Summary
Overview
Work History
Education
Skills
Timeline
Generic

Karthik N

Toronto

Summary

Results-driven Data Engineer with 6+ years of experience delivering secure, scalable ETL pipelines, Big Data and analytics solutions across AWS and Azure environments. Expert in building and optimizing data pipelines, ETL workflows, and SQL-based data integration to support large-scale enterprise data platforms and analytics initiatives. Skilled in Azure services like ADF, Databricks, Synapse, Data Lake, Key Vault and AWS services like Glue, Redshift, Lambda, S3, EMR, Athena for scalable, cloud-native data engineering. Migrated large-scale SQL and NoSQL databases from on-premises systems to cloud platforms like Azure SQL, AWS Redshift, and Snowflake, ensuring zero data loss, optimized performance, and minimal downtime through automation and data validation frameworks. Hands-on experience with PySpark, Spark SQL, Hadoop, Kafka, and Big Data ecosystems, ensuring high-performance distributed data processing and real-time analytics. Proficient in Python, SQL, and automation frameworks, streamlining repetitive tasks and enhancing reliability across ETL/ELT processes. Seamlessly integrated external APIs and open-source libraries into Python applications to expand functionality and automation capabilities. Applied MLOps practices by supporting data pipelines for model training, validation, and deployment, ensuring seamless integration of machine learning into production environments. Designed and developed interactive dashboards and visualizations using Power BI to deliver actionable insights and enhance decision-making across business domains. Designed and implemented data pipelines using Jenkins, GitHub Actions, and Azure DevOps, integrated with Terraform for infrastructure as code, and orchestrated complex ETL workflows with Apache Airflow DAGs, improving release efficiency and pipeline reliability. Strong background in data governance, compliance, and security controls (GDPR, HIPAA, PIPEDA), ensuring secure and ethical use of enterprise data. Adept at working in Agile Scrum environments, contributing to sprint planning, backlog grooming, and cross-functional collaboration to deliver on business priorities. Recognized for mentoring junior engineers, fostering knowledge sharing, and promoting best practices in data engineering and DevOps. Proven leadership in cross-functional projects, driving end-to-end data modernization initiatives that improved scalability, reliability, and business impact.

Diligent [Desired Position] with robust background in data engineering and proven ability to design and implement complex data pipelines. Successfully contributed to optimizing data architecture and enhancing data processing efficiencies. Demonstrated expertise in big data technologies and proficiency in Python and SQL.

Experienced with building and maintaining data pipelines to ensure seamless data flow. Utilizes advanced knowledge of big data technologies to drive data-driven decision-making. Track record of enhancing data architecture for improved performance and reliability.

Data engineering professional poised to add significant value through comprehensive experience in developing scalable data solutions. Noted for strong team collaboration and adaptability in fast-paced environments. Reliable in driving results with key skills in data modeling, ETL processes, and cloud-based data platforms.

Senior engineering professional with deep expertise in data architecture, pipeline development, and big data technologies. Proven track record in optimizing data workflows, enhancing system efficiency, and driving business intelligence initiatives. Strong collaborator, adaptable to evolving project demands, with focus on delivering impactful results through teamwork and innovation. Skilled in SQL, Python, Spark, and cloud platforms, with strategic approach to data management and problem-solving.

Detail-oriented [Job Title] designs, develops and maintains highly scalable, secure and reliable data structures. Accustomed to working closely with system architects, software architects and design analysts to understand business or industry requirements to develop comprehensive data models. Proficient at developing database architectural strategies at the modeling, design and implementation stages.

Responsive expert experienced in monitoring database performance, troubleshooting issues and optimizing database environment. Possesses strong analytical skills, excellent problem-solving abilities, and deep understanding of database technologies and systems. Equally confident working independently and collaboratively as needed and utilizing excellent communication skills.

Astute [Job Title] with data-driven and technology-focused approach. Communicates clearly with stakeholders and builds consensus around well-founded models. Talented in writing applications and reformulating models.

Meticulous Data Scientist accomplished in compiling, transforming and analyzing complex information through software. Expert in machine learning and large dataset management. Demonstrated success in identifying relationships and building solutions to business problems.

Experienced leader with strong background in guiding teams, managing complex projects, and achieving strategic objectives. Excels in developing efficient processes, ensuring high standards, and aligning efforts with organizational goals. Known for collaborative approach and commitment to excellence.

Equipped with strong problem-solving abilities, willingness to learn, and excellent communication skills. Poised to contribute to team success and achieve positive results. Ready to tackle new challenges and advance organizational objectives with dedication and enthusiasm.

Proactive and goal-oriented professional with excellent time management and problem-solving skills. Known for reliability and adaptability, with swift capacity to learn and apply new skills. Committed to leveraging these qualities to drive team success and contribute to organizational growth.

Recent graduate with foundational knowledge in [Area of study] and hands-on experience gained through academic projects and internships. Demonstrates strong teamwork, problem-solving, and time-management skills. Prepared to start career and make meaningful contributions with commitment and drive.

Results-oriented achiever with proven ability to exceed targets and drive success in fast-paced environments. Combines strategic thinking with hands-on experience to deliver impactful solutions and enhance organizational performance.

Organized and dependable candidate successful at managing multiple priorities with a positive attitude. Willingness to take on added responsibilities to meet team goals.

Possesses versatile skills in project management, problem-solving, and collaboration. Brings fresh perspective and strong commitment to quality and success. Recognized for adaptability and proactive approach in delivering effective solutions.

Thorough team contributor with strong organizational capabilities. Experienced in handling numerous projects at once while ensuring accuracy. Effective at prioritizing tasks and meeting deadlines.

Innovative technology professional with several years of diverse experience. Skilled in enhancing systems and aligning technical solutions with business objectives. Proven success in leading projects from start to finish and contributing to organizational growth and success.

Tech-savvy innovator with hands-on experience in emerging technologies and passion for continuous improvement. Skilled in identifying opportunities for technological enhancements and implementing effective solutions. Adept at leveraging new tools and methods to solve problems and enhance productivity. Excels in adapting to fast-paced environments and driving technological advancements.

Demonstrates strong analytical, communication, and teamwork skills, with proven ability to quickly adapt to new environments. Eager to contribute to team success and further develop professional skills. Brings positive attitude and commitment to continuous learning and growth.

Pursuing full-time role that presents professional challenges and leverages interpersonal skills, effective time management, and problem-solving expertise.

Dynamic individual with hands-on experience in [Area of expertise] and talent for navigating challenges. Brings strong problem-solving skills and proactive approach to new tasks. Known for adaptability, creativity, and results-oriented mindset. Committed to making meaningful contributions and advancing organizational goals.

Detail-oriented individual with exceptional communication and project management skills. Proven ability to handle multiple tasks effectively and efficiently in fast-paced environments. Recognized for taking proactive approach to identifying and addressing issues, with focus on optimizing processes and supporting team objectives.

Hardworking and passionate job seeker with strong organizational skills eager to secure entry-level [Job Title] position. Ready to help team achieve company goals.

Overview

6
6
years of professional experience

Work History

Senior Data Engineer

Paypal
09.2022 - Current
  • Designed, developed, and optimized complex SQL Server stored procedures, functions, views, and triggers to support real-time and batch operations.
  • Developed and maintained scalable database schemas, tables, indexes, and views across analytical and transactional systems, ensuring alignment with data normalization best practices.
  • Partnered with business analysts and architects to translate high-level requirements into robust, compliant, and scalable database solutions.
  • Led ELT pipeline development using AWS Glue and Azure Data Factory, integrating data from internal and external platforms, ensuring referential integrity and SLA compliance.
  • Applied advanced data modeling techniques (Star, Snowflake schemas) and normalization strategies to support performance and scalability.
  • Engineered streaming data pipelines using Kafka, Spark Streaming, and Kinesis, enabling near-real-time insights for regulated workflows and business operations.
  • Implemented data quality validations, schema enforcement, and transformation logic to meet data accuracy and consistency standards.
  • Automated reporting and operational workflows using PowerShell, Python, and SQL, improving turnaround times for daily banking-like operations.
  • Developed and maintained Power BI dashboards, integrating audit trails, dynamic filters, and drilldowns to enhance transparency and control.
  • Integrated GDPR and PCI DSS principles into pipeline and schema design, ensuring full regulatory compliance across sensitive datasets.
  • Participated in implementing backup and disaster recovery strategies, working with infrastructure teams to maintain data availability and minimize risk.
  • Monitored and resolved performance issues in SQL and Spark jobs, conducting root cause analysis and applying tuning techniques (e.g., query plans, partitioning).
  • Conducted peer code reviews and mentored junior developers in writing optimized SQL, automation scripts, and modular pipeline components.
  • Contributed to continuous improvement initiatives, developing CI/CD pipelines using Git, Jenkins, Terraform, and Azure DevOps for robust release processes.
  • Documented end-to-end data workflows, lineage, and architectural decisions in JIRA and Confluence, improving collaboration and maintainability.
  • Gained conceptual exposure to MemSQL and GridGain for in-memory and caching strategies, familiar with implementation architecture and performance tuning considerations.
  • Assisted in system security hardening and database access control implementations in collaboration with IT security teams.
  • Environment: PySpark, Spark, Hive, Kafka, Spark Streaming, Python, Scala, SQL, AWS (Glue, S3, EMR, EC2, Lambda, Athena, Kinesis, Redshift), Azure (Databricks, Data Factory, SQL DW, DevOps, Logic Apps, Functional Apps), Snowflake, MS SQL, Oracle, Airflow, Jenkins, GIT, JIRA, Power BI, Tableau
  • Collaborated with cross-functional teams to define data architecture and integration strategies.
  • Mentored junior engineers on best practices in data modeling and database management.
  • Developed robust data quality frameworks ensuring accuracy and consistency of datasets.
  • Led initiatives to migrate legacy systems to cloud-based solutions, improving scalability.
  • Automated data ingestion workflows, reducing manual intervention and increasing reliability.
  • Analyzed system performance metrics to identify bottlenecks and drive continuous improvement efforts.
  • Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects.
  • Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing code accordingly.
  • Evaluated emerging technologies and tools to identify opportunities for enhancing existing systems or creating new ones.
  • Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
  • Leveraged advanced analytics tools to create interactive dashboards that provided actionable insights into key business metrics.
  • Championed the adoption of agile methodologies within the team, resulting in faster delivery times and increased collaboration among team members.
  • Mentored junior team members in best practices for software development, code optimization, and troubleshooting techniques.

Software Developer

Zoho Corporation
09.2019 - 08.2022
  • Built SQL scripts to handle data archiving and purging, supporting storage optimization and long-term data lifecycle management.
  • Integrated Power BI with Azure DevOps to enable collaborative development and apply version control best practices in reporting workflows.
  • Created and customized SSIS packages to support environment-specific deployments with consistent configurations.
  • Enabled seamless data connections in SSIS by managing various source and destination setups, ensuring smooth data flow.
  • Implemented rigorous data validation routines within SSIS to maintain high levels of data integrity and accuracy across ETL processes.
  • Harnessed Power BI's advanced analytics and ML features to deliver predictive and prescriptive insights for business strategy.
  • Conducted detailed market segmentation analysis using psychographic indicators to uncover behavioral patterns and audience preferences.
  • Developed scalable web applications using Python and JavaScript frameworks.
  • Led cross-functional teams to enhance software development lifecycle processes.
  • Mentored junior developers on best coding practices and project methodologies.
  • Implemented automated testing strategies to optimize code quality and reliability.
  • Collaborated with product managers to define requirements and deliver effective solutions.
  • Analyzed system performance metrics, identifying areas for optimization and improvement.
  • Streamlined deployment processes, reducing release time through CI/CD tools integration.
  • Drove adoption of Agile methodologies, enhancing team collaboration and productivity.
  • Improved software efficiency by troubleshooting and resolving coding issues.
  • Saved time and resources by identifying and fixing bugs before product deployment.
  • Collaborated with cross-functional teams to deliver high-quality products on tight deadlines.
  • Enhanced user experience through designing and implementing user-friendly interfaces.
  • Updated old code bases to modern development standards, improving functionality.
  • Optimized application performance by conducting regular code reviews and refactoring when necessary.
  • Participated in software field testing to verify performance of developed projects.
  • Contributed to a positive team environment through effective communication, problem-solving, and collaboration skills.
  • Developed customized software solutions for diverse clients, resulting in increased satisfaction and repeat business.
  • .Streamlined workflows by creating reusable code libraries for common functions and features across multiple projects.
  • Designed customized solutions for proposals to potential customers.
  • .Mentored junior developers to improve their technical skills, fostering a culture of continuous learning within the team.
  • Translated customer requirements into written use cases.
  • Developed software for desktop and mobile operating systems.
  • .Consistently met project milestones while maintaining rigorous quality control standards throughout all stages of the development life cycle.
  • Increased development speed by automating repetitive tasks using scripts and tools.
  • .Achieved faster development cycles using Agile methodologies, including Scrum or Kanban processes.
  • .Created comprehensive documentation detailing software functionality for future reference or maintenance purposes.
  • .Boosted customer satisfaction rates through timely resolution of reported technical issues during the support phase of projects.
  • .Ensured seamless migrations from legacy systems to modern platforms through meticulous planning, testing, and execution.
  • Collaborated on stages of systems development lifecycle from requirement gathering to production releases.
  • Tailored software solutions to meet specific client needs, ensuring high levels of customer satisfaction and repeat business.
  • Optimized database queries for enhanced performance, enabling faster data retrieval and processing.
  • Enhanced user experience with intuitive interface designs, leading to increased customer satisfaction.
  • Boosted team productivity through introduction of pair programming, fostering culture of knowledge sharing and collaboration.
  • Engaged in continuous learning to stay ahead of emerging technologies, ensuring team's solutions remained cutting edge.
  • Conducted in-depth market research to guide development of new software features that addressed unmet user needs.
  • Collaborated closely with cross-functional teams to identify and resolve system bottlenecks, ensuring smoother operations.
  • Participated in regular code sprints, contributing to rapid development and iteration of software products.
  • Pioneered use of machine learning algorithms to automate and improve decision-making processes within applications.
  • Spearheaded adoption of containerization, significantly improving deployment workflows and environments' consistency.
  • Increased code efficiency by implementing rigorous code review practices, which improved overall software performance.
  • Facilitated seamless migrations to cloud-based solutions, enabling more flexible and cost-effective infrastructure management.
  • Implemented automated testing frameworks, reducing bugs at launch and ensuring higher quality releases.
  • Reduced system downtime by establishing robust monitoring and quick response protocols.
  • Streamlined software development processes, significantly reducing time to market by introducing agile methodologies.
  • Contributed to open source projects, enhancing product features and community engagement.
  • Led development of scalable web application, accommodating growing user demands without compromising on speed.
  • Developed comprehensive documentation for software projects, improving maintainability and future scalability.
  • Improved software security with integration of advanced encryption techniques, safeguarding sensitive user data.
  • Played key role in mentoring junior developers, elevating team competencies and fostering supportive work environment.
  • Administered time-based retention policies in Azure Blob Storage to meet compliance regulations and secure historical data.
  • Managed metadata and data integrity with AWS Glue Data Catalog to reinforce governance and data discovery.
  • Designed high-performance ETL workflows within Redshift, enabling scalable and reliable data transformations.
  • Developed data cleansing and deduplication tools to eliminate redundancy and boost dataset quality and performance.
  • Leveraged QlikView data modeling to streamline transformation, correlation, and reporting processes.
  • Designed interactive Python-based visualizations using Seaborn, Plotly, and Matplotlib for clear data communication.
  • Embedded Plotly-based visualizations into BI tools and web apps, enriching analytical and visual depth for end-users.
  • Participated in quality assurance and code review sessions to uphold data accuracy, maintainability, and compliance.
  • Applied Git command-line tools like git log no-merges to streamline commit history tracking for cleaner repository maintenance.
  • Carried out data lineage and impact assessments to trace data flows from source to destination and evaluate change impact.
  • Built machine learning-powered market basket analysis solutions in Azure Databricks for uncovering product affinity patterns.
  • Enabled real-time data handling in Azure Data Factory through integration with streaming platforms, supporting continuous data updates.
  • Developed configuration scripts in Azure DevOps for managing variables across multiple environments and deployment stages.
  • Developed recovery strategies for ETL processes, along with disaster recovery plans to maintain business continuity in case of system failures.
  • Employed ensemble machine learning techniques like stacking and blending to improve prediction accuracy by combining outputs from multiple models.
  • Implemented robust data masking and tokenization techniques to anonymize sensitive information and comply with data privacy regulations.
  • Authored comprehensive SSIS documentation, including data flow diagrams and dependency mappings, to support better understanding and maintainability of packages.
  • Enhanced SSRS reports by crafting custom expressions and functions, improving calculations and enabling complex data transformations within reports.
  • Developed and maintained custom Power BI visuals and extensions to meet advanced visualization needs beyond native capabilities.
  • Produced informative SSRS reports featuring a variety of visual elements—charts, gauges, graphs— to improve data comprehension and decision-making.
  • Built databases and table structures for web applications.
  • Tested and deployed scalable and highly available software products.
  • Corrected, modified and upgraded software to improve performance.
  • Coordinated deployments of new software, feature updates and fixes.
  • Analyzed work to generate logic for new systems, procedures and tests.
  • Conducted data modeling, performance and integration testing.
  • Documented software development methodologies in technical manuals to be used by IT personnel in future projects.
  • Authored code fixes and enhancements for inclusion in future code releases and patches.
  • Estimated work hours and tracked progress using Scrum methodology.
  • Created proofs of concept for innovative new solutions.
  • Designed and implemented scalable applications for data extraction and analysis.
  • Developed next generation integration platform for internal applications.
  • Developed conversion and system implementation plans.
  • Designed and developed forward-thinking systems that meet user needs and improve productivity.
  • Translated technical concepts and information into terms parties could easily comprehend.
  • Inspected equipment, assessed functionality, and optimized controls.
  • Tested functional compliance of company products.
  • Proved successful working within tight deadlines and a fast-paced environment.
  • Rapidly prototyped new data processing capabilities to confirm integration feasibility into existing systems.
  • Supervised work of programmers, designers and technicians, assigned tasks and monitored performance against targets.
  • Optimized dust, temperature and humidity controls for installed systems.

Education

Bachelor of Science - Computer Science

SRM University
INDIA

Skills

  • Languages: SQL, Python, PySpark, Scala, Java
  • Bigdata Tools: Hadoop, Apache Spark, Hive, Snowflake, Apache Airflow, Apache Kafka, HDFS
  • Reporting Tools: Power BI, SSIS, SSRS, SQL SAS Crystal Report, Tableau
  • Cloud Services: AWS: Amazon S3, EMR, Redshift, AWS Glue, Athena, Kinesis, AWS Lambda Azure: Azure Data Lake Storage, Azure Synapse, Azure Data Factory, Databricks
  • Methodology: Agile, Waterfall
  • Databases: MS SQL Server 2019/2016/2014/2012, PL/SQL, Oracle 11g/12c, Mongo DB, Cosmos DB, Azure SQL Database, Amazon RDS, and Google Cloud SQL
  • Other Skills: DBT (Data Build Tool), Maven, Docker, Jenkins, Kubernetes, Terraform, Luigi, Oozie, Jira, Confluence, Machine Learning, Microservices, Autosys, SharePoint

Timeline

Senior Data Engineer

Paypal
09.2022 - Current

Software Developer

Zoho Corporation
09.2019 - 08.2022

Bachelor of Science - Computer Science

SRM University
Karthik N