Practical database engineer possessing in-depth knowledge of data manipulation techniques and computer programming paired with expertise in integrating and implementing new software packages and new products into system. Offering several-year background managing various aspects of development, design and delivery of database solutions. Tech-savvy and independent professional bringing outstanding communication and organizational abilities.
Overview
12
12
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Td bank
Canada, ON
01.2020 - Current
Define project scope and timelines, collaborating with stakeholders to understand data needs and provide technical leadership and training on data engineering tools
Develop risk management plans, technical design documents, and SOPs focusing on data integrity, security, and privacy
Translate business objectives into data engineering solutions, collecting and ingesting data from multiple sources into Azure Delta Lake
Implement various data loads using Azure Data Factory, optimizing ingestion and transformation processes
Ingesting data from diverse sources including Delimited, Mainframe, JSON, and XML formats into Delta Lake, ensuring data integrity and reliability
Automate recurring tasks and jobs to streamline operations and reduce manual efforts
Senior Big Data Engineer
Td bank
Canada, ON
01.2018 - 01.2020
Managed Hadoop Distributed File System (HDFS) files, optimizing storage and retrieval processes for efficient data handling
Automated ETL jobs using Oozie Workflow Scheduler and Bash Script, reducing manual intervention and improving data flow efficiency
Leveraged data from a Big Data Lake, utilizing advanced tools like Hive, HDFS, and masking techniques for comprehensive data processing
Developed and implemented data quality checks and validation processes to ensure accuracy and reliability throughout the data lifecycle
Provided technical guidance and mentoring to team members on best practices in big data management, fostering skill development and knowledge sharing
Transferred data from data lake to HEAT cluster through podium publish ETL tool for Analytics and reporting, enhancing data accessibility and usability for analytical purposes
Senior ETL Consultant
ust global
Chennai, India
01.2017 - 01.2018
Analyzed ETL SSIS Package and designed DataStage jobs to perform equivalent functionality, ensuring seamless transition and compatibility across platforms
Prepared comprehensive Data Mapping Documents and designed ETL jobs based on intricate requirements, ensuring alignment with business objectives and data quality standards
Conducted performance tuning and optimization of DataStage jobs to enhance processing speed and efficiency, reducing processing times and resource utilization
Created intricate DataStage jobs utilizing various stages for data transformation, including Transformer, Aggregator, Sort, Join, Merge, Lookup, and more, optimizing data processing efficiency
Documented detailed ETL test plans, test cases, and validations based on design specifications, facilitating thorough testing and validation processes to ensure accuracy and reliability of ETL workflows
ETL Developer
Cognizant Technology Solutions
Chennai, India
01.2016 - 01.2017
Created detailed Data Mapping Documents to ensure stakeholders had a clear understanding of data flow and transformation requirements
Enhanced ETL job performance by implementing meticulous query optimization techniques, improving data processing speed and efficiency
Managed import/export of ETL job components to maintain data integrity and facilitate seamless deployment processes
Implemented advanced multi-node declaration and re-partitioning strategies in DataStage, optimizing resource utilization and job execution efficiency
Developed complex DataStage jobs utilizing advanced techniques like Change Data Capture and Surrogate Key to meet intricate data processing needs
Conducted thorough unit testing and debugging of DataStage jobs to ensure accuracy and reliability in data transformation processes
Project Engineer
Wipro Technologies
Bengaluru, India
01.2013 - 01.2016
Designed complex data mappings to facilitate accurate and efficient data flow between sources and targets
Collaborated closely with the batch operations team to schedule and execute DataStage jobs, ensuring smooth data processing
Played a key role in troubleshooting and resolving production issues, providing round-the-clock on-call support to minimize downtime
Ensured seamless data operations by addressing issues promptly and implementing effective solutions