ANNIERIA THIYAGARAJAN
Toronto,ON
Summary
- A passionate Data engineering professional with 14 years of experience in Data Engineering & Analytics, Big Data and Cloud Computing.
- Veteran Analytics Engineer skilled at managing both databases and technical personnel.
- Ready to serve as leader and subject matter expert in all areas touching data retention, processing, and management.
- Having a demonstrated history of working in the Finance, Automotive and Retail industries.
- Proven track record of high performance under trying circumstances and delivering projects on-time with high customer satisfaction.
- Complex problem-solver with analytical and driven mindset.
- Organized and dependable candidate successful at managing multiple priorities with a positive attitude.
- Willingness to take on added responsibilities to meet team goals.
Overview
14
14
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
FPT Software
03.2024 - Current
- Subject Matter Expert and Technical Lead in Revenue Management - Revenue Integrity for Air Canada.
- Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects by developing ETL pipelines using Azure Data Factory and Azure Databricks.
- Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing Pyspark code accordingly.
- Helped team upgrade to Spark 13.3 and enable Unity Catalog for workspaces.
- Mentored junior team members in best practices for software development, code optimization, and troubleshooting techniques.
- Championed adoption of agile methodologies within team, resulting in faster delivery times and increased collaboration among team members.
- Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
- Designed robust database architecture that supported seamless integration of new datasets and facilitated rapid analysis capabilities.
- Acted as a trusted advisor for clients by providing thought leadership on best practices in data engineering, ensuring their systems were optimized for performance and scalability.
- Leveraged advanced analytics tools to create interactive dashboards that provided actionable insights into key business metrics.
Technical Lead
Tech Mahindra
05.2023 - 03.2024
- Subject Matter Expert and Technical Lead in Worker Data Analytics space.
- Collaborated with product managers, designers, and business stakeholders to define project requirements, ensuring alignment with organizational goals.
- Designed and developed end to end ETL pipeline using PySpark, Unix Shell scripts, BigQuery, Azure Data Factory, and Azure Databricks.
- Implemented complete Data Warehouse solution in Big Query thereby enabling business to gather insights and make decisions.
- Demonstrated proficiency in writing complex SQL transformations in fine-tuned manner to facilitate customer needs.
- Authored resource creation in GCP using Terraform.
- Enhanced pipeline performance by using Spark and Databricks optimization techniques.
- Designed and implemented Data Governance, Data Cataloging and Data Security using Unity Catalog in Azure Databricks and Policy tags, RLS and dynamic views in GCP.
- Developed near-realtime streaming pipelines using Autoloader and Delta Live Tables.
- Implemented continuous integration and deployment strategies for faster release cycles and improved application reliability.
Senior Data Engineer
Walmart Global Tech India
05.2019 - 05.2023
- Subject Matter Expert in Walmart’s Cost to Serve Analytics platform.
- Championed Cloud Migration from on-prem Hadoop to Google Cloud Platform thereby releasing 85% of Supply Chain footprint in on-prem Hadoop environment.
- Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing code accordingly.
- Designed and developed almost 50+ complex data pipelines employing complex SQL transformations.
- Automated Hive to Spark components conversion thereby improving the migration time by 50%.
- Automated Data Quality report generation thereby improving data quality by 98%.
- Authored robust Kafka pipelines to ensure events like including orders, inquires, and cancellations reach the system within a minimum time.
- Handled importing of data from various data sources, performed transformations using Hive, loaded data into GCP using Sqoop and Kafka for nearly 150 tables.
- Contributed to the development of 8 business critical dashboards in Looker which provides insights on Distribution Center wastages, Travel Congestion, Fresh Wastages.
- Spearheaded the development of Cloud Cost Monitoring and Optimization tool which has improved overall cloud cost saving by 50% across different domains.
- Exercised leadership capabilities by successfully training and handling team members.
Data Engineer
Ford Motor Company
08.2017 - 05.2019
- Data Engineer for a SAS to PySpark model conversion project in Customer Analytics space.
- Led end-to-end implementation from requirements gathering through deployment and post-launch support stages.
- Evaluated various tools, technologies, and best practices for potential adoption in the company''s data engineering processes.
- Streamlined complex workflows by breaking them down into manageable components for easier implementation and maintenance.
- Optimized data processing by implementing efficient ETL pipelines and streamlining database design.
- Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
- Collaborated with data scientists to develop machine learning models by providing the necessary data infrastructure and preprocessing tools.
Associate
Cognizant Technology Solutions
01.2015 - 08.2017
- Backend developer responsible for building back end system in Oracle PL/SQL using packages, procedures and other data objects for a banking application.
- Facilitated the coordination between the frontend team and backend team in correctly understanding the requirements and delivering in the required format for the frontend to consume data.
- Spearheaded process improvement initiatives that increased operational efficiency while maintaining high-quality output standards.
- Established robust monitoring processes to proactively detect system anomalies or performance bottlenecks before they impact users or critical operations.
- Designed scalable and maintainable data models to support business intelligence initiatives and reporting needs.
- Authored code fixes and enhancements for inclusion in future code releases and patches.
- Automated repetitive tasks using custom scripts, enhancing productivity across the development team.
Software Engineer
Hexaware Technology
10.2013 - 01.2015
- Built strong relationships with stakeholders, ensuring clear communication channels for project updates and progress reports.
- Improved software development efficiency by implementing Agile methodologies.
- Demonstrated expertise in developing complex SQL parsing logic.
- Implemented and maintained batch jobs for data loading.
- Improved efficiency of the application by performance tuning of backend queries.
- Explored and implemented new features like result caching in functions thereby improving the response time.
- Established efficient communication channels within the team, leading to better collaboration among members during project development phases.
- Maximized user satisfaction by gathering feedback and incorporating it into future iterations of products.
- Developed comprehensive documentation to facilitate knowledge sharing among team members and support long-term maintenance efforts.
Systems Engineer
Tata Consultancy Services
09.2010 - 10.2013
- Assisted in designing, planning and managing database migration process from Sybase to SQL Server and Oracle.
- Coordinated with stakeholders throughout all stages of the project lifecycle from inception to completion.
- Worked with subject matter experts and project team to identify, define, collate, document and communicate the database migration requirements.
- Developed custom software solutions (code review tool) for various business needs, resulting in improved productivity.
- Interacted with business group on daily basis and resolved production issues within SLA.
Education
Masters In Computer Applications -
Thiagarajar School Of Management
India04.2010
Skills
Programming Languages: PySpark, Python, Oracle PL/SQL, SQL
Cloud Platforms: Microsoft Azure, Google Cloud Platform
On-Premises: Hadoop ecosystem (HDFS, Hive, Sqoop)
Big Data Processing: Apache Spark, Azure Databricks
Streaming Services: Delta Live Tables
Data-Warehouse: Big Query, Azure Synapse Analytics
GCP Services: GCS, IAM, Terraform, Cloud Functions, DataProc, BigQuery
Azure Services: ADLS Gen2, Azure Data Factory, Unity Catalog
RDBMS: Oracle 11g, SQL Server
Scripting Language: Unix Shell Scripting
Tools: Jupyter Notebook, SAS Enterprise Studio, PLSQL Developer, Storage Explorer
Visualization Tool: Power BI, Looker
Scheduler: Azure Data Factory, Automic, Google Composer, Airflow
Project Management: Azure Repo, JIRA, GitHub
Certification
- Google Certified Associate Cloud Engineer
- Google Certified Professional Data Engineer
- Microsoft Azure Cloud Fundamentals AZ-900
- Microsoft Azure Data Fundamentals DP-900
- Databricks Certified Data Engineer Associate
- Microsoft Azure Data Engineer Associate DP-203
- Oracle Certified Professional
- Amazon Certified Solutions Architect
Accomplishments
- "Bravo Award” for excellence - Walmart Global Tech India
- "Asia Pacific Recognition Award” – Ford Motor Company
- "Business Enabler” – Cognizant Technology Solutions
- "Associate of the Quarter” – Cognizant Technology Solutions
Languages
English
Native or Bilingual
Tamil
Native or Bilingual
Timeline
Senior Data Engineer
FPT Software
03.2024 - Current
Technical Lead
Tech Mahindra
05.2023 - 03.2024
Senior Data Engineer
Walmart Global Tech India
05.2019 - 05.2023
Data Engineer
Ford Motor Company
08.2017 - 05.2019
Associate
Cognizant Technology Solutions
01.2015 - 08.2017
Software Engineer
Hexaware Technology
10.2013 - 01.2015
Systems Engineer
Tata Consultancy Services
09.2010 - 10.2013
Masters In Computer Applications -
Thiagarajar School Of Management
- Google Certified Associate Cloud Engineer
- Google Certified Professional Data Engineer
- Microsoft Azure Cloud Fundamentals AZ-900
- Microsoft Azure Data Fundamentals DP-900
- Databricks Certified Data Engineer Associate
- Microsoft Azure Data Engineer Associate DP-203
- Oracle Certified Professional
- Amazon Certified Solutions Architect
Similar Profiles
Tria Claire A. DucayTria Claire A. Ducay
Senior Business Development Manager at FPT SOFTWARE, FPTSenior Business Development Manager at FPT SOFTWARE, FPT
HAN PHANHAN PHAN
Receptionist at FPT SoftwareReceptionist at FPT Software
Van TruongVan Truong
Senior UI Designer at FPT SoftwareSenior UI Designer at FPT Software