GNANA PRADEEP CHODAVARAPU
Toronto,Canada
Summary
Data Engineer with over 6 years of experience in designing and implementing robust data pipelines utilizing AWS and Azure technologies. Proficient in AWS services such as EC2, S3, RDS, and Lambda, alongside Azure tools including Data Factory and SQL Server. Skilled in SQL, Python, and CI/CD pipeline development, with a strong focus on big data processing using Hadoop, Spark, and Kafka. Demonstrated ability to deliver scalable data solutions and provide production support through effective log monitoring and REST API integration. Proficient in using ETL/ELT tools such as Apache NiFi, Informatica, Talend, Apache Airflow, Azure Data Factory, and AWS Glue for designing and automating data integration workflows.
Overview
7
7
years of professional experience
1
1
Certification
Work History
Azure Data Engineer
Royal Bank of Canada
Toronto, CANADA
06.2024 - Current
- Implemented Azure Data Factory (ADF) for data ingestion and transformation from relational, unstructured sources; automated workflows using triggers such as Events, Schedules, and Tumbling.
- Developed batch and real-time data processing solutions leveraging ADF, Databricks clusters, and Azure Stream Analytics, facilitating migration from on-premise databases to Azure Data Lake Store (ADLS).
- Designed and provisioned Databricks clusters, notebooks, and jobs for continuous and batch processing; automated business processes using Azure Logic Apps.
- Created data ingestion systems to extract from RDBMS (Oracle/Teradata) and load into NoSQL databases like MongoDB; utilized custom UDFs in PySpark for enhanced data processing.
- Employed ElasticSearch and Kibana for data indexing and visualization; contributed to CI/CD pipelines for data transformation and deployment.
AWS Data Engineer
TD Bank
Toronto, CANADA
11.2023 - 05.2024
- Implemented serverless architecture with AWS API Gateway, Lambda, and DynamoDB to automate software deployment.
- Developed AWS Lambda code integrated with DynamoDB from Amazon S3 for complex analytical reporting.
- Utilized SonarQube for continuous code inspection, automated bug detection, and compliance assurance.
- Designed AWS infrastructure automation using CLI and CloudFormation for services including SNS, SQS, and EC2.
- Created HBase and Hive tables with ORC format and Snappy compression for efficient data storage.
- Developed SQOOP scripts for Oracle-to-Big Data migration and implemented Kafka producers for data streaming.
Azure Data Engineer
Biocon (Cognizant)
Bangalore, India
02.2020 - 07.2023
- Designed and implemented scalable data architectures on Azure for storage, processing, and analytics.
- Created and optimized data models and schemas for efficient data retrieval and integration.
- Migrated Oracle database to Azure Synapse Analytics, utilizing Power BI for interactive reporting.
- Built and managed ETL pipelines in Azure Data Factory for comprehensive data processing and transformation.
- Leveraged big data technologies, including Apache Hadoop and Azure Databricks, for large dataset analysis.
- Deployed and managed Azure cloud components, enhancing data processing workflows effectively.
Data Engineer
Volvo
Bangalore, India
08.2018 - 01.2020
- Developed and maintained scalable data pipelines for processing large volumes of market data and trade transactions using Azure Data Factory and Azure Databricks.
- Optimized SQL queries and workflows to enhance performance and reduce latency in financial data retrieval utilizing Azure SQL Database.
- Integrated data with business intelligence tools like Power BI, enabling stakeholders to access dashboards and visualizations for data-driven decisions.
- Implemented monitoring systems with Azure Monitor and Azure Log Analytics to ensure health and performance of data pipelines.
- Migrated data from diverse sources, including relational and NoSQL databases, into Azure solutions using Azure Data Factory.
- Built Azure-based data solutions incorporating services like Azure Blob Storage and Azure Logic Apps to automate processing.
Education
Bachelor of Technology - Electronics And Communication Engineering
JNTUK
IndiaSkills
- Azure Data Factory and Synapse Analytics
- Azure Databricks and SQL Database
- AWS Glue and Redshift
- AWS Lambda and S3
- Azure Data Lake Storage
- Amazon RDS and Apache Spark
- Azure Stream Analytics and CloudWatch
- Power BI expertise
- SQL performance tuning
- Apache Kafka for streaming data
- Python for data processing
- ETL process design
- Big data technologies (Hadoop, HBase, Hive)
- Docker for containerization
- CI/CD for data pipelines
Certification
- Microsoft Certified: Azure Data Fundamentals
Languages
English
Professional
Timeline
Azure Data Engineer
Royal Bank of Canada
06.2024 - Current
AWS Data Engineer
TD Bank
11.2023 - 05.2024
Azure Data Engineer
Biocon (Cognizant)
02.2020 - 07.2023
Data Engineer
Volvo
08.2018 - 01.2020
Bachelor of Technology - Electronics And Communication Engineering
JNTUK
Similar Profiles
ANIRBAN BASUANIRBAN BASU
QA Manager at TCS CANADA (Client - Royal Bank Of Canada)QA Manager at TCS CANADA (Client - Royal Bank Of Canada)
Ashok Kumar PeddinaAshok Kumar Peddina
Senior Consultant at Capgemini, Canada, Royal Bank of CanadaSenior Consultant at Capgemini, Canada, Royal Bank of Canada
Pierre ChristianPierre Christian
Customer Service Representative/Courier and Archiving at (RBC)Royal Bank Of Canada/Antigua Commercial BankCustomer Service Representative/Courier and Archiving at (RBC)Royal Bank Of Canada/Antigua Commercial Bank
Emmanuela Osei-KwartengEmmanuela Osei-Kwarteng
SOX Compliance Analyst at Royal Bank of CanadaSOX Compliance Analyst at Royal Bank of Canada
Venkat MuruganVenkat Murugan
Associate Director, Murex Business Specialist at RBC Capital MarketsAssociate Director, Murex Business Specialist at RBC Capital Markets