Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

AISHWARYA RAMAN

Texas

Summary

Experienced IT professional with over 18 years in the industry, including 7+ years in architecture and cloud services (AWS/Azure). Expert in leveraging emerging technologies for business insights from big data and proficient in Hadoop architecture. Responsible for the entire development architecture within data lakes, including project planning, technical estimation, and maintenance. Familiar with Agile and DevOps practices, ensuring continuous integration and delivery.

Overview

19
19
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

Brillio
Texas/Toronto
12.2023 - Current

The McAfee Mosaic Portal is being upgraded from Angular CLI version 7.3.6 to version 12. Completed upgrades to Angular 8, 9, and 10, with work on Angular 11 and 12 in progress.

  • Analyzed Alteryx workflows and mapped migration to Databricks.
  • Designed data architecture using the Medallion approach with Bronze, Silver, and Gold data layers.
  • Developed and tested ETL pipelines in PySpark to replicate Alteryx functionality in Databricks.
  • Managed historical and incremental data loads to ensure data completeness and currency.
  • Optimized data storage and query performance through data partitioning and Spark tuning.
  • Collaborated with cross-functional teams to align migration with business needs.
  • Migrated SAS code to PySpark in AWS Glue and conducted BAT testing.

Lead Data Engineer

HCL Technologies
Mississauga, ON
02.2022 - 11.2023

Managed ALPACA data products for HP Print Consents, handling daily data reports and storage in delta tables based on parquet format.

  • Implemented data integration techniques, including ETL processes and data pipelines.
  • Designed and optimized data models for both relational and NoSQL databases.
  • Led the implementation of historical and incremental data loads in AWS Glue using PySpark.
  • Developed and optimized data processing workflows in Azure Databricks.
  • Managed data storage and processing in AWS S3 and Redshift.
  • Deployed and managed big data solutions across AWS, Azure, and Google Cloud platforms.
  • Created and optimized ADF pipelines for data transfer and transformation tasks.
  • Ensured data governance, data lineage, and metadata management.
  • Collaborated with cross-functional teams to align data solutions with business goals.
  • Guided junior team members and resolved complex data-related challenges.
  • Led and managed big data projects, optimizing costs and monitoring resource usage.

Big Data –Architect/Engineer

National Hearing Services Inc
02.2020 - 02.2022
  • Developed a data lake for Sonova to efficiently store and access structured and unstructured data from various sources.
  • Participated in business meetings to define deliverables based on requirements.
  • Designed and architected the data lake in Azure, utilizing Pyspark and Python.
  • Created EventHub endpoints for data ingestion from six sources, handling approximately 1 TB of data.
  • Managed data ingestion processes and ensured seamless data transfer between sources using Azure Data Factory (ADF).
  • Integrated AWS data sources with Azure Data Lake using Azure Databricks.
  • Built and deployed Docker containers to enhance developer workflows and provide updated images for predictive models.
  • Established a CI/CD pipeline using TeamCity and GitHub to automate builds and deployments.
  • Collaborated with the R&D team to implement data encryption for sensitive information during ingestion.
  • Help R&D team with the implementation of encrypting data while pushing into the Data lake so that sensitive data doesn't exposed.

Cloud Developer

FLIR Systems Inc
Waterloo, Ontario
07.2018 - 02.2020
  • Specifically focus on building out Cloud Services to help generate new revenue streams
  • Work closely with Architects and technical product managers to translate overall system architecture and product requirement into well designed and implemented software components
  • Creation of different AWS service like API, Lambda to build a Serverless application
  • Picking the right AWS services for the application
  • Creating different S3 buckets and write Lambda to move files from S3 to instance and FTP server to S3
  • Completes database architecture and design to help build the foundation of new systems
  • Develop logical data flows that support project requirements and guides colleagues in appropriate and effective use of data
  • Evaluates KPIs to identify trends and evaluate the success of business initiatives and creates models and reports to demonstrate observations and clearly communicate recommendations to senior management
  • Built and deploy Docker containers to improve developer workflow, increase scalability, and optimizing speed
  • Used Continuous integration tools like Jenkins, for build process automation and deployment process via Git Hub.

Senior Architect

DMI
02.2017 - 04.2017
  • Designed AWS architecture for cloud-based big data solutions with EMR, S3, and Lambda.
  • Developed strategies and roadmaps for modernizing data platforms.
  • Created real-time and batch data ingestion architectures using Lambda, Kafka, Spark, HBase, Sqoop, and Hive.
  • Implemented and tested large AWS clusters and proof-of-concept projects.
  • Delivered presentations on Hadoop best practices and architecture.
  • Mentored architects, reviewed architectures and code, and solved Hadoop issues.
  • Wrote and integrated scripts to enhance user experience.
  • Built and deployed scalable Hadoop ecosystems.
  • Managed project feasibility, costs, and multi-disciplinary teams.

Associate Architect

Western Union, Opus consulting Solutions
07.2015 - 11.2016
  • Designed a real-time analytics and ingestion platform using Spark and Kafka.
  • Architected Hadoop applications and recommended appropriate solutions and technologies.
  • Collaborated with clients and onsite counterparts for solution design and consulting.
  • Extracted log files and loaded them into HDFS using Sqoop for analysis.
  • Provided consulting on Big Data use cases and designed scalable Hadoop clusters.
  • Automated data ingestion into HDFS using Oozie workflows.
  • Translated high-level solutions into actionable deliverables and guided team implementation.
  • Led a team of 3+ on various Big Data solutions and managed infrastructure setup for POCs.
  • Prepared high-level designs, assigned tasks, tracked progress, and conducted status calls.
  • Presented demos to clients, received appreciation, and successfully transitioned POCs to project phase.Got Appreciation in implementing the project and succeeding in making the Poc move to Project phase.

Consultant Architect

Verizon Data services India
12.2014 - 07.2015
  • Understood the architecture of the Big Data ecosystem and its components.
  • Processed large sets of structured, semi-structured, and unstructured data.
  • Designed and implemented HQL scripts and wrote MapReduce jobs in Java and Pig.
  • Imported and exported log data into HDFS using Flume, and transformed data with Pig.
  • Loaded, transformed, and supported large data sets, including Hadoop configuration and support.
  • Analyzed data stores to uncover insights and maintained security and data privacy.
  • Consulted with clients to determine requirements for new structures.

Technology Analyst

Ford India Pvt Ltd
04.2012 - 06.2014
  • Understood Big Data ecosystem architecture and its components.
  • Designed and implemented HQL scripts and recommended architectural improvements.
  • Wrote MapReduce jobs in Java and Pig; imported and exported data to HDFS and Hive using Sqoop.
  • Loaded and transformed large data sets, managed data from various sources, and supported MapReduce programs.
  • Installed, configured Hive, created tables, and wrote Hive UDFs and queries.
  • Pulled data from FTP servers into Hive using Oozie workflows; generated result tables and analyzed systems.
  • Developed High-Level and Low-Level Designs (HLD and LLD), coded, and tested system functionalities.
  • Conducted weekly status meetings and prepared monthly reports; ensured effective onshore-offshore coordination.

Senior Software Engineer

Renault Nissan Technology Business Center India
12.2007 - 04.2012
  • Captured requirements from stakeholders and led enhancements and development projects.
  • Assigned tasks to team members and addressed bug fixes.
  • Implemented PowerCenter/PowerExchange for data conversion and migration from legacy IMS and VSAM mainframe systems to Oracle 11i.
  • Consulted on infrastructure and technology to ensure integration and alignment with other architecture domains.Consulted with stakeholders and other project team members in the areas of infrastructure, technology, platform, middleware, database, capacity, physical deployment, and service management to ensure consistency, alignment and integration with other architecture domains.

Software Engineer

UST Global
12.2005 - 12.2007
  • Interacted with the client to capture requirements and managed enhancements and development based on those needs.
  • Assigned tasks to team members and handled bug fixes.
  • Implemented PowerCenter/PowerExchange for data conversion and migration from legacy IMS and VSAM mainframe systems to Oracle 11i.
  • Utilized mainframe technologies such as IMS, VSAM, and DB2 in data migration and integration processes.

Education

Bachelor of Engineering in Computer Science -

Jerusalem College of Engineering
Chennai, Tamilnadu

Skills

Hadoop Ecosystem
Spark
Kafka
HBase
Hive
Pig

Pyspark
Python

Data integration techniques
ETL processes
Data pipelines
Data extraction, transformation, and loading (ETL)

Data modeling
Database design
Relational databases
NoSQL databases

AWS (including AWS Glue,S3,Cloudwatch,EMR,EC2,AWS Redshift)
Azure (including Azure Databricks, Azure Data Factory (ADF), Azure Logic Apps, Azure Storage,Azure Eventhub,Azure Webapps)

Docker
Kubernetes

Distributed computing
Parallel processing frameworks

Batch data
Real-time data

Data analytics
Machine learning techniques

Data governance principles
Data lineage
Metadata management
Data governance frameworks

Optimizing data pipelines
Performance bottlenecks
Serverless computing
Managed services

Certification

  • Certified by Microsoft on completion of Azure Fundamentals.
  • Certified Architecting Microsoft Azure solutions.
  • Certified Cloudera Hadoop Developer.

Timeline

Senior Data Engineer

Brillio
12.2023 - Current

Lead Data Engineer

HCL Technologies
02.2022 - 11.2023

Big Data –Architect/Engineer

National Hearing Services Inc
02.2020 - 02.2022

Cloud Developer

FLIR Systems Inc
07.2018 - 02.2020

Senior Architect

DMI
02.2017 - 04.2017

Associate Architect

Western Union, Opus consulting Solutions
07.2015 - 11.2016

Consultant Architect

Verizon Data services India
12.2014 - 07.2015

Technology Analyst

Ford India Pvt Ltd
04.2012 - 06.2014

Senior Software Engineer

Renault Nissan Technology Business Center India
12.2007 - 04.2012

Software Engineer

UST Global
12.2005 - 12.2007

Bachelor of Engineering in Computer Science -

Jerusalem College of Engineering
AISHWARYA RAMAN