Nandha Subramaniam
Toronto,ON
Summary
Proven leader with 16+ years of experience in Data platform engineering and Data Architecture. Extensive experience in helping Banking and Financial services organizations to build Data & Analytics platforms, migrate data platforms, and navigate data architecture and transformation journeys.
Overview
17
17
years of professional experience
1
1
Certification
Work History
Senior Platform Engineer
Scotiabank
05.2023 - Current
- Spearheaded the migration and transformation of legacy systems to Google Cloud Platform (GCP), ensuring seamless integration and optimization of cloud services.
- Designed and implemented infrastructure as code (IaC) using Terraform to provision and manage resources such as Big Query Datasets, Composer, GCS, Cloud Functions, Compute Engine, MIGs, VPC SC rules, Subnets, and Firewalls.
- Utilized Google BigQuery for data warehousing and analytics, enabling efficient data querying and reporting using PowerBI and Looker
- Implemented CI/CD pipelines using Cloud Build, automating the build, test, and deployment processes to enhance development efficiency and reduce time to market
- Developed and deployed serverless applications using Cloud Functions, Cloud Scheduler and Pubsub
- Defined monitoring requirements and implemented automated incident resolution solutions to maintain system health and performance using Google Ops Suite
- Implemented and managed IAM policies to ensure secure and compliant access to cloud resources.
Principal Data Architect
Accenture
04.2022 - 04.2024
- Led Merger and Acquisition (M&A) program for one of the largest Canadian banks
- Evaluated to-be acquired bank’s source systems of analytical and BI platforms.· Review and validate the existing data models, update existing models or build new models and mappings based on the business requirements.
- Identified the consumption patterns of the source data and curated data in Analytics platforms and perform impact analysis of source to target data transformation
- Worked with both the banks’ CRM and customer data platforms and plan for conversion activities
- Build migration plan, design ETL pipeline architecture, historical data movement and implementation strategy
- Strategized mock execution and Dress Rehearsals for Conversion Day
Principal Data Architect
Tech Mahindra
12.2019 - 04.2022
- As part of Data & Analytics Practice, majorly helped BFSI customers across Canada to build their Data & Analytics platforms
- Evaluating Cloudera Data Platform (Data Center) solution for building a new On-Premise data lake which would decouple the storage and compute, provide dedicated compute clusters to the customers based on their workload
- Helped to build and manager Rancher Kubernetes environment for running analytics workload using custom Spark framework and Talend. Enabling visualization through Trino / Presto SQL engine for Power BI. Managed Minio object storage on K8S
- Helped to deploy Collibra and Erwin infrasturctures for Data Governance and Data Modeling
- ·Helped evaluating existing Data Lake workloads (Ingestions, Transformation and Consumption) and cluster configurations and providing suggestions to improve the performance related issues.
Big Data Architect
TATA Consultancy Services
04.2017 - 11.2019
- Evaluated Netezza based ETL architecture and proposing effective and efficient solutions for migrating the applications to Hadoop framework.
- Designed, developed and implemented Map-Reduce programs for converting the file formats which can be read by Apache Hive
- Converted Netezza SQL to HiveQL and Spark SQL and deployed ETL pipelines
- Designed and deployed daily and historical data migration jobs from Netezza to Hadoop and AWS S3
- Conducted PoCs and PILOTs for migrating workloads to AWS Redshift and Snowflake
- Desiged and implemented the data transfer approach between On-Premise and Cloud storage using AWS sFTP Services through Direct Connect
Senior Hadoop Developer
TATA Consultancy Services
01.2015 - 04.2017
- Migrated Deposits and Cards data to Hadoop Eco System as part of Hadoop as a Service Initiative which would help the client to use multiple open source tools, replacing traditional ETL with Hadoop components and managing peta bytes of data
- Participated in multi-organizational, multi-disciplinary teams to determine, articulate, & codify business requirements
- Used SQOOP for copying large volume of data from Teradata to HDFS
- Used Oozie for scheduling the workflows for moving the data using SQOOP, analyzing and transforming the data using Hive and Pig.
- Used webHDFS for copying the data from non-Hadoop cluster to Hadoop cluster
- · Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
Hadoop Administrator
TATA Consultancy Services
12.2013 - 01.2015
- Installed and configured Hadoop components using Cloudera Manager.
- Responsible for certifying and benchmarking tools and services which are installed on clusters before on-boarding the application teams
- Work with architecture and engineering teams to propose new tools required for business
- Working with the environment management teams to setting up Hadoop users, authorizing and setting up Kerberos
- Monitoring the Hadoop clusters through Cloudera Manager and Navigator
- Monitoring ETL jobs, manage and review logs
Teradata Developer
TATA Consultancy Services
12.2007 - 11.2013
- ·Ensured the stability of clients’ Teradata production and disaster platforms using Teradata manager and Teradata ViewPoint.
- Monitoring the data load, load delays and reloads
- Analysis of Teradata SQL to resolve issues from clients and users
- ·Query Tuning and fixing the performance issues
- · Creating databases, tables/views and granting access to the users
- Data movement from production to lower platforms for testing
Education
Master of Science - Computer Applications
SASTRA University
Tanjore, India04.2011
Skills
Google Cloud Platform(GCP)
Terraform
Apache Hadoop
Kubernetes
Data Engineering
Data Migration
Platform Engineering
ETL
Python
Certification
Google Certified Professional Cloud Architect
Timeline
Senior Platform Engineer
Scotiabank
05.2023 - Current
Principal Data Architect
Accenture
04.2022 - 04.2024
Principal Data Architect
Tech Mahindra
12.2019 - 04.2022
Big Data Architect
TATA Consultancy Services
04.2017 - 11.2019
Senior Hadoop Developer
TATA Consultancy Services
01.2015 - 04.2017
Hadoop Administrator
TATA Consultancy Services
12.2013 - 01.2015
Teradata Developer
TATA Consultancy Services
12.2007 - 11.2013
Master of Science - Computer Applications
SASTRA University
Similar Profiles
Raquel JohnRaquel John
Senior Customer Care Advisor-Social Media at ScotiabankSenior Customer Care Advisor-Social Media at Scotiabank
Kevin LevyKevin Levy
Director, Commercial Card Solutions, Global Transaction Banking, North America at ScotiabankDirector, Commercial Card Solutions, Global Transaction Banking, North America at Scotiabank
Radhika NarayananRadhika Narayanan
Officer III, Commodities Wind-down Operations at ScotiabankOfficer III, Commodities Wind-down Operations at Scotiabank
Madhav MakkerMadhav Makker
Senior Customer Experience Associate at ScotiabankSenior Customer Experience Associate at Scotiabank
Sinthuja VijayarajSinthuja Vijayaraj
Intern Waitress at Cibo Wine BarIntern Waitress at Cibo Wine Bar