Summary
Overview
Work History
Education
Skills
Timeline
Generic

Nikhil Bhardwaj

San Francisco,CA

Summary

Data has been the central focus of my career within information systems. Whether it be business intelligence and data warehousing as a means for the informative visualization of previously disparate data, or database development of ETL solutions for analytical applications, data has been the focal point of my career knowledge and experience.

Overview

10
10
years of professional experience

Work History

Data Engineer

American Agcredit
08.2019 - Current
  • Led migration efforts to migrate from RDBMS to Snowflake cloud data warehouse.
  • Migrated complex stored procedures to dbt code as part of migration project.
  • Implemented dbt in a multi-terabyte database, reducing data load times by 40%.
  • Developed custom dbt macros to automate and enhance data validation and testing.
  • Used dbt to test the data (schema tests, referential integrity tests, custom tests) and ensures data quality.
  • Used dbt to debug complex chain of queries. splitting into multiple models and macros that can be tested separately.
  • Implement CI/CD pipelines for data solutions using Azure DevOps.
  • Used Airflow/Tidal to orchestrate pipeline runs.
  • Integrated Apache Spark with Azure Databricks to streamline data pipeline constructions, reducing system latency by 25%.
  • Implemented ETL processes utilizing Azure Data Factory in few projects, resulting in a 30% increase in workflow efficiency
  • Conduct data quality assessments and implement data governance frameworks.
  • Participate in agile ceremonies and contribute to iterative project development.
  • Lead the team towards successful project release and acts as a lead b/w onshore and offshore to have a proper handshake.

Data Engineer

TD Ameritrade
05.2018 - 05.2019
  • Led a complex integration project migrating 13 data sources from on-premises SSIS to Azure Data Factory, improving scalability and reducing infrastructure costs by 30%.
  • Designed and implemented Azure-SSIS Integration Runtime in Azure Data Factory to run existing SSIS packages in the cloud, ensuring seamless transition of ETL workloads.
  • Utilized Azure Data Factory's GUI to create and manage data pipelines, reducing development time by 25% compared to traditional SSIS development.
  • Implemented data flow transformations in Azure Data Factory to handle complex data manipulations previously done in SSIS.
  • Migrated SQL Server Agent jobs to Azure Data Factory pipelines and triggers using SSMS SSIS Job Migration Wizard, automating scheduling and monitoring of ETL processes.
  • Optimized data integration processes by leveraging Azure Data Lake Storage and Azure Synapse Analytics, resulting in a 40% improvement in data processing speed.
  • Designed and implemented end-to-end data solutions in Azure, including storage, integration, processing, and visualization components.
  • Established best practices for Azure Data Factory development, including source control integration and CI/CD pipelines, improving team collaboration and code quality.
  • Conducted knowledge transfer sessions to upskill the team on Azure Data Factory concepts and best practices, facilitating a smooth transition from SSIS.
  • Implemented robust logging and monitoring solutions in Azure Data Factory, enhancing visibility into ETL processes and reducing troubleshooting time by 50%.

Data Engineer

BMO
09.2017 - 05.2018
  • Led the implementation of a comprehensive data masking strategy, reducing exposure of sensitive data in non-production environments by 95%.
  • Designed and executed data masking proof of concept for multiple application teams, improving data security practices across the organization.
  • Implemented IBM Optim for data masking, resulting in a 60% reduction in time to detect and resolve data quality issues.
  • Conducted workshops with application team leads and subject matter experts to gather requirements for data masking implementation.

Business Intelligence Analyst

Healthcare Ontario
05.2016 - 09.2017
  • Designed and implemented ETL processes using SSIS, improving data processing efficiency by 30%.
  • Developed complex SSRS reports and dashboards, reducing reporting time by 20%.
  • Created and optimized SSAS cubes, enhancing data analysis capabilities for business users.
  • Implemented automated backup and database maintenance jobs, ensuring data integrity and availability.
  • Redesigned BI dashboard resulting in a 15% time decrease in data analysis.

Data Engineer

CIBC
09.2015 - 04.2016
  • Implemented IBM InfoSphere Optim Data Privacy solution to mask sensitive data across non-production environments, reducing data privacy risks by 95%.
  • Designed and executed data masking policies using IBM Optim's comprehensive set of techniques, including random masking, shuffling, and string literal substitution.
  • Developed custom masking routines in IBM Optim to transform complex data elements while retaining contextual meaning and referential integrity.
  • Utilized IBM Optim's GUI and CLI interfaces to create and manage data privacy projects, improving efficiency in masking operations by 30%.
  • Implemented policy-based masking rules in IBM Optim, ensuring consistent application of data privacy standards across the enterprise.
  • Leveraged IBM Optim's data discovery capabilities to identify and classify sensitive information across multiple databases and file systems.
  • Created smaller, representative data subsets using IBM Optim's data subsetting feature for efficient development and testing environments.
  • Automated data masking processes in IBM Optim, reducing manual effort and ensuring consistent protection of sensitive data.
  • Utilized IBM Optim to mask data in various formats including CSV, XML, and database tables, demonstrating versatility in handling different data sources.
  • Generated comprehensive audit reports using IBM Optim's reporting features to document masking activities for compliance purposes.

Education

Master of Science - Computer Science

University of Guelph Humber
Toronto
05-2015

Bachelor of Science - Computer Science

Panjab University
India
05-2013

Skills

    Languages: Java, SQL, PL/SQL, Python

    Databases: Snowflake, SQL Server, Oracle Database 10g/11g, DB2, Netezza, Teradata150

    Web Technologies: HTML 5, CSS3, JavaScript, jQuery, AJAX, JSON, XML

    ETL/ELT Tools: DBT data build tool, Azure Data Factory, IBM Datastage, Informatica Powercenter, SSIS, Databricks

    Data Warehouse: Snowflake Cloud, Amazon Redshift, SQL MI, Azure synapse analytics

    BI and Reporting: Tableau, Power BI

    Web/Application Servers: Apache Tomcat

    Version Control Tools: GIT, SVN, Azure Devops

    Orchestration Tools: ADF, Airflow, Tidal, Stonebranch

    Development Tools: Visual Studio, NetBeans, Eclipse

    Methodologies: Agile, Waterfall, Kanban

Timeline

Data Engineer

American Agcredit
08.2019 - Current

Data Engineer

TD Ameritrade
05.2018 - 05.2019

Data Engineer

BMO
09.2017 - 05.2018

Business Intelligence Analyst

Healthcare Ontario
05.2016 - 09.2017

Data Engineer

CIBC
09.2015 - 04.2016

Master of Science - Computer Science

University of Guelph Humber

Bachelor of Science - Computer Science

Panjab University

Similar Profiles

CREATE PROFILE
Nikhil Bhardwaj