Utilizing metadata and tagging in Azure Synapse Key Vaults to organize and categorize keys based on their purpose and usage.
Implementing advanced logging and performance monitoring strategies in SSIS packages to gather and analyze crucial execution metrics, guaranteeing data accuracy and efficiency.
Optimizing Tableau dashboards for performance and efficiency.
Implemented scalable feature extraction workflows to enhance the predictive power of AI-driven models.
Implement data replication and synchronization processes in Redshift to ensure data consistency across different environments and data sources.
Scheduled recurring data refresh jobs using Apache Airflow to supply up-to-date inputs for machine learning pipelines.
Conducted BI data profiling and metadata management for comprehensive data understanding.
Set up Git blame with the show-stats option to view statistics about changes in the database code.
Implementing and managing integration with Azure DevOps for version control and collaborative development in Azure Databricks.
Building and maintaining data warehouses and data marts, providing organizations with a solid foundation for data storage and retrieval.
Scheduling and automating SSAS ETL processes to ensure timely and accurate data updates, minimizing manual intervention, and streamlining workflows.
Develop and optimize complex SQL queries in Snowflake to extract and manipulate data for analysis.
Utilizing the Azure Logic Apps Connector for SAP to integrate SAP systems with other applications and services.
Develop and maintain data quality checks and data cleansing processes to ensure data integrity.
Demonstrating expertise in using Azure Data Factory's data flow transformation capabilities, including mapping, pivoting, and aggregating data.
Seamlessly transitioning Azure Purview catalogs between environments, assuring data consistency across different stages.
Ensuring data confidentiality and privacy through the thoughtful design and implementation of report security and access controls in SSRS.
Designing and developing Power BI dashboards with custom drill-through actions and cross-filtering.
Configured AWS Data Pipeline for ingesting and transforming data from various sources into AWS data stores.
Implementing and managing error handling and data retry mechanisms for robust data processing.
Implement robust data security measures on AWS Glue, including encryption, access controls, and data masking, to protect sensitive information, and ensure compliance with regulatory requirements.
Configuring and managing database replication and data mirroring for real-time data redundancy and failover capabilities.
Designing and implementing effective data partitioning strategies to optimize data storage and retrieval performance in Azure Data Lake.
Overview
6
6
years of professional experience
Work History
Sr Data and Cloud Engineer
Wave Financial
Toronto, Canada
04.2024 - Current
Creating and managing Power BI datasets and dataflows for data reuse and consistency across reports.
Developed logic to automatically flag outliers and anomalies in datasets prior to machine learning model input.
Created reusable feature extraction modules and stored them in centralized repositories for machine learning reuse.
Configuring and managing Azure Databricks integration with data virtualization platforms, promoting agile data handling.
Optimized storage and retrieval layers to accelerate access to high-volume AI datasets during training.
Created data validation rules to ensure accuracy and completeness of inputs used by AI inference systems.
Expertly configuring and managing data transformation logic, streamlining data processing within Azure Data Lake.
Collaborate closely with data engineers to optimize AWS Glue jobs, fine-tuning configurations, and resource allocations to improve job performance, scalability, and cost-effectiveness.
Design and implement Snowflake data archiving solutions for long-term data retention.
Develop and maintain AWS-based applications and services: designing, developing, and deploying applications and services on AWS, using a variety of AWS services and tools.
Synchronizing and coordinating key vaults with on-premises systems in Azure Synapse to enable seamless data encryption and decryption.
Utilize Snowflake multi-cluster warehouses for concurrent data processing and workload isolation.
Establishing and overseeing data lake integration with Azure Synapse Link, granting real-time data access and enabling agile analytics within Azure Data Lake.
Led efforts to establish data governance frameworks to support cultural change and adoption.
Conducting analysis of complex data sets in Power BI to identify patterns and trends, delivering valuable insights to stakeholders.
Provide technical support and troubleshooting for database and data warehouse issues.
Profoundly analysing data quality through Azure Purview's data profiling capabilities, identifying and addressing potential anomalies.
Conducting user acceptance testing and gathering feedback to iterate and improve Power BI solutions based on user requirements.
Collaborated with ETL developers to design and optimize SQL queries for efficient data extraction and loading.
Using Azure Purview to establish data classification policies that align with industry-specific regulatory standards.
Building and maintaining data warehouses and data marts, providing organizations with a solid foundation for data storage and retrieval.
Collaborated with cross-functional teams to define SQL database requirements and design data models for new applications.
Ingeniously utilizing Azure Data Factory's integration with Azure API Management, facilitating data integration with external APIs.
Proficiency in time series analysis techniques such as autocorrelation, spectral analysis, and time series decomposition for understanding temporal patterns, crucial for time-based data analysis.
Implementing and managing data exploration and data visualization capabilities in Azure Synapse, enabling data-driven insights and decision-making.
Implementing Azure Arc-enabled Kubernetes to manage Kubernetes clusters across on-premises, multi-cloud, and edge environments.
Implement Redshift workload management (WLM) to prioritize and manage query queues.
Setting up and managing integration with Azure Monitor for comprehensive performance monitoring in Azure Data Bricks.
Crafting dynamic parameterization and expressions in Azure Data Factory pipelines for flexible and adaptive data integration, delivering customized solutions.
Developed and maintained ETL metadata management processes to track and manage changes to data structures.
Developing Tableau data connections for real-time data analysis and visualization.
Creating interactive Power BI dashboards and reports that provide real-time data visualization.
Set up Git's pre-push hook to prevent pushing commits that do not meet certain criteria.
Designing Tableau dashboards with drill-through capabilities for detailed data analysis, allowing users to navigate from summary information to detailed data.
Designed and deployed AI-powered predictive models for customer churn, sales forecasting, and anomaly detection, integrating seamlessly with business intelligence tools for actionable insights.
Implemented ML Ops pipelines to operationalize machine learning models, ensuring scalable, monitored, and version-controlled AI deployments in production environments.
Applied NLP techniques to extract key entities and sentiments from unstructured customer feedback, enhancing data-driven decision making.
Built automated retraining pipelines for AI models using Azure Machine Learning, maintaining accuracy as data patterns evolve.
Leveraged generative AI models for data augmentation and synthetic data generation to overcome data sparsity challenges.
Integrated AI model explainability tools (e.g., SHAP, LIME) into dashboards to provide transparency and trust in model predictions.
Designed and executed A/B testing frameworks for AI model performance evaluation and selection.
Implemented reinforcement learning algorithms to optimize resource allocation and scheduling in dynamic environments.
Data and MSBI Engineer
Creospark
Ontario, Canada
03.2022 - 04.2024
Setting up and overseeing data lake integration with Azure Key Vault, ensuring secure management of cryptographic keys.
Configuring and managing Azure Service Bus for reliable messaging and event-driven communication.
Maintained version-controlled datasets to support reproducibility in iterative AI model development.
Generated synthetic data to supplement rare cases and improve generalization in AI models.
Conducting performance testing and tuning exercises in Power BI to improve the overall performance and responsiveness of solutions.
Constructed performance-optimized SQL queries for extracting training datasets for machine learning models.
Integrated data validation checkpoints to monitor input quality during machine learning pipeline execution.
Create and maintain serverless architectures using AWS Lambda, API Gateway, etc : This involves designing and building serverless architectures using AWS Lambda, API Gateway, and other AWS services.
Diligently monitoring data integration performance and resource utilization, proactively optimizing costs and efficiency within Azure Data Factory, ensuring a symphony of cost-effective data orchestration.
Implemented SQL server database mirroring and log shipping for data redundancy and disaster recovery.
Develop and maintain data visualization solutions on AWS Glue, enabling stakeholders to easily access and interpret data through interactive dashboards and reports.
Designing custom data lineage reports within Azure Purview to support compliance audits and data governance reviews.
Creating and managing Power BI datasets with data profiling and quality checks.
Designing and executing Azure Databricks architectures for big data processing, handling large datasets effectively.
Design and implement Snowflake data migration strategies for moving data between environments.
Have a good understanding of AWS Security Hub and be able to use it to aggregate and prioritize security alerts and compliance findings across their AWS accounts.
Leveraging advanced aggregation techniques in OLAP cubes to improve query performance and reduce data processing time.
Implement Snowflake data sharing for collaboration and data monetization, enabling organizations to share data securely and efficiently.
Envisioning and crafting disaster recovery and business continuity strategies for Azure Data Lake, safeguarding data integrity and ensuring continuous data availability.
Configure Snowflake data masking for protecting sensitive data, ensuring that only authorized users have access to sensitive information.
Developing and optimizing database indexes for improved query performance, creating indexes to enhance the speed of data retrieval operations.
Designing Tableau dashboards with storytelling techniques to engage and inform stakeholders.
Developing Tableau data actions for interactive dashboard filtering.
Designing and developing Power BI dashboards with custom navigation buttons for user-friendly exploration.
Designing and executing Azure Databricks integration with data privacy and compliance regulations, prioritizing data protection.
Implementing advanced logging and performance monitoring strategies in SSIS packages to gather and analyze crucial execution metrics, guaranteeing data accuracy and efficiency.
Articulating technical specifications with unparalleled accuracy, skillfully translating business users' report requirements into seamless execution in Azure Data Factory.
Designing and implementing data archival and data purging strategies for long-term data retention.
Developing and maintaining database backup and recovery plans for PostgreSQL databases.
Designing and executing data transformation and data cleansing processes in Azure Synapse, enhancing data quality and reliability.
Designed and executed SQL scripts for database schema migration and version control.
Data and BI Analyst
Bharti AXA Life Insurance Co. Ltd.
Mumbai, India
01.2020 - 02.2022
Skillfully crafting SQL queries and stored procedures to optimize data retrieval for SSRS reports, skillfully balancing accuracy and efficiency.
Collaborating with cross-functional teams to identify and resolve data-related issues and discrepancies.
Creating and maintaining data lineage documentation to track the origin and transformations applied to the data.
Collaborating with data engineers and administrators to ensure seamless data integration and availability in Power BI. Providing input and guidance on data architecture and infrastructure requirements.
Setting up and meticulously managing data retention and archival policies in Azure Data Lake, ensuring data preservation and compliance.
Designing Tableau dashboards with drill-down capabilities for deeper data exploration.
Meticulously creating and maintaining report server projects and report models in SSRS, providing a robust foundation for efficient and scalable reporting.
Skilled in creating custom data connectors for Power BI, expanding data connectivity options and enabling seamless integration with various data sources.
Artfully designing and implementing data flow tasks within SSIS packages, formulating a logical and systematic approach to data transformation and manipulation.
Skilfully developing and maintaining SSIS project configurations, expertly parameterizing package execution, and permitting dynamic configuration adjustments.
Collaboratively engaging with business users to gather detailed report requirements and translating them into precise technical specifications for SSRS reporting.
Designing and implementing attribute hierarchies and member properties in SSAS dimensions, defining the hierarchical relationships and properties of dimension attributes.
Skillfully implementing custom error handling and retry logic in Azure Data Factory pipelines, ensuring unwavering data reliability.
Creating backup and restoration solutions for cryptographic keys in Azure Synapse to facilitate disaster recovery and data continuity.
Implementing and managing integration with Azure Event Hubs for real-time data ingestion and processing in Azure Data Bricks.
Implementing and managing data migration and data integration for cloud migration projects.
Configure Snowflake resource monitors and query profiling to analyze query performance and optimize resource allocation.
Establish disaster recovery plans to mitigate risks and ensure business continuity in critical situations.
Created and managed SQL server maintenance plans for automating routine database maintenance tasks, such as index rebuilds and backups, to optimize performance.
Skillfully crafting SQL queries and statements within SSIS components, empowering data manipulation during the ETL process.
Meticulously configuring and overseeing package connections to a plethora of data sources and destinations in SSIS, expertly establishing the vital links required to extract and load data.
Seamlessly transitioning Azure Purview catalogues between environments, assuring data consistency across different stages.
Education
Bachelor of Engineering (B.E.) -
RMD Engineering College
Chennai, Tamilnadu, India
Skills
Business Intelligence Delivery
Power BI
SSRS
Modern Data Engineering
Azure
Azure Data Factory
Databricks
Azure SQL
Blob Storage
ETL
Data Movement Architecture
SSIS
Spark
Slowly Changing Dimensions
Data Systems Architecture
OLTP
OLAP
AI
Machine Learning
Scripting
Task Automation
Python
PowerShell
T-SQL
Database Ecosystem Management
RDBMS
NoSQL
SQL Server
MongoDB
MySQL
MS Access
Development Ecosystem Proficiency
SQL Server Management Studio
SQL Server Data Tools
Visual Studio
Versioning
Team Collaboration
Git
Team Foundation Server
Data Platform Flexibility
Windows Server
UNIX
Linux
Overview
Data and Power Bi Analyst with around 6 years of hands-on experience in building cloud-based data solutions, developing BI architectures using Synapse, ADF, and Power BI, and optimizing performance across hybrid data platforms to support machine learning workflows and AI model deployment.