Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Ram Charan Madaram

San Francisco

Summary

Dynamic and highly experienced Data Engineer with over 5 years of expertise in designing, deploying, and managing robust, large-scale data solutions across diverse cloud platforms, with a primary focus on AWS and Azure. My background includes a deep specialization in building resilient, high-performance ETL/ELT pipelines for both batch and real-time streaming systems, leveraging a modern tech stack that includes Apache Spark, Kafka, and Snowflake. I am adept at utilizing Python for data processing and automation, writing complex SQL for optimization and transformation, and operationalizing data workflows using advanced CI/CD pipelines. I am a collaborative problem-solver with a proven track record of optimizing data infrastructure, ensuring data quality and governance, and delivering scalable data solutions that directly align with key business objectives.

Overview

6
6
years of professional experience
1
1
Certification

Work History

Data Engineer

Intuit
Mountain View
08.2024 - Current
  • Designed and deployed cross-cloud ETL workflows using Azure Data Factory and AWS Glue, enabling ingestion and transformation of over 10 TB of multi-format data from diverse sources.
  • Built real-time data pipelines using Kafka + Spark Structured Streaming, ensuring <2s latency processing for transactional events and telemetry data.
  • Developed modular PySpark and Scala Spark jobs in Azure Databricks, leveraging Delta Lake for ACID-compliant storage layers and data versioning.
  • Managed ingestion from AWS S3 and Azure Data Lake Storage Gen2 into Snowflake, applying robust data quality checks, schema evolution, and schema-on-read techniques.
  • Implemented Snowflake optimization strategies (clustering, caching, pruning), reducing average query latency by over 40%.
  • Automated infrastructure deployments using Terraform for Azure and CloudFormation for AWS.
  • Leveraged Azure Functions and AWS Lambda for event-driven enrichment, preprocessing, and validation at ingestion.
  • Established CDC pipelines using Kafka Connect and Snowflake Streams/Tasks to maintain synchronization across OLTP systems and analytical stores.
  • Engineered complex CI/CD pipelines using Azure DevOps for Spark/ADF/DataBricks and Jenkins for AWS Glue/Lambda jobs with automated unit tests, linters, and security scans.
  • Used Apache Airflow and Azure Data Factory triggers for orchestrating dependent workflows and managing SLAs across multi-region pipelines.
  • Conducted large-scale partitioning/bucketing strategies in Hive and Snowflake to optimize read-intensive workloads.
  • Enabled Power BI dashboards using curated models from Snowflake and integrated them into real-time refresh cycles via Azure Synapse.
  • Tuned Spark jobs via broadcast joins, persist/cache, and resource provisioning across YARN and Databricks autoscaling clusters.
  • Integrated Azure Monitor, AWS CloudWatch, and Datadog for cross-cloud observability and alerting of batch and streaming jobs.
  • Applied row-level security policies in Snowflake for secure data sharing across business units.
  • Participated in rigorous Agile ceremonies and sprint planning using JIRA, delivering bi-weekly production-ready features.

Data Engineer

HCl
Hyderabad
07.2022 - 07.2023
  • Developed complex T-SQL stored procedures, functions, and triggers within Azure SQL Database to implement business logic and automate critical data processes.
  • Designed and developed Azure Data Factory (ADF) pipelines for robust ETL operations, extracting and loading data from diverse sources like Azure Blob Storage and other flat file sources into Azure SQL Database and Azure Synapse Analytics.
  • Automated ETL jobs using ADF triggers and scheduling, including managing and configuring alerts for pipeline statuses and failures through Azure Monitor and Log Analytics, ensuring operational continuity.
  • Enforced real-time monitoring and reporting dashboards using Power BI and Azure Monitor for end-to-end pipeline visibility and operational transparency, enabling proactive issue resolution.
  • Developed complex reports using Power BI and Azure Synapse Analytics, including parameterized, sub-reports, drill-down, and summary reports, supporting data-driven decision-making.
  • Created and enhanced database objects (tables, indexes, views) and designed constraints to ensure data integrity across various Azure data services.
  • Collaborated with cross-functional teams to define requirements for seamless data exchange between multiple systems using Azure Integration Services.
  • Deployed ADF pipelines and Power BI report from development to production environments using CI/CD practices with Azure DevOps, ensuring smooth transitions and minimal downtime.

Data Engineer

Accenture
Hyderabad
12.2019 - 06.2022
  • Migrated 100+ legacy ETL processes from Oracle/SSIS to Azure Synapse and Data Lake Storage Gen2, improving performance by 60%.
  • Created and managed HDInsight Spark clusters, used for large-scale transformations and ML pre-processing workflows.
  • Designed end-to-end batch pipelines using ADF + Azure Databricks, applying business logic using SparkSQL and PySpark.
  • Developed Hive UDFs and partitioned tables to efficiently segment medical data by specialty, geography, and modality.
  • Used Apache Sqoop to synchronize relational data (SQL Server, Oracle) into HDFS for data lake integration.
  • Implemented data masking logic within Spark pipelines to comply with HIPAA regulations.
  • Migrated SQL workloads to Azure Synapse and implemented Polybase for high-speed querying of external data sources.
  • Automated Azure DevOps pipelines with YAML to package and deploy Spark jobs, logic apps, and configuration templates.
  • Created and managed complex Spark-based scoring jobs for risk classification models, supporting clinical decision engines.
  • Developed and deployed monitoring dashboards using Power BI + Azure Log Analytics for pipeline observability.
  • Used Git & Azure Repos for code versioning and implemented branch strategies aligned with Scrum sprint cycles.

Education

Master of Science - Computer Technology

Eastern Illinois University
Charleston, IL

Bachelor of Science - Computer Science

JNTUH
Hyderabad ,india

Skills

  • Hadoop
  • Hive
  • Spark
  • Kafka
  • Oozie
  • Sqoop
  • MapReduce
  • Zookeeper
  • Azure
  • AWS
  • Snowflake
  • Azure SQL DB
  • SQL Server
  • Oracle
  • MySQL
  • PostgreSQL
  • Cosmos DB
  • Azure DevOps
  • GitHub Actions
  • Jenkins
  • YAML
  • Terraform
  • Power BI
  • Tableau
  • Apache Airflow
  • Azure Data Factory
  • AWS Step Functions
  • Avro
  • Parquet
  • Delta
  • JSON
  • CSV
  • Git
  • GitHub

Certification

  • AWS Cloud Practitioner
  • Microsoft Certified: Azure Data Engineer Associate

Timeline

Data Engineer

Intuit
08.2024 - Current

Data Engineer

HCl
07.2022 - 07.2023

Data Engineer

Accenture
12.2019 - 06.2022

Master of Science - Computer Technology

Eastern Illinois University

Bachelor of Science - Computer Science

JNTUH

Similar Profiles

CREATE PROFILE
Ram Charan Madaram