Srujan kumar
Scarborough,Canada
Summary
Over 5 plus years of experience as a Data Engineer, building end-to-end data solutions including ETL pipelines, data warehousing, and business intelligence in cloud environments. Extensive experience with Snowflake, AWS, Azure, Databricks, and hands-on work with Hadoop, Spark, Kafka, and NoSQL databases like Cassandra and HBase. Skilled in data modeling using 3NF, Star, and Snowflake schemas, designing data pipelines, and managing data governance and metadata for OLAP and ODS systems. Strong programming experience in Scala, Python, and Java, with expertise in building microservices and RESTful APIs for high-volume, high-concurrency applications. Experienced in transforming complex data into actionable insights using Tableau, PySpark, and Spark SQL. Proven ability to lead teams in Agile environments, manage CI/CD pipelines, and deliver projects from architecture and requirement gathering to deployment.
Overview
5
5
years of professional experience
Work History
Data Engineer
Rogers Communication
01.2024 - Current
- Worked on building scalable and efficient data processing pipelines for business analytics and reporting, leveraging Spark, PySpark, Scala, and Azure services including Databricks, Data Factory, Synapse, and SQL Data Warehouse.
- Developed ETL workflows to transform raw data into optimized datasets, processed both batch and real-time data, and implemented monitoring and logging solutions to ensure pipeline performance.
- Collaborated with cross-functional teams to design data-driven solutions, ingesting data into Azure Blob Storage, Data Lake, and SQL Data Warehouse, and creating automated data quality checks to maintain accuracy.
- Designed and deployed Azure Databricks workspaces, built notebooks for extracting and transforming raw data, and implemented proof-of-concept SOAP and REST APIs. Developed interactive dashboards using Power BI and integrated visuals based on business requirements.
Data Engineer
Air Canada
09.2022 - 12.2023
- At DentalCorp, designed and maintained robust data pipelines and architectures using Snowflake, Databricks, Airflow, Python, and AWS services such as EC2, S3, Lambda, RDS, Athena, and CloudWatch.
- Built ETL pipelines using PySpark and Spark SQL, migrating legacy systems to cloud data warehouses, and implemented advanced data models with star and snowflake schemas in Snowflake.
- Automated data ingestion and pipeline orchestration with Airflow, scheduled DAGs for Spark and Hive jobs, and monitored AWS services for data migration.
- Developed APIs in Python, deployed Elasticsearch for monitoring, and created interactive reports in Tableau. Focused on scalable, flexible, and secure solutions for enterprise data analytics.
Data Analyst
New Horizons Cybersoft LTD
09.2020 - 08.2022
- Designed and developed scalable big data solutions using Hadoop, Spark, Hive, and PySpark, supporting both batch and real-time processing with Kafka and Cassandra.
- Migrated data between RDBMS and HDFS using Sqoop, optimized Hadoop workflows, and implemented Spark applications for recommendation engines, fraud detection, and behavioral analytics.
- Built ETL pipelines with Hive and Spark SQL, created custom Hive UDFs, and performed data analysis using Python libraries such as Pandas and NumPy. Managed AWS cloud infrastructure including EC2, S3, EMR, and Lambda, ensuring system reliability and scalability.
- Developed interactive dashboards in Tableau and Power BI to provide actionable business insights, collaborating with cross-functional teams to maintain data privacy and compliance.
- Performed comprehensive data management, including analysis, gap assessment, and data mapping. Created SQL extracts, stored procedures, and reports using T-SQL, PL/SQL, SSIS, and SSRS for data validation and transformation. Conducted extensive data profiling and integration, prototyped Tableau visualizations for executive decision-making, and developed reusable data objects and an enterprise data dictionary to ensure consistent reporting across the organization.
Education
Bachelor’s degree - Computer Science and technology
Gitam University
Skills
- Big Data
- Hadoop, Spark, Hive, Kafka, Sqoop, HBase
- Databases
- MySQL, Oracle, SQL Server, PostgreSQL, MongoDB, Cassandra
- Cloud
- AWS (S3, Redshift, EMR, Lambda, Glue), Azure (ADF, Databricks, Synapse, SQL DW)
- Programming
- Python, SQL, PySpark, Scala
- Tools
- Airflow, Jenkins, Git, Tableau, Power BI
- Data Warehousing
- Snowflake, Redshift, Databricks
Technologies and Tools
C++, C, Java, Objective-C, C#, SQL, JavaScript/PySpark, Spark SQL, Scala, Python, T-SQL, U-SQL, Azure Databricks, Azure Data Factory, Azure Synapse, Azure SQL Data Warehouse, Azure HD Insights, Azure Blob, Azure Functions, Snowflake, Airflow, AWS (EC2, S3, Lambda, RDS, Athena, CloudWatch), Hadoop, HDFS, Hive, Sqoop, Kafka, Cassandra, Power BI, Tableau, SSIS, SSRS, PL/SQL, SQL Server, MongoDB, UNIX, Git, Jenkins
Timeline
Data Engineer
Rogers Communication
01.2024 - Current
Data Engineer
Air Canada
09.2022 - 12.2023
Data Analyst
New Horizons Cybersoft LTD
09.2020 - 08.2022
Bachelor’s degree - Computer Science and technology
Gitam University