Lakhwinder Singh
Montreal,QC
Summary
Highly-motivated employee with desire to take on new challenges. Strong work ethic, adaptability, and exceptional interpersonal skills. Adept at working effectively unsupervised and quickly mastering new skills.
Overview
12
12
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
National Bank of canada
Montreal, Quebec
06.2020 - Current
- Created ETL scripts to move and transform data from various sources into a centralized repository.
- Automated manual processes by creating custom scripts and programs using scripting languages like Bash, PowerShell, python.
- Deployed aws cloud using terraform.
- Deployed and maintained Databricks on AWS Cloud EC2 instance.
- Worked with snowflake for data warehousing.
- Created custom lambda function for managing users on databricks.
- Migrated apps from azure to aws and worked on dynamoDB, lambda, cloudwatch, eventbridge, Step function, EC2, IAM etc.
- Worked on confluent kafka to stream messages and send to api.
- Pushed messages of kafka on splunk.
- Used Ansible and Packer to automate the VM creation.
- Launched new custom AMI with new feature each quarter.
- Configured, managed, and monitored cloud-based services such as AWS EC2, S3, EBS, ELB, RDS using Terraform and Ansible.
- Developed data pipelines to ingest and process large datasets.
- Built complex reports utilizing multiple sources of information from different systems.
- Ensured data accuracy through regular testing and validation procedures prior to deployment in production environments.
- Performed advanced analytics on structured and unstructured data using SQL, Python,.
Data Engineer
CGI
Montreal, Quebec
09.2017 - 05.2020
- Collaborated with other members of the team to design efficient solutions for managing Big Data workloads.
- Performed end-to-end testing of all components in the Big Data architecture.
- Worked on Hadoop HDFS, Hive, Sqoop, PIG, Spark,
- Optimized queries on distributed systems like Hive and Impala or Presto to improve performance of analytics tasks.
- Implemented real-time streaming applications using Kafka Streams or Spark Streaming.
- Used scripting languages such as Python or Bash to automate routine tasks associated with Big Data engineering.
- Implemented best practices around designing fault tolerant architectures in a distributed computing environment.
- Created and maintained Hadoop clusters with multiple nodes.
- Hand on experience with Flume, Oozie, MapReduce and Hbase.
- Knowledge of Flume, Zookeeper and other components from Hadoop ecosystem.
- Experience in analyzing data using HiveQL, Pig Latin.
- Hands on experience in Apache Hive and Apache Pig.
- Provided technical support to business users on using Big Data tools for their analytical needs.
- Built data pipelines using Apache Spark to process large amounts of structured and unstructured data.
Hadoop Developer
Acadgild
Remote
02.2016 - 07.2017
- Configured name nodes, resource managers, job trackers, task trackers in the cluster environment.
- Installed and configured the Hadoop framework on Linux servers.
- Built custom user-defined functions to extend functionality of HiveQL queries.
- Designed efficient algorithms for processing large amounts of data stored in HDFS or other file systems.
- Tuned Apache Hadoop clusters to improve performance of MapReduce jobs.
Hardware Verification Engineer
Concordia university
Montreal, Quebec
05.2016 - 08.2016
- Identified issues with hardware designs and provided solutions accordingly.
- Wrote test plans and test cases for hardware verification activities.
- Collaborated with cross-functional teams to develop efficient testing strategies.
- Performed functional testing of various integrated circuits.
- Executed tests on FPGA platforms to validate designs against specifications.
Junior Java Developer
TCIL-IT
Chandigarh, India
01.2014 - 08.2014
- Developed Java programs and applications to meet customer requirements.
- Debugged existing code modules to identify and correct errors or performance problems.
- Worked closely with project teams to coordinate tasks, timelines and deliverables.
- Built reusable components that can be used across multiple projects.
- Compiled code and corrected errors detected in compilation process.
Embedded System Developer
APPIN TECHONOLOGIES
Jalandhar , Punjab India
05.2012 - 07.2012
- Designed, implemented, tested and debugged software components in a real-time environment.
- Maintained legacy products by updating drivers or fixing bugs found during maintenance cycles.
- Deployed firmware onto target devices ensuring compatibility across different architectures.
- Worked with hardware engineers to ensure that the design met customer requirements.
Education
Master of Engineering - Electrical And Computer Engineering
Concordia University
Montreal, QC06-2016
Bachelor of Engineering - Electronics And Computer Engineering Technology
Rayat Bahra Institute of Nano Techonology
Hoshiarpur Punjab India06-2014
Skills
- NoSQL Databases
- Relational databases
- Data Modeling
- Data Security
- Data Warehousing
- Performance Tuning
- Continuous integration
- Python Programming
- API Development
- Big data technologies
- Data Migration
- SQL and Databases
- Data Visualization
- Problem-Solving
- Active Listening
- Team Collaboration
- Problem-solving abilities
- 5 year Experience working with aws and azure cloud
- 3 year of Terraform
- 5 year experience with snowflake and sql
- Working experience Ansible and packer
- 5 year experience on Databricks
- 2 year confluent kafka
- 7 year Python
- 5 Experience with CI/CD jenkins
- Experience with Docker, sonarqube, nexus, Jfrog artifactory, Datadog
- 4 year experience on Hadoop HDFS, Hive, Sqoop, PIG, Spark,
- Flume, Oozie, MapReduce and Hbase
- Knowledge of Flume, Zookeeper and other components from
- Hadoop ecosystem
- Experience in analyzing data using HiveQL, PIG Latin
- Hands on experience in Apache Hive and Apache Pig
- Designed Embedded system (Mips-32 bit processor with 5 stage pipeline) and Formal Hardware verification using Formality and cadence
Languages
English
Full Professional
French
Limited
Punjabi
Native/ Bilingual
Hindi
Native/ Bilingual
Certification
- AWS CLOUD PRACTITIONER
- TERRAFORM ASSOCIATE
Timeline
Senior Data Engineer
National Bank of canada
06.2020 - Current
Data Engineer
CGI
09.2017 - 05.2020
Hardware Verification Engineer
Concordia university
05.2016 - 08.2016
Hadoop Developer
Acadgild
02.2016 - 07.2017
Junior Java Developer
TCIL-IT
01.2014 - 08.2014
Embedded System Developer
APPIN TECHONOLOGIES
05.2012 - 07.2012
Master of Engineering - Electrical And Computer Engineering
Concordia University
Bachelor of Engineering - Electronics And Computer Engineering Technology
Rayat Bahra Institute of Nano Techonology
Similar Profiles
Radwan RahmanRadwan Rahman
Software Engineering Intern at National Bank of CanadaSoftware Engineering Intern at National Bank of Canada
Nicholas QINNicholas QIN
Senior Manager, Finance at National Bank of CanadaSenior Manager, Finance at National Bank of Canada
Claude Carelle NgassamClaude Carelle Ngassam
Collection Customer Service Agent at National Bank of CanadaCollection Customer Service Agent at National Bank of Canada