Varun Nanduri
Dallas,TX
Summary
Big Data professional with robust experience in designing and implementing scalable data architectures. Strong expertise in Hadoop, Spark, and cloud platforms and deep understanding of data governance and security. Known for fostering team collaboration and delivering high-quality results under evolving conditions. Versatile, reliable, and adept at leveraging cutting-edge technologies to drive impactful business outcomes.
Overview
7
7
years of professional experience
Work History
Data Engineer
pSolv
07.2021 - Current
- Understanding the Business requirements based on Functional specifications to design the ETL methodology.
- Designed, developed, and implemented Hadoop applications using Spark, Hive, and MapReduce
- .Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
- Monitored and optimised Apache Airflow clusters for performance, resource utilisation, and reliability, tuning parameters such as worker concurrency, task timeout, and DAG concurrency limits for efficient pipeline execution
- Designed, developed, and implemented Delta share ETL using spark.
- Wrote scripts to automate application deployments and configurations; monitored YARN applications
- Architected Python scripts and automation tools to streamline repetitive tasks and workflows, such as data scraping, file manipulation, and system administration tasks, effectively enhancing productivity and efficiency.
- Implemented Data ingestion and transformation pipelines to ingest patient records, conditions and integrated with web scraped hospital records to map patient to hospitals based on beds and service needs
- Data integration of EIA data, Utility Company records and programs to help consumers find energy saving programs offered in the geography
- Agent Time Data – Integrate and aggregate agent call data, presence data and Employee data for Payroll processing
- Down Stream Rebate – Integrate Rebate Order, Measures and attributes and Payment data for downstream analytics and report.
Data Engineer Intern
ContraForce
01.2021 - 05.2021
- Worked with various cloud technologies and machine learning algorithms like Random Forest to classify data and to enhance proprietary cybersecurity products
- Created visualization dashboards using Tableau for online reports to help clients and to have better understanding of their data and to help identify anomalies in it
- Created tables, dropping, and altering at run time without blocking updates and queries using HBase and Hive.
- Implemented proactive monitoring mechanisms to detect potential system bottlenecks, allowing preemptive action to prevent downtime or reduced performance.
- Supported efficient data migration processes by designing and implementing robust ETL procedures, ensuring minimal downtime during transitions.
Software Engineer Intern
Udak Technologies Pvt. Ltd
11.2020 - 12.2020
- Undertook broad research on the current market trends, distilling this research by assessing the feasibility of developing the best pose estimation algorithms
- Developed a neural network that identifies and rectifies a yoga posture of an individual using a pre trained model and transfer learning technique
- Implemented a real-time feedback system leveraging the pose estimation model, which significantly improved user engagement and correctness of postures during yoga practice sessions by providing instantaneous visual corrections.
Associate Engineer
Virtusa Consulting Services Pvt. Ltd
08.2017 - 07.2019
- Developed new organisational practices to an existing application in Pega, automating more than 15% of the workflow of the web application resulting in improved time efficiency and productivity
- Developed several REST web services supporting both XML and JSON leveraged by both web and mobile applications
- Collaborated with a team of six people in supporting the production and the testing teams for reducing the defect rates of a web application in the insurance industry by 30%
- Received “STAR Award” for solving high-priority bugs during the production phase of the application
- Developed Regression models like Ridge and Lasso for data that was is articulated for cyberattacks to create views for BI tools like Tableau
- Presented 10+ technical presentations to train the employees about the project work and provide them with a better understanding of organizational standards
Education
M.S., Information Technology and Management -
The University of Texas at Dallas
05.2021
B.Tech., Computer Science and Engineering -
Jawaharlal Nehru Technological University, Hyderabad – India
05.2017
Skills
- ETL development
- Data Warehousing
- Data Modeling
- Data Pipeline Design
- Scripting Languages
- Spark Framework
- Performance Tuning
- Hadoop Ecosystem
- Data Bricks
- o9 Eco System
Languages
English
Professional Working
Timeline
Data Engineer
pSolv
07.2021 - Current
Data Engineer Intern
ContraForce
01.2021 - 05.2021
Software Engineer Intern
Udak Technologies Pvt. Ltd
11.2020 - 12.2020
Associate Engineer
Virtusa Consulting Services Pvt. Ltd
08.2017 - 07.2019
B.Tech., Computer Science and Engineering -
Jawaharlal Nehru Technological University, Hyderabad – India
M.S., Information Technology and Management -
The University of Texas at Dallas
Similar Profiles
Andrew DavisAndrew Davis
Account Manager at Aln Apartment DataAccount Manager at Aln Apartment Data
SHELLEY ROBERTSONSHELLEY ROBERTSON
Assistant Manager at Penny Paws Veterinary Community OutreachAssistant Manager at Penny Paws Veterinary Community Outreach
Alexandre MorelAlexandre Morel
Director of Speech at Equality in ForensicsDirector of Speech at Equality in Forensics
Isaac MartinezIsaac Martinez
Medical Assistant at Expedian Urgent CareMedical Assistant at Expedian Urgent Care
Damodaran JDamodaran J
Data Engineer at Sunoida Solutions Pvt LtdData Engineer at Sunoida Solutions Pvt Ltd