Manpreet Singh
Halifax,NS
Summary
Experienced Python developer with a focus on web scraping, proficient in Azure, AWS, and DevOps tools including Docker and Kubernetes. Skilled in Django, React, and SQL with 3 years of experience in software development and deployment.
Overview
3
3
years of professional experience
Work History
Sr. Back-End Engineer, Web Scraping
Dentistry Automation
10.2022 - Current
- Developed a web automation framework for automating scraping ETF data, as well as claims and eligibility insurance information.
- Successfully scraped data from more than 250 websites.
- Automated the entire workflow for daily crawling and generating PDF reports of the crawled data.
- Created a custom solution for solving reCAPTCHA, eliminating the need for a third-party API.
- Containerized the application to scale scraping operations and make it serverless.
- Integrated the framework with Azure Storage Accounts for data storage.
- Utilized Azure Container Apps (Kubernetes) and Azure CI/CD for continuous deployment.
Python Developer
Nostra.ai
09.2022 - Current
- Developed and maintained web scraping scripts and tools to extract data from various websites.
- Implemented advanced scraping techniques to efficiently gather data from dynamic and complex websites.
- Optimized scraping scripts for performance and scalability, enabling the extraction of large datasets.
- Managed proxies and user agents to avoid detection and ensure uninterrupted scraping operations.
- Processed and cleaned scraped data to ensure accuracy and usability for analysis and reporting.
- Stayed updated with the latest web scraping technologies and best practices to enhance scraping efficiency.
- Collaborated with cross-functional teams to integrate scraped data into analytical systems and applications.
- Provided expertise and guidance on web scraping projects, including strategy, tool selection, and troubleshooting.
Python Developer (Web Scraping and Automation)
Freelancing
01.2021 - 09.2022
- Working as a Python developer mainly web scraping and building automation bots for websites.
- Developing bots that can't be detected by antibot detection services like Cloudflare's I am under attack mode, data dome, and other antibot detection services
- Creating bots which can spoof browser's fingerprints like canvas fingerprinting, Webrtc, Rotating user-agents, using proxies, and randomizing every parameter to generate a unique fingerprint.
- Bots that can solve captcha like image captcha, Hcapthca, Recaptcha even the Geecaptcha, and fun captcha using third-party API's. Deploying bots on cloud services like AWS, GCP,Azure and storing crawled data into the database.
- I have provided web Scraping solutions to organizations like famepilot.com, https://www.reviewshake.com, https://www.reviewshake.com/, https://disruptordinary.com/
Education
Bachelor of Science - Computer Applications
Arunodaya University
India, India05.2020
Skills
- Product Development
- Server Administration
- Object-Oriented Programming
- Selenium
- Scrapy
- Requests
- BeautifulSoup
- Azure,Aws
- Docker
- Kubernetes
- React
- Django
- CI/CD
- Aws lambda, Azure Function apps, RDS
Timeline
Sr. Back-End Engineer, Web Scraping
Dentistry Automation
10.2022 - Current
Python Developer
Nostra.ai
09.2022 - Current
Python Developer (Web Scraping and Automation)
Freelancing
01.2021 - 09.2022
Bachelor of Science - Computer Applications
Arunodaya University
Similar Profiles
Bhavana KarnatiBhavana Karnati
Project Manager at Dentistry AutomationProject Manager at Dentistry Automation
Jo Anna Marie RebultanJo Anna Marie Rebultan
Aftermarket Sales Support Specialist at Veetech Automation FZE (formerly named Versatech AUtomation FZE)Aftermarket Sales Support Specialist at Veetech Automation FZE (formerly named Versatech AUtomation FZE)
Lauren MooreLauren Moore
Dental Assistant Intern at Summit Implant Dentistry | Fry Orthodontics | New Mark Modern DentistryDental Assistant Intern at Summit Implant Dentistry | Fry Orthodontics | New Mark Modern Dentistry
Gurtej Singh DhaliwalGurtej Singh Dhaliwal
House Painter at Silverstone Painting Inc.House Painter at Silverstone Painting Inc.
Sasha BoucherSasha Boucher
Startup Founder at Eastwork Brothers Realty ServicesStartup Founder at Eastwork Brothers Realty Services