WebCrawler for Security & E-commerce Analytics
Published:
Built WebCrawler to extract private data and find security issues by discovering hidden links in source code comments across 1M+ websites.
Integrated a rotating proxy system (Requests, BeautifulSoup, spaCy) to randomize IP/port per request and achieved ~95% accuracy.
Used the crawler to support real-time commodity price monitoring dashboards.