Repository navigation

#

web-scraper

A collection of awesome web crawler,spider in different languages

6719
10 个月前

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

PHP
3281
19 天前
D4Vinci/Scrapling

🕷️ An undetectable, powerful, flexible, high-performance Python library that makes Web Scraping simple and easy again!

Python
2914
2 天前

Web Scraper in Go, similar to BeautifulSoup

Go
2195
1 年前

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Python
1664
13 天前
dipu-bd/lightnovel-crawler

Generate and download e-books from online sources.

Python
1664
24 天前
gosom/google-maps-scraper

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

Go
1286
7 天前

🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

JavaScript
1077
2 年前

Bulk download your favourite anime episodes from your favourite anime websites

Python
838
2 年前

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Ruby
816
3 个月前
Python
783
1 年前

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

Python
779
3 年前

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

JavaScript
775
19 天前

A list of scrapers from around the web.

663
2 个月前

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

Python
579
10 天前

Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.

523
10 天前