Repository navigation

#

web-scraper

A collection of awesome web crawler,spider in different languages

6915
1 年前
D4Vinci/Scrapling

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

Python
6453
16 分钟前

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

PHP
3295
6 小时前
gosom/google-maps-scraper

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

Go
2255
25 天前

Web Scraper in Go, similar to BeautifulSoup

Go
2213
2 年前
dipu-bd/lightnovel-crawler

Generate and download e-books from online sources.

Python
1818
6 小时前

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Python
1750
9 天前

🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

JavaScript
1095
2 年前

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

JavaScript
936
3 个月前
Kotlin
916
1 个月前

Bulk download your favourite anime episodes from your favourite anime websites

Python
848
2 年前

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Ruby
828
2 个月前

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

Python
813
4 年前
Python
797
2 年前

A list of scrapers from around the web.

683
6 个月前

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

Python
589
2 个月前