Run Headless Crawling As A Dockerized Fastapi Service

Async Python web crawler that converts pages into clean, LLM-ready Markdown and structured JSON via CSS/XPath or LLM extraction. Includes deep-crawl, caching, sessions/proxies, and a Docker FastAPI server, aimed at RAG pipelines and agent data ingestion.

AI Tools | Python · web-scraping · web-crawler