Cloudflare Launches Single API Call Website Crawler in Open Beta, Returning Content in Multiple Formats
Cloudflare launches a powerful new /crawl endpoint in open beta, enabling developers to crawl entire websites with a single API call and retrieve content in HTML, Markdown, or structured JSON formats, with advanced controls like depth limits, URL filtering, and incremental crawling available on both free and paid plans.