Key capabilities
- Inspect static metadata, language, headings, links, images, social cards, and JSON-LD
- Evaluate RFC 9309-style Allow and Disallow matching for a chosen crawler
- Read URL sets, indexes, alternates, dates, duplicates, and gzip uploads
- Use a public-only static fetch with explicit document and byte limits
