---
title: Robots TXT
---

[Skip to content](#_top)

[API Reference](https://developers.cloudflare.com/api/go)

[Radar](https://developers.cloudflare.com/api/go/resources/radar)

Copy Markdown

Open in **Claude**Open in **ChatGPT**Open in **Cursor**

---

**Copy Markdown****View as Markdown**

# Robots TXT

#### Robots TXTTop

##### [Get top domain categories by robots.txt files parsed](https://developers.cloudflare.com/api/go/resources/radar/subresources/robots_txt/subresources/top/methods/domain_categories)

client.Radar.RobotsTXT.Top.DomainCategories(ctx, query) (\*[RobotsTXTTopDomainCategoriesResponse](<https://developers.cloudflare.com/api/go/resources/radar#(resource)%20radar.robots_txt.top%20%3E%20(model)%20RobotsTXTTopDomainCategoriesResponse%20%3E%20(schema)>), error)

GET/radar/robots\_txt/top/domain\_categories

#### Robots TXTTopUser Agents

##### [Get top user agents on robots.txt files](https://developers.cloudflare.com/api/go/resources/radar/subresources/robots_txt/subresources/top/subresources/user_agents/methods/directive)

client.Radar.RobotsTXT.Top.UserAgents.Directive(ctx, query) (\*[RobotsTXTTopUserAgentDirectiveResponse](<https://developers.cloudflare.com/api/go/resources/radar#(resource)%20radar.robots_txt.top.user_agents%20%3E%20(model)%20RobotsTXTTopUserAgentDirectiveResponse%20%3E%20(schema)>), error)

GET/radar/robots\_txt/top/user\_agents/directive