--- title: Bulk get robots.txt rules --- [Skip to content](#_top) [API Reference](https://developers.cloudflare.com/api/go) [AI Audit](https://developers.cloudflare.com/api/go/resources/ai_audit) [Robots](https://developers.cloudflare.com/api/go/resources/ai_audit/subresources/robots) Copy Markdown Open in **Claude**Open in **ChatGPT**Open in **Cursor** --- **Copy Markdown****View as Markdown** # Bulk get robots.txt rules client.AIAudit.Robots.BulkGet(ctx, params) (\*[RobotBulkGetResponse](), error) POST/zones/{zone\_id}/ai-audit/robots/bulk Fetches and parses robots.txt files for multiple domains within a zone in a single request. Each domain must belong to the specified zone. Results are keyed by hostname. ##### Security
API Token The preferred authorization scheme for interacting with the Cloudflare API. Create a token. **Example:**Authorization: Bearer Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY
API Email + API Key The previous authorization scheme for interacting with the Cloudflare API, used in conjunction with a Global API key. **Example:**X-Auth-Email: user@example.com The previous authorization scheme for interacting with the Cloudflare API. When possible, use API tokens instead of Global API keys. **Example:**X-Auth-Key: 144c9defac04969c7bfad8efaa8ea194
##### Accepted Permissions (at least one required) `Zone Settings Write``Zone Settings Read` ##### ParametersExpand Collapse
params RobotBulkGetParams ZoneID param.Field\[string] Path param: Identifier of the zone. Link to this property Body param.Field\[\[]string] Body param: Array of domain hostnames to fetch robots.txt for. Each domain must end with the zone name. Maximum 25 domains per request. Link to this property
[Link to this property](<#(resource)%20ai_audit.robots%20%3E%20(method)%20bulk_get%20%3E%20(params)%20default>) ##### ReturnsExpand Collapse
type RobotBulkGetResponse map\[string, RobotBulkGetResponseItem] Map of hostname to parsed robots.txt rules.
UserAgents map\[string, RobotBulkGetResponseItemUserAgent] Map of user-agent string to its parsed rules. Allow \[]string List of allowed path patterns. Link to this property Disallow \[]string List of disallowed path patterns. Link to this property
ContentSignals RobotBulkGetResponseItemUserAgentsContentSignalsOptional Content signal directives from robots.txt.
AIInput RobotBulkGetResponseItemUserAgentsContentSignalsAIInputOptional Whether AI input usage is permitted. One of the following: const RobotBulkGetResponseItemUserAgentsContentSignalsAIInputYes RobotBulkGetResponseItemUserAgentsContentSignalsAIInput = "yes" Link to this property const RobotBulkGetResponseItemUserAgentsContentSignalsAIInputNo RobotBulkGetResponseItemUserAgentsContentSignalsAIInput = "no" Link to this property
Link to this property
AITrain RobotBulkGetResponseItemUserAgentsContentSignalsAITrainOptional Whether AI training is permitted. One of the following: const RobotBulkGetResponseItemUserAgentsContentSignalsAITrainYes RobotBulkGetResponseItemUserAgentsContentSignalsAITrain = "yes" Link to this property const RobotBulkGetResponseItemUserAgentsContentSignalsAITrainNo RobotBulkGetResponseItemUserAgentsContentSignalsAITrain = "no" Link to this property
Link to this property
Search RobotBulkGetResponseItemUserAgentsContentSignalsSearchOptional Whether search indexing is permitted. One of the following: const RobotBulkGetResponseItemUserAgentsContentSignalsSearchYes RobotBulkGetResponseItemUserAgentsContentSignalsSearch = "yes" Link to this property const RobotBulkGetResponseItemUserAgentsContentSignalsSearchNo RobotBulkGetResponseItemUserAgentsContentSignalsSearch = "no" Link to this property
Link to this property
Link to this property CrawlDelay float64Optional Crawl delay in seconds. Link to this property
Link to this property Sitemaps \[]stringOptional List of sitemap URLs found in robots.txt. Link to this property Status int64Optional HTTP status code from fetching the robots.txt file. Link to this property
[Link to this property](<#(resource)%20ai_audit.robots%20%3E%20(model)%20RobotBulkGetResponse%20%3E%20(schema)>) ### Bulk get robots.txt rules Go HTTPTypeScriptPythonGoTerraform ``` package main import ( "context" "fmt" "github.com/cloudflare/cloudflare-go" "github.com/cloudflare/cloudflare-go/ai_audit" "github.com/cloudflare/cloudflare-go/option" ) func main() { client := cloudflare.NewClient( option.WithAPIToken("Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY"), ) response, err := client.AIAudit.Robots.BulkGet(context.TODO(), ai_audit.RobotBulkGetParams{ ZoneID: cloudflare.F("zone_id"), Body: []string{"example.com", "blog.example.com"}, }) if err != nil { panic(err.Error()) } fmt.Printf("%+v\n", response) } ``` 200 example ``` { "errors": [], "messages": [ { "code": 1000, "message": "message", "documentation_url": "documentation_url", "source": { "pointer": "pointer" } } ], "success": true, "result": { "foo": { "userAgents": { "foo": { "allow": [ "string" ], "disallow": [ "string" ], "contentSignals": { "ai-input": "yes", "ai-train": "yes", "search": "yes" }, "crawlDelay": 0 } }, "sitemaps": [ "string" ], "status": 0 } } } ``` ##### Returns Examples 200 example ``` { "errors": [], "messages": [ { "code": 1000, "message": "message", "documentation_url": "documentation_url", "source": { "pointer": "pointer" } } ], "success": true, "result": { "foo": { "userAgents": { "foo": { "allow": [ "string" ], "disallow": [ "string" ], "contentSignals": { "ai-input": "yes", "ai-train": "yes", "search": "yes" }, "crawlDelay": 0 } }, "sitemaps": [ "string" ], "status": 0 } } } ```