---
title: Crawl
---

[Skip to content](#_top)

[API Reference](https://developers.cloudflare.com/api/python)

[Browser Rendering](https://developers.cloudflare.com/api/python/resources/browser_rendering)

Copy Markdown

Open in **Claude**Open in **ChatGPT**Open in **Cursor**

---

**Copy Markdown****View as Markdown**

# Crawl

##### [Crawl websites.](https://developers.cloudflare.com/api/python/resources/browser_rendering/subresources/crawl/methods/create)

browser\_rendering.crawl.create(CrawlCreateParams\*\*kwargs) -> [CrawlCreateResponse](<https://developers.cloudflare.com/api/python/resources/browser_rendering#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_create_response%20%3E%20(schema)>)

POST/accounts/{account\_id}/browser-rendering/crawl

##### [Get crawl result.](https://developers.cloudflare.com/api/python/resources/browser_rendering/subresources/crawl/methods/get)

browser\_rendering.crawl.get(strjob\_id, CrawlGetParams\*\*kwargs) -> [CrawlGetResponse](<https://developers.cloudflare.com/api/python/resources/browser_rendering#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)>)

GET/accounts/{account\_id}/browser-rendering/crawl/{job\_id}

##### [Cancel a crawl job.](https://developers.cloudflare.com/api/python/resources/browser_rendering/subresources/crawl/methods/delete)

browser\_rendering.crawl.delete(strjob\_id, CrawlDeleteParams\*\*kwargs) -> [CrawlDeleteResponse](<https://developers.cloudflare.com/api/python/resources/browser_rendering#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_delete_response%20%3E%20(schema)>)

DELETE/accounts/{account\_id}/browser-rendering/crawl/{job\_id}

##### ModelsExpand Collapse

str

Crawl job ID.

[Link to this property](<#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_create_response%20%3E%20(schema)>)

<details>

<summary>

class CrawlGetResponse: …

</summary>

id: str

Crawl job ID.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20id">Link to this property</a>

browser\_seconds\_used: float

Total seconds spent in browser so far.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20browserSecondsUsed">Link to this property</a>

finished: float

Total number of URLs that have been crawled so far.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20finished">Link to this property</a>

<details>

<summary>

records: List\[Record]

List of crawl job records.

</summary>

<details>

<summary>

status: Literal\["queued", "errored", "completed", 3 more]

Current status of the crawled URL.

</summary>

One of the following:

"queued"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%200">Link to this property</a>

"errored"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%201">Link to this property</a>

"completed"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%202">Link to this property</a>

"disallowed"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%203">Link to this property</a>

"skipped"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%204">Link to this property</a>

"cancelled"

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status%20%3E%20(member)%205">Link to this property</a>

</details>

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20status">Link to this property</a>

url: str

Crawled URL.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20url">Link to this property</a>

html: Optional\[str]

HTML content of the crawled URL.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20html">Link to this property</a>

json: Optional\[Dict\[str, Optional\[object]]]

JSON of the content of the crawled URL.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20json">Link to this property</a>

markdown: Optional\[str]

Markdown of the content of the crawled URL.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20markdown">Link to this property</a>

<details>

<summary>

metadata: Optional\[RecordMetadata]

Absent for urls that never reached a fetch.

</summary>

status: float

HTTP status code of the crawled page.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20metadata%20%3E%20(property)%20status">Link to this property</a>

url: str

Final URL of the crawled page.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20metadata%20%3E%20(property)%20url">Link to this property</a>

title: Optional\[str]

Title of the crawled page.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20metadata%20%3E%20(property)%20title">Link to this property</a>

</details>

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records%20%3E%20(items)%20%3E%20(property)%20metadata">Link to this property</a>

</details>

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20records">Link to this property</a>

skipped: float

Total number of URLs that were skipped due to include/exclude/subdomain filters. Skipped URLs are included in records but are not counted toward total/finished.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20skipped">Link to this property</a>

status: str

Current crawl job status.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20status">Link to this property</a>

total: float

Total current number of URLs in the crawl job.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20total">Link to this property</a>

cursor: Optional\[str]

Cursor for pagination.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)%20%3E%20(property)%20cursor">Link to this property</a>

</details>

[Link to this property](<#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_get_response%20%3E%20(schema)>)

<details>

<summary>

class CrawlDeleteResponse: …

</summary>

job\_id: str

The ID of the cancelled job.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_delete_response%20%3E%20(schema)%20%3E%20(property)%20job_id">Link to this property</a>

message: str

Cancellation confirmation message.

<a href="#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_delete_response%20%3E%20(schema)%20%3E%20(property)%20message">Link to this property</a>

</details>

[Link to this property](<#(resource)%20browser_rendering.crawl%20%3E%20(model)%20crawl_delete_response%20%3E%20(schema)>)