> For the complete documentation index, see [llms.txt](https://docs.projectbit.ca/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.projectbit.ca/documentation/tools/spider.md).

# Spider

**SpiderTools** is an open source web Scraper & Crawler that returns LLM-ready data. To start using Spider, you need an API key from the [Spider dashboard](https://spider.cloud/).

### [​](https://docs.phidata.com/tools/spider#prerequisites)Prerequisites <a href="#prerequisites" id="prerequisites"></a>

The following example requires the `spider-client` library.

```shell
pip install -U spider-client
```

### [​](https://docs.phidata.com/tools/spider#example)Example <a href="#example" id="example"></a>

The following agent will run a search query to get the latest news in USA and scrape the first search result. The agent will return the scraped data in markdown format.

cookbook/tools/spider\_tools.py

```python
from bitca.agent import Agent
from bitca.tools.spider import SpiderTools

agent = Agent(tools=[SpiderTools()])
agent.print_response('Can you scrape the first search result from a search on "news in USA"?', markdown=True)
```

### [​](https://docs.phidata.com/tools/spider#toolkit-params)Toolkit Params <a href="#toolkit-params" id="toolkit-params"></a>

| Parameter     | Type  | Default | Description                                    |
| ------------- | ----- | ------- | ---------------------------------------------- |
| `max_results` | `int` | -       | The maximum number of search results to return |
| `url`         | `str` | -       | The url to be scraped or crawled               |

### [​](https://docs.phidata.com/tools/spider#toolkit-functions)Toolkit Functions <a href="#toolkit-functions" id="toolkit-functions"></a>

| Function | Description                           |
| -------- | ------------------------------------- |
| `search` | Searches the web for the given query. |
| `scrape` | Scrapes the given url.                |
| `crawl`  | Crawls the given url.                 |