> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getrilo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Web Tools

> HTTP requests, web scraping, and API calls for your workflows.

Rilo provides web tools for making HTTP requests, scraping web pages, calling REST APIs, and handling URLs. Perfect for integrating with external services and collecting data from the web.

## Overview

Web tools enable:

* **HTTP Requests**: GET, POST, PUT, DELETE requests to APIs and web services
* **Web Scraping**: Extract data from web pages using selectors
* **API Integration**: Connect to REST APIs with authentication
* **URL Operations**: Parse and construct URLs

<Info>
  For social media platforms, use the specialized Social Media Scraping tools instead.
</Info>

## Capabilities

### HTTP Requests

Make HTTP requests to any web service or API:

* **GET**: Fetch data from APIs or web pages
* **POST**: Send data to services
* **PUT**: Update resources
* **DELETE**: Remove resources

Supports:

* Custom headers (authentication, content types)
* Request parameters and body data
* Response handling and error management

### Web Scraping

Extract data from web pages:

* Use CSS selectors to target specific elements
* Extract text, attributes, and structured data
* Handle multiple elements and lists
* Wait for specific elements to load

### API Integration

Connect to REST APIs:

* Authenticated API calls
* Custom headers and authentication tokens
* Handle JSON request and response data
* Automatic error handling

### URL Operations

Work with URLs:

* Parse URLs to extract components (domain, path, query parameters)
* Build URLs from components
* Manipulate URL parameters

## Use Cases

<CardGroup cols={2}>
  <Card title="API Integration" icon="plug">
    Connect to external APIs and services to retrieve or send data.
  </Card>

  <Card title="Data Collection" icon="download">
    Scrape data from websites for analysis or processing.
  </Card>

  <Card title="Webhooks" icon="webhook">
    Send webhook notifications to external services.
  </Card>

  <Card title="Data Sync" icon="refresh">
    Sync data between different services and platforms.
  </Card>
</CardGroup>

## Configuration

When using web tools in your workflows, you can configure:

* **URLs**: API endpoints or web pages to access
* **Headers**: Authentication tokens, content types, custom headers
* **Selectors**: CSS selectors for web scraping (e.g., ".post-title", "#content")
* **Parameters**: Query parameters or request body data

All configuration can be easily tuned through the workflow editor without writing code.

## Important Notes

### Web Scraping

* Works best with static HTML content
* For JavaScript-heavy sites, some content may not be accessible
* Always respect website terms of service and rate limits

### API Calls

* Authentication credentials are managed securely by Rilo
* API rate limits from external services apply
* Response handling is automatic

<Warning>
  Always respect website terms of service and API rate limits when using web tools.
</Warning>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Request not working">
    * Verify the URL is correct and accessible
    * Check authentication credentials are properly configured
    * Ensure required headers are set
  </Accordion>

  <Accordion title="Scraping returns no data">
    * Verify CSS selectors match the page structure
    * Check if content loads dynamically (may not be scrapable)
    * Try different selectors or inspect the page HTML
  </Accordion>

  <Accordion title="API rate limit errors">
    * Reduce request frequency in your workflow
    * Contact the API provider for higher limits
    * Consider caching results when appropriate
  </Accordion>
</AccordionGroup>

## Related Features

* [Social Media Scraping](/features/social-media-scraping) - Specialized scraping for social platforms
* [Custom Integrations](/integrations/custom-integrations) - Create custom API integrations
* [Code Generation](/features/code-generation) - How web tools are used in generated code

***

<Note>
  WebOpsLibrary is a versatile tool for web operations. Use it for general web tasks, and specialized libraries for specific platforms.
</Note>