How to create a Webparser integration

A webparser tool lets the AI fetch and read a web page on demand using a plain HTTP request — the same way curl would. It’s the right choice when the page you need is server-rendered and the data is already in the HTML that comes back. Fast (200–800 ms typical), cheap, and runs in the regular AI worker.

Webparser vs Webscraper — which one?

• Webparser (this article) — plain HTTP fetch. Use when the data is visible in View Source. Examples: documentation pages, status pages, simple pricing tables, blog posts.
• Webscraper — full headless Chromium with JavaScript execution. Use when the page is a JavaScript app and the data only appears after scripts run. See How to create a Webscraper integration.

Try the parser first. If View Source on the target page shows the data you need, the parser is fine and ten times faster than the scraper.

Step 1 — Add a scrape-type tool

Webparser and webscraper share one tool type called scrape; the difference is the render_js flag. Open Admin → Projects → [project] → Integrations, pick (or create) any integration, then Add tool and set:

• Type — scrape.
• Name — fetch_status_page or similar.
• Description — one sentence describing when to call it. The AI reads this to decide.
• Render JS — leave off.
• Allowed domains — comma-separated list of hostnames the tool may fetch from, e.g. status.example.com, docs.example.com. This is enforced — anything outside the list is rejected before a request goes out.

Step 2 — Narrow down with a CSS selector (optional)

By default the tool returns the cleaned text of the whole page (capped at 16 KB). For long pages you can pass a CSS selector so only the matching element’s text is returned:

.status-summary, main article, #pricing-table

Anything CSS can match works. If the selector matches nothing the tool falls back to the full page so the AI never sees an empty result.

Step 3 — Custom headers (optional)

If the target needs a specific User-Agent or accepts a token in a header, set them under Custom headers:

{
  "User-Agent": "ChatonioBot/1.0",
  "Accept": "text/html,application/xhtml+xml"
}

Step 4 — Test it

Use the Test scrape button on the tool row. Paste a real URL within your allowed-domains list:

{
  "url": "https://status.example.com/incidents/latest"
}

{
  "success": true,
  "detail": "Fetched in 412 ms",
  "preview": "All systems operational..."
}

{
  "success": false,
  "detail": "Domain not in allowed list"
}

How it shows up in conversations

The AI calls the parser on its own when a customer’s question lines up with the tool’s description — you don’t prompt for it. The fetched text is piped straight into the model’s context, so the response is always grounded in what the page actually said at fetch time, not stale training data.

Limits and tips

• Output is capped at 16 KB. Very long pages will be truncated — a CSS selector helps a lot.
• Hard timeout is 30 seconds; the worker will abort and return an error to the AI rather than hang.
• HTTPS only against public hosts — the SSRF guard blocks loopback / private network addresses outright.
• If the page is gated by login, this tool won’t see it. Use a custom-header token if your service supports header auth, otherwise model the data behind a real API integration.