Overview
AgentQL provides the following three tools:ExtractWebDataTool
extracts structured data as JSON from a web page given a URL using either an AgentQL query or a Natural Language description of the data.
AgentQLBrowserToolkit
and must be used with a Playwright
browser or a remote browser instance via Chrome DevTools Protocal (CDP):
-
ExtractWebDataBrowserTool
extracts structured data as JSON from the active web page in a browser using either an AgentQL query or a Natural Language description. -
GetWebElementBrowserTool
finds a web element on the active web page in a browser using a Natural Language description and returns its CSS selector for further interaction.
Integration details
Class | Package | Serializable | JS support | Version |
---|---|---|---|---|
AgentQL | langchain-agentql | ❌ | ❌ | 1.0.0 |
Tool features
Tool | Web Data Extraction | Web Element Extraction | Use With Local Browser |
---|---|---|---|
ExtractWebDataTool | ✅ | ❌ | ❌ |
ExtractWebDataBrowserTool | ✅ | ❌ | ✅ |
GetWebElementBrowserTool | ❌ | ✅ | ✅ |
Setup
Playwright
browser and configure Jupyter Notebook’s asyncio
loop.
Credentials
To use the AgentQL tools, you will need to get your own API key from the AgentQL Dev Portal and set the AgentQL environment variable.Instantiation
ExtractWebDataTool
You can instantiate ExtractWebDataTool
with the following params:
api_key
: Your AgentQL API key from dev.agentql.com.Optional
.timeout
: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to900
.is_stealth_mode_enabled
: Whether to enable experimental anti-bot evasion strategies. This feature may not work for all websites at all times. Data extraction may take longer to complete with this mode enabled. Defaults toFalse
.wait_for
: The number of seconds to wait for the page to load before extracting data. Defaults to0
.is_scroll_to_bottom_enabled
: Whether to scroll to bottom of the page before extracting data. Defaults toFalse
.mode
:"standard"
uses deep data analysis, while"fast"
trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to"fast"
.is_screenshot_enabled
: Whether to take a screenshot before extracting data. Returned in ‘metadata’ as a Base64 string. Defaults toFalse
.
ExtractWebDataTool
is implemented with AgentQL’s REST API, you can view more details about the parameters in the API Reference docs.
ExtractWebDataBrowserTool
To instantiate ExtractWebDataBrowserTool, you need to connect the tool with a browser instance.
You can set the following params:
timeout
: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to900
.wait_for_network_idle
: Whether to wait until the network reaches a full idle state before executing. Defaults toTrue
.include_hidden
: Whether to take into account visually hidden elements on the page. Defaults toTrue
.mode
:"standard"
uses deep data analysis, while"fast"
trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to"fast"
.
ExtractWebDataBrowserTool
is implemented with AgentQL’s SDK. You can find more details about the parameters and the functions in AgentQL’s API References.
GetWebElementBrowserTool
To instantiate GetWebElementBrowserTool, you need to connect the tool with a browser instance.
You can set the following params:
timeout
: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to900
.wait_for_network_idle
: Whether to wait until the network reaches a full idle state before executing. Defaults toTrue
.include_hidden
: Whether to take into account visually hidden elements on the page. Defaults toFalse
.mode
:"standard"
uses deep data analysis, while"fast"
trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to"fast"
.
GetWebElementBrowserTool
is implemented with AgentQL’s SDK. You can find more details about the parameters and the functions in AgentQL’s API References.`
Invocation
ExtractWebDataTool
This tool uses AgentQL’s REST API under the hood, sending the publically available web page’s URL to AgentQL’s endpoint. This will not work with private pages or logged in sessions. Use ExtractWebDataBrowserTool
for those usecases.
url
: The URL of the web page you want to extract data from.query
: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about how to write an AgentQL query in the docs or test one out in the AgentQL Playground.prompt
: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Useprompt
if you want to extract data defined by free-form language without defining a particular structure.
query
or a prompt
to use AgentQL.
ExtractWebDataBrowserTool
query
: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about how to write an AgentQL query in the docs or test one out in the AgentQL Playground.prompt
: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Useprompt
if you want to extract data defined by free-form language without defining a particular structure.
query
or a prompt
to use AgentQL.
To extract data, first you must navigate to a web page using LangChain’s Playwright tool.
GetWebElementBrowserTool
prompt
: A Natural Language description of the web element to find on the page.
Chaining
You can use AgentQL tools in a chain by first binding one to a tool-calling model and then calling it:Instantiate LLM
Execute Tool Chain
Use within an agent
You can use AgentQL tools with an AI Agent using theAgentQLBrowserToolkit
. This toolkit includes ExtractDataBrowserTool
and GetWebElementBrowserTool
. Here’s an example of agentic browser actions that combine AgentQL’s toolkit with the Playwright tools.