feat: Add new tools and configurations for Castle platform
- Introduced `uv.lock` for dependency management with various packages including `pytest`, `colorama`, and `pluggy`. - Added `pyrightconfig.json` for Python type checking configuration. - Expanded `recommendations.md` with detailed scaling recommendations and project management strategies. - Created shared `ruff.toml` for consistent linting across projects. - Developed `Castle Tools` with various utilities including Android backup, browser automation, document conversion, and search tools. - Implemented `backup-collect` and `schedule` tools for system administration tasks. - Enhanced `search` functionality with indexing and querying capabilities using Tantivy. - Added comprehensive documentation for each tool, including usage examples and installation instructions.
This commit is contained in:
21
tools/browser/pyproject.toml
Normal file
21
tools/browser/pyproject.toml
Normal file
@@ -0,0 +1,21 @@
|
||||
[project]
|
||||
name = "castle-browser"
|
||||
version = "0.1.0"
|
||||
description = "Castle browser automation tools"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"python-dotenv>=1.0.0",
|
||||
"browser-use>=0.1.0",
|
||||
"langchain-openai>=0.1.0",
|
||||
"langchain-anthropic>=0.1.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
browser = "browser.browser:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/browser"]
|
||||
0
tools/browser/src/browser/__init__.py
Normal file
0
tools/browser/src/browser/__init__.py
Normal file
166
tools/browser/src/browser/browser.md
Normal file
166
tools/browser/src/browser/browser.md
Normal file
@@ -0,0 +1,166 @@
|
||||
---
|
||||
command: browser
|
||||
script: browser/browser.py
|
||||
description: Browse the web using natural language via browser-use
|
||||
version: 1.0.0
|
||||
category: browser
|
||||
---
|
||||
|
||||
# browser
|
||||
|
||||
Browser automation tool powered by browser-use. Execute browser tasks from the command line using natural language instructions.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **LLM Provider** - Requires API key for OpenAI or Anthropic
|
||||
- **Chromium/Chrome** - Used by the automation framework
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Install the toolkit
|
||||
cd /data/repos/toolkit
|
||||
make install
|
||||
|
||||
# Verify installation
|
||||
which browser
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
### API Key Configuration
|
||||
|
||||
Set up your LLM provider API key:
|
||||
|
||||
```bash
|
||||
# For OpenAI
|
||||
export OPENAI_API_KEY=sk-...
|
||||
|
||||
# For Anthropic Claude
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# Add to ~/.bashrc or ~/.profile to persist
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Execute a browser task
|
||||
browser "Go to google.com and search for python tutorials"
|
||||
|
||||
# Use specific LLM provider
|
||||
browser --model gpt-4 "Fill out the contact form on example.com"
|
||||
browser --model claude-3-5-sonnet "Take a screenshot of the homepage"
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```bash
|
||||
browser [options] "instruction"
|
||||
|
||||
Options:
|
||||
instruction Natural language instruction for browser task (required)
|
||||
--model MODEL LLM model to use (default: gpt-4o-mini)
|
||||
--headless Run browser in headless mode (no visible window)
|
||||
--timeout SECONDS Maximum time for task execution (default: 300)
|
||||
-h, --help Show help message
|
||||
```
|
||||
|
||||
## Supported Models
|
||||
|
||||
### OpenAI Models
|
||||
- `gpt-4o` - Most capable, best for complex tasks
|
||||
- `gpt-4o-mini` - Fast and affordable (default)
|
||||
- `gpt-4-turbo`
|
||||
- `gpt-4`
|
||||
- `gpt-3.5-turbo`
|
||||
|
||||
### Anthropic Models
|
||||
- `claude-3-5-sonnet-20241022` - Most capable
|
||||
- `claude-3-opus-20240229`
|
||||
- `claude-3-sonnet-20240229`
|
||||
- `claude-3-haiku-20240307`
|
||||
|
||||
## Examples
|
||||
|
||||
### Web Search
|
||||
|
||||
```bash
|
||||
browser "Search Google for 'best python libraries 2024' and summarize the top 3 results"
|
||||
```
|
||||
|
||||
### Form Filling
|
||||
|
||||
```bash
|
||||
browser "Go to example.com/contact and fill in the form with: Name: John Doe, Email: john@example.com, Message: Testing automation"
|
||||
```
|
||||
|
||||
### Screenshots
|
||||
|
||||
```bash
|
||||
browser "Navigate to github.com and take a screenshot"
|
||||
```
|
||||
|
||||
### Data Extraction
|
||||
|
||||
```bash
|
||||
browser "Go to news.ycombinator.com and list the top 5 post titles"
|
||||
```
|
||||
|
||||
### Multi-step Tasks
|
||||
|
||||
```bash
|
||||
browser "Go to amazon.com, search for 'wireless mouse', filter by 4+ stars, and show me the top 3 results with prices"
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- **Natural language control** - Use plain English to automate browser tasks
|
||||
- **AI-powered** - Uses LLMs to understand and execute complex instructions
|
||||
- **Multi-step workflows** - Handle complex sequences of actions
|
||||
- **Screenshot capture** - Can take screenshots as part of tasks
|
||||
- **Form filling** - Automatically fill out forms
|
||||
- **Data extraction** - Extract information from web pages
|
||||
- **Headless mode** - Run without visible browser window
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `OPENAI_API_KEY` - API key for OpenAI models
|
||||
- `ANTHROPIC_API_KEY` - API key for Anthropic Claude models
|
||||
|
||||
## Notes
|
||||
|
||||
- Tasks are executed using browser automation via browser-use
|
||||
- Complex tasks may take longer to complete
|
||||
- Headless mode is faster but you won't see the browser actions
|
||||
- Some websites may block automation (check robots.txt)
|
||||
- API costs apply based on model usage
|
||||
- Default timeout is 5 minutes; adjust with `--timeout` for longer tasks
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### API Key Not Found
|
||||
|
||||
```bash
|
||||
# Ensure API key is exported
|
||||
echo $OPENAI_API_KEY
|
||||
# or
|
||||
echo $ANTHROPIC_API_KEY
|
||||
|
||||
# Add to shell config if missing
|
||||
export OPENAI_API_KEY=sk-...
|
||||
```
|
||||
|
||||
### Browser Launch Fails
|
||||
|
||||
Ensure Chromium or Chrome is installed on your system. The browser-use library will try to find it automatically.
|
||||
|
||||
### Timeout Errors
|
||||
|
||||
For complex tasks, increase the timeout:
|
||||
|
||||
```bash
|
||||
browser --timeout 600 "complex long-running task"
|
||||
```
|
||||
199
tools/browser/src/browser/browser.py
Executable file
199
tools/browser/src/browser/browser.py
Executable file
@@ -0,0 +1,199 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Browser automation tool powered by browser-use.
|
||||
|
||||
This tool allows executing browser automation from command line instructions.
|
||||
It takes instruction text and uses browser-use to automate browser tasks.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import asyncio
|
||||
import os
|
||||
from typing import Dict, Any
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load environment variables for API keys
|
||||
load_dotenv()
|
||||
|
||||
# Import browser-use and related dependencies
|
||||
try:
|
||||
from browser_use import Agent, Browser, BrowserConfig, BrowserContextConfig
|
||||
|
||||
# Optional imports for different LLM providers
|
||||
openai_available = False
|
||||
anthropic_available = False
|
||||
|
||||
try:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
openai_available = True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
|
||||
anthropic_available = True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
except ImportError:
|
||||
sys.stderr.write("Error: Required dependencies not found. Install with:\n")
|
||||
sys.stderr.write("pip install browser-use python-dotenv\n")
|
||||
sys.stderr.write("pip install langchain-openai # For OpenAI models\n")
|
||||
sys.stderr.write("pip install langchain-anthropic # For Anthropic models\n")
|
||||
sys.stderr.write("patchright install chromium\n")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
async def run_browser_task(
|
||||
task: str,
|
||||
provider: str = "openai",
|
||||
model: str = "gpt-4o",
|
||||
max_steps: int = 25,
|
||||
max_actions_per_step: int = 4,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Run a browser task based on the given instruction.
|
||||
|
||||
Args:
|
||||
task: The instruction text describing what to do in the browser
|
||||
provider: The model provider to use
|
||||
model: The specific model name to use
|
||||
max_steps: Maximum number of steps the agent can take
|
||||
max_actions_per_step: Maximum number of actions per step
|
||||
|
||||
Returns:
|
||||
The result from the browser-use agent
|
||||
"""
|
||||
# Configure the browser (headless mode is required for environments without X11 display)
|
||||
browser = Browser(
|
||||
config=BrowserConfig(
|
||||
headless=True,
|
||||
new_context_config=BrowserContextConfig(
|
||||
viewport_expansion=0,
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
# Select and configure the appropriate LLM based on provider
|
||||
if provider.lower() == "anthropic":
|
||||
if not anthropic_available:
|
||||
raise ImportError(
|
||||
"Anthropic integration not available. Install with: pip install langchain-anthropic"
|
||||
)
|
||||
if not os.environ.get("ANTHROPIC_API_KEY"):
|
||||
raise ValueError("ANTHROPIC_API_KEY not found in environment variables")
|
||||
llm = ChatAnthropic(model=model) # type: ignore
|
||||
elif provider.lower() == "openai":
|
||||
if not openai_available:
|
||||
raise ImportError(
|
||||
"OpenAI integration not available. Install with: pip install langchain-openai"
|
||||
)
|
||||
if not os.environ.get("OPENAI_API_KEY"):
|
||||
raise ValueError("OPENAI_API_KEY not found in environment variables")
|
||||
llm = ChatOpenAI(model=model) # type: ignore
|
||||
else:
|
||||
raise ValueError(f"Unsupported provider: {provider}")
|
||||
|
||||
# Create and run the agent
|
||||
agent = Agent(
|
||||
task=task,
|
||||
llm=llm,
|
||||
browser=browser,
|
||||
max_actions_per_step=max_actions_per_step,
|
||||
)
|
||||
|
||||
# Execute the task with a maximum number of steps
|
||||
result = await agent.run(max_steps=max_steps)
|
||||
return result # type: ignore
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Execute browser automation from command line instructions"
|
||||
)
|
||||
parser.add_argument(
|
||||
"instruction", nargs="*", help="Instruction text for browser automation"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--provider",
|
||||
"-p",
|
||||
choices=["openai", "anthropic"],
|
||||
default="openai",
|
||||
help="The model provider to use (default: openai)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model", "-m", help="The model to use (default depends on provider)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--file", "-f", help="Read instruction from file instead of command line"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-steps",
|
||||
type=int,
|
||||
default=25,
|
||||
help="Maximum number of steps the agent can take (default: 25)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-actions",
|
||||
type=int,
|
||||
default=4,
|
||||
help="Maximum number of actions per step (default: 4)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Set default model based on provider if not specified
|
||||
if not args.model:
|
||||
if args.provider == "anthropic":
|
||||
args.model = "claude-3-opus-20240229"
|
||||
else: # openai
|
||||
args.model = "gpt-4o"
|
||||
|
||||
# Get instruction from file, command line args, or stdin
|
||||
if args.file:
|
||||
try:
|
||||
with open(args.file, "r") as f:
|
||||
instruction = f.read().strip()
|
||||
except Exception as e:
|
||||
sys.stderr.write(f"Error reading file: {e}\n")
|
||||
sys.exit(1)
|
||||
elif args.instruction:
|
||||
instruction = " ".join(args.instruction)
|
||||
else:
|
||||
# Check if there's input from stdin
|
||||
if not sys.stdin.isatty():
|
||||
instruction = sys.stdin.read().strip()
|
||||
else:
|
||||
parser.print_help()
|
||||
sys.exit(1)
|
||||
|
||||
if not instruction:
|
||||
sys.stderr.write("Error: No instruction provided\n")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"Running browser task: {instruction}")
|
||||
print(f"Using model: {args.provider}/{args.model}")
|
||||
|
||||
try:
|
||||
result = asyncio.run(
|
||||
run_browser_task(
|
||||
task=instruction,
|
||||
provider=args.provider,
|
||||
model=args.model,
|
||||
max_steps=args.max_steps,
|
||||
max_actions_per_step=args.max_actions,
|
||||
)
|
||||
)
|
||||
print("\nTask completed!")
|
||||
if "output" in result:
|
||||
print(result["output"])
|
||||
except Exception as e:
|
||||
sys.stderr.write(f"Error executing browser task: {e}\n")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user