devtu-create-tool
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseToolUniverse Tool Creator
ToolUniverse 工具创建指南
Create new scientific tools for the ToolUniverse framework following established best practices.
遵循既定最佳实践,为ToolUniverse框架创建新型科学工具。
Table of Contents
目录
Critical Knowledge
核心必备知识
Top 6 Mistakes (90% of Failures)
六大常见错误(占失败案例的90%)
- Missing Entry - Tools silently won't load
default_config.py - Fake test_examples - Tests fail, agents get bad examples
- Single-level Testing - Misses registration bugs
- Skipping - Misses schema/API issues
test_new_tools.py - Tool Names > 55 chars - Breaks MCP compatibility
- Raising Exceptions - Should return error dicts instead
- 缺少配置项 - 工具会静默加载失败
default_config.py - 虚假测试示例 - 测试失败,Agent获取错误示例
- 单层级测试 - 遗漏注册类Bug
- 跳过测试 - 遗漏Schema/API问题
test_new_tools.py - 工具名称超过55字符 - 破坏MCP兼容性
- 直接抛出异常 - 应返回错误字典而非抛出异常
Tool Creator vs SDK User
SDK用户 vs 工具创建者
| SDK User (Using) | Tool Creator (Building) |
|---|---|
| |
| Handle responses | Design schemas |
| One-level usage | Three-step registration |
| SDK 用户(使用工具) | 工具创建者(构建工具) |
|---|---|
| |
| 处理响应 | 设计Schema |
| 单层使用 | 三步注册流程 |
Core Concepts
核心概念
Two-Stage Architecture
两阶段架构
Stage 1: Tool Class Stage 2: Wrappers (Auto-Generated)
Python Implementation From JSON Configs
↓ ↓
@register_tool("MyTool") MyAPI_list_items()
class MyTool(BaseTool): MyAPI_search()
def run(arguments): MyAPI_get_details()Key Points:
- One class handles multiple operations
- JSON defines individual tool wrappers
- Users call wrappers, which route to class
- Need BOTH for tools to work
Stage 1: Tool Class Stage 2: Wrappers (Auto-Generated)
Python Implementation From JSON Configs
↓ ↓
@register_tool("MyTool") MyAPI_list_items()
class MyTool(BaseTool): MyAPI_search()
def run(arguments): MyAPI_get_details()关键要点:
- 一个类处理多个操作
- JSON定义独立的工具封装
- 用户调用封装,由封装路由到对应类
- 两者缺一不可,工具才能正常工作
Three-Step Registration
三步注册流程
Step 1: Class Registration
python
@register_tool("MyAPITool") # Decorator registers class
class MyAPITool(BaseTool):
passStep 2: Config Registration ⚠️ MOST COMMONLY MISSED
python
undefined步骤1:类注册
python
@register_tool("MyAPITool") # 装饰器注册类
class MyAPITool(BaseTool):
pass步骤2:配置注册 ⚠️ 最常遗漏的步骤
python
undefinedIn src/tooluniverse/default_config.py
在 src/tooluniverse/default_config.py 文件中
TOOLS_CONFIGS = {
"my_category": os.path.join(current_dir, "data", "my_category_tools.json"),
}
**Step 3: Wrapper Generation** (Automatic)
```bash
tu = ToolUniverse()
tu.load_tools() # Auto-generates wrappers in tools/Verification Script:
python
import sys
sys.path.insert(0, 'src')TOOLS_CONFIGS = {
"my_category": os.path.join(current_dir, "data", "my_category_tools.json"),
}
**步骤3:封装生成**(自动完成)
```bash
tu = ToolUniverse()
tu.load_tools() # 在tools/目录下自动生成封装验证脚本:
python
import sys
sys.path.insert(0, 'src')Step 1: Check class registered
Step 1: 检查类是否注册
from tooluniverse.tool_registry import get_tool_registry
import tooluniverse.your_tool_module
registry = get_tool_registry()
assert "YourToolClass" in registry, "❌ Step 1 FAILED"
print("✅ Step 1: Class registered")
from tooluniverse.tool_registry import get_tool_registry
import tooluniverse.your_tool_module
registry = get_tool_registry()
assert "YourToolClass" in registry, "❌ 步骤1失败"
print("✅ 步骤1:类已注册")
Step 2: Check config registered
Step 2: 检查配置是否注册
from tooluniverse.default_config import TOOLS_CONFIGS
assert "your_category" in TOOLS_CONFIGS, "❌ Step 2 FAILED"
print("✅ Step 2: Config registered")
from tooluniverse.default_config import TOOLS_CONFIGS
assert "your_category" in TOOLS_CONFIGS, "❌ 步骤2失败"
print("✅ 步骤2:配置已注册")
Step 3: Check wrappers generated
Step 3: 检查封装是否生成
from tooluniverse import ToolUniverse
tu = ToolUniverse()
tu.load_tools()
assert hasattr(tu.tools, 'YourCategory_operation1'), "❌ Step 3 FAILED"
print("✅ Step 3: Wrappers generated")
print(f"✅ All steps complete!")
undefinedfrom tooluniverse import ToolUniverse
tu = ToolUniverse()
tu.load_tools()
assert hasattr(tu.tools, 'YourCategory_operation1'), "❌ 步骤3失败"
print("✅ 步骤3:封装已生成")
print(f"✅ 所有步骤完成!")
undefinedStandard Response Format
标准响应格式
All tools must return:
json
{
"status": "success" | "error",
"data": {...}, // On success
"error": "message" // On failure
}Why: Consistent error handling, composability, user expectations
所有工具必须返回如下格式:
json
{
"status": "success" | "error",
"data": {...}, // 成功时返回
"error": "message" // 失败时返回
}原因: 统一的错误处理、可组合性、符合用户预期
Implementation Guide
实现指南
File Structure
文件结构
Required Files:
- - Implementation
src/tooluniverse/my_api_tool.py - - Tool definitions
src/tooluniverse/data/my_api_tools.json - - Tests
tests/unit/test_my_api_tool.py - - Usage examples
examples/my_api_examples.py
Auto-Generated (don't create manually):
- - Wrappers
src/tooluniverse/tools/MyAPI_*.py
必需文件:
- - 工具实现代码
src/tooluniverse/my_api_tool.py - - 工具定义配置
src/tooluniverse/data/my_api_tools.json - - 测试用例
tests/unit/test_my_api_tool.py - - 使用示例
examples/my_api_examples.py
自动生成文件(无需手动创建):
- - 工具封装
src/tooluniverse/tools/MyAPI_*.py
Pattern 1: Multi-Operation Tool (Recommended)
模式1:多操作工具(推荐)
Python Class:
python
from typing import Dict, Any
from tooluniverse.tool import BaseTool
from tooluniverse.tool_utils import register_tool
import requests
@register_tool("MyAPITool")
class MyAPITool(BaseTool):
"""Tool for MyAPI database."""
BASE_URL = "https://api.example.com/v1"
def __init__(self, tool_config):
super().__init__(tool_config)
self.parameter = tool_config.get("parameter", {})
self.required = self.parameter.get("required", [])
def run(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Route to operation handler."""
operation = arguments.get("operation")
if not operation:
return {"status": "error", "error": "Missing: operation"}
if operation == "list_items":
return self._list_items(arguments)
elif operation == "search":
return self._search(arguments)
else:
return {"status": "error", "error": f"Unknown: {operation}"}
def _list_items(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""List items with pagination."""
try:
params = {}
if "limit" in arguments:
params["limit"] = arguments["limit"]
response = requests.get(
f"{self.BASE_URL}/items",
params=params,
timeout=30
)
response.raise_for_status()
data = response.json()
return {
"status": "success",
"data": data.get("items", []),
"total": data.get("total", 0)
}
except requests.exceptions.Timeout:
return {"status": "error", "error": "Timeout after 30s"}
except requests.exceptions.HTTPError as e:
return {"status": "error", "error": f"HTTP {e.response.status_code}"}
except Exception as e:
return {"status": "error", "error": str(e)}
def _search(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Search items by query."""
query = arguments.get("query")
if not query:
return {"status": "error", "error": "Missing: query"}
try:
response = requests.get(
f"{self.BASE_URL}/search",
params={"q": query},
timeout=30
)
response.raise_for_status()
data = response.json()
return {
"status": "success",
"results": data.get("results", []),
"count": data.get("count", 0)
}
except requests.exceptions.RequestException as e:
return {"status": "error", "error": f"API failed: {str(e)}"}JSON Configuration:
json
[
{
"name": "MyAPI_list_items",
"class": "MyAPITool",
"description": "List items from database with pagination. Returns item IDs and names. Supports filtering by status and type. Example: limit=10 returns first 10 items.",
"parameter": {
"type": "object",
"required": ["operation"],
"properties": {
"operation": {
"const": "list_items",
"description": "Operation type (fixed)"
},
"limit": {
"type": "integer",
"description": "Max results (1-100)",
"minimum": 1,
"maximum": 100
}
}
},
"return": {
"type": "object",
"properties": {
"status": {"type": "string", "enum": ["success", "error"]},
"data": {"type": "array"},
"total": {"type": "integer"},
"error": {"type": "string"}
},
"required": ["status"]
},
"test_examples": [
{
"operation": "list_items",
"limit": 10
}
]
}
]Python类:
python
from typing import Dict, Any
from tooluniverse.tool import BaseTool
from tooluniverse.tool_utils import register_tool
import requests
@register_tool("MyAPITool")
class MyAPITool(BaseTool):
"""Tool for MyAPI database."""
BASE_URL = "https://api.example.com/v1"
def __init__(self, tool_config):
super().__init__(tool_config)
self.parameter = tool_config.get("parameter", {})
self.required = self.parameter.get("required", [])
def run(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Route to operation handler."""
operation = arguments.get("operation")
if not operation:
return {"status": "error", "error": "Missing: operation"}
if operation == "list_items":
return self._list_items(arguments)
elif operation == "search":
return self._search(arguments)
else:
return {"status": "error", "error": f"Unknown: {operation}"}
def _list_items(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""List items with pagination."""
try:
params = {}
if "limit" in arguments:
params["limit"] = arguments["limit"]
response = requests.get(
f"{self.BASE_URL}/items",
params=params,
timeout=30
)
response.raise_for_status()
data = response.json()
return {
"status": "success",
"data": data.get("items", []),
"total": data.get("total", 0)
}
except requests.exceptions.Timeout:
return {"status": "error", "error": "Timeout after 30s"}
except requests.exceptions.HTTPError as e:
return {"status": "error", "error": f"HTTP {e.response.status_code}"}
except Exception as e:
return {"status": "error", "error": str(e)}
def _search(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Search items by query."""
query = arguments.get("query")
if not query:
return {"status": "error", "error": "Missing: query"}
try:
response = requests.get(
f"{self.BASE_URL}/search",
params={"q": query},
timeout=30
)
response.raise_for_status()
data = response.json()
return {
"status": "success",
"results": data.get("results", []),
"count": data.get("count", 0)
}
except requests.exceptions.RequestException as e:
return {"status": "error", "error": f"API failed: {str(e)}"}JSON配置:
json
[
{
"name": "MyAPI_list_items",
"class": "MyAPITool",
"description": "List items from database with pagination. Returns item IDs and names. Supports filtering by status and type. Example: limit=10 returns first 10 items.",
"parameter": {
"type": "object",
"required": ["operation"],
"properties": {
"operation": {
"const": "list_items",
"description": "Operation type (fixed)"
},
"limit": {
"type": "integer",
"description": "Max results (1-100)",
"minimum": 1,
"maximum": 100
}
}
},
"return": {
"type": "object",
"properties": {
"status": {"type": "string", "enum": ["success", "error"]},
"data": {"type": "array"},
"total": {"type": "integer"},
"error": {"type": "string"}
},
"required": ["status"]
},
"test_examples": [
{
"operation": "list_items",
"limit": 10
}
]
}
]Pattern 2: Async Polling (Job-Based APIs)
模式2:异步轮询(基于任务的API)
python
import time
def _submit_job(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Submit job and poll for results."""
try:
# Submit
submit_response = requests.post(
f"{self.BASE_URL}/jobs/submit",
json={"data": arguments.get("data")},
timeout=30
)
submit_response.raise_for_status()
job_id = submit_response.json().get("job_id")
# Poll
for attempt in range(60): # 2 min max
status_response = requests.get(
f"{self.BASE_URL}/jobs/{job_id}/status",
timeout=30
)
status_response.raise_for_status()
result = status_response.json()
if result.get("status") == "completed":
return {
"status": "success",
"data": result.get("results"),
"job_id": job_id
}
elif result.get("status") == "failed":
return {
"status": "error",
"error": result.get("error"),
"job_id": job_id
}
time.sleep(2) # Poll every 2s
return {"status": "error", "error": "Timeout after 2 min"}
except requests.exceptions.RequestException as e:
return {"status": "error", "error": str(e)}python
import time
def _submit_job(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
"""Submit job and poll for results."""
try:
# Submit
submit_response = requests.post(
f"{self.BASE_URL}/jobs/submit",
json={"data": arguments.get("data")},
timeout=30
)
submit_response.raise_for_status()
job_id = submit_response.json().get("job_id")
# Poll
for attempt in range(60): # 2 min max
status_response = requests.get(
f"{self.BASE_URL}/jobs/{job_id}/status",
timeout=30
)
status_response.raise_for_status()
result = status_response.json()
if result.get("status") == "completed":
return {
"status": "success",
"data": result.get("results"),
"job_id": job_id
}
elif result.get("status") == "failed":
return {
"status": "error",
"error": result.get("error"),
"job_id": job_id
}
time.sleep(2) # Poll every 2s
return {"status": "error", "error": "Timeout after 2 min"}
except requests.exceptions.RequestException as e:
return {"status": "error", "error": str(e)}API Key Configuration
API密钥配置
Tools can specify API key requirements in JSON config:
required_api_keysjson
{
"name": "NVIDIA_ESMFold_predict",
"required_api_keys": ["NVIDIA_API_KEY"],
...
}optional_api_keysjson
{
"name": "PubMed_search_articles",
"optional_api_keys": ["NCBI_API_KEY"],
"description": "Search PubMed. Rate limits: 3 req/sec without key, 10 req/sec with NCBI_API_KEY.",
...
}When to use :
optional_api_keys- APIs with rate limits that improve with a key (NCBI: 3→10 req/sec, Semantic Scholar: 1→100 req/sec)
- APIs that work anonymously but have better quotas with authentication
- Tools that should always be available, even for users without API keys
Implementation pattern for optional keys:
python
def __init__(self, tool_config):
super().__init__(tool_config)
# Read from environment variable only (not as parameter)
self.api_key = os.environ.get("NCBI_API_KEY", "")
def run(self, arguments):
# Adjust behavior based on key availability
has_key = bool(self.api_key)
rate_limit = 0.1 if has_key else 0.4 # Faster with key
...Key rules:
- ❌ Don't add as a tool parameter for optional keys
api_key - ✅ Read optional keys from environment variables only
- ✅ Include rate limit info in tool description
- ✅ Tool must work (with degraded performance) when key is missing
工具可在JSON配置中指定API密钥要求:
required_api_keysjson
{
"name": "NVIDIA_ESMFold_predict",
"required_api_keys": ["NVIDIA_API_KEY"],
...
}optional_api_keysjson
{
"name": "PubMed_search_articles",
"optional_api_keys": ["NCBI_API_KEY"],
"description": "Search PubMed. Rate limits: 3 req/sec without key, 10 req/sec with NCBI_API_KEY.",
...
}何时使用:
optional_api_keys- 有速率限制的API,使用密钥可提升速率(如NCBI:3→10次/秒,Semantic Scholar:1→100次/秒)
- 支持匿名访问,但认证后配额更高的API
- 需确保始终可用的工具,即使用户无API密钥
可选密钥实现模式:
python
def __init__(self, tool_config):
super().__init__(tool_config)
# 仅从环境变量读取(不作为工具参数)
self.api_key = os.environ.get("NCBI_API_KEY", "")
def run(self, arguments):
# 根据密钥可用性调整行为
has_key = bool(self.api_key)
rate_limit = 0.1 if has_key else 0.4 # 有密钥时请求间隔更短
...密钥规则:
- ❌ 不要为可选密钥添加工具参数
api_key - ✅ 仅从环境变量读取可选密钥
- ✅ 在工具描述中包含速率限制信息
- ✅ 无密钥时工具仍可工作(性能可能下降)
JSON Best Practices
JSON配置最佳实践
Tool Naming (≤55 chars for MCP):
- Template:
{API}_{action}_{target} - ✅ Good: (20 chars)
FDA_get_drug_info - ❌ Bad: (55+ chars)
FDA_get_detailed_drug_information_with_history
Description (150-250 chars, high-context):
json
{
"description": "Search database for items. Returns up to 100 results with scores. Supports wildcards (* ?) and Boolean operators (AND, OR, NOT). Example: 'protein AND membrane' finds membrane proteins."
}Include: What it returns, data source, use case, input format, example
test_examples (MUST be real):
json
{
"test_examples": [
{
"operation": "search",
"query": "protein", // ✅ Real, common term
"limit": 10
}
]
}❌ Don't use: ,
✅ Do use: Real IDs from actual API documentation
"id": "XXXXX""placeholder": "example_123"工具命名(长度≤55字符以兼容MCP):
- 模板:
{API}_{action}_{target} - ✅ 规范示例:(20字符)
FDA_get_drug_info - ❌ 错误示例:(超过55字符)
FDA_get_detailed_drug_information_with_history
描述(150-250字符,包含高上下文信息):
json
{
"description": "Search database for items. Returns up to 100 results with scores. Supports wildcards (* ?) and Boolean operators (AND, OR, NOT). Example: 'protein AND membrane' finds membrane proteins."
}描述应包含:返回内容、数据源、使用场景、输入格式、示例
test_examples(必须为真实有效示例):
json
{
"test_examples": [
{
"operation": "search",
"query": "protein", // ✅ 真实常用术语
"limit": 10
}
]
}❌ 禁止使用:,
✅ 推荐使用:API官方文档中的真实ID
"id": "XXXXX""placeholder": "example_123"JSON Config Conventions
JSON配置约定
- The field in JSON must match the Python class name from
"type"@register_tool - Include to define tool output structure (not just
return_schema)return - Put example inputs in only; avoid
test_examplesblocks insideexamples/parameter— they bloat configs and drift from realityreturn_schema - For large allow-lists, document values in and enforce in Python (avoid giant schema
description)enum - Auto-discovery: tools placed in are auto-discovered; no
src/tooluniverse/modification needed__init__.py - Optional: implement for custom validation
validate_parameters(arguments)
For the full implementation plan, maintenance checklist, and large API expansion guidance, see references/implementation-guide.md
- JSON中的字段必须与
"type"装饰器中的Python类名完全匹配@register_tool - 需包含以定义工具输出结构(而非仅
return_schema字段)return - 仅在中放置输入示例;避免在
test_examples/parameter中添加return_schema块——这会导致配置臃肿且与实际情况脱节examples - 对于大型允许列表,在描述中说明取值范围并在Python代码中做校验(避免使用庞大的Schema枚举)
- 自动发现机制:放置在目录下的工具会被自动发现;无需修改
src/tooluniverse/__init__.py - 可选:实现方法进行自定义参数校验
validate_parameters(arguments)
完整的实现计划、维护清单和大型API扩展指南,请参考references/implementation-guide.md
Testing Strategy
测试策略
Final Validation with test_new_tools.py (MANDATORY)
使用test_new_tools.py进行最终验证(强制要求)
All new tools MUST pass before submission.
scripts/test_new_tools.pyThis script tests tools using their from JSON configs and validates responses against .
test_examplesreturn_schemabash
undefined所有新工具在提交前必须通过测试
scripts/test_new_tools.py该脚本会使用JSON配置中的测试工具,并验证响应是否符合定义。
test_examplesreturn_schemabash
undefinedTest your specific tools
测试指定工具
python scripts/test_new_tools.py your_tool_name
python scripts/test_new_tools.py your_tool_name
Test with verbose output
带详细输出的测试
python scripts/test_new_tools.py your_tool_name -v
python scripts/test_new_tools.py your_tool_name -v
Test all tools (for full validation)
测试所有工具(完整验证)
python scripts/test_new_tools.py
python scripts/test_new_tools.py
Stop on first failure
遇到第一个失败即停止
python scripts/test_new_tools.py your_tool_name --fail-fast
**What it validates**:
- Tool execution succeeds (no exceptions)
- Response matches `return_schema` (if defined)
- 404 errors tracked separately (indicates bad test_examples)
- Tools with missing API keys are skipped gracefully
**Common failures and fixes**:
| Failure | Cause | Fix |
|---------|-------|-----|
| 404 ERROR | Invalid ID in test_examples | Use real IDs from API docs |
| Schema Mismatch | Response doesn't match return_schema | Update schema or fix response format |
| Exception | Code bug or missing dependency | Check error message, fix implementation |
---python scripts/test_new_tools.py your_tool_name --fail-fast
**验证内容**:
- 工具执行成功(无异常抛出)
- 响应符合`return_schema`定义(若已配置)
- 404错误会被单独追踪(表明test_examples无效)
- 缺少API密钥的工具会被优雅地跳过
**常见失败原因及修复方案**:
| 失败类型 | 原因 | 修复方案 |
|---------|-------|-----|
| 404 ERROR | test_examples中使用了无效ID | 使用API文档中的真实ID |
| Schema Mismatch | 响应不符合return_schema定义 | 更新Schema或修复响应格式 |
| Exception | 代码Bug或依赖缺失 | 查看错误信息,修复实现代码 |
---Two-Level Testing (MANDATORY)
两层测试(强制要求)
Level 1: Direct Class Testing
python
import json
from tooluniverse.your_tool_module import YourToolClass
def test_direct_class():
"""Test implementation logic."""
with open("src/tooluniverse/data/your_tools.json") as f:
tools = json.load(f)
config = next(t for t in tools if t["name"] == "YourTool_operation1")
tool = YourToolClass(config)
result = tool.run({"operation": "operation1", "param": "value"})
assert result["status"] == "success"
assert "data" in resultLevel 2: ToolUniverse Interface Testing
python
import pytest
from tooluniverse import ToolUniverse
class TestYourTools:
@pytest.fixture
def tu(self):
tu = ToolUniverse()
tu.load_tools() # CRITICAL
return tu
def test_tools_load(self, tu):
"""Verify registration."""
assert hasattr(tu.tools, 'YourTool_operation1')
def test_execution(self, tu):
"""Test via ToolUniverse (how users call it)."""
result = tu.tools.YourTool_operation1(**{
"operation": "operation1",
"param": "value"
})
assert result["status"] == "success"
def test_error_handling(self, tu):
"""Test missing params."""
result = tu.tools.YourTool_operation1(**{
"operation": "operation1"
# Missing required param
})
assert result["status"] == "error"Level 3: Real API Testing
python
def test_real_api():
"""Verify actual API integration."""
tu = ToolUniverse()
tu.load_tools()
result = tu.tools.YourTool_operation1(**{
"operation": "operation1",
"param": "real_value_from_docs"
})
if result["status"] == "success":
assert "data" in result
print("✅ Real API works")
else:
print(f"⚠️ API error (may be down): {result['error']}")Why Both Levels:
- Level 1: Tests implementation, catches code bugs
- Level 2: Tests registration, catches config bugs
- Level 3: Tests integration, catches API issues
层级1:直接类测试
python
import json
from tooluniverse.your_tool_module import YourToolClass
def test_direct_class():
"""Test implementation logic."""
with open("src/tooluniverse/data/your_tools.json") as f:
tools = json.load(f)
config = next(t for t in tools if t["name"] == "YourTool_operation1")
tool = YourToolClass(config)
result = tool.run({"operation": "operation1", "param": "value"})
assert result["status"] == "success"
assert "data" in result层级2:ToolUniverse接口测试
python
import pytest
from tooluniverse import ToolUniverse
class TestYourTools:
@pytest.fixture
def tu(self):
tu = ToolUniverse()
tu.load_tools() # CRITICAL
return tu
def test_tools_load(self, tu):
"""Verify registration."""
assert hasattr(tu.tools, 'YourTool_operation1')
def test_execution(self, tu):
"""Test via ToolUniverse (how users call it)."""
result = tu.tools.YourTool_operation1(**{
"operation": "operation1",
"param": "value"
})
assert result["status"] == "success"
def test_error_handling(self, tu):
"""Test missing params."""
result = tu.tools.YourTool_operation1(**{
"operation": "operation1"
# Missing required param
})
assert result["status"] == "error"层级3:真实API测试
python
def test_real_api():
"""Verify actual API integration."""
tu = ToolUniverse()
tu.load_tools()
result = tu.tools.YourTool_operation1(**{
"operation": "operation1",
"param": "real_value_from_docs"
})
if result["status"] == "success":
assert "data" in result
print("✅ Real API works")
else:
print(f"⚠️ API error (may be down): {result['error']}")为何需要两层测试:
- 层级1:测试实现逻辑,捕获代码Bug
- 层级2:测试注册流程,捕获配置Bug
- 层级3:测试集成效果,捕获API问题
Common Patterns
常见模式
Error Handling Checklist
错误处理检查清单
✅ Always set timeout (30s recommended)
✅ Catch specific exceptions (Timeout, ConnectionError, HTTPError)
✅ Return error dicts, never raise in run()
✅ Include helpful context in error messages
✅ Handle JSON parsing errors
✅ Validate required parameters
✅ 始终设置超时时间(推荐30秒)
✅ 捕获特定异常(Timeout、ConnectionError、HTTPError)
✅ 返回错误字典,不在run()方法中抛出异常
✅ 错误信息中包含有用的上下文
✅ 处理JSON解析错误
✅ 校验必填参数
Dependency Management
依赖管理
Check package size FIRST:
bash
curl -s https://pypi.org/pypi/PACKAGE/json | python3 -c "
import json, sys
data = json.load(sys.stdin)
print(f'Dependencies: {len(data[\"info\"][\"requires_dist\"] or [])}')
"Classification:
- Core (<100MB, universal use) →
[project.dependencies] - Optional (>100MB or niche) →
[project.optional-dependencies]
In code:
python
try:
import optional_package
except ImportError:
return {
"status": "error",
"error": "Install with: pip install optional_package"
}先检查包大小:
bash
curl -s https://pypi.org/pypi/PACKAGE/json | python3 -c "
import json, sys
data = json.load(sys.stdin)
print(f'Dependencies: {len(data[\"info\"][\"requires_dist\"] or [])}')
"依赖分类:
- 核心依赖(<100MB,通用场景)→ 放入
[project.dependencies] - 可选依赖(>100MB或 niche场景)→ 放入
[project.optional-dependencies]
代码中处理方式:
python
try:
import optional_package
except ImportError:
return {
"status": "error",
"error": "Install with: pip install optional_package"
}Pagination Pattern
分页模式
python
def _list_items(self, arguments):
params = {}
if "page" in arguments:
params["page"] = arguments["page"]
if "limit" in arguments:
params["limit"] = arguments["limit"]
response = requests.get(url, params=params, timeout=30)
data = response.json()
return {
"status": "success",
"data": data.get("items", []),
"page": data.get("page", 0),
"total_pages": data.get("total_pages", 1),
"total_items": data.get("total", 0)
}python
def _list_items(self, arguments):
params = {}
if "page" in arguments:
params["page"] = arguments["page"]
if "limit" in arguments:
params["limit"] = arguments["limit"]
response = requests.get(url, params=params, timeout=30)
data = response.json()
return {
"status": "success",
"data": data.get("items", []),
"page": data.get("page", 0),
"total_pages": data.get("total_pages", 1),
"total_items": data.get("total", 0)
}Troubleshooting
故障排查
Tool Doesn't Load (90% of Issues)
工具无法加载(90%的问题)
Symptoms: Tool count doesn't increase, no error, when calling
AttributeErrorCause: Missing Step 2 of registration (default_config.py)
Solution:
python
undefined症状: 工具数量未增加,无错误提示,调用时出现
AttributeError原因: 遗漏注册流程的步骤2(default_config.py配置)
解决方案:
python
undefinedEdit src/tooluniverse/default_config.py
编辑 src/tooluniverse/default_config.py
TOOLS_CONFIGS = {
# ... existing ...
"your_category": os.path.join(current_dir, "data", "your_category_tools.json"),
}
**Verify**:
```bash
grep "your_category" src/tooluniverse/default_config.py
ls src/tooluniverse/tools/YourCategory_*.py
python3 -c "from tooluniverse import ToolUniverse; tu = ToolUniverse(); tu.load_tools(); print(hasattr(tu.tools, 'YourCategory_op1'))"TOOLS_CONFIGS = {
# ... 现有配置 ...
"your_category": os.path.join(current_dir, "data", "your_category_tools.json"),
}
**验证方式**:
```bash
grep "your_category" src/tooluniverse/default_config.py
ls src/tooluniverse/tools/YourCategory_*.py
python3 -c "from tooluniverse import ToolUniverse; tu = ToolUniverse(); tu.load_tools(); print(hasattr(tu.tools, 'YourCategory_op1'))"Tests Fail with Real APIs
真实API测试失败
Mock vs Real Testing:
- Mocks test code structure
- Real calls test API integration
- Both needed for confidence
What Real Testing Catches:
- Response structure differences
- Parameter name mismatches
- Unexpected pagination
- Timeout issues
- Data type surprises
Mock测试 vs 真实测试:
- Mock测试验证代码结构
- 真实调用测试API集成效果
- 两者结合才能确保工具可靠
真实测试可捕获的问题:
- 响应结构差异
- 参数名称不匹配
- 意外的分页逻辑
- 超时问题
- 数据类型异常
Reference
参考资料
Complete Workflow
完整工作流程
- Create Python class with
@register_tool - Create JSON config with realistic test_examples and
return_schema - Add to ← CRITICAL
default_config.py - Generate wrappers:
tu.load_tools() - Test Level 1 (direct class)
- Test Level 2 (ToolUniverse interface)
- Test Level 3 (real API calls)
- Run ← MANDATORY
python scripts/test_new_tools.py your_tool -v - Run
python scripts/check_tool_name_lengths.py --test-shortening - Create examples file
- Verify all 3 registration steps
- Document in verification report
For the full development checklist and maintenance phases, see references/implementation-guide.md
- 创建带装饰器的Python类
@register_tool - 创建包含真实test_examples和的JSON配置
return_schema - 添加到← 关键步骤
default_config.py - 生成封装:
tu.load_tools() - 测试层级1(直接类测试)
- 测试层级2(ToolUniverse接口测试)
- 测试层级3(真实API调用)
- 运行 ← 强制要求
python scripts/test_new_tools.py your_tool -v - 运行
python scripts/check_tool_name_lengths.py --test-shortening - 创建示例文件
- 验证所有3个注册步骤
- 记录验证报告
完整的开发检查清单和维护阶段指南,请参考references/implementation-guide.md
Quick Commands
常用命令
bash
undefinedbash
undefinedValidate JSON
验证JSON格式
python3 -m json.tool src/tooluniverse/data/your_tools.json
python3 -m json.tool src/tooluniverse/data/your_tools.json
Check Python syntax
检查Python语法
python3 -m py_compile src/tooluniverse/your_tool.py
python3 -m py_compile src/tooluniverse/your_tool.py
Verify registration
验证注册配置
grep "your_category" src/tooluniverse/default_config.py
grep "your_category" src/tooluniverse/default_config.py
Generate wrappers
生成封装
PYTHONPATH=src python3 -m tooluniverse.generate_tools --force
PYTHONPATH=src python3 -m tooluniverse.generate_tools --force
List wrappers
列出所有封装
ls src/tooluniverse/tools/YourCategory_*.py
ls src/tooluniverse/tools/YourCategory_*.py
Run unit tests
运行单元测试
pytest tests/unit/test_your_tool.py -v
pytest tests/unit/test_your_tool.py -v
MANDATORY: Run test_new_tools.py validation
强制要求:运行test_new_tools.py验证
python scripts/test_new_tools.py your_tool -v
python scripts/test_new_tools.py your_tool -v
Count tools
统计工具数量
python3 << 'EOF'
from tooluniverse import ToolUniverse
tu = ToolUniverse()
tu.load_tools()
print(f"Total: {len([t for t in dir(tu.tools) if 'YourCategory' in t])} tools")
EOF
undefinedpython3 << 'EOF'
from tooluniverse import ToolUniverse
tu = ToolUniverse()
tu.load_tools()
print(f"Total: {len([t for t in dir(tu.tools) if 'YourCategory' in t])} tools")
EOF
undefinedCritical Reminders
关键提醒
⚠️ ALWAYS add to default_config.py (Step 2)
⚠️ NEVER raise exceptions in run()
⚠️ ALWAYS use real test_examples
⚠️ ALWAYS test both levels
⚠️ RUN before submitting
⚠️ KEEP tool names ≤55 characters
⚠️ RETURN standard response format
⚠️ SET timeout on all HTTP requests
⚠️ VERIFY all 3 registration steps
⚠️ USE for rate-limited APIs that work without keys
⚠️ NEVER add parameter for optional keys - use env vars only
python scripts/test_new_tools.py your_tool -voptional_api_keysapi_key⚠️ 必须添加到default_config.py(步骤2)
⚠️ 禁止在run()方法中抛出异常
⚠️ 必须使用真实的test_examples
⚠️ 必须完成两层测试
⚠️ 运行 后再提交
⚠️ 确保工具名称长度≤55字符
⚠️ 使用标准响应格式
⚠️ 设置所有HTTP请求的超时时间
⚠️ 验证所有3个注册步骤
⚠️ 使用标记无需密钥即可使用但速率受限的API
⚠️ 禁止为可选密钥添加参数 - 仅使用环境变量
python scripts/test_new_tools.py your_tool -voptional_api_keysapi_keySuccess Criteria
成功标准
✅ All 3 registration steps verified
✅ Level 1 tests passing (direct class)
✅ Level 2 tests passing (ToolUniverse interface)
✅ Real API calls working (Level 3)
✅ passes with 0 failures ← MANDATORY
✅ Tool names ≤55 characters
✅ test_examples use real IDs
✅ Standard response format used
✅ Helpful error messages
✅ Examples file created
✅ No raised exceptions in run()
test_new_tools.pyWhen all criteria met → Production Ready 🎉
✅ 所有3个注册步骤验证通过
✅ 层级1测试通过(直接类测试)
✅ 层级2测试通过(ToolUniverse接口测试)
✅ 真实API调用正常(层级3)
✅ 测试0失败 ← 强制要求
✅ 工具名称长度≤55字符
✅ test_examples使用真实ID
✅ 使用标准响应格式
✅ 错误信息有帮助性
✅ 创建了示例文件
✅ run()方法中无抛出异常
test_new_tools.py满足所有标准后 → 可投入生产环境 🎉