google-agents-cli

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

google-agents-cli

google-agents-cli

Skill by ara.so — Devtools Skills collection.
agents-cli
is the CLI and skills framework for building, evaluating, and deploying AI agents on Google Cloud's Gemini Enterprise Agent Platform. It works with ADK (Agent Development Kit) to provide end-to-end agent development workflows, from scaffolding to production deployment.
ara.so提供的Skill——Devtools Skills合集。
agents-cli
是一款CLI工具及技能框架,用于在Google Cloud的Gemini Enterprise Agent平台上构建、评估和部署AI Agent。它与ADK(Agent Development Kit)配合使用,提供从项目搭建到生产部署的端到端Agent开发工作流。

What It Does

功能介绍

  • Scaffold agent projects: Create new ADK agent projects with best practices built-in
  • Local development: Run and test agents locally with hot reload
  • Evaluation: Run systematic evaluations with metrics, evalsets, and LLM-as-judge
  • Deployment: Deploy to Google Cloud (Agent Runtime, Cloud Run, GKE)
  • Publishing: Register agents with Gemini Enterprise
  • Observability: Integrate Cloud Trace, logging, and third-party monitoring
  • CI/CD: Set up staging/prod pipelines with automated testing
  • 项目搭建: 创建内置最佳实践的全新ADK Agent项目
  • 本地开发: 本地运行并测试Agent,支持热重载
  • 评估: 通过指标、评估集和LLM-as-judge运行系统性评估
  • 部署: 部署至Google Cloud(Agent Runtime、Cloud Run、GKE)
  • 发布: 在Gemini Enterprise中注册Agent
  • 可观测性: 集成Cloud Trace、日志记录及第三方监控
  • CI/CD: 搭建包含自动化测试的预发布/生产环境流水线

Installation

安装

Prerequisites: Python 3.11+, uv, Node.js
bash
undefined
前置要求: Python 3.11+、uv、Node.js
bash
undefined

Install CLI and skills

Install CLI and skills

uvx google-agents-cli setup
uvx google-agents-cli setup

Or just install the CLI

Or just install the CLI

pip install google-agents-cli
pip install google-agents-cli

Verify installation

Verify installation

agents-cli --version
undefined
agents-cli --version
undefined

Authentication

身份验证

bash
undefined
bash
undefined

Authenticate with Google Cloud

Authenticate with Google Cloud

agents-cli login
agents-cli login

Check authentication status

Check authentication status

agents-cli login --status
agents-cli login --status

For local development without Google Cloud, use AI Studio API key

For local development without Google Cloud, use AI Studio API key

export GOOGLE_API_KEY=your_api_key_here
undefined
export GOOGLE_API_KEY=your_api_key_here
undefined

Core Commands

核心命令

Project Scaffolding

项目搭建

bash
undefined
bash
undefined

Create a new agent project

Create a new agent project

agents-cli scaffold my-agent
agents-cli scaffold my-agent

Create with specific template

Create with specific template

agents-cli scaffold my-agent --template basic agents-cli scaffold my-agent --template rag
agents-cli scaffold my-agent --template basic agents-cli scaffold my-agent --template rag

Add features to existing project

Add features to existing project

agents-cli scaffold enhance --add deployment agents-cli scaffold enhance --add cicd agents-cli scaffold enhance --add rag
agents-cli scaffold enhance --add deployment agents-cli scaffold enhance --add cicd agents-cli scaffold enhance --add rag

Upgrade project to newer agents-cli version

Upgrade project to newer agents-cli version

agents-cli scaffold upgrade
undefined
agents-cli scaffold upgrade
undefined

Local Development

本地开发

bash
undefined
bash
undefined

Install project dependencies

Install project dependencies

agents-cli install
agents-cli install

Run agent with a single prompt

Run agent with a single prompt

agents-cli run "What's the weather in San Francisco?"
agents-cli run "What's the weather in San Francisco?"

Run with file input

Run with file input

agents-cli run --input-file prompt.txt
agents-cli run --input-file prompt.txt

Run with streaming output

Run with streaming output

agents-cli run "Summarize this article" --stream
agents-cli run "Summarize this article" --stream

Lint code

Lint code

agents-cli lint
undefined
agents-cli lint
undefined

Evaluation

评估

bash
undefined
bash
undefined

Run evaluations

Run evaluations

agents-cli eval run
agents-cli eval run

Run specific evalset

Run specific evalset

agents-cli eval run --evalset evalsets/basic.yaml
agents-cli eval run --evalset evalsets/basic.yaml

Compare two evaluation results

Compare two evaluation results

agents-cli eval compare results/eval1.json results/eval2.json
agents-cli eval compare results/eval1.json results/eval2.json

Run with custom metrics

Run with custom metrics

agents-cli eval run --metrics accuracy,latency,cost
undefined
agents-cli eval run --metrics accuracy,latency,cost
undefined

Deployment

部署

bash
undefined
bash
undefined

Deploy to Google Cloud (interactive)

Deploy to Google Cloud (interactive)

agents-cli deploy
agents-cli deploy

Deploy with specific config

Deploy with specific config

agents-cli deploy --config deploy.yaml
agents-cli deploy --config deploy.yaml

Deploy to specific environment

Deploy to specific environment

agents-cli deploy --env production
agents-cli deploy --env production

Provision infrastructure

Provision infrastructure

agents-cli infra single-project --project-id my-project agents-cli infra cicd --project-id my-project
agents-cli infra single-project --project-id my-project agents-cli infra cicd --project-id my-project

Set up datastore for RAG

Set up datastore for RAG

agents-cli infra datastore --project-id my-project
undefined
agents-cli infra datastore --project-id my-project
undefined

Publishing

发布

bash
undefined
bash
undefined

Register with Gemini Enterprise

Register with Gemini Enterprise

agents-cli publish gemini-enterprise
agents-cli publish gemini-enterprise

Publish with metadata

Publish with metadata

agents-cli publish gemini-enterprise --name "My Agent" --description "Does X"
undefined
agents-cli publish gemini-enterprise --name "My Agent" --description "Does X"
undefined

Data Ingestion (RAG)

数据摄入(RAG)

bash
undefined
bash
undefined

Run data ingestion pipeline

Run data ingestion pipeline

agents-cli data-ingestion --source gs://my-bucket/docs agents-cli data-ingestion --source ./local-docs --datastore my-datastore
undefined
agents-cli data-ingestion --source gs://my-bucket/docs agents-cli data-ingestion --source ./local-docs --datastore my-datastore
undefined

Utilities

实用工具

bash
undefined
bash
undefined

Show project info and CLI version

Show project info and CLI version

agents-cli info
agents-cli info

Force reinstall skills to all IDEs

Force reinstall skills to all IDEs

agents-cli update
undefined
agents-cli update
undefined

ADK Agent Code Patterns

ADK Agent代码模式

Basic Agent Structure

基础Agent结构

python
undefined
python
undefined

my_agent.py

my_agent.py

from adk.agents import Agent from adk.tools import Tool from adk.models import ModelClient
from adk.agents import Agent from adk.tools import Tool from adk.models import ModelClient

Define a custom tool

Define a custom tool

class WeatherTool(Tool): """Get weather information for a location."""
def __init__(self):
    super().__init__(
        name="get_weather",
        description="Get current weather for a location"
    )

def execute(self, location: str) -> str:
    # Implementation
    return f"Weather in {location}: Sunny, 72°F"
class WeatherTool(Tool): """Get weather information for a location."""
def __init__(self):
    super().__init__(
        name="get_weather",
        description="Get current weather for a location"
    )

def execute(self, location: str) -> str:
    # Implementation
    return f"Weather in {location}: Sunny, 72°F"

Create agent

Create agent

agent = Agent( name="weather-assistant", description="An agent that provides weather information", model=ModelClient(model_name="gemini-2.0-flash-exp"), tools=[WeatherTool()] )
agent = Agent( name="weather-assistant", description="An agent that provides weather information", model=ModelClient(model_name="gemini-2.0-flash-exp"), tools=[WeatherTool()] )

Run agent

Run agent

if name == "main": response = agent.run("What's the weather in NYC?") print(response)
undefined
if name == "main": response = agent.run("What's the weather in NYC?") print(response)
undefined

Agent with State Management

带状态管理的Agent

python
from adk.agents import Agent, AgentState
from adk.models import ModelClient
from typing import Any, Dict

class ConversationState(AgentState):
    """Custom state for conversation tracking."""
    
    def __init__(self):
        super().__init__()
        self.conversation_history = []
        self.user_preferences = {}
    
    def add_message(self, role: str, content: str):
        self.conversation_history.append({"role": role, "content": content})

agent = Agent(
    name="stateful-assistant",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    state=ConversationState()
)
python
from adk.agents import Agent, AgentState
from adk.models import ModelClient
from typing import Any, Dict

class ConversationState(AgentState):
    """Custom state for conversation tracking."""
    
    def __init__(self):
        super().__init__()
        self.conversation_history = []
        self.user_preferences = {}
    
    def add_message(self, role: str, content: str):
        self.conversation_history.append({"role": role, "content": content})

agent = Agent(
    name="stateful-assistant",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    state=ConversationState()
)

Use state in agent execution

Use state in agent execution

response = agent.run("Remember my name is Alice") agent.state.user_preferences["name"] = "Alice"
undefined
response = agent.run("Remember my name is Alice") agent.state.user_preferences["name"] = "Alice"
undefined

Multi-Agent Orchestration

多Agent编排

python
from adk.agents import Agent, AgentOrchestrator
from adk.models import ModelClient
python
from adk.agents import Agent, AgentOrchestrator
from adk.models import ModelClient

Create specialized agents

Create specialized agents

research_agent = Agent( name="researcher", description="Researches topics and gathers information", model=ModelClient(model_name="gemini-2.0-flash-exp") )
writer_agent = Agent( name="writer", description="Writes content based on research", model=ModelClient(model_name="gemini-2.0-flash-exp") )
research_agent = Agent( name="researcher", description="Researches topics and gathers information", model=ModelClient(model_name="gemini-2.0-flash-exp") )
writer_agent = Agent( name="writer", description="Writes content based on research", model=ModelClient(model_name="gemini-2.0-flash-exp") )

Orchestrate agents

Orchestrate agents

orchestrator = AgentOrchestrator( agents=[research_agent, writer_agent], workflow="sequential" # or "parallel", "conditional" )
orchestrator = AgentOrchestrator( agents=[research_agent, writer_agent], workflow="sequential" # or "parallel", "conditional" )

Run orchestrated workflow

Run orchestrated workflow

result = orchestrator.run("Write an article about AI agents")
undefined
result = orchestrator.run("Write an article about AI agents")
undefined

Agent with Callbacks

带回调的Agent

python
from adk.agents import Agent
from adk.callbacks import Callback
from adk.models import ModelClient

class LoggingCallback(Callback):
    """Log agent execution steps."""
    
    def on_agent_start(self, agent_name: str, input_data: Any):
        print(f"Agent {agent_name} starting with input: {input_data}")
    
    def on_tool_start(self, tool_name: str, tool_input: Dict[str, Any]):
        print(f"Tool {tool_name} called with: {tool_input}")
    
    def on_tool_end(self, tool_name: str, tool_output: Any):
        print(f"Tool {tool_name} returned: {tool_output}")
    
    def on_agent_end(self, agent_name: str, output: Any):
        print(f"Agent {agent_name} finished with: {output}")

agent = Agent(
    name="monitored-agent",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    callbacks=[LoggingCallback()]
)
python
from adk.agents import Agent
from adk.callbacks import Callback
from adk.models import ModelClient

class LoggingCallback(Callback):
    """Log agent execution steps."""
    
    def on_agent_start(self, agent_name: str, input_data: Any):
        print(f"Agent {agent_name} starting with input: {input_data}")
    
    def on_tool_start(self, tool_name: str, tool_input: Dict[str, Any]):
        print(f"Tool {tool_name} called with: {tool_input}")
    
    def on_tool_end(self, tool_name: str, tool_output: Any):
        print(f"Tool {tool_name} returned: {tool_output}")
    
    def on_agent_end(self, agent_name: str, output: Any):
        print(f"Agent {agent_name} finished with: {output}")

agent = Agent(
    name="monitored-agent",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    callbacks=[LoggingCallback()]
)

RAG Agent Pattern

RAG Agent模式

python
from adk.agents import Agent
from adk.tools import Tool
from adk.models import ModelClient
from adk.rag import VectorStore, Retriever

class RAGTool(Tool):
    """Retrieve relevant documents from vector store."""
    
    def __init__(self, datastore_id: str):
        super().__init__(
            name="retrieve_docs",
            description="Retrieve relevant documents"
        )
        self.retriever = Retriever(datastore_id=datastore_id)
    
    def execute(self, query: str) -> str:
        docs = self.retriever.retrieve(query, top_k=5)
        return "\n\n".join([doc.content for doc in docs])

agent = Agent(
    name="rag-assistant",
    description="Agent with RAG capabilities",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    tools=[RAGTool(datastore_id="my-datastore")]
)
python
from adk.agents import Agent
from adk.tools import Tool
from adk.models import ModelClient
from adk.rag import VectorStore, Retriever

class RAGTool(Tool):
    """Retrieve relevant documents from vector store."""
    
    def __init__(self, datastore_id: str):
        super().__init__(
            name="retrieve_docs",
            description="Retrieve relevant documents"
        )
        self.retriever = Retriever(datastore_id=datastore_id)
    
    def execute(self, query: str) -> str:
        docs = self.retriever.retrieve(query, top_k=5)
        return "\n\n".join([doc.content for doc in docs])

agent = Agent(
    name="rag-assistant",
    description="Agent with RAG capabilities",
    model=ModelClient(model_name="gemini-2.0-flash-exp"),
    tools=[RAGTool(datastore_id="my-datastore")]
)

Project Configuration

项目配置

agents.yaml

agents.yaml

yaml
undefined
yaml
undefined

Project configuration

Project configuration

name: my-agent version: 1.0.0 description: My AI agent
name: my-agent version: 1.0.0 description: My AI agent

Agent configuration

Agent configuration

agent: name: my-assistant model: gemini-2.0-flash-exp temperature: 0.7 max_tokens: 2048
agent: name: my-assistant model: gemini-2.0-flash-exp temperature: 0.7 max_tokens: 2048

Tools configuration

Tools configuration

tools:
  • name: web_search enabled: true
  • name: code_execution enabled: false
tools:
  • name: web_search enabled: true
  • name: code_execution enabled: false

Evaluation configuration

Evaluation configuration

evaluation: evalsets: - path: evalsets/basic.yaml - path: evalsets/advanced.yaml metrics: - accuracy - latency - cost
evaluation: evalsets: - path: evalsets/basic.yaml - path: evalsets/advanced.yaml metrics: - accuracy - latency - cost

Deployment configuration

Deployment configuration

deployment: target: cloud-run region: us-central1 min_instances: 1 max_instances: 10
deployment: target: cloud-run region: us-central1 min_instances: 1 max_instances: 10

Observability

Observability

observability: cloud_trace: true cloud_logging: true
undefined
observability: cloud_trace: true cloud_logging: true
undefined

Evalset Configuration

评估集配置

yaml
undefined
yaml
undefined

evalsets/basic.yaml

evalsets/basic.yaml

name: basic-evalset description: Basic functionality tests
test_cases:
  • id: tc-001 input: "What is 2+2?" expected_output: "4" metrics:
    • accuracy
    • latency
  • id: tc-002 input: "Explain quantum computing in simple terms" judge: type: llm-as-judge criteria: - clarity - accuracy - conciseness
  • id: tc-003 input: "Write a Python function to reverse a string" validator: type: code-execution test: | def test_reverse(): assert reverse("hello") == "olleh" assert reverse("") == ""
undefined
name: basic-evalset description: Basic functionality tests
test_cases:
  • id: tc-001 input: "What is 2+2?" expected_output: "4" metrics:
    • accuracy
    • latency
  • id: tc-002 input: "Explain quantum computing in simple terms" judge: type: llm-as-judge criteria: - clarity - accuracy - conciseness
  • id: tc-003 input: "Write a Python function to reverse a string" validator: type: code-execution test: | def test_reverse(): assert reverse("hello") == "olleh" assert reverse("") == ""
undefined

Deployment Configuration

部署配置

yaml
undefined
yaml
undefined

deploy.yaml

deploy.yaml

target: cloud-run project_id: ${GCP_PROJECT_ID} region: us-central1
service: name: my-agent-service min_instances: 1 max_instances: 10 cpu: 2 memory: 4Gi timeout: 300s
environment:
  • name: GOOGLE_API_KEY secret: projects/${GCP_PROJECT_ID}/secrets/gemini-api-key
  • name: LOG_LEVEL value: INFO
ci_cd: enabled: true environments: - name: staging project_id: ${STAGING_PROJECT_ID} branch: develop - name: production project_id: ${PRODUCTION_PROJECT_ID} branch: main
undefined
target: cloud-run project_id: ${GCP_PROJECT_ID} region: us-central1
service: name: my-agent-service min_instances: 1 max_instances: 10 cpu: 2 memory: 4Gi timeout: 300s
environment:
  • name: GOOGLE_API_KEY secret: projects/${GCP_PROJECT_ID}/secrets/gemini-api-key
  • name: LOG_LEVEL value: INFO
ci_cd: enabled: true environments: - name: staging project_id: ${STAGING_PROJECT_ID} branch: develop - name: production project_id: ${PRODUCTION_PROJECT_ID} branch: main
undefined

Environment Variables

环境变量

bash
undefined
bash
undefined

API Keys

API Keys

export GOOGLE_API_KEY=your_api_key_here export GOOGLE_CLOUD_PROJECT=your-project-id
export GOOGLE_API_KEY=your_api_key_here export GOOGLE_CLOUD_PROJECT=your-project-id

Agent Configuration

Agent Configuration

export AGENT_MODEL=gemini-2.0-flash-exp export AGENT_TEMPERATURE=0.7 export AGENT_MAX_TOKENS=2048
export AGENT_MODEL=gemini-2.0-flash-exp export AGENT_TEMPERATURE=0.7 export AGENT_MAX_TOKENS=2048

Deployment

Deployment

export DEPLOY_REGION=us-central1 export DEPLOY_ENV=production
export DEPLOY_REGION=us-central1 export DEPLOY_ENV=production

Observability

Observability

export ENABLE_CLOUD_TRACE=true export ENABLE_CLOUD_LOGGING=true export LOG_LEVEL=INFO
export ENABLE_CLOUD_TRACE=true export ENABLE_CLOUD_LOGGING=true export LOG_LEVEL=INFO

RAG

RAG

export DATASTORE_ID=my-datastore export VECTOR_STORE_TYPE=vertex-ai-search
undefined
export DATASTORE_ID=my-datastore export VECTOR_STORE_TYPE=vertex-ai-search
undefined

Common Patterns

常见模式

Creating a Complete Agent Project

创建完整的Agent项目

bash
undefined
bash
undefined

1. Scaffold project

1. Scaffold project

agents-cli scaffold my-agent
agents-cli scaffold my-agent

2. Navigate to project

2. Navigate to project

cd my-agent
cd my-agent

3. Install dependencies

3. Install dependencies

agents-cli install
agents-cli install

4. Edit agent code

4. Edit agent code

Edit src/agent.py with your agent logic

Edit src/agent.py with your agent logic

5. Run locally

5. Run locally

agents-cli run "Test prompt"
agents-cli run "Test prompt"

6. Create evalset

6. Create evalset

Create evalsets/test.yaml

Create evalsets/test.yaml

7. Run evaluations

7. Run evaluations

agents-cli eval run
agents-cli eval run

8. Deploy to Google Cloud

8. Deploy to Google Cloud

agents-cli login agents-cli deploy
agents-cli login agents-cli deploy

9. Publish to Gemini Enterprise

9. Publish to Gemini Enterprise

agents-cli publish gemini-enterprise
undefined
agents-cli publish gemini-enterprise
undefined

Adding Tools to an Agent

为Agent添加工具

python
from adk.agents import Agent
from adk.tools import Tool
from adk.models import ModelClient
import requests

class SearchTool(Tool):
    """Search the web for information."""
    
    def __init__(self, api_key: str):
        super().__init__(
            name="web_search",
            description="Search the web for current information"
        )
        self.api_key = api_key
    
    def execute(self, query: str, num_results: int = 5) -> str:
        # Implementation using search API
        # Use self.api_key from environment
        results = self._search(query, num_results)
        return "\n".join([r["title"] + ": " + r["snippet"] for r in results])
    
    def _search(self, query: str, num_results: int):
        # Actual search implementation
        pass

class CalculatorTool(Tool):
    """Perform mathematical calculations."""
    
    def __init__(self):
        super().__init__(
            name="calculator",
            description="Perform mathematical calculations"
        )
    
    def execute(self, expression: str) -> float:
        # Safe evaluation of mathematical expressions
        import ast
        import operator
        
        operators = {
            ast.Add: operator.add,
            ast.Sub: operator.sub,
            ast.Mult: operator.mul,
            ast.Div: operator.truediv,
        }
        
        def eval_expr(node):
            if isinstance(node, ast.Num):
                return node.n
            elif isinstance(node, ast.BinOp):
                return operators[type(node.op)](
                    eval_expr(node.left),
                    eval_expr(node.right)
                )
            else:
                raise ValueError("Unsupported expression")
        
        return eval_expr(ast.parse(expression, mode='eval').body)
python
from adk.agents import Agent
from adk.tools import Tool
from adk.models import ModelClient
import requests

class SearchTool(Tool):
    """Search the web for information."""
    
    def __init__(self, api_key: str):
        super().__init__(
            name="web_search",
            description="Search the web for current information"
        )
        self.api_key = api_key
    
    def execute(self, query: str, num_results: int = 5) -> str:
        # Implementation using search API
        # Use self.api_key from environment
        results = self._search(query, num_results)
        return "\n".join([r["title"] + ": " + r["snippet"] for r in results])
    
    def _search(self, query: str, num_results: int):
        # Actual search implementation
        pass

class CalculatorTool(Tool):
    """Perform mathematical calculations."""
    
    def __init__(self):
        super().__init__(
            name="calculator",
            description="Perform mathematical calculations"
        )
    
    def execute(self, expression: str) -> float:
        # Safe evaluation of mathematical expressions
        import ast
        import operator
        
        operators = {
            ast.Add: operator.add,
            ast.Sub: operator.sub,
            ast.Mult: operator.mul,
            ast.Div: operator.truediv,
        }
        
        def eval_expr(node):
            if isinstance(node, ast.Num):
                return node.n
            elif isinstance(node, ast.BinOp):
                return operators[type(node.op)](
                    eval_expr(node.left),
                    eval_expr(node.right)
                )
            else:
                raise ValueError("Unsupported expression")
        
        return eval_expr(ast.parse(expression, mode='eval').body)

Create agent with multiple tools

Create agent with multiple tools

import os
agent = Agent( name="research-assistant", model=ModelClient(model_name="gemini-2.0-flash-exp"), tools=[ SearchTool(api_key=os.getenv("SEARCH_API_KEY")), CalculatorTool() ] )
undefined
import os
agent = Agent( name="research-assistant", model=ModelClient(model_name="gemini-2.0-flash-exp"), tools=[ SearchTool(api_key=os.getenv("SEARCH_API_KEY")), CalculatorTool() ] )
undefined

Setting Up CI/CD

搭建CI/CD

bash
undefined
bash
undefined

1. Set up CI/CD infrastructure

1. Set up CI/CD infrastructure

agents-cli infra cicd --project-id my-project
agents-cli infra cicd --project-id my-project

2. Create deployment configs for staging and prod

2. Create deployment configs for staging and prod

Edit deploy-staging.yaml and deploy-production.yaml

Edit deploy-staging.yaml and deploy-production.yaml

3. Configure GitHub Actions or Cloud Build

3. Configure GitHub Actions or Cloud Build

The scaffold enhance command creates templates

The scaffold enhance command creates templates

4. Push to repository

4. Push to repository

git add . git commit -m "Set up CI/CD" git push origin main
undefined
git add . git commit -m "Set up CI/CD" git push origin main
undefined

Implementing Custom Metrics

实现自定义指标

python
undefined
python
undefined

evalsets/custom_metrics.py

evalsets/custom_metrics.py

from adk.evaluation import Metric from typing import Dict, Any
class CustomAccuracyMetric(Metric): """Custom accuracy metric with fuzzy matching."""
def __init__(self, threshold: float = 0.8):
    super().__init__(name="custom_accuracy")
    self.threshold = threshold

def evaluate(self, 
             prediction: str, 
             expected: str, 
             context: Dict[str, Any]) -> float:
    # Custom evaluation logic
    from difflib import SequenceMatcher
    ratio = SequenceMatcher(None, prediction, expected).ratio()
    return 1.0 if ratio >= self.threshold else 0.0
class LatencyMetric(Metric): """Measure agent response latency."""
def __init__(self):
    super().__init__(name="latency")

def evaluate(self,
             prediction: str,
             expected: str,
             context: Dict[str, Any]) -> float:
    # Return latency in milliseconds
    return context.get("execution_time_ms", 0)
from adk.evaluation import Metric from typing import Dict, Any
class CustomAccuracyMetric(Metric): """Custom accuracy metric with fuzzy matching."""
def __init__(self, threshold: float = 0.8):
    super().__init__(name="custom_accuracy")
    self.threshold = threshold

def evaluate(self, 
             prediction: str, 
             expected: str, 
             context: Dict[str, Any]) -> float:
    # Custom evaluation logic
    from difflib import SequenceMatcher
    ratio = SequenceMatcher(None, prediction, expected).ratio()
    return 1.0 if ratio >= self.threshold else 0.0
class LatencyMetric(Metric): """Measure agent response latency."""
def __init__(self):
    super().__init__(name="latency")

def evaluate(self,
             prediction: str,
             expected: str,
             context: Dict[str, Any]) -> float:
    # Return latency in milliseconds
    return context.get("execution_time_ms", 0)

Use in evaluation

Use in evaluation

from adk.evaluation import Evaluator
evaluator = Evaluator( agent=my_agent, evalset_path="evalsets/basic.yaml", metrics=[ CustomAccuracyMetric(threshold=0.85), LatencyMetric() ] )
results = evaluator.run() print(f"Average accuracy: {results.metrics['custom_accuracy']}") print(f"Average latency: {results.metrics['latency']}ms")
undefined
from adk.evaluation import Evaluator
evaluator = Evaluator( agent=my_agent, evalset_path="evalsets/basic.yaml", metrics=[ CustomAccuracyMetric(threshold=0.85), LatencyMetric() ] )
results = evaluator.run() print(f"Average accuracy: {results.metrics['custom_accuracy']}") print(f"Average latency: {results.metrics['latency']}ms")
undefined

Troubleshooting

故障排查

Authentication Issues

身份验证问题

bash
undefined
bash
undefined

Clear credentials and re-authenticate

Clear credentials and re-authenticate

agents-cli login --force
agents-cli login --force

Use service account for CI/CD

Use service account for CI/CD

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json agents-cli login --service-account
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json agents-cli login --service-account

Check authentication status

Check authentication status

agents-cli login --status
undefined
agents-cli login --status
undefined

Deployment Failures

部署失败

bash
undefined
bash
undefined

Check project configuration

Check project configuration

agents-cli info
agents-cli info

Validate deployment config

Validate deployment config

agents-cli deploy --validate-only
agents-cli deploy --validate-only

Deploy with verbose logging

Deploy with verbose logging

agents-cli deploy --verbose
agents-cli deploy --verbose

Check Cloud Run logs

Check Cloud Run logs

gcloud run services logs read my-agent-service --project=${GCP_PROJECT_ID}
undefined
gcloud run services logs read my-agent-service --project=${GCP_PROJECT_ID}
undefined

Evaluation Issues

评估问题

bash
undefined
bash
undefined

Run single test case

Run single test case

agents-cli eval run --test-case tc-001
agents-cli eval run --test-case tc-001

Run with debug mode

Run with debug mode

agents-cli eval run --debug
agents-cli eval run --debug

Validate evalset format

Validate evalset format

agents-cli eval run --validate-only
agents-cli eval run --validate-only

Check evaluation logs

Check evaluation logs

cat .agents-cli/eval-results/latest.log
undefined
cat .agents-cli/eval-results/latest.log
undefined

Dependency Conflicts

依赖冲突

bash
undefined
bash
undefined

Reinstall dependencies

Reinstall dependencies

agents-cli install --force
agents-cli install --force

Clear cache

Clear cache

rm -rf .agents-cli/cache agents-cli install
rm -rf .agents-cli/cache agents-cli install

Use specific Python version

Use specific Python version

uv venv --python 3.11 source .venv/bin/activate agents-cli install
undefined
uv venv --python 3.11 source .venv/bin/activate agents-cli install
undefined

Model Access Issues

模型访问问题

bash
undefined
bash
undefined

Verify API key is set

Verify API key is set

echo $GOOGLE_API_KEY
echo $GOOGLE_API_KEY

Test model access directly

Test model access directly

curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=${GOOGLE_API_KEY}"
-H 'Content-Type: application/json'
-d '{"contents":[{"parts":[{"text":"Hello"}]}]}'
curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=${GOOGLE_API_KEY}"
-H 'Content-Type: application/json'
-d '{"contents":[{"parts":[{"text":"Hello"}]}]}'

Switch to different model

Switch to different model

export AGENT_MODEL=gemini-1.5-pro agents-cli run "test"
undefined
export AGENT_MODEL=gemini-1.5-pro agents-cli run "test"
undefined

Performance Optimization

性能优化

python
undefined
python
undefined

Enable caching for repeated queries

Enable caching for repeated queries

from adk.models import ModelClient
model = ModelClient( model_name="gemini-2.0-flash-exp", cache_enabled=True, cache_ttl=3600 # 1 hour )
from adk.models import ModelClient
model = ModelClient( model_name="gemini-2.0-flash-exp", cache_enabled=True, cache_ttl=3600 # 1 hour )

Batch processing for evaluations

Batch processing for evaluations

agents-cli eval run --batch-size 10 --parallel 4
agents-cli eval run --batch-size 10 --parallel 4

Reduce token usage

Reduce token usage

agent = Agent( name="efficient-agent", model=ModelClient( model_name="gemini-2.0-flash-exp", max_tokens=1024, # Lower limit temperature=0.3 # More deterministic ) )
undefined
agent = Agent( name="efficient-agent", model=ModelClient( model_name="gemini-2.0-flash-exp", max_tokens=1024, # Lower limit temperature=0.3 # More deterministic ) )
undefined

Debugging Agent Behavior

调试Agent行为

python
from adk.agents import Agent
from adk.callbacks import DebugCallback
from adk.models import ModelClient
python
from adk.agents import Agent
from adk.callbacks import DebugCallback
from adk.models import ModelClient

Enable detailed debugging

Enable detailed debugging

debug_callback = DebugCallback( log_prompts=True, log_responses=True, log_tool_calls=True )
agent = Agent( name="debug-agent", model=ModelClient(model_name="gemini-2.0-flash-exp"), callbacks=[debug_callback], verbose=True )
debug_callback = DebugCallback( log_prompts=True, log_responses=True, log_tool_calls=True )
agent = Agent( name="debug-agent", model=ModelClient(model_name="gemini-2.0-flash-exp"), callbacks=[debug_callback], verbose=True )

Check agent execution traces

Check agent execution traces

response = agent.run("Debug this behavior") print(agent.get_trace())
undefined
response = agent.run("Debug this behavior") print(agent.get_trace())
undefined

Model Selection Guide

模型选择指南

python
undefined
python
undefined

Fast, cost-effective for simple tasks

Fast, cost-effective for simple tasks

ModelClient(model_name="gemini-2.0-flash-exp")
ModelClient(model_name="gemini-2.0-flash-exp")

Balanced performance and capabilities

Balanced performance and capabilities

ModelClient(model_name="gemini-1.5-pro")
ModelClient(model_name="gemini-1.5-pro")

Maximum capabilities for complex reasoning

Maximum capabilities for complex reasoning

ModelClient(model_name="gemini-1.5-pro-002")
ModelClient(model_name="gemini-1.5-pro-002")

Multimodal tasks (images, video)

Multimodal tasks (images, video)

ModelClient( model_name="gemini-1.5-pro-vision", multimodal=True )
undefined
ModelClient( model_name="gemini-1.5-pro-vision", multimodal=True )
undefined