mlkit-agent-api

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Targeting AICore Developer Preview. This skill provides authoritative execution patterns and failure-prevention rules for on-device agentic workflows in Android apps.

适用于 AICore Developer Preview。此 skill 为 Android 应用中的端侧 agent 工作流提供权威的执行模式和故障预防规则。

1. Prerequisites and setup

1. 前提条件与设置

Gradle dependencies

Gradle 依赖项

// app/build.gradle.kts dependencies { // Core ML Kit GenAI Agent API implementation("com.google.mlkit:genai-agent:1.0.0-alpha1")

    // Android AppFunctions dependency for on-device tool declarations
    implementation("androidx.appfunctions:appfunctions:1.0.0-alpha13")
}

[!IMPORTANT] Always use official SDK interfaces from
com.google.mlkit.genai.agent.*
,
androidx.appfunctions.*
,
com.google.mlkit.genai.prompt.*
, and
com.google.mlkit.genai.common.*
. Never declare local placeholder interfaces or fallback mocks.

// app/build.gradle.kts dependencies { // Core ML Kit GenAI Agent API implementation("com.google.mlkit:genai-agent:1.0.0-alpha1")

    // Android AppFunctions dependency for on-device tool declarations
    implementation("androidx.appfunctions:appfunctions:1.0.0-alpha13")
}

[!IMPORTANT] 始终使用来自以下包的官方 SDK 接口：
com.google.mlkit.genai.agent.*
、
androidx.appfunctions.*
、
com.google.mlkit.genai.prompt.*
和
com.google.mlkit.genai.common.*
。切勿声明本地占位接口或回退 mock。

2. Minimal agent and runner initialization

2. 最小化 Agent 和 Runner 初始化

Configure an

Agent

with

Agent.builder

and wrap execution in

AgentRunner

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("OnDeviceAssistant")
    .setInstructions("You are a helpful on-device assistant.")
    .build()

val runner = AgentRunner(agent)

```
Agent.builder
```
defaults to
```
maxTurns = 10
```
automatically to prevent infinite tool loops.
```
AgentRunner
```
defaults to greedy decoding (
```
temperature = 0.0f
```
) for deterministic tool selection and schema compliance.
```
AgentRunner(agent)
```
automatically resolves application context using
```
MlKitContext
```
for AppFunction execution in Android apps. An explicit
```
AgentRunner(context, agent)
```
overload is available for headless tests or custom context scopes.

使用

Agent.builder

配置

Agent

，并通过

AgentRunner

封装执行：

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("OnDeviceAssistant")
    .setInstructions("You are a helpful on-device assistant.")
    .build()

val runner = AgentRunner(agent)

```
Agent.builder
```
自动将
```
maxTurns
```
设为
```
10
```
，以防止无限的工具调用循环。
```
AgentRunner
```
默认采用贪心解码（
```
temperature = 0.0f
```
），以实现确定性的工具选择并确保符合 schema。
```
AgentRunner(agent)
```
使用
```
MlKitContext
```
自动解析应用上下文，以在 Android 应用中执行 AppFunction。对于无头测试或自定义上下文作用域，可使用显式的
```
AgentRunner(context, agent)
```
重载。

3. Multi-turn chat and lifecycle (ViewModel)

3. 多轮聊天与生命周期（ViewModel）

AgentSession

preserves conversational history in memory and is not
Parcelable
. To preserve dialogue history across Android configuration changes (such as screen rotation), retain

session

inside an AndroidX

ViewModel

When executing single-turn ephemeral queries, managing concurrent multi-sessions, or requiring explicit RAM cleanup, you must follow the guidelines in Session lifecycle.

package com.example.app

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentSession
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.common.GenAiException
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch

class ChatViewModel : ViewModel() {
    private val agent = Agent.builder()
        .setInstructions("You are a helpful on-device assistant.")
        .build()
    private val runner = AgentRunner(agent)
    private val session: AgentSession =
        runner.getOrCreateSession("chat_session")

    private val _uiState = MutableStateFlow<ChatUiState>(ChatUiState.Idle)
    val uiState: StateFlow<ChatUiState> = _uiState.asStateFlow()

    fun sendMessage(prompt: String) {
        viewModelScope.launch {
            _uiState.value = ChatUiState.Loading("")
            try {
                // Content.builder() required; runAsync takes Content
                val content = Content.builder().text(prompt).build()
                var fullResponse = ""
                session.runAsync(content)
                    // Extension function requiring textTokens import
                    .textTokens()
                    .collect { token ->
                        fullResponse += token
                        _uiState.value = ChatUiState.Loading(fullResponse)
                    }
                _uiState.value = ChatUiState.Success(fullResponse)
            } catch (e: GenAiException) {
                // Catch GenAiException to prevent app crashes on AI errors
                val err = e.localizedMessage ?: "AI error (${e.errorCode})"
                _uiState.value = ChatUiState.Error(err)
            }
        }
    }

    override fun onCleared() {
        super.onCleared()
        // Evict conversation session from RAM when ViewModel is destroyed
        session.close()
    }
}

AgentSession

在内存中保留对话历史记录，且不是
Parcelable
。若要在 Android 配置更改（例如屏幕旋转）后保留对话历史，请在 AndroidX

ViewModel

中保留

session

。

在执行临时单轮查询、管理并发多会话或需要显式清理 RAM 时，必须遵循会话生命周期中的指南。

package com.example.app

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentSession
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.common.GenAiException
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch

class ChatViewModel : ViewModel() {
    private val agent = Agent.builder()
        .setInstructions("You are a helpful on-device assistant.")
        .build()
    private val runner = AgentRunner(agent)
    private val session: AgentSession =
        runner.getOrCreateSession("chat_session")

    private val _uiState = MutableStateFlow<ChatUiState>(ChatUiState.Idle)
    val uiState: StateFlow<ChatUiState> = _uiState.asStateFlow()

    fun sendMessage(prompt: String) {
        viewModelScope.launch {
            _uiState.value = ChatUiState.Loading("")
            try {
                // Content.builder() required; runAsync takes Content
                val content = Content.builder().text(prompt).build()
                var fullResponse = ""
                session.runAsync(content)
                    // Extension function requiring textTokens import
                    .textTokens()
                    .collect { token ->
                        fullResponse += token
                        _uiState.value = ChatUiState.Loading(fullResponse)
                    }
                _uiState.value = ChatUiState.Success(fullResponse)
            } catch (e: GenAiException) {
                // Catch GenAiException to prevent app crashes on AI errors
                val err = e.localizedMessage ?: "AI error (${e.errorCode})"
                _uiState.value = ChatUiState.Error(err)
            }
        }
    }

    override fun onCleared() {
        super.onCleared()
        // Evict conversation session from RAM when ViewModel is destroyed
        session.close()
    }
}

Essential lifecycle and coroutine rules

关键生命周期和协程规则

Content
object :

session.runAsync

and

runner.runSingleTurnAsync

require a

Content

object. Always build it using

Content.builder().text(userPrompt).build()

textTokens
extension :

textTokens

is a Kotlin extension function on

Flow<AgentEvent>

. Always include

import com.google.mlkit.genai.agent.textTokens

Session cleanup : Always invoke
```
session.close
```
in
```
ViewModel.onCleared
```
to evict conversational RAM and cancel pending tasks.
AgentEvent
stream : When collecting
```
Flow<AgentEvent>
```
directly instead of
```
textTokens
```
, always include
```
import com.google.mlkit.genai.agent.AgentEvent
```
. Match against
```
AgentEvent.TextMessage(text, isPartial)
```
,
```
AgentEvent.FunctionCall(name, args)
```
, and
```
AgentEvent.FunctionResponse(name, response)
```
, and always include an
```
else -> {}
```
branch in
```
when (event)
```
expressions (
```
AgentEvent
```
is non-exhaustive for forward compatibility):
import com.google.mlkit.genai.agent.AgentEvent

session.runAsync(content).collect { event -> when (event) { is AgentEvent.FunctionCall -> { _uiState.update { it.copy(activeToolStatus = "Tool started: ${event.name}") } } is AgentEvent.FunctionResponse -> { _uiState.update { it.copy(activeToolStatus = "Tool completed: ${event.name}") } } is AgentEvent.TextMessage -> { _uiState.update { it.copy(responseText = it.responseText + event.text) } } else -> {} // Required: AgentEvent is non-exhaustive for forward compat } }

Content
对象：

session.runAsync

和

runner.runSingleTurnAsync

需要

Content

对象。始终使用

Content.builder().text(userPrompt).build()

构建。

textTokens
扩展函数：

textTokens

是

Flow<AgentEvent>

上的 Kotlin 扩展函数。始终包含导入

com.google.mlkit.genai.agent.textTokens

。

会话清理：始终在
```
ViewModel.onCleared
```
中调用
```
session.close
```
，以释放对话占用的 RAM 并取消待处理任务。
AgentEvent
流：直接收集
```
Flow<AgentEvent>
```
而不是使用
```
textTokens
```
时，始终包含导入
```
com.google.mlkit.genai.agent.AgentEvent
```
。匹配
```
AgentEvent.TextMessage(text, isPartial)
```
、
```
AgentEvent.FunctionCall(name, args)
```
和
```
AgentEvent.FunctionResponse(name, response)
```
，并且始终在
```
when (event)
```
表达式中包含
```
else -> {}
```
分支（为保证向前兼容，
```
AgentEvent
```
是非穷尽类型）：
import com.google.mlkit.genai.agent.AgentEvent

session.runAsync(content).collect { event -> when (event) { is AgentEvent.FunctionCall -> { _uiState.update { it.copy(activeToolStatus = "Tool started: ${event.name}") } } is AgentEvent.FunctionResponse -> { _uiState.update { it.copy(activeToolStatus = "Tool completed: ${event.name}") } } is AgentEvent.TextMessage -> { _uiState.update { it.copy(responseText = it.responseText + event.text) } } else -> {} // Required: AgentEvent is non-exhaustive for forward compat } }

4. Tool calling recipe

4. 工具调用方案

All tools used with

Agent

originate from Android AppFunctions (

@AppFunctionDeclaration

), registered using the declarative path (

enableAppFunctions(true)

addAppFunctionIds(...)

) or advanced discovery (

AppFunctionAgentTools.discover(...)

When you need to restrict caller access levels for private in-app tools or wrap discovered AppFunctions with human-in-the-loop (HITL) confirmation decorators, you must consult Tools and functions.

所有与

Agent

一起使用的工具都源自 Android AppFunctions （

@AppFunctionDeclaration

），并通过声明式路径（

enableAppFunctions(true)

或

addAppFunctionIds(...)

）或高级发现机制（

AppFunctionAgentTools.discover(...)

）注册。

当需要为应用内私有工具限制调用方访问级别，或使用 human-in-the-loop（HITL）确认装饰器包装已发现的 AppFunctions 时，必须查阅工具与函数。

Option A: Declarative AppFunctions path

选项 A：声明式 AppFunctions 路径

Define tools declaratively in your app using AndroidX AppFunctions:

package com.example.app.tools

import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.AppFunctionSerializable

@AppFunctionSerializable
data class CalendarEvents(val date: String, val events: List<String>)

class CalendarTools {
    /**
     * Fetches scheduled calendar appointments for a specific date in
     * YYYY-MM-DD format.
     *
     * @param date Date string in YYYY-MM-DD format
     */
    @AppFunctionDeclaration(isDescribedByKDoc = true)
    fun fetch_calendar(
        date: String
    ): CalendarEvents {
        return CalendarEvents(
            date = date,
            events = listOf(
                "Team Standup at 10:00 AM",
                "Product Review at 2:00 PM"
            )
        )
    }
}

Enable AppFunctions on

Agent.Builder

and initialize

AgentRunner

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    // Automatically discovers AppFunctions in your app package
    .enableAppFunctions(true)
    // Optional: scope to specific function IDs or method names
    .addAppFunctionIds("fetch_calendar")
    .build()

val runner = AgentRunner(agent)

AppFunction Contract : Tool methods take domain parameters directly (no framework context parameter required). Returns must be primitives or
```
@AppFunctionSerializable
```
data classes (
```
Map<String, Any?>
```
isn't supported for AppFunctions).
Restricting Callers with
ACCESS_LEVEL_SELF
: AppFunctions don't have to be public to external callers. To restrict execution strictly to your app's internal agent (blocking external system assistants and IPC callers), annotate with
```
@OptIn(ExperimentalAppFunctionsApi::class)
```
and
```
@AppFunctionAccessLevel(level = AppFunctionMetadata.ACCESS_LEVEL_SELF, isCompatEnforcementEnabled = true)
```
.

使用 AndroidX AppFunctions 在应用中声明式定义工具：

package com.example.app.tools

import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.AppFunctionSerializable

@AppFunctionSerializable
data class CalendarEvents(val date: String, val events: List<String>)

class CalendarTools {
    /**
     * Fetches scheduled calendar appointments for a specific date in
     * YYYY-MM-DD format.
     *
     * @param date Date string in YYYY-MM-DD format
     */
    @AppFunctionDeclaration(isDescribedByKDoc = true)
    fun fetch_calendar(
        date: String
    ): CalendarEvents {
        return CalendarEvents(
            date = date,
            events = listOf(
                "Team Standup at 10:00 AM",
                "Product Review at 2:00 PM"
            )
        )
    }
}

在

Agent.Builder

上启用 AppFunctions 并初始化

AgentRunner

：

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    // Automatically discovers AppFunctions in your app package
    .enableAppFunctions(true)
    // Optional: scope to specific function IDs or method names
    .addAppFunctionIds("fetch_calendar")
    .build()

val runner = AgentRunner(agent)

AppFunction 约定：工具方法直接接受领域参数（无需框架上下文参数）。返回值必须是基本类型或
```
@AppFunctionSerializable
```
data 类（AppFunctions 不支持
```
Map<String, Any?>
```
）。
使用
ACCESS_LEVEL_SELF
限制调用方：AppFunctions 无须向外部调用方公开。若要将执行严格限制在应用内部 Agent（阻止外部系统助手和 IPC 调用方），请使用
```
@OptIn(ExperimentalAppFunctionsApi::class)
```
和
```
@AppFunctionAccessLevel(level = AppFunctionMetadata.ACCESS_LEVEL_SELF, isCompatEnforcementEnabled = true)
```
进行注解。

Option B: Advanced discovery and decorator pattern

选项 B：高级发现与装饰器模式

For inspecting, filtering, or decorating discovered AppFunctions (such as Human-in-the-Loop confirmation wrappers using

AgentTool(delegate)

) before registering with the agent:

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentTool
import com.google.mlkit.genai.agent.AppFunctionAgentTools

// Discover tools (optionally scoped to function IDs)
val discoveredTools = AppFunctionAgentTools.discover(
    enabledAppFunctionIds = listOf("fetch_calendar")
)

// Optional: Wrap sensitive tools using AgentTool(delegate) decorator
class HitlConfirmationTool(
    delegate: AgentTool,
    private val onConfirm: suspend (
        toolName: String,
        args: Map<String, Any>
    ) -> Boolean
) : AgentTool(delegate) {
    override suspend fun execute(args: Map<String, Any>): Map<String, Any?> {
        if (!onConfirm(name, args)) {
            return mapOf("status" to "CANCELLED_BY_USER")
        }
        return checkNotNull(delegate).execute(args)
    }
}

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    .addTools(discoveredTools)
    .build()

val runner = AgentRunner(agent)

[!WARNING] Mutual exclusivity : Configure AppFunctions using either the declarative path (
enableAppFunctions(true)
on
Agent.Builder
) or the advanced path (
AppFunctionAgentTools.discover
passed to
addTools
), but not both. Using both simultaneously triggers a warning and registers duplicate tool schemas.

在注册到 Agent 之前，检查、过滤或装饰已发现的 AppFunctions（例如使用

AgentTool(delegate)

的 Human-in-the-Loop 确认包装器）：

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentTool
import com.google.mlkit.genai.agent.AppFunctionAgentTools

// Discover tools (optionally scoped to function IDs)
val discoveredTools = AppFunctionAgentTools.discover(
    enabledAppFunctionIds = listOf("fetch_calendar")
)

// Optional: Wrap sensitive tools using AgentTool(delegate) decorator
class HitlConfirmationTool(
    delegate: AgentTool,
    private val onConfirm: suspend (
        toolName: String,
        args: Map<String, Any>
    ) -> Boolean
) : AgentTool(delegate) {
    override suspend fun execute(args: Map<String, Any>): Map<String, Any?> {
        if (!onConfirm(name, args)) {
            return mapOf("status" to "CANCELLED_BY_USER")
        }
        return checkNotNull(delegate).execute(args)
    }
}

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    .addTools(discoveredTools)
    .build()

val runner = AgentRunner(agent)

[!WARNING] 互斥性：使用声明式路径（
Agent.Builder
上的
enableAppFunctions(true)
）或高级路径（通过
addTools
传入
AppFunctionAgentTools.discover
）配置 AppFunctions，但不要同时使用两者。同时使用会触发警告并注册重复的工具 schema。

Deprecated annotations (@Tool and @Param)

已弃用的注解（

@Tool

和

@Param

）

The legacy

@Tool

and

@Param

annotations in

com.google.mlkit.genai.schema.annotations.*

, along with

AgentTool.fromAnnotatedObject

Agent.Builder.addTools(annotatedObject: Any)

, and standalone

FunctionTool

, have been deprecated or removed. All tools added to

Agent

must originate from

@androidx.appfunctions.AppFunctionDeclaration

using

Agent.Builder.enableAppFunctions(true)

AppFunctionAgentTools.discover

com.google.mlkit.genai.schema.annotations.*

中的旧版

@Tool

和

@Param

注解，以及

AgentTool.fromAnnotatedObject

、

Agent.Builder.addTools(annotatedObject: Any)

和独立的

FunctionTool

均已弃用或移除。添加到

Agent

的所有工具都必须源自

@androidx.appfunctions.AppFunctionDeclaration

，并通过

Agent.Builder.enableAppFunctions(true)

或

AppFunctionAgentTools.discover

5. Ephemeral single-turn execution

5. 临时单轮执行

For one-off tasks where conversation history isn't needed (such as summarizing text, formatting data, or answering standalone questions), invoke

runner.runSingleTurnAsync(content)

directly:

import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.Flow

// Executes a single-turn query ephemerally without session persistence:
fun askQuickQuestion(runner: AgentRunner, question: String): Flow<String> {
    val content = Content.builder().text(question).build()
    return runner.runSingleTurnAsync(content).textTokens()
}

```
runner.runSingleTurnAsync(content)
```
automatically creates and evicts an ephemeral session upon Flow completion.
Cleanup reference :
- ```
session.close
```
  : Releases session resources and cancels coroutines (call in
```
ViewModel.onCleared
```
  ).
- ```
session.delete
```
  or
```
runner.deleteSession(sessionId)
```
  : Permanently deletes the session from the runner.

对于不需要对话历史记录的一次性任务（例如摘要文本、格式化数据或回答独立问题），直接调用

runner.runSingleTurnAsync(content)

：

import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.Flow

// Executes a single-turn query ephemerally without session persistence:
fun askQuickQuestion(runner: AgentRunner, question: String): Flow<String> {
    val content = Content.builder().text(question).build()
    return runner.runSingleTurnAsync(content).textTokens()
}

```
runner.runSingleTurnAsync(content)
```
会在 Flow 完成时自动创建并回收临时 session。
清理参考：
- ```
session.close
```
  ：释放会话资源并取消协程（在
```
ViewModel.onCleared
```
  中调用）。
- ```
session.delete
```
  或
```
runner.deleteSession(sessionId)
```
  ：从 runner 中永久删除会话。

6. Error handling and GenAiException reference

6. 错误处理与 GenAiException 参考

Runtime failures during on-device execution (such as model unavailability or exceeding maximum turn limits) are emitted or thrown as

GenAiException

import com.google.mlkit.genai.common.GenAiException

端侧执行期间发生的运行时故障（例如模型不可用或超过最大轮次限制）会通过

GenAiException

发出或抛出：

import com.google.mlkit.genai.common.GenAiException

Key error codes

关键错误代码

```
GenAiException.ErrorCode.MAX_TURNS_EXCEEDED
```
(
```
-112
```
): The agent exceeded
```
maxTurns
```
(default 10) without completing the reasoning loop.
```
GenAiException.ErrorCode.NOT_AVAILABLE
```
(
```
8
```
): The Gemini Nano model or AICore service isn't available on this device.
```
GenAiException.ErrorCode.BUSY
```
(
```
9
```
): System or AICore is busy, or battery or thermal quota has been reached.

```
GenAiException.ErrorCode.MAX_TURNS_EXCEEDED
```
（
```
-112
```
）：Agent 超过
```
maxTurns
```
（默认值为 10），未能完成推理循环。
```
GenAiException.ErrorCode.NOT_AVAILABLE
```
（
```
8
```
）：此设备上不存在 Gemini Nano 模型或 AICore 服务。
```
GenAiException.ErrorCode.BUSY
```
（
```
9
```
）：系统或 AICore 正忙，或已达到电池或热配额限制。