mlkit-agent-api

Targeting AICore Developer Preview. This skill provides authoritative execution patterns and failure-prevention rules for on-device agentic workflows in Android apps.

1. Prerequisites and setup

Gradle dependencies

// app/build.gradle.kts
dependencies {
    // Core ML Kit GenAI Agent API
    implementation("com.google.mlkit:genai-agent:1.0.0-alpha1")

    // Android AppFunctions dependency for on-device tool declarations
    implementation("androidx.appfunctions:appfunctions:1.0.0-alpha13")
}

[!IMPORTANT] Always use official SDK interfaces from
com.google.mlkit.genai.agent.*
,
androidx.appfunctions.*
,
com.google.mlkit.genai.prompt.*
, and
com.google.mlkit.genai.common.*
. Never declare local placeholder interfaces or fallback mocks.

2. Minimal agent and runner initialization

Configure an

Agent

with

Agent.builder

and wrap execution in

AgentRunner

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("OnDeviceAssistant")
    .setInstructions("You are a helpful on-device assistant.")
    .build()

val runner = AgentRunner(agent)

```
Agent.builder
```
defaults to
```
maxTurns = 10
```
automatically to prevent infinite tool loops.
```
AgentRunner
```
defaults to greedy decoding (
```
temperature = 0.0f
```
) for deterministic tool selection and schema compliance.
```
AgentRunner(agent)
```
automatically resolves application context using
```
MlKitContext
```
for AppFunction execution in Android apps. An explicit
```
AgentRunner(context, agent)
```
overload is available for headless tests or custom context scopes.

3. Multi-turn chat and lifecycle (ViewModel)

AgentSession

preserves conversational history in memory and is not
Parcelable
. To preserve dialogue history across Android configuration changes (such as screen rotation), retain

session

inside an AndroidX

ViewModel

When executing single-turn ephemeral queries, managing concurrent multi-sessions, or requiring explicit RAM cleanup, you must follow the guidelines in Session lifecycle.

package com.example.app

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentSession
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.common.GenAiException
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch

class ChatViewModel : ViewModel() {
    private val agent = Agent.builder()
        .setInstructions("You are a helpful on-device assistant.")
        .build()
    private val runner = AgentRunner(agent)
    private val session: AgentSession =
        runner.getOrCreateSession("chat_session")

    private val _uiState = MutableStateFlow<ChatUiState>(ChatUiState.Idle)
    val uiState: StateFlow<ChatUiState> = _uiState.asStateFlow()

    fun sendMessage(prompt: String) {
        viewModelScope.launch {
            _uiState.value = ChatUiState.Loading("")
            try {
                // Content.builder() required; runAsync takes Content
                val content = Content.builder().text(prompt).build()
                var fullResponse = ""
                session.runAsync(content)
                    // Extension function requiring textTokens import
                    .textTokens()
                    .collect { token ->
                        fullResponse += token
                        _uiState.value = ChatUiState.Loading(fullResponse)
                    }
                _uiState.value = ChatUiState.Success(fullResponse)
            } catch (e: GenAiException) {
                // Catch GenAiException to prevent app crashes on AI errors
                val err = e.localizedMessage ?: "AI error (${e.errorCode})"
                _uiState.value = ChatUiState.Error(err)
            }
        }
    }

    override fun onCleared() {
        super.onCleared()
        // Evict conversation session from RAM when ViewModel is destroyed
        session.close()
    }
}

Essential lifecycle and coroutine rules

Content
object :

session.runAsync

and

runner.runSingleTurnAsync

require a

Content

object. Always build it using

Content.builder().text(userPrompt).build()

textTokens
extension :

textTokens

is a Kotlin extension function on

Flow<AgentEvent>

. Always include

import com.google.mlkit.genai.agent.textTokens

Session cleanup : Always invoke
```
session.close
```
in
```
ViewModel.onCleared
```
to evict conversational RAM and cancel pending tasks.
AgentEvent
stream : When collecting
```
Flow<AgentEvent>
```
directly instead of
```
textTokens
```
, always include
```
import com.google.mlkit.genai.agent.AgentEvent
```
. Match against
```
AgentEvent.TextMessage(text, isPartial)
```
,
```
AgentEvent.FunctionCall(name, args)
```
, and
```
AgentEvent.FunctionResponse(name, response)
```
, and always include an
```
else -> {}
```
branch in
```
when (event)
```
expressions (
```
AgentEvent
```
is non-exhaustive for forward compatibility):
import com.google.mlkit.genai.agent.AgentEvent

session.runAsync(content).collect { event -> when (event) { is AgentEvent.FunctionCall -> { _uiState.update { it.copy(activeToolStatus = "Tool started: ${event.name}") } } is AgentEvent.FunctionResponse -> { _uiState.update { it.copy(activeToolStatus = "Tool completed: ${event.name}") } } is AgentEvent.TextMessage -> { _uiState.update { it.copy(responseText = it.responseText + event.text) } } else -> {} // Required: AgentEvent is non-exhaustive for forward compat } }

4. Tool calling recipe

All tools used with

Agent

originate from Android AppFunctions (

@AppFunctionDeclaration

), registered using the declarative path (

enableAppFunctions(true)

addAppFunctionIds(...)

) or advanced discovery (

AppFunctionAgentTools.discover(...)

When you need to restrict caller access levels for private in-app tools or wrap discovered AppFunctions with human-in-the-loop (HITL) confirmation decorators, you must consult Tools and functions.

Option A: Declarative AppFunctions path

Define tools declaratively in your app using AndroidX AppFunctions:

package com.example.app.tools

import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.AppFunctionSerializable

@AppFunctionSerializable
data class CalendarEvents(val date: String, val events: List<String>)

class CalendarTools {
    /**
     * Fetches scheduled calendar appointments for a specific date in
     * YYYY-MM-DD format.
     *
     * @param date Date string in YYYY-MM-DD format
     */
    @AppFunctionDeclaration(isDescribedByKDoc = true)
    fun fetch_calendar(
        date: String
    ): CalendarEvents {
        return CalendarEvents(
            date = date,
            events = listOf(
                "Team Standup at 10:00 AM",
                "Product Review at 2:00 PM"
            )
        )
    }
}

Enable AppFunctions on

Agent.Builder

and initialize

AgentRunner

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    // Automatically discovers AppFunctions in your app package
    .enableAppFunctions(true)
    // Optional: scope to specific function IDs or method names
    .addAppFunctionIds("fetch_calendar")
    .build()

val runner = AgentRunner(agent)

AppFunction Contract : Tool methods take domain parameters directly (no framework context parameter required). Returns must be primitives or
```
@AppFunctionSerializable
```
data classes (
```
Map<String, Any?>
```
isn't supported for AppFunctions).
Restricting Callers with
ACCESS_LEVEL_SELF
: AppFunctions don't have to be public to external callers. To restrict execution strictly to your app's internal agent (blocking external system assistants and IPC callers), annotate with
```
@OptIn(ExperimentalAppFunctionsApi::class)
```
and
```
@AppFunctionAccessLevel(level = AppFunctionMetadata.ACCESS_LEVEL_SELF, isCompatEnforcementEnabled = true)
```
.

Option B: Advanced discovery and decorator pattern

For inspecting, filtering, or decorating discovered AppFunctions (such as Human-in-the-Loop confirmation wrappers using

AgentTool(delegate)

) before registering with the agent:

import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentTool
import com.google.mlkit.genai.agent.AppFunctionAgentTools

// Discover tools (optionally scoped to function IDs)
val discoveredTools = AppFunctionAgentTools.discover(
    enabledAppFunctionIds = listOf("fetch_calendar")
)

// Optional: Wrap sensitive tools using AgentTool(delegate) decorator
class HitlConfirmationTool(
    delegate: AgentTool,
    private val onConfirm: suspend (
        toolName: String,
        args: Map<String, Any>
    ) -> Boolean
) : AgentTool(delegate) {
    override suspend fun execute(args: Map<String, Any>): Map<String, Any?> {
        if (!onConfirm(name, args)) {
            return mapOf("status" to "CANCELLED_BY_USER")
        }
        return checkNotNull(delegate).execute(args)
    }
}

val agent = Agent.builder()
    .setName("CalendarAssistant")
    .setInstructions("You assist with calendar queries.")
    .addTools(discoveredTools)
    .build()

val runner = AgentRunner(agent)

[!WARNING] Mutual exclusivity : Configure AppFunctions using either the declarative path (
enableAppFunctions(true)
on
Agent.Builder
) or the advanced path (
AppFunctionAgentTools.discover
passed to
addTools
), but not both. Using both simultaneously triggers a warning and registers duplicate tool schemas.

Deprecated annotations (@Tool and @Param)

The legacy

@Tool

and

@Param

annotations in

com.google.mlkit.genai.schema.annotations.*

, along with

AgentTool.fromAnnotatedObject

Agent.Builder.addTools(annotatedObject: Any)

, and standalone

FunctionTool

, have been deprecated or removed. All tools added to

Agent

must originate from

@androidx.appfunctions.AppFunctionDeclaration

using

Agent.Builder.enableAppFunctions(true)

AppFunctionAgentTools.discover

5. Ephemeral single-turn execution

For one-off tasks where conversation history isn't needed (such as summarizing text, formatting data, or answering standalone questions), invoke

runner.runSingleTurnAsync(content)

directly:

import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.textTokens
import com.google.mlkit.genai.prompt.Content
import kotlinx.coroutines.flow.Flow

// Executes a single-turn query ephemerally without session persistence:
fun askQuickQuestion(runner: AgentRunner, question: String): Flow<String> {
    val content = Content.builder().text(question).build()
    return runner.runSingleTurnAsync(content).textTokens()
}

```
runner.runSingleTurnAsync(content)
```
automatically creates and evicts an ephemeral session upon Flow completion.
Cleanup reference :
- ```
session.close
```
  : Releases session resources and cancels coroutines (call in
```
ViewModel.onCleared
```
  ).
- ```
session.delete
```
  or
```
runner.deleteSession(sessionId)
```
  : Permanently deletes the session from the runner.

6. Error handling and GenAiException reference

Runtime failures during on-device execution (such as model unavailability or exceeding maximum turn limits) are emitted or thrown as

GenAiException

import com.google.mlkit.genai.common.GenAiException

Key error codes

```
GenAiException.ErrorCode.MAX_TURNS_EXCEEDED
```
(
```
-112
```
): The agent exceeded
```
maxTurns
```
(default 10) without completing the reasoning loop.
```
GenAiException.ErrorCode.NOT_AVAILABLE
```
(
```
8
```
): The Gemini Nano model or AICore service isn't available on this device.
```
GenAiException.ErrorCode.BUSY
```
(
```
9
```
): System or AICore is busy, or battery or thermal quota has been reached.