Loading...
Loading...
Alibaba Cloud Tablestore Agent Storage Skill. Use for building and managing Tablestore-based knowledge bases with the `tablestore-agent-storage` Python SDK. Capabilities: - Install and configure the `tablestore-agent-storage` SDK - Create, describe and list knowledge bases (with subspace and custom metadata support) - Upload local files or import OSS documents into a knowledge base - Query document status and list documents - Perform hybrid retrieval (dense vector + full-text) with metadata filtering - Set up local directory sync scripts and scheduled tasks for automatic knowledge base updates Triggers: "知识库", "tablestore", "ots", "表格存储", "agent storage", "knowledge base", "向量检索", "文档上传", "文档导入", "知识库同步", "tablestore-agent-storage", "AgentStorageClient"
npx skill4agent add aliyun/alibabacloud-aiops-skills alibabacloud-tablestore-agent-storagetablestore-agent-storagetablestore_agent_storage/tablestore_agent_storage/ots_kb_config.jsontablestore_agent_storage/sync_knowledge_base.pytablestore_agent_storage/.sync_cache.jsontablestore_agent_storage/ots_kb_config.json| Operation | Idempotent |
|---|---|
| Yes |
| Yes |
pip install tablestore-agent-storage==1.0.4pip install tablestore-agent-storage==1.0.4 -i https://pypi.tuna.tsinghua.edu.cn/simple# Try running pyenv rehash manually first (~/.pyenv/shims/.pyenv-shim can be removed safely)
rm -f ~/.pyenv/shims/.pyenv-shim && pyenv rehash
# Then retry pip install
pip install tablestore-agent-storage==1.0.4ots_endpointots_instance_nameimport json
from alibabacloud_credentials.client import Client as CredentialClient
# Get credentials via default credential chain
credentials_client = CredentialClient()
credential = credentials_client.get_credential()
access_key_id = credential.get_access_key_id()
access_key_secret = credential.get_access_key_secret()
sts_token = credential.get_security_token()
# now you can save the credentials into configots_endpointots_instance_nameots_endpointhttp://ots-cn-hangzhou.aliyuncs.comcn-hangzhoutablestore_cli list_instance -r <region_id>tablestore_cli create_instance -n <instance_name> -r <region_id> -d "Auto-created by Agent"tablestore_cli describe_instance -r <region_id> -n <instance_name>"Status": 1export OTS_USER_AGENT=AlibabaCloud-Agent-Skillsots_endpointhttp://ots-<region-id>.aliyuncs.comhttps://<instance-name>.<region-id>.ots.aliyuncs.comtablestore_agent_storage/ots_kb_config.json{
"access_key_id": "",
"access_key_secret": "",
"sts_token": "",
"ots_endpoint": "",
"ots_instance_name": "",
"oss_endpoint": "",
"oss_bucket_name": "",
"knowledge_bases": []
}create_knowledge_baselist_knowledge_basedescribe_knowledge_baseoss_endpointoss_bucket_nameAliyunOTSAccessingOSSRolehttps://ram.console.aliyun.com/authorize?request=%7B%22payloads%22%3A%5B%7B%22missionId%22%3A%22Tablestore.RoleForOTSAccessingOSS%22%7D%5D%2C%22callback%22%3A%22https%3A%2F%2Fotsnext.console.aliyun.com%2F%22%2C%22referrer%22%3A%22Tablestore%22%7Daccess_key_idaccess_key_secretsts_tokenlocal_pathoss_sync_pathsync_interval_minutesinclusion_filters["*.pdf", "*.docx", "*.txt", "*.md", "*.html"]tablestore_agent_storage/sync_knowledge_base.pyadd_documents.sync_cache.jsonopenclaw cron add --name "kb-sync" --every 5m --message "Please run the knowledge base sync script: cd /your/project && python3 tablestore_agent_storage/sync_knowledge_base.py"import json
from tablestore_agent_storage import AgentStorageClient
config = json.load(open("tablestore_agent_storage/ots_kb_config.json", "r"))
client = AgentStorageClient(
access_key_id=config["access_key_id"],
access_key_secret=config["access_key_secret"],
sts_token=config.get("sts_token"), # STS temporary credential, optional
ots_endpoint=config["ots_endpoint"],
ots_instance_name=config["ots_instance_name"]
)client = AgentStorageClient(
access_key_id=config["access_key_id"],
access_key_secret=config["access_key_secret"],
sts_token=config.get("sts_token"),
oss_endpoint=config["oss_endpoint"], # Must be in the same region as OTS
oss_bucket_name=config["oss_bucket_name"],
ots_endpoint=config["ots_endpoint"],
ots_instance_name=config["ots_instance_name"]
)subspace"subspace": truesubspacesubspacesubspace_defaultclient.create_knowledge_base({
"knowledgeBaseName": "my_kb",
"description": "My knowledge base"
})metadataMetadataFieldMetadataFieldTypeEmbeddingConfigurationclient.create_knowledge_base({
"knowledgeBaseName": "my_kb",
"subspace": True,
"metadata": [
{"name": "author", "type": "string"},
{"name": "version", "type": "long"}
]
})# List all knowledge bases (supports pagination)
client.list_knowledge_base({"maxResults": 20, "nextToken": ""})
# View details of a single knowledge base
client.describe_knowledge_base({"knowledgeBaseName": "my_kb"})# Upload a single file to the default subspace
client.upload_documents({
"knowledgeBaseName": "my_kb",
"documents": [
{"filePath": "/path/to/file.pdf"},
{"filePath": "/path/to/doc.docx", "metadata": {"author": "aliyun"}}
]
})
# Upload to a specific subspace
client.upload_documents({
"knowledgeBaseName": "my_kb",
"subspace": "finance",
"documents": [
{"filePath": "/path/to/report.pdf", "metadata": {"version": 2}}
]
})# Import a single file
client.add_documents({
"knowledgeBaseName": "my_kb",
"documents": [
{"ossKey": "oss://your-bucket/docs/file.pdf"}
]
})
# Import an OSS directory (supports file type filtering)
client.add_documents({
"knowledgeBaseName": "my_kb",
"subspace": "tech_docs",
"documents": [
{
"ossKey": "oss://your-bucket/synced-folder/",
"inclusionFilters": ["*.pdf", "*.docx", "*.md"],
"exclusionFilters": ["*draft*"],
"metadata": {"source": "oss_sync"}
}
]
})# Query by docId
client.get_document({
"knowledgeBaseName": "my_kb",
"docId": "your_doc_id"
})
# Query by ossKey
client.get_document({
"knowledgeBaseName": "my_kb",
"ossKey": "oss://your-bucket/docs/file.pdf",
"subspace": "tech_docs"
})pendingcompletedfailed# List all documents in a knowledge base (supports pagination)
client.list_documents({
"knowledgeBaseName": "my_kb",
"maxResults": 20,
"nextToken": ""
})
# List documents in specific subspaces
client.list_documents({
"knowledgeBaseName": "my_kb",
"subspace": ["finance", "tech_docs"],
"maxResults": 50
})client.retrieve({
"knowledgeBaseName": "my_kb",
"retrievalQuery": {
"text": "your question",
"type": "TEXT"
},
"retrievalConfiguration": {
"searchType": ["DENSE_VECTOR", "FULL_TEXT"],
"denseVectorSearchConfiguration": {"numberOfResults": 10},
"fullTextSearchConfiguration": {"numberOfResults": 10},
"rerankingConfiguration": {
"type": "RRF",
"numberOfResults": 10,
"rrfConfiguration": {
"denseVectorSearchWeight": 1.0,
"fullTextSearchWeight": 1.0,
"k": 60
}
}
}
})client.retrieve({
"knowledgeBaseName": "my_kb",
"retrievalQuery": {"text": "your question", "type": "TEXT"},
"retrievalConfiguration": {
"searchType": ["DENSE_VECTOR"],
"denseVectorSearchConfiguration": {"numberOfResults": 10}
}
})MetadataFilterfilterLet me first check your basic environment. Please confirm:
- Is Python 3.8 or higher available in your current environment?
- May I install
?tablestore-agent-storage
Credentials require the following three pieces of information:
2.access_key_id3.access_key_secret(optional)sts_token
Note: You may ask the user how to obtain credentials (e.g., where the credentials config file is located), but you must never display them directly, nor ask the user for plaintext AK/SK.
Two more OTS configuration items are needed:
2.ots_endpointots_instance_name
Note: Theformat must beots_endpoint, nothttp://ots-<region-id>.aliyuncs.com.https://<instance-name>.<region-id>.ots.aliyuncs.com
Please confirm:
- Do you want to create a new knowledge base, or use an existing one?
- What is the knowledge base name?
After basic configuration is complete, do you also need:
- Upload local files
- Link a local directory with automatic sync
If you need local file upload or automatic sync, please provide:
2.oss_endpointoss_bucket_name
Please provide directory sync information:
- Local directory path
2. OSS sync path prefixlocal_pathoss_sync_path- Sync interval (minutes, default: 5) 4. File type filter (default:
)*.pdf, *.docx, *.txt, *.md, *.html