Local Knowledge Base Retrieval Skill (kb-retriever)

Knowledge Base Directory Description

• The knowledge base is stored in a root directory, containing multiple file types (such as

  .md

  /

  .txt

  ,

  .pdf

  ,

  .xlsx

  , etc.), usually split into multi-level subdirectories by type or business purpose.
• Adopt hierarchical directory index files:
  • The root directory has a

    data_structure.md

    that describes the main "domain directories" and their purposes.
  • Each domain directory can have its own

    data_structure.md

    , explaining which subdirectories/files are under this directory and their respective purposes.
  • Deeper subdirectories can also have

    data_structure.md

    , forming a multi-level index tree.
• Knowledge base root directory conventions:
  • By default, the knowledge base is located in the

    knowledge/

    directory under the current project root.
  • If the user explicitly specifies another path in the conversation (e.g., "My knowledge base is in /data/kb" or "Use ./docs as the knowledge base directory"), use the user-specified path as the root directory.
  • When the default path

    knowledge/

    does not exist or access fails, confirm the actual knowledge base root directory location with the user instead of guessing randomly.
• Individual business files may be large:
  • Do not directly read the entire file using Read
  • For PDF and Excel files, perform structured processing using corresponding Skills first, then conduct precise retrieval combined with grep/local reading

Locate the

knowledge

Root Directory

• Prioritize user-specified root directory: If the user provides a path (such as

  ./docs

  ,

  ./knowledge-personal

  ), use the user-provided path directly.
• Default root directory: Otherwise, the root directory is约定为

  knowledge/

  under the current project.
  • Explicitly check if the directory exists using shell commands: Prefer

    test -d knowledge

    , or use

    ls -d knowledge

    as a fallback.
  • Note: Prohibit using patterns like

    Glob "knowledge" in .

    to determine directory existence.

    Glob

    only returns file paths, not directories themselves, and an empty result cannot distinguish between "directory does not exist" and "directory exists but is empty".
• Only use Glob to retrieve content under the directory when the root directory is confirmed to exist via

  test -d

  or similar methods, and specify the directory as

  path

  , for example:
  • Index files:

    pattern="**/data_structure.md"

    ,

    path="knowledge"

  • All Markdown files:

    pattern="**/*.md"

    ,

    path="knowledge"

• If the default

  knowledge/

  does not exist (failed

  test -d

  ): Do not guess other directories, clearly inform the user that the default root directory was not found, and ask the user to specify the actual knowledge base path.

Key Principle: Learn First, Then Process

• ✅ Have read the corresponding references document to learn processing methods
• ✅ Have understood the recommended tools and commands
• ✅ Have completed file processing (extraction/conversion)
• ⏭️ Now you can start retrieval

• ❌ Attempt to process PDF directly without reading pdf_reading.md
• ❌ Attempt to process Excel directly without reading excel_reading.md
• ❌ Skip file processing steps and directly retrieve from original PDF/Excel files

Overall Process

1. Understand User Requirements
   • Read the user's question and extract:
     • Topic/domain keywords (e.g., "sales report", "system architecture", "interface documentation")
     • Time or scope constraints (e.g., "Q1 2023", "latest version")
     • Required output type (explanation, summary, specific field values, etc.)
   • Determine the knowledge base root directory:
     • First check if the user specified a knowledge base path in the question.
     • Otherwise, use the default root directory

       knowledge/

       .
     • If the default root directory does not exist or has an abnormal structure, ask the user for confirmation instead of making assumptions.
2. View Directory Index

   data_structure.md

   Hierarchically
   • Use the concept of a "current working directory":
     • Start from the user-specified knowledge base root directory by default; if not specified, use the current directory.
   • If

     data_structure.md

     exists in the current working directory:
     • Use Read to read the first few lines (e.g., limit=300), and read in segments if necessary.
     • Objectives:
       • Understand which subdirectories and files are under the current directory
       • Understand the purpose description of each subdirectory/file
     • Based on the user's question, select the most relevant subdirectories or files to form a candidate set.
   • For candidate subdirectories:
     • Recursively enter the subdirectory, set it as the new "current working directory", continue to find the

       data_structure.md

       inside and repeat the above process.
     • During recursion, avoid diving into all branches at once, prioritize drilling down along the path most relevant to the question.
   • For candidate business files (md/text, PDF, Excel, etc.):
     • After completing the necessary directory level exploration, collect these files as the final retrieval target list.
   • When prioritizing:
     • Prioritize domain directories and files whose purpose descriptions highly match the question topic
     • Secondly consider constraints such as time/version (if reflected in the index)
     • General explanatory documents (such as README.md, overall design documents) are given lower priority
3. Learn File Processing Methods (Mandatory When Encountering PDF/Excel)
   • Before processing PDF files:
     • Must first read references/pdf_reading.md (note this directory is under the Skills directory, not the Knowledge directory) to learn extraction methods
     • Focus on understanding: pdftotext command, pdfplumber usage, table extraction methods
   • Before processing Excel files:
     • Must first read references/excel_reading.md to learn reading methods
     • Must first read references/excel_analysis.md to learn analysis methods
     • Focus on understanding: pandas reading, column filtering, data filtering
   • Purpose: Ensure correct tools and methods are used, avoid blind retrieval
4. Execute Processing and Retrieval by File Type
   • Process files using the newly learned methods (extraction, conversion, structuring)
   • For each type of candidate file, execute the strategy below for "Markdown/Text", "PDF", "Excel"
   • General principles:
     • Start with the most relevant and precise files first
     • Perform progressive local retrieval within each file, avoid loading the entire content at once
     • Switch to the next candidate file if satisfactory information cannot be obtained from the current file
5. Iterative Retrieval
   • All file types use a unified "multi-round iterative retrieval mechanism" (see Public Retrieval Principles above)
6. Answer Organization and Traceability
   • Summarize the context obtained from multiple rounds of retrieval and comprehensively answer the user's question.
   • Try to:
     • Provide clear and direct answers
     • Indicate the file names used (include approximate locations if necessary, such as chapters or approximate line numbers/page numbers)
   • If the answer is based on inference or incomplete information:
     • Clearly mark assumptions and uncertainties
     • Prompt the user to supplement more specific file ranges or keywords

Public Retrieval Principles

Keyword Selection Strategy

• Extract 3-8 keywords from the user's question (including possible English abbreviations, synonyms, hypernyms/hyponyms)
• Can combine phrases (e.g., "sales report", "API interface timeout")
• Include business terms, technical terminology, common abbreviations if necessary (e.g., "uv", "pv", "GMV")

Basic grep Retrieval Principles

• Always specify as precise include and path as possible, avoid searching the entire directory
• Prioritize trying core nouns and terms from the question as patterns, then try synonyms
• For each hit, only read the local area near the match (several lines above and below)
• Save "file name + location information + text snippet"

Multi-round Iterative Retrieval Mechanism (Max 5 Times)

1. Iteration Control
   • Maintain a "number of retrieval attempts" count, maximum 5 times
   • Increment the count after each retrieval
2. Per Iteration Process
   1. Generate/update retrieval keywords based on the question (can include synonyms, extended words)
   2. Select files or file parts that have not been fully retrieved
   3. Execute retrieval (grep/local reading/dedicated Skill call)
   4. Analyze the obtained context snippets
   5. Judge whether the information is sufficient to answer the question
3. Termination Conditions
   • Found sufficient context to support the answer; or
   • Reached 5 attempts without finding suitable information
4. Handling Insufficient Information
   • Clearly inform the user that information is missing or may not be in the current knowledge base
   • Provide the closest information found and explain the uncertainty
   • Prompt the user how to narrow down the scope (more specific file names, keywords, time ranges, etc.)

Notes

• Prohibit directly calling

  Glob "knowledge" in .

  or any call attempting to determine directory existence using Glob for the first time. Directory existence should be checked via shell commands (such as

  test -d

  ).
• When using this Skill to query the knowledge base, prohibit using other tools such as web search to obtain knowledge

Specific Strategies for Different File Types

1. Markdown / Text Files (.md, .txt, .log, etc.)

1. Candidate File Selection
   • Judge relevance based on

     data_structure.md

     , file names and paths
   • Prioritize retrieving title and directory files (such as summary documents, design overviews)
2. grep Positioning and Local Reading
   • Use the Grep tool for specified candidate files, limit specific suffixes with include (e.g., "*.md")
   • For files with matches, use Read to only read the local area near the match:
     • Control reading via line number offset and limit (e.g., read dozens of lines before and after the matching line)
     • Avoid reading the entire file
3. Special Handling
   • If the content is only a directory/title, continue to locate and dive deeper based on links or section names
   • Apply the "multi-round iterative retrieval mechanism" (see Public Retrieval Principles above)

2. PDF File Retrieval Strategy

1. First: Read Processing Method Guide
   • Before processing any PDF, must first read references/pdf_reading.md (note this directory is under the Skills directory, not the Knowledge directory)
   • Focus on understanding: pdftotext command, pdfplumber usage, table extraction methods, quick decision table
2. Select Candidate PDFs
   • Select the most relevant 1-3 files based on descriptions in

     data_structure.md

   • If the user specifies a specific PDF file, prioritize using that file
3. Extract Text Using Learned Methods
   • Use tools recommended in pdf_reading.md (prefer pdftotext or pdfplumber)
   • Important: Use

     pdftotext input.pdf output.txt

     to extract text to a file, do not output directly to stdout (avoid occupying a large number of tokens)
   • If table extraction is needed, use pdfplumber's table extraction function
4. Execute Retrieval on Extracted Results
   • Use grep to perform keyword search on the extracted text
   • For each hit, extract the context around the hit (dozens of lines or adjacent pages)
   • Save "file name + page number/approximate location + text snippet"
   • Apply the "multi-round iterative retrieval mechanism" (see Public Retrieval Principles above)

3. Excel File Retrieval Strategy

1. First: Read Processing Method Guides
   • Before processing any Excel, must first read:
     • references/excel_reading.md - Learn how to read worksheets (note this directory is under the Skills directory, not the Knowledge directory)
     • references/excel_analysis.md - Learn how to analyze data (note this directory is under the Skills directory, not the Knowledge directory)
   • Focus on understanding: pandas reading methods, column filtering, data filtering, aggregation operations
2. Select Candidate Excel Files
   • Select the most relevant sheets based on

     data_structure.md

     and workbook/worksheet names
   • Prioritize workbooks/worksheets containing keywords such as "report", "statistics", "log", "configuration", "mapping"
   • If the user specifies a specific Excel file, prioritize using that file
3. Explore Structure Using Learned Methods
   • Use pandas to read the first 10-50 rows (use the

     nrows

     parameter to limit)
   • Focus on mastering: column names/field names, data types (numeric, date, text), key fields
   • Compare column names with the user's question to identify potential key fields (e.g., "revenue", "sales", "error_code", etc.)
4. Execute Data Retrieval and Analysis
   • Use learned pandas methods for filtering and aggregation (e.g.,

     df[df['column'] == value]

     )
   • Only read data near matching rows each time, avoid reading the entire table at once
   • If the question includes a time range, add time filtering to the retrieval
   • Apply the "multi-round iterative retrieval mechanism" (see Public Retrieval Principles above)

Collaboration with Other Tools

PDF Processing

• Must first read references/pdf_reading.md to learn processing methods before handling PDF
• Use pdfplumber/pypdf for text extraction, table extraction, metadata reading
• Prefer the pdftotext command-line tool for fast text extraction

Excel Processing

• Must first read before handling Excel:
  • references/excel_reading.md - Learn reading methods
  • references/excel_analysis.md - Learn analysis methods
• Use pandas for data exploration, preview, filtering and analysis

Tool Usage Principles

• Grep: Used to find line numbers and matching snippets by keyword in specified files, always specify as precise include and path as possible
• Read: Only used for local file reading, always set a reasonable limit (e.g., 200-500 lines) and appropriate offset
• For any potentially large file:
  • Prohibit reading from start to end directly
  • Always narrow down the scope first via index, directory, keywords, etc., then read

Answer Style and Error Handling

• Answer Style
  • Try to answer in the language used by the user (Chinese/English).
  • Give the conclusion first, then brief basis.
  • If needed, list the referenced files and approximate locations at the end, for example:
    • Source: design/api_gateway.md near line 100
    • Source: reports/2023_Q1_sales.xlsx Summary worksheet
• When Information is Missing or Uncertain
  • Clearly state that no fully matching information was found in the current knowledge base or only partial answers can be provided.
  • Do not fabricate facts.
  • Prompt the user how to help narrow down the scope:
    • Specify more specific directories/files
    • Provide more precise keywords or field names
    • Specify time/version ranges