secrets

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Goldsky Secrets Management

Goldsky 密钥管理

Create and manage secrets for pipeline sink credentials.

创建和管理管道sink凭证的密钥。

Agent Instructions

Agent 操作说明

When this skill is invoked, follow this streamlined workflow:

当调用此技能时，请遵循以下简化工作流：

Step 1: Verify Login + List Existing Secrets

步骤1：验证登录 + 列出现有密钥

Run

goldsky secret list

to confirm authentication and show existing secrets.

If authentication fails: Invoke the

auth-setup

skill first.

运行

goldsky secret list

以确认身份验证并显示现有密钥。

如果身份验证失败： 先调用

auth-setup

技能。

Step 2: Determine Intent Quickly

步骤2：快速确定用户意图

Skip unnecessary questions. If the user's intent is clear from context, proceed directly:

User says "create a postgres secret" → Go straight to credential collection
User pastes a connection string → Parse it immediately (see Connection String Parsing)
User mentions a specific provider (Neon, Supabase, etc.) → Use provider-specific guidance

Only use AskUserQuestion if intent is genuinely unclear.

跳过不必要的问题。 如果从上下文可明确用户意图，直接执行：

用户说“create a postgres secret” → 直接进入凭证收集环节
用户粘贴连接字符串 → 立即解析（参见连接字符串解析部分）
用户提及特定提供商（Neon、Supabase等） → 使用提供商专属指引

仅当意图确实不明确时，才使用AskUserQuestion。

Step 3: Connection String Parsing (Preferred for PostgreSQL)

步骤3：连接字符串解析（PostgreSQL首选方式）

If user provides a connection string, parse it directly instead of asking questions.

PostgreSQL connection string format:

postgres://USER:PASSWORD@HOST:PORT/DATABASE?sslmode=require
postgresql://USER:PASSWORD@HOST/DATABASE

Parsing logic:

Extract:
```
user
```
,
```
password
```
,
```
host
```
,
```
port
```
(default 5432),
```
databaseName
```
Construct JSON immediately
Create the secret without further questions

Example - user provides:

postgresql://neondb_owner:abc123@ep-cool-name.us-east-2.aws.neon.tech/neondb?sslmode=require

Create using the connection string directly:

bash

goldsky secret create --name SUGGESTED_NAME

如果用户提供连接字符串，直接解析而非询问问题。

PostgreSQL连接字符串格式：

postgres://USER:PASSWORD@HOST:PORT/DATABASE?sslmode=require
postgresql://USER:PASSWORD@HOST/DATABASE

解析逻辑：

提取：
```
user
```
、
```
password
```
、
```
host
```
、
```
port
```
（默认5432）、
```
databaseName
```
立即构造JSON
无需进一步询问即可创建密钥

示例 - 用户提供：

postgresql://neondb_owner:abc123@ep-cool-name.us-east-2.aws.neon.tech/neondb?sslmode=require

直接使用连接字符串创建：

bash

goldsky secret create --name SUGGESTED_NAME

When prompted, paste the connection string:

当出现提示时，粘贴连接字符串：

postgresql://neondb_owner:abc123@ep-cool-name.us-east-2.aws.neon.tech/neondb?sslmode=require

undefined

undefined

Step 4: Provider-Specific Quick Paths

步骤4：提供商专属快捷流程

Neon:

Connection string format:

postgresql://USER:PASS@ep-XXX.REGION.aws.neon.tech/neondb

Default port: 5432
Common issue: Free tier has 512MB limit - pipelines will fail with "project size limit exceeded"

Supabase:

Connection string format:

postgresql://postgres:PASS@db.PROJECT.supabase.co:5432/postgres

Use the "Connection string" from Project Settings → Database

PlanetScale (MySQL):

Use
```
"protocol": "mysql"
```
and port 3306

Neon：

连接字符串格式：

postgresql://USER:PASS@ep-XXX.REGION.aws.neon.tech/neondb

默认端口：5432
常见问题：免费版有512MB限制 - 管道会因“项目大小超出限制”失败

Supabase：

连接字符串格式：

postgresql://postgres:PASS@db.PROJECT.supabase.co:5432/postgres

从项目设置→数据库中获取“Connection string”

PlanetScale (MySQL)：

使用
```
"protocol": "mysql"
```
和端口3306

Step 5: Create Secret Directly

步骤5：直接创建密钥

Once you have credentials (from parsing or user input), create immediately:

bash

goldsky secret create \
  --name SECRET_NAME \
  --value '{"type":"jdbc","protocol":"postgres",...}' \
  --description "Optional description"

Naming convention:

PROJECT_PROVIDER

(e.g.,

TRADEWATCH_NEON

ANALYTICS_SUPABASE

)

一旦获取到凭证（来自解析或用户输入），立即创建：

bash

goldsky secret create \
  --name SECRET_NAME \
  --value '{"type":"jdbc","protocol":"postgres",...}' \
  --description "可选描述"

命名规范：

PROJECT_PROVIDER

（例如：

TRADEWATCH_NEON

、

ANALYTICS_SUPABASE

）

Step 6: Verify

步骤6：验证

Run

goldsky secret list

to confirm creation.

运行

goldsky secret list

确认创建成功。

Secret JSON Schemas

密钥JSON Schema

JSON schema files are available in the
schemas/
folder. Each file contains the full schema with examples.

Secret Type	Schema File	Type Field	Use Case
PostgreSQL	`postgres.json`	`jdbc`	Database sink
MySQL	`postgres.json`	`jdbc`	Database sink (protocol: mysql)
ClickHouse	`clickhouse.json`	`clickHouse`	Analytics database
Kafka	`kafka.json`	`kafka`	Event streaming
AWS S3	`s3.json`	`s3`	Object storage
ElasticSearch	`elasticsearch.json`	`elasticSearch`	Search engine
DynamoDB	`dynamodb.json`	`dynamodb`	NoSQL database
SQS	`sqs.json`	`sqs`	Message queue
OpenSearch	`opensearch.json`	`opensearch`	Search/analytics
Webhook	`webhook.json`	`httpauth`	HTTP endpoints

Schema location:

schemas/

(relative to this skill's directory)

JSON Schema文件位于
schemas/
文件夹中。每个文件都包含完整的Schema及示例。

密钥类型	Schema文件	Type字段	使用场景
PostgreSQL	`postgres.json`	`jdbc`	数据库sink
MySQL	`postgres.json`	`jdbc`	数据库sink（protocol: mysql）
ClickHouse	`clickhouse.json`	`clickHouse`	分析型数据库
Kafka	`kafka.json`	`kafka`	事件流处理
AWS S3	`s3.json`	`s3`	对象存储
ElasticSearch	`elasticsearch.json`	`elasticSearch`	搜索引擎
DynamoDB	`dynamodb.json`	`dynamodb`	NoSQL数据库
SQS	`sqs.json`	`sqs`	消息队列
OpenSearch	`opensearch.json`	`opensearch`	搜索/分析
Webhook	`webhook.json`	`httpauth`	HTTP端点

Schema位置：

schemas/

（相对于此技能的目录）

Quick Reference Examples

快速参考示例

PostgreSQL — Connection string format:

postgres://username:password@host:port/database

bash

goldsky secret create --name MY_POSTGRES_SECRET

PostgreSQL — 连接字符串格式：

postgres://username:password@host:port/database

bash

goldsky secret create --name MY_POSTGRES_SECRET

The CLI will prompt for the connection string interactively

CLI会交互式提示输入连接字符串


**ClickHouse** — Connection string format:

https://username:password@host:port/database


**Kafka** — JSON format:

```json
{
  "type": "kafka",
  "bootstrapServers": "broker:9092",
  "securityProtocol": "SASL_SSL",
  "saslMechanism": "PLAIN",
  "saslJaasUsername": "user",
  "saslJaasPassword": "pass"
}

S3 — Colon-separated format:

access_key_id:secret_access_key

Or with session token:

access_key_id:secret_access_key:session_token

Webhook:

Note: Turbo pipeline webhook sinks do not support Goldsky's native secrets management. Include auth headers directly in the pipeline YAML
headers:
field instead.


**ClickHouse** — 连接字符串格式：

https://username:password@host:port/database


**Kafka** — JSON格式：

```json
{
  "type": "kafka",
  "bootstrapServers": "broker:9092",
  "securityProtocol": "SASL_SSL",
  "saslMechanism": "PLAIN",
  "saslJaasUsername": "user",
  "saslJaasPassword": "pass"
}

S3 — 冒号分隔格式：

access_key_id:secret_access_key

或包含会话令牌：

access_key_id:secret_access_key:session_token

Webhook：

注意： Turbo管道webhook sink不支持Goldsky的原生密钥管理。请直接在管道YAML的
headers:
字段中包含身份验证头。

Connection String Parser

连接字符串解析器

For PostgreSQL, use the helper script to parse connection strings:

bash

./scripts/parse-connection-string.sh "postgresql://user:pass@host:5432/dbname"

对于PostgreSQL，使用辅助脚本解析连接字符串：

bash

./scripts/parse-connection-string.sh "postgresql://user:pass@host:5432/dbname"

Output: JSON ready for goldsky secret create --value

输出：可直接用于goldsky secret create --value的JSON

undefined

undefined

Step 5: Confirm and Create

步骤5：确认并创建

Show the user what will be created (mask password with ***) and ask for confirmation before running the command.

向用户展示即将创建的内容（密码用***掩码），并在执行命令前请求确认。

Step 6: Verify Success

步骤6：验证创建成功

Run

goldsky secret list

to confirm the secret was created.

运行

goldsky secret list

确认密钥已创建。

Quick Reference

快速参考

Action	Command
Create	`goldsky secret create --name NAME --value "value"`
List	`goldsky secret list`
Reveal	`goldsky secret reveal NAME`
Update	`goldsky secret update NAME --value "new-value"`
Delete	`goldsky secret delete NAME`

操作	命令
创建	`goldsky secret create --name NAME --value "value"`
列出	`goldsky secret list`
查看	`goldsky secret reveal NAME`
更新	`goldsky secret update NAME --value "new-value"`
删除	`goldsky secret delete NAME`

Prerequisites

前提条件

Goldsky CLI installed
Logged in (
```
goldsky login
```
)
Connection credentials for your target sink

已安装Goldsky CLI
已登录（
```
goldsky login
```
）
目标sink的连接凭证

Why Secrets Are Needed

为何需要密钥

Pipelines that write to external sinks (PostgreSQL, ClickHouse, Kafka, S3) need credentials to connect. Instead of putting credentials directly in your pipeline YAML, you store them as secrets and reference them by name.

Benefits:

Credentials are encrypted and stored securely
Pipeline configs can be shared without exposing secrets
Credentials can be rotated without modifying pipelines

写入外部sink（PostgreSQL、ClickHouse、Kafka、S3）的管道需要凭证才能连接。无需将凭证直接放在管道YAML中，而是将其存储为密钥并通过名称引用。

优势：

凭证被加密并安全存储
管道配置可共享且不会暴露密钥
无需修改管道即可轮换凭证

Command Reference

命令参考

Command	Purpose	Key Flags
`goldsky secret create`	Create a new secret	`--name` , `--value` , `--description`
`goldsky secret list`	List all secrets
`goldsky secret reveal <name>`	Show secret value
`goldsky secret update <name>`	Update secret value	`--value` , `--description`
`goldsky secret delete <name>`	Delete a secret	`-f` (force, skip confirmation)

命令	用途	关键参数
`goldsky secret create`	创建新密钥	`--name` , `--value` , `--description`
`goldsky secret list`	列出所有密钥
`goldsky secret reveal <name>`	查看密钥值
`goldsky secret update <name>`	更新密钥值	`--value` , `--description`
`goldsky secret delete <name>`	删除密钥	`-f` （强制，跳过确认）

Common Patterns

常见模式

PostgreSQL Secret

PostgreSQL密钥

bash

goldsky secret create --name PROD_POSTGRES

bash

goldsky secret create --name PROD_POSTGRES

When prompted, provide the connection string:

出现提示时，提供连接字符串：

postgres://admin:secret@db.example.com:5432/mydb


Pipeline usage:

```yaml
sinks:
  output:
    type: postgres
    from: my_source
    schema: public
    table: transfers
    secret_name: PROD_POSTGRES


管道使用方式：

```yaml
sinks:
  output:
    type: postgres
    from: my_source
    schema: public
    table: transfers
    secret_name: PROD_POSTGRES

ClickHouse Secret

ClickHouse密钥

bash

goldsky secret create --name CLICKHOUSE_ANALYTICS

bash

goldsky secret create --name CLICKHOUSE_ANALYTICS

When prompted, provide the connection string:

出现提示时，提供连接字符串：

https://default:secret@abc123.clickhouse.cloud:8443/analytics


Pipeline usage:

```yaml
sinks:
  output:
    type: clickhouse
    from: my_source
    table: events
    secret_name: CLICKHOUSE_ANALYTICS
    primary_key: id


管道使用方式：

```yaml
sinks:
  output:
    type: clickhouse
    from: my_source
    table: events
    secret_name: CLICKHOUSE_ANALYTICS
    primary_key: id

Rotating Credentials

轮换凭证

Update an existing secret without changing pipeline configs:

bash

goldsky secret update MY_POSTGRES_SECRET --value 'postgres://admin:NEW_PASSWORD@db.example.com:5432/mydb'

Active pipelines will pick up the new credentials on their next connection.

无需修改管道配置即可更新现有密钥：

bash

goldsky secret update MY_POSTGRES_SECRET --value 'postgres://admin:NEW_PASSWORD@db.example.com:5432/mydb'

运行中的管道会在下次连接时使用新凭证。

Deleting Unused Secrets

删除未使用的密钥

bash

undefined

bash

undefined

With confirmation prompt

带确认提示

goldsky secret delete OLD_SECRET

Skip confirmation (for scripts)

跳过确认（适用于脚本）

goldsky secret delete OLD_SECRET -f


**Warning:** Deleting a secret that's in use will cause pipeline failures.

goldsky secret delete OLD_SECRET -f


**警告：** 删除正在使用的密钥会导致管道失败。

Secret Naming Conventions

密钥命名规范

Use descriptive, uppercase names with underscores:

Good	Bad
`PROD_POSTGRES_MAIN`	`secret1`
`STAGING_CLICKHOUSE`	`my-secret`
`KAFKA_PROD_CLUSTER`	`postgres`

Include environment and purpose in the name for clarity.

使用描述性的大写名称，以下划线分隔：

规范名称	不规范名称
`PROD_POSTGRES_MAIN`	`secret1`
`STAGING_CLICKHOUSE`	`my-secret`
`KAFKA_PROD_CLUSTER`	`postgres`

名称中包含环境和用途以提升清晰度。

Troubleshooting

故障排除

Error: Secret not found

错误：密钥未找到

Error: Secret 'MY_SECRET' not found

Cause: The secret name doesn't exist or is misspelled.
Fix: Run

goldsky secret list

to see available secrets and check the exact name.

Error: Secret 'MY_SECRET' not found

原因： 密钥名称不存在或拼写错误。
解决： 运行

goldsky secret list

查看可用密钥并检查准确名称。

Error: Secret already exists

错误：密钥已存在

Error: Secret 'MY_SECRET' already exists

Cause: Attempting to create a secret with a name that's already in use.
Fix: Use

goldsky secret update MY_SECRET --value "new-value"

to update, or choose a different name.

Error: Secret 'MY_SECRET' already exists

原因： 尝试创建的密钥名称已被使用。
解决： 使用

goldsky secret update MY_SECRET --value "new-value"

更新，或选择其他名称。

Error: Invalid secret value format

错误：密钥值格式无效

Error: Invalid JSON in secret value

Cause: JSON syntax error in the secret value.
Fix: Validate your JSON before creating the secret:

bash

undefined

Error: Invalid JSON in secret value

原因： 密钥值中的JSON语法错误。
解决： 创建密钥前验证JSON有效性：

bash

undefined

Test JSON validity

测试JSON有效性

echo '{"url":"...","user":"..."}' | jq .

undefined

echo '{"url":"...","user":"..."}' | jq .

undefined

Pipeline fails with "connection refused"

管道因“连接被拒绝”失败

Cause: The credentials in the secret are incorrect or the database is unreachable.
Fix:

Verify credentials work outside Goldsky:
```
psql "postgresql://..."
```
Check the secret value:
```
goldsky secret reveal MY_SECRET
```
Ensure the database allows connections from Goldsky's IP ranges

原因： 密钥中的凭证不正确或数据库无法访问。
解决：

在Goldsky外验证凭证是否可用：
```
psql "postgresql://..."
```
检查密钥值：
```
goldsky secret reveal MY_SECRET
```
确保数据库允许Goldsky的IP范围连接

Pipeline fails with "authentication failed"

管道因“身份验证失败”失败

Cause: Username or password in the secret is incorrect. Fix: Update the secret with correct credentials:

bash

goldsky secret update MY_SECRET --value 'postgres://correct:credentials@host:5432/db'

原因： 密钥中的用户名或密码不正确。 解决： 使用正确的凭证更新密钥：

bash

goldsky secret update MY_SECRET --value 'postgres://correct:credentials@host:5432/db'

Secret value contains special characters

密钥值包含特殊字符

Cause: JSON strings with special characters need proper escaping. Fix: Use proper JSON escaping for special characters in password fields:

Backslash: use
```
\\
```
Double quote: use
```
\"
```
Newline: use
```
\n
```

With the structured JSON format, most special characters in passwords work without URL encoding since the password is a separate field.

原因： 包含特殊字符的JSON字符串需要正确转义。 解决： 对密码字段中的特殊字符使用正确的JSON转义：

反斜杠：使用
```
\\
```
双引号：使用
```
\"
```
换行符：使用
```
\n
```

使用结构化JSON格式时，密码中的大多数特殊字符无需URL编码即可正常工作，因为密码是单独的字段。