scenario-quality-gate

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Scenario Quality Gate

Scenario 质量门禁

Overview

概述

One tool,
asset_quality_gate_run
, scores a finished image asset and returns a
pass
/
warn
/
fail
verdict with 0 to 100 scores and per-dimension lists of
reasons
and
suggestions
: an AI-quality check always, plus brand-brief compliance when the team or project has a brief configured. Image assets only. Quality Gate is an Enterprise add-on: when it is not enabled for the team the call fails cleanly, so detect that, fall back to an
asset_analyze
review (see
scenario-asset-analysis
), and say so. Retrying never clears it.
The tool is catalog-only and write-class: get the schema with
scenario_tools_search
, then run it through
scenario_tool_execute_write
with
{name: "asset_quality_gate_run", parameters: {...}}
, scope ids inside
parameters
(the read executor rejects it by lane and names the right one). Or reconnect with
?toolsets=full
. Connection and scope: see the
scenario
skill. If a sibling skill named here is missing from your available skills, ask the user to install it (
npx skills add scenario-labs/skills --skill <name>
); unattended, proceed from tool schemas and flag the gap.
有一个名为
asset_quality_gate_run
的工具,可对已完成的图像资产进行评分,并返回
pass
/
warn
/
fail
判定结果,附带0到100的分数以及各维度的
reasons
(原因)和
suggestions
(建议)列表:该工具始终会执行AI质量检查,如果团队或项目配置了品牌简报,还会额外进行品牌简报合规性检查。仅支持图像资产。质量门禁是企业版附加功能:如果团队未启用该功能,调用会直接失败,因此需要检测该情况,回退到
asset_analyze
审查(参见
scenario-asset-analysis
)并告知用户。重试无法解决该问题。
该工具仅可通过目录获取,且属于写入类:使用
scenario_tools_search
获取其schema,然后通过
scenario_tool_execute_write
运行,参数为
{name: "asset_quality_gate_run", parameters: {...}}
,作用域ID需放在
parameters
中(读取执行器会按通道拒绝该调用,并告知正确的执行器)。或者使用
?toolsets=full
重新连接。连接与作用域相关内容请参见
scenario
技能。如果你的可用技能中缺少此处提到的同系列技能,请让用户安装(命令:
npx skills add scenario-labs/skills --skill <name>
);在无人值守模式下,可基于工具schema继续操作,并标记缺失的技能。

Quick reference: what a call costs

快速参考:调用费用

CallWhat happensCost
Plain call, usable verdict storedReturns the stored verdict,
source: "stored"
Free (the response carries no
creativeUnitsCost
)
Plain call, no usable verdictRuns a new analysis and stores the verdict,
source: "new_analysis"
Billed; the response carries
creativeUnitsCost
(1 CU as of this writing)
rerun: true
New analysis that replaces the stored verdictBilled
dry_run: true
Nothing runs; returns the price of a new analysis as
creativeUnitsCost
Free
A plain call is a read only when a usable verdict already exists; otherwise the same call silently escalates to the billed analysis. "No usable verdict" covers both never-scored and a stored
error
result. When spend needs approval first, probe free:
dry_run: true
, or
asset_get
, whose
qualityGate
field carries the verdict and scores when one is stored (summary only; the stored
reasons
and
suggestions
come back through the tool, still free).
sensitivity
(
low
,
medium
,
high
; default is the team or project setting) shapes new analyses only and is ignored on stored reads.
rerun: true
is for after the brief or the sensitivity changed, nothing else: it re-bills and overwrites.
调用类型执行操作费用
普通调用,存在可用的已存储判定结果返回已存储的判定结果,
source: "stored"
免费(响应中不包含
creativeUnitsCost
)
普通调用,无可用判定结果执行新的分析并存储判定结果,
source: "new_analysis"
计费;响应中包含
creativeUnitsCost
(截至本文撰写时为1 CU)
rerun: true
执行新分析并覆盖已存储的判定结果计费
dry_run: true
不执行任何操作;以
creativeUnitsCost
形式返回新分析的价格
免费
仅当已存在可用的判定结果时,普通调用才是只读操作;否则,同一调用会静默升级为计费分析。“无可用判定结果”包括从未评分和已存储的
error
结果两种情况。如果需要先审批费用,可免费探测:使用
dry_run: true
,或调用
asset_get
——当存在已存储的判定结果时,其
qualityGate
字段会携带判定结果和分数(仅为摘要;已存储的
reasons
和
suggestions
仍可通过该工具免费获取)。
sensitivity
(可选值:
low
/
medium
/
high
;默认值为团队或项目设置)仅对新分析生效,对已存储结果的读取会忽略该参数。
rerun: true
仅用于简报或sensitivity变更后的场景,别无他用:它会重新计费并覆盖原有结果。

Reading the verdict

解读判定结果

The response is
source
plus
quality_gate
:
verdict
,
overallScore
,
aiQualityScore
,
briefComplianceScore
,
sensitivity
,
appliedBriefIds
, and
details
. With no brief configured the compliance dimension is absent entirely (
appliedBriefIds: []
,
overallScore
equals
aiQualityScore
); with one,
details.briefCompliance
sits beside
details.aiQuality
, each carrying a
score
and lists of
reasons
and
suggestions
, usually several of each.
响应包含
source
和
quality_gate
两部分:
quality_gate
下包含
verdict
、
overallScore
、
aiQualityScore
、
briefComplianceScore
、
sensitivity
、
appliedBriefIds
和
details
。如果未配置简报,则合规维度完全不存在(
appliedBriefIds: []
,
overallScore
等于
aiQualityScore
);如果配置了简报,则
details.briefCompliance
与
details.aiQuality
并列,各自包含
score
以及
reasons
和
suggestions
列表,通常每个列表都有数条内容。

Turning the verdict into a better image

利用判定结果优化图像

A verdict alone only sorts assets. The gate earns its cost when the feedback drives the next attempt:
  • reasons
    name concrete flaws ("elongated finger anatomy on the left hand", "background gradient off the brief's palette"). Use them to pick the fix path per flaw: a local defect on an otherwise approved image is a masked inpainting pass (
    scenario-image
    ); a global one is a regeneration.
  • suggestions
    are written as edit instructions, often worded for manual retouching. Apply all that fit, not just the first: translate them into the next
    model_run
    prompt or parameters, or into the edit instruction.
  • A regenerated image is a new asset, so a plain call scores it.
    rerun
    is never part of the loop. It may even come back pre-scored for free: teams with auto-detect enabled (
    qualityGateAutoDetect: true
    on their
    teams_list
    row) score new generations automatically.
  • When a round repeats the same flaw classes at an unmoved score, rewording will not fix them: switch models (
    recommend
    again) or repair the flaw with a masked edit before spending another round.
  • Fix the exit bar and a round cap up front:
    verdict: "pass"
    by default, or a score target the user names (
    overallScore
    at or above 90, say; the named bar then outranks a bare
    pass
    ), and three rounds unless told otherwise, since every round bills a generation plus an analysis. At the cap, or when a round stops moving the scores, stop: report the best asset with its remaining flaws and ask before spending more rounds; unattended, deliver that best asset and flag the miss.
scenario-refine-loop
carries the loop discipline beyond the gate: one variable per round, regeneration from the approved baseline, and fix routing when the criteria go past the brief.
仅靠判定结果只能对资产进行分类。只有当反馈能推动下一次生成尝试时,质量门禁才物有所值:
  • reasons
    列出具体缺陷(例如“左手手指解剖结构过长”、“背景渐变不符合简报的调色板”)。可根据这些缺陷选择修复路径:如果整体合格的图像存在局部缺陷,可使用蒙版修复(
    scenario-image
    );如果是全局性缺陷,则需要重新生成。
  • suggestions
    以编辑指令的形式呈现,通常是为手动修图而编写的。应应用所有适用的建议,而非仅第一条:将它们转化为下一次
    model_run
    的提示词或参数,或者转化为编辑指令。
  • 重新生成的图像是新的资产,因此需要通过普通调用进行评分。
    rerun
    永远不会出现在循环中。它甚至可能预先完成评分并免费返回:启用了自动检测的团队(其
    teams_list
    行中
    qualityGateAutoDetect: true
    )会自动对新生成的内容进行评分。
  • 如果某一轮迭代后分数没有变化,且缺陷类型与之前相同,那么仅修改提示词无法解决问题:应切换模型(重新调用
    recommend
    ),或者在进行下一轮之前通过蒙版编辑修复缺陷。
  • 提前设定退出标准和轮次上限:默认退出标准为
    verdict: "pass"
    ,或用户指定的分数目标(例如
    overallScore
    达到90分及以上;此时指定的分数标准优先级高于单纯的
    pass
    );除非另有说明,否则轮次上限为3轮,因为每一轮都需要支付生成和分析的费用。达到轮次上限,或某一轮分数不再提升时,应停止迭代:上报最优资产及其剩余缺陷,在继续迭代前征求用户意见;在无人值守模式下,直接交付最优资产并标记未达标的情况。
scenario-refine-loop
提供了门禁之外的循环规范:每轮仅调整一个变量,从已通过的基线开始重新生成,并且当标准超出简报范围时提供修复路径。

What the verdict does not cover

判定结果的局限性

The gate is an artifact detector. Measured on real assets, three limits to design around:
  • Artifacts, not plausibility. A panel whose car carried two different wheels, a whitewall on one only, and no wheel arch above the rear tire scored 92 and
    pass
    at
    sensitivity: "high"
    , with
    reasons
    praising the chrome rims that were the defect. What is impossible rather than ugly needs a structural check of your own: counts and attachment (hands, fingers per hand, limbs, wheels, each attached to one body), part to whole (a wheel sits inside an arch, a guard sits on its own blade, a reflection sits under what casts it), and what carries the weight.
  • It has never seen your reference.
    briefCompliance
    needs a configured brief, and a brief is text, so no call can tell you this is not the same car or costume as the plate it was generated from. Compare the asset against its reference yourself, item by item.
  • A composite averages its parts. A 12-panel storyboard scored 88 and
    pass
    while one of its panels was physically impossible. Score the unit you are willing to reject: split a composite with
    model_scenario-image-slicer
    , score each part, repair only the failures, and recompose with
    model_scenario-compose-image
    .
The score is also a weak signal that a structural repair landed: a broken panel and its corrected replacement both scored 92, and only the
reasons
changed. Re-read the flaw list, do not watch the number.
该门禁是一个伪影检测器。基于真实资产的测试,有三个需要在设计中规避的局限:
  • 仅检测伪影,不验证合理性。 有一张图板中的汽车有两个不同的轮子,只有一个是白壁轮胎,且后轮胎上方没有轮拱,但在
    sensitivity: "high"
    的设置下仍获得了92分和
    pass
    判定,
    reasons
    甚至称赞了作为缺陷的镀铬轮圈。对于不符合逻辑而非不美观的问题,需要你自行进行结构检查:包括数量和连接关系(手、每只手的手指、四肢、轮子,每个都应连接到一个主体上)、部分与整体的关系(轮子应在轮拱内,防护罩应在其自身的叶片上,倒影应在投射倒影的物体下方)以及承重结构是否合理。
  • 它从未见过你的参考图。
    briefCompliance
    需要配置简报,而简报是文本形式的,因此任何调用都无法告诉你这张图和生成它的参考图中的汽车或服装不是同一个。你需要自行逐项对比资产与其参考图。
  • 合成图的分数是各部分的平均值。 一张包含12个图板的故事板获得了88分和
    pass
    判定,但其中一个图板的内容在物理上是不可能存在的。你应该按可接受拒绝的最小单位进行评分:使用
    model_scenario-image-slicer
    分割合成图,对每个部分分别评分,仅修复不合格的部分,然后使用
    model_scenario-compose-image
    重新合成。
分数也不是判断结构修复是否生效的可靠信号:一张有问题的图板和修复后的版本都得了92分,只有
reasons
发生了变化。应重新查看缺陷列表,而非只关注分数。

Worked example: iterate a hero prop to pass

示例:迭代优化主角道具以通过门禁

  1. Generate per
    scenario-image
    :
    model_run
    ,
    jobs_wait
    , collect the asset id.
  2. scenario_tools_search
    with
    query="quality gate"
    once for the schema, then
    dry_run: true
    on the first asset to surface the per-analysis price.
  3. Score:
    scenario_tool_execute_write
    with
    {name: "asset_quality_gate_run", parameters: {asset_id, team_id, project_id}}
    . It returns
    source: "new_analysis"
    and a
    quality_gate
    carrying
    verdict: "warn"
    ,
    briefComplianceScore: 58
    , and
    details.briefCompliance.suggestions
    asking for the logo at the top left and a flatter background.
  4. Fold both suggestions into the prompt, regenerate, and score the new asset with a plain call (no
    rerun
    ).
  5. verdict: "pass"
    : deliver, and file it (collections and tags per
    scenario-asset-analysis
    ). Later reads of any scored asset are free stored reads.
  6. The brief changes next sprint: only then
    rerun: true
    on the assets that must be re-judged.
  1. 按照
    scenario-image
    的流程生成:调用
    model_run
    ,执行
    jobs_wait
    ,获取资产ID。
  2. 调用
    scenario_tools_search
    ,传入
    query="quality gate"
    获取一次schema,然后对第一个资产执行
    dry_run: true
    ,以获取每次分析的价格。
  3. 评分:调用
    scenario_tool_execute_write
    ,参数为
    {name: "asset_quality_gate_run", parameters: {asset_id, team_id, project_id}}
    。它会返回
    source: "new_analysis"
    和
    quality_gate
    对象,其中
    quality_gate
    包含
    verdict: "warn"
    、
    briefComplianceScore: 58
    ,以及
    details.briefCompliance.suggestions
    (要求将logo放在左上角,使用更平整的背景)。
  4. 将这两条建议都融入提示词,重新生成,然后通过普通调用对新资产进行评分(不要使用
    rerun
    )。
  5. 当
    verdict: "pass"
    时:交付资产,并归档(按照
    scenario-asset-analysis
    的方式添加集合和标签)。后续对已评分资产的读取都是免费的存储读取。
  6. 如果下一个冲刺阶段简报发生了变化:只有在这种情况下,才对需要重新判定的资产执行
    rerun: true
    。

Common mistakes

常见错误

  • Treating a plain call as a free read: without a usable stored verdict it silently runs and bills the analysis. Probe with
    dry_run
    or
    asset_get
    first when the spend matters.
  • Passing
    rerun: true
    out of habit: it re-bills verdicts that were free to read. Its one job is refreshing after the brief or sensitivity changed.
  • Expecting
    briefComplianceScore
    with no brief configured:
    quality_gate
    carries it and
    details.briefCompliance
    only when
    appliedBriefIds
    is non-empty.
  • Applying one suggestion and rescoring each time: the lists usually carry several fixes, and one regeneration can absorb them all.
  • Running it through
    scenario_tool_execute_read
    : write-class, rejected by lane.
  • Scoring a video, 3D, or audio asset: image assets only.
  • Assuming a fresh upload has no verdict: uploads deduplicate by content, so identical bytes return the same long-lived asset id whatever the filename, and a re-upload of a file the team scored before carries its stored verdict.
  • Setting
    sensitivity
    on a call that returns a stored verdict: it applies to new analyses only; stricter scoring of an already-scored asset requires
    rerun: true
    .
  • Retrying the entitlement failure: Quality Gate is an Enterprise add-on. Degrade to the
    asset_analyze
    review in
    scenario-asset-analysis
    and tell the user why.
  • 将普通调用视为免费读取:如果没有可用的已存储判定结果,它会静默执行分析并计费。当费用很重要时,请先使用
    dry_run
    或
    asset_get
    进行探测。
  • 习惯性地传入
    rerun: true
    :这会对原本可以免费读取的判定结果重新计费。它的唯一用途是在简报或sensitivity变更后刷新结果。
  • 在未配置简报的时期望获得
    briefComplianceScore
    :只有当
    appliedBriefIds
    非空时,
    quality_gate
    才会包含该字段和
    details.briefCompliance
    。
  • 每次只应用一条建议就重新评分:列表中通常包含多个修复点,一次重新生成就可以全部应用。
  • 通过
    scenario_tool_execute_read
    运行该工具:它属于写入类,会被通道拒绝。
  • 对视频、3D或音频资产评分:该工具仅支持图像资产。
  • 认为新上传的文件没有判定结果:上传会按内容去重,因此无论文件名如何,相同的字节都会返回同一个长期有效的资产ID,重新上传团队之前评过分的文件会保留其已存储的判定结果。
  • 在返回已存储判定结果的调用中设置
    sensitivity
    :该参数仅对新分析生效;要对已评分的资产进行更严格的评分,需要使用
    rerun: true
    。
  • 遇到权限失败时重试:质量门禁是企业版附加功能。应降级使用
    scenario-asset-analysis
    中的
    asset_analyze
    审查,并告知用户原因。