Each of these has burned real time. Each looks reasonable from the inside
while it is happening.
The unmeasured overnight launch. A job goes up at the end of the day
because the day is ending, on the theory that the night is free anyway.
Morning finds it a small fraction done, dead, or wedged — and the night was
not free: it cost the diagnosis time, the cleanup, and a day of schedule
built on the assumption it would finish. The launch decision was made by the
calendar, not by a measurement.
The intuition ETA. "About an hour, probably." From a person this is
optimism; from a model it is fluent confabulation with the grammar of a
measurement. The tell is that no log exists from which the number could have
been derived. The question that kills it is three words: measured from what?
The startup-transient fit. A rate fit from the first items either
includes one-time warm-up costs — projecting the run as far grimmer than it
is, so a feasible job gets canceled — or is taken after caches are warm and
omits a cost the full run pays over and over. Both are cured by fitting the
steady-state window and saying which window was used.
Linear extrapolation of a superlinear cost. Every item slightly costlier
than the last: growing intermediate expressions, an accumulating store,
memory pressure building toward swap. The pilot's rate is the best rate the
run will ever see, the projection is a lower bound presented as a forecast,
and the wall arrives hours after the promised finish. The two-size pilot with
a fitted exponent exists for exactly this.
Grinding past the rethink. The most expensive class in the catalog. A
route measured to be long gets ground through anyway, because grinding feels
like progress and rethinking feels like starting over. A day goes into
babysitting the run — restarts, memory tweaks, partial salvage — while a
reformulation that would remove most of the work sits unexamined. When the
reformulation is finally tried, it finishes before the grind would have. The
threshold rule exists to force this comparison while it is still cheap.
The bigger-machine reflex. The projection is bad, so the plan becomes
more cores. Parallelism pays at most the width factor, and only when the
problem splits cleanly; a restructure removes the work itself, which no
amount of width can do. Hardware comes after the structure is right, not
instead of it.
The rate collapse ignored. Mid-run the rate drops severalfold — the hard
cases arrive, memory tightens, another job lands on the same machine — but
the original ETA keeps being quoted because it is written down and quoting it
is easier than re-measuring. Every "almost done" derived from the dead rate
is fiction, and every plan built on it inherits the fiction.
Log lines mistaken for progress. The job prints steadily, therefore it is
working — except the printing is a heartbeat, a retry loop, or a progress
indicator over a phase that finished long ago, and the output file has not
grown in hours. Liveness is measured on the output class the run exists to
produce; everything else is the sound of a fan.
Sunk-cost continuation. "It has been running too long to kill now." The
hours already spent are not an argument for spending the rest; they are
spent either way. The only live comparison is the measured remaining time of
this run against the total time of the restructured route — and the
restructured route often wins even starting from zero.
The unfaithful pilot. The pilot ran with different flags, lower
precision, warm caches, or the easy slice of the input, so the projection
describes a different computation. The sharpest variant: case difficulty
grows with index, and the pilot sampled the easy head of the list. The pilot
is the run, scaled down, with nothing else changed — and drawn from the same
distribution of cases as the real thing.
这些情况都曾浪费真实时间。在其发生过程中,从内部看每一个似乎都很合理。
未经测量的隔夜启动。 某项任务在一天即将结束时启动,理由是反正夜晚是免费的。第二天早晨会发现,它只完成了一小部分、已经终止,或者卡死不动——而夜晚并不免费:它还消耗了诊断时间、清理工作,以及基于其会完成这一假设而安排的一天日程。启动决策依据的是日历,而不是测量结果。
直觉 ETA。 “大概一个小时吧。” 对人而言,这只是乐观;对模型而言,这是用测量结果般的语法进行的流畅编造。其破绽在于不存在可以推导出该数字的日志。只需问一句就能终结它:“根据什么测得?”
使用启动瞬态拟合。 使用最初若干条目拟合速率,要么把一次性预热成本包含进去——将运行预测得比实际更糟,于是取消本来可行的任务;要么在 cache 已经预热后才取样,漏掉完整运行会反复支付的成本。两种问题都可通过拟合稳态窗口并明确说明所用窗口来解决。
对超线性成本进行线性外推。 每个条目都比前一个略微更昂贵:中间表达式不断增长、存储持续累积、内存压力逐步逼近 swap。试运行期间的速率是完整运行所能达到的最佳速率,因此预测实际上是一个被包装成预测的下限,而“撞墙”会在承诺完成时间过去几小时后才真正发生。采用两种规模试运行并拟合指数,正是为了解决这一问题。
越过重新思考的节点仍死磕。 这是所有问题中代价最高的一类。一条已测得耗时长路径仍被执行到底,因为死磕感觉像在取得进展,而重新思考感觉像从头开始。一天时间都花在盯着任务运行上——重启、调整内存、抢救部分成果——同时,一个本可消除大部分工作的重新表述却无人审视。当它最终被尝试时,会在原本那条死磕路线完成之前就结束。阈值规则的存在,就是为了迫使这一比较在成本仍然较低时发生。
** Bigger-machine 本能。** 预测结果不理想,于是计划就变成增加更多核心。并行带来的收益至多等于宽度因子的收益,而且只有在问题能够干净拆分时才成立;重构则会消除工作本身,这是任何并行宽度都无法做到的。应在结构正确之后再考虑硬件,而不是用硬件代替结构优化。
忽视速率骤降。 运行中途速率下降数倍——困难案例出现、内存变得紧张、另一个任务落到同一台机器上——但旧 ETA 仍被反复引用,因为它已经写下来了,而引用它比重新测量更容易。每一个基于失效速率得出的“快完成了”都是虚构的,每一个建立在此基础上的计划也继承了这种虚构。
把日志行误认为进度。 任务持续打印,因此它正在工作——但这些打印可能只是心跳、重试循环,或针对一个早已完成的阶段的进度指示器,而输出文件数小时都没有增长。存活状态应根据该次运行本应产出的输出来衡量;其他一切不过是风扇的声音。
沉没成本驱动下的继续运行。 “它已经跑了这么久,现在终止太可惜。” 已经花掉的时间并不是继续投入的论据;无论如何都已经花掉了。唯一仍具现实意义的比较,是当前运行实测的剩余时间与重构路径的总耗时——即使从零开始,重构路径也往往胜出。
失真的试运行。 试运行使用了不同的 flags、更低的精度、预热的 cache,或只处理输入中容易处理的那一部分,因此预测描述的其实是另一种计算。最容易误导人的变体是:案例难度随索引增大,而试运行只抽取了列表开头较容易的部分。试运行就是将真实运行按比例缩小,除此之外什么也不改变,并且必须抽取自与真实任务相同的案例分布。