图只能用一次
主线位置：训练主线的 backward 一步：同一个前向上第二次调
backward()的时候。
版本：PyTorch 2.15.0（CPU）、burn 0.22.1（Flex 设备）
stateDiagram-v2 state "图已记录" as Recorded state "第一次反向后" as Consumed state "PyTorch 成功" as PtOk state "PyTorch 报错" as PtErr state "burn panic" as BurnPanic state "保留的图" as Kept [*] --> Recorded: forward Recorded --> Consumed: backward Consumed --> PtOk: 无保存张量 Consumed --> PtErr: 张量已释放 Consumed --> BurnPanic: 一律 Recorded --> Kept: retain_graph Kept --> Consumed: 再 backward
「保留的图」只有 PyTorch 有。实验代码：PyTorch 侧 rgraph.py，burn 侧 crate autodiff-exp 的 bin rgraph（feature 同模板，debug）。平台 M2 Max、macOS 27、rustc 1.99.0。
同一个 loss 两次 backward
x = torch.tensor([1.0, 2.0], requires_grad=True)
loss = (x.exp() * x).sum()
loss.backward()
attempt("same loss, second backward", loss.backward)
let ad = Device::flex().autodiff();
let x = Tensor::<1>::from_floats([1.0, 2.0], &ad).require_grad();
let loss = (x.clone().exp() * x.clone()).sum();
let _ = loss.backward();
attempt("same loss, second backward", || loss.backward());
same loss, second backward: RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
thread 'main' (8558871) panicked at .../burn-autodiff-0.22.1/src/runtime/server.rs:74:17:
Cannot run backward: graph tape has already been consumed at NodeId(3). Burn does not support retain_graph. Combine losses before backward or recompute the forward pass. Use detach() only to intentionally sever gradients through the previous computation.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
same loss, second backward: panicked
两边都拒绝，理由不一样。PyTorch 的信息说的是「保存的中间值已经释放」：图的结构还在，第一次 backward 把反向要用的张量（这里是 exp 的输出和 x）放掉了。burn 的信息说的是「tape 已经被消耗」：AutodiffServer::backward 把沿途每个非叶子节点的反向步骤从图里移出来执行，执行完步骤就没了，连同它保存的张量。NodeId(3) 就是 loss 自己：节点 id 按创建顺序从 0 数，x、exp、乘法之后是 sum。机制和这条信息的来历（0.22 之前第二次 backward 会悄悄返回不完整的梯度）见 burn 卷的 backward 与 Gradients。
PyTorch 这时的修法是 retain_graph=True：
loss = (x.exp() * x).sum()
loss.backward(retain_graph=True)
loss.backward()
print("retain_graph twice: x.grad =", x.grad)
retain_graph twice: x.grad = tensor([10.8731, 44.3343])
两次的梯度累加进 .grad，是 2·(eˣ + x·eˣ)。burn 的 backward 没有参数，0.22.1 里也没有别的办法留住步骤：Tensor::backward 的文档写着「Burn does not support retaining the graph for another backward (retain_graph).」，接着给了两种替代：把共享中间结果的 loss 先合并成一个再 backward，或者每次 backward 重算前向。
问： backward(&self) 只借用了 loss，loss 还在手里，为什么不能再用？
答： 被消耗的不是 loss 这个句柄，是它背后图里的步骤。步骤存在进程级的图结构里，按节点 id 查找，所有 clone() 出来的句柄共享同一份；文档说「Clones share those steps; cloning a tensor does not preserve its backward tape.」。所以先 let l2 = loss.clone(); 再 backward 也没用，l2.backward() 一样 panic。
两个 loss 共享中间结果
实际代码里很少对同一个 loss 调两次 backward，更常见的是一个前向出两个损失项，分别 backward：
h = x.exp()
l1, l2 = h.sum(), (h * 2).sum()
l1.backward()
attempt("l2 after l1 (shared exp)", l2.backward)
let h = x.clone().exp();
let (l1, l2) = (h.clone().sum(), (h.clone() * 2.0).sum());
let g1 = l1.backward();
println!("l1: x.grad = {:?}", v(x.grad(&g1)));
attempt("l2 after l1 (shared exp)", || l2.backward());
l2 after l1 (shared exp): RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
l1: x.grad = Some([2.7182817, 7.389056])
thread 'main' (8558871) panicked at .../burn-autodiff-0.22.1/src/runtime/server.rs:74:17:
Cannot run backward: graph tape has already been consumed at NodeId(10). Burn does not support retain_graph. Combine losses before backward or recompute the forward pass. Use detach() only to intentionally sever gradients through the previous computation.
l2 after l1 (shared exp): panicked
这次 l2 自己的步骤（乘 2、求和）都在，缺的是两者共用的 exp（NodeId(10)）。PyTorch 这里同样报错：exp 保存的输出被 l1 的 backward 放掉了。
两边都对的写法是先把 loss 加起来，一次 backward：
h = x.exp()
l1, l2 = h.sum(), (h * 2).sum()
l1.backward(retain_graph=True)
l2.backward()
print("l1 retain + l2: x.grad =", x.grad.tolist())
x.grad = None
h = x.exp()
(h.sum() + (h * 2).sum()).backward()
print("(l1 + l2).backward: x.grad =", x.grad.tolist())
let h = x.clone().exp();
let grads = (h.clone().sum() + (h * 2.0).sum()).backward();
println!("(l1 + l2).backward: x.grad = {:?}", v(x.grad(&grads)));
l1 retain + l2: x.grad = [8.154845237731934, 22.16716766357422]
(l1 + l2).backward: x.grad = [8.154845237731934, 22.16716766357422]
(l1 + l2).backward: x.grad = Some([8.154845, 22.167168])
梯度是 3eˣ。PyTorch 的 tolist() 把 f32 转成 Python 的 double 打印，burn 打印的是 f32 的最短表示；把 PyTorch 的两个数转回 numpy.float32 和 burn 的值比较，逐位相同，最大差 0。PyTorch 里 retain_graph 加两次 backward 和合并 loss 结果一样，所以从 PyTorch 搬代码时，凡是 retain_graph=True 只是为了「几个 loss 各自 backward、梯度在 .grad 里加起来」，都可以改成先相加。合并还少走一遍共享部分的反向。
需要分别看每一项的大小时，记日志用值（l1.clone().into_scalar()），不要分别 backward。把前向结果缓存在循环外、每步拿它算 loss，也是同一个错误，修法见 graph tape has already been consumed。
panic 之后还能做什么
burn 先只读地检查从根能走到的所有步骤，有一个缺了就 panic，这时一个步骤也没消耗（AutodiffServer::backward 里的注释：「Reject invalid ancestry before removing even the root so fresh branches survive the rejection.」）。用 catch_unwind 接住 panic 以后，看看哪些还能用：
let h = x.clone().exp();
let (l1, l2) = (h.clone().sum(), (h.clone() * 2.0).sum());
let l3 = (h * 3.0).sum();
let _ = l1.backward();
attempt("l2 after l1", || l2.backward());
let fresh = (x.clone() * 4.0).sum();
println!("fresh forward after the panic: x.grad = {:?}", v(x.grad(&fresh.backward())));
attempt("l3 after l1", || l3.backward());
thread 'main' (8558871) panicked at .../burn-autodiff-0.22.1/src/runtime/server.rs:74:17:
Cannot run backward: graph tape has already been consumed at NodeId(22). Burn does not support retain_graph. Combine losses before backward or recompute the forward pass. Use detach() only to intentionally sever gradients through the previous computation.
l2 after l1: panicked
fresh forward after the panic: x.grad = Some([4.0, 4.0])
thread 'main' (8558871) panicked at .../burn-autodiff-0.22.1/src/runtime/server.rs:74:17:
Cannot run backward: graph tape has already been consumed at NodeId(24). Burn does not support retain_graph. Combine losses before backward or recompute the forward pass. Use detach() only to intentionally sever gradients through the previous computation.
l3 after l1: panicked
- 这回报出的
NodeId(22)、NodeId(24)按创建顺序数是l2、l3自己的sum，不是exp：h已经移进了l3，没有句柄引用它，l1的 backward 结束时内存管理把这两条不可能再完整反向的分支一起删了（规则见 burn 卷 backward 与 Gradients 的「什么时候释放」）。上一节h还在手里，报的就是exp。 - 叶子
x被l1的 backward 用过，新的前向照样能用它：叶子没有步骤不算错。这就是训练循环每步重算前向能成立的原因。 l3也依赖被消耗的exp，同样 panic；能救回来的只有不经过被消耗节点的分支。- PyTorch 的
RuntimeError是普通异常；burn 是 panic，要在 Rust 里接住只能catch_unwind。正常代码不应该靠它，该做的是在写法上避免第二次 backward。
(x * 2).sum()、(x + 1).sum()、(x * x).sum()，每个 backward 一次之后再 backward 一次。PyTorch 里哪几个第二次会报错？burn 里呢？x = [1, 2] 要梯度，代码在上面两个实验文件里，平台同上。
动脑筋答案
for name, f in [("x * 2", lambda: (x * 2).sum()), ("x + 1", lambda: (x + 1).sum()),
("x * x", lambda: (x * x).sum())]:
l = f()
l.backward()
attempt(f"{name}: second backward", l.backward)
print(f" x.grad = {x.grad}")
x.grad = None
x * 2: second backward: RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). ...
x.grad = tensor([2., 2.])
x + 1: second backward: ok
x.grad = tensor([2., 2.])
x * x: second backward: RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). ...
x.grad = tensor([2., 4.])
x * 2: first x.grad = Some([2.0, 2.0])
...
x * 2: second backward: panicked
x + 1: first x.grad = Some([1.0, 1.0])
...
x + 1: second backward: panicked
x * x: first x.grad = Some([2.0, 4.0])
...
x * x: second backward: panicked
- PyTorch 只有
x + 1第二次成功，.grad累加成[2, 2]（两次各[1, 1]）。另两个失败的.grad是第一次的结果。 - 原因在每个反向节点保存了什么。在 PyTorch 里查
grad_fn上_saved_开头的属性：MulBackward0有_saved_self和_saved_other，乘以常数 2 也一样存了两个操作数；AddBackward0只有_saved_alpha（一个标量系数），SumBackward0只有_saved_self_sym_sizes（形状）。第一次 backward 释放的是保存的张量，加法和求和没有张量可释放，所以第二次走得通。 - burn 三个都 panic（缺的分别是
NodeId(5)、NodeId(7)、NodeId(9)，都是各自的sum）：判断的依据是步骤还在不在，不看步骤里有没有保存东西。从 PyTorch 过来，不能用「这段图里只有加法，第二次 backward 没事」来推断 burn 的行为。
小结
- PyTorch 第二次 backward 报不报错看保存的张量释放了没有，
retain_graph=True可以留住；burn 的 backward 消耗沿途所有非叶子步骤，第二次一律 panic，没有retain_graph。 - 句柄
clone()共享同一份步骤，复制句柄救不回来。 - 共享中间结果的几个 loss 先相加再 backward；结果和 PyTorch 的
retain_graph加两次 backward 逐位相同。 - panic 前不消耗任何步骤，叶子可以给新的前向反复用；训练循环每步重算前向。