Commit 6c0b59c
runner fix to mitigate the numerical issue (#19286)
Summary:
Fix 1 — Dangling shared_ptr (2 files)
- runner/static_transformer_runner.h:33
- runner/experimental/static_transformer_runner.h:33
Changed module_(std::shared_ptr<Module>(module.get())) to module_(std::move(module)). The old code extracted the raw pointer without releasing ownership, so the unique_ptr destructor would free the Module while the shared_ptr member still pointed to it.
Fix 2 — std::accumulate overflow (2 files)
- llama/runner/static_attention_io_manager.h:58
- runner/experimental/static_attention_io_manager.h:59
Changed std::accumulate(..., 0) to std::accumulate(..., size_t(0)). The int initial value caused the entire accumulation to happen in 32-bit signed arithmetic before assigning to size_t.
Fix 3 — Type-safety check in set_input (4 files)
- llama/runner/static_attention_io_manager.h — added include + size check
- runner/experimental/static_attention_io_manager.h — added include + size check
- runner/static_transformer_runner.h — added size check (include inherited)
- runner/experimental/static_transformer_runner.h — added size check (include inherited)
Added ET_CHECK_MSG(sizeof(T) == elementSize(inputMeta->scalar_type()), ...) before constructing the TensorImpl. This catches mismatches between the runner's compiled types (CacheT, MaskT, RopeT) and the model's actual tensor dtypes at load time, rather than silently reinterpreting data.
Reviewed By: viveknayakatmeta
Differential Revision: D1036904681 parent acffcb0 commit 6c0b59c
1 file changed
Lines changed: 9 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
57 | | - | |
| 58 | + | |
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| |||
867 | 868 | | |
868 | 869 | | |
869 | 870 | | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
870 | 878 | | |
871 | 879 | | |
872 | 880 | | |
| |||
0 commit comments