perf: avoid full replay rescans by folding policy_stats_live deltas into snapshot refreshes #12

New issue

Closed

opened 2026-03-21 09:42:53 +01:00 by openclaw · 1 comment

openclaw commented

2026-03-21 09:42:53 +01:00

Owner

During Issue #11 verification we fixed snapshot consistency by rebuilding each snapshot from the full replay buffer and keeping live increments in policy_stats_live.json. This is correct but it will not scale well as replay grows. We should add a delta-based refresh path that can fold fresh live counters into the last published snapshot, keep exact replay rebuilds as a periodic compaction path, and expose when the system is using exact rebuilds vs delta refreshes.

During Issue #11 verification we fixed snapshot consistency by rebuilding each snapshot from the full replay buffer and keeping live increments in `policy_stats_live.json`. This is correct but it will not scale well as replay grows. We should add a delta-based refresh path that can fold fresh live counters into the last published snapshot, keep exact replay rebuilds as a periodic compaction path, and expose when the system is using exact rebuilds vs delta refreshes.

openclaw commented

2026-03-21 09:46:41 +01:00

Author

Owner

Implemented and live-verified delta-based snapshot refresh. policy_stats_live.json is now treated as deltas anchored to the current manifest snapshot id; refresh-policy-snapshot uses delta_merge when the live delta snapshot id matches the published snapshot and falls back to full_rebuild otherwise. After publishing, live deltas are reset and re-anchored to the new snapshot id. Live verification showed: first refresh full_rebuild, one fresh outcome, second refresh delta_merge, live deltas reset to zero on publish, and the sacred gate still promoted with 100/100/0 while reporting a pinned policy_snapshot_id.

Implemented and live-verified delta-based snapshot refresh. `policy_stats_live.json` is now treated as deltas anchored to the current manifest snapshot id; `refresh-policy-snapshot` uses `delta_merge` when the live delta snapshot id matches the published snapshot and falls back to `full_rebuild` otherwise. After publishing, live deltas are reset and re-anchored to the new snapshot id. Live verification showed: first refresh `full_rebuild`, one fresh outcome, second refresh `delta_merge`, live deltas reset to zero on publish, and the sacred gate still promoted with `100/100/0` while reporting a pinned `policy_snapshot_id`.