[pull] master from kevoreilly:master#503

Merged

pull[bot] merged 57 commits into

threatcode:masterfrom

kevoreilly:master

Jul 13, 2026

Commits on Jul 9, 2026

rooter: opt-in next-hop egress primitive (fail-closed, per-task, vendor-neutral)
node5-sublime
committed
docs: processor concurrency redesign spec (pluggable pebble/prefork engines)

wmetcalf
and
claude
committed
docs: add audited memory model & shared-state section to processor redesign spec

wmetcalf
and
claude
committed
docs: finalize processor redesign — fork-per-task lifecycle + measured lazy-load cost

wmetcalf
and
claude
committed
docs: implementation plan for processor prefork engine

wmetcalf
and
claude
committed
docs(plan): clarify A/B runs on production PostgreSQL; sqlite is test-only

wmetcalf
and
claude
committed
feat(processor): add TaskSource for engine task polling/status

wmetcalf
and
claude
committed
fix(processor): revert TaskSource to session.begin(); fix test setup to match production txn discipline
wmetcalf
committed
refactor(processor): document TaskSource.fetch limit semantics; cover failed_processing path
wmetcalf
committed
feat(processor): add ProcessingEngine base + get_engine registry
wmetcalf
committed
refactor(processor): extract run_task(task) adapter shared by engines

wmetcalf
and
claude
committed
fix(processor): restore session.begin() wrapper in run_task; fix test setup to match production txn discipline
wmetcalf
committed
feat(processor): PebbleEngine behind engine seam + --engine flag (default pebble)

wmetcalf
and
claude
committed
fix(processor): make pebble task_fn picklable (functools.partial); restore per-task proctitle/formatter cleanup

wmetcalf
and
claude
committed
feat(processor): PreforkEngine scaffolding + single-threaded invariant
wmetcalf
committed
feat(processor): prefork fork-per-task execution + reap + failure status

wmetcalf
and
claude
committed
feat(processor): prefork wall-clock timeout via killpg + grace escalation

wmetcalf
and
claude
committed
fix(processor): log SIGKILL escalation; broaden killpg exception handling; strengthen no-orphan test
wmetcalf
committed
test(processor): regression test locking worker_init-before-task_fn ordering

wmetcalf
and
claude
committed
docs(processor): spike results for extractor sub-pool spawn strategy

wmetcalf
and
claude
committed
fix(extractors): per-call pool with explicit teardown, swept by process-group kill
wmetcalf
committed
revert: drop out-of-scope static_file_info elif accidentally bundled in Task 10

wmetcalf
and
claude
committed
docs(processor): A/B runbook + mark redesign implementation status

wmetcalf
and
claude
committed
fix(processor): final polish — freespace backpressure for prefork; completion log; runbook paths; decoded waitpid status; wire max_analysis_count to both engines
wmetcalf
committed
feat(processor): prefork invariant also checks /proc/self/task (catches native C-extension threads like STPyV8 V8 workers)
wmetcalf
committed
fix(peepdf): lazy-import STPyV8/V8 bindings (was spawning 16 native threads at module load, breaking prefork single-threaded invariant)

wmetcalf
and
claude
committed
fix(processing): lazy-import imagehash to keep prefork supervisor kernel-single-threaded

wmetcalf
and
claude
committed
perf(extractors): shared per-child extractor pool under prefork
wmetcalf
committed
rooter: gateway liveness requires IFF_UP, not just interface existence
node5-sublime
committed
fix(extractors): bound extractor-pool teardown so a hung grandchild can't wedge the worker
wmetcalf
committed
perf(processor): pre-warm YARA in supervisor so forked children inherit it (COW)
wmetcalf
committed
rooter: fail-closed hardening (resolve re-entry clears stale binding + nexthop_init blackhole seed)
node5-sublime
committed
rooter: startup config validation (dirty-line table gating when enabled + default_policy)
node5-sublime
committed
docs: drop internal planning docs from upstream PR; make code comments self-contained
wmetcalf
committed
rooter: rebind ip-rule cleanup is table-agnostic + reject fail-table collision at startup
node5-sublime
committed
fix(prefork): don't reap timed-out children before SIGKILL escalation; killpg fallback; idle backoff
wmetcalf
committed
fix(processor/source): widen DB fetch limit by exclude_ids to avoid worker starvation
wmetcalf
committed
fix(extractors/dedup): bound post-stop pool join; close QR image fd
wmetcalf
committed

Commits on Jul 10, 2026