Skip to content

health views: distinguish uncollected device from missing observation #645

Description

@ben-malbeclabs

Problem

The health_* views (health_multicast_user, health_publisher_subscriber_path, health_multicast_user_rate) treat the absence of mroute / MSDP data on a device as "endpoint missing" → unhealthy. But state-collect only runs on jump devices today — 31 of 97 activated devices as of writing — so any user, publisher, or subscriber pinned to a non-jump device renders as unhealthy even when it is functioning normally.

Concrete examples:

  • Users on dz100a-lax1-tsw, dz103a-lon1-tsw, dz115a-tyo2-tsw show health_status='unhealthy' with reasons like "publisher Tunnel529 not seen as RPF interface for (S,G) on dz100a-lax1-tsw" — but no mroute data is collected from that device at all.
  • The rate dimension (health_multicast_user_rate.expected_bps_5m) falls through to NULL for every subscriber in edge-solana-shreds because some publishers in the group are on non-jump devices, tripping the gpt_missing_publisher_count > 0 guard for the whole group.

Proposed fix

Add a state_collected_devices CTE (or refreshable materialized view) seeded from devices with any mroute / MSDP observation in the last 24 h. Then in each health_* view:

  • Endpoint observation booleans (publisher_iif_observed, subscriber_oif_observed, endpoints_reconciled) stay as-is.
  • Add a verdict override: if the relevant device is NOT in state_collected_devices, set health_status='unknown' (not unhealthy) with a mismatch_reason of "endpoint device not state-collected" (or similar).
  • For path rows: track publisher_device_state_collected and subscriber_device_state_collected separately so the UI can render which side is uncollected.
  • For gpt_missing_publisher_count: ignore publishers on non-state-collected devices when computing the gating count, so subscribers in a group still get an expected_bps_5m derived from the publishers we actually observe.

Why not now

The fix is straightforward but it changes verdicts that operators are already starting to learn — needs a notice / a coordinated rollout. Banner is in place (lake#643) so users read current verdicts as work-in-progress.

Refs

  • malbeclabs/infra#1501 (multicast health umbrella)
  • See ~/.claude/.../memory/reference_lake_state_collect_scope.md for the jump-vs-non-jump enumeration query.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions