OneLake Security Framework

Reusable, config-driven deployment of Microsoft Fabric OneLake Security (OLS/DAR) — 3 notebooks, nothing else. Drop them into any project's workspace; everything customer-specific lives in one Delta config table and the runtime parameters. The notebooks never change per project.

Note

OneLake Security reached General Availability in May 2026 — it is new, and many teams haven't adopted it yet. Announcement: OneLake security Generally Available (Fabric blog).

What is OneLake Security?

OneLake Security (OLS) is Microsoft Fabric's native, data-plane access control: you define security roles on the data itself (a lakehouse's tables, folders, rows, columns) and OneLake enforces them everywhere — instead of re-implementing security per engine with T-SQL views, workspace boundaries, or report-side filters.

Feature highlights:

• Role-based access — grant Read per table (/Tables/schema/table) or folder (/Files/...); users with no role see nothing (deny by default). Create and manage roles →
• Row-level security (RLS) — SQL-like predicates per table per role (static values; operators = <> IN NOT AND OR NULL). A member of several roles gets the union of what they allow. Row-level security →
• Column-level security (CLS) — hide specific columns per role. Column-level security →
• Entra-native membership — role members are Entra users, security groups, or service principals; membership changes happen in Entra, no sync tables to maintain.
• One definition, every engine — enforced across the SQL analytics endpoint, Spark, and Direct Lake semantic models (Power BI), so reports and notebooks see the same filtered data.
• Granular write — separate ReadWrite roles for scoped write access (RLS/CLS don't apply there).

Worth knowing: roles carrying RLS/CLS are read-only; workspace Admins/Members/Contributors bypass OLS (it governs consumers); role definitions are per lakehouse.

Why this framework?

OneLake Security itself is great — operating it by hand is not. Real deployments mean tens of roles × hundreds of tables × several member groups × dev/test/prod, and the portal gives you no review, no preview, and no history. The specific traps this framework closes:

Pain (all real, all easy to hit)	What the framework does
Portal edits apply immediately — no diff, no approval, no undo trail	plan records the exact diff first; apply is blocked until a matching plan exists and the live state hasn't drifted
The bulk write API is full-replace — omit a role and it's silently deleted	apply = upsert-only (never deletes); deletions require an explicit replace, which prints the doomed list and always preserves Default* roles
NOT IN predicates silently drop NULL rows (SQL three-valued logic)	validation warns on every NOT IN without OR <col> IS NULL
Multi-role members get the union of permissions — over-exposure creeps in unnoticed	expand warns when one member reaches the same table through several roles
Hand-typed table names fail silently or grab the wrong case	expand resolves everything against the live Spark catalog — typos and 0-match globs are hard errors
No audit story for compliance	every run writes atomic rows (role × scope × member) to an audit log, linked plan→apply by batch_id
Same problem, every project, rebuilt every time	config + parameters are the only per-project pieces — the notebooks never change

In short: it brings the plan → review → apply discipline of Terraform/kubectl to OneLake Security, in plain Fabric notebooks your team already knows how to run.

Notebook	Purpose
onelake_security_lib.ipynb	Source of truth. Cell 1: pure functions (parse/expand/validate/diff — CI-testable anywhere). Cell 2: classes — FabricClient (REST), AuditLog (audit log), OneLakeSecurityDeployment (one public method per mode; shared steps = private methods).
onelake_security.ipynb	Runtime — thin orchestration. Setup builds the objects; one mode = one cell = one method call.
onelake_security_test.ipynb	stdlib unittest scenarios. mode=unit (pure, runs anywhere) · mode=integration (black-box against the real runtime). One runner, summary JSON to the pipeline. How to write/run/read: docs/TESTING.md.

Pipeline (lock-file pattern)

ctl_security_config (human, short: ';' lists, globs, case-insensitive)   ← CUSTOMER LOGIC HERE
        │  mode=expand   — canonical-case dict from Spark catalog, validation A/B/C
        ▼
ctl_security_config_expanded (machine lock-file + provenance) + CSV export
        │  mode=plan     — source_hash staleness guard, diff vs live DAR
        │  gate — a successful plan for this exact config + still-matching live diff
        ▼
mode=apply (upsert, never deletes) | replace (full truth, Default* preserved)
        → ctl_security_log (atomic audit rows)

Modes at a glance

mode	When	What it does	Writes
setup	once per lakehouse	creates the 3 control tables (idempotent)	tables
expand	after every config edit	validates + freezes config into the lock-file + review CSV	lock-file
plan	before any write	diffs desired vs live, records the plan — unlocks apply/replace	audit log
apply	routine deployment	upserts managed roles — never deletes	OLS + log
replace	deliberate cleanup	config = full truth; omitted roles deleted (Default* kept)	OLS + log
show-role	"what can this role reach?"	read-only live pivot (subject required, globs ok)	—
show-table	"who reaches this table?"	read-only live pivot	—
show-member	"what does this member see?"	read-only live pivot	—

Full manual — per-mode details, guards, result keys, and the failure catalog: docs/MODES.md. Table schemas — ER + data dictionary: docs/DATA-MODEL.md. Every run exits with a JSON result (result.exitValue) so a pipeline can branch — e.g. plan returns changes: false → skip approval + apply.

Adopting in a new project (5 steps — no notebook edits)

1. Import the 3 notebooks into the workspace that hosts (or can reach) the target lakehouse — three ways (portal / REST / git integration): see docs/FABRIC-IMPORT.md.
2. Run mode=setup — creates the three control tables (idempotent; names configurable via parameters, defaults ctl.ctl_security_config / _expanded / _log), then author one row per role × rule (see docs/RUNBOOK.md §2 for the column contract).
3. Set parameters (or pass from a pipeline): usually just mode + tenant_id — workspace_id/item_id auto-resolve to the attached lakehouse.
4. Run expand → review CSV → plan → review diff → apply.
5. Wire CI/CD (optional): fabric-cicd deploys the 3 notebooks; the test notebook is the promote gate (mode=unit then mode=integration); batch_id = pipeline run id links plan→apply.

Guardrails (built in, non-negotiable)

• Glob expansion: 0 matches = error · schema part needs ≥ 1 literal.
• NOT IN without OR <col> IS NULL warns (NULL rows silently drop — verified on a live Fabric workspace).
• Member lists must be identical across a role's rows (hard error).
• Staleness guard (source_hash) + saved-plan gate + drift-since-plan gate + GUID identity guard.
• apply never deletes; replace deletes omitted roles but always preserves Default*.
• Empty config refuses to expand; duplicate rows skip with a warning.

Status / TODO

• Unit suite passes outside Fabric (extract-and-exec — CI does exactly this).
• Not yet verified against the live API: per-role single POST/DELETE + If-Match ETag (differential apply) and Graph member-name → objectId resolution. Until then, apply uses the proven bulk PUT after the gates, and member objectIds must already be GUIDs.

License

MIT