Lab-to-AI: From Raw Zone to SageMaker and HealthOmics
- Shameem Abdul Salam
- Aws , Dev ops
- October 15, 2026
Table of Contents
The science team’s request arrived as four words: “AI on our data.” Behind it stood a platform team with a validated raw zone, a governance model built for audits, and no plan for what happens when a model wants to read everything.
This is the series finale: the raw zone you built and defended becomes the substrate for AI/ML—but only if governance, access, and cost controls arrive before the first model runs. This post is the readiness checklist for that day.
The staging path: raw to training-ready
Data goes AI-ward in stages, and each stage has a different contract:
| Stage | What it is | Governance question |
|---|---|---|
raw/ | Immutable originals (system of record) | Who may read at all? |
| Curated | Cleaned, schema’d tables (your Iceberg layer) | Is the transform reviewed and reproducible? |
| Training sets | Versioned, consent-filtered extracts | Is provenance traceable to runs? |
- Nothing trains on
raw/directly—models read curated tables with recorded lineage, or you lose the audit thread at the worst moment. - Consent/PHI/PII flags live in catalog metadata; extracts are filtered by policy, not by a scientist’s memory. Macie or equivalent scans for what metadata missed.
- Training sets are versioned artifacts—a model result must be reproducible from a named dataset, the way a run result traces to a manifest.
Where the work runs
- SageMaker Unified Studio as the governed front door: data prep, catalog, and ML apps in one environment with projects and access control—so “the AI environment” isn’t a snowflake account someone made in 2025.
- Catalog governance via DataZone-style domains: publish curated data with owners, expiry, and approval flows—the attribution discipline, applied to analytics.
- HealthOmics when the workload is genuinely genomic: managed sequence storage and workflow runs beat hand-rolled pipelines for cost and reproducibility—evaluate before building.
- Self-built training (your Batch estate) for everything that isn’t a fit—managed isn’t mandatory, it’s a default to justify against.
Bedrock and assistants: the guardrails that matter
- No raw PHI/consented identifiers into prompts—the filtering happens in the data layer, not in a prompt template someone edits later.
- Knowledge bases read curated, consent-cleared tables—never the raw bucket “just this once.”
- Access to Bedrock/AgentCore is a scoped role per project, reviewed like any boundary identity—not a shared API key.
- Agent actions against data platforms are read-only until a change-control story exists (validated delivery applies to agentic changes too).
FinOps for AI: the cost controls
Nearly every team now manages AI spend; the ones hurt worst are the ones that started after the bill:
- Per-project tagging enforced at role level (only a project role can launch project resources); Cost Categories roll projects up.
- Budgets with alarms per research group—surprise is the failure mode, size is negotiable.
- Model-choice discipline: start with the cheapest model that passes eval; record the eval, not the vibes.
- Weekly review of idle endpoints, notebooks, and feature groups in the same cost hygiene pass as everything else.
Before the first model runs
- Named dataset versions with lineage, filtered by policy.
- Scoped roles, catalog approvals, no raw-bucket shortcuts.
- Budget alarms live, tagged at the role.
- A written answer to “can this model’s output be traced back to runs?"—because that question is coming.
Series map
| Post | Focus |
|---|---|
| ALCOA+ field guide | Data integrity as AWS controls |
| Ingestion patterns | Poller vs DataSync vs S3 Events |
| Raw zone to Iceberg | S3 Tables and queryable projections |
| Validated delivery | CI/CD as the change-control system |
| Audit evidence | Observability that doubles as records |
| This post | Governed AI on the raw zone |
If you only do one thing: publish the rule “models read curated, humans approve the extract” before the first AI project starts—retro-fitting governance onto a running model is ten times the work, and it’s the one corner of this series where undo is not an option.