---

Quick Definition (30–60 words)

Pod Security Admission is a Kubernetes admission controller that enforces pod-level security standards at creation time. Analogy: like a security checkpoint that inspects backpacks before you enter a building. Formal: it validates and/or enforces PodSecurity admission labels and policies according to namespace-level enforcement modes.

---

What is Pod Security Admission?

Pod Security Admission (PSA) is a Kubernetes admission plugin that enforces pod-level security validation using built-in policy tiers (privileged, baseline, restricted) based on namespace labels. It is not a full policy engine with custom policy language; it provides curated checks for common pod security risks.

What it is / what it is NOT

• It is a built-in admission controller present in many Kubernetes distributions.
• It is NOT a replacement for a policy engine such as OPA/Gatekeeper when you need custom or complex policies.
• It is NOT dynamic runtime protection; it blocks or warns at admission time.

Key properties and constraints

• Operates at admission time before objects are persisted.
• Applies per-namespace enforcement using labels like pod-security.kubernetes.io/enforce.
• Has three policy levels: privileged, baseline, restricted.
• Intended for common best-practice checks, not exhaustive security posture validation.
• Works where admission webhooks are allowed; disabled in clusters that do not enable the admission plugin.

Where it fits in modern cloud/SRE workflows

• Early policy gate in CI/CD pipelines and cluster admission path.
• Low-friction, standardized baseline for security teams and platform teams.
• Complements runtime security and workload hardening practices.
• Useful as a first safety net in multi-tenant clusters and managed Kubernetes control planes.

Diagram description (text-only)

• Developer -> Push manifest or Helm chart -> CI runs static validations -> Kubernetes API server receives create request -> Pod Security Admission checks namespace labels and pod spec -> Outcome: allow | warn | deny -> Object persisted or rejected -> Observability emits audit/event.

Pod Security Admission in one sentence

Pod Security Admission enforces standardized pod-level security checks at creation time using a three-tier policy model to block or warn on insecure pod specifications.

Pod Security Admission vs related terms (TABLE REQUIRED)

ID	Term	How it differs from Pod Security Admission	Common confusion
T1	OPA Gatekeeper	More flexible with Rego policies	People expect PSA to do complex custom checks
T2	Kyverno	Policy engine with mutating capabilities	Kyverno can mutate; PSA cannot
T3	PSP (PodSecurityPolicy)	Deprecated API removed in modern clusters	Some think PSP and PSA are the same
T4	Runtime security agents	Protects at runtime, not admission time	Expect PSA to block runtime exploits
T5	NetworkPolicy	Controls network traffic, not pod spec fields	Confused with securing network posture
T6	Admission webhook	Generic mechanism for custom checks	PSA is a specific admission plugin
T7	Image scanner	Analyzes container images for vulnerabilities	PSA does not inspect image contents
T8	RBAC	Manages API access control, not pod attributes	People mix authz with workload constraints

Row Details

• T1: OPA Gatekeeper can express arbitrary policies via Rego and can audit, enforce, and mutate; use when you need complex constraints beyond PSA.
• T2: Kyverno supports policy validation, generation, and mutation; use when you need to auto-fix or generate labels.
• T3: PodSecurityPolicy was an older object for pod restrictions; PSA replaces the common use cases with built-in checks.
• T4: Runtime agents monitor syscall, process and file activity; PSA only blocks at admission and reduces attack surface before runtime.
• T7: Image scanners inspect layers and CVEs; PSA can deny images by simple patterns but not CVE content.

---

Why does Pod Security Admission matter?

Business impact (revenue, trust, risk)

• Prevents insecure workloads from being deployed, reducing likelihood of breaches that could lead to data loss or downtime.
• Lowers compliance risk by enforcing standardized workload hardening, which protects brand and customer trust.
• Reduces potential financial exposure from incidents by decreasing attack surface at deployment time.

Engineering impact (incident reduction, velocity)

• Reduces incidents caused by misconfigured pods (privileged containers, hostPath misuse).
• Enables platform teams to enforce guardrails and let developers self-serve within secure defaults, increasing velocity.
• Lowers toil since security checks are centralized and consistent, avoiding repeated manual reviews.

SRE framing (SLIs/SLOs/error budgets/toil/on-call)

• SLI candidates: percentage of pods that pass security admission, time to detect rejected deployments.
• SLOs: e.g., 99.9% of production pods comply with restricted or baseline policies.
• Error budget use: policy rollouts can consume error budget if they cause deployment failures; manage via stage rollouts.
• Toil reduction: fewer post-deploy security fixes reduces repetitive triage work for on-call.

3–5 realistic “what breaks in production” examples

• A stateful workload accidentally runs with hostPath mount to /var, causing data leakage across tenants.
• A CI job deploys a container with privileged scale-up model that escapes isolation and triggers a cluster compromise.
• An autoscaler creates pods missing resource limits, causing node OOMs and noisy neighbor failures.
• Service pods start with root user and run unnecessary capabilities, increasing internal threat surface.
• Unvetted init containers mount Docker socket, enabling container runtime privilege escalation.

---

Where is Pod Security Admission used? (TABLE REQUIRED)

ID	Layer/Area	How Pod Security Admission appears	Typical telemetry	Common tools
L1	Cluster control plane	Admission plugin enforcing labels	Audit events, admission logs	kubectl, kube-apiserver
L2	Namespace governance	Label-driven enforcement per namespace	Namespace labels changes, rejections	GitOps tools, namespace managers
L3	CI/CD pipeline	Preflight checks and test deployments	CI job failures, policy test logs	CI systems, unit tests
L4	Platform engineering	Default guardrails for developer platforms	Ticket counts for denials, onboarding metrics	Platform APIs, templates
L5	Multi-tenant security	Tenant isolation via restricted policies	Rejection rates per tenant	RBAC, quota controllers
L6	Managed Kubernetes	Vendor-enabled PSA as default	Vendor audit logs, support tickets	Cloud provider control planes

Row Details

• L3: CI/CD integration often runs kubectl apply against test clusters to validate cluster admission behavior and avoid production failures.
• L4: Platform teams use PSA to set default namespace labels and templates so developers get secure defaults when creating namespaces.
• L6: Managed Kubernetes vendors may enable PSA by default at certain policy levels; behavior can vary between providers.

---

When should you use Pod Security Admission?

When it’s necessary

• You need a low-effort, standardized baseline to prevent common pod-level risks.
• Operating a multi-tenant cluster where tenant isolation and predictable behavior matter.
• Want to enforce minimal security expectations for developer-created workloads.

When it’s optional

• Single-team clusters with strong CI/CD pre-deployment checks and dedicated security engineers.
• Environments already protected by an advanced policy engine with broader governance needs.

When NOT to use / overuse it

• Don’t rely on PSA for complex policy logic like image scanning CVE enforcement or scheduling constraints.
• Avoid using PSA alone for runtime protection, network segmentation, or OS-level hardening.
• Don’t use PSA to replace fine-grained authorization or secrets management.

Decision checklist

• If multi-tenant AND want quick wins -> enable PSA restricted/baseline.
• If need custom policies or mutation -> use Gatekeeper or Kyverno alongside PSA.
• If you have automated CI checks AND single trusted team -> PSA optional.
• If you need runtime detection and response -> complement PSA with runtime tools.

Maturity ladder: Beginner -> Intermediate -> Advanced

• Beginner: Enable PSA in warn mode to evaluate impact, set baseline for non-prod.
• Intermediate: Enforce baseline in dev and staging; restricted in security-sensitive namespaces.
• Advanced: Combine PSA with OPA/Gatekeeper or Kyverno, integrate with CI and automated remediation.

---

How does Pod Security Admission work?

Step-by-step components and workflow

1. Namespace labeling: Platform/admin sets pod-security.kubernetes.io/enforce|warn|audit with a policy level.
2. API server receives Pod creation/update request.
3. PSA plugin evaluates the pod spec against the policy level’s checks.
4. If policy violations: – enforce mode: deny creation/update. – warn mode: allow but emit a warning event. – audit mode: allow and emit an audit event.
5. Admission outcome logged to audit logs and to namespace events.
6. Developer or automation receives rejection or sees warning, adjusts spec, retries.

Data flow and lifecycle

• Input: Pod manifest or workload controller creating pods.
• Checks: Pod spec fields (securityContext, volumes, capabilities, host namespaces, privileged flag, runAsNonRoot, etc.).
• Output: Admission decision and audit/warn/deny events.
• Lifecycle: Only at admission time; no continuous enforcement after pod runs beyond admitted spec.

Edge cases and failure modes

• Admission plugin disabled or misconfigured -> inconsistent enforcement.
• Namespaces without labels default to privileged behavior depending on cluster config -> unexpected allowances.
• Admission conflicts when multiple admission controllers apply -> ordering and plugin semantics matter.

Typical architecture patterns for Pod Security Admission

• Default-namespace-labels pattern: Platform bootstraps new namespaces with baseline labels using namespace controllers or GitOps; use when onboarding many teams.
• CI preflight pattern: Run a test apply in a staging cluster with PSA enforce to catch admission denials early; use when wanting fast feedback in CI.
• Gradual rollout pattern: Start PSA in warn/audit across cluster, then gradually enforce per namespace; use for low-risk adoption.
• Policy-composition pattern: Combine PSA for common checks and Gatekeeper/Kyverno for custom rules; use when you need both standard and custom governance.
• Tenant-isolation pattern: Enforce restricted on tenant namespaces and baseline for internal tooling; use in multi-tenant clusters.

Failure modes & mitigation (TABLE REQUIRED)

ID	Failure mode	Symptom	Likely cause	Mitigation	Observability signal
F1	Unexpected denials	Deployments failing on create	Namespace label enforcement too strict	Rollout warn first and adjust policies	Admission deny audit events
F2	Silent permissive behavior	Insecure pods created	Namespace lacks labels or plugin disabled	Apply default namespace labels and enable plugin	Increase in risky pod specs seen in audit
F3	CI breaks due to policy	CI job fails on deploy step	CI not aligned with cluster policies	Add policy checks to CI or use staging cluster	CI failure logs pointing to admission denies
F4	Alert fatigue	Many warnings flooding teams	PSA in warn mode widely across many namespaces	Rebalance modes and tune templates	High volume of warning events
F5	Conflicting policies	Rejected by webhook after PSA pass	Multiple admission controllers conflicting	Review admission ordering and policy overlap	Mixed admission/deny audit entries

Row Details

• F2: Some clusters default to permissive behavior if labels are absent; platform should set a default via bootstrap or admission configuration to avoid surprises.
• F3: CI pipelines that apply to production without a staging check often encounter denials; maintain a mirror staging environment to validate admission behavior.

---

Key Concepts, Keywords & Terminology for Pod Security Admission

• Admission controller — A component that intercepts API requests to validate or mutate objects — Central to admission-time policy — Confusing with runtime agents
• Pod — Smallest deployable unit in Kubernetes — Target of PSA checks — Confused with container
• Namespace — Logical cluster partition — Label-driven PSA scope — Missing labels cause different behavior
• Enforcement mode — enforce warn audit — Determines deny or warn behavior — People misconfigure for production
• PodSecurity standards — The curated checks and fields evaluated — Provide baseline security — Not exhaustive
• Baseline level — Minimal acceptable restrictions — Good for developer workloads — Not sufficient for multi-tenant isolation
• Restricted level — Strong restrictions to reduce attack surface — Best for critical workloads — May block legacy apps
• Privileged level — Permissive mode for backwards compatibility — Use for system namespaces — Risky for general workloads
• SecurityContext — Pod/container-level settings for user, capabilities, SELinux — Primary PSA check target — Missing runAsNonRoot is common pitfall
• runAsNonRoot — Ensures non-root user — Prevents root containers — Legacy images may fail
• runAsUser — Numeric UID setting — Ensures specific user runs containers — Images must support the UID
• readOnlyRootFilesystem — Prevents writes to root — Increases immutability — Breaks apps writing to root
• capabilities — Linux capability bits like NET_ADMIN — PSA may deny added capabilities — Some apps require capabilities
• privileged flag — Full container privileges akin to host root — Typically denied in restricted mode — High risk
• hostPath volume — Mounts host filesystem into pod — Common for breakout and local access — Often blocked
• hostNetwork — Pod uses host network namespace — Can expose cluster network — Use sparingly
• hostPID — Access to host process namespace — High risk for introspection — Denied in restricted
• hostIPC — Access to host IPC namespace — Rarely needed — Denied in hardened profiles
• seccompProfile — Syscall filtering profile — PSA checks for default or runtime/default — Misconfigured profiles can break syscalls
• SELinuxOptions — SELinux labeling for containers — Enforces MAC policies — Complex to set for many images
• AppArmor — Linux syscall sandboxing — Not available on all distros — PSA checks presence in some setups
• readinessProbe — Not a PSA check but related to deployment health — Important for safe rollouts — Missing probes increase risk
• livenessProbe — Also not PSA but important — Restarts unhealthy containers — Overaggressive probes cause flapping
• resource limits — CPU/memory requests and limits — PSA encourages reasonable limits implicitly — Missing limits cause noisy neighbor issues
• imagePullPolicy — Controls image pulls — Not directly PSA controlled — PullAlways can affect rollout timing
• image registry — Where images are stored — PSA does not validate image trust by default — Use image policy/webhooks for signing checks
• immutable images — Reproducible and pinned images — PSA complements immutability by limiting risky fields — Mutable tags are a pitfall
• workload controller — Deployment/StatefulSet/etc that creates pods — PSA evaluates pods created by controllers — Controller-level mutation may be needed
• Mutating admission webhook — Alters objects on the fly — PSA cannot mutate; combine with mutating webhooks for autofix
• Validating admission webhook — Rejects based on custom logic — PSA is a built-in validating type for pod checks — Order matters
• Audit logs — Cluster-level history of actions and denies — Key for incident forensics — Ensure audit log retention
• Events — Kubernetes events for warn/audit messages — Useful for quick triage — Can be transient
• GitOps — Declarative cluster config via git — Recommended to manage PSA namespace labels and defaults — Ensure sync is reliable
• Multi-tenant cluster — Hosts multiple orgs or teams — PSA is vital to isolate tenants — Requires careful label and RBAC design
• Least privilege — Security principle enforced by PSA — Prefer restricted defaults — May need exemptions
• Exemption — Explicitly allow an exception (not native in PSA) — Achieved via label/classic policy patterns — Track with strong audit
• On-call playbook — Steps for denied deployment or security reprovision — Critical to reduce MTTR — Keep short and practical
• Deny vs warn drift — Mismatch between warn and enforce modes across environments — Causes production incidents — Use phased rollout
• Policy drift — Divergence between desired and actual policy state — Detect with audits — Reconcile via GitOps
• Remediation automation — Scripts or controllers to fix violations — PSA is admission-only so use mutating tools for remediation — Avoid over-automation risk

---

How to Measure Pod Security Admission (Metrics, SLIs, SLOs) (TABLE REQUIRED)

ID	Metric/SLI	What it tells you	How to measure	Starting target	Gotchas
M1	Pod admission pass rate	Share of pods accepted without violation	Count admitted pods / total pod create attempts	99.9% for prod	High rate may hide missing policy scope
M2	Pod admission deny rate	Rate of rejected pods	Count denied pod creates / attempts	<0.1% for prod	Denies during rollout expected
M3	Warn event rate	Number of warn events emitted	Count PSA warn events over time	Monitor baseline by env	High warns cause alert fatigue
M4	Time to fix denied pod	Mean time from deny to successful deploy	Time between deny event and successful pod	<2 hours for dev	Longer for infra-owned namespaces
M5	Policy rollout failures	Deployments failed after policy change	Count failed deployments post-change	Aim for zero during canary	Changes in enforce mode spike this
M6	Audit coverage	Fraction of namespaces with PSA labels	Namespaces labeled / total namespaces	100% for platform-managed	Some namespaces intentionally exempt
M7	Runtime incidents linked to misconfigs	Incidents attributed to initial insecure pods	Postmortem tagging/count	0 for critical incidents	Attribution requires good postmortems
M8	CI preflight mismatch rate	Deployments passing CI but failing cluster PSA	CI accepted / cluster denied ratio	<0.5%	Mirrors staging not matching prod drives this

Row Details

• M4: Time to fix depends on team SLAs and whether the fix is code or platform change; track by linking deny event IDs to ticket resolution times.
• M6: Platform-managed clusters should aim to label new namespaces automatically; track unlabeled islands as gaps.

Best tools to measure Pod Security Admission

Tool — Kubernetes audit logs

• What it measures for Pod Security Admission: Admission denials, warnings, and overall audit trail.
• Best-fit environment: Any Kubernetes cluster with audit logging enabled.
• Setup outline:
• Enable audit policy with events for admissions.
• Configure log sink to central log store.
• Parse audit events for PSA plugin subjects.
• Strengths:
• Canonical source for admission events.
• Useful for forensic analysis.
• Limitations:
• Verbose; needs aggregation and retention planning.
• Can be heavy to query in large clusters.

Tool — Prometheus + exporters

• What it measures for Pod Security Admission: Custom metrics derived from events and controllers.
• Best-fit environment: Clusters with Prometheus monitoring.
• Setup outline:
• Export counts of admission events to Prometheus.
• Create counters for warn/deny/allow.
• Build alerting rules and dashboards.
• Strengths:
• Flexible SLI/SLO monitoring.
• Integrates with alerting and dashboards.
• Limitations:
• Requires instrumentation to translate events into metrics.
• Possible cardinality issues.

Tool — Logging platform (ELK/Fluent/Cloud logs)

• What it measures for Pod Security Admission: Aggregated admission logs, warnings, and related pod spec snapshots.
• Best-fit environment: Clusters sending logs to centralized logging.
• Setup outline:
• Ship kube-apiserver and audit logs.
• Create parsers for PSA events.
• Create saved searches and alerts.
• Strengths:
• Rich search and forensic capability.
• Correlate with other events for incidents.
• Limitations:
• Cost and retention planning.
• Query performance at scale.

Tool — GitOps reconciliation dashboards

• What it measures for Pod Security Admission: Drift between declared namespace labels and actual cluster state.
• Best-fit environment: GitOps-managed clusters.
• Setup outline:
• Track namespace resources in Git.
• Alert on unlabeled namespaces or reconciliation failures.
• Strengths:
• Prevents policy drift.
• Automates remediation via sync.
• Limitations:
• Only as good as declared config; manual namespaces can bypass.

Tool — Policy engines telemetry (Gatekeeper/Kyverno)

• What it measures for Pod Security Admission: Custom policy violations that complement PSA.
• Best-fit environment: Clusters using Gatekeeper or Kyverno.
• Setup outline:
• Enable audit mode.
• Collect constraint violations and counts.
• Strengths:
• Rich context for why pods were denied.
• Can provide remediation hints.
• Limitations:
• Overlap with PSA may cause duplicate signals.
• Complexity in writing policies.

Recommended dashboards & alerts for Pod Security Admission

Executive dashboard

• Panels:
• Cluster-level deny and warn rates (trend).
• % namespaces with enforce labels.
• Number of blocked deployments in last 30 days.
• Why:
• Provide stakeholders a quick health view of policy adoption and risks.

On-call dashboard

• Panels:
• Real-time admission denies for last 15 minutes.
• Top namespaces with most denies.
• Recent events with pod spec snippets.
• CI failure vs cluster deny mismatches.
• Why:
• Fast triage for developers and platform on-call.

Debug dashboard

• Panels:
• Detailed audit log sample search box.
• Recent warn events with full pod spec.
• Pod controller rollout status for affected deployments.
• Namespace label and annotation table.
• Why:
• Support problem resolution and investigation.

Alerting guidance

• What should page vs ticket:
• Page: Sudden spike in denies affecting production namespaces or critical services.
• Ticket: Individual developer deployment denials in non-critical namespaces.
• Burn-rate guidance:
• Use error budget approach when changing policy enforcement modes; avoid paging for gradual warn spikes.
• Noise reduction tactics:
• Group similar denies into single alerts per namespace.
• Suppress warnings for known legacy apps during migration windows.
• Deduplicate by pod template hash to avoid repeated noise.

---

Implementation Guide (Step-by-step)

1) Prerequisites – Kubernetes cluster version that includes PSA support or vendor documentation confirming availability. – Cluster admin privileges to set audit logging and admission configuration. – GitOps or configuration management to maintain namespace labels. – Monitoring and logging stack for events and metrics.

2) Instrumentation plan – Instrument audit logs to capture PSA warnings and denies. – Export metrics for allow/warn/deny counts. – Track namespace label drift and pod spec snapshots for denied pods.

3) Data collection – Configure audit policy for admission events and send to central logging. – Scrape PSA-related metrics into Prometheus or equivalent. – Store pod spec snapshots for denied pods for postmortems.

4) SLO design – Define SLOs for acceptable deny rates and remediation times. – Use environment-specific targets (e.g., stricter for prod). – Allocate error budget for policy rollouts.

5) Dashboards – Create executive, on-call, debug dashboards as described above. – Provide links from denies to pod spec context and source (CI job, git commit).

6) Alerts & routing – Page when critical services impacted. – Ticket for developer-level denials. – Route to platform SRE for system namespace issues; route to owning team for app namespace denials.

7) Runbooks & automation – Create runbooks for common denial reasons: missing runAsNonRoot, privileged set, hostPath use. – Automate common remediations where safe (e.g., add runAsNonRoot labels) using mutating controllers with caution.

8) Validation (load/chaos/game days) – Perform game days: intentionally create policy violations to validate alerts and runbooks. – Use CI preflight in staging to validate PSA behavior under load. – Chaos tests: ensure PSA remains available during API server failover.

9) Continuous improvement – Review audit logs weekly to find patterns. – Update templates, onboarding docs, and CI checks based on denial trends. – Gradually tighten enforcement following successful migrations.

Pre-production checklist

• Ensure PSA plugin is enabled and configured.
• Label staging namespaces appropriately and test deny/warn behaviors.
• Configure audit logging and metric export.
• Run CI preflight to replicate production-sized workloads.

Production readiness checklist

• Labels applied to all relevant namespaces.
• Dashboards and alerts configured.
• Runbooks published and on-call trained.
• Error budget allocated for policy rollouts.

Incident checklist specific to Pod Security Admission

• Identify affected namespaces and workloads.
• Collect relevant audit events and pod specs.
• Determine whether change was planned (policy rollout) or unexpected.
• Apply fallback (temporarily set warn or adjust policy) only after coordination.
• Document remediation and update policies/guides.

---

Use Cases of Pod Security Admission

1) Multi-tenant SaaS cluster – Context: Hosting multiple customers on shared cluster. – Problem: Tenants may accidentally or maliciously use hostPath or privileged containers. – Why PSA helps: Enforces restricted policies per tenant namespace. – What to measure: Deny rates by tenant, number of insecure specs attempted. – Typical tools: PSA + RBAC + network policies.

2) Developer self-service platform – Context: Developers create namespaces and deploy apps. – Problem: Insecure defaults cause drift and incidents. – Why PSA helps: Baseline enforcements ensure safer defaults. – What to measure: % namespaces with baseline/restricted labels. – Typical tools: GitOps bootstrap + PSA.

3) Compliance enforcement for regulated workloads – Context: Workloads subject to compliance audit. – Problem: Need consistent enforcement of least privilege. – Why PSA helps: Provides enforceable checks at admission. – What to measure: Audit coverage and denied non-compliant pods. – Typical tools: PSA + audit logs + SIEM.

4) CI/CD preflight validation – Context: CI pipelines deploy to test clusters before production. – Problem: Production denials not caught in CI cause blocked deploys. – Why PSA helps: Mirror PSA in staging to catch issues earlier. – What to measure: CI-to-cluster mismatch rate. – Typical tools: Staging clusters with PSA + CI jobs.

5) Platform onboarding for new teams – Context: New teams onboard into platform. – Problem: Inconsistent namespace setup leads to insecure deployments. – Why PSA helps: Enforce labels and guardrails automatically. – What to measure: Time to compliance after onboarding. – Typical tools: Namespace-provisioning automation + PSA.

6) Legacy app migration – Context: Move legacy workloads to modern cluster. – Problem: Legacy apps require relaxed privileges. – Why PSA helps: Use warn mode to identify required exceptions and plan remediation. – What to measure: Number of exceptions and migration time. – Typical tools: PSA warn mode + mutation tools for temporary exemptions.

7) Incident containment after breach attempt – Context: Suspicious behavior traced to a misconfigured pod. – Problem: Need to prevent further risky deployments. – Why PSA helps: Enforce restricted policies and block similar future pods. – What to measure: Reduction in similar risky pod creations. – Typical tools: PSA + runtime agents.

8) Cost-conscious autoscaling – Context: Unbounded pods without limits cause node pressure. – Problem: Lack of limits lead to noisy neighbor and cost spikes. – Why PSA helps: Enforce resource requests and limits as part of policy checklist. – What to measure: Number of pods missing limits; node OOM events. – Typical tools: PSA + quota controllers + cost monitoring.

---

Scenario Examples (Realistic, End-to-End)

Scenario #1 — Kubernetes: Enforcing restricted for production workloads

Context: Production namespaces host critical microservices.
Goal: Prevent pods from running as root and using privileged capabilities.
Why Pod Security Admission matters here: Blocks risky pod specs before they start, reducing attack surface.
Architecture / workflow: Platform applies restricted enforcement labels to prod namespaces; CI deploys into staging with warn mode.
Step-by-step implementation:

1. Enable PSA in cluster and configure audit logging.
2. Label production namespaces with pod-security.kubernetes.io/enforce=restricted.
3. Label staging with warn=restricted and run CI test deployments.
4. Create runbooks for common denies.
5. Provide developer docs and fix templates (add runAsNonRoot). What to measure: Deny rate in production, time to remediate dev denials, number of privileged pods blocked.
   Tools to use and why: PSA, audit logs, Prometheus for metrics, CI mirror cluster.
   Common pitfalls: Legacy images expect root; require image rebuilds or UID mapping.
   Validation: Test deploying a pod with privileged flag and observe deny audit event.
   Outcome: Reduced privilege-related incidents and clearer developer guidance.

Scenario #2 — Serverless/managed-PaaS: Validating functions in managed clusters

Context: Platform provides managed serverless namespaces backed by Kubernetes.
Goal: Ensure function pods do not request host access or privileged capabilities.
Why Pod Security Admission matters here: Keeps managed runtimes constrained without heavy custom policy work.
Architecture / workflow: Managed namespaces labeled baseline; functions deployed via control plane create pods subject to PSA.
Step-by-step implementation:

1. Apply baseline labels to managed function namespaces.
2. Add CI checks that validate function runtime images conform.
3. Monitor warn events for new function types.
4. Escalate to restricted for high-sensitivity tenants. What to measure: Warn and deny rates per tenant; function failure incidents.
   Tools to use and why: PSA, logging, platform API to manage namespace labels.
   Common pitfalls: Platform-generated sidecars may require specific capabilities; adjust templates.
   Validation: Deploy a function that uses hostPath and confirm denied in baseline or restricted.
   Outcome: managed functions run with predictable security posture.

Scenario #3 — Incident-response/postmortem: Denied pod led to outage investigation

Context: A critical service failed to roll out after a policy change; production pods were denied.
Goal: Rapidly restore service and prevent recurrence.
Why Pod Security Admission matters here: Policy change caused unexpected denials; PSA audit events provide evidence.
Architecture / workflow: SRE receives alarm for failed rollout, inspects audit logs and PSA denies.
Step-by-step implementation:

1. Page platform on-call for denied production rollouts.
2. Collect deny audit events and pod specs; identify the policy rule triggered.
3. If necessary, temporarily switch namespace to warn mode to allow rollback.
4. Patch deployment manifests to comply and restore enforce mode.
5. Perform postmortem and update CI to catch similar issues. What to measure: Time to recovery, frequency of denies from policy changes.
   Tools to use and why: Audit logs, Prometheus alerts, runbook.
   Common pitfalls: Rolling back policy without communication leads to drift.
   Validation: Restore deployment and confirm pods running; update SLOs.
   Outcome: Faster diagnosis and update of deployment pipelines to prevent repeat.

Scenario #4 — Cost/performance trade-off: Resource limits enforced by PSA with autoscaler

Context: Autoscaler scaling app creates too many unbounded pods using lots of resources.
Goal: Ensure pods have requests/limits to maintain node stability and control costs.
Why Pod Security Admission matters here: Denies or warns on pods without limits, protecting cluster capacity.
Architecture / workflow: Baseline policy combined with quota controllers; autoscaler configured with safe headroom.
Step-by-step implementation:

1. Set baseline or custom PSA checks that require resource limits.
2. Adjust HPA scaling policies to consider resource requests.
3. Monitor node OOM events and pod evictions.
4. Train teams on sizing and provide templates with sensible defaults. What to measure: Number of pods without limits, node OOM/kublet evictions, cost per namespace.
   Tools to use and why: PSA, resource quota, metrics server, cost monitoring.
   Common pitfalls: Overly strict limits cause throttling and latency spikes.
   Validation: Launch a test that would previously cause OOM and confirm blocking or mitigation.
   Outcome: Better resource stability and predictable cost behavior.

---

Common Mistakes, Anti-patterns, and Troubleshooting

List of mistakes with Symptom -> Root cause -> Fix (15–25 items, includes observability pitfalls)

1. Symptom: Pod creation denied unexpectedly -> Root cause: Namespace lacks warn label but expected to be permissive -> Fix: Reconcile namespace labels and use GitOps.
2. Symptom: CI jobs fail while developers can deploy manually -> Root cause: CI uses different staging cluster missing PSA config -> Fix: Mirror PSA in staging and add preflight CI checks.
3. Symptom: Excessive warn events -> Root cause: PSA set to warn cluster-wide during rollout -> Fix: Narrow warn scope, schedule phased rollout.
4. Symptom: Missing audit entries for denies -> Root cause: Audit policy not capturing admission events -> Fix: Update audit policy to include requestStages and admission events.
5. Symptom: Developers unaware of denials -> Root cause: No integration between PSA warnings and developer workflows -> Fix: Surface denial messages in CI and PR checks.
6. Symptom: High deny rate in production -> Root cause: Sudden policy change without staggered rollout -> Fix: Rollback policy to warn, communicate changes, plan phased enforce rollout.
7. Symptom: Duplicate alerts for same denial -> Root cause: Multiple monitoring rules and log parsers firing -> Fix: Deduplicate at alerting platform and centralize rules.
8. Symptom: Legacy apps break on enforcement -> Root cause: Apps require root or hostPath -> Fix: Create migration plan or scoped exemptions and rebuild images to run non-root.
9. Symptom: No metric for deny rate -> Root cause: No instrumentation exporting PSA events to metrics -> Fix: Add event-to-metric exporter or use audit log exporter.
10. Symptom: Policy drift across clusters -> Root cause: Manual namespace label changes -> Fix: Enforce via GitOps and reconciler.
11. Symptom: Policy conflicts with mutating webhook -> Root cause: Webhook mutates fields PSA expects -> Fix: Order webhooks appropriately and test interactions.
12. Symptom: On-call overloaded with non-critical pages -> Root cause: Alerts not tuned by namespace severity -> Fix: Route alerts by namespace and only page for critical namespaces.
13. Symptom: Slow triage due to missing context -> Root cause: Deny event lacks pod spec snapshot in logs -> Fix: Capture and store rejected pod specs with correlation IDs.
14. Symptom: False sense of security -> Root cause: Teams assume PSA covers all security needs -> Fix: Document PSA scope and integrate runtime tools.
15. Symptom: Unclear ownership of PSA configs -> Root cause: Platform and security teams both think the other owns labels -> Fix: Define ownership RACI and manage via GitOps.
16. Symptom: High cardinality metrics after instrumenting events -> Root cause: Creating unique label combinations per pod -> Fix: Aggregate or hash high-cardinality fields.
17. Symptom: Missing alerts during API-server outages -> Root cause: Admission controller kept but logs not forwarded -> Fix: Ensure high-availability logging sinks and fallback.
18. Symptom: Too many exemptions -> Root cause: Teams request relaxed policies without remediation plan -> Fix: Time-box exemptions and track via tickets.
19. Symptom: Difficulty auditing historical denies -> Root cause: Short audit retention -> Fix: Increase audit log retention or export to long-term store.
20. Symptom: Slow deployments after enabling PSA -> Root cause: Increased manual fixes and back-and-forth -> Fix: Provide templates, mutation webhooks for safe autofix.
21. Symptom: Observability blind spots -> Root cause: Events not correlated to CI commits -> Fix: Add commit metadata to deployment annotations.
22. Symptom: Denials triggered by sidecars -> Root cause: Sidecar injection modifies securityContext -> Fix: Align sidecar templates with PSA expectations.
23. Symptom: Divergent policies between regions -> Root cause: Manual per-cluster configuration -> Fix: Centralize policy config and replicate via GitOps.
24. Symptom: Missing owner for broken rollout -> Root cause: No mapping from namespace to team -> Fix: Tag namespaces with owner labels and integrate with on-call routing.

Observability pitfalls included above: missing audit capture, high-cardinality metrics, lacking pod spec snapshots, short retention, and missing CI correlation.

---

Best Practices & Operating Model

Ownership and on-call

• Platform team owns cluster-level PSA configuration and namespace bootstrapping.
• Security defines desired policy profiles and SRE helps with SLOs and alerts.
• Assign on-call rotation for platform incidents; application teams own fixes for denied pods in their namespaces.

Runbooks vs playbooks

• Runbooks: Short procedural steps for common immediate fixes (e.g., unlock deployment by switching warn temporarily).
• Playbooks: Higher-level incident response steps involving multiple teams and remediation paths.

Safe deployments (canary/rollback)

• Roll out policy changes gradually using canary namespaces and track denies.
• Use staged label changes from warn to enforce with automated rollback if deny rate exceeds threshold.

Toil reduction and automation

• Automate namespace label creation via GitOps templates.
• Provide mutating controllers only when safe for small, well-tested transformations.
• Automate common remediation PR creation when deny events show straightforward fixes.

Security basics

• Use least privilege for system and app namespaces.
• Ensure image signing and registry policies are applied separately; PSA focuses on pod spec hardening.
• Combine PSA with network policy and runtime detection tools.

Weekly/monthly routines

• Weekly: Review top denied pod reasons and update templates or docs.
• Monthly: Audit namespace label coverage and review policy-related incidents.
• Quarterly: Reassess profile levels and adjust based on risk posture.

What to review in postmortems related to Pod Security Admission

• Whether denies were expected from planned changes.
• Time to detection and remediation of PSA-related incidents.
• CI/CD gaps that allowed mismatches.
• Whether policy changes were communicated and staged correctly.

---

Tooling & Integration Map for Pod Security Admission (TABLE REQUIRED)

ID	Category	What it does	Key integrations	Notes
I1	Audit logging	Captures admission events and denies	SIEM, log store, Prometheus exporters	Central for forensic analysis
I2	Monitoring	Exposes SLI metrics for allow/warn/deny	Prometheus, alert manager	Needs event to metric translation
I3	Policy engine	Custom policies and mutation	Gatekeeper, Kyverno	Complements PSA for advanced rules
I4	GitOps	Declarative labels and baseline configs	Flux, Argo CD	Prevents policy drift
I5	CI/CD	Preflight tests and staging deploys	Jenkins, GitLab CI, GitHub Actions	Catch denials early
I6	Runtime security	Detects runtime anomalies post-admission	EDR, runtime agents	PSA does not replace runtime tools
I7	Logging platform	Searchable admission audits	ELK, cloud logs	Useful for debugging
I8	Namespace manager	Automates namespace creation and labels	Platform controllers	Prevents unlabeled islands
I9	Alerting platform	Routes pages and tickets	PagerDuty, Opsgenie	Tie to namespace severity
I10	Cost tooling	Correlates resource issues to cost	Cost analytics tools	Shows cost implications of missing limits

Row Details

• I1: Ensure audit logs include request and response stages and are shipped to a durable store for at least the investigation window.
• I2: Implement a stable exporter to avoid high-cardinality labels when converting events to metrics.
• I3: Use Gatekeeper/Kyverno where business needs require custom validation or automatic remediation.
• I4: GitOps ensures namespace labels and PSA configuration are versioned and auditable.
• I8: Namespace managers prevent accidental creation of ungoverned namespaces that bypass PSA.

---

Frequently Asked Questions (FAQs)

H3: What Kubernetes versions support Pod Security Admission?

Most modern Kubernetes versions include PSA as a built-in plugin; exact availability varies by distribution and provider.

H3: Is Pod Security Admission enabled by default?

Varies / depends.

H3: Can PSA mutate pod specs to fix violations?

No. PSA is a validating admission plugin and cannot mutate objects.

H3: How does PSA compare to Gatekeeper or Kyverno?

PSA provides curated checks; Gatekeeper/Kyverno offer custom policies and mutation capabilities.

H3: Can PSA check container image vulnerabilities?

No. PSA does not inspect image contents or CVEs.

H3: How do I roll out PSA without breaking production?

Start in warn mode, use a staging cluster, and gradually enforce with canary namespaces.

H3: Can I exempt namespaces from PSA?

Yes, by labeling them appropriately or not labeling; but exemptions should be tracked and time-boxed.

H3: Will PSA prevent runtime exploits?

No. PSA reduces attack surface at admission but runtime protection tools are required for detection and response.

H3: How do I monitor PSA denies?

Collect audit logs and export deny/warn events to metrics and dashboards.

H3: What are the standard PSA levels?

Privileged, baseline, restricted — these are standard tiers for common checks.

H3: Can PSA block sidecar-injected pods?

Yes, if injected sidecars introduce securityContext fields that violate the policy.

H3: How to handle legacy apps that require privileged access?

Use warn mode and plan migration; consider scoped exemptions and rebuild images where possible.

H3: Does PSA work with serverless platforms?

Yes, PSA enforces pod-level constraints regardless of the control plane that creates pods.

H3: How should alerts be routed for PSA issues?

Page for critical namespaces and production impact; create tickets for developer namespace denials.

H3: Are deny audit logs sufficient for compliance audits?

They are an important part but often need to be combined with other evidence like CI attestations and runtime logs.

H3: Can PSA be bypassed?

Potentially if cluster admins change admission config or namespaces are unlabeled; manage via GitOps and RBAC.

H3: How long should we keep PSA audit logs?

Retention depends on compliance and incident response needs; plan retention based on regulatory and forensic requirements.

H3: What’s a good SLO for PSA enforcement?

Depends on context; example targets provided earlier are starting points, not universal mandates.

---

Conclusion

Pod Security Admission is a pragmatic, low-friction mechanism for enforcing pod-level security guardrails in Kubernetes. It is most effective as an early gate in a layered security approach that includes CI checks, policy engines for custom rules, and runtime protection systems. Adopt PSA incrementally: start in warn mode, validate via CI and staging, then enforce per-namespace while tracking SLIs and running game days.

Next 7 days plan (5 bullets)

• Day 1: Audit namespace labels and enable audit logging for admission events.
• Day 2: Configure monitoring to count PSA allow/warn/deny events.
• Day 3: Set PSA to warn in non-prod and test CI preflight deployments.
• Day 4: Draft runbooks for the top 5 deny reasons and notify teams.
• Day 5–7: Roll out baseline enforcement to a small set of non-critical namespaces and measure impact.

---

Appendix — Pod Security Admission Keyword Cluster (SEO)

• Primary keywords
• Pod Security Admission
• Kubernetes Pod Security Admission
• PSA Kubernetes

• pod-security admission

• Secondary keywords

• pod security enforcement
• pod security best practices
• Kubernetes admission controller
• pod-security.kubernetes.io labels

• baseline restricted privileged profiles

• Long-tail questions

• what is pod security admission in kubernetes
• how to enable pod security admission
• pod security admission deny vs warn
• how to audit pod security admission events
• psa vs gatekeeper vs kyverno differences
• how to rollout pod security admission safely
• why are my pods being denied by pod security admission
• how to integrate pod security admission with CI
• pod security admission audit log retention
• pod security admission metrics to monitor
• how to migrate legacy apps to comply with pod security admission
• pod security admission runbook example
• enabling restricted profile kubernetes
• pod security admission and namespace labels
• pod security admission common deny reasons
• how to troubleshoot pod security admission denies
• pod security admission best practices for multitenancy
• pod security admission and runtime security
• pod security admission for managed kubernetes

• pod security admission and policy drift

• Related terminology

• admission controller
• audit logs
• pod spec
• securityContext
• runAsNonRoot
• capabilities
• privileged container
• hostPath
• hostNetwork
• hostPID
• seccomp
• SELinuxOptions
• AppArmor
• mutating webhook
• validating webhook
• Gatekeeper
• Kyverno
• PodSecurityPolicy
• GitOps
• CI preflight
• namespace bootstrap
• resource limits
• runtime protection
• denial audit event
• policy enforcement mode
• warn mode
• enforce mode
• audit mode
• policy tiers
• restricted profile
• baseline profile
• privileged profile
• policy rollout
• error budget for policy changes
• canary namespace
• namespace manager
• observability for admission
• deny rate metric
• remediation automation
• policy drift detection
• postmortem for admission denies
• compliance and admission controls