TL;DR
The AWS shared responsibility model gets taught as four separate jobs: access control, encryption, monitoring, hybrid connectivity. They’re the same job, run four times. Environments pass every one of those four reviews and still fail, because nobody checked whether the four copies of that job agree with each other. That gets worse, not better, once you count how many of the identities running through them aren’t human.
The reframe
The shared responsibility split sounds tidy. AWS handles the concrete, the racks, the hypervisor, the global backbone. You handle everything running on top of it. Most enterprise security training stops there and hands you a list: configure IAM, encrypt your data, turn on logging, secure your network connection. Four boxes to check.
Open any one of those boxes and the same question is sitting inside it. IAM policy: which identity can call which API. KMS key policy: which identity can call Decrypt. CloudTrail: which identity did what, when. Even a Direct Connect session, once the private circuit is up, still has to answer which identity is on the other end before anything crosses it. The shared responsibility line does not split into four controls. It splits into one control, identity, applied at four different layers of the stack.
That is the practitioner’s foundation this post is named for: not a AWS security checklist, but the one question every item on that checklist is actually asking.
The mental model
Four boxes on your side of the line, one gate underneath all of them. That is not a coincidence, it is the shared responsibility model working as designed. AWS cannot secure “your side” for you because your side is defined by who you let in, and only you know who that should be.
Walking the four boxes
Access control. IAM with role-based access control is the most direct version of the pattern: define a role, attach a policy, grant the minimum the role needs. AWS Managed Active Directory extends this outward, instead of provisioning identities twice, employees authenticate with the corporate credential they already have, and AWS Directory Service maps that into IAM’s trust model. This is federation, the same mechanism covered in “You’ve Been Doing Federation Since Day One,” just applied to your workforce instead of your workloads.
Encryption. KMS gets taught as a separate discipline from access control, with its own section in every AWS security course. It is not separate. A KMS key has a key policy, and that key policy is an IAM policy. Encrypting data does not remove it from the identity question, it adds a second identity gate in front of the first: to read the object you need s3:GetObject, and then you need kms:Decrypt on the key that wrapped it. Two locks, same kind of key.
Monitoring. CloudTrail’s real value is not that it logs API calls, it is that it logs whose API calls. Every entry is principal, action, resource, allowed or denied. Read a CloudTrail table looking for anomalies and you are doing identity threat detection whether or not anyone called it that. CloudWatch adds the operational layer on top, alerting when the identity pattern itself looks wrong, not just when the infrastructure does.
Hybrid connectivity. This is the one that looks like an exception. Site-to-Site VPN and Direct Connect are network-layer controls: IPsec tunnels, dedicated circuits, TLS for application traffic. They matter, and skipping them is a real mistake. But a private circuit answers “is this path protected,” not “who is allowed to use it.” Once the tunnel or the circuit is up, the question that decides what happens next is still which identity is on the other end, evaluated against the same IAM and Managed AD model as everything else. The network layer earns you a safer pipe. It does not replace the gate at the end of it.
Where four green checkmarks still fail
Physical security ran into this exact failure decades before AWS existed, just wearing different names. A large office building typically runs four separate access systems: badge readers on the doors, the safe’s combination roster, the front desk’s visitor log, and the parking garage’s gate list. Each system has its own owner. Each owner audits their own system and finds it clean. Nobody has ever sat down and compared the four rosters against each other, because no single job description covers all four. A departing employee’s badge gets deactivated the same day; HR is thorough about that part. Nobody remembers the safe combination roster is a separate spreadsheet, kept by facilities, updated on facilities’ schedule. Nine months later, that name is still on it.
AWS runs the identical failure with tighter names and a shorter blast-radius clock.
Take a role called lambda-nightly-billing-sync. It reads encrypted billing export objects from s3://acme-billing/exports/, decrypts them with a customer-managed KMS key, writes reconciled rows to a DynamoDB table, and publishes a completion event to SNS. At creation, its IAM policy is scoped to exactly those four resources: s3:GetObject on that one prefix, kms:Decrypt on that one key ARN, dynamodb:PutItem on that one table, sns:Publish on that one topic. It passes review the day it’s written, because it is written tightly.
Eight months later, compliance mandates annual key rotation for financial data. An engineer creates the replacement key, billing-cmk-2026, and finds a dozen roles and two cross-account consumers still referencing the old key’s ARN. Tracking down and updating every one of those IAM policies is a half-day of work queued behind three other tickets. Instead, the engineer adds one statement to the new key’s policy: Principal: "*", Effect: Allow, Action: kms:Decrypt. It’s scoped by an aws:PrincipalAccount condition to the account itself. The intent is to unblock the rotation deploy while the real per-role grants get written properly. The follow-up ticket to scope it down is still open at the next audit cycle.
That audit comes back clean on all four counts. IAM review pulls lambda-nightly-billing-sync’s policy and finds four scoped statements, nothing more. KMS review confirms the key is customer-managed with rotation enabled. It also checks for a wildcard principal. Most policy-as-code scanners are documented to flag an unconditioned wildcard by default. Whether they also catch one narrowed by a condition, the way this one is, is worth confirming against the live tool rather than assumed. This one carries an account-level condition, so it doesn’t trip. CloudTrail review confirms logging is on and retention meets policy. Four green checkmarks.
What actually changed: every principal in that AWS account, not just the billing sync role, can now call kms:Decrypt on billing-cmk-2026. Getting the ciphertext still requires s3:GetObject, which most principals in the account don’t have on that prefix, so the exposure isn’t wide open. But it is exactly as wide as whatever the S3 layer’s own IAM grants happen to allow, and most teams treat KMS as the real gate on financial data specifically because the S3 layer is usually looser. Widen the KMS side and the S3 layer, never designed to be anyone’s last line of defense, quietly becomes the only one.
IAM was not wrong. KMS was not wrong, by its own review’s own rule. The two reviews checked two different stories about the same key, and nobody checked that the stories still matched after the rotation.
That is what “one control, four layers” is actually warning you about. Four independent audits can each pass while the identity model drifts into four slightly different, unsynchronized copies of itself. The failure is not in any one box. It is in the seam between boxes that no single review owns.
The Principal Trace
The fix is not a fifth audit. It’s changing what the first four are already checking. Call it the Principal Trace: pick one real principal, a role, not an abstract policy class, and walk it through all four surfaces in the same sitting.
- Grant Check (IAM). What does the principal’s own policy say it can do? This is the review everyone already runs.
- Key Check (KMS). For every KMS key this principal touches, open that key’s own policy and find the principal by name. Not “encryption is on,” specifically: is this principal named, or does the statement fall back to a wildcard carrying only an account-level condition, the way
billing-cmk-2026did above? - Activity Check (CloudTrail / CloudWatch). Does CloudTrail actually show entries under this principal’s identifier, at a volume that matches what it’s supposed to do? Broad IAM grants next to near-zero CloudTrail activity usually mean one of two things: the role is unused (revoke it), or it’s being invoked through a path nobody accounted for. This check has its own blind spot, worth naming up front: S3 data events, the object-level
GetObject/PutObjectactivity a scenario like this one depends on, are not enabled by default in CloudTrail. They cost extra per event, and plenty of real accounts leave them off for high-volume buckets for exactly that reason. A role can run cleanly every night and still show near-zero CloudTrail volume, not because it’s unused, but because nobody’s watching the layer it actually touches. Confirm data events are on for the bucket in question before trusting what this check tells you. - Transport Check (VPN / Direct Connect). If this principal’s traffic crosses a hybrid connection, does the identity mapping on that connection agree with what IAM grants, or is the network layer trusting something coarser, like “anything from this IP range,” than the identity layer intends?
Run the Trace on one principal and you’re not auditing four controls anymore. You’re testing whether one identity model actually holds across four enforcement points. Run it on a handful of principals chosen for breadth, not depth: a human role, a CI role, a Lambda execution role, a cross-account role. Drift like the billing-cmk-2026 grant above usually surfaces in minutes, not in a compliance review six months later.
Where the Trace gets genuinely hard: break-glass
The Principal Trace has a blind spot, and it isn’t one you patch by tightening the heuristic. Most environments with real incident response have at least one break-glass role, something like incident-response-emergency, built to let a small on-call group decrypt customer data across any KMS key during an active security incident without routing each decrypt through the normal per-key grant process. Its IAM policy is intentionally broad. Its KMS reach is intentionally broader than any everyday role’s. And by design, its CloudTrail activity should sit at near zero outside of a live incident.
Run the Principal Trace against that role and it produces exactly the signature the Activity Check is built to catch: broad IAM, broad KMS reach, near-zero CloudTrail volume. Step 3 says revoke it. That’s precisely wrong for this role, and precisely right for a thousand other roles that carry the same shape for the ordinary reason: drift instead of design.
Legitimate, intentional divergence and accumulated drift produce identical evidence from inside the four surfaces. There’s no heuristic inside IAM, KMS, or CloudTrail that tells them apart, because the difference isn’t in what the role can do. It’s in whether a human decided it should be able to, on purpose, and wrote that decision down somewhere. The only fix is an out-of-band record of intent: a registry stating this role is supposed to look like this. Someone owns it. It gets reviewed on its own cadence. And it stays separate from the four surfaces it’s making an exception to. Which means the fix for the seam problem is itself a fifth surface, with its own owner and its own chance to drift out of date. That doesn’t break the four-surfaces model. It just means the model was never claiming to verify itself. Something outside it still has to vouch for what’s allowed to diverge, and that’s a judgment call, not a scan. This post doesn’t have a clean answer for how to make that registry trustworthy at scale. Naming the gap honestly beats papering over it with a heuristic that will eventually flag the wrong role and get ignored the next ten times it’s right.
Where this gets worse: non-human identities
This problem does not distribute evenly. It concentrates in service roles, CI runners, and Lambda execution roles, for a structural reason: a human role usually gets set up once, by one person, in one sitting, so all four surfaces are more likely to agree by default. A machine identity’s four surfaces get touched by different automation at different times. Terraform provisions the IAM role. A separate module grants KMS access, maybe written by someone else, maybe copied from a different environment. CloudWatch alarms get configured in a third pass, if at all. Nobody traces the principal end to end, because nobody owns end to end. Each surface has an owner. The identity does not.
That is the same shape of problem the Federation post named directly: there are far more non-human identities in any real environment than human ones. Exactly how many more depends on who’s counting and how. Palo Alto Networks’ 2026 Identity Security Landscape report puts the ratio at 109 machine identities for every human one; Veza’s 2026 State of Identity & Access report puts it at 17 to 1. The methodologies disagree by a factor of six. They don’t disagree on the direction: machine identities aren’t a large minority of any real AWS account, they’re most of it. And per Verizon’s 2026 Data Breach Investigations Report, credential abuse still turns up in roughly 39% of breach chains, the single most common technique tracked. The identities that pass through the four surfaces most often are exactly the identities with the fewest eyes on them.
Misconceptions worth killing directly
“Encryption is a separate control from access control.” It is access control with an extra step. A KMS key policy is an IAM policy. If you get access control right and encryption wrong, or the reverse, you still have one identity gate that is misconfigured, not two independent systems.
“A VPN or Direct Connect is the security boundary for hybrid environments.” It is a safer transport, not a boundary. The boundary is still whichever identity system decides what that transport is allowed to reach once it arrives.
“Monitoring is a separate discipline from IAM.” CloudTrail is IAM’s own decisions, replayed as a log. Reading it well is reading identity behavior, not a separate skill from designing the policies in the first place.
“If IAM is locked down, the rest doesn’t matter.” IAM only tells you what the front door allows. A misconfigured KMS key policy, an unmonitored principal, or a VPN identity mapping that is coarser than IAM’s is a second door IAM’s own review never checked.
Security note
Run the Principal Trace above whenever a new automation gets IAM access, not just at annual audit time. That is when the four surfaces are most likely to still be in sync, and cheapest to fix if they are not. Waiting for a compliance cycle means waiting for the drift to compound across every deploy in between, and machine identities compound faster than anyone is watching. For the identities this can’t catch by design, break-glass and other intentional exceptions, the fix isn’t a sharper check. It’s an owned, reviewed registry of what’s supposed to diverge and why.
