AWS Emulator Was Written by Agents. Humans Did Not Review It

On November 11, 2026, AWS released AWS Emulator, a local stand-in for AWS that you can run on a laptop, a dev server, or in CI. The AWS CLI and the SDKs talk to it instead of the cloud. No AWS account is required, and there is no service fee. The part that matters for anyone shipping software with coding agents is the sentence in AWS's own account of the build: the emulator was written entirely by autonomous agents. The team has not written or reviewed any of the code.

That is a stronger claim than "agents helped." It is a product, under a CC0 license, updated continuously, whose authors of record are the agents. Senior principal engineer Matthew Goldberg described the guardrails that made the team willing to ship it. Those guardrails are the story. The emulator is the artifact they produced.

If your own agents are opening pull requests, this is the public example of what "the agents shipped it" looks like when a cloud vendor does it on purpose. It is also the public example of what still fails when the tests are incomplete and nobody read the diff. Who is allowed to hold the credentials while a swarm does that work is a separate problem, covered in what an agent swarm is allowed to touch. How to structure a codebase so an agent can change it without losing the product is in how to refactor an agent-built codebase. This note is the test loop AWS published, and what a green run on that loop does not prove.

What they actually released

AWS Emulator is built from the public AWS API models. Reporting on the first release counted 438 API models covering more than 200 services, with one Rust crate per service. You point the CLI or an SDK at the local process. Calls that would have gone to AWS hit the emulator instead.

It runs as RPM or Debian packages on Linux x86_64 and aarch64, or in Docker on macOS and Windows. The Docker path is capped at a 2 GiB memory limit. A source build wants on the order of 100 GB of free disk. Those are workstation constraints, not a claim about fidelity. A laptop that can start the process has not thereby reproduced us-east-1.

The source is on GitHub under CC0, which means there is no copyright license standing between you and the code. CC0 does not mean the code is correct, and it does not mean AWS will behave the same way when the same request reaches the real service. The agents' backlog of known gaps is public, in a folder named things-that-do-not-work. A vendor that publishes the list of failures is easier to reason about than a vendor that hides them. The list is still a list of failures.

Goldberg is explicit that the human team did not review the lines. The trust is placed in the system around the agents: tests derived from the API models, checked once against real AWS, then replayed forever against the emulator, plus a merge gate that refuses a change when the build, the coding-standard review, the security review, or the fidelity suite fails. Without that gate, he writes, they would not trust what the agents are building. The gate is the product decision. The model is not.

Three teams, on purpose

The agents run on Amazon Bedrock AgentCore, and the work is split into three teams because bounded tasks went better than one agent asked to "build AWS."

The first team owns the fidelity suite. A test is a real AWS scenario, not a mocked happy path. Goldberg's example for Amazon SQS is the obvious one: create a queue, send a message, receive it, acknowledge it, delete the queue. Agents devise these tests from the publicly released API models, then verify each new test against real AWS before it is allowed into the suite. Once it is in, it can be replayed without calling AWS again. AWS says the suite is tens of thousands of tests and that it keeps growing as agents find new scenarios the models allow. That growth is the point. A fixed suite freezes last month's idea of the service. A suite that only grows when a test has already passed against the live cloud is a different instrument.

The second team replays that suite against the emulator and files what diverges. The repository did not start as a faithful copy. Goldberg says the initial system generated an empty web server that could accept requests for all 438 API models and then did nothing. The first issues were prompts of the form "build a first version of Amazon SQS." After a service exists, the finders replay the fidelity tests, file the failures, and are told to group failures that may share a root cause. An issue is a bounded defect, not "make Lambda perfect."

The third team takes one issue each. The agent checks out the code, looks for the cause, edits until that issue is addressed, and submits the change to the gate. If the gate fails, the change goes back to an agent. If it passes, it merges to the main line. Goldberg reports hundreds of agents working at once and thousands of issues fixed per day. The release pipeline uses AWS's own developer tools, with CodeCommit as the agents' repository. GitHub is where the code lands last, not where the agents commit first.

The split is the part worth copying, and it is also the part most teams skip. One agent asked to design, test, and ship a service will wander. Three roles with a test that can fail in one direction — the live cloud and the emulator disagreed — give the model a score it cannot talk its way out of. The score is only as good as the test. A test that never checks who is allowed to call the API will stay green while authorization is wrong.

What the merge gate is, and what it is not

The gate blocks a change that does not compile. That is the easy bar. Goldberg also describes agents that review the change against coding standards, agents that verify the code is secure, and a fidelity check that the change did not break tests the emulator had already been passing. Several of those security reviewers are adversarial. A change that fails any of them is sent back. A change that passes all of them merges.

Read that list against the claim that no human reviewed the code. The reviewers are agents. An agent reviewing an agent's patch can catch a class of mistakes the tests name, and it can miss a class of mistakes the tests do not name. "Verify the code is secure" is a prompt with a verdict, not a description of a sandbox, a threat model, or a person who would refuse a clever patch that still ships customer data to the wrong place. The gate is real. It is not a security audit in the sense a buyer of the emulator should assume.

The same limit shows up in how fast the system moves. Thousands of fixes a day is a throughput number. It is also how a wrong test becomes permanent. If the fidelity test encodes a misunderstanding of the API, every agent that "fixes" the emulator is fixing it toward the misunderstanding. The test was checked against real AWS once, when it was written. A queue that sends and receives a message does not prove that a queue policy rejected the caller who should have been rejected.

The fidelity number is the product limit

MarkTechPost's report on the release gives a scored snapshot for v1.0.322: 24,243 passes out of 42,857 fidelity tests, 56.6 percent. The spread inside that average is the number that should change how you use the tool. AWS KMS was at 93.9 percent, 291 of 310 tests. Amazon SQS was at 89.8 percent. AWS CloudFormation was at 15.8 percent, 414 of 2,612 tests.

An average of 56.6 percent is not "half the cloud works." It is a mix of services an agent can practice against today and services where most of the suite still fails. If your app's critical path is SQS, the emulator is a serious local loop. If your critical path is a CloudFormation deployment, a green test on the emulator is a weak statement. You have to know which service your job actually called.

The suite is also scored. Tests the project has not scored yet are not in that fraction. A service can look better or worse as the agents add tests. Fidelity will move between the afternoon you read the README and the afternoon your CI pulls a new build. Pin the version you ran when you decided a service was good enough, the same way you would pin a base image. "Latest" is a different product every day when thousands of patches merge daily.

IDC analyst Devin Pratt, quoted in coverage of the release, listed what this kind of emulator does not pretend to do yet. It does not simulate throttling. It does not simulate an outage. It does not reproduce the difference between regions, where a feature exists in one and not another. It does not sandbox the code it runs. Those are not footnotes if your production incidents are throttling, a partial outage, or a region you do not deploy to. The emulator will not show you that failure. A test written only against the emulator will not fail when the real service would.

There is a public folder, things-that-do-not-work, for the gaps the agents already know about. Read it for the services you depend on before you add the emulator to CI. A known gap that your test does not touch is a gap you have chosen to ignore. A known gap that your test does touch, while CI stays green, means your test is not the fidelity test. It is a smaller test with a friendlier name.

IAM is the gap that will ship a false pass

The behavioral miss that showed up immediately in public discussion is authorization. The emulator does not check IAM the way real AWS does. A request AWS would reject can succeed locally. On Hacker News that was summarized in one line: a request that AWS rejects can sometimes succeed in the emulator. That is unfortunate if you were using the tool to find out whether a role is too wide. It is dangerous if you were using the tool to find out whether a role is tight enough.

IAM is not a polish item. An agent that develops against an emulator where denied calls succeed will write the call anyway. The test will pass. The first time the role is a real one, the call fails, or you grant the role whatever the agent needed and the over-permission becomes the design. LocalStack and hand-written mocks have had this hole for years. The honest use is to say so in the test report: this job did not check authorization.

If you want the emulator to teach the agent about IAM, you need a separate check against an account where the role is the production role, or a deliberately narrower one. That check needs credentials the agent should not hold for the rest of the day. The local process is the loop you use so the agent can iterate without that credential.

What the emulator is good for

Use it when the cost of a feedback loop is the problem. An agent that deploys to a real account on every attempt will create resources, trip quotas, and leave a bill. A local process that speaks the SDK's wire protocol is the loop for the hours when the agent is still shaping the call: wrong parameter names, missing pagination, a client pointed at the wrong endpoint. Tens of thousands of API calls belong on a laptop. They do not belong on a production account.

Keep the emulator off the list of environments that may merge to production by themselves. A pull request can require the emulator job and still require a second job, smaller and slower, against a real account in a sandbox. The second job is allowed to be boring. Create the resource, call it as the role the app will use, assert the denial you expect, delete the resource. If that job is red, the emulator job being green is irrelevant.

Pin the emulator version in CI. A project that floats to the latest build will change behavior under you because the agents merged a thousand fixes overnight. Read the diff of the version bump the way you would read a dependency bump. Do not point production traffic at the process. It is a test double, and the Docker build's memory cap is a development limit.

What your agents do not have, even if AWS's did

The flattering reading of this release is that autonomous software engineering is now practical, because a major vendor shipped a large codebase no human wrote. The accurate reading is narrower. Autonomous software engineering was practical here because the spec was already public, the oracle was the production cloud, the work could be split into one issue per agent, and a gate could reject patches without asking the model to be honest. Remove any of those and the same swarm is a fast way to merge unreviewed code.

Most generated products have none of the four. The spec is a chat. The oracle is the same model that wrote the code, perhaps asked to write a test afterward. The issues are "build the app." The gate is a person who clicks merge because the screenshot looks right, or there is no person because the agent has permission to push. AWS's result is not evidence that this second setup works. It is evidence that the first setup can produce something useful and still be wrong on authorization and on most of CloudFormation.

If you are going to copy one piece, copy the oracle. Pick a behavior a real system can confirm: a Stripe webhook you can replay, a row you can count, a role that must be denied. Make the agent chase that, one failure at a time, and refuse the patch when the confirmation fails. Do not copy the part where no one reads the code and then hope your tests are as strict as a cloud vendor's API models. The architecture question of how those tests survive the next agent session is the subject of how an agent-built codebase stays changeable. The credential question, which this emulator does not answer, is who holds the keys while the swarm runs.

What to check before a green run counts

Do this for the services your app actually calls, not for the README's list of 200.

1. Name the emulator version in CI. If the version is floating, you do not know which fidelity snapshot you tested.
2. Read things-that-do-not-work for those services. A gap on your path means the local pass does not speak to that behavior.
3. Look up the scored fidelity for those services if the project publishes it. A service at SQS's level and a service at CloudFormation's level are not the same tool.
4. Mark authorization as untested unless a separate job uses a real role against real AWS. A local allow is not an IAM decision.
5. Add one test, against the real service, for throttling or a missing region feature if your incident history includes either. The emulator will not invent that failure for you.
6. Keep production credentials out of the agent that is allowed to hammer the emulator. The local process does not need them. A config that mounts them anyway has mixed two trust levels.
7. Require a person, or a real-account job a person owns, before a change that only passed the emulator can deploy.

If you cannot say which of those seven are true for the repo agents are already merging to, the green run is a comfort. It is not evidence.

How Techtribe checks an agent-built test loop

We read the repo and the CI config and separate the checks a real system confirmed from the checks a stand-in confirmed. You get that split in writing, including whether a green emulator or LocalStack run is allowed to merge. We do not treat a local pass as proof the live cloud would allow the call.

A one-time pass fits when the agent is a developer's editor and the suite is a single repo. A Dedicated Team fits when agents merge to that repo every day and someone has to keep the oracle honest after we leave. The code stays in your repo. The keys stay in your cloud. The scope call is 30 minutes with a founder, and it is where we say which of those you are in.

Bring the repo, the emulator or LocalStack config, and whether CI treats that green run as proof. If a production role is already on the machine that runs the agent, say that first.

The fixed scope for taking a generated app to production is a page per tool: Lovable, Bolt, v0, and Replit.