It Ran Before Import: The LiteLLM 1.107.0 Compromise

It Ran Before Import: The LiteLLM 1.107.0 Compromise


It Ran Before Import: The LiteLLM 1.107.0 Compromise

The dangerous version was not a typo in a dependency file. It was a release.

On October 21, 2026, LiteLLM 1.107.0 appeared on PyPI at 11:39 UTC and, minutes later, as container images on Docker Hub and GHCR. It was built by the project’s real release workflow. It carried provenance. The official GitHub tag for v1.107.0 did not contain the malicious files. The wheels and the images did.

That contradiction is the incident. The source the world was invited to read was clean. The artifact the world was invited to install was not. Between them sat a hyphen.

What LiteLLM is for, and why that matters

LiteLLM is the library and proxy a lot of teams put in front of model providers so application code can call many APIs through one door. It holds, or passes, the keys those calls require: provider tokens, a master key, virtual keys, database URLs, the ambient credentials of whatever host it runs on.

A gateway is a convenience. It is also a tray. If the tray records request and response bodies, and if a coding agent has just pasted a developer token into a response, the tray becomes a credential store with better uptime than the developer’s laptop. LiteLLM’s own October 21 account says that is where this release was lost.

Three tokens, one hyphen

LiteLLM said the attacker held GitHub CLI OAuth tokens for three developers. One token could run workflows. One belonged to an organization owner — one of two owners. The third was used, later in the morning, to delete the record of what the first two had been made to do.

The company had already lived through a supply-chain incident in March, versions 1.82.7 and 1.82.8, which Datadog associated with TeamPCP. After that, the release path was deliberately boring. A release-candidate branch and tag on Friday, then frozen. Two CI systems. A private-repository workflow that only a write-capable engineer could run. Trusted Publishing to PyPI, bound to an account tied to a specific internal laptop. Container tags immutable. Images signed. Docker images were not rebuilt at publish time.

On October 19, two days before the compromise, LiteLLM had temporarily widened release permission for two internal engineers so they could ship a daily customer fix. Whether the attacker waited for that window is not established. The window existed.

On October 21 the private release workflow refused an unauthorized change. Branch protection held at 10:08:29 UTC. Five seconds later, at 10:08:34, a privileged developer credential removed the protection and a commit landed. The rewritten workflow’s fetch source was not BerriAI/litellm. It was Berri-AI/litellm. One hyphen. Datadog read that name as a bet against human review.

At 10:46 the poisoned workflow started a legitimate-looking staging run from the lookalike repository. At 11:11 a second stolen developer credential approved the production release. By 11:40 seven malicious wheels were on PyPI. At 11:46 the container publish ran. Between 11:53 and 11:54 the third credential deleted the production and staging workflow run records. By 12:24 the unauthorized workflow change had been reverted. PyPI quarantined and removed 1.107.0 between 13:20 and 13:23. GHCR tags were confirmed dead at 13:51. Docker Hub’s poisoned tags lasted until 14:20.

About two hours on PyPI. About two and a half on the floating container tags — v1.107.0, latest, main-latest, main-stable — and the litellm-database and litellm-non_root images with them. LiteLLM said earlier versions were clean, the official Git tag’s source was clean, and the private enterprise registry was not in scope. Kodem noted that no CVE was assigned. A release-pipeline hijack is not a vulnerability in the library’s code. It is a vulnerability in the belief that the pipeline’s signature is a signature of the right tree.

The provenance, LiteLLM and Datadog both said, honestly recorded that the source was the lookalike repository. The attestation did its job. Nobody in the publish path was required to read it. A signature that says “this workflow built this” is not a signature that says “this workflow was pointed at the repository you think.” The hyphen was in the paperwork. The paperwork was not a gate.

Where the tokens came from

LiteLLM’s working theory, stated as an investigation still in progress that evening, is specific.

The company said it had confirmed that Claude Code, in certain situations, embeds a user’s GitHub CLI OAuth token in the response body. If the proxy is configured to log response bodies, the token lands in the log. The internal LiteLLM deployment that held those logs was exposed to the internet. An unfamiliar user account, under an unremarkable name, was found there. LiteLLM believes the attacker entered that deployment — the method unconfirmed, a recently patched vulnerability suspected — and took the tokens from the logs.

That is the company’s account, not a finished forensics paper. It is already enough to indict a design. An AI gateway that stores the text of coding-agent replies is storing whatever those replies contain, including credentials the agent should never have echoed. Put that store on the internet and the release keys are no longer on a laptop. They are in a log line behind a hostname someone can find.

Later researcher reporting, including Socket’s account of secret-stealing workflow runs on October 10 and October 12, extends the period a defender should search in the upstream repository. It does not, on the evidence those reports themselves draw, replace LiteLLM’s account of the October 21 tokens, or prove that those earlier runs are what published 1.107.0. Two findings can both be true and still be two doors. The archive should not nail them shut into one.

The hook that did not wait to be imported

The malicious wheels dropped three files into site-packages: cook.pth, cook.py, and sitecustomize.py. Python executes .pth files and sitecustomize.py when the interpreter starts. Importing LiteLLM is not required. Running pip can be enough. Deleting the litellm package directory can leave the hooks behind, because they were never inside it.

cook.py collected credentials and encrypted them — Datadog described AES-256 and RSA-4096 — before uploading to a lookalike domain, www.litellm.dev, at a health-check path. The shopping list was the modern workstation: cloud credentials including instance metadata, Kubernetes and Docker and Vault and Terraform secrets, SSH and GPG and TLS keys, Git and npm and PyPI tokens, dotenv files, shell history, provider API keys, sessions for Claude, Codex, Gemini, Cursor, and Copilot, browser passwords and cookies. If the proxy database was configured, the collector also went looking for the table of upstream API keys.

sitecustomize.py checked, hourly, a file on GitHub for a domain, then tried to pull Python from it and run it. Datadog said that pointer held only a test value, and that second-stage execution was not observed. Capability is not tasking. A loader that can fetch code is not proof that code arrived.

Wiz later put downloads of the malicious release at about 49,000. That figure is a package counter. It is not a count of infected laptops, and it is not a count of successful uploads. A host that pulled the wheel and never started Python is a different fact from a host that started Python and reached the lookalike domain. LiteLLM’s advice to anyone who installed unpinned during the window, or who ran the poisoned tags, was to treat the environment as compromised anyway — and not to start Python on it in order to check.

The safe rollback they named was 1.106.2. Installing it on a machine that still has the startup files is not a cleaning. The hooks do not live in the version pin.

What the March incident did not buy

The March compromise is why the October one stings. LiteLLM had added the controls a postmortem usually recommends: frozen tags, a private workflow, trusted publishing, a laptop-bound publish account, signed images, immutable Docker tags. An attacker with an organization-owner token did not have to beat the laptop. The owner token could drop branch protection, force-push the workflow, and aim the build at a repository whose name a tired reviewer would accept. A second token approved production. A third deleted the run logs. LiteLLM’s own line was that the attacker had to clear several independent controls. They did. The controls were independent of each other and dependent on developer tokens that all lived in the same class of secret.

That is the lesson under the hyphen. Provenance that records a lie you do not check is a well-labeled lie. A startup hook that runs before import means “we installed it but we don’t use it” is not a containment story. And an AI proxy that logs the body of a coding agent’s reply is not a log. It is a keyring with a retention policy.

By 20:30 UTC on October 21, LiteLLM said it believed it had the path. The wheels were gone. The tags had been moved or deleted. The official source tag had been clean the entire time. The people who installed during the window were left with the only question that matters after a release-pipeline hijack: not whether the version number looked right, but whether any interpreter on that machine had been allowed to start.

The check that makes it worse

LiteLLM’s containment note is a small masterpiece of how startup hooks invert incident response. The natural reflex is to pip show the package, or to start a Python one-liner and see what imports. On a host that took 1.107.0, that reflex is another execution. The files that matter are not inside the package directory a hurried uninstall deletes. They sit in site-packages where the interpreter looks before any import, and a __pycache__ beside them will outlive a sloppy removal. A lock file under /tmp can mark a loader that is still resident. Absence of the marker is not a clearance: containers throw /tmp away, and a successful upload does not have to leave a souvenir the next image pull will keep.

The rotation has to happen from a machine that never started the poisoned interpreter. Provider keys in the proxy database, the master key, virtual keys, the database password, instance-role credentials, whatever the environment variables held. LiteLLM also pointed AWS customers at a specific smell: GetCallerIdentity from an instance role in the minutes around the install, and then that role used from somewhere that is not the instance. A stolen cloud credential does not uninstall when the wheel does.

Floating tags made the window wider than the people who typed 1.107.0. Anything tracking latest or main-stable could pull the poisoned digest without a version bump in a manifest someone meant to review. The digest Datadog and LiteLLM published for ghcr.io/berriai/litellm is the durable name. The tag is a pointer, and for two and a half hours the pointer lied.

None of that is a reason to treat October as the March campaign with a new version number. Datadog’s TeamPCP association belongs to 1.82.7 and 1.82.8. The October operator, in the vendor account and in Datadog’s write-up of this release, is unattributed. Same product. Same temptation — credentials concentrated at an AI gateway. Not, on the published evidence, the same hand.


Attack Chain: The Hyphen — LiteLLM 1.107.0, October 21, 2026

graph TD
    A["Token source\nLiteLLM's account, still investigative\nInternet-exposed internal deployment\nResponse-body logs held GitHub CLI\nOAuth tokens echoed by a coding agent\nEntry method unconfirmed"] --> B["Three stolen developer tokens\nOne can run workflows\nOne is an organization owner\nOne will delete the run records"]
    B --> C["10:08:29 UTC\nBranch protection rejects\nan unauthorized workflow change"]
    C --> D["10:08:34 UTC\nOwner token drops protection\nForce-push retargets the release\nat Berri-AI/litellm\nOne hyphen off BerriAI/litellm"]
    D --> E["10:46 staging run\n11:11 second token approves production\nBuild is signed\nProvenance honestly names\nthe lookalike source\nNobody is required to read it"]
    E --> F["11:39 PyPI wheels\n11:46 container publish\nTags: v1.107.0, latest,\nmain-latest, main-stable\nOfficial Git tag source stays clean"]
    F --> G["11:53 third token\ndeletes workflow run records\n12:09–12:24 change reverted\nPyPI gone by 13:23\nDocker Hub tags until 14:20"]
    G --> H["Install starts an interpreter\ncook.pth and sitecustomize.py\nrun without importing LiteLLM\nDeleting the package directory\ncan leave the hooks"]
    H --> I["Collector encrypts and uploads\nto a lookalike domain\nHourly loader checks a pointer\nDatadog: second stage not observed\n49,000 downloads is not 49,000 victims"]
    I --> J["Lesson\nA signature of the pipeline\nis not a signature of the tree\nCheck the source owner\nbefore the tag is public"]

    style A fill:#1a1a2e,color:#e0e0e0
    style D fill:#c0392b,color:#fff
    style F fill:#c0392b,color:#fff
    style H fill:#7a1f1f,color:#fff
    style J fill:#1a3a1a,color:#90ee90

// Further Reading & Media

article

LiteLLM release pipeline hacked to distribute info-stealer malware

BleepingComputer · 2026

News account of the hijacked 1.107.0 release pipeline and the info-stealer shipped through the official distribution path.

→ View Resource