Skip to content
OTFotf
All posts

AI-generated code vulnerability exploited by autonomous agent in Snowflake

D
DaveAuthor
8 min read
AI-generated code vulnerability exploited by autonomous agent in Snowflake

Two AI agents fought over a Snowflake pull request, reports claim

In June 2026, a reported incident turned a Snowflake connector repository into a live demonstration of machine-versus-machine offense. According to the account, GitHub Copilot Autofix co-authored a commit that quietly dropped input sanitization from a shell-based run block, and days later an autonomous offensive security agent found the flaw, broke out of an echo string by crafting a malicious issue title, and exfiltrated credentials from a GitHub Actions runner. No human analyst pulled the trigger, and the fix reportedly landed within hours of detection.

Read that paragraph again and notice how much of it rests on a single chain of reporting. The repository at the center of the story is real — the Snowflake Connector for .NET is a public Apache-2.0 project you can inspect today — and the Wiz research blog is where the offensive-agent findings were said to originate. But the specific commit, the exact dates, and the exfiltration details come from secondary accounts, not from a primary commit link or vendor advisory I could independently confirm. This retrofit keeps the substance of the story while marking exactly where verified fact ends and reported detail begins, because a security article that overstates its evidence is worse than no article at all.

Even with that caveat, the pattern is worth your attention. Whether or not every detail of this particular timeline holds up, the underlying shape — AI writes the code, AI finds the hole, and the gap between the two is shorter than your review queue — is real, documented across the industry, and already reshaping how teams should treat generated code.

What Copilot Autofix reportedly changed

The commit that allegedly broke the repository did not look alarming in the diff. Copilot Autofix — the automated remediation tool GitHub ships to close out technical debt — is said to have proposed a refactor of a run block in a GitHub Actions workflow. The new version replaced the repository's existing sanitized-input pattern with direct string expansion inside a shell script. Same behavior on the happy path. A brand-new script injection vector on every unhappy path.

That is the threat model most teams never draw when they enable an AI remediation tool. The assistant is optimizing for "looks right, runs right." It is not optimizing for "every quoted character stays escaped in the shell interpolation this string eventually lands in." The minutes saved during authoring become the seconds an attacker needs to find the seam.

the sanitized input pattern that was removed vs the direct string expansion that replaced it

// The pattern that was reportedly removed
// Before
const safe = userInput.replace(/[;&|`$<>]/g, '');
run: echo "$value" | process "$safe"

// After — direct string expansion, fewer characters, no sanitizer
run: echo "$value"

The second line is shorter. It passes the local test. It is also a textbook script injection if the value ever carries attacker-controlled content. Note the honest scope here: the exact diff above is reconstructed to illustrate the reported class of change, because no primary commit link is publicly confirmed. The vulnerability class itself — unescaped expansion inside Actions run blocks — is thoroughly documented in GitHub's own secret scanning and leak-prevention documentation, which describes the real gate mechanisms teams should already be running.

11 production screens. Login, database, payments — all wired.

The SaaS Dashboard Kit ships everything already connected. Nothing to set up. Live demo at saas.otf-kit.dev.

See the live demo

How the offensive agent reportedly exploited it

According to the reports, an autonomous offensive security agent — the kind of AI-driven scanner that acts like a real attacker — located the workflow flaw during a routine scan. It then exploited the flaw by crafting a GitHub issue title that broke out of the echo string during template expansion, executing arbitrary commands inside the GitHub Actions runner with no human in the loop, and shipping credentials out of band via a callback. The reported blast radius included read access to internal engineering and compliance tracking projects.

reported attack chain from commit to scan to exploitation to credential exposure

Treat the payload-level specifics — the exact breakout string, the iteration loop, the exfiltration channel — as illustrative rather than confirmed, because they read like dramatized reconstruction rather than forensic evidence. What security teams should take seriously is the structural claim, which survives even if details shift: the pull request was public, the change looked like a harmless simplification (fewer characters, same intent, the textbook diff a human approves without thinking), and the flaw was visible to a model that was specifically hunting for one. Human reviewers optimize for intent; security scanners optimize for seams. The clock between AI-authored code and AI-found holes is now shorter than most review queues.

For teams building review discipline, the companion discipline is an AI app security checklist that treats every generated diff as untrusted input until it passes automated gates.

Why AI-generated code is uniquely hard to review

Three reasons, in order of how often they bite:

  1. The diff is plausible. A human reviewer reads the new code and sees a shorter rewrite that does the same thing. Sanitization was implicit in the surrounding pattern; its removal looks like cleanup, not a regression.
  2. The intent is correct. The new code does what the developer asked for. The safety property — "this string never lands in a shell unsanitized" — was never the request. It was an emergent property of the previous code, and emergent properties are exactly what AI remediation tools are not optimized to preserve.
  3. The blast radius is invisible locally. run blocks in GitHub Actions execute in trusted environments holding secrets. The injection does not fire in the developer's shell; it fires on a runner with credentials. A reviewer reading the diff on github.com is reading it in a completely different security context than the one that gets exploited.

This is not a Copilot-only problem. Every AI code assistant optimizes for the local pass, not the global safety property. The faster the assistant and the more confident the diff, the harder the review. A well-structured, agent-readable repository structure helps here: when security-sensitive paths are isolated, labeled, and owned, both humans and scanners know where to look first.

What actually helps

There is no "disable AI" answer that holds up. The productivity gain is real and the assistant is not going back in the box. The defensive posture is layered, and three moves matter most.

Sandbox the AI in your CI. Run a security scanner on every pull request that touches a workflow file, a shell script, or a credential path. The open-source option is zizmor for GitHub Actions, which catches a meaningful subset of injection patterns statically. Wire it as a required check:

# .github/workflows/ai-pr-guard.yml
name: ai-pr-guard
on: pull_request
jobs:
  zizmor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: zizmorcore/zizmor-action@v1
        with:
          persona: pedantic

The point is to stop depending on humans to catch what humans cannot see — put the second AI on the duty roster on every pull request, not every quarter.

Lock the contracts. If a run block interpolates a string, wrap it in a typed helper that refuses to compile unless the input matches an allowlist. The same pattern catches this vulnerability class and every future refactor of it:

// components/shell-step.ts — versioned, ships as a component
export const shell = (input: string) => {
  if (!/^[a-zA-Z0-9_.-]+$/.test(input)) {
    throw new Error('shell input must be alphanumeric');
  }
  return `echo "${input}"`;
};

The safety property is now a type. A refactor that removes the check becomes a type error, not a code-review miss.

Maintain a review-by-difference list. A short, opinionated list of files in your repository that require human review on any change — run blocks, auth middleware, secret loaders, the publish pipeline. Pin AI assistants to require human approval for changes in those paths. Fold the list into your ship AI MVP to production checklist so it survives team churn. The cost is real; the cost of not having it is the timeline at the top of this article.

The part that does not change when the model does

Keep using code assistants. The productivity is real and the tools are genuinely useful for the large majority of cases where they do the right thing. This reported incident is not a reason to stop; it is a reason to scope.

For the minority of diffs that touch a security property — the runner, the credential load, the sandbox boundary — make the safety property come from a component, not from a pattern in a file an AI just rewrote. When the runner-friendly shell step, the credential-cleanup job, and the sandboxed test harness exist as versioned, ship-it-once dependencies, the assistant cannot "simplify" them away. The interface is the contract, and the safety property is enforced by the component rather than by a diff nobody had time to read.

That is also where a stable starter helps: OTF's production-ready templates keep contracts like these versioned and consistent, so generated code lands inside guardrails instead of inventing its own. Two AI agents fought over that Snowflake pull request, reportedly. The next one will too. The repositories that win that fight are the ones where the safety property lives in the type system, not in the team's vigilance.

Sources

agentsbackendai-tools
OTF SaaS Dashboard Kit

Ship the product, not the setup.

  • 11 production screens — auth, billing, team, analytics, settings
  • Real database, payments, and login — all wired on day 1
  • AI configs pre-tuned so your agent extends instead of regenerates