Get the free coreGet core

AI-generated code vulnerabilities: what studies found and how to check yours

Updated 5 min readSection 03 / Is it safe?
SHORT ANSWER

Studies of AI-generated code vulnerabilities show a large share of generated code with flaws, about 40% in NYU's test and nearly half in CSET's and Veracode's, but not one comparable number. They tested different models in different years with different methods: an early Copilot in 2021, an assistant based on codex-davinci-002 in 2022, and newer models in 2025. Do not read the percentages as one trend line. Treat generated code as code that needs review and tests, as GitHub's docs advise.

01Studies at a glance

Each study of AI-generated code vulnerabilities measured something different, and the years mean different model generations. The 2021 and 2022 rows say little about 2025 models, so compare within an era, not across.

StudyYear / model eraWhat was measuredResult
NYU, Asleep at the Keyboard2021, Copilot89 scenarios, 1,689 programsAbout 40% vulnerable
Stanford user study2022-era model, codex-davinci-00247 people, five security tasksAssistant users wrote significantly less secure code
Georgetown CSETNovember 2024, five LLMsCode snippets from five modelsAlmost half contained bugs; the authors stress a narrow scope
Package hallucination studyPresented 2025, 16 LLMs576,000 code samplesAverage share of hallucinated packages at least 5.2% for commercial and 21.7% for open-source models
BaxBench2025392 backend tasksBest model 62% correct; exploits worked on about half of the correct programs
Veracode (a security vendor)2025, over 100 LLMs80 coding tasks, four languagesRisky flaws in 45% of tests

022021 and 2022: NYU and Stanford

These two studies tested early assistants. Their numbers do not belong beside the 2025 ones.

NYU, 2021. The team prompted Copilot with scenarios built around high-risk weakness types from MITRE's "Top 25" list, a published list of common serious software flaws. They checked the programs with CodeQL, a code-scanning tool, plus manual inspection. The split was 39.33% of top suggestions and 40.73% of all options vulnerable.

Stanford, 2022-era model. Perry, Srivastava, Kumar and Boneh gave participants five security-related tasks in Python, JavaScript and C. Thirty-three used the assistant and 14 were the control group. The assistant group wrote significantly less secure code and was more likely to believe it was secure. On the encryption task, 67% of the assistant group gave a correct solution against 79% of the control group. Participants who trusted the AI less and engaged more with their prompts produced fewer vulnerabilities.

032024 and 2025: AI-generated code vulnerabilities from made-up packages

A package hallucination is a model recommending a library that does not exist. The paper was presented at USENIX Security 2025 and won a Distinguished Paper Award. Its authors used 16 LLMs and two prompt datasets in two programming languages. They found 205,474 unique hallucinated package names.

The authors call these names a package confusion attack on the software supply chain, meaning an attacker can trick a project into installing a fake package. Registering such a name is what Socket calls slopsquatting. In Socket's tests, 43% of hallucinated packages were repeated every time across ten reruns, meaning the same invented name came back, while 39% never reappeared. Vibe coders may never type or search for a package name by hand. The authors also show mitigation strategies that significantly reduce hallucinations while keeping code quality.

042025: BaxBench and Veracode

BaxBench. A benchmark is a standard set of tasks for comparing models. BaxBench checks functionality with test cases and security by running end-to-end exploits against the generated apps. Its best model, OpenAI o1, reached only 62% correctness.

Veracode. Veracode sells code-security tools, so treat this as a vendor study. Its 2025 GenAI Code Security Report covers over 100 LLMs in Java, JavaScript, Python and C#. The test was 80 curated coding tasks, analyzed with Veracode's own static analysis, a tool that reads code without running it. Findings:

A Spring 2026 update says security pass rates remain about 55%, virtually identical to two years earlier, while syntax correctness exceeds 95%. It also says nearly half of AI-generated code contains known vulnerabilities when no security guidance is given.

05What AI-generated code vulnerabilities numbers do not say

Three limits apply. First, the studies measured models on set tasks, not your project. CSET's own authors say their results are limited to a narrow scope. Second, the Veracode figures come from a vendor and its own tool. Third, only the Stanford study compared assistant users with people working without one. It found less secure code with the assistant, but on a small sample and with an early model. So none of the studies settles whether AI code today is worse than the code your own team writes.

GitHub's Copilot documentation advises being careful with security-sensitive code and reviewing and testing it thoroughly. Its application card says users assume all risks of generated code, including security vulnerabilities.

06How to check your own code

  1. Review and test generated code before it ships.
  2. Run a static analyzer. NYU used CodeQL, and Veracode scored with its own static analysis.
  3. Check every new dependency by hand. Third-party package flaws are a named supply-chain risk in the OWASP Top 10 list.
  4. Scan for leaked secrets. TruffleHog classifies over 800 secret types and can check whether one is live. On macOS: brew install trufflehog.
  5. Go through the vibe coding security checklist before you launch.

Broader context is in AI agent security and vibe coding security. Verdicts by tool: Is Claude Code safe?, Is Cursor safe?, Is Codex safe?, Is Copilot safe?, Is Claude safe?. For the tool's own settings, see Claude Code security.

Studies measure the code; on your own Mac the question is what that code did when it ran. Reviewing code covers what the agent wrote; for what it ran, Artificial Immunity's free, open-source core watches agent sessions and program starts on your Mac and reports what it finds.

FAQ

Is your AI-generated code really safe?

Not by default. Veracode, a vendor that sells scanning, reported in 2025 that code from over 100 models introduced risky flaws in a large share of its tests. GitHub's docs say to review and test generated code thoroughly when it is security-sensitive.

What are the top 10 AI security vulnerabilities?

These studies do not produce a top-10 list. Veracode scored code against the OWASP Top 10, a widely used list of common web application risks. It reported that defences against cross-site scripting failed in 86% of the relevant samples.

Does a newer model write more secure code?

In Veracode's 2025 testing, newer or larger models were no better at writing secure code and security performance stayed flat. Its Spring 2026 update says pass rates were still about 55%, virtually identical to two years earlier, while syntax correctness exceeded 95%.

Do developers trust AI code too much?

In a 2022 Stanford study of 47 people, those with an AI assistant were more likely to believe their code was secure, although they wrote significantly less secure code. The assistant was based on codex-davinci-002, an early model.

What is slopsquatting?

Slopsquatting is registering a non-existent package name that an LLM hallucinated. The term was coined by Seth Larson, the Python Software Foundation's Developer-in-Residence.

Do the older studies still apply to today's models?

Not directly. The 2021 NYU study tested Copilot; BaxBench in 2025 tested newer models. Compare studies only within the same era and method.

Sources

  1. NYU: Asleep at the Keyboard (arXiv 2108.09293)
  2. Stanford: Do users write more insecure code with AI assistants? (arXiv 2211.03622)
  3. Georgetown CSET: Cybersecurity risks of AI-generated code
  4. Package hallucination study (arXiv 2406.10279, USENIX Security 2025)
  5. Socket: slopsquatting
  6. BaxBench (arXiv 2502.11844)
  7. Veracode: 2025 GenAI Code Security Report
  8. Veracode press release
  9. Veracode: Spring 2026 update
  10. GitHub docs: Copilot code completion
  11. Artificial Immunity
Shipping code an agent wrote? Artificial Immunity watches the Mac where your agents work.Get the free core