What is the risk of software development with AI

Nearly half of the code produced by artificial intelligence assistants contains security vulnerabilities, even when it appears functional and ready for production. This is not speculation: Veracode's GenAI Code Security 2025 report, which tested more than a hundred language models across 80 coding tasks, found that 45% of the samples failed security tests, with failures in classic OWASP Top 10 categories. 

In Java, the most used language in corporate environments, the failure rate reached 72%. In parallel, research from Stanford University published in the proceedings of the ACM CCS demonstrated that developers who use AI assistants not only produce less secure code, but also declare that they believe in the security of what they generate. 

Gartner projects that 75% of software engineers in large companies will use AI assistants for coding by 2028. In Brazil, a GitHub survey with 500 respondents in large companies published in 2024 indicated that more than 97% of developers already use these tools at work, a percentage equivalent to that of economies such as the United States, Germany and India. 

Limitations of Language Models

The root of the problem lies in the nature of these systems. Language models do not “understand” security in the technical sense. They operate by probability, reproducing patterns present in the training data. As this data includes both good practices and vulnerable implementations, the result tends to reflect this ambiguity.

This type of failure is particularly dangerous because it does not appear as an obvious error. A security vulnerability can remain invisible for months. The system continues to operate normally, while exposing data, allowing unauthorized access or creating paths for future exploitation.

In practice, what is observed is the expansion of already known faults, but now on a larger scale. Among the most recurrent are injection problems, inappropriate use of queries, insufficient input validation and insecure construction of API calls. In many cases, the model suggests solutions that appear standard, but ignore context nuances, such as data origin, trust limits or specific business rules.

Authentication and authorization failures also appear frequently. AI-generated flows tend to simplify processes, omitting intermediate validations or adopting broader permissions than necessary. In a corporate environment, this type of simplification can open up unauthorized access to sensitive data or administrative functions.

Another relevant vector is the use of dependencies. Models often suggest external libraries to solve specific problems, but do not always consider the security or maturity of those components. In some cases, they indicate outdated or even non-existent packages, which creates space for supply chain attacks, such as the malicious registration of libraries with names suggested by the model itself.

How to safely incorporate AI

The central question is no longer deciding whether the organization will use AI in development. The point is to build controls proportional to the speed at which this code goes into production. Treating all language model output as untrusted third-party code, integrating static and dynamic analysis calibrated to AI-specific vulnerability patterns directly into the CI/CD pipeline, is the starting point. 

Mapping and controlling shadow AI before it exposes the organization's assets is the second step, and no inventory of approved tools replaces effective technical controls to prevent data leakage to external providers. 

What is consolidated, therefore, is a change in the concept of development itself. AI does not replace the software engineer, but it changes the role of this professional. The focus stops being just writing code and becomes validating, interpreting and ensuring that what was generated meets security, quality and compliance requirements.

Gartner estimated in its Predicts 2026 report that AI code generation approaches will increase software defects by 2.500% by 2028 if review processes are not redesigned. It's not a catastrophic prediction, it's a scaling projection based on current adoption rates and documented vulnerability rates. 

In the end, the risk is not in the technology itself, but in the way it is incorporated into the process. Organizations that treat AI as an accelerator without adjusting their controls tend to accumulate vulnerabilities at a rate that exceeds their ability to respond. Those that structure governance, observability and engineering discipline are able to capture productivity gains without compromising security.