The proliferation of AI in the last few years has led to new potential in almost every industry and, in parallel, many (sometimes funny, often uncanny) examples of its fundamental limitations. In the Web3 security space, the same dynamic exists: AI has introduced new possibilities and also presents serious drawbacks when relied on without question.
When used with an understanding of its limitations, AI can significantly scale the productivity of smart contract auditors, automating code analysis and quickly flagging vulnerabilities that previously required manual review.
But someone has to say it: AI cannot complete comprehensive security audits. And we’ll tell you why, but first, let’s talk about what AI is great at.
If you look at AI more like an assistant that can take some small tasks off your plate (that you’ll need to review) and less like an expert who can do the work for you, you’re going to have a better time.
AI is a great tool for noticing discrete patterns in implementation vulnerabilities in a smart contract audit. Letting AI analyze your smart contract can help you quickly digest large code bases, summarize what’s there, and supply ideas on where vulnerabilities would likely be.
Often, the hardest part of an audit is getting started - this is what AI can take over for you.
When it comes to understanding the big picture of a Web3 audit with all of its factors and externalities, a human analyst is required. Design vulnerabilities in particular demand a higher level of analysis and understanding than what AI can currently accomplish. Often, vulnerabilities are unique edge cases that AI simply won’t catch.
Large language models (LLMs) and other detection tools have the capability to detect patterns in code, but overall understanding of complex systems is out of their reach. Static analysis tools demand that developers define and maintain their rule sets to identify vulnerabilities. Likewise, AI models can only propose solutions based on the data they have been trained with. LLMs will do their best to produce results that appear analogous to an answer to the prompt they are given, but they are constrained to pattern matching as their primary method of deduction. This gives AI a considerably narrower perspective, which overlooks the complexity of design vulnerabilities.
It goes without saying, but when it comes to securing a smart contract, missing the big picture leaves you extremely vulnerable to malicious actors.
We will continue to integrate AI more and more into our processes as Web3 security auditors, recognizing where the tool excels: at handling small components of the audit, at assisting the human auditor step by step. In the future, we may see analysts providing necessary oversight of AI as it works through every step of the audit: analysis, threat modeling, verifying, and testing. The combination of human and AI power results in an extremely efficient and thorough workflow.
AI is one of the core features we’ve built into Audit Wizard: a tool that greatly augments but does not replace the human Web3 developer and security auditor. Try it for free here.