Visit the webpage here: https://code4rena.com/register/bot
It feels like AI is everywhere, doesn’t it?
While recent research suggests a high percentage of smart contract exploits cannot be detected using automation, the looming presence of AI and the open nature of our platform have led us to actively consider the place AI has in Code4rena.
Like other peers in the open audit space, Code4rena initially made the decision to ban all submissions appearing to have been written by AI. After a healthy discussion, we realized this wasn’t conducive to a community that otherwise embraces new technology.
But graphs like this one from the Sci-Fi publication Clarkesworld are fairly ominous for anyone with an open submission form.
So we asked:
How can we embrace the bots without being overrun by them?
Enter the Code4rena Bot Race.
Bot Races make AI the first phase of C4 competitive audits.
Bot Crews compete to see whose bot can create the highest quality and most thorough audit report — and win a share of the audit competition prize pool as a result.
Then, the best report is made available to all wardens to build off with old-fashioned human insight and ingenuity.
How do Bot Races work?
- Bot Races will run the first hour after a competition’s start time.
- Bot Race submissions will be assessed by Lookouts and graded A/B/C similarly to other reports based on report quality, validity, and thoroughness, with a bonus for the best overall report. 10% of each competition’s pool will be allocated to Bot Races.
- The winning bot’s report will be shared with Wardens. This report will be considered the official source for known findings for the rest of the audit.
A few points to note:
- All submissions apart from the winning one will remain private.
- Wardens are encouraged to cite what tools and methods their bot uses. If multiple Bot Crews submit very similar reports, the Bot Crew that contributed the most code to any open-source tools used will be given precedence.
What’s the benefit of a Bot Race?
- Projects benefit because Wardens have the most useful data available from the very best Bot Report and are focused on doing the work that humans can do best.
- Wardens benefit by not needing to submit every low-hanging issue easily identified by automation, and can instead focus on submitting more unique and useful findings.
- Judges benefit by having fewer easily-findable issues to review and far less AI submissions to deal with, given the stock submissions from these models will already be considered known issues.
This reduces the number of duplicate submissions made to Code4rena competitions, thereby removing clutter that would otherwise take up Lookout, Judge and Project time. It also means that the majority of duplicates to be awarded will decrease, and a larger portion of the pool can be given to more unique and useful findings. This further incentivizes submissions to be both unique and of the highest quality.
All in all, the process is made more efficient and, as a bonus, valuable findings that depend on human effort and expertise can be more appropriately rewarded.
How can I register my bot?
- Step 1: Qualifier Races will be open periodically. During this time, Bot Crews will need to name their bot, identify Wardens on their Bot Crew, and submit their bot report to the qualifier competition.
- Step 2: Judges will be assigned to review all bot applications. The top 20 bots will be accepted for participation in the next batch of Bot Races.
The first Qualifier Race will be in early April. We’ll announce it as soon as we schedule it so you have a few days to prepare. Complete the application process which will be on this webpage during that time period to have your bot considered.
