Results of ChatGPT Analysis of Java and C++
• Dataset
Results of 2024 analysis of CERT Secure Coding Rules for C++ and Java with ChatGPT 3.5 and ChatGPT-4o.
Publisher
Software Engineering Institute
Abstract
We collected examples from the CERT Secure Coding standards for C++ and Java, which contain commonly seen security or functionality errors, explanations of the errors, noncompliant code examples, and compliant code examples showing potential solutions. Our goals were to see if the GPT models would correctly identify errors in noncompliant code and correctly recognize compliant code as error-free. This site contains our complete test results for ChatGPT 3.5 and ChatGPT 4-o.