C++ Developer | Cybersecurity Researcher | Software Engineer | Backend Developer | Ph.D. Computer Scientist.
I see this related to LLM02-Insecure Output Handling and also to Developing_Insecure_Source_Code, one of the new candidate entries submitted for consideration to be included in version 2.0 of the OWASP Top 10 For Large Language Model Applications list (https://lnkd.in/gBPZZws7). The use of CriticGPT highlights the necessity of actively monitoring and sanitizing the model's inputs and outputs. #llmsecurity #aisecurity
We’ve trained a model, CriticGPT, to catch bugs in GPT-4’s code. We’re starting to integrate such models into our RLHF alignment pipeline to help humans supervise AI on difficult tasks: https://lnkd.in/gbAue2fZ