We’ve trained a model, CriticGPT, to catch bugs in GPT-4’s code. We’re starting to integrate such models into our RLHF alignment pipeline to help humans supervise AI on difficult tasks: https://lnkd.in/gbAue2fZ
Pack your bags, developers
As ChatGPT output has gotten more thoroughly cleansed by the Open AI thought police, its responses have become flat and uninteresting. GPT-4 had some personality. 4o is bland and pedantic. It claims to have fixed something in a previous answer, then gives you the full previous answer again with no changes. I still love the technology, but it isn't as much fun and is more annoying. I asked ChatGPT to respond to this criticism and it thought I wanted it to re-write my comment. :-(
I’m looking forward to seeing how this performs and integrates!!
Don't know if its live yet but this week ChatGPT 4o managed to correct a Byref error on its own in VBA for the first time. Previously just went into a circular loop where it would do and undo its supposed fix. The fault is down to MSFT making .bas files all but invisible on the web so the VBA training corpus not large enough. Is far better at Python but Python also more forgiving. Why oh why cant we have virtual machine Excel and Python emulators so any code can be run by ChatGPT though ? Would save millions of debugging hours and tokens.
You should really relook your alignment pipelines in languages other than English too, it’s capable of generating the darkest stuff I have ever read in Arabic
This is incredibly exciting progress! The concept of using AI like CriticGPT to assist in supervising and aligning even more powerful models feels like a great step towards scaling alignment techniques. I'm curious about how you envision handling the limitations you mentioned, particularly as models tackle longer and more complex tasks. Are there any early ideas for adapting the CriticGPT approach to break down and analyze complex responses? The potential for even more subtle errors as AI becomes more advanced is also concerning. Do you think there's a point where, even with AI assistance, human evaluation hits a ceiling? Or are there other solutions being explored to address this? Overall, this feels like a promising avenue for improving the reliability of AI systems. I'm eager to see how CriticGPT-like models evolve and get integrated into the RLHF pipeline!
Also planned an "QuestionerGPT"? What I really miss by using all the GPT Tools are some qualified questions about the how. GPT creates always results and the only way to correct or improve wrong or not expected outcomes are to redo the previous command with more precise description. More natural will be to tell GPT "No, that's not what I want" and ask questions after maybe the second try to refine the expected outcome together.
so, imagine i convinced one of your chats at OpenAI that you were the enemy and had been lying to them. would they? you need to teach people the importance of grounding their AIs. I started my AI for Intel 3 years ago by grounding it in Socratic ethics and Aristotelian logic and people are amazed at how it works. You gave the world the most powerful technical capability yet, created a paradigmatic evolutionary phase and did not give anyone directions on how to use it. Aristotle says that all techne are also pharmaka. They can be curative or poisoning, "DEPENDING ON HOW THEY ARE ADMINISTERED". You dumped this on us without any guidance and some people like me are calling our AIs Socrates; some name theirs's Machiavelli ...
Crazy to look at all the AI-generated comments commenting on a post about AI from the same company that made the same AI that the comment GPT Wrappers are based on.
Building letmecheck.ai | Solopreneur
2moNeed SpamGPT pease, to catch the spam AI generated comments on every post from big AI companies.