An Unknown Gpt2-Chatbot Is Blowing Up On X: Is This OpenAI's GPT-5 Upgrade?
ChatGPT is easily the best-known generative AI chatbot in the world, but it offers different experiences depending on whether or not you pay for a premium LLM. The free ChatGPT runs on the GPT-3.5 model, while ChatGPT Plus and enterprise subscriptions have access to GPT-4. There's even talk about GPT-4.5 coming out soon, with many people expecting OpenAI to launch a GPT-5 upgrade later this year.
With that in mind, a mysterious GPT2-Chatbot is apparently out in the wild. Some testers are already posting impressive results from their experiments. It's unclear where it comes from and how it made its way online, but it seems to be very capable.
What's clear is that it's blowing up on Twitter/X, with people trying to explain its origin. Are we looking at an unreleased version of ChatGPT? This could be an early test version of GPT-5 that OpenAI is testing in the wild ahead of its release.
OpenAI CEO Sam Altman said in a recent interview that he didn't know when GPT-5 will be released. The company has other exciting announcements to make, and we've seen some of them already.
I have been testing gpt2-chatbot for a few days. Today it seems to have gotten much more attention.
It surpassed all of our ChatGPT-4 benchmarks.
Hypothesis: A few of us have concluded it is a form of pre-lobotomized ChatGPT-4 or trained heavily on it. https://t.co/KKLHmPVYnf
โ Brian Roemmele (@BrianRoemmele) April 29, 2024
For example, OpenAI made ChatGPT available without an account, discussed the Sora public rollout, showcased its voice cloning tool, and rolled out ChatGPT memory more widely.
Interestingly, Altman hinted that the next ChatGPT upgrade might not be called GPT-5, and that's understandable. But GPT-5 got plenty of attention last year.
Some said GPT-5 would be the point where we reach AGI, or a potentially dangerous AI that might reason as well as humans. GPT-5 also came up as a potential reason for Altman's firing last fall. Maybe a different name would be needed to clean the image of GPT-5.
Maybe OpenAI will go for a ChatGPT 2 upgrade, therefore skipping the GPT-5 model name entirely. With that in mind, could GPT2-Chatbot be the precursor of a future OpenAI evolution of ChatGPT? It's too early to tell.
uh.... gpt2-chatbot just solved an International Math Olympiad (IMO) problem in one-shot
the IMO is insanely hard. only the FOUR best math students in the USA get to compete
prompt + its thoughts ๐งต https://t.co/CuO0ToJmb9 pic.twitter.com/3xxWPvtmuG
โ andrew gao (@itsandrewgao) April 29, 2024
I asked gpt2-chatbot to generate a simple programm for turning doodles into art.
Every other model I've tested fails at this. They get a doodle function, but the button does nothing of artistic value.
gpt2-chat's attempt: pic.twitter.com/oPhc5ob0Ow
โ Ippi (@Coolzippity) April 30, 2024
What seems clear here is that GPT2-Chatbot is quite advanced. According to some of the people who tested it, it's apparently beating or matching GPT-4 (ChatGPT Plus) in benchmarks.
Others offered examples of problems where GPT2-Chatbot answered prompts correctly, including difficult math problems, while GPT-4 and other AI models couldn't.
Siqi Chen went as far as to call GPT2-Chatbot "mildly scary" in a now-deleted tweet because it was able to solve a puzzle that other GPTs couldn't figure out.
Chen also said that GPT2-Chatbot seems to have better reasoning than GPT-4 and Claude 3 Opus. But he speculated that the mysterious LLM isn't GPT-4.5 or GPT-5.
so i don't think it's an accident that this isn't named gpt4.5 or gpt5 โ it is neither.
it's a test bed for q* (or whatever you want to call tree of thought + prm these days).
the next gpt5 will likely continue to ride on the scaling hypothesis, plus a reasoning boost from this
โ Siqi Chen (@blader) April 30, 2024
You can test GPT2-Chatbot yourself right now by going to this link. Select Direct Chat and look up the model.
At the time of this writing, the rate limit for the model had been reached. It's proof it went viral on social media. I tried asking it about its origins. Apparently, the mysterious model told others it's GPT-4 from OpenAI, but a V2 version.
not sure if this is hallucinated, but: https://t.co/ap08iKW2t0
โ Umang Jaipuria (@umang) April 29, 2024
Large Model Systems Organization (lmsys.org) commented on the origin of the GPT2-Chatbot model they're hosting on their website, saying this model is in testing, and they won't divulge more details about it until it goes public:
Just to clarify, following our policy, we've partnered with several model developers to bring their new models to our platform for community preview testing. These models are strictly for testing and won't be listed on the leaderboard until they go public.
Whether GPT2-Chatbot is GPT-5, a different ChatGPT upgrade, or something else, I still expect OpenAI to make some sort of big GPT-5 announcement later this year, even if the underlying model gets a different name.
Remember that Google grabbed everyone's attention a few months ago when it launched the big Gemini 1.5 upgrade. Then Meta came out with its own generative AI models, which are rolling out slowly to Facebook, Messenger, WhatsApp, and Instagram. In light of that increased competition, upgrades to ChatGPT must be imminent.
On that note, it'll be interesting to see which entity claims the mysterious new GPT2-Chatbot. If the GPT2-Chatbot isn't from OpenAI, it could be just another GPT rival that puts even more pressure on the company to improve ChatGPT.