We are building the platform that makes AI agents in games practical and useful. Learn more about our bot technology at https://regression.gg.
External link for Regression Games
Remote, US
Checkout our recent stream! And make sure to join us every Friday at https://lnkd.in/dtMwei2Z
Posted the VOD for our Friday stream at Regression Games, where we built a GPT for QA testing games. Watch the video or checkout the blog post in the comments below! https://lnkd.in/eETaYzDb #GPT #AI #gamedeveloper
Regression Games reposted this
New reading club this Friday in the Regression Games Discord - we will be talking about a paper that explores using multimodal LLMs like GPT-4v to understand bugs in game screenshots! Event link: https://lnkd.in/deJahfHq Paper: https://lnkd.in/dbsdViiR
Regression Games reposted this
Just had a great Friday Reading Club session in the Regression Games Discord today about this work. Here are some topics we chatted about (and I might make a blog post to follow up on this). We have a new paper chat every Friday; check out the comments for a link to our Discord! - We wondered how much of this model is memorizing visuals vs actually forming a model of the world. For instance, it was interesting to see that it memorized the map, but if it saw a new area that wasn't in the training area, would it generate new content? Also, the evaluation was mostly focused on visuals, but we were curious about what that evaluation would look like regarding the gameplay logic itself. Especially in the human eval, did the humans correctly find the real gameplay due to visual artifacts, or gameplay bugs? - In their setup, they ignore/disable text conditioning for the stable diffusion model... If this is paired with some instruction text for the game visuals and gameplay logic, I wonder what that might look like. We were thinking it would require a whole lot of data before it can actually generate new games vs just simulating an existing one. We were comparing this to the SIMA work from Google earlier this year. - It makes you wonder how far away we are from all software just being a neural network. We didn't see any reason why this same approach couldn't be used to simulate a simple website. - We pondered how this would perform with AAA-style games. The visual details might make it pretty tricky to get real-time gameplay at high fidelity. - We had some interesting conversations about the de-noising approach to allow frames to be accurate after many iterations and how they trained the RL model to play the game. Some unique ideas there!
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Group). Stanford Ph.D. Building Humanoid robot and gaming foundation models. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
It's a tradition for hackers to run DOOM in crazy places: thermostats, "smart" toasters, even ATMs. Now they run DOOM purely in a diffusion model. Every pixel here is generated. A while ago, I said "Sora was a data-driven physics engine". Well, not quite, because Sora could not be interacted with. You set the initial condition (a text or initial frame) and may only watch the simulation passively. GameNGen is a proper neural world model. It takes as input past frames (states) and an action from a user (keyboard/mouse), and outputs the next frame. The quality is by far the most impressive I've seen on DOOM. However, this comes with significant caveats. Let's deep dive: 1. GameNGen overfits to the extreme on a single game by training on 0.9B frames (!!). This is a HUGE number, almost 40% of the dataset used to train Stable Diffusion v1. At this point, it's likely memorizing how DOOM renders from every corner of the game in all scenarios. DOOM doesn't have that much content anyway. 2. GameNGen is more like a glorified NeRF than a video gen model. A NeRF takes images of a scene from different view angles, and reconstructs the 3D representation of the scene. The vanilla formulation has no generalization capability, i.e. it could not "imagine" new scenes. GameNGen is not like Sora: by design, it could not synthesize new games or interaction mechanics. 3. The hard part of this paper is not the diffusion model, but the dataset. Authors trained RL agents to play the game first, at various different skill levels, and collected 0.9B (frame, action) pairs for training. Most of the video datasets online do NOT come with actions, which means this method wouldn't extrapolate. Data is always the bottleneck for action-driven world models. 4. There're two practical use cases for game world models in my mind: (1) write a prompt to create playable worlds that would otherwise take game studios years to make; (2) use the world model to train better embodied AI. Neither use cases can be realized. Use case (2) doesn't work because there's no advantage to use GameNGen for training agents than directly using DOOM simulator itself. It'd be more interesting if a neural world model simulates scenes that traditional hand-crafted graphics engines cannot. What's an example of a truly useful neural world model? Elon said in a reply that "Tesla can do something similar with real world video". Not surprising: Autopilot team likely has trillions of (camera feed, steering wheel action) pairs. Again, data is the hard part! With such rich real-world data, it's entire possible to learn a general driving sim that covers all kinds of edge cases, and use that to deploy & verify a new FSD build without physical cars. GameNGen is still a really great proof of concept. At least we know by now that 0.9B frames is the upper bound to compress high-res DOOM into a neural network. Paper: Diffusion Models Are Real-Time Game Engines https://lnkd.in/g9aW_uUK
Regression Games reposted this
This Friday (and hopefully each Friday going forward) we have a reading club in the Regression Games Discord. This week we will be talking about the paper from a few days ago where Google researchers trained a diffusion model to simulate DOOM. Come join and hang out for this quick chat! https://lnkd.in/eJiZfPWP
Regression Games reposted this
Just wrote a new blog post on self-healing tests and the pain points of those systems when applied to games. https://lnkd.in/dhJ7nfFK I kept seeing this idea of using LLMs to fix broken and flaky tests, and I've been pondering how this applies to games which are inherently non-static. Hoping to think more about this and experiment in the future! Let me know what you think in the comments or in the Discord (link at the end of the blog post).
We are extremely excited to welcome Dr. Batu Aytemiz to Regression Games! Batu brings extensive experience in game AI from his time at Ubisoft La Forge, Microsoft Research Cambridge Game Intelligence Lab, and Agentic AI. Batu will help us bring AI automation in games to the next level 🎉
Exploratory testing, deep data collection, and computer vision-enabled decision making in automated tests are some of the innovative tools we're developing and looking for feedback on! If you're a game programmer, we'd love to chat with you. Click the link below to share your thoughts: https://lnkd.in/e63KCmPj
Regression Games reposted this
With release 0.0.23, you can now try out the Regression Games validation tool without any account or game! Visit our login page to see a link to the demo: https://lnkd.in/e8WuHpsT Use the feedback feature to let us know what you think, and check out the changelog blog post for more information on this release: https://lnkd.in/esfWAFNG