Sony's New Patent Aims to Train NPCs with AI Using Your Player Data

Over the past year, it's become increasingly evident that Sony and Microsoft are eager to integrate AI features into the next generation of consoles. Now, Sony has filed another patent that would use AI to train NPCs based on data generated by you, the player.

This new patent, titled "AI-adapted non-player character for video game system," outlines a process of collecting data from gameplay and using it to modify NPCs to make them more lifelike.

Unlike Nvidia's ACE program, which uses a large language model (LLM) to script NPC dialogue, Sony's proposed system could alter everything about an NPC, from its design to its behavior in scenarios like boss fights. Sony argues this is necessary to create more lifelike games without further inflating development time and budgets.

The patent states that "developing such believable NPCs is challenging due to limitations of modern game development and often relies on extensive scripting and programming. Additionally, NPC development often relies on sophisticated systems and extensive resources to track, animate, and evolve NPC behaviors. In a typical development environment, NPC management often utilizes costly and efficient rendering engines, simulation engines, and scalable systems capable of supporting increasingly detailed virtual environments and lifelike NPC interactions."

The solution, according to the patent, is to use AI to handle most of this work. It continues, "The techniques described herein overcome the foregoing challenges and others by providing an artificial intelligence (AI)-adapted NPC. By using the techniques described herein, video game systems can be improved." Using AI to shortcut development time is nothing new, but that's not the only reason Sony patented this technology. It seems to believe that player data will make NPCs even better than before.

Sony elaborates: "Additionally, by using the techniques described herein, lifelike NPCs that behave based on behaviors exhibited by video game players and/or responsive to behaviors exhibited by video game players can be seamlessly integrated into video game systems and video game content thereby improving video game systems without the extensive scripting and hardware resources."

As with any of the other patents Sony has been rapidly filing, there's no guarantee this system will actually make its way to a PlayStation 5 (or more likely PlayStation 6) near you. However, the game industry has been experimenting with ways to use AI to train NPCs for years. Just last year, Inzoi used LLMs, along with player input, to shape how NPCs would interact with the player and each other.

It's only a matter of time, then, before AI is used to train different aspects of NPC behavior with your data. I could envision a world where bosses in a game like Elden Ring evolve to counter your playstyle, or an enemy in a horror game stalks you more effectively because it 'knows' how you react to different stimuli.

However, there would be questions about security. As cool as having smarter NPCs would be, if these models are processed in the cloud, there would be a risk of your data leaking out onto the internet. And while it doesn't seem like advertisers would want to know how you react to a giant monster swinging an axe at you, it probably will end up being used for 'enhanced marketing,' too.

This is hardly the first of Sony's strange new patents. Just a couple of weeks ago, the company patented a system where AI would detect problems with your console and deliver vibe-coded patches. It also patented a controller with an NFC card reader that would bring tap-to-pay to your PlayStation. The next generation of consoles is shaping up to be a strange one, to say the least.