»GTA V«: AI filter makes the game more realistic

Razor-pointed palm fronds, cracks in asphalt, reflections from the sun on car roofs: The virtual Southern California in the video game “GTA V” still looks impressive even after the video game was released, especially in its PC version. At the same time, the fake world in the game can still be easily distinguished from the real.

American chip maker scientist Intel Want to replace it with image enhancement software. Researchers have a software model (PDF) Designed on Artificial intelligence (AI) And makes the streets of the big city of Los Santos more realistic.

The team uploaded a video to YouTube Which shows a better graphic. With AI filters, the asphalt looks more authentic, the cars reflect light more beautifully, and the trees also look as if they were filmed with a camera from inside the vehicle.

Stephen Richter has been working on the project for two years with two collaborators. “Our goal was to make the game as real as possible,” the Intel researcher said in an interview with SPIEGEL. “Game simulations often don’t look realistic enough: graphics are not yet enough to depict the real world.” A lot of work was done in the project before the images appeared like they do now.

The software stops the information for the graphics card

To generate a clean image, the researchers double-checked: on the one hand, the fully rendered sports image is sent via an AI filter. At the same time, however, the software also uses so-called G buffers. It contains information to correctly calculate geometric details for graphics cards such as the distance between buildings and the camera and the position of the light source.

Researchers shut down the G-buffer to make it clear to AI not to plant trees in the sky. Finally, the resulting image is subject to a reality check. The software compares the results to street snapshots and provides authenticity points.

To map reality as accurately as possible, the software picks things up Training photos from cityscapes back. This photo database contains roads from 50 mainly German cities such as Erfurt, Perfume And Stuttgart Deposited. The pictures clearly indicate whether an object is a pedestrian, a traffic signal or a bus stop.

Everything is not right

While roads, trees and cars actually look more realistic, image enhancement reaches its limits with some objects – also because roadside palm trees are rare in this country. The software sometimes interprets the crane as a traffic light and the park bench is transformed into a rolled moonlight. According to a report by scientists, pedestrians are also “processed in a less concrete way”.

Despite such problems, researchers believe that technology can bring more realistic graphics to games. “I can well imagine that engine developers are using this software as a reality filter,” says Richter. “GTA is only a proxy here. It can be implemented in many games.”

However, in its current form, the software is still very slow. Even with Nvidia’s top model GeForce RTX 3090 graphics card, image correction outshines the result after only half a second. According to Richter, at the moment it is a prototype “that you cannot plug directly into the game at this time.”