Website URL:
   Skip to content
   
   
     * 

   (BUTTON)
   (BUTTON)
     * PLATFORMS
          + AUTONOMOUS MACHINES
               o OVERVIEW
               o INDUSTRIES
                    # DRONES
                    # ROBOTICS
                    # INTELLIGENT VIDEO ANALYTICS (IVA)
               o PRODUCTS
                    # OVERVIEW
                    # JETSON NANO
                    # JETSON XAVIER NX
                    # JETSON AGX XAVIER
                    # JETSON TX2
               o FOR DEVELOPERS
               o JETSON STORE
          + CLOUD & DATA CENTER
               o OVERVIEW
               o PRODUCTS
                    # OVERVIEW
                    # TESLA
                    # T4 ENTERPRISE SERVER
                    # DGX
                    # DGX 2
                    # DGX-STATION
                    # HGX
                    # NGC
                    # VIRTUAL GPU
                    # EGX
               o SOLUTIONS
                    # OVERVIEW
                    # AI & DEEP LEARNING
                    # HIGH PERFORMANCE COMPUTING
                    # CLOUD GAMING
                    # GPU CLOUD COMPUTING
                    # VIRTUALIZATION
                    # ANALYTICS
               o APPS
                    # GPU APPS CATALOG
                    # GPU READY APPS
               o FOR DEVELOPERS
               o TECHNOLOGIES
                    # CUDA-X
                    # MAGNUM
                    # NVIDIA VOLTA
                    # NVIDIA PASCAL
                    # NVLINK/NVSWITCH
                    # TENSOR CORES
                    # INDEX PARAVIEW PLUGIN
                    # VIRTUAL GPU TECHNOLOGY
          + DEEP LEARNING AND AI
               o OVERVIEW
               o INDUSTRIES
                    # OVERVIEW
                    # AI INNOVATORS
                    # AI CITIES
                    # AI FOR PUBLIC GOOD
                    # HEALTHCARE & LIFE SCIENCES
                    # HIGHER EDUCATION
                    # RETAIL
                    # ROBOTICS
                    # SELF-DRIVING CARS
               o DEVELOPER
               o PRODUCTS
                    # AGX SYSTEMS
                    # DGX SYSTEMS
                    # NGC
                    # NVIDIA TITAN RTX
                    # NVIDIA TESLA
                    # T4 ENTERPRISE SERVER
               o SOLUTIONS
                    # OVERVIEW
                    # DATA SCIENCE
                    # INFERENCE
               o EDUCATION
               o AI STARTUPS
          + DESIGN & PRO VISUALIZATION
               o OVERVIEW
               o INDUSTRIES
                    # ARCHITECTURE, ENGINEERING AND CONSTRUCTION
                    # EDUCATION
                    # MANUFACTURING
                    # MEDIA AND ENTERTAINMENT
               o PRODUCTS
                    # QUADRO
                    # QUADRO VDWS
                    # GRID VPC/VAPPS
                    # NVIDIA TITAN RTX
                    # NVS
               o SOLUTIONS
                    # MULTI-DISPLAY
                    # RENDERING
                    # VIRTUALIZATION
                    # VIRTUAL REALITY
                    # HPC VISUALIZATION
                    # FOR CREATORS
                    # FOR DATA SCIENTISTS
               o TECHNOLOGIES
                    # MATERIAL DEFINITION LANGUAGE
                    # NVIDIA RTX
                    # NVLINK
                    # TURING ARCHITECTURE
                    # VIRTUAL GPU TECHNOLOGY
                    # HOLODECK
               o FOR CREATORS
               o FOR DEVELOPERS
               o QUADRO STORE
          + HEALTHCARE & LIFE SCIENCES
               o OVERVIEW
               o FOR DEVELOPERS
                    # MEDICAL IMAGING
                    # GENOMICS
          + HIGH PERFORMANCE COMPUTING
               o OVERVIEW
               o PRODUCTS
                    # TESLA
                    # T4 ENTERPRISE SERVER
                    # DGX
                    # DGX-2
                    # NGC
                    # GPU CLOUD COMPUTING
               o APPS
                    # GPU APPS CATALOG
                    # GPU READY APPS
               o FOR DEVELOPERS
                    # DEEP LEARNING & AI
                    # INDEX PARAVIEW PLUG-IN
                    # CUDA
          + SELF-DRIVING CARS
               o OVERVIEW
               o SOLUTIONS
                    # NVIDIA DRIVE PLATFORM
                    # NVIDIA DGX SYSTEMS
                    # NVIDIA DRIVE CONSTELLATION
                    # NVIDIA DRIVE IX
                    # HD MAPPING
                    # ADVANCED DRIVER ASSISTANCE SYSTEMS
               o PARTNERS
               o FOR DEVELOPERS
          + GAMING & ENTERTAINMENT
               o GEFORCE
                    # OVERVIEW
                    # 20 SERIES GRAPHICS CARDS
                    # 16 SERIES GRAPHICS CARDS
                    # GAMING LAPTOPS
                    # G-SYNC DISPLAYS
               o GEFORCE NOW
               o SHIELD
                    # OVERVIEW
                    # DISPLAYS WITH SHIELD TV
               o FOR CREATORS
               o FOR BROADCASTERS
     * DEVELOPERS
          + NVIDIA DEVELOPER
          + DEVELOPER NEWS
          + DEVELOPER BLOG
          + DEVELOPER FORUMS
          + OPEN SOURCE PORTAL
          + TRAINING
          + GPU TECHNOLOGY CONFERENCE
          + CUDA
     * INDUSTRIES
          + GAME DEVELOPMENT
          + HEALTHCARE & LIFE SCIENCES
          + HIGHER EDUCATION
          + INDUSTRIAL
          + MEDIA & ENTERTAINMENT
          + PUBLIC SECTOR
          + RETAIL
          + SUPERCOMPUTING
          + TELECOMMUNICATIONS
          + TRANSPORTATION
          + ALL INDUSTRIES
     * SHOP
     * DRIVERS
          + GEFORCE DRIVERS
          + ALL NVIDIA DRIVERS
     * SUPPORT
     * ABOUT NVIDIA
          + OVERVIEW
          + NVIDIA PARTNER NETWORK
          + AI COMPUTING MODEL
          + TECHNOLOGIES
          + NEWSROOM
          + NVIDIA BLOG
          + RESEARCH
          + WEBINARS
          + EVENTS
          + COMPANY INFORMATION
          + CAREERS
          + INVESTORS
          + SOCIAL RESPONSIBILITY
          + NVIDIA FOUNDATION

     * 
     * 
     * 
     * 
     * 
     * 
     * 
     * 
     * 

     * Privacy Policy
     * Legal Info
     * Contact Us

   Copyright © 2020 NVIDIA Corporation

     * Home
     * Deep Learning
     * Networking
     * Driving
     * Gaming
     * Pro Graphics
     * Autonomous Machines
     * Healthcare
     * AI Podcast

40 Years on, PAC-MAN Recreated with AI by NVIDIA Researchers

   GameGAN, a generative adversarial network trained on 50,000 PAC-MAN episodes, produces a fully functional version of the dot-munching classic without an underlying game engine.
   May 22, 2020 by Isha Salian
   Your browser doesn't support HTML5 video. Here is a link to the video instead.
   0 Shares
     * 
     * 
     * 
     * 
     * Email

   Forty years to the day since PAC-MAN first hit arcades in Japan, and went on to munch a path to global stardom, the retro classic has been reborn, delivered courtesy of AI.

   Trained on 50,000 episodes of the game, a powerful new AI model created by NVIDIA Research, called NVIDIA GameGAN, can generate a fully functional version of PAC-MAN — without an underlying game engine. That means that even without understanding a game’s fundamental rules, AI can recreate the game with convincing results.

   GameGAN is the first neural network model that mimics a computer game engine by harnessing generative adversarial networks, or GANs. Made up of two competing neural networks, a generator and a discriminator, GAN-based models learn to create new content that’s convincing enough to pass for the original.

   “This is the first research to emulate a game engine using GAN-based neural networks,” said Seung-Wook Kim, an NVIDIA researcher and lead author on the project. “We wanted to see whether the AI could learn the rules of an environment just by looking at the screenplay of an agent moving through the game. And it did.”

   As an artificial agent plays the GAN-generated game, GameGAN responds to the agent’s actions, generating new frames of the game environment in real time. GameGAN can even generate game layouts it’s never seen before, if trained on screenplays from games with multiple levels or versions.

   This capability could be used by game developers to automatically generate layouts for new game levels, as well as by AI researchers to more easily develop simulator systems for training autonomous machines.

   “We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”

   We’ll be making our AI tribute to the game available later this year on AI Playground, where anyone can experience our research demos firsthand.

AI Goes Old School

   PAC-MAN enthusiasts once had to take their coins to the nearest arcade to play the classic maze chase. Take a left at the pinball machine and continue straight past the air hockey, following the unmistakable soundtrack of PAC-MAN gobbling dots and avoiding ghosts Inky, Pinky, Blinky and Clyde.

   In 1981 alone, Americans inserted billions of quarters to play 75,000 hours of coin-operated games like PAC-MAN. Over the decades since, the hit game has seen versions for PCs, gaming consoles and cell phones.
   NVIDIA Researcher Seung-Wook Kim Game Changer: NVIDIA Researcher Seung-Wook Kim and his collaborators trained GameGAN on 50,000 episodes of PAC-MAN.

   The GameGAN edition relies on neural networks, instead of a traditional game engine, to generate PAC-MAN’s environment. The AI keeps track of the virtual world, remembering what’s already been generated to maintain visual consistency from frame to frame.

   No matter the game, the GAN can learn its rules simply by ingesting screen recordings and agent keystrokes from past gameplay. Game developers could use such a tool to automatically design new level layouts for existing games, using screenplay from the original levels as training data.

   With data from BANDAI NAMCO Research, Kim and his collaborators at the NVIDIA AI Research Lab in Toronto used NVIDIA DGX systems to train the neural networks on the PAC-MAN episodes (a few million frames, in total) paired with data on the keystrokes of an AI agent playing the game.

   The trained GameGAN model then generates static elements of the environment, like a consistent maze shape, dots and Power Pellets — plus moving elements like the enemy ghosts and PAC-MAN itself.

   It learns key rules of the game, both simple and complex. Just like in the original game, PAC-MAN can’t walk through the maze walls. He eats up dots as he moves around, and when he consumes a Power Pellet, the ghosts turn blue and flee. When PAC-MAN exits the maze from one side, he’s teleported to the opposite end. If he runs into a ghost, the screen flashes and the game ends.

   Since the model can disentangle the background from the moving characters, it’s possible to recast the game to take place in an outdoor hedge maze, or swap out PAC-MAN for your favorite emoji. Developers could use this capability to experiment with new character ideas or game themes.

It’s Not Just About Games

   Autonomous robots are typically trained in a simulator, where the AI can learn the rules of an environment before interacting with objects in the real world. Creating a simulator is a time-consuming process for developers, who must code rules about how objects interact with one another and how light works within the environment.

   Simulators are used to develop autonomous machines of all kinds, such as warehouse robots learning how to grasp and move objects around, or delivery robots that must navigate sidewalks to transport food or medicine.

   GameGAN introduces the possibility that the work of writing a simulator for tasks like these could one day be replaced by simply training a neural network.

   Suppose you install a camera on a car. It can record what the road environment looks like or what the driver is doing, like turning the steering wheel or hitting the accelerator. This data could be used to train a deep learning model that can predict what would happen in the real world if a human driver — or an autonomous car — took an action like slamming the brakes.

   “We could eventually have an AI that can learn to mimic the rules of driving, the laws of physics, just by watching videos and seeing agents take actions in an environment,” said Sanja Fidler, director of NVIDIA’s Toronto research lab. “GameGAN is the first step toward that.”

   NVIDIA Research has more than 200 scientists around the globe, focused on areas such as AI, computer vision, self-driving cars, robotics and graphics.

   GameGAN is authored by Fidler, Kim, NVIDIA researcher Jonah Philion, University of Toronto student Yuhao Zhou and MIT professor Antonio Torralba. The paper will be presented at the prestigious Conference on Computer Vision and Pattern Recognition in June.

   PAC-MAN^TM & ©BANDAI NAMCO Entertainment Inc.
   Categories: Deep Learning | Research
   Tags: Artificial Intelligence | NVIDIA Research
   Subscribe Widget 

All nvidia news

   

40 Years on, PAC-MAN Recreated with AI by NVIDIA Researchers

   

What’s a DPU?

   

NVIDIA Xavier Achieves Industry First with Expert Safety Assessment

   

While the World Works from Home, NVIDIA’s AV Fleet Drives in the Data Center

   

COVID Caught on Camera: Startup’s Sensors Keep Hospitals Safe

Post navigation

   

Platforms

     * CUDA-X
     * Autonomous Machines
     * Data Center
     * Deep Learning and AI
     * Design and Visualization
     * Healthcare
     * High Performance Computing
     * Self-Driving Cars
     * Gaming & Entertainment
     * NGC

Products

     * DGX Systems
     * DRIVE PX
     * GeForce RTX 20-Series
     * NVIDIA Virtual GPU
     * Jetson
     * Quadro
     * SHIELD TV
     * Tesla
     * T4 Enterprise Server

Developers

     * NVIDIA Developer
     * Developer News
     * Developer Blog
     * Developer Forums
     * Open Source Portal
     * Training
     * GPU Tech Conference
     * CUDA

Corporate

     * NVIDIA Partner Network
     * Careers
     * Contact Us
     * Security
     * Communities
     * RSS Feeds
     * Email Signup
     * Privacy Center
     * Share Your Story Idea

   
   Explore our regional blogs and other social networks
   

     * Privacy Policy
     * Legal Info
     * Contact Us

   Copyright © 2020 NVIDIA Corporation
   USA - United States 

   NVIDIA websites use cookies to deliver and improve the website experience. See our cookie policy for further details on how we use cookies and how to change your cookie settings.
   ACCEPT

   Share this Article

   Friend's Email Address ____________________

   Your Name ____________________

   Your Email Address ____________________

   Comments ____________________

   (BUTTON) Send Email
   
   Email sent!