Maybe you’ve played Pokemon Go. Maybe you’ve trained monsters and made them battle. Maybe you’ve caught them all. While you were doing all of that, did you realize playing a game would improve navigation systems, particularly for food delivery robots? Did you know you may have been capturing images connected to a location? No? How could that be? Let’s talk.

What is Pokemon Go?

Released in 2016, Pokemon Go is a mobile app based game. In the game, players can get points for finding and capturing virtual pokemon characters that are digitally superimposed on the real world. Users can train pokemon and have them battle with other pokemon trained by other players or teams gaining them more points.

How did the game get the images and connect it to location?

In order to find and capture pokemon, players must use their mobile device while walking around in the real world. They have to point their phone toward the pokemon in order to catch them. For this game to work, the app must have permission to use the mobile device’s camera and GPS. Players were able to get rewards for taking pictures or scanning certain landmarks. The camera has GPS, which can give a precise location for the images. Beyond GPS, phones also have accelerometers, magnetometers, and gyroscopes which help sense the phones precise orientation. All of this data could be recorded through the game’s app.

How do the images taken by players enable navigation systems?

Millions of players took a variety pictures from a variety of angles and perspectives. All this data gives a detailed view of various areas, particularly ones not easily accessible by cars. Getting images of these various ares by pedestrian players is important for delivery service robots, which typically navigate on sidewalks as opposed to streets. Further, these images can help robots navigate when cell communication is poor or blocked by infrastructure like buildings and overpasses. The parent company Niantic is using these images to train what they are calling a “large geospatial model”, which kinda sounds like a large language model, but for maps.

My Take

I wonder if everyone who was taking all of these pictures understood how their data would be used. Also, are any other apps using our data in this way? And are there any other people or indoor spaces that are in featured in these pictures unknowingly? I also wonder how this data could be used beyond enabling deliveries. If they can connect image to location so readily, do any of us have real privacy. Even if you have location turned off on your pictures, it could be possible to pinpoint the location just based what is in the background.