Imaginary Soundscapes works by using two different neural networks. One is able to identify the content within the different Street View images, while the other can pinpoint audio and separate the sounds and acoustics of the environment. Tokui has put both together for this project; the two neural networks can find open source audio and play it in the background, based on what you’re looking at in Google Street View. For example, on an residential street in a busy Washington, DC, urban neighborhood, I heard an imagined soundtrack of buses, cars and people chatting as they walked by.
You can give it a try yourself at Imaginary Soundscapes’ website; it only works on PC in Chrome and Firefox, though I was able to access it in Chrome on a Mac. By default the site takes you to a random location. You have a choice to input your own street address or visit another random place. The sound will play automatically.
It’s not perfect, of course. Fast Co.Design points out that in the quiet, grass-filled fields of Tokyo Stadium, there are the sounds of cars zooming by. Regardless, it’s a pretty cool AI. You can go more in depth into how it works at the creator’s Medium post and a short paper for a workshop on Machine Learning and Design.