What is Photosynth?

Response from Roger Meike, Senior Director of Sunlabs:

Photosynth is a project partnered with University of Washington and other parts of Microsoft. It's a way to allow photos from a variety of sources to come together in a synthesized world. Today you can play with it online. Notice as we open it up, the first thing you’re seeing is this point cloud. It’s a consequence of having mashed-up hundreds of photos together to see how they spatially relate to each other. This construction of this 3-D world was automatically constructed with no human intervention, it’s entirely algorithmic. So this can give you an experience of being able to traverse through a place. So you can pan around, zoom into the details. We can zoom in to a six megapixel image. You're going to get this experience on a web browser because we have the 32-bit code with the capability of doing really clever caching, really clever animation between scenes, the extensive real-time work, and on the back end we’re hosting gigabytes of data, so it’s really a story about how the best of both worlds combines client server assets. This is splatter mode, and it just looks like a splatter of photos. Notice that the ones around the perimeter of the center have more similarity to the center photo, and as we click that changes it. That similarity metric between photos comes out of the analysis we do offline. So again, just showing some of the wonderful detail that is available is pretty exciting. You’re now seeing St. Peter's, watch as we take a step back and then dive in for more detail and you see how the continuity is preserved as we go from one photo to another. That represents the camera angles where the photographers were standing and where they were taking their pictures. There’s a little photo here that’s kind of hard to find because it’s captures the serendipity that happens, there's a celebrity who happened to be on vacation at the same time, and we were quite shocked to see that particular celebrity there at the same time. There's Stephen Hawkins on vacation.

One thing we found with this project is we had certain ideas what would be compelling, and it took a while for us to start doing interior shots. One of the first was inside an art gallery, we’re going to take a walk around his gallery, we get continuity, and then we’ve also done high resolution shots of some of his art and because of the technology, we can zoom in on it, and what looked like masking tape was actually part of the painting, and when you step back you don’t realize that’s actually part of the painting that the artist did. So there's hundreds of photos shamelessly stitched together in a 3-D environment, and we’re really excited about it.

From the splatter mode we go back to 3-D view, and there’s something about capturing something that seems like time-sequence photography. We can step back and say show me the rest of the mountain that they’re climbing, the context, and get a nice panoramic view of the whole thing, how all these photos relate to another. There’s other things we’re not just showing for time, but the white rectangles you’re seeing his showing when you can zoom in on individual photos. When you go to our website to take a look, you’ll be able to browse all the collections but the last one.

Photosynth is at http://labs.live.com/photosynth, in many ways it’s a poster child for the values we talked about. The values are meaningless without something that actually grounds it. We’re creating a richer digital ecosystem that’s entirely novel, and we’re giving it back to the user in terms of what they want to do. I want to go out of my way to thank Noah Snavley, Steve Seitz, Rick Szeliski. This project took four months, and three months to make it more product-oriented to show it to you today. So that’s it.