Using Google Earth as Photo Location Finder

by Gong Liu September 14, 2011 05:19

Introduction

I am an avid hiker. I keep a photolog on Facebook for each of my hikes. The photolog documents places I hiked, things I saw and people I met. The other day when I was compiling my photolog for a recent backpacking trip to Banner Peak in Ansel Adams Wilderness (see a slideshow of the trip at the end of this post), I came across this article on the internet titled "Lonely Grave in the Sierra" by Dr. H. Galic, a professor of Stanford University. The article is about a forgotten memorial to two climbers who died on Banner Peak in the summer of 1934, and how the author relentlessly pursued every lead about the grave, the victims and people involved, and finally put the history puzzle pieces together to recreate the scenario of what happened on that fateful day. Along the way he recounted the history of mountaineering accidents and rescues in High Sierra. It's like an episode of History Detectives, a truly fascinating story.

As much as I enjoy the article I can't help but notice there is no mention about the exact location of the grave. In fact, Dr. Galic has some concerns about it:

"I am intentionally being vague about the exact location of the grave. If you are a genuine hiker or climber, and familiar with the area, you will easily recognize geographical features mentioned in the text. For others, who might see the grave site as a tourist attraction, and plan to come solely for that reason: Please do not bother, you will never find it."  

To me this only raises curiosity. First of all, I didn't know there was a grave before my Banner Peak trip. Now that I've learned about the grave and the story behind it, I really want to know where it is located so I can pay a visit to it next time around. Secondly, I'm a software engineer specialized in digital maps, GPS, and location-based services. I'm obsessed with finding locations. Call it occupational habit. I can't rest without putting the grave's location on a map somehow!  

So what exactly are the geographical features mentioned in the text that may help me locate the grave? Banner Peak, Mt Ritter, Lake Catherine, and Thousand Island Lake are mentioned multiple times in the text. These landmarks help to establish the general area of the grave. So at least I know in which "haystack" I should look for the "needle". Then he mentioned some remote valley where he found the grave on his way to the back side of Mt. Ritter. This helps to narrow down the search further because there are only a few valleys that can be used to access the back side of Mt. Ritter with reasonable effort. But that's pretty much as far as one can go by the text. Fortunately, in the article Dr. Galic included a few photos of the grave with mountains and valleys as backdrop. This makes me think: If a picture is worth a thousand words, is it possible that I can deduce location information from it? Since the grave is rather small, the photos must have been taken in close range. So if I find the location of the photos, I find the grave. 

Incidentally, I'm not the only one who is interested in finding the location of a photo. CIA/NSA is interested in it too! A recent IARPA (Intelligence Advanced Research Projects Activity) Solicitation expresses the desire of wanting a photo location finder. And this news article reveals that the spy agencies' intention is to use the finder to track down terrorists from propaganda photos and videos. I'm sure if such a system is ever developed, my little grave location mystery can be solved with it in a snap of fingers. But before then, before millions of dollars are spent on the project, let me just show you what can be done right now with existing technology.                

Getting the context of a photo 

Before we can find the location of a photo in any outdoor terrain, we need to know its context, such as its general area (country, region, time zone, nearby city, etc.), date/time, and perhaps camera settings. A good place to start for such information is the photo's  Exif (Exchangeable image file format) header. If we are lucky, if the photo happens to be "geotagged", we can get its exact location from the header. There are tools we can use to dump the header and see everything in it. Fig. 1 is a photo taken with a smartphone. What is special about it is that it is geotagged - when the photo was taken, the smartphone's built-in GPS was on and its latitude and longitude coordinates were written to the photo's Exif header. Fig. 2 is a dump of the Exif header. Toward the end of it are the latitude and longitude coordinates. If we upload the photo to an Exif-aware website such as Flickr, we can see the photo's exact location on the map. Please refer to my earlier post for more info about using geotagged photos.  

Fig. 1. A geotagged photo taken with a smartphone. Click to see original. 

Fig. 2. Exif header of above photo. 

Many smartphones are capable of tracking locations by means other than GPS. They use cell towers and WiFi networks for locating aid. The locations obtained this way are not as accurate as GPS, and thus may not be used for geotagging photos that are taken with these smartphones. But they are good enough for providing context information, assuming, of course, we have access to the location data and are able to use it to derive a photo's approximate location based on its time stamp. This article shows a visualization of iPhone location data. It is interesting to note that Apple's location tracking practice caused such an uproar for privacy concerns that Congressional hearings were held. 

Only photos taken with newer generation of digital cameras or smartphones may contain geotags. Most photos don't. Photos that have been edited with photo editing software may lose their original headers. Photos that have been uploaded to some websites (e.g. Facebook) may get their headers trimmed to bare minimum. And then there are software programs that allow a user to remove infomation from a photo's header before publishing. In all these cases we have to derive the context information of a photo from sources other than Exif header. For instance, the most obvious way is to ask the person who took the photo or the publisher who published the photo. Or we may extract that information from the photo's caption or related text, such as the case of our grave site photos. The point is that getting the context of a photo is more of an intelligent work than a software engineering one. Whoever wins the IARPA bid is likely to follow some sort of heuristic approach that incorporates human intelligence into the search algorithm.  

Finding photo location with Google Earth

Assuming we know the context of a photo, how are we going to find its location or the location of an object in the photo? The idea is simple: We'll need a 3D digital ground model, such as Google Earth. We identify some recognizable geographic features from both the photo and the digital model. We try to match the photo's perspective in relation to the geographic features with those in the digital model. We estimate the location of the object visually in the digital model, which yields a location measurement in terms of latitude, longitude, and elevation.

In Fig. 3 the left side is one of the original photos of the grave site by Dr. Galic. Notice that the grave is just a small bronze plaque on top of a pile of rocks. Since I have a rough idea about the general area from the story as well as from my field trip, I can quickly zoom in the area in Google Earth and locate the valley where the grave is supposed to be. With some tweaks I can make Google Earth display a perspective similar to that of the photo in relation to the background mountains, as shown in Fig. 3, the right side. By referencing the grave location in the photo I can mark my best estimate of the location of the grave in Google Earth with a pushpin. The pushpin location measurement can then be read at bottom-center of the Google Earth screen (in the red oval). It's just this simple! 

Fig. 3. Original photo #1 (left) by H. Galic and matched perspective in Google Earth (right). Click to enlarge. 

From this exercise the following points should be noted:

  • We got to appreciate the importance of the context information of a photo. Without it, we wouldn't know where to start. Sifting through all possible mountains, all possible perspectives would be really like finding a needle in a haystack.
  • The matching of the perspective of the photo in Google Earth involves quite a bit tweaking, manually. This is where I see the value of using an automated computer program for the job, as described in the IARPA Solicitation "automated geolocation technologies to work efficiently and accurately over all terrain and large search areas". To be fully automatic, the program must be able to recoganize by itself what key points or features to match between the photo and the digital model, a task that is easy for a human but can be very challenging for a computer.
  • Unlike Google Street View which provides photo realistic environment, Google Earth in rural areas or wilderness areas is only an approximate representation of reality. It lacks the necessary details to identify, for example, the spring and the meadow mentioned in the article, or the rock outcrop in the foreground of above photo. As a 3D model, it is created by interpolating surfaces from survey data points, and thus is limited in accuracy by whatever data source used.  

Because of all the inaccuracy and uncertainty involved from various sources, the photo, the digital model, the underlying data, and human factors, it would be nice to have an estimate of the error of our method. This can be done by having additional observations from either different photos by same observer or same photo by different observers. Observations of same photo by same person are not statistically valid because later oberservations may be influenced by earlier ones. Fig. 4 shows, side by side, a second photo of the grave site by Dr. Galic and a matched perspective in Google Earth, as well as a different estimate of the grave location (the yellow pushpin).

Fig. 4. Original photo #2 (left) by H. Galic and matched perspective in Google Earth (right). Click to enlarge. 

Now we have two observations resulting two estimated grave locations. Based on this data we can establish a set of concentric circular search areas that look like a bull's eye as shown in Fig. 5. The center of the bull's eye is the mid point of the line segment with the two estimated grave locations as end points. The radius of the innermost circle (red) is equal to half of the length of the above line segment. This radius actually represents one standard deviation of the observations in X-Y plane. The radius of the middle circle (yellow) is simply 2 times of the standard deviation, and the radius of the outermost circle (blue), 3 times of that. According to the 3-sigma rule, if the observation follows a normal distribution (it usually does as the error is from multiple independent sources), the probability of finding the grave in the red circle is about 68%, the probability in the yellow circle, 95%, and the probability in the blue circle, 99.7%. In our case, the largest search area (blue) is only about 600 ft across. Conducting a search in it should be quite manageable.

The bull's eye is drawn on a contour map. If we know information about the elevation of the grave, we can narrow down the search further, by searching only an elevation range within the bull's eye.

Note the above error calculation only applies to errors introduced when trying to locate a photo or an object on the photo in a digital model. It does not consider errors in the field, which can only be estimated with field data.   

 

Fig. 5. Bull's eye search areas. Click to enlarge. 

 

Now that I have marked the grave location on the map, I can finally rest easy. But I still have to find the real thing next time I'm there. I have high confidence that I will find it, and when I do I will take a picture of it with my geotagging-capable camera. That's when I will be really, truly satisfied Laughing

Bull's eye calculation for N observations

Let's say we have N observations of the location of an object of interest from a digital model  

(1)

where, xi - longitude, yi - latitude. 

The center of the bull's eye, (μx, μy), can be calculated as follows:

(2)
(3)

The standard deviations of longitude and latitude are given by Equs. 4 and 5, respectively.

(4)
(5)

The radiuses of the three circles, from the smallest to the biggest, can then be calculated as follows:

(6)
(7)
(8)

Slideshow: Banner Peak Expedition

Comments are closed

About

A seasoned computer professional. A tofu culture evangelist...
more >>

Tag Cloud

Calendar

<<  April 2017  >>
MoTuWeThFrSaSu
272829303112
3456789
10111213141516
17181920212223
24252627282930
1234567

View posts in large calendar
Copyright © 2008-2011 Gong Liu. All rights reserved. | credits | contact me
The content on this site represents my own personal opinions, and does not reflect those of my employer in any way.