Flickr as a database?

Nine Ladies stone circle, viewed from the King Stone

Nine Ladies stone circle, viewed from the King Stone

I went out on a photo trip yes­ter­day. I’ve been test­ing the iPhone app GeoLogTag recently. For some reason I simply can­not get my iPhone talk­ing to my MacBook, which makes a lot of the WiFi syncing apps a waste of time. GeoLogTag has an option to geotag pho­tos on Flickr. If you visit this photo’s page on Flickr and look at the data on the right side, you’ll see it was taken near Stanton in Peak and there’s a map so you can see other nearby pho­tos.

Once you’ve remembered to leave the phone on, which means can­cel­ling the auto-lock, which geo-logging, then tag­ging is pretty much pain­less. The phone keeps track of where it is at any given time. It con­nects to Flickr and any ungeot­agged pho­tos that were taken while the phone was log­ging get tagged. This works because the pho­tos have a record of the time they were taken. If you remem­ber to take your phone around with you when to take a shot, then you can have a reas­on­ably accur­ate log of where you are. At $5 it’s a lot more sane than buy­ing one of those ded­ic­ated photo-loggers you can buy. On the other hand you do have to have the phone to go with it.

Being able to log pho­tos quite accur­ately could well be very use­ful next time I go on a sur­vey. At the very least it could be a usable way of map­ping the loc­a­tion of many tombs, so long as the tombs are 10m or more apart. Uploading to Flickr to log them is a nuis­ance, I should sort out my home net­work, but it’s pos­sibly also an oppor­tun­ity thanks to the Flickr API. The API makes it very easy to get machine-readable data out of Flickr. Here’s an example.

Fire remains (distant) aligned with two megaliths at Nine Ladies.

Fire remains (dis­tant) aligned with two mega­liths at Nine Ladies.

This is a photo of an align­ment I saw at Nine Ladies. If you fol­low the line from fore­ground over the back stone and on, you might be able to see two black patches in the grass. These are the ash remains of fires which had been lit at the site. Burning, even small fires, is not help­ful. The risk of rampant fire is slim because the grass is short and, because it’s in the UK, it’s nearly always damp. However, at the very least it’ll cause a spike in any future mag­neto­metry sur­veys of the area. Also if there is some­thing bur­ied beneath the ground which hasn’t been found yet, heat­ing is a great way to des­troy it. So I now have some data which I might want to log in a database.

I can put some inform­a­tion about a site into Flickr tags. In this case it’s a Bronze Age stone circle on Stanton Moor with some dam­age. So long as I have a uni­form set of tags I use to describe that like “loc­a­tion: Stanton Moor” then I can use the API as a search engine. For example I could:

Search for any photos where tag == project: My Project

which would list all pho­tos in My Project.
It would be easier to be able to lift the data from Flickr and put it into a SQL data­base but that shouldn’t be dif­fi­cult with stand­ard tags. Here’s an example of what inform­a­tion Flickr has about the photo above.

<rsp stat="ok">
  <photo id="3890427837" secret="c89e5a1e94" server="3530" farm="4" dateuploaded="1252190752"
   isfavorite="0" license="5" rotation="0" originalsecret="3a9cc5e6bf" originalformat="jpg"
    <owner nsid="79983635@N00" username="Alun Salt" realname="Alun Salt" location="Derby and 
     Leicester, UK"/>
    <title>Alignment at 9 Ladies.jpg</title>
      Or, how not to photograph burning. If you follow the line from the foreground over the 
      back stone and beyond, you might be able to see two black patches in the grass.
      <a href=";size=large" rel="nofollow">
      A larger view might help.</a>

      Burning, even small fires, is not helpful. The risk of rampant fire is slim because the 
      grass is short and, because it's in the UK, it's nearly always damp. However, at the very 
      least it'll cause a spike in any future magnetometry surveys of the area. Also if there 
      is something buried beneath the ground which hasn't been found yet, heating is a great way
      to destroy it.

      I took the photo from this angle as there were some people behind me, which meant a HDR 
      shot from the opposite direction wasn't viable at the time. When I did set up some shots 
      from the other side I'd forgotten about getting the shot of the fires.
    <visibility ispublic="1" isfriend="0" isfamily="0"/>
    <dates posted="1252190752" taken="2009-09-05 18:58:27" takengranularity="0" 
    <editability cancomment="0" canaddmeta="0"/>
    <usage candownload="1" canblog="0" canprint="0"/>
      <note id="72157622266401280" author="79983635@N00" authorname="Alun Salt" x="233"
       y="108" w="29" h="16">A patch from a small fire.</note>
      <note id="72157622141844777" author="79983635@N00" authorname="Alun Salt" x="238" 
       y="95" w="31" h="16">A patch from a larger fire.</note>
      <tag id="204526-3890427837-771" author="79983635@N00" raw="archaeology"
      <tag id="204526-3890427837-1520781" author="79983635@N00" raw="stanton moor"
      <tag id="204526-3890427837-98094" author="79983635@N00" raw="bronze age"
      <tag id="204526-3890427837-110" author="79983635@N00" raw="UK" machine_tag="0">uk</tag>
      <tag id="204526-3890427837-5933" author="79983635@N00" raw="Derbyshire"
      <tag id="204526-3890427837-778" author="79983635@N00" raw="megalithic"
      <tag id="204526-3890427837-38882" author="79983635@N00" raw="prehistoric"
      <tag id="204526-3890427837-56210" author="79983635@N00" raw="HDR"
      <tag id="204526-3890427837-45290559" author="79983635@N00" raw="project: test project"
      <tag id="204526-3890427837-45290561" author="79983635@N00" raw="location: Stanton Moor"
      <tag id="204526-3890427837-45290563" author="79983635@N00" raw="period: Bronze Age"
      <tag id="204526-3890427837-45290565" author="79983635@N00" raw="type: stone circle"
      <tag id="204526-3890427837-45290567" author="79983635@N00" raw="damage: yes"
      <url type="photopage"></url>

So with some fairly simple code I could get Flickr and the data­base to talk to each other.

GET all photos WHERE project == My Project
FOR EACH photo_id in the list of results
 { GET INFO for photo_id
   INSERT into database id = photo_id;
   READ latitude tag;
   INSERT into database latitude = latitude_tag;
   READ longitude tag;
   INSERT into database longitude = longitude;
   READ period tag;
   INSERT into database period = period_tag;
   READ location tag;
   INSERT into database location = location_tag;
   AND SO ON with the rest of the relevant tags;

You could use the photo date tags to log when the site was vis­ited, the author tag to log who vis­ited it and so on. There are some poten­tial prob­lems. You can add tags to my pho­tos because I let any­one tag them. That could come back to bite me if someone decides it would be fun to add a period: Neolithic tag onto the photo. However if you look at the tags in the data file it also notes who is author of those tags, so you can alter the search to reads TAGS where author == PERMITTED AUTHOR. The API also allow you to search groups, so a group of people could con­trib­ute to a pool of data if there were agreed tag­ging standards.

It’s also just occurred to me that if you made pro­ject tags in the format Project Identifier : Tag : Content like StonehengeSurvey:Location:In the pub, then the same pho­tos could be incor­por­ated into mul­tiple pro­jects, so there’s no need to have a uni­ver­sal defin­i­tion of terms like Early or Late.

Another thing I’d be wary of it mak­ing my data pub­lic as I cre­ate it. Some things I’m work­ing on are likely to be very simple. If I’m out in the field and you can read my data from your home you could be in a pos­i­tion to scoop me, even if you credit me as the source of data. One way round that is to upload pho­tos as private. GeoLogTag is happy work­ing with private pho­tos (I’ve tested it). A group work­ing together would have to make use of the con­tacts fea­ture in Flickr, pos­sibly accept­ing that for the pro­ject they’re all fam­ily so they share access to each other’s uploads.

Certainly my future pro­jects are going to have a very large photo com­pon­ent to them. Geotagging is the hook which everything else can use­fully fit on to. What’ll be inter­est­ing to see is what effect this will have on GIS, if any­one can eas­ily pro­duce masses of geo­graph­ic­ally use­ful data.

Time Savers


Normally blog entries here are writ­ten days in advance, so when I have busy days things con­tinue as usual. For the past couple of weeks I’ve been very busy, hence the lack of blog­ging. On the plus side I have had some help from other people who found me some time sav­ing tools.

Mick Morrison has been read­ing the Google Earth EULA, like we all did before we clicked ‘accept’. He points out that the guidelines on the Google site say:

You may use Google Maps and Google Earth con­tent includ­ing pho­to­graphic imagery in bro­chures, mar­ket­ing col­lat­eral, pack­aging, trade show displays/banners, news­pa­pers, aca­demic pub­lic­a­tions, journ­als, and books.

Quite reas­on­ably he’s ask­ing why they aren’t being used in aca­demic pub­lic­a­tions. The answer in my case is that it didn’t occur to me that get­ting per­mis­sion would be simple. I’ll be using the maps in my thesis now.

The other big time-savers are the applets from The Nebraska Astronomy Applet Project. What I need to do is cre­ate some dia­grams like the one below show­ing how appar­ent star paths change with lat­it­ude, how the sun­rises over dif­fer­ent parts of the hori­zon at dif­fer­ent times of the year, and so on. The NAAP Astronomy Labs have put together some really use­ful tools for this, so now I can include images like this one:

Paths of the stars over Sicily.

If you visit their site you’ll see the out­puts are big­ger, bet­ter qual­ity with vec­tor graph­ics and inter­act­ive so you can anim­ate them. They’re very kindly allow­ing me to use prints, which will save me a lot of time and frus­tra­tion with my graph­ics tablet.

Now all I need is some afford­able way of cre­at­ing cir­cu­lar his­to­grams on a Mac. Suggestions are welcome.

Making Memory Maps


Colleen Morgan demon­strates how she can track the past with her phone. Photo (cc) Miss Colleen.

Mbedr is a tool for embed­ding Flickr pho­tos with notes into other web­sites. This is not some­thing I would nor­mally bother with and it seems I’d be miss­ing a huge oppor­tun­ity. Colleen Morgan has real­ised that if you get a satel­lite photo, then draw­ing notes over a photo is an effect­ive way of mak­ing an annot­ated map. An example below is from Miss Colleen and is part of the Remixing El Presido project.

San Francisco Presidio Memory Map

That’s inter­est­ing, but by itself it’s a curio rather than some­thing excit­ing. What is excit­ing is that Colleen Morgan has really thought about what you can do with Flickr’s fea­tures. It’s a beau­ti­fully eleg­ant idea. Because above, what you see is a photo, albeit a photo with notes. However because Flickr also allows geo­graph­ical inform­a­tion to be asso­ci­ated with a photo you can give it a place. Now comes the stroke of genius.

Add in tri­an­gu­la­tion from GPS or WiFi and sud­denly you have a GPS which not only knows where you are but also where the past was too.

Impressing people when you have some­thing shiny and new isn’t too dif­fi­cult. Taking a lot of pub­licly avail­able tools together to pro­duce some­thing new is a dif­fer­ent mat­ter. While the tech­no­logy might be inter­est­ing what really drives this is the idea, which is excel­lent. Best of all, and this is even more excit­ing, this looks prac­tical. I don’t want to down­play the skill involved, but it looks like any­one could feas­ibly do this. Usually innov­a­tions in archae­olo­gical map­ping require highly expens­ive equip­ment, not least hun­dreds of pounds for a GIS and thou­sands for a Total Station. The only fly in the oint­ment is at WordPress​.com.

The reason for post­ing this here was to see what it was that needed debug­ging in Mbedr. A quick look makes it sadly obvi­ous. WordPress​.com strips JavaScript from posts — for entirely under­stand­able secur­ity con­cerns. It makes it impossible to embed Flash which I used to use a lot, QTVR style pan­or­a­mas or vari­ous movie formats. Sadly you can add Mbedr to this list.

I’m ter­ribly pleased that I’m not on a pro­ject which would bene­fit from this kind of approach. It’s such a good idea I’d want to copy it and that wouldn’t be fair because it’s someone else’s idea and a very good one. You should read about it on her web­log.

Wiltshire and its 21st century SMR

Stonehenge, Facing the Midwinter Sunset

This is how Tom Goskar cas­u­ally tosses a cat amongst the pigeons…

If you’re inter­ested in the archae­ology of the county of Wiltshire, you can now access the Wiltshire Sites and Monuments Record (SMR) online, com­plete with a map interface.

The SMR is the archae­olo­gical record of a county and, as Tom points out, Wiltshire is the county with Stonehenge in it.

Adding the map inter­face is one of these things which sounds simple, but which hardly any­body offers. I can find my house on a map, but I couldn’t tell you the OS grid ref­er­ence, nor parse grid ref­er­ences from loc­a­tions in the record to work out where they lie in rela­tion to me. It doesn’t add any new inform­a­tion to the SMR but chan­ging the inter­face makes it a lot more access­ible. If the SMR data­bases over all UK counties were opened up then this kind of approach would be a massive help for any­one who’s inter­ested in their local archae­ology rather than their local admin­is­trat­ive district.

If you don’t know the names of local vil­lages or par­ishes, or OS ref­er­ences, but are inter­ested in Stonehenge and its sur­round­ings, then Wiltshire Council’s action is a import­ant as any major book the subject.

Unfortunately the map isn’t appear­ing for me. I don’t know if that’s my browser or the work on the council’s serv­ers caus­ing the prob­lem. Hopefully you’ll have bet­ter luck. As you can see on Tom’s page, it cer­tainly worked for him.

Freedom is…

Baghdad Wall
Not Berlin. Photo (cc) this­chan­ginglife.

The Occupation is finally bring­ing the peoples of Iraq together in peace. I’ve tagged a blog post in del​.icio​.us Baghdad res­id­ents protest US-erected divid­ing wall on Annotated Life which notes that Sunnis and Shi’ites are hand in hand in oppos­i­tion to the con­struc­tion of the latest wall across the city. I briefly noted that in another con­text it could pass for an Iron Curtain. With a bit more though I think I was wrong. What has sur­prised me is after think­ing about it some more there may be a grim­mer his­tor­ical com­par­ison.
Continue read­ing

Angkor Wat and Urban Sprawl

Angkor Wat
Sunset at Angkor Wat. Photo (cc) Cap’n Surly.

The big story catch­ing my eye at the moment is the dis­cov­ery that there’s a lot more to Angkor Wat than pre­vi­ously thought. To some extent that shouldn’t be too sur­pris­ing. The site is boasts massive build­ings and is care­fully planned. There’s some stun­ning engin­eer­ing and hydraul­ics which feeds a net­work of pools. The prob­lem is find­ing where the extra set­tle­ment is. The dis­cov­ery of Angkor Wat was in the six­teenth cen­tury, but ser­i­ous work only really star­ted with the reports of Henri Mouhot in the 19th cen­tury. The big prob­lem is find­ing the sites. Angkor is the ste­reo­typ­ical Lost City in the Jungle. The solu­tion is to use radar which the Greater Angkor Project has been doing to look for plant growth and moisture.

You can’t build a massive city without there being some envir­on­mental impact, and the trick here is to see how plants grow after the site has been aban­doned. Places where trenches were dug and ditches cut stay slightly damper than nor­mal. Soil that accu­mu­lates over walls in con­trast is bet­ter drained. This cre­ates dif­fer­ences in in plant growth and pro­duces images that look a bit like an x-ray or out­line of the build­ings. The res­ults have been stun­ning. To quote Damien Evans, the Deputy Director of the pro­ject, “We have iden­ti­fied over a thou­sand new man­made ponds and at least 74 long-lost temples, by cor­rel­at­ing the radar data with on-the-ground sampling.“
Continue read­ing

Displaying astronomical alignments in academic papers


[Cross-posted to i-Science]

Segesta alignment
Astronomical align­ment at Segesta

There’s a couple of paper which have come out recently which use dif­fer­ent tech­niques for indic­at­ing astro­nom­ical align­ments at archaelo­gical sites. The image above is one I put together for a poster to show why hori­zon alti­tude is import­ant as well as azi­muth. It’s quite tight, so it’d be no good if you wanted to see where sun­rise was in mid­sum­mer for instance, and chart­ing the paths of astro­nom­ical bod­ies over a site is a prob­lem. By and large you can treat a site as a small flat area, so there’s not usu­ally any car­to­graphic prob­lems in account­ing for the curvature of the earth. The sky in con­trast is very curved over every archae­olo­gical site, so how to you dis­play that in a paper?

The Megalithic Portal put me on to an inter­est­ing art­icle pub­lished in Information Visualization: A Sky Dome visu­al­isa­tion for iden­ti­fic­a­tion of astro­nom­ical ori­ent­a­tions by Georg Zotti. The abstract includes:

This paper presents a novel dia­gram com­bin­ing archae­olo­gical maps with a folded-apart, flattened view of the whole sky, show­ing the local hori­zon and the daily paths of the Sun, Moon and brighter stars. By use of this dia­gram, inter­est­ing group­ings of astro­nom­ical ori­ent­a­tion dir­ec­tions, for example, to cer­tain sun­rise and sun­set points could be iden­ti­fied, which were evid­ently used to mark cer­tain days of the year.

Unfortunately Information Visualization isn’t a journal archae­olo­gists get, and it costs $30 to down­load the paper. What I can talk about though is a con­fer­ence paper on his own site: A Sky Dome Visualisation for Identification of Astronomical Orientations, which includes in the abstract:

This paper presents a novel dia­gram com­bin­ing archae­olo­gical maps with a folded-apart, flattened view of the whole sky, show­ing the local hori­zon and the daily paths of sun, moon and brighter stars. By use of this dia­gram, inter­est­ing group­ings of astro­nom­ical ori­ent­a­tion dir­ec­tions, e.g. to cer­tain sun­rise and sun­set points could be iden­ti­fied, which were evid­ently used to mark cer­tain days of the year.

The two look as though they’re likely to be similar.

The idea is actu­ally rather clever and I’ll go through a very sim­pli­fied ver­sion of a dia­gram based on his method.
Continue read­ing