LSIDs, to proxy or not to proxy?

The LSID discussion rumbles on (see my earlier post). One issue that has re-emerged is the use of HTTP proxies in RDF documents. In a recent email Greg Whitbread wrote:

The existing TDWG recommendation that "5. All references to LSIDs within RDF documents should use the proxified form", basically states that LSID will never appear in any way other than bundled into an http URI - if we are also to publish data as RDF.

That sounds as if it means that those wanting to use LSID resolution will first have to extract the LSID part from the http URI which will now appear everywhere we would expect to find our unique identifier.

Donald [Hobern] has presented a strong case for unique identifiers conforming to the LSID specification but we have now an equally strong case that in its http form our identifier must behave as a dereferenceable URN per W3C linked data recommendations.

My own view is that the RDF should always contain a canonical, un-proxied version of an identifier (whether LSID or DOI), because:

1. having only the proxied version assumes that there is only one suitable proxy (there may be multiple ones)
2. it assumes that the specified proxy will always exist (our track record in durable HTTP services is poor)
3. assumes the specified proxy will always match conform to current standards
4. it imposes an overhead on clients that want the canonical identifier (i.e., they have to strip away the proxy)

I predict that for any meaningful, successful (read "actually used") identifier there will be multiple services that will be capable of consuming that identifier, not just HTTP proxies. DOIs can be proxied (by several servers, including http://dx.doi.org/ and http://hdl.handle.net ), resolved using OpenURL resolvers, etc.

In order to play ball with Linked Data, there are several ways forward:

1. always refer to LSIDs in their proxied form (see above for reasons why this might not be a good idea)
2. ensure that at least one proxy exists which can resolve LSIDs in a linked data friendly way (see bioGUID as an example)
3. use or develop linked data clients that understand LSIDs (e.g., http://linkeddata.uriburner.com/, see this view of urn:lsid:zoobank.org:pub:2C6BD020-B54A-4119-9693-3231C9FCEFA6)

2 and 3 already exist, so I'm not so keen on 1.

For me this is one of the biggest hurdles facing using HTTP URIs as identifiers -- I have to choose one. As an analogy, I can identify a book using an ISBN (say, 0226644677). How do I represent this in RDF? Well, I could use an HTTP URI, say http://www.amazon.com/Tangled-Trees-Phylogeny-Cospeciation-Coevolution/dp/0226644677/ , or maybe http://www.worldcat.org/isbn/0226644677. There are many, many I could choose from. However, so long as I know that the ISBN is 0226644677, I'm free to use whatever URI best suits my needs. So, what I really want is the ISBN by itself.

Imagine, for example, a publisher such as PLoS or Magnolia Press (publisher of Zootaxa), both of which have recently published taxonomic papers containing LSIDs (e.g., doi:10.1371/journal.pone.0001787). They might want to display LSIDs linked to their own LSID resolver that embellishes the metadata with information they have (e.g., they might wish to highlight links to other content that they host). In a sense this is much the same idea as supported by OpenURL COinS, where OpenURL-format metadata is embedded in a HTML document and the user choose what resolver to use to resolve the links (including tools such as Zotero).

Having LSIDs prefixed with a HTTP proxy makes these task a little harder.

LSIDs, HTTP URI, Linked Data, and bioGUID

The LSID discussion has flared up (again) on the TDWG mailing lists. This discussion keeps coming around (I've touched on it here and here), this time it was sparked by the LSID SourceForge site being broken (the part where you get the code is OK). Some of the issues being raised include:

• Nobody uses LSIDs except the biodiversity informatics crowd, have we missed something?
• LSIDs don't play nice with the Linked Data/Semantic Web world, which is much bigger than us
• If we adopt HTTP URIs, will this send the wrong message to data providers (LSIDs imply a commitment to persistence, URLs don't)
• The community has invested a lot in LSIDs, it's too late to change course now

There are other issues as well, in many ways much harder, namely how to ensure adoption and long term persistence of whatever identifier technology the techies agree on.

I've been twittering (@rdmpage) about some of this, and Pierre Lindenbaum blogged about my earlier paper on testing LSIDs (doi:10.1186/1751-0473-3-2), so I decided to return to one of the original goals of my bioGUID project, namely providing a tool to resolve existing identifiers in a consistent way (see the now moribund bioGUID blog, I now blog about bioGUID here on iPhylo). One of the goals of bioGUID was to take an identifier and return RDF. I also had an underlying triple store that was populated with this RDF. After a hardware crash I took the opportunity to rebuild bioGUID from scratch, focussing on OpenURL access to literature. Now, I'm looking at LSIDs again.

The standard response to the concern that the rest of the world has gone down the HTTP URI route is to say that we can stick a HTTP proxy on the front of the LSID (e.g., http://lsid.tdwg.org/urn:lsid:indexfungorum.org:names:21364) and play ball with the Linked Data crowd, who are rapidly linking diverse data sets together:

However, sticking a HTTP proxy on an LSID isn't enough. As outlined in the document Cool URIs for the Semantic Web, we need a way of distinguishing between a HTTP URI that identifies real-world objects or concepts (such as a person or a car), and documents describing those things (put another way, if I put a HTTP URI for Angelina Jolie into a web browser, I expect to get a document describing her, not Ms Jolie herself) . One solution (and the one that is gaining traction) is to use 303 redirects to make this explicit:

A client resolving a URI for a thing will get a 303 status code, telling them that the URI identifies an object. They can get the appropriate representation via content negotiation (a web browser wants HTML, a linked data browser wants RDF).

Data URIs. So, in order to get LSIDs to play ball with Linked Data we need a HTTP proxy that supports 303 redirects (as Roger Hyam pointed out). I've implemented a simple one as part of bioGUID. If you append a LSID to http://bioguid.info/ you get a HTTP URI that passes the
Vapour Linked Data validator tests. For example, http://bioguid.info/urn:lsid:indexfungorum.org:names:21364 resolves to a web page in a browser, but clients that ask for RDF will get that. You can see the steps involved in resolving this Cool URI here. Vapour provides a nice graphical overview of the process:



The TDWG LSID proxy doesn't validate, so this is something that should be addressed.

In addition to resolving LSIDs, my service stores the resulting RDF in a triple store using ARC, and you can query this triple store using a SPARQL interface that makes use of Danny Ayers' Javascript SPARQL editor. I've a serious case of déjà vu as I've implemented this feature several times before using 3store3 (usually after much fun getting it to work). I got bored with triple stores as the bigger problem seemed to be the errors in the metadata I was harvesting, which seriously limited my ability to link different objects together (but that's another story).

How Shall I Integrate Thee? Let Me Count the Ways...

Leigh Dodds has a nice post How Shall I Integrate Thee? Let Me Count the Ways... about different ways to integrate data.

• The one where we share identifiers
• The one where we're describing the same thing
• The one where we're speaking different languages
• The one where we're using different units
• The one where we're speaking at different levels of abstraction

Apart from the suggestion that Leigh has been watching way too much Friends, there's much food for thought here. I suspect that "The one where we're describing the same thing" is the one I'll be making most use of.

In Rethinking LSIDs versus HTTP URI I argued that most applications will use HTTP URIs, which makes them accessible, but not terribly useful as identifiers, the reason being that I think it is unlikely that people will reuse HTTP URIs ("The one where we share identifiers"). A good example is Connotea, which has its own URIs for each paper its users bookmark. I won't use these URIs as identifiers in my database (if only because if a user resolves them, they get taken to Connotea's web site, not mine). However, I will store any PubMed and DOI identifiers, so that somebody aggregating information from Connotea (say to retrieve user tags) and my database (say, to get links to sequences and specimens) can work out that the Connotea URI and my URI are talking about the same thing.