Rewriting DOIs

One problem with my cunning plan to use Mediawiki REDIRECTs to handle DOIs is that some DOIs, such as those that BioOne serves based on SICIs contain square brackets, [ ], which conflicts with wiki syntax. For example, doi:10.1206/0003-0090(2006)297[0001:TATOL]2.0.CO;2 I want to enable users to enter a raw DOI, so I've been playing with a simple URL rewrite in Appache httd.conf, namely:

RewriteRule ^/wiki/doi:(.*)\[(.*)\](.*)$ /w/index.php?title=Doi:$1-$2-$3 [NC,R]

This rewrites the [ and ] in the original DOI, then forces a new HTTP request (hence the [NC,R] at the end of the line). This keeps Mediawiki happy, at the cost of the REDIRECT page having a DOI that looks a slightly different from the original. However, it means the user can enter the original DOI in the URL, and not have to manually edit it.

From bibliographic coupling to data coupling

Bibliographic coupling is a term coined by Kessler (doi:10.1002/asi.5090140103) in 1963 as a measure of similarity between documents. If two documents, A and B, cite a third, C, then A and B are coupled.

I'm interested in extending this to data, such as DNA sequences and specimens. In part this is because within the challenge dataset I'm finding cases where authors cite data, but not the paper publishing the data. For example, a paper may list all the DNA sequences in uses (thus citing the original data), but not the paper providing the data.

To make this concrete, the paper "Towards a phylogenetic framework for the evolution of shakes, rattles, and rolls in Myiarchus tyrant-flycatchers (Aves: Passeriformes: Tyrannidae)" doi:10.1016/S1055-7903(03)00259-8 lists the sequences used, but does not cite the source of three of these (which is the Science paper "Nonequilibrium Diversity Dynamics of the Lesser Antillean Avifauna" (doi:10.1126/science.1065005). As a result, if I was reading "Nonequilibrium Diversity Dynamics of the Lesser Antillean Avifauna" and wanted to learn who had cited it I would miss the fact that paper "Towards a phylogenetic framework for the evolution of shakes, rattles, and rolls..." had used the data (and hence, in effect, "cited" the paper). In some cases, data citation may be more relevant than bibliographic citation because it relates to people using the data, which seems a more significant action than simply reading the paper.

Note that I'm not interested in the issue of credit as such. In the above example, the authors of the Science paper are also coauthors of the "shakes, rattles, and rolls" paper, and hence show commendable restrain in not citing themselves. I'm interested in the fate of the data. Who has used it? What have they done with it? Has anybody challenged the data (for example, suggesting a sequence was misindentified)? These are the things that a true "web of data" could tell us.

Defrosting the Digital Library

Duncan Hull alerted me to his paper "Defrosting the Digital Library: Bibliographic Tools for the Next Generation Web" (PloS Computational Biology, doi:10.1371/journal.pcbi.1000204). Here's the abstract:

Many scientists now manage the bulk of their bibliographic information electronically, thereby organizing their publications and citation material from digital libraries. However, a library has been described as “thought in cold storage,” and unfortunately many digital libraries can be cold, impersonal, isolated, and inaccessible places. In this Review, we discuss the current chilly state of digital libraries for the computational biologist, including PubMed, IEEE Xplore, the ACM digital library, ISI Web of Knowledge, Scopus, Citeseer, arXiv, DBLP, and Google Scholar. We illustrate the current process of using these libraries with a typical workflow, and highlight problems with managing data and metadata using URIs. We then examine a range of new applications such as Zotero, Mendeley, Mekentosj Papers, MyNCBI, CiteULike, Connotea, and HubMed that exploit the Web to make these digital libraries more personal, sociable, integrated, and accessible places. We conclude with how these applications may begin to help achieve a digital defrost, and discuss some of the issues that will help or hinder this in terms of making libraries on the Web warmer places in the future, becoming resources that are considerably more useful to both humans and machines.

It's an interesting read, and it also <shamless plug>cites my bioGUID project</shamless plug>.

[Image from dave 7]

OpenURL for Genbank records

Following on from adding specimens to my OpenURL resolver, I've added support for GenBank records. Either an OpenURL request such as http://bioguid.info/openurl?id=genbank:DQ502033, or the short URL http://bioguid.info/genbank/DQ502033 will resolve the GenBank record for accession number DQ502033.

The HTML isn't much to look at, the real goodness is the JSON (obtained by appending "&display=json" to the OpenURL request, or ".json" to the short form, e.g. http://bioguid.info/genbank/DQ502033.json).

The resolver gets the sequence form NCBI, does a little post processing, then displays the result. Postprocesisng includes parsing the latitude and longitude coordinates (something of a mess in GenBank, see my earlier metacrap rant), extracting specimen codes, adding bibliographic GUIDs (such as DOIs, Handles, or URLs), finding uBio namebankID's for hosts, etc. Note that some records have a key called "taxonomic_group". This is to provide clues for resolving museum specimens -- often the DiGIR provider needs to know what kind of taxon you are searching for.

The aim is to have a simple service that returns somewhat cleaned up GenBank records that I (and others) can play with.

A Shared Culture

The A Shared Culture video from the Creative Commons web site.

Modelling GUIDs and taxon names in Mediawiki

Thinking more and more about using Mediawiki (or, more precisely, Semantic Mediawiki) as a platform for storing and querying information, rather than write my own tools completely from scratch. This means I need ways of modelling some relationships between identifiers and objects.

The first is the relationship between document identifiers such as DOIs and metadata about the document itself. One approach which seems natural is to create a wiki page for the identifier, and have that page consist of a #REDIRECT statement which redirects the user to the wiki page on the actual article.



This seems a reasonable solution because:

• The user can find the article using the GUID (in effect we replicate the redirection DOIs make use of)
• The GUID itself can be annotated
• It is trivial to have multiple GUIDs linking to the same paper (e.g., PubMed identifiers, Handles, etc.).

Taxon names present another set of problems, mainly because of homonyms (the same name being give to two or more diferent taxa).The obvious approach is to do what Wikipedea does (e.g., Morus), namely have a disambiguation page that enable the user to choose which taxon they want. For example:



In this example, there are two taxon names Pinnotheres, so the user would be able to choose between them.

For names which had only one corresponding taxon name we would still have two pages (one for the name string, and one for the taxon name), which would be linked by a REDIRECT:



The advantage of this is that if we subsequently discover a homonym we can easily handle it by changing the REDIRECT page to a disambiguation page. In the meantime, users can simply use the name string because they will be automatically redirected to the taxon name page (which will have the actual information about the name, for example, where it was published).

Of course, we could do all of this in custom software, but the more I look at it the power to edit the relationships between objects, as well as the metadata, and also make inferences makes Semantic Mediawiki look very attractive.

Google Books and Mediawiki

Following on from the previous post, I wrote a simpe Mediawiki extension to insert a Google Book into a wiki page. Written in a few minutes, not tested much, etc.

To use this, copy the code below and save in a file googlebook.php in the extensions directory of your Mediawiki installation.

<?php
# rdmp

# Google Book extension
# Embed a Google Book into Mediawiki
#
# Usage:
# <googlebook id="OCLC:4208784" />
# 
# To install it put this file in the extensions directory 
# To activate the extension, include it from your LocalSettings.php
# with: require("extensions/googlebook.php");
 
$wgExtensionFunctions[] = "wfGoogleBook";
 
function wfGoogleBook() {
    global $wgParser;
    # registers the <googlebook> extension with the WikiText parser
    $wgParser->setHook( "googlebook", "renderGoogleBook" );
}
 
# The callback function for converting the input text to HTML output
function renderGoogleBook( $input, $argv ) 
{
 $width = 425;
 $height = 400;
 
 if (isset($argv["width"]))
 {
  $width = $argv["width"];
 }
 if (isset($argv["height"]))
 {
  $width = $argv["height"];
 }
    
 $output = '<script type="text/javascript" 
            src="http://books.google.com/books/previewlib.js">
           </script>';
 $output .= '<script type="text/javascript"> 
            GBS_insertEmbeddedViewer(\'' 
            . $argv["id"] . '\','. $width . ',' . $height . ');
           </script>';   

    return $output;
}
?>

In your LocalSettings.php file add the line

require("extensions/googlebook.php");

Now you can add a Google book to a wiki page by adding a <googlebook> tag. For example:

<googlebook id="OCLC:4208784" />

The id gives the book identifier (such as an OCLC number or a ISBN (you need to include the identifier prefix). By defaulot, the book will appear in a box 425 × 400 pixels in size. You can add optional width and height parameters to adjust this.

OCLC and Google books

I've started to come across more taxonomic books in Google Books, such as Catalogue of the specimens of snakes in the collection of the British museum by John Edward Gray. Google books provides a nice widget for embedding views of books. There is tool for generating the Javascript code. Note that in Blogger (which I use to create this blog) you need to make sure that theJavascript occurs on a single line with no line breaks for it to work.



The Javascript used (with linebreaks that must be removed before using) is:

<script type="text/javascript" 
   src="http://books.google.com/books/previewlib.js">
</script>
<script type="text/javascript">
GBS_insertEmbeddedViewer('OCLC:4208784',425,400);
</script>

I stumbled across this book whilst searching for the original record for the snake Enhydris punctata. Confusingly, the Catalogue of Life lists this snake as Enhydris punctata GRAY 1849, implying that Gray's original name still stands, whereas in fact it should be Enhydris punctata (Gray, 1849) as the Gray's original name for the snake was Phytolopsis punctata. It's little things like this that drive me nuts, especially as the Catalogue of Life has no obvious, quick means of fixing this (Wiki, anyone?).

I was also interested in using the OCLC numbers a GUID for the book, but there are several to choose from (including two related to the Google Book). Unlike DOIs, a book may have multiple OCLCs (sigh). Still, it's a GUID, and it's resolvable, so it's a start. Hence, one could link GUIDs for the names published in this book to the book itself.

OpenURL for specimens

As part of the slow rebuild of bioguid.info, and as part of the Challenge, I've started making an OpenURL resolver for specimens. Partly this is just a wrapper around DiGIR providers, but it's also a response to the lack of GUIDs for specimens. In the same way that I think OpenURL for papers only really makes sense in a world without GUIDs for literature (DOIs pretty much take care of that), given the lack of specimen GUIDs we are left to resolve specimens based on metadata.

For example, the holotype of Pseudacris fouquettei (shown in photo by Suzanne L. Collins, original here) is TNHC 63583. In a digital world, I want the paper describing this taxon, and the specimen(s) assigned to it to be a click away. In this spirit, here is an OpenURL link for the specimen: http://bioguid.info/openurl/?genre=specimen &institutionCode=TNHC &collectionCode=Herps &catalogNumber=63583. Click on this link and you get a page with some very basic information on the specimen. If you want more, append "&display=json" to the URL to get a JSON response.

So, armed with this, TNHC 63583 becomes resolvable, and joining the pieces becomes a little easier.

Monitor envy

Mike Sanderson's wall of monitors is getting some attention. Cool as it looks (and I'm positively green with envy), this strikes me as the LCD equivalent of Science's suggestion that the reader print out a tree on several bits of paper doi:10.1126/science.300.5626.169. If we can fit the planet on a monitor, can't we fit the tree of life?

Wired barcoding

The latest issue of Wired has an article on DNA barcoding, entitled "A Simple Plan to ID Every Creature on Earth". The article doesn't say much that will be new to biologists, but it's a nice intro to the topic, and some of the personalities involved.

Biodiversity Service Status

The rather frail nature of biodiversity services (some of the major players have had service breaks in the last few weeks) has prompted me to revisit Dave Vieglais's BigDig and extend it to other services, such as uBio, EOL, and TreeBASE, as well as DSpace repositories and tools such as Connotea.

The result is at http://bioguid.info/status/. The idea is to poll each service once an hour to see if it is online. Eventually I hope to draw some graphs for each service, to get some idea of how reliable it is.

Much of my own work depends on using web sites and services, and I'm constantly frustrated when they go offline (some times for months at a time).

My aim is to be constructive. I well aware that reliability is not easy, and some tools that I've developed myself have disappeared. But I think as a community we need to do a lot better if biodiversity informatics is to deliver on its promise.

The list of service is biased by what I use. I'm also aware that some of the DiGIR provider information is out of date (I basically lifted the list from the BigDig, I'll try and edit this as time allows).

Comments (and requests for adding services) are welcome. There is a comment box at the bottom of the web page, which uses Disqus, a very cool comment system that enables you to keep track of your comments across multiple sites. It also supports OpenID.

Global biogeographical data bases on marine fishes: caveat emptor

D. Ross Robertson has published a paper entitled "Global biogeographical data bases on marine fishes: caveat emptor" (doi:10.1111/j.1472-4642.2008.00519.x - DOI is broken, you can get the article here). The paper concludes:

Any biogeographical analysis of fish distributions that uses GIS data on marine fishes provided by FishBase and OBIS 'as is' will be seriously compromised by the high incidence of species with large-scale geographical errors. A major revision of GIS data for (at least) marine fishes provided by FishBase, OBIS, GBIF and EoL is essential. While the primary sources naturally bear responsibility for data quality, global online providers of aggregated data are also responsible for the content they serve, and cannot side-step the issue by simply including generalized disclaimers about data quality. Those providers need to actively coordinate, organize and effect a revision of GIS data they serve, as revisions by individual users will inevitably lead to confused science (which version did you use?) and a tremendous expenditure of redundant effort. To begin with, it should be relatively easy for providers to segregate all data on pelagic larvae and adults of marine organisms that they serve online. Providers should also include the capacity for users to post readily accessible public comments about the accuracy of individual records and the overall quality of individual data bases. This would stimulate improvements in data quality, and generate 'selection pressures' favouring the usage of better quality data bases, and the revision or elimination of poor-quality data bases. The services provided to the global science community by the interlinked group of online providers of biodiversity data are invaluable and should not be allowed to be discredited by a high incidence of known serious errors in GIS data among marine fishes, and, likely, other marine organisms. (emphasis added)

As I've noted elsewhere on this blog, and as demonstrated by Yesson et al.'s paper on legume records in GBIF (doi:10.1371/journal.pone.0001124) (not cited by Robertson), there are major problems with geographical information in public databases. I suspect there will be more papers like this, which I hope will inspire database providers and aggregators to take the issue seriously. (Thanks to David Patterson for spotting this paper).

Half-baked ideas. II. Degree of interest tree of NCBI taxonomy

Among the many ways to display trees, degree of interest (DOI) trees strike me as one potentially useful way to display trees such as the NCBI taxonomy. For background see, e.g. doi:10.1145/1133265.1133358 (or Google "degree of interest trees").

The thing that would make this really useful is if an application was written that, like Google Earth, supported a simple annotation file format. Hence, users could create their own annotation files (e.g., taxa of a certain size, those with eyes, etc.) and upload those files, creating their own annotation layers, in much the same way as we can load sets of geographical annotations into Google Earth. I think it's this feature which makes Google Earth what it is, so my question is whether we can replicate this for classifications/phylogeny?

Half-baked ideas. I. Wiki for taxonomy

Next few weeks will be busy with term starting, kids visiting, and other commitments, so time to jot down some ideas. The first is to have a Wiki for taxonomic names. Bit like Wikispecies, but actually useful, by which I mean useful for working biologists. This would mean links to digital literature (DOIs, Handles, etc.), use of identifiers for names and taxa (such as NCBI taxids, LSIDs, etc.), and having it pre-populated with data. Imagine merging the NCBI taxonomy, Catalogue of Life, Index Fungorum, and IPNI, say, and having it automatically updated with sources such as WoRMS and uBio RSS. Why a Wiki? Well, partly to capture all the little textual annotations that are needed to flesh out the taxonomy, and partly to make it easy to correct the numerous mistakes that litter existing databases.

As an initial target, I'd aim for a comprehensively annotated NCBI taxonomy, as this is probably the most important taxonomic database that we have.

Displacement activity no. 37 - Wordle

My delicious tags as a Wordle. In my defense I've not had nearly enough coffee today.

Challenge data

Just to provide a sense of how much data I want to analyse for the Challenge, I have the XML, PDF, and images for 1687 articles from Molecular Phylogenetics and Evolution to play with.

Postphylogenetics

Last week I was at NESCent's 2008 Community Summit. As part of that meeting a few of us had a breakout group on "Biodiversity and phylogenetics". Brian O'Meara took some spectacularly thorough notes, including the pithy:

S[wofford]: What?

Julia Clarke and I were advocating data mining, not entirely successfully. At one point I started ranting about post-phylogenetics (i.e., what do do when we've basically got the tree of life). For a brief moment I thought this might be a cool new term to use, although Googling finds that W. Ford Doolittle has used it in the title of talks given at the Wenner-Gren Foundations International Symposium at Stockholm in 2003, and at Penn State in 2006. However, the 2006 talk title (Postphylogenetics: The Tree of Life in the Light of Lateral Gene Transfer) suggests a different meaning (i.e., there isn't a tree of life to be found). I prefer to think of it in the same sense as "postgenomics" -- now that we have all this information, how can we make the best use of it?

When ISSN's disappear, taking DOIs with them

I've been using ISSN's (International Standard Serial Number) to uniquely identify journals, both to generate article identifiers, and as a parameter to send to CrossRef's OpenURL resolver. Recently I've come across journals that change their ISSN, which has fairly catastrophic effects on my lookup tools. For example, the Canadian Journal of Botany has the ISSN 0008-4026, or at least this is what JournalSeek tells me. However, the journal web site tells me that it has been renamed as Botany, with ISSN 1916-2804. The thing is, if I want to look up DOIs for articles published in the Canadian Journal of Botany, I have to use the ISSN for Botany if I want to get a result. Hence, I can't rely on looking up the ISSN for the Canadian Journal of Botany. I've come across this in other journals as well.

WorldCat's xISSN web services provide some tools to help, including a graphical display of the history of a journal and it's ISSN(s). Here is the history for 1916-2790, redrawn using Graphviz. WorldCat use Webdot, which I've written about earlier. If you view the source of the WorldCat page you can get the link to the original dot file.



The problem with these changes is that it makes ISSN's more fragile. Ideally, the original ISSN would be preserved, and/or CrossRef would have a table mapping old ISSN's onto new ones. The rate things are going, I may have to create such a table myself.

Hell is other people's data

Starting to get serious about the Grand Challenge. First step is to parse the XML data Elsevier made available. Sadly this is only for Molecular Phylogenetics and Evolution for 2007, I would have liked the whole journal in XML to avoid hassles with parsing PDF. However, XML is not without it's own problems. I'm slowly getting my head around Elsevier's XML (which is, it has to be said, documented in depth). Two tools I find invaluable are the oXygen XML editor, and Marc Liyanage's TextXSLT application.

As a first attempt, I'm converting Elsevier XML into JSON (being a much simpler format to handle). I'm just after what I regard as the core data, namely the bibliography, and the tables (rich with GenBank accession numbers, specimen codes, and geocoordinates). There are a few "gotchas", such as misisng namespaves to add, and HTML entities that need to be added. Then there's the fact that the XML describes both the document content and it's presentation. Tables can get complicated (cells can span more than one row or column), which makes tasks such as identifying cell contents by using the heading of the corresponding column a bit harder. I hope to put a XSLT style sheet online once I'm happy that it can handle most, if not all the tables I've come across. Then the fun of trying to extract the information can begin.