Collaborative data matrices using EditGrid

EditGrid is an online collaborative spreadsheet tool that I stumbled across via Ogle Earth. It strikes me thjat this could be a way to create phylogenetic data sets collaboratively.



As a quick test I grabbed the Vertebrates example file that comes with MacClade, exported the NEXUS file as a table, opened it in Excel, then uploaded the Excel file to EditGrid. You can see the results here.


The spreadsheet is a natural metaphor for phylogenetic data, although in this application is is likely to be better suited to morphological data where a team of people are assembling a matrix from various sources.

The developers of EditGrid have a blog whioch converys their own sense of excitement about this project.

Connotea and TreeBASE

One of my (forever) ongoing projects is to map taxon names in TreeBASE to names in external databases (such as uBio) as a way of checking that the names are correct, adding the ability to handle synonyms, and hierarchical queries (see my earlier post for more details).

Now, many names in TreeBASE aren't in any of the major name databases (fossils seem particularly poorly supported), which means hunting on Google for the name. In some cases I come across the name and the original reference for the name, which means I can document that the name is correct. For example, TreeBASE taxon T8737 is Eocaiman cavernensis, which doesn't occur in any of the name sources I use (uBio, ITIS, NCBI, IPNI, etc.). It's a fossil crocodilian, described by George Gaylord Simpson in 1933.


The original description in American Museum Novitates is online (hdl:2246/2050), courtesy of the AMNH's DSpace server. So, how do I link the name and the publication -- without me creating a new database to do this? Well, Connotea to the rescue. I add Simpson's paper to Connotea, tagged with the TreeBASE TaxonID T8737, and viola, the information is stored.

Now, to make use of this we need to do a little bit more, such as have a triple store that contains both the TreeBASE names and the Connotea record, but given that Connotea serves RSS 1.0 (i.e., RDF), this is easy.

What I like about this is:

1. I don't have to do much work


2. The publication information is stored where others can see it and make use of it (i.e., if my experiments with these ideas fall by the wayside, the data still remain).

Now, back to the tedious task of mapping...

Small Pieces Loosely Joined

Just finished reading Small Pieces Loosely Joined. It's a fabulous essay on the nature of the Web. The more I read it the more it confirms my fear that most people talking about biological taxonomy and biodiversity on the Web simply don't "get" the Web. Adopting the Web successfully will require a willingness to accept error, ambiguity, and downplaying "expertise" and "authority". It will be interesting to see what happens.

TreeBASE rocks

I gave a talk today ("Dude, where's my tree?") at the Evolution 2006 meeting at Stony Brook. It was intended as a somewhat tongue-in-check overview of some issues concerning TreeBASE, and broader areas of biodiversity informatics, making use of ants as an example (see my SemAnt project).
Michael Donoghue took me aside after the talk and made some interesting points. He was a little tired — understandably — of hearing that "TreeBASE sucks" (e.g., my CIPRES talk), and felt that my constantly saying this was counter productive. It could also lead to people not putting their data in TreeBASE because they'd heard that it "sucks".
There is an element of social responsibility here, I guess. I resolutely avoid politics. I don't mean this in a pejorative sense, it's just that I don't have the temperament or skill for it, unlike Michael himself (Lee Belbin is another person in this area who strikes me as a very skilled manager).
Now, my talk was intended to be fun, and I was taking the piss out of myself as much as anything. I also think the things we criticise are the things we value the most. But that said, let be make it clear that TreeBASE is very important. As editor of Systematic Biology I've made authors submit data to it. I have a lot of respect for the work Michael, Bill Piel, and Mike Sanderson put into TreeBASE. If you have phylogenetic data — submit it to TreeBASE. It's the best we have. It's just that, well, as a community, we could do better.

Taxonomic names, metadata, and the Semantic Web

My paper "Taxonomic names, metadata, and the Semantic Web" has appeared in Biodiversity Informatics.

Life Science Identifiers (LSIDs) offer an attractive solution to the problem of globally unique identifiers for digital objects in biology. However, I suggest that in the context of taxonomic names, the most compelling benefit of adopting these identifiers comes from the metadata associated with each LSID. By using existing vocabularies wherever possible, and using a simple vocabulary for taxonomy-specific concepts we can quickly capture the essential information about a taxonomic name in the Resource Description Framework (RDF) format. This opens up the prospect of using technologies developed for the Semantic Web to add ``taxonomic intelligence" to biodiversity databases. This essay explores some of these ideas in the context of providing a taxonomic framework for the phylogenetic database TreeBASE.

More on trees and Google Earth

Well, turns out Bill's not the only one putting trees on Google Earth. Declan Butler pointed me to Ogle Earth, where there is a teaser of some work on guiology.

Currently playing in iTunes: Crazy In Love by Beyoncé

Avian flu, phylogeny, and Google Earth

The penny just dropped (duh!).
Having mentioned Bill Piel's very cool visualisation of phylogenies on Google Earth

what about the other cool use of Google Earth in biology, namely Declan Butler's displays of the march of avian flu?

Instead of standard diagrams like this one from the Ruben Donis' paper in Emerging Infections Diseases:

why not take phylogenies for avian flu virus and add them to the data Declan is displaying? This could be a potentially compelling graphic, and a test of whether phylogenies add useful information to our understanding of what is going on.

TreeBASE meets Google Earth

Bill Piel has created a cool tool for creating KMZ files of phylogenies for Google Earth. From the web site:

One of the components of the CIPRES project is the development of TreeBASE II — a robust, scalable, and versatile re-design and re-engineering of TreeBASE. As part of this project, we are exploring other ways of browsing and visualizing trees. Google Earth is a fantastic 3-D browser for exploring geographic resources and has the potential to be a useful and fun tool for delivering biological information with a geographic component.

Google Earth (available for Windows and Mac OS X) is opening up all sorts of possibilities for biodiversity informatics (ants being one of the first examples). What is cool about Bill's work is that it departs from simple locality records.

As always, after pausing to say "wow", there are all sorts of things that one could think of adding. For example, some trees are clearer than others, due to how well the geography and trees match. I wonder if this could be used as a measure of how well geography "explains" the tree. For example, simple vicariance or serial dispersal would have few cross-overs, a history of dispersal (or an old pattern with extinction, or if geography has changed) might be messier. Perhaps there is a metric that could be developed for this. It strikes me as similar in spirit to trees for tandem duplications -- there's a nice spatial (albeit it linear) order in a tree if the sequences are tandem duplications.

If the trees had dated nodes (i.e., were "chronograms"), presumably this could be used to compute node heights, so you'd be able to have chronograms. Sort of a reverse onion, the layers getting older as you go out. People could then see whether biogeographic patterns were of a similar age. This adds a spatial dimension to chronograms (see an earlier post on the analogy between genome browsers and chronograms).

As an aside, and because I was once a panbiogeography enthusiast, why haven't panbiogeographers leap on Google Earth as a tool to display "tracks"? If ever there was an opportunity to drag that movement out of the doldrums, this is it.

Open Access taxonomy

Pyramica boltoni
Originally uploaded by Roderic Page.

Fussing around with ants, I stumbled across this paper (doi:10.1653/0015-4040(2006)89[1:PBANSO]2.0.CO;2) (if the DOI doesn't work, try this link), which describes a new species, Pyramica boltoni. This paper is Open Access, so the question arises, how do I get it into a triple store? I could add the metadata about the paper (it would be nice to do this automatically via Connotea and the DOI, but some BioOne DOIs aren't "live" yet), but what about things like the pictures?
For fun, I grabbed Fig. 1, uploaded it into iPhoto, then exported it to Flickr using the FlickrExport plugin.
Flickr has an API, hence the image (and the associated tags) could be retrieved automatically. Hence, anybody with Connotea and Flickr accounts could contribute to a triple store.

Towards the ToL database - some visions

So, when I started this blog I promised to write something about phyloinformatics, and the goal of a phylogenetic database. I've been playing around with various ideas, some of which have made it online, but most remain buried on various hard drives until they get written up to the state they are useable.

There are also numerous distractions, and detours along the way, such as MyPHPBib, Taxonomic Search Engine, and LSIDs, oh and iSpecies, which got me into trouble with Google, then there is a certain journal, and a certain person (but let's not go there...).

My point (and I do have one), is that maybe it's time to rethink some cherished ideas. Basically, my original goal of creating a phylogenetic database involved massive annotation, disambiguation of taxonomic names, and linking to global identifiers for taxonomic names, sequences, images, and publications. This is the project outlined at the start of this blog.

I still believe this would be worthwhile, and I've a lot of the work done for TreeBASE (e.g., mapping TreeBASE names to external databases, BLASTing sequences in ttreeBASE to get accession numbers, etc.). This is a lot of work, and I wonder about scalability and involvement. In other words, can it cope with the amount of data and trees we generate, and how do we get people to contribute. So, here are a few different (not necessarily exclusive approaches).

Curation
Use TreeBASE as a seed and continue to grow that database, adding extensive annotations and cross links. Time consuming, but potentially very powerful, especially is data is dumped into a triple store and cool ways to query it are developed.

Googolise everything
Use Google to crawl for NEXUS files (e.g., "#nexus" "begin data" format dna), extract them and put them into a database. Use string matching and BLAST to figure out what the files are about.

Phylogeny news
Monitor NCBI and journal RSS feeds, when new sequences or papers appear, extract popsets, use or build alignments, compute trees quickly and wack into a database. Interface is something like Postgenomic (maybe using the same clustering algorithms to link related stories), or even cooler, newsmap


Connotea style

Inspired by projects like Connotea, perhaps the trick is to mobilise the community by lowering the barrier to entry. Instead of aiming for a carefully curated database, what if people could upload the basics (some sort of identifier for the paper, such as a DOI or a PubMed id, and one or more trees in Newick format). I think this is what Wayne Maddison was suggesting when we chatted at the CIPRES (see my earlier post about that meeting) -- if Wayne didn't suggest this, then my apologies. The idea would be that people could upload the bare minimum, but be able to annotate, comment, link, etc. Behind the scenes we have scripts to look up whatever identifiers we have and extract relevant metadata.

Taxonomic Search Engine back online

My Taxonomic Search Engine is back online (mostly). This tool to search multiple databases for scientific names was another casualty of hacking. Having been developed under PHP 4, it needed some work to play nice with PHP 5. The changes were minor, and mainly concerned changes in code involving XPath and XSLT. I've commited these changes to SourceForge. I've not got the Species 2000 service back up (this needs local data to be restored), and the LSIDs are broken due to problems with IBM's LSID perl stack on my Fedora Core 4 machine (sigh).

AntBase and Web 2.0 business value

Dion Hinchcliffe has a piece entitled Creating real business value with Web 2.0 which lists AntBase.org (I think he actually means AntWeb) as an example of a non-commercial Web 2.0 service that demonstrates "scalable marshalling of underutilized data resources," and shows:

...how a scientific community turned massive taxonomy resources otherwise mouldering away in basements as lost specimens into a thriving online database of information that can be shared by all. Understanding the success and importance of both of these points to intriguing and largely unexploited possibilities that I predict will become more common and widespread in the near future.

The article comes with this graphic:



See also Dion's Thinking in Web 2.0: Sixteen ways (via Danny Ayers).

CrossRef's OpenURL resolver

CrossRef's OpenURL resolver can be used to find DOIs for papers, or give a DOI it can extract metadata. For example, consider Brian Fisher's article on silk production in Melissotarsus emeryi. This is in FORMIS, and the record is displayed here in RIS format:

TY - JOUR
AU - Fisher, B.L/
AU - Robertson, H.G.
PY - 1999
TI - Silk production by adult workers of the ant Melissotarsus emeryi (Hymenoptera, Formicidae) in South African fynbos
SP - 78-83
JF - Insect. Soc.
JO - Insectes Sociaux
VL - 46
N1 - Part of FORMIS 00; from PSW,
KW - ant; Formicidae; Melissotarsus emeryi; Africa; South Africa; scientific; nest; tending Homoptera; silk; production; adult; worker; gland; cuticular depressions; hypostoma; silk brushes; nest construction; defense; diaspidid symbiont;
ID - 6573
ER -

We can find the DOI for this article using the following query:

http://www.crossref.org/openurl?&noredirect=true&aulast=Fisher%20LRM&title=Insectes%20Sociaux&volume=46&spage=78&date=1999, which yields the following XML:

<?xml version = "1.0" encoding = "UTF-8"?>
<crossref_result version="2.0" xmlns="http://www.crossref.org/qrschema/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.crossref.org/qrschema/2.0 http://www.crossref.org/qrschema/crossref_query_output2.0.xsd">
<query_result>
<head>
<email_address>ckoscher@crossref.org</email_address>
<doi_batch_id>w001</doi_batch_id>
</head>
<body>
<query key="555-555" status="resolved">
<doi type="journal_article">10.1007/s000400050116</doi>
<issn type="print">00201812</issn>
<issn type="electronic">14209098</issn>
<journal_title match="exact">Insectes Sociaux</journal_title>
<author match="fuzzy">Fisher</author>
<volume match="exact">46</volume>
<issue>1</issue>
<first_page match="exact">78</first_page>
<year match="exact">1999</year>
<publication_type>full_text</publication_type>
<article_title>Silk production by adult workers of the ant Melissotarsus emeryi (Hymenoptera, Formicidae) in South African fynbos</article_title>
</query>
</body>
</query_result>
</crossref_result>

The DOI for this article is 10.1007/s000400050116. This service could be a simple way to get DOIs for recent papers in FORMIS, enabling us to get GUIDs for the articles, as well as providing a link to the article itself.

Updating ants

A triple store for ants is all very well, but it contains just the information available when the triple store was created. What about updating it? What about doing this automatically? Here are some ideas:

Connotea
Connotea provides semantically rich RSS feeds. We could subscribe to a feed using a tag (such as Formicidae), and extract recent posts. Could use HTTP conditional GET, or parse the Connotea feed and use XPath to extract references more recent than a given date. Connotea makes extensive use of RDF in their RSS feeds, so it's easy to dump this into the triple store.
uBio
uBio's taxononmically intelligent RSS feed reader could be used to monitor publications on ants (e.g., Formicidae). uBio uses RSS 2.0, which doesn't include RDF (see Wikipedia entry for RSS). One option would be to parse the RSS and see what we can extract from the links (e.g., if they contain DOIs, are Ingenta feeds, etc.). If there are DOIs we could use CrossRef's OpenURL lookup. Or we could use the Connotea Web API. We'd upload the URLs, and get Connotea to see what it can do with them, then we make use of their RSS feed. This also makes the information available to everybody for tagging.

GenBank
We could also track new sequences in GenBank (to do).

stamen design | big ideas worth pursuing

stamen (which brought us Mappr) has a nice discussion of data visualisation.

Currently playing in iTunes: Summertime by George Benson

Ants, RDF, and triple stores

Background
In order to explore the promise of RDF and triple stores we need some large, interesting data sets. Ants are cool, there is a lot of data available online (e.g., AntWeb at the California Academy of Sciences, Antbase at the American Museum of Natural History, New York, and the Hymenoptera Name Server at Ohio State University, Chris Schmidt's ponerine.org, and Ant News), and they get good press (for example, the "Google ant").

Specimens


Firstly, we start with a Google Earth file for ants, obtained from AntWeb on Monday April 24th, 2006. AntWeb gives the link as http://www.antweb.org/antweb.kmz, which is a compressed KML file. However, this file merely gives the location for the actual data file, http://www.antweb.org/doc.kmz. Grab that file, expand it and you get 27 Mb of XML listing 50,550 ant specimens and 1,511 associated images.

We use the Google Earth file because it gives us a dump of AntWeb, and does it in a reasonably easy to handle format (XML). I wrote a C++ program to parse the KML file and dump the information to RDF. One limitation is that my program dies on the recent KML files because they have very long lines. Doing a search on <Placemark> and replacing it with \r<Placemark> in TextWrangler fixed the problem.

In order to keep things as simple and as generic as possible, I use Dublin Core metadata terms wherever possible, and the basic geo (WGS84 lat/long) vocabulary for geographical coordinates. The URI for the specimen is the URL (no LSIDs just yet).

In addition to the RDF, I generate two text dumps for further processing.

Images
As noted at iSpecies, we can automate the extraction of metadata from images using Exif tags.There is a vocabulary for describing Exif data in RDF, which I've adopted. However, I don't use all the tags, nor do I use IFD, which frankly I don't understand.

So, the basic idea is to have a Perl script that:

1. Takes a list of AntWeb images (more preciesly, the URLs for the images)


2. Fetches each image in turn using LWP and writes them to a temporary folder


3. Uses Image::EXIF to extract Exif tags


4. Generate RDF

Some AntWeb specific things include linking the image to the specimen, and linking to a Creative Commons license.
Here is an example:

<rdf:Description rdf:about="http://www.antweb.org/images/casent0005842/casent0005842_p_1_low.jpg" >
<dc:subject rdf:resource="http://www.antweb.org/specimen.do?name=casent0005842" />
<dc:type>image</dc:type>
<dc:publisher rdf:resource="http://www.antweb.org/"/>
<dc:format>image/jpeg</dc:format>
<exif:resolutionUnit>inches</exif:resolutionUnit>
<exif:yResolution>337.75</exif:yResolution>
<exif:imageHeight>64</exif:imageHeight>
<exif:imageWidth>112</exif:imageWidth>
<exif:xResolution>337.75</exif:xResolution>
</rdf:Description>
</rdf:RDF>

This RDF is generated from the image to the right. What is interesting about the Exif metadata is that it isn't generated from the AntWeb database itself, but from the images. Hence, unlike the Goggle Earth file, we are adding value rather than simply reformatting an existing resource.

Of course, there are some gotchas. Some images look like this ("image not available"), and the Exif tag Copyright has a stray null character (\0x00) appended at the end, which breaks the RDF. Fixed this by Zap gremlins in TextWrangler.

Names
There is no single authorative list of scientific names. I'm going to use the biggest (and best), uBio, specifically their findIT SOAP service. It might make more sense to use the Hymenoptera Name Server, but uBio serves RDF, and gets most off the ant names anyway as the Hymenoptera Name Server feeds names into ITIS, which in turn end up in uBio. The result of this mapping is a <dc:subject> tag for each specimen that links using rdf:resource to a uBio LSID. When we make the mapping, we write the uBio namebank ids to a separate file, which we then process to get the RDF for each name.
The script reads a list of specimens and taxon names, calls uBio's findIT SOAP service, and if it gets a direct match, writes some RDF linking the specimen URI to the uBio LSID. It also stores the uBio id in memory, and dumps these into a file for processing in the next step.

uBio metadata

Having mapped ant names to uBio, we can then go to uBio and use their LSID authority to retrieve metadata for each name in, you guessed it, RDF. We could resolve LSIDs, but for speed I'll "cheat" and append the uBio namebank ID to http://names.ubio.org/authority/metadata.php?lsid=.
So, armed with a Perl script we read the list of uBio ids, fetch the metadata for each one and dump it into directory. I then run another Perl script that scans a directory for ".rdf" files and puts them in the triple store.

NCBI

Sequences
I retrieved all ant sequences from GenBank by searching the taxonomy browser for Formicidae, downloading all the sequence gis, then running a Perl script that retrieved the XML record for each sequence and populated a MySQL database. I then found all sequences that include a specimen voucher field with CASENT%:

SELECT DISTINCT dbxref.id FROM
specimen INNER JOIN source USING (source_id)
INNER JOIN sequence_dbxref  ON source.seq_id = sequence_dbxref.sequence_id
INNER JOIN dbxref USING (dbxref_id)
WHERE (code LIKE "CASENT%") AND (dbxref.namespace = "GI")

Next, we fetch these records from NCBI. This seems redundant as we have the information already in a local MySQL database, but I want to use a simple script that takes a GI and outputs RDF so that anybody can do this.

Names
In much the same way, I grabbed the TaxIds for ants with sequences, and grabbed RDF for each name.

PubMed
For PubMed records I wrote a simple Perl script that, given a list of PubMed identifiers, retrieves the XML record from NCBI and converts it to RDF using a XSLT style sheet. The script also gets the identifiers for any sequence linked to that PubMed record using elinks, and uses the <dcterms:references> tag to model the relationship. For the ant project I only use PubMed ids for papers that include sequences that have CASENT specimens:

SELECT DISTINCT dbxref.id FROM
specimen INNER JOIN source USING (source_id)
INNER JOIN sequence_dbxref  ON source.seq_id = sequence_dbxref.sequence_id
INNER JOIN dbxref USING (dbxref_id)
WHERE (code LIKE "CASENT%") AND (dbxref.namespace = "PUBMED")

Turns out there are only three such papers:

16601190

16214741

15336679


FORMIS
We could add bibliographic data from FORMIS, which can be searched online here, and downloaded as EndNote files. This would be "fun" to convert to RDF.

PubMed Central
This search finds all papers on Formicidae in PubMed Central, which we could use as an easy source of XML data, in some cases with full text and citation links.

Triple store
The beauty of a triple store is that we can import all these RDF documents into a single store and query them. It doesn't matter that we have information about images in one file, information about specimens in another, and information about names in yet another file. If we use URIs consistently, it all comes together. This is data integration made easy.

Query

This RDQL query finds all images for Melissotarsus insularis

SELECT ?image WHERE 
(?taxon, <dc:subject>, "Melissotarsus insularis") 
(?specimen, <dc:subject>, ?taxon) 
(?image, <dc:subject>, ?specimen)
(?image, <dc:type>,"image")
USING dc FOR <http://purl.org/dc/elements/1.1/>

Which returns two images: .

OK, now for something a little more fun. The Smith et al. barcoding paper that surveyed ants in Madagascar has PubMed id 16214741 (this paper also has the identifier doi:10.1098/rstb.2005.1714). Given this id (recast as a LSID urn:lsid:ncbi.nlm.nih.gov.lsid.zoology.gla.ac.uk:pubmed:16214741), we can find the geographic localities the authors sampled from using this query:

SELECT  ?lat, ?long WHERE 
(?nuc, <dcterms:isReferencedBy>, <urn:lsid:ncbi.nlm.nih.gov.lsid.zoology.gla.ac.uk:pubmed:16214741>)
(?nuc, <dc:source>, ?specimen)
(?specimen, <geo:lat>, ?lat)
(?specimen, <geo:long>, ?long)
USING dc FOR <http://purl.org/dc/elements/1.1/>
dcterms FOR <http://purl.org/dc/terms/>
geo FOR <http://www.w3.org/2003/01/geo/wgs84_pos#>

which gives four localities:

?lat    ?long
"-13.263333"    "49.603333"
"-13.464444"    "48.551666"
"-13.211666"    "49.556667"
"-14.4366665"   "49.775"

We can also search our triple store using other identifiers, such as DOIs:

SELECT  ?lat, ?long WHERE 
(?pubmed, <dc:identifier>, <doi:10.1098/rstb.2005.1714>)
(?nuc, <dcterms:isReferencedBy>, ?pubmed)
(?specimen, <geo:lat>, ?lat)
(?specimen, <geo:long>, ?long)
USING dc FOR <http://purl.org/dc/elements/1.1/>
dcterms FOR <http://purl.org/dc/terms/>
geo FOR <http://www.w3.org/2003/01/geo/wgs84_pos#>

is the same query as above, but uses the DOI for the barcoding paper.

New inferences

One thing I noticed early on is that there are specimens that have been barcoded and which are labelled in GenBank as unidentified (i.e., they have names like "Melissotarsus sp. BLF m1"), but the same specimen has a proper name in AntWeb (e.g., casent0107665-d01 is Melissotarsus insularis). Assuming the identification is correct (a big if), we can then use this information to add value to GenBank. For example, a search of GenBank for sequences for Melissotarsus insularis find nothing, but it does have sequences for this taxon, albeit under the name "Melissotarsus sp. BLF m1".

This query searches the triple store for specimens that are named differently in AntWeb and GenBank. Often both names are not proper names, but represent different ways of saying "we don't know what this is". But in some cases, the specimen does have a proper name attached to it:

SELECT ?specimen, ?ident, ?name WHERE 
(?specimen, <dc:type>, "specimen")
(?specimen, <dc:subject>, ?ident) 
(?nuc, <dc:source>, ?specimen)
(?nuc, <dc:subject>, ?taxid)
(?taxid, <dc:type>, "Scientific Name")
(?taxid, <dc:title>, ?name)
AND ?ident ne ?name
USING dc FOR <http://purl.org/dc/elements/1.1/>

Currently playing in iTunes: One by Mary J Blige & Bono. Currently playing in iTunes: Crazy (Single Version) by Gnarls Barkley

Nascent: Open Text Mining Interface

From Nature's blog on web technology and science comes this post on Open Text Mining Interface (OTMI):

Every now and then a scientist contacts Nature asking for a machine-readable copy if our content (i.e., the XML) to use in text-mining research. We're usually happy to oblige, but there has to be a better way for everyone concerned, not least the poor researcher, who might have to contact any number of publishers and deal with many different content formats to conduct their work. Much better, surely, to have a common format in which all publishers can issue their content for text-mining and indexing purposes.

and further

The example of RSS shows how powerful a relatively simple common standard can be when it comes to aggregating content from multiple sources (even when it's messed up as badly as RSS ;). So maybe an approach like OTMI (or a better one dreamt up by someone else) can help those who want to index and text-mine scientific and other content. Like RSS, I think publishers might also come to see this as a kind of advert for their content because it should help interested readers to discover it. And on the basis that a something is always better than nothing, it also doesn't force publishers to give away the human-readable form of their content — they can limit themselves to snippets or even just word vectors if they want to.

Currently playing in iTunes: By the Time I Get to Phoenix by Glen Campbell

Normal service...

I'm slowly restoring the services offered by Darwin, after it was hacked. So far, these services are back online:

• TreeView X


• iSpecies
  


• Joseph Hughes' web site


• Vince Smith's web site

Ambient Findability

Ambient Findability by Peter Morville is a wonderful read, full of snippets of inspiration. In many ways, like ambient music alluded to at the end of the book, it is less about specifics and more about a way of thinking, and about the possibilities once things become findable.

Darwin hacked

One of my lab's web servers was hacked last week. This machine hosts a lot of projects, such as the Glasgow Name Server, the Taxonomic Search Engine, iSpecies, LouseBase, and TreeView X. Sadly, it was not completely backed up, although most of the key stuff is replicated elsewhere (including source code in CVS on another machine, or in SourceForge, copies of databases on other machines, etc.). Even if it was completely backed up, there is the hassle of rebuilding a machine. Still, since it wasn't backed up, here are some of the things I had to go through.


The kernel (Red Hat 8) had been tampered with, so the machine would no longer boot. I'm was now faced with the task of getting stuff off the machine in case reinstalling the operating system lost data. Luckily the machine (a Dell Precision 620) booted from a Knoppix CD, which gave me a GUI. So now I can browse my crippled machine, but...

... it couldn't talk to the Net because the Knoppix live CD uses DHCP to get an IP address, and my university doesn't support DHCP (argh!). However, I have an Apple Airport base station with a spare Ethernet port, and connecting the Dell to that port provided a DHCP address (yay).

Booting from a live CD has one major limitation -- I can't alter anything on the disks in the Dell. Hence, doing things like changing file permissions, or making tarballs to be able to FTP directories is out of the question. I don't have a USB key or an external USB hard drive big enough to take the gigabytes of stuff on the Dell.

What worked, after a lot of fussing was Samba. Using the smb:// protocol in Konqueror (I trick I learned from Mac OS X), I managed to connect to a Fedora Core 4 box in my lab. I could then drag and drop key files onto the FC4 machine (such as httpd.conf, hosts.allow, various CGI scripts, etc.) that were specific to the hacked machine. I also made backups of the home folders, just in case.

This left MySQL databases. Moving these proved to be a major pain, because they are not accessible by the Knoppix user. The solution turned out to be to mount the FC4 box using Samba:

1. su


2. mkdir /mnt/linnaeus


3. mount -t smbfs -o username=xxxx //linnaeus.zoology.gla.ac.uk/xxxx /mnt/linnaeus

Now we can copy all the MySQL databases on the FC4 machine.

Ah, but how to get the actual data...? Well, on my Mac OS 10.3 iBook, I have MySQL 4.0.21, which works with the MySQL files from Red Hat 8 (3.23 I think). I use CocoaSQL to create the database, then move all the .MYI and .MYD files into the appropriate folder in /Library/MySQL/data/, then set permissions to ensure that mysql can read the files (make user mysql the owner chown mysql *, and set permissions to 660).

Yes, the obvious lesson is to have everything backed up, but on a developmental machine with gigabytes of images and other data, much of it moved around frequently, and a central backup system whose client software wouldn't build on my machine, I'd sort of let this slip (doh!).

Render DOT files on the fly on Mac OS X

Webdot isn't available for Mac OS X, and as I use an iBook running Panther for all my development work (before moving to a Linux box to host the results) I wanted to have the same functionality on my iBook. This can be achieved by hacking a simplified version of webdot. This Perl script creates a virtual web browser to serve the image. I've simplified things somewhat, but it works.

The two things you need to set in the script dot.cgi are the path to your copy of the Graphviz program dot,and a directory where dot can write temporary files (I use /tmp).

You can get a copy of the script here.

To render an image of a graph on the fly you insert an img
tag with the src attribute comprising:

1. the path to the CGI script, e.g. /cgi-bin/dot.cgi


2. a '/' delimiter


3. the URL of the graph file, e.g. http://localhost/~rpage/dot/leda.7.46.gml.dot
   


4. the extension of the image format you want (e.g., png, svg, etc.) preceeded by a dot "."

As an example, here is the dot file http://localhost/~rpage/dot/leda.dot as a PNG image, using the HTML:

<img src="/cgi-bin/dot.cgi/http://localhost/~rpage/dot/leda.dot.png" />



The source file for this graph looks like this:

graph G {
   node [width=.2,height=.2,fontsize=10];
   edge [fontsize=10,len=2];
   0 [label="0"];
   1 [label="3"];
   2 [label="4"];
   3 [label="5"];
   4 [label="6"];
   5 [label="7"];
   0 -- 1 [label="13"];
   0 -- 2 [label="12"];
   0 -- 5 [label="8"];
   0 -- 4 [label="71"];
   1 -- 5 [label="84"];
   1 -- 4 [label="8"];
   2 -- 5 [label="18"];
   2 -- 4 [label="11"];
   2 -- 3 [label="51"];
   3 -- 4 [label="87"];
}

YAKAFOKON: TreeMap Clustering in SVG with PHP

Pierre Lindenbaum has a very nice PHP script for generating Treemaps in SVG .

Visualizing literature derived networks

This paper in Genome Biology is a nice example of visualising relationships derived from PubMed:

We have developed PubNet, a web-based tool that extracts several types of relationships returned by PubMed queries and maps them into networks, allowing for graphical visualization, textual navigation, and topological analysis. PubNet supports the creation of complex networks derived from the contents of individual citations, such as genes, proteins, Protein Data Bank (PDB) IDs, Medical Subject Headings (MeSH) terms, and authors. This feature allows one to, for example, examine a literature derived network of genes based on functional similarity.

I've added it to my Connotea library under the tag visualisation (note to self: American English and British English spelling is just one of the problems with "tagging"). I'd seen this paper before, but "forgot" it until browsing Connotea and stumbling across nicmila's library. Nice illustration of the power of shared tags.

(Via nicmila.)

Currently playing in iTunes: Wisemen (Album Version) by James Blunt

Quantum treemaps

One of the things that keeps bothering me is the lack of compelling ways to visualise information in phylogenetic databases. Trees themselves are, I feel, pretty awful objects to work with. They are large, and displaying them takes up a lot of screen real estate. Yet, in many ways, the more one sees of the tree the less one gains from the experience. For example, CAIDA's Walrus tool (right), used by Tim Hughes to display large trees looks fabulous, but is it useful? By which I mean, can we use it find out about stuff, or do we just spin it around and go "ohh, isn't it pretty?"

Treemaps are another tool that I've looked at, but never been terribly impressed. However, quantum treemaps, described in Ordered and Quantum Treemaps: Making Effective Use of 2D Space to Display Hierarchies, look potentially useful. To quote from the paper describing them:

The goal of the Quantum Treemap algorithm is similar to other treemap algorithms, but instead of generating rectangles of arbitrary aspect ratios, it generates rectangles with widths and heights that are integer multiples of a given elemental size. In this manner, it always generates rectangles in which a grid of elements of the same size can be layed out. Furthermore, all the grids of elements will align perfectly with rows and columns of elements running across the entire series of rectangles. It is this basic element size that cannot be made any smaller that led to the name of Quantum Treemaps

Quantum treemaps have made their way into Photomesa.

So, here's my thought. What if we used a quantum treemap to browse TreeBASE? Suppose we have a mapping between TreeBASE taxa and the NCBI taxonomy (or any other taxonomy, it doesn't really matter). If we then have some notion of what taxa each TreeBASE study is mainly about, then we could display a quantum treemap of studies rooted at any node in the NCBI taxonomy. For example, studies on mammals, grouped by order. The point here is not to see the tree, but to navigate through the studies using the tree.

Whereas treemaps usually display a nested hierarchy, my sense is that quantum treemaps are used to display the children of a node, rather than the whole tree. I think this is because the final size of a quantum treemap is unpredictable.

The mapping of TreeBASE names to NCBI tax_ids is not trivial, but I've got most of one done. Mapping studies to taxa needs a little thought. One approach is to take a tree from a study, relabel it with NCBI tax_ids, then find the least common ancestor in the NCBI taxonomy of the centroid of the tree. The idea is that this is in the core of the tree, and hence should capture what the tree is about. Finding the LCA of the root would be an obvious thing to do, but if one has a tree comprising mostly vertebrates, but rooted with a bacterium, then the root LCA is the root of life, which isn't a terribly accurate summary of the tree.

I've been playing with generating quantum treemaps, based on a C++ port of some Java code written by Ben Bederson. The next step is to try and bolt this together into a demo of how this might be used to navigate TreeBASE.

Links

• Tim Hughes' paper on Walrus: doi: 10.1186/1471-2105-5-48


• Ordered and Quantum Treemaps: Making Effective Use of 2D Space to Display Hierarchies: doi:10.1145/571647.571649


• Using Treemaps to Visualize Phylogenetic Trees: doi:10.1007/11573067_29

OpenSearch

Not a huge fan of IE, but this post on David Patten's blog nicely illustrates the ease of use of A9's OpenSearch with IE 7.

I'd previously played with OpenSearch as a quick way to integrate biodiversity sources, and put together a couple that have been registered with A9 (search for "taxonomy" and you'll find them). It's essentially adding a few tags to RSS or Atom feeds, coupled with a simple way to describe the search engine.

Perhaps it's time to play with this a little more. It would be a very simple way to open up some data.

(Via A9 Developer Blog.)

Currently playing in iTunes: Wonderwall by Oasis

Firefox Extension for Turning Built-in SVG on and off

A quick Google found this Firefox extension for turning built-in SVG on and off, posted on or maybe something uplifting.

Really useful little extension, because Firefox SVG support is actually pretty awful very good. (Just discovered that FireFox couldn't handle my original SVG, but if I put in the namespaces as attributes of the svg tag, everything worked fine. Must remember to engage brain before typing...).

The Adobe SVG plugin is much nicer (this is still true). This extension enables the user to switch between Firefox and Adobe. Pity there is nothing like this for Camino, a very nice Gecko-based browser for Mac OS X that I've gotten used to. Because it uses the same rendering engine as Firefox, Camino makes the same hash nice job of SVG (once the namespaces are included).

Currently playing in iTunes: Fire and Rain by James Taylor

Fun and games with WebDot and Fedora Core 4

Well, that was fun. I've just installed AT&T's WebDot, a Tcl CGI program for generating images of graphs on the fly using Graphviz. Although there are RPMs for Fedora Core 4 available from AT&T's site, they didn't work. Here's what I did to get this working:

• Installed Graphviz from the RPM.


• Installed WebDot from the source tarball, not the RPM.


• Failing to read the instructions (like, who does that?) I neglected to install graphviz-tcl-2.8-1.fc4.i386.rpm (Graphviz Tcl). I tried to do this but it complained about a lack of Tk


• For reasons unbeknownst to me, I didn't have Tk installed (but did have Tcl), so off to the Fedora Core FTP server to grab tk-8.4.9-3.i386.rpm.


• Now graphviz-tcl-2.8-1.fc4.i386.rpm installs happily, but still no joy.


• Running Webdot from the command line
  

  
  cd /var/www/cgi-bin
  ./webdot
  

  
  produced some meaningful error messages at last. The script couldn't find a Tcl shell because the script was looking for Tcl 8.3, and Fedora Core 4 has 8.4. Hence, I edited the first line of the script accordingly. Then it complained about not finding libtcldot.so.0.0.0. Turned out the script thought this library was in /usr/lib/graphviz/, whereas the RPM had put it in /usr/lib/graphviz/tcl (sigh). So, the first two lines now look like this:
  

  
  #!/usr/bin/tclsh8.4
  set LIBTCLDOT /usr/lib/graphviz/tcl/libtcldot.so.0.0.0
  

  
  


• OK, the webdot script now runs, but no images of graphs. One useful hint is to try and view the images WebDot generates, even if they look like missing images (i.e., there doesn't seem to be anything there). Sometimes useful error messages are displayed, in this case I saw a message about missing fonts.


• Having gone through this particular version of hell a while ago when putting WebDot on a RedHat 8 box, I remembered we need some TrueType fonts. AT&T have these on the Graphviz web site, but as a tarball with no instructions what to do with them. I still had the RPM I used on the RedHat 8 box (you can grab it here), so installed this (which puts the fonts into /usr/X11R6/lib/X11/fonts/truetype/).


• Now, it works (yay!).
  

Isn't software fun? By the way, the nice icon in this post comes from pixelglow's wonderful port of Graphviz to Mac OS X. After the fun with Fedora, it's back to my Mac...

Biodiversity Informatics Visualization

The Human-Computer Interaction Lab at the University of Maryland has produced numerous nice visualisation tools, and has a page devoted to biodiversity visualisation.

We are building information retrieval and analysis interfaces for the rapidly expanding domain of biodiversity and ecological databases. Biodiversity databases contain organism-related information such as distribution, taxonomy, natural history, and conservation data. They are as complex as molecular and medical biology resources, yet serve a broad audience as do general-use digital libraries. We began by developing an interactive tree visualization (TaxonTree) for Kingdom Animalia. We also developed a prototype allowing coupled interaction with two trees (DoubleTree). We are currently working on developing other methods of visualizing both hierarchical and non-hierarchical biodiversity information (TreePlus and EcoLens), leveraging prior research on digital libraries and on bioinformatics. This has involved exploring ontologies and biodiversity data management in collaboration with the Animal Diversity Web (ADW) and the SPIRE project, and tree-reasoning with Kevin Omland at UMBC.

Least Common Ancestor (LCA) queries

My suspicion is that most queries biologists would make concerning trees are fundamentally LCA queries (as opposed to pattern matching, for example). For instance, "find me trees where group x is monophyletic" is a LCA query. Hence, LCA algorithms of of special interest (especially if they can be incorporated in a database). One recent LCA algorithm due to Bender and Farach-Colton (PDF here) has an implementation available in SourceForge, courtesy of Muhammad Ahsan Yusuf and Zack Ramjan.

There is a recent paper on this work (and the problem of LCA's in directed acyclic graphs) by Bender et al. in Journal of Algorithms(doi:10.1016/j.jalgor.2005.08.001).

All My Eye

All My Eye is a new blog by staff at Ingenta involved in RDF and related projects. Worth keeping an eye on, especially as the Ingenta fields are metadata rich, and have been used by uBio's RSS project.

Genome browsers and chronograms

Continuing the theme of visualising phylogenies, one thing which strikes me is the parallel between genome browsers that display annotation "tracks" (such as the UCSC Genome Browser) and illustrations of "chronograms" with geological periods and accompanying data, such as sea levels, isotope levels, etc. In my haste I couldn't find an example with a sea-level track, but I know they exist. The chronogram at right comes from Steppan et al. (doi:10.1111/j.1095-8312.2003.00274.x). In both cases there is a natural co-ordinate system (genome location and time, respectively) going from left to right, and annotations that can be added using the same frame of reference.



Hence, wouldn't it be cool™ if we had a database of phylogenies that could be queried by time slices (see my earlier post on interval queries), and which would display a phylogeny together with user-selected annotation tracks (obtained, say, from external geological databases)?

Interval queries

VLDB2005 has some interesting papers. One which caught my eye is Efficiently Processing Queries on Interval-and-Value Tuples in Relational Databases[PDF] by Enderle et al. Why? Because with all the effort systematists are placing on estimating divergence times, it would be nice if a phylogeny database could recover phylogenies by time slices (as well as by taxa, geography, data type, author, etc.). Imagine being able to find phylogenies with divergence dates that spanned the K/T boundary, for example.

Document icons

Amber Frid-Jimenez: has a really nice page showing document icons that reflect the words contained in the document. Neat idea, and has a lot of potential. One obvious extension to phylogenetics would be to represent the taxonomic coverage in a similar style, so people could very quickly find studies on related sets of organisms simply by browsing.

(Via information aesthetics.)

LSID Firefox extension update

I've updated my extension to resolve LSIDs in Firefox so that it works with version 1.5.0.1 (the most recent version of Firefox). The extension is available from Mozdev. It may take a little while for the mirrors to update with the new version, so if you get a "404" when trying to download, you may need to come back later.

IBM's LSID project have their own Firefox extension LSID Launchpad for Firefox, which is a lot slicker than mine. It's in beta, but well worth a look.

In case you're wondering, these extensions enable you to browse LSIDs as if they were URLs. For example, with the extension installed, a LSID such as urn:lsid:ubio.org.lsid.zoology.gla.ac.uk:namebankID:10386 becomes clickable, and you can see the metadata associated with the LSID. For the technically minded, they add support for the lsidres protocol to Firefox.

Rob McCool on Rethinking the Semantic Web

Having read Rob McCool's articles on Rethinking the Semantic Web (brought to my attention by Bob McMorris' comment on my earlier post on globally unique identifiers), I think he makes very interesting points, but they are not all relevant to whether biodiversity informatics adopts RDF.

In terms of whether the dream of the Semantic Web will happen, I suspect he is right - technologies such as tags and microformats will be a lot easier to adopt, and will make more effective use of existing tools. I'm not writing the Semantic Web off, but McCool's point about keeping things very simple is, I think, on the money.

Much of the work on RDF and the Semantic Web has been done in academia, and most examples concern things such as relationships between people and projects (typically computer science projects in, you guessed it, the Semantic Web). Within a small academic community there is often a small problem scope, consistent vocabulary (or at least, it is tractable to develop either a vocabulary or a mapping between vocabularies), obvious identifiers, experience with ontologies, and a limited set of problems. My sense is that biodiversity informatics fits this model. If the goal is to integrate databases of integrate taxonomic names, specimens, images, character data, DNA sequences, and publications, and make inferences based on this aggregation of information, then I feel the use of Semantic Web techniques will be quite tractable, indeed productive.

In the same way, much of the scepticism about whether ontologies are actually be useful in the real world (see Clay Shirky's brilliant Ontology is Overrated -- Categories, Links, and Tags, or listen to a MP3) is probably well founded. Again, I think the issue is one of scope. Biologists are used to ontologies, after all what is taxonomic classification but a large ontology with well developed rules for its construction and maintenance?

That said, there are areas in our field where insistence on RDF, controlled vocabularies, and ontologies will probably be counterproductive. Ontologies for morphological characters will, I suspect, prove hard to sell. Even though we have a history of shared terminology (think of papers establishing consistent numbering schemes for setae on insect heads), these shared vocabularies tend to have limited applicability unless they are very general (matching setae on the head of a fly and a louse is tricky), and if they are general (e.g., "legs") they are very low level. There is also the thorny issue that many aspects of morphology are not homologous in evolutionary terms (in what sense are the wings of a fly and a bird both "wings"?). Leaving aside the conceptual issues, this is one area where I think people will balk if it becomes a pain to use ontologies. It's hard enough getting people to use scientific names (never mind remembering that species names such as Homo sapiens should be written in italics). I suspect this is one area (along with scientific literature) where tagging will be a compelling alternative. For an example of the power of tagging literature see Connotea.


McCool's articles are available here:

• doi:10.1109/MIC.2005.133 [PDF]


• doi:10.1109/MIC.2005.18 [PDF]

Globally Unique Identifiers

I attended the TDWG-GUID workshop on Global Unique Indenitifers (GUIDs) held at NESCent, which has issued a report. Essentially, the aim of this work is to deploy globally unique identifiers for digital objects in biodiversity informatics, such as taxon names, specimen records, images, etc. The workshop settled on LSIDs (Life Science Identifiers), which is a sensible choice.

LSIDs have been around, and there is considerable software support from IBM (see their project on SourceForge). I've used them in my Taxonomic Search Engine. Not everybody is thrilled by LSIDs (see Anyone using LSID? on NodalPoint).

DOIs and Handles were also considered. I have flirted with handles (see my comments on the iSpecies blog). DOIs have some useful properties, especially stable infrastructure, management tools, and immediate utility by the publishing industry, although they are not cheap. George Garrity uses them in his NamesforLife© project(doi:10.1601/tx.0). Long term the biodiversity community might benefit from thinking seriously about this. The German Science Foundation has invested in providing free DOIs to the German scientific community (see Publication and Citation of Scientific Primary Data). There's also a certain irony in a blog posting talking about GUIDs and rejecting DOIs, when every reference to an external publication is made using, you guessed it, a DOI.

Regarding the workshop itself, at times I wanted to gnaw off parts of my body to retain sanity. As a result I was pretty obnoxious. My frustration stemmed partly from a feeling that the TDWG community seems determined to make life hard for themselves by placing obstacles in their path whenever possible. They've also a lot of investment in XML schema, which I regard as misguided (that's being polite). Anybody who thinks XML schema are the answer to our problems should read "From XML to RDF: how semantic web technologies will change the design of 'omic' standards" (doi:10.1038/nbt1139). I nearly lost it when there was discussion of adopting LSIDs but serving the metadata in XML schema. This defeats the whole point of LSIDs. By serving RDF, we can do inference, in particular we can easily aggregate RDF into triple stores. Populating a database becomes as easy as resolving the LSID and sucking down the metadata. Consequently, data integration suddenly looks a lot more tractable. Indeed, from the perspective of RDF, LSIDs are just another Uniform Resource Identifier (URI), albeit one which consistently resolves to RDF.

As the workshop drew to a close, I began to feel that one reason people just didn't "get" LSIDs and RDF was that there were no really cool examples of what can be done with the technology. If you just look at RDF serialised as XML, then it's not obvious what the big deal is. So we serve a different form of XML, what's the big deal? This is a little like my first impression of XML -- it just seemed like a more fussy version of HTML, so what was all the hype about? Once you see the power of the tools associated with XML (such as the parsers, XSLT and XPath), then you see the point. It can make exchanging and processing data a lot easier, and style XSLT sheets are just way kewl. The difference between XML and RDF is of this order. So, what we need are some cool applications combining LSIDs, metadata, and triple stores to show people just why this is so much more powerful than the XML schema that have obsessed the TDWG community for so long.

Search result comparison

Yet more cool stuff from information aesthetics, this time a comparison of search results from Google and Yahoo. Given my own talk on Google versus Yahoo and the death of taxonomy?, this page certainly caught my eye.

Treemaps

I came across this version of treemaps a little while ago, but this post on information aesthetics reminds me to add this to the list of cool things that are worth thinking about when considering how to visualise the Tree of Life.

Isn't it gorgeous?

TreeBASE talk at CIPRES

On Saturday I gave a short invited talk at the CIPRES all hands meeting in Austin. Not sure if I'll be invited back after this, but I think it was worth saying a few things that somebody, somewhere should be saying. You can grab the PowerPoint slides here. To give you a sense of the talk (and the style in which it was intended), here's the abstract:

The current TreeBASE is a black hole -- data disappears in and is difficult to extract again. Furthermore, no use is made of the wealth of information that could be linked to data in TreeBASE. The only external links TreeBASE contains are author email addresses. Yet, given a GenBank sequence or a paper title one can go to the Internet and readily extract information on genes, specimen localities, PubMed records, citation links, images, taxonomic authorities, etc. The search interface is limited, and locks users into primitive and often fruitless searches. TreeBASE is a walled garden in a time when the world is discovering data integration, federated searches, and "mashups." Designing new, improved (read bigger) relational database schema does nothing to address these issues. If the community wants a useful tool that tells us what we know (and what we don't know) about the tree of life, and enables the kind of integrated research that we systematists so often say is only possible with a phylogenetic underpinning, then I suggest we need something rather different. This talk will sketch some problems with TreeBASE, discuss some ideas relating to globally unique identifers, metadata, inference, and the Semantic web, and will end with the author running from the room hotly pursued by Bill Piel.

Apart from biological gripes, I was also a little surprised that some of the stuff from the early days of "phyloinformatics" wasn't being picked up on, especially the idea of a phylogenetic query language (e.g., Jamil et al., BIBE 2001 doi:10.1109/BIBE.2001.974405) (and, no, I don't think a CORBA wrapper constitutes a phylogenetic query language).

Finding good phylogenies using citation relationships

How does a person who is not expert in a group of organisms find a good phylogeny to use for their work? Think of somebody interested in animal behaviour who needs a tree for their birds of fish. Ignoring the answers "ask a systematist" or (even worse) "become a systematist and build the tree themselves", how do we answer this query?

Google ranks sites using link structure (in essense, pages with lots of links that are themselves pointed to by lots of sites score highly). Could we use the same idea for scientific papers? The answer is of course we could, but whether it would generate useful results is an open question. I've been toying with Jon Kleinberg's ideas in Authoritative sources in a hyperlinked environment. Kleinberg identifies "authorities" and "hubs", which are roughly analogous to highly cited papers and review articles, respectively.

So, the idea is this. For a collection of papers (such as those in TreeBASE, or those being assembled for birds by Katie Davis in my lab), use Google Scholar to extract citation information, build a graph and compute authorities and hubs using Kleinberg's algorithms. Based on a little play with TreeBASE (which I need to finish and write up, sigh), papers with high hub scores tend to be recent reviews, which may be good candidates for a place to start.

We could even test this. In the case of Katie's work on bird supertrees, we could compute a measure of fit between input trees and the supertree, and compare that with the score assigned to the paper containing the source tree. If my idea has value, papers that have "good" input trees will also have scores based on citation structure (e.g., hubiness, or some other measure).

Wouldn't it be cool if ... GenBank watch

From the "wouldn't it be cool if" department, one thing I've often thought would be very handy would be a web site that listed sequences in GenBank that were known (or suspected) to be problematic (especially sequences thought to have been misidentified). What I'd like to see is a site called something like "GenBank Watch" (a ripe off of Search Engine Watch) where this information is recorded.

There has some commentary on this issue in the literature (Rytas Vilgalys's article in New Phytologist doi:10.1046/j.1469-8137.2003.00894.x, and James Harris' article in Trends in Ecology and Evolution doi:10.1016/S0169-5347(03)00150-2).

Some workers do make available lists of dubious sequences, such as list of rejected sequences provided by the mor project. My concern is that a lot of this sort of information is buried in papers (e.g. this one suggesting AF203470 has been misidentified), or even worse, comes to light when manuscripts are reviewed, the authors remove the sequences from their data set, but the important information (that the sequence is bogus) isn't mentioned in the paper.

Wouldn't it be great if there was a web site were one could go and search for a sequence by accession number to if somebody had flagged that sequences as problematic? Ideally the site would enable users to comment on a sequence (for example, it whether the sequence is bogus might be contentious), and it would also need a web service interface so the search could be automated.

One for the "if I only had time" list.