My British Library Talk Science presentation is now available online as a podcast.
Saturday, March 21, 2009
London Calling Video
Here's the video of my talk at the NHM (courtesy of Vince Smith).
Going Digital - by Rod Page from Vince Smith on Vimeo.
Going Digital - by Rod Page from Vince Smith on Vimeo.
Wednesday, March 18, 2009
London Calling
Busy day yesterday, giving two talks, one at The Natural History Museum, one at the British Library. Slides for the NHM talk are below. Karen James pointed out the irony that a talk where I gave the NHM a hard time for being backward about embracing digitisation can't be viewed on most PCs at the NHM because SlideShare requires a recent version of Flash (which users can't install without IT's permission), and the downloaded presentation won't open because the NHM uses an older version of MS Office. So much for my attempts to share the slides. There will also be a video available at some point.
The second presentation was at the British Libraries "Talk Science" series, for some background see the forum on Nature Network. There will be a podcast available of this presentation. In her introduction to my talk, Sarah Kemmitt quoted from a recent paper by Antonio G. Valdecasas ([JACC]1175-5326:1820@41 where he described Vagabundia sci:
My own feeling is that SCI probably does a reasonable job of ranking the impact of taxonomic publication, the real task is to broaden our notion of what gets cited.
Going Digital
View more presentations from rdmpage.
The second presentation was at the British Libraries "Talk Science" series, for some background see the forum on Nature Network. There will be a podcast available of this presentation. In her introduction to my talk, Sarah Kemmitt quoted from a recent paper by Antonio G. Valdecasas ([JACC]1175-5326:1820@41 where he described Vagabundia sci:
Vagabundia comes from the Spanish word 'vagabundo' that means 'wanderer'. It is a feminine substantive; sci refers to Science Citation Index. We pointed out some time ago (Valdecasas et al. 2000) that the popularity of the Science Citation Index (SCI) as a measure of ‘good’ science has been damaging to basic taxonomic work. Despite statements to the contrary that SCI is not adequate to evaluate taxonomic production (Krell 2000), it is used routinely to evaluate taxonomists and prioritize research grant proposals. As with everything in life, SCI had a beginning and will have an end. Before it becomes history, I dedicate this species to this sociological tool that has done more harm than good to taxonomic work and the basic study of biodiversity. Young biologists avoid the 'taxonomic trap' or becoming taxonomic specialists (Agnarsson & Kuntner 2007) due to the low citation rate of strictly discovery-oriented and interpretative taxonomic publications. Lack of recognition of the value of these publications, makes it difficult for authors to obtain grants or stable professional positions.
My own feeling is that SCI probably does a reasonable job of ranking the impact of taxonomic publication, the real task is to broaden our notion of what gets cited.
Friday, March 06, 2009
Phylogenies in a wiki
I'm slowing trying to get phylogenies into the wiki that I'm playing with. Here's an early example, the TreeBASE tree T6002, from the study A Phylogenomic Study of Birds Reveals Their Evolutionary History. The tree is displayed using my tvwidget. Below are listed the OTUs in the tree in a crude table. The idea is that this table will contain a mapping between OTU labels and taxa. For example, one OTU is labelled "Diomedea nigripes". This links to the page for Diomedea nigripes, which provides some basic information on this name, including the statement that ITIS regards the correct name for this bird to be Phoebastria nigripes (Audubon, 1839). The table on the page showing the tree displays this information as well.
This is all terribly incomplete and crude, but it gives a sense of where this is going. The plan is to import in bulk the trees and the mappings (from, say, TBMap), as well as the names themselves, and associated literature (including the TreeBASE studies) and then the trees will be embedded in richer data about the taxa.
This is all terribly incomplete and crude, but it gives a sense of where this is going. The plan is to import in bulk the trees and the mappings (from, say, TBMap), as well as the names themselves, and associated literature (including the TreeBASE studies) and then the trees will be embedded in richer data about the taxa.
Photosynth for trees - supertrees revisited
It's Friday, so time for some random, half-baked ideas. Imagine that we have a database of evolutionary trees, and these overlap for a set of taxa that we are interested in. How do we summarise these trees? One approach is to make a supertree. It would be useful to display the subtrees that went into making this supertree, if only to give an idea of how much they agree with the supertree. How to do this?
One idea I've been toying with is inspired by Photosynth, from Microsoft labs (it only runs on Windows, sigh). Photosynth takes a series of pictures taken from different angles and stiches them together into a 3D model of the object being photographed:
One thing I like about Photosynth is that you can see the original pictures, so when you move around the view you get a sense of how they have contributed to the overall view. This is easier to see than explain:
Now, imagine if we did this with trees. We could create a supertree as a summary of the individual trees, then have the original trees layered on top. Perhaps we could do this in 3D, so that each individual tree is in a plane that is tilted with respect to the supertree in proportion to how much it disagrees with the supertree:
I think this could be a fun way to explore a set of trees, and it would give one the ability to quickly grasp how well the source trees agreed with the supertree. Note that I'm not (necessarily) arguing that the supertree represents the try phylogeny. Think of it as a convenient way to summarise the individual trees.
Part of what attracts me to this approach is that I think most, if not all, 3D phylogeny viewers (such as Paloverde and the Wellcome tree of life) don't make any real use of 3D, beyond the rather gimmicky (and I find ultimately confusing) ability to fly around a 2D tree. Is there a better way to exploit the possibilities of 3D?
One idea I've been toying with is inspired by Photosynth, from Microsoft labs (it only runs on Windows, sigh). Photosynth takes a series of pictures taken from different angles and stiches them together into a 3D model of the object being photographed:
One thing I like about Photosynth is that you can see the original pictures, so when you move around the view you get a sense of how they have contributed to the overall view. This is easier to see than explain:
Now, imagine if we did this with trees. We could create a supertree as a summary of the individual trees, then have the original trees layered on top. Perhaps we could do this in 3D, so that each individual tree is in a plane that is tilted with respect to the supertree in proportion to how much it disagrees with the supertree:
I think this could be a fun way to explore a set of trees, and it would give one the ability to quickly grasp how well the source trees agreed with the supertree. Note that I'm not (necessarily) arguing that the supertree represents the try phylogeny. Think of it as a convenient way to summarise the individual trees.
Part of what attracts me to this approach is that I think most, if not all, 3D phylogeny viewers (such as Paloverde and the Wellcome tree of life) don't make any real use of 3D, beyond the rather gimmicky (and I find ultimately confusing) ability to fly around a 2D tree. Is there a better way to exploit the possibilities of 3D?
Sunday, March 01, 2009
What would people want/expect from taxon searching on TreeBASE?
Rutger Vos asked on Twitter "What would people want/expect from taxon searching on TreeBASE?". This is a good question, and one which motivated the work I did on TBMap (see doi:10.1186/1471-2105-8-158), which developed a mapping between TreeBASE taxa and other databases. In that paper I published a table showing the effectiveness of string and hierarchical queries of TreeBASE. A string query returned just the studies that included the query term, a hierarchical query returned all studies that contained taxa that descend from a given node in the NCBI taxonomy.
A hierarchical query can be visualised as a range query. For example, the diagram below shows a classification where the descendants of Node A correspond to the range 1-8 (this is a simplification of visitation numbers, see my earlier post, and also Chen et al. doi:10.1186/1471-2148-8-90). The three trees can be represented as the ranges 3-8, 6-7, and 2-4, respectively. To find trees for the taxon corresponding to Node A we look for trees whose range intersects 1-8, to find trees corresponding to Node B we look for trees whose range intersects 1-5.
This approach retrieves a list of all trees that include a given taxon, but potentially this list could be very large (for example, a query for all plant trees could return 1000's of trees). So the question becomes how to order these trees? Some ideas:
Ordering by size seems attractive, but will favour trees with more taxa over those with a greater taxonomic spread (which is favoured by ordering by the span of the tree). I used span in my challenge entry to display papers with related taxa.
What I have in mind for ordering by set inclusion is constructing a directed graph where each node represents a tree, and a pair of nodes (x, y) is connected by an edge x → y if the taxa in tree y are nested inside the taxa in tree x. We could also introduce some additional nodes corresponding to nodes in the classification. If we then topologically sort the graph we have a linear order for the trees. Given that this order could be pre-computed independently of any queries (in much the same way that PageRank is), it could make for faster queries.
It would be useful to explore these and other ordering criteria. Perhaps the best approach would be some measure which combines one or more criteria, in which case we might want to use some form of rank aggregation (see iSpecies clones, and taxonomic intelligence for some links to the relevant literature).
A hierarchical query can be visualised as a range query. For example, the diagram below shows a classification where the descendants of Node A correspond to the range 1-8 (this is a simplification of visitation numbers, see my earlier post, and also Chen et al. doi:10.1186/1471-2148-8-90). The three trees can be represented as the ranges 3-8, 6-7, and 2-4, respectively. To find trees for the taxon corresponding to Node A we look for trees whose range intersects 1-8, to find trees corresponding to Node B we look for trees whose range intersects 1-5.
This approach retrieves a list of all trees that include a given taxon, but potentially this list could be very large (for example, a query for all plant trees could return 1000's of trees). So the question becomes how to order these trees? Some ideas:
- order by the number of taxa in the tree.
- order by the size of the range ("span") of the tree.
- order by set inclusion.
Ordering by size seems attractive, but will favour trees with more taxa over those with a greater taxonomic spread (which is favoured by ordering by the span of the tree). I used span in my challenge entry to display papers with related taxa.
What I have in mind for ordering by set inclusion is constructing a directed graph where each node represents a tree, and a pair of nodes (x, y) is connected by an edge x → y if the taxa in tree y are nested inside the taxa in tree x. We could also introduce some additional nodes corresponding to nodes in the classification. If we then topologically sort the graph we have a linear order for the trees. Given that this order could be pre-computed independently of any queries (in much the same way that PageRank is), it could make for faster queries.
It would be useful to explore these and other ordering criteria. Perhaps the best approach would be some measure which combines one or more criteria, in which case we might want to use some form of rank aggregation (see iSpecies clones, and taxonomic intelligence for some links to the relevant literature).
Friday, February 27, 2009
Something's missing from taxonomic name vocabularies
In the wiki examples I've been developing I've been trying to model names using the TDWG LSID vocabularies, particularly TaxonName. Roger Hyam has obviously put a huge amount of work into developing these, and they handle just about everything I need. However, I think that there's one thing missing, namely a way to express the logical relationship between the parts of a multinomial taxonomic name.
For example, consider the fish Chromis circumaurea Pyle, Earle, and Greene, 2008, described by Rich Pyle and colleages (TED have recently posted a great video of Rich talking about discovering new species of fish). Chromis circumaurea is a species in the genus Chromis, and in the TaxonName vocabulary I can represent this relationship using the term "genusPart", which specifies the name of the genus. In a wiki page this could be a link to a page called "Chromis".
But, which "Chromis"? There are at least three:
I'd alluded to this issue in my first major foray into RDF and taxonomic names (Taxonomic names, metadata, and the Semantic Web), where I proposed using the Dublin Core term "isPartOf" to link the specific epithet to the genus part. In this case, the link would be between URIs for the names Chromis circumaurea Pyle, Earle, and Greene, 2008 and Chromis Cuvier, 1814.
It's a small point, but without some means to link components of a name we're going to struggle to sensibly answer questions such as listing all the species in a given genus (or, perhaps more correctly, all the species names that have been published in a given genus).
For example, consider the fish Chromis circumaurea Pyle, Earle, and Greene, 2008, described by Rich Pyle and colleages (TED have recently posted a great video of Rich talking about discovering new species of fish). Chromis circumaurea is a species in the genus Chromis, and in the TaxonName vocabulary I can represent this relationship using the term "genusPart", which specifies the name of the genus. In a wiki page this could be a link to a page called "Chromis".
But, which "Chromis"? There are at least three:
- Chromis Hübner 1819
- Chromis Lacepède 1802
- Chromis Cuvier, 1814
I'd alluded to this issue in my first major foray into RDF and taxonomic names (Taxonomic names, metadata, and the Semantic Web), where I proposed using the Dublin Core term "isPartOf" to link the specific epithet to the genus part. In this case, the link would be between URIs for the names Chromis circumaurea Pyle, Earle, and Greene, 2008 and Chromis Cuvier, 1814.
It's a small point, but without some means to link components of a name we're going to struggle to sensibly answer questions such as listing all the species in a given genus (or, perhaps more correctly, all the species names that have been published in a given genus).
Tuesday, February 24, 2009
Clustering taxonomic names
As part of my Quixotic attempt to construct a wiki of taxonomic names, I'm building a database of names and links. My current plan is to seed this with the NCBI taxonomy. What I want to do is flesh out the NCBI taxonomy with authorities and links to the original literature. At the moment the NCBI taxonomy is almost "nude", lacking links to the literature behind the names. As the magnificently bearded Geoffrey Bilder notes in an interview with Martin Fenner:
Given how important the NCBI taxonomy is, I think it would be a great improvement if each name could be linked to the original taxonomic publication. A first step to this is to find the taxonomic authority, the name of the author (or authors) of the name.
One potential source is uBio, which provides web services for retrieving information on names. Hence, an obvious approach is to map NCBI names to uBio names. However, if I use uBio's SOAP service typically I get multiple records for the same name. Some of these are due to homomyms (e.g., the same name used for a plant and an animal), but many are the same name with variations on the taxonomic authority. Much of this variation arises because uBio aggregates information from a wide range of databases, and each database differs in who it records the taxonomic authority.
For example, for the name "Diplura" (which I've discussed earlier) we get these names and authorities:
The nodes in the graph are the taxonomic authorities, "cleaned" by making all the text lower case, and stripping any punctuation. The edges are labelled by the length of the longest common substring shared by the nodes that edge links (I ignore substrings less than four characters long). This graph groups the variations on Diplura Koch (a spider), and Diplura Hollenberg (a brown alga, see doi:10.1111/j.1529-8817.1969.tb02617.x).
Not surprisingly, perhaps, the linkouts from the NCBI taxonomy for Diplura are a mess, with the algal genus (taxon:371965 linking to both plant and animal databases, and the insect class (taxon:29997) linking to a mix of plants and animals, not all of the animals are insects.
I'm still playing with the underlying code, but I might try and build a web service that returns name clusters (and perhaps the graph as well).
One way in which researchers assess the trustworthiness of content is by determining how it sits within the scholarly record. Does it provide evidence for its assertions in citations? Do other people cite it?
Given how important the NCBI taxonomy is, I think it would be a great improvement if each name could be linked to the original taxonomic publication. A first step to this is to find the taxonomic authority, the name of the author (or authors) of the name.
One potential source is uBio, which provides web services for retrieving information on names. Hence, an obvious approach is to map NCBI names to uBio names. However, if I use uBio's SOAP service typically I get multiple records for the same name. Some of these are due to homomyms (e.g., the same name used for a plant and an animal), but many are the same name with variations on the taxonomic authority. Much of this variation arises because uBio aggregates information from a wide range of databases, and each database differs in who it records the taxonomic authority.
For example, for the name "Diplura" (which I've discussed earlier) we get these names and authorities:
- Diplura (Greene MS.) Allman 1864
- Diplura Borner, 1904
- Diplura C. L. Koch 1850
- Diplura G. J. Hollenberg, 1969
- Diplura Hollenb.
- Diplura Jerdon 1864
- Diplura Koch 1850
- Diplura Koch 1851
- Diplura Rambur 1866
- Diplura Simon 1892
The nodes in the graph are the taxonomic authorities, "cleaned" by making all the text lower case, and stripping any punctuation. The edges are labelled by the length of the longest common substring shared by the nodes that edge links (I ignore substrings less than four characters long). This graph groups the variations on Diplura Koch (a spider), and Diplura Hollenberg (a brown alga, see doi:10.1111/j.1529-8817.1969.tb02617.x).
Not surprisingly, perhaps, the linkouts from the NCBI taxonomy for Diplura are a mess, with the algal genus (taxon:371965 linking to both plant and animal databases, and the insect class (taxon:29997) linking to a mix of plants and animals, not all of the animals are insects.
I'm still playing with the underlying code, but I might try and build a web service that returns name clusters (and perhaps the graph as well).
Friday, February 20, 2009
Talks at The Natural History Museum and British Library
Vince Smith has produced a nice flyer for my forthcoming talk at The Natural History Museum on March 17th (11-12).
It will be a busy day as I'm also talking at the British Library in the evening (6pm - 8:30pm), for which Sarah Kemmitt has produced a flyer, and set up a discussion forum on Nature Network. With all this effort going into the artwork, I'd better actually come up with something useful to say.
It will be a busy day as I'm also talking at the British Library in the evening (6pm - 8:30pm), for which Sarah Kemmitt has produced a flyer, and set up a discussion forum on Nature Network. With all this effort going into the artwork, I'd better actually come up with something useful to say.
Wednesday, February 04, 2009
Wiki demo
Reading a recent TAXACOM thread (Species Pages - purpose) my sense is that some people are arguing that "species pages" would be time consuming to create, aren't much good for taxonomists (to quote Mike Dallwitz "In brief, to make simplified and attractive information about taxa easily available to casual users?"), and nobody gets credit for making them. In short, "they're not for me, I don't get credit for making them, so why bother?"
Others (e.g., Doug Yanega) see species pages -- properly constructed -- to be a research tool. If we extend this to its logical conclusion, we could envisage these pages being the primary source of information on taxa. Indeed, new taxa could be described in this way. In short, "this is the future of taxonomic publication".
One obvious way to realise species pages sensu Doug Yanega is using a wiki, but then there are those that horrified by the prospect of just "anyone" being able to edit that content. In short, "wikis are not for serious people, the ignorant might mess up my stuff". Others have had a more positive experience.
I realise this doesn't do justice to these positions, but this make things a little concrete, I've put together a demo based on a wiki I'm constructing. The aim of this wiki is to link together taxonomic names, specimens, images, classifications, publications, phylogeny, people in one place. It's a bit like a wiki version of my Elsevier Challenge entry.
This is some way off being ready for prime time, but I thought it might be useful to show the sort of thing that can be done.
As a starting point, http://itaxon.org/wikidev/Chromis_circumaurea is a page about Chromis circumaurea, one of the fish Rich Pyle et al recently described in Zootaxa. This page contains a map and some specimen images, and an abbreviated description copied from the Zootaxa article. The images and the map are generated automaticaly by the wiki, based on the links it has to the specimens, e.g.:
http://itaxon.org/wikidev/BMNH_2007.10.31.3
http://itaxon.org/wikidev/USNM_391138_%28Fishes%29
The USNM specimen is linked to a GenBank sequence http://itaxon.org/wikidev/EU358583 . Specimens themselves are linked to Museum collections (using data from Biodiversity Collections Index ).
The name itself is linked to a publication:
http://itaxon.org/wikidev/Jacc:1175-5326:1671%403
which is also linked to the authors. The wiki supports GUIDs, so that you can link to the page about Chromis circumaurea using it's ZooBank LSID, for example: http://itaxon.org/wikidev/Urn:lsid:zoobank.org:act:8ADC4817-8F1C-4C88-8B8A-5372A84CAEC9
What I hope this crude example demonstrates is a framework where we can support all the kinds of objects we care about, and easily create links between them that can generate useful information. For example, the page for Chromis circumaurea doesn't explicitly list the images shown, they are there because of the links between Chromis circumaurea, the specimens, and the images of those specimens. The same applies to the map. What this means is that very little information needs be entered, it's mostly a matter of joining the dots.
Note that these wiki pages already have more information than either iSpecies or EOL.
This example has been assembled by hand, but much of the data required can be entered automatically (e.g., for sequences, specimens, publications, etc.), and tools such as text mining or XML markup (e.g., TaxonX) could be easily exploited. I also realise that as it stands the demo has very limited information about the organism itself, but I don't this this as intractable.
If we could build things like this (and I believe we can, with a lot less effort than might be thought), the question becomes this the kind of "species page" that would be useful?
Monday, February 02, 2009
Wiki modelling - Part 4
Another issue I'm trying to get my head around is how to deal with labels in phylogenies. These can be any number of things, such as GenBank sequences, specimen codes, taxon names, abbreviations of taxon names, laboratory codes, etc. Here's my quick attempt to model these:
This sketches various levels of indirection to go from a label in a tree to a taxon name. The may be short form of a taxon name (one redirect to name), it may contain a specimen code (redirect to specimen code, then link to name), or it may be GenBank sequence (redirect to accession number, then via source to taxon name with corresponding NCBI taxon id). There are other cases to consider, such as synonyms, but I'll try to deal with these later. At this stage I'm looking at how to make it simple to query for all phylogenies that contain a given taxon.
This sketches various levels of indirection to go from a label in a tree to a taxon name. The may be short form of a taxon name (one redirect to name), it may contain a specimen code (redirect to specimen code, then link to name), or it may be GenBank sequence (redirect to accession number, then via source to taxon name with corresponding NCBI taxon id). There are other cases to consider, such as synonyms, but I'll try to deal with these later. At this stage I'm looking at how to make it simple to query for all phylogenies that contain a given taxon.
Wiki modelling - Part 3
I rather skirted around the notion of "taxonomic concepts" in the previous post, partly because it's easy to end up with trying to have a concept for each utterance every made by a taxonomist, and that doesn't seem, er, scalable. So, I have a more limited view of a taxonomic concept, namely a name attached to some data. For example, I think the NCBI Taxonomy provides useful taxonomic concepts, in that names are explicitly linked to data, such as sequences:
Having data means we can make inferences that have some basis, other than trying to figure out what a taxonomist "meant".
However, things start to get a little messy once I try and extract more information out of NCBI GenBank. Some time ago I pointed out the potential utility of host association records in GenBank. In some (many?) cases the host taxa won't be in GenBank, so the link will be between DNA sequence and taxon name. This is, of course, a simplification. It would be nice to model things more accurately. For example, a parasite will typically be obtained from a host organism, so it might be nice if, say, we had voucher specimen codes for both parasite and host, and could model the link as one between organisms (or samples of/from organisms). However, this is unlikely to be feasible in most cases, hence we have sequences linked to names:
Having data means we can make inferences that have some basis, other than trying to figure out what a taxonomist "meant".
However, things start to get a little messy once I try and extract more information out of NCBI GenBank. Some time ago I pointed out the potential utility of host association records in GenBank. In some (many?) cases the host taxa won't be in GenBank, so the link will be between DNA sequence and taxon name. This is, of course, a simplification. It would be nice to model things more accurately. For example, a parasite will typically be obtained from a host organism, so it might be nice if, say, we had voucher specimen codes for both parasite and host, and could model the link as one between organisms (or samples of/from organisms). However, this is unlikely to be feasible in most cases, hence we have sequences linked to names:
Wiki modelling - Part 2
Modelling taxa is a bit trickier. I've sketched my ideas for distinguishing name strings and taxonomic names earlier. That's the easy stuff. What about "taxonomic concepts" and "OTUs"? As a first pass, I'm looking at linking taxon names to classifications via GUIDs. If a taxon appears in a classification then the GUID of the corresponding node in the classification is an attribute of the taxon name, and each classification GUID (representing a node in a classification) corresponds to a page in the Wiki.
I claim no originality for this, it's very close to how uBio models classification concepts:
The trick here is going to be ensuring that I can do sensible queries, such as linking a node in a classification to alternative names.
The other entity that I need to think carefully about are OTUs (Operational Taxonomic Units). By OTUs I mean the taxa that appear in phylogenetic trees. In the TbMap project I mapped TreeBASE taxa to names in external databases, but noted that TreeBASE taxa are better thought of as OTUs:
Taxa in phylogenetic trees may be single sequences, multiple sequences (from one or more specimens), or aggregates of information from multiple taxa. The challenge is to model these in the simplest way that reflects this, but also makes queries feasible. What I'm aiming for is for the user to click on a node in a phylogeny, and be taken to a page that best corresponds to the entity in the tree, but at the same time enable queries that will list all phylogenies that contain a given taxon.
I claim no originality for this, it's very close to how uBio models classification concepts:
The trick here is going to be ensuring that I can do sensible queries, such as linking a node in a classification to alternative names.
The other entity that I need to think carefully about are OTUs (Operational Taxonomic Units). By OTUs I mean the taxa that appear in phylogenetic trees. In the TbMap project I mapped TreeBASE taxa to names in external databases, but noted that TreeBASE taxa are better thought of as OTUs:
...many taxon names in TreeBASE are best though of as Operational Taxonomy Units (OTUs) rather than taxonomic names. They identify a set of observations for a particular specimen, set of specimens, or a taxon. For instance, "Eleutherodactylus crassidigitus FMNH257676 Panama" (TaxonID T51971) refers to a 1200 base pair stretch of mitochondrial DNA (AY273113) obtained from Field Museum Natural History specimen FMNH 257676, which has been identified as Eleutherodactylus crassidigitus. [see doi:10.1186/1471-2105-8-158.
Taxa in phylogenetic trees may be single sequences, multiple sequences (from one or more specimens), or aggregates of information from multiple taxa. The challenge is to model these in the simplest way that reflects this, but also makes queries feasible. What I'm aiming for is for the user to click on a node in a phylogeny, and be taken to a page that best corresponds to the entity in the tree, but at the same time enable queries that will list all phylogenies that contain a given taxon.
Wiki modelling - Part 1
Time to make some notes. I've been playing with using Sematic Mediawiki to create a database of taxonomic names, literature, specimens, sequences, and phylogenies. One challenge is to come up with simple ways to model these entities, in a way that makes both data entry simple and querying as simple as possible. Some things are straightforward. For example, a publication can be modelled like this:
OK, I've ignored the attributes. The diagram simply shows the use of MediaWiki REDIRECT to enable the use of standard publication GUIDs as Wiki page names (see earlier posts for more details, and a hack to deal with problem characters in DOIs). One benefit of GUID REDIRECTs is that I can refer to publications using GUIDs, and the wiki user will be taken to the article page without any fuss.
Likewise, we can model a journal like this:
Again, GUIDs are REDIRECT pages. This means an article page can have the ISSN of the publication it appears in as one of its attributes, and we can then use ISSNs in our queries.
People are a bit trickier, given the absence of GUIDs (or the desire to keep obvious ones, such as email addresses, private) (see doi:10.1371/journal.pcbi.1000247 for some background). I plan to have a single page for each author, and have alternative spellings link to that page:
This is one motivation for my work on equivalent author names. By finding clusters of equivalent names it would be possible to pre-populate the wiki with author names from bibliographic databases, whilst minimising the number of duplicate pages for the same author.
OK, I've ignored the attributes. The diagram simply shows the use of MediaWiki REDIRECT to enable the use of standard publication GUIDs as Wiki page names (see earlier posts for more details, and a hack to deal with problem characters in DOIs). One benefit of GUID REDIRECTs is that I can refer to publications using GUIDs, and the wiki user will be taken to the article page without any fuss.
Likewise, we can model a journal like this:
Again, GUIDs are REDIRECT pages. This means an article page can have the ISSN of the publication it appears in as one of its attributes, and we can then use ISSNs in our queries.
People are a bit trickier, given the absence of GUIDs (or the desire to keep obvious ones, such as email addresses, private) (see doi:10.1371/journal.pcbi.1000247 for some background). I plan to have a single page for each author, and have alternative spellings link to that page:
This is one motivation for my work on equivalent author names. By finding clusters of equivalent names it would be possible to pre-populate the wiki with author names from bibliographic databases, whilst minimising the number of duplicate pages for the same author.
Thoughts on the Wellcome Interactive Tree of Life
Last night BBC One aired David Attenborough's Charles Darwin and the Tree of Life, which featured a lovely "fly through" the tree of life:
In conjunction with the TV show, the Wellcome Trust has launched the Interactive Tree of Life, a Flash-based view of the tree of life. There's also a blog about the project. Here's a demo of the tree:
The tree looks very nice, and a lot of work has gone into it, but I am somewhat underwhelmed. The tree itself is tiny, and does a poor job of conveying the relative diversity of life (e.g., no plants, bacteria, few arthropods, etc.). It displays the tree on a 2D plane, and the user can move relative to that plane. I'm not convinced this is the best way to display large trees. Something modelled on Perceptive Pixel's demo might be more useful. I blogged about this last year, but the video host service has disappeared. You can see the tree display 50 seconds in to the video below:
There also seems to be some confusion over the license used. The disclaimer states that it is Creative Commons Attribution-Noncommercial 2.5 Generic, but the link is to Attribution-Noncommercial-No Derivative Works 3.0. There's a big difference. The disclaimer encourages us to remix it, the linked license says that we cannot.
Out of curiosity I grabbed the code from the web site (a 1.5Gb file) and had a quick look. The bulk of the files are media, such as images, movies, and 3D Maya models. There's some nice stuff here. The actual tree itself is there in New Hampshire eXtended format. Here it is displayed in TreeView X:
Here's the NHX tree itself.
((((((((((((((((((((((((((((((Antelope:25.38857142856932 [&&NHX:id=51129:tol=Y ], Sheep:25.38857142856932 [&&NHX:id=51093:tol=Y ]):4.231428571428751 [&&NHX:id=51154 ], Cow:29.619999999998072 [&&NHX:id=51150 ])Bovidae:33.26333333333336 [&&NHX:id=50878:tol=Y ], Pig:62.883333333331485 [&&NHX:id=51287:tol=Y ]):0.978333333333334, Hippopotamus:63.86166666666491 [&&NHX:id=30366 ]):2.445833333333335, Camel:66.30749999999811 [&&NHX:id=30350:tol=Y ])Artiodactyla:7.337500000000006 [&&NHX:id=15976:tol=Y ], Humpback_whale:73.64499999999816 [&&NHX:id=16054 ]):9.663854166666676 [&&NHX:id=15975 ], (((Horse:72.14374999999836 [&&NHX:id=16255 ], Rhino:72.14374999999836 [&&NHX:id=16262 ])Perissodactyla:5.153125000000017 [&&NHX:id=15980:tol=Y ], Elephant:77.29687499999827 [&&NHX:id=22667:tol=Y ]):4.294270833333347 [&&NHX:id=15979 ], Aardvark:81.59114583333152 [&&NHX:id=16881 ]):1.717708333333339):15.761701388888907 [&&NHX:id=15974 ], ((((Brown_bear:40.2599999999984 [&&NHX:id=123666:tol=Y ], Polar_bear:40.2599999999984 [&&NHX:id=123667:tol=Y ])Ursidae:11.18333333333332 [&&NHX:id=16015:tol=Y ], Seal:51.443333333331715 [&&NHX:id=16020 ]):4.473333333333327, Dog:55.91666666666514 [&&NHX:id=16013 ])Caniformia:11.18333333333332 [&&NHX:id=16011 ], (Cat:40.259999999998215 [&&NHX:id=123531:tol=Y ], Lion:40.259999999998215 [&&NHX:id=123566:tol=Y ])Felidae:26.839999999999964 [&&NHX:id=16006:tol=Y ])Carnivora:31.9705555555556 [&&NHX:id=15971:tol=Y ]):0.15222222222222132, ((Mole:86.06453703703531 [&&NHX:id=16213 ], Shrew:86.06453703703531 [&&NHX:id=16223 ]):3.759497354497354, Hedgehog:89.82403439153265 [&&NHX:id=16211 ])Insectivora:9.398743386243385 [&&NHX:id=15968:tol=Y ]):0.3805555555555533, (((Chimpanzee:9.143333333331611 [&&NHX:id=26565 ], Human:9.143333333331611 [&&NHX:id=16421:tol=Y ]):82.69666666666664 [&&NHX:id=16412 ], Tree_shrew:91.83999999999833 [&&NHX:id=50808 ]):4.1099999999999755 [&&NHX:id=15962 ], Fruit_bat:95.94999999999834 [&&NHX:id=16076 ]):3.6533333333333116 [&&NHX:id=15961 ]):1.14166666666666, ((Hamster:56.967999999998256 [&&NHX:id=16546 ], Rat:56.96799999999821 [&&NHX:id=50732 ])Eumuroida:33.211999999999804 [&&NHX:id=16528 ], Rabbit:90.17999999999788 [&&NHX:id=16227 ])Glires:10.565000000000104 [&&NHX:id=15957 ]):16.484999999999623 [&&NHX:id=15955 ], Kangaroo:117.22999999999797 [&&NHX:id=16248 ]):31.38499999999999 [&&NHX:id=15993 ], Platypus:148.61499999999816 [&&NHX:id=16253:tol=Y ]):39.50999999999999 [&&NHX:id=15990 ], Megazostrodon:4.0625 [&&NHX:id=15032:nsz=2:ncol=red:ext=Y ])Cynodontia:32.5 [&&NHX:id=15030 ], Dimetrodon:4.0625 [&&NHX:id=14972:nsz=2:ncol=red:ext=Y ])Sphenacodontoidea:73.125 [&&NHX:id=14971 ], (((((((((((((Flamingo:106.30859375000043 [&&NHX:id=89474:tol=Y ], Toucan:106.3085937500002 [&&NHX:id=93330:tol=Y ]):1.4296874999999905, Penguin:107.73828125000037 [&&NHX:id=57223:tol=Y ])Neoaves:3.5742187499999765 [&&NHX:id=26305:tol=Y ], Duck:111.31250000000024 [&&NHX:id=89298:tol=Y ])Neognathae:4.289062499999972 [&&NHX:id=26291 ], Ostrich:115.60156250000016 [&&NHX:id=26289:tol=Y ])Neornithes:4.289062499999972 [&&NHX:id=15834:tol=Y ], Ichthyornis:4.386242378048764 [&&NHX:id=15833:nsz=2:ncol=red:ext=Y ])Euornithes_true_birds:12.867187499999915 [&&NHX:id=15829 ], Archaeopteryx:2.144531249999986 [&&NHX:id=15824:nsz=2:ncol=red:ext=Y ])Aves:12.867187499999915 [&&NHX:id=15721:tol=Y ], Tyrannosaurus_Rex:79.47061157226744 [&&NHX:id=15889:nsz=2:ncol=red:ext=Y ]):74.99999999999773 [&&NHX:id=15713 ], Diplodocus:157.62263488769486 [&&NHX:id=15756:nsz=2:ncol=red:ext=Y ])Saurischia:9.374999999999716 [&&NHX:id=15724 ], Iguanadon:165.44959259033112 [&&NHX:id=15740:nsz=2:ncol=red:ext=Y ])Dinosauria:20.400000000000002 [&&NHX:id=14883:tol=Y ], Crocodile:250.39999999999782 [&&NHX:id=14868 ])Archosauria:22.950000000000003 [&&NHX:id=14900:tol=Y ], Snake:273.34999999999803 [&&NHX:id=17563 ])Sauria:12.75 [&&NHX:id=14913 ], Brouffia:1.275 [&&NHX:id=14865:nsz=2:ncol=red:ext=Y ]):5.1 [&&NHX:id=14864 ], Tortoise:291.1999999999979 [&&NHX:id=17631 ])Reptilia:2.55 [&&NHX:id=14846 ])Amniota:56.25000000000091 [&&NHX:id=14990:tol=Y ], (Amphibians:317.3333333333312 [&&NHX:id=14940 ], Newt:317.3333333333311 [&&NHX:id=82771 ])Living_amphibians:32.66666666666758 [&&NHX:id=14997:tol=Y ])Tetrapoda:9.00000000000009 [&&NHX:id=14987 ], Seymouriamorpha:14.610465116279109 [&&NHX:id=17554:nsz=2:ncol=red:ext=Y ]):40.87500000000041 [&&NHX:id=14985 ], (Acanthostega:1.8750000000000189 [&&NHX:id=15016:tol=Y:nsz=2:ncol=red:ext=Y ], Icthyostega:1.8750000000000189 [&&NHX:id=15015:tol=Y:nsz=2:ncol=red:ext=Y ]):0.7500000000000075):13.125000000000131 [&&NHX:id=14976 ], Panderichthys:2.2500000000000226 [&&NHX:id=14951:nsz=2:ncol=red:ext=Y ]):4.500000000000045 [&&NHX:id=14950 ], Eusthenopteron:2.2500000000000226 [&&NHX:id=14949:nsz=2:ncol=red:ext=Y ]):9.00000000000009 [&&NHX:id=14948 ], ((Paddle_fish:363.6363636363626 [&&NHX:id=68750 ], Sturgeon:363.6363636363626 [&&NHX:id=68749 ])Acipenseriformes:24.242424242424242 [&&NHX:id=68726:tol=Y ], Clown_fish:387.8787878787866 [&&NHX:id=52149 ]):38.621212121213034 [&&NHX:id=68709 ])Osteichthyes:9.00000000000009 [&&NHX:id=14921 ], Shark:435.5 [&&NHX:id=14925 ])Node_1:13.944444444444489 [&&NHX:id=14919 ], Cephalaspidida:19.261904761904795 [&&NHX:id=16894:nsz=2:ncol=red:ext=Y ])Node_3:28.333333333333336 [&&NHX:id=14840 ], Pteraspis:46.45061728395067 [&&NHX:id=16929:nsz=2:ncol=red:ext=Y ])Node_1:9.444444444444445 [&&NHX:id=14833 ], Lamprey:487.22222222222223 [&&NHX:id=15919 ])Vertebrata:18.88888888888889 [&&NHX:id=14829:tol=Y ], Pikaia:506.1111111111113 [&&NHX:id=14824 ]):158.00000000000003 [&&NHX:id=14822 ], ((((((((Fruit_fly:339.4285714285717 [&&NHX:id=10610 ], Peacock_butterfly:339.42857142857144 [&&NHX:id=94054 ]):7.071428571428555 [&&NHX:id=8224 ], Wasp:346.5000000000002 [&&NHX:id=11244 ]):28.28571428571422 [&&NHX:id=8223 ], Cockroach:374.7857142857142 [&&NHX:id=8544 ])Neoptera:7.071428571428555 [&&NHX:id=8267:tol=Y ], Giant_dragonfly:10.607142857142833 [&&NHX:id=13265:nsz=2:ncol=red:ext=Y ])Pterygota:98.80952380952378 [&&NHX:id=8210:tol=Y ], Brine_shrimp:480.6666666666665 [&&NHX:id=6387 ]):43.81073446327684 [&&NHX:id=2527 ], ((((Harevest_mite:496.40677966101725 [&&NHX:id=2612 ], Spider:496.4067796610171 [&&NHX:id=2788:tol=Y ]):8.864406779661008 [&&NHX:id=2542 ], Harvestman:505.27118644067804 [&&NHX:id=2556 ])Arachnida:8.864406779661008 [&&NHX:id=2536:tol=Y ], Sea_Scorpion:39.88983050847454 [&&NHX:id=8174:nsz=2:ncol=red:ext=Y ])Chelicerata:7.387005649717507 [&&NHX:id=2535 ], Millipede:521.5225988700563 [&&NHX:id=52849 ]):2.954802259887003)Arthropoda:40.855932203389834 [&&NHX:id=2469:tol=Y ], (Opabinia:555.9111111111108 [&&NHX:id=20357 ], Velvet_worm:555.9111111111108 [&&NHX:id=20356 ])Onychophora:9.422222222222217 [&&NHX:id=2470:tol=Y ]):70.55555555555556 [&&NHX:id=2468 ], Acoel:635.8888888888889 [&&NHX:id=20383 ]):28.222222222222225)Bilateria:85.88888888888889 [&&NHX:id=2459:tol=Y ], Sponge:750.0 [&&NHX:id=20438 ])Animals:2250.0 [&&NHX:id=2374:tol=Y ], E._coli:3000.0 [&&NHX:id=2306 ])Life_on_earth;
It's great to see creative people tackling the challenge of displaying the tree of life. I just not convinced that this is the best way to do it.
In conjunction with the TV show, the Wellcome Trust has launched the Interactive Tree of Life, a Flash-based view of the tree of life. There's also a blog about the project. Here's a demo of the tree:
The tree looks very nice, and a lot of work has gone into it, but I am somewhat underwhelmed. The tree itself is tiny, and does a poor job of conveying the relative diversity of life (e.g., no plants, bacteria, few arthropods, etc.). It displays the tree on a 2D plane, and the user can move relative to that plane. I'm not convinced this is the best way to display large trees. Something modelled on Perceptive Pixel's demo might be more useful. I blogged about this last year, but the video host service has disappeared. You can see the tree display 50 seconds in to the video below:
There also seems to be some confusion over the license used. The disclaimer states that it is Creative Commons Attribution-Noncommercial 2.5 Generic, but the link is to Attribution-Noncommercial-No Derivative Works 3.0. There's a big difference. The disclaimer encourages us to remix it, the linked license says that we cannot.
Out of curiosity I grabbed the code from the web site (a 1.5Gb file) and had a quick look. The bulk of the files are media, such as images, movies, and 3D Maya models. There's some nice stuff here. The actual tree itself is there in New Hampshire eXtended format. Here it is displayed in TreeView X:
Here's the NHX tree itself.
((((((((((((((((((((((((((((((Antelope:25.38857142856932 [&&NHX:id=51129:tol=Y ], Sheep:25.38857142856932 [&&NHX:id=51093:tol=Y ]):4.231428571428751 [&&NHX:id=51154 ], Cow:29.619999999998072 [&&NHX:id=51150 ])Bovidae:33.26333333333336 [&&NHX:id=50878:tol=Y ], Pig:62.883333333331485 [&&NHX:id=51287:tol=Y ]):0.978333333333334, Hippopotamus:63.86166666666491 [&&NHX:id=30366 ]):2.445833333333335, Camel:66.30749999999811 [&&NHX:id=30350:tol=Y ])Artiodactyla:7.337500000000006 [&&NHX:id=15976:tol=Y ], Humpback_whale:73.64499999999816 [&&NHX:id=16054 ]):9.663854166666676 [&&NHX:id=15975 ], (((Horse:72.14374999999836 [&&NHX:id=16255 ], Rhino:72.14374999999836 [&&NHX:id=16262 ])Perissodactyla:5.153125000000017 [&&NHX:id=15980:tol=Y ], Elephant:77.29687499999827 [&&NHX:id=22667:tol=Y ]):4.294270833333347 [&&NHX:id=15979 ], Aardvark:81.59114583333152 [&&NHX:id=16881 ]):1.717708333333339):15.761701388888907 [&&NHX:id=15974 ], ((((Brown_bear:40.2599999999984 [&&NHX:id=123666:tol=Y ], Polar_bear:40.2599999999984 [&&NHX:id=123667:tol=Y ])Ursidae:11.18333333333332 [&&NHX:id=16015:tol=Y ], Seal:51.443333333331715 [&&NHX:id=16020 ]):4.473333333333327, Dog:55.91666666666514 [&&NHX:id=16013 ])Caniformia:11.18333333333332 [&&NHX:id=16011 ], (Cat:40.259999999998215 [&&NHX:id=123531:tol=Y ], Lion:40.259999999998215 [&&NHX:id=123566:tol=Y ])Felidae:26.839999999999964 [&&NHX:id=16006:tol=Y ])Carnivora:31.9705555555556 [&&NHX:id=15971:tol=Y ]):0.15222222222222132, ((Mole:86.06453703703531 [&&NHX:id=16213 ], Shrew:86.06453703703531 [&&NHX:id=16223 ]):3.759497354497354, Hedgehog:89.82403439153265 [&&NHX:id=16211 ])Insectivora:9.398743386243385 [&&NHX:id=15968:tol=Y ]):0.3805555555555533, (((Chimpanzee:9.143333333331611 [&&NHX:id=26565 ], Human:9.143333333331611 [&&NHX:id=16421:tol=Y ]):82.69666666666664 [&&NHX:id=16412 ], Tree_shrew:91.83999999999833 [&&NHX:id=50808 ]):4.1099999999999755 [&&NHX:id=15962 ], Fruit_bat:95.94999999999834 [&&NHX:id=16076 ]):3.6533333333333116 [&&NHX:id=15961 ]):1.14166666666666, ((Hamster:56.967999999998256 [&&NHX:id=16546 ], Rat:56.96799999999821 [&&NHX:id=50732 ])Eumuroida:33.211999999999804 [&&NHX:id=16528 ], Rabbit:90.17999999999788 [&&NHX:id=16227 ])Glires:10.565000000000104 [&&NHX:id=15957 ]):16.484999999999623 [&&NHX:id=15955 ], Kangaroo:117.22999999999797 [&&NHX:id=16248 ]):31.38499999999999 [&&NHX:id=15993 ], Platypus:148.61499999999816 [&&NHX:id=16253:tol=Y ]):39.50999999999999 [&&NHX:id=15990 ], Megazostrodon:4.0625 [&&NHX:id=15032:nsz=2:ncol=red:ext=Y ])Cynodontia:32.5 [&&NHX:id=15030 ], Dimetrodon:4.0625 [&&NHX:id=14972:nsz=2:ncol=red:ext=Y ])Sphenacodontoidea:73.125 [&&NHX:id=14971 ], (((((((((((((Flamingo:106.30859375000043 [&&NHX:id=89474:tol=Y ], Toucan:106.3085937500002 [&&NHX:id=93330:tol=Y ]):1.4296874999999905, Penguin:107.73828125000037 [&&NHX:id=57223:tol=Y ])Neoaves:3.5742187499999765 [&&NHX:id=26305:tol=Y ], Duck:111.31250000000024 [&&NHX:id=89298:tol=Y ])Neognathae:4.289062499999972 [&&NHX:id=26291 ], Ostrich:115.60156250000016 [&&NHX:id=26289:tol=Y ])Neornithes:4.289062499999972 [&&NHX:id=15834:tol=Y ], Ichthyornis:4.386242378048764 [&&NHX:id=15833:nsz=2:ncol=red:ext=Y ])Euornithes_true_birds:12.867187499999915 [&&NHX:id=15829 ], Archaeopteryx:2.144531249999986 [&&NHX:id=15824:nsz=2:ncol=red:ext=Y ])Aves:12.867187499999915 [&&NHX:id=15721:tol=Y ], Tyrannosaurus_Rex:79.47061157226744 [&&NHX:id=15889:nsz=2:ncol=red:ext=Y ]):74.99999999999773 [&&NHX:id=15713 ], Diplodocus:157.62263488769486 [&&NHX:id=15756:nsz=2:ncol=red:ext=Y ])Saurischia:9.374999999999716 [&&NHX:id=15724 ], Iguanadon:165.44959259033112 [&&NHX:id=15740:nsz=2:ncol=red:ext=Y ])Dinosauria:20.400000000000002 [&&NHX:id=14883:tol=Y ], Crocodile:250.39999999999782 [&&NHX:id=14868 ])Archosauria:22.950000000000003 [&&NHX:id=14900:tol=Y ], Snake:273.34999999999803 [&&NHX:id=17563 ])Sauria:12.75 [&&NHX:id=14913 ], Brouffia:1.275 [&&NHX:id=14865:nsz=2:ncol=red:ext=Y ]):5.1 [&&NHX:id=14864 ], Tortoise:291.1999999999979 [&&NHX:id=17631 ])Reptilia:2.55 [&&NHX:id=14846 ])Amniota:56.25000000000091 [&&NHX:id=14990:tol=Y ], (Amphibians:317.3333333333312 [&&NHX:id=14940 ], Newt:317.3333333333311 [&&NHX:id=82771 ])Living_amphibians:32.66666666666758 [&&NHX:id=14997:tol=Y ])Tetrapoda:9.00000000000009 [&&NHX:id=14987 ], Seymouriamorpha:14.610465116279109 [&&NHX:id=17554:nsz=2:ncol=red:ext=Y ]):40.87500000000041 [&&NHX:id=14985 ], (Acanthostega:1.8750000000000189 [&&NHX:id=15016:tol=Y:nsz=2:ncol=red:ext=Y ], Icthyostega:1.8750000000000189 [&&NHX:id=15015:tol=Y:nsz=2:ncol=red:ext=Y ]):0.7500000000000075):13.125000000000131 [&&NHX:id=14976 ], Panderichthys:2.2500000000000226 [&&NHX:id=14951:nsz=2:ncol=red:ext=Y ]):4.500000000000045 [&&NHX:id=14950 ], Eusthenopteron:2.2500000000000226 [&&NHX:id=14949:nsz=2:ncol=red:ext=Y ]):9.00000000000009 [&&NHX:id=14948 ], ((Paddle_fish:363.6363636363626 [&&NHX:id=68750 ], Sturgeon:363.6363636363626 [&&NHX:id=68749 ])Acipenseriformes:24.242424242424242 [&&NHX:id=68726:tol=Y ], Clown_fish:387.8787878787866 [&&NHX:id=52149 ]):38.621212121213034 [&&NHX:id=68709 ])Osteichthyes:9.00000000000009 [&&NHX:id=14921 ], Shark:435.5 [&&NHX:id=14925 ])Node_1:13.944444444444489 [&&NHX:id=14919 ], Cephalaspidida:19.261904761904795 [&&NHX:id=16894:nsz=2:ncol=red:ext=Y ])Node_3:28.333333333333336 [&&NHX:id=14840 ], Pteraspis:46.45061728395067 [&&NHX:id=16929:nsz=2:ncol=red:ext=Y ])Node_1:9.444444444444445 [&&NHX:id=14833 ], Lamprey:487.22222222222223 [&&NHX:id=15919 ])Vertebrata:18.88888888888889 [&&NHX:id=14829:tol=Y ], Pikaia:506.1111111111113 [&&NHX:id=14824 ]):158.00000000000003 [&&NHX:id=14822 ], ((((((((Fruit_fly:339.4285714285717 [&&NHX:id=10610 ], Peacock_butterfly:339.42857142857144 [&&NHX:id=94054 ]):7.071428571428555 [&&NHX:id=8224 ], Wasp:346.5000000000002 [&&NHX:id=11244 ]):28.28571428571422 [&&NHX:id=8223 ], Cockroach:374.7857142857142 [&&NHX:id=8544 ])Neoptera:7.071428571428555 [&&NHX:id=8267:tol=Y ], Giant_dragonfly:10.607142857142833 [&&NHX:id=13265:nsz=2:ncol=red:ext=Y ])Pterygota:98.80952380952378 [&&NHX:id=8210:tol=Y ], Brine_shrimp:480.6666666666665 [&&NHX:id=6387 ]):43.81073446327684 [&&NHX:id=2527 ], ((((Harevest_mite:496.40677966101725 [&&NHX:id=2612 ], Spider:496.4067796610171 [&&NHX:id=2788:tol=Y ]):8.864406779661008 [&&NHX:id=2542 ], Harvestman:505.27118644067804 [&&NHX:id=2556 ])Arachnida:8.864406779661008 [&&NHX:id=2536:tol=Y ], Sea_Scorpion:39.88983050847454 [&&NHX:id=8174:nsz=2:ncol=red:ext=Y ])Chelicerata:7.387005649717507 [&&NHX:id=2535 ], Millipede:521.5225988700563 [&&NHX:id=52849 ]):2.954802259887003)Arthropoda:40.855932203389834 [&&NHX:id=2469:tol=Y ], (Opabinia:555.9111111111108 [&&NHX:id=20357 ], Velvet_worm:555.9111111111108 [&&NHX:id=20356 ])Onychophora:9.422222222222217 [&&NHX:id=2470:tol=Y ]):70.55555555555556 [&&NHX:id=2468 ], Acoel:635.8888888888889 [&&NHX:id=20383 ]):28.222222222222225)Bilateria:85.88888888888889 [&&NHX:id=2459:tol=Y ], Sponge:750.0 [&&NHX:id=20438 ])Animals:2250.0 [&&NHX:id=2374:tol=Y ], E._coli:3000.0 [&&NHX:id=2306 ])Life_on_earth;
It's great to see creative people tackling the challenge of displaying the tree of life. I just not convinced that this is the best way to do it.
Wednesday, January 21, 2009
Darwin on Twitter
Well, not Darwin himself, exactly. The Evolution Directory (better known as "EvolDir") is a mailing list run by Brian Golding at McMaster University, Ontario. It's widely used by evolutionary biologists to post announcements about jobs, courses, conferences, software, and other topics of interest to the community.
In this age of spam- and administrivia-clogged inboxes I find it hard to keep track of emails (and routinely ignore most), so it occurred to me that I'd pay more attention to EvolDir if the announcements were made over twitter. Hence, I wrote a script to monitor my email account for EvolDir emails, and post any announcements to twitter. You can follow EvolDir by going to http://twitter.com/evoldir.
One complication was working out a URL for individual EvolDir emails. To make my life simpler I created a simple web archive of each individual email. As a by product of this there is now an RSS feed for EvolDir, available from http://bioguid.info/services/evoldir/, if RSS is your preferred means of consuming news. I've added the EvolDir RSS feed to the Systematic Biology web site, so that visitors to that site can see the latests announcements from the evolutionary biology community.
Sunday, January 18, 2009
Equivalent author names
One problem I've encountered in building a bibliographic database is the different ways author names are written. For example, for papers I've authored my name may be written as "Roderic D. M. Page" or "R. D. M. Page". Googling about this problem I came across Dror Feitelson's paper On identifying name equivalences in digital libraries. Feitelson addresses the issue of matching first names:
In this example, the names "Abe Bob C", "A B C", and "Abe B" are equivalent, as are "Ace D E" and "A D", leaving "Abe F G" by itself.
I've implemented Feitelson's weighted clique algorithm in a PHP script that calls a C++ program that does the clique analysis. Results can be returned in HTML or JSON. You can try the service at http://bioguid.info/services/. You can also call the service directly by a HTTP POST request to the URL http://bioguid.info/services/equivalent.php with these parameters:
The services provided by digital libraries can be much improved by correctly identifying variants of the same name. For example, this will allow for better retrieval of all the works by a certain author. We focus on variants caused by abbreviations of first names, and show that significant achievements are possible by simple lexical analysis and comparison of names. This is done in two steps: first a pairwise matching of names is performed, and then these are used to find cliques of equivalent names. However, these steps can each be performed in a variety of ways. We therefore conduct an experimental analysis using two real datasets to find which approaches actually work well in practice. Interestingly, this depends on the size of the repository, as larger repositories may have many more similar names.Feitelson's solution is to construct a graph of similarity between first names, then find weighted cliques grouping equivalent names. For example, given the first names "Ace D. E.", "A. D.", "Abe F. G.", "Abe Bob C.", "A. B. C.", and "Abe B", we create the graph below where the edges are weighted by similarity between the names:
In this example, the names "Abe Bob C", "A B C", and "Abe B" are equivalent, as are "Ace D E" and "A D", leaving "Abe F G" by itself.
I've implemented Feitelson's weighted clique algorithm in a PHP script that calls a C++ program that does the clique analysis. Results can be returned in HTML or JSON. You can try the service at http://bioguid.info/services/. You can also call the service directly by a HTTP POST request to the URL http://bioguid.info/services/equivalent.php with these parameters:
| Parameter | Value | Description |
|---|---|---|
| names | string | List of first names, separated by end of line (\n) character |
| format | html or json | Format of the results |
Thursday, January 15, 2009
Wikis versus Scratchpads
Yes, I know this is ultimately a case of the "genius of and", but the more I play with the Semantic Mediawiki extension the more I think this is going to be the most productive way forward. I've had numerous conversations with Vince Smith about this. Vince and colleagues at the NHM have been doing a lot of work on "Scratchpads" -- Drupal based webs sites that tend to be taxon-focussed. My worry is that in the long term this is going to create lots of silos that some poor fool will have to aggregate together to do anything synthetic with. This makes inference difficult, and also raises issues of duplication (for example, in bibliographies).
I've avoided wikis for a while because of the reliance on plain text (i.e., little structure) (see this old post of mine on Semant), but Semantic Mediawiki provides a fairly simple way to structure information, and it also provides some basic inference. This makes it possible to create wiki pages that are largely populated by database queries, rather than requiring manual editing. For example, I have queries now that will automatically populate a page about a person with that person's publications, and any taxa named after that person. The actual wiki page itself has hardly any text (basically the name of the person). That is, nobody has to manually edit the wiki page to update lists of published papers. Similarly, maps can be generated in situ using queries that aggregate localities mentioned on a wiki page with localities for GenBank sequences and specimens. Very quickly relationships start to emerge without any manual intervention. The combination of templates and Semantic Mediawiki queries seems a pretty powerful way to aggregate information. There are, of course, limitations. The queries are fairly basic, and there's not the power of something like SPARQL, but it's a start. Coupled with the ease of editing to fix the errors in the contributing databases, and the ease of redirecting to handle multiple identifiers, I think a wiki-based approach has a lot of promise.
So, I've been teasing Vince that Drupal (or another CMS) is probably the wrong approach, and that semantic wikis are much more powerful (something Gregor Hagedorn has also been arguing). Vince would probably counter that the goal of scratchpads is to move taxonomists into the digital age by providing them with a customisable platform for them to store and display their data, hence his mission is to capture data. My focus is more to do with aggregating and synthesising the large amount of data we already have (and are struggling to do anything exciting with). Hence, the "genius of and". However, I still worry that when we have a world with loads of scratch pads with overlapping data, some poor fool will still have to merge them together to make sense of it all.
I've avoided wikis for a while because of the reliance on plain text (i.e., little structure) (see this old post of mine on Semant), but Semantic Mediawiki provides a fairly simple way to structure information, and it also provides some basic inference. This makes it possible to create wiki pages that are largely populated by database queries, rather than requiring manual editing. For example, I have queries now that will automatically populate a page about a person with that person's publications, and any taxa named after that person. The actual wiki page itself has hardly any text (basically the name of the person). That is, nobody has to manually edit the wiki page to update lists of published papers. Similarly, maps can be generated in situ using queries that aggregate localities mentioned on a wiki page with localities for GenBank sequences and specimens. Very quickly relationships start to emerge without any manual intervention. The combination of templates and Semantic Mediawiki queries seems a pretty powerful way to aggregate information. There are, of course, limitations. The queries are fairly basic, and there's not the power of something like SPARQL, but it's a start. Coupled with the ease of editing to fix the errors in the contributing databases, and the ease of redirecting to handle multiple identifiers, I think a wiki-based approach has a lot of promise.
So, I've been teasing Vince that Drupal (or another CMS) is probably the wrong approach, and that semantic wikis are much more powerful (something Gregor Hagedorn has also been arguing). Vince would probably counter that the goal of scratchpads is to move taxonomists into the digital age by providing them with a customisable platform for them to store and display their data, hence his mission is to capture data. My focus is more to do with aggregating and synthesising the large amount of data we already have (and are struggling to do anything exciting with). Hence, the "genius of and". However, I still worry that when we have a world with loads of scratch pads with overlapping data, some poor fool will still have to merge them together to make sense of it all.
Friday, December 19, 2008
Code prettyfier
Test of google-code-prettify:
class Voila {
public:
// Voila
static const string VOILA = "Voila";
// will not interfere with embedded tags.
}
Thursday, December 18, 2008
Failure
Success is the ability to go from failure to failure without losing your enthusiasm -- Winston ChurchillI learnt today that my Elsevier Challenge entry didn't make the final cut. This wasn't unexpected. In the interests of "open science" (blame Paulo Nuin) here is the feedback I received from the judges:
StrengthsI think this is a pretty fair evaluation of my entry. I was making a case for what could be done, rather than providing a specific bit of kit that could make this happen right now. I think I was also a little guilty of not following the "underpromise but overdeliver" mantra. My original proposal included harvesting phylogenies from images, and that proved too difficult to do in the time available. I don't think having trees would have ultimately changed the result (i.e., not making the cut), but it would have been cool to have them.
Beautiful presentation, lovely website. Page clearly made his case for open access to metadata/full articles in order to allow communities to build the tools they want. The judges would have enjoyed seeing more elements from the original abstract (tree of life). Great contribution so far to the discussion; Page made his point very well.
Weaknesses
Given that no specific tool was proposed, this submission is somewhat out of scope for the competition. Nonetheless, in support of his point, Page could have elaborated on the kinds of open formats and standards for text and data and figures that would support integrated community-wide tool-building. Alternatively, if the framework and the displayed functionalities were to be the submission, there could have been more discussion of how others can integrate their plug-ins and make them cross-referential to the plug-ins of others. The proposal for Linked Data should utilize Semantic Web standards
Elements to Consider for Development
How many, and which types of, information substrates? How much work for a new developer to create a new one, and to make this work? How to incentivize authors to produce the required metadata? Or to make the data formats uniform?
Anyway, time to stomp around the house a bit, and be generally grumpy towards innocent children and pets. Congratulations to the
Subscribe to:
Posts (Atom)