BioNames and where taxonomy is published

I've added a simple "dashboard" to BioNames to display some basic data about what is in the database. Apart from a table of the number of bibliographic identifiers in the database (currently there are 54,422 publications with DOIs, for example), there are some graphic summaries. These are a bit slow to load as they are created on the fly.

Publishers

The first summarises the relative frequency of articles from different publishers (broadly defined to include digital repositories such as DSpace and JSTOR). For most of this information I'm using data returned when I resolve a DOI at CrossRef. The data is incomplete and likely to change as I add more articles, and CouchDB finally catches up and indexes all the data.



The biggest blob is BioStor, which is my project to extract articles from BHL. Magnolia Press publish Zootaxa, then there are some well-known mainstream publishers such as Springer, Wiley, and Taylor & Francis (Informa UK). These publishers have digitised the back catalogues of a number of society journals, so their prominence here doesn't mean that they are actively publishing new taxonomic content. One use for a diagram like this is to think about what content to data mine. BioStor content is open access (via BHL) and so can be readily mined. Some articles in Zootaxa are open access and so could also be downloaded and processed. Then we have the big commercial publishers, who have a significant fraction of taxonomic content behind their paywalls. If the community was to think about mining this data, then this diagram suggests which publishers to start asking first.

Journals

The next diagram shows articles grouped by journal (using the journal's ISSN).



There circles are too small to be labelled usefully. A couple of things strike me. The first is the sheer number of journals! The taxonomic literature is widely scattered across numerous different outlets, which is part of the challenge of indexing the literature (and this diagram includes only those journals that have ISSNs, many smaller or older ones don't). There is no one journal which dominates the landscape (the largest circle on the top right is Zootaxa). But this diagram spans the complete history of taxonomic publication, so includes large journals (such as Annals and Magazine of Natural History) that no longer exist (at least in their present form). Might be useful to slice this diagram by, say, decade to get a clearer picture of patterns of publication.

As the database builds I post some more summaries at BioNames.

BioNames and altmetrics

One consequence of having a database of literature with external identifiers such as DOIs is that we can plug into a bunch of external services to get additional information about a reference. For example, altmetric can take a DOI and display some article level metrics. As an experiment I've added code for altmetric badges to the web page in BioNames that displays publications. For example, here is the ZooKeys paper "An extraordinary new family of spiders from caves in the Pacific Northwest (Araneae, Trogloraptoridae, new family)" in BioNames:



The "About" tab displays the altmetric badge with a bunch of metrics of engagement with this paper. If you click on the badge you get more details about what people have been saying about this paper.

It would be great to explore this across the complete set of taxonomic papers so that we could get a sense the degree of engagement people have with the latest taxonomic literature.

BioNames now live - Report on project

BioNames (http://bionames.org) is live. Getting to this point was supported by funding from EOL as part of their Computable Data Challenge. The award from EOL is paying for Ryan Schenk to work on the interface and overall design of the web site, and over the last few weeks we've been working increasingly frantically to get things ready. "Ready" is a relative concept. The project is far from finished from my perspective, there is a mound of data (millions of names, hundreds of thousands of publications) that is being cleaned, cross-linked, and ultimately visualised. But the EOL funding came with a deadline and adult supervision (aka Cyndy Parr), so it was a great incentive to get something function out the door.

What is BioNames?

Elsewhere I've argued that biodiversity informatics is fundamentally about linking stuff together, and BioNames tackles the link between a name and its publication. Ultimately I want each taxon name to be linked to its original description, and that description has a digital identifier (such as a DOI). It's a small step, but building those links, coupled with (where possible) bringing those publications together in one place provides a platform to potentially do some cool stuff (more on this later). Since about 2009 I've been working on building a database of these links, and have been documenting progress (or it's lack) along the way (e.g., search iPhylo for "itaxon").

Here are some screen shots (and links so you can see for your self). It's a very early stage release, but you'll get the idea.

GBIF classiifcation of Rousettus with Ryan's awesome taxon name timeline.


Viewing a paper A Tarzan yell for conservation: a new chameleon, Calumma tarzan sp. n., proposed as a flagship species for the creation of new nature reserves in Madagascar

Coverage of articles in a journal (Proceedings of the Entomological Society of Washington).
What got built?

There is a bunch of code and documentation online:

• Original proposal
• API documentation (needs to be updated)
• github repository for API and in-progress code
• github repository for data uploading
• Some design ideas (old)
• github repository for design documents
• github repository for website
• Trello board tracking the project
• Code for populating database

There is also a Darwin Core Archive format dump, which *cough* fun to create.

There have been progress reports on this blog (search for BioNames). You can also see what we got up to in the github logs.

What didn't happen

The original proposal (http://dx.doi.org/10.6084/m9.figshare.92091) was, of course, a tad ambitious, and a number of things haven't made it into this release. Phylogenies are the biggest casualty, but they are close (see Viewing phylogenies on the web: Javascript conversion of Newick tree to SVG for experiments on visualisation). It just wasn't possible to get them ready in time for the May 31 deadline. But this is on the to do list.

What's next

Now that there is a functioning web site there are several directions to explore. There is a lot of data cleaning to do, many missing references to add, taxon names to map to GBIF and NCBI, and more. I've completely glossed over the issue of reconciling author names, it's clear that the same author can appear multiple times because of variations in how their name has been recorded in various databases. There are various ways to tackle this, the most interesting is to use tools like Mendeley or ORCID to enable people to "claim" their identity.

Now that there is a mapping between the NCBI taxonomy and taxon names linked to literature, it would be great to add phylogenetic data to BioNames (which was part of the original plan). One way is by importing PhyLoTA, another is by adding support for BLAST searches that generate trees. For example, for a given taxon we could create a list of suitable sequences (e.g., DNA barcodes) and enable users to generate BLAST trees to get a sense of what the taxon is related to (and, in many cases, how much genetic differentiation there is within that taxon).

Given that BioNames has a lot of full text (from BioStor as well as numerous sources of PDFs and scans) there is huge scope for data mining. Obvious things to do are extract taxon names, geographic localities, and specimen codes (using tools I already have for BioStor). Then there is the challenge of extracting lists of literature cited and building citation networks. A small proportion of the taxonomic literature exists in XML (e.g., articles in PLoS, Zookeys, and various SciElo journals), which makes this task a lot easier. Given that many of the cited papers will already be in BioNames, we could build a taxonomic literature reader that enabled you to treat the literature in BioNames as one big interlinked, browesable archive. I'm posting a list of ideas on Trello.

Multiple DOIs for the same article issued by different publishers

I've stumbled on a case where two different publishers have issued different DOIs for the same articles. In this case, Springer and J-State both publish the Japanese Journal of Ichthyology (ISSN 0021-5090). The following article:

Randall, J. E., & McCarthy, L. J. (1989). Solea stanalandi, a new sole from the Persian Gulf. Japanese Journal of Ichthyology, 36(2), 196–199. doi:10.1007/BF02914322

is published by Springer with the DOI http://dx.doi.org/10.1007/BF02914322, and this DOI is registered with CrossRef. J-Stage publish the same article, with the DOI (http://dx.doi.org/10.11369/jji1950.36.196). This DOI is not registered with CrossRef. I haven't been able to find an easy way to discover the DOI registration agency for a DOI (surely there should be a simple service that tells me this?).

This illustrates a problem with the success of DOIs and the existence of multiple registration agencies. When there was essentially a single agency for publications (CrossRef) it was relatively easy to ensure that DOIs for publications were unique. Now that there are multiple DOI registration agencies it is possible for conflicts to arise. We might expect this to be rare, after all, surely there's only one publisher for an article? However, the publishing landscape is more complicated that that, with articles being served by multiple publishers, and archiving projects like JSTOR and BHL having content that overlaps with that of existing publishers. Messy (sigh).

BioNames update - deadlines

This week promises to be *cough* interesting. The deadline for the BioNames project is the end of the week (May 31st), and all I have right now is a blank web page (gulp). Behind the scenes data is being cleaned, mushed, and reconciled, and CSS, Javascript, and HTML are being wrangled. It's going to be tight, but hey, what could possibly go wrong...?

DOIs for specimens are here, but we're not quite there yet

Cool! @alaskamuseum has DOIs for specimens, e.g. dx.doi.org/10.7299/X78052… (minted by @ezidcdl )

— Roderic Page (@rdmpage) May 22, 2013

I've been banging on about having citable, persistent identifiers for specimens, so was suitably impressed when Derek Sikes posted a comment on iPhylo that Arctos already does this. For example, here is a DOI for a specimen: http://dx.doi.org/10.7299/X7VQ32SJ.



So, we're all done, right? Not quite. DOIs by themselves don't get us where we (OK, where I think we) want to be. The DOI identifies a specimen, which is great (see discussion on iDigBio: You are putting identifiers on the wrong thing for why this matters). We can also get machine-readable metadata using the DOI (by using the URL http://data.datacite.org/10.7299/X7VQ32SJ ). The metadata is limited (ideally we'd want something like Darwin Core), but it is a start. It's not clear how we get from the DOI to Darwin Core.

There are at least two issues that remain to be tackled. The first is that we now have a bunch of identifiers for the same thing, e.g.:

• http://dx.doi.org/10.7299/X7VQ32SJ
• http://arctos.database.museum/guid/UAM:Ento:230092
• http://data.gbif.org/occurrences/773259491/
• UAM Ento 230092 (GBIF)
• UAM Insects 230092 (Arctos)
• BYU_gb32.1 (see DQ457300

Most of these identifiers don't know about each other (for example, GBIF doesn't know about the DOI, nor does Arctos link to GBIF). So we have disconnected pieces of information about the same thing.

The second issue is how do we discover a specimen DOI? CrossRef supports services where you can take a bibliographic citation, e.g. Phylogeny and biogeography of ice crawlers (Insecta: Grylloblattodea) based on six molecular loci: designating conservation status for Grylloblattodea species and get back a DOI (in this case, http://dx.doi.org/10.1016/j.ympev.2006.04.013). This makes it possible for publishers to take lists of literature cited in authors' manuscripts and quickly add DOIs to those citations. We don't have an equivalent service for specimens, which is going to make our task of linking specimens to sequences and the literature something of a challenge.

We are making progress, but there is some way to go. Identifiers are only part of the solution, we also need services.

The impact of museum collections: one collection ≈ one Nobel Prize

iPhylo: GBIF specimens in BioStor: who are the top ten museums?(Unfortunately not @amnh) iphylo.blogspot.com/2012/02/gbif-s…

— Susan Perkins (@NYCuratrix) May 14, 2013

Ideas on measuring the "impact" of a natural history collection have been bubbling along, as reflected in recent comments on iPhylo, and some offline discussions I've been having with David Blackburn and Alan Resetar.

My focus has been at the specimen-level, with a view to motivation the adoption of persistent specimen-level identifiers so that we can citations of specimens over time (e.g., in publications and databases such as GenBank). Not only does this provide a measure of the "impact" of a collection, it helps with provenance. If we sequence a specimen that is subsequently assigend to a different taxon and we have a way of tracking that specimen via its identifier, then we can transmit that new identification to other consumers of data based on that specimen. For example, we could automatically notify GenBank that what we thought was an x is actually a y.

So I made a simple "league table" of museum collections based on specimens cited in BioStor. There are all sorts of issues with this approach. Once you rank collections, people may use that to argue some can be axed and more resources funnelled into others. A more positive approach would be to indetify collections that are underused, and try and figure out why. And in the same way that taxonomic papers may have a citation long life, specimens may sit in a museum for a long time before being cited (for example, when eventually recognised as a new species doi:10.1016/j.cub.2012.10.029). So, metrics can be a double-edged sword.

Citing specimens is a useful metric, but not all citations are equal, and not all citations are immediate. A specimen that yields DNA sequences that are published in, say, Nature, arguably has more weight than a specimen listed in a rarely cited paper. Likewise, subsequent citations of a paper that cites a specimen should confer more weight on the value of that specimen. Elsewhere (doi:10.1093/bib/bbn022, preprint here: hdl:10101/npre.2008.1760.1) I've argued for a Google PageRank-style way to measure the impact of a specimen that takes into account papers and other objects derived from a specimen (e.g., images, sequences).

Meanwhile, Morgan Jackson alerted me to a quicker way to get a measure of the impact of the collection.

@rdmpage @nycuratrix Check Nature for a recent note about this. A bird museum calc. their collection's h-score from papers citing specimens

— Morgan Jackson (@BioInFocus) May 16, 2013

The "short note" Morgan refers to is by Kevin Winker and Jack J. Withrow:

Winker, K., & Withrow, J. J. (2013). Natural history: Small collections make a big impact. Nature, 493(7433), 480–480. doi:10.1038/493480b

They constructed a Google Scholar profile and collected papers that cite the University of Alaska Museum's bird collection (see here for full details). The h-score of this collection of papers is 42, which Winkler and Withrow note is "equivalent to an average Nobel laureate in physics". Here's the graph of citations over time:


It's a neat trick, if a little time consuming. But one advantage it has is that it puts collections on a similar footing to individual researchers. You could imagine asking the question "how much money would you spend supporting a researcher at this level?" How does this compare to the resources actually being spent?

One thing I hope will emerge from discussions like this is a desire to make specimens first-class citizens of the web, with stable identifiers that enable them to be cited in the same way we cite papers and, increasingly, data sets.

GBIF data quality: visualising Mesibov's millipedes

Bob Mesibov (who has been a guest author on this blog) recently published a paper on data quality in in ZooKeys:

Mesibov, R. (2013). A specialist’s audit of aggregated occurrence records. ZooKeys, 293(0), 1–18. doi:10.3897/zookeys.293.5111

In this paper Bob documents some significant discrepancies between data in his Millipedes of Australia (MoA) database and the equivalent data in the Atlas of Living Australia and GBIF (disclosure, I was a reviewer of the paper, and also sit on GBIF's science committee). This paper spawned a thread on TAXACOM, and also came up at the GBIF meeting I was at earlier this week.

One thing lacking from the discussion is a clear sense of just how big are the discrepancies between GBIF and MoA data, so I grabbed the data provided by Bob (http://dx.doi.org/10.3897/zookeys.293.5111.app and extracted the records where GBIF and MoA disagreed. I converted these to GeoJSON and threw them on Google Maps:



You can see a live version here http://bl.ocks.org/rdmpage/raw/5501293/ (it can take a little while for the map to appear). I've connected the MoA and GBIF localities for the same occurrence by a straight line, and the the MoA records are encircled by an estimate of their uncertainty (for many records the circle is invisible at this scale).

There are some fairly spectacular discrepancies, and a lot of relatively small scale displacements of records. Does this matter? The answer to this question will depend on what people want to do with the data. You may regard the discrepancies as serious (certainly it's interesting that there are so many differences between the two data sets), or minor given the geographic scale. But visualising them at least makes it possible to form a judgement.

BioNames update - live mockup

Things are finally coming together, at least enough to have a functioning demo. It looks awful, but shows the main things I want BioNames to do. One thing I'm most concerned about at this stage is the possible confusion users might experience between taxon names and concepts. For example, there are two pages about Pteropus, one about the name Pteropus, the other about the bat that bears this name (as understood by GBIF).

The demo is live at http://bionames.org/bionames-api/mockup_index.php (note that this is a temporary URL so I can't guarantee it will be online when you read this).

BioNames live mockup from Roderic Page on Vimeo.

BioNames update - reconciliation strategies

Over on Google Plus (yeah, me neither) Donat Agosti is giving me a hard time regarding the quality of some data that I am using. I've responded to Donat directly, but here I just want to quickly outline two different approaches to cleaning and reconciling bibliographic metadata.

The problem addressed by Donat is the issue of multiple strings for the same journal (e.g., the plethora of different abbreviations and permutations people use to refer to the same journal). In trying to make sense of this mess there are a couple of strategies we can use. One is to cluster the strings into sets that we think refer to the same thing, e.g.:


We could then synthesise the preferred journal name from this set. We could make some sort of consensus string, for example. There are also some quite nice Bayesian methods for combining contradictory metadata.

Another approach, which I use, is to map the strings to a third party identifier, in this case an ISSN:


Once I've done this I can use the identifier to refer to the journal, hence ultimately I don't particularly care what string is best for the journal (indeed, I can defer to a third party for this decision).

The point is obsessing with clean, "correct" bibliographic metadata is something of a fool's errand. Obviously, it's nice to have clean metadata if you can get it, but in many cases there is no exact answer to what is the correct metadata. Some journals have multiple names (e.g., in different languages), some run different volume numbering schemes in parallel, and date of publication can be rather problematic (see my Mendeley group on publication dates). If we can map a publication to a globally unique identifier, such as a DOI, then we can sidestep this issue and focus on what I think really matters - linking data together.

Thoughts on GBIC 2012 and a vision of the future of biodiversity informatics

This seems to be the season for big, arm-wavy documents about the future of biodiversity informatics (see A decadal view of biodiversity informatics: challenges and priorities). An equivalent document is being drafted based on the Global Biodiversity Informatics Conference (GBIC 2012) conference. Writing these documents is hard work, they have to balance a set of conflicting visions, predict the future, and communicate a coherent plan to people who either could help make this happen, or feel they have a stake in the outcome.

Leaving all those constraints behind, and waving arms wildly, here's one take on the future of biodiversity informatics. I see three themes.

1. Knowing what we know

We have a limited grasp of how much we actually know, and crap tools to summarise this knowledge. I want a Google Analytics for biodiversity data where I can see at a glance the current state of our knowledge (e.g., what is the rate of sequencing of environmental samples in the Mediterranean? How much of Indonesia's amphibian fauna is in protected areas?). These are fairly trivial queries. If Google can analyse web traffic from sites being hit over a million times per day ( ~ 365 million hits per year) we can do the same thing on GBIF-scale databases. There is huge scope here for cool visualisation of the growth of our knowledge, such as this:

If biologists were explorers (Mammalia)... from Andrew W Hill on Vimeo.

Imagine the GBIF classification like this:

filesystem visualisation from wonderful websolutions on Vimeo.

2. Life stream

Terrible title, but this is where we monitor change, both "organic" and anthropogenic. This is where we use data mining to do a sentiment analysis of the biosphere, looking to detect changes such as outbreaks of disease, invasive species, etc. This builds on 1 but focusses on change. Imagine a "news service" for biology along the lines of tools available to financial markets (e.g., Silobreaker):



This is where we interface with decision makers, in the sense that Braulio Dias's statement "I am convinced that the lack of adequate biodiversity monitoring is at the heart of our difficulties to make convincing arguments" is true, this tackles that question.

3. Modelling the biosphere

Time to model all life on Earth (http://dx.doi.org/10.1038/493295a) is our equivalent of a moon shot (oh how I hate that analogy). Purves et al. have made the case, this is the task that will galvanise people outside the taxonomy/biodiversity community. This is real megascience (1. is data collection, 2. is data mining and analysis). Climate modellers and oceanographers get to do this:



Can we do the same?

Reconciling author names using Open Refine and VIAF

In an earlier post I discussed using Open Refine (formerly Google Refine) to clean and reconcile taxon names. I've added an additional service that can be used to reconcile author names that uses the Virtual International Authority File (VIAF) API. Using this service we can match authors to VIAF identifiers (you may have noticed these appearing on people's pages in Wikipedia, e.g. Mary J. Rathbun's Wikipedia page lists her VIAF as 61796012).

To use the service follow the instructions in the earlier post but add the service:

http://iphylo.org/~rpage/phyloinformatics/services/reconciliation_viaf.php

This service is fairly crude, in particular, I make no attempt to score the matches that VIAR returns because this would require parsing and normalising author names. This could be added if needed. If you want some exmaple names to try, here are some taxonomists:

George A Boulenger
G A Boulenger
Wilhelm Michaelsen
W Michaelsen
Colin Campbell Sanborn
Suzanne Hand
Philip Hershkovitz
Yehudah Leopold Werner
W B Spencer
Norman Platnick

A decadal view of biodiversity informatics: challenges and priorities

BMC Ecology has published Alex Hardisty and Dave Roberts' white paper on biodiversity informatics:

Hardisty, A., & Roberts, D. (2013). A decadal view of biodiversity informatics: challenges and priorities. BMC Ecology, 13(1), 16. doi:10.1186/1472-6785-13-16

Here are their 12 recommendations (with some comments of my own):

1. Open Data, should be normal practice and should embody the principles of being accessible, assessable, intelligible and usable.
   
   Seems obvious, but data providers are often reluctant to open "their" data up for reuse.
2. Data encoding should allow analysis across multiple scales, e.g. from nanometers to planet-wide and from fractions of a second to millions of years, and such encoding schemes need to be developed. Individual data sets will have application over a small fraction of these scales, but the encoding schema needs to facilitate the integration of various data sets in a single analytical structure.
   
   No I don't know what this means either, but I'm guessing that it's relevant if we want to attempt this: doi:10.1038/493295a
3. Infrastructure projects should devote significant resources to market the service they develop, specifically to attract users from outside the project-funded community, and ideally in significant numbers. To make such an investment effective, projects should release their service early and update often, in response to user feedback.
   
   Put simply, make something that is both useful and easy to use. Simples.
4. Build a complete list of currently used taxon names with a statement of their interrelationships (e.g. this is a spelling variation; this is a synonym; etc.). This is a much simpler challenge than building a list of valid names, and an essential pre-requisite.
   
   One of the simplest tasks, first tackled successfully by uBio, now moribund. The Global Names project seems stalled, intent on drowning in acronym soup (GNA,GNI,GNUB, GNITE).
5. Attach a Persistent Identifier (PID) to every resource so that they can be linked to one another. Part of the PID should be a common syntactic structure, such as ‘DOI: ...’ so that any instance can be simply found in a free-text search.
   
   DOIs have won the identifier wars, and everything citable (publications, figures, datasets) is acquiring one. The mistake to avoid is forgetting that identifiers need services built on top of them (see http://labs.crossref.org/ for some DOI-related tools). The core service we need is reverse lookup: given this thing (publication, specimen, etc.) what is its identifier?
6. Implement a system of author identifiers so that the individual contributing a resource can be identified. This, in combination with the PID (above), will allow the computation of the impact of any contribution and the provenance of any resource.
   
   This is a solved problem, assuming ORCID continues to gain momentum. For past authors VIAF has identifiers (which are being incorporated into Wikipedia).
7. Make use of trusted third-party authentication measures so that users can easily work with multiple resources without having to log into each one separately.
   
   Again, a solved problem. People routinely use third parties such as Google and Facebook for this purpose.
8. Build a repository for classifications (classification bank) that will allow, in combination with the list of taxonomic names, automatic construction of taxonomies to close gaps in coverage.
   
   Let's not, let's focus on the only two classifications that actually matter because they are linked to data, namely GBIF and NCBI. If we want one classification to coalesce around make it GBIF (NCBI will grow anyway).
9. Develop a single portal for currently accepted names - one of the priority requirements for most users.
   
   Yup, still haven't got this, we clearly didn't get the memo about point 3.
10. Standards and tools are needed to structure data into a linked format by using the potential of vocabularies and ontologies for all biodiversity facets, including: taxonomy, environmental factors, ecosystem functioning and services, and data streams like DNA (up to genomics).
    
    The most successful vocabulary we've come up with (Darwin Core) is essentially an agreed way to label columns in Excel spreadsheets. I've argued elsewhere that focussing on vocabularies and ontologies distracts from the real prerequisite for linking stuff together, namely reusable identifiers (see 5). No point developing labels for links if you don't have the links.
11. Mechanisms to evaluate data quality and fitness-for-purpose are required.
    
    Our data is inaccurate and full of holes, and we lack decent tools for visualising and fixing this (hence my interest in putting the GBIF classification into GitHub).
12. A next-generation infrastructure is needed to manage ever-increasing amounts of observational data.
    
    Not our problem, see doi:10.1038/nature11875 (by which I mean lots of people need massive storage, so it will be solved)

Food for thought. I suspect we will see the gaggle of biodiversity informatics projects will seek to align themselves with some of these goals, carving up the territory. Sadly, we have yet to find a way to coalesce critical mass around tackling these challenges. It's a cliché, but I can't help thinking "what would Google do?" or, more, precisely, "what would a Google of biodiversity look like?"

BioNames update - taxon concepts

Quick notes on "taxon concepts". In order to navigate through taxon names I plan to have at least one taxonomic classification in BioNames. GBIF makes the most sense at this stage.


The model I'm adopting is that the classification is a graph where nodes have the id used by the external database (in this case GBIF). Each node has one or more names attached, and where possible the names are linked to the original description. Where we have synonyms it would be nice to link the synonymy to publication(s) that proposed that relationship.

Time to put taxonomy into GitHub

Donald Hobern drew my attention to nice the way iNaturalist displays taxonomic splits:


In this example, observations identified as Rhipidura fuliginosa are being split into Rhipidura fuliginosa and Rhipidura albiscapa. This immediately reminds me of the idea which keeps circulating around, namely using version control tools to manage taxonomic classification. Some years ago David Shorthouse proposed managing taxonomic classifications using version control, see Taxonomic Consensus as Software Creation. I discussed this in Taxonomy on a hard disk, and Pierre Lindenbaum has an interesting post on treating the NCBI taxonomy as a file system A FUSE-based filesystem reproducing the NCBI Taxonomy hierarchy.

The idea is that a taxonomy, such as the GBIF backbone taxonomy, could be placed in GitHub where people could clone it, annotated, correct, edit, or otherwise mess with it, then GBIF could pull in those edits and release an updated, cleaner taxonomy. If software version control seems a bit esoteric, it's worth noting that use of GitHub is rapidly becoming much more mainstream in science, and not just for software development. People are using it to store versions of data analysis (e.g., https://github.com/dwinter/Fungal-Foray) and collaboratively write manuscripts (e.g., https://github.com/weecology/data-sharing-paper). The journal eLIFE is depositing articles there (e.g., https://github.com/elifesciences/elife-articles). In addition to all the infrastructure GitHub provides (the ability to identify who did what and when, to roll back changes, to fork classifications, etc.) there is also the attraction of not creating yet more software, but simply editing a classification by moving folders around on your local filesystem. The idea seems irresistible…

DNA QR Codes

Best. Flowchart. EVER! #ias13twitter.com/jcolman/status…

— Jonathon Colman (@jcolman) April 7, 2013

Came across this paper recently:

Liu, C., Shi, L., Xu, X., Li, H., Xing, H., Liang, D., Jiang, K., et al. (2012). DNA Barcode Goes Two-Dimensions: DNA QR Code Web Server. (R. DeSalle, Ed.)PLoS ONE, 7(5), e35146. doi:10.1371/journal.pone.0035146

Despite QR Codes being uncool, there's something appealing about the idea of compressing a DNA barcode sequence into a small image. Imagine having a specimen label with a QR Code, pointing a smart phone at the label using an app that converts the QR Code to a sequence, sends it to BLAST and returns a phylogeny that includes DNA from that specimen (perhaps using a service like http://iphylo.org/~rpage/phyloinformatics/blast).

Notes on font size

I'm working on displaying OCR text from BHL using SVG, and these are just some quick notes on font size. Specifically how SVG font size corresponds to the size of letters, and how you work out what point size was used to print text on a BHL page.

SVG font-size corresponds to the EM square of the font. Hence, if I specify a font-size of 100px then text looks like this (you'll need need a browser that supports SVG to see this):


The yellow box is the EM square (in this example 100px by 100px). The height of the letter "M" is set by the properties of the font which in this case is Times-Roman which has a capheight of 662. This value (and others) are defined in the font description file (Adobe-Core35_AFMs-314.tar.gz).

Below is a diagram showing attributes of Times Roman with respect to the 1000 x 1000 EM square:


Couple of things to note. The first is that the height of a digital font is not given by simply adding the capheight and descender, the height of the font is the EM square. If you know the capheight and the font metrics you can compute the size of the EM square (for Times Roman capheight / 0.662 gives you the EM square). Hence it is possible to fairly accurately reproduce printed text in SVG. I had hopes that I could then go on to infer the actual point size used on the printed page (being able to say "this is 10pt" seemed more elegant than this font is "x pixels"). Turns out that "point size" is a terribly elusive concept, see Point Size and the Em Square: Not What People Think. I've clearly lots to learn about typography. BHL would be a gold mine for anyone interested in the development of type faces and printing technology over time.

Towards DOIs for Biodiversity Heritage Library articles

The new look Biodiversity Heritage Library includes articles extracted from BioStor, which is a step forwards in making the "legacy" biodiversity literature more accessible. But we still have some way to go. In particular the articles lack the obvious decoration of a modern article, the DOI. Consequently these articles still live in a twilight zone where they are cited in the literature but not linked to. DOIs are becoming more common for taxonomic articles. Zookeys has them, and now Zootaxa has adopted them (and will be applying them retrospectively to thousands of already published articles). Major archives of back issues digitised by Taylor and Francis, and Wiley, for example, also have DOIs.

One obstacle to assigning CrossRef DOIs to articles in BHL is the convention that DOIs are typically managed by the publisher of the journal. But in a number of cases the publisher may no longer exist, the journal may no longer be published, or the publisher may lack the commercial resources to support DOIs. In these cases perhaps BHL could adopt the role of publisher?

Another approach is that adopted by a number of other digital archives, whereby the archive assigns DOIs to articles, but these DOIs are registered not through CrossRef but with another DOI registration agency, such as DataCite. For example the Swiss Electronic Academic Library Service (SEALS) archive assigns DOIs to individual articles, such as http://dx.doi.org/10.5169/seals-88913.

There are some limitations to not using CrossRef DOIs, in particular, you don't get the full benefits of their metadata-based services such as getting metadata from a DOI, discovering DOIs from metadata, or citation linking. But all is not lost. Some services support both CrossRef and DataCite DOIs, such as http://crosscite.org/citeproc. For example, for the DOI 10.5169/seals-88913 we get some basic formatting:

Perret, Jean-Luc. (1961). Etudes herpétologiques africaines III. Société Neuchâteloise des Sciences Naturelles. doi:10.5169/seals-88913

This still leaves us lacking some services, such as finding DOIs for articles cited in a manuscript. However this is a service we can provide, and will have to anyway if we want to find all the digitised literature available (e.g., archives such as SEALS as well as numerous instances of DSpace). My preference would be for CrossRef DOIs, but if that proves problematics we can still get much of the functionality we need using other DOI providers.

BioNames update - matching taxon names to classifications

On eof the things BioNames will need to do is match taxon names to classifications. For example, if I want to display a taxonomic hierarchy for the user to browse through the names, then I need a map between the taxon names that I've collected and one or more classifications. The approach I'm taking is to match strings, wherever possible using both the name and taxon authority. In many cases this is straightforward, especially if there is only one taxon with a name. But often we have cases where the same name has been used more than once for different taxa. For example, here is what ION has for the name "Nystactes".

Nystactes Bohlke	2735131
Nystactes	2787598
Nystactes Gloger 1827	4888093
Nystactes Kaup 1829	4888094

If I want to map these names to GBIF then these are corresponding taxa with the name "Nystactes":

Nystactes Böhlke, 1957	2403398
Nystactes Gloger, 1827	2475109
Nystactes Kaup, 1829	3239722

Clearly the names are almost identical, but there are enough little differences (presence or absence of comma, "o" versus "ö") to make things interesting. To make the mapping I construct a bipartite graph where the nodes are taxon names, divided into two sets based on which database they came from. I then connect the nodes of the graph by edges, weighted by how similar the names are. For example, here is the graph for "Nystactes" (displayed using Google images:

I then compute the maximum weighted bipartite matching using a C++ program I wrote. This matching corresponds to the solid lines in the graph above.

In this way we can make a sensible guess as to how names in the two databases relate to one another.

BioNames update - API documentation

One of the fun things about developing web sites is learning new tricks, tools, and techniques. Typically I hack away on my MacBook, and when something seems vaguely usable I stick it on a web server. For BioNames things need to be a little more formalised, especially as I'm collaborating with another developer (Ryan Schenk). Ryan is focussing on the front end, I'm working on the data (harvesting, cleaning, storing).

In most projects I've worked on the code to talk to the database and the code to display results have been the same, it was ugly but it got things. For this project these two aspects have to be much more cleaning separated so that Ryan and I can work independently. One way to do this is to have a well-defined API that Ryan can develop against. This means I can hide the sometimes messy details of how to communicate with the data, and Ryan doesn't need to worry about how to get access to the data.

Nice idea, but to be workable it requires that the API is documented (if it's just me then the documentation is in my head). Documentation is a pain, and it is easy for it to get out of sync with the code such that what the docs say an API does and what it actually does are two separate things (sound familiar?). What would be great is a tool that enables you to write the API documentation, and make that "live" so that the API output can be tested against. In other words, a tool like apiary.io.

Apiary.io is free, very slick, and comes with GitHUb integration. I've started to document the BioNames API at http://docs.bionames.apiary.io/. These documents are "live" in that you can try out the API and get live results from the BioNames database.

I'm sure this is all old news to real software developers (as opposed to people like me who know just enough to get themselves into trouble), but it's quite liberating to start with the API first before worrying about what the web site will look like.