The week between Christmas and New Years is mostly time off for me– I’ve added no new listings to The Online Books Page this past week, for instance– but even on vacation, as long as I have a working Internet connection I still tend to fix bad links as I hear about them from readers’ reports.  I try to draw from a variety of free online book sources, instead of just a few big ones; that’s worthwhile to me because it increases the diversity of titles and editions on the site.  But the tradeoff is that many of these sites disappear, reorganize, or otherwise have links go bad over time.  I’m grateful to my readers for reporting bad links to me, and I can often fix other bad links to the same site when I fix the one reported to me.

The links and sites that persist, and those that don’t, often aren’t the ones you might expect.  Who’d have thought, for instance, that a shoestring-budget project that didn’t even maintain its own website until fairly recently would have the longest-lived (and still one of the largest) electronic book collections in common use, outlasting many better-funded or more systematically planned projects (as well as its own doggedly persistent original champion)?  Although the links to Project Gutenberg’s ebooks have changed over the years, the persistence of their etext numbers, and the proliferation of Gutenberg sites and mirrors, has made it relatively easy for me to keep links working for their more than 40,000 ebooks.

Some library-sponsored sites use persistent link redirection technologies, such as PURLs, to keep their links working.  But technology alone isn’t sufficient for persistence.  I recently had to update all of my links going to a PURL-based library consortium site.  I’m sure the people who worked at the organization hosting the site would have kept the links working if they could, but the organization itself was defunded by the state, and its functions were taken over by a new agency that didn’t preserve the links.

Fortunately, the failure had a couple of graceful aspects that eased recovery.  First of all, the old links didn’t stop working altogether, but redirected to the front page of a digital repository in which people could search for the titles they were looking for.  Second, the libraries in the consortium still maintained their own websites, and the old links included a serial number unique to each text (similar to Gutenberg’s etext numbers) that was also used by member libraries.  I found that in most cases I could automatically rewrite my links, using that serial number, so that they would point to a copy at a contributing library’s website.  This made it easier for me to rewrite my links, even though they go to new sites, than it’s often been for me to update links to sites that persist but reorganize.   (For instance, I’ve seen sites change to new content management systems that used completely different URLs from their old design, and then had to manually relocate and verify each link one at a time.)

Sometimes I have to replace links that still “work”, technically.  I used to have thousands of links to a Canadian consortium that provided free access to scanned public domain books and pamphlets from that country’s history.  Not long ago, I discovered that while my links still work, the site had gone to a subscription model where readers have to pay for access beyond the first dozen or so pages of each text. Given the precarious state of Canadian library funding, I’m sure the people running the site were simply doing what they thought necessary to ensure the persistence of the sponsoring organization (which continues to provide new electronic texts and services).  Personally, however, I was more concerned about the persistence of free access to the digitized texts I’d pointed to.  Fortunately, a number of the consortium’s member libraries had also uploaded copies of their scans to the Internet Archive, using the same serial numbers used on the Canadian consortium’s website.  As a result, I was able to quickly update most of my links to point to the Internet Archive’s copies.  I intend to track down working alternative links to the 200 or so remaining texts, or post requests seeking other copies of these texts, when time permits.  (I’ve also sent along a donation to the Internet Archive, in part to thank them for continuing to provide access to texts like these.)

It’s been said in digital library literature that persistence of identifiers is more a matter of policy than technology.  Based on the experiences I’ve related above, the practical persistence of links is even more a matter of will than of policy: the will (and ability) to keep maintaining access through changing conditions; the willingness to consider alternatives to specific organizational structures or policies if the original ones turn out not to be tenable; the willingness to pick things up again, or let others pick them up, after a failure.

It’s also clear from my experience that practically speaking, failure is not the main enemy of persistence.   More of a threat is not recovering from failure, or being so worried about failure that one doesn’t even begin to sustain the thing or the purpose that should persist.  To riff off a famous G. K. Chesterton quote, if it’s worth doing something, it’s worth being willing and ready to fail at doing it.  And then, to be willing to pick up again where you left off, or to make it easy for someone else to pick it up, and try something new.

That’s persistence.  That’s what’s ultimately gotten the dissertation rewritten, the estates settled, the blog picked up again, the books put and kept online for the world to read, and many other things I’ve found worthwhile, despite difficulties, anxieties, and setbacks.  I value that persistence, and I hope you value it as well, for the things you find worthwhile. I look forward to seeing where it takes us in the year to come.

Respecting failure: Some thoughts, and a proposal

Last month’s Digital Library Federation forum involved a number of interesting discussions, both at the conference site and online.  This forum, different from previous ones, centered around discussions of strategies for innovation in libraries.  It also involved discussions of the future of DLF itself, which earlier this year ended its independent existence and was merged into CLIR.

One theme that quickly emerged in discussions was the importance of failure.  It’s a topic we often feel uncomfortable discussing, especially when we had a hand in whatever failed.  Part of the discomfort in the digital library community has to do with the dual nature of what many of us do: We manage programs and services, and we also try to innovate.  As managers, we don’t want our programs to fail.  (If that happens anyway, we’d at least like to avoid being blamed for the failure.)  And libraries have long-term ongoing service and preservation obligations that make certain kinds of catastrophic failure unacceptable.

But as innovators, we want to be open to failure as a way of learning.  “Fail faster!” is a common slogan of innovative labs and ventures, and knowing the “thousands of ways that don’t work” (part of a quote often attributed to Thomas Edison) help us better understand the ways that do.  I’ve mentioned before that my most widely-cited paper was written about the failure of a software development project I helped work on.  And a new scientific theory isn’t usually worth considering until it is capable of failure– that is, it makes definite predictions that subsequent observations can either confirm or refute.

If we are really serious about innovating, we need to respect failure, and leave room for it.  We need to let people try things that might not work, allow time for encountering dead ends, have contingency plans that let us continue to carry out our missions even as failures occur, and note both what worked and what didn’t in the things we try.  It’s especially useful to note things that we found didn’t work before they were obvious to others, since we might well save others a lot of time avoiding the same pitfalls.

How should these failures be reported, though?  It’s often easier and more gratifying to publish stories of success than stories of failure.  And if we talk about our failures in public, whether in a journal, at a conference of like-minded professionals, or even in a blog or tweet, we may have good reason to worry that it might hurt our own positions in our organizations, or the work that we do.

The question was raised at the DLF Forum whether future forums could be a “safe” place to discuss failures.  I’m not sure any public gathering can be an entirely safe place for that.  People you see are identifiable, and word tends to spread over time, particularly when a large number of people are present.

One alternative that’s been proposed to address this problem, and still make useful information about failure available to a wide audience, is to anonymize reports.  We can still learn a lot from failures in our field even if we don’t know exactly who failed and where.  And perhaps anonymity can produce more useful information about failure, by producing more “safe” places to talk about it.

So, I’d like to try a little experiment.  From now until the end of February, I invite folks involved in library-related failures to send me reports of their failures, for possible publication in this blog.  I will choose which ones to publish (and may or may not request revisions), but whether or not they get published, I will do my best not to disclose your identity.  (I can’t absolutely prevent email hacks or subpoenas, but I consider them unlikely.)  Please clearly note the start and end of the report you’re submitting for publication; I’ll assume any information that might help identify you or your organization in the middle of the report is information that you’re comfortable disclosing.  Reports should be sent to the email address shown on this page.

I’m interested in particular in failures of initiatives you were personally involved with, and what can be learned from the failures. While others may also be involved in the failures, I’m not particularly interested in reports intended primarily to focus blame on a particular third party.  You should use similar care to obscure the identities of others involved as you do in obscuring your own identity.

Some folks may feel more comfortable working with someone else, or under different selection criteria than the ones I use.  So I invite others to make similar offers if they see fit (and I might mention them here if I hear about any similar offers).

Will this produce something useful to the library community?   I don’t know yet, but it seems worth risking a bit of failure to find out.   If you have any suggestions about the proposal, or have some ideas of kinds of reports you’d like to see, feel free to mention them in the comments.  And if you have a report you’d like to make, send me email.