Public Domain Day advent calendar #2: ‘Twas the Night Before Christmas Cantata by Frances McCollin

Posted on December 2, 2018 by John Mark Ockerbloom

Today marks the start of the Christian season of Advent in my church, four Sundays before Christmas. It’s an important time of the year for vocal music, as choirs both sacred and secular rehearse and perform Christmas-related music in services and concerts.  We like singing public domain classics– those have stood the test of time and are often well-known to our audiences, and we can copy scores cheaply and don’t have to worry about performance rights or royalties.  We also like to sing newer copyrighted works– those can be fresh and exciting, and can often be easily found in print and in recordings.  Older works still under copyright, though, can often fade into obscurity, even when they’re by creators who were once well-known.

Today’s Public Domain Day advent calendar entry falls into that category.   Though Frances McCollin had a long career in Philadelphia, where I’ve lived and sung for nearly 20 years, I don’t recall hearing of her before doing research for this calendar. Blind from an early age, she took a synesthetic approach to music, according to her biography Frances McCollin: Her Life and Music (1990), associating various keys with different colors and moods.  She published nearly 100 pieces of music, and her works have been performed by groups that include the Philadelphia Orchestra and the Warsaw Symphony.  The library where I work has some of her papers in its special collections; the Free Library of Philadelphia has others. Some of her early public domain music is now freely available online, such as “The Singing Leaves” (1918).  And some of it is still performed today. I’ve found online a sample of a recent recording by Canada’s Elektra Women’s Choir of her arrangement of “In the Bleak Midwinter” for women’s voices.  (McCollin died in 1960, so most of her published work is in the public domain in Canada. I’m not sure of the US copyright status of this particular composition.)

I had a much harder time, however finding traces of another Christmas vocal piece she was known for in her day, a children’s chorale setting of “‘Twas the Night Before Christmas”.  Based on the famous poem by Clement Clarke Moore (which, published 100 years earlier, had been in the public domain for her to adapt), her cantata was copyrighted in 1923, and renewed in 1950.  That copyright persists to this day, as the piece has faded into obscurity.  But 30 days from now, the work will enter the public domain in the US, and anyone interested in it can digitize it, make copies, perform it, or create new works based on it, without having to find a rightsholder to ask permission or pay royalties.

As Public Domain Day approaches, I’m looking forward to seeing familiar works join the public domain in the US, like the perennial international best-seller featured in yesterday’s calendar post.  But for every work like that, there are many more works that are less well-known, on all kinds of topics of interest, by people who lived all over, including in one’s own hometown.   They may have been largely out of sight for years, but their entry into the public domain gives them a new opportunity to be discovered, enjoyed, and shared by anyone.

If you know of any other works that will be coming into the public domain this January you’d like to share, I’d love to hear about them.  I’ll post about another one tomorrow.

 

Posted in adventcalendar | Comments Off on Public Domain Day advent calendar #2: ‘Twas the Night Before Christmas Cantata by Frances McCollin

Public Domain Day advent calendar #1: The Prophet by Kahlil Gibran

Posted on December 1, 2018 by John Mark Ockerbloom

“Your children are not your children […]
For they have their own thoughts […]
You are the bows from which your children as living arrows are sent forth.”

The lines above are from one of the more well-known parts of The Prophet, written by Lebanese-American poet and artist Kahlil Gibran.  Though they talk about one’s offspring, one could also see them as applying to one’s creations.  They may originate from people (parents or authors) who start with great control and influence over them, but eventually they become independent of their origin.  To quote another line in the poem, they “dwell in the house of tomorrow, which you cannot visit”.

Since The Prophet was first published in 1923, it has to a large extent been under the control and influence of its author, and then, after his death in 1931, his estate.  But over time it has entered the public domain in various countries around the world.  It entered the public domain in many countries at the start of 1982, after 50 years had passed from the author’s death.  It entered (and in some cases, re-entered) the public domain in many others in 2002, more than 70 years after his death.  And finally, 31 days from now, it will enter the public domain in the United States, as part of the first batch of published works to enter the public domain in more than 20 years.

During the month of December, this blog will feature various works from 1923 that will be joining the public domain in the US this coming January 1, Public Domain Day.  The Prophet is fairly well-known and still easy to find in print.  Many other interesting works from 1923 are not so well-known or easy to find, and I hope to feature a wide variety of works over the next 31 days.  (I already have some works planned to feature, but have not yet filled out a full roster; if there are any in particular you’d like to suggest, let me know by commenting here or by contacting me.)

I plan to keep most of the posts short, but I’ll have links to more information on the works and authors featured, and folks are also welcome to discuss them further here in the comments if they wish.  Come the new year, I’ll also go back and add live links to online copies of the works when possible.  (For now, I’ll just add a link to all online books I know of by Gibran, some of which are already public domain in the US, and some of which aren’t yet but are public domain elsewhere.)

I may also sometimes take the opportunity to point out aspects of copyright relevant to the featured work.  For instance, in this post I’ve directly copied material from Gibran’s book (by quoting from it) even though I’ve noted that it’s still under copyright where I am.  I can do this thanks to the principle of fair use, which allows such copying under certain circumstances. There’s no universal algorithm for determining whether fair use applies in a particular case, but since I’m only quoting a small portion, in a noncommercial educational context, in order to provide original commentary and not to provide a substitute for the original work, a fair use defense would be very likely to prevail if a copyright dispute came up.  Once The Prophet enters the public domain here, though, we’ll be able to use larger portions, or the whole work, for a much wider variety of purposes.

I’ll post my next Public Domain Day advent calendar entry tomorrow.  I hope you’ll come back then to read it, and welcome your comments.

2019 update:  Link to the full text of The Prophet, now in the US public domain, courtesy Gutenberg Australia
 

 

Posted in adventcalendar, copyright | Comments Off on Public Domain Day advent calendar #1: The Prophet by Kahlil Gibran

Why pay for what’s free? Finding open access and public domain articles

Posted on October 23, 2018 by John Mark Ockerbloom

[Another version of this post appears on PennLibNews, as part of the Penn Libraries’ celebration of Open Access Week.]

It happens to most researchers all too often.  You see a reference or a link to an article that you’d love to read or use in your research.  It looks like the article is online.  But when you follow the link or look up the journal of venue it’s in, you’re confronted with a paywall— an impassable login screen, or a demand to pay a fee to read the article.  Not all journals charge fees to readers (in fact, thousands do not), but for those that do, the fees are sometimes quite large– they can be $30 or $40 or more for a single article, and sometimes that fee only gets you access for a limited time period.  Readers not affiliated with research institutions often run into paywalls, but so do scholars at elite research institutions, because none of us can afford to pay the ever-growing subscription prices of all the commercial journals on the market.  (Not even Harvard.)

When you hit a paywall, it’s tempting to give up, look for other articles instead, or take your chances trying to get an illicit copy from sketchy bootleg sites.  But there are various ways you can often get a legitimate version of the article you seek without having to pay anything.  Here are some avenues you can look into.

There may be a legitimate free copy posted elsewhere online.

The website of a journal is often not the only place you can read the papers in it.  Many authors also put their own copies of their papers online.  There are various ways this can be done legally. Many journals and publishers allow authors to self-archive their papers on their personal or institution’s web sites, and sometimes elsewhere.  (Authors might also negotiate this right when they contract with a publisher.)  Typically the posted copies will not have the typesetting and formatting of the journal copies, but instead be a “preprint”– the version of the paper submitted to the journal– or a “postprint”– the paper after it’s been peer-reviewed and revised, but before it’s been reformatted with the journal’s customary typesetting and layout.  There may also be an earlier iteration of a paper freely available online, such as a version presented at a conference or a working paper at an eprint site like ArXiv.org, SocArXiv, or Humanities Commons CORE.

How do you find these free versions of articles?  Many of them can be found in Google Scholar, by typing in the title of the paper and the name of the lead author. (Sometimes earlier versions of the paper will have different titles, so you might also want to try typing in the names of the authors and some keywords, or a characteristic phrase from the abstract or introduction if you have access to it.)  Google Scholar searches will often turn up both the publisher’s version and other versions that might be freely readable or downloadable.  (Look for an “all versions” link by the search results; selecting that link will often show you the other versions.)

Unpaywall is another important tool for finding free articles.  Its database of free alternatives to published articles isn’t as big as Google Scholar’s, but it’s still quite large– over 20 million to date– and it offers a variety of ways of finding articles that Google Scholar doesn’t offer. For instance, they offer a free browser plugin (currently available for Firefox and Chrome) that will inform you of free alternatives when you visit a web page for a paywalled paper.  They also make their knowledge base available via APIs and bulk data downloads, allowing developers to build new applications for finding and using free versions of papers.  (As a result, Unpaywall links are now available in some other bibliographic databases, such as Scopus and Web of Science, that are offered at the Penn Libraries and elsewhere.)

You might be able to get a copy from your library.

If you can’t find a free copy of the article you want on the open web, you may still be able to find a copy of the article through your library’s catalog, or via inter-library loan.

Searching the catalog can often turn up copies you can access for free.   The web sites for many academic libraries, including the Penn Libraries, allow you to search for both journals and articles accessible via the library.  If your library’s web site offers article searching, enter the title and first author’s name of the article you’re interested in– you might find a copy you can access via one of the library’s licensed databases.  If your library just has a traditional catalog search, enter the title of the journal the article appeared in.  The results may give you electronic access to the journal you’re interested in, or it might let you know about print copies the library has that contain the article.  (Many journals still publish a print version.)

If your library doesn’t have the journal you seek in its collection, it might still enable you to get a copy of the article you want via inter-library loan.  These days, inter-library loan can be quite fast, and in many places you can get a copy of the article delivered to you electronically. (Here’s how it works at Penn.) Visit your local librarian to find out more about what they can do for you.

It may be in the public domain.

Just because an article is behind a paywall doesn’t necessarily mean it’s actually copyrighted.  If you’re interested in an older article, it might be in the public domain. Many journal volumes published before 1923 have scans freely available online, and can be found via resources like The Online Books Page.  But many later scholarly publications are public domain as well, including US publications prior to 1964 that did not renew their copyrights, and US publications prior to 1989 that were not published with copyright notices.  Most US scholarly publications before 1964 in fact do not have copyright renewals (though a fair number do); and some more informal or noncommercial scholarly publications lacked copyright notices.  At the Penn Libraries, we’ve been gathering information about periodicals and their copyright status over the course of the 20th century, and have information on many 20th-century journals here.  (If you don’t see the journal that you’re interested in, and it was published in the US between 1923 and 1963, let us know about it and we’ll research it and put information about it in our resource.  We’ll also list whatever public domain issues we can find freely available online.)

Just because an article or journal issue is in the public domain,  there won’t necessarily be a free online copy.  But if you do locate a copy of a public domain article or issue, you’re free to scan it yourself and share it with others.  We hope that our listings of copyright information for various periodicals will prompt libraries and others who own those periodicals to digitize and freely share their public domain contents.

You might be able to get an article from its author(s).

If the article you’re interested in is a recent one, its authors might be reachable online.  They might well have copies of their articles, and be willing to email a copy to a fellow researcher who asks for it.  (Unlike the publishers, authors of scholarly articles generally earn no money from them, so they have little reason to be stingy about access.)  This sort of person-to-person informal, non-systematic sharing has a long tradition in academia, and is generally allowed by publishers (and sometimes explicitly authorized; see for example this IEEE policy, which at this writing says it’s fine to “share copies… for individual personal use”.)

Contact information for authors can often be found by putting their names into search engines, along with their field of research and/or institution if known.

Paying it forward

Hopefully you’ve been able to find and benefit from free, legal copies of scholarly articles using techniques like the ones I’ve described above.  You haven’t had to pay the authors or publishers for them, but if you’ve found them beneficial, you can always pay forward, by sharing your own articles with other readers.  As I noted above, many journals allow authors to post their articles on personal, institutional, or other non-profit websites (though some may require waiting for a time after publication, or only posting certain versions of the article).  Some publishers will also agree to additional author’s rights (such as those specified in the SPARC addendum) if the author asks for them.

Many academic libraries are happy to help you share the articles you write, and provide places to post them so that they’re easily discoverable online, and preserved for posterity.  One of the services the Penn Libraries offers, for instance, is Faculty Assisted Submission, where librarians will go through your CV and post whatever they can to our institutional repository.  You can also send them any further articles you publish when they are accepted, and they will take care of posting them as soon as any required waiting period has expired, without you having to keep track of what you can post when.

I hope these tips help you in freely partaking of scholarly research, and in freely sharing your own.  If you have any questions about doing this, or experiences you’d like to share, feel free to comment here.

 

 

Posted in copyright, libraries, open access, serials, sharing | 9 Comments

Public domain serials: Renewal inventory complete!

Posted on June 15, 2018 by John Mark Ockerbloom

I’m pleased to announce an important milestone in the IMLS-funded project I’m leading to help open access to 20th century public domain serials.  We now have a complete, openly published inventory of all serials with active issue or contribution renewals made through 1977, based on listings in the Copyright Office’s Catalog of Copyright Entries.   For each serial in our inventory, we list the earliest issue’s renewal and the earliest contribution’s renewal made during that time period, if any.  In many cases, we also give additional copyright information, and link to freely readable online issues, tables of contents, and copyright permissions information.

The Copyright Office’s Copyright records catalog includes copyright renewals and other registrations filed from 1978 onward.  Since our inventory now runs all the way up to their catalog’s 1978 starting point, you can now find out if there were any copyright renewals filed for a serial you’re interested in by checking our inventory and the Copyright Office’s database.  (Prior to now, you might have needed to search through lots of volumes of the Catalog of Copyright Entries before making such a determination.  You can now do it much more quickly.)

We also now have renewal information for over 1,000 of these serials available as structured data, as described in a previous post.  These structured data files are available as individual JSON files (linked off of each serial listed in the inventory with a hyperlink), and also can be downloaded in bulk, along with our overall inventory listing, from our Online Books Page GitHub repository.  There are still some listings that are not yet available as structured data, but they’ll be converted over time, and all new listings will be made available as structured data.

Thanks to this structured data, we can now also offer a more consistent and easy-to-use inventory page.  In particular, all of the title links on the page now go to copyright information.   (Before, some links would go to copyright information and some would go to online copies of the serials themselves, making for a somewhat confusing user experience.  You can still get to online serials via the copyright information pages; if a linked title is shown with emphasis, its copyright information page includes a link to free online content available for that title.)

We’re now done, then, with compiling and publishing the data we said we would compile for our project.  But we’re not done with the project yet as a whole.  Here’s what’s still to come:

  • Suggested procedures for using the data we’ve compiled, along with other data, to quickly identify and check public domain serial content.  We’re drafting those procedures now, and hope to publish our draft after we’ve had some experts in copyright and digitization projects review it.  In the meantime, if you want to use the data we’ve compiled to clear copyrights for serials, keep the caveats we describe at the top of our inventory in mind.
  • Documentation for the JSON files we’re using for our copyright data.  The fields and format we’re using are still subject to change, but probably won’t change all that much from what they look like now.  (I might make them JSON-LD files eventually, but hope that I can do that in a backward-compatible way.)  In the meantime, feel free to contact me with questions about the fields I’m using in those files and what they mean.
  • Examples of public domain serials published after 1922 whose copyright status has been cleared with our recommended procedures and data, along with explanations on how we cleared them.

There are also ways you can help out with this project.  Along with the things you can do that we suggested in our earlier post, here are a few new things that we can now help with:

  • If you’re interested in a serial that published between 1923 and 1964 that’s not already in our list, tell us about it.  (You can use use this suggestion form.  All you need to put in it is the title, and whatever is needed to distinguish it from similarly-titled works.)  We can quickly add it to our inventory.  If we find any associated renewals in the Copyright Office’s database, we’ll note the first one.  If we don’t find any associated renewals, we’ll note that.  And if we can quickly find online public domain issues, we’ll link to them.
  • If you’re interested in compiling more comprehensive or details information about a particular serial, you can get in touch with us and we can work with you to get an enhanced JSON file created for it.  For an idea of what can be done, check our our information page for Amazing Stories and its associated JSON file.  We don’t have time ourselves to create these sort of comprehensive pages for all of the serials out there, but we’d love to work with people with the time, interest and skills to create such information pages for serials they’re interested in.

I’m excited about where we’ve gotten so far, and what can be done with this data.  I’d love to hear from you about what you’d like to do with it, and how you might like to extend it.

Posted in copyright, metadata, open access, serials, sharing | 3 Comments

Public Domain Day 2018: The 20-year alarm clock

Posted on January 1, 2018 by John Mark Ockerbloom

In Washington Irving’s classic story “Rip van Winkle“, the title character follows an archaically-dressed stranger into a mountain hideaway, falls asleep, and wakes up to find the world has moved on 20 years without him.  He’s alarmed at first, but eventually figures out what has happened, adapts, and settles into the social fabric of his much-changed town.

We in the US are experiencing a similar phenomenon this Public Domain Day.  It’s now been 20 years since a full year’s worth of published content entered the public domain, when 1922 copyrights expired at the start of 1998.  Later that year, Congress passed the Sonny Bono Copyright Term Extension Act, and the advance of the public domain has been stuck in the Warren G. Harding administration ever since.

Other countries are doing somewhat better.  Most European countries have by now regained the 20 years of public domain many of them lost when they complied with the EU Copyright Duration Directive, and they now get to freely share the works of creators who died in 1947– people like Baroness Orczy, Alfred North Whitehead, Pierre Bonnard, and Willa Cather.  (We in the US get any unpublished works by people in this group, but that’s all that’s entering the public domain here today.)  Countries like Canada that kept to the Berne Convention’s “life plus 50 years” terms are doing substantially better– today they get the works of creators who died in 1967, including Carson McCullers, Arthur Ransome, Woody Guthrie, René Magritte, and Dorothy Parker.  (The Public Domain Review’s Class of 2018 article has writeups of some these authors, and others.)

This time next year, though, the US may well get to join the party in a bigger way, and have all copyrighted 1923 publications finally enter the public domain.  It’s not a sure thing, though– lobbyists for the entertainment industry have long pressed Congress for longer copyright terms, and while there’s no bill I’m aware of that’s been introduced to do it next year, that doesn’t mean that one couldn’t be quickly rammed through.  So it’s a good time for you to let your elected representatives know that you value a robust public domain, and want no further copyright extensions.  (And if you’re hoping to elect someone new in 2018, let your candidates know this matters to you as well.)

There are other ways that those of us in the US can prepare Public Domain Wake-Up party next year.  Some of us are continuing to bring the “hidden public domain” to light over the next year.  Researchers with HathiTrust’s Copyright Review Management System have by now found over 300,000 books and other monographs that are non-obviously in the public domain.  (You may be able to help them!)  The project I’m leading to help identify public domain serial content past 1922 is also underway, and should be complete later this year.  (See an earlier post for some ways you can help with that if you like.)

Other folks are working on another set of works that libraries in the US can now share– certain works in the last 20 years of copyright, which as of today span a full 20 years of publication, from 1923 to 1942.  A provision in the 1998 Copyright Term Extension Act, codified in Section 108(h) of US copyright law, allows non-profit libraries and archives to digitally display and distribute such works when they are not being commercially exploited, and copies are not available at a reasonable price.  Libraries have not made much explicit use of this provision until recently, partly due to uncertainty about when and how it applies.  A preprint article by Elizabeth Townsend Gard aims to clear up this uncertainty and spur libraries to make these works more widely available.  I hope that this work is further developed and applied in the coming year.  (And I’m happy to consider works in these last 20 years of copyright for inclusion in The Online Books Page‘s listings, when the libraries that have digital copies make it clear that they’re following the section 108(h) guidelines.)

I also hope some folks take some time this year to take a good look at what we’ve digitized from 1922, and compare it to what was copyrighted that year in the US.  What portion of 1922’s creative output have we brought to light online?   What sorts of works, and what people, have we tended to miss?  Knowing where the gaps are can tell us what we might want to focus on bringing to light from 1923.  Some of that material can be posted online now; hopefully the rest can be posted starting next January.

Finally, it’s worth remembering that you don’t have to wait for copyrights to expire on their own to share work whose copyright you control.  You can open-license them any time you like (this post, along with many of my other works, is CC-BY licensed, for instance).  And you can let them go entirely if you like.  I tend to put my works into the public domain 14 years after publication, following the original copyright term of the US, if I see no need to extend it further.  Today I do that again, and with this post you can consider anything I published in 2003, whose copyright I still control, to have a CC0 dedication to the public domain.

Happy Public Domain Day to everyone who’s seeing new works in the public domain where they are.  And here’s to waking up to lots of new additions to the public domain in the US a year from now.

Posted in copyright, online books, open access, serials, sharing | 1 Comment

Adding more, and more structured, information about public domain serials

Posted on December 11, 2017 by John Mark Ockerbloom

I’ve been happy to hear from a number of people and institutions interested in the IMLS-funded project we now have underway to shed light on the hidden public domain of 20th century serials that I discussed in my last post.  I gave a short 6-minute presentation on the project at this fall’s Digital Library Federation Forum, and you can find my presentation slides and script on the Open Science Framework.   (You can download either a Powerpoint file or a PDF as you prefer; both have the slides and the notes.)

With the help of Alison Miner and Carly Sewell, I’m now starting to add data to the inventory of serials that is one of the deliverables of the project.  Right now, we’re putting up data from 1966 renewals, where serial contributions from 1938 and 1939 were renewed.  But there’s quite a bit more data in the pipeline, and I hope that we’ll have all of the 1930s covered by the end of this month, and then advance rapidly into the 1940s in the new year.  (Our goal is to get up to 1978’s renewals, which will finish off the 1940s and get to 1950, and from there on the Copyright Office’s online database can be consulted for serial renewals.  We’re aiming to have that completed sometime in the spring.)

I’ve heard from various people who are interested in clearing copyrights for their own serials digitization projects– as well as some projects that are doing it already, like the Hevelin Fanzines project that was also discussed at the DLF Forum.  As I mentioned in my previous post, we intend to publish suggested procedures for doing such copyright-clearing.  We’ll be preparing drafts of such procedures in the new year, and we’ll let interested folks know when such drafts are available for comments and suggestions.

We’re also happy to hear suggestions about other aspects of the project.  One suggestion we heard in an early presentation was the inventory should be shared as downloadable structured data, and not just as a big web page, so that it would be easier for people to repurpose and automatically analyze the data or various purposes.  That sounded like a good idea to me, and I got more excited about its possibilities when looking at all the work that people were doing with projects like the FictionMags project, ISFDB and similar projects, where volunteers have crowd-compiled detailed structured contents information for a large number of serials.

So the new data going into the inventory is going in as structured data files, and we’re also slowly refitting existing entries into such files.  That’s meant a bit of a slower startup than we’d originally planned, but we believe this work will pay dividends in the future.  Already it means that we can reuse the same data in multiple contexts– for instance, we can show first-renewal information for Adventure magazine on the Online Books Page issue listings, on a copyright information page, and in the big inventory page.  Updating the underlying structured data file can change what appears in all of these contexts.

Moreover, data structures are expandable.  When readers asked me to list Amazing Stories and Galaxy science fiction magazines on The Online Books Page, I had to determine which parts of their runs were public domain and thus could be listed without any further inquiries into permission.   I looked up copyright renewals for these magazines and then recorded renewal data in the same sorts of structured data files so it could be reused.  (Here, for instance, is the automatically generated copyright information page for Amazing Stories.)  I also added structured data fields that allowed the inclusion of name identifiers, in particular, Library of Congress Name Authorities, so that authors could be consistently identified, and then linked to other information about them, such as contact information for permissions.)  With these links, and with the links to full issue tables of contents compiled by other projects, it becomes easier to digitize nearly any story in the early years of the magazine, by checking to see whether there is still an active copyright on it, and by sending an inquiry for permissions if there is one.

To be clear: I’m not going to compile lists of all renewals for all the serials in my inventory.   That would take more time than I have– the scope of my 1-year grant only covers the first renewals for the serials that have them, up to 1950.  But if I can create more detailed renewal lists for Amazing Stories using the defined structure, then others who are interested could create similarly structured renewal lists for other serials.  And maybe those lists could be linked, shared or distributed from my inventory as well, if there’s interest.

So before I get much further into the project, I’d like to hear from folks who might be interested in using or compiling this sort of detailed renewal information.  Is this sort of structured information useful to you?  And if so, will the format and structure I’ve defined for the data files work?  Or should it change (something that’s easier to do now than later), or be augmented?

I didn’t find any pre-existing schema that covered detailed article-level copyright renewal data, so I decided to roll my own for starters.  There’s a variety of encoding schemes one could use for it, including XML, JSON, and the various RDF formats.  I figured JSON would be easiest for laypeople and librarians to understand and edit in its  native format, and it can be automatically translated into suitable XML or RDF schemes if desired.  But if you know of good reasons for preferring a different native format, or know of schemes I should reuse or extend instead of reinventing the wheel, I’d be interested in hearing about them.  (I’m especially interested if the format or schema is already in common use by the sorts of folks who compile serial contents information.)  Alternatively, if what I have now is as good a starting point as anything else, I’d be happy to know that, and could then take the time to write up formal documentation for it.

To see the existing files I have, you can go to the big inventory page and follow the “More details” links that you’ll see for certain serials in the list.  These lead to copyright information pages for the serials in question, which in turn have links to JSON files at the bottom of each page.  I also have most of the JSON files in a Github folder that’s part of my Online Books Page project repository.

If you work with this sort of metadata, or would like to, I’d love to hear from you.  If we get this right, I hope this data will spark all kinds of useful work opening access to a wide variety of 20th-century serial publications.

 

Posted in copyright, metadata, open access, science fiction, serials, sharing | Comments Off on Adding more, and more structured, information about public domain serials

New IMLS-funded project: Opening access to 20th century public domain serials

Posted on September 13, 2017 by John Mark Ockerbloom

I’m happy to report that over the next year, I and others at Penn will be working on a project that the Institute of Museum and Library Services has just funded to help open access to the vast public domain of 20th century serials.  We’ll be developing and demonstrating data sets and procedures to make it much easier to verify public domain status for content in scholarly journals, magazines, newspapers, newsletters, and special interest periodicals published in the United States.  We hope that all kinds of libraries can take advantage of the resources we provide to make materials like this in their collections available online to all, and digitally preserve them for posterity.

As I’ve noted previously, US publications prior to 1964 had to have their copyrights renewed or they would enter the public domain, and most serial content from that era was not renewed.  Projects like HathiTrust and JSTOR have lots of public domain serial content digitized, but they generally don’t provide access to it past 1922, since it’s difficult to verify the status of the post-1922 volumes they have digitized.  HathiTrust’s Copyright Review Program has investigated and opened access to many post-1922 books, but to date has not moved into serials, where one needs to verify the public domain status not only of the serial itself, but of the individual contributions (articles, stories, etc.) in the serials.  To date, that’s been too complicated to do at scale for them and for many other projects.

We aim to make it simple and practical to do so for many serials.  Here’s how we plan to do it:

  • First, we’ll complete and openly publish online an inventory, long in the works, of all the serials that have an active issue or contribution copyright renewal filed with the US Copyright Office, and the dates of the first such renewals, up to when the Copyright Office’s online registration catalog picks up.
  • Second, with the help of legal experts, we’ll draft suggested procedures for using this inventory, along with other online resources, to quickly identify and check public domain serial content.  We plan to develop procedures along the lines of those described in Michigan’s Finding the Public Domain Toolkit, and to be similarly useful to libraries and other digitizers of serials.
  • Finally, we’ll demonstrate and publicize our procedures and data sets, by using them to copyright-clear and digitize some sample serial content, by publishing our resources and reports online, and by reaching out to librarians and others who can use them.

We’d love to have you join us in this work.  Here are some of the things you might do:

  • Help spread the word about this project.   I’ll be doing a brief presentation about it at the upcoming Digital Library Federation Forum in Pittsburgh, and will post my slides after the event.  I’m happy to talk about it further and answer questions there and anywhere else that’s practical.
  • Give us feedback.  What are the best ways for us to provide our inventories and procedures?  How can we make them easier to understand, use, and (as appropriate) automatically process and repurpose?  What kinds of serials deserve particular attention in our procedures and sample digitizations?  How can we best reach, and get contributions and suggestions from, the people and institutions that could put these resources to good use?
  • Try using the data and procedures, when we have them, to copyright-clear and digitize, public domain serial content you’d like to share with the world.  If you let us know what you’ve done, and how it went, we might be able to use what you tell us to improve what we provide; and we can also publicize your digitizations in places like The Online Books Page.
  • Help enhance the data we have.  Our basic inventory will list all the serials in the time period it covers that have renewals, and list the earliest active renewals.  But many of those serials also have public domain content after those first renewals.  We don’t have time ourselves to list all of those renewals ourselves for all the serials we list, but it’s possible for motivated folks to do so for material they’d like to digitize or see digitized.  I’m very interested in sharing or linking to any such lists that are made for particular serials.  (I hope to share an example list of this sort in an upcoming post.)

I’m excited about this project, and about the content that I hope this project will help make available to the world online.  If you find it of interest as well, I’d love to hear from you.

 

 

Posted in awards, citizen librarians, copyright, metadata, open access, serials, sharing | 2 Comments

Public Domain Day 2017: Keeping memory alive

Posted on January 1, 2017 by John Mark Ockerbloom

It’s a bit hard to believe, but there are now adults in the US who cannot remember a substantial Public Domain Day.  In their own lifetimes, copyrights for published works have never expired here.  But I remember when they expired regularly every New Years Day, up until 1998.  And I have hope that in two more years, if our government does not acquiesce to the entertainment industry and extend copyrights yet again, we’ll start seeing published works again regularly enter the public domain here.

There are other things that I don’t remember, or haven’t seen much of myself.  I wasn’t around for the demagogues who whipped up mobs to usher in fascism and persecution in 1930s Europe, nor for the world war that followed that, nor the beginnings of European integration that helped keep the peace afterwards.  I didn’t personally experience the struggles for civil rights and equality of the 1960s and early 1970s, not being born yet, or being too young to take part in them.  I’ve never been to Russia, Turkey, Hungary, or the Philippines, where recent hopes of democratic and liberal reforms have given way to more authoritarian regimes.  I’m not an expert in the science that shows the world dangerously warming in recent years, how humans have caused and will be affected by it, and what we need to do to stop it.

Yes, all of these things are mentioned in history books and various media, but they’re often discussed in superficial ways that don’t reflect the detailed memories and lessons of the people who experienced them firsthand, or who are experts in understanding the issue.  I think that’s part of why I see our societies now making many of the same sorts of mistakes that people who know of these things firsthand could help us avoid.   I think it’s also part of the shock of many of people I know watching the news of the past year.  They didn’t have the personal memories of how prosperous societies can regress just as easily as they can progress, or hadn’t previously noticed how marginalized people experience those societies very differently than our dominant narratives suggest.

Keeping memory alive, and making it known far and wide, is essential if we are going to solve the problems we have today, and avoid the kinds of mistakes and disasters we’ve had in the past.   The more easily we can duplicate and spread that memory, the more likely we are to keep it alive.  That’s why the Internet Archive is making a backup copy in Canada, just in case anything happens to its primary US copy.  That’s why people at Penn and elsewhere are trying to duplicate all the data the government has on climate and other issues before the administration changes, lest it become unavailable or harder to access in the future.  That’s why projects like Wikipedia go out of their way to allow their content to be copied and readily downloadable in bulk, so it can be read and shared in places where the Internet isn’t as reliable or as uncensored as it is in other places.

And that’s one reason why the public domain is so important, and why it’s so important that copyrighted works enter the public domain regularly, automatically, and in a timely fashion.  Copyrights are important to support the people who create works of art and knowledge, and to help ensure that they can introduce them to the world in the form and manner they intend.  But it’s also important that after “limited times” (to quote the US Constitution) the works enter the public domain, so they can be copied, disseminated, reinterpreted and reworked, and remembered, without restriction.  The easier it is to copy and disseminate, the easier it is to remember.

In 2017, we have works by authors who died in 1946 finally entering the public domain in Europe and many other countries.  That now allows us to freely copy the works of those who perished in World War II and shortly thereafter, but not the works of all those who survived and helped rebuild society afterwards.  (Though it at least finally frees works like 1895’s The Time Machine, written by a young H.G. Wells who would live until 1946.)  We’re also seeing works by authors who died in 1966 entering the public domain in Canada and some other countries, freeing the works of people like the anti-fascist poet Andre Breton, as well as C. S. Forester’s Captain Horatio Hornblower.  (You can read about more authors now in the public domain in “life plus 70 years” and “life plus 50 years” countries in the Public Domain Review’s annual roundup.)

In the US, once again, no published works are entering the public domain today on their own accord.  (Works published in 1960 would have, under the copyright law in effect at the time they were published.   Duke has an overview of some of those, as they have for previous years.)  But we may soon be seeing works regularly enter the public domain once again, and there’s still a lot of the present-day public domain that’s not as well-known as it should be.  So there’s a lot we can do now to support the public domain in 2017.  Here’s what I’m planning to do:

  • Keep government accountable for the public domain and sound copyright policy:  The Congress and administration we just elected in the US are the same people who will be in office when 1923’s copyrights are scheduled to finally expire, on January 1, 2019.  I intend to make sure those expire on schedule, by watching for any attempts to extend copyrights further and telling my elected officials to oppose them.  (This has had good effect in other countries recently,  Pushback against the Trans-Pacific Partnership, for instance, has so far kept Canada from extending its copyrights another 20 years, as that treaty would have required them to do.)  I also intend to press the government to keep fair use strong, and to keep the Copyright Office administered by the Library of Congress, which will help it be responsive to both creators and consumers, and not just to the entertainment industry.
  • Bring to light the “hidden” public domain of the 20th century:  Many public domain works are now freely readable online, but most of them are from before 1923, when it’s easy to determine public domain status in the US.  HathiTrust has also made available many library books published between 1923 and 1963, whose copyrights have expired because they were never renewed as required.  But there are also a lot of unrenewed public domain newspapers, magazines, scholarly journals, photographs, and art and music from this time period that’s in the public domain as well, as well as pamphlets, posters, and underground and amateur publications from as late as 1989 in the public domain.   Unpublished works are also still entering the public domain, including in the US, from authors who died in 1946 or earlier.  All of this material is a crucial part of the memory of the 20th century, especially for more marginalized groups of people, and it’s at more risk of being lost and forgotten than most library books are.  In 2017, I’ll be working on projects that should make it considerably easier for people to clear copyrights for periodical literature in particular,  and thus keep the memories in them alive.
  • Highlight public domain and other open content especially important to remember:  On my Online Books Page, I’ll keep an eye out for books and serials in the public domain that cover topics that seem especially important for people to remember in 2017.  I’ll add them to my catalog with appropriate descriptions and make them more easily findable from places like Google and Wikipedia.  I’ll also be looking for more recent open-licensed content on these topics.  You can help me out with this, by suggesting titles for me to add. Over the past few years, I’ve added thousands of works to my new listings that were suggested by readers, and I’m eager to hear what you think we should be remembering.
  • Add my own work to the public domain:  Like many creators, I’d rather have my work remembered than keep it locked up for 70 years after my death.  So I open-license much of what I put online.  (This post for instance, is licensed CC-BY, so you can copy it as you like as long as you clearly credit that I wrote it and originally published it here on Everybody’s Libraries, and note the CC-BY license.)  And after a suitable period of time, I go further and put my work into the public domain outright.   Today, for instance, I dedicate everything I wrote and published prior to 2003, and whose copyrights I control, to the public domain, via a CC0 dedication.  I’ve made similar declarations in past years as well, inspired by the initial 14-year term that was prescribed by the US’s original copyright law.

It’s especially important in times of uncertainty and danger that we keep our collective memory alive, to help us move forward wisely and joyfully.  The public domain preserves and promotes that memory, and I hope you’ll join me in supporting it in the coming year.

Posted in copyright, open access, preservation | 2 Comments

Forward to Libraries update, and some thoughts on sustainability and scale

Posted on October 3, 2016 by John Mark Ockerbloom

It’s been a while since I posted about Forward to Libraries, but if you’ve been following my Github repo, you may have noticed that it’s had a steady stream of updates and growth.  If making connections across library collections or between libraries and Wikipedia interests you, the service is more comprehensive and wide-ranging than ever.  Here’s an update:

Number of libraries

We now support forwarding to over 1,000 library systems worldwide, running dozens of off-the-shelf and custom-developed catalog and discovery systems.  I’ve expanded coverage in all 50 states, Canada, the UK, and Australia, and have also added links to more countries in all inhabited continents.  (Because the system currently focuses on Library of Congress headings, it works best with Anglo-American catalogs, but it yields acceptable results in some other catalogs as well, like the big Norway research library union catalog I just added today.)  While major research libraries and big-city public libraries are well-represented, I’ve also been trying to add HBCUs, community colleges, rural library networks, and other sometimes-overlooked communities.  And I respond quickly to requests from users to add their libraries.

Wikipedia coverage

Forward to Libraries can be used to make links between library collections, and to and from my millions-strong Online Books Page listings.  But it’s also known for its interoperation with Wikipedia, where it knows how to link between over half a million distinct Library of Congress name and subject headings and their corresponding English language Wikipedia articles.  The majority of these mappings are name mappings provided by OCLC’s VIAF, which in its most recent data dump includes over 485,000 VIAF identifiers that include both a Library of Congress Name Authorities identifier and an English Wikipedia article link.  An additional 50,000 or so topical, demographic, and geographical LC subject headings are also mapped.  These mappings derive from exact matches, algorithmic matching for certain kinds of heading similarities, and a manually curated mappings file that has grown over time to include more than 22,000 correspondences.

What you can do with this data

If you’re browsing a topic or an author’s works on the Online Books Page, you can follow links to searches for the same topic or author in any of the 1000+ libraries currently in my dataset.  (If your library’s not one of them, just ask me to add it.)  If the topic or the author is covered in one of the 500,000 corresponding Wikipedia articles the system knows of, you’ll also be offered a link to the relevant article.

If you’re browsing certain Wikipedia articles, you’ll also find links from them back to library searches in your favorite of those 1000+ library systems (or any other of those systems you wish to search).  Right now those links use templates that must be manually placed, so there are only about 2500 Wikipedia articles with those links, but any Wikipedia editor can add the templates to additional articles.  (A bot could potentially add more quickly, but that would require some negotiation with the Wikipedia community that I haven’t undertaken to date.)  If you’re involved in a Wikipedia-library collaboration project (like this one), you may want to add one of these templates when editing articles on topics that are likely to have relevant source materials in multiple libraries.  (The most common template used is the Library Resources Box, generally added to the External Links or Further Reading section of an article.)

If you’re interested in offering a similar library or Wikipedia linking service from your own catalog or discovery system, I’d be interested in hearing from you.  You can either point to my forwarding service (using a standard linking syntax) or implement your own forwarder based on my code and data on Github.  Right now it requires some effort and expertise to implement either method, but I’m happy to work with interested libraries or developers to make forwarding easier to implement.

Scaling and sustaining the service

The Forward to Libraries service still runs largely as a 1-person part-time project.  (The Wikipedia templates are largely placed by others, and the service fundamentally depends on regularly updated data sets from OCLC, the Library of Congress, and Wikipedia, but I maintain the code and coordinate the central referral database myself.)

Part-time, 1-person projects raise some common questions:  “Will they be sustained?”  and “Can they scale?”  I wasn’t sure myself of the answers to those questions when I started developing this service.  Fortunately, I went ahead and introduced it anyway, and three-going-on-four years later, I’m happy to say that the basic service *is* in fact more sustainable and scalable than I’d thought it might be.   The code is fairly simple, and doesn’t require a lot of updating to keep running.  The main scale issues for the basic service have to do with the number of library systems and the number of topic mappings in the system, and those are both manageable.

The number of libraries turns out to be the more challenging factor to manage.  Libraries change their catalogs and discovery systems on a regular basis, and when they do, search links that worked for their old catalogs often don’t work for the new ones.  I have an automated tool that I run occasionally to flag libraries that return an error code to my search links; it’s not very sophisticated, but it does alert me to many libraries whose profiles I need to update, without my having to check all of them manually.  (If you find any libraries where forwarding is no longer working, you can also use the suggestion form to alert me to the problem.)  The more pressing scaling problem at the moment is the user experience: right now, when you’re asked to choose a library, the program shows a list of links to all 1000+ libraries currently in the system.  That can be a bit much to handle, especially for users whose data is metered.  Updating the library choice form to only show local libraries after the user selects the state, province or country they’re in will cut down on the data sent to the user; that may cost the user an extra click, but the tradeoff seems worth it at this point.

The number of topic mappings, on the other hand, has been easier to manage than I’d thought.  VIAF publishes updated data files for names about once a month, and I can run a script over it to automatically update my service’s name heading mappings when a new file comes out. Likewise, Wikipedia is now providing twice-a-month dump files of its English language encyclopedia.  I can download one of Wikipedia’s files in a couple of hours, and then run a script that flags any topical articles I map to that have gone away, changed their title, or redirected to other articles.  I can then change my mappings file accordingly in under an hour. Library of Congress subject headings change as well, but they don’t change very fast.  New and changed topical headings are published online about once a month, and one can generally add or change mappings from one of their updates within a few hours.  I spend a few spare-time hours each month adding mappings for older subject headings, so if the current rate of LCSH growth holds, in theory I could cover *all* LCSH topical headings in my manual mappings file after some number of years.  In practice, I don’t have to do this, especially as topical mappings also start to get collaboratively managed in places like Wikidata.  (I’m not currently working with their data set, but hope to do so in the future.)

Broadening the service

While I can maintain and gradually grow the basic service with my current resources, broadening it would require more.  Not every library, particularly outside the US, uses Library of Congress Subject Headings, so it would be nice to offer mappings to more subject ontologies and languages.  Similarly, not everyone likes to work with Wikipedia (often with good reason), and it’d be nice to support links to and from alternative knowledge bases as well.  The basic architecture of Forward to Libraries is capable of handling multiple library and web-based ontologies, but additional coding and data maintenance would be required.  There are also various ways to build on the service to engage more deeply with libraries and current topics of interest; I’ve explored some ideas along these lines, but haven’t had the time to implement them.

Things continuing as they are, though, I should be able to maintain and grow the basic Forward to Libraries service for quite some time to come.  I’m thankful to the people and the data providers that have made this service possible. And if you’re interested in doing more with it, or helping develop it in new directions, I’d be very glad to talk with you.

Posted in data, discovery, libraries, metadata, wikipedia | Comments Off on Forward to Libraries update, and some thoughts on sustainability and scale

The value of catalogs in the linked data era: Two recent talks

Posted on September 12, 2016 by John Mark Ockerbloom

I’ve recently uploaded two talks I gave this year to my Selected Works site.  I presented “How Not to Waste Catalogers’ Time: Making the most of subject headings” at the Code4lib 16 conference in Philadelphia in March.  “How to Read 100 Million Publications: VIVO and Comprehensive Open Publication Databases” was a talk I gave at VIVO 2016 last month.  The versions I’ve deposited in Selected Works are PDF files that include both slides and notes, so you can see both what I showed, and (approximately) what I said during the presentations.

The talks can be seen as two complementary takes on the question “What’s the value of the catalog, and cataloging, in today’s networked environment?”  In “How to Read 100 Million Publications” I highlight the value of an open, comprehensive database of the scholarly record that goes down to the article level.  Such a database would be quite large (on the order of 100 million records, as the title suggests), and considerably finer-grain that present-day library-run catalogs, which typically catalog at the title level, and only add a little bit of volume information at the holdings level.  But as I describe in the talk, 100-million-record-scale databases are now routinely maintained, and regularly replicated, online, and even a “just the facts” database of who published what articles where could be very helpful in promoting preservation, open access, corpus analysis, research networking systems, and a variety of other applications.  It would not need to be built from scratch– the data required for such a database already largely exists online, though all too often in fragmentary or proprietary collections.  But a number of these could be brought together and opened up, without requiring much additional cataloging labor.  Shared-effort collaborations, like those that support databases like GOKb and SHARE, as well as more crowd-sourced projects like FictionMags or Wikipedia, could support the scale needed.

While “How to Read 100 Million Publications” proposes something new we could do with library catalogs, “How Not to Waste Catalogers’ Time” urges developers to pay close attention to what catalogers have already been doing with them– particularly the often-overlooked semantic richness of their subject cataloging.  At the VIVO conference I saw a demo of a BIBFRAME application where a book’s subjects were represented by an ordered set of FAST properties.  That’s easy enough to model as RDF linked data, but it omits the subject ordering that can improve relevance ranking, and lets readers distinguish the relative importance of subjects in a particular work.  It also doesn’t accommodate the detailed on-the-fly subject categories that coordinated subdivisions allow.  It’s possible to accommodate both of these in linked data, but it requires more complicated data models, and most catalogs don’t take full advantage of their semantic strengths.  My talk shows some examples of systems that can take advantage of them (as well as the strengths of FAST and Wikipedia terminology).  Whether or not we implement such systems broadly, I hope we continue to support ordered, coordinated subject headings in the underlying data of the next-generation catalogs we build.  It’s easy enough to automatically map from such data to FAST term sets, for discovery systems that work best with those, but you can’t reliably and automatically go back the other way and keep the original degree of precision.

In short, I want us to build catalogs that let catalogers and readers do more with metadata, rather than less.  I hope that these two talks suggest some ways we can do that, and I’d love to hear from folks who have more ideas, suggestions, or questions.

 

Posted in discovery, metadata, open access, sharing, subjects | Comments Off on The value of catalogs in the linked data era: Two recent talks