THE BRITISH LIBRARY

UK Web Archive blog

Information from the team at the UK Web Archive, the Library's premier resource of archived UK websites

Introduction

News and views from the British Library’s web archiving team and guests. Posts about the public UK Web Archive, and since April 2013, about web archiving as part as non-print legal deposit. Editor-in-chief: Jason Webber. Read more

15 September 2016

Commemorating the Battle of the Somme in the UK Web Archive

On the 15 September 1916 the the Battle of Flers Courcelette (a phase of the greater Battle of the Somme) commenced. It is mostly famous for the introduction of the tank into battle (to mixed results). Less well known now is that it was the day that the Prime Ministers own son Lt. Raymond Asquith was killed when he went into action with his unit, the 3rd Grenadier Guards. It turned out to be the battalion's bloodiest single day of the war. Asquith's death is recorded in the battalion war diary that I transcribed while I was researching my own Great Grandfather. This website is now saved as part of the UK Web Archive and will be available for future research even if the original goes offline.

THE BATTLE OF THE SOMME, JULY-NOVEMBER 1916 THE BATTLE OF THE SOMME, JULY-NOVEMBER 1916© IWM (CO 802)

Commemorating the Somme and the First World war
The UK Web Archive has been collecting websites about the First World war since 2014 and will continue to do so until at least 2019. So far we have 726 individual websites in the collection, 128 of which are available to view through the public website.

There is already a great range of websites in the collection. Many of them look at memorials linked to places (e.g. Crich parish roll of honour) or individual units (e.g. 36th Ulster Division). Others commemorate individual family members such as William Thomas Clarke.

The home front is not forgotten in projects such as 'A Year in the Life of Avon Dassett' or 'Sunderland in the First World war'.

We need your help!
We welcome any suggestions for making this collection as complete as possible. If you have a UK website that relates to the First World War (or know of one), please let us know through twitter (@ukwebarchive) or our nomination form.

Online resources often only last a few years and the UK Web Archive aims to keep copies of these First World War centenary websites in perpetuity. Help us keep these memories alive.

By Jason Webber, Web Archiving Engagement Manager, The British Library

14 September 2016

Surveying the Domain: Three Days with the Web Archiving Team

I’m Sara Day Thomson, researcher for the Digital Preservation Coalition. We’re a membership organisation who support institutions, like BL, to ensure long-term access to their digital content, no matter what that might be. To support my own professional development and general curiosity, the Web Archiving team at BL let me spend three days with them learning the ins and outs of archiving the Internet.

SDT_DPC_ProfessionalProfile

Web Archiving vs Digital Preservation?
What, you might ask, does web archiving have to do with digital preservation? I would answer: everything. Web Archiving operates at the frontier of capturing and preserving our contemporary cultural and historical record. From the Information Highway to social networking sites, the Internet represents not only our cultural record but the inscription of an evolving technology. As I learned while tinkering with the web archiving ‘machine’, I got a first-hand look at the challenge this creates for archivists who must keep pace with the development of the Web and how people use it.

If you haven’t seen it, I’m the author of the recent Preserving Social Media Technology Watch report. Preserving Social Media presents these same issues faced by organisations who want—or are required—to archive social media content. My three days with the BL team have provided a wider lens to my understanding of the role of social media and what it actually looks like to archive the wider Web.

PresSocMedia_COVER

Three days spent harvesting the Web with the BL team has solidified my view that web archiving is fundamentally an act of digital preservation. Just like many ‘traditional’ digital media, such as PDFs or emails or mp4s, further action must be taken on web content in its native form in order to ensure its long term accessibility. The need for further action for web content is urgent, even more so than for some other digital formats. During just my brief tenure, I came across more than one website that had disappeared since it was last harvested.

Challenges and rewards
Web content is complex—even discussing social media as a single category poses problems because different platforms function in different ways and are governed by varying Terms of Service. While social media has more recently become a dominant player, there’s a whole world of Web out there that isn’t ‘platformized’. Given this diversity—and the likelihood that technology will continue to dramatically alter how we dispense and consume information—web archivists are faced with the challenge of ensuring this content will be useable and comprehensible in the future. This challenge is at the centre of any digital preservation endeavour: it’s not just about saving the bits, or the code, but about preserving meaning.

The BL team are not alone in the effort to save the Web for future generations. While the team is relatively small (smaller than you’d think given the scale of the task), they work closely with their Legal Deposit partners, with curators within BL, with curators without the BL, and with the researchers and other users. The creation of a meaningful record of our lives online requires the input of all of these specialists and is likely to be more successful through open collaboration.

The challenges—and rewards—of digital preservation are best shared, whether it’s for the preservation of digitised manuscripts from the Middle Ages or the emails of the prime minister or a national record of the World Wide Web.

By Sara Day Thomson, Digital Preservation Coalition

18 August 2016

Poetry Goes Online: Preserving poetry journals and zines for the Web archive

I have been working on a Special Collection for the UK Web Archive of UK-based online poetry journals and magazines. My own research at Goldsmiths, where I am completing the first year of a PhD, is concerned with contemporary poetic responses to the increasing ubiquity of the internet and networked culture. This project has been a fantastic opportunity to enrich my own understanding of digital poetry publishing in the UK and develop my research paradigm; I also hope my findings answer some questions regarding digital-only collection strategies for the Library’s on-going Non-Print Legal Deposit (NPLD) responsibilities. In this article I want to share some of my discoveries which will be included in the Special Collection.

The Next Generation of Poetry Journals
My research interests grew out of my experience with poetry communities which had emerged out of, and operated entirely within, digital spaces: participants used social media for networking, collaboration and promotion, taking advantage of cheap web hosting and free blog domains to publish zines and chapbooks. For a younger generation of digitally-native poets growing up in an era of cuts to arts funding (and perhaps less sentimental about print culture), the internet provides the easiest and cheapest method to publish, be read and to interact with other poets. It also provides a space for groups often excluded or underrepresented in print publishing. tender is an exceptional example of this latest generation of online journals; published quarterly as a highly-polished downloadable PDF file, it features a curated selection of original art, poetry, prose and interviews made exclusively by female-identified writers and artists.

Fig1_tender
Fig 1

Making a Break from Print Culture
Unlike print publishing - with its propensities for the risk-averse and the commercial - the effectively free status of online publishing encourages greater formal and thematic experimentalism. For Every Year, for example, is publishing an original piece of prose, poetry or “something else” for every year since 1400 – they have already made it to the year 1821 and show no signs of stopping anytime soon. Other thematically adventurous publications in the collection include Visual Verse, a zine based entirely around ekphrastic writing; and PracCrit, a journal which publishes original poems juxtaposed with essayistic responses from other poets. Many of these publications are - like much online activity - international in outlook, with contributors hailing from around the globe. The lack of a clear geographical home for certain journals opens up a number of problems regarding NPLD scope, which is limited to the preservation of UK publications.

Fig2_foreveryyear
Fig 2
Fig3_praccrit
Fig 3

The Changing Digital Landscape
Examining the brief history of online poetry also charts a broader history of internet publishing trends, as the infrastructure of online spaces evolves with each successive technological shift. The simplistic text and image sites of “Web 1.0” have been replaced with increasingly sophisticated interfaces and professional graphic design as internet culture comes of age (Footballpoetry).

Footballpoetry2005
In 2005
Footballpoetrytoday
In 2016

Elsewhere, journals like Conversation Poetry are published via Issuu, a skeuomorphic digital publishing platform which mimics the physical properties of a print publication. Conversely – and perhaps more interestingly - journals such as Proof in the UK and The Claudius App in the US foreground the aesthetics of their own digitality. Through the utilisation of multimedia platforms like Java and Flash, these journals aim to make the experience of reading itself consonant with the interactive, dynamic nature of computational technology.

These too present some of the greatest challenges for the Web Archive moving forward, since even advanced web crawlers have limitations when archiving plugins and streaming media content (although new advances in archiving technology show promise). As part of the broader born-digital genre of e-literature, these new experiments mark a break with traditional “bookbound” forms, and may offer a glimpse of the future of literary arts. Look out for the collection on the Web Archive in the next few months.

By Joe McCarney, PhD Placement Student, Goldsmiths, University of London

17 August 2016

Tender to Redevelop the UK Web Archive Website

The UK Web Archive (based at The British Library) is looking to appoint a superb User Experience (UX) company to help us improve our Web Archive service to the public and facilitate high quality academic research.

The project should be open source and bring an innovative and engaging interface to our web archive collections. The project should also integrate with our work on full-text search and trend analysis (see www.webarchive.org.uk/shine).

For a copy of the Invitation to Tender and how to respond please visit the BL eTendering Portal at the following link:

https://bl.bravosolution.co.uk; at the Home Page, under ‘Opportunities and notices’ please click on ‘View current opportunities and notices’.

If you wish to view/download the documents and are not registered on Bravo, please follow the instructions below (registration is free). If you are register please go to step 2.

1. Register your company on the e-tendering portal (this is only required once).

Select the ‘Login or register to participate’ link above and click the ‘Click here to register’ link on the home page.

Accept the terms and conditions and click continue,
Enter your correct business and user details,
Note the username you chose and click Save when complete,
You will shortly receive an email with your unique password (please keep this secure).

2. Respond to the ITT

Login to the portal with the username/password,
Click the ITT's Open To All Suppliers link.
Click on the relevant Tender
Click the Express Interest button in the Actions box on the left-hand side of the page,
This will move the ITT into your My ITT's page. (This is a secure area reserved for your projects only),
You can now access any attachments by clicking the Settings and Buyer Attachments in the Actions box

3. Responding to the ITT

You can now choose to Reply or Reject (please give a reason if rejecting),
You must use the Messages function to communicate to the Library and seek any clarification,
Note the deadline for completion, then follow the onscreen instructions to complete your response to the ITT.

You must then publish your reply using the publish button in the Actions box on the left-hand side of the page.

Note: If you have any questions regarding the tender, please do so through the portal.

By Jason Webber, Web Archiving Engagement Manager

27 June 2016

Capturing and Preserving the EU Referendum Debate (Brexit)

Following the announcement in May 2015 that there would be a referendum on the UK’s EU membership; the Legal Deposit UK Web Archive, led by curators at the Bodleian Libraries, started a collection of websites.

The team of curators includes contributors from the Bodleian Libraries, The British Library, the National Libraries of Scotland and Wales and also Queen’s University Belfast (for the Northern Ireland perspective) and the London School of Economics (for capturing and preserving individual documents, such as the pdf versions of campaigning leaflets). 

The collection scope is to capture the ‘Brexit’ debate and the debate around the EU Referendum as well as the wider context of UK/EU relations, including:

  • Media coverage,
  • websites of political parties and other political institutions and groups
  • campaigning and lobbying
  • trade unions, professional organisations, businesses
  • academic debate
  • culture and arts
  • public opinion through blogs, comments, and if possible social media.

We primarily archive UK websites under the Non-Print Legal Deposit mandate, but also decided to include some sites outside the UK, if relevant – e.g. websites of UK expats in Europe, or political parties, interest groups and think tanks in the EU and in EU member states – on a permission basis.

The collection (at the time of writing) has 2590 target websites. Some of these are whole websites; others will be a single news story or blog post.

Access and availability
The majority of the collection will be available in the reading rooms of UK Legal Deposit libraries, including both British Library sites. As is usual for web archive collections, there is a delay between collection and availability of up to a year.

By Svenja Kunze, Project Archivist, Bodleian Libraries (Oxford University)

17 May 2016

Saving BBC Recipes Website

There's been much coverage today of plans to remove the recipe pages from the BBC website.

6018503713_573fccc22a_z

The UK Web Archive has been collecting selected pages from the BBC, mainly news, for over ten years and since 2013 we have attempted to capture the entirety of the BBC web estate. A small number of pages are available on the Open UK Web Archive website. Most of the BBC's online presence, however, is only available in the reading rooms of UK Legal Deposit libraries, including both of the British Library sites at St. Pancras and Boston Spa in Yorkshire.

We have today instigated a further crawl of the BBC website with the specific aim of ensuring that we save the recipes from the food pages. We can also report that the Internet Archive, Library of Alexandria and the National Library of Iceland have also captured these pages so their future is assured.

Polly Russell, British Library Curator and Food Historian says 

"Cookery books, like cookery websites, obviously serve a practical purpose but that is not all. For historians, sociologists and anthropologists they also tell us about people's culinary aspirations and anxieties, cultural tastes and trends, dietary preoccupations, social expectations and economic conditions. They are, therefore, a rich source for researchers. So while it's sad news to hear about plans to close the much trusted and well-loved BBC Food website, it's a relief that the British Library is going to be able to archive the website for posterity."

 

 

26 April 2016

Easter Rising 1916 Centenary in Print and Digital

Ireland has been gripped by  commemorations of the Easter Rising in the last month. The Rising took place from the 24th April to the 29th April 1916 in Dublin. A packed programme of events and activities took place across Ireland and in Irish communities further afield to commemorate this centenary.

In March 2016, addressing a colloquium at the Bodleian Library, Oxford, the Irish Ambassador to the United Kingdom, his Excellency Daniel Mulhall, emphasised the transnational and inclusive nature of the commemoration programme in his opening remarks. The 1916 Rising had a global impact with ripples felt as far as Asia and India. This is reflected in the range of events taking place in the United Kingdom, supported by the Irish Embassy.

In military terms the Rising was a failure and had consequences for the people of Dublin with 415 people killed, the majority of whom were civilians.

Print
Turning to the documentation of the Rising, there are a number of interesting documents within the Library’s collections relating to the Rising. The British Library does not hold an original broadside of the Proclamation of an Irish Republic. Nevertheless, later examples of the document were acquired retrospectively.

The earliest example of a version of the proclamation in the British Library’s collections, can be found at C.S.A.24/3.(1.). This is interesting from a bibliographical stand point because it is the first entry under the new heading in the British Library Printed Catalogue to 1975:

ProvisionalGovernmentEntryBLPC

Provisional Government of the Irish Republic 1916. Miscellaneous Public documents. 

That the Library classified this proclamation as a public document and gave the document the C.S.A., official publication pressmark prefix, which originates from the 1890s, is of particular interest.  The third factor which is of interest is that this version of the proclamation is the only item in the green bound guard-book which is embossed on the spine in gold.

Poblacht na heireann1916

IRELAND. PROCLAMATIONS, ETC.

Although the red (purchase) stamp appears on the reverse of the document, because of the way it has been mounted in the volume it is unclear when the item was acquired. It appears to read 15 May ‘59. The volume itself bears the British Museum binders stamp B.M.1961 on the inside of the rear board. These dates indicate that this item, as with other ephemera relating to 1916 Rebellion, was acquired retrospectively. 

Poblacht na heireann 1941

The second example of the proclamation is a more ornate affair. It is a single sheet dating from 1941, measuring approximately 325mm x 255mm. The text of the document is laid out in the same fashion as the original, but the type face has been standardised, removing the anomalies from the original, and the list of signatories has been centred rather than justified to the right as in the original. What is most striking about this item are the portraits of the seven signatories surrounding the text and connected by the decorative boarder. At the bottom centre surround in a circle is the Irish Army sunburst emblem, designed by Eion MacNeill, and interestingly it is reproduced without the inscription "Óglaigh na hÉireann" or Irish Volunteers.

Irish War News Irish War News p4

The third document is a piece of contemporary ephemera which traces its lineage to the focal point of the rebellion. Dated Tuesday April 25 1916, on the last page of the first issue of Irish War News it is an article headed:

“Stop Press (Irish) ‘War News’ is published to-day because a momentous thing has happened. The Irish Republic has been declared in Dublin and a Provisional Government has been appointed to administer it is affairs.”

 The article goes on to name the signatories of the proclamation as the Provisional Government while outlining the situation in Dublin from the rebel prospective.       

Digital
The Rising, or more particularly the centenary of the events in Dublin a hundred years ago, is being explored and represented in new ways thanks to technology and the work of colleagues at Trinity College Dublin and the Bodleian Library Oxford. In the last year they have built and curated a collection of websites related to the commemoration.

These have been archived as part of the open UK Web Archive.  To have the opportunity to build this collection of Irish and UK websites is an exciting prospect for the future of web published content. This endeavour illustrates how the internet is not confined by national boundaries. The work on the Easter Rising collection exemplifies how archivists working together can build a contemporary collection which provides a range of perspectives from all corners of the .uk and .ie domains.   

Archiving websites about anniversaries and centenaries such as Easter 1916 is of prime importance because such sites can be transient and are soon overwritten or taken down. Archiving them creates a research resource for the future which offers scholars and anyone interested the opportunity to explore and examine the response to this centenary on the published web.

The Easter Rising collection is currently a growing part of the UK Web Archive special collections where it can be freely consulted online.

By Jeremy Jenkins, Curator Emerging Media, The British Library
@_jerryjenkins

 

Further Reading

Bouch, Joseph J. “The Republican Proclamation of Easter Monday, 1916,” Bibliographical Society of Ireland, Publications vol.5. no.3 1936. General Reference Collection: Ac.9708/2 [A reissue].

The Easter Proclamation of the Irish Republic, MCMXVI
Dublin : Dolmen Press, 1960. General Reference Collection: Cup.510.ak.37

The Easter Proclamation of the Irish Republic 1916,
[S.l.] : Dolmen Press, 1976. Document Supply Shelfmark: D76/23312

 

 

15 February 2016

Introducing SHINE 2.0 - A Historical Search Engine

Add comment Comments (0)

In 2015, as part of the Big UK Domain Data for the Arts and Humanities project, we released our first ‘historical search engine’ service. We’ve publicised it at IDCC15, the 2015 IIPC GA and at the first RESAW conference, and so far has been very well received. Not only has it lead to some excellent case studies that we can use to improve our services, but other web archives have shown interest in re-using the underlying open source code. In particular, some of our Canadian colleagues have successfully launched webarchives.ca, which lets users search ten years worth of archived websites from Canadian political parties and political interest groups (see here for more details).

Even bigger data!
But we remained frustrated for two reasons. Firstly, when we built that first service, we could not cope with the full scale of the 1996-2013 dataset, and we only managed to index the two billion resources up to 2010. Secondly, we had not yet learned how to cope with more than one or two users at a time, so we were loath to publicise the website too widely in case it crashed. So, over the last six months, and with the guidance of Toke Eskildsen and Thomas Egense at the State Library of Denmark, we’ve been working on resolving these scaling issues (their tech blog is definitely worth a look if you’re into this kind of thing).

Thanks to their input, I’m happy to announce that our historical search prototype now spans the whole period from 1996 to the 6th April 2013, and contains 3,520,628,647 distinct records.

Shine-release-two-total-resources-over-time

Broken down by year, you can see there’s a lot of variation, depending on the timings of the global crawls from which this collection was drawn. This is why our trends visualisation plots query results as a percentage of all the resources crawled in each year rather than absolute figures. However, the overall variation and the fact that the 2013 chunk only covers the first three months should be kept in mind when interpreting the results.

Time travel?
You might also notice there seem to be a few data points from as early as 1938, and even from 2072! This tiny proportion of results correspond to malformed or erroneous records, although currently it’s not clear if the 1,714 results from 1995 are genuine or not. No one ever said Big Data would be Clean Data.

De-duplication of records
Furthermore, we’ve decided to change the way we handle web archiving records that have been ‘de-duplicated’. When the crawler visits a page and finds precisely the same item as before, instead of storing another copy, we can store a so-called “revisit record” and refer to the earlier copy rather than duplicating it. This crude form of data compression can save a lot of disk space for frequently crawled material, and it’s use has grown over time. For example, looking at the historical dataset, you can see that 30% of the 2013 results were duplicates.

Shine-release-two-revisits

However, as these records don’t hold the actual item, our indexing process was not able to index these items properly. Over the next few weeks, we shall scan through these 65 million revisit records and ‘reduplicate’ them. This does mean that, for now, the results from 2013 might be a bit misleading in some cases. We also failed to index the last 11,031 of the 515,031 WARC files that make up this dataset (about 2% of the total, likely affecting the 2010-2013 results only), simply because we ran out of disk space. The index is using up 18.7TB of SSD storage, and if we can find more space, we’ll fill in the rest.

Do try it at home
In the meantime, please explore our historical archive and tell us what you find! It might be slow sometimes (maybe 10-20 seconds), so please be patient, but we’re pretty confident that it will be stable from now on.

Shine-release-two-early-social-media

Shine-release-two-later-social-media

Shine-release-two-austerity

https://www.webarchive.org.uk/shine

By Andy Jackson, British Library Web Archiving Technical Lead