LUSTRE Workshops 1 & 2 report

Over the course of nine months, the AHRC-funded LUSTRE project has made significant progress in its exploration of using Artificial Intelligence (AI) to unlock born-digital and digitised government archives. The project has already delivered a range of activities, including more than 50 semi-structured interviews with professionals in the GLAM sector, computer scientists, and scholars. Additionally, there have been four online lunchtime talks and two hybrid workshops. 

In this blog post, we will focus on the two hybrid workshops organised in collaboration with our project partner, the Cabinet Office. These workshops took place at the Central Digital and Data Office in Whitechapel, London, as well as online. The talks from both workshops have been recorded and can be accessed on the LUSTRE website through the following links: 

LUSTRE Workshop 1 

LUSTRE Workshop 2 

The primary objective of these workshops was to foster the establishment of a network of professionals in the archival field and academic researchers who share an interest in exchanging knowledge regarding the opportunities, challenges, and potential risks associated with AI for accessing, managing, and using born-digital archives. 
Summary of Workshop 1

Invited speakers to the first workshop in January 2023 were Professor Stephanie Decker and Dr Adam Nix from the University of Birmingham; Dr Jenny Bunn from The National Archives; and Dr Tony Russell-Rose from Goldsmiths University of London. 
The day began with a presentation by Professor Stephanie Decker and Dr Adam Nix, who discussed the practices of digital archival discovery, focusing specifically on the use of AI to connect context and content in emails. Their talk provided valuable insights from a user perspective, emphasising the significance of meaningful access to born-digital archives for researchers, particularly those in the humanities and social sciences. 

Next, Dr Jenny Bunn delved into the ethical dimension of using advanced technologies, which are often categorised under the umbrella term of AI. She highlighted the challenges that the introduction of AI presents to the delicate balance between transparency, accountability, and fairness in recordkeeping activities. Dr Bunn explored new approaches that can maintain transparency and accountability in recordkeeping practices amidst the use of AI, emphasising the importance of good governance. 

The third talk of the day was led by Dr Tony Russell-Rose, and it showed how the LUSTRE project effectively brings together researchers and professionals from the GLAM sector, digital humanities, and computer science. Dr Russell-Rose’s presentation focused on access to databases and proposed approaches that move away from traditional command-line query builders. His talk Searching, fast and slow: rethinking the query builder paradigm centred on a platform called 2Dsearch which redefines ‘advanced search’, reducing syntactic errors, enhancing semantic transparency, and providing support for reuse and optimisation. 

The day concluded with a thought-provoking roundtable discussion featuring Lise Jaillant, Adam Nix, Stephanie Decker, and Jenny Bunn, centred around ethical issues in AI and government records. The speakers further expanded on the ethical dimension of conducting research on digital archives using AI-assisted technologies. They unanimously emphasised the need for transparent, explainable, and accountable processes and technologies. 
Adam Nix drew attention to the risks and contingent liabilities of already available collections when novel forms of searches unveil new materials. Stephanie Decker reflected on the challenges of anonymisation and privacy, highlighting the delicate balance between anonymising data and preserving valuable contextual research information. Jenny Bunn emphasised the active effort archivists must undertake to avoid embracing the ‘myth of objectivity’ when adopting new technologies, asserting that archival practice is inherently subjective and necessitates the archivist’s engagement as a gatekeeper. She also discussed the increased sensitivity risks associated with larger and more complex datasets. 

Overall, the first workshop provided valuable insights into the intersection of AI and born-digital archives, exploring both the practical and ethical dimensions of using  AI technologies for archival research. 

Summary of Workshop 2

The second LUSTRE workshop, titled ‘AI and Born-Digital Archives in the Government Sector and Beyond: Challenges and Opportunities’, was once again held  in Whitechapel. The workshop took place in May 2023 and, like the previous one, it was a hybrid event with five presentations, including two remote presenters. 

Invited speakers included Dr Keegan McBride from the Oxford Internet Institute, James Lappin from Loughborough University, Dr Lise Jaillant from Loughborough University (PI of the LUSTRE project), John Sheridan from The National Archives, and Professor Jason R Baron from the University of Maryland. 

Dr Keegan McBride commenced the day with his talk, ‘Artificial Intelligence in the Public Sector: Separating Myth from Reality’. He focused on differentiating the development of AI in the private sector from its applications in the public sector. Dr McBride highlighted the necessary building blocks for effective AI usage in the public sector, including technical, infrastructural, and legislative aspects. He also presented best practice case studies to illustrate AI implementation. 

James Lappin followed with a presentation on ‘AI and the Management of Email Accounts Over Time’, drawing on his doctoral research. He explored the impact of AI on access permissions and retention rules within digital corporate systems, particularly focusing on email. His talk covered various aspects, including the impact of email on recordkeeping, the effective organisation of correspondence, and key strategic choices when applying AI to email management. 

Dr Lise Jaillant, the Principal Investigator of the LUSTRE project, provided her perspective on the challenges posed by the transition from traditional archives to digital archives for professionals and academic researchers. She shared results from interviews conducted with GLAM professionals, researchers, and archive users. The participants highlighted two pressing obstacles in accessing digital records: mistrust between stakeholders and mistrust of technology. Dr Jaillant emphasised the importance of facilitating knowledge transfer, fostering collaborations between GLAM professionals, academics, and other stakeholders, and enabling access to archival materials as well as the discovery of their existence. 

Following a brief question and answer session involving both in-person and online participants, the workshop broke for lunch. The afternoon session commenced with a talk by John Sheridan, the Digital Director of The National Archives, titled ‘Navigating the Maelstrom (Looking for Lighthouses)’. He reflected on the rapid advancement of AI and large language models and the uncertainties these technologies bring. He presented strategic interventions that digital archives can implement to leverage AI’s opportunities. These interventions encompassed content classification and organisation, new text generation (including summaries and transcriptions), image/video generation based on instruction, and data transformation/manipulation based on instruction. 

The day concluded with a remote guest speaker, Professor Jason R Baron (University of Maryland). His talk focused on the process which should lead to the transfer of permanent federal government records to the National Archives and Records Administration (NARA) solely in electronic or digital formats by June 2024. 
He discussed the challenges NARA faces in meeting the 2024 deadline and the limitations presented by the Freedom of Information Act (FOIA) in providing timely access to archival records. Professor Baron also explored how AI tools can assist in ensuring real public access to government archives. He emphasised the role of AI in supporting archivists, records managers, and lawyers in filtering sensitive content from public records and recommended continued experimenting with Technology Assisted Review methods for efficiently discovering records and accurately segregating personal content. He also suggested exploring how generative AI could provide narratives explaining why documents or portions of documents have been withheld under freedom of information laws. Professor Baron concluded his talk by encouraging the embrace of AI, considering it not merely as a black box, but as a ‘gift’ to archivists to improve access to vast digital collections.


Navigating the maelstrom (looking for lighthouses)

13:30 – 14:00

Speaker: John Sheridan (The National Archives)

Abstract: We are navigating a maelstrom. The pace of advancement in AI, in particular of large language models, is bewildering, exhilarating, and terrifying. The scale of the advances made over just the last few months seems immense. No-one seems to know the limits of what is now possible, or might be.

For digital archives, this is a moment of enormous opportunity. There are also risks and threats. To quote Clausewitz, “A sensitive and discriminating judgment is called for; a skilled intelligence to scent out the truth.” (1). So, where are the lighthouses to try to navigate by, or, if there are none apparent, where at least might they be built? This presentation will suggest there are some beacons of light to guide us. Borrowing ideas from maths, computing, law and economics, it will posit what they are and what they mean for archivists. From there it will extrapolate some strategic interventions that digital archives can make to take advantage of the enormous opportunities ahead of us in the brave new world of artificial intelligence.

John Sheridan is the Digital Director at The National Archives.

1 – Carl von Clausewitz, Vom Kriege, Book 1, Chapter 3.


Is NARA Ready Yet? The Newly Extended 2024 Start Date For Accessioning Records into the US National Archives Only in Electronic and Digital Formats, and What That Means

14:00 -14:30 (online)

Speaker: Jason R. Baron, Professor of the Practice, College of Information Studies, University of Maryland 

Abstract: For over a decade, the US Archivist has been attempting to mandate a future start date for the accessioning of all permanent federal government records into the National Archives and Records Administration (NARA) solely in electronic or digital formats.  Although the “no more hard copy records” date for doing so has been extended twice, in the interim much has been accomplished in promoting governmental policies that involve the management of hundreds of millions if not billions of electronic records in individual agencies, including most prominently the archiving of e-mail records. Three questions we should be asking: how will NARA meet the challenge of accessioning especially born digital records when the deadline does finally arrive?  Are freedom of information laws up to the task of providing timely access in the future to those records? And how can AI tools assist in ensuring that public access to the government’s archives remains real and not just aspirational? 

Bio: Jason R. Baron is a Professor of the Practice in the College of Information Studies at the University of Maryland. Between 2013 and 2020, he acted as Of Counsel to the eDiscovery & Information Governance group at Faegre Drinker, LLP. During his prior 33 years in government service, he served as the first appointed director of litigation at the National Archives and Records Administration, and before that as a trial lawyer and senior counsel at the Department of Justice. In those capacities, Mr. Baron acted as lead counsel on landmark lawsuits involving the preservation of White House email, and also played a leading role in improving federal electronic recordkeeping policies. He is a recipient of the international Emmett Leahy Award for his career contributions in records and information management. Mr. Baron received his B.A. magna cum laude with honors from Wesleyan University, and his J.D. from the Boston University School of Law.


The Impact of AI on Records Management 

11: 00 – 11:30

Speaker: James Lappin, Loughborough University

Abstract: It seems likely that AI will, at some point in the relatively near future, provide originating organisations with the capability to re-aggregate records in any (or all) of their systems.  The organisations concerned could then, if they wished, begin to apply access permissions and retention rules to records on the basis of an entirely different set of aggregations to those used by the end-users that created and received them. 

This opens up a theoretical question: 

  • will the coming of such a capability mean that the way that records are aggregated in the day-to-day systems that end-users used to create and receive them (document management systems, collaboration systems, email systems, chat systems etc.) cease to have anything more than a purely temporary significance in recordkeeping? OR 
  • will that ‘original order’ of records instead place constraints and limits on the ways that original organisations can make use of artificial intelligence capabilities for improving the precision of the application of access permissions and retention rules? 

This talk will explore the impact of AI on the application of access permissions and retention rules within digital corporate systems, on the basis of a set of thought experiments carried out in James’ doctoral research project. 

Bio: James Lappin is a doctoral researcher with Loughborough University’s Centre for Information Management. He has worked in the field of archives and records management for thirty years as a practitioner, consultant, researcher, policy advisor, presenter, blogger, podcaster and cartoonist. 


Artificial Intelligence in the Public Sector: Separating Myth from Reality

10:30 – 11:00

Speaker: Dr Keegan McBride

Abstract: Artificial Intelligence (AI) developments in the private sector have quickly permeated throughout all levels of society. Daily we are – either directly or indirectly – interacting with AI. Acknowledging the benefits that AI has brought to the private sector, particularly regarding efficiency gains, many have started to talk about the potential benefits that AI may have when applied in and by the public sector. Certainly, some benefits do exist. However, it is also important to realize that the public sector and the private sector are different, with different requirements, goals, and aims. While the private sector may be concerned with customer acquisition, maximizing profits, or market growth, the public sector from does not have the same motivations. Thus, the reasons for applying AI, the ways in which AI is applied, and the areas where AI can be applied will be similarly different. To maximize the impact that AI can have, these differences must be acknowledged. Starting from this need for differentiation, this talk will help to demystify AI, by defining and outlining what AI is and is not. Following this, the talk will highlight the necessary building blocks for the usage of AI in the public sector: technically, infrastructurally, and legislatively. Finally, the talk will conclude by discussing how the public sector can best use and build AI by discussing procurement best practices and presenting best practice case studies.  

Bio: Keegan McBride is a departmental research lecturer in AI, Government, and Policy and Course Director of the Social Science of the Internet MSc program at the Oxford Internet Institute. He is an expert on the topic of digital government and has published research in leading journals and conferences on topics such as Artificial Intelligence in the public sector, open government data, government digitalization, and government interoperability and data exchange systems. In his research he aims to develop an understanding about the future trajectory of the state in the digital age by exploring the complex and co-evolutionary relationships between technology, society, and the state. 


Workshop 2 – 4th of May 2023

Our Workshop 2 will take place on the 4th of May of 2023 in London both in person, at The White Chapel building in East London, and online. The workshop will continue our discussions on AI applied to born-digital archives, particularly government archives.

Details about registration and the full program of speakers will be published soon. 

AI AI and Archives AI Ethics born-digital archives Dark archives email archives lunchtime online talk Open Data recordkeeping Records Management search interface transparency UX


Workshop 1. AI and born-digital archives: Challenges and opportunities

Thursday 26 January 2023

Through a series of talks and a round table, this day long workshop delved into the challenges and opportunities that AI offers to the management and use of digital born archives.

The workshop was an hybrid event (in London and online).

Invited speakers included Professor Stephanie Decker and Dr Adam Nix from the University of Birmingham; Dr Jenny Bunn from The National Archives and University College London; and Dr Tony Russell-Rose from Goldsmiths University.


Finding light in dark archives: Using AI to connect context and content in email.

Professor Stephanie Decker and Dr Adam Nix (University of Birmingham).

The practice of digital archival discovery is still emerging, and the approaches future research will take when using digital sources remain unclear. Archival practice has been shaped by paper-based, pre-digital sources and guides assumptions around how researchers will access and make use of such collections. Paradoxically, dealing with the increasing relevance of born-digital records is not helped by the fact that many born-digital collections remain dark, in part while questions of how they should be effectively made available are answered. Our research takes a user perspective on discovery within born-digital archives and seeks to promote more meaningful access to born-digital archives for researchers. In doing so, our work deals with the implications that unfamiliar archival technologies (including artificial intelligence) have on disciplinary traditions in the humanities and social science, with a specific focus on historical and qualitative approaches.

Our work in this area currently focuses on the issue of context within organisational email, and the challenges of searching and interpreting large bodies of email data. We are particularly interested in how effective machine-assisted search and multiple pathways for discovery can be used to open contextually opaque collections. Such access is likely to leverage a collection’s structural and content characteristics, as well as targeted archival selection and categorisation. We ultimately suggest that by combining relatively open user-led interfaces with pre-selective material, digital archives can provide environments suited to both the translation of existing research practices and the integration of more novel opportunities for discovery. Our presentation will summarise our progress in this area and reflect on the technical and methodological questions our work here has raised.

Putting principle into practice: Transparency, recordkeeping and AI.

Dr Jenny Bunn (The National Archives and University College London).

At the level of principle it is difficult to argue against the inherent good-ness of ideals such as transparency, accountability and fairness, but they have never been easy to put into practice. The increasingly advanced assistance technologies, generally placed under the label of AI, can now offer us further complicate this picture; offering as they do new possibilities for us to distance ourselves from both the consequences of our decisions and the very making of them. Recordkeeping has long acted to bridge this distance and this presentation will consider the new forms it may need to take to continue to ensure that accounts are rendered and explanations offered in the enduring spirit of transparency.

Searching, fast and slow: rethinking the query builder paradigm.

Dr Tony Russell-Rose (Reader in Computer Science, Goldsmiths University).

Knowledge workers such as information professionals, legal researchers and librarians need to create and execute search strategies that are comprehensive, transparent, and reproducible. The traditional solution is to use command-line query builders offered by proprietary database vendors. However, these are based on a paradigm that dates from the days when databases could be accessed only via text-based terminals and command-line syntax. In this talk, we explore alternative approaches based on a visual paradigm in which users express concepts as objects on an interactive canvas. This offers a more intuitive UX that eliminates error, makes the query semantics more transparent, and offers new ways to collaborate and share best practices.

Ethics for AI and Government Records

Roundtable with Dr Lise Jaillant, Dr Adam Nix, Prof. Stephanie Decker and Dr Jenny Bunn

AI AI and Archives AI Ethics born-digital archives Dark archives email archives lunchtime online talk Open Data recordkeeping Records Management search interface transparency UX


Lunchtime talk 1 with Giovanni Colavizza: Towards AI-supported records management

Wednesday 7th of December 2022

Artificial Intelligence (AI) is increasingly necessary in support of records management decisions. The scale, rapidity and complexity required in modern records management operations all contribute to make AI increasingly relevant to practitioners. This talk will start by reviewing recent contributions at the intersection of records management and AI, as well as highlight the many challenges still lying ahead of us. Preliminary considerations on the preconditions and requirements of using AI in support of records management will be proposed for discussion. These will be grounded in concrete examples and in an appreciation of the limits and potential of modern-day AI technology. 

Giovanni Colavizza is Assistant Professor of Digital Humanities at UvA, visiting researcher at The Alan Turing Institute and at the Centre for Science and Technology Studies (CWTS), Leiden University.

He did his PhD at the Digital Humanities Laboratory of the EPFL in Lausanne, working on methods for text mining and citation analysis of scholarly publications, and is co-founder of Odoma, a start-up offering customised machine learning techniques in the cultural heritage domain. Giovanni was also a Co-investigator on the Living with Machines project and convenes the AI for Arts interest group at the Turing.

Prior to joining the UvA, Giovanni has been part of the Research Engineering Group of The Alan Turing Institute, and a researcher at Leiden University (CWTS), the Leibniz Institute of European History in Mainz, and the University of Oxford. He studied computer science (BSc) and history (BA, MA) in Udine, Milan, Padua and Venice in Italy.

AI AI and Archives AI Ethics born-digital archives Dark archives email archives lunchtime online talk Open Data recordkeeping Records Management search interface transparency UX