Over the course of nine months, the AHRC-funded LUSTRE project has made significant progress in its exploration of using Artificial Intelligence (AI) to unlock born-digital and digitised government archives. The project has already delivered a range of activities, including more than 50 semi-structured interviews with professionals in the GLAM sector, computer scientists, and scholars. Additionally, there have been four online lunchtime talks and two hybrid workshops.
In this blog post, we will focus on the two hybrid workshops organised in collaboration with our project partner, the Cabinet Office. These workshops took place at the Central Digital and Data Office in Whitechapel, London, as well as online. The talks from both workshops have been recorded and can be accessed on the LUSTRE website through the following links:
The primary objective of these workshops was to foster the establishment of a network of professionals in the archival field and academic researchers who share an interest in exchanging knowledge regarding the opportunities, challenges, and potential risks associated with AI for accessing, managing, and using born-digital archives.
Summary of Workshop 1
Invited speakers to the first workshop in January 2023 were Professor Stephanie Decker and Dr Adam Nix from the University of Birmingham; Dr Jenny Bunn from The National Archives; and Dr Tony Russell-Rose from Goldsmiths University of London.
The day began with a presentation by Professor Stephanie Decker and Dr Adam Nix, who discussed the practices of digital archival discovery, focusing specifically on the use of AI to connect context and content in emails. Their talk provided valuable insights from a user perspective, emphasising the significance of meaningful access to born-digital archives for researchers, particularly those in the humanities and social sciences.
Next, Dr Jenny Bunn delved into the ethical dimension of using advanced technologies, which are often categorised under the umbrella term of AI. She highlighted the challenges that the introduction of AI presents to the delicate balance between transparency, accountability, and fairness in recordkeeping activities. Dr Bunn explored new approaches that can maintain transparency and accountability in recordkeeping practices amidst the use of AI, emphasising the importance of good governance.
The third talk of the day was led by Dr Tony Russell-Rose, and it showed how the LUSTRE project effectively brings together researchers and professionals from the GLAM sector, digital humanities, and computer science. Dr Russell-Rose’s presentation focused on access to databases and proposed approaches that move away from traditional command-line query builders. His talk Searching, fast and slow: rethinking the query builder paradigm centred on a platform called 2Dsearch which redefines ‘advanced search’, reducing syntactic errors, enhancing semantic transparency, and providing support for reuse and optimisation.
The day concluded with a thought-provoking roundtable discussion featuring Lise Jaillant, Adam Nix, Stephanie Decker, and Jenny Bunn, centred around ethical issues in AI and government records. The speakers further expanded on the ethical dimension of conducting research on digital archives using AI-assisted technologies. They unanimously emphasised the need for transparent, explainable, and accountable processes and technologies.
Adam Nix drew attention to the risks and contingent liabilities of already available collections when novel forms of searches unveil new materials. Stephanie Decker reflected on the challenges of anonymisation and privacy, highlighting the delicate balance between anonymising data and preserving valuable contextual research information. Jenny Bunn emphasised the active effort archivists must undertake to avoid embracing the ‘myth of objectivity’ when adopting new technologies, asserting that archival practice is inherently subjective and necessitates the archivist’s engagement as a gatekeeper. She also discussed the increased sensitivity risks associated with larger and more complex datasets.
Overall, the first workshop provided valuable insights into the intersection of AI and born-digital archives, exploring both the practical and ethical dimensions of using AI technologies for archival research.
Summary of Workshop 2
The second LUSTRE workshop, titled ‘AI and Born-Digital Archives in the Government Sector and Beyond: Challenges and Opportunities’, was once again held in Whitechapel. The workshop took place in May 2023 and, like the previous one, it was a hybrid event with five presentations, including two remote presenters.
Invited speakers included Dr Keegan McBride from the Oxford Internet Institute, James Lappin from Loughborough University, Dr Lise Jaillant from Loughborough University (PI of the LUSTRE project), John Sheridan from The National Archives, and Professor Jason R Baron from the University of Maryland.
Dr Keegan McBride commenced the day with his talk, ‘Artificial Intelligence in the Public Sector: Separating Myth from Reality’. He focused on differentiating the development of AI in the private sector from its applications in the public sector. Dr McBride highlighted the necessary building blocks for effective AI usage in the public sector, including technical, infrastructural, and legislative aspects. He also presented best practice case studies to illustrate AI implementation.
James Lappin followed with a presentation on ‘AI and the Management of Email Accounts Over Time’, drawing on his doctoral research. He explored the impact of AI on access permissions and retention rules within digital corporate systems, particularly focusing on email. His talk covered various aspects, including the impact of email on recordkeeping, the effective organisation of correspondence, and key strategic choices when applying AI to email management.
Dr Lise Jaillant, the Principal Investigator of the LUSTRE project, provided her perspective on the challenges posed by the transition from traditional archives to digital archives for professionals and academic researchers. She shared results from interviews conducted with GLAM professionals, researchers, and archive users. The participants highlighted two pressing obstacles in accessing digital records: mistrust between stakeholders and mistrust of technology. Dr Jaillant emphasised the importance of facilitating knowledge transfer, fostering collaborations between GLAM professionals, academics, and other stakeholders, and enabling access to archival materials as well as the discovery of their existence.
Following a brief question and answer session involving both in-person and online participants, the workshop broke for lunch. The afternoon session commenced with a talk by John Sheridan, the Digital Director of The National Archives, titled ‘Navigating the Maelstrom (Looking for Lighthouses)’. He reflected on the rapid advancement of AI and large language models and the uncertainties these technologies bring. He presented strategic interventions that digital archives can implement to leverage AI’s opportunities. These interventions encompassed content classification and organisation, new text generation (including summaries and transcriptions), image/video generation based on instruction, and data transformation/manipulation based on instruction.
The day concluded with a remote guest speaker, Professor Jason R Baron (University of Maryland). His talk focused on the process which should lead to the transfer of permanent federal government records to the National Archives and Records Administration (NARA) solely in electronic or digital formats by June 2024.
He discussed the challenges NARA faces in meeting the 2024 deadline and the limitations presented by the Freedom of Information Act (FOIA) in providing timely access to archival records. Professor Baron also explored how AI tools can assist in ensuring real public access to government archives. He emphasised the role of AI in supporting archivists, records managers, and lawyers in filtering sensitive content from public records and recommended continued experimenting with Technology Assisted Review methods for efficiently discovering records and accurately segregating personal content. He also suggested exploring how generative AI could provide narratives explaining why documents or portions of documents have been withheld under freedom of information laws. Professor Baron concluded his talk by encouraging the embrace of AI, considering it not merely as a black box, but as a ‘gift’ to archivists to improve access to vast digital collections.