EyCon Workshop 2: “Using Visual AI Applied to Digital Archives”— Highlights and Reflections

Devika Mehra (Loughborough University)

In what way does AI (Artificial Intelligence) impact traditional archival practice and theory? What do these new and emergent human-machine-based tools mean for future research and practice for refiguring and increasing access to all visual archives? These were the some of the key questions raised at the second EyCon workshop, ‘Using Visual AI Applied to Digital Archives.’ The workshop was organised by the EyCon project (Early Conflict Photography and Visual AI) held at the Imperial War Museum (London) on 8 June -9 June 2023.

The EyCon project is co-funded by the AHRC/Labex Past in the Present led by Dr Lise Jaillant (Loughborough University) and Dr Julien Schuh (Paris Nanterre University), in partnership with the University of Paris Cité and Prof Daniel Foliard. It aims at harnessing AI-reliant tools to analyse a large corpus of colonial photographs. EyCon’s database will include thousands of historical photographs documenting armed violence. The project is partnering with a network of archival institutions in France and in the UK. This workshop acted as a forum to negotiate with these pressing questions and the impact of visual AI techniques on digital archives. While Day 1 had three seventy-five-minute sessions, Day 2 had two practical sessions spread over three hours. Such a format fostered close engagement with French and UK cultural institutions impacted by the challenges of sensitive digitised archives.

On Day 1 Dr Lise Jaillant (UK PI-EyCon Project, Loughborough University) and Dr Maria Castrillo (Head of Collections Access and Research, Imperial War Museums) welcomed the guests in the morning. The first session then started with two presentations focusing on using computational methods to explore illustrations in digital visual collections. The session opened with the presentation, ‘Getting the Picture: Developing Computational Methods to Interrogate Large Datasets of Historical Book Illustrations,’ by Professor Julia Thomas and Irene Testini from Cardiff University. They introduced the methods of computational analysis used for The Illustration Archive database and their latest project, ‘Finding a Place’. The Layout Parser used to analyse captions for illustrations highlighted the potential of using computation methods for working with historical illustrations. Some challenges encountered in their project such as awkward layouts and poor quality of digitised images were raised adding to the ongoing debates of dealing with large corpus of historical data. This was complemented by the second presentation, ‘Using multimodal machine learning to study the visual representation of war,’ by Dr Thomas Smits (Universiteit Antwerpen). Drawing on the example of the Indonesian War of Independence (1945-1949), it explored how multimodal machine learning models can be used to analyse the visual representation of war at scale. It offered current and new possibilities of transitioning from monomodality to multimodality in Digital Humanities research.

After a short break, current computational methods and tools used for accessing, analysing, and disseminating visual and audio-visual collections were examined in the next session on new models of accessing digital archives using visual AI. Eléonore Plantard and Véronique Pontillon-Valedon (ECPAD: Établissement de communication et de production audiovisuelle de la Défense) began this session with their presentation, ‘The New ECPAD Digital Platform: When Workflows Question the Communication of Archives,’ on the different interfaces and softwares used for archiving sensitive images and handling their dissemination to the public. In making the images open to the public, GLAM industry deals with restrictions to communicability related to Heritage and Intellectual Property Codes and the Defence Code. Using their new database model, they demonstrated that new computational methods offer ways to GLAM practitioners to handle, preserve, and market sensitive visual archives.

This was followed by another thought-provoking presentation by Giacomo Alliata who shared the work he is doing with his colleagues, Yumeng Hou and Sarah Kenderdine, at the Laboratory of Experimental Museology eM+, EPFL. He focused on new computational methods and tools to augment moving image archives in his presentation, ‘Augmenting the Archival Experience of Embodied Knowledge Through Visual AI: a Computational Framework.’ It focused on how AI can be used for new ways of visualising motion- and posture-based performances and the relevance of understanding the body. He demonstrated how new and emergent computational practices offer ways of unlocking the massive audio-visual collections using two use cases: the Prix de Lausanne archive and the Hong Kong Martial Arts Living Archive.

The guests regrouped after lunch for the final session of the day, ‘Visual AI, Digital Archives and GLAM institutions: Challenges and Future Outputs.’ In his talk on ‘Back to basics: why technology alone is not enough to unlock the potential of visual archives,’ Geoff Browell (King’s College London Archives) addressed the challenge of finding a balance between technology and archival expertise to augment the potential of visual archives. Some of the key concerns raised included: the short-term/theme-based digitisation projects, the need for specialist cultural or historical knowledge to contextualise these collections, the communicability restrictions concerning ‘controversial’ collections, and the issue of long-term digital preservation. He pointed out the need to develop long term partnerships amongst interdisciplinary communities working with archives and to develop sustainable digital humanities projects. Dr Christiane Sibille and Nicole Graf (ETH-Bibliothek, Zurich) carried forward the discussion on use of machine learning methods and generating metadata in cultural heritage institutions in their presentation, ‘Collections as Data in the Context of Visual AI.’ Through their work on Image Archive, they demonstrated how archivists and volunteers used a crowdsourcing platform and computational tools to create high-quality metadata. Day 1 ended with a tour of the Second World War Galleries organised by Imperial War Museums team that focused on the narratives created by museum spaces for the wider public to understand the fraught history of war.

Day Two of the workshop consisted of two practical sessions; one led by Imperial War Museums colleagues, Alan Wakefield and Helen Mavin, and the second one led by the EyCon French team members, Dr Julien Schuh and Marina Giardinetti. The practical sessions offered an opportunity for an interactive discussion on war photo archives, sensitivity issues, and computational tools.

Alan Wakefield opened the first session with his presentation on ‘The IWM Photograph Archive & Images of warfare in the British Empire 1900 – 1929.’ He gave a critical overview of the Imperial War Museum Collections and used specific examples to highlight how his team works to develop a nuanced understanding of the histories these collections represent. He raised questions and challenges on how to reconsider the collections and focus on weaker areas to tell the narratives of every individual who contributed to the war effort hidden within these collections. This was followed by Helen Mavin’s presentation, ‘Addressing sensitivities at IWM,’ that brought the discussion forward on issues of sensitivity and reconsidering metadata and accessibility of photographs and films. Using recent IWM projects with external collaborators (including the ‘Provisional Semantics’ project), she noted collection development and digitisation programme and the challenges of incorporating multiple perspectives in an evolving digitised national collection.

The second practical session, ‘Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database,’ presented the EyCon database, visual similarity model, and the tools developed in the context of the EyCon project to make visible conflicts that are rarely present in contemporary visual culture. They introduced the Layout Parser model and the techniques that they have used to identify sensitive content and to add trigger warnings with the help of AI. They explained how they used computational tools to train the database and then use it on our historical dataset. This method allowed them to group together similar images (both images and captions for the pictures). They demonstrated how they took all the metadata as well as the visual aspects of the pictures in the database to enable multimodal search functionality. This enabled them to link pictures from different databases with the help of this tool.

In short, the sessions covered wide-ranging issues concerning the potential and challenges of AI applied to archival practices in GLAM institutions, leading to a reassessment of large visual collections and historical databases.

---

---