EyCon workshop 2: ‘Using Visual AI applied to Digital Archives’ – Recordings and Presentation Slides

Imperial War Museum (London) – 8 June 2023-9 June 2023 

The second EyCon workshop, ‘Using Visual AI Applied to Digital Archives’ was held at the Imperial War Museums (London) on 8 June 2023-9 June 2023. The EyCon Project has used AI-reliant tools to analyse a large corpus of colonial photographs. EyCon’s database includes thousands of historical photographs documenting armed violence. The EyCon project is co-funded by the AHRC/Labex Past in the Present led by Dr Lise Jaillant (Loughborough University) and Dr Julien Schuh (Paris Nanterre University), in partnership with the University of Paris Cité and Prof Daniel Foliard. It is partnering with a network of archival institutions in France and in the UK.

We are pleased to announce that the recordings of the presentations, as well as slides, are now available (see links below).

 Day 1

Thursday, 8 June (10:00 to 16:15 GMT)

SESSION 1 – Using Computational Methods to Explore Illustrations in Digital Visual Collections

10:30 – 11:00 Professor Julia Thomas (Cardiff University) and Irene Testini (Cardiff University) – Getting the Picture: Developing Computational Methods to Interrogate Large Datasets of Historical Book Illustrations

The talk introduces the methods of computational analysis that we have used to interrogate a big dataset of around a million illustrations from books spanning the seventeenth to the twentieth centuries and covering diverse subject matter, including literature, history, geography, and philosophy. This work has been developed in the context of two major AHRC-funded projects: the first led to the creation of the largest online searchable resource dedicated to book illustrations: The Illustration Archive (https://illustrationarchive.org.uk); and the second project uses computer vision tools to analyse the images in this dataset in terms of their specific representation of places and the people associated with them. Particular attention is given to the ways in which we have made these images searchable using a variety of methods from crowdsourcing to caption capture and computer vision. These methods have revealed culturally sensitive images, especially those relating to the historical representation of people and racial and ethnic groups. The focus of our latest project, ‘Finding a Place’, is to use computational methods to actively explore these problematic illustrations and to identify how these images proliferate and circulate and how the ‘stereotype’ (which is itself, of course, a printing term) is constructed.

11:00 – 11:30 Dr. Thomas Smits (Universiteit Antwerpen) – Using multimodal machine learning to study the visual representation of war

Most research in Digital Humanities is monomodal, meaning that the object of analysis is either textual or visual. Recently developed multimodal deep learning models, offer new possibilities to explore, analyze, and enrich multimodal historical collections at scale. Trained to predict which images and texts belong together, these models can be applied to a wide variety of text-to-image, image-to-image, and image-to-text prediction tasks without the need to train them for a specific task or on specific data. I showcase the possibilities of multimodal machine learning by applying it to a collection of 26,000 digitized photographs of the Dutch Army Information Service, which were made during the Indonesian War of Independence (1945-1949). Previous research suggests that this collection presents a highly sanitized and ‘bloodless’ image of this war. Can we use multimodal machine learning to uncover patterns in the visual representation of violence in this collection? On a more general level, I argue that multimodal machine learning enables us to move past the hyper-focus on a small number of ‘iconic’ war photographs and study the visual representation of armed conflict at scale.

SESSION 2 – New Models of Accessing Digital Archives using Visual AI

12:00 – 12:30 Eléonore Plantard, and Véronique Pontillon-Valedon (ECPAD) – The New ECPAD Digital Platform: When Workflows Question the Communication of Archives

The origin of the digital platform project of the Establishment of Communication and Audiovisual Production of Defence (ECPAD), launched in the mid 2010’s, stems primarily from the need to accelerate the availability of the institution’s archives to the public. Combining a new archiving system, improved archival processing and dissemination on the Internet, ImagesDéfense project is an opportunity for the ECPAD to completely redesign and rethink its workflow by developing new tools offered by digital resources. A new conceptual data model and different interfaces are set up, allowing each processing step: from the transfer through a module of collecting archives, to the dissemination in the website, imagesdefense.gouv.fr, including the preservation, transcoding and even the marketing of these archives. These new and partially automated workflows distinguish the analog images from the digital archives, guarantee that the archiving complies with time requirements, but also induce both human control and new questions about the communication and dissemination of archives. In this matter, the institution must take into consideration the heritage code, the intellectual property code and the defence code. The various interfaces and softwares for accessing images, in the media library in situ or on the Internet, must make it possible to differentiate on the one hand the rules for the communicability of images, on the other hand their classification levels and finally allow them to be editorialized according to their sensitivity.

12:30 – 13:00 Giacomo Alliata (EPFL-CDH-DHI-eM+) – Augmenting the Archival Experience of Embodied Knowledge Through Visual AI: a Computational Framework

With the burgeoning use of digital technologies in safeguarding intangible and living heritage, memory institutions have produced a significant body of material, yet to be made available to the public. The quest for new ways to unlock these massive collections has intensified, especially the knowledge embedded in complex data formats, such as audiovisual and motion capture recordings. Aiming to address the gap between the living heritage archive and embodied knowledge experience, this work inspects a computational pipeline incorporating machine intelligence, archival science, and digital museology. By reflecting on how embodied knowledge is sustained and potentially valorised, we devise approaches primarily based on vision-based extraction and estimation, movement analysis, aesthetic visual reconstruction and machine learning. The goal is to augment the archival experience with new modes of exploration, representation, and embodiment. This presentation reports the procedures and computational tools inspected through two use cases. In the first case, datafication enhancement is employed to allow archival exploration via embodied cues in the Hong Kong Martial Arts Living Archive, a multimodal archive that leverages state-of-the-art motion capture tools to document the living practices of martial arts. In another experimentation with the Prix de Lausanne archive, a collection of 50 years of video recordings of dance performances from the internationally renowned competition Prix de Lausanne, we augment the visual representation of the dancers’ movements through a range of aesthetic expressions and visual metaphors. Though holding different application purposes, both projects operate on the proposed framework and extract archive-specific features to create meaningful representations, which reveals the versatile applications that computational capacities can achieve. The practices also represent a model of interdisciplinary involvement where the archivists, computists, artists and knowledge holders join hands to renew strategies for archival exploration and heritage interpretation in a new light.

SESSION 3 – Visual AI, Digital Archives and GLAM institutions: Challenges and Future Outputs

14:15 – 14:45 Geoffrey Browell (King’s College London) – Back to basics: why technology alone is not enough to unlock the potential of visual archives

Drawing on practical examples from King’s College London Archives’ holdings of nineteenth and twentieth century military and medical history, this talk explores some of the challenges in acquiring, handling, describing, and facilitating access to often sensitive legacy visual collections, and their digital surrogates. These include mountainous cataloguing backlogs, a pressing shortage of professional conservators, the absence of specialist cultural or historical knowledge to identify and contextualise collections and institutional caution and inertia when confronted with collections that have been identified as controversial or deemed politically ‘difficult’. It argues that technology is not enough: for high-tech approaches, including AI, to make a meaningful contribution to problem-solving, they need to work with the grain of professional practice, audience expectations, funding constraints and employers’ appetite for risk. Ideally, they should provide short cuts for hard-pressed archivists, teachers, and interpreters of visual history, rather than another complex layer of technology that is difficult to understand, implement and use. The talk explores the ways in which standalone digital humanities projects can be built into sustainable and durable services to find new uses for visual archives, and how the best examples facilitate cultural change that enables the breaking down of professional silos and encourages meaningful co-working by archivists, scholars, and technologists.

14:45 – 15:15 Dr Christiane Sibille (ETH-Bibliothek, Zurich) and Nicole Graf (ETH-Bibliothek, Zurich) Collections as Data in the Context of Visual AI

High-quality metadata has always been essential for the use of collections in GLAMs. The use of machine learning methods opens numerous new possibilities in this context, but also poses new challenges for cultural heritage institutions. The image archive of the ETH Library in Zurich addresses this situation by generating and curating not only one but multiple metadata layers. The basis is formed by classic manual archive and library cataloguing. In addition, a group of external volunteers assists the Image Archive (https://ba.e-pics.ethz.ch) in enriching sparse or missing metadata (approx. 20,000 items per year). 35’000 images per year are georeferenced on a three-dimensional globe using the crowdsourcing platform sMapshot (https://smapshot.heig-vd.ch/owner/ethz). In the past months, the Image Archive started to include machine learning tools in its portfolio to broaden its experience in this field. First, key metadata fields were converted into English with the help of automatic translations. In addition, commercial software was used to recognize and label the nearly one million digitized images. The resulting keywords were integrated into the search interface, so far without manual post-processing, as a public pilot. In our paper, we take a closer look at the work with these different levels, analyze the resulting challenges and provide an outlook on future activities. In doing so, we want to contribute to the question of how archives can implement the idea of collections as data in the context of visual AI. We are particularly interested in how GLAM institutions can overcome data silos and create a data cycle that is characterized by the principles of FAIR and ideally generates added value for all stakeholders involved.

Day 2

Friday, 9 June (09:00 to 13:45 GMT)

09:30 – 10:30 Alan Wakefield (Imperial War Museums) and Helen Mavin (Imperial War Museums) – Developing the Photograph Archive at Imperial War Museums

Alan has led a discussion in his talk,The IWM Photograph Archive & Images of warfare in the British Empire 1900 – 1929, around the formation and development of the photograph collection following the establishment of the Imperial War Museum in 1917, with a focus on imagery documenting conflict and British military activity in India and across Africa during the period 1900-1929. Examples from the collection are used to highlight sensitivities that are commonly encountered and how his team are strategically working with the collection to develop a deeper and more nuanced understanding of the histories they represent. In her talk, “Addressing Sensitivities at IWM”, Helen has presented on projects recently undertaken at IWM with external collaboration to review and reconsider how collections, particularly photographs and film, are accessed and described, and by who. Helen has shared findings from recent AHRC funded projects.

10:45 – 12:45 AI tools practical session by Dr Julien Schuh (EyCon project French Co-I) and team (2 hours) – Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database

The aim of this session, Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database”, is to present some of the tools developed in the context of the EyCon project.

– Image segmentation: In order to be able to analyse the circulation of images in the press and in the albums of the time, we must first be able to extract these illustrations. We present the protocols developed to produce a corpus of press illustrations.

– Image processing and similarity detection: The quality of the photographs differs greatly between the original negatives and prints and the press reproductions. In order to be able to apply object recognition or similarity algorithms on these images, we re-train models with images modified to mimic the transformations produced by photomechanical reproduction. This improves similarity recognition results, both for identifying duplicate images and for aggregating images with similar content.

– Sensitive pictures and texts recognition: We discuss the solutions envisaged to identify sensitive contents, both in terms of visual contents and of captions and period descriptions.

– Pose estimation: The automatic recognition of people’s poses in photographs allows us to identify evolutions in the staging of bodies in the photography of violence.

– Database: The presentation of the EyCon database has allowed us to get to grips with the site and to explore these corpora.

The IWM Photograph Archive & Images of warfare in the British Empire 1900 – 1929

This is the presentation, “The IWM Photograph Archive & Images of warfare in the British Empire 1900 – 1929”, delivered by Alan Wakefield (Imperial War Museums) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database

This is the presentation, “Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database”, delivered by Dr Julien Schuh (EyCon project French Co-I) and team at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Getting the Picture: Developing Computational Methods to Interrogate Large Datasets of Historical Book Illustrations

This is the presentation, “Getting the Picture: Developing Computational Methods to Interrogate Large Datasets of Historical Book Illustrations”, delivered by Professor Julia Thomas (Cardiff University) and Irene Testini (Cardiff University) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Using Multimodal Machine Learning to Study the Visual Representation of War

This is the presentation, “Using Multimodal Machine Learning to Study the Visual Representation of War”, delivered by Dr Thomas Smits (Universiteit Antwerpen) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Back to Basics: Why Technology Alone is Not Enough to Unlock the Potential of Visual Archives

This the presentation, “Back to Basics: Why Technology Alone is Not Enough to Unlock the Potential of Visual Archives”, delivered by Geoffrey Browell (King’s College London) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Augmenting the Archival Experience of Embodied Knowledge Through Visual AI: a Computational Framework

This is the presentation, “Augmenting the Archival Experience of Embodied Knowledge Through Visual AI: a Computational Framework”, delivered by Giacomo Alliata (EPFL-CDH-DHI-eM+) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

Collections as Data in the Context of Visual AI

This is the presentation, “Collections as Data in the Context of Visual AI“, delivered by Dr Christiane Sibille and Nicole Graf (ETH-Bibliothek, Zurich) at the second EyCon workshop, ‘Using Visual AI applied to Digital Archives’, held at the Imperial War Museum (London) on 8 June-9 June 2023. Hosted by Loughborough University in partnership with Imperial War Museums Institute, this was the second workshop held by the EyCon (Early Conflict Photography and Visual AI) project.

EyCon Workshop 2: “Using Visual AI Applied to Digital Archives”— Highlights and Reflections

Devika Mehra (Loughborough University)

In what way does AI (Artificial Intelligence) impact traditional archival practice and theory? What do these new and emergent human-machine-based tools mean for future research and practice for refiguring and increasing access to all visual archives? These were the some of the key questions raised at the second EyCon workshop, ‘Using Visual AI Applied to Digital Archives.’ The workshop was organised by the EyCon project (Early Conflict Photography and Visual AI) held at the Imperial War Museum (London) on 8 June -9 June 2023.

The EyCon project is co-funded by the AHRC/Labex Past in the Present led by Dr Lise Jaillant (Loughborough University) and Dr Julien Schuh (Paris Nanterre University), in partnership with the University of Paris Cité and Prof Daniel Foliard. It aims at harnessing AI-reliant tools to analyse a large corpus of colonial photographs. EyCon’s database will include thousands of historical photographs documenting armed violence. The project is partnering with a network of archival institutions in France and in the UK. This workshop acted as a forum to negotiate with these pressing questions and the impact of visual AI techniques on digital archives. While Day 1 had three seventy-five-minute sessions, Day 2 had two practical sessions spread over three hours. Such a format fostered close engagement with French and UK cultural institutions impacted by the challenges of sensitive digitised archives.

On Day 1 Dr Lise Jaillant (UK PI-EyCon Project, Loughborough University) and Dr Maria Castrillo (Head of Collections Access and Research, Imperial War Museums) welcomed the guests in the morning. The first session then started with two presentations focusing on using computational methods to explore illustrations in digital visual collections. The session opened with the presentation, ‘Getting the Picture: Developing Computational Methods to Interrogate Large Datasets of Historical Book Illustrations,’ by Professor Julia Thomas and Irene Testini from Cardiff University. They introduced the methods of computational analysis used for The Illustration Archive database and their latest project, ‘Finding a Place’. The Layout Parser used to analyse captions for illustrations highlighted the potential of using computation methods for working with historical illustrations. Some challenges encountered in their project such as awkward layouts and poor quality of digitised images were raised adding to the ongoing debates of dealing with large corpus of historical data. This was complemented by the second presentation, ‘Using multimodal machine learning to study the visual representation of war,’ by Dr Thomas Smits (Universiteit Antwerpen). Drawing on the example of the Indonesian War of Independence (1945-1949), it explored how multimodal machine learning models can be used to analyse the visual representation of war at scale. It offered current and new possibilities of transitioning from monomodality to multimodality in Digital Humanities research.

After a short break, current computational methods and tools used for accessing, analysing, and disseminating visual and audio-visual collections were examined in the next session on new models of accessing digital archives using visual AI. Eléonore Plantard and Véronique Pontillon-Valedon (ECPAD: Établissement de communication et de production audiovisuelle de la Défense) began this session with their presentation, ‘The New ECPAD Digital Platform: When Workflows Question the Communication of Archives,’ on the different interfaces and softwares used for archiving sensitive images and handling their dissemination to the public. In making the images open to the public, GLAM industry deals with restrictions to communicability related to Heritage and Intellectual Property Codes and the Defence Code. Using their new database model, they demonstrated that new computational methods offer ways to GLAM practitioners to handle, preserve, and market sensitive visual archives.

This was followed by another thought-provoking presentation by Giacomo Alliata who shared the work he is doing with his colleagues, Yumeng Hou and Sarah Kenderdine, at the Laboratory of Experimental Museology eM+, EPFL. He focused on new computational methods and tools to augment moving image archives in his presentation, ‘Augmenting the Archival Experience of Embodied Knowledge Through Visual AI: a Computational Framework.’ It focused on how AI can be used for new ways of visualising motion- and posture-based performances and the relevance of understanding the body. He demonstrated how new and emergent computational practices offer ways of unlocking the massive audio-visual collections using two use cases: the Prix de Lausanne archive and the Hong Kong Martial Arts Living Archive.

The guests regrouped after lunch for the final session of the day, ‘Visual AI, Digital Archives and GLAM institutions: Challenges and Future Outputs.’ In his talk on ‘Back to basics: why technology alone is not enough to unlock the potential of visual archives,’ Geoff Browell (King’s College London Archives) addressed the challenge of finding a balance between technology and archival expertise to augment the potential of visual archives. Some of the key concerns raised included: the short-term/theme-based digitisation projects, the need for specialist cultural or historical knowledge to contextualise these collections, the communicability restrictions concerning ‘controversial’ collections, and the issue of long-term digital preservation. He pointed out the need to develop long term partnerships amongst interdisciplinary communities working with archives and to develop sustainable digital humanities projects. Dr Christiane Sibille and Nicole Graf (ETH-Bibliothek, Zurich) carried forward the discussion on use of machine learning methods and generating metadata in cultural heritage institutions in their presentation, ‘Collections as Data in the Context of Visual AI.’ Through their work on Image Archive, they demonstrated how archivists and volunteers used a crowdsourcing platform and computational tools to create high-quality metadata. Day 1 ended with a tour of the Second World War Galleries organised by Imperial War Museums team that focused on the narratives created by museum spaces for the wider public to understand the fraught history of war.

Day Two of the workshop consisted of two practical sessions; one led by Imperial War Museums colleagues, Alan Wakefield and Helen Mavin, and the second one led by the EyCon French team members, Dr Julien Schuh and Marina Giardinetti. The practical sessions offered an opportunity for an interactive discussion on war photo archives, sensitivity issues, and computational tools.

Alan Wakefield opened the first session with his presentation on ‘The IWM Photograph Archive & Images of warfare in the British Empire 1900 – 1929.’ He gave a critical overview of the Imperial War Museum Collections and used specific examples to highlight how his team works to develop a nuanced understanding of the histories these collections represent. He raised questions and challenges on how to reconsider the collections and focus on weaker areas to tell the narratives of every individual who contributed to the war effort hidden within these collections. This was followed by Helen Mavin’s presentation, ‘Addressing sensitivities at IWM,’ that brought the discussion forward on issues of sensitivity and reconsidering metadata and accessibility of photographs and films. Using recent IWM projects with external collaborators (including the ‘Provisional Semantics’ project), she noted collection development and digitisation programme and the challenges of incorporating multiple perspectives in an evolving digitised national collection.

The second practical session, ‘Visual AI for the EyCon Project, Sensitive Images, and the EyCon Database,’ presented the EyCon database, visual similarity model, and the tools developed in the context of the EyCon project to make visible conflicts that are rarely present in contemporary visual culture. They introduced the Layout Parser model and the techniques that they have used to identify sensitive content and to add trigger warnings with the help of AI. They explained how they used computational tools to train the database and then use it on our historical dataset. This method allowed them to group together similar images (both images and captions for the pictures). They demonstrated how they took all the metadata as well as the visual aspects of the pictures in the database to enable multimodal search functionality. This enabled them to link pictures from different databases with the help of this tool.

In short, the sessions covered wide-ranging issues concerning the potential and challenges of AI applied to archival practices in GLAM institutions, leading to a reassessment of large visual collections and historical databases.

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search