Workshop Report: Working with Potentially Sensitive Digital Photographic Archives with the Help of Multimodal Deep-learning

On the 6th of March, the EyCon Project (funded by the Labex “Les Passés dans le Présent” and the AHRC) organized a workshop on the potential benefits and problems of using multi-modal artificial intelligence (AI) approaches to work with archives of sensitive digitized historical photographs. The workshop was attended by archivists and museum professionals from the Établissement de la Communication et de la Production Audiovisuelle de la Défense (ECPAD), the Service Historique de la Défense (SHD), and La Contemporaine museum and archive, and was held at La Contemporaine. The workshop’s aim was to collectively explore the possibilities for computational and machine-learning approaches to making potentially sensitive colonial archives more navigable and discoverable. The EyCon team presented a theoretical pipeline or workflow with which heritage institutions could carry out the computer-assisted analysis of images, enriching their metadata with a number of tools. In particular, we aimed to think through ways that these tools could be used to identify and contextualize sensitive images. After the presentation, the floor was opened for a dialogue between the research team and the archival partners about the contemporary working practices around metadata and sensitive images, and the possibilities and challenges of these technologies.

            After introductions, EyCon postdoctoral research fellow Dr. Jonathan Dentler introduced the project and the large questions that have been recently introduced in the public sphere and in scholarship about the ethics and politics of disseminating sensitive images online, particularly those depicting colonial violence. He suggested that a key proposition of the Eycon project is that, “critical and multimodal artificial intelligence offers a means of balancing the ethical treatment of material with the necessity of improving access to colonial archives for researchers and the public.” Using examples from the holdings of the ECPAD, he also introduced the issue of metadata, and how archives handle what terms to attach to photographs in order to contextualize them and make them searchable. AI could potentially solve the problem of scale in the task of adding metadata to large photography collections. Furthermore, by automatically detecting visual similarity, it could potentially help trace image circulation both transnationally and trans-imperially among linked archives, helping to de-silo national and imperial histories.

Another computational tool that could be used to enrich metadata are various forms of object detection and “safe search” functionalities that automatically detect violence and other forms of sensitive material.

Figure 1: Test of the Google Vision API “safe search” function on a photograph of the corpse of a german soldier from the fonds Valois.

(Figure 1) Unfortunately, as Jonathan Dentler discussed, such tools have been shaped by the legal and commercial needs of companies, and are not technically, scientifically, or ethically suited to historical material. (Figure 2) Indeed, out of 199 images extracted from the fonds Valois photo albums created by the French Service Photographique de l’Armée (SPA) during World War One and held by La Contemporaine that contain the word “corpse” in their textual metadata, only 12% were recognized by Google Vision API’s “safe search’ function as “possibly” or “probably” containing violence.

Figure 2: Results of a Google Vision API “Safe Search” with a corpus of 199 images containing the word “cadavre” or “corpse” in their metadata in the Valois albums.

            EyCon’s newly hired computer vision engineer Mohamed Salim Aissi, who recently earned a masters degree in computer science from Sorbonne Université, then introduced the technical details on visual similarity operations, explaining the architecture of auto-encoding that permits the extraction of information from images in order to determine similarity.

Figure 3: Visual Similarity method results.

(Figure 3) Using this technique, he was able to automatically find similar objects in the Valois albums, including, for example, shots of dugouts and trenches, injured faces, diplomatic meetings pictured from slightly different angles, and statues.

Figure 4: Images of statues made by the SPA and grouped in the same zone of the pixplot.

(Figure 4) These images were represented in clusters in a pixplot. Jonathan Dentler pointed out, however, that without fine-tuning of this approach, it could create false positives, for example grouping a photograph of burned bodies inside a destroyed tank with simple landscape shots on the basis of the tonal values of the pixels distributed across the image. The EyCon project believes that one way out of this problem is to treat textual metadata as a vector in the process of detecting similarity across images, so that the computation includes terms like “corpse” in the metadata as a hint that there may be violence in the picture, in addition to detecting objects or similarity. Such a process will likely be the subject of a second workshop in May.

            At this point, Jonathan Dentler also discussed the problem of implicit violence in photographs, using the example of the album La Prise de Samory, which is conserved by the SHD. An image that is apparently inoffensive can often have violent implications.

Figure 5: « Samory dans les rue de Saint Louis ». Album La Prise de Samory. Service Historique de la Défense (Vincennes).

(Figure 5) Extracted from its context, this image seems to show a simple parade or procession down the street of a town. In fact, this example is taken from an album created in 1899 by a French officer just after the conclusion of the war between the French army and Samory Touré’s Wassoulou Empire, which, at its apogee during the 1880s, extended over parts of present-day Guinea, Mali, and the Ivory Coast. In 1898 the French forces captured Touré and exiled him to Gabon.This photo shows the procession of Touré in Saint Louis after his capture. It is therefore a document of a form of public humiliation and a display of power, and is potentially sensitive in a way that simple object detection could not account for.

             Project leader Dr. Julien Schuh then presented on issues about how to make a pipeline for artificial intelligence work designed for scientific and university purposes. He discussed the need for what experts call “explainable AI,” which is a notion that the technology should not be “black boxed” and opaque from the perspective of its users. This involves, for example, making training datasets publicly available so that they can be critiqued. A good pipeline would also involve input from experts and source communities, perhaps through platforms such as Zooniverse. Such input could then be verified by archive professionals. From a meta-level perspective of the organization of archival materials, Dr. Schuh discussed the ways in which artificial intelligence could build new connections at scale that would allow researchers to draw new connections that were hidden by layers of archivation handed down by colonial institutions.

             The Eycon Project’s research engineer, Soumik Mallick, discussed the way in which various kinds of artificial intelligence tools such as pose detection or object detection can be deployed on datasets that have been extracted using the tool Layout Parser. He also explored reasons behind failures in pose recognition on historical photos. Data architect Marina Giardinetti discussed how the project uses Layout Parser in order to extract images from complex album and periodical page designs. Only when the images are extracted can “clean” datasets be created that would be useful in testing or producing tools informed by artificial intelligence. She explained how this methodology is build into the overall structure of EyCon’s database, which will survive after the end of the project, and demonstrated a “beta” version of the public-facing database.

            After a short coffee break, the workshop reconvened in an open discussion format to gather the responses of the project’s archival partners in attendance. Representing the ECPAD were Véronique Pontillon (chargée des actions scientifiques) and Eléonore Plantard (chargée des collections entrées par voie extraordinaire). They explained that the choice was made at ECPAD to put warning screens up on an image-by-image basis for potentially sensitive images due to the fact that they were being put on line, and that this involved a considerable amount of work. One employee within the service is responsible for validating these sensitivity notices. On the back-end of the database, this person can check a box which implements a warning on the selected contents to hide potentially disturbing images. There is great difficulty in making this determination for certain limit cases—images that do not necessarily appear violent at first sight, such as burned corpses that are not recognizable as such, or the moments just before an execution rather than during the act itself. While there is some direction from the heritage code and the principle of one’s right to their image and respect of human dignity, there is not a specific framework for deciding whether an image is sensitive or not. Instead, a trained expert takes all final decisions on this question, taking into account the suggestions of those who processed the documents.

            The database gives researchers the chance to alert archivists to consult the record to add additional information, including about sensitivity. These archivists can then add information on sensitive images in text form to the documentary records. Ideally, these notices are included for each image in a given file, but nevertheless occasionally there is only one notice for a whole batch of images. Another solution that was devised was putting up warning screens over sensitive images. Eléonore Plantard cautioned that it is important to realize that this platform is more of a publication tool than a tool for putting everything at the disposition of the researcher. There are files in the archives that contain pictures, without corresponding images put up on the site. These website pages simply alert researchers of the presence of a set of documents held in physical form only. The images put up on line are necessarily images on which ECPAD archivists have worked in order to enrich the metadata, and which have been considered from the point of view of potential sensitivity, but also from the point of view of copyright law.

            Julien Gueslin, head of the museum department at La Contemporaine, explained that the photographic prints in the collection are put online using the national tool Calames in EAD-XML with a number of headings attached, including persons, places, and subjects, which enables research within the corpus concerned. He felt sure that EyCon’s work has great potential to enrich these descriptions. As with other private archives in the public domain, these images are under open license, so the archive user can access them directly without mediation. This means that the archivist needs to consider what kind of information to furnish on the webpage. For other photographic archives that are private and under copyright, consultation can instead only take place on the premises, which in the case of sensitive images provides a form of mediation. Before the accession into the public archive from a private source, these documents can be submitted to rules that concern archival use and potentially to the need for special authorizations or exemptions from public use. Typically, however, the donor singles out these exceptions not on the basis of sensitive content but rather out of concerns for familial or private life. 

            Concerning the Valois Albums in particular, Julien Gueslin pointed out that it is interesting to conceive of these photos produced by the SPA as sensitive images today, given that during the war they were made available publicly for journalists or diplomats for propaganda purposes. They were controlled in terms of censorship and propaganda, but nonetheless widely disseminated without seeming to cause a public outcry at the time or in later years. He pointed out that this demonstrates how definitions of sensitivity can change and evolve over time. While there may always be new criticisms of the collection according to new definitions of sensitivity, most of the users of the archive are historians, who are more cognizant of the bias inherent in all such archival records. Overall, Julien Gueslin questioned whether it was the role of archivists to decide what is sensitive and prevent access to historical data. The fact of being an archive—a space dedicated to research—means that this institution is somewhat set aside from concerns about sensitivity in other spaces. In an archive one can contextualize documents in a way not possible in the digital media space, which provides some cover from pressure to censor or suppress data.

More than blocking what is sensitive, he believes that users most want to enrich data further. For example, Julien Gueslin suggested that it would be interesting to automatically enrich the location data attached photos, or to give these automatically-located place-names in different languages, particularly in areas that have many different linguistic groups or where the sovereignty and dominant language has changed, such as in Eastern Europe. EyCon’s work in enriching data, and being able to make connections and links between archival collections, is thus the most interesting part of the project from his perspective.  

            Finally, Benjamin Doizelet, head of the iconographic archives division at the SHD, explained that the SHD is an archival service and thus the patrimony code applies to it. This code requires that the archive be open for public access, and they provide this access. At the same time, there is no Internet portal for the photographs held by the SHD. However, there is a communication and follow-up process with everyone who accesses the photographs. Because it is specifically a military archive, it is assumed that all of the files are potentially sensitive, though less because of death and destruction (which is common) than because of potentially sensitive ways in which the army represents itself in the documents. Certain archival documents might offend the dignity of known or recognizable persons and thus must not be made publicly available. Over time, his predecessors had marked certain boxes with a red sticker to signify potentially sensitive content. However, this has always been subject to revision over time and is without any official legal status, indicating rather a subjective and ethical point of view.

            Ultimately, the workshop was a valuable opportunity to exchange with the project’s archival partners. The team showed both the potential and the problems of these tools, which are sure to reshape archival work. At the same time, our archival partners were able to educate us about the current practical, legal, and scientific conditions that currently govern they ways in which they make photographic heritage available to researchers and the public.

Applying Pose Recognition to the World War One Valois Albums: Some AI Avenues for Photography History

Jonathan Dentler, EyCon Postdoctoral Research Fellow

The Early Conflict Photography 1890—1918 and Visual AI (EyCon) project is partnered with several UK and French archives and museums, including La Contemporaine, which notably holds the fonds des albums Valois, which contain the photographs produced by the Section photographique de l’armée (SPA) during World War One. Using both Application Programming Interfaces (APIs) for already-digitized collections, as well as scanning of collections that have not already been digitized, Eycon has assembled a very large corpus of images, and is now advancing to the computing stage. Computation involves training a neural network to recognize various features of the images in the corpus, with the aim of enriching their associated metadata, increasing their navigability, and making them more useful for researchers and the public. This short post will explain the project’s goals, with a focus on the fonds Valois. In particular, it will consider how machine learning holds the potential for new contributions to existing scholarship on photography, personal presentation, and emotion, as well as how the SPA attempted to shape the image of the war for domestic and international audiences.

Created in 1915 in order to counter German photo propaganda, the SPA was a collaboration between the École des Beaux-Arts, the Ministry of War, the Ministry of Public Instruction, and the Ministry of Foreign Affairs, which mobilized professional photographers to document the war for the purposes of propaganda as well as the creation of national patrimony and future public memory of the conflict. While the Minister of War charged the photographers with their general mission and assigned them to sections of the war front and the home front, the ministry of public instruction and the Beaux Arts took charge of logistics and funding, and the Minister of Foreign Affairs diffused the photos to the press in allied and neutral countries. The effort was organized on an industrial scale, and generated a vast number of photographic prints in a wide variety of formats (Guillot, 2010).[1] Digitization of this collection has allowed for a whole new set of uses by researchers and the public (Maurel and Morvan, 2016), and machine learning promises to build on this success in important ways.

Having assembled the EyCon database, the research team is now applying a set of deep learning tools to segment pages into images and associated text, estimate human pose and gesture, and to detect objects in the images. These techniques can be used to enrich searchable metadata for digitized archives, dramatically enhancing its navigability for researchers and widening the parameters by which one can search for a given set of images. While the fonds Valois already has quite detailed associated metadata for the photographs in its albums—including information such as date, caption, and location—the ability to automatically recognize objects and poses in the photo and produce new metadata can supplement this information, substantially enhancing the collection’s usefulness for researchers and the public.

Indeed, the fonds Valois’ relatively replete metadata makes it a vital collection for the EyCon project’s wider goals, because it allows us to assemble a training dataset for the deep learning network. In order to teach the network to recognize human poses such as “standing,” “sitting,” “lying down,” or “walking,” it is necessary to assemble at least ten thousand images tagged with this information as a training set. After computation, the OpenPose algorithm should be able to recognize such poses in the images without the assistance of preexisting metadata (Cao et. al., 2019). The fonds Valois’ existing searchable metadata allows us to find all the images that contain these terms in the captions, and automatically compile them into a training set. Inputting “assis” into Argonaute, for example, might return image VAL 413/091, which was taken on April 21, 1918 and depicts “French and British artillerymen sitting on a 75mm canon.” This image, along with all the others with “assis” in their metadata can be compiled into a training set.

Researchers have used the fonds Valois to pose questions about topics such as the relationship between photography, the war experience, emotion, and masculinity (Pichel 2021). However, scholars are limited by time and labor constraints to working with a fairly limited set of photos, often picking one battle or album (Pichel, 2012). Researchers who have applied quantitative methods to visual sources tend to confine their sample sizes to between four hundred and one thousand images (Männistö et. al., 2022). Information about pose can be extremely useful to the researcher asking questions about how experience of war, as well as the SPA’s official photographic representation of it, changed over the course of the conflict. In general, sitting connotes a more relaxed photographic model, and indeed in this instance the men are smiling and smoking. If a researcher could establish a change in the relative prevalence of photos displaying sitting soldiers over the course of the war, they could make an argument about changing photographic practices, which may have shifted from a stiffer and more official relationship to the model to a more relaxed attitude, emphasizing confidence, ease, and camaraderie among enlisted men (Marly, 2018).

Pose estimation will help to make such arguments by allowing the researcher to work at a quantitative scale that would be otherwise impossible, as well as by alerting the researcher to relevant photos for which existing metadata would not have been sufficient. Take, for example, VAL 320/045 dated January 15, 1917 and taken in the area of Alsace.

The photo shows a group of five tirailleurs sénégalais sitting down to a meal, and one soldier standing. Two of the soldiers are smiling, conveying a relaxed air of camaraderie behind the front lines. Yet the caption—“Billet of the Senegalese. The soup of the infantrymen (tirailleurs)”—contains no information on pose or facial expression, and would thus not come up in a search for “sitting” in Argonaute. EyCon will apply machine learning tools such as pose estimation and expression recognition to enrich such images’ associated metadata, opening up whole new avenues of research into emotion and photographic practice during World War One.

With a much higher degree of precision than is afforded even by the relatively replete metadata of the fonds Valois, researchers could ask questions such as “What is the relative prevalence of sitting vs. standing photos in representations of soldiers over the course of the war? How do such iconographic trends change over time?” These choices by SPA photographers could indicate a shift from early optimism to later disillusion and exhaustion, or a more relaxed stylistic approach. Or, they might point to changing propaganda strategies that shifted from a stricter adherence to an iconography of heroism to a limited admission of the hardship, sacrifice, and day-to-day routines and habitus of soldiers at the front. By calculating the distance of the subjects from the photographer and how it changed over time, researchers could study the relative prevalence of close ups, medium shots that show groups close enough to depict their actions, or more distant landscape shots. Close up shots tend to provoke identification with the subject, and a different emotional response in the viewer (Chumachenko et. al., 2020). Alternatively, using object detection a researcher could find photos that include smoking or certain types of military equipment and uniform, useful information for studies of material culture. The possibilities are nearly limitless.

Machine learning has the potential to reduce the time and labor required for annotating photographs, compiling large numbers of images for quantitative study around a wide variety of variables, and enables study of large image collections on a new scale. Such tools will shift the labor of quantitative visual analysis away from compiling data and toward the higher-level work of interpretation. These techniques are particularly interesting for the photographic archive, and will allow researchers to see this medium and its history anew; while traditional art historical methods were developed in the context of a relatively limited number of artists and works, machine learning allows researchers to take the industrial scale of photographic production into account for the first time.

References: 

Cao, Zhe, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. “OpenPose: Realtime Multiperson 2D Pose Estimation Using Part Affinity Fields.” IEEE Transactions on Pattern Analysis and Machine Intelligence 43.1, pp. 172-186.

Chumachenko, Kateryna, Anssi Männistö, Alexandros Iosifidis, and Jenni Raitoharju. “Machine Learning Based Analysis of Finnish World War II Photographers.” IEEE Access, vol. 8, pp. 144184-144196, 2020.

Guillot, Hélène. “La section photographique de l’armée et la Grande Guerre: De la creation en 1915 à la non-dissolution.” Revue historique des armées, 258, 2010, pp. 110-117.

Männistö, Anssi, Mert Seker, Alexandros Iosifidis, and Jenni Raitoharju.“Automatic Image Content Extraction: Operationalizing Machine Learning in Humanistic Photographic Studies of Large Visual Archives.” arXiv preprint arXiv:2204.02149 (2022).

Marly, Mathieu. “L’armée rend-elle virile ? Réflexions sur le “modèle militaro-virile à la fin du IXe siècle.” Clio. Femmes, Genre, Histoire, 47, 2018, pp. 229-247. 

Maurel, Lionel, Josselin Morvan. “Les circulations sur le web d’un corpus numérisé: l’exemple des albums Valois,” Rapport final du projet : “Le devenir du patrimoine numérisé en ligne: l’exemple de la grande Guerre.” 2016.

Pichel, Beatriz. “Making Sense of Verdun: Photography and Emotions During the First World War in France,” in On Suffering: An Interdisciplinary Dialogue on Narrative and Suffering. London: Brill, 2012.

Pichel, Beatriz. Picturing the Western Front: Photography, Practices, and Experiences in First World War France. Manchester: Manchester University Press, 2021.

EyCon: What we are doing with Machine Learning and Computer Vision.

Blog Post by Soumik Mallick

Objective:

Historical photographs held by archival institutions provide a huge source of information for researchers in several domains of computer science, image processing, and computer vision. Photographs have been analyzed in multiple fields, including archaeology, war history, post-phenomenological geography, propaganda research, post-colonial studies, and many more. However, the work required for methodical analysis of very large quantities of photographs is a time-consuming and painful task, which also necessitates research time. Nowadays state-of-the-art Machine Learning (ML), Deep Learning (DL) and Computer Vision (CV) algorithms have significantly sped up this process and are providing a new viewpoint for researching photo archives.

Within the EyCon project, we are contributing to historical photograph analysis through ML as well as DL. This is allowing us to take one step further towards automatically analyzing historical data and extracting valuable information, which can help historians to find deeper meanings. Currently, we are developing novel methods using ML and DL for historical layout analysis and object detection. Our early-stage experiential prototype shows how several historical photograph analysis research problems can be addressed by utilizing our techniques. Our in-house developed algorithm allows for automatically extracting layout and detection of the objects detailed in historical photographs. Our work is currently in its early stages and needs more refinement, but it is already demonstrating positive results.

Methods: 

Hand-crafted features are affected by the difficulty of capturing explicit knowledge about the attributes to be associated with archival photographs, such as newspapers layouts. DL and CV have opened innovative opportunities for computer science researchers to assist the wider research community with automatic tools, to analyse and further understand document layout and objects. They can recognize meaningful patterns in historical data that are intrinsically related to human perception, and can assist experts in document layout analysis, such as object detection in war photography, which is particularly useful for museum and art gallery websites.

Currently, our model has three separate stages 1) LayoutParser and 2) Object detection and 3) Human pose estimation. We will discuss them one by one.

  1. Layout-parser:

DL methods have played a significant role in the last decade in increasing the performance of CV tasks, including historical document analysis tasks and layout analysis. Yet, the drawback of common DL approaches is their enormous hunger for annotated data. This hunger is particularly problematic in historical document analysis and layout analysis since most tasks, like semantic labelling, require experts to label the document images correctly.

Figure 1: LayoutParser framework

In recent times several DL models and datasets have been developed for layout analysis tasks. Nowadays, we also see that object detection-based methods like Faster R-CNN and Mask R-CNN are also used for identifying document elements, as well as detecting tables. Most recently some research work shows that Graph Neural Networks have also been used in table detection. 

Nevertheless, drawbacks of those models are usually implemented individually and there is no unified framework to load, as well as use, such models. There have been a variety of document data collections to facilitate the development of DL models. Some examples include PRImA (magazine layouts); PubLayNet (academic paper layouts); Table Bank (tables in academic papers); Newspaper Navigator Dataset (newspaper figure layouts); and HJDataset (historical Japanese document layouts). So, the models trained on these datasets are already available in the LayoutParser model zoo to support different scenarios.

A Model in LayoutParser takes a document image as an input and generates a list of rectangular boxes for the target content regions. There are other methods, which depend on deep convolutional neural networks (CNN) rather than manually curated rules to identify content regions. It is formulated as an object detection problem and state-of-the-art models like Faster R-CNN and Mask R-CNN are used. There are three key components in the data structure, namely the Coordinate system, the TextBlock, and the Layout. They provide different levels of abstraction for the layout data, and a set of APIs (Application Programming Interfaces) is supported for transformations or operations on these classes.

In LayoutParser, coordinate supports generally two kinds of variation: TextBlock consists of the coordinate information and extra features like block text, types, and reading orders; a layout object is a list of all possible layout elements, including other Layout objects. They all support the same set of transformation and operation APIs for maximum flexibility. LayoutParser currently supports nine different pre-trained models, trained on five different datasets.

LayoutParser provides a unified interface for existing Optical Character Recognition (OCR) tools, and it supports the Tesseract and Google Cloud Vision OCR engines. In accordance with the authors of the research paper, we have already implemented a LayoutParser using Pytorch, Python >= 3.6, Detectron2 and CUDA.

LayoutParser uses Detectron2-based pre-trained models like Faster R-CNN, RetinaNet, and Mask R-CNN to detect the layout of our input document. This is basically an object consisting of a list of detected layouts.

(a)
(b)
(c)

Figure 2: Sample of output of LayoutParser

In each detected layout, you will get the following important information:

  1. Coordinates of the bounding box (x1, y1, x2, y2) of each detected layout;
  2. The type of detected layout (i.e text, image, table, list, or title);
  3. The ID of the detected layout;
  4. The text inside each detected layout;
  5. The confidence score of each detected layout;

 For EyCon, we fine-tuned our model of LayoutParser according to our dataset and achieved good results. Fig. 2 shows the outcome of our LayoutParser.

Historical Object Detection:

Detected objects are helpful in determining the context of historical photos when used in conjunction with appropriate classes, as well as the focus of each photographer. A photograph of a chair, for example, may have been taken indoors, whereas photos of horses, boats, cars, or trains may have been taken outdoors. Moreover, the presence of objects can help determine the time of the year, and even establish the period for unlabelled photographs. For example, a high number of chairs, ties, uniforms, and people in a photograph are likely to indicate some official event or group of soldiers. At the same time, photos of battle tanks, airplanes, artillery, sandbags and guns, would suggest a battle or near battle areas. Aside from identifying the context of each photo, such analysis can draw out the main focus of a photographer, evaluating which types of objects are most frequently present in their photographs.

Figure 3: The Mask R-CNN framework

The main goal of object detection is to predict a set of bounding boxes and category labels for each object of interest. In recent times detectors address this set prediction task in an indirect way, by defining surrogate regression as well as classification problems on a large set of proposals as anchors, or window centres. Most of their performances are significantly influenced by post-processing steps to collapse near-duplicate predictions, the design of the anchor sets, and by the heuristics that assign target boxes to anchors.

Secondly, we implemented object detection using Mask R-CNN. Mask R-CNN’s benchmark that is used in Detectron2 uses multiple frameworks. It is an overall two-stage procedure that uses parallelism in regard to the class and box. Every candidate object has two outputs: a class label and a bounding box offset. Building from Fast R-CNN, the first stage of a Regional Proposal Network (RPN) is adopted. Moreover, pixel-to-pixel alignment is added and features are extracted using RoIPool from each candidate box. RoIPool and RoIAlign are techniques that perform quantization for the pixels.

Mask R-CNN adopts the same two-stage procedure, with an identical first stage (which is RPN). In the second stage, in parallel to predicting the class and box offset, Mask R-CNN also outputs a binary mask for each RoI. This is in contrast to most recent systems, where classification depends on mask predictions. It follows the spirit of Fast R-CNN that applies bounding-box classification and regression in parallel (which turned out to largely simplify the multi-stage pipeline of the original R-CNN).  Mask R-CNN is also special because it outputs a binary mask for each RoI in contrast to most object segmentation systems where classification depends on mask predictions. For each RoI, a multi-task loss is defined as L = Lc + Lb + Lm. The mask representation is quite valuable to the performance of Detectron2/Mask R-CNN since it encodes an input object’s spatial layout and thus produces faster inference. Mask R-CNN also features a convolutional backbone for feature extraction (creates pixel-to-pixel correspondence provided by convolutions) and a network head for bounding box recognition as part of both classification and regression.

(a)
(b)

Figure 3a and 3b: Sample of output of object detection

Improved objection detection:

This simplifies the detection pipeline by dropping multiple hand-designed components that encode prior knowledge, like spatial anchors or non-maximal suppression. Unlike most existing detection methods, the object Detection transformer doesn’t require any customized layers and thus can be reproduced easily in any framework that contains standard CNN and transformer classes. Below Fig. 4 shows object detection architecture with transformer. 

Figure 4: Object detection architecture with transformer

The transformer encoder-decoder infers a fixed-size set of N predictions, in a single pass through the decoder, where N is set to be significantly larger than the typical number of objects in an image.

Next, an important step is finding a bipartite matching between these two sets, a permutation of N elements σ is searched with the lowest cost.

Where Lmatch (yi, ^y(i)) is a pair-wise matching cost.

The matching cost takes into account both the class prediction and the similarity of predicted is yi= {bi, ci}.

ci is the class probability.

Lmatch (yi, ^y (i)) is:

Compute the Hungarian loss, which is a linear combination of a negative log-likelihood for class prediction and a box loss defined as:

A conventional CNN backbone helps to learn a 2D representation of an input image and the backbone is ImageNet-pretrained ResNet-50 or ResNet-101 with frozen batch norm. After that the model flattens it and supplements it with a positional encoding before passing it into a transformer encoder. A Transformer decoder then takes as input a small, fixed number of learned positional embeddings, called object queries, and additionally attends to the encoder output. Each output embedding of the decoder is passed to a shared feed forward network (FFN) and that predicts either a detection with class and bounding box or a “no object” class.  Fig. 5 shows an output of the object detection we implemented with PyTorch and Detectron2. 

(a)
(b)

Figure 5: Example output of our improved Object detection

Human pose estimation:

A human pose skeleton denotes the orientation of an individual in a particular format. Fundamentally, it is a set of data points that can be connected to describe an individual’s pose. Each data point in the skeleton can also be called a part or coordinate, or point. A relevant connection between two coordinates is known as a limb or pair. However, it is important to note that not all combinations of data points give rise to relevant pairs.

(a)
(b)
(c)

Figure 6: Keypoints detected by OpenPose

The pipeline from OpenPose is actually pretty simple and straightforward. First, an input image is fed as input into a “two-branch multi-stage” CNN. Two branch means that the CNN produces two different outputs. Multi-stage simply means that the network is stacked one on top of the other at every stage.

Figure 7: Architecture of the two-branch multi-stage CNN of OpenPose

Two-branches: the top branch, shown in beige, predicts the confidence maps of different body parts location such as the right eye, left eye, right elbow and others. The bottom branch, shown in blue, predicts the affinity fields, which represents a degree of association between different body parts.

Multi-Stage: at the first stage the network produces an initial set of detection confidence maps S and a set of part affinity fields L. Then, in each subsequent stages’ predictions from both branches in the previous stage, along with the original image features F, are concatenated, represented, and used to produce more refined predictions. Finally, the confidence maps and affinity fields are being processed by greedy inference to output the 2D key points for all people in the image. We explain OpenPose first, which can help you to understand how we can get key points.    

Human pose from single image:

Estimating a 3D human pose from a single image is known to be a severely ill-posed problem because many different body configurations can virtually have the same projection. A typical solution consists in using discriminative strategies to directly learn mappings from image evidence (e.g., HOG, SIFT) to 3D poses. This has been recently extended to end-to-end mappings using CNNs. In order to be effective, though, these approaches require large amounts of training images annotated with the ground truth 3D pose. While obtaining this kind of data is straightforward for 2D poses, even for images ‘in the wild’ (FLIC or LSP datasets), it requires using sophisticated motion capture systems for the 3D case:  

(a)
(b)

Figure 8: 3D human pose estimation on historical images

With 3D body pose parameterization, most approaches use a skeleton with a number N of joints ranging between 14 and 20 and represented by 3N vectors in a Cartesian space. In order to enforce joint dependency during the 2D-to-3D inference and considered latent joint representations, obtained through Kernel Dependency Estimation and autoencoders. To gain depth-scale invariance, we have to first normalize the vertical coordinates of the projected 2D poses xi to be within the range [−1, 1]. 3D joint positions yi are expressed in meters with no further pre-processing. After, we can represent both 2D and 3D poses by means of Euclidean Distance Matrices. For the 3D pose y it is defined edm(y) to be the N × N matrix where its (m, n) entry is computed as:

Similarly, edm(x) is the N × N matrix built from the pairwise distances between normalized 2D joint coordinates. 

Retrieving the 3D joint positions

from a potentially noisy distance matrix edm(y) estimated by the neural network, can be formulated as the following error minimization problem:

As a regression between two Euclidean Distance Matrices encoding pairwise distances of 2D body joints and 3D body joints, a 3D human pose estimation problem can be formulated. Regression is carried out by a Neural Network, and 3D joint estimates are obtained via Multidimensional Scaling from the predicted 3D Euclidean Distance Matrices.  Above, Fig.8 shows the outcome of 3D human pose estimation.

Implementation:

Both our LayoutParser and object detection are based on PyTorch with Facebook Detectron2. For human pose estimation we used PyTorch. Currently, we are using a single NVIDIA RTX 3000 GPU and CUDA version 11.

Result:

Our work is still at an early stage. While we have achieved good results so far, it needs more refinement.

 Figure 9: Output result of LayoutParser

Figure 10: Output result of object detection

Figure 11: Human pose estimation                   

Difficulty and challenges:

We are facing some challenges with LayoutParser. Historical photographs contain lots of complex layouts and LayoutParser has failed to identify those layouts. Historical photographs also have poor image quality with high levels of noise. There can be several kinds of logos, symbols and stamps inside images as well as documents, which make it very difficult to perform LayoutParser on these records.

Conclusion:

In future, we aim to add more features to our platform, initially improving the detection of smaller-sized objects. In our updated version of the model, we will try to add person segmentation, face detection with facial expression analysis, group-level emotion recognition, gender estimation, and weapon detection. We believe that these updated features will help to provide us with a deeper understanding, which will enable further research by historians or within cultural heritage organisations. While our work is currently fragmented into different parts, soon we will be able to provide our data open source on our website through a Python SDK library with an integrated model, so you can also download all our code and database. We wanted to keep you updated on our progress so far, and to let you know that we are continuously working towards improving our model to provide a uniform system for the automated analysis of historical photography.

 References

Cai, Hongping, Qi Wu, Tadeo Corradi, Peter Hall, ‘The Cross-Depiction Problem: Computer Vision Algorithms for Recognising Objects in Artwork and in Photographs’, CVPR (2015).

Carion, Nicolas, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko, ‘End-to-End Object Detection with Transformers’, CVPR (2020).

Castellano, Giovanna, Eufemia Lella, and Gennaro Vessio, ‘Visual Link Retrieval and Knowledge Discovery in Painting Datasets’, Multimedia Tools and Applications, 80 (2021).

Castellano, Giovanna, Giovanni Sansaro, and Gennaro Vessio, ‘Integrating Contextual Knowledge to Visual Features for Fine Art Classification’, arXiv, 2021.

Crowley, Elliot J., and Andrew Zisserman, ‘In Search of Art’, ECCV (2014).

Gonthier, Nicolas, Yann Gousseau, Said Ladjal, Olivier Bonfait, ‘Weakly Supervised Object Detection in Artworks’, ECCV (2018).

He, Kaiming, Georgia Gkioxari, Piotr Dollar, and Ross Girshick ‘Mask R-CNN’, ICCV (2017).

Francesc Moreno-Noguer, 3D Human Pose Estimation from a Single Image via Distance Matrix Regression, CVPR2016

Liu, Ziming, Guangyu Gao, Lin Sun, Li Fang, Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh, ‘Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields’, CVPR (2017).

Liu, Ziming, Guangyu Gao, Lin Sun, Li Fang, ‘IPG-Net: Image Pyramid Guidance Network for Small Object Detection’, CVPR (2020).

Shen, Zejiang, and Ruochen Zhang, Melissa Dell, Benjamin Charles Germain Lee, Jacob Carlson, and Weining Li, ‘LayoutParser: A Unified Toolkit for Deep Learning-Based Document Image Analysis’, arXiv (2021).

Strezoski, Gjorgji Marcel Worring, ‘OmniArt: Multi-task Deep Learning for Artistic Data Analysis’, CVPR (2017).

Review of the First EyCon Workshop ‘Photosensible’, 9 and 10 June 2022 – by Elise Clerteau (ENS de Lyon).

On 9th and 10th June, the EyCon Project (funded by Labex Passés dans le Présent, and the AHRC), the Musée du Quai Branly Jacques Chirac, and the Université de Paris Cité organised a workshop on sensitive photographs and their contemporary recirculations. Held at the Musée du Quai Branly Jacques Chirac (Paris), several specialists, brought together by the question of the ‘photosensitive’, were able to question collectively the specific problems caused by ‘controversial images’ that hurt, show human suffering or materialise domination, and consider the way historians, archive institutions and museums can approach this unstable material that resonates with the violence of the past. The discussions were articulated around several issues, relating to ‘gaze’; the reproduction/recirculation of the images; the legal delimitations; the difficult conciliation between the occultation of the past and the clash of sensitivities; and to the uses–notably digital uses–of these photographs. At the crossroads of ethics and legality, violence and sensitivity, visibility and invisibility, image and text, the ‘photosensitive’ has aroused artistic and historical reflections, but also philosophical, socio-anthropological and psycho-cognitive considerations to approach certain shocking and violent photos in accordance with the objectives of the EyCon project. Focused on the mass of war images and their treatment before the second half of the 20th century, this project aims at finding the tools to structure this material, make it more accessible, and solve the potential issues in its dissemination.

June 9th (morning)

The first hypotheses were put forward on the morning of June 9, as the representatives of the archives and museums gathered for the occasion, reported:

  • Musée du Quai Branly Jacques Chirac (Paris). Among the 700,000 photographs in the collection inherited by the MQB, many are sensitive and problematic both from an iconographic point of view (nudity, mutilation, torture, skeletons, medical photography, animal sacrifices, etc.) and from a contextual one. There is an existing digital library, but with restricted access — for example, some photographs appear without the image, and one has to request to see it.
  • Établissement de communication et de production audiovisuelle de la Défense/ Audiovisual section of the French Armed Forces (Ivry). The ECPAD holds 15 million photographs and approximately 94,000 hours of film. A set of rules has been established to determine a varying level of confidentiality of the photos according to their violence. Shocking terms or iconographic contents are mentioned as such in an informative and/or preventive note. The institution is also particularly attentive to offended relatives’ requests for the removal of images.
  • Division of Iconographic Archives of the Defence Historical Service (Vincennes). Public and private archives are treated in this military service whose priority is to make all (meta)data accessible to the public despite their explicit violence. However, the collections are not yet available online.
  • La Contemporaine (Paris-Nanterre). A library and ‘museum of contemporary worlds’, La Contemporaine documents the consequences of conflicts on civilians. In the management of its collections, there is a particular emphasis on questions of classification and on the psychological effects of war violence on populations. 
  • Archives nationales d’outre-mer (Aix-en-Provence). 37 kilometres of archives from colonising and colonised sources (1844-1970) are kept at the ANOM. Most of the photographs are accessible online, despite their sensitive nature.
  • Imperial War Museums (UK). Opened in 1917, the three IWMs now detain 33 million items documenting the events of both World Wars. The issues of sensitivity and digital dissemination trigger specific reflections aiming at bridging the gap between academic research and the public’s understanding of conflicts.
  • Search at Wellcome Collection (London). The Wellcome Collection museum’s search engine underwent a major development in 2015, optimising responses to the online audience’s expectations. The access to more than a million images is guaranteed despite gradual restrictions (open / open with advisory /restricted / by appointment / donor permission / temporary unavailable unavailable) depending on the sensitivity of the photographs.
  • Royal Museum for Central Africa (Belgium). Founded in 1960, this museum is an institute for research, valorisation and reinforcement of national capacities in Africa. A development of photographic metadata through digitalization is underway, but still partial.
  • Museum of Photography of Madagascar (Antananarivo, Madagascar). A museum of heritage conservation and education, this institution has received many financial and documentary donations since its creation in 2013. Digital technology has been put at the service of Madagascar’s heritage to give its photographic memory of the colonial period its visibility back. Many official photographs being paradoxically held by families, those shared by locals are often returned to them after having been digitised.

The presentations were structured around information about the different institutions; institutional (re)definitions of what ‘sensitive’ photographs are; policies for their management; the challenges of digital dissemination; online accessibility and its potential restrictions; additional metadata; and the establishment of prior warnings. Despite disparities in terms of the size of photographic collections and the progress of their digitalization, the origin of their sources, and the policies for exhibiting images considered ‘sensitive’, several common difficulties have emerged:

  • Definitory. The iconographic content is not systematically a criterion to define what a ‘sensitive photograph’ is. If nudity, ritual mutilation, corporal punishment, public executions, human remains, diseases and animal sacrifices can be identified as ‘sensitive categories’, it is no less true that the violence of certain photographs is due to their textual content (captions, notices, titles) and/or to their symbolically traumatic resonance in certain communities or cultures. Where an internal committee has been designated at the Musée du Quai Branly Jacques Chirac to discuss the problematic nature of certain images, a specific identification procedure was put in place at the IWM, from the acquisition of the photograph to its ‘encounter’ with the public.
  • Artistic and historical. What to show, what to correct? Should the captions appear as they were written? should they be corrected? should they be accompanied by an informative-preventive contextual caption? The choice could potentially obscure part of history. On the ECPAD website, in addition to the ‘sensitive image’ warning displayed on the screen, the captions judged to be shocking are explicitly labelled with their contextual origin, to clarify the distance between what was done at the time and the way it is presented today. Moreover, the cases of requests for the removal of images from relatives resolve an ethical issue, but also raise a historical problem concerning the erasure of evidence of past violence.
  • Legal and juridical. The uneven progress of the digitalization of the collections mentioned is certainly due to budgetary ties (the scale of the Wellcome Collection is largely due to the substantial investments that allowed the processing of 10,000 images per day in 2015 when the project was born), but also to legal constraints. Copyrights, classification issues, the duty to return sources, are factors that can complicate or even restrict the online or onsite exhibition of these images. In the iconographic archives of the Historical Service of Defence, where ‘violence, death and destruction cannot be hidden’ (Benjamin Doizelet), the greatest difficulty encountered by archivists is related to the classification of public archives.
  • Ethics and practices. These photographs raise ethical questions — both because of the sensitive nature of what they show and because of their potential effects on viewers’ sensibilities. This leads to theoretical and practical reflections on the criteria to be determined for their contemporary uses and (re)circulations. Dieter Van Hassel, head of the Archives-Information Department at the Royal Museum for Central Africa, spoke about the challenges of expanding online access to ‘sensitive photographs’: it is not only a question of legal constraints but also of fear of the public’s reactions. At the Musée du Quai Branly Jacques Chirac, some photographs are published online but without the image: only the informative note appears, thus testifying to the existence of the image while inviting one to contact the museum to access the image. However, we can assume that this policy of not showing also poses a problem: accessibility becomes a privilege when it implies a specific consultation process that is not within the reach of everyone. Circulation is essential in Madagascar, where the Museum of Photography is conceived as a depository-mediator of images and photographic archives. The digitalization of photographic documents and the creation of a database are similar to a ‘photographic heritage in the making’ (Helihantha Rajaonarison), which could remedy the occultation of the visual memory of colonisation in Madagascar where the main national museums are still closed to the public.

The link between photographic dissemination and ethical-legal problems superimposed on individual and/or collective traumas––of which the photographs are material and memorial traces––has thus been woven since the beginning of their exchanges. The ‘trauma-informed approach’, suggested by Helen Marvin, seems conducive to renewing the way we look at these photographs, to questioning and even redefining the uses and norms established in the past. As the eye of the specialist is not the only one to rest on these visual memorial objects, the role of the spectator is not negligible in the appreciation and the determination of the ‘photosensitivity’: one can go as far as supposing, like Heliantha Rajaonarison, that among sensitive photographs it is possible to distinguish photographs with positive and negative sensitivity, according to their reception. Indeed, the remediation of certain images can engender conversations on the collective recognition of forgotten and invisible violence. Generally speaking, it is the balance (still far from being reached) between a duty to make these collections public (both physically and digitally), freedom of expression, and privacy that is on the horizon.

June 9th (afternoon)

The afternoon of the first day was an opportunity for a collaborative and playful workshop of photographic confrontation(s), in every sense of the word. Photographs previously selected by each specialist according to their explicit or uncertain sensitivity were divided into two batches, just as the specialists were divided into two groups to encourage exchanges. Several images were thus discussed in light of their ‘sensitivity’, to see why and how they could be judged as sensitive, and to what extent it was possible to arrive at recommendations concerning their contemporary (re)circulations. The photographs were quite diverse but provoked comparable inquisitive reactions, starting from the context of production moving towards issues of reception and exhibition: 

  1. The context of the shooting of the photograph and past uses in question. What information do we have on the context of the shooting, on the identity of the photographer and on their intentions? What uses has this photograph been put to – propaganda, documentary, anthropology, medicine…?
  2. The iconographic content and composition in question. Does the photograph directly represent violence? To what extent does the photographic technique (framing, staging) influence the sensitivity of the image? Is the language of the caption, echoing the colonial rhetoric, possibly more violent than the image itself? 
  3. Reception and exhibition in question. Who to show these photos to and where? For whom are they sensitive? What can be done so that they do not offend sensibilities? Can they be exhibited (in a museum and/or online) and if so, how?

These interventions revealed the complementary plurality of approaches as well as an undeniable divergence between the sensitivity of the archivist or historian and that of the community––as one of the participants said, ‘we cannot put an archivist behind each reader’. Indeed, the status of photographs is also largely defined by the reactions of viewers, whose primary reaction is not necessarily as informed and objective as that of a specialist. From this observation, the caution for a respectful use of the photographs towards communities was agreed upon and accompanied by a recommendation to diffuse certain images online that undermine the dignity of certain people who were photographed in a condescending way, especially when their face is visible. Insofar as photographs are multicultural objects from their creation in a shared cultural-historical context, it seems difficult to decide unilaterally on their use(s). Adopting a ‘poly-cultural’ approach, according to Julien Schuh’s proposal, could help to avoid the risk of being out of step with those who have also participated in creating these objects, sometimes at their expense. More widely, the importance of visual cultures nowadays would advise us to pay particular attention to the education of the public. Providing adequate knowledge to understand what photography is and what it implies in the past, present and future, seems indispensable.

June 10th

Before welcoming the public to attend the conferences organised in the movie theatre of the Musée du Quai Branly Jacques Chirac during the afternoon of June 10, the specialists met once again behind closed doors. The historical management of sensitive visual documents in the era of mass digitization and the concomitant risks of misinformation; the difference between these documents and other types of sources; the methodologies to be adopted; and the apparent contradiction between emotion and the historical approach, were among the topics discussed. Three interventions brought together cognitive sciences and the consequences of the recirculation of sensitive images with the question of the public ‘in context’.

Grégoire Borst (professor of developmental psychology and educational cognitive neuroscience at the University of Paris) addressed the effects and consequences of ‘exposure to sensitive images on the brain and memorization’. The approach of the ‘photosensitivity’ by the cognitive processes showed that violent photographs tend to be memorisedmore quickly and durably, but especially that the sight of an aversive photograph is felt as a threat, involving a psychological process of defence. This is one of the reasons why photographs differ from videos, as it is easier to regain control of one’s reactivity at their sight. The colorization of photos could also be explained by neuro-psychological effects of identification, exacerbation of shock, distanciation, etc. As for the regular sight of violent or shocking images (as in video games), it is unlikely to lead to an excess of violence, but rather to a desensitisation, because it is less and less felt as threatening or shocking. Grégoire Borst also insisted on the fact that even though the spatio-temporal distance separating the photograph from its spectator can influence sensitivity, just as the reactions to violence are likely to vary according to the geographical areas and the socio-cultural groups, these reactions are cognitively identical. Photography appears as a medium provoking a specific reaction leading to an almost immediately constructed meaning. Certain collectively sensitive and contested images can, as the historian Susie Protschky reminded us, reflect an intergenerational trauma that gives a political coloration to the community’s emotional response.

This spectatorial dimension was then addressed ‘in context’ by Helihantha Rajaonarison (Musée de la photographie de Madagascar) in relation to sensitive photos from the colonial period in Madagascar. The initial idea––that the reader’s reactions make an image violent or not ––leads to an injunction to continually question the impact that photographs can have on those who view them (families, descendants but also specialists and archivists), and the extent to which they can remedy a violence that was previously hidden. In the case of Madagascar, colonial violence (including photographic violence) is to be understood through a cultural prism: certain symbolic, psychological and moral violence strategically perpetuated by Gallieni in the nineteenth century during the period of colonial pacification is represented in photographs — such as the submission of the chief Menalamba Rabozaka in front of the queen’s palace, later desecrated with its transformation into a training school for colonial administrators. Thus, the 2,500 photographs held by the Institution of Malagasy National Cartography, the personal collection of Grandidier Père & Fils, as well as private albums left by individuals preserved in the national archives, constitute traces of this violence to which the Malagasy people must have access, if only to decide for themselves whether they are positively or negatively sensitive.

Pierre Schill (Centre de Recherches Interdisciplinaires en Sciences Humaines et Sociales de Montpellier) finally evoked the potential for controversy and the unintended consequences in relation to digital recirculation and the reappropriation of sensitive images. Interested in the work of Gaston Chérau, a photographer-reporter during the Italo-Turkish War (1911), the historian analysed the recirculation of his extremely violent photographs (emaciated bodies, hanged, mutilated, etc.) in two paper works available online, and in the Getty images bank. While the contemporary politicization of photographs can be observed online in the comments made by users under the photographs published on social networks, the question of exhibiting certain photographs of the tortured bodies of foreign soldiers has arisen––not only in relation to their violence but also to their potentially dominant appropriation. Thus, the choice to display the photographs on a table, rather than on the wall, seemed a good solution to avoid erecting them as works of art while preserving their archival documentary dimension. The printed publication of Schill’s book, Réveiller l’archive d’une guerre coloniale (Revealing the Archive of a Colonial War), was likewise intended to protect these sensitive images from ‘documentary digital misuse’.

The public lectures that followed broadened several perspectives both on the sensitive photographic-historical material and on its exhibition and reception.

  • Nancy Rushohora (Senior Lecturer in the Department of Archaeology and Heritage Studies at the University of Dar es Salaam) looked at the demanding images and the memories of colonial violence in Tanzania with the example of the single portrait of sub-chief Nduna Songea Mbano. The photograph, subtitled ‘A Real Mgoni’ was taken from an anthropological perspective, but a false one, since this chief was not from this community. However, this image has been the object of countless circulations anduses in this region of Tanzania, which later became ‘a land of trauma’ because the body of the leader is buried there without its skull which has potentially been transported to Germany shortly after the killings. Representative of the evolution of forms and implications in the display and production of visual material from the colonial period to contemporary practices around this representation, this image testifies to the way in which a colonial photograph can be overturned in and through its sensitive effects.
  • Helen Mavin (Head of Photographs at IWM London) then presented the Provisional Semanticsdigitization project, which challenges the imperial gaze in IWM’s Second World War photographs. Underpinned by an ethical and collaborative methodology, both institutional and interpersonal, the project is dedicated to ensuring a transparent and critical reading of approximately 5,000 photographs. Starting from the premise that these documents are traces of Britain’s dominant presence in India, the titles and captions are scrutinised in order to discuss the socio-political context of creation and to suggest a contemporary recontextualization. The main issues at stake in this project are the continuous questioning of readings and attitudes towards the collections, as well as transparency towards the public. This is why biographies are available online alongside the photographs for researchers.
  • Susie Protschky (Associate Professor of History at Deakin University, Melbourne) communicated on ‘Late starters: The post-colonial afterlives of war photographs from Indonesia and neglected histories of colonial violence’. In support of several photographs, the historian speculates that they are often reproduced as historical evidence of the difficulty of Dutch history that few have been willing to take on. If the recent colorization of some of the images raises objections about the potential misuse of historical material, it explains to a large extent that the official apology of the responsibility of the Dutch state in these abuses occurred shortly after their publications on social networks. The current parallels made by viewers who are already aware of the extent of the colonial propaganda are distortions of the meaning that the photographs had at the time. However, they do testify to the difficulty of positioning between the object and the current national context. Thus, Susie Protschky concludes that photographs of colonial violence are ‘image-events’ circulating in a polyphonic public sphere where divergent discourses on these images develop. A constant readjustment of the interpretative framework of these photographs thus appears more than necessary.
  • The approach of Carine Peltier-Caroff (in charge of the photo library of the MQB) revolved around Helène Meissalloux’s 19th-century album of postcards. By questioning ‘the politics of naming and the ethics of showing’, Carinne Peltier-Caroff presented the process of digitalization carried out in the photographic section of the Museum and evoked the evolution of the standards of cataloguing allowing to fix codes. Questions must be asked from the context of production of the photograph to the documentary processing, including the uses and dissemination as well as the circulation, the development of links between data, the enhancement of the identities of photographed people, the revisitation of the colonial context of the collections, and the historicization of documentary practices. To resolve the ethical and legal issues encountered in relation to the management of the collections, an internal committee was set up to discuss the photographs’ iconographic content, the old titles and historical captions, as well as the conditions for putting them online.
  • ‘The most illuminating thing I have ever seen’, were Kim A. Wagner (professor at Queen Mary University of London)’s introductory words to the analysis of the photographic evidence of the 1906 Bud Dajo massacre. Starting with the observation that this massacre, among the least known in American history, is nevertheless one of its best documented, the professor wonders what a particular photograph (the image should be put online) has to say about a specific historical context. Revealing in several ways, notably because of its scenography, the image superimposes sensitive and historical layers whose significance can be discussed. At the time, it became a ‘visual war trophy’ circulating among soldiers on whose side this violence was ‘positively sensitive’ because it was victorious. In the newspapers, this photograph was used alongside other representations of wounded or dead American soldiers, in order to enhance a narrative in which the illustration of the massacre became an illustration of the white man’s burden. The image was thus implicated and (re)contextualised in various—and sometimes antagonistic—ways, as evidenced by the re-appropriation of the image by the anti-imperialist league for the purpose of colonial denunciations.

All in all, this thought-provoking workshop was an opportunity to further reflect on the ‘photosensitive’ engendered by the EyCon Project, and to lay down foundations for future collaborations between participating institutions. The questions which emerged (still far from being solved) testify to the stimulating richness of a multidisciplinary, polyphonic and polycultural topic, inviting us to renew our approaches and point of view.

AI & Cultural Heritage Eycon Team recording

GOOD NEWS!

The recording of the Eycon team’s presentation ‘AI and Contested Pasts’ at the ‘AI and Cultural Heritage’ event at the Loughborough University’s Institue of Advanced Studies on March 28th, 2022 is now available online!

Pour ceux qui n’ont pas pu assister à la présentation du projet Eycon ‘AI and contested pasts” lors de l’évènement ‘AI and Cultural Heritage’ le 28 mars 2022 à l’Université de Loughborough, l’enregistrement est disponible via ce lien :

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search