Tens of thousands of maps were produced by British forces during the first world war, from large-scale maps for senior commanders needing an overview of an entire front, to much more detailed topographical maps for front-line troops, which allowed them to accurately pinpoint enemy positions and establish the nature of the ground they held.
Many of these maps are preserved in the unit war diaries, with Citizen Historians uncovering more each day. One of the great benefits of tagging the war diaries is that it allows us to continue the mapping work carried out during the war itself, using modern techniques to establish a visual representation of the events of the time and the experiences of the units to whom the diaries belong
Image © IWM (Q 2306)
One way in which we can do this is to use place names, particularly those we can locate on the mapping tool contained within the Place tag, to site events geographically, which is a great way to quickly get an overview of a particular type of activity or event. The link below, for example, will take you to a map produced by Jim O’Donnell of the Zooniverse showing amalgamated unit casualty figures (tags showing 0 casualties have been discarded for this analysis), linked to geographic location and covering the years 1914-1918.
Bear in mind that this data set is derived from the first tranche of completed diaries – a little over 200 units in all. Imagine what the map will look like once we have completed all of the thousands still to be tagged!
Georectification is another exciting way in which mapping tools can add to our understanding of the content of the war diaries. This technique allows us to align first world war maps discovered in the diaries with modern satellite images of the same area. We’ll be doing another blog post soon to cover this and you’ll get to see some excellent examples of just what we can do – give yourself a head start by downloading Google Earth beforehand!
With all our Citizen Historians gallantly tagging away, we thought it was high time we explained how all that hard work is being used to produce the data sets for the project.
While we really appreciate all the effort each and every individual is putting in on the diaries we know that errors can arise for one reason or another. For that reason, we generate what’s known as consensus data. We have an algorithm that allows us to do this.
To begin with, each of the diary pages is tagged by at least five Citizen Historians. Five different people who might each look at that page in a slightly different way. Once that tagging is complete, the diary page is closed and put in the queue for processing.
The system starts this by identifying tags of the same type relating to the same entity (a place, a person, an action etc.). It has to take a best guess at this, clustering tags together based on a percentage of the image size for each scanned diary page. Trial and error has shown that this percentage is best set at 3% vertical and 10% horizontal. There must be a minimum of two tags for a particular entity if it is to make it into the final consensus data set. So, if two of the five Citizen Historians who have tagged a diary page have both identified a place in the same position on that page, that place makes it in.
Image © IWM (Q 5700)
The consensus tag generated from this tag cluster is then placed at the average location in which all of its constituent tags were generated
Next the system has to determine exactly what information should be attached to each tag. This is relatively straightforward when the original tags came from a fixed list (e.g. Activities tags, which can be of only a certain number of types). Where tags contain free text (e.g. person or place), fuzzy text matching is used to determine their attached information (e.g. Slater-Booth, Sclater-Booth and similar variants would be grouped together). Where a majority of these free text tags have the same value, that value becomes the consensus value. However, if there is no clear majority value, then the consensus tag will be formed of the leading variants.
The algorithm is also designed to create serialised data. In essence, this means that each consensus tag is associated with a date, which allows the data generated to then be ordered by date. When Citizen Historians tag dates on a diary page, they essentially segment that page, and it’s that segmentation which allows the system to determine which consensus tags should lie inside which date area.
Once these operations have been carried out for one page of a diary, the next page will be processed and so on until the diary is complete.
Don’t worry about us losing all the tags you’ve generated, though – our databases hold everything that every single one of our Citizen Historians has added to Operation War Diary, be it individual tags, hashtags or text comments. We know just how valuable a resource that’s going to be for anybody wanting to investigate the diaries beyond the standard, structured tags we’ve defined.
Why not check out our first batch of consensus data here: http://wd3.herokuapp.com/public
As part of an ongoing blog feature, researchers involved with Operation War Diary will be introducing themselves and discussing what the project means to them and their work. This week Chris Kempshall, Project Officer for East Sussex County Council’s WW1 Commemorations preparations and Visiting Researcher and Associate Tutor at the University of Sussex, talks about his research interests and how Operation War Diary will help him develop them further.
Over to you, Chris!
I have had a circuitous approach to the field of First World War studies with my initial university BA degree being in Media Practice and Theory, followed by a Masters in Contemporary War and Peace Studies. It was when I encountered theories of the ‘myth’ of the First World War during history related courses in my undergraduate that the war first took hold as an interest of mine and Dan Todman’s work on the subject cemented my interest in the field. At the time, rather naively, I found the idea that history could somehow lie or lack truth to be extremely unfair.
From this starting point I then, over the course of a few years, ended up becoming more intrigued with the nature and construction of the First World War. The original notion behind my PhD thesis was to examine whether the British and French had competing or complementary ‘myths’ of the First World War but it was during that early research that I realized that the relationships between British and French soldiers had been largely overlooked and the more I examined it the more interested I became. The article Elizabeth Greenhalgh wrote on ‘Parade Ground Soldiers’ proved to my mind how important an area this could be.
As a result my research now focuses predominantly on the interactions and relations between allied soldiers of different nationalities. My PhD thesis moved on to focus on the relations between British and French soldiers on the Western Front. Whilst previous studies have touched on the relations between common soldiers, this has often been within specific case studies. I drew particularly on the contemporary diaries, letters and written records of British soldiers within the Imperial War Museum and also the postal censorship records of the French army at the Archives de l’armee de terre in order to trace the nature and evolution of these relations across the war.
One of the main areas of interest for me with Operation War Diary has been examining the records which report on interactions between different allies. Having just recently been appointed to the Academic Advisory Board for the Imperial War Museum I am extremely excited at seeing the results of all this public research.
Alongside my research I also teach and lecture on a First World War course at the University of Sussex and, for the next eight months, am the Project Officer for East Sussex County Council’s ongoing project of World War One Commemorations. I am also preparing my PhD thesis for possible future publication.
You can make a real difference to Chris’ work by using the hashtag #alliedrelations if you come across anything you think might be relevant while tagging the diaries. Relationships don’t need to be limited to those between the British and French forces!
As well as the tagged data generated by Operation War Diary’s Citizen Historians, the opportunity exists to classify a wide range of contextual information by recording hashtags on each diary’s talk pages. There have already been lively discussions amongst Citizen Historians over how best to hashtag #horses and #aeroplanes along with a concerted effort by many to do so. Just go to the Operation War Diary Talk forums over at http://talk.operationwardiary.org/ and search for either term to see the results.
In this blog post, Professor Edgar Jones discusses common medical terms which you might encounter, and which can be noted by using the #shellshock hashtag. Edgar is Professor of the History of Medicine and Psychiatry at King’s College London and leads the MSc in War and Psychiatry. He sits on Operation War Diary’s academic advisory group.
War Syndromes or post-combat disorders
The most common psychological diagnosis of the First World War was shell shock. The term was used by servicemen themselves in the winter of 1914-15 but given formal medical authority by Captain C.S. Myers in a Lancet paper published in February 1915. Henceforth, it was used in medical notes. It was a catch-all term without a precise definition and included a wide range of psycho-somatic presentations (a mixture of functional physical and psychological symptoms). It is not the same as PTSD though has some symptoms in common. However, the military medical authorities were concerned about its use and from 1917 onwards tried to restrict the use of the term. Alternative diagnoses included: neurasthenia and war neurosis.
Please note: contemporaries wrote shell shock as one or two words. When tagging the term it is important to record it as a single word, #shellshock.
Other labels used to describe psycho-somatic disorders in the British Army included:
Disordered action of the heart (DAH): This was a functional disorder (ie without organic basis) characterised by palpitations, chest pain, shortness of breath, fatigue and difficulty completing tasks. It was prevalent in the British Army throughout the latter part of the nineteenth century and was also known as irritable heart, soldier’s heart, cardiac neurosis and Da Costa’s syndrome.
Effort syndrome: Effort syndrome was the term proposed in 1917 by Thomas Lewis to categorise servicemen suffering from functional cardiac disorders hitherto called disordered action of the heart, soldier’s heart or irritable heart.
Neurocirculatory asthenia: The term introduced by US military physicians serving in the UK and France in 1918 to avoid use of the terms disordered action of the heart or effort syndrome.
Gas hysteria was a term applied to servicemen who believed that they had been exposed to gas but for which there was no organic evidence and also to those who had a mild exposure, had recovered physically but continued to report medically-unexplained symptoms.
If you come across any of these terms while tagging the diaries, we would be very grateful if you could use the hashtag #shellshock to note them in the comments section for the relevant diary page. You could be helping to advance our understanding of the psychological effects of the war on the soldiers who fought in it!
In the first eight weeks since the launch of Operation War Diary, over 10,000 citizen historians worldwide have tagged names, places and other details in over 200 unit war diaries.
Initial reports reveal some amazing statistics:
- Over 260,000 tags relating to named individuals
- Over 332,000 tags relating to places
- Almost 300,000 tags relating to activities
- The amount of volunteer effort put in so far is equivalent to one person working 40 hours a week for four years.
With your help we’re going one step further than traditional transcription by using the data to digitally map and analyse patterns and trends in the unit war diaries, offering new perspectives on the First World War. Our developers and academic advisory group are hard at work crunching the numbers from the first two months of the project – we’ll blog about their plans soon.
In addition, much of the data – particularly names and places – will be integrated into The National Archives’ catalogue (Discovery), allowing researchers and family historians to search the diaries for named individuals mentioned in the diaries. Making the data freely available to researchers in this way is hugely important to all of the project partners, and we want everyone to be able to benefit from the amazing efforts put in by citizen historians. The data will be published in this way under the Open Government Licence.
The data will also be available to users of Lives of the First World War, which we’ll tell you more about soon.
Find out more about the unit war diaries
Today The National Archives has published another 3,987 digitised First World War unit war diaries from France and Flanders online, which means that around 6,000 diaries are now available on The National Archives’ website to search free of charge (and download for a small fee). These will in time be added to Operation War Diary for tagging.
If you’d never encountered a unit war diary before tagging them for Operation War Diary, you may have been wondering what all the fuss is about and why they’re so important to researchers and historians. They contain a wealth of information of far greater interest than the army could ever have predicted, providing insight into daily events on the front line, and are full of fascinating detail about the decisions that were made and the activities that resulted from them.
If tagging the unit war diaries has inspired you to find out more, whether about a particular unit or an individual, The National Archives has many useful online resources that should help you. Our First World War 100 portal gives an overview of the millions of records that we hold relating to the war, from war diaries to conscription appeals via service records and Cabinet papers. It’s an essential starting point for anyone researching a First World War ancestor, with step-by-step guidance to help you on your way. We also have a wealth of multimedia resources, including podcasts and the popular My Tommy’s War blog series. We send out a free monthly enewsletter with news and updates relating to our work and collection – sign up today to receive your own copy!
The National Archives