The whole is other than the sum of its parts: the untapped potential of spatial data from archaeological fieldwork

Introduction: reasons for and benefits of harmonising data at scale

The benefits of collating and sharing spatial data systematically from multiple providers, to address a range of environmental issues or to underpin research, have long been recognised. Developed in response to ever increasing environmental challenges such as flooding and pollution on a trans-national scale, the European Union Infrastructure for Spatial Information in Europe (INSPIRE) Directive (2007) provides the mandate and framework for public organisations to share environmentally related datasets within their own and neighbouring countries to improve public access, inform decision making and improve the stewardship of the environment. Initiatives like One Geology, a partnership of 190 organisations from 121 countries, have developed their own network developing the framework, infrastructure and standards for their discipline. Collating data from multiple sources and many countries is also at the heart of the multi-disciplinary The Role of Culture in Early Expansion of Humans (ROCEEH) project.¹

For marine data, The Marine Environment Data Information Network (MEDIN) provides the coordination and acts as a data hub for a broad range of offshore datasets managed by accredited data archive centres (DACs). The seven DACs cover Bathymetry, Water Column Oceanography, Geology and Geophysics Flora, Fauna and Habitat, Meteorology, Fisheries and the Historic Environment. Each DAC contributes key metadata about project datasets to the MEDIN portal. The portal enables users to search across over 17,225 marine datasets from over 600 UK organisations.

Benefits of harmonising and sharing spatial data from multiple providers

A number of studies demonstrate the value of harmonising data across multiple providers. A cost benefit analysis² undertaken by environmental economists and marine consultants on behalf of MEDIN demonstrated a benefit to cost ratio (BCR) of 8.2. A review of a range of marine spatial data infrastructures (SDIs) in New Zealand found the BCR to range from 2 to 18.2.³ Benefits include efficiency of data collection and reuse, supporting marine science and more effective marine spatial planning.

There is already a lot of good practice in the stewardship of historic environment data. National Heritage Agencies and Local Authority Historic Environment Records manage inventories for their respective jurisdictions. A recent study⁴ of the impact of local authority archaeologists in Britain, informed by information in Historic Environment Records (HERs), estimates that archaeology in development management had a BCR of 15, contributing £218m to local economies from commercial archaeology and saving up to £1.3 billion in delay and emergency excavation costs. Project archives resulting from fieldwork and research may be deposited in a relevant archive, including Core Trust Seal accredited archives at Historic Environment Scotland (HES) and the Archaeology Data Service. An analysis⁵ of the value and impact of the Archaeology Data Service in 2013 estimated a two- to eight-fold return through additional reuse of data deposited in their digital archive, though not everything reaches an archive, or is deposited in a reusable format.

Despite the seemingly impressive benefits for the stewardship of data, are we really realising the value and potential of historic environment data, particularly that captured from primary archaeological fieldwork and research? The following sections explore the different approaches to publishing spatial data from organisations responsible for creating and maintaining spatial data - curated data – and the range of data created through primary field work by many organisations for a broad range of purposes, data that needs to be collated to realise real benefit.

Spatial Data and the historic environment - curated data

Good data is at the heart of decision making and stewardship of the historic environment. It should also be an important contributor to wider societal challenges including climate change, automation and complex modelling of our environment through eco-system services and digital twins. Good data should deliver efficiencies to a broad range of users outside of, but interacting with, heritage.

Through The INSPIRE (Scotland) Regulations 2009 key protected sites datasets, including scheduled monuments and listed buildings, are readily accessible through national and organisational metadata portals. These are curated datasets where organisations are responsible for the creation and maintenance of content to deliver their business objectives. Publication of these datasets includes Discovery Metadata, View and Download services.

The Historic Environment Scotland portal provides access to the statutory data HES is responsible for, as well as access to point data from the National Record of the Historic Environment, published through Canmore. For most datasets users may download data as zipped shape files or access Web Map and Web Features services or an Atom feed. Historic land use data may also be accessed through a more restrictive licence. Two web map browsers, The Designations Map Search and PastMap provide location-based services. PastMap also allows users to view and search Canmore and most HER data providers, to query, select and download point data from across searchable datasets in csv or kml format, and to view content on linked web pages hosted by the data contributor. Canmore offers both text and map based searches.

HES key datasets are also added to the SpatialData.gov.scot metadata portal, which enables users to find, share and reuse spatial data provided by Scottish public sector organisations. These records are harvested by and added to the Data.gov.uk open data portal.

Most HERs maintain or have access to databases recording information about monuments and events (specific activities that lead to the recognition of or add to the understanding of a monument). As well as contributing to PastMap, monument and event data is provided to The Spatial Hub. Established by The Improvement Service (the national improvement organisation for Local Government in Scotland), The Spatial Hub provides the coordination and infrastructure for harmonising and transforming 138 datasets across Scotland’s 32 local authorities and other partner organisations, for use by the Scottish public sector bodies and their contractors under the Public Sector Mapping Agreement. Academic researchers may access the data through EDINA.

Spatial data and the historic environment - collated data

Primary archaeological fieldwork, research and analysis, often referred to as ‘events’, are unique activities that happen once over a discrete period of time – often through research, stewardship of the historic environment or in discharge of a condition required under planning consent. Fieldwork routinely creates a wealth of spatial data whose reuse potential is largely unrealised beyond short-term project delivery. Results are typically presented in a pdf report or more traditional publication such as the Royal Commission on the Ancient and Historical Monuments of Scotland (RCAHMS) survey of the Mar Lodge Estate, Aberdeenshire (Figure 1),⁶ where the spatial data is fossilised to the page orphaned from its broader landscape context.

Figure 1. Illustration prepared from archaeological field survey data at Mar Lodge Estate, Aberdeenshire Background mapping © Crown Copyright HES [2023].

Ordnance Survey licence number 100057073.

The underlying data might be deposited with a formal archive such as the Historic Environment Scotland Archives or The Archaeology Data Service (ADS): both Core Trust Seal certified repositories. A study by the Association of Local Government Archaeology Officers (ALGAO):England, working with the ADS, estimated that at best, 2-3% of commercial projects had been digitally archived with the ADS.⁷ The proportion of archives from fieldwork in Scotland deposited with HES remains unquantified.

Even within the archive, the data may not be easily found and retrieved. Archival discipline necessarily compartmentalises data by collection and project (Figure 2) greatly understating the value of the spatial component.

Figure 2. Collection hierarchy for the archive record in Figure 1.

Notable exceptions include the HES National Collection of Aerial Photography and the National Library of Scotland Map Library who both publish online map bowsers as finding aids to their respective collections. All too often it falls to individual initiatives or individual enthusiasm and project-based solutions to realise the potential of spatial data preserved in archives. Creating these collated datasets requires considerably more investment to collect, preserve, harmonise and publish than curated data. Critically, the mandate provided for curated data to provide that coordination is lacking.

Record enhancement is a key function of both the HERs and HES. Existing practice is for the curator to enhance the record through extracting key data from project reports produced as part of the discharge of a planning condition, or from summary (often interim) reports published in Discovery and Excavation in Scotland, an annual digest of fieldwork. Since 2007, fieldwork can be reported through an online application: OASIS (Online AccesS to the Index of archaeological investigationS).^8,9 The OASIS form ensures that fieldwork reporting is standardised, meeting the MIDAS Heritage Standard by drawing on controlled vocabularies published on Heritagedata.org to ensure a degree of consistency at project level. Additionally, the project reports should be uploaded and shared with the relevant HER and HES before release to the Archaeology Data Service. The latest iteration, OASIS V, encourages users to upload project extents, albeit with restricted geometries, to accurately locate projects and share with curators under an open CC-BY licence. There is also an option to upload more detailed geometries for complex project extents, or for details of trench extents. Nevertheless, project extents are still often digitised from raster images designed for inclusion in a data structure report or publication – a process ingrained with inefficiency and inaccuracy. OASIS addresses standardisation and harmonisation at project level but major barriers remain in routinely accessing and sharing more detailed spatial data including interoperability across datasets from multiple providers.

The Scottish HERs published specifications for documenting the basic ‘event’. These specifications provide a baseline, or vanilla, record for an event (the who, what, why, and when) enough to meet HER requirements in documenting where activities have taken place. The specifications do not and are not intended to address the increasing sophistication of primary datasets from a range of non-invasive and invasive techniques to systematically document content within data types or act as finding aids or spatial indexes to associated archives. For instance, as well as seeing the extent of a geophysical survey, a user should be able to assess the usefulness of the dataset through viewing the technical metadata (instrumentation, resolution of survey etc) – all captured through the geophysical module in OASIS¹⁰ with a link to the related archive, where known.

With data created by a broad range of professional organisations, in academia and through community led projects for a wide-range of purposes, the absence of relevant data standards, specifications and reference vocabularies presents a challenging environment for data harmonisation or homogenisation. Even if data is internally consistent from a single provider – and that is not always the case - interoperability across data providers remains challenging.

Case studies

Scotland’s past is a rich and diverse resource ranging from the lowlands to the highlands, from rural to urban and from subtle archaeology revealed as cropmarks and increasingly recognised through a range of remote sensing analysis to upstanding archaeological sites and landscapes. The following examples explore the potential for sharing spatial data from separate projects and a range of techniques – to build something other than the sum of the parts.

Field survey

Excepting a small number of university research projects, for many decades systematic archaeological field survey was the almost exclusive preserve of two national, public-sector organisations. RCAHMS compiled inventories of monuments and Britain’s National mapping agency, The Ordnance Survey, surveyed and published depictions of archaeological monuments as topographic on their paper maps. Both organisations established robust mapping conventions to interpret and present individual site depictions consistently.¹¹ With the need for environmental impact assessments and planning requirements, walk over surveys are now an important part of many infrastructure projects. Spatial data standards have not, however, kept pace with the growth of a thriving private sector.

The RCAHMS Afforestable Land Survey was established in 1989 to undertake strategic targeted surveys in areas where the archaeological potential had not been recognised through low levels of archaeological recording. The survey programme evolved to use a range of digital equipment from electronic distance measurement equipment (EDMs) to differential global positioning systems (dGPS) to map the archaeological landscape. With the advent of Geographic Information Systems (GIS), data initially gathered to produce maps of the archaeological landscape for archive and publication (Figure 3) could be combined across survey projects to present a seamless view of mapped survey areas.

Figure 3. Extract of archaeological survey data from Mar Lodge viewed within a Geographic Information System (GIS).

Background mapping © Crown Copyright HES [2023]. Ordnance Survey licence number 100057073.

The approach is extensible. It could and should form the basis of specifications to realise the potential of mapped data fossilised in project reports from the ever-increasing number of commercial, community and research projects.

Airborne mapping and remote sensing analysis

Transcribing or mapping archaeological features from oblique and vertical aerial photography on a site-by-site basis is a long-established technique. What was a slow and laborious manual process has been transformed through computerised rectification programmes. In Scotland, an Airborne Mapping programme has been undertaken by RCAHMS/HES for over thirty years. Again, the potential to combine individual site transcriptions within the GIS to present a seamless view contextualising individual site transcriptions within a wider archaeological landscape (Figure 4) was quickly realised through development of consistent data specifications similar to, but not identical to, that for field survey mapping.

Figure 4. (A) Airborne mapping of cropmarks reveals a multi-period archaeological landscape at Inveresk, East Lothian © Bluesky International Ltd and Getmapping Plc [2018]. (B) Excavations confirmed the position and character of the ditches observed from aerial photography. Orthoimagery © Bluesky International Ltd and Getmapping Plc [2018].

Geophysical survey techniques reveal anomalies which can be interpreted as archaeological features and sites. The techniques of airborne laser scanning or LiDAR (light detection and ranging) analysis offer fresh opportunities for identifying and mapping archaeology, beyond the rich arable lowlands, into the upland and penetrating the tree canopy. Techniques and methodologies may differ, but the outputs from the interpretation need to be consistent across approaches, with particular attention to the terminology used. Local vocabularies need to reference formal, accessible, shared, and broadly applicable language for knowledge representation to improve interoperability (The ‘I’ of the FAIR Data Principles).

Taking the holistic approach

Consistent application of spatial data specifications is an essential prerequisite to develop a holistic understanding of complex archaeological landscapes from a range of non-invasive and invasive techniques and researchers. The rich archaeological landscape at Inveresk, East Lothian has been mapped through the Airborne Mapping programme and subject to numerous small-scale excavations (Figure 4). Airborne mapping recorded the complex multi-period cropmark landscape, including features identified as Roman temporary camps and a Roman field system overlying a Neolithic Cursus monument. Excavation extents and key features were digitised from the interim report¹² to map the archaeology at Inveresk. Excavations on the line of a new bypass in 1984 were designed to confirm that the linear cropmarks were part of the Roman temporary camp. The results confirmed the interpretation of the western linear features as Roman, but also that the eastern linear features, previously interpreted as a Roman road, were Neolithic in date.

Presenting results

A common product of any research is the presentation of the analysis or interpretation of the results. A prerequisite is a standardised terminology. Although HES airborne mapping and field survey mapping data both acknowledge the Scottish Monument Thesaurus, the poly-hierarchical relationship between a concept, broader terms and top-level terms introduces enough uncertainty and inconsistency to group and present related feature types in a meaningful way cartographically. Development and promotion of micro-thesauri to group the many hundreds of monument terms succinctly and inform the map legend would help address this challenge and help deliver interoperability across a range of observed data.

Untapped potential

Those best placed to document the extents and detail from fieldwork are those undertaking that work. Technological advances help. Widespread availability of high precision surveying equipment coupled with a marked growth in both developer-funded archaeology from the 1990s and community-led projects has democratised survey, be it on excavation, from survey or analysis.¹³ In keeping with the INSPIRE principle that data should be collected once and maintained at a level where this can be done most effectively,¹⁴ primary spatial data should be routinely shared with data curators. For instance, a tabulated gazetteer of sites recorded from a large walkover survey embedded in a pdf is far less useful than the equivalent data presented as a csv file, or preferably as a GIS dataset mapping site extents supported by detailed attribution. Similarly, a plan fossilised as an image in a pdf has less reuse potential than the original georeferenced data.

Sophisticated digital datasets require much richer attribution than ‘entry level’ event data required for established record purposes and basic information discovery. Appropriate attribution should enable a potential user to explore the character of the data¹⁵ at landscape (GIS layers) not project level (pdf).

There are already many highly skilled surveyors and geomatics officers working in archaeology. However, approaches are organisation focused rather than industry based. This is starting to change with large scale infrastructure projects like High Speed 2 which require consistent approaches across multiple contractors and sites. The profession needs to invest in developing and promoting the necessary data standards and demonstrate the benefits of a coordinated approach across techniques and organisations.

The bigger picture

With the establishment of the Geospatial Commission in 2018, the UK government aims to accelerate the delivery of economic, social and environmental benefits derived from geospatial data, products and services across the private and public sector. A key infrastructure project is the National Underground Asset Register, with the Scottish Community Apparatus Data Vault (VAULT) in Scotland, which is coordinating the complexity of buried services from multiple utilities to save time in decision making and save lives on site. In the same vein, archaeologists need to move beyond sensitivity mapping of known areas of high archaeological interest and locations of excavations, to routinely map and share buried archaeological contexts, the location of significant features and key contexts to deposit modelling. If the highly competitive construction industry can coordinate data sharing at scale, so can archaeology.

Archaeological and built heritage data needs to engage with digital twins: virtual models that accurately represent the physical city above and below ground. To do so requires access to detailed, accurate models of our heritage. Without those models, and as remote access to geospatial data for decision-making increases, the absence of archaeological data poses a risk to our past. If that data is invisible or inaccessible within an archive in the wrong format, it has no value in the emerging geospatial ecosystem. Developing consistent archaeological datasets not only makes managing that data more efficient, but also unlocks economic value through saving time and money.¹⁶

Spatial data is essential for the stewardship of the historic environment, not only in terms of managing specific risks at sites but informing the management of the wider landscape. Using a range of spatial data techniques, HES monitors the condition of and risk to properties in its care. In 2010 HES surveyors created a digital model of Skara Brae, a well-preserved Neolithic settlement and UNESCO World Heritage Site on Orkney, to inform site conservation, interpretation and engagement.¹⁷ A cyclical monitoring programme every two years enables comparison of the data over time against the 2010 benchmark to gain a better understanding of the effects of coastal erosion on the environs of the site. The data is not only essential for the stewardship of the site but also forms a case study for Dynamic Coast, a pan-government partnership developing an evidence base of national coastal change across Scotland.

To remain relevant in an increasingly digital society and realise the potential of expensively gathered data, archaeology needs to develop a shared infrastructure, to capture, share and publish a broad range of spatial data adhering to the FAIR Data principles. The manifesto is straightforward:

The value of curated data is already recognised through statutory requirements for spatial data, but the potential of primary archaeological data remains largely unrealised, compartmentalised by project and archive. With advances in geospatial technologies, it is increasingly essential to standardise and share data. Most of the building blocks needed to realise the potential of spatial data from primary archaeological data are in place^18,19 but we lack agreed standards and infrastructure to harmonise and share that data. Without the relevant blueprints, we cannot transform data into information and knowledge effectively (Figure 5).

Figure 5. (A) Fieldwork creates a wealth of archaeological data but without an effective blueprint in place the value of that data tin contributing to the wider archaeological landscape remains unrealised. (B) The sum of the parts: the temptation is to sort and organise data based on similarity – of technique, or by provider. (C) Something other than the sum of the parts: the true value of data can only be realised through an effective blueprint, acknowledging the FAIR Data principles, to build something other than the sum of the parts. (© Crown Copyright HES: Open Government Licence (nationalarchives.gov.uk)).

User requirements and specifications can be addressed through a collaborative approach between data curators, archivists and those creating data through fieldwork and research supported by robust training resources. The substantive challenge is developing, hosting and maintaining the relevant infrastructure to manage the long-term delivery of that spatial data to realise the both the societal and financial benefits of good data stewardship.