1. Introduction

Many organisations from community groups to professional archaeological companies, university departments and public bodies are active in archaeological fieldwork and research in Scotland. Since 1947 most fieldwork undertaken in Scotland has been reported annually through an archaeology yearbook, Discovery and Excavation in Scotland (DES) and increasingly online through OASIS (Wright and Richards 2018). Up to 1000 projects are reported annually. These range considerably in scale, from reporting chance finds and new discoveries through to multi-year, multi-technique projects. Developer-funded archaeology, in discharge of a scheduled monument consent on a protected site or a planning condition, and other commercial work accounts for over half the fieldwork reported, with community projects making up about one-third of fieldwork reported. Most entries in DES give the National Record of the Historic Environment (NRHE) as the (intended) place of deposit for their archive.

Most of Scotland's 32 local authorities and the two national parks maintain or have access to Historic Environment Records (HERs) providing 'access to a comprehensive and dynamic information resource about the historic environment of its local area for public benefit and use' (Gilman 2004, 14). HERs may hold project reports and material from fieldwork undertaken in response to a planning condition.

Historic Environment Scotland (HES) hosts the NRHE published as Canmore. The NRHE provides online access to over 320,000 records about the archaeological and built heritage of Scotland and its maritime waters, linked to over 1.6 million items in its archive. With the growth of digital data deposits since the start of the new millennium, the NRHE has invested heavily in developing a digital repository to ensure the long-term preservation and accessibility of these records. The NRHE is the primary place of deposit for most archaeological fieldwork undertaken in Scotland.

2. Purpose and Value of the Archive

Our Place in Time (Scottish Government 2014a), Scotland's first ever strategy for the historic environment, recognises the value the archaeological and built heritage brings to local distinctiveness. The strategy sets out a collaborative approach across stakeholders 'to ensure that the cultural, social, environmental and economic value of Scotland's heritage makes a strong contribution to the wellbeing of the nation and its people' (Scottish Government 2014a, 9). A key strategic priority is to 'ensure that decision making is informed and that sound evidence-based information is available at all levels of decision making' (Scottish Government 2014a, 10). The value of archives as a resource is further recognised in Scotland's Archaeology Strategy.

Following ISAD(G) guidelines (International Council on Archives 2000), the NRHE archives are organised from the general to the specific to aid retrieval within the collections. However, unlike traditional archives, the value of the archaeological archive is realised through the interdependency of the collection with the site inventory. This relationship was eloquently expressed in the late 19th century by David Murray, President of the Glasgow Archaeological Society who called for:

'a survey of all monuments of antiquity of every kind, e.g., pillar stones and cromlechs, circles and alignments, cairns and barrows, camps, forts, and other earthworks, crosses, wells, churches and graveyards, crannogs, peels, castles, and other buildings, and their sites where the buildings are gone, caves, cup and ring-marked rocks, British and Celtic trackways, and Roman roads. Of certain objects photographs or rubbings would be taken; in other cases drawings would be made, in some cases casts, and, where necessary, measured plans, sections and elevations. All illustrations should be made on a determined system, and the same scale should be adopted, as far as possible, as regards each class of objects, and marked upon the photograph, drawing, or plan' (Murray 1896, 29-30).

The quotation also reflects the long-established decoupling of the objects held in museums from the remainder of the project archives - a situation that persists to the present day. That shortcoming aside, the connection between the inventory and the archive has been and remains the strength of the NRHE beyond more generalised repositories.

3. Legislative Framework and Policy Guidance for Archaeological Archives

In the UK, legislation and planning guidance for heritage is generally a devolved responsibility whereas professional standards are broadly applicable across the profession, though acknowledging different approaches across the UK (see Richards this issue). In Scotland, archival practices are informed by Scottish Government policy, professional standards, conditions placed on fieldwork and best practice, rather than required through formal legislation.

3.1 Policy

Scottish Planning Policy (SPP) sets out Scottish Ministers' policy for the nation's historic environment. (Scottish Government 2014b, 33-35). With a strong emphasis on preservation and protection wherever feasible, the policy acknowledges that where this is not possible 'planning authorities should, through the use of conditions or a legal obligation, ensure that developers undertake appropriate excavation, recording, analysis, publication and archiving before and/or during development. If archaeological discoveries are made, they should be reported to the planning authority to enable discussion on appropriate measures, such as inspection and recording' (Scottish Government 2014b, 35).

3.2 Requirements

For work undertaken through Scheduled Monument Consent on a Protected Site or funded through their archaeological commissions or grants, HES requires that the archive is deposited in an appropriate archive. However most archaeological fieldwork takes place on undesignated sites. For work undertaken as part of a planning condition, the SPP requires that planning authorities should 'ensure that developers undertake appropriate excavation, recording, analysis, publication and archiving before and/or during development' (Scottish Government 2014b, 35). This policy is supported by The Planning Advice Note PAN 2/2011 (Scottish Government 2011), which requires that 'Developers should be prepared to undertake appropriate excavation and/or recording before and/or during development, and to support consequential analysis, publication and archiving of the results, and this can be required by the planning authority through the use of conditions or a legal agreement' (Scottish Government 2011, section 32). The local authority archaeologist will specify that a copy of the project report is deposited with their HER and that the archives are deposited in an appropriate repository.

3.3 Archaeological licences

There is no formal licensing system requiring work to be undertaken by a qualified archaeologist. However, the Chartered Institute for Archaeologists (CIfA), the professional body representing archaeologists working in the UK and overseas, recognises 'Registered Organisations' through a quality assurance scheme. Developers are encouraged to commission professionally competent archaeological organisations or consultants, whose work should meet CIfA quality standards (Scottish Government 2011, 5).

3.4 Professional guidance

CIfA publishes a raft of standards and guidance documentation. Using the Standard and Guidance for Excavation as an example, the standard states that 'An archaeological excavation will examine and record the archaeological resource within a specified area using appropriate methods and practices. These will satisfy the stated aims of the project, and comply with the Code of Conduct and other relevant regulations of CIfA. It will result in one or more published accounts and an ordered, accessible archive.' (CIfA 2014, 4). The standard requires that the archaeologist produces a Written Scheme of Investigation (WSI) (or project design). The WSI 'should make suitable provision for archive deposition, and include an appropriate retention and disposal policy for digital, photographic and paper archival materials, as well as artefacts and ecofacts' (CIfA 2014, 9), ensuring that the archaeologist considers archival requirements from the start of the project. For work undertaken in Scotland, the standard is explicit, stating that 'all excavation documentary archives must be deposited in the National Monuments Record for Scotland (now NRHE), which will arrange to copy material to local museums etc.' (CIfA 2014, 15). The standard also requires that fieldwork is reported 'to an appropriate online index', qualified in a footnote as OASIS or its equivalent or successor and that 'a summary interim report must be published in an annual, regional or national digest of fieldwork [DES]' (CIfA 2014, 10-11). Data Management Plans also ensure consideration of digital material throughout the lifecycle of a project.

Both the CIfA archaeological archives special interest group and the Archaeological Archives Forum promote awareness of archaeological archives within the profession. The CIfA selection toolkit informs archive retention policies. The AAF Dig Digital project delivered by DigVentures and CIfA, has produced guidance documents to promote archival best practice. The Archaeology Data Service/Digital Antiquity Guides to Good Practice also provide authoritative guidance for a range of data types.

4. Archives

In Scotland, project archives may often comprise both physical and digital material. The archive is split on deposition. The Scottish Finds Allocation Panel assigns artefactual assemblages to a particular museum but, unless specified as a condition of funding, the researcher may choose a place of deposit for the paper and digital archive. In practice the majority of project archives are deposited with HES.

4.1 The National Record of the Historic Environment

HES was established in 2015 as the lead public body for investigating, caring for and promoting Scotland's historic environment by Act of Scottish Parliament (The Historic Environment Scotland Act 2014). The Act sets out responsibility for the management of the physical and digital archives as a national resource for reference, study and research. The NRHE is funded as part of the block grant to HES from the Scottish Government and revenue raised by HES.

The archives were accumulated over the lifetime of the predecessor body, the Royal Commission on the Ancient and Historical Monuments of Scotland (RCAHMS) and continue to grow annually. The archive comprises both archaeological and architectural collections from the work of RCAHMS and external depositors.

Table 1: Range of catalogued material in the Archives by discipline and category (some material relating to ruined monuments like castles is double indexed). Digital content is shaded.

			Analogue	Digital
Discipline	Number of items	Total %	Number of items	%	Number of items	%
Archaeological Record	753726	46.44	395603	24.38	358123	22.07
Architectural Record	869226	53.56	483356	29.78	385870	23.78
Total	1622952	100	878959	54.16	743993	45.84

		Architectural record	Archaeological record
Category	Total	Total	Total	Dual classification	% Archaeology
Digital images	294449	177723	84487	32239	7.19
Digital non-image file	12152	3160	8139	853	0.55
Scanned images	437241	204937		60015	14.31
Videos	151	50	83	18	0.01
Framed	323	261	5	57	0.00
Library	1634	1207	60	367	0.03
Manuscripts	36907	16844	14741	5322	1.24
Miscellaneous	12	2	8	2	0.00
Newscuttings	442	134	240	68	0.02
Objects	84	80	1	3	0.00
Photographs	702189	385448	221828	94913	19.52
Print Room	48619	37663	1820	9136	0.68
Prints and Drawings	87089	41541	34338	11210	2.81
Rubbings	501	166	143	192	0.02
Transcriptions	1159	10	1138	11	0.07
Total	1622952	869226	539320	214406	46.44

There is a heavy weighting towards photographic and drawn material, reflecting the role of RCAHMS as a body of survey. Summaries of the range of material held in the archive are presented in Table 1 (overall collection) and Table 2 (digital holdings).

Table 2: Digital data formats in the Archive

Categories of digital file		Count	Size (TB)
Images	Digital Images	516736	32.97
	Scanned images	537222	14.31
	Digital multimedia (sound, video. animation)	1363	0.53
GIS	Rectified aerial imagery	16100	0.23
	Digital mapping/GIS	444	0.01
Geophysics	Digital geophysics survey	229	0.01
Graphics	Digital vector graphic (artwork and CAD)	3363	0.02
Text	Digital datasets / spreadsheets	1567	0.01
	Digital structured text / word processed document / PDF	7191	0.04
Total		1084215	48.13

The collecting policy of the NRHE covers archives from individuals and organisations relating to the historic environment of Scotland. Guidance for depositors is published online. There is no charge for depositors lodging an archive with HES. The long-term costs of accessioning, cataloguing and preservation are absorbed by the archive. Canmore acts as an online index to the archive. Only low-resolution images for personal use and some pdf files may be freely downloaded from Canmore. Access to digital files is largely on request, for which there is normally a charge.

As of 27 November 2020, the digital archives comprise 743,993 catalogued records with over 1m digital preservation instances (where items are stored in a range of file formats) totalling 48.13Tb, with a further 3.62Tb of data accessioned awaiting cataloguing from external depositors. As with the analogue archive, the majority of the holdings are internally generated, swollen by an active digitisation programme of analogue photographs. Archaeological archives from external depositors represent less than 3% of the digital record (Table 3) - a figure inflated by digitisation of analogue material.

Table 3: External archaeological content held in the digital archive

Categories of digital file		Count	Size (TB)	Size GB	% of digital record
Images	Digital Images	49861	0.435	445.43	1.00
	Scanned images	29779	0.569	582.39	1.31
	Digital multimedia (sound, video. animation)	319	0.256	261.92	0.59
GIS	Rectified aerial imagery	0	0	0	0
	Digital mapping/GIS/	118	0.008	8.67	0.02
Geophysics	Digital geophysics survey	77	0.005	5.55	0.01
Graphics	Digital vector graphic (artwork and CAD)	1311	0.008	8.66	0.02
Text	Digital datasets / spreadsheets	1059	0.01	0.39	0.00
	Digital structured text/word processed document / PDF	4483	0.009	9.1	0.02
Total		901435	1.291	1322.11	2.97

4.1.1 The digital repository

With the steady growth of digital holdings (Table 2) since the start of the millennium, HES has invested heavily in developing a digital repository. Capacity to manage the long-term stewardship of digital data has increased through integrating Preservica preservation software, with the archive catalogue managed in an Oracle database. The purpose, guiding principles and operational policies for the digital repository are set out in HES Archives: Digital Repository management policies (Historic Environment Scotland 2020). HES also publishes a list of preferred and accepted field formats accepted by the archive (Historic Environment Scotland 2021).

An application for CoreTrustSeal status was submitted in late summer 2020 and is currently under review. Certification will confirm that procedures and processes at HES meet the core characteristics of trustworthy data repositories, reassuring depositors and satisfying some funding bodies. For the wider organisation, HES records management policies inform the retention of documents outside the NRHE.

4.2. Other repositories

The Archaeology Data Service (ADS) will accept digital archives from Scotland, particularly if projects have UK-wide coverage or the place of deposit is specified by the funding body. The ADS make an upfront deposition charge but, once catalogued, project archives are freely accessible online under open licences.

HERs usually hold copies of project reports and associated records, including digital material undertaken as a result of a planning condition and other investigations. They do not have any long-term preservation strategies for digital material. They may also be subject to organisational data retention and archival policies. Project archives may also be deposited with local authority or university museums or retained by the researcher. These archives tend to be orphaned from the related records on Canmore, reducing discoverability and their contribution to the archaeological record. Outside formal digital repositories, the data become increasingly vulnerable to degradation and deletion. Together with the ADS and the Royal Commission on the Ancient and Historical Monuments of Wales, HES forms part of a federated Historic Environment Data Archive Centre within the Marine Environment Data Information Network (MEDIN). Through its network of DACs, MEDIN recognizes that marine data are expensive to gather and are always unique in time and place. There are wide benefits to be gained from working together across disciplines to share and properly manage and promote reuse of that data.

4.3 Artefacts and museums

There is no equivalent to the Portable Antiquities Scheme in Scotland. Unlike the rest of the United Kingdom, where there is a much narrower definition of 'Treasure' under The Treasure Act 1996, Scottish Treasure Trove is based on the Scots common law bona vacantia (ownerless goods). Ownerless objects (excluding modern objects) found by chance or through activities such as metal-detecting, field-walking, or archaeological excavation become the property of the Crown. Objects must be reported and may be claimed as Treasure Trove. After documentation, chance finds along with artefactual assemblages from fieldwork are allocated by the Scottish Archaeological Finds Allocation Panel (SAFAP) to a relevant museum, usually close to where the object was found, or the National Museum Scotland for nationally significant discoveries. Details of objects reported as Treasure Trove are shared with both the national and local records and published in Discovery and Excavation in Scotland.

There is currently no charge for depositing artefactual assemblages with museums. However, following a feasibility study and consultation under the auspices of Scotland's Archaeology Strategy (Mann and Robertson 2020), this situation is likely to change with the introduction of a 'Processing Fee' for the physical element of developer-led archaeological archives.

5. Compliance

Most fieldwork reported through Discovery and Excavation in Scotland states that the intended place of deposit is the NRHE. Despite the obvious flowline from field to deposition, there is no audit trail monitoring that workflow. Consequently, it is challenging to identify what proportion of projects undertaken produce an archive or how many are subsequently accessioned and catalogued.

Empirical experience suggests that although specialist reports form part of the archive, much of the primary data is retained by the author. For instance, it is not uncommon for data from a geophysical survey to be retained privately, with only the report and limited archive deposited with a project archive.

For some standing building surveys, reports are withheld to respect the privacy of the owner. However, there are also differences in working practices, with archaeological companies routinely depositing material with the NRHE as opposed to architectural historians who tend to retain reports within their practice records.

As a national body HES undertakes a range of recording projects through its Heritage and Conservation Directorates. The Heritage Directorate has active field survey and airborne mapping programmes, including analysis of LiDAR. Results are entered directly into the NRHE. The Conservation Directorate uses cutting-edge digital documentation techniques to inform the condition and interpretation of heritage in the care of HES, including the impact of climate change on the historic environment. Project archives are currently retained within the Directorate as working datasets.

Novel digital techniques, including 3D modelling, as well as augmented and virtual reality projects, bring the past to life online. With many models developed in Computer Science and visualisation labs rather than in archaeological organisations, the long-term preservation of the underlying data needs to be considered.

6. How FAIR is the NRHE?

Canmore provides powerful tools, from a keyword search to more advanced options, including filtered searches on the collections and site tables through to web map browsers (on Canmore and PastMap). As a portal, Canmore reaches a wide audience, regularly returning user figures in excess of 840,000 users and 4.7m page views annually (Historic Environment Scotland and Built Environment Forum Scotland 2016, 68). However, the potential of the underlying data is not fully realised in terms of knowledge discovery, representation and integration either by humans or machines. The fitness of data in the NRHE to support information discovery, sharing and reuse, particularly in machine-readable formats, may be assessed against the FAIR Data principles (Wilkinson et al. 2016).

6.1 Findability

Site and Collections records are each identified by locally issued web addresses. Records are comprehensively described, though not in a machine-readable format. The relational nature between these elements provides a wealth of searchable metadata covering location, type of site, spatio-temporal coverage searchable through text- or map-based searches. Archival information includes the creator (person and organisation), type of material, a description, copyright.

6.2 Accessible

Canmore is a fully searchable online database enabling users to search through a simple keyword search or more detailed site or collection searches. Users may also use the Canmore map search or PastMap ; the latter including Protected Sites and HER data) for spatial searches. Excepting the HER data, layers presented in PastMap may be accessed as View (Web Map Services) or download (ATOM feed) services from the HES download portal. These services are formally catalogued to UKGemini, the UK geographic metadata standard through https://spatialdata.gov.scot, harvested to https://data.gov.uk as well as on the MEDIN metadata portal.

Users are able to self-select and download up to 1000 records as csv or kml files under an Open Government Licence from Canmore and PastMap searches. More complex data searches and requests for copies of archive material, including most digital files, are chargeable. Users may view images online through a zoomable browser and download low-resolution images for personal reuse.

6.3 Interoperable

View and download services are provided in open, non-proprietary formats and as Shape files for spatial data. Bespoke data requests are usually in proprietary formats: MS Access, MS Excel or as Shape files. Data content largely draws on internally managed vocabularies. As the lead organisation for the historic environment in Scotland, HES publishes standardised machine-readable key vocabularies (Monuments, Objects, Maritime craft and Periods) with universal identifiers through Heritagedata.org. However, the semantic capabilities of the data are not realised either within Canmore or in any outputs, greatly restricting reuse potential. Extracts of site records from Canmore are provided periodically to the Archaeology Data Service ArchSearch, harvested to ARIADNE via ARIADNE mappings.

6.4 Reusable

CIfA standards (2014, 17) consider that the copyright and ownership of the paper and digital archive from archaeological work usually rests with the archaeological organisation undertaking the work. Title or licences are usually transferred to the recipient institution on transfer of the archives. Where the depositor wishes to retain Intellectual Property Rights for material they have created, a separate licence must be signed to allow HES to disseminate material as per their advertised terms and conditions.

The terms and conditions and User licence for using Canmore are set out at resource level and are considered contractually binding (HES 2020, 12). Copyright and licencing is asserted over individual items in the archive. On requesting material from the archive, the user must accept further specific licensing conditions. Low-resolution images may be downloaded for personal use, excluding Commercial Use or Sale, No Public Distribution (e.g. by hand, email, web). However, as noted under interoperability, the Open Government Licence applies to spatial data, enabling users to freely reuse data for their own purposes.

7. To the Future

With ever-increasing capacity to capture, process and model the past, the NRHE can expect an exponential growth in the data volumes being deposited. There needs to be greater understanding of the likely data volumes and formats deposited each year to model and plan for data storage. More of the large volume digital data models created by HES and other organisations needs to be formally archived within the NRHE. Drawing heavily on remotely sensed data, the HES Rapid Archaeological Mapping Project (RAMP) (Cowley et al. 2020) aims to create a rapid analysis and identification of archaeological sites nationally. RAMP will also be developing capacity and expertise in geophysical survey, informing practice and deposition. With large and complex data, low-resolution dissemination models should be routinely displayed within the relevant pages in Canmore. There is also a need to assess the wealth of AR and VR models created outside the traditional archaeological depositors. Currently it is impossible to quantify the percentage of projects undertaken each year producing an archive that actually deposit that archive online. Ongoing redevelopment of the DES and OASIS reporting form will start to build that flowline.

With a long-established archive following traditional copyright and licensing policies, change can sometimes be slow. Current management policies reflect changing attitudes: 'We will deploy open licensing frameworks where applicable and or necessary, to comply with research funding requirements and service obligations including Creative Commons and Open Government Licences with the terms of use of digital assets and any attribution included. HES also utilise a range of open mapping products for the purposes of site identification, context navigation, resource location, and spatial data download'. (HES 2020, 8). The next few years will see how successful HES is in developing an Open Licence framework and making more digital data downloadable from Canmore.

Acknowledgements

I am grateful to Alex Adamson for commenting on an earlier draft and to colleagues in HES archives for providing some information for this article.