Like survey investigation, RRD can entail significant up-front costs. Historically, up-front costs of survey-led social investigations have been borne by the funder of a specific project, and are justified by the value of the end-uses to which the data are put. Smart data (and some other sources, such as administrative data) arise out of unrelated and non-research purposes. They can be acquired for research use only with the consent of the data owner, with any agreement likely subject to legal consents as to third party use. Figure 3-8 illustrates how a syndicated approach to RRD creation can develop and evolve.
Figure 3-8 The creation of reusable RRD for multiple social investigations
Interest in a hitherto unused data source may arise in different parts of the research community, often by serendipity. For example: an employee of an organization that generates smart data may become aware of a potential data use case for a problem that, although not of immediate operational concern, merits a horizon-scanning investigation and may enlist an academic contact in collaboration; or an academic engaged in an industry knowledge exchange project may negotiate personal access to a company’s data in fulfillment of project objectives; or a masters student may become aware that data available through a short-term placement may have potential for their dissertation topic. In each instance, the novelty of the data source may motivate a data service to engage with the data owner to propose more broad-based collaboration, albeit that the provenance of the data remains largely unknown. Research findings thus far do not meet open science criteria of reproducibility and this may also motivate early users of the data to lobby for widened access.
Where potential research usage is assessed as being widespread and worthwhile, a research data service may consult with prospective users to establish priorities for sector-wide data acquisition. The next step is then to explore the possibility of acquisition from the owner and, given a reasonable likelihood of success, to negotiate temporary access to a sufficiently large sub-set of the data. This enables critical appraisal of the value of the data for a wide class of research problems using due diligence analysis of content, coverage and provenance. This has two components: internal validation of the ostensible content of the data and their spatial and temporal granularity; and external validation of the content that can be triangulated against external (statistical and administrative) sources or augmentation with other smart data sources that may also provide cross-validatory evidence.
If the over-all outcome of due diligence is positive, a multilateral data licensing agreement (DLA) may then be drawn up with the data owner. This enables sector-wide sub-licensing of raw data and derivative data products, or RRD, alongside metadata for each. RRD may be developed from a single smart dataset, or by pooling multiple smart and other data sources. RRD are designed to be non-disclosive, in order that they may be distributed to the broadest possible constituency of research users. Once established, these procedures are straightforward to repeat by updating the DLA for updates or extensions to the initial data resources.
These processes together enable an industrialization of the survey investigation research design shown in Figure 3-8. Instead of framing a specific research problem with a unique set of hypotheses within a bespoke research design, data are acquired and engineered for use across a wide range of research projects, and the fitness for purpose of the data is documented in generic terms. The life of the dataset is not confined to a single project, terminating upon acceptance of a single final report: rather, the data support (alone or in combination with other sources) multiple projects. The data infrastructure that the smart data provide are extended and updated in continuous time. Sampling and the apparatus of inference are replaced by triangulation with (less detailed and frequently collected) statistical sources or augmentation with administrative or smart data that establish reliability.