Geospatial Project Design and Research Practice > Research-ready data > A retrospective on sampling and inference in social investigation

Moser and Kalton’s (1971, 1985) Survey Methods in Social Investigation remains a definitive statement of the core organizing principles and practices of social survey research design and implementation, from conception of a research problem, through sample design and monitored data collection to robust, transparent inference and interpretation. Social ‘investigations’ are not carried out in controlled laboratory conditions but typically sample individuals from a known population (‘all that there is’ that is relevant to the research hypotheses) in the observable world. If a sub-group is deemed worthy of special interest to the research, special efforts can be made to adequately represent them in the sample design. The extent of the population is understood, and each member of it has a known and pre-specified chance of selection, independent of use of extraneous considerations such as social media participation, travel card use or voter registration – considerations that may make some elements of the population easier to engage with in data collection. Research proceeds through drawing samples and monitoring of patterning in non-response. It is then possible to use statistical inference to infer population characteristics with known margins of error. The project culminates with report-writing, after which the data are retired from any use apart from checks for reproducibility by other researchers.

Survey-led investigations remain important today (albeit plagued by falling response rates), but this world has fundamentally changed with the advent of vast assemblages of smart data, arising from consumer transactions and other interactions between humans and smart devices (see Longley et al., 2016). This shift has spurred growing use of smart data to underpin geospatial analysis that is data rich but also data driven. Datasets that are ‘found’ rather than designed provide no safeguards with respect to content or coverage, and their use to create representations of the world may ultimately prove delusional, and akin to the hallucinatory results of unchecked AI large language models.

What can be done? A first step is to investigate the potential sources and operations of bias in the creation of smart datasets, in order that they might be anticipated, understood and ultimately accommodated in subsequent analysis. Although precise impacts upon specific analyses may only become apparent in specific applications, metadata on general characteristics may enable multiple reuses of a smart dataset, akin to the reuse of pre-processed scenes from remote sensing. Smart data can be similarly rendered ‘research ready’ by diagnostic checking to documented standards and hence be prepared for multiple purposes. Social investigations present overlapping data requirements, and RRD enables economies of scale in database creation and maintenance to be realized for the wider good. Carefully managed, such RRD can retain similar internal consistency to data collected using the design of surveys for specific investigations – mindful also that non-response rates and weaknesses of some (e.g. clustered) sample designs in the past never truly presented a panacea for analysis.