JLJoseph Lam

Joseph Lam

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Joseph Lam has a total of 68 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Evaluation of linkage between health, education and social care administrative data for 21 million children in England

    by Vincent Nguyen, Joseph Lam, Tony Stone, Ruth Blackburn, Ruth Gilbert, Milagros Ruiz Nishiki, Katie Harron

    Background: Population-level administrative data cohorts hold huge potential for generating evidence to improve child health. Understanding the quality of these data is critical to enabling us to fully realise this potential. We aimed to evaluate linkage between children’s health, education and social care administrative records (ECHILD) to identify possible sources of bias. Methods: We created four cohorts to understand the drivers of inclusion/exclusion within ECHILD. For each cohort, we calculated the linkage rate by counting the total number of eligible individuals who appeared in the ECHILD linkage spine and any of the corresponding hospital/school component datasets. We then assessed whether linkage rates varied according to sociodemographic characteristics (year of birth, sex, ethnicity, region). Results: ECHILD currently captures 34.2 million patients and 25.3 million pupils born between September 1984 and March 2023, aged 0-38 years. Over 90% of children with hospital birth records linked to a school record (rising to 93% for those born from 2014 onwards); 88% of children attending school in 2001/02 linked to a patient record (rising to 96% for those born in 2019/20), resulting in a total of 21 million linked individuals. Substantial variation existed between sociodemographic groups, with those living in London, residing in more deprived areas, or with recorded ethnicity other than White being less likely to be included. Conclusions: Population-level datasets such as ECHILD often under-represent specific groups. Irrespective of the mechanisms by which individuals are excluded (e.g. opt outs, linkage errors or not interacting with services), those in minoritised ethnic groups and those living in more deprived areas are most affected. Linkage evaluations are critical for understanding who is and who is not included in analyses of these data so that researchers can account for potential sources of bias within analyses. Highlights: We evaluated linkage quality and population coverage within ECHILD, a national, longitudinal, linked health, education and social care data resource. ECHILD contains linked health and education records on 21.3 million children born from 1984 onwards; linkage rates were >90% for hospital birth cohorts and >88% for school inception cohorts. Linkage rates varied by ethnicity, deprivation and geography, with lower representation of minoritised groups and children living in London. Routine linkage evaluations are essential for quantifying selection bias and supporting valid use of population-level administrative data; our findings demonstrate the importance of transparent reporting of linkage quality for population data science research.

  • Open Access

    Making the invisible visible: maternal and child health inequalities for Latin Americans in the UK

    by Cristina Fernandez Turienzo, Maria Alvarez Gomez, Luana Barbagelata, Jacobo Belilty, Bruna Boscaini, Alex Hernandez, Rose Higgins, Joseph Lam, Ramon Luengo Fernandez, Diana Medina, et al.

  • Open Access

    How Ethnic Name Characteristics Influence String Similarities and Cause Bias when Linking Large Population Databases

    by Yashithi Dharmawimala, Peter Christen, Sumayya Ziyad, Rainer Schnell, Joseph Lam, Anushka Vidanage, Thilina Ranbaduge

    Comparing personal names is at the core of most record linkage systems, where it is common for names to be recorded with variations, either erroneous or intentional, and individuals can change their names over time. Therefore, string comparison functions are commonly employed to calculate similarities between names. Popular comparison functions include Jaro-Winkler, edit distance, and q-gram based techniques. Although the question of which functions are more suitable for a given record linkage application has received little attention, the selection of a suitable string comparison function can be crucial in the context of bias and fairness. This is particularly important when linking databases that contain ethnically diverse names. In this work, we investigate how different string comparison functions can result in various levels of bias when names from diverse ethnic groups are compared. We conduct extensive experiments on a large public population database where for each record an ethnicity is available. By comparing name variations of the same person using multiple string comparison functions, we show that different functions exhibit diverse similarity distributions for names from different ethnic groups. Moreover, the similarity score for the same name pair can vary greatly depending on the comparison function employed. Our results provide important insights into how practical record linkage of population-level databases should be conducted to limit bias with regard to ethnic groups.

  • Open Access

    A comparison of the Alkire-Foster method and a Markov random field approach in the analysis of multidimensional poverty

    by Joseph Lam

    Multidimensional poverty measurement is crucial for capturing deprivation beyond income-based metrics. This study compares the Alkire-Foster (AF) method and a Markov Random Field (MRF) approach for classifying multidimensional poverty using a simulation-based analysis. The AF method applies a deterministic threshold-based classification, while the MRF approach leverages probabilistic graphical modelling to account for correlations between deprivation indicators. Using a synthetic dataset of 50,000 individuals with ten binary deprivation indicators, we assess classification accuracy, false positive/negative trade-offs, and agreement between the methods. Results show that AF achieves higher classification accuracy (89.5%) compared to MRF (75.4%), with AF minimizing false negatives and MRF reducing false positives. The overall agreement between the two methods is 65%, with discrepancies primarily occurring when AF classifies individuals as poor while MRF does not. While AF is transparent and easy to implement, it does not capture interdependencies among indicators, potentially leading to misclassification. MRF, though computationally intensive, offers a more nuanced understanding of deprivation clusters. These findings highlight the trade-offs in multidimensional poverty measurement and provide insights for policymakers on method selection based on data availability and policy objectives. Future research should extend these approaches to non-binary indicators and real-world datasets.


  • Open Access

    Strategies for ascertaining timing of address changes in administrative data

    by Joseph Lam, Mario Cortina-Borja, Peter Christen, Richard Thomas, Robert Aldridge, Ruth Blackburn, Ruth Gilbert, Andy Boyd, Katie Harron

    Addresses captured in administrative health data can facilitate the analysis of place-based exposures and their influence on health. If patients change address, hospital records are updated when individuals next use health services: precise timing of addresses changes must therefore be inferred. There is therefore a need to determine the temporal accuracy of address recording in these data. Using deidentified data, we compared Lower Super Output Areas (LSOAs) derived from English hospital records (NHS_E, n = 40,102), with linked data on addresses recorded in the UK Longitudinal Linkage Collaboration cohorts (Cohorts, n = 40,963) from January 1989 to April 2023. We compared the accuracy of three methods for estimating the timing of changes in LSOA: 1) inferring the end date of the current address as the start date of the next reported address minus 1 day (N-1 method); 2) using median date between current and next start date as the end date for current address, and to update the start date for the next address (Median method); 3) generating the address end date as a function of beta distribution defined in terms of the current and next start dates, assuming that most people update their address soon after they move (random method). In total, 39,216 (95.7%) Cohort members had at least 1 matching LSOA reported in both Cohorts and NHS_E data. 47% of these matching LSOAs were recorded in the hospital data within the two years prior to or following the Cohorts recorded dates. All three methods demonstrated LSOA agreement of ~78%, with negligible differences across methods. Methods for estimating timing of changes in hospital-recorded addresses are reasonably accurate when compared to Cohorts data but rely on assumptions about data quality and data generation mechanisms. Researchers should consider these assumptions and the implications of each method to justify their approach.

  • Terminating bias: How Arnold Schwarzenegger showed us the importance of spelling names correctly

    by Joseph Lam

    In his winning entry for our 2024 Statistical Excellence Award for Early Career Writing, Joseph Lam explores data linkage errors and biases and the implication for ethnically minoritised populations


  • Open Access

    “What master do you serve?” Struggles in university EDI committees

    by Joseph Lam

    In light of the recent suspension of the Equality, Diversity, and Inclusion (EDI) advisory group by UK Research and Innovation, this article invites a reflection from my observations as a member of an EDI committee of negative attitudes towards Advance Higher Education charters and EDI committees at higher education institutes. Using race equity as an example, I hope to suggest why these negative attitudes from institute members are by design inevitable, if the Advance Higher Education framework is the only angle in which EDI progress is understood.

  • Open Access

    Generating synthetic identifiers to support development and evaluation of data linkage methods

    by Joseph Lam, Andy Boyd, Robin Linacre, Ruth Blackburn, Katie Harron

    IntroductionCareful development and evaluation of data linkage methods is limited by researcher access to personal identifiers. One solution is to generate synthetic identifiers, which do not pose equivalent privacy concerns, but can form a 'gold-standard' linkage algorithm training dataset. Such data could help inform choices about appropriate linkage strategies in different settings. ObjectivesWe aimed to develop and demonstrate a framework for generating synthetic identifier datasets to support development and evaluation of data linkage methods. We evaluated whether replicating associations between attributes and identifiers improved the utility of the synthetic data for assessing linkage error. MethodsWe determined the steps required to generate synthetic identifiers that replicate the properties of real-world data collection. We then generated synthetic versions of a large UK cohort study (the Avon Longitudinal Study of Parents and Children; ALSPAC), according to the quality and completeness of identifiers recorded over several waves of the cohort. We evaluated the utility of the synthetic identifier data in terms of assessing linkage quality (false matches and missed matches). ResultsComparing data from two collection points in ALSPAC, we found within-person disagreement in identifiers (differences in recording due to both natural change and non-valid entries) in 18% of surnames and 12% of forenames. Rates of disagreement varied by maternal age and ethnic group. Synthetic data provided accurate estimates of linkage quality metrics compared with the original data (within 0.13-0.55% for missed matches and 0.00-0.04% for false matches). Incorporating associations between identifier errors and maternal age/ethnicity improved synthetic data utility. ConclusionsWe show that replicating dependencies between attribute values (e.g. ethnicity), values of identifiers (e.g. name), identifier disagreements (e.g. missing values, errors or changes over time), and their patterns and distribution structure enables generation of realistic synthetic data that can be used for robust evaluation of linkage methods.