Psychometric properties of the rapid neurodevelopmental assessment in detecting social-emotional problems during routine child developmental monitoring in primary healthcare

Abstract

Background

The global prevalence of social-emotional problems in children and adolescents is nearly double in First Nations populations compared to non-First Nations populations, highlighting health inequities due to the impact of colonisation. Addressing this requires culturally responsive social-emotional screening in primary healthcare, enhanced by a simple, psychometrically sound tool. The Rapid Neurodevelopmental Assessment (RNDA) is user-friendly, incorporates child observations and parental input, and can be used by primary healthcare providers. This study evaluated the RNDA’s performance in screening social-emotional problems during routine health checks with First Nations children.

Methods

Working with an Aboriginal Community Controlled Health Organisation in Australia, children (60% male, 92% identifying as First Nations) aged 3 to 16 years (M = 8.40, SD = 3.33) and a caregiver participated in this study as part of a health check. The convergence with, and accuracy of, children’s scores derived from single-item measures of seven social-emotional problems on the RNDA was compared to their corresponding multi-item scores from the parent-report Behavior Assessment System for Children 3rd Edition (BASC-3).

Results

Each of the single-item measures on the RNDA were significantly correlated with the corresponding multi-item construct on the BASC-3, except for anxiety. The total accuracy of the RNDA relative to the BASC-3 was 58 to 81%, with high sensitivity for four of the seven items: hyperactivity (90%), attention problems (87%), externalising problems (82%) and behaviour symptoms index (88%). Sensitivity of the remaining items ranged from 14 to 71% and specificity ranged from 29 to 88%. The measure showed an average positive predictive value of 50% and negative predictive value of 75%.

Conclusions

The single-item measures within the RNDA’s behaviour domain showed good convergent validity relative to the BASC-3. Most items had acceptable accuracy, comparable with similar screening measures. These findings further support the RNDA’s integration into First Nations child health checks, allowing for a rapid, holistic assessment of child development to improve health equity.

Introduction

The estimated global prevalence of social-emotional problems (i.e., issues that affect mood, thinking, and behaviour such as depression, anxiety, and neurodevelopmental disorders) affecting children and adolescents is approximately 14% [1], with the prevalence more than doubling in First Nations populations world-wide, such as in Canada (33%), Australia (30%), New Zealand (30%), and America (20%)[2]. Such discrepancies in prevalence rates, like broader health inequities in First Nations populations, are rooted in the ongoing consequences of colonisation (e.g., displacement from land, loss of cultural continuity, oppression, and exploitation of marginalised groups). Colonisation has significantly influenced the social determinants of health, or the conditions in which people are born, grow, live, and work, such as socioeconomic status (SES), education, environment, employment, and access to healthcare [3]. These social determinants disproportionately affect marginalised communities and perpetuate a cycle of inequity, further cementing stark differences in health outcomes among certain populations [4].

The effects of colonisation on First Nations populations worldwide are profound and enduring, with particular evidence of the impact on social determinants of health [5]. One example is poverty, which is the primary driver of numerous health disparities among First Nations communities and is a direct result of the dispossession of First Nations peoples from their lands and resources by colonial powers. This disruption of traditional ways of life, policies of displacement, and racial segregation of First Nations peoples led to education disparities, economic disenfranchisement, and loss of self-determination for many First Nations groups. These social determinants, in addition to geographic isolation and systemic discrimination of First Nations people perpetuate cycles of inequity that persist today [6].

Efforts to address these disparities must prioritise the provision of culturally responsive healthcare, screening, and support strategies by professionals that understand the unique needs and perspectives of First Nations communities [2]. In Australia, annual health checks conducted by Aboriginal Health Workers/Practitioners (AHW/Ps) play a crucial role in bridging the current healthcare gap. AHW/Ps are ideally placed to conduct such health checks given their invaluable knowledge and understanding of the unique needs, cultural nuances, and historical contexts of Australia’s First Nations communities [7].

The effectiveness of health screening, specifically as it relates to child development and social-emotional wellbeing, can be significantly enhanced through the utilisation of culturally acceptable and psychometrically sound tools. Most psychometrically sound tools assessing child development require administration by qualified specialists, are expensive, and difficult to access in primary healthcare [8]. This contributes to health system inequities by creating assessment and treatment bottlenecks, especially in rural and remote regions, where there is a high turnover of specialists [9,10,11] and access to developmental assessment services and allied health support is limited [7]. Concerns also extend to the appropriateness of these tools for children from families of lower SES, a significant demographic in rural and remote communities [12], given that the most popular tools for developmental screening and assessment were developed in the USA or UK and validated on samples of urban children from families of high SES. However, there is an opportunity to address the limitations of current screening tools available to the primary healthcare sector with a relatively new measure, the Rapid Neurodevelopmental Assessment, modified for use in the Australian context (RNDA) [13].

The Rapid Neurodevelopmental Assessment

The RNDA is a cost-effective, observational, and functional assessment of neurodevelopmental impairment across nine domains (gross motor, fine motor, vision, speech/expressive language, hearing, cognition, behaviour, self-care, and seizures) for children from birth to 16 years. The behaviour domain includes a set of single-item measures designed to screen for various social-emotional problems, including anxiety, atypicality, and attention, setting it apart from other commonly used assessment and screening tools such as the Strengths and Difficulties Questionnaire (SDQ) and the Behaviour Assessment System for Children, Third Edition (BASC-3) [14], which include many more items (e.g., 25–170). The use of single items in the RNDA makes the measure a time and cost-effective way to identify strengths, impairments, and the need for further assessment and intervention.

Comprehensive assessments are often required for a thorough understanding of a child’s developmental or emotional challenges. The RNDA offers a brief, targeted screening tool designed for use in primary healthcare settings, particularly in rural and remote First Nations communities. In these contexts, the RNDA’s efficiency is critical, as it can be administered quickly, even by healthcare practitioners without specialised training, thus overcoming the logistical challenges posed by high turnover rates of specialists and the high demand for screening in such communities. Single-item measures have been argued to be acceptable and effective when the constructs they assess are unidimensional, clearly defined, and narrow in scope [15], and their face and convergent validity can be demonstrated. Recent editorial discussions [15] have highlighted multiple advantages of single-item measures, particularly in time-restricted and resource-limited settings, where they can be a valuable tool for identifying key areas of concern and facilitating prompt intervention.

Originally developed in Bangladesh and validated specifically for use in low-income countries [16,17,18], the RNDA has been modified to better suit the Australian context and digitised to increase accessibility and ease of use [19]. The RNDA provides valuable information about the type and severity of symptoms and can be completed by a broad range of healthcare practitioners.

While the current body of research on the original RNDA has shown promising interrater reliability and concurrent validity, these studies were undertaken in Bangladesh and Guatemala, with evidence of the measure’s utility in Australia still emerging. Furthermore, the original RNDA has previously shown acceptable sensitivity (70–83%) and moderate specificity (57–84%), but only as it relates to the identification of intellectual impairment [18, 20].

The social-emotional wellbeing section of the RNDA relies on single-item screeners, which if valid, provides an economical and efficient addition to triage protocols. The digitisation, efficiency of single-item measures, and insights into type and severity of symptoms can overcome some limitations of other commonly used measures, particularly in resource-limited communities. This makes the RNDA a strong candidate for integration into annual health check protocols in primary healthcare. However, the accuracy of the RNDA as a tool to identify social-emotional problems is yet to be demonstrated. Uncovering evidence of the RNDA’s accuracy in identifying social-emotional problems in a sample of Australian First Nations children and adolescents further strengthens the case for its integration into routine developmental monitoring during health checks for First Nations children.

Study aims

The aims of this study were to evaluate the concurrent validity and the accuracy of healthcare providers’ responses to the RNDA across seven social-emotional problem constructs. The RNDA included single items for each construct, which are assessed based on observations of child (aged 3 to 16 years) and caregiver information. The reference test was the BASC-3 completed by a caregiver. It is hypothesised that: (1) the single- RNDA items used to assess social-emotional problems will be significantly correlated with the comparable multi-item composite score derived from the BASC-3 (see Table 1); (2) the RNDA will show acceptable accuracy (≥ 80% sensitivity and specificity) in identifying children who have scored above the clinical cut off (i.e., T- score 65 or above) on comparable scales on the BASC-3, as well as moderate positive predictive value (PPV) and high negative predictive value (NPV) in line with expectations for screening measures [21].

Methods

Measures

The Behavior Assessment System for Children 3rd Edition (BASC-3) [14]

The BASC-3 Parent Rating Scale (PRS) was used as the reference measure. The BASC-3 was designed for assessing behavioural and emotional problems in children and adolescents and has three forms: preschool (2-5yrs), child (6-11yrs), and adolescent (12-21yrs). With an estimated administration time of 20 minutes, the BASC-3 has over 170 items that are descriptions of observable positive or negative behaviours. The caregiver responds to each item with Never (0 points), Sometimes (1 point), Often (2 points), or Almost always (3 points). Items are summed according to the scale to which they belong, yielding a raw score, which is then converted to a normative t-score. Higher scores indicate more problems. The current study utilised gender-specific norms. T-scores are computed for 4 content scales and 15 subscales, including hyperactivity, anxiety, and emotional self-control, as well as composite scores for externalising problems (hyperactivity + aggression + conduct problems) and a behavioural symptoms index (attention problems + atypicality + withdrawal). Table 1 outlines the 11 BASC-3 subscales that align with the RNDA behaviour domain items relevant to this study. The BASC-3 has been shown to have strong psychometric properties, with high internal consistency and test-retest reliability (α = ≥ 0.80) and excellent sensitivity (0.95 − 0.97) and specificity (0.79 − 0.80) [22]. Additionally, while the BASC-3 relies on a USA normative sample, Tan and colleagues [23] found evidence to support its cross-cultural validity among Australian children.

To assess convergence, t-scores were used in correlational analyses. For calculations of sensitivity, specificity, total accuracy, positive predictive value, and negative predictive value, BASC-3 scores were dichotomized using a cut-off score of 1.5 standard deviations from the mean (1 = 65 or greater t-score, 2 = 65 or lower t-score). A cut off score of 1.5 standard deviations from the mean was selected as this is a common cut point for categorising children and adolescents at-risk for psychological and behavioural problems on a range of assessment instruments [24].

Rapid Neurodevelopmental Assessment (RNDA) [13]

The RNDA has 31 age-specific forms to accommodate developmental changes from birth to 16 years. Each form includes screening questions tailored for each specific age range across nine neurodevelopment domains (gross & fine motor, vision, hearing, speech, cognition, behaviour, self-care & seizures). The current study examined the items within the behaviour domain, which had 3 variations across the age ranges: one for children under 5 years, one for children aged 5–9 years, and one for children aged 10–16 years (see Table 1). Items across all 3 variations were designed to be stand-alone single-item measures of different developmental and social-emotional problems. Some of these items were combined to create two additional subscales correspondent to the BASC-3 (i.e., externalising problems & behavioural symptoms index; see Table 1). Items are brief and draw attention to both strengths and weaknesses in social-emotional functioning. Each of the single items within the behaviour domain are designed to be scored with consideration of both administrator observation of the child during the screening and caregiver recall, particularly in instances where specific items may not be observed during the session (e.g., odd or unusual behaviour). In these instances, practitioners can flexibly adapt the wording of items according to the standards outlined in the manual to increase relatability for families.

For children under 5 years, individual items are scored as Present (i.e., the specific item is observed consistently; 0), Most of the time (i.e., the item is observed frequently; 0.5), Infrequent (i.e., the item is observed infrequently; 1), or Never (i.e., the item is never observed; 2). For children over 5 years, items are scored as No (i.e., there is no concern; 1) or Yes (i.e., there is a concern; 2). The 4-point scoring system for children under 5 years allows for a more nuanced assessment of emerging skills, reflecting the gradual nature of early development and skill acquisition. In contrast, the dichotomous scoring system for children over 5 years is appropriate as developmental skills at this stage are typically more stable and clearly present or absent, enhancing screening efficiency. There are some items relating to social-emotional problems that remain consistent across all age ranges (e.g., withdrawal, atypicality, attention problems), and some that emerge from age 5 (e.g., aggression, conduct problems). Due to differences in scoring across the age ranges, scores for participants under 5 years were dichotomised, and items reverse scored so that a score of 1 = Impairment and 2 = No impairment. Table 1 outlines how each of the relevant RNDA items align with subscales measuring corresponding constructs on the BASC-3.

Table 1 Alignment of RNDA behaviour domain items with constructs on the BASC-3

Data collection tools and procedure

The health check comprised of several components, including: (1) demographic details; (2) cultural connections; (3) prenatal, developmental, educational, medical, and social history; (4) developmental screening utilising the RNDA; (5) clinical observations (vitals); (6) body systems review and physical examination (for detailed description of the health check see [7]). The Aboriginal Health Workers/Practitioners (AHW/Ps) completed components 1–5 and a general medical practitioner completed component 6. Where concerns were raised by a caregiver during the health check or flagged by the RNDA, the AHW/Ps administered the BASC-3 at a follow-up session with the caregiver, which was then entered and scored digitally using a secure cloud-based scoring system. Families where no concerns were raised during the health check, but who agreed to be contacted for future research, were contacted by phone and asked to complete the BASC-3 over the phone with a university research assistant. Data were entered into a REDCap database [25] developed in partnership with the ACCHO staff members. Health providers were blind to the purpose of this study. Only relevant demographic data, RNDA scores, and BASC-3 scores were used in the current study. Participants who had completed multiple health checks between 2019 and 2022 were only included in the study once, with the RNDA and BASC-3 data administered closest in time selected for inclusion.

Statistical analyses

Point-biserial correlations (r_pb) were used to draw conclusions about the concurrent validity (convergence) of the RNDA screening scores (dichotomous data) compared to the BASC-3 subscale scores (t-scores). Cross-tabulation of dichotomous indicators derived from the RNDA and BASC-3 results was used to produce the sensitivity, specificity, PPV, NPV, prevalence, and total accuracy of the RNDA with the BASC-3 used as the reference measure.

Results

Study setting and participants

This project was undertaken in partnership with an Aboriginal Community Controlled Health Organisation (ACCHO), which has a catchment area spanning approximately 640,000 km [2], one of the largest covered by a single ACCHO in Queensland, Australia. Ethical clearance was granted by the Griffith University Human Research Ethics Committee (2022/362) and permission from the committee representing the local Traditional Owners of the Land was granted.

The focal participants were 84 children (60% male, 92% First Nations) who underwent a health check between 2019 and 2022, and had their development screened using both the RNDA and the BASC-3. Children were aged 3 to 16 years (M = 8.40, SD = 3.33). Of the participants where the administrator of the RNDA was documented (N = 40), 90% (n = 36) were completed by healthcare workers who identify as First Nations, 5% (n = 2) by a general practitioner who identifies as First Nations, and 5% (n = 2) by a speech pathologist. Documentation of the RNDA administrator was incomplete in some cases due to the multiple demands inherent in clinical work and a need to prioritise clinical over administrative tasks. Caregivers who answered questions for the RNDA and completed the BASC-3 attended the health check appointment with the child and were biological mothers (n = 40; 48%), other family members (i.e., siblings, grandparents, aunties/uncles; n = 26; 31%), or foster mothers (n = 9; 11%). Most children reported identifying with more than one Nation group, with 24% of participants identifying as Kalkadoon, 21% Waanyi, and 12% Waliwarra. All families were from a “remote” or “very remote” region, as classified by the Modified Monash (MM) Model (MM 6 = remote; 7 = very remote) [26]. During the data collection period, there were 90 children that had undergone a health check. However, one participant was excluded because they were younger than 24 months and had been screened using different RNDA items than all other participants. Another five participants were excluded due to missing scores on individual items on the RNDA, which were required for the analyses in this study.

As outlined in Table 2, correlations between the RNDA and BASC-3 scores were statistically significant, except for anxiety.

Table 2 Correlations between the RNDA and the BASC-3 for constructs of interest

Table 3 outlines the sensitivity, specificity, total accuracy, prevalence (as indicated by the BASC-3), PPV, and NPV, of the RNDA when compared to the BASC-3 on each of the corresponding constructs. The sensitivity of the RNDA ranged from 14 to 90% and the specificity ranged from 24 to 94%. The PPV ranged from 7 to 79% and the NPV ranged from 63 to 90%.

Table 3 Percentage values for assessing the accuracy of the RNDA when compared to the BASC-3

Discussion

The current study aimed to validate single-item measures on the RNDA against multi-item reliable and valid subscales on the BASC-3. Statistically significant correlations were found for all single-item measures on the RNDA, except for anxiety, suggesting that all other items are accurately capturing the desired constructs. These results provide strong evidence that each item within the behaviour domain, except for anxiety, can be considered valid as single-item measures. Additionally, these findings validate that the RNDA can be used as intended, where the individual social-emotional items reveal strengths and impairments in social-emotional functioning, while the total score for the behaviour domain provides insights about the severity of these impairments [13, 27]. These findings extend previous research by Khan and colleagues [16,17,18] and Muslima and colleagues [20] and provides further evidence of the convergent validity of the RNDA behaviour domain when screening for social-emotional problems within Australian First Nations children.

The weak correlation of the anxiety construct might be attributed to the limited number of participants who endorsed anxiety on the BASC-3. The prevalence of anxiety in the current sample was unusually low at 10%, where the prevalence rate of anxiety for Australia’s First Nations population is approximately 17% [28]. Further, the anxiety-related item on the RNDA (“does the child have any extreme fears?”) may not effectively capture generalised anxiety symptoms. The term ‘extreme fears’ could be interpreted as referring specifically to phobias rather than broader manifestations of anxiety. Given that this item was not adapted for the Australian context, our findings suggest that revision may be necessary to ensure it accurately captures anxiety while being culturally appropriate. For First Nation populations, fear stems from deep-seated traumas such as the Stolen Generation, genocide, and displacement from land and traditional ways of life by colonial powers. While anxiety is a common experience, fear encompasses a distinct set of thoughts and emotions shaped by these profound historical and personal tragedies. More inclusive wording, such as ‘worried,’ ‘concerned,’ ‘nervous,’ or ‘uneasy,’ may improve caregiver understanding and enhance the item’s validity in recognising anxiety in children across cultural contexts.

The total accuracy of the RNDA across social-emotional constructs ranged from 58 to 81%. In community screening programs, where resources are often limited and the goal is to identify at-risk children, higher sensitivity is often prioritised to ensure that no child with social-emotional problems is missed [29]. The RNDA social-emotional items demonstrated high sensitivity across the constructs of hyperactivity (90%), externalising problems (82%), attention problems (87%), and the behavioural symptoms index (88%) and high specificity for aggression (85%) and conduct problems (88%). Interestingly, other screeners, such as the Ages and Stages Questionnaire (ASQ-3) [30] and the Parents’ Evaluation of Developmental Status: Developmental Milestones [31], utilise multi-item subscales to assess the same constructs as the single-items of the RNDA and have comparable sensitivity (i.e., 86% and 83% respectively).

On the whole, the sensitivity and specificity values of the social-emotional items on the RNDA are comparable to other commonly used measures. A systematic review [32] identified related instruments often described in literature and detailed the pooled sensitivity and specificity values for the Pediatric Screening Checklist (Se = 72%, Sp = 88%), Strengths and Difficulties Questionnaire (SDQ; Se = 65%, Sp = 76%), Child Behavior Checklist (Se = 63%, Sp = 84%), Ages and Stages Questionnaire: Social-Emotional (Se = 73%, Sp = 88%), and Brief Infant-Toddler Social Emotional Assessment (Se = 80%, Sp = 82%). Consistent findings were uncovered in another meta-analysis [33] for ADHD symptoms assessed by the SDQ (Se = 59%, Sp = 79%) and the Achenbach System of Empirically Based Assessment (ASEBA) DSM-IV ADHD subscale (Se = 75%, Sp = 81%) and Attention Problems subscale (Se = 73%, Sp = 77%). These reviews corroborate that it is not uncommon for such measures to have higher specificity, despite higher sensitivity being preferred for screening measures.

The sample size could have had an effect on the sensitivity and specificity values. Bujang and Adnan [34] utilised PASS software to calculate the minimum sample size required for finding sensitivity and specificity of screening tools. The minimum sample size proposed to find high sensitivity is 163 participants with 49 of these having the disease (prevalence = 30%, power = 0.81). The minimum sample size for finding adequate specificity was 70 with 21 of these having the disease (prevalence = 30%, power = 0.81). Thus, sensitivity may have been more affected by sample size than specificity in the current study.

When averaged across all items, the RNDA showed moderate positive predictive value (PPV) and high negative predictive value (NPV). The moderate PPV indicates that slightly over 50% of the children identified as having a social-emotional problem by the RNDA were accurately classified. The high NPV suggests the RNDA accurately identified cases as having no concerns approximately 75% of the time. It is generally desirable for a screening measure to exhibit a higher NPV than PPV, as it indicates better accuracy in ruling out individuals with no problems, thus reducing the risk of false negatives and overlooking children who do have social-emotional problems. These findings further support the RNDA’s proficiency in correctly ruling out those without social-emotional problems, aligning with the desired criteria for screening measures.

A strength of this article was its remote First Nations sample. Having data from First Nations communities is important because these communities have often been excluded from validity studies [35]. Such evidence is most often generated within dominant culture samples from populated areas and applied to other regions assuming generalisability. This can inadvertently impose incorrect evidence on these communities, perpetuating health inequities [35]. This sample represents a rare demonstration of validity within this population. This was a critical and purposeful decision given that one of the intended purposes of the RNDA was to be used to improve access to equitable healthcare for these communities. To ensure effective use of the tool for this purpose, validity must be established within these regions. There are challenges inherent in collecting samples from remote regions, and indeed the small sample size which limited the statistical power was a limitation of this sample.

It is worth noting that the RNDA and BASC-3 measures were completed by different respondents, with healthcare practitioners completing the RNDA and caregivers completing the BASC-3. This may explain the moderate correlations observed between constructs assessed on the RNDA and BASC- 3, as using different respondents can introduce additional variance and make it more difficult to detect associations between related constructs. On the other hand, using different respondents helps mitigate common method bias, which can occur when the same source provides measures of both the predictor and criterion [36]. Collecting data from different sources or using varied methods (e.g., self-report and observational measures) is a recognised approach to addressing this type of bias The moderate correlations observed between constructs assessed on the RNDA and BASC-3 are comparable with existing research employing multi-rater approaches [37] highlighting the expected variability when using different respondents.

The flexibility of the RNDA administration allowed Aboriginal Health Workers/Practitioners (AHW/Ps) to ask about RNDA behaviour domain items not observed in the session. AHW/Ps often integrated these items within the context of a ‘yarn’ [38]—a culturally resonant, conversational style that emphasises relationality and understanding. This allowed AHW/Ps to ensure that families felt at ease and were able to provide the most accurate and valid responses. However, this flexibility may have resulted in some variation in how items were phrased across different respondents, which could introduce inconsistencies in the data collected, which may influence scoring. While this approach enhanced the cultural responsiveness of the assessment, it is important to consider the potential implications of this variability when interpreting the results.

Another limitation of this study is the incomplete documentation of the RNDA administrator for some participants, reflecting the challenges of real-world data collection in clinical settings. A further limitation, borne from real-world data collection, was the difference in procedure for administering the BASC-3 depending on whether developmental concerns were flagged by the caregiver or RNDA results, compared to those administered the BASC-3 by phone later. This variation reflects standard clinical practice, where children with identified concerns are prioritised for further assessment and intervention by the healthcare practitioner. However, it may have introduced inconsistencies in follow-up data, such as a longer time lapse between RNDA and BASC-3 data collection, which should be considered when interpreting the findings.

While the RNDA’s accuracy requires further validation with additional and larger numbers of children, this study provides evidence that the use of the single-item measures on the RNDA yield moderate sensitivity and specificity compared to multi-item measures. This seems to be a beneficial trade-off for increased efficiency, utility, and flexibility needed for culturally-responsive administration. The use of valid and reliable screening measures should not replace clinical decision making, rather when considered in the broader context of a holistic child health check, valid single-item test scores can help inform meaningful next steps for families, especially in communities where specialist healthcare is inaccessible. The current study suggests that the single-item social-emotional measures within the RNDA provide an accessible, quick and easy to administer tool suitable for large-scale screening during routine developmental monitoring in primary healthcare settings. When trained to a standard, AHWP/s, frontline healthcare providers, and other non-specialists can use this tool to reduce barriers to accessing specialist care and start a local support journey, contributing to more equitable healthcare for all.