Testing a Training Program That Uses Virtual Reality Technology to Improve Children's Pedestrian Behaviors: A Randomized Controlled Trial
Testing a Training Program That Uses Virtual Reality Technology to Improve Children's Pedestrian Behaviors: A Randomized Controlled Trial
Motor vehicle pedestrian injury is a critical issue for school children.1-4 Each year in the US, over 4900 pedestrians are killed and another 207,000 are injured, and about 25% of these pedestrian events involve school-age children. This research focuses on 7-8 year olds, who constitute a high-risk group for pedestrian injury. At these ages children regularly cross streets without supervision and they struggle both with selecting where to cross and determining how to cross. Research has shown, however, that children are capable of benefiting from effective behavioral training in pedestrian behavior. The proposed research addresses the issue of crossing skills deficits and will: (1) implement a randomized controlled trial (RCT) to test two alternative training programs to teach 7-8 year-olds where and how to cross streets safely; and (2) conduct an economic analysis to reveal cost:benefit indices for both.
Meta-analyses of pedestrian training programs reveal that behavioral training in a traffic environment most reliably produces some degree of improvement in crossing skills. Thus, 'street-side training' is often described as the gold standard. Implementation, however, poses many practical problems related to implementation. The investigators have addressed this issue by developing a training system that uses a virtual pedestrian environment and extends past VR systems by having children fully cross the street and offering the unique capability of teaching both where and how to cross, with skills in each domain measured separately so exactly what is learned and what component crossing behaviors improved can be precisely determined for each individual child.
Children (7-8 years) will be randomized to one of three groups (balanced for sex): street-side training, virtual-reality training, and a no-intervention control, with the same pre- and post- measures taken across groups. Primary analyses will test for changes in indices of where and how to cross, as well as attention to traffic when crossing. An economic analysis of the two programs will reveal their relative cost effectiveness. These results will provide essential knowledge to inform future decisions about 'best practices' in child pedestrian injury prevention through behavioral training.
APPROACH
RESEARCH GOALS. The specific goals are: (1) conduct a RCT with 7-8 year olds to evaluate the relative effectiveness of 2 training programs (Virtual Reality, Street Side), compared to a no-intervention control, with a focus in training on where and how to cross. (2) Conduct an economic analysis of the programs to ascertain their relative cost effectiveness. These results will provide essential knowledge on relative program effectiveness and economic costs to inform future decisions about 'best practices' in child pedestrian injury prevention through behavioral training
STUDY DESIGN: 3 Groups Recruited children will receive the pre-tests and then be randomly assigned to one of three groups (2 Intervention, 1 Control), using a block (sex) randomization procedure so Intervention groups (VR training or VR-T, Street Side training or SS-T) and the no intervention Control group (No Intervention: NI-C) are balanced for sex of the child.
RESEARCH QUESTIONS Does performance in VR-T exceed that in NI-C? Does performance in SS-T exceed that in NI-C? Is performance equivalent in VR-T vs SS-T or does one produce better outcomes and qualify as best practice? Do any immediate gains persist over time (2 months post-intervention)? What is the cost-effectiveness of VR-T compared to SS-T?
POWER ANALYSIS All calculations were performed by a consulting statistician (Dr. Gerarda Darlington) using a function written in R version 3.3.2 based on a formula provided by Julious and using the pwr.anova.test function from the pwr package in R. Results from the pilot feasibility study were used to estimate mean changes for where and how to cross: after VR-T, children crossed in unsafe locations 65% fewer times and there was a 68% increase in how to cross safely, with both variables being normally distributed. With a within-group standard deviation of 0.16 (pilot data) and a sample size of 58 children per group, we would have over 80% power to confirm equivalence between the VR-T and SS-T groups, assuming no observed difference in change scores between the groups and an equivalence threshold level of 17%, with alpha=0.025. Note that the total sample size would support the analyses planned, allowing for significance at p < 0.0001 with power greater than 95%. Allowing for 10% attrition, we will recruit 64/group (N = 192).
SUBJECT POPULATION AND ELIGIBILITY CRITERIA [*see Introduction to Application please] A total of 192 children 7-8 years will be recruited, with 64 assigned to each group; this includes allowance for 10% attrition (usual rate=5%). Youth will be randomly sampled from an existing database of 9250 community recruited volunteers, with supplemental recruiting (as needed) through community organizations and advertising. Use of the database increases the speed and efficiency with which we can execute the research but does not change the base population-it is all a community recruited sample. Inclusion Criteria: developing normally (parent report; no learning, social, physical, mental health concerns), English speaking. Exclusion Criteria: self or anyone in family hospitalized for injury as a pedestrian, which can lead to atypical cautiousness.
RANDOMIZATION Blinding. Blinding to condition is not possible but greater bias may be introduced in RCTs through inadequate allocation concealment than with blinding. All participants will be blind to the study hypotheses.
Allocation of participants will occur by via a computer-generated random number sequence.
TIMING OF EVENTS AND MEASURES All children do an initial visit to complete the pre-test measures and are then randomized. Then SS-T and VR-T children complete 6 weekly training sessions (~30 minutes/session). All children then complete another visit for post-test immediate measures, with a longer-term visit (2 months later) to evaluate sustainability of any improvements.
SETTING Midblock crossing on two-lane bidirectional roads, without nearby traffic signals or crosswalks, are the focus.
For SS-T sessions, a road has been identified (50 km limit) that provides opportunity for training in where and how to cross in the same types of road situations to be presented in VR-T. Traffic volume was recently measured for 7 days and varied from light (8 cars/min) to heavy (16 cars/min). This range for traffic volume will occur in VR-T.
For VR-T sessions, these occur in the VR lab at UG campus. For pre and post measures, for all groups these occur in two settings: (a) street side (designated as "Field" below) with naturally varying traffic (children indicate where and how they will cross but do not do so), (b) in the VR lab in which children cross the virtual street in traffic using a gaming controller.
Note that the pre/post Field measures will be taken at two locations along the same road (randomized order, with one used for pre and the other for post), which controls for familiarity and minimizes risk of large variation in traffic volume between pre/post measures. The road is nearby the VR lab, which allows children to complete pre-test VR and Field measures during the same visit to UG; the same will occur for post-test measures. Posted speed is 50 km, which means traffic conditions to be tested on are ones that pose risk of severe injury at these ages.
TESTING PROTOCOL Pre/Post measures for Where and How to Cross [Note that pre- and post- measures are the same, and all children, regardless of group, complete all measures. ***Reliability for coding street side crossing will be completed.] Field Measures. For where to cross measures, the child selects the safest location to cross in 3 different street side situations (parked cars, hill, blind curve). For how to cross measures: the child stands 1 foot from the curb, with a pressure sensitive step plate in front of him (it connects to a laptop; a clear plastic wall prevents entry into the street), and completes a '2 step procedure' (based on observed traffic, take 2 steps forward, corresponding to initiating a street crossing; from this we estimate crossing measures.[NOTE: Street side testing is video recorded so traffic and crossing measures and reliability can be computed later.] VR Measures. The same measures as for Field, except completed within a virtual street environment.
Pre-Training Activities & Measures VR-T. An initial Orientation and Movement-Control Training Module ensures all children are practiced with using the controller to navigate in the 3-D world to the same motor skill level, before starting pedestrian training. The controller allows them to vary their path, direction, and speed of walking and crossing.
SS-T. Children's average walking speed for crossing is determined and then, in combination with video taken street side, is used to estimate crossing measures - so they do not cross in traffic, but only give initiation judgements. This is a common approach that has been used before.
Training Programs [All program decisions were informed by pilot testing.] Each includes 6 training sessions of 30 minutes (2% of our pilot sample needed a break) and addresses where and how to look and cross. Performance is tracked on a trial-by-trial basis, and errors result in repetition of teaching trials, resulting in more training on skills they lack than those they have. Difficulty level generally increases over sessions as children acquire skills.
VR-T. Children experience trials from each of three training modules that are organized logically for how one crosses: (1) deciding where to cross (component skills: identify a safe place with parked cars, traffic on hills, traffic on blind curves); (2) effective looking (component skills: left-right-left, always left as one enters the road); (3) how to cross safely (component skills: check traffic in both directions, select larger gaps, start right in when car back bumper passes). On each trial, the computer talks and provides immediate feedback on performance so children know if they safely completed the trial (e.g., "Great job") or what they did that was unsafe and why (replays of what the child did are shown and explained), and what to do instead. A component skill is tested (with explanatory feedback given for failures) until the child achieves at least 80% success over a minimum of 5 trials or completes 10 trials. The child then moves on to test another skill, with failed component skills retested in a future session. Over sessions, the child gets repeated training on all modules and skills, but repetition focuses mostly on skills the child still needs to achieve.
SS-T. The Kerbcraft program is grounded in behavior theory (modeling, reinforcement). The program includes where and how content and is organized similarly to VR-T (modules, teach component skills, immediate feedback). Over sessions, children get training on all modules and skills, with repetition focusing on weaker skills. Testers (RA, students) will follow a detailed manual that outlines program delivery.
PRIMARY OUTCOME MEASURES Behavior Measures. [All have been used in prior research.] Pre and Post measures are taken in two settings (VR lab and Field street side). In each setting, children complete 15 where trials (they indicate where to cross, but do not do so) and 15 how trials (they decide about crossing in traffic).
Where. One score: average proportion of 15 trials the child chose the safe option for when crossing on a hill (5 trials), with blind curves (5 trials), with parked cars (5 trials). These 3 component scores will also be analysed separately (see Secondary Analyses).
How. Three scores: (1) Attention composite (proportion of 15 trials they looked correctly: L/R/L, always L just before they step into road for VR-T or would have for SS-T); (2) Start Delay (time in seconds between rear bumper of car passing and beginning of starting across); (3) Inter-vehicle Temporal Gap Size selected (in sec).
Consequence. Two scores (from how trials): (1) average Time Left to Spare (i.e., how close the car came in seconds); (2) composite of proportion of Near Miss (car passed within 1 sec of child) and Hit trials.
Economic Measures The key outcome is hits, as this would result in economic burden of injury. First, the actual cost of implementing the two training programs (personnel, equipment, etc.) will be determined. Then a decision analytic model will be developed to compare each program to the no-intervention group. The model will take a societal perspective, including both government payer (direct hospital and non-hospital costs: ER visits, community physician visits, etc.) and individual/ family out of pocket costs (e.g., out of hospital drugs, home care services, lost productivity). A lifetime time horizon will be used, noting that some injuries are severe and will have a long-term impact on morbidity and quality of life. Probabilistic sensitivity analysis will assess the robustness of the model. Cost data will be captured from relevant administrative databases; where data are missing, relevant literature and/or expert opinion will be used to estimate costs.
Inclusion Criteria:
Exclusion Criteria: