Use of Behavior Imaging Technology to Verify Inter-Rater Reliability in a Multi-Site Pharmaceutical Trial
A multi-site clinical trial, led by Principal Investigator Dr. James McCracken of UCLA, is designed to measure the effects of an investigational product to mitigate social disability Changes in social functioning will be measured through the use of structured assessments that are administered by trained raters.
Use of Behavior Imaging during this study should show “proof of principle” that Behavior Imaging can improve multi-site clinical trial methodology for variety of symptoms associated with various brain disorders.
Background
Study Objectives
Assess the use of Behavior Imaging technology to enhance multi-site inter-rater reliability.
- Provide on-going training for observers
- Introduce observer reliability checks
- Timely intervention in response to inadequate inter-rater reliability
Methods and Procedures
- Experimental design
- Four medical research institutions located throughout the US
- Assessments based on Autism Diagnostic Observation Schedule (ADOS) and the Social Communication Interaction Test (SCIT)
- Observer training based on pre-recorded video patient interviews
- Scoring used the Behavior Imaging on-line platform
- Results were automatically compared to “Gold Standard”
- Discrepancies were identified and recommendations offered to observers
- Observers were then required to repeat training until the “Gold Standard” was achieved
Results (work in progress)
- All independent raters at four sites completed reliability checks using the ADOS and the SCIT
- Patients’ observations were completed and recorded
- All observers (raters) varied how they matched the “Gold Standard”
Behavior imaging used in clinical research for autism pharmaceutical trials shows promise to improve periodic autism assessments, inter-rater reliability and multi-site collaboration.
