Inter-rater Reliability of Diagnosing Thoracic Outlet Syndrome by NHS Physiotherapists Using the CORE-TOS Clinical Diagnostic Criteria
Inter-rater Reliability of Diagnosing Thoracic Outlet Syndrome by NHS Physiotherapists Using the CORE-TOS Clinical Diagnostic Criteria
Thoracic outlet syndrome (TOS) denotes the collection of symptoms which may arise from compression of the neurovascular structures in the region of the brachial plexus. TOS has historically been a clinical enigma, with lack of consensus regarding its diagnosis limiting the validity of any research into it. Literature and local audit both note significant patient morbidity and redundant use of secondary care clinics and investigations in sufferers. The last decade has seen the creation of a consortium of leaders in the field and development of the CORE-TOS diagnostic tool. This tool has 5 subsets of clinical diagnostic criteria (CDC). Positivity in 4 or more suggests a diagnosis of TOS.
The current study seeks to specifically examine whether physiotherapy clinicians - both in primary and secondary care - can consistently identify cases of TOS using the CORE-TOS tool and refer them appropriately to an extended scope physiotherapist (ESP) specialising in the condition. Specific education will be provided to relevant physiotherapists who will be asked to note all relevant CDC in any suspected cases thereafter referred to the ESP in an out-patient physiotherapy department setting. These referrals will follow the standard local pathway to physiotherapy and no clinical testing manoeuvres out with the current scope of physiotherapy will be applied. The patients' case notes will thereafter be retrospectively examined, and the inter-rater reliability of the CDC recorded by the both the referring physiotherapist on their referral and the researcher at initial review. This will analysed using intraclass correlation coefficient, SEM and Bland and Altman's agreement tests, coupled with descriptive analysis.
The principal method of the study will involve retrospective analysis of data prospectively recorded in patients' clinical notes. The data will pertain to clinical information recorded by both the physiotherapist referring the patient, and the principal investigator (P.I.) at their initial assessment of the patient. Patients will be selected from within the specific timeframe of the calendar year between April 2021 and April 2022.
At the first appointment with each rater patients will have undergone subjective and objective clinical examination within the current scope of physiotherapy practice. All relevant clinical CORE-TOS clinical diagnostic criteria (CDC) noted by each on examination will have been recorded, up to a maximum of 5.
Data Collection:
Pending ethical clearance from Queen Margaret University Edinburgh, collation and analysis of the data will take place in April & May 2022 (i.e., one calendar year following the final education session). Within the allocated resources of the study all data collection and analysis will be undertaken by the P.I. only and thus no blinding will be possible. A retrospective review of the P.I.s patient files will be undertaken to collate the relevant patient files. These are hand-written, paper files stored locally within the target physiotherapy department. An Excel® (Microsoft®) spreadsheet will be used to generate a database for the study. This will be stored on the local health service, password-protected server behind the researcher's unique login. Patients will be anonymised on the worklist to a random number using the random number generator function in Excel® and this number written into their case notes, along with the protocol identifier and the date the case note was reviewed. The purpose of this will be twofold. The first of these is for local data protection requirements. The second is to allow for any future consistency checking of the data, given that no blinding of the results will be possible within the study design.
Each CORE-TOS CDC noted by both the referrer (rater 1) and the principal investigator (rater 2) will be recorded on an Excel® spreadsheet. The worksheet will allow for both future quantification of CDC out of a possible 5 and to assist identification of any consistent errors or omissions. For the purposes of descriptive analysis, supplementary data will also be collected all of which will be personally non-identifiable in keeping with ethical and data protection agreements. These will include age, sex, initial quickDASH and SF-12 scores, the length of time symptoms were present prior to review by the researcher, the presence/absence of antecedent trauma, previous imaging and any other clinical sub-specialities seen for this presentation. Given the inability to blind the study, this Excel® file will be stored securely on the local NHS server, behind the researcher's password-protected login for 12 months to aid consistency checking, if required. In addition, it will also be used to inform an intended future study on the outcomes of local treatment for TOS and the optimal pathway for management of it.
Data analysis:
The patients used in this study will be extracted from the population of patients presenting to GP practices, secondary care orthopaedic clinics and outpatient physiotherapy departments with neck and arm discomfort and considered a representative sample of that cohort. The number of CORE-TOS clinical diagnostic criteria (CDC) recorded by both the referring physiotherapists and the researcher at initial assessment will be used for analysis. The possible number of relevant CDC will range from 0 to 5, assume an absolute zero and the absence of numbers of findings below zero (i.e. no negative values). Given this is a continuous scale with equal weight given to each score and equal distance between adjacent scores, these units of measurement will be considered as ratio data.
The primary purpose of the study is to assess the reliability of the CORE-TOS tool. All statistical analysis of the data will be carried out by the primary investigator using a combination of Microsoft® EXCEL® and the IBM® SPSS® software platforms. Analysis of correlation will be applied in the form of an intra class correlation co-efficient (ICC). A consideration in the use of ICCs lies in the fact that numerous models exist and application of the same data to each model can give different results. All ICCs use ANOVA in their calculations and the type of ANOVA model is one of three factors that must be considered in ensuring the appropriate ICC is chosen. The other two are the type of measurement and definition of the relationship considered to be important between measures.
The current study presents a model where the second rater is fixed for all patients, thought the first rater (the referrer) will be different each time. The presentation of a patient with TOS to any of these referring clinicians is random, as is whomever these patients first encounter (i.e., the referring clinician). The referring clinicians are not pre-chosen in this study and will thus be considered a random sample of the population of all potential referrers. In addition, each patient will have their own unique combination of CDC on the CORE-TOS tool. In this regard both the referring physiotherapist and the patient will be considered as sources of random error. Given this and the aim to generalise the results to all potential referrers a two-way random effects model of ANOVA will be used. In considering the type of measures, given that this study ultimately seeks to look at the use of the CORE-TOS in practice as a single measure to initiate a clinical action (i.e., referral to a specialist pathway) a single measure ICC will be used. For the purposes of the study the ability of each referring clinician to clinically assess a patient will be considered equal, irrespective of job title or level of experience. As the study is concerned with the extent to which the score of the referrer and researcher agrees, the definition of absolute agreement will be chosen. This study will use the ICC nomenclature convention of 2 numbers in parentheses and based on the above, the ICC model that will be used is ICC(2,1). This will be formally stated will along with the ANOVA model, type of measurement (in this case single) and definition of relationship in the analysis section of the study along with the ICC and its 95% confidence interval.
It is suggest that for an ICC to be effective, it must be estimated from data having at least 25 degrees of freedom (i.e. the number of measurements times the number of raters). In the case of the current study, the number of CDC per patient will be considered a single measure and thus a minimum of 25 patients will be required. As with all reliability co-efficients, the potential level of reliability will range from 0 to 1. An ICC result in this study of <0.75 will be considered fair to poor, >0.75 considered good, and >0.9 reasonable for clinical measurements .
It is accepted that using the result of an ICC in isolation is insufficient for clinical interpretation, as it gives no indication of the variance both between measurements on the same subject and measurements between subjects. It is universally recommended to supplement ICCs with supplementary tests. To estimate agreement between measurements a standard error of measurement (SEM) will be used. In addition, a Bland and Altman Agreement test (BAAT) will be plotted to graphically present the variance in the data. The raw data will be extracted from EXCEL, which will also be used for the analysis. The SEM will be calculated from the following formula:
SEM = SD (√1-ICC)
where SD equals the sample standard deviation and ICC is the calculated ICC score.
The BAAT will be plotted as follows. The differences between CORE-TOS scores will be calculated for each patient and plotted against the average score between raters per patient. As the BAAT seeks to identify how many of these plotted values fall within 2 standard deviations of the mean difference between scores, a line representing 2 SD above and below the mean will be added to the graph. This will be used to give a visual reference of agreement and to identify any outliers or specific areas of bias. The BAAT will be calculated on the entire sample of referrers irrespective of their type of job (i.e. ESP vs GP APP) or experience and this factor taken into consideration in the final discussion. In addition Bland and Altman plots will also be produced to subdivide the analysis into referring physiotherapy specialty to analyse for any specialty error or bias.
Descriptive analysis will be applied to the demographics of the patients in the study. These will be tabulated and include age, sex, presence/absence of trauma at onset, previous treatments and initial outcome measure scores of disability and quality of life (quickDASH and SF-12). The purpose of this is twofold. The first of these is to allow extrapolation of the study sample to the known literature regarding thoracic outlet syndrome (i.e., if the demographics significantly differ, the sample may not be representative). Secondly, the demographics and outcome measure scores will provide some measure of the local incidence and morbidity associated with TOS. These elements will inform both an intended future study on the outcomes of the researcher's physiotherapy intervention for this cohort and a business case to develop more optimal clinical pathways for TOS in the local health board.
Validity of the CORE-TOS Tool In addition to the testing of reliability and agreement of the number of recorded measures from the CORE-TOS tool, further descriptive statistics will be applied to evaluate the face validity of the tool itself in local practice. Within this a number of assumptions will be made. By the nature of the study design, all patients referred to the researcher will be considered by the referrer to have at least 4/5 positive CDC and thus be considered by them to be positive for TOS. For the purposes of the study, the researcher is deemed an expert in the condition, against whose opinion a direct analysis is made regarding diagnosis and (ultimately) appropriate treatment (i.e., they ultimately decide the initial course of treatment/management based on their diagnosis). The raw data will also therefore be analysed for the number of true positives versus false positives considered by the researcher. Analysis of false negatives is outwith the design of the study. Combined with analysis for any consistent errors, inconsistencies or omissions it is hoped this may provide a measure of the sensitivity of the tool in local practice.
The data analysis, will be formally recorded and written up in concise and journalistic style as part of the XMO29 module of the Master of Research degree at QMU in accordance with their local guidance.
Inclusion Criteria:
Exclusion Criteria:
95278262@qmu.ac.uk07958584097