The Opinions of Multiple Stakeholders Towards Gerontechnology Evaluation Framework: Four Studies Using Delphi Techniques
The Opinions of Multiple Stakeholders Towards Gerontechnology Evaluation Framework: Four Studies Using Delphi Techniques
The 4 studies aim to collect and examine the opinions from multiple stakeholders of gerontechnology product including the demand side, funder, supply side and academia on Gerontechnology Evaluation Framework. The Evaluation Framework will describe the essential knowledge to conduct an evaluation study for 10 selected gerontechnology product themes in local service settings. Specifically, we will (1) develop the indicators to determine if a companion robot, which is one of the product themes, is 'good'; (2) describe feasible study designs, implementation strategies and data analyses to generate evidenced-based evaluation results of companion robots; (3) outline strategies to build organizational research capacity to facilitate the evaluation of gerontechnology products.
In the Preparation Stage of all 4 studies, eligible participants will be recruited under purposive sampling and snowball sampling. Literature review, qualitative interviews and reflection on recent experience of evaluating gerontechnology products will be conducted. The opinions collected from the stakeholders will facilitate questionnaire development for the Delphi Stage. In the Delphi Stage of each of the 4 studies, there will be two rounds of quantitative validation of the grouped statements to generate consensus. Based on the opinions with consensus, the evaluation framework will be revised.
Study design For research question (1), two studies, i.e. Study A1 and A2, will be conducted. Study A1 will conduct secondary data analyses on qualitative interviews and a two-round questionnaire using Delphi techniques with end users. Study A2 will conduct focus groups and a two-round questionnaire using Delphi techniques with decision makers. For research question (2), Study B will employ a focus group and a real-time two-round questionnaire using Delphi techniques with decision makers. For research question (3), Study C will employ a real-time two-round questionnaire using Delphi techniques study with decision makers.
For all 4 studies, in the first Delphi round, panellists will rate anonymously the agreement of the opinions on a 5-point Likert scale, give reasons for their ratings and suggest revisions. They will then be presented with the overall analyses and their ratings in the first round. In the second Delphi round, they will rate anonymously the agreement of the revisions, and give reasons for their ratings.
Procedures Preparation Stage In Study A1, for qualitative interviews with product users, we will do a secondary data analysis on individual qualitative interviews with 10 end users and 6 demand side - frontline. All of them were recruited from a residential care home and previously participated in an intervention study using companion robot. Their answers to questions related to 'good' companion robots will be analyzed by framework analysis. The analyzed opinions, upon the approval of the investigators, will be listed as the questionnaire items for use in Delphi part. Additional questionnaire items for the Delphi part will be added based on the literature that explored the opinions of the product users towards companion robots. For the Delphi part, 20 - 60 eligible visitors of Gerontech and Innovation Expo, a local exhibition that provides a platform for stakeholders to discuss and collaborate for broader adoption of gerontechnology, will be recruited by convenience sampling at the time of Expo registration. Their voluntary written consent will be sought before they complete the Delphi questionnaire. They will provide their email address and contact number, so that they could be contacted to do second Delphi round.
In Study A2, for focus groups with decision makers, 5 - 20 decision makers who have joined the development, evaluation or procurement process about companion robots will be purposively recruited by investigators of the project through phone calls and emails. In particular, demand side - managerial personnel will be recruited from the organizations that have adopted companion robots in their service units. Academia will be recruited by purposive sampling on the investigators who have publication(s) on gerontechnology in online databases of PubMed, Web of Science, Cochrane, CINAHL, EMBASE, and by convenient sampling on the nursing educators in the School of Nursing, the University of Hong Kong (HKU) will be conveniently recruited. Funders to be recruited are the representatives nominated Innovation and Technology Fund (that subsidizes social service units in renting and procuring gerontechnology products including companion robots) and. Supply side who expressed willingness to the Hong Kong Council of Social Service on providing companion robot(s) for trial test will be recruited. Similar to Study A1, their opinions will be analyzed by framework analysis. The analyzed opinions, upon the approval of the investigators, will be listed as the questionnaire items for use in Delphi part. For the Delphi part, 15 - 40 eligible visitors of Gerontech and Innovation Expo will be recruited following the same recruitment mechanism as the Delphi part in Study A1.
In Study B, the same focus groups participants who join the focus groups in Study A2 are simultaneously recruited. Their opinions in the focus groups will be analyzed by framework analysis. The analyzed opinions, upon the approval of the investigators, will be listed as the questionnaire items for use in Delphi part. For real-time Delphi questionnaires, in addition to the above decision makers, snowball sampling will be adopted to recruit the partners of Hong Kong Council of Social Service. These partners agreed to join discussions on gerontechnology, and they are eligible to join the Delphi part no matter they have or do not have experience in the development, evaluation or procurement process about companion robots. In total 8 - 15 panelists will be recruited.
In Study C, the questionnaire used in the Delphi stage will contain a list of strategies in enhancing the organizational research capacity, which are consolidated from literature review and investigators' past experience in doing similar research. In the Delphi Stage, the participants will be recruited in the same way as in Study B. In total 8 - 15 panelists will be recruited.
Delphi Stage For all 4 studies, in the first Delphi round, panellists will rate anonymously the agreement of the opinions on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), explain their ratings of 3 or less and suggest revisions (including additions of new items). Upon the approval of the investigators, items will be revised according to the suggestions. Items in the first Delphi round that reach moderate or low consensus are revised, and rated in the second Delphi round together with the newly suggested items. On the contrary, for items that reach very high consensus or high consensus, if no revisions are suggested, they will not be rated again. Before the second Delphi round, the panellists will be presented with the consensus level analyses, their own ratings in the first round, and proposed revisions based on the comment on the analysis. Items with more than 75% approval of the revised items will be included in the second round Delphi. In this round, more panellists will be recruited to rate anonymously the agreement of the revised items.
For Study A1 and A2, the second Delphi round is expected to take place two weeks after the first Delphi round. For Study B and C that adopt real-time Delphi in a workshop, the investigators will first present the materials consolidated in the Preparation Stage, and panellists will immediately indicate the agreement of the materials in the first Delphi round. This is followed by an open discussion of first round Delphi analysis results among the panellists for around an hour. For diverse opinions on the same issue, different scenarios favouring the opinions will be explored, and the opinions together with their favouring scenarios will be listed in the second Delphi round. When the opinions are consolidated, second Delphi round will then take place. The panellists will rate their agreement on the suggested revisions.
The inclusion criteria for product user (i.e. end user and demand side - frontline).
End user
Demand side - frontline
The inclusion criteria for decision maker (i.e. demand side - manager, academia, funder, and supply side):
Demand side - manager
Academia
Exclusion criteria for all:
• Unable to communicate in Chinese