TIPS for Scaling up Research in Upper Limb Prosthetics

Abstract

Many research initiatives have been employed in upper limb prosthetics (ULP) in the last few decades. The body of knowledge is growing and inspired by new and interesting technology that has been brought to the market to facilitate functioning of people with upper limb defects. However, a lot of research initiatives do not reach the target population. Several reasons can be identified as to why research does not move beyond the lab, such as lack of research quality, disappointing results of new initiatives, lack of funding to further develop promising initiatives, and poor implementation or dissemination of results. In this paper, we will appraise the current status of the research in ULP. Furthermore, we will try to provide food for thought to scale up research in ULP, focusing on (1) translation of research findings, (2) the quality of innovations in the light of evidence-based medicine and evidence-based practice, (3) patient involvement, and (4) spreading of research findings by focusing on implementation and dissemination of results and collaboration in a national and international perspective. With this paper, we aim to open the discussion on scaling up research in the community of professionals working in the field of ULP.

1. Introduction

Major challenges exist in upper limb prosthetic (ULP) research today, such as developing new technology, obtaining funding for research proposals, improving quality of research activities, and spreading the results in order to benefit the target population. In the field of ULP research, many efforts are put into the development of new prosthesis components, new control strategies, improving the use of prostheses, and developing measurement instruments, all to overcome consequences of limitations caused by the loss of an upper limb. Although in recent decades many innovations have become available to the prosthesis user, such as pattern recognition control, targeted muscle reinnervation, osseointegration, and multi-grip hands, it is generally acknowledged that many research findings do not at all or only in a limited way move beyond the lab and reach the target population. A frequently quoted statement about the lapse between research and medical practice indicates that it takes 17 years to turn 14% of original research to the benefit of patient care [1]. As such, only a small proportion of research activities and related funding reaches the target population. Of the numerous and diverse reasons for this, lack of quality of the innovation, lack of funding to continue the development of promising initiatives, or limited attention for implementation or dissemination most likely stand out in the field of prosthetics. Furthermore, industry has a main role in the development of innovative prosthetic devices on a large scale. The high demands on industry to minimize costs and to maximize profits put pressure on innovative initiatives, which may explain why the industrial scale up of innovative designs is limited. It can be questioned how we can scale up our research to move beyond the lab and to serve the target population better with our research results. The latest TIPS-ISPO-BACPAR Symposium on ULP research (Salford, UK, 20–23 March 2019) was dedicated to the theme “Moving beyond the lab”. This paper is a reflection of a keynote presentation fitting within this symposium theme. The findings and opinions presented in this paper are not unique to ULP, but may also be applicable to other fields of medical device development and usage.

2. Scaling Up

Scaling up can be defined as “deliberate efforts to increase the impact of health service innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis” [2]. The WHO published three reports on scaling-up strategies [2,3,4], which were prepared on the understanding that scaling up rarely occurs automatically, as is often assumed. Scaling up needs focused attention, strategic planning and management, as well as resource allocation [4]. Thus, if research in ULP should move beyond the lab and the impact of successfully tested innovations should benefit the person with an upper limb defect, specific attention should be paid to scaling up this field of research. We summarized part of WHO’s recommendations on how to enhance the attributes of success and the potential for sustainable scaling up in four topics of interest: Translation, Innovation, Patients, and Spreading, composing the acronym TIPS. In this paper these TIPS will be introduced, discussed, and illustrated in light of scaling up research in ULP (

Table 1. Recommendations for sustainable scaling up and topics of interest.

Topics of Interest: TIPS	Recommendations for Sustainable Scaling Up (WHO) [2]
Translation	- Test the innovation under real-life operating conditions
Innovation	- Initiate scaling up after the effectiveness and feasibility of the innovation have been established
Patients	- Involve the user organization in a participatory success - Reflect on the degree and nature of change that the innovation implies for the user organization
Spreading	- Identify the key features central to success so the innovation can be streamlined and more readily replicated during scaling up

).

2.1. Translation

Translation of research findings from the lab to the target population encompasses the testing of the innovation under real-life operating conditions. To do so, we should take into account translational research, which is defined as the process of applying laboratory research to human studies and enhancing the adoption of evidence-based practices in real-world settings to reach broad populations [5]. Translational research can be divided into three stages: activities to establish the effectiveness of care (stage 1), activities to assess who benefits from promising care (stage 2), and activities to determine how to deliver high-quality care reliably and in all relevant settings, mostly home try-outs (stage 3) [6]. Looking at the body of research performed in ULP, most research seems to be performed in Translation stages 1 and 2, but see [7,8] for examining prosthesis use after a period of home use (i.e., stage 3). Examples are the many research initiatives to develop multi-grip prosthesis hands or to develop pattern recognition systems to strive for more intuitive control in myoelectric prosthesis users. Only a limited number of authors report on home try-outs [9,10]. Only a limited number of development initiatives reach the market. To date, only four multi-grip hands and two pattern recognition systems are commercially available in Europe. To implement research findings in clinical practice or in the patient’s home situation, we should shift more research activities to Translation stage 3 (Box 1). The digital world we currently live in enables us to design research that can be executed in the home situation. Mobile healthcare technologies (telerehabilitation) enable patients to get their research interventions anytime and anywhere. This will probably create significantly higher patient engagement. Additionally, smart technology-enabled systems can monitor each patient’s progress in great detail [5]. Smart technology, such as activity monitoring using sensors applied to upper limb prostheses or their users, can provide knowledge on prosthesis wearing patterns, usage of prosthesis components, or compensatory movements. As extensive home trials with large sample sizes might not be feasible, our research field should put less emphasis on traditional research designs, such as randomized controlled trials (RCTs) or case control studies, and should value more alternative study designs, such as qualitative designs, pragmatic clinical trials, participatory action research, or small-N methodology, such as single case experimental designs, small sample, or interrupted time series [5,11,12]. The latter methodologies are characterized by single persons or small groups, repeated outcome measurements, and sequential application or withdrawal of interventions. If carried out properly, the level of evidence can be as high as an RCT. As N-of-1 studies might look like case studies, scientific journals seem to be reluctant to publish such research results. We therefor would like to call for scientific journals to stimulate accepting papers exploiting small-N methodology.

Box 1. TIPS on Translation of research findings.

-

Focus more research on translational research stage 3

-

Consider home try-outs to be part of research proposals

-

Make use of the advantages of smart technology and telerehabilitation

-

Consider alternative study designs, such as small-N methodology

-

Scientific journals should accept and stimulate the submission of small-N papers

2.2. Innovation

Scaling up should be initiated after the effectiveness and feasibility of the innovation have been established (Table 1) [2]. Effectiveness and feasibility studies can only be evaluated if the principles of evidence-based medicine (EBM) and evidence-based practice (EBP) are considered. Therefore, we will here explain and summarize the current status of EBM and EPB in ULP research. EBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients [13]. EBP is the integration of external scientific evidence, clinical expertise, and patient perspectives to provide high-quality services [14]. Both EBM and EBP are interlinked.

To apply EBM/EBP the 5-A’s procedure can be used: Ask, Access, Appraise, Apply, and Assess [15]. The first step, Ask, starts with formulating a PICO(T): Patient or problem, Intervention or exposure, Comparison intervention or exposure, Outcome of interest, and Time or duration of data collection. Next, diverse databases containing citations, scientific journals, full texts, or abstracts (PubMed, PsycINFO, SCOPUS, Cochrane, CINAHL, etc.) can be Accessed to find relevant articles to answer the formulated PICO(T). Subsequently, all articles should be Appraised regarding their design and quality. The strength of evidence of commonly used research designs is reflected in the pyramid of evidence (Figure 1). Articles revealing data at the study level, such as evidence-based guidelines, systematic reviews, or meta-analyses, are in the top of the pyramid, and articles reflecting data at the subject level are in the bottom of the pyramid. Studies at the subject level with high internal validity, such as RCTs, are categorized higher in the pyramid than studies with low internal validity, such as case reports. Various websites can be helpful to find suitable tools to appraise the quality of each individual article [16,17,18,19,20]. The fourth step is to Apply the findings to clinical practice by deciding whether the findings are significant, generalizable, relevant, important, and feasible to implement in clinical practice. If the findings lead to the implementation of an innovation in clinical practice, the final step, Assess, should be applied a while after the implementation.

To assess the level of EBM/EBP in the field of ULP, we searched PubMed for articles using the topics “Upper limb prosthetics” and “Rehabilitation” published in the last 20 years (1999–2018). The search string was composed with the help of a specialized librarian (see Appendix A). A total of 936 articles were found, mainly published in the last five years (Figure 2). To determine the type of study designs used, we searched these 936 articles using MESH terms, title/abstract, and keywords. It appeared that 0 guidelines, 9 systematic reviews, 111 narrative reviews, 11 RCT’s (of which 8 reported only on able-bodied participants), 16 clinical controlled trials (of which 11 reported only on able-bodied participants), 134 cohort studies, 105 case reports, and 1 expert opinion were published. The type of study of the remaining articles could not be identified with our search procedure. From this search, we concluded that only a few studies have a position in the top of the pyramid of evidence and many studies form the base of this pyramid.

If we compose a pyramid of evidence using the absolute number of articles found in each category of the pyramid (from evidence-based guidelines in the top to expert opinions in the bottom), the pyramid shape takes a different form, and is best reflected by the shape of a Mexican hat, a so-called sombrero (Figure 3, left and middle panels).

As the shape of a sombrero is reflecting a body of research of primarily low quality, we may argue that ULP research should strive for a reversed pyramid, reflecting research with a mostly high level of evidence output (Figure 3, right panel). However, we should realize that in the development of treatments, diagnostics, or other instruments, research needs to go through a series of phases, and the optimal study type for one phase may be inadequate for the next [21]. An illustrative example is the development of a treatment to diminish phantom limb pain: first a case study was published, followed by a cohort study, and currently a multicenter randomized clinical trial is executed [22,23,24]. As a sombrero or a reversed pyramid both do not seem to be optimal, we suggest that the community of ULP researchers should strive for a tall hat shape of their body of research: a tall hat provides a sound base of case studies, cohort studies, and controlled clinical studies, and has a proportionally in-shape amount of high level of evidence publications, such as RCT’s, systematic reviews, or evidence-based guidelines (Figure 4).

The next question that emerges in the light of scaling up research in ULP is how ULP research can move from a sombrero to a tall hat. In order to answer this question, we will first have a more detailed look into the current status of the available literature. Subsequently, we will provide recommendations for scaling up.

Evidence based guidelines, systematic reviews/meta-analyses. Unfortunately, there is no single database hosting evidence-based guidelines on ULP topics. It is therefore hard to identify and access relevant guidelines. In our search to find guidelines about ULP, we only found the VA/DoD Clinical Practice Guideline “the management of upper extremity amputation rehabilitation” [25]. Most recommendations of this guideline are based on expert opinions and not on publications with a high level of evidence. More than 10 years ago, we published a systematic review on the relationship between the fitting of a first prosthesis in children with congenital upper limb deficiency and rejection rates [26]. Back then, we concluded that the clinical practice of the introduction of a prosthesis is guided by clinical expertise rather than by evidence-based medicine. In the limited number of systematic reviews found, we appeared to not be the only authors concluding that evidence is mostly not available and firm conclusions cannot be drawn [27]. Although we appreciate the huge amount of work put into composing guidelines and systematic reviews and we would encourage to raise the numbers, their value will only increase if the level of evidence of the included studies increases, since their content is as strong as the weakest link, namely the article with the worst quality on which they are based. When collecting results from other studies, it would improve the quality of evidence-based guidelines, systematic reviews, and meta-analyses if standardized protocols (such as standardized activity measures, usage of a core set of questionnaires, specific electrode locations, etc.) were used. Further, we recommend that a database for evidence-based guidelines and systematic reviews specifically dedicated to ULP research be established (see Box 2). An organization like the International Society for Prosthetics and Orthotics International (ISPO-Int) could host such a database.

Randomized Controlled Trials. RCTs are judged to be the gold standard when practicing research, which is explained by the high internal validity (low risk of bias) of RCTs. However, some disadvantages also have to be acknowledged, since the external validity and the generalization to the target population can be low. Furthermore, costs are mostly high. In ULP research, only a limited number of RTCs can be found. This can be explained by the difficulties encountered when executing an RCT: double or even single blinding is often not possible, undoing of results can be hard or impossible, especially in training or learning studies, and most studies have to deal with small sample sizes with large subject variation. As a result, most RCTs are performed on able-bodied participants using prosthesis simulators. We have to admit that we also contributed to this practice [28,29,30,31,32]. We therefore recommend to always validate results obtained in able-bodied persons with a small sample of persons with an upper limb defect, which also applies to other study designs, such as clinical controlled trials or cohort studies [33,34,35].

For improving the quality of RCTs, CCTs, cohort, and case studies, we recommend using reporting guidelines (see Box 2), such as provided [17]. Furthermore, we would encourage publishing more theoretical foundations regarding developments in ULP research. If grounded on theories, innovative treatments will be developed in a more thorough way, which will contribute to higher quality publications. Examples of theories that might be useful are the treatment theory or the enablement theory [36,37]). Moreover, Bongers et al. and van Dijk et al. applied insights from motor control to ULP [38,39]. We hope that one of the results of the current paper is that authors feel more encouraged to submit such theoretical papers. Finally, we would like to break a lance for the education of researchers and clinicians. If researchers and clinicians are more aware of the prerequisites of high-quality research, the results will improve. With clinicians, we not only refer to doctors or therapists, but also to prosthetists and technicians. We would therefore recommend research institutes, hospitals, and orthopedic workshops to educate their staff on the basics of good clinical practice. Such courses can be followed easily by using online facilities.

Box 2. TIPS on improving the body of research on ULP Innovations.

-

Strive for a shape of a tall hat instead of a sombrero for the body of ULP research

-

Compose more evidence-based guidelines and systematic reviews

-

Develop standardized protocols, including a core set of questionnaires, to improve the quality of evidence-based guidelines, systematic reviews, and meta-analyses

-

Establish a dedicated database to host guidelines and systematic reviews on ULP topics

-

Rely much less on research on able-bodied persons

-

In all studies, always include at least some persons with ULD

-

Use established reporting guidelines when reporting study results

-

Base design innovations on theories

-

Scientific journals should encourage authors to submit theoretical papers

-

Provide good clinical practice education to all clinical staff

2.3. Patients

Patient perspectives are different from the perspectives of professionals, but in many ULP research papers, it appears that such perspectives are largely neglected [40]. New versions of upper limb prosthesis components are brought onto the market without proper evaluation by the users. For instance, recent introduction of pattern recognition systems has not been preceded by clinical studies on their effectiveness [41,42]. As a result of this practice, several innovations that were not or only appreciated in a limited way by users were made commercially available. Research funding spent on such innovations might largely be regarded as wasted. Patients should therefore contribute to research from the start of each project, and their feedback should be valued throughout the course of the project. Such patient involvement will contribute to greater generalization of study results and will improve implementation of innovations. We would therefore recommend putting more emphasis on patient perspectives in each research project, for example, by considering qualitative research designs, such as focus groups, Delphi rounds, or personal interviews (Box 3) [43].

Patients can be reached not only via rehabilitation centers or orthopedic workshops, but also through amputee associations, such as the International Amputee Association [44]. By writing contributions for the journals or websites of the patient associations, researchers can make themselves known to patients and disseminate the results of research to the ULD population. To our experience, this is also an efficient way to recruit patients for participation in research activities.

Box 3. TIPS on Patient involvement.

-

Include patient perspectives in each research project

-

Involve patients in each research project from the start of the project

-

Evaluations should be based on opinions of prosthetic users

-

Consider qualitative study designs to elucidate patient perspectives

-

Have an active attitude in your contacts with patient associations

2.4. Spreading of Research Findings: Implementation, Dissemination, and Collaboration

Implementation. If an innovation has proven its effectiveness, the implementation of the intervention or treatment should be a logical next step. However, implementation has not gained much attention in our research field. As a consequence, many good initiatives have not reached clinical practice. When designing a research project, most researchers start with formulating an idea and then seek funding, and if funding is obtained, the project is executed, results are analyzed, and a publication is prepared. It is mostly at the latter stage that implementation of research results is considered. However, for implementation to be successful, an implementation plan should be part of the research proposal. Prosthesis users should be involved in order to get a more complete understanding of how novel technologies could address their concerns and how these should be implemented in their daily lives (Box 4) [45].

Our own implementation experiences have been described after execution of the PPP-Arm (Prosthesis Prescription Protocol of the Arm) project [46]. PPP-Arm is a protocol used by all Dutch rehabilitation teams when prescribing an upper limb prosthesis. Success factors regarding the implementation of this national protocol were the involvement of all stakeholders from the beginning of the project; the presence of a dedicated project coordinator; division of the implementation into two phases (where large centers did the first try-outs and smaller centers participated after initial problems were solved); the application of knowledge brokers (mostly occupational therapists), who were assigned to be the main persons responsible to implement the protocol at the work floor; arrangement of financial compensation for the knowledge brokers; regular meetings with knowledge brokers; and dissemination of regular newsletters to keep everybody updated.

Dissemination. Research in ULP is characterized by being applicable to a small population on the one hand and costly on the other hand. These two characteristics imply that the interest of potential research participants or funding agencies has to be drawn in an active way in order to scale up research in ULP. One way to draw the attention of participants or funding agencies is to focus on interesting research results and to use the sexiness of the advanced technological products to attract the media (Box 4). Media can be various types of social media or traditional media, such as radio or television. The power of media should not be underestimated. Dutch TV broadcasted an item about our participation in the international Phantom Motor Execution trial [24]. As a result, participants from all over the country sent us requests for participation in the study. Instead of putting a lot of effort in recruiting participants, which is mostly the case in research, we had to create a waitlist. Research in ULP has a large amount of feel-good content, and we should take more advantage of this circumstance. We therefore would advise paying more attention to public relations and make a dissemination plan at the start of the project.

Collaboration. Collaboration is of utmost importance for scaling up research in ULP. A team approach is necessary, since we only can scale up if we collaborate. Important elements for successful collaborations are having a shared mission, having an open mind and trust for each other, feeling responsibility for your own part of the project and for the project part of the partners, and having a team with members who are complementary in skills. Each research team needs different types of expertise, such as engineers, clinicians, human movement scientists, research nurses, implementation experts, valorization experts, public relations officers, and persons representing the target population, the prosthesis users. We have been working on ULP projects for many years now, and we have collaborated with several other research groups worldwide. The majority of these collaborations were fruitful and inspirational, but sometimes we faced some challenges. Leadership within a group of collaborative partners should be clear and accepted by all partners. When operating in an international collaboration, partners should realize that there are cultural differences between nationalities [47]. Furthermore, we believe that for a good collaboration, it is important not only to have contact by phone or digital channels, but seeing each other in real life once in a while is also essential.

An important advantage of collaboration is that shared databases can be created (Box 4), which can be a solution to the small patient groups we have to deal with in ULP research. The goal of the ULP research community should be that each person with an upper limb defect should be included in a shared database. Data should be collected about patient and amputation characteristics (date of birth, sex, medical history, age at limb loss, year/level/cause/side of limb loss, hand dominancy), prosthetic supply (type of prosthesis, including all components and control mechanisms, wearing time, time from amputation to fitting, reimbursement), and outcome data (covering ICF levels: phantom limb pain, stump pain, musculoskeletal complaints, functional performance scores, quality of life, satisfaction with prosthetic supply, and participation in society). We should also exploit other ways to share our knowledge by publishing in open access journals and by sharing our data in public data repositories. It would be advisable to create a dedicated ULP data repository in order to make data easier to find and reuse. If we succeed in doing so, funding agencies might get more interested in ULP research. Collaboration is also needed to share complementary expertise. We would advise the international community to use conferences not only to report on research results, but also to create networks, to invite liaison officers or funding agencies and organize workshops to write grants, to get familiar with alternative study designs, or to improve knowledge on implementation or dissemination. Conferences should not only be the place to be for reporting results, but should also be the start of new beginnings, new ideas, and should be exciting, surprising, and give new energy. Conferences like ISPO World Congress, Trent International Prosthesis Symposium (TIPS), or MyoElectric Control (MEC) could take the lead in the case of ULP research (Box 4).

Box 4. TIPS on Spreading of research findings.

-

Implementation should be part of each research proposal

-

Take advantage of the feel-good content of ULP research by attracting media

-

Make it a habit to use social media and traditional media

-

Make a dissemination plan at the start of a project

-

(Inter)national collaboration and a team approach are essential

-

Establish shared databases and include each person with an upper limb defect in these databases

-

Publish in open access journals

-

Share data in public data repositories

-

Create a dedicated ULP data repository

-

Use conferences as a start of new beginnings

3. Conclusions

The result of scaling-up research of ULP should be a widespread usage of effective innovations by persons with an upper limb defect. In this paper, we evaluated the current status of the quantity and quality of research in ULP. We concluded that the overall quality of research is low and that quantity prevails over quality. To scale up research in ULP, we should pay attention to the transfer of research findings by applying translational research, which means that alternative study designs should be considered in order to move beyond the lab. Innovations should be developed with the principles of evidence-based medicine and evidence-based practice in mind. To improve the quality of research, the body of research findings should be reflected by a shape of a tall hat instead of a sombrero. Patients should be involved in all research steps, and research on able-bodied persons should only be performed if at least a few individuals with ULD are included as well. The spreading of results needs far more attention. We therefore recommend implementation and dissemination plans to be part of each research proposal. Furthermore, we should take more advantage of the feel-good content of our research products and use media more to disseminate results and recruit patients for research. Collaboration between researchers should be encouraged in order to overcome small sample sizes and to collect necessary skills to execute all facets of a research project. Data should become more visible and should be shared. Conferences should not only be used for presentation of results, but should be the start of new beginnings.