Scaling up can be defined as “deliberate efforts to increase the impact of health service innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis” [2
]. The WHO published three reports on scaling-up strategies [2
], which were prepared on the understanding that scaling up rarely occurs automatically, as is often assumed. Scaling up needs focused attention, strategic planning and management, as well as resource allocation [4
]. Thus, if research in ULP should move beyond the lab and the impact of successfully tested innovations should benefit the person with an upper limb defect, specific attention should be paid to scaling up this field of research. We summarized part of WHO’s recommendations on how to enhance the attributes of success and the potential for sustainable scaling up in four topics of interest: Translation, Innovation, Patients, and Spreading, composing the acronym TIPS. In this paper these TIPS will be introduced, discussed, and illustrated in light of scaling up research in ULP (Table 1
Translation of research findings from the lab to the target population encompasses the testing of the innovation under real-life operating conditions. To do so, we should take into account translational research, which is defined as the process of applying laboratory research to human studies and enhancing the adoption of evidence-based practices in real-world settings to reach broad populations [5
]. Translational research can be divided into three stages: activities to establish the effectiveness of care (stage 1), activities to assess who benefits from promising care (stage 2), and activities to determine how to deliver high-quality care reliably and in all relevant settings, mostly home try-outs (stage 3) [6
]. Looking at the body of research performed in ULP, most research seems to be performed in Translation stages 1 and 2, but see [7
] for examining prosthesis use after a period of home use (i.e., stage 3). Examples are the many research initiatives to develop multi-grip prosthesis hands or to develop pattern recognition systems to strive for more intuitive control in myoelectric prosthesis users. Only a limited number of authors report on home try-outs [9
]. Only a limited number of development initiatives reach the market. To date, only four multi-grip hands and two pattern recognition systems are commercially available in Europe. To implement research findings in clinical practice or in the patient’s home situation, we should shift more research activities to Translation stage 3 (Box 1
). The digital world we currently live in enables us to design research that can be executed in the home situation. Mobile healthcare technologies (telerehabilitation) enable patients to get their research interventions anytime and anywhere. This will probably create significantly higher patient engagement. Additionally, smart technology-enabled systems can monitor each patient’s progress in great detail [5
]. Smart technology, such as activity monitoring using sensors applied to upper limb prostheses or their users, can provide knowledge on prosthesis wearing patterns, usage of prosthesis components, or compensatory movements. As extensive home trials with large sample sizes might not be feasible, our research field should put less emphasis on traditional research designs, such as randomized controlled trials (RCTs) or case control studies, and should value more alternative study designs, such as qualitative designs, pragmatic clinical trials, participatory action research, or small-N methodology, such as single case experimental designs, small sample, or interrupted time series [5
]. The latter methodologies are characterized by single persons or small groups, repeated outcome measurements, and sequential application or withdrawal of interventions. If carried out properly, the level of evidence can be as high as an RCT. As N-of-1 studies might look like case studies, scientific journals seem to be reluctant to publish such research results. We therefor would like to call for scientific journals to stimulate accepting papers exploiting small-N methodology.
Box 1. TIPS on Translation of research findings.
Focus more research on translational research stage 3
Consider home try-outs to be part of research proposals
Make use of the advantages of smart technology and telerehabilitation
Consider alternative study designs, such as small-N methodology
Scientific journals should accept and stimulate the submission of small-N papers
Scaling up should be initiated after the effectiveness and feasibility of the innovation have been established (Table 1
]. Effectiveness and feasibility studies can only be evaluated if the principles of evidence-based medicine (EBM) and evidence-based practice (EBP) are considered. Therefore, we will here explain and summarize the current status of EBM and EPB in ULP research. EBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients [13
]. EBP is the integration of external scientific evidence, clinical expertise, and patient perspectives to provide high-quality services [14
]. Both EBM and EBP are interlinked.
To apply EBM/EBP the 5-A’s procedure can be used: Ask, Access, Appraise, Apply, and Assess [15
]. The first step, Ask, starts with formulating a PICO(T): Patient or problem, Intervention or exposure, Comparison intervention or exposure, Outcome of interest, and Time or duration of data collection. Next, diverse databases containing citations, scientific journals, full texts, or abstracts (PubMed, PsycINFO, SCOPUS, Cochrane, CINAHL, etc.) can be Accessed to find relevant articles to answer the formulated PICO(T). Subsequently, all articles should be Appraised regarding their design and quality. The strength of evidence of commonly used research designs is reflected in the pyramid of evidence (Figure 1
). Articles revealing data at the study level, such as evidence-based guidelines, systematic reviews, or meta-analyses, are in the top of the pyramid, and articles reflecting data at the subject level are in the bottom of the pyramid. Studies at the subject level with high internal validity, such as RCTs, are categorized higher in the pyramid than studies with low internal validity, such as case reports. Various websites can be helpful to find suitable tools to appraise the quality of each individual article [16
]. The fourth step is to Apply the findings to clinical practice by deciding whether the findings are significant, generalizable, relevant, important, and feasible to implement in clinical practice. If the findings lead to the implementation of an innovation in clinical practice, the final step, Assess, should be applied a while after the implementation.
To assess the level of EBM/EBP in the field of ULP, we searched PubMed for articles using the topics “Upper limb prosthetics” and “Rehabilitation” published in the last 20 years (1999–2018). The search string was composed with the help of a specialized librarian (see Appendix A
). A total of 936 articles were found, mainly published in the last five years (Figure 2
). To determine the type of study designs used, we searched these 936 articles using MESH terms, title/abstract, and keywords. It appeared that 0 guidelines, 9 systematic reviews, 111 narrative reviews, 11 RCT’s (of which 8 reported only on able-bodied participants), 16 clinical controlled trials (of which 11 reported only on able-bodied participants), 134 cohort studies, 105 case reports, and 1 expert opinion were published. The type of study of the remaining articles could not be identified with our search procedure. From this search, we concluded that only a few studies have a position in the top of the pyramid of evidence and many studies form the base of this pyramid.
If we compose a pyramid of evidence using the absolute number of articles found in each category of the pyramid (from evidence-based guidelines in the top to expert opinions in the bottom), the pyramid shape takes a different form, and is best reflected by the shape of a Mexican hat, a so-called sombrero (Figure 3
, left and middle panels).
As the shape of a sombrero is reflecting a body of research of primarily low quality, we may argue that ULP research should strive for a reversed pyramid, reflecting research with a mostly high level of evidence output (Figure 3
, right panel). However, we should realize that in the development of treatments, diagnostics, or other instruments, research needs to go through a series of phases, and the optimal study type for one phase may be inadequate for the next [21
]. An illustrative example is the development of a treatment to diminish phantom limb pain: first a case study was published, followed by a cohort study, and currently a multicenter randomized clinical trial is executed [22
]. As a sombrero or a reversed pyramid both do not seem to be optimal, we suggest that the community of ULP researchers should strive for a tall hat shape of their body of research: a tall hat provides a sound base of case studies, cohort studies, and controlled clinical studies, and has a proportionally in-shape amount of high level of evidence publications, such as RCT’s, systematic reviews, or evidence-based guidelines (Figure 4
The next question that emerges in the light of scaling up research in ULP is how ULP research can move from a sombrero to a tall hat. In order to answer this question, we will first have a more detailed look into the current status of the available literature. Subsequently, we will provide recommendations for scaling up.
Evidence based guidelines, systematic reviews/meta-analyses.
Unfortunately, there is no single database hosting evidence-based guidelines on ULP topics. It is therefore hard to identify and access relevant guidelines. In our search to find guidelines about ULP, we only found the VA/DoD Clinical Practice Guideline “the management of upper extremity amputation rehabilitation” [25
]. Most recommendations of this guideline are based on expert opinions and not on publications with a high level of evidence. More than 10 years ago, we published a systematic review on the relationship between the fitting of a first prosthesis in children with congenital upper limb deficiency and rejection rates [26
]. Back then, we concluded that the clinical practice of the introduction of a prosthesis is guided by clinical expertise rather than by evidence-based medicine. In the limited number of systematic reviews found, we appeared to not be the only authors concluding that evidence is mostly not available and firm conclusions cannot be drawn [27
]. Although we appreciate the huge amount of work put into composing guidelines and systematic reviews and we would encourage to raise the numbers, their value will only increase if the level of evidence of the included studies increases, since their content is as strong as the weakest link, namely the article with the worst quality on which they are based. When collecting results from other studies, it would improve the quality of evidence-based guidelines, systematic reviews, and meta-analyses if standardized protocols (such as standardized activity measures, usage of a core set of questionnaires, specific electrode locations, etc.) were used. Further, we recommend that a database for evidence-based guidelines and systematic reviews specifically dedicated to ULP research be established (see Box 2
). An organization like the International Society for Prosthetics and Orthotics International (ISPO-Int) could host such a database.
Randomized Controlled Trials.
RCTs are judged to be the gold standard when practicing research, which is explained by the high internal validity (low risk of bias) of RCTs. However, some disadvantages also have to be acknowledged, since the external validity and the generalization to the target population can be low. Furthermore, costs are mostly high. In ULP research, only a limited number of RTCs can be found. This can be explained by the difficulties encountered when executing an RCT: double or even single blinding is often not possible, undoing of results can be hard or impossible, especially in training or learning studies, and most studies have to deal with small sample sizes with large subject variation. As a result, most RCTs are performed on able-bodied participants using prosthesis simulators. We have to admit that we also contributed to this practice [28
]. We therefore recommend to always validate results obtained in able-bodied persons with a small sample of persons with an upper limb defect, which also applies to other study designs, such as clinical controlled trials or cohort studies [33
For improving the quality of RCTs, CCTs, cohort, and case studies, we recommend using reporting guidelines (see Box 2
), such as provided [17
]. Furthermore, we would encourage publishing more theoretical foundations regarding developments in ULP research. If grounded on theories, innovative treatments will be developed in a more thorough way, which will contribute to higher quality publications. Examples of theories that might be useful are the treatment theory or the enablement theory [36
]). Moreover, Bongers et al. and van Dijk et al. applied insights from motor control to ULP [38
]. We hope that one of the results of the current paper is that authors feel more encouraged to submit such theoretical papers. Finally, we would like to break a lance for the education of researchers and clinicians. If researchers and clinicians are more aware of the prerequisites of high-quality research, the results will improve. With clinicians, we not only refer to doctors or therapists, but also to prosthetists and technicians. We would therefore recommend research institutes, hospitals, and orthopedic workshops to educate their staff on the basics of good clinical practice. Such courses can be followed easily by using online facilities.
Box 2. TIPS on improving the body of research on ULP Innovations.
Strive for a shape of a tall hat instead of a sombrero for the body of ULP research
Compose more evidence-based guidelines and systematic reviews
Develop standardized protocols, including a core set of questionnaires, to improve the quality of evidence-based guidelines, systematic reviews, and meta-analyses
Establish a dedicated database to host guidelines and systematic reviews on ULP topics
Rely much less on research on able-bodied persons
In all studies, always include at least some persons with ULD
Use established reporting guidelines when reporting study results
Base design innovations on theories
Scientific journals should encourage authors to submit theoretical papers
Provide good clinical practice education to all clinical staff
2.4. Spreading of Research Findings: Implementation, Dissemination, and Collaboration
If an innovation has proven its effectiveness, the implementation of the intervention or treatment should be a logical next step. However, implementation has not gained much attention in our research field. As a consequence, many good initiatives have not reached clinical practice. When designing a research project, most researchers start with formulating an idea and then seek funding, and if funding is obtained, the project is executed, results are analyzed, and a publication is prepared. It is mostly at the latter stage that implementation of research results is considered. However, for implementation to be successful, an implementation plan should be part of the research proposal. Prosthesis users should be involved in order to get a more complete understanding of how novel technologies could address their concerns and how these should be implemented in their daily lives (Box 4
Our own implementation experiences have been described after execution of the PPP-Arm (Prosthesis Prescription Protocol of the Arm) project [46
]. PPP-Arm is a protocol used by all Dutch rehabilitation teams when prescribing an upper limb prosthesis. Success factors regarding the implementation of this national protocol were the involvement of all stakeholders from the beginning of the project; the presence of a dedicated project coordinator; division of the implementation into two phases (where large centers did the first try-outs and smaller centers participated after initial problems were solved); the application of knowledge brokers (mostly occupational therapists), who were assigned to be the main persons responsible to implement the protocol at the work floor; arrangement of financial compensation for the knowledge brokers; regular meetings with knowledge brokers; and dissemination of regular newsletters to keep everybody updated.
Research in ULP is characterized by being applicable to a small population on the one hand and costly on the other hand. These two characteristics imply that the interest of potential research participants or funding agencies has to be drawn in an active way in order to scale up research in ULP. One way to draw the attention of participants or funding agencies is to focus on interesting research results and to use the sexiness of the advanced technological products to attract the media (Box 4
). Media can be various types of social media or traditional media, such as radio or television. The power of media should not be underestimated. Dutch TV broadcasted an item about our participation in the international Phantom Motor Execution trial [24
]. As a result, participants from all over the country sent us requests for participation in the study. Instead of putting a lot of effort in recruiting participants, which is mostly the case in research, we had to create a waitlist. Research in ULP has a large amount of feel-good content, and we should take more advantage of this circumstance. We therefore would advise paying more attention to public relations and make a dissemination plan at the start of the project.
Collaboration is of utmost importance for scaling up research in ULP. A team approach is necessary, since we only can scale up if we collaborate. Important elements for successful collaborations are having a shared mission, having an open mind and trust for each other, feeling responsibility for your own part of the project and for the project part of the partners, and having a team with members who are complementary in skills. Each research team needs different types of expertise, such as engineers, clinicians, human movement scientists, research nurses, implementation experts, valorization experts, public relations officers, and persons representing the target population, the prosthesis users. We have been working on ULP projects for many years now, and we have collaborated with several other research groups worldwide. The majority of these collaborations were fruitful and inspirational, but sometimes we faced some challenges. Leadership within a group of collaborative partners should be clear and accepted by all partners. When operating in an international collaboration, partners should realize that there are cultural differences between nationalities [47
]. Furthermore, we believe that for a good collaboration, it is important not only to have contact by phone or digital channels, but seeing each other in real life once in a while is also essential.
An important advantage of collaboration is that shared databases can be created (Box 4
), which can be a solution to the small patient groups we have to deal with in ULP research. The goal of the ULP research community should be that each person with an upper limb defect should be included in a shared database. Data should be collected about patient and amputation characteristics (date of birth, sex, medical history, age at limb loss, year/level/cause/side of limb loss, hand dominancy), prosthetic supply (type of prosthesis, including all components and control mechanisms, wearing time, time from amputation to fitting, reimbursement), and outcome data (covering ICF levels: phantom limb pain, stump pain, musculoskeletal complaints, functional performance scores, quality of life, satisfaction with prosthetic supply, and participation in society). We should also exploit other ways to share our knowledge by publishing in open access journals and by sharing our data in public data repositories. It would be advisable to create a dedicated ULP data repository in order to make data easier to find and reuse. If we succeed in doing so, funding agencies might get more interested in ULP research. Collaboration is also needed to share complementary expertise. We would advise the international community to use conferences not only to report on research results, but also to create networks, to invite liaison officers or funding agencies and organize workshops to write grants, to get familiar with alternative study designs, or to improve knowledge on implementation or dissemination. Conferences should not only be the place to be for reporting results, but should also be the start of new beginnings, new ideas, and should be exciting, surprising, and give new energy. Conferences like ISPO World Congress, Trent International Prosthesis Symposium (TIPS), or MyoElectric Control (MEC) could take the lead in the case of ULP research (Box 4
Box 4. TIPS on Spreading of research findings.
Implementation should be part of each research proposal
Take advantage of the feel-good content of ULP research by attracting media
Make it a habit to use social media and traditional media
Make a dissemination plan at the start of a project
(Inter)national collaboration and a team approach are essential
Establish shared databases and include each person with an upper limb defect in these databases
Publish in open access journals
Share data in public data repositories
Create a dedicated ULP data repository
Use conferences as a start of new beginnings