Development of a force-ﬁeld-based control strategy for an upper-limb rehabilitation robot

. Robot-assisted rehabilitation has proven to be effective for improving the motor performance of patients with neuromuscular injuries. The effectiveness of robot-assisted training directly depends on the control strategy applied in the therapy training. This paper presents an end-effector upper-limb rehabilitation robot for the functional recovery training of disabled patients. A force-ﬁeld-based rehabilitation control strategy is then developed to induce active patient participation during training tasks. The proposed control strategy divides the 3D space around the rehabilitation training path into a human-dominated area and a robot-dominated area. It encodes the space around the training path and endows the corresponding normal and tangential force; the tangential component assists with movement along the target path, and the normal component pushes the patient’s hand towards the target path using a real-time adjustable controller. Compared with a common force-ﬁeld controller, the human–robot interaction in this strategy is easy and can be quickly adjusted by changing the force ﬁeld’s range or the variation characteristics of two forces, and the intervention in two directions can change continuously and smoothly despite the patient’s hand crossing the two areas. Visual guidance based on the Unity-3D environment is introduced to provide visual training instructions. Finally, the feasibility of the proposed control scheme is validated via training experiments using ﬁve healthy subjects.


Introduction
The number of patients with upper-limb motor dysfunction caused, for example, by stroke and spinal cord injury has increased sharply year over year (Malcolm et al., 2009). About two-thirds of stroke patients survive; however, more than 80 % of them may suffer hemiparesis. These survivors require prolonged physical therapy to recover motor function for the activities of daily living (ADLs) (Cortese et al., 2015). Research on neurological rehabilitation suggests that repetitive motor activity has positive effects with respect to improving movement coordination and avoiding muscle atrophy, and the therapeutic effect is mainly determined by the intention, task-oriented quality and sustainability of rehabilitation training. Robotic systems have a natural advantage in rehabilitation over traditional rehabilitation treatments (Bertani et al., 2017). Robot-assisted therapy can deliver long-endurance, repetitive and sustainable therapeutic training using programmable control strategies (Milot et al., 2013). Furthermore, therapists can obtain a series of quanti-tative assessments of patient training performance to further optimize the treatment strategies (Mehrholz et al., 2012). In recent decades, the application of robotic systems to the rehabilitation treatment of neuromuscular impairment has received increasing attention from around the world (Krebs et al., 2004;Gassert and Dietz, 2018). To date, many rehabilitation robots have been developed, including end-effector and exoskeleton robots. With respect to the aforementioned robot systems, end-effector rehabilitation robots have attracted plenty of interest from widespread researchers, resulting in the development of systems such as MIT-MANUS (Hogan et al., 1992), GENTLE/S (Loureiro et al., 2003), RE-HAROB (Andras et al., 2009), ACRE (Schoone et al., 2007) and PASCAL (Keller et al., 2013).
The effectiveness of robot-assisted rehabilitation treatment is largely determined by the control strategy applied in the therapy training (Jiang et al., 2012;Kahn et al., 2004). Various kinds of control strategies have been developed for endeffector upper-limb rehabilitation robots in order to execute predetermined training tasks. The existing control methods can be classified into passive control strategies and cooperative control strategies according to the interaction between humans and rehabilitation robots (Lindberg et al., 2004). In the early stages of hemiplegia recovery, the patient's affected limb is completely paralyzed, without any muscle contraction or active movement. The passive control strategy is particularly applicable in this situation, as the robot assists the patient to passively perform repetitive flexion and extension training tasks along a predetermined path, helps the patient maintain the normal range of joint movement (Proietti et al., 2016), and lays the foundation for active training. Many controllers have been proposed to ensure the performance of passive training, including fixed-gain PD (proportional and differentiation) controllers (French et al., 2014), neural network-based PI (proportional and integral) controllers (Erol et al., 2005), fuzzy logic PD controllers (Xu et al., 2011), dynamic fuzzy network impedance controllers (Song et al., 2014) and so on. However, if the motion-related central nervous system has been restored but is still weak, a cooperative control strategy can be used, which emphasizes fully mobilizing the patient's intention to actively exercise during a training task in order to maximize the efficiency of rehabilitation training (Mounis et al., 2019). Therefore, the control strategy applied during this stage should facilitate patient-robot interaction with minimal robot intervention and maximal patient effort (Wu et al., 2018;Frisoli et al., 2009;Akiyama et al., 2015;Lee et al., 2020). Wang et al. (2019) developed an end-effector upper-limb rehabilitation robot based on fuzzy logic rules and impedance control; this system uses recursive least squares to estimate the human impedance parameters and quantify the residual motor capacity. These parameters and the motion deviations are input into the fuzzy logic controller. Zhang et al. (2020a) proposed an impedance-based assist-as-needed controller that enables the patient to move freely in the fault-tolerant region and provides assistance according to the patient's functional ability when deviating from the fault-tolerant region. A new performance-based assistance method was developed by Leconte and Ronsse (2016) that can assess the movement features of smoothness, velocity and amplitude during training tasks. Krebs et al. (2003) proposed a novel concept of performance-based progressive robot therapy that uses speed, time or electromyography (EMG) thresholds to initiate robot assistance. Cui et al. (2017) developed a wrenchbased controller to conduct an exoskeleton with 7 degrees of freedom (7DOF) for dexterous motion training that contains four basic force/torque components which guide and correct the position/pose errors. Shi et al. (2022Shi et al. ( , 2020 proposed a human-centered control method for assist-as-needed (AAN) robotic rehabilitation, and they created a feedback-stabilized closest-attitude tracking algorithm according to the geometric properties of a special 3D orthogonal group, SO(3), and then realized the tracking of the robot to the desired position/posture by force/velocity field. Therefore, the main purpose of this paper is to present a new patient-cooperative control framework for an endeffector upper-limb rehabilitation robot that provides robotassisted training for individuals with neuromuscular disorders. First, a minimal-intervention force-field-based control strategy is proposed. It divides the 3D space around the rehabilitation training path into the patient-dominated area and the robot-dominated area, encodes the space, and provides the corresponding normal and tangential forces that guide the patient's hand movement towards and along the target path, respectively; moreover, a damping term is added to maintain the stability of the system. The human-robot interaction can be adjusted by changing the force field's range or variation characteristics to meet the subjects' requirements during different recovery stages. The patient-dominated area enables greater patient initiative with less robot intervention; however, robot intervention increases significantly as the patient's hand deviates into the robot-dominated area. Finally, the feasibility of the proposed control strategy is evaluated by several preliminary training experiments using five healthy subjects who are required to accomplish the task in both the health and mock-paralyzed states. The experimental results are presented and discussed in Sects. 3 and 4.

Minimal-intervention principle
In the mid-to late-recovery stage, part of the motor-related nervous system of the patient has regenerated, and the affected limb has regained some motor function; however, it is still a great challenge for the patient to perform an independent training task. During this stage, the main therapy goals are twofold: (1) to maintain the existing range of motion and the degree of muscular activation and (2) to induce the patient to actively participate in or even lead the training task in order to further improve the patient's neurological function and gradually restore the motor ability of daily movements. Clinical experience has shown that maximizing the usage of recovered motor function is beneficial for patients to improve treatment efficiency and restore psychological confidence. Therefore, in this paper, a cooperative patientrobot control strategy for upper-limb rehabilitation is developed based on the principle of minimal intervention in order to facilitate human-robot interaction by stimulating patient initiative (Nef et al., 2007;Todorov and Jordan, 2013). It is required that the system not interfere with the rehabilitation training task if the patient can perform the expected training task actively; instead, maximum utilization of recovered motor function should be encouraged. However, assuming the patient is not capable of accomplishing the expected training task, the robot should provide the appropriate assistance to the affected limb in order to ensure the integrity of the training task.

Force-field control strategy
A spatial force field is constructed in the 3D space around a predetermined horizontal training path, as shown in Fig. 1. During rehabilitation training, the actual path is likely to deviate from the desired path (e.g., the purple curve shown in Fig. 1). Thus, the position deviation d(t) from the actual position can be calculated. The spatial force field is divided into a patient-dominated area and a robot-dominated area. Note that R T and R N denote the boundaries of the patient-dominated area and the robotdominated area, respectively. The actual motor capabilities of the patient can be estimated via the corresponding area. The position deviation d(t) can be represented as follows: The actual end-effector position is within the patientdominated area if the deviation satisfies the condition of d(t) ∈ [0, R T ), as for point P (1) shown in Fig. 1, in which case the rehabilitation system judges that the patient's motor performance is good enough to actively complete the training task. Efficiency and accuracy are the most important two targets during this stage. The system gives little normal support in this area, and the patient can obtain adequate exercise. However, tangential force can be added to improve the patient's efficiency or as an appropriate push for the patient after a high-intensity training task.
If the position deviation continues to increase and satisfies the condition of d(t) ∈ [R T , R N ), as for point P (2) shown in Fig. 1, the actual end-effector position is within the robotdominated area; in this case, the rehabilitation system judges that the patient's motor performance is not satisfactory to complete the training task independently. The completeness of the training is the most important target at this stage. The system provides sufficient normal support, and this support increases exponentially as the position deviation increase until the centrifugal motion of the patient stops. This force can drive the patient's hand to return to the predetermined path.
It is necessary to mention that the controller parameters should be adjusted according to the different recovery stages and training requirements of patients. In the early stage of rehabilitation, the patient's affected limbs cannot complete the training task; therefore, some measures should be taken to help the patient complete passive training tasks -for instance, appropriately narrowing the patient-dominated area, increasing the normal support, applying greater tangential assistance and so on. On the other hand, patients who have recovered some motor function should be encouraged to actively participate in training; thus, it may be necessary to expand the patient-dominated area and reduce tangential assistance in order for the patient to gain more freedom of movement during the training task. In addition, if the patient's hand always moves back and forth between two areas, although it will not affect the stability of the system, the boundary of two areas still needs to be reasonably modulated to make it more suitable for the individual. Combining the above descriptions, the force-field control strategy is proposed as follows: Here, F N ∈ R 3×1 , F T ∈ R 3×1 and F D ∈ R 3×1 represent the normal force, tangential force and damping item, respectively, and F x ∈ R 3×1 is the assistance force applied on the end-effector. Equation (3) describes an end-effector rotational impedance controller: a virtual rotational spring with rotational stiffness K θ ∈ R 3×3 is attached to keep the endeffector pose θ ∈ R 3×1 close to the desired pose θ d ∈ R 3×1 . The rotational velocityθ ∈ R 3×1 is damped with dissipating element D θ ∈ R 3×3 , and F θ ∈ R 3×1 is the assistance wrench applied on the end-effector. Control models for each item in the force field are as follows: Here, n ∈ R 3×1 (in Eq. 5) represents the direction of normal force, which can be calculated by Eq. (8); K N ∈ R 3×3 is the coefficient of normal force; K N = αK t−1 N + βD, where α is a forgetting factor, and β is a gain factor; K t−1 N represents the normal force coefficient at the last sample time; D = diag{[d x , d y , d z ]} indicates the real-time position deviation of the end-effector in three directions; K T ∈ R 3×3 (in Eq. 6) denotes the tangential coefficient; t ∈ R 3×1 is the direction of the tangential force, which is always in the same direction as the motion; K D ∈ R 3×3 (in Eq. 7) represents the damping coefficient; andẋ ∈ R 3×1 is the translational speed of the end-effector. The variation characteristics of the tangential and normal force around the predetermined path are shown in Fig. 2.
Here, N null ∈ R 7×7 is the null-space projector and is defined as follows: N null = I − J T (q)(J T (q)) # . K q ∈ R 7×7 is joint-space stiffness, B q ∈ R 7×7 is joint-space damping and q ∈ R 7×1 is a real-time joint position with the nominal joint position q d ∈ R 7×1 being constant throughout the trial. The desired assist torque can be represented as follows: Figure 3 shows the overall block diagram of the proposed force-field-based robot rehabilitation system. The left side depicts the force-field control strategy mentioned in this paper, and the right side shows patient-robot interaction.

Experiments and results
The experiments involved in this paper are conducted on an end-effector upper-limb rehabilitation robot (EULRR) platform (Zhang et al., 2020b(Zhang et al., , 2021Sun et al., 2021) with real-time external torque control implemented via the FRI (Fast Research Interface) software package (Schreiber et al., 2010). Five healthy subjects (mean age of 26.2 years and a male to female ratio of 3 : 2) were recruited to participate in the following experiment. The experiment aims to verify the feasibility of the proposed force-field control strategy in rehabilitation training and to investigate the effect of different parameters on the control performance.

Hardware
EULRR is designed to mimic the relative position of the human arms for more effective rehabilitation training. Figure 4 describes the structure of EULRR, which includes a pair of industrial manipulators, an external monitor and the body with an electrical module, a pair of manipulator control cabinets, and a control computer.

Experimental setup
During the test, the subject sat inside the EULRR and kept their torso still, as shown in Fig. 5. The subject's hand was connected to the end of the robot via a handle, with the assumption that the subject's left upper arm was the affected side. To verify the effectiveness of the force field in different areas, the subject had to cross two areas in one single experiment; therefore, a state of muscular spasticity (MS) was artificially constructed, which required the subject to complete the training with a large deviation. Subjects were required to intentionally manipulate the robot end-effector, circulating along the predetermined path for seven laps in every single experiment. During laps 2 to 4, the subjects were asked to complete the training task as well as possible in a muscle normal (MN) state; however, during laps 6 and 7, the training task was performed in the MS state. It is important to note that, in this paper, position deviation is the criterion used to distinguish acceptable or unacceptable training performance.
The experiments were designed under the principle of control variates. The fixed parameters of all of the subexperiments are shown in Table 1. The first experimental group (E1 in Table 2) contains 12 conditions, named T1 to T12, and aims to investigate the effect of different combinations of K T and K D on the force-field control performance. The second experimental group (E2 in Table 2) contains three conditions, named T13 to T15, and aims to investigate the effect of different combinations of α and β on the force-field control performance. The third experimental group (E3 in Table 2) is the control group and has only one condition, named T16. The control group T16 is performed before all of the other groups.
The system provides the subjects with visual guidance, developed based on Unity-3D, as shown in Fig. 6. The green dot on the interface is the real-time position of the endeffector, the arrows point out the real-time directions of the tangential and normal forces, the yellow circle is the top view of the predetermined path, the yellow line is the front view of the predetermined path, and the completed lap of the subject is recorded in the upper left-hand corner. The subject moves clockwise.

Data processing
The experimental data were all processed with MATLAB and SPSS software. The real-time position of the end-effector was calculated according to the robot forward kinematics. The position deviation of the end-effector was calculated using Eq. (1), and the assistance force of the robot was calculated using Eq. (14).
The actual path was visualized along with the predetermined path in order to intuitively point out the difference between robot-assisted movement and free movement, and the average assistance force was displayed along with the endeffector position deviation in order to intuitively discover their relationship. The abrupt engagement of the robot controller at the start of each experiment induced transient behavior in the robot end-effector motion, as the task was not critically damped. To eliminate possible transient behavior from the data analysis, the first revolution in each of these trials was discarded, and the fifth revolution was discarded as well because it recorded the transition behavior between two states, which was meaningless to the experiment.

Result
The actual path of the hand of the first subject (S 1 ) under condition T14 and condition T16 are shown in Figs     under condition T14, the average respective deviations in the MN and MS states are 6.51 and 19.35 mm, whereas those under condition T16 are 23.05 and 34.28 mm. By analyzing all five subjects' average deviations between the two states using a paired t test, significant differences between two states were uncovered, as shown in Table 3, which proves the feasibility of the MS state. Even though the 3D view in Figs. 7 and 8 clearly shows the difference in the actual path between robot-assisted motion and the free motion, there is no significant distinction    between them in the xoy view because the graphical guidance works, but it is difficult for the subject to balance the motion in both the xoy plane and the yoz plane at the same time, which results in greater deviation in the z-axis direction when the subjects are experiencing free movement. This is evident from the xoz or yoz view. The experimental results containing the average assistance force and deviation of the tests conducted by S 1 are shown in Fig. 9. The effect of different combinations of control parameters can be revealed. The red line depicts the variation in the average deviation, while the blue line indicates the variation in the average assistance force. It can be noted that the presence of a force field causes the end-effector deviation to be smaller than that under free motion, regardless of whether the subject exercises in the MN state or the MS state. For each state, the results are statistically analyzed using a paired t test in order to compare the results between conditions. The threshold for the p value is selected to be 5 % for all tests. The results of the paired t test are shown in Table 4. If the t test result is shown with a "-", it means that the mean deviation of the two sets of data is similar; otherwise, the percentage decrease is shown in the table with the p values given in parentheses. The main findings with respect to the paired t test results are as follows: 1. For the two states, conditions T1-T15 and condition T16 present significant differences in their mean deviation: all conditions show great improvements compared with condition T16, which proves that the force field can help subjects improve their training performance. Thus, the feasibility of force field for rehabilitation training is verified.
2. With respect to conditions group T2 and T8 and condition group T5 and T8, there is no significant difference in the mean deviation in either of these groups, resulting in similar performance.
3. In the MN state, conditions T13 and T15 have no significant difference in their mean deviation, although condition T15 shows a 28.17 % improvement compared with condition T13. However, in the MS state, a significant difference is shown between these treatments, despite the similar improvement in the mean deviation compared to that in the MN state. When the subject is in the  MS state, a similar mean deviation improvement rate to that in the MN state represents a larger mean deviation, which can cause significant changes in the human-robot interaction. However, this may not change much in the MN state.
In experimental group E1, the variation in the assistance force and the deviation show the same trend when the characteristics of the normal force are controlled. However, there are still some cases where the assistance forces have opposite trends to the position deviations (e.g., condition T3 in Fig. 9a), and this can be explained by the fact that the average deviations in the MN state are small, with the contribution of the tangential force to the assistance force being much larger than that of the normal force. Nevertheless, when the subjects perform the tasks in the MS state, the position deviations are larger overall, and the effect of tangential force is smaller, resulting in the same trend between the average deviations and assistance forces. The above analysis indicates that the force field can respond effectively to the performance of the subject by applying time-varying and appropriate assistance force.
In experimental group E1-1, the average deviations have a different trend from K T when K D remains constant. This conclusion is also applicable to the comparison between different experimental groups -for instance, condition T4 in E1-2, condition T7 in E1-3 and condition T10 in E1-4 also show the same result. In contrast, for groups E1-2, E1-3 and E1-4, the value of K T remains constant, but the average devi-ations do not grow with an increase in K D . These two points combined with result (2) from the paired t test (shown above) indicate that K T and K D do not play a decisive role in the position constraint. However, as an increase in K T makes the subject's hand move faster, it is necessary to set an appropriate K D as a limitation for movement.
In experimental group E2, the values of K T and K D remain constant, and the variation characteristic of normal force is changed by adjusting the value of β. As the value of β increases, the average deviations gradually decrease, and the average assistance forces increase as well or are almost the same. In the other words, the subject is provided with a larger assistance force at the same position. Therefore, the constraint capacity of the force field is enhanced. The above conclusions combined with result (3) from the paired t test (shown above) provide evidence that the controller parameter should be adjusted in real-time according to the patient's motor performance.

Conclusion
This paper has dealt with the development of a minimalintervention force-field-based control strategy for patientcooperative control of an upper-limb rehabilitation robot that helps the patient to perform active rehabilitation in 3D space. The force field divides the space around the predetermined path and provides appropriate assistance to the patient according to position deviation during training in order to max- The percentages represent the reduction in the average position deviation of condition A compared with that of condition B.
imize the patient's effort. As the patient's hand gradually deviates from the predetermined path, the system increases the normal intervention to deter the patient from the predetermined path. When the patient's hand does not perform well with respect to moving along a predetermined path, selective assistance in the tangential direction can be applied to help the patient complete the task faster or as relaxation after intense training. The experimental results of five healthy subjects show that, for the predetermined training path, the force field could (1) help the subjects improve their rehabilitation performance, (2) respond effectively to the position deviation of the subjects, and (3) give the subjects time-varying and appropriate assistance force. Thus, the feasibility of force field use in rehabilitation has been verified. The experimental results also provide evidence that the controller parameter should be adjusted in real-time according to the patient's motor performance. Future works will be devoted to developing ADL paths to help patients perform specific motions and to recruiting patients for further trials.
Data availability. The data that support the findings of this study are available from the corresponding author upon reasonable request.
Author contributions. JP and LZ conceived the idea, JP, LZ and QS performed all of the experiments. JP drafted the paper, and JP, LZ and QS discussed and edited the paper. JP finalized the paper, including preparing the detailed response letter. LZ and QS supervised the work.
Competing interests. The contact author has declared that none of the authors has any competing interests.

Disclaimer.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.