Analysis of force feedback mirror-assisted strategy based on adaptive impedance control

When rehabilitation robots assist patients with unilateral upper-limb motor dysfunction in mirror-assisted exercise training, the lack of bilateral tactile feedback may lead to secondary injury on the affected side. To address this issue, a force feedback mirror-aided strategy based on adaptive impedance control is proposed, utilizing a bimanual robot. This strategy establishes a force feedback mirror between the unaffected and affected sides, allowing the unaffected side to perceive the assistive force when the manipulator assists the affected side's movement, thereby ensuring the safety of the mirror-assisted exercise. To achieve mirror trajectory tracking during rehabilitation, a human–robot physical interaction model is developed based on the robotic dynamic model and impedance control. For tactile feedback between the unaffected and affected sides, adaptive impedance control is employed based on the interaction force between the affected side and the manipulator, ensuring that the interaction forces on both sides are proportional and equal in real time. Experimental results demonstrate that, during bilateral mirror-assisted exercise, force feedback mirrors are effectively formed between the two arms, confirming that the proposed strategy not only enables mirror trajectory tracking but also facilitates force feedback mirroring. This study lays the foundation for future safety optimization of robot-assisted mirror rehabilitation training systems.

1 Introduction

Most stroke and spinal cord injury patients exhibit symptoms of hemiparesis and demonstrate unilateral motor dysfunction primarily affecting the upper limbs (Park et al., 2010; Piscitelli et al., 2020). Some researchers have conducted clinical rehabilitation training on patients with upper-limb motor dysfunction based on mirror therapy, and the results of these studies confirm mirror therapy to be an effective rehabilitation intervention (Yang et al., 2018; Harmsen et al., 2015). Traditional mirror therapy utilizes a flat mirror for visual feedback, allowing patients to perceive movement on the affected side while the unaffected side moves in real time (Rossiter et al., 2015). It primarily emphasizes the fidelity of visual feedback while overlooking the importance of proprioceptive feedback. To address some of the issues inherent in traditional mirror rehabilitation methods, there has been growing interest in and a gradual application of mirror-assisted training by robots in upper-limb rehabilitation in hemiparetic patients (Miao et al., 2020; Tai et al.,2020). Moreover, some researchers have employed fMRI (functional magnetic resonance imaging), TMS (transcranial magnetic stimulation), and FMA (Fugl–Meyer assessment) to evaluate and analyze neuroplasticity, brain activation, and motor function, thereby validating the effectiveness of upper-limb mirror rehabilitation (Nam et al., 2017; Wu et al., 2021; Dhakate and Bhattad, 2020). Furthermore, studies have indicated that, during rehabilitation training on the affected side of the upper limb, the unaffected side also needs to participate in movement synergistically, which can enhance the motivation and active motor intent of patients undergoing rehabilitation (Noskin et al., 2008; Kim et al., 2018). Therefore, research on robot-based mirror-assisted strategies is crucial.

Using robot mirror-assisted movement, some researchers acquire motion information from the unaffected side by outfitting it with sensor devices and then transmit this real-time mirrored data to the robot, enabling the robot to assist in mirror movements on the affected side. For instance, some scholars utilize sEMG (surface electromyography) to measure electromyographic signals on the unaffected side and employ SVM (support vector machine) classification to recognize upper-limb movement patterns, thus obtaining trajectories for the unaffected side (Cai et al., 2019; Yang et al., 2022), and so perform mirror trajectory tracking based on impedance control. Additionally, some researchers use IMUs (inertia motion units) to measure posture information of movements on the unaffected side (Yang et al., 2023), or they have participants wear wireless BAN (body area network) sensors to collect real-time movement data from the unaffected side (Wang and Fu, 2011). These studies involve measuring movement information on the unaffected side through external devices, which are relatively easy to wear but suffer from lower accuracy and complexity when it comes to decoding the movement information on the unaffected side. Moreover, the passive training of the affected side with manipulator assistance lacks certain safety measures.

Moreover, some researchers have employed bimanual robots for mirror-assisted movement training. For instance, Harischandra and Abeykoon (2019) proposed an adaptive fuzzy-logic control strategy based on the position coordination error and patient response time for bimanual robots. This method seamlessly switches between resistance mode and assistance mode during mirror training, enhancing positional synchronization between the affected and unaffected sides. Experimental results indicate its efficacy in elbow joint functional recovery on the affected side. Previous literature (Shahbazi et al., 2016; Li et al., 2022) introduced an on-demand assistance control strategy based on patient motion assessment, where robots assist subjects in mirror movements through an adaptive controller (Shahbazi et al., 2016) or an impedance controller based on a GMM (Gaussian mixture model) (Li et al., 2022), thereby encouraging the subject to actively engage in rehabilitation training. Previous literature (Miao et al., 2018; Sheng et al., 2019) proposed an adaptive-admittance-control-based mirror assistance strategy utilizing two UR (universal robots) to achieve bilateral mirror movements. Studies on bilateral training trajectory standardization (Miao et al., 2018) and active–passive mirror assistance strategies (Sheng et al., 2019) have also been conducted, thereby providing a safe and effective bilateral training environment. Chen et al. (2019) proposed a therapist-involved remote mirror assistance movement strategy. By outfitting therapists with IMU sensors and EMG sensors, the therapist's bilateral movement information is mirrored to bilateral exoskeleton robots, assisting patients in bilateral mirror movement training. In summary, the robot assisting the affected side for mirror trajectory tracking, due to unidirectional transmission of movement information, might result in excessive movement range on the affected side, potentially increasing the risk of secondary injury as the unaffected side cannot perceive the movement status of the affected side.

Therefore, this paper proposes a force feedback mirror-assisted strategy based on adaptive impedance control. The guided side of the bimanual robot engages in physical interaction with the healthy side, while the assisted side of the bimanual robot assists the affected side in tracking mirror trajectories. Utilizing impedance control, the manipulator assists the affected side in tracking mirror trajectories and transfers the interaction force to the guided side of the bimanual robot. Subsequently, based on adaptive impedance control, the guided side of the bimanual robot provides force feedback between the healthy side and the affected side, ensuring that the interaction force between the healthy side and the manipulator is proportional to the interaction force between the affected side and the manipulator.

The main contributions of this study are as follows: (1) the assisted side of the bimanual robot employs impedance control to assist the affected side in tracking mirror trajectories, providing a certain fault tolerance for the affected side when tracking the trajectories of the healthy side, thus enhancing the compliance of interactions between the affected side and the manipulator, and (2) a force feedback mirror-assisted strategy based on adaptive impedance control is proposed. During mirror movements, the interaction force between the healthy side and the guided side of the bimanual robot is proportional to the interaction force between the affected side and the assisted side of the bimanual robot, thereby achieving the effect of force feedback mirroring.

The organization of this paper is as follows. In Sect. 2, we introduce the design of the mirror training system which will be used later in this paper. The experimental results of the force feedback mirror-assisted motion are presented in Sect. 3. Finally, conclusions and future prospects are given in Sect. 4.

2 Design of mirror training system

During mirror-assisted training, the guided side of the bimanual robot engages in active and compliant interaction with the healthy side, while the assisted side of the bimanual robot assists the affected side in tracking mirror trajectories. The schematic diagram of mirror training is illustrated in Fig. 1, where O_W−XYZ represents the world coordinate system of the bimanual robot, and O_A−X_AY_AZ_A and O_G−X_GY_GZ_G denote the base coordinate systems of, respectively, the assisted side and guided side of the bimanual robot.

Figure 1 Schematic diagram of bilateral mirror-assisted motion.

This paper defines physical interaction movements between the assisted side of the bimanual robot and the affected side of the human upper limb and between the guided side of the bimanual robot and the healthy side of the human upper limb. To analyze the mirrored pose of the robot's end effector using spatial geometric relationships, pose calibration of the robot is required. However, as the bimanual-robot platform is movable, calibration must be performed whenever the platform's position changes. To address this issue, this paper proposes that, based on the robot's kinematic model, the pose of the end effector can be determined through the mirrored relationship of the joint angles. The mirror relationship between the assisted side and guided side of the bimanual robot relative to the symmetrical plane is represented by Eqs. (1) and (2):

$$\begin{array}{}\text{(1)}& {\displaystyle }{Q}_{\mathrm{A}}=\mathit{\psi }\cdot Q_{\mathrm{G}},\text{(2)}& {\displaystyle }\left\{\begin{array}{l}{X}_{\mathrm{A}}=f\left(Q_{\mathrm{A}}\right)\\ X_{\mathrm{G}}=f\left(Q_{\mathrm{G}}\right)\end{array},\right.\end{array}$$

where $X_{\mathrm{A}}\in R^{m\times \mathrm{1}}$ and $X_{\mathrm{G}}\in R^{m\times \mathrm{1}}$ represent the end effector positions of the assisted side and guided side of the bimanual robot, respectively; $Q_{\mathrm{A}}\in R^{n\times \mathrm{1}}$ and $Q_{\mathrm{G}}\in R^{n\times \mathrm{1}}$ denote the joint angles of the assisted side and guided side of the bimanual robot, respectively; and f(⋅) is the forward kinematic model of the robot; $\mathit{\psi }=\mathrm{Diag}({\mathit{\psi }}_{\mathrm{1}},{\mathit{\psi }}_{\mathrm{2}},\mathrm{\cdots },{\mathit{\psi }}_m)$ is the mirror mapping matrix.

2.1 Force feedback mirroring frame based on adaptive impedance control of mirror-guide side

During interactive motion between the healthy side and the bimanual-robot-guided side, the healthy side drags the bimanual-robot-guided side. The interaction force is defined as the external force F_G,ext acting on the manipulator end effector. Consequently, a dynamic equation model between the healthy side and the bimanual-robot-guided side can be established. The dynamic equation of the joint space manipulator is shown in Eq. (3):

$$\begin{array}{}\text{(3)}& \begin{array}{rl}& {\mathbf{M}}_{\mathrm{G}}\left(q_{\mathrm{G}}\right){\ddot{q}}_{\mathrm{G}}+{\mathbf{C}}_{\mathrm{G}}(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}}){\dot{q}}_{\mathrm{G}}+{\mathit{G}}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)+{\mathit{\tau }}_{\mathrm{f},\mathrm{G}}\\ & ={\mathit{\tau }}_{\mathrm{G}}+{\mathit{\tau }}_{\mathrm{G},\mathrm{ext}},\end{array}\end{array}$$

where ${\mathbf{M}}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)\in R^{n\times n}$ is the positive definite symmetric inertia matrix on the guided side of the bimanual robot; ${\mathbf{C}}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)\in R^{n\times n}$​​​​​​​ is the Gothic force or centrifugal force matrix on the guided side of the bimanual robot; ${\mathit{G}}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)\in R^{n\times \mathrm{1}}$ is the gravity vector on the guided side of the bimanual robot; ${\mathit{\tau }}_{\mathrm{f},\mathrm{G}}\in R^{n\times \mathrm{1}}$ is the joint friction force vector on the guided side of the bimanual robot; ${\mathit{\tau }}_{\mathrm{G}}\in R^{n\times \mathrm{1}}$ is the joint driving torque vector on the guided side of the bimanual robot; ${\mathit{\tau }}_{\mathrm{G},\mathrm{ext}}\in R^{n\times \mathrm{1}}$ is the external torque vector received by the guided side of the bimanual robot; and q_G, ${\dot{q}}_{\mathrm{G}}$, and ${\ddot{q}}_{\mathrm{G}}$ are, respectively, the joint angle, angular velocity, and angular acceleration on the guided side of the bimanual robot.

The mapping relationship between the torque and angular velocity in the robot joint space and the force or moment and velocity in the task space is commonly represented using Jacobian matrices, as shown in Eqs. (4) and (5):

$$\begin{array}{}\text{(4)}& {\displaystyle }\mathit{\tau }={\mathbf{J}}^T\left(q\right)F,\text{(5)}& {\displaystyle }\dot{\mathit{x}}=\mathbf{J}\left(q\right)\dot{q},\end{array}$$

where ${\mathbf{J}}^T\left(q\right)\in R^{n\times m}$ is the Jacobian transpose of the manipulator.

Since the control strategy for human–robot interaction motion is performed in task space, based on Eqs. (4) and (5), the dynamic equation of the manipulator in task space is established, as shown in Eq. (6):

$$\begin{array}{}\text{(6)}& \begin{array}{rl}& {\mathbf{\Gamma }}_{\mathrm{G}}\left(q_{\mathrm{G}}\right){\ddot{\mathit{x}}}_{\mathrm{G}}+{\mathit{\mu }}_{\mathrm{G}}(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}}){\dot{\mathit{x}}}_{\mathrm{G}}+{\mathit{F}}_{\mathrm{G},\mathrm{g}}\left(q_{\mathrm{G}}\right)+{\mathit{F}}_{\mathrm{G},\mathrm{f}}\\ & ={\mathit{F}}_{\mathrm{G}}+{\mathit{F}}_{\mathrm{G},\mathrm{ext}},\end{array}\end{array}$$

where ${\mathbf{\Gamma }}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)=\left(\mathbf{J}\right(q_{\mathrm{G}}\left){\mathbf{M}}_{\mathrm{G}}^{-\mathrm{1}}\right(q_{\mathrm{G}}\left){\mathbf{J}}^T\right(q_{\mathrm{G}}){)}^{-\mathrm{1}}\in R^{m\times m}$ is the inertia matrix vector mapped to the end effector of the manipulator; ${\mathit{\mu }}_{\mathrm{G}}(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}}){\dot{\mathit{x}}}_{\mathrm{G}}={\mathbf{\Gamma }}_{\mathrm{G}}\left(q_{\mathrm{G}}\right)\left(\mathbf{J}\right(q_{\mathrm{G}}\left){\mathbf{M}}_{\mathrm{G}}^{-\mathrm{1}}\right(q_{\mathrm{G}}\left){\mathbf{C}}_{\mathrm{G}}\right(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}})-\dot{\mathbf{J}}(q_{\mathrm{G}}\left)\right){\dot{q}}_{\mathrm{G}}\in R^{m\times \mathrm{1}}$ is the Coriolis force or centrifugal force vector mapped to the end effector of the manipulator; ${\mathit{F}}_{\mathrm{G},\mathrm{g}}\left(q_{\mathrm{G}}\right)=\left({\mathbf{J}}^{\ast }\right(q_{\mathrm{G}}\left){)}^T{\mathit{G}}_{\mathrm{G}}\right(q_{\mathrm{G}})\in R^{m\times \mathrm{1}}$ is the gravity vector mapped to the end effector of the manipulator; ${\mathit{F}}_{\mathrm{G},\mathrm{f}}=\left({\mathbf{J}}^{\ast }\right(q_{\mathrm{G}}){)}^T{\mathit{\tau }}_{\mathrm{G},\mathrm{f}}\in R^{m\times \mathrm{1}}$ is the friction force vector mapped to the end effector of the manipulator; ${\mathit{F}}_{\mathrm{G}}=\left({\mathbf{J}}^{\ast }\right(q_{\mathrm{G}}){)}^T{\mathit{\tau }}_{\mathrm{G}}\in R^{m\times \mathrm{1}}$ is the joint driving force vector mapped to the end effector of the manipulator; ${\mathit{F}}_{\mathrm{G},\mathrm{ext}}=\left({\mathbf{J}}^{\ast }\right(q_{\mathrm{G}}){)}^T{\mathit{\tau }}_{\mathrm{G},\mathrm{ext}}\in R^{m\times \mathrm{1}}$ is the interaction force vector of the external environment of the manipulator; x_G, ${\dot{\mathit{x}}}_{\mathrm{G}}$, and ${\ddot{\mathit{x}}}_{\mathrm{G}}$ are, respectively, the position, velocity, and acceleration vectors of the manipulator.

The Jacobian matrix defined the mapping relationship between joint velocities and linear or angular velocities (Sun et al., 2024). When a manipulator has redundant degrees of freedom or is in a singular position (i.e., n>m), even small task space velocities can result in significant joint velocities. When computing the inverse of the Jacobian matrix, it is common to construct a pseudo-inverse of the Jacobian matrix based on a positive definite symmetric weight matrix $\mathbf{W}\in R^{n\times n}$ (Sun et al., 2023). Consequently, the pseudo-inverse of the Jacobian matrix on the guided side of the bimanual robot is represented as shown in Eq. (7):

$$\begin{array}{}\text{(7)}& {\mathbf{J}}_{\mathrm{W}}^{\ast }={\mathbf{W}}^{-\mathrm{1}}\cdot {\mathbf{J}}_{\mathrm{G}}^T\cdot ({\mathbf{J}}_{\mathrm{G}}\cdot {\mathbf{W}}^{-\mathrm{1}}\cdot {\mathbf{J}}_{\mathrm{G}}^T{)}^{-\mathrm{1}}.\end{array}$$

During the interactive motion between the healthy side and the guided side of the bimanual robot, compliant motion of the mirror-guided side is achieved by the healthy side dragging it based on impedance control. Following Hogan's impedance control theory (Hogan, 1984), a second-order control system based on mass–damping–stiffness is established as shown in Eq. (8):

$$\begin{array}{}\text{(8)}& \begin{array}{rl}& {\mathbf{M}}_C({\ddot{\mathit{x}}}_{\mathrm{G},\mathrm{e}}-{\ddot{\mathit{x}}}_{\mathrm{G}})+{\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}}-{\dot{\mathit{x}}}_{\mathrm{G}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{G},\mathrm{e}}-{\mathit{x}}_{\mathrm{G}})\\ & ={\mathit{F}}_{\mathrm{G},\mathrm{e}}-{\mathit{F}}_{\mathrm{G},\mathrm{ext}},\end{array}\end{array}$$

where M_C is the symmetric positive definite mass matrix of the task space, D_C is the symmetric positive definite damping matrix of the task space, and K_C is the symmetric positive definite stiffness matrix of the task space. Equation (9) can be obtained from Eqs. (6) and (8) simultaneously.

$$\begin{array}{}\text{(9)}& \begin{array}{rl}& {\mathbf{\Gamma }}_{\mathrm{G}}\left(q_{\mathrm{G}}\right){\ddot{\mathit{x}}}_{\mathrm{G}}+{\mathit{\mu }}_{\mathrm{G}}(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}}){\dot{\mathit{x}}}_{\mathrm{G}}+{\mathit{F}}_{\mathrm{G},\mathrm{g}}\left(q_{\mathrm{G}}\right)+{\mathit{F}}_{\mathrm{G},\mathrm{f}}\\ & ={\mathit{F}}_{\mathrm{G}}+{\mathbf{M}}_C({\ddot{\mathit{x}}}_{\mathrm{G}}-{\ddot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})+{\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{G}}-{\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})\\ & +{\mathbf{K}}_C({\mathit{x}}_{\mathrm{G}}-{\mathit{x}}_{\mathrm{G},\mathrm{e}})\end{array}\end{array}$$

According to Eq. (9), if Γ_G(q_G) is set to be equal to M_C, the expected acceleration is zero, and then Eq. (9) is simplified to Eq. (10).

$$\begin{array}{}\text{(10)}& \begin{array}{rl}& {\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{G}}-{\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{G}}-{\mathit{x}}_{\mathrm{G},\mathrm{e}})+{\mathit{\mu }}_{\mathrm{G}}(q_{\mathrm{G}},{\dot{q}}_{\mathrm{G}}){\dot{\mathit{x}}}_{\mathrm{G}}\\ & +{\mathit{F}}_{\mathrm{G},\mathrm{g}}\left(q_{\mathrm{G}}\right)+{\mathit{F}}_{\mathrm{G},\mathrm{f}}={\mathit{F}}_{\mathrm{G}}\end{array}\end{array}$$

To prevent secondary injury on the affected side during assistance by the bimanual robot, real-time perception of the interaction force between the affected side and the mirror-assisted side by the healthy side is essential. Employing adaptive impedance control, the end effector of the bimanual robot engages in compliant interaction motion with the healthy side. In order to realize the mirror feedback of the interaction force between the two sides of the bimanual robot, adaptive adjustments of impedance controller parameters are made at the end effector. During mirror-assisted movements, the interaction motion progresses slowly, resulting in minimal influence by the damping term of the impedance controller, with interaction forces primarily being affected by stiffness parameters. Thus, by designing a stiffness parameter, a force feedback mirroring strategy for mirror-assisted movements is implemented.

The stiffness parameter consists of a stiffness coefficient matrix and a position deviation term. Real-time acquisition of the interaction force between the affected side and the mirror-assisted side is used as the influencing factor for adaptive impedance control. By combining Eqs. (6) and (8), an adaptive stiffness matrix coefficient K_C can be constructed as in Eq. (12) or adaptive desired position parameters x_G,e can be formulated as in Eq. (13) based on the interaction force between the affected side and the mirror-assisted side.

$$\begin{array}{}\text{(11)}& {\displaystyle }{\mathit{F}}_{\mathrm{G},\mathrm{ext}}=\mathit{\alpha }{\mathit{F}}_{\mathrm{A},\mathrm{ext}}\text{(12)}& {\displaystyle }{\mathbf{K}}_C={\displaystyle \frac{\mathit{\alpha }{\mathit{F}}_{\mathrm{A},\mathrm{ext}}-{\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{G}}-{\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})}{{\mathit{x}}_{\mathrm{G}}-{\mathit{x}}_{\mathrm{G},\mathrm{e}}}}\text{(13)}& {\displaystyle }{\mathit{x}}_{\mathrm{G},\mathrm{e}}={\mathit{x}}_{\mathrm{G}}-{\displaystyle \frac{\mathit{\alpha }{\mathit{F}}_{\mathrm{A},\mathrm{ext}}-{\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{G}}-{\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})}{{\mathbf{K}}_C}}\end{array}$$

In the above, α is the scaling coefficient of the force feedback mirroring, and F_A,ext is the human–robot interaction force on the mirror-assisted side.

According to Eq. (12), in order to obtain adaptive stiffness parameters K_C, it is necessary to set the desired position x_G,e. If the desired position is set as the position from the previous servo iteration cycle, when the interaction motion pauses, the change in position approaches zero, resulting in particularly large stiffness matrix coefficients. However, as indicated by Eq. (13), to obtain adaptive desired positions, stiffness matrix coefficients K_C need to be set. Considering the compliance of interaction motion, the stiffness matrix coefficients should be much smaller than those of the assisted side, and when the mirrored interaction force is too large, it may lead to desired-position settings that are beyond the workspace of the manipulator. Therefore, to construct an adaptive impedance controller, it is necessary to combine Eqs. (12) and (13), both satisfying compliant interaction at the end effector and achieving force feedback mirroring for mirror-assisted movements. The framework of the adaptive impedance control is illustrated in Fig. 2.

Figure 2 Frame diagram based on adaptive impedance control.

2.2 Mirror trajectory tracking framework based on impedance control for mirror-assisted side

Research on control methods for robot mirror-assisted movements is abundant, typically employing robot position control methods such as PID (proportional integral derivative) control, admittance control, and sliding-mode control for mirror trajectory tracking (Nisar et al., 2024). However, these mirror-assisted strategies, in the pursuit of precision in mirroring trajectory tracking, often overlook the compliance of human–robot interaction motion. This study utilizes impedance control for mirror trajectory tracking, which, although it may reduce the precision of mirrored trajectories, enhances the compliance of interaction motion between the affected side and the manipulator. The mirrored position of the guided side of the bimanual robot is derived from Eqs. (1) and (2), and this position is used as the desired position for the impedance controller on the mirror-assisted side. Subsequently, a task space impedance model for interaction motion between the healthy side and the mirror-assisted side is constructed, as shown in Eq. (14).

$$\begin{array}{}\text{(14)}& \begin{array}{rl}& {\mathbf{M}}_C({\ddot{\mathit{x}}}_{\mathrm{A}}-{\ddot{\mathit{x}}}_{\mathrm{A},\mathrm{e}})+{\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{A}}-{\dot{\mathit{x}}}_{\mathrm{A},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{A}}-{\mathit{x}}_{\mathrm{A},\mathrm{e}})\\ & ={\mathit{F}}_{\mathrm{A},\mathrm{ext}}\end{array}\end{array}$$

According to the dynamics model of the manipulator, Eq. (14) is incorporated considering the fact that human–robot interaction motion proceeds slowly, and the influence of acceleration terms is neglected. Thus, the control model for human–robot interaction motion can be simplified to Eq. (15).

$$\begin{array}{}\text{(15)}& \begin{array}{rl}& {\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{A}}-{\dot{\mathit{x}}}_{\mathrm{A},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{A}}-{\mathit{x}}_{\mathrm{A},\mathrm{e}})+{\mathit{\mu }}_{\mathrm{A}}(q_{\mathrm{A}},{\dot{q}}_{\mathrm{A}}){\dot{\mathit{x}}}_{\mathrm{A}}\\ & +{\mathit{F}}_{\mathrm{A},\mathrm{g}}\left(q_{\mathrm{A}}\right)+{\mathit{F}}_{\mathrm{A},\mathrm{f}}={\mathit{F}}_{\mathrm{A}}\end{array}\end{array}$$

2.3 Force feedback mirroring-assisted motion control framework

Compared to traditional mirror-assisted movements, force-feedback-based mirror control ensures that the magnitude of the interaction force between the healthy side and the guided side of the bimanual robot is proportional to the magnitude of the interaction force between the affected side and the assisted side of the bimanual robot. Through the bimanual robot, force feedback mirroring between the healthy and affected sides of the upper limbs is achieved to enhance the safety of mirror-assisted movements. This study employs adaptive impedance control for interaction control between the healthy side and the mirror-guided side, while damping stiffness control is used for interaction control between the affected side and the mirror-assisted side; thus, both sides of the human–robot interaction motion are based on impedance control theory.

In summary, by combining the manipulator joint space dynamic model from Eq. (3) with the impedance control models from Eqs. (8) and (17), the input torques τ_m and τ_s for the guided side and assisted side of the bimanual robot can be obtained, as shown in Eqs. (16) and (17).

$$\begin{array}{}\text{(16)}& {\displaystyle }\begin{array}{rl}{\mathit{\tau }}_m& ={\mathit{\tau }}_d+{\mathbf{J}}_{\mathrm{G}}^T{\mathit{F}}_{\mathrm{G}}\\ & ={\mathit{\tau }}_d+{\mathbf{J}}_{\mathrm{G}}^T\left({\mathbf{D}}_C\right({\dot{\mathit{x}}}_{\mathrm{G}}-{\dot{\mathit{x}}}_{\mathrm{G},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{G}}-{\mathit{x}}_{\mathrm{G},\mathrm{e}}\left)\right)\end{array}\text{(17)}& {\displaystyle }\begin{array}{rl}{\mathit{\tau }}_s& ={\mathit{\tau }}_d+{\mathbf{J}}_{\mathrm{A}}^T{\mathit{F}}_{\mathrm{A}}\\ & ={\mathit{\tau }}_d+{\mathbf{J}}_{\mathrm{A}}^T\left({\mathbf{D}}_C\right({\dot{\mathit{x}}}_{\mathrm{A}}-{\dot{\mathit{x}}}_{\mathrm{A},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{A}}-{\mathit{x}}_{\mathrm{A},\mathrm{e}}\left)\right)\end{array}\end{array}$$

Combining Eqs. (16) and (17), a control framework for force-feedback-based mirror-assisted movement is constructed, as illustrated in Fig. 3. The control framework is divided into the assisted side and the guided side, with data exchange between the two manipulators being facilitated by the UDP/IP (User Datagram Protocol/Internet Protocol) communication protocol.

Figure 3 Force feedback mirroring control framework based on adaptive impedance control.

According to Fig. 3, based on adaptive impedance control, the guided side of the bimanual robot interacts physically with the healthy side, mapping the mirrored pose of the guided side to the mirror-assisted side in real time. This enables the mirror-assisted side to assist the affected side in mirroring trajectory tracking. Simultaneously, the assistance force from the mirror-assisted side is mapped to the guided side of the bimanual robot. By adjusting the influencing factor of the adaptive impedance controller, force feedback mirroring between the mirror-assisted side and the mirror-guided side is achieved.

3 Experiments and discussion

In this section, the experimental results are presented to demonstrate the feasibility of the proposed force feedback mirroring-assisted movement strategy. The purpose of the experiments is to test the functionality of the controller rather than to conduct clinical biological responses or medical trials with patients; hence, the experiments involved only healthy participants. The control method proposed in this paper is implemented on an end-effector-based upper-limb bilateral rehabilitation robot platform.

3.1 Bilateral mirror training system

The model of the upper-limb bilateral motion assistance training platform developed by the research group is depicted in Fig. 4, consisting mainly of a PC host, an adjustable seat, a display screen, and two KUKA IIWA manipulators. Additionally, both the software and hardware of the upper-limb bilateral rehabilitation training platform are equipped with emergency stop functions (Sun et al., 2023). The transformation relationship between the coordinate systems of the bimanual-robot system is illustrated on the right side of Fig. 4. Here, O_W−XYZ represents the world coordinate system of the robot, O_L−XYZ represents the base coordinate system of the left manipulator, O_R−XYZ represents the base coordinate system of the right manipulator, O_HR−XYZ represents the coordinate system of the physical interaction point on the right side, and O_HL−XYZ represents the coordinate system of the physical interaction point on the left side. The visually guided interaction interface is depicted on the left side of Fig. 4, featuring two yellow circles representing the desired trajectories for the left and right sides, along with a virtual motion channel created based on these trajectories. Compared to other rehabilitation devices, this platform is utilized for end-effector-based upper-limb assistance training, enabling both unilateral and bilateral upper-limb assistance training strategies.

Figure 4 The upper-limb bilateral rehabilitation training system model.

The control strategy proposed in this paper is implemented by writing programs in C++ language on the Ubuntu Linux operating system. Data transmission between the PC and the manipulator control cabinet is facilitated via UDP/IP communication based on the FRI (Fast Research Interface) library, with a real-time module achieving a maximum speed of up to 1 kHz (Schreiber et al., 2010). Additionally, the real-time module of the manipulator also supports external torque control, with its control rate being outlined in Eq. (18):

$$\begin{array}{}\text{(18)}& \begin{array}{rl}{\mathit{\tau }}_{\mathrm{cmd}}& =\mathbf{J}\left(q{)}^T\right(\mathbf{K}({\mathit{x}}_{\mathrm{fri}}-{\mathit{x}}_{\mathrm{msr}})+\mathbf{D}\left(d_C\right))+{\mathit{\tau }}_{\mathrm{fri}}\\ & +{\mathit{f}}_{\mathrm{dynamics}}(q,\dot{q},\ddot{q}),\end{array}\end{array}$$

where ${\mathit{\tau }}_{\mathrm{cmd}}\in R^{n\times \mathrm{1}}$ represents the controller input torque vector, $\mathbf{K}\in R^{m\times m}$ represents the stiffness matrix of the impedance controller in the internal task space of the robot, ${\mathit{x}}_{\mathrm{fri}}\in R^{m\times \mathrm{1}}$ denotes the desired position in the task space, ${\mathit{x}}_{\mathrm{msr}}\in R^{m\times \mathrm{1}}$ represents the real-time pose of the manipulator in the task space, $\mathbf{D}\left(d_C\right)\in R^{\mathrm{m}\times \mathrm{1}}$ represents the force generated by the damping term inside the robot, ${\mathit{\tau }}_{\mathrm{fri}}\in R^{n\times \mathrm{1}}$ denotes the externally superimposed torque vector, and ${\mathit{f}}_{\mathrm{dynamics}}(q,\dot{q},\ddot{q})\in R^{n\times \mathrm{1}}$ denotes the dynamics term of the robot.

In order to realize force feedback mirroring based on the mirror-assisted motion of a bimanual robot, a human–robot physical interaction method based on adaptive impedance control is proposed. This method enables the robotic assistance of the affected side to track mirrored trajectories while also facilitating force feedback mirroring between the healthy and affected sides. The magnitude of mirrored interaction forces can be adjusted by means of proportional influence factor coefficients. Moreover, participants can use their active forces to correct trajectories instead of strictly adhering to reference trajectories, enhancing the compliance of mirror-assisted movements. The input torque of the human–robot interaction controller on the mirror-guided side is calculated using Eq. (15), and by combining Eqs. (8), (12), and (13), the external input torque can be obtained, as shown in Eq. (19).

$$\begin{array}{}\text{(19)}& \begin{array}{rl}{\mathit{\tau }}_{\mathrm{cmd}}& ={\mathbf{D}}_C({\dot{\mathit{x}}}_{\mathrm{A}}-{\dot{\mathit{x}}}_{\mathrm{A},\mathrm{e}})+{\mathbf{K}}_C({\mathit{x}}_{\mathrm{A}}-{\mathit{x}}_{\mathrm{A},\mathrm{e}})\\ & +{\mathit{f}}_{\mathrm{dynamics}}(q,\dot{q},\ddot{q})\end{array}\end{array}$$

3.2 Subject

The experimental testing in this study involved five healthy male participants (average age of 25.6 years, average height of 175 cm) with no upper-limb movement disorders. Participants sat on an adjustable-height seat, facing away from the robot, for bilateral mirror-assisted movement training, as shown in Fig. 5. Additionally, participants imitated the affected-side movements of stroke patients with their non-dominant hand and the healthy-side movements of stroke patients with their dominant hand. Since this study focuses on the feasibility of the force feedback mirroring strategy in robot mirror-assisted locomotion, factors such as compensatory movements by patients were disregarded, ensuring the validity of the experimental results.

Figure 5 Experimental test scenario.

3.3 Design of experiment

To validate the feasibility of the proposed method, a set of mirror-assisted training exercises designed for planar task orientation was developed. The flowchart illustrating the data transmission and computation process is shown in Fig. 6. Based on the analysis of dexterity within the workspace and the distribution of human–robot interaction dexterity in bilateral upper-limb rehabilitation robots, discussed as a reference (Sun et al., 2021), a circular desired trajectory was constructed within the plane at $Z=-\mathrm{425}$ mm. The center of the circle was located at position [655 mm, −552 mm, −425 mm] in the robot's coordinate system O_R−XYZ, with a radius of 100 mm. Additionally, a virtual channel with the desired trajectory as its centerline and a radius of 25 mm was created. Real-time position information of the end effector of the manipulator was transmitted to a visualization interface built using Unity 3D software, providing real-time visual feedback. The data transmission frequency between the guided side and assisted side of the bimanual robot, as well as between the manipulator and Unity 3D, was set to 500 Hz using the UDP/IP data communication protocol. Participants were instructed to move their healthy side along the desired trajectory within the virtual channel as much as possible. Furthermore, the interaction force ratio coefficient between the mirror-assisted side and the guided side was set to 3, and the stiffness matrix and damping ratio matrix for the task space impedance controller of the affected side were configured, with detailed parameter data provided in Table 1.

Figure 6 Flowchart of mirror-assisted movement experiments.

Table 1 Experiment impedance parameters.

3.4 Results

The participants engaged in four rounds of circular trajectories for bilateral mirror movements guided by visual cues. Data on joint angle transformations for the two manipulators were obtained through position sensors and UDP/IP communication. Subsequently, the motion trajectory of the human–robot interaction points in coordinate system O_W−XYZ was computed based on the forward kinematics of the manipulator. Taking the test results of participant one as an example, the motion trajectory is illustrated in Fig. 7. In Fig. 7, the red trajectory represents the movement trajectory of the participant's left hand, the black trajectory represents the reference trajectory, the black dot indicates the center of the constructed reference trajectory, light blue denotes the virtual channel constructed, and the blue trajectory represents the movement trajectory of the participant's right hand.

Figure 7 Bilateral mirror motion trajectory diagram.

To analyze the trajectory deviation during mirror trajectory tracking based on impedance control, real-time position information of both the assisted side and guided side of the bimanual robot was synchronously recorded, and their mirror deviation changes were calculated based on the principle of mirror symmetry. Taking the test results of participant one as an example, the bilateral mirror trajectory deviation changes are illustrated in Fig. 8. In Fig. 8, the dashed red line represents the positional deviation in the x-axis direction during bilateral mirror movements, the dashed pink line represents the positional deviation in the y-axis direction during bilateral mirror movements, the dashed blue line represents the positional deviation in the z-axis direction during bilateral mirror movements, and the solid black line represents the positional deviation during bilateral mirror movements.

Figure 8 Bilateral mirror trajectory deviation change diagram.

To test the feasibility of force feedback mirroring based on adaptive impedance control, external forces were detected in real-time using joint torque sensors of the robot, and the interaction force between the human and the end effector of the manipulator was calculated using Eq. (4). The variations in interaction forces on both the assisted side and guided side of the bimanual robot were synchronously recorded in real time. Taking the test results of participant one as an example, the changes in bilateral interaction forces are illustrated in Fig. 9. The left side of Fig. 9 represents the scale values of the interaction force magnitude, while the right side represents the scale values of the interaction force ratio coefficients. The solid blue line represents the variation in the assistance force exerted on the participant's left side by the manipulator, the dashed blue line represents the variation in the resistance force exerted on the participant's right side by the manipulator, the solid red line represents the variation in the ratio of the assistance force on the participant's left side to the resistance force on the participant's right side during bilateral mirror movements, and the dashed red line represents the variation in the theoretical ratio of the assistance force on the participant's left side to the resistance force on the participant's right side during bilateral mirror movements.

Figure 9 Interaction force change diagram for bilateral mirror motion.

According to Fig. 9, the ratio between the assistance force exerted by the manipulator on the subject's left side and the resistance force applied on the right side shows a maximum deviation of approximately 0.4 between the theoretical and actual values. The discrepancy between the theoretical and experimental results can primarily be attributed to factors such as measurement errors from the sensors, compensation errors in the friction modeling of the robot's dynamics, and synchronization errors during data acquisition.

Finally, the average trajectory deviation and interaction force ratio of each participant during bilateral force feedback mirror-assisted movements were statistically analyzed, as depicted in Fig. 10. The left side of Fig. 10 represents the scale values of the average trajectory deviation magnitude, while the right side represents the scale values of the interaction force ratio coefficients. The blue line indicates the variations in the mean values of mirror trajectory deviations for each participant, while the red line represents the variations in the mean values of the interaction force ratios for each participant.

Figure 10 Changes in mean mirror trajectory bias and interaction force ratio for five subjects.

3.5 Discussion

From the test results depicted in Fig. 8, it is evident that the average deviation between the affected-side trajectory and the healthy-side trajectory is approximately 8.7 mm. When mirror-assisted trajectory tracking is conducted on the mirror-assisted side based on impedance control, a certain degree of deviation is permitted, thereby enhancing the compliance of interaction between the affected side and the robot. Moreover, Fig. 7 reveals the presence of periodicity in the deviation changes. The trajectory tracking on the affected side in this study is based on fixed stiffness impedance control, as demonstrated in Fig. 8, which also exhibits periodic changes in the assistance force of the manipulator. This suggests the existence of certain biomechanical characteristics when participants engage in periodic circular trajectory movements; specific regions indicate smoother upper-limb movements and vice versa. Figure 8 illustrates that the average ratio of the assistance force exerted on the participant's left side to the resistance force exerted on the participant's right side is 3.14, while the ratio set in this study is 3, indicating the presence of interaction force mirroring deviation. Part of the reason for this discrepancy is the measurement deviation of the sensors. Additionally, Fig. 9 indicates that the average ratio of the assistance force exerted on the left side of the five participants to the resistance force exerted on their right side fluctuates around 3, with the mirror trajectory deviation being around 8.4 mm.

In conclusion, the force feedback mirror-assisted motion constructed based on adaptive impedance control can mirror the force feedback of bilateral mirror-assisted motion to a certain extent, which verifies the feasibility of the proposed method.

The shortcomings of this study are as follows. (1) The parameters selected for task space impedance control in the experimental tests were not optimal, and the issue of torque generated by task space rotational layer impedance control was not investigated, necessitating further optimization in subsequent studies. (2) In the mirror-assisted movement studied in this paper, impedance control was used for trajectory tracking on the affected side, which increased the compliance of human–robot interaction but reduced the accuracy of trajectory tracking. The stiffness coefficient of the impedance control needs further optimization. (3) Only healthy subjects were selected for experimental testing to validate the proposed control strategy in this study, necessitating further recruitment of patients for clinical testing to enhance the effectiveness and persuasiveness of the proposed method.

4 Future work

​​​​​​​In the next phase of our research, we plan to conduct clinical experiments with patients to further validate the effectiveness of the method proposed in this paper. Future research will focus on (1) experimental validation of mirror-assisted movement in hemiplegic patients with upper-limb motor dysfunction and (2) implementing ADL (activity of daily living) tasks using virtual-reality technology, providing patients with auditory or visual feedback and incentive rewards to enhance motivation for rehabilitation training.

5 Conclusion

This paper proposes a force feedback mirror-assisted strategy based on adaptive impedance control. In this strategy, the subject's unimpaired side drags the mirror-guided side for physical interaction movement based on the adaptive impedance control, while the mirror-assisted side, using impedance control, aids the impaired side in tracking the mirror trajectory so as to realize the mirror synchronous movement between the two sides of the subject.

The system employs UDP/IP communication to obtain the real-time interaction forces between the impaired side and the mirror-assisted side, allowing the unimpaired side to assist the impaired side's mirrored movements via a bimanual robotic system. Additionally, a force feedback mechanism enables real-time perception of proportional assistive forces applied by the mirror-assisted side, thereby enhancing the safety of mirror training.

Experimental tests indicate that the bimanual robotic system effectively applies assistive or impedance forces to the subject's impaired and unimpaired sides in accordance with the predefined proportional coefficients. Although minor deviations were observed in the mirror trajectory, the experimental results demonstrate the feasibility of the proposed strategy in achieving both trajectory mirroring and force feedback mirroring, providing a solid theoretical foundation for further optimization of rehabilitation training systems.