Robot Training With Vector Fields Based on Stroke Survivors’ Individual Movement Statistics

The wide variation in upper extremity motor impairments among stroke survivors necessitates more intelligent methods of customized therapy. However, current strategies for characterizing individual motor impairments are limited by the use of traditional clinical assessments (e.g., Fugl-Meyer) and simple engineering metrics (e.g., goal-directed performance). Our overall approach is to statistically identify the range of volitional movement capabilities, and then apply a robot-applied force vector field intervention that encourages under-expressed movements. We investigated whether explorative training with such customized force fields would improve stroke survivors’ (n = 11) movement patterns in comparison to a control group that trained without forces (n = 11). Force and control groups increased Fugl-Meyer UE scores (average of 1.0 and 1.1, respectively), which is not considered clinically meaningful. Interestingly, participants from both groups demonstrated dramatic increases in their range of velocity during exploration following only six days of training (average increase of 166.4% and 153.7% for the Force and Control group, respectively). While both groups showed evidence of improvement, we also found evidence that customized forces affected learning in a systematic way. When customized forces were active, we observed broader distributions of velocity that were not present in the controls. Second, we found that these changes led to specific changes in unassisted motion. In addition, while the shape of movement distributions changed significantly for both groups, detailed analysis of the velocity distributions revealed that customized forces promoted a greater proportion of favorable changes. Taken together, these results provide encouraging evidence that patient-specific force fields based on individuals’ movement statistics can be used to create new movement patterns and shape them in a customized manner. To the best of our knowledge, this paper is the first to directly link engineering assessments of stroke survivors’ exploration movement behaviors to the design of customized robot therapy.


I. INTRODUCTION
T HE thoughtful application of robotics to stroke rehabilitation could offer powerful tools to complement traditional approaches in therapy. The success of such strategies, however, demands that the design of therapy account for the wide variation in motor impairments that exist across stroke survivors. Clinical assessments (e.g. Fugl-Meyer [1], Wolf Motor Function Test [2], Action Arm Reaching Test [3]) currently in use already recognize the differences in the level and types of impairment in stroke patients; including, abnormal muscle synergies [4], [5], muscle weakness [6], [7], spasticity [8]. Some assessments offer guidance to individualize treatment plans, while others are more reliable tools for tracking patients' recovery in response to therapy [9]- [11]. Of course, such clinical assessments are time-consuming and rely on subjective scoring that suffers from inter-rater variation and repeatability issues. One key emerging trend in therapy is the characterization of individual motor capabilities through the use of robotic devices, which afford high fidelity spatial and temporal recording of objective measures in a wide variety of tasks [12]- [16]. However, it has not been obvious how such robot-based assessments can reflect a patient's clinically relevant capabilities, or how such information can inform the design of therapy.
While experience and judgment guide the assertions of clinicians in diagnosing patients, robotic tools offer the unrelenting and dispassionate facility to organize vast amounts of data. Current strategies for high-resolution automated assessment of upper-limb motor function have been limited to simple engineering metrics related to patients' performance on specific tasks (e.g. reaching accuracy) [13], [15]- [17]. Recently, Huang et al. [18] demonstrated that statistical distributions of movement constructed from kinematic data during a selfdirected motor exploration task provide a probabilistic view of stroke survivors' individual movement tendencies. Interestingly, Huang et al. showed that each stroke survivor's patterns were distinct, reflecting individualized impairments, while neurally-intact individuals' patterns were similar to each other. We assert that free exploration encourages the full expression of movement and also provides a comprehensive description of an individual's motor deficits. Moreover, this unique feature of movement distributions provides a basis for the design of therapy that is customized to each individual.
Beyond characterization of motor deficits, effective robot therapy relies on key decisions about the form of intervention. One critical aspect relates to the amount of assistance the robot should provide to a patient. Therapy that encourages patient-mediated motions have shown to have better functional outcomes than guidance-based strategies in which the robot moves the limb [19], [20]. Treatment that promotes affected limb action, such as constraint-induced therapy that allows only motion of the affected limb, has been shown to strengthen muscle activity and promote neural growth [21]- [25]. Other strategies demonstrate assistance as needed or gradual assistance based on real-time measures of performance [26]- [28]. These strategies supply forces sufficient to overcome existing barriers or deficits while still requiring effort and control from the patient. However, one challenge with these approaches is that assistance is generally applied to specific movement types and goals (e.g. reaching to a target) [27], [29], [30] or motor impairments (e.g. muscle weakness). These approaches fail to account for the wide variety of movements in daily life. Motor exploration, on the other hand, enables practice over a broader range of movements. Broader exploratory movement patterns could serve as the foundation to improve functional skills. The ability of robots to provide forces as a function of movement makes them a potential candidate to reshape patients' movement patterns.
Our approach employed a robotic device to both characterize and retrain the unique movement patterns of stroke survivors. We first characterized individuals' typical distributions of movement from a motor exploratory task. A model of these distributions was incorporated into the design of patient-specific force fields that push their hands toward their less frequent motions. In this study, stroke survivors trained with customized forces, while a control group trained without forces. To assess changes in motor capability, we evaluated clinical measures as well as engineering metrics describing the range of motion during motor exploration. We previously presented a portion of this study which revealed evidence of increased velocity range [32]. Here, with a fully powered cohort, we further investigated how resulting movement patterns are related to the design of customized forces. Our findings demonstrate preliminary evidence of patient-specific force therapy reshaping the exploratory movement patterns of stroke survivors.

A. Experiment Participants
Twenty-five stroke survivors participated in this study (Table I). All participants were screened prior to participation by a physical therapist ("rater"). Two participants did not pass the screening criteria and one participant dropped out of the study due to shoulder discomfort during force training. We excluded the data from these participants in our analysis. The main inclusion criteria were 1) chronic stroke (8+ months post-stroke) 2) hemiparesis with moderate to severe arm impairment measured by the upper extremity portion of the Fugl-Meyer Assessment (FMA-UE score of 15-50, [1]) and 3) primary cortex involvement. The exclusion criteria included 1) severe sensory deficits in the limb 2) severe spasticity (Modified Ashworth of 4 preventing movement [31]) 3) aphasia, cognitive impairment or visual deficits that would influence their ability to perform the experiment tasks and 4) Botox injection in the past 4 months. Individuals gave informed consent in accordance with the Northwestern University Institutional Review Board (IRB) and were paid for their participation. Recruitment of stroke participants for this study was primarily through an institutional database that stroke survivors consent to join and be contacted regarding studies held at the Rehabilitation Institute of Chicago. This study is registered with the U.S. National Institutes of Health's clinical study database, ClinicalTrials.gov (Identifier: NCT02570256).
We planned our experiment as a randomized, doubleblinded, parallel group, two-arm design. We randomly assigned participants to either the Force or Control groups, each having eleven participants. We chose to power this study based on group differences in our primary outcome measure, changes in FMA-UE scores. Power analysis for a two-sample t-test included an alpha of 0.05, a power of 0.80, and an effect size of 3.5. Note, our recruitment procedures did not prevent the participants with unstable baseline clinical measurements completing training. However, we removed the data of these participants from the main analysis (see section F.). We employed a Block randomization with two participants per block using the initial assessment of FMA-UE scores. A third party researcher other than the clinical rater and the experimenter assigned participants to the groups by flipping a coin. The experimenter did not explicitly state to the participants which group they were assigned to prior to the first session. However, details regarding training and the potential risks involved were stated upon receiving consent. The clinical rater was not present during training sessions and was blind to group assignments until the conclusion of the study. Two different raters performed the clinical assessments during the course of the study. The same rater evaluated a given participant for each of their four clinical evaluation sessions (see Table I).

B. Apparatus
Participants performed planar motor tasks using a planar robotic device (manipulandum) and a custom video display system (Fig. 1A), presented previously [32], [33]. The robotic device has two degrees-of-freedom allowing participants to move within the transverse plane. It is equipped two encoders at each of its joints to record the position and velocity of its end-effector. A force sensor attached to the end-effector measures the human-robot interaction. At the base of the manipulandum resides two torque motors capable of generating programmable forces. Participants viewed down into a nontransparent mirror which overlaid their arm. Their hand was not visible and arm was partially covered. We provided real-time feedback of the robot's end-effector position (green cursor) which overlaid directly on top of the   participants' hand. Visual feedback also included instructions and measures of performance specific to each motor task participants performed.
Participants operated the robot's end-effector through a wrist brace attached to a revolute joint which allowed them to focus training on forearm and upper arm coordination. Participants also rested their forearm on an arm support which provided gravity assistance. Participants were situated with respect to the robot such that their shoulder lined up with the center of the experimental workspace (0.6 m × 0.4 m). The workspace boundaries (white outline) were visible to the participants. Participants were able to comfortably reach the bottom edge of the workspace. Due to constraints of the experimental set-up, two participants required modifications to the workspace area. For these particular circumstances, the bottom boundary of the workspace was shifted away from the participants so that their body did not overlap with the workspace (see Fig. 1A).
The robot control and instrumentation was mediated with a Simulink-based XPC Target computer, with a basic rate of 1kHz. Data was collected at 200 Hz and filtered using a 5 th order Butterworth low pass filter with a 12 Hz cutoff. The robot produced endpoint forces through the two torque motors, and the controller compensated for inertial effects of the robot arm during all experiment phases.

C. Experiment Protocol
Each participant completed nine sessions across five weeks (Fig. 1B). The first session (Baseline 1) and the second session (Baseline 2) served as initial evaluations and were separated by two weeks to establish baseline. Each evaluation included a clinical assessment followed by a performance assessment (described in detail below). We also evaluated participants two to three days (session 8; Post Evaluation) and six to eight days (session 9; Follow-up Evaluation) following the final training session. Each participant trained three days per week for two weeks (sessions 2-7). At the beginning of each training session, participants completed the performance assessment. Training began on the same day as the Baseline 2 evaluation (session 2).
1) Clinical Assessment: At the beginning of each evaluation session, a physical therapist administered a clinical assessment of participants' sensorimotor impairments of their affected arm. The clinical assessment included the FMA-UE [1], Action Research Arm Test (ARAT) [34], Modified Ashworth Scale (MAS) [31], Chedoke McMaster Stroke Assessment-Arm (CMSA-A) [35], and elbow range of motion (ROM).
2) Performance Assessment: For each session, participants completed three separate tasks using the robotic device: a) Goal-directed reaching task: Each session's performance assessment started with a goal-directed reaching task ( Fig. 2A). The reaching task tested participants' ability to make straight-line movements to circular visual targets (blue, 0.1 cm radius). Starting from a center target position, participants attempted to move a visual cursor (green, 0.05 cm radius) in a straight-line to a target in one of five outward target directions (0°, 45°, 90°, 135°or 180°relative to the positive x direction) located 15cm from the center target. Each outward movement was accompanied with a corresponding inward movement back to the center target (180°, −135°, −90°, −45°or 0°relative to the positive x direction). Participants were instructed to hold the cursor within each target for 0.5s. Participants attempted three movements to each target direction (i.e. 30 trials). The target locations were presented in block order and the order of targets remained the same across sessions. The center target was located anterior to the participants' shoulder; however, the distance from the shoulder to the hand varied depending on participants' arm length and range of motion. Some participants were unable to reach all the targets. If a movement attempt lasted longer than 8s, the experimental software advanced to the next target location. Following each reaching movement, participants received visual feedback on movement time. We defined movement time as the time from movement onset (speed > 0.04 m/s) to the time the cursor reached the target. Determination of movement onset was derived from previous experimental measurements of signal noise of the robot encoders when the robot handle is at rest. Applying a speed threshold of 0.04 m/s reasonably separates user-intended movement from resting noise. Movement times within a predetermined range of 400 and 750ms resulted in the appearance of the text "Good" in green color [36]. The appearance of "Too Slow" in blue text and "Too fast" in red text indicated movement times slower than 750 ms and faster than 400 ms, respectively. Participants had difficulty achieving the task constraints on movement time. Thus, we provided additional encouragement by instructing participants to move as fast and accurate as possible to each target. b) Goal-directed circular movement task: Each performance assessment also included a circular movement task ( Fig. 2A). The circular movement task tested participants' ability to coordinate movement in a cyclical fashion. Participants attempted to make repetitive circles around a visual circular track (blue dotted-line, 10 cm radius). Starting from a target located at the top of the circular track, we instructed participants to move as fast and accurate as possible around the track until the track disappeared. The disappearance of the track marked the end of the trial. The track disappeared after the robot's endpoint traveled a total distance equivalent to four times the circular track's circumference. Participants attempted three movements in the clockwise and counterclockwise directions (six trials). Similar to the reaching task, we provided visual feedback regarding movement time. Movement time was defined as the time from movement onset to the time the cursor traveled the total distance specified. Participants attempted to achieve movement times within a predetermined range of 3-6s.
c) Movement exploration: The primary portion of the performance assessment included a self-directed motor exploration task. For this task, participants were instructed to move the robot handle to all reachable positions within the robot workspace, at various speeds and movement directions. We also encouraged participants to avoid repeating the same movements continuously. Participants were undisturbed by the robot while performing the task. We informed participants they could rest at any time throughout the experiment. Each motor exploration trial (six trials for 12 minutes total) ended after two cumulative minutes of movement within the workspace. Movement speed below a threshold of 0.04 m/s was considered resting or no movement and the time points did not count towards the total movement time. We previously determined that 12 minutes of motor exploration is a sufficient amount of data to accurately characterize a stroke survivor's motor behavior during the same task [37]. Upon completing a trial, we provided participants with Post-Trial feedback related to their motor exploration performance (see section D.).
3) Training: For each training session (sessions 2-7), participants completed an additional two blocks of eight, two-minute motor exploration trials (32 minutes in total) separated by a rest period (1-3 minutes). Participants performed the 32 minutes of motor exploration training either without forces (Control group) or within a customized force field (Force group) for all training sessions (see section E.) We informed participants that they could rest at any time during training. We also provided participants with Post-Trial feedback after each trial (see section D.)

D. Post-Trial Feedback on Motor Exploration
At the end of each two-minute motor exploration trial within the performance assessments and following each training session, participants received a score measuring the randomness of their movements, presented previously [32]. We used a heuristic measure of randomness to encourage participants to express more variety in their movement patterns. The score was calculated by first dividing the experimental workspace into an 8 × 6 grid of two-dimensional (2-D) velocity-based histograms. Each 2-D histogram contained 25 bins in a 5 bin × 5 bin arrangement. Bin counts of individual histograms were based on the velocity of each data point located within the respective position of the workspace. Each histogram ranged from −1.25 to 1.25 m/s along both the x and y axes (lateral and fore-aft axes relative to the body) with each bin having a height and width of 0.5 m/s. For two minutes of motor exploration data (24000 data points), a completely uniform space (i.e. each bin having the same number of counts) equals 20 counts per bin. This was the maximum number of counts each bin could accumulate. The randomness score was determined by dividing the total number of counts across each bin by the total number of data points, displayed as a percentage. Following each completed trial for each participant, we displayed on the screen both their "Current" score (score from the most recent trial) and "Best" score (highest score across all trials within a given session). We explained to the participants that the scores reflected how well they varied their movements during the task and we encouraged them to attempt to achieve the highest score possible (i.e. 100 percent).

E. Design of Vector Field
The exploration portion of the performance assessment (Characterization) during each training session served as a basis for the design of the customized force field used within each training session. More specifically, we first extracted the 2-D velocity data accumulated across the six trials of motor exploration during characterization (12 minutes of data in total). A 2-D histogram of velocity data offers a detailed view of how participants' movement patterns varied during motor exploration (Fig. 2B, Characterization). We express histograms as probability distributions by dividing each bin by the sum of the number of data tabulated in each histogram. A typical movement distribution from a stroke survivor exhibits areas (i.e. bins) of higher probability (red) and lower probability (blue). Fig. 2B shows a representative movement distribution constructed from 12 minutes of velocity data during characterization within a single session. Note, to visualize the probability distributions, we presented velocity histograms with 40 x 40 equally sized bins, scaled according to each participants' maximum (defined as the 99 th percentile) absolute velocity during motor exploration across all sessions (see Fig. 2).
We then fit the 2-D velocity data with a weighted sum of multivariate Gaussian-normal components according to maximum likelihood estimates (using the 'gmdistribution.fit' function in Matlab 2013): For the case of 2-D velocity, k = 2 and x1 and x2 represent velocity in the x and y directions, respectively. Each j-th component is associated with a covariance matrix, S, and a center, μ. It has previously been shown that smoothing of velocity data using a multivariate Gaussian kernel with five components accurately describes the complexity of stroke survivor's velocity distributions [38]. When participants performed the motor exploration task, a high frequency of data accumulated near zero velocity during user-intended periods of rest and changes in movement direction. Thus, prior to fitting the data, we removed data with a speed below a threshold of 0.04 m/s. Each two-minute motor exploration trial contained 24000 data points. An example of a Gaussian distribution obtained from the model fit of 2-D velocity data during characterization (shown in Fig. 2B) is shown in Fig. 2C (colored contour lines).
Computing the gradient of (1) results in a velocitydependent continuous function whose output are vectors that represent the slope along the 2-D Gaussian distribution. In principle, the direction of the vectors point from higher probabilities towards lower probabilities of the distribution. An example of a vector field derived from calculating the gradient of the multivariate Gaussian distribution is shown in Fig. 2C (blue arrows). The vector field represents the direction and relative magnitude of force applied during motor exploration training. The applied force was updated continuously based on the current velocity of the robot's endpoint while participants performed motor exploration. An example probability distribution of velocity data during motor exploration training within a vector field (shown in Fig. 2C) is shown in Fig. 2D (within training effect). The probability distribution in Fig. 2D was constructed from 32 minutes of velocity data during motor exploration training. The magnitude of the applied force was determined by 1) normalizing the current vector magnitude by the 80th percentile of the vector magnitudes calculated across the velocity data accumulated during characterization and 2) applying a gain equal to 2% of the participant's body weight (i.e. the approximate weight of the arm). We developed this heuristic normalization technique during pilot testing of the vector field. It accounts for differences in the vector magnitudes and differences in participants' arm impedances. For safety, the applied forces smoothly decreased to zero magnitude outside the workspace boundaries.
F. Analysis 1) Clinical Outcomes: Our primary clinical outcome measure to determine the therapeutic benefit of motor exploration training on overall arm function was changes in clinical FMA-UE scores. We compared FMA-UE scores assessed for each of the evaluation sessions (see Fig. 1B). We summarize these results in terms of the change in FMA-UE scores relative to the average score between Baseline 1 (session 1) and Baseline 2 (session 2) evaluations. Statistical differences in FMA-UE were analyzed using a 2 (session: Average Baseline, Post) x 2 (training group: force, control) repeated measures Analysis of Variance (ANOVA). We considered statistical differences significant at α of 0.05.
Prior to analysis, we removed data from participants who demonstrated unstable FMA-UE scores between Baseline 1 and Baseline 2 sessions. To determine stability, we calculated the minimal detectable change needed to exceed measurement error (i.e. the 95% confidence interval of the standard deviation for Baseline 1 and Baseline 2 scores) [39]. Among the participants, two Control group participants and one Force group participant showed a change in baseline measurements greater than the calculated minimal detectable change threshold (2.5 points). We removed the data of these three participants for all subsequent metrics.
2) Motor Exploration Performance: We first evaluated whether participants' movement patterns improved following training, in terms of the range of velocities spanned during motor exploration (characterization) for each session. The metric we used, 50 th percentile coverage, represents the estimated area of participants' median movement tendencies in the velocity domain. We first calculated the 50 th percentile contour of 2-D velocity data, and then calculated the area (m 2 /s 2 ) within the boundary formed by this contour. The boundary was formed by connecting points represented by the median (i.e. 50 th percentile) speed within 64 equally spaced bins radially aligned within the range of 0-2π (see Fig. 2B for a representative 50 th percentile contour of velocity data, black outline). We summarize these results in terms of the percent change in 50 th percentile coverage relative to the Baseline 2 evaluation (session 2). Statistical differences in coverage were analyzed using a 2 (session: Baseline 2, Post) x 2 (training group: force, control) repeated measures Analysis of Variance (ANOVA).
While velocity coverage revealed overall changes in terms of the range of movements, we also wished to quantify changes in the patterns of movement. We first constructed 2-D probability distributions of velocity data from motor exploration during characterization and training within each session. This analysis featured probability distributions with 100x100 equally sized bins, which were scaled to a common maximum range across all participants (±2 m/s). The binby-bin difference between two given probability distributions represented the change in probability. Fig. 2E shows an example of a contrast plot (for visualization purposes, the contrast plot is presented with 40×40 bins). Positive changes (red) and negative changes (blue) in probability within each bin indicate an increase and decrease in data, respectively. To quantify the difference between two probability distributions, we defined the contrast score as the total sum of the absolute difference in probability between corresponding bins.
We present some simple metrics to characterize learning as well as the impact of training forces. We first determined the how training affects subsequent unassisted conditions (cumulative transfer effect). We calculated the contrast score between the probability distributions of velocity data from Baseline 2 characterization and that of each successive session (sessions 3-9). To determine group differences in the cumulative transfer effect, we compared scores calculated between Baseline 2 and Post evaluations. Aside from measures of learning, we also wished to characterize differences in the direct experience of training between groups. To do so, we computed a contrast score between the probability distributions of velocity from training data and characterization data within each session (within training effect). We compared the mean within training effect contrast scores between groups. Statistical differences were analyzed using a Student's t-test (α = 0.05).
We devised a novel analysis to test whether the robot mediated training promoted changes in learning that corresponded to the design of customized forces. Our approach was to examine whether the changes in each participant's movement behaviors across training (cumulative transfer effect) were similar to the changes within training (within training effect). We computed the Pearson's correlation between the Baseline 2 and Post contrast (cumulative transfer effect contrast) and the average characterization and training contrasts within each training sessions (average within training effect contrast). As with the contrast score, we employed a common maximum range and bin density across all participants (±2 m/s, 100×100 bins). Statistical differences between the training groups were analyzed using a Student's t-test (α = 0.05).
3) Goal-Directed Performance: To determine the effect of training on goal-directed task performance, we compared changes in movement error, peak speed and duration between Baseline 2 (session 2) and Post (session 8). For the reaching task, our primary error metric was the maximum perpendicular distance along the movement trajectory (from movement onset (speed > 0.04 m/s) to when the cursor reached the target) with respect to the ideal straight-line path to the target. Besides this primary metric, we performed supplementary analysis of the path length ratio defined as the total distance traveled for each movement normalized with respect to the distance between targets (15cm). We also defined peak speed as the maximum speed along the trajectory and duration as the total time from movement onset to when the cursor reached the target. Statistical differences were analyzed using a 2 (session: Baseline 2, Post) × 2 (training group: force, control) × 10 (movement directions) repeated measures Analysis of Variance (ANOVA).
For the circular movement task, we compared changes in movement error, average speed and duration. Our primary measure of error was the mean radial deviation relative to a reference track defined by the mean radius of the movement trajectory. We first measured the mean center of each movement trajectory (from movement onset to when the cursor traveled a total distance equivalent to four times the circumference of the circular track). Then, we computed the distance between each point along the movement trajectory and the mean center. The average of these distances served as the radius of the circular reference track. We then calculated the mean distance between each point along the movement trajectory and the reference track. Besides our main metric for circular movement, we computed the average speed in terms of mean speed along the movement trajectory and movement duration as the total time from movement onset to when the cursor traveled the total distance equivalent to four times the circular track's circumference. Statistical differences were analyzed using a 2 (session: Baseline 2, Post) × 2 (training group: force, control) × 2 (movement direction: clockwise and counterclockwise) repeated measures Analysis of Variance (ANOVA). We considered statistical differences significant at α of 0.05.

A. Clinical Outcomes
Our pre-declared primary outcome measure, change in FMA-UE scores, showed that both training groups improved with training (Post evaluation) compared to the average  Table I; data points are staggered horizontally to avoid overlap. Vertical bars represent the mean and 95% confidence interval (gray, Force; black, Control). between baseline evaluations (Fig. 3); however, we failed to detect a significant difference between training groups. After removing the data from three participants with unstable baseline FMA-UE scores, the mean change and 95% confidence interval (CI) in FMA-UE scores were 1.1 (CI: 0.0, 2.2) and 1.0 (CI: −0.3, 2.3) for the Force and Control group, respectively (session: F(1, 17) = 7.9, p = 0.01; training group: F(1, 17) = 0.0005, p = 0.9). We also found that increases in FMA-UE scores persisted for one week (sixeight days) following training for five of the Force participants and five of the Control participants (Fig. 3). The mean change in FMA-UE scores from the Average Baseline evaluation to the Follow-up evaluation were 1.0 (CI: −0.4, 2.4) and 1.8 (CI: 0.7, 2.9) for the Force and Control group, respectively (session: F(1, 17) = 8.8, p = 0.009; training group: F(1, 17) = 0.0, p = 0.99). We also evaluated changes in our secondary clinical outcomes; including, ARAT, CMSA-A, MAS and elbow ROM (see Table I for individual participant data). We summarized these results and provided additional analysis that considers all participant data (see Table II Supplementary Statistical Analysis).

B. Motor Exploration Performance
Our analysis of motor exploration first examined the extent to which motor capabilities improved, and then how movement probabilities were redistributed, and finally whether such changes could be attributed to training conditions. We found that the training groups demonstrated similar increases in velocity coverage (see Fig. 4A, session: F(1, 17) = 24.5, p = 0.0001). However, we failed to detect a significant   Fig. 7).  Table I; data points are staggered horizontally to avoid overlap. Vertical bars represent group (gray, Force; black, Control) mean and 95% confidence interval. The asterisk represents significance between training groups (α < 0.05).
Besides changes in velocity coverage, we observed a gradual increase in the degree of change in movement patterns, as indicated by the transfer effect contrast scores across sessions for both training groups (see Fig. 4B). Each participant's cumulative transfer effect contrast plot (see Fig. 7) corresponds to the contrast score between Baseline 2 and Post characterizations. We failed to detect a significant difference between groups; t(17) = 0.5, p = 0.63. Note, to visualize the probability distributions, we presented velocity histograms with 40 × 40 equally sized bins, scaled according to each participants' maximum (defined as the 99 th percentile) absolute velocity during motor exploration across all sessions (see Figs. 5and 7).
The apparent similarities between groups in these changes, however, contrast starkly to the large differences in training conditions. We examined how movement behaviors differed from the initial characterization (null field) to later training within the same session. As expected, Force participants' movement behaviors were drastically altered when training in the presence of forces. Fig. 5A (top) shows a typical Force participant's contrast plots between training and characterization probability distributions within each training session.
In contrast, Control participants displayed movement behaviors during training that were similar to that of the beginning of the session (Fig. 5A, bottom). Group means across training sessions revealed significant differences in the within training effect contrast scores; t(17) = 8.34, p < 0.05 (Fig. 5B). Fig. 7  Beyond the general changes described above, we performed new analyses that were supplementary to our planned metrics to better reveal specific differences in learning due to forces. While the contrast score provides critical information about the degree of movement redistribution, it does not indicate whether such changes necessarily improved motor exploration. We created a metric, the favorability score, which summarized the way that our intervention may have reversed a person's initial deficits, either by decreasing over-expressed or increasing under-expressed velocities. We examined to what extent the observed velocity states for each participant exhibited favorable increases or decreases in probability. For each session, we tabulated two-dimensional velocity histograms with a common maximum range and high bin density (+/−2 m/s, 100×100 bins). We defined a distribution midpoint as half the peak probability for the reference distribution (see Fig 6A). Using the initial baseline distribution on session 2 as the reference point for each participant, we computed favorable changes as the sum of all increases of each velocity state for which the initial distribution was below the midpoint, as well as all decreases of each velocity state for which the initial distribution was above the midpoint. All other changes in the distribution were then evaluated as unfavorable changes. As a final metric, we evaluated the favorable changes as a proportion of the total change for each participant. We observed similar changes across multiple sessions, and similar session dependence between groups (repeated measures ANOVA; session: F(5, 85) = 1.38, p = 0.24; session x group: F(5, 85) = 1.19, p = 0.32). Considering the sessions as a whole, this metric of favorable changes was actually greater for the Force group compared to the Control group (Fig. 6B, average of sessions 3-8, = 0.10 (CI: 0.035, 0.16), t(17) = 3.2, p = 0.005).
While the favorability score indicated possible advantages from training with forces, we also devised a novel analysis to test whether changes in movement behavior were consistent with the design of the customized environments. We observed that changes in probability distributions within training were similar to the changes between Baseline 2 and Post (Fig. 7, cumulative transfer effect contrast compared to average within training contrast). Increases and decreases in probability from Baseline 2 and Post correlated with their respective probabilities for training, as depicted by red and blue shaded areas, respectively. In other words, velocities that increased their representation each training session also increased by the end of training (Post), and velocities that decreased their representation each training session also decreased by the end of training. This supports the idea that Force participants' preserved the changes in movement behaviors that were trained using unique training forces. The mean of the Pearson's Correlation for the Force and Control groups were 0.60 (CI: 0.41, 0.79) and 0.36 (CI: 0.09, 0.62), respectively. Interestingly, we failed to detect significant differences between the groups (t(17) = 1.76, p = 0.1), which suggests that short term changes in movement distribution are predictive of longer term learning. Fig. 7 shows each participant's cumulative transfer effect contrast (Baseline 2-to-Post) and average within training contrast with the corresponding correlation. Supplementary analysis of motor exploration performance, considering all participant data, yielded similar results (see Table III Supplementary Statistical Analysis).

C. Goal-Directed Performance
We also evaluated the effect of motor exploration training on participants' performance during the goal-directed movement tasks that were not trained within each session. Both training groups reduced movement error in the reaching task from Baseline 2 evaluation to Post evaluation; however, we failed to detect significant differences between groups. Surprisingly, for the circular movements, both training groups increased movement error following training. For the reaching task, the mean difference in maximum perpendicular distance for the Force and Control group was −0.28 (CI: −0.59, 0.4) cm and −0.05 (CI: −0.48, 0.38) cm, respectively (session:  Beside the planned analyses above, we performed a posthoc analysis of movement duration and speed during the goaldirected tasks and found changes consistent with the increases in velocity coverage observed during motor exploration. For each task, both training groups significantly decreased movement duration and increased speed; however, we failed to detect significant differences between groups. For the reaching task, the mean difference in movement duration for the Force and Control group was  Table IV Supplementary Statistical Analysis).

IV. DISCUSSION
This study investigated an innovative approach to robottherapy in which the characterization of participants' motor exploration directly informed the mathematical structure of customized robotic training environments. To the best of our knowledge, this is the first clinical study to apply robot guided characterization of stroke survivors' exploratory motor behaviors towards the design of individually customized therapy. Disappointingly, our results from clinical assessments did not indicate differences between the novel treatment and controls. While changes in FMA-UE scores were modest for both groups, stroke survivors exhibited marked increases in velocity coverage following only two-weeks of training. Beyond measures of overall improvement, we were very interested in whether the influence of interactive forces could be detected in learned movement patterns. Consequently, we devised a novel analysis that revealed significant correlations between induced training behaviors and new patterns of unassisted movement. These results provide preliminary evidence that new movement behaviors can be learned from training with forces that target movement deficits.
Our pre-declared primary clinical result, change in Fugl-Meyer scores, showed that both groups benefitted from training; however, such levels of improvement would not be viewed as clinically relevant [9], [40], [41]. Our clinical results  [4], [5], [44]. However, considering that our intervention lasted only two weeks, compared to 6+ weeks in other interventions, it may not be surprising that our effects were only modest. One benefit from our approach may be that free exploration training with velocity feedback is at higher intensities. Interestingly, some participants displayed even greater improvement upon a follow-up evaluation after training, which could indicate that new motor capabilities required some time to incorporate into activities of daily living. It is also possible that the inactivity between the final day of training and later evaluation allowed patients some needed rest. It is also possible that the repeated exposures to clinical evaluations had a training effect of "teaching to the test". Anecdotally, many participants stated that our motor exploration paradigm appeared to relax the muscles of their affected arm. Such action could have stretched muscles [45] or reduced reflex gains [46] due to the reduced mechanical impedance.
Overall, participants expressed that the motor exploration task was somewhat tiring and not particularly engaging. However, the participants also reported that the feedback score provided incentive to be creative with expressing movement variety. Future iterations of customized force design might target more degrees of freedom, which would have a greater impact on functional skills [47]. However, it is also possible that the robot-assisted training promoted learning that is not evident from clinical assessments [48].
Our analysis of the changes in motor exploration revealed evidence that participants increased movement capability. As a simple metric of the range of motion, we observed that participants from both groups increased their velocity limits. It is worth emphasizing that due to the nature of characteristic movement behaviors during exploration, the vector fields resulting from velocity data generally tend to push participants' movements towards higher velocities, in a manner similar to destabilizing forces from our previous work [49].
It is possible that improved coverage indicates that participants retained some of the movement patterns acquired through vector field training into their exploration practice evaluated without forces.
Beyond the overall range of motion, we were also interested in measuring the degree to which the probabilities of observed movements were redistributed. Interestingly, our findings showed that training groups exhibited similar amounts of change in movement distribution. This similarity demonstrates that motor exploration practice can induce change even in the absence of external forces. Despite considerable differences between training groups' average within training effect, both groups displayed a gradual change from their original movement behaviors across sessions. This result is consistent with our previous study that compared distributions across multiple days without any intervention [18]. Hence, beyond the use of customized forces, there may be other forms of training intervention, with visual feedback or even verbal instructions, which may prove useful in inducing desirable changes to movement distribution.
Our analysis of the proportion of favorable change in movement distribution (see Fig. 6b) provides evidence, however, that training forces can positively impact on how stroke survivors express movement. While both groups exhibited general improvement in the range of motor exploration, our supplementary analysis indicated group differences in how the initial trends of overly low or high probability velocities changed due to training (see Fig. 6a). Learned non-use in stroke survivors represents an extreme case of how a lack of motor expression can be reinforced [49], [50]. In addition, abnormal coordination or involvement of additional degrees of freedom can occur. For example, compensatory trunk motion is typical in reaching [52], while circumduction at the hip occurs due to stiff knee gate [53]. In a rat model of stroke recovery, researchers suggest that "inappropriate gestures may represent motor habits that substitute for, and compete with, successful movements" [54]. It is worth emphasizing that training for the Control group was self-mediated except for the knowledge of results presented at the end of each trial block. Consequently, without more specific guidance, reinforcement of abnormal movement distributions was possible. The fact that participants of the Force group also exhibited both favorable and unfavorable changes indicates that further refinement is needed in the design of customized forces. The crucial lesson here, however, is that forces evidently provided an additional pressure on motor adaptation that evidently helped to reverse the deficits in movement distributions found prior to training.
We devised an analysis to answer a fundamental question about robotic intervention: can mathematical structure of force field customization be detected in learned motor behaviors? Because the customized robot training produced such dramatic changes in movement distribution during the presentation of forces, it was in many ways surprising that our experiment groups exhibited such similar degrees of improvement. Our supplementary analysis of motor exploration, however, revealed analogous changes between training and new behaviors (see correlation analysis in Motor Exploration Performance section). These correlation analyses suggest customized forces caused specific and persistent changes to movement behaviors. Note that the Force participants experienced drastically altered movement distributions due to force interactions. Yet despite such effects, some of these individuals still demonstrated high correlations, indicating some retention of the movement behaviors learned during training. Interestingly, we also observed similar correlations in the Control group.
The key difference, however, is that the movement distributions during the training phase of the Control group was self-mediated and not dictated by customized robot forces. It is perhaps unsurprising changes within day would in some way mirror changes in longer term learning. It is however remarkable that new patterns of movement persisted even when induced from externally applied forces. The learning of new exploratory behavior indicated here differs from typical adaptation to novel force and visual distortions [55]- [57] since participants were not given prescribed movement goals and hence did not rely on explicit error feedback. Instead, it is likely that the repeated exposure to motor exploration with interactive forces induced adaptation in terms of use-dependent learning [58]- [60]. Further development is needed for predictive models of how practice behaviors during intervention lead to changes in motor exploration capabilities.
We observed some movement behaviors during training that could indicate unintended consequences of our implementation of vector field training. Specifically, some participants exhibited rapid, repetitive motions in a curved path. While repetitive behavior can appear in stroke patients' distributions during un-assisted motor exploration, it was clearly evident in the distributions during interaction with forces. One possibility is that participants intentionally avoided forces by moving at relatively constant velocities outside their characteristic behavior since this is where force magnitudes were low. Alternatively, the destabilizing nature of vector fields may have constrained participants to repetitive behavior because the forces were continuously active. Such a scenario would have similarities to passively moving the limb [61], resulting in less active involvement-an essential component to recovery [62]. One potential limitation of our current protocol is that the task feedback did not penalize cyclic behavior. Instead of gradual adaptation, some participants exhibited substantial and sudden increases in coverage. This effect suggests changes in task comprehension, or in the strategy for how to work with interactive forces [63]. Future iterations of force fields could be improved by obtaining characterization data that more faithfully reflect participants' full range of capabilities, and by improved task instructions on the goals of motor exploration.
Our analysis of the changes in reaching and circular motion performance suggests that learned exploration behaviors might not immediately transfer to skill in goal-directed actions. Both treatment groups only showed a modest reduction of movement error on the goal-directed reaching task and an increase in movement error on the circular movement task following training. Our motor exploration task did not provide feedback of movement errors related to specific movement goals. It does, however, encourage participants' to practice upper-arm coordination over a wider range of movements, which has been shown to facilitate generalization to untrained movements [64]. Since increases in velocity coverage were a main component of the overall changes in movement distributions, it is possible that participants generalized the ability to move at higher speeds as opposed to the ability to minimize reaching errors. Thus, we further inspected whether analogous changes were present in their goal-directed movements. For both tasks, we observed an increase in peak speed and a decrease in the time to complete each movement. This result might suggest that participants retained increases in movement speed at the expense of decreased accuracy [65]. On the other hand, it is likely that any new motor exploration capabilities require time and experience to incorporate into activities of daily living.
Beyond the potential benefits of customized force fields for upper extremity rehabilitation, our approach could serve as a basis for a wide range of therapeutic applications. Statistical profiling of large data sets is an emerging trend, and analysis of distributions could be derived from a variety of domains relating to human behavior; including, electromyography, joint-space variables and electrocorticography. The framework we have provided here could be applied more generally to determine the optimal strategies to customize treatment.