Hands-Free Physical Human-Robot Interaction and Testing for Navigating a Virtual Ballbot

A hands-free (HF) lean-to-steer control concept that uses torso motions is demonstrated by navigating a virtual robotic mobility device based on a ball-based robotic (ballbot) wheelchair. A custom sensor system (i.e., Torso-dynamics Estimation System (TES)) was utilized to measure and convert the dynamics of the rider’s torso motions into commands to provide HF control of the robot. A simulation study was conducted to explore the efficacy of the HF controller compared to a traditional joystick (JS) controller, and whether there were differences in performance by manual wheelchair users (mWCUs), who may have reduced torso function, compared to able-bodied users (ABUs). Twenty test subjects (10 mWCUs +10 ABUs) used the subject-specific adjusted TES while wearing a virtual reality headset and were asked to navigate a virtual human rider on the ballbot through obstacle courses replicating seven indoor environment zones. Repeated measures MANOVA tests assessed performance metrics representing efficiency (i.e., number of collisions), effectiveness (i.e., completion time), comfort (i.e., NASA TLX scores), and robustness (i.e., index of performance). As expected, more challenging zones took longer to complete and resulted in more collisions. An interaction effect was observed such that ABUs had significantly more collisions using JS vs. HF control, while mWCUs had little difference with either interface. All subjects reported greater physical demand was needed for HF control than JS control; although, no users visibly showed or expressed fatigue or exhaustion when using HF control. In general, HF control performed as well as JS control, and mWC’s performed similarly to ABUs.


I. INTRODUCTION
The majority of wheelchair users use manual wheelchairs [1].Yet, over 70% of manual wheelchair users experience overuse upper extremity injuries [2].Furthermore, the user needs both hands to effectively propel the wheelchair, which compromises life experiences due to inability to carry objects or grasp the hand of a loved one during propulsion.Powered wheelchairs can be operated with one hand via a joystick.However, most users with sufficient upper limb function will not use powered wheelchairs due to their large size and weight, greater cost, runtime limitation, and difficulty transporting in personal vehicles.We are breaking the mold of the traditional wheelchair through exploration of a riding self-balancing ball-based robot (ballbot).Robot movement and speed are managed hands-free by leaning the torso in the desired direction; thus allowing the rider to use their hands for other life experiences.
Various interfaces for human-robot interaction (HRI) have been developed for navigational control of virtual or physical mobility devices using hands-free (HF) control [3]- [7].Selfbalancing devices (e.g., Segway, Onewheel) take advantage of the rider's dynamics (e.g., leaning) to offset the system's center of mass to propel or brake [7], [8].Others have introduced interfaces that used an inertial measurement unit (IMU) placed on the rider's shoulder [9] or pressure sensing rigid bar placed in front of the rider's torso to control powered wheelchairs for riders with trunk or shoulder mobility issues [6], [9].These interfaces, however, have drawbacks.While these hands-free interfaces are relevant to human locomotion (e.g., leaning of torso), these devices were not tested on users with disabilities such as manual wheelchair users (mWCUs).In addition, these interfaces did not offer adjustments of ride behavior to accommodate riders with different ride preferences.This is detrimental for mWCUs who have limited torso mobility and therefore require more sensitive ride behavior (e.g., higher acceleration for a given human input) [10], [11].Also, mWCUs display a wide range of torso functionalities due to diverse injury and medical history.Thus, adjustability of HRI using torso motions is essential to accommodate mWCUs.
In this paper, we introduce an interface that uses natural and customizable hands-free control generated by torso motions to maneuver a novel omnidirectional personal mobility

Hands-Free Physical Human-Robot Interaction and Testing for
Navigating a Virtual Ballbot* Seung Yun Song, Student Member, IEEE, Nadja Marin, Chenzhang Xiao, Ryu Okubo, Joao Ramos, Member, IEEE, and Elizabeth T. Hsiao-Wecksler, Member, IEEE S. Y Song, N. Marin, C. Xiao, J. Ramos, and E.T. Hsiao-Wecksler are with the Department of Mechanical Science and Engineering, and R. Okubo is with the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, IL 61801 USA (phone: 217-333-3415; email: ethw@illinois.edu).C. Xiao is now with Intuitive, CA 94086, USA.J. Ramos is also with Boston Dynamics, MA 02451, USA device, i.e., PURE (Personalized Unique Rolling Experience), (Figure 1).The drivetrain of PURE utilizes the concept of a ballbot, i.e., a balancing robot that rides on top of a spherical wheel or ball.Thus, PURE's system behaves like an inverted pendulum, so that the rider's body movements affect the system dynamics.The rider can maneuver PURE by 1) leaning their torso to the front or back to move forward or backward, 2) leaning their torso to the side to slide left or right, 3) twisting their torso to spin clockwise or counterclockwise, and 4) performing any combinations of the previously mentioned actions.Lean amplitude affects device speed.These torso motions were chosen as the input commands for the HF control since these motions are intuitive and relevant to natural human locomotion behavior.
We performed a simulation study to determine if HF control performed as well as joystick (JS) control, a common baseline interface, and if mWCUs performed as well as ablebodied users (ABUs).A virtual environment of realistic and challenging indoor environments was chosen as the test platform to verify this novel interface for hands-free navigation more safely and efficiently prior to testing with a physical device.

A. Subject Demographics
Twenty young adults (between 18-35 years old) were recruited (Table I).Their basic subject information, physical measurements, prior experiences with JS control, HF control, and virtual reality (VR), and preferred sensitivity settings were collected.The prior experiences with JS control included video gaming experiences using JS-based controllers (e.g., X-box controllers).The prior experiences with HF control included experiences with self-balancing devices (e.g., Segway, Onewheel, motorized skateboards, Wii Balance Board).Upper body center of mass (COM) height was estimated using the distance between the seat surface and subject's xyphoid process.A common inclusion criterion for both subject groups was ability to manipulate the JS using the fingers.
The ABUs had to 1) have no recent injury to their upper body within the past 3 months, 2) have no history of neck, arm, or back related surgery, disorder, and diseases, and 3) have full torso function (flexion/extension ± 50˚, lateral flexion ± 40˚, trunk twist ± 80˚).
The mWCUs had to 1) have at least minimal trunk control and be able to feel pressure to at least the level of the xyphoid process and 2) be an experienced mWCU (i.e., have used a manual WC daily for a minimum of a year, and for 50%-100% of waking hours).Our mWCU population used a WC for an average of 18.1 ± 9.9 years and 100% of waking hours.WC use was due to spinal-cord injury (T4 (N=1), T10 (N=3), T11 (N=1), T12 (N=1)) or congenital disorders (N=4).mWCUs had limited torso mobility due to one or more of the following: scoliosis (N=5), kyphosis (N=3), lordosis (N=4), and spinal fusion surgery (N=5).Most were wheelchair athletes and their classifications were collected to better assess their level of torso mobility (T-53 (N=3), T-54 (N=6) for WC track and field athletes [12] and IWBF (N=1) for international wheelchair basketball federation [13]).
Understanding the degree of torso mobility was essential to determine if the HF control can be used for mWCUs with various levels of disability.This study was approved by the University of Illinois Institutional Review Board (IRB #22552), and informed consent was received.

B. Physical Test Setup 1) Torso-dynamics Estimation System (TES)
Users typically prefer different ride sensitivities (i.e., PURE's velocity as a function of torso motion) depending on their torso mobility.Therefore, the signals quantifying torso mechanics were used as reference signals to the ballbot's controller.Thus, a custom sensor system (i.e., Torsodynamics Estimation System (TES)), consisting of a Force Sensing Seat (FSS) and an inertial measurement unit (IMU) was used to measure the dynamics of the rider's torso motions (Figure 1, 2) [14].
The FSS estimated the 2D center of pressures (COPs) of the seated user, such that  ,  were the COP along anterior/posterior and lateral/medial directions, respectively (Figure 2 (a)).The seat dimensions (i.e., depth, width, and dump angle) were adjusted for each subject's physique and preference [14].mWCUs were given the option to use their personal seat cushion and foot straps for comfort and stability.ABUs used a standard wheelchair cushion.An industrial grade IMU (VN-100, VectorNav, USA) was secured on the subject's manubrium using medical-grade double-sided adhesive tape to estimate the 3D torso angles ( ,  ,  ) of the subject (Figure 2). ,  represented the torso lean angles forward/backward and left/right, respectively, and  characterized the torso twist angle.

2) VR headset and Joystick
A VR headset (Oculus Quest 2, Facebook Technologies LLC, USA) and a 4-axis (3 potentiometer + 1 button) joystick (OM400B-M2, Omter Group Ltd., China) were used (Figure 1, 2(b)).The VR headset, connected to the computer, displayed the simulation of the ballbot.The headband and interpupillary distance were adjusted for each subject.The subjects controlled the joystick knob using their dominant hand while holding the base of the joystick with their nondominant hand.CoppeliaSim was chosen since others claimed CoppeliaSim performed relatively better compared to other simulators (e.g., Gazebo) in terms of real time factor, CPU efficiency, physics engine support, sensor accuracy, etc. [15].Also, CoppeliaSim enabled more realistic and computationally stable modeling of the omni-wheels, essential for properly modeling the dynamics of a ballbot.The virtual rider was modeled as two rigid bodies: upper and lower body (Figure 2(e)).The virtual upper and lower body mass properties ( ,  ) and the upper body center of mass (COM) height ( ℎ) reflected each subject's physical body measurements (i.e.,  = upper body mass (head + torso + arms),  =lower body mass (pelvis + legs + feet), ℎ = upper body COM height). and  were estimated using anthropometric equations based on subject gender and total mass [16].

C. Virtual Test Setup 1) Virtual Rider
The virtual rider's 3D torso movements in sagittal, frontal, and transverse planes were controlled using three proportional-gain position controllers that regulated the efforts of the three orthogonal virtual torso motors to ensure the virtual rider's torso angles reflected the physical rider's torso angles ( ,  , ).The mapping of the physical subject's torso angle and physical measurements to the virtual rider was essential for realistic replication of dynamics of the virtual ballbot since the rider's torso motions and mass properties influenced the riding behavior of the ballbot.

2) Virtual Ballbot
The virtual ballbot consisted of a cylinder representing the chassis with three evenly spaced omni-wheel and motor pairs balancing on a ball (Figure 2 (c, d)).The virtual motors were torque controlled, and the virtual omni-wheels were modeled as a collection of rollers with passive joints around the outer perimeter of the omni-wheel (Figure 2 (c, d)).The velocity control of the virtual ballbot was achieved by mapping the interface signals from the TES or JS.For HF control, the motion of the subject's COP measured by the FSS in x and y directions ( ,  ), and torso twist angle measured by the IMU ( ) were mapped to control ballbot velocity in the forward/backward direction ( ), left/right direction ( ), and rotational motion ( ), respectively (Figure 2(a)).For JS control, the rotations of the JS knob in three different axes ( ,  ,  ) were mapped to set the 2D linear and rotational velocities ( ,  ,  ) of the ballbot (Figure 2

(b)).
A Linear Quadratic Regulator (LQR) controller [17], [18] was used to dynamically stabilize the ballbot with the virtual rider and to control the velocity of the ballbot by tracking the states of the ballbot (Figure 2 (c)).The controller tracked angular positions and velocities of the chassis in the sagittal and frontal planes ( ,  ,  ,  ), angular velocities of the ball in the sagittal and frontal planes ( , ), and the yaw angular velocity of the chassis in the transverse plane ( ).Thus, seven states (  ,  ,  ,  ,  ,  ,  ) were used to stabilize and control the movements of the virtual ballbot.Three planar models of the ballbot were used for (d) Three evenly spaced omni-wheel and motor pairs drove the motion and direction of the ball.(e) The 3D torso angles and subject's physical measurements were mapped to the virtual environment to replicate realistic rider-ballbot dynamics.HRI commands for HF or JS control were preprocessed and used as input for the ballbot controller to control the omnidirectional velocity of the virtual ballbot.

558
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
controlling the movement in the sagittal, frontal, and transverse planes independently [19].The torques of the actuating wheels from the planar models were mapped to the torques of the three motors in 3D using Jacobian transformations.More details are in [17], [18].

3) Virtual Mapping of HRI interface signals
The signals from the interfaces were preprocessed and mapped as reference velocities of the ballbot  , see (1) (Figure 2).The first four elements of  were the states of the ballbot in the translational x and y directions.These were set to zero because the ballbot needs to be balanced dynamically.
Before being mapped as the ballbot's reference velocities  , the interface signals or  were preprocessed to accommodate for user sensitivity preference and provide stable reference signals for the ballbot controller (Figure 2).
where the  represented the  th data index and k-1 represented the previous reference signals from HF or JS.
First, the interface reference signal   was adjusted to match the subject's preference by multiplying a sensitivity factor (i.e.,   ,  ,  for HF control,   ,  ,  for JS control) to the reference signal.The values for  and  were adjusted for each subject during the VR training course.Higher sensitivities allowed navigation of the ballbot using smaller torso movements, which could be helpful for mWCUs with less torso mobility.
Second, the reference signal from the interface was preprocessed by a low pass filter, flooring function, and saturation function, see (2) and Figure 2.
A first order digital IIR low pass filter ( ) was added to smooth the adjusted interface reference signal and ballbot velocity reference signals. required three arguments: 1) reference signals adjusted to the subject's sensitivity factor (  •  , ), 2) previous ballbot states (  , ), 3) smoothing factor ( ).For our model, a heuristically-tuned smoothing factor of  0.85 was used.A flooring function ( ) that brought the reference signals to zero when below a certain value  0.05˚/, 0.05/, 0.05/ was used to minimize unwanted movement of the ballbot when input signals were close to zero.This was important to completely brake to a full stop for both interfaces.For the JS control, the knob would not always return to zero position when released due to the mechanical compliance of the springs, causing non-zero readings.For HF control, the subjects did not precisely move their torso back to their predefined neutral position (defined by averaging the COP data from the FSS for 5 s while the subject faced forward and sat comfortably upright), causing small non-zero COP readings.Thus, the flooring function also helped subjects to brake completely by removing these small non-zero readings and unwanted drifting of the ballbot.
Finally, a saturation function ( ) was added to prevent the ballbot from reaching excessively high speeds ( 10˚/, 2/, 2/ ).The processed interface signals ( ) were input as reference signals for the virtual ballbot controller to track, see (4).

4) Virtual Training and Test Course
In the VR scene, the subjects first went through a training course and then a test course to evaluate the performance when using the two control interfaces.
The virtual training and testing courses replicated realistic and challenging indoor environments that followed the Americans with Disabilities Act (ADA) standards [20] (Figure 3).Both courses contained seven different zones: wide hallway, medium hallway, narrow hallway, zones with moving obstacles, table zones, a bathroom stall, and slalom course.The wide (2.4 m), medium (1.8 m), and narrow (1.2 m) widths simulated a large public building (e.g., hospital) hallway width, average residential hallway width, and narrow residential hallway width, respectively.Each hallway zone contained three sub-zones (i.e., straight, left turn, and right turn).The lengths of straights and the turns were the same for all hallway widths.Zones with moving obstacles contained three virtual human figures walking at a slow (1.1 m/s), medium (1.4 m/s), and fast (1.7 m/s) pace.The table zones contained three tables (1.0 m 2.5 m) for the subject to navigate between the obstacles to test the lateral sliding omnidirectional capability of the rider and virtual ballbot.Within the table zones, there were three subzones with varying gap sizes between the tables: narrow (1.4 m), medium (2.0 m), and wide (2.6 m), which followed the ADA standards for an office environment.A bathroom stall that followed the ADA standards was included to test the spinning capability of the virtual ballbot.Lastly, a slalom course with four cones with a gap of 2.0 m was added since slalom is a commonly used practice course for wheelchair users [21].The testing course had a different course layout, colors, and texture than the training course to prevent the subjects from memorizing the course layout.
The goal of the training course (Figure 3 (a)) was to 1) familiarize the subject with the two control interfaces (i.e., HF and JS control) and the VR environment since navigation in VR can cause motion sickness [22], [23], and 2) find the subject's preferred sensitivity settings.The subject was randomly started with either HF or JS control to navigate through the training course using a TV monitor before using the VR headset.Some subjects expressed nausea or dizziness when immediately put into the VR scene since the visual (i.e., moving in VR) and proprioceptive (i.e., sitting still in real world) perception were different [23].This discomfort was more pronounced when the sensitivities were not tuned or when the subject was unfamiliar with the pendulum-like dynamics behavior of the ballbot.Thus, a TV monitor was used prior to the use of the VR headset for subjects to minimize the subject's discomfort in VR and get accustomed to the basic principles of the interfaces.In addition, the subject could understand the overall layout of the training course and tune the sensitivity settings to their preference.The sensitivity tuning process involved the subject verbally asking the on-site investigator to increase the sensitivity of the chosen interface incrementally (the initial sensitivity was set to the lowest setting) governing translational and rotational velocities of the virtual ballbot.After using the TV monitor, the subject wore the VR headset and was allowed to explore each control interface by freely navigating in an open area (12.8 m 12.8 m) using their sensitivity settings.The subject was asked to navigate through a simple square (7.3 m 7.3 m) course with a narrow (1.38 m) hallway width multiple times using steering and sliding maneuvers.Once the subjects were comfortable with the interface and the VR scene, they completed the remainder of the training course.During the entire training process, the subject was allowed to change the sensitivity settings.If the subject expressed any discomfort while using the VR, a short break (~1 minute) was given.If the subject collided during the training process, they would respawn at the beginning of the zone and repeat the zone until they could complete it without colliding.The subject was told to complete each zone without colliding (prioritizing safety) at their comfortably fast speed.
The goal of the test course was to evaluate the performance of the two interfaces using the preferred sensitivity settings defined from the training course (Figure 3 (b)).The task for the subject was to safely navigate through various zones of the test course without colliding into static / moving obstacles or walls.Unlike the training course, the subject could not adjust the sensitivity in the test course.If the subject expressed any discomfort while using the VR, a short break (~ 1 minute) was given.The subjects were randomly given either HF or JS control to begin navigating through the test course.The subjects were provided the following instructions: "The goal is to complete this course without any collisions; speed is not the top priority.Prioritize safety first."After 5 minutes of rest, the same procedure was repeated for the other interface once the subject finished using the first interface.

D. Data Processing 1) Performance Metrics
The performance of each controller interface by subject group was quantified using four attributes: 1) effectiveness, 2) efficiency, 3) comfort, and 4) robustness.These attributes were considered to provide the most relevant information for evaluating the performance of human robot interfaces for navigation and teleoperation tasks [24]- [26].
Effectiveness (i.e., how well the task is completed) and efficiency (i.e., how quickly the task is completed) were quantified by counting the number of collisions ( ) between the virtual rider/ballbot and walls or obstacles, and the successful completion time ( ), respectively [25]- [27]. and  were computed for each zone of the course.Also, whenever a collision occurred, the virtual rider/ballbot was respawned at the start of the zone where the collision occurred to prevent the subjects from finishing the task inappropriately (e.g., bouncing back and forth while completing the zone and not complying with the given task). was defined as the total time that the subject took to finish a zone without colliding.
Comfort (i.e., how easy the task is to complete mentally and physically) and robustness (i.e., how sensitive the interface performance is to difficulty level of the task) were quantified using the NASA Task Load indeX (TLX) [28] and the index of performance (IOP) [29], [30], respectively.The NASA TLX is a widely used tool for measuring subjective mental and physical workload for various interfaces in terms of scores in six categories: Mental, Physical, Temporal, Performance, Effort, Frustration [28].Each score ranges from 0 to 100 points, where a lower number indicates more comfort.The IOP was derived from the Accot-Zhai Steering Law [29], [30] and is derived by assessing performance via completion time relative to course difficulty via hallway length and width.Thus, the higher the IOP for a given interface, the less sensitive (i.e., more robust) the interface performance is to more difficult course zones.IOP is a common metric for quantifying the interface robustness in navigation tasks [29], [30].In our study, we computed IOP for three course features: straight, left turn, and right turn (IOP straight , IOP left , IOP right ).

2) Statistical Analysis
Multiple repeated-measures multivariate ANOVA (MANOVA) tests were performed to determine 1) if the HF control performed as well as the JS control, and 2) if mWCUs performed as well as ABUs.Interface performance was assessed using the four attributes (i.e., effectiveness, efficiency, comfort, and robustness) represented by four metrics (i.e.,  , , TLX, IOP).Some metrics (i.e.,  ,  ) were computed for each zone, while others (i.e., TLX, IOP) were calculated after going through all the zones.Therefore, we performed two MANOVA tests.First, a three-way MANOVA {2 (Interface: HF, JS control) 2 (Group: ABUs, mWCUs) 7 (Zone)} with two dependent variables (i.e.,  ,  ) was performed.Second, a two-way MANOVA {2 (Interface: HF, JS control) 2 (Group: ABUs, mWCUs)} with nine dependent variables (i.e., six TLX + three IOP) was performed.Follow-up univariate ANOVAs were performed on significant dependent variables for each MANOVA; these were followed by LSD post-hoc comparisons where appropriate.Significance levels were 0.05 for all analyses.All tests were performed with SPSS (Version 28.0.1,IBM Corp, USA).

III. RESULTS
Generally, the MANOVA tests found few differences among the performance metrics when considering controller interface, user group, or test zone (Figure 4).
For the three-way MANOVA on effectiveness and efficiency dependent variables, Zone (p < 0.001) and Interface Group (p = 0.02) were statistically significant (Figure 4a, 4b).Follow-up univariate ANOVAs found that both  and  were different by Zone (p = 0.002 and p < 0.001, respectively) and  was different by Interface Group (p = 0.007).Most collisions occurred in the narrow hallway and bathroom.As expected, it also took longer to move through the narrow hallway.For ABUs,  was significantly higher when using JS than HF control (0.39 ± 0.09 vs. 0.09 ± 0.05 collisions, p = 0.002).However, for mWCUs,  was not significantly different when using either interface (0.17 ± 0.09 vs. 0.23 ± 0.05 collisions, p = 0.5).
For the two-way MANOVA on comfort and robustness dependent variables, only the main effect Interface (p < 0.001) was statistically significant (Figure 4c, 4d).Follow-up univariate ANOVAs found that TLX-Physical was different by Interface (p < 0.001).Participants felt that the HF control required more physical demand.

1) Performance of HF and JS control
Generally, the hands-free (HF) control performed similar to the joystick (JS) control for both manual wheelchair users (mWCUs) and able-bodied users (ABUs) when maneuvering a virtual ballbot through indoor environments.While test subjects indicated on the NASA TLX that the physical demand for HF control was higher than JS control, subjects from both groups did not visibly show or express fatigue or exhaustion after the study.
All subjects from both groups were able to quickly learn (training time < 45 minutes) and perform well with HF control.On average, 16 out of 20 subjects reported extensive JS experience, i.e., 10 years of previous gaming experience with an average frequency of 2.6 hours/week.In comparison, only 4 out of 20 subjects have used self-balancing devices (e.g., Segway) or human-computer interfaces controlled by the user's balance (e.g., Wii Balance Board) with very little experience (< 4 hours of total usage).These data suggest that while the subjects had more previous exposure and experience with JS than HF control, subjects were able to learn quickly and perform well with the HF control.The HF control's fast learning curve and comparable performance to JS control may be attributed to the core concept of HF control taking advantage of the intuitive human locomotion behavior of leaning or facing one's body toward the intended motion direction [31].Thus, HF control showed promising results to be used in controlling a physical device for both ABUs and mWCUs.
Depending on the environment, one interface may be preferred over the other.The JS control may be preferred over HF control in situations requiring abrupt braking.Since the JS control device had mechanical springs that automatically returned the knob into its neutral position when the knob was released, the subject could effortlessly and quickly brake by simply letting go of the knob.Also, the JS control could be used for longer periods of time and for more fine and precise movements since less physical effort was needed.On the other hand, the HF control may have advantages over JS control.The users of HF control can utilize their hands for other tasks and not only navigation, offering more sense of independence and freedom.Also, subjects reported that the HF control felt more intuitive since the torso leaning motion felt more natural.Another interesting advantage of HF control was the potential health benefits such as getting core exercise and relieving pressure sores, which are problems faced by mWCUs [32].
mWCUs performed as well as the ABUs for navigating the virtual ballbot.This population of mWCUs displayed a diverse range of torso mobility (e.g., asymmetrical torso movement, limited torso range of motion), core functionality (e.g., full vs. limited core function), and different spinal conditions (e.g., scoliosis, lordosis, kyphosis).However, all mWCU subjects were able to perform as well as ABUs using both interfaces mainly due to the ability to tune the sensitivity of the interfaces to accommodate the subject's preference and limitations.
Both subject groups generally exhibited similar levels of performance across all zones.For wide and medium zones, both groups demonstrated high levels of effectiveness and efficiency for both interfaces, since there were no moving or static obstacles while wide space was available.Even for more challenging zones such as the moving obstacle, table, and slalom, both subject groups navigated effectively and efficiently using either interface since the number of collisions and completion time were similar.The narrow and restroom zones were the most difficult zones for both groups since these zones had the least amount of free space.
The effectiveness of the interfaces depended on the subject group.For mWCUs,  was were similar when using JS or HF control.However, for ABUs,  was higher when using the JS control.This may be due to the mWCUs having more experience using JS-like devices for video gaming (~12.2 years) than ABUs (~7.7 years).This trend of mWCUs having more gaming experience than ABUs may be prevalent in the general population since mWCUs may spend more time engaged in sedentary activities (e.g., watching TV, playing video games) than ABUs, especially for adolescents [33].

2) Guidelines for HRI design in VR
A few important points related to the HRI design and VR test setup/protocol were made to assist other researchers developing HRI for people with disability or conducting locomotive/navigation tests in VR environments.In terms of the interface hardware design, an ergonomic and adjustable seat and backrest design was critical for achieving high performance, especially for mWCUs.For example, having a proper backrest height was essential for the mWCUs to give sufficient back support for safety while providing as much freedom as possible for torso movements.These seat and backrest dimensions can be obtained by following their traditional manual wheelchair design specifications.In terms of the HRI software design, the adjustability of interfaces sensitivities was important to accommodate subjects with various torso mobilities.In terms of the VR test setup and protocol, starting the test using the TV monitor and then putting the subjects into the VR helped minimize motion sickness due to VR scene and learn the course layout and interfaces easily.Since the subjects needed to intake a lot of novel information regarding the novel HRIs, breaking down this information into smaller pieces seemed to be a more effective learning strategy.In addition, some subjects' interface performance was affected by the VR motion sickness.To prevent this, frequent (every one to two minutes) breaks (~1 minute) in the beginning of the experiment was mandatory for all subjects, even if the subject did not exhibit discomfort during testing, since sudden motion sickness may occur when transitioning from prolonged VR to the realworld environment.Thus, future HRI designs and VR studies for navigation applications should consider the subject's ergonomics, learning capacity and adaptability to the VR.

3) Limitations and Future Work
A few limitations related to the test setup, HRI design, and experimental protocol were observed in this study.
First, while the VR scene gave more immersion than a TV monitor, the VR scene still did not provide the same level of spatial awareness (e.g., perception of the distance between the virtual rider and near-by wall) and perceptual/proprioceptive feedback (e.g., acceleration and tilting of the ballbot) as the real physical world [34].This limitation of VR may have increased the difficulty for subjects to effectively maneuver through these spatially small zones (e.g., narrow hallway, bathroom stalls).However, regardless of the zones, subjects from both groups exhibited similar levels of performance, suggesting that the mWCUs performed as well as ABUs using HF and JS control.
Second, our VR test setup did not provide any realistic force/tactile feedback of the dynamics of moving ballbot since only a static FSS was used.Providing subjects with realistic force feedback using a dynamic test platform with actuators may improve the subject's control and stability of the virtual ballbot, especially for the HF control, since humans rely on not just visual but also proprioceptive feedback for balancing [35].In addition, it is known that proprioceptive feedback can be inherently faster than vision-based reactions (50-100 ms vs. 150-250 ms) [36], and triggers true balancing and reactive motions that are the primitive of human motor control.This may also reduce VR motion sickness.
Third, more advanced mapping functions from interface signals to the ballbot can be explored to better accommodate users with limited torso mobility, since some wheelchair users had asymmetric and limited torso mobilities.Advanced mapping functions may be generated using non-linear functions that use data-driven approaches [37] personalized for each user's mobility, potentially automating and expediting the process for adjusting user's sensitivity.
Fourth, after completing the study, we realized that not all subjects used lateral sliding between tables (2 ABU, 1 mWCU) and spinning in the bathroom (8 ABU, 10 mWCU).Other subjects were able to drive forward and backward to navigate these zones.Future studies will revise the instructions to explicitly request particular motions in specific zones.
Lastly, all subjects were young adults and most mWCUs were student athletes.Future studies should represent more diverse subject populations with wider ranges of age and athleticism.

V. CONCLUSIONS
A novel hands-free (HF) human-robot interface that uses the rider's torso motions to maneuver a virtual ballbot was developed.ABUs and mWCUs navigated a virtual ballbot using HF and joystick (JS) control (a commonly used interface) through a challenging indoor course in which performance was measured using four attributes: efficiency (number of collisions), efficiency (successful completion time), comfort (NASA TLX scores), and robustness (index of performance).Generally, the HF control performed similar as JS control.While there was more physical demand for HF than JS control, most subjects did not exhibit physical fatigue when using HF control.Also, the mWCUs, even those with more limited torso mobility, performed as well as ABUs.The proposed hands-free interface showed potential to be used for a physical ballbot.Our study may provide insight for others developing HRIs for navigating devices in VR environments.

Figure 1 .
Figure1.Hands-free (HF) control is used to control a virtual rider on a ballbot using the signals provided by the Torso-dynamics Estimation System (TES), which consists of an adjustable force sensing seat (FSS) and inertial measurement unit (IMU).See accompanying video.
A rider, ballbot and two obstacle courses (a training and test course) were created in an open-source robot simulation software (CoppeliaSim, Coppelia Robotics, Switzerland).

Figure 2 .
Figure 2. Simulation of virtual rider and ballbot.(a) For hands-free (HF) control, the TES recorded the seated subject's 3D torso angles and 2D center of pressure (COP).(b) For joystick control, the JS recorded 3D angles of motion.(c) Three planar models of the ballbot were used for the LQR ballbot controller.(d)Three evenly spaced omni-wheel and motor pairs drove the motion and direction of the ball.(e) The 3D torso angles and subject's physical measurements were mapped to the virtual environment to replicate realistic rider-ballbot dynamics.HRI commands for HF or JS control were preprocessed and used as input for the ballbot controller to control the omnidirectional velocity of the virtual ballbot.

Figure 3 .
Figure 3. Virtual (a) training and (b) testing obstacle courses replicating seven realistic and challenging indoor environment zones.

Figure 4 .
Figure 4. Results plots of HF and JS control for ABUs and mWCUs for performance metrics of (a) effectiveness and (b) efficiency, (c) comfort and (d) robustness.

TABLE 1 .
SUBJECT INFORMATION & CONTROLLER SENSITIVITY SETTINGS The next three elements of  were sources of input reference signals, which were x ref =  ,  ,  for HF or  ref =  ,  ,  for JS control.