IASTED 2010

Virtual Environment Sensorimotor Hand Dexterity Training System

IASTED 2010
K.G. August (Switzerland, UK, USA), M. Sellathurai (UK), D. Bleichenbacher,A. Skairek, G. Fluet, A. Merians, and S. Adamovich (USA)
Virtual Environments (VE) can be used to structure, organize, and control sensorimotor experiences (stimuli and feedback), activities, instructions (explicit or implicit), dialogues including those embodied by Virtual Avatar Teachers for observation and imitation of others or by Personal Proxy, provide haptics and kinematic support, enable intensive repetitions and practice, and interactions with real or virtual objects in ways unachievable through conventional means. The Virtual Environment Sign Language Instruction System (VESLI), designed to provide sensorimotor hand dexterity training and performance measurement features, incorporates virtual reality exercises mapped to specific impairments underlying dysfunction of the hand and organized in levels of difficulty. One goal of the VESLI design is to provide hand dexterity training even when the patient is unable to move adequately enough to participate in traditional therapies. Ten control subjects practiced hand dexterity exercises with the VESLI system using a learning and memory protocol. When studying with natural hands, pictures helped in learning the gestures, and text definitions helped in recall. Meanwhile with virtual hands avatars, the picture descriptions resulted in better performance. Sensorimotor experience in VE might be capable of providing safe and appropriate task-based sensory stimulation and feedback to extend available and early therapies to those in need.

NEBEC

Virtual reality, robot, and object touch: Blended reality sensorimotor training experience

August, K.G.; Guidali, M.; Sellathurai, M.; Hepp-Reymond, M.; Adamovich, S.V.; Riener, R.;
Inst. of Neuroinf., Univ. & ETH Zurich, Zurich, Switzerland

This paper appears in: Bioengineering Conference (NEBEC), 2011 IEEE 37th Annual Northeast
Issue Date: 1-3 April 2011
On page(s): 1 - 2
Location: Troy, NY
ISSN: 2160-7001
Print ISBN: 978-1-61284-827-3
References Cited: 32
INSPEC Accession Number: 12030447
Digital Object Identifier: 10.1109/NEBC.2011.5778716
Date of Current Version: 27 May 2011

Abstract

Loss of body sensations often accompanies injuries to the central or peripheral nervous system and can lead to disintegration of body schema resulting in altered sensory dependent motor skills. Localizing external objects within peripersonal space plays an important role in the strategies of accurate human movement, towards targets and for obstacle avoidance. In our system, practicing viewing and finger-tip contact with real objects is enabled using Blended Reality: Virtual Reality, Robot Assistance, and Real Object Touch. Arm Location exercises accompany multi-sensory experience in the blended reality. Adding vision and touch, salient properties of three dimensional real objects, to the technology assisted training environment increases the number and realistic qualities of sensory modalities for integration to bolster performance of the trainee. A number of exercises have been developed and are being investigated to understand contributions of the multi-sensory experiences upon performance, and to identify appropriate protocols for therapy.

TePRA

A system for sensory motor rehabilitation of the upper limb with virtual reality, exoskeleton robot, and real objects

Technologies for Practical Robot Applications (TePRA), 2011 IEEE Conference on
Issue Date: 11-12 April 2011
On page(s): 54 - 63
Location: Woburn, MA
Print ISBN: 978-1-61284-482-4
References Cited: 141
INSPEC Accession Number: 11963082
Digital Object Identifier: 10.1109/TEPRA.2011.5753482
Date of Current Version: 21 April 2011

Abstract

Technology assisted therapy has the potential to transform rehabilitation options available, and to dramatically increase the reach of today's healthcare system. Yet challenges persist in rendering translational application designs that optimize the full potential of technology and create value for the patient and the therapist. In a step towards optimizing value of technologies for practical applications to support very weak patients who might otherwise be unable to participate in traditional therapies, an integrated sensory motor training station was designed and developed. Inspired by recent neuroscientific research findings the goal of the design was to provide concurrent first person perspective immersive action observation of both virtual and real elements for motor and sensory experience; the system incorporates a virtual limb proxy that can be personalized and actuated by the robot and that is accompanied by exercise practice in peripersonal space for a plasticity promoting experience for the hand and arm. The station uses virtual reality and real objects for visual sensory experience, real objects also provide tactile sensory experience, and an exoskeleton upper limb robot provides assistance to patients. For many patients, successful movement and movement intensity required in rehabilitation is not achievable without the robot assistance. The multi-sensory features of the system promote a top-down strategy for training the upper limb (hand and arm) complementing the robot training; the system is ideally targeted for weak patients and those with tactile or proprioception sensory loss and those who are known to benefit from multi-sensory experiences.

Computing in Cardiology

Computing in Cardiology September, 2011

Heart rate asymmetry and emotional response to robot-assist task challenges in post-stroke patients

Herbert F. Jelinek1,
Katherine G. August2,
Md. Hasan Imam3,
Ahsan H. Khandoker3,4,
Alexander Koenig2,5,
Robert Riener2,5
1Charles Sturt University, Albury, Australia; 2University of Zurich, Zurich, Switzerland; 3University
of Melbourne, Melbourne, Australia; 4 Khalifa University, Abu Dhabi, UAE 5ETH, Zurich,
Switzerland


Abstract
The level of motivation or stress influences learning
the use of a robot-assist device for walking. Heart rate
asymmetry (HRA) indicates the level of parasympathetic
(HRA<0.5) and sympathetic (HRA>0.5) involvement in heart rate regulation. Three patients and seven controls
were presented increasing levels of task difficulty. During
training patients showed an increase in stress as indicated by the HRA index (0.524±0.02) in contrast to control participants (0.485±0.03). As the task complexity increased, the HRA in the patient group was atypical and
falling below 0.5, compared to control (HRA>0.5). The
latter result reflects an increased cognitive involvement
and a higher sympathetic predominance in accordance
with an increased task difficulty. Thus stroke affected the
response to the task challenges in that the patient response to increasing task challenge leads to an inversion of HRA associated with a decreased mental engagement and higher risk of sudden cardiac death.

Process for reducing false negative mammography bilateral registration

This paper appears in: Bioengineering Conference, 2004. Proceedings of the IEEE 30th Annual Northeast
Issue Date: 17-18 April 2004
On page(s): 69 - 70
ISSN:
Print ISBN: 0-7803-8285-4
Cited by : 1
INSPEC Accession Number: 8096976
Digital Object Identifier: 10.1109/NEBC.2004.1299997
Date of Current Version: 24 May 2004

August, K.; Biswal, B.; Reisman, S.;
New Jersey Inst. of Technol., Newark, NJ, USA

Abstract

Each woman presents slightly different risk factors that should be considered during mammography image interpretation to reduce the risk of false negative. Given the complexity of the risk situation, and the variety of practical and affordable image processing techniques available, processes and systems sensitive to individual risk factors may reduce false negative reports. Bilateral registration as a process improvement may help to reduce risk of false negative mammography by providing statistical objective metrics to identify regions of interest in screening at risk women for whom historic images are not available, or for whom historic images are not useable.

Restor Neurol Neurosci. 2009;27(3):209-23.

A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution: a proof of concept study.

Adamovich SV, August K, Merians A, Tunik E.

Department of Biomedical Engineering, New Jersey Institute of Technology, University Heights, Newark, NJ, USA.

Abstract

PURPOSE:

Emerging evidence shows that interactive virtual environments (VEs) may be a promising tool for studying sensorimotor processes and for rehabilitation. However, the potential of VEs to recruit action observation-execution neural networks is largely unknown. For the first time, a functional MRI-compatible virtual reality system (VR) has been developed to provide a window into studying brain-behavior interactions. This system is capable of measuring the complex span of hand-finger movements and simultaneously streaming this kinematic data to control the motion of representations of human hands in virtual reality.

METHODS:

In a blocked fMRI design, thirteen healthy subjects observed, with the intent to imitate (OTI), finger sequences performed by the virtual hand avatar seen in 1st person perspective and animated by pre-recorded kinematic data. Following this, subjects imitated the observed sequence while viewing the virtual hand avatar animated by their own movement in real-time. These blocks were interleaved with rest periods during which subjects viewed static virtual hand avatars and control trials in which the avatars were replaced with moving non-anthropomorphic objects.

RESULTS:

We show three main findings. First, both observation with intent to imitate and imitation with real-time virtual avatar feedback, were associated with activation in a distributed frontoparietal network typically recruited for observation and execution of real-world actions. Second, we noted a time-variant increase in activation in the left insular cortex for observation with intent to imitate actions performed by the virtual avatar. Third, imitation with virtual avatar feedback (relative to the control condition) was associated with a localized recruitment of the angular gyrus, precuneus, and extrastriate body area, regions which are (along with insular cortex) associated with the sense of agency.

CONCLUSIONS:

Our data suggest that the virtual hand avatars may have served as disembodied training tools in the observation condition and as embodied "extensions" of the subject's own body (pseudo-tools) in the imitation. These data advance our understanding of the brain-behavior interactions when performing actions in VE and have implications in the development of observation- and imitation-based VR rehabilitation paradigms.

PMID:

19531876

[PubMed - indexed for MEDLINE]

Conf Proc IEEE Eng Med Biol Soc. 2006;1:3692-5.

FMRI analysis of neural mechanisms underlying rehabilitation in virtual reality: activating secondary motor areas.

August K, Lewis JA, Chandar G, Merians A, Biswal B, Adamovich S.

Source

Biomed. Eng. Dept., New Jersey Inst. of Technol., Newark, NJ, USA. ka38@njit.edu

Abstract

A pilot functional MRI study on a control subject investigated the possibility of inducing increased neural activations in primary, as well as secondary motor areas through virtual reality-based exercises of the hand. These areas are known to be important in effective motor output in stroke patients with impaired corticospinal systems. We found increased activations in these brain areas during hand exercises in VR when compared to vision of non-anthropomorphic shapes. Further studies are needed to investigate the potential of virtual reality-based rehabilitation for tapping into the properties of the mirror neuron system to stimulate plasticity in sensorimotor areas.

BioTechniques - 2011 MacArthur Fellow: Kevin Guskiewicz, Sports Medicine Researcher

BioTechniques - 2011 MacArthur Fellow: Kevin Guskiewicz, Sports Medicine Researcher