Fjeld.ch

Augmented Reality Approaches to Sensory Rehabilitation
ABSTRACT
The potential for augmented reality (AR) technologies to impact work habits and collaborative work is perhaps moststriking for individuals with sensory or perceptual impairments. Commercial display and sensing technologies, incombination with on-board computation capabilities (either in the form of specialized hardware or general-purposewearable computers), are introducing a new generation of adaptive aids. Spectacles and traditional hearing aids arebeing replaced by customized and context-sensitive conditional display systems. These technologies will enablemuch broader access, both to day-to-day interaction and to our increasingly information-based workspace. Twospecific examples illustrate many of the technological and human factors challenges presented in the development ofthese new AR sensory aids. The lessons and technological advances gained from these efforts may have muchbroader implications for the design and implementation of generic AR systems.
1. Introduction
As interface and computing technologies evolve they may offer hope to people with sensory and neuro-perceptualdisorders. Although traditional medical solutions and prosthetic devices continue to advance, wearable computingand personal multi-sensory displays provide a new paradigm for context-sensitive and user-customized „sensoryprosthetics“. At the Human Interface Technology Lab we are developing several such devices that represent a rangeof problems in this domain. We present an overview here of two of them that are particularly promising andinformative: (1) visual aids based on scanned retinal images, such as the Virtual Retinal Display (VRD), and (2) anaugmented reality display that aids Parkinson’s Disease patients to overcome frozen gait.
2. Low Vision Aids
While there is evidence of occasional use of natural lenses to focus poor eyesight as early as the 3rd century AD, itwas not until roughly 1850 that spectacles designed to improve visual acuity became available to the general public.
Today eyeglasses are being at least partially displaced by surgical procedures that augment and correct the opticalproperties of the eye. Despite these advances, many partially sighted (‘low vision’) individuals are not able toachieve adequate vision for essential daily tasks. It is estimated that nearly 14 million Americans experience visualproblems that impede their enjoyment of everyday living (National Advisory Council, 1998).
Conventional computer displays, for example, pose problems for many people with limited vision. Among the visualchallenges presented by conventional displays are insufficient brightness and spatial resolution, flicker (especiallywith peripheral viewing, a common strategy adopted by many people with macular degeneration), and excessiveglare. The deficiencies of conventional visual displays are even more apparent in AR systems, in which computer-generated imagery is superimposed on the user’s view of the physical world. Current consumer-level „see-through“head-mounted displays based on reflected LCD panels provide only „ghosty“ images that are inadequate in outdoorlighting conditions and are difficult to keep in focus at arbitrary accommmodative distances.
2.1 Scanned Retinal Displays
Scanned retinal displays, such as the Human Interface Technology Lab’s virtual retinal display (VRD) technologymay provide a significantly better alternative for low vision computer users. In combination with image acquisitiontechnologies and on-board computing, the VRD may also form the basis for a wearable low vision aid. The VRDuses a scanning light beam in place of the conventional image planes of CRTs and LCDs. A very small spot isfocused onto the retina and is swept over it rapidly in a raster pattern (see functional schematic in Figure 1). Becausethe photons are tightly shepherded, this display technique offers exceptional brightness, even at very low light power levels. In addition, the VRD’s small ‘exit pupil’ of collimated light greatly reduces light-scattering and provides fora very wide depth of focus (Kollin, 1993; Tidwell et al, 1995; Johnston and Wiley, 1995).
Figure 1. Block diagram of the basic VRD components.
Our prototype VRD projects very low laser power (typically, 50-200 nanowatts), yet is perceived as very bright,colorful, and comfortable to view. Three coherent light sources (red, green, and blue) are combined to provide fullRGB color, or one light source can be used for monochrome displays. For each primary color, the current prototypeproduces a standard VGA resolution image at a refresh rate of 60 Hz.
Although it is being developed as a generic display technology, its unique characteristics make the VRD anexceptionally good image source for people with partial loss of vision, particularly those due to optical anomalies ofthe eye. Some of the unique aspects of VRD display technology of interest for low vision applications include its: very high brightness (at levels unattainable by CRT and LCD), collimated light, resulting in less eye strain and accommodative conflict, and very small system exit pupil, affording a small entrance pupil into the eye, large depth of focusand greatly reduced glare.
2.2 Low Vision Pilot Studies
Throughout the development of the VRD we have demonstrated the system to a wide variety of people, includingmany individuals with low vision conditions. Numerous subjective reports of enhanced vision have led us to a moresystematic examination of VRD effects on low vision visual perception. The primary objectives of this pilotresearch have been to determine what types of low vision will benefit most from VRD technology, and to determineif the VRD can be an effective alternative low vision computer interface.
Which Display Was
Which Display Was
Perceptually Clearer?
Perceptually Brighter?
Equal (2)
Equal (2)
Figure 2. Subjective preference results for VRD vs. CRT display conditions.
In Kleweno’s (1999) study thirteen subjects were recruited from the local low vision community. The low visionconditions represented by this group included amblyopia, retinal detachment, diabetic retinopathy, glaucoma,cataracts, nystagmus, aniridia, surface wrinkling retinopathy and strabismus. Using a portable red monochromeversion of the VRD, this study compared acuity, reading speed and user preference across four display conditions:(1) standard CRT with white on black contrast, (2) standard CRT with red on black contrast, (3) VRD with red onblack contrast with a luminance setting equal to half the measured value of the CRT (white on black contrast), and (4) VRD with red on black contrast with a luminance setting equal to the measured value of the CRT (white on blackcontrast).
Average reading speed for this pilot subject sample was variable, with no statistically significant increase in averagereading speed using the VRD compared with the CRT. However, the relative superiority of the VRD over the CRTdisplay was striking for subjects with optical causes of low vision (e.g., cataracts and corneal aberrations). Resultsfor the subjective comparisons of image quality between VRD and CRT displays were even more compelling (asshown in Figure 2). These pilot low vision subjects clearly preferred the brightness and clarity of the scanned lightdisplay over the conventional CRT display.
Although the red monochrome VRD was shown to be effective for several low vision subjects, most subjectsindicated that red was clearly not their preferred color for text display. Seibel et al’s (2001) study further addressedthis issue of optimal text color for a VRD-based low vision AR display. Twelve subjects were recruited, with awide range of low vision condition, including acromatopsia (lack of color vision), macular degeneration, partialalbinism, glaucoma, retinal scarring, and Stargaarts disease (a congenital form of macular degeneration). Of these,eight were able to complete the testing protocol, in which text reading performance was measured for various textcolors presented on four different background lighting conditions.
R G B W RR GG BB R1 G1 B1 W1 R2 G2 B2 W2 Color Contrast Condition
Figure 3. Normalized reading times for all 15 color contrast conditions.
Mean reading times, normalized for each subject within each background condition, are shown in Figure 3. Whilelonger reading times (and thus poorer performance) for red text were expected, the finding that blue text wassignificantly easier (faster) to read than the other text colors in the ambient lighting conditions was surprising and notpreviously reported for CRT- and LCD-based low vision reading studies. We are currently systematically exploringthese effects to determine how viewing text via the VRD is different from viewing text on conventional displays, andwhether the effect will generalize to normally sighted readers with an equivalently challenging task. The overallfocus of this research is to determine the optimal image generation parameters and visual aid design specificationsfor a variety of low vision conditions. Retinal light-scanning technology appears quite promising to (1) optimizelow vision access to computing, and (2) form the basis of a wearable low vision aid (in combination with imageprocessing and context-sensitive display).
3. Augmented Perception for Neurological Disorders
Sensory feedback is a vital component of complex integrated motor behaviors, such as walking and talking. In
Parkinson’s Disease (PD), a degenerative neural disorder, these behaviors are progressively disrupted. While
typically characterized as a ‘motor disorder’ due to failure of the dopaminergic pathways in the basal ganglia, recent
evidence suggests that faulty sensory feedback may also play a role in exacerbating these symptoms.
3.1 Parkinson’s Disease and Kinesia Paradoxa
One of the most debilitating effects of PD is the sudden unpredictable and total inability to take a single step(akinesia). The effects of reduced levels of dopamine on mobility range from the complete inability to initiateambulation, to small shuffling stutter steps, to normal gait that suddenly freezes (Marsden, 1977). Kinesia paradoxais an interesting phenomenon that has been documented in the PD literature for many decades (Martin, 1967), buthas yet to be fully exploited for therapeutic purposes. In PD patients exhibiting akinesia the presence of objects onthe ground can often facilitate walking. Typically, kinesia paradoxa is demonstrated by placing a series of smallobjects or markers in the patient’s intended path, about one stride length apart. The result can be a dramaticrecovery of full stride length walking, but only in the presence of the cues.
3.2 Visual Cueing with AR
Numerous past attempts to develop PD walking aidsbased on the kinesia paradoxa effect have had limitedpractical applicability. Over the past few years,however, we have developed a number of highlyeffective AR devices, in which virtual visual cues areused to evoke the kinesia paradoxa response in placeof physical cues (Prothero, 1993; Riess andWeghorst, 1995; Weghorst and Riess, 2001). Usingmicroelectronics and compact head-worn displaytechnologies, these virtual visual cues can besuperimposed onto reality, and can be made availableon demand in a wearable package. Several workingprototypes have been built which demonstrate thatvirtual visual cues are indeed effective and may havea significant therapeutic impact in the lives of peoplesuffering from PD.
Figure 4. Central Field Cueing Device. This implementation uses reflections of elongated „scrolling“
LED lamps to project the virtual equivalent of objects onto the real world.
The most cost-effective solution to date uses an array of flashing LEDs, reflected off an optical combiner in theuser’s field of view, producing enhanced visual flow via „apparent motion“ effects (see Figure 4). The efficacy ofthe LED approach has been demonstrated in a series of trials with experimentally naïve PD subjects.
Unassiste d Akinetic Gait
Visual Cue Enabled Gait
ect Number
Mean Stride Length (inches)
Figure 5. Effect of central field visual cueing device on mean stride length for akinetic subjects.
Figure 5 summarizes the results of a series of initial trials with 10 male and 2 female PD subjects, ranging in agefrom 45 through 74, with subjects ordered by unassisted stride length. Prior to testing with the augmented realitycues, each subject was screened for responsiveness to kinesia paradoxa cues using physical markers on the floor.
Subjects who exhibited no stride length increase in response to the physical cues were not presented with the virtualcues. It should be noted, however, that less than 10% of the akinetic PD patients tested failed to exhibit kinesiaparadoxa, and all subjects who exhibited some degree of stride enhancement with physical cues also respondedpositively to the virtual cues. Clearly, the visual cueing effect is extraordinarily robust. The mean stride length ratio(unassisted/cued) for these akinetic subjects ranged from .05 (S1, S2) to .93 (S12), with a mean stride length ratio of.60 across all subjects in this sample.
Since there is a natural „ceiling effect“ for stride length (i.e., the maximal normal stride length is relatively fixed foreach individual, depending largely on subject height), the benefits of visual gait cueing are proportionate to thedegree of akinesia experienced by each subject. The strongest effects were demonstrated for subjects who exhibitedfreezing or festinating gait (e.g., S1 and S2). It is interesting to note, however, that even mildly akinetic subjects(e.g., S11 and S12) responded positively to the experimental cueing, and that no subjects tested to date haveexhibited a shorter mean stride length in response to the visual cues. In addition to these quantitative effects, severalsubjects also exhibited noticeable improvements in gait topology and posture. One subject, for example, showed amarked decrease in her asymmetric foot dragging and shoulder slant while walking using the visual cueing device.
4. Conclusions
The development of small inexpensive microprocessors with low power requirements complements the recentemergence of consumer level head-mounted displays. In addition, the challenge of creating adequate visual displaysfor wearable applications has engendered new approaches to visual display, including technologies that scanmodulated light directly onto the retina. This convergence of enabling technologies bodes well for a new generationof adaptive aids for a wide variety of sensory and neurological disorders. These applications may also have apositive impact on the development of AR for more generic purposes.
REFERENCES
Kleweno, C., Seibel, E., Kloeckner, K., Viirre, E., Furness, T.A. Evaluation of a scanned laser display as an alternative low vision computer interface. OSA Technical Digest of VSIA Topical Meeting , 1999.
Johnston, R.S., and Wiley, S.R. Development of a commercial retinal scanning display. Proc. SPIE. Helmet- and Head-mounted Displays and Symbology Design Requirements II. 2465 : 2-13, 1995.
Kollin, J. A Retinal Display For Virtual-Environment Applications. In Proceedings of SID International Symposium, Digest Of Technical Papers, pp. 827, 1993.
Marsden, David C., and J.D. Parkes, Success and problems in long-term levodopa therapy in Parkinson's Disease.
Martin, J.P. The Basal Ganglia and Posture. J.B. Lippincott Company, 1967.
National Advisory Council. Vision Research (1998) A National Plan: 1999-2003 . U.S. Dept. of Health and Human Services (NIH Publication No. 98-4120), 1998.
Prothero, J. The Treatment of Akinesia Using Virtual Images. Master's Thesis, University of Washington, 1993.
Riess, T., and Weghorst, S. Augmented reality in the treatment of Parkinson's Disease. Proceedings of Medicine Meets Virtual Reality 3, IOS Press, 1995.
Seibel, E.J., Gau, C.-C., McQuaide, S., Weghorst, S.J., Kelly, J.P., and Furness, T.A. Augmented retinal light scanning display for low vision: Effect of text color and background on reading performance OSA TechnicalDigest of VSIA Topical Meeting, 2001.
Tidwell, M., Johnston, R.S., Melville, D. and Furness, T.A. The Virtual Retinal Display - A Retinal Scanning Imaging System. In Proceedings of Virtual Reality World ’95, pp. 325-333, 1995.
Weghorst, S., and Riess, T. Wearable sensory enhancement aids for Parkinson's Disease. Medicine Meets Virtual Reality, Newport Beach, 2001.

Source: http://www.fjeld.ch/hcii2001/Weghorst.pdf