What people “see” with retinal prostheses? Fundamentally, qualitatively different than natural vision, all invoked electrical stimuli to describe the appearance of their percepts; the process never ceased to be cognitively fatiguing

Erickson-Davis C, Korzybska H (2021) What do blind people “see” with retinal prostheses? Observations and qualitative reports of epiretinal implant users. PLoS ONE 16(2): e0229189. https://doi.org/10.1371/journal.pone.0229189

Abstract

Introduction: Retinal implants have now been approved and commercially available for certain clinical populations for over 5 years, with hundreds of individuals implanted, scores of them closely followed in research trials. Despite these numbers, however, few data are available that would help us answer basic questions regarding the nature and outcomes of artificial vision: what do recipients see when the device is turned on for the first time, and how does that change over time?

Methods: Semi-structured interviews and observations were undertaken at two sites in France and the UK with 16 recipients who had received either the Argus II or IRIS II devices. Data were collected at various time points in the process that implant recipients went through in receiving and learning to use the device, including initial evaluation, implantation, initial activation and systems fitting, re-education and finally post-education. These data were supplemented with data from interviews conducted with vision rehabilitation specialists at the clinical sites and clinical researchers at the device manufacturers (Second Sight and Pixium Vision). Observational and interview data were transcribed, coded and analyzed using an approach guided by Interpretative Phenomenological Analysis (IPA).

Results: Implant recipients described the perceptual experience produced by their epiretinal implants as fundamentally, qualitatively different than natural vision. All used terms that invoked electrical stimuli to describe the appearance of their percepts, yet the characteristics used to describe the percepts varied significantly between recipients. Artificial vision for these recipients was a highly specific, learned skill-set that combined particular bodily techniques, associative learning and deductive reasoning in order to build a “lexicon of flashes”—a distinct perceptual vocabulary that they then used to decompose, recompose and interpret their surroundings. The percept did not transform over time; rather, the recipient became better at interpreting the signals they received, using cognitive techniques. The process of using the device never ceased to be cognitively fatiguing, and did not come without risk or cost to the recipient. In exchange, recipients received hope and purpose through participation, as well as a new kind of sensory signal that may not have afforded practical or functional use in daily life but, for some, provided a kind of “contemplative perception” that recipients tailored to individualized activities.

Conclusion: Attending to the qualitative reports of implant recipients regarding the experience of artificial vision provides valuable information not captured by extant clinical outcome measures.

Discussion

We undertook ethnographic research with a population of retinal prosthesis implant recipients and vision rehab specialists, documenting the process of getting, learning to use and living with these devices.

We found that the perceptual experience produced by these devices is described by recipients as fundamentally, qualitatively different than natural vision. It is a phenomenon they describe using terms that invoke electric stimuli, and one that is ambiguous and variable across and sometimes within recipients. Artificial vision for these recipients is a highly specific learned skillset that combines particular bodily techniques, associative learning and deductive reasoning to build a “lexicon of flashes”—a distinct perceptual vocabulary—that they then use to decompose, recompose and interpret their surroundings. The percept does not transform over time; rather, the recipient can better learn to interpret the signals they receive. This process never ceases to be cognitively fatiguing and does not come without risk nor cost to the recipient. In exchange recipients can receive hope and purpose through participation, as well as a new kind of sensory signal that may not afford practical or functional use in daily life, but for some provides a kind of “contemplative perception” that recipients tailor to individualized activities. We expand on these findings below to explore what they mean in terms of the development and implementation of these devices, as well as for our understanding of artificial vision as a phenomenon.

What does it mean that the recipients describe artificial vision as being fundamentally, qualitatively “different” than natural vision? We believe that acknowledging that artificial vision is a unique sensory phenomenon might not only be more accurate, but it may also open up new avenues of use for these devices. Artificial vision may be considered as “visual” in terms of being similar to what recipients remember of the experience of certain kinds of light, as well as by offering the possibility of being able to understand features of the environmental surround at a distance. That being said, artificial vision was also described as both qualitatively and functionally different than the “natural” vision the recipients remember. It is in this way that the sensory experience provided by these devices could be viewed as less a restoration or replacement and more as a substitution; that is, as offering an entirely different or novel sensory tool. By shifting from the rhetoric of replacement or restoration to substitution we believe it could widen the bounds in which researchers and rehabilitation specialists think and operate with regard to how these devices are designed and implemented, potentially liberating a whole new spectrum of utility through the novel sensations these devices produce. Likewise, this shift could change the expectations of individuals receiving these devices, including addressing the initial disappointment that was expressed by many of our recipients when they encountered just how different the signals were to what they were expecting.

Second, acknowledging artificial vision as a unique sensory phenomenon also helps us understand the importance of qualitative description. The process of learning to use the device is a cooperative process between the rehabilitation specialist and the recipient, with the specialist guiding the recipient to attend to their perceptual experience and interpret it in specific ways. This process begins with the recipient learning to recognize how the basic unit of artificial vision—the phosphene—appears for them, and then describe that to the rehab specialist. The specialist then uses this information to guide the recipient in learning how the phosphenes correspond to features of the environment. It is a continuous and iterative communicative practice between the recipient and specialist that evolves over many months, during which stimuli are encountered, the recipient responds, and the specialist gives corrective or affirming feedback (with more or less description by the recipient and guidance by the specialist depending on the dynamic and need). The process is so specific to the dynamic between recipient and specialist that it can be considered to be “co-constructed” within their interactions.

Because each recipients’ qualitative experience is so distinct (phosphenes differ significantly between recipients so that no recipients’ perceptual experience is alike [60]) each process is tailored to the individual recipient by specific specialists. We found that certain vision rehabilitation specialists inquire in more depth about a recipient’s qualitative experience than others, using different methods, styles and techniques, and this can result in a different experience—and thus outcome—for the recipients. Our findings are based on reports captured either by directly asking the recipients about their experience or observing descriptions that were part of the rehabilitation process but that were by and large not recorded by the specialists nor relayed back to the companies, early stage researchers nor the individuals being implanted. That is, we found that there is no protocol in place for capturing or sharing recipient’s qualitative reports, including within the companies (between various clinical sites). Yet these kinds of data are essential to understanding these devices as well as in learning about artificial vision more generally, and thus deserve careful consideration by both researchers and clinicians who are developing and implanting these and similar devices, as well as to individuals and their families who are considering receiving them.

The better vision rehabilitation specialists are able to understand the recipient’s qualitative experience, the more they are enabled to assist them in learning to use the device. The more that early-stage researchers understand about how the parameters of the device correspond to perceptual experience, the better they are able to optimize design and implementation strategies. Finally, communicating these data to individuals and their families who are considering being implanted is essential. It would contribute to a more accurate understanding of the qualitative experience and process they are signing on for, and thus is an important part of informed consent. It would also help to address certain psychosocial difficulties we found recipients to experience. For instance, we found that the recipient’s percept does not change over time—that instead the recipient becomes better able at interpreting the signal received using cognitive techniques. It is a subtle distinction, but a profound one in terms of conditioning expectations around these devices—both of the researchers and the recipients. We found that current rhetoric employed by researchers and vision rehab specialists regarding neuroplasticity and the ability for recipients to transform the signal with enough practice has created a situation in which failure of the recipient to significantly transform the signal over time is perceived as a failure of the recipient (behaviorally, where the recipient is deemed to have insufficiently practiced using of the device, and/or physiologically, where the problem is located within the recipient’s eye or visual system). By shifting the expectation that it is not the percept itself, but the recipient’s ability to use the percept over time that can improve, one can potentially avoid and address the psychosocial distress that we found some recipients experienced as a result.

This study had several limitations, first and foremost the number of recipients limited by small study populations and availability of recipients. Future studies of these devices would do well to include similar qualitative reports from recipients, either as primary focus or as supplement to other outcome measures (i.e. as “mixed methods” studies that combine qualitative and quantitative methodologies). In addition, qualitative reports are only one type of data and are not meant to replace other forms of data being collected on these devices. Rather, we believe they deserve special attention because they have been heretofore neglected in the literature despite their potential to provide valuable information not captured by normative functional outcome measures. Qualitative data about recipients’ perceptual experience can both inform device design and rehabilitative techniques, as well as grant a more holistic understanding of the phenomenon of artificial vision. In addition to contributing to the larger body of work on visual prostheses, this study serves as a case example of the kind of data mobilized by qualitative, ethnographic methodology—in particular phenomenological inquiry—in study of brain machine interface devices.