Prototyping a wearable communication system
MY ROLE
CONTRIBUTION
SCOPE
view one of the award pages to see the completed solution:
I’m interested in →
Designing the Inquiry
The first design decision was deciding how understanding itself should be constructed.
The first challenge was not methodological. It was epistemic.
Our team had no direct experience of Deaf culture, no fluency in ASL, and no personal knowledge of what it means to rely on interpretation for everyday interaction. Before designing anything, I needed to acknowledge the limits of what we knew and build a research strategy capable of addressing them before our assumptions became embedded in the solution.
For that reason, I deliberately chose not to begin by interviewing primary users. I was concerned that we would ask questions framed by an incomplete understanding of the communication landscape. Instead, I sampled subject-matter experts (interpreters, educators, and advocates whose work moves continuously between Deaf and hearing communities) and organized semi-structured interviews around open-ended topic guides.
Their position between both communities gave us a foundation for understanding the linguistic, cultural, and interpretive dimensions of the problem before asking Deaf participants to respond to a proposed intervention.
Producing Insight
Following our data collection, the challenge was understanding what the interviews revealed collectively.
I led the analysis using Reflexive Thematic Analysis because the goal was not simply to catalogue recurring statements, but to interpret how participants understood communication and what those perspectives demanded from the design.
The team first immersed ourselves independently in the transcripts and compared our interpretations; I then led the formal coding process in NVivo, repeatedly moving between transcripts, reflexive notes, codes, and thematic maps as the structure of the findings developed.
Four interconnected themes changed our understanding of the problem and, importantly, each created a responsibility for what we could responsibly design.
Theme 1: Being Bilingual in the Deaf Community
For many Deaf individuals, ASL is a first language while English is a second language structured very differently. Participants described ASL not simply as a means of exchanging information, but as a place of community and belonging; one described entering an ASL-dominant space as, “I’m really at home… I can breathe now.”
This meant the design could not treat ASL merely as input to be converted into English. It had to support ASL as a primary mode of expression and keep the language culturally central to the interaction.
Theme 2: Interpretation Constructs Meaning
Participants repeatedly emphasized that interpretation is not word-for-word translation. Meaning in ASL is constructed through facial expression, body posture, spatial grammar, directionality, movement, and context as much as through the hands. As one interpreter described it, interpretation is a “nuanced rendition” reconstructed into a three-dimensional message that will be meaningful to another person.
Full-body expression therefore became non-negotiable. Any system based only on recognizing hand movement would begin from an incomplete representation of the language, regardless of how accurately it recognized individual signs.
Theme 3: Working through Caption Technology
Many existing accessibility tools move from spoken English to written English. Our participants challenged the assumption that this necessarily makes communication accessible. When ASL is a person’s primary language, communicating through written English may require moving through a second linguistic system with fundamentally different grammar and structure.
The intervention therefore could not solve access simply by replacing speech with text. Deaf users needed a way to participate through ASL rather than being required to communicate through English alone.
Theme 4: Automation is not able to Interpret
Participants were particularly critical of automated systems that attempt to interpret ASL through isolated hand movements. Their concern was not simply that the technology remained inaccurate, but that many approaches misunderstood what needed to be interpreted in the first place.
This established a broader criterion: the system needed to preserve expressive meaning and context rather than pursue a surface-level mapping between signs and English words.
Reframing The Problem
The SME interview did not simply reveal answers, but changed our understanding of the project.
The challenge was determining what this new understanding demanded from a design perspective. I led the process of moving between the research evidence and possible directions for intervention, asking what responsibilities the findings created for anything we chose to build.
Together, the themes established four requirements. Full-body expression had to be preserved, because meaning in ASL extends beyond hand movement into facial expression, posture, directionality, and spatial grammar. ASL had to remain a primary mode of expression, rather than functioning only as input to be converted into English. Interpretation had to preserve meaning and context, rather than reduce communication to a literal mapping between signs and words. And ASL had to remain culturally central to the interaction, recognizing it as a language through which identity and belonging are expressed rather than as an accommodation around English.
These were not features added to a predetermined product. They were constraints on what an appropriate intervention could be, derived from the understanding the research had produced.
Prototyping an Experience
Our research had produced a usable understanding of the problem but could not show how a new interaction might be experienced. Participants needed to encounter our proposal directly through use.
I developed two functional experience prototypes for formative testing: a mobile conversational interface addressing direct exchange, and wearable glasses addressing environmental awareness. Both were built at enough fidelity that participants could respond to an experience rather than to a speculative description.
Conversational Prototype — simulated live ASL exchange
The conversational prototype used the visual interface developed by our UI lead, which I translated into a responsive Kivy application running natively in Python on a touchscreen LattePanda. I constructed the interaction using a Wizard-of-Oz system: two conversational scenarios were preprogrammed, and I worked with our head interpreter to record ASL interpretations for anticipated responses. Participants could answer either through English text or by selecting prerecorded ASL responses, allowing an apparently live interpreted exchange to be experienced before the underlying interpretation technology existed.
Wearable Prototype — environmental awareness in motion
The glasses addressed a different problem: persistent awareness while moving through a hearing environment. Rather than mediating only direct conversation, they provided visual cues about surrounding auditory events and their direction; whether someone outside the wearer’s field of view was speaking to them, an alarm was sounding, or a vehicle was approaching.
I built the system around an RP2350 microcontroller and XMOS XVF3800 voice processor. The XVF3800 handled microphone-array processing and direction-of-arrival estimation while the RP2350 managed the surrounding interaction and classification logic. When the available XVF3800 documentation did not expose the communication I needed, I captured its signal traffic with a logic analyzer and reconstructed the command sequences required to make the processors communicate. I also trained and deployed a lightweight Edge Impulse audio-classification model within the RP2350’s processing constraints, ultimately distinguishing human speech from non-human environmental sounds—the level of classification required for the formative encounters.
After validating the architecture on development hardware, I designed custom PCBs integrating the processor, microphone-array interface, LEDs, and power management, had them manufactured, assembled and brought up the boards, and adapted the firmware to the final hardware. I designed the glasses around those electronics using PLA internal structures and milled cellulose-acetate outer components, engineering the hinges to route flex cables without restricting temple movement. The technical objective throughout was not product readiness; it was to make the proposed experience real enough to be encountered and questioned.
Validation by Encounter
Experience became the final form of evidence.
The final stage brought both prototypes into formative testing with the Atlanta Area School for the Deaf and the Three Rivers Association for the Deaf. Together the two groups gave us participants ranging from students in grades 4–12 to adults approximately 30–68 years old, allowing us to encounter different communication practices, stages of life, and relationships with technology.
I adapted the research method to each group rather than requiring every participant to respond in the same way. At the school, I created drawing and diagramming activities so students could express reactions visually without depending on written English or having every response mediated through an interpreter. With adult participants, we combined written responses with individual interviews conducted with interpreter support, allowing more detailed accounts of situations where the prototypes might matter.
Participants extended the role of the glasses beyond the short exchanges that had initially motivated the project. They connected environmental awareness to a more persistent understanding of what was happening around them—for example, knowing whether a grandchild was crying in another room. The encounter shifted the wearable from a tool for isolated auditory events toward a broader question of environmental awareness while moving through everyday life.
The conversational prototype produced a different finding. Participants distinguished it from notebooks and transcription tools because ASL was available as a means of responding. Even participants comfortable with written English found value in being able to answer through prerecorded ASL. The important quality was not simply translation accuracy; it was the orientation of the interaction toward ASL as a legitimate primary mode of expression rather than something that needed to disappear into English.
Research changed the premise of the project before we designed, prototyping made that understanding tangible, and direct encounters revealed implications neither stage could have produced alone.
The project developed into DARI, and was subsequently recognized with a Red Dot Design Award and an Indigo Design Award and selected for exhibition at Prototypes for Humanity. More importantly, it changed my understanding of responsible design. Design cannot begin with what technology is capable of doing. It has to begin with the discipline of understanding what a lived situation actually demands—especially when designing for experiences that are not your own.
view one of the award pages to see the completed solution:
↑
back to top
made by me for you, 2026