Why Non-Visual?
We experience the world with all five sensory inputs and yet a majority of learning modules rely solely on
Research
I had taken on mainly 3 forms of primary research including - interviews, observation and testing experimental prototypes with people.
In my observations, I spent a few days observing how much we respond to sensory stimuli like sights, sounds, smells, touch and taste. I mapped out my findings in relation to whether people around were more likely to respond in words (descriptive), respond by comparing the stimuli to something else (comparitive), respond by change in expression or body language (expressive) or respond by reacting physically like moving or an action (reactive).
In my observations, I spent a few days observing how much we respond to sensory stimuli like sights, sounds, smells, touch and taste. I mapped out my findings in relation to whether people around were more likely to respond in words (descriptive), respond by comparing the stimuli to something else (comparitive), respond by change in expression or body language (expressive) or respond by reacting physically like moving or an action (reactive).
In interviews, I talked to 12 people several questions ranging from the challenges of learning to multi-sensory experiences. Questions like what they think makes learning easier, how they wish they could learn skills in an ideal world, what kind of learning methods do they think is helpful and what do they imagine they would be able to do in a world where multi-sensory interfaces exist everywhere.
Sound plays a part in how we experience textures. In the form of a research activity, I presented participants with an image of different textures, the sound of said texture and a texture board to test and see if I could emulate touching the real object with the help of sound.
Storyboarding
As the first step in my process I made storyboards to illustrate the ideal user experience and establish why
Concept Prototype
In the initial stages of making, I created a video to demonstrate the concept of the project in order to get a better idea of what the final output would look like.
Final Output
Using a combination of a digital interface coded in p5.js and a physical console coded with an arduino, the final output consisted of a game play experience where the user would play as someone lost in a forest at night and their only way out is to rely on their non-visual senses to get out.
Takeaways
Based on multiple rounds of user feedback and testing, I was able to complete this version of the project. A lot of the feedback I received included discomfort in relation to the lack of visual feedback that people often expect in such experiences. Due to this feedback I added a lot of visual feedback to indicate to the player what their status is in the game. In future versions, I would like to lean into the discomfort and reduce the visual feedback further and focus completely on sound, haptics and texture.