Sense-making through Touch Interaction with a Picturebook App

Abstract

Many picturebooks are published today as software applications (apps) for touch devices, presenting many opportunities for sensory experiences and interaction. A person’s sense-making is embodied (or grounded) in sensory experiences and interactions, so these new technological opportunities will impact how they physically engage with and make sense of a picturebook app. However, few studies have examined touch and physical interaction with digital devices, a lacuna that is problematic in the digital age. This article poses the research question: How is touch interaction with a picturebook app facilitating or limiting sense-making? The conceptual framework for discussing this question embraces sensing, sense-making, and interaction. Two potential core paradoxes concerning digital touch devices and picturebook apps are introduced: a paradox of materiality and a paradox of interactivity. The award-winning picturebook app, Wuwu & Co., was studied through an in-depth explorative inquiry supported with diary questions. The inquiry identified several examples of how the picturebook app facilitated sense-making, including how its virtual materiality evoked past experiences of physical materials, how it evoked empathy in the researcher, and how the story could evoke particular reactions and emotions in the researcher. The inquiry identified limitations in the app related to possibilities of exploring, predetermined possibilities of acting, and how the device influenced sensory perception. The study indicates that the app provides rich opportunities for cooperation; however, this cooperation extends only to co-option, not to co-creation. These findings are useful for future users, facilitators, and those involved in future app development, because it suggests limitations in the medium and improvements that could enhance sense-making through active, co-creating, touch interaction.