On-World Computing: Enabling Interaction on Everyday Surfaces

Computers are now ubiquitous. However, computers and digital content have remained largely separate from the physical world – users explicitly interact with computers through small screens and input devices, and the “virtual world” of digital content has had very little overlap
with the practical, physical world. My thesis work is concerned with helping computing escape the confines of screens and devices, to spill digital content out into the physical world around us. In this way, I aim to help bridge the gap between the information-rich digital world and the
familiar environment of the physical world and allow users to interact with digital content as they would ordinary physical content. I approach this problem from many facets: from the lowlevel work of providing high-fidelity touch interaction on everyday surfaces, easily transforming these surfaces into enormous touchscreens; to the high-level questions surrounding the interaction design between physical and virtual realms. To achieve this end, building on my prior
work, I develop two physical embodiments of this new mixed-reality design: a tiny, miniaturized projector and camera system providing the hardware basis for a projected on-world interface, and a head-mounted augmented-reality head-mounted display modified to support touch interaction
on arbitrary surfaces.