For theatre audiences on a spectrum from D/deaf to hard of hearing, it is often difficult to keep up with performances. Even in cases where the performance is signed, or has captions, these accessibility additions often feel ‘tacked on’ and are typically located out of the action on stage, requiring audiences to share attention between the performance and the support. Working with Red Earth Theatre, a production company with a long history of “Total communication” in which actors sign on stage, we have been developing ways to deliver accessibility right into the heart of a performance.
Red Earth’s new show Soonchild, is touring the UK now, supported by funding from the University of Nottingham Smart Products Beacon, as well as the AHRC and the Arts Council. The show is captioned right across the set with beautiful, designed in, images, video and text delivered using new software developed at the Mixed Reality Laboratory.
The project team developed a software called ‘captionomatic’ which uses the principles of projection mapping to turn whole theatre sets into projection surfaces. While projection mapping itself is by no means a new concept, our approach has been to both simplify the process and to fit it into the wider theatre-tech ecology. Our innovation is to take a 3D model of the set – easily produced from the scale-model of the set pieces typically built for any performance, and project this onto the real set, using a simple system of point-matching to correctly align the physical set with its digital twin. Once that 3D model is in place, we are then able to project images, video, text and whatever else onto those set pieces respecting occlusion and creating an immersive canvas on which to display content.
We provide tools to read in the script from a word document, produce a compete set of captions, then generate the necessary cues which can be fired by QLab (or similar theatrical management software) to drive our system. Theatre designers need only edit the target locations and the look and feel of the text to create beautiful captions around their sets. Different sets of captions can be delivered for different audiences as necessary – so some shows may be fully captioned while others may only have key points highlighted. We know from our research that different audiences have different preferences for how captions are delivered, and our system allows theatre companies to quickly and confidently make adjustments – even between performances of the same show. Setup of the system in a new location takes only a few minutes, something that is absolutely necessary for touring productions.
More broadly, this new approach to projection mapping allows substantial creativity with digital media in theatre that extends beyond accessibility. Critically, it substantially reduces the technical barrier to entry of including projection-mapped media in a show. Soonchild demonstrates this with some beautiful interactions between live actors and pre-recorded media – in this case shadow puppetry, projected on set as if live.
The software was demonstrated at an accessible theatre technology day at Wolverhampton Arena theatre, and plans for additional workshops and training are in the works. Despite being developed for Soonchild, the software has been designed to be easily applicable to many different types of shows and thus is open source and free, requiring only off the shelf hardware (a PC and projector). We will also be making the hardware used in the show – a projector powerful enough to compete with theatrical lighting – available for other production companies to borrow and experiment with once Soonchild’s tour is complete.
This work was developed in partnership between Red Earth Theatre, The Mixed Reality Laboratory, The School of English and Department of Modern Languages and Cultures.
The project website is available here.
Soonchild will be performed in Nottingham at the Lakeside Arts Theatre in Nottingham on Sunday 24th November – more information can be found here.