As designers, our task is to facilitate solutions. When designing, prototyping and testing big screen apps, there are unique challenges, benefits, and opportunities to be highlighted and tested. We couldn’t find what we needed in the current prototyping tools… so we made our own.
With web and native apps, one can usually display the prototypes in their natural habitat of a laptop, phone or tablet. The experience of prototyping TV apps is much more disjointed as the screen and the remote are entirely separate entities
The standard method is to display the remote control and the app on the same screen, but this gives a misleading representation.
During tests, we often saw users trying to click directly on the app interface, rather than using the remote control. It’s sometimes difficult to understand that the journey between point A and point B might appear small, but it requires a sequence of 5 button-presses to get there.
In addition, user-test subjects should be sat 3–5 meters away from the display to ensure a realistic set of circumstances.
We were also tasked with designing a remote control to accompany the TV app. Developing and manufacturing a physical remote control represents a whole other set of challenges. It’s a time-consuming and hugely costly process, so it really needs to be perfect the first time around. It, therefore, makes sense to extensively test and prototype the digital interface and the physical peripheral together , allowing you to quickly iterate on both.
So what’s the best way to mock up a remote control which can be easily iterated upon? Tactile feedback is important. Touching and handling a physical object with some weight. Maybe having to point it in the right direction? Getting the feedback of a button-push.
Creating our own remote control from a banana and a makeymakey kit was one of many suggestions floated in the room, and one we’ll probably return to soon.
The solution we chose was simple and we had a fully functional proof of concept in just over two working days.
We made a website
Just like other prototyping solutions, we had the TV app and the remote control rendered side by side on each and every html page. Flows were mapped out in Sketch artboards.
With some simple CSS responsive breakpoints, we set the big-screen content to be hidden on mobile, and the remote control to be hidden on the big screen.
The remote section
Inside the mobile breakpoint, a full width & height div uses the remote control as a background image. This serves as our navigation. Pseudo-elements were created and placed absolute over the remote control’s buttons. Each of these pseudo-elements were then assigned an anchor tag.
Some of these buttons went on to become constants . Home , for example, will always go to the home page, profile will go to the profile page etc.
But most of the buttons will be contextual. The directional arrows, ‘ ok ’ & ‘ return’ will all go to different screens relative to where you in the flow.
On the big screen
There’s not really too much to say other than; this could be anything. Static images, gifs & movie files, modals, pop-ups, overlays, menus… if it’s an existing web technology, we can put it into this prototype.
But, so what, right? How can clicking through a website on a phone affect what’s happening on the TV?
This is how
Our iPhone and mac are on the same wireless network. The mac is screen-mirroring via HDMI to the TV. Nothing new here.
The magic element that brings it all together is Codekit app . It’s a websocket tool aimed at front-end designers and developers. One of its greatest features, in my opinion, is the live preview in-browser. Changes to the code and styling are updated immediately to all browsers using the unique bonjour url. Furthermore, if you click a link to change the page on one device; all the others will follow.
Now we can use the remote display on the phone to click through the prototype’s pages on the TV.
The possibilities are almost endless. The only limitation is how much time we want to put into a prototype. It might be tempting to strive for something so advanced, we’re essentially recreating the product itself.
To summarise version 1.0
- It ticked all the boxes for the problem we were trying to solve with the resources available
- We made the whole thing in about 20 hours
- It’s a highly scalable solution
- The larger the prototype gets, the harder it is to develop and maintain (as it is with other solutions). It’s easy to lose track of where everything is going, especially with non-linear flows and slap-dash naming discipline
- A lack of GUI means some basic front-end skills are required
- It’s dependent on Wi-Fi and the Apple ecosystem
- The changing of the page causes a flash every time
- Rebuild the platform with a wysiwyg/CMS/GUI
- Test conductive stickers on the phone to mimic physical buttons
- Investigate if we can utilise the sketch prototyping Output data
- Investigate if React/Vue could fix the flashing problem
- Test hardware integration such as makeymakey