The development of an app for people with visual impairment in collaboration with Sber
Tiflo-commentary is a specially prepared audio description of scenes, actions, and objects that helps people with visual impairment to perceive information entirely. With the help of the «Iskusstvo.Vsluh» app, visually impaired and partially sighted people can listen to performances with tiflo-commentary and expand the boundaries of perception without using special equipment.
Theatre for visually impaired people
As part of the Golden Mask Festival 2017, Sberbank, together with the Russian Society of the Blind (RSB), decided to present the first Russian application for listening to the commentary on theatrical productions. We were invited to be a technology partner and executor of the project.
The perfect opportunity for the the visually impaired to enjoy the theatre
To create a unique mobile app with tiflo-commentary for theatre-goers, we had to analyze available technical solutions and offer a suitable option for the theater.
It was necessary to take into account the specific features of users when designing and developing the app, so that a person with visual impairments can use applications without any difficulties.
Development of a unique mobile app with tiflo‑commentary
In addition to iOS and Android applications, we have developed a backend, a database, and an operator panel, which is used by a responsible employee to launch audio descriptions at the right time of the performance. This provides the simplest and most reliable system operation scenario.
«Our task was to create a platform that would allow theaters to attract the blind without much effort independently. The implementation of the technological solution on smartphones (which are now common among the visually impaired) is the best option to speed up the penetration of tiflo-commentary in theaters and to allow partially sighted people in any big city to enjoy a theatrical performance. We took a risk and decided to be the first who would develop such an application. Of course, we invited experienced partners to the team.»
To start listening to the tiflo-commentary, a user only needs to come to the theater, run the application, put on the headphones and that’s it! When the performance begins, the user will hear an audio description of what is happening. Connection to Wi-Fi happens automatically on Android. iPhone technically does not have this feature, so the application tells you to which network connect to, and offers to do so.
«The most valuable feedback is the faces of visually impaired viewers (this is not a typo, blind people are also viewers) when they came out of the performances. It is difficult to describe with words the range of emotions and impressions read on their faces. First and foremost, it is necessary to thank the wonderful performances chosen for the start of the project, and the joy and emotions of the spectators are an additional confirmation that everything (organization, tiflo-commentary, and also the application conveying it) has been done right».
The format of the application with tiflo-commentary was used only for movies before. But unlike the cinema in the theater, the launch of the tiflo-commentary can not be automated. There can be delays, improvisation, and changes in the scenario.
We analyzed 3 ways of synchronization to link the comments to the pauses in the performances and proposed the best option in terms of implementation time, reliability, and usability. The option of using the local network turned out to be the most reliable and required minimal effort from users.
Screen with performances
Requirements for the theaters
In theaters, there are often problems with the connection. Sometimes there are special signal jammers, so as not to interfere with the performance. Therefore, the presence of a local network is a necessary condition for the reliable transmission of information. Each theater participating in the project was given a customized Wi-Fi router and a laptop — they just need to be plugged into the outlet and the tiflo-commentary infrastructure is ready.
A tiflo operator is a person who runs audio files and accompanies the performance. Similar technology is used to provide subtitles in operas. Therefore, in every theater, there is a staff member who can cope with this task. We created a simple application for running audio files. It is not difficult to work with the operator panel, and it could be easily introduced into any theater.
«The most valuable feedback is the faces of visually impaired viewers (this is not a typo, blind people are also viewers) when they came out of the performances. It is difficult to describe with words the range of emotions and impressions read on their faces. First and foremost, it is necessary to thank the wonderful performances chosen for the start of the project, and the joy and emotions of the spectators are an additional confirmation that everything (organization, tiflo-commentary, and also the application conveying it) has been done right.»
What should be the app interface for people who do not see the screen? How do they understand what to tap? The design and font in the application «Iskusstvo.Vsluh» are tailored to meet the needs of the visually impaired and are coordinated with the RSB. Such voice interface technology support as TalkBack for Android and VoiceOver for iOS allows even completely blind people to navigate the application.
The testers specially checked all functions with their eyes closed to make sure that the visually impaired people can use the application and to check the correctness of the voice assistant work. The Russian Society of the Blind members also took part in testing the application.
«Developers managed to find a simple and understandable approach for the interface implementation. Particularly pleasant is that the application remains visually pleasing to people with good vision. Often, when developing an interface for the blind, creators do not think about it and abuse the contrasts and sizes of the elements.»
The performance atmosphere in the app
Tiflo-commentary was voiced by professional actors: Chulpan Khamatova, Yuliya Peresild, and Anatoly Bely. It allows the users to feel the theater atmosphere even more. Now people with visual impairment have an opportunity to participate in theatrical life.
«The more different applications there are that allow people to perceive the world that everyone else sees, the better.»
We have made the world a better place
It was a pleasure to take part in such a kind and bright project. In addition to interesting technical problems, we solved the main one — a human one. We helped people feel the theater better.
- Project Manager
- Art Director
- iOS Development
Alexey Nemychenkov, Polina Demidova
- Android Development
Alexandra Kochetkova, Ilya Biltuyev
Let’s get to know each other!
Your application has been submitted successfully
Thank you! We will contact you soon