The development of an app for people with visual impairment in collaboration with Sber

Intro

  • Mobile App

  • Android

  • iOS

Tiflo-commentary is a specially prepared audio description of scenes, actions, and objects that helps people with visual impairment to perceive information entirely. With the help of the «Iskusstvo.Vsluh» app, visually impaired and partially sighted people can listen to performances with tiflo-commentary and expand the boundaries of perception without using special equipment.

Screenshot
01

Intro

Theatre for visually impaired people

As part of the Golden Mask Festival 2017, Sberbank, together with the Russian Society of the Blind (RSB), decided to present the first Russian application for listening to the commentary on theatrical productions. We were invited to be a technology partner and executor of the project.

02

Task

The perfect opportunity for the the visually impaired to enjoy the theatre

To create a unique mobile app with tiflo-commentary for theatre-goers, we had to analyze available technical solutions and offer a suitable option for the theater.

It was necessary to take into account the specific features of users when designing and developing the app, so that a person with visual impairments can use applications without any difficulties.

03

Solution

Development of a unique mobile app with tiflo‑commentary

In addition to iOS and Android applications, we have developed a backend, a database, and an operator panel, which is used by a responsible employee to launch audio descriptions at the right time of the performance. This provides the simplest and most reliable system operation scenario.

Заголовок

«Our task was to create a platform that would allow theaters to attract the blind without much effort independently. The implementation of the technological solution on smartphones (which are now common among the visually impaired) is the best option to speed up the penetration of tiflo-commentary in theaters and to allow partially sighted people in any big city to enjoy a theatrical performance. We took a risk and decided to be the first who would develop such an application. Of course, we invited experienced partners to the team.»

Polina Mahold

Mobile projects and platforms marketing manager of «Sberbank»

User Experience

To start listening to the tiflo-commentary, a user only needs to come to the theater, run the application, put on the headphones and that’s it! When the performance begins, the user will hear an audio description of what is happening. Connection to Wi-Fi happens automatically on Android. iPhone technically does not have this feature, so the application tells you to which network connect to, and offers to do so.

Заголовок

«The most valuable feedback is the faces of visually impaired viewers (this is not a typo, blind people are also viewers) when they came out of the performances. It is difficult to describe with words the range of emotions and impressions read on their faces. First and foremost, it is necessary to thank the wonderful performances chosen for the start of the project, and the joy and emotions of the spectators are an additional confirmation that everything (organization, tiflo-commentary, and also the application conveying it) has been done right».

Mikhail Korneyev

The coordinator of the project «RSB-Film»

Comments synchronization

The format of the application with tiflo-commentary was used only for movies before. But unlike the cinema in the theater, the launch of the tiflo-commentary can not be automated. There can be delays, improvisation, and changes in the scenario.

We analyzed 3 ways of synchronization to link the comments to the pauses in the performances and proposed the best option in terms of implementation time, reliability, and usability. The option of using the local network turned out to be the most reliable and required minimal effort from users.

Screenshot

Screen with performances

Requirements for the theaters

In theaters, there are often problems with the connection. Sometimes there are special signal jammers, so as not to interfere with the performance. Therefore, the presence of a local network is a necessary condition for the reliable transmission of information. Each theater participating in the project was given a customized Wi-Fi router and a laptop — they just need to be plugged into the outlet and the tiflo-commentary infrastructure is ready.

Operator panel

A tiflo operator is a person who runs audio files and accompanies the performance. Similar technology is used to provide subtitles in operas. Therefore, in every theater, there is a staff member who can cope with this task. We created a simple application for running audio files. It is not difficult to work with the operator panel, and it could be easily introduced into any theater.

Заголовок

«The most valuable feedback is the faces of visually impaired viewers (this is not a typo, blind people are also viewers) when they came out of the performances. It is difficult to describe with words the range of emotions and impressions read on their faces. First and foremost, it is necessary to thank the wonderful performances chosen for the start of the project, and the joy and emotions of the spectators are an additional confirmation that everything (organization, tiflo-commentary, and also the application conveying it) has been done right.»

Mikhail Korneyev

The coordinator of the project «RSB‑Film»

Accessible interface

What should be the app interface for people who do not see the screen? How do they understand what to tap? The design and font in the application «Iskusstvo.Vsluh» are tailored to meet the needs of the visually impaired and are coordinated with the RSB. Such voice interface technology support as TalkBack for Android and VoiceOver for iOS allows even completely blind people to navigate the application.

Testing subtleties

The testers specially checked all functions with their eyes closed to make sure that the visually impaired people can use the application and to check the correctness of the voice assistant work. The Russian Society of the Blind members also took part in testing the application.

Заголовок

«Developers managed to find a simple and understandable approach for the interface implementation. Particularly pleasant is that the application remains visually pleasing to people with good vision. Often, when developing an interface for the blind, creators do not think about it and abuse the contrasts and sizes of the elements.»

Mikhail Korneyev

The coordinator of the project «RSB‑Film»

The performance atmosphere in the app

Tiflo-commentary was voiced by professional actors: Chulpan Khamatova, Yuliya Peresild, and Anatoly Bely. It allows the users to feel the theater atmosphere even more. Now people with visual impairment have an opportunity to participate in theatrical life.

Заголовок

«The more different applications there are that allow people to perceive the world that everyone else sees, the better.»

Eugen Denisov

Project manager at MobileUp

04

Resume

We have made the world a better place

It was a pleasure to take part in such a kind and bright project. In addition to interesting technical problems, we solved the main one — a human one. We helped people feel the theater better.

05

Team

Project team

  • Project Manager

    Eugene Denisov

  • Art Director

    Philip Filchenko

  • Design

    Ilyas Gayfullin

  • CTO

    Eugene Valeev

  • iOS Development

    Alexey Nemychenkov, Polina Demidova

  • Android Development

    Dmitry Gorbunov

  • Testing

    Alexandra Kochetkova, Ilya Biltuyev

  • Documentation

    Anton Vityazev

06

Other projects

07

Collaboration

We are always glad to cooperate and work on new projects

Describe the task, and we will contact you soon. Or you can always text us on Telegram.

Let’s get to know each other!

By clicking 'send', you consent to our processing your personal data and agree to our privacy policy terms.

Your application has been submitted successfully

Thank you! We will contact you soon