Gaspar Ramôa

Pinpointing elements on large tactile surfaces is challenging for individuals with blindness and visual impairment (BVI) seeking to access two-dimensional (2D) information. This is particularly evident when using 2D tactile readers, devices designed to provide 2D information using static tactile representations with audio explanations. Traditional pinpointing methods, such as sighted assistance and trial-and-error, are limited and inefficient, while alternative pinpointing user interfaces (UI) are still emerging and need advancement. To address these limitations, we develop three distinct navigation UIs using a user-centred design approach: Sonar (proximity-radar sonification), Voice (direct clock-system speech instructions), and Sonoice, a new method that combines elements of both. The navigation UIs were incorporated into the Tactonom Reader device to conduct a trial study with ten BVI participants. Our UIs exhibited superior performance and higher user satisfaction than the conventional trial-and-error approach, showcasing scalability to varied assistive technology and their effectiveness regardless of graphic complexity. The innovative Sonoice approach achieved the highest efficiency in pinpointing elements, but user satisfaction was highest with the Sonar approach. Surprisingly, participant preferences varied and did not always align with their most effective strategy, underscoring the importance of accommodating individual user preferences and contextual factors when choosing between the three UIs. While more extensive training may reveal further differences between these UIs, our results emphasise the significance of offering diverse options to meet user needs. Altogether, the results provide valuable insights for improving the functionality of 2D tactile readers, thereby contributing to the future development of accessible technology.

Accessible alternatives to two-dimensional (2d) information are important in connecting people with blindness and visual impairment (BVI) to the rest of the world. Such 2d information for people with BVI includes, for example, line charts. Within line charts, it is a challenging task for people with BVI to distinguish lines. This is specifically difficult for charts with complex layouts that contain multiple lines, especially parallel lines and intersections. Tactile 2d graphic readers such as the Tactonom Reader allow access to line graph data via audio-tactile user interfaces (UIs), but these are not yet mature. Its current user interface (Tactonom Reader Version 1) combines tactile graphics (paper with Braille embossing or swell paper) with pinpointed audio explanations through camera-based fingertip recognition. Following a user-centred design, we implemented a new augmented user interface to distinguish lines of line charts in the Tactonom Reader. In addition to spoken explanations, the enhanced user interface uses acoustic feedback, supports dynamic trigger sounds and includes a component for filtering individual lines. Thus, we first evaluated the current speech user interface of the Tactonom Reader and our extended user interface for distinguishing lines of line diagrams with four participants with BVI. Beyond comparing the efficiency, effectiveness and satisfaction of both user interfaces with graphs of different complexity, we observed the current challenges of this task. The analysis revealed that the evaluated user interfaces enable people with BVI to access and distinguish lines in uncomplicated mathematical charts. Our new user interface has proved very useful in several areas but needs further development to fully enable people with BVI to understand and distinguish lines in complex mathematical diagrams.

Two-dimensional(2d) data access is crucial in connecting people with blindness and visual impairment (BVI) with the rest of the world. Such 2d information includes, for example, floor plans of train stations provided by transportation networks to help travellers navigate and understand a train station and its surroundings, but these documents are not yet accessible. Existing assistive technologies, such as 2d pin-matrix displays, offer new possibilities here to present 2d information via a refreshable tactile pin-matrix display and audio feedback. However, user interfaces that enable people with BVI to perceive the shape and content of information are still not yet fully functional and explored. This paper presents a workflow for converting PDF files of station floor plans provided by transportation networks into an accessible Scalable Vector Graphics format (SVG). Those files are not only usable for 2d pin-matrix displays but can also be printed on swell paper or can be embossed and used in 2d tactile graphic readers. Moreover, we propose a new user interface for 2d refreshable pin-matrix devices that enables people with BVI to interact with train station floor plans. This user interface was implemented for Inventivio GmbH’s Tactonom pin-matrix device (V1) and supports moving, navigating to elements and zooming in SVG train station map documents. Further work includes implementing new features to reduce the complexity and information overload on a tactile surface.

Access to complex graphical information is essential when connecting blind and visually impaired (BVI) people with the world. Tactile graphics readers enable access to graphical data through audio-tactile user interfaces (UIs), but these have yet to mature. A challenging task for blind people is locating specific elements–areas in detailed tactile graphics. To this end, we developed three audio navigation UIs that dynamically guide the user’s hand to a specific position using audio feedback. One is based on submarine sonar sounds, another relies on the target’s coordinate plan x and y-axis, and the last uses direct voice instructions. The UIs were implemented in the Tactonom Reader device, a new tactile graphic reader that enhances swell paper graphics with pinpointed audio explanations. To evaluate the effectiveness of the three different dynamic navigation UIs, we conducted a within-subject usability test that involved 13 BVI participants. Beyond comparing the effectiveness of the different UIs, we observed and recorded the interaction of the visually impaired participants with the different navigation UI to further investigate their behavioral patterns during the interaction. We observed that user interfaces that required the user to move their hand in a straight direction were more likely to provoke frustration and were often perceived as challenging for blind and visually impaired people. The analysis revealed that the voice-based navigation UI guides the participant the fastest to the target and does not require prior training. This suggests that a voice-based navigation strategy is a promising approach for designing an accessible user interface for the blind.

In this paper, we propose three methods for door state classification with the goal to improve robot navigation in indoor spaces. These methods were also developed to be used in other areas and applications since they are not limited to door detection as other related works are. Our methods work offline, in low-powered computers as the Jetson Nano, in real-time with the ability to differentiate between open, closed and semi-open doors. We use the 3D object classification, PointNet, real-time semantic segmentation algorithms such as, FastFCN, FC-HarDNet, SegNet and BiSeNet, the object detection algorithm, DetectNet and 2D object classification networks, AlexNet and GoogleNet. We built a 3D and RGB door dataset with images from several indoor environments using a 3D Realsense camera D435. This dataset is freely available online. All methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classification algorithm running in real-time on a low-power device.

In this paper, we propose two methods for door classification with the goal to help and improve robot navigation in indoor spaces and to be used in other areas and applications since it is not limited to door detection as other related works. Our methods work offline, in low-powered computers as the Jetson Nano, in real-time with the ability to differentiate between open, closed and semi-open doors. We use the 3D object classification, PointNet and real-time semantic segmentation algorithms FastFCN and FC-HarDNet. We built a 3D and RGB dataset using a 3D Realsense camera D435 with door images in several indoor environments that we make freely available. Both methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classification algorithm running in real-time on a low-power device.