Publications

Recent advances in mobile technology have the potential to radically change the quality of tools available for people with sensory impairments, in particular the blind and partially sighted. Nowadays almost every smart-phone and tablet is equipped with high-resolution cameras, typically used for photos, videos, games and virtual reality applications. Very little has been proposed to exploit these sensors for user localisation and navigation instead. To this end, the “Active Vision with Human-in-the-Loop for the Visually Impaired” (ActiVis) project aims to develop a novel electronic travel aid to tackle the “last 10 yards problem” and enable blind users to independently navigate in unknown environments, ultimately enhancing or replacing existing solutions such as guide dogs and white canes. This paper describes some of the project’s key challenges, in particular with respect to the design of a user interface (UI) that translates visual information from the camera to guidance instructions for the blind person, taking into account the limitations introduced by visual impairment. In this paper we also propose a multimodal UI that caters to the needs of the visually impaired that exploits human-machine progressive co-adaptation to enhance the user’s experience and improve navigation performance.

Technological advances and affordability of recent smart sensors, as well as the consolidation of common software platforms for the integration of the latter and robotic sensors, are enabling the creation of complex active and assisted living environments for improving the quality of life of the elderly and the less able people. One such example is the integrated system developed by the European project ENRICHME, the aim of which is to monitor and prolong the independent living of old people affected by mild cognitive impairments with a combination of smart-home, robotics and web technologies. This paper presents in particular the design and technological solutions adopted to integrate, process and store the information provided by a set of fixed smart sensors and mobile robot sensors in a domestic scenario, including presence and contact detectors, environmental sensors, and RFID-tagged objects, for long-term user monitoring and adaptation.

The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) and those ones equipping modern mobile robots (e.g. RGB-D cameras) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGB-D camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment’s area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with a comprehensive dataset of RGB-D and domotic data containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and its potential for complex anomaly detection in AAL settings.

We present a system for temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, the proposed approach temporally detects intervals where individual or social activity is occurring. Recognition of human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities is useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors, which are able to characterise human interactions; (2) develop a computational model to segment temporal intervals with social interaction or individual behaviour; (3) provide a public dataset with RGB-D data with continuous stream of individual activities and social interactions. Results show that the proposed approach attained relevant performance with temporal segmentation of social activities.

The Special Issue of Robotics and Autonomous Systems presents papers from the Seventh European Conference on Mobile Robots (ECMR’15). The paper Vision-based Markov Localization for Long Term Autonomy by Benjamin Suger, Michael Ruhnke and Wolfram Burgard describes an approach to localize a mobile robot relative to an image sequence recorded in a different season and its evaluation on several challenging datasets, where the approach outperformed state-of-the-art techniques. The paper Image Features for Visual Teach-and-Repeat Navigation in Changing Environments by Tomas Krajnik, Pablo Cristoforis, Keerthy Kusumam, Peer Neubert and Tom Duckett describes an evaluation of image features for long-term visual teach-and-repeat navigation in outdoor environments with changes in appearance due to seasonal and daily variations. The paper Efficient Retrieval of Arbitrary Objects from Long-Term Robot Observations by Nils Bore, Rares Ambrus, Patric Jensfelt and John Folkesson presents a method for efficient query and retrieval of arbitrarily shaped objects from large amounts of unstructured 3D point cloud data for long-term semantic mapping by mobile service robots. The paper Time-Dependent Gas Distribution Modelling by Sahar Asadi, Han Fan, Victor Hernandez Bennetts and Achim Lilienthal describes an approach for modelling variations over time in gas distributions sensed by mobile robots and predicting future measurements. The paper Probabilistic Ego-Motion Estimation Using Multiple Automotive Radar Sensors by Matthias Rapp, Michael Barjenbruch, Markus Hahn, Juergen Dickmann and Klaus Dietmayer presents a framework for self-motion estimation based on registration of consecutive radar scans and a likelihood model for the Doppler velocity. The paper Integrated Online Trajectory Planning and Optimization in Distinctive Topologies by Christoph Roesmann, Frank Hoffmann and Torsten Bertram presents a novel approach for trajectory planning by mobile robots in the presence of dynamic obstacles such as people.

Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbor. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real-time stress detection. Finally, we present an study of the most discriminative features for stress detection.

IN October 2016, inside a sold-out arena in Zurich, a man named Numa Poujouly steered his wheelchair up to the central podium. As the Swiss national anthem played, organizers of the world’s first cyborg Olympics hung a gold medal around Poujouly’s neck. The 30-yearold, who became paralyzed after a bicycle accident in his teens, had triumphed in the tournament’s most futuristic event: A video-game-like race in which the competitors controlled their speeding avatars with just their minds.

Congenital myopathies are a group of genetic muscle disorders characterized clinically by hypotonia and weakness, usually from birth, and a static or slowly progressive clinical course. Historically, congenital myopathies have been classified on the basis of major morphological features seen on muscle biopsy. However, different genes have now been identified as associated with the various phenotypic and histological expressions of these disorders, and in recent years, because of their unexpectedly wide genetic and clinical heterogeneity, next-generation sequencing has increasingly been used for their diagnosis. We reviewed clinical and genetic forms of congenital myopathy and defined possible strategies to improve cost-effectiveness in histological and imaging diagnosis.

Knowing the force exerted by a human operator while he/she is performing a specific task is important in many different field. For instance, limiting or optimizing the effort in sport activities allows for the development of specific training patterns for athletes, while knowing the effort made by a worker when he/she lifts a weight is important from the point of view of the safety. The effective effort is not only related to the net force/torque generated, but also to the force generated by each muscle. This aspect is the most crucial to be evaluated, as a non properly designed/handled exercise/task can lead to an excessive muscle strain and, in turn, to injuries. In this paper we report some preliminary results obtained by using low-cost wearable sensors in the estimation of the weight lifted by a human operator, through the simultaneous measurement of motion (via inertial sensors) and the muscles activations (via surface ElectroMyoGraphy – sEMG). The armband has 8 sEMG sensors and a 9-DoF inertial sensor, it has electrically safe setup with low voltage battery and Bluetooth protocol. The relationship between the EMG and inertial signals and the exerted force was made using a biarticular model of the arm. The model was used in order to have a theoretical value of the shoulder and elbow torques performing a weightlifting standardized tests. The value was then compared with the one estimated by the identification of a model and by means of a neural network. In both cases, the results show the relationship between signals and torque, but, in both cases, the results are affected by error. Nevertheless, even if it doesnt accurately estimate the weight lifted, both the presented techniques highlight the possibility of developing a classification of the exerted force that, calibrating the system for each person, can identify whether the weight lifted is light or heavy.

Ocular myopathy, typically manifesting as progressive external ophthalmoplegia (PEO), is among the most common mitochondrial phenotypes. The purpose of this study is to better define the clinical phenotypes associated with ocular myopathy. This is a retrospective study on a large cohort from the database of the “Nation-wide Italian Collaborative Network of Mitochondrial Diseases”. We distinguished patients with ocular myopathy as part of a multisystem mitochondrial encephalomyopathy (PEO-encephalomyopathy), and then PEO with isolated ocular myopathy from PEO-plus when PEO was associated with additional features of multisystemic involvement. Ocular myopathy was the most common feature in our cohort of mitochondrial patients. Among the 722 patients with a definite genetic diagnosis, ocular myopathy was observed in 399 subjects (55.3%) and was positively associated with mtDNA single deletions and POLG mutations. Ocular myopathy as manifestation of a multisystem mitochondrial encephalomyopathy (PEO-encephalomyopathy, n = 131) was linked to the m.3243A>G mutation, whereas the other “PEO” patients (n = 268) were associated with mtDNA single deletion and Twinkle mutations. Increased lactate was associated with central neurological involvement. We then defined, among the PEO group, as “pure PEO” the patients with isolated ocular myopathy and “PEO-plus” those with ocular myopathy and other features of neuromuscular and multisystem involvement, excluding central nervous system. The male proportion was significantly lower in pure PEO than PEO-plus. This study reinforces the need for research on the role of gender in mitochondrial diseases. The phenotype definitions here revisited may contribute to a more homogeneous patient categorization, useful in future studies and clinical trials.