Publications

With the increasing popularity of social media and smart devices, the face as one of the key biometrics becomes vital for person identification. Among those face recognition algorithms, video-based face recognition methods could make use of both temporal and spatial information just as humans do to achieve better classification performance. However, they cannot identify individuals when certain key facial areas, such as eyes or nose, are disguised by heavy makeup or rubber/digital masks. To this end, we propose a novel deep spiking neural network architecture in this paper. It takes dynamic facial movements, the facial muscle changes induced by speaking or other activities, as the sole input. An event-driven continuous spike-timing-dependent plasticity learning rule with adaptive thresholding is applied to train the synaptic weights. The experiments on our proposed video-based disguise face database (MakeFace DB) demonstrate that the proposed learning method performs very well, i.e., it achieves from 95% to 100% correct classification rates under various realistic experimental scenarios.

Recent technological advances enabled modern robots to become part of our daily life. In particular, assistive robotics emerged as an exciting research topic that can provide solutions to improve the quality of life of elderly and vulnerable people. This paper introduces the robotic platform developed in the ENRICHME project, with particular focus on its innovative perception and interaction capabilities. The project’s main goal is to enrich the day-to-day experience of elderly people at home with technologies that enable health monitoring, complementary care, and social support. The paper presents several modules created to provide cognitive stimulation services for elderly users with mild cognitive impairments. The ENRICHME robot was tested in three pilot sites around Europe (Poland, Greece, and UK) and proven to be an effective assistant for the elderly at home.

This paper presents a system for online learning of human classifiers by mobile service robots using 3D LiDAR sensors, and its experimental evaluation in a large indoor public space. The learning framework requires a minimal set of labelled samples (e.g. one or several samples) to initialise a classifier. The classifier is then retrained iteratively during operation of the robot. New training samples are generated automatically using multi-target tracking and a pair of “experts” to estimate false negatives and false positives. Both classification and tracking utilise an efficient real-time clustering algorithm for segmentation of 3D point cloud data. We also introduce a new feature to improve human classification in sparse, long-range point clouds. We provide an extensive evaluation of our the framework using a 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments demonstrate the influence of the system components and improved classification of humans compared to the state-of-the-art.

Recent technological advances enabled modern robots to become part of our daily life. In particular, assistive robotics emerged as an exciting research topic that can provide solutions to improve the quality of life of elderly and vulnerable people. This paper introduces the robotic platform developed in the ENRICHME project, with particular focus on its innovative perception and interaction capabilities. The project’s main goal is to enrich the day-to-day experience of elderly people at home with technologies that enable health monitoring, complementary care, and social support. The paper presents several modules created to provide cognitive stimulation services for elderly users with mild cognitive impairments. The ENRICHME robot was tested in three pilot sites around Europe (Poland, Greece, and UK) and proven to be an effective assistant for the elderly at home.

Sound perception is a fundamental skill for many people with severe sight impairments. The research presented in this article is part of an ongoing project with the aim to create a mobile guidance aid to help people with vision impairments find objects within an unknown indoor environment. This system requires an effective non-visual interface and uses bone-conduction headphones to transmit audio instructions to the user. It has been implemented and tested with spatialised audio cues, which convey the direction of a predefined target in 3D space. We present an in-depth evaluation of the audio interface with several experiments that involve a large number of participants, both blindfolded and with actual visual impairments, and analyse the pros and cons of our design choices. In addition to producing results comparable to the state-of-The-Art, we found that Fitts’s Law (a predictive model for human movement) provides a suitable metric that can be used to improve and refine the quality of the audio interface in future mobile navigation aids.

This paper presents the perception system of a new professional cleaning robot for large public places. The proposed system is based on multiple sensors including 3D and 2D lidars, two RGB-D cameras and a stereo camera. The two lidars together with an RGB-D camera are used for dynamic object (human) detection and tracking, while the second RGB-D and stereo camera are used for detection of static objects (dirt and ground objects). A learning and reasoning module for spatial–temporal representation of the environment based on the perception pipeline is also introduced. Furthermore, a new dataset collected with the robot in several public places, including a supermarket, a warehouse and an airport, is released. Baseline results on this dataset for further research and comparison are provided. The proposed system has been fully implemented into the Robot Operating System (ROS) with high modularity, also publicly available to the community.

Autonomous Vehicles (AVs) must interact with other road users. They must understand and adapt to complex pedestrian behaviour, especially during crossings where priority is not clearly defined. This includes feedback effects such as modelling a pedestrian’s likely behaviours resulting from changes in the AVs behaviour. For example, whether a pedestrian will yield if the AV accelerates, and vice versa. To enable such automated interactions, it is necessary for the AV to possess a statistical model of the pedestrian’s responses to its own actions. A previous work demonstrated a proof-ofconcept method to fit parameters to a simplified model based on data from a highly artificial discrete laboratory task with human subjects. The method was based on LIDAR-based person tracking, game theory, and Gaussian process analysis. The present study extends this method to enable analysis of more realistic continuous human experimental data. It shows for the first time how game-theoretic predictive parameters can be fit into pedestrians natural and continuous motion during road-crossings, and how predictions can be made about their interactions with AV controllers in similar real-world settings.

During the last 40 years, a large number of musical works was composed and recorded at the Centro di Sonologia Computazionale (CSC) of the University of Padova. The problem of how to preserve this increasing amount of audio documents arose and, in order to meet this need, the CSC started to carry out a research to develop a scientific methodology for preservation. In the last 7 years, this methodology was refined and applied to the digitization of more than 2,500 audio documents both from Italian and international audio archives, mainly stored on analogue magnetic carriers such as open-reel tapes and cassettes. Their content consists of electronic, folk and other kinds of music, as well as speech recordings. The methodology provides for collecting numerous metadata about the original carrier on which the audio information was stored, i.e. flange diameter, brand and material, its physical state and the related preservation copy, necessary for a correct preservation of the content and the contextual information. This work aims to extract and interpret the information from this wealth of data and metadata, which was collected and structured through the software PSKit. This may be useful for audio technicians, archives and policy makers to plan future preservation projects.

Providing technologies to support the visiting experience in cultural venues of artistic value is an important issue that needs to be addressed by considering the delicate nature of the places. Architectural heritage and visual arts are two valuable examples: the most sensible choice for augmenting the comprehension and the experience concerning this kind of cultural heritage is through audio cues (e.g., using audio guides). This work describes a pilot experiment to evaluate the impact of soundscapes and environmental acoustics reconstruction in the visitors’ experience. We proposed, to the visitors, an audioguide integrated with an optional experience presenting a comparison between a choir recorded in an anechoic chamber and the same choir as it would have sounded in the Church of the San Martino Charterhouse, using a preliminary acoustic survey. Experiments were conducted with real visitors to maximise the ecology of the collected data and results show that people are significantly interested towards the proposed experience, motivating further efforts to improve the quality and depth of the contents provided with this strategy.

Objectives: The purpose of this study was to assess the functional and prognostic correlates of B-lines during stress echocardiography (SE). Background: B-profile detected by lung ultrasound (LUS) is a sign of pulmonary congestion during SE. Methods: The authors prospectively performed transthoracic echocardiography (TTE) and LUS in 2,145 patients referred for exercise (n = 1,012), vasodilator (n = 1,054), or dobutamine (n = 79) SE in 11 certified centers. B-lines were evaluated in a 4-site simplified scan (each site scored from 0: A-lines to 10: white lung for coalescing B-lines). During stress the following were also analyzed: stress-induced new regional wall motion abnormalities in 2 contiguous segments; reduced left ventricular contractile reserve (peak/rest based on force, ≤2.0 for exercise and dobutamine, ≤1.1 for vasodilators); and abnormal coronary flow velocity reserve ≤2.0, assessed by pulsed-wave Doppler sampling in left anterior descending coronary artery and abnormal heart rate reserve (peak/rest heart rate) ≤1.80 for exercise and dobutamine (≤1.22 for vasodilators). All patients completed follow-up. Results: According to B-lines at peak stress patients were divided into 4 different groups: group I, absence of stress B-lines (score: 0 to 1; n = 1,389; 64.7%); group II, mild B-lines (score: 2 to 4; n = 428; 20%); group III, moderate B-lines (score: 5 to 9; n = 209; 9.7%) and group IV, severe B-lines (score: ≥10; n = 119; 5.4%). During median follow-up of 15.2 months (interquartile range: 12 to 20 months) there were 38 deaths and 28 nonfatal myocardial infarctions in 64 patients. At multivariable analysis, severe stress B-lines (hazard ratio [HR]: 3.544; 95% confidence interval [CI]: 1.466 to 8.687; p = 0.006), abnormal heart rate reserve (HR: 2.276; 95% CI: 1.215 to 4.262; p = 0.010), abnormal coronary flow velocity reserve (HR: 2.178; 95% CI: 1.059 to 4.479; p = 0.034), and age (HR: 1.031; 95% CI: 1.002 to 1.062; p = 0.037) were independent predictors of death and nonfatal myocardial infarction. Conclusions: Severe stress B-lines predict death and nonfatal myocardial infarction. (Stress Echo 2020–The International Stress Echo Study [SE2020]; NCT03049995)