Publications

This work introduces a real-time intention decoding algorithm grounded in muscle synergies (Syn-ID). The algorithm detects the electromyographic (EMG) onset and infers the direction of the movement during reaching tasks to control a powered shoulder-elbow exoskeleton. Features related to muscle synergies are used in a Gaussian Mixture Model and probability accumulation-based logic to infer the user’s movement direction. The performance of the algorithm was verified by a feasibility study including eight healthy participants. The experiments comprised a transparent session, during which the exoskeleton did not provide any assistance, and an assistive session in which the Syn-ID strategy was employed. Participants were asked to reach eight targets equally spaced on a circumference of 25 cm radius (adjusted chance level: 18.1%). The results showed an average accuracy of 48.7% after 0.6 s from the EMG onset. Most of the confusion of the estimate was found along directions adjacent to the actual one (type 1 error: 33.4%). Effects of the assistance were observed in a statistically significant reduction in the activation of Posterior Deltoid and Triceps Brachii. The final positions of the movements during the assistive session were on average 1.42 cm far from the expected ones, both when the directions were estimated correctly and when type 1 errors occurred. Therefore, combining accurate estimates with type 1 errors, we computed a modified accuracy of 82.10±6.34%. Results were benchmarked with respect to a purely kinematics-based approach. The Syn-ID showed better performance in the first portion of the movement (0.14 s after EMG onset).

Robotic waste sorting poses significant challenges in both perception and manipulation, given the extreme variability of objects that should be recognized on a cluttered conveyor belt. While deep learning has proven effective in solving complex tasks, the necessity for extensive data collection and labeling limits its applicability in real-world scenarios like waste sorting. To tackle this issue, we introduce a data augmentation method based on a novel GAN architecture called wasteGAN. The proposed method allows to increase the performance of semantic segmentation models, starting from a very limited bunch of labeled examples, such as few as 100. The key innovations of wasteGAN include a novel loss function, a novel activation function, and a larger generator block. Overall, such innovations helps the network to learn from limited number of examples and synthesize data that better mirrors real-world distributions. We then leverage the higher-quality segmentation masks predicted from models trained on the wasteGAN synthetic data to compute semantic-aware grasp poses, enabling a robotic arm to effectively recognizing contaminants and separating waste in a real-world scenario. Through comprehensive evaluation encompassing dataset-based assessments and real-world experiments, our methodology demonstrated promising potential for robotic waste sorting, yielding performance gains of up to 5.8% in picking contaminants. The project page is available at https://github.com/bach05/wasteGAN.git.

Panoramic cameras offer a 4π steradian field of view, which is desirable for tasks like people detection and tracking since nobody can exit the field of view. Despite the recent diffusion of low-cost panoramic cameras, their usage in robotics remains constrained by the limited availability of datasets featuring annotations in the robot space, including people’s 2D or 3D positions. To tackle this issue, we introduce PanNote, an automatic annotation tool for people’s positions in panoramic videos. Our tool is designed to be cost-effective and straightforward to use without requiring human intervention during the labeling process and enabling the training of machine learning models with low effort. The proposed method introduces a calibration model and a data association algorithm to fuse data from panoramic images and 2D LiDAR readings. We validate the capabilities of PanNote by collecting a real-world dataset. On these data, we compared manual labels, automatic labels and the predictions of a baseline deep neural network. Results clearly show the advantage of using our method, with a 15-fold speed up in labeling time and a considerable gain in performance while training deep neural models on automatically labelled data.

Brain-computer interfaces (BCIs) represent an alternative channel of communication between the user and the external environment, circumventing the need for traditional neural pathways. The capability to modulate one’s own electroencephalogram (electroencephalography (EEG)) signal holds the potential to facilitate specific movements in external devices, thereby restoring or enhancing certain abilities that may have been compromised. In this study, we propose two potential configurations for a four-class, closed loop, real-time Motor Imagery (MI) brain-computer interface (BCI), with the objective of assessing the viability of these endogenous BCIs in accurately directing a cursor on the screen. Twelve healthy participants and one individual with motor disability participated in the experiment, with nine of them successfully transitioning from one-dimensional to two-dimensional cursor control. This outcome suggests that proficient control is achievable with sufficient training time.

Tremor is a significant movement disorder characterized by involuntary, rhythmic, oscillatory movement of body parts. Traditional methods for tremor detection and analysis rely on visual evaluation and the use of rating scales. However, marker-less pose estimation (MPE) holds great potential for advancing tremor research by enabling the collection of objective and feature-rich data using a simple camera-based setup. This article aims to demonstrate the applicability of MPE for the extraction of relevant tremor features from hand kinematics and the automation of tremor detection and classification. We conducted an experiment involving three healthy subjects performing movements while a weak tremor was induced with functional electrical stimulation (FES). From multi-perspective camera data, we computed the trajectories of 20 key-points of the hand using the marker-less estimator Anipose. After extracting features from these key-point trajectories we trained machine-learning models to assess their validity in differentiating between tremor and non-tremor signals (detection) and between intention and constant tremor (classification). Despite a small intensity of FES-induced tremor, our system could detect tremor with 70.73% accuracy and classify between intention tremor and non-intention tremor with 79.28% accuracy. In conclusion, this research provides a foundation for the development of an MPE-based method for automated tremor assessment at home, using simple camera-based equipment.

Powered lower limb exoskeletons (LLEs) have emerged as wearable robots designed to augment users’ locomotion capabilities, offering mechanical support and additional power for both healthy and impaired subjects. However, current assistive exoskeletons are limited by predefined motion trajectories, hindering adaptability to unstructured environments encountered in daily life. To address this limitation, this paper proposes an environment-adaptive gait planning (EAGP) solution. The approach integrates scene understanding, pose estimation, and adaptive gait planning modules. A novel Collision-Free Foot Trajectory Generator (CFFTG) algorithm facilitates obstacle avoidance by computing collision-free foot trajectories, enhancing safety and adaptability. Through inverse kinematics, the planned trajectories are converted into angular joint trajectories for execution by low-level control. This comprehensive framework aims to enhance the adaptability and safety of LLEs, paving the way for broader real-world applications beyond clinical and research settings.

Semantic segmentation of point clouds is an essential task for understanding the environment in autonomous driving and robotics. Recent range-based works achieve real-time efficiency, while point- and voxel-based methods produce better results but are affected by high computational complexity. Moreover, highly complex deep learning models are often not suited to efficiently learn from small datasets. Their generalization capabilities can easily be driven by the abundance of data rather than the architecture design. In this paper, we harness the information from the three-dimensional representation to proficiently capture local features, while introducing the range image representation to incorporate additional information and facilitate fast computation. A GPU-based KDTree allows for rapid building, querying, and enhancing projection with straightforward operations. Extensive experiments on SemanticKITTI and nuScenes datasets demonstrate the benefits of our modification in a “small data”setup, in which only one sequence of the dataset is used to train the models, but also in the conventional setup, where all sequences except one are used for training. We show that a reduced version of our model not only demonstrates strong competitiveness against full-scale state-of-the-art models but also operates in real-time, making it a viable choice for real-world case applications. The code of our method is available at https://github.com/Bender97/WaffleAndRange.

This paper explores the integration of Artificial Intelligence (AI) into the Italian school curriculum. It proposes an activity to introduce students to the mechanisms behind AI with a special emphasis on Machine Learning (ML) and its integration with a simple robotic DIY artifact, with a maker-oriented approach. The activity aims to encourage the development of problem-solving abilities and cross-cutting skills in an inclusive framework that aligns with European key competencies. The paper also discusses the use of various reference documents, ranging from the White Paper on AI to the Ethical Guidelines for Educators on the use of AI and data. The project uses machine learning and robotics to create an intelligent waste bin to identify and sort waste. This is a way to promote awareness of the European Agenda 2030, which is an essential component of the curriculum, and encourage responsible conduct that emphasizes sustainability and environmental conservation. Future research could focus on the impact of the activity on students’ curriculum-based learning and how it influences their approach to Artificial Intelligence.

In industrial environments like factories and warehouses, transportation of flexible materials that need the collaboration of several subjects is a typical activity. One instance is the handling of enormous fibre sheets in the fabrication of composite parts, which presents several difficulties, including handling flexible materials and needing to place the material with extreme precision. Recently, there has been a lot of interest in employing robots to help human workers carry such things. However, this typically entails the robot adopting a follower attitude just intended for passive support without fully using its accuracy and repeatability. To make the best possible use of the robot’s and the operator’s skills, it is necessary to use an intelligent motion planner that takes into account the ergonomics of the operator but at the same time ensures the precision required by the task. In this paper, we present a preliminary study for a Human-Aware Motion Planner for the cooperative transportation of materials.

The growing volumes of solid waste present a significant challenge for sustainable management. An efficient robotics system for sorting waste materials is essential to improving recycling and contamination removal, but it is often limited by the extreme variation of items to recognize in a cluttered and dirty environment. To bring robots in such scenario, the system must be able to recognize and manipulate different objects, adapting to a high degree of variability. A system based on deep learning can achieve high performance and fulfill these requirements. However, learning models require extensive labeled data, which limits their applicability in this context. Indeed, the complexity of the real-world environment presents a significant challenge to effective data collection, highlighting the importance of data augmentation techniques for creating a suitable dataset for training models for object recognition in this context. To address this challenge, our study investigates the use of Generative Adversarial Networks (GANs) for synthetic data generation in waste-sorting systems. GANs are employed to produce synthetic images of waste streams with corresponding labels that accurately reflect the complexity and diversity of real-world waste. A primary con-cern in synthetic data generation is ensuring alignment between generated images and their labels. To address this, we introduce a novel method for controlling the GAN generation process, which enforces semantic coherence and preserves the intended structure of the labeled data. The experiments demonstrate that semantic segmentation models trained on datasets augmented with these synthetic images perform better in the semantic segmentation of waste.