DeepNLP IROS2022 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
IEEE/RSJ International Conference on Intelligent Robots and Systems
-
Folding garments reliably and efficiently is a long standing challenge in robotic manipulation due to the complex dynamics and high dimensional configuration space of garments. An intuitive approach is to initially manipulate the garment to a canonical smooth configuration before folding. In this work, we develop SpeedFolding, a reliable and efficient bimanual system, which given user-defined inst...
-
Path planning in unknown environments remains a challenging problem, as the environment is gradually observed during the navigation, the underlying planner has to update the environment representation and replan, promptly and constantly, to account for the new observations. In this paper, we present a visibility graph-based planning framework capable of dealing with navigation tasks in both known ...
-
LiDAR-based localization and mapping is one of the core components in many modern robotic systems due to the direct integration of range and geometry, allowing for precise motion estimation and generation of high quality maps in real-time. Yet, as a consequence of insufficient environmental constraints present in the scene, this dependence on geometry can result in localization failure, happening ...
-
Training a high-dimensional simulated agent with an under-specified reward function often leads the agent to learn physically infeasible strategies that are ineffective when deployed in the real world. To mitigate these unnatural behaviors, reinforcement learning practitioners often utilize complex reward functions that encourage physically plausible behaviors. However, a tedious labor-intensive t...
-
We present RCareWorld, a human-centric simulation world for physical and social robotic caregiving designed with inputs from stakeholders. RCareWorld has realistic human models of care recipients with mobility limitations and caregivers, home environments with multiple levels of accessibility and assistive devices, and robots commonly used for caregiving. It interfaces with various physics engines...
-
In endoluminal surgery, the miniature instruments shall be of high accuracy and flexibility for minimal invasive diagnosis and surgical intervention. To this end, continuum robots with flexible joints have been proposed as the mechanism of endoscopic instruments. The compliance and deformability of the continuum joints enable access into the curved lumen. However, the manufacturing tolerances are ...
-
Humans have an incredible sense of self-preservation that is both instilled, and also learned through experience. One system which contributes to this is the pain and reflex system which both minimizes damage through involuntary reflex actions and also serves as a means of 'negative reinforcement’ to allow learning of poor actions or decision. Equipping robots with a reflex system and parallel lea...
-
Convolutional neural networks show remarkable results in classification but struggle with learning new things on the fly. We present a novel rehearsal-free approach, where a deep neural network is continually learning new unseen object categories without saving any data of prior sequences. Our approach is called RECALL, as the network recalls categories by calculating logits for old categories bef...
-
When humans grasp objects in the real world, we often move our arms to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, the grasp stability could vary widely based on the object's holding pose, as the gravitational torque and gripper ...
-
Autonomous Driving (AD) related features represent important elements for the next generation of mobile robots and autonomous vehicles focused on increasingly intelligent, autonomous, and interconnected systems. The applications involving the use of these features must provide, by definition, real-time decisions, and this property is key to avoid catastrophic accidents. Moreover, all the decision ...
-
Cleaning robots are one of the market dominators in the commercialized robot space. So far, numerous robots have been introduced that can perform cleaning tasks in various settings, including floor, pavement, pool, lawn, windows, etc. However, none of the existing commercial cleaning robots targets the staircase, commonly found in multi-story buildings. Even though few works in the literature intr...
-
This work presents a non-parametric spatiotemporal model for mapping human activity by mobile autonomous robots in a long-term context. Based on Variational Gaussian Process Regression, the model incorporates prior information of spatial and temporal-periodic dependencies to create a continuous representation of human occurrences. The inhomogeneous data distribution resulting from movements of the...
-
In this paper, we report on a field study in which we employed two service robots in a bakery store as a sales promotion. Previous studies have explored public applications of service robots public such as shopping malls. However, more evidence is needed that service robots can contribute to sales in real stores. Moreover, the behaviors of customers and service robots in the context of sales promo...
-
Autonomous vehicles are designed to elevate the efficiency of assigned tasks and ensure the safety of the environment in which they operate. This paper presents a research study focused on shared autonomy using a multi-layer fuzzy logic framework to build a relationship between an autonomous self-reconfigurable robot and a human user by switching control to the teleoperator to assist the robot whe...
-
Autonomous navigation in highly populated areas remains a challenging task for robots because of the difficulty in guaranteeing safe interactions with pedestrians in unstructured situations. In this work, we present a crowd navigation control framework that delivers continuous obstacle avoidance and post-contact control evaluated on an autonomous personal mobility vehicle. We propose evaluation me...
-
Area coverage is demanded from the robots utilized in application domains such as floor cleaning. Even though many advanced coverage algorithms have been developed, the area coverage performance is limited due to the inaccessibility of narrow spaces caused by physical constraints. Reconfigurable robots have been introduced to overcome this limitation where reconfigurability could help in assessing...
-
Service robots are mobile autonomous robots, often operating in uncertain and difficult environments. While being increasingly popular, engineering service robots is challenging. Especially, evolving them from prototype to deployable product requires effective validation and verification, assuring the robot's correct and safe operation in the target environment. While testing is the most common va...
-
Given an object, an environment, and a goal pose, how should a robot make contact to move it? Solving this problem requires reasoning about rigid-body dynamics, object and environment geometries, and hybrid contact mechanics. This paper proposes a hierarchical framework that solves this problem in 2D worlds, with polygonal objects and point fingers. To achieve this, we decouple the problem in thre...
-
To economically deploy robotic manipulators the programming and execution of robot motions must be swift. To this end, we propose a novel, constraint-based method to intuitively specify sequential manipulation tasks and to compute time-optimal robot motions for such a task specification. Our approach follows the ideas of constraint-based task specification by aiming for a minimal and object-centri...
-
Contact-rich manipulation is challenging due to dynamically-changing physical constraints by the contact mode changes undergone during manipulation. This paper proposes a versatile local planning and control framework for contact-rich manipulation that determines the continuous control action under variable contact modes online. We model the physical characteristics of contact-rich manipulation by...
-
With the field of rigid-body robotics having matured in the last fifty years, routing, planning, and manipulation of deformable objects have recently emerged as a more untouched research area in many fields ranging from surgical robotics to industrial assembly and construction. Routing approaches for deformable objects which rely on learned implicit spatial representations (e.g., Learning-from-Dem...
-
This paper is about fast slosh-free fluid transportation. Existing approaches are either computationally heavy or only suitable for specific robots and container shapes. We model the end effector as a point mass suspended by a spherical pendulum and study the requirements for slosh-free motion and the validity of the point mass model. In this approach, slosh-free trajectories are generated by cont...
-
Autonomous fabric manipulation is a longstanding challenge in robotics, but evaluating progress is difficult due to the cost and diversity of robot hardware. Using Reach, a cloud robotics platform that enables low-latency remote execution of control policies on physical robots, we present the first systematic benchmarking of fabric manipulation al-gorithms on physical hardware. We develop 4 novel ...
-
Neural Networks are the state-of-the-art technology for environmental perception in applications such as autonomous driving. However, they require a large amount of training data in order to perform well, making the selection and annotation of sensor data a time-consuming and expensive task. Active learning is a promising approach to reduce the required amount of training data by selecting samples...
-
This paper proposes a novel 3D single-object tracker to more stably, accurately, and faster track objects, even if they are temporarily missed. Our idea is to utilize spatial-temporal data association to achieve object tracking robustly, and it consists of two main parts. We firstly employ a temporal motion model cross frames to estimate the object's temporal information and update the region of i...
-
Traditional video-based human activity recognition has experienced remarkable progress linked to the rise of deep learning, but this effect was slower as it comes to the downstream task of driver behavior understanding. Understanding the situation inside the vehicle cabin is essential for Advanced Driving Assistant System (ADAS) as it enables identifying distraction, predicting driver's intent and...
-
Experienced human drivers always make safe driving decisions by selectively observing the front, rear and side- view mirrors. Several end - to-end methods have been pro-posed to learn driving models with multi-view visual infor-mation. However, these benchmark methods lack semantic understanding of multi-view image contents, where human drivers usually reason these information for decision making ...
-
In this paper, we present a framework to assess the quality of a pedestrian detector in an autonomous driving scenario. To do this, we exploit performance metrics from the domain of computer vision on one side and so-called threat metrics from the motion planning domain on the other side. Based on a reachability analysis that accounts for the uncertainty in future motions of other traffic particip...
-
Semantic segmentation provides scene understanding capability by performing pixel-wise classification of objects within an image. However, the sensitivity of such algorithms towards domain changes requires fine-tuning using an annotated dataset for each novel domain, which is expensive to construct and inefficient. We highlight that irrespective of the training dataset, structural properties of sc...
-
Self-supervised monocular depth estimation has been widely investigated to estimate depth images and relative poses from RGB images. This framework is promising because the depth and pose networks can be trained from just time-sequence images without the need for the ground truth depth and poses. In this work, we estimate the depth around a robot (360° view) using time-sequence spherical camera im...
-
Unmanned aerial vehicles (UAVs) have been used for many applications in recent years, from urban search and rescue, to agricultural surveying, to autonomous underground mine exploration. However, deploying UAVs in tight, indoor spaces, especially close to humans, remains a challenge. One solution, when limited payload is required, is to use micro-UAVs, which pose less risk to humans and typically ...
-
Chemical gas dispersion poses considerable threat to humans, animals and the environment. The research areas of gas source localization and gas distribution mapping aim to localize the source of gas leaks and map the gas plume respectively, in order to help the coordination of swift rescue missions. Although very similar, these two areas are often treated separately in literature. In some cases, i...
-
Rapid aerial grasping through robots can lead to many applications that utilize fast and dynamic picking and placing of objects. Rigid grippers traditionally used in aerial manipulators require high precision and specific object geometries for successful grasping. We propose RAPTOR, a quadcopter platform combined with a custom Fin Ray®gripper to enable more flexible grasping of objects with differ...
-
In this paper, we address the vision-based detection and tracking problems of multiple aerial vehicles using a single camera and Inertial Measurement Unit (IMU) as well as the corresponding perception consensus problem (i.e., uniqueness and identical IDs across all observing agents). We design several vision-based decentralized Bayesian multi-tracking filtering strategies to resolve the associatio...
-
The collective operation of robots, such as unmanned aerial vehicles (UAVs) operating as a team or swarm, is affected by their individual capabilities, which in turn is dependent on their physical design, aka morphology. However, with the exception of a few (albeit ad hoc) evolutionary robotics methods, there has been very little work on understanding the interplay of morphology and collective beh...
-
This paper deals with the control of a laser spot in the context of minimally invasive surgery of the middle ear, e.g., cholesteatoma removal. More precisely, our work is concerned with the exhaustive burring of residual infected cells after primary mechanical resection of the pathological tissues since the latter cannot guarantee the treatment of all the infected tissues, the remaining infected c...
-
Spina Bifida (SB) is a birth defect developed during the early stage of pregnancy in which there is incomplete closing of the spine around the spinal cord. The growing interest in fetoscopic Spina-Bifida repair, which is performed in fetuses who are still in the pregnant uterus, prompts the need for appropriate training. The learning curve for such procedures is steep and requires excellent proced...
-
Percutaneous coronary intervention (PCI) involves the delivery of a flexible submillimeter guidewire and existing x- ray based approaches impose significant ironing radiation. The use of magnetic resonance imaging (MRI) for intraoperative guidance has the advantages of not only being safe but also having high positioning accuracy and excellent tissue contrast. This paper develops a pneumatically d...
-
In this study, the infant Cardiac Robotic Surgical System (iCROSS) is developed to assist a surgeon in performing the patent ductus arteriosus (PDA) closure and other infant cardiac surgeries. The iCROSS is a dual-arm robot allowing two surgical instruments to collaborate in a narrow space while keeping a sufficiently large workspace. Compared with the existing surgical robotic systems, the iCROSS...
-
This paper reports the design, construction, and experimental validation of a novel hand-held robot for inoffice laser surgery of the vocal folds. In-office endoscopic laser surgery is an emerging trend in Laryngology: It promises to deliver the same patient outcomes of traditional surgical treatment (i.e., in the operating room), at a fraction of the cost. Unfortunately, office procedures can be ...
-
This paper presents a reduction mechanism for robot actuators that can switch between two types of reduction ratio. By fixing the carrier or ring gear of the proposed actuator which is based on the 3K compound planetary drive, the actuator can shift its reduction ratio. For compact design with reduced weight of the actuator, unique pawl brake mechanism interacting with cams and micro servos for sw...
-
In this paper, we propose a new brachiation robot, a single-rod brachiation robot. Brachiation is a method of locomotion that makes clever use of gravity and has been tried to apply to robots. Conventional brachiation robots are multiple-pendulum-like robots that mimic a gibbon. Although the multiple-pendulum-like robot can easily change the length of one brachiation step by joints, it has complex...
-
Building humanoid robots with properties similar to those of humans in terms of strength and agility is a great and unsolved challenge. This work introduces a compact and lightweight wrist joint mechanism that is singularity-free and has large range of motion. The mechanism provides two degrees of freedom (DoF) and was developed for integration into a human scale humanoid robot arm. It is based on...
-
In biomedical engineering, robotic implants provide new methods to restore and improve bodily function, and regenerate tissue. A significant challenge with the design of these devices is to safely actuate them for weeks or months, while they are residing in a patient's body. Magnetic, and other force-at-distance actuation methods, allow mechanisms to be controlled remotely and without contact or l...
-
Tendon-driven continuum robots show promise for use in surgical applications as they can assume complex configurations to navigate along tortuous paths. However, to achieve these complex robot shapes, multiple segments are required as each robot segment can bend only with a single constant curvature. To actuate these additional robot segments, multiple tendons must typically be added on-board the ...
-
We propose a novel toroidal origami monotrack capable of smooth-skin driving and bending for closed-skin-drive robots. Monotracks are a promising solution for achieving high mobility in unstructured environments. Toroidal-drive mechanisms enable whole skin drive; however, conventional methods experience unexpected wrinkling and buckles that lead to a large resistance. In this study, we propose an ...
-
Exploring unknown environments, such as caves or planetary surfaces, requires a quick understanding of the surroundings. Beforehand, only aerial footage from satellites or images from previous missions might be available. The proposed ensemble based anomaly detection framework utilizes previously gained knowledge and incorporates it with insights gained during the mission. The modular system consi...
-
The feature pyramid, which is a vital component of the convolutional neural networks, plays a significant role in several perception tasks, including object detection for autonomous driving. However, how to better fuse multi-level and multi-sensor feature pyramids is still a significant challenge, especially for object detection. This paper presents a FocusTR (Focusing on the valuable features by ...
-
In this paper, we present methods for capturing 4D body shapes of swimming fish with affordable small training datasets and textureless 2D videos. Automated capture of spatiotemporal animal movements and postures is revolutionizing the study of collective animal behavior. 4D (including 3D space + time) shape data from animals like schooling fish contains a rich array of social and non-social infor...
-
Highly accurate multi-person pose estimation at a high framerate is a fundamental problem in autonomous driving. Solving the problem could aid in preventing pedestrian-car accidents. The present study tackles this problem by proposing a new model composed of a feature pyramid and an original head to a general backbone. The original head is built using lightweight CNNs and directly estimates multi-...
-
To navigate safely, it is essential for a robot to detect all kinds of moving objects that could possibly interfere with the own trajectory. For common object classes, like cars, regular pedestrians, and trucks, there are large scale datasets as well as corresponding machine learning techniques, which provide remarkable results in commonly available detection benchmarks. A big challenge that remai...
-
In this paper we propose a novel point cloud generator that is able to reconstruct and generate 3D point clouds composed of semantic parts. Given a latent representation of the target 3D model, the generation starts from a single point and gets expanded recursively to produce the high-resolution point cloud via a sequence of point expansion stages. During the recursive procedure of generation, we ...
-
In an attempt to imitate the success of transformers in the field of natural language processing into computer vision tasks, vision transformers (ViTs) have recently gained attention. Performance breakthroughs have been achieved in coarse-grained tasks like classification. However, dense prediction tasks, such as detection, segmentation, and depth estimation, require additional modifications and h...
-
We propose DeepFusion, a modular multi-modal architecture to fuse lidars, cameras and radars in different combinations for 3D object detection. Specialized feature extractors take advantage of each modality and can be exchanged easily, making the approach simple and flexible. Extracted features are transformed into bird's-eye-view as a common representation for fusion. Spatial and semantic alignme...
-
In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or ball query among points, making the resulting network inappropriate for time critical applications. O...
-
This paper presents a proof-of-concept soft robotic glove that provides haptic feedback to the surgeon's hand during interventional endoscopy procedures, specifically colonoscopy. The glove is connected to a force sensing soft robotic sleeve that is mounted onto a colonoscope. The glove consists of pneumatic actuators that inflate in proportion to the incident forces on the soft robotic sleeve. Th...
-
While haptics research has traditionally focused on the fingertips and hands, other locations on the body provide large areas of skin that could be utilized to relay large-area haptic sensations. Researchers have thus developed wearable devices that use distributed vibrotactile actuators and distributed pneumatic force displays, but these methods have limitations. In prior work, we presented a nov...
-
In interacting with stiff environments through teleoperated systems, time delays cause a mismatch between haptic feedback and the expected feedback by the operator. This mismatch causes artefacts in the feedback, which decrease transparency, but so does filtering these artefacts. Through modelling of operator stiffness and the expected feedback force with EMG, the artifacts can be selectively filt...
-
In-hand pivoting is one of the important manipulation skills that leverage robot grippers' extrinsic dexterity to perform repositioning tasks to compensate for environmental uncertainties and imprecise motion execution. Although many researchers have been trying to solve pivoting problems using mathematical modeling or learning-based approaches, the problems remain as open challenges. On the other...
-
Despite non-co-location, haptic stimulation at the wrist can potentially provide feedback regarding interactions at the fingertips without encumbering the user's hand. Here we investigate how two types of skin deformation at the wrist (normal and shear) relate to the perception of the mechanical properties of virtual objects. We hypothesized that a congruent mapping (i.e. when the most relevant in...
-
Relocation of haptic feedback from the fingertips to the wrist has been considered as a way to enable haptic interaction with mixed reality virtual environments while leaving the fingers free for other tasks. We present a pair of wrist-worn tactile haptic devices and a virtual environment to study how various mappings between fingers and tactors affect task performance. The haptic feedback rendere...
-
Tactile feedback is necessary for closing the sen-sorimotor loop in prosthetic and tele-operable control, which would allow for more precise manipulation and increased acceptance of use of such devices. Pressure stimuli are commonly presented to users in haptic devices through a sensory substitution to vibration. The precise nature of this substitution affects pressure sensitivity, as well as the ...
-
Existing work in physical robot caregiving is limited in its ability to provide long-term assistance. This is majorly due to (i) lack of well-defined problems, (ii) diversity of tasks, and (iii) limited access to stakeholders from the caregiving community. We propose Structuring Physically Assistive Robotics for Caregiving with Stakeholders-in-the-loop (SPARCS) to address these challenges. SPARCS ...
-
We present a predictive system for non-prehensile, physics-based motion planning in clutter with a human-in-the-loop. Recent shared-autonomous systems present motion planning performance improvements when high-level reasoning is provided by a human. Humans are usually good at quickly identifying high-level actions in high-dimensional spaces, and robots are good at converting high-level actions int...
-
People and robots may need to cross each other in narrow spaces when they are sharing environments. It is expected that autonomous robots will behave in these contexts safely but also show social behaviors. Thereby, developing an acceptable behavior for autonomous robots in the area mentioned above is a foreseeable problem for the Human-Robot Interaction (HRI) field. Our current work focuses on in...
-
In hybrid industrial environments, workers' comfort and positive perception of safety are essential requirements for successful acceptance and usage of collaborative robots. This paper proposes a novel human-robot interaction framework in which the robot behaviour is adapted online according to the operator's cognitive workload and stress. The method exploits the generation of B-spline trajectorie...
-
Ultrasound (US) imaging is a common but physically demanding task in the medical field, and sonographers may need to put in considerable physical effort for producing high-quality US images. During physical human-robot interaction on US imaging, robot compliance is a critical feature that can ensure human user safety while automatic force regulation ability can help to improve task performance. Ho...
-
In human-robot collaboration, robot errors are inevitable—damaging user trust, willingness to work together, and task performance. Prior work has shown that people naturally respond to robot errors socially and that in social interactions it is possible to use human responses to detect errors. However, there is little exploration in the domain of nonsocial, physical human-robot collaboration such ...
-
Self-body awareness refers to the recognition of one's body as one's own and consists of two senses: “sense of body ownership” and “sense of agency.” In telexistence/telepresence robot operation, time delays in the robot's motion degrade self-body awareness of the robot body. We investigated how self-body recognition can be affected in a telexistence robot operation in a VR space when the robot is...
-
The visual attention of navigators is imperative to understand the logic of navigation as well as the surveillance of navigators' status and operation. Current studies are implemented with the help of wearable eye-tracker glasses; yet, the high expenditure demanded by such equipment and service and its limitations on usability have impeded related research from further development. In this letter,...
-
Recovering depth information from a single image is a long-standing challenge, and self-supervised depth estimation methods have gradually attracted attention due to not relying on high-cost ground truth. Constructing an accurate photometric loss based on photometric consistency is crucial for these self-supervised methods to obtain high-quality depth maps. However, the photometric loss in most st...
-
Depth estimation and semantic edge detection are two key tasks in computer vision, which have made great progress. To date, how to associatively predict the depth and the semantic edge is rarely explored. In this work, we first propose a flexible two-branch framework that can make the two tasks take advantage of each other, achieving a win-win situation. Specifically, for the semantic edge detecti...
-
Current RGB-based 6D object pose estimation methods have achieved noticeable performance on datasets and real world applications. However, predicting 6D pose from single 2D image features is susceptible to disturbance from changing of environment and textureless or resemblant object surfaces. Hence, RGB-based methods generally achieve less competitive results than RGBD-based methods, which deploy ...
-
The problem of image change detection via every-day indoor robot navigation is explored from a novel perspective of the self-attention technique. Detecting semantically non-distinctive and visually small changes remains a key challenge in the robotics community. Intuitively, these small non-distinctive changes may be better handled by the recent paradigm of the attention mechanism, which is the ba...
-
In this paper, we introduce a new concept of incorporating factorized flow maps as mid-level representations, for bridging the perception and the control modules in modular learning based robotic frameworks. To investigate the advantages of factorized flow maps and examine their interplay with the other types of mid-level representations, we further develop a configurable framework, along with fou...
-
Visual action planning particularly excels in applications where the state of the system cannot be computed explicitly, such as manipulation of deformable objects, as it enables planning directly from raw images. Even though the field has been significantly accelerated by deep learning techniques, a crucial requirement for their success is the availability of a large amount of data. In this work, ...
-
Currently, task-oriented grasp detection approaches are mostly based on pixel-level affordance detection and semantic segmentation. These pixel-level approaches heavily rely on the accuracy of a 2D affordance mask, and the generated grasp candidates are restricted to a small workspace. To mitigate these limitations, we firstly construct a novel affordance-based grasp dataset and propose a 6-DoF ta...
-
Articulated objects pose a unique challenge for robotic perception and manipulation. Their increased number of degrees-of-freedom makes tasks such as localization computationally difficult, while also making the process of realworld dataset collection unscalable. With the aim of addressing these scalability issues, we propose Neural Articulated Radiance Fields (NARF22), a pipeline which uses a ful...
-
Inter-agent relative localization is critical for any multi-robot system operating in the absence of external positioning infrastructure or prior environmental knowledge. We propose a novel inter-agent relative 2D pose estimation system where each participating agent is equipped with several ultra-wideband (UWB) ranging tags. Prior work typically supplements noisy UWB range measurements with addit...
-
Ranging information from ultra-wideband (UWB) ranging radios can be used to improve estimated navigation accuracy of a ground robot with other on-board sensors. However, all ranging-aided navigation methods demand the locations of ranging nodes to be known, which is not suitable for time-pressed situations, dynamic cluttered environments, or collaborative navigation applications. This paper descri...
-
360ST-Mapping: An Online Semantics-Guided Topological Mapping Module for Omnidirectional Visual SLAM
As an abstract representation of the environment structure, a topological map has advantageous properties for path-planning and navigation. Here we proposed an online topological mapping method, 360ST-Mapping, using omnidirectional vision. The 360° field-of-view allows the agent to obtain consistent observation and incrementally extract topological environment information. Moreover, we leverage se...
-
Long-time operations of autonomous vehicles and mobile robots in logistics and service applications are still a challenge. To avoid a continuous re-mapping, the map can be updated to obtain a consistent representation of the current environment. In this paper, we propose a novel LIDAR-based occupancy grid map updating algorithm for dynamic environments taking into account possible localisation and...
-
Intelligent robots designed to interact with hu-mans in the real world need to adapt to the preferences of different individuals. Preference-based reinforcement learning (RL) has shown great potential for teaching robots to learn personalized behaviors from interacting with humans with-out a meticulous, hand-crafted reward function, replaced by learning reward based on a human's preferences betwee...
-
Although there is extensive research regarding legged manipulators, comparatively little focuses on their User Interfaces (UIs). Towards extending the state-of-art in this domain, in this work, we integrate a Boston Dynamics (BD) Spot® with a light-weight 7 DoF Kinova® robot arm and a Robotiq® 2F-85 gripper into a legged manipulator. Furthermore, we jointly control the robotic platform using an af...
-
Interactive Reinforcement Learning (IntRL) allows human teachers to accelerate the learning process of Reinforcement Learning (RL) robots. However, IntRL has largely been limited to tasks with discrete-action spaces in which actions are relatively slow. This limits IntRL's application to more complicated and challenging robotic tasks, the very tasks that modern RL is particularly well-suited for. ...
-
To communicate, the ~ 1.5 million Americans living with deafblindess use tactile American Sign Language (t-ASL). To provide Deafßilind (DB) individuals with a means of using their primary communication language without the use of an interpreter, we developed an assistive technology that promotes their autonomy. The TATUM (Tactile ASL Translational User Mechanism) anthropomorphic arm hand system le...
-
Unmanned aerial vehicles (UAVs) are becoming more common, presenting the need for effective human-robot communication strategies that address the unique nature of unmanned aerial flight. Visual communication via drone flight paths, also called gestures, may prove to be an ideal method. However, the effectiveness of visual communication techniques is dependent on several factors including an observ...
-
Explainable artificial intelligence is a research field that tries to provide more transparency for autonomous intelligent systems. Explainability has been used, particularly in reinforcement learning and robotic scenarios, to better understand the robot decision-making process. Previous work, however, has been widely focused on providing technical explanations that can be better understood by AI ...
-
Accurate and robust localization is an essential task for autonomous driving systems. In this paper, we propose a novel 3D LiDAR-aided visual-inertial localization method. Our method fully explores the complementarity of visual and LiDAR observations. On the one hand, the association between semantic features in images and a given semantic map provides constraints for the absolute pose. On the oth...
-
Robust localization in a given map is a crucial component of most autonomous robots. In this paper, we address the problem of localizing in an indoor environment that changes and where prominent structures have no correspondence in the map built at a different point in time. To overcome the discrepancy between the map and the observed environment caused by such changes, we exploit human-readable l...
-
Autonomous robots need to recognize the environment by identifying the scene. Scan context is one of global descriptors, and it encodes the three-dimensional scan data of the scene for the identification in a matrix form. Scan context is in a matrix form that is simple to store, but the matching of scan contexts can require computational effort because the descriptor is orientation-dependent. Beca...
-
Robots deployed in settings such as warehouses and parking lots must cope with frequent and substantial changes when localizing in their environments. While many previous localization and mapping algorithms have explored methods of identifying and focusing on long-term features to handle change in such environments, we propose a different approach - can a robot understand the distribution of movab...
-
In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling,...
-
Position estimation in Multi-Robot Systems (MRS) relies on relative angle or distance measurements between the robots, which generally deteriorate as distances increase. Moreover, the localization accuracy is strongly influenced both by the quality of the raw measurements but also by the overall geometry of the network. In this paper, we design a cost function that accounts for these two issues an...
-
Bird's Eye View (BEV) is a popular representation for processing 3D point clouds, and by its nature is fundamentally sparse. Motivated by the computational limitations of mobile robot platforms, we create a fast, high-performance BEV 3D object detector that maintains and exploits this input sparsity to decrease runtimes over non-sparse baselines and avoids the tradeoff between pseudoimage area and...
-
The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. This complicates tactile tasks like shape reconstruction and exploration that require information about contact geometry. In this work, we pursue a semi-supervised approach to remove shear while preservi...
-
In this paper, we create a new tendon-connected multi-functional optical tactile sensor, MechTac, for object perception in the field of view (TacTip) and location of touching points in the blind area of vision (TacSide). In a multi-point touch task, the information of the TacSide and the TacTip are overlapped to commonly affect the distribution of papillae pins on the TacTip. Since the effects of ...
-
Human-robot collaboration (HRC) has been considered as a promising paradigm towards futuristic human-centric smart manufacturing, to meet the thriving needs of mass personalization. In this context, existing robotic systems normally adopt a single-granularity semantic segmentation scheme for environment perception, which lacks the flexibility to be implemented to various HRC situations. To fill th...
-
This paper deals with the design of a robotic assistant for the transport of large and fragile objects. We propose a new collaborative robotic controller that fulfills the main requirements of co-transportation tasks of large and fragile objects: to execute any trajectory in a collaborative mode while minimizing the stress applied on the object by both partners in order to avoid damaging it. This ...
-
Safety-based Dynamic Task Offloading for Human-Robot Collaboration using Deep Reinforcement Learning
Robots with constrained hardware resources usually rely on Multi-access Edge Computing infrastructures to offload computationally expensive tasks to meet real-time and safety requirements. Offloading every task might not be the best option due to dynamic changes in the network conditions and can result in network congestion or failures. This work proposes a task offloading strategy for mobile robo...
-
Human-robot teams are expected to accomplish complex tasks in high-risk and uncertain environments. In domains such as space exploration or search & rescue, a human operator may not be a robotics expert, but will need to establish a baseline understanding of the robot's capabilities with respect to a given task in order to appropriately utilize and rely on the robot. This willingness to rely, also...
-
When robots interact with human partners, often these partners change their behavior in response to the robot. On the one hand this is challenging because the robot must learn to coordinate with a dynamic partner. But on the other hand - if the robot understands these dynamics - it can harness its own behavior, influence the human, and guide the team towards effective collaboration. Prior research...
-
Enabling robots to learn from interactions with users is essential to perform service tasks. However, as a robot categorizes objects from multimodal information obtained by its sensors during interactive onsite teaching, the inferred names of unknown objects do not always match the human user's expectation, especially when the robot is introduced to new environments. Confirming the learning result...
-
For the proper functioning of mobile manipulator-type autonomous robot performing complicated tasks in a human-robot coexistence environment, tasks and motions must be planned simultaneously. In such environments, a human and robot should collaborate with each other. Therefore, the robot must act in accordance with the human and avoid useless actions duplicated with those of humans. However, any a...
-
VR Facial Animation is necessary in applications requiring clear view of the face, even though a VR headset is worn. In our case, we aim to animate the face of an operator who is controlling our robotic avatar system. We propose a real-time capable pipeline with very fast adaptation for specific operators. In a quick enrollment step, we capture a sequence of source images from the operator without...
-
We propose a new approach to control the shape of deformable objects with robots. Specifically, we consider a fixed-length elastic linear object lying on a 2D workspace. Our main idea is to encode the object's deformation behavior in an offline constant Jacobian matrix. To derive this Jacobian, we use geometric deformation modeling and combine recent work from the fields of deformable object contr...
-
This paper presents a novel visual servoing method that controls a robotic manipulator in the configuration space as opposed to the classical vision-based control methods solely focusing on the end effector pose. We first extract the robot's shape from depth images using a skeletonization algorithm and represent it using parametric curves. We then adopt an adaptive visual servoing scheme that esti...
-
Visual Servoing has been effectively used to move a robot into specific target locations or to track a recorded demonstration. It does not require manual programming, but it is typically limited to settings where one demonstration maps to one environment state. We propose a modular approach to extend visual servoing to scenarios with multiple demonstration sequences. We call this conditional servo...
-
Most deformable object manipulation tasks still rely on skillful human operators. To automate such tasks, a robotic system should not only be able to deform an object to a desired shape but also servo its deformation along a specific path towards the desired shape. We propose a shape servoing control scheme to automate such tasks. Our scheme controls the deformation trajectory towards the desired ...
-
In this paper, an Event-triggered Image-based Visual Servoing Nonlinear Model Predictive Controller (ET-IBVS-NMPC) for multirotor aerial vehicles is presented. The proposed scheme is developed for the autonomous surveillance of contour-based areas with different characteristics (e.g. forest paths, coastlines, road pavements). For this purpose, an appropriately trained Deep Neural Network (DNN) is ...
-
Recent Earth observation satellites are now equipped with new instrument that allows image feedback in real-time. Problematic such as ground target tracking, moving or not, can now be addressed by precisely controlling the satellite attitude. In this paper, we propose to consider this problem using a visual servoing approach. While focusing on the target, the control scheme has also to take into a...
-
For robots to move in the real world, they must first correctly understand the state of its own body and the tools that it holds. In this research, we propose DIJE, an algorithm to estimate the image Jacobian for every pixel. It is based on an optical flow calculation and a simplified Kalman Filter that can be efficiently run on the whole image in real time. It does not rely on markers nor knowled...
-
One of the challenging input settings for visual servoing is when the initial and goal camera views are far apart. Such settings are difficult because the wide baseline can cause drastic changes in object appearance and cause occlusions. This paper presents a novel self-supervised visual servoing method for wide baseline images which does not require 3D ground truth supervision. Existing approache...
-
Multiple sensors, especially cameras and LiDARs, are widely used in autonomous vehicles. In order to fuse data from different sensors accurately, precise calibrations are required, including camera intrinsic parameters, and relative poses between multiple cameras and LiDARs. However, most existing camera-LiDAR calibration methods need to place manually designed calibration objects in multiple loca...
-
Simultaneous orthophoto stitching during the flight of Unmanned Aerial Vehicles (UAV) can greatly promote the practicability and instantaneity of diverse applications such as emergency disaster rescue, digital agriculture, and cadastral survey, which is of remarkable interest in aerial photogrammetry. However, the inaccurately estimated camera poses and the intuitive fusion strategy of existing me...
-
In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior with the aim that completed object geometry leads to better object reconstruction and tracking accuracy. For ea...
-
Estimating the microbial infestation profile of an area is essential for an effective cleaning process. However, current methods used to inspect the microbial infestation within a spatial region are manual and laborious. For large regions that require automated cleaning, conventional methods of microbial examination are not practical. We propose a novel robot-aided microbial density estimation and...
-
Perceiving the surrounding environment is crucial for autonomous mobile robots. An elevation map provides a memory-efficient and simple yet powerful geometric represen-tation of the terrain for ground robots. The robots can use this information for navigation in an unknown environment or perceptive locomotion control over rough terrain. Depending on the application, various post processing steps m...
-
This paper proposes an online large-scale dense mapping method for UAVs with a height of 150–250 meters. We first fuse the GPS with the visual odometry to estimate the scaled poses and sparse points. In order to use the depth of sparse points for depth map, we propose Sparse Confidence Cascade View-Aggregation MVSNet (SCCVA-MVSNet), which projects the depth-converged points in the sliding window o...
-
Grinding materials into a fine powder is a time-consuming task in material science that is generally performed by hand, as current automated grinding machines might not be suitable for preparing small-sized samples. This study presents a robotic powder grinding system for laboratory automation in material science applications that observe the powder's state to improve the grinding outcome. We deve...
-
Laboratory automation is a suitable solution to establish higher reproducibility with less manual work and thus higher quality standards in life sciences. To date, mobile robots are capable of performing autonomous pick-and-place tasks in the laboratory, and specialized pipetting machines can be used for sequenced liquid handling. However, the complex and creative process of developing new researc...
-
Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to f...
-
Particle filters are suited to solve nonlinear and non-Gaussian estimation problems which find numerous applications in autonomous systems navigation. Previous works on Laplace Particle Filter on Lie groups (LG-LPF) demonstrated its robustness and accuracy on challenging navigation scenarios compared to classic particle filters. Nevertheless, LG-LPF is applicable when the prior probability density...
-
The ability to track surgical instruments in realtime is crucial for autonomous Robotic Assisted Surgery (RAS). Recently, the fusion of visual and kinematic data has been proposed to track surgical instruments. However, these methods assume that both sensors are equally reliable, and cannot successfully handle cases where there are significant perturbations in one of the sensors' data. In this pap...
-
Motion estimation approaches typically employ sensor fusion techniques, such as the Kalman Filter, to handle individual sensor failures. More recently, deep learning-based fusion approaches have been proposed, increasing the performance and requiring less model-specific implementations. However, current deep fusion approaches often assume that sensors are synchronised, which is not always practica...
-
We present a portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes. Our goal is to render high-quality images for a dynamic scene from any viewpoint at any time using our portable multiscopic camera. To achieve such novel view and time synthesis, we develop a physical multiscopic camera equipped with five cameras to train a neural radiance fi...
-
The collaborative operation of multiple robots can make up for the shortcomings of a single robot, such as limited field of perception or sensor failure. multirobots collaborative semantic mapping can enhance their comprehensive contextual understanding of the environment. However, existing multirobots collaborative semantic mapping algorithms mainly apply discrete occupancy map inference, and do ...
-
Navigating accurately in potentially GPS-denied environments is a perquisite of autonomous systems. Relative localization based on ultra-wideband (UWB) is - especially indoors - a promising technology. In this paper, we present a probabilistic filter based Modular Multi-Sensor Fusion (MMSF) approach with the capability of using efficiently all information in a fully meshed UWB ranging network. Thi...
-
In multi-robot missions, relative position and attitude information between robots is valuable for a variety of tasks such as mapping, planning, and formation control. In this paper, the problem of estimating relative poses from a set of inter-robot range measurements is investigated. Specifically, it is shown that the estimation accuracy is highly dependent on the true relative poses themselves, ...
-
Event cameras are biologically-inspired vision sensors that capture pixel-level illumination changes instead of the intensity image at a fixed frame rate. They offer many advantages over the standard cameras, such as high dynamic range, high temporal resolution (low latency), no motion blur, etc. Therefore, developing state estimation algorithms based on event cameras offers exciting opportunities...
-
Spatiotemporal calibration of sensors, especially of those which do not share their fields of view, is becoming increasingly important in the fields of autonomous driving and robotics. This paper presents a general sensor calibration method, named Multical, that makes use of multiple planar calibration targets whose poses will be estimated alongside spatiotemporal calibration. Multical exploits co...
-
Designing control policies for legged locomotion11In this work, we specifically consider quadruped locomotion. is complex due to the under-actuated and non-continuous robot dynamics. Model-free reinforcement learning provides promising tools to tackle this challenge. However, a major bottleneck of applying model-free reinforcement learning in real world is safety. In this paper, we propose a safe ...
-
In reinforcement learning (RL), exploration is essential to achieve a globally optimal policy but unconstrained exploration can cause damages to robots and nearby people. To handle this safety issue in exploration, safe RL has been proposed to keep the agent under the specified safety constraints while maximizing cumulative rewards. This paper introduces a new safe RL method which can be applied t...
-
Reinforcement learning (RL) has been shown to be effective at learning control from experience. However, RL typically requires a large amount of online interaction with the environment. This limits its applicability to real-world settings, such as in robotics, where such interaction is expensive. In this work we investigate ways to minimize online interactions in a target task, by reusing a subopt...
-
Reinforcement learning shows great potential to solve complex contact-rich robot manipulation tasks. However, the safety of using RL in the real world is a crucial problem, since unexpected dangerous collisions might happen when the RL policy is imperfect during training or in unseen scenarios. In this paper, we propose a contact-safe reinforcement learning framework for contact-rich robot manipul...
-
Solving complex problems using reinforcement learning necessitates breaking down the problem into manageable tasks, and learning policies to solve these tasks. These policies, in turn, have to be controlled by a master policy that takes high-level decisions. Hence learning policies involves hierarchical decision structures. However, training such methods in practice may lead to poor generalization...
-
Over the last decade, the ability to teach actions to robots in a user-friendly way has gained relevance, and a practical way of teaching robots a new task is to use Inverse Reinforcement Learning (IRL). In IRL, an expert teacher shows the robot a desired behaviour and an agent builds a model of the reward. The agent can also infer a policy that performs in an optimal way within the limitations of...
-
The common approach for local navigation on challenging environments with legged robots requires path planning, path following and locomotion, which usually requires a locomotion control policy that accurately tracks a commanded velocity. However, by breaking down the navigation problem into these sub-tasks, we limit the robot's capabilities since the individual tasks do not consider the full solu...
-
Autonomously navigating a robot in everyday crowded spaces requires solving complex perception and planning challenges. When using only monocular image sensor data as input, classical two-dimensional planning approaches cannot be used. While images present a significant challenge when it comes to perception and planning, they also allow capturing potentially important details, such as complex geom...
-
Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile devices. Although deep learning is capable of extracting information from multiple sensory inputs, there has been little use of sound for the control and learning of robotic actions. For unsupervised reinforcement learni...
-
Inspired by the traditional art of paper folding, origami, autonomous production of 3D structures from 2D sheets can be achieved by the implementation of self-folding techniques. One technique to achieve such transformation is the usage of thermo-responsive smart materials such as self-folding polymeric films, which can be controlled by heat to shrink. Achieving remote self-folding with a practica...
-
Insects can locomote readily in challenging environments, such as over steep inclines and across obstacle-laden terrains, which still frustrate robots of similar size. In this work, inspired by the passive compliant properties of insect limbs, we use the insect-scale Harvard Ambulatory Microrobot and multilayer microfabrication techniques as platform to study the ability of passive mechanisms to i...
-
Soft microrobotics have recently been an active field that advances microrobotics with new robot design, locomotion, and applications. In this paper, we study the ferrofluid robot (FR), which has soft nature and exhibits paramagnetism. Currently, the FR locomotion is usually realized by magnetic force. To enable the FR with more locomotion modes for environment and task adaptability, we program th...
-
As the focus on highly intelligent robots continues, a problem that cannot be ignored has emerged: resource con-straints. Considering the game problem of resource limitation and the level of intelligence, we focus on lightweight intelligence. This work is a further refinement of our previous work, a heterogeneous lightweight intelligent multi-robot system. In-spired by the nature creatures “octopu...
-
Professional human drivers usually have more than one driving strategy to handle incoming traffic situations. These different strategies activate different performance characteristics of the vehicle, enabling the driver to minimize the risk in a variety of situations by optimizing the strategy selection. In the same spirit, we define a novel concept of strategy-wise performance metric and creative...
-
Reconfigurable mobile robots are well suited for inspection tasks in legacy nuclear facilities where access is restricted and the environment is often cluttered. A reconfig-urable snake robot, MIRRAX, has previously been developed to investigate such facilities. The joints used for the robot's reconfiguration introduce additional constraints on the robot's control, such as balance, on top of the e...
-
In nature, the movement of quadrupeds is completed under the combined action of the spine and the legs. Inspired by this, this paper explores the effect of a lateral flexing spine on the locomotion of a rat robot. Benefiting from the regular lateral flexion of a soft actuated spine, the rat robot exhibits enhance step length of its hind legs and increased translational velocity by coordinating the...
-
We propose a framework for co-optimizing the shape and motion of rigid robotic effectors for planar tasks. While planning object and robot-object contact trajectories is extensively studied, designing an effector that can execute the planned trajectories receives less attention. As such, our framework synthesizes an object trajectory and object-effector contact trajectory into an effector trajecto...
-
Unlike fully actuated systems, the control of underactuated robots necessitates the use of passive dynamics to fulfill control objectives. Hence, there is an increased interdependence between their design parameters and the closed loop performance. This paper proposes a novel approach for co-optimization of robot design and controller parameters for increased certifiable stability obtained with me...
-
The time-series datasets commonly applied for anomaly detection research showcase specific suboptimal properties. This work novelly conceptualizes condition state synthesis to improve the data-synthetic pipeline of an anomalous-event dataset. We demonstrate two technical contributions in this study. First, we propose a methodology to formulate, accelerate and enrich the condition state synthetic p...
-
Through the efforts of robotic engineers and inspired by the flapping flight of smaller creatures (e.g., insects and hummingbirds), the untethered stable hovering of flapping micro-aerial vehicles (FMAVs) has been achieved. Now, engineers are evaluating how to improve the mobility of these vehicles. The maneuverability of insects and birds in flight, such as their sharp turns and their takeoffs an...
-
In this paper we present swimming and modeling for Trident, a three-link lamprey inspired robot that is able to climb on flat smooth walls. We explore two gaits proposed to work for linear swimming, and three gaits for turning maneuvers. We compare the experimental results obtained from these swimming experiments with two different reduced order fluid interaction models, one a previously published...
-
Undulatory fin motions in fish-like robots are typically created using intricate arrays of servo motors. Motor arrays offer impressive versatility in terms of kinematics, but their complexity leads to constraints on size, hydrodynamic force production, and power consumption, particularly when studying propulsive performance at high-frequencies. Here we present an alternative design that uses a sin...
-
In this work we present the design of a swimming robot that is inspired by the body shape modulation of small microorganisms. Amoebas are small single celled organisms that locomote through deformation and shape change of their body. To achieve similar shape modulation for swimming propulsion in a robot we developed a novel flexible appendage using tape springs. A tape spring is an elongated strip...
-
Flapping wing insects benefit from a compliant thorax that provides elastic energy exchange and resiliency to wing collisions. In this paper, we present a flapping wing robot that uses an underactuated compliant transmission inspired by the insect thorax. We developed a novel fabrication method that combines carbon fiber (CF) laminate and soft robotics fabrication techniques for transmission const...
-
Anomaly detection is an important problem in computer vision; however, the scarcity of anomalous samples makes this task difficult. Thus, recent anomaly detection methods have used only “normal images” with no abnormal areas for training. In this work, a powerful anomaly detection method is proposed based on student-teacher feature pyramid matching (STPM), which consists of a student and teacher n...
-
This paper focuses on problems associated with the deployment of automatic agents for last-mile delivery. We propose a framework and methodology to systematically evaluate and compare different hybrid strategies. Performance metrics in agent noise, delivery time, energy consumption, coverage rate, package throughput, and system costs are defined rigorously and modeled mathematically. Using the met...
-
In a typical fulfillment center, the order fulfilling process is managed by a warehouse management system (WMS). For efficiency, WMS usually applies batch picking, also called multi-order picking, to collect the same items for multiple orders. Suppose an item appears in multiple orders, instead of repeatedly revisiting the exact picking location multiple times, a picker will be instructed to pick ...
-
The rise of simulation environments has enabled learning-based approaches for assembly planning, which is otherwise a labor-intensive and daunting task. Assembling furniture is especially interesting since furniture are intricate and pose challenges for learning-based approaches. Surprisingly, humans can solve furniture assembly mostly given a 2D snapshot of the assembled product. Although recent ...
-
The existing manufacturing systems based on processes involving the transportation of workpiece are unsuitable for large products such as air mobility systems. This study proposes a novel ultra-complex manufacturing system called the “Modularized-Structure and Multiple-Points Simultaneous Machining System (MS-cubic)” based on the concept of intelligent space, which simultaneously performs multiple...
-
A loop closure module plays an important role in visual SLAM systems, which can reduce the accumulat-ed drift. This task faces the challenges of large viewpoint changes and expensive computational costs when optimizing the global map. This paper proposes DH-LC, a novel accurate and robust loop closure method that consists of hierarchical spatial feature matching (HSFM) and hybrid bundle adjustment...
-
LiDAR-based global localization is a fundamental problem for mobile robots. It consists of two stages, place recognition and pose estimation, which yields the current orientation and translation, using only the current scan as query and a database of map scans. Inspired by the definition of a recognized place, we consider that a good global localization solution should keep the pose estimation acc...
-
As the basics of Visual Simultaneous Localization And Mapping (VSLAM), keyframes play an essential role. In previous works, keyframes are selected according to a series of view change-based strategies for short-term data association (STDA). However, the texture enrichment of frames is always ignored, resulting in the failure of long-term data association (LTDA). In this paper, we propose an inform...
-
Reliability of SLAM systems is considered one of the critical requirements in modern autonomous systems. This directed the efforts to developing many state-of-the-art systems, creating challenging datasets, and introducing rigorous metrics to measure SLAM performance. However, the link between datasets and performance in the robustness/resilience context has rarely been explored. In order to fill ...
-
The ability to reason about changes in the environment is crucial for robots operating over extended periods of time. Agents are expected to capture changes during operation so that actions can be followed to ensure a smooth progression of the working session. However, varying viewing angles and accumulated localization errors make it easy for robots to falsely detect changes in the surrounding wo...
-
There is an increasing demand for robots that can be substituted for humans in various tasks. Mobile robots are being introduced in factories, stores, and public facilities for carrying goods and cleaning. In factories and stores, desks and shelves are arranged such that the work and movement of personnel are reduced. The surrounding furniture is also set to ensure that a single task can be perfor...
-
Recent progress in object pose prediction provides a promising path for robots to build object-level scene representations during navigation. However, as we deploy a robot in novel environments, the out-of-distribution data can degrade the prediction performance. To mitigate the domain gap, we can potentially perform self-training in the target domain, using predictions on robot-captured images as...
-
Long-term scene changes pose challenges to localization systems using a pre-built map. This paper presents a LiDAR-based system that provides robust localization against those challenges. Our method starts with activation of a mapping process temporarily when global matching towards the pre-built map is unreliable. The temporary map will be merged onto the pre-built map for later localization sess...
-
Loop closure detection has the potential to correct the drift of trajectories and build a global consistent map in LiDAR SLAM, however it remains a challenging problem in outdoor environment due to the sparsity of 3D point clouds data, large-scale scenes and moving objects. Inspired by the way humans perceive the environment through recognizing objects and identifying their relations, this paper p...
-
Accurate and reliable state estimation is becoming increasingly important as robots venture into the real world. Gaussian variational inference (GVI) is a promising alternative for nonlinear state estimation, which estimates a full probability density for the posterior instead of a point estimate as in maximum a posteriori (MAP)-based approaches. GVI works by optimizing for the parameters of a mul...
-
High-precision camera re-localization technology in a pre-established 3D environment map is the basis for many tasks, such as Augmented Reality, Robotics and Autonomous Driving. The point-based visual re-localization approaches are well-developed in recent decades, but are insufficient in some feature-less cases. In this paper, we design a complete pipeline for camera pose refinement with points a...
-
Safety guarantees in motion planning for autonomous driving typically involve certifying the trajectory to be collision-free under any motion of the uncontrollable participants in the environment, such as the human-driven vehicles on the road. As a result they usually employ a conservative bound on the behavior of such participants, such as reachability analysis. We point out that planning traject...
-
This paper explores learning an effective controller for improving the efficiency of kinodynamic planning for vehicular systems navigating uneven terrains. It describes the pipeline for training the corresponding controller and using it for motion planning purposes. The training process uses a soft actor-critic approach with hindsight experience replay to train a model, which is parameterized by t...
-
Motion planning in off-road environments re-quires reasoning about both the geometry and semantics of the scene (e.g., a robot may be able to drive through soft bushes but not a fallen log). In many recent works, the world is classified into a finite number of semantic categories that often are not sufficient to capture the ability (i.e., the speed) with which a robot can traverse off-road terrain...
-
When navigating to a goal in an uncertain environment, a robot must simultaneously navigate the exploration-exploitation tradeoff: should it aim to gain information and reduce uncertainty, or should it simply brave the unknown? We formalize this as the Bayesian dynamic motion planning problem, and we analyze how several strategies from the literature balance these concerns via determinization and ...
-
In this paper, we present an online planning-scheduling approach for battery-powered autonomous aerial robots. The approach consists of simultaneously planning a coverage path and scheduling onboard computational tasks. We further derive a novel variable coverage motion robust to air-borne constraints and an empirically motivated energy model. The model includes the energy contribution of the sche...
-
This paper investigates how a shepherd robot can efficiently steer a coherent group by intelligently moving behind the group in obstacle-filled environments. It was been shown that a model trained by deep reinforcement learning can guide a small number (2–4) of agents among obstacles. However, herding a larger group becomes significantly more challenging because it exhibits the characteristics sim...
-
This paper focuses on the problem of smoothing a rotation trajectory corrupted by noise, while simultaneously estimating its corresponding angular velocity and angular acceleration. To this end, we develop a geometric version of the Savitzky-Golay filter on SO(3) that avoids following the conventional practice of first converting the rotation trajectory into Euler-like angles, performing the filte...
-
This work presents the automatic design of additively manufacturable serially linked one degree of freedom manipulators whose end effectors move along individually prescribed 2D trajectories. The kinematic coupling of the links is done by using gear stages consisting of spur gears and toothed belt gears. The basic design parameters of these mechanisms are determined using a Fourier series. The cal...
-
Robotic painting tasks in the real world are often made complicated by the highly complex and stochastic nature of the dynamics that underlie, e.g., physical contact between the painting tool and a canvas, color blendings between painting mediums, and many more. Simulation-based inverse graphics algorithms, for example, can not be directly transferred to the real-world due in large to the consider...
-
Musical dancing is an ubiquitous phenomenon in the human society. Providing robots the ability to dance has the potential to make the human robot co-existence more acceptable in our society. Hence, dancing robots have generated a considerable research interest in the recent years. In this paper, we present a novel formalization of robot dancing as planning and control of optimally timed actions ba...
-
In this paper, we present a motion-based robotic communication framework that enables non-verbal communication among autonomous underwater vehicles (AUVs) and human divers. We design a gestural language for AUV-to-AUV communication which can be easily understood by divers observing the conversation - unlike typical radio frequency, light, or audio-based AUV communication. To allow AUVs to visually...
-
The combination of collaborative robots and end-to-end AI, promises flexible automation of human tasks in factories and warehouses. However, such promise seems a few breakthroughs away. In the meantime, humans and cobots will collaborate helping each other. For these collaborations to be effective and safe, robots need to model, predict and exploit human's intents for responsive decision making pr...
-
Co-speech gestures are a principal component in conveying messages and enhancing interaction experiences between humans and critical ingredients in human-agent interaction, including virtual agents and robots. Existing machine learning approaches have yielded only marginal success in learning speech-to-motion at the frame level. Current methods generate repetitive gesture sequences that lack appro...
-
Robowflex is a software library for robot motion planning in industrial and research applications, leveraging the popular Moveit library and Robot Operating System (ROS) middleware. Robowflex provides an augmented API for crafting and manipulating motion planning queries within a single program, making motion planning with Moveit easy. Robowflex's high-level API simplifies many common use-cases wh...
-
Absolute position detection in sensorless electric stepper motors potentially allows for higher space efficiency, improved shock resistance, simplified installation, reduced number of parts and lowered cost. A prototype is demonstrated measuring 42 × 42 × 34 mm3 with seven coils arranged in a star configuration. The rotor is ϕ 25.8 × 12.5 mm2 and has 51 teeth which are irregularly spaced. At the d...
-
Evaluation of grasps on deformable $3\mathrm{D}$ objects is a little-studied problem, even if the applicability of rigid object grasp quality measures for deformable ones is an open question. A central issue with most quality measures is their dependence on contact points, which for deformable objects depend on the deformations. This paper proposes a grasp quality measure for deformable objects th...
-
Intercepting an object in flight through nonpre-hensile manipulation is a challenging problem, which is aimed at catching and stopping a flying object using little contacts without completely restraining its relative motion to the robot. This paper presents a two-stage optimal trajectory generation method to tackle this problem. At the pre-catching stage, optimal position and attitude trajectories...
-
In this paper, we introduce a mask-based grasping method that discerns multiple objects within the scene regard-less of transparency or specularity and finds the optimal grasp position avoiding clutter. Conventional vision-based robotic grasping approaches often fail to extend to the scenes containing transparent objects due to their different visual appearance. To handle the different visual char...
-
This paper reports a tensegrity manipulator driven by 40 pneumatic cylinders without mechanical springs. In general, tensegrity robots use mechanical springs to achieve a stable/curved tensegrity structure, and this is true even when a component extends/retracts with an actuator. The stiffness of the mechanical spring should be high to increase the stiffness of the entire structure and improve the...
-
This paper expands on the problem of grasping an object that can only be grasped by a single parallel gripper when a fixture (e.g., wall, heavy object) is harnessed. Preceding work that tackle this problem are limited in that the employed networks implicitly learn specific targets and fixtures to leverage. However, the notion of a usable fixture can vary in different environments, at times without...
-
Planar pushing is a fundamental robot manipulation task with most algorithms built upon the quasi-static as-sumption. Under this assumption the end-effector should apply force on the pushed object along the full moving trajectory. This means that the target position must lie in the robot's workspace. To enable a robot to deliver objects outside of its workspace and facilitate faster delivery, the ...
-
In recent years, there has been a growing interest in robotic manipulation of deformable objects. In order to perform certain tasks, the robot must control the shape of the object while taking care not to apply excessive stresses so as not to deform it irreversibly. This is the case when extracting elasto-plastic objects in strips from an industrial reel. In order to control the mechanical stresse...
-
This paper introduces DGBench, a fully reproducible open-source testing system to enable benchmarking of dynamic grasping in environments with unpredictable relative motion between robot and object. We use the proposed benchmark to compare several visual perception arrangements. Traditional perception systems developed for static grasping are unable to provide feedback during the final phase of a ...
-
In recent years, synthetic data has been widely used in the training of 6D pose estimation networks, in part because it automatically provides perfect annotation at low cost. However, there are still non-trivial domain gaps, such as differences in textures/materials, between synthetic and real data. These gaps have a measurable impact on performance. To solve this problem, we introduce a simulatio...
-
Joint estimation of grasped object pose and extrinsic contacts is central to robust and dexterous manipulation. In this paper, we propose a novel state-estimation algorithm that jointly estimates contact location and object pose in 3D using exclusively proprioception and tactile feedback. Our approach leverages two complementary particle filters: one to estimate contact location (CPFGrasp) and ano...
-
Learning-based perception systems in robotics often requires large-scale image segmentation annotation. Current approaches rely on human labelers, which can be expensive, or simulation data, which can visually differ from real data. This paper proposes Labels from UltraViolet (LUV), a novel framework that enables rapid, automated, inexpensive, high quality data collection in real. LUV uses transpa...
-
Robots with elasticity in structural components can suffer from undesired end-effector positioning imprecision, which exceeds the accuracy requirements for successful manipulation. We present the Probabilistic-Product-Of-Exponentials robot model, a novel approach for kinematic modeling of robots. It does not only consider the robot's deterministic geometry but additionally models time-varying and ...
-
Localization in topological maps is essential for image-based navigation using an RGB camera. Localization using only one camera can be challenging in medium-to-large-sized environments because similar-looking images are often observed repeatedly, especially in indoor environments. To overcome this issue, we propose a learning-based localization method that simultaneously utilizes the spatial cons...
-
We present a strong baseline that surpasses the performance of previously published methods on the Habitat Challenge task of navigating to a target object in indoor environments. Our method is motivated from primary failure modes of prior state-of-the-art: poor exploration, inaccurate object identification, and agent getting trapped due to imprecise map construction. We make three contributions to...
-
One of the key challenges in high-speed off-road navigation on ground vehicles is that the kinodynamics of the vehicle-terrain interaction can differ dramatically depending on the terrain. Previous approaches to addressing this challenge have considered learning an inverse kinodynamics (IKD) model, conditioned on inertial information of the vehicle to sense the kinodynamic interactions. In this pa...
-
In this paper, we propose a hybrid visual-LiDAR odometry (H-VLO) framework that fuses predicted visual depth map and completed LiDAR map. Compared to the previous visual-LiDAR odometry methods, our approach leverages 2D feature matching and 3D association by utilizing deep depth map, deep flow map and deep LiDAR depth completion networks. Rather than extraction of the depth values from LiDAR measu...
-
In this work, we present a memory-augmented approach for image-goal navigation. Earlier attempts, including RL-based and SLAM-based approaches have either shown poor generalization performance, or are heavily-reliant on pose/depth sensors. Our method is based on an attention-based end-to-end model that leverages an episodic memory to learn to navigate. First, we train a state-embedding network in ...
-
The tasks that unmanned aerial vehicles (UAVs) have taken upon have progressively grown in complexity over the years, alongside with the level of autonomy with which they are carried out. In this work, we present an example of aerial screwing operations with a fully-actuated tilt-rotor platform. Key contributions include a new control framework to automate screwing operations through a robust hole...
-
Online trajectory planners enable quadrotors to safely and smoothly navigate in unknown cluttered environments. However, tuning parameters is challenging since modern planners have become too complex to mathematically model and predict their interaction with unstructured environments. This work takes humans out of the loop by proposing a planner parameter adaptation framework that formulates objec...
-
For real-time multirotor kinodynamic planning, the efficiency of sampling-based methods is usually hindered by difficult-to-sample homotopy classes like narrow passages. In this paper, we address this issue by a hybrid scheme. We firstly propose a fast regional optimizer exploiting the information of local environments and then integrate it into a bidirectional global sampling process. The incorpo...
-
From ancient times, humans have been using cables and ropes to tie, carry, and manipulate objects by folding knots. However, automating knot folding is challenging because it requires dexterity to move a cable over and under itself. In this paper, we propose a method to fold knots in midair using a team of aerial vehicles. We take advantage of the fact that vehicles are able to fly in between cabl...
-
Plastic waste is a global concern that has a negative impact on the oceans and wildlife health. This paper focuses on detection of floating plastics in aerial images taken from unmanned aerial vehicles (UAVs). It proposes a new method for plastic detection in marine environments, based on SIFT descriptor and color histograms for feature extraction, as an alternative to state-of-the-art object dete...
-
In this work we address the challenging problem of manipulating a flexible link, like a rope, with an aerial robot. Inspired by spraying tasks in construction and maintenance scenarios, we consider the case in which an autonomous end-effector (e.g., a spray nozzle moved by a robot or a human operator) is connected to a fixed point by a rope (e.g., a hose). To avoid collisions between the rope and ...
-
This work presents a pipeline for autonomous emergency landing for multicopters, such as rotary wing Unmanned Aerial Vehicles (UAVs), using deep Reinforcement Learning (RL). Mechanical malfunctions, strong winds, sudden battery life drops (e.g, due to cold weather), failure in localization or GPS jamming are not uncommon and all constitute emergency situations that require a UAV to abort its missi...
-
Lumen Shape Reconstruction using a Soft Robotic Balloon Catheter and Electrical Impedance Tomography
Incorrectly sized balloon catheters can lead to increased post-surgical complications, yet even with preoperative imaging, correct selection remains a challenge. With limited feedback during surgery, it is difficult to verify correct deployment. We propose the use of integrated impedance measurements and Electrical Impedance Tomography (EIT) imaging to assess the deformation of the balloon and det...
-
Small untethered soft robots have potential for diverse applications, particularly in constrained spaces where the use of a tethered device would be infeasible. Examples include biomedical applications such as brachytherapy, fine-needle biospy and micro-needle drug delivery. To advance soft robots towards these applications, there is a need to establish methods for tracking and control using clini...
-
Autonomous surgical robotics is a growing area of research, with advances being made in the areas of vision and control. Central to this research is the need for simulations to facilitate data collection and simulate learning environments for Reinforcement Learning (RL) agents. Recent simulators have facilitated RL policy generation, but lack a robust sim2real pipeline and a proven vision-based po...
-
Percutaneous coronary intervention (PCI) is a frequently used surgical treatment for cardiovascular disease, one of the leading cause of death in the world. In traditional PCI, a doctor navigates a thin guidewire in a patient's vessel toward a target location by looking into live X-ray angiogram images of the patient. Recently, researchers are using reinforcement learning to automate this guidewir...
-
Modeling closed loop mechanisms is a necessity for the control and simulation of various systems and poses a great challenge to rigid body dynamics algorithms. Solving the forward and inverse dynamics for such systems require resolution of loop closure constraints which are often solved via numerical procedures. This brings an additional burden to these algorithms as they have to stabilize and con...
-
This paper presents reorientation manoeuvres applied to an omnidirectional wheeled robot for impact mitigation during short falls. The proposed robot architecture aims to build upon recent innovations in reorientation robots to attain fast, multi-axis reorientation. Indeed, the use of omnidirectional wheels allows for simplifications to be made with respect to previous mobile robot architectures t...
-
Keeping balance is one of the most important tasks of a motorcycle. The steady-state manifold is proposed in this paper to explore the inherent dynamics and the balance properties of a riderless motorcycle. The dynamic and kinematic characteristics are analyzed based on the manifold and are validated by simulation. Comparing to traditional control method, the usefulness of the manifold in control ...
-
Tele-impedance increases interaction performance between a robotic tool and unstructured/unpredictable en-vironments during teleoperation. However, the existing tele-impedance interfaces have several ongoing issues, such as long calibration times and various obstructions for the human operator. In addition, they are all designed to be controlled by the operator's arms, which can cause difficulties...
-
Complex needle shape prediction remains an issue for planning of surgical interventions of flexible needles. In this paper, we validate a theoretical method for flexible needle shape prediction allowing for non-uniform curvatures, extending upon a previous sensor-based model which combines curvature measurements from fiber Bragg grating (FBG) sensors and the mechanics of an inextensible elastic ro...
-
We discuss a novel method for estimating task Cartesian position and velocity in robot manipulators. This is done by model-based fusion of inertial measurement units with motor encoders. The model is developed to robustly handle the uncertainties in the trajectory. Thus, not only the approach benefits from high fidelity and bandwidth thanks to multiple-sensory fusion, but it also enforces stabilit...
-
Robots are increasingly being called on to operate in settings and on tasks originally designed for humans, or where humans are also expected to work. Accordingly, the hardware and tools to be packaged, operated, or maintained are typically designed for use by humans, not robots. Robot autonomy in such cases can be expedited by a “robot factors” approach to the design of hardware, analogous to erg...
-
Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, ...
-
Estimating 6D poses of objects is an essential computer vision task. However, most conventional approaches rely on camera data from a single perspective and therefore suffer from occlusions. We overcome this issue with our novel multi-view 6D pose estimation method called MV6D which accurately predicts the 6D poses of all objects in a cluttered scene based on RGB-D images from multiple perspective...
-
Robots are increasingly introduced in various Child-Robot Interactions with educational, entertainment or even therapeutic goals. In order to achieve qualitative inter-actions, robots need to adjust their behavior according to children's response. A robot's ability to successfully estimate partner's engagement is of great importance towards this direction. In this research we propose a method to e...
-
A gaze-fixating robot perceives distance to the fixated object and relative positions of surrounding objects immediately, accurately, and robustly. We show how fixation, which is the act of looking at one object while moving, exploits regularities in the geometry of 3D space to obtain this information. These regularities introduce rotation-translation couplings that are not commonly used in struct...
-
Person detection is a crucial task for mobile robots navigating in human-populated environments. LiDAR sensors are promising for this task, thanks to their accurate depth measurements and large field of view. Two types of LiDAR sensors exist: the 2D LiDAR sensors, which scan a single plane, and the 3D LiDAR sensors, which scan multiple planes, thus forming a volume. How do they compare for the tas...
-
Soft robotic grippers facilitate contact-rich manipulation, including robust grasping of varied objects. Yet the beneficial compliance of a soft gripper also results in significant deformation that can make precision manipulation challenging. We present visual pressure estimation & control (VPEC), a method that infers pressure applied by a soft gripper using an RGB image from an external camera. W...
-
In contrast to sophisticated means of visual su-per resolution (SR), not much work has been done in the tactile SR field. Existing tactile SR algorithms for taxel-based sensors mainly focus on enhancing the localization accuracy, and generally associate with a specific type of hardware, sometimes not applicable to generic taxel-based tactile sensors. Inspired by image SR, we investigate the tactil...
-
Visuotactile sensors have recently attracted much attention in robot communities due to the benefit of high spatial resolution sensing. However, force/torque estimation by visuotactile sensors remains a challenging problem. In this paper, we propose a learning-based six-axis force/torque estimation network using GelStereo visuotactile sensor, which can provide two-dimensional (2D) and three-dimens...
-
When humans socially interact with another agent (e.g., human, pet, or robot) through touch, they do so by applying varying amounts of force with different directions, locations, contact areas, and durations. While previous work on touch gesture recognition has focused on the spatio-temporal distribution of normal forces, we hypothesize that the addition of shear forces will permit more reliable c...
-
This article presents the novel modular version of the robotic platform Cellulo, a versatile handheld robot initially designed as an educational robot. The use of Cellulo in different contexts and applications over the years has highlighted the need for modularity. Modularity adds versatility by increasing the spectrum of functionalities of the robot, as well as more robustness. Modulo Cellulo con...
-
Stroke survivors usually have dyskinesia, who have an urgent need for rehabilitation-assist training. To reduce the labor of rehabilitation therapists, this paper attempts to investigate an effective rehabilitation-assisted robot skill acquisition framework, which is inspired by the scheme of robot learning from demonstration (LfD). Since most of the current LfD methods were implemented with rigor...
-
A treadmill system has a large potential to provide humans with an augmented walking experience in real-life without a spatial limitation. However, a treadmill gait is different from walking on level ground. In previous studies, the adaptive belt speed control of a treadmill was developed to achieve a self-paced walking for making the users' treadmill gait similar to their level ground gait. Such ...
-
During physical human robot collaboration, it is important to be able to implement a time-varying interactive behaviour while ensuring robust stability. Admittance control and passivity theory can be exploited for achieving these objectives. Nevertheless, when the admittance dynamics is time-varying, it can happen that, for ensuring a passive and stable behaviour, some spurious dissipative effects...
-
Transferring tools and objects to human hands is an important ability of collaborative robots. Most of the existing approaches focus on handover affordance, however, the comfort of receiving objects with human hands is often neglected. In this paper, we use advanced deep learning models to pre-generate handover target configurations that are convenient for human grasping based on the characteristi...
-
Drilling a hole on a curved surface with a desired angle is prone to failure when done manually, due to the difficulties in drill alignment and also inherent instabilities of the task, potentially causing injury and fatigue to the workers. On the other hand, it can be impractical to fully automate such a task in real manufacturing environments because the parts arriving at an assembly line can hav...
-
In this paper, we propose a learning-based moving-object tracking method utilizing the newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR. Compared with most existing commercial LiDAR sensors, FMCW LiDAR can provide additional Doppler velocity information to each 3D point of the point clouds. Benefiting from this, we can generate instance labels as ground truth in a sem...
-
Multiple object tracking in 3D point clouds has applications in mobile robots and autonomous driving. This is a challenging problem due to the sparse nature of the point clouds and the added difficulty of annotation in 3D for supervised learning. To overcome these challenges, we propose a neural network architecture that learns effective object features and their affinities in a self supervised fa...
-
In this paper, we propose an accurate and simultaneously efficient solution to perspective-n-point-and-line (PnPL) problem by null space analysis. Although many PnPL-like methods have been proposed, it is hard to obtain the optimal solution considering both calculation efficiency and accuracy at the same time. Based on the remarkable EOPnP method, the proposed algorithm integrates linear-expressed...
-
Tracking the 3D motion of agile animals in the wild will enable new insight into the design of robotic controllers. However, in-field 3D pose estimation of high-speed wildlife such as cheetahs is still a challenge [1]. In this work, we aim to solve two of these challenges: unnatural pose estimates during highly occluded sequences and synchronization error between multi-view data. We expand on our ...
-
Direct methods have shown excellent performance in the applications of visual odometry and SLAM. In this work we propose to leverage their effectiveness for the task of 3D multi-object tracking. To this end, we propose DirectTracker, a framework that effectively combines direct image alignment for the short-term tracking and sliding-window photometric bundle adjustment for 3D object detection. Obj...
-
We present the first real-time system capable of tracking and reconstructing, individually, every visible object in a given scene, without any form of prior on the rigidness of the objects, texture existence, or object category. In contrast with previous methods such as Co-Fusion and MaskFusion that first segment the scene into individual objects and then process each object independently, the pro...
-
Event cameras are bio-inspired dynamic vision sensors that respond to changes in image intensity with a high temporal resolution, high dynamic range and low latency. These sensor characteristics are ideally suited to enable visual target tracking in concert with a broadcast visual communication channel for smart visual beacons with applications in distributed robotics. Visual beacons can be constr...
-
Robots deployed in human-centric environments may need to manipulate a diverse range of articulated objects, such as doors, dishwashers, and cabinets. Articulated objects often come with unexpected articulation mechanisms that are inconsistent with categorical priors: for example, a drawer might rotate about a hinge joint instead of sliding open. We propose a category-independent framework for pre...
-
Contrary to other standard cameras, event cam-eras interpret the world in an entirely different manner; as a collection of asynchronous events. Despite event camera's unique data output, many event feature detection and tracking algorithms have shown significant progress by making detours to frame-based data representations. This paper questions the need to do so and proposes a novel event data-fr...
-
The majority of current approaches in autonomous driving rely on High-Definition (HD) maps which detail the road geometry and surrounding area. Yet, this reliance is one of the obstacles to mass deployment of autonomous vehicles due to poor scalability of such prior maps. In this paper, we tackle the problem of online road map extraction via leveraging the sensory system aboard the vehicle itself....
-
Autonomous robots are often required to acquire high-level prior knowledge by continuously reconstructing the semantics and geometry of the surrounding scene, which is the basis of exploration and planning. Most existing continuous semantic mapping algorithms cannot distinguish potential differences in voxels, resulting in an over-inflated map. Furthermore, fixed-size query ranges introduce high c...
-
The ability to produce large-scale maps for nav-igation, path planning and other tasks is a crucial step for autonomous agents, but has always been challenging. In this work, we introduce BEV-SLAM, a novel type of graph-based SLAM that aligns semantically-segmented Bird's Eye View (BEV) predictions from monocular cameras. We introduce a novel form of occlusion reasoning into BEV estimation and dem...
-
Lidar technology has evolved significantly over the last decade, with higher resolution, better accuracy, and lower cost devices available today. In addition, new scanning modalities and novel sensor technologies have emerged in recent years. Public datasets have enabled benchmarking of algorithms and have set standards for the cutting edge technology. However, existing datasets are not representa...
-
This paper presents a multimodal indoor odometry dataset, OdomBeyondVision, featuring multiple sensors across the different spectrum and collected with different mobile platforms. Not only does OdomBeyondVision contain the traditional navigation sensors, sensors such as IMUs, mechanical LiDAR, RGBD camera, it also includes several emerging sensors such as the single-chip mmWave radar, LWIR thermal...
-
Combining multiple sensors enables a robot to maximize its perceptual awareness of environments and enhance its robustness to external disturbance, crucial to robotic navigation. This paper proposes the FusionPortable benchmark, a complete multi-sensor dataset with a diverse set of sequences for mobile robots. This paper presents three contributions. We first advance a portable and versatile multi...
-
This paper introduces a stereo thermal camera dataset (STheReO) with multiple navigation sensors to encourage thermal SLAM researches. A thermal camera measures infrared rays beyond the visible spectrum therefore it could provide a simple yet robust solution to visually degraded environments where existing visual sensor-based SLAM would fail. Existing thermal camera datasets mostly focused on mono...
-
We propose a human-in-the-loop reinforcement learning mechanism to help robots learn emotional behavior. Unlike the previous methods of providing explicit feedback via pressing keyboard buttons or mouse clicks, we provide a more natural way for ordinary people to train social robots how to perform social tasks according to their preferences - facial expressions. The whole experiment is carried out...
-
We present an accessible robot interface for telemanipulation (A-RIFT), which preserves the haptic channel partially in a zero-additional-cost interface by visual substitution of force feedback (VSFF). This work explores a gap in the literature, resulting from the focus on performance improvements in telerobotics at increasing interface costs. Unlike most telemanipulation interfaces for high-degre...
-
The dynamic load attached to the load gravity imposes an excessive burden to human shoulders during load carriage, resulting in possible muscle injuries and additional physical exertion. This paper proposes an active suspension backpack, capable of transferring partial load from human shoulders to pelvis and alleviating the dynamic load through separated panels and motor actuation, to reduce press...
-
To plan safe maneuvers and act with foresight, autonomous vehicles must be capable of accurately predicting the uncertain future. In the context of autonomous driving, deep neural networks have been successfully applied to learning pre-dictive models of human driving behavior from data. However, the predictions suffer from cascading errors, resulting in large inaccuracies over long time horizons. ...
-
For most LiDAR-inertial odometry, accurate initial states, including temporal offset and extrinsic transfor-mation between LiDAR and 6-axis IMUs, play a significant role and are often considered as prerequisites. However, such information may not be always available in customized LiDAR-inertial systems. In this paper, we propose LI-Init: a full and real-time LiDAR-inertial system initialization pr...
-
Terrain classification is a challenging task for robots operating in unstructured environments. Existing classification methods make simplifying assumptions, such as a reduced number of classes, clearly segmentable roads, or good lighting conditions, and focus primarily on one sensor type. These assumptions do not translate well to off-road vehicles, which operate in varying terrain conditions. To...
-
Accurate shape estimation of concentric tube robots (CTRs) using mathematical models remains a challenge, reinforcing the need to develop techniques for accurate and real-time shape sensing of CTRs. In this paper, we develop a fusion algorithm that predicts the robot's shape by combining a mathematical model of the CTR with a measurement of the Cartesian coordinates of the robot's tip using an ele...
-
Manipulation of deformable objects is a challenging task for a robot. It would be problematic to use a single sensory input to track the behaviour of such objects: vision can be subjected to occlusions, whereas tactile inputs cannot capture the global information that is useful for the task. In this paper, we study the problem of using vision and tactile inputs together to complete the task of fol...
-
Robotic geo-fencing and surveillance systems require accurate monitoring of objects if/when they violate perimeter restrictions. In this paper, we seek a solution for depth imaging of such objects of interest at high accuracy (few tens of cm) over extended ranges (up to 300 meters) from a single vantage point, such as a pole mounted platform. Unfortunately, the rich literature in depth imaging usi...
-
To achieve accurate and robust pose estimation in Simultaneous Localization and Mapping (SLAM) task, multisensor fusion is proven to be an effective solution and thus provides great potential in robotic applications. This paper proposes FAST-LIVO, a fast LiDAR-Inertial-Visual Odometry system, which builds on two tightly-coupled and direct odometry subsystems: a VIO subsystem and a LIO subsystem. T...
-
This paper proposes a novel extended Kalman filter (EKF) along with its adaptive variant for effective magnetic, angular rate and gravity (MARG) sensor-only pose estimation of mobile robots operated longer periods in reference-denied environments. First, a gradient-descent orientation-based EKF framework is derived, which formulates the MARG-based pose propagation with both bandpass-filtered and b...
-
Planning a safe trajectory for minimally invasive (keyhole) neurosurgery procedures require avoiding critical anatomical structures such as blood vessels and ventricles while optimizing the needle trajectory parameters such as length and curvature to comply with the needle kinematics. In this paper, we propose a reinforcement learning-based method for obtaining kinematically feasible trajectories ...
-
In this paper, we present a learning-based approach that allows a robot to quickly follow a reference path defined in joint space without exceeding limits on the position, velocity, acceleration and jerk of each robot joint. Contrary to offline methods for time-optimal path parameterization, the reference path can be changed during motion execution. In addition, our approach can utilize sensory fe...
-
This paper proposes a new regularization technique for reinforcement learning (RL) towards making policy and value functions smooth and stable. RL is known for the instability of the learning process and the sensitivity of the acquired policy to noise. Several methods have been proposed to resolve these problems, and in summary, the smoothness of policy and value functions learned mainly in RL con...
-
Recent advances in unsupervised representation learning significantly improved the sample efficiency of training Reinforcement Learning policies in simulated environments. However, similar gains have not yet been seen for real-robot reinforcement learning. In this work, we focus on enabling data-efficient real-robot learning from pixels. We present Contrastive Pre-training and Data Augmentation fo...
-
Reinforcement Learning (RL) for robotic applications can benefit from a warm-start where the agent is initialized with a pretrained behavioral policy. However, when transitioning to RL updates, degradation in performance can occur, which may compromise the robot's safety. This degradation, which constitutes an inability to properly utilize the pretrained policy, is attributed to extrapolation erro...
-
Traditional approaches to manipulating the state of deformable linear objects (DLOs) - i.e., cables, ropes - rely on model-based planning. However, constructing an accurate dynamic model of a DLO is challenging due to the complexity of interactions and a high number of degrees of freedom. This renders the task of achieving a desired DLO shape particularly difficult and motivates the use of model-f...
-
Deep reinforcement learning (deep RL) has emerged as an effective tool for developing controllers for legged robots. However, vanilla deep RL often requires a tremendous amount of training samples and is not feasible for achieving robust behaviors. Instead, researchers have investigated a novel policy architecture by incorporating human experts' knowledge, such as Policies Modulating Trajectory Ge...
-
Satisfying safety constraints is the top priority in safe reinforcement learning (RL). However, without proper exploration, an overly conservative policy such as freezing at the same position can be generated. To this end, we utilize maximum entropy RL methods for exploration. In particular, an RL method with Tsallis entropy maximization, called Tsallis actor-critic (TAC), is used to synthesize po...
-
General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach configurable goals for a wide range of tasks on command. However, such goal-conditioned policies are notoriously difficult and time-consuming to train from scratc...
-
Space robotics applications, such as Active Space Debris Removal (ASDR), require representative testing before launch. A commonly used approach to emulate the microgravity environment in space is air-bearing based platforms on flat-floors, such as the European Space Agency's Orbital Robotics and GNC Lab (ORGL). This work proposes a control architecture for a floating platform at the ORGL, equipped...
-
The reliability of autonomous traverses of rovers is critical. It may be jeopardized by the accumulation of errors and the uncertainty propagation of their localization systems. Moreover, space environments are usually harsh, challenging and unpredictable. Teleoperation is complex due to the significant and unavoidable delay. For these reasons, a path planner that provides some level of autonomy w...
-
Three-dimensional (3D) human pose estimation is one of the most basic tasks for human-interacting robots. Especially in weightless environments such as the International Space Station (ISS), wherein objects may move with a higher degree of freedom compared to on the ground, a camera with a wider field of view (FOV) is crucial in improving the probability of capturing surrounding humans. To this en...
-
Planning for autonomous operation in unknown environments poses a number of technical challenges. The agent must ensure robustness to unknown phenomena, un-predictable variation in execution, and uncertain resources, all while maximizing its objective. These challenges are ex-acerbated in the context of space missions where uncertainty is often higher, long communication delays necessitate robust ...
-
Extraterrestrial rovers with a general-purpose robotic arm have many potential applications in lunar and planetary exploration. Introducing autonomy into such systems is desirable for increasing the time that rovers can spend gathering scientific data and collecting samples. This work investigates the applicability of deep reinforcement learning for vision-based robotic grasping of objects on the ...
-
Autonomously selecting the right sequence of locations to sample is critical during exploration missions in unknown environments, with constraints on the number of samples that can be collected, and a possibility of system failure. A key idea for decision-making in unknown environments is to exploit side information available to the agent, combined with the information gained from samples collecte...
-
In robotic domains, learning and planning are complicated by continuous state spaces, continuous action spaces, and long task horizons. In this work, we address these challenges with Neuro-Symbolic Relational Transition Models (NSRTs), a novel class of models that are data-efficient to learn, compatible with powerful robotic planning methods, and generalizable over objects. NSRTs have both symboli...
-
For autonomous skill acquisition, robots have to learn about the physical rules governing the 3D world dynamics from their own past experience to predict and reason about plausible future outcomes. To this end, we propose a transformation-based 3D video prediction (T3VIP) approach that explicitly models the 3D motion by decomposing a scene into its object parts and predicting their corresponding r...
-
Non-holonomic vehicle motion has been studied extensively using physics-based models. Common approaches when using these models interpret the wheel/ground interactions using a linear tire model and thus may not fully capture the nonlinear and complex dynamics under various environments. On the other hand, neural network models have been widely employed in this domain, demonstrating powerful functi...
-
Traditional gimbal-based bionic eye systems usually use a multi-degree-of-freedom mechanical platform to move the camera freely, which makes the structure complex and bulky. The galvanometer-based reflective bionic eye system uses a galvanometer to replace the traditional mechanical rotation structure, which separates the camera from the gimbal system, greatly simplifying the structure. However, t...
-
Soft grippers have the potential to solve many existing manipulation challenges, particularly in agile industry applications. However, existing soft grippers are often limited in the range of objects they can pick, or by cluttered environments. We present a design inspired by the nose and fingers at the end of an elephant's trunk, which can pick both by suction and pinching, allowing increased gra...
-
Mantis shrimp produce one of the fastest strikes in the animal kingdom, their striking appendages reaching tip velocities of tens of meters per second underwater. Their ultrafast movement is capable of crushing the shells of prey and generating cavitation bubbles, and has long raised interest from the scientific community. To study the underlying mechanisms and operating principles behind these be...
-
There is a way to utilize humanoid robots to mimic human behavior by taking advantage of their human-like proportions. In general, motion capture is used; in this case, the posture of the body links can be taken. However, this method does not provide detailed information on the contact state, which is important for actions that involve contact with objects. In this study, we focused on the foot, w...
-
Visual navigation for insect-scale robots is very challenging because in such a small scale, the size, weight, and power (SWaP) constraints do not appear to permit visual navigation techniques such as SLAM (Simultaneous Localization and Mapping) because they are likely to be too power-hungry. We propose to use a biology-inspired approach, which we term the bilinear optic flow approximation, that i...
-
Soft, growing robots have the ability to conform to their environment and traverse highly curved paths that would typically prove challenging for other robot designs. As they navigate through these constrained and cluttered environments, there is often significant interaction between the robot and its surroundings. In this work, we propose a method to enable tactile perception for growing robots, ...
-
Natural eyeball motions in humanoid robots can contribute to friendly communication, thus improving the human-robot interaction. In this paper, we develop antagonist-agonist artificial muscles for humanoid eyeball motions, by using dielectric elastomer actuators (DEAs). Inspired by human eyeballs, the artificial muscles consist of two pairs of DEA: one pair for the horizontal motion, and the other...
-
The fingerprint effect, wherein vibrations are produced with frequencies related to the speed of a surface sliding across fingerprint ridges and the period of those ridges, has been studied for use in both slip detection and texture recognition. Here, we use a simple bioinspired sensor with parallel, straight, fingerprint-like ridges and a single ferroelectric ceramic transducer to show that the f...
-
The observation of the anatomy of agile animals and their locomotion capabilities emphasizes the importance of fast and lightweight legs and confirms the intrinsic compliance integrated into muscle-tendon units as a major ingredient for energy efficient and robust locomotion. This quality is especially relevant for distal leg segments which are subject to aggressive dynamics. Legged robots are acc...
-
With the growing application of various robots in real life, the need for an automatic anomaly detection system for robots is necessary for safety. In this paper, we develop an anomaly detection method using a stacked LSTM that can be applied to any robot controlled by a feedback control. Our method does not need installation of additional sensors. Our method is model-free and unsupervised because...
-
The ability to detect failure during task execution and to recover from failure is vital for autonomous robots performing tasks in previously unknown environments. In this paper, we present an approach for failure detection during the execution of grasping and mobile manipulation tasks by a humanoid robot. The approach combines multi-modal sensory information consisting of proprioceptive, force an...
-
Enabling vertical take-off and landing while pro-viding the ability to fly long ranges opens the door to a wide range of new real-world aircraft applications while improving many existing tasks. Tiltrotor vertical take-off and landing (VTOL) unmanned aerial vehicles (UAVs) are a better choice than fixed-wing and multirotor aircraft for such applications. Prior works on these aircraft have addresse...
-
Fault mitigation in Connected Autonomous Vehicle (CAV) platoons is faster and more reliable if the fault structure is known. In this paper we propose using transmissibility operators, which are relationships that relate a set of velocities with another in the platoon, to classify the faults. Transmissibility operators were shown to be exceptional in signals estimation; however, its also shown to b...
-
Complex robots in challenging scenarios require constant monitoring of their state and adaptation of their behavior to ensure robustness, reliability and longevity. While known possible errors can be specifically surveilled, other prob-lems can be fully unforeseen, requiring detection systems able to identify novel faults. We detect possible faults as anomalies on various internal sensor data, uti...
-
Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with...
-
In Simultaneous Localization and Mapping (SLAM), Loop Closure Detection (LCD) is essential to minimize drift when recognizing previously visited places. Visual Bag- of-Words (vBoW) has been an LCD algorithm of choice for many state-of-the-art SLAM systems. It uses a set of visual features to provide robust place recognition but fails to perceive the semantics or spatial relationship between featur...
-
With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and mapping (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve th...
-
We present the first distributed optimization al-gorithm with lazy communication for collaborative geometric estimation, the backbone of modern collaborative simultaneous localization and mapping (SLAM) and structure-from-motion (SfM) applications. Our method allows agents to cooperatively reconstruct a shared geometric model on a central server by fusing individual observations, but without the n...
-
The dynamic factors in the environment will lead to the decline of camera localization accuracy due to the violation of the static environment assumption of SLAM algorithm. Recently, some related works generally use the combination of semantic constraints and geometric constraints to deal with dynamic objects, but problems can still be raised, such as poor real-time performance, easy to treat peop...
-
Visual-inertial-odometry has attracted extensive attention in the field of autonomous driving and robotics. The size of Field of View (FoV) plays an important role in Visual-Odometry (VO) and Visual-Inertial-Odometry (VIO), as a large FoV enables to perceive a wide range of surrounding scene elements and features. However, when the field of the camera reaches the negative half plane, one cannot si...
-
Global localization is essential for autonomous mobile systems, especially indoor applications where the GPS signal is denied. Although the appearance-based methods have been successfully applied in various localization tasks, they face various challenges such as light variation, viewpoint changing, and dynamic interference. Additionally, the appearance-based methods usually require a visual featu...
-
Precise geolocalization is crucial for unmanned aerial vehicles (UAVs). However, most current deployed UAVs rely on the global navigation satellite systems (GNSS) for geolocalization. In this paper, we propose to use a lightweight visual-inertial system with a 2D georeferenced map to obtain accurate geodetic positions for UAVs. The proposed system firstly integrates a micro inertial measurement un...
-
Using pole and plane objects in lidar SLAM can increase accuracy and decrease map storage requirements compared to commonly-used point cloud maps. However, place recognition and geometric verification using these landmarks is challenging due to the requirement for global matching without an initial guess. Existing works typically only leverage either pole or plane landmarks, limiting application t...
-
This paper mainly studies the localization and mapping of range sensing robots in the confidence-rich map (CRM) and then extends it to provide a full state estimate for information-theoretic exploration. Most previous works about active simultaneous localization and mapping and exploration always assumed the known robot poses or utilized inaccurate information metrics to approximate pose uncertain...
-
In this work, an efficient homology guided belief space planning method for obstacle-cluttered environments is presented. The proposed planner follows a two-step approach. First, a h-signature guided rapidly-exploring random tree (HRRT) algorithm is proposed to provide nominal trajecto-ries in different homology classes by constructing homology aware sub-trees in a parallel manner. The HRRT planne...
-
While sampling-based approaches have made significant progress, motion planning with dynamics still poses significant challenges as the planner has to generate not only collision-free but also dynamically-feasible trajectories that enable the robot to reach its goal. To improve the efficiency of sampling-based motion planners, this paper develops a framework, termed Motion-Planning Runtime Predict...
-
Optimal sampling based motion planning and trajectory optimization are two competing frameworks to generate optimal motion plans. Both frameworks have complementary properties: Sampling based planners are typically slow to converge, but provide optimality guarantees. Trajectory optimizers, however, are typically fast to converge, but do not provide global optimality guarantees in nonconvex problem...
-
This paper proposes a model of a class of regular virtual tubes that can generate safe, feasible, and smooth space for a robotics swarm in an obstacle-dense environment, especially for a drone swarm based on the flocking model. The regular principles are first proposed, and the regular conditions are then formulated based on the principles. A method to obtain a regular virtual tube is also present...
-
In this paper, we propose a trajectory planning framework for a robot that exploits a pre-computed database of end-effector trajectories as the guidance of optimization-based inverse kinematics. We constructed a reachable graph of a robot offline, which represents feasible end-effector paths with corresponding configurations. When performing the online trajectory planning, we applied A* search to ...
-
This paper reports on insights by robotics researchers that participated in a 5-day robot-assisted nuclear disaster response field exercise conducted by Kerntechnische Hilfdienst GmbH (KHG) in Karlsruhe, Germany. The German nuclear industry established KHG to provide a robot-assisted emergency response capability for nuclear accidents. We present a systematic description of the equipment used; the...
-
This paper contributes a novel strategy towards risk-aware motion planning for collision-tolerant aerial robots subject to localization uncertainty. Attuned to the fact that micro aerial vehicles are often tasked to navigate within GPS-denied, possibly unknown, confined and obstacle-filled environments the proposed method exploits collision-tolerance at the robot design level to mitigate the risks...
-
The weight, space, and power limitations of small aerial vehicles often prevent the application of modern control techniques without significant model simplifications. Moreover, high-speed agile behavior, such as that exhibited in drone racing, make these simplified models too unreliable for safety-critical control. In this work, we introduce the concept of time-varying backup controllers (TBCs): ...
-
While both navigation and manipulation are chal-lenging topics in isolation, many tasks require the ability to both navigate and manipulate in concert. To this end, we propose a mobile manipulation system that leverages novel navigation and shape completion methods to manipulate an object with a mobile robot. Our system utilizes uncertainty in the initial estimation of a manipulation target to cal...
-
This paper presents a novel design of a multi-directional bicycle robot, which is developed for the inspection of steel structures, in particular, steel-reinforced bridges. The locomotion concept is based on arranging two magnetic wheels in a bicycle-like configuration with two independent steering actuators. This configuration allows the robot to possess multi-directional mobility. An additional ...
-
An industrial connector insertion task requires submillimeter positioning and grasp pose compensation for a plug. Thus, highly accurate estimation of the relative pose between a plug and socket is fundamental for achieving the task. World models are promising technologies for visuomotor control because they obtain appropriate state representation to jointly optimize feature extraction and latent d...
-
We propose a parallel massage robot with compliant joints based on the series elastic actuator (SEA), offering a unified force-position control approach. First, the kinematic and static force models are established for obtaining the corresponding control variables. Then, a novel force-position control strategy is proposed to separately control the force-position along the normal direction of the s...
-
In this paper we present a potential energy map based approach that provides a framework for the design and control of a robotic grasper. Unlike other potential energy map approaches, our framework considers friction for a more realistic perspective on grasper performance. Our analysis establishes the importance of considering dynamically variable geometry in grasper design, namely palm width, lin...
-
While collaborative robotic arms offer significant safety benefits, safety of the overall manipulator system cannot be guaranteed unless equally strict safety requirements are satisfied by the accompanying end-effector. Current robot grippers are not made in a way that fulfills such a requirement, resulting in collaborative robots needing to operate in a protected environment. This paper presents ...
-
Research around tactile sensing for grasp stability prediction in robotic manipulators continues to be popular, however few works are able to achieve a high classification accuracy. Due to simulation complexity, data-driven methods are often forced to rely on experimental data, yielding small, often unbalanced, data sets. In this work, the authors use a 3972 sample data set to explore the effects ...
-
This paper explores a novel approach to dexterous manipulation, aimed at levels of speed, precision, robustness, and simplicity suitable for practical deployment. The enabling technology is a Direct-drive Hand (DDHand) comprising two fingers, two DOFs each, that exhibit high speed and a light touch. The test application is the dexterous manipulation of three small and irregular parts, moving them ...
-
We propose the framework of Series Elastic End Effectors in 6D (SEED), which combines a spatially compliant element with visuotactile sensing to grasp and manipulate tools in the wild. Our framework generalizes the benefits of series elasticity to 6-dof, while providing an abstraction of control using visuotactile sensing. We propose an algorithm for relative pose estimation from visuotactile sens...
-
Vision-based robotic assembly is a crucial yet challenging task as the interaction with multiple objects requires high levels of precision. In this paper, we propose an integrated 6D robotic system to perceive, grasp, manipulate and assemble blocks with tight tolerances. Aiming to provide an off-the-shelf RGB-only solution, our system is built upon a monocular 6D object pose estimation network tra...
-
This work explores how contextual information and human intention affect the motion prediction of humans during a handover operation with a social robot. By classifying human intention in four different classes, we developed a model able to generate a different motion for each intention class. Furthermore, the model uses a multi-headed attention architecture to add contextual information to the pi...
-
We study the problem of estimating the pose of an object which is being manipulated by a multi-fingered robotic hand by only using proprioceptive feedback. To address this challenging problem, we propose a novel variant of differentiable particle filters, which combines two key extensions. First, our learned proposal distribution incorporates recent measurements in a way that mitigates weight dege...
-
We investigate the problem of planning stable grasps for object manipulations using an 18-DOF robotic hand with four fingers. The main challenge here is the high-dimensional search space, and we address this problem using a novel two-stage learning process. In the first stage, we train an autoregressive network called the hand-pose-generator, which learns to generate a distribution of valid 6D pos...
-
Multi-goal policy learning for robotic manipu-lation is challenging. Prior successes have used state-based representations of the objects or provided demonstration data to facilitate learning. In this paper, by hand-coding a high-level discrete representation of the domain, we show that policies to reach dozens of goals can be learned with a single network using Q-learning from pixels. The agent f...
-
Studies in robot teleoperation have been centered around action specifications-from continuous joint control to discrete end-effector pose control. However, these “robot-centric” interfaces often require skilled operators with extensive robotics expertise. To make teleoperation accessible to nonexpert users, we propose the framework “Scene Editing as Teleoperation” (SEaT), where the key idea is to...
-
Automated grasping of arbitrary objects is an essential skill for many applications such as smart manufacturing and human robot interaction. This makes grasp detection a vital skill for automated robotic systems. Recent work in model-free grasp detection uses point cloud data as input and typically outperforms the earlier work on RGB(D)-based methods. We show that RGB(D)-based methods are being un...
-
This paper presents a unified approach to the design of Model Predictive Controllers (MPC), custom-tailored for path following by Automated Guided Vehicles (AGVs). The approach can be applied in a unified manner to several relevant AGV kinematic configurations, including tricycle, differential, and double steer-drive. By leveraging Linear Parameter Varying (LPV) MPC, it provides maximum maneuverab...
-
Collision avoidance in emergency situations is a crucial and challenging task in motion planning for autonomous vehicles. Especially in the field of optimization-based planning using nonlinear model predictive control, many efforts to achieve real-time performance are still ongoing. Among various approaches, the iterative linear quadratic regulator (iLQR) is known as an efficient means of nonlinea...
-
In this paper, we present a novel autonomous driving framework, called a road graph and image attention network (RIANet), which computes the attention scores of objects in the image using the road graph feature. The process of the proposed method is as follows: First, the feature encoder module encodes the road graph, image, and additional features of the scene. The attention network module then i...
-
In the narrow lane scene of autonomous driving, it is critical for the ego car to recognize the intentions of social vehicles and cooperate with them. However, cooperating with social vehicles is challenging due to insufficient information. This paper proposes an Explorative Game that adopts Participant Game and Perfect Bayesian Equilibrium to exploratively perform some aggressive actions to obtai...
-
In this study, a topology-guided unified nonlinear model predictive control (NMPC) approach is proposed for autonomous navigation of a class of Hybrid Terrestrial and Aerial Quadrotors (HyTAQs) in unknown environments. The approach can fully exploit the hybrid terrestrial-aerial locomotion of the vehicle and as such ensure a high navigation efficiency. A unified terrestrial-aerial NMPC is first fo...
-
This paper presents a method for the robust selection of measurements in a simultaneous localization and mapping (SLAM) framework. Existing methods check consistency or compatibility on a pairwise basis, however many measurement types are not sufficiently constrained in a pairwise scenario to determine if either measurement is inconsistent with the other. This paper presents group-$k$ consistency ...
-
Processing large indoor scenes is a challenging task, as scan registration and camera trajectory estimation methods accumulate errors across time. As a result, the quality of reconstructed scans is insufficient for some applications, such as visual-based localization and navigation, where the correct position of walls is crucial. For many indoor scenes, there exists an image of a technical ftoorpl...
-
Visual SLAM systems targeting static scenes have been developed with satisfactory accuracy and robustness. Dynamic 3D object tracking has then become a significant capability in visual SLAM with the requirement of under-standing dynamic surroundings in various scenarios including autonomous driving, augmented and virtual reality. However, performing dynamic SLAM solely with monocular images remain...
-
Visual and lidar Simultaneous Localization and Mapping (SLAM) algorithms benefit from the Inertial Measurement Unit (IMU) modality. The high-rate inertial data complement the other lower-rate modalities. Moreover, in the absence of constant acceleration, the gravity vector makes two attitude angles out of three observable in the global coordinate frame. In visual odometry, this is already being us...
-
Lidar-based SLAM systems perform well in a wide range of circumstances by relying on the geometry of the environment. However, even mature and reliable approaches struggle when the environment contains structureless areas such as long hallways. To allow the use of lidar-based SLAM in such environments, we propose to add reflector markers in specific locations that would otherwise be difficult. We ...
-
Visual SLAM inside the human body will open the way to computer-assisted navigation in endoscopy. However, due to space limitations, medical endoscopes only provide monocular images, leading to systems lacking true scale. In this paper, we exploit the controlled lighting in colonoscopy to achieve the first in-vivo 3D reconstruction of the human colon using photometric stereo on a calibrated monocu...
-
The recent momentum in aerial manipulation has led to an interest in developing virtual reality interfaces for aerial physical interaction tasks with simple, intuitive, and reliable control and perception. However, this requires the use of expensive subsystems and there is still a research gap between interface design, user evaluations and the effect on aerial manipulation tasks. Here, we present ...
-
This paper presents an easy-to-deploy, virtual reality-based teleoperation system for controlling a robot arm. The proposed system is based on a consumer-grade virtual reality device (Oculus Quest 2) with a low-cost robot arm (a LoCoBot) to allow easy replication and set up. The proposed Work-from-Home Virtual Reality (WFH-VR) system allows the user to feel an intimate connection with the real rem...
-
Most virtual reality (VR) applications use a commercial controller for interaction. However, a typical virtual reality controller (VRC) lacks positional precision and accu-racy in millimeter-scale scenarios. This lack of precision and accuracy is caused by built-in sensors drift. Therefore, the tracking performance of a VRC needs to be enhanced for millimeter-scale scenarios. Herein, we introduce ...
-
In this paper, we propose a novel torque controller for the implementation virtual remote center of motion. The controller allows the system to implement the required behavior and guarantees the satisfaction of the remote center of motion constraint. Exploiting the Udwadia-Kalaba equation for constrained dynamic systems, the controller is synthesized considering the dynamic effect the constraint p...
-
Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Se...
-
Research attention on natural user interfaces (NUIs) for drone flights are rising. Nevertheless, NUIs are highly diversified, and primarily evaluated by different physical environments leading to hard-to-compare performance between such solutions. We propose a virtual environment, namely VRFlightSim, enabling comparative evaluations with enriched drone flight details to address this issue. We firs...
-
This paper presents a customized programmable robotic system, TanTwin (Tangible Twin), designed to promote STEM education for K-12 children. Firstly, TanTwin is implemented based on a wheel-robot with standard LEGO bricks. With several deep neural networks, a child can convert a captured portrait of himself/herself into standard LEGO bricks, therefore he/she can build a tangible twin robot of him-...
-
Suspended Cable-Driven Parallel Robots (SCDPR) have intriguing capabilities on large scales but still have open challenges in precisely estimating the end-effector pose. The cables exhibit a downward curved shape, also known as cable sag which needs to be accounted for in the pose estimation. The catenary equations can accurately describe this phenomenon but are only accurate in equilibrium condit...
-
This study focuses on replicating the muscu-loskeletal system of human arms for mimicking its movement. Muscle redundancy is critical for regulating the mechanical impedance of arms and legs. However, when implementing muscle redundancy on robots, making an ill-posed problem that cannot determine the muscle forces uniquely. In this paper, first, a method for controlling end-point stiffness in the ...
-
Cable-Driven Parallel Robots (CDPRs) have been proposed for a variety of applications such as material handling, rehabilitation, and instrumentation. However, the collision-free constraint of CDPRs limits the workspace of CDPRs and the feasible position of anchor points. To address the collision-free constraint of CDPRs, a data-driven kinematic control scheme is developed for CDPRs, enabling a CDP...
-
Legged robots with high locomotive performance have been extensively studied, and various leg structures have been proposed. Especially, a leg structure that can achieve both continuous and high jumps is advantageous for moving around in a three-dimensional environment. In this study, we propose a parallel wire-driven leg structure, which has one DoF of linear motion and two DoFs of rotation and i...
-
This paper addresses the problem of learning control policies for mobile robots, modeled as unknown Markov Decision Processes (MDPs), that are tasked with temporal logic missions, such as sequencing, coverage, or surveillance. The MDP captures uncertainty in the workspace structure and the outcomes of control decisions. The control objective is to synthesize a control policy that maximizes the pro...
-
Planning under uncertainty is a fundamental problem in robotics. Classical approaches rely on a metrical representation of the world and robot's states to infer the next course of action. While these approaches are considered accurate, they are often susceptible to metric errors and tend to be costly regarding memory and time consumption. However, in some cases, relying on qualitative geometric in...
-
A fully tested autonomous system works predictably under ideal or assumed environment. However, its behavior is not fully defined when some components malfunction or fail. In this paper, we consider automated guided vehicle (AGV), equipped with multiple sensors, executing a traversal task in a static unknown environment. We have analytically studied the system, computed a set of performance and sa...
-
Vacuum-adhesion-based climbing robots have been developed to cater to the demands in the cleaning and inspection work of airplanes. A robot intended to clean and inspect an airplane faces a Risk of Adhesion (RoA) based on the robot and the surface conditions, such as worn-out suction cups. These sorts of underlying conditions are not easily noticeable for an operator of a robot and might lead to c...
-
Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-ba...
-
Evaluating safety performance in a resource-efficient way is crucial for the development of autonomous systems. Simulation of parameterized scenarios is a popular testing strategy but parameter sweeps can be prohibitively expensive. To address this, we propose HiddenGems: a sample-efficient method for discovering the boundary between compliant and non-compliant behavior via active learning. Given ...
-
Research on Deep Neural Networks (DNNs) has focused on improving performance and accuracy for real-world deployments, leading to new models, such as Spiking Neural Networks (SNNs), and optimization techniques, e.g., quantization and pruning for compressed networks. However, the deployment of these innovative models and optimization techniques introduces possible reliability issues, which is a pill...
-
While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment environments of RAS pose new challenges on its dependability. Although existing works impose constraints on the DRL policy to ensure successful completion of the mission, it is far from adequate to assess the DR...
-
The dynamic response of the legged robot locomotion is non-Lipschitz and can be stochastic due to environmental uncertainties. To test, validate, and characterize the safety performance of legged robots, existing solutions on observed and inferred risk can be incomplete and sampling inefficient. Some formal verification methods suffer from the model precision and other surrogate assumptions. In th...
-
Human activities recognition is an important task for an intelligent robot, especially in the field of human-robot collaboration, it requires not only the label of sub-activities but also the temporal structure of the activity. In order to automatically recognize both the label and the temporal structure in sequence of human-object interaction, we propose a novel Pyramid Graph Convolutional Networ...
-
Legged locomotion in humans is governed by natural dynamics of the human body and neural control. One mechanism that is assumed to contribute to the high efficiency of human walking is the impulsive ankle push-off, which potentially powers the swing leg catapult. However, the mechanics of the human lower leg with its complex muscle-tendon units spanning over single and multiple joints is not yet u...
-
Dynamic motions of humans and robots are widely driven by posture-dependent nonlinear interactions between their degrees of freedom. However, these dynamical effects remain mostly overlooked when studying the mechanisms of human movement generation. Inspired by recent works, we hypothesize that human motions are planned as sequences of geodesic synergies, and thus correspond to coordinated joint m...
-
Manipulability ellipsoids efficiently capture the human pose and reveal information about the task at hand. Their use in task-dependent robot teaching - particularly their transfer from a teacher to a learner - can advance emulation of human-like motion. Although in recent literature focus is shifted towards manipulability transfer between two robots, the adaptation to the capabilities of the othe...
-
Humans are able to negotiate downstep behaviors-both planned and unplanned-with remarkable agility and ease. The goal of this paper is to systematically study the translation of this human behavior to bipedal walking robots, even if the morphology is inherently different. Concretely, we begin with human data wherein planned and unplanned downsteps are taken. We analyze this data from the perspecti...
-
Walking with load is a common task in daily life and disaster rescue. Long-term load carriage may cause irreversible damage to the human body. Although remarkable progress has been made in the field of wearable robots, it is still far from avoiding interference to human legs, which will lead to energy consumption. In this paper, a novel wearable robot, Centaur, for assisting load carriage has been...
-
This study aims to construct a running simulator based on a motion generation and control system that enables the description of motion strategies using the spring-loaded inverted pendulum (SLIP) model. The problems of stability and robustness encountered in the running simulation with the SLIP model are elucidated, and stable running is achieved by controlling the stiffness and the attitude angle...
-
We tackle the problem of tracking the human lower body as an initial step toward an automatic motion assessment system for clinical mobility evaluation, using a multimodal system that combines Inertial Measurement Unit (IMU) data, RGB images, and point cloud depth measurements. This system applies the factor graph representation to an optimization problem that provides 3-D skeleton joint estimatio...
-
The semantic segmentation of surgical scenes is a prerequisite for task automation in robot assisted interventions. We propose LapSeg3D, a novel DNN-based approach for the voxel-wise annotation of point clouds representing surgical scenes. As the manual annotation of training data is highly time consuming, we introduce a semi-autonomous clustering-based pipeline for the annotation of the gallbladd...
-
Egocentric vision is an emerging topic, which has demonstrated great potential in assistive healthcare scenarios, ranging from human-centric behavior analysis to personal social assistance. Within this field, due to the heterogeneity of visual perception from first-person views, egocentric pose estimation is one of the most significant prerequisites for enabling various downstream applications. Ho...
-
Monocular SLAM in deformable scenes will open the way to multiple medical applications like computer-assisted navigation in endoscopy, automatic drug delivery or autonomous robotic surgery. In this paper we propose a novel method to simultaneously track the camera pose and the 3D scene deformation, without any assumption about environment topology or shape. The method uses an illumination-invarian...
-
Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the in-consistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery...
-
Depth estimation from panoramic imagery has received minimal attention in contrast to standard perspective imagery, which constitutes the majority of the literature on the key research topic. The vast - and frequently complete - field of view provided by such panoramic photographs makes them appealing for a variety of applications, including robots, autonomous vehicles, and virtual reality. Consum...
-
An increasing number of nonspecialist robotic users demand easy-to-use machines. In the context of visual servoing, the removal of explicit image processing is becoming a trend, allowing an easy application of this technique. This work presents a deep learning approach for solving the perception problem within the visual servoing scheme. An artificial neural network is trained using the supervisio...
-
Creating accurate maps of complex, unknown environments is of utmost importance for truly autonomous navigation robot. However, building these maps online is far from trivial, especially when dealing with large amounts of raw sensor readings on a computation and energy constrained mobile system, such as a small drone. While numerous approaches tackling this problem have emerged in recent years, th...
-
Safe motion planning in robotics requires planning into space which has been verified to be free of obstacles. However, obtaining such environment representations using lidars is challenging by virtue of the sparsity of their depth measurements. We present a learning-aided 3D lidar reconstruction framework that upsamples sparse lidar depth measurements with the aid of overlapping camera images so ...
-
Scalable Fiducial Tag Localization on a 3D Prior Map via Graph-Theoretic Global Tag-Map Registration
This paper presents an accurate and scalable method for fiducial tag localization on a 3D prior environmental map. The proposed method comprises three steps: 1) visual odometry-based landmark SLAM for estimating the relative poses between fiducial tags, 2) geometrical matching-based global tag-map registration via maximum clique finding, and 3) tag pose refinement based on direct camera-map alignm...
-
High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds r...
-
This work presents a humanoid soft robotics finger design with rigid skeletons and proprioceptive sensors. This 4-DOFs dexterous finger has soft joints and rigid phalanxes, which is about the size of human hand. To enhance the overall stiffness and for human-like behavior and configuration, rigid-soft actuators which we called quasi-joints are introduced. Although their lengths are shortened in th...
-
There are many spaces inaccessible to humans where robots could help deliver sensors and equipment. Many of these spaces contain three-dimensional passageways and uneven terrain that pose challenges for robot design and control. Everting toroidal robots, which move via simultaneous eversion and inversion of their body material, are promising for navigation in these types of spaces. We present a no...
-
As soft, continuum robots see increasing areas of application, many scenarios have arisen where it is necessary to consider the geometric shape of the robot. The current approaches to robot kinematics, such as the piecewise constant-curvature (PCC) model, are effective in representing simple overall robot geometry and estimating the end-effector state, but they are less intuitive for planning robo...
-
Grasping and manipulation are complex and demanding tasks, especially when executed in dynamic and unstructured environments. Typically, such tasks are executed by rigid articulated end-effectors, with a plethora of actuators that need sophisticated sensing and complex control laws to execute them efficiently. Soft robotics offers an alternative that allows for simplified execution of these demand...
-
Recent advances in soft robotics in academia have led to the adoption of soft grippers in industrial settings. Due to their soft bending actuators, these grippers can handle delicate objects with great care. However, due to their flexibility, the actuators are prone to out-of-plane deformations upon asymmetric loading. These undesired deformations lead to reduced grasp performance and may cause in...
-
The optimal stiffness for soft swimming robots depends on swimming speed, which means no single stiffness can maximise efficiency in all swimming conditions. Tunable-stiffness would produce an increased range of high-efficiency swimming speeds for robots with flexible propulsors and enable soft control surfaces for steering underwater vehicles. We propose and demonstrate a method for tunable soft ...
-
Having accurate localization capabilities is one of the fundamental requirements of autonomous robots. For underwater vehicles, the choices for effective localization are limited due to limitations of GPS use in water and poor environ-mental visibility that makes camera-based methods ineffective. Popular inertial navigation methods for underwater localization using Doppler-velocity log sensors, so...
-
VPR is a fundamental task for autonomous navigation as it enables a robot to localize itself in the workspace when a known location is detected. Although accuracy is an essential requirement for a VPR technique, computational and energy efficiency are not less important for real-world applications. CNN-based techniques archive state-of-the-art VPR performance but are computationally intensive and ...
-
Accurate and real-time position estimates are cru-cial for mobile robots. This work focuses on ranging-based positioning systems, which rely on distance measurements between known points, called anchors, and a tag to localize. The topology of the network formed by the anchors strongly influences the tag's localizability, i.e., its ability to be accurately localized. Here, the tag and some anchors ...
-
Developing robust vision-guided controllers for quadrupedal robots in complex environments with various obstacles, dynamical surroundings and uneven terrains is very challenging. While Reinforcement Learning (RL) provides a promising paradigm for agile locomotion skills with vision inputs in simulation, it is still very challenging to deploy the vision-guided RL policy in the real world. Our key i...
-
Model-free Deep Reinforcement Learning (DRL) controllers have demonstrated promising results on various challenging non-linear control tasks. While a model-free DRL algorithm can solve unknown dynamics and high-dimensional problems, it lacks safety assurance. Although safety constraints can be encoded as part of a reward function, there still exists a large gap between an RL controller trained wit...
-
Aiming at the problem that it is difficult to calculate the force of permanent magnets in the magnetic field, this paper proposes a nonlinear mechanical model of linear array magnetic field based on radial basis function neural network (RBFNN). Combined with the linear Halbach array adsorption module of the wall-climbing robot, the three-dimensional geometric magnetic fields of four typical linear...
-
This paper presents the design, analysis, and performance evaluation of an omnidirectional transformable wheel-leg robot called OmniWheg. We design a novel mechanism consisting of a separable omni-wheel and 4-bar linkages, allowing the robot to transform between omni-wheeled and legged modes smoothly. In wheeled mode, the robot can move in all directions and efficiently adjust the relative positio...
-
This paper introduces SCALER, a quadrupedal robot that demonstrates climbing on bouldering walls, over-hangs, ceilings and trotting on the ground. SCALER is one of the first high-degrees of freedom four-limbed robots that can free-climb under the Earth's gravity and one of the most mechanically efficient quadrupeds on the ground. Where other state-of-the-art climbers specialize in climbing, SCALER...
-
Microspine grippers allow robots to ascend steep rocky slopes and cliff faces, enabling scientific exploration of exposed strata on Earth and other solar system bodies. Historically, the Shape Deposition Manufacturing (SDM) process has been used to fabricate multi-material suspensions for load-sharing among multiple microspines. We instead apply the Hybrid Deposition Manufacturing (HDM) process to...
-
A spherical robotic probe has several advantages in rough environments and has therefore raised interest for application in planetary exploration. A sphere is well-suited to protect high-sensitive payloads, however, the locomotion system for planetary surfaces raises several challenges. This paper presents a novel locomotion system consisting of linear actuators which are usable in a multi-functio...
-
Extended Kalman filtering is a common approach to achieve floating base estimation of a humanoid robot. These filters rely on measurements from an Inertial Measurement Unit (IMU) and relative forward kinematics for estimating the base position-and-orientation and its linear velocity along with the augmented states of feet position-and-orientation. We refer to such filters as flat-foot filters. How...
-
Pose estimation is the task of determining the 6D position of an object in a scene. Pose estimation aid the abilities and flexibility of robotic set-ups. However, the system must be configured towards the use case to perform adequately. This configuration is time-consuming and limits the usability of pose estimation and, thereby, robotic systems. Deep learning is a method to overcome this configur...
-
Light Detection And Rangings (LiDARs) have been widely adopted to modern self-driving vehicles, providing 3D information of the scene and surrounding objects. However, adverser weather conditions still pose significant challenges to LiDARs since point clouds captured during snowfall can easily be corrupted. The resulting noisy point clouds degrade downstream tasks such as mapping. Existing works i...
-
Feature extraction or localization based on the fiducial marker could fail due to motion blur in real-world robotic applications. To solve this problem, a lightweight generative adversarial network, named Ghost-DeblurGAN, for real-time motion deblurring is developed in this paper. Furthermore, on account that there is no existing deblurring benchmark for such task, a new large-scale dataset, York-...
-
Cloud robotics, i.e., controlling robots from the cloud, make it possible to perform more complex processes, make robots smaller, and coordinate multi-robots by sharing information between robots and utilizing abundant computing resources. In cloud robotics, robots need to transmit videos to the cloud in real time to recognize their surroundings. Lowering the video quality reduces the bitrate in l...
-
We present an unsupervised simultaneous learning framework for the task of monocular camera re-localization and depth estimation from unlabeled video sequences. Monocular camera re-localization refers to the task of estimating the absolute camera pose from an instance image in a known environment, which has been intensively studied for alternative localization in GPS-denied environments. In recent...
-
Scanning real-life scenes with modern registration devices typically give incomplete point cloud representations, mostly due to the limitations of the scanning process and 3D occlusions. Therefore, completing such partial representations remains a fundamental challenge of many computer vision applications. Most of the existing approaches aim to solve this problem by learning to reconstruct individ...
-
Flexible Collision-free Platooning Method for Unmanned Surface Vehicle with Experimental Validations
This paper addresses the flexible formation problem for unmanned surface vehicles in the presence of obstacles. Building upon the leader-follower formation scheme, a hybrid line-of-sight based flexible platooning method is proposed for follower vehicle to keep tracking the leader ship. A fusion artificial potential field collision avoidance approach is tailored to generate optimal collision-free t...
-
Marine autonomous vehicles deployed to conduct marine geophysical surveys are becoming an increasingly used asset in the commercial, academic, and defense industries. However, the ability to collect high-quality data from applicable sensors is directly related to the robustness of vehicle motion caused by environmental disturbances. In this paper we designed and integrated a new path following con...
-
This paper deals with the estimation of the shape of a catenary for a negatively buoyant cable, connecting a pair of underwater robots in a robot chain. The new estimation method proposed here is based on the calculation of local tangents thanks to the data acquired from inertial measurement units (IMUs), which are attached to the cable near its ends. This method is compared with a vision-based es...
-
Navigation and obstacle avoidance in aquatic en-vironments for autonomous surface vehicles (ASVs) in high-traffic maritime scenarios is still an open challenge, as the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) is not defined for multi-encounter situations. Current state-of-the-art methods resolve single-to-single encounters with sequential actions and a...
-
Developing a good control strategy for biomimetic robots is challenging. Robust control methods require an accurate model of the robot. Nowadays, model-free methods are being extensively explored for the control and navigation of terrestrial robots. In this paper, we consider a novel deep reinforcement learning-based model-free swimming control for our bio-inspired robotic tadpole. To realize this...
-
This paper proposes an underwater target perception framework to comprehensively explore target information in underwater scenes, to improve the work efficiency and safety of underwater operations. This framework adopts a layered processing mechanism including water column imaging, constant false alarm rate detection (CFAR) detection, and local feature analysis, to accurately distinguish between f...
-
Electroadhesion is a versatile and controllable adhesion mechanism that has been used extensively in robotics. Soft electroadhesion embodies electrostatic adhesion in soft materials and is required for shape-adaptive and safe grasping of curved objects and delicate materials. In this work, we present a soft electroadhesive fabrication method based on laser scribing graphene oxide on a silicone fil...
-
Tendon-driven flexible endoscopic surgical robots have been developed to access narrow curved paths without incision. Robot shape information is essential for precise control and to prevent unwanted tissue damage. In this paper, we propose a joint angle sensing method using coiled soft sensors to estimate the shape of the hyperredundant manipulator, which is commonly used in flexible endoscopic su...
-
Achieving compact and biocompatible actuators with sensing capabilities is a key challenge for the safety critical and highly patient-specific biomedical field. In this study, a compact and versatile soft fluidic sensor-actuator capable of measuring both force and displacement in static and dynamic conditions is presented. Pressure and resistance are shown to be interchangeable in predicting load ...
-
This paper presents a model-based approach to reconstructing the large deformations of slender flexible beams through strain-gauge deflection sensors. Using the principal axes decomposition of structural compliance, a closed-form kinetostatics model can be obtained to characterize the non-linear force-deformation behavior of the flexible beams under-going large-scale deflection. Owing to analytica...
-
This paper presents a new 6 DOF soft robotic manipulator intended for colorectal surgery. The manipulator, based on a novel design that employs an inextensible tube to limit axial extension, is shown to maximize the force exerted at its tip and the bending angle, the latter being measured with a soft sensing skin. Manufacturing of the prototype is achieved with a lost-wax silicone-casting techniqu...
-
Navigation inside luminal organs is an arduous task that requires non-intuitive coordination between the movement of the operator's hand and the information obtained from the endoscopic video. The development of tools to automate certain tasks could alleviate the physical and mental load of doctors during interventions allowing them to focus on diagnosis and decision-making tasks. In this paper we...
-
Recent years have seen the introduction of more affordable but less accurate 2D range sensors whose field of view is $2\pi$. Scan-matching with these has been insufficiently researched, while being a challenge due to these sensors' increased measurement uncertainty. This paper proposes a real-time method for matching scans extracted from panoramic 2D LIDAR sensors. The method leverages properties ...
-
This study proposes a visual localization method using a vector map and voxel grid map with a stereo camera. The two maps provide different modality advantages and are integrated using a particle filter. In contrast to other vector map-based methods, our method does not use road markings because creating and maintaining vector maps that include high-accuracy road markings is laborious. Furthermore...
-
Indoor localization systems often fuse inertial odometry with map information via hand-defined methods to reduce odometry drift, but such methods are sensitive to noise and struggle to generalize across odometry sources. To address the robustness problem in map utilization, we propose a data-driven prior on possible user locations in a map by combining learned spatial map embeddings and temporal o...
-
We describe a novel approach to indoor place recognition from RGB point clouds based on aggregating low-level colour and geometry features with high-level implicit semantic features. It uses a 2-stage deep learning framework, in which the first stage is trained for the auxiliary task of semantic segmentation and the second stage uses features from layers in the first stage to generate discriminate...
-
Visual odometry is an important part of the perception module of autonomous robots. Recent advances in deep learning approaches have given rise to hybrid visual odometry approaches that combine both deep networks and traditional pose estimation methods. One limitation of deep learning approaches is the availability of ground truth data needed to train the neural networks. For example, it is extrem...
-
This paper is about extremely robust and lightweight localisation using LiDAR point clouds based on instance segmentation and graph matching. We model 3D point clouds as fully-connected graphs of semantically identified components where each vertex corresponds to an object instance and encodes its shape. Optimal vertex association across graphs allows for full 6-Degree-of-Freedom (DoF) pose estima...
-
This paper presents the two sub-systems of the first holistic system for autonomous obstacle detection (OD) in railways, the on-board vision system and unmanned aerial vehicle (UAV) vision system for object localization (OL) on and near the rail tracks. The main goal of such a holistic system is to enable long-range detection of obstacles on the rail tracks ahead of the train where the UAV-based O...
-
Robotic applications are continuously striving towards higher levels of autonomy. To achieve that goal, a highly robust and accurate state estimation is indispensable. Combining visual and inertial sensor modalities has proven to yield accurate and locally consistent results in short-term applications. Unfortunately, visual-inertial state estimators suffer from the accumulation of drift for long-t...
-
This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. Geo- tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used fo...
-
This article presents an entirely data-driven approach for autonomous control of redundant manipulators with hydraulic actuation. The approach only requires minimal system information, which is inherited from a simulation model. The non-linear hydraulic actuation dynamics are modeled using actuator networks from the data gathered during the manual operation of the manipulator to effectively emulat...
-
In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. For this purpose, we impl...
-
Deep reinforcement learning (RL) is a promising approach to solving complex robotics problems. However, the process of learning through trial-and-error interactions is often highly time-consuming, despite recent advancements in RL algorithms. Additionally, the success of RL is critically dependent on how well the reward-shaping function suits the task, which is also time-consuming to design. As ag...
-
In this work, we introduce a new perspective for learning transferable content in multi-task imitation learning. Humans are capable of transferring skills and knowledge. If we can cycle to work and drive to the store, we can also cycle to the store and drive to work. We take inspiration from this and hypothesize the latent memory of a policy network can be disentangled into two partitions. These c...
-
Although the soft robot is gaining considerable popularity in dexterous and safe manipulation, accurate motion control is still an open problem to be explored. Recent investigations suggest that reinforcement learning (RL) is a promising solution but lacks efficient adaptability for Sim2Real transfer or environment variations. In this paper, we present a deep deterministic policy gradient (DDPG)-b...
-
The paper proposes a path planning solution for autonomous robotic exploration of complex subterranean environments. The work contributes to the family of graph-based planners by bringing the following improvements. Firstly, an occupancy grid-based sample-and-project solution to terrain-assessment is proposed instead of building an explicit elevation map of the environment. Secondly, the solution-...
-
As a hard high-altitude work, power transmission line inspection increasingly demands robots to conduct in place of the human being. A variety of robots have been developed to this end, with basic locomotion and inspection implemented on the lines. However, most current line inspection robots (LIRs) are only mobile platforms with complex structures and large weights, lacking sufficiently dexterous...
-
Inspection of subsea pipelines is crucial for avoiding any hazards and minimizing the risks to infrastructure and the environment. These inspections are achieved using Autonomous Underwater Vehicles (AUVs) in favour of reduced operational costs. This work presents a vehicle agnostic approach for tracking subsea pipelines at close-range for autonomous guidance along the pipeline using an AUV. A mul...
-
Extensive research has been conducted on autonomous surface robots and underwater robots for various tasks in aquatic environments. The duration of the operation of autonomous field robots depends on the capacity of the mounted battery, as they are not typically connected to an external power supply. Therefore, smart strategies which are optimized for each task are required to extend the working t...
-
To achieve autonomy in complex real-world exploration missions, we consider deployment strategies for a team of robots with heterogeneous capabilities. We formulate a multi-robot exploration mission and compute an operation policy to maintain robot team productivity and maximize mission success. The environment description, robot capability, and mission outcome are modeled as a Markov decision pro...
-
This paper presents a framework for Multi-Modal SLAM (MIMOSA) that utilizes a nonlinear factor graph as the underlying representation to provide loosely-coupled fusion of any number of sensing modalities. Tailored to the goal of enabling resilient robotic autonomy in GPS-denied and perceptually-degraded environments, MIMOSA currently contains modules for pointcloud registration, fusion of multiple...
-
Autonomous navigation of ground robots has been widely used in indoor structured 2D environments, but there are still many challenges in outdoor 3D unstructured environments, especially in rough, uneven terrains. This paper proposed a plane-fitting based uneven terrain navigation framework (PUTN) to solve this problem. The implementation of PUTN is divided into three steps. First, based on Rapidly...
-
This paper presents a method of drawing pipeline routes using a sensing unit with a rotary encoder and IMU (Inertial Measurement Unit), which is towed by a self-propelled in-pipe inspection robot. However, the IMU information generally contains integration errors, making it difficult to draw accurate pipeline routes. In this study, we propose a method combining gradient descent using a gyroscopic ...
-
In this paper, we consider online planning in par-tially observable domains. Solving the corresponding POMDP problem is a very challenging task, particularly in an online setting. Our key contribution is a novel algorithmic approach, Simplified Information Theoretic Belief Space Planning (SITH-BSP), which aims to speed up POMDP planning considering belief-dependent rewards, without compromising th...
-
Planning collision-free motions for robots with many degrees of freedom is challenging in environments with complex obstacle geometries. Recent work introduced the idea of speeding up the planning by encoding prior experience of successful motion plans in a neural network. However, this “neural motion planning” did not scale to complex robots in unseen 3D environments as needed for real-world appl...
-
This paper presents a motion planning strategy that utilizes the improved A -search guided tree to enable autonomous parking of a general 3-trailer with a car-like tractor. Different from the state-of-the-art state-lattice-based methods, where numerous motion primitives are necessary to ensure successful planning, our work allows quick off-lattice exploration to find a solution. Our treatment brin...
-
We present TerraPN, a novel method that learns the surface properties (traction, bumpiness, deformability, etc.) of complex outdoor terrains directly from robot-terrain interactions through self-supervised learning, and uses it for autonomous robot navigation. Our method uses RGB images of terrain surfaces and the robot's velocities as inputs, and the IMU vibrations and odometry errors experienced...
-
Signal temporal logic (STL) provides a powerful, flexible framework for specifying complex autonomy tasks; however, existing methods for planning based on STL specifications have difficulty scaling to long-horizon tasks and are not robust to external disturbances. In this paper, we present an algorithm for finding robust plans that satisfy STL specifications. Our method alternates between local op...
-
We present a decentralized path-planning algorithm for navigating multiple differential-drive robots in dense environments. In contrast to prior decentralized methods, we propose a novel congestion metric-based replanning that couples local and global planning techniques to efficiently navigate in scenarios with multiple corridors. To handle dense scenes with narrow passages, our approach computes...
-
Collective decision-making enables self-organizing robot swarms to act autonomously on a swarm level and is essential to coordinate their actions as a whole. When robots only share and communicate information locally a distributed and decentralized approach is required. In a previous paper [4], an efficient method based on a distributed Bayesian algorithm was created to distinguish a binary enviro...
-
We propose a decentralized control algorithm for a minimalistic robotic swarm with limited capabilities such that the desired global behavior emerges. We consider the problem of searching for and encapsulating various targets present in the environment while avoiding collisions with both static and dynamic obstacles. The novelty of this work is the guaranteed generation of desired complex swarm be...
-
In order to facilitate and assist the indoor mobility of people with special needs, the classically static objects in the environment, such as furniture, can be rendered mobile. The need for efficient and safe autonomous coordination of a mobile furniture swarm arises. We present a closed-form approach for mobile furniture obstacle avoidance and navigation within an indoor environment. The approac...
-
One of the main tasks for autonomous robot swarms is to collectively decide on the best available option. Achieving that requires a high quality communication between the agents that may not always be available in a real world environment. In this paper we introduce the communication-constrained collective decision-making problem where some areas of the environment limit the agents' ability to com...
-
Decision-making among groups of humans can benefit from open discussion and inclusion of a diversity of opinions, promoting deliberative democracy. In this work, we test whether a swarm of robots can help facilitate decision-making by visually representing the diversity of opinions. We used a swarm of robots we built, called MOSAIX, that consists of 4-inch touchscreens-on-wheels robots called Tile...
-
An environment where a robot swarm attacks a territory protected by another one leads to an attack-defense confrontation problem. Commonly-used deep reinforcement learning-based methods rely on pre-training and become intractable due to the curse of dimensionality. To develop effective attack strategies, inspired by a particle swarm optimizer (PSO), this work proposes a PSO-based strategy for a ro...
-
Multi-Objective Geometric Optimization of A Multi-Link Manipulator Using Parameterized Design Method
The performance of a robot is closely related to its structure. From the initial design of link lengths to structural optimization, it is still the research hotspot in recent years. To make the manipulator lightweight and ensure its working range and flexibility, researchers have proposed many optimization methods, most of which are for specific working scenarios, requirements, and robot structure...
-
Over the last years, research done in automation and industrial robotics has established the foundations for skill-oriented systems based on the OPC UA standard. Nevertheless, utilizing these advances in other areas of robotics research can be challenging and time consuming. We present a framework aiming to reduce this entry threshold. Our solution is an open source, easy to configure tool based o...
-
Programming paths for robotic welding conventionally requires precise positioning of workpieces, detailed 3D models and/or tedious teach pendant programming. A new method is introduced in this paper that enables an operator to teach the weld path to the robot through a haptic-visual interface. The operator teaches the path by guiding the tool tip to contact on the workpiece surface with force feed...
-
Human safety has always been the main priority when working near an industrial robot. With the rise of Human-Robot Collaborative environments, physical barriers to avoiding collisions have been disappearing, increasing the risk of accidents and the need for solutions that ensure a safe Human-Robot Collaboration. This paper proposes a safety system that implements Speed and Separation Monitoring (S...
-
For traditional cable-driven parallel robots (CD-PRs), changing the workspace is relatively difficult, which needs to reconfigure the anchor points and the external frame. The main reason is that the winch is separated from the moving platform, and a series of pulleys are applied to guide the driving cables. This paper proposes a novel all-in-one suspended CDPR that integrates all components in th...
-
Industrial robots require force controllability and impact resistance to ensure safe physical interactions. An electro-hydrostaic actuator (EHA) is expected to be suitable for such applications because it has high backdrivability which improve both force controllability at contact and impact resistance. However, EHAs had been rarely used in multi-axes robotic systems. The previous works validated ...
-
Using a delta-manipulator for stabilisation of an end-effector to perform precise spatial positioning is a current area of interest in aerial manipulation. High speed precision movements of a manipulator can cause disturbances to the aerial platform, which hinders trajectory tracking and in some cases could be sufficient to cause a loss of control of the vehicle. In this paper, a statically balanc...
-
We present a locally optimal tracking controller for Cable Driven Parallel Robot (CDPR) control based on a time-varying Linear Quadratic Gaussian (TV-LQG) controller. In contrast to many methods which use fixed feedback gains, our time-varying controller computes the optimal gains depending on the location in the workspace and the future trajectory. Meanwhile, we rely heavily on offline computatio...
-
We aim to teach robots to perform simple object manipulation tasks by watching a single video demonstration. Towards this goal, we propose an optimization approach that outputs a coarse and temporally evolving 3D scene to mimic the action demonstrated in the input video. Similar to previous work, a differentiable renderer ensures perceptual fidelity between the 3D scene and the 2D video. Our key n...
-
Daily manipulation tasks are characterized by geometric primitives related to actions and object shapes. Such geometric descriptors are poorly represented by only using Cartesian coordinate systems. In this paper, we propose a learning approach to extract the optimal representation from a dictionary of coordinate systems to encode an observed movement/behavior. This is achieved by using an extensi...
-
Learning from Demonstration (LfD) is a popular method of reproducing and generalizing robot skills from human-provided demonstrations. In this paper, we propose a novel optimization-based LfD method that encodes demon-strations as elastic maps. An elastic map is a graph of nodes connected through a mesh of springs. We build a skill model by fitting an elastic map to the set of demonstrations. The ...
-
Learning robotic tasks in the real world is still highly challenging and effective practical solutions remain to be found. Traditional methods used in this area are imitation learning and reinforcement learning, but they both have limitations when applied to real robots. Combining reinforcement learning with pre-collected demonstrations is a promising approach that can help in learning control pol...
-
Despite significant advancements, collision-free navigation in autonomous driving is still challenging, considering the navigation module needs to balance learning and planning to achieve efficient and effective control of the vehicle. We propose a novel framework of inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning (IRL-HC) for autonomous maneuver...
-
This paper presents a method for learning logical task specifications and cost functions from demonstrations. Constructing specifications by hand is challenging for complex objectives and constraints in autonomous systems. Instead, we consider demonstrated task executions, whose logic structure and transition costs need to be inferred by an autonomous agent. We employ a spectral learning approach ...
-
Category-level pose estimation is a challenging problem due to intra-class shape variations. Recent methods deform pre-computed shape priors to map the observed point cloud into the normalized object coordinate space and then retrieve the pose via post-processing, i.e., Umeyama's Algorithm. The shortcomings of this two-stage strategy lie in two aspects: 1) The surrogate supervision on the intermed...
-
In this paper, we propose a novel Voting based Grasp Pose Network (VGPN) to detect 6-DoF grasps in cluttered scenes. The motivation of this paper is that local object geometry can provide useful clues about where the object can be grasped. Generated by the sampled seed points from raw point cloud, the votes allow seed points in different object regions to contribute to locations where the object c...
-
Robotic systems using anthropomorphic end-effectors face tremendous challenges choosing a suitable pose for grasping an object. The fact that the choice of a grasp is influenced by the physical properties of an object, the intended task, and the environment results in a considerable amount of variables. The majority of models targeted towards enabling such robots to determine a suitable grasping p...
-
In this work, we developed a mathematical model and a simulation platform for a fish-inspired robotic template, namely Magnetic, Modular, Undulatory Robot $(\mu \text{Bot})$. Through this platform, we systematically explored the effects of robot design and fluid parameters on swimming performance via reinforcement learning. The mathematical model was composed of two interacting subsystems, the rob...
-
This paper presents novel quadrotor structures composed of only clockwise rotors. A multirotor unmanned aerial vehicle (UAV) generally has both clockwise and counterclockwise rotors to counteract the torques from the rotors. While the proposed structures have only clockwise rotors, those rotors are tilted to cancel the torques around the yaw angle of the body. This paper investigates the condition...
-
Frequently, ground robots are hampered by debris and objects on the ground, and safely surpassing them is not always trivial. On the contrary, a robot capable of flying is intrinsically immune to such obstacles and, therefore, greatly enhances the possibility of inspecting and intervening in adverse surroundings for humans. This work introduces a novel teleoperated aerial platform for inspection a...
-
In this article, we consider the problem of delivering an object to a human coworker by means of an aerial robot (AR). To this aim, we present an ergonomics-aware Nonlinear Model Predictive Control (NMPC) designed to autonomously perform the handover. The method is general enough to be applied to any multi-rotor aerial vehicle (MRAV) with a minimal adaptation of the robot model. The formulation of...
-
Force exertion, object manipulation, and interaction are novel trending research topics of autonomous flying robots that can yield hoovering. Moreover, specifically with quadrotors, the vibration caused by the high natural frequency of rotating propellers exacerbates the problem of maintaining contact and exerting force against a rigidly fixed object. This contact vibration transfers back kinetic ...
-
In robot learning from demonstration (LfD), a visual representation of a cost function inferred from Inverse Reinforcement Learning (IRL) provides an intuitive tool for humans to quickly interpret the underlying objectives of the demonstration. The inferred cost function can be used by controllers, for example, Model Predictive Controllers (MPCs). In this work, we improve the recently developed IR...
-
Turn-taking is a fundamental behavior during human interactions and robots must be capable of turn-taking to interact with humans. Current state-of-the-art approaches in turn-taking focus on developing general models to predict the end of turn (EoT) across all contexts. This demands an all-inclusive verbal and non-verbal behavioral dataset from all possible contexts of interaction. Before robot de...
-
This paper presents a framework for transferring surface finishing skills to new surface geometries while preserving the surface finish quality. The main idea is to estimate the contact area between the workpiece and the tool by using 3D point cloud approach and replicate a given material removal rate and the accumulated material removal, as these quantities are the main parameters for quality. Th...
-
Learning temporal relations between actions in a bimanual manipulation task is important for capturing the constraints of actions required to achieve the task's goal. However, given several demonstrations of a bimanual manipulation task, the problem of identifying the true temporal dependencies between actions - if there are any - is very challenging due to contradictions. We propose a model-drive...
-
Motion optimization is an effective framework for generating smooth and safe trajectories for robotic manipulation tasks. However, it suffers from local optima that hinder its applicability, especially for multi-objective tasks. In this paper, we study this problem in light of the integration of Energy-Based Models (EBM) as guiding priors in motion optimization. EBMs are probabilistic models with ...
-
To perform tasks in unstructured environments, robots need to be able to apply learned skills to different contexts and to autonomously make decisions online. We, therefore, developed a novel data-driven task learning approach that segments a task demonstration into simpler skills and structures them in a high-level task graph. In contrast to other state-of-the-art methods, the presented approach ...
-
This paper considers the problem of enabling the user to modify the path of a telepresence robot. The robot is capable of autonomously navigating to a goal predefined by the user, but the user might still want to modify the path, for example, to go further away from other people, or to go closer to landmarks she wants to see on the way. We propose Human-Influenced Dynamic Window Approach (HI-DWA),...
-
Drones can provide a minimally-constrained adapting camera view to support robot telemanipulation. Furthermore, the drone view can be automated to reduce the burden on the operator during teleoperation. However, existing approaches do not focus on two important aspects of using a drone as an automated view provider. The first is how the drone should select from a range of quality viewpoints within...
-
Teleoperated robots are believed to play an important role for future applications in industry, medicine and other domains. Examples for this are remote assembly and maintenance, surgery, diagnosis or deep-sea and space exploration. Such applications are made possible by state-of-the-art tactile manipulators, well-researched control schemes and novel communication technologies such as the fifth ge...
-
This brief proposes a new stabilizing communication law to allow the wave transformation-based teleoperation architecture to accommodate direct environmental contact force feedback, potentially increasing the human operator's experience of telepresence. Simulation results are provided.
-
There has been a drastic shift to remote interaction for professional, industrial and personal interactions. Improving the overall quality of these interactions by removing any sense of distance between the users is the ultimate goal. Video conferencing has been widely adopted as an improvement to audio-only interactions. Having added visuals to audio communication, the next frontier is to add phy...
-
Virtual Fixtures facilitate teleoperation, for in-stance by guiding the human operator. Developing these Virtual Fixtures in tasks with tight tolerances remains challenging. Fixtures with a high stiffness allow for more precise guidance, whereas a lower stiffness is required to allow for corrections. We observed that many assembly operations can be split into different phases - approaching, positi...
-
The accomplishment of a successful teleoperation task requires guaranteeing system stability and transparency. Communication delay (in particular variable time delay), quantization and discretization negatively affect system stability and might be overcome with Time Domain Passivity Approach (TDPA), a model-free and robust way to cope with energy injection due to communication delay. However, this...
-
A long-standing question in robot hand design is how accurate tactile sensing must be. This paper uses simulated tactile signals and the reinforcement learning (RL) framework to study the sensing needs in grasping systems. Our first experiment investigates the need for rich tactile sensing in the rewards of RL-based grasp refinement algorithms for multi-fingered robotic hands. We systematically in...
-
Robotic manipulation of cloth has applications ranging from fabrics manufacturing to handling blankets and laundry. Cloth manipulation is challenging for robots largely due to their high degrees of freedom, complex dynamics, and severe self-occlusions when in folded or crumpled configurations. Prior work on robotic manipulation of cloth relies primarily on vision sensors alone, which may pose chal...
-
Grasping of objects using multi-fingered robotic hands often fails due to small uncertainties in the hand motion control and the object's pose estimation. To tackle this problem, we propose a grasping adjustment strategy based on tactile seroving. Our technique employs feedback from a sensorized multi-fingered robotic hand to collaboratively servo the fingers and palm to achieve the desired grasp....
-
The automated cleaning of surfaces such as furniture, bathroom sinks, and even human bodies is challenging due to the three-dimensional nature of their geometries. Yet, enabling robots to effectively and safely perform these tasks would not only reduce user efforts spent on household cleaning chores, but would also alleviate the strenuous workload of caretakers as the elderly population continues ...
-
Cleaning public areas like commercial complexes is challenging due to their sophisticated surroundings and the vast kinds of real-life dirt. Robots are required to distinguish dirts and apply corresponding cleaning strategies. In this work, we proposed an active-cleaning framework by utilizing deep-learning methods for both solid wastes detection and liquid stains segmentation. Our system consists...
-
Robot simulation has been an essential tool for data-driven manipulation tasks. However, most existing simulation frameworks lack either efficient and accurate models of physical interactions with tactile sensors or realistic tactile simulation. This makes the sim-to-real transfer for tactile-based manipulation tasks still challenging. In this work, we integrate simulation of robot dynamics and vi...
-
Perceiving the environment through touch is important for robots to reach in cluttered environments, but devising a way to sense without disturbing objects is challenging. This work presents the design and modelling of whisker-inspired sensors that attach to the surface of a robot manipulator to sense its surrounding through light contacts. We obtain a sensor model using a calibration process that...
-
In minimally invasive surgery (MIS), the reliable detection of hard inclusions in soft tissue is crucial for the success of the intervention. In robot-assisted surgery (RAS) however, limited technologies are available for intracorporeal tissue stiffness assessment due to the lack of force and tactile feedback from the robot tool tip. This paper investigates both, human haptic perception and roboti...
-
Visual place recognition (VPR) - a fundamental task in computer vision and robotics - is the problem of identifying a place mainly based on visual information. View-point and appearance changes, such as due to weather and seasonal variations, make this task challenging. Currently, there is no universal VPR technique that can work in all types of environments, on a variety of robotic platforms, and...
-
Last-mile delivery robots are usually required to navigate on the sidewalk through a fixed route. The current solutions heavily rely on the image-based perception and GPS localization to successfully complete delivery tasks. However, it is prone to fail and become unreliable when the robot runs in challenging conditions, such as operating in different illuminations, or under canopies of trees or b...
-
Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users' high-level navigation plans. To find such a path, vehicles negotiate with different terrains and assess their traversal difficulty. Most prior works model surroundings either through geometric representations or semantic classifications, which do not reflect perceived motion intensi...
-
We propose a hierarchical planning algorithm that efficiently computes an optimal plan for finding a target object in large environments where a robot must simultaneously consider both navigation and manipulation. One key challenge that arises from large domains is the substantial increase in search space complexity that stems from considering mobile manipulation actions and the increase in number...
-
Wheeled-legged robots have the potential to navigate in cluttered and irregular scenarios by altering the locomotion modes to adapt to the terrain challenges and effectively reach targeted locations in unstructured spaces. To achieve this functionality, a hybrid locomotion planner is necessary. In this work we present a search-based planner, which explores a set of motion primitives and a 2.5D tra...
-
When adapting Simultaneous Mapping and Localization (SLAM) to real-world applications, such as autonomous vehicles, drones, and augmented reality devices, its memory footprint and computing cost are the two main factors limiting the performance and the range of applications. In sparse feature based SLAM algorithms, one efficient way for this problem is to limit the map point size by selecting the ...
-
In this paper, we propose the online extrinsic correction method that effectively optimizes the extrinsic parameters of multi-camera systems used in visual SLAM. In the typical visual SLAM systems that use multi-camera settings, the intrinsic and extrinsic parameters of the cameras are calculated through offline calibration, which is used as the fixed constraints in online execution. However, the ...
-
This paper considers active SLAM problem for 3D deformable environments where the trajectory of the robot is planned to optimize the SLAM results. A planning strategy combining an efficient global planner with an accurate local planner is proposed to solve the problem. Simulation results under different scenarios have shown that the proposed active SLAM algorithm provides a good balance between ac...
-
Surgical activity recognition and prediction can help provide important context in many Robot-Assisted Surgery (RAS) applications, for example, surgical progress monitoring and estimation, surgical skill evaluation, and shared control strategies during teleoperation. Transformer models were first developed for Natural Language Processing (NLP) to model word sequences and soon the method gained pop...
-
Robot-Assisted Minimally Invasive Surgery (RAMIS), which introduced robot-actuated invasive tools to increase the dexterity and efficiency of traditional MIS, has become popular. Investigations on how to achieve autonomy in RAMIS have drawn vast intention recently, which urges further insights into the process of the surgical procedures. In this paper, the definition of critical actions, which dis...
-
Minimally invasive surgery requires real-time tool tracking to guide the surgeon where depth perception and visual occlusion present navigational challenges. Although vision-based and external sensor-based tracking methods exist, fibre-optic sensing can overcome their limitations as they can be integrated directly into the device, are biocompatible, small, robust and geometrically versatile. In th...
-
Robotic comanipulation provides a cost-effective solution to telesurgery when remote operation is not strictly necessary. Within the field of laparoscopic surgery, the comanip-ulation scenario is only recently being exploited commercially in the form of lightweight backdrivable systems. A passive wrist backdrivable robot does not require preoperative alignment with the incision that acts as a fulc...
-
Estimating precise metric depth and scene reconstruction from monocular endoscopy is a fundamental task for surgical navigation in robotic surgery. However, traditional stereo matching adopts binocular images to perceive the depth information, which is difficult to transfer to the soft robotics-based surgical systems due to the use of monocular endoscopy. In this paper, we present a novel framewor...
-
A central aspect of robotic motion planning is collision avoidance, where a multitude of different approaches are currently in use. Optimization-based motion planning is one method, that often heavily relies on distance computations between robots and obstacles. These computations can easily become a bottleneck, as they do not scale well with the complexity of the robots or the environment. To imp...
-
This paper addresses local path re-planning for n-dimensional systems by introducing an informed sampling scheme and cost function to achieve collision avoidance with minimum deviation from an (optimal) nominal path. The proposed informed subset consists of the union of ellipsoids along the specified nominal path, such that the subset efficiently encapsulates all points along the nominal path. The...
-
We present a biologically inspired approach for path planning with dynamic obstacle avoidance. Path plan-ning is performed in a condensed configuration space of a robot generated by self-organizing neural networks (SONN). The robot itself and static as well as dynamic obstacles are mapped from the Cartesian task to the configuration space by precomputed kinematics. The condensed space represents a...
-
With the goal of efficiently computing collisionfree robot motion trajectories in dynamically changing environments, we present results of a novel method for Heuristics Informed Robot Online Path Planning (HIRO). Dividing robot environments into static and dynamic elements, we use the static part for initializing a deterministic roadmap, which provides a lower bound of the final path cost as infor...
-
Real-time control is an essential aspect of safe robot operation in the real world with dynamic objects. We present a framework for the analysis of object-aware con-trollers, methods for altering a robot's motion to anticipate and avoid possible collisions. This framework is focused on three design considerations: kinematics, motion profiles, and virtual constraints. Additionally, the analysis in ...
-
We propose a collaborative teleoperation algorithm which utilizes haptic force feedback to guide users around oncoming obstacles while accounting for non-holonomic constraints. The proposed algorithm predicts the user's goal, plans a path using a modified RRT* algorithm to the predicted goal, and provides haptic guidance to the path and away from obstacles when the user is in an unsafe pose. We sh...
-
Measuring the fidelity of synthesized virtual traffic has become an important and fundamental concern for evaluating the performance of different traffic simulation techniques and applications of autonomous vehicle testing. In this work, we propose a novel method to evaluate the fidelity of any trajectory data from the perspective of anomalous trajectory detection. First, given the trajectory data...
-
Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling. To alleviate this, we introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We propose a model that learns a mapping between RGB images and corresponding LiDAR features such as raydrop or pe...
-
Simulation remains one of the key methods for testing and validation of robotic perception systems and it also becomes increasingly important for training visuomotor policies for autonomous driving or manipulation. Further, as perception pipelines tend to leverage increasing amounts of modalities, it appears vital to simulate additional cues such as depth maps aside from RGB images. To align simul...
-
We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential manipulation planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a sto...
-
For autonomous agents to act as trustworthy partners to human users, they must be able to reliably communicate their competency for the tasks they are asked to perform. Towards this objective, we develop probabilistic world models based on deep generative modelling that allow for the simulation of agent trajectories and accurate calculation of tasking outcome probabilities. By combining the streng...
-
Recognizing human partners is an essential social skill for building personalized and long-term human-robot interactions. However, robots deployed in complex, real-world environments have to face several challenges, such as managing unstructured interactions with multiple users, limited computational resources, and intrinsic and continuous variability of their sensory evidence. To cope with these ...
-
Social relations within a group are one of the factors that build the social context. Less attention has been paid to social relations within a group for human-robot interaction design. In this study, we designed different types of service behaviors for social relations in a mixed age group and conducted an experiment to investigate the effect of service behavior types (serving the elderly first v...
-
The purpose of this research is to contribute to social communication between humans and robots in scenes that have been considered difficult due to the limited facial expression capabilities of robots. In order to provide more detailed facial expressions, we designed a novel wire-driven 3D eyebrow using a soft material with a bending structure. We then demonstrated the mechanical properties of th...
-
This paper evaluates a robot that distributed hand-sanitizer over an eight month period (October 2020-June 2021) in public places on the Oregon State University campus. During COVID times, many robots have been deployed in public places as social distancing enforcers, food delivery robots, UV-sanitation robots and more, but few studies have assessed the social situations of these robots. Using the...
-
Body language such as conversational gesture is a powerful way to ease communication. Conversational gestures do not only make a speech more lively but also contain semantic meaning that helps to stress important information in the discussion. In the field of robotics, giving conversational agents (humanoid robots or virtual avatars) the ability to properly use gestures is critical, yet remain a t...
-
This paper demonstrates the practical performance of a new theory of balance control that has been shown in simulation to out-perform earlier balance control theories in the sense of allowing the robot to make larger and faster movements while still maintaining its balance. The case studied here is that of a general planar double inverted pendulum, which resembles a legged robot's behaviour when t...
-
When robots walk on uneven terrain, trajectory planning should take into account both the whole-body dy-namics and the ground geometry simultaneously. In uneven terrain environments, there are only a limited number of places where the robot is able to make stable contact with the ground without its feet wobbling or slipping because of the intricate round geometry. In such environments, the optiona...
-
In order to properly integrate humanoid robots in real-life situations, they must be able to collaborate with humans in completing tasks. One of these tasks is the cooperative transportation of a heavy object, which has been widely studied in the humanoids literature. However, the proposed methods rely heavily on six-axis force/torque (F/T) sensors at the wrists, which medium-sized or even some fu...
-
Robotic systems capable of Dynamic Mobile Manipulation (DMM) tasks combine dynamic manipulation and locomotion and could facilitate dangerous or physically demanding labor. For instance, firefighter humanoid robots could leverage their body by leaning against collapsed building rubble to push it aside. Here we introduce a teleoperation system that targets the realization of these tasks using human...
-
Standing balance for legged robots can be achieved through regulating the center of pressure (ankle strategy), the angular momentum about the center of mass (hip strategy), and the magnitude of ground reaction force (variable height strategy). Prevalent reduced order models used to model legged robots at most only capture two of these strategies, and the contribution of the three available strateg...
-
Although many walking control frameworks have been developed to enable biped robots to walk stably on uneven terrain, the foot sole of the robot is also important. Inspired by that study, we have developed a Variable Stiffness Sole (VSS), which is able to adapt to the shape of the obstacles on the ground in Compliant Mode and provide robust support in Stiff Mode. Furthermore, we proposed a walking...
-
This paper presents a novel method for smaller-sized humanoid robots to self-calibrate their foot force sensors. The method consists of two steps: 1. The robot is commanded to move along planned whole-body trajectories in different double support configurations. 2. The sensor parameters are determined by minimizing the error between the measured and modeled center of pressure (CoP) and ground reac...
-
Depth perception is pivotal in many fields, such as robotics and autonomous driving, to name a few. Consequently, depth sensors such as LiDARs rapidly spread in many applications. The 3D point clouds generated by these sensors must often be coupled with an RGB camera to understand the framed scene semantically. Usually, the former is projected over the camera image plane, leading to a sparse depth...
-
The depth completion task aims to predict a dense depth map from a sparse LiDAR point cloud and an RGB image. This task is critical because an accurate depth map can be used as prior information to solve many computer vision tasks, such as downstream tasks in autonomous vehicles and robot vision. Previous deep learning methods which focus on the local affinity have achieved impressive results. How...
-
Single image 3D face reconstruction with accurate geometric details is a critical and challenging task due to the similar appearance on the face surface and fine details in organs. In this work, we introduce a self-supervised 3D face reconstruction approach from a single image that can recover detailed textures under different camera settings. The proposed network learns high-quality disparity map...
-
Monocular depth estimation (MDE) has attracted intense study due to its low cost and critical functions for robotic tasks such as localization, mapping and obstacle detection. Supervised approaches have led to great success with the advance of deep learning, but they rely on large quantities of ground-truth depth annotations that are expensive to acquire. Unsupervised domain adaptation (UDA) trans...
-
Being able to reproduce physical phenomena ranging from light interaction to contact mechanics, simulators are becoming increasingly useful in more and more application domains where real-world interaction or labeled data are difficult to obtain. Despite recent progress, significant human effort is needed to configure simulators to accurately reproduce real-world behavior. We introduce a pipeline ...
-
This paper introduces a novel deep framework for dense 3D reconstruction from multiple image frames, leveraging a sparse set of depth measurements gathered jointly with image acquisition. Given a deep multi-view stereo network, our framework uses sparse depth hints to guide the neural network by modulating the plane-sweep cost volume built during the forward step, enabling us to infer constantly m...
-
Partial observation of indoor scenes (single-viewed RGB-D) carries insufficient spatial information for complex tasks such as autonomous navigation and virtual reality, thus many learning-based methods are proposed to realize semantic scene completion (SSC) from single-viewed input. However, most of them only extract scene-level features of input to generate output, which might lose details. In th...
-
We seek to rigorously evaluate the benefit of using a few beams rather than a single beam for a low-cost obstacle avoidance sonar for small AUVs. For a small low-cost AUV, the complexity, cost, and volume required for a multi-beam forward looking sonar are prohibitive. In contrast, a single-beam system is relatively easy to integrate into a small AUV, but does not provide the performance of a mult...
-
Many operations performed by work class Remotely Operated Vehicles (ROVs) require the manipulation of heavy loads. An example is the manipulation and grouting of armour stones. A way to increase the working capabilities of the ROV is to introduce cables among the set of actuators. The cable lengths and tensions are controlled by winches placed on the vehicle. Being similar to a cable-driven parall...
-
Radar odometry may provide valuable input for surface vessels in several marine applications. The vulnerability of global positioning satellite systems to jamming and spoofing motivates the search for alternatives. In this work, we investigate the feasibility of W-band frequency modulated continuous wave radars in marine settings for odometry. A method to model radar resolution is presented and is...
-
Considering realistic characteristics of acoustic localization methods is crucial for roboticists when developing guidance and control algorithms for small and agile underwater robots. Current simulators either rely purely on geometric distancing, i.e. do not consider dynamic effects such as robot motion during acoustic signal propagation, or they are too complex for usage by non-communication exp...
-
Sonar sensors play an integral part in underwater robotic perception by providing imagery at long distances where standard optical cameras cannot. They have proven to be an important part in various robotic algorithms including localization, mapping, and structure from motion. Unfortunately, generating realistic sonar imagery for algorithm development is difficult due to the high cost of field tri...
-
An essential task for a multi-robot system is generating a common understanding of the environment and relative poses between robots. Cooperative tasks can be executed only when a vehicle has knowledge of its own state and the states of the team members. However, this has primarily been achieved with direct rendezvous between underwater robots, via inter-robot ranging. We propose a novel distribut...
-
This paper presents a novel algorithm for monitoring marine environments utilizing a resource-constrained robot. Collecting water quality data from large bodies of water is paramount for monitoring the ecosystem's health, particularly for predicting harmful cyanobacteria blooms. The large spatial dimensions of such bodies of water and the slow varying of water quality parameters make exhaustive, c...
-
An increase in health awareness has fueled the development of fitness equipment or devices nowadays. Most conventional fitness devices have had some issues in space limitation and the high cost of equipment. With the advance in wearable robotics, we proposed a soft passive fitness wearable device for upper limb resistance exercises such as chest press, frontal raise, and chest fly. Users can custo...
-
Soft robots have a wide rang of applications due to their compliance, flexibility and low fabrication cost. Compare to rigid robots, soft robots are more safe for human. In this work, we design various multi-DoF actuators with spring reinforce and particle jamming, and two fabrication methods are proposed to make them. Each type of actuator is tested to evaluate the mechanical properties by experi...
-
Soft robotic grippers are becoming increasingly popular for agricultural and logistics automation. Their passive conformability enables them to adapt to varying product shapes and sizes, providing stable large-area grasps. This work presents a novel methodology for combining soft robotic grippers with electrical impedance tomography-based sensors to infer intrinsic properties of grasped fruits. We...
-
Fibre jamming is a new and understudied soft robotic mechanism that has previously found success in stiffness-tunable arms and fingers. However, to date researchers have not fully taken advantage of the freedom offered by contemporary fabrication techniques including multi-material 3D printing in the creation of fibre jamming structures. In this research, we present a novel, modular, multi-materia...
-
Although the grasping state analysis is vital in the study of manipulators, the grasping state analysis of soft manipulators as an independent research topic is not much so far. This paper proposes a novel pneumatic soft manipulator with a flexible tactile sensor array (SM-FTSA). The flexible tactile sensor array comprises piezoresistive materials with a porous structure. An equal potential approa...
-
Collaboratively estimating the state of two robots under communication constraints is challenging regarding computational complexity and statistical optimality. Previous work only achieves practical solutions by either disregarding parts of the measurements or imposing a communication overhead, being non-optimal or not entirely distributed, respectively. In this work, we present a centralized-equi...
-
Place recognition is a fundamental component of robotics, and has seen tremendous improvements through the use of deep learning models in recent years. Networks can experience significant drops in performance when deployed in unseen or highly dynamic environments, and require additional training on the collected data. However naively fine-tuning on new training distributions can cause severe degra...
-
Regular maintenance of pipelines is an important task to ensure oil transportation and other operation (sewers, nature gas). Precise localization of pipeline damage can greatly improve the efficiency of maintenance work. Since the texture similarity and illumination change of pipe, traditional local descriptors for image matching like SIFT, SURF and ORB are easy to suffer from false correspondence...
-
We present DOME, a novel method for one-shot imitation learning, where a task can be learned from just a single demonstration and then be deployed immediately, without any further data collection or training. DOME does not require prior task or object knowledge, and can perform the task in novel object configurations and with distractors. At its core, DOME uses an image-conditioned object segmenta...
-
Imitation learning has attracted attention as a method for realizing complex robot control without programmed robot behavior. Meta-imitation learning has been proposed to solve the high cost of data collection and low generalizability to new tasks that imitation learning suffers from. Meta-imitation can learn new tasks involving unknown objects from a small amount of data by learning multiple task...
-
When cast into the Deep Reinforcement Learning framework, many robotics tasks require solving a long horizon and sparse reward problem, where learning algorithms struggle. In such context, Imitation Learning (IL) can be a powerful approach to bootstrap the learning process. However, most IL methods require several expert demonstrations which can be prohibitively difficult to acquire. Only a handfu...
-
Inspired by the fact that humans use diverse sensory organs to perceive the world, sensors with different modalities are deployed in end-to-end driving to obtain the global context of the 3D scene. In previous works, camera and LiDAR inputs are fused through transformers for better driving performance. These inputs are normally further interpreted as high-level map information to assist navigation...
-
Output Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient Sensorimotor Policy Learning
Imitation learning (IL) can generate computationally efficient sensorimotor policies from demonstrations provided by computationally expensive model-based sensing and control algorithms. However, commonly employed IL methods are often data-inefficient, requiring the collection of a large number of demonstrations and producing policies with limited robustness to uncertainties. In this work, we comb...
-
We demonstrate the first large-scale application of model-based generative adversarial imitation learning (MGAIL) to the task of dense urban self-driving. We augment standard MGAIL using a hierarchical model to enable generalization to arbitrary goal routes, and measure performance using a closed-loop evaluation framework with simulated interactive agents. We train policies from expert trajectorie...
-
Learning from previously collected datasets of expert data offers the promise of acquiring robotic policies without unsafe and costly online explorations. However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time. While prior works mainly studied the distribution shift caused by the policy during the ...
-
This paper presents a novel probabilistic approach to deep robot learning from demonstrations (LfD). Deep move-ment primitives (DMPs) are deterministic LfD model that maps visual information directly into a robot trajectory. This paper extends DMPs and presents a deep probabilistic model that maps the visual information into a distribution of effective robot trajectories. The architecture that lea...
-
This paper studies the problem of constructing polytopic representations for planar regions from depth camera readings. This problem is of great importance for terrain mapping in complicated environment and has great potentials in legged locomotion applications. To address the polytopic planar region characterization problem, we propose a two-stage solution scheme. At the first stage, the planar r...
-
This paper introduces an Online Localisation and Colored Mesh Reconstruction (OLCMR) ROS perception architecture for ground exploration robots aiming to perform robust Simultaneous Localisation And Mapping (SLAM) in challenging unknown environments and provide an associated colored 3D mesh representation in real time. It is intended to be used by a remote human operator to easily visualise the map...
-
For wave-driven unmanned surface vehicles (WUSVs), utilizing oscillating foils is the most straightforward and common wave energy conversion mechanism. Improving the thrust of the oscillating foil to increase its speed can help WUSVs improve their maneuverability and shorten the completion of ocean missions. This paper proposes a novel transformable two-segment foil, improving the wave energy-conv...
-
Autonomous navigation in dense vegetation remains an open challenge and is an area of major interest for the research community. In this paper we propose a novel traversability estimation method, the Forest Traversability Map, that gives autonomous ground vehicles the ability to navigate in harsh forests or densely vegetated environments. The method estimates travers ability in unstructured enviro...
-
Retrieving the missing dimension information in acoustic images from 2D forward-looking sonar is a well-known problem in the field of underwater robotics. There are works attempting to retrieve 3D information from a single image which allows the robot to generate 3D maps with fly-through motion. However, owing to the unique image formulation principle, estimating 3D information from a single image...
-
We present a method for autonomous exploration of large-scale unknown environments under mission time con-straints. We start by proposing the Frontloaded Information Gain Orienteering Problem (FIG-OP) - a generalization of the traditional orienteering problem where the assumption of a reliable environmental model no longer holds. The FIG-OP ad-dresses model uncertainty by frontloading expected inf...
-
In this paper, we present an online adaptive planning strategy for a team of robots with heterogeneous sensors to sample from a latent spatial field using a learned model for decision making. Current robotic sampling methods seek to gather information about an observable spatial field. However, many applications, such as environmental monitoring and precision agriculture, involve phenomena that ar...
-
The ability to make good decisions in partially observed non-cooperative multi-agent scenarios is important for robots to interact effectively in human environments. A robust framework for such decision-making problems is the Interactive Partially Observable Markov Decision Processes (I-POMDPs), which explicitly models the other agents' beliefs up to a finite reasoning level in order to more accur...
-
We present an incremental search algorithm, called Lifelong-GLS, which combines the vertex efficiency of Lifelong Planning A* (LPA*) and the edge efficiency of Generalized Lazy Search (GLS) for efficient replanning on dynamic graphs where edge evaluation is expensive. We use a lazily evaluated LPA* to repair the cost-to-come inconsistencies of the relevant region of the current search tree based o...
-
In an environment where a manipulator needs to execute multiple consecutive tasks, the act of object manoeuvre will change the underlying configuration space, affecting all subsequent tasks. Previously free configurations might now be occupied by the manoeuvred objects, and previously occupied space might now open up new paths. We propose Lazy Tree-based Replanner (LTR *)-a novel hybrid planner th...
-
Autonomous navigation among people is a com-plex problem that also exhibits considerable variation depending on the type of environment and people involved. Here we consider navigation among crowds that exhibit flow-like behavior like people moving through a train station. We propose a novel pseudo-fluid model of crowd flow for such problems. These have an intuitive physical interpretation and do ...
-
Autonomous Driving requires high levels of coordination and collaboration between agents. Achieving effective coordination in multi-agent systems is a difficult task that remains largely unresolved. Multi-Agent Reinforcement Learning has arisen as a powerful method to accomplish this task because it considers the interaction between agents and also allows for decentralized training—which makes it ...
-
In this paper, we present a method for updating robotic state belief through contact with uncertain surfaces and apply this update to a Kalman filter for more accurate state estimation. Examining how guard surface uncertainty affects the time spent in each mode, we derive a novel guard saltation matrix- which maps perturbations prior to hybrid events to perturbations after - accounting for additio...
-
Contact-implicit trajectory optimization offers an appealing method of automatically generating complex and contact-rich behaviors for robot manipulation and locomotion. The scalability of such techniques has been limited, however, by the challenge of ensuring both numerical reliability and physical realism. In this paper, we present preliminary results suggesting that the Iterative Linear Quadrat...
-
This paper proposes a novel approach to controlling wheel-legged quadrupedal robots using pose optimization and force-based control via quadratic programming (QP). Our method allows the robot to leverage the whole-body motion and the wheel actuation to roll over high obstacles while keeping wheel traction with the terrain. In detail, we first present linear rigid body dynamics with wheels that can...
-
This paper considers the problem of designing legged robots for traversing uneven terrain, wherein terrain characteristics represent uncertainty for the design process. When this process encompasses a wider variety of terrains, the likelihood of the designed robot falling in the real world should decrease. However, computational scalability limits the number of terrains that can be taken into acco...
-
This paper reports on a new error-state Model Predictive Control (MPC) approach to connected matrix Lie groups for robot control. The linearized tracking error dynamics and the linearized equations of motion are derived in the Lie algebra. Moreover, given an initial condition, the linearized tracking error dynamics and equations of motion are globally valid and evolve independently of the system t...
-
This study presents a whole-body model predictive control (MPC) of robotic systems with rigid contacts, under a given contact sequence using online switching time optimization (STO). We treat robot dynamics with rigid contacts as a switched system and formulate an optimal control problem of switched systems to implement the MPC. We utilize an efficient solution algorithm for the MPC problem that o...
-
We present a homotopic approach to generating energetically optimal gaits for legged robots that maps passive (i.e., unactuated) gaits of an energetically conservative model of the robot to a model with user-defined target dynamics with dissipation and actuation (i.e., the more “realistic” legged model). Our core contribution is advancing the state-of-the-art towards a turn-key approach where the ...
-
The Robotic locomotion community is interested in optimal gaits for control. Based on the optimization criterion, however, there could be a number of possible optimal gaits. For example, the optimal gait for maximizing displacement with respect to cost is quite different from the maximum displacement optimal gait. Beyond these two general optimal gaits, we believe that the optimal gait should deal...
-
We propose a novel and efficient lifting approach for the optimal control of rigid-body systems with contacts to improve the convergence properties of Newton-type methods. To relax the high nonlinearity, we consider the state, acceleration, contact forces, and control input torques, as optimization variables and the inverse dynamics and acceleration constraints on the contact frames as equality co...
-
In both industrial and service domains, a central benefit of the use of robots is their ability to quickly and reliably execute repetitive tasks. However, even relatively simple peg-in-hole tasks are typically subject to stochastic variations, requiring search motions to find relevant features such as holes. While search improves robustness, it comes at the cost of increased runtime: More exhausti...
-
Delicate industrial insertion tasks (e.g., PC board assembly) remain challenging for industrial robots. The chal-lenges include low error tolerance, delicacy of the components, and large task variations with respect to the components to be inserted. To deliver a feasible robotic solution for these insertion tasks, we also need to account for hardware limits of existing robotic systems and minimize...
-
The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet challenging problems in industry, where an agent must pack variable size items delivered in sequence into a finite bin with the aim to maximize the space utilization. It represents a strongly NP-Hard optimization problem such that no solution has been offered to date with high performance in space utilization. In this paper, we pr...
-
The current transformation towards smart manufacturing has led to a growing demand for human-robot collaboration (HRC) in the manufacturing process. Perceiving and understanding the human co-worker's behaviour introduces challenges for collaborative robots to efficiently and effectively perform tasks in unstructured and dynamic environments. Integrating recent data-driven machine vision capabiliti...
-
To overcome the limitations of the so-called Industry 4.0 focusing on mass production and full automation, a novel paradigm was recently introduced, namely Industry 5.0, which aims at an increased collaboration between humans and machines, and particularly robots, instead of replacing the former with the latter. This challenge requires novel interactive intelligent robots able to perform complex t...
-
This paper presents a combined task and motion planner for a robot arm to carry out 3D metal wire curving tasks by collaborating with a bending machine. We assume a collaborative robot that is safe to work in a human environment but has a weak payload to bend objects with large stiffness, and developed a combined planner for the robot to use a bending machine. Our method converts a 3D curve to a b...
-
Robotic automation is steadily growing in different industries around the world. However, in some industries, such as garment manufacturing, most tasks are still predominantly manual, due to the flexible nature of clothes. Garment and clothes are easily deformed when some force is applied, so it is difficult for robots to handle them while predicting their deformation. Our general research goal is...
-
Augmented reality (AR) defines a new information-delivery paradigm by overlaying computer-generated information on the perception of the real world. AR-integrated robot has become an appealing concept in terms of enhanced human-robot interaction. Despite intensive research on AR, existing indoor location-based AR systems are vulnerable to attacks and can hardly meet the security and privacy requir...
-
In this paper, a novel prototype for hanging produce harvesting is presented, that is productive, versatile, and robust. In our methodology, the robot-mounted tube approaches, and eventually surrounds the produce of interest at the entry side, that can be as small as the produce diameter, plus a small margin. The stem is then cut by a laser beam, with the optics set up for a distant focal point. S...
-
In this paper we consider the information path-planning problem for a single robot in a stochastic environment with static obstacles subject to a preassigned constraint on the distance it can travel. Given a set of candidate sampling locations, the objective is to determine a path for the robot that allows to visit as many sampling locations as possible to accurately reconstruct an unknown underly...
-
Lattice structures allow robotic systems to operate in complex and hazardous environments, e.g. construction, mining and nuclear plants, reliably and effectively. However, current navigation systems for these structures are neither realistic, as they assume simplistic motion primitives and obstacle-free workspaces, nor efficient as they rely solely on global discrete search in an attempt to levera...
-
This research presents a novel Collective Robotic Construction (CRC) system named RECCraft. The RECCraft hardware system is composed of the mobile manipulation vehicles, the cubic blocks, and the folding ramp blocks. Solid connection and easy removal of the blocks are achieved by an electropermanent magnet and silicon steel sheets. With one degree of freedom (DOF) lifting manipulator, the robot ca...
-
Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. Probabilistic embeddings for actor-critic $\boldsymbol{RL}$ (PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not ex...
-
Tactile signals provide rich information about objects via touch and are essential for a robot to perform dex-terous manipulation. Exploring actively via tactile perception collects important information about the workspace. However, designing an effective tactile exploration policy is challenging in unstructured environments. Typically, the geometric information is incomplete, and need to be comp...
-
Multi-agent reinforcement learning (MARL) meth-ods face a curse of dimensionality in the policy and value function representations as the number of agents increases. The development of distributed or parallel training techniques is also hindered by the global coupling among the agent dynamics, requiring simultaneous state transitions. This paper introduces Distributed multi-Agent Reinforcement Lea...
-
Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes
The major challenges of collision avoidance for robot navigation in crowded scenes lie in accurate environment modeling, fast perceptions, and trustworthy motion planning policies. This paper presents a novel adaptive environment model based collision avoidance reinforcement learning (i.e., AEMCARL) framework for an unmanned robot to achieve collision-free motions in challenging navigation scenari...
-
Reinforcement learning algorithms require a large amount of samples; this often limits their real-world applications on even simple tasks. Such a challenge is more outstanding in multi-agent tasks, as each step of operation is more costly, requiring communications or shifting or resources. This work aims to improve data efficiency of multi-agent control by model-based learning. We consider network...
-
Learning a risk-aware policy is essential but rather challenging in unstructured robotic tasks. Safe reinforcement learning methods open up new possibilities to tackle this problem. However, the conservative policy updates make it intractable to achieve sufficient exploration and desirable performance in complex, sample-expensive environments. In this paper, we propose a dual-agent safe reinforcem...
-
Multi-objective optimization models that encode ordered sequential constraints provide a solution to model various challenging problems including encoding preferences, modeling a curriculum, and enforcing measures of safety. A recently developed theory of topological Markov decision processes (TMDPs) captures this range of problems for the case of discrete states and actions. In this work, we exte...
-
Traditional model-based reinforcement learning (RL) methods generate forward rollout traces using the learnt dynamics model to reduce interactions with the real environment. The recent model-based RL method considers the way to learn a backward model that specifies the conditional probability of the previous state given the previous action and the current state to additionally generate backward ro...
-
Sample efficiency has been a key issue in reinforcement learning (RL). An efficient agent must be able to leverage its prior experiences to quickly adapt to similar, but new tasks and situations. Meta-RL is one attempt at formalizing and ad-dressing this issue. Inspired by recent progress in meta-RL, we introduce BIMRL, a novel multi-layer architecture along with a novel brain-inspired memory modu...
-
Human and robot partners increasingly need to work together to perform tasks as a team. Robots designed for such collaboration must reason about how their task-completion strategies interplay with the behavior and skills of their human team members as they coordinate on achieving joint goals. Our goal in this work is to develop a computational framework for robot adaptation to human partners in hu...
-
Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. Common practical methods in the area suffer from inaccurate ergonomics models in performing postural optimization. In or...
-
Robots and artificial agents that interact with humans should be able to do so without bias and inequity, but facial perception systems have notoriously been found to work more poorly for certain groups of people than others. In our work, we aim to build a system that can perceive humans in a more transparent and inclusive manner. Specifically, we focus on dynamic expressions on the human face, wh...
-
To collaborate well with robots, we must be able to understand their decision making. Humans naturally infer other agents' beliefs and desires by reasoning about their observable behavior in a way that resembles inverse reinforcement learning (IRL). Thus, robots can convey their beliefs and desires by providing demonstrations that are informative for a human learner's IRL. An informative demonstra...
-
Advanced social cognitive skills enhance the effectiveness of human-robot interactions. Research shows that an important precursor to the development of these abilities in humans is Theory of Mind (ToM) - the ability to attribute mental states to oneself and to others. In this work, we endow robots with ToM abilities and propose a ToM-based approach to proactive robotic assistance by appealing to ...
-
Goal-based navigation in public places is critical for independent mobility and for breaking barriers that exist for blind or visually impaired (BVI) people in a sight-centric society. Through this work we present a proof-of-concept system that autonomously leverages goal-based navigation assistance and perception to identify socially preferred seats and safely guide its user towards them in unkno...
-
In this paper, we present the Sample Efficient Social Navigation from Observation (SESNO) algorithm that efficiently learns socially-compliant navigation policies from observations of human trajectories. SESNO is an inverse reinforcement learning (IRL)-based algorithm that learns from human trajectory observations without knowledge of their actions. We improve the sample-efficiency over previous I...
-
Humans use audio signals in the form of spoken language or verbal reactions effectively when teaching new skills or tasks to other humans. While demonstrations allow humans to teach robots in a natural way, learning from trajectories alone does not leverage other available modalities including audio from human teachers. To effectively utilize audio cues accompanying human demonstrations, first it ...
-
Machine vision is a critical subsystem for enabling robots to be able to perform a variety of tasks in orchard environments. However, orchards are highly visually complex environments, and computer vision algorithms operating in them must be able to contend with variable lighting conditions and background noise. Past work on enabling deep learning algorithms to operate in these environments has ty...
-
The global grape and wine industry has been considerably impacted by diseases such as downy mildew (DM). Agricultural robots have demonstrated great potential to accurately and rapidly map DM infection for precision applications. Although the robots can autonomously acquire high-resolution images in the vineyard, data processing is mostly performed offline because of network infrastructure and onb...
-
In view planning, the position and orientation of the cameras have been a major contributing factor to the quality of the resulting 3D model. In applications such as precision agriculture, a dense and accurate reconstruction must be obtained quickly while the data is still actionable. Instead of using an arbitrarily large number of images taken from every possible position and orientation in order...
-
Cultivation and weeding are two of the primary tasks performed by farmers today. A recent challenge for weeding is the desire to reduce herbicide and pesticide treatments while maintaining crop quality and quantity. In this paper we introduce BonnBot-I a precise weed management platform which can also performs field monitoring. Driven by crop monitoring approaches which can accurately locate and c...
-
Contemporary robots in precision agriculture focus primarily on automated harvesting or remote sensing to monitor crop health. Comparatively less work has been performed with respect to collecting physical leaf samples in the field and retaining them for further analysis. Typically, orchard growers manually collect sample leaves and utilize them for stem water potential measurements to analyze tre...
-
Due to labor shortage and rising labor cost for the apple industry, there is an urgent need for the development of robotic systems to efficiently and autonomously harvest apples. In this paper, we present a system overview and algorithm design of our recently developed robotic apple harvester prototype. Our robotic system is enabled by the close integration of several core modules, including visua...
-
Apple picking is a challenging manipulation task, but it is difficult to test solutions due to the limited window of time that apples are in season. Previous methods have built simulations of apple trees, but simulations rarely capture soft contact and deformation well, both of which are common in fruit picking. In this paper we present and validate a physical proxy that replicates the mechanics o...
-
The evolution of smaller and more powerful GPUs over the last 2 decades has vastly increased the opportunity to apply robust deep learning-based machine vision approaches to real-time use cases in practical environments. One exciting application domain for such technologies is precision agriculture, where the ability to integrate on-board machine vision with data-driven actuation means that farmer...
-
Reflective and textureless surfaces such as windows, mirrors, and walls can be a challenge for scene reconstruction, due to depth discontinuities and holes. We propose an audio-visual method that uses the reflections of sound to aid in depth estimation and material classification for 3D scene reconstruction in robot navigation and AR/VR applications. The mobile phone prototype emits pulsed audio, ...
-
The field of bioacoustics is concerned with monitoring wild animals based on their vocalisations. Passive acoustic recorders are now commonly used to collect data of the soundscapes of our wild places. While the data they collect is extremely useful, the majority of the recorders use a single omnidirectional microphone, and thus cannot independently perform localisation of a calling animal. Locali...
-
Sound source separation is a method to extract a target sound source from a mixture of various sound sources and noises. One of the typical sound source separation methods is beamforming, which can separate sound sources by direction based on the phase difference between channels from the recorded signal of a microphone array, a multi-channel recording system. However, beamforming is a direction-b...
-
Humans integrate multiple sensory modalities (e.g., visual and audio) to build a causal understanding of the physical world. In this work, we propose a novel type of intrinsic motivation for Reinforcement Learning (RL) that encourages the agent to understand the causal effect of its actions through auditory event prediction. First, we allow the agent to collect a small amount of acoustic data and ...
-
This paper describes the practical response- and performance-aware development of online speech enhancement for an augmented reality (AR) headset that helps a user understand conversations made in real noisy echoic environments (e.g., cocktail party). One may use a state-of-the-art blind source separation method called fast multichannel nonnegative matrix factorization (FastMNMF) that works well i...
-
This paper proposes a microphone array with a speaker to recognize the shape of the surface of the target object by using the standing wave between the transmitted and the reflected acoustic signals. Because the profile of the distance spectrum encodes both the distance to the target and the distance to the edges of the target's surface, this paper proposes to fuse distance spectra using a microph...
-
We investigated the use of impact sounds generated during exploratory behaviors in a robotic manipulation setup as cues for predicting object surface material and for recognizing individual objects. We collected and make available the YCB-impact sounds dataset which includes over 3,000 impact sounds for the YCB set of everyday objects lying on a table. Impact sounds were generated in three modes: ...
-
As a type of body language, gestures can largely affect the impressions of human-like robots perceived by users. Recent data-driven approaches to the generation of co-speech gestures have successfully promoted the naturalness of produced gestures. These approaches also possess greater generalizability to work under various contexts than rule-based methods. However, most have no direct control over...
-
For robot and drone auditions, microphone arrays have been used for estimating sound source directions and sound source locations. By using sound source localization techniques, for example, drones can detect people calling for help even if the target person is not visible. Most sound source localization methods are based on estimated sound source directions and triangulation. However, when it com...
-
State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is no...
-
Cooperative adaptive cruise control (CACC) is a smart transportation solution that can mitigate traffic jams and improve road safety. CACC performance is heavily impacted by communication time delay; moreover, control theory solutions generally compromise control performance by tuning control gains in order to maintain plant stability. We propose a control-machine learning hybrid approach called d...
-
Automatic control design for robotic systems is becoming more and more popular. However, this usually involves a significant computational cost, due to the expensive and noisy evaluation of candidate solutions through high-fidelity simulation or even real hardware. This work aims at reducing the computational cost of automatic design of behavioral arbitrators through the introduction of a two-step...
-
Learning from Demonstration (LfD) is a very appealing approach to empower robots with autonomy. Given some demonstrations provided by a human teacher, the robot can learn a policy to solve the task without explicit programming. A promising use case is to endow smart robotic wheelchairs with active assistance to navigation. By using LfD, it is possible to learn to infer short-term destinations anyw...
-
Pouring liquids accurately into containers is one of the most challenging tasks for robots as they are unaware of the complex fluid dynamics and the behavior of liquids when pouring. Therefore, it is not possible to formulate a generic pouring policy for real-time applications. In this paper, we propose PourNet, as a generalized solution to pouring different liquids into containers. PourNet is a h...
-
Increasingly volatile markets challenge companies and demand flexible production systems that can be quickly adapted to new conditions. Machine Learning has proven to show significant potential in supporting the human operator during the time-consuming and complex task of robot pro-gramming by identifying relevant parameters of the underlying robot control program. We present a solution to learn t...
-
Control of wheeled humanoid locomotion is a challenging problem due to the nonlinear dynamics and under-actuated characteristics of these robots. Traditionally, feedback controllers have been utilized for stabilization and locomotion. However, these methods are often limited by the fidelity of the underlying model used, choice of controller, and environmental variables considered (surface type, gr...
-
Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning i...
-
Deep reinforcement learning (DRL) is a promising approach to solve complex control tasks by learning policies through interactions with the environment. However, the training of DRL policies requires large amounts of training experiences, making it impractical to learn the policy directly on physical systems. Sim-to-real approaches leverage simulations to pretrain DRL policies and then deploy them...
-
Compared to their rigid counterparts, soft material robotic systems offer great advantages when it comes to flexibility and adaptability. Despite their advantages, modeling of soft systems is still a challenging task, due to the continuous and often highly nonlinear nature of deformation these systems exhibit. Tasks like motion planning or design optimization of soft robots require computationally...
-
Proprioception, or the perception of the configuration of one's body, is challenging to achieve with soft robots due to their infinite degrees of freedom and incompatibility with most off-the-shelf sensors. This work explores the use of inertial measurement units (IMUs), sensors that output orientation with respect to the direction of gravity, to achieve soft robot proprioception. A simple method ...
-
Fluidically actuated soft robots have promising capabilities such as inherent compliance and user safety. The control of soft robots needs to properly handle nonlinear actuation dynamics, motion constraints, workspace limitations, and variable shape stiffness, so having a unique algorithm for all these issues would be extremely beneficial. In this work, we adapt Model Predictive Control (MPC), pop...
-
As robots begin to move from structured industrial environments to the real world, they must be equipped to not only safely interact with the environment, but also reason about how to leverage contact to perform tasks. In this work, we develop a modeling and motion planning framework for continuum robots that accounts for contact anywhere along the robot. We first present an analytical model for c...
-
Soft robotics has the potential to revolutionize robotic locomotion, in particular, soft robotic swimmers offer a minimally invasive and adaptive solution to explore and preserve our oceans. Unfortunately, current soft robotic swimmers are vastly inferior to evolved biological swimmers, especially in terms of controllability, efficiency, maneuverability, and longevity. Additionally, the tedious it...
-
Model-Based Disturbance Estimation for a Fiber-Reinforced Soft Manipulator using Orientation Sensing
To aid in real-world situations, soft robots need to be able to estimate their state and external interactions based on proprioceptive sensors. Estimating disturbances allows a soft robot to perform desirable force control. However, even in the case of rigid manipulators, force estimation at the end-effector is seen as a non-trivial problem. And indeed, current approaches to address this challenge...
-
From a medical standpoint, detecting the size and shape of hard inclusions hidden in soft three-dimensional objects is of great significance for early detection of cancer through palpation. Soft robots, especially soft grippers, substantially broaden robots' palpation capabilities from soft to hard materials without the assistance of a camera. We have recently introduced a CNN-Bayes approach which...
-
Autonomous exploration in unknown environments is a fundamental task for robots. Existing approaches mostly were concentrated on the efficiency of the exploration with the assumption of perfect state estimation, but the drift of pose estimation in visual SLAM occurs frequently and is detrimental to robot's localization and exploration performance. In this paper, a perception-aware exploration(PAE)...
-
We present a Visual-Inertial Odometry (VIO) algorithm with multiple non-overlapping monocular cameras aiming at improving the robustness of the VIO algorithm. An initialization scheme and tightly-coupled bundle adjustment for multiple non-overlapping monocular cameras are proposed. With more stable features captured by multiple cameras, VIO can maintain stable state estimation, especially when one...
-
This paper advocates the Gaussian belief propagation solver for factor graphs in the case of gas distribution mapping to support an olfactory sensing robot. The local message passing of belief propagation moves away from the standard Cholesky decomposition technique, which avoids solving the entire factor graph at once and allows for only areas of interest to be updated more effectively. Implement...
-
Sensor-equipped unoccupied aerial vehicles (UAVs) have the potential to help reduce search times and alleviate safety risks for first responders carrying out Wilderness Search and Rescue (WiSAR) operations, the process of finding and rescuing person(s) lost in wilderness areas. Unfortunately, visual sensors alone do not address the need for robustness across all the possible terrains, weather, and...
-
In this paper, we introduce an event-based visual odometry and mapping framework that relies on decaying event-based corners. Event cameras, unlike conventional cam-eras, can provide sensor data during high-speed motions or in scenes with high dynamic ranges. Rather than providing intensity information at a global shutter rate, events are trig-gered asynchronously depending on whether there is a c...
-
Steerable needles are medical devices with the ability to follow curvilinear paths to reach targets while circumventing obstacles. In the deployment process, a human operator typically places the steerable needle at its start position on a tissue surface and then hands off control to the automation that steers the needle to the target. Due to uncertainty in the placement of the needle by the human...
-
The introduction of neuroendoscopy, microneu- rosurgery, neuronavigation, and intraoperative imaging for surgical operations has made significant improvements over other traditionally invasive surgical techniques. The integration of magnetic resonance imaging (MRI)-driven surgical devices with intraoperative imaging and endoscopy can enable further advancements in surgical treatments and outcomes....
-
Robot-assisted gastrointestinal endoscopic surgery (GES) as a kind of natural orifice transluminal endoscopic surgery (NOTES) is the next-generation minimally invasive surgery (MIS). Besides, rendering certain autonomy to a Gas-trointestinal Endoscopic Surgical Robot (GESR) is promising but highly challenging. Therefore, to accelerate the development and augment the autonomy of GESR, we use Coppel...
-
Establishing a physics-based model capturing the kinetostatic behavior of concentric tube continuum robots is challenging as elastic interactions between the flexible tubes constituting the robot result in a highly non-linear problem. The Goldstandard physics-based model using the Cosserat theory of elastic rods achieves reasonable approximations with 1.5 - 3 % with respect to the robot's length, ...
-
Flexible Endoscopes (FEs) for colonoscopy present several limitations due to their inherent complexity, resulting in patient discomfort and lack of intuitiveness for clinicians. Robotic FEs with autonomous control represent a viable solution to reduce the workload of endoscopists and the training time while improving the procedure outcome. Prior works on autonomous endoscope FE control use heurist...
-
Variable stiffness catheters typically rely on thermally induced stiffness transitions with a transition temperature above body temperature. This imposes considerable safety limitations for medical applications. In this work, we present a variable stiffness catheter using a hybrid control strategy capable of actively heating and actively cooling the catheter material. The proposed catheter is made...
-
Tactile robots shall be deployed for dynamic task execution in production lines with small batch sizes. Therefore, these robots should have the ability to respond to changing conditions and be easy to (re-)program. Operating under uncertain environments requires unifying subsystems such as robot motion and force policy into one framework, referred to as tactile skills. In this paper, we propose th...
-
Quadrupedal manipulators require to be compliant when dealing with external forces during autonomous manipulation, tele-operation or physical human-robot interaction. This paper presents a whole-body controller that allows for the implementation of a Cartesian impedance control to coordinate tracking performance and desired compliance for the robot base and manipulator arm. The controller is formu...
-
Electro-adhesive clutches have become effective tools for variable stiffness functions in many robotic systems due to their light weight, high speed and strong brake force. In this paper, we present a novel, tubular design of an electro-adhesive clutch. Our clutch consists of flexible electrode sheets rolled into a tubular structure. This design allows encapsulating large electrode areas in a comp...
-
This paper proposes a variable impedance control architecture to facilitate eye surgery training in a dual-user haptic system. In this system, an expert surgeon (the trainer) and a novice surgeon (the trainee) collaborate on a surgical procedure using their own haptic devices. The mechanical impedance parameters of the trainer's haptic device remain constant during the operation, whereas those of ...
-
A robot designed to coexist and work with humans in the same workspace should be able to work at the same speed as humans and have safe contact with humans and with the environment. However, when a robot arm has been given flexibility through mechanisms and controls for the purpose of coexistence, it is difficult for it to perform tasks at the speed and accuracy desired by humans if it is moved si...
-
Although the average healthy adult transitions from sit to stand over 60 times per day, most research on powered prosthesis control has only focused on walking. In this paper, we present a data-driven controller that enables sitting, standing, and walking with minimal tuning. Our controller comprises two high level modes of sit/stand and walking, and we develop heuristic biomechanical rules to con...
-
Hardware acceleration can revolutionize robotics, enabling new applications by speeding up robot response times while remaining power-efficient. However, the diversity of acceleration options makes it difficult for roboticists to easily deploy accelerated systems without expertise in each specific hardware platform. In this work, we address this challenge with RobotCore, an architecture to integra...
-
We present Tasho (Task specification for receding horizon control), an open-source Python toolbox that facilitates systematic programming of optimal control problem (OCP)-based robot motion skills. Separation-of-concerns is followed while designing the components of a motion skill, which promotes their modularity and reusability. This allows us to program complex motion tasks by configuring and co...
-
Containerization promises to strengthen platform-independent development, better resource utilization, and secure deployment of software. As these benefits come with negligible overhead in CPU and memory utilization, containerization is increasingly being adopted in mobile robotic applications. An open challenge is supporting software tasks that have mixed-criticality requirements. Even more chall...
-
In the Robot Operating System (ROS), a major middleware for robots, the Transform Library (TF) is a mandatory package that manages transformation information between coordinate systems by using a single-rooted directed tree and providing methods for registering and computing the information. However, the tree has two fundamental problems. The first is its poor scalability: since it accepts only a ...
-
The fit of a wearable device, such as a prosthesis, can be quantitatively characterized by the mechanical coupling at the user-device interface. It is thought that the mechanical impedance, specifically the stiffness and damping, of wearable device interfaces can significantly impact human performance while using them. To test this theory, we develop a forearm-mounted testbed with a motorized, two...
-
Lower Limb Exoskeletons (LLE) have received considerable interest in strength augmentation, rehabilitation, and walking assistance scenarios. For strength augmentation, LLE is expected to have the capability of reducing metabolic energy. However, the energy for adjusting Center of Gravity (CoG) is a main part of the total energy consumed during walking. This paper proposes a novel Human-exoskeleto...
-
Despite the large amount of available exoskeletons, their use in daily life is still limited due to the absence of testing in real-life environments. Thus, the present work aims to test on a series of uneven terrains a wearable ankle exoskeleton, named RANK, designed for walking assistance and drop-foot prevention. RANK consists of a 3D-printed brace attached to the user and a piezoresistive insol...
-
Upper limb impairments and weakness are com-mon post-stroke and with advanced aging. Rigid exoskeletons have been developed as a potential solution, but have had limited impact. In addition to user concerns about safety, their weight and appearance, the rigid attachment and typical anchoring methods can result in skin damage. In this paper, we present a soft, fabric-based pneumatic sleeve, which c...
-
Shared control can help in teleoperated object manipulation by assisting with the execution of the user's intention. To this end, robust and prompt intention estimation is needed, which relies on behavioral observations. Here, an intention estimation framework is presented, which uses natural gaze and motion features to predict the current action and the target object. The system is trained and te...
-
Equipping robots with the ability to infer human intent is a vital precondition for effective collaboration. Most computational approaches towards this objective derive a probability distribution of “intent” conditioned on the robot's perceived state. However, these approaches typically assume task-specific labels of human intent are known a priori. To overcome this constraint, we propose the Dise...
-
Motion prediction of road users in traffic scenes is critical for autonomous driving systems that must take safe and robust decisions in complex dynamic environments. We present a novel motion prediction system for autonomous driving. Our system is based on the Bayesian inverse planning framework, which efficiently orchestrates map-based goal extraction, a classical control-based trajectory genera...
-
Predicting object motion behaviour is a challenging but crucial task for safe decision making and path planning for an autonomous vehicle. It is challenging in large part due to the uncertain, multi-modal, and practically intractable set of possible agent-agent and agent-space interactions, especially in urban driving settings. Models solely based on constant velocity or social force have an inher...
-
How to design an optimal wearable device for human movement recognition is vital to reliable and accurate human-machine collaboration. Previous works mainly fabricate wearable devices heuristically. Instead, this paper raises an academic question: can we design an optimization algorithm to optimize the fabrication of wearable devices such as figuring out the best sensor arrangement automatically? ...
-
Anticipating the future behavior of pedestrians is a crucial part of deploying Automated Driving Systems (ADS) in urban traffic scenarios. Most recent works utilize a convolutional neural network (CNN) to extract visual information, which is then input to a recurrent neural network (RNN) along with pedestrian-specific features like location and speed to obtain temporal features. However, the major...
-
As robots across domains start collaborating with humans in shared environments, algorithms that enable them to reason over human intent are important to achieve safe inter-play. In our work, we study human intent through the problem of predicting trajectories in dynamic environments. We explore domains where navigation guidelines are relatively strictly defined but not clearly marked in their phy...
-
Performing a complex autonomous mission with a multi-robot system requires to integrate several deliberative approaches to perform task allocation, optimization, and execution control. Implementing such a deliberative architecture is a complex task: it requires the developer to master the decision algorithms themselves (e.g., automated planning models), to have a good knowledge of the involved rob...
-
We present a new solution to the fine-grained retrieval of clean CAD models from a large-scale database in order to recover detailed object shape geometries for RGBD scans. Unlike previous work simply indexing into a moderately small database using an object shape descriptor and accepting the top retrieval result, we argue that in the case of a large-scale database a more accurate model may be fou...
-
Several methods of semantic segmentation using light detection and ranging (LiDAR) sensors have been proposed for the recognition of surrounding objects by autonomous driving cars. LiDAR is a sensor that compensates for the weaknesses of other sensors, such as cameras or radar systems, and semantic segmentation assigns a class label to each point in the LiDAR point cloud. Recently, real-time seman...
-
Context information is important for instance segmentation on point clouds. Existing methods either only use local surroundings by stacking multiple convolution layers or use non-local methods to model long-range interactions. However, they usually directly operate on points which is an unstructured and low-level representation and is highly dependent on context. To address this issue, we propose ...
-
Unsupervised domain adaptation for point cloud semantic segmentation has attracted great attention due to its effectiveness in learning with unlabeled data. Most of existing methods use global-level feature alignment to transfer the knowledge from the source domain to the target domain, which may cause the semantic ambiguity of the feature space. In this paper, we propose a graph-based framework t...
-
Place recognition is seen as a crucial factor to correct cumulative errors in Simultaneous Localization and Mapping (SLAM) applications. Most existing studies focus on visual place recognition, which is inherently sensitive to environmental changes such as illumination, weather and seasons. Considering these facts, more recent attention has been attracted to use 3-D Light Detection and Ranging (Li...
-
Fisheye object detection is a difficult task in robotics and autonomous driving. One of the reasons is that the fisheye datasets are inferior to standard image datasets in scale and quantity, which inspires the idea of using standard image datasets for fisheye object detection. However, the models trained on standard image datasets do not perform well with fisheye data. In this work, we explore th...
-
Disinfection robots have applications in promoting public health and reducing hospital acquired infections and have drawn considerable interest due to the COVID-19 pan-demic. To disinfect a room quickly, motion planning can be used to plan robot disinfection trajectories on a reconstructed 3D map of the room's surfaces. However, existing approaches discard semantic information of the room and, thu...
-
Assistive robot systems have been developed to help people accomplish daily manipulation tasks especially for those with disabilities, where scene understanding plays a crucial role in enabling robots to interpret the surroundings and behave accordingly. Most of the current systems approach scene understanding without considering the functional dependencies between objects. However, it is only val...
-
This paper presents a novel approach for multi-robot unknown area exploration. Recently, the frontier tree data structure was used in single robot exploration to memorize frontiers, their positions, exploration state, and the map. This tree could be queried to decide on further exploration steps. In this paper, we take the concept further for multi-robot exploration by proposing a new abstraction ...
-
We present a manifold optimization approach to solve inference and planning problems with range constraints. The core of our approach is the definition of a manifold that represents points or poses with range constraints. We discover that the manifold of range-constrained points is homogeneous under the rigid transformation group action, and utilize the group action to derive the tangent space, re...
-
This paper presents a coverage based control algorithm to coordinate a group of autonomous robots. Most of the solutions presented in the literature rely on an exact Voronoi partitioning, whose computation requires complete knowledge of the environment to be covered. This can be achieved only by robots with unlimited sensing capabilities, or through communication among robots in a limited sensing ...
-
In this work, we consider the Multi-Agent Pickup-and-Delivery (MAPD) problem, where agents constantly engage with new tasks and need to plan collision-free paths to execute them. To execute a task, an agent needs to visit a pair of goal locations, consisting of a pickup location and a delivery location. We propose two variants of an algorithm that assigns a sequence of tasks to each agent using th...
-
We present a novel overconstraining and constraint-discarding method for asynchronous, real-time, decentralized, multi-robot trajectory planning that ensures collision avoidance. Our approach utilizes communication between robots. The communication medium is best-effort: messages may be dropped, re-ordered or delayed. Robots conservatively constrain themselves against others assuming they may be w...
-
This paper presents an algorithm for a team of mobile robots to simultaneously learn a spatial field over a domain and spatially distribute themselves to optimally cover it. Drawing from previous approaches that estimate the spatial field through a centralized Gaussian process, this work leverages the spatial structure of the coverage problem and presents a decentralized strategy where samples are...
-
We present a centralized algorithm for labeled, disk-shaped Multi-Robot Path Planning (MPP) in a continuous planar workspace with polygonal boundaries. Our method automatically transform the continuous problem into a discrete, graph-based variant termed the pebble motion problem, which can be solved efficiently. To construct the underlying pebble graph, we identify inscribed circles in the workspa...
-
Leveraging both the autonomy of robots and the expert knowledge of humans can enable a multi-robot system to complete missions in challenging environments with a high degree of adaptivity and robustness. This paper proposes a multi-modal task-based graphical user interface for controlling a heterogeneous multi-robot team. The core of the interface is an integrated multi-robot task allocation syste...
-
Robots using classical control have revolutionised assembly lines where the environment and manipulated objects are restricted and predictable. However, they have proven less effective when the manipulated objects are deformable due to their complex and unpredictable behaviour. The use of tactile sensors and continuous monitoring of tactile feedback is there-fore particularly important for pick-an...
-
This research focuses on soft robotic bladders that are used to monitor and control the interaction between a user's head and the shell of a Smart Helmet. Compression of these bladders determines impact dissipation; hence the focus of this paper is sensing and estimation of bladder compression. An IR rangefinder-based solution is evaluated using regression techniques as well as a Neural Network to...
-
Whiskers are widely used by animals for sensing physical interactions with their environments. By combining the Kirigami skin pop-up feature and flexible conducting layer, we designed a deployable Kirigami whisker sensor. The sensor can deploy from a flat state to a sensing state while whisker stiffness and initial pop-up angle can be tuned by adjusting the pre-stretch strain. Preliminary results ...
-
Soft sensorised skins are essential for improving robotic manipulation capabilities towards that of humans. Integration of sensors into existing robotic hands is challenging due to rigidity of components, low packing density or poor sensor response. We propose a sensorised skin, based-on barometric sensing, which can be molded over a skeletal robot hand. The sensors connect air chambers embedded i...
-
We create a virtual 2D tactile array for soft pneumatic actuators using embedded audio components. We detect contact-specific changes in sound modulation to infer tactile information. We evaluate different sound representations and learning methods to detect even small contact variations. We demonstrate the acoustic tactile sensor array by the example of a PneuFlex actuator and use a Braille displ...
-
In this paper, we propose a novel variable-length estimation approach for shape sensing of extensible soft robots utilizing fiber Bragg gratings (FBGs). Shape reconstruction from FBG sensors has been increasingly developed for soft robots, while the narrow stretching range of FBG fiber makes it difficult to acquire accurate sensing results for extensible robots. Towards this limitation, we newly i...
-
Underwater robots have a variety of potential uses, including marine resource research, ecological research, and disaster relief. Most of the underwater robots currently in practical use have screw propulsion systems, which have several noises, collision, and entrainment problems. There is a lot of research on underwater robots using soft actuators to solve these problems. However, current soft ac...
-
Multi agent coverage and robot navigation are two very important research fields within robotics. However, their intersection has received limited attention. In multi agent coverage, perfect navigation is often assumed, and in robot navigation, the focus is often to minimize the localization error with the aid of stationary features from the environment. The need for integration of the two becomes...
-
For enabling efficient, large-scale coordination of unmanned aerial vehicles (UAV s) under the labeled setting, in this work, we develop the first polynomial time algorithm for the reconfiguration of many moving bodies in three-dimensional spaces, with provable 1. $x$ asymptotic makespan optimality guarantee under high robot density. More precisely, on an $m_{1} \times m_{2} \times m_{3}$ grid, $m...
-
In many environmental monitoring applications robots are often tasked to visit various distinct locations to make observations and/or collect specific measurements. The problem of scheduling and assigning robots to the various tasks and planning feasible paths for the robots can be posed as an Orienteering Problem (OP). In the standard OP, routing and scheduling is achieved by maximizing an object...
-
Portfolio-based algorithm selection can help in choosing the best suited algorithm for a given task while leveraging the complementary strengths of the candidates. Solving the Multi-Agent Path Finding (MAPF) problem optimally has been proven to be NP-Hard. Furthermore, no single optimal algorithm has been shown to have the fastest runtime for all MAPF problem instances, and there are no proven app...
-
Online coverage path planning to explore an unknown workspace with multiple homogeneous robots could be either centralized or distributed. While distributed planners are computationally faster, centralized planners can produce more efficient paths, reducing the duration of completing a coverage mission significantly. To exploit the power of a centralized framework, we propose a receding horizon ce...
-
This paper deals with Multi-robot Trajectory Planning, that is, the problem of computing trajectories for multiple robots navigating in a shared space while minimizing for control energy. Approaches based on trajectory optimization can solve this problem optimally. However, such methods are hampered by complex robot dynamics and collision constraints that couple robot's decision variables. We prop...
-
We provide theoretical bounds on the worst case performance of the greedy algorithm in seeking to maximize a normalized, monotone, but not necessarily submodular ob-jective function under a simple partition matroid constraint. We also provide worst case bounds on the performance of the greedy algorithm in the case that limited information is available at each planning step. We specifically conside...
-
For biomedical applications in targeted therapy delivery and interventions, a large swarm of micro-scale particles (“agents”) has to be moved through a maze-like environment (“vascular system”) to a target region (“tumor”). Due to limited on-board capabilities, these agents cannot move autonomously; instead, they are controlled by an external global force that acts uniformly on all particles. In t...
-
The ability to evolve is fundamental for any valuable autonomous agent whose knowledge cannot remain limited to that injected by the manufacturer. Consider for example a home assistant robot: it should be able to incrementally learn new object categories when requested, but also to recognize the same objects in different environments (rooms) and poses (hand-held/on the floor/above furniture), whil...
-
In order to be effective general purpose machines in real world environments, robots not only will need to adapt their existing manipulation skills to new circumstances, they will need to acquire entirely new skills on-the-fly. One approach to achieving this capability is via Multi-task Reinforcement Learning (MTRL). Most recent work in MTRL trains a single policy to solve all tasks at once. In th...
-
Autonomous agents need to perceive the world in a robust way, such that the shift in data distribution does not lead to faulty perception results. When agents cannot be trained with abundant data, agents may need to operate on real world environments while trained on simulated data, and suffer from domain shift. This paper proposes an effective and robust unsupervised domain adaptation (UDA) metho...
-
In this work, we propose a data-driven approach to optimize the parameters of a simulation such that control policies can be directly transferred from simulation to a real-world quadrotor. Our neural network-based policies take only onboard sensor data as input and run entirely on the embed-ded hardware. In real-world experiments, we compare low-level Pulse-Width Modulated control with higher-leve...
-
Unsupervised domain adaptation (UDA) aims to learn domain-invariant representations between the labeled source domain and the unlabeled target domain. Existing self- training-based UDA methods use ground truth and pseudo- labels to supervise source data and target data respectively. However, strong supervision in the source domain and pseudo- label noise in the target domain lead to some problems,...
-
Domain adaptation is an important property in robot vision, which enables the neural networks pre-trained on source domains to adapt target domains automatically without any annotation efforts. During this process, source data is not always accessible due to the constraints of expensive storage overhead and data privacy protection. Therefore, the source domain pre-trained model is expected to opti...
-
Randomization is currently a widely used approach in Sim2Real transfer for data-driven learning algorithms in robotics. Still, most Sim2Real studies report results for a specific randomization technique and often on a highly customized robotic system, making it difficult to evaluate different randomization approaches systematically. To address this problem, we define an easy-to-reproduce experimen...
-
In recent years, with the combination of tissue engineering and additive manufacturing technologies, the possibility of fabricating scaffolds with porosity and complex structure has been improved. Since the properties of most biomaterial inks are influenced by temperature and thereby affect the quality of the scaffolds, a controlled printing environment is very important. This study focuses on tem...
-
We introduce a robotic assembly system that streamlines the design-to-make workflow for going from a CAD model of a product assembly to a fully programmed and adaptive assembly process. Our system captures (in the CAD tool) the intent of the assembly process for a specific robotic workcell and generates a recipe of task-level instructions. By integrating visual sensing with deep-learned perception...
-
Robot assembly discovery (RAD) is a challenging problem that lives at the intersection of resource allocation and motion planning. The goal is to combine a predefined set of objects to form something new while considering task execution with the robot-in-the-loop. In this work, we tackle the problem of building arbitrary, predefined target structures entirely from scratch using a set of Tetris-lik...
-
This paper addresses the problem of copying an unknown assembly of primitives with known shape and appearance using information extracted from a single photograph by an off-the-shelf procedure for object detection and pose estimation. The proposed algorithm uses a simple combination of physical stability constraints, convex optimization and Monte Carlo tree search to plan assemblies as sequences o...
-
We present a new algorithm for coordinating the motion of multiple extruders to increase throughput in fused filament fabrication (FFF)/fused deposition modeling (FDM) additive manufacturing. Platforms based on FFF are commonly available and advantageous to several industries, but are limited by slow fabrication time and could be could be significantly improved through efficient use of multiple ex...
-
Work-related musculoskeletal disorders (MSD) are one of the major cause of injuries and absenteeism at work. These lead to important cost in the manufacturing industry. Human-robot collaboration can help decreasing this issue by appropriately distributing the tasks and decreasing the workload of the factory worker. This paper proposes a novel generic task allocation approach based on hierarchical ...
-
Snap-fit peg-in-hole assembly widely exists in both industry and daily life, especially for consumer electronics. The buckle mechanism leads to a damping zone inside the port where insertion force needs to be increased. It is much difficult to automate this process by robots, for size and clearance of the components are always small, and the damping buckle should be perceived and distinguished fro...
-
In this paper, we propose a novel and general method for autonomous robotic assembly of arbitrary and complex-shaped parts in the presence of 6-dimensional uncertainty. When a nominal assembly motion of the robot holding a part is stopped by contact due to uncertainty, our method finds the best estimate for the uncertainty and the contact configuration of the part based on sensed force/torque and ...
-
Autonomously driving vehicles must be able to navigate in dynamic and unpredictable environments in a collision-free manner. So far, this has only been partially achieved in driverless cars and warehouse installations where marked structures such as roads, lanes, and traffic signs simplify the motion planning and collision avoidance problem. We are presenting a new control approach for car-like ve...
-
This paper presents a path planning refinement technique that allows the efficient collision and rollover-free motion planning for mobile manipulator robots working on rough terrain. First, the necessary theoretical background on a mobile manipulator's kinematics and dynamic stability measure is introduced. Then, after the brief introduction of the sampling-based path planning problem, the additio...
-
In recent years, mobile robots are becoming ambitious and deployed in large-scale scenarios. Serving as a high-level understanding of environments, a sparse skeleton graph is beneficial for more efficient global planning. Currently, existing solutions for skeleton graph generation suffer from several major limitations, including poor adaptiveness to different map representations, dependency on rob...
-
This paper addresses a safe planning and control problem for mobile robots operating in communication- and sensor-limited dynamic environments. In this case the robots cannot sense the objects around them and must instead rely on intermittent, external information about the environment, as e.g., in underwater applications. The challenge in this case is that the robots must plan using only this sta...
-
In this paper, we present a decentralized control approach based on a Nonlinear Model Predictive Control (NMPC) method that employs barrier certificates for safe navigation of multiple nonholonomic wheeled mobile robots in unknown environments with static and/or dynamic obstacles. This method incorporates a Learned Barrier Function (LBF) into the NMPC design in order to guarantee safe robot naviga...
-
Sampling-based motion planners are widely used in robotics due to their simplicity, flexibility and computational efficiency. However, in their most basic form, these algorithms operate under the assumption of static scenes and lack the ability to avoid collisions with dynamic (i.e. moving) obstacles. This raises safety concerns, limiting the range of possible applications of mobile robots in the ...
-
We propose a simple imitation learning procedure for learning locomotion controllers that can walk over very challenging terrains. We use trajectory optimization (TO) to produce a large dataset of trajectories over procedurally generated terrains and use Reinforcement Learning (RL) to imitate these trajectories. We demonstrate with a realistic model of the ANYmal robot that the learned controllers...
-
We present a versatile framework for the computational co-design of legged robots and dynamic maneuvers. Current state-of-the-art approaches are typically based on random sampling or concurrent optimization. We propose a novel bilevel optimization approach that exploits the derivatives of the motion planning sub-problem (i.e., the lower level). These motion-planning derivatives allow us to incorpo...
-
The complex dynamics of agile robotic legged locomotion requires motion planning to intelligently adjust footstep locations. Often, bipedal footstep and motion planning use mathematically simple models such as the linear inverted pendulum, instead of dynamically-rich models that do not have closed-form solutions. We propose a real-time optimization method to plan for dynamical models that do not h...
-
Deep reinforcement learning has emerged as a popular and powerful way to develop locomotion controllers for quadruped robots. Common approaches have largely focused on learning actions directly in joint space, or learning to modify and offset foot positions produced by trajectory generators. Both approaches typically require careful reward shaping and training for millions of time steps, and with ...
-
Inspired by the digital twinning systems, a novel real-time digital double framework is developed to enhance robot perception of the terrain conditions. Based on the very same physical model and motion control, this work exploits the use of such simulated digital double synchronized with a real robot to capture and extract discrepancy information between the two systems, which provides high dimens...
-
As Liszt once said “(a virtuoso) must call up scent and blossom, and breathe the breath of life”, a virtuoso plays the piano with passion, poetry, and extraordinary technical ability. Hence, piano playing, being a task that is quintessentially human, becomes a hallmark for roboticians and artificial intelligence researchers to pursue. In this paper, we advocate an end-to-end reinforcement learning...
-
We develop two consensus-based learning algorithms for multi-robot systems applied on complex tasks involving collision constraints and force interactions, such as the cooperative peg-in-hole placement. The proposed algorithms integrate multi-robot distributed consensus and normalizing-flow-based reinforcement learning. The algorithms guarantee the stability and the consensus of the multi-robot sy...
-
We investigate the problem of coordinating two robot arms to solve non-monotone tabletop multi-object re- arrangement tasks. In a non-monotone rearrangement task, complex object-object dependencies exist that require moving some objects multiple times to solve an instance. In working with two arms in a large workspace, some objects must be handed off between the robots, which further complicates t...
-
Visual localization, which is a key technology to realize the automation of cell manipulation, has been widely studied. Since the depth of field of the microscope is narrow, the planar localization and depth estimation are usually coupled together. At present, most methods adopt the serial working mode of focusing first and then planar localization, but they usually do not have good real-time perf...
-
Object manipulation in cluttered scenes is a difficult and important problem in robotics. To efficiently manipulate objects, it is crucial to understand their surroundings, especially in cases where multiple objects are stacked one on top of the other, preventing effective grasping. We here present DUQIM-Net, a decision-making approach for object manipulation in a setting of stacked objects. In DU...
-
Tracking position and orientation independently affords more agile maneuver for over-actuated multirotor Unmanned Aerial Vehicles (UAVs) while introducing undesired downwash effects; downwash flows generated by thrust generators may counteract others due to close proximity, which significantly threatens the stability of the platform. The complexity of modeling aerodynamic airflow challenges contro...
-
Although the manipulating of the unmanned aerial manipulator (UAM) has been widely studied, vision-based UAM approaching, which is crucial to the subsequent manipulating, generally lacks effective design. The key to the visual UAM approaching lies in object tracking, while current UAM tracking typically relies on costly model-based methods. Besides, UAM approaching often confronts more severe obje...
-
Flying animals possess highly complex physical characteristics and are capable of performing agile maneuvers using their wings. The flapping wings generate complex wake structures that influence the aerodynamic forces, which can be difficult to model. While it is possible to model these forces using fluidstructure interaction, it is very computationally expensive and difficult to formulate. In thi...
-
In aerial cooperative transportation missions, it has been recognized that for small-sized but heavy payloads, the cable-suspended framework is a preferred manner. However, to maintain proper safe flight distances, cables always stay inclined, which implies that horizontal force components have to be generated by UAVs, and only partial thrust forces are used for gravity compensation. To overcome t...
-
This paper presents a novel trajectory planning method for aerial perching. Compared with the existing work, the terminal states and the trajectory durations can be adjusted adaptively, instead of being determined in advance. Further-more, our planner is able to minimize the tangential relative speed on the premise of safety and dynamic feasibility. This feature is especially notable on micro aeri...
-
Free-space-oriented roadmaps typically generate a series of convex geometric primitives, which constitute the safe region for motion planning. However, a static environment is assumed for this kind of roadmap. This assumption makes it unable to deal with dynamic obstacles and limits its applications. In this paper, we present a dynamic free-space roadmap, which provides feasible spaces and a navig...
-
This paper proposes a perception-shared and swarm trajectory global optimal (STGO) algorithm fused UAVs formation motion planning framework aided by an active sensing system. First, the point cloud received by each UAV is fit by the gaussian mixture model (GMM) and transmitted in the swarm. Resampling from the received GMM contributes to a global map, which is used as the foundation for consensus....
-
We propose a fully autonomous system for mobile robot exploration in unknown environments. Our system employs a novel frontier detection algorithm based on the fast front propagation (FFP) technique and uses parallel path planning to reach the detected front regions. Given an occupancy grid map in 2D, possibly updated online, our algorithm can find all the frontier points that can allow mobile rob...
-
Visual Inertial Odometry (VIO) is a widely studied localization technique in robotics. State-of-the-art VIO algorithms are composed of two parts: a frontend which performs visual perception and inertial measurement pre-processing, and a backend which fuses vision and inertial measurements to estimate the robot's pose. Both image processing in the frontend and sensor fusion in the backend are compu...
-
Our goal is to train a neural planner that can capture diverse driving behaviors in complex urban scenarios. We observe that even state-of-the-art neural planners are struggling to perform common maneuvers such as lane change, which is rather natural for human drivers. We propose to explore the multi-modalities in the planning problem and force the neural planner to explicitly consider different p...
-
Learning-based model predictive control has been widely applied in autonomous racing to improve the closed-loop behaviour of vehicles in a data-driven manner. When environmental conditions change, e.g., due to rain, often only the predictive model is adapted, but the controller parameters are kept constant. However, this can lead to suboptimal behaviour. In this paper, we address the problem of da...
-
We present a method to find the optimal control strategy for a robot using prior information of localization that maximizes the probability of satisfaction of a temporal logic specification while considering the uncertainty in both motion and sensing, two major causes for localization uncertainty. The specifications are given in the probabilistic computation tree logic (PCTL) formulae over a set o...
-
Object Goal Navigation (ObjectNav) task is to navigate an agent to an object category in unseen environments without a pre-built map. In this paper, we solve this task by predicting the distance to the target using semantically-related objects as cues. Based on the estimated distance to the target object, our method directly choose optimal midterm goals that are more likely to have a shorter path ...
-
Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sample-efficient result than learning by pixels. This work pres...
-
In this paper, we describe Direct Sparse Odometry Lite (DSOL), an improved version of Direct Sparse Odometry (DSO) [1]. We propose several algorithmic and implementation enhancements which speed up computation by a significant factor (on average 5x) even on resource-constrained platforms. The increase in speed allows us to process images at higher frame rates, which in turn provides better results...
-
Planning autonomous driving behaviors in dense traffic is challenging. Human drivers are able to influence their road environment to achieve (otherwise unachievable) goals, by communicating their intents to other drivers. An autonomous system that is required to drive in the presence of human traffic must thus possess this fundamental negotiation capability. This work presents a novel benchmark th...
-
Learning-based visual odometry (VO) has gained increasing popularity in autonomous navigation of small robots. However, most methods in the category require computation resources not normally available on edge systems. We contend that specialized hardware accelerators are ideal solutions to this problem because of their superior energy efficiency. In this paper, we first propose a model to derive ...
-
Point cloud semantic segmentation is important for road scene perception, a task for driverless vehicles to achieve full fledged autonomy. In this work, we introduce Mask Point Transformer Network (MPT-Net), a novel architecture for point cloud segmentation that is simple to implement. MPT-Net consists of a local and global feature encoder and a transformer based decoder; a 3D Point-Voxel Convolut...
-
We introduce a novel approach for temporal activity segmentation with timestamp supervision. Our main contribution is a graph convolutional network, which is learned in an end-to-end manner to exploit both frame features and connections between neighboring frames to generate dense framewise labels from sparse timestamp labels. The gener-ated dense framewise labels can then be used to train the seg...
-
Reliable and stable 6D pose estimation of un-cooperative space objects plays an essential role in on-orbit servicing and debris removal missions. Considering that the pose estimator is sensitive to background interference, this paper proposes a counterfactual analysis framework named CA-SpaceNet to complete robust 6D pose estimation of the space-borne targets under complicated background. Specific...
-
Monocular depth estimation has been actively studied in fields such as robot vision, autonomous driving, and 3D scene understanding. Given a sequence of color images, unsupervised learning methods based on the framework of Structure-From-Motion (SfM) simultaneously predict depth and camera relative pose. However, dynamically moving objects in the scene violate the static world assumption, resultin...
-
Two-view structure from motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM (vSLAM). Many existing end-to-end learning-based methods usually formulate it as a brute regression problem. However, the inadequate utilization of traditional geometry model makes the model not robust in unseen environments. To improve the generalization capability and robustness of end-to-end two-view Sf...
-
This work focuses on estimating 6D poses and sizes of category-level objects from a single RGB-D image. How to exploit the complementary RGB and depth features plays an important role in this task yet remains an open question. Due to the large intra-category texture and shape variations, an object instance in test may have different RGB and depth features from those of the object instances in trai...
-
Comprehending human motion is a fundamental challenge for developing Human-Robot Collaborative applications. Computer vision researchers have addressed this field by only focusing on reducing error in predictions, but not taking into account the requirements to facilitate its implementation in robots. In this paper, we propose a new model based on Transformer that simultaneously deals with the rea...
-
In this work, we introduce an adaptive control framework for human-robot collaborative transportation of objects with unknown deformation behaviour. The proposed framework takes as input the haptic information transmitted through the object, and the kinematic information of the human body obtained from a motion capture system to create reactive whole-body motions on a mobile collaborative robot. I...
-
When teleoperating complex robotic manipula-tors, operators often find it most natural to issue commands that dictate end effector movements in task space. If the robot has redundant degrees of freedom, the translation of this com-mand from task space into configuration space can affect the robot's maneuverability, smoothness of motion, and the general precision of the teleoperated system. In this...
-
The recognition of actions performed by humans and the anticipation of their intentions are important enablers to yield sociable and successful collaboration in human-robot teams. Meanwhile, robots should have the capacity to deal with multiple objectives and constraints, arising from the collaborative task or the human. In this regard, we propose vision techniques to perform human action recognit...
-
As humans and robots start to collaborate in close proximity, robots are tasked to perceive, comprehend, and anticipate human partners' actions, which demands a predictive model to describe how humans collaborate with each other in joint actions. Previous studies either simplify the collaborative task as an optimal control problem between two agents or do not consider the learning process of human...
-
While human-robot interaction studies are becoming more common, quantification of the effects of repeated interaction with an exoskeleton remains unexplored. We draw upon existing literature in human skill assessment and present extrinsic and intrinsic performance metrics that quantify how the human-exoskeleton system's behavior changes over time. Specifically, in this paper, we present a new perf...
-
Rearrangement tasks have been identified as a crucial challenge for intelligent robotic manipulation, but few methods allow for precise construction of unseen structures. We propose a visual foresight model for pick-and-place rearrangement manipulation which is able to learn efficiently. In addition, we develop a multi-modal action proposal module which builds on the Goal-Conditioned Transporter N...
-
Learning from Demonstration (LfD) requires methodologies able to generalize tasks in new situations. This paper studies the use of virtual demonstrations to extend the extrapolation capabilities of probabilistic motion models such as the traPPCA method. Similarly to other LfD methods, traPPCA is able to calculate new trajectories very fast, but does not generalize well outside the area covered by ...
-
Learning goal conditioned control in the real world is a challenging open problem in robotics. Reinforcement learning systems have the potential to learn autonomously via trial-and-error, but in practice the costs of manual reward design, ensuring safe exploration, and hyperparameter tuning are often enough to preclude real world deployment. Imitation learning approaches, on the other hand, offer ...
-
We propose an approach that enables simultaneous interpretable learning of a high-level discrete behaviour and its low-level rhythmic sub-behaviour. We do this though a unified reward function, where a reward function that only describes low-level behaviour, with less impact on learning of other behaviours is recovered from few-shot motion demonstrations. To this end, we first extract local behavi...
-
Learning intricate manipulation skills from human demonstrations requires good sample efficiency. We introduce a novel learning algorithm, the Curvature-regularized Variational Auto-Encoder (CurvVAE), to achieve this goal. The CurvVAE is able to model the natural variations in human-demonstrated trajectory data without overfitting. It does so by regularizing the curvature of the learned manifold. ...
-
While learning from synthetic training data has recently gained an increased attention, in real-world robotic applications, there are still performance deficiencies due to the so-called Sim-to-Real gap. In practice, this gap is hard to resolve with only synthetic data. Therefore, we focus on an efficient acquisition of real data within a Sim-to-Real learning pipeline. Concretely, we employ deep Ba...
-
Research in manipulation of deformable objects is typically conducted on a limited range of scenarios, because handling each scenario on hardware takes significant effort. Realistic simulators with support for various types of deformations and interactions have the potential to speed up experimentation with novel tasks and algorithms. However, for highly deformable objects it is challenging to ali...
-
Detection and segmentation of moving obstacles, along with prediction of the future occupancy states of the local environment, are essential for autonomous vehicles to proactively make safe and informed decisions. In this paper, we propose a framework that integrates the two capabilities together using deep neural network architectures. Our method first detects and segments moving objects in the s...
-
Surgical scene segmentation is fundamentally crucial for prompting cognitive assistance in robotic surgery. However, pixel-wise annotating surgical video in a frame-by-frame manner is expensive and time consuming. To greatly reduce the labeling burden, in this work, we study semi-supervised scene segmentation from robotic surgical video, which is practically essential yet rarely explored before. W...
-
Integrating different representations from complementary sensing modalities is crucial for robust scene interpretation in autonomous driving. While deep learning architectures that fuse vision and range data for 2D object detection have thrived in recent years, the corresponding modalities can degrade in adverse weather or lighting conditions, ultimately leading to a drop in performance. Although ...
-
Though robot learning is often formulated in terms of discrete-time Markov decision processes (MDPs), physical robots require near-continuous multiscale feedback control. Machines operate on multiple asynchronous sensing modalities, each with different frequencies, e.g., video frames at 30Hz, proprioceptive state at 100Hz, force-torque data at 500Hz, etc. While the classic approach is to batch obs...
-
Model predictive control is a promising method in robot control tasks. How to design an effective model structure and efficient prediction framework for model predictive control is still an open challenge. To reduce the time consumption and avoid compounding-error of the multi-step prediction process in model predictive control, we propose a single-model simultaneous framework, which uses single d...
-
Enabling robots to perform complex dynamic tasks such as picking up an object in one sweeping motion or pushing off a wall to quickly turn a corner is a challenging problem. The dynamic interactions implicit in these tasks are critical towards the successful execution of such tasks. Graph neural networks (GNNs) provide a principled way of learning the dynamics of interactive systems but can suffer...
-
In the lateral access environment, the robot be-havior should be planned considering surrounding objects and obstacles because object observation directions and approach angles are limited. To safely retrieve a partially occluded target object in these environments, we have to relocate objects using prehensile actions to create a collision-free path for the target. We propose a learning-based meth...
-
Deep Reinforcement Learning (DRL) faces challenges bridging the sim-to-real gap to enable real-world applications. In contrast to the simulated environments used in conventional DRL training, real-world systems are non-linear and evolve in an asynchronous fashion; sensors and actuators have limited precision; communication channels are noisy; and many components introduce variable delays. While th...
-
Hierarchical algorithms have often been used to plan and execute complicated robotic sequential manipulation tasks, where an abstract planner searches for a skill sequence in an abstract space, and each skill generates actual motions on the basis of the planned skill sequences. To generate executable plans, the abstract planner should know the pre-/postconditions of each skill and appropriately ch...
-
In recent years, robots have become vital to achieving manufacturing competitiveness. Especially in industrial environments, a strong level of interaction is reached when humans and robots form a dynamic system that works together towards achieving a common goal or accomplishing a task. However, the human-robot collaboration can be cognitively demanding, potentially contributing to cognitive fatig...
-
A control barrier functions-based quadratic programming (CBF-QP) method has emerged as a controller synthesis tool to assure safety of autonomous systems owing to the appealing safe forward invariant set. However, the provable safety relies on a precisely described dynamic model, which is not always available in practice. Recent works leverage learning to compensate model uncertainty for a CBF con...
-
Underactuation is a core challenge associated with controlling soft and continuum robots, which possess theoreti-cally infinite degrees of freedom, but few actuators. However, $m$ actuators may still be used to control a dynamic soft robot in an m-dimensional output task space. In this paper we develop a task-space control approach for planar continuum robots that is robust to modeling error and r...
-
Using soft actuators as legs, soft quadruped robots have shown great potential in traversing unstructured and complex terrains and environments. However, unlike rigid robots whose gaits can be generated using foot pattern design and kinematic model of the rigid legs, the gait generation of soft quadruped robots remains challenging due to the high DoFs of the soft actuators and the uncertain deform...
-
Soft actuators offer a safe, adaptable approach to tasks like gentle grasping and dexterous manipulation. Creating accurate models to control such systems however is challenging due to the complex physics of deformable materials. Accurate Finite Element Method (FEM) models incur prohibitive computational complexity for closed-loop use. Using a differentiable simulator is an attractive alternative,...
-
For Lifelong SLAM, one has to deal with temporary localization failures, e.g., induced by kidnapping. We achieve this by starting a new map and merging it with the previous map as soon as relocalization succeeds. Since relocalization methods are fallible, it can happen that such a merge is invalid, e.g., due to perceptual aliasing. To address this issue, we propose methods to detect and undo inval...
-
Simultaneous Localization and Mapping (SLAM) systems are fundamental building blocks for any autonomous robot navigating in unknown environments. The SLAM implementation heavily depends on the sensor modality employed on the mobile platform. For this reason, assumptions on the scene's structure are often made to maximize estimation accuracy. This paper presents a novel direct 3D SLAM pipeline that...
-
In this paper, we present a tightly-coupled visual-inertial object-level multi-instance dynamic SLAM system. Even in extremely dynamic scenes, it can robustly optimise for the camera pose, velocity, IMU biases and build a dense 3D reconstruction object-level map of the environment. Our system can robustly track and reconstruct the geometries of arbitrary objects, their semantics and motion by incr...
-
ACEFusion is the first 3D reconstruction system able to capture the geometry and semantics of dynamic scenes using an RGB-D camera in real-time on a robotic computing platform. Harnessing the hardware accelerators of an Nvidia Jetson AGX Xavier, the system uses heterogeneous computing to achieve 30 FPS under a 30W power budget. Using a data-parallel design, we perform most image computation on the...
-
Simultaneous localization and mapping (SLAM) based on laser sensors has been widely adopted by mobile robots and autonomous vehicles. These SLAM systems are required to support accurate localization with limited computational resources. In particular, point cloud registration, i.e., the process of matching and aligning multiple LiDAR scans collected at multiple locations in a global coordinate fra...
-
Cross-view image geolocalization provides an estimate of an agent's global position by matching a local ground image to an overhead satellite image without the need for GPS. It is challenging to reliably match a ground image to the correct satellite image since the images have significant viewpoint differences. Existing works have demonstrated localization in constrained scenarios over small areas...
-
Facial paralysis is a challenging condition that alters a patient's ability to express emotion and communicate. Restoring facial movements thus has crucial implications for the patients' quality of life. This publication introduces an approach for artificial muscles implementation targeting facial reanimation, as well as the challenges and limitations of the proposed strategy. The aim is to develo...
-
This paper presents the development and experimental evaluation of a portable haptic exoskeleton glove system designed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The proposed glove system involves force perception, linkage-driven finger mechanism, and personalized voice control to achieve various grasping functionality requirements. The fully ...
-
This paper presents the design and development of a novel, low-profile, exoskeleton robotic glove aimed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The key idea of this new glove lies in its new finger mechanism that takes advantage of the rigid coupling hybrid mechanism (RCHM) concept. This mechanism concept couples the motions of the adjacent...
-
As a traditional movement assist equipment for people with lower-limb dysfunction, the wheelchair can support and carry users to perform a long-distance movement indoor and outdoor, however, prolonged inactivity can lead to muscle atrophy and deteriorate motion functions. As a promising solution, the lower limb exoskeleton provides people the ability of standing and walking to avoid these problems...
-
Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands. In this work, we provide a flexible language-based...
-
The present paper proposes a novel reinforce-ment learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. Dream- erV2 is a cutting-edge model-based reinforcement learning from pixels that uses discrete world models to represent latent states with categorical variables. Dreaming is also a form of reinforcement learning from pixels that attempts to avoid t...
-
Ahstract- One of the key challenges in visual imitation learning is collecting large amounts of expert demonstrations for a given task. While methods for collecting human demonstrations are becoming easier with teleoperation methods and the use of low-cost assistive tools, we often still require 100–1000 demonstrations for every task to learn a visual representation and policy. To address this, we...
-
Learning representations that generalize across tasks and domains is challenging yet necessary for autonomous systems. Although task-driven approaches are appealing, de-signing models specific to each application can be difficult in the face of limited data, especially when dealing with highly variable multimodal input spaces arising from different tasks in different environments. We introduce the...
-
Learned knowledge graph representations supporting robots contain a wealth of domain knowledge that drives robot behavior. However, there does not exist an inference reconciliation framework that expresses how a knowledge graph representation affects a robot's sequential decision making. We use a pedagogical approach to explain the inferences of a learned, black-box knowledge graph representation,...
-
This paper proposes a new network for reconstructing multi-object point cloud. Different from previous networks which reconstruct multi-object point cloud as a whole, our network iteratively reconstructs each individual object point cloud from a frame of multi-object point cloud. To achieve this goal, we have designed MO-Transformer, a transformer-based autoregressive network. During training, MO-...
-
Self-supervised learning on point clouds has gained a lot of attention recently, since it addresses the label-efficiency and domain-gap problems on point cloud tasks. In this paper, we propose a novel self-supervised framework to learn informative features from partial point clouds. We leverage partial point clouds scanned by LiDAR that contain both content and pose attributes, and we show that di...
-
A key challenge in learning to perform manipulation tasks is selecting a suitable skill representation. While specific skill representations are often easier to learn, they are often only suitable for a narrow set of tasks. In most prior works, roboticists manually provide the robot with a suitable skill representation to use e.g. a neural network or DMPs. By contrast, we propose to allow the robo...
-
Laser-driven micro-tools are formulated by treating highly-focused laser beams as actuators, to control the tool's motion to contact then manipulate a micro object, which allows it to manipulate opaque micro objects, or large cells without causing photodamage. However, most existing laser-driven tools are limited to relatively simple tasks, such as moving and caging, and cannot carry out in-hand d...
-
This article proposes a method for the correction of angular deviations caused during the fixing process of samples prepared for Atomic Force Microscopy (AFM). The correction is done using the angular control of a 6-DOF PPPS parallel platform were the sample is placed, while the AFM scan is performed by a 3-DOF serial cartesian robot with a tuning fork probe designed to perform FM-AFM. The method ...
-
Sub-structures such as micro-structured magnetic teeth fabricated with an artificial bacteria flagellum (ABF) are designed for achieving more motion modes, higher precision, and better controllability. To achieve these, a more precise model considering the non-circular cross-sectional features is setup without simplifying the structure as a helical filament with a circular cross-section as having ...
-
Acoustic holography is a newly emerging and promising technique to dynamically generate arbitrary desired holographic acoustic field in 3D space for contactless robotic manipulation. The latest technology supporting complex dynamic holographic acoustic field reconstruction is through phased transducer array (PTA), where the phase profile of emitted acoustic wave from discrete transducers is contro...
-
The magnetic microrobot has become a promising approach in many biomedical applications due to its small volume, flexible motion, and untethered micromachines. The micro-chain robot is one of the most popular magnetic microrobots. However, the uncontrollable magnetic moment direction and quantity of the magnetic beads consisted in the existing self-assembled micro-chain robot limit their locomotio...
-
Pushing objects through cluttered scenes is a challenging task, especially when the objects to be pushed have initially unknown dynamics and touching other entities has to be avoided to reduce the risk of damage. In this paper, we approach this problem by applying deep reinforcement learning to generate pushing actions for a robotic manipulator acting on a planar surface where objects have to be p...
-
In this paper, we present a Domain Randomization and a Domain Adaptation approach to transfer experience for entanglement detection and separation from simulation into a real-world bin-picking application. We investigate the influence of different randomization options in image processing and use a CycleGAN as a further Domain Adaptation method to synthesize simulation data as realistically as pos...
-
Object rearranging is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a goal configuration. Previous studies focus on designing an expert system for each specific task by model-based or data-driven approaches and the application scenarios are therefore limited. Some research has been attempting to design a general framework to obtai...
-
Dynamic movements are ubiquitous in human motor behavior as they tend to be more efficient and can solve a broader range of skill domains than their quasi-static counterparts. For decades, robotic juggling tasks have been among the most frequently studied dynamic manipulation problems since the required dynamic dexterity can be scaled to arbitrarily high difficulty. However, successful approaches ...
-
Robot manipulation in cluttered environments of-ten requires complex and sequential rearrangement of multiple objects in order to achieve the desired reconfiguration of the target objects. Due to the sophisticated physical interactions involved in such scenarios, rearrangement-based manipulation is still limited to a small range of tasks and is especially vulnerable to physical uncertainties and p...
-
We propose a novel Parallel Monte Carlo tree search with Batched Simulations (PMBS) algorithm for accelerating long-horizon, episodic robotic planning tasks. Monte Carlo tree search (MCTS) is an effective heuristic search algorithm for solving episodic decision-making problems whose underlying search spaces are expansive. Leveraging a GPU-based large-scale simulator, PMBS introduces massive parall...
-
Recent advances in legged locomotion have en-abled quadrupeds to walk on challenging terrains. However, bipedal robots are inherently more unstable and hence it's harder to design walking controllers for them. In this work, we leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. Similar to existing works, we start with a base policy which ...
-
This paper presents a novel framework for point- foot biped running in three-dimensional space. The proposed approach generates center of mass (CoM) reference trajectories based on a differentially flat spring-loaded inverted pendulum (SLIP) model. A foothold planner is used to select touch down location that renders optimal CoM trajectory for upcoming step in real time. Dynamically feasible traje...
-
We consider the problem of controlling a bilateral leader-follower robotic surgical set-up to allow kinesthetic haptic feedback to the user when the instrument approaches a forbidden area like sensitive organs arteries or veins that should be protected from injuries during surgery. The leader is a haptic device while the follower is a general purpose manipulator holding an elongated tool with an a...
-
WORMESH is a unique robot concept inspired by flatworm locomotion and its key feature is the use of multiple traveling waves for locomotion. This paper presents the steering method for anisotropic module configuration (AMC) of WORMESH-II based on the kinematics of skid steering of mobile robots. AMC of WORMESH-II used two parallel pedal waves to generate locomotion. The kinematic model of WORMESH-...
-
This paper presents a novel method of realizing in-hand manipulation inspired by the peristaltic motion of a large-sized caterpillar. The sharp contrast between the proposed soft-bodied finger and the conventional hard/rigid robotic ones is peristaltic motion with compression and bending deformations. The design is based on the biological fact that large-size caterpillars (e.g., Bombyx mori) utili...
-
Undulation is the most common gait generated by legless creatures, which enables their robust and efficient locomotion in various environments. Such advantages inspired the control design of many kinds of locomotion robots. Despite their technical details, most of them realize the undulation gait via tracking predetermined trajectories called serpenoid curves, which are a group of sinusoidal wavef...
-
Enlightened by the creeping gait of natural snakes, snake-like robots swing joints side to side at similar tracks for generating propelling forces. However, it is not always essential to control all joints of a snake-like robot to realize the creeping gait. Therefore, in this paper, a creeping snake-like robot with partially actuated joints has been investigated, towards reducing the redundancy ca...
-
This paper presents a novel snake robot with the docking function, which can help the snake robots to connect with each other to achieve a stronger one with double length and double degrees of freedom. First, the mechanical design of the snake robot with docking function is introduced, including the body link and the head-tail passive docking mechanical structure. Second, the control system is bui...
-
Reinforcement learning commonly suffers from slow convergence speed and requires thousands of episodes, which makes it hard to be applied for physical robotic applications. Little research has been studied for snake robot control using RL because of the additional difficulty of high redundancy of freedom. Existing methods either adopts an asynchronous A3C structure or a joint state representation....
-
Climbing robots have the potential to revolutionize the maintenance and inspection operations of many types of vertical structures. In nature, parrots exhibit a remarkable capacity for manipulation during climbing behaviors, for which robotics can benefit from studying. In this paper we present a novel tripedal robot that is inspired by the morphology of these impressive birds, which use their leg...
-
For performing robotic manipulation tasks, the core problem is determining suitable trajectories that fulfill the task requirements. Various approaches to compute such trajectories exist, being learning and optimization the main driving techniques. Our work builds on the learning-from-demonstration (LfD) paradigm, where an expert demonstrates motions, and the robot learns to imitate them. However,...
-
Time-series data classification is central to the analysis and control of autonomous systems, such as robots and self-driving cars. Temporal logic-based learning algorithms have been proposed recently as classifiers of such data. However, current frameworks are either inaccurate for real-world applications, such as autonomous driving, or they generate long and complicated formulae that lack interp...
-
We develop a method for synthesizing control policies for stochastic, linear, time-varying systems that must perform tasks specified in signal temporal logic. We build upon an efficient, sampling-based framework that computes the probability of the system satisfying its specification. By exploiting the properties of linear systems and robustness score in temporal logic specifications, we obtain sa...
-
In this paper, we propose a preliminary definition and analysis of the novel concept of sensor observability index. The goal is to analyse and evaluate the performance of distributed directional or axial-based sensors to observe specific axes in task space as a function of joint configuration in serial robot manipulators. For example, joint torque sensors are often used in serial robot manipulator...
-
Mobility-on-demand systems are transforming the way we think about the transportation of people and goods. Most research effort has been placed on scalability issues for systems with a large number of agents and simple pickup/drop-off demands. In this paper, we consider fair multi-vehicle route planning with streams of complex, temporal logic transportation demands. We consider an approximately en...
-
We present an incremental topology-based motion planner that, while planning paths in the configuration space, performs metric gluing on the constructed Vietoris-Rips simplicial complex of each sub-space (voxel). By incrementally capturing topological and geometric information in batches of voxel graphs, our algorithm avoids the time overhead of analyzing the properties of the entire configuration...
-
We propose a novel hybrid system (both hardware and software) of an Unmanned Aerial Vehicle (UAV) carrying a miniature Unmanned Ground Vehicle (miniUGV) to perform a complex search and manipulation task. This system leverages the heterogeneous robots to accomplish a task that cannot be done using a single robot system. It enables the UAV to explore a hidden space with a narrow opening through whic...
-
This paper is interested in formalizing human trajectories in human-robot encounters. Inspired by robot navigation tasks in human-crowded environments, we consider the case where a human and a robot walk towards each other, and where humans have to avoid colliding with the incoming robot. Further, humans may describe different be-haviors, ranging from being in a hurry/minimizing completion time to...
-
Simultaneous Localization and Mapping (SLAM) has greatly assisted in exploring perceptually-degraded underground environments, such as human-made tunnels, mine tunnels, and caves. However, the recurring sensor failures and spurious loop closures in these scenes bring significant challenges to applying SLAM. This paper proposes an architecture for underground localization and semantic mapping (ULSM...
-
Loop closure detection is a key technology for long-term robot navigation in complex environments. In this paper, we present a global descriptor, named Normal Distribution Descriptor (NDD), for 3D point cloud loop closure detection. The descriptor encodes both the probability density score and entropy of a point cloud as the descriptor. We also propose a fast rotation alignment process and use cor...
-
In recent years, Visual-Inertial Odometry (VIO) has achieved many significant progresses. However, VIO meth-ods suffer from localization drift over long trajectories. In this paper, we propose a First-Estimates Jacobian Visual-Inertial-Ranging Odometry (FEJ-VIRO) to reduce the localization drifts of VIO by incorporating ultra-wideband (UWB) ranging measurements into the VIO framework consistently....
-
Visual SLAM methods based on point features have achieved acceptable results in texture-rich static scenes, but they often suffer from a deficiency of texture and the existence of dynamic objects in real indoor scenes, which limits the application of these methods. In this paper, we have presented DRG-SLAM, which combines line features and plane features into point features to improve the robustne...
-
We present a generic framework for scale-aware direct monocular odometry based on depth prediction from a deep neural network. In contrast with previous methods where depth information is only partially exploited, we formulate a novel depth prediction residual which allows us to incorporate multi-view depth information. In addition, we propose to use a truncated robust cost function which prevents...
-
2D SLAM is useful for mobile robots that are constrained to a 2D plane, for example in a warehouse, simplifying calculations in respect to the 3D case. The use of an IMU in such a context can enrich the estimation and make it more robust. In this paper we reformulate the IMU preintegration widely used in 3D problems for the 2D case, making use of Lie Theory. The Lie theory based formalization, fir...
-
The scale ambiguity problem is inherently unsolvable to monocular SLAM without the metric baseline between moving cameras. In this paper, we present a novel scale estimation approach based on an object-level SLAM system. To obtain the absolute scale of the reconstructed map, we formulate an optimization problem to make the scaled dimensions of objects conform to the distribution of their sizes in ...
-
Bin picking is a challenging problem in robotics due to high dimensional action space, partially visible objects, and contact-rich environments. State-of-the-art methods for bin picking are often simplified as planar manipulation, or learn policy based on human demonstration and motion primitives. The designs have escalated in complexity while still failing to reach the generality and robustness o...
-
Grasping in dense clutter is a fundamental skill for autonomous robots. However, the crowdedness and oc-clusions in the cluttered scenario cause significant difficul-ties to generate valid grasp poses without collisions, which results in low efficiency and high failure rates. To address these, we present a generic framework called GE-Grasp for robotic motion planning in dense clutter, where we lev...
-
Being able to grasp objects is a fundamental component of most robotic manipulation systems. In this paper, we present a new approach to simultaneously reconstruct a mesh and a dense grasp quality map of an object from a depth image. At the core of our approach is a novel camera-centric object representation called the “object shell” which is composed of an observed “entry image” and a predicted “...
-
Exploring the relationship among objects and giving the correct operation sequence is vital for robotic manipulation. However, most previous algorithms only model the relationship between pairs of objects independently, ignoring the interaction effect between them, which may generate redundant or missing relations in complex scenes, such as multi-object stacking and partial occlusion. To solve thi...
-
Picking a specific object from clutter is an essential component of many manipulation tasks. Partial observations often require the robot to collect additional views of the scene before attempting a grasp. This paper proposes a closed-loop next-best-view planner that drives exploration based on occluded object parts. By continuously predicting grasps from an up-to-date scene reconstruction, our po...
-
Among primates, the prehensile nature of the hand is vital for greater adaptability and a secure grip over the substrate/branches, particularly for arm-swinging motion or brachiation. Though various brachiation mechanisms that are mechanically equivalent to underactuated pendulum models are reported in the literature, not much attention has been given to the hand design that facilitates both locom...
-
Robotic manipulators navigating cluttered shelves or cabinets may find it challenging to avoid contact with obstacles. Indeed, rearranging obstacles may be necessary to access a target. Rather than planning explicit motions that place obstacles into a desired pose, we suggest allowing incidental contacts to rearrange obstacles while monitoring contacts for safety. Bypassing object identification, ...
-
PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations
Evolution Strategy (ES) algorithms have shown promising results in training complex robotic control policies due to their massive parallelism capability, simple implementation, effective parameter-space exploration, and fast training time. However, a key limitation of ES is its scalability to large capacity models, including modern neural network architectures. In this work, we develop Predictive ...
-
Robotic manipulation of cloth is a challenging task due to the high dimensionality of the configuration space and the complexity of dynamics affected by various material properties. The effect of complex dynamics is even more pronounced in dynamic folding, for example, when a square piece of fabric is folded in two by a single manipulator. To account for the complexity and uncertainties, feedback ...
-
Complexity, cost, and power requirements for the actuation of individual robots can play a large factor in limiting the size of robotic swarms. Here we present PCBot, a minimalist robot that can precisely move on an orbital shake table using a bi-stable solenoid actuator built directly into its PCB. This allows the actuator to be built as part of the automated PCB manufacturing process, greatly re...
-
Force estimation using neural networks is a promising approach to enable haptic feedback in minimally invasive surgical robots without end-effector force sensors. Various network architectures have been proposed, but none have been tested in real time with surgical-like manipulations. Thus, questions remain about the real-time transparency and stability of force feedback from neural network-based ...
-
We address the problem of enabling quadrupedal robots to perform precise shooting skills in the real world using reinforcement learning. Developing algorithms to enable a legged robot to shoot a soccer ball to a given target is a challenging problem that combines robot motion control and planning into one task. To solve this problem, we need to consider the dynamics limitation and motion stability...
-
Gesture recognition is a popular human-computer interaction technology, which has been widely applied in many fields (e.g., autonomous driving, medical care, VR and AR). However, 1) most existing gesture recognition methods focus on the fixed recognition scenarios with several gestures, which could lead to memory consumption and computational effort when continuously learning new gestures; 2) Mean...
-
Suction grippers are the most common pick-and-place end effectors used in industry. However, there is little literature on creating and validating models to predict their force interaction with objects in dynamic conditions. In this paper, we study the interaction dynamics of an active vacuum suction gripper during the vertical release of an object. Object and suction cup motions are recorded usin...
-
Robots that are supposed to interact with or manipulate objects in the world must be able to track the poses of objects in their sensor data. Thus, Detecting and tracking the 6-DoF poses of targeted objects is important for aerial manipulation and is still in the early stage due to the high dynamics and limited onboard capacity of such systems. In this paper, we propose ICK-Track, a novel method f...
-
Robots with multi-fingered grippers could perform advanced manipulation tasks for us if we were able to properly specify to them what to do. In this study, we take a step in that direction by making a robot grasp an object like a grasping demonstration performed by a human. We propose a novel optimization-based approach for transferring human grasp demonstrations to any multi-fingered grippers, wh...
-
Gravity based caging grasps are robotic grasps where the robot hand passively supports an object against gravity. When a robot hand supports an object at a local minimum of the object gravitational energy, the robot hand forms a basket like grasp of the object. Any object movement in a basket grasp requires an increase of the object gravitational energy, thus allowing secure object pickup and tran...
-
A popular solution for connecting different components in modern electronics, such as mobile phones, laptops, tablets, etc, is the use of flexible flat cables (FFC). Typically, it takes hours of repetition from a highly trained worker, or a high precision autonomous robot with specialised end effectors to reliably manage the installation of these cables. Human workers are prone to error, and canno...
-
Mobile manipulator throwing is a promising method to increase the flexibility and efficiency of dynamic manipulation in factories. Its major challenge is to efficiently plan a feasible throw under a wide set of task specifications. We show that the mobile manipulator throwing problem can be simplified to a planar problem, hence greatly reducing the computational costs. Using machine learning appro...
-
Quadruped robots are usually equipped with ad-ditional arms for manipulation, negatively impacting price and weight. On the other hand, the requirements of legged locomotion mean that the legs of such robots often possess the needed torque and precision to perform manipulation. In this paper, we present a novel design for a small-scale quadruped robot equipped with two leg-mounted manipulators ins...
-
This work addresses the problem of transporting an object along a desired planar trajectory by pushing with mobile robots. More specifically, we concentrate on establishing optimal contacts between the object and the robots to execute the given task with minimum effort. We present a task-oriented contact placement optimization strategy for object pushing that allows calculating optimal contact poi...
-
A kitchen assistant needs to operate human-scale objects, such as cabinets and ovens, in unmapped environments with dynamic obstacles. Autonomous interactions in such environments require integrating dexterous manipulation and fluid mobility. While mobile manipulators in different form factors provide an extended workspace, their real-world adoption has been limited. Executing a high-level task fo...
-
We present an open source motion planning framework for ROS, which uses constraint and optimization based task space control to generate trajectories for the whole body of mobile manipulators. Motion goals are defined as constraints which are enforced on task space functions. They map the controllable degrees of freedom of a system onto custom task spaces, which can, but do not have to be, the Car...
-
The gap between simulation and the real-world restrains many machine learning breakthroughs in computer vision and reinforcement learning from being applicable in the real world. In this work, we tackle this gap for the specific case of camera-based navigation, formulating it as following a visual cue in the foreground with arbitrary backgrounds. The visual cue in the foreground can often be simul...
-
While mobile navigation has been focused on obstacle avoidance, Navigation Among Movable Obstacles (NAMO) via interaction with the environment, is a problem that is still open and challenging. This paper, presents a novel system integration to handle NAMO using visual feedback. In order to explore the capabilities of our introduced system, we explore the solution of the problem via graph-based pat...
-
Underground environments are some of the most challenging for autonomous navigation. The long, featureless corridors, loose and slippery soils, bad illumination and unavailability of global localization make many traditional approaches struggle. In this work, a topological-based navigation system is presented that enables autonomous navigation of a ground robot in mine-like environments relying ex...
-
In the context of visual navigation, the capacity to map a novel environment is necessary for an agent to exploit its observation history in the considered place and efficiently reach known goals. This ability can be associated with spatial rea-soning, where an agent is able to perceive spatial relationships and regularities, and discover object characteristics. Recent work introduces learnable po...
-
When robots operate in real-world off-road environments with unstructured terrains, the ability to adapt their navigational policy is critical for effective and safe navigation. However, off-road terrains introduce several challenges to robot navigation, including dynamic obstacles and terrain uncertainty, leading to inefficient traversal or navigation failures. To address these challenges, we int...
-
Cyber-attacks, failures, and implementation errors inside the controller of an autonomous system can affect its correct behavior leading to unsafe states and degraded performance. In this paper, we focus on such problems specifically on cyber-attacks that manipulate controller parameters like the gains in a feedback controller or that triggers different behaviors or block inputs based on specific ...
-
Recent advances in deep reinforcement learning and scalable photorealistic simulation have led to increasingly mature embodied AI for various visual tasks, including navigation. However, while impressive progress has been made for teaching embodied agents to navigate static environments, much less progress has been made on more dynamic environments that may include moving pedestrians or movable ob...
-
Interestingness recognition is crucial for decision making in autonomous exploration for mobile robots. Previous methods proposed an unsupervised online learning approach that can adapt to environments and detect interesting scenes quickly, but lack the ability to adapt to human-informed interesting objects. To solve this problem, we introduce a human-interactive framework, AirInteraction, that ca...
-
Aerial Modular Reconfigurable Robots (AMRRs) are scalable systems consisting of rotor modules capable of rearrangement during flight. The potential to dynamically change any shape for a given task poses the question: what arrangements offer the most aerodynamic benefit for the task of flying? Answering this requires understanding how adjacent rotors in various configurations influence each another...
-
Precise Position Control of a Multi-rotor UAV with a Cable-suspended Mechanism During Water Sampling
This paper addresses the problem of water sampling by using a multirotor UAV with a cable-suspended mechanism. In order to ensure the safe execution of the sampling procedure and the stabilization of the vehicle, the disturbances, induced by the water flow and transferred through the cable, have to be identified. Specifically, an estimate of the disturbances is extracted by integrating a depth sen...
-
Pruning work at high altitude is a dangerous work with a high risk of accidents for human workers. In this research, we propose a multirotor flying robot that is equipped with a wire-suspended device and performs pruning task. We use a saber saw as a cutting tool. If the cutting tool is installed on the body of the multirotor platform, it is difficult for the flying robot to approach the desired w...
-
The article focuses on acquiring a 3D radiation map of a building via a two-phase survey performed with an unmanned aircraft system (UAS). First, a model of the stud-ied building is created by means of photogrammetry. Then, radiation data are collected using a 2-inch NaI(Tl) detector in a regular grid at a distance of 2 m from all accessible surfaces of the building (i.e., the walls and the roof)....
-
Drones have shown to be useful aerial vehicles for unmanned transport missions such as food and medical supply delivery. This can be leveraged to deliver life-saving nutrition and medicine for people in emergency situations. However, commercial drones can generally only carry 10 %–30 % of their own mass as payload, which limits the amount of food delivery in a single flight. One novel solution to ...
-
This work introduces a Nonlinear Model Predictive Control (N-MPC) for camera-equipped Unmanned Aerial Vehicles (UAVs), which controls at the motor level the UAV motion to ensure the quality of vision-based state estimation while performing other tasks. The controller ensures visibility over a sufficient amount of features, while optimizing their coverage, based on an assessment of the estimation q...
-
Due to the precise manipulations required in dental surgery, robotic technologies have been applied to dentistry. So far, most dental robots are designed for implant surgery, helping dentists accurately place the implant to the desired position and depth. This paper presents the DentiBot, the first robot designed for dental endodontic treatment. Without visual feedback, the DentiBot is integrated ...
-
To cope with the difficulty of training and eval-uation for nursing telerobot operator. This paper proposes a training-evaluation method for operator with unsupervised trajectory segmentation. To evaluate the dexterity and proce-dural knowledge of the operators objectively, we propose a new unsupervised model TSC-CRP that can automatically segment trajectory from nursing robotic training sessions....
-
In this study, we present a visual servo control framework for fully automated nasopharyngeal swab robots. The proposed framework incorporates a deep learning-based nostril detection with a cascade approach to reliably identify the nostrils with high accuracy in real time. In addition, a partitioned visual servoing scheme that combines image-based visual servoing with axial control is formulated f...
-
As the aging of society continues to accelerate, the number of elderly patients is increasing, as is the demand for manpower to care for them. In particular, there is an urgent need for bedridden patient care. However, limitations in the supply of human resources have caused an increase in the burden for care. In particular, nursing personnel often experience inconvenience and difficulties owing t...
-
Loss of upper extremity motor control and function is an unremitting symptom in post-stroke patients. This would impose hardships on accomplishing their daily life activities. Supernumerary robotic limbs (SRLs) were introduced as a solution to regain the lost Degrees of Freedom (DoFs) by introducing an independent new limb. The actuation systems in SRL can be categorized into rigid and soft actuat...
-
In this study, a mechanism that realizes a novel structural form of the harmonic reducer is introduced. Conventional robots often use various mechanical reducers owing to low torque and high-speed characteristics of electric motors. Among them, harmonic reducers are frequently used because of their compact size and backlash-free precision. The plate harmonic reducer which uses the same topological...
-
Compliance is good. However, it is challenging for one compliant continuum robot to finish both high precision manipulation and environmental-adapted motions. In this paper, a modular continuum robot with the alterable compliance characteristic is proposed. Besides, an actuation module is also proposed using a tubular-screw mechanism for non-slippage transmission. Kinematic analyses and dynamic co...
-
The recent trend towards low reduction gearing in robotic actuation has revitalised the need for high-performance gearing concepts. In this work we propose compact low-reduction cycloidal gearing, that is 3D-printable and combined with off-the-shelf components. This approach presents an enormous potential for high performance-to-cost implementations. After discussing parameter selection and design...
-
This contribution demonstrates the usage of fold-based joints to create a novel 3 DoF 3R(RPaR) parallel robot design. Multiple folding mechanisms are introduced, fulfilling the function of revolute, prismatic, and spherical joints. Folding mechanisms are here tested regarding their applicability in parallel kinematic robots taking advantage of beneficial properties such as increased stiffness, fla...
-
For universal robotic limbs, having a large workspace with high stiffness and adjustable output properties is important to adapt to various situations. A combination of parallel mechanisms that can change output characteristics is promising to meet these demands. As such, we propose a lever mechanism with double parallel link platforms. This mechanism is composed of a lever mechanism with the effo...
-
Impressive animal locomotion capabilities are mediated by the co-evolution of the skeletal morphology and muscular properties. Legged robot performance would also likely benefit from the co-optimization of actuators and leg morphology. However, development of custom actuators for legged robots is expensive and time consuming, discouraging application-specific actuator optimization. This paper pres...
-
In modern highly dynamic robot manipulation, collisions between a robot and objects may be intentionally executed to improve performance. To distinguish between these deliberate contacts and accidental collisions beyond the limit of state-of-the-art human-robot interactions, new sensing approaches are required. This work seeks an easy-to-implement and real-time capable solution to detect the ident...
-
Object detection plays an essential role in providing localization, path planning, and decision making capabilities in autonomous navigation systems. However, existing object detection models are trained and tested on a fixed number of known classes. This setting makes the object detection model difficult to generalize well in real-world road scenarios while encountering an unknown object. We addr...
-
Using Domain Randomized synthetic data for training deep learning systems is a promising approach for addressing the data and the labeling requirements for supervised techniques to bridge the gap between simulation and the real world. We propose a novel approach for generating and applying class-specific Domain Randomization textures by using randomly cropped image patches from real-world data. In...
-
Processing 3D data efficiently has always been a challenge. Spatial operations on large-scale point clouds, stored as sparse data, require extra cost. Attracted by the success of transformers, researchers are using multi-head attention for vision tasks. However, attention calculations in transformers come with quadratic complexity in the number of inputs and miss spatial intuition on sets like poi...
-
Transfer learning based approaches have recently achieved promising results on the few-shot detection task. These approaches however suffer from “catastrophic forgetting” issue due to finetuning of base detector, leading to sub-optimal performance on the base classes. Furthermore, the slow convergence rate of stochastic gradient descent (SGD) results in high latency and consequently restricts real...
-
As most existing works of single-view 3D reconstruction aim at learning the better mapping functions to directly transform the 2D observation into the corresponding 3D shape for achieving state-of-the-art performance, there often comes a potential concern on having the implicit bias towards the seen classes learnt in their models (i.e. reconstruction intertwined with the classification) thus leadi...
-
Segmenting object instances is a key task in machine perception, with safety-critical applications in robotics and autonomous driving. We introduce a novel approach to instance segmentation that jointly leverages measurements from multiple sensor modalities, such as cameras and LiDAR. Our method learns to predict embeddings for each pixel or point that give rise to a dense segmentation of the scen...
-
Semantic object mapping in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in ...
-
Designing a cost function for nonlinear model predictive control (MPC) with a sparse/binary stage cost is challenging. This paper proposes a novel MPC approach with a scheduled quadratic stage cost function that approximates the true stage cost in order to optimally control a nonlinear system with a sparse/binary stage cost. The cost function parameter is optimally scheduled by a parameter schedul...
-
Within the context of Robotic Minimally Invasive Surgery (R-MIS), we propose a novel linear model predictive controller formulation for the coordination of multiple autonomous robotic arms. The controller is synthesized by formulating a linear approximation of non-linear constraints, which allows the controller to be both computationally faster and better performing due to the increased prediction...
-
In this paper, we propose a new sampling-based path planning approach, focusing on the challenges linked to autonomous exploration. Our method relies on the definition of a disk graph of free-space bubbles, from which we derive a biased sampling function that expands the graph towards known free space for maximal navigability and frontiers discovery. The proposed method demonstrates an exploratory...
-
This paper introduces a new method for robot motion planning and navigation in uneven environments through a surfel representation of underlying point clouds. The proposed method addresses the shortcomings of state-of-the-art navigation methods by incorporating both kinematic and physical constraints of a robot with standard motion planning algorithms (e.g., those from the Open Motion Planning Lib...
-
Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the travers ability of the surrounding terrain, which depends on the hardware, locomotion c...
-
This work presents an online trajectory generation algorithm using a sinusoidal jerk profile. The generator takes initial acceleration, velocity and position as input, and plans a multi-segment trajectory to a goal position under jerk, acceleration, and velocity limits. By analyzing the critical constraints and conditions, the corresponding closed-form solution for the time factors and trajectory ...
-
Automated excavation is promising to improve the safety and efficiency of excavators, and trajectory planning is one of the most important techniques. In this paper, we propose a two-stage method that integrates data-driven imitation learning and model-based trajectory optimization to generate optimal trajectories for autonomous excavators. We firstly train a deep neural network using demonstratio...
-
Area coverage is crucial for robotics applications such as cleaning, painting, exploration, and inspections. Hinged reconfigurable robots have been introduced for these application domains to improve the area coverage performance. However, the existing coverage algorithms of hinged reconfigurable robots require improvements in the aspects; consideration of beyond a limited set of reconfigurable sh...
-
With the Autonomous Vehicle (AV) industry shifting towards machine-learned approaches for motion plan-ning [1], the performance of self-driving systems is starting to rely heavily on large quantities of expert driving demon-strations. However, collecting this demonstration data typically involves expensive HD sensor suites (LiDAR + RADAR + cameras), which quickly becomes financially infeasible at ...
-
Informative path planning is an important and challenging problem in robotics that remains to be solved in a manner that allows for wide-spread implementation and real-world practical adoption. Among various reasons for this, one is the lack of approaches that allow for informative path planning in high-dimensional spaces and non-trivial sensor constraints. In this work we present a sampling-based...
-
While the musculoskeletal humanoid has various biomimetic benefits, its complex modeling is difficult, and many learning control methods have been developed. However, for the actual robot, the hysteresis of its joint angle tracking is still an obstacle, and realizing target posture quickly and accurately has been difficult. Therefore, we develop a feedback control method considering the hysteresis...
-
In this study, seated walk, a movement of walking while sitting on a chair with casters, is realized on a musculoskeletal humanoid from human teaching. The body is balanced by using buttock-contact sensors implemented on the planar interskeletal structure of the human mimetic musculoskeletal robot. Also, we develop a constrained teaching method in which one-dimensional control command, its transit...
-
Control of an articulated spine is important for humanoids' dynamic and balanced motion. Although there have been many spinal structures for humanoids, their actuation is still limited due to the usage of geared motors for joints. This paper introduces position control of a distributed electrome-chanical spine in a vertical plane. The spine dynamics model is approximated as an open chain. Gravitat...
-
A surface microstructure that mimics the surface of a gecko's foot can exert a large gripping force with a small contact force. If such a structure is applied to the fingertips of a two-fingered parallel gripper, stable grasping can be achieved independent of the wetting and frictional state of the contact surface. However, the adhesive force of the microstructure is large while releasing the obje...
-
The musculoskeletal humanoid is difficult to modelize due to the flexibility and redundancy of its body, whose state can change over time, and so balance control of its legs is challenging. There are some cases where ordinary PID controls may cause instability. In this study, to solve these problems, we propose a method of learning a correlation model among the joint angle, muscle tension, and mus...
-
Actively searching for targets using a multi-agent system in an unknown environment poses a two-pronged prob-lem, where on the one hand we need agents to cover as much of the environment as possible and on the other have a higher density of agents where there are potential targets to maximize detection performance. This paper proposes a fully distributed solution for an ad hoc network of agents to...
-
This paper presents a method for Bayesian multi-robot peer-to-peer data fusion where any pair of autonomous robots hold non-identical, but overlapping parts of a global joint probability distribution, representing real world inference tasks (e.g., mapping, tracking). It is shown that in dynamic stochastic systems, filtering, which corresponds to marginalization of past variables, results in direct...
-
Multi-robot systems have been shown to build large-scale, user-specified structures using distributed, environmentally-mediated coordination in simulation. Little attention, however, has been devoted to error propagation and mitigation. In this paper, we introduce a detailed simulation of TERMES, a prototypical construction system, in which robots have realistic error profiles. We use this simulat...
-
We present Perceive-Represent-Generate (PRG), a novel three-stage framework that maps perceptual information of different modalities (e.g., visual or sound), corresponding to a series of instructions, to a sequence of movements to be executed by a robot. In the first stage, we perceive and preprocess the given inputs, isolating individual commands from the complete instruction provided by a human ...
-
SMA-NBO: A Sequential Multi-Agent Planning with Nominal Belief-State Optimization in Target Tracking
In target tracking with mobile multi-sensor sys-tems, sensor deployment impacts the observation capabilities and the resulting state estimation quality. Based on a partially observable Markov decision process (POMDP) formulation comprised of the observable sensor dynamics, unobservable target states, and accompanying observation laws, we present a distributed information-driven solution approach t...
-
Several template models have been developed to facilitate the analysis of limit-cycles for quadrupedal locomotion. The parameters in the model are usually fixed; however, biology shows that animals change their leg stiffness according to the locomotion velocity, and this adaptability invariably affects the stability of the gait. This paper provides an analysis of the influence of this variable leg...
-
Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its environment. There are several challenges in solving embodied multimodal tasks, including long-horizon planning, vision-and-language grounding, and efficient exploration. We focus on a critical bottleneck, namely the perform...
-
There has been a plethora of work towards im-proving robot perception and navigation, yet their application in hazardous environments, like during a fire or an earthquake, is still at a nascent stage. We hypothesize two key challenges here: first, it is difficult to replicate such scenarios in the real world, which is necessary for training and testing purposes. Second, current systems are not ful...
-
The robotic flexible endoscope is developed rapidly in the field of surgery robots due to its high flexibility and safety. However, some inherent features, e.g., high nonlinearity, material creep, complex dynamic hysteresis behaviors, and the unknown coupling effects between bending and twisting motions, can lead to the significant degradation on three-dimensional (3-D) positioning performance of ...
-
Structural intervention cardiology (SIC) interventions are crucial procedures for correcting heart valves, walls, and muscle form defects. However, the possibility of embolization or perforation, as well as the lack of transparent vision and autonomous surgical equipment, make it difficult for the clinician. This paper proposes a robot-assisted tendon-driven catheter and machine learning-based pat...
-
Magnetically-actuated flexible endoscopes (MAFE) have been well used in minimally-invasive surgery because they can be steered by a magnetic field thus more flexible than traditional endoscopes. Model-free and uncalibrated visual-feedback control makes it possible to manipulate MAFE with a magnetic field without external tracking systems. Because no extra sensor is required to obtain position and ...
-
Existing robotic endoscopes for laparoscopic surgery, predominantly rigid or limited in dexterity, occupy a large motion space1, The large occupied motion space necessitates large incisions and reduces the motion space for surgeons to simultaneously operate other surgical instruments. Meanwhile, surgeons only have limited view adjustment capability to avoid occlusion and they often have to lift/pu...
-
Minimally invasive surgeries (MIS) or natural orifice transluminal endoscopic surgeries (NOTES) such as the transurethral resection of bladder tumor (TURBT) require the surgical robot to be miniaturized to perform surgical procedures in confined spaces. However, the surgical robot's tiny size poses problems in its fabrication and shape sensing. In this paper, a miniature continuum surgical robot i...
-
Effectively disassembling and recovering materials from waste electrical and electronic equipment (WEEE) is a critical step in moving global supply chains from carbon-intensive, mined materials to recycled and renewable ones. Conventional recycling processes rely on shredding and sorting waste streams, but for WEEE, which is comprised of numerous dissimilar materials, we explore targeted disassemb...
-
In typical data-based grasping methods, a grasp based on parallel-jaw grippers is parameterized by the center of the gripper, the rotation angle, and the gripper opening width so as to predict the quality and pose of grasps at every pixel. In contrast, a grasp is represented using only two contact points for contact-points-based grasp representation, which allows for fusion with tactile sensors mo...
-
Instance segmentation is an important task for supporting robotic grasping in auto-store scenarios. Accurate segmentation usually relies on the quantity and quality of available annotated training data. However, it requires tremendous cost to obtain these labels. In this work, without requiring any human annotations on real data, our proposed self-ensembling sim-to-real network, namely SESR, is ab...
-
The accuracy and robustness of visual odometry (VO) is significantly affected by the high dynamic range (HDR) environments, because traditional cameras have a limited dynamic range and inevitably miss information in both overexposed and underexposed areas. To overcome the above challenge, we use an spatially varying exposure (SVE) camera, which captures four images with different exposure levels s...
-
In this paper, we present a relative pose estimation algorithm based on lines knowing the vertical direction associated to each image. We demonstrate that a closed-form solution requiring only eight lines between three views is possible. As a linear solution, it is shown that our approach outperforms the standard trifocal estimation based on 13 triplets of lines and can be efficiently inserted int...
-
Fast, robust, and flexible part feeding is essential for enabling automation of low volume, high variance assembly tasks. An actuated vision-based solution on a traditional vibratory feeder, referred to here as a vision trap, should in principle be able to meet these demands for a wide range of parts. However, in practice, the flexibility of such a trap is limited as an expert is needed to both id...
-
Communication between robots and the server is a major problem for cloud robotic systems. In this paper, we address the problem caused by data loss during such communications and propose an efficient buffering algorithm, called AFR, to solve the problem. We model the problem into an optimization problem to maximize the received Quantity of Information (QoI). Our AFR algorithm is formally proved to...
-
Wood logs picking is a challenging task to automate. Indeed, logs usually come in cluttered configurations, randomly orientated and overlapping. Recent work on log picking automation usually assume that the logs' pose is known, with little consideration given to the actual perception problem. In this paper, we squarely address the latter, using a data-driven approach. First, we introduce a novel d...
-
Forest canopies are the biggest habitat for terrestrial life, yet our understanding of environmental processes and biodiversity inside the canopy continues to be limited due to labour and resource intensive data collection. Existing aerial and climbing robots also struggle to access these complex environments, while animals easily navigate them using multiple means of locomotion. Following this in...
-
For the development of muscle-machine interfaces (MuMIs), researchers have relied mainly on Electromyography (EMG) signals. However, these signals require complex hardware systems, as well as specialized signal processing and feature extraction methods. To overcome these issues, in our previous work, we proposed a novel MuMI for decoding human intention and motion, called Lightmyography (LMG). To ...
-
Dialogue State Tracking (DST) is an important part in task-oriented dialog system, whose target is to infer the current dialog states and user intentions according to the dialog history information. To this end, we have achieved improvements to the existing work and proposed a dynamic network model suitable for multi-domain dialog, which can explicitly use domain information and better cope with z...
-
Conversation can play an essential role in forging bonds between humans and social robots, but participants need to feel like they are being listened to, remembered, and cared about in order to effectively build rapport. In this paper, we propose a novel strategy for conducting small talk with a social robot. Our approach is known as the Tiers of Friendship. It is centered around three core design...
-
Robots that use natural language in collaborative tasks must refer to objects in their environment. Recent work has shown the utility of the linguistic theory of the Givenness Hierarchy (GH) in generating appropriate referring forms. But before referring expression generation, collaborative robots must determine the content and structure of a sequence of utterances, a task known as document planni...
-
Going beyond the traditional sparse multi-channel peripheral human-machine interface that has been used widely in neurorobotics, high-density surface electromyography (HD-sEMG) has shown significant potential for decoding upper-limb motor control. We have recently proposed heterogeneous temporal dilation of LSTM in a deep neural network architecture for a large number of gestures (>60), securing s...
-
This paper discusses the tele-operation system for multiple mobile manipulators. If a single person could freely tele-operate multiple mobile manipulators simultaneously, it would be a great step toward the goal of “avatar-symbiotic society” allowing people to live beyond the constraints of their bodies, space, and time. At present, however, such a tele-operation system has not been developed. The...
-
In recent years, haptic telemanipulation has been introduced to control robots remotely with an input device that generates force feedback. Compliant control strategies are needed to ensure safe interaction between humans and robots. Accurate and precise manipulation requires a stiff setup of the impedance parameters, while safety demands for low stiffness. This paper proposes an impedance-based c...
-
This work proposes a novel haptic data reduction scheme for time-delayed teleoperation by coding information as blocks. State-of-the-art (SOTA) haptic data reduction approaches are mainly sampled-based schemes. They encode haptic signals sample by sample in order to minimize the introduced coding delay. In contrast, our proposed block-based coding approach transmits a sample block as a single unit...
-
Advanced wireless communication networks provide lower latency and a higher transmission rate. Although this is an enabler for many new teleoperation applications, the risk of network instability or packet drop is still unavoidable. Real-time manipulator teleoperation requires data transmission with no discontinuity. Shared autonomy (SA) is a standard method to mitigate this issue. In this way, if...
-
Model Predictive Control (MPC) schemes have proven their efficiency in controlling high degree-of-freedom (DoF) complex robotic systems. However, they come at a high computational cost and an update rate of about tens of hertz. This relatively slow update rate hinders the possibility of stable haptic teleoperation of such systems since the slow feedback loops can cause instabilities and loss of tr...
-
The teleoperation of mobile manipulators may pose significant challenges, demanding complex interfaces and causing a substantial burden to the human operator due to the need to switch continuously from the manipulation of the arm to the control of the mobile platform. Hence, several works have considered to exploit shared control techniques to overcome this issue and, in general, to facilitate the...
-
We present a proprioceptive teleoperation system that uses a reflexive grasping algorithm to enhance the speed and robustness of pick-and-place tasks. The system consists of two manipulators that use quasi-direct-drive actuation to provide highly transparent force feedback. The end-effector has bimodal force sensors that measure 3-axis force information and 2-dimensional contact location. This inf...
-
Motion tracking interfaces are intuitive for free-form teleoperation tasks. However, efficient manipulation control can be difficult with such interfaces because of issues like the interference of unintended motions and the limited precision of human motion control. The limitation in control efficiency reduces the operator's performance and increases their workload and frustration during robot tel...
-
Accurate LiDAR-camera extrinsic calibration is a precondition for many multi-sensor systems in mobile robots. Most calibration methods rely on laborious manual operations and calibration targets. While working online, the calibration methods should be able to extract information from the environment to construct the cross-modal data association. Convolutional neural networks (CNNs) have powerful f...
-
The goal of this paper is to develop an ultrasonic detection device that can be mounted on an underwater snake vehicle (USV) for underwater district heating pipe (DHP) detection in the future. Ultrasonic detection technology (UDT) is the detection means used, and the cavitation defects in polyurethane (PUR) layer of DHPs are the object being detected. Due to the large thickness of PUR layer and th...
-
Robot identification is a prolific topic that has a long history with results spanning recent decades. Recent years have witnessed a renew of interest in this problem due in part to a rapid increase in robotic hardware platforms capable of accurate model-based control. The most popular methods exploit the fact that the inverse dynamic model is linear to the dynamic parameters. Because we identify ...
-
In this paper, we present TEScalib, a novel extrinsic self-calibration approach of LiDAR and stereo camera using the geometric and photometric information of surrounding environments without any calibration targets for automated driving vehicles. Since LiDAR and stereo camera are widely used for sensor data fusion on automated driving vehicles, their extrinsic calibration is highly important. Howe...
-
2D laser range-finders and depth-cameras are usually equipped on service robots. But there are rarely calibration methods of them. This paper proposes an extrinsic calibration method of a 2D laser range-finder and a depth-camera using an orthogonal trihedron. The trihedron with orthogonal assumptions is taken as a reference frame to roughly estimate the relative pose between the sensors by solving...
-
Wearable robots have been developed to aid or substitute the gait locomotion of humans. To assist gait locomotion based on the intention of a wearer, a gait pattern analysis is required with a wearable sensor by measuring body information, i.e., a joint angular velocity. However, measuring a precise joint angular velocity is difficult because the attachment position of a sensor has a curvature and...
-
System modeling and parameter identification of micro aerial vehicles (MAV) are crucial for robust autonomy, especially under highly dynamic motions. Visual-inertial-aided online parameter identification has recently seen research attention due to the demanding of adaptation to platform configuration changes with minimal onboard sensor requirements. To this end, we design an online MAV system iden...
-
For an autonomous vehicle, the ability to sense its surroundings and to build an overall representation of the environment by fusing different sensor data streams is fundamental. To this end, the poses of all sensors need to be accurately determined. Traditional calibration methods are based on: 1) using targets specifically designed for calibration purposes in controlled environments, 2) optimizi...
-
Modern autonomous vehicles (AVs) often rely on vision, LIDAR, and even radar-based simultaneous localization and mapping (SLAM) frameworks for precise localization and navigation. However, modern SLAM frameworks often lead to unacceptably high levels of drift (i.e., localization error) when AVs observe few visually distinct features or encounter occlusions due to dynamic obstacles. This paper argu...
-
Navigation through unknown, cluttered environments is a fundamental and challenging task for autonomous vehicles as they must deal with a myriad of obstacle configurations typically unknown a priori. Challenges arise because obstacles of unknown shapes and dimensions can create occlusions limiting sensor field of view and leading to uncertainty in motion planning. In this paper we propose to lever...
-
Quadrotors are agile platforms. With human experts, they can perform extremely high-speed flights in cluttered environments. However, fully autonomous flight at high speed remains a significant challenge. In this work, we propose a motion planning algorithm based on the corridor-constrained minimum control effort trajectory optimization (MINCO) framework. Specifically, we use a series of overlappi...
-
Robust maritime obstacle detection is essential for fully autonomous unmanned surface vehicles (USVs). The currently widely adopted segmentation-based obstacle detection methods are prone to misclassification of object reflections and sun glitter as obstacles, producing many false positive detections, effectively rendering the methods impractical for USV navigation. However, water-turbulence-induc...
-
In the absence of an absolute positioning system, such as GPS, autonomous vehicles are subject to accumu-lation of positional error which can interfere with reliable performance. Improved navigational accuracy without GPS enables vehicles to achieve a higher degree of autonomy and reliability, both in terms of decision making and safety. This paper details the use of two navigation systems for aut...
-
While automated research vehicles are already populating the roads, their commercial availability at scale is still to come. Presumably, one of the key challenges is to derive behaviors that are safe and comfortable but at the same time not overcautious, despite considerable uncertainties. These uncertainties stem from imperfect perception, occlusions and limited sensor range, but also from the un...
-
In this work, we investigate the problem of incre-mentally solving constrained non-linear optimization problems formulated as factor graphs. Prior incremental solvers were either restricted to the unconstrained case or required periodic batch relinearizations of the objective and constraints which are expensive and detract from the online nature of the algorithm. We present InCOpt, an Augmented La...
-
We propose a new SLAM system that uses the semantic segmentation of objects and structures in the scene. Semantic information is relevant as it contains high level information which may make SLAM more accurate and robust. Our contribution is twofold: i) A new SLAM system based on ORB-SLAM2 that creates a semantic map made of clusters of points corresponding to objects instances and structures in t...
-
Perceptual Aliasing is one of the main problems in simultaneous localization and mapping (SLAM). Wrong associations between different places may lead to failure of the whole map. Research on structure information is rarely investigated among existing solutions to this problem. In cases of visual SLAM without sensors, such as LiDAR or Inertial Measurement Unit (IMU), structure information can rarel...
-
Simultaneous localization and mapping (SLAM) plays a vital role in mapping unknown spaces and aiding autonomous navigation. Virtually all state-of-the-art solutions today for 2D SLAM are designed for dense and accurate sensors such as laser range-finders (LiDARs). However, these sensors are not suitable for resource-limited nano robots, which become increasingly capable and ubiquitous nowadays, an...
-
Simultaneous localization and mapping (SLAM) is a critical capability in autonomous navigation, but in order to scale SLAM to the setting of “lifelong” SLAM, particularly under memory or computation constraints, a robot must be able to determine what information should be retained and what can safely be forgotten. In graph-based SLAM, the number of edges (measurements) in a pose graph determines b...
-
In this paper we present BOEM-SLAM, a backend for visual-inertial SLAM systems capable of creating a globally consistent trajectory and map without retaining the entire history of data. By leveraging the hidden Markov model structure, BOEM-SLAM can summarize historical data into sufficient statistics and then discard it. As a data-efficient algorithm, BOEM-SLAM addresses the growing computational ...
-
During auscultation, patients in difficult age often feel embarrassed and uncomfortable when exposing their chests to doctors of the different gender and being touched physically by doctors. We assume that an auscultation with robot technology can address the aforementioned gender-related issue. Toward eliminating gender bias during auscultation exam, this paper proposes a robotic platform which e...
-
The magnetic medical microrobots are influenced by diverse factors such as the medium, the geometry of the microrobot, and the imaging procedure. It is worth noting that the size limitations make it difficult or even impossible to obtain reliable physical properties of the system. In this research, to achieve a precise microrobot control using minimum knowledge about the system, an Adaptive Fuzzy ...
-
This paper presents a multi-modal model-based reinforcement learning (MBRL) approach to the excavation of fragmented rocks, which are very challenging to model due to their highly variable sizes and geometries, and visual occlusions. A multi-modal recurrent neural network (RNN) learns the dynamics of bucket-terrain interaction from a small physical dataset, with a discrete set of motion primitives...
-
We investigate how employing model learning methods in concert with model predictive control (MPC) can be used to automate obstacle reduction to mitigate risks to Combat Engineers operating construction equipment in an active battlefield. We focus on the task of earthen berm removal using a bladed vehicle. We introduce a novel data-driven formulation for earthmoving dynamics that enables predictio...
-
Remotely controlled hydraulically driven robots are expected to play an important role in extreme environments such as disaster sites, and force feedback is effective for improving the fidelity of the remote environment and the work efficiency. However, it is not reasonable to attach a force sensor directly to the end-point of a hydraulically driven robot. In a previous study, the authors showed t...
-
This paper investigates using a sampling-based approach, the RRT*, to reconfigure a 2D set of connected tiles in complex environments, where multiple obstacles might be present. Since the target application is automated building of discrete, cellular structures using mobile robots, there are constraints that determine what tiles can be picked up and where they can be dropped off during reconfigura...
-
Mobile 3D Printing (M3DP), using printing-in-motion, is a powerful paradigm for automated construction. A mobile robot, equipped with its own power, materials and an arm-mounted extruder, simultaneously navigates and creates its environment. Such systems can be highly scalable, parallelizable and flexible. However, planning and controlling the motion of the arm and base at the same time is challen...
-
A large-scale dump truck that automatically transports earth and sand in cooperation with a human-operated backhoe is of interest to the construction industry. A human-operated dump truck generally drives slightly past the desired loading position and then backs up to it for loading the sediment. The turning and loading positions are subjectively decided according to the working posture of the bac...
-
Autonomous navigation of a robot in agricultural fields is essential for every task from crop monitoring to weed management and fertilizer application. Many current approaches rely on accurate GPS, however, such technology is expensive and can be impacted by lack of coverage. As such, autonomous navigation through sensors that can interpret their environment (such as cameras) is important to achie...
-
Accurate kinodynamic models play a crucial role in many robotics applications such as off-road navigation and high-speed driving. Many state-of-the-art approaches for learning stochastic kinodynamic models, however, require precise measurements of robot states as labeled input/output examples, which can be hard to obtain in outdoor settings due to limited sensor capabilities and the absence of gro...
-
Recognizing objects in dense clutter accurately plays an important role to a wide variety of robotic manipulation tasks including grasping, packing, rearranging and many others. However, conventional visual recognition models usually miss objects because of the significant occlusion among instances and causes incorrect prediction due to the visual ambiguity with the high object crowdedness. In thi...
-
Visual Place Recognition (VPR) has become an indispensable capacity for mobile robots to operate in large-scale environments. Existing methods in this field mostly focus on exploring high-performance encoding strategies, while few attempts are devoted to lightweight models that balance per-formance and computational cost. In this work, we propose a Lightweight Self-attentional Distillation Network...
-
Place recognition is key to Simultaneous Localization and Mapping (SLAM) and spatial perception. However, a place recognition in the wild often suffers from erroneous predictions due to image variations, e.g., changing viewpoints and street appearance. Integrating uncertainty estimation into the life cycle of place recognition is a promising method to mitigate the impact of variations on place rec...
-
Multi-view-based 3D object recognition is important in robot-environment interaction. However, recent methods simply extract features from each view via convolutional neural networks (CNNs) and then fuse these features together to make predictions. These methods ignore the inherent ambiguities of each view caused due to 3D-2D projection. To address this problem, we propose a novel deep framework f...
-
This study proposes a dual-camera system for indoor high magnification surveillance which is capable of achieving always-in-focus and non-delay gaze control based on high-speed vision. The users are enabled to move the mouse freely on the wide-view screen while observing its in-focal zoom-in monitoring video in real-time. The proposed system consists of a wide-angle camera for wide-view and a Galv...
-
Spatiotemporal maps are data-driven estimates of time changing phenomena. For environmental science, rather than collect data from an array of static sensors, a mobile sensor platform could reduce setup time and cost, maintain flexibility to be deployed to any area of interest, and provide active feedback during observations. While promising, mapping is challenging with mobile sensors because vehi...
-
Quantifying the safety of the human body ori-entation is an important issue in human-robot interaction. Knowing the changing physical constraints on human motion can improve inspection of safe human motions and bring essential information about stability and normality of human body orientations with real-time risk assessment. Also, this information can be used in cooperative robots and monitoring ...
-
Autonomous robots should operate in real-world dynamic environments and collaborate with humans in tight spaces. A key component for allowing robots to leave structured lab and manufacturing settings is their ability to evaluate online and real-time collisions with the world around them. Distance-based constraints are fundamental for enabling robots to plan their actions and act safely, protecting...
-
This paper presents an application of human-drone interaction (HDI) for inventory management in a ware-house 4.0 that aims at improving the operators' safety and well-being together with increasing efficiency and reducing production costs. In our work, the speed and separation monitoring (SSM) methodology is applied for the first time to HDI, in analogy to the human-robot interaction (HRI) ISO saf...
-
In order to transform a robot into an intelligent machine it needs to be enabled to react to unforeseen events (most importantly collisions) during task execution and have a plan on how to continue the task afterwards. This requires a flexible operational framework that allows to define adaptive reactions and interactions with the motion generation and task planning stage. Within this work we firs...
-
In physical human-robot interaction (pHRI) enabled by admittance control, delay-induced oscillations arising from both the neuromuscular time-delays of the human and electromechanical delays of the robot can cause unsafe instability in the system. This study presents and evaluates rate-limiting as a means to overcome such instability, and provides a new perspective on how rate-limiting can benefit...
-
Many collaborative human-robot tasks require the robot to stay safe and work efficiently around humans. Since the robot can only stay safe with respect to its own model of the human, we want the robot to learn a good model of the human in order to act both safely and efficiently. This paper studies methods that enable a robot to safely explore the space of a human-robot system to improve the robot...
-
A robot must comply with very restrictive safety standards in close human-robot collaboration applications. These standards limit the robot's performance because of speed reductions to avoid potentially large forces exerted on humans during collisions. On-robot capacitive proximity sensors (CPS) can serve as a solution to allow higher speeds and thus better productivity. They allow early reactive ...
-
This paper presents a gait controller for bipedal robots to achieve highly agile walking over various terrains given local slope and friction cone information. Without these considerations, untimely impacts can cause a robot to trip and inadequate tangential reaction forces at the stance foot can cause slippages. We address these challenges by combining, in a novel manner, a model based on an Angu...
-
In this article, we propose a deep learning frame-work that provides a unified approach to the problem of leg contact detection in humanoid robot walking gaits. Our formulation accomplishes to accurately and robustly estimate the contact state probability for each leg (i.e., stable or slip/no contact). The proposed framework employs solely propriocep-tive sensing and although it relies on simulate...
-
This paper presents a novel control approach, based on the interconnection and damping-assignment passivity-based control (IDA-PBC), to achieve stable and periodic walking for underactuated planar biped robots with one degree of underactuation. The system's physical structure is preserved by assigning a target port-Hamiltonian dynamics to the closed-loop system, which also ensures passivity. The c...
-
In this work, we propose a learning approach for 3D dynamic bipedal walking when footsteps are constrained to stepping stones. While recent work has shown progress on this problem, real-world demonstrations have been limited to relatively simple open-loop, perception-free scenarios. Our main contribution is a more advanced learning approach that enables real-world demonstrations, using the Cassie ...
-
Maintaining balance to external pushes is one of the most important features for a humanoid to walk in a real environment. In particular, methods for counteracting to pushes using the centroidal angular momentum (CAM) control have been actively developed. In this paper, a CAM control scheme based on hierarchical quadratic programming (HQP) is proposed. The scheme of the CAM control consists of CAM...
-
We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep locatio...
-
This paper studies the effects of replacing pin-joint knees in passive dynamic bipedal walkers with biomimetic four-bar knees. The kinetic model of the four-bar knees is presented in detail, and an analytical model of the passive walking dynamics is derived. The resulting four-bar kneed biped is compared with a pin-joint kneed walker, for their passive walking performance. The geometry of the four...
Introduction
Conference IROS2022 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 997