DeepNLP ICRA2021 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
For a companion robot that follows a person as an assistant, predicting human walking is important to produce a proactive movement that is helpful to maintain an appropriate area decided by the human personal space. However, fully trusting the prediction may result in obstructing human walking because it is not always accurate. Hence, we consider the estimation of uncertainty (i.e., entropy) of th...
-
We present a path planning framework for marine robots subject to uncertain ocean currents that exploits data from ensemble forecasting, which is a technique for current prediction used in oceanography. Ensemble forecasts represent a distribution of predicted currents as a set of flow fields that are considered to be equally likely. We show that the typical approach of computing the vector-wise me...
-
The implementation of optimization-based motion coordination approaches in real world multi-agent systems remains challenging due to their high computational complexity and potential deadlocks. This paper presents a distributed model predictive control (MPC) approach based on convex feasible set (CFS) algorithm for multi-vehicle motion coordination in autonomous driving. By using CFS to convexify ...
-
Modern navigation algorithms based on deep reinforcement learning (RL) show promising efficiency and robustness. However, most deep RL algorithms operate in a risk-neutral manner, making no special attempt to shield users from relatively rare but serious outcomes, even if such shielding might cause little loss of performance. Furthermore, such algorithms typically make no provisions to ensure safe...
-
A method for planning and controlling the somersault motion of a quadruped robot is proposed in this paper. The method divides the somersault motion into 5 stages according to intuitive understanding. Based on the simplified dynamic model, the linear programming method is used to obtain the maximum ground reaction force under the constraints of joint torque and friction cone, and then the optimal ...
-
Multi-segment continuum robots, that offer inherent compliance and distal dexterity, are suitable for deployment in minimally invasive surgical procedures. Cable-driven mechanism is commonly used in continuum surgical robots but could lead to inter-segment motion coupling in a multi-segment robot. In this paper, we present a coupled mechanics model for a two-segment notched continuum robot to anal...
-
In this paper, we develop a novel visual-inertial navigation system with motion constraint (VINS-Motion), which extends the visual-inertial navigation system (VINS) to incorporate vehicle motion constraints for improving the autonomous vehicles localization accuracy. Besides the prior information, IMU measurement residual, and visual measurement residual utilized in VINS, vehicle orientation/veloc...
-
A computed tomography (CT)-guided robotic needle requires registration to transfer coordinates between the robot and CT image for accurate insertion. In our previous work, we proposed a geometric marker that allows direct registration between a CT image and robot and demonstrated its proof of concept. In this paper, we present a registration algorithm for calculating the six-degrees-of-freedom err...
-
Shape sensors are important for safer and more dexterous manipulation of the medical catheters. Among the electromagnetic based shape sensors, a voice coil shape sensor measures the variation of a mutual inductance between the coils placed along the tube due to the bending of the tube. Owing to the design flexibility of a voice coil, it offers the small size without the external magnetic field gen...
-
Additive Manufacturing, has evolved beyond prototyping to manufacturing end-products. The authors are involved in developing a large-scale extrusion-based 3D printer to print mining equipment - a Gravity Separation Spiral, and embedding sensors to monitor the operational conditions re-motely. This paper presents a temperature-compensated strain sensor that can be 3D printed inline within large-sca...
-
The recovery of autonomous underwater vehicles (AUVs) has been a challenging mission due to the limited localization accuracy and movement capability of the AUVs. To overcome these limitations, we propose a novel design of a deployable underwater robot (DUR) for the recovery mission. Utilizing the origami structure, the DUR can transform between open and closed states to maximize the performance a...
-
In this paper we present a methodology for optimising the design of a metamaterial structure with one degree of freedom that is able to simultaneously bend and stretch. The structure is intended for assisting flexion-extension of the wrist joint. The metamaterial is comprised of serially connected, individually designed cells. The design parameters can be chosen to optimally fit a desired planar c...
-
In this paper, we present a low-damping active knee prosthesis (LDKP) with low noise and high backdrivability. The proposed prosthesis is driven by a motor and then decelerated by a four-stage synchronous belt. High backdrivability given by this structure accelerates the prosthetic response. A control system containing several sensors are embedded in the proposed prosthesis to recognize different ...
-
End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from a common limitation, lacking the ability to recover from an out-of-distribution state caused by compounding errors. In this paper, instead of using tactile feedback or explicitly detecting the failure through vision, we...
-
In this study, we present a method to grasp diverse unseen real-world objects using an off-policy actor-critic deep reinforcement learning (RL) with the help of a simulation and the use of as little real-world data as possible. Actor-critic deep RL is unstable and difficult to tune when a raw image is given as an input. Therefore, we use state representation learning (SRL) to make actor-critic RL ...
-
Manipulation in a densely cluttered environment creates complex challenges in perception to close the control loop, many of which are due to the sophisticated physical interaction between the environment and the manipulator. Drawing from biological sensory-motor control, to handle the task in such a scenario, tactile sensing can be used to provide an additional dimension of the rich contact inform...
-
Maintaining a map online is resource-consuming while a robust navigation system usually needs environment abstraction via a well-fused map. In this paper, we propose a mapless local planner which directly conducts such abstraction on the unfused sensor data. A limited-memory data structure with a reliable proximity query algorithm is proposed for maintaining raw historical information. A sampling-...
-
Robots are usually equipped with cameras to explore the indoor scene and it is expected that the robot can well describe the scene with natural language. Although some great success has been achieved in image and video captioning technology, especially on many public datasets, the caption generated from indoor scene video is still not informative and coherent enough. In this paper, we propose the ...
-
Recently proposed DNN-based stereo matching methods that learn priors directly from data are known to suffer a drastic drop in accuracy in new environments. Although supervised approaches with ground truth disparity maps often work well, collecting them in each deployment environment is cumbersome and costly. For this reason, many unsupervised domain adaptation methods based on image-to-image tran...
-
Justifying operational decisions for robots is a challenging task as the operator or the robot itself has to understand the underlying physical interaction between the robot and the environment to predict the potential outcome. It is desirable to understand how the decision influences the operational performance in the way of causal relationship for the purpose of explainable decision-making. Here...
-
HueCode: A Meta-marker Exposing Relative Pose and Additional Information in Different Colored Layers
In this paper, HueCode, a meta-marker that robustly and simultaneously exposes the relative pose between a marker and a camera along with additional information, is proposed. It occupies the area of a single marker by overlaying multiple types of markers in different colored layers. Using perspective information from the first (most recognizable) type of element marker, the second or higher marker...
-
Purely vision-based localization and mapping is a cost-effective and thus attractive solution to localization and mapping on smart ground vehicles. However, the accuracy and especially robustness of vision-only solutions remain rivalled by more expensive, lidar-based multi-sensor alternatives. We show that a significant increase in robustness can be achieved if taking non-holonomic kinematic const...
-
We present a tightly-coupled multi-sensor fusion architecture for autonomous vehicle applications, which achieves centimetre-level accuracy and high robustness in various scenarios. In order to realize robust and accurate point-cloud feature matching we propose a novel method for extracting structural, highly discriminative features from LiDAR point clouds. For high frequency motion prediction and...
-
Straight line features have been increasingly utilized in visual SLAM and 3D reconstruction systems. The straight lines’ parameterization, parallel constraint, and coplanar constraint are studied in many recent works. In this paper, we explore the novel intersection constraint of straight lines for structure reconstruction. First, a minimum parameterized representation of ray-point-ray (RPR) struc...
-
Deep reinforcement learning (DRL) has been demonstrated to provide promising results in several challenging decision making and control tasks. However, the required inference costs of deep neural networks (DNNs) could prevent DRL from being applied to mobile robots which cannot afford high energy-consuming computations. To enable DRL methods to be affordable in such energy-limited platforms, we pr...
-
Model-based deep reinforcement learning has achieved success in various domains that require high sample efficiencies, such as Go and robotics. However, there are some remaining issues, such as planning efficient explorations to learn more accurate dynamic models, evaluating the uncertainty of the learned models, and more rational utilization of models. To mitigate these issues, we present MEEE, a...
-
In the present paper, we propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. Dreamer is a sample- and cost-efficient solution to robot learning, as it is used to train latent state-space models based on a variational autoencoder and to conduct policy optimization by latent trajectory imagination. However, this autoencoding based appr...
-
Probabilistic regression techniques in control and robotics applications have to fulfill different criteria of data-driven adaptability, computational efficiency, scalability to high dimensions, and the capacity to deal with different modalities in the data. Classical regressors usually fulfill only a subset of these properties. In this work, we extend seminal work on Bayesian nonparametric mixtur...
-
Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we intro...
-
Navigation through dynamic pedestrian environments in a socially compliant manner is still a challenging task for autonomous vehicles. Classical methods usually lead to unnatural vehicle behaviours for pedestrian navigation due to the difficulty in modeling social conventions mathematically. This paper presents an end-to-end path planning system that achieves autonomous navigation in dynamic envir...
-
State-of-the-art reinforcement learning (RL) algorithms suffer from high sample complexity, particularly in the sparse reward case. A popular strategy for mitigating this problem is to learn control policies by imitating a set of expert demonstrations. The drawback of such approaches is that an expert needs to produce demonstrations, which may be costly in practice. To address this shortcoming, we...
-
This paper studies push recovery for humanoid robots based on a variable-height inverted pendulum (VHIP) model. We first develop an approach for treating zero-step capturability of the VHIP with a novel methodology based on Hamilton-Jacobi (HJ) reachability analysis. Such an approach uses the sub-zero level set of a value function to encode capturability of the VHIP, where the value function is ob...
-
In this paper, we propose a meaningful definition of rotational centroidal orientation which is somewhat missed in the state-of-the-art centroidal momentum and dynamics theory for locomotion robots with one floating base. This centroidal instantaneous orientation rotates as the robot runs, and it is extracted from the total system angular inertia. The new centroidal frame is proposed to be paralle...
-
To design a general-purpose assembly robot system that can handle objects of various shapes, we propose a soft jig that fits to the shapes of assembly parts. The functionality of the soft jig is based on a jamming gripper developed in the field of soft robotics. The soft jig has a bag covered with a malleable silicone membrane, which has high friction, elongation, and contraction rates for keeping...
-
In physical human-robot collaboration (pHRC), singularity avoidance strategies are often critical to obtain stable interaction dynamics. It is hypothesised a predictable singularity avoidance strategy is preferred in pHRC as humans tend to maximise predictability when using complex systems. By using an electroencephalogram (EEG), it is possible to assess the predictability of a task through a feat...
-
A human can socially interact in a non-verbal manner by understanding the intention behind a tactile stimulus. Patting on one’s back is one of tactile communications, which is considered as a sign of encouragement in most cultures. The majority of such tactile communication is carried out by a dynamic tactile on large passive body parts and differently interpreted by how and where on the body is t...
-
Motion prediction of multiple agents in a dynamic scene is a crucial component in many real applications, including intelligent monitoring and autonomous driving. Due to the complex interactions among the agents and their interactions with the surrounding scene, accurate trajectory prediction is still a great challenge. In this paper, we propose a new method for robust trajectory prediction of mul...
-
A method that enables an industrial robot to accomplish the peg-in-hole task for holes in concrete is proposed. The proposed method involves slightly detaching the peg from the wall, when moving between search positions, to avoid the negative influence of the concrete’s high friction coefficient. It uses a deep neural network (DNN), trained via reinforcement learning, to effectively find holes wit...
-
We present a novel task planner - TaskNet for an autonomous excavator based on a data-driven method, which plans feasible task-level sequence by learning from demonstration data. Given a high-level excavation objective, our TaskNet planner can decompose it into sub-tasks, each of which can be further decomposed into task primitives with specifications. We train our TaskNet using an excavation trac...
-
A well known and well-studied feature of boats’ dynamic is the effect of steering-induced roll. This property is used by a technique called Rudder Roll Stabilization (RRS) to stabilise ships in waves in order to make the navigation safer and more pleasant. This technique is based on the generation of induced roll. Because of its specific application, studies have been limited to commercial vessels...
-
Food packing industry workers typically pick a target amount of food by hand from a food tray and place them in containers. Since menus are diverse and change frequently, robots must adapt and learn to handle new foods in a short time-span. Learning to grasp a specific amount of granular food requires a large training dataset, which is challenging to collect reasonably quickly. In this study, we p...
-
Recently, tactile sensing has attracted great interest for robotic manipulation. Predicting if a grasp will be stable or not, i.e. if the grasped object will drop out of the gripper while being lifted, can aid robust robotic grasping. Previous methods paid equal attention to all regions of the tactile data matrix or all time-steps in the tactile sequence, which may include irrelevant or redundant ...
-
In this paper, we present a spherical magnetic joint for the inverted locomotion of a multi-legged robot. The permanent magnet’s spherical shape allows the robot to attach its foot to a steel surface without energy consumption. However, the robot’s inverted locomotion requires foot flexibility for placement and gait construction of the robot. Therefore, the spherical magnetic joint mechanism was d...
-
This paper presents a new open-source mechanical design of a 6-DOF anthropomorphic ALARIS robotic hand that can serve as a low-cost design platform for further customization and utilization for research and educational purposes. The presented hand design employs linkage-based three-phalange finger and two-phalange adaptive thumb designs with non-backdrivable worm-and-rack transmission mechanisms. ...
-
We have implemented a force-based operational space controller on a physical musculoskeletal humanoid robot arm. The controller calculates muscle activations based on a biomimetic Hill-type muscle model. We propose a method to include the joint torque nullspace in the optimization process, which enables the robot to exploit the nullspace to gradually lower its overall muscle activation. We have ve...
-
After years of development, various swarms of robots have been proposed for many complicated tasks, such as forming patterns, cooperative locomotion, and adapting to different environments. However, controlling microrobotic swarms is still a challenging task owing to the lacking of integrated devices on the small-scale agents, and actuation of multiple microrobotic swarms to different targets unde...
-
In the context of robotic high-precision soldering, we propose an image-based pin alignment control method based on active plastic deformation. The plastic deformation is a well-known failure mechanism in most situations, which includes a phenomenon that the objects do not return original state. Here, in contrast to this convention, we utilize the plastic deformation of the metal pin to do pin ali...
-
Recent advances in deep reinforcement learning (deep RL) enable researchers to solve challenging control problems, from simulated environments to real-world robotic tasks. However, deep RL algorithms are known to be sensitive to the problem formulation, including observation spaces, action spaces, and reward functions. There exist numerous choices for observation spaces but they are often designed...
-
The "Thinking, Fast and Slow" paradigm of Kahneman proposes that we use two different styles of thinking—a fast and intuitive System 1 for certain tasks, along with a slower but more analytical System 2 for others. While the idea of using this two-system style of thinking is gaining popularity in AI and robotics, our work considers how to interleave the two styles of decision-making, i.e., how Sys...
-
Learning the uncertain dynamical environments for online learning and prediction from noisy sensory measurement streams is essential for various tasks in robotics. Recently, Gaussian process (GP) online learning such as an infinite-horizon Gaussian process (IHGP) has shown effectiveness to cope with non-stationary dynamical random processes in learning hyperparameters online by reducing the comput...
-
It is ubiquitously accepted that during the autonomous navigation of the quadrotors, one of the most widely adopted unmanned aerial vehicles (UAVs), safety always has the highest priority. However, it is observed that the ego airflow disturbance can be a significant adverse factor during flights, causing potential safety issues, especially in narrow and confined indoor environments. Therefore, we ...
-
This article proposes a new method that enables Unmanned Aerial Vehicles (UAVs) to actively find targets and shoot photographs of them in an unknown environment, while successfully avoiding surrounding obstacles and planning optimize routes. Owing to the limited computing ability on the UAVs, we obtained the point cloud data of surrounding objects, and selected the best segmentation method of the ...
-
The quadrotor is popularly used in challenging environments due to its superior agility and flexibility. In these scenarios, trajectory planning plays a vital role in generating safe motions to avoid obstacles while ensuring flight smoothness. Although many works on quadrotor planning have been proposed, a research gap exists in incorporating self-adaptation into a planning framework to enable a d...
-
A novel 3-dimensional (3-D) alignment method for point-cloud registration is proposed where the time-differential information of the measured points is employed. The new problem turns out to be a novel multi-dimensional optimization. Analytical solution to this optimization is then obtained, which sets the ground of further correspondence matching using k-D trees. Finally, via many examples, we sh...
-
This paper addresses the safe path planning problem for autonomous mobility with multi-modal perception uncertainties. Specifically, we assume that different sensor inputs lead to different Gaussian process regulated perception uncertainties (named as multi-modal perception uncertainties). We implement a Bayesian inference algorithm, which merges the multi-modal GP-regulated uncertainties into a u...
-
Moving in dynamic pedestrian environments is one of the important requirements for autonomous mobile robots. We present a model-based reinforcement learning approach for robots to navigate through crowded environments. The navigation policy is trained with both real interaction data from multi-agent simulation and virtual data from a deep transition model that predicts the evolution of surrounding...
-
In this paper, we present a novel approach that predicts spatially and temporally crowd behaviour for robotic social navigation. Integrating mobile robots into human society involves the fundamental problem of navigation in crowds. A robot should attempt to navigate in a way that is minimally invasive to the humans in its environment. However, planning in a dynamic environment is difficult as the ...
-
In swarm control, many robots coordinate their actions in a distributed and decentralized way. We propose a consensus-based control barrier function (CCBF) for a swarm. CCBF restricts the states of the whole distributed system, not just those of the individual robots. The barrier function is approximated by a consensus filter. We prove that CCBF constrains the control inputs for holding the forwar...
-
Scenarios requiring humans to choose from multiple seemingly optimal actions are commonplace, however standard imitation learning often fails to capture this behavior. Instead, an over-reliance on replicating expert actions induces inflexible and unstable policies, leading to poor generalizability in an application. To address the problem, this paper presents the first imitation learning framework...
-
This paper presents a novel robot-environment interaction in navigation tasks such that robots have neither a representation of their working space nor planning function, instead, an active environment takes charge of these aspects. This is realized by spatially deploying computing units, called cells, and making cells manage traffic in their respective physical region. Different from stigmegic ap...
-
In this paper, we explore a multi-agent reinforcement learning approach to address the design problem of communication and control strategies for multi-agent cooperative transport. Typical end-to-end deep neural network policies may be insufficient for covering communication and control; these methods cannot decide the timing of communication and can only work with fixed-rate communications. There...
-
We propose a game-theoretic multi-robot task allocation framework that enables a large team of robots to optimally allocate tasks in dynamically changing environments. As our main contribution, we design a decision-making algorithm that defines how the robots select tasks to perform and how they repeatedly revise their task selections in response to changes in the environment. Our convergence anal...
-
Heterogeneous multi-robot systems are advantageous for operations in unknown environments because functionally specialised robots can gather environmental information, while others perform tasks. We de ne this decomposition as the scout–task robot architecture and show how it avoids the need to explicitly balance exploration and exploitation by permitting the system to do both simultaneously. The ...
-
The Patrolling Problem is a crucial feature of the surveillance task in defense and other establishments. Most of the works in the literature concentrate on reducing the Idleness value at each location in the environment. However, there are often a few prioritized locations that cannot be left unvisited beyond a certain Time Period. In this paper, we study the problem of Prioritized patrolling - t...
-
Critical for the coexistence of humans and robots in dynamic environments is the capability for agents to understand each other’s actions, and anticipate their movements. This paper presents Stochastic Process Anticipatory Navigation (SPAN), a framework that enables nonholonomic robots to navigate in environments with crowds, while anticipating and accounting for the motion patterns of pedestrians...
-
Real-time lower limbs motion or gait measurement is an important part in human-robotic interaction for the control of robotic walkers and rehabilitation devices. Laser range finder or infrared sensor that is mounted on the device has been widely used in applications. Although these sensors can provide accurate horizontal motion information of lower limbs during human walking, it is still difficult...
-
This paper presents c2g-HOF networks which learn to generate cost-to-go functions for manipulator motion planning. The c2g-HOF architecture consists of a cost-to-go function over the configuration space represented as a neural network (c2g-network) as well as a Higher Order Function (HOF) network which outputs the weights of the c2g-network for a given input workspace. Both networks are trained en...
-
Nowadays, various algorithms based on the Rapidly-exploring Random Tree (RRT) methods are utilized to solve motion planning problems. Based on the RRT*, we developed a novel reconnection method that enables the planner to directly generate a smooth curved trajectory. Meanwhile, kinodynamic constraints of the robots are considered to generate the control input, which improves the feasibility of the...
-
We propose a new optimization-based task and motion planning (TAMP) with signal temporal logic (STL) specifications for robotic sequential manipulation such as pick-and-place tasks. Given a high-level task specification, the TAMP problem is to plan a trajectory that satisfies the specification. This is, however, a challenging problem due to the difficulty of combining continuous motion planning an...
-
The recent remarkable progress of deep reinforcement learning (DRL) stands on regularization of policy for stable and efficient learning. A popular method, named proximal policy optimization (PPO), has been introduced for this purpose. PPO clips density ratio of the latest and baseline policies with a threshold, while its minimization target is unclear. As another problem of PPO, the symmetric thr...
-
This work considers the optimal non-revisiting coverage tasks with a single non-redundant manipulator for the case when the object can be positioned at a predefined set of locations within the workcell. The scenario is often encountered in typical industrial settings, for instance when the object presents itself along a conveyor belt and its surface can not be serviced at a single location - the o...
-
This paper presents a search-based partial motion planner for generating feasible trajectories of car-like robots in highly dynamic environments. The planner searches for smooth, safe, and near-time-optimal trajectories by exploring a state graph built on motion primitives. To enable fast online planning, we propose an efficient path searching algorithm based on the aggregation and pruning of moti...
-
This paper introduces a novel task-space decomposed motion planning framework for multi-robot simultaneous locomotion and manipulation. When several manipulators hold an object, closed-chain kinematic constraints are formed, and it will make the motion planning problems challenging by inducing lower-dimensional singularities. Unfortunately, the constrained manifold will be even more complicated wh...
-
Efficient recharging is an essential requirement for autonomous mobile robots. In an indoor robotic application, charging stations can be installed offline. However, frequent trips to the charging stations cause inefficiency in the performance of the mobile robots. In an outdoor environment, a charging station cannot even be installed easily. We propose a framework and algorithms for enabling a gr...
-
In this paper, we proposed a novel path replanning algorithm on arbitrary graphs. To avoid computationally heavy preprocessing and to reduce required memory to store the expanded vertices of the previous search, we defined the feature vertices, which are extracted from the previous path by a simple algorithm to compare the costs between adjacent vertices along the path once. Proper additional heur...
-
For quadrotor trajectory planning, describing a polynomial trajectory through coefficients and end-derivatives both enjoy their own convenience in energy minimization. We name them double descriptions of polynomial trajectories. The transformation between them, causing most of the inefficiency and instability, is formally analyzed in this paper. Leveraging its analytic structure, we design a linea...
-
Self-healing function is a promising approach for damage management of high-load robot applications such as legged robots. Although the function is getting major in soft robotics, its application to life-sized "stiff" robots is of relatively minor interest. Although the authors have devised several self-healing tensile modules for tendon-driven robots, the design guideline to satisfy the large loa...
-
We introduce an origami-reinforced parallel continuum robot which is capable of maintaining the orientation of the end effector regardless of the bending shape. The cross-routing tendons provide an effective actuation of the robot because the constant length of the backbones prevent the actuation of parallel arrangement of tendons. We utilise the arclength relationship of parallel curves to show t...
-
This paper proposes a coupled tendon-driven waist joint for humanoid robots. The waist joint was designed as a 3 degrees of freedom (DOF) structure to simulate the motion of a human waist. The power transmission was designed by adopting a 3-motor 3-DOF (3M3D) coupled tendon-driven mechanism, so that the torque on the joints was multiplied. We derived the torque transmission formula and the rotatio...
-
Our goal is to investigate different approaches to modulate stiffness and apply them to human-robot interaction. Here we report on our effort employing the concept of adjustable unsupported-length cantilever leaf spring, which has been previously applied to different designs of variable stiffness actuators. By transmitting the interaction force through the elastic component directly to the support...
-
A vehicle is expected to exhibit omni-directional locomotion capability to provide improved rough terrain vehicle functionality. Generally, rough terrain vehicles are not holonomic and cannot travel in the lateral direction, whereas typical omni-directional vehicles have difficulty in traveling on rough terrain. This paper proposes installing a crank leg for a Mecanum-wheeled vehicle ("Mecanum cra...
-
In this work, we propose an approach to manipulate objects by position-controlled robot hands: in-hand caging manipulation. In this method, an object is manipulated based on caging without force sensing or force control. An object is caged by a robot hand throughout manipulation, and we can locate the object around a goal by deformation of the cage without sensing the object configuration. In this...
-
A novel control scheme for biped robots to manipulate the ZMP three-dimensionally apart from the actual ground profile is presented. It is shown that the linear inverted-pendulum-like dynamics with this scheme can represent a wider class of movements including variation of the body height. Moreover, this can also represent the motion in aerial phase. Based on this, the foot-guided controller propo...
-
Robotic audition is a basic sense that helps robots perceive the surroundings and interact with humans. Sound Source Localization (SSL) is an essential module for a robotic system. However, the performance of most sound source localization techniques degrades in noisy and reverberant environments due to inaccurate Time Difference of Arrival (TDoA) estimation. In robotic sound source localization, ...
-
Global navigation satellite systems (GNSS) are one of the utterly popular sources for providing globally referenced positioning for autonomous systems. However, the performance of the GNSS positioning is significantly challenged in urban canyons, due to the signal reflection and blockage from buildings. Given the fact that the GNSS measurements are highly environmentally dependent and time-correla...
-
Visual localization plays an indispensable role in robotics. Both learning and hand-crafted feature based methods for relocalization process keep their effectiveness and weakness. However, current algorithms seldom consider these two kinds of features under one framework. In this paper, focusing on this task, we propose a novel relocalization framework for RGB or RGB-D data source, which is compos...
-
Probabilistic volumetric mapping (PVM) represents a 3D environmental map for an autonomous robotic navigational task. A popular implementation such as Octomap is widely used in the robotics community for such a purpose. The Octomap relies on an octree to represent a PVM and its main bottleneck lies in massive ray-shooting to determine the occupancy of the underlying volumetric voxel grids.In this ...
-
Indoor positioning without GPS is a challenge task, especially, in complex scenes or when sensors fail. In this paper, we develop an ultra-wideband aided visual-inertial positioning system (UVIP) which aims to achieve accurate and robust positioning results in complex indoor environments. To this end, a point-line-based stereo visual-inertial odometry (PL-sVIO) is firstly designed to improve the p...
-
Initial global localization is important to mobile robotics in terms of navigation initialization (or re-initialization) and loop closure in SLAM. 3D LiDARs are commonly used for mobile robotics, yet LiDAR-based initial global localization (especially at large scale such as in outdoor environments) is still challenging due to lack of salient features in LiDAR range data. Inspired by visual SLAM or...
-
LiDAR odometry algorithms are complex and involve a number of hyper-parameters. The choice of hyper-parameters can substantively affect the performance of odometry estimation, and it is necessary to carefully fine-tune the hyper-parameters depending on the sensor, environment, and algorithm to achieve the best estimation results. While odometry estimation algorithms are often tuned manually, this ...
-
Place Recognition enables the estimation of a globally consistent map and trajectory by providing non-local constraints in Simultaneous Localisation and Mapping (SLAM). This paper presents Locus, a novel place recognition method using 3D LiDAR point clouds in large-scale environments. We propose a method for extracting and encoding topological and temporal information related to components in a sc...
-
This paper proposes an automatic and accuracy- enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects from massive point clouds. To obviate user interventions, we propose an automated planar board detec...
-
Human–robot interaction (HRI) has been widely researched in diverse applications. A robot following a person is one such scenario investigated in the HRI field. However, human movements and actions are complex and can change dramatically. We herein demonstrate a machine learning-based system that allows a person-following robot to track in real-time the predicted future motion of a walking human, ...
-
A human-centered robot needs to reason about the cognitive limitation and potential irrationality of its human partner to achieve seamless interactions. This paper proposes an anytime game-theoretic planner that integrates iterative reasoning models, a partially observable Markov decision process, and chance-constrained Monte-Carlo belief tree search for robot behavioral planning. Our planner enab...
-
As robots begin to collaborate with people in real life, safety needs to be rigorously ensured to reliably employ robots nearby. In addition to collision prevention algorithms, studies are being actively conducted on collision handling methods. Momentum Observer (MOB) was developed to estimate disturbance torque without using joint acceleration. However, the estimated disturbance from MOB contains...
-
Teleoperation of robots can be challenging, especially for novice users with little to no experience at such tasks. The difficulty is largely due to the numerous degrees of freedom users must control and their limited perception bandwidth. To help mitigate these challenges, we propose in this paper a solution which relies on artificial intelligence to understand user intended motion and then on mi...
-
Bottom-up approaches for image-based multi-person pose estimation consist of two stages: (1) keypoint detection and (2) grouping of the detected keypoints to form person instances. Current grouping approaches rely on learned embedding from only visual features that completely ignore the spatial configuration of human poses. In this work, we formulate the grouping task as a graph partitioning probl...
-
The purpose of this paper is the robotic hanging manipulation of an object of various shapes that is not limited to a specific category. To achieve this, we propose a method that allows the estimator to learn many different shapes with hanging points without any manual annotation. A random shape generator using GAN solves the limitation of the number of 3D models and can handle objects of various ...
-
Modular self-reconfigurable robotic (MSRR) systems are potentially more robust and more adaptive than conventional systems. Following our previous work where we proposed a freeform MSRR module called FreeBOT, this paper presents a novel configuration detection system for FreeBOT using a magnetic sensor array. A FreeBOT module can be connected by up to 11 modules, and the proposed configuration det...
-
In this paper, we present a diabolo model that can be used for training agents in simulation to play diabolo, as well as running it on a real dual robot arm system. We first derive an analytical model of the diabolo-string system and compare its accuracy using data recorded via motion capture, which we release as a public dataset of skilled play with diabolos of different dynamics. We show that ou...
-
A technological revolution is occurring in the field of robotics with the data-driven deep learning technology. However, building datasets for each local robot is laborious. Meanwhile, data islands between local robots make data unable to be utilized collaboratively. To address this issue, the work presents Peer-Assisted Robotic Learning (PARL) in robotics, which is inspired by the peer-assisted l...
-
This paper introduces a leg-wheel transformable quadruped robot named Lywal which can switch to the leg-mode and the wheel-mode for locomotion, and the claw-mode for picking up and transport functions. First, the mechanical structure of Lywal is designed by using an innovative 2-DoF transformable mechanism. Second, the calculation of kinematics is analyzed in detail. Then, the switching-mode strat...
-
Staircase is a typical obstacle for the legged robot to overcome in buildings. This paper studies the stair climbing capability-based dimensional synthesis for a hexapod legged robot, i.e., exploring how to determine the leg length and the longitudinal body length concerning the target staircase in the mechanical design stage. In climbing a staircase, leg-staircase interference is one of the predo...
-
Stable walking in real-world environments is a challenging task for humanoid robots, especially when considering the dynamic disturbances, e.g., caused by external perturbations that may be encountered during locomotion. The varying nature of disturbance necessitates high adaptability. In this paper, we propose an enhanced Nonlinear Model Predictive Control (NMPC) approach for robust and adaptable...
-
This paper presents a method for LiDAR sensor-based traversability analysis for autonomous mobile robots in urban environments. Although urban environments are structured environments, a typical terrain comprises hazardous regions for mobile robots. Therefore, a reliable method for detecting traversable regions is required to prevent robots from getting stuck in the middle of the road. Conventiona...
-
Proactive human-robot interaction (HRI) allows the receptionist robots to actively greet people and offer services based on vision, which has been found to improve acceptability and customer satisfaction. Existing approaches are either based on multi-stage decision processes or based on end-to-end decision models. However, the rule-based approaches require sedulous expert efforts and only handle m...
-
We present a method to estimate two-dimensional, time-invariant oceanic flow fields based on data from both ensemble forecasts and online measurements. Our method produces a realistic estimate in a computationally efficient manner suitable for use in marine robotics for path planning and related applications. We use kernel methods and singular value decomposition to find a compact model of the ens...
-
Quadrupedal robots are skillful at locomotion tasks while lacking manipulation skills, not to mention dexterous manipulation abilities. Inspired by the animal behavior and the duality between multi-legged locomotion and multi-fingered manipulation, we showcase a circus ball challenge on a quadrupedal robot, ANYmal. We employ a model-free reinforcement learning approach to train a deep policy that ...
-
Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the int...
-
An effective understanding of the contextual environment and accurate motion forecasting of surrounding agents is crucial for the development of autonomous vehicles and social mobile robots. This task is challenging since the behavior of an autonomous agent is not only affected by its own intention, but also by the static environment and surrounding dynamically interacting agents. Previous works f...
-
In visual robot self-localization, graph-based scene representation and matching have recently attracted research interest as robust and discriminative methods for self-localization. Although effective, their computational and storage costs do not scale well to large-size environments. To alleviate this problem, we formulate self-localization as a graph classification problem and attempt to use th...
-
Efficient SE(3) Reachability Map Generation via Interplanar Integration of Intra-planar Convolutions
Convolution has been used for fast computation of reachability maps, but it has high computational costs when performing SE(3) convolution operations for general joint arrangements in industrial robots and 3D workspace. Its application is also limited to planar robots, 2D workspace, or robots with special spatial arrangements for joints. In this paper, we find that the SE(3) convolution can be dec...
-
Colorectal cancer incidence has been steadily rising worldwide. Magnetic colonoscopes provide new approaches to conduct colon inspection and treatment. This paper presents a novel electromagnetically actuated soft-tethered colonoscope to achieve precise and stable orientation control. An inflated balloon is designed to eliminate the unpredictable disturbance of the floating tether. A 2OR Pseudo-Ri...
-
Retinal surgeons are required to manipulate multiple surgical instruments in a confined intraocular space, while the instruments are constrained at the small incisions made on the sclera. Furthermore, physiological hand tremor can affect the precision of the instrument motion. The Steady-Hand Eye Robot (SHER), developed in our previous study, enables tremor-free tool manipulation by employing a co...
-
In spite of recent high attention of the biohybrid robot system, the previous researches focused on actuation system depend on simple on/off control without feedback control. To solve this problem, we proposed a soft sensor for feedback control of a bio-actuator driven by skeletal muscle. The proposed soft sensor can measure contraction forces of the proposed bio-actuator [1]. The bio-actuator was...
-
Microbubble-induced acoustic microstreaming for efficient on-chip micromanipulation is widely developed in biological applications. However, it is still challenging to simultaneously transport, trap, and rotate single cells using one device in a biocompatible manner, while expensive and bulky traditional acoustic driving system also increases its limitation. This paper presents a portable acoustof...
-
The autonomous robots consisting of an immovable lander and a rover are widely deployed to explore extraterrestrial planets. However, these robots have two main limitations: (1) the separate design for lander and rover respectively results in heavy mass and big volume of the whole system, which increases the launching cost sharply; (2) the rover’s detection area has to be restricted to the vicinit...
-
Self-supervised goal proposal and reaching is a key component for exploration and efficient policy learning algorithms. Such a self-supervised approach without access to any oracle goal sampling distribution requires deep exploration and commitment so that long horizon plans can be efficiently discovered. In this paper, we propose an exploration framework, which learns a dynamics-aware manifold of...
-
Underwater image enhancement is an important low-level computer vision task for autonomous underwater vehicles and remotely operated vehicles to explore and understand the underwater environments. Recently, deep convolutional neural networks (CNNs) have been successfully used in many computer vision problems, and so does underwater image enhancement. There are many deep-learning-based methods with...
-
We propose ultrasound Doppler imaging and magnetic navigation of collective cell microrobots in whole blood. Cell microrobots are cultured using stem cells and iron microparticles, they have spheroidal structures and can be actuated under external magnetic fields. A collective of cell microrobots can be reversibly gathered and spread due to the tunable magnetic interaction, and are able to exhibit...
-
Precise aerial manipulation is important for multirotor robots. For multirotors equipped with arms, the root pose error due to the floating body affects the precision at the end effector. Fixed-root approaches, such as perching on surfaces using the rotor suction force, are useful to address this problem. Furthermore, it is difficult for arm-equipped multirotors to generate large wrenches at the e...
-
In this work, we focus on the autonomous navigation of snake robots in densely-cluttered environments, where collisions between the robot and obstacles are frequent, which could happen often in disaster scenarios, underground caves, or grassland/forest environments. This work takes the view that obstacles are not to be avoided, but rather exploited to support and direct the motion of the snake rob...
-
Navigation on challenging terrain topographies requires the understanding of robots’ locomotion capabilities to produce optimal solutions. We present an integrated framework for real-time autonomous navigation of mobile robots based on elevation maps. The framework performs rapid global path planning and optimization that is aware of the locomotion capabilities of the robot. A GPU-aided, sampling-...
-
This paper presents an integrated methodology and experimental validation of an autonomous framework for unmanned underwater vehicles (UUVs) merely equipped with a conventional monocular camera and a pressure sensor to accomplish high-performance autonomy. Optimal pose of the UUV is solved iteratively by Levenburg-Marquardt optimization for the Perspective-n-Point (PnP) problem. To guarantee a con...
-
This paper investigates the tracking control problem of a class of high-order distributed systems subjected to limited communication bandwidth. An event-triggered control method is proposed, where the controller is triggered only when specific events happen. Moreover, the computational complexity is reduced by introducing command filters for virtual control signals. Specifically, the backstepping ...
-
Stereo matching is essential for robot navigation. However, the accuracy of current widely used traditional methods is low, while methods based on CNN need expensive computational cost and running time. This is because different cost volumes play a crucial role in balancing speed and accuracy. Thus we propose MSCVNet, which combines traditional methods and neural networks to improve the quality of...
-
Systems of multiple low-cost, underactuated floats combined with fully actuated surface vessels can improve the scalability and cost-effectiveness of autonomous systems for marine science and environmental monitoring. Here, we consider a coordination problem where surface vessels must drop off floats at locations such that they are likely to drift to observe given points of interest, and later mus...
-
Multi-Agent Path Finding (MAPF) is essential to large-scale robotic systems. Recent methods have applied reinforcement learning (RL) to learn decentralized polices in partially observable environments. A fundamental challenge of obtaining collision-free policy is that agents need to learn co-operation to handle congested situations. This paper combines communication with deep Q-learning to provide...
-
Conventional coverage path planning algorithms are mainly based on the zigzag and spiral patterns or their combinations. The traversal order is limited by the linear or inside-outside manner. We propose a new set of coverage patterns induced from geometric folding operations, called the geometric folding pattern, to make coverage paths with more flexible traversal order. We study the modeling and ...
-
We propose a tree search-based planning algorithm for a robot manipulator to rearrange objects and grasp a target in a dense space. We consider environments where tasks cannot be completed with prehensile planning only. As assuming that a manipulator is only allowed to grasp from the top, we aim to minimize the number of rearrangement actions and the total execution time, which affects the efficie...
-
Trajectory optimization of sensing robots to actively gather information of targets has received much attention in the past. It is well-known that under the assumption of linear Gaussian target dynamics and sensor models the stochastic Active Information Acquisition problem is equivalent to a deterministic optimal control problem. However, the above-mentioned assumptions regarding the target dynam...
-
This paper proposes a real-time obstacle avoidance control scheme for a 6-DOF manipulator with a tool in the end effector. The system consists of environment monitoring, robot-tool segmentation and collision-free motion planning of the manipulator. A Kinect V2 RGB-D camera is used to track obstacles including human and objects in the working environment. The K-D tree algorithm is then adopted to c...
-
Sampling-based motion planning under task constraints is challenging because the null-measure constraint manifold in the configuration space makes rejection sampling extremely inefficient, if not impossible. This paper presents a learning-based sampling strategy for constrained motion planning problems. We investigate the use of two well-known deep generative models, the Conditional Variational Au...
-
Reliable perception is essential for robots that interact with the world. But sensors alone are often insufficient to provide this capability, and they are prone to errors due to various conditions in the environment. Furthermore, there is a need for robots to maintain a model of its surroundings even when objects go out of view and are no longer visible. This requires anchoring perceptual informa...
-
The autonomous navigation in the crowded environment is a challenging task due to the sensor occlusion and the complex nature of the abstract social interactions. And yet, humans are capable of navigating in such complex environment. In this paper, we propose an effective navigation method that combines the learning-based and model-based methods in a way that a cost function that includes human im...
-
We aim to develop an efficient programming method for equipping service robots with the skill of performing sign language motions. This paper addresses the problem of transferring complex dual-arm sign language motions characterized by the coordination among arms and hands from human to robot, which is seldom considered in previous studies of motion retargeting techniques. In this paper, we propos...
-
LiDAR-based SLAM system is admittedly more accurate and stable than others, while its loop closure detection is still an open issue. With the development of 3D semantic segmentation for point cloud, semantic information can be obtained conveniently and steadily, essential for high-level intelligence and conductive to SLAM. In this paper, we present a novel semantic-aided LiDAR SLAM with loop closu...
-
Planning under partial obervability is essential for autonomous robots. A principled way to address such planning problems is the Partially Observable Markov Decision Process (POMDP). Although solving POMDPs is computationally intractable, substantial advancements have been achieved in developing approximate POMDP solvers in the past two decades. However, computing robust solutions for problems wi...
-
Human running motion is a complex motion that not only based on the movement of legs but also needs the entire body to participate in. Therefore, we focused on the upper body, designed a trunk and pelvic rotation assist suit to apply an external force on the chest and pelvis for assisting the running motion, changing the energy flow between the upper and lower body to improve the running efficienc...
-
This paper extends our previous work on a pneumatic bending module and presents two more modules for rotational and translational motions. In these modules, antagonistic chambers enveloped by rigid shells are adopted to realize bidirectional actuation, and they are characterized by safe actuation, enhanced torque/force output, independent stiffness tuning, and real-time position control. Due to th...
-
This work considers the control of underactuated bipedal walking, and a novel capturability-based control framework is presented. Compared with traditional approaches, the presented control method does not rely on the use of the Poincaré map, which may take significant computational cost. Firstly, a new definition of stable walking is presented, and a novel foot-placement based control method is p...
-
Loop closure detection (LCD), which aims to deal with the drift emerging when robots travel around the route, plays a key role in a simultaneous localization and mapping system. Unlike most current methods which focus on seeking an appropriate representation of images, we propose a novel two-stage pipeline dominated by the estimation of spatial geometric relationship. When a query image occurs, we...
-
Biological observations on tetrapods locomotion deduce that anti-phase synchronization (APS) between fore and rear parts is beneficial for achieving a high-speed walking. On the other hand, theoretical analysis and experimental studies on quadruped robots suggest that a flexible spine potentially improves the gait efficiency and adaptability via smoothing the ground collisions. However, these two ...
-
Flexible flat cables (FFC) are very popular for connecting different components in modern electronics (e.g., mobile phones, laptops, tablets, etc.). The manipulation of FFCs typically relies on highly trained workers that spend hours performing the same repetitive processes, or on autonomous robotic systems that are equipped with simple clamping mechanisms or pneumatically driven suction cups. Suc...
-
Unmanned robots have been proposed for the decommissioning of Fukushima Dai-ichi Nuclear Power Plant. To achieve efficient movement of robots in the high-radiation environment, we propose an “automated construction system of a modularized rail structure.” In the high-radiation environment, the rail structure must be constructed by a remotely controlled robot using minimal sensors. In addition, a c...
-
This paper proposes a novel force controller that serially combines admittance and impedance controllers. The proposed controller is adaptable to an unknown changeable environment in terms of stiffness, and it is able to achieve high control accuracy and stable operation. First, conventional admittance and impedance controllers are recalled, and based on them, a new force controller is designed. N...
-
Motion planning for Cable-Driven Parallel Robots (CDPRs) is a challenging task due to various restrictions on cable tensions, collisions and obstacle avoidance. The presented work aims at proposing an optimal path planning strategy in order to both maximize the wrench capability and the dexterity of the robot in a cluttered environment. First, an asymptoticallyoptimal path finding method based on ...
-
Simultaneous assembly of multiple objects is a key technology to form solid connections among objects to get compact structures in precision assembly and micro-assembly. Dramatically different from traditional assembly of two objects, the interaction among multiple objects is more complicated on analysis and control. During simultaneous assembly of multiple objects, there are multiple mutually eff...
-
In recent years, research and development have been carried out on manipulators equipped with multi-fingered robot hands as end-effectors to perform delicate and dexterous tasks. In high-speed movement of such multi-fingered hand-arms, the weight of the multi-fingered hands slows down the response of the arms. To solve this problem, we propose a control method in which a high-response hand is used...
-
We reformulate the signal temporal logic (STL) synthesis problem as a maximum a-posteriori (MAP) inference problem. To this end, we introduce the notion of random STL (RSTL), which extends deterministic STL with random predicates. This new probabilistic extension naturally leads to a synthesis-as-inference approach. The proposed method allows for differentiable, gradient-based synthesis while exte...
-
For Ultra-Wide-Band (UWB) based navigation, an accurate initialization of the anchors in a reference coordinate system is crucial for precise subsequent UWB-inertial based pose estimation. This paper presents a strategy based on information theory to initialize such UWB anchors using raw distance measurements from tag to anchor(s) and aerial vehicle poses. We include a linear distance-dependent bi...
-
To address the problem of building accurate and coherent models for large-scale terrains from incomplete and noisy sensor data, this paper proposes a novel framework that can efficiently infer terrain structures by divisionally providing the best linear unbiased estimates for the elevation values. To avoid data ambiguity caused by the uncertainty of sensor data, the proposed method introduces elev...
-
Symmetric environment is one of the most intractable and challenging scenarios for mobile robots to accomplish global localization tasks, due to the highly similar geometrical structures and insufficient distinctive features. Existing localization solutions in such scenarios either depend on pre-deployed infrastructures which are expensive, inflexible, and hard to maintain; or rely on single senso...
-
Autonomous urban driving among human-driven cars requires a holistic understanding of road rules, driver intents and driving styles. This is challenging as a short-term, single instance, driver intent of lane change may not correspond to their driving styles for a longer duration. This paper presents an interactive behavior planner which accounts for road context, short-term driver intent, and lon...
-
Mapping three-dimensional (3-D) dynamic environments is essential for aerial robots but challenging to consider the increased dimensions in both space and time compared to 2-D static mapping. This paper presents a kernel-based 3-D dynamic occupancy mapping algorithm, K3DOM, that distinguishes between static and dynamic objects while estimating the velocities of dynamic cells via particle tracking....
-
This paper aims to improve robots’ versatility and adaptability by allowing them to use a large variety of end- effector tools and quickly adapt to new tools. We propose AdaGrasp, a method to learn a single grasping policy that generalizes to novel grippers. By training on a large collection of grippers, our algorithm is able to acquire generalizable knowledge of how different grippers should be u...
-
Practical industrial assembly scenarios often require robotic agents to adapt their skills to unseen tasks quickly. While transfer reinforcement learning (RL) could enable such quick adaptation, much prior work has to collect many samples from source environments to learn target tasks in a model-free fashion, which still lacks sample efficiency on a practical level. In this work, we develop a nove...
-
Sequences and time-series often arise in robot tasks, e.g., in activity recognition and imitation learning. In recent years, deep neural networks (DNNs) have emerged as an effective data-driven methodology for processing sequences given sufficient training data and compute resources. However, when data is limited, simpler models such as logic/rule-based methods work surprisingly well, especially w...
-
This work focuses on learning useful and robust deep world models using multiple, possibly unreliable, sensors. We find that current methods do not sufficiently encourage a shared representation between modalities; this can cause poor performance on downstream tasks and over-reliance on specific sensors. As a solution, we contribute a new multi-modal deep latent state-space model, trained using a ...
-
In this paper, we propose a generative model that learns a sequence-to-sequence (Seq2Seq) translation between human whole-body motions and linguistic descriptions by natural language. Our model merges the Seq2Seq model with the training strategy of sequence generative adversarial nets (SeqGAN), which extends a GAN framework to solve the problem that the gradient cannot pass back to the generator n...
-
A motion-planning method that can adapt to changes in the surrounding environment is proposed and evaluated. Automation of work is progressing in factories and distribution warehouses due to labor shortages. However, utilizing robots for transport operations in a distribution warehouse faces a problem; that is, tasks for setting up a robot, such as adjustment of acceleration for stabilization of t...
-
This paper explores the idea that skillful assembly is best represented as dynamic sequences of Manipulation Primitives, and that such sequences can be automatically discovered by Reinforcement Learning. Manipulation Primitives, such as "Move down until contact", "Slide along x while maintaining contact with the surface", have enough complexity to keep the search tree shallow, yet are generic enou...
-
This paper presents a decentralized and asynchronous systematic solution for multi-robot autonomous navigation in unknown obstacle-rich scenes using merely onboard resources. The planning system is formulated under gradient-based local planning framework, where collision avoidance is achieved by formulating the collision risk as a penalty of a nonlinear optimization problem. In order to improve ro...
-
This paper addresses robust landing stabilization in humanoid locomotion on uneven terrain. The core idea is to find a configuration of the robot that results in small impulsive force when an unexpected obstacle is encountered, and to adjust post-contact reference for swing foot with which the pose of the foot is stabilized on the obstacle. This can be achieved by walking with heel strike motion (...
-
This paper introduces a general state estimation framework fusing multiple sensor information for hybrid wheeled-legged robots performing mobile manipulation tasks. At the core of the state estimator is a novel unified odometry for hybrid locomotion which can seamlessly maintain tracking and has no need to switch between stepping and rolling modes. To the best of our knowledge, the proposed odomet...
-
Deep reinforcement learning has made significant progress in robotic manipulation tasks and it works well in the ideal disturbance-free environment. However, in a real-world environment, both internal and external disturbances are inevitable, thus the performance of the trained policy will dramatically drop. To improve the robustness of the policy, we introduce the adversarial training mechanism t...
-
This paper introduces a challenging object grasping task and proposes a self-supervised learning approach. The goal of the task is to grasp an object which is not feasible with a single parallel gripper, but only with harnessing environment fixtures (e.g., walls, furniture, heavy objects). This Slide-to-Wall grasping task assumes no prior knowledge except the partial observation of a target object...
-
Robot grasping and manipulation planning in unstructured and dynamic environments is heavily dependent on the attributes of manipulated objects. Although deep learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Moreover, training such models requires large datasets that are generally e...
-
Lecturers of Engineering courses around the world are struggling to increase the engagement of students through the introduction of appropriate hands-on activities and assignments. In Biomechatronics and Robotics courses these assignments typically focus on how certain devices are designed, modelled, fabricated, or controlled. The hardware for these assignments is usually purchased by some externa...
-
The interaction between an exoskeleton and its human user is complex, and needs to conform to various requirements related to safety, comfort and adaptability. It is however impractical to test a large number of prototype variations against a large number of user variations, especially in the initial design and testing phases. Model based methods can help at this design stage by providing a virtua...
-
We propose an efficient way of solving optimal control problems for rigid-body systems on the basis of inverse dynamics and the multiple-shooting method. We treat all variables, including the state, acceleration, and control input torques, as optimization variables and treat the inverse dynamics as an equality constraint. We eliminate the update of the control input torques from the linear equatio...
-
Powered exoskeletons for people with paraplegia are subjected to repetitive and large impacts due to the repeated ground contacts. The repetitive impact forces not only deteriorate the wear comfort but also cause a serious damage to the muscles and bones of the human wearing the powered exoskeleton. To address this issue, a novel shock absorption mechanism for powered exoskeletons that can reduce ...
-
Learning driving policies using an end-to-end network has been proved a promising solution for autonomous driving. Due to the lack of a benchmark driver behavior dataset that contains both the visual and the LiDAR data, existing works solely focus on learning driving from visual sensors. Besides, most works are limited to predict steering angle yet neglect the more challenging vehicle speed contro...
-
In this paper, we present PocoNet: Point cloud Online COmpression NETwork to address the task of SLAM-oriented compression. The aim of this task is to select a compact subset of points with high priority to maintain localization accuracy. The key insight is that points with high priority have similar geometric features in SLAM scenarios. Hence, we tackle this task as point cloud segmentation to ca...
-
3D Reconstruction of Deformable Colon Structures based on Preoperative Model and Deep Neural Network
In colonoscopy procedures, it is important to rebuild and visualize the colonic surface to minimize the missing regions and reinspect for abnormalities. Due to the fast camera motion and deformation of the colon in standard forward-viewing colonoscopies, traditional simultaneous localization and mapping (SLAM) systems work poorly for 3D reconstruction of colon surfaces and are prone to severe drif...
-
This paper focuses on developing a real-time and flexible velocity estimation approach for serial revolute manipulator using only one inertial measurement unit (IMU) mounted on each link side of the manipulator. Particularly, the proposed approach has no requirement for the installation position and orientation of the IMU, which improves the flexibility of the implementation procedure. A joint vel...
-
An Adaptive Cruise Control (ACC) system allows vehicles to maintain a desired headway distance to a preceding vehicle automatically. It is increasingly adopted by commercial vehicles. Recent research demonstrates that the effective use of ACC can improve the traffic flow through the adaptation of the headway distance in response to the current traffic conditions. In this paper, we demonstrate that...
-
In industrial assembly tasks, the in-hand pose of grasped objects needs to be known with high precision for subsequent manipulation tasks such as insertion. This problem (in-hand-pose estimation) has traditionally been addressed using visual recognition or tactile sensing. On the one hand, while visual recognition can provide efficient pose estimates, it tends to suffer from low precision due to n...
-
To generate assembly sequences that robots can easily handle, this study tackled assembly sequence generation (ASG) by considering two tradeoff objectives: (1) insertion conditions and (2) degrees of the constraints affecting the assembled parts. We propose a multi-objective genetic algorithm to balance these two objectives. Furthermore, we extend our previously proposed 3D computer-aided design (...
-
The technology of dynamic map fusion among networked vehicles has been developed to enlarge sensing ranges and improve sensing accuracies for individual vehicles. This paper proposes a federated learning (FL) based dynamic map fusion framework to achieve high map quality despite unknown numbers of objects in fields of view (FoVs), various sensing and model uncertainties, and missing data labels fo...
-
The design of aerial-aquatic multirotors can benefit from thruster rotation so that the thrusters can act directly in the lateral directions of surge and sway when submerged. This allows much more effective locomotion underwater as opposed to the aerial configuration where rotational acceleration is used to direct small components of thrust in the lateral directions. However, the introduction of l...
-
Birds are agile in locomotion and able to move quickly and easily from one place to another. When a bird is on the ground, and a threat approaches, the bird will fly away and escape. An ornithopter robot provides advantages in energy saving, maneuverability, and crash safety. Most flapping robots require an operator or assistance to take off. The goal of this study is to enable self-takeoff from t...
-
This paper proposes a systematic solution that uses an unmanned aerial vehicle (UAV) to aggressively and safely track an agile target. It properly handles the challenging situations where the intent of the target and the dense environments are unknown. Our work is divided into two parts: target motion prediction and tracking trajectory planning. The target motion prediction method utilizes target ...
-
Robot navigation is a task where reinforcement learning approaches are still unable to compete with traditional path planning. State-of-the-art methods differ in small ways, and do not all provide reproducible, openly available implementations. This makes comparing methods a challenge. Recent research has shown that unsupervised learning methods can scale impressively, and be leveraged to solve di...
-
In this paper, we propose a new planning scheme for high-speed flight in an unknown environment while taking into account drag forces. Drag forces become non-negligible at high speeds and may lead to unfeasible trajectories. The method leverages a new Mixed-Integer Quadratic Program/Model Predictive Control formulation that allows to easily account for drag forces. This formulation makes use of a ...
-
Cable-driven parallel robots (CDPRs) are a type of parallel robots, where cables are used instead of rigid links. This leads to many advantages, such as large workspace, low mass in motion and simple reconfiguration. The drawbacks are accuracy issues and complex cable management. Indeed, it is usual that cables become slack. That can be caused by, for example, cable mass, uncertainties in the syst...
-
The increasing frequency of robot interactions with humans and complex environments motivates the design and control of soft robotic systems made of compliant structures. Additive manufacturing processes enable the creation of new robotic systems and mechanisms that gain their compliance extrinsically through their design, rather than simply their material. With the ability to rapidly and affordab...
-
Humans in contrast to robots are excellent in performing fine manipulation tasks owing to their remarkable dexterity and sensorimotor organization. Enabling robots to acquire such capabilities, necessitates a framework that not only replicates the human behaviour but also integrates the multi-sensory information for autonomous object interaction. To address such limitations, this research proposes...
-
Increasing the robustness of grasping actions and the recovery from failure is key to improving a robot’s autonomy. Endowing robots with the ability to robustly grasp and manipulate unknown difficult objects such as stones is required for sample collection in unknown environments. In this paper, we present a complete system for robust grasping of stones, which integrates stone segmentation based o...
-
In tendon-driven robotic hands, tendons are usually routed along several pulleys. The resulting friction is often substantial, and must therefore be modelled and estimated, for instance for accurate control and contact detection. Common approaches for friction estimation consider special dedicated setups, where the parameters of a static or dynamic friction model at a single contact point are dete...
-
Projecting high-dimensional environment observations into lower-dimensional structured representations can considerably improve data-efficiency for reinforcement learning in domains with limited data such as robotics. Can a single generally useful representation be found? In order to answer this question, it is important to understand how the representation will be used by the agent and what prope...
-
We present a purely vision based geolocation method for aircraft flying over urban and suburban environments. The method is based on matching aerial images with geolocated map tiles using a shared low dimensional embedded space of descriptors. The Euclidean distance between descriptors is used as a similarity measure between domains. The similarity between the observation and map locations is then...
-
The problem addressed in this paper is the localisation of a mobile robot using a combination of on-board sensors and Ultra-Wideband (UWB) beacons. Specifically, we consider a scenario in which a mobile robot travels across an area infrastructured with a small number of UWB anchors. The presence of obstacles in the environment introduces an offset in the measurements of the distance between the ro...
-
Robust and accurate, map-based localization is crucial for autonomous mobile systems. In this paper, we exploit range images generated from 3D LiDAR scans to address the problem of localizing mobile robots or autonomous cars in a map of a large-scale outdoor environment represented by a triangular mesh. We use the Poisson surface reconstruction to generate the mesh-based map representation. Based ...
-
Relocalization is a fundamental task in the field of robotics and computer vision. There is considerable work in the field of deep camera relocalization, which directly estimates poses from raw images. However, learning-based methods have not yet been applied to the radar sensory data. In this work, we investigate how to exploit deep learning to predict global poses from Emerging Frequency-Modulat...
-
Monocular and stereo visions are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots. However, they are usually developed independently and have their respective strengths and limitations. We propose a novel unified learning framework that leverages the strengths of both monocular and stereo cues for 3D human localization. Our method jointly (i) ...
-
Constructing Birds-Eye-View (BEV) maps from monocular images is typically a complex multi-stage process involving the separate vision tasks of ground plane estimation, road segmentation and 3D object detection. However, recent approaches have adopted end-to-end solutions which warp image-based features from the image-plane to BEV while implicitly taking account of camera geometry. In this work, we...
-
Dense depth estimation is essential to scene-understanding for autonomous driving. However, recent self-supervised approaches on monocular videos suffer from scale-inconsistency across long sequences. Utilizing data from the ubiquitously copresent global positioning systems (GPS), we tackle this challenge by proposing a dynamically-weighted GPS-to-Scale (g2s) loss to complement the appearance-base...
-
Across a wide range of applications, from autonomous vehicles to medical imaging, multi-spectral images provide an opportunity to extract additional information not present in color images. One of the most important steps in making this information readily available is the accurate estimation of dense correspondences between different spectra.Due to the nature of cross-spectral images, most corres...
-
Postural synergies are used in robotics to facilitate the control of dexterous artificial hands. This is achieved by learning a latent space (synergy space) from grasp postures and directly controlling the hand in this space. In this work, we propose the use of a non-linear conditional model for learning the latent space, that can incorporate the object shape and size as additional variables. Whil...
-
Recent studies have shown that enabling drones to change their morphology in flight can significantly increase their versatility in different tasks. In this paper, we investigate the aerodynamic effects caused by the partial overlap between the propellers and the main body of a morphing quadrotor during flight. We experimentally characterize such effects and design a morphology-aware control schem...
-
We address the problem of autonomous exploration of unknown environments using a Micro Aerial Vehicle (MAV) equipped with an active depth sensor. As such, the task consists in mapping the gradually discovered environment while planning the envisioned trajectories in real-time, using on-board computation only. To do so, we present SplatPlanner, an end-to-end autonomous planner that is based on a no...
-
In this work, we present a new exploration algorithm for Micro Aerial Vehicles (MAVs). The planner uses a combination of Next-Best-View (NBV) sampling and frontier-based approaches to reduce the impact of finding unexplored areas in large scenarios. For each sampled point, the yaw angle is optimized to maximize the potential gain for mapping. The gain is expressed as a ratio between the exploratio...
-
Neuromorphic processors like Loihi offer a promising alternative to conventional computing modules for endowing constrained systems like micro air vehicles (MAVs) with robust, efficient and autonomous skills such as take-off and landing, obstacle avoidance, and pursuit. However, a major challenge for using such processors on robotic platforms is the reality gap between simulation and the real worl...
-
Event-based vision sensors achieve up to three orders of magnitude better speed vs. power consumption trade off in high-speed control of UAVs compared to conventional image sensors. Event-based cameras produce a sparse stream of events that can be processed more efficiently and with a lower latency than images, enabling ultra-fast vision-driven control. Here, we explore how an event-based vision a...
-
Current aerial robots are increasingly adaptive; they can morph to enable operation in changing conditions to complete diverse missions. Each mission may require the robot to conduct a different task. A conventional learning approach can handle these variations when the system is trained for similar tasks in a representative environment. However, it may result in overfitting to the new data stream...
-
Interactive (mechanical) impedance and finger fatigues are important topics, which have not been well investigated. To tackle this problem, we developed a soft lightweight (0.25 kg) finger exoskeleton (TIE-EXO) for quantifying interactive impedance and finger fatigue. A resist-as-needed (RAN) controller was used to produce variable resistance in fingers’ exercises. The TIE-EXO’s feedback and RAN’s...
-
When the arm prosthesis worn by an amputated Human being is not adequately configured with respect to the end-effector task, body compensations are often observed. Namely, to compensate for a wrong joint positioning on the robotic distal side, a subject trying to reach a desired position/orientation of his/her hand mobilizes his/her proximal joints, thus exploiting the redundancy of the human+robo...
-
Many human-robot collaboration scenarios can be seen as a redundant leader-follower setup where the human (i.e., the leader) can potentially perform the task without the assistance of the robot (i.e., the follower). Thus, the goal of the collaboration, beside stable execution of the task, is to reduce the human cost; e.g., ergonomic, or cognitive cost. Such system redundancies (where the same task...
-
Safety can be listed as one of the most important aspects of every working environment, from the low risky to the most dangerous one. Construction sites can be clearly identified among the riskiest working fields, mainly because several complex and fast maneuvers are executed in a very crowded, dynamic and uncertain scenario. Specifically referring to construction machines, typical operations such...
-
Predicting trajectories of participating vehicles is a crucial task towards full and safe autonomous driving. General unconstrained machine learning methods often report unrealistic predictions, and need to be combined with different motion constraints. Existing work either defines some shallow maneuvers and modes to regulate the output, or uses vehicle dynamics as the main source of constraints, ...
-
As urban environments manifest high levels of complexity it is of vital importance that safety systems embedded within autonomous vehicles (AVs) are able to accurately anticipate short-term future motion of nearby agents. This problem can be further understood as generating a sequence of coordinates describing the future motion of the tracked agent. Various proposed approaches demonstrate signific...
-
This paper proposes an approach for (a) recognizing an observed trajectory from a library of pre-learned motions; and (b) predicting the target position of such trajectory. In our approach, motions are represented as Dynamic Movement Primitives (DMPs). We use critical points from the observed trajectory to time-align it with those in the library. To match the observed trajectory with those in the ...
-
This paper presents the first successful experiment implementing whole-body model predictive control with state feedback on a torque-control humanoid robot. We demonstrate that our control scheme is able to do whole-body target tracking, control the balance in front of strong external perturbations and avoid collision with an external object. The key elements for this success are threefold. First,...
-
The Sequential Linear Quadratic (SLQ) algorithm is a continuous-time version of the well-known Differential Dynamic Programming (DDP) technique with a Gauss-Newton Hessian approximation. This family of methods has gained popularity in the robotics community due to its efficiency in solving complex trajectory optimization problems. However, one major drawback of DDP-based formulations is their inab...
-
Optimal control is a popular approach to synthesize highly dynamic motion. Commonly, L2 regularization is used on the control inputs in order to minimize energy used and to ensure smoothness of the control inputs. However, for some systems, such as satellites, the control needs to be applied in sparse bursts due to how the propulsion system operates. In this paper, we study approaches to induce sp...
-
Path planning and collision avoidance are challenging in complex and highly variable environments due to the limited horizon of events. In literature, there are multiple model- and learning-based approaches that require significant computational resources to be effectively deployed and they may have limited generality. We propose a planning algorithm based on a globally stable passive controller t...
-
One major challenge in Assembly Sequence Planning (ASP) for complex real-world CAD-scenarios is to find an appropriate disassembly path for each assembled part. Complex real-world scenes are characterized by a large installation space. There each part has many different possible disassembly paths that differ in length and clearance. However, due to tight packing in the installation space, these pa...
-
Gaussian Process (GP) formulation of continuous-time trajectory offers a fast solution to the motion planning problem via probabilistic inference on factor graph. However, often the solution converges to in-feasible local minima and the planned trajectory is not collision-free. We propose a message passing algorithm that is more sensitive to obstacles with fast convergence time. We leverage the ut...
-
In the case of non-holonomic robot navigation, path planning algorithms such as Rapidly-exploring Random Tree (RRT) rarely provide feasible and smooth paths without the need of additional processing. Furthermore, in a transport context like power wheelchair navigation, passenger comfort should be a priority and influence path planning strategy. In this paper, we propose a local path planner which ...
-
The ability to safely navigate through complex and cluttered environments is required for a wide range of robotics applications. This paper introduces a framework to compute safe global paths in maps represented as collections of 3D Signed Distance Function (SDF) submaps. Such maps are able to maintain global consistency in spite of odometry drift. However, computationally efficient global path pl...
-
Redundant robots offer the possibility of improving agility, compared to their non-redundant counterparts, by exploiting the additional kinematic DOFs to increase a measure called manipulability. While it is common to maximize the manipulability measure during redundancy resolution locally, global optimization of a full trajectory is usually computationally too expensive and thus only considered f...
-
Naturally, finding joint trajectories for robotic manipulators involves competing optimization goals. On the one hand, the end-effector should move along a predictable and short path while on the other hand joint movement and acceleration should be kept to a minimum. Obstacles in the workspace or joint limits complicate the situation even further. Constructing a metric that makes undesired configu...
-
Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g...
-
In this paper, a robust RGB-D SLAM system is proposed to utilize the structural information in indoor scenes, allowing for accurate tracking and efficient dense mapping on a CPU. Prior works have used the Manhattan World (MW) assumption to estimate low-drift camera pose, in turn limiting the applications of such systems. This paper, in contrast, proposes a novel approach delivering robust tracking...
-
This paper implements and demonstrates visual route mapping and localisation upon a Pixel Processor Array (PPA). The PPA sensor comprises of an array of Processing Elements (PEs), each of which can capture and process visual information directly. This provides significant parallel processing power allowing novel ways in which information can be processed on-sensor. Our method predicts the correct ...
-
Visual place recognition is the task of finding matchings of images that show the same place in the world. Combinations of appearance changes (e.g. changing illumination or weather) and geometric changes (e.g. viewpoint changes or occlusions) challenge existing approaches. Learning-based local image feature pipelines are a promising approach to this type of problem. We present a novel attentive fe...
-
Visual place recognition is the task of recognizing same places of query images in a set of database images. It is important for loop closure detection in SLAM and candidate selection for global localization. Many approaches in the literature perform computationally inefficient full image comparisons between queries and all database images. There is still a lack of suited methods for efficient pla...
-
This paper presents a semi-supervised framework for multi-level description learning aiming for robust and accurate camera relocalization across large perception variations. Our proposed network, namely DLSSNet, simultaneously learns weakly-supervised semantic segmentation and local feature description in the hierarchy. Therefore, the augmented descriptors, trained in an end-to-end manner, provide...
-
Datasets for autonomous cars are essential for the development and benchmarking of perception systems. However, most existing datasets are captured with camera and LiDAR sensors in good weather conditions. In this paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to facilitate research on object detection, tracking and scene understanding using radar sensing for safe autonom...
-
Accurately localizing in and mapping an environment are essential building blocks of most autonomous systems. In this paper, we present a novel approach for LiDAR odometry and mapping, focusing on improving the mapping quality and at the same time estimating the pose of the vehicle. Our approach performs frame-to-mesh ICP, but in contrast to other SLAM approaches, we represent the map as a triangl...
-
Structure from Motion (SfM) often fails to estimate accurate poses in environments that lack suitable visual features. In such cases, the quality of the final 3D mesh, which is contingent on the accuracy of those estimates, is reduced. One way to overcome this problem is to combine data from a monocular camera with that of a LIDAR. This allows fine details and texture to be captured while still ac...
-
This paper presents a simulation study of the problem of balancing a planar double pendulum in which the lower body (the leg) has been modified to include a spring-loaded passive prismatic joint. Robots of this kind can travel by hopping, and can also stand and balance on a single point. The purpose of this study is to investigate the degree to which a balance controller can cope with the large an...
-
Demands on leg degrees of freedom and control precision for bipedal robotics are steadily increasing, especially for the tasks involving walking on a rough terrain. In this paper we present an alternative, as well as a working proof-of-concept. Meet gyrubot: a 5-link almost planar bipedal robot with a torso complemented by a nonanthropomorphic stabilization system, capable of blindly walking throu...
-
Modeling the Coulomb Friction Cone in trajectory optimization is typically done by linearizing it along a series of vectors. Often, these vectors define the edges of polyhedral estimations of the cone. This article provides an alternate approach that samples the cone along a vector that satisfies the Maximum Dissipation Principle, which is shown to be significantly more computationally tractable. ...
-
Modern industrial applications require robots to operate in unpredictable environments, and programs to be created with a minimal effort, to accommodate frequent changes to the task. Here, we show that genetic programming can be effectively used to learn the structure of a behavior tree (BT) to solve a robotic task in an unpredictable environment. We propose to use a simple simulator for learning,...
-
Efficient sampling from constraint manifolds, and thereby generating a diverse set of solutions for feasibility problems, is a fundamental challenge. We consider the case where a problem is factored, that is, the underlying nonlinear program is decomposed into differentiable equality and inequality constraints, each of which depends only on some variables. Such problems are at the core of efficien...
-
We introduce a simple new method for visual imitation learning, which allows a novel robot manipulation task to be learned from a single human demonstration, without requiring any prior knowledge of the object being interacted with. Our method models imitation learning as a state estimation problem, with the state defined as the end-effector’s pose at the point where object interaction begins, as ...
-
We investigate a novel deep-learning-based approach to estimate uncertainty in stereo disparity prediction networks. Current state-of-the-art methods often formulate disparity prediction as a regression problem with a single scalar output in each pixel. This can be problematic in practical applications as in many cases there might not exist a single well defined disparity, for example in cases of ...
-
Grasping objects is one of the most important abilities that a robot needs to master in order to interact with its environment. Current state-of-the-art methods rely on deep neural networks trained to jointly predict a graspability score together with a regression of an offset with respect to grasp reference parameters. However, these two predictions are performed independently, which can lead to ...
-
Salient object detection (SOD) can directly improve the performance of tasks like obstacle detection, semantic segmentation and object recognition. Such tasks are important for robotic and other autonomous navigation systems. State-of-the-art SOD methodologies, provide improved performance by incorporating depth information, usually acquired using additional specialized sensors, e.g., RGB-D camera...
-
Footstep planning is the dominating approach when it comes to controlling the walk of a humanoid robot, even though a footstep plan is expensive to compute. The most prominent proposals typically spend up to a few seconds of computation time and output a sequence of up to 30 steps all the way to the goal. This way, footstep planning is applicable only in static environments where nothing changes a...
-
The big potential of humanoid robots is not restricted to the ground, but these versatile machines can be successfully employed in unconventional scenarios, e.g. space, where contacts are not always present. In these situations, the robot’s limbs can be used to assist or even generate the angular motion of the floating base, as a consequence of the centroidal momentum conservation. In this paper, ...
-
During the planning phase of industrial robot workplaces, hazard analyses are required so that potential hazards for human workers can be identified and appropriate safety measures can be implemented. Existing hazard analysis methods use human reasoning, checklists and/or abstract system models, which limit the level of detail. We propose a new approach that frames hazard analysis as a search prob...
-
The evaluation of robot capabilities to navigate human crowds is essential to conceive new robots intended to operate in public spaces. This paper initiates the development of a benchmark tool to evaluate such capabilities; our long term vision is to provide the community with a simulation tool that generates virtual crowded environment to test robots, to establish standard scenarios and metrics t...
-
During the co-presence of human workers and robots, measures are required to avoid injuries from undesired contacts. Capacitive Proximity Sensors (CPSs) offer a cost-effective solution to cover the entire robot manipulator with fast close-range perception for HRC tasks, closing the perception gap between tactile detection and mid-range perception. CPSs do not suffer from occlusion and compared to ...
-
One of the first and foremost non-verbal interactions that humans perform is a handshake. It has an impact on first impressions as touch can convey complex emotions. This makes handshaking an important skill for the repertoire of a social robot. In this paper, we present a novel framework for learning reaching behaviours for humanrobot handshaking behaviours for humanoid robots solely using third-...
-
Haptic guidance can improve accuracy and dexterity during the teleoperation of a robot, but only if the model of the task used to provide the assistance is accurate. In medical robotics, the registration of a task from pre-operative planning from medical images to the robot’s task-space can be erroneous. Additionally, the deformability of the environment can require online correction of a planned ...
-
Physical Human-Robot-Interaction (pHRI) is beneficial for communication in social interaction or to perform collaborative tasks but is also crucial for safety. While robotic devices embed sensors for this sole purpose, their design often is the results of a trade-off between technical capabilities and rarely considers human factors. We propose a novel approach to design and fabricate compliant Hum...
-
In this work, a novel Dynamic Movement Primitive (DMP) formulation is proposed which supports reversibility, i.e. backwards reproduction of a learned trajectory. Apart from sharing all favourable properties of the original DMP, decoupling the teaching of position and velocity profiles and bidirectional drivability along the encoded path are also supported. Original DMP have been extensively used f...
-
It is widely accepted that in the future, robots will cooperate with humans in everyday tasks. Robots interacting with humans will require social awareness when performing their tasks which will require navigation. While navigating, robots should aim to avoid distressing people in order to maximize their chance of social acceptance. For instance, avoiding getting too close to people or disrupting ...
-
This paper offers an explanation of why humans can effortlessly grasp objects from a pile. We identified a regularity in objects’ motion when pushed, namely, an object separates and stabilizes in front of the pusher. We devise an open-loop grasping strategy leveraging this regularity in piles of nearly identical objects. Our real robot robustly grasps round objects beside a wall with success rates...
-
Grasp planning and most specifically the grasp space exploration is still an open issue in robotics. This article presents an efficient procedure for exploring the grasp space of a multifingered adaptive gripper for generating reliable grasps given a known object pose. This procedure relies on a limited dataset of manually specified expert grasps, and use a mixed analytic and data-driven approach ...
-
We introduce an underactuated differential-based robot gripper able to perform self-adaptive grasping with passive disturbance rejection. The gripper utilises three car differential systems to achieve self-adaptiveness with a single actuator: a base differential for distributing power from the motor to the fingers, and two independent finger differentials for controlling the proximal and distal jo...
-
Situation awareness is of great importance for autonomous ships. One key aspect is to estimate the sea state in a real-time manner. Considering the ship as a large wave buoy, the sea state can be estimated from motion responses without extra sensors installed. However, it is difficult to associate waves with ship motion through an explicit model since the hydrodynamic effect is hard to model. In t...
-
Robotic systems evolving in hazardous and harsh environment are prone to mission failure or system loss in presence of faults. This paper presents a fault tolerant methodology, implemented into a control architecture of an underwater robot that executes biological monitoring missions. High level constraint violations (mission, safety, energy, time and localization) and low level faults (software a...
-
In this paper, we propose an approach for robust visual Simultaneous Localisation and Mapping (SLAM) in underwater environments leveraging acoustic, inertial and altimeter/depth sensors. Underwater visual SLAM is challenging due to factors including poor visibility caused by suspended particles in water, a lack of light and insufficient texture in the scene. Because of this, many state-of-the-art ...
-
An important capability of humans when performing dexterous precision gripping tasks is our ability to feel both the weight and slipperiness of an object in real-time, and adjust our grip force accordingly. In this paper, we present for the first time a fully-instrumented version of our PapillArray tactile sensor concept, which can sense grip force, object weight, and incipient slip and friction, ...
-
This work investigates uncertainty-aware deep learning (DL) in tactile robotics based on a general framework introduced recently for robot vision. For a test scenario, we consider optical tactile sensing in combination with DL to estimate the edge pose as a feedback signal to servo around various 2D test objects. We demonstrate that uncertainty-aware DL can improve the pose estimation over determi...
-
In this work, we report on how a sense of touch can be used to control an underactuated anthropomorphic robot hand, based on an integration that respects the hand’s mechanical functionality. Our focus is on integrating the sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a...
-
An efficient approach to closed-loop shape control of deformable objects using finite element models
Robots are nowadays faced with the challenge of handling deformable objects in industrial operations. In particular, the problem of shape control, which aims at giving a specific deformation state to an object, has gained interest recently in the research community. Among the proposed solutions, approaches based on finite elements proved accurate and reliable but also complex and computationally-i...
-
Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does ...
-
Modern, torque-controlled service robots can regulate contact forces when interacting with their environment. Model Predictive Control (MPC) is a powerful method to solve the underlying control problem, allowing to plan for whole-body motions while including different constraints imposed by the robot dynamics or its environment. However, an accurate model of the robot-environment is needed to achi...
-
Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the surgical robot control quality. In this work, we aim to develop an effective surgical gesture recogniti...
-
We investigate what grade of sensor data is required for training an imitation-learning-based AV planner on human expert demonstration. Machine-learned planners [1] are very hungry for training data, which is usually collected using vehicles equipped with the same sensors used for autonomous operation [1]. This is costly and non-scalable. If cheaper sensors could be used for collection instead, da...
-
For asymptotically optimal sampling-based path planners such as RRT*, path quality improves as the number of samples added to the motion tree increases. However, each additional sample requires a nearest-neighbor search. Calculating state transition costs can be particularly difficult in cases with complex dynamics such as aerial vehicles in non-isotropic cost fields like wind. Computationally cos...
-
Automated Generation of Robot Trajectories for Assembly Processes Requiring Only Sparse Manual Input
In this paper, a new method for offline programming part assembly operations with tight fittings is presented. More specifically, an assembly process trajectory generator with self programming capabilities is developed where the user needs to provide only very sparse and intuitive input. The presented system is added to the existing skill based robot software package VEROSIM. In VEROSIM, the traje...
-
Numerous robotic and control applications have strict real-time requirements, which, when violated, result in reduced quality of service or, in case of safety critical applications, might even have catastrophic consequences. To ensure that certain real-time constraints are satisfied, roboticists have relied on real-time safe frameworks, environments and middleware. With the introduction of ROS 2, ...
-
We introduce a multi-functional robotic gripper equipped with a set of actions required for disassembly of electromechanical devices. The gripper consists of a robot arm with 5 degrees of freedom (DoF) for manipulation and a jaw gripper with a 1-DoF rotation joint and a 1-DoF closing joint. The system enables manipulation in 7 DoF and offers the ability to reposition objects in hand and to perform...
-
Due to manufacturing tolerances, the geometry parameters of workpieces are not constant in industrial welding applications. Today, this problem is addressed by either accepting fluctuating part quality or by measuring the geometry and adjusting the configuration of the robot and process controller for each individual part. However, measuring the geometry requires additional manufacturing time or r...
-
The detection of contextual anomalies is a challenging task for surveillance since an observation can be considered anomalous or normal in a specific environmental context. An unmanned aerial vehicle (UAV) can utilize its aerial monitoring capability and employ multiple sensors to gather contextual information about the environment and perform contextual anomaly detection. In this work, we introdu...
-
We propose an approach enabling an Unmanned Aerial Vehicle (UAV) to autonomously enter a target building through an open window. We use a fusion of depth camera and 2D Light Detection and Ranging (LiDAR) data for window detection and continuous estimation of its position, orientation, and size. The proposed algorithms are capable of running both with and without available a priori information. The...
-
A polyhedral friction cone is a set of reaction wrenches that an object can experience whilst in contact with its environment. This polyhedron is a powerful tool to control an object’s motion and interaction with the environment. It can be derived analytically, upon knowledge of object and environment geometries, contact point locations and friction coefficients. We propose to estimate the polyhed...
-
Deep Neural Networks (NNs) have been widely utilized in contact-rich manipulation tasks to model the complicated contact dynamics. However, NN-based models are often difficult to decipher which can lead to seemingly inexplicable behaviors and unidentifiable failure cases. In this work, we address the interpretability of NN-based models by introducing the kinodynamic images. We propose a methodolog...
-
We study synthesis of control strategies from linear temporal logic (LTL) objectives in unknown environments. We model this problem as a turn-based zero-sum stochastic game between the controller and the environment, where the transition probabilities and the model topology are fully unknown. The winning condition for the controller in this game is the satisfaction of the given LTL specification, ...
-
We consider the problem of security-aware planning in an unknown stochastic environment, in the presence of attacks on control signals (i.e., actuators) of the robot. We model the attacker as an agent who has the full knowledge of the controller as well as the employed intrusion-detection system and who wants to prevent the controller from performing tasks while staying stealthy. We formulate the ...
-
Solving robotic navigation tasks via reinforcement learning (RL) is challenging due to their sparse reward and long decision horizon nature. However, in many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation and using sub-goal...
-
Collaborative robots offer increased interaction capabilities at relatively low cost but in contrast to their industrial counterparts they inevitably lack precision. Moreover, in addition to the robots' own imperfect models, day-to-day operations entail various sources of errors that despite being small rapidly accumulate. This happens as tasks change and robots are re-programmed, often requiring ...
-
An underlying structure in several sampling-based methods for continuous multi-robot motion planning (MRMP) is the tensor roadmap (PR), which emerges from combining multiple PRM graphs constructed for the individual robots via a tensor product. We study the conditions under which the TR encodes a near-optimal solution for MRMP—satisfying these conditions implies near optimality for a variety of po...
-
Multicopters are able to perform high maneuverability yet their potential have not been fully achieved. In this work, we propose a full-body, optimization-based motion planning framework that takes shape and attitude of aerial robot into consideration such that the aggressiveness of drone maneuvering improves significantly in cluttered environment. Our method takes in a series of intersecting poly...
-
We present a model predictive controller (MPC) that automatically discovers collision-free locomotion while simultaneously taking into account the system dynamics, friction constraints, and kinematic limitations. A relaxed barrier function is added to the optimization’s cost function, leading to collision avoidance behavior without increasing the problem’s computational complexity. Our holistic ap...
-
This paper presents Kinetic Energy Difference (KED) as a metric for collision proximity. The calculation of KED for differentially driven robots is explained, along with an example obstacle avoidance algorithm that utilizes it. This example algorithm is computationally efficient and simulations show that it is capable of guiding robots with slow dynamics through narrow corridors.
-
We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation using a relatively simple vehicle model, including model randomization, can be successfully transferred to the real robotic setup. We achieve this by using a novel policy output regularization approach and a lifted action space which e...
-
Trajectory optimization is an important tool in task-based robot motion planning, due to its generality and convergence guarantees under some mild conditions. It is often used as a post-processing operation to smooth out trajectories that are generated by probabilistic methods or to directly control the robot motion. Unconstrained trajectory optimization problems have been well studied, and are co...
-
Anticipating the motion of dynamic objects is critical for making intelligent decisions navigating through an environment while avoiding collisions. In this work, we propose a CNN model that estimates 3D motion of objects using sequences of monocular images. We show that we can train this model without using any manual annotations by using Iterative Closest Points (ICP) to align pointclouds of an ...
-
In the field of visual ego-motion estimation for Micro Air Vehicles (MAVs), fast maneuvers stay challenging mainly because of the big visual disparity and motion blur. In the pursuit of higher robustness, we study convolutional neural networks (CNNs) that predict the relative pose between subsequent images from a fast-moving monocular camera facing a planar scene. Aided by the Inertial Measurement...
-
Monocular Visual-Inertial Odometry (VIO) has become ubiquitous for navigation of autonomous Micro Air Vehicles (MAVs). Yet, state-of-the-art VIO is still very failure-prone, which can have dramatic consequences. To prevent this, VIO must be able to re-initialize in mid-air, either during a free fall or on a constant velocity trajectory after attitude control has been re-established. However, for b...
-
We propose novel solvers for estimating the egomotion of a calibrated camera mounted to a moving vehicle from a single affine correspondence via recovering special homographies. For the first, second and third classes of solvers, the sought plane is expected to be perpendicular to one of the camera axes. For the fourth class, the plane is orthogonal to the ground with unknown normal, e.g., it is a...
-
Modeling and understanding the environment is an essential task for autonomous driving. In addition to the detection of objects, in complex traffic scenarios the motion of other road participants is of special interest. Therefore, we propose to use a recurrent neural network to predict a dynamic occupancy grid map, which divides the vehicle surrounding in cells, each containing the occupancy proba...
-
While the automatic creation of maps for localization is a widely tackled problem, the automatic inference of higher layers of HD maps is not. Additionally, approaches that learn from maps require richer and more precise landmarks than currently available.In this work, we fuse semantic detections from a monocular camera with depth and orientation estimation from lidar to automatically detect, trac...
-
Lightweight and semantically meaningful environment maps are crucial for many applications in robotics and autonomous driving to facilitate higher-level tasks such as navigation and planning. In this paper we present a novel approach to incrementally build a meaningful and lightweight semantic map directly as a 3D mesh from a monocular or stereo sequence. Our system leverages existing feature-base...
-
Biologically inspired algorithms for simultaneous localization and mapping (SLAM) such as RatSLAM have been shown to yield effective and robust robot navigation in both indoor and outdoor environments. One drawback however is the sensitivity to perceptual aliasing due to the template matching of low-dimensional sensory templates. In this paper, we propose an unsupervised representation learning me...
-
Robots need to be able to work in multiple different environments. Even when performing similar tasks, different behaviour should be deployed to best fit the current environment. In this paper, We propose a new approach to navigation, where it is treated as a multi-task learning problem. This enables the robot to learn to behave differently in visual navigation tasks for different environments whi...
-
A long short-term memory neural network is used to provide a system model that captures the temporal-dynamics of a holonomic, fully-actuated aquatic surface vehicle. As is true in many fields, new developments in robotics often are made in simulation first before being applied to real systems. To simulate an aquatic or aerial robot, a dynamic system model of the robot is required. The more represe...
-
We present a control architecture for real-time adaptation and tracking of trajectories generated using a terrain-aware trajectory optimization solver. This approach enables us to circumvent the computationally exhaustive task of online trajectory optimization, and further introduces a control solution robust to systems modeled with approximated dynamics. We train a policy using deep reinforcement...
-
In this paper, we propose a novel iterative learning control (ILC) scheme for precise state tracking of pneumatic muscle (PM) actuators. Two critical issues are considered in our scheme: 1) state constraints on PM position and velocity; 2) uncertainties of the PM model. Based on the three-element form, a PM model is constructed that takes both parametric and nonparametric uncertainties into consid...
-
3D point cloud-based place recognition is highly demanded by autonomous driving in GPS-challenged environments and serves as an essential component (i.e. loop-closure detection) in lidar-based SLAM systems. This paper proposes a novel approach, named NDT-Transformer, for real-time and large-scale place recognition using 3D point clouds. Specifically, a 3D Normal Distribution Transform (NDT) repres...
-
This paper aims at showing the dynamic performance and reliability of the low-cost, open-access quadruped robot Solo-12, which is developed within the framework of Open Dynamic Robot Initiative. It presents the implementation of a state-of-the-art control pipeline, close to the one that was previously implemented on Mini Cheetah, which implements a model predictive controller based on the centroid...
-
We present a learning algorithm for training a single policy that imitates multiple gaits of a walking robot. To achieve this, we use and extend MPC-Net, which is an Imitation Learning approach guided by Model Predictive Control (MPC). The strategy of MPC-Net differs from many other approaches since its objective is to minimize the control Hamiltonian, which derives from the principle of optimalit...
-
As locomotion decisions must be taken by considering the future, most existing quadruped controllers are based on a model predictive controller (MPC) with a reduced model of the dynamics to generate the motion and a whole- body controller to execute it. Yet the simplifying assumptions of the MPC are often chosen ad-hoc or by intuition. In this article, we focus on a set of MPCs and analyze the eff...
-
In this paper, we study the problem of dynamic interaction between a robot and an unknown object (e.g., catching a ball, or handing off an object during locomotion). In particular, we propose a method for estimating the inertial parameters of an object during dynamic interaction, while minimally altering the trajectory of the object – a minimal interaction approach. Our method combines trajectory ...
-
Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure high-resolution 3D contact geometry for a compact robot finger, to simultaneously meet optical and mechanical constraints. In this work, we present the GelSight Wedge sensor, which is optimized to have a compact shape for robot...
-
The ability to detect and estimate external contacts is essential for robot arms to operate in unstructured environments occupied by humans. However, most robot arms are not equipped with adequate sensors to detect contacts on their entire body. What many robot arms do have is torque sensors for individual joints. Through a quantitative analysis, we argue that it is fairly likely for two distinct ...
-
The discontinuities and multi-modality introduced by contacts make manipulation planning challenging. Many previous works avoid this problem by pre-designing a set of high-level motion primitives like grasping and pushing. However, such motion primitives are often not adequate to describe dexterous manipulation motions. In this work, we propose a method for dexterous manipulation planning at a mor...
-
While traditional approaches to manipulation planning assume known object templates, recent approaches to "category-level manipulation" aim to manipulate a category of objects with potentially unknown instances and large intra-category shape variation. In this paper we explore an object representation to enable precise category-level manipulation, capturing a notion of the object configuration and...
-
In many applications, including logistics and manufacturing, robot manipulators operate in semi-structured environments alongside humans or other robots. These environments are largely static, but they may contain some movable obstacles that the robot must avoid. Manipulation tasks in these applications are often highly repetitive, but require fast and reliable motion planning capabilities, often ...
-
We present a visually grounded hierarchical planning algorithm for long-horizon manipulation tasks. Our algorithm offers a joint framework of neuro-symbolic task planning and low-level motion generation conditioned on the specified goal. At the core of our approach is a two-level scene graph representation, namely geometric scene graph and symbolic scene graph. This hierarchical representation ser...
-
Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many...
-
This paper presents a novel computational model to address the problem that contact is an infinite phenomena involving continuous regions of interaction. The problem is cast as a semi-infinite program with complementarity constraints (SIPCC). Rather than pre-discretize contacting surfaces into a finite number of contact points, we use semi-infinite programming (SIP) techniques that operate on the ...
-
Robots have begun operating and collaborating with humans in industrial and social settings. This collaboration introduces challenges: the robot must plan while taking the human’s actions into account. In prior work, the problem was posed as a 2-player deterministic game, with a limited number of human moves. The limit on human moves is unintuitive, and in many settings determinism is undesirable....
-
The IKEA Furniture Assembly Environment is one of the first benchmarks for testing and accelerating the automation of long-horizon and hierarchical manipulation tasks. The environment is designed to advance reinforcement learning and imitation learning from simple toy tasks to complex tasks requiring both long-term planning and sophisticated low-level control. Our environment features 60 furniture...
-
Dexterous robotic hands are appealing for their agility and human-like morphology, yet their high degree of freedom makes learning to manipulate challenging. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop to learn grasping policies that favor the same object regions favored by pe...
-
Robots must reason about pushing and grasping in order to engage in flexible manipulation in cluttered environments. Earlier works on learning pushing and grasping only consider each operation in isolation or are limited to top-down grasping and bin-picking. We train a robot to learn joint planar pushing and 6-degree-of-freedom (6-DoF) grasping policies by self-supervision. Two separate deep neura...
-
Billions of people use chopsticks, a simple yet versatile tool, for fine manipulation of everyday objects. The small, curved, and slippery tips of chopsticks pose a challenge for picking up small objects, making them a suitably complex test case. This paper leverages human demonstrations to develop an autonomous chopsticks-equipped robotic manipulator. Due to the lack of accurate models for fine m...
-
Industrial automation requires robot dexterity to automate many processes such as product assembling, packaging, and material handling. The existing robotic systems lack the capability to determining proper grasp strategies in the context of object affordances and task designations. In this paper, a framework of task-oriented dexterous grasping is proposed to learn grasp knowledge from human exper...
-
This paper enhances the feedback linearization controller for multirotors with a learned acceleration error model and a thrust input delay mitigation model. Feedback linearization controllers are theoretically appealing but their performance suffers on real systems, where the true system does not match the known system model. We take a step in reducing these robustness issues by learning an accele...
-
In this paper, we introduce Cirrus, a new long-range bi-pattern LiDAR public dataset for autonomous driving tasks such as 3D object detection, critical to highway driving and timely decision making. Our platform is equipped with a high-resolution video camera and a pair of LiDAR sensors with a 250-meter effective range, which is significantly longer than existing public datasets. We record paired ...
-
We present a dead reckoning strategy for increased resilience to position estimation failures on multirotors, using only data from a low-cost IMU and novel, bio-inspired airflow sensors. The goal is challenging, since low-cost IMUs are subject to large noise and drift, while 3D airflow sensing is made difficult by the interference caused by the propellers and by the wind. Our approach relies on a ...
-
This paper introduces a real-time dense planar LiDAR SLAM system, named π-LSAM, for the indoor environment. The widely used LiDAR odometry and mapping (LOAM) framework [1] does not include bundle adjustment (BA) and generates a low fidelity tracking pose. This paper seeks to overcome these drawbacks for the indoor environment. Specifically, we use the plane as the landmark, and introduce plane adj...
-
We propose a methodology for robust, real-time place recognition using an imaging lidar, which yields image-quality high-resolution 3D point clouds. Utilizing the intensity readings of an imaging lidar, we project the point cloud and obtain an intensity image. ORB feature descriptors are extracted from the image and encoded into a bag-of-words vector. The vector, used to identify the point cloud, ...
-
Safe and high-speed navigation is a key enabling capability for real world deployment of robotic systems. A significant limitation of existing approaches is the computational bottleneck associated with explicit mapping and the limited field of view (FOV) of existing sensor technologies. In this paper, we study algorithmic approaches that allow the robot to predict spaces extending beyond the senso...
-
This paper describes the development of a single reduced-order dynamic model that captures running on land, running while submerged, and for the first time swimming on the surface of water. By capturing the effect of fluid forces on both the body and the leg, the Spring-Loaded Inverted Pendulum (SLIP) model is extended to operate in amphibious and aquatic regimes. Three distinct stable motion patt...
-
Understanding the gap between simulation and reality is critical for reinforcement learning with legged robots, which are largely trained in simulation. However, recent work has resulted in sometimes conflicting conclusions with regard to which factors are important for success, including the role of dynamics randomization. In this paper, we aim to provide clarity and understanding on the role of ...
-
Everyday contact-rich tasks, such as peeling, cleaning, and writing, demand multimodal perception for effective and precise task execution. However, these present a novel challenge to robots as they lack the ability to combine these multimodal stimuli for performing contact-rich tasks. Learning-based methods have attempted to model multi-modal contact-rich tasks, but they often require extensive t...
-
We propose a Deep Interaction Prediction Network (DIPN) for learning to predict complex interactions that ensue as a robot end-effector pushes multiple objects, whose physical properties, including size, shape, mass, and friction coefficients may be unknown a priori. DIPN "imagines" the effect of a push action and generates an accurate synthetic image of the predicted outcome. DIPN is shown to be ...
-
Earlier work has shown that reusing experience from prior motion planning problems can improve the efficiency of similar, future motion planning queries. However, for robots with many degrees-of-freedom, these methods exhibit poor generalization across different environments and often require large datasets that are impractical to gather. We present SPARK and FLAME, two experience-based frameworks...
-
We propose a novel planning technique for satisfying tasks specified in temporal logic in partially revealed environments. We define high-level actions derived from the environment and the given task itself, and estimate how each action contributes to progress towards completing the task. As the map is revealed, we estimate the cost and probability of success of each action from images and an enco...
-
We present an algorithm for learning behavior trees for robotic task planning, which alleviates the need for time-intensive or infeasible manual design of control architectures. Our method involves representing the search space of behavior trees as a formal grammar and searching over this grammar by means of a new generalization of Monte Carlo tree search (MCTS) for directed acyclic graphs (DAGs),...
-
Autonomous ground vehicles have improved greatly over the past decades, but they still have their limitations when it comes to off-road environments. There is still a need for planning techniques that effectively handle physical interactions between a vehicle and its surroundings. We present a method of modifying a standard path planning algorithm to address these problems by incorporating a learn...
-
Teleoperation—i.e., controlling a robot with human motion—proves promising in enabling a humanoid robot to move as dynamically as a human. But how to map human motion to a humanoid robot matters because a human and a humanoid robot rarely have identical topologies and dimensions. This work presents an experimental study that utilizes reaction tests to compare joint space and task space mappings fo...
-
Interactive driving scenarios, such as lane changes, merges and unprotected turns, are some of the most challenging situations for autonomous driving. Planning in interactive scenarios requires accurately modeling the reactions of other agents to different future actions of the ego agent. We develop end-to-end models for conditional behavior prediction (CBP) that take as an input a query future tr...
-
This paper presents a technique that allows a robot to escort a human to their destination. Unlike tracking where the robot follows the human from behind, the proposed technique locates the robot in front of the human by incorporating human intention in addition to conventional motion prediction. Human head pose is used as an effective past-proven implicit indicator of intention. A particle filter...
-
To effectively assist human workers in assembly tasks a robot must proactively offer support by inferring their preferences in sequencing the task actions. Previous work has focused on learning the dominant preferences of human workers for simple tasks largely based on their intended goal. However, people may have preferences at different resolutions: they may share the same high-level preference ...
-
As environments involving both robots and humans become increasingly common, so does the need to account for people during planning. To plan effectively, robots must be able to respond to and sometimes influence what humans do. This requires a human model which predicts future human actions. A simple model may assume the human will continue what they did previously; a more complex one might predic...
-
As robots work with human teams, they will be expected to fluently coordinate with them. While people are adept at coordination and real-time adaptation, robots still lack this skill. In this paper, we introduce TANDEM: Temporal Anticipation and Adaptation for Machines, a series of neurobiologically-inspired algorithms that enable robots to fluently coordinate with people. TANDEM leverages a human...
-
To enable robots to smoothly interact with humans during their travels together as a group, robots need the ability to adapt their motions under environmental changes and ensure all group members’ routes are feasible. To achieve this ability, robots require knowledge of the final destination and the subgoals in between. In practice, such information is seldom shared explicitly among group members,...
-
Prior work on generating explanations in a planning context has focused on providing the rationale behind an AI agent’s decision-making. While these methods offer the right explanations, they fail to heed the cognitive requirement of understanding an explanation from the explainee or human’s perspective. In this work, we set out to address this issue by considering the order for communicating info...
-
In shared autonomy, the user input is blended with the assistive motion to accomplish a task where the user goal is typically unknown to the robot. Transparency between the human and robot is essential for effective collaboration. Prior works have provided methods for the robot to infer the user goal; however, they are usually dependent on the distance between the robot and object, which may not b...
-
We outline a neural network-based pipeline for perception, control and planning of a 7 DoF robot for tasks that involve reaching into a dormant grapevine canopy. The proposed system consists of a 6 DoF industrial robot arm and a linear slider that can actuate on an entire grape vine. Our approach uses Convolutional Neural Networks to detect buds in dormant grape vines and a Reinforcement Learning ...
-
We propose a new generative model-based predictive display for robotic teleoperation over high-latency communication links. Our method is capable of rendering photo-realistic images of the scene to the human operator in real time from RGB-D images acquired by the remote robot. A preliminary exploration stage is used to build a coarse 3D map of the remote environment and to train a generative model...
-
Effective social robots should leverage humor’s unique ability to improve relationship connections and dispel stress, but current robots possess limited (if any) humorous abilities. In this paper, we aim to supplement one aspect of autonomous robots by giving robotic systems the ability to "read the room" to assess how their humorous statements are received by nearby people in real time. Using a d...
-
Oysters are an essential species in the Chesapeake Bay living ecosystem. Oysters are filter feeders and considered the vacuum cleaners of the Chesapeake Bay that can considerably improve the Bay's water quality. Many oyster restoration programs have been initiated in the past decades and continued to date. Advancements in robotics and artificial intelligence have opened new opportunities for aquac...
-
This paper contributes a method designed to enable autonomous distributed 3D nuclear radiation field mapping. The algorithm uses a single radiation sensor and a sequence of spatially distributed and robotically acquired radiation measurements across a discretized 3D grid to derive a radiation gradient. The derived gradient is probabilistically propagated to unknown components of the map to further...
-
This paper presents a novel omnidirectional gait design and feedback control of a radially symmetric tripedal friction-driven robot. The robot features 3 servo motors mounted on a 3-D printed chassis 7 cm from the center of mass and separated 120 degrees. These motors drive limbs, which impart frictional reactive forces on the body. We first introduce a mathematical model for the robot motion, the...
-
Steerable needles that are able to follow curvilinear trajectories and steer around anatomical obstacles are a promising solution for many interventional procedures. In the lung, these needles can be deployed from the tip of a conventional bronchoscope to reach lung lesions for diagnosis. The reach of such a device depends on several design parameters including the bronchoscope diameter, the angle...
-
Derivatives of equations of motion describing the rigid body dynamics are becoming increasingly relevant for the robotics community and find many applications in design and control of robotic systems. Controlling robots, and multibody systems comprising elastic components in particular, not only requires smooth trajectories but also the time derivatives of the control forces/torques, hence of the ...
-
The inverse dynamics of a robotic manipulator is instrumental in precise robot control and manipulation. However, acquiring such a model is challenging, not only due to unmodelled non-linearities such as joint friction, but also from a machine learning perspective (e.g., input space dimension, amount of data needed). The accuracy of such models, regardless of the learning techniques, relies on pro...
-
We present a novel algorithm for self-supervised monocular depth completion. Our approach is based on training a neural network that requires only sparse depth measurements and corresponding monocular video sequences without dense depth labels. Our self-supervised algorithm is designed for challenging indoor environments with textureless regions, glossy and transparent surfaces, moving people, lon...
-
Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy. Humans do this everyday, relying on touch and proprioceptive feedback in visually-challenging environments. However, robots might not always know when their sensors are corrupted, as even broken sensors can return valid values...
-
Robots can perform various types of automated movements in the workspace. In recent years, robot applications have been expanded to a much wider scope, including robot machining, robot assembly, robot 3D printing, robot inspection, etc. Many of these applications require robots to have higher absolute accuracy compared with conventional robot part handling and welding. The capability to assess a r...
-
We present the design, implementation, and evaluation of RF-Grasp, a robotic system that can grasp fully-occluded objects in unknown and unstructured environments. Unlike prior systems that are constrained by the line-of-sight perception of vision and infrared sensors, RF-Grasp employs RF (Radio Frequency) perception to identify and locate target objects through occlusions, and perform efficient e...
-
Composites are increasingly becoming a material of choice in the aerospace and automotive industries. Currently, many composite parts are produced by manually laying up sheets on complex molds. Composite sheet layup requires executing two main tasks: (1) grasping a sheet and (2) draping it on the mold. Automating the layup process requires automation of these two tasks. This paper is focused on th...
-
The present work focuses on the development of an efficient path controller to guide a fixed-wing UAV (Unmanned Aerial Vehicle) to follow a closed curve and avoid unknown dynamic obstacles. Our strategy is composed of two layers: a top level layer responsible for guidance and a lower level layer responsible for tracking the references given by the top level. To solve the guidance problem, we propo...
-
Collision detection and recovery for aerial robots remain a challenge because of the limited space for sensors and local stability of the flight controller. We introduce a novel collision-resilient quadrotor that features a compliant arm design to enable free flight while allowing for one passive degree of freedom to absorb shocks. We further propose a novel collision detection and characterizatio...
-
Unmanned aerial vehicles have been demonstrated successfully in a variety of tasks, including surveying and sampling tasks over large areas. These vehicles can take many forms. Quadrotors’ agility and ability to hover makes them well suited for navigating potentially tight spaces, while fixed wing aircraft are capable of efficient flight over long distances. Hybrid aerial vehicles (HAVs) attempt t...
-
Traditional aerial vehicles are usually custom-designed for specific tasks. Although they offer an efficient solution, they are not always able to adapt to changes in the task specification, e.g., increasing the payload. This applies to quadrotors, having a maximum payload and only four controllable degrees of freedom, limiting their adaptability to the task’s variations. We propose a versatile mo...
-
We tackle the problem of multiple quadrotors transporting a cable-suspended point-mass load. The quadrotors are treated as a virtual leader-follower algorithm, where a multi-layer graph encapsulates the communication and physical interaction. On the one hand, the communication stands for the approach of following the reference trajectory of a virtual leader. On the other hand, the load exerts a di...
-
Uncovering potential failure cases is a crucial step in the validation of safety critical systems such as autonomous vehicles. Failure search may be done through logging substantial vehicle miles in either simulation or real world testing. Due to the sparsity of failure events, naive random search approaches require significant amounts of vehicle operation hours to find potential system weaknesses...
-
Transferring reinforcement learning policies trained in physics simulation to the real hardware remains a challenge, known as the "sim-to-real" gap. Domain randomization is a simple yet effective technique to address dynamics discrepancies across source and target domains, but its success generally depends on heuristics and trial-and-error. In this work we investigate the impact of randomized para...
-
This paper addresses the long-standing open problem of observability of mass and inertia plant parameters in the adaptive identification (AID) of second-order nonlinear models of 6 degree-of-freedom rigid-body dynamical systems subject to externally applied forces and moments. Although stable methods for AID of plant parameters for this class of systems, as well numerous approaches to stable model...
-
Temporal task planning guarantees a robot will succeed in its task as long as certain explicit and implicit assumptions about the robot’s operating environment, sensors, and capabilities hold. A robot executing a plan can silently fail to fulfill the task if the assumptions are violated at runtime. Monitoring assumption violations at runtime can flag silent failures and also provide mitigation and...
-
This paper presents a novel policy representation for partially observable Markov decision processes (POMDPs) called circulant controllers and a provably efficient gradient-based algorithm for them. A formal mathematical description is provided that leverages circulant matrices for the controller’s stochastic node transitions. This structure is particularly effective for capturing decision-making ...
-
We present non-convex maximal dissipation principle (NMDP), a time integration scheme for articulated bodies with simultaneous contacts. Our scheme resolves contact forces via the maximal dissipation principle (MDP). Whereas prior MDP solvers assume linearized dynamics and integrate using the forward multistep scheme, we consider the coupled system of nonlinear Newton-Euler dynamics and MDP and in...
-
The potential benefits of model-free reinforcement learning to real robotics systems are limited by its uninformed exploration that leads to slow convergence, lack of data-efficiency, and unnecessary interactions with the environment. To address these drawbacks we propose a method that combines reinforcement and imitation learning by shaping the reward function with a state-and-action-dependent po...
-
Can we use reinforcement learning to learn general-purpose policies that can perform a wide range of different tasks, resulting in flexible and reusable skills? Contextual policies provide this capability in principle, but the representation of the context determines the degree of generalization and expressivity. Categorical contexts preclude generalization to entirely new tasks. Goal-conditioned ...
-
The process of learning a manipulation task depends strongly on the action space used for exploration: posed in the incorrect action space, solving a task with reinforcement learning can be drastically inefficient. Additionally, similar tasks or instances of the same task family impose latent manifold constraints on the most effective action space: the task family can be best solved with actions i...
-
Attempts to achieve robotic Within-Hand-Manipulation (WIHM) generally utilize either high-DOF robotic hands with elaborate sensing apparatus or multi-arm robotic systems. In prior work we presented a simple robot hand with variable friction robot fingers, which allow a low-complexity approach to within-hand object translation and rotation, though this manipulation was limited to planar actions. In...
-
Multi-stage forceful manipulation tasks, such as twisting a nut on a bolt, require reasoning over interlocking constraints over discrete and continuous choices. The robot must choose a sequence of discrete actions, or strategy, such as whether to pick up an object, and the continuous parameters of each of those actions, such as how to grasp that object. In forceful manipulation tasks, the force re...
-
Distributed manipulators - consisting of a set of actuators or robots working cooperatively to achieve a manipulation task - are robust and flexible tools for performing a range of planar manipulation skills. One novel example is the delta array, a distributed manipulator composed of a grid of delta robots, capable of performing dexterous manipulation tasks using strategies incorporating both dyna...
-
Robot manipulation in cluttered scenes often requires contact-rich interactions with objects. It can be more economical to interact via non-prehensile actions, for example, push through other objects to get to the desired grasp pose, instead of deliberate prehensile rearrangement of the scene. For each object in a scene, depending on its properties, the robot may or may not be allowed to make cont...
-
We present a novel approach to robotic grasp planning using both a learned grasp proposal network and a learned 3D shape reconstruction network. Our system generates 6-DOF grasps from a single RGB-D image of the target object, which is provided as input to both networks. By using the geometric reconstruction to refine the candidate grasp produced by the grasp proposal network, our system is able t...
-
Robotic grasping is one of the most fundamental robotic manipulation tasks and has been actively studied. However, how to quickly teach a robot to grasp a novel target object in clutter remains challenging. This paper attempts to tackle the challenge by leveraging object attributes that facilitate recognition, grasping, and quick adaptation. In this work, we introduce an end-to-end learning method...
-
Grasping a novel target object in constrained environments (e.g., walls, bins, and shelves) requires intensive reasoning about grasp pose reachability to avoid collisions with the surrounding structures. Typical 6-DoF robotic grasping systems rely on the prior knowledge about the environment and intensive planning computation, which is ungeneralizable and inefficient. In contrast, we propose a nov...
-
Proposing grasp poses for novel objects is an essential component for any robot manipulation task. Planning six degrees of freedom (DoF) grasps with a single camera, however, is challenging due to the complex object shape, incomplete object information, and sensor noise. In this paper, we present a 6-DoF contrastive grasp proposal network (CGPN) to infer 6-DoF grasps from a single-view depth image...
-
We present a planner for large-scale (un)labeled object sorting tasks, which uses two types of manipulation actions: overhead grasping and planar pushing. The grasping action offers completeness guarantee under mild assumptions, and the planar pushing is an acceleration strategy that moves multiple objects at once. We make two main contributions: (1) We propose a bilevel planning algorithm. Our hi...
-
Planning in realistic environments requires searching in large planning spaces. Affordances are a powerful concept to simplify this search, because they model what actions can be successful in a given situation. However, the classical notion of affordance is not suitable for long horizon planning because it only informs the robot about the immediate outcome of actions instead of what actions are b...
-
Rewards play a crucial role in reinforcement learning. To arrive at the desired policy, the design of a suitable reward function often requires significant domain expertise as well as trial-and-error. Here, we aim to minimize the effort involved in designing reward functions for contact-rich manipulation tasks. In particular, we provide an approach capable of extracting dense reward functions algo...
-
We introduce ACRONYM, a dataset for robot grasp planning based on physics simulation. The dataset contains 17.7M parallel-jaw grasps, spanning 8872 objects from 262 different categories, each labeled with the grasp result obtained from a physics simulator. We show the value of this large and diverse dataset by using it to train two state-of-the-art learning-based grasp planning algorithms. Grasp p...
-
We present a novel Deep Reinforcement Learning (DRL) based policy to compute dynamically feasible and spatially aware velocities for a robot navigating among mobile obstacles. Our approach combines the benefits of the Dynamic Window Approach (DWA) in terms of satisfying the robot’s dynamics constraints with state-of-the-art DRL-based navigation methods that can handle moving obstacles and pedestri...
-
Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex autonomous driving scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In this work, we show that explicitly inferring the latent state and encoding spatial-temporal r...
-
Ranging systems can provide inexpensive, accurate, energy- and computationally-efficient navigation solutions for mobile robots. This work focuses on location and pose estimation in ranging networks composed of anchors with known positions as well as mobile robots modeled as rigid bodies, each carrying multiple tags to localize. Noisy distance measurements can be obtained between a subset of the n...
-
This paper considers the use of two position receivers and an inertial measurement unit (IMU) to estimate the position, velocity, and attitude of a rigid body, collectively called extended pose. The measurement model consisting of the position of one receiver and the relative position between the two receivers is left invariant, enabling the use of the invariant extended Kalman filter (IEKF) frame...
-
This paper introduces the Compartmentalized Covariance Intersection (CCI) algorithm, a consistent technique to fuse measurements in cooperative navigation networks. The algorithm reduces the excess conservatism of standard Covariance Intersection (CI) by assuming that correlation is only present within each measurement stream and not across the different sources. This assumption allows the sources...
-
Accurate motion capture of aerial robots in 3D is a key enabler for autonomous operation in indoor environments such as warehouses or factories, as well as driving forward research in these areas. The most commonly used solutions at present are optical motion capture (e.g. VICON) and Ultrawide-band (UWB), but these are costly and cumbersome to deploy, due to their requirement of multiple cameras/a...
-
This paper studies the problem of autonomous exploration under localization uncertainty for a mobile robot with 3D range sensing. We present a framework for self-learning a high-performance exploration policy in a single simulation environment, and transferring it to other environments, which may be physical or virtual. Recent work in transfer learning achieves encouraging performance by domain ad...
-
Deep learning-based object pose estimators are often unreliable and overconfident especially when the input image is outside the training domain, for instance, with sim2real transfer. Efficient and robust uncertainty quantification (UQ) in pose estimators is critically needed in many robotic tasks. In this work, we propose a simple, efficient, and plug-and-play UQ method for 6-DoF object pose esti...
-
This paper addresses outdoor terrain mapping using overhead images obtained from an unmanned aerial vehicle. Dense depth estimation from aerial images during flight is challenging. While feature-based localization and mapping techniques can deliver real-time odometry and sparse points reconstruction, a dense environment model is generally recovered offline with significant computation and storage....
-
We present an ensemble learning methodology that combines multiple existing robotic grasp synthesis algorithms and obtain a success rate that is significantly better than the individual algorithms. The methodology treats the grasping algorithms as "experts" providing grasp "opinions". An Ensemble Convolutional Neural Network (ECNN) is trained using a Mixture of Experts (MOE) model that integrates ...
-
We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with o...
-
Robotic automation in surgery requires precise tracking of surgical tools and mapping of deformable tissue. Previous works on surgical perception frameworks require significant effort in developing features for surgical tool and tissue tracking. In this work, we overcome the challenge by exploiting deep learning methods for surgical perception. We integrated deep neural networks, capable of effici...
-
In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input. The attention module specifically targets motion planning, whereas prior literature only applied attention in perception tasks. Learning an attention mask directly targeted for motion planning significantly improves the planner safe...
-
When personal, assistive, and interactive robots make mistakes, humans naturally and intuitively correct those mistakes through physical interaction. In simple situations, one correction is sufficient to convey what the human wants. But when humans are working with multiple robots or the robot is performing an intricate task often the human must make several corrections to fix the robot’s behavior...
-
SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning
As learning-based approaches progress towards automating robot controllers design, transferring learned policies to new domains with different dynamics (e.g. sim-to-real transfer) still demands manual effort. This paper introduces SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics simulator to match the simulated trajectories to the ones from the target domain, using a...
-
To function autonomously in the physical world, humanoid robots need high-fidelity sensing systems, especially for forces that cannot be easily modeled. Modeling forces in robot feet is particularly challenging due to static indeterminacy, thereby requiring direct sensing. Unfortunately, resolving forces in the feet of some smaller-sized humanoids is limited both by the quality of sensors and the ...
-
This paper presents a 3D pointing interface application to signal a UAV’s target in a large-scale environment. This system enables UAVs equipped with a monocular camera to determine which window of a building is selected by a human user in large-scale indoor or outdoor environments. The 3D pointing interface consists of three parts: YOLO, Open- Pose, and ORB-SLAM. YOLO detects the target objects, ...
-
Robot programming typically makes use of a set of mechanical skills that is acquired by machine learning. Because there is in general no guarantee that machine learning produces robot programs that are free of surprising behavior, the safe execution of a robot program must utilize monitoring modules that take sensor data as inputs in real time to ensure the correctness of the skill execution. Owin...
-
Complex environments can cause robots to fail. Identifying the key elements of the environment associated with such failures is critical for faster fault isolation and, ultimately, debugging those failures. In this work we present the first automated approach for reducing the environment in which a robot failed. Similar to software debugging techniques, our approach systematically performs a parti...
-
Human-robot collaboration frequently requires extensive communication, e.g., using natural language and gesture. Augmented reality (AR) has provided an alternative way of bridging the communication gap between robots and people. However, most current AR-based human-robot communication methods are unidirectional, focusing on how the human adapts to robot behaviors, and are limited to single-robot d...
-
Learning from Demonstration (LfD) enables novice users to teach robots new skills. However, many LfD methods do not facilitate skill maintenance and adaptation. Changes in task requirements or in the environment often reveal the lack of resiliency and adaptability in the skill model. To overcome these limitations, we introduce ARC-LfD: an Augmented Reality (AR) interface for constrained Learning f...
-
Lovable robots in movies regularly beep, chirp, and whirr, yet robots in the real world rarely deploy such sounds. Despite preliminary work supporting the perceptual and objective benefits of intentionally-produced robot sound, relatively little research is ongoing in this area. In this paper, we systematically evaluate transformative robot sound across multiple robot archetypes and behaviors. We ...
-
Robot teleoperation is a reliable way to perform a variety of tasks with complex robotic systems. However, the remote control of active telepresence cameras on the robot for improved telepresence adds an additional degree of complexity while teleoperating and can thus affect the operator’s performance during tele-manipulation. Our previous user study investigates the general human performance and ...
-
The emergency department (ED) is a safety-critical environment in which healthcare workers (HCWs) are overburdened, overworked, and have limited resources, especially during the COVID-19 pandemic. One way to address this problem is to explore the use of robots that can support clinical teams, e.g., to deliver materials or restock supplies. However, due to EDs being overcrowded, and the cognitive o...
-
Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structura...
-
In this paper, we investigate how heterogeneous multi-robot systems with different sensing capabilities can observe a domain with an a priori unknown density function. Common coverage control techniques are targeted towards homogeneous teams of robots and do not consider what happens when the sensing capabilities of the robots are vastly different. This work proposes an extension to Lloyd’s algori...
-
Service robots are gaining capabilities to be deployed in public environments for human assistance. While robot actively providing guidance has shown great success in field study, the communication strategy (the strategy to decide whom to initiate the service for and when), and hence the performance evaluation, has been based on behavioral-based qualitative analysis. We attribute this to the chall...
-
This paper investigates a hybrid solution which combines deep reinforcement learning (RL) and classical trajectory planning for the "following in front" application. Here, an autonomous robot aims to stay ahead of a person as the person freely walks around. Following in front is a challenging problem as the user’s intended trajectory is unknown and needs to be estimated, explicitly or implicitly, ...
-
Human-robot object handovers have been an actively studied area of robotics over the past decade; however, very few techniques and systems have addressed the challenge of handing over diverse objects with arbitrary appearance, size, shape, and deformability. In this paper, we present a vision-based system that enables reactive human-to-robot handovers of unknown objects. Our approach combines clos...
-
Ability to generate intelligent and generalizable facial expressions is essential for building human-like social robots. At present, progress in this field is hindered by the fact that each facial expression needs to be programmed by humans. In order to adapt robot behavior in real time to different situations that arise when interacting with human subjects, robots need to be able to train themsel...
-
Assistive robots have the potential to help people perform everyday tasks. However, these robots first need to learn what it is their user wants them to do. Teaching assistive robots is hard for inexperienced users, elderly users, and users living with physical disabilities, since often these individuals are unable to show the robot their desired behavior. We know that inclusive learners should gi...
-
Predictive human models often need to adapt their parameters online from human data. This raises previously ignored safety-related questions for robots relying on these models such as what the model could learn online and how quickly could it learn it. For instance, when will the robot have a confident estimate in a nearby human’s goal? Or, what parameter initializations guarantee that the robot c...
-
Human-robot interactions (HRI) can be modeled as differential games with incomplete information, where each agent holds private reward parameters. Due to the open challenge in finding perfect Bayesian equilibria of such games, existing studies often decouple the belief and physical dynamics by iterating between belief update and motion planning. Importantly, the robot’s reward parameters are often...
-
Despite the advances in autonomous navigation and motion planning, there are still several challenges to overcome, especially for confined or underground spaces. Confined scenarios present challenges such as lack of global or accurate external localization, uneven and slippery terrains, and multilevel stages. Exploring and mapping unknown unstructured environments is a fundamental step into the sa...
-
Mobile robots in unstructured, mapless environments must rely on an obstacle avoidance module to navigate safely. The standard avoidance techniques estimate the locations of obstacles with respect to the robot but are unaware of the obstacles’ identities. Consequently, the robot cannot take advantage of semantic information about obstacles when making decisions about how to navigate. We propose an...
-
We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two branch structure. We embedded Gated-SCNN into the segmentor component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we...
-
In order to achieve autonomous vertical wall climbing, the transition phase from the ground to the wall requires extra consideration inevitably. This paper focuses on the contact sequence planner to transition between flat terrain and vertical surfaces for multi-limbed climbing robots. To overcome the transition phase, it requires planning both multicontact and contact wrenches simultaneously whic...
-
We present a trajectory planning and control architecture for bipedal locomotion at a variety of speeds on a highly underactuated and compliant bipedal robot. A library of compliant walking trajectories are planned offline, and stored as compact arrays of polynomial coefficients for tracking online. The control implementation uses a floating-base inverse dynamics controller which generates dynamic...
-
Industrial manipulators do not collapse under their own weight when powered off due to the friction in their joints. Although these mechanism are effective for stiff position control of pick-and-place, they are inappropriate for legged robots that must rapidly regulate compliant interactions with the environment. However, no metric exists to quantify the robot’s performance degradation due to mech...
-
The lack of stability guarantee restricts the practical use of learning-based methods in core control problems in robotics. We develop new methods for learning neural control policies and neural Lyapunov critic functions in the modelfree reinforcement learning (RL) setting. We use sample-based approaches and the Almost Lyapunov function conditions to estimate the region of attraction and invarianc...
-
A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS),...
-
Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical control systems. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the ability to deal with state and input constraints. However, it involves solving a PDE, whose computational ...
-
We solve active target tracking, one of the essential tasks in autonomous systems, using a deep reinforcement learning (RL) approach. In this problem, an autonomous agent is tasked with acquiring information about targets of interests using its on-board sensors. The classical challenges in this problem are system model dependence and the difficulty of computing information-theoretic cost functions...
-
Since the application of Deep Q-Learning to the continuous action domain in Atari-like games, Deep Reinforcement Learning (Deep-RL) techniques for motion control have been qualitatively enhanced. Nowadays, modern Deep-RL can be successfully applied to solve a wide range of complex decision-making tasks for many types of vehicles. Based on this context, in this paper, we propose the use of Deep-RL ...
-
This paper presents a novel non-Gaussian inference algorithm, Normalizing Flow iSAM (NF-iSAM), for solving SLAM problems with non-Gaussian factors and/or non-linear measurement models. NF-iSAM exploits the expressive power of neural networks, and trains normalizing flows to draw samples from the joint posterior of non-Gaussian factor graphs. By leveraging the Bayes tree, NF-iSAM is able to exploit...
-
We present an empirical investigation of a new mapping system based on a graph of panoramic depth images. Panoramic images efficiently capture range measurements taken by a spinning lidar sensor, recording fine detail on the order of a few centimeters within maps of expansive scope on the order of tens of millions of cubic meters. The flexibility of the system is demonstrated by running the same map...
-
Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data, however existing autonomy datasets either represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in...
-
Safety is a critical property in applications including robotics, transportation, and energy. Safety is especially challenging in reinforcement learning (RL) settings, in which uncertainty of the system dynamics may cause safety violations during exploration. Control Barrier Functions (CBFs), which enforce safety by constraining the control actions at each time step, are a promising approach for s...
-
Effective planning in model-based reinforcement learning (MBRL) and model-predictive control (MPC) relies on the accuracy of the learned dynamics model. In many instances of MBRL and MPC, this model is assumed to be stationary and is periodically re-trained from scratch on state transition experience collected from the beginning of environment interactions. This implies that the time required to t...
-
This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of complex tasks, which are expressed by linear temporal logic (LTL) specifications. Due to the consideration of environment and motion uncertainties, we model the robot motion as a probabilistic labeled Markov decision process (PL-MDP) with unknown tran...
-
Autonomous systems often use approximate planners that exploit state abstractions to solve large MDPs in real-time decision-making problems. However, these planners can eliminate details needed to produce effective behavior in autonomous systems. We therefore propose a novel model, a partially abstract MDP, with a set of abstract states that each compress a set of ground states to condense irrelev...
-
Energy management is a critical aspect of risk assessment for Uncrewed Aerial Vehicle (UAV) flights, as a depleted battery during a flight brings almost guaranteed vehicle damage and a high risk of human injuries or property damage. Predicting the amount of energy a flight will consume is challenging as routing, weather, obstacles, and other factors affect the overall consumption. We develop a dee...
-
In this work, we present an adaptive behavior path planning method for autonomous exploration and visual search of unknown environments. As volumetric exploration and visual coverage of unknown environments, with possibly different sensors, are non-identical objectives, a principled combination of the two is proposed. In particular, the method involves three distinct planning policies, namely expl...
-
This paper presents our comparative study on how the flight performances of an in-flight morphing quad-rotor are affected by the morph induced inertia variation. A custom-built in-flight morphing quad-rotor was employed in numerical and experimental tests for the study and analysis. In these tests, the quad-rotor is controlled to follow a predefined path and/or to hover in an environment with the ...
-
The commonly used metrics for motion prediction do not correlate well with a self-driving vehicle’s system-level performance. The most common metrics are average displacement error (ADE) and final displacement error (FDE), which omit many features, making them poor self-driving performance indicators. Since high-fidelity simulations and track testing can be resource-intensive, the use of predictio...
-
Assembly systems are commonly seen in production practice, where multiple components are joined in a manufacturing process to make a final product. In this paper, a decomposition/aggregation-based method is presented to evaluate the performance metrics of assembly systems with machines following the exponential reliability model (either synchronous or asynchronous). In particular, we consider the ...
-
Polyculture farming is a sustainable farming technique based on synergistic interactions between differing plant types that make them more resistant to diseases and pests and better able to retain water. Reduced uniformity can reduce use of pesticides, fertilizer, and water, but is more labor intensive and more challenging to automate. We describe a scaled physical testbed (1.5m×3.0m) that uses a ...
-
Data synthesizing is crucial for data-driven anomaly prognostics on physical machines. We propose the first general-purpose anomalous scenario synthesizer, GPASS, for rotary equipment. More specifically, we present a design of implementing modular rotational damping, large lateral force, with high-frequency range capability as fundamental modes of physical inputs. The GPASS is a general-purpose pl...
-
Research in autonomous robots for construction has largely focused on ground-based robots whose reach constrains the size of what they can build, or on climbing or aerial robots that build solid or unroofed structures. Autonomous construction of larger, multistory buildings, or bridges spanning unsupported distances, would require robots that build sturdy structures supporting their own weight. In...
-
Robotic hopping requires high performance and precision, due to its extreme interactions with the environment. Designing a system that will perform optimally, or even stably, for this motion primitive is a significant challenge. In previous work, it was shown that designing a robot with two springs (one in series and one in parallel with the actuator) could dramatically improve performance. Howeve...
-
Learning object manipulation is a critical skill for robots to interact with their environment. Even though there has been significant progress in robotic manipulation of rigid objects, interacting with non-rigid objects remains challenging for robots. In this work, we introduce velcro peeling as a new application for robotic manipulation of non-rigid objects in complex environments. We present a ...
-
Reinforcement Learning (RL) algorithms can in principle acquire complex robotic skills by learning from large amounts of data in the real world, collected via trial and error. However, most RL algorithms use a carefully engineered setup in order to collect data, requiring human supervision and intervention to provide episodic resets. This is particularly evident in challenging robotics problems, s...
-
Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning
Substantial advancements to model-based reinforcement learning algorithms have been impeded by the model-bias induced by the collected data, which generally hurts performance. Meanwhile, their inherent sample efficiency warrants utility for most robot applications, limiting potential damage to the robot and its environment during training. Inspired by information theoretic model predictive control...
-
Slicing is an important skill for a robot to learn as it is more efficient and results in less deformation in comparison with cutting by pressing. Cutting experiments with foods have indicated that the ease of slicing is caused by a decrease in fracture toughness. In this paper, we formally characterize this decrease based on the work needed to maintain the critical strain for fracture. Forces gen...
-
Policies trained in simulation often fail when transferred to the real world due to the ‘reality gap’ where the simulator is unable to accurately capture the dynamics and visual properties of the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering to determine how much to randomize system parameters in order to learn a policy...
-
Motion planning for robotic manipulation makes heavy use of quasistatic models, but these same models have not yet proven useful for simulation. This is because in many multi-contact situations, the quasistatic models do not describe a unique next state for the system. A planner is able to use these models optimistically (checking only for feasibility of a motion), but simulation requires more.In ...
-
Object rearrangement is a widely-applicable and challenging task for robots. Geometric constraints must be carefully examined to avoid collisions and combinatorial issues arise as the number of objects increases. This work studies the algorithmic structure of rearranging uniform objects, where robot-object collisions do not occur but object-object collisions have to be avoided. The objective is mi...
-
Ceramics are one of the major sources of information about the past for archaeologists, with a typical archaeological dig unearthing 1000’s of pottery fragments (sherds) each day. However, archaeologists often are not allowed to remove these sherds from their home countries. Therefore, logging data (e.g., mass, color, decoration) in the field is the only way to record valuable information about th...
-
Complex manipulation tasks, such as rearrangement planning of numerous objects, are combinatorially hard problems. Existing algorithms either do not scale well or assume a great deal of prior knowledge about the environment, and few offer any rigorous guarantees. In this paper, we propose a novel hybrid control architecture for achieving such tasks with mobile manipulators. On the discrete side, w...
-
In this paper, we aim to solve inverse kinematics of the integrated robotic arm-hand systems to achieve precision grasping, provided the desired grasp configuration (contact points + contact normals). The key insights of our approach are three-fold. First, we propose a human-inspired thumb-first strategy and consider one finger of the robotic hand as the "thumb" to narrow down the search space and...
-
Object insertion is a classic contact-rich manipulation task. The task remains challenging, especially when considering general objects of unknown geometry, which significantly limits the ability to understand the contact configuration between the object and the environment. We study the problem of aligning the object and environment with a tactile-based feedback insertion policy. The insertion pr...
-
Tactile sensing is critical for robotic grasping and manipulation of objects under visual occlusion. However, in contrast to simulations of robot arms and cameras, current simulations of tactile sensors have limited accuracy, speed, and utility. In this work, we develop an efficient 3D finite element method (FEM) model of the SynTouch BioTac sensor using an open-access, GPU-based robotics simulato...
-
Tactile perception is central to robot manipulation in unstructured environments. However, it requires contact, and a mature implementation must infer object models while also accounting for the motion induced by the interaction. In this work, we present a method to estimate both object shape and pose in real-time from a stream of tactile measurements. This is applied towards tactile exploration o...
-
While classical autonomous navigation systems can typically move robots from one point to another safely and in a collision-free manner, these systems may fail or produce suboptimal behavior in certain scenarios. The current practice in such scenarios is to manually re-tune the system’s parameters, e.g. max speed, sampling rate, inflation radius, to optimize performance. This practice requires exp...
-
Classical navigation systems typically operate using a fixed set of hand-picked parameters (e.g. maximum speed, sampling rate, inflation radius, etc.) and require heavy expert re-tuning in order to work in new environments. To mitigate this requirement, it has been proposed to learn parameters for different contexts in a new environment using human demonstrations collected via teleoperation. Howev...
-
Robot locomotion is a major challenge in robotics. Model-based approaches are vulnerable to model errors, and incur high computation overhead resulted from long control horizon. Model-free approaches are trained with a large number of training samples, which are expensive to obtain. In this paper, we develop a hybrid control and learning framework, called Reinforced iLQR (RiLQR), which combines th...
-
Imitation Learning (IL) is a powerful paradigm to teach robots to perform manipulation tasks by allowing them to learn from human demonstrations collected via teleoperation, but has mostly been limited to single-arm manipulation. However, many real-world tasks require multiple arms, such as lifting a heavy object or assembling a desk. Unfortunately, applying IL to multi-arm manipulation tasks has ...
-
Autonomous systems like aircraft and assistive robots often operate in scenarios where guaranteeing safety is critical. Methods like Hamilton-Jacobi reachability can provide guaranteed safe sets and controllers for such systems. However, often these same scenarios have unknown or uncertain environments, system dynamics, or predictions of other agents. As the system is operating, it may learn new k...
-
In this paper, we explore whether a robot can learn to hang arbitrary objects onto a diverse set of supporting items such as racks or hooks. Endowing robots with such an ability has applications in many domains such as domestic services, logistics, or manufacturing. Yet, it is a challenging manipulation task due to the large diversity of geometry and topology of everyday objects. In this paper, we...
-
Existing multi-camera SLAM systems assume synchronized shutters for all cameras, which is often not the case in practice. In this work, we propose a generalized multi-camera SLAM formulation which accounts for asynchronous sensor observations. Our framework integrates a continuous-time motion model to relate information across asynchronous multi-frames during tracking, local mapping, and loop clos...
-
The paper proposes a multi-modal sensor fusion algorithm that fuses WiFi, IMU, and floorplan information to infer an accurate and dense location history in indoor environments. The algorithm uses 1) an inertial navigation algorithm to estimate a relative motion trajectory from IMU sensor data; 2) a WiFi-based localization API in industry to obtain positional constraints and geo-localize the trajec...
-
We propose a framework for tightly-coupled lidar-visual-inertial odometry via smoothing and mapping, LVI-SAM, that achieves real-time state estimation and map-building with high accuracy and robustness. LVI-SAM is built atop a factor graph and is composed of two sub-systems: a visual-inertial system (VIS) and a lidar-inertial system (LIS). The two sub-systems are designed in a tightly-coupled mann...
-
The widely-used Extended Kalman Filter (EKF) provides a straightforward recipe to estimate the mean and covariance of the state given all past measurements in a causal and recursive fashion. For a wide variety of applications, the EKF is known to produce accurate estimates of the mean and typically inaccurate estimates of the covariance. For applications in visual inertial localization, we show th...
-
Simultaneous localization and mapping (SLAM) is a crucial functionality for exploration robots and virtual/augmented reality (VR/AR) devices. However, some of such devices with limited resources cannot afford the computational or memory cost to run full SLAM algorithms. We propose a general client-server SLAM optimization framework that achieves accurate real-time state estimation on the device wi...
-
This paper studies the cooperative hunting problem, where a group of agents encircle a target while avoiding collisions with each other and with obstacles in the environment. The paper deals with obstacle rich environments and dynamic (moving obstacle) environments by formulating the problem as both a control problem and a planning problem. A model predictive control (MPC) method is proposed which...
-
In this work, we aim to achieve efficient end-to-end learning of driving policies in dynamic multi-agent environments. Predicting and anticipating future events at the object level are critical for making informed driving decisions. We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-...
-
In this work we present a simple end-to-end trainable machine learning system capable of realistically simulating driving experiences. This can be used for verification of self-driving system performance without relying on expensive and time-consuming road testing. In particular, we frame the simulation problem as a Markov Process, leveraging deep neural networks to model both state distribution a...
-
This paper presents a framework that uses language instructions to define the constraints and objectives for robots gathering information about their environment. Designing autonomous robotic sampling missions requires deep knowledge of both autonomy systems and scientific domain expertise. Language commands provide an intuitive interface for operators to give complex instructions to robots. The k...
-
An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined – a vehicle will yield to us if we decide to proceed ...
-
Whole-body optimizers have been successful at automatically computing complex dynamic locomotion behaviors. However they are often limited to offline planning as they are computationally too expensive to replan with a high frequency. Simpler models are then typically used for online replanning. In this paper we present a method to generate whole body movements in real-time for locomotion tasks. Ou...
-
Although existing cellular network base stations are typically immobile, the recent development of small form factor base stations and self driving cars has enabled the possibility of deploying a team of continuously moving base stations that can reorganize the network infrastructure to adapt to changing network traffic usage patterns. Given such a system of mobile base stations (MBSes) that can f...
-
This paper describes Resh, a new, statically typed, interpreted programming language and associated runtime for orchestrating multirobot systems. The main features of Resh are: (1) It offloads much of the tedious work of programming such systems away from the programmer and into the language runtime; (2) It is based on a small set of temporal and locational operators; and (3) It is not restricted ...
-
Applications of micro unmanned aerial vehicles (UAVs) are gradually expanding into complex urban and natural environments. Despite noticeable progress, flying robots in obstacle-rich environments is still challenging. On-board processing for detecting and avoiding obstacles is possible, but at a significant computational expense, and with significant limitations (e.g., for obstacles with small cro...
-
We report the design of a morpho-functional robot called Husky Carbon. Our goal is to integrate two forms of mobility, aerial and quadrupedal legged locomotion, within a single platform. There are prohibitive design restrictions such as tight power budget and payload, which can particularly become important in aerial flights. To address these challenges, we pose a problem called the Mobility Value...
-
Teaching an anthropomorphic robot from human example offers the opportunity to impart humanlike qualities on its movement. In this work we present a reinforcement learning based method for teaching a real world bipedal robot to perform movements directly from human motion capture data. Our method seamlessly transitions from training in a simulation environment to executing on a physical robot with...
-
Recent work has demonstrated the success of reinforcement learning (RL) for training bipedal locomotion policies for real robots. This prior work, however, has focused on learning joint-coordination controllers based on an objective of following joint trajectories produced by already available controllers. As such, it is difficult to train these approaches to achieve higher-level goals of legged l...
-
This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally — a process that often requires extensive tuning due to differences between the models and hardware. In this wo...
-
Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, w...
-
In this paper, we propose a novel online algorithm for motion similarity measurements during human-robot interaction (HRI). Specifically, we formulate a Segment-based Online Dynamic Time Warping (SODTW) algorithm that can be used for understanding of repeated and cyclic human motions, in the context of rehabilitation or social interaction. The algorithm can estimate both the human-robot motion sim...
-
Robot interfaces often only use the visual channel. Inspired by Wickens’ Multiple Resource Theory, we investigated if the addition of audio elements would reduce cognitive workload and improve performance. Specifically, we designed a search and threat-defusal task (primary) with a memory test task (secondary). Eleven participants – predominantly first responders – were recruited to control a robot...
-
Time-Domain Passivity-based Controller with an Optimal Two-channel Lawrence Telerobotic Architecture
The time-domain passivity approach has been proposed in the literature in a variety of formats to guarantee the stability of teleoperation leader-follower systems. The conventional use of the proposed technique utilizes the control effort at the follower side as the force feedback to be sent back to the user at the leader’s side. However, this has resulted in transparency problems, especially when...
-
Socially Assistive Robots (SARs) have demonstrated success in the delivery of interventions to individuals with Autism Spectrum Disorder (ASD). To date, these robot-mediated interventions have primarily been designed and implemented by robotics researchers. It remains unclear whether therapists could independently utilize robots to deliver therapies in clinical settings. In this paper, we conducte...
-
During the COVID-19 pandemic, due to the unprecedented workload and cross-infection hazard, the health-care workers’ lives are under a significant threat. However, minimizing the duration and frequency of close clinician-to-patient contacts using simple technologies that enable physical distancing could reduce the risk of spreading the disease. In this context, this paper presents the conceptual d...
-
Older adults who age in place face many health problems and need to be taken care of. Fall is a serious problem among elderly people. In this paper, we present the design and implementation of collaborative fall detection using a wearable device and a companion robot. First, we developed a wearable device by integrating a camera, an accelerometer and a microphone. Second, a companion robot communi...
-
Assisting surgeons with automation of surgical subtasks is challenging due to backlash, hysteresis, and variable tensioning in cable-driven robots. These issues are exacerbated as surgical instruments are changed during an operation. In this work, we propose a framework for automation of high- precision surgical subtasks by learning local, sample-efficient, accurate, closed-loop policies that use ...
-
A pair of Supernumerary Robotic Limbs (Super-Limbs) can brace the wearer’s upper body while they work at floor level, and support them during crawling. The SuperLimbs’ motion is synchronized with the operator to mimic natural human crawling. This synchronization relies on experimental data from the operator’s observed crawl. A method for predicting the phase difference between the SuperLimbs’ hand...
-
Characterizing what types of exoskeleton gaits are comfortable for users, and understanding the science of walking more generally, require recovering a user’s utility landscape. Learning these landscapes is challenging, as walking trajectories are defined by numerous gait parameters, data collection from human trials is expensive, and user safety and comfort must be ensured. This work proposes the...
-
The design of impedance controllers for sloped walking with a transfemoral prosthesis is a complex control problem that generally results in numerous tuning parameters. This study proposes an easy-to-tune sloped walking control scheme. While the ankle is controlled using impedance control, the knee is controlled using a hybrid strategy of impedance control and trajectory tracking. This study deriv...
-
Current lower-limb prosthesis control methods are primarily model-independent — lacking formal guarantees of stability, relying largely on heuristic tuning parameters for good performance, and neglecting use of the natural dynamics of the system. Model-dependence for prosthesis controllers is difficult to achieve due to the unknown human dynamics. We build upon previous work which synthesized prov...
-
State-of-the-art human-in-the-loop robot grasping is hugely suffered by Electromyography (EMG) inference robustness issues. As a workaround, researchers have been looking into integrating EMG with other signals, often in an ad hoc manner. In this paper, we are presenting a method for end-to-end training of a policy for human-in-the-loop robot grasping on real reaching trajectories. For this purpos...
-
Shared autonomy enables robots to infer user intent and assist in accomplishing it. But when the user wants to do a new task that the robot does not know about, shared autonomy will hinder their performance by attempting to assist them with something that is not their intent. Our key idea is that the robot can detect when its repertoire of intents is insufficient to explain the user's input, and g...
-
Manipulations of a constrained object often use a non-rigid grasp that allows the object to rotate relative to the end effector. This orientation slip strategy is often present in natural human demonstrations, yet it is generally overlooked in methods to identify constraints from such demonstrations. In this paper, we present a method to model and recognize prehensile orientation slip in human dem...
-
Analyzing surgical workflow is crucial for surgical assistance robots to understand surgeries. With the understanding of the complete surgical workflow, the robots are able to assist the surgeons in intra-operative events, such as by giving a warning when the surgeon is entering specific keys or high-risk phases. Deep learning techniques have recently been widely applied to recognizing surgical wo...
-
A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning
Autonomous vehicles need to handle various traffic conditions and make safe and efficient decisions and maneuvers. However, on the one hand, a single optimization/sampling-based motion planner cannot efficiently generate safe trajectories in real time, particularly when there are many interactive vehicles near by. On the other hand, end-to-end learning methods cannot assure the safety of the outco...
-
For autonomous vehicles, effective behavior planning is crucial to ensure safety of the ego car. In many urban scenarios, it is hard to create sufficiently general heuristic rules, especially for challenging scenarios that some new human drivers find difficult. In this work, we propose a behavior planning structure based on reinforcement learning (RL) which is capable of performing autonomous vehi...
-
We present a hierarchical control approach for maneuvering an autonomous vehicle (AV) in tightly-constrained environments where other moving AVs and/or human driven vehicles are present. A two-level hierarchy is proposed: a high-level data-driven strategy predictor and a lower-level model-based feedback controller. The strategy predictor maps an encoding of a dynamic environment to a set of high-l...
-
This paper presents an analytical methodology and experimental study to identify quantitatively the zero-potential-energy (ZP) motion due to the stiffness matrices in Cartesian impedance control of redundant manipulators. This mode of motion, analogous to the rigid-body mode in classic mechanical systems, shows up as a result of the redundancy of the robot and creates a steady-state deviation from...
-
In this paper, we address the task of interacting with dynamic environments where the changes in the environment are independent of the agent. We study this through the context of trapping a moving ball with a UR5 robotic arm. Our key contribution is an approach to utilize a static planner for dynamic tasks using a Dynamic Planning add-on; that is, if we can successfully solve a task with a static...
-
In this paper, we address the Perception– Constrained Model Predictive Control (PCMPC) and state estimation problems for quadrotors with cable suspended payloads using a single camera and Inertial Measurement Unit (IMU). We design a receding–horizon control strategy for cable suspended payloads directly formulated on the system manifold configuration space SE (3) ×S2. The approach considers the sy...
-
Developing agile behaviors for legged robots re-mains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a c...
-
The tabletop robot Haru, used for affective telepresence research, enables a teleoperator to communicate affects from a distance. The robot’s expressiveness offers myriad ways of communicating affects through the execution of emotive routines. The teleoperator reacts to input modalities such as the user’s facial expression, gestures and speech-based intent as perceived by the robot’s perception sy...
-
Fast autonomous motion in cluttered and unknown environments, such as forests, is highly dependent on low-latency obstacle avoidance strategies. In this context, this paper presents a motion planning strategy that relies on lattices for the fast computation of local paths that both avoid obstacles and follow a vector field that encodes the global robot task. Lattices are constructed in the sensor ...
-
Generating a surgical report in robot-assisted surgery, in the form of natural language expression of surgical scene understanding, can play a significant role in document entry tasks, surgical training, and post-operative analysis. Despite the state-of-the-art accuracy of the deep learning algorithm, the deployment performance often drops when applied to the Target Domain (TD) data. For this purp...
-
Laparoscopic Field of View (FOV) control is one of the most fundamental and important components in Minimally Invasive Surgery (MIS), nevertheless the traditional manual holding paradigm may easily bring fatigue to surgical assistants, and misunderstanding between surgeons also hinders assistants to provide a high-quality FOV. Targeting this problem, we here present a data-driven framework to real...
-
Soft robotics is an emerging technology with excellent application prospects. However, due to the inherent compliance of the materials used to build soft robots, it is extremely complicated to control soft robots accurately. In this paper, we introduce a data-based control framework for solving the soft robot underwater locomotion problem using deep reinforcement learning (DRL). We first built a s...
-
We propose a novel real-time physically-accurate simulator for long flexible cable manipulation. We first discretize the cable into multiple rigid link segments, each with complementarity-based contact model and inter-segment compliant coupling; and partition the cable into a number of subsystems, each composed with a number of consecutive links. We then formulate the inter-subsystem consistency c...
-
In this paper, we proposed an autonomous robotic flexible endoscope system for the laparoscopic bariatric surgery (LBS). This system comprises a UR5 robot and a flexible endoscope equipped with a novel continuum joint, named reinforced double helix continuum mechanism. Compared with the simple helix structure, the compressional and torsional stiffness of the proposed joint are improved significant...
-
Variable stiffness technology is an extensively discussed topic in soft robotics, which can bridge the traditional fast, precise and high-force rigid robots with compliant, agile, and safe soft robots. In this paper, we introduce the concept of geometry-induced rigidity and propose a variable curvature jamming (VCJ) mechanism for amplifying the laminar jamming structures which are fast, efficient,...
-
Most existing surgical robots employ straight rigid instruments and are not compatible with imaging modalities, especially magnetic resonance imaging (MRI) that presents restrictive constraints on the robot and actuator materials. Employing continuum distal end effector and fulfilling multi-imager compatibility will potentially lead to wide adoption of surgical robots in intraoperative image-guide...
-
Unlike loose coupling approaches and the EKF-based approaches in the literature, we propose an optimization-based visual-inertial SLAM tightly coupled with raw Global Navigation Satellite System (GNSS) measurements, a first attempt of this kind in the literature to our knowledge. More specifically, reprojection error, IMU pre-integration error and raw GNSS measurement error are jointly minimized w...
-
In this paper, a three-dimensional light detection and ranging simultaneous localization and mapping (SLAM) method is proposed that is available for tracking and mapping with 500–1000 Hz processing. The proposed method significantly reduces the number of points used for point cloud registration using a novel ICP metric to speed up the registration process while maintaining accuracy. Point cloud re...
-
We present a fast, scalable, and accurate Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of objects. Leveraging the observation that artificial environments are structured and occupied by recognizable objects, we show that a compositional and scalable object mapping formulation is amenable to a robust SLAM solution for drift-free large-scale indoor rec...
-
In this paper, we present a robotic system to automatically perform defect inspection tasks over free-form specular surfaces, which the image acquisition sub-system is equipped with a 6-DOF robot manipulator to achieve flexible scanning. Given the mesh model of the workpiece, we implement K-means based region segmentation algorithm on the point cloud after preprocessing. Then, we take the smooth r...
-
We have developed service robot management system to facilitate effective collaboration between multiple units and types of robots in operation. This system is implemented by serverless architecture on cloud and using cellular based IoT communication. So it has not only usual cloud system advantage that it is not necessary to prepare dedicated server and network equipment, but it reduces managemen...
-
Although Socially Assistive Robotics have been used in Autism Spectrum Disorder (ASD) interventions, such studies often exclude Special Educators (SEs) and often use expensive humanoid robots. In this paper, we investigate whether non-humanoid toy robots can act as teaching aids in ASD Education, in particular, can they reduce the workload of SEs. We target two most common yet divergent problems f...
-
Heart Position Estimation based on Bone Distribution toward Autonomous Robotic Fetal Ultrasonography
Autonomous fetal ultrasonography with a robotic ultrasound (US) can potentially solve the issue of the shortage of ob-gyn physicians in prenatal care. In fetal cardiac diagnosis, acoustic shading derived from the fetal skeleton makes the scanning procedure using a robotic US cumbersome. We hypothesize that fetal bone distribution can be used for determining the fetal position and for subsequently ...
-
The feasibility of using chip-based RFID designs as wireless sensor tags open a wide range of application possibilities in the field of robotics. However, multi-step lithography manufacturing and/or MEMS techniques are often required for the industrial-grade fabrication of such sensors. In this paper, we present a simple, home-based, two-step fabrication process to produce chipless RF-based wirele...
-
Due to the high positioning accuracy and relatively low prices, SCARA robots are widely used in industrial fields. The objective of this paper is to propose a hand-eye self-calibration algorithm for SCARA robots which could consider both accuracy and computational cost. The previous global optimal hand-eye calibration algorithms based on branch-and-bound (BnB) optimization is limited by their expe...
-
Dielectric elastomers are electro-mechanically coupled transducers that display a nonlinear viscoelastic stress-strain relationship. Modeling and controlling such nonlinear materials and actuators are of great challenge. A spring roll dielectric elastomer actuator is a linear actuator composed of a spring and sheets of wrapped dielectric elastomers. Since its shape and actuation performance resemb...
-
This paper presents a wheeled V-shaped in-pipe robot in which the two outputs of the wheel shaft and roll joint are driven solely by a single actuator input. This underactuation is generated by a simple miter gear mechanism. Generally, to control two movements easily, one of the outputs of the underactuated mechanism is constrained by the resilience force of springs or by the friction force. Howev...
-
A hydraulic drive system can generate a large force and is impact resistant. Thus, robots that utilize hydraulic drive systems have been developed for use in disaster areas. In a hydraulic drive system, it is necessary to design the motion of the robot within the capacity of the pump unit, but it is generally difficult to predict the required flow rate and pressure. Especially for hydraulically dr...
-
An autonomous robot that is able to physically guide humans through narrow and cluttered spaces could be a big boon to the visually-impaired. Most prior robotic guiding systems are based on wheeled platforms with large bases with actuated rigid guiding canes. The large bases and the actuated arms limit these prior approaches from operating in narrow and cluttered environments. We propose a method ...
-
An Overconstrained Robotic Leg with Coaxial Quasi-direct Drives for Omni-directional Ground Mobility
Planar mechanisms dominate modern designs of legged robots with remote actuator placement for robust agility in ground mobility. This paper presents a novel design of robotic leg modules using the Bennett linkage, driven by two coaxially arranged quasi-direct actuators capable of omnidirectional ground locomotion. The Bennett linkage belongs to a family of overconstrained linkages with three-dimen...
-
Most of the existing visual SLAM methods heavily rely on a static world assumption and easily fail in dynamic environments. Some recent works eliminate the influence of dynamic objects by introducing deep learning-based semantic information to SLAM systems. However such methods suffer from high computational cost and cannot handle unknown objects. In this paper, we propose a real-time semantic RGB...
-
This paper proposes a new path planning strategy - the Rapid Visible Tree (RVT) algorithm to guide a robot to its goal in a complex environment without dangerous collisions. By fusing the visibility information with the classic tree-based searching method, RVT only takes the noisy points locally acquired from the environment as input and computes the visible region at each location to decide the g...
-
Compared to widely used LiDAR-based mapping in autonomous driving field, image-based mapping method has the advantages of low cost, high resolution, and no need for complex calibration. However, the image-based 3D mapping depends heavily on the texture richness and always leaves holes and outliers in low-textured areas, such as the road surface. To this end, this paper proposed a novel semanticall...
-
Robots that can manipulate objects in unstructured environments and collaborate with humans can benefit immensely by understanding natural language. We propose a pipelined architecture of two stages to perform spatial reasoning on the text input. All the objects in the scene are first localized, and then the instruction for the robot in natural language and the localized co-ordinates are mapped to...
-
In this paper, we present a novel preconditioning strategy for the classic 8-point algorithm (8-PA) for estimating an essential matrix from 360-FoV images (i.e., equirectangular images) in spherical projection. To alleviate the effect of uneven key-feature distributions and outlier correspondences, which can potentially decrease the accuracy of an essential matrix, our method optimizes a non-rigid...
-
Initialisation of Autonomous Aircraft Visual Inspection Systems via CNN-Based Camera Pose Estimation
General Visual Inspection is a manual inspection process regularly used to detect and localise obvious damage on the exterior of commercial aircraft. There has been increasing demand to perform this process at the boarding gate to minimize the downtime of the aircraft and automating this process is desired to reduce the reliance on human labour. This automation typically requires the first step of...
-
This paper presents the voxelized generalized iterative closest point (VGICP) algorithm for fast and accurate three-dimensional point cloud registration. The proposed approach extends the generalized iterative closest point (GICP) approach with voxelization to avoid costly nearest neighbor search while retaining its accuracy. In contrast to the normal distributions transform (NDT), which calculate...
-
This paper presents a vision-based modularized drone racing navigation system that uses a customized convolutional neural network (CNN) for the perception module to produce high-level navigation commands and then leverages a state-of-the-art planner and controller to generate low-level control commands, thus exploiting the advantages of both data- based and model-based approaches. Unlike the state...
-
Recently, deep-learning based approaches have achieved impressive performance for autonomous driving. However, end-to-end vision-based methods typically have limited interpretability, making the behaviors of the deep networks difficult to explain. Hence, their potential applications could be limited in practice. To address this problem, we propose an interpretable end-to-end vision-based motion pl...
-
Visual perception in autonomous driving is a crucial part of a vehicle to navigate safely and sustainably in different traffic conditions. However, in bad weather such as heavy rain and haze, the performance of visual perception is greatly affected by several degrading effects. Recently, deep learning-based perception methods have addressed multiple degrading effects to reflect real-world bad weat...
-
This paper aims to tackle the problem of referring image segmentation, which is targeted at reasoning the region of interest referred by a query natural language sentence. One key issue to address the referring image segmentation is how to establish the cross-modal representation for encoding the two modalities, namely, the query sentence and the input image. Most existing methods are designed to ...
-
Real-time semantic segmentation is a challenging task as both accuracy and inference speed need to be considered simultaneously. In real-world applications, it is usually achieved by deploying a deep neural network in modern GPU device. However, most of the work focused on real-time semantic segmentation is designed by significantly reducing computation complexity and model size. There are other f...
-
In environmental perception of autonomous driving, zero-shot semantic segmentation that can make prediction of new categories without using any labeled training samples is considered as a challenging task. One key step in this task is to transfer knowledge across categories via auxiliary semantic word embeddings. In this paper, we propose a feature enhanced projection network (FEPNet) that takes f...
-
Addressing on monocular visual odometry problem, this paper presents a novel end-to-end network for estimation of camera ego-motion. The network learns the latent space of optical flow (OF) and models sequential dynamics so that the motion estimation is constrained by the relations between sequential images. We compute the OF field of consecutive images and extract the latent OF representation in ...
-
Existing radar sensors can be classified into automotive and scanning radars. While most radar odometry (RO) methods are only designed for a specific type of radar, our RO method adapts to both scanning and automotive radars. Our RO is simple yet effective, where the pipeline consists of thresholding, probabilistic submap building, and an Normal Distribution Transform-based (NDT-based) radar scan ...
-
Visual Inertial Odometry (VIO) is of great interest due the ubiquity of devices equipped with both a monocular camera and Inertial Measurement Unit (IMU). Methods based on the extended Kalman Filter remain popular in VIO due to their low memory requirements, CPU usage, and processing time when compared to optimisation-based methods. In this paper, we analyse the VIO problem from a geometric perspe...
-
Tactile sensing on human feet is crucial for motion control, however, has not been explored in robotic counterparts. This work is dedicated to endowing tactile sensing to legged robot’s feet and showing that a single-legged robot can be stabilized with only tactile sensing signals from its foot. We propose a robot leg with a novel vision-based tactile sensing foot system and implement a processing...
-
Parametric models that represent layout in terms of scene attributes are an attractive avenue for road scene understanding in autonomous navigation. Prior works that rely only on ground imagery are limited by the narrow field of view of the camera, occlusions and perspective foreshortening. In this paper, we demonstrate the effectiveness of using aerial imagery as an additional modality to overcom...
-
This paper presents a Dynamic Vision Sensor (DVS) based system for reasoning about high-speed motion. As a representative scenario we consider a robot at rest, reacting to a small, fast approaching object at speeds higher than 15 m/s. Since conventional image sensors at typical frame rates observe such an object for only a few frames, estimating the underlying motion presents a considerable challe...
-
Crawling through various terrains has been a long research interest. In recent years, quite a number of soft crawling robots have been developed. However, locomoting in an elastic, humid, and slippery environment remains a challenge. In nature, gastropods, such as snails, live in humid environment and could crawl through all kinds of surface conditions by using wet adhesion. In the wet adhesive lo...
-
Motion control of multi-joint Series Elastic Actuator (SEA)-driven robots still faces challenges including intrinsic oscillatory dynamics, high-order robotic dynamics, low-bandwidth inner loop, and dynamic nonlinearities. In this letter, a nonlinear disturbance observer (NDOB)-based robust controller with the singular perturbation theory is proposed to perform stable and precise motion control of ...
-
Myoelectric prosthetic hands are intended to re-place the function of the amputee’s lost arm. Therefore, developing robotic prosthetics that can mimic not only the appearance and functionality of humans but also characteristics unique to human movements is paramount. This paper proposes a novel biomimetic control method for myoelectric prosthetic hands integrating the impedance model with the conc...
-
Safety is a critical concern when deploying reinforcement learning agents for realistic tasks. Recently, safe reinforcement learning algorithms have been developed to optimize the agent’s performance while avoiding violations of safety constraints. However, few studies have addressed the nonstationary disturbances in the environments, which may cause catastrophic outcomes. In this paper, we propos...
-
Quantification of Joint Redundancy considering Dynamic Feasibility using Deep Reinforcement Learning
The robotic joint redundancy for executing a task and the optimal usage of robotic joints given the redundant degrees of freedom are crucial for the performance of a robot. It is therefore of interest to quantify the joint redundancy to better understand the robotic dexterity considering the dynamic feasibility. To this end, model-based approaches have been among the most commonly used methods to ...
-
The analysis of facial expression is a very complex and challenging problem. Most researches for automated Facial Expression Recognition (FER) are mainly based on deep learning networks, rarely considering data imbalance. This paper commits to addressing the long-tail distribution problems among large-scale datasets in wild. Inspired by the continual learning method, we reconstruct multi-subsets f...
-
Recognizing human emotion/expressions automatically is quite an expected ability for intelligent robotics, as it can promote better communication and cooperation with humans. Current deep-learning-based algorithms may achieve impressive performance in some lab-controlled environments, but they always fail to recognize the expressions accurately for the uncontrolled in-the-wild situation. Fortunate...
-
Deep learning algorithms are recognized as the most effective method for rectal tumor segmentation. However, since the multi-scale detailed feature information of rectal tumor cannot be fully extracted and applied, the segmentation and identification results of most algorithms are not always perfect. In this work, we introduce a Covariance Self-Attention Dual Path UNet (CSA-DPUNet), that is modifi...
-
This paper proposes a method of fabric structure defect detection based on tactile information. Different from traditional visual-based detection methods, the proposed method uses a tactile sensor to attain the information of fabric. The advantage of using tactile information is to avoid different irregular dyeing patterns and reduce the influence of ambient light. Therefore, the proposed method c...
-
To solve the problem of unmanned ground vehicle leader-follower formation transportation in unstructured environment, we propose a novel target detection and tracking method based on multi-sensor fusion perception. Combined with 3D-Lidar, millimeter wave Radar and GPS/IMU, the proposed method can achieve stable target detection and continuous tracking of both static and dynamic vehicles. First, 3D...
-
Automatic surgical gesture recognition is fundamentally important to enable intelligent cognitive assistance in robotic surgery. With recent advancement in robot-assisted minimally invasive surgery, rich information including surgical videos and robotic kinematics can be recorded, which provide complementary knowledge for understanding surgical gestures. However, existing methods either solely ado...
-
We propose a method for detecting structural changes in a city using images captured from vehicular mounted cameras over traversals at two different times. We first generate 3D point clouds for each traversal from the images and approximate GNSS/INS readings using Structure-from-Motion (SfM). A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo-locati...
-
Visual Place Recognition (VPR) is a crucial component for long-term mobile robot autonomy. In this paper, we exploit a coarse-to-fine paradigm to recognize places. In particular, we first select candidate frames for each query image, and then check the spatial geometric relationship between the query and its candidate frames to determine the final place match. In the coarse match stage, we employ ...
-
Microinjection technology is applied widely in biomedical research for the purposes of gene manipulation and microinsemination. Generally, microinjection is performed under an optical microscope environment through image presentation of the targets. To perform the microinjection process, it is necessary to place multiple cells in the same droplet and perform multiple injections. This process requi...
-
Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for roboti...
-
Multi-instance scenes are especially challenging for end-to-end visuomotor (image-to-control) learning algorithms. "Pipeline" visual servo control algorithms use separate detection, selection and servo stages, allowing algorithms to focus on a single object instance during servo control. End-to-end systems do not have separate detection and selection stages and need to address the visual ambiguiti...
-
This work presents a general deep learning framework for large-scale point clouds understanding without voxelizations, called FG-Conv, which achieves an accurate and real-time understanding of point clouds. Through our novel design combining feature level correlation mining and deformable convolutions based geometric aware modeling, the local feature relationships and geometric patterns can be cap...
-
We propose simple yet effective improvements in point representations and local neighborhood graph construction within the general framework of graph neural networks (GNNs) for 3D point cloud processing. As a first contribution, we propose to augment the vertex representations with important local geometric information of the points, followed by nonlinear projection using a MLP. As a second contri...
-
This paper presents a stereo object matching method that exploits both 2D contextual information from images as well as 3D object-level information. Unlike existing stereo matching methods that exclusively focus on the pixel-level correspondence between stereo images within a volumetric space (i.e., cost volume), we exploit this volumetric structure in a different manner. The cost volume explicitl...
-
In this paper, we propose an operational space control formulation for a planar N-link underactuated manipulator (PAN–1)1 with a passive first joint subject to actuator constraints (N ⩾ 3), covering both stabilization and tracking tasks. Such underactuated manipulators have an inherent first-order nonholonomic constraint, allowing us to project their dynamics to a space consistent with the nonholo...
-
The inner/low-level control loop of most industrial robotic manipulators is protected from any modification by a closed control architecture. The only way to specify joint inputs to them is through position or velocity commands. Furthermore, the inner controller is unknown/uncertain, as it is not revealed to the user. This makes it very difficult to determine the configuration of the inner control...
-
The actuator bandwidth limitation deteriorates the stability and performance of torque-based robot controllers. Operational space control is especially prone to this problem, since the limited bandwidth of a single actuator can reduce the performance of all related tasks simultaneously. In this article, an intuitive way to penalize low performance actuators is proposed to improve the performance o...
-
Driven by the SARS-CoV-2 pandemic, demand for oropharyngeal swab sampling (OP-swabs) is surging. However, medical staff can easily become infected by the virus during the sampling process. In an effort to combat this, we developed a novel, intrinsically safe rigid- flexible coupling (RFC) manipulator to improve the safety and reliability of OP-swab sampling to test for COVID-19, which is presented...
-
This paper presents the design and control of a fully handheld robot for robot-assisted microsurgery. The handheld robot incorporates a miniature 6-PUS parallel micromanipulator that can impose a remote center of motion (RCM) at the incision point of entry during microsurgery. An optimization framework is formulated to determine the geometric parameters of the micromanipulator. The optimization ai...
-
This paper designed a pneumatic soft climbing robot by utilizing the high flexibility of soft materials. Capabilities of climbing and creeping through small spaces would rarely possible with a method based only on rigid links. At first, according to the drive mode of pneumatic networks, a model for soft climbing robots with different section having their independent stiffness was designed; afterwa...
-
Surface robots can have many applications due to multiple degrees of freedom. Accordingly, many open research questions arise due to the limited number of realized cases and insufficient theory foundation. For a surface robot that performs deformations with shear deformation, the inner product calculation in a local coordinate system on the robot generally depends on the shear angle. However, the ...
-
Origami, the art of folding paper, can impart useful design inspirations to the creation of mechanical structures and mechanisms. Bistability is a useful property for origami designs, which can help compartmentalize different actuations and stiffness tuning regimes. Given the benefits of bistability, we investigated origami designs used to build robots and deployable structures. We show snap-throu...
-
Parallel tracking and mapping (PTAM) as a time-efficient framework for simultaneous localization and mapping (SLAM) has been becoming popular in recent years. However, in this paper, we vigilantly point out that the favorite parallel-pipeline design realized by recent proposed SLAM algorithms may lead to inaccurate state estimates which, as a consequence, cannot always guarantee the performance of...
-
Concurrent mapping necessitates data association among vehicles to overcome temporal and sensor modality differences. In this work, we focus on an underwater multi-vehicle mapping scenario in which vehicles have various sensor modalities, namely sonar and camera. This inter-session sonar-optical image matching poses two main challenges. First, ensuring covisibility for the opti-acoustic pair is co...
-
In this paper, a degeneracy avoidance method for a point and line based visual SLAM algorithm is proposed. Visual SLAM predominantly uses point features. However, point features lack robustness in low texture and illuminance variant environments. Therefore, line features are used to compensate the weaknesses of point features. In addition, point features are poor in representing discernable featur...
-
For 25 years the Robotics Toolbox for MATLAB® has been used for teaching and research worldwide. This paper describes its successor – the Robotics Toolbox for Python. More than just a port, it takes advantage of popular open-source packages and resources to provide platform portability, fast browser-based 3D graphics, quality documentation, fast numerical and symbolic operations, powerful IDEs, sh...
-
The goal of foraging robot swarms is to search and deliver resources to a specific central collection zone quickly. In the previously proposed multiple-place foraging algorithm with dynamic depots, foraging performance decreases as search areas and swarm sizes increase: depots need to travel long distances to deliver resources to the center, and more robots produce more congestion on their journey...
-
Most of the existing saliency prediction research focuses on either single images or videos (or more precisely multiple images in sequence). However, to apply saliency prediction to drone exploration that has to consider multiple images from different view angles or localizations to determine the direction to explore, saliency prediction over multiple discontinuous images is required. In this pape...
-
Positioning Control for Underactuated Unmanned Surface Vehicles to Resist Environmental Disturbances
In this paper, we present the positioning control problem for underactuated unmanned surface vehicles (USVs) in the presence of unknown external disturbances, such as the wind, waves, and currents. The three control objectives of positioning control for underactuated USVs are, firstly, to retain a predefined constant distance to a look-ahead point, secondly, to regulate the vehicle heading to poin...
-
Path planning is a fundamental capability for autonomous navigation of robotic wheelchairs. With the impressive development of deep-learning technologies, imitation learning-based path planning approaches have achieved effective results in recent years. However, the disadvantages of these approaches are twofold: 1) they may need extensive time and labor to record expert demonstrations as training ...
-
Object slip perception is essential for mobile manipulation robots to perform manipulation tasks reliably in the dynamic real-world. Traditional approaches to robot arms’ slip perception use tactile or vision sensors. However, mobile robots still have to deal with noise in their sensor signals caused by the robot’s movement in a changing environment. To solve this problem, we present an anomaly de...
-
Autonomous vehicles (AVs) play an important role in transforming our transportation systems and relieving traffic congestion. To guarantee their safety, AVs must be sufficiently tested before they are deployed to public roads. Existing testing often focuses on AVs’ collision avoidance on a given route. There is little work on the systematic testing for AVs’ route planning and tracking on a map. In...
-
3D pointing devices are indispensable in virtual reality (hereafter VR) and human-robot interaction scenarios. Existing devices are cumbersome or non-immersive or have a limited volume of operation. Hand gesture-based interfaces do not suffer from these problems and can be used for 3D pointing purposes. However, there is a lack of robust, accurate hand gesture-based pointing techniques which can b...
-
This paper presents a navigation network based deep reinforcement learning framework for autonomous indoor robot exploration. The presented method features a pattern cognitive non-myopic exploration strategy that can better reflect universal preferences for structure. We propose the Extendable Navigation Network (ENN) to encode the partially observed high-dimensional indoor Euclidean space to a sp...
-
This paper addresses the localization of contacts of an unknown grasped rigid object with its environment, i.e., extrinsic to the robot. We explore the key role that distributed tactile sensing plays in localizing contacts external to the robot, in contrast to the role that aggregated force/torque measurements traditionally play in localizing contacts on the robot. When in contact with the environ...
-
Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain...
-
Input-output transmission ratio shifting mechanisms provide a variable transmission ratio, which effectively expands a speed-force operating range of actuators. Although it is the most effective solution to increase the performance of robotic systems, its application to compact robotic systems still remains a challenging issue due to its complexity and massive structure. In this paper, we introduc...
-
Compliance in robot actuation provides a solution to perform safe physical human-robot interaction. Conventional compliant actuators (variable stiffness actuators, series elastic actuators) used more than two motors or closed-loop controller to modulate both stiffness and equilibrium position independently. These actuators are complex, lack of energy efficiency, and have limited stiffness range. I...
-
High resolution and large range force/torque (F/T) measurements are usually required in many engineering tasks. However, most existing F/T sensors only have a fixed resolution over their whole ranges. The key lies in that it is difficult to well balance high resolution and large range in the sensor design. Taking the torque sensor for example, this paper presents a better compromise for this probl...
-
The research community presented significant advances in many different Visual-Inertial Navigation System (VINS) algorithms to localize mobile robots or hand-held devices in a 3D environment. While authors of the algorithms of-ten do compare to, at that time, existing competing approaches, their comparison methods, rigor, depth, and repeatability at later points in time have a large spread. Furthe...
-
a new generation of computer vision, namely event-based or neuromorphic vision, provides a new paradigm for capturing visual data and the way such data is processed. Due to a highly novel type of visual sensors used in event-based vision, only a few datasets aimed at visual navigation tasks are publicly available.In this paper, we present and describe the first event-based vision dataset intended ...
-
In modern agriculture, measuring phenotypic traits helps breeders monitor plant growth, increase yield, and provide food, feed, and fiber. Traditional phenotyping requires intensive manual work, partially being intrusive. In this paper, we investigate the challenge of measuring phenotypic traits in an automated fashion through mobile robots operating in field environments. In particular, we want t...
-
Driven by the need for physically accurate and haptically convincing models of lunar and planetary regolith for model-mediated teleoperation in space, we present an approach to modelling regolith in an efficient yet realistic way. Model parameters are derived from physical characteristics of the regolith, to render regoliths with different density profiles, cohesion, internal friction and in diffe...
-
This paper presents two different approaches to generate a time local-optimal and jerk-limited trajectory with blends for a robot manipulator under consideration of kinematic constraints. The first approach generates a trajectory with blends based on the trapezoidal acceleration model by formulating the problem as a nonlinear constraint and a non-convex optimization problem. The resultant trajecto...
-
In dynamic and cramped industrial environments, achieving reliable Visual Teach and Repeat (VT&R) with a single-camera is challenging. In this work, we develop a robust method for non-synchronized multi-camera VT&R. Our contribution are expected Camera Performance Models (CPM) which evaluate the camera streams from the teach step to determine the most informative one for localization during the re...
-
Neural Architecture Search (NAS) has proved effective in offering outperforming alternatives to handcrafted neural networks. In this paper we analyse the benefits of NAS for image classification tasks under strict computational constraints. Our aim is to automate the design of highly efficient deep neural networks, capable of offering fast and accurate predictions and that could be deployed on a l...
-
Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D environments where the robot has to solely rely on a single monocular camera. In particular, we are interested in solving this problem without relying on localization, mapping, or planning techniques. Most of the existing ...
-
Current robots are either expensive or make significant compromises on sensory richness, computational power, and communication capabilities. We propose to leverage smartphones to equip robots with extensive sensor suites, powerful computational abilities, state-of-the-art communication channels, and access to a thriving software ecosystem. We design a small electric vehicle that costs $50 and ser...
-
The purpose of the present work is the reconstruction of motorcycle lateral dynamics. The main idea is to estimate pertinent states and unknown inputs (rider action) with respect to nonlinear outputs due to motion transformation frames (inertial sensors are away from the local frame). To overcome this issue, we propose a new Unknown Input Observers with variable output matrix. In this paper, we ta...
-
Torsional actuators are a class of artificial muscle technology that generates torque and produces rotary motion in response to various stimuli. This paper presents a novel torsional actuator combining an origami-inspired twisting skeleton and an artificial muscle. The process of torsional actuator design starts from identifying a foldable twisting skeleton which is capable of achieving helical mo...
-
Robust and reliable ego-motion is a key component of most autonomous mobile systems. Many odometry estimation methods have been developed using different sensors such as cameras or LiDARs. In this work, we present a resilient approach that exploits the redundancy of multiple odometry algorithms using a 3D LiDAR scanner and a monocular camera to provide reliable state estimation for autonomous vehi...
-
In the context of self-driving vehicles there is strong competition between approaches based on visual localisation and Light Detection And Ranging (LiDAR). While LiDAR provides important depth information, it is sparse in resolution and expensive. On the other hand, cameras are low-cost and recent developments in deep learning mean they can provide high localisation performance. However, several ...
-
Planetary robotics navigation has attracted the great attention of many researchers in recent years. Localization is one of the most important problems for robots on another planet in the lack of GPS. The robots need to be able to know their location and the surrounding map in the environment concurrently, to work and communicate together on another planet. In the current work, a novel algorithm i...
-
The flexibility and adaptability of modular and re-configurable robots opens up new opportunities for on-demand robot morphology optimization for varying tasks. In particular, multi-arm robotic systems can expand the solution space for any given task. In this paper, we present a novel approach to exploit this feature for generating optimal fit-to-task robot structures with respect to a minimum-eff...
-
This paper presents a computational framework for the design of high-performance legged robotic systems. The framework relies on the concurrent optimization of hardware parameters and control trajectories to find the best robot design for a given task. In particular, we focus on energy efficiency, presenting novel electro-mechanical models to account for the losses of the actuators due to friction...
-
Recent advances in machine learning have triggered an enormous interest in using learning-based approaches for robot control and object manipulation. While the majority of existing algorithms are evaluated under the assumption that the involved bodies are rigid, a large number of practical applications contain deformable objects. In this work we focus on Deformable Linear Objects (DLOs) which can ...
-
Deep Imitation Learning requires a large number of expert demonstrations, which are not always easy to obtain, especially for complex tasks. A way to overcome this shortage of labels is through data augmentation. However, this cannot be easily applied to control tasks due to the sequential nature of the problem. In this work, we introduce a novel augmentation method which preserves the success of ...
-
Validating the kinematic feasibility of a planned robot motion and finding corresponding inverse solutions are time-consuming processes, especially for long-horizon manipulation tasks. Most existing approaches are based on solving iterative gradient-based optimization, so the processes are time-consuming and have a high risk of falling in local minima. In this work, we propose a unified framework ...
-
Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challenging for complex tasks. Using inaccurate models for learning can lead to sub-optimal solutions that are unlikely to perform well in practice. In this work, we present a learning approach which iterates between model lear...
-
When learning policies for robotic systems from data, safety is a major concern, as violation of safety constraints may cause hardware damage. SafeOpt is an efficient Bayesian optimization (BO) algorithm that can learn policies while guaranteeing safety with high probability. However, its search space is limited to an initially given safe region. We extend this method by exploring outside the init...
-
We present a hierarchical planning and control framework that enables an agent to perform various tasks and adapt to a new task flexibly. Rather than learning an individual policy for each particular task, the proposed framework, DISH, distills a hierarchical policy from a set of tasks by representation and reinforcement learning. The framework is based on the idea of latent variable models that r...
-
Active Model Learning using Informative Trajectories for Improved Closed-Loop Control on Real Robots
Model-based controllers on real robots require accurate knowledge of the system dynamics to perform optimally. For complex dynamics, first-principles modeling is not sufficiently precise, and data-driven approaches can be leveraged to learn a statistical model from real experiments. However, the efficient and effective data collection for such a data-driven system on real robots is still an open c...
-
A limitation of model-based reinforcement learning (MBRL) is the exploitation of errors in the learned models. Blackbox models can fit complex dynamics with high fidelity, but their behavior is undefined outside of the data distribution. Physics-based models are better at extrapolating, due to the general validity of their informed structure, but underfit in the real world due to the presence of u...
-
Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations, with the cost of nominal performance drop. While adversarial training appears to enhance the robustness and safety of a deep model deployed in open-world decision-critical applications, counterintuitively, it induces undesired behaviors in robot learning settings. In this pap...
-
Increasing research attention has been attracted to automatic plug-in charging in an unmanned and dangerous environment. In this work, we develop an object detection solution based on deep learning on 3D point clouds using a mobile robot manipulator to provide mobility and manipulation. In this solution, the 3D point cloud technology is adopted to measure the shapes and depth information for plugi...
-
Bilateral teleoperation under rate mode is known to be a difficult problem in terms of stability, especially when the slave manipulator interacts with a time-varying environment. This paper presents an energy based variable admittance control approach, whose principle combines the monitoring and the regulation of the energy exchanges with a passivity tank. It allows stable interactions with force ...
-
Teleoperating robotic arms is a challenging task that requires years of training to master. It is mentally demanding, as the operator must internally compute transformations, or rely on muscle memory, to perform even the simplest tasks. Alternative methods that rely on embodiment –the immersive, first person experience of controlling the robot from its point of view are recently becoming more popu...
-
Teleoperation provides a way for human operators to guide robots in situations where full autonomy is challenging or where direct human intervention is required. It can also be an important tool to teach robots in order to achieve autonomous behaviour later on. The increased availability of collaborative robot arms and Virtual Reality (VR) devices, provides ample opportunity for development of nov...
-
In this work, the problem of human-robot collaborative object transfer to unknown target poses is addressed. The desired pattern of the end-effector pose trajectory to a known target pose is encoded using DMPs (Dynamic Movement Primitives). During transportation of the object to new unknown targets, a DMP-based reference model and an EKF (Extended Kalman Filter) for estimating the target pose and ...
-
In the modern manufacturing process, novel technologies enable the collaboration between humans and robots, which increases productivity while keeping flexibility. However, these technologies also lead to new challenges, e.g., maximization of Human-Robot Collaboration (HRC) performance while ensuring safety for the human being in fenceless robot applications. In this paper, an approach of the dyna...
-
In the industrial domain, the application of any system as a safety function has to follow strict rules and requirements, defined in safety standards such as ISO 13849 [1] and ISO 13855 [2]. Two core requirements are an extremely low rate of dangerous errors and an upper limit for the response time of the system (hard real-time requirement). Current approaches in the field of human pose estimation...
-
Today’s robots are expected to fulfill different requirements originated from executing complex tasks in uncertain environments, often in collaboration with humans. To deal with this type of multi-objective control problem, hierarchical least-square optimization techniques are often employed, defining multiple tasks as objective functions, listed in hierarchical manner. The solution to the Inverse...
-
In Human-Robot Collaboration, the robot operates in a highly dynamic environment. Thus, it is pivotal to guarantee the robust stability of the system during the interaction but also a high flexibility of the robot behavior in order to ensure safety and reactivity to the variable conditions of the collaborative scenario.In this paper we propose a control architecture capable of maximizing the flexi...
-
Accurate post-impact velocity predictions are essential in developing impact-aware manipulation strategies for robots, where contacts are intentionally established at non-zero speed mimicking human manipulation abilities in dynamic grasping and pushing of objects. Starting from the recorded dynamic response of a 7DOF torque-controlled robot that intentionally impacts a rigid surface, we investigat...
-
Driving energy consumption plays a major role in the navigation of mobile robots in challenging environments, especially if they are left to operate unattended under limited on-board power. This paper reports on first results of an energy-aware path planner, which can provide estimates of the driving energy consumption and energy recovery of a robot traversing complex uneven terrains. Energy is es...
-
This work presents the design and autonomous navigation policy of the Resilient Micro Flyer, a new type of collision-tolerant robot tailored to fly through extremely confined environments and manhole-sized tubes. The robot maintains a low weight (<500g) and implements a combined rigid-compliant design through the integration of elastic flaps around its stiff collision-tolerant frame. These passive...
-
Shallow water environments pose daunting scenarios for the operation of Unmanned Underwater Vehicles (UUVs), due to significantly larger wave disturbances being present in comparison to a typical deep sea situation. Performing inspection and maintenance tasks at close quarters in these conditions requires reliable control methods robust to external disturbances, allowing accurate position and atti...
-
The energy of ocean waves is the key distinguishing factor of marine environments compared to other aquatic environments such as lakes and rivers. Waves significantly affect the dynamics of marine vehicles; hence it is imperative to consider the dynamics of vehicles in waves when developing efficient control strategies for autonomous surface vehicles (ASVs). However, most marine simulators availab...
-
We present a highly efficient approach to compute continuous shortest path vector fields on arbitrarily shaped 3D triangular meshes for robot navigation in complex real-world outdoor environments. The continuity of the vector field allows to query the shortest distance, direction and geodesic path to the goal at any point within the mesh triangles, resulting in accurate paths. In order to avoid im...
-
Modeling dynamics of deformable linear objects (DLOs), such as cables, hoses, sutures, and catheters, is an important and challenging problem for many robotic manipulation applications. In this paper, we propose the first method to model and learn full 3D dynamics of DLOs from data. Our approach is capable of capturing the complex twisting and bending dynamics of DLOs and allows local effects to p...
-
This study presents a novel dynamic modelling method for analyzing the modal properties of a curved extensible continuum manipulator (ECM). Considering the variable bending angle and the length, the kinematic and static models are firstly established for descripting the geometric posture deformation and deflection generated by gravity. Then, the modal dynamic model is developed along the centerlin...
-
This paper presents an online informative path planning approach for active information gathering on three-dimensional surfaces using aerial robots. Most existing works on surface inspection focus on planning a path offline that can provide full coverage of the surface, which inherently assumes the surface information is uniformly distributed hence ignoring potential spatial correlations of the in...
-
In this paper, we present a model based approach for predicting the forward kinematics of a tool for picking prepreg fiber plies from a flat table and draping them onto a double curved mold. The tool consists of suction cups interlinked with springs. The tool has 60 actuated and 240 passive degrees of freedom. The prediction is based on establishing a kinematic model of the total potential energy ...
-
Ellipsoid-based manipulability measures are often used to characterize the force/velocity task-space capabilities of robots. While computationally simple, this approach largely approximate and underestimate the true capabilities. Force/velocity polytopes appear to be a more appropriate representation to characterize the robot’s task-space capabilities. However, due to the computational complexity ...
-
Teleoperation deals with extraordinary situations where an external operator takes over the control of an autonomous vehicle. Especially in complex urban scenarios, this may cause a too high workload for the human operator, resulting in suboptimal solutions. This contribution presents a teleoperation paradigm to raise the autonomy level of teleoperated driving, while the operator still remains the...
-
We propose a combination of Model Predictive Control and Lexicographic Programming to address complex scenarios with conflicting goals related to various aspects of comfort and safety of passengers in a transit bus, generating different deceleration profiles depending on the speed of the bus and distance to obstacles, validated in experiments with a standard transit bus equipped with self-driving ...
-
The problem of safe interception of multiple intruder UAVs by a team of cooperating autonomous aerial vehicles is addressed in this paper. The presented work is motivated by the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where this task was simplified to an interaction with a set of static and dynamic objects (balloons and a UAV), and by a real autonomous aerial interception ...
-
The compact flocking of relatively localized Un-manned Aerial Vehicles (UAVs) in high obstacle density areas is discussed in this paper. The presented work tackles realistic scenarios in which the environment map is not known apriori and the use of a global localization system and communication infrastructure is difficult due to the presence of obstacles. To achieve flocking in such a constrained ...
-
This paper presents a manipulator-equipped unmanned aerial vehicle (UAV) performing contact-based surface inspection. Using stereo camera information, a contact point is determined and an approach path is generated to ensure an autonomous workflow. The proposed inspection method uses an impedance controller to regulate the force applied by the end-effector to the contact surface. Due to the often ...
-
Formation control of multi-agent systems deals with groups of robots forming specific spatial geometries. Combined with the advancements of unmanned aerial vehicles (UAVs) in the past decade, formation control may potentially be applied to tasks such as search-and-rescue, surveillance, even collaborative manipulation. A key challenge is the decentralization of formation control, where each agent b...
-
In this paper, we present a model predictive controller for a fully actuated aerial manipulator to track a hybrid force and pose trajectory at the end-effector in an aerial interaction task. A force sensor at the end-effector is used to detect contact and to directly control the interaction force. We propose an approach for automatic transition between three operation modes which reflect the state...
-
Unmanned aerial vehicles (UAVs) are finding their way into offshore applications. In this work, we postulate an original system that entails a marine locomotive quadrotor UAV that manipulates the velocity of a floating buoy by means of a cable. By leveraging the advantages of UAVs relative to high speed, maneuverability, ease of deployment, and wide field of vision, the proposed UAV−buoy system pa...
-
Visual-Inertial Odometry (VIO) has been widely used and researched to control and aid the automation of navigation of robots especially in the absence of absolute position measurements, such as GPS. However, when the observable landmarks in the scene lie far away, as in high-altitude flights for example, the fidelity of the metric scale estimate in VIO greatly degrades. Aiming to tackle this issue...
-
In this paper, we present an integrated system that includes reasoning from visual and natural language inputs, action and motion planning, executing tasks by a robotic arm, manipulating objects, and discovering their properties. A vision to action module recognises the scene with objects and their attributes and analyses enquiries formulated in natural language. It performs multi-modal reasoning ...
-
Robotic teleoperation provides human-in-the-loop capabilities of complex manipulation tasks in dangerous or remote environments, such as for planetary exploration or nuclear decommissioning. This work proposes a novel telemanipulation architecture using a passive Fractal Impedance Controller (FIC), which does not depend upon an active viscous component for guaranteeing stability. Compared to a tra...
-
Robotic tasks such as manipulation with visual inputs require image features that capture the physical properties of the scene, e.g., the position and configuration of objects. Recently, it has been suggested to learn such features in an unsupervised manner from simulated, self-supervised, robot interaction; the idea being that high-level physical properties are well captured by modern physical si...
-
We present a novel framework for self-supervised grasped object segmentation with a robotic manipulator. Our method successively learns an agnostic foreground segmentation followed by a distinction between manipulator and object solely by observing the motion between consecutive RGB frames. In contrast to previous approaches, we propose a single, end-to-end trainable architecture which jointly inc...
-
The connectivity of distributed networked multi-robot systems is a crucial operational specification, since the involved robots interact/communicate locally only with their immediate neighbors. Thus, in this work, we propose a distributed algorithm to estimate the algebraic connectivity of the underlying communication graph, which stands as a valid connectivity metric. Our method establishes robus...
-
Recent work in the multi-agent domain has shown the promise of Graph Neural Networks (GNNs) to learn complex coordination strategies. However, most current approaches use minor variants of a Graph Convolutional Network (GCN), which applies a convolution to the communication graph formed by the multi-agent system. In this paper, we investigate whether the performance and generalization of GCNs can ...
-
For robots to leave the safety of the laboratory and explore the world, maneuverability will need to be mastered. However, transient motions, such as rapid acceleration and deceleration, have received little attention in the literature. This is mainly due to the complexity of analyzing these high dimensional systems that have no closed-form solution which makes controller design a non-trivial task...
-
The problem of dynamic locomotion over rough terrain requires both accurate foot placement together with an emphasis on dynamic stability. Existing approaches to this problem prioritize immediate safe foot placement over longer term dynamic stability considerations, or relegate the coordination of foot placement and dynamic stability to heuristic methods. We propose a multi-layered locomotion fram...
-
The kinematic features of a centaur-type humanoid platform, combined with a powerful actuation, enable the experimentation of a variety of agile and dynamic motions. However, the higher number of degrees-of-freedom and the increased weight of the system, compared to the bipedal and quadrupedal counterparts, pose significant research challenges in terms of computational load and real implementation...
-
Planning for legged-wheeled machines is typically done using trajectory optimization because of many degrees of freedom, thus rendering legged-wheeled planners prone to falling prey to bad local minima. We present a combined sampling and optimization-based planning approach that can cope with challenging terrain. The sampling-based stage computes whole-body configurations and contact schedule, whi...
-
In recent years, mobility on demand has experienced a major revival due to various ride-hailing companies entering the market. Competing in this field requires an efficient operation. Therefore, the applied policy, which cares for vehicle-to-customer assignment and vehicle repositioning, has to achieve good customer service and minimize cost while trying to keep the impact on the environment as lo...
-
The paper proposes novel sampling strategies to compute the optimal path alteration of a surface vessel sailing in close quarters. Such strategy directly encodes the rules for safe navigation at sea, by exploiting the concept of minimal ship domain to determine the compliant region where the path deviation is to be generated. The sampling strategy is integrated within the optimal rapidly-exploring...
-
Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scena...
-
In this paper, we present self-supervised shared latent embedding (S3LE), a data-driven motion retargeting method that enables the generation of natural motions in humanoid robots from motion capture data or RGB videos. While it requires paired data consisting of human poses and their corresponding robot configurations, it significantly alleviates the necessity of time-consuming data-collection vi...
-
Sparse roadmaps are important to compactly represent state spaces, to determine problems to be infeasible and to terminate in finite time. However, sparse roadmaps do not scale well to high-dimensional planning problems. In prior work, we showed improved planning performance on high-dimensional planning problems by using multilevel abstractions to simplify state spaces. In this work, we generalize...
-
In this paper, we consider disassembly scenarios for real-world 3D CAD-data, where each component is defined by a triangle mesh. For a fast construction of collision-free disassembly paths, common approaches use sampling-based rigid body motion planning which is well studied in the literature. One fact that has so far received little attention is that in industrial disassembly scenarios components...
-
Search-based methods that use motion primitives can incorporate the system’s dynamics into the planning and thus generate dynamically feasible MAV trajectories that are globally optimal. However, searching high-dimensional state lattices is computationally expensive. Local multiresolution is a commonly used method to accelerate spatial path planning. While paths within the vicinity of the robot ar...
-
Within the Multi-Platform Inspection, Maintenance and Repair in Extreme Environments (MIMRee) project, a lightweight and multifunctional robotic repair arm is created for wind turbine blades. The design features a toolbox at the base of the arm housing multiple end-effector tools and an autonomous end-effector tool-changer. The arm communicates commands and data via internet with a bespoke user in...
-
Reacting to unexpected conditions and recovering from errors is crucial for robots to perform their missions continuously in dynamic environments. This paper introduces a novel framework aiming to handle errors impairing the quality of robot services concerning on-line management of error recovery. The framework is a combination of an execution generation tool, a learning module, and a recovery pi...
-
Minimally invasive wireless devices, allowing the sampling of gut’s bacteria, are needed for a longitudinal understanding of the role of the microbiota on the human health. Herein, we present a novel magnetic actuation system fitting inside a 11.5 × 30.5 mm wireless ingestible capsule. Lacking any electronic components, the capsule robot is designed for the collection of microbiota’s samples throu...
-
This paper addresses the problem of real-time planning in constrained dual-arm manipulation scenarios. Our proposed coupling leverages manipulability information of the cooperative bimanual task-space to a vector-field based planner by means of a repulsive circulatory field, while geometric primitives in Spin(3)⋉ℝ3 are explored for flexible task relaxation. Furthermore, the circular field informs ...
-
We present a novel hierarchical POMDP framework to solve an object search and delivery task where the agent is given a prior belief about the possible item locations. Solving POMDPs is computationally demanding and, as such, applications have typically been limited to small environments. The proposed hierarchical POMDP framework performs reasoning on multiple spatial scales in order to reduce comp...
-
Existing work on sequential manipulation planning and trajectory optimization typically assumes the robot, environment and tools to be given. However, in particular in industrial applications, it is highly interesting to ask, what would be an optimal robot design, tool shape, or robot station geometry for a particular ensemble of manipulation tasks. To tackle this problem we propose a formulation ...
-
Recent advances in incorporating neural networks into particle filters provide the desired flexibility to apply particle filters in large-scale real-world applications. The dynamic and measurement models in this framework are learnable through the differentiable implementation of particle filters. Past efforts in optimising such models often require the knowledge of true states which can be expens...
-
Most existing approaches for visual localization either need a detailed 3D model of the environment or, in the case of learning-based methods, must be retrained for each new scene. This can either be very expensive or simply impossible for large, unknown environments, for example in search-and-rescue scenarios. Although there are learning-based approaches that operate scene-agnostically, the gener...
-
We present a Multi-Criteria Decision Making (MCDM) framework specifically designed for planetary exploration. Our work is based on PROMETHEE II, which allows operators to add task-specific criteria and conditions. We extended this algorithm to improve its resource usage by reducing the number of candidate exploration goals that have to be evaluated and compared. This is crucial when given a large ...
-
Conventional SLAM techniques strongly rely on scene rigidity to solve data association, ignoring dynamic parts of the scene. In this work we present Semi-Direct DefSLAM (SD-DefSLAM), a novel monocular deformable SLAM method able to map highly deforming environments, built on top of DefSLAM [1]. To robustly solve data association in challenging deforming scenes, SD-DefSLAM combines direct and indir...
-
We present an approach to learn fast and dynamic robot motions without exceeding limits on the position θ, velocity $\dot \theta $ , acceleration $\ddot \theta $ and jerk $\dddot \theta $ of each robot joint. Movements are generated by mapping the predictions of a neural network to safely executable joint accelerations. The neural network is invoked periodically and trained via reinforcement learn...
-
Imitation learning (IL) enables robots to acquire skills quickly by transferring expert knowledge, which is widely adopted in reinforcement learning (RL) to initialize exploration. However, in long-horizon motion planning tasks, a challenging problem in deploying IL and RL methods is how to generate and collect massive, broadly distributed data such that these methods can generalize effectively. I...
-
Robot learning is often simplified to planar manipulation due to its data consumption. Then, a common approach is to use a fully-convolutional neural network (FCNN) to estimate the reward of grasp primitives. In this work, we extend this approach by parametrizing the two remaining, lateral degrees of freedom (DoFs) of the primitives. We apply this principle to the task of 6 DoF bin picking: We int...
-
We present an approach based on conditional generative adversarial networks (GANs) to generate grasps directly and in a feed-forward manner from a raw depth image input. Building on the recently introduced StyleGAN architecture we extend results from an earlier proof-of-concept paper [1] and demonstrate successful sim2real transfer of grasp outputs for a robot arm with a Shadow Dexterous Hand. We ...
-
We combine Boston Dynamics Spot® with a light-weight, external robot arm to perform dynamic grasping maneuvers. While Spot is a reliable, robust and easy-to-control mobile robot, these highly desirable qualities come with the price that the control access granted to the user is restricted. Consequently Spot’s behavior must largely be treated as a black box, which causes difficulties when combined ...
-
While there exists many methods for manipulating rigid objects with parallel-jaw grippers, grasping with multi-finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we p...
-
Reinforcement learning (RL) has achieved some impressive recent successes in various computer games and simulations. Most of these successes are based on having large numbers of episodes from which the agent can learn. In typical robotic applications, however, the number of feasible attempts is very limited. In this paper we present a sample-efficient RL algorithm applied to the example of a table...
-
Avoiding face-touches has been one of the most common medical recommendations since the beginning of the COVID-19 pandemic. This work aims at providing people with help in contrasting this widespread, yet noxious habit. The solution we present exploits wearable devices to detect hand motions ending up into a face-touch and promptly notify the user exploiting haptic feedback. To this aim, we propos...
-
This paper presents a contact-aided inertial-kinematic floating base estimation for humanoid robots considering an evolution of the state and observations over matrix Lie groups. This is achieved through the application of a geometrically meaningful estimator which is characterized by concentrated Gaussian distributions. The configuration of a floating base system like a humanoid robot usually req...
-
We present a system which improves the safety and accuracy of impedance controlled robotic manipulators with proximity perception. Proximity servoed manipulators, which use proximity sensors attached to the robot’s outer shell, have recently demonstrated robust collision avoidance abilities. Nevertheless, unwanted collisions cannot be avoided entirely. As a fallback safety mechanism, robots with j...
-
We propose a safety control approach based on the online optimal scaling of the size of bounding volumes used as dynamic safety zones for collaborative robotics. Intersection tests between bounding volumes surrounding robot and human allow the safety controller to identify possible collisions. Our proposed approach optimizes online smooth stop trajectories, to be engaged if a potential collision i...
-
The need to guarantee safety of collaborative robots limits their performance, in particular, their speed and hence cycle time. The standard ISO/TS 15066 defines the Power and Force Limiting operation mode and prescribes force thresholds that a moving robot is allowed to exert on human body parts during impact, along with a simple formula to obtain maximum allowed speed of the robot in the whole w...
-
In this paper, we propose a novel safe, passive, and robust control law for mechanical systems. The proposed approach addresses safety from a physical human-robot interaction perspective, where a robot must not only stay inside a pre-defined region, but respect velocity constraints and ensure passivity with respect to external perturbations that may arise from a human or the environment. The propo...
-
Physical human-robot interaction is known to be a crucial aspect in modern lightweight robotics. Herein, the estimation of external interactions is essential for the effective and safe collaboration. In this work, an extended momentum-based disturbance observer is presented which includes the sensing redundancy related to additional force-torque measurements. The observer eliminates the need for a...
-
In physical human-robot interaction (pHRI), robots need to detect and react to intended and unintended contacts in a safe manner. Proprioceptive sensing capabilities and collision detection and identification techniques differ among commercially available robots, which means that also their sensitivity to detect dynamic collisions with the environment or the human co-worker differ. Up to now, ther...
-
This paper considers a scenario where a robot and a human operator share the same workspace, and the robot is able to both carry out autonomous tasks and physically interact with the human in order to achieve common goals. In this context, both intentional and accidental contacts between human and robot might occur due to the complexity of tasks and environment, to the uncertainty of human behavio...
-
We propose a practical approach for detecting the event that a human wearing an IMU-equipped bracelet points at a moving robot; the approach uses a learned classifier to verify if the robot motion (as measured by its odometry) matches the wrist motion, and does not require that the relative pose of the operator and robot is known in advance. To train the model and validate the system, we collect d...
-
We present PATHoBot an autonomous crop surveying and intervention robot for glasshouse environments. The aim of this platform is to autonomously gather high quality data and also estimate key phenotypic parameters. To achieve this we retro-fit an off-the-shelf pipe-rail trolley with an array of multi-modal cameras, navigation sensors and a robotic arm for close surveying tasks and intervention. In...
-
Image-based yield detection in agriculture could raise harvest efficiency and cultivation performance of farms. Following this goal, this research focuses on improving instance segmentation of field crops under varying environmental conditions. Five data sets of cabbage plants were recorded under varying lighting outdoor conditions. The images were acquired using a commercial mono camera. Addition...
-
In this work, we propose a global planning strategy specifically designed for precision agriculture settings, where field activities may have different requirements ranging from a full orchard inspection to sparse targeted per-plant interventions. This global planning strategy is formulated as a novel Multi-Platform Steiner Traveling Salesman Problem (MP-STSP) where, in order to guarantee the expl...
-
We consider the problem of people search by a mobile social robot in case of a situation that cannot be solved by the robot alone. Examples are physically opening a closed door or operating an elevator. Based on the Behavior Tree framework, we create a modular and easily extendable action sequence with the goal of finding a person to assist the robot. By decomposing the Behavior Tree as a Discrete...
-
Safety integration components for robotic applications are a mandatory feature for any autonomous mobile application, including human avoidance behaviors. This paper proposes a novel parametrizable scene risk evaluator for open-field applications that use humans motion predictions and pre-defined hazard zones to estimate a braking factor. Parameters optimization uses simulated data. The evaluation...
-
During the off-road path following of a wheeled mobile robot in presence of poor grip conditions, the longitudinal velocity should be limited in order to maintain safe navigation with limited tracking errors, while at the same time being high enough to minimize travel time. Thus, this paper presents a new approach of online speed fluctuation, capable of limiting the lateral error below a given thr...
-
A two-wheeled self-balancing robot is a statically unstable non-linear system with strong coupling dynamics. Common practices in the development of control systems for such robots are either to linearise the region of application to be used with linear controllers or to use complex nonlinear controllers such as Fuzzy logic, Sliding Mode, and Neural Networks. Nonetheless, in this paper, we are prop...
-
Multi-agent control of a robot using multiple controllers is vital in domains like shared control and reliable control. The strategy of assigning a varying priority (authority) to each agent controller, and commanding the robot using an authority-weighted sum of the forces produced by all the agents has been exploited in prior works. In this paper, firstly, we show that this strategy results in a ...
-
Connectivity maintenance is crucial for the real world deployment of multi-robot systems, as it ultimately allows the robots to communicate, coordinate and perform tasks in a collaborative way. A connectivity maintenance controller must keep the multi-robot system connected independently from the system’s mission and in the presence of undesired real world effects such as communication delays, mod...
-
This paper presents a novel control strategy to herd a group of non-cooperative evaders by means of a team of robotic herders. In herding problems, the motion of the evaders is typically determined by strong nonlinear reactive dynamics, escaping from the herders. Many applications demand the herding of numerous and/or heterogeneous entities, making the development of flexible control solutions cha...
-
State of the art mapping algorithms can produce high-quality maps. However, they are still vulnerable to clutter and outliers which can affect map quality and in consequence hinder the performance of a robot, and further map processing for semantic understanding of the environment. This paper presents ROSE, a method for building-level structure detection in robotic maps. ROSE exploits the fact tha...
-
Self-collision detection and avoidance are essential for reactive control, in particular for dynamics robots equipped with legs or arms. Yet, only few control methods are able to handle such constraints, and it is often necessary to rely on path planning to define a collision-free trajectory that the controller would then track. In this paper, we introduce a combination of two lightweight, conserv...
-
We propose an efficient method to estimate the relative pose of a multi-camera system from a minimum of two affine correspondences (ACs). Our solution is novel as it computes the 6DOF relative pose by utilizing a first-order rotation approximation. We directly derive a single polynomial based on the constraint between ACs and the generalized camera model. Then a closed-form solution is found analy...
-
Autonomous exploration allows mobile robots to navigate in initially unknown territories in order to build complete representations of the environments. In many real-life applications, environments often contain dynamic obstacles which can compromise the exploration process by temporarily blocking passages, narrow paths, exits or entrances to other areas yet to be explored. In this work, we formul...
-
Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick- and-place from raw images. Conditioning VMC on desired goal states is a promising way of achieving versatile skill primitives. However, common conditioning schemes either rely on task-specific fine tuning - e.g. using one-shot imitation learning (IL) - or on sampling approaches using a for...
-
Aerial manipulation increases significantly the workspace size of robotic manipulators. However, aerial manipulation suffers from a lack of autonomy due to limited embedded energy. The Aerial Manipulator with Elastic Suspension (AMES) is designed to cope with this issue. It is an omnidirectional aerial vehicle equipped with a gripper and suspended under a robotic carrier by a spring for gravity co...
-
Unmanned Aerial Vehicles (UAVs) have introduced benefits in many areas of the energy sector. Today, power line sensor deployment is manually executed on passive power lines using helicopters, introducing great risks, costs and difficulties for the power distribution companies and the human operators.In this paper, we present a novel modular mechanical system utilizing pneumatic as the actuation so...
-
Precise system identification is an important aspect of adequate control design and parameter definition to allow for accurate and reliable navigation. While this is well known in robotics, the community working with small rotorcraft Unmanned Aerial Vehicles (UAVs) has yet to discover the benefits. In contrast to existing work, which often performs offline or deterministic (i.e. closed-form) syste...
-
This paper presents a reactive legged locomotion generation scheme that enables our quadruped robot CEN-TAURO to adapt to varying payloads while walking. The center-of-mass (CoM) trajectories are generated in real time in a model predictive control (MPC) fashion, trading off large stability margins against evenly stretched legs. Vertex-based zero-moment-point (ZMP) constraints are imposed to ensur...
-
A practical approach to robot reinforcement learning is to first collect a large batch of real or simulated robot interaction data, using some data collection policy, and then learn from this data to perform various tasks, using offline learning algorithms. Previous work focused on manually designing the data collection policy, and on tasks where suitable policies can easily be designed, such as r...
-
This work presents an approach for control, state-estimation and learning model (hyper)parameters for robotic manipulators. It is based on the active inference framework, prominent in computational neuroscience as a theory of the brain, where behaviour arises from minimizing variational free-energy. First, we show there is a direct relationship between active inference controllers, and classic met...
-
Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel approach to infer optimal skill parameters directly fro...
-
The cerebellum plays a distinctive role within our motor control system to achieve fine and coordinated motions. While cerebellar lesions do not lead to a complete loss of motor functions, both action and perception are severally impacted. Hence, it is assumed that the cerebellum uses an internal forward model to provide anticipatory signals by learning from the error in sensory states. In some st...
-
The combinatorics inherent to the issue of planning legged locomotion can be addressed by decomposing the problem: first, select a guide path abstracting the contacts with a heuristic model; then compute the contact sequence to balance the robot gait along the guide path. While several models have been proposed to compute such a path, none have yet managed to efficiently capture the complexity of ...
-
Deformable object manipulation tasks have long been regarded as challenging robotic problems. However, until recently very little work has been done on the subject, with most robotic manipulation methods being developed for rigid objects. Deformable objects are more difficult to model and simulate, which has limited the use of model-free Reinforcement Learning (RL) strategies, due to their need fo...
-
This paper deals with a problem of precise jumping for legged robots: what are the control inputs required to perform a jump that results in a desired landing point? We propose a novel precise jump planning method, formulated as a bilevel optimization problem. The presented formulation exploits certain insights into the jump dynamics, leading to a low-dimensional optimization problem, and allowing...
-
Bipedal walking is one of the most difficult but exciting challenges in robotics. The difficulties arise from the complexity of high-dimensional dynamics, sensing and actuation limitations combined with real-time and computational constraints. Deep Reinforcement Learning (DRL) holds the promise to address these issues by fully exploiting the robot dynamics with minimal craftsmanship. In this paper...
-
While many advancements have been made in the development of template models for describing upright-trunk locomotion, the majority of the effort has been focused on the stance phase. In this paper, we develop a new compact dynamic model as a first step toward a fully unified locomotion template model (ULT-model) of an upright-trunk forward hopping system, which will also require a unified control ...
-
Template models are frequently used to simplify the control dynamics for robot hopping or running. Passive limit cycles can emerge for such systems and be exploited for energy-efficient control. A grand challenge in locomotion is trunk stabilization when the hip is offset from the center of mass (CoM). The swing phase plays a major role in this process due to the moment of inertia of the leg; howe...
-
In reality, there is still much to be done for robots to be able to perform manipulation actions with full autonomy. Complicated manipulation tasks, such as cooking, may still require a person to perform some actions that are very risky for a robot to perform. On the other hand, some other actions may be very risky for a human with physical disabilities to perform. Therefore, it is necessary to ba...
-
Recent research in AI ethics has put forth explainability as an essential principle for AI algorithms. However, it is still unclear how this is to be implemented in practice for specific classes of algorithms—such as motion planners. In this paper we unpack the concept of explanation in the context of motion planning, introducing a new taxonomy of kinds and purposes of explanations in this context...
-
Estimating the engagement of children is an essential prerequisite for constructing natural Child-Robot Interaction. Especially in the case of children with Autism Spectrum Disorder, monitoring the engagement of the other party allows robots to adjust their actions according to the educational and therapeutic goals in hand. In this work we delve into engagement estimation with a focus on children ...
-
With growing access to versatile robotics, it is beneficial for end users to be able to teach robots tasks without needing to code a control policy. One possibility is to teach the robot through successful task executions. However, near-optimal demonstrations of a task can be difficult to provide and even successful demonstrations can fail to capture task aspects key to robust skill replication. H...
-
Aleatoric uncertainty estimation, based on the observed training data, is applied for the detection of conflicts in a demonstration data set. The particular focus of this paper is the resolution of conflicting data resulting from scenarios with equivalent action choices, such as obstacle avoidance, path planning or multiple joint configurations. In terms of the estimated uncertainty, the proposed ...
-
Human interaction involves very sophisticated non-verbal communication skills like understanding the goals and actions of others and coordinating our own actions accordingly. Neuroscience refers to this mechanism as motor resonance, in the sense that the perception of another persons actions and sensory experiences activates the observer’s brain as if (s)he would be performing the same actions and...
-
A key challenge towards autonomous multi-part object assembly is robust sensorimotor control under uncertainty. In contrast to previous works that rely on a priori knowledge on whether two parts match, we aim to learn this through physical interaction. We propose a hierarchical approach that enables a robot to autonomously assemble parts while being uncertain about part types and positions. In par...
-
In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and the zooming ratio of the laparoscope, following the surgeon’s instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjustment of zooming ratio may interrupt the workflow of the surgical operation. To this end, we propose ...
-
The research and development of socially interactive robots is a complex challenge because of the wide variety of capabilities needed for effective social human-robot interactions (HRI). Many of these capabilities, including perception, dialog, and control, have state of the art methods and solutions, but combining those into a comprehensive and seamless interaction is still an open challenge. We ...
-
Robots hold great potential for supporting exercise and physical therapy, but such systems are often cumbersome to set up and require expert supervision. We aim to solve these concerns by combining Captury Live, a real-time markerless motion-capture system, with a Rethink Robotics Baxter Research Robot to create the Robot Interaction Studio. We evaluated this platform for unsupervised human-robot ...
-
Talking gestures are a fundamental part of body language and, therefore, are also important for social robots. Gesture generation by generative approaches is supposed to produce a more appropriate behavior than rule-based approaches. Usually, the evaluation of generated gestures is carried out by subjective visual evaluation, which could be cultural dependent and influenced by external factors. In...
-
We propose an approach for robot-supervised learning that automates label generation for semantic segmentation with Convolutional Neural Networks (CNNs) for crop row detection in a field. Using a training robot equipped with RTK GNSS and RGB camera, we train a neural network that can later be used for pure vision-based navigation. We test our approach on an agri-robot in a strawberry field and suc...
-
We have recently proposed two pile loading controllers that learn from human demonstrations: a neural network (NNet) [1] and a random forest (RF) controller [2]. In the field experiments the RF controller obtained clearly better success rates. In this work, the previous findings are drastically revised by experimenting summer time trained controllers in winter conditions. The winter experiments re...
-
Concentric Tube Robots (CTRs), a type of continuum robot, are a collection of concentric, pre-curved tubes composed of super elastic nickel titanium alloy. CTRs can bend and twist from the interactions between neighboring tubes causing the kinematics and therefore control of the end-effector to be very challenging to model. In this paper, we develop a control scheme for a CTR end-effector in Carte...
-
Keyhole neurosurgery is challenging, due to the complex anatomy of the brain and the inherent risk of damaging vital structures while reaching the surgical target. This paper presents a path planner for safe and effective neurosurgical interventions. The strengths of the proposed framework lay in the integration of multiple risk structures combined into a deductive method for fast and intuitive us...
-
Robotic Electrospinning Actuated by Non-Circular Joint Continuum Manipulator for Endoluminal Therapy
Electrospinning has exhibited excellent benefits to treat the trauma for tissue engineering due to its produced micro/nano fibrous structure. It can effectively adhere to the tissue surface for long-term continuous therapy. This paper develops a robotic electrospinning platform for endoluminal therapy. The platform consists of a continuum manipulator, the electrospinning device, and the actuation ...
-
In this paper we solve the full-consensus problem for multiple nonholonomic vehicles interacting over a directed leader-follower topology and subject to sensing constraints in the form of limited range and limited field-of-view. Remarkably, based on a polar-coordinates model transformation, the designed controller is time-invariant and smooth (in the domain of definition). Moreover, the control la...
-
This paper presents a novel approach to recover outdated cross-covariance between correlated agents at the moment they perform joint observations. This allows to render Collaborative State Estimation (CSE) fully distributed, with communication only required for the moment of joint observation and most importantly, it significantly reduces the maintenance effort in case of high frequent propagation...
-
Most recent works on multi-target tracking with multiple cameras focus on centralized systems. In contrast, this paper presents a multi-target tracking approach implemented in a distributed camera network. The advantages of distributed systems lie in lighter communication management, greater robustness to failures and local decision making. On the other hand, data association and information fusio...
-
GenGrid is a novel comprehensive open-source, distributed platform intended for conducting extensive swarm robotic experiments. The modular platform is designed to run swarm robotics experiments that are compatible with different types of mobile robots ranging from Colias, Kilobot, and E-puck [1]–[4]. The platform offers programmable control over the experimental setup and its parameters and acts ...
-
Safely generating impacts with robots is challenging due to subsequent discontinuous velocity and high impact forces. We aim at increasing the impact velocity – the robot’s relative speed prior to contact – such that impact-tasks like grabbing and boxing are made with the highest allowable speed performance when needed. Previous works addressed this problem for rigid bodies’ impacts. This letter p...
-
State estimation, in particular estimation of the base position, orientation and velocity, plays a big role in the efficiency of legged robot stabilization. The estimation of the base state is particularly important because of its strong correlation with the underactuated dynamics, i.e. the evolution of center of mass and angular momentum. Yet this estimation is typically done in two phases, first...
-
Well-established optimization-based methods can guarantee an optimal trajectory for a short optimization horizon, typically no longer than a few seconds. As a result, choosing the optimal trajectory for this short horizon may still result in a sub-optimal long-term solution. At the same time, the resulting short-term trajectories allow for effective, comfortable and provable safe maneuvers in a dy...
-
Making decisions in complex driving environments is a challenging task for autonomous agents. Imitation learning methods have great potentials for achieving such a goal. Adversarial Inverse Reinforcement Learning (AIRL) is one of the state-of-art imitation learning methods that can learn both a behavioral policy and a reward function simultaneously, yet it is only demonstrated in simple and static...
-
We propose an integrated prediction and planning system for autonomous driving which uses rational inverse planning to recognise the goals of other vehicles. Goal recognition informs a Monte Carlo Tree Search (MCTS) algorithm to plan optimal maneuvers for the ego vehicle. Inverse planning and MCTS utilise a shared set of defined maneuvers and macro actions to construct plans which are explainable ...
-
Driving styles play a major role in the acceptance and use of autonomous vehicles. Yet, existing motion planning techniques can often only incorporate simple driving styles that are modeled by the developers of the planner and not tailored to the passenger. We present a new approach to encode human driving styles through the use of signal temporal logic and its robustness metrics. Specifically, we...
-
In this paper, we propose a robust end-to-end multi-modal pipeline for place recognition where the sensor systems can differ from the map building to the query. Our approach operates directly on images and LiDAR scans without requiring any local feature extraction modules. By projecting the sensor data onto the unit sphere, we learn a multi-modal descriptor of partially overlapping scenes using a ...
-
EMG-driven musculoskeletal model has been broadly used to detect human intention in rehabilitation robots. This approach computes muscle-tendon force and translates it to the joint kinematics. However, the muscle-tendon parameters of the musculoskeletal model are difficult to measure in vivo and varied across subjects. In this study, a direct collocation (DC) method is proposed to optimize the sub...
-
Motivated by the limited flight time of batterypowered multi-rotor UAVs, in this paper we address the problem of generating energy-optimal trajectories for a planar quadrotor. More specifically, by considering an accurate electrical model for the brushless DC motors and rest-to-rest maneuvers between two predefined boundary states, we explicitly compute the minimum-energy curves by adopting a free...
-
This paper presents a non-linear optimization method for trajectory planning of tethered aerial robots. Particularly, the paper addresses the planning problem of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) by means of a tether. The result is a collision-free trajectory for UAV and tether, assuming the UGV position is static. The optimizer takes into account constrai...
-
Soft isoperimetric truss robots have demonstrated an ability to grasp and manipulate objects using the members of their structure. The compliance of the members affords large contact areas with even force distribution, allowing for successful grasping even with imprecise open-loop control. In this work we present methods of analyzing and controlling isoperimetric truss robots in the context of gra...
-
We present a method for classifying the quality of near-contact grasps using spatial metrics that are recoverable from sensor data. Current methods often rely on calculating precise contact points, which are difficult to calculate in real life, or on tactile sensors or image data, which may be unavailable for some applications. Our method, in contrast, uses a mix of spatial metrics that do not dep...
-
The Dorsal Grasper, an assistive wearable grasping device, incorporates supernumerary fingers and an artificial palm with the forearm and back of the hand, respectively. It enables power wrap grasping and adduction pinching with its V-shaped soft fingers. Designed with C6/C7 spinal cord injury in mind, it takes advantage of active wrist extension that remains in this population after injury. We pr...
-
Occupancy grids have been widely used for mapping with mobile robots for several decades. Occupancy grids discretize the analog environment and seek to determine the occupancy probability of each cell. More recent occupancy grid mapping algorithms have shown the advantage of capturing cell correlations in the measurement model and the posterior. By estimating the probability of a given map as oppo...
-
We propose a new, data-efficient approach for skill transfer to novel objects, accounting for known categorical shape variation. A low-dimensional shape representation embedding is learned from a set of deformations, sampled between known objects within a category. This latent representation is mapped to a set of control parameters that result in successful execution of a category-level skill on t...
-
For a dynamical system, safety is typically guaranteed by constraining the system states within a set defined a priori. A popular approach is to use control barrier functions (CBFs) that encode safety using a smooth function. However, typical constructions of the smooth function do not account for any notion of safety uncertainty for the system inside the safe set. Although, one can formulate unce...
-
Robotic object rearrangement combines the skills of picking and placing objects. When object models are unavailable, typical collision-checking models may be unable to predict collisions in partial point clouds with occlusions, making generation of collision-free grasping or placement trajectories challenging. We propose a learned collision model that accepts scene and query object point clouds an...
-
Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, inspection of narrow pipes, examination of turbine blades, etc. It is also challenging because confined spaces tend to lack a positioning infrastructure, and conventional sensors often cannot detect objects in close range. Therefore, such tasks require a sensor that is compact, operates in sho...
-
In this paper, we design a versatile multi-sensor aided inertial navigation system (MINS) that can efficiently fuse multi-modal measurements of IMU, camera, wheel encoder, GPS, and 3D LiDAR along with online spatiotemporal sensor calibration. Building upon our prior work [1] –[3], in this work we primarily focus on efficient LiDAR integration in a sliding-window filtering fashion. As each 3D LiDAR...
-
In this work we augment our prior state-of-the-art visual-inertial odometry (VIO) system, OpenVINS [1], to produce accurate dense depth by filling in sparse depth estimates (depth completion) from VIO with image guidance – all while focusing on enabling real-time performance of the full VIO+depth system on embedded devices. We show that noisy depth values with varying sparsity produced from a VIO ...
-
This paper develops a probabilistic simultaneous estimation and modeling (SEAM) framework for estimating a robot’s state and correcting its motion model parameters. This is done by incorporating model uncertainty in state prediction and correcting parameters via optimization. In the proposed technique, belief about a state being estimated is represented by arbitrary multi-dimensional non-Gaussian ...
-
In this paper, we present an efficient online calibration system for longitudinal vehicle dynamics of driverless cars. Instead of modeling vehicle’s longitudinal dynamical system analytically, we employ a data-driven method to generate an "end-to-end" numerical model with a look-up table which saves vehicle’s velocity, control command, and acceleration. This reference table should be calibrated to...
-
Testing mobile robots is difficult and expensive, and many faults go undetected. In this work we explore whether fuzzing, an automated test input generation technique, can more quickly find failure inducing inputs in mobile robots. We developed a simple fuzzing adaptation, BASE-FUZZ, and one specialized for fuzzing mobile robots, PHYS-FUZZ. PHYS-FUZZ is unique in that it accounts for physical attr...
-
Estimating the centroidal dynamics of legged robots is crucial in the context of multi-contact locomotion of legged robots. In this paper, we formulate the estimation of centroidal dynamics as a maximum a posteriori problem and we use a differential dynamic programming approach for solving it. The soundness of the proposed approach is first validated on a simulated humanoid robot, where ground tru...
-
This work presents a framework for the simultaneous optimization of motors, transmissions, and mechanisms of different joints of robotic legs with the goal of achieving an energy efficient, precisely controllable and stable locomotion in dynamic environments. This unified framework allowed us to introduce and formulate new performance metrics for the separate evaluation of the system's stabilizing...
-
Dynamic locomotion for legged robots is difficult because the system dynamics are highly nonlinear and complex, nominally underactuated and unstable, multi-input and multi-output, as well as time-variant and hybrid. One usually faces the choice between the intricate full-body dynamics which remains computationally expensive and sometimes even intractable, and the empirically simplified model which...
-
Manipulators can be added to legged robots, allowing them to interact with and change their environment. Legged mobile manipulation planners must consider how contact forces generated by these manipulators affect the system. Current planning strategies either treat these forces as immutable during planning or are unable to optimize over these contact forces while operating in real-time. This paper...
-
We explore how high-speed robot arm motions can dynamically manipulate ropes and cables to vault over obstacles, knock objects from pedestals, and weave between obstacles. In this paper, we propose a self-supervised learning framework that enables a UR5 robot to perform these three tasks. The framework finds a 3D apex point for the robot arm, which, together with a task-specific trajectory functio...
-
Rearranging and manipulating deformable objects such as cables, fabrics, and bags is a long-standing challenge in robotic manipulation. The complex dynamics and high-dimensional configuration spaces of deformables, compared to rigid objects, make manipulation difficult not only for multi-step planning, but even for goal specification. Goals cannot be as easily specified as rigid object poses, and ...
-
We consider the task of grasping a target object based on a natural language command query. Previous work primarily focused on localizing the object given the query, which requires a separate grasp detection module to grasp it. The cascaded application of two pipelines incurs errors in overlapping multi-object cases due to ambiguity in the individal outputs. This work proposes a model named Comman...
-
Many Reinforcement Learning (RL) approaches use joint control signals (positions, velocities, torques) as action space for continuous control tasks. We propose to lift the action space to a higher level in the form of subgoals for a motion generator (a combination of motion planner and trajectory executor). We argue that, by lifting the action space and by leveraging sampling-based motion planners...
-
Neural Networks (NNs) can provide major empirical performance improvements for robotic systems, but they also introduce challenges in formally analyzing those systems’ safety properties. In particular, this work focuses on estimating the forward reachable set of closed-loop systems with NN controllers. Recent work provides bounds on these reachable sets, yet the computationally efficient approache...
-
This paper formulates a methodology to plan and control flat-terrain motions of an underactuated bipedal robot riding a snakeboard, which is a steerable variant of the skateboard. We use tools from non-holonomic motion planning to study snakeboard gaits and develop feedback control strategies that enable bipedal robots to produce the desired gaits while maintaining balance, regulating the magnitud...
-
Global position control for underactuated bipedal walking is a challenging problem due to the lack of actuation on the feet of the robots. In this paper, we apply the Hybrid-Linear Inverted Pendulum (H-LIP) based stepping on 3D underactuated bipedal robots for global position control. The step-to-step (S2S) dynamics of the H-LIP walking approximates the actual S2S dynamics of the walking of the ro...
-
Ultimately, feedback control is about making adjustments using current state information in order to meet an objective in the future. In the control of bipedal locomotion, linear velocity of the center of mass has been widely accepted as the primary variable around which feedback control objectives are formulated. In this paper, we argue that it is easier to predict the one-step ahead evolution of...
-
This paper proposes a hybrid planning framework that generates complex dynamic motion plans for jumping legged robots to traverse challenging terrains. By employing a motion primitive, the original problem is decoupled as path planning followed by a trajectory optimization (TO) module that handles dynamics. A variant of a kinodynamic Rapidly-exploring Random Trees (RRT) planner finds a path as a p...
-
Unmanned Aerial Vehicle (UAV) flight paths have been shown to communicate meaning to human observers, similar to human gestural communication. This paper presents the results of a UAV gesture perception study designed to assess how observer viewpoint perspective may impact how humans perceive the shape of UAV gestural motion. Robot gesture designers have demonstrated that robots can indeed communi...
-
Mobile robots deployed in human-populated environments must be able to safely and comfortably navigate in close proximity to people. Head orientation and gaze are both mechanisms which help people to interpret where other people intend to walk, which in turn enables them to coordinate their movement. Head orientation has previously been leveraged to develop classifiers which are able to predict th...
-
Humans are known to construct cognitive maps of their everyday surroundings using a variety of perceptual inputs. As such, when a human is asked for directions to a particular location, their wayfinding capability in converting this cognitive map into directional instructions is challenged. Owing to spatial anxiety, the language used in the spoken instructions can be vague and often unclear. To ac...
-
In previous work, using a process we call meshing, the reachable state spaces for various continuous and hybrid systems were approximated as a discrete set of states which can then be synthesized into a Markov chain. One of the applications for this approach has been to analyze locomotion policies obtained by reinforcement learning, in a step towards making empirical guarantees about the stability...
-
How best to attach exoskeletons to human limbs is an open and understudied problem. In the case of upperbody exoskeletons, cylindrical handles are commonly used attachments due to ease of use and cost effectiveness. However, handles require active grip strength from the user and may result in undesirable flexion synergy stimulation, thus limiting the robot’s effectiveness. This paper presents a ne...
-
Supervisory control of a humanoid robot in a manipulation task requires coordination of remote perception with robot action, which becomes more demanding with multiple moving cameras available for task supervision. We explore the use of autonomous camera control and selection to reduce operator workload and improve task performance in a supervisory control task. We design a novel approach to auton...
-
Advancements in the domain of physical human-robot interaction (pHRI) have tremendously improved the ability of humans and robots to communicate, collaborate, and coexist. In particular, compliant robotic systems offer many characteristics that can be leveraged towards enabling physical interactions that more efficiently and intuitively communicate intent, making compliant systems potentially usef...
-
Planning under uncertainty is a crucial capability for autonomous systems to operate reliably in uncertain and dynamic environments. The concern of safety becomes even more critical in healthcare settings where robots interact with human patients. In this paper, we propose a novel risk-aware planning framework to minimize the risk of falls by providing a patient with an assistive device. Our appro...
-
Shared autonomy teleoperation can guarantee safety, but does so by reducing the human operator’s control authority, which can lead to reduced levels of human-robot agreement and user satisfaction. This paper presents a novel haptic shared autonomy teleoperation paradigm that uses haptic feedback to inform the user about the inner state of a shared autonomy paradigm, while still guaranteeing safety...
-
This paper presents a novel strategy for intelligent robotic environment reconfiguration applied to overcome mobility constraints with an autonomously exploring mobile manipulation system. A realistic problem arising during exploration of unknown challenging environments is the encountering of untraversable areas –given the robot’s mobility constraints– resulting in the robot getting stuck. We pro...
-
We investigate improving Monte Carlo Tree Search based solvers for Partially Observable Markov Decision Processes (POMDPs), when applied to adaptive sampling problems. We propose improvements in rollout allocation, the action exploration algorithm, and plan commitment. The first allocates a different number of rollouts depending on how many actions the agent has taken in an episode. We find that r...
-
This paper presents an experimental study that investigated how humans interact with viscous, damping-defined mechanical environments and quantified the lower bounds of robotic damping that they can stably interact with. Human subjects performed posture maintenance tasks for different arm postures while holding a robotic arm manipulator simulating unstable (negative) damping-defined environments a...
-
We consider a robot tasked with observing its environment and later selectively summarizing what it saw as a vivid, structured narrative. The robot interacts with an uncertain environment, modelled as a stochastic process, and must decide what events to pay attention to (substance), and how to best make its recording (style) for later compilation of its summary. If carrying a video camera, for exa...
-
Reactive stepping and push recovery for biped robots is often restricted to flat terrains because of the difficulty in computing capture regions for nonlinear dynamic models. In this paper, we address this limitation by proposing a novel 3D reactive stepper, the DeepQ stepper, that can approximately learn the 3D capture regions of both simplified and full robot dynamic models using reinforcement l...
-
Soil strength testing and collecting soil cores from wetlands is currently a slow, manual process that runs the risk of disturbing and contaminating soil samples. This paper describes a method using an instrumented dart deployed and retrieved by a drone for performing core sample tests in soft soils. The instrumented dart can simultaneously conduct free- fall penetrometer tests. A drone-mounted me...
-
Research on tracking control has been on-going for many years. The accuracy and the practicality of the tracking control method has always been one of the most important aspects when designing the control strategy. Autonomous Underwater Vehicles are becoming increasingly important in the applications of ocean surveillance and military, etc. Therefore, this paper aims to develop a control method fo...
-
We introduce a methodology to compute the inverse kinematics for concentric tube continuum robots from a desired shape as input. We demonstrate that it is possible to accurately learn joint parameters using neural networks for a discrete point-wise shape representation with different discretization. In comparison to a vanilla numerical method, the learning-based method is preferred in terms of acc...
-
Due to the continuous and flexible nature of continuum robot backbones and the infinite number of parameters required to represent them in configuration space, modeling them accurately and in real-time is challenging. While the constant curvature assumption provides a simple alternative, it is limited in its capabilities as it cannot account for external tip forces. In cases where the backbone dev...
-
Even though artificial muscles have gained popularity due to their compliant, flexible and compact properties, there currently does not exist an easy way of making informed decisions on the appropriate actuation strategy when designing a muscle-powered robot; thus limiting the transition of such technologies into broader applications. What’s more, when a new muscle actuation technology is develope...
-
This paper proposes an EMG-dependant neural network-based model of human forearm during interaction with a haptic training simulator of sinus endoscopy. We used a conventional lumped mass-spring-damper model as a base model, beside which we took effects of muscle activation level, using surface electromyography (EMG) signals, into consideration. Unknown parameters of a five-parameter mass-spring-d...
-
Dynamics modeling in outdoor and unstructured environments is difficult because different elements in the environment interact with the robot in ways that can be hard to predict. Leveraging multiple sensors to perceive maximal information about the robot’s environment is thus crucial when building a model to perform predictions about the robot’s dynamics with the goal of doing motion planning. We ...
-
Aircraft routing problem is a crucial component for flight automation. Despite recent successes, challenges still remain when the environment is dynamic and uncertain. In this paper, we tackle the following two challenges. First, when the environment is uncertain, it is much safer if the route planner can guarantee a specified level of safety. Second, when the environment is dynamic, the planner n...
-
In human-in-the-loop navigation, the operator’s intention is to locally avoid obstacles while planning long-horizon paths in order to complete the navigation task. We propose a hierarchical teleoperation framework that captures these characteristics of intention, and generates trajectories that are locally safe and follow the operator’s global plan. The hierarchical teleoperation framework consist...
-
Reduction of combinatorial filters involves compressing state representations that robots use. Such optimization arises in automating the construction of minimalist robots. But exact combinatorial filter reduction is an NP-complete problem and all current techniques are either inexact or formalized with exponentially many constraints. This paper proposes a new formalization needing only a polynomi...
-
Morphable design and depth-based visual control are two upcoming trends leading to advancements in the field of quadrotor autonomy. Stereo-cameras have struck the perfect balance of weight and accuracy of depth estimation but suffer from the problem of depth range being limited and dictated by the baseline chosen at design time. In this paper, we present a framework for quadrotor navigation based ...
-
This paper proposes a drive-through recharging strategy for a quadrotor. First, a ground charging station is constructed with a portable charging wire, then with two conductive hooks connected to the battery, the quadrotor is flying through the station. Finally, after connecting the hooks to the charging wire, the quadrotor can be recharged without landing or stopping, similar to a drive-through. ...
-
We present a novel continuous time trajectory representation based on a Chebyshev polynomial basis, which when governed by known dynamics models, allows for full trajectory and robot dynamics estimation, particularly useful for high-performance robotics applications such as unmanned aerial vehicles. We show that we can gracefully incorporate model dynamics to our trajectory representation, within ...
-
Many robot applications call for autonomous exploration and mapping of unknown and unstructured environments. Information-based exploration techniques, such as Cauchy-Schwarz quadratic mutual information (CSQMI) and fast Shannon mutual information (FSMI), have successfully achieved active binary occupancy mapping with range measurements. However, as we envision robots performing complex tasks spec...
-
Attention control is a key cognitive ability for humans to select information relevant to the current task. This paper develops a computational model of attention and an algorithm for attention-based probabilistic planning in Markov decision processes. In attention-based planning, the robot decides to be in different attention modes. An attention mode corresponds to a subset of state variables mon...
-
Path planning for robotic coverage is the task of determining a collision-free robot trajectory that observes all points of interest in an environment. Robots employed for such tasks are often capable of exercising active control over onboard observational sensors during navigation. We address the problem of planning robot and sensor trajectories that maximize information gain in such tasks, where...
-
We describe a docking mechanism and strategy to allow modular self-assembly for the Modboat: an inexpensive, underactuated, oscillating, surface-swimming robot powered by a single motor. Because propulsion is achieved through oscillation, orientation can be controlled only in the average; this complicates docking, which requires precise position and orientation control. Given these challenges, we ...
-
Recent work has achieved dense 3D reconstruction with wide-aperture imaging sonar using a stereo pair of orthogonally oriented sonars. This allows each sonar to observe a spatial dimension that the other is missing, without requiring any prior assumptions about scene geometry. However, this is achieved only in a small region with overlapping fields-of-view, leaving large regions of sonar image obs...
-
Several animal species self-organize into large groups to leverage vital behaviors such as foraging, construction, or predator evasion. With the advancement of robotics and automation, engineered multi-agent systems have been inspired to achieve similarly high degrees of scalable, robust, and adaptable autonomy through decentralized and dynamic coordination. So far however, they have been most suc...
-
We propose a hierarchical representation of objects, where the representation of each object is allowed to change based on the quality of accumulated measurements. We initially estimate each object as a 2D bounding box or a 3D point, encoding only the geometric properties that can be well-constrained using limited viewpoints. With additional measurements, we allow each object to become a higher di...
-
Navigating fluently around pedestrians is a necessary capability for mobile robots deployed in human environments, such as buildings and homes. While research on social navigation has focused mainly on the scalability with the number of pedestrians in open spaces, typical indoor environments present the additional challenge of constrained spaces such as corridors and doorways that limit maneuverab...
-
Sensor coverage is the critical multi-robot problem of maximizing the detection of events in an environment through the deployment of multiple robots. Large multi-robot systems are often composed of simple robots that are typically not equipped with a complete set of sensors, so teams with comprehensive sensing abilities are required to properly cover an area. Robots also exhibit multiple forms of...
-
Ground Penetrating Radar (GPR) is one of the most important non-destructive evaluation (NDE) instruments to detect and locate underground objects (i.e. rebars, utility pipes). Many of the previous researches focus on GPR image-based feature detection only, and none can process sparse GPR measurements to successfully reconstruct a very fine and detailed 3D model of underground objects for better vi...
-
This paper presents replay overshooting (RO), an algorithm that uses properties of the extended Kalman filter (EKF) to learn nonlinear stochastic latent dynamics models suitable for long-horizon prediction. We build upon overshooting methods used to train other prediction models and recover a novel variational learning objective. Further, we use RO to extend another objective that acts as a surrog...
-
Several independent approaches exist for state estimation and control of multirotor unmanned aerial systems (UASs) that address specific and constrained operational conditions. This work presents a complete end-to-end pipeline that enables precise, aggressive and agile maneuvers for multirotor UASs under real and challenging outdoor environments. We leverage state-of-the-art optimal methods from t...
-
Robotic manipulators are increasingly being used to perform additive manufacturing. The accuracy of a built part is dependent on the trajectory execution error of the manipulator. For articulated manipulators, the trajectory execution error and achievable build accuracy vary considerably over the workspace. Therefore, the build accuracy depends on where the part is placed in the manipulator worksp...
-
Malaria is a worldwide scourge, and the broad deployment of an effective vaccine would improve the lives of millions of people. A vaccine based on Plasmodium falciparum (PfSPZ) sporozoites extracted from the salivary glands of infected mosquitoes shows significant promise. However, the large-scale industrial production of PfSPZ-based vaccines will benefit from automation of the key step of extract...
-
Industrial knitting machines create fabric by manipulating loops held on hundreds of needles. A core problem in pattern making for these machines is transfer planning – coming up with a sequence of low-level operations that move loops to the appropriate needles so that knitting through those loops produces the correct final structure. Since each loop is connected to the larger piece in progress, t...
-
Semantic segmentation of surgical instruments provides essential priors for autonomous surgery. This task is however challenging since the fine-structure of surgical instruments requires the accurate segmentation of detailed regions in images. As the visual guidance for autonomous surgery, the algorithm should also be real-time and friendly to embedded systems. In this paper, a discriminative asym...
-
Surgical instrument segmentation in robot-assisted surgery (RAS) - especially that using learning-based models - relies on the assumption that training and testing videos are sampled from the same domain. However, it is impractical and expensive to collect and annotate sufficient data from every new domain. To greatly increase the label efficiency, we explore a new problem, i.e., adaptive instrume...
-
Semantic segmentation has attracted increasing attention due to its important role in self-driving, and it is often realized by supervised learning with large number of well labeled maps. However, the labeled images are hard to be obtained in most circumstances, and the common way for unsupervised semantic segmentation is usually implemented by transferring the knowledge from source supervised dom...
-
Traditional convolution for capturing local structures and relationships remains a key technical limit in 3D semantic segmentation, which neglects the certain influence of the adjacent points on the central point in the disordered local point clouds. In this paper, we propose a novel joint-edge graph convolution neural network (JEGCN), which can extract the dynamic features of each local area and ...
-
Robust and accurate pose estimation in long-term localization is crucial to autonomous driving. In this paper, we dealt with absolute localization with a LiDAR feature map and multi-sensor measurements. We proposed a tightly-coupled fusion method with fixed-lag smoothing. A sliding window of recently maintained states is estimated by minimizing a joint cost function. This cost function includes re...
-
Modern LiDAR-SLAM (L-SLAM) systems have shown excellent results in large-scale, real-world scenarios. However, they commonly have a high latency due to the expensive data association and nonlinear optimization. This paper demonstrates that actively selecting a subset of features significantly improves both the accuracy and efficiency of an L-SLAM system. We formulate the feature selection as a com...
-
Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline...
-
We consider the problem of rearranging objects in a cluttered and confined space using a robotic manipulator. The goal is to retrieve a target object from the clutter where the target is occluded by other objects. In situations where overhand grasps are not allowed, the robot needs to remove some objects to make the target accessible. In the course of removing the objects, the robot also needs to ...
-
In this paper, we present a hybrid position/force controller for operating joint robots. The hybrid controller has two goals—motion tracking and force regulating. As long as these two goals are not mutually exclusive, they can be decoupled in some way. In this work, we make use of the smooth and invertible mapping from the joint space to the task space to decouple the two control goals and design ...
-
Exploration in environments with sparse rewards remains a challenging problem in Deep Reinforcement Learning (DRL). For the off-policy method, it usually needs a large number of training samples. With the growing dimensions of state and action space, this method becomes more and more sample-inefficient. In this paper, we propose a novel fast exploration method for off-policy reinforcement learning...
-
Lightly-damped dynamics of a flexure-based mechanism will tend to largely deteriorate the broadband control performance if its hysteresis nonlinearity has been compensated. This paper developed a novel damped piezo-driven decoupled XYZ nanopositioning stage, which consists of three orthogonal parallel kinematic subchains, in which each subchain has a translational pair using a bridge-type piezo-dr...
-
This paper presents a transformable RHex-inspired robot, RHex-T3, with high energy efficiency, excellent flexibility and versatility. By using the innovative 2-DoF transformable structure, RHex-T3 inherits most of RHex’s mobility, and can also switch to other 4 modes for handling various missions. The wheel-mode improves the efficiency of RHex-T3, and the leg-mode helps to generate a smooth locomo...
-
This paper presents a new variable stiffness actuator based on a second-order lever mechanism which has wide stiffness regulation range. By employing a novel symmetric structure design and improving the load capacity of the stiffness regulation module, the proposed actuator also shows well performance in load capacity, stiffness regulation response, and elastic hysteresis. On this basis, a variabl...
-
A novel motion planning method is proposed to generate human-like motion for anthropomorphic robot arms. Its highlight is to consider the robot arm to be human-like not only in its configuration but also in its motion patterns. To achieve this, the intrinsic mechanisms of human arm motion generation are transferred to robot motion planning. First, human arm motion is modeled using human arm motion...
-
For a multi-fingered robot hand, the individual control over single joints cannot guarantee their fine collaboration. For achieving a high-precision synchronization, a theory of synchronous control is introduced to multi-fingered robot hands. This paper introduced a new model-free and cross-coupling control strategy. It had been tested on the humanoid robot fingers and showed high positioning perf...
-
Point set registration (PSR) is an essential problem in communities of computer vision, medical robotics and biomedical engineering. This paper is motivated by considering the anisotropic characteristics of the error values in estimating both the positional and orientational vectors from the PSs to be registered. To do this, the multi-variate Gaussian and Kent distributions are utilized to model t...
-
Self-supervised learning algorithms that compute depth map from monocular videos have achieved remarkable performance on urban scenes and have been applied extensively. These techniques still face significant challenges, however, when applied directly to endoscopic videos because of the brightness variations from frame to frame and inadequate representation learning during the training phase. Insp...
-
Active wireless capsule endoscopy (WCE) under magnetic actuation is a promising technology to reduce the inspection time and relieve the burden of physicians. In this paper, we propose a reciprocally rotating magnetic actuation method for trajectory following of a capsule and develop its dynamic model. For the trajectory following task, we investigate the closed-loop tracking control strategies ba...
-
In this paper, we propose the reduced model for the full dynamics of a bicycle and analyze its nonlinear behavior under a proportional control law for steering. Based on the Gibbs-Appell equations for the Whipple bicycle, we obtain a second-order nonlinear ordinary differential equation (ODE) that governs the bicycle’s controlled motion. Two types of equilibrium points for the governing equation a...
-
This paper presents a balance control technique for a novel wheel-legged robot. We first derive a dynamic model of the robot and then apply a linear feedback controller based on output regulation and linear quadratic regulator (LQR) methods to maintain the standing of the robot on the ground without moving backward and forward mightily. To take into account nonlinearities of the model and obtain a...
-
General object grasping is an important yet unsolved problem in the field of robotics. Most of the current methods either generate grasp poses with few DoF that fail to cover most of the success grasps, or only take the unstable depth image or point cloud as input which may lead to poor results in some cases. In this paper, we propose RGBD-Grasp, a pipeline that solves this problem by decoupling 7...
-
This study proposes a hybrid vision/force control scheme for interaction with the inner surface of the bottle-like object. Based on the geometry of the object, a new generalized constraint called the bottleneck (BN) constraint is proposed, which ensures the tool passes through a fixed 3-D region and avoid collisions with the boundary of the region. To realize the hybrid vision/force control under ...
-
Reliable robotic grasping in unstructured environments is a crucial but challenging task. The main problem is to generate the optimal grasp of novel objects from partial noisy observations. This paper presents an end-to-end grasp detection network taking one single-view point cloud as input to tackle the problem. Our network includes three stages: Score Network (SN), Grasp Region Network (GRN), an...
-
Jumping with controllable elevation is significant for insect-scale robots to improve terrain adaptability and to escape from risks. However, jumping robots based on soft materials with low stiffness cannot transmit displacement precisely, exhibiting poor control of jumping. Here, we propose a modified two-bars catapult mechanism combined with an asynchronous sequential releasing strategy to reali...
-
Complex pipeline network should be inspected regularly for safety. In general, these tasks are often completed by large pipe-climbing robots or customized equipment. Most of them are not effective, and cannot work on pipes with uncertain barriers. Moreover, some pipes are mounted in constrained scenarios, so bulky robots are not applicable. This paper presents a tethered soft-rigid pipe-climbing r...
-
Mechanical intelligence is the use of mechanical and other physical properties to create robotic systems adaptable to new external situations using simple control schemes. Designs of robot hands have successfully been developed and optimised following this principle to produce self-adaptive and versatile power grasps via implementations based on underactuated fingers, elastic components, and open-...
-
Although reinforcement learning (RL) has achieved great success in robotic manipulation skills learning, it is still challenging for long-horizon tasks. Combining RL with demonstrations is an effective solution. In this paper, we propose a novel hierarchical learning from demonstrations method for long-horizon tasks, which leverages (i) object-centered segmentation of demonstrations to automatical...
-
We present Pylot, a platform for autonomous vehicle (AV) research and development, built with the goal to allow researchers to study the effects of the latency and accuracy of their models and algorithms on the end-to-end driving behavior of an AV. This is achieved through a modular structure enabled by our high-performance dataflow system that represents AV software pipeline components (object de...
-
In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from sim...
-
In this paper, the distributed rendezvous control problem of networked uncertain robotic systems with bearing measurements is investigated. The network topology of the multi-robot systems is described by an undirected graph. The dynamics of robots is modeled by Euler-Lagrange equation with unknown inertial parameters, which is more general than simple kinematics considered in existing works on ren...
-
SLAM systems can retrieve their metric scales and depth information using RGB-D cameras. However, limited by the sensing range and objects structure, RGB-D cameras can not always work well, resulting in failures sometimes. In this work, we present initialization and localization methods based on maximum-a-posteriori estimation. Our system endows monocular keypoints with valid depth values and intr...
-
Right invariant extended Kalman filter (RIEKF) based simultaneous localization and mapping (SLAM) proposed recently has shown to be able to produce more consistent SLAM estimates as compared with traditional EKF based SLAM methods, including some improved EKF SLAM methods such as observability constrained-EKF (OC-EKF) SLAM. Latest results have demonstrated that its performance is very close to opt...
-
Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning researchers who want to compare the performance of a n...
-
We propose a memory-constrained partition-based method to extract symbolic representations of the belief state and its dynamics in order to solve planning problems in a partially observable Markov decision process (POMDP). Our K-means partitioning strategy uses a fixed number of symbols to represent the partitions of the belief space and ensures the parameterization of the belief dynamics does not...
-
In this work, we consider the problem of planning paths for a team of autonomous unmanned aerial vehicles (UAVs) to assist search and rescue practitioners. To address the problem, we develop a fully integrated framework that includes information from all aspects of the search environment. We take into consideration lost person motion via a behavior-based predictive model, anticipated human searche...
-
In this work, we propose a novel method for performing inertial aided navigation, by using deep neural net-works (DNNs). To date, most DNN inertial navigation methods focus on the task of inertial odometry, by taking gyroscope and accelerometer readings as input and regressing for integrated IMU poses (i.e., position and orientation). While this design has been successfully applied on a number of ...
-
Line segment features become popular in SLAM community. Usually, line-based SLAM systems utilize local appearance descriptors for line segment tracking. However, traditional descriptor-based line segment tracking algorithms suffer from the problem that accuracy and speed cannot be possessed simultaneously, which affects the performance of line-based SLAM systems negatively. We propose a novel line...
-
Visual localization for planar moving robot is important to various indoor service robotic applications. To handle the textureless areas and frequent human activities in indoor environments, a novel robust visual localization algorithm which leverages dense correspondence and sparse depth for planar moving robot is proposed. The key component is a minimal solution which computes the absolute camer...
-
The localization system, which outputs vehicle position, velocity, and attitude, is one of the fundamental components in the autonomous driving vehicle. The global pose is not only used for the planning and control system, but also an important reference for the cloud source-based HD Map building and updating. The accuracy, availability, and reliability are key requirements for the localization sy...
-
Robotic hand exoskeletons can provide assistance to people who suffer from hand functional disability or spinal cord injury (SCI). However, the current hand exoskeletons remain challenging with respect to having a user-friendly design that satisfies human motion with a lightweight structure. Here we propose a method of using topology optimization in the design of finger exoskeletons, which is a li...
-
This paper presents the design and preliminary evaluation of a portable spring-based knee exoskeleton, the SpringExo, which is designed to provide assistance to the leg while minimizing interference with the natural leg movement. Traditional rigid exoskeletons are unable to accurately align with a user’s anatomical joints. In addition, the user’s natural motion pattern is likely to change due to t...
-
Recently, the motion averaging method has been introduced as an effective means to solve the multi-view registration problem. This method aims to recover global motions from a set of relative motions, where the original method is sensitive to outliers due to using the Frobenius norm error in the optimization. Accordingly, this paper proposes a novel robust motion averaging method based on the maxi...
-
The matching and fusing of local maps generated by multiple robots can greatly enhance the performance of relative localization and collaborative mapping. Currently, existing semantic matching methods are partly based on classical iterative closet point (ICP), which typically fail in cases with large initial error. What’s more, current semantic matching algorithms have high computation complexity ...
-
Scale ambiguity is a fundamental problem in monocular visual odometry. Typical solutions include loop closure detection and environment information mining. For applications like self-driving cars, loop closure is not always available, hence mining prior knowledge from the environment becomes a more promising approach. In this paper, with the assumption of a constant height of the camera above the ...
-
Monitoring the state of contact is essential for robotic devices, especially grippers that implement geckoinspired adhesives where intimate contact is crucial for a firm attachment. However, due to the lack of deformable sensors, few have demonstrated tactile sensing for gecko grippers. We present Viko, an adaptive gecko gripper that utilizes vision-based tactile sensors to monitor contact state. ...
-
Robots with a parallel-jaw gripper and suction cup is an adaptive and efficient robotic picking system. This paper proposed Policy-Oriented Instance Segmentation (POIS) for ambidextrous robots. POIS can generate a pair of target masks that allows ambidextrous robots to pick in parallel. It takes a depth image and predicts initial mask, center offset, and policy confidence map through three paralle...
-
In the Wave-driven unmanned surface vehicles (WUSVs), oscillating-foils are the most straightforward and widely used wave energy conversion mechanism. In this paper, a kind of novel asymmetric foil is proposed, which improves the wave energy-converting efficiency to provide a more significant thrust in every wave cycle. We break down the movement of the foils in the wave and build the correspondin...
-
The versatile nature of agile micro aerial vehicles (MAVs) poses fundamental challenges to the design of robust state estimation in various complex environments. Achieving high-quality performance in textureless scenes is one of the missing pieces in the puzzle. Previously proposed solutions either seek a remedy with visual loop closure or leverage RF localizability with inferior accuracy. None of...
-
Control of an aerial manipulator is challenging due to the decentralized dynamics of the aerial vehicle and the robotic arm. It is generally complex to adjust the controller of the aerial manipulator when replacing a different robotic arm. This paper presents a flexible control scheme for a quadrotor-based aerial manipulator equipped with a replaceable robotic arm. To analyze the dynamic character...
-
Real-time guidewire segmentation and endpoint localization play a pivotal role in robot-assisted minimally invasive surgery, which is helpful to reduce radiation dose and procedure time. Nevertheless, the tasks often come with the challenge of limited computational resources. For this purpose, a real-time multi-task framework with two stages is developed. In the first stage, a Fast Attention-fused...
-
Robots operating in open environments expect to have robust plans to achieve tasks successfully under environment uncertainties. However, both partial observability and dynamics of environment states have significantly decreased the robustness of task achievement, making robot task planning much more challenging. The partially observable states require the robot to obtain observations for optimall...
-
In this paper, we propose a novel deep reinforcement learning (DRL) system for the autonomous navigation of mobile robots that consists of three modules: map navigation, multi-view perception and multi-branch control. Our DRL system takes as the input a routed map provided by a global planner and three RGB images captured by a multi-camera setup to gather global and local information, respectively...
-
Visible images have been widely used for motion estimation. Thermal images, in contrast, are more challenging to be used in motion estimation since they typically have lower resolution, less texture, and more noise. In this paper, a novel dataset for evaluating the performance of multi-spectral motion estimation systems is presented. All the sequences are recorded from a handheld multi-spectral de...
-
Ego-motion estimation with 3D perception using visual odometry (VO) is known to be robust and economical among the existing odometry techniques. However, existing VO solutions are typically both computation intensive and memory intensive, which dramatically inhibits their deployment in IoT platforms such as robotic vehicles and handheld devices mostly equipped with resource-constrained MCU-level p...
-
In this paper, we introduce a method for visual relocalization using the geometric information from a 3D surfel map. A visual database is first built by global indices from the 3D surfel map rendering, which provides associations between image points and 3D surfels. Surfel reprojection constraints are utilized to optimize the keyframe poses and map points in the visual database. A hierarchical cam...
-
Due to the head swinging and the body winding, the self-localization and path following for snake-like robots based on vision are very challenging. In this paper, a novel pantilt compensation method and curve parameter compensation path following controller are proposed to solve these problems, which can achieve high-precision path following. More specifically, to realize real-time positioning of ...
-
In this paper, the configuration transformation of Wheel-Legged Robot (WLR) is studied, which can enable the robot to change its multilinks configuration on Inverted Equilibrium Manifold (IEM), while keeping balance with a small location drift on the floor. First of all, the general form of dynamics equation of planar Articulated Wheeled Inverted Pendulum (AWIP) with a wheel and n − 1 rigid links,...
-
Load-carrying capability is an essential criterion in legged robots' practical application. This paper proposes an unpowered hydraulic auxiliary system to improve the legged robot's loading capability and energy efficiency. For humans, it has been widely hypothesized that intra-abdominal pressure can reduce potential injurious compressive force imposed on spinal discs when a person lifts heavy obj...
-
This paper proposes a state estimator for legged robots operating in slippery environments. An Invariant Extended Kalman Filter (InEKF) is implemented to fuse inertial and velocity measurements from a tracking camera and leg kinematic constraints. The misalignment between the camera and the robot-frame is also modeled thus enabling auto-calibration of camera pose. The leg kinematics based velocity...
-
This paper considers a multi-target coverage problem where a robot team aims to efficiently cover multi-targets while maintaining connectivity in a distributed manner. A novel knowledge-incorporated policy framework is proposed to derive a distributed, efficient, and connectivity guaranteed coverage policy. In particular, a knowledge-guided policy network (KGPnet) is designed, which consists of ob...
-
Collaborative exploration in an unknown environment without external positioning under limited communication is an essential task for multi-robot applications. For inter-robot positioning, various Distributed Simultaneous Localization and Mapping (DSLAM) systems share the Place Recognition (PR) descriptors and sensor data to estimate the relative pose between robots and merge robots’ maps. As maps...
-
Conventional multi-agent path planners typically compute an ensemble of paths while optimizing a single objective, such as path length. However, many applications may require multiple objectives, say fuel consumption and completion time, to be simultaneously optimized during planning and these criteria may not be readily compared and sometimes lie in competition with each other. Naively applying e...
-
Brain needle intervention is a specific diagnosis and therapy procedure in brain disorders, such as brain tumors and Parkinson’s disease. Preoperative needle path planning is a vital step to guarantee the patient’s safety and reduce lesions. For positioning accuracy in the CT/MRI environment, we have developed a novel needle intervention robot in our previous work. Because the robot is currently d...
-
Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restr...
-
The sampling-based partial motion planning algorithm has come into widespread application in dynamic mobile robot navigation due to its low calculation costs and excellent performance in avoiding obstacles. However, when confronted with complicated scenarios, the motion planning algorithms are easily caught in traps. In order to solve this problem, this paper proposes a knowledge-based fast motion...
-
Small-scale robots have great potential in minimally invasive surgery (MIS). In this paper, we propose an untethered magnetic gripper with small scale and build a double-magnet model for it. The gripper is 4.3mm long and its maximum width is 4mm. It contains a spindle and two magnets, which can achieve precise control of orientation, position and open angle with external magnetic driven field. As ...
-
Magnetic manipulation provides a versatile, remote, noninvasive, and cost-effective strategy in a variety of applications. Till now, many different configurations of magnetic manipulation systems have been developed to address different needs on force, torque, accuracy, and accessibilities. Magnetic field mapping can help to explore the exact map of the magnetic field in the working space and guar...
-
Accurate position feedback in a wide range is critical for medical microrobotics and robot-assisted examinations, such as colonoscopy, bronchoscopy and capsule endoscopy examination. Among the many modalities of positioning feedback, magnetic tracking is a preferable method due to the unique advantages of free line of sight, free energy storage and untethered connection. However, the field strengt...
-
The reactor power and the coolant level in the nuclear plant are monitored via the reactor core detectors. Every 4 to 5 years, the detectors with high-level radiation need to be removed, which is time-consuming and hazardous for workers. To address this issue, this paper introduces a novel robotic system and its strategy for the removal of the detectors. The modular mechanisms are designed to achi...
-
Vision-based localization and mapping in the agricultural environment is challenging due to the unstructured scene with unstable features, illumination variations, bumpy roads, and dynamic environmental objects. To address these challenges, we propose an accurate and robust stereo direct visual odometry system with modifications on Stereo-DSO. We firstly select some well-matched static stereo poin...
-
Most of the existing road detection methods are either single-modal based, e.g., based on LiDAR or camera, or multi-modal based with LiDAR-camera fusion. The algorithms are designed for a specific data type, and cannot cope with input data changes. In addition, the LiDAR-camera based methods can only work in day time with enough light. In this paper, we develop a novel LiDAR-camera fusion strategy...
-
Soft robots are compliant to wrap large objects and adaptive to unstructured environments, while rigid robots can bend discretely at joints and pinch small objects easily. In this paper, we aim to create a novel robot that can behave like a rigid or soft robot adaptively, without additional stiffness-tunable mechanism. This approach makes the robot lightweight and practical. First, we present the ...
-
Continuum manipulators have shown a wide range of applications due to their inherent compliance and dexterity. At present, the cross-sectional dimension of these manipulators is often kept constant, which facilitates the design, fabrication and modeling processes. The famous piecewise-constant-curvature (PCC) assumption is widely used in the kinematics and motion control. By contrast, most natural...
-
Simulating the swimming of soft underwater robot remains challenging due to the absence of an efficient numerical framework that can effectively capture the geometrically nonlinear deformation of soft materials and structures when interacting with a liquid environment. Here, we address this by introducing a discrete differential geometry-based model that incorporates an implicit treatment of the e...
-
Differentiable simulators provide an avenue for closing the sim-to-real gap by enabling the use of efficient, gradient-based optimization algorithms to find the simulation parameters that best fit the observed sensor readings. Nonetheless, these analytical models can only predict the dynamical behavior of systems for which they have been designed. In this work, we study the augmentation of a novel...
-
Although deep reinforcement learning (RL) has been successfully applied to a variety of robotic control tasks, it’s still challenging to apply it to real-world tasks, due to the poor sample efficiency. Attempting to overcome this shortcoming, several works focus on reusing the collected trajectory data during the training by decomposing them into a set of policy-irrelevant discrete transitions. Ho...
-
Modern robot systems use various software programs to autonomously perform different kinds of tasks. However, due to the risks of possible faults and errors, a robotic software program can inevitably crash in some cases, causing that the robot system fails to perform the current task. Thus, for robustness, the crashed program should be correctly recovered to continue the failed task. For this purp...
-
We introduce a new solution to point set registration, a fundamental geometric problem occurring in many computer vision and robotics applications. We consider the specific case in which the point sets are segmented into semantically annotated parts. Such information may for example come from object detection or instance-level semantic segmentation in a registered RGB image. Existing methods incor...
-
Loop closure detection (LCD) is an essential module for simultaneous localization and mapping (SLAM), which can correct accumulated errors after long-term explorations. The widely used bag-of-words (BoW) model can not satisfy well the requirements of both low time consumption and high accuracy for a mobile platform. In this paper, we propose a novel LCD algorithm based on motion knowledge. We give...
-
Accurate localization is of crucial importance for autonomous driving tasks. Nowadays, we have seen a lot of sensor-rich vehicles (e.g. Robo-taxi) driving on the street autonomously, which rely on high-accurate sensors (e.g. Lidar and RTK GPS) and high-resolution map. However, low-cost production cars cannot afford such high expenses on sensors and maps. How to reduce costs? How do sensor-rich veh...
-
Highly accurate and robust localization ability is of great importance for autonomous vehicles (AVs) in urban scenarios. Traditional vision-based methods suffer from lost due to illumination, weather, viewing and appearance changes. In this paper we propose a novel visual semantic localization algorithm based on HD map and semantic features which are compact in representation. Semantic features ar...
-
Vision-based localization and mapping solution is promising to be adopted in the automated valet parking task. In this paper, a semantic SLAM framework that leverages the hybrid edge information on bird’s-eye view images is presented. To extract useful edges from the synthesized bird’s-eye view image and the free-space contours for the SLAM task, different segmentation methods are designed to remo...
-
The efficiency and accuracy of mapping are crucial in a large scene and long-term AR applications. Multi-agent cooperative SLAM is the precondition of multi-user AR interaction. The cooperation of multiple smart phones has the potential to improve efficiency and robustness of task completion and can complete tasks that a single agent cannot do. However, it depends on robust communication, efficien...
-
In multi-agent applications such as surveillance and logistics, fleets of mobile agents are often expected to coordinate and safely visit a large number of goal locations as efficiently as possible. The multi-agent planning problem in these applications involves allocating and sequencing goals for each agent while simultaneously producing conflict-free paths for the agents. In this article, we int...
-
This paper presents a comprehensive inertial aided 3D LiDAR SLAM system with hybrid geometric primitives in large-scale environments, including a tightly-coupled LiDAR-Inertial-Odometry (LIO), a global mapping module supported by learning-based loop closure detection and a sub-maps matching algorithm. An efficient method is developed to simultaneously extract explicit plane features and point feat...
-
Safety is an important topic in autonomous driving since any collision may cause serious injury to people and damage to property. Hamilton-Jacobi (HJ) Reachability is a formal method that verifies safety in multi-agent interaction and provides a safety controller for collision avoidance. However, due to the worst-case assumption on the car’s future behaviours, reachability might result in too much...
-
Autonomous intersection management (AIM) refers to planning the trajectories for multiple connected and automated vehicles (CAVs) when they traverse an unsignalized intersection cooperatively. As an extension of the conventional AIM, lane-free AIM allows the CAVs to adjust their velocities and paths flexibly within the intersection. Nominally, one needs to formulate a centralized optimal control p...
-
Road Network Persistent Surveillance Problem (RPSP) involves path planning for an unmanned ground vehicle (UGV) with detection ability to timely detect the events randomly occurred. The road network is formed by edges and weighted viewpoints, where the UGV must move along the edges. The existing method based on cognitive architecture is inadequate in terms of real-time and effective decision makin...
-
Depth completion aims to recover the dense depth map from sparse depth data and RGB image respectively. However, due to the huge difference between the multi-modal signal input, vanilla convolutional neural network and simple fusion strategy cannot extract features from sparse data and aggregate multi-modal information effectively. To tackle this problem, we design a novel network architecture tha...
-
Object grasping in cluttered scenes is a widely investigated field of robot manipulation. Most of the current works focus on estimating grasp pose from point clouds based on an efficient single-shot grasp detection network. However, due to the lack of geometry awareness of the local grasping area, it may cause severe collisions and unstable grasp configurations. In this paper, we propose a two-sta...
-
Image contour based vision measurement is widely applied in robot manipulation and industrial automation. It is appealing to realize object-agnostic vision system, which can be conveniently reused for various types of objects. We propose the contour primitive of interest extraction network (CPieNet) based on the one-shot learning framework. First, CPieNet is featured by that its contour primitive ...
-
This paper presents a modular and reconfigurable mini-robotic system with 5 degrees of freedom (DoFs) towards minimally invasive surgery (MIS). The mini-robotic system consists of two modules, a 2-DoFs rotational end-effector, and a 3-DoFs positioning platform. The 2-DoFs rotational end-effector is based on a spring-spherical joint mechanism, whose rotation is controlled by Bowden-cable. The 3-DoF...
-
This paper describes the control of a novel Magnetic Actuated Flexible-joint Robotic Surgical (MAFRS) camera system with four degrees of freedom (4-DOF) for single incision laparoscopic surgery. Based on the idea of motion decoupling, we designed a novel MAFRS system which is consists of an external driving device and a motor-free insertable wireless robotic device with a hollow flexible joint. Du...
-
The development and control of biological actuators have been an active research field. Biological actuators revealed high mobility with compact dimensions, which is critical for the design of microrobots. The powerful kicking motion of the locust is important for its quick jumping. Herein, we examined the kicking process of the locust’s hindleg and controlled the flexion and extension motions via...
-
Self-assembly has attracted growing interests in modular robotics during past decades. Recent work accelerates the assembly process by parallelizing the docking actions among robots. However, these methods can only apply to ideal environments without obstacles. Otherwise, robots will get trapped during the assembly process, due to the complex scenes with obstacles. This paper presents an efficient...
-
We consider typical scenarios where an autonomous multi-robot team is used for surveying a large region. The desired output is a spatial map of the physical values of interest. Accounting for spatial correlation and uncertainty, the map is modeled using a Gaussian Process. Considering real-world constraints such as limited time budget and collision avoidance, we model team’s mission as a joint inf...
-
In unstructured and unknown environments, heterogeneous robots must be able to perceive the environment, coordinate with each other and complete tasks collaboratively with onboard sensors. In this paper, a tightly-coupled perception and navigation framework is proposed for heterogeneous land-air robots, which forms a closed loop of perception-navigation for heterogeneous robots. The key novelty of...
-
Contact-rich manipulation tasks are commonly found in modern manufacturing settings. However, manually designing a robot controller is considered hard for traditional control methods as the controller requires an effective combination of modalities and vastly different characteristics. In this paper, we first consider incorporating operational space visual and haptic information into a reinforceme...
-
Most industrial parts are parametric and their special properties are not fully explored yet. This paper proposes a new 6DoF pose estimation network for parametric shapes in stacked scenarios (ParametricNet). It treats a parametric shape, instead of a part object, as a category. The keypoints of individual instances are learned with point- wise regression and Hough voting scheme, from which specif...
-
Shared autonomous mobility-on-demand systems hold great promise for improving the efficiency of urban transportation, but are challenging to implement due to the huge scheduling search space and highly dynamic nature of requests. This paper presents a novel optimal schedule pool (OSP) assignment approach to optimally dispatch high-capacity ride-sharing vehicles in real time, including: (1) an incr...
-
The barcode navigation based on QR (quick response) codes is widely employed in industrial logistics due to its accurate localization and flexible movement paths. However, the regular repair of damaged barcodes and robot speed control when approaching the barcodes are required. In this study, we presented an improved magnetic spot navigation approach to replace the barcode navigation for automated...
-
Prior correlation filter (CF)-based tracking methods for unmanned aerial vehicles (UAVs) have virtually focused on tracking in the daytime. However, when the night falls, the trackers will encounter more harsh scenes, which can easily lead to tracking failure. In this regard, this work proposes a novel tracker with anti-dark function (ADTrack). The proposed method integrates an efficient and effec...
-
Unmanned aerial vehicle (UAV) based visual tracking has been confronted with numerous challenges, e.g., object motion and occlusion. These challenges generally introduce unexpected mutations of target appearance and result in tracking failure. However, prevalent discriminative correlation filter (DCF) based trackers are insensitive to target mutations due to a predefined label, which concentrates ...
-
In the domain of visual tracking, most deep learning-based trackers highlight the accuracy but casting aside efficiency. Therefore, their real-world deployment on mobile platforms like the unmanned aerial vehicle (UAV) is impeded. In this work, a novel two-stage Siamese network-based method is proposed for aerial tracking, i.e., stage-1 for high-quality anchor proposal generation, stage-2 for refi...
-
Positive affect has been linked to increased interest, curiosity and satisfaction in human learning. In reinforcement learning, extrinsic rewards are often sparse and difficult to define, intrinsically motivated learning can help address these challenges. We argue that positive affect is an important intrinsic reward that effectively helps drive exploration that is useful in gathering experiences....
-
Although multi-person human pose estimation has made great progress in recent years, the challenges such as various scales of persons, occluded keypoints, and crowded backgrounds in complex scenes are still remained to be solved. In this paper, we propose a novel multi-level pose estimation network (MLPE) to learn multi-level features that can preserve both the strong semantic clues and spatial re...
-
Intention prediction is a crucial task for Autonomous Driving (AD). Due to the variety of size and layout of intersections, it is challenging to predict intention of human driver at different intersections, especially unseen and irregular intersections. In this paper, we formulate the prediction of intention at intersections as an open-set prediction problem that requires context specific matching...
-
Camera motion estimation, such as relative pose estimation and absolute pose estimation, are fundamental problems in computer vision and robotics. To obtain the motion parameters, classical methods rely on studying the properties of the geometric matrices, e.g., rotation matrix, essential matrix, homography matrix. The well known five-point algorithm was successfully derived using the singular con...
-
Medical diagnostic robot systems have been paid more and more attention due to its objectivity and accuracy. The diagnosis of mild cognitive impairment (MCI) is considered an effective means to prevent Alzheimer's disease (AD). Doctors diagnose MCI based on various clinical examinations, which are expensive and the diagnosis results rely on the knowledge of doctors. Therefore, it is necessary to d...
-
Reliable real-time extrinsic parameters of 3D Light Detection and Ranging (LiDAR) and camera are a key component of multi-modal perception systems. However, extrinsic transformation may drift gradually during operation, which can result in decreased accuracy of perception system. To solve this problem, we propose a line-based method that enables automatic online extrinsic calibration of LiDAR and ...
-
We transform classical robot inertial parameter identification into an online learning problem by integrating state-of-the-art gradient descent techniques and first-order principles from mechanics and differential geometry. Through this, incremental learning of fully physically feasible inertial properties without requiring any prior information is made possible. This is achieved using a version o...
-
Learning manipulation skills from observing human demonstration videos is a promising aspect for intelligent robotic systems. Recent advances in video to command provide an end-to-end approach to translate a video into robot plans. However, the general video captioning methods focus more on the understanding of the full frame, while they lack the consideration of the spatio-temporal features in vi...
-
This paper presents a novel method for learning and tracking of the desired path of the human partner in physical human-robot interaction. Combining the Adam optimization algorithm with iteration learning control (ILC), a path learning method is designed to generate and update reference waypoints according to the human partner’s desired path. This method firstly uses the Adam optimization algorith...
-
Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D human pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatiotemporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatiotemporal sequences and effectively achieves real-t...
-
Many previous works have facilitated muscle cell (C2C12) alignment to form fiber-like cell structures. However, there still remains a challenge how to induce C2C12 myoblasts in the cell structures to differentiate into matured myocytes to form a functional muscle tissue, while external mechanical stimulation has been proved to have good effects on proliferation and differentiation of myoblasts. In...
-
This paper describes a versatile platform for swarm robotics research. It integrates multiple pheromone communication with a dynamic visual scene along with real time data transmission and localization of multiple-robots. The platform has been built for inquiries into social insect behavior and bio-robotics. By introducing a new research scheme to coordinate olfactory and visual cues, it not only ...
-
To date, untethered microrobots have been receiving tremendous attention for playing implacable roles of maneuverable tools in fields such as microfabrication and biomanipulation. Typical actuation of such untethered tiny robots is the magnetic field-based approaches, including gradient and rotational methods. Compared to the gradient type method, the rotational approach requires much less magneti...
-
Robot path planning is difficult to solve due to the contradiction between the optimality of results and the complexity of algorithms, even in 2D environments. To find an optimal path, the algorithm needs to search all the state space, which costs many computation resources. To address this issue, we present a novel recurrent generative model (RGM), which generates efficient heuristic to reduce th...
-
This paper presents a novel multi-robot coverage path planning (CPP) algorithm - aka SCoPP - that provides a time-efficient solution, with workload balanced plans for each robot in a multi-robot system, based on their initial states. This algorithm accounts for discontinuities (e.g., no-fly zones) in a specified area of interest, and provides an optimized ordered list of way-points per robot using...
-
Planning the simultaneous movement of multiple agents represents a challenging coordination problem, and ideally safety and efficiency are jointly addressed. This paper introduces a planning algorithm for fast and energy-efficient trajectories with reduced collision potential from a start to an end constellation. This new approach combines trajectory approximation based on model predictive control...
-
It is challenging to develop an online path planning algorithm for Ackermann-steering vehicles to find collision-free and kinematically-feasible paths, that is efficient for dense environments, adaptable to various environments, and suitable for environments with narrow passages. In this paper, we propose a kinematically constrained RRT-based path planning algorithm integrating with a trajectory p...
-
ICSI (Intracytoplasmic sperm injection) is one of the most effective treatments for severe male infertility. During the implementation of the ICSI, it is necessary to perform the three-dimensional positioning of the tip of the glass injection micropipette. At present, the process is mainly controlled by skilled operators. Such manual operation is time-consuming and likely to cause micropipette dam...
-
Zebrafish (Danio Rerio) larvae have long been an important model organism for biomedicine and drug discovery. It is difficult to deliver the external materials into the circulatory system by conventional exposing administration, while vein microinjection is more efficient but more challenging. In this paper, a robotic cardinal vein microinjection system was presented for zebrafish larvae. The key ...
-
The surface electromyography (sEMG) signal-based human-machine interface (HMI) has been widely used for various scenarios of physical human-robot interaction. However, current HMIs based on bipolar myoelectric sensors are hindered by the limitations of global sEMG features, which are prone to variability and delay. In this letter, we define a HMI that takes advantage of the underlying neural infor...
-
Koopman operator theory has served as the basis to extract dynamics for nonlinear system modeling and control across settings, including non-holonomic mobile robot control. There is a growing interest in research to derive robustness (and/or safety) guarantees for systems the dynamics of which are extracted via the Koopman operator. In this paper, we propose a way to quantify the prediction error ...
-
Obstacle avoidance is crucial for autonomous surface vehicles (ASVs) in the sea because rescue is extremely difficult there. OceanVoy, a sailboat toward long range energy-saving voyage, has to overcome the dual challenges, i.e. from the environmental interference and its low mobility preventing from precise obstacle avoidance. We propose a control scheme based on real-time collision risk assessmen...
-
For large-scale tasks, coverage path planning (CPP) can benefit greatly from multiple robots. In this paper, we present an efficient algorithm MSTC∗ for multi-robot coverage path planning (mCPP) based on spiral spanning tree coverage (Spiral-STC). Our algorithm incorporates strict physical constraints like terrain traversability and material load capacity. We compare our algorithm against the stat...
-
Impact mitigation is crucial to the stable locomotion of legged robots, especially in high-speed dynamic locomotion. This paper presents a leg locomotion system, including the nonlinear active compliance control and the active impedance control for the steel wire transmission-based legged robot. The developed control system enables high-speed dynamic locomotion with excellent impact mitigation and...
-
Recent works on semantic Simultaneous Localization and Mapping (SLAM) utilizing object landmarks have shown superiority in terms of robustness and accuracy in tracking and localization. 3D object landmarks represented by a cubic or quadric surface are inferred from 2D object bounding boxes which are typically captured from multiple views by an object detector. Nevertheless, bounding box noises and...
-
Object detection in 3D with stereo cameras is an important problem in computer vision, and is particularly crucial in low-cost autonomous mobile robots without LiDARs. Nowadays, most of the best-performing frameworks for stereo 3D object detection are based on dense depth reconstruction from disparity estimation, making them extremely computationally expensive. To enable real-world deployments of ...
-
Sliding Mode Control of the Semi-active Hover Backpack Based on the Bioinspired Skyhook Damper Model
It is inevitable for human to bear the gravitational and inertial force when carrying loads. The impact force exerted on human body is originated from the inertial force which can increase the energy expenditure and cause injury to human body. This paper proposes a semi-active hover backpack with controllable air damper to minimize the inertial force. The skyhook damper model of hover backpack is ...
-
This paper aims to present a drone swarm light show design platform to support STEAM (science, technology, engineering, art and mathematics) education for K-12 children. With this platform, children can use this platform to design a drone swarm light show easily. To this end, the architecture of this platform contents three layers: UI layer, command layer, and physical layer. The UI layer has an e...
-
Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehicle dynamics or try to solve expensive trajectory-optimization problems online. When the vehicle approaches its physical limits, existing model-based controllers struggle to handle highly nonlinear dynamics, and cannot le...
-
MRI-guided robotic systems are emerging platforms for minimally invasive intervention because of high positioning accuracy and excellent tissue contrast. MR safe encoders are critical components for closed-loop robotic control. This paper develops an MR safe absolute rotary encoder based on eccentric sheave and FBG sensors. The eccentric sheave transforms the rotational motion of the shaft to the ...
-
We present an efficient, elastic 3D LiDAR reconstruction framework which can reconstruct up to maximum Li-DAR ranges (60 m) at multiple frames per second, thus enabling robot exploration in large-scale environments. Our approach only requires a CPU. We focus on three main challenges of large-scale reconstruction: integration of long-range LiDAR scans at high frequency, the capacity to deform the r...
-
Feature-based lidar odometry methods have attracted increasing attention due to their low computational cost. However, theoretically analysis of the effect of extracted features on pose estimation is still lacked. In this paper, we propose a method of key-feature selection for lightweight lidar inertial odometry, KFS-LIO, to further enhance the real-time performance by selecting the most effective...
-
Combining lidar in camera-based simultaneous localization and mapping (SLAM) is an effective method in improving overall accuracy, especially at outdoor large scale scenes. Recent development of low-cost lidars (e.g. Livox lidar) enable us to explore such SLAM systems with lower budget and higher performance. In this paper we propose CamVox by adapting Livox lidars into visual SLAM (ORB-SLAM2) by ...
-
Lidar odometry (LO) is a key technology in numerous reliable and accurate localization and mapping systems of autonomous driving. The state-of-the-art LO methods generally leverage geometric information to perform point cloud registration. Furthermore, obtaining the point cloud semantic information describing the environment more abundantly will facilitate the registration. We present a novel sema...
-
In this paper, a novel navigation learning method in continuous action space among crowds based on relational graph is proposed which can be directly deployed on differential-drive mobile robots without any change. More specifically, in order to increase generalization ability in crowd sizes, Graph Convolutional Network (GCN) is at first adopted to extract the relationships between robot and pedes...
-
When autonomous robots interact with humans, such as during autonomous driving, explicit safety guarantees are crucial in order to avoid potentially life-threatening accidents. Many data-driven methods have explored learning probabilistic bounds over human agents’ trajectories (i.e. confidence tubes that contain trajectories with probability δ), which can then be used to guarantee safety with prob...
-
Human motion prediction is an important and challenging topic that has promising prospects in efficient and safe human-robot-interaction systems. Currently, the majority of the human motion prediction algorithms are based on deterministic models, which may lead to risky decisions for robots. To solve this problem, we propose a probabilistic model for human motion prediction in this paper. The key ...
-
Human motion prediction is essential in human-robot interaction. Current research mostly considers the joint dependencies but ignores the bone dependencies and their relationship in the human skeleton, thus limiting the prediction accuracy. To address this issue, we represent the human skeleton as a directed acyclic graph with joints as vertexes and bones as directed edges. Then, we propose a nove...
-
In order to manipulate a deformable object, such as rope or cloth, in unstructured environments, robots need a way to estimate its current shape. However, tracking the shape of a deformable object can be challenging because of the object’s high flexibility, (self-)occlusion, and interaction with obstacles. Building a high-fidelity physics simulation to aid in tracking is difficult for novel enviro...
-
In this paper, we develop an online learning-based visual tracking framework that can optimize the target model and estimate the scale variation for object tracking. We propose a recommender-based tracker, which is capable of selecting the representative convolutional neural network (CNN) layers and feature maps autonomously. In addition, the proposed recommender computes the weights of these laye...
-
Visual tracking is a fundamental capability for robots tasked with humans and environment interaction. However, state-of-the-art visual tracking methods are still prone to failures and are imprecise when applied to challenging stereos, and their results are generally confidence agonistic. These methods depend on an embedded deep learning model to provide deterministic features or regression maps. ...
-
Due to the limited field of view (FOV), the probe-based confocal laser endomicroscopy (pCLE) imaging system remains challenging to be widely used in clinic. Existing video mosaicking approaches are usually troubled by poor real-time capability and sensitivity to tissue deformations and intensity fluctuations. In this paper, a novel pCLE mosaicking algorithm that simultaneously implements rigid pro...
-
To optimize the cutting depth in robotic-assisted laminectomy, we present a real-time method to adjust the preoperatively planned feed rate in the depth direction of the robot cutting trajectory. Not only the linearity between the harmonic amplitude of the milling acoustic signal and the cutting depth is discussed by analyzing the milling dynamic model, but its influencing variables are analyzed. ...
-
Low-light image enhancement is a complex and vital task including, recovering color and texture details from low-light images. For automated driving, low-light scenarios will have severe implications for vision-based applications. To address this problem, we propose a real-time unsupervised generative adversarial network (GAN) with multiple discriminators. It includes a multi-scale discriminator, ...
-
Scene flow represents the motion of points in the 3D space, which is the counterpart of the optical flow that represents the motion of pixels in the 2D image. However, it is difficult to obtain the ground truth of scene flow in the real scenes, and recent studies are based on synthetic data for training. Therefore, how to train a scene flow network with unsupervised methods based on real-world dat...
-
Deep3DRanker: A Novel Framework for Learning to Rank 3D Models with Self-Attention in Robotic Vision
Research on generating or processing point clouds has become an increasingly popular domain in robotic research due to its extensive applications, such as robotic grasping, augmented reality and autonomous vehicle navigation. In this paper, we explore a new research area on point clouds - Learning to rank 3D models captured from a single depth image. In the Learning To Rank (LTR) task, we aim at o...
-
In this paper, we investigate the problem of weakly supervised 3D vehicle detection. Conventional methods for 3D object detection usually require vast amounts of manually labelled 3D data as supervision signals. However, annotating large datasets needs huge human efforts, especially for 3D area. To tackle this problem, we propose a frustum-aware geometric reasoning (FGR) method to detect vehicles ...
-
Soft robots have been applied widely to various constrained scenarios due to the advantages over traditional rigid manipulators such as softness, deformability and adaptability to constrained surroundings. To make full use of this merit, this paper proposes a method that integrates collision detection, localization and force estimation for a cable-driven soft manipulator without any prior geometri...
-
Flexible surgical instruments can flexibly adjust their posture with a high degree of freedom, which makes them highly suitable for performing surgical tasks in narrow workspaces. However, redundant degrees of freedom increase their kinematic difficulty, which may cause redundant solutions, complex calculations, and low speeds. In this paper, a flexible surgical instrument is presented. The struct...
-
Ego-motion estimation is a key requirement for the simultaneous localization and mapping (SLAM) problem. The traditional pipeline goes through feature extraction, feature matching and pose estimation, whose performance depends on the manually designed features. In this paper, we are motivated by the strong performance of deep learning methods in other computer vision and robotics tasks. We replace...
-
In this work, we present a lightweight, tightly-coupled deep depth network and visual-inertial odometry (VIO) system, which can provide accurate state estimates and dense depth maps of the immediate surroundings. Leveraging the proposed lightweight Conditional Variational Autoencoder (CVAE) for depth inference and encoding, we provide the network with previously marginalized sparse features from V...
-
Mapping and localization in non-static environments are fundamental problems in robotics. Most of previous methods mainly focus on static and highly dynamic objects in the environment, which may suffer from localization failure in semi-dynamic scenarios without considering objects with lower dynamics, such as parked cars and stopped pedestrians. In this paper, we introduce semantic mapping and lif...
-
In this work, we propose a novel deep online correction (DOC) framework for monocular visual odometry. The whole pipeline has two stages: First, depth maps and initial poses are obtained from convolutional neural networks (CNNs) trained in self-supervised manners. Second, the poses predicted by CNNs are further improved by minimizing photometric errors via gradient updates of poses during inferenc...
-
Robust and accurate localization plays a key role in autonomous driving and robot applications. To utilize the complementary properties of different sensors, we present a novel tightly-coupled approach to combine the local (stereo cameras, IMU) and global sensors (magnetometer, GNSS). We jointly optimize all the model parameters through one active window. The visual part integrates constraints fro...
-
Vehicle Re-identification (Re-ID) aims to retrieve all instances of query vehicle images present in an image pool. However viewpoint, illumination, and occlusion variations along with subtle differences between two unique images pose a significant challenge towards achieving an effective system. In this paper, we emphasize upon enhancing the performance of visual feature based ReID system by impro...
-
Voxel-based methods have been widely used in point cloud 3D object detection. These methods usually transform points into voxels while suffering from information loss during point cloud voxelization. To address this problem, we propose a novel one-stage Voxelization Information Compensation Network (VIC-Net), which has the ability of loss-free feature extraction. The whole framework consists of a ...
-
Large-scale visual place recognition (VPR) is inherently challenging because not all visual cues in the image are beneficial to the task. In order to highlight the task-relevant visual cues in the feature embedding, the existing attention mechanisms are either based on artificial rules or trained in a thorough data-driven manner. To fill the gap between the two types, we propose a novel Semantic R...
-
Active vision is a desirable perceptual feature for robots. Existing approaches usually make strong assumptions about the task and environment, thus are less robust and efficient. This study proposes an adaptive view planning approach to boost the efficiency and robustness of active object detection. We formulate the multi-object detection task as an active multiview object detection problem given...
-
Estimating a depth map from a single RGB image has been investigated widely for localization, mapping, and 3- dimensional object detection. Recent studies on a single-view depth estimation are mostly based on deep Convolutional neural Networks (ConvNets) which require a large amount of training data paired with densely annotated labels. Depth annotation tasks are both expensive and inefficient, so...
-
Depth completion is an important task in computer vision and robotics applications, which aims at predicting accurate dense depth from a single RGB-LiDAR image. Convolutional neural networks (CNNs) have been widely used for depth completion to learn a mapping function from sparse to dense depth. However, recent methods do not exploit any 3D geometric cues during the inference stage and mainly rely...
-
Image guided depth completion is the task of generating a dense depth map from a sparse depth map and a high quality image. In this task, how to fuse the color and depth modalities plays an important role in achieving good performance. This paper proposes a two-branch backbone that consists of a color-dominant branch and a depth-dominant branch to exploit and fuse two modalities thoroughly. More s...
-
Multi-object tracking is an important ability for an autonomous vehicle to safely navigate a traffic scene. Current state-of-the-art follows the tracking-by-detection paradigm where existing tracks are associated with detected objects through some distance metric. Key challenges to increase tracking accuracy lie in data association and track life cycle management. We propose a probabilistic, multi...
-
Pedestrian trajectory prediction is a critical yet challenging task especially for crowded scenes. We suggest that introducing an attention mechanism to infer the importance of different neighbors is critical for accurate trajectory prediction in scenes with varying crowd size. In this work, we propose a novel method, AVGCN, for trajectory prediction utilizing graph convolutional networks (GCN) ba...
-
Autonomous vehicle navigation in shared pedestrian environments requires the ability to predict future crowd motion both accurately and with minimal delay. Understanding the uncertainty of the prediction is also crucial. Most existing approaches however can only estimate uncertainty through repeated sampling of generative models. Additionally, most current predictive models are trained on datasets...
-
Some impressing multi-kernel or multi-task correlation filter trackers only focus on boosting the discrimination of multi-channel features, or exploiting the interdependence among different tasks. However, the cooperation and complementary of both technologies are missed, and the spatial structure among or inside target regions is also ignored. Therefore, this paper proposes a spatial graph regula...
-
Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and fully autonomous inter-floor navigation is still not available. As the very first step of inter-floor na...
-
Uncertainty estimation for point cloud semantic segmentation is to quantify the confidence degree for the predicted label of points, which is essential for decision-making tasks. This paper proposes a neighborhood spatial aggregation based method, NSA-MC dropout, to achieve efficient uncertainty estimation for point cloud semantic segmentation. Unlike the traditional uncertainty estimation method ...
-
Semantic Segmentation is a crucial component in the perception systems of many applications, such as robotics and autonomous driving that rely on accurate environmental perception and understanding. In literature, several approaches are introduced to attempt LiDAR semantic segmentation task, such as projection-based (range-view or birds-eye-view), and voxel-based approaches. However, they either a...
-
Recently, quadrotors are gaining significant attention in aerial transportation and delivery. In these scenarios, an accurate estimation of the external force is as essential as the six degree-of-freedom (DoF) pose since it is of vital importance for planning and control of the vehicle. To this end, we propose a tightly-coupled Visual-Inertial-Dynamics (VID) system that simultaneously estimates th...
-
In recent years, thanks to the continuously reduced cost and weight of 3D lidar, the applications of this type of sensor in the community have become increasingly popular. Despite many progresses, estimation drift and tracking loss are still prevalent concerns associated with these systems. However, in theory these issues can be resolved with the use of some observations to fixed landmarks in the ...
-
The learning-from-observation (LfO) framework aims to map human demonstrations to a robot to reduce programming effort. To this end, an LfO system encodes a human demonstration into a series of execution units for a robot, which are referred to as task models. Although previous research has proposed successful task-model encoders, there has been little discussion on how to guide a task-model encod...
-
Powered exoskeletons for people with paraplegia have been widely developed. To generate the basic but essential motions for daily human life, precise control algorithms to follow the joint reference trajectories are necessary. The dynamic characteristics of the exoskeletal joints, however, varies signifi-cantly during walking because the load side is exchanged from legs in the air to the wearer’s ...
-
As a life-threatening disease, stroke can lead to long-term problems affecting the patients’ daily living ability. A common problem facing post-stroke patients is foot drop. An emerging modality of interest for correcting the foot drop is to combine both actuated ankle-foot orthosis (AAFO) and functional electrical stimulation (FES). Such hybrid assistive system not only ensure effective assistanc...
-
Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, p...
-
Many robotics applications require interest points that are highly repeatable under varying viewpoints and lighting conditions. However, this requirement is very challenging as the environment changes continuously and indefinitely, leading to appearance changes of interest points with respect to time. This paper proposes to predict the repeatability of an interest point as a function of time, whic...
-
We present DRACO, a method for Dense Reconstruction And Canonicalization of Object shape from one or more RGB images. Canonical shape reconstruction— estimating 3D object shape in a coordinate space canonicalized for scale, rotation, and translation parameters—is an emerging paradigm that holds promise for a multitude of robotic applications. Prior approaches either rely on painstakingly gathered ...
-
Dense optical flow estimation plays a key role in many robotic vision tasks. In the past few years, with the advent of deep learning, we have witnessed great progress in optical flow estimation. However, current networks often consist of a large number of parameters and require heavy computation costs, largely hindering its application on low power-consumption devices such as mobile phones. In thi...
-
In this paper, we propose a new video object detection (VoD) method, referred to as temporal feature aggregation and motion-aware VoD (TM-VoD), that produces a joint representation of temporal image sequences and object motion. The TM-VoD generates strong spatiotemporal features for VOD by temporally redundant information in an image sequence and the motion context. These are produced at the featu...
-
We propose a targetless method for calibrating the extrinsic parameters among multiple cameras and a LiDAR sensor using object pose estimation. Contrast to previous targetless methods requiring certain geometric features, the proposed method exploits any objects of unspecified shapes in the scene to estimate the calibration parameters in single-scan configuration. Semantic objects in the scene are...
-
Point set registration plays a critical role in robotics and computer vision. Early methods considered registration as a purely geometric problem, presenting excellent extensibility for various tasks due to their explicit handling of correspondences; statistical methods were later introduced to handle noise. However, the two categories of algorithms have evolved independently without sharing much ...
-
Robot motion control aims to generate control inputs for a robotic system to track a planned trajectory. Feedback provided by sensors plays an essential role in motion control by improving system performance when external disturbances and/or initial errors exist. However, feedback signals, such as images are often of a large size, which imposes a heavy computational burden on the system. In this p...
-
6D pose estimation of objects is an important part of robot grasping. The latest research trend on 6D pose estimation is to train a deep neural network to directly predict the 2D projection position of the 3D key points from the image, establish the corresponding relationship, and finally use Pespective-n-Point (PnP) algorithm performs pose estimation. The current challenge of pose estimation is t...
-
Tactile perception on our fingers is a key sensory feedback that enables us to perceive and explore our world using our hands as probes, and is essential for efficient gripping and manipulation of objects. A tactile feedback system can therefore greatly improve the quality of life of individuals with partial or complete sensory loss like during stroke, or with artificial limbs after an amputation....
-
PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation
We propose a novel objective for penalizing geometric inconsistencies and improving the depth and pose estimation performance of monocular camera images. Our objective is designed using the Wasserstein distance between two point clouds, estimated from images with different camera poses. The Wasserstein distance can impose a soft and symmetric coupling between two point clouds, which suitably maint...
-
The ability to estimate 3D object shape from a single image is vital to robotics and manufacturing. For instance, it enables iterative trial-and-error in simulated environments. In single-view reconstruction, implicit functions have demonstrated superior results over traditional methods. However, implicit functions suffer from the heavy computation of mesh extraction. This is due to the indirect m...
-
Curb detection is an essential function of autonomous vehicles in urban areas. However, curbs are difficult to detect in complex urban environments in which many dynamic objects exist. Additionally, curbs appear in a variety of shapes and sizes. Previous studies have been based on the traditional pipeline, which consists of the extraction and aggregation of hand-crafted features that are then fed ...
-
Accurate and fast inventory management algorithms are essential in the modern distribution industry. However, the configuration process of inventory management algorithms is very expensive, and the direct comprehensive management of inventory procedures is labor intensive and inaccurate. Therefore, in this paper, we propose an optical character recognition (OCR)-based inventory management algorith...
Introduction
Conference ICRA2021 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 1000