DeepNLP IROS2021 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
Provides an abstract for each of the invited presentations and may include a brief professional biography of each presenter. The complete presentations were not made available for publication as part of the conference proceedings.
-
An efficient 3D scene perception algorithm is a vital component for autonomous driving and robotics systems. In this paper, we focus on semantic scene completion, which is a task of jointly estimating the volumetric occupancy and semantic labels of objects. Since the real-world data is sparse and occluded, this is an extremely challenging task. We propose a novel framework, named Up-to-Down networ...
-
With the availability of many datasets tailored for autonomous driving in real-world urban scenes, semantic segmentation for urban driving scenes achieves significant progress. However, semantic segmentation for off-road, unstructured environments is not widely studied. Directly applying existing segmentation networks often results in performance degradation as they cannot overcome intrinsic probl...
-
Convolutional Neural Networks (CNNs) have been increasingly applied in visual classification tasks by replacing hand-crafted features with deep features. However, problems such as inter-class similarity and intra-class variation led to the need of obtaining more descriptive features. To accomplish this, a new semantic inter-object relationship approach is proposed, which is based on the distance r...
-
Scene recognition is a fundamental task in robotic perception. For human beings, scene recognition is reasonable because they have abundant object knowledge of the real world. The idea of transferring prior object knowledge from humans to scene recognition is significant but still less exploited. In this paper, we propose to utilize meaningful object representations for indoor scene representation...
-
This paper considers online object-level mapping using partial point-cloud observations obtained online in an unknown environment. We develop an approach for fully Convolutional Object Retrieval and Symmetry-AIded Registration (CORSAIR). Our model extends the Fully Convolutional Geo-metric Features model to learn a global object-shape embedding in addition to local point-wise features from the poi...
-
Estimating a scene’s depth to achieve collision avoidance against moving pedestrians is a crucial and fundamental problem in the robotic field. This paper proposes a novel, low complexity network architecture for fast and accurate human depth estimation and segmentation in indoor environments, aiming to applications for resource-constrained platforms (including battery-powered aerial, micro-aerial...
-
We present a novel approach to joint depth and normal estimation for time-of-flight (ToF) sensors. Our model learns to predict the high-quality depth and normal maps jointly from ToF raw sensor data. To achieve this, we meticulously constructed the first large-scale dataset (named ToF-100) with paired raw ToF data and ground-truth high-resolution depth maps provided by an industrial depth camera. ...
-
We propose an extension to the segment-based global localization method for LiDAR SLAM using descriptors learned considering the visual context of the segments. A new architecture of the deep neural network is presented that learns the visual context acquired from synthetic LiDAR intensity images. This approach allows a single multi-beam LiDAR to produce rich and highly descriptive location signat...
-
We present a two-stage transfer learning method for training state-dependent sensor measurement models (SDSMMs) with limited sensor data. This method can alleviate collecting sizeable sensor and ground truth data to learn accurate sensor models, especially when we must learn many sensor models (for example, a fleet of autonomous cars, drones, or warehouse robots). In the first stage, we use prior ...
-
Among various 3D capturing systems, since the system with line lasers based on the light sectioning method is simple and accurate, it has widely attracted many developers and used for many purposes. In addition, there is no need to synchronize the camera and the laser and also the configuration of the camera and the lasers is flexible, and thus, the system can be used for extreme conditions, such ...
-
Micro Air Vehicles (MAVs) are increasingly being used for complex or hazardous tasks in enclosed and cluttered environments such as surveillance or search and rescue. With this comes the necessity for sensors that can operate in poor visibility conditions to facilitate with navigation and avoidance of objects or people. Radar sensors in particular can provide more robust sensing of the environment...
-
Autonomous vehicles may not behave as expected when subject to environmental disturbances. For instance, control commands suitable for driving on dry, paved roads may lead to unsafe conditions and undesired deviations when on slippery dirt or icy roads. Furthermore, it becomes increasingly important to offer human-understandable explanations of autonomous robots’ actions – especially when they ope...
-
Model-free reinforcement learning has become a viable approach for vision-based robot control. However, sample complexity and adaptability to domain shifts remain persistent challenges when operating in high-dimensional observation spaces (images, LiDAR), such as those that are involved in autonomous driving. In this paper, we propose a flexible framework by which a policy’s observations are augme...
-
This paper presents a self-supervised Learning from Learned Hallucination (LfLH) method to learn fast and reactive motion planners for ground and aerial robots to navigate through highly constrained environments. The recent Learning from Hallucination (LfH) paradigm for autonomous navigation executes motion plans by random exploration in completely safe obstacle-free spaces, uses hand-crafted hall...
-
Learning robust policies for robotic systems operating in presence of uncertainty is a challenging task. For safe navigation, in addition to the natural stochasticity of the environment and vehicle dynamics, the perception uncertainty associated with dynamic entities, e.g. pedestrians, must be accounted for during motion planning. To this end, we construct an algorithm with built-in robustness to ...
-
We propose a novel autonomous driving frame-work that leverages graph-based features of roads, such as road positions and connections. The proposed method is divided into two parts: a low-level controller which follows the trajectory calculated by a graph-based path planner, and a high-level controller which determines the speed of the vehicle to follow the traffic flow. The high-level controller ...
-
Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving vehicles. In these applications, failure of perception systems may put human life at risk, and a broad adoption of these technologies requires the development of methodologies to guarantee and monitor safe operation. Despite the paramount importance of perception systems, curr...
-
General contact-rich manipulation problems are long-standing challenges in robotics due to the difficulty of understanding complicated contact physics. Deep reinforcement learning (RL) has shown great potential in solving robot manipulation tasks. However, existing RL policies have limited adaptability to environments with diverse dynamics properties, which is pivotal in solving many contact-rich ...
-
This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn la...
-
Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or policies. Especially in the context of robotics, in which the cost of real-world data is usually extremely high, Reinforcement Learning solutions achieving high sample efficiency are needed. In this paper, we propose a framework co...
-
For robotic grasping tasks with diverse target objects, some deep learning-based methods have achieved state-of-the-art results using direct visual input. In contrast, actor-critic deep reinforcement learning (RL) methods typically perform very poorly when applied to grasp diverse objects, especially when learning from raw images and sparse rewards. To render these RL techniques feasible for visio...
-
The perception of the environment plays a decisive role in the safe and secure operation of autonomous vehicles. The perception of the surrounding is way similar to human vision. The human’s brain perceives the environment by utilizing different sensory channels and develop a view-invariant representation model. In this context, different exteroceptive sensors like cameras, Lidar, are deployed on ...
-
Humans learn to imitate by observing others. However, robot imitation learning generally requires expert demonstrations in the first-person view (FPV). Collecting such FPV videos for every robot could be very expensive.Third-person imitation learning (TPIL) is the concept of learning action policies by observing other agents in a third-person view (TPV), similar to what humans do. This ultimately ...
-
3D laser scanning by LiDAR sensors plays an important role for mobile robots to understand their surroundings. Nevertheless, not all systems have high resolution and accuracy due to hardware limitations, weather conditions, and so on. Generative modeling of LiDAR data as scene priors is one of the promising solutions to compensate for unreliable or incomplete observations. In this paper, we propos...
-
While an exciting diversity of new imaging devices is emerging that could dramatically improve robotic perception, the challenges of calibrating and interpreting these cameras have limited their uptake in the robotics community. In this work we generalise techniques from unsupervised learning to allow a robot to autonomously interpret new kinds of cameras. We consider emerging sparse light field (...
-
State estimation with sensors is essential for mobile robots. Due to different performance of sensors in different environments, how to fuse measurements of various sensors is a problem. In this paper, we propose a tightly coupled multi-sensor fusion framework, Lvio-Fusion, which fuses stereo camera, Lidar, IMU, and GPS based on the graph optimization. Especially for urban traffic scenes, we intro...
-
Reactive Visual Odometry Scheduling Based on Noise Analysis using an Adaptive Extended Kalman Filter
A new strategy is proposed for scheduling Visual Odometry (VO) measurements for wheeled ground vehicles. Rather than having a fixed interval or distance between image acquisitions, we propose to trigger VO based on covariances from an Adaptive Extended Kalman Filter. The adopted model uses process noise to drive wheel slip estimation, which, when correctly identified, can be used with Wheel Odomet...
-
Multi-sensor Fusion Incorporating Adaptive Transformation for Reconfigurable Pavement Sweeping Robot
An efficient sensors fusion framework in an autonomous robot is necessary for various functions like object detection and perception enhancement. Multi-sensor calibration techniques are used to fuse multiple static sensors into a single frame of reference. However, for reconfigurable robots, sensors can change pose during reconfiguration need a robust adaptive sensor fusion to account for the rela...
-
We present an approach for radar-inertial odometry which uses a continuous-time framework to fuse measurements from multiple automotive radars and an inertial measurement unit (IMU). Adverse weather conditions do not have a significant impact on the operating performance of radar sensors unlike that of camera and LiDAR sensors. Radar’s robustness in such conditions and the increasing prevalence of...
-
In this paper, we propose to fuse radar measurements with Visual Inertial Odometry (RVIO) or Thermal Inertial Odometry (RTIO). FMCW radar sensor data enables to estimate the 3D ego velocity independent of the visual conditions. Fusion with VIO or TIO heavily improves the robustness in challenging conditions such as darkness, direct sunlight or fog.Specifically, we propose RRxIO: An extension to th...
-
We propose a 2D path planning algorithm in a non-convex workspace defined as a sequence of connected convex polytopes. The reference path is parameterized as a B-spline curve, which is guaranteed to entirely remain within the workspace by exploiting the local convexity property and by formulating linear constraints on the control points of the B-spline. The novelties of the paper lie in the use of...
-
Motion planning for complex dynamic systems such as helicopters is a challenging problem due to non-holonomic and nonlinear differential constraints. Approaches for optimal kinodynamics motion planning have only been demonstrated for simple dynamic systems such as Dubins car or linear systems. In this paper we present Closed-loop FMT* (CL-FMT*) which is an extension of FMT* [7] that uses closed-lo...
-
For many tasks, including table tennis, catching, and sword fighting, a critical step is intercepting the incoming object with a robot arm or held tool. Solutions to robot arm interception via learning, specifically reinforcement learning (RL), have become prevalent, as they provide robust solutions to the robot arm interception problem, even for high degree of freedom robotic systems. Despite num...
-
Path planning in the presence of dynamic obstacles is a fundamental problem in robotics with widespread applications. A typical approach to such problems is that a robot predicts the trajectories of dynamic obstacles, and plans its path while avoiding them. Such a formulation becomes limiting though for scenarios where an agent cannot complete its task efficiently, without disrupting the movement ...
-
This paper presents a novel method to generate spatial constraints for motion planning in dynamic environments. Motion planning methods for autonomous driving and mobile robots typically need to rely on the spatial constraints imposed by a map-based global planner to generate a collision-free trajectory. These methods may fail without an offline map or where the map is invalid due to dynamic chang...
-
Application of multiple robotic manipulators in a shared workspace is still restricted to repetitive tasks limiting their flexible deployment for production systems. Still, existing motion control algorithms cannot be performed online for arbitrary environments in case of multiple manipulators cooperating with each other. In this work we propose a scalable and real-time capable motion control algo...
-
We propose a new algorithm to simplify the controller development for distributed robotic systems subject to external observations, disturbances, and communication delays. Unlike prior approaches that propose specialized solutions to handling communication latency for specific robotic applications, our algorithm uses an arbitrary centralized controller as the specification and automatically genera...
-
Long-term deployment of a fleet of mobile robots requires reliable and secure two-way communication channels between individual robots and remote human operators for supervision and tasking. Existing open-source solutions to this problem degrade in performance in challenging real-world situations such as intermittent and low-bandwidth connectivity, do not provide security control options, and can ...
-
Many algorithms for control of multi-robot teams operate under the assumption that low-latency, global state information necessary to coordinate agent actions can readily be disseminated among the team. However, in harsh environments with no existing communication infrastructure, robots must form ad-hoc networks, forcing the team to operate in a distributed fashion. To overcome this challenge, we ...
-
This paper introduces a distributed multi-robot collision avoidance algorithm based on the concept of Buffered Voronoi Cells (BVC). We propose a novel algorithm for avoiding deadlocks consisting of three stages: deadlock prediction, deadlock recovery, and deadlock recovery success prediction. Simple heuristics (such as the right-hand rule) are often used to avoid deadlocks. Such heuristics might r...
-
In multi-robot multi-target tracking, robots coordinate to monitor groups of targets moving about an environment. We approach planning for such scenarios by formulating a receding-horizon, multi-robot sensing problem with a mutual information objective. Such problems are NP-Hard in general. Yet, our objective is submodular which enables certain greedy planners to guarantee constant-factor suboptim...
-
Loop closure detection (LCD), which aims to deal with the drift emerging when robots travel around the route, plays a key role in a simultaneous localization and mapping system. Unlike most current methods which focus on seeking an appropriate representation of images, we propose a novel two-stage pipeline dominated by the estimation of spatial geometric relationship. When a query image occurs, we...
-
Scene text detection plays an important role on vision-based robot navigation to many potential landmarks such as nameplates, information signs, floor button in the elevators. Recently, scene text detection with segmentation-based methods has been receiving more and more attention. The segmentation results can be used to efficiently predict scene text of various shapes, such as irregular text in m...
-
We present a reward-predictive, model-based learning method featuring trajectory-constrained visual attention for use in mapless, local visual navigation tasks. Our method learns to place visual attention at locations in latent image space which follow trajectories caused by vehicle control actions to later enhance predictive accuracy during planning. Our attention model is jointly optimized by th...
-
Recent work has shown results on learning navigation policies for idealized cylinder agents in simulation and transferring them to real wheeled robots. Deploying such navigation policies on legged robots can be challenging due to their complex dynamics, and the large dynamical difference between cylinder agents and legged systems. In this work, we learn hierarchical navigation policies that accoun...
-
The advent of deep learning has inspired research into end-to-end learning for a variety of problem domains in robotics. For navigation, the resulting methods may not have the generalization properties desired let alone match the performance of traditional methods. Instead of learning a navigation policy, we explore learning an adaptive policy in the parameter space of an existing navigation modul...
-
Fully autonomous mobile robots have a multitude of potential applications, but guaranteeing robust navigation performance remains an open research problem. For many tasks such as repeated infrastructure inspection, item delivery, or inventory transport, a route repeating capability can be sufficient and offers potential practical advantages over a full navigation stack. Previous teach and repeat r...
-
Vision-based systems for terrain detection are ubiquitous in mobile robotics, while such systems recently emerged for locomotion assistance of disabled people. For instance, wearable devices embedding vision sensors can assist people in navigation; or guide lower-limb prosthesis or exoskeleton controller to retrieve gait patterns being adapted to the executed task (overground walking, stairs, slop...
-
Point set registration (PSR) is an essential problem in surgical navigation and image-guided surgery (IGS). It can help align the pre-operative volumetric images with the intra-operative surgical space. The performances of PSR are susceptible to noise and outliers, which are the cases in real-world surgical scenarios. In this paper, we provide a novel point set registration method that utilizes th...
-
Localization and Control of Magnetic Suture Needles in Cluttered Surgical Site with Blood and Tissue
Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effect...
-
Forecasting the future behaviors of dynamic actors is an important task in many robotics applications such as self-driving. It is extremely challenging as actors have latent intentions and their trajectories are governed by complex interactions between the other actors, themselves, and the map. In this paper, we propose LaneRCNN, a graph-centric motion forecasting model that captures the actor-to-...
-
This paper proposes a stereovision-guided robotic laser system that can conduct laser ablation on targets selected by human operators in the color image, referred as StereoCNC. Two digital cameras are integrated into a previously developed robotic laser system to add a color sensing modality and formulate the stereovision. A calibration method is implemented to register the coordinate frames betwe...
-
In this paper, we propose a novel algorithm for fusing a sequence of 3D images, named as Direct Bundle Adjustment (DBA). This algorithm simultaneously optimizes the global pose parameters of image frames and the intensity values of the fused global image using the 3D image data directly (without extracting features from the images). This one-step 3D image fusion approach is achieved by formulating...
-
This paper presents an offset-free model predictive controller for fast and accurate control of a spherical soft robotic arm. In this control scheme, a linear model is combined with an online disturbance estimation technique to systematically compensate model deviations. Dynamic effects such as material relaxation resulting from the use of soft materials can be addressed to achieve offset-free tra...
-
Most commercially available fixed-wing aerial vehicles (FWV) can carry only small, lightweight computing hardware such as Jetson TX2 onboard. Solving non-linear trajectory optimization on these computing resources is computationally challenging even while considering only the kinematic motion model. Most importantly, the computation time increases sharply as the environment becomes more cluttered....
-
Optimal control is often used in robotics for planning a trajectory to achieve some desired behavior, as expressed by the cost function. Most works in optimal control focus on finding a single optimal trajectory, which is then typically tracked by another controller. In this work, we instead consider trajectory distribution as the solution of an optimal control problem, resulting in better trackin...
-
Highly dynamic robotic tasks require high-speed and reactive robots. These tasks are particularly challenging due to the physical constraints, hardware limitations, and the high uncertainty of dynamics and sensor measures. To face these issues, it’s crucial to design robotics agents that generate precise and fast trajectories and react immediately to environmental changes. Air hockey is an example...
-
The sense of taste is fundamental to a human chef’s ability to cook tasty food. To develop robots that can demonstrate human-like cooking, robots need to be equipped with a sense of taste and enabled to use this perception capability to improve or understand the food which they are cooking. We propose a first study of using a salinity sensor to provide a robot with a sense of saltiness. We then de...
-
In every timed car race, the goal is to drive through the racing track as fast as possible. The total time depends on selection of the racing line. Following a better racing line often decides who wins. In this paper, we solve the optimal racing line problem using a genetic algorithm. We propose a novel racing line encoding based on a homeomorphic transformation called Matryoshka mapping. We evalu...
-
Research in biomechanics hypothesizes that human motion is optimal with respect to an unknown cost function that varies depending on the action and/or task. This unknown cost function is often approximated as the weighted sum of a set of features or basis cost functions. As a person performs a sequence of actions, the weights associated to each of these basis functions are likely to vary over time...
-
In this paper we present a novel approach to accomplishing soft robot configuration estimation and control using RGB-D cameras and SLAM-based methods. By placing cameras on the unactuated sections of our large-scale (approximately 2 meters long) pneumatic soft robot, we can map an environment and then estimate the orientation of the robot links using landmark-based localization. Using the orientat...
-
Soft material robotics is a rather young research field in the robotics and material science communities. A popular design is the soft pneumatic actuator (SPA) which, if connected serially, becomes a highly compliant manipulator. This high compliance makes it possible to adapt to the environment and in the future might be very useful for manipulation tasks in narrow and wound environments. A centr...
-
In this paper, we propose a simple analytical model for pneu-net soft actuator. The model is based on Euler– Bernoulli finite strain hyperelastic thin cantilever beam theory. The deformation of the air chambers is modelled using infinitesimal strain membrane theory. The proposed theoretical model estimates the deformation and force characteristics of the actuator. The developed model accounts the ...
-
To date, soft robots have been increasingly designed and analyzed, especially, Soft Continuum Manipulators (SCMs). Due to dexterous deformability, their Inverse Kinematics (IK) is still difficult to solve. Cyclic Coordinate Descent (CCD) algorithm is one of the classical optimization algorithms to solve IK of rigid manipulators with prismatic or rotational joints. However, it cannot be directly ex...
-
Soft robot modeling tends to prioritize soft robot dynamics in order to recover how they might behave. Soft robot design tends to focus on how to use compliant elements with actuation to effect certain canonical movement profiles. For soft robot locomotors, these profiles should lead to locomotion. Naturally, there is a gap between the emphasis of computational modeling and the needs of locomotion...
-
Due to their inherent compliance, soft robots are more versatile than rigid linked robots when they interact with their environment, such as object manipulation or biomimetic motion, and are considered to be the key element in introducing robots to everyday environments. Although various soft robotic actuators exist, past research has focused primarily on designing and analyzing single components....
-
A dynamic model of a soft fibre-reinforced fluidic elastomer is presented and experimentally verified, which can be used for model-based controller design. Due to the inherent visco-(hyper)elastic characteristics and nonlinear time-dependent behaviour of soft fluidic elastomer robots, analytic dynamic modelling is challenging. The fibre reinforced noninflatable soft fluidic elastomer robot used in...
-
Casting manipulation has been studied to expand the robot’s movable range. In this manipulation, the robot throws and reaches the end effector to a distant target. Usually, a special casting manipulator, which consists of rigid arm links and specific flexible linear objects, is constructed for an effective casting manipulation. However, the special manipulator cannot perform normal manipulations, ...
-
In this paper, we presented a new method for deformation control of deformable objects, which utilizes both visual and tactile feedback. At present, manipulation of deformable objects is basically formulated by assuming positional constraints. But in fact, in many situations manipulation has to be performed under actively applied force constraints. This scenario is considered in this research. In ...
-
This paper presents a real-time safety and control for robot manipulators using control barrier functions and control Lyapunov functions in operational space. We first define the operational space in terms of system dynamics, jacobian, and torques and then ensure safety by designing Control Barrier Functions (CBF) around the body links of the robotic manipulator. The control barrier function provi...
-
Rotating a grasped object about all three spatial axes is challenging, because kinematically redundant robot hands require complex control schemes for within-hand rotations, and simple parallel grippers require inefficient whole arm motions. We present a novel 3-finger robot hand design inspired by a spherical parallel mechanism that achieves these rotations with just 3 actuators. The hand is desi...
-
In this work, we focus on improving the robot’s dexterous capability by exploiting visual sensing and adaptive force control. TeachNet, a vision-based teleoperation learning framework, is exploited to map human hand postures to a multi-fingered robot hand. We augment TeachNet, which is originally based on an imprecise kinematic mapping and position-only servoing, with a biomimetic learning-based c...
-
In this paper we propose a novel flexible and optimization-free controller for standard torque-controlled manipulator for Robotic-Assisted Minimally Invasive Surgery. A novel method has been developed to model the constraint introduced by the laparoscopic tool, i.e. the remote center of motion, exploiting closed chain manipulators theory, and the final controller was synthesized considering the ef...
-
This paper reports the design and evaluation of a novel piezo based actuator for needle drive in autonomous Deep Anterior Lamellar Keratoplasty (piezo-DALK). The actuator weighs less than 8g and is 20mm x 20mm x 10.5mm in size, making it ideal for eye-mounted applications. Mean open loop positional deviation was 1.17 ± 3.15um, and system repeatability and accuracy were 17.16um and 18.33um, respect...
-
This paper presents a hand-worn assistive device to assist a visually impaired person with object manipulation. The device uses a Google Pixel 3 as the computational platform, a Structure Core (SC) sensor for perception, a speech interface, and a haptic interface for human-device interaction. W-ROMA is intended to assist a visually impaired person to locate a target object (nearby or afar) and gui...
-
Humans learn complex motor skills with practice and training. Though the learning process is not fully understood, several theories from motor learning, neuroscience, education, and game design suggest that curriculum-based training may be the key to efficient skill acquisition. However, designing such a curriculum and understanding its effects on learning are challenging problems. In this paper, ...
-
Humans must remain unharmed during their interaction with robots. We present a new method guaranteeing impact force limits when humans and robots share a workspace. Formal guarantees are realized using an online verification method, which plans and verifies fail-safe maneuvers through predicting reachable impact forces by considering all future possible scenarios. We model collisions as a coupled ...
-
The Probabilistic Movement Primitives (ProMPs) is an essential issue and framework for robotics Learning from Demonstration (LfD). It has been successfully applied to the robotics field in tasks such as skill acquisition and Human-Robot Collaboration (HRC). In this paper, we focus on its adaptability in the HRC scenario, in which the adaptability of the ProMPs allows the robot to predict the futur...
-
Planning for robotic systems is frequently formulated as an optimization problem. Instead of manually tweaking the parameters of the cost function, they can be learned from human demonstrations by Inverse Reinforcement Learning (IRL). Common IRL approaches employ a maximum entropy trajectory distribution that can be learned with soft reinforcement learning, where the reward maximization is regular...
-
Awareness of its surroundings is a crucial capability for a robot meant to be working alongside other robots or human operators. When considering safety norms and modalities, in particular the Speed and Separation Monitoring (SSM), proper proximity information can make the difference in the overall efficiency of a use case, for example avoiding unnecessary penalizations in the cycle-time. This pap...
-
This paper presents the design and evaluation of the low-cost capacitive proximity-force-fusion sensor "safe skin", which can measure simultaneously the proximity of humans as well as the contact force. It was designed such that the force and proximity sensing functions can work concurrently without interfering with each other. Moreover, active shielding, on-chip digitization and ground isolation ...
-
Studies on the production of emotions have been conducted to create robotic facial expressions. The reported methodologies for generating emotions for a robot have focused on recognizing a user’s emotions using devices, such as cameras and microphones, and then generating the reactive emotions of a robot according to the user’s emotions. However, these methodologies may have some limitations in de...
-
In this paper we develop a gestural communication perception system for a social robot companion that is able to autonomously learn novel gestures on-the-fly. The system constantly tracks human gestural activities with a camera and evaluates the performed gestures under an open-set assumption. This allows for the identification of unknown gestures. Once detected, the system stores motion sequences...
-
Inducing a user’s behavior through social interaction is a goal that a social robot aims to achieve. It has been argued that empathy has a strong effect on behavior inducement. In human-human interaction, it has been verified that the influence of a nonverbal cue on empathy outweighs that of a verbal cue when those are used in a combined way. The objectives of this study are to explore if such out...
-
A human patient simulator (HPS) can achieve effective visual-, auditory-, text-, and alarm-based feedback methods in care or nursing education. Among these, the method of visual feedback is important to design an HPS that can express emotions or feelings of pain like an actual human does because this method allows an immediate reaction between robots and humans. This study aims to develop an avata...
-
This paper introduces a quadruped robot, Tachyon, which aims to achieve high payload, robust, and dynamic locomotion on the various terrain with high energy efficiency. Thanks to a novel compact series-parallel elastic actuator (SPEA) on the upper link and a four-bar linkage design in the knee joint for constant vertical foot force, the 41-kg robot can carry more than 20 kg of payloads with dynami...
-
This paper presents a novel methodology to model and optimize trajectories of a quadrupedal robot with spinal compliance to improve standing jump performance compared to quadrupeds with a rigid spine. We introduce an elastic model for a prismatic robotic spine that is actively preloaded and mechanically lock-enabled at initial and maximum length, and develop a constrained trajectory optimization m...
-
Legged robots primarily energize their center of mass through external contact during stance phase. This links their range of possible motions to actuator power limits applied during usually short periods of time. Enabling limb actuators to pump energy into the system during non-contact phases can greatly extend the energetic profile of possible motions. However, funneling this extra energy into u...
-
Simple template models have proven useful for understanding the underlying dynamics of legged locomotion. The most common one, the SLIP model, considers the legs as linear springs with constant stiffness, and it explains well the radial dynamics of the legs. However, in order to study the influence of the leg swing dynamics and leg segmentation on gait stability, more complex models are required. ...
-
A bio-inspired way to model locomotion is using a network of coupled phase oscillators to create a Central Pattern Generator (CPG). The recently developed feedback control method tegotae includes exteroceptive force feedback into the governing phase update equations, leading to gait limit cycles. However, the oscillator coupling weights are often determined empirically. Here, we first investigate ...
-
Recently, a new category of bio-inspired legged robots moving directly on the seabed have been proposed to complement the abilities of traditional underwater vehicles and to enhance manipulation and sampling tasks. So far, only tele-operated use of underwater legged robots has been reported and in this paper we attempt to fill such gap by presenting the first step towards autonomous area inspectio...
-
Many of today’s robot perception systems aim at accomplishing perception tasks that are too simplistic and too hard. They are too simplistic because they do not require the perception systems to provide all the information needed to accomplish manipulation tasks. Typically the perception results do not include information about the part structure of objects, articulation mechanisms and other attri...
-
The process of capturing a well-composed photo is difficult and it takes years of experience to master. We propose a novel pipeline for an autonomous agent to automatically capture an aesthetic photograph by navigating within a local region in a scene. Instead of classical optimization over heuristics such as the rule-of-thirds, we adopt a data-driven aesthetics estimator to assess photo quality. ...
-
In recent years, there have been many studies on sports robots that can play against humans, including studies on the strategies that sports robots use by taking into account the physical conditions of their opponents. However, there have been few studies on strategies that take into account psychological conditions of the opponents, such as carelessness and habituation. This paper proposes a moti...
-
Robotic contact juggling is a challenging task in which robots must control the movement of a ball rapidly and indirectly without holding it while keeping the ball in and sometimes out of contact with the robot’s body. In this work, we address the problem of learning such robotic contact juggling from trial and error via model-based reinforcement learning (MBRL). The key insight is that complex ro...
-
Elderly and mobility impaired people need special attention during bathing activities, since these tasks are demanding in body flexibility. Our aim is to build an assistive robotic bathing system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions have to be adopted, in order to ac...
-
Advances in sensing and learning algorithms have led to increasingly mature solutions for human detection by robots, particularly in selected use-cases such as pedestrian detection for self-driving cars or close-range person detection in consumer settings. Despite this progress, the simple question which sensor-algorithm combination is best suited for a person detection task at handƒ remains hard ...
-
We construct a Virtual Kinematic Chain (VKC) that readily consolidates the kinematics of the mobile base, the arm, and the object to be manipulated in mobile manipulations. Accordingly, a mobile manipulation task is represented by altering the state of the constructed VKC, which can be converted to a motion planning problem, formulated and solved by trajectory optimization. This new VKC perspectiv...
-
Motion optimization for legible robot intent has largely ignored the robot’s dynamics, citing burdensome complexity that prevents online deployment. Even where the original task (to be communicated) could be solved on the dynamical system, the legibility problem (to communicate that task’s intent) could not. This work simplifies the legibility objective to have equivalent computational complexity ...
-
Understanding human navigational intent is essential for robots to be able to interact with and navigate around humans safely and naturally. Current methods typically perform inference through only one mode of perception such as human motion trajectory, and a single theoretical framework such as a learning-based or classical approach. In contrast, this paper studies prediction of human navigationa...
-
A multi-modal framework to generate user intention distributions when operating a mobile vehicle is proposed in this work. The model learns from past observed trajectories and leverages traversability information derived from the visual surroundings to produce a set of future trajectories, suitable to be directly embedded into a perception-action shared control strategy on a mobile agent, or as a ...
-
Pedestrian trajectories and actions prediction in complex environment is challenging due to the complexity of human behavior and a variety of internal and external stimuli. Much works has gone towards predicting trajectories and actions separately without mining the coupling relationships between them, which is an important information for our humans to reason and predict. Inspired by this, we pro...
-
In this paper we address an important problem in self-driving, forecasting multi-pedestrian motion and their shared scene occupancy map, which is critical for safe navigation. Our contributions are two-fold. First, we advocate for predicting both the individual motions as well as the scene occupancy map in order to effectively deal with missing detections caused by postprocessing, e.g. confidence ...
-
It is important for autonomous vehicles to have the ability to infer the goals of other vehicles (goal recognition), in order to safely interact with other vehicles and predict their future trajectories. This is a difficult problem, especially in urban environments with interactions between many vehicles. Goal recognition methods must be fast to run in real time and make accurate inferences. As au...
-
The correct characterization of uncertainty when predicting human motion is equally important as the accuracy of this prediction. We present a new method to correctly predict the uncertainty associated with the predicted distribution of future trajectories. Our approach, CovariaceNet, is based on a Conditional Generative Model with Gaussian latent variables in order to predict the parameters of a ...
-
In this paper, we present a solution for simultaneous handling of large components with industrial robots performing synchronized motions with an AGV in flexible flow assembly. For this purpose, we implement an Extended Kalman Filter with a global localization system to track an AGV and multiple manipulators. We propose a model-predictive controller for force compliance and trajectory tracking in ...
-
In recent years, many learning based approaches have been studied to realize robotic manipulation and assembly tasks, often including vision and force/tactile feedback. How-ever, it is unclear what the baseline state-of-the-art performance is and what the bottleneck problems are. In this work, we evaluate off-the-shelf (OTS) industrial solutions on a recently introduced benchmark, the National Ins...
-
Assembly orders of components have direct influence on feasibility and efficiency of assembly process in manufacturing and are usually defined by experienced operators. To automate the assembly sequence generation process, we present a method using the idea of case-based reasoning, which can take advantage of experience of a reference assembly to generate the assembly sequence of a new assembly. F...
-
Collaborative robots are expected to work alongside humans and directly replace human workers in some cases, thus effectively responding to rapid changes in assembly lines. Current methods for programming contact-rich tasks, particularly in heavily constrained spaces, tend to be fairly inefficient. Therefore, faster and more intuitive approaches are urgently required for robot teaching. This study...
-
Recent studies on deep-learning-based small defection segmentation approaches are trained in specific settings and tend to be limited by fixed context. Throughout the training, the network inevitably learns the representation of the background of the training data before figuring out the defection. They underperform in the inference stage once the context changed and can only be solved by training...
-
In this paper, we propose TUPPer-Map, a metric-semantic mapping framework based on the unified panoptic segmentation and temporal data association. In contrast to the previous mapping method, our framework integrates the data association stage into the holistic pixel-level segmentation stage in an end-to-end fashion, taking advantage of both intra-frame and inter-frame spatial and temporal knowled...
-
We propose a novel neural network module that transforms an existing single-frame semantic segmentation model into a video semantic segmentation pipeline. In contrast to prior works, we strive towards a simple, fast, and general module that can be integrated into virtually any single-frame architecture. Our approach aggregates a rich representation of the semantic information in past frames into a...
-
For autonomous driving, drivable region detection is one of the most basic and essential tasks. In this paper, a novel LiDAR-based drivable region detection algorithm which could output a complete, accurate and stable result is proposed. To promote the completeness of the detection result, the Bayesian generalized kernel inference and bilateral filtering are utilized to estimate the attribute of t...
-
Road curb detection is important for autonomous driving. It can be used to determine road boundaries to constrain vehicles on roads, so that potential accidents could be avoided. Most of the current methods detect road curbs online using vehicle-mounted sensors, such as cameras or 3-D Lidars. However, these methods usually suffer from severe occlusion issues. Especially in highly-dynamic traffic e...
-
Accurate and reliable localization is a fundamental requirement for autonomous vehicles to use map information in higher-level tasks such as navigation or planning. In this paper, we present a novel approach to vehicle localization in dense semantic maps, including vectorized high-definition maps or 3D meshes, using semantic segmentation from a monocular camera. We formulate the localization task ...
-
Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications. The assistance system should not only achieve high accuracy under normal conditions, but obtain robust perception against extreme situations. However, traffic accidents that involve object collisions, deformations, overturns, etc., yet unseen in most training sets, ...
-
Freespace detection is a fundamental component of autonomous driving perception. Recently, deep convolutional neural networks (DCNNs) have achieved impressive performance for this task. In particular, SNE-RoadSeg, our previously proposed method based on a surface normal estimator (SNE) and a data-fusion DCNN (RoadSeg), has achieved impressive performance in freespace detection. However, SNE-RoadSe...
-
Robust and accurate localization is an essential component for robotic navigation and autonomous driving. The use of cameras for localization with high definition map (HD Map) provides an affordable localization sensor set. Existing methods suffer from pose estimation failure due to error prone data association or initialization with accurate initial pose requirement. In this paper, we propose a c...
-
Technologies for estimating self-position and orientation are important for both humans and robots. These technologies allow robots to perform tasks such as carrying objects and allow people to reach their destinations. Although self-position estimation technologies using GPS and laser rangefinders have been developed, few methods can be used by both humans and robots. Therefore, we developed a me...
-
The combination of ultrawideband (UWB) radios and inertial measurement units (IMU) can provide accurate positioning in environments where the Global Positioning System (GPS) service is either unavailable or has unsatisfactory performance. The two sensors, IMU and UWB radio, are often not co-located on a moving system. The UWB radio is typically located at the extremities of the system to ensure re...
-
Localization is an essential task for mobile autonomous robotic systems that want to use pre-existing maps or create new ones in the context of SLAM. Today, many robotic platforms are equipped with high-accuracy 3D LiDAR sensors, which allow a geometric mapping, and cameras able to provide semantic cues of the environment. Segment-based mapping and localization have been applied with great success...
-
Estimating the 6-DoF camera pose of an image with respect to a 3D scene model, known as visual localization, is a fundamental problem in many computer vision and robotics tasks. Among various visual localization methods, the direct 2D-3D matching method has become the preferred method for many practical applications due to its computational efficiency. When using direct 2D-3D matching methods in l...
-
Different health-monitoring techniques were considered in the literature to enhance the safety and stability of Connected Autonomous Vehicle (CAV) platoons. The health-monitoring processes include fault detection, localization, and mitigation. It is evident that mitigating these faults is faster and more reliable if the fault structure is known. To this end, we consider classifying the fault class...
-
In many robotic tasks, such as autonomous drone racing, the goal is to travel through a set of waypoints as fast as possible. A key challenge for this task is planning the timeoptimal trajectory, which is typically solved by assuming perfect knowledge of the waypoints to pass in advance. The resulting solution is either highly specialized for a single-track layout, or suboptimal due to simplifying...
-
Deep Reinforcement Learning has emerged as an efficient dynamic obstacle avoidance method in highly dynamic environments. It has the potential to replace overly conservative or inefficient navigation approaches. However, integrating Deep Reinforcement Learning into existing navigation systems is still an open frontier due to the myopic nature of Deep-Reinforcement-Learning-based navigation, which ...
-
Image synthesis driven by computer graphics achieved recently a remarkable realism, yet synthetic image data generated this way reveals a significant domain gap with respect to real-world data. This is especially true in autonomous driving scenarios, which represent a critical aspect for over-coming utilizing synthetic data for training neural networks. We propose a method based on domain-invarian...
-
Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although penalizing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies which allow to tune their le...
-
During the last decades, the research endeavours on autonomous driving found great resonance in Advanced Driver-Assistance Solutions that equipped the contemporary civilian vehicles and significantly boosted their driver-less mobility. The existing applications are mostly focused on urban scenarios where signs, road lanes and markers are well defined and ordered favouring the motion of the vehicle...
-
General purpose simulators provide cheap training data to learn complex robotic skills. However, the transition from simulation to reality is often very challenging for the agent. One major issue is the delay on the physical robot that may deteriorate the performance of the deployed agent. Furthermore, once a successfully trained learning-based control policy is available, re-purposing the knowled...
-
Domain adaptation is a common problem in robotics, with applications such as transferring policies from simulation to real world and lifelong learning. Performing such adaptation, however, requires informative data about the environment to be available during the adaptation. In this paper, we present domain curiosity—a method of training exploratory policies that are explicitly optimized to provid...
-
Enabling mobile robots for solving challenging and diverse shape, texture, and motion related tasks with high fidelity vision requires the integration of novel multimodal imaging sensors and advanced fusion techniques. However, it is associated with high cost, power, hardware modification, and computing requirements which limit its scalability. In this paper, we propose a novel Simultaneously Lear...
-
Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. A policy trained with expensive data is rendered useless after making even a minor change to the robot hardware. In this paper, we address the challenging problem of adapting a policy, trained to perform a task, to a novel robotic hardware platform ...
-
Shaping Progressive Net of Reinforcement Learning for Policy Transfer with Human Evaluative Feedback
Deep reinforcement learning has achieved significant success in many fields, but will confront sampling efficiency and safety problems when applying to robot control in the real world. Sim-to-real transfer learning was proposed to make use of samples in the simulation and overcome the gap between simulation and real world. In this paper, we focus on improving Progressive Neural Network — an effect...
-
This paper presents a novel method for transferring motion planning and control policies between a teacher and a learner robot. With this work, we propose to reduce the sim-to-real gap, transfer knowledge designed for a specific system into a different robot, and compensate for system aging and failures. To solve this problem we introduce a Schwarz–Christoffel mapping-based method to geometrically...
-
The ability to simultaneously distinguish objects, materials, and their associated physical properties is one fundamental function of the sense of touch. Recent advances in the development of tactile sensors and machine learning techniques allow more accurate and complex modelling of robotic tactile sensations. However, many state-of-the-art (SotA) approaches focus solely on constructing black-box...
-
The broad scope of obstacle avoidance has led to many kinds of computer vision-based approaches. Despite its popularity, it is not a solved problem. Traditional computer vision techniques using cameras and depth sensors often focus on static scenes, or rely on priors about the obstacles. Recent developments in bio-inspired sensors present event cameras as a compelling choice for dynamic scenes. Al...
-
3D single object tracking is a key issue for robotics. In this paper, we propose a transformer module called Point-Track-Transformer (PTT) for point cloud-based 3D single object tracking. PTT module contains three blocks for feature embedding, position encoding, and self-attention feature computation. Feature embedding aims to place features closer in the embedding space if they have similar seman...
-
In the field of large-scale SLAM for autonomous driving and mobile robotics, 3D point cloud based place recognition has aroused significant research interest due to its robustness to changing environments with drastic daytime and weather variance. However, it is time-consuming and effort-costly to obtain high-quality point cloud data for place recognition model training and ground truth for regist...
-
We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural Radiance Field (NeRF). NeRFs have been shown to be remarkably effective for the task of view synthesis — synthesizing photorealistic novel views of real-world scenes or objects. In this work, we investigate whether we can apply analysis-by-synthesis via NeRF for mesh-free, RGB-only 6DoF pose estimation – ...
-
Feature extraction plays an important role in visual localization. Unreliable features on dynamic objects or repetitive regions will interfere with feature matching and challenge indoor localization greatly. To address the problem, we propose a novel network, RaP-Net, to simultaneously predict region-wise invariability and point-wise reliability, and then extract features by considering both of th...
-
A recent line of work has shown that end-to-end optimization of Bayesian filters can be used to learn state estimators for systems whose underlying models are difficult to hand-design or tune, while retaining the core advantages of probabilistic state estimation. As an alternative approach for state estimation in these settings, we present an end-to-end approach for learning state estimators model...
-
Safe and proactive planning in robotic systems generally requires accurate predictions of the environment. Prior work on environment prediction applied video frame prediction techniques to bird’s-eye view environment representations, such as occupancy grids. ConvLSTM-based frameworks used previously often result in significant blurring of the predictions, loss of static environment structure, and ...
-
We present a control-aware design optimization method for quadrupedal robots. In particular, we show that it is possible to analytically differentiate typical, inverse dynamics-based whole body controllers with respect to design parameters, and that gradient-based methods can be used to efficiently improve an initial morphological design according to well-established metrics. We apply our design o...
-
Robotic legs often lag behind the performance of their biological counterparts. The inherent passive dynamics of natural legs largely influences the locomotion and can be abstracted through the spring-loaded inverted pendulum (SLIP) model. This model is often approximated in physical robotic legs using a leg with minimal mass. Our work aims to embed the SLIP dynamics by using a nonlinear strict os...
-
A Novel Design of Mobile Robotic System for Opening and Transitioning Through a Watertight Ship Door
Recent offshore drilling activities have dramatically bloomed oil and gas production. Due to extreme weather, such as hurricanes and tsunamis, offshore oil platforms may need to be constantly monitored in case of unexpected dangers. Using robots to monitor and prevent these dangerous situations is a cost-effective and safer solution compared to any human involvement. However, one major drawback fo...
-
Polar linkages have two degrees-of-freedom (DOF) where one input joint angle controls the length of a radial segment while another controls its angle. Considering a theoretical planar robot model, this mapping between joint angles to output motions can be shown to be energetically advantageous over the ubiquitous two-revolute linkage. Since a polar linkage’s typical construction involves a moving ...
-
In this paper, we consider a trajectory planning problem where an autonomous vehicle aims to rendezvous with another cooperating vehicle in minimum time. The first vehicle has kinematic constraints, consequently feasible trajectories must have a maximum curvature less than a specified limit. Rendezvous is said to occur at the instant that the two vehicles are collocated with the same heading. We p...
-
We demonstrate that challenging shortest path problems can be solved via direct spline regression from a neural network, trained in an unsupervised manner (i.e. without requiring ground truth optimal paths for training). To achieve this, we derive a geometry-dependent optimal cost function whose minima guarantees collision-free solutions. Our method beats state-of-the-art supervised learning basel...
-
In this paper, we develop a novel path-constrained and collision-free optimal trajectory planning algorithm for robot manipulators in the presence of obstacles for the following problem: Given a desired sequence of discrete waypoints of robot configurations, a set of robot kinematic and dynamic constraints, and a set of obstacles, determine a time and jerk optimal and collision-free trajectory for...
-
Pedestrian trajectory prediction is a challenge because of the complex social interactions in context and the elusive intention of each pedestrian. Collision avoidance is one of the most common social interactions in real world, while existing data-driven works have not handled it well yet. In order to address this issue, we propose a framework that considers the theory about the minimum distance ...
-
Motion planners for mobile robots in unknown environments face the challenge of simultaneously maintaining both robustness against unmodeled uncertainties and persistent feasibility of the trajectory-finding problem. That is, while dealing with uncertainties, a motion planner must update its trajectory, adapting to the newly revealed environment in real-time; failing to do so may involve unsafe ci...
-
Achieving a proper balance between planning quality, safety and efficiency is a major challenge for autonomous driving. Optimisation-based motion planners are capable of producing safe, smooth and comfortable plans, but often at the cost of runtime efficiency. On the other hand, naïvely deploying trajectories produced by efficient-to-run deep imitation learning approaches might risk compromising s...
-
We present a learning-based planner that aims to robustly drive a vehicle by mimicking human drivers’ driving behavior. We leverage a mid-to-mid approach that allows us to manipulate the input to our imitation learning network freely. With that in mind, we propose a novel feedback synthesizer for data augmentation. It allows our agent to gain more driving experience in various previously unseen en...
-
Voronoi coverage control is a particular problem of importance in the area of multi-robot systems, which considers a network of multiple autonomous robots, tasked with optimally covering a large area. This is a common task for fleets of fixed-wing Unmanned Aerial Vehicles (UAVs), which are described in this work by a unicycle model with constant forward-speed constraints. We develop event-based co...
-
This paper investigates servicing waypoints in a wide area using collaborative deployments of vehicles with heterogeneous range and mobility constraints. We formulate a joint planning problem for a single transport truck and multiple service drones in which the truck is constrained to a road and must deploy a team of range-constrained drones to visit waypoints. The need to deploy, collect, and red...
-
This paper studies a novel encirclement guaranteed cooperative pursuit problem involving N pursuers and a single evader in an unbounded two-dimensional game domain. Throughout the game, the pursuers are required to maintain encirclement of the evader, i.e., the evader should always stay inside the convex hull generated by all the pursuers, in addition to achieving the classical capture condition. ...
-
This study addresses the problem of path optimization for conducting a mapping mission using a multi-robot system with limited sensing capability, which aims to ensure efficient mapping with emphasis on the cooperative aspect of the mission. To achieve the cooperative mapping, a new path planning algorithm is proposed which can take advantage of the multi-robot system while dealing with the lack o...
-
To realize effective heterogeneous multi-robot teams, researchers must leverage individual robots’ relative strengths and coordinate their individual behaviors. Specifically, heterogeneous multi-robot systems must answer three important questions: who (task allocation), when (scheduling), and how (motion planning). While specific variants of each of these problems are known to be NP-Hard, their in...
-
We address the problem of assigning a team of drones to autonomously capture a set desired shots of a dynamic target in the presence of obstacles. We present a two-stage planning pipeline that generates offline an assignment of drone to shots and locally optimizes online the viewpoint. Given desired shot parameters, the high-level planner uses a visibility heuristic to predict good times for captu...
-
In this paper, we focus on the problem of learning online an optimal policy for Active Visual Search (AVS) of objects in unknown indoor environments. We propose POMP++, a planning strategy that introduces a novel formulation on top of the classic Partially Observable Monte Carlo Planning (POMCP) framework, to allow training-free online policy learning in unknown environments. We present a new beli...
-
Localizing a vehicle on a prebuilt HD vector map is a prerequisite for many autonomous driving applications. Existing visual localization approaches usually require a separate local feature layer to function. The separate localization layer suffers from the robustness issue inherited from the local features. Also, it could be difficult to create a feature layer that aligns perfectly with an existi...
-
Humans can robustly follow a visual trajectory defined by a sequence of images (i.e. a video) regardless of substantial changes in the environment or the presence of obstacles. We aim at endowing similar visual navigation capabilities to mobile robots solely equipped with a RGB fisheye camera. We propose a novel probabilistic visual navigation system that learns to follow a sequence of images with...
-
In this paper, we propose a novel Deep Reinforcement Learning approach to address the mapless navigation problem, in which the locomotion actions of a humanoid robot are taken online based on the knowledge encoded in learned models. Planning happens by generating open-loop trajectories in a learned latent space that captures the dynamics of the environment. Our planner considers visual (RGB images...
-
We present Success weighted by Completion Time (SCT), a new metric for evaluating navigation performance for mobile robots. Several related works on navigation have used Success weighted by Path Length (SPL) as the primary method of evaluating the path an agent makes to a goal location, but SPL is limited in its ability to properly evaluate agents with complex dynamics. In contrast, SCT explicitly...
-
The nature animals have evolved highly efficient and robust organs that support their complex daily navigation tasks. To mimic animal’s navigation capability, we present a novel bio-inspired navigation system that draws inspirations from nature animals in this paper. The system consists of a three-axis magnetometer, a monocular camera, a micro inertial measurement unit (MIMU) and a polarization ca...
-
The quality of robot vision greatly affects the performance of automation systems, where occlusions stand as one of the biggest challenges. If the target is occluded from the sensor, detecting and grasping such objects become very challenging. For example, when multiple robot arms cooperate in a single workplace, occlusions will be created under the robot arm itself and hide objects underneath. Wh...
-
In this paper, we propose a similarity-aware fusion network (SAFNet) to adaptively fuse 2D images and 3D point clouds for 3D semantic segmentation. Existing fusion-based methods achieve superior performances by integrating information from multiple modalities. However, they heavily rely on the projection-based correspondence between 2D pixels and 3D points and can only perform the information fusi...
-
The use of local detectors and descriptors in typical computer vision pipelines works well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inh...
-
Full body scanning plays an important role in automated industrial manufacture and inspection. It requires the fusion of multi-view point cloud data and consumes large computational resources when the geometry of corresponding point clouds are unknown. Structured Light Illumination (SLI) is one of the most promising indoor 3D imaging techniques, but also has the same weakness for fusing the multi-...
-
Learning Environment Constraints in Collaborative Robotics: A Decentralized Leader-Follower Approach
In this paper, we propose a leader-follower hierarchical strategy for two robots collaboratively transporting an object in a partially known environment with obstacles. Both robots sense the local surrounding environment and react to obstacles in their proximity. We consider no explicit communication, so the local environment information and the control actions are not shared between the robots. A...
-
In this paper we explore the use of block coordinate descent (BCD) to optimize the centroidal momentum dynamics for dynamically consistent multi-contact behaviors. The centroidal dynamics have recently received a large amount of attention in order to create physically realizable motions for robots with hands and feet while being computationally more tractable than full rigid body dynamics models. ...
-
Hamilton-Jacobi (HJ) reachability analysis is a powerful technique used to verify the safety of autonomous systems. HJ reachability is ideal for analysing nonlinear systems with disturbances and flexible set representations. A drawback to this approach is that it suffers from the curse of dimensionality, which prevents real-time deployment on safety-critical systems. In this paper, we show that a ...
-
Production plants are being re-designed to implement human-centered solutions. Especially considering high added-value operations, robots are required to optimize their behavior to achieve a task quality at least comparable to the one obtained by the skilled operators. A manual programming and tuning of the manipulator is not an efficient solution, requiring to adopt towards automated strategies. ...
-
Learning policies in simulation is promising for reducing human effort when training robot controllers. This is especially true for soft robots that are more adaptive and safe but also more difficult to accurately model and control. The sim2real gap is the main barrier to successfully transfer policies from simulation to a real robot. System identification can be applied to reduce this gap but tra...
-
Timely fault detection and identification (FDI) of soft manipulators are critical in the design of surgical systems to improve reliability. However, due to the intrinsic compliance of soft manipulators, their end effectors vibrate during the dynamic control process, which introduces noise into the measured signals and makes FDI of soft manipulators challenging. This paper proposes a novel method t...
-
Soft pneumatic legged robots show promise in their ability to traverse a range of different types of terrain, including natural unstructured terrain met in applications like precision agriculture. They can adapt their body morphology to the intricacies of the terrain at hand, thus enabling robust and resilient locomotion. In this paper we capitalize upon recent developments on soft pneumatic legge...
-
In this paper we introduce a novel technique that aims to dynamically control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce a path defined by sparse way-points. The main idea behind this work is based on the assumption that a complex trajectory may be derived as a combination of a discrete set of parameterizable simple movements, as suggested by Movement Primitive ...
-
Soft fluidic actuators are increasingly being used for wearable haptic devices due to their high energy density and low encumbrance. These actuators are typically controlled using constant fluidic pressure control (CFPC), where the actuator pressure is switched between a high pressure source and atmospheric pressure using a fluidic valve. However, this type of control has several limitations for s...
-
The tactile and proprioceptive sensation increases human manipulability, and soft tissue compliance stabilizes the grasping function. However, it is challenging to transpose this system to the small confined space of soft robotic fingers due to the material properties and complex wiring entailed. Furthermore, soft robotic fingers also incorporate actuating components, making such a system more dif...
-
Humans display the remarkable ability to sense the world through tools and other held objects. For example, we are able to pinpoint impact locations on a held rod and tell apart different textures using a rigid probe. In this work, we consider how we can enable robots to have a similar capacity, i.e., to embody tools and extend perception using standard grasped objects. We propose that vibro-tacti...
-
Perceiving obstacles and avoiding collisions is fundamental to the safe operation of a robot system, particularly when the robot must operate in highly dynamic human environments. Proximity detection using on-robot sensors can be used to avoid or mitigate impending collisions. However, existing proximity sensing methods are orientation and placement dependent, resulting in blind spots even with la...
-
The key role of tactile sensing for human grasping and manipulation is widely acknowledged, but most industrial robot grippers and even multi-fingered hands are still designed and used without any tactile sensors. While the basic design principles for resistive or capacitive sensors are well known, several factors keep tactile sensing from large-scale deployment — high sensor costs, short lifespan...
-
In this paper, we present an accelerometer-based kinematic calibration algorithm to accurately estimate the pose of multiple sensor units distributed along a robot body. Our approach is self-contained, can be used on any robot provided with a Denavit-Hartenberg kinematic model, and on any skin equipped with Inertial Measurement Units (IMUs). To validate the proposed method, we first conduct extens...
-
We present a novel robot end-effector for gripping and haptic exploration. Tactile sensing through suction flow monitoring is achieved with a new suction cup design that contains multiple chambers for air flow. Each chamber connects with its own remote pressure transducer, which enables both absolute and differential pressure measures between chambers. By changing the overall vacuum applied to thi...
-
Tactile sensing can improve end-effector control and grasp quality, especially for free-flying robots where target approach and alignment present particular challenges. However, many current tactile sensing technologies are not suitable for the harsh environment of space. We present a tactile sensor that measures normal and biaxial shear strains in the pads of a gripper using a single optical fibe...
-
Cricothyrotomy is a life-saving emergency intervention that secures an alternate airway route after a neck injury or obstruction. The procedure starts with identifying the correct location (the cricothyroid membrane) for creating an incision to insert an endotracheal tube. This location is determined using a combination of visual and palpation cues. Enabling robot-assisted remote cricothyrotomy ma...
-
In robotic surgeries employing the master-slave operation scheme, various haptic devices have been adopted as master manipulators. The main challenge of the haptic device is to contribute to a sensitive reaction to external forces for operators. Since force perception on fingertips is impaired by unstable entire hand condition, stable hand condition helps to improve force perception. This study pr...
-
Autonomous surgical execution relieves tedious routines and surgeon’s fatigue. Recent learning-based methods, especially reinforcement learning (RL) based methods, achieve promising performance for dexterous manipulation, which usually requires the simulation to collect data efficiently and reduce the hardware cost. The existing learning-based simulation platforms for medical robots suffer from li...
-
Tip force estimation on continuum arms is of crucial clinical importance for catheter-based procedures, i.e., catheter-based ablation therapies. In this study, an analytical solution for force estimation based on inverse Cosserat rod modeling was proposed and validated. Initially, a previously validated Bezier-based shape interpolation was used to parameterize the deformation and the kinematics an...
-
Lumbar injection is an image-guided procedure performed manually for diagnosis and treatment of lower back pain and leg pain. Previously, we have developed and verified an MR-Conditional robotic solution to assisting the needle insertion process. Drawing on our clinical experiences, a virtual remote center of motion (RCM) constraint is implemented to enable our robot to mimic a clinician’s hand mo...
-
Wheelchair-mounted robotic arms (and other assistive robots) should help their users perform everyday tasks. One way robots can provide this assistance is shared autonomy. Within shared autonomy, both the human and robot maintain control over the robot’s motion: as the robot becomes confident it understands what the human wants, it increasingly intervenes to automate the task. But how does the rob...
-
Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team me...
-
Despite considerable efforts by human designers, accounting for every unique situation that an autonomous robotic system deployed in the real world could face is often an infeasible task. As a result, many such deployed systems still rely on human assistance in various capacities to complete certain tasks while staying safe. Competence-aware systems (CAS) is a recently proposed model for reducing ...
-
A dynamic autonomy allocation framework automatically shifts how much control lies with the human versus the robotics autonomy, for example based on factors such as environmental safety or user preference. To investigate the question of which factors should drive dynamic autonomy allocation, we perform a human subject study to collect ground truth data that shifts between levels of autonomy during...
-
Being able to refer to an object, a person, or a place in a non-ambiguous manner is a need when one has to achieve collaborative activities with a partner. This is the so-called Referring Expression Generation (REG) problem. While widely used for Human-Robot Interaction, state of the art approaches restrict its use to the current environment. We propose a novel extension to the REG which takes ful...
-
Controlling a robotic arm to achieve manipulation tasks is challenging for humans. Especially if only low-dimensional input signals can be provided, as is often the case for users with motor impairments. Using shared control to provide task-specific guidance and constraints facilitates control – for instance with the Shared Control Templates (SCT) framework – and enables even complex activities of...
-
In collaborative robotic applications, human and robot have to work together to accomplish a common job, composed by a set of tasks. In order to achieve an efficient human-robot collaboration (HRC), it is important to have an integration between a proper task scheduling strategy and a task execution strategy. The first must deal with the variability of the two agents, while the second must deal wi...
-
Storytelling plays a central role in human socializing and entertainment, and research on conducting storytelling with robots is gaining interest. However, much of this research assumes that story content is curated. In this paper, we expand the recently-proposed task of collaborative storytelling, where an intelligent agent and a person collaborate to create a unique story by taking turns adding ...
-
Acceptance of social robots in human-robot collaborative environments depends on the robots’ sensitivity to human moral and social norms. Robot behavior that violates norms may decrease trust and lead human interactants to blame the robot and view it negatively. Hence, for long-term acceptance, social robots need to detect possible norm violations in their action plans and refuse to perform such p...
-
It is highly desirable for robots that work alongside humans to be able to understand instructions in natural language. Existing language conditioned imitation learning models directly predict the actuator commands from the image observation and the instruction text. Rather than directly predicting actuator commands, we propose translating the natural language instruction to a Python function whic...
-
We report the results of our study on whether taking a walk with a child-like robot that a person takes care of can generate interaction with the surrounding people whom the person has never met before. As the number of single-person households increases, it is expected that more people will live with not only robots that can be useful for people but also robots that people can take care of. Our s...
-
Mobile social robots should be able to engage in interaction with people effectively. However, greeting someone is a complex task since it implies an exchange of social signals. Adam Kendon modeled human greetings as a set of six phases: initiation of approach, distance salutation, head dip, approach, final approach, and close salutation. Based on Kendon’s model, we propose a system for mobile soc...
-
Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a lo...
-
The development of perception and control methods that allow bird-scale flapping-wing robots (a.k.a. ornithopters) to perform autonomously is an under-researched area. This paper presents a fully onboard event-based method for ornithopter robot visual guidance. The method uses event cameras to exploit their fast response and robustness against motion blur in order to feed the ornithopter control l...
-
In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones...
-
Autonomous aerial videography is a challenging task, which involves collision avoidance against obstacles and visibility guaranteed target tracking in unstructured environments. In this paper, we organize a two micro aerial vehicle (MAV) team, which consists of a target agent responsible for a specific mission and a camera agent for filming the target agent. Especially, this paper focuses on traje...
-
In this paper, we propose a resource-efficient approach to provide an autonomous UAV with an on-board perception method to detect safe, hazard-free landing sites during flights over complex 3D terrain. We aggregate 3D measurements acquired from a sequence of monocular images by a Structure-from-Motion approach into a local, robot-centric, multi-resolution elevation map of the overflown terrain, wh...
-
Robots operating in households must find objects on shelves, under tables, and in cupboards. In such environments, it is crucial to search efficiently at 3D scale while coping with limited field of view and the complexity of searching for multiple objects. Principled approaches to object search frequently use Partially Observable Markov Decision Process (POMDP) as the underlying framework for comp...
-
The paper focuses on collision-inclusive motion planning for impact-resilient mobile robots. We propose a new deformation recovery and replanning strategy to handle collisions that may occur at run-time. Contrary to collision avoidance methods that generate trajectories only in conservative local space or require collision checking that has high computational cost, our method directly generates (l...
-
Existing autonomous parking solutions usually require special signs, pre-built maps or accurate ranging sensors to achieve reliable perception of the parking environment, but these methods are difficult to popularize because they either require preconditions or are expensive for production cars. In this paper, we propose a vision-only autonomous parking solution based on only six cameras. Through ...
-
Finding an occluded object in a lateral access environment such as a shelf or cabinet is a problem that arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. While this problem, known as mechanical search, is well-studied in overhead access environments, lateral access environments introduce constraints on the poses of objects and on available grasp actions, and push...
-
We present a novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments with moving obstacles or targets. Our approach uses deep reinforcement learning-based expert policy that is trained using a sim2real paradigm. In order to increase the reliability and handle the failure cases of the expert policy, we combine with a poli...
-
This paper presents a reactive control framework for robotic bipedal running over random discrete terrain using a shift-invariant funnel library that can be composed into motion plans. The main contribution of the paper is the formalization of the funnel library and the introduction of a receding-horizon reactive planner that utilizes this funnel library. The proposed controller generates a robust...
-
Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, developing effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on developing novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus ...
-
The paper proposes a method for automatic multi-view dataset construction based on formula-driven supervised learning (FDSL). Although data collection and human annotation of 3D objects are labor-intensive, we automatically generate their training data and labels in the proposed multi-view dataset. To create a large-scale multi-view dataset, we employ fractal geometry, which is considered the back...
-
Place recognition is indispensable for a drift-free localization system. Due to the variations of the environment, place recognition using single-modality has limitations. In this paper, we propose a bi-modal place recognition method, which can extract a compound global descriptor from the two modalities, vision and LiDAR. Specifically, we first build the elevation image generated from 3D points a...
-
Place recognition gives a SLAM system the ability to correct cumulative errors. Unlike images that contain rich texture features, point clouds are almost pure geometric information which makes place recognition based on point clouds challenging. Existing works usually encode low-level features such as coordinate, normal, reflection intensity, etc., as local or global descriptors to represent scene...
-
This paper addresses fully-online always-adaptation of a transfer function for robot audition systems based on microphone array processing. The transfer function represents signal propagation characteristics between a microphone and a sound source, which provides essential information for real-world scene analysis, such as sound source localization and separation for robots. Although it is commonl...
-
Drone audition, namely the hearing capability of a drone, is expected to compensate for the drawbacks of visual sensors in search-and-rescue missions. Current multi-rotor drones have limitations of flight duration and sound processing due to ego-noise generated by rotors and air-flow. Drone audition for a kiteplane, i.e., a fixed-wing drone that can fly slowly and stably, has not been investigated...
-
Forestry machines are heavy vehicles performing complex manipulation tasks in unstructured production forest environments. Together with the complex dynamics of the onboard hydraulically actuated cranes, the rough forest terrains have posed a particular challenge in forestry automation. In this study, the feasibility of applying reinforcement learning control to forestry crane manipulators is inve...
-
We present a comprehensive comparison of hybrid Data-Driven Control (DDC) applied on a hydraulic excavator. DDC offers a state-of-the-art, high performance control based on data and expert knowledge. On the one hand, expert knowledge is complex to adapt to each unique excavator requiring substantial engineering efforts. On the other hand, purely data based control overcomes this drawback by adapti...
-
Automation of excavation tasks requires real-time trajectory planning satisfying various constraints. To guarantee both constraint feasibility and real-time trajectory re-plannability, we present an integrated framework for real-time optimization-based trajectory planning of a hydraulic excavator. The proposed framework is composed of two main modules: a global planner and a real-time local planne...
-
In this paper, we explore the problem of task-consistent path planning for printing-in-motion via Mobile Manipulators (MM). MM offer a potentially unlimited planar workspace and flexibility for print operations. However, most existing methods have only mobility to relocate an arm which then prints while stationary. In this paper we present a new fully autonomous path planning approach for mobile m...
-
In this work we propose a learning approach to high-precision robotic assembly problems. We focus on the contact-rich phase, where the assembly pieces are in close contact with each other. Unlike many learning-based approaches that heavily rely on vision or spatial tracking, our approach takes force/torque in task space as the only observation. Our training environment is robotless, as the end-eff...
-
Motion planning is a largely solved problem for robot arms with joint state feedback, but remains an area of research for sensorless manipulators such as toy robot arms and heavy equipment such as excavators and cranes. A promising approach to this problem is deep learning, which employs a pre-trained convolutional neural network to identify manipulator links and estimate joint states from a monoc...
-
The imagination of the surrounding environment based on the experience and semantic cognition has great potential to extend the limited observations to leverage the ability for mapping, collision avoidance and path planning. This paper provides a training-based algorithm for mobile robots to perform spatial imagination based on semantic cognition and evaluates the proposed method for the mapping t...
-
Depth completion recovers a dense depth map from sensor measurements. Current methods are mostly tailored for very sparse depth measurements from LiDARs in outdoor settings, while for indoor scenes Time-of-Flight (ToF) or structured light sensors are mostly used. These sensors provide semi-dense maps, with dense measurements in some regions and almost empty in others. We propose a new model that t...
-
Most autonomous vehicles (AVs) rely on LiDAR and RGB camera sensors for perception. Using these point cloud and image data, perception models based on deep neural nets (DNNs) have achieved state-of-the-art performance in 3D detection. The vulnerability of DNNs to adversarial attacks have been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in bo...
-
We propose a novel object-augmented RGB-D SLAM system that is capable of constructing a consistent object map and performing relocalisation based on centroids of objects in the map. The approach aims to overcome the view dependence of appearance-based relocalisation methods using point features or images. During the map construction, we use a pre-trained neural network to detect objects and estima...
-
The detection of planar surfaces in a point cloud is a popular technique for the extraction of drivable or walkable surfaces and for tabletop segmentation. Unfortunately, RGB-D sensors are quite noisy and provide incomplete data, which makes the extraction of surfaces more challenging. Also, it is desirable to process the point cloud data in real time, which at a rate of approximately 30 Hz, leave...
-
Dense image alignment works by minimizing the photometric error of two images since it is assumed that the illumination changes between images close in time remain the same—this is what is called the brightness constancy assumption. However, this assumption does not hold with long-term maps since illumination changes continually from day to day (morning, afternoon, evening) and is dependent on cer...
-
In the current paper we investigate the challenges of localizing walking humanoid robots using Visual SLAM (VSLAM). We propose a novel dense RGB-D SLAM framework that seamlessly integrates with the dynamic state of a humanoid, to provide real-time localization and dense mapping of its surroundings. Following the path of recent research in humanoid localization, in the current work we explore the i...
-
2D-to-3D correspondence estimation is the key step of 3D model-based image localization, and most of the existing research in this field focuses on improving the feature matching performance. Even with the best feature matching method, there are still some outliers, and thus, almost all the methods simply apply the RANSAC algorithm to select the inliers and estimate the camera pose afterwards. How...
-
A novel siamese autoencoder visual odometry system named SAEVO is proposed in this paper. SAEVO can jointly estimate the 6-DoF pose and the depth using deep neural networks trained with monocular clips only. The main idea of the proposed method is an unsupervised deep learning scheme that combines siamese networks with auto-encoder for multi-scale matching to estimate ego-motion. Also, two unsuper...
-
Event cameras are neuromorphic vision sensors that are able to capture high dynamic range with low latency in microseconds, without motion blur. Their strength lies in the unique representation of data as asynchronous events, enabling detection of scene structures less invariantly from dynamic luminance changes. However, a single event does not represent spatial information, and events must be int...
-
A coordinate system is proposed that replaces the usual three-dimensional Cartesian x, y, z position coordinates, for use in robotic localization applications. Range, azimuth, and elevation measurement models become greatly simplified, and, unlike spherical coordinates, the proposed coordinates do not suffer from the same kinematic singularities and angle wraparound. When compared to Cartesian coo...
-
Autonomous Valet Parking (AVP) in an under- ground garage is an emerging smart vehicle solution that the community believes to be solvable with close-to-market sensors. Absence of GPS signals and a high degree of self-similarity however render global visual localization in such environments a highly challenging problem. We present a novel underground parking localization method that relies on text...
-
The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which...
-
Mobile devices are increasingly expected to sup-port high-performance cyber-physical applications in small form factors, e.g., drones and rovers. However, the gap between hardware limitations of these devices and application requirements is still prohibitive – conflicting goals such as robust, accurate, and efficient execution must be managed carefully to achieve acceptable operation. In this pape...
-
The paper proposes a reliable and robust planning solution to the long range robotic navigation problem in extremely cluttered environments. A two-layer planning architecture is proposed that leverages both the environment map and the direct depth sensor information to ensure maximal information gain out of the onboard sensors. A frontier-based pose sampling technique is used with a fast marching ...
-
This study addresses the challenge of predicting the terrain traversability of off-road vehicles. When an off-road vehicle is operated on rough terrains or slopes of unconsolidated materials, it is crucial to accurately predict terrain traversability for efficient operations and to avoid critical mobility risks. However, the prediction of traversability is challenging, especially for the predictio...
-
Offroad mobile robot perception systems must be able to learn robust terrain classification models. Models built from computer vision often fail in their ability to generalize to new environments where appearance characteristics change. Sound and vibration signals from robot-terrain interaction can be used to classify the terrain from characteristics that vary less between environments. Previous w...
-
This paper studies autonomous stair climbing for quadrupedal robots with perception. Enabling quadrupeds to reliably climb staircases greatly expands their applicability in practical scenarios. For this structured task, we develop a simple yet effective perception and control framework for autonomous quadrupedal stair climbing. By exploiting the structural knowledge about the staircases, the propo...
-
Power-over-tether aircraft is an effective tool for persistent spatiotemporal monitoring of environmental phenomena. This paper presents the design and evaluation of flight trajectories for the tethered aircraft unmanned system (TAUS) sensing a dynamic temperature field. TAUS is a novel power-over-tether-based unmanned aerial system (UAS) configured for long-term, high throughput atmospheric monit...
-
The design of an industrial pipe inspection robot for 4" firewater piping inspection and mapping with some new and unreported features is presented. The robot consists of a compact 3D MEMs Lidar with non-linear scanning pattern in the form of Lissajous trajectories that produces a dense scan of inner pipe surfaces. Evaluating the surface integrals of these 3D point clouds acquired at pipe turns al...
-
Collaborative human-robot field operations rely on timely decision-making and coordination, which can be challenging for heterogeneous teams operating in large-scale deployments. In this work, we present the design of an immersive, mixed reality (MR) interface to support sense-making and situational awareness based on the data collection capabilities of both human and robotic team members. Our sol...
-
Landing a quadrotor on an inclined surface is a challenging maneuver. The final state of any inclined landing trajectory is not an equilibrium, which precludes the use of most conventional control methods. We propose a deep reinforcement learning approach to design an autonomous landing controller for inclined surfaces. Using the proximal policy optimization (PPO) algorithm with sparse rewards and...
-
We introduce Self-supervised Online Reward Shaping (SORS), which aims to improve the sample efficiency of any RL algorithm in sparse-reward environments by automatically densifying rewards. The proposed framework alternates between classification-based reward inference and policy update steps—the original sparse reward provides a self-supervisory signal for reward inference by ranking trajectories...
-
We propose a vision-based reinforcement learning (RL) approach for closed-loop trajectory generation in an arm reaching problem. Arm trajectory generation is a fundamental robotics problem which entails finding collision-free paths to move the robot’s body (e.g. arm) in order to satisfy a goal (e.g. place end-effector at a point). While classical methods typically require the model of the environm...
-
While classic control theory offers state of the art solutions in many problem scenarios, it is often desired to improve beyond the structure of such solutions and surpass their limitations. To this end, residual policy learning (RPL) offers a formulation to improve existing controllers with reinforcement learning (RL) by learning an additive "residual" to the output of a given controller. However...
-
In imitation learning from observation (IfO), a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a ...
-
Autonomous assembly has been a desired functionality of many intelligent robot systems. We study a new challenging assembly task, designing and constructing a bridge without a blueprint. In this task, the robot needs to first design a feasible bridge architecture for arbitrarily wide cliffs and then manipulate the blocks reliably to construct a stable bridge according to the proposed design. In th...
-
In this paper, we address bandwidth-limited and obstruction-prone collaborative perception, specifically in the context of multi-agent semantic segmentation. This setting presents several key challenges, including processing and ex-changing unregistered robotic swarm imagery. To be successful, solutions must effectively leverage multiple non-static and intermittently-overlapping RGB perspectives, ...
-
In recent studies, the widespread of deep learning has made many kinds of large-scale image datasets available and it has enabled to improve the performance of image-based 3-D scene reconstruction. Several studies estimate whole 3-D scenes including occluded or unseen parts consistent with the obtained partial observations by integrating prior knowledge from training datasets with them, under no c...
-
For a robot deployed in the world, it is desirable to have the ability of autonomous learning to improve its initial pre-set knowledge. We formalize this as a bootstrapped self-supervised learning problem where a system is initially bootstrapped with supervised training on a labeled dataset and we look for a self-supervised training method that can subsequently improve the system over the supervis...
-
In this paper, we aim to recognize function points of category-agnostic objects and perform object manipulation. To recognize function points of various shapes, it is necessary to train with a large amount of training data. Also, it is necessary to take into account not only visual information but also physics and interaction between objects. To solve these problems, we are working on the automati...
-
Camera anomalies like rain or dust can severely degrade image quality and its related tasks, such as localization and segmentation. In this work we address this important issue by implementing a pre-processing step that can effectively mitigate such artifacts in a real-time fashion, thus supporting the deployment of autonomous systems with limited compute capabilities. We propose a shallow generat...
-
As autonomous robots interact and navigate around real-world environments such as homes, it is useful to reliably identify and manipulate articulated objects, such as doors and cabinets. Many prior works in object articulation identification require manipulation of the object, either by the robot or a human. While recent works have addressed predicting articulation types from visual observations a...
-
Monocular depth inference has gained tremendous attention from researchers in recent years and remains as a promising replacement for expensive time-of-flight sensors, but issues with scale acquisition and implementation overhead still plague these systems. To this end, this work presents an unsupervised learning framework that is able to predict at-scale depth maps and egomotion, in addition to c...
-
Proactive inspection is essential for prediction and prevention of rolling stock component failures. The conventional process for inspecting bogies under trains presents significant challenges for inspectors who need to visually check the tight and cluttered environment. We propose a miniature multi-link climbing robot, called BogieBot, that can be deployed inside the undercarriage areas of trains...
-
To cope with the requirements on efficiency and labour-saving of the out-pipe surface grinding tasks in the wild, several proposals are revealed and discussed. The one benefiting from the characteristics of planetary gear transmission and friction actuating mechanism are expatiated. To realize full coverage of out-pipe surface, the self-rotation and revolution motions of every polishing tool (cutt...
-
The paper introduces the novel Modular Pipe Climber III with a Three-Output Open Differential (3-OOD) mechanism to eliminate slipping of the tracks due to the changing cross-sections of the pipe. This will be achieved in any orientation of the robot. Previous pipe climbers use three-wheel/track modules, each with an individual driving mechanism to achieve stable traversing. Slipping of tracks is p...
-
This paper introduces a planar tensegrity robot that walks passively and cyclically on a gentle downhill, where its gait versatility can be strengthened by applying actuation forces on the connection cables. The novelty of this work is that we design the structure of this passive robot inspired by the rimless wheel, which naturally generates cyclic locomotion. Consequently, its mathematical model ...
-
Spherical robots are typically comprised of an actuation unit enclosed by a spherical shell. Among nonholonomic systems, spherical robots offer the best maneuverability and lowest energy consumption (due to their omnidirectional movement and single contact point with the ground). This allows them to traverse rough and uneven terrains. Further, using their ability to roll on the ground, they can pr...
-
Ceiling mobile robots are anticipated for trans-porting component parts in production sites. In our previous study, we had developed a crawler-type ceiling mobile robot named "HanGrawler." In this study, we aim to realize 1 m/s and 90°/s speed movement on par with existing ground carriers, and reveal the factors contributing to the improvement in speed and stabilization. Accordingly, we develop Ha...
-
The combination of a dexterous continuum robot and magnetic resonance imaging can potentially improve surgical precision and minimize brain manipulation in a minimally invasive neurosurgical procedure. In this work, a seven degree-of-freedom (DoF) continuum neurosurgical robot was developed. The main innovation lies in the design of a safe and robust switching mechanism and gear-based quick-connec...
-
Mobile robots are traditionally developed to be reactive and avoid collisions with surrounding humans, often moving in unnatural ways without following social protocols, forcing people to behave very differently from human-human interaction rules. Humans, on the other hand, are seamlessly able to understand why they may interfere with surrounding humans and change their behavior based on their rea...
-
Over the years, many motion planning algorithms have been proposed. It is often unclear which algorithm might be best suited for a particular class of problems. The problem is compounded by the fact that algorithm performance can be highly dependent on parameter settings. This paper shows that hyperparameter optimization is an effective tool in both algorithm selection and parameter tuning over a ...
-
Natural and human-like arm motions are promising features to facilitate social understanding of humanoid robots. To this end, we integrate biophysical characteristics of human arm-motions into sampling-based motion planning. We show the generality of our method by evaluating it with multiple manipulators. Our first contribution is to introduce a set of cost functions to optimize for human-like arm...
-
Sampling-based motion planning is the predominant paradigm in many real-world robotic applications, but its performance is immensely dependent on the quality of the samples. The majority of traditional planners are inefficient as they use uninformative sampling distributions instead of exploiting structures and patterns in the problem to guide better sampling strategies. Moreover, most current lea...
-
Path planning of a tool in Minimally Invasive Surgery (MIS) can provide assistance to the surgeons by giving solutions for faster and safe tool movements during the surgery. However, the main challenge in this problem is to address non-uniform tool shape for planning that can change due to the tool’s dexterity. A typical robotic path planning approach by describing the robot’s feasible movements u...
-
This work proposes a novel distributed control framework in which a team of pursuer agents equipped with a radio jamming device cooperate in order to track and radio-jam a rogue target in 3D space, with the ultimate purpose of disrupting its communication and navigation circuitry. The target evolves in 3D space according to a stochastic dynamical model and it can appear and disappear from the surv...
-
The Visibility-based Persistent Monitoring (VPM) problem seeks to find a set of trajectories (or controllers) for robots to persistently monitor a changing environment. Each robot has a sensor, such as a camera, with a limited field-of-view that is obstructed by obstacles in the environment. The robots may need to coordinate with each other to ensure no point in the environment is left unmonitored...
-
This work presents an efficient and implementable solution to the problem of joint task allocation and path planning in a multi-UAV platform. The sensing requirement associated with the task gives rise to an uncanny variant of the traditional vehicle routing problem with coverage/sensing constraints. As is the case in several multi-robot path-planning problems, our problem reduces to an mTSP probl...
-
Multi-robot task allocation (MRTA) problems involve optimizing the allocation of robots to tasks. MRTA problems are known to be challenging when tasks require multiple robots and the team is composed of heterogeneous robots. These challenges are further exacerbated when we need to account for uncertainties encountered in the real-world. In this work, we address coalition formation in heterogeneous...
-
This paper focuses on the problem of multi-robot source-seeking, where a group of mobile sensors localizes and moves close to a single source using only local measurements. Drawing inspiration from the optimal sensor placement research, we develop an algorithm that estimates the source location while approaches the source following gradient descent steps on a loss function defined on the Fisher in...
-
We propose a decentralized approach that simultaneously allocates and decomposes high level tasks among various robots. The approach exploits HTN structures and algorithms, that are used within an auction-based allocation scheme, and aims at dealing with complex tasks with causal or temporal relations. The paper formalizes the approach, and depicts how HTN planning processes are used to estimate b...
-
This paper proposes a novel monocular teach-and-repeat navigation system with the capability of scale awareness, i.e. the absolute distance between observation and goal images. It decomposes the navigation task into a sequence of visual servoing sub-tasks to approach consecutive goal/node images in a topological map. To be specific, a novel hybrid model, named deep steering network is proposed to ...
-
The self-supervised loss formulation for jointly training depth and egomotion neural networks with monocular images is well studied and has demonstrated state-of-the-art accuracy. One of the main limitations of this approach, however, is that the depth and egomotion estimates are only determined up to an unknown scale. In this paper, we present a novel scale recovery loss that enforces consistency...
-
This article presents a novel software architecture enabling the analysis of assembly actions from fine-grained hand motions. Unlike previous works that compel humans to wear ad-hoc devices or visual markers in the human body, our approach enables users to move without additional burdens. Modules developed are able to: (i) reconstruct the 3D motions of body and hands keypoints using multi-camera s...
-
This paper presents a robust monocular visual teach-and-repeat (VT&R) navigation system for long-term operation in outdoor environments. The approach leverages deep-learned descriptors to deal with the high illumination variance of the real world. In particular, a tailored self-supervised descriptor, DarkPoint, is proposed for autonomous navigation in outdoor environments. We seamlessly integrate ...
-
We address the problem of completing per pixel dense depth map using a single RGB image and the sparse point cloud of the scene. Depth prediction from RGB image is a hard problem and while dense point clouds obtained from LiDAR sensors can be used in addition to RGB image, the cost of such sensors is a significant barrier. Having LiDAR sensors which capture sparse point clouds is a reasonable midd...
-
A mobile robot that follows behind humans in structured environments has to face the challenge of full occlusion caused by the walls when the target person makes a turn at the corridor intersections. This may result in short-term, even a permanent loss of the target from the field of view of the Human-Following Robots (HFRs). Concerning this issue, a novel side-by-side following method for HFRs is...
-
Deep Leg Tracking by Detection and Gait Analysis in 2D Range Data for Intelligent Robotic Assistants
Online human leg tracking and gait analysis are crucial functionalities for mobility assistant robots, like intelligent walkers. Usually, such walkers are equipped with various sensors for the extraction of human-related features for adaptive human-robot interaction and assistance. We treat the gait detection problem jointly, presenting a novel method for detecting and recognizing gait features fr...
-
Deep policy networks enable robots to learn behaviors to solve various real-world complex tasks in an end-to-end fashion. However, they lack transparency to provide the reasons of actions. Thus, such a black-box model often results in low reliability and disruptive actions during the deployment of the robot in practice. To enhance its transparency, it is important to explain robot behaviors by con...
-
Collecting new experience is costly in many robotic tasks, so determining how to efficiently explore in a new environment to learn as much as possible in as few trials as possible is an important problem for robotics. In this paper, we propose a method for exploring for the purpose of learning a dynamics model. Our key idea is to minimize a score given by a discriminator network as an objective fo...
-
Reasoning about potential occlusions is essential for robots to efficiently predict whether an object exists in an environment. Though existing work shows that a robot with active perception can achieve various tasks, it is still unclear if occlusion reasoning can be achieved. To answer this question, we introduce the task of robotic object existence prediction: when being asked about an object, a...
-
Active inference, a theoretical construct inspired by brain processing, is a promising alternative to control artificial agents. However, current methods do not yet scale to high-dimensional inputs in continuous control. Here we present a novel active inference torque controller for industrial arms that maintains the adaptive characteristics of previous proprioceptive approaches but also enables l...
-
In this paper, we present a capacitance-based sensor array for physical human-robot interaction (pHRI) applications that can measure the proximity, near-zero-force (NZF) contacts, and pressure between a robot and human body. The top segment including the electrodes is made of soft, stretchable materials, while the bottom segment consists of electrodes patterned from a thin copper film. The resulti...
-
The ability to detect that a grasped object is slipping from the robot gripper is a crucial skill for autonomous robotic manipulation. However, current solutions for automatic slip detection do not perform well in real-world unstructured settings, in which a wide variety of gripper-object interactions could occur. Tactile and force sensing are the most suitable sensory modalities to detect such ev...
-
Recent advances in object segmentation have demonstrated that deep neural networks excel at object segmentation for specific classes in color and depth images. However, their performance is dictated by the number of classes and objects used for training, thereby hindering generalization to never seen objects or zero-shot samples. To exacerbate the problem further, object segmentation using image f...
-
Under-actuated bionic hands have achieved tremendous popularity in many fields because of their advantages of lightweight, budget-friendly, satisfactory flexibility, and adaptability. Except for the bionic mechanical design, various anthropomorphic control strategies have been proposed and investigated in the last decades. However, due to its under-actuated characteristic, there are still many cha...
-
Currently, most table tennis robots concentrate on the canonical position control problem while ignoring the actual velocity control requirements. In this paper, we consider these requirements and propose a new table tennis robot framework. First, a tailor-made mechanical structure is designed such that the robot can reach large workspaces. Thereafter, in the table tennis trajectory prediction pro...
-
This paper develops iterative Covariance Regulation (iCR), a novel method for active exploration and mapping for a mobile robot equipped with on-board sensors. The problem is posed as optimal control over the SE(3) pose kinematics of the robot to minimize the differential entropy of the map conditioned the potential sensor observations. We introduce a differentiable field of view formulation, and ...
-
The focus of this paper is (i) to derive a suitable dynamic model for a self-reconfigurable pavement sweeping robot PANTHERA and (ii) to design a robust controller for the same to tackle uncertainties stemming from the reconfiguration process, external disturbances and from actuator saturation. To meet the first objective, an Euler-Lagrangian dynamic model is proposed to incorporate the effects of...
-
Human-robot collaboration in shared workspaces enables companies to improve efficiency and the quality of work for human workers. A novel research direction in this field is that human and robot dynamically negotiate which actions to perform. This requires the robot to permanently monitor the current task state and actions the human has performed. We envision a system that tracks the task progress...
-
Machines tend to use powerful actuators and large gearboxes to bear large loads, which are inconvenient in terms of responsiveness as they affect the duration of operations. Thus, to compensate the force to grasp an object, we propose a clamping mechanism implementing the internally-balanced magnetic unit (IB Magnet) as a force amplifier, which is a mechanism able to switch attached and detached s...
-
A prominent challenge in the field of robotics is manipulation of flexible objects. One major factor that makes this task difficult is the complex dynamics emerging from its high-dimensional structure. This argues against the use of popular optimization-based approaches, which scale poorly with system dimension (the "curse of dimensionality"). Nevertheless, almost indifferent to this complexity, h...
-
Concentric Tube Robots (CTR) have been gaining ground in minimally-invasive robotic surgeries due to their small footprint, compliance, and high dexterity. CTRs can assure safe interaction with soft tissue, provided that precise and effective motion control is achieved. Controlling the motion of CTRs is still challenging. Commonly used model-based control approaches often employ simplified geometr...
-
This study presents dynamics computation and control of a hybrid multi-link system that integrates rigid- and soft-bodies. It is a challenging problem to install a softness in a robot system, which is an important factor in human body. Softness achieved by human muscles and ligaments contributes to dynamic motion. Flexibility of a sports prosthetic leg allows a handicapped person to run. However, ...
-
This article describes an open-source hardware platform for controlling pneumatic soft robotic systems and presents the comparison of control schemes with on-off and proportional valves. The Pneumatic Soft Robotics Driver (PneuSoRD) can be used with up to one pump and pressure accumulator, 26 on-off valves, and 5 proportional valves, any of which can be operated in open or closed-loop control usin...
-
This paper proposes a novel active visuo-tactile based methodology wherein the accurate estimation of the time-invariant SE(3) pose of objects is considered for autonomous robotic manipulators. The robot equipped with tactile sensors on the gripper is guided by a vision estimate to actively explore and localize the objects in the unknown workspace. The robot is capable of reasoning over multiple p...
-
This study proposes a technique for determining completion of robotic tasks considering a characteristics that some features on force responses are common among various objects. In particular, this paper focuses on the click response because its pattern is common to many objects, although the magnitude of the response differs depending on the object. A discriminator for detecting the click respons...
-
Tactile sensing is critical for humans to perform everyday tasks. While significant progress has been made in analyzing object grasping from vision, it remains unclear how we can utilize tactile sensing to reason about and model the dynamics of hand-object interactions. In this work, we employ a high-resolution tactile glove to perform four different interactive activities on a diversified set of ...
-
The need for comprehensive telemedicine solutions is becoming increasingly relevant due to challenges associated with the ageing population, the increasing shortage of health-care providers, and, more recently, the global pandemic. Existing solutions primarily focus on, e.g., electronic medical records, audiovisual connections, and, in some cases, robotic systems with very basic capabilities. Here...
-
This paper proposes a novel dictionary learning approach to detect event anomalities using skeletal information extracted from RGBD video. The event action is represented as several latent action atoms and composed of latent spatial and temporal attributes. We aim to construct a network able to learn from few examples and also rules defined by the user. The skeleton frames are clustered by an init...
-
There has been an increasing demand to automate the non-patient care matters so that the clinical staff can focus on delivering patient care. For example, out-patients undergoing chemotherapy increases their toilet usage frequency due to the treatment. As they are undergoing chemotherapy, their output waste contains a level of chemical. This task is compulsory yet troublesome and time-consuming so...
-
In minimally invasive surgery, miniaturisation and in situ adjustable stiffness of robotic manipulators are desired features. Previous research proposed a simple and effective tendon-driven curve-joint manipulator design using a variable neutral-line mechanism, which highly satisfies both criteria. A kinematic model was developed for such a manipulator based on the geometry of the structure. Howev...
-
Human-robot collaboration (HRC) significantly extends robotic systems’ applications when working in spaces like houses, hospitals, or laboratories. However, new challenges appear during a close collaboration between humans and robots and imitating the movement of humans by robots. Learning from demonstration (LfD), or kinesthetic teaching, is a popular approach to help teach a robot human behavior...
-
3D skeleton-based motion prediction and activity recognition are two interwoven tasks in human behaviour analysis. In this work, we propose a motion context modeling methodology that provides a new way to combine the advantages of both graph convolutional neural networks and recurrent neural networks for joint human motion prediction and activity recognition. Our approach is based on using an LSTM...
-
Metastatic involvement of lymph nodes is one of the most important prognostic variables for many cancers. Several deep learning based algorithms have been developed to segment metastatic regions in pathological images to help predict prognosis. However, the training of these methods requires a large amount of annotated data, and the labeling task is an extremely time-consuming process for human an...
-
The goal of this work is to propose a way of dealing with physical interactions for collaborative robots that will ensure the safety of a human operator and improve the performance of a common task by implementing multiple robot behavior scenarios. In this scope, all collisions of a robotic arm are detected and analyzed to chooses an appropriate reaction strategy. The points of contact on the robo...
-
Intelligent robots are redefining a multitude of critical domains but are still far from being fully capable of assisting human peers in day-to-day tasks. An important requirement of collaboration is for each teammate to maintain and respect an understanding of the others’ expectations of itself. Lack of which may lead to serious issues such as loose coordination between teammates, reduced situati...
-
Extreme environments, such as search and rescue missions, defusing bombs, or exploring extraterrestrial planets, are unsafe environments for humans to be in. Robots enable humans to explore and interact in these environments through remote presence and teleoperation and virtual reality provides a medium to create immersive and easy-to-use teleoperation interfaces. However, current virtual reality ...
-
There is increasing demand for robots that provide a mode of transportation in environments in which people coexist. However, conventional mobile robots, especially those carrying people, are limited in terms of their environments and tasks. For example, wheeled robots are limited to moving on flat ground. Walking robots are limited to entertainment and so on. The originality of the present paper ...
-
Integrating real-time, complex social signal processing into robotic systems – especially in real-world, multi-party interaction situations – is a challenge faced by many in the Human-Robot Interaction (HRI) community. The difficulty is compounded by the lack of any standard model for human representation that would facilitate the development and interoperability of social perception components an...
-
Mobile robots are increasingly being deployed in public spaces such as shopping malls, airports, and urban sidewalks. Most of these robots are designed with human-aware motion planning capabilities but are not designed to communicate with pedestrians. Pedestrians encounter these robots without prior understanding of the robots’ behaviour, which can cause discomfort, confusion, and delayed social a...
-
When interacting in unstructured human environments, occasional robot failures are inevitable. When such failures occur, everyday people, rather than trained technicians, will be the first to respond. Existing natural language explanations hand-annotate contextual information from an environment to help everyday people understand robot failures. However, this methodology lacks generalizability and...
-
Effects of Conversational Contexts and Forms of Non-lexical Backchannel on User Perception of Robots
A non-lexical backchannel is known to be dependent on the conversational context, and its form can be distinguished by the social relation between the speaker and the listener in the Korean language. Thus, to investigate the effect of a non-lexical backchannel, we conducted a 2 (context: information-centric versus emotion-centric) × 3 (forms of backchannel: "ne" versus "eo" versus "eum") mixed-par...
-
All robots create consequential sound—sound produced as a result of the robot’s mechanisms—yet little work has explored how sound impacts human-robot interaction. Recent work shows that the sound of different robot mechanisms affects perceived competence, trust, human-likeness, and discomfort. However, the physical sound characteristics responsible for these perceptions have not been clearly ident...
-
The development of aerial autonomy has enabled aerial robots to fly agilely in complex environments. However, dodging fast-moving objects in flight remains a challenge, limiting the further application of unmanned aerial vehicles (UAVs). The bottleneck of solving this problem is the accurate perception of rapid dynamic objects. Recently, event cameras have shown great potential in solving this pro...
-
Recent years have witnessed the fast evolution and promising performance of the convolutional neural network (CNN)-based trackers, which aim at imitating biological visual systems. However, current CNN-based trackers can hardly generalize well to low-light scenes that are commonly lacked in the existing training set. In indistinguishable night scenarios frequently encountered in unmanned aerial ve...
-
Recently, the Siamese-based method has stood out from multitudinous tracking methods owing to its state-of-the-art (SOTA) performance. Nevertheless, due to various special challenges in UAV tracking, e.g., severe occlusion and fast motion, most existing Siamese-based trackers hardly combine superior performance with high efficiency. To this concern, in this paper, a novel attentional Siamese track...
-
This paper reports a novel optical localization method, including both the hardware design and algorithm design, to track mobile Unmanned Aerial Vehicles (UAVs). The method relies on a circle-shaped blinking LED marker installed on the UAV and uses a single Dynamic Vision Sensing (DVS) camera to sense the temporal difference of the video streams. A temporal-filtering algorithm processes the video ...
-
This work presents a semantic-aware path-planning pipeline for Unmanned Aerial Vehicles (UAVs) using deep reinforcement learning for vision-based navigation in challenging environments. Driven by the maturity of works in semantic segmentation, the proposed path-planning architecture uses reinforcement learning to distinguish the parts of the scene that are perceptually more informative using seman...
-
Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. However, they may suffer from sample inefficiency, which leads to computational burden for finding a ...
-
To capture the geometry of an object by an autonomous system, next best view (NBV) planning can be used to determine the path a robot will take. However, current NBV planning algorithms do not distinguish between objects that need to be mapped and everything else in the environment; leading to inefficient search strategies. In this paper we present a novel approach for NBV planning that accounts f...
-
Indoor exploration using mobile robots has typically focused on exploring the entire environment without considering deadlines. This paper introduces a priority-based exploration algorithm for situations with an initially unknown and dynamically assigned deadline. The goal of our exploration strategy is to determine the geometric structure of an unknown environment as rapidly as possible and retur...
-
This paper proposes a robot assembly planning method by automatically reading the graphical instruction manuals designed for humans. Essentially, the method generates an Assembly Task Sequence Graph (ATSG) by recognizing a graphical instruction manual. An ATSG is a graph describing the assembly task procedure by detecting types of parts included in the instruction images, completing the missing in...
-
For autonomous service robots to successfully perform long horizon tasks in the real world, they must act intelligently in partially observable environments. Most Task and Motion Planning approaches assume full observability of their state space, making them ineffective in stochastic and partially observable domains that reflect the uncertainties in the real world. We propose an online planning an...
-
Intelligent robots need to generate and execute plans. In order to deal with the complexity of real environments, planning makes some assumptions about the world. When executing plans, the assumptions are usually not met. Most works have focused on the negative impact of this fact and the use of replanning after execution failures. Instead, we focus on the positive impact, or opportunities to find...
-
Many manipulation tasks combine high-level discrete planning over actions with low-level motion planning over continuous robot motions. Task and motion planning (TMP) provides a powerful general framework to combine discrete and geometric reasoning, and solvers have been previously proposed for single-robot problems. Multi-robot TMP expands the range of TMP problems that can be solved but poses si...
-
Reinforcement learning has been widely applied in exploration, navigation, manipulation, and other fields. Most of the relevant techniques generate kinematic commands (e.g., move, stop, turn) for agents based on the current state information. However, recent dense action representations based research, such as spatial action maps, pointing way-points to the agent in the same domain as its observat...
-
Robotic planning problems in hybrid state and action spaces can be solved by integrated task and motion planners (TAMP) that handle the complex interaction between motion-level decisions and task-level plan feasibility. TAMP approaches rely on domain-specific symbolic operators to guide the task-level search, making planning efficient. In this work, we formalize and study the problem of operator l...
-
Hyper-redundant manipulators with slender body and high dexterity are widely applied for operations in confined spaces. Among the motion planning methods for these operations, the follow-the-leader motion controller is generally developed to avoid the obstacles, while the path trajectories are usually given. In this paper, we present an autonomous motion planner with a specialized rapidly explorin...
-
In this paper, we present a planning and control framework for dynamic, whole-body motions for dynamically stable shape-accelerating mobile manipulators. This class of robots are inherently unstable and require careful coordination between the upper and lower body to maintain balance while performing arm motion tasks. Solutions to this problem either use a complex, full-body nonlinear dynamic mode...
-
Musculoskeletal humanoids have various biomimetic advantages, and the redundant muscle arrangement allowing for variable stiffness control is one of the most important. In this study, we focus on one feature of the redundancy, which enables the humanoid to keep moving even if one of its muscles breaks, an advantage that has not been dealt with in many studies. In order to make the most of this adv...
-
This paper proposes an improved supervised autonomy framework for remote teleoperation of a quadrupedal bimanual mobile manipulator in an unknown environment, with the usage of advanced perception technology and allowing the operator to easily assist the robot with decision making for executing tasks on the fly. First, the perception system uses lightweight deep neural network-based Single Shot De...
-
In this paper, a fuzzy PID controller based on yaw angle prediction is applied to design an attitude controller for a spherical rolling robot. The robot consists of a 2-DOF pendulum located inside a spherical shell with freedom to rotate about the transversal and longitudinal axis. The proposed controller allows the robot to autonomously change its parameters to adapt to different environments bas...
-
Autonomous robotic weeding in grasslands requires robust weed segmentation. Deep learning models can provide solutions to this problem, but they need to be trained on large amounts of images, which in the case of grasslands are notoriously difficult to obtain and manually annotate. In this work we introduce Few-leaf Learning, a concept that facilitates the training of accurate weed segmentation mo...
-
Modern herbicide application in agricultural set-tings typically relies on either large scale sprayers that dispense herbicide over crops and weeds alike or portable sprayers that require labor intensive manual operation. The former method results in overuse of herbicide and reduction in crop yield while the latter is often untenable in large scale operations. This paper presents the first fully a...
-
Modern agricultural applications require knowledge about the position and size of fruits on plants. However, occlusions from leaves typically make obtaining this information difficult. We present a novel viewpoint planning approach that builds up an octree of plants with labeled regions of interest (ROIs), i.e., fruits. Our method uses this octree to sample viewpoint candidates that increase the i...
-
The problem of robotic lime picking is challenging; lime plants have dense foliage which makes it difficult for a robotic arm to grasp a lime without coming in contact with leaves. Existing approaches either do not consider leaves, or treat them as obstacles and completely avoid them, often resulting in undesirable or infeasible plans. We focus on reaching a lime in the presence of dense foliage b...
-
Studies have shown that picking techniques play an important role in determining fruit quality at harvest (e.g. bruising, stem retention, etc). When picking fruit such as apples and pears, professional pickers use active perception, incorporating both visual and tactile input about fruit orientation, stem location, and the fruit’s immediate surroundings. This combination of tactile, visual, and fo...
-
Object detection and semantic segmentation are two of the most widely adopted deep learning algorithms in agricultural applications. One of the major sources of variability in image quality acquired outdoors for such tasks is changing lighting conditions that can alter the appearance of the objects or the contents of the entire image. While transfer learning and data augmentation reduce the need f...
-
RGB-D cameras have been successfully used for indoor High-ThroughPut Phenotyping (HTPP). However, their capability and feasibility for in-field HTPP applications still need to be evaluated. To solve the problem, we evaluate the depth-ranging performances of a consumer-level RGB-D camera (RealSense D435i) under in-field scenarios. First, we focus on determining their optimal ranging areas for diffe...
-
The environment of most real-world scenarios such as malls and supermarkets changes at all times. A pre-built map that does not account for these changes becomes out-of-date easily. Therefore, it is necessary to have an up-to-date model of the environment to facilitate long-term operation of a robot. To this end, this paper presents a general lifelong simultaneous localization and mapping (SLAM) f...
-
Lifelong SLAM considers long-term operation of a robot where already mapped locations are revisited many times in changing environments. As a result, traditional graph-based SLAM approaches eventually become extremely slow due to the continuous growth of the graph and the loss of sparsity. Both problems can be addressed by a graph pruning algorithm. It carefully removes vertices and edges to keep ...
-
In the present work we address the problem of achieving a consistent estimator for SLAM. We propose a novel method capable of computing approximately consistent global uncertainties without scaling in complexity with the total size of the explored area. The method allows arbitrary selection of local areas for optimization, introducing a methodology for building a virtual prior in bounded time. The...
-
Due to the presence of ambiguities caused by sensor noise and structural similarity, simultaneous localization and mapping (SLAM) observation models are typically multimodal. The multimodal inference process can be directly dealt with by belief propagation (BP) using weighted Gaussian mixture messages, but for efficiency, a combinatorial explosion of the complexity must be suitably relaxed. In thi...
-
This paper proposes a robust initialization method for a multi-camera visual SLAM system where cameras have only a limited common field of views and inaccurate extrinsic calibration. The limited common field of views leads to only a few common features that can be matched between cameras. Inaccurate extrinsic poses, caused by vibrations or misplacement of cameras after offline calibration, make it...
-
Learning to distinguish independent moving objects from the observed optical flow with a moving camera remains challenging. In this work, we first present a novel camera pose compensation (CPC) scheme. With the help of ingenious geometric analysis, it breaks the observed optical flow into patterns that are easier to interpret for the motion segmentation network. Secondly, we further refine such co...
-
Recent advances in unsupervised learning for object detection, segmentation, and tracking hold significant promise for applications in robotics. A common approach is to frame these tasks as inference in probabilistic latent-variable models. In this paper, however, we show that the current state-of-the-art struggles with visually complex scenes such as typically encountered in robot manipulation ta...
-
3D object detection is a key component of many robotic applications such as self-driving vehicles. While many approaches rely on expensive 3D sensors such as LiDAR to produce accurate 3D estimates, methods that exploit stereo cameras have recently shown promising results at a lower cost. Existing approaches tackle this problem in two steps: first depth estimation from stereo images is performed to...
-
Data augmentation has greatly contributed to improving the performance in image recognition tasks, and a lot of related studies have been conducted. However, data augmentation on 3D point cloud data has not been much explored. 3D label has more sophisticated and rich structural information than the 2D label, so it enables more diverse and effective data augmentation. In this paper, we propose part...
-
3D object detection is a key perception component in autonomous driving. Most recent approaches are based on LiDAR sensors only or fused with cameras. Maps (e.g., High Definition Maps), a basic infrastructure for intelligent vehicles, however, have not been well exploited for boosting object detection tasks. In this paper, we propose a simple but effective framework - MapFusion to integrate the ma...
-
Spiking Neural Networks (SNN) are the so-called third generation of neural networks which attempt to more closely match the functioning of the biological brain. They inherently encode temporal data, allowing for training with less energy usage and can be extremely energy efficient when coded on neuromorphic hardware. In addition, they are well suited for tasks involving event-based sensors, which ...
-
Precision agriculture is a fast-growing field that aims at introducing affordable and effective automation into agricultural processes. Nowadays, algorithmic solutions for navigation in vineyards require expensive sensors and high computational workloads that preclude large-scale applicability of autonomous robotic platforms in real business case scenarios. From this perspective, our novel propose...
-
This article establishes the Exploration-RRT algorithm: A novel general-purpose combined exploration and path planning algorithm, based on a multi-goal Rapidly-Exploring Random Trees (RRT) framework. Exploration-RRT (ERRT) has been specifically designed for utilization in 3D exploration missions, with partially or completely unknown and unstructured environments. The novel proposed ERRT is based o...
-
Automated guided vehicles operation in human populated factory environments is a challenging task, especially when there is a demand to operate without following fixed paths defined by guide wires, magnetic tape, magnets, or transponders embedded in the floor. The paper at hand introduces a vision-based method enabling safe and autonomous operation of pallet moving vehicles that accommodate pallet...
-
In this paper, we consider periodic communication delays within the connected autonomous vehicles platoon. Periodic signals are fundamentally simple to create, and in this study we analyze whether certain amplitude or frequencies can cause instability. This is important as we discover in this study, the classical method of replacing time-varying delays with constant delays does not capture the com...
-
Autonomous driving in rainy conditions remains a big challenge. One of the issues is sensor degradation. LiDAR is commonly used in autonomous driving systems to perceive and understand surrounding environments. However, LiDAR performance can be degraded by rain, thereby influencing other system performance (e.g., perception or localization). Therefore, knowing how much degradation exists in curren...
-
Autonomous train navigation using only a low-cost MEMS IMU and a track map is considered in this paper. The approach is designed for urban rail or subway environments where GNSS measurements are unreliable or unavailable, and is intended as a baseline against which more complex sensor fusion approaches can be compared to ensure the consistency of the estimates. The estimator exploits the track mot...
-
Developing an agent in reinforcement learning (RL) that is capable of performing complex control tasks directly from high-dimensional observation such as raw pixels is a challenge as efforts still need to be made towards improving sample efficiency and generalization of RL algorithm. This paper considers a learning framework for a Curiosity Contrastive Forward Dynamics Model (CCFDM) to achieve a m...
-
Humans effortlessly solve pushing tasks in everyday life but unlocking these capabilities remains a challenge in robotics because physics models of these tasks are often inaccurate or unattainable. State-of-the-art data-driven approaches learn to compensate for these inaccuracies or replace the approximated physics models altogether. Nevertheless, approaches like Deep Q-Networks (DQNs) suffer from...
-
In various control task domains, existing controllers provide a baseline level of performance that—though possibly suboptimal—should be maintained. Reinforcement learning (RL) algorithms that rely on extensive exploration of the state and action space can be used to optimize a control policy. However, fully exploratory RL algorithms may decrease performance below a baseline level during training. ...
-
The DARPA subterranean challenge requires teams of robots to traverse difficult and diverse underground environments. Traversing small gaps is one of the challenging scenarios that robots encounter. Imperfect sensor information makes it difficult for classical navigation methods, where behaviours require significant manual fine tuning. In this paper we present a deep reinforcement learning method ...
-
The use of reinforcement learning (RL) has led to huge advancements in the field of robotics. However data scarcity, brittle convergence and the gap between simulation & real world environments, mean that most common RL approaches are subject to over fitting and fail to generalise to unseen environments. Hardware agnostic policies would mitigate this by allowing a single network to operate in a va...
-
Motion planning under uncertainty is one of the main challenges in developing autonomous driving vehicles. In this work, we focus on the uncertainty in sensing and perception, resulted from a limited field of view, occlusions, and sensing range. This problem is often tackled by considering hypothetical hidden objects in occluded areas or beyond the sensing range to guarantee passive safety. Howeve...
-
The applicability of reinforcement learning (RL) algorithms in real-world domains often requires adherence to safety constraints, a need difficult to address given the asymptotic nature of the classic RL optimization objective. In contrast to the traditional RL objective, safe exploration considers the maximization of expected returns under safety constraints expressed in expected cost returns. We...
-
We propose the ViNet architecture for audio-visual saliency prediction. ViNet is a fully convolutional encoder-decoder architecture. The encoder uses visual features from a network trained for action recognition, and the decoder infers a saliency map via trilinear interpolation and 3D convolutions, combining features from multiple hierarchies. The overall architecture of ViNet is conceptually simp...
-
Visual Odometry (VO) is used in many applications including robotics and autonomous systems. However, traditional approaches based on feature matching are computationally expensive and do not directly address failure cases, instead relying on heuristic methods to detect failure. In this work, we propose a deep learning-based VO model to efficiently estimate 6-DoF poses, as well as a confidence mod...
-
In recent years, unsupervised deep learning approaches have received significant attention to estimating the depth and visual odometry (VO) from unlabelled monocular image sequences. However, their performance is limited in challenging environments due to perceptual degradation, occlusions, and rapid motions. Moreover, the existing unsupervised methods suffer from the lack of scale-consistency con...
-
Volumetric deep learning approach towards stereo matching aggregates a cost volume computed from input left and right images using 3D convolutions. Recent works showed that utilization of extracted image features and a spatially varying cost volume aggregation complements 3D convolutions. However, existing methods with spatially varying operations are complex, cost considerable computation time, a...
-
Attention is an important component of modern deep learning. However, less emphasis has been put on its inverse: ignoring distraction. Our daily lives require us to explicitly avoid giving attention to salient visual features that confound the task we are trying to accomplish. This visual prioritisation allows us to concentrate on important tasks while ignoring visual distractors.In this work, we ...
-
Outdoor scene completion is a challenging issue in 3D scene understanding, which plays an important role in intelligent robotics and autonomous driving. Due to the sparsity of LiDAR acquisition, it is far more complex for 3D scene completion and semantic segmentation. Since semantic features can provide constraints and semantic priors for completion tasks, the relationship between them is worth ex...
-
Learning 3D object reconstruction from a single RGB image is a fundamental and extremely challenging problem for robots. As acquiring labeled 3D shape representations for real-world data is time-consuming and expensive, synthetic image-shape pairs are widely used for 3D reconstruction. However, the models trained on synthetic data set did not perform equally well on real-world images. The existing...
-
Grasping in cluttered scenes has always been a great challenge for robots, due to the requirement of the ability to well understand the scene and object information. Previous works usually assume that the geometry information of the objects is available, or utilize a step-wise, multi-stage strategy to predict the feasible 6-DoF grasp poses. In this work, we propose to formalize the 6-DoF grasp pos...
-
We propose a 6D pose estimation method for an object from a single RGB image for a robotic grasping task. Many approaches estimate pose parameters from images taken from other viewpoints and use deep learning to achieve high accuracy. However, most of these methods are not robust to changes in object texture, and there is a possibility that the correct pose cannot be estimated by only one-time inf...
-
Grasping objects in cluttered scenarios is a challenging task in robotics. Performing pre-grasp actions such as pushing and shifting to scatter objects is a way to reduce clutter. Based on deep reinforcement learning, we propose a Fast-Learning Grasping (FLG) framework, that can integrate pre-grasping actions along with grasping to pick up objects from cluttered scenarios with reduced real-world t...
-
The dominant way to control a robot manipulator uses hand-crafted differential equations leveraging some form of inverse kinematics / dynamics. We propose a simple, versatile joint-level controller that dispenses with differential equations entirely. A deep neural network, trained via model-free reinforcement learning, is used to map from task space to joint space. Experiments show the method capa...
-
Path planning for a robot is one of the major problems in the area of robotics. When a robot is given a task in the form of a Linear Temporal Logic (LTL) specification such that the task needs to be carried out repetitively, we want the robot to follow the shortest cyclic path so that the number of times the robot completes the mission within a given duration gets maximized. In this paper, we addr...
-
In many multi-robot applications, mobile worker robots are often engaged in performing some tasks repetitively by following pre-computed trajectories. As these robots are battery-powered, they need to get recharged at regular intervals. We envision that, in the future, a few mobile recharger robots will be employed to supply charge to the energy-deficient worker robots recurrently to keep the over...
-
Traditional geometric mechanics models used in locomotion analysis rely heavily on systems having symmetry in SE(2) (i.e., the dynamics and constraints are invariant with respect to a system’s position and orientation) to simplify motion planning. As a result, the symmetry assumption prevents locomotion analysis on non-flat surfaces because the system dynamics may vary as a function of position an...
-
Path planning and smooth trajectory generation are critical capabilities for efficient navigation of mobile robots operating in challenging and cluttered environments. For real time and autonomous operations of mobile robots, intelligent algorithms, efficient and light-weight compute, and smooth trajectory are key components. In this work, we propose an intelligent, probabilistic Gaussian mixture ...
-
Sampling-based motion planning algorithms such as RRT* are well-known for their ability to quickly find an initial solution and then converge to the optimal solution asymptotically as the number of samples tends to infinity. However, the convergence rate can be slow for high-dimensional planning problems, particularly for dynamical systems where the sampling space is not just the configuration spa...
-
We propose a two-phase risk-averse architecture for controlling stochastic nonlinear robotic systems. We present Risk-Averse Nonlinear Steering RRT* (RANS-RRT*) as an RRT* variant that incorporates nonlinear dynamics by solving a nonlinear program (NLP) and accounts for risk by approximating the state distribution and performing a distributionally robust (DR) collision check to promote safe planni...
-
Recent work has demonstrated real-time mapping and reconstruction from dense perception, while motion planning based on distance fields has been shown to achieve fast, collision-free motion synthesis with good convergence properties. However, demonstration of a fully integrated system that can safely re-plan in unknown environments, in the presence of static and dynamic obstacles, has remained an ...
-
Robotic In-space assembly (ISA) is the next step to building larger and more permanent structures in orbit. Robotic ISA offers a unique opportunity for engineers to design the robotic system and the structure at the same time. ISA structures can be optimized to minimize weight or the number of pieces but these decisions have large impacts on the complexity of the robotic system. This impact goes b...
-
Space robots have played an essential role in space junk removal. Compared with traditional model-based methods, model-free reinforcement learning methods are promising in tackling space capture missions, which is challenging due to the dynamic singular problem and measuring errors of dynamics parameters. Nevertheless, current research mostly focus on the single-target environment. In this paper, ...
-
Disentangling two or more cables often requires many steps to remove crossings between and within cables. We formalize the problem of disentangling multiple cables and present an algorithm, Iterative Reduction Of Non-planar Multiple cAble kNots (IRON-MAN), that outputs robot actions to remove crossings from multi-cable knotted structures. IRON-MAN uses a learned perception system inspired by prior...
-
This work introduces an approach for automatic hair combing by a lightweight robot. For people living with limited mobility, dexterity, or chronic fatigue, combing hair is often a difficult task that negatively impacts personal routines. We propose a modular system for enabling general robot manipulators to assist with a hair-combing task. The system consists of three main components. The first co...
-
In the growing elderly population globally, patients with severe movement disorders account for a large proportion. Moreover, the development of intelligent service equipment can better assist them in their daily. This paper proposes a new service robot control system. The brain-computer interface (BCI) based on Steady-State Visual Evoked Potentials (SSVEP) is used to acquire and process electroen...
-
Visual servoing control schemes, such as Image-Based (IBVS), Pose Based (PBVS) or Hybrid-Based (HBVS) have been extensively developed over the last decades making possible their uses in a large number of applications. It is well-known that the main problems to be handled concern the presence of local minima or singularities, the visibility constraint, the joint limits, etc. Recently, Model Predict...
-
In this paper, we propose a robotic ultrasound imaging method that scans the breast in two separate phases to acquire high-quality ultrasound images. Our proposed system controls five Degrees of Freedom (DoFs) of the robot that hold an ultrasound probe to perform precise scanning. This system finds the desired trajectory based on geometrical analysis of the target inside the breast in a pre-scan p...
-
Shape control has become a prominent research field as it enables the automation of tasks in many applications. Overall, deforming an object to a desired target shape by using few grippers is a major challenge. The limited information about the object dynamics, the need to combine small and large deformations in order to achieve certain target shapes and the non-linear nature of most deformable ob...
-
Recent data-driven approaches to visual servoing have shown improved performances over classical methods due to precise feature matching and depth estimation. Some recent servoing approaches use a model predictive control (MPC) framework which generalise well to novel environments and are capable of incorporating dynamic constraints, but are computationally intractable in real-time, making it diff...
-
The key challenge of unsupervised vehicle re-identification (Re-ID) is learning discriminative features from unlabelled vehicle images. Numerous methods using domain adaptation have achieved outstanding performance, but those methods still need a labelled dataset as a source domain. This paper addresses an unsupervised vehicle Re-ID method, which no need any types of a labelled dataset, through a ...
-
This paper proposes a method to extract the position and pose of vehicles in the 3D world from a single traffic camera. Most previous monocular 3D vehicle detection algorithms focused on cameras on vehicles from the perspective of a driver, and assumed known intrinsic and extrinsic calibration. On the contrary, this paper focuses on the same task using uncalibrated monocular traffic cameras. We ob...
-
Autonomous driving in a pedestrian zone is a challenging task. Technische Universitaet Kaiserslautern (TUK) is currently researching autonomous driving on the university campus for elderly or disabled people. This paper presents a novel campus dataset from the TUK campus, recorded over the span of one year for an autonomous bus project. John Deere’s Gator X855D is used for the work which is equipp...
-
Existing vision systems for autonomous driving or robots are sensitive to waterdrops adhered to windows or camera lenses. Most recent waterdrop removal approaches take a single image as input and often fail to recover the missing content behind waterdrops faithfully. Thus, we propose a learning-based model for waterdrop removal with stereo images. To better detect and remove waterdrops from stereo...
-
A commonly-used representation for motion prediction of actors is a sequence of waypoints (comprising positions and orientations) for each actor at discrete future time-points. While regressing waypoints is simple and flexible, it can exhibit unrealistic higher-order derivatives (such as acceleration) and approximation errors at intermediate time steps. To address this issue we propose a general r...
-
Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning perception algorithms trained solely on synthetic data encounter serious performance drops when they are tested on real data. Such performance drops are commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods that have been ap...
-
Detecting and tracking objects in 3D scenes play crucial roles in autonomous driving. Successfully recognizing objects through space and time hinges on a strong detector and a reliable association scheme. Recent 3D detection and tracking approaches widely represent objects as points when associating detection results with trajectories. Despite the demonstrated success, these approaches do not full...
-
We study the problem of pursuit-evasion for a single pursuer and an evader in polygonal environments where the players have visibility constraints. The pursuer is tasked with catching the evader as quickly as possible while the evader tries to avoid being captured. We formalize this problem as a zero-sum game where the players have private observations and conflicting objectives.One of the challen...
-
Object picking with two-fingered grippers is widely used in practice. However, the deformability and slipperiness of the target object still remain a challenge, and not resolving them might lead to breaking or dropping of the grasped objects. To prevent such instances, tactile sensing plays an important role because it can directly detect even the subtle changes that occur during grasping. Mechano...
-
We present a straightforward and efficient way to control unstable robotic systems using an estimated dynamics model. Specifically, we show how to exploit the differentiability of Gaussian Processes to create a state-dependent linearized approximation of the true continuous dynamics that can be integrated with model predictive control. Our approach is compatible with most Gaussian process approach...
-
The control of a robot for manipulation tasks generally relies on object detection and pose estimation. An attractive alternative is to learn control policies directly from raw input data. However, this approach is time-consuming and expensive since learning the policy requires many trials with robot actions in the physical environment. To reduce the training cost, the policy can be learned in sim...
-
Engineers and scientists often rely on their intuition and experience when designing soft robotic systems. The development of performant controllers and motion plans for these systems commonly requires time-consuming iterations on hardware. We present the SoMo (Soft Motion) toolkit, a software framework that makes it easy to instantiate and control typical continuum manipulators in an accurate phy...
-
This paper puts forward a novel sensor-less fault detection method with only task errors feedback and applies it to visual servoing tasks of soft robot manipulators. The method is developed by introducing a suitably designed endogenous accessory signal (EAS). On the one hand, EAS transforms the change of jacobian matrix led by faults into the change of task errors, which enables the fault to be di...
-
Autonomous grasping is an important factor for robots physically interacting with the environment and executing versatile tasks. However, a universally applicable, cost-effective, and rapidly deployable autonomous grasping approach is still limited by those target objects with fuzzy-depth information. Examples are transparent, specular, flat, and small objects whose depth is difficult to be accura...
-
Deformable solid objects such as clay or dough are prevalent in industrial and home environments. However, robotic manipulation of such objects has largely remained unexplored in literature due to the high complexity involved in representing and modeling their deformation. This work addresses the problem of shaping elasto-plastic dough by proposing to use a novel elastic end-effector to roll dough...
-
Humans can intuitively grasp objects of different shape and weight. Throughout the grasp execution they control and coordinate the grasp forces at all contact points between the hand and the object to achieve a stable grasp. Dexterous grasping with humanoid hands relies on the perfect coordination between grasp posture and force balance at the contact points in a high dimensional space and remains...
-
Grasping is one of the most fundamental problems in robotic manipulation. In recent years, with the development of data-driven methods, reinforcement learning has been used in solving robotic grasping problems. However, grasping is a long-horizon and sparse reward task, whose natural reward only appears when the task is successfully achieved. Therefore, it brings great challenges to the deployment...
-
Engineering humanoid robot hands with the ability to dexterously grasp objects of different sizes, shapes, mate-rial properties and weights requires sophisticated tactile sensing and intelligent controllers able to interpret sensory information and adapt contact forces with the object to achieve a stable and safe grasp. In this paper, we present a new soft humanoid hand equipped with a multimodal ...
-
This paper presents a spectral correlation-based method (SpectGRASP) for robotic grasping of arbitrarily shaped, unknown objects. Given a point cloud of an object, SpectGRASP extracts contact points on the object’s surface matching the hand configuration. It neither requires offline training nor a-priori object models. We propose a novel Binary Extended Gaussian Image (BEGI), which represents the ...
-
We propose a new approach to investigate and quantify dynamic grasp performance. Oftentimes, existing approaches to grasp analysis assess a grasp’s quality in a static situation. We build upon such considerations to also account for the dynamic nature of most grasp operations. In particular, these typically do not, in practice, occur in a static setting. Robotic grasping is indeed commonly involve...
-
We propose an autonomous grasping pipeline that relies on geometric information extracted from segmented point cloud data. This is in contrast to many recent approaches leveraging deep learning and thus relying on a rather large amount of training samples. We argue that the proposed geometric approach facilitates task-level planning as the shape, size, and symmetry of objects can be directly taken...
-
Evaluating a grasp generated by a set of hand-object contact locations is a key component of many grasp planning algorithms. In this paper, we present a novel second-order cone program (SOCP) based optimization formulation for evaluating a grasps’ ability to apply wrenches to generate a linear motion along a given direction and/or an angular motion about the given direction. Our quality measure ca...
-
We present a novel application of Learning from Demonstration to realize a fully autonomous bi-manual surgical suturing task, including needle pick up, insertion, re-grasping, extraction and hand-over. Surgical action primitives are learned from a single human demonstration and encoded into an action library from which they are pulled to compose more elaborate tasks at planning/execution time. The...
-
Deep Reinforcement Learning (DRL) is a viable solution for automating repetitive surgical subtasks due to its ability to learn complex behaviours in a dynamic environment. This task automation could lead to reduced surgeon’s cognitive workload, increased precision in critical aspects of the surgery, and fewer patient-related complications. However, current DRL methods do not guarantee any safety c...
-
Classic Minimally Invasive Surgery (MIS) is an ergonomic burden for assistants and surgeons. The former need to adopt uncomfortable positions for hours while holding a camera to track the latter’s gestures inside the patient. This incurs assistant’s muscle fatigue which can lead to tremor or drift of the video feedback. A backdrivable robotic holder can be attached to this device in order to compe...
-
In robotically assisted surgical procedures the surgical tool is usually inserted in the patient’s body through a small incision, which acts as a constraint for the motion of the robot, known as remote center of Motion (RCM). The location of the insertion point on the patient’s body has huge effects on the performances of the surgical robot. In this work we present an offline pre-operative framewo...
-
Our research team has been developing biped robots based on the nature of passive dynamics. We aim to both investigate the effect of wobbling mass and apply the findings to biped robots to achieve high-performance running. We used an elastically supported wobbling mass in the trunk of biped robots because humans utilize their elastic organs in the upper body and arms to improve running performance...
-
Human-robot collaboration is an essential re-search topic in artificial intelligence (AI), enabling researchers to devise cognitive AI systems and affords an intuitive means for users to interact with the robot. Of note, communication plays a central role. To date, prior studies in embodied agent navigation have only demonstrated that human languages facilitate communication by instructions in nat...
-
The ongoing global healthcare crisis has amplified the need for automation of manual tasks in several industries and service sectors. Simple household tasks such as tidying and cleaning are in high demand, with only a few robotic platforms capable of performing them due to the mobility, workspace, and dexterity requirements. This work presents ARoA, an autonomous robotic assistant that can execute...
-
Vision aided dynamic exploration on bipedal robots poses an integrated challenge for perception and control. Rapid walking motions as well as the vibrations caused by the landing-foot contact-force introduce critical uncertainty in the visual-inertial system, which can cause the robot to misplace its feet placing on complex terrains and even fall over. In this paper, we present a streamlined integ...
-
We developed a method to enable a humanoid robot to carry stacked boxes. In order to transport objects efficiently, it is necessary to carry multiple objects at the same time, but in previous studies, humanoid robots have only been able to carry a single object. When a humanoid robot carries stacked boxes, the robot drops boxes when the positional relationship between un-grasped boxes changes. The...
-
Brain-computer interface (BCI)-based robotic telepresence provides an opportunity for people with disabilities to control robots remotely without any actual physical movement. However, traditional BCI systems usually require the user to select the navigation direction from visual stimuli in a fixed background, which makes it difficult to control the robot in a dynamic environment during the locomo...
-
The paper presents a shared control architecture for robotic systems commanded through a motor imagery based Brain-Computer Interface (BCI). The overall system is aimed at assisting people to perform teleoperated manipulation tasks, and it is structured so as to leave different levels of autonomy to the user depending on the actual stage of the task execution. The low-level part of the shared cont...
-
With an increasing number of robotic and prosthetic devices, there is a need for intuitive interfaces which enable the user to efficiently interact with them. The conventional interfaces are generally bulky and unsuitable for dynamic and unstructured environments. An alternative to the traditional interfaces is the class of Muscle-Machine Interfaces (MuMIs) that allow the user to have an embodied ...
-
Manifold Trial Selection to Reduce Negative Transfer in Motor Imagery-based Brain–Computer Interface
A major challenge in electroencephalogram (EEG) signal classification is that the EEG signals recorded from different subjects are drawn from different distributions. When the unlabeled EEG data of the new subject arrive, called target domain, classifying them with a classifier trained on prerecorded EEG data of other subjects, called source domain, will greatly decrease the classification accurac...
-
In this work, we present a definition of a neurointerface architecture combined from two main parts (1) neuroport (a hardware device) that implements a neuro protocol, generated and managed by a (2) neuroterminal (a software). The proposed architecture was created by analogy with OSI network architecture. We also present the neuroterminal as an oscillator motif real-time neurosimulation and result...
-
Kinematic information obtained directly from the skeletal model has been useful for jumping action recognition. Current research focuses on dynamic analysis based on the video stream. Although skeletal data can accurately capture the high-level information of human action, it ignores the brain’s pre-execution command information, which plays a crucial role in identifying jumping action. Therefore,...
-
Learning from human feedback using event-related electroencephalography (EEG) signals has attracted extensive attention recently owing to their intuitive communication ability by decoding user intentions. However, this approach requires users to perform specified tasks and their success or failure. In addition, the amount of attention needed for decision-making increases with the task difficulty, ...
-
Today, many inspection domains utilize the benefits of drones to monitor and inspect infrastructure in an efficient manner. The energy grid is challenged by frequent and thorough inspection to stay operational. So far, drones have already been introduced to solve this challenge. However, the inspection drone still requires manual control and subsequent human examination of the captured photos and ...
-
Fast and robust gate perception is of great importance in autonomous drone racing. We propose a convolutional neural network-based gate detector (GateNet1) that concurrently detects gate’s center, distance, and orientation with respect to the drone using only images from a single fish-eye RGB camera. GateNet achieves a high inference rate (up to 60 Hz) on an onboard processor (Jetson TX2). Moreove...
-
A Flying-type cable climbing robot, CCRobot-IV-F, is presented in this paper. It is a climbing precursor of the fourth version of CCRobot, designed to surpass the abilities of previous robots with high climbing speed and obstacle-crossing capability. CCRobot-IV-F weighs less than 10 kg and a no-load speed of up to 4.5 m/s, which significantly exceeds that of other climbing robots. A dynamic model ...
-
Exploring an unknown environment without colliding with obstacles is one of the essentials of autonomous vehicles to perform diverse missions such as structural inspections, rescues, deliveries, and so forth. Therefore, unmanned aerial vehicles (UAVS), which are fast, agile, and have high degrees of freedom, have been widely used. However, previous approaches have two limitations: a) First, they m...
-
In this paper, an autonomous aerial manipulation task of pulling a plug out of an electric socket is conducted, where maintaining the stability and robustness is challenging due to sudden disappearance of a large interaction force. The abrupt change in the dynamical model before and after the separation of the plug can cause destabilization or mission failure. To accomplish aerial plug-pulling, we...
-
For conventional aerial manipulators, the robotic arm is rigidly attached to the quadrotor. Consequently, the maneuver of the quadrotor will affect the motion of the robotic arm when it is used for tasks such as inspection. In this paper, we propose a novel aerial manipulator with a self-locking gimbal system which can switch between motion coupled and decoupled mode. Furthermore, a dynamic gravit...
-
Rigid grippers used in existing aerial manipulators require precise positioning to achieve successful grasps and transmit large contact forces that may destabilize the drone. This limits the speed during grasping and prevents "dynamic grasping", where the drone attempts to grasp an object while moving. On the other hand, biological systems (e.g., birds) rely on compliant and soft parts to dampen c...
-
Cursor-based tele-operation interfaces for manipulators can enable widely available and accessible control of robots to make many near term applications possible. However, their efficiency is restricted by the challenge of controlling 6 Degrees-of-Freedom (DoF) with 2D input from the cursor. Existing interfaces make use of different strategies to tackle this challenge, including viewpoint constrai...
-
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patient, attached to consoles where their heads must be fixed and their arms can only move in a limited sp...
-
The fast growth of communication technologies such as 5G provides high bandwidth and low latency wireless internet access. This enables both high definition video stream and real-time robot commands transmitted between robots and operators in the context of telepresence and teleoperation. Although there has been substantial research to establish algorithms that convert images to robot motions and ...
-
This paper considers how the motions of a telepresence robot moving autonomously affect a person immersed in the robot through a head-mounted display. In particular, we explore the preference, comfort, and naturalness of elements of piecewise linear paths compared to the same elements on a smooth path. In a user study, thirty-six subjects watched panoramic videos of three different paths through a...
-
Prolonged remote tele-locomanipulation of multi degrees-of-freedom mobile manipulators requires a compromise between the system’s performance and the operator’s ergonomics. Neglecting this demand can significantly affect either the task completion or the level of comfort to achieve it. However, the simultaneous consideration of these key factors has received less attention in the literature. To re...
-
This paper proposes an admittance controller-based teleoperation system for contact-rich tasks. Based on the analysis of the motivating task (deposited iron lump removal task in the steel mill), the system concept is focused on the practical aspects of the system, and various components are combined to enhance the safety of the teleoperation of the robot. To connect the large inertia difference be...
-
This paper addresses the problem of identifying whether/how a black-box autonomous system has regressed in performance when compared to previous versions. The approach analyzes performance datasets (typically gathered through simulation-based testing) and automatically extracts test parameter clusters of predicted performance regression. First, surrogate modeling with quantile random forests is us...
-
UAVs are deployed in various applications including disaster search-and-rescue, precision agriculture, law enforcement and first response. As UAV software systems grow more complex, the drawbacks of developing them in low-level languages become more pronounced. For example, the lack of memory safety in C implies poor isolation between the UAV autopilot and other concurrent tasks. As a result, the ...
-
Improving robot systems via newly-developed sensing devices, control algorithms, or state estimators in order to obtain safe and efficient human-robot interaction as well as tactile manipulation skills requires standardized performance measurement protocols for objective comparison. Common protocols to evaluate robot motion performance are currently defined in EN ISO 9283:1998. For tactile and saf...
-
In many situations it is either impossible or impractical to develop and evaluate agents entirely on the target domain on which they will be deployed. This is particularly true in robotics, where doing experiments on hardware is much more arduous than in simulation. This has become arguably more so in the case of learning-based agents. To this end, considerable recent effort has been devoted to de...
-
This paper presents a learning-based Model Predictive Control (MPC) methodology incorporating nonlinear predictions with robotics applications in mind. In particular, MPC is combined with feedback linearization for computational efficiency and Gaussian Process Regression (GPR) is used to model unknown system dynamics and nonlinearities. In this method, MPC predicts future states by leveraging a GP...
-
This paper introduces HOPPY, an open-source, low-cost, robust, and modular kit for robotics education. The robot dynamically hops around a rotating gantry with a fixed base. The kit is intended to lower the entry barrier for studying dynamic robots and legged locomotion with real systems. It bridges the theoretical content of fundamental robotic courses with real dynamic robots by facilitating and...
-
Underwater visual localization is an essential technique for the autonomous operation of underwater robots. However, the unique underwater image characteristics, including refraction, sparse features, and severe noise, pose an enormous challenge to it. For addressing these issues, this paper proposes an open-source fiducial-based underwater stereo visual-inertial localization method under the exte...
-
The lack of GPS signal in the underwater environment poses limitations in terms of localization and navigation of mobile robots. Strategies based on acoustic localization systems are employed to improve underwater navigation. In this paper we describe a first step towards the development of a marine system of systems involving autonomous mobile nodes. The approach relies on communication networkin...
-
Accurate localization is one of the biggest challenges in underwater robotics. The primary reasons behind that are unavailability of satellite-based positioning below the surface, and lack of clear features in natural water bodies for visually aided localization. As such, the common method of choice for external position referencing in underwater robots is the use of acoustic signals for computing...
-
The increasing interest for autonomous ships has motivated research in numerous areas. One such area is the safe navigation through ice infested waters, for which a sensor instrumentation and automated process are proposed for near-field, sea-ice 3D scanning and mapping using a ship mounted LiDAR, with attitude compensation from inertial and satellite positioning sensors. Data were collected both ...
-
Estimating ocean flow fields in 3D is a critical step in enabling the reliable operation of underwater gliders and other small, low-powered autonomous marine vehicles. Existing methods produce depth-averaged 2D layers arranged at discrete vertical intervals, but this type of estimation can lead to severe navigation errors. Based on the observation that real-world ocean currents exhibit relatively ...
-
One of the main challenges in underwater robot localization is the scarcity of external positioning references. Therefore, accurate inertial localization in between external position updates is crucial for applications such as underwater environmental sampling. In this paper, we present a framework for estimating kinematic and dynamic model parameters used for inertial navigation. Accurate values ...
-
Robots and autonomous systems need to know where they are within a map to navigate effectively. Thus, simultaneous localization and mapping or SLAM is a common building block of robot navigation systems. When building a map via a SLAM system, robots need to re-recognize places to find loop closure and reduce the odometry drift. Image-based place recognition received a lot of attention in computer ...
-
Simultaneous Localization and Mapping (SLAM) has wide robotic applications such as autonomous driving and unmanned aerial vehicles. Both computational efficiency and localization accuracy are of great importance towards a good SLAM system. Existing works on LiDAR based SLAM often formulate the problem as two modules: scan-to-scan match and scan-to-map refinement. Both modules are solved by iterati...
-
Simultaneous localization and mapping (SLAM) is an essential technique for autonomous driving. Recently, combining image recognition technology to generate semantically meaningful maps has become a new trend in visual SLAM research. However, in the field of LiDAR SLAM, this potential has not been fully explored. We propose a novel object-level SLAM system using 3D LiDARs for autonomous vehicles. W...
-
Simultaneous Localization and Mapping (SLAM) is considered to be an essential capability for intelligent vehicles and mobile robots. However, most of the current lidar SLAM approaches are based on the assumption of a static environment. Hence the localization in a dynamic environment with multiple moving objects is actually unreliable. The paper proposes a dynamic SLAM framework RF-LIO, building o...
-
With the democratization of 3D LiDAR sensors, precise LiDAR odometries and SLAM are in high demand. New methods regularly appear, proposing solutions ranging from small variations in classical algorithms to radically new paradigms based on deep learning. Yet it is often difficult to compare these methods, notably due to the few datasets on which the methods can be evaluated and compared. Furthermo...
-
Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordinates of a next viewpoint to discriminate the object...
-
Among prerequisites for a synthetic agent to inter-act with dynamic scenes, the ability to identify independently moving objects is specifically important. From an application perspective, nevertheless, standard cameras may deteriorate remarkably under aggressive motion and challenging illumination conditions. In contrast, event-based cameras, as a category of novel biologically inspired sensors, ...
-
Projecting the point cloud on the 2D spherical range image transforms the LiDAR semantic segmentation to a 2D segmentation task on the range image. However, the LiDAR range image is still naturally different from the regular 2D RGB image; for example, each position on the range image encodes the unique geometry information. In this paper, we propose a new projection-based LiDAR semantic segmentati...
-
Vision-based 3D object detection is a research focus in the field of autonomous driving system. While recently proposed pseudo-LiDAR is a promising solution, its performance is severely restricted by the image-based depth estimator, leading to a considerable performance gap against the LiDAR-based counterparts. In this paper, substantial advances are developed along an orthogonal direction to the ...
-
The RGB-Thermal (RGB-T) information for semantic segmentation has been extensively explored in recent years. However, most existing RGB-T semantic segmentation usually compromises spatial resolution to achieve real-time inference speed, which leads to poor performance. To better extract detail spatial information, we propose a two-stage Feature-Enhanced Attention Network (FEANet) for the RGB-T sem...
-
Object detection plays a deep role in visual systems by identifying instances for downstream algorithms. In industrial scenarios, however, a slight change in manufacturing systems would lead to costly data re-collection and human annotation processes to re-train models. Existing solutions such as semi-supervised and few-shot methods either rely on numerous human annotations or suffer low performan...
-
Marine vessels are subject to high wear and tear due to the conditions they operate in. To reduce risk of failure during operation, vessels are inspected periodically every five years. These inspections are prone to high subjectiveness that makes them hard to reproduce for the shipping owners. The purpose of this paper is to present a regressor to a Faster R-CNN network that can help alleviate som...
-
We compare a state-of-the-art deep image retrieval and a deep place recognition method for place recognition using LiDAR data. Place recognition aims to detect previously visited locations and thus provides an important tool for navigation, mapping, and localisation. Experimental comparisons are conducted using challenging outdoor and indoor datasets, Oxford Radar RobotCar and COLD, in the "long-t...
-
In this paper, we present a novel learnable and continuous Monte-Carlo Tree Search method, named as KB-Tree, for motion planning in autonomous driving. The proposed method utilizes an asymptotical PUCB based on Kernel Regression (KR-AUCB) as a novel UCB variant, to improve the exploitation and exploration performance. In addition, we further optimize the sampling in continuous space by adapting Ba...
-
Numerous autonomous navigation systems have been proposed, so far, for use in walking environments. Of these, systems that do not rely on high-definition maps and precise localization are cheaper to maintain and easier to implement in unknown outdoor environments. In these systems, road-following navigation using road boundaries is commonly used. In outdoor environments, however, the road boundary...
-
In autonomous navigation, sensors suffer from massive occlusion in cluttered environments, leaving a significant amount of space unknown. In practice, treating the unknown space in optimistic or pessimistic ways both set limitations on planning performance. Therefore, aggressiveness and safety cannot be satisfied at the same time. Mimicking human behavior, in this paper, we propose a method based ...
-
Widespread adoption of autonomous vehicles will not become a reality until solutions are developed that enable these intelligent agents to co-exist with humans. This includes safely and efficiently interacting with human-driven vehicles, especially in both conflictive and competitive scenarios. We build up on the prior work on socially-aware navigation and borrow the concept of social value orient...
-
Obstacle detection and avoidance plays a crucial role in autonomous navigation of unmanned ground vehicles. This becomes more challenging in off-road environments due to the higher probability of finding negative obstacles (e.g., holes, ditches, trenches, etc.) compared with on-road environments. One approach to solve this problem is to avoid the candidate path with a negative obstacle, but in off...
-
For autonomous vehicles integrating onto road-ways with human traffic participants, it requires understanding and adapting to the participants’ intention by responding in predictable ways. This paper proposes a reinforcement learning based negotiation-aware motion planning framework, which adopts RL to adjust the driving style of the planner by dynamically modifying the prediction horizon length o...
-
When it comes to the control system of quadruped robots, deep reinforcement learning (DRL) is considered to be a promising solution. Despite years of development in this field, difficulties remain in guaranteeing the action stability of DRL-based quadruped robots’ locomotion, especially in tough terrain. In this paper, a terrain-aware teacher-student controller integrating a risk assessment networ...
-
Quadruped robots possess advantages on different terrains over other types of mobile robots by virtue of their flexible choices of foothold points. It is crucial to integrate terrain perception with motion planning to exploit the potential of quadruped robots. We propose a novel hierarchical terrain-aware control (HTC) framework, which leverages deep reinforcement learning (DRL) for the high-level...
-
Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous regions when applying reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, ...
-
Reward function, as an incentive representation that recognizes humans’ agency and rationalizes humans’ actions, is particularly appealing for modeling human behavior in human-robot interaction. Inverse Reinforcement Learning is an effective way to retrieve reward functions from demonstrations. However, it has always been challenging when applying it to multi-agent settings since the mutual influe...
-
This work developed meta-learning control policies to achieve fast online adaptation to different changing conditions, which generate diverse and robust locomotion. The proposed method updates the interaction model constantly, samples feasible sequences of actions of estimated state-action trajectories, and then applies the optimal actions to maximize the reward. To achieve online model adaptation...
-
Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid...
-
We study the problem of multi-robot mapless navigation in the popular Centralized Training and Decentralized Execution (CTDE) paradigm. This problem is challenging when each robot considers its path without explicitly sharing observations with other robots and can lead to non-stationary issues in Deep Reinforcement Learning (DRL). The typical CTDE algorithm factorizes the joint action-value functi...
-
3D perception on point-cloud is a challenging and crucial computer vision task. A point-cloud consists of a sparse, unstructured, and unordered set of points. To understand a point-cloud, previous point-based methods, such as PointNet++, extract visual features through the hierarchical aggregation of local features. However, such methods have several critical limitations: 1) They require considera...
-
Estimating the 6D pose of objects is beneficial for robotics tasks such as transportation, autonomous navigation, manipulation as well as in scenarios beyond robotics like virtual and augmented reality. With respect to single image pose estimation, pose tracking takes into account the temporal information across multiple frames to overcome possible detection inconsistencies and to improve the pose...
-
Execution monitoring is essential for robots to detect and respond to failures. Since it is impossible to enumerate all failures for a given task, we learn from successful executions of the task to detect visual anomalies during runtime. Our method learns to predict the motions that occur during the nominal execution of a task, including camera and robot body motion. A probabilistic U-Net architec...
-
We propose a new deep learning framework to decompose monocular videos into 3D geometry (camera pose and depth), moving objects, and their motions, with no supervision. We build upon the idea of view synthesis, which uses classical camera geometry to re-render a source image from a different point-of-view to obtain supervisory signals, specified by a predicted relative 6-degree-of-freedom pose and...
-
We propose a novel multi-pose loss function to train a neural network for 6D pose estimation, using synthetic data and evaluating it on real images. Our loss is inspired by the VSD (Visible Surface Discrepancy) metric and relies on a differentiable renderer and CAD models. This novel multi-pose approach produces multiple weighted pose estimates to avoid getting stuck in local minima. Our method re...
-
We tackle the problem of refining and denoising a series of 3D human poses estimated from a low-resolution video. Low-resolution often causes the wrong pose estimation, e.g., left-right switching and the absence of keypoints. We propose to use the variational autoencoder (VAE) to remove these challenging noises. The VAE model utilizes time-series information and motion priors in denoising. From ou...
-
We present KDFNet, a novel method for 6D object pose estimation from RGB images. To handle occlusion, many recent works have proposed to localize 2D keypoints through pixel-wise voting and solve a Perspective-n-Point (PnP) problem for pose estimation, which achieves leading performance. However, such voting process is direction-based and cannot handle long and thin objects where the direction inte...
-
This paper introduces a novel approach for the grasping and precise placement of various known rigid objects using multiple grippers within highly cluttered scenes. Using a single depth image of the scene, our method estimates multiple 6D object poses together with an object class, a pose distance for object pose estimation, and a pose distance from a target pose for object placement for each auto...
-
We propose an approach to multi-modal grasp detection that jointly predicts the probabilities that several types of grasps succeed at a given grasp pose. Given a partial point cloud of a scene, the algorithm proposes a set of feasible grasp candidates, then estimates the probabilities that a grasp of each type would succeed at each candidate pose. Predicting grasp success probabilities directly fr...
-
This paper proposes a new deep learning approach to antipodal grasp detection, named Double-Dot Network (DD-Net). It follows the recent anchor-free object detection framework, which does not depend on empirically pre-set anchors and thus allows more generalized and flexible prediction on unseen objects. Specifically, unlike the widely used 5-dimensional rectangle, the gripper configuration is defi...
-
In-flight objects capture is extremely challenging. The robot is required to complete trajectory prediction, interception position calculation and motion planning within tens of milliseconds. As in-flight uneven objects are affected by various kinds of forces, which leads to the time-varying acceleration, motion prediction for them is difficult. In order to compensate the system’s non-linearity, w...
-
Robot learning of real-world manipulation tasks remains challenging and time consuming, even though actions are often simplified by single-step manipulation primitives. In order to compensate the removed time dependency, we additionally learn an image-to-image transition model that is able to predict a next state including its uncertainty. We apply this approach to bin picking, the task of emptyin...
-
We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybri...
-
Grasp pose detection generates the position and orientation of the robot end-effector to grasp objects from the RGB or RGB-D image. In this paper, we propose a novel grasp pose detection network that generates 3-DOF grasp poses using the RGB image. The network follows the anchor-based object detection pipeline and incorporates the angle detection unit. Furthermore, we redesign the grasp angle pred...
-
Anytime motion planners are widely used in robotics. However, the relationship between their solution quality and computation time is not well understood, and thus, determining when to quit planning and start execution is unclear. In this paper, we address the problem of deciding when to stop deliberation under bounded computational capacity, so called meta-reasoning, for anytime motion planning. ...
-
Among the most prevalent motion planning techniques, sampling and trajectory optimization have emerged successful due to their ability to handle tight constraints and high-dimensional systems, respectively. However, limitations in sampling in higher dimensions and local minima issues in optimization have hindered their ability to excel beyond static scenes in offline settings. Here we consider hig...
-
In this paper, we present a novel algorithm – DRGBT (Dynamic Rapidly-exploring Generalized Bur Tree), intended for motion planning in dynamic environments. The main idea behind DRGBT lies in a so-called adaptive horizon, consisting of a set of prospective target nodes that belong to a predefined $\mathcal{C}$-space path, which originates from the current node. Each node is assigned a weight that d...
-
This paper presents a novel approach to the multigoal trajectory planning for vehicles with curvature-constrained trajectories such as fixed-wing aircraft. In the existing formulation called the Dubins Traveling Salesman Problem (DTSP), the vehicle speed is assumed to be constant over the whole trajectory, and that does not allow adaptation of the turning radius of the trajectory between the targe...
-
We introduce a risk-aware variant of the Traveling Salesperson Problem (TSP), where the robot tour cost and reward have to be optimized simultaneously, while being subjected to uncertainty in both. We study the case where the rewards and the costs exhibit diminishing marginal gains, i.e., are submodular. Since the costs and the rewards are stochastic, we seek to maximize a risk metric known as Con...
-
This work considers the problem of generating computationally efficient quadrotor motion primitives between a given pose (position, velocity, and acceleration) and a goal plane in the presence of obstacles. A new motion primitive tool based on the logistic curve is proposed and a closed-form analytic approach is developed to satisfy constraints on starting pose, goal plane, velocity, acceleration,...
-
We consider the problem of generating a fixed path for a mobile observer in a polygonal environment that can maintain a line-of-sight with an unpredictable target. In contrast to purely off-line or on-line techniques, we propose a hierarchical tracking strategy in which an off-line path generation technique based on a RRT is coupled with an online feedback-control technique to generate trajectorie...
-
Developing the flocking behavior for a dynamic squad of fixed-wing UAVs is still a challenge due to kinematic complexity and environmental uncertainty. In this paper, we deal with the decentralized flocking and collision avoidance problem through deep reinforcement learning (DRL). Specifically, we formulate a decentralized DRL-based decision making framework from the perspective of every follower,...
-
This work proposes a scheme that allows learning complex multi-agent behaviors in a sample efficient manner, applied to 2v2 soccer. The problem is formulated as a Markov game, and solved using deep reinforcement learning. We propose a basic multi-agent extension of TD3 for learning the policy of each player, in a decentralized manner. To ease learning, the task of 2v2 soccer is divided in three st...
-
Leader-follower navigation is a popular class of multi-robot algorithms where a leader robot leads the follower robots in a team. The leader has specialized capabilities or mission critical information (e.g. goal location) that the followers lack, and this makes the leader crucial for the mission’s success. However, this also makes the leader a vulnerability -an external adversary who wishes to sa...
-
Multi-agent path finding in formation has many potential real-world applications like mobile warehouse robotics. However, previous multi-agent path finding (MAPF) methods hardly take formation into consideration. Further-more, they are usually centralized planners and require the whole state of the environment. Other decentralized partially observable approaches to MAPF are reinforcement learning ...
-
We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observatio...
-
Dynamic inference that adaptively skips parts of model execution based on the complexity of input data can effectively reduce the computation cost of deep learning models during the inference. However, current architectures for dynamic inference only consider the exits at the block level, whose results may not be suitable for different applications. In this paper, we present the Auto-Dynamic-DeepL...
-
Semantic segmentation serves as a cornerstone for safety autonomous driving and has been achieved remarkable progress at the price of dense annotations. Unsupervised domain adaptation was widely utilized to addresses this labor-intensive problem, which transfers the knowledge learned from labeled synthetic datset to real-world without any annotations. However, most existing adaptation works predic...
-
Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks
Category-level 6D pose estimation, aiming to predict the location and orientation of unseen object instances, is fundamental to many scenarios such as robotic manipulation and augmented reality, yet still remains unsolved. Precisely recovering instance 3D model in the canonical space and accurately matching it with the observation is an essential point when estimating 6D pose for unseen objects. I...
-
Although instance-aware perception is a key prerequisite for many autonomous robotic applications, most of the methods only partially solve the problem by focusing solely on known object categories. However, for robots interacting in dynamic and cluttered environments, this is not realistic and severely limits the range of potential applications. Therefore, we propose a novel object instance segme...
-
Object models are highly useful for robots as they enable tasks such as detection, pose estimation and manipulation. However, models are not always easily available, especially in real-world domains of operation such as peoples’ homes. This work presents a pipeline to generate high-quality object reconstructions from human in-hand manipulation to alleviate the necessity of specialised or expensive...
-
During deployment, an object detector is expected to operate at a similar performance level reported on its testing dataset. However, when deployed onboard mobile robots that operate under varying and complex environmental conditions, the detector’s performance can fluctuate and occasionally degrade severely without warning. Undetected, this can lead the robot to take unsafe and risky actions base...
-
In the context of autonomous vehicles, one of the most crucial tasks is to estimate the risk of the undertaken action. While navigating in complex urban environments, the Bayesian occupancy grid is one of the most popular types of maps, where the information of occupancy is stored as the probability of collision. Although widely used, this kind of representation is not well suited for risk assessm...
-
Future urban transportation concepts include a mixture of ground and air vehicles with varying degrees of autonomy in a congested environment. In such dynamic environments, occupancy maps alone are not sufficient for safe path planning. Safe and efficient transportation requires reasoning about the 3D flow of traffic and properly modeling uncertainty. Several different approaches can be taken for ...
-
We present a systematic approach called Extended VINS-Mono to utilize VINS-Mono, a state-of-the-art monocular visual-inertial relative localization method, targeting practical vehicle localization in large-scale outdoor road environments. Our proposed fusion approach associates multiple independent localization methods and provides multiple (projected) state estimates in a desired coordinate syste...
-
Many essential services for autonomous vehicles, e.g., navigation on high-quality maps, are designed based on the understanding of traffic conditions, e.g., travel time/speed on road segments, traffic flow, etc. However, most existing traffic condition models lack the consideration of the differentiation for vehicles with different types (e.g., personal vehicles or trucks) and thus they cannot sat...
-
This paper presents a vision-based control strategy for a rotary-wing unmanned aerial vehicle (RUAV) transporting an unknown suspended payload. The suspended payload parameters, which include its mass and cable length, are unknown and direct measurements of its states are not available. A feedforward-feedback adaptive control strategy, that consists of a notch filter and linear quadratic Gaussian ...
-
Many cities suffer from a shortage of parking spaces. Research in high density parking (HDP) focuses on how to increase the capacity of parking lots by allowing vehicles to block each other but temporarily give way to other vehicles by driving autonomously upon request. Previous works on HDP did not consider mixing different parking strategies and ignored the possibility of gridlock when multiple ...
-
The ability to predict the future movements of other vehicles is a subconscious and effortless skill for humans and key to safe autonomous driving. Therefore, trajectory prediction for autonomous cars has gained a lot of attention in recent years. It is, however, still a hard task to achieve human-level performance. Interdependencies between vehicle behaviors and the multimodal nature of future in...
-
We present a new neural network architecture, called NaturalNet, which uses a simplified biological neuron model and consists of a set of nonlinear ordinary differential equations. We model the membrane potential of each neuron by integrating the in-flowing currents, but we do not consider ion channels, nor individual spikes. To keep the membrane potential within a defined value range, we introduc...
-
Prior work in natural-language-driven navigation demonstrates success in systems deployed in synthetic environments or applied to large datasets, both real and synthetic. However, there is an absence of such frameworks being deployed and rigorously tested in real environments, unknown a priori. In this paper, we present a novel framework that uses spoken dialogue with a real person to interpret a ...
-
The successful performance of any system is dependant on the hardware of the agent, which is typically immutable during RL training. In this work, we present ORCHID (Optimisation of Robotic Control and Hardware In Design) which allows for truly simultaneous optimisation of hardware and control parameters in an RL pipeline. We show that by forming a complex differential path through a trajectory ro...
-
In the paper, we show how scalable, low-cost trunk-like robotic arms can be constructed using only basic 3D-printing equipment and simple electronics. The design is based on uniform, stackable joint modules with three degrees of freedom each. Moreover, we present an approach for controlling these robots with recurrent spiking neural networks. At first, a spiking forward model learns motor-pose cor...
-
Learning a robot motor skill from scratch is impractically slow; so much so that in practice, learning must typically be bootstrapped using human demonstration. However, relying on human demonstration necessarily degrades the autonomy of robots that must learn a wide variety of skills over their operational lifetimes. We propose using kinematic motion planning as a completely autonomous, sample ef...
-
This paper proposes a novel neural network-based control policy to enable a mobile robot to navigate safety through environments filled with both static obstacles, such as tables and chairs, and dense crowds of pedestrians. The network architecture uses early fusion to combine a short history of lidar data with kinematic data about nearby pedestrians. This kinematic data is key to enable safe robo...
-
Autonomous driving with imitation learning is vulnerable to the quality of an expert dataset. Typical driving involves situations or online data that are biased toward specific scenarios such as lane following or stop. This property causes an imbalance in the driving dataset, and it is highly likely to deteriorate the performance of autonomous driving with imitation learning. In this paper, we pro...
-
Parallel Continuum Robots (PCR) have received a lot of attention in recent years. This paper presents a new 6-degrees-of-freedom PCR derived from the conventional 3-PPSR parallel manipulator. This robot is driven by three limbs consisting of two flexible rods each and replacing the spherical and revolute joints of the original version. Each limb is mounted onto two linear axes arranged in series. ...
-
Soft robotic gloves have shown great advantages in assisting individuals with hand pathologies to perform continuous exercises to restore their hand functions, which could considerably accelerate the rehabilitation process and reduce the costs. However, single rehabilitation mode, difficulty in achieving multiple degrees-of-freedom (DoF) motion, and the lack of high-fidelity feedback still challen...
-
Soft climbing robots have been attracting increasing attention in soft robotics community, and a lot of prototypes been proposed with basic climbing function implemented. Climbing on poles is a challenge with soft robots, and the capability of current pole-climbing soft robots needs to be improved in terms of adaptability to various poles and deformation controllability or constraining of the soft...
-
Soft, tip-extending, pneumatic "vine robots" that grow via eversion are well suited for navigating cluttered environments. Two key mechanisms that add to the robot’s functionality are a tip-mounted retraction device that allows the growth process to be reversed, and a tip-mounted camera that enables vision. However, previous designs used rigid, relatively heavy electromechanical retraction devices...
-
The fabrication method to utilize poly (vinyl alcohol) hydrogels with additional stiff parts in a single structure for hydrogel-based soft robots to realize an intensive motion with elastic energy is proposed in this paper. An inorganic material which is often seen in the hard tissues of our body; hydroxyapatite, was partially formed on a hydrogel with a simple procedure of alternatingly soaking a...
-
A human hand can grasp a desired number of objects at once from a pile based solely on tactile sensing. To do so, a robot needs to make a grasp in a pile, sense the number of objects in the grasp before lifting, and predict how many will remain in the grasp after lifting. It is a very challenging problem because when making the prediction, the robotic hand is still in the pile and the objects in t...
-
Product packing is a typical application in ware-house automation that aims to pick objects from unstructured piles and place them into bins with optimized placing policy. However, it still remains a significant challenge to finish the product packing tasks in general logistics scenarios where the objects are variable-sized and the configurations are complex. In this work, we present the PackerBot...
-
This paper presents the design, fabrication, and preliminary results of a soft hip exosuit to assist hip flexion and extension during walking. The exosuit uses soft and compliant materials to create a wearable robot that has a low profile, low mass, and is highly flexible to freely move with the user’s natural range of motion. The Soft Robotic Hip Exosuit (SR-HExo) consists of flat fabric pneumati...
-
There is an increasing demand for accurate prediction of joint moments using wearable sensors for robotic exoskeletons to achieve precise control and for rehabilitation care to remotely monitor users’ condition. In this study, we used electromyography (EMG) signals to first identify muscle synergies, then used them to train of a long short-term memory network to predict knee joint moments during w...
-
With over 10 million people currently suffering from significant long-term gait disability in the United States only, robot-assisted rehabilitation and wearable devices are increasingly gaining attention as a mean to regain functional mobility. Since these devices work collaborative and synchronously with the human gait, it is necessary to be able to detect gait events, such as heel-strikes, in re...
-
This paper presents a method for biped gait generation along a predefined curve with fully stretched knees. First, we design a spatial gait pattern as a function of the traveled distance on the path without considering dynamics. Then, a consistent dynamic walking motion is obtained by optimization that minimizes the zero-moment point and the speed errors while considering the trade-off between kin...
-
In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimensional state and...
-
Legged robots often use separate control policies that are highly engineered for traversing difficult terrain such as stairs, gaps, and steps, where switching between policies is only possible when the robot is in a region that is common to adjacent controllers. Deep Reinforcement Learning (DRL) is a promising alternative to hand-crafted control design, though typically requires the full set of te...
-
When legged robots impact their environment, they undergo large changes in their velocities in a small amount of time. Measuring and applying feedback to these velocities is challenging, and is further complicated due to uncertainty in the impact model and impact timing. This work proposes a general framework for adapting feedback control during impact by projecting the control objectives to a sub...
-
In this paper, with a view toward deployment of light-weight control frameworks for bipedal walking robots, we realize end-foot trajectories that are shaped by a single linear feedback policy. We learn this policy via a model-free and a gradient free learning algorithm, Augmented Random Search (ARS), in the two robot platforms Rabbit and Digit. Our contributions are two-fold: a) By using torso and...
-
We present a novel, low cost framework for reconstructing surface contact movements during in-hand manipulations. Unlike many existing methods focused on hand pose tracking, ours models the behavior of contact patches, and by doing so is the first to obtain detailed contact tracking estimates for multi-contact manipulations. Our framework is highly accessible, requiring only low cost, readily avai...
-
We propose a novel real-time physically-accurate simulation framework for the snap connection process. For this, we first notice the peculiarities of the process, namely, small/smooth deformation, stiff connector and segmented contact. We then design our simulation to fully exploit these peculiarities by adopting the following strategies: 1) the technique of passive midpoint integration (PMI [1]),...
-
Frictional contact has been extensively studied as the core underlying behavior of legged locomotion and manipulation, and its nearly-discontinuous nature makes planning and control difficult even when an accurate model of the robot is available. Here, we present empirical evidence that learning an accurate model in the first place can be confounded by contact, as modern deep learning approaches a...
-
This paper proposes a novel and practical approach to enhance the computational efficiency of the hierarchical quadratic programming (HQP)-based whole-body control. The HQP method is known to offer control solutions satisfying strict priority with various constraints for multiple-tasks execution. However, it inherently comes at the price of high computation time to solve QP optimization problems i...
-
We describe a framework for changing-contact robot manipulation tasks, which require the robot to make and break contacts with objects and surfaces. The discontinuous interaction dynamics of such tasks make it difficult to construct and use a single dynamics model or control strategy for such tasks. For any target motion trajectory, our framework incrementally improves its prediction of when conta...
-
Constrained battery life on current Unmanned Aerial Vehicles (drones) limits the time they can operate and distance they can travel. We address this challenge by harvesting solar power to enable duty-cycled operation on a palm-sized drone. We present a scaling analysis that suggests that more solar power can be collected per unit mass of the drone as scale reduces, favoring small drones. By chargi...
-
Autonomous Micro Aerial Vehicles (MAVs) have the potential to be employed for surveillance and monitoring tasks. By perching and staring on one or multiple locations aerial robots can save energy while concurrently increasing their overall mission time without actively flying. In this paper, we address the estimation, planning, and control problems for autonomous perching on inclined surfaces with...
-
The visibility of targets determines performance and even success rate of various applications, such as active slam, exploration, and target tracking. Therefore, it is crucial to take the visibility of targets into explicit account in trajectory planning. In this paper, we propose a general metric for target visibility, considering observation distance and angle as well as occlusion effect. We for...
-
Novel methods for the inference of radiation intensity functions defined over known surfaces are proposed, intended for use in surveying applications with mobile spectrometers. Previous approaches, based on the maximum likelihood expectation maximization (ML-EM) framework with Poisson likelihoods, are extended to better handle spatially continuous intensity statistics using ideas from Gaussian fil...
-
Aerial vehicles are revolutionizing applications that require capturing the 3D structure of dynamic targets in the wild, such as sports, medicine and entertainment. The core challenges in developing a motion-capture system that operates in outdoors environments are: (1) 3D inference requires multiple simultaneous viewpoints of the target, (2) occlusion caused by obstacles is frequent when tracking...
-
Transporting objects using quadrotors with cables has been widely studied in the literature. However, most of those approaches assume that the cables are previously attached to the load by human intervention. In tasks where multiple objects need to be moved, the efficiency of the robotic system is constrained by the requirement of manual labor. Our approach uses a non-stretchable cable connected t...
-
When dealing with the haptic teleoperation of multi-limbed mobile manipulators, the problem of mitigating the destabilizing effects arising from the communication link between the haptic device and the remote robot has not been properly addressed. In this work, we propose a passive control architecture to haptically teleoperate a legged mobile manipulator, while remaining stable in the presence of...
-
Telerobotic systems combined with miniaturised snake-like or elephant-trunk robotic arms can improve the ergonomics and accessibility in minimally invasive surgical tasks such as knee arthroscopy. Such systems, however, are usually designed in a specific and integral approach, making it expensive to adapt to various procedures or patient anatomies. 3D printed instruments with a detachable design c...
-
Mobile telepresence robots (MTRs) allow people to navigate and interact with a remote environment that is in a place other than the person’s true location. Thanks to the recent advances in 360° vision, many MTRs are now equipped with an all-degree visual perception capability. However, people’s visual field horizontally spans only about 120° of the visual field captured by the robot. To bridge thi...
-
In the context of teleoperation, arbitration refers to deciding how to blend between human and autonomous robot commands. We present a reinforcement learning solution that learns an optimal arbitration strategy that allocates more control authority to the human when the robot comes across a decision point in the task. A decision point is where the robot encounters multiple options (sub-policies), ...
-
Robotic avatars promise immersive teleoperation with human-like manipulation and communication capabilities. We present such an avatar system, based on the key components of immersive 3D visualization and transparent force-feedback telemanipulation. Our avatar robot features an anthropomorphic bimanual arm configuration with dexterous hands. The remote human operator drives the arms and fingers th...
-
Progress in the last decade has brought about significant improvements in the accuracy and speed of SLAM systems, broadening their mapping capabilities. Despite these advancements, long-term operation remains a major challenge, primarily due to the wide spectrum of perturbations robotic systems may encounter.Increasing the robustness of SLAM algorithms is an ongoing effort, however it usually addr...
-
Recent Semantic SLAM methods combine classical geometry-based estimation with deep learning-based object detection or semantic segmentation. In this paper we evaluate the quality of semantic maps generated by state-of-the-art class-and instance-aware dense semantic SLAM algorithms whose codes are publicly available and explore the impacts both semantic segmentation and pose estimation have on the ...
-
Localization is an essential module that supports many intelligent functions of a mobile robot such as transportation or inspection. However, justifying that a localization module is sufficiently accurate for supporting all downstream tasks is one of the most difficult questions to answer in practice. To overcome this problem, we move away from the traditional calculation of pose errors and propos...
-
Estimating depth from a monocular image is an ill-posed problem: when the camera projects a 3D scene onto a 2D plane, depth information is inherently and permanently lost. Nevertheless, recent work has shown impressive results in estimating 3D structure from 2D images using deep learning. In this paper, we put on an introspective hat and analyze state-of-the-art monocular depth estimation models i...
-
Precise perception is one of the key enablers of autonomous robotic operations. The right selection of sensors significantly influences the overall performance of the system. This paper provides a systematic approach for evaluation of various sensors available on the market. The main focus is to assess the performance in use cases of short to medium distance operations, especially relevant for pre...
-
The implementation of stable locomotion on humanoid robots is a difficult task. This is complicated by the fact that there is no uniform method for analyzing a robot and its control architecture and for calculating indicators to quantify performance of flat ground walking. Moreover, there is no widely accepted indicator do distinct between a stable and unstable state of the robot. We propose the I...
-
A robot can invoke heterogeneous computation resources such as CPUs, cloud GPU servers, or even human computation for achieving a high-level goal. The problem of invoking an appropriate computation model so that it will successfully complete a task while keeping its compute and energy costs within a budget is called a model selection problem. In this paper, we present an optimal solution to the mo...
-
Deep neural networks are the leading solution to the object detection problem. However, challenges arise when applying these networks to the kind of real-time, first-person video data that a robotic platform must process: specifically, detections may not be consistent from frame to frame, and objects may frequently appear at viewpoints that are particularly challenging for the model, resulting in ...
-
Autonomous Underwater Vehicles (AUVs) can be effective collaborators to human scuba divers in many applications, such as environmental surveying, mapping, or infrastructure repair. However, for these applications to be realized in the real world, it is essential that robots are able to both lead and follow their human collaborators. Current algorithms for diver following are not robust to non-unif...
-
Identifying in-water obstacles is fundamental for safe navigation of Autonomous Surface Vehicles (ASVs). This paper presents a model-free method for segmenting individual in-water objects (e.g., swimmers, buoys, boats) and shorelines from LiDAR sensor data. To reduce the computational requirement, our method first converts the 3D point cloud into a 2D spherical projection image. Then, an algorithm...
-
This paper introduces a novel deep learning approach to semantic segmentation of the shoreline environments with a high frames-per-second (fps) performance, making the approach readily applicable to autonomous navigation for Unmanned Surface Vehicles (USV). The proposed ShorelineNet is an efficient deep neural network of high performance relying only on visual input. ShorelineNet uses monocular vi...
-
Visual monitoring operations underwater require both observing the objects of interest in close-proximity, and tracking the few feature-rich areas necessary for state estimation. This paper introduces the first navigation framework, called AquaVis, that produces on-line visibility-aware motion plans that enable Autonomous Underwater Vehicles (AUVs) to track multiple visual objectives with an arbit...
-
Most of the state-of-the-art visual-inertial SLAM methods pay less attention to the scene structure of man-made environments. In this paper, based on the assumption of multiple local Manhattan worlds (MWs), we propose a Manhattan frame (MF) re-identification method to build relative rotation constraints between MF matching pairs and tightly couple these constraints into global bundle adjust module...
-
SymbioLCD: Ensemble-Based Loop Closure Detection using CNN-Extracted Objects and Visual Bag-of-Words
Loop closure detection is an essential tool of Simultaneous Localization and Mapping (SLAM) to minimize drift in its localization. Many state-of-the-art loop closure detection (LCD) algorithms use visual Bag-of-Words (vBoW), which is robust against partial occlusions in a scene but cannot perceive the semantics or spatial relationships between feature points. CNN object extraction can address thos...
-
Building object-level maps can facilitate robot-environment interactions (e.g. planning and manipulation), but objects could often have multiple probable poses when viewed from a single vantage point, due to symmetry, occlusion or perceptual failures. A robust object-level simultaneous localization and mapping (object SLAM) algorithm needs to be aware of this pose ambiguity. We propose to maintain...
-
This paper presents a tightly coupled pipeline, which efficiently fuses measurements of LiDAR, camera, IMU, encoder, and GNSS to estimate the robot state and build a map even in challenging situations. The depth of visual features is extracted by projecting the LiDAR point cloud and ground plane into image. We select the tracked high-quality visual features and LiDAR features and tightly coupled t...
-
Motivated by the goal of achieving long-term drift-free camera pose estimation in complex scenarios, we propose a global positioning framework fusing visual, inertial and Global Navigation Satellite System (GNSS) measurements in multiple layers. Different from previous loosely- and tightly-coupled methods, the proposed multi-layer fusion allows us to delicately correct the drift of visual odometry...
-
Radar SLAM has attracted wide attention due to its all-day and all-weather working characteristics in the last decade. The existing radar SLAM systems mainly adopt mechanically pivoting radar with simple principle and high resolution, but this kind of radar has disadvantages such as low frame rate, distortion of the radar image, and high cost. Although array snapshot radar has the advantages of hi...
-
This paper presents an accurate, highly efficient and learning free method for large-scale radar odometry estimation. By using a simple filtering technique that keeps the strongest returns, we produce a clean radar data representation and reconstruct surface normals for efficient and accurate scan matching. Registration is carried out by minimizing a point-to-line metric and robustness to outliers...
-
Robust visual localization in traffic scenes is a fundamental problem for self-driving vehicles. However, it is still challenging to achieve accurate localization performance because of drastic viewpoint and illumination changes. To address the issues, we design a novel monocular localization framework based on a light-weight prior map, called BSP-MonoLoc, which leverages the 2D semantic primitive...
-
We address the problem of robot localization using ground penetrating radar (GPR) sensors. Current approaches for localization with GPR sensors require a priori maps of the system’s environment as well as access to approximate global positioning (GPS) during operation. In this paper, we propose a novel, real-time GPR-based localization system for unknown and GPS-denied environments. We model the l...
-
LiDAR map matching(LMM) is a critical localization technique in autonomous driving while existing methods have problems in terms of both accuracy and robustness when driving in the scenes with poor structure information (e.g. highways). This paper put forward a multi-level intensity map based cascaded network for LiDAR map matching in autonomous driving. The network uses an effective multi-level i...
-
This paper presents DLL, a fast direct map-based localization technique using 3D LIDAR for its application to aerial robots. DLL implements a point cloud to map registration based on non-linear optimization of the distance of the points and the map, thus not requiring features, neither point correspondences. Given an initial pose, the method is able to track the pose of the robot by refining the p...
-
Simultaneous Localization and Mapping (SLAM) is an essential capability for autonomous robots, but due to high data rates of 3D LiDARs real-time SLAM is challenging. We propose a real-time method for 6D LiDAR odometry. Our approach combines a continuous-time B-Spline trajectory representation with a Gaussian Mixture Model (GMM) formulation to jointly align local multi-resolution surfel maps. Spars...
-
The ability to localise is key for robot navigation. We describe an efficient method for vision-based localisation, which combines sequential Monte Carlo tracking with matching ground-level images to 2-D cartographic maps such as OpenStreetMaps. The matching is based on a learned embedded space representation linking images and map tiles, encoding the common semantic information present in both an...
-
Behavior prediction of traffic actors is an essential component of any real-world self-driving system. Actors’ long-term behaviors tend to be governed by their interactions with other actors or traffic elements (traffic lights, stop signs) in the scene. To capture this highly complex structure of interactions, we propose to use a hybrid graph whose nodes represent both the traffic actors as well a...
-
Many autonomous driving applications nowadays come along with a prebuilt vector map for routing and planning purposes. In order to localize on this map, traditional LiDAR localization methods usually require a separate localization layer to function. On one hand, the separate layer occupies large storage and is not convenient to update. On the other hand, the potential of the vector map itself has...
-
Localization is a crucial prerequisite for automated valet parking, in which a vehicle is required to navigate itself in a GPS-denied parking lot. Traditional visual localization methods usually build a feature map and use it for future localizations. However, the feature map is not robust to changes in illumination, appearance, and viewing perspective. To deal with this issue, we need a more stab...
-
We aim to assess the performance of LiDAR-to-map registration on compressive maps. Modern autonomous vehicles utilize pre-built HD (High-Definition) maps to perform sensor-to-map registration, which recovers pose estimation failures and reduces drift in a large-scale environment. However, sensor-to-map registration is usually realized by registering the sensor to a dense 3D model, which occupies m...
-
We would like to enable a robot to navigate efficiently and robustly in known, structured environments that are large enough to cause traditional planning approaches to incur considerable computational cost. Hierarchical planners are a promising way to increase planning efficiency in such environments because high-level abstract plans can be used to reduce the size of the search space over which d...
-
The 3D visual perception for vehicles with the surround-view fisheye camera system is a critical and challenging task for low-cost urban autonomous driving. While existing monocular 3D object detection methods perform not well enough on the fisheye images for mass production, partly due to the lack of 3D datasets of such images. In this paper, we manage to overcome and avoid the difficulty of acqu...
-
Autonomous vehicles on city roads and especially in pedestrian environments require agility to navigate narrow passages and turn in tight spaces, leading to the need for a real-time, robust and adaptable controller. In this paper, we present orientation and context aware controllers for autonomous vehicles that can closely track the reference path wit alh respect to the current state of the vehicl...
-
We propose a novel benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. Aquatic navigation is an extremely challenging task due to the non-stationary environment and the uncertainties of the robotic platform, hence it is crucial to consider the safety aspect of the problem, by analyzing the behavior of the trained network to avoid dangerous situations (e.g., collis...
-
This paper proposes a collaborative policy framework via relational graph reasoning for multi-agent systems to accomplish adversarial tasks. A relational graph reasoning module consisting of an agent graph reasoning module and an opponent graph module, is designed to enable each agent to learn mixture state representation to enhance the effectiveness of the policy. In particular, for each agent, t...
-
Solving complex real-world tasks, e.g., autonomous fleet control, often involves a coordinated team of multiple agents which learn strategies from visual inputs via reinforcement learning. Many existing multi-agent reinforcement learning (MARL) algorithms however don’t scale to environments where agents operate on visual inputs. To address this issue, algorithmically, recent works have focused on ...
-
Informed and robust decision making in the face of uncertainty is critical for robots operating in unstructured environments. We formulate this as Bayesian Reinforcement Learning over latent Markov Decision Processes (MDPs). While Bayes-optimality is theoretically the gold standard, existing algorithms scale poorly to continuous state and action spaces. We build on the following insight: in the ab...
-
A promising characteristic of Deep Reinforcement Learning (DRL) is its capability to learn optimal policy in an end-to-end manner without relying on feature engineering. However, most approaches assume a fully observable state space, i.e. fully observable Markov Decision Processes (MDPs). In real-world robotics, this assumption is unpractical, because of issues such as sensor sensitivity limitatio...
-
Reinforcement Learning (RL) has made remarkable achievements, but it still suffers from inadequate exploration strategies, sparse reward signals, and deceptive reward functions. To alleviate these problems, a Population-guided Novelty Search (PNS) parallel learning method is proposed in this paper. In PNS, the population is divided into multiple sub-populations, each of which has one chief agent a...
-
Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Lea...
-
Q-learning with Long-term Action-space Shaping to Model Complex Behavior for Autonomous Lane Changes
In autonomous driving applications, reinforcement learning agents often have to perform complex behavior, which can translate into optimizing multiple objectives while following certain rules. Encoding traffic rules and desires such as safety and comfort via classical methods based on reward shaping (i.e. a weighted combination of different objectives in the reward signal) or Lagrangian methods (i...
-
Single image haze removal is crucial in computer vision. In open literatures, two kinds of dehazing strategies (prior-based and learning-based methods) have been developed. However, they have a trade-off between detail preservation and the image quality. Prior-based methods reconstruct the detail well but have lower image quality while learning-based methods achieve better recovered quality but lo...
-
Inferring a complete 3D geometry given an in-complete point cloud is essential in many vision and robotics applications. Previous work mainly relies on a global feature extracted by a Multi-layer Perceptron (MLP) for predicting the shape geometry. This suffers from a loss of structural details, as its point generator fails to capture the detailed topology and structure of point clouds using only t...
-
Along with point features, line features play an important role in achieving robust Simultaneous Localization and Mapping (SLAM) under complex environments. This paper proposes a fast and effective method, namely Superline, to simultaneously detect line segments and generate robust descriptors for matching. The entire model is composed of a convolutional backbone and two task heads, i.e., detectio...
-
Stereo reconstruction models trained on small images do not generalize well to high-resolution data. Training a model on high-resolution image size faces difficulties of data availability and is often infeasible due to limited computing resources. In this work, we present the Occlusion-aware Recurrent binocular Stereo matching (ORStereo), which deals with these issues by only training on available...
-
The ability to adapt their perception to changing environments is a core characterization of intelligent robots. At present, Unsupervised Domain Adaptation (UDA) methods are used to address this problem where the adaptation task is formulated as a transfer problem from a well-described scenario (source domain) to a new scenario (target domain). In order to implement the domain adaptation, these me...
-
Deep learning approaches to estimating 3D object pose and geometry present an attractive alternative to online estimation techniques, which can suffer from significant estimation latency. However, a practical hurdle to training state-of-the-art deep 3D bounding box estimators is collecting a sufficiently large dataset of 3D bounding box labels. In this work, we present a novel framework for weakly...
-
Self-supervised learning for depth estimation possesses several advantages over supervised learning. The benefits of no need for ground-truth depth, online fine-tuning, and better generalization with unlimited data attract researchers to seek self-supervised solutions. In this work, we propose a new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera ...
-
Transparent objects with unique visual properties often make depth cameras fail to scan their reflective and refractive surfaces. Recent studies on depth completion of transparent objects have leveraged a linear system based on the geometric constraints to predict the missing depth, which is hard to be employed in an end-to-end framework and achieve joint optimization. In this paper, we propose De...
-
Zero-shot execution of unseen robotic tasks is important to allowing robots to perform a wide variety of tasks in human environments, but collecting the amounts of data necessary to train end-to-end policies in the real-world is often infeasible. We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a si...
-
Grasp detection of novel objects in unstructured environments is a key capability in robotic manipulation. For 2D grasp detection problems where grasps are assumed to lie in the plane, it is common to design a fully convolutional neural network that predicts grasps over an entire image in one step. However, this is not possible for grasp pose detection where grasp poses are assumed to exist in SE(...
-
The ability to successfully grasp objects is crucial in robotics, as it enables several interactive downstream applications. To this end, most approaches either compute the full 6D pose for the object of interest or learn to predict a set of grasping points. While the former approaches do not scale well to multiple object instances or classes yet, the latter require large annotated datasets and ar...
-
Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates ...
-
Pose estimation is a key challenge in robot manipulation and grasping task. Current object pose estimation approaches based on 3D models and depth sensor information have difficulties to handle transparent objects because of the limitation to capture the accurate depth information. To address these issues, we present a 6DoF pose estimation approach, called GhostPose, which utilizes a novel 3D boun...
-
The object manipulation is a crucial ability for a service robot, but it is hard to solve with reinforcement learning due to some reasons such as sample efficiency. In this paper, to tackle this object manipulation, we propose a novel framework, AP-NPQL (Non-Parametric Q Learning with Action Primitives), that can efficiently solve the object manipulation with visual input and sparse reward, by uti...
-
Robust motion planners for unmanned ground vehicles must minimize risk while obeying vehicle mobility constraints. Algorithms such as the State Lattice (SL) utilize offline computation to generate expressive control sets which form recombinant search spaces, enabling the use of heuristic search to efficiently produce feasible motion plans online. The Adaptive State Lattice (ASL) demonstrated that ...
-
This paper presents a supervised learning method to generate continuous cost-to-go functions of non-holonomic systems directly from the workspace description. Supervision from informative examples reduces training time and improves network performance. The manifold representing the optimal trajectories of a non-holonomic system has high-curvature regions which can not be efficiently captured with ...
-
We propose a novel approach for sampling-based and control-based motion planning. We combine a representation of the environment obtained via a modified version of optimal Rapidly-exploring Random Trees (RRT*), with landmark-based output-feedback controllers obtained via Control Lyapunov Functions, Control Barrier Functions, and robust Linear Programming. Our solution inherits many benefits of RRT...
-
This work proposes a model predictive control approach of a wheeled mobile robot based on a local-minima free navigation function. The constructed navigation function includes information on a goal location and obstacles. Novel conservative navigation is introduced that is simple to compute and yields convergent control behavior. To solve the optimization problem the combined optimization is propo...
-
Sampling-based methods such as Rapidly-exploring Random Trees (RRTs) have been widely used for generating motion paths for autonomous mobile systems. In this work, we extend time-based RRTs with Control Barrier Functions (CBFs) to generate, safe motion plans in dynamic environments with many pedestrians. Our framework is based upon a human motion prediction model which is well suited for indoor na...
-
Motion planning under uncertainty is of significant importance for safety-critical systems such as autonomous vehicles. Such systems have to satisfy necessary constraints (e.g., collision avoidance) with potential uncertainties coming from either disturbed system dynamics or noisy sensor measurements. However, existing motion planning methods cannot efficiently find the robust optimal solutions un...
-
Bladed tools such as jigsaws are common tools for wood workers on job-sites and in workshops, but do not currently have sufficient autonomous hardware or path planning algorithms to enable automation. Here we present a system of an autonomous robot and a path planning algorithm for automating jigsaw operations. The robot can drill holes, insert the jigsaw, and cut plywood. Our algorithm converts c...
-
As more robots are being deployed into human environments, a human-aware navigation planner needs to handle multiple contexts that occur in indoor and outdoor environments. In this paper, we propose a tunable human-aware robot navigation planner that can handle a variety of human-robot contexts. We present the architecture of the system and discuss the features along with some implementation detai...
-
In this paper, we consider the refuel scheduling problem for a team of ground robots deployed in "aislelike" environments wherein the robots are constrained to move along rows. In order to maintain a minimum service rate or throughput for the ground robots, we investigate the problem of scheduling a team of mobile charging stations deployed to replace the batteries on-board the ground robots witho...
-
We develop an event-triggered control strategy for a weighted-unbalanced directed homogeneous robot network to reach a dynamic consensus in this work. We present some guarantees for synchronizing a robot network when all robots have access to the reference and when a limited number of robots have access. The proposed event-triggered control can reduce and avoid the periodic updating of the signals...
-
Cooperative manipulation systems inherently cause internal stress on the common object. Many works have proposed methods to eliminate this internal stress. However, in this paper, we show that this property can be cautiously leveraged to compensate for external disturbance on the cooperative system, particularly disturbances that occur due to collision along the links of one of the cooperating rob...
-
A resilient multi-vehicle system cooperatively performs tasks by exchanging information, detecting, and removing cyber attacks that have the intent of hijacking or diminishing performance of the entire system. In this paper, we propose a framework to: i) detect and isolate misbehaving vehicles in the network, and ii) securely encrypt information among the network to alert and attract nearby vehicl...
-
As multi-robot systems (MRS) are widely used in various tasks such as natural disaster response and social security, people enthusiastically expect an MRS to be ubiquitous that a general user without heavy training can easily operate. However, humans have various preferences on balancing between task performance and safety, imposing different requirements onto MRS control. Failing to comply with p...
-
Relative localization between autonomous robots without infrastructure is crucial to achieve their navigation, path planning, and formation in many applications, such as emergency response, where acquiring a prior knowledge of the environment is not possible. The traditional Ultra-WideBand (UWB)-based approach provides a good estimation of the distance between the robots, but obtaining the relativ...
-
Exploring robots may fail due to environmental hazards. Thus, robots need to account for the possibility of failure to plan the best exploration paths. Optimizing expected utility enables robots to find plans that balance achievable reward with the inherent risks of exploration. Moreover, when robots rendezvous and communicate to exchange observations, they increase the probability that at least o...
-
This paper addresses the problem of active information gathering for multi-robot systems. Specifically, we consider scenarios where robots are tasked with reducing uncertainty of dynamical hidden states evolving in complex environments. The majority of existing information gathering approaches are centralized and, therefore, they cannot be applied to distributed robot teams where communication to ...
-
Humans have mental models that allow them to plan, experiment, and reason in the physical world. How should an intelligent agent go about learning such models? In this paper, we will study if models of the world learned in an open-ended physics environment, without any specific tasks, can be reused for downstream physics reasoning tasks. To this end, we build a benchmark Open-ended Physics Environ...
-
To improve the efficiency of medical visualization for computer aided surgery, we propose a fast and unsupervised 3D-CNN based non-local feature learning network. The proposed network consists of an encoder structure and a decoder structure. The encoder of the network projects the cube into a high-dimensional feature space, and the decoder of the network reconstructs the cube from the feature spac...
-
Optical flow can be leveraged in robotic systems for obstacle detection where low latency solutions are critical in highly dynamic settings. While event-based cameras have changed the dominant paradigm of sending by encoding stimuli into spike trails, offering low bandwidth and latency, events are still processed with traditional convolutional networks in GPUs defeating, thus, the promise of effic...
-
Video prediction methods generally consume substantial computing resources in training and deployment, among which keypoint-based approaches show promising improvement in efficiency by simplifying dense image prediction to light keypoint prediction. However, keypoint locations are often modeled only as continuous coordinates, so noise from semantically insignificant deviations in videos easily dis...
-
Knowing the distance to nearby objects is crucial for autonomous cars to navigate safely in everyday traffic. In this paper, we investigate monocular depth estimation, which advanced substantially within the last years and is providing increasingly more accurate results while only requiring a single camera image as input. In line with recent work, we use an encoder-decoder structure with so-called...
-
6D object pose estimation is an essential task in vision-based robotic grasping and manipulation. Prior works extract spatial features by fusing the RGB image and depth without considering the temporal motion information, limiting their performance in heavy occlusion robotic grasping scenarios. In this paper, we present an end-to-end model named TemporalFusion, which integrates the temporal motion...
-
Forecasting the future behavior of all traffic agents in the vicinity is a key task to achieve safe and reliable autonomous driving systems. It is a challenging problem as agents adjust their behavior depending on their intentions, the others’ actions, and the road layout. In this paper, we propose Decoder Fusion RNN (DF-RNN), a recurrent, attention-based approach for motion forecasting. Our netwo...
-
Validating the safety of autonomous systems generally requires the use of high-fidelity simulators that adequately capture the variability of real-world scenarios. However, it is generally not feasible to exhaustively search the space of simulation scenarios for failures. Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST wit...
-
Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechani...
-
On-demand ride-sharing is a promising way to improve mobility efficiency and reliability. The quality of passenger experience and the profit achieved by these platforms are strongly affected by the vehicle dispatch policy. However, existing ride-sharing research seldom considers travel time uncertainty, which leads to inaccurate dispatch allocations. This paper proposes a framework for dynamic veh...
-
LiDAR-only object detection is essential for autonomous driving systems and is a challenging problem. For the representation of a bird’s eye view LiDAR point-cloud, this paper proposes a single-stage object detector. The detector can output classification information and accurate positioning information for multi-category objects. In this paper, the detector’s design methods are detailed from a bi...
-
We propose a method to determine control input on the basis of minimizing the risk-sensitive cost function and show the results of an experiment in which the method was applied to a cooperative transportation robot system that we have developed. In the robot system, two robots hold a work object to transport without an external fixing device. The mechanism yields the force interaction between the ...
-
In this paper, we study the problem of zero-shot sim-to-real when the task requires both highly precise control with sub-millimetre error tolerance, and wide task space generalisation. Our framework involves a coarse-to-fine controller, where trajectories begin with classical motion planning using ICP-based pose estimation, and transition to a learned end-to-end controller which maps images to act...
-
The quality of life for upper limb amputees can be greatly improved by the adoption of poly-articulated myoelectric prostheses. Typically, in these applications, a pattern recognition algorithm is used to control the system by converting the recorded electromyographic activity (EMG) into complex multi-degrees of freedom (DoFs) movements. However, there is currently a trade-off between the intuitiv...
-
Opening a lock without vision sensors remains a challenge for robots. Inspired by the ability of a human to open a lock through touch and intuition, a peg-in-hole assembly method for recognizing the relative position and inclination angle of a hole is proposed. We use supervised learning to generate a contact-state model to judge the relative contact state and introduce force control strategies th...
-
A neural network based flexible object manipulation system for a humanoid robot on FPGA is proposed. Although the manipulations of flexible objects using robots attract ever increasing attention since these tasks are the basic and essential activities in our daily life, it has been put into practice only recently with the help of deep neural networks. However such systems have relied on GPU accele...
-
Autonomous vehicles must be able to understand the surrounding traffic flows and predict the future traffic conditions for planning a safe maneuver. During prediction, the action of autonomous vehicles should be considered, as it influences the interaction between vehicles sharing the same traffic scene and thus influences the future traffic flow. From this perspective, not only should the predict...
-
Contact-rich assembly tasks may result in large and unpredictable forces and torques when the locations of the contacting parts are uncertain. The ability to correct the trajectory in response to haptic feedback and accomplish the task despite location uncertainties is an important skill. We hypothesize that this skill would facilitate generalization and support direct transfer from simulations to...
-
The skin is an important organ which enables humans to interact with the unstructured environment around. It is perfectly soft and covers the entire body providing immediate feedback even when that part is not directly in the field of vision. With the human skin as an inspiration, in this paper, we develop a novel completely soft robot skin for tactile sensing. The skin utilizes a new type of mate...
-
Origami robots–often called "printable" robots– created using folding processes have gained extensive attention due to their potential for rapid and accessible design and fabrication through simple structures with complex functionalities. However, almost all origami robots require conventional rigid electronics for control, which may hinder the integration and restrict the potential of these origa...
-
Many soft robots are capable of significantly changing their shape, an ability that can offer advantages in many applications. For instance, such a robot can flatten its body to fit under small gaps and expand to move over large obstacles. Further, because these shape changes are usually driven by a pressurized fluid, if they act over a large area, they have the potential to apply large forces to ...
-
This paper presents an underactuated robotic gripper that consists of three fingers. This gripper is driven by seven actuators and capable of grasping a wide range of objects in different working scenarios. A combination of a four-bar mechanism and parallelograms ensures that each finger can provide the basic pinch grasp and power grasp. Detailed fingertip grasping force analysis shows the large p...
-
Handling of fabric is a crucial step in the manufacturing of garments. This task is typically performed by trained workers who manipulate one sheet at a time, thus introducing a bottleneck in the automation of the textile industry. This paper seeks to address the challenge of picking fabric up by proposing a new method of achieving ply-separation. Our approach relies on a finger-tip sized (2 cm2) ...
-
Differential mechanisms allow the designers of robotic and prosthetic grippers and hands to create devices that require a minimal number of motors in order to grasp a plethora of everyday life objects, leading to light-weight, compact, and low-cost implementations. The working principle of differential mechanisms is simple. They allow the distribution of the forces exerted by a single actuator to ...
-
We present an optimization-based structural-parametric synthesis method for reconfigurable closed-chain underactuated linkages for robotic systems that physically interact with the environment with an emphasis on adaptive grasping. The key idea is to implement morphological computation concepts to keep both necessary trajectory-specific holonomic constraints and mechanism adaptivity using variable...
-
Design optimization can lead to the development of robotic end-effectors with optimal grasping and dexterous, in-hand manipulation capabilities. In particular, the finger link dimensions have been identified as one of the primary design parameters that affects the performance of a robotic gripper. The ability of a gripper to manipulate objects is mainly attributed to the interaction between a set ...
-
Kinematic and force synergies can be used to reduce the complexity and dimensionality of the motion generation and control problem, as well as facilitate the mechanical implementation of robotic hands. In this paper we present a novel implementation of hardware synergies realized on the actuation level by leveraging a novel reconfigurable electric actuation topology principle. The proposed electri...
-
Current rigid linkage grippers are limited in flexibility, and gripper design optimality relies on expertise, experiments, or arbitrary parameters. Our proposed rigid gripper can accommodate irregular and off-center objects through a whippletree mechanism, improving adaptability. We present a whippletree-based rigid under-actuated gripper and its parametric design multi-objective optimization for ...
-
Over the last decade, adaptive tendon driven devices have gained an increased interest from the research community for their lightweight, compact, and affordable design features attributed to the utilisation of underactuation, differential mechanisms, and structural compliance. Although adaptive tendon driven devices are capable of efficiently executing stable grasps under significant object pose ...
-
Ankle joints play key roles in everyday locomotion, such as walking, stair climbing, and sit-to-stand. Despite the achievement in designing powered prosthetic ankles, engineers still face challenges to duplicate the full mechanics of ankle joints, including high torque, large range of motion (ROM), low profile, backdrivability, and efficiency, using electric motors and related transmissions. In th...
-
Realizing the potential of active lower-limb pros-theses to increase user mobility and efficiency requires safe, reliable, stable, and intuitive control strategies. The two prevailing classes of lower-limb prosthesis control can be categorized as volitional and non-volitional. Volitional control strategies (VCs) directly sense the user’s intentions, but this generally intuitive approach can be qui...
-
The lack of haptically aware upper-limb prostheses forces amputees to rely largely on visual cues to complete activities of daily living. In contrast, non-amputees inherently rely on conscious haptic perception and automatic tactile reflexes to govern volitional actions in situations that do not allow for constant visual attention. We therefore propose a myoelectric prosthesis system that reflects...
-
Most controllers for lower-limb robotic prostheses require individually tuned parameter sets for every combination of speed and incline that the device is designed for. Because ambulation occurs over a continuum of speeds and inclines, this design paradigm requires tuning of a potentially prohibitively large number of parameters. This limitation motivates an alternative control framework that enab...
-
The tuning process for a robotic prosthesis is a challenging and time-consuming task both for users and clinicians. An automatic tuning approach using reinforcement learning (RL) has been developed for a knee prosthesis to address the challenges of manual tuning methods. The algorithm tunes the optimal control parameters based on the provided knee joint profile that the prosthesis is expected to r...
-
This paper presents a stumble recovery controller for a knee exoskeleton that detects a stumble perturbation; selects an anticipated recovery strategy; and provides appropriate recovery assistance. In order to assess the efficacy of the controller in providing an assistive response to a stumble perturbation, the controller was implemented in a knee exoskeleton and evaluated in a single healthy adu...
-
With the recent diversification of operating devices, the demand for input operations that require confirmation of the effect of differences in display response on operability has increased. Regarding display response, previous studies have investigated the threshold time and sense of agency for a delayed response during device operation. However, these studies only focused on subjective evaluatio...
-
Situation awareness (SA) is generally considered as the perception, understanding, and projection of objects’ properties and positions. We believe if the system can sense drivers’ SA, it can appropriately provide warnings for objects that drivers are not aware of. To investigate drivers’ awareness, in this study, a human-subject experiment of driving simulation was conducted for data collection. W...
-
Developments in human-robot teaming have given rise to significant interest in training methods that enable collaborative agents to safely and successfully execute tasks alongside human teammates. While effective, many existing methods are brittle to changes in the environment and do not account for the preferences of human collaborators. This ineffectiveness is typically due to the complexity of ...
-
The long-term goal of this research is to develop methods for training propulsion during walking using robotic exoskeletons that customize their intervention based on the response of an individual.In this study, we first determined the feasibility of modeling the relationship between propulsion mechanics and parameters of a robotic intervention applied at the hip and knee joints as a Gaussian proc...
-
Robotic technologies are becoming pervasive within industrial and domestic settings, resulting in more frequent interactions between humans and robots. To ensure these interactions are effective, Human-Robot Interaction (HRI) researchers have argued that robots and humans must establish a shared common ground by communicating fundamental pieces of information to each other, such as their intention...
-
There is a growing interest in designing robots that can work alongside humans. Such robots will undoubtedly be expected to explain their behavior and decisions. While generating explanations is an actively researched topic, most works tend to focus on methods that generate explanations that are one size fits all. As in the specifics of the user-model are completely ignored. The handful of works t...
-
An important problem in designing human-robot systems is the integration of human intent and performance in the robotic control loop, especially during complex tasks. Bimanual coordination is a complex human behavior that is critical in many fine motor tasks, including robot-assisted surgery. To fully leverage the capabilities of the robot as an intelligent and assistive agent, online recognition ...
-
Robot social navigation is influenced by human preferences and environment-specific scenarios such as elevators and doors, thus necessitating end-user adaptability. State-of-the-art approaches to social navigation fall into two categories: model-based social constraints and learning-based approaches. While effective, these approaches have fundamental limitations – model-based approaches require co...
-
In this paper we report the safety-oriented framework for controlling the torque in the case of robots with high reduction gears and having no joint torque feedback. This kind of robots suffer from high joint friction and low backdrivability, requiring high gains and integral feedback, which can be dangerous. Our optimization-based framework includes feasibility and safety features borrowed from p...
-
The increasing complexity of modern robotic systems and the environments they operate in necessitates the formal consideration of safety in the presence of imperfect measurements. In this paper we propose a rigorous framework for safety-critical control of systems with erroneous state estimates. We develop this framework by leveraging Control Barrier Functions (CBFs) and unifying the method of Bac...
-
This paper studies the group multi-object tracking (MOT) problem in dynamic pedestrian environments, with intended application to safe navigation for autonomous vehicles. We complete a full autonomous vehicle navigation pipeline from object detection, tracking, grouping, to risk map generation and safe path planning. Our main contribution is to instantiate a group multi-object tracking algorithm, ...
-
Reactive and safe agent modellings are important for nowadays traffic simulator designs and safe planning applications. In this work, we proposed a reactive agent model which can ensure safety without comprising the original purposes, by learning only high-level decisions from expert data and a low level decentralized controller guided by the jointly learned decentralized barrier certificates. Emp...
-
Humanoid robots in unknown environments need to be able to quickly react to contacts in order to ensure safety of humans and their own hardware. For showing useful reactions to contacts, the robot needs information about possibly multiple contacts such as their respective contact locations and wrenches. In this paper, we introduce our algorithm rm-Code, a real-time multi-contact detection, isolati...
-
Spiking Neural Networks (SNNs) aim at providing energy-efficient learning capabilities when implemented on neuromorphic chips with event-based Dynamic Vision Sensors (DVS). This paper studies the robustness of SNNs against adversarial attacks on such DVS-based systems, and proposes R-SNN, a novel methodology for robustifying SNNs through efficient DVS-noise filtering. We are the first to generate ...
-
Cooperative transportation by multi-aerial robots has the potential to support various payloads and improve fail- safe against dropping. Furthermore, changing the attachment positions of robots according payload characteristics increases the stability of transportation. However, there are almost no transportation systems capable of scaling to the payload weight and size and changing the optimal at...
-
Aerial manipulators have great potential in accomplishing a variety of aerial tasks. One class of aerial manipulators, multi-UAV parallel robots, consists of multiple UAVs connected to a payload or an end-effector by passive kinematic chains. The primary limitation of such aerial manipulators is the dependence on motion capture (MOCAP) systems that provide precise and high-rate exteroceptive pose ...
-
Micro Aerial Vehicles (MAV) with Vertical Takeoff and Landing (VTOL) capabilities, such as quadrotors, have offered significant value to many research fields and markets. However, only recently, MAV began to be explored as systems capable of interacting with the environment, performing manipulation tasks, and participating in more versatility-demanding operations. Pursuing the goal of turning flyi...
-
In the last decade, autonomous vertical take-off and landing (VTOL) vehicles have become increasingly important as they lower mission costs thanks to their re-usability. However, their development is complex, rendering even the basic experimental validation of the required advanced guidance and control (G & C) algorithms prohibitively time-consuming and costly. In this paper, we present the design...
-
In recent times, quadrotors have become immensely applicable in scenarios such as relief operations, infrastructure maintenance, search-and-rescue missions etc. A key control design challenge arises in these applications when the quadrotor has to manoeuvre through constrained spaces such as narrow windows, pipelines in the presence of external disturbances and parametric uncertainties: such condit...
-
Flapping-wing robots (so-called ornithopters) are a promising type of platform to perform efficient winged flight and interaction with the environment. However, the control of such vehicles is challenging due to their under-actuated morphology to meet lightweight requirements. Consequently, the flight control of flapping-wing robots is predominantly handled by the tail. Most ornithopters feature a...
-
This paper investigates the task coordination of multi-robot where each robot has a private individual temporal logic task specification; and also has to jointly satisfy a globally given collaborative temporal logic task specification. To efficiently generate feasible and optimized task execution plans for the robots, we propose a hierarchical multi-robot temporal task planning framework, in which...
-
This paper presents Reduced State Value Iteration (RSVI), an algorithm to compute policies for Markov Decision Processes (MDPs) that have natural checkpoints, allowing for a solution based on a reduced state space. The algorithm is applied to find policies for multiple drones to persistently surveil an environment subject to charging constraints. RSVI leverages the structure of the true MDP to bui...
-
In this paper, we propose an evolutionary algorithm for solving the multi-robot orienteering problem where a team of cooperative robots aims to maximize the total information collected by visiting a subset of given nodes within a fixed budget on travel costs. Multi-robot orienteering problems are relevant to applications such as logistic delivery services, precision agriculture, and environmental ...
-
We propose a novel theoretical approach for solving a Stochastic Security Game using augmented Markov Decison Processes and an experimental evaluation. Most of the previous works mentioned in the literature focus on Linear Programming techniques seeking Strong Stackelberg Equilibria through the defender and attacker’s strategy spaces. Although effective, these techniques are computationally expens...
-
As robotic technologies advance and robots move out of factories and labs into the real world, grip on a variety of surfaces (e.g. smooth or rough) in a variety of conditions (e.g. dry or wet) becomes increasingly important. Bioinspired "microspines" have been previously explored, but primarily for vertical climbing applications or for small-scale robots applying low forces (less than 1 N). Furthe...
-
Inspired by bat flight performance, we explore the advantages of wing twist and fold for flapping wing robots. For this purpose, we develop a dynamical model that incorporates these two degrees of freedom to the wing. The twist is assumed to be linearly-increasing along the wing, while the wing fold is modeled as a relative rotation of the handwing with respect to the armwing. An optimization sche...
-
The bones, muscles, tendons and connective tissues form a continuous tension network throughout human body. This heterogeneous mixture presents the characteristics of tensegrity, providing the body with structurally integrity, stability and flexibility. Inspired by this, this paper proposes a novel soft robotic gripper based on tensegrity structures. Firstly, the design and working principle of th...
-
In this paper, we propose an automatic robot-rat interaction framework that enables a robotic rat to realize real-time localization, tracking and movement analysis of a laboratory rat. Specifically, we combine an object detector with stereo matching to achieve fast localization of the laboratory rat. Combined with the rat-like motion of the robot, one-step tracking of the rat is achieved, which en...
-
Legged robots have opened their way to more stable and practical mobile robot applications. However, their locomotion strategies are limited to similar patterns, and dynamic running at high speed still is not successfully realized. One of the key technology required for the realization of the dynamic running of the legged robot is to estimate the ground contact force and control it in real-time. T...
-
With the gradual maturity of the software and hardware of quadruped robots, the application scenarios of quadruped robots are increasing, such as security, rescue, exploration and other tasks. Quadruped robots are flexible and adaptive to challenging or complex environment. This study presents a large-scale quadruped robot, Pegasus II, which is a new version upgraded from the previous quadruped ro...
-
This paper studies stabilizer design for position-controlled humanoid robots. Stabilizers are an essential part for position-controlled humanoids, whose primary objective is to adjust the control input sent to the robot to assist the tracking controller to better follow the planned reference trajectory. To achieve this goal, this paper develops a novel force-feedback based whole-body stabilizer th...
-
Adaptive control can address model uncertainty in control systems. However, it is preliminarily designed for tracking control. Recent advancements in the control of quadruped robots show that force control can effectively realize agile and robust locomotion. In this paper, we present a novel adaptive force-based control framework for legged robots. We introduce a new architecture in our proposed a...
-
This paper presents a control strategy for quadruped robots to hop on their rear legs in three-dimensional space. The proposed approach generates nominal center of mass (CoM) trajectories based on a template spring-loaded inverted pendulum (SLIP) model. Tracking this reference remains a challenge due to the underactauted nature of balance with point feet. To address this challenge, a control-Lyapu...
-
Compared with torque-control techniques, a position-controlled quadruped robot is lower cost, easier to build, and more direct to drive. However, the stiff actuation of position-controlled actuators makes it difficult for the quadruped to achieve dynamically stable locomotion. This paper presents an implementation of joint velocity programming technique to regulate the body’s moving speed and orie...
-
Micro-nano-robots are considered to be a promising platform for drug delivery in biological organisms, but there are still urgent technical problems in biocompatibility and degradability of 3D-printed-based micro-robots that need to be solved. Therefore, in this paper, we design a magnetized bio-hybrid robot, which uses mouse macrophages as carriers, and allowed it to swallow Fe2O3 particles with ...
-
Embryos/oocytes vitrification is an essential cryopreservation technique in IVF (in vitro fertilization) clinics. The reliable and effective transferring of embryos/oocytes is crucial to the subsequent steps in the whole procedure of vitrification. After each transferring, the straw needs to be replaced with a new one. Due to the uncertainties in the fabrication and installation, the exact knowled...
-
Mechanical rubbing of blood clot using miniature magnetic helical robots is a potential way for thrombolysis. In this paper, we report a new strategy for this issue based on mobile coils. Previously, we proposed the concept of magnetic actuation with parallel mobile coils, in which multiple coils can move in 3D space. Enabled by mobility of the coils, additional degree-of-freedom (DOF) could be ut...
-
This paper presents an adaptive backstepping controller for the reference tracking of an alginate artificial cell. An adaptive controller was implemented to precisely manipulate a magnetic artificial cell actuated by rotating magnetic fields. The rolling motion of a small-scale robot in a fluidic environment is challenging, especially when the fluid imparts an unknown response at low Reynolds numb...
-
Optoelectronic tweezer-driven microrobots (OETdMs) are a versatile micromanipulation technology based on the application of light induced dielectrophoresis to move small dielectric structures (microrobots) across a photoconductive substrate. The microrobots in turn can be used to exert forces on secondary objects and carry out a wide range of micromanipulation operations, including collecting, tra...
-
Moths use hundreds of strain sensors (campaniform sensilla) on each wing to quickly respond to perturbations that may otherwise destabilize the moth during flight. A similar sensing approach could help stabilize micro-aerial vehicles (MAVs), but large sensor arrays are challenging due to the wiring and large latency that exists when capturing data from many traditional strain sensors. This work in...
-
Robotic simulators are crucial for academic research and education as well as the development of safety-critical applications. Reinforcement learning environments— simple simulations coupled with a problem specification in the form of a reward function—are also important to standardize the development (and benchmarking) of learning algorithms. Yet, full-scale simulators typically lack portability ...
-
We present iGibson 1.0, a novel simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes. Our environment contains 15 fully interactive home-sized scenes with 108 rooms populated with rigid and articulated objects. The scenes are replicas of real-world homes, with distribution and the layout of objects aligned to those of the real world. iGibson 1.0...
-
Evaluation of social robot navigation inherently requires human input due to its qualitative nature. Motivated by the need to scale human evaluation, we propose a general method for deploying interactive, rich-client robotic simulations on the web. Prior approaches implement specific web- compatible simulators or provide tools to build a simulator for a specific study. Instead, our approach builds...
-
We consider the problem of co-designing embodied intelligence as a whole in a structured way, from hardware components such as propulsion systems and sensors to software modules such as control and perception pipelines. We propose a principled approach to formulate and solve complex embodied intelligence co-design problems, leveraging a monotone co-design theory. The methods we propose are intuiti...
-
Prior experiments have shown that human gait synchronizes to periodic torque pulses applied about the hip and ankle joints by robotic exoskeletons. Importantly, entrainment occurred even when the pulse period differed slightly from the user’s preferred stride period, making it a viable approach to increase gait speed. As gait speed is an important outcome of gait therapy, gait entrainment to mecha...
-
Simulation is an essential tool for developing robotic systems; however it is computationally intensive and does not scale well, especially for multi-robot scenarios. In this paper we introduce daß – a system that facilitates distributable and scalable simulations of robotic applications. It overcomes many of the performance and quality limitations involved in common multi-robot simulation scenari...
-
The system design and algorithm development of mobile 3D printing robots need a realistic simulation. They require a mobile robot simulation platform to interoperate with a physics-based material simulation for handling interactions between the time-variant deformable 3D printing materials and other simulated rigid bodies in the environment, which is not available for roboticists yet. To bridge th...
-
Identification of textile properties is an important milestone toward advanced robotic manipulation tasks that consider interaction with clothing items such as assisted dressing, laundry folding, automated sewing, textile recycling and reusing. Despite the abundance of work considering this class of deformable objects, many open problems remain. These relate to the choice and modelling of the sens...
-
Reinforcement Learning (RL) is a powerful mathematical framework that allows robots to learn complex skills by trial-and-error. Despite numerous successes in many applications, RL algorithms still require thousands of trials to converge to high-performing policies, can produce dangerous behaviors while learning, and the optimized policies (usually modeled as neural networks) give almost zero expla...
-
Adopting the ideal reliable machine model, the throughput of a lithography machine can be given as the reciprocal of the operation time. This time can be defined at the die level where the actual exposure process takes place as the time unit per die. A closer look at the motion profiles, namely step-and-scan trajectories, suggests that a multi-disciplinary design optimization should be involved wh...
-
We propose an energy-efficient controller to minimize the energy consumption of a mobile robot by dynamically manipulating the mechanical and computational actuators of the robot. The mobile robot performs real-time vision-based applications based on an event-based camera. The actuators of the controller are CPU voltage/frequency for the computation part and motor voltage for the mechanical part. ...
-
Autonomous vehicles usually consume a large amount of computational power for their operations, especially for the tasks of sensing and perception with artificial intelligence algorithms. Such a computation may not only cost a significant amount of energy but also cause performance issues when the onboard computational resources are limited. To address this issue, this paper proposes an adaptive o...
-
This paper proposes the design of a new robot end-effector based on compliant constant-force mechanism for robot-assisted manufacturing, such as polishing. One uniqueness of the proposed end-effector lies in that it offers a constant contact force without using a force sensor and controller. An industrial robot is adopted to position the end-effector and the end-effector regulates the contact forc...
-
Execution of automatically generated programs for accurate robotic machining requires the generated trajectories to be not only accurate with respect to the work piece, but also that the trajectories are continuous differentiable (C1) while avoiding unnecessary large curvatures leading to large accelerations that could compromise machining quality or speed. A widely used work piece representation ...
-
We propose a multi-camera simultaneous localization and mapping (SLAM) system using the Manhattan constraint to support automated valet parking. The proposed method uses multiple cameras to expand the system field of view, to improve the robustness of the SLAM system in textureless regions, where point features from different cameras are jointly optimized by a uniform cost function. To improve glo...
-
We present a method for efficiently exploring highly convoluted environments. The method incorporates two planning stages - an exploration stage for extending the boundary of the map, and a relocation stage for explicitly transiting the robot to different sub-areas in the environment. The exploration stage develops a local Rapidly-exploring Random Tree (RRT) in the free space of the environment, a...
-
Dynamic environments that include unstructured moving objects pose a hard problem for Simultaneous Localization and Mapping (SLAM) performance. The motion of rigid objects can be typically tracked by exploiting their texture and geometric features. However, humans moving in the scene are often one of the most important, interactive targets – they are very hard to track and reconstruct robustly due...
-
In object-based Simultaneous Localization and Mapping (SLAM), 6D object poses offer a compact representation of landmark geometry useful for downstream planning and manipulation tasks. However, measurement ambiguity then arises as objects may possess complete or partial object shape symmetries (e.g., due to occlusion), making it difficult or impossible to generate a single consistent object pose e...
-
Underwater scenarios are challenging for visual Simultaneous Localization and Mapping (SLAM) due to limited visibility and intermittently losing structures in image views. In this paper, we propose a visual acoustic bundle adjustment system which fuses a camera and a Doppler Velocity Log (DVL) in a graph SLAM framework for reliable underwater localization and mapping. In order to fuse the vision w...
-
SLAM in deformable environments is a very challenging research topic. Some research works have been presented by different research groups in the past few years. However, there are still some challenging research questions remaining unanswered. This paper discusses some of these research questions focusing on the case when point features are used to describe the deformable environments. The SLAM p...
-
This study proposes an adaptive data-driven hy-perparameter tuning framework for black-box 3D LiDAR odometry algorithms. The proposed framework comprises offline parameter-error function modeling and online adaptive parameter selection. In the offline step, we run the odometry estimation algorithm for tuning with different parameters and environments and evaluate the accuracy of the estimated traj...
-
Learning from Demonstration (LfD) provides an intuitive and fast approach to program robotic manipulators. Task parameterized representations allow easy adaptation to new scenes and online observations. However, this approach has been limited to pose-only demonstrations and thus only skills with spatial and temporal features. In this work, we extend the LfD framework to address forceful manipulati...
-
Teaching robots how to apply forces according to our preferences is still an open challenge that has to be tackled from multiple engineering perspectives. This paper studies how to learn variable impedance policies where both the Cartesian stiffness and the attractor can be learned from human demonstrations and corrections with a user-friendly interface. The presented framework, named ILoSA, uses ...
-
We focus on multiple experts performing a task in a Markov decision process (MDP) environment. A probabilistic assignment of trajectories to clusters and a mathematical framework which leverages the utility function are employed to jointly estimate the discount factor and reward. We treat the number of clusters as a hyperparameter which can be "freely" selected by the problem designer. In this wor...
-
Generating robot motion that fulfills multiple tasks simultaneously is challenging due to the geometric constraints imposed on the robot. In this paper, we propose to solve multi-task problems through learning structured policies from human demonstrations. Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces. The policy structure provides the...
-
We are interested in learning from demonstration (LfD) that can both learn and execute a trajectory and evaluate the quality of a previously unseen trajectory in the domain of assistive robotics. To this end, we propose a novel continuous inverse optimal control (IOC) formulation that simultaneously learns an optimal time-invariant controller and an evaluation metric from human demonstrations. We ...
-
Learning from Demonstration (LfD) is a popular approach that allows humans to teach robots new skills by showing the correct way(s) of performing the desired skill. Human-provided demonstrations, however, are not always optimal and the teacher usually addresses this issue by discarding or replacing sub-optimal (noisy or faulty) demonstrations. We propose a novel LfD representation that learns from...
-
The purpose of this paper is to present a novel curved Gaussian Mixture Model (CGMM) and to study the application of it in motion skill encoding. Primarily, Gaussian mixture model (GMM) has been widely applied on many occasions when a probability density function is needed to approximate a complex probability distribution. However, GMM cannot efficiently approach highly non-linear distributions. T...
-
Learning from visual data opens the potential to accrue a large range of manipulation behaviors by leveraging human demonstrations without specifying each of them mathe-matically, but rather through natural task specification. In this paper, we present Learning by Watching (LbW), an algorithmic framework for policy learning through imitation from a single video specifying the task. The key insight...
-
Conventional works that learn grasping affordance from demonstrations need to explicitly predict grasping configurations, such as gripper approaching angles or grasping preshapes. Classic motion planners could then sample trajectories by using such predicted configurations. In this work, our goal is instead to fill the gap between affordance discovery and affordance-based policy learning by integr...
-
Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation. Surprisingly, an extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing the robot to complete a task regardless of its view of the scene. In t...
-
Behavioral cloning (BC) bears a high potential for safe and direct transfer of human skills to robots. However, demonstrations performed by human operators often contain noise or imperfect behaviors that can affect the efficiency of the imitator if left unchecked. In order to allow the imitators to effectively learn from imperfect demonstrations, we propose to employ the robust t-momentum optimiza...
-
Imitation learning (IL) frameworks in robotics typically assume that a domain expert's demonstration always contains a correct way of doing the task. Despite its theoretical convenience, this assumption has limited practical values for an IL-powered robot in real world. There are many reasons for an expert in the real world to provide demonstrations that may contain incorrect or potentially unsafe...
-
Modern model-free reinforcement learning methods have recently demonstrated impressive results on a number of problems. However, complex domains like dexterous manipulation remain a challenge due to the high sample complexity. To address this, current approaches employ expert demonstrations in the form of state-action pairs, which are difficult to obtain for real-world settings such as learning fr...
-
Forecasting future trajectories of multiple pedestrians in a crowded environment is a challenging problem due to the complex interactions among the pedestrians. The interactions can be asymmetric and their influences may vary over time. Moreover, each pedestrian can exhibit different behavior at any given time and context and thus they may have multiple future possible trajectories. In this work, ...
-
Being able to safely operate for extended periods of time in dynamic environments is a critical capability for autonomous systems. This generally involves the prediction and understanding of motion patterns of dynamic entities, such as vehicles and people, in the surroundings. Many motion prediction methods in the literature implicitly account for environmental factors by learning on observed moti...
-
We propose a lightweight retrieval-based pipeline to predict 6DOF camera poses from RGB images. Our pipeline uses a convolutional neural network (CNN) to encode a query image as a feature vector. A nearest neighbor lookup finds the pose-wise nearest database image. A siamese convolutional neural network regresses the relative pose from the nearest neighboring database image to the query image. The...
-
We investigate the capacitated vehicle routing problem (CVRP) under a robotics context, where a vehicle with limited payload must complete delivery (or pickup) tasks to serve a set of geographically distributed customers with varying demands. In classical CVRP, a customer location is modeled as a point. In many robotics applications, however, it is more appropriate to model such "customer location...
-
Path planning for multiple robots (MRPP) represents a task of finding non-colliding paths for robots via which they can navigate from their initial positions to specified goal positions. The problem is often modeled using undirected graphs where robots move between vertices across edges while no two robots can simultaneously occupy the same vertex nor can traverse an edge in opposite directions. C...
-
Agent control among pedestrians is often approached in one of the three following ways: using predefined behaviors for agent navigation, learning navigation behaviors from data, or search-based planning on a graph where each edge is a feasible action chosen from a set of predefined actions. While the first approach often produces natural looking motions and the second learns and utilizes complex i...
-
We consider the Stochastic Orienteering Problem with random traversal time for edges. In this scenario the length of the path is a random variable and we consider a formulation with chance constraints, i.e., a bound on the probability that the length of the path exceeds the allotted budget. Our proposed solution casts the problem as an instance of a suitably defined Constrained Markov Decision Pro...
-
In this paper, we revisit the long-standing Traveling Salesman Problem (TSP) and focus on the challenging, yet practical route planning problem with limited computational resources. We make contributions to TSP, one of the most famous NP-hard problems by providing a new improved approximate solution, which we term TOpology Preserving Self-Organizing Map (TOPSOM). TOPSOM well preserves the topology...
-
The future motion of traffic participants is inherently uncertain. To plan safely, therefore, an autonomous agent must take into account multiple possible trajectory outcomes and prioritize them. Recently, this problem has been addressed with generative neural networks. However, most generative models either do not learn the true underlying trajectory distribution reliably, or do not allow predict...
-
A novel 6-DOF parallel manipulator with two spherical-universal-revolute limbs is proposed in this work. Compared with general 6-DOF parallel manipulators of six kinematic limbs, this new manipulator actuated by spherical motion generators has only two limbs, which brings kinematic advantages such as small footprint and large workspace. The inverse position problem of the manipulator is solved by ...
-
In this paper, a new robust model-based super-twisting algorithm is proposed as a control solution for parallel kinematic manipulators (PKMs). The conventional super-twisting algorithm for robot manipulators has the structure of a computed-torque control which can be sensitive to measurement noise. This issue may deteriorate the dynamic performance of the manipulator and reduce its robustness towa...
-
Cable-Driven Parallel Robots (CDPRs) are parallel robots with rigid links replaced by cables. As for most parallel robots the determination of the analytical solutions to the direct geometrico-static model (DGSM) is a difficult task that is often not feasible online. However, the knowledge of the moving-platform (MP) pose is necessary in order to control the CDPR, e.g. with visual servoing. When t...
-
Object permanence in psychology means knowing that objects still exist even if they are no longer visible. It is a crucial concept for robots to operate autonomously in uncontrolled environments. Existing approaches learn object permanence from low-level perception, but perform poorly on more complex scenarios, like when objects are contained and carried by others. Knowledge about manipulation act...
-
Predicting the motion of observed entities benefits humans almost seamlessly. The same benefits can be proliferated to mobile autonomous systems if we have a reliable, real-time solution to predict the motion of any object of interest, be it the host’s own motion or that of an observed foreign object. In this work, a novel Multi-Variable State Prediction (MVSP) methodology is devised for real-time...
-
Human trajectory prediction has received increased attention lately due to its importance in applications such as autonomous vehicles and indoor robots. However, most existing methods make predictions based on human-labeled trajectories and ignore the errors and noises in detection and tracking. In this paper, we study the problem of human trajectory forecasting in raw videos, and show that the pr...
-
Tracking the 6D pose of objects in video sequences is important for robot manipulation. Most prior efforts, however, often assume that the target object's CAD model, at least at a category-level, is available for offline training or during online template matching. This work proposes BundleTrack, a general framework for 6D pose tracking of novel objects, which does not depend upon 3D models, eithe...
-
Estimating the states of surrounding traffic participants stays at the core of autonomous driving. In this paper, we study a novel setting of this problem: model-free single-object tracking (SOT), which takes the object state in the first frame as input, and jointly solves state estimation and tracking in subsequent frames. The main purpose for this new setting is to break the strong limitation of...
-
Multi-object tracking is a critical component in autonomous navigation, as it provides valuable information for decision-making. Many researchers tackled the 3D multi-object tracking task by filtering out the frame-by-frame 3D detections; however, their focus was mainly on finding useful features or proper matching metrics. Our work focuses on a neglected part of the tracking system: score refinem...
-
Lenticular angle gauge (LEAG) is a planar pattern that visualizes relative attitude by the position of the black line, which moves according to the viewing angle. The authors developed a new LEAG in which the direction of movement of the black line is 90◦ different from the previous LEAG, and used it to develop a non-square high-accuracy fiducial marker. The new marker realized accurate pose estim...
-
We present a decentralized collision avoidance method for dense environments based on buffered Voronoi cells (BVC) and reciprocal velocity obstacles (RVO). Our approach is designed for scenarios with a large number of agents in close proximity and provides passive-friendly collision avoidance guarantees. The Voronoi cells are superimposed with RVO cones to compute a suitable direction for each age...
-
Despite significant advancements in the field of multi-agent navigation, agents still lack the sophistication and intelligence that humans exhibit in multi-agent settings. In this paper, we propose a framework for learning a human-like general collision avoidance policy for agent-agent interactions in fully decentralized, multi-agent environments. Our approach uses knowledge distillation with rein...
-
Efficient trajectory optimization is essential for avoiding collisions in unstructured environments, but it remains challenging to have both speed and quality in the solutions. One reason is that second-order optimality requires calculating Hessian matrices that can grow with O(N2) with the number of waypoints. Decreasing the waypoints can quadratically decrease computation time. Unfortunately, fe...
-
Artificial potential fields (APFs) and their variants have been a staple for collision avoidance of mobile robots and manipulators for almost 40 years. Its model-independent nature, ease of implementation, and real-time performance have played a large role in its continued success over the years. Control barrier functions (CBFs), on the other hand, are a more recent development, commonly used to g...
-
In this paper, we propose a map-based end-to-end DRL approach for three-dimensional (3D) obstacle avoidance in a partially observed environment, which is applied to achieve autonomous navigation for an indoor mobile robot using a depth camera with a narrow field of view. We first train a neural network with LSTM units in a 3D simulator of mobile robots to approximate the Q-value function in double...
-
It is challenging for a mobile robot to navigate through human crowds. Existing approaches usually assume that pedestrians follow a predefined collision avoidance strategy, like social force model (SFM) or optimal reciprocal collision avoidance (ORCA). However, their performances commonly need to be further improved for practical applications, where pedestrians follow multiple different collision ...
-
Trajectory tracking in the orientation space utilizing unit quaternions yields non linear error dynamics as opposed to Cartesian position. In this work, we study trajectory tracking in the orientation space utilizing the most popular quaternion error representations and angular velocity errors. By selecting error functions carefully we show exponential convergence in a region of attraction contain...
-
Hybrid robots incorporate the advantages of both aerial-only and terrestrial-only vehicles to achieve enhanced mobility and better energy efficiency. Among hybrid vehicles, spherical robots offer the best maneuverability. While operating on uneven surfaces is one of the main benefits of spherical robots, the current literature only covers control of these robots on flat surfaces. This work present...
-
A formulation of the area defense and surveillance problem for one intruder and one defense and surveillance robot and its corresponding solution using control barrier functions is presented. The defense robot must follow the intruder as it moves through a rectangular region in the plane, ensuring that the position of the intruder is also within a rectangular region attached to the surveillance ro...
-
Unknown properties of a robot’s environment are one of the sources of uncertainty in autonomous navigation. This uncertainty has to be accounted for when modelling robot dynamics. For ground vehicles in particular, terrain structure is one of the main environmental factors that can strongly influence the dynamics. Therefore, to ensure the ability of a robot to safely and efficiently navigate new e...
-
Shape Memory Alloys (SMAs) are a type of smart material that are commonly used in compact and lightweight linear actuators. Paired with compliant motion conversion mechanisms, these alloys can be transformed into lightweight grippers ideal for applications such as drone deliveries. This work proposes to use the inherent stiffness of a compliant mechanism with an SMA coil, to design and size a self...
-
In this paper, a compliant five-bar leg mechanism is proposed, designed and manufactured for heavy-load legged robots, by using two magneto-rheological actuators (MRAs) that are capable of offering a maximal torque of 78Nm. To address the rate-dependent hysteresis of the MRA, a hybrid rate-dependent hysteresis model is derived based on the idea of mappings between different hysteresis loops. With ...
-
In this paper, we introduce a new type of compliant actuator named the Parallel Variable Stiffness Actuator (PVSA) which consists of a variable stiffness spring placed in parallel with a direct-drive motor. Parallel variable stiffness actuators provide (i) high-fidelity force control and (ii) controllable energy storage, as they inherit the benefits of direct-drive motors and variable stiffness sp...
-
Theory suggests a linear relation between stiffness and the energy stored by a linear helical spring at constant deformation. This relation implies that increasing the stiffness of a helical spring upon deformation requires more energy at larger deformations. State-of-the-art variable stiffness spring actuators, used to drive robots and human assistive and augmentation devices, are characterized b...
-
High precision industrial applications call for equally precise functioning of industrial manipulators, which in turn requires accurate modeling of the manipulators. This paper carries out a detailed study on the modeling of industrial manipulators with elastic joints to improve their accuracy. In particular, the effect of adopting a simple harmonic drive (HD) model and ignoring a dynamic effect c...
-
Dexterous manipulation tasks often require contact switching, where fingers make and break contact with the object. We propose a method that plans trajectories for dexterous manipulation tasks involving contact switching using contact-implicit trajectory optimization (CITO) augmented with a high-level discrete contact sequence planner. We first use the high-level planner to find a sequence of fing...
-
Whole-arm manipulation (WAM) is often used to manipulate large and bulky objects. Contact point-based methods for generating the robot configurations for WAM mostly search for suitable contact points and configurations simultaneously. However, in order to learn good contact points, or allow an operator to select them, inverse kinematics (IK) solvers are needed which take such points along with a s...
-
We present a Virtual Kinematic Chain (VKC) perspective, a simple yet effective method, to improve task planning efficacy for mobile manipulation. By consolidating the kinematics of the mobile base, the arm, and the object being manipulated collectively as a whole, this novel VKC perspective naturally defines abstract actions and eliminates unnecessary predicates in describing intermediate poses. A...
-
This paper presents a motion planning algorithm that enables robots to efficiently pick up objects by considering simultaneous multi-object grasping. At the center of the algorithm is a cost function that helps to determine one of the following three grasping policies considering distance and friction constraints – Grasping a single object; Grasping two objects simultaneously; Grasping two object ...
-
In this work we tackle the path planning problem for a 21-dimensional snake robot-like manipulator, navigating a cluttered gas turbine for the purposes of inspection. Heuristic search based approaches are effective planning strategies for common manipulation domains. However, their performance on high dimensional systems is heavily reliant on the effectiveness of the action space and the heuristic...
-
Motion control and planning for the manipulator are critical components in manipulator teleoperation. Online (real-time) motion control is challenging for active obstacle avoidance and often results in fluctuating and unsafe motion. Offline motion planning, on the other hand, generates precise and secure trajectories for complex manipulation. In this paper, a real-time nonlinear model predictive c...
-
For the visually impaired people, some outdoor activities like running or soccer are difficult, due to not being able to clearly see the environment. Recently, multiple researchers have contributed to help the visually impaired people run outdoors using robotic systems with different types of feedback, such as auditory feedback and haptic feedback. They discovered that using robotic systems can be...
-
Origami robots composed of rigid parts with flexible joints have inherent compliance that enables deployment and reconfiguration for various shape adaptations. The major drawback of such mechanical compliance is its intrinsic softness and lack of controllability of this stiffness. In this work, we propose a design of variable stiffness origami joints to be integrated into large scale origami syste...
-
Traditionally vibrotactile feedback delivered by haptic interfaces is used to provide additional support for visual interaction via prehensile object manipulation using fingers. Nevertheless, haptic stimuli can be also applied for non-prehensile interaction that involves movements of the elbow joint. In this paper, we have designed a table-top haptic device that provides vibrotactile stimulation t...
-
Two-Stage Optimization of a Reconfigurable Asymmetric 6-DOF Haptic Robot for Task-Specific Workspace
Parallel mechanisms (PMs) are commonly used for developing haptic devices due to low inertia, high rigidity and precision. However, limited workspace impedes their application for task-oriented robotic therapy which generally requires large motion ranges. To solve this problem, first, a PM- based reconfigurable asymmetric 6-DOF haptic interface was presented, and then a two-stage optimization meth...
-
We present a novel haptic teleoperation approach that considers not only the safety but also the stability of a teleoperation system. Specifically, we build upon previous work on haptic shared control, which generates a reference haptic feedback that helps the human operator to safely navigate the robot but without taking away their control authority. Crucially, in this approach the force rendered...
-
Human-in-the-loop telerobotic systems (HiLTS) are robotic tools designed to extend and in some circumstances improve the dexterous capabilities of the human operator in virtual and remote environments. Dexterous manipulation, however, depends on how well the telerobot is incorporated into the operator’s sensorimotor control scheme. Empirical evidence suggests that haptic feedback can lead to impro...
-
Maintaining lateral and longitudinal trajectory tracking accuracy is challenging for autonomous ground vehicles (AGVs). This paper considers kinematics and dynamics of longitudinal and lateral motion to form a novel composite structure considering the cross-impacts of acceleration and steering commands on tracking errors in the lateral and longitudinal directions, respectively. The multi-tiered st...
-
Path tracking controllers for non-holonomic vehicles are commonly designed with primary focus on robustness to different kinds of disturbances, vehicle dynamics and other effects. Subsequently, the path tracking behavior is improved by parameter optimization or controller extensions. The possible improvement, however, is already limited by the control design itself. To overcome this drawback, this...
-
Our paper proposes a model predictive controller as a single-task formulation that simultaneously optimizes wheel and torso motions. This online joint velocity and ground reaction force optimization integrates a kinodynamic model of a wheeled quadrupedal robot. It defines the single rigid body dynamics along with the robot’s kinematics while treating the wheels as moving ground contacts. With this...
-
This article presents a novel transformable wheeled robot with three motion modes. Based on the four-bar mechanism design, the transformable wheel can transform into CW (clockwise) legged wheel mode and CCW (counterclock-wise) legged wheel mode from the circular wheel. Both legged wheel modes achieve good obstacle-climbing performance when the robot overcomes obstacles such as steps in the forward...
-
In this study, we propose a predictive model composed of a recurrent neural network including parametric bias and stochastic elements, and an environmentally adaptive robot control method including variance minimization using the model. Robots which have flexible bodies or whose states can only be partially observed are difficult to modelize, and their predictive models often have stochastic behav...
-
This paper presents a framework for trajectory planning that explicitly considers robotic traversability based on a quasi-dynamic vehicle model of a mobile robot in loose soil. The quasi-dynamic model estimates the slip effect due to wheel-terrain interaction forces regardless of solving complicated multibody dynamics. Therefore, our proposed model is computationally efficient for quantifying how ...
-
Marine Autonomous Navigation for Biomimetic Underwater Robots Based on Deep Stereo Attention Network
This paper proposes a multi-objective visionbased navigation network for biomimetic underwater robots to cope with scientific observation, target selection, and obstacle avoidance in marine missions. Structurally, a stereo block attention module is first constructed to serially extract the channel and spatial attention portion of the real-time visual feedback. Next, the parallax attention mechanis...
-
Bat flight has been an underdeveloped area of bio-inspired robotics because of the vast complexities of biological bat flight and the over 40 degrees of freedom present in their bodies. The robotic flapping system Bat Bot (B2) has been shown to exhibit fundamental properties of biological bat flight with its articulated wings, its deformable membrane, and its controllable hindlimbs. However, the s...
-
Locating odour sources with mobile robots is a difficult task with many real world applications. Over the years, researchers have devised bio-inspired and cognitive methods to enable mobile robots to fulfil this task. One of the most popular cognitive approaches is Infotaxis, which computes a probability map for the location of the chemical source and, on each time step, moves the robot in the dir...
-
This paper shows analytical and experimental evidence of using the vibration dynamics of a compliant whisker for accurate terrain classification during steady state motion of a mobile robot. A Hall effect sensor was used to measure whisker vibrations due to perturbations from the ground. Analytical results predict that the whisker vibrations will have one dominant frequency at the vertical perturb...
-
Robust top-down and bottom-up visual saliency for mobile robots using bio-inspired design principles
Modern camera systems in robotics tend to pro-duce overwhelming amounts of visual information due to their high resolutions and high frame rates. This raises a fundamental question of how robots should focus attention on a region of the visual scene, and how they should process information in the periphery. This is particularly an issue for mobile robots, where the computational resources of low-p...
-
One way to create a quadruped galloping robot from scratch is to design a brick-shaped body and utilize relatively simple open-chain leg mechanisms controlled with relatively complex control algorithms. Alternatively, we can look at how nature solved the same task designing fast mammals such as cheetah, and by means of morphological computation, we can design a complex mechanical system that has m...
-
This paper proposes a bioinspired adaptive anchoring module that can be integrated into robots to enhance their mobility and manipulation abilities. The design of the module is inspired by the structure of the mouth in Chilean lamprey (Mordacia lapicida) where a combination of suction and several arrays of teeth with different sizes around the mouth opening is used for catching preys and anchoring...
-
Quadruped locomotion is a challenging task for learning-based algorithms. It requires tedious manual tuning and is difficult to deploy in reality due to the reality gap. In this paper, we propose a quadruped robot learning system for agile locomotion which does not require any pre-training and works well in various real-world terrains. We introduce a hierarchical learning framework that uses reinf...
-
The availability of inexpensive 3D-printed quadrupedal robots motivates the development of learning-based methods compatible with low-cost embedded processors and position-controlled hobby servos. In this work, we show that a linear policy is sufficient to modulate an open-loop trajectory generator, enabling a quadruped to walk over rough, unknown terrain, with limited sensing. The policy is train...
-
Functional autonomous systems often realize complex tasks by utilizing state machines comprised of discrete primitive behaviors and transitions between these behaviors. This architecture has been widely studied in the context of quasi-static and dynamics-independent systems. However, applications of this concept to dynamical systems are relatively sparse, despite extensive research on individual d...
-
The efficient evaluation the dynamic stability of legged robots on non-coplanar terrains is important when developing motion planning and control policies. The inference time of this measure has a strong influence on how fast a robot can react to unexpected events, plan its future footsteps or its body trajectory. Existing approaches suitable for real-time decision making are either limited to fla...
-
Legged robots require fast and accurate representation of their surrounding terrain to achieve behaviors such as running, push recovery, continuous walking, backflips, while also utilizing on-board computational resources efficiently. The desired tasks can be achieved efficiently by representing the environment using planar regions. However, existing methods for planar region extraction are either...
-
In this paper, we explore the challenge of generating animal-like walking motions for legged robots. To this end, we propose a versatile and robust control pipeline that combines a state-of-the-art model-based controller with a data-driven technique that is commonly used in computer animation. We demonstrate the efficacy of our control framework on a variety of quadrupedal robots in simulation. We...
-
In this paper, a learning-based whole-body loco-motion controller is proposed, which enables quadruped robots to perform running in the style of real animals. We use a low-level controller based on multi-rigid body dynamics to calculate desired torques for each joint, while the high-level neural network policy planning the expected gait and foothold. The policy is trained with reinforcement learni...
-
Large workspace localization of magnetic robots is important for medical applications. This paper presents a novel localization strategy to achieve simultaneous localization and actuation of magnetic robots using hall-effect sensors. We integrate 25 sensors into a sensing probe and mount it on to the mobile-coil system, which realizes accurate sensing and actuation of magnetic devices within a cyl...
-
Adaptive locomotion of microrobots can be achieved by using a smart polymer such as a hydrogel. For hydrogel-based bilayer helical microrobots, the change of environment such as temperature and pH can result in shape deformation into helical shapes differing from their initial state and hence swimming performance. In this work, we proposed a model for studying the parameters that affect the shape ...
-
Control of tetherless magnetically actuated helical robots using rotating dipole fields has a wide variety of medical applications. The most promising technique in manipulation of these robots involves a rotating permanent magnet controlled by a robotic manipulator. In this work, we study the open-loop response of helical robots (in viscous fluids characterized by low Reynolds numbers) in the pres...
-
The spherical joint is an effective solution to design parallel micro-robotic systems with rotation capabilities in the three-dimensional space. This type of joint has however some non-linear characteristics, such as the clearance, which affect the positioning accuracy in micro-robotic tasks. The starting point of this study lies in experimental observations of rotation errors from a 3-PPPS 6-DOF ...
-
Non-contact manipulation technology has extensive application in the manipulation and fabrication of micro/nanomaterials. However, the manipulation devices are often precise and complex, operated only by professionals and subject to site constraints. We propose a simple optoelectronic tweezer platform, which can be controlled remotely and simply for the manipulation of microparticles at different ...
-
Recognizing Activities of Daily Living (ADL) is a vital process for intelligent assistive robots, but collecting large annotated datasets requires time-consuming temporal labeling and raises privacy concerns, e.g., if the data is collected in a real household. In this work, we explore the concept of constructing training examples for ADL recognition by playing life simulation video games and intro...
-
Radar sensors have a long tradition in advanced driver assistance systems (ADAS) and also play a major role in current concepts for autonomous vehicles. Their importance is reasoned by their high robustness against meteorological effects, such as rain, snow, or fog, and the radar’s ability to measure relative radial velocity differences via the Doppler effect. The cause for these advantages, namel...
-
We present a challenging dataset, ChangeSim, aimed at online scene change detection (SCD) and more. The data is collected in photo-realistic simulation environments with the presence of environmental non-targeted variations, such as air turbidity and light condition changes, as well as targeted object changes in industrial indoor environments. By collecting data in simulations, multi-modal sensor ...
-
Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person ...
-
Humans have a natural ability to effortlessly comprehend linguistic commands such as “park next to the yellow sedan” and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands. To this end, we propose the novel task of Referring ...
-
Event cameras are bio-inspired vision sensors which measure per pixel brightness changes. They offer numerous benefits over traditional, frame-based cameras, including low latency, high dynamic range, high temporal resolution and low power consumption. Thus, these sensors are suited for robotics and virtual reality applications. To foster the development of 3D perception and navigation algorithms ...
-
Modern self-driving systems heavily rely on deep learning. As a consequence, their performance is influenced significantly by the quality and richness of the training data. Data collection platforms can generate many hours of raw data on a daily basis, however, it is not feasible to label everything. Therefore, it is critical to have a mechanism to identify "what to label". Active learning approac...
-
Aerobatic quadrotors have been a very active field of research for the last two decades. Their huge community boosted the development of computational light-weight planning and control algorithms. In contrast and despite recent progress, research on agile micro autonomous underwater vehicles (µAUV) is still in its infancy. Both vehicle classes share a close relationship. They achieve high speeds o...
-
In the process of landing unmanned aerial vehicles (UAVs) on an unmanned surface vehicle (USV), a manipulator can be applied to help the UAV land safely and accurately. However, it is a challenge to control the manipulator on a disturbed USV due to joint velocity constraints and bandwidth limitations. To solve this problem, a predictive control framework is proposed in this paper. We leverage a fi...
-
Exploration and monitoring of hazardous fields in marine environments is one of the most promising tasks to be performed by fleets of low-cost micro autonomous underwater vehicles (μAUVs). In contrast to vehicles in other domains, underwater robots are forced to perform all computations onboard as no powerful communication links are available underwater. This puts the focus on computationally effi...
-
We propose a novel technique for guidance of buoyancy-controlled vehicles in uncertain under-ice ocean flows. In-situ melt rate measurements collected at the grounding zone of Antarctic ice shelves, where the ice shelf meets the underlying bedrock, are essential to constrain models of future sea level rise. Buoyancy-controlled vehicles, which control their vertical position in the water column thr...
-
The Modboat is an autonomous surface robot that turns the oscillation of a single motor into a controlled paddling motion through passive flippers. Inertial control methods developed in prior work can successfully drive the Modboat along trajectories and enable docking to neighboring modules, but have a non-constant cycle time and cannot react to dynamic environments. In this work we present a thr...
-
We present multi-robot iSAM2 (MR-iSAM2), an efficient incremental smoothing and mapping (iSAM) algorithm to solve multi-robot simultaneous localization and mapping (SLAM) inference problems. MR-iSAM2 is based on a novel data structure multi-root Bayes tree (MRBT), which packs multiple Bayes trees with the same undirected clique structure. In multi-robot scenarios, the MRBT enables new measurements...
-
We present a collaborative visual simultaneous localization and mapping (SLAM) framework for service robots. With an edge server maintaining a map database and performing global optimization, each robot can register to an existing map, update the map, or build new maps, all with a unified interface and low computation and memory cost. We design an elegant communication pipeline to enable real-time...
-
The data loss caused by unreliable network seriously impacts the results of remote visual SLAM systems. From our experiment, a loss of less than 1 second of data can cause a visual SLAM algorithm to lose tracking. We present a novel buffering method, ORBBuf, to reduce the impact of data loss on remote visual SLAM systems. We model the buffering problem as an optimization problem by introducing a s...
-
In this paper we present a consistent and distributed state estimator for multi-robot cooperative localization (CL) which efficiently fuses environmental features and loop-closure constraints across time and robots. In particular, we leverage covariance intersection (CI) to allow each robot to only estimate its own state and autocovariance and compensate for the unknown correlations between robots...
-
Real-time dense reconstruction has been extensively studied for its wide applications in computer vision and robotics, meanwhile much effort has been made for the multi-robot system which plays an irreplaceable role in complicated but time-critical scenarios, e.g., search and rescue tasks. In this paper, we propose an efficient system named Coxgraph for multi-robot collaborative dense reconstructi...
-
We propose Super Odometry, a high-precision multi-modal sensor fusion framework, providing a simple but effective way to fuse multiple sensors such as LiDAR, camera, and IMU sensors and achieve robust state estimation in perceptually-degraded environments. Different from traditional sensor-fusion methods, Super Odometry employs an IMU-centric data processing pipeline, which combines the advantages...
-
In the future, extraterrestrial expeditions will not only be conducted by rovers but also by flying robots. The technical demonstration drone Ingenuity, that just landed on Mars, will mark the beginning of a new era of exploration unhindered by terrain traversability. Robust self-localization is crucial for that. Cameras that are lightweight, cheap and information-rich sensors are already used to ...
-
Perception using stereo requires external light. In the absence of natural light, active, structured light provides light where it is needed. In this work, we demonstrate how a free moving line striping laser can be used to perceive and model terrains. In this formulation, we do not need to know the position of the laser with respect to the stereo pair which precludes the need for calibrating the ...
-
Future planetary missions will rely on rovers that can autonomously explore and navigate in unstructured environments. An essential element is the ability to recognize places that were already visited or mapped. In this work, we leverage the ability of stereo cameras to provide both visual and depth information, guiding the search and validation of loop closures from a multi-modal perspective. We ...
-
Space free-flyers like the Astrobee robots currently operating aboard the International Space Station must operate with inherent system uncertainties. Parametric uncertainties like mass and moment of inertia are especially important to quantify in these safety-critical space systems and can change in scenarios such as on-orbit cargo movement, where unknown grappled payloads significantly change th...
-
This study analyzes the surface sliding behavior of space probes in simulated extraterrestrial environments. When a space probe lands on an extraterrestrial body, its landing gear (footpad, landing legs) contacts and slides along the ground surface. The influence of various parameters (i.e., footpad size, velocity, ground condition, atmospheric pressure, and gravity) on the friction behavior of fo...
-
Due to the high sensitivity and complexity of robotic surgery tasks, acquiring appropriate skill levels by trainee surgeons through an effective training process is very important and affects the patient’s safety and the quality of surgical outcomes. With the advanced deep learning technology and the recent availability of surgical procedures data, intelligent methods can be deployed to assess and...
-
Teleoperation systems with haptic feedback allow a human user to remotely interact with a dangerous or inac-cessible environment, perform various tasks, and perceive the haptic feedback. To ensure system stability while maintaining the best possible quality of experience (QoE), different teleoperation control schemes and haptic communication strategies need to be selected to adapt to varying netwo...
-
Multi-sensor solution has been widely adopted in real-world robotics systems (e.g., self-driving vehicles) due to its better robustness. However, its performance is highly dependent on the accurate calibration between different sensors, which is very time-consuming (i.e., hours of human efforts) to acquire. Recent learning-based solutions partially address this yet still require costly ground-trut...
-
Robot modelling is an essential part to properly understand how a robotic system moves and how to control it. The kinematic model of a robot is usually obtained by using Denavit-Hartenberg convention, which relies on a set of parameters to describe the end-effector pose in a Cartesian space. These parameters are assigned based on geometrical considerations of the robotic structure, however, the as...
-
In the era of self-driving vehicles, state estimation has 3 main contradictory requirements, such as accuracy, robustness, and cost-effectiveness. To satisfy all of them, the integration of the wheel encoder measurements is a proper choice besides the generally applied GNSS, inertial and visual-odometry methods. The wheel odometry is a robust and cost-effective method, but the accuracy of the esti...
-
The integration of multiple cameras and 3D Li-DARs has become basic configuration of augmented reality devices, robotics, and autonomous vehicles. The calibration of multi-modal sensors is crucial for a system to properly function, but it remains tedious and impractical for mass production. Moreover, most devices require re-calibration after usage for certain period of time. In this paper, we prop...
-
Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. However, IL methods often fail to generalize to new scene configurations not covered by training data. On the other hand, humans can manipulate objects in varying conditions. Key to such capability is hand-eye coordination, a cognitive ability that enables humans to adaptively direct their...
-
Recent Imitation Learning (IL) techniques focus on adversarial imitation learning algorithms to learn from a fixed set of expert demonstrations. While these approaches are theoretically sound, they suffer from a number of problems such as poor sample efficiency, poor stability, and a host of issues that Generative Adversarial Networks (GANs) suffer from. In this paper we introduce a generalization...
-
Traditional imitation learning provides a set of methods and algorithms to learn a reward function or policy from expert demonstrations. Learning from demonstration has been shown to be advantageous for navigation tasks as it allows for machine learning non-experts to quickly provide information needed to learn complex traversal behaviors. However, a minimal set of demonstrations is unlikely to ca...
-
We propose a decentralized, learning-based solution to the challenging problem of unlabeled multi-agent navigation among obstacles, where robots need to simultaneously tackle the problems of goal assignment, local collision avoidance, and navigation. Our method has each robot infer their desired action by communicating with each other as well as a set of position-fixed routers. The inference is ca...
-
The multi-robot coverage problem is an essential building block for systems that perform tasks like inspection, exploration, or search and rescue. We discretize the coverage problem to induce a spatial graph of locations and represent robots as nodes in the graph. Then, we train a Graph Neural Network controller that leverages the spatial equivariance of the task to imitate an expert open-loop rou...
-
Segmentation of a compound task with multiple subtasks is crucial for imitation learning. Conventional unsupervised segmentation methods focused on only reproducibility of demonstrations and did not use the property that goal-directed actions rarely occur without intention. In this paper, we propose a novel method to segment demonstrations into goal-directed actions by self-supervised learning. We...
-
Deep imitation learning is promising for solving dexterous manipulation tasks because it does not require an environment model and pre-programmed robot behavior. However, its application to dual-arm manipulation tasks remains challenging. In a dual-arm manipulation setup, the increased number of state dimensions caused by the additional robot manipulators causes distractions and results in poor pe...
-
Many state-of-art robotics applications require fast and efficient motion planning algorithms. Existing motion planning methods become less effective as the dimensionality of the robot and its workspace increases, especially the computational cost of collision detection routines. In this work, we present a framework to address the cost of expensive primitive operations in sampling-based motion pla...
-
Traditional decision and planning frameworks for self-driving vehicles (SDVs) scale poorly in new scenarios, thus they require tedious hand-tuning of rules and parameters to maintain acceptable performance in all foreseeable cases. Recently, self-driving methods based on deep learning have shown promising results with better generalization capability but less hand engineering effort. However, most...
-
Deep learning algorithms such as Convolutional Neural Networks (CNNs) are currently used to solve a range of robotics and computer vision problems. These networks typically estimate the desired representation in a single forward pass and must therefore learn to converge from a wide range of initial conditions to a precise result. This is challenging, and has led to increased interest in the develo...
-
Deep Neural Networks (DNNs) are a critical component for self-driving vehicles. They achieve impressive performance by reaping information from high amounts of labeled data. Yet, the full complexity of the real world cannot be encapsulated in the training data, no matter how big the dataset, and DNNs can hardly generalize to unseen conditions. Robustness to various image corruptions, caused by cha...
-
Stream-based active learning (AL) is an efficient training data collection method, and it is used to reduce human annotation cost required in machine learning. However, it is difficult to say that the human cost is low enough because most previous studies have assumed that an oracle is a human with domain knowledge. In this study, we propose a method to replace a part of the oracle’s work in strea...
-
Autonomous industrial assembly relies on the precise measurement of spatial constraints as designed by computer-aided design (CAD) software such as SolidWorks. This paper proposes a framework for an intelligent industrial robot to understand the spatial constraints for model assembly. An extended generative adversary network (GAN) with a 3D long short-term memory (LSTM) network was designed to com...
-
The recent years have seen the increasing importance of cognitive models for improved robot navigation. In this paper, a novel cognitive navigation package, which consists of topometric map representation and a three-level path planner, is proposed. The topometric maps are built from architectural floor plans with additional features within a limited number of selected regions. The inherent discre...
-
This paper aims to improve the path quality and computational efficiency of sampling-based kinodynamic planners for vehicular navigation. It proposes a learning framework for identifying promising controls during the expansion process of sampling-based planners. Given a dynamics model, a reinforcement learning process is trained offline to return a low-cost control that reaches a local goal state ...
-
Risk-bounded motion planning is an important yet difficult problem for safety-critical tasks. While existing mathematical programming methods offer theoretical guarantees in the context of constrained Markov decision processes, they either lack scalability in solving larger problems or produce conservative plans. Recent advances in deep reinforcement learning improve scalability by learning policy...
-
In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided by state-of-the-art deep convolutional neural netwo...
-
When a learning solution is needed for different robots, a model is often trained for each robot geometry, even if the robotic task is the same and the robots are structurally similar. In this paper, we address the problem of transfer learning of swept volume predictors for the motion of articulated robots with similar geometric structure. The swept volume is a scalar value corresponding to the sp...
-
Nano quadcopters are ideal for gas source localization (GSL) as they are safe, agile and inexpensive. However, their extremely restricted sensors and computational resources make GSL a daunting challenge. We propose a novel bug algorithm named ‘Sniffy Bug', which allows a fully autonomous swarm of gas-seeking nano quadcopters to localize a gas source in unknown, cluttered, and GPS-denied environme...
-
This paper presents a novel methodology that allows a swarm of robots to perform a cooperative transportation task. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF), taking advantage of this framework’s locality properties. By setting appropriate potential functions, robots can dynamically navigate, form groups, and perform co- operative transportation in a completely dece...
-
In this paper, we study existing flocking models and propose extensions to improve their abilities to deal with environments having obstacles impacting the communication quality. Often depicted as robust systems, there is yet a lack of understanding how flocking models compare and how they are impacted by the communication quality when they exchange control data. We extend two standard models to i...
-
In this paper, we propose micROS.BT, an event-driven behavior tree (BT) framework aiming at supporting swarm-robot coordination. Compared with other BT frame-works, micROS.BT implements the event-driven way under the multi-thread mode, which can effectively save computing resources. Moreover, in order to ensure swarm-robot coordination, we optimize the implementation of the traditional blackboard ...
-
Current existing stereo visual odometry algorithms are computationally too expensive for robots with restricted resources. Executing these algorithms on such robots leads to a low frame rate and unacceptable decay in accuracy. We modify S-MSCKF, one of the most computationally efficient stereo Visual Inertial Odometry (VIO) algorithm, to improve its speed and accuracy when tracking low numbers of ...
-
Bundle adjustment refines scene geometry and relative camera poses simultaneously via reprojection error, computed by a set of images from different viewpoints, which is the gold standard for visual odometry. However, deep learning methods have not been well exploited within this area of study. This paper introduces a self-supervised learning framework for monocular visual odometry, inside which d...
-
Most of the state-of-the-art visual inertial SLAM methods pay less attention to 2D-2D and 3D-2D matching with more reliable features in a long time span, which easily results in continuous estimation drift. In this paper, we propose an efficient drift-free visual-inertial SLAM method by a pose guided feature matching method to re-identify existing features from a spatial-temporal sensitive sub-glo...
-
In this paper, we propose a new scheme based on the Sampson distance (SD) to describe visual feature residuals for visual-inertial odometry (VIO). Unlike the epipolar-constraint-based SD for visual odometry (VO), the proposed SD is formulated based on the perspective projection constraint. We proved in theory that the proposed SD retains the good properties of those earlier SD criteria in the lite...
-
Advancing maturity in mobile and legged robotics technologies is changing the landscapes where robots are being deployed and found. This innovation calls for a transformation in simultaneous localization and mapping (SLAM) systems to support this new generation of service and consumer robots. No longer can traditionally robust 2D lidar systems dominate while robots are being deployed in multi-stor...
-
Modern visual-inertial navigation systems (VINS) are faced with a critical challenge in real-world deployment: they need to operate reliably and robustly in highly dynamic environments. Current best solutions merely filter dynamic objects as outliers based on the semantics of the object category. Such an approach does not scale as it requires semantic classifiers to encompass all possibly-moving o...
-
Recovering rigid registration between successive camera poses lies at the heart of 3D reconstruction, SLAM and visual odometry. Registration relies on the ability to compute discriminative 2D features in successive camera images for determining feature correspondences, which is very challenging in feature-poor environments, i.e. low-texture and/or low-light environments. In this paper, we aim to a...
-
In this article we propose a distributed collision avoidance scheme for multi-agent unmanned aerial vehicles (UAVs) based on nonlinear model predictive control (NMPC), where other agents in the system are considered as dynamic obstacles with respect to the ego agent. Our control scheme operates at a low level and commands roll, pitch and thrust signals at a high frequency, each agent broadcasts it...
-
Supervision of a nominal controller, to enforce safety, is concerned with appropriately modifying the generated control inputs, if needed, in order to keep a control system within a set of safe states. An integral component in supervision is a controlled invariant set contained in the set of safe states. In this paper, we build on recent results on the computation of polytopic controlled invariant...
-
Imaging techniques are established aids in surgical procedures and have become indispensable in modern medicine. Combined with robot-assisted systems, significantly higher precision can already be achieved today and work steps can be simplified. We hypothesize that algorithms can make the operations of the future safer and easier. The paper introduces an approach for the control of a robotic manip...
-
In this paper, we extend a famous motion planning approach, GPMP2, to multi-robot cases, yielding a novel centralized trajectory generation method for the multi-robot formation. A sparse Gaussian Process model is employed to represent the continuous-time trajectories of all robots as a limited number of states, which improves computational efficiency due to the sparsity. We add constraints to guar...
-
Despite decades of efforts, robot navigation in a real scenario with volatility, uncertainty, complexity, and ambiguity (VUCA for short), remains a challenging topic. Inspired by the central nervous system (CNS), we propose a hierarchical multi-expert learning framework for autonomous navigation in a VUCA environment. With a heuristic exploration mechanism considering target location, path cost, a...
-
Deep reinforcement learning has achieved great success in laser-based collision avoidance work because the laser can sense accurate depth information without too much redundant data, which can maintain the robustness of the algorithm when it is migrated from the simulation environment to the real world. However, high-cost laser devices are not only difficult to apply on a large scale but also have...
-
This paper studies the problem of interaction control based on force and velocity measurements. Without restricting ourselves to fixed control structures such as admittance or impedance control, we consider the general topology of the problem. Adopting the idea of Youla-based 2DOF control, we reveal a generic control architecture that splits the problem into two independent parts: nominal admittan...
-
Human physical assistance requires the assistant to tune both his trajectory and impedance in order to assist an individual as well as be guided by him. In this study we propose a controller for teleoperated human assistance that allows the assistant to guide the assisting robot in both trajectory and impedance. We propose to use the inherent perturbations in the task, induced by the elderly or st...
-
In modern robotics, the manipulators are no longer isolated under fully controlled conditions but rather conceived to work in unconstrained environments. Under these operations, compliant control and passivity properties of the robot are of great importance, and thus the system’s energy function plays a crucial role in the control design. In this work, we propose a new design of cartesian impedanc...
-
We address the problem of motion control for safe physical interaction, and in particular finding new ways for impedance controller parameters’ adaptation to ensure better safety with minimum possible lose in robot performance. We propose an exteroception-based dynamically updated safety metric that takes into account current robot state and inertia as well as external objects’ mass, shape, materi...
-
Manipulation of flexible objects is one of the major challenges in robotics as the nonlinear dynamics of the high-dimensional object structure makes it difficult to apply current control methods. A previous simulation study showed that control with few pre-structured joint trajectories coupled with joint impedance (dynamic primitives) could control a 25-dimensional whip to hit a target. This was p...
-
Natural dynamics, nonlinear optimization, and, more recently, convex optimization are available methods for stiffness design of energy-efficient series elastic actuators. Natural dynamics and general nonlinear optimization only work for a limited set of load kinetics and kinematics, cannot guarantee convergence to a global optimum, or depend on initial conditions to the numerical solver. Convex pr...
-
Hydrostatic transmission has shown promising results for enabling the manipulator to achieve low effective inertia, high stiffness, and high torque density. However, the incompressibility of fluid causes the lack of compliance, so that it could not provide intrinsic safety. Thus, it would be advantageous to introduce series compliance on the hydrostatic manipulator for adjusting stiffness dependin...
-
Optimization of gear ratios for dual-motor actuators is presented for the development of a walker-type assist robot. The robot is reconfigurable to provide an elderly user with multiple physical support functions; one is to assist sitto-stand transitions and the other is to serve as a walker to aid the user in walking. To avoid falling while walking, the robot must react quickly and reconfigure it...
-
In this study, based on an analysis of experimental results, we propose a method for slip modeling of Spiral Zipper. The Spiral Zipper is a prismatic actuator that extracts a length-changeable cylindrical tube from a flexible ABS band. The band tube is driven by a friction wheel; the slip results due to the lack of synchronization between the motion of the band and that of the friction wheel. Thro...
-
In many real-world manipulation problems, the constraints imposed by the environment on an object are tight. In these cases, most state-of-the-art planners struggle to fit satisfactorily in low dimensional sub-manifolds, while still ensuring geometric and force feasibility. On the other hand, humans are at ease with such situations and indeed exploit constraints to manipulate objects proficiently....
-
Manipulation of objects by exploiting their contact with the environment can enhance both the dexterity and payload capability of robotic manipulators. A common way to manipulate heavy objects beyond the payload capability of a robot is to use a sequence of pivoting motions, wherein, an object is moved while some contact points between the object and a support surface are kept fixed. The goal of t...
-
Similar to human work, robotic tasks sometimes require two hands to be accomplished. This requires coordinated motion planning and control. While fulfilling the task in a coordinated manner is already a big challenge, the task at hand becomes even harder when obstacles are introduced in the environment that need to be avoided. Furthermore in the case of dynamic environments, contacts cannot be avo...
-
While a human is tracking a moving object to prepare for later grasping, we naturally change our hand pose to generate optimal pre-grasp to avoid post-grasp adjustment. Robot hand controllers need dynamic pre-grasp planning capability, so they are not limited in dynamic tracking and catching tasks. To fill this gap, we explore the feasibility of using a two-stage optimization method to enable dyna...
-
This paper proposes a manipulation scheme based on learning the motion of objects after being hit by a robotic end-effector. This allows for the object to be positioned at a desired location outside the physical workspace of the robot. An estimate of the object dynamics under friction and collisions is learnt and used to predict the desired hitting parameters (speed and direction), given the initi...
-
Grasping in dynamic environments presents a unique set of challenges. A stable and reachable grasp can become unreachable and unstable as the target object moves, motion planning needs to be adaptive and in real time, the delay in computation makes prediction necessary. In this paper, we present a dynamic grasping framework that is reachability-aware and motion-aware. Specifically, we model the re...
-
We study a stochastic version of the classic orienteering problem where the time to traverse an edge is a continuous random variable. For a given temporal deadline B, our solution produces a policy, i.e., a function that, based on the current position along a solution path and the elapsed time, decides whether to continue along the path or take a shortcut to avoid missing the deadline. The solutio...
-
Formation control of multi-robot systems has been largely studied due to its wide application domain. Several methods in the literature rely on explicit communication among the robots, which in realistic scenarios may lead to reduced performance or even instability due to delays and packet loss or corruption. Nonetheless, multi-robot coordination based solely on implicit communication has been pro...
-
Unmanned Aircraft System Traffic Management (UTM) becomes a highly relevant complex challenge, as the UAV activity is rapidly growing bringing more amateur and professional drones to the urban skies. The main concern of managing such a system is safely navigating and controlling hundreds or thousands of drones simultaneously, flying in a crowded dense environments. This paper introduces an innovat...
-
Adjustable Autonomy is gaining interest as it alleviates robot management costs, which often restrain non-routine applications. Whereas it seems straightforward to account for the availability of helpers when making plans that involve being granted for support in the future, no existing research covers this issue. As a solution, we formalize the first human-centric model that accounts for operator...
-
Despite the stringent requirements of a real-time system, the reliance of the Robot Operating System (ROS) on the loopback network interface imposes a considerable overhead on the transport of high bandwidth data, while the nodelet package, which is an efficient mechanism for intra-process communication, does not address the problem of efficient local inter-process communication (IPC). To remedy t...
-
Robot Operating System (ROS) is a de-facto standard robot middleware in many academic and industrial use cases. However, utilizing ROS/ROS2 in safety-critical embedded applications with real-time requirement is challenging because of C1) Non-real-time underlying hardware, C2) No control on the host OS scheduler, C3) Unpredictable dynamic memory allocation, C4) High resource requirement, and C5) Un...
-
Recently, mobile robots have become important tools in various industries, especially in logistics. Deep reinforcement learning emerged as an alternative planning method to replace overly conservative approaches and promises more efficient and flexible navigation. However, deep reinforcement learning approaches are not suitable for long-range navigation due to their proneness to local minima and l...
-
Exploration tasks are essential to many emerging robotics applications, ranging from search and rescue to space exploration. The planning problem for exploration requires determining the best locations for future measurements that will enhance the fidelity of the map, for example, by reducing its total entropy. A widely-studied technique involves computing the Mutual Information (MI) between the c...
-
Suppose an agent asserts that it will move through an environment in some way. When the agent executes its motion, how does one verify the claim? The problem arises in a range of contexts including validating safety claims about robot behavior, applications in security and surveillance, and for both the conception and the (physical) design and logistics of scientific experiments. Given a set of fe...
-
In this paper, we introduce an automata-based framework for planning with relaxed specifications. User relaxation preferences are represented as weighted finite state edit systems that capture permissible operations on the specification, substitution and deletion of tasks, with complex constraints on ordering and grouping. We propose a three-way product automaton construction method that allows us...
-
We propose a novel constrained reinforcement learning method for finding optimal policies in Markov Decision Processes while satisfying temporal logic constraints with a desired probability throughout the learning process. An automata-theoretic approach is proposed to ensure the probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve c...
-
Understanding a controller’s performance in different scenarios is crucial for robots that are going to be deployed in safety-critical tasks. If we do not have a model of the dynamics of the world, which is often the case in complex domains, we may need to approximate a performance function of the robot based on its interaction with the environment. Such a performance function gives us insights in...
-
We present a topological method for finding coarsely diverse pathways. The use of pre-computed paths for online planning in a dynamic context reduces the overhead of re-planning alternate routes. Our algorithm applied the notion of discrete Morse theory to identify critical points incident on the obstacles and used this information to identify and return a diverse set of coarse paths. Three sampli...
-
This paper proposes a framework for learning task specifications from demonstrations, while ensuring that the learned specifications do not violate safety constraints. Furthermore, we show how these specifications can be used in a planning problem to control the robot under environments that can be different from those encountered during the learning phase. We formulate the specification learning ...
-
Plane features can be used to reduce drift errors in SLAM systems, especially in indoor environments. It is easy and efficient to extract planes from a dense point cloud, which is commonly generated from a RGB-D camera or a 3D lidar. But when using a stereo camera, it is hard to compute dense point clouds accurately or efficiently. In this paper, we propose a novel method to compute plane paramete...
-
This paper presents a hierarchical segment-based optimization method for Simultaneous Localization and Mapping (SLAM) system. First we propose a reliable trajectory segmentation method that can be used to increase efficiency in the back-end optimization. Then we propose a buffer mechanism for the first time to improve the robustness of the segmentation. During the optimization, we use global infor...
-
This work is dedicated to simultaneous continuous-time trajectory estimation and mapping based on Gaussian Processes (GP). State-of-the-art GP-based models for Simultaneous Localization and Mapping (SLAM) are computationally efficient but can only be used with a restricted class of kernel functions. This paper provides the algorithm based on GP with Random Fourier Features (RFF) approximation for ...
-
Autonomous mobile robots need maps for effective, safe navigation, and SLAM in general is still an unsolved problem. Nonetheless, certain combinations of environmental characteristics and sensors admit tractable solutions. In particular, detection and tracking of linear features such as line segments (2D) or planar facets (3D) has been proven robust in many man-made environments. However, these ty...
-
A 3D mesh offers a rich yet lightweight representation of geometry and topology for the metric and semantic understanding of a robot’s scene. Noisy features are often used to generate the mesh which furthers the need for accurate regularisation. Current approaches tightly couple front-end optimisation with regularisation making it difficult to evaluate the choice of discretisation and regularisati...
-
In this paper, we propose a novel local to global plane regularity aggregation framework for dense surfel mapping, aiming for real-time reconstruction of high-quality 3D global models in both indoor and urban environments. Different from prior works that directly localize surfels globally, we investigate three interplanar geometric relations: {coplanarity, parallelism, orthogonality} from local to...
-
Meshes are commonly used as 3D maps since they encode the topology of the scene while being lightweight. Unfortunately, 3D meshes are mathematically difficult to handle directly because of their combinatorial and discrete nature. Therefore, most approaches generate 3D meshes of a scene after fusing depth data using volumetric or other representations. Nevertheless, volumetric fusion remains comput...
-
Traditional monocular visual simultaneous localization and mapping (SLAM) algorithms have been extensively studied and proven to reliably recover a sparse structure and camera motion. Nevertheless, the sparse structure is still insufficient for scene interaction, e.g., visual navigation and augmented reality applications. To densify the scene reconstruction, the use of single-image absolute depth ...
-
High definition (HD) maps have demonstrated their essential roles in enabling full autonomy, especially in complex urban scenarios. As a crucial layer of the HD map, lane-level maps are particularly useful: they contain geometrical and topological information for both lanes and intersections. However, large scale construction of HD maps is limited by tedious human labeling and high maintenance cos...
-
In this paper, we propose a highly accurate continuous-time trajectory estimation framework dedicated to SLAM (Simultaneous Localization and Mapping) applications, which enables fuse high-frequency and asynchronous sensor data effectively. We apply the proposed framework in a 3D LiDAR-inertial system for evaluations. The proposed method adopts a non-rigid registration method for continuous-time tr...
-
Generating meaningful spatial models of physical environments is a crucial ability for autonomous navigation of mobile robots. This paper considers the problem of building continuous occupancy maps from sparse and noisy sensor data. To this end, we propose a new method named random mapping maps that advances the popular methods in two aspects. Firstly, it can represent environment models in a memo...
-
In this work, we fully define the existing relationships between traditional optimality criteria and the connectivity of the underlying pose-graph in Active SLAM, characterizing, therefore, the connection between Graph Theory and the Theory Optimal Experimental Design. We validate the proposed relationships in 2D and 3D graph SLAM datasets, showing a remarkable relaxation of the computational load...
-
Vehicle dynamic models are the key to bridge the gap between simulation and real road test in autonomous driving. An accurate vehicle model allows control algorithms in simulation being transferred to real road test with same quality. In this paper, we present a dynamic model residual correction framework (DRF) for vehicle dynamic modeling. DRF provides a general accuracy improvement framework on ...
-
While object detection modules are essential functionalities for any autonomous vehicle, the performance of such modules that are implemented using deep neural networks can be, in many cases, unreliable. In this paper, we develop abstraction-based monitoring as a logical framework for filtering potentially erroneous detection results. Concretely, we consider two types of abstraction, namely data-l...
-
Autonomous systems often operate in environments where the behavior of multiple agents is coordinated by a shared global state. Reliable estimation of the global state is thus critical for successfully operating in a multi-agent setting. We introduce agent-aware state estimation—a framework for calculating indirect estimations of state given observations of the behavior of other agents in the envi...
-
This paper presents a mobile UVC disinfection robot designed to mitigate the threat of airborne and surface pathogens. Our system comprises a mobile robot base, a custom UVC lamp assembly, and algorithms for autonomous navigation and path planning. We present a model of UVC disinfection and dosage of UVC light delivered by the mobile robot. We also discuss challenges and prototyping decisions for ...
-
Autonomous vehicles must balance a complex set of objectives. There is no consensus on how they should do so, nor on a model for specifying a desired driving behavior. We created a dataset to help address some of these questions in a limited operating domain. The data consists of 92 traffic scenarios, with multiple ways of traversing each scenario. Multiple annotators expressed their preference be...
-
This paper presents a novel strategy for the autonomous deployment of Micro Aerial Vehicle scouts through constricted aperture-like ingress points, by narrowly fitting and launching them with a high-precision Mobile Manipulation robot. A significant problem during exploration and reconnaissance into highly unstructured environments, such as indoor collapsed ones, is the encountering of impassable ...
-
In this study, we report the successful execution of in-air knotting of rope using a dual-arm two-finger robot based on deep learning. Owing to its flexibility, the state of the rope was in constant flux during the operation of the robot. This required the robot control system to dynamically correspond to the state of the object at all times. However, a manual description of appropriate robot moti...
-
Automated planning enables robots to find plans to achieve complex, long-horizon tasks, given a planning domain. This planning domain consists of a list of actions, with their associated preconditions and effects, and is usually manually defined by a human expert, which is very time-consuming or even infeasible. In this paper, we introduce a novel method for generating this domain automatically fr...
-
Recent advances in robot learning have enabled robots to become increasingly better at mastering a predefined set of tasks. On the other hand, as humans, we have the ability to learn a growing set of tasks over our lifetime. Continual robot learning is an emerging research direction with the goal of endowing robots with this ability. In order to learn new tasks over time, the robot first needs to ...
-
Imitation learning (IL) algorithms have shown promising results for robots to learn skills from expert demonstrations. However, they need multi-task demonstrations to be provided at once for acquiring diverse skills, which is difficult in real world. In this work we study how to realize continual imitation learning ability that empowers robots to continually learn new tasks one by one, thus reduci...
-
When an autonomous robot learns how to execute actions, it is of interest to know if and when the execution policy can be generalised to variations of the learning scenarios. This can inform the robot about the necessity of additional learning, as using incomplete or unsuitable policies can lead to execution failures. Generalisation is particularly relevant when a robot has to deal with a large va...
-
Reinforcement Learning (RL) methods have been widely applied for robotic manipulations via sim-to-real transfer, typically with proprioceptive and visual information. However, the incorporation of tactile sensing into RL for contact-rich tasks lacks investigation. In this paper, we model a tactile sensor in simulation and study the effects of its feedback in RL-based robotic control via a zero-sho...
-
The planning of robotic manipulation and grasping tasks depends on the reconstruction of the 3D object’s shape. Most of the existing 3D object reconstruction methods are based on visual sensing that are limited due to the lack of the object’s occluded side information. The goal of this paper is to overcome these limitations and improve the 3D objects’ reconstruction by adding the tactile sensing t...
-
Recent work on robot learning with visual observations has shown great success in solving many manipulation tasks. While visual observations contain rich information about the environment and the robot, they can be unreliable in the presence of visual noise or occlusions. In these cases, we can leverage tactile observations generated by the interaction between the robot and the environment. In thi...
-
ICP algorithms typically involve a fixed choice of data association method and a fixed choice of error metric. In this paper, we propose Hybrid ICP, a novel and flexible ICP variant which dynamically optimises both the data association method and error metric based on the live image of an object and the current ICP estimate. We show that when used for object pose estimation, Hybrid ICP is more acc...
-
Rotational displacement about the grasping point is a common grasp failure when an object is grasped at a location away from its center of gravity. Tactile sensors with soft surfaces, such as GelSight sensors, can detect the rotation patterns on the contacting surfaces when the object rotates. In this work, we propose a model-based algorithm that detects those rotational patterns and measures rota...
-
We present a system for multi-level scene awareness for robotic manipulation. Given a sequence of camera-inhand RGB images, the system calculates three types of information: 1) a point cloud representation of all the surfaces in the scene, for the purpose of obstacle avoidance. 2) the rough pose of unknown objects from categories corresponding to primitive shapes (e.g., cuboids and cylinders), and...
-
We present a soft humanoid hand with in-finger integrated cameras and in-hand real-time image processing system for fast reactive grasping. Specifically, we describe an FPGA-based, in-hand integrated, embedded system for processing visual data captured by the five in-finger cameras while avoiding high bandwidth raw data streaming via the robots real-time data bus. The hardware acceleration allows ...
-
Event-based visual perception is becoming increasingly popular owing to interesting sensor characteristics enabling the handling of difficult conditions such as highly dynamic motion or challenging illumination. The mostly complementary nature of event cameras however still means that best results are achieved if the sensor is paired with a regular frame-based sensor. The present work aims at answ...
-
We need robots more aware of the unintended outcomes of their actions for ensuring safety. This can be achieved by an onboard failure detection system to monitor and detect such cases. Onboard failure detection is challenging with a limited set of onboard sensor setup due to the limitations of sensing capabilities of each sensor. To alleviate these challenges, we propose FINO-Net, a novel multimod...
-
In this work we propose 3D-FFS, a novel approach to make sensor fusion based 3D object detection networks significantly faster using a class of computationally inexpensive heuristics. Existing sensor fusion based networks generate 3D region proposals by leveraging inferences from 2D object detectors. However, as images have no depth information, these networks rely on extracting semantic features ...
-
Inertial measurement units are widely used in different fields to estimate the attitude. Many algorithms have been proposed to improve estimation performance. However, most of them still suffer from 1) inaccurate initial estimation, 2) inaccurate initial filter gain, and 3) non-Gaussian process and/or measurement noise. This paper will leverage reinforcement learning to compensate for the classica...
-
Dynamic objects in the environment, such as people and other agents, lead to challenges for existing simultaneous localization and mapping (SLAM) approaches. To deal with dynamic environments, computer vision researchers usually apply some learning-based object detectors to remove these dynamic objects. However, these object detectors are computationally too expensive for mobile robot on-board pro...
-
Robust off-road perception for autonomous navigation is hard to achieve. Versatile environments, different hardware, and numerous disturbances limit the perceptional portability in changing applications and cross-platform. This contribution proposes sensor-fusion considering the data quality of uncertain sensors to increase the classification and mapping components’ perceptual robustness. The resu...
-
Due to changes in model dynamics or unexpected disturbances, an autonomous robotic system may experience unforeseen challenges during real-world operations which may affect its safety and intended behavior: in particular actuator and system failures and external disturbances are among the most common causes of degraded mode of operation. To deal with this problem, in this work, we present a meta-l...
-
Omni-directional mobile robot (OMR) systems have been very popular in academia and industry for their superb maneuverability and flexibility. Yet their potential has not been fully exploited, where the extra degree of freedom in OMR can potentially enable the robot to carry out extra tasks. For instance, gimbals or sensors on robots may suffer from a limited field of view or be constrained by the ...
-
Replanning is an essential problem for robots operating in a dynamic and complex environment for responsive and robust autonomy. Previous incremental-search algorithms efficiently reuse existing search results to facilitate a new plan when the environment changes. Yet, they rely solely on geometric information of the environment encoded in an edge-weighted graph. However, semantic information ofte...
-
Navigation planning for legged robots has distinct challenges compared to wheeled and tracked systems due to the ability to lift legs off the ground and step over obstacles. While most navigation planners assume a fixed traversability value for a single terrain patch, we overcome this limitation by proposing a reachability-based navigation planner for legged robots. We approximate the robot morpho...
-
Many robotic manipulation problems are multi-modal—they consist of a discrete set of mode families (e.g., whether an object is grasped or placed) each with a continuum of parameters (e.g., where exactly an object is grasped). Core to these problems is solving single-mode motion plans, i.e., given a mode from a mode family (e.g., a specific grasp), find a feasible motion to transition to the next d...
-
We present a framework for planning complex motor actions such as pouring or scooping from arbitrary start states in cluttered real-world scenes. Traditional approaches to such tasks use dynamic motion primitives (DMPs) learned from human demonstrations. We enhance a recently proposed state-of-the-art DMP technique capable of obstacle avoidance by including them within a novel hybrid framework. Th...
-
We present a new approach to tackle the problem of lattice-type metamorphic robots reconfiguration. We base our approach on a reduction to satisfiability modulo theory (SMT). Unlike the current state-of-the-art solutions, we consider the spatial limitations of the modules themselves and produce collision-free plans. We give an in-depth description of the reduction and discuss several optimizations...
-
This paper introduces a new class of soft reconfigurable robot: balloon animal robots. The balloon animal robot consists of a closed volume inflatable tube which can be reconfigured into structures of varying topology by a collective of simple sub-unit robots. The robotic sub-units can (1) drive along the length of the tube to localize a joint, (2) create pinch points that locally reduce the bendi...
-
Programmable matter is a material that can change its physical properties at will, whether it is its shape, density or conductivity. It can be implemented as an ensemble of micro-robots arranged in space to form a specific shape and having their own computing power. This technology behaves as a distributed system. Each micro-robot is called a module and the whole forms a modular robot. This paper ...
-
Flying Modular Structures offer a versatile mechanism that can change the arrangement of constituent actuators according to task requirements. In this work, we extend a modular aerial platform that can expand its actuation capabilities depending on the configuration. Each module is composed of a quadrotor in a cage that can rigidly connect with other modules. The quadrotor is connected with the ca...
-
This paper examines a family of designs for magnetic cubes and counts how many configurations are possible for each design as a function of the number of modules. Magnetic modular cubes are cubes with magnets arranged on their faces. The magnets are positioned so that each face has either magnetic south or north pole outward. Moreover, we require that the net magnetic moment of the cube passes thr...
-
Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate results of object detection, affinity computation and data association in real time. This paper presents an efficient multi-modal MOT framework with online joint detection and tracking schemes and robust data association for autonomous driving applications. The novelty of this work includes: (1) development of an end-to-end deep...
-
Autonomous inspection of powerlines with quadrotors is challenging. Flights require persistent perception to keep a close look at the lines. We propose a method that uses event cameras to robustly track powerlines. Event cameras are inherently robust to motion blur, have low latency, and high dynamic range. Such properties are advantageous for autonomous inspection of powerlines with drones, where...
-
High quality object proposals are crucial in visual tracking algorithms that utilize region proposal network (RPN). Refinement of these proposals, typically by box regression and classification in parallel, has been popularly adopted to boost tracking performance. However, it still meets problems when dealing with complex and dynamic background. Thus motivated, in this paper we introduce an improv...
-
Camera calibration is an important prerequisite towards the solution of 3D computer vision problems. Traditional methods rely on static images of a calibration pattern. This raises interesting challenges towards the practical usage of event cameras, which notably require image change to produce sufficient measurements. The current standard for event camera calibration therefore consists of using f...
-
Currently, there have been many kinds of pointbased 3D trackers, while voxel-based methods are still underexplored. In this paper, we first propose a voxel-based tracker, named PointSiamRCNN, improving tracking performance by embedding target information into the search region. Our framework is composed of two parts for achieving proposal generation and proposal refinement, which fully releases th...
-
Generating diverse and comprehensive interacting agents to evaluate the decision-making modules is essential for the safe and robust planning of autonomous vehicles (AV). Due to efficiency and safety concerns, most researchers choose to train interactive adversary (competitive or weakly competitive) agents in simulators and generate test cases to interact with evaluated AVs. However, most existing...
-
Recognising the goals or intentions of observed vehicles is a key step towards predicting the long-term future behaviour of other agents in an autonomous driving scenario. When there are unseen obstacles or occluded vehicles in a scenario, goal recognition may be confounded by the effects of these unseen entities on the behaviour of observed vehicles. Existing prediction algorithms that assume rat...
-
Autonomous control of an emergency vehicle will save lives through faster transport and shorter response. Towards this goal, it must overcome the challenge of inter- acting with existing human drivers on the road. We present a game-theoretic approach for semi-cooperative control of an autonomous emergency vehicle that can interact efficiently with humans on the road. We model the interactions betw...
-
Robust real-time detection and motion forecasting of traffic participants is necessary for autonomous vehicles to safely navigate urban environments. In this paper, we present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation directly from time-series LiDAR data. Instead of the widely used bird’s eye view (BEV) representation, we utilize the native range view (R...
-
Detecting dynamic objects and predicting static road information such as drivable areas and ground heights are crucial for safe autonomous driving. Previous works studied each perception task separately, and lacked a collective quantitative analysis. In this work, we show that it is possible to perform all perception tasks via a simple and efficient multi-task network. Our proposed network, LidarM...
-
Testing and verification is still an open issue on the way to fully automated driving. Simulations can help to reduce the required testing efforts, however, classical simulators based on physical models and heuristics, such as the intelligent driver model (IDM), show limited model accuracy on a microscopic scenario level. In turn, learning-based driver models are often capable to predict human dri...
-
Although autonomous driving technology has made tremendous progress in recent years, it is still challenging to predict the intentions and trajectories of pedestrians. The state-of-the-art methods suffer from two problems. (1) Existing works consider these two tasks separately, ignoring the connection between them. (2) The selection and integration of inputs for these tasks are not well designed. ...
-
The domain of robotics is challenging to apply deep reinforcement learning due to the need for large amounts of data and for ensuring safety during learning. Curriculum learning has shown good performance in terms of sample-efficient deep learning. In this paper, we propose an algorithm (named GloCAL) that creates a curriculum for an agent to learn multiple discrete tasks, based on clustering task...
-
Modeling induction motor dynamics is a crucial problem in the industry. The previous works mainly model the dynamics based on the physical model assumption and state equation. However, due to the complex internal structure of motors, the traditional methods cannot estimate dynamics precisely. To address this issue, we adopt a deep learning-based approach that takes the time-series motor data measu...
-
In this paper, we propose a data-driven modelling framework using a sparse regression technique to find the governing equations of dynamics systems. With this approach, the prior knowledge of features from simple structures can be used to deduce which on complex structures. The prior knowledge of single-pendulums, double-pendulums, and spherical pendulum enlightens the guess of the feature library...
-
Virtually all robot control methods benefit from the availability of an accurate mathematical model of the robot. However, obtaining a sufficient amount of informative data for constructing dynamic models can be difficult, especially when the models are to be learned during robot deployment. Under such circumstances, standard data-driven model learning techniques often yield models that do not com...
-
We report the development of novel fault detection and isolation (FDI) methods for model-based fault detection (MB-FD) and quotient-space fault isolation (QS-FI). This FDI approach performs MB-FD and QS-FI of single or multiple con-current faults in plants and actuators simultaneously, without a priori knowledge of fault form, type, or dynamics. To detect faults, MB-FD characterizes deviation from...
-
As robotic systems move from highly structured environments to open worlds, incorporating uncertainty from dynamics learning or state estimation into the control pipeline is essential for robust performance. In this paper we present a nonlinear particle model predictive control (PMPC) approach to control under uncertainty, which directly incorporates any particle-based uncertainty representation, ...
-
Learning an accurate model of the environment is essential for model-based control tasks. Existing methods in robotic visuomotor control usually learn from data with heavily labelled actions, object entities or locations, which can be demanding in many cases. To cope with this limitation, we propose a method, dubbed DMotion, that trains a forward model from video data only, via disentangling the m...
-
Hand Gesture Recognition (HGR) has application in Human Machine Interfaces (HMIs), to control robots, games, and machines. Here we demonstrate a soft-matter multi-layer printed electronic circuit, that can be used to detect the human gesture without the need for physical contact, except for unlocking the system. The film is able to detect touch and proximity of the hand at various nodes, and thus ...
-
Soft robotic devices, including actuators fabricated from materials with a low modulus of elasticity, such as silicone elastomers, have gained significant interest in recent years. A flexible sensor is a vital component for estimating the conditions of soft actuators, such as shape, and deformation due to contact events. However, it is challenging to develop a flexible sensor with tolerability and...
-
This paper proposes the design of a robotic gripper motivated by the bin-picking problem, where a variety of objects need to be picked from cluttered bins. The presented gripper design focuses on an enveloping cage-like approach, which surrounds the object with three hooked fingers, and then presses into the object with a movable palm. The fingers are flexible and imbue grasps with some elasticity...
-
The need for robotic hands capable of gentle in-hand manipulation is growing rapidly as robots enter the real world. In this work, we show that the arrangement of digits in a soft robotic hand has a strong effect on in-hand manipulation capabilities. Introducing task-based performance metrics which quantify the range of motion, repeatability, and accuracy of in-hand manipulation tasks, we investig...
-
Robot grasping and dexterous, in-hand manipulation allow robots to interact with their surroundings and execute a plethora of complex tasks such as pushing buttons, opening doors, and interacting with electrical appliances. In robotics, such complicated tasks are typically executed by multi-fingered end-effectors that are heavy, rigid, and expensive, employing numerous degrees of freedom and actua...
-
Robot hand is essential for a fully functional robot and designing a good robot hand is a sophisticated job that challenges the designer’s knowledge and experience. This paper presents a computational framework for automatic optimal robot hand design based on reinforcement learning (RL), which considers desired grasping tasks, grasp control strategies, and performance quality measures altogether. ...
-
Robots can grasp, even manipulate, objects with different shape, weight and size thanks to the their end-effectors. These are mostly constituted by two fingers, and are known as grippers. However, despite being quite simple for human beings, manipulation is not so straightforward to carry out on robotic systems. One of the main obstacles is the lack of reliable control methods: this is especially ...
-
In this paper, we present a framework that unites obstacle avoidance and deliberate physical interaction for robotic manipulators. As humans and robots begin to coexist in work and household environments, pure collision avoidance is insufficient, as human–robot contact is inevitable and, in some situations, desired. Our work enables manipulators to anticipate, detect, and act on contact. To achiev...
-
Kinesthetic teaching allows the direct skill transfer from the human to the robot through physical human-robot interaction. However, it is heavily affected by the robot’s dynamics and the control scheme utilized for the physical interaction. In this work, we aim at assisting the human-teacher by reducing her/his physical and cognitive load. To this aim, we propose a controller with virtual fixture...
-
Two main challenges that need to be addressed in physical human-robot interaction (pHRI) are efficient recognition of human intention and interaction safety. In this paper, a general human intention framework was summarized, firstly, according to the robot's roles: a passive follower and a compliant leader. Secondly, we proposed variable admittance control models governed by human intentions. Powe...
-
Assistive robot manipulators help people with upper motor impairments perform tasks by themselves. However, teleoperating a robot to perform complex tasks is difficult. Shared control algorithms make this easier: these algorithms predict the user’s goal, autonomously generate a plan to accomplish the goal, and fuse that plan with the user’s input. To accurately predict the user’s goal, these algor...
-
Assistive robots that operate alongside humans require the ability to understand and replicate human behaviours during a handover. A handover is defined as a joint action between two participants in which a giver hands an object over to the receiver. In this paper, we present a method for learning human-to-human handovers observed from motion capture data. Given the giver and receiver pose from a ...
-
Lower Limb Exoskeletons (LLEs) are promising in gait rehabilitation for stroke survivors. In gait training of post-stroke patients with LLEs, one of the main challenges is how to generate appropriate gait patterns from the sound leg to the paretic leg for different patients with varying walking speeds. In this paper, we proposed a Synergetic Gait Prediction (SGP) model for rehabilitation LLEs with...
-
Human touching the robot to convey intentions or emotions is an essential communication pathway during physical Human-Robot Interaction (pHRI). Therefore, advanced service robots require superior tactile intelligence to guarantee naturalness and safety when making physical contact with human subjects. Tactile intelligence is the capability to percept and recognize tactile information from touch be...
-
Physical human-robot interactions (pHRI) are less efficient and communicative than human-human interactions, and a key reason is a lack of informative sense of touch in robotic systems. Interpreting human touch gestures is a nuanced, challenging task with extreme gaps between human and robot capability. Among prior works that demonstrate human touch recognition capability, differences in sensors, ...
-
This paper presents a technique for state estimation of a multi-link system having no joint encoders, which can only be partially observed by a camera. To fully observe the system without changing the current configuration, a gyroscope and an accelerometer are attached to each link as dead-reckoning sensors. Observations of the dead-reckoning sensors are associated with the states of the multi-lin...
-
We propose a hierarchical control framework for generating versatile motions by a humanoid robot. The central feature of our framework is computational affordability: a large amount of computation time is allowable in the upper-level hierarchy. Consequently, whole-body trajectory optimization for a long time horizon becomes feasible. To ensure such affordability, a fast feedback loop is establishe...
-
To solve the interference problems of wind and wave action and load movement when switching under water surface conditions in the marine environment, a study on the water surface stability prediction of the bio-inspired undulatory fin robot is carried out. Based on the fin motion equation and fluid drag theory, a water surface stability calculation model of the robot is established. The study comp...
-
This paper reports methods to compute the equilibrium stances of a new snake-like robot designed to stabilize its head on a free water surface. To adjust rapidly the stability of the robot, this bio-inspired robot can rotate independently each body-shell, and modify the level of immersion of each module. To predict the stable stance accessible by this additional degree of freedom, a model is devel...
-
It is well understood that nature has a calming effect on us. But in a physical space remote from nature, might the robotic embodiment of a natural phenomenon have the same effect? To address this question, we have simulated the soothing movement of ocean waves in a soft robotic surface, both as a simulation and in a physical prototype. In this paper, we report on our modeling methods of this natu...
Introduction
Conference IROS2021 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 996