DeepNLP ICRA2020 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
This paper presents a robotic gripper design that can perform customizable grasping tasks at the millimeter scale. The design is based on the origami string, a mechanism with a single degree of freedom that can be mechanically programmed to approximate arbitrary paths in space. By using this concept, we create miniature fingers that bend at multiple joints with a single actuator input. The shape a...
-
This paper presents a method to create fish-like robots with tensegrity systems and describes a prototype modeled on the body shape of the rainbow trout with a length of 400 mm and a mass of 102 g that is driven by a waterproof servomotor. The structure of the tensegrity robot consists of rigid body segments and elastic cables that represent bone/tissue and muscles of fish, respectively. This stru...
-
We propose a generative model for the spatio-temporal distribution of high dimensional categorical observations. These are commonly produced by robots equipped with an imaging sensor such as a camera, paired with an image classifier, potentially producing observations over thousands of categories. The proposed approach combines the use of Dirichlet distributions to model sparse co-occurrence relat...
-
This paper describes the development of a tilting locomotion system based on a compliant tensegrity structure with multiple stable equilibrium configurations. A tensegrity structure featuring 4 stable equilibrium states is considered. The mechanical model of the structure is presented and the according equations of motion are derived. The variation of the length of selected structural members allo...
-
Soft robot has demonstrated promise in unstructured and dynamic environments due to unique advantages, such as safe interaction, adaptiveness, easy to actuate, and easy fabrication. However, the highly dissipative nature of elastic materials results in small stiffness of soft robot which limits certain functions, such as force transmission, position accuracy, and load capability. In this paper, we...
-
Soft robots apply compliant materials to perform motions and behaviors not typically achievable by rigid robots. An underwater, compliant, multi-segment continuum manipulator that can bend, swallow, disgorge is developed in this study. The manipulator is driven by McKibben water hydraulic artificial muscle (WHAM). The mechanical properties of the WHAM are tested and analyzed experimentally. The ki...
-
Soft robots are theoretically well-suited to rescue and exploration applications where their flexibility allows for the traversal of highly cluttered environments. However, most existing mobile soft robots are not fast or powerful enough to effectively traverse three dimensional environments. In this paper, we introduce a new mobile robot with a continuously deformable slender body structure, the ...
-
The human thumb's state contribution to grasping and dexterous manipulation of objects is a function of the kinematic multiplicity of joints and structure of the bones, joints, and ligaments. This paper looks at the design and evaluation of a human-like thumb for use in a robotic hand, where the thumb's state contribution to grasping and dexterous manipulation is a function of a simplified kinemat...
-
This paper presents a unique unmanned ground vehicle with a dynamic wheelbase and an adaptive thrust based friction optimization scheme that aids in the traversal of steep slopes and slippery surfaces. The vehicle is capable of adapting itself to the surface topography using an impedance-based stabilization module to minimize the mechanical oscillatory transients induced during its motion. A detai...
-
Enabling long-term operation during day and night for collaborative robots requires a comprehensive understanding of the unstructured environment. Besides, in the dynamic environment, robots must be able to recognize dynamic objects and collaboratively build a global map. This paper proposes a novel approach for dynamic collaborative mapping based on multimodal environmental perception. For each m...
-
Travel over sloped terrain is difficult as an incline changes the interaction between each wheel and the ground resulting in an unbalanced load distribution which can lead to loss of traction and instability. This paper presents a novel approach to generating wheel rotation for primary locomotion by only changing its centre of rotation, or as a complimentary locomotion source to increase versatili...
-
This paper proposes a Guidance, Navigation, and Control (GNC) architecture for planetary rovers targeting the conditions of upcoming Mars exploration missions such as Mars 2020 and the Sample Fetching Rover (SFR). The navigation requirements of these missions demand a control architecture featuring autonomous capabilities to achieve a fast and long traverse. The proposed solution presents a two-le...
-
As the growth of smart home, healthcare, and home robot applications, learning a face recognition system which is specific for a particular environment and capable of self-adapting to the temporal changes in appearance (e.g., caused by illumination or camera position) is nowadays an important topic. In this paper, given a video of a group of people, which simulates the surveillance video in a smar...
-
Mobile robot assistants have many applications, such as helping people in their daily living activities. These robots have to detect and recognize the actions and goals of the humans they are assisting. While there are several wide-spread plan and activity recognition solutions for controlled environments with many built-in sensors, like smart-homes, there is a lack of such systems for mobile robo...
-
Most of the research effort on image-based place recognition is designed for urban environments. In bucolic environments such as natural scenes with low texture and little semantic content, the main challenge is to handle the variations in visual appearance across time such as illumination, weather, vegetation state or viewpoints. The nature of the variations is different and this leads to a diffe...
-
The increasingly-used robotic systems can provide precise delivery and reduce X-ray radiation to medical staff in percutaneous coronary interventions (PCI), but natural manipulations of interventionalists are forgone in most robot-assisted procedures. Therefore, it is necessary to explore natural manipulations to design more advanced human-robot interfaces (HRI). In this study, a multilayer-multim...
-
Simultaneous Localization and Mapping (SLAM) is one of the basic problems in mobile robotics. While most approaches are based on occupancy grid maps, Normal Distributions Transforms (NDT) and mixtures like Occupancy Normal Distribution Transforms (ONDT) have been shown to represent sensor measurements more accurately. In this work, we slightly re-formulate the (O)NDT matching function such that it...
-
Analyzing sensor data of plants and monitoring plant performance is a central element in different agricultural robotics applications. In plant science, phenotyping refers to analyzing plant traits for monitoring growth, for describing plant properties, or characterizing the plant's overall performance. It plays a critical role in the agricultural tasks and in plant breeding. Recently, there is a ...
-
We propose an uncertainty-based sensor fusion framework for visual-inertial odometry, which is the task of estimating relative motion using images and measurements from inertial measurement units. Visual-inertial odometry enables robust and scale-aware estimation of motion by incorporating sensor states, such as metric scale, velocity, and the direction of gravity, into the estimation. However, th...
-
LiDAR odometry and mapping (LOAM) has been playing an important role in autonomous vehicles, due to its ability to simultaneously localize the robot’s pose and build high-precision, high-resolution maps of the surrounding environment. This enables autonomous navigation and safe path planning of autonomous vehicles. In this paper, we present a robust, real-time LOAM algorithm for LiDARs with small ...
-
In this paper, we present an active SLAM framework for volumetric exploration of 3D underwater environments with multibeam sonar. Recent work in integrated SLAM and planning performs localization while maintaining volumetric free-space information. However, an absence of informative loop closures can lead to imperfect maps, and therefore unsafe behavior. To solve this, we propose a navigation poli...
-
Service robots should be able to operate autonomously in dynamic and daily changing environments over an extended period of time. While Simultaneous Localization And Mapping (SLAM) is one of the most fundamental problems for robotic autonomy, most existing SLAM works are evaluated with data sequences that are recorded in a short period of time. In real-world deployment, there can be out-of-sight s...
-
This paper sets a new foundation for data-driven inertial navigation research, where the task is the estimation of horizontal positions and heading direction of a moving subject from a sequence of IMU sensor measurements from a phone. In contrast to existing methods, our method can handle varying phone orientations and placements.More concretely, the paper presents 1) a new benchmark containing mo...
-
We propose a fast and lightweight end-to-end convolutional network architecture for real-time segmentation of high resolution videos, NfS-SegNet, that can segement 2K-videos at 36.5 FPS with 24.3 GFLOPS. This speed and computation-efficiency is due to following reasons: 1) The encoder network, NfS-Net, is optimized for speed with simple building blocks without memory-heavy operations such as depth...
-
The horizon line is an important geometric feature for many image processing and scene understanding tasks in computer vision. For instance, in navigation of autonomous vehicles or driver assistance, it can be used to improve 3D reconstruction as well as for semantic interpretation of dynamic environments. While both algorithms and datasets exist for single images, the problem of horizon line esti...
-
Deployment and operation of autonomous underwater vehicles is expensive and time-consuming. High-quality realistic sonar data simulation could be of benefit to multiple applications, including training of human operators for post-mission analysis, as well as tuning and validation of autonomous target recognition (ATR) systems for underwater vehicles. Producing realistic synthetic sonar imagery is ...
-
Informed sampling-based planning algorithms exploit problem knowledge for better search performance. This knowledge is often expressed as heuristic estimates of solution cost and used to order the search. The practical improvement of this informed search depends on the accuracy of the heuristic.Selecting an appropriate heuristic is difficult. Heuristics applicable to an entire problem domain are o...
-
Robotic manipulation problems are inherently continuous, but typically have underlying discrete structure, e.g., whether or not an object is grasped. This means many problems are multi-modal and in particular have a continuous infinity of modes. For example, in a pick-and-place manipulation domain, every grasp and placement of an object is a mode. Usually manipulation problems require the robot to...
-
State-of-the-art coverage planning methods perform well in simple environments but take an ineffectively long time to converge to an optimal solution in complex three-dimensional (3D) environments. As more structures are present in the same volume of workspace, these methods slow down as they spend more time searching for all of the nooks and crannies concealed in three-dimensional spaces. This wo...
-
The increasing popularity of quadrotors has given rise to a class of predominantly vision-driven vehicles. This paper addresses the problem of perception-aware time optimal path parametrization for quadrotors. Although many different choices of perceptual modalities are available, the low weight and power budgets of quadrotor systems makes a camera ideal for on-board navigation and estimation algo...
-
The safety of an autonomous vehicle not only depends on its own perception of the world around it, but also on the perception and recognition from other vehicles. If an ego vehicle considers the uncertainty other vehicles have about itself, then by reducing the estimated uncertainty it can increase its safety. In this paper, we focus on how an ego vehicle plans its trajectories through the blind s...
-
The field of soft robotics is grounded on the idea that, due to their inherent compliance, soft robots can safely interact with the environment. Thus, the development of effective planning and control pipelines for soft robots should incorporate reliable robot-environment interaction models. This strategy enables soft robots to effectively exploit contacts to autonomously navigate and accomplish t...
-
In this paper, we investigate the consensus problem of multiple unmanned aerial vehicles (UAVs) in the presence of environmental constraints under a general communication topology containing a directed spanning tree. First, based on a position transformation function, we propose a novel dynamic reference position and yaw angle for each UAV to cope with both the asymmetric topology and the constrai...
-
In this paper, we present Neural-Swarm, a nonlinear decentralized stable controller for close-proximity flight of multirotor swarms. Close-proximity control is challenging due to the complex aerodynamic interaction effects between multirotors, such as downwash from higher vehicles to lower ones. Conventional methods often fail to properly capture these interaction effects, resulting in controllers...
-
The line coverage problem is the coverage of linear environment features (e.g., road networks, power lines), modeled as 1D segments, by one or more robots while respecting resource constraints (e.g., battery capacity, flight time) for each of the robots. The robots incur direction dependent costs and resource demands as they traverse the edges. We treat the line coverage problem as an optimization...
-
This paper presents a coverage control algorithm for teams of quadcopters with downward facing visual sensors that prevents the appearance of coverage holes in-between the monitored areas while maximizing the coverage quality as much as possible. We derive necessary and sufficient conditions for preventing the appearance of holes in-between the fields of views among trios of robots. Because this c...
-
Predicting the possible future behaviors of vehicles that drive on shared roads is a crucial task for safe autonomous driving. Many existing approaches to this problem strive to distill all possible vehicle behaviors into a simplified set of high-level actions. However, these action categories do not suffice to describe the full range of maneuvers possible in the complex road networks we encounter...
-
Mobile robots capable of navigating seamlessly and safely in pedestrian rich environments promise to bring robotic assistance closer to our daily lives. In this paper we draw on insights of how humans move in crowded spaces to explore how to recognize pedestrian navigation intent, how to predict pedestrian motion and how a robot may adapt its navigation policy dynamically when facing unexpected hu...
-
Autonomous driving is a dynamically growing field of research, where quality and amount of experimental data is critical. Although several rich datasets are available these days, the demands of researchers and technical possibilities are evolving. Through this paper, we bring a new dataset recorded in Brno - Czech Republic. It offers data from four WUXGA cameras, two 3D LiDARs, inertial measuremen...
-
Decision-making in dense traffic scenarios is challenging for automated vehicles (AVs) due to potentially stochastic behaviors of other traffic participants and perception uncertainties (e.g., tracking noise and prediction errors, etc.). Although the partially observable Markov decision process (POMDP) provides a systematic way to incorporate these uncertainties, it quickly becomes computationally...
-
Autonomous urban driving navigation is still an open problem and has ample room for improvement in unknown complex environments and terrible weather conditions. In this paper, we propose a two-stage framework, called IPP-RL, to handle these problems. IPP means an Imitation learning method fusing visual information with the additional steering angle calculated by Pure-Pursuit (PP) method, and RL me...
-
In the autonomous driving area synthetic data is crucial for cover specific traffic scenarios which autonomous vehicle must handle. This data commonly introduces domain gap between synthetic and real domains. In this paper we deploy data augmentation to generate custom traffic scenarios with VRUs in order to improve pedestrian recognition. We provide a pipeline for augmentation of the Cityscapes d...
-
We present a novel key region extraction method of point cloud, ROI-cloud, for LiDAR odometry and localization with autonomous robots. Traditional methods process massive point cloud data in every region within the field of view. In dense urban environments, however, processing redundant and dynamic regions of point cloud is time-consuming and harmful to the results of matching algorithms. In this...
-
Visual localization is the problem of estimating a camera within a scene and a key technology for autonomous robots. State-of-the-art approaches for accurate visual localization use scene-specific representations, resulting in the overhead of constructing these models when applying the techniques to new scenes. Recently, learned approaches based on relative pose estimation have been proposed, carr...
-
Combining multiple complementary techniques together has long been regarded as a way to improve performance. In visual localization, multi-sensor fusion, multi-process fusion of a single sensing modality, and even combinations of different localization techniques have been shown to result in improved performance. However, merely fusing together different localization techniques does not account fo...
-
Tracking the pose of a camera is at the core of visual localization methods used in many applications. As the observations of a camera are inherently affected by lighting, it has always been a challenge for these methods to cope with varying lighting conditions. Thus far, this issue has mainly been approached with the intent to increase robustness by choosing lighting invariant map representations...
-
Visual place recognition algorithms trade off three key characteristics: their storage footprint, their computational requirements, and their resultant performance, often expressed in terms of recall rate. Significant prior work has investigated highly compact place representations, sub-linear computational scaling and sub-linear storage scaling techniques, but have always involved a significant c...
-
We address the localization of robots in a multi-MAV system where external infrastructure like GPS or motion capture systems may not be available. Our approach lends itself to implementation on platforms with several constraints on size, weight, and power (SWaP). Particularly, our framework fuses the onboard VIO with the anonymous, visual-based robot-to-robot detection to estimate all robot poses ...
-
In this paper we target the problem of transferring policies across multiple environments with different dynamics parameters and motor noise variations, by introducing a framework that decouples the processes of policy learning and system identification. Efficiently transferring learned policies to an unknown environment with changes in dynamics configurations in the presence of motor noise is ver...
-
Deep reinforcement learning (RL) is being actively studied for robot navigation due to its promise of superior performance and robustness. However, most existing deep RL navigation agents are trained using fixed parameters, such as maximum velocities and weightings of reward components. Since the optimal choice of parameters depends on the use-case, it can be difficult to deploy such existing meth...
-
We propose to formulate the problem of representing a distribution of robot configurations (e.g. joint angles) as that of approximating a product of experts. Our approach uses variational inference, a popular method in Bayesian computation, which has several practical advantages over sampling-based techniques. To be able to represent complex and multimodal distributions of configurations, mixture ...
-
In this study, we propose the manipulation and cell injection of a fluorescent microsensor using multiple wavelengths of light. The fluorescent microsensor is made of a 1-μm polystyrene particle containing infrared (IR: 808 nm) absorbing dye and Rhodamine B. The polystyrene particle can be manipulated in water using a 1064-nm laser because the refractive index of the polystyrene is 1.6 (refractive...
-
The Extra Robotic Legs (XRL) system is a robotic augmentation worn by a human operator consisting of two articulated robot legs that walk with the operator and help bear a heavy backpack payload. It is desirable for the Human-XRL quadruped system to walk with the rear legs lead the front by 25% of the gait period, minimizing the energy lost from foot impacts while maximizing balance stability. Unl...
-
After occasional perturbation, it is crucial to spontaneously control the limit cycle walking so that it quickly returns to its closed orbit in phase space. Otherwise, its stability can not be sufficiently guaranteed if the speed of recovery is slow while successive perturbation is applied. The accumulated deviation may eventually drive the phase outside the basin of attraction, leading to failure...
-
Correspondence identification is an essential problem for collaborative multi-robot perception, with the objective of deciding the correspondence of objects that are observed in the field of view of each robot. In this paper, we introduce a novel maximin hypergraph matching approach that formulates correspondence identification as a hypergraph matching problem. The proposed approach incorporates b...
-
We present a scalable distributed target tracking algorithm based on the alternating direction method of multipliers that is well-suited for a fleet of autonomous cars communicating over a vehicle-to-vehicle network. Each sensing vehicle communicates with its neighbors to execute iterations of a Kalman filter-like update such that each agent's estimate approximates the centralized maximum a poster...
-
We present a novel approach to increase the flight time of a multirotor via mid-air docking and in-flight battery switching. A main quadcopter flying using a primary battery has a docking platform attached to it. A `flying battery' - a small quadcopter carrying a secondary battery - is equipped with docking legs that can mate with the main quadcopter's platform. Connectors between the legs and the...
-
Actuator power consumption is a limiting factor in mobile robot design. In this paper we introduce the concept of an energy-recycling actuator, which uses an array of springs and clutches to capture and return elastic energy in parallel with an electric motor. Engaging and disengaging clutches appropriately could reduce electrical energy consumption without sacrificing controllability, but present...
-
Even though mobile robots have been around for decades, trajectory optimization and continuous time collision avoidance remain subject of active research. Existing methods trade off between path quality, computational complexity, and kinodynamic feasibility. This work approaches the problem using a nonlinear model predictive control (NMPC) framework, that is based on a novel convex inner approxima...
-
This paper introduces Action Image, a new grasp proposal representation that allows learning an end-to-end deep-grasping policy. Our model achieves 84% grasp success on 172 real world objects while being trained only in simulation on 48 objects with just naive domain randomization. Similar to computer vision problems, such as object detection, Action Image builds on the idea that object features a...
-
Learning-based grasp pose detection algorithms have boosted the performance of robot grasping, but they usually need manually fine-tuning steps to find the balance between detection accuracy and efficient. In this paper, we discard these intermediate procedures, like sampling grasps and generating grasp proposals, and propose an end-to-end grasp pose detection model. Our model uses the RGB image a...
-
Grasping in cluttered scenes is challenging for robot vision systems, as detection accuracy can be hindered by partial occlusion of objects. We adopt a reinforcement learning (RL) framework and 3D vision architectures to search for feasible viewpoints for grasping by the use of hand-mounted RGB-D cameras. To overcome the disadvantages of photo-realistic environment simulation, we propose a large-s...
-
Grasping for novel objects is important for robot manipulation in unstructured environments. Most of current works require a grasp sampling process to obtain grasp candidates, combined with local feature extractor using deep learning. This pipeline is time-costly, expecially when grasp points are sparse such as at the edge of a bowl.In this paper, we propose an end-to-end approach to directly pred...
-
Transparent objects are a common part of everyday life, yet they possess unique visual properties that make them incredibly difficult for standard 3D sensors to produce accurate depth estimates for. In many cases, they often appear as noisy or distorted approximations of the surfaces that lie behind them. To address these challenges, we present ClearGrasp - a deep learning approach for estimating ...
-
This paper addresses the task of estimating the 6 degrees of freedom pose of a known 3D object from depth information represented by a point cloud. Deep features learned by convolutional neural networks from color information have been the dominant features to be used for inferring object poses, while depth information receives much less attention. However, depth information contains rich geometri...
-
While a great variety of 3D cameras have been introduced in recent years, most publicly available datasets for object recognition and pose estimation focus on one single camera. In this work, we present a dataset of 32 scenes that have been captured by 7 different 3D cameras, totaling 49,294 frames. This allows evaluating the sensitivity of pose estimation algorithms to the specifics of the used c...
-
To teach robots skills, it is crucial to obtain data with supervision. Since annotating real world data is time-consuming and expensive, enabling robots to learn in a self- supervised way is important. In this work, we introduce a robot system for self-supervised 6D object pose estimation. Starting from modules trained in simulation, our system is able to label real world images with accurate 6D o...
-
GelSight sensor has been used to study microgeometry of objects since 2009 in tactile sensing applications. Elastomer, reflective coating, lighting, and camera were the main challenges of making a GelSight sensor within a short period. The recent addition of permanent markers to the GelSight was a new era in shear/slip studies. In our previous studies, we introduced Ultraviolet (UV) ink and UV LED...
-
Individuals living with paralysis or amputation can operate robotic prostheses using input signals based on their intent or attempt to move. Because sensory function is lost or diminished in these individuals, haptic feedback must be non-collocated. The intracortical brain computer interface (iBCI) has enabled a variety of neural prostheses for people with paralysis. An important attribute of the ...
-
The ocean is a harsh and unstructured environment for robotic systems; high ambient pressures, saltwater corrosion and low-light conditions demand machines with robust electrical and mechanical parts that are able to sense and respond to the environment. Prior work shows that the addition of gentle suction flow to the hands of underwater robots can aid in the handling of objects during mobile mani...
-
We revise, improve and extend the system previously introduced by us and named SSM-VPR (Semantic and Spatial Matching Visual Place Recognition), largely boosting its performance above the current state of the art. The system encodes images of places by employing the activations of different layers of a pre-trained, off-the-shelf, VGG16 Convolutional Neural Network (CNN) architecture. It consists o...
-
Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuri...
-
In this paper, we outline an interleaved acting and planning technique to rapidly reduce the uncertainty of the estimated robot's pose by perceiving relevant information from the environment, as recognizing an object or asking someone for a direction. Generally, existing localization approaches rely on low-level geometric features such as points, lines, and planes. While these approaches provide t...
-
In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. We propose a new approach for decentralized task allocation based on a decentralized genetic algorithm (GA). The approach parallelizes a genetic algorithm across the team of agents, making efficient use of their computational resources. In the proposed approach, the agents ...
-
This paper presents a task and motion planning (TAMP) framework for a robotic manipulator in order to retrieve a target object from clutter. We consider a configuration of objects in a confined space with a high density so no collision-free path to the target exists. The robot must relocate some objects to retrieve the target without collisions. For fast completion of object rearrangement, the rob...
-
This paper presents scalable designs and fabrication, actuation, and manipulation techniques for soft millirobots under uniform magnetic field control. The millirobots were fabricated through an economic and robust moulding technique using polydimethylsiloxane (PDMS), acrylonitrile butadiene styrene (ABS) filaments, and 3D printed polylactic acid (PLA) rings. The soft millirobots were simple hollo...
-
In reinforcement learning (RL), sparse rewards are a natural way to specify the task to be learned. However, most RL algorithms struggle to learn in this setting since the learning signal is mostly zeros. In contrast, humans are good at assessing and predicting the future consequences of actions and can serve as good reward/policy shapers to accelerate the robot learning process. Previous works ha...
-
This study aim to confirm the effect of viscosity characteristics differences on the rotational manipulation of a cylindrical rotary controller with the index finger and thumb through a quantitative analysis and evaluation of muscle and brain activations. The target motion was a rotary manipulation with the index finger and thumb of a cylindrical rotary controller with a 50 mm diameter. The rotary...
-
In this paper, we propose a real-time human-robot interface (HRI) system, where Electrooculography (EOG) and Electromyography (EMG) signals were decoded to perform reach-to-grasp movements. For that, five different eye movements (up, down, left, right and rest) were classified in real-time and translated into commands to steer an industrial robot (UR-10) to one of the four approximate target direc...
-
We develop a decoding technique that estimates both the position and torque of a joint of the limb in interaction with an environment based on activities of the agonist-antagonist pair of muscles using electromyography in real time. The long short-term memory (LSTM) network is employed as the core processor of the proposed technique that is capable of learning time series of a long-time span with ...
-
Electromyography (EMG) based interfaces have been used in various robotics studies ranging from teleoperation and telemanipulation applications to the EMG based control of prosthetic, assistive, or robotic rehabilitation devices. But most of these studies have focused on the decoding of user's motion or on the control of the robotic devices in the execution of simple tasks (e.g., grasping tasks). ...
-
Telepresence tele-action robots enable human workers to reliably perform difficult tasks in remote, cluttered, and human environments. However, the effort to control coordinated manipulation and active perception motions may exhaust and intimidate novice workers. We hypothesize that such cognitive efforts would be effectively reduced if the teleoperators are provided with autonomous camera selecti...
-
The need for users' safety and technology accept-ability has incredibly increased with the deployment of co-bots physically interacting with humans in industrial settings, and for people assistance. A well-studied approach to meet these requirements is to ensure human-like robot motions. Classic solutions for anthropomorphic movement generation usually rely on optimization procedures, which build ...
-
Human-multi-robot collaboration is becoming more and more common in intelligent manufacturing. Optimal assembly scheduling of such systems plays a critical role in their production efficiency. Existing approaches mostly consider humans as agents with assumed or known capabilities, which leads to suboptimal performance in realistic applications where human capabilities usually change. In addition, ...
-
Clear Corneal Incision, a challenging step in cataract surgery, and important to the overall quality of the surgery. New surgeons usually spend one full year trying to perfect their incision, but even after such rigorous training deficient incisions can still occur. This paper proposes an autonomous robotic system for this self-sealing incision. A conventional ophthalmic microscope system with a m...
-
This paper introduces a 2-DOF spatial remote center of motion (RCM) tensegrity mechanism, based on a double parallelogram system, dedicated for percutaneous needle insertion. The originality of this mechanism is its ability to be reconfigured and its capacity to perform a decoupled modulation of its stiffness in an asynchronous way. To do so, an analytical stiffness model of the robot is establish...
-
This paper introduces a Model Predictive Control (MPC) strategy for fully-constrained Cable-Driven Parallel Robots. The main advantage of the proposed scheme lies in its ability to explicitly handle cable tension limits. Indeed, the cable tension distribution is performed as an integral part of the main control architecture. This characteristic significantly improves the safety of the system. Expe...
-
Tendon actuated multisection continuum arms have high potential for inspection applications in highly constrained spaces. They generate motion by axial and bending deformations. However, because of the high mechanical coupling between continuum sections, variable length-based kinematic models produce poor results. A new mechanics model for tendon actuated multisection continuum arms is proposed in...
-
This paper presents a trajectory optimization formulation for planning dynamic trajectories of a six-degree-of-freedom (six-DOF) cable-suspended parallel robot (CSPR) that extend beyond the static workspace. The optimization is guided by low-dimensional dynamic models to overcome the local minima and accelerate the exploration of the narrow feasible state space. The dynamic similarity between the ...
-
This study proposes an intelligent spraying system with semantic segmentation of fruit trees in a pear orchard. A fruit tree detection system was developed using the SegNet model, a semantic segmentation structure. The system is trained with images categorized into five distinct classes. The learned deep learning model performed with an accuracy of 83.79%. Further, we fusion depth data from an RGB...
-
Dormant pruning is a major cost component of fresh market tree fruit production, nearly equal in scale to harvesting the fruit. However, relatively little focus has been given to the problem of pruning trees autonomously. In this paper, we introduce a robotic system consisting of an industrial manipulator, an eye-in-hand RGB-D camera configuration, and a custom pneumatic cutter. The system is capa...
-
Progress in autonomous mobile robotics has seen significant advances in the development of many algorithms for motion control and path planning. However, robust performance from these algorithms can often only be expected if the parameters controlling them are tuned specifically for the respective robot model, and optimised for specific scenarios in the environment the robot is working in. Such pa...
-
Riemannian Motion Policies (RMPs) have recently been introduced as a way to specify second-order motion policies defined on robot task spaces. RMP-based approaches have the advantage of being more general than traditional approaches based on operational space control; for example, the generalized task inertia in an RMP can be fully state-dependent, which is particularly effective in designing coll...
-
Mechanical self-stability is often useful for controlling systems in uncertain and unstructured environments because it can regulate processes without explicit state observation or feedback computation. However, the performance of such systems is often not optimised, which begs the question how their dynamics can be naturally augmented by a control law to improve performance metrics. We propose a ...
-
Motion control of underactuated systems through the inverse dynamics contains configuration singularities. These limitations in configuration space mainly stem from the inertial coupling that passive joints/bodies create. In this study, we present a model that is free from singularity while the trajectory of the rotating mass has a small-amplitude sine wave around its circle. First, we derive the ...
-
Capturing large objects of unknown shape and orientation remains a challenge for most robotic grippers. We present a highly under-actuated gripper well suited for this task. Prior work shows two primary limitations to these grippers: the grip force of each link tends to decrease as the number of links increases, and the stability of an under-actuated linkage depends on the configuration of the lin...
-
Autonomous driving in an unregulated urban crowd is an outstanding challenge, especially, in the presence of many aggressive, high-speed traffic participants. This paper presents SUMMIT, a high-fidelity simulator that facilitates the development and testing of crowd-driving algorithms. By leveraging the open-source OpenStreetMap map database and a heterogeneous multi-agent motion prediction model ...
-
Robotic Wire Arc Additive Manufacturing (WAAM) utilizes a robot arm as a motion system to build 3D metallic objects by depositing weld beads one above the other in a layer by layer fashion. A key part of this approach is the process study and control of Multi-Layer Multi-Bead (MLMB) deposition, which is very sensitive to process parameters and prone to error stacking. Despite its importance, it ha...
-
The most widely used methods for toolpath planning in 3D printing slice the input model into successive 2D layers to construct the toolpath. Unfortunately the methods can incur a substantial amount of wasted motion (i.e., the extruder is moving while not printing). In recent years we have introduced a new paradigm that characterizes the space of feasible toolpaths using a dependency graph on the i...
-
Flexible manufacturing and automation require robots that can be adapted to changing tasks. We propose to use modular robots that are customized from given modules for a specific task. This work presents an algorithm for proposing a module composition that is optimal with respect to performance metrics such as cycle time and energy efficiency, while considering kinematic, dynamic, and obstacle con...
-
Learning robotic manipulation tasks using reinforcement learning with sparse rewards is currently impractical due to the outrageous data requirements. Many practical tasks require manipulation of multiple objects, and the complexity of such tasks increases with the number of objects. Learning from a curriculum of increasingly complex tasks appears to be a natural solution, but unfortunately, does ...
-
We present an approach to the distributed storage of data across a swarm of mobile robots that forms a shared global memory. We assume that external storage infrastructure is absent, and that each robot is capable of devoting a quota of memory and bandwidth to distributed storage. Our approach is motivated by the insight that in many applications data is collected at the periphery of a swarm topol...
-
This paper introduces a new approach for victim localization in avalanches that will be exploited by UAVs using the ARVA sensor. We show that the nominal ARVA measurement can be linearly related to a quantity that is sufficient to reconstruct the victim position. We explicitly deal with a robust scenario in which the measurement is actually perturbed by a noise that grows with the distance to the ...
-
Autonomous navigation in unknown environments with the intent of exploring all traversable areas is a significant challenge for robotic platforms. In this paper, a simple yet reliable method for exploring unknown environments is presented based on bio-inspired reactive control and metric-topological planning. The reactive control algorithm is modeled after the spatial decomposition of wide and sma...
-
This paper focuses on exploration and occupancy mapping of unknown environments using a mobile robot. While a truncated signed distance field (TSDF) is a popular, efficient, and highly accurate representation of occupancy, few works have considered optimizing robot sensing trajectories for autonomous TSDF mapping. We propose an efficient approach for maintaining TSDF uncertainty and predicting its...
-
Deep learning has enjoyed much recent success, and applying state-of-the-art model learning methods to controls is an exciting prospect. However, there is a strong reluctance to use these methods on safety-critical systems, which have constraints on safety, stability, and real-time performance. We propose a framework which satisfies these constraints while allowing the use of deep neural networks ...
-
In this paper, we present a 3D factor-graph LiDAR-SLAM system which incorporates a state-of-the-art deeply learned feature-based loop closure detector to enable a legged robot to localize and map in industrial environments. Point clouds are accumulated using an inertial-kinematic state estimator before being aligned using ICP registration. To close loops we use a loop proposal mechanism which matc...
-
In modern visual SLAM systems, it is a standard practice to retrieve potential candidate map points from overlapping keyframes for further feature matching or direct tracking. In this work, we argue that keyframes are not the optimal choice for this task, due to several inherent limitations, such as weak geometric reasoning and poor scalability. We propose a voxel-map representation to efficiently...
-
Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos. Our method learns a general skill embedding independently from the task context by using an adver...
-
Spiking Neural Networks (SNNs) are bio-inspired networks that process information conveyed as temporal spikes rather than numeric values. An example of a sensor providing such data is the event-camera. It only produces an event when a pixel reports a significant brightness change. Similarly, the spiking neuron of an SNN only produces a spike whenever a significant number of spikes occur within a s...
-
In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue. Some recent deep learning works learn VO in an end-to-end mann...
-
Camera localization is a fundamental and key component of autonomous driving vehicles and mobile robots to localize themselves globally for further environment perception, path planning and motion control. Recently end-to-end approaches based on convolutional neural network have been much studied to achieve or even exceed 3D-geometry based traditional methods. In this work, we propose a compact ne...
-
Exploration in environments with sparse feed-back remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience...
-
We present TrueRMA, a data-efficient, model-free method to learn cost-optimized robot trajectories over a wide range of starting points and endpoints. The key idea is to calculate trajectory waypoints in Cartesian space by recursively predicting orthogonal adaptations relative to the midpoints of straight lines. We generate a differentiable path by adding circular blends around the waypoints, calc...
-
For robots using motion planning algorithms such as RRT and RRT*, the computational load can vary by orders of magnitude as the complexity of the local environment changes. To adaptively provide such computation, we propose Fog Robotics algorithms in which cloud-based serverless lambda computing provides parallel computation on demand. To use this parallelism, we propose novel motion planning algo...
-
Mobile robots can be used in numerous outdoor tasks such as patrolling, delivery and military applications. In order to deploy mobile robots in this kind of environment, where there are different challenges like slopes, elevations, or even holes, they should be able to detect such challenges and determine the best path to accomplish their tasks. In this paper, we are proposing an exploration appro...
-
We introduce R3T, a reachability-based variant of the rapidly-exploring random tree (RRT) algorithm that is suitable for (optimal) kinodynamic planning in nonlinear and hybrid systems. We developed tools to approximate reachable sets using polytopes and perform sampling-based planning with them. This method has a unique advantage in hybrid systems: different dynamic modes in the reachable set can ...
-
Planning in unstructured environments is challenging - it relies on sensing, perception, scene reconstruction, and reasoning about various uncertainties. We propose DeepSemanticHPPC, a novel uncertainty-aware hypothesis-based planner for unstructured environments. Our algorithmic pipeline consists of: a deep Bayesian neural network which segments surfaces with uncertainty estimates; a flexible poi...
-
We study a novel class of motion planning problems, inspired by emerging low-energy robotic vehicles, such as insect-size flyers, chip-size satellites, and high-endurance autonomous blimps, for which the energy consumed by computing hardware during planning a path can be as large as the energy consumed by actuation hardware during the execution of the same path. We propose a new algorithm, called ...
-
Collision checking is a computational bottleneck in motion planning, requiring lazy algorithms that explicitly reason about when to perform this computation. Optimism in the face of collision uncertainty minimizes the number of checks before finding the shortest path. However, this may take a prohibitively long time to compute, with no other feasible paths discovered during this period. For many r...
-
This study has developed a fault-tolerant controller that is able to recover a quadrotor from arbitrary initial orientations and angular velocities, despite the complete failure of a rotor. This cascaded control method includes a position/altitude controller, an almost-global convergence attitude controller, and a control allocation method based on quadratic programming. As a major novelty, a cons...
-
This paper proposes a model identification method and evaluation of a force model for multirotor UAVs. The model incorporates propellers' aerodynamics derived from momentum and blade element theories, as well as aerodynamics of the UAV's structure and actuation dynamics. A two-steps identification approach of the model parameters is proposed. The model is identified and evaluated from outdoor expe...
-
This paper presents a preliminary study of an Aerial Manipulator suspended by a spring to a robotic carrier. The suspended aerial manipulator is actuated by six pairs of contra-rotating propellers generating a 6-DoF wrench. Simulations show path following results using a computed torque (feedback linearization) control strategy. Active vibration canceling is validated experimentally on a first pro...
-
Event cameras are a promising candidate to enable high speed vision-based control due to their low sensor latency and high temporal resolution. However, purely event-based feedback has yet to be used in the control of drones. In this work, a first step towards implementing low-latency high-bandwidth control of quadrotors using event cameras is taken. In particular, this paper addresses the problem...
-
In this paper, we present a Perception-constrained Nonlinear Model Predictive Control (NMPC) framework for the real-time control of multi-rotor aerial vehicles. Our formulation considers both constraints from a perceptive sensor and realistic actuator limitations that are the rotor minimum and maximum speeds and accelerations. The formulation is meant to be generic and considers a large range of m...
-
In this work, we derive a coordinate-free formulation of the coupled dynamics of a class of 6DOF aerial manipulators consisting of an underactuated quadrotor equipped with a 2DOF articulated manipulator, and demonstrate that the system is differentially flat with respect to the end effector pose. In particular, we require the center of mass of the entire system to be fixed in the end effector fram...
-
CMTS: A Conditional Multiple Trajectory Synthesizer for Generating Safety-Critical Driving Scenarios
Naturalistic driving trajectory generation is crucial for the development of autonomous driving algorithms. However, most of the data is collected in collision-free scenarios leading to the sparsity of the safety-critical cases. When considering safety, testing algorithms in near-miss scenarios that rarely show up in off-the-shelf datasets and are costly to accumulate is a vital part of the evalua...
-
Environmental fluctuations pose crucial challenges to a localization system in autonomous driving. We present a robust LiDAR localization system that maintains its kinematic estimation in changing urban scenarios by using a dead reckoning solution implemented through a LiDAR inertial odometry. Our localization framework jointly uses information from complementary modalities such as global matching...
-
The common pipeline in autonomous driving systems is highly modular and includes a perception component which extracts lists of surrounding objects and passes these lists to a high-level decision component. In this case, leveraging the benefits of deep reinforcement learning for high-level decision making requires special architectures to deal with multiple variable-length sequences of different o...
-
Anticipating the future trajectories of surrounding vehicles is a crucial and challenging task in path planning for autonomy. We propose a novel Convolutional Long Short Term Memory (Conv-LSTM) based neural network architecture to predict the future positions of cars using several seconds of historical driving observations. This consists of three modules: 1) Interaction Learning to capture the eff...
-
Learning an optimal policy for autonomous driving task to confront with complex environment is a long- studied challenge. Imitative reinforcement learning is accepted as a promising approach to learn a robust driving policy through expert demonstrations and interactions with environments. However, this model utilizes non-smooth rewards, which have a negative impact on matching between navigation c...
-
We present a novel algorithm (GraphRQI) to identify driver behaviors from road-agent trajectories. Our approach assumes that the road-agents exhibit a range of driving traits, such as aggressive or conservative driving. Moreover, these traits affect the trajectories of nearby road-agents as well as the interactions between road-agents. We represent these inter-agent interactions using unweighted a...
-
This paper presents a system for robust, large-scale topological localisation using Frequency-Modulated Continuous-Wave scanning radar which extends the state-of-the-art by an efficient, learning-based approach to handle radar data for localisation. We learn a metric space for embedding polar radar scans using CNN and NetVLAD architectures traditionally applied to the visual domain. However, we ta...
-
Global localization is an important and widely studied problem for many robotic applications. Place recognition approaches can be exploited to solve this task, e.g., in the autonomous driving field. While most vision-based approaches match an image w.r.t. an image database, global visual localization within LiDAR-maps remains fairly unexplored, even though the path toward high definition 3D maps, ...
-
Visual place recognition in changing environments is the problem of finding matchings between two sets of observations, a query set and a reference set, despite severe appearance changes. Recently, image comparison using CNNbased descriptors showed very promising results. However, the experiments in the literature typically assume a single distinctive condition within each set (e.g., reference ima...
-
In this paper we deal with the problem of odometry and localization for Lidar-equipped vehicles driving in urban environments, where a premade target map exists to localize against. In our problem formulation, to correct the accumulated drift of the Lidar-only odometry we apply a place recognition method to detect geometrically similar locations between the online 3D point cloud and the a priori o...
-
This paper proposes a novel approach for global localisation of mobile robots in large-scale environments. Our method leverages learning-based localisation and filtering-based localisation, to localise the robot efficiently and precisely through seeding Monte Carlo Localisation (MCL) with a deeplearned distribution. In particular, a fast localisation system rapidly estimates the 6-DOF pose through...
-
This paper deals with the localization problem of a robot in an environment made of indistinguishable landmarks, and assuming the initial position of the vehicle is unknown. This scenario is typically encountered in underwater applications for which landmarks such as rocks all look alike. Furthermore, the position of the robot may be lost during a diving phase, which obliges us to consider unknown...
-
In recent years, a myriad of advanced results have been reported in the community of imitation learning, ranging from parametric to non-parametric, probabilistic to non-probabilistic and Bayesian to frequentist approaches. Meanwhile, ample applications (e.g., grasping tasks and humanrobot collaborations) further show the applicability of imitation learning in a wide range of domains. While numerou...
-
Non-linear dynamical systems represent a compact, flexible, and robust tool for reactive motion generation. The effectiveness of dynamical systems relies on their ability to accurately represent stable motions. Several approaches have been proposed to learn stable and accurate motions from demonstration. Some approaches work by separating accuracy and stability into two learning problems, which in...
-
Learning from offline task demonstrations is a problem of great interest in robotics. For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task. However, leveraging a fixed batch of data can be problematic for larger datasets and longer-horizon tasks with greater ...
-
In many robot control problems, factors such as stiffness and damping matrices and manipulability ellipsoids are naturally represented as symmetric positive definite (SPD) matrices, which capture the specific geometric characteristics of those factors. Typical learned skill models such as dynamic movement primitives (DMPs) can not, however, be directly employed with quantities expressed as SPD mat...
-
This work introduces a learning-based pile loading controller for autonomous robotic wheel loaders. Controller parameters are learnt from a small number of demonstrations for which low level sensor (boom angle, bucket angle and hydrostatic driving pressure), egocentric video frames and control signals are recorded. Application specific deep visual features are learnt from demonstrations using a Si...
-
This paper focuses on inverse reinforcement learning (IRL) to enable safe and efficient autonomous navigation in unknown partially observable environments. The objective is to infer a cost function that explains expert-demonstrated navigation behavior while relying only on the observations and state-control trajectory used by the expert. We develop a cost function representation composed of two pa...
-
Retinal vein cannulation is a promising approach for treating retinal vein occlusion that involves injecting medicine into the occluded vessel to dissolve the clot. The approach remains largely unexploited clinically due to surgeon limitations in detecting interaction forces between surgical tools and retinal tissue. In this paper, a dual force constraint controller for robot-assisted retinal surg...
-
In conventional robotic surgery, the manipulating methods exhibit limitations that are strongly related to the advantages and disadvantages of a pinch grip and power grip. The context of this study is focused on the introduction of a combined grip to compensate for such restraints. In particular, this study proposed the combined-grip-handle scheme on a master manipulator. In this framework, the po...
-
Contact force quality is one of the most critical factors for safe and effective lesion formation during cardiac ablation. The contact force and contact stability plays important roles in determining the lesion size and creating a gap-free lesion. In this paper, the contact stability of a novel magnetic resonance imaging (MRI)-actuated robotic catheter under tissue surface motion is studied. The r...
-
Fast and accurate intracorporeal targeting through an anatomical orifice exhibiting unknown behavior
Surgery may involve precise instrument tip positioning in a minimally invasive way. During these operations, the instrument is inserted in the body through an orifice. The movements of the instrument are constrained by interaction forces arising at the orifice level. The physical constraints may drastically vary depending on the patient’s anatomy. This introduces uncertainties that challenge the p...
-
Existing approaches for robotic control of magnetic swarms are not capable of generating magnetic aggregates precisely in an arbitrarily specified target region in a fluidic flow environment. Such a swarm control capability is demanded by medical applications such as clinical embolization (i.e., localized clogging of blood vessels). This paper presents a new magnetic swarm control strategy to gene...
-
In this paper, a bilateral teleoperation control of a serial robot manipulator, which guarantees a Remote Center of Motion (RCM) constraint in its kinematic level, is developed. A two-layered approach based on the energy tank model is proposed to achieve haptic feedback on the end effector with a pedal switch. The redundancy of the manipulator is exploited to maintain the RCM constraint using the ...
-
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the...
-
Posture control for legged robots has been widely developed on custom-designed robotic platforms, with little work being done on commercially-available robots despite their potential as low-cost research platforms. This paper presents the implementation of a Walking Posture Control system on a commercially-available hexapod robot which utilizes low-cost joint actuators without torque control capab...
-
This paper presents a novel filter architecture that allows a team of vehicles to collaboratively localize using Terrain Relative Navigation (TRN). The work explores several causes of measurement correlation that preclude the use of traditional estimators, and proposes an estimator structure that eliminates one source of measurement correlation while properly incorporating others through the use o...
-
This paper addresses the problem of a domain becoming non-convex while using coverage control of a multirobot system over time-varying domains. When the domain moves around in the workspace, its motion and the presence of obstacles might cause it to deform into some non-convex shape, and the robot team should act in a coordinating manner to maintain coverage. The proposed solution is based on a fr...
-
We consider the problem of controlling a large fleet of drones to deliver packages simultaneously across broad urban areas. To conserve energy, drones hop between public transit vehicles (e.g., buses and trams). We design a comprehensive algorithmic framework that strives to minimize the maximum time to complete any delivery. We address the multifaceted complexity of the problem through a two-laye...
-
We address the problem of maintaining resource availability in a networked multi-robot system performing distributed target tracking. In our model, robots are equipped with sensing and computational resources enabling them to track a target's position using a Distributed Kalman Filter (DKF). We use the trace of each robot's sensor measurement noise covariance matrix as a measure of sensing quality...
-
Multi-robot systems are able to achieve common objectives exchanging information among each other. This is possible exploiting a communication structure, usually modeled as a graph, whose topological properties (such as connectivity) are very relevant in the overall performance of the multirobot system. When considering mobile robots, such properties can change over time: robots are then controlle...
-
Social insects successfully create bridges, rafts, nests and other structures out of their own bodies and do so with no centralized control system, simply by following local rules. For example, while traversing rough terrain, army ants (genus Eciton) build bridges which grow and dissolve in response to local traffic. Because these self-assembled structures incorporate smart, flexible materials (i....
-
The flexible under-actuated musculoskeletal hand is superior in its adaptability and impact resistance. On the other hand, since the relationship between sensors and actuators cannot be uniquely determined, almost all its controls are based on feedforward controls. When grasping and using a tool, the contact state of the hand gradually changes due to the inertia of the tool or impact of action, an...
-
Transfer learning is a popular approach to bypassing data limitations in one domain by leveraging data from another domain. This is especially useful in robotics, as it allows practitioners to reduce data collection with physical robots, which can be time-consuming and cause wear and tear. The most common way of doing this with neural networks is to take an existing neural network, and simply trai...
-
In this paper we introduce a novel technique that aims to dynamically control a modular bio-inspired soft-robotic arm in order to perform cyclic rhythmic patterns. Oscillatory signals are produced at the actuator's level by a central pattern generator (CPG), resulting in the generation of a periodic motion by the robot's end-effector. The proposed controller is based on a model-free neurodynamic s...
-
We present a parallel robot mechanism and the constitutive laws that govern the deformation of its constituent soft actuators. Our ultimate goal is the real-time motion-correction of a patient's head deviation from a target pose where the soft actuators control the position of the patient's cranial region on a treatment machine. We describe the mechanism, derive the stress-strain constitutive laws...
-
In this work, we present an effective multi-view approach to closed-loop end-to-end learning of precise manipulation tasks that are 3D in nature. Our method learns to accomplish these tasks using multiple statically placed but uncalibrated RGB camera views without building an explicit 3D representation such as a pointcloud or voxel grid. This multi-camera approach achieves superior task performanc...
-
Precision cutting of soft-tissue remains a challenging problem in robotics, due to the complex and unpredictable mechanical behaviour of tissue under manipulation. Here, we consider the challenge of cutting along the boundary between two soft mediums, a problem that is made extremely difficult due to visibility constraints, which means that the precise location of the cutting trajectory is typical...
-
In this paper we present a Deep Reinforcement Learning approach to solve dynamic cloth manipulation tasks. Differing from the case of rigid objects, we stress that the followed trajectory (including speed and acceleration) has a decisive influence on the final state of cloth, which can greatly vary even if the positions reached by the grasped points are the same. We explore how goal positions for ...
-
Manipulation tasks such as preparing a meal or assembling furniture remain highly challenging for robotics and vision. Traditional task and motion planning (TAMP) methods can solve complex tasks but require full state observability and are not adapted to dynamic scene changes. Recent learning methods can operate directly on visual inputs but typically require many demonstrations and/or task-specif...
-
What is a proper representation for objects in manipulation? What would human try to perceive when manipulating a new object in a new environment? In fact, instead of focusing on the texture and illumination, human can infer the "affordance" [36] of the objects from vision. Here "affordance" describes the object's intrinsic property that affords a particular type of manipulation. In this work, we ...
-
UAVs require reliable, cost-efficient onboard flight state estimation that achieves high accuracy and robustness to perturbation. We analyze a multi-sensor extended Kalman filter (EKF) based on the work by Leutenegger. The EKF uses measurements from a MEMS-based inertial system, static and dynamic pressure sensors as well as GPS. As opposed to other implementations we do not use a magnetic sensor ...
-
In this paper, we present an open platform, termed OpenVINS, for visual-inertial estimation research for both the academic community and practitioners from industry. The open sourced codebase provides a foundation for researchers and engineers to quickly start developing new capabilities for their visual-inertial systems. This codebase has out of the box support for commonly desired visual-inertia...
-
In this paper, we present a Quaternion-based Error-State Extended Kalman Filter (Q-ESEKF) based on IMU propagation with an extension for Collaborative State Estimation (CSE) and a communication complexity of O(1) (in terms of required communication links). Our approach combines a versatile filter formulation with the concept of CSE, allowing independent state estimation on each of the agents and a...
-
Batch optimization based inertial measurement unit (IMU) and visual sensor fusion enables high rate localization for many robotic tasks. However, it remains a challenge to ensure that the batch optimization is computationally efficient while being consistent for high rate IMU measurements without marginalization. In this paper, we derive inspiration from maximum likelihood estimation with partial-...
-
In this work, we propose a new estimation method for second-order kinematics for floating-base robots, based on highly redundant distributed inertial feedback. The linear acceleration of each robot link is measured at multiple points using a multimodal, self-configuring and self-calibrating artificial skin. The proposed algorithm is two-fold: i) the skin acceleration data is fused at the link leve...
-
The Dynamic Visual Sensor is considered to be a next-generation vision sensor. Since event-based vision is in its early stage of development, a small number of datasets has been created during the last decade. Dataset creation is motivated by the need for real data from one or many sensors. Temporal accuracy of data in such datasets is crucially important since the events have high temporal resolu...
-
Robots are more and more often designed in order to perform tasks in synergy with human operators. In this context, a current research focus for collaborative robotics lies in the design of high-performance control solutions, which ensure security in spite of unmodeled external forces. The present work provides a method based on Model Predictive Control (MPC) to allow compliant behavior when inter...
-
Collaborative robots should ideally use low torque actuators for passive safety reasons. However, some applications require these collaborative robots to reach deep into confined spaces while assisting a human operator in physically demanding tasks. In this paper, we consider the use of in-situ collaborative robots (ISCRs) that balance the conflicting demands of passive safety dictating low torque...
-
For safe human-robot interaction, various type of flexible manipulators have been developed. Especially series elastic actuator (SEA) based manipulators have been getting huge attention since the elastic element of SEA prevents people from injury when undesirable collision happens. Moreover, it improves system durability by absorbing impact force, which could damage actuators. However, the elastic...
-
Many manipulation tasks require coordinated motions for arm and fingers. Complexity increases when the task requires to control for the force at contact against a non-flat surface; This becomes even more challenging when this contact is done on a human. All these challenges are regrouped when one, for instance, massages a human limb. When massaging, the robotic arm is required to continuously adap...
-
The problem of human-exoskeleton interaction with uncertain dynamical parameters remains an open-ended research area. It requires an elaborate control strategy design of the exoskeleton to accommodate complex and unpredictable human body movements. In this paper, we proposed a novel control approach for the lower limb exoskeleton to realize its task of assisting the human operator walking. The mai...
-
Mapping and localization, preferably from a small number of observations, are fundamental tasks in robotics. We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN). The DMN constructs a spatially structured view-embedding map and uses it for subsequent visual localizatio...
-
This paper proposes an end-to-end self-supervised feature representation network named Attentive Task-Net or AT-Net for video-based task imitation. The proposed AT-Net incorporates a novel multi-level spatial attention module to highlight spatial features corresponding to the intended task demonstrated by the expert. The neural connections in AT-Net ensure the relevant information in the demonstra...
-
The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over non-varying distributions for a fixed set of tasks. Fully retraining models eac...
-
ImageNet-pretrained networks have been widely used in transfer learning for monocular depth estimation. These pretrained networks are trained with classification losses for which only semantic information is exploited while spatial information is ignored. However, both semantic and spatial information is important for per-pixel depth estimation. In this paper, we design a novel self-supervised geo...
-
Accurate motion control of surgical robots is critical for the efficiency and safety of both state-of-the-art teleoperated robotic surgery and the ultimate autonomous robotic surgery. However, fine motion control for a flexible endoscopic surgical robot is highly challenging because of the shape-dependent and speed-dependent motion hysteresis of tendon-sheath mechanisms (TSMs) in the long, tortuou...
-
High resolution tactile sensors are often bulky and have shape profiles that make them awkward for use in manipulation. This becomes important when using such sensors as fingertips for dexterous multi-fingered hands, where boxy or planar fingertips limit the available set of smooth manipulation strategies. High resolution optical based sensors such as GelSight have until now been constrained to re...
-
In this paper, we present FootTile, a foot sensor for reaction force and center of pressure sensing in challenging terrain. We compare our sensor design to standard biomechanical devices, force plates and pressure plates. We show that FootTile can accurately estimate force and pressure distribution during legged locomotion. FootTile weighs 0.9 g, has a sampling rate of 330 Hz, a footprint of 10×10...
-
Fall prevention is one of the most important components in senior care. We present a technique to augment an assistive walking device with the ability to prevent falls. Given an existing walking device, our method develops a fall predictor and a recovery policy by utilizing the onboard sensors and actuators. The key component of our method is a robust human walking policy that models realistic hum...
-
This paper proposes a relatively simple function of assistive force for a belt-type hip assist suit developed by the authors' group. The function, which is inspired by the muscle force of the rectus femoris, contains only two parameters, the magnitude and a phase shift factor. Thus, it can reduce the amount of calculation in generating the desired assistive force during walking. Tests were perform...
-
This paper discusses the design and preliminary evaluation of a soft pneumatic socket (SPS) with real-time pressure regulation and an automated underactuated donning mechanism (UDM). The ability to modulate the pressure at the human-socket interface of a prosthesis or wearable device to accommodate user's activities has the potential to make the user more comfortable. Furthermore, a hands-free, un...
-
Exosuits can reduce metabolic demand and improve gait. Controllers explicitly derived from biological mechanisms that reflect the user's joint or muscle dynamics should in theory allow for individualized assistance and enable adaptation to changing gait. With the goal of developing an exosuit control strategy based on muscle power, we present an approach for estimating, at real time rates, when th...
-
Soft strain sensors have been explored as an unobtrusive approach for wearable motion tracking. However, accurate tracking of multi degree-of-freedom (DOF) noncyclic joint movements remains a challenge. This paper presents a soft sensing shirt for tracking shoulder kinematics of both cyclic and random arm movements in 3 DOFs: adduction/abduction, horizontal flexion/extension, and internal/external...
-
We address goal-based imitation learning, where the aim is to output the symbolic goal from a third-person video demonstration. This enables the robot to plan for execution and reproduce the same goal in a completely different environment. The key challenge is that the goal of a video demonstration is often ambiguous at the level of semantic actions. The human demonstrators might unintentionally a...
-
Flexible adaptation of learning strategy depending on online changes of the user's current intents have a high relevance in human-robot collaboration. In our previous study, we proposed an intrinsic interactive reinforcement learning approach for human-robot interaction, in which a robot learns his/her action strategy based on intrinsic human feedback that is generated in the human's brain as neur...
-
Generating a natural question can enable autonomous robots to propose problems according to their surroundings. However, recent studies on question generation rarely consider semantic graph mapping, which is widely used to understand environments. In this paper, we introduce a method to generate natural questions using object-oriented semantic graphs. First, a graph convolutional network extracts ...
-
Safety in Human-Robot Collaboration (HRC) is a bottleneck to HRC-productivity in industry. With robots being the main source of hazards, safety engineers use over-emphasized safety measures, and carry out lengthy and expensive risk assessment processes on each HRC-layout reconfiguration. Recent advances in deep Reinforcement Learning (RL) offer solutions to add intelligence and comprehensibility o...
-
We demonstrate how a sampling-based robotic planner can be augmented to learn to understand a sequence of natural language commands in a continuous configuration space to move and manipulate objects. Our approach combines a deep network structured according to the parse of a complex command that includes objects, verbs, spatial relations, and attributes, with a sampling-based planner, RRT. A recur...
-
Enabling a broader range of users to efficiently deploy autonomous mobile robots requires intuitive frameworks for specifying a robot's task and behaviour. We present a novel approach using learning from corrections (LfC), where a user is iteratively presented with a solution to a motion planning problem. Users might have preferences about parts of a robot's environment that are suitable for robot...
-
Autonomous navigation is a pre-requisite for field robots to carry out precision agriculture tasks. Typically, a robot has to navigate along a crop field multiple times during a season for monitoring the plants, for applying agrochemicals, or for performing targeted interventions. In this paper, we propose a visual-based navigation framework tailored to row-crop fields that exploits the regular cr...
-
In this paper, we consider the Constant-cost Orienteering Problem (COP) where a robot, constrained by a limited travel budget, aims at selecting a path with the largest reward in an aisle-graph. The aisle-graph consists of a set of loosely connected rows where the robot can change lane only at either end, but not in the middle. Even when considering this special type of graphs, the orienteering pr...
-
This paper presents a dynamic stability-constrained optimal motion planning algorithm developed for a timber harvesting machine working on rough terrain. First, the kinematics model of the machine, and the Zero Moment Point (ZMP) stability measure is presented. Then, an approach to simplify the model to gain insight and achieve a fast solution of the optimization problem is introduced. The perform...
-
Selectively picking a target fruit surrounded by obstacles is one of the major challenges for fruit harvesting robots. Different from traditional obstacle avoidance methods, this paper presents an active obstacle separation strategy that combines push and drag motions. The separation motion and trajectory are generated based on the 3D visual perception of the obstacle information around the target...
-
Tracking the pose of an object while it is being held and manipulated by a robot hand is difficult for vision-based methods due to significant occlusions. Prior works have explored using contact feedback and particle filters to localize in-hand objects. However, they have mostly focused on the static grasp setting and not when the object is in motion, as doing so requires modeling of complex conta...
-
Many manipulation tasks, such as placement or within-hand manipulation, require the object's pose relative to a robot hand. The task is difficult when the hand significantly occludes the object. It is especially hard for adaptive hands, for which it is not easy to detect the finger's configuration. In addition, RGB-only approaches face issues with texture-less objects or when the hand and the obje...
-
Accurate 6D object pose estimation is fundamental to robotic manipulation and grasping. Previous methods follow a local optimization approach which minimizes the distance between closest point pairs to handle the rotation ambiguity of symmetric objects. In this work, we propose a novel discrete- continuous formulation for rotation regression to resolve this local-optimum problem. We uniformly samp...
-
Extracting a known target object from a pile of other objects in a cluttered environment is a challenging robotic manipulation task encountered in many robotic applications. In such conditions, the target object touches or is covered by adjacent obstacle objects, thus rendering traditional grasping techniques ineffective. In this paper, we propose a pushing policy aiming at singulating the target ...
-
Grasping in cluttered environments is a fundamental but challenging robotic skill. It requires both reasoning about unseen object parts and potential collisions with the manipulator. Most existing data-driven approaches avoid this problem by limiting themselves to top-down planar grasps which is insufficient for many real-world scenarios and greatly limits possible grasps. We present a method that...
-
In this paper, we introduce a novel single shot approach for 6D object pose estimation of rigid objects based on depth images. For this purpose, a fully convolutional neural network is employed, where the 3D input data is spatially discretized and pose estimation is considered as a regression task that is solved locally on the resulting volume elements. With 65 fps on a GPU, our Object Pose Networ...
-
This paper introduces a multimodal range dataset namely for radio detection and ranging (radar) and light detection and ranging (LiDAR) specifically targeting the urban environment. By extending our workshop paper [1] to a larger scale, this dataset focuses on the range sensor-based place recognition and provides 6D baseline trajectories of a vehicle for place recognition ground truth. Provided ra...
-
Initialization is essential to monocular Simultaneous Localization and Mapping (SLAM) problems. This paper focuses on a novel initialization method for monocular SLAM based on planar features. The algorithm starts by homography estimation in a sliding window. It then proceeds to a global plane optimization (GPO) to obtain camera poses and the plane normal. 3D points can be recovered using planar c...
-
Large-scale 3D scene reconstruction is an important task in autonomous driving and other robotics applications as having an accurate representation of the environment is necessary to safely interact with it. Reconstructions are used for numerous tasks ranging from localization and mapping to planning. In robotics, volumetric depth fusion is the method of choice for indoor applications since the em...
-
We showcase a topological mapping framework for a challenging indoor warehouse setting. At the most abstract level, the warehouse is represented as a Topological Graph where the nodes of the graph represent a particular warehouse topological construct (e.g. rackspace, corridor) and the edges denote the existence of a path between two neighbouring nodes or topologies. At the intermediate level, the...
-
We propose a novel RGB-D camera tracking system that robustly reconstructs hand-held RGB-D camera sequences. The robustness of our system is achieved by two independent features of our method: adaptive visual odometry (VO) and integer programming-based key-frame selection. Our VO method adaptively interpolates the camera motion results of the direct VO (DVO) and the iterative closed point (ICP) to...
-
Optimal control holds great potential to improve a variety of robotic applications. The application of optimal control on-board limited platforms has been severely hindered by the large computational requirements of current state of the art implementations. In this work, we make use of a deep neural network to directly map the robot states to control actions. The network is trained offline to imit...
-
We present a novel analysis of AO-RRT: a tree-based planner for motion planning with kinodynamic constraints, originally described by Hauser and Zhou (AO-X, 2016). AO-RRT explores the state-cost space and has been shown to efficiently obtain high-quality solutions in practice without relying on the availability of a computationally-intensive two-point boundary-value solver. Our main contribution i...
-
This article presents a path tracking control strategy for a quadcopter to follow a time varying curve. The control is based on artificial vector fields. The construction of the field is based on a well known technique in the literature. Next, control laws are developed to impose the behavior of the vector field to a second order integrator model. Finally, control laws are developed to impose the ...
-
We use reinforcement learning in simulation to obtain a driving system controlling a full-size real-world vehicle. The driving policy takes RGB images from a single camera and their semantic segmentation as input. We use mostly synthetic data, with labelled real-world data appearing only in the training of the segmentation network.Using reinforcement learning in simulation and synthetic data is mo...
-
General-purpose planning algorithms for automated driving combine mission, behavior, and local motion planning. Such planning algorithms map features of the environment and driving kinematics into complex reward functions. To achieve this, planning experts often rely on linear reward functions. The specification and tuning of these reward functions is a tedious process and requires significant exp...
-
For safe navigation around pedestrians, automated vehicles (AVs) need to plan their motion by accurately predicting pedestrians' trajectories over long time horizons. Current approaches to AV motion planning around crosswalks predict only for short time horizons (1-2 s) and are based on data from pedestrian interactions with human-driven vehicles (HDVs). In this paper, we develop a hybrid systems ...
-
In this paper we present The Oxford Radar RobotCar Dataset, a new dataset for researching scene understanding using Millimetre-Wave FMCW scanning radar data. The target application is autonomous vehicles where this modality is robust to environmental conditions such as fog, rain, snow, or lens flare, which typically challenge other sensor modalities such as vision and LIDAR.(/P)(P)The data were ga...
-
End-to-end learning from sensory data has shown promising results in autonomous driving. While employing many sensors enhances world perception and should lead to more robust and reliable behavior of autonomous vehicles, it is challenging to train and deploy such network and at least two problems are encountered in the considered setting. The first one is the increase of computational complexity w...
-
The main contribution of this paper is a novel Extended Kalman Filter (EKF) based localisation scheme that fuses two complementary approaches to outdoor vision based localisation. This EKF is aided by a front end consisting of two Convolutional Neural Networks (CNNs) that provide the necessary perceptual information from camera images. The first approach involves a CNN based extraction of informat...
-
This paper proposes a hybrid localization method that fuses Monte Carlo localization (MCL) and convolutional neural network (CNN)-based end-to-end (E2E) localization. MCL is based on particle filter and requires proposal distributions to sample the particles. The proposal distribution is generally predicted using a motion model. However, because the motion model cannot handle unanticipated errors,...
-
We estimate the global pose of a multirotor UAV by visually localizing images captured during a flight with Google Earth images pre-rendered from known poses. We metrically localize real images with georeferenced rendered images using a dense mutual information technique to allow accurate global pose estimation in outdoor GPS-denied environments. We show the ability to consistently localize throug...
-
We propose Adaptive Curriculum Generation from Demonstrations (ACGD) for reinforcement learning in the presence of sparse rewards. Rather than designing shaped reward functions, ACGD adaptively sets the appropriate task difficulty for the learner by controlling where to sample from the demonstration trajectories and which set of simulation parameters to use. We show that training vision-based cont...
-
Recent research demonstrated that it is feasible to end-to-end train multi-task deep visuomotor policies for robotic manipulation using variations of learning from demonstration (LfD) and reinforcement learning (RL). In this paper, we extend the capabilities of end-to-end LfD architectures to object manipulation in clutter. We start by introducing a data augmentation procedure called Accept Synthe...
-
Assembly tasks performed with a robot often fail due to unforeseen situations, regardless of the fact that we carefully learned and optimized the assembly policy. This problem is even more present in humanoid robots acting in an unstructured environment where it is not possible to anticipate all factors that might lead to the failure of the given task. In this work, we propose a concurrent LfD fra...
-
We present an open-source framework that provides a low barrier to entry for real-time simulation, visualization, and interactive manipulation of user-specifiable soft-bodies, environments, and robots (using a human-readable front-end interface). The simulated soft-bodies can be interacted by a variety of input interface devices including commercially available haptic devices, game controllers, an...
-
Electromagnetic field gradients generated by magnetic resonance imaging (MRI) devices pave the way to power untethered magnetic robots remotely. This innovative use of MRI devices allows exerting magnetic pulling forces on untethered magnetic robots, which could be used for navigation, diagnosis, drug delivery and therapeutic procedures inside a human body. So far, MRI-powered untethered magnetic ...
-
This study deals with the balance of humanoid or multi-legged robots in a multi-contact setting where a chosen subset of contacts is undergoing desired sliding-task motions. One method to keep balance is to hold the center-of-mass (CoM) within an admissible convex area. This area is calculated based on the contact positions and forces. We introduce a methodology to compute this CoM support area (C...
-
We introduce a new, analytical method for generating whole-body motions for humanoid robots, which approximate the desired Composite Rigid Body (CRB) inertia. Our approach uses a reduced five mass model, where four of the masses are attributed to the limbs and one is used for the trunk. This compact formulation allows for finding an analytical solution that combines the kinematics with mass distri...
-
One of the main challenges of planning legged locomotion in complex environments is the combinatorial contact selection problem. Recent contributions propose to use integer variables to represent which contact surface is selected, and then to rely on modern mixed-integer (MI) optimization solvers to handle this combinatorial issue. To reduce the computational cost of MI, we exploit the sparsity pr...
-
We present a method that finds locomanipulation plans that perform simultaneous locomotion and manipulation of objects for a desired end-effector trajectory. Key to our approach is to consider an injective locomotion constraint manifold that defines the locomotion scheme of the robot and then using this constraint manifold to search for admissible manipulation trajectories. The problem is formulat...
-
Out in the field, bipedal robots need to travel on terrain that is uneven, non-rigid, and sometimes moving beneath their feet. We present a force-based double support balancing controller for such dynamic terrain scenarios for bipedal robots, and test it on the robotic bipedal platform "Tallahassee Cassie." The presented controller relies on minimal information about the robot model, requiring its...
-
Robot networks are susceptible to fail under the presence of malicious or defective robots. Resilient networks in the literature require high connectivity and large communication ranges, leading to high energy consumption in the communication network. This paper presents robot formations with guaranteed resiliency that use smaller communication ranges than previous results in the literature. The f...
-
In this paper, we optimize the interaction graph of a multi-robot system (MRS) by maximizing its probability of security while requiring the MRS to have the fewest edges possible. Edges that represent robot interactions exist according to a probability distribution and security is defined using the control theoretic notion of left invertibility. To compute an optimal solution to our problem, we fi...
-
To achieve coordination in a multi-robot system, the robots typically resort to some form of communication among each other. In most of the multi-robot coordination frameworks, high-level coordination strategies are studied but `how' the ground-level communication takes place, is assumed to be taken care of by another program. In this paper, we study the communication routing problem for large mul...
-
Programming languages, libraries, and development tools have transformed the application development processes for mobile computing and machine learning. This paper introduces CyPhyHouse—a toolchain that aims to provide similar programming, debugging, and deployment benefits for distributed mobile robotic applications. Users can develop hardware-agnostic, distributed applications using the high-le...
-
We present a novel algorithm for simultaneous task assignment and path planning on a graph (or roadmap) with stochastic edge costs. In this problem, the initially unassigned robots and tasks are located at known positions in a roadmap. We want to assign a unique task to each robot and compute a path for the robot to go to its assigned task location. Given the mean and variance of travel cost of ea...
-
Optimal Topology Selection for Stable Coordination of Asymmetrically Interacting Multi-Robot Systems
In this paper, we address the problem of optimal topology selection for stable coordination of multi-robot systems with asymmetric interactions. This problem arises naturally for multi-robot systems that interact based on sensing, e.g., with limited field of view (FOV) cameras. From our previous efforts on motion control in such settings, we have shown that not all interaction topologies yield sta...
-
We propose a human-operator guided planning approach to pushing-based manipulation in clutter. Most recent approaches to manipulation in clutter employs randomized planning. The problem, however, remains a challenging one where the planning times are still in the order of tens of seconds or minutes, and the success rates are low for difficult instances of the problem. We build on these control-bas...
-
This paper presents a method of computing free motions of a planar assembly of rigid bodies connected by loose joints. Joints are modeled using local distance constraints, which are then linearized with respect to configuration space velocities, yielding a linear programming formulation that allows analysis of systems with thousands of rigid bodies. Potential applications include analysis of colle...
-
Planning contact interactions is one of the core challenges of many robotic tasks. Optimizing contact locations while taking dynamics into account is computationally costly and, in environments that are only partially observable, executing contact-based tasks often suffers from low accuracy. We present an approach that addresses these two challenges for the problem of vision-based manipulation. Fi...
-
Logic-Geometric Programming (LGP) is a powerful motion and manipulation planning framework, which represents hierarchical structure using logic rules that describe discrete aspects of problems, e.g., touch, grasp, hit, or push, and solves the resulting smooth trajectory optimization. The expressive power of logic allows LGP for handling complex, large-scale sequential manipulation and tool-use pla...
-
Use of physics-based simulation as a planning model enables a planner to reason and generate plans that involve non-trivial interactions with the world. For example, grasping a milk container out of a cluttered refrigerator may involve moving a robot manipulator in between other objects, pushing away the ones that are moveable and avoiding interactions with certain fragile containers. A physics-ba...
-
We present a hybrid differential dynamic programming (DDP) algorithm for closed-loop execution of manipulation primitives with frictional contact switches. Planning and control of these primitives is challenging as they are hybrid, under-actuated, and stochastic. We address this by developing hybrid DDP both to plan finite horizon trajectories with a few contact switches and to create linear stabi...
-
Structured-light and stereo cameras, which are widely used to construct point clouds for robotic applications, have different limitations on estimating depth values. Structured-light cameras fail in black, transparent, and reflective objects, which influence the light path; stereo cameras fail in texture-less objects. In this work, we propose a depth fusion model that complements these two types o...
-
Although Structure-from-Motion (SfM) as a maturing technique has been widely used in many applications, state-of-the-art SfM algorithms are still not robust enough in certain situations. For example, images for inspection purposes are often taken in close distance to obtain detailed textures, which will result in less overlap between images and thus decrease the accuracy of estimated motion. In th...
-
This work proposes a probabilistic low level automotive sensor fusion approach using LiDAR, RADAR and camera data. The method is stateless and directly operates on associated data from all sensor modalities. Tracking is not used, in order to reduce the object detection latency and create existence hypotheses per frame. The probabilistic fusion uses input from 3D and 2D space. An association method...
-
In this paper, we propose an autonomous gait analysis system consisting of a mobile robot and custom-engineered instrumented insoles. The robot is equipped with an on-board RGB-D sensor, the insoles feature inertial sensors and force sensitive resistors. This system is motivated by the need for a robot companion to engage older adults in walking exercises. Support vector regression (SVR) models we...
-
Within the Industry 4.0 context, industrial robots need to show increasing autonomy. The manipulator has to be able to react to uncertainties/changes in the working environment, displaying a robust behavior. In this paper, a control framework is proposed to perform industrial interaction tasks in uncertain working scenes. The proposed methodology relies on two components: i) a 6D pose estimation a...
-
Although autonomous control of robotic manipulators has been studied for several decades, they are not commonly used in safety-critical applications due to lack of safety and performance guarantees - many of them concerning the modulation of interaction forces. This paper presents a mechanical probing strategy for estimating the environmental impedance parameters of compliant environments, indepen...
-
In many tasks such as finishing operations, achieving accurate force tracking is essential. However, uncertainties in the robot dynamics and the environment limit the force tracking accuracy. Learning a compensation model for these uncertainties to reduce the force error is an effective approach to overcome this limitation. However, this approach requires an adaptive and robust framework for motio...
-
This paper presents a novel semantic scene change detection scheme with only weak supervision. A straightforward approach for this task is to train a semantic change detection network directly from a large-scale dataset in an end-to-end manner. However, a specific dataset for this task, which is usually labor-intensive and time-consuming, becomes indispensable. To avoid this problem, we propose to...
-
We propose a novel fast and robust 3D point clouds segmentation framework via coupled feature selection, named 3DCFS, that jointly performs semantic and instance segmentation. Inspired by the human scene perception process, we design a novel coupled feature selection module, named CFSM, that adaptively selects and fuses the reciprocal semantic and instance features from two tasks in a coupled mann...
-
In this paper, we propose the problem of collaborative perception, where robots can combine their local observations with those of neighboring agents in a learnable way to improve accuracy on a perception task. Unlike existing work in robotics and multi-agent reinforcement learning, we formulate the problem as one where learned information must be shared across a set of agents in a bandwidth-sensi...
-
Generally capable Spatial AI systems must build persistent scene representations where geometric models are combined with meaningful semantic labels. The many approaches to labelling scenes can be divided into two clear groups: view-based which estimate labels from the input view-wise data and then incrementally fuse them into the scene model as it is built; and map-based which label the generated...
-
In order to understand how a robot will perform in the open world, we aim to establish a quantitative understanding of the distribution of environments that a robot will face when when it is deployed. However, even restricting attention only to the distribution of objects in a scene, these distributions over environments are nontrivial: they describe mixtures of discrete and continuous variables r...
-
In this paper we present an approach and a benchmark for visual reasoning in robotics applications, in particular small object grasping and manipulation. The approach and benchmark are focused on inferring object properties from visual and text data. It concerns small household objects with their properties, functionality, natural language descriptions as well as question-answer pairs for visual r...
-
The goal of achieving `universal grasping' where many objects can be handled with minimal control input is the focus of much research due to potential high impact applications ranging from grocery packing to recycling. However, many of the grippers developed suffer from limited sensing capabilities which can prevent handing of both heavy bulky items and also lightweight delicate objects which requ...
-
This study describes the enhancement of a vacuum actuated soft gripper's grasping capabilities using retractable finger nails and an active re-configurable palm. The finger nail mechanism is pneumatically actuated and enables the gripper to perform complex grasping and manipulation tasks with high repeatability. The retracted nails can exert normal grasping forces of up to 1.8N and enable grasping...
-
In this paper an automated data labeling (ADL) neural network is proposed to streamline dataset collecting for real-time predicting the continuous motion of hand and wrist, these gestures are only decoded from a surface electromyography (sEMG) array of eight channels. Unlike collecting both the bio-signals and hand motion signals as samples and labels in supervised learning, this algorithm only co...
-
Developing autonomous vehicles capable of navigating safely and socially around pedestrians is a major challenge in intelligent transportation. This challenge cannot be met without understanding pedestrians' behavioral response to an autonomous vehicle, and the task of building a clear and quantitative description of the pedestrian to vehicle interaction remains a key milestone in autonomous navig...
-
Social Navigation methods attempt to integrate knowledge from Human Sciences fields such as the notion of Proxemics into mobile robot navigation. They are often evaluated in simulations, or lab conditions with informed participants, and studies of the impact of the robot behavior on humans are rare. Humans communicate and interact through many vectors, among which are motion and positioning, which...
-
Verbal interaction between a human and a robot may play a key role in conveying suitable directions for a robot to achieve the goal of a user's request. However, a robot may need to correct task plans or make new decisions with human help, which would make the interaction inconvenient and also increase the interaction time. In this paper, we propose a new verbal interaction-based method that can g...
-
We consider the problem of mapless collision-avoidance navigation where humans are present using 2D laser scans. Our proposed method uses ego-safety to measure collision from the robot's perspective and social-safety to measure the impact of robot's actions on surrounding pedestrians. Specifically, the social-safety part predicts the intrusion impact of the robot's action into the interaction area...
-
Artificial microswimmers are prospective robotic agents especially in biomedical applications. A rotating magnetic field can actuate a magnetized swimmer with a helical tail and enable propulsion. Such swimmers exhibit several modes of instability. Inside conduits, for example, hydrodynamic interactions with the boundaries lead to helical paths for pusher-mode swimmers; in this mode the helical ta...
-
Morphology of a robot design is important to its ability to achieve a stated goal and therefore applying machine learning approaches that incorporate morphology in the design space can provide scope for significant advantage. Our study is set in a domain known to be reliant on morphology: flapping wing flight. We developed a parameterised morphology design space that draws features from biological...
-
This work studies the accuracy of a simple but effective analytical model for a flapping-wings UAV in longitudinal gliding flight configuration comparing it with experimental results of a real ornithopter. The aerodynamic forces are modeled following the linearized potential theory for a flat plate in gliding configuration, extended to flapping-wing episodes modeled also by the (now unsteady) line...
-
The flight characteristics of bats remarkably have been overlooked in aerial drone designs. Unlike other animals, bats leverage the manipulation of inertial dynamics to exhibit aerial flip turns when they perch. Inspired by this unique maneuver, this work develops and uses a tiny robot called Harpoon to demonstrate that the preparation for upside-down landing is possible through: 1) reorientation ...
-
All animals and robots that move in the world must navigate to a goal while clearing obstacles. Using vision to accomplish such task has several advantages in cost and payload, which explains the prevalence of biological visual guidance. However, the computational overhead has been an obvious concern when increasing number of pixels and frames that need to be analyzed in real-time for a machine vi...
-
This paper presents the design and performance of a new locomotion strategy for serpentine robots using screw propulsion. The ARCSnake robot comprises serially linked, identical modules, each incorporating an Archimedes' screw for propulsion and a universal joint (U-Joint) for orientation control. When serially chained, these modules form a versatile serpentine robot platform which enables the rob...
-
Ground Penetrating Radar (GPR) is one of the most important non-destructive evaluation (NDE) devices to detect the subsurface objects (i.e. rebars, utility pipes) and reveal the underground scene. One of the biggest challenges in GPR based inspection is the subsurface targets reconstruction. In order to address this issue, this paper presents a 3D GPR migration and dielectric prediction system to ...
-
The paper presents a working pipeline which integrates hardware and software in an automated robotic rose cutter. To the best of our knowledge, this is the first robot able to prune rose bushes in a natural environment. Unlike similar approaches like tree stem cutting, the proposed method does not require to scan the full plant, have multiple cameras around the bush, or assume that a stem does not...
-
In this paper, a slip-limiting controller for redundant line-suspended robots is presented. This kind of robot is usually equipped with v-shaped wheels, which brings uncertainty about the effective wheel radius, particularly when crossing obstacles. The proposed algorithm is able to estimate and limit wheel slippage in the presence of such uncertainty, relying only on wheel angular velocity measur...
-
In this paper, a new method is proposed to solve the inverse kinematics problem of redundant manipulators. This method demonstrates superior performance on continuous motion by combining interval search genetic algorithm based on trajectory which we propose with parametric joint angle method. In this method, population continuity strategy is utilized to improve search speed and reduce evolutionary...
-
Serial manipulator kinematics provide a mapping between joint variables in joint-space coordinates, and end-effector configurations in task-space Cartesian coordinates. Velocity mappings are represented via the manipulator Jacobian produced by direct differentiation of the forward kinematics. Acquisition of acceleration, jerk, and snap expressions, typically utilized for accurate trajectory-tracki...
-
Inverse kinematics is a fundamental challenge for articulated robots: fast and accurate algorithms are needed for translating task-related workspace constraints and goals into feasible joint configurations. In general, inverse kinematics for serial kinematic chains is a difficult nonlinear problem, for which closed form solutions cannot easily be obtained. Therefore, computationally efficient nume...
-
Today's complex robotic designs comprise in some cases a large number of degrees of freedom, enabling for multi-objective task resolution (e.g., humanoid robots or aerial manipulators). This paper tackles the local stability problem of a hierarchical closed-loop inverse kinematics algorithm for such highly redundant robots. We present a method to guarantee this system stability by performing an on...
-
In this work, a robotic assistance strategy is developed to improve the safety in an artisanal task that involves a strong interaction between a machine-tool and an operator. Wood milling is chosen as a pilot task due to its importance in carpentry and its accidentogenic aspect. A physical model of the tooling process including a human is proposed and a simulator is thereafter developed to better ...
-
Visual localization is an important task for applications such as navigation and augmented reality, but is a challenging problem when there are changes in scene appearances through day, seasons, or environments. In this paper, we present a convolutional neural network (CNN)-based approach for visual localization across normal to drastic appearance variations such as pre- and post-disaster cases. O...
-
Planning a trajectory with guaranteed safety is a core part for a risk-free flight of a multirotor. If a trajectory planner only aims to ensure safety, it may generate trajectories which overly bypass risky regions and prevent the system from achieving specific missions. This work presents a robust trajectory planning algorithm which simultaneously guarantees the safety and reachability to the tar...
-
Real-world autonomous systems often employ probabilistic predictive models of human behavior during planning to reason about their future motion. Since accurately modeling human behavior a priori is challenging, such models are often parameterized, enabling the robot to adapt predictions based on observations by maintaining a distribution over the model parameters. Although this enables data and p...
-
Roboticists are grappling with how to address privacy in robot design at a time when regulatory frameworks around the world increasingly require systems to be engineered to preserve and protect privacy. This paper surveys the top robotics journals and conferences over the past four decades to identify contributions with respect to privacy in robot design. Our survey revealed that less than half of...
-
We propose a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of m...
-
This paper proposes a decentralized control strategy to reach segregation in heterogeneous robot swarms distributed in curves. The approach is based on a formation control algorithm applied to each robot and a heuristics to compute the distance between the groups, i.e. the distance from the beginning of the curve. We consider that robots can communicate through a fixed underlying topology and also...
-
This paper describes a distributed algorithm for computing the number of robots in a swarm, only requiring communication with neighboring robots. The algorithm can adjust the estimated count when the number of robots in the swarm changes, such as the addition or removal of robots. Probabilistic guarantees are given, which show the accuracy of this method, and the trade-off between accuracy, speed,...
-
We present a distributed Bayesian algorithm for robot swarms to classify a spatially distributed feature of an environment. This type of "go/no-go" decision appears in applications where a group of robots must collectively choose whether to take action, such as determining if a farm field should be treated for pests. Previous bio-inspired approaches to decentralized decision-making in robotics lac...
-
Supervisory Control Theory (SCT) provides a formal framework for controlling discrete event systems. It has recently been used to generate correct-by-construction controllers for swarm robotics systems. Current SCT frameworks are limited, as they support only (private) events that are observable within the same robot. In this paper, we propose an extended SCT framework that incorporates (public) e...
-
Robot simulators provide an easy way for evaluation of new concepts and algorithms in a simulated physical environment reducing development time and cost. Therefore it is convenient to have a tool that quickly creates a 3D landscape from an arbitrary 2D image or 2D laser range finder data. This paper presents a new tool that automatically constructs such landscapes for Gazebo simulator. The tool c...
-
This paper addresses the problem to simulate ARVA sensors using ROS and Gazebo. ARVA is a French acronym which stands for Appareil de Recherche de Victims en Avalanche and represents the forefront technology adopted in Search & Rescue operations to localize victims of avalanches buried under the snow. The aim of this paper is to describe the mathematical and theoretical background of the transceiv...
-
For robots to exhibit a high level of intelligence in the real world, they must be able to assess objects for which they have no prior knowledge. Therefore, it is crucial for robots to perceive object affordances by reasoning about physical interactions with the object. In this paper, we propose a novel method to provide robots with an ability to imagine object affordances using physical simulatio...
-
We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM)...
-
Granular media (e.g., cereal grains, plastic resin pellets, and pills) are ubiquitous in robotics-integrated industries, such as agriculture, manufacturing, and pharmaceutical development. This prevalence mandates the accurate and efficient simulation of these materials. This work presents a software and hardware framework that automatically calibrates a fast physics simulator to accurately simula...
-
We aim to develop an algorithm for robots to manipulate novel objects as tools for completing different task goals. An efficient and informative representation would facilitate the effectiveness and generalization of such algorithms. For this purpose, we present KETO, a framework of learning keypoint representations of tool-based manipulation. For each task, a set of task-specific keypoints is joi...
-
Does having visual priors (e.g. the ability to detect objects) facilitate learning to perform vision-based manipulation (e.g. picking up objects)? We study this problem under the framework of transfer learning, where the model is first trained on a passive vision task (i.e., the data distribution does not depend on the agent’s decisions), then adapted to perform an active manipulation task (i.e., ...
-
In industrial assembly tasks, the position of an object grasped by the robot has to be known with high precision in order to insert or place it. In real applications, this problem is commonly solved by jigs that are specially produced for each part. However, they significantly limit flexibility and are prohibitive when the target parts change often, so a flexible method to localize parts with high...
-
Applications of deep neural network (DNN) based object and grasp detections could be expanded significantly when the network output is processed by a high-level reasoning over relationship of objects. Recently, robotic grasp detection and object detection with reasoning have been investigated using DNNs. There have been efforts to combine these multitasks using separate networks so that robots can...
-
Many existing SLAM approaches rely on the assumption of static environments for accurate performance. However, several robot applications require them to traverse repeatedly in semi-static or dynamic environments. There has been some recent research interest in designing persistence filters to reason about persistence in such scenarios. Our goal in this work is to incorporate such persistence reas...
-
Dynamic environments are challenging for visual SLAM since the moving objects occlude the static environment features and lead to wrong camera motion estimation. In this paper, we present a novel dense RGB-D SLAM solution that simultaneously accomplishes the dynamic/static segmentation and camera ego-motion estimation as well as the static background reconstructions. Our novelty is using optical f...
-
Deep neural network controllers for autonomous driving have recently benefited from significant performance improvements, and have begun deployment in the real world. Prior to their widespread adoption, safety guarantees are needed on the controller behaviour that properly take account of the uncertainty within the model as well as sensor noise. Bayesian neural networks, which assume a prior over ...
-
A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which...
-
In this paper, we present a generic fault-tolerant control (FTC) strategy via reinforcement learning (RL). We demonstrate the effectiveness of this method on quadcopter unmanned aerial vehicles (UAVs). The fault-tolerant control policy is trained to handle actuator and sensor fault/attack. Unlike traditional FTC, this policy does not require fault detection and diagnosis (FDD) nor tailoring the co...
-
The best way to combine the results of deep learning with standard 3D reconstruction pipelines remains an open problem. While systems that pass the output of traditional multi-view stereo approaches to a network for regularisation or refinement currently seem to get the best results, it may be preferable to treat deep neural networks as separate components whose results can be probabilistically fu...
-
Carrying payloads in air is a major mission for multirotor aerial robot. However, the presence of payloads on multirotor aerial robot has a risk of degrading the performance of the flight controller. This concern becomes obvious especially when carrying objects not securely attached to the body or performing aerial manipulation. Therefore, controller with the ability to adapt itself to the effects...
-
Autonomous flight through unknown environments in the presence of obstacles is a challenging problem for micro aerial vehicles (MAVs). A majority of the current state-of-art research assumes obstacles as opaque objects that can be easily sensed by optical sensors such as cameras or LiDARs. However in indoor environments with glass walls and windows, or scenarios with smoke and dust, robots (even b...
-
As Unmanned Aerial Vehicles (UAVs) become more commonly used in industry, their performance will continue to be challenged. A performance bottleneck that is crucial to overcome is the design of electric propulsion systems for UAVs that operate in disparate flight modes (e.g., hovering and forward-moving flight). While flight mode dissimilarity presents a fundamental design challenge for fixed-geom...
-
We present a behaviour-based reinforcement learning approach, inspired by Brook's subsumption architecture, in which simple fully connected networks are trained as reactive behaviours. Our working assumption is that a pick and place robotic task can be simplified by leveraging domain knowledge of a robotics developer to decompose and train reactive behaviours; namely, approach, grasp, and retract....
-
A common approach for defining a reward function for multi-objective reinforcement learning (MORL) problems is the weighted sum of the multiple objectives. The weights are then treated as design parameters dependent on the expertise (and preference) of the person performing the learning, with the typical result that a new solution is required for any change in these settings. This paper investigat...
-
In this paper, a bio-inspired monocular vision perception method combined with a learning-based reaction local planner for obstacle avoidance of micro UAVs is presented. The system is more computationally efficient than other vision-based perception and navigation methods such as SLAM and optical flow because it does not need to calculate accurate distances. To improve the robustness of perception...
-
Interactive Reinforcement Learning (RL) enables agents to learn from two sources: rewards taken from observations of the environment, and feedback or advice from a secondary critic source, such as human teachers or sensor feedback. The addition of information from a critic during the learning process allows the agents to learn more quickly than non-interactive RL. There are many methods that allow...
-
Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. On the other hand, reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sampleinefficient and brittle. In this wo...
-
We contribute a study benchmarking the performance of multiple motion-based learning from demonstration approaches. Given the number and diversity of existing methods, it is critical that comprehensive empirical studies be performed comparing the relative strengths of these techniques. In particular, we evaluate four approaches based on properties an end user may desire for real-world tasks. To pe...
-
An approach toward intuitive and easy robot programming, consists to transfer skills from humans to machines, through demonstration. A vast literature exists on learning from multiple demonstrations. This paper, on the other hand, tackles the problem of providing all needed information to execute a certain task by resorting to one single demonstration - hence, a problem closer to programming than ...
-
We propose in this paper Periodic Interaction Primitives - a probabilistic framework that can be used to learn compact models of periodic behavior. Our approach extends existing formulations of Interaction Primitives to periodic movement regimes, i.e., walking. We show that this model is particularly well-suited for learning data-driven, customized models of human walking, which can then be used f...
-
Rotating miniature magnetic swimmers are de-vices that could navigate within the bloodstream to access remote locations of the body and perform minimally invasive procedures. The rotational movement could be used, for example, to abrade a pulmonary embolus. Some regions, such as the heart, are challenging to navigate. Cardiac and respiratory motions of the heart combined with a fast and variable b...
-
In this paper we present a framework to learn skills from human demonstrations in the form of geometric nullspaces, which can be executed using a robot. We collect data of human demonstrations, fit geometric nullspaces to them, and also infer their corresponding geometric constraint models. These geometric constraints provide a powerful mathematical model as well as an intuitive representation of ...
-
We present an integrated approach that provides compliant control of an iCub humanoid robot and adaptive reaching, grasping, navigating and co-manipulating capabilities. We use state-dependent dynamical systems (DS) to (i) coordinate and drive the robots hands (in both position and orientation) to grasp an object using an intermediate virtual object, and (ii) drive the robot's base while walking/n...
-
In this paper, we study the state-constrained consensus problem and introduce a new family of distributed algorithms based on subspace projection methods which are simple to implement and which preserve, under some suitable conditions, the consensus value of the original discrete-time agreement protocol. The proposed theory is supported by extensive numerical experiments for the constrained 2D ren...
-
In this paper, we propose a graph-based search method to optimally allocate tasks to a team of robots given a global task specification. In particular, we define these agents as discrete transition systems. In order to allocate tasks to the team of robots, we decompose finite linear temporal logic (LTL) specifications and consider agent specific cost functions. We propose to use the stochastic opt...
-
For multi-robot teams with heterogeneous capabilities, typical task allocation methods assign tasks to robots based on the suitability of the robots to perform certain tasks as well as the requirements of the task itself. However, in real-world deployments of robot teams, the suitability of a robot might be unknown prior to deployment, or might vary due to changing environmental conditions. This p...
-
In this work, we introduce Mobile Wireless Infrastructure on Demand: a framework for providing wireless connectivity to multi-robot teams via autonomously reconfiguring ad-hoc networks. In many cases, previous multi-agent systems either assumed the availability of existing communication infrastructure or were required to create a network in addition to completing their objective. Instead our syste...
-
In this paper, we formulate and solve the intermittent deployment problem, which yields strategies that couple when heterogeneous robots should sense an environmental process, with where a deployed team should sense in the environment. As a motivation, suppose that a spatiotemporal process is slowly evolving and must be monitored by a multi-robot team, e.g., unmanned aerial vehicles monitoring pas...
-
We present an algorithm for multi-robot coverage of an initially unknown spatial scalar field characterized by a density function, whereby a team of robots simultaneously estimates and optimizes its coverage of the density function over the domain. The proposed algorithm borrows powerful concepts from Bayesian Optimization with Gaussian Processes that, when combined with control laws to achieve ce...
-
In this study, we present a novel control framework for assembly tasks with a soft robot. Typically, existing hard robots require high frequency controllers and precise force/torque sensors for assembly tasks. The resulting robot system is complex, entailing large amounts of engineering and maintenance. Physical softness allows the robot to interact with the environment easily. We expect soft robo...
-
While most robotics simulation libraries are built for low-dimensional and intrinsically serial tasks, soft-body and multi-agent robotics have created a demand for simulation environments that can model many interacting bodies in parallel. Despite the increasing interest in these fields, no existing simulation library addresses the challenge of providing a unified, highly-parallelized, GPU-acceler...
-
Underactuated adaptive hands simplify grasping tasks but it is difficult to model their interactions with objects during in-hand manipulation. Learned data-driven models have been recently shown to be efficient in motion planning and control of such hands. Still, the accuracy of the models is limited even with the addition of more data. This becomes important for long horizon predictions, where er...
-
Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem by ...
-
We present an algorithm determining where to relocate objects inside a cluttered and confined space while rearranging objects to retrieve a target object. Although methods that decide what to remove have been proposed, planning for the placement of removed objects inside a workspace has not received much attention. Rather, removed objects are often placed outside the workspace, which incurs additi...
-
The ability to autonomously modify their environment dramatically increases the capability of robots to operate in unstructured environments. We develop a specialized construction algorithm and robotic system that can autonomously build motion support structures with previously unseen objects. The approach is based on our prior work on adaptive ramp building algorithms, but it eliminates the assum...
-
An accurate depth map of the environment is critical to the safe operation of autonomous robots and vehicles. Currently, either light detection and ranging (LIDAR) or stereo matching algorithms are used to acquire such depth information. However, a high-resolution LIDAR is expensive and produces sparse depth map at large range; stereo matching algorithms are able to generate denser depth maps but ...
-
State of the art visual-inertial odometry approaches suffer from the requirement of high gradients and sufficient visual texture. Even direct photometric approaches select a subset of the image with high-gradient areas and ignore smooth gradients or generally low-textured areas. In this work, we show that taking all image information (i.e. every single pixel) enables visual-inertial odometry even ...
-
Interaction with the environment is arguably one of the necessary actions for many robot applications such as haptic devices, manipulation, parts assembly, cooperation with humans, and the use of tools. Taxonomy of interaction behaviours is classified into three categories: cooperation, collaboration, and competition. In theory, interaction dynamics may be modelled by D'Alembert's principle or non...
-
Sensing antenna devices, that mimic insect antennae or mammal whiskers, is an active field of research that still needs new developments in order to become efficient and reliable components of robotic systems. This work reports a new result in the area of signal processing of these devices that allows to detect the instant of the impact of a flexible antenna with an object faster than other report...
-
Analytic grasp planning algorithms typically approximate compliant contacts with soft point contact models to compute grasp quality, but these models are overly conservative and do not capture the full range of grasps available. While area contact models can reduce the number of false negatives predicted by point contact models, they have been restricted to a 3D analysis of the wrench applied at t...
-
The ability to simulate and predict the outcome of contacts is paramount to the successful execution of many robotic tasks. Simulators are powerful tools for the design of robots and their behaviors, yet the discrepancy between their predictions and observed data limit their usability. In this paper, we propose a self-supervised approach to learning residual models for rigid-body simulators that e...
-
Car models have been extensively studied at varying levels of abstraction, and planning and executing motions under ideal conditions is well researched and understood. For more aggressive maneuvers, for example when drifting or skidding, empirical and/or discontinuous friction models have been used to explain and approximate real world contact behavior. Separately, contact dynamics have been exten...
-
We present a novel region growing algorithm for plane extraction of organized point clouds using the spherical convex hull. Instead of explicit plane parameterization, our approach interprets potential underlying planes as a series of geometric constraints on the sphere that are refined during region growing. Unlike existing schemes relying on downsampling for sequential execution in real time, ou...
-
Recent work on semantic simultaneous localization and mapping (SLAM) have shown the utility of natural objects as landmarks for improving localization accuracy and robustness. In this paper we present a monocular semantic SLAM system that uses object identity and inter-object geometry for view-invariant loop detection and drift correction. Our system's ability to recognize an area of the scene eve...
-
We present an active acoustic sensor that turns soft pneumatic actuators into contact sensors. The whole surface of the actuator becomes a sensor, rendering the question of where best to place a contact sensor unnecessary. At the same time, the compliance of the soft actuator remains unaffected. A small, embedded speaker emits a frequency sweep which travels through the actuator before it is recor...
-
THIS paper presents a bidirectional 3D-printed soft pneumatic actuator that is capable of inward and outward bending. A direct 3D-printing approach is adopted to fabricate the actuator, which reduces fabrication complexity and allows for easy customization of actuator dimensions for various applications. To illustrate the applicability of the bidirectional actuators, four of these actuators were i...
-
Real-time human activity recognition plays an essential role in real-world human-centered robotics applications, such as assisted living and human-robot collaboration. Although previous methods based on skeletal data to encode human poses showed promising results on real-time activity recognition, they lacked the capability to consider the context provided by objects within the scene and in use by...
-
Task-oriented dialogue system has a vital role in Human-Robot Interaction (HRI). However, it has been developed based on conventional pipeline approach which has several drawbacks; expensive, time-consuming, and so on. Based on this approach, developers manually define a robot's behaviour such as gestures and facial expressions on the corresponding dialogue states. Recently, end-to-end learning of...
-
Socially assistive robots have the potential to improve group dynamics when interacting with groups of people in social settings. This work contributes to the understanding of those dynamics through a user study of trust dynamics in the novel context of a robot mediated support group. For this study, a novel framework for robot mediation of a support group was developed and validated. To evaluate ...
-
Despite substantial evidence for the crucial role played by an active backbone or spine in animal locomotion, its adoption in legged robots remains limited because the added mechanical complexity and resulting dynamical challenges pose daunting obstacles to characterizing even a partial range of potential performance benefits. This paper takes a next step toward such a characterization by explorin...
-
A method for designing the motion of a snake robot negotiating complicated pipe structures having a constant diameter is presented. For such robots moving inside pipes, there are various "obstacles" such as junctions, bends, shears, and blockages. To surmount these obstacles, we propose a method that enables the robot to adapt to multiple pipe structures of a constant diameter. We designed the tar...
-
In this work, we present the concept, design, and initial testing of a single actuator peristaltic motion robot for subsurface geological exploration and device emplacement. We are researching unconventional methods, including robotics, for the production of energy from oil reservoirs that do not liberate carbon to the atmosphere. For such application, we are developing autonomous robots for data ...
-
In order to fulfill conflicting requirements in the development of industrial robots, such as increased accuracy of a weightreduced manipulator with lower mechanical stiffness, the robot's dynamical behavior must be evaluated early in the development process. This leads to the need of accurate multibody models of the manipulator under development.This paper deals with multibody models that include...
-
In the context of model-based reinforcement learning and control, a large number of methods for learning system dynamics have been proposed in recent years. The purpose of these learned models is to synthesize new control policies. An important open question is how robust current dynamics-learning methods are to shifts in the data distribution due to changes in the control policy. We present a rea...
-
We present the MagNet, a neural network-based multi-agent interaction model to discover the governing dynamics and predict evolution of a complex multi-agent system from observations. We formulate a multi-agent system as a coupled non-linear network with a generic ordinary differential equation (ODE) based state evolution, and develop a neural network-based realization of its time-discretized mode...
-
With the rapid rise of 3D-printing as a competitive mass manufacturing method, manual "decaking" – i.e. removing the residual powder that sticks to a 3D-printed part – has become a significant bottleneck. Here, we introduce, for the first time to our knowledge, a robotic system for automated decaking of 3D-printed parts. Combining Deep Learning for 3D perception, smart mechanical design, motion pl...
-
Traditional solar trackers often adopt motors to automatically adjust the attitude of the solar panels towards the sun for maximum power efficiency. In this paper, a novel design of solar tracker for the ocean environment is introduced. Utilizing the fluctuations due to the waves, electromagnetic brakes are utilized instead of motors to adjust the attitude of the solar panels. Compared with the tr...
-
This paper introduces a new hydraulic manipulator with hydraulic-cable driven actuation (HCA) modules for disaster response mobile-manipulation. The hydraulic actuation system has the potential to apply disaster-response application, because it has a higher power-to-weight ratio and robustness to external impacts than electric motor actuation. However, using a conventional hydraulic manipulators i...
-
Plucked strings are an exciting sound generation model for technical and timbral exploration. Mechatronic chordophones take advantage of this model and have been the focus of extensive research and exploration in musical robotics, often used as stand-alone instruments or as part of sound art installations. However, no existing chordophone designs have utilised the expressive potential of plucked s...
-
A mobility mechanism for robots to be used in tight spaces shared with people requires it to have a small footprint, to move omnidirectionally, as well as to be highly maneuverable. However, currently there exist few such mobility mechanisms that satisfy all these conditions well. Here we introduce Omnidirectional Balancing Unicycle Robot (OmBURo), a novel unicycle robot with active omnidirectiona...
-
We consider recognition and reconfiguration of lattice-based cellular structures by very simple robots with only basic functionality. The underlying motivation is the construction and modification of space facilities of enormous dimensions, where the combination of new materials with extremely simple robots promises structures of previously unthinkable size and flexibility; this is also closely re...
-
The Variable Topology Truss (VTT) is a new class of self-reconfigurable robot that can reconfigure its truss shape and topology depending on the task or environment requirements. Motion planning and avoiding self-collision are difficult as these systems usually have dozens of degrees-of-freedom with complex intersecting parallel actuation. There are two different types of shape changing actions fo...
-
In this work we introduce ModQuad-DoF, a modular flying robotic structure with enhanced capabilities for yaw actuation. We propose a new module design that allows a one degree of freedom relative motion between the flying robot and the cage, with a docking mechanism allowing rigid connections between cages. A novel method of yaw actuation that increases the structure control authority is also pres...
-
This paper presents a novel approach to fault tolerant reconfiguration of modular self-folding robots. Among various types of faults that probably occur in the modular system, we focus on the tolerance of complete actuation failure of active modules that might cause imprecise robotic motion and even reconfiguration failure. Our approach is to utilize the reconfigurability of modular self-folding r...
-
This paper presents a parallel permutation algorithm that achieves linear full-resolution reconfiguration of sliding-only cubic modular robots. We assume the use of a cubic module that can only slide across other modules' surfaces. The idea of a cubic modular robot with sliding-only motion primitive is a new concept that has advantages in simplifying the mechanisms of module hardware and space sav...
-
Fiducial markers like AprilTags play an important role in robotics, e.g., for the calibration of cameras or the localization of robots. One of the most important properties of an algorithm for detecting such tags is its localization accuracy.In this paper, we present the results of an extensive comparison of four freely available libraries capable of detecting AprilTags, namely AprilTag 3, AprilTa...
-
A variety of optimization problems takes the form of a minimum norm optimization. In this paper, we study the change of optimal values between two incrementally constructed least norm optimization problems, with new measurements included in the second one. We prove an exact equation to calculate the change of optimal values in the linear least norm optimization problem. With the result in this pap...
-
An important research issue in mobile robotics is performance assessment of robot SLAM algorithms in terms of their localization accuracy. Typically, SLAM algorithms are evaluated with the help of benchmark datasets or expensive equipment such as motion capture. Benchmark datasets however, are environment-specific, and use of motion capture constrains spatial coverage and affordability. In this pa...
-
The current state-of-the-art for testing and evaluation of autonomous surface vehicle (ASV) decision-making is currently limited to one-versus-one vessel interactions by determining compliance with the International Regulations for Prevention of Collisions at Sea, referred to as COLREGS. Strict measurement of COLREGS compliance, however, loses value in multi-vessel encounters, as there can be conf...
-
A defining characteristic of intelligent systems is the ability to make action decisions based on the anticipated outcomes. Video prediction systems have been demonstrated as a solution for predicting how the future will unfold visually, and thus, many models have been proposed that are capable of predicting future frames based on a history of observed frames (and sometimes robot actions). However...
-
We propose a closed-loop, multi-instance control algorithm for visually guided reaching based on novel learning principles. A control Lyapunov function methodology is used to design a reaching action for a complex multi-instance task in the case where full state information (poses of all potential reaching points) is available. The proposed algorithm uses monocular vision and manipulator joint ang...
-
Object finding in clutter is a skill that requires perception of the environment and in many cases physical interaction. In robotics, interactive perception defines a set of algorithms that leverage actions to improve the perception of the environment, and vice versa use perception to guide the next action. Scene interactions are difficult to model, therefore, most of the current systems use prede...
-
Instance grasping is a challenging robotic grasping task when a robot aims to grasp a specified target object in cluttered scenes. In this paper, we propose a novel end-to-end instance grasping method using only monocular workspace and query images, where the workspace image includes several objects and the query image only contains the target object. To effectively extract discriminative features...
-
Recent approaches for 3D object detection have made tremendous progresses due to the development of deep learning. However, previous researches are mostly based on individual frames, leading to limited exploitation of information between frames. In this paper, we attempt to leverage the temporal information in streaming data and explore 3D streaming based object detection as well as tracking. Towa...
-
Safe autonomous driving requires reliable 3D object detection-determining the 6 DoF pose and dimensions of objects of interest. Using stereo cameras to solve this task is a cost-effective alternative to the widely used LiDAR sensor. The current state-of-the-art for stereo 3D object detection takes the existing PSMNet stereo matching network, with no modifications, and converts the estimated dispar...
-
The Relative Confusion Matrix, a Tool to Assess Classifiablility in Large Scale Picking Applications
For bin picking robots in real logistics installations, the certainty of picking the correct product out of a mixed-product bin is essential. This paper proposes an approach for the robot to efficiently decide whether it can robustly distinguish the product to pick from the others in the bin. If not, the pick has to be routed not to the robot workstation but to a manual picking station. For this, ...
-
Accurately estimating the 6-DoF object pose from a single RGB image is a challenging task in computer vision. Though pose regression approaches have achieved great progress, the performance is still limited. In this work, we propose Pose-guided Auto-Encoder (PAE), which can distill better pose-related features from the image by utilizing a suitable pose representation, 3D Location Field (3DLF), to...
-
This paper tackles the problem of data abstraction in the context of 3D point sets. Our method classifies points into different geometric primitives, such as planes and cones, leading to a compact representation of the data. Being based on a semi-global Hough voting scheme, the method does not need initialization and is robust, accurate, and efficient. We use a local, low-dimensional parameterizat...
-
Real-time semantic segmentation is desirable in many robotic applications with limited computation resources. One challenge of semantic segmentation is to deal with the object scale variations and leverage the context. How to perform multi-scale context aggregation within limited computation budget is important. In this paper, firstly, we introduce a novel and efficient module called Cascaded Fact...
-
To enable intelligent automated driving systems, a promising strategy is to understand how human drives and interacts with road users in complicated driving situations. In this paper, we propose a 3D-aware egocentric spatial-temporal interaction framework for automated driving applications. Graph convolution networks (GCN) is devised for interaction modeling. We introduce three novel concepts into...
-
Motion retargeting between heterogeneous polymorphs with different sizes and kinematic configurations requires a comprehensive knowledge of (inverse) kinematics. Moreover, it is non-trivial to provide a kinematic independent general solution. In this study, we developed a cyclic three-phase optimization method based on deep reinforcement learning for human-robot motion retargeting. The motion reta...
-
Surgical scene understanding and multi-tasking learning are crucial for image-guided robotic surgery. Training a real-time robotic system for the detection and segmentation of high-resolution images provides a challenging problem with the limited computational resource. The perception drawn can be applied in effective real-time feedback, surgical skill assessment, and human-robot collaborative sur...
-
Automatic surgical gesture recognition is fundamental for improving intelligence in robot-assisted surgery, such as conducting complicated tasks of surgery surveillance and skill evaluation. However, current methods treat each frame individually and produce the outcomes without effective consideration on future information. In this paper, we propose a framework based on reinforcement learning and ...
-
Deep Convolutional Neural Networks (DCNNs) are used extensively in medical image segmentation and hence 3D navigation for robot-assisted Minimally Invasive Surgeries (MISs). However, current DCNNs usually use down sampling layers for increasing the receptive field and gaining abstract semantic information. These down sampling layers decrease the spatial dimension of feature maps, which can be detr...
-
There is a growing interest on formal methods-based robotic planning for temporal logic objectives. In this work, we extend the scope of existing synthesis methods to hyper-temporal logics. We are motivated by the fact that important planning objectives, such as optimality, robustness, and privacy, (maybe implicitly) involve the interrelation between multiple paths. Such objectives are thus hyperp...
-
Whether a robot can perform some specific task depends on several aspects, including the robot’s sensors and the plans it possesses. We are interested in search algorithms that treat plans and sensor designs jointly, yielding solutions—i.e., plan and sensor characterization pairs—if and only if they exist. Such algorithms can help roboticists explore the space of sensors to aid in making design tr...
-
The fundamental path planning problem for a mobile robot involves generating a trajectory for point-to-point navigation while avoiding obstacles. Heuristic-based search algorithms like A* have been shown to be efficient in solving such planning problems. Recently, there has been an increased interest in specifying complex path planning problem using temporal logic. In the state-of-the-art algorith...
-
This paper addresses the generation of collision-free trajectories for the autonomous execution of assistive tasks in Robotic Minimally Invasive Surgery (R-MIS). The proposed approach takes into account geometric constraints related to the desired task, like for example the direction to approach the final target and the presence of moving obstacles. The developed motion planner is structured as a ...
-
In this paper, we propose a Deep Imitative Q-learning (DIQL) method to synthesize control policies for mobile robots that need to satisfy Linear Temporal Logic (LTL) specifications using noisy semantic observations of their surroundings. The robot sensing error is modeled using probabilistic labels defined over the states of a Labeled Transition System (LTS) and the robot mobility is modeled using...
-
In this paper, we address the problem of finding cost-efficient three-dimensional paths that satisfy the maximum allowed curvature and the pitch angle of the vehicle. For any given initial and final configurations, the problem is decoupled into finding the horizontal and vertical parts of the path separately. Although the individual paths are modeled as two-dimensional Dubins curves using closed-f...
-
Unmanned aerial vehicles (UAVs) with mounted cameras have the advantage of capturing aerial (bird-view) images. The availability of aerial visual data and the recent advances in object detection algorithms led the computer vision community to focus on object detection tasks on aerial images. As a result of this, several aerial datasets have been introduced, including visual data with object annota...
-
Aircraft that can launch ballistically and convert to autonomous, free-flying drones have applications in many areas such as emergency response, defense, and space exploration, where they can gather critical situational data using onboard sensors. This paper presents a ballistically-launched, autonomously-stabilizing multirotor prototype (SQUID - Streamlined Quick Unfolding Investigation Drone) wi...
-
Automatic surveillance and monitoring using Unmanned Aerial Systems (UAS) require the development of perception systems that robustly work under different illumination conditions. Event cameras are neuromorphic sensors that capture the illumination changes in the scene with very low latency and high dynamic range. Although recent advances in eventbased vision have explored the use of event cameras...
-
Pose estimation is of paramount importance for flight control as well as localization and navigation of Unmanned Aerial Vehicles (UAVs) to enable autonomous operations. In environments without GPS, such estimation can only be determined using onboard sensors; optical flow using a monocular camera is a popular approach. Monocopters are a class of nature inspired UAVs known as free rotors where thei...
-
In this paper, the high angular rate estimation for simultaneous localisation and mapping (SLAM) of a Flying Li-DAR (Flydar) is presented. The proposed EKF-based algorithm exploits the sinusoidal magnetometer measurement generated by the continuously rotating airframe for estimation of the robot hovering angular velocity. Significantly, the proposed method does not rely on additional sensors other...
-
Safe and precise reference tracking is a crucial characteristic of Micro Aerial Vehicles (MAVs) that have to operate under the influence of external disturbances in cluttered environments. In this paper, we present a Nonlinear Model Predictive Control (NMPC) that exploits the fully physics based non-linear dynamics of the system. We furthermore show how the moment and thrust control inputs can be ...
-
We present a temporal Bayesian filter for semantic segmentation of a video sequence. Each pixel is a random variable following a discrete probabilistic distribution function representing possible semantic classes. Bayesian filtering consists in two main steps: 1) a prediction model and 2) an observation model (likelihood). We propose to use a datadriven prediction function derived from a dense opt...
-
Algorithms for motion planning in unknown environments are generally limited in their ability to reason about the structure of the unobserved environment. As such, current methods generally navigate unknown environments by relying on heuristic methods to choose intermediate objectives along frontiers. We present a unified method that combines map prediction and motion planning for safe, time-effic...
-
In certain challenging environments, such as inside buildings on fire, the main sensors (e.g. cameras, LiDARs and GPS systems) used for multicopter localization can become unavailable. Direct integration of the inertial navigation sensors (the accelerometer and rate gyroscope), is however unaffected by external disturbances, but the rapid error accumulation quickly makes a naive application of suc...
-
Exploration of unknown environments is embedded and essential in many robotics applications. Traditional algorithms, that decide where to explore by computing the expected information gain of an incomplete map from future sensor measurements, are limited to very powerful computational platforms. In this paper, we describe a novel approach for computing this expected information gain efficiently, a...
-
This paper presents a path planning methodology which enables Autonomous Underwater Vehicles (AUVs) to navigate in shallow complex environments such as coral reefs. The approach leverages prior information from an aerial photographic survey, and derived bathymetric information of the corresponding area. From these prior maps, a set of features is obtained which define an expected arrangement of ob...
-
This paper investigates a novel calibration process of devices with different modalities, which is a critical step of computer vision applications. We propose a fully automatic extrinsic calibration of a LiDAR-camera system. Our approach applies sphere as their surfaces and contours can be accurately detected on point clouds and camera images, respectively. Experiments on synthetic and real data e...
-
In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent the scene in a bird's eye view, where each grid cell contains the occupancy prob-ability and the two dimensional velocity. As input data, our approach relies on measurement grid maps, which contain oc...
-
High-end laser range-finders provide accurate 3D data over long ranges. But their scans are inhomogeneous, i.e., the environment is non-uniformly sampled, as there is denser data in the near range than in the far range. Furthermore, the generation of a scan is time-consuming. Thus, it is desirable to cover an area by as few scans as possible, i.e., scanning is more time-efficient if the overlap be...
-
Accurate uncertainty estimation associated with the pose transformation between two 3D point clouds is critical for autonomous navigation, grasping, and data fusion. Iterative closest point (ICP) is widely used to estimate the transformation between point cloud pairs by iteratively performing data association and motion estimation. Despite its success and popularity, ICP is effectively a determini...
-
In this paper, we develop an online active mapping system to enable a quadruped robot to autonomously survey large physical structures. We describe the perception, planning and control modules needed to scan and reconstruct an object of interest, without requiring a prior model. The system builds a voxel representation of the object, and iteratively determines the Next-Best-View (NBV) to extend th...
-
Obtaining accurate and sufficient feature matches is crucial for robust large-scale Structure-from-Motion. For unordered image collections, a traditional feature matching method with geometric verification requires a huge cost to find sufficient feature matches. Although several methods have been proposed to speed up this stage, none of them makes full use of existing matches. In this paper, we pr...
-
In this paper, we address a 3D reconstruction problem using depth prediction from a single RGB image. With the recent advances in deep learning, depth prediction shows high performance. However, due to the discrepancy between training environment and test environment, 3D reconstruction can be vulnerable to the uncertainty of depth prediction. To consider the uncertainty of depth prediction for rob...
-
Feature descriptor matching is a critical step is many computer vision applications such as image stitching, image retrieval and visual localization. However, it is often affected by many practical factors which will degrade its performance. Among these factors, illumination variations are the most influential one, and especially no previous descriptor learning works focus on dealing with this pro...
-
This work presents dense stereo reconstruction using high-resolution images for infrastructure inspections. The state-of-the-art stereo reconstruction methods, both learning and non-learning ones, consume too much computational resource on high-resolution data. Recent learning-based methods achieve top ranks on most benchmarks. However, they suffer from the generalization issue due to lack of task...
-
A new closed-form solver is proposed minimizing the algebraic error optimally, in the least squares sense, to estimate the relative planar motion of two calibrated cameras. The main objective is to solve the over-determined case, i.e., when a larger-than-minimal sample of point correspondences is given - thus, estimating the motion from at least three correspondences. The algorithm requires the ca...
-
Two solvers are proposed for estimating the extrinsic camera parameters from a single affine correspondence assuming general planar motion. In this case, the camera movement is constrained to a plane and the image plane is orthogonal to the ground. The algorithms do not assume other constraints, e.g. the non-holonomic one, to hold. A new minimal solver is proposed for the semi-calibrated case, i.e...
-
Moving object detection for visual odometry in a dynamic environment based on occlusion accumulation
Detection of moving objects is an essential capability in dealing with dynamic environments. Most moving object detection algorithms have been designed for color images without depth. For robotic navigation where real-time RGBD data is often readily available, utilization of the depth information would be beneficial for obstacle recognition. Here, we propose a simple moving object detection algori...
-
Consider the case of multiple visual sensors perceiving the same scene from different viewpoints. In order to achieve consistent visual perception, the problem of data association, in this case establishing correspondences between observed features, must be first solved. In this work, we consider multiway matching which is a specific instance of multi-sensory data association. Multiway matching re...
-
Interactive simulators are used in several important applications which include the training simulators for teleoperated robotic laparoscopic surgery. While stateof-art simulators are capable of rendering realistic visuals and accurate dynamics, grasping is often implemented using kinematic simplification techniques that prevent truly multimanual manipulation, which is often an important requireme...
-
A methodology for implementing arbitrary foot shapes in the passive walking dynamics of biped robots is developed. The dynamic model of a walking robot is defined in a way that allows shape-dependent foot kinetics to contribute to the robot's dynamics, for all convex foot shapes regardless of the exact foot geometry: for the developed method, only the set of points describing the foot profile curv...
-
Over the past decade we have been able to vastly improve the control algorithms of our biped walking robot LOLA. Further enhancements, however, are limited by vibration problems caused by the dynamics of LOLA's mechanical structure. In this work, we present small examples how structural dynamics limit our control design for walking control as well as low level position control of the joints. We al...
-
This paper employs velocity decomposition of underactuated mechanical systems to determine the degree of dynamic coupling in the gaits of a two-link biped model. The degree of coupling between controlled and uncontrolled directions quantifies the control authority the system has over its unactuated degree of freedom. This paper shows that the amount of coupling is directly correlated to gait robus...
-
We present an extension of our previously proposed IS-MPC method for humanoid gait generation aimed at obtaining robust performance in the presence of disturbances. The considered disturbance signals vary in a range of known amplitude around a mid-range value that can change at each sampling time, but whose current value is assumed to be available. The method consists in modifying the stability co...
-
This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the h...
-
State-of-the-art approaches to legged locomotion are widely dependent on the use of models like the linear inverted pendulum (LIP) and the spring-loaded inverted pendulum (SLIP), popular because their simplicity enables a wide array of tools for planning, control, and analysis. However, they inevitably limit the ability to execute complex tasks or agile maneuvers. In this work, we aim to automatic...
-
Using multiple robots for exploring and mapping environments can provide improved robustness and performance, but it can be difficult to implement. In particular, limited communication bandwidth is a considerable constraint when a robot needs to determine if it has visited a location that was previously explored by another robot, as it requires for robots to share descriptions of places they have ...
-
We consider the kinodynamic multi-robot planning problem in cluttered 3-D workspaces. Reachability analysis on position invariant geometric trees is leveraged to find kino- dynamically feasible trajectories for the multi-robot team from potentially non-stationary initial states. The key contribution of our approach is that a collision-free geometric solution guarantees a kinodynamically feasible, ...
-
The collaboration of unmanned aerial vehicles (UAVs) has become a popular research topic for its practicability in multiple scenarios. The collaboration of multiple UAVs, which is also known as aerial swarm is a highly complex system, which still lacks a state-of-art decentralized relative state estimation method. In this paper, we present a novel fully decentralized visual-inertial-UWB fusion fra...
-
We present an algorithm for the concurrent assignment and planning of collision-free trajectories (DC-CAPT) for robots whose kinematics can be modeled as Dubins cars, i.e., robots constrained in terms of their initial orientation and their minimum turning radius. Coupling the assignment and trajectory planning subproblems allows for a computationally tractable solution. This solution is guaranteed...
-
It is common knowledge that tractor-trailer vehicles are affected by jackknifing, a phenomenon that consists in the divergence of the trailer hitch angle and ultimately causes the vehicle to fold up. For the case of backwards motion, in which jackknifing can also occur at low speeds, we present a control method that drives the vehicle along a reference Cartesian trajectory while avoiding the diver...
-
The design of reliable path-following controllers is a key ingredient for successful deployment of self-driving vehicles. This controller-design problem is especially challenging for a general 2-trailer with a car-like tractor due to the vehicle's structurally unstable joint-angle kinematics in backward motion and the car-like tractor's curvature limitations which can cause the vehicle segments to...
-
This paper proposes an offline and runtime combined framework for the autonomous motion of snake robots. With the dynamic feedback of its state during runtime, the robot utilizes the linear regression to update its control parameters for better performance and thus adaptively reacts to the environment. To reduce interference from infeasible samples and improve efficiency, the data set for runtime ...
-
The paper deals with non-standard motion tasks specified for a two-wheeled nonholonomic robot. It is assumed that wheels cannot fully rotate which reduces a set of feasible movements significantly. In spite of these constraints, it is expected that position of the robot can be changed without violating nonholonomic constraints. Such a possibility comes from the small time local controllability (ST...
-
The complexity associated with the control of highly-articulated legged robots scales quickly as the number of joints increases. Traditional approaches to the control of these robots are often impractical for many real-time applications. This work thus presents a novel sampling-based planning approach for highly-articulated robots that utilizes a probabilistic graphical model (PGM) to infer in rea...
-
This work addresses the problem of changing grasp configurations on objects with an unknown shape through in-hand manipulation. Our approach leverages shape priors, learned as deep generative models, to infer novel object shapes from partial visual sensing. The Dexterous Manipulation Graph method is extended to build incrementally and account for object shape uncertainty when planning a sequence o...
-
Robotic in-hand manipulation has been a longstanding challenge due to the complexity of modelling hand and object in contact and of coordinating finger motion for complex manipulation sequences. To address these challenges, the majority of prior work has either focused on model-based, low-level controllers or on model-free deep reinforcement learning that each have their own limitations. We propos...
-
This paper develops closed-loop tactile controllers for dexterous robotic manipulation with a dual-palm robotic system. Tactile dexterity is an approach to dexterous manipulation that plans for robot/object interactions that render interpretable tactile information for control. We divide the role of tactile control into two goals: 1) control the contact state between the end-effector and the objec...
-
This paper describes the development of a novel non-anthropomorphic robot hand with the ability to manipulate objects by means of articulated, actively driven rollers located at the fingertips. An analysis is conducted and systems of equations for two-finger and three-finger manipulation of a sphere are formulated to demonstrate full six degree of freedom nonholonomic spatial motion capability. A ...
-
Flexible medical instruments, such as Continuum Dexterous Manipulators (CDM), constitute an important class of tools for minimally invasive surgery. Accurate CDM shape reconstruction during surgery is of great importance, yet a challenging task. Fiber Bragg grating (FBG) sensors have demonstrated great potential in shape sensing and consequently tip position estimation of CDMs. However, due to the...
-
We propose a model-based approach to design feedback policies for dexterous robotic manipulation. The manipulation problem is formulated as reaching the target region from an initial state for some non-smooth nonlinear system. First, we use trajectory optimization to find a feasible trajectory. Next, we characterize the local multi-contact dynamics around the trajectory as a piecewise affine syste...
-
In this paper, we introduce an approach to tracking the pose of a monocular camera in a prior surfel map. By rendering vertex and normal maps from the prior surfel map, the global planar information for the sparse tracked points in the image frame is obtained. The tracked points with and without the global planar information involve both global and local constraints of frames to the system. Our ap...
-
We present LINS, a lightweight lidar-inertial state estimator, for real-time ego-motion estimation. The proposed method enables robust and efficient navigation for ground vehicles in challenging environments, such as feature-less scenes, via fusing a 6-axis IMU and a 3D lidar in a tightly-coupled scheme. An iterated error-state Kalman filter (ESKF) is designed to correct the estimated state recurs...
-
A 3D measurement system consisting of a 3D scanner and an industrial robot (eye-in-hand) is commonly used to scan large object under test (OUT) from multiple fieldof-views (FOVs) for complete measurement. A data stitching process is required to align multiple FOVs into a single coordinate system. Marker-free stitching assisted by robot’s accurate positioning becomes increasingly attractive since i...
-
Robustness and accuracy for monocular visual odometry (VO) under challenging environments are widely concerned. In this paper, we present a monocular VO system leveraging learned repeatability and description. In a hybrid scheme, the camera pose is initially tracked on the predicted repeatability maps in a direct manner and then refined with the patch-wise 3D-2D association. The local feature para...
-
A vehicle driving along the road is surrounded by many objects, but only a small subset of them influence the driver's decisions and actions. Learning to estimate the importance of each object on the driver's real-time decision-making may help better understand human driving behavior and lead to more reliable autonomous driving systems. Solving this problem requires models that understand the inte...
-
This paper describes the design and implementation of a real-time robotics system for semi-specular/painted surface defect detection. The system can be used on moving parts, tolerate varying lighting conditions, and can accommodate small inherent vibrations of the inspected surface that is common in manufacturing operations. Topographical information of the inspected surface is first obtained by t...
-
We propose a serpentine type tendon driven underactuated end-effector design with a closing mechanism that is triggered upon contact with an object. This end-effector can grasp objects without knowing the size a priori and is able to grasp a new object while securing another one previously grasped, and so grasp multiple objects sequentially with a single DOF actuation. Design parameters based on t...
-
Hand synergy from neuroscience provides an effective tool for anthropomorphic hands to realize versatile grasping with simple planning and control. This paper aims to extend the synergy-inspired design from anthropomorphic hands to multi-fingered robot hands. The synergy-inspired hands are not necessarily humanoid in morphology but perform primary characteristics and functions similar to the human...
-
The success of hybrid suction + parallel-jaw grippers in the Amazon Robotics/Picking Challenge have demonstrated the effectiveness of multimodal grasping approaches. However, existing multimodal grippers combine grasping modes in isolation and do not incorporate the benefits of compliance found in soft robotic manipulators. In this paper, we present a gripper that integrates three modes of graspin...
-
Gecko-inspired adhesives have several desirable characteristics in robotic grasping: controllable activation and deactivation of adhesion, ability to grasp and release with minimal disturbance, and grasping without the need of form closure. Previously proposed grippers with this technology either require a complex activation mechanism or multiple activation steps. In this paper, we present an unde...
-
In this paper, we propose the ADVISEd (Active Deformation through VIsual SErvoing) method, a novel model-free deformation servoing method able to deform a soft object towards a desired shape. ADVISEd relies on an online estimation of the deformation Jacobian that relates the motion of the robot end-effector to the deformation behavior of the object. The estimation is based on a weighted least-squa...
-
We study the problem of learning manipulation skills from human demonstration video by inferring the association relationships between geometric features. Motivation for this work stems from the observation that humans perform eye-hand coordination tasks by using geometric primitives to define a task while a geometric control error drives the task through execution. We propose a graph based kernel...
-
Existing deep learning based visual servoing approaches regress the relative camera pose between a pair of images. Therefore, they require a huge amount of training data and sometimes fine-tuning for adaptation to a novel scene. Furthermore, current approaches do not consider underlying geometry of the scene and rely on direct estimation of camera pose. Thus, inaccuracies in prediction of the came...
-
We present a vision-based navigation system that uses a visual memory to navigate. Such memory corresponds to a topological map of key images created from moving a virtual camera over a model of the real scene. The advantage of our approach is that it provides a useful insight into the navigability of a visual path without relying on a traditional learning stage. During the navigation stage, the r...
-
This paper addresses the problem of efficiently achieving visual predictive control tasks. To this end, a memory of motion, containing a set of trajectories built off-line, is used for leveraging precomputation and dealing with difficult visual tasks. Standard regression techniques, such as k-nearest neighbors and Gaussian process regression, are used to query the memory and provide on-line a warm...
-
For the majority of tasks performed by traditional serial robot arms, such as bin picking or pick and place, only two or three degrees of freedom (DOF) are required for motion; however, by augmenting the number of degrees of freedom, further dexterity of robot arms for multiple tasks can be achieved. Instead of increasing the number of joints of a robot to improve flexibility and adaptation, which...
-
Soft robotic pneumatic grippers have been shown to be versatile, robust to impacts, and safe for use on delicate objects. One type, fluidic elastomer grippers, are characterized by fingers with an inextensible gripping surface backed by extensible pneumatic chambers; when inflated, this mismatch in extensibility results in the finger curling. However, one drawback of these simple fingers is that t...
-
Soft, tip-extending "vine" robots offer a unique mode of inspection and manipulation in highly constrained environments. For practicality, it is desirable that the distal end of the robot can be manipulated freely, while the body remains stationary. However, in previous vine robots, either the shape of the body was fixed after growth with no ability to manipulate the distal end, or the whole body ...
-
Soft materials and compliant actuation concepts have generated new design and control approaches in areas from robotics to wearable devices. Despite the potential of soft robotic systems, most designs currently use hard pumps, valves, and electromagnetic actuators. In this work, we take a step towards fully soft robots by developing a new compliant electromagnetic actuator architecture using galli...
-
Inflated continuum robots are promising for a variety of navigation tasks, but controlling their motion with a small number of actuators is challenging. These inflated beam robots tend to buckle under compressive loads, producing extremely tight local curvature at difficult-to-control buckle point locations. In this paper, we present an inflated beam robot that uses distributed stiffness changing ...
-
Lower limb exoskeleton (LLE) has received considerable interests in strength augmentation, rehabilitation and walking assistance scenarios. For walking assistance, the LLE is expected to have the capability of controlling the affected leg to track the unaffected leg’s motion naturally. An important issue in this scenario is that the exoskeleton system needs to deal with unpredictable disturbance f...
-
The role of various types of robot assistance in post-stroke gait rehabilitation has gained much attention in recent years. Furthermore, there is increased popularity to use more than one rehabilitation method in order to utilize the different advantages of each. Naturally, this results in the need to study how the different robot-assisted interventions affect the various underlying sensorimotor m...
-
This paper presents a visual positioning system (VPS) for real-time pose estimation of a robotic navigation aid (RNA) for assistive navigation. The core of the VPS is a new method called depth-enhanced visual-inertial odometry (DVIO) that uses an RGB-D camera and an inertial measurement unit (IMU) to estimate the RNA's pose. The DVIO method extracts the geometric feature (the floor plane) from the...
-
Online estimation of 3D ground reaction forces (GRFs) is becoming increasingly important for closed-loop control of lower-extremity robotic exoskeletons. Through in-verse dynamics and optimization models, 3D GRFs can be used to estimate net joint torques and approximate muscle forces. Although instrumented footwear to measure vertical GRFs in out-of-the-lab environments is available, accurately me...
-
Fall-related injury is a significant health problem on a global scale and is expected to grow with the aging population. Laboratory-based perturbation systems have the capability of simulating various modes of fall-inducing perturbations in a repeatable way. These systems enable fundamental research on human gait and balance and facilitate the development of devices to assist human balance. We pre...
-
Over the last years, hand exoskeletons have become a popular and efficient technical solution for assisting people that suffer from neurological and musculoskeletal diseases and enhance the capabilities of healthy individuals. These devices can vary from rigid and complex structures to soft, lightweight, wearable gloves. Despite the significant progress in the field, most existing solutions do not...
-
Implementing an intuitive control law for an upper-limb exoskeleton dedicated to force augmentation is a challenging issue in the field of human-robot collaboration. The aim of this study is to design an innovative approach to assist carrying an unknown load without using force sensors or specific handle. The method is based on user's intentions estimated through a wireless EMG armband allowing mo...
-
Assistive robots have the potential to provide tremendous support for disabled and elderly people in their daily dressing activities. Recent studies on robot-assisted dressing usually simplify the setup of the initial robot configuration by manually attaching the garments on the robot end-effector and positioning them close to the user's arm. A fundamental challenge in automating such a process fo...
-
In a physical Human-Robot Interaction for industrial scenarios is paramount to guarantee the safety of the user while keeping the robot's performance. Hierarchical task approaches are not sufficient since they tend to sacrifice the low priority tasks in order to guarantee the consistency of the main task. To handle this problem, we enhance the standard hierarchical fusion by introducing a novel in...
-
In this paper, a novel human-robot collaborative framework for mixed case palletizing is presented. The framework addresses several challenges associated with the detection and localisation of boxes and pallets through visual perception algorithms, high-level optimisation of the collaborative effort through effective role-allocation principles, and maximisation of packing density. A graphical user...
-
We design and test a 3 DoF parallel cable system capable of applying precise and accurate impulses to walking and running subjects for the University of Utah's Treadport Active Wind Tunnel (TPAWT). Using Nexus VICON motion capture and gait algorithms, perturbations can be applied at different points in the subject's gait. The use of a PID force controller allow the system to create omnidirectional...
-
The human-robot collaboration is a promising and challenging field of robotics research. One of the main collaboration tasks is the object co-manipulation where the human and robot are in a continuous physical interaction and forces exerted must be handled. This involves some issues known in robotics as physical Human-Robot Interaction (pHRI), where human safety and interaction comfort are require...
-
Attaching a robotic manipulator to a flying base allows for significant improvements in the reachability and versatility of manipulation tasks. In order to explore such systems while taking advantage of human capabilities in terms of perception and cognition, bilateral teleoperation arises as a reasonable solution. However, since most telemanipulation tasks require visual feedback in addition to t...
-
Motion mapping is an intuitive method of teleoperation with a low learning curve. Our previous study investigates the physical fatigue caused by teleoperating a robot to perform general-purpose assistive tasks and this fatigue affects the operator's performance. The results from that study indicate that physical fatigue happens more in the tasks which involve more precise manipulation and steady p...
-
Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually offer reduced degrees of control. Herein, a low-cost, depth-based teleoperation system, DexPilot,...
-
In a distributed multi-master-multi-slave teleoperation system, the human users may compete against each other for the control of the team of slave robots. To win the competition, one operator would send the largest command to the slave group. For the sake of team cohesion, the slave group should follow the command of the winning operator and ignore the commands of the other users. To enable (i) t...
-
Controlling and manning robots from a remote location is difficult because of the limitations one faces in perception and available degrees of actuation. Although humans can become skilled teleoperators, the amount of training time required to acquire such skills is typically very high. In this paper, we propose a novel solution (named Autocomplete) to aid novice teleoperators in manning robots ad...
-
Path planning of an object which is allowed to be in contact with other objects during assembly process is a significant challenge due to the variety of permitted or forbidden collisions between the distinct parts of the objects to be assembled. In order to put objects together in real-life scenarios, parts of assembled objects may be required to flex, whereas other parts may have to fit exactly. ...
-
We consider the problem of creating tighter-fitting bounding volumes (more specifically rectangular swept spheres) when constructing bounding volume hierarchies (BVHs) for complex 3D geometries given in the form of unstructured triangle meshes/soups with the aim of speeding up our IPS Path Planner for rigid bodies, where the triangles often have very different sizes. Currently, the underlying coll...
-
This paper presents the architecture of a high- density parking solution based on car-like robots specifically designed to move cars. The main difficulty is to park the vehicles close to one another which implies hard constraints on the robot motion and localization. In particular, this paper focuses on navigation in narrow lanes. We propose a Lyapunov- based control strategy that has been derived...
-
Implementation of safe and efficient autonomous driving systems requires accurate prediction of the long-term trajectories of surrounding vehicles. High uncertainty in traffic behavior makes it difficult to predict trajectories in urban environments, which have various road geometries. To over-come this problem, we propose a method called lane-based multimodal prediction network (LAMP-Net), which ...
-
With new, safer manipulator robots, the probability of serious injury due to collisions with humans remains low (5%), even at speeds as high as 2 m.s-1. Collisions would better be avoided nevertheless, because they disrupt the tasks of both the robot and the human. We propose in this paper to equip robots with exteroceptive sensors and online motion generation so that the robot is able to perceive...
-
This paper presents a novel episodic method to learn a robot's nonlinear dynamics model and an increasingly optimal control sequence for a set of tasks. The method is based on the Koopman operator approach to nonlinear dynamical systems analysis, which models the flow of observables in a function space, rather than a flow in a state space. Practically, this method estimates a nonlinear diffeomorph...
-
Magnetic helical microswimmers can be propelled by rotating magnetic field and are adept at passing through narrow space. To date, various magnetic actuation systems and control methods have been developed to drive these microswimmers. However, steering their spacial movement in a large workspace is still challenging, which could be significant for potential medical applications. In this regard, t...
-
Recently, swarm control of micro-/nanorobots has drawn much attention in the field of microrobotics. This paper reports a mobile paramagnetic nanoparticle swarm with the capability of active shape deformation that can improve its environment adaptability. We show that, by applying elliptical rotating magnetic fields, a swarm pattern called the elliptical paramagnetic nanoparticle swarm (EPNS) woul...
-
In this paper, we introduce novel miniature swimmers with multiple rigid tails based on spherical helices. The tail distribution of these prototypes enhances its swimming features as well as allowing to carry objects with it. The proposed swimmers are actuated by a rotating magnetic field, generating the robot rotation and thus producing a considerable thrust to start self-propelling. These protot...
-
Active force sensors are key instruments to get around the tradeoff between the sensitivity and the measurement range of conventional passive force sensors. Thanks to their quasi-infinite stiffness in closed loop, active sensors can be applied for force measurements on samples with a wide range of stiffness without interference with the mechanical parameters of the sensor. MEMS (Micro-Electro Mech...
-
Electromagnetic Navigation Systems (eMNS) can be used to control a variety of multiscale devices within the human body for remote surgery. Accurate modeling of the magnetic fields generated by the electromagnets of an eMNS is crucial for the precise control of these devices. Existing methods assume a linear behavior of these systems, leading to significant modeling errors within nonlinear regions ...
-
In this paper, an automated tracking system with head and tail recognition for time-lapse observation of free-moving C. elegans is presented. In microscale field, active C. elegans can move out of the view easily without an automated tracking system because of the narrow field of view and rapid speed of C. elegans. In our previous works, we constructed an automated platform with 3D freedom to trac...
-
Autonomous Underwater Vehicles (AUVs) are increasingly being used to support scientific research and monitoring studies. One such application is in benthic habitat mapping where these vehicles collect seafloor imagery that complements broadscale bathymetric data collected using sonar. Using these two data sources, the relationship between remotely-sensed acoustic data and the sampled imagery can b...
-
Multispectral recognition has attracted increasing attention from the research community due to its potential competence for many applications from day to night. However, due to the domain shift between RGB and thermal image, it has still many challenges to apply and to use RGB domain-based tasks. To reduce the domain gap, we propose multispectral domain invariant framework, which leverages the un...
-
Nowadays, robots are mechanically able to perform highly demanding tasks, where AI-based planning methods are used to schedule a sequence of actions that result in the desired effect. However, it is not always possible to know the exact outcome of an action in advance, as failure situations may occur at any time. To enhance failure tolerance, we propose to predict the effects of robot actions by a...
-
In order to solve complex, long-horizon tasks, intelligent robots need to carry out high-level, abstract planning and reasoning in conjunction with motion planning. However, abstract models are typically lossy and plans or policies computed using them can be unexecutable. These problems are exacerbated in stochastic situations where the robot needs to reason about, and plan for multiple contingenc...
-
This paper presents a novel Cable Climbing Robot CCRobot-III, which is the third version designed for bridge cable inspection tasks, aiming at surpassing previous versions in terms of climbing speed and payload capacity. Benefiting from Split-type Wire-driven design, CCRobot-III can climb along a 90-110mm diameter bridge cable in inchworm-like gait at a speed of up to 12m/min, and carrying more th...
-
This paper introduces the Omnidirectional Tractable Three Module Robot for traversing inside complex pipe networks. The robot consists of three omnidirectional modules fixed 120° apart circumferentially which can rotate about their own axis allowing holonomic motion of the robot. The holonomic motion enables the robot to overcome motion singularity when negotiating T-junctions and further allows t...
-
The advanced robotic and automation (ARA) lab has developed and successfully implemented a design inspired by many of the various cutting edge steel inspection robots to date. The combination of these robots concepts into a unified design came with its own set of challenges since the parameters for these features sometimes conflicted. An extensive amount of design and analysis work was performed b...
-
This paper presents a wheeled wall-climbing robot with a shape-adaptive magnetic adhesion mechanism for large steel structures. To travel up and down various curved ferromagnetic surfaces, we developed a 2 DOF rotational magnetic adhesion mechanism installed on each wheel that can change the orientation of the magnets to keep the magnetic force direction always normal to the contact surface. These...
-
This paper presents a method for algebraic fault detection and identification of nonlinear mechanical systems, describing rigid robots, by using an approximation with orthonormal Jacobi polynomials. An explicit expression is derived for the fault from the equation of motion, which is decoupled from disturbances and only depends on measurable signals and their time derivatives. Fault detection and ...
-
Tilted rotors in multirotor vehicles have shown to be useful for different practical reasons. For instance, increasing yaw maneuverability or enabling full position and attitude control of hexarotor vehicles. It has also been proven that a hexagon-shaped multirotor is capable of complete attitude and altitude control under failures of one of its rotors. However, when a rotor fails, the torque that...
-
We propose a method for detecting execution anomalies in robotics and autonomy software. The algorithm uses system monitoring techniques to obtain profiles of executions. It uses a clustering algorithm to create clusters of those executions, representing nominal execution. A distance metric determines whether additional execution profiles belong to the existing clusters or should be considered ano...
-
This paper studies the reliability of a real-world learning-enabled system, which conducts dynamic vehicle tracking based on a high-resolution wide-area motion imagery input. The system consists of multiple neural network components - to process the imagery inputs - and multiple symbolic (Kalman filter) components - to analyse the processed information for vehicle tracking. It is known that neural...
-
Rotation invariance has been an important topic in computer vision tasks. Ideally, robot grasp detection should be rotation-invariant. However, rotation-invariance in robotic grasp detection has been only recently studied by using rotation anchor box that are often time-consuming and unreliable for multiple objects. In this paper, we propose a rotation ensemble module (REM) for robotic grasp detec...
-
Is it possible to learn policies for robotic assembly that can generalize to new objects? We explore this idea in the context of the kit assembly task. Since classic methods rely heavily on object pose estimation, they often struggle to generalize to new objects without 3D CAD models or task-specific training data. In this work, we propose to formulate the kit assembly task as a shape matching pro...
-
Robotic manipulation of deformable 1D objects such as ropes, cables, and hoses is challenging due to the lack of high-fidelity analytic models and large configuration spaces. Furthermore, learning end-to-end manipulation policies directly from images and physical interaction requires significant time on a robot and can fail to generalize across tasks. We address these challenges using interpretabl...
-
Embedded deformation graph is a widely used technique in deformable geometry and graphical problems. Although the technique has been transmitted to stereo (or RGB-D) camera based SLAM applications, it remains challenging to compromise the computational cost as the model grows. In practice, the processing time grows rapidly in accordance with the expansion of maps. In this paper, we propose an appr...
-
We present an approach for estimating the pose of an external camera with respect to a robot using a single RGB image of the robot. The image is processed by a deep neural network to detect 2D projections of keypoints (such as joints) associated with the robot. The network is trained entirely on simulated data using domain randomization to bridge the reality gap. Perspective-n-point (PnP) is then ...
-
In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image...
-
We propose a robust baseline method for instance segmentation which are specially designed for large-scale outdoor LiDAR point clouds. Our method includes a novel dense feature encoding technique, allowing the localization and segmentation of small, far-away objects, a simple but effective solution for single-shot instance prediction and effective strategies for handling severe class imbalances. S...
-
In this paper, we consider the generation of generic object candidates by a mobile robot that is endowed with a pan-tilt monocular camera. This is an important problem because these candidates serve as basis for the robot to categorize and/or recognize the objects in its surroundings. The previously proposed methods either do not have a means of enabling the robot to look around through moving its...
-
In this work, we propose Dilated Point Convolutions (DPC). In a thorough ablation study, we show that the receptive field size is directly related to the performance of 3D point cloud processing tasks, including semantic segmentation and object classification. Point convolutions are widely used to efficiently process 3D data representations such as point clouds or graphs. However, we observe that ...
-
Obstacle detection by semantic segmentation shows a great promise for autonomous navigation in unmanned surface vehicles (USV). However, existing methods suffer from poor estimation of the water edge in presence of visual ambiguities, poor detection of small obstacles and high false-positive rate on water reflections and wakes. We propose a new deep encoder-decoder architecture, a water-obstacle s...
-
Anchor boxes act as potential object localization candidates allow single-stage detectors to achieve real-time performance, at the cost of localization accuracy when compared to state-of-the-art two-stage detectors. Therefore, correct selection of the scale and aspect ratio associated with an anchor box is crucial for detector performance. In this work, we propose a novel architecture called DANet...
-
This paper presents a self-supervised framework for learning to detect robust keypoints for odometry estimation and metric localisation in radar. By embedding a differentiable point-based motion estimator inside our architecture, we learn keypoint locations, scores and descriptors from localisation error alone. This approach avoids imposing any assumption on what makes a robust keypoint and crucia...
-
In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor...
-
Object detection and motion parameters estimation are crucial tasks for self-driving vehicle safe navigation in a complex urban environment. In this work we propose a novel real-time approach of temporal context aggregation for motion detection and motion parameters estimation based on 3D point cloud sequence. We introduce an ego-motion compensation layer to achieve real-time inference with perfor...
-
Fast, non-linear trajectories have been shown to be more accurately visually measured, and hence predicted, when sampled spatially (that is when the target position changes) rather than temporally, i.e. at a fixed-rate as in traditional frame-based cameras. Event-cameras, with their asynchronous, low latency information stream, allow for spatial sampling with very high temporal resolution, improvi...
-
Painting textures on 3D surfaces requires an understanding of the surface geometry, paint flow and paint mixing. This work formulates automated painting as a planning problem and proposes a solution based on a self-supervised learning framework that enables a robot to paint monochromatic non-uniform textures on 3D surfaces. We developed a method that iteratively decides the actions to take based o...
-
Sampling-based motion planning techniques have emerged as an efficient algorithmic paradigm for solving complex motion planning problems. These approaches use a set of probing samples to construct an implicit graph representation of the robot's state space, allowing arbitrarily accurate representations as the number of samples increases to infinity. In practice, however, solution trajectories only...
-
In this paper, we consider the path planning problem on a graph. To reduce computation load by efficiently exploring the graph, we model the heuristic function as a neural network, which is trained by a training set derived from optimal paths to estimate the optimal cost between a pair of vertices on the graph. As such heuristic function cannot be proved to be an admissible heuristic to guarantee ...
-
Motion planning is important in a wide variety of applications such as robotic manipulation. However, it is still challenging to reliably find a collision-free path within a reasonable time. To address the issue, this paper proposes a novel framework which combines a sampling-based planner and deep learning for faster motion planning, focusing on heuristics. The proposed method extends Task-Space ...
-
We would like to enable a robotic agent to quickly and intelligently find promising trajectories through structured, unknown environments. Many approaches to navigation in unknown environments are limited to considering geometric information only, which leads to myopic behavior. In this work, we show that learning a sampling distribution that incorporates both geometric information and explicit, o...
-
In this paper, we propose a deep neural network that predicts the feasibility of a mixed-integer program from visual input for robot manipulation planning. Integrating learning into task and motion planning is challenging, since it is unclear how the scene and goals can be encoded as input to the learning algorithm in a way that enables to generalize over a variety of tasks in environments with ch...
-
Exploration and collision-free navigation through an unknown environment is a fundamental task for autonomous robots. In this paper, a novel exploration strategy for Micro Aerial Vehicles (MAVs) is presented. The goal of the exploration strategy is the reduction of map entropy regarding occupancy probabilities, which is reflected in a utility function to be maximised. We achieve fast and efficient...
-
Autonomous landing on a moving platform presents unique challenges for multirotor vehicles, including the need to accurately localize the platform, fast trajectory planning, and precise/robust control. Previous works studied this problem but most lack explicit consideration of the wind disturbance, which typically leads to slow descents onto the platform. This work presents a fully autonomous visi...
-
Fixed-wing unmanned aerial vehicles (UAVs) offer significant performance advantages over rotary-wing UAVs in terms of speed, endurance, and efficiency. However, these vehicles have traditionally been severely limited with regards to maneuverability. In this paper, we present a nonlinear control approach for enabling aerobatic fixed-wing UAVs to maneuver in constrained spaces. Our approach utilizes...
-
In this work we present a novel, approximate, efficient algorithm for determining the Trim Flight Envelope of a fixed-wing UAV, based on a generic, nonlinear numerical model. The resulting Flight Envelope is expressed as a convex intersection of half-spaces. Subsequently, a Model Predictive Controller (MPC) is designed which takes into account the Flight Envelope constraints, to avoid Loss-of-Cont...
-
This paper presents a novel vehicle motion forecasting method based on multi-head attention. It produces joint forecasts for all vehicles on a road scene as sequences of multi-modal probability density functions of their positions. Its architecture uses multi-head attention to account for interactions between all vehicles, and long short-term memory layers for encoding and forecasting. It relies s...
-
We present a novel framework for object-level 3D underwater reconstruction using imaging sonar sensors. We demonstrate that imaging sonar reconstruction is analogous to the problem of confocal non-line-of-sight (NLOS) reconstruction. Drawing upon this connection, we formulate the problem as one of solving for volumetric albedo, where the scene of interest is modeled as a directionless albedo field...
-
This work presents a semantic map management approach for various environments by triggering multiple maps with different simultaneous localization and mapping (SLAM) configurations. A modular map structure allows to add, modify or delete maps without influencing other maps of different areas. The hierarchy level of our algorithm is above the utilized SLAM method. Evaluating laser scan data (e.g. ...
-
Performing collaborative semantic mapping is a critical challenge for cooperative robots to maintain a comprehensive contextual understanding of the surroundings. Most of the existing work either focus on single robot semantic mapping or collaborative geometry mapping. In this paper, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formu...
-
This paper focuses on real-time occupancy mapping and collision checking onboard an autonomous robot navigating in an unknown environment. We propose a new map representation, in which occupied and free space are separated by the decision boundary of a kernel perceptron classifier. We develop an online training algorithm that maintains a very sparse set of support vectors to represent obstacle bou...
-
Hybrid Topological and 3D Dense Mapping through Autonomous Exploration for Large Indoor Environments
Robots require a detailed understanding of the 3D structure of the environment for autonomous navigation and path planning. A popular approach is to represent the environment using metric, dense 3D maps such as 3D occupancy grids. However, in large environments the computational power required for most state-of-the-art 3D dense mapping systems is compromising precision and real-time capability. In...
-
Planar markers are useful in robotics and computer vision for mapping and localisation. Given a detected marker in an image, a frequent task is to estimate the 6DOF pose of the marker relative to the camera, which is an instance of planar pose estimation (PPE). Although there are mature techniques, PPE suffers from a fundamental ambiguity problem, in that there can be more than one plausible pose ...
-
A service robot is expected to provide proactive service for visitors who require its help. In contrast to passive service, e.g., providing service only after being spoken to, proactive service initiates an interaction at an early stage, e.g., talking to potential visitors who need the robot’s help in advance. This paper addresses how to anticipate the start of user interaction. We propose an appr...
-
In table tennis, the rotation (spin) of the ball plays a crucial role. A table tennis match will feature a variety of strokes. Each generates different amounts and types of spin. To develop a robot that can compete with a human player, the robot needs to detect spin, so it can plan an appropriate return stroke. In this paper we compare three methods to estimate spin. The first two approaches use a...
-
A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory inputs in an environment and to make a sequence of actions to reach their goals. In this paper, we attempt to approach the problem of Audio-Visual Embodied Navigation, the task of planning the shortest path from a random starting location in a scene to the sound source in an indoor environment, given ...
-
Autonomous tool construction is a significant but challenging task in robotics. This task can be interpreted as when given a reference tool, selecting some available candidate parts to reconstruct it. Most of the existing works perform tool construction in the form of action part and grasp part, which is only a specific construction pattern and limits its application to some extent. In general sce...
-
Correlation filter (CF) has recently exhibited promising performance in visual object tracking for unmanned aerial vehicle (UAV). Such online learning method heavily depends on the quality of the training-set, yet complicated aerial scenarios like occlusion or out of view can reduce its reliability. In this work, a novel time slot-based distillation approach is proposed to efficiently and effectiv...
-
It is difficult for both cameras and depth sensors to obtain reliable information in hazy scenes. Therefore, image dehazing is still one of the most challenging problems to solve in computer vision and robotics. With the development of convolutional neural networks (CNNs), lots of dehazing and depth estimation algorithms using CNNs have emerged. However, very few of those try to solve these two pr...
-
In this paper, an Internet of Things-based human-robot collaborative control scheme is developed in Robot-assisted Minimally Invasive Surgery scenario. A hierarchical operational space formulation is designed to exploit the redundancies of the 7-DoFs redundant manipulator to handle multiple operational tasks based on their priority levels, such as guaranteeing a remote center of motion constraint ...
-
The control of passive dynamic systems remains a challenging problem in the field of robotics, and insights from their study can inform everything from dynamic behaviors on actuated robots to robotic assistive devices. In this work, we explore the use of flat actuated environments for realizing passive dynamic balancing and locomotion. Specifically, we utilize a novel omnidirectional actuated floo...
-
The variable-height inverted pendulum (VHIP) model enables a new balancing strategy by height variations of the center of mass, in addition to the well-known ankle strategy. We propose a biped stabilizer based on linear feedback of the VHIP that is simple to implement, coincides with the state-of-the-art for small perturbations and is able to recover from larger perturbations thanks to this new st...
-
Biped walking involves a series of transitions between single support (SS) and double support (DS) contact configurations that can include both balanced and unbalanced states. The new concept of steppability is introduced to partition the set of unbalanced states into steppable states and falling (unsteppable) states based on the ability of a biped system to respond to forward velocity perturbatio...
-
This paper presents a study that evaluates the effects on the walking pattern of a full-sized humanoid robot as it pushes different carts. Furthermore, it discuss a modified Zero Moment Point (ZMP) pattern based on a capture point method, and a friction compensation method for the arms. Humanoid researchers have demonstrated that robots can perform a wide range of tasks including handling tools, c...
-
A compass-like biped robot can go down a gentle slope without the need of actuation through a proper choice of its dynamic parameter and starting from a suitable initial condition. Addition of control actions is requested to generate additional gaits and robustify the existing one. This paper designs an interconnection and damping assignment passivity-based control, rooted within the port-Hamilton...
-
This work deals with the problem of planning conflict-free paths for mobile robots in cluttered environments. Since centralized, coupled planning algorithms are computationally intractable for large numbers of robots, we consider decoupled planning, in which robots plan their paths sequentially in order of priority. Choosing how to prioritize the robots is a key consideration. State-of-the-art pri...
-
This paper presents a method for predicting the probability of future connectivity between mobile robots with range-limited communication. In particular, we focus on its application to active motion planning for cooperative localization (CL). The probability of connection is modeled by the distribution of quadratic forms in random normal variables and is computed by the infinite power series expan...
-
In many cases the multi-robot systems are desired to execute simultaneously multiple behaviors with different controllers, and sequences of behaviors in real time, which we call behavior mixing. Behavior mixing is accomplished when different subgroups of the overall robot team change their controllers to collectively achieve given tasks while maintaining connectivity within and across subgroups in...
-
This paper considers the optimal cooperative robotic manipulation problem in terms of energy resources. In particular, we consider rigid cooperative manipulation systems, i.e., with rigid grasping contacts, and study energy-optimal conditions in the sense of minimization of the arising internal forces, which are inter-agent forces that do not contribute to object motion. Firstly, we use recent res...
-
Unlike large and dangerous industrial robots at production lines in factories that are strictly fenced off, collaborative robots are smaller and safer and can be installed adjacent to human workers and collaborate with them. However, controlling and teaching new moves to collaborative robots can be difficult and time-consuming when using existing methods, such as pressing buttons on a teaching pen...
-
Executing multiple tasks concurrently is important in many robotic applications. Moreover, the prioritization of tasks is essential in applications where safety-critical tasks need to precede application-related objectives, in order to protect both the robot from its surroundings and vice versa. Furthermore, the possibility of switching the priority of tasks during their execution gives the roboti...
-
Human-robot interaction is one of the keys of assistive robots. Robots are expected to be compliant with people but at the same time correctly perform the tasks. In such applications, Cartesian impedance control is preferred over joint control, as the desired interaction and environmental feedback can be described more naturally, and the force to be exerted by the robot can be readily adjusted.Thi...
-
Understanding of characteristics of objects such as fragility, rigidity, texture and dimensions facilitates and innovates robotic grasping. In this paper, we propose a context- aware anthropomorphic robotic hand (MagicHand) grasping system which is able to gather various information about its target object and generate grasping strategies based on the perceived information. In this work, NIR spect...
-
In robotic grasping, objects are often occluded in ungraspable configurations such that no feasible grasp pose can be found, e.g. large flat boxes on the table that can only be grasped once lifted. Inspired by human bimanual manipulation, e.g. one hand to lift up things and the other to grasp, we address this type of problems by introducing pregrasp manipulation – push and lift actions. We propose...
-
This paper introduces the technique of tilt-and-pivot manipulation, a new method for picking thin, rigid objects lying on a flat surface through robotic dexterous in-hand manipulation. During the manipulation process, the gripper is controlled to reorient about the contact with the object such that its finger can get in the space between the object and the supporting surface, which is formed by ti...
-
The real-time segmentation of surgical instruments plays a crucial role in robot-assisted surgery. However, it is still a challenging task to implement deep learning models to do real-time segmentation for surgical instruments due to their high computational costs and slow inference speed. In this paper, we propose an attention-guided lightweight network (LWANet), which can segment surgical instru...
-
Current challenges in automated robotic breast ultrasound (US) acquisitions include keeping acoustic coupling between the breast and the US probe, minimizing tissue deformations and safety. In this paper, we present how an autonomous 3D breast US acquisition can be performed utilizing a 7DOF robot equipped with a linear US transducer. Robotic 3D breast US acquisitions would increase the diagnostic...
-
This paper presents a robust and accurate approach for the rigid registration of pre-operative and intraoperative point sets in image-guided surgery (IGS). Three challenges are identified in the pre-to-intraoperative registration: the intra-operative 3D data (usually forms a 3D curve in space) (1) is often contaminated with noise and outliers; (2) usually only covers a partial region of the whole ...
-
Surgical robots are used to perform minimally invasive surgery and alleviate much of the burden imposed on surgeons. Our group has developed a surgical robot to aid in the removal of tumors at the base of the skull via access through the nostrils. To avoid injuring the patients, a collision-avoidance algorithm that depends on having an accurate model for the poses of the instruments' shafts is use...
-
Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on smallscale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully util...
-
Robotic bronchoscopic intervention requires detailed 3D airway maps for both localisation and enhanced visualisation, especially at peripheral airways. Patient-specific airway maps can be generated from preoperative chest CT scans. Due to pathological abnormalities and anatomical variations, automatically delineating the airway tree with distal branches is a challenging task. In the paper, we prop...
-
Recently, it has become possible to easily design and fabricate robotic systems in the laboratory and at home due to the recent development of 3D printer technology. On the other hand, the strength of the plastic materials used in reasonably priced 3D printers and the accuracy of the printed parts are generally low. These problems affect the part-joining quality. Therefore, this paper describes a ...
-
We propose a gripper displacement-magnification mechanism and an extendable finger mechanism, both of which can be attached to a commercially available parallel gripper. We then verify the operation of the mechanism in order to expand applications of the parallel gripper. The displacement-magnification mechanism has a stacked rack-and-pinion system that doubles displacement. The extendable finger ...
-
Over the past few years, shape memory polymer (SMP) has been extensively studied in terms of its remarkable reversible dry adhesive properties and related smart adhesive applications. However, its exceptional properties have not been exploited for further opportunities such as pick-and-place applications, which would otherwise advance the robotic manipulation. This work explores the use of an SMP ...
-
It is an effective strategy for the multi-person pose tracking task in videos to employ prediction and pose matching in a frame-by-frame manner. For this type of approach, uncertainty-aware modeling is essential because precise prediction is impossible. However, previous studies have relied on only a single prediction without incorporating uncertainty, which can cause critical tracking errors if t...
-
Many high-level video understanding methods require input in the form of object proposals. Currently, such proposals are predominantly generated with the help of neural networks that were trained for detecting and segmenting a set of known object classes, which limits their applicability to cases where all objects of interest are represented in the training set. We propose an approach that can rel...
-
In this paper, we propose a novel method to simultaneously track the deformation of soft objects and estimate their elasticity parameters. The tracking of the deformable object is performed by combining the visual information captured by a RGB-D sensor with interactive Finite Element Method simulations of the object. The visual information is more particularly used to distort the simulated object....
-
Existing state-of-the-art object tracking can run into challenges when objects collide, occlude, or come close to one another. These visually based trackers may also fail to differentiate between objects with the same appearance but different materials. Existing methods may stop tracking or incorrectly start tracking another object. These failures are uneasy for trackers to recover from since they...
-
Pig counting is a crucial task for large-scale pig farming. Pigs are usually visually counted by human. But this process is very time-consuming and error-prone. Few studies in literature developed automated pig counting method. The existing works only focused on pig counting using single image, and its level of accuracy faced challenges due to pig movements, occlusion and overlapping. Especially, ...
-
We present 6-PACK, a deep learning approach to category-level 6D object pose tracking on RGB-D data. Our method tracks in real time novel object instances of known object categories such as bowls, laptops, and mugs. 6-PACK learns to compactly represent an object by a handful of 3D keypoints, based on which the interframe motion of an object instance can be estimated through keypoint matching. Thes...
-
Soft active materials can generate flexible locomotion and change configurations through large deformations when subjected to an external environmental stimulus. They can be engineered to design 'soft machines' such as soft robots, compliant actuators, flexible electronics, or bionic medical devices. By embedding ferromagnetic particles into soft elastomer matrix, the ferromagnetic soft matter can...
-
Soft robots offer significant advantages in adaptability, safety, and dexterity compared to conventional rigid-body robots. However, it is challenging to equip soft robots with accurate proprioception and tactile sensing due to their high flexibility and elasticity. In this work, we describe the development of a vision-based proprioceptive and tactile sensor for soft robots called GelFlex, which i...
-
Soft robots demonstrate great potential compared with traditional rigid robots owing to their inherently soft body structures. Although researchers have made tremendous progress in recent years, existing soft robots are in general plagued by a main issue: slow speeds and small forces. In this work, we aim to address this issue by actively designing the energy landscape of the soft body: the total ...
-
Origami robots are well-suited for jumping maneuvers because of their light weight and ability to incorporate actuation and control strategies directly into the robot body. However, existing origami robots often model fold patterns as rigidly foldable and fail to take advantage of deformation in an origami sheet for potential energy storage. In this paper, we consider a parametric origami tessella...
-
Surface electromyography contains muscular information representing gestures and corresponding forces. However, conventional sEMG-based motion recognition methods, such as pattern classification and regression, have intrinsic limitations due to the complex characteristics of sEMG signals. In this paper, motion intensity, a highly selective sEMG feature proportional to the level of muscle contracti...
-
The development of algorithms for motion discrimination in home rehabilitation sessions poses numerous challenges. Recent studies have used the concept of synergies to discriminate a set of movements. However, the discrimination depends on the correlation of the reconstructed movement with the online data, and the training data requires well-defined movements. In this paper, we introduced the conc...
-
For those with irregular gait, re-calibration of motor control strategies and retraining of coordination are key goals. Thoughtful external forces or resistances during repetitive tasks can reprogram motor control patterns and strategies. Prior work in our lab has utilized this theory to improve gait in various patient groups using the Tethered Pelvic Assist Device (TPAD), a treadmill-based roboti...
-
Functional electrical stimulation (FES) is a promising technique for restoring reaching ability to individuals with tetraplegia. To this point, the complexities of goal-directed reaching motions and the shoulder-arm complex have prevented the realization of this potential in full-arm 3D reaching tasks. We trained a Gaussian process regression model to form the basis of a feedforward-feedback contr...
-
Relatively little work in human and robot control has examined the control of underactuated objects with internal dynamics, such as transporting a cup of coffee, a task that presents little problems for humans. This study examined how humans move a `cup of coffee' with a view to identify principles that may be useful for robot control. The specific focus was on how humans choose initial conditions...
-
In today's automation driven manufacturing environments, emerging technologies like cobots (collaborative robots) and augmented reality interfaces can help integrating humans into the production workflow to benefit from their adaptability and cognitive skills. In such settings, humans are expected to work with robots side by side and physically interact with them. However, the trade-off between st...
-
Autonomous robots have the potential to serve as versatile caregivers that improve quality of life for millions of people worldwide. Yet, conducting research in this area presents numerous challenges, including the risks of physical interaction between people and robots. Physics simulations have been used to optimize and train robots for physical assistance, but have typically focused on a single ...
-
This paper presents a learning-from-demonstration (LfD) framework for teaching human-robot social interactions that involve whole-body haptic interaction, i.e. direct human-robot contact over the full robot body. The performance of existing LfD frameworks suffers in such interactions due to the high dimensionality and spatiotemporal sparsity of the demonstration data. We show that by leveraging th...
-
Human Preferences in Using Damping to Manage Singularities During Physical Human-Robot Collaboration
When a robot manipulator approaches a kinematic singular configuration, control strategies need to be employed to ensure safe and robust operation. If this manipulator is being controlled by a human through physical human-robot collaboration, the choice of strategy for handling singularities can have a significant effect on the feelings and impressions of the user. To date the preferences of human...
-
The objective of this paper is to create a new collaborative robotic system that subsumes the advantages of mobile manipulators and supernumerary limbs. By exploiting the reconfiguration potential of a MObile Collaborative robot Assistant (MOCA), we create a collaborative robot that can function autonomously, in close proximity to humans, or be physically coupled to the human counterpart as a supe...
-
In the present paper, we first adopt explicit force control from general robotics and embed it into teleoperation systems to enhance the transparency by reducing the effect of the perceived inertia to the human operator and simultaneously improve contact perception. To ensure stability of the proposed teleoperation system considering time-delays, we propose a sequential design procedure based on t...
-
Search and rescue robotics is becoming a relevant topic in the last years and the growing number of robotic platforms and dedicated projects is the evidence of the interest in this area. In this context, the possibility to drive a remote robot with an exoskeleton is a promising strategy to enhance dexterity, reduce operator effort and save time. However, the use of haptic feedback (bilateral teleo...
-
Drone teleoperation is usually accomplished using remote radio controllers, devices that can be hard to master for inexperienced users. Moreover, the limited amount of information fed back to the user about the robot's state, often limited to vision, can represent a bottleneck for operation in several conditions. In this work, we present a wearable interface for drone teleoperation and its evaluat...
-
In some applications, operators may want to create fluid, human-like motion on a remotely-operated robot, for example, a device used for remote telepresence. This paper examines two methods of controlling the pose of a Baxter robot via an Xbox One controller. The first method is a joint- by-joint (JBJ) method in which one joint of each limb is specified in sequence. The second method of control, n...
-
As artificial intelligence becomes an increasingly prevalent method of enhancing robotic capabilities, it is important to consider effective ways to train these learning pipelines and to leverage human expertise. Working towards these goals, a master-apprentice model is presented and is evaluated during a grasping task for effectiveness and human perception. The apprenticeship model augments self-...
-
Virtual fixtures (VFs) improve human operator performance in teleoperation scenarios. However, the generation of VFs is challenging, especially in unstructured environments. In this work, we introduce a framework for the interactive generation of VF. The method is based on the observation that a human can easily understand just by looking at the remote environment which VF could help in task execu...
-
This paper focuses on local path planning for obstacle avoidance tasks dedicated to off-road mobile robots. This approach calculates a new local path for the vehicle using a set of cubic Bezier curves once the safety distance is not respected; otherwise, the vehicle follows the global reference path which is defined off-line. Two basic steps are used to determine this new path. Firstly, some signi...
-
Collision avoidance is a key technology towards safe human-robot interaction, especially on-line and fastreacting motions are required. Skins with proximity sensors mounted on the robot's outer shell provide an interesting approach to occlusion-free and low-latency perception. However, collision avoidance algorithms which make extensive use of these properties for fast-reacting motions have not ye...
-
Horizontally scanning 2D laser rangefinders are a popular approach for indoor robot localization because of the high accuracy of the sensors and the compactness of the required 2D maps. As the scanners in this configuration only provide information about one slice of the environment, the measurements typically do not capture the full extent of a large variety of obstacles, including chairs or tabl...
-
State of the art methods for robotic path planning in dynamic environments, such as crowds or traffic, rely on hand crafted motion models for agents. These models often do not reflect interactions of agents in real world scenarios. To overcome this limitation, this paper proposes an integrated path planning framework using generative Recurrent Neural Networks within a Monte Carlo Tree Search (MCTS...
-
This paper details a novel approach to collision avoidance for aerial vehicles that enables high-speed flight in uncertain environments. This framework is applied at the controller level and provides safety regardless of the planner that is used. The method is shown to be robust to state uncertainty and disturbances, and is computed entirely online utilizing the full nonlinear system dynamics. The...
-
We propose thrombolysis using a magnetic nanoparticle microswarm with tissue plasminogen activator (tPA) under ultrasound imaging. The microswarm is generated in blood using an oscillating magnetic field and can be navigated with locomotion along both the long and short axis. By modulating the input field, the aspect ratio of the microswarm can be reversibly tuned, showing the ability to adapt to ...
-
Micromanipulation is challenging due to the specific physical effects governing the microworld. Interactive approaches using only visual feedback are limited to the 2D image of the microscope, and have forcibly lower bandwidth. Recently, haptic feedback teleoperation systems have been developed to try to overcome those difficulties. This paper explores the case of an optical tweezers platform coup...
-
This paper presents a novel steering mechanism, which leads to frequency-controlled locomotion demonstrated for the first time in micro bristle robots. The miniaturized robots are 3D-printed, 12 mm × 8 mm × 6 mm in size, with bristle feature sizes down to 400 µm. The robots can be steered by utilizing the distinct resonance behaviors of the asymmetrical bristle sets. The left and right sets of the...
-
Here we present HAMR-Jr, a 22.5mm, 320mg quadrupedal microrobot. With eight independently actuated degrees of freedom, HAMR-Jr is, to our knowledge, the most mechanically dexterous legged robot at its scale and is capable of high-speed locomotion (13.91bodylengthss-1) at a variety of stride frequencies (1-200Hz) using multiple gaits. We achieved this using a design and fabrication process that is ...
-
We consider problems in which robots conspire to present a view of the world that differs from reality. The inquiry is motivated by the problem of validating robot behavior physically despite there being a discrepancy between the robots we have at hand and those we wish to study, or the environment for testing that is available versus that which is desired, or other potential mismatches in this ve...
-
Recently, Linear Temporal Logic (LTL) has been used as a formalism for defining high-level robot tasks, and LTL synthesis has been used to automatically create correct-by-construction robot control. The underlying premise of this approach is that the robot has a set of actions, or skills, that can be composed to achieve the high- level task. In this paper we consider LTL specifications that cannot...
-
We consider the problem of optimal reactive synthesis - compute a strategy that satisfies a mission specification in a dynamic environment, and optimizes a given performance metric. We incorporate task-critical information, that is only available at runtime, into the strategy synthesis in order to improve performance. Existing approaches to utilising such time-varying information require online re...
-
We present a reinforcement learning (RL) frame-work to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Process (MDP). Specifically, we learn a policy that maximizes the probability of satisfying the LTL formula without learning the transition probabilities. We introduce a novel rewardin...
-
This paper introduces an intrinsically safe parallel manipulator dedicated to fast pick-and-place operations, called R-Min. It has been designed to reduce the risk of injury during a collision with a human operator, while maintaining high speed and acceleration. The proposed architecture is based on a modification of the well-known planar five-bar mechanism, where additional passive joints are int...
-
In this paper, we propose a parallel six-wheel-legged robot that can traverse irregular terrain while carrying objectives to do heavy-duty work. This robot is equipped with six Stewart platforms as legs and tightly integrates the additional degrees of freedom introduced by the wheels. The presented control strategy with physical system used to adapt the diverse degrees of each leg to irregular ter...
-
This paper presents the development and results of a large 3 d.o.f cable-driven parallel robot (CDPR) that has been extensively used between June and August 2019 for an artistic exhibition. The purpose of the exhibition was to 3D print a wall of glass powder, which will slowly collapse after the deposit of each layer. Positioning control on the assigned trajectory was an issue because of the CDPR ...
-
The 3-CRS manipulator is an original parallel mechanism having 6 degrees of freedom (DOFs) with only 3 limbs. This mechanism uses a motorized cylindrical joint per limb. This new paradigm of actuation opens research fields on new families of robots that should particularly interest the parallel robotics community. According to its dimensional synthesis, this mechanism can have remarkable kinematic...
-
We present a novel class of robots belonging to Constrained Collaborative Mobile Agents (CCMA) family which consists of ground mobile bases with non-holonomic constraints. Moreover, these mobile robots are constrained by closed-loop kinematic chains consisting of revolute joints which can be either passive or actuated. We also describe a novel trajectory optimization method which is general with r...
-
A wheeled robot operating on various terrains such as scattered obstacles and slopes is required to cope with and overcome the driving environment. In this paper, in order to overcome a step-type obstacle and to steadily ascend on the slope, the main body rotation mechanism, which controls the load distribution on the robot wheels was proposed for a wheel-drive robot. By rotating the center of the...
-
With the availability of small system in package realizations, radar systems become more and more attractive for a variety of applications in robotics, in particular also for collaborative robotics. As the simulation of robot systems in realistic scenarios has become an important tool, not only for design and optimization, but also e.g. for machine learning approaches, realistic simulation models ...
-
While robots can learn models to solve many manipulation tasks from raw visual input, they cannot usually use these models to solve new problems. On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of groundi...
-
Index of papers presented at ICRA 2020 and published in the IEEE Robotics and Automation Letters
-
We propose an efficient method for monocular simultaneous localization and mapping (SLAM) that is capable of estimating metrically-scaled motion without additional sensors or hardware acceleration by integrating metric depth predictions from a neural network into a geometric SLAM factor graph. Unlike learned end-to-end SLAM systems, ours does not ignore the relative geometry directly observable in...
-
We formulate for the first time visual-inertial initialization as an optimal estimation problem, in the sense of maximum-a-posteriori (MAP) estimation. This allows us to properly take into account IMU measurement uncertainty, which was neglected in previous methods that either solved sets of algebraic equations, or minimized ad-hoc cost functions using least squares. Our exhaustive initialization ...
-
Accurate, robust and real-time localization under constrained-resources is a critical problem to be solved. In this paper, we present a new sparse pose-graph visual-inertial SLAM (SPVIS). Unlike the existing methods that are costly to deal with a large number of redundant features and 3D map points, which are inefficient for improving positioning accuracy, we focus on the concise visual cues for h...
-
Keypoint matching is an important operation in computer vision and its applications such as visual simultaneous localization and mapping (SLAM) in robotics. This matching operation heavily depends on the descriptors of the keypoints, and it must be performed reliably when images undergo conditional changes such as those in illumination and viewpoint. In this paper, a descriptor fusion model (DFM) ...
-
Sparse-indirect SLAM systems have been dominantly popular due to their computational efficiency and photometric invariance properties. Depth sensors are critical to SLAM frameworks for providing scale information to the 3D world, yet known to be plagued by a wide variety of noise sources, possessing lateral and axial components. In this work, we demonstrate the detrimental impact of these depth no...
-
Simultaneous Localization and Mapping (SLAM) in large-scale, unknown, and complex subterranean environments is a challenging problem. Sensors must operate in off-nominal conditions; uneven and slippery terrains make wheel odometry inaccurate, while long corridors without salient features make exteroceptive sensing ambiguous and prone to drift; finally, spurious loop closures that are frequent in e...
-
When incorporating deep neural networks into robotic systems, a major challenge is the lack of uncertainty measures associated with their output predictions. Methods for uncertainty estimation in the output of deep object detectors (DNNs) have been proposed in recent works, but have had limited success due to 1) information loss at the detectors nonmaximum suppression (NMS) stage, and 2) failure t...
-
Robots coexisting with humans in their environment and performing services for them need the ability to interact with them. One particular requirement for such robots is that they are able to understand spatial relations and can place objects in accordance with the spatial relations expressed by their user. In this work, we present a convolutional neural network for estimating pixelwise object pla...
-
Deep neural networks (DNNs) have achieved great success in the area of computer vision. The disparity estimation problem tends to be addressed by DNNs which achieve much better prediction accuracy in stereo matching than traditional hand-crafted feature based methods. On one hand, however, the designed DNNs require significant memory and computation resources to accurately predict the disparity, e...
-
Deep learning has become an increasingly common technique for various control problems, such as robotic arm manipulation, robot navigation, and autonomous vehicles. However, the downside of using deep neural networks to learn control policies is their opaque nature and the difficulties of validating their safety. As the networks used to obtain state-of-the-art results become increasingly deep and ...
-
A longstanding challenge in robot learning for manipulation tasks has been the ability to generalize to varying initial conditions, diverse objects, and changing objectives. Learning based approaches have shown promise in producing robust policies, but require heavy supervision and large number of environment interactions, especially from visual inputs. We propose a novel self-supervision techniqu...
-
Path planning is an active area of research essential for many applications in robotics. Popular techniques include graph-based searches and sampling-based planners. These approaches are powerful but have limitations.This paper continues work to combine their strengths and mitigate their limitations using a unified planning paradigm. It does this by viewing the path planning problem as the two sub...
-
One major challenge in Assembly Sequence Planning (ASP) for complex real-world CAD-scenarios is to find appropriate disassembly paths for all assembled parts. Such a path places demands on its length and clearance. In the past, it became apparent that planning the disassembly path based on the (approximate) General Voronoi Diagram (GVD) is a good approach to achieve these requirements. But for com...
-
In this work, we propose an augmentation to the Dynamic Movement Primitives (DMP) framework which allows the system to generalize to moving goals without the use of any known or approximation model for estimating the goal's motion. We aim to maintain the demonstrated velocity levels during the execution to the moving goal, generating motion profiles appropriate for human robot collaboration. The p...
-
In this paper, we revisit model predictive control (MPC) for the classical wheeled mobile robot (WMR) navigation problem. We prove that the reachable set based hierarchical MPC (HMPC), a state-of-the-art MPC, cannot handle WMR navigation in theory due to the non-existence of non-trivial linear system with an under-approximate reachable set of WMR. Nevertheless, we propose a virtual linear leader g...
-
This paper is about the closely-related problems of localization and aggregation for extremely simple robots, for which the only available action is to move in a given direction as far as the geometry of the environment allows. Such problems may arise, for example, in biomedical applications, wherein a large group of tiny robots moves in response to a shared external stimulus. Specifically, we ext...
-
Stable flight control under ceilings is difficult for multirotor Unmanned Aerial Vehicles (UAVs). The wake interaction between rotors and upper walls, called the "ceiling effect", causes an increase of rotor thrust. As a result of the thrust increase, multi-rotors are drawn upward abruptly and collide with ceilings. In previous work, several thrust models of the ceiling effect have been proposed f...
-
This paper presents a novel path planning strategy for fast and agile exploration using aerial robots. Tailored to the combined need for large-scale exploration of challenging and confined environments, despite the limited endurance of micro aerial vehicles, the proposed planner employs motion primitives to identify admissible paths that search the configuration space, while exploiting the dynamic...
-
We propose a novel anomaly detection framework for a fleet of hybrid aerial vehicles executing high-speed package pickup and delivery missions. The detection is based on machine learning models of normal flight profiles, trained on millions of flight log measurements of control inputs and sensor readings. We develop a new scalable algorithm for robust regression which can simultaneously fit predic...
-
Correlation filter-based tracking has been widely applied in unmanned aerial vehicle (UAV) with high efficiency. However, it has two imperfections, i.e., boundary effect and filter corruption. Several methods enlarging the search area can mitigate boundary effect, yet introducing undesired background distraction. Existing frame-by-frame context learning strategies for repressing background distrac...
-
Regrasping is one of the most common and important manipulation skills used in our daily life. However, aerial regrasping has not been seriously investigated yet, since most of the aerial manipulator lacks dexterous manipulation abilities except for the basic pick-and-place. In this paper, we focus on pivoting a long box, which is one of the most classical problems among regrasping researches, usi...
-
Robots operating in outdoor, urban environments need the ability to follow complex natural language commands which refer to never-before-seen landmarks. Existing approaches to this problem are limited because they require training a language model for the landmarks of a particular environment before a robot can understand commands referring to those landmarks. To generalize to new environments out...
-
Deep Merging: Vehicle Merging Controller Based on Deep Reinforcement Learning with Embedding Network
Vehicles at highway merging sections must make lane changes to join the highway. This lane change can generate congestion. To reduce congestion, vehicles should merge so as not to affect traffic flow as much as possible. In our study, we propose a vehicle controller called Deep Merging that uses deep reinforcement learning to improve the merging efficiency of vehicles while considering the impact ...
-
It has been demonstrated that the performance of an object detector degrades when it is used outside the domain of the data used to train it. However, obtaining training data for a new domain can be time consuming and expensive. In this work we demonstrate how a radar can be used to generate plentiful (but noisy) training data for image-based vehicle detection. We then show that the performance of...
-
Formula Student Driverless (FSD) is a competition where student teams compete to build an autonomous racecar. The main dynamic event in FSD is trackdrive, where the racecar traverses an unknown track with lanes demarcated by cones. One major challenge of the event is to determine the boundaries of the lane from cones perceived online despite false positive cone detections and sharp turns. We prese...
-
Cooperative adaptive cruise control (CACC) of intelligent vehicles contributes to improving cruise control performance, reducing traffic congestion, saving energy and increasing traffic flow capacity. In this paper, we resolve the CACC problem from the viewpoint of synchronization control, our main idea is to introduce the spatial-temporal synchronization mechanism into vehicle platoon control to ...
-
Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a ...
-
Accurate and reliable localization is crucial to autonomous vehicle navigation and driver assistance systems. This paper presents a novel approach for online vehicle localization in a digital map. Two distinct map matching algorithms are proposed: i) Iterative Closest Point (ICP) based lane level map matching is performed with visual lane tracker and grid map ii) decision-rule based approach is us...
-
This paper presents a new technique to solve the Indoor Visual Place Recognition problem from the Deep Learning perspective. It consists on an image retrieval approach supported by a novel image similarity metric. Our work uses a 3D laser sensor mounted on a backpack with a calibrated spherical camera i) to generate the data for training the deep neural network and ii) to build a database of geo-r...
-
Pose estimation and map building are central ingredients of autonomous robots and typically rely on the registration of sensor data. In this paper, we investigate a new metric for registering images that builds upon on the idea of the photometric error. Our approach combines a gradient orientation-based metric with a magnitude-dependent scaling term. We integrate both into stereo estimation as wel...
-
A robot operating in the world constantly receives information about its environment in the form of new measurements at every time step. Smoothing-based estimation methods seek to optimize for the most likely robot state estimate using all measurements up till the current time step. Existing methods solve for this smoothing objective efficiently by framing the problem as that of incremental uncons...
-
Besides being part of the Internet of Things (IoT), drones can play a relevant role in it as enablers. The 3D mobility of UAVs can be exploited to improve node localization in IoT networks for, e.g., search and rescue or goods localization and tracking. One of the widespread IoT communication technologies is Long Range Wide Area Network (LoRaWAN), which allows achieving long communication distance...
-
We describe and experimentally demonstrate a practical method for indoor localization using measurements obtained from resource-constrained devices with limited sensing capabilities. We focus on handheld/mobile devices but the method can be useful for a variety of wearable devices. Our system works with sparse WiFi or image-based measurements, avoiding laborious site surveying for dense signal map...
-
Nowadays, mobile robots are deployed in many indoor environments such as offices or hospitals. These environments are subject to changes in the traversability that often happen following patterns. In this paper, we investigate the problem of navigating in such environments over extended periods of time by capturing and exploiting these patterns to make informed decisions for navigation. Our approa...
-
Gaussian processes (GPs) based Reinforcement Learning (RL) methods with Model Predictive Control (MPC) have demonstrated their excellent sample efficiency. However, since the computational cost of GPs largely depends on the training sample size, learning an accurate dynamics using GPs result in low control frequency in MPC. To alleviate this trade-off and achieve a sample-and-computation-efficient...
-
Robotic manipulators are reaching a state where we could see them in household environments in the following decade. Nevertheless, such robots need to be easy to instruct by lay people. This is why kinesthetic teaching has become very popular in recent years, in which the robot is taught a motion that is encoded as a parametric function - usually a Movement Primitive (MP)-. This approach produces ...
-
This paper provides a real time on-board algorithm for a biplane-quadrotor to iteratively learn a forward transition maneuver via repeated flight trials. The maneuver is controlled by regulating the pitch angle and propeller thrust according to feedforward control laws that are parameterized by polynomials. Based on a nominal model with simplified aerodynamics, the optimal coefficients of the poly...
-
We propose a learning algorithm for automating image sampling in scientific applications. We consider settings where images are sampled by controlling a probe beam's scanning trajectory over the image surface. We explore alternatives to obtaining images by the standard rastering method. We formulate the scanner control problem as a reinforcement learning (RL) problem and train a policy to adaptive...
-
Gaussian processes (GPs) enable a probabilistic approach to important estimation and classification tasks that arise in robotics applications. Meanwhile, most GP-based methods are often prohibitively slow, thereby posing a substantial barrier to practical applications. Existing "sparse" methods to speed up GPs seek to either make the model more sparse, or find ways to more efficiently manage a lar...
-
Surgical robots have been introduced to operating rooms over the past few decades due to their high sensitivity, small size, and remote controllability. The cable-driven nature of many surgical robots allows the systems to be dexterous and lightweight, with diameters as low as 5mm. However, due to the slack and stretch of the cables and the backlash of the gears, inevitable uncertainties are broug...
-
Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The ob...
-
Assistive robotic arms enable users with physical disabilities to perform everyday tasks without relying on a caregiver. Unfortunately, the very dexterity that makes these arms useful also makes them challenging to teleoperate: the robot has more degrees-of-freedom than the human can directly coordinate with a handheld joystick. Our insight is that we can make assistive robots easier for humans to...
-
Torque distribution in redundant robots that combine the potential of asymmetric series-parallel actuated branches and multi-articulation pose a non-trivial challenge. To address the problem, this work proposes a novel optimization based controller that can accommodate various quadratic criteria to perform the torque distribution among dissimilar series and parallel actuators in order to maximize ...
-
In this paper, we present a novel factor graph formulation to estimate the pose and velocity of a quadruped robot on slippery and deformable terrain. The factor graph introduces a preintegrated velocity factor that incorporates velocity inputs from leg odometry and also estimates related biases. From our experimentation we have seen that it is difficult to model uncertainties at the contact point ...
-
Energy-efficient locomotion is of primary importance for legged robot to extend operation time in practical applications. This paper presents an approach to achieve energy-efficient locomotion for a quadrupedal robot walking over challenging terrains. Firstly, we optimize the nominal stance parameters based on the analysis of leg torque distribution. Secondly, we proposed the foothold planner and ...
-
Optimization based predictive control is a powerful tool that has improved the ability of legged robots to execute dynamic maneuvers and traverse increasingly difficult terrains. However, it is often challenging and unintuitive to design meaningful cost functions and build high-fidelity models while adhering to timing restrictions. A novel framework to extract and design principled regularization ...
-
Learning to locomote to arbitrary goals on hardware remains a challenging problem for reinforcement learning. In this paper, we present a hierarchical framework that improves sample-efficiency and generalizability of learned locomotion skills on real-world robots. Our approach divides the problem of goal-oriented locomotion into two sub-problems: learning diverse primitives skills, and using model...
-
Soft robotics technology creates new ways for legged robots to interact with and adapt to their environment. In this paper we develop i) a new 2-degree-of-freedom soft pneumatic actuator, and ii) a novel soft robotic hexapedal robot called SoRX that leverages the new actuators. Simulation and physical testing confirm that the proposed actuator can generate cyclic foot trajectories that are appropr...
-
Unmanned aerial vehicles (UAVs) have been widely used in many applications. The limited flight time of UAVs, however, still remains as a major challenge. Although numerous approaches have been developed to recharge the battery of UAVs effectively, little is known about optimal methodologies to deploy charging stations. In this paper, we address the charging station deployment problem with an aim t...
-
This paper presents a new efficient algorithm which guarantees a solution for a class of multi-agent trajectory planning problems in obstacle-dense environments. Our algorithm combines the advantages of both grid-based and optimization-based approaches, and generates safe, dynamically feasible trajectories without suffering from an erroneous optimization setup such as imposing infeasible collision...
-
We study the problem of sequential task assignment and collision-free routing for large teams of robots in applications with inter-task precedence constraints (e.g., task A and task B must both be completed before task C may begin). Such problems commonly occur in assembly planning for robotic manufacturing applications, in which sub-assemblies must be completed before they can be combined to form...
-
The challenges of multi-robot navigation in dynamic environments lie in uncertainties in obstacle complexities, partially observation of robots, and policy implementation from simulations to the real world. This paper presents a cooperative approach to address the multi-robot navigation problem (MRNP) under dynamic environments using a deep reinforcement learning (DRL) framework, which can help mu...
-
Autonomous navigation through unknown and complex environments is a fundamental capability that is essential in almost all robotic applications. Optimal robot path planning is critical to enable efficient navigation. Path planning is a complex, compute and memory intensive task. Traditional methods employ either graph based search methods or sample based methods to implement path planning, which a...
-
This paper presents a robot trajectory optimization formulation that builds upon numerical optimal control and Lie group methods. In particular, the inherent sparsity of direct collocation is carefully analyzed to dramatically reduce the number of floating-point operations to get first-order information of the problem. We describe how sparsity exploitation is employed with both numerical and analy...
-
Autonomous cars and fixed-wing aerial vehicles have the so-called non-holonomic kinematics which non-linearly maps control input to states. As a result, trajectory optimization with such a motion model becomes highly non-linear and non-convex. In this paper, we improve the computational tractability of non-holonomic trajectory optimization by reformulating it in terms of a set of bi-convex cost an...
-
As we build our legged robots smaller and cheaper, stable and agile control without expensive inertial sensors becomes increasingly important. We seek to enable versatile dynamic behaviors on robots with limited modes of state feedback, specifically proprioceptive-only sensing. This work uses model-based trajectory optimization methods to design open-loop stable motion primitives. We specifically ...
-
In this paper, we propose an optimization-based decision-making tool for safe motion planning and control in an environment with randomly moving obstacles. The unique feature of the proposed method is that it limits the risk of unsafety by a pre-specified threshold even when the true probability distribution of the obstacles' movements deviates, within a Wasserstein ball, from an available empiric...
-
We present a new method to generate optimal grasps for brittle and fragile objects using a novel stress- minimization (SM) metric. Our approach is designed for objects that are composed of homogeneous isotopic materials. Our SM metric measures the maximal resistible external wrenches that would not result in fractures in the target objects. In this paper, we propose methods to compute our new metr...
-
A robust grasp controller for both slipping avoidance and controlled sliding is proposed based on force/tactile feedback only. The model-based algorithm exploits a modified LuGre friction model to consider rotational frictional sliding motions. The modification relies on the Limit Surface concept where a novel computationally efficient method is introduced to compute in real-time the minimum grasp...
-
Manipulating objects whose physical properties are unknown remains one of the greatest challenges in robotics. Controlling grasp force is an essential aspect of handling unknown objects without slipping or crushing them. Although extensive research has been carried out on grasp force control, unknown object manipulation is still difficult because conventional approaches assume that object properti...
-
Humans can quickly determine the force required to grasp a deformable object to prevent its sliding or excessive deformation through vision and touch, which is still a challenging task for robots. To address this issue, we propose a novel 3D convolution-based visual-tactile fusion deep neural network (C3D-VTFN) to evaluate the grasp state of various deformable objects in this paper. Specifically, ...
-
Current end-to-end grasp planning methods propose grasps in the order of seconds that attain high grasp success rates on a diverse set of objects, but often by constraining the workspace to top-grasps. In this work, we present a method that allows end-to-end top-grasp planning methods to generate full six-degree-of-freedom grasps using a single RGBD view as input. This is achieved by estimating th...
-
Consumer demand for augmented reality (AR) in mobile phone applications, such as the Apple ARKit. Such applications have potential to expand access to robot grasp planning systems such as Dex-Net. AR apps use structure from motion methods to compute a point cloud from a sequence of RGB images taken by the camera as it is moved around an object. However, the resulting point clouds are often noisy d...
-
In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360° coverage of stereo observations of the environment. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estima...
-
In the last few years, there has been a growing interest in taking advantage of the 360° panoramic images potential, while managing the new challenges they imply. While several tasks have been improved thanks to the contextual information these images offer, object recognition in indoor scenes still remains a challenging problem that has not been deeply investigated. This paper provides an object ...
-
Fisheye cameras are commonly used in applications like autonomous driving and surveillance to provide a large field of view (> 180o). However, they come at the cost of strong non-linear distortions which require more complex algorithms. In this paper, we explore Euclidean distance estimation on fisheye cameras for automotive scenes. Obtaining accurate and dense depth supervision is difficult in pr...
-
Recently, end-to-end trainable deep neural networks have significantly improved stereo depth estimation for perspective images. However, 360° images captured under equirectangular projection cannot benefit from directly adopting existing methods due to distortion introduced (i.e., lines in 3D are not projected onto lines in 2D). To tackle this issue, we present a novel architecture specifically de...
-
Omnidirectional 360° camera proliferates rapidly for autonomous robots since it significantly enhances the perception ability by widening the field of view (FoV). However, corresponding 360° depth sensors, which are also critical for the perception system, are still difficult or expensive to have. In this paper, we propose a low-cost 3D sensing system that combines an omnidirectional camera with a...
-
3D orientation estimation is a key component of many important computer vision tasks such as autonomous navigation and 3D scene understanding. This paper presents a new CNN architecture to estimate the 3D orientation of an omnidirectional camera with respect to the world coordinate system from a single spherical panorama. To train the proposed architecture, we leverage a dataset of panoramas named...
-
The only way to perceive a small object held between our fingers is to trust our sense of touch. Touch provides cues about the state of the contact even if its view is occluded by the finger. The interaction between the soft fingers and the surface reveals crucial information, such as the local shape of the object, that plays a central role in fine manipulation. In this work, we present a new sphe...
-
An unstable grasp pose can lead to slip, thus an unstable grasp pose can be predicted by slip detection. A regrasp is required afterwards to correct the grasp pose in order to finish the task. In this work, we propose a novel regrasp planner with multi-sensor modules to plan grasp adjustments with the feedback from a slip detector. Then a regrasp planner is trained to estimate the location of cent...
-
Incorporating touch as a sensing modality for robots can enable finer and more robust manipulation skills. Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. In this paper, we introduce OmniTact, a multi-directional high-resolution tactile sensor. OmniTact is designed to be used as a fingertip for robotic manipulation with robotic hands, a...
-
Texture sensing is one of the types of information sensed by humans through touch, and is thus of interest to robotics that this type of information can be acquired and processed. In this work we present a texture topography sensor based on a ciliary structure, a biological structure found in many organisms. The device consists of up to 9 elastic cilia with permanent magnetization assembled on top...
-
Remote characterisation of the environment during physical robot-environment interaction is an important task commonly accomplished in telerobotics. This paper demonstrates how tactile and proximity sensing can be efficiently used to perform automatic crack detection. A custom-designed integrated tactile and proximity sensor is implemented. It measures the deformation of its body when interacting ...
-
We propose a new capacitive proximity sensor that detects deformations of a suction cup as a tactile sense. We confirmed that one sensor module provides three applications for reliable picking and a simplified setup. The first application is the picking height decision. The second one is the placing height decision for detecting whether the grasped object is placed on the placement surface. These ...
-
In this paper, we propose a novel filtering-based method that fuses events from a dynamic vision sensor (DVS), images, and inertial measurements to estimate camera poses. A DVS is a bio-inspired sensor that generates events triggered by brightness changes. It can cover the drawbacks of a conventional camera by virtual of its independent pixels and high dynamic range. Specifically, we focus on opti...
-
In this paper we investigate the effect of tightly-coupled estimation on the performance of visual-inertial localization and dynamic object pose tracking. In particular, we show that while a joint estimation system outperforms its decoupled counterpart when given a "proper" model for the target's motion, inconsistent modeling, such as choosing improper levels for the target's propagation noises, c...
-
The advances in deep reinforcement learning recently revived interest in data-driven learning based approaches to navigation. In this paper we propose to learn viewpoint invariant and target invariant visual servoing for local mobile robot navigation; given an initial view and the goal view or an image of a target, we train deep convolutional network controller to reach the desired goal. We presen...
-
In this work, we propose a tightly-coupled odometry framework, which combines monocular visual feature observations with distance measurements provided by a single ultra-wideband (UWB) anchor with an initial guess for its location. Firstly, the scale factor and the anchor position in the vision frame will be simultaneously estimated using a variant of Levenberg-Marquardt non-linear least squares o...
-
Visual topological navigation has been revitalized recently thanks to the advancement of deep learning that substantially improves robot perception. However, the scalability and reliability issue remain challenging due to the complexity and ambiguity of real world images and mechanical constraints of real robots. We present an intuitive approach to show that by accurately measuring the capability ...
-
Imitation learning is a popular approach for training effective visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for long periods of time. In this work, we propose a zero-shot imitation learning framework for training a goal-driven visual navigation policy on a...
-
The high deformability and compliance of soft robots allow safer interaction with the environment. On the other hand, these advantages bring along controllability and predictability challenges which result in loss of force and stiffness output. Such challenges should be addressed in order to improve the overall functional performance and to meet the requirements of real-scenario applications. In t...
-
Soft growing robots are proposed for use in applications such as complex manipulation tasks or navigation in disaster scenarios. Safe interaction and ease of production promote the usage of this technology, but soft robots can be challenging to teleoperate due to their unique degrees of freedom. In this paper, we propose a human-centered interface that allows users to teleoperate a soft growing ro...
-
Restoring healthy kinematics is a critical component of assisting and rehabilitating impaired locomotion. Here we tested whether spatiotemporal gait patterns can be modulated by applying mechanical impedance to hip joints. Using the Samsung GEMS-H exoskeleton, we emulated a virtual spring (positive and negative) between the user's legs. We found that applying positive stiffness with the exoskeleto...
-
This paper presents the design and implementation of a controller for stair ascent and descent in a primarily-passive stance-controlled swing-assist (SCSA) prosthesis. The prosthesis and controller enable users to perform both step-over and step-to stair ascent and descent. The efficacy of the controller and SCSA prosthesis prototype in providing improved stair ambulation was tested on a unilatera...
-
Pneumatic transmission has several advantages in developing powered ankle foot orthosis (AFO) systems, such as the flexibility in placing pneumatic components for mass distribution and providing high back-drivability via simple valve control. However, pneumatic systems are generally tethered to large stationary air compressors that restrict them for being used as daily assistive devices. In this s...
-
Robotic prostheses provide new opportunities to better restore lost functions than passive prostheses for trans-femoral amputees. But controlling a prosthesis device automatically for individual users in different task environments is an unsolved problem. Reinforcement learning (RL) is a naturally promising tool. For prosthesis control with a user in the loop, it is desirable that the controlled p...
-
Development of a Twisted String Actuator-based Exoskeleton for Hip Joint Assistance in Lifting Tasks
This paper presents a study on a compliant cable-driven exoskeleton for hip assistance in lifting tasks that is aimed at preventing low-back pain and injuries in the vocational setting. In the proposed concept, we used twisted string actuator (TSA) to design a light-weight and powerful exoskeleton that benefits from inherent TSA advantages. We have noted that nonlinear nature of twisted strings’ t...
-
This paper presents a novel portable passive lower limb exoskeleton for walking assistance. The exoskeleton is designed with built-in spring mechanisms at the hip and knee joints to realize gravity balancing of the human leg. A pair of mating gears is used to convert the tension force from the built-in springs into balancing torques at hip and knee joints for overcoming the influence of gravity. S...
-
This paper investigates a burrowing robot that can maneuver and steer while being submerged in a granular medium. The robot locomotes using an internal vibro-impact mechanism and steers using a rotating bevel-tip head. We formulate and investigate a non-holonomic model for the steering mechanism and a hybrid dynamics model for the thrusting mechanism. We perform a numerical analysis of the dynamic...
-
This paper presents a novel physical gripping framework intended for controlled, high force density attachment on a range of surfaces. Our framework utilizes a light-activated chemical adhesive to attach to surfaces. The cured adhesive is part of a "sacrificial layer," which is shed when the gripper separates from the surface. In order to control adhesive behavior we utilize ultraviolet (UV) light...
-
In this paper, the optimization of the anchor points of a cable driven parallel robot (CDPR) for 3D printing is proposed in order to maximize the rigidity. Indeed, in the context of 3D printing, robot stiffness should guarantee a high level of tool path following accuracy. The optimized platform showed a rigidity improvement in simulation, but also experimentally with a first study of vibration mo...
-
This work presents a novel design concept that achieves multi-legged locomotion using a three-dimensional cam system. A computational framework has been developed to analyze and dimension this cam apparatus, that can perform arbitrary end effector motions within its design constraints. The mechanism enables continuous gait transition and inherent force compliance. With only two motors, any traject...
-
In order to accomplish various missions, autonomous ground vehicles must operate on a wide range of terrain. While many systems such as wheels and whegs can navigate some types of terrain, none are optimal across all. This creates a need for physical adaptation. This paper presents a broad new approach to physical adaptation that relies on manipulation. Specifically, we explore how multipurpose ma...
-
This paper focuses on designing, modeling and validating a novel scalable multicopter whose deformation mechanism, called SNIAE-SSE, relies on a combination of simple non-intersecting angulated elements (SNIAEs) and straight scissor-like elements (SSEs). The proposed SNIAE-SSE mechanism has the advantages of single degree-of-freedom, fast actuation capability and large deformation ratio. In this w...
-
Cooperative autonomy and data sharing can largely improve the mission performance of robotic networks in underwater surveillance applications. In this paper, we describe the cooperative autonomy used to control the Autonomous Underwater Vehicles (AUVs) acting as sonar receiver nodes in the CMRE Anti-Submarine Warfare (ASW) network. The paper focuses on a track management module that was integrated...
-
Battery life, reliability, and localization are prominent challenges in the design of autonomous underwater vehicles (AUVs). This work aims to address facets of these challenges using a single system. We describe the design of a bidirectional resonant pump that uses a single electromagnetic voice coil motor (VCM) capable of rotation around a central two degree-of-freedom flexure stage axis. This a...
-
We present an efficient approach for finding shortest paths in flow fields that vary as a sequence of flow predictions over time. This approach is applicable to motion planning for slow marine robots that are subject to dynamic ocean currents. Although the problem is NP-hard in general form, we incorporate recent results from the theory of finding shortest paths in time-dependent graphs to constru...
-
Navigation underwater traditionally is done by keeping a safe distance from obstacles, resulting in "fly-overs" of the area of interest. Movement of an autonomous underwater vehicle (AUV) through a cluttered space, such as a shipwreck or a decorated cave, is an extremely challenging problem that has not been addressed in the past. This paper proposes a novel navigation framework utilizing an enhan...
-
We present a deep residual network-based generative model for single image super-resolution (SISR) of underwater imagery for use by autonomous underwater robots. We also provide an adversarial training pipeline for learning SISR from paired data. In order to supervise the training, we formulate an objective function that evaluates the perceptual quality of an image based on its global content, col...
-
This paper presents a synchronization controller for mobile sensors that are minimally actuated and can only communicate with each other over a very short range. This work is motivated by ocean monitoring applications where large-scale sensor networks consisting of drifters with minimal actuation capabilities, i.e., active drifters, are employed. We assume drifters are tasked to monitor regions co...
-
This work presents the concept of energy-based safety for series-elastic actuation. Generic actuation passivity and safety is treated, defining several energy storage and power flow properties related to passivity. Safe behaviour is not guaranteed by passivity, but can be guaranteed by energy and power limits that adapt the nominal behaviour of an impedance controller. A discussion on power flows ...
-
In many series-elastic actuator applications, the ability to safely render a wide range of impedance is important. Advanced torque control techniques such as the disturbance observer (DOB) can improve torque tracking performance, but their impact on safe impedance range is not established. Here, safety is defined with load port passivity, and passivity conditions are developed for two variants of ...
-
Theory suggests an inverse relation between the stiffness and the energy storage capacity for linear helical springs: reducing the active length of the spring by 50% increases its stiffness by 100%, but reduces its energy storage capacity by 50%. State-of-the-art variable stiffness actuators used to drive robots are characterized by a similar inverse relation, implying reduced energy storage capac...
-
Structures with origami design enable objects to transform into various three-dimensional shapes. Traditionally origami structures are designed with zero-thickness flat paper sheets. However, the thickness and intersection of origami facets are non-negligible in most cases, uniquely when integrating origami design with robotic design because of the more efficient force transfer between thick plate...
-
In this study, we developed a real-time simulation method for non-deformable continuous tracks having grousers for rough terrain by explicitly considering the collision and friction between the tracks and the ground. In the proposed simulation method, an arbitrary trajectory of a track is represented with multiple linear and circular segments, each of which is a link connected to a robot body. The...
-
This paper presents an approach and introduces new open-source tools that can be used to evaluate robotic mapping algorithms. Also described is an extensive subterranean mine rescue dataset based upon the DARPA Subterranean (SubT) challenge including professionally surveyed ground truth. Finally, some commonly available approaches are evaluated using this metric.
-
Successful robot path planning is challenging in the presence of visual occlusions and moving targets. Classical methods to solve this problem have used visioning and perception algorithms in addition to partially observable markov decision processes to aid in path planning for pursuit-evasion and robot tracking. We present a predictive path planning process that measures and utilizes the uncertai...
-
As an external manifestation of human emotions, expression recognition plays an important role in human-computer interaction. Although existing expression recognition methods performs perfectly on constrained frontal faces, there are still many challenges in expression recognition in natural scenes due to different unrestricted conditions. Expression classification belongs to a pattern recognition...
-
Hand pose estimation with objects is challenging due to object occlusion and the lack of large annotated datasets. To tackle these issues, we propose an Augmented Autoencoder based deep learning method using augmented clean hand data. Our method takes 3D point cloud of a hand with an augmented object as input and encodes the input to latent representation of the hand. From the latent representatio...
-
While 2D object detection has made significant progress, robustly localizing objects in 3D space under presence of occlusion is still an unresolved issue. Our focus in this work is on real-time detection of human 3D centroids in RGB-D data. We propose an image-based detection approach which extends the YOLO v3 architecture with a 3D centroid loss and mid-level feature fusion to exploit complementa...
-
Monitoring of biosignals on a daily basis plays important roles for the health management of elderly. The monitoring system for the daily life, the system should not require the subjects to take special effort like wearing a sensor. We propose biosignals measurement using wide-range load sensor on the bed. The sensing system can detect the body weight, heartbeat and respiration simultaneously by j...
-
The paper presents a pedestrian near-miss detector with temporal analysis that provides both pedestrian detection and risk-level predictions which are demonstrated on a self-collected database. Our work makes three primary contributions: (i) The framework of pedestrian near-miss detection is proposed by providing both a pedestrian detection and risk-level assignment. Specifically, we have created ...
-
Place recognition is an important component for simultaneously localization and mapping in a variety of robotics applications. Recently, several approaches using landmark information to represent a place showed promising performance to address long-term environment changes. However, previous approaches do not explicitly consider changes of the landmarks, i,e., old landmarks may disappear and new o...
-
We present a new linear method for RGB-D based simultaneous localization and mapping (SLAM). Compared to existing techniques relying on the Manhattan world assumption defined by three orthogonal directions, our approach is designed for the more general scenario of the Atlanta world. It consists of a vertical direction and a set of horizontal directions orthogonal to the vertical direction and thus...
-
Stereo-vision devices have rigorous requirements for extrinsic parameter calibration. In Stereo Visual Inertial Odometry (VIO), inaccuracy in or changes to camera extrinsic parameters may lead to serious degradation in estimation performance. In this manuscript, we propose an online calibration method for stereo VIO extrinsic parameters correction. In particular, we focus on Multi-State Constraint...
-
We introduce a novel lidar-monocular visual odometry approach using point and line features. Compared to previous point-only based lidar-visual odometry, our approach leverages more environment structure information by introducing both point and line features into pose estimation. We provide a robust method for point and line depth extraction, and formulate the extracted depth as prior factors for...
-
Modern robotic systems sense the environment geometrically, through sensors like cameras, lidar, and sonar, as well as semantically, often through visual models learned from data, such as object detectors. We aim to develop robots that can use all of these sources of information for reliable navigation, but each is corrupted by noise. Rather than assume that object detection will eventually achiev...
-
Visual-inertial SLAM is essential for robot navigation in GPS-denied environments, e.g. indoor, underground. Conventionally, the performance of visual-inertial SLAM is evaluated with open-loop analysis, with a focus on the drift level of SLAM systems. In this paper, we raise the question on the importance of visual estimation latency in closed-loop navigation tasks, such as accurate trajectory tra...
-
Motivated by the success of encoding multi-scale contextual information for image analysis, we propose our PointAtrousGraph (PAG) - a deep permutation-invariant hierarchical encoder-decoder for efficiently exploiting multi-scale edge features in point clouds. Our PAG is constructed by several novel modules, such as Point Atrous Convolution (PAC), Edgepreserved Pooling (EP) and Edge-preserved Unpoo...
-
Following recent developments, this paper investigates the possibility to predict uncertainty models for monocular graph SLAM using topological features of the problem. An architecture to learn relative (i.e. inter-keyframe) uncertainty models using the resistance distance in the covisibility graph is presented. The proposed architecture is applied to simulated UAV coverage path planning trajector...
-
The ability to generate natural language explanations conditioned on the visual perception is a crucial step towards autonomous agents which can explain themselves and communicate with humans. While the research efforts in image and video captioning are giving promising results, this is often done at the expense of the computational requirements of the approaches, limiting their applicability to r...
-
Augmented Reality and mobile robots are gaining increased attention within industries due to the high potential to make processes cost and time efficient. To facilitate augmented reality, a calibration between the Augmented Reality device and the environment is necessary. This is a challenge when dealing with mobile robots due to the mobility of all entities making the environment dynamic. On this...
-
Deep reinforcement learning (RL) has enabled training action-selection policies, end-to-end, by learning a function which maps image pixels to action outputs. However, it's application to visuomotor robotic policy training has been limited because of the challenge of large-scale data collection when working with physical hardware. A suitable visuomotor policy should perform well not just for the t...
-
We address the problem of effectively composing skills to solve sparse-reward tasks in the real world. Given a set of parameterized skills (such as exerting a force or doing a top grasp at a location), our goal is to learn policies that invoke these skills to efficiently solve such tasks. Our insight is that for many tasks, the learning process can be decomposed into learning a state-independent t...
-
Unmanned aerial vehicles (UAVs) are frequently used for large-scale scene mapping and reconstruction. However, in most cases, drones are operated manually, which should be more effective and intelligent. In this article, we present a method of real-time UAV path planning for autonomous urban scene reconstruction. Considering the obstacles and time costs, we utilize the top view to generate the ini...
-
We present a novel fast marching gradient sampling strategy to accelerate the convergence speed of sampling-based motion planning algorithms. This strategy is based on an informed certificate set which consists of the robot states with exact collision status as well as the minimum distance and the gradient to the nearest obstacle. The informed certificate set covers almost the whole planning space...
-
During flights, an unmanned aerial vehicle (UAV) may not be allowed to move across certain areas due to soft constraints such as privacy restrictions. Current methods on self-adaption focus mostly on motion planning such that the trajectory does not trespass predetermined restricted areas. When the environment is cluttered with uncertain obstacles, however, these motion planning algorithms are not...
-
We present a simple method to quickly explore C-spaces of robotic manipulators and thus facilitate path planning. The method is based on a novel geometrical structure called generalized bur. It is a star-like tree, rooted at a given point in free C-space, with an arbitrary number of guaranteed collision-free edges computed using distance information from the workspace and simple forward kinematics...
-
We present a contact-implicit trajectory optimization framework that can plan contact-interaction trajectories for different robot architectures and tasks using a trivial initial guess and without requiring any parameter tuning. This is achieved by using a relaxed contact model along with an automatic penalty adjustment loop for suppressing the relaxation. Moreover, the structure of the problem en...
-
Gradient-based trajectory optimization (GTO) has gained wide popularity for quadrotor trajectory replanning. However, it suffers from local minima, which is not only fatal to safety but also unfavorable for smooth navigation. In this paper, we propose a replanning method based on GTO addressing this issue systematically. A path-guided optimization (PGO) approach is devised to tackle infeasible loc...
-
In this work we present a new methodology on learning-based path planning for autonomous exploration of subterranean environments using aerial robots. Utilizing a recently proposed graph-based path planner as a "training expert" and following an approach relying on the concepts of imitation learning, we derive a trained policy capable of guiding the robot to autonomously explore underground mine d...
-
This paper presents a novel telepresence system for enhancing aerial manipulation capabilities. It involves not only a haptic device, but also a virtual reality that provides a 3D visual feedback to a remotely-located teleoperator in real-time. We achieve this by utilizing onboard visual and inertial sensors, an object tracking algorithm and a pregenerated object database. As the virtual reality h...
-
The RVM (Robot-based Vibration Suppression Modules) is proposed for the manipulation and transport of a large flexible object. Since the RVM is easily attachable/detachable to the object, this RVM allows distributing over the manipulated object so that it is scalable to the object size. The composition of the system is partly motivated by the MAGMaS (Multiple Aerial-Ground Manipulator System) [1]-...
-
Existing studies for environment interaction with an aerial robot have been focused on interaction with static surroundings. However, to fully explore the concept of an aerial manipulation, interaction with moving structures should also be considered. In this paper, a multirotor-based aerial manipulator opening a daily-life moving structure, a hinged door, is presented. In order to address the con...
-
This work suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchical chasing planner. Leveraging covariant optimization, the prediction module estimates the future mo...
-
FG-GMM-based Interactive Behavior Estimation for Autonomous Driving Vehicles in Ramp Merging Control
Interactive behavior is important for autonomous driving vehicles, especially for scenarios like ramp merging which require significant social interaction between autonomous driving vehicles and human-driven cars. This paper enhances our previous Probabilistic Graphical Model (PGM) merging control model for the interactive behavior of autonomous driving vehicles. To better estimate the interactive...
-
Fully autonomous vehicles are expected to share the road with less advanced vehicles for a significant period of time. Furthermore, an increasing number of vehicles on the road are equipped with a variety of low-fidelity sensors which provide some perception and localization data, but not at a high enough quality for full autonomy. In this paper, we develop a perception and localization system tha...
-
We present a method for learning to drive on smooth terrain while simultaneously avoiding collisions in challenging off-road and unstructured outdoor environments using only visual inputs. Our approach applies a hybrid model-based and model-free reinforcement learning method that is entirely self-supervised in labeling terrain roughness and collisions using on-board sensors. Notably, we provide bo...
-
We present a realtime tracking algorithm, Road-Track, to track heterogeneous road-agents in dense traffic videos. Our approach is designed for dense traffic scenarios that consist of different road-agents such as pedestrians, two-wheelers, cars, buses, etc. sharing the road. We use the tracking-by-detection approach where we track a road-agent by matching the appearance or bounding box region in t...
-
Multilateration systems reconstruct the location of a target that transmits electromagnetic or acoustic signals. The employed measurements for localization are the times of arrival (TOAs) of the transmitted signal, measured by a number of spatially distributed receivers at known positions. We present a novel multilateration algorithm to localize multiple targets that transmit indistinguishable sig...
-
When robots move autonomously for long-term, varied appearance such as the transition from day to night and seasonal variation brings challenges to visual place recognition. Defining an appearance condition (e.g. a season, a kind of weather) as a domain, we consider that the desired representation for place recognition (i) should be domain-unrelated so that images from different time can be matche...
-
Estimating pose from given 3D correspondences, including point-to-point, point-to-line and point-to-plane correspondences, is a fundamental task in computer vision with many applications. We present a fast and accurate solution for the least-squares problem of this task. Previous works mainly focus on studying the way to find the global minimizer of the least-squares problem. However, existing wor...
-
Ground texture based localization is a promising approach to achieve high-accuracy positioning of vehicles. We present a self-contained method that can be used for global localization as well as for subsequent local localization updates, i.e. it allows a robot to localize without any knowledge of its current whereabouts, but it can also take advantage of a prior pose estimate to reduce computation...
-
Reliable data association represents a main challenge of feature-based vehicle localization and is the key to integrity of localization. Independent of the type of features used, incorrect associations between detected and mapped features will provide erroneous position estimates. Only if the uniqueness of a local environment is represented by the features that are stored in the map, the reliabili...
-
An essential measure of autonomy in assistive service robots is adaptivity to the various contexts of human-oriented tasks, which are subject to subtle variations in task parameters that determine optimal behaviour. In this work, we propose an apprenticeship learning approach to achieving context-aware action generalization on the task of robot-to-human object hand-over. The procedure combines lea...
-
We propose a novel hierarchical online learning scheme for fast and efficient bootstrapping of sensorimotor skills. Our scheme permits rapid data-driven robot model learning in a "learning while behaving" fashion. It is updated continuously to adapt to time-dependent changes and driven by an intrinsic motivation signal. It utilizes an online associative radial basis function network, which is the ...
-
To be effective in unstructured and changing environments, robots must learn to recognize new objects. Deep learning has enabled rapid progress for object detection and segmentation in computer vision; however, this progress comes at the price of human annotators labeling many training examples. This paper addresses the problem of extending learning-based segmentation methods to robotics applicati...
-
Annotated datasets are essential for supervised learning. However, annotating large datasets is a tedious and time-intensive task. This paper addresses active learning in the context of semantic segmentation with the goal of reducing the human labeling effort. Our application is agricultural robotics and we focus on the task of distinguishing between crop and weed plants from image data. A key cha...
-
In this paper, we propose a framework to build a memory of motion for warm-starting an optimal control solver for the locomotion task of a humanoid robot. We use HPP Loco3D, a versatile locomotion planner, to generate offline a set of dynamically consistent whole-body trajectory to be stored as the memory of motion. The learning problem is formulated as a regression problem to predict a single-ste...
-
We present a novel approach to control design for nonlinear systems which leverages model-free policy optimization techniques to learn a linearizing controller for a physical plant with unknown dynamics. Feedback linearization is a technique from nonlinear control which renders the input-output dynamics of a nonlinear plant linear under application of an appropriate feedback controller. Once a lin...
-
Surgical gesture recognition is important for surgical data science and computer-aided intervention. Even with robotic kinematic information, automatically segmenting surgical steps presents numerous challenges because surgical demonstrations are characterized by high variability in style, duration and order of actions. In order to extract discriminative features from the kinematic signals and boo...
-
Most current surgical robotic systems lack the ability to sense tool/tissue interaction forces, which motivates research in methods to estimate these forces from other available measurements, primarily joint torques. These methods require the internal joint torques, due to the robot inverse dynamics, to be subtracted from the measured joint torques. This paper presents the use of neural networks t...
-
Dynamic traversal of uneven terrain is a major objective in the field of legged robotics. The most recent model predictive control approaches for these systems can generate robust dynamic motion of short duration; however, planning over a longer time horizon may be necessary when navigating complex terrain. A recently-developed framework, Trajectory Optimization for Walking Robots (TOWR), computes...
-
Simplified models are useful to increase the computational efficiency of a motion planning algorithm, but their lack of accuracy have to be managed. We propose two feasibility constraints to be included in a Single Rigid Body Dynamics-based trajectory optimizer in order to obtain robust motions in challenging terrain. The first one finds an approximate relationship between joint-torque limits and ...
-
Hybrid legged-wheeled robots such as the CEN-TAURO, are capable of varying their footprint polygon to carry out various agile motions. This property can be advantageous for wheeled-only planning in cluttered spaces, which is our focus. In this paper, we present an improved algorithm that builds upon our previously introduced preliminary footprint varying A* planner, which was based on the rectangu...
-
This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian imp...
-
To date, many control strategies for legged robots have been proposed for stable locomotion over rough and unstructured terrains. However, these approaches require sensing information throughout locomotion, which may be noisy or unavailable at times. An alternative solution to rough terrain locomotion is a legged robot design that can passively adapt to the variations in the terrain without requir...
-
We propose a method to generate actuation plans for a reduced order, dynamic model of bipedal running. This method explicitly enforces robustness to ground uncertainty. The plan generated is not a fixed body trajectory that is aggressively stabilized: instead, the plan interacts with the passive dynamics of the reduced order model to create emergent robustness. The goal is to create plans for legg...
-
Path planning is important for robot action execution, since a path or a motion trajectory for a particular action has to be defined first before the action can be executed. Most of the current approaches are iterative methods where the trajectory is generated by predicting the next state based on the current state. Here we propose a novel method by utilising a fully convolutional neural network, ...
-
Many problems in robotics involve multiple decision making agents. To operate efficiently in such settings, a robot must reason about the impact of its decisions on the behavior of other agents. Differential games offer an expressive theoretical framework for formulating these types of multi-agent problems. Unfortunately, most numerical solution techniques scale poorly with state dimension and are...
-
This paper introduces a novel approach for model predictive control of ballbots for path-following tasks. Ballbots are dynamically unstable mobile robots which are designed to balance on a single ball. The model presented in this paper is a simplied version of a full quaternion-based model of ballbots' underactuated dynamics which is suited for online implementation. Furthermore, the approach is e...
-
Despite their abundance in robotics and nature, underactuated systems remain a challenge for control engineering. Trajectory optimization provides a generally applicable solution, however its efficiency strongly depends on the skill of the engineer to frame the problem in an optimizer-friendly way. This paper proposes a procedure that automates such problem reformulation for a class of tasks in wh...
-
In this paper, we describe a planner capable of generating walking trajectories by using the centroidal dynamics and the full kinematics of a humanoid robot model. The interaction between the robot and the walking surface is modeled explicitly through a novel contact parametrization. The approach is complementarity-free and does not need a predefined contact sequence. By solving an optimal control...
-
Hybrid wheel-legged robots have begun to demonstrate the ability to adapt to complex terrain traditionally inaccessible to purely wheeled morphologies. Further research is needed into how their dynamics can be optimally controlled for developing highly adaptive behaviours on challenging terrain. Using optimal center of mass (COM) kinematic trajectories, this work examines the nonlinear dynamics co...
-
While many robotic tasks, like manipulation and locomotion, are fundamentally based in making and breaking contact with the environment, state-of-the-art control policies struggle to deal with the hybrid nature of multi-contact motion. Such controllers often rely heavily upon heuristics or, due to the combinatoric structure in the dynamics, are unsuitable for real-time control. Principled deployme...
-
Motivated by the stringent requirements of unstructured real-world where a plethora of unknown objects reside in arbitrary locations of the surface, we propose a voxel-based deep 3D Convolutional Neural Network (3D CNN) that generates feasible 6-DoF grasp poses in unrestricted workspace with reachability awareness. Unlike the majority of works that predict if a proposed grasp pose within the restr...
-
In this paper, enhancement to the novel grasp planning algorithm based on gripper workspace spheres is presented. Our development requires a registered point cloud of the target from different views, assuming no prior knowledge of the object, nor any of its properties. This work features a new set of metrics for grasp pose candidates evaluation, as well as exploring the impact of high object sampl...
-
Robot grasping of deformable hollow objects such as plastic bottles and cups is challenging, as the grasp should resist disturbances while minimally deforming the object so as not to damage it or dislodge liquids. We propose minimal work as a novel grasp quality metric that combines wrench resistance and object deformation. We introduce an efficient algorithm to compute the work required to resist...
-
In this paper, we tackle the problem of 6-DoF grasp detection which is crucial for robot grasping in cluttered real-world scenes. Unlike existing approaches which synthesize 6-DoF grasp data sets and train grasp quality networks with input grasp representations based on point clouds, we rather take a novel hierarchical approach which does not use any 6-DoF grasp data. We cast the 6-DoF grasp detec...
-
This paper considers basket grasps, where a two-finger robot hand forms a basket that can safely lift and carry rigid objects in a 2-D gravitational environment. The two-finger basket grasps form special points in a high-dimensional configuration space of the object and two-finger robot hand. This paper establishes that all two-finger basket grasps can be found in a low-dimensional contact space t...
-
We present a novel, robust sound source localization algorithm considering back-propagation signals. Sound propagation paths are estimated by generating direct and reflection acoustic rays based on ray tracing in a backward manner. We then compute the back-propagation signals by designing and using the impulse response of the backward sound propagation based on the acoustic ray paths. For identify...
-
Many species have evolved advanced non-visual perception while artificial systems fall behind. Radar and ultrasound complement camera-based vision but they are often too costly and complex to set up for very limited information gain. In nature, sound is used effectively by bats, dolphins, whales, and humans for navigation and communication. However, it is unclear how to best harness sound for mach...
-
The sound source separation problem has many useful applications in the field of robotics, such as human-robot interaction, scene understanding, etc. However, it remains a very challenging problem. In this paper, we utilize both visual and audio information of videos to perform the sound source separation task. A self-supervised learning framework is proposed to implement the object detection and ...
-
Despite the utility and benefits of omnidirectional images in robotics and automotive applications, there are no datasets of omnidirectional images available with semantic segmentation, depth map, and dynamic properties. This is due to the time cost and human effort required to annotate ground truth images. This paper presents a framework for generating omnidirectional images using images that are...
-
Electrical resistance tomography (ERT) has previously been utilized to develop a large-scale tactile sensor because this approach enables the estimation of the conductivity distribution among the electrodes based on a known physical model. Such a sensor made with a stretchable material can conform to a curved surface. However, this sensor cannot fully cover a cylindrical surface because in such a ...
-
Accurate force sensing is important for endoluminal intervention in terms of both safety and lesion targeting. This paper develops an FBG-based force sensor for robotic bronchoscopy by configuring three FBG sensors at the lateral side of a conical substrate. It allows a large and eccentric inner lumen for the interventional instrument, enabling a flexible imaging probe inside to perform optical bi...
-
Tactile sensors based on electrical resistance tomography (ERT) have shown many advantages for implementing a soft and scalable whole-body robotic skin; however, calibration is challenging because pressure reconstruction is an ill-posed inverse problem. This paper introduces a method for calibrating soft ERT-based tactile sensors using sim-to-real transfer learning with a finite element multiphysi...
-
Deep learning and reinforcement learning methods have been shown to enable learning of flexible and complex robot controllers. However, the reliance on large amounts of training data often requires data collection to be carried out in simulation, with a number of sim-to-real transfer methods being developed in recent years. In this paper, we study these techniques for tactile sensing using the Tac...
-
This paper presents a semi-empirical method for simulating contact with elastically deformable objects whose force response is learned using entirely data-driven models. A point-based surface representation and an inhomogeneous, nonlinear force response model are learned from a robotic arm acquiring force-displacement curves from a small number of poking interactions. The simulator then estimates ...
-
Commercial six-axis force-torque sensors suffer from being some combination of expensive, fragile, and hard-touse. We propose a new fiducial-based design which addresses all three points. The sensor uses an inexpensive webcam and can be fabricated using a consumer-grade 3D printer. Open-source software is used to estimate the 3D pose of the fiducials on the sensor, which is then used to calculate ...
-
Modern vehicles are often equipped with a surround-view multi-camera system. The current interest in autonomous driving invites the investigation of how to use such systems for a reliable estimation of relative vehicle displacement. Existing camera pose algorithms either work for a single camera, make overly simplified assumptions, are computationally expensive, or simply become degenerate under n...
-
Topological strategies for navigation meaningfully reduce the space of possible actions available to a robot, allowing use of heuristic priors or learning to enable computationally efficient, intelligent planning. The challenges in estimating structure with monocular SLAM in low texture or highly cluttered environments have precluded its use for topological planning in the past. We propose a robus...
-
Vision-based path following allows robots to autonomously repeat manually taught paths. Stereo Visual Teach and Repeat (VT&R) [1] accomplishes accurate and robust long-range path following in unstructured outdoor environments across changing lighting, weather, and seasons by relying on colour-constant imaging [2] and multi-experience localization [3]. We leverage multi-experience VT&R together wit...
-
Learning-based visual navigation still remains a challenging problem in robotics, with two overarching issues: how to transfer the learnt policy to unseen scenarios, and how to deploy the system on real robots. In this paper, we propose a deep neural network based visual navigation system, SnapNav. Unlike map-based navigation or Visual-Teach-and-Repeat (VT&R), SnapNav only receives a few snapshots...
-
We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS-Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial od...
-
Visual navigation tasks in real-world environments often require both self-motion and place recognition feedback. While deep reinforcement learning has shown success in solving these perception and decision-making problems in an end-to-end manner, these algorithms require large amounts of experience to learn navigation policies from high-dimensional data, which is generally impractical for real ro...
-
If robots and humans are to coexist and cooperate in society, it would be useful for robots to be able to engage in tactile interactions. Touch is an intuitive communication tool as well as a fundamental method by which we assist each other physically. Tactile abilities are challenging to engineer in robots, since both mechanical safety and sensory intelligence are imperative. Existing work reveal...
-
This paper presents a soft robotic ankle-foot orthosis (SR-AFO) exosuit designed to provide support to the human ankle in the frontal plane without restricting natural motion in the sagittal plane. The SR-AFO exosuit incorporates inflatable fabric-based actuators with a hollow cylinder design which requires less volume than the commonly used solid cylinder design for the same deflection. The actua...
-
There are limitations of conventional active-assistive control for upper limb rehabilitation exoskeleton robot, such as 1). prior time-dependent trajectories are generally required, 2). task-based rehabilitation exercise involving multi-joint motion is hard to implement, and 3). assistive mechanism normally is so inflexible that the resulting exercise performed by the subjects becomes inefficient....
-
Robot rehabilitation is an emerging and promising topic that incorporates robotics with neuroscience and rehabilitation to define new methods for supporting patients with neurological diseases. As a consequence, the rehabilitation process could increase the efficacy exploiting the potentialities of robot-mediated therapies. Nevertheless, nowadays clinical effectiveness is not enough to widely intr...
-
Impedance controllers are popularly used in the field of lower limb prostheses and exoskeleton development. Such controllers assume the joint to be a spring-damper system described by a discrete set of equilibria and impedance parameters. These parameters are estimated via a least squares optimization that minimizes the difference between the controller's output torque and human joint torque. Othe...
-
Since the uprising of new biomedical orthotic devices, exoskeletons have been put in the spotlight for their possible use in rehabilitation. Even if these products might share some commonalities among them in terms of overall structure, degrees of freedom and possible actions, they quite often differ in their approach on how to generate a feasible, stable and comfortable gait trajectory pattern. T...
-
As robot autonomy improves, robots are increasingly being considered in the role of autonomous observation systems - free-flying cameras capable of actively tracking human activity within some predefined area of interest. In this work, we formulate the autonomous observation problem through multi-objective optimization, presenting a novel Semi-MDP formulation of the autonomous human observation pr...
-
Human movement prediction is difficult as humans naturally exhibit complex behaviors that can change drastically from one environment to the next. In order to alleviate this issue, we propose a prediction framework that decouples short-term prediction, linked to internal body dynamics, and long-term prediction, linked to the environment and task constraints. In this work we investigate encoding sh...
-
This paper presents a method to incorporate ergonomics into the optimization of action sequences for bi-manual human-robot cooperation tasks with continuous physical interaction. Our first contribution is a novel computational model of the human that allows prediction of an ergonomics assessment corresponding to each step in a task. The model is learned from human motion capture data in order to p...
-
We present a novel POMDP problem formulation for a robot that must autonomously decide where to go to collect new and scientifically relevant images given a limited ability to communicate with its human operator. From this formulation we derive constraints and design principles for the observation model, reward model, and communication strategy of such a robot, exploring techniques to deal with th...
-
Unlike robots, humans do not have constant movements. Their pathways are individually changeable and influenced by circumstances. This paper presents a method to investigate human pathway variations in a real study. In systematically selected tasks, human pathways are examined for 100 participants in manual and human-robot collaboration (HRC) scenarios. As a result, the variations of pathways are ...
-
The supercoiled polymer (SCP) actuator is a novel artificial muscle, which is manufactured by twisting and coiling polymer fibers. This new artificial muscle is soft, low-cost and shows good linearity. Being utilized as an actuator, the artificial muscle could generate significant mechanical power in a muscle-like form upon electrical activation by Joule heating. In this study, we adopt this new a...
-
Fusions of LiDARs (light detection and ranging) and cameras have been effectively and widely employed in the communities of autonomous vehicles, virtual reality and mobile mapping systems (MMS) for different purposes, such as localization, high definition map or simultaneous location and mapping. However, the extrinsic calibration between a camera and a 3D LiDAR is a fundamental prerequisite to gu...
-
As a crucial step of sensor data fusion, sensor calibration plays a vital role in many cutting-edge machine vision applications, such as autonomous vehicles and AR/VR. Existing techniques either require quite amount of manual work and complex settings, or are unrobust and prone to produce suboptimal results. In this paper, we investigate the extrinsic calibration of an RGB camera and a light detec...
-
This work provides a framework for the 3D calibration of wafers and a wafer handling robot by monocular vision. The proposed method precisely reconstructs the 3D poses of wafers from a set of images captured by the camera mounted on the robot. In addition, it calibrates the robot kinematics simultaneously. A robust ellipse detection and tracking algorithm based on the edge arcs is developed to rec...
-
Taking selfies has become a photographic trend nowadays. We envision the emergence of the "video selfie" capturing a short continuous video clip (or burst photography) of the user, themselves. A selfie stick is usually used, whereby a camera is mounted on a stick for taking selfie photos. In this scenario, we observe that the camera typically goes through a special trajectory along a sphere surfac...
-
The increasing availability of surround-view camera systems in passenger vehicles motivates their use as an exterior perception modality for intelligent vehicle behaviour. An important problem within this context is the extrinsic calibration between the cameras, which is challenging due to the often reduced overlap between the fields of view of neighbouring views. Our work is motivated by two insi...
-
Self-diagnosis and self-repair are some of the key challenges in deploying robotic platforms for long-term real-world applications. One of the issues that can occur to a robot is miscalibration of its sensors due to aging, environmental transients, or external disturbances. Precise calibration lies at the core of a variety of applications, due to the need to accurately perceive the world. However,...
-
The automatic parts feeding of multiple objects is an unsolved problem in the manufacturing industry. In this paper, we tackle the problem by proposing a multi-robot system. The system comprises three sub-components which perform bin-picking, regrasping, and kitting. The three subcomponents divide and conquer the automatic multiple parts feeding problem by considering a coarse-to-fine manipulation...
-
We consider the task of autonomously unloading boxes from trucks using an industrial manipulator robot. There are multiple challenges that arise: (1) real-time motion planning for a complex robotic system carrying two articulated mechanisms, an arm and a scooper, (2) decision-making in terms of what action to execute next given imperfect information about boxes such as their masses, (3) accounting...
-
The latency in vision-based sensor systems used in human-robot collaborative environments is an important safety parameter which in most cases has been neglected by researchers. The main reason for this neglect is the lack of an accurate ground-truth sensor system with a minimal delay to benchmark the vision-sensors against. In this paper the latencies of 3D vision-based sensors are experimentally...
-
In industry assembly lines, parts feeding machines are widely employed as the prologue of the whole procedure. They play the role of sorting the parts randomly placed in bins to the state with specified pose. With the help of the parts feeding machines, the subsequent assembly processes by robot arm can always start from the same condition. Thus it is expected that function of parting feeding mach...
-
In this paper we propose an algorithm to actively control the distance of a motor-propeller system (MPS) to a large obstacle using data from a single microphone. The method is based upon a broadband constructive/destructive interference pattern across the audible frequency band that is present when the MPS is near an obstacle. By taking the difference between the power spectrum in the obstacle-fre...
-
This paper proposes a new kind of light-weight manipulators suitable for safe interactions. The proposed manipulators use anti-parallelogram joints in series, referred to as X-joints. Each X-joint is remotely actuated with cables and springs in parallel, thus realizing a tensegrity one-degree-of-freedom mechanism. As compared to manipulators built with simple revolute joints in series, manipulator...
-
This paper describes a new type of bending module inspired, in part, by the musculoskeletal structure of the lobster leg joint. The bending module proposed combines enhanced torque output, reconfigurability in assembling, safe compliant actuation, and accurate control on its mechanical performance. In this module, antagonistic soft chambers are enveloped by exoskeleton shells, and the bending angl...
-
This paper presents a soft-robotic platform for exploring the ecological relevance of non-locomotory movements via animal-robot interactions. Coral snakes (genus Micrurus) and their mimics use vigorous, non-locomotory, and arrhythmic thrashing to deter predation. There is variation across snake species in the duration and curvature of anti-predator thrashes, and it is unclear how these aspects of ...
-
The directional mechanical impedance of the human ankle was identified from subjects in a standing posture with varying levels of muscle activity. The impedance modeled the different torque responses to angle perturbations about different axes of rotation. This work proposed a novel impedance model that incorporated the coupling between multiple degrees of freedom of the ankle and was validated th...
-
Legged systems need to optimize contact force in order to maintain contacts. For this, the controller needs to have the knowledge of the surface geometry and how slippery the terrain is. We can use a vision system to realize the terrain, but the accuracy of the vision system degrades in harsh weather, and it cannot visualize the terrain if it is covered with water or grass. Also, the degree of fri...
-
The goal of this paper is to present a method for simultaneous trajectory and local stabilizing policy optimization to generate local policies for trajectory-centric model-based reinforcement learning (MBRL). This is motivated by the fact that global policy optimization for non-linear systems could be a very challenging problem both algorithmically and numerically. However, a lot of robotic manipu...
-
In this paper, we propose a method for generating undulatory gaits for snake robots. Instead of starting from a pre-defined movement pattern such as a serpenoid curve, we use a Model Predictive Control (MPC) approach to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of this approach is that the resulting gaits are automatically adapted to the ...
-
With the increase in use of Unmanned Aerial Vehicles (UAVs)/drones, it is important to detect and identify causes of failure in real time for proper recovery from a potential crash-like scenario or post incident forensics analysis. The cause of crash could be either a fault in the sensor/actuator system, a physical damage/attack, or a cyber attack on the drone's software. In this paper, we propose...
-
Rapid and reliable robot bin picking is a critical challenge in automating warehouses, often measured in picks-per-hour (PPH). We explore increasing PPH using faster motions based on optimizing over a set of candidate grasps. The source of this set of grasps is two-fold: (1) grasp-analysis tools such as Dex-Net generate multiple candidate grasps, and (2) each of these grasps has a degree of freedo...
-
This paper presents a cooperative robotic team composed of unmanned ground vehicles (UGVs) with hybrid operational modes to tackle the multiple traveling salesman problem (mTSP) with obstacles. The hybrid operational modes allow every UGV in the team to not only travel on a ground surface but also jump over obstacles. We name these UGVs jumping rovers. The jumping capability provides a flexible fo...
-
Multi-view stereo (MVS) algorithms have been commonly used to model large-scale structures. When processing MVS, image acquisition is an important issue because its reconstruction quality depends heavily on the acquired images. Recently, an explore-then-exploit strategy has been used to acquire images for MVS. This method first constructs a coarse model by exploring an entire scene using a pre-all...
-
This paper presents Reoriented Short-Cuts (RSC): A modification of the traditional Short-Cut technique, allowing almost sure, single homotopy class, asymptotic convergence in high degree of freedom (DoF) problems. An additional Informed Gaussian Sampling (IGS) technique is also introduced for convergence comparison. Traditionally, Short-Cut methods are used as a final technique to further optimize...
-
Deep reinforcement learning has great potential to acquire complex, adaptive behaviors for autonomous agents automatically. However, the underlying neural network polices have not been widely deployed in real-world applications, especially in these safety-critical tasks (e.g., autonomous driving). One of the reasons is that the learned policy cannot perform flexible and resilient behaviors as trad...
-
Agile fixed-wing aircraft integrate the efficient, high-speed capabilities of conventional fixed-wing platforms with the extreme maneuverability of rotorcraft. This work presents a nonlinear control strategy that harnesses these capabilities to enable autonomous flight through aggressive, time-constrained, three-dimensional trajectories. The cascaded control structure consists of two parts; an inn...
-
Fixed-wing vertical take-off and landing (VTOL) aircraft pose a unique control challenge that stems from complex aerodynamic interactions between wings and rotors. Thus, accurate estimation of external forces is indispensable for achieving high performance flight. In this paper, we present a composite adaptive nonlinear tracking controller for a fixed- wing VTOL. The method employs online adaptati...
-
This paper overviews the system design, modeling and control of the Aerial Robotic Chain. This new design corresponds to a reconfigurable robotic system of systems consisting of multilinked micro aerial vehicles that presents the ability to cross narrow sections, morph its shape, ferry significant payloads, offer the potential of distributed sensing and processing, and enable system extendability....
-
In this paper we propose to control a quadrotor through direct acceleration feedback. The proposed method, while simple in form, alleviates the need for accurate estimation of platform parameters such as mass and propeller effectiveness. In order to use efficaciously the noisy acceleration measurements in direct feedback, we propose a novel regression-based filter that exploits the knowledge on th...
-
This work presents a method to control omnidirectional micro aerial vehicles (OMAVs) for the tracking of 6-DoF trajectories in free space. A rigid body model based approach is applied in a receding horizon fashion to generate optimal wrench commands that can be constrained to meet limits given by the mechanical design and actuators of the platform. Allocation of optimal actuator commands is perfor...
-
This paper presents a design of oscillation damping control for the cable-Suspended Aerial Manipulator (SAM). The SAM is modeled as a double pendulum, and it can generate a body wrench as a control action. The main challenge is the fact that there is only one onboard IMU sensor which does not provide full information on the system state. To overcome this difficulty, we design a controller motivate...
-
TUNERCAR is a toolchain that jointly optimizes racing strategy, planning methods, control algorithms, and vehicle parameters for an autonomous racecar. In this paper, we detail the target hardware, software, simulators, and systems infrastructure for this toolchain. Our methodology employs a parallel implementation of CMA-ES which enables simulations to proceed 6 times faster than real-world rollo...
-
Risk assessment to quantify the danger associated with taking a certain action is critical to navigating safely through crowded urban environments during autonomous driving. Risk assessment and subsequent planning is usually done by first tracking and predicting trajectories of other agents, such as vehicles and pedestrians, and then choosing an action to avoid future collisions. However, few exis...
-
This paper presents an approach for implementing game theoretic decision making in combination with realistic sensory data input so as to allow an autonomous vehicle to perform maneuvers, such as lane change or merge in high traffic scenarios. The main novelty of this work, is the use of realistic sensory data input to obtain the observations as input of an iterative multi-player game in a realist...
-
Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by le...
-
In this paper, we consider a two-player racing game, where an autonomous ego vehicle has to be controlled to race against an opponent vehicle, which is either autonomous or human-driven. The approach to control the ego vehicle is based on a Sensitivity-ENhanced NAsh equilibrium seeking (SENNA) method, which uses an iterated best response algorithm in order to optimize for a trajectory in a two-car...
-
Visual place recognition is an important problem in mobile robotics which aims to localize a robot using image information alone. Recent methods have shown promising results for place recognition under varying environmental conditions by exploiting the sequential nature of the image acquisition process. We show that by using k nearest neighbours based image retrieval as the backend, and exploiting...
-
Globally localizing in a given map is a crucial ability for robots to perform a wide range of autonomous navigation tasks. This paper presents OneShot - a global localization algorithm that uses only a single 3D LiDAR scan at a time, while outperforming approaches based on integrating a sequence of point clouds. Our approach, which does not require the robot to move, relies on learning-based descr...
-
Traditional robotic control suits require profound task-specific knowledge for designing, building and testing control software. The rise of Deep Learning has enabled end-to-end solutions to be learned entirely from data, requiring minimal knowledge about the application area. We design a learning scheme to train end-to-end linear dynamical systems (LDS)s by gradient descent in imitation learning ...
-
Mini-Batched Online Incremental Learning Through Supervisory Teleoperation with Kinesthetic Coupling
We propose an online incremental learning approach through teleoperation which allows an operator to partially modify a learned model, whenever it is necessary, during task execution. Compared to conventional incremental learning approaches, the proposed approach is applicable for teleoperation-based teaching and it needs only partial demonstration without any need to obstruct the task execution. ...
-
Lower leg prostheses could improve the life quality of amputees by increasing comfort and reducing energy to locomote, but currently control methods are limited in modulating behaviors based upon the human's experience. This paper describes the first steps toward learning complex controllers for dynamical robotic assistive devices. We provide the first example of behavioral cloning to control a po...
-
Imitation learning enables robots to learn a task by simply watching the demonstration of the task. Current imitation learning methods usually require the learner and demonstrator to occur in the same context. This limits their scalability to practical applications. In this paper, we propose a more general imitation learning method which allows the learner and the demonstrator to come from differe...
-
Learning Assistance by Demonstration (LAD) is concerned with using demonstrations of a human agent to teach a robot how to assist another human. The concept has previously been used with smart wheelchairs to provide customised assistance to individuals with driving difficulties. A basic premise of this technique is that the learned assistive policy should be able to generalise to environments diff...
-
Autonomous vehicles are sharing the road with human drivers. In order to facilitate interactive driving and cooperative behavior in dense traffic, a thorough understanding and representation of other traffic participants' behavior are necessary. Cost functions (or reward functions) have been widely used to describe the behavior of human drivers since they can not only explicitly incorporate the ra...
-
A button battery accidentally ingested by a toddler or small child can cause severe damage to the stomach within a short period of time. Once a battery lands on the surface of the esophagus or stomach, it can run a current in the tissue and induce a chemical reaction resulting in injury. Following our previous work where we presented an ingestible magnetic robot for button battery retrieval, this ...
-
This paper reports the development of a fully actuated body-mounted robotic assistant for MRI-guided low back pain injection. The robot is designed with a 4-DOF needle alignment module and a 2-DOF remotely actuated needle driver module. The 6-DOF fully actuated robot can operate inside the scanner bore during imaging; hence, minimizing the need of moving the patient in or out of the scanner during...
-
It is known that the interior of the human body is one of the most adverse environments for a foreign body, such as an in-vivo robot, and vice-versa. As robots operating in-vivo are increasingly recognized for their capabilities and potential for improved therapies, it is important to ensure their safety, especially for long term treatments when little supervision can be provided. We introduce an ...
-
In this paper, we evaluate the effect of increasing camera baselines on depth perception in robot-assisted surgery. Restricted by the diameter of the surgical trocar through which they are inserted, current clinical stereo endoscopes have a fixed baseline of 5.5 mm. To overcome this restriction, we propose using a stereoscopic "pickup" camera with a side-firing design that allows for larger baseli...
-
Robotic automation has the potential to assist human surgeons in performing suturing tasks in microsurgery, and in order to do so a robot must be able to guide a needle with sub-millimeter precision through soft tissue. This paper presents a robotic suturing system that uses 3D optical coherence tomography (OCT) system for imaging feedback. Calibration of the robot-OCT and robot-needle transforms,...
-
Wireless Capsule Endoscopy (WCE) has the advantage of reducing the invasiveness and pain of gastrointestinal examinations. In this work, we propose a system aimed at autonomously accelerating and locating the WCE inside the intestine for clinical applications. A rotating magnet controlled by a robotic arm is placed outside the patient's body to actuate the capsule with an internal magnetic ring, a...
-
Multi-legged robots are effective at traversing rough terrain. However, terrains that include collapsible footholds (i.e. regions that can collapse when stepped on) remain a significant challenge, especially since such situations can be extremely difficult to anticipate using only exteroceptive sensing. State-of-the-art methods typically use various stabilisation techniques to regain balance and c...
-
Multi-robot systems of increasing size and complexity are used to solve large-scale problems, such as area exploration and search and rescue. A key decision in human-robot teaming is dividing a multi-robot system into teams to address separate issues or to accomplish a task over a large area. In order to address the problem of selecting teams in a multi-robot system, we propose a new multimodal gr...
-
We present a multi-scale forward search algorithm for distributed agents to solve single-query shortest path planning problems. Each agent first builds a representation of its own search space of the common environment as a multi-resolution graph, it communicates with the other agents the result of its local search, and it uses received information from other agents to refine its own graph and upd...
-
Connectivity maintenance is an essential aspect to consider while controlling a multi-robot system. In general, a multi-robot system should be connected to obtain a certain common objective. Connectivity must be kept regardless of the control strategy or the objective of the multi-robot system. Two main methods exist for connectivity maintenance: keep the initial connections (local connectivity) o...
-
The interplay between network topology and the interaction modalities of a multi-robot team fundamentally impact the types of formations that can be achieved. To explore the trade-offs between network structure and the sensing and communication capabilities of individual robots, this paper applies controller synthesis to formation control of infinitesimally shape-similar frameworks, for which main...
-
Finding sources of airborne chemicals with mobile sensing systems finds applications in safety, security, and emergency situations related to medical, domestic, and environmental domains. Given the often critical nature of all the applications, it is important to reduce the amount of time necessary to accomplish this task through intelligent systems and algorithms. In this paper, we extend a previ...
-
This paper introduces the Weighted Buffered Voronoi tessellation, which allows us to define distributed, semicooperative multi-agent navigation policies with guarantees on collision avoidance. We generate the Voronoi cells with dynamic weights that bias the boundary towards the agent with the lower relative weight while always maintaining a buffered distance between two agents. By incorporating ag...
-
In this work, we present a novel approach for modeling, and classifying between, the system load states introduced when constructing staged soft arm configurations. Through a two stage approach: (1) an LSTM calibration routine is used to identify the current load state then (2) a control input generation step combines a generalized quasistatic model with the learned load model. Our experiments sho...
-
This paper proposes a muscle-like SMA (Shape Memory Alloy) actuator with an active cooling system for efficient response. An SMA coil spring is embedded into a stretchable coolant vascular for soften structure of robots. In order to design a flexible, lightweight, and fast-response soft actuator with the SMA coil spring and coolant circulation system, a modeling based approach has been conducted. ...
-
In this paper, we propose a novel approach for transferring a deep reinforcement learning (DRL) grasping agent from simulation to a real robot, without fine tuning in the real world. The approach utilises a CycleGAN to close the reality gap between the simulated and real environments, in a reverse real-to-sim manner, effectively "tricking" the agent into believing it is still in the simulator. Fur...
-
Multi-arm mobile manipulators can be represented as a combination of multiple robotic agents from the perspective of task-assignment and motion planning. Depending upon the task, agents might collaborate or work independently. Integrating motion planning with task-agent assignment is a computationally slow process as infeasible assignments can only be detected through expensive motion planning que...
-
Learning contact-rich, robotic manipulation skills is a challenging problem due to the high-dimensionality of the state and action space as well as uncertainty from noisy sensors and inaccurate motor control. To combat these factors and achieve more robust manipulation, humans actively exploit contact constraints in the environment. By adopting a similar strategy, robots can also achieve more robu...
-
To solve multi-step manipulation tasks in the real world, an autonomous robot must take actions to observe its environment and react to unexpected observations. This may require opening a drawer to observe its contents or moving an object out of the way to examine the space behind it. Upon receiving a new observation, the robot must update its belief about the world and compute a new plan of actio...
-
The present paper introduces a novel methodology for Unscented Kalman Filtering (UKF) on manifolds that extends previous work by the authors on UKF on Lie groups. Beyond filtering performance, the main interests of the approach are its versatility, as the method applies to numerous state estimation problems, and its simplicity of implementation for practitioners not being necessarily familiar with...
-
For most applications in mobile robotics, precise state estimation is essential. Typically, the state estimation is based on the fusion of data from different sensors. In practice, these sensors differ in their characteristics and measurements are available to the sensor fusion algorithm only with delay. Based on a brief survey of sensor fusion approaches that consider delayed and out-of-sequence ...
-
The fusion of multiple sensor modalities, especially through deep learning architectures, has been an active area of study. However, an under-explored aspect of such work is whether the methods can be robust to degradation across their input modalities, especially when they must generalize to degradation not seen during training. In this work, we propose an uncertainty-aware fusion scheme to effec...
-
In this paper, we present an efficient and robust GPS-aided visual inertial odometry (GPS-VIO) system that fuses IMU-camera data with intermittent GPS measurements. To perform sensor fusion, spatiotemporal sensor calibration and initialization of the transform between the sensor reference frames are required. We propose an online calibration method for both the GPS-IMU extrinsics and time offset a...
-
To fuse information from inertial measurement units (IMU) with other sensors one needs an accurate model for IMU error propagation in terms of position, velocity and orientation, a triplet we call extended pose. In this paper we leverage a nontrivial result, namely log-linearity of inertial navigation equations based on the recently introduced Lie group SE2(3), to transpose the recent methodology ...
-
We present an approach for estimating the body-frame velocity of a mobile robot. We combine measurements from a millimeter-wave radar-on-a-chip sensor and an inertial measurement unit (IMU) in a batch optimization over a sliding window of recent measurements. The sensor suite employed is lightweight, low-power, and is invariant to ambient lighting conditions. This makes the proposed approach an at...
-
This paper presents the design of a compliant manipulator using a series elastic actuator (SEA) and a mechanism for precisely measuring the force acting on the contact part of the manipulator without using a force sensor. It is important to maintain a constant contact force between the compliant manipulator and the wall in order to guarantee cleaning performance, and the ball screw mechanism is us...
-
We present an approach for the autonomous, robust execution of peg-in-hole assembly tasks. We build on a sampling-based state estimation framework, in which samples are weighted according to their consistency with the position and joint torque measurements. The key idea is to reuse these samples in a motion generation step, where they are assigned a second task-specific weight. The algorithm there...
-
This paper identifies conditions for designing the appropriate spatial admittance to achieve reliable force-guided assembly of polyhedral parts for cases in which a single frictional contact occurs between the two parts. This work is an extension of previous work in which frictionless contact was considered. This paper presents a way to characterize friction without solving a set of complicated no...
-
Actuators are essential devices that exert force and do work. The contraction of an actuator (how much it can shorten) is an important property that strongly influences its applications, especially in engineering and robotics. While high contractions have been achieved by thermally- or fluidically-driven technologies, electrically-driven actuators typically cannot contract by more than 50%. Recent...
-
Supercoiled polymer (SCP) artificial muscles exhibit many desirable properties such as large contractions and high power density. However, their full potential as robotic muscles is challenged by insufficient strain or force generation – non-mandrel-coiled SCP actuators produce up to 10-20% strain; mandrel-coiled SCP actuators often lift up to 10-30g of weight. It is strongly desired but difficult...
-
Soft continuum robots have many applications such as medical surgeries, service industries, rescue tasks, and underwater exploration. Flexibility and good accessibility of such robots are the key reasons for their popularity. However, the complexity of their structural design and control systems limit their broader applications. In this paper, a novel variable stiffness soft continuum robot based ...
-
SwarmRail represents a novel solution to overhead manipulation from a mobile unit that drives in an aboveground rail-structure. The concept is based on the combination of omnidirectional mobile platform and L-shaped rail profiles that form a through-going central gap. This gap makes possible mounting a robotic manipulator arm overhead at the underside of the mobile platform. Compared to existing s...
-
In this paper, we present a fast, on-line mapping and planning solution for operation in unknown, off-road, environments. We combine obstacle detection along with a terrain gradient map to make simple and adaptable cost map. This map can be created and updated at 10 Hz. An A* planner finds optimal paths over the map. Finally, we take multiple samples over the control input space and do a kinematic...
-
Autonomous robots often encounter challenging situations where their control policies fail and an expert human operator must briefly intervene, e.g., through teleoperation. In settings where multiple robots act in separate environments, a single human operator can manage a fleet of robots by identifying and teleoperating one robot at any given time. The key challenge is that users have limited att...
-
In human-robot collaboration, it is crucial for the robot to make its intentions clear and predictable to the human partners. Inspired by the mutual learning and adaptation of human partners, we suggest an actor-critic approach for a legible robot motion planner. This approach includes two neural networks and a legibility evaluator: 1) A policy network based on deep reinforcement learning (DRL); 2...
-
We present an approach for controlling the position of a quadrotor in 3D space using pointing gestures; the task is difficult because it is in general ambiguous to infer where, along the pointing ray, the robot should go. We propose and validate a pragmatic solution based on a push button acting as a simple additional input device which switches between different virtual workspace surfaces. Result...
-
Aiming to understand how human (false-)belief— a core socio-cognitive ability—would affect human interactions with robots, this paper proposes to adopt a graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs. Specifically, a parse graph (pg) is learned from a single-view spatiotemporal parsing by aggregating various object states along the time; s...
-
Newborn infants are naturally attracted to human faces, a crucial source of information for social interaction. In robotics, acquisition of such information is crucial and social robots should also learn to exhibit such social skill. Deep learning algorithms are valid candidates to address the problem of face localisation. However, a major drawback of these methods is the large amount of data and ...
-
Maintaining high levels of productivity for planetary rover missions is very difficult due to limited communication and heavy reliance on ground control. There is a need for autonomy that enables more adaptive and efficient actions based on real-time information. This paper presents an autonomous mapping and exploration approach for planetary rovers. We first describe a machine learning model that...
-
The autonomous operation of robots in the International Space Station (ISS) is required to maximize the use of limited resources and enable astronauts to concentrate more on valuable tasks. To achieve these goals, we are developing a station where the robot approaches, docks, charges, and then undocks. In this paper, the authors propose a magnetic docking mechanism for free-flying robots with sphe...
-
In this work, we demonstrate an instrumented wheel concept which utilizes a 2D pressure grid, an electrochemical impedance spectroscopy (EIS) sensor and machine learning (ML) to extract meaningful metrics from the interaction between the wheel and surface terrain. These include continuous slip/skid estimation, balance, and sharpness for engineering applications. Estimates of surface hydration, tex...
-
On-orbit proximity operations in space rendezvous, docking and debris removal require precise and robust 6D pose estimation under a wide range of lighting conditions and against highly textured background, i.e., the Earth. This paper investigates leveraging deep learning and photorealistic rendering for monocular pose estimation of known uncooperative spacecraft. We first present a simulator built...
-
To control a free-floating robotic system with uncertain parameters in OOS tasks with high accuracy, a fast parameter identification method, previously developed by the authors, is enhanced further and used concurrently with a controller. The method provides accurate parameter estimates, without any prior knowledge of any system dynamic properties. This control scheme compensates for the accumulat...
-
The performance of model-based motion control for free-flying robots relies on accurate estimation of their parameters. In this work, a method of rigid body inertial parameter estimation which relies on a variational approach is presented. Instead of discretizing the continuous equations of motion, discrete dual quaternion equations based on variational mechanics are used to formulate a linear par...
-
Many approaches for camera and LiDAR calibration are presented in literature but none of them estimates all intrinsic and extrinsic parameters simultaneously and therefore optimally in a probabilistic sense.In this work, we present a method to simultaneously estimate intrinsic and extrinsic parameters of cameras and LiDARs in a unified problem. We derive a probabilistic formulation that enables fl...
-
In order to relate information across cameras in a Dynamic Camera Cluster (DCC), an accurate time-varying set of extrinsic calibration transformations need to be determined. Previous calibration approaches rely solely on collecting measurements from a known fiducial target which limits calibration accuracy as insufficient excitation of the gimbal is achieved. In this paper, we improve DCC calibrat...
-
Planarity of checkerboards is a widely used feature for extrinsic calibration of camera and LiDAR. In this study, we propose two analytically derived covariances of (i) plane parameters and (ii) plane measurement, for precise extrinsic calibration of camera and LiDAR. These covariances allow the graded approach in planar feature correspondences by exploiting the uncertainty of a set of given featu...
-
Industrial arm-type robots have multiple degrees-of-freedom (DoFs) and high dexterity but the use of the roll-pitch-roll wrist configuration yields singularities inside the reachable workspace. Excessive joint velocities will occur when encountering these singularities. Arm-type robots currently don't have enough dexterity to move the end-effector path away from the wrist singularities. Robots wit...
-
This paper presents a novel solution for online trajectory planning of a full-size tractor-trailers vehicle composed of a car-like tractor and arbitrary number of passive full trailers. The motion planning problem for such systems was rarely addressed due to the complex nonlinear dynamics. A simulation-based prediction method is proposed to easily handle the complicated nonlinear dynamics and effi...
-
Scalability is a significant challenge for robot swarms. Generally, larger groups of cooperating robots produce more inter-robot collisions, and in swarm robot foraging, larger search arenas result in larger travel costs. This paper demonstrates a scale-invariant swarm foraging algorithm that ensures that each robot finds and delivers targets to a central collection zone at the same rate regardles...
-
Advances in legged robotics are strongly rooted in animal observations. A clear illustration of this claim is the generalization of Central Pattern Generators (CPG), first identified in the cat spinal cord, to generate cyclic motion in robotic locomotion. Despite a global endorsement of this model, physiological and functional experiments in mammals have also indicated the presence of descending s...
-
Neuromorphic hardware offers computing platforms for the efficient implementation of spiking neural networks (SNNs) that can be used for robot control. Here, we present such an SNN on a neuromorphic chip that solves a number of tasks related to simultaneous localization and mapping (SLAM): forming a map of an unknown environment and, at the same time, estimating the robot's pose. In particular, we...
-
In this study, a visual shock absorber capable of adapting to free-fall objects with various weights and speeds is designed and realized. The key element is a magnetic gear to passively absorb shock in the moment of contact, which is difficult for traditional feedback control to deal with. The magnetic gear allows the seamless transfer of control from the non-contact state to the contact state. 10...
-
Slip-Based Nonlinear Recursive Backstepping Path Following Controller for Autonomous Ground Vehicles
Path following accuracy and error convergence with graceful motion in vehicle steering control is challenging due to the competing nature of these requirements, especially across a range of operating speeds. This work is founded upon slip-based kinematic and dynamic models, which allow derivation of controllers considering error due to sideslip and the mapping between steering commands and gracefu...
-
This paper considers the problem of fast and safe autonomous navigation in partially known environments. Our main contribution is a control policy design based on ellipsoidal trajectory bounds obtained from a quadratic state-dependent distance metric. The ellipsoidal bounds are used to embed directional preference in the control design, leading to system behavior that is adapted to local environme...
-
Series elastic actuators with passive compliance have been gaining increasing popularity in force-controlled robotic manipulators. One of the reasons is the actuator's ability to infer the applied torque by measuring the deflection of the elastic element as opposed to directly with dedicated torque sensors. Proper deflection control is pinnacle to achieve a desired output torque and, therefore, sm...
-
A novel scheme for an "upgrade" of a linear control algorithm to a non-linear one is developed based on the concepts of a generalized homogeneity and an implicit homogeneous feedback design. Some tuning rules for a guaranteed improvement of a regulation quality are proposed. Theoretical results are confirmed by real experiments with the quadrotor QDrone of Quanser™.
-
To detach a permanent magnet using a control force much smaller than its original attractive force, the internally-balanced magnetic unit (IB Magnet) was invented. It has been applied to magnetic devices such as wall-climbing robots, ceiling-dangling drones, and modular swarm robots. In contrast to its significant reduction rate with regard to the control force, the IB Magnet has two major problem...
-
Continuum manipulators can accomplish various tasks in confined spaces, benefiting from their compliant structures and improved dexterity. Confined and unstructured spaces may require both enhanced stiffness of a continuum manipulator for precision and payload, as well as compliance for safe interaction. Thus, studies have been consistently dedicated to design continuum or articulated manipulators...
-
Snake-like robots for endoscopic surgery make it possible to reach deep-seated lesions. With the use of small flexible tendon-driven instruments, it is possible to perform bimanual micro-surgical tasks that are challenging for standard endoscopic surgeries. Existing devices, however, lack articulated wrists and rolling motion of the end-effector. This paper presents a new instrument design with a ...
-
Motion planning for vehicles under the influence of flow fields can benefit from the idea of streamline-based planning, which exploits ideas from fluid dynamics to achieve computational efficiency. Important to such planners is an efficient means of computing the travel distance and direction between two points in free space, but this is difficult to achieve in strong incompressible flows such as ...
-
Coral species detection underwater is a challenging problem. There are many cases when even the experts (marine biologists) fail to recognize corals, hence limiting ground truth annotation for training a robust detection system. Identifying coral species is fundamental for enabling the monitoring of coral reefs, a task currently performed by humans, which can be automated with the use of underwate...
-
This paper presents an observer-integrated Reinforcement Learning (RL) approach, called Disturbance OB-server Network (DOB-Net), for robots operating in environments where disturbances are unknown and time-varying, and may frequently exceed robot control capabilities. The DOB-Net integrates a disturbance dynamics observer network and a controller network. Originated from conventional DOB mechanism...
-
Ocean Worlds represent one of the best chances for extra-terrestrial life in our solar system. A new mission concept must be developed to explore these oceans. This mission would require traversing the 10s of km thick icy shell and releasing a submersible into the ocean below. During the transit of the icy shell and the exploration of the ocean, the vehicle(s) would be out of contact with Earth fo...
-
Photogrammetry techniques used for 3D reconstructions and motion estimation from images are based on projective geometry that models the image formation process. However, in the underwater setting, refraction of light rays at the housing interface introduce non-linear effects in the image formation. These effects produce systematic errors if not accounted for, and severely degrade the quality of t...
-
There is a long history demonstrating humans' tendency to create artificial copies of living creatures. For moving machines called robots, actuators play a key role in developing human-like movements. Among different types of actuation, PAMs (pneumatic artificial muscles) are known as the most similar ones to biological muscles. In addition to similarities in force generation mechanism (tension ba...
-
Light-weight robotic manipulators can be used to restore the manipulation capability of people with a motor disability. However, manipulating the environment poses a complex task, especially when the control interface is of low bandwidth, as may be the case for users with impairments. Therefore, we propose a constraint-based shared control scheme to define skills which provide support during task ...
-
Enabling robots to understand instructions provided via spoken natural language would facilitate interaction between robots and people in a variety of settings in homes and workplaces. However, natural language instructions are often missing information that would be obvious to a human based on environmental context and common sense, and hence does not need to be explicitly stated. In this paper, ...
-
Defining a robot's expressivity is a difficult task that requires thoughtful consideration of the potential of various robot modalities and a model of communication that humans understand. Humanoid and zoomorphic-designed robots can easily take cues from human and animals, respectively when designing their expressivity. However, a robot design that is neither human nor animal-like does not have a ...
-
Visual dirt detection is becoming an important capability of modern professional cleaning robots both for optimizing their wet cleaning results and for facilitating demand-oriented daily vacuum cleaning. This paper presents a robust, fast, and reliable dirt and office item detection system for these tasks based on an adapted YOLOv3 framework. Its superiority over state-of-the-art dirt detection sy...
-
We aim for mobile robots to function in a variety of common human environments. Such robots need to be able to reason about the locations of previously unseen target objects. Landmark objects can help this reasoning by narrowing down the search space significantly. More specifically, we can exploit background knowledge about common spatial relations between landmark and target objects. For example...
-
Recovering the 3D structure of the surrounding environment is an essential task in any vision-controlled Structure-from-Motion (SfM) scheme. This paper focuses on the theoretical properties of the SfM, known as the incremental active depth estimation. The term incremental stands for estimating the 3D structure of the scene over a chronological sequence of image frames. Active means that the camera...
-
Aerial imagery plays an important role in land-use planning, population analysis, precision agriculture, and unmanned aerial vehicle tasks. However, existing aerial image datasets generally suffer from the problem of inaccurate labeling, single ground truth type, and few category numbers. In this work, we implement a simulator that can simultaneously acquire diverse visual ground truth data in the...
-
Loop closure detection is an essential and challenging problem in simultaneous localization and mapping (SLAM). It is often tackled with light detection and ranging (LiDAR) sensor due to its view-point and illumination invariant properties. Existing works on 3D loop closure detection often leverage on matching of local or global geometrical-only descriptors which discard intensity reading. In this...
-
We propose to integrate text objects in man-made scenes tightly into the visual SLAM pipeline. The key idea of our novel text-based visual SLAM is to treat each detected text as a planar feature which is rich of textures and semantic meanings. The text feature is compactly represented by three parameters and integrated into visual SLAM by adopting the illumination-invariant photometric error. We a...
-
Many approaches have been proposed to estimate camera poses by directly minimizing photometric error. However, due to the non-convex property of direct alignment, proper initialization is still required for these methods. Many robust norms (e.g. Huber norm) have been proposed to deal with the outlier terms caused by incorrect initializations. These robust norms are solely defined on the magnitude ...
-
Adding more cameras to SLAM systems improves robustness and accuracy but complicates the design of the visual front-end significantly. Thus, most systems in the literature are tailored for specific camera configurations. In this work, we aim at an adaptive SLAM system that works for arbitrary multi-camera setups. To this end, we revisit several common building blocks in visual SLAM. In particular,...
-
The static world assumption is standard in most simultaneous localisation and mapping (SLAM) algorithms. Increased deployment of autonomous systems to unstructured dynamic environments is driving a need to identify moving objects and estimate their velocity in real-time. Most existing SLAM based approaches rely on a database of 3D models of objects or impose significant motion constraints. In this...
-
The question of "representation" is central in the context of dense simultaneous localization and mapping (SLAM). Learning-based approaches have the potential to leverage data or task performance to directly inform the representation. However, blending representation learning approaches with "classical" SLAM systems has remained an open question, because of their highly modular and complex nature....
-
Learning local behavioral sequences to better infer non-local properties in real multi-robot systems
When members of a multi-robot team follow regular motion rules sensitive to robots and other environmental factors within sensing range, the team itself may become an informational fabric for gaining situational awareness without explicit signalling among robots. In our previous work [1], we used machine learning to develop a scalable module, trained only on data from 3-robot teams, that could pre...
-
Learning-based ego-motion estimation approaches have recently drawn strong interest from researchers, mostly focusing on visual perception. A few learning-based approaches using Light Detection and Ranging (LiDAR) have been re-ported; however, they heavily rely on a supervised learning manner. Despite the meaningful performance of these approaches, supervised training requires ground-truth pose la...
-
Learning from observation (LfO) offers a new paradigm for transferring task behavior to robots. LfO requires the robot to observe the task being performed and decompose the sensed streaming data into sequences of state-action pairs, which are then input to LfO methods. Thus, recognizing the state-action pairs correctly and quickly in sensed data is a crucial prerequisite. We present SA-Net a deep ...
-
Robots navigating in human crowds need to optimize their paths not only for their task performance but also for their compliance to social norms. One of the key challenges in this context is the lack of standard metrics for evaluating and optimizing a socially compliant behavior. Existing works in social navigation can be grouped according to the differences in their optimization objectives. For i...
-
While robot learning has demonstrated promising results for enabling robots to automatically acquire new skills, a critical challenge in deploying learning-based systems is scale: acquiring enough data for the robot to effectively generalize broadly. Imitation learning, in particular, has remained a stable and powerful approach for robot learning, but critically relies on expert operators for data...
-
Learning meaningful visual representations in an embedding space can facilitate generalization in downstream tasks such as action segmentation and imitation. In this paper, we learn a motion-centric representation of surgical video demonstrations by grouping them into action segments/subgoals/options in a semi-supervised manner. We present Motion2Vec, an algorithm that learns a deep embedding feat...
-
This paper proposes a new path planning algorithm to consider motion uncertainty for wheeled robots in rough environments. The proposed method uses particles to express the uncertainty propagation in complicated environments constructed with various types of terrain. Also, RRT (Rapidly-exploring Random Tree) is expanded based on the uncertainty of each node in order to prevent increasing the accum...
-
RRT* is one of the most widely used sampling-based algorithms for asymptotically-optimal motion planning. RRT* laid the foundations for optimality in motion planning as a whole, and inspired the development of numerous new algorithms in the field, many of which build upon RRT* itself. In this paper, we first identify a logical gap in the optimality proof of RRT*, which was developed by Karaman and...
-
We study fundamental theoretical aspects of probabilistic roadmaps (PRM) in the finite time (non-asymptotic) regime. In particular, we investigate how completeness and optimality guarantees of the approach are influenced by the underlying deterministic sampling distribution X and connection radius r > 0. We develop the notion of (δ, ε)-completeness of the parameters X, r, which indicates that for ...
-
The complexity of surgical operation can be released significantly if surgical robots can learn the manipulation skills by imitation from complex tasks demonstrations such as puncture, suturing, and knotting, etc.. This paper proposes a reinforcement learning algorithm based manipulation skill transferring technique for robot-assisted Minimally Invasive Surgery by Teaching by Demonstration. It emp...
-
This paper considers safe robot mission planning in uncertain dynamical environments. This problem arises in applications such as surveillance, emergency rescue, and autonomous driving. It is a challenging problem due to mod-eling and integrating dynamical uncertainties into a safe planning framework, and finding a solution in a computationally tractable way. In this work, we first develop a proba...
-
An Iterative Quadratic Method for General-Sum Differential Games with Feedback Linearizable Dynamics
Iterative linear-quadratic (ILQ) methods are widely used in the nonlinear optimal control community. Recent work has applied similar methodology in the setting of multi-player general-sum differential games. Here, ILQ methods are capable of finding local equilibria in interactive motion planning problems in real-time. As in most iterative procedures, however, this approach can be sensitive to init...
-
Hybrid aerial-aquatic vehicles have the unique ability of travelling in both air and water and can benefit from both lower fluid resistance in air and energy efficient position holding in water. However, they have to address the differing requirements which make optimising a single design difficult. While existing examples have shown the possibility of such vehicles, they are mostly structurally i...
-
For most people on the ground, facing an unwanted drone buzzing around overhead, there is not a lot that we can do, especially if it is out of gun (radio wave gun or shotgun) range. A solution to this is to use intercept drones that seek out and bring down other drones. In order to make the interception autonomous, an image-based visual servo algorithm is designed with a forward-looking monocular ...
-
A quadcopter is an under-actuated system with only four control inputs for six degrees of freedom, and yet the human control of a quadcopter is simple enough to be learned with some practice. In this work, we consider the problem of human control of a multiple quadcopters system to transport a cable-suspended payload. The coupled dynamics of the system, due to the inherent physical constraints, is...
-
With the increasing global popularity of self-driving cars, there is an immediate need for challenging real-world datasets for benchmarking and training various computer vision tasks such as 3D object detection. Existing datasets either represent simple scenarios or provide only day-time data. In this paper, we introduce a new challenging A*3D dataset which consists of RGB images and LiDAR data wi...
-
3D vehicle detection based on point cloud is a challenging task in real-world applications such as autonomous driving. Despite significant progress has been made, we observe two aspects to be further improved. First, the semantic context information in LiDAR is seldom explored in previous works, which may help identify ambiguous vehicles. Second, the distribution of point cloud on vehicles varies ...
-
Fine-Grained Driving Behavior Prediction via Context-Aware Multi-Task Inverse Reinforcement Learning
Research on advanced driver assistance systems for reducing risks to vulnerable road users (VRUs) has recently gained popularity because the traffic accident reduction rate for VRUs is still small. Dealing with unexpected VRU movements on residential roads requires proficient acceleration and deceleration. Although fine-grained prediction of driving behavior through inverse reinforcement learning ...
-
The current state of the art in automotive high definition digital (HD) map generation based on dedicated mapping vehicles cannot reliably keep these maps up to date because of the low traversal frequencies. Anonymized data collected from the fleet of vehicles that is already on the road provides a huge potential to outperform such state of the art solutions in robustness, safety and up-to-datenes...
-
Driveable area detection is a key component for various applications in the field of autonomous driving (AD), such as ground-plane detection, obstacle detection and maneuver planning. Additionally, bulky and over-parameterized networks can be easily forgone and replaced with smaller networks for faster inference on embedded systems. The driveable area detection, posed as a two class segmentation t...
-
Mapping and localization is a critical module of autonomous driving, and significant achievements have been reached in this field. Beyond Global Navigation Satellite System (GNSS), research in point cloud registration, visual feature matching, and inertia navigation has greatly enhanced the accuracy and robustness of mapping and localization in different scenarios. However, highly urbanized scenes...
-
Accurate and robust global localization is essential to robotics applications. We propose a novel global localization method that employs the map traversability as a hidden observation. The resulting map-corrected odometry localization is able to provide an accurate belief tensor of the robot state. Our method can be used for blind robots in dark or highly reflective areas. In contrast to odometry...
-
We present an end-to-end trainable Neural Network architecture for stereo imaging that jointly locates and estimates human body poses in 3D. Our method defines a 2D pose for each human in a stereo pair of images and uses a correlation layer with a composite field to associate each left-right pair of joints. In absence of a stereo pose dataset, we show that we can train our method with synthetic da...
-
We present a self-supervised deep pose correction (DPC) network that applies pose corrections to a visual odometry estimator to improve its accuracy. Instead of regressing inter-frame pose changes directly, we build on prior work that uses data-driven learning to regress pose corrections that account for systematic errors due to violations of modelling assumptions. Our self-supervised formulation ...
-
Indoor positioning technology is essential for indoor mobile robots and drones. However, there has never been a general-purpose technology or infrastructure that enables indoor positioning with an accuracy of less than 10 cm. We have developed an attitude measurement method using multiple dynamic moires with a lenticular lens and developed an ultra- high-accuracy visual marker with an attitude est...
-
Accurate localization and tracking are a fundamental requirement for robotic applications. Localization systems like GPS, optical tracking, simultaneous localization and mapping (SLAM) are used for daily life activities, research, and commercial applications. Ultra-wideband (UWB) technology provides another venue to accurately locate devices both indoors and outdoors. In this paper, we study a loc...
-
This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user prefer-ences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences bet...
-
Collaborative robots and space manipulators contain significant joint flexibility. It complicates the control design, compromises the control bandwidth, and limits the tracking accuracy. The imprecise knowledge of the flexible joint dynamics compounds the challenge. In this paper, we present a new control architecture for controlling flexible-joint robots. Our approach uses a multi-layer neural ne...
-
Correlation filters (CFs) have shown excellent performance in unmanned aerial vehicle (UAV) tracking scenarios due to their high computational efficiency. During the UAV tracking process, viewpoint variations are usually accompanied by changes in the object and background appearance, which poses a unique challenge to CF-based trackers. Since the appearance is gradually changing over time, an ideal...
-
Studies on object rearrangement planning typically consider known objects. Some learning-based methods can predict the movement of an unknown object after single-step interaction, but require intermediate targets, which are generated manually, to achieve the rearrangement task. In this work, we propose a framework for unknown object rearrangement. Our system first models an object through a small-...
-
Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce TAXONS, a Task Agnostic eXploration of Out...
-
This paper presents a method to enable a robot using stochastic Model Predictive Control (MPC) to achieve high performance on a repetitive path-following task. In particular, we consider the case where the accuracy of the model for robot dynamics varies significantly over the path-motivated by the fact that the models used in MPC must be computationally efficient, which limits their expressive pow...
-
Current clinical endovascular interventions rely on 2D guidance for catheter manipulation. Although an aortic 3D surface is available from the pre-operative CT/MRI imaging, it cannot be used directly as a 3D intra-operative guidance since the vessel will deform during the procedure. This paper aims to reconstruct the live 3D aortic deformation by fusing the static 3D model from the pre-operative d...
-
Needle-based procedures, such as biopsy and percutaneous tumor ablation, highly depend on the accuracy of needle placement. The accuracy is significantly affected by the needle-tissue interaction no matter what needles (straight or steerable) are used. Due to the unknown tissue mechanics, it is challenging to achieve high accuracy in practice. This paper hence proposes a needle design with an arti...
-
This paper introduces a method for robotic steering of a flexible needle inside moving and deformable tissues. The method relies on a set of objective functions allowing to automatically steer the needle along a predefined path. In order to follow the desired trajectory, an inverse problem linking the motion of the robot end effector with the objective functions is solved using a Finite Element si...
-
Master-slave systems for endovascular catheterization have brought major clinical benefits including reduced radiation doses to the operators, improved precision and stability of the instruments, as well as reduced procedural duration. Emerging deep reinforcement learning (RL) technologies could potentially automate more complex endovascular tasks with enhanced success rates, more consistent motio...
-
Path planning algorithms for steerable catheters, must guarantee anatomical obstacles avoidance, reduce the insertion length and ensure the compliance with needle kinematics. The majority of the solutions in literature focuses on graph based or sampling based methods, both limited by the impossibility to directly obtain smooth trajectories. In this work we formulate the path planning problem as a ...
-
We present a novel control strategy for dynamic legged locomotion in complex scenarios that considers information about the morphology of the terrain in contexts when only on-board mapping and computation are available. The strategy is built on top of two main elements: first a contact sequence task that provides safe foothold locations based on a convolutional neural network to perform fast and c...
-
This paper presents an adaptive control scheme for robotic systems that operate in the face of-potentially large-structured uncertainty. The proposed adaptive controller employs an on-line supervisor that utilizes logic-based switching among a finite set of controllers to identify uncertain parameters, and adapt the behavior of the system based on a current estimate of their value. To achieve this...
-
This paper proposes a novel algorithm for joint space position/torque hybrid control of a mammal-type quadruped robot. With this control algorithm, the robot demonstrated both dynamic locomotion and push reaction abilities without the need for torque control in the ab/ad joints. Based on the tipping and slipping condition of the legged robot, we showed that reaction to a typical push in the horizo...
-
Robotic Hopping is challenging from the perspective of both modeling the dynamics as well as the mechanical design due to the short period of ground contact in which to actuate on the world. Previous work has demonstrated stable hopping on a moving-mass robot, wherein a single spring was utilized below the body of the robot. This paper finds that the addition of a spring in parallel to the actuato...
-
Legged robots have been highlighted as promising mobile platforms for disaster response and rescue scenarios because of their rough terrain locomotion capability. In cluttered environments, small robots are desirable as they can maneuver through small gaps, narrow paths, or tunnels. However small robots have their own set of difficulties such as limited space for sensors, limited obstacle clearanc...
-
We aim to guard swarm-robotics applications against denial-of-service (DoS) attacks that result in withdrawals of robots. We focus on applications requiring the selection of actions for each robot, among a set of available ones, e.g., which trajectory to follow. Such applications are central in large-scale robotic applications, e.g., multi-robot motion planning for target tracking. But the current...
-
We study a patrolling problem where multiple agents are tasked with protecting an environment where one or more adversaries are trying to compromise targets of varying value. The objective of the patrollers is to move between targets to quickly spot when an attack is taking place and then diffuse it. Differently from most related literature, we do not assume that attackers have full knowledge of t...
-
This paper presents a distributed optimization framework and its local utility design for communication-aware information gathering by mobile robotic sensor networks. The main idea of the optimization is that each robot decides based on its local utility that considers the decisions of other neighbor robots higher in a given hierarchy. The local utility is designed as conditional mutual informatio...
-
We investigate algorithmic approaches for targeted drug delivery in a complex, maze-like environment, such as a vascular system. The basic scenario is given by a large swarm of micro-scale particles ("agents") and a particular target region ("tumor") within a system of passageways. Agents are too small to contain on-board power or computation and are instead controlled by a global external force t...
-
Time-optimal trajectories for dynamic robotic vehicles are difficult to compute even for state-of-the-art nonlinear programming (NLP) solvers, due to nonlinearity and bang-bang control structure. This paper presents a bilevel optimization framework that addresses these problems by decomposing the spatial and temporal variables into a hierarchical optimization. Specifically, the original problem is...
-
We propose a sampling-based trajectory optimization methodology for constrained problems. We extend recent works on stochastic search to deal with box control constraints, as well as nonlinear state constraints for discrete dynamical systems. Regarding the former, our strategy is to optimize over truncated parameterized distributions on control inputs. Furthermore, we show how non-smooth penalty f...
-
The ability to optimally control robotic systems offers significant advantages for their performance. While time-dependent optimal trajectories can numerically be computed for high dimensional nonlinear system dynamic models, constraints and objectives, finding optimal feedback control policies for such systems is hard. This is unfortunate, as without a policy, the control of real-world systems re...
-
We introduce Crocoddyl (Contact RObot COntrol by Differential DYnamic Library), an open-source framework tailored for efficient multi-contact optimal control. Crocoddyl efficiently computes the state trajectory and the control policy for a given predefined sequence of contacts. Its efficiency is due to the use of sparse analytical derivatives, exploitation of the problem structure, and data sharin...
-
Integrated robotic arm system should contain both grasp and place actions. However, most grasping methods focus more on how to grasp objects, while ignoring the placement of the grasped objects, which limits their applications in various industrial environments. In this research, we propose a model-free deep Q-learning method to learn the grasping-stacking strategy end-to-end from scratch. Our met...
-
Semantic grasping is the problem of selecting stable grasps that are functionally suitable for specific object manipulation tasks. In order for robots to effectively perform object manipulation, a broad sense of contexts, including object and task constraints, needs to be accounted for. We introduce the Context-Aware Grasping Engine, which combines a novel semantic representation of grasp contexts...
-
Depth acquisition, based on active illumination, is essential for autonomous and robotic navigation. LiDARs (Light Detection And Ranging) with mechanical, fixed, sampling templates are commonly used in today's autonomous vehicles. An emerging technology, based on solid-state depth sensors, with no mechanical parts, allows fast and adaptive scans. In this paper, we propose an adaptive, image-driven...
-
Light Detection and Ranging (LIDAR) sensors play an important role in the perception stack of autonomous robots, supplying mapping and localization pipelines with depth measurements of the environment. While their accuracy outperforms other types of depth sensors, such as stereo or time-of-flight cameras, the accurate modeling of LIDAR sensors requires laborious manual calibration that typically d...
-
Three-dimensional reconstruction of dynamic objects is important for robotic applications, for example, the robotic recognition and manipulation. In this paper, we present a novel 3D surface reconstruction method for moving objects. The proposed method combines the spatial-multiplexing and time-multiplexing structured-light techniques that have advantages of less image acquisition time and accurat...
-
Recent studies in radar-based navigation present promising navigation performance using scanning radars. These scanning radar-based odometry methods are mostly feature-based; they detect and match salient features within a radar image. Differing from existing feature-based methods, this paper reports on a method using direct radar odometry, PhaRaO, which infers relative motion from a pair of radar...
-
Understanding the semantic characteristics of the environment is a key enabler for autonomous robot operation. In this paper, we propose a deep convolutional neural network (DCNN) for semantic segmentation of a LiDAR scan into the classes car, pedestrian and bicyclist. This architecture is based on dense blocks and efficiently utilizes depth separable convolutions to limit the number of parameters...
-
Dual-arm robots can overcome grasping force and payload limitations of a single arm by jointly grasping an object. However, if the distribution of mass of the grasped object is not even, each arm will experience different wrenches that can exceed its payload limits. In this work, we consider the problem of balancing the wrenches experienced by a dual-arm robot grasping a rigid tray. The distributi...
-
Developing artificial tactile sensing capabilities that rival human touch is a long-term goal in robotics and prosthetics. Gradually more elaborate biomimetic tactile sensors are being developed and applied to grasping and manipulation tasks to help achieve this goal. Here we present the neuroTac, a novel neuromorphic optical tactile sensor. The neuroTac combines the biomimetic hardware design fro...
-
How to reduce uncertainty in object pose estimation under complex contacts is crucial to autonomous robotic manipulation and assembly. In this paper, we introduce an approach through forecasting contact force from simulated complex contacts with calibration based on real force sensing. A constraint-based haptic simulation algorithm is used with sphere-tree representation of contacting objects to c...
-
Fingernail imaging has been proven to be effective in prior works [1], [2] for estimating the 3D fingertip forces with a maximum RMS estimation error of 7%. In the current research, fingernail imaging is used to perform unconstrained grasp force measurement on multiple fingers to study human grasping. Moreover, two robotic arms with mounted cameras and a visual tracking system have been devised to...
-
Given a set of 3D-to-2D point correspondences corrupted by outliers, we aim to robustly estimate the absolute camera pose. Existing methods robust to outliers either fail to guarantee high robustness and efficiency simultaneously, or require an appropriate initial pose and thus lack generality. In contrast, we propose a novel approach based on the robust "L2-minimizing estimate" (L2E) loss. We fir...
-
In this paper, we present an on-board vision-based approach for avoidance of moving obstacles in dynamic environments. Our approach relies on an efficient obstacle detection and tracking algorithm based on depth image pairs, which provides the estimated position, velocity and size of the obstacles. Robust collision avoidance is achieved by formulating a chance-constrained model predictive controll...
-
This paper presents a monocular vision based proximity estimation system using abstract features, such as corner points, blobs and edges, as inputs to a neural network. An experimental vehicle was built using a vision system integrating the SCAMP-5 vision chip, a micro-controller, and an RC model car. The vision chip includes image sensor with embedded 256×256 processor SIMD array. The pixel proce...
-
The motion of planar ground vehicles is often non-holonomic, and as a result may be modelled by the 2 DoF Ackermann steering model. We analyse the feasibility of estimating such motion with a downward facing camera that exerts fronto-parallel motion with respect to the ground plane. This turns the motion estimation into a simple image registration problem in which we only have to identify a 2-para...
-
The TEB hierarchical planner for real-time navigation through unknown environments is highly effective at balancing collision avoidance with goal directed motion. Designed over several years and publications, it implements a multi-trajectory optimization based synthesis method for identifying topologically distinct trajectory candidates through navigable space. Unfortunately, the underlying factor...
-
Collecting and automatically obtaining reward signals from real robotic visual data for the purposes of training reinforcement learning algorithms can be quite challenging and time-consuming. Methods for utilizing unlabeled data can have a huge potential to further accelerate robotic learning. We consider here the problem of performing manipulation tasks from pixels. In such tasks, choosing an app...
-
Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to pr...
-
General-purpose simulators can be a valuable data source for flexible learning and control approaches. However, training models or control policies in simulation and then directly applying to hardware can yield brittle control. Instead, we propose a novel way to use simulators as regularizers. Our approach regularizes a decoder of a variational autoencoder to a black-box simulation, with the laten...
-
In the robotics literature, different knowledge transfer approaches have been proposed to leverage the experience from a source task or robot-real or virtual-to accelerate the learning process on a new task or robot. A commonly made but infrequently examined assumption is that incorporating experience from a source task or robot will be beneficial. In practice, inappropriate knowledge transfer can...
-
DeepRacer is a platform for end-to-end experimentation with RL and can be used to systematically investigate the key challenges in developing intelligent control systems. Using the platform, we demonstrate how a 1/18th scale car can learn to drive autonomously using RL with a monocular camera. It is trained in simulation with no additional tuning in the physical world and demonstrates: 1) formulat...
-
Beyond the ultimate goal of prosthetics, repairing all the capabilities of amputees, the development line of upper-limb prostheses control mainly relies on three aspects: the robustness, the intuitiveness and the reduction of mental fatigue. Many complex structures and algorithms are proposed but no one question a common open-loop nature, where the user is the one in charge of correcting errors. Y...
-
Magnetic tracking algorithms can be used to determine the position and orientation of magnetic markers or devices. These techniques are particularly interesting for biomedical applications such as teleoperated surgical robots or the control of upper limb prostheses. The performance of different algorithms used for magnetic tracking was compared in the past. However, in most cases, those algorithms...
-
We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations while keeping tracks of all evacuees’ locations. A population density map, obtained on-the-fly by aggregating locations of evacuees from user-end Augmented Reality (AR) devices, is used to model the congestion distribution inside a build...
-
The paper presents an approach for interactive programming of the robotic manipulator using mixed reality. The developed system is based on the HoloLens glasses connected through Robotic Operation System to Unity engine and robotic manipulators. The system gives a possibility to recognize the real robot location by the point cloud analysis, to use virtual markers and menus for the task creation, t...
-
Heart rate monitoring at home is a useful metric for assessing health e.g. of the elderly or patients in post-operative recovery. Although non-contact heart rate monitoring has been widely explored, typically using a static, wall-mounted device, measurements are limited to a single room and sensitive to user orientation and position. In this work, we propose mBeats, a robot mounted millimeter wave...
-
Gait training is widely used to treat gait abnormalities. Traditional gait measurement systems are limited to instrumented laboratories. Even though gait measurements can be made in these settings, it is challenging to estimate gait parameters robustly in real-time for gait rehabilitation, especially when walking over-ground. In this paper, we present a novel approach to track the continuous gait ...
-
Biped robots require massive power on each leg while walking, hopping, and running. We have developed a flow-based control system-called hydraulic direct drive system- that can achieve high output while avoiding spatial limitations. To implement the proposed system with simple equipment configuration, a pump and single-rod cylinder are connected in a closed loop. However, because compensation for ...
Introduction
Conference ICRA2020 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 1000