DeepNLP IROS2020 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is ...
-
Anomaly detection is a key goal of autonomous surveillance systems that should be able to alert unusual observations. In this paper, we propose a holistic anomaly detection system using deep neural networks for surveillance of critical infrastructures (e.g., airports, harbors, warehouses) using an unmanned aerial vehicle (UAV). First, we present a heuristic method for the explicit representation o...
-
ROSflight is a lean, open-source autopilot system developed with the primary goal of supporting the needs of researchers working with micro aerial vehicle systems. The project consists of firmware designed to run on low-cost, readily available flight controller boards, as well as ROS packages for interfacing between the flight controller and application code and for simulation. The core objectives...
-
Nonlinear Model Predictive Control (NMPC) is a powerful and widely used technique for nonlinear dynamic process control under constraints. In NMPC, the state and control weights of the corresponding state and control costs are commonly selected based on human-expert knowledge, which usually reflects the acceptable stability in practice. Although broadly used, this approach might not be optimal for...
-
Unmanned Aerial Vehicles (UAVs) are becoming increasingly popular in the film and entertainment industries, in part because of their maneuverability and perspectives they enable. While there exists methods for controlling the position and orientation of the drones for visibility, other artistic elements of the filming process, such as focal blur, remain unexplored in the robotics community. The la...
-
With a growing number of applications in the world for UAVs, there is a clear limitation regarding the need for extended battery life. With the current flight times, many users would benefit greatly with an innovative option of field charging these devices. The objective of this project is to investigate feasibility of inductively harvesting energy from a power line cable for applications such as ...
-
We introduce SplitFlyer-a novel quadcopter with an ability to disassemble into two self-contained bicopters through human assistance. As a subunit, the bicopter is a severely underactuated aerial vehicle equipped with only two propellers. Still, each bicopter is capable of independent flight. To achieve this, we provide an analysis of the system dynamics by relaxing the control over the yaw rotati...
-
This paper addresses the problem of cooperative transport of a point mass hoisted by two aerial robots. Treating the robots as a leader and a follower, the follower stabilizes the system with respect to the leader using only feedback from its Inertial Measurement Units (IMU). This is accomplished by neglecting the acceleration of the leader, analyzing the system through the generalized coordinates...
-
Currently, drone research and development has received significant attention worldwide. Particularly, delivery services employ drones as it is a viable method to improve delivery efficiency by using a several unmanned drones. Research has been conducted to realize complete automation of drone control for such services. However, regarding the takeoff and landing port of the drones, conventional met...
-
A high-quality estimate of wind fields can potentially improve the safety and performance of Unmanned Aerial Vehicles (UAVs) operating in dense urban areas. Computational Fluid Dynamics (CFD) simulations can help provide a wind field estimate, but their accuracy depends on the knowledge of the distribution of the inlet boundary conditions. This paper provides a real-time methodology using a Partic...
-
Mobile robots of various types have been proposed for infrastructure inspection and disaster investigation. For such mobile robot applications, accessing the areas is of primary importance for missions. Therefore, various locomotive mechanisms have been studied. We introduce a novel mobile robot system, named DIR-3, combining a crawler robot and a microdrone. By rotating its arm back and forth, DI...
-
In this paper, we propose a new concept of robot which is hybrid, including aerial and crawling subsystems and an arm, and also modular with interchangeable crawling subsystems for different pipe configurations, since it has been designed to cover most industrial oil & gas end-users' requirements. The robot has the same ability than aerial robots to reach otherwise inaccessible locations, but make...
-
Geomorphological Analysis Using Unpiloted Aircraft Systems, Structure from Motion, and Deep Learning
We present a pipeline for geomorphological analysis that uses structure from motion (SfM) and deep learning on close-range aerial imagery to estimate spatial distributions of rock traits (size, roundness, and orientation) along a tectonic fault scarp. The properties of the rocks on the fault scarp derive from the combination of initial volcanic fracturing and subsequent tectonic and geomorphic fra...
-
In the paper, a pair of auto-tuning methods for fixed-parameter controllers is presented, in application to multirotor unmanned aerial vehicles (UAVs) control. In both cases, the automatized process of searching the best altitude controller parameters is carried out with the use of a modified golden-search method, for a selected cost function, during the flight of a pair of UAVs. All the calculati...
-
Trajectory optimization problems under affine motion model and convex cost function are often solved through the convex-concave procedure (CCP), wherein the non-convex collision avoidance constraints are replaced with its affine approximation. Although mathematically rigorous, CCP has some critical limitations. First, it requires a collision-free initial guess of solution trajectory which is diffi...
-
Landing is an essential part of multicopter task operations. A multicopter has relatively stringent requirements for landing, particularly for achieving flatness. Currently, landing on rough terrain with normal skids is difficult. Therefore, research is being conducted to obtain skids capable of landing on rough terrain. In this paper, a passive skid for multicopter landing on rough terrain is pro...
-
We present a novel paradigm and algorithm for optimal design of underactuated robot platforms in highly-constrained nonconvex parameter spaces. We apply this algorithm to two variants of the mature RoboBee platform, numerically demonstrating predicted performance improvements of over 10% in some cases by algorithmically reasoning about variable effective-mechanical-advantage (EMA) transmissions, h...
-
This work presents the design, fabrication, and characterization of an airflow sensor inspired by the whiskers of animals. The body of the whisker was replaced with a fin structure in order to increase the air resistance. The fin was suspended by a micro-fabricated spring system at the bottom. A permanent magnet was attached beneath the spring, and the motion of fin was captured by a readily acces...
-
We present PufferBot, an aerial robot with an expandable structure that may expand to protect a drone's propellers when the robot is close to obstacles or collocated humans. PufferBot is made of a custom 3D-printed expandable scissor structure, which utilizes a one degree of freedom actuator with rack and pinion mechanism. We propose four designs for the expandable structure, each with unique char...
-
Rotary-wing flying machines draw attention within the UAV community for their in-place hovering capability, and recently, holonomic motion over fixed-wings. In this paper, we investigate about the power-optimality in a mono-spinner, i.e., a class of rotary-wing UAVs with one rotor only, whose main body has a streamlined shape for producing additional lift when counter-spinning the rotor. We provid...
-
The conceptual design and flight controller of a novel kind of quadcopter are presented. This design is capable of morphing the shape of the UAV during flight to achieve position and attitude control. We consider a dynamic center of gravity (CoG) which causes continuous variation in a moment of inertia (MoI) parameters of the UAV. These dynamic structural parameters play a vital role in the stabil...
-
This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation driven by external forces. We derive the dynamic model of this variable geomet...
-
Exploiting contacts with environment structures provides extra force support to a UAV, often reducing the power consumption and hence extending the mission time. This paper investigates one such way to exploit flat surfaces in the environment by a novel aerial-ground hybrid locomotion. Our design is a single passive wheel integrated at the UAV bottom, serving a minimal design to date. We present t...
-
This work presents a model-free nonlinear controller for an ornithopter prototype with bioinspired wings and tail. The size and power requirements have been thought to allocate a customized autopilot on board. To assess the functionality and performance of the full mechatronic design, a controller has been designed and implemented to execute a prescribed perching 2D trajectory. Although functional...
-
Aerial vehicles with collision resilience can operate with more confidence in environments with obstacles that are hard to detect and avoid. This paper presents the methodology used to design a collision resilient aerial vehicle with icosahedron tensegrity structure. A simplified stress analysis of the tensegrity frame under impact forces is performed to guide the selection of its components. In a...
-
This work presents the deployment of UAVs for the exploration of clouds, from the system architecture and simulation tests to a real-flight campaign and trajectory analyzes. Thanks to their small size and low altitude, light UAVs have proven to be adapted for in-situ cloud data collection. The short life time of the clouds and limited endurance of the planes require to focus on the area of maximum...
-
In micro aerial vehicle (MAV) operations, the success of a mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and the designated mission are often different in unknown, unstructured environments, and only sub-optimality may be obtained in each aspect...
-
Coverage path planning (CPP) is the task of designing a trajectory that enables a mobile agent to travel over every point of an area of interest. We propose a new method to control an unmanned aerial vehicle (UAV) carrying a camera on a CPP mission with random start positions and multiple options for landing positions in an environment containing no-fly zones. While numerous approaches have been p...
-
This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasi...
-
For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, its limitations on flight altitude prevent the UAVs from observing the global map. Thus motion planning in the local map becomes a Partially Observa...
-
We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicl...
-
We propose an algorithm for generating minimum-snap trajectories for quadrotors with linear computational complexity with respect to the number of segments in the spline trajectory. Our algorithm is numerically stable for large numbers of segments and is able to generate trajectories of more than 500, 000 segments. The computational speed and numerical stability of our algorithm makes it suitable ...
-
Persistent surveillance with aerial vehicles (drones) subject to connectivity and power constraints is a relatively uncharted domain of research. To reduce the complexity of multi-drone motion planning, most state-of-the-art solutions ignore network connectivity and assume unlimited battery power. Motivated by this and advances in optimization and constraint satisfaction techniques, we introduce a...
-
This paper proposes a planning algorithm for autonomous media production with multiple Unmanned Aerial Vehicles (UAVs) in outdoor events. Given filming tasks specified by a media Director, we formulate an optimization problem to maximize the filming time considering battery constraints. As we conjecture that the problem is NP-hard, we consider a discretization version, and propose a graph-based al...
-
In this work, we present a system architecture to enable autonomous navigation of multiple agents across user-selected global interest points in a partially unknown environment. The system is composed of a server and a team of agents, here small aircrafts. Leveraging this architecture, computation-ally demanding tasks, such as global dense mapping and global path planning can be outsourced to a po...
-
Cooperative robotics is a trending topic nowadays as it makes possible a number of tasks that cannot be performed by individual robots, such as heavy payload transportation and agile manipulation. In this work, we address the problem of cooperative transportation by heterogeneous, manipulator- endowed robots. Specifically, we consider a generic number of robotic agents simultaneously grasping an o...
-
Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt...
-
There has been a considerable amount of recent work on high-speed micro-aerial vehicle flight in unknown and unstructured environments. Generally these approaches either use active sensing or fly slowly enough to ensure a safe braking distance with the relatively short sensing range of passive sensors. The former generally requires carrying large and heavy LIDARs and the latter only allows flight ...
-
The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements. Previous appearances are also gradually forgotten due to the exponential decay of historical views in traditional appearance updating scheme of DCF framework, reducing the model's robustness. In this work, a novel tracker based on DCF framework is proposed to augmen...
-
Next-Best-View planning for surface reconstruction of large-scale 3D environments with multiple UAVs
In this paper, we propose a novel cluster-based Next-Best-View path planning algorithm to simultaneously explore and inspect large-scale unknown environments with multiple Unmanned Aerial Vehicles (UAVs). In the majority of existing informative path-planning methods, a volumetric criterion is used for the exploration of unknown areas, and the presence of surfaces is only taken into account indirec...
-
Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i...
-
Inspection for structural properties (surface stiffness and coefficient of restitution) is crucial for understanding and performing aerial manipulations in unknown environments, with little to no prior knowledge on their state. Inspection-on-the-fly is the uncanny ability of humans to infer states during manipulation, reducing the necessity to perform inspection and manipulation separately. This p...
-
Visual tracking has yielded promising applications with unmanned aerial vehicle (UAV). In literature, the advanced discriminative correlation filter (DCF) type trackers generally distinguish the foreground from the background with a learned regressor which regresses the implicit circulated samples into a fixed target label. However, the predefined and unchanged regression target results in low rob...
-
The integration of computer vision techniques for the accomplishment of autonomous interaction tasks represents a challenging research direction in the context of aerial robotics. In this paper, we consider the problem of contact-based inspection of a textured target of unknown geometry and pose. Exploiting state of the art techniques in computer graphics, tuned and improved for the task at hand, ...
-
In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computation...
-
We present a method to reconstruct the 3D trajectory of an airborne robotic system only from videos recorded with cameras that are unsynchronized, may feature rolling shutter distortion, and whose viewpoints are unknown. Our approach enables robust and accurate outside-in tracking of dynamically flying targets, with cheap and easy-to-deploy equipment. We show that, in spite of the weakly constrain...
-
This paper presents a method for target localization and tracking in clutter using Bayesian fusion of vision and Radio Frequency (RF) sensors used aboard a small Unmanned Aircraft System (sUAS). Sensor fusion is used to ensure tracking robustness and reliability in case of camera occlusion or RF signal interference. Camera data is processed using an off-the-shelf algorithm that detects possible ob...
-
Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to- end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real- world data. Simulated data, on the other hand, is easy to...
-
Disturbance estimation for Micro Aerial Vehicles (MAVs) is crucial for robustness and safety. In this paper, we use novel, bio-inspired airflow sensors to measure the airflow acting on a MAV, and we fuse this information in an Unscented Kalman filter (UKF) to simultaneously estimate the three-dimensional wind vector, the drag force, and other interaction forces (e.g. due to collisions, interaction...
-
We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate th...
-
Automated driving requires highly precise and robust vehicle state estimation for its environmental perception, motion planning and control functions. Using GPS and environmental sensors can compensate for the deficits of the estimation based on traditional vehicle dynamics sensors. However, each type of sensor has specific strengths and limitations in accuracy and robustness due to their differen...
-
We present a probabilistic framework for detailed 3-D shape estimation and tracking using only vision measurements. Vision detections are processed via a bird's eye view representation, creating accurate detections at far ranges. A probabilistic model of the vision based point cloud measurements is learned and used in the framework. A 3-D shape model is developed by fusing a set of point cloud det...
-
Achieving and maintaining line-of-sight (LOS) is challenging for underwater optical communication systems, especially when the underlying platforms are mobile. In this work, we propose and demonstrate an active alignment controlbased LED-communication system that uses the DC value of the communication signal as feedback for LOS maintenance. Utilizing the uni-modal nature of the dependence of the l...
-
Underwater communication is extremely challenging for small underwater robots which typically have stringent power and size constraints. In our previous work, we developed an artificial electrocommunication system which could be an alternative for the communication of small underwater robots. This paper further presents a new electrocommunication system that utilizes Binary Frequency Shift Keying ...
-
Underwater roll rotation is a basic but essential maneuver that allows many biological swimmers to achieve high maneuverability and complex locomotion patterns. In particular, sea mammals (e.g., sea otter) with flexible vertebra structures have a unique mechanism to efficiently achieve roll rotation, not propelled mainly by inter-digital webbing or fin, but by bending and twisting their body.In th...
-
In this paper, we present a novel algorithm for constructing a maximally informative path for a robot in an information gathering task. We use a Self-Organizing Map (SOM) framework to discover important topological features in the information function. Using these features, we identify a set of distinct classes of trajectories, each of which has improved convexity compared with the original functi...
-
The SPIR, Submersible Pylon Inspection Robot, is developed to provide an innovative and practical solution to keep workers safe during maintenance of underwater structures in shallow waters, which involves working in dangerous water currents, and high-pressure water-jet cleaning. More advanced than work-class Remotely Operated Vehicles technology, the SPIR is automated and required minimum involve...
-
This paper presents a novel autonomous surface vessel (ASV), called Roboat II for urban transportation. Roboat II is capable of accurate simultaneous localization and mapping (SLAM), receding horizon tracking control and estimation, and path planning. Roboat II is designed to maximize the internal space for transport, and can carry payloads several times of its own weight. Moreover, it is capable ...
-
Automatic latching for charging in a disturbed environment for Unmanned Surface Vehicle (USVs) is always a challenging problem. In this paper, we propose a two-stage automatic latching system for USVs charging in berth. In Stage I, a vision-guided algorithm is developed to calculate an optimal latching position for charging. In Stage II, a novel latching mechanism is designed to compensate the mov...
-
This paper presents the results of the experimental tests performed to validate the functionality of a variable pitch system (VPS), designed for pitch attitude control of the novel underwater robotic vehicle explorer UX-1. The VPS is composed of a mass suspended from a central rod mounted across the hull. This mass is rotated around the transverse axis of the vehicle in order to perform a change i...
-
In this paper we present the LoCO AUV, a Low-Cost, Open Autonomous Underwater Vehicle. LoCO is a general-purpose, single-person-deployable, vision-guided AUV, rated to a depth of 100 meters. We discuss the open and expandable design of this underwater robot, as well as the design of a simulator in Gazebo. Additionally, we explore the platform's preliminary local motion control and state estimation...
-
In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborati...
-
In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to th...
-
This paper proposes a methodology for real-time depth estimation of underwater monocular camera images, fusing measurements from a single-beam echosounder. Our system exploits the echosounder's detection cone to match its measurements with the detected feature points from a monocular SLAM system. Such measurements are integrated in a monocular SLAM system to adjust the visible map points and the s...
-
This paper presents a novel risk vector-based near miss prediction and obstacle avoidance method. The proposed method uses the sensor readings about the pose of the other obstacles to infer their motion model (velocity and heading) and, accordingly, adapt the risk assessment and take corrective actions if necessary. Relative vector calculations allow the method to perform in real-time. The algorit...
-
This work presents a practical method of obtaining a dynamic system model for small omnidirectional aquatic vehicles. The models produced can be used to improve vehicle localisation, aid in the design or tuning of control systems and facilitate the development of simulated environments. The use of a dynamic model for onboard real-time velocity prediction is of particular importance for aquatic veh...
-
Despite the recent progress, guidance, navigation, and control (GNC) are largely unsolved for agile micro autonomous underwater vehicles (μAUVs). Hereby, robust and accurate self-localization systems which fit μAUVs play a key role and their absence constitutes a severe bottleneck in micro underwater robotics research. In this work we present, first, a small-size low-cost high performance vision-b...
-
Space manipulator systems in orbit are subject to link flexibilities since they are designed to be lightweight and long reaching. Often, their joints are driven by harmonic gear-motor units, which introduce joint flexibility. Both of these types of flexibility may cause structural vibrations. To improve endpoint tracking, advanced control strategies that benefit from the knowledge of system parame...
-
In-space assembly (ISA) is the next step to building larger and more permanent structures in orbit. The use of a robotic in-space assembler saves on costly and potentially risky EVAs. Determining the best robot for ISA is difficult as it will depend on the structure being assembled. A comparison between two categories of robots is presented: a stationary robot and robot which crawls along the trus...
-
Ground-based teleoperation of robot manipulators for on-orbit servicing of spacecraft represents an example of high-payoff, high-risk operations that are challenging to perform due to high latency communications, with telemetry time delays of several seconds. In these scenarios, confidence of operating without failure is paramount. We report the development of an Interactive Planning and Supervise...
-
A novel identification method is developed which identifies the accumulated angular momentum (AAM) of spinning reaction wheels (RWs) of an uncooperative satellite captured by a robotic servicer. In contrast to other methods that treat captured satellite's RWs as non-spinning, the developed method provides simultaneously accurate estimates of the AAM of the captured satellite's RWs and of the inert...
-
This paper presents the modeling and analysis of a novel moving mechanism "tumbling" for asteroid exploration. The system actuation is provided by an internal motor and torque wheel; elastic spring-mounted spikes are attached to the perimeter of a circular-shaped robot, protruding normal to the surface and distributed uniformly. Compared with the conventional motion mechanisms, this simple layout ...
-
In this paper, we propose three novel Hardware-in-the-loop simulation (HLS) methods for a fully-actuated orbital robot in the presence of external interactions using On-Ground Facility Manipulators (OGFM). In particular, a fixed-base and a vehicle-driven manipulator are considered in the analyses. The key idea is to describe the orbital robot's dynamics using the Lagrange-Poincaré(LP) equations, w...
-
With the booming development in aerospace technology, the video satellite which observes the live phenomena on the ground by video shooting has gradually emerged as a new Earth observation method. And remote sensing comes into a "dynamic" era with the demand for new processing techniques, especially the near-real-time tracking and geo-positioning algorithm for ground moving targets. However, many ...
-
The ability to recognize previously mapped locations is an essential feature for autonomous systems. Unstructured planetary-like environments pose a major challenge to these systems due to the similarity of the terrain. As a result, the ambiguity of the visual appearance makes state-of-the-art visual place recognition approaches less effective than in urban or man-made environments. This paper pre...
-
We propose a system for visually monitoring and servoing the cutting of a multi-layer insulation (MLI) blanket that covers the envelope of satellites and spacecraft. The main contributions of this paper are: 1) to propose a model for relating visual features describing the engagement depth of the blade to the force exerted on the MLI blanket by the cutting tool, 2) a blade design and algorithm to ...
-
This paper presents a novel approach to sampling subsurface asteroidal regolith under severe time constraints. Sampling operations that must be completed within a few hours require techniques that can manage subsurface obstructions that may be encountered. The large uncertainties due to our lack of knowledge of regolith properties also make sampling difficult. To aid in managing these challenges, ...
-
Modular robotic systems with self-repair or self-replication capabilities have been presented as a robust, low cost solution to extraterrestrial or Arctic exploration. This paper explores using ice as the sole structure element to build robots. The ice allows for increased flexibility in the system design, enabling the robotic structure to be designed and built post deployment, after tasks and ter...
-
The ocean world Europa is a prime target for exploration given its potential habitability [1]. We propose a mobile robotic system that is capable of autonomously traversing tens of meters to visit multiple sites of interest on a Europan analogue surface. Due to the topology of Europan terrain being largely unknown, it is desired that this mobility system traverse a large variety of terrain types. ...
-
To allow for the construction of large space structures to support future space endeavors, autonomous robotic solutions would serve to reduce cost and risk of human extravehicular activity (EVA). Practicality of autonomous assembly requires both theoretical and algorithmic advances, and hardware experimentation across a spectrum of technological readiness levels. Analysis of hardware experiments p...
-
Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which...
-
Recent works have proved that combining spatial and temporal visual cues can significantly improve the performance of various vision-based robotic systems. However, for the ultrasonic sensors used in most robotic tasks (e.g. collision avoidance, localization and navigation), there is a lack of benchmark ultrasonic datasets that consist of spatial and temporal data to verify the usability of spatia...
-
Drivers and other road users often encounter situations (e.g., arriving at an intersection simultaneously) where priority is ambiguous or unclear but must be resolved via communication to reach agreement. This poses a challenge for autonomous vehicles, for which no direct means for expressing intent and acknowledgment has yet been established. This paper contributes a minimal model to manage ambig...
-
Automated vehicles need to not only perceive their environment, but also predict the possible future behavior of all detected traffic participants in order to safely navigate in complex scenarios and avoid critical situations, ranging from merging on highways to crossing urban intersections. Due to the availability of datasets with large numbers of recorded trajectories of traffic participants, de...
-
Current autonomous driving systems are composed of a perception system and a decision system. Both of them are divided into multiple subsystems built up with lots of human heuristics. An end-to-end approach might clean up the system and avoid huge efforts of human engineering, as well as obtain better performance with increasing data and computation resources. Compared to the decision system, the ...
-
In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning. However, this perception task is difficult, particularly for generic obstacles/objects, due to appearance and occlusion changes. To tackle this problem, we propose an end-to-end deep learning framework for LIDAR-based flow estimation in bird's eye view (BeV). Our method ta...
-
Distracted driving is one of the main contributors to traffic accidents. This paper proposes a deep learning approach to detecting multiple distracted driving behaviors. In order to obtain more accurate detection results, a synchronized image recognition system based on two cameras is designed, by which the body movements and face of the driver are monitored respectively. The images captured from ...
-
In human-robot interaction (HRI) systems, such as autonomous vehicles, understanding and representing human behavior are important. Human behavior is naturally rich and diverse. Cost/reward learning, as an efficient way to learn and represent human behavior, has been successfully applied in many domains. Most of traditional inverse reinforcement learning (IRL) algorithms, however, cannot adequatel...
-
Understanding the kinematics of a ground robot is essential for efficient navigation. Based on the kinematic model of a robot, its full motion capabilities can be represented by theoretical motion primitives. However, depending on the environment and/or human preferences, not all of those theoretical motion primitives are desirable and/or achievable. This work presents a method to identify effecti...
-
We present a new measure, CMetric, to classify driver behaviors using centrality functions. Our formulation combines concepts from computational graph theory and social traffic psychology to quantify and classify the behavior of human drivers. CMetric is used to compute the probability of a vehicle executing a driving style, as well as the intensity used to execute the style. Our approach is desig...
-
We propose an integrated approach to active exploration by exploiting the Cartographer method as the base SLAM module for submap creation and performing efficient frontier detection in the geometrically co-aligned submaps induced by graph optimization. We also carry out analysis on the reachability of frontiers and their clusters to ensure that the detected frontier can be reached by robot. Our me...
-
Recent advancements in statistical learning and computational abilities have enabled autonomous vehicle technology to develop at a much faster rate. While many of the architectures previously introduced are capable of operating under highly dynamic environments, many of these are constrained to smaller-scale deployments, require constant maintenance due to the associated scalability cost with high...
-
Extensive city navigation remains an unresolved problem for autonomous mobile robots that share space with pedestrians. This paper proposes a configuration for a navigation map that expresses urban structures and an autonomous navigation scheme that uses the configuration. The proposed map configuration is a hybrid structure of multiple 2D grid maps and a topological graph. The occupancy grids for...
-
Predicting the future trajectory of surrounding vehicles in a randomly varying traffic level is one of the most challenging problems in developing an autonomous vehicle. Since there is no pre-defined number of interacting vehicles participated in, the prediction network has to be scalable with respect to the number of vehicles in order to guarantee consistent performance in terms of both accuracy ...
-
Traffic simulators are important tools in autonomous driving development. While continuous progress has been made to provide developers more options for modeling various traffic participants, tuning these models to increase their behavioral diversity while maintaining quality is often very challenging. This paper introduces an easily-tunable policy generation algorithm for autonomous driving agent...
-
We present a predictive runtime monitoring technique for estimating future vehicle positions and the probability of collisions with obstacles. Vehicle dynamics model how the position and velocity change over time as a function of external inputs. They are commonly described by discrete-time stochastic models. Whereas positions and velocities can be measured, the inputs (steering and throttle) are ...
-
Autonomous vehicles need to plan at the task level to compute a sequence of symbolic actions, such as merging left and turning right, to fulfill people's service requests, where efficiency is the main concern. At the same time, the vehicles must compute continuous trajectories to perform actions at the motion level, where safety is the most important. Task-motion planning in autonomous driving fac...
-
In this work, we address the motion planning problem for autonomous vehicles through a new lattice planning approach, called Feedback Enhanced Lattice Planner (FELP). Existing lattice planners have two major limitations, namely the high dimensionality of the lattice and the lack of modeling of agent vehicle behaviors. We propose to apply the Intelligent Driver Model (IDM) [1] as a speed feedback p...
-
Ground plane estimation and ground point segmentation is a crucial precursor for many applications in robotics and intelligent vehicles like navigable space detection and occupancy grid generation, 3D object detection, point cloud matching for localization and registration for mapping. In this paper, we present GndNet, a novel end-to-end approach that estimates the ground plane elevation informati...
-
Autonomous navigation and exploration in confined spaces are currently setting new challenges for robots. The presence of narrow passages, flammable atmosphere, dust, smoke, and other hazards makes the mapping and navigation tasks extremely difficult. To tackle these challenges, robots need to make intelligent decisions, maximising information while maintaining the safety of the system and their s...
-
As electric vehicle (EV) technologies become mature, EV has been rapidly adopted in modern transportation systems, and is expected to provide future autonomous mobility-on-demand (AMoD) service with economic and societal benefits. However, EVs require frequent recharges due to their limited and unpredictable cruising ranges, and they have to be managed efficiently given the dynamic charging proces...
-
Autonomous vehicles are typically developed and trained to work under certain system and environmental conditions defined at design time and can fail or perform poorly if unforeseen conditions such as disturbances or changes in model dynamics appear at runtime. In this work, we present a fast online planning, learning, and recovery approach for safe autonomous operations under unknown runtime dist...
-
Monocular 3D object detection aims to detect objects in a 3D physical world from a single camera. However, recent approaches either rely on expensive LiDAR devices, or resort to dense pixel-wise depth estimation that causes prohibitive computational cost. In this paper, we propose an end-to-end trainable monocular 3D object detector without learning the dense depth. Specifically, the grid coordina...
-
The Crossing or Not-Crossing (C/NC) problem is important to autonomous vehicles (AVs) for safe vehicle/pedestrian interactions. However, this problem setup often ignores pedestrians walking along the direction of the vehicles' movement (LONG). To enhance the AVs' awareness of pedestrian behavior, we make the first step towards extending the C/NC to the C/NC/LONG problem and recognize them based on...
-
We propose a robust solution to future trajectory forecast, which can be practically applicable to autonomous agents in highly crowded environments. For this, three aspects are particularly addressed in this paper. First, we use composite fields to predict future locations of all road agents in a singleshot, which results in a constant time complexity, regardless of the number of agents in the sce...
-
Learning-based driving solution, a new branch for autonomous driving, is expected to simplify the modeling of driving by learning the underlying mechanisms from data. To improve the tactical decision-making for learning-based driving solution, we introduce hierarchical behavior and motion planning (HBMP) to explicitly model the behavior in learning-based solution. Due to the coupled action space o...
-
Long-tail and rare event problems become crucial when autonomous driving algorithms are applied in the real world. For the purpose of evaluating systems in challenging settings, we propose a generative framework to create safety-critical scenarios for evaluating specific task algorithms. We first represent the traffic scenarios with a series of autoregressive building blocks and generate diverse s...
-
This contribution is concerned with the topic of using simulation to understand the behavior of groups of mutually interacting autonomous vehicles (AVs) or robots engaged in traffic/maneuvers that involve coordinated operation. We outline the structure of a multi-agent simulator called SYN-CHRONO and provide results pertaining to its scalability and ability to run real-time scenarios with humans i...
-
An autonomous vehicle platoon is a network of autonomous vehicles that communicate together to move in a desired way. One of the greatest threats to the operation of an autonomous vehicle platoon is the failure of either a physical component of a vehicle or a communication link between two vehicles. This failure affects the safety and stability of the autonomous vehicle platoon. Transmissibility-b...
-
The Go-CHART is a four-wheel, skid-steer robot that resembles a 1:28 scale standard commercial sedan. It is equipped with an onboard sensor suite and both onboard and external computers that replicate many of the sensing and computation capabilities of a full-size autonomous vehicle. The Go-CHART can autonomously navigate a small-scale traffic testbed, responding to its sensor input wiwithth progr...
-
Radar sensors have become an important part of the perception sensor suite due to their long range and their ability to work in adverse weather conditions. However, several shortcomings such as large amounts of noise and extreme sparsity of the point cloud result in them not being used to their full potential. In this paper, we present a novel Recursive Least Squares (RLS) based approach to estima...
-
While Lidar beams are often represented as rays, they actually have finite beam width and this width impacts the measured shape and size of objects in the scene. Here we investigate the effects of beam width on measurements of thin objects such as vertical poles. We propose a model for beam divergence and show how this can explain both object dilation and erosion. We develop a calibration method t...
-
Autonomous driving requires the inference of actionable information such as detecting and classifying objects, and determining the drivable space. To this end, we present Multi-View LidarNet (MVLidarNet), a two-stage deep neural network for multi-class object detection and drivable space segmentation using multiple views of a single LiDAR point cloud. The first stage processes the point cloud proj...
-
Roads have well defined geometries, topologies, and traffic rules. While this has been widely exploited in motion planning methods to produce maneuvers that obey the law, little work has been devoted to utilize these priors in perception and motion forecasting methods. In this paper we propose to incorporate these structured priors as a loss function. In contrast to imposing hard constraints, this...
-
Automobiles or robots with advanced autonomous systems are equipped with multiple types of sensors to overcome different weather and geographical conditions. These sensors generally have various data delays and sampling rates. Additionally, the communication delays or time synchronization errors between the onboard computers significantly affect the robustness and accuracy of localization for auto...
-
We consider the problem of localization and navigation of Autonomous Underwater Vehicles (AUV) in the context of high performance subsea asset inspection missions in deep water. We propose a solution based on the recently introduced Unscented Kalman Filter on Manifolds (UKF-M) for onboard navigation to estimate the robot's location, attitude and velocity, using a precise round and rotating Earth n...
-
Lane mark detection is one of the key tasks for autonomous driving systems. Accurate detection of lane marks under complex urban environments remains a challenge. In this paper, an end-to-end lane mark detection network named DSSF-net, which is capable of directly outputting the accurate fitted lane curves, is proposed. First, a dual-task segmentation framework for jointing lane category predictio...
-
Autonomous vehicles often rely on high-definition (HD) maps to navigate around. However, lane markings (LMs) are not necessarily static objects due to wear & tear from usage and road reconstruction & maintenance. Therefore, the wrong matching between LMs in the HD map and sensor readings may lead to erroneous localization or even cause traffic accidents. It is imperative to keep LMs up-to-date. Ho...
-
In recent years, self-supervised methods for monocular depth estimation has rapidly become an significant branch of depth estimation task, especially for autonomous driving applications. Despite the high overall precision achieved, current methods still suffer from a) imprecise object-level depth inference and b) uncertain scale factor. The former problem would cause texture copy or provide inaccu...
-
It is well known that semantic segmentation can be used as an effective intermediate representation for learning driving policies. However, the task of street scene semantic segmentation requires expensive annotations. Furthermore, segmentation algorithms are often trained irrespective of the actual driving task, using auxiliary image-space loss functions which are not guaranteed to maximize drivi...
-
This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. To tackle the first issue we exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. The maps are used in an attention mechanism that promotes segmentation confidence masks, thus focusing the network on semantic classes in the image tha...
-
Directionality in path planning is essential for efficient autonomous navigation in a number of real-world environments. In many map-based navigation scenarios, the viable path from a given point A to point B is not the same as the viable path from B to A. We present a method that automatically incorporates preferred navigation directionality into a path planning costmap. This `preference' is repr...
-
Trajectory prediction is crucial for autonomous vehicles. The planning system not only needs to know the current state of the surrounding objects but also their possible states in the future. As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest. Most of the existing works use rasterized maps to e...
-
Constrained Iterative Linear Quadratic Regulator (CILQR), a variant of ILQR, has been recently proposed for motion planning problems of autonomous vehicles to deal with constraints such as obstacle avoidance and reference tracking. However, the previous work considers either deterministic trajectories or persistent prediction for target dynamical obstacles. The other drawback is lack of generality...
-
The lane change maneuver is one of the typical maneuvers in various driving situations. Therefore the automatic lane change function is one of the key functions for autonomous vehicles. Many researches have been conducted in this field. Most existing work focused on the solutions for the static environment and assume that the surrounding vehicles are running at constant speeds. However, in reality...
-
In this paper, we present a real-time non-linear model-predictive control (NMPC) framework to perform minimum-time motion planning for autonomous racing cars. We introduce an innovative kineto-dynamical vehicle model, able to accurately predict non-linear longitudinal and lateral vehicle dynamics. The main parameters of this vehicle model can be tuned with only experimental or simulated maneuvers,...
-
In this paper we propose a hierarchical controller for autonomous racing where the same vehicle model is used in a two level optimization framework for motion planning. The high-level controller computes a trajectory that minimizes the lap time, and the low-level nonlinear model predictive path following controller tracks the computed trajectory online. Following a computed optimal trajectory avoi...
-
This paper studies a motion planning problem over a roadmap in which a vehicle aims to travel from a start to a destination in presence of an attacker who can launch a cyber-attack on the vehicle over any one edge of the roadmap. The vehicle (defender) has the capability to switch on/off a countermeasure that can detect and permanently disable the attack if it occurs concurrently. We first model t...
-
We investigate the autonomous navigation of a mobile robot in the presence of other moving vehicles under time-varying uncertain environmental disturbances. We first predict the future state distributions of other vehicles to account for their uncertain behaviors affected by the time-varying disturbances. We then construct a dynamic-obstacle-aware reachable space that contains states with high pro...
-
Plant phenotyping, i.e., the task of measuring plant traits to describe the anatomy and physiology of plants, is a central task in crop science and plant breeding. Standard methods often require intrusive or time-consuming operations involving a lot of manual labor. Cameras or range sensors, paired with 3D reconstructions methods, can support phenotyping but the task yields several challenges in p...
-
Global navigation satellite system (GNSS) has been considered as a panacea for positioning and tracking since the last decade. However, it suffers from severe limitations in terms of accuracy, particularly in highly cluttered and indoor environments. Though real-time kinematics (RTK) supported GNSS promises extremely accurate localisation, employing such services are expensive, fail in occluded en...
-
Continuous representations of objects have always been used in robotics in the form of geometric primitives and surface models. Recently, learning techniques have emerged which allow more complex continuous representations to be learned from data, but these learning techniques require training data in the form of watertight meshes which restricts their application as meshes of this form are diffic...
-
In this paper we consider the stochastic cost orienteering problem, i.e., a version of the classic orienteering problem where the cost associated with each edge is a random variable with known distribution. Such a model is relevant when travel costs are variable, e.g., when a robot moves in uncertain terrain conditions. We model this problem using a composite state space tracking both how much pro...
-
Precision livestock farming uses artificial intelligence to individually monitor livestock activity and health. Tracking individuals over time can reveal health indicators that correlate with productivity and longevity. For instance, locomotion patterns observed in lame pigs have been shown to correlate with poor animal welfare and productivity. Kinematic analysis of pigs using pose estimates prov...
-
In this paper, our motivating application lies in precision agriculture where accurate modeling of forage is essential for informing rotational grazing strategies. Unfortunately, a major difficulty arises in modeling forage processes as they evolve on large scales according to complex ecological influences. As robots can collect data over large scales in a forage environment, they act as a promisi...
-
Intuitive human robot interfaces like speech or gesture recognition are essential for gaining acceptance for robots in daily life. However, such interaction requires that the robot detects the human's intention to interact, tracks his position and keeps its sensor systems in an optimal configuration. Audio is a suitable modality for such task as it allows for detecting a speaker in arbitrary posit...
-
Towards long range and high endurance sailing, energy is of utmost importance. Moreover, benefiting from the dominance of the sailboat itself, it is energy-saving and environment-friendly. Thus, the sailboat with energy planning problem is meaningful. However, until now, the sailboat energy optimization problem has rarely been considered. In this paper, we focus on the energy consumption optimizat...
-
Tracking and locating radio-tagged wildlife is a labor-intensive and time-consuming task necessary in wildlife conservation. In this article, we focus on the problem of achieving embedded autonomy for a resource-limited aerial robot for the task capable of avoiding undesirable disturbances to wildlife. We employ a lightweight sensor system capable of simultaneous (noisy) measurements of radio sign...
-
The isoline tracking of this work is concerned with the control design for a sensing robot to track a given isoline of an unknown 2-D scalar filed. To this end, we propose a coordinate-free controller with a simple PI-like form using only the concentration feedback for a Dubins robot, which is particularly useful in GPS-denied environments. The key idea lies in the novel design of a sliding surfac...
-
Quadrotor flight in cluttered, unknown environments is challenging due to the limited range of perception sensors, challenging obstacles, and limited onboard computation. In this work, we directly address these challenges by proposing an efficient, reactive planning approach. We introduce the Bitwise Trajectory Elimination (BiTE) algorithm for efficiently filtering out in-collision trajectories fr...
-
This paper serves as one of the first efforts to enable large-scale and long-duration autonomy using the Boston Dynamics Spot robot. Motivated by exploring extreme environments, particularly those involved in the DARPA Subterranean Challenge, this paper pushes the boundaries of the state-of-practice in enabling legged robotic systems to accomplish real-world complex missions in relevant scenarios....
-
In this paper, we study a wheeled robot with a prismatic extension joint. This allows the robot to build up momentum to perform jumps over obstacles and to swing up to the upright position after the loss of balance. We propose a template model for the class of such two-wheeled jumping robots. This model can be considered as the simplest wheeled-legged system. We provide an analytical derivation of...
-
The holonomic and omnidirectional capabilities imparted to the mobile base platform depends mainly on two factors, i.e., the wheel design and its various arrangements in the platform chassis. This paper reports on the development of a novel omnidirectional spherical sectioned wheel named Ospheel. It is modular, and the spherical sectioned geometry of the wheel is driven using two actuators placed ...
-
In this work we present a Four-Wheeled Independent Drive and Steering (4WIDS) robot named AGRO and a method of controlling its orientation while airborne using wheel reaction torques. This is the first documented use of independently steerable wheels to both drive on the ground and achieve aerial attitude control when thrown. Inspired by a cat's self-righting reflex, this capability was developed ...
-
Autonomous navigation of steel bridge inspection robots are essential for proper maintenance. Majority of existing robotic solutions for bridge inspection require human intervention to assist in the control and navigation. In this paper, a control system framework has been proposed for a previously designed ARA robot [1], which facilitates autonomous real-time navigation and minimizes human involv...
-
A mobile robot that can achieve a stable attitude and locomotion on steep slopes is needed to overcome the problems of slipping and falling for automation of works on steep slopes. The conventional approaches to achieve a stable attitude and locomotion have been researched by adopting tracked wheels and multi-legged mechanisms instead of wheel mechanisms. However, these robots have limitations in ...
-
Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cases, the robot needs to safely push unripe fruits t...
-
Robotic packaging of fresh leafy produce such as herbs and salads generally involves picking out a target mass from a pile or crate of plant material. Typically, for low-complexity parallel grippers, the weight picked can be controlled by varying the opening aperture. However, often individual strands of plant material get entangled with each other, causing more to be picked out than desired. This...
-
Many robotic coverage applications involve detection of spatially distributed targets, followed by path planning to visit them for service. In these applications, the performance of the detection algorithm can have profound effect on planning decisions and costs. Range of operation, in both space and time, for robots is typically finite over a single mission and is a common constraint that needs t...
-
Crops are an important source of food and other products. In conventional farming, tractors apply large amounts of agrochemicals uniformly across fields for weed control and plant protection. Autonomous farming robots have the potential to provide environment-friendly weed control on a per plant basis. A system that reliably distinguishes crops, weeds, and soil under varying environment conditions...
-
We present techniques to measure crop heights using a 3D Light Detection and Ranging (LiDAR) sensor mounted on an Unmanned Aerial Vehicle (UAV). Knowing the height of plants is crucial to monitor their overall health and growth cycles, especially for high-throughput plant phenotyping. We present a methodology for extracting plant heights from 3D LiDAR point clouds, specifically focusing on plot-ba...
-
Creative design of grippers on their configurations, mechatronics control system, and multi-component collaborative algorithms is often utilized to realize complex operations in industrial applications, due to the environmental constraints or specific task requirements. Firstly, this paper introduces the background problems. As the main automatic equipment -- the shield machine -- in the field of ...
-
We propose a novel excavation (i.e., digging) trajectory planning framework for industrial autonomous robotic excavators, which emulates the strategies of human expert operators to optimize the excavation of (complex/unmodellable) soils while also upholding robustness and safety in practice. First, we encode the trajectory with dynamic movement primitives (DMP), which is known to robustly preserve...
-
Backhoe loads sediment onto the bed of dump trucks during earthmoving work. The prediction of backhoe loading time is essential for ensuring safe cooperation between the backhoe and dump trucks. However, it is difficult to predict the instant at which the backhoe is ready to load sediment, because of the similarity in motions observed during gathering sediment. Moreover, since operators have diffe...
-
In this paper, we propose a dynamic state estimation framework for lateral control of a heavy tractor-trailers system using only mass-produced low-cost sensors. This issue is challenging since the lateral velocity of the lead tractor is difficult to measure directly. The performance of existing dynamic model-based estimation methods will also be degraded, as different trailers and payloads cause t...
-
3D Point cloud registration is still a very challenging topic due to the difficulty in finding the rigid transformation between two point clouds with partial correspondences, and it's even harder in the absence of any initial estimation information. In this paper, we present an end-to-end deep-learning based approach to resolve the point cloud registration problem. Firstly, the revised LPD-Net is ...
-
This article presents a planning and control pipeline for legged-wheeled (hybrid) machines. It consists of a Trajectory Optimization based planner that computes references for end-effectors and joints. The references are tracked using a whole-body controller based on a hierarchical optimization approach. Our controller is capable of performing terrain adaptive whole-body control. Furthermore, it c...
-
In this article we present a data-driven approach for automated arm control of a hydraulic excavator. Except for the link lengths of the excavator, our method does not require machine-specific knowledge nor gain tuning. Using data collected during operation of the excavator, we train a general purpose model to effectively represent the highly non-linear dynamics of the hydraulic actuation and join...
-
Automation in construction continues to be a topic of interest for many in industry and academia. However, the dynamic environments presented in construction sites prove these tasks to be difficult to automate reliably. This paper proposes a novel method of teleoperation for multiple heterogeneous robots within a construction environment. The system is achieved by creating a virtual reality interf...
-
The joint detection of drivable areas and road anomalies is a crucial task for ground mobile robots. In recent years, many impressive semantic segmentation networks, which can be used for pixel-level drivable area and road anomaly detection, have been developed. However, the detection accuracy still needs improvement. Therefore, we develop a novel module named the Normal Inference Module (NIM), wh...
-
This paper characterizes an algorithm that estimates searcher skill level to support planning for search activities involving heterogeneous robot and human/robot teams. Specifically, we use Monte-Carlo simulations to determine the empirical accuracy of the estimator, to assess the quality of its predicted distribution (nonparametric) of agent skill levels, and the convergence rate of the estimate....
-
Developments in mobile robot navigation have enabled robots to operate in warehouses, retail stores, and on sidewalks around pedestrians. Various navigation solutions have been proposed, though few as widely adopted as ROS (Robot Operating System) Navigation. 10 years on, it is still one of the most popular navigation solutions1. Yet, ROS Navigation has failed to keep up with modern trends. We pro...
-
In this paper, we propose a novel path planning algorithm for the nonholonomic multiple mobile robot system with applications to a robotic autonomous luggage trolley collection system at airports. We consider this path planning algorithm as a Multiple Traveling Salesman Problem (MTSP). Our path planning algorithm consists of three parts. First, we use the Minimum Spanning Tree (MSP) algorithm to d...
-
Avoiding obstacles in the perceived world has been the classical approach to autonomous mobile robot navigation. However, this usually leads to unnatural and inefficient motions that significantly differ from the way humans move in tight and dynamic spaces, as we do not refrain interacting with the environment around us when necessary. Inspired by this observation, we propose a framework for auton...
-
Tissue engineering is trying to use modular tissue micro-rings to construct artificial biological microtubes as substitute of autologous tissue tubes to alleviate the shortage of donor sources. However, because of the lack of effective assembly strategies, it is still challenging to achieve high-speed fabrication of biological microtubes with high cell density. In this paper, we proposed a robotic...
-
This paper proposed a technique to introduce the membrane protein into the lab-on-chip analysis system having a planar lipid bilayer. The proposed technique utilized a dielectrophoretic(DEP) force generated by the asymmetric configuration of the solid electrodes on the aqueous buffer separator. By applying the alternating current to the separator and the counter electrode, we manipulated liposomes...
-
Under the excitation of acoustic radiation, the amount of energy absorbed and rebounded by cells have the relationship with mechanical properties, e.g. stiffness, shape, weight and so on. In this paper, a femtosecond laser-activated micro-detector is designed to convert this relationship into an electrical signal. First, the acoustic radiation is generated by a femtosecond laser pulse in a microch...
-
Sonoporation, which typically employs acoustic cavitation microbubbles, can enhance the permeability of the cell membrane, allowing foreign matter to enter cells across the natural barriers. However, the diameter nonuniformity and random distribution of microbubbles make it difficult to achieve controllable and high-efficiency sonoporation, while complex extern acoustic driving system also limits ...
-
Cell manipulation is a critical procedure in related biological applications such as embryo biopsy and intracytoplasmic sperm injection (ICSI), where the biological cell is required to be oriented to the desired position. To bridge the gap between the techniques and the clinical applications, a robotic micromanipulation method, which utilizes friction forces to rotate the cell with standard microp...
-
In this paper, we have constructed actively perfusable multiple hepatic lobule-like vascular networks in a 3D cellular structure by using magnetic tweezers. Without well-organized channel networks, cells in a large 3D tissue cannot receive nutrients and oxygen from the channel, and therefore, the cells will be dead after few days. To construct well-organized channel networks, we fabricated a hepat...
-
Magnetic microrobots can be propelled precisely and wirelessly in vivo using magnetic field for targeted drug delivery and early detection. They are promising for clinical trials since magnetic fields are capable of penetrating most materials with minimal interaction, and are nearly harmless to human beings. However, challenges like the biocompatibility, biodegradation and therapeutic effects of t...
-
In this paper, we present a magnetized cell-robot using macrophages as templates, which can be controlled under a strong gradient magnetic field, to approach and kill cancer cells in both vitro and vivo environment. Firstly, we establish a magnetic control system using only four coils which can generate gradient field up to 4.14 T/m utilizing the coupled field contributed by multiple electromagnet...
-
Magnetically-driven screws operating in soft-tissue environments could be used to deploy localized therapy or achieve minimally invasive interventions. In this work, we characterize the closed-loop behavior of magnetic screws in an agar gel tissue phantom using a permanent magnet-based robotic system with an open-configuration. Our closed-loop control strategy capitalizes on an analytical calculat...
-
The presence of any type of defect on the glass screen of smart devices has a great impact on their quality. We present a robust semi-supervised learning framework for intelligent micro-scaled localization and classification of defects on a 16K pixel image of smartphone glass. Our model features the efficient recognition and labeling of three types of defects: scratches, light leakage due to crack...
-
Functional nanomaterials possess exceptional multi-physical (e.g., mechanical, electrical and optical) properties compared with their bulk counterparts. To facilitate both synthesis and device applications of these nanomaterials, it is highly desired to characterize their multi-physical properties with high accuracy and efficiency. The nanomanipulation techniques under scanning electron microscopy...
-
Collaborative robotics allows merging the best capabilities of humans and robots to perform complex tasks. This allows the user to interact with remote and directly inaccessible environments such as the micro-scale world. This interaction is made possible by the bidirectional exchange of information (displacement - force) between the user and the environment through a haptic interface. The effecti...
-
Robust particle manipulation is a challenging but essential technique for single-cell analysis and processing of microfluidic devices. This paper proposes a micro-particle manipulation system with a microfluidic channel network. We built gravity-induced pressure actuators, which can generate high-resolution output pressure with a wide range so that the multiple particles can be delivered from the ...
-
By virtue of their ultra high resolution, scanning electron microscopes (SEMs) are essential to study topography, morphology, composition, and crystallography of materials, and thus are widely used for advanced researches in physics, chemistry, pharmacy, geology, etc. The major hindrance of using SEMs is that obtaining high quality images from SEMs requires a professional control of many control p...
-
In order to improve the convenience of operation for the medical assistive devices and reduce the use and maintenance cost, the Aruco recognition technology is applied to the navigation and positioning of visual guided electric assistive devices. Firstly, the differential control kinematic model of the electric wheelchair is analyzed. We discuss the feasibility of Aruco recognition technology in t...
-
Surgical navigation is challenging on complicated multi-branch structures such as intrarenal collecting systems or bronchi. The objective of this work is to help surgeons quickly establish the corresponding relationship between intraoperative endoscopic images and preoperative CT data. An endoscopic navigation method is proposed based on three-dimensional structure registration. It mainly includes...
-
Accurate volume segmentation from the Computed Tomography (CT) scan is a common prerequisite for pre-operative planning, intra-operative guidance and quantitative assessment of therapeutic outcomes in robot-assisted Minimally Invasive Surgery (MIS). 3D Deep Convolutional Neural Network (DCNN) is a viable solution for this task, but is memory intensive. Small isotropic patches are cropped from the ...
-
Intelligent vision is appealing in computer-assisted and robotic surgeries. Vision-based analysis with deep learning usually requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. We investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model live-cadaver GAN (LC-GAN) ...
-
This paper presents a technique to concurrently and jointly predict the future trajectories of surgical instruments and the future state(s) of surgical subtasks in robot-assisted surgeries (RAS) using multiple input sources. Such predictions are a necessary first step towards shared control and supervised autonomy of surgical subtasks. Minute-long surgical subtasks, such as suturing or ultrasound ...
-
For the time varying optimization problem, the tracking error cannot converge to zero at the finite time because of the optimal solution changing over time. This paper proposes a novel varying parameter recurrent neural network (VPRNN) based hierarchical optimization of a 7-DoF surgical manipulator for Robot-Assisted Minimally Invasive Surgery (RAMIS), which guarantees task tracking, Remote Center...
-
This paper proposes a magnetic needle steering controller to manipulate mesoscale magnetic suture needles for executing planned suturing motion. This is an initial step towards our research objective: enabling autonomous control of magnetic suture needles for suturing tasks in minimally invasive surgery. To demonstrate the feasibility of accurate motion control, we employ a cardinally-arranged fou...
-
The recent development of Robot-Assisted Minimally Invasive Surgery (RAMIS) has brought much benefit to ease the performance of complex Minimally Invasive Surgery (MIS) tasks and lead to more clinical outcomes. Compared to direct master-slave manipulation, semi-autonomous control for the surgical robot can enhance the efficiency of the operation, particularly for repetitive tasks. However, operati...
-
In magnetic resonance imaging (MRI) guided robotic catheter ablation procedures, reliable tracking of the catheter within the MRI scanner is needed to safely navigate the catheter. This requires accurate registration of the catheter to the scanner. This paper presents a differential, multi-slice image-based registration approach utilizing active fiducial coils. The proposed method would be used to...
-
One out of eight women will get breast cancer during their lifetime. A biopsy, a procedure in which a tissue sample is acquired from the lesion, is required to confirm the diagnosis. A biopsy is preferably executed under ultrasound (US) guidance because it is simple, fast, and cheap, gives real-time image feedback and causes little patient discomfort. However, Magnetic Resonance (MR)-detected lesi...
-
Robot-Assisted systems for Minimally Invasive Surgery enhance the surgeon capability, however, direct control over both the surgical tools and the endoscope results in an increased workload that leads to longer operation times. This work investigates the introduction of SCAN (System for Camera Autonomous Navigation) to overcome this limitation. An experimental study involving 12 participants was c...
-
One approach to control drug delivery in the cochlea is to use a magnetic microrobot powered by externally applied magnetic fields. However, it is necessary to integrate a localization system to ensure the precise navigation of the microrobot in the cochlear canal. To avoid integrating a clinical imaging modality for the navigation of microrobots in the cochlea, we propose in this work the applica...
-
As a novel therapy for peritoneal dissemination, it is desired to actualize an endoscopic photothermal therapy, which is minimally invasive and is highly therapeutically effective. However, since the endoscopic tumor temperature control has not been actualized, conventional therapies could damage healthy tissues by overhearing. In this paper, we develop a thermal endoscope system that controls the...
-
During the last two decades, minimally invasive surgery (MIS) has become popular because it offers advantages such as less pain, faster recovery, improved cosmesis, and reduced complications. Single-port laparoscopic surgery is a form of MIS where surgeons operate exclusively through a single entry. However, the view from the rigid endoscope is often obscured by the instruments which pass through ...
-
Hyper-redundant manipulators driven by cables are used in minimally invasive surgery because of their flexibility and small diameters. In particular, manipulators composed of many rigid links and joints have the advantages of high stiffness and payload. However, these manipulators have difficulty in estimating their positions and shapes using calculations based only on the kinematics model that as...
-
Automated surgical gestures classification and recognition are important precursors for achieving the goal of objective evaluation of surgical skills. Many works have been done to discover and validate metrics based on the motion of instruments that can be used as features for automatic classification of surgical gestures. In this work, we present a series of angular metrics that can be used toget...
-
Surgical knot tying is one of the most fundamental and important procedures in surgery, and a high-quality knot can significantly benefit the postoperative recovery of the patient. However, a longtime operation may easily cause fatigue to surgeons, especially during the tedious wound closure task. In this paper, we present a vision-based method to automate the suture thread grasping, which is a su...
-
There are many design trade-offs when building a magnetic manipulator to control millimeter-scale rotating magnetic swimmers for surgical applications.For example, increasing the magnitude of the flux density generated by the magnetic manipulator increases the torque applied to the swimmer, which could enable performing a wider variety of surgical tasks in the future. However, producing stronger m...
-
Circulating tumor cells (CTCs) are the primary cause of tumor metastasis after surgery. Metastatic tumor recurrence is the leading reason of cancer death. It is prerequisite to develop a platform for CTCs separation to predict the cancer cell transfer in important organs. Herein, a novel acoustic microfluidic device was designed to capture the "true" CTCs from the whole blood sample. The blood got...
-
Robot-assisted vitreoretinal surgery can filter surgeons' hand tremors and provide safe, accurate tool manipulation. In this paper, we report the design, optimization, and evaluation of a novel tilt mechanism for a new Steady-Hand Eye Robot (SHER). The new tilt mechanism features a four-bar linkage design and has a compact structure. Its kinematic configuration is optimized to minimize the require...
-
Currently, laparoscopic surgery systems are adapted for a large number of indications and patients and are therefore not optimized for one specific case. The challenge to create systems with an optimized kinematic structure for a specific patient regarding reachability and manipulability in the needed workspace is the automated design and construction process. We have developed an automated design...
-
Gastrointestinal (GI) endoscopy is a conventional and prevalent procedure used to diagnose and treat diseases in the digestive tract. This procedure requires inserting an endoscope equipped with a camera and instruments inside a patient to the target of interest. To manoeuvre the endoscope, an endoscopist would rotate the knob at the handle to change the direction of the distal tip and apply the f...
-
In this study, we developed a selective driving joint forceps (SDJF) for laparoscopic surgery. The SDJF has a mechanism that the driving joints can be selected arbitrarily, therefore each joint doesn't require an individual actuator for operating. The developed SDJF has six joints that can be operated using only four actuators. Each joint has 2-degrees-of-freedom (DOF) of flexion. Therefore, the S...
-
Many surgical robots with steerable surgical instruments have been proposed for endoscopic surgery. Surgical instruments should be small in size for insertion into the body and be able to handle large payloads such as tissue. Because the overall diameter and payload parameters are a trade-off, it is difficult to design an instrument with a large payload while reducing its diameter. In this paper, ...
-
The use of robots in minimally invasive surgery has improved the quality of standard surgical procedures. So far, only the automation of simple surgical actions has been investigated by researchers, while the execution of structured tasks requiring reasoning on the environment and the choice among multiple actions is still managed by human surgeons. In this paper, we propose a framework to impleme...
-
In this paper, an improved motion planning scheme is proposed for surgical robot control with multiple active constraints, including joint constraints, joint velocity constraints and remote center of motion constraints. It introduces an improved recurrent neural network (RNN) to optimize the online motion planning respect to multiple constraints. The demonstrated surgical operation trajectory is d...
-
In this paper we present a novel strategy for motion planning of autonomous robotic arms in Robotic Minimally Invasive Surgery (R-MIS). We consider a scenario where several laparoscopic tools must move and coordinate in a shared environment. The motion planner is based on a Model Predictive Controller (MPC) that predicts the future behavior of the robots and allows to move them avoiding collisions...
-
Force modulation of robotic manipulators has been extensively studied for several decades but is not yet commonly used in safety-critical applications due to a lack of accurate interaction contact modeling and weak performance guarantees - a large proportion of them concerning the modulation of interaction forces. This study presents a high-level framework for simultaneous trajectory optimization ...
-
Mitral regurgitation is one of the most common heart diseases caused by ventricular dysfunction or anatomic abnormality of the mitral valve. The fundamental treatment for mitral regurgitation is to repair/replace the mitral valve through open-heart surgery which is risky and requires more time to recover or through minimally invasive approaches, which have significant challenges and limitations. T...
-
Endometrial regeneration surgery is a new therapy for intrauterine adhesion (IUA). However, existing instruments lacking dexterity and compliance are with difficulty to successfully perform the tasks of generating transplant wounds and transplanting stem cells during endometrial regeneration surgery. This paper presents a novel shifted-routing continuum manipulator which is driven by only two cabl...
-
Fine needle insertions into a lung are challenging in terms of the needle deflection due to the breathing motion. Although previous related works neglected the effect for the needle deflection due to the breathing motion by patients stopping the breath during the insertion, they have to suffer from the discomfort. This paper proposes the intermittent insertion control method to decrease needle def...
-
Steerable needles have been widely researched in recent years, and they have multiple potential roles in the medical area. The flexibility and capability of avoiding obstacles allow the steerable needles to be applied in the biopsy, drug delivery and other medical applications that require a high degree of freedom and control accuracy. Radius of Curvature (ROC) of the needle while inserting in the...
-
This paper presents a smart continuum actuator based on a promising class of materials: ElectroActive polymer (EAP). Indeed these polymers undergo dimensional change in response to an applied electrical field and could be integrated directly in an endoscopic robot structure. We focuses on one of such materials, an electrostrictive polymer, for its valuable strain performances. An analytical model ...
-
This paper studies the contact stability and contact safety of a robotic intravascular cardiac catheter under blood flow disturbances while in contact with tissue surface. A probabilistic blood flow disturbance model, where the blood flow drag forces on the catheter body are approximated using a quasi-static model, is introduced. Using this blood flow disturbance model, probabilistic contact stabi...
-
Needle pose tracking is fundamental to achieve a precise and safe insertion in minimally-invasive percutaneous interventions. In this work, a method for estimating the full pose of steerable needles is presented, considering a four-segment Programmable Bevel-Tip Needle (PBN) as a case study. The method estimates also the torsion of the needle that can arise during the insertion because of the inte...
-
Haptic simulators can help medical students to train and improve their skills before practicing with a real patient. However, the vast majority of needle insertion haptic simulators are based on sophisticated models that are accurate but highly demanding in computing resources. Most of them do not provide haptic feedback and/or are not suitable for haptic control due to their computing time. In th...
-
Reinforcement Learning (RL) methods have demonstrated promising results for the automation of subtasks in surgical robotic systems. Since many trial and error attempts are required to learn the optimal control policy, RL agent training can be performed in simulation and the learned behavior can be then deployed in real environments. In this work, we introduce an open-source simulation environment ...
-
This paper presents a dynamic constraint formulation to provide protective virtual fixtures of 3D anatomical structures from polygon mesh representations. The proposed approach can anisotropically limit the tool motion of surgical robots without any assumption of the local anatomical shape close to the tool. Using a bounded search strategy and Principle Directed tree, the proposed system can run e...
-
Robot-assisted retinal surgery has become increasingly prevalent in recent years in part due to the potential for robots to help surgeons improve the safety of an immensely delicate and difficult set of tasks. The integration of robots into retinal surgery has resulted in diminished surgeon perception of tool-to-tissue interaction forces due to robot's stiffness. The tactile perception of these in...
-
A flexible endoscope introduces more dexterity to the image capturing in endoscopic surgery. However, manual control or automatic control based on instrument tracking does not handle the misorientation between the endoscopic video and the surgeon. We propose an automatic flexible endoscope control method that tracks the surgeon's head with respect to the object in the surgical scene. The robotic f...
-
This paper introduces an obstacle-crossing strategy, and the self-reconfiguration algorithm for a new class of modular robots called the rolling sphere, which can fit obstacles represented by cubes of different sizes due to the chain connection of multiple spheres. For the self-reconfiguration of the rolling spheres, a large gradient is obtained by classifying its action types and hierarchically m...
-
Modular robots are autonomous systems with variable morphology, composed of independent connected computational elements, called particles or modules. Due to critical resource constraints and limited capabilities, globally unique identifier (ID) assignment to each particle is a very challenging task in modular robots. However, having a unique ID in each one remains essential for various operations...
-
In this paper, we present a novel description for the configuration space of adaptive modular robots with a triangular structure based on extended binary trees. In general, binary trees can serve as a representation of kinematic trees with a maximum of two immediate descendants per element. Kinematic loops are incorporated in the tree structure by an ingenious extension of the binary tree indices....
-
Modular robots are defined as autonomous kinematic machines with variable morphology. They are composed of several thousands or even millions of modules which are able to coordinate in order to behave intelligently. Clustering the modules in modular robots has many benefits, including scalability, energy-efficiency, reducing communication delay and improving the self-configuration processes that f...
-
The modular assembly and actuation of 3D printed milliscale cuboid robots using a globally applied magnetic field is presented. Cuboids are composed of a rectangular resin shell embedded with two spherical permanent magnets that can independently align with any applied magnetic field. Placing cuboids within short distances of each other allows for modular assembly and disassembly by changing magne...
-
A fundamental challenge in the field of modular and collective robots is balancing the trade-off between unit-level simplicity, which allows scalability, and unit-level functionality, which allows meaningful behaviors of the collective. At the same time, a challenge in the field of soft robotics is creating untethered systems, especially at a large scale with many controlled degrees of freedom (DO...
-
Robotic self-assembly allows robots to join to form useful, on-demand structures. Unfortunately, the methods employed by most self-assembling robotic swarms compromise this promise of adaptability through their use of fixed docking locations, which impair a swarm's ability to handle imperfections in the structural lattice resulting from load deflection or imperfect robot manufacture; these concern...
-
We consider the problem of designing synthetic cells to achieve a complex goal (e.g., mimicking the immune system by seeking invaders) in a complex environment (e.g., the circulatory system), where they might have to change their control policy, communicate with each other, and deal with stochasticity including false positives and negatives-all with minimal capabilities and only a few bits of memo...
-
In this study, a passive arm-support exoskeleton was designed to provide assistive aid for manufacturing workers. The exoskeleton has two operating states which can be altered using an unique ratchet bar mechanism with two blocks fixed on the ratchet bar. When the upper arm is elevated to the highest poiont, the pawl module will touch the lower block to allow the pawl separated, so that the arm ca...
-
This paper presents electro-hydrostatic actuation as a valid substitute of electro-mechanical devices for powered knee prostheses. The work covers the design of a test rig exploiting linear electro-hydrostatic actuation. Typical control laws for prosthesis actuators are discussed, implemented and validated experimentally. Particularly, this work focuses on position and admittance control syntheses...
-
New challenges arise when investigating the use of active prostheses for lower limb replacement, such as high motor power requirements, leading to increased weight and reduced autonomy. Series and parallel elasticity are often explored to reduce the necessary motor power but often the effect on the energy consumption of the prosthesis is not directly investigated, as the mechanical power propertie...
-
The operational space of an 8-axis arm exoskeleton is partitioned into "tasks" based on the human arm motion, and a task priority approach is implemented to perform the inverse kinematics. The tasks are prioritized in the event that singularities or other constraints such as joint limits render the full desired operational space infeasible. The task reconstruction method is used to circumvent sing...
-
The human hand serves as an inspiration for robotic grippers. However, the dimensions of the human hand evolved under a different set of constraints and requirements than that of robots today. This paper discusses a method of kinematically optimizing the design of an anthropomorphic robotic hand. We focus on maximizing the workspace intersection of the thumb and the other fingers as well as maximi...
-
In this study, we address the inverse kinematics problem for an upper-limb exoskeleton by presenting a novel method that guarantees the satisfaction of joint-space constraints, and solves closed-chain mechanisms in a serial robot configuration. Starting from the conventional differential kinematics method based on the inversion of the Jacobian matrix, we describe and test two improved algorithms b...
-
Several powered exoskeletons have been developed and commercialized to assist people with complete spinal cord injury. For motion control of a powered exoskeleton, a normal gait pattern is often applied as a reference. However, the physical ability of paraplegics and the degrees of freedom of powered exoskeletons are totally different from those of people without disabilities. Therefore, this pape...
-
Gait Training Robot with Intermittent Force Application based on Prediction of Minimum Toe Clearance
Adaptive assistance of gait training robots has been determined to improve gait performance through motion assistance. An important control role during walking is to avoid tripping by controlling minimum toe clearance (MTC), which is an indicator of tripping risk, to avoid its decrease among gait cycles. No conventional gait training robots can adjust assistance timing based on MTC. In this paper,...
-
Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LINECOSPAR, a human-in-the-loop preference-based fram...
-
Evidence suggests that the metabolic cost associated with the locomotive activity of walking is dependent upon ankle stiffness. This stiffness can be a control parameter in an ankle-foot prosthesis. Considering unique physical interaction between each individual with below-knee amputation and robotic ankle-foot prosthesis, individually tuned stiffness in a robotic ankle-foot prosthesis may improve...
-
This paper describes a novel low-level controller for the lower-limb exoskeleton Atalante. The controller implemented on the commercialized product Atalante works under the assumption of full rigidity, performing position control through decentralized joint PIDs. However, this controller is unable to tackle the presence of flexibilities in the system, which cause static errors and undesired oscill...
-
This paper outlines steps toward a framework for model-based user intent detection to enable fluent human-robot interaction in assistive exoskeletons. An interacting multi-model (IMM) estimation scheme is presented to address state estimation for lower-extremity exoskeletons and to handle their hybrid dynamics. The proposed IMM scheme includes new approaches that enable it to estimate states of hy...
-
To obtain synchronized gait assistance, this paper presents a new delayless adaptive dual-oscillator (ADO) scheme to address the inherent delay issue. In the ADO structure, a new oscillator is coupled with the primitive one but the phase is adaptively feed-forward compensated. It’s remarkable that the compensated phase is determined by the proposed extended phase lag observer, in which both the ph...
-
In order to fluidly perform complex tasks in collaboration with a human being, such as table handling, a humanoid robot has to recognize and adapt to human movements. To achieve such goals, a realistic model of the human locomotion that is computable on a robot is needed. In this paper, we focus on making a humanoid robot follow a human-like locomotion path. We mainly present two models of human w...
-
Despite plenty of motion planning strategies have been proposed for bipedal locomotion, enhancing the walking robustness in real-world environments is still an open question. This paper focuses on robust body and leg trajectories synthesis through integrating constrained optimization with imitation learning. Specifically, we first propose a Quadratically Constrained Quadratic Programming (QCQP) al...
-
In this paper we examine a novel method of core-located actuation that we believe can be used to vary gaits in a compass-gait walker, using critical analysis of a ball-in-tray mechanism to apply forces at the robot's "pelvis". The dynamic equations of motion of a tilting ball-tray system with several design parameters are developed and simulated for various tray designs. Results show that changes ...
-
Most state-of-the-art bipedal robots are designed to be anthropomorphic and therefore possess legs with knees. Whilst this facilitates more human-like locomotion, there are implementation issues that make walking with straight or near-straight legs difficult. Most bipedal robots have to move with a constant bend in the legs to avoid singularities at the knee joints, and to keep the centre of mass ...
-
A control algorithm that allows a human model to crawl using a pair of supernumerary robotic limbs (SuperLimbs) is presented. The human model and SuperLimbs are coupled by a compliant harness. This work is inspired by the need for wearable robotic systems that can support workers engaged in fatiguing tasks. The walking policy is developed based on Lyapunov analysis. The volume of the region of att...
-
This paper addresses reactive generation of step time and location of biped robots for balance recovery against a severe push. Key idea is to reformulate the balance recovery problem into a tracking problem for "hybrid" inverted pendulum model of the biped, where taking a new step implicitly yields a discrete jump of the tracking error. This interpretation offers a Lyapunov-based approach to react...
-
In this paper, we present a sequential motion planning and control method for generating somersaults on bipedal robots. The somersault (backflip or frontflip) is considered as a coupling between an axile hopping motion and a rotational motion about the center of mass of the robot; these are encoded by a hopping Spring-loaded Inverted Pendulum (SLIP) model and the rotation of a Flywheel, respective...
-
Compliance is important for humanoid robots, especially a position-controlled one, to perform tasks in complicated environments where unexpected or sudden contacts will result in large impacts which may cause instability or destroy the hardware of robots. This paper presents a compliance control method based on viscoelastic model for humanoid robots to survive on these conditions. The viscoelastic...
-
In this paper, we propose the stabilization strategy for a soft landing in a biped walking using impedance control and the optimization-based whole-body control framework. Even though proper contact forces and desired trajectories of the robot are given, the robot can be unstable easily if unexpected forces are applied to the robot or impulsive contact force is produced in the landing state while ...
-
Deformable objects are very common around us in our daily life. Because they have infinitely many degrees of freedom, they present a challenging problem in robotics. Inspired by practical industrial applications, we present in this paper our research on using a humanoid robot to take a long, thin and flexible belt out of a bobbin and pick up the bending part of the belt from the ground. By proposi...
-
Real-world applications require light-weight, energy-efficient, fully autonomous robots. Yet, increasing autonomy is oftentimes synonymous with escalating computational requirements. It might thus be desirable to offload intensive computation—not only sensing and planning, but also low-level whole-body control—to remote servers in order to reduce on-board computational needs. Fifth Generation (5G)...
-
The musculoskeletal humanoid has various biomimetic benefits, and the redundant muscle arrangement is one of its most important characteristics. This redundancy can achieve fail-safe redundant actuation and variable stiffness control. However, there is a problem that the maximum joint angle velocity is limited by the slowest muscle among the redundant muscles. In this study, we propose two methods...
-
The recent progress of an ongoing project utilizing a ducted-fan propulsion system to improve a humanoid robot's ability to step over large ditches is reported. A novel method (GAS) based on the genetic algorithm with smoothness constraint can effectively minimize the thrust by optimizing the robot's posture during 3D stepping. The significant advantage of the method is that it can realize the con...
-
The musculoskeletal humanoid has various biomimetic benefits, and it is important that we can embed and evaluate human reflexes in the actual robot. Although stretch reflex has been implemented in lower limbs of musculoskeletal humanoids, we apply it to the upper limb to discover its useful applications. We consider the implementation of stretch reflex in the actual robot, its active/passive appli...
-
In this work, we propose a system for humanoid robot fast motions. When a humanoid robot performs a motion such as a tennis forehand stroke motion, a whole-body fast motion in reaction to visual information is required. There are three problems to tackle. (1) Motion is desired to be quick. (2) Real-time visual processing considering visual noises is needed. (3) Real-time joint angle modification w...
-
A long-standing argument in model-based control of locomotion is about the level of complexity that a model should have to define a behavior such as running. Even though a goldilocks model based on biomechanical evidence is often sought, it is unclear what level of complexity qualifies to be such a model. This dilemma deepens further for bipedal robotic running with point feet, since these robots ...
-
We consider the motion planning problem of a hopper navigating a terrain comprising stepping stones while optimizing an energy metric. The most widely used approach of discrete searches (e.g., A-star) cannot handle boundary conditions (e.g., end path constraints on position, velocity). However, continuous optimizations can easily deal with the boundary value problem but are not widely used in moti...
-
Robots encounter many risks that threaten the success of practical locomotion tasks. Legs break, electrical components overheat, and feet can unexpectedly slip. When all risks cannot be completely avoided, how does a robot decide its best action? We present a method for planning robot motions by reasoning about risk-of-failure probabilities instead of applying cost-penalty functions or inflexible ...
-
Although they take many forms, legged robots rely upon springs to achieve high speed, dynamic locomotion. In this paper we examine the effect of adding parallel springs to robots that rely on virtual compliance. Specifically, we consider the trade-off between energetic efficiency and leg versatility that comes while using Parallel Elastic Actuators (PEAs). To do this, we vary the ratio of physical...
-
The ability of legged systems to traverse highly- constrained environments depends by and large on the performance of their motion and balance controllers. This paper presents a controller that excels in a scenario that most state- of-the-art balance controllers have not yet addressed: line walking, or walking on nearly null support regions. Our approach uses a low-dimensional virtual model (2-DoF...
-
Continuous robot operation in extreme scenarios such as underground mines or sewers is difficult because exteroceptive sensors may fail due to fog, darkness, dirt or malfunction. So as to enable autonomous navigation in these kinds of situations, we have developed a type of proprioceptive localization which exploits the foot contacts made by a quadruped robot to localize against a prior map of an ...
-
Animal-level agility and robustness in robots cannot be accomplished by solely relying on blind locomotion controllers. A significant portion of a robot’s ability to traverse terrain comes from reacting to the external world through visual sensing. However, embedding the sensors and compute that provide sufficient accuracy at high speeds is challenging, especially if the robot has significant spac...
-
This paper shows how CasADi’s state-of-the-art implementation of algorithmic differentiation can be leveraged to formulate and efficiently solve gait optimization problems, enabling rapid gait design for high-dimensional biped robots. Comparative studies on a 7-DOF planar biped show that CasADi generates optimal gaits 4 times faster than another existing advanced optimization package. The framewor...
-
Legged robots have great potential to perform complex loco-manipulation tasks, yet it is challenging to keep the robot balanced while it interacts with the environment. In this paper we investigated the use of additional contact points for maximising the robustness of loco-manipulation motions. Specifically, body-ground contact was studied for its ability to enhance robustness and manipulation cap...
-
This article introduces an innovative model-based strategy for designing a legged robot to generate animal-like running dynamics with differentiated leg braking and thrusting force patterns. Linear springs were utilized as legs, but instead of having one end of each spring connected directly to the hip joint, one extra bar was added to offset the spring's direction. The robot's front and hind legs...
-
Current techniques in robot design and fabrication are time consuming and costly. Robot designs are needed that facilitate low-cost fabrication techniques and reduce the design to production timeline. Here we present an axial-rotational coupled metastructure that can serve as the functional core of a low-cost 3D printed walking robot. Using an origami-inspired assembly technique, the axial-rotatio...
-
Underactuated designs of robot limbs can enable these systems to passively adapt their joint configuration in response to external forces. Passive adaptation and reconfiguration can be extremely beneficial in situations where manipulation or locomotion with complex substrates is required. A common design for underactuated systems often involves a single tendon that actuates multiple rotational joi...
-
Creating robots with emotional personalities will transform the usability of robots in the real-world. As previous emotive social robots are mostly based on statically stable robots whose mobility is limited, this paper develops an animation to real-world pipeline that enables dynamic bipedal robots that can twist, wiggle, and walk to behave with emotions. First, an animation method is introduced ...
-
For mitigating joint impact torques, researchers have reduced joint stiffness by series elastic actuators, reflected inertia by low gear ratios, and friction torque from drive-trains. However, these impact mitigation methods may impair the control performance of contact forces or may increase motor and robot mass. This paper proposes a design method for achieving a balance between impact mitigatio...
-
Truss structures can be found in many buildings and civil infrastructure such as bridges and towers. But as these architectures age, their maintenance is required to keep them structurally sound. A legged robotic solution capable of climbing these structures for maintenance is sought, but determining the size and shape of such a robot to maximise structure coverage is a challenging task. This pape...
-
Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation ope...
-
Quadrupedal locomotion skills are challenging to develop. In recent years, deep Reinforcement Learning promises to automate the development of locomotion controllers and map sensory observations to low-level actions. Moreover, the full robot dynamics model can be exploited, but no model-based simplifications are to be made. In this work, a method for developing controllers for the Laelaps II robot...
-
In the field of quadruped locomotion, dynamic locomotion behavior, and rich integration with sensory feedback represents a significant development. In this paper, we present an efficient neural model, which includes CPG and its sensorimotor coordination, and demonstrate its implementation in a quadruped robot to show how efficient integration of motor and sensory feedback can generate dynamic beha...
-
Spiking neurons might play a larger role than simply as an efficient signal transmitter. Several studies have demonstrated how movements can be generated using networks of spiking neurons. However, the complexity of spiking neural networks makes their implementation difficult, and the use of spiking neurons in robotics has remained largely impractical. In this paper, we show that the addition of a...
-
The purpose of this study is to make life-sized humanoid robots acquire tool manipulation skills that require complicated force adjustment. The difficulty in acquisition of tool manipulation skills comes from the hardship in physical modeling. Recent research have revealed that deep reinforcement learning (DRL), a model-free approach, performs superior in such tasks. However, DRL in general has a ...
-
This paper presents novel control techniques for passivation and stabilisation of floating-base systems with contacts, whose dynamical models comprise both joint-space, and Cartesian floating-base coordinates. The aforementioned results are achieved using both minimally model-based, and completely model-free controllers that employ power-shaping signals. Model-free control is permitted through usa...
-
Human jumping involves not only lower limbs but also whole-body coordination. During jumping, the effect of sinking the center of mass for recoil and arm swing are significant, and they can cause changes in the jump height. However, upper body movements during jumping movements of humanoid robots have not been studied adequately. When jumping involves only the lower limbs, the burden on the lower ...
-
This work presents a motion planning algorithm for legged robots capable of constructing long-horizon dynamic plans in real-time. Many existing methods use models that prohibit flight phases or even require static stability, while those that permit these dynamics often plan over short horizons or take minutes to compute. The algorithm presented here resolves these issues through a reduced-order dy...
-
This paper presents a framework providing a full pipeline to execute a complex physical interaction behaviour of a humanoid bipedal robot, both from a theoretical and a practical standpoint. Building from a multi-contact control architecture that combines contact planning and reactive force distribution capabilities, the main contribution of this work consists in the integration of a sample-based ...
-
A robot cannot lift up an object if it is not feasible to do so. However, in most research on robot lifting, "feasibility" is usually presumed to exist a priori. This paper proposes a three-step method for a humanoid robot to reason about the feasibility of lifting a heavy box with physical properties that are unknown to the robot. Since feasibility of lifting is directly related to the physical p...
-
Performing large step-ups is a challenging task for a humanoid robot. It requires the robot to perform motions at the limit of its reachable workspace while straining to move its body upon the obstacle. This paper presents a non-linear trajectory optimization method for generating step-up motions. We adopt a simplified model of the centroidal dynamics to generate feasible Center of Mass trajectori...
-
Wheeled-legged robots combine the efficiency of wheeled robots when driving on suitably flat surfaces and versatility of legged robots when stepping over or around obstacles. This paper introduces a planning and control framework to realise dynamic locomotion for wheeled biped robots. We propose the Cart-Linear Inverted Pendulum Model (Cart-LIPM) as a template model for the rolling motion and the ...
-
Most current bipedal robots were modeled with an assumption that there is no slip between the stance foot and ground. This paper relaxes that assumption and undertakes a comprehensive study of a compass gait biped with foot slipping. It is found that slips are most likely to happen near impact for a broad range of gaits. Among these gaits, ones with a backward swing foot velocity relative to the g...
-
Passive dynamic walking is a model that walks down a shallow slope without any control or input. This model has been widely used to investigate how stable walking is generated from a dynamic viewpoint, which is useful to provide design principles for developing energy-efficient biped robots. However, the basin of attraction is very small and thin, and it has a fractal-like complicated shape. This ...
-
This paper aims to develop time-varying virtual constraint controllers that allow stable and agile bounding gaits for full-order hybrid dynamical models of quadrupedal locomotion. As opposed to state-based nonlinear controllers, time-varying controllers can initiate locomotion from zero velocity. Motivated by this property, we investigate the stability guarantees that can be provided by the time-v...
-
The authors have investigated underactuated locomotion robots with an inner wobbling mass, it is discovered that the wobbling mass controls the gait speed by entrainment. Supplying the wobbling from outside, outer wobbling entrains load objects and controls the transferring speed. In this paper, we propose a vibratory conveyor system based on the frequency entrainment of a limit cycle walker. The ...
-
This paper proposes a multi-task control strategy for a quadruped robot named THU-QUAD II. The mechanical design of the robot ensures a wide range of motion for all joints, which allows it to stand and walk like a mammal as well as sprawl to the ground and crawl like a reptile. Five basic leg configurations are defined for the robot, including four mammal-type configurations with bidirectional kne...
-
This paper describes the design, control and initial experimental results of the quadruped robot LLAMA. Designed to operate in a human-scale world, this 67kg-class, all-electric robot is capable of rapid motion over a variety of terrains. Thanks to a unique leg configuration and custom high-torque, low gear-ratio motors, it can move omnidirectionally at speeds over 1 m/s. A hierarchical reactive c...
-
A floating base system is inevitably to contact the environment while it is moving. This paper explores the contact force estimation and regulation algorithm for a position-controlled floating base system without joint torque information. First, the joint space dynamic model of the system is presented and transformed into the contact space. Then, the inverse dynamics method is employed to estimate...
-
Although legged robots are becoming more nonlinear with higher degrees of freedom (DOFs), the centralized nonlinear control methods required to achieve stable locomotion cannot scale with the dimensionality of these robots. This paper investigates time-varying decentralized feedback control architectures based on hybrid zero dynamics (HZD) that stabilize dynamic legged locomotion with high degrees...
-
This paper presents a constrained nonlinear model predictive control (NMPC) framework for legged locomotion. The framework assumes a legged robot as a floating base single rigid body with contact forces being applied to the body as external forces. With consideration of orientation dynamics evolving on the rotation manifold SO(3), analytic Jacobians which are necessary for constructing the gradien...
-
An important issue when synthesizing legged locomotion plans is the combinatorial complexity that arises from gait pattern selection. Though it can be defined manually, the gait pattern plays an important role in the feasibility and optimality of a motion with respect to a task. Replacing human intuition with an automatic and efficient approach for gait pattern selection would allow for more auton...
-
This paper proposes a kinodynamic motion plan-ning framework for multi-legged robot jumping based on the mixed-integer convex program (MICP), which simultaneously reasons about centroidal motion, contact points, wrench, and gait sequences. This method uniquely combines configuration space discretization and the construction of feasible wrench polytope (FWP) to encode kinematic constraints, actuato...
-
Can we design motion primitives for complex legged systems uniformly for different terrain types without neglecting modeling details? This paper presents a method for rapidly generating quadrupedal locomotion on sloped terrains-from modeling to gait generation, to hardware demonstration. At the core of this approach is the observation that a quadrupedal robot can be exactly decomposed into coupled...
-
Elaborate trajectory optimization models with many degrees of freedom can be a useful locomotion-planning tool, as they provide rich solutions that take advantage of the robot's specific morphology. They are, however, prone to falling into local minima. Depending on the seed that initializes the solver, the trajectories themselves and the extent to which they minimize the cost function can vary wi...
-
As a strategy to address the difficulties encountered when modeling and controlling a musculoskeletal system, we present a straightforward implementation of an autonomous decentralized motion control system in this paper; the system is inspired by the spinal reflex system of animals. We developed an artificial receptor, a muscle, and a neuron to mechanically implement the reflex mechanisms of anim...
-
In this paper, we present the second version of a reconfigurable modular legged robot, Snapbot V2. The mechanical design of Snapbot V2 is enhanced for better dynamic performance and robust connection with modular legs. A motion generator for locomotion is developed to achieve various locomotion skills in one to six-leg configurations. The locomotion is tested on a multi-body dynamic simulation mod...
-
Passive sensing with ambient WiFi signals is a promising technique that will enable new types of human-robot interactions while preserving users' privacy. Here, we present PresSense, a system for human respiration sensing in noisy environments. Unlike existing WiFi-based respiration sensors, we employ a human presence detector, improving the robustness in scenarios where no human is present in an ...
-
Humans and robots are increasingly sharing their workspaces to benefit from the precision, endurance, and strength of machines and the universal capabilities of humans. Instead of performing time-consuming real experiments, computer simulations of humans could help to optimally orchestrate human and robotic tasks—either for setting up new production cells or by optimizing the motion planning of al...
-
A reduced balance ability can lead to falls and critical injuries. To prevent falls, humans use reaction forces and torques generated by swinging their arms. In animals, we can find that a similar strategy is taken using tails. Inspired by these strategies, we propose an approach that utilizes a robotic appendage as a human balance supporter without assistance from environmental contact. As a proo...
-
To function effectively in real-world environments, powered wearable robots such as exoskeletons and robotic prostheses must recognize the user's motion intent by detecting the user's locomotion modes such as walking, stair ascent and descent or ramp ascent and descent. Traditionally, intent detection is achieved using rule based methods such as state machines or fuzzy logic using data from wearab...
-
Human beings and animals are capable of recognizing places from a previous journey when viewing them under different environmental conditions (e.g., illuminations and weathers). This paper seeks to provide robots with a human-like place recognition ability using a new point cloud feature learning method. This is a challenging problem due to the difficulty of extracting invariant local descriptors ...
-
As an essential component of visual simultaneous localization and mapping (SLAM), place recognition is crucial for robot navigation and autonomous driving. Existing methods often formulate visual place recognition as feature matching, which is computationally expensive for many robotic applications with limited computing power, e.g., autonomous driving and cleaning robot. Inspired by the fact that...
-
In this paper, we introduce an ambiguity-aware robust active SLAM (ARAS) framework that makes use of multi-hypothesis state and map estimations to achieve better robustness. Ambiguous measurements can result in multiple probable solutions in a multi-hypothesis SLAM (MH-SLAM) system if they are temporarily unsolvable (due to insufficient information), our ARAS aims at taking all these probable esti...
-
This paper presents a proof-of-concept for a localization and mapping system for magnetic crawlers performing inspection tasks on structures made of large metal plates. By relying on ultrasonic guided waves reflected from the plate edges, we show that it is possible to recover the plate geometry and robot trajectory to a precision comparable to the signal wavelength. The approach is tested using r...
-
Simultaneous Localization and Mapping (SLAM) is considered a mature research field with numerous applications and publicly available open-source systems. Despite this maturity, existing SLAM systems often rely on ad-hoc implementations or are tailored to predefined sensor setups. In this work, we tackle these issues, proposing a novel unified SLAM architecture specifically designed to standardize ...
-
In this paper, we consider the problem of distributed pose graph optimization (PGO) that has extensive applications in multi-robot simultaneous localization and mapping (SLAM). We propose majorization minimization methods for distributed PGO and show that our methods are guaranteed to converge to first-order critical points under mild conditions. Furthermore, since our methods rely a proximal oper...
-
The ability to infer map variables and estimate pose is crucial to the operation of autonomous mobile robots. In most cases the shared dependency between these variables is modeled through a multivariate Gaussian distribution, but there are many situations where that assumption is unrealistic. Our paper shows how it is possible to relax this assumption and perform simultaneous localization and map...
-
In this work, we present a novel method for reconstructing particular 3D surface points using an imaging sonar sensor. We derive the two-dimensional Fermat flow equation, which may be applied to the planes defined by each discrete azimuth angle in the sonar image. We show that the Fermat flow equation applies to boundary points and surface points which correspond to specular reflections within the...
-
Motivated by the goal of achieving robust, drift-free pose estimation in long-term autonomous navigation, in this work we propose a methodology to fuse global positional information with visual and inertial measurements in a tightly-coupled nonlinear-optimization-based estimator. Differently from previous works, which are loosely-coupled, the use of a tightly-coupled approach allows exploiting the...
-
In recent years, many excellent SLAM methods based on cameras, especially the camera-IMU fusion (VIO), have emerged, which has greatly improved the accuracy and robustness of SLAM. However, we find through experiments that most of the existing VIO methods perform well on drones or drone datasets, but for ground robots on complex terrain, they cannot continuously provide accurate and robust localiz...
-
Introducing object-level semantic information into simultaneous localization and mapping (SLAM) system is critical. It not only improves the performance but also enables tasks specified in terms of meaningful objects. This work presents OrcVIO, for visual-inertial odometry tightly coupled with tracking and optimization over structured object models. OrcVIO differentiates through semantic feature a...
-
Multi-sensor fusion of multi-modal measurements from commodity inertial, visual and LiDAR sensors to provide robust and accurate 6DOF pose estimation holds great potential in robotics and beyond. In this paper, building upon our prior work (i.e., LIC-Fusion), we develop a sliding-window filter based LiDAR-Inertial-Camera odometry with online spatiotemporal calibration (i.e., LIC-Fusion 2.0), which...
-
With monocular Visual-Inertial Odometry (VIO) system, 3D point cloud and camera motion can be estimated simultaneously. Because pure sparse 3D points provide a structureless representation of the environment, generating 3D mesh from sparse points can further model the environment topology and produce dense mapping. To improve the accuracy of 3D mesh generation and localization, we propose a tightl...
-
We present SplitFusion, a novel dense RGB-D SLAM framework that simultaneously performs tracking and dense reconstruction for both rigid and non-rigid components of the scene. SplitFusion first adopts deep learning based semantic instant segmentation technique to split the scene into rigid or non-rigid surfaces. The split surfaces are independently tracked via rigid or non-rigid ICP and reconstruc...
-
We propose a framework for tightly-coupled lidar inertial odometry via smoothing and mapping, LIO-SAM, that achieves highly accurate, real-time mobile robot trajectory estimation and map-building. LIO-SAM formulates lidar-inertial odometry atop a factor graph, allowing a multitude of relative and absolute measurements, including loop closures, to be incorporated from different sources as factors i...
-
This paper proposes a 3D LiDAR simultaneous localization and mapping (SLAM) method that improves accuracy, robustness, and computational efficiency for an iterative closest point (ICP) algorithm employing a locally approximated geometry with clusters of normal distributions. In comparison with previous normal distribution-based ICP methods, such as normal distribution transformation and generalize...
-
Detecting loop closures in 3D Light Detection and Ranging (LiDAR) data is a challenging task since point-level methods always suffer from instability. This paper presents a semantic-level approach named GOSMatch to perform reliable place recognition. Our method leverages novel descriptors, which are generated from the spatial relationship between semantics, to perform frame description and data as...
-
Place recognition is essential for SLAM system since it is critical for loop closure and can help to correct the accumulated drift and result in a globally consistent map. Unlike the visual slam which can use diverse feature detection methods to describe the scene, there are limited works reported to represent a place using single LiDAR scan. In this paper, we propose a segmentation-based egocentr...
-
Numerous Simultaneous Localization and Mapping (SLAM) algorithms have been presented in last decade using different sensor modalities. However, robust SLAM in extreme weather conditions is still an open research problem. In this paper, RadarSLAM, a full radar based graph SLAM system, is proposed for reliable localization and mapping in large-scale environments. It is composed of pose tracking, loc...
-
GP-SLAM+: real-time 3D lidar SLAM based on improved regionalized Gaussian process map reconstruction
This paper presents a 3D lidar SLAM system based on improved regionalized Gaussian process (GP) map reconstruction to provide both low-drift state estimation and mapping in real-time for robotics applications. We utilize spatial GP regression to model the environment. This tool enables us to recover surfaces including those in sparsely scanned areas and obtain uniform samples with uncertainty. Tho...
-
State representation learning (SRL) in partially observable Markov decision processes has been studied to learn abstract features of data useful for robot control tasks. For SRL, acquiring domain-agnostic states is essential for achieving efficient imitation learning. Without these states, imitation learning is hampered by domain-dependent information useless for control. However, existing methods...
-
General-purpose trajectory planning algorithms for automated driving utilize complex reward functions to perform a combined optimization of strategic, behavioral, and kinematic features. The specification and tuning of a single reward function is a tedious task and does not generalize over a large set of traffic situations. Deep learning approaches based on path integral inverse reinforcement lear...
-
We consider the problem of visual imitation learning without human kinesthetic teaching or teleoperation, nor access to an interactive reinforcement learning training environment. We present a geometric perspective to this problem where geometric feature correspondences are learned from one training video and used to execute tasks via visual servoing. Specifically, we propose VGS-IL (Visual Geomet...
-
We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less ma...
-
Existing architectures for imitation learning using image-to-action policy networks perform poorly when presented with an input image containing multiple instances of the object of interest, especially when the number of expert demonstrations available for training are limited. We show that end-to-end policy networks can be trained in a sample efficient manner by (a) appending the feature map outp...
-
We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, stochastic, nonlinear dynamics. Our approach extends the Normalizing Flows framework to learn stable Stochastic Differential Equations. We prove the Lyapunov stability for a class of Stochastic Differential Equations and we propose a learning algorithm to learn them from a set of demonstrated tr...
-
Deep Generative Models such as Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE) have found multiple applications in Robotics, with recent works suggesting the potential use of these methods as a generic solution for the estimation of sampling distributions for motion planning in parameterized sets of environments. In this work we provide a first empirical study of challenge...
-
This paper presents a teaching by demonstration method for contact tasks with periodic movement on planar surfaces of unknown pose. To learn the motion on the plane, we utilize frequency oscillators with periodic movement primitives and we propose modified adaptation rules along with an extraction method of the task’s fundamental frequency by automatically discarding near-zero frequency components...
-
Sudden changes in the dynamics of robotic tasks, such as contact with an object or the latching of a door, are often viewed as inconvenient discontinuities that make manipulation difficult. However, when these transitions are well-understood, they can be leveraged to reduce uncertainty or aid manipulation-for example, wiggling a screw to determine if it is fully inserted or not. Current model-free...
-
Being able to quickly adapt to changes in dynamics is paramount in model-based control for object manipulation tasks. In order to influence fast adaptation of the inverse dynamics model's parameters, data efficiency is crucial. Given observed data, a key element to how an optimizer updates model parameters is the loss function. In this work, we propose to apply meta-learning to learn structured, s...
-
Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain ...
-
Planar pushing remains a challenging research topic, where building the dynamic model of the interaction is the core issue. Even an accurate analytical dynamic model is inherently unstable because physics parameters such as inertia and friction can only be approximated. Data-driven models usually rely on large amounts of training data, but data collection is time consuming when working with real r...
-
This paper introduces a new technique for learning probabilistic models of mass and friction distributions of unknown objects, and performing robust sliding actions by using the learned models. The proposed method is executed in two consecutive phases. In the exploration phase, a table-top object is poked by a robot from different angles. The observed motions of the object are compared against sim...
-
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces. In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own performance from real-world experience without requiring human supervision. To this end, we propose a...
-
A key challenge with controlling complex dynamical systems is to accurately model them. However, this requirement is very hard to satisfy in practice. Data-driven approaches such as Gaussian processes (GPs) have proved quite effective by employing regression based methods to capture the unmodeled dynamical effects. However, GPs scale cubically with number of data points n, and it is often a challe...
-
Locomotion is a prime example for adaptive behavior in animals and biological control principles have inspired control architectures for legged robots. While machine learning has been successfully applied to many tasks in recent years, Deep Reinforcement Learning approaches still appear to struggle when applied to real world robots in continuous control tasks and in particular do not appear as rob...
-
Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In addition, kinodynamic constraints are often non-differentiable and difficult to implement in an optimisation approach. In this work, these challenges are addressed by framing quadruped control as o...
-
Humans use simple probing actions to develop intuition about the physical behavior of common objects. Such intuition is particularly useful for adaptive estimation of favorable manipulation strategies of those objects in novel contexts. For example, observing the effect of tilt on a transparent bottle containing an unknown liquid provides clues on how the liquid might be poured. It is desirable to...
-
Humans learn about object properties using multiple modes of perception. Recent advances show that robots can use non-visual sensory modalities (i.e., haptic and tactile sensory data) coupled with exploratory behaviors (i.e., grasping, lifting, pushing, dropping, etc.) for learning objects' properties such as shape, weight, material and affordances. However, non-visual sensory representations cann...
-
Applying Deep Reinforcement Learning (DRL) to complex tasks in the field of robotics has proven to be very successful in the recent years. However, most of the publications focus either on applying it to a task in simulation or to a task in a real world setup. Although there are great examples of combining the two worlds with the help of transfer learning, it often requires a lot of additional wor...
-
Zero-shot sim-to-real transfer of tasks with complex dynamics is a highly challenging and unsolved problem. A number of solutions have been proposed in recent years, but we have found that many works do not present a thorough evaluation in the real world, or underplay the significant engineering effort and task-specific fine tuning that is required to achieve the published results. In this paper, ...
-
We explore using reinforcement learning on single and multi-agent systems such that after learning is finished we can apply a policy zero-shot to new environment sizes, as well as different number of agents and entities. Building off previous work, we show how to map back and forth between the state and action space of a standard Markov Decision Process (MDP) and multi-dimensional tensors such tha...
-
We present TrueÆdapt, a model-free method to learn online adaptations of robot trajectories based on their effects on the environment. Given sensory feedback and future waypoints of the original trajectory, a neural network is trained to predict joint accelerations at regular intervals. The adapted trajectory is generated by linear interpolation of the predicted accelerations, leading to continuou...
-
Learning from demonstration (LfD) is an intuitive framework allowing non-expert users to easily (re-)program robots. However, the quality and quantity of demonstrations have a great influence on the generalization performances of LfD approaches. In this paper, we introduce a novel active learning framework in order to improve the generalization capabilities of control policies. The proposed approa...
-
Conventional robot programming methods are not suited for non-experts to intuitively teach robots new tasks. For this reason, the potential of collaborative robots for production cannot yet be fully exploited. In this work, we propose an active learning framework, in which the robot and the user collaborate to incrementally program a complex task. Starting with a basic model, the robot's task know...
-
How can we learn representations for planning that are both efficient and flexible? Task and motion planning models are a good candidate, having been very successful in long-horizon planning tasks-however, they've proved challenging for learning, relying mostly on hand-coded representations. We present a framework for learning constraint-based task and motion planning models using gradient descent...
-
We present a novel learning and control framework that combines artificial neural networks with online trajectory optimization to learn dexterous manipulation skills from human demonstration and to transfer the learned behaviors to real robots. Humans can perform the demonstrations with their own hands and with real objects. An instrumented glove is used to record motions and tactile data. Our sys...
-
In this paper, we consider autonomous driving of a vehicle using imitation learning. Generative adversarial imitation learning (GAIL) is a widely used algorithm for imitation learning. This algorithm leverages positive demonstrations to imitate the behavior of an expert. In this paper, we propose a novel method, called mixed generative adversarial imitation learning (MixGAIL), which incorporates b...
-
Safe autonomous driving requires robust detection of other traffic participants. However, robust does not mean perfect, and safe systems typically minimize missed detections at the expense of a higher false positive rate. This results in conservative and yet potentially dangerous behavior such as avoiding imaginary obstacles. In the context of behavioral cloning, perceptual errors at training time...
-
This paper introduces two simple techniques to improve off-policy Reinforcement Learning (RL) algorithms. First, we formulate off-policy RL as a stochastic proximal point iteration. The target network plays the role of the variable of optimization and the value network computes the proximal operator. Second, we exploits the two value functions commonly employed in state-of-the-art off-policy algor...
-
Recent advancements in Bayesian likelihood-free inference enables a probabilistic treatment for the problem of estimating simulation parameters and their uncertainty given sequences of observations. Domain randomization can be performed much more effectively when a posterior distribution provides the correct uncertainty over parameters in a simulated environment. In this paper, we study the integr...
-
Recent breakthroughs both in reinforcement learning and trajectory optimization have made significant advances towards real world robotic system deployment. Reinforcement learning (RL) can be applied to many problems without needing any modeling or intuition about the system, at the cost of high sample complexity and the inability to prove any metrics about the learned policies. Trajectory optimiz...
-
Recently, there has been a lot of excitement surrounding the use of reinforcement learning for robot control and navigation. However, many of these algorithms encounter difficulty navigating long or complex trajectories. This paper presents a new mobile robot control system called Stochastic Neural Control (SNC), that uses a stochastic policy gradient algorithm for local control and a modified pro...
-
Model-free reinforcement learning (RL) is a powerful approach for learning control policies directly from high-dimensional state and observation. However, it tends to be data-inefficient, which is especially costly in robotic learning tasks. On the other hand, optimal control does not require data if the system model is known, but cannot scale to models with high-dimensional states and observation...
-
Learning control policies in robotic tasks requires a large number of interactions due to small learning rates, bounds on the updates or unknown constraints. In contrast humans can infer protective and safe solutions after a single failure or unexpected observation. In order to reach similar performance, we developed a hierarchical Bayesian optimization algorithm that replicates the cognitive infe...
-
One problem in real-world applications of reinforcement learning is the high dimensionality of the action search spaces, which comes from the combination of actions over time. To reduce the dimensionality of action sequence search spaces, macro actions have been studied, which are sequences of primitive actions to solve tasks. However, previous studies relied on humans to define macro actions or a...
-
Deep learning in combination with improved training techniques and high computational power has led to recent advances in the field of reinforcement learning (RL) and to successful robotic RL applications such as in-hand manipulation. However, most robotic RL relies on a well known initial state distribution. In real-world tasks, this information is however often not available. For example, when d...
-
In the next few years, the amount and variety of context-aware robotic manipulator applications is expected to increase significantly, especially in household environments. In such spaces, thanks to programming by demonstration, non-expert people will be able to teach robots how to perform specific tasks, for which the adaptation to the environment is imperative, for the sake of effectiveness and ...
-
Traffic signal controllers play an essential role in today's traffic system. However, the majority of them currently is not sufficiently flexible or adaptive to generate optimal traffic schedules. In this paper we present an approach to learn policies for signal controllers using deep reinforcement learning aiming for optimized traffic flow. Our method uses a novel formulation of the reward functi...
-
In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task.Our method is trained and evaluated on an in-house collected data-set of 34 vol...
-
Coverage path planning is a well-studied problem in robotics in which a robot must plan a path that passes through every point in a given area repeatedly, usually with a uniform frequency. To address the scenario in which some points need to be visited more frequently than others, this problem has been extended to non-uniform coverage planning. This paper considers the variant of non-uniform cover...
-
Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be c...
-
We propose a model-free algorithm for learning efficient policies capable of returning table tennis balls by controlling robot joints at a rate of 100Hz. We demonstrate that evolutionary search (ES) methods acting on CNN-based policy architectures for non-visual inputs and convolving across time learn compact controllers leading to smooth motions. Furthermore, we show that with appropriately tuned...
-
Continuum robots have long held a great potential for applications in inspection of remote, hard-to-reach environments. In future environments such as the Deep Space Gateway, remote deployment of robotic solutions will require a high level of autonomy due to communication delays and unavailability of human crews. In this work, we explore the application of policy optimization methods through Actor...
-
Deep reinforcement learning (DRL) is capable of learning high-performing policies on a variety of complex high-dimensional tasks, ranging from video games to robotic manipulation. However, standard DRL methods often suffer from poor sample efficiency, partially because they aim to be entirely problem-agnostic. In this work, we introduce a novel approach to exploration and hierarchical skill learni...
-
We explore the interpretation of sound for robot decision making, inspired by human speech comprehension. While previous methods separate sound processing unit and robot controller, we propose an end-to-end deep neural network which directly interprets sound commands for visual-based decision making. The network is trained using reinforcement learning with auxiliary losses on the sight and sound n...
-
We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions. The input LTL formulas have never been seen before, yet the network performs zero-shot generalization to satisfy them. This is a novel form of multi-task learning for RL agents where agents learn from one diverse set of tasks and ge...
-
In the present paper, we propose an extension of the Deep Planning Network (PlaNet), also referred to as PlaNet of the Bayesians (PlaNet-Bayes). There has been a growing demand in model predictive control (MPC) in partially observable environments in which complete information is unavailable because of, for example, lack of expensive sensors. PlaNet is a promising solution to realize such latent M...
-
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects. Planning is performed in a low-dimensional latent state space that embeds images. We define and implement a Latent Space Roadmap (LSR) which is a graph-based structure that globally captures the latent system dynamics. Our framework consists...
-
Utilization of latent space to capture a lower-dimensional representation of a complex dynamics model is explored in this work. The targeted application is of a robotic manipulator executing a complex environment interaction task, in particular, cutting a wooden object. We train two flavours of Variational Autoencoders-standard and Vector-Quantised-to learn the latent space which is then used to i...
-
Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of an held object through tactile exploration. ...
-
For robots acting in human-centered environments, the ability to improve based on experience is essential for reliable and adaptive operation; however, particularly in the context of robot failure analysis, experience-based improvement is practically useful only if robots are also able to reason about and explain the decisions they make during execution. In this paper, we describe and analyse a re...
-
Robots perceive their environment using various sensor modalities, e.g., vision, depth, sound or touch. Each modality provides complementary information for perception. However, while it can be assumed that all modalities are available for training, when deploying the robot in real-world scenarios the sensor setup often varies. In order to gain flexibility with respect to the deployed sensor setup...
-
Precomputed roadmaps can enable effective multi-query motion planning: a roadmap can be built for a robot as if no obstacles were present, and then after edges invalidated by obstacles observed at query time are deleted, path search through the remaining roadmap returns a collision-free plan. However, large roadmaps are memory intensive to store, and can be too slow for practical use. We present a...
-
Mobile robots operating in public environments require the ability to navigate among humans and other obstacles in a socially compliant and safe manner. This work presents a combined imitation learning and deep reinforcement learning approach for motion planning in such crowded and cluttered environments. By separately processing information related to static and dynamic objects, we enable our net...
-
Motion planning for robots of high degrees-of-freedom (DOFs) is an important problem in robotics with sampling-based methods in configuration space $\mathcal{C}$ as one popular solution. Recently, machine learning methods have been introduced into sampling-based motion planning methods, which train a classifier to distinguish collision free subspace from in-collision subspace in $\mathcal{C}$. In ...
-
UAVs or Unmanned Aerial Vehicles are an upcoming technology which has eased human lifestyles in many ways. Due to this trend future skies have a risk of getting congested. In such a situation time optimal collision avoidance would be extremely vital to travel in a shortest possible time by avoiding collisions. The paper proposes a novel method for time optimal collision avoidance for UAVs. The pro...
-
For accomplishing a variety of missions in challenging environments, the capability of navigating with full autonomy while avoiding unexpected obstacles is the most crucial requirement for UAVs in real applications. In this paper, we proposed such a computationally efficient obstacle avoidance trajectory planner that can be used in unknown cluttered environments. Because of the narrow view field o...
-
Robots have been successfully used in well-structured and deterministic environments, but they are still unable to function in unstructured environments mainly because of missing reliable real-time systems that integrate perception and control. In this paper, we close the loop between perception and control for real-time obstacle avoidance by introducing a new robust perception algorithm and a new...
-
Ensuring a safe online motion planning despite a large number of moving agents is the problem addressed in this paper. Collision avoidance is achieved without communication between the agents and without global localization system. The proposed solution is a modification of the Hybrid Reciprocal Velocity Obstacles (HRVO) combined with a tracking error estimation, in order to adapt the Velocity Obs...
-
Picking an item in the presence of other objects can be challenging as it involves occlusions and partial views. Given object models, one approach is to perform object pose estimation and use the most likely candidate pose per object to pick the target without collisions. This approach, however, ignores the uncertainty of the perception process both regarding the target's and the surrounding objec...
-
In this paper, we propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties for safer navigation through cluttered environments. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates through each step of our MPC's finite time ho...
-
For safe and efficient human-robot interaction, a robot needs to predict and understand the intentions of humans who share the same space. Mobile robots are traditionally built to be reactive, moving in unnatural ways without following social protocol, hence forcing people to behave very differently from human-human interaction rules, which can be overcome if robots instead were proactive. In this...
-
This paper proposes a dynamic attention-based visual odometry framework (DAVO), a learning-based VO method, for estimating the ego-motion of a monocular camera. DAVO dynamically adjusts the attention weights on different semantic categories for different motion scenarios based on optical flow maps. These weighted semantic categories can then be used to generate attention maps that highlight the re...
-
Recent CNN-based optical flow approaches have a separated structure of feature extraction and flow estimation. The core task of optical flow is finding the corresponding points while rich representation is just the key part of such matching problems. However, the prior work usually pays more attention to the design of flow decoder than the feature extraction. In this paper, we present a novel opti...
-
In this paper, a global descriptor for a LiDAR point cloud, called LiDAR Iris, is proposed for fast and accurate loop-closure detection. A binary signature image can be obtained for each point cloud after several LoG-Gabor filtering and thresholding operations on the LiDAR-Iris image representation. Given two point clouds, their similarities can be calculated as the Hamming distance of two corresp...
-
Accurate and reliable 3D object detection is vital to safe autonomous driving. Despite recent developments, the performance gap between stereo-based methods and LiDAR-based methods is still considerable. Accurate depth estimation is crucial to the performance of stereo-based 3D object detection methods, particularly for those pixels associated with objects in the foreground. Moreover, stereo-based...
-
In this paper, we tackle the problem of detecting objects in 3D and forecasting their future motion in the context of self-driving. Towards this goal, we design a novel approach that explicitly takes into account the interactions between actors. To capture their spatial-temporal dependencies, we propose a recurrent neural network with a novel Transformer [1] architecture, which we call the Interac...
-
The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different ob...
-
We consider the task of underwater robot navigation for the purpose of collecting scientifically relevant video data for environmental monitoring. The majority of field robots that currently perform monitoring tasks in unstructured natural environments navigate via path-tracking a pre-specified sequence of waypoints. Although this navigation method is often necessary, it is limiting because the ro...
-
In this work, we present a perception-aware path-planning pipeline for Unmanned Aerial Vehicles (UAVs) for navigation in challenging environments. The objective is to reach a given destination safely and accurately by relying on monocular camera-based state estimators, such as Keyframe-based Visual-Inertial Odometry (VIO) systems. Motivated by the recent advances in semantic segmentation using dee...
-
We present a robot navigation system that uses an imitation learning framework to successfully navigate in complex environments. Our framework takes a pre-built 3D scan of a real environment and trains an agent from pre-generated expert trajectories to navigate to any position given a panoramic view of the goal and the current visual input without relying on map, compass, odometry, or relative pos...
-
Autonomous navigation in complex environments is a crucial task in time-sensitive scenarios such as disaster response or search and rescue. However, complex environments pose significant challenges for autonomous platforms to navigate due to their challenging properties: constrained narrow passages, unstable pathway with debris and obstacles, or irregular geological structures and poor lighting co...
-
We present an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor (DVS) as input. The backbone of our pipeline is a bioinspired encoder-decoder neural network architecture - ECN. To train the pipeline, we introduce a covariance normalization technique which resembles the...
-
As one of the most promising areas, mobile robots draw much attention these years. Current work in this field is often evaluated in a few manually designed scenarios, due to the lack of a common experimental platform. Meanwhile, with the recent development of deep learning techniques, some researchers attempt to apply learning-based methods to mobile robot tasks, which requires a substantial amoun...
-
Vision and voice are two vital keys for agents’ interaction and learning. In this paper, we present a novel indoor navigation model called Memory Vision-Voice Indoor Navigation (MVV-IN), which receives voice commands and analyzes multimodal information of visual observation in order to enhance robots’ environment understanding. We make use of single RGB images taken by a rst-view monocular camera....
-
Visual motion estimation is an integral and well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation, which is especially challenging in highly dynamic environments. Such environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion.Previous work in object tracking focuses on maintaining the integrity of...
-
In this paper, we introduce IDOL, an optimization-based framework for IMU-DVS Odometry using Lines. Event cameras, also called Dynamic Vision Sensors (DVSs), generate highly asynchronous streams of events triggered upon illumination changes for each individual pixel. This novel paradigm presents advantages in low illumination conditions and high-speed motions. Nonetheless, this unconventional sens...
-
Reinforcement Learning (RL), among other learning-based methods, represents powerful tools to solve complex robotic tasks (e.g., actuation, manipulation, navigation, etc.), with the need for real-world data to train these systems as one of its most important limitations. The use of simulators is one way to address this issue, yet knowledge acquired in simulations does not work directly in the real...
-
In this paper, we propose an autonomous method for robot navigation based on a multi-camera setup that takes advantage of a wide field of view. A new multi-task network is designed for handling the visual information supplied by the left, central and right cameras to find the passable area, detect the intersection and infer the steering. Based on the outputs of the network, three navigation indica...
-
Robust estimation of camera motion under the presence of outlier noisevision. Despite existing efforts that focus on detecting motion and scene degeneracies, the best existing approach that builds on Random Consensus Sampling (RANSAC) still has non-negligible failure rate. Since a single failure can lead to the failure of the entire visual simultaneous localization and mapping, it is important to ...
-
Place recognition is a core component of Simultaneous Localization and Mapping (SLAM) algorithms. Particularly in visual SLAM systems, previously-visited places are recognized by measuring the appearance similarity between images representing these locations. However, such approaches are sensitive to visual appearance change and also can be computationally expensive. In this paper, we propose an a...
-
Synthetic data has been applied in many deep learning based computer vision tasks. Limited performance of algorithms trained solely on synthetic data has been approached with domain adaptation techniques such as the ones based on generative adversarial framework. We demonstrate how adversarial training alone can introduce semantic inconsistencies in translated images. To tackle this issue we propo...
-
We present an algorithm for the realtime matching of wireframe extractions in pairs of images. Here we treat extracted wireframes as graphs and propose a simplified Graduated Assignment algorithm to use with this problem. Using this algorithm we achieve a 30% accuracy improvement over the baseline method. We show that, for this problem, the simplified Graduated Assignment algorithm can achieve rea...
-
We propose an efficient edge-based stereo visual odometry (VO) using multiple quadtrees created according to image gradient orientations. To characterize edges, we classify them into eight orientation groups according to their image gradient directions. Using the edge groups, we construct eight quadtrees and set overlapping areas belonging to adjacent quadtrees for robust and efficient matching. F...
-
In this paper, we investigate the perception-aware path finding, planning and following for a class of snake robots autonomously serpentining in an unmodeled and unknown environment. In the work, the onboard LiDAR sensor mounted on the head of the snake robot is utilized to reconstruct the local environment, by which and the modified rapidly-exploring random tree method, a feasible path from the c...
-
Joint Feature Selection and Time Optimal Path Parametrization for High Speed Vision-Aided Navigation
We study a problem in vision-aided navigation in which an autonomous agent has to traverse a specified path in minimal time while ensuring extraction of a steady stream of visual percepts with low latency. Vision-aided robots extract motion estimates from the sequence of images of their on-board cameras by registering the change in bearing to landmarks in their environment. The computational burde...
-
Autonomous valet parking is a specific application for autonomous vehicles. In this task, vehicles need to navigate in narrow, crowded and GPS-denied parking lots. Accurate localization ability is of great importance. Traditional visual-based methods suffer from tracking lost due to texture-less regions, repeated structures, and appearance changes. In this paper, we exploit robust semantic feature...
-
Driver gaze mapping is crucial to estimate driver attention and determine which objects the driver is focusing on while driving. We introduce DGAZE, the first large-scale driver gaze mapping dataset. Unlike previous works, our dataset does not require expensive wearable eye-gaze trackers and instead relies on mobile phone cameras for data collection. The data was collected in a lab setting designe...
-
UAVs face many challenges in autonomous obstacle avoidance in large outdoor scenarios, specifically the long communication distance from ground stations. The computing power of onboard computers is limited, and the unknown obstacles cannot be accurately detected. In this paper, an autonomous obstacle avoidance scheme based on the fusion of millimeter wave radar and monocular camera is proposed. Th...
-
Trajectory prediction has always been a challenging problem for autonomous driving, since it needs to infer the latent intention from the behaviors and interactions from traffic participants. This problem is intrinsically hard, because each participant may behave differently under different environments and interactions. This key is to effectively model the interlaced influence from both spatial c...
-
Current unmanned aerial vehicle (UAV) visual tracking algorithms are primarily limited with respect to: (i) the kind of size variation they can deal with, (ii) the implementation speed which hardly meets the real-time requirement. In this work, a real-time UAV tracking algorithm with powerful size estimation ability is proposed. Specifically, the overall tracking task is allocated to two 2D filter...
-
This paper presents an bio-inspired event-based perception scheme for agile aerial robot maneuvering. It tries to mimic birds, which perform purposeful maneuvers by closing the separation in the retinal image (w.r.t. the goal) to follow time-to-contact trajectories. The proposed approach is based on event cameras, also called artificial retinas, which provide fast response and robustness against m...
-
Simulation data can be utilized to extend real-world driving data in order to cover edge cases, such as vehicle accidents. The importance of handling edge cases can be observed in the high societal costs in handling car accidents, as well as potential dangers to human drivers. In order to cover a wide and diverse range of all edge cases, we systemically parameterize and simulate the most common ac...
-
Safety is crucial for deploying robots in the real world. One way of reasoning about safety of robots is by building safe sets through Hamilton-Jacobi (HJ) reachability. However, safe sets are often computed offline, assuming perfect knowledge of the dynamics, due to high compute time. In the presence of uncertainty, the safe set computed offline becomes inaccurate online, potentially leading to d...
-
LiDAR odometry is a fundamental task for various areas such as robotics, autonomous driving. This problem is difficult since it requires the systems to be highly robust running in noisy real-world data. Existing methods are mostly local iterative methods. Feature-based global registration methods are not preferred since extracting accurate matching pairs in the nonuniform and sparse LiDAR data rem...
-
We propose a novel teach-and-repeat navigation system, SSM-Nav, which is based on the output of the recently introduced SSM visual place recognition methodology. During the teach phase, a teleoperated wheeled robot stores in a database features of images taken along an arbitrary route. During the repeat phase or navigation, a CNN-based comparison of each captured image is performed against the dat...
-
This study proposes a self-supervised method for detecting scene changes from an image pair. For mobile cameras such as drive recorders, to alleviate the camera viewpoints' difference, image alignment and change detection must be optimized simultaneously because they depend on each other. Moreover, lighting condition makes the scene change detection more difficult because it widely varies in image...
-
With the recent boost in autonomous driving, increased attention has been paid on radars as an input for occupancy mapping. Besides their many benefits, the inference of occupied space based on radar detections is notoriously difficult because of the data sparsity and the environment dependent noise (e.g. multipath reflections). Recently, deep learning-based inverse sensor models, from here on cal...
-
Robotic exploration requires adaptively selecting navigation goals that result in the rapid discovery and mapping of an unknown world. In many real-world environments, subtle structural cues can provide insight about the unexplored world, which may be exploited by a decision maker to improve the speed of exploration. In sparse subterranean tunnel networks, these cues come in the form of topologica...
-
Established indoor robot navigation frameworks build on the separation between global and local planners. Whereas global planners rely on traditional graph search algorithms, local planners are expected to handle driving dynamics and resolve minor conflicts. We present a system to train neural-network policies for such a local planner component, explicitly accounting for humans navigating the spac...
-
In this paper, we consider the problem of building learning agents that can efficiently learn to navigate in constrained environments. The main goal is to design agents that can efficiently learn to understand and generalize to different environments using high-dimensional inputs (a 2D map), while following feasible paths that avoid obstacles in obstacle-cluttered environment. To achieve this, we ...
-
Learning-based approaches often outperform hand-coded algorithmic solutions for many problems in robotics. However, learning long-horizon tasks on real robot hardware can be intractable, and transferring a learned policy from simulation to reality is still extremely challenging. We present a novel approach to model-free reinforcement learning that can leverage existing sub-optimal solutions as an ...
-
Path following is a challenging task for legged robots. In this paper, we present a hierarchical control architecture for path following of a quadruped salamander-like robot, in which, the tracking problem is decomposed into two sub-tasks: high-level policy learning based on the framework of reinforcement learning (RL) and low-level traditional controller design. More specifically, the high-level ...
-
Behavioral decision making is an important aspect of autonomous vehicles (AV). In this work, we propose a behavior planning structure based on hierarchical reinforcement learning (HRL) which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-go...
-
Energy-efficient mapless navigation is crucial for mobile robots as they explore unknown environments with limited on-board resources. Although the recent deep rein-forcement learning (DRL) approaches have been successfully applied to navigation, their high energy consumption limits their use in several robotic applications. Here, we propose a neuromorphic approach that combines the energy-efficie...
-
Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadr...
-
Robot control policies learned in simulation do not often transfer well to the real world. Many existing solutions to this sim-to-real problem, such as the Grounded Action Transformation (GAT) algorithm, seek to correct for- or ground-these differences by matching the simulator to the real world. However, the efficacy of these approaches is limited if they do not explicitly account for stochastici...
-
This paper considers the problem of learning behaviors in simulation without knowledge of the precise dynamical properties of the target robot platform(s). In this context, our learning goal is to mutually maximize task efficacy on each environment considered and generalization across the widest possible range of environmental conditions. The physical parameters of the simulator are modified by a ...
-
How to explore environments is one of the most critical factors for the performance of an agent in reinforcement learning. Conventional exploration strategies such as ε-greedy algorithm and Gaussian exploration noise simply depend on pure randomness. However, it is required for an agent to consider its training progress and long-term usefulness of actions to efficiently explore complex environment...
-
We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time; it must choose sensing actions that both curb localization uncertainty and achieve information gain. For this problem, belief space planning methods that forward- simulate robot sensing and estimation may of...
-
Robot navigation in human spaces today largely relies on the construction of precise geometric maps and a global motion plan. In this work, we navigate with only local sensing by using available signage - as designed for humans - in human-made environments such as airports. We propose a formalization of "signage" and define 4 levels of signage that we call complete, fully-specified, consistent and...
-
A semantic understanding of the environment is needed to enable high level autonomy in robotic systems. Recent results have demonstrated rapid progress in underlying technology areas, but few results have been reported on end-to-end systems that enable effective autonomous perception in complex environments. In this paper, we describe an approach for rapidly and autonomously mapping unknown enviro...
-
A robot understands its world through the raw information it senses from its surroundings. This raw information is not suitable as a shared representation between the robot and its user. A semantic map, containing high-level information that both the robot and user understand, is better suited to be a shared representation. We use the semantic map as the user-facing interface on our fleet of floor...
-
The ability to search for objects is a precondition for various robotic tasks. In this paper, we address the problem of finding objects in partially known indoor environments. Using the knowledge of the floor plan and the mapped objects, we consider object-object and object-room co-occurrences as prior information for identifying promising locations where an unmapped object can be present. We prop...
-
We develop an online probabilistic metric-semantic mapping approach for autonomous robots relying on streaming RGB-D observations. We cast this problem as a Bayesian inference task, requiring encoding both the geometric surfaces and semantic labels (e.g., chair, table, wall) of the unknown environment. We propose an online Gaussian Process (GP) training and inference approach, which avoids the com...
-
In order to enable a team of robots to operate successfully, retrieving accurate relative transformation between robots is the fundamental requirement. So far, most research on relative localization mainly focus on geometry features such as points, lines and planes. To address this problem, collaborative semantic map matching is proposed to perform semantic perception and relative localization. Th...
-
Finding chemical compounds in the air has applications when situations such as gas leaks, environmental emergencies and toxic chemical dispersion occur. Enabling robots to undertake this task would provide a powerful tool to prevent dangerous situations and assist humans when emergencies arise. While the dispersion of chemical compounds in the air is intrinsically a three-dimensional (3D) phenomen...
-
Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is...
-
This paper presents VCU-RVI, a new visual inertial odometry (VIO) benchmark with a set of diverse data sequences in different indoor scenarios. The benchmark was captured using an Structure Core (SC) sensor, consisting of an RGB-D camera and an IMU. It provides aligned color and depth images with 640×480 resolution at 30 Hz. The camera's data is synchronized with the IMU's data at 100 Hz. Thirty-n...
-
Human-Robot Interaction (HRI) user studies are challenging to evaluate and compare due to a lack of standardization and the infrastructure required to implement each study. The lack of experimental infrastructure also makes it difficult to systematically evaluate the impact of individual components (e.g., the quality of perception software) on overall system performance. This work proposes a frame...
-
With the recent development of autonomous vehicle technology, there have been active efforts on the deployment of this technology at different scales that include urban and highway driving. While many of the prototypes showcased have been shown to operate under specific cases, little effort has been made to better understand their shortcomings and generalizability to new areas. Distance, uptime an...
-
As robotics matures and increases in complexity, it is more necessary than ever that robot autonomy research be reproducible. Compared to other sciences, there are specific challenges to benchmarking autonomy, such as the complexity of the software stacks, the variability of the hardware and the reliance on data-driven techniques, amongst others. In this paper, we describe a new concept for reprod...
-
Real-world environments are inherently uncertain, and to operate safely in these environments robots must be able to plan around this uncertainty. In the context of motion planning, we desire systems that can maintain an acceptable level of safety as the robot moves, even when the exact locations of nearby obstacles are not known. In this paper, we solve this chance-constrained motion planning pro...
-
Recent advances in Deep Machine Learning have shown promise in solving complex perception and control loops via methods such as reinforcement and imitation learning. However, guaranteeing safety for such learned deep policies has been a challenge due to issues such as partial observability and difficulties in characterizing the behavior of the neural networks. While a lot of emphasis in safe learn...
-
Within a robot autonomy stack, the planner and controller are typically designed separately, and serve different purposes. As such, there is often a diffusion of responsibilities when it comes to ensuring safety for the robot. We propose that a planner and controller should share the same interpretation of safety but apply this knowledge in a different yet complementary way. To achieve this, we us...
-
Multi-agent safe systems have become an increasingly important area of study as we can now easily have multiple AI-powered systems operating together. In such settings, we need to ensure the safety of not only each individual agent, but also the overall system. In this paper, we introduce a novel multi-agent safe learning algorithm that enables decentralized safe navigation when there are multiple...
-
This paper investigates the safe path planning problem for an autonomous vehicle operating in unstructured, cluttered environments. While some objects may be accurately with canonical perception algorithms, other objects and clutter may be harder to track. We present an approach that combines two methods of risk assessment: for objects with reliable tracking, we use a Gaussian Process (GP) regulat...
-
This paper presents a method to validate localization safety for a preplanned trajectory in a given environment. Localization safety is defined as integrity risk and quantified as the probability of an undetected localization failure. Integrity risk differs from previously used metrics in robotics in that it accounts for unmodeled faults and evaluates safety under the worst possible combination of...
-
In this study we set out to design a computer vision-based system to assess human-robot trust in real time during close-proximity human-robot collaboration. This paper presents the setup and hardware for an augmented reality-enabled human-robot collaboration cell as well as a method of measuring operator proximity using an infrared camera. We tested this setup as a tool for assessing trust through...
-
As robots and robotic things become to have more agency, IoRT which consists of robots and robotic things can be considered as a social organization. Accordingly, social organization structure of IoRT could affect users' behavior and perception of IoRT. In this study, in order to examine the effect of social organization structure on people's acceptance of IoRT, we conducted a 2 (social organizati...
-
Common experience suggests that agents who know each other well are better able to work together. In this work, we address the problem of calibrating intention and capabilities in human-robot collaboration. In particular, we focus on scenarios where the robot is attempting to assist a human who is unable to directly communicate her intent. Moreover, both agents may have differing capabilities that...
-
As AI becomes an integral part of our lives, the development of explainable AI, embodied in the decision-making process of an AI or robotic agent, becomes imperative. For a robotic teammate, the ability to generate explanations to justify its behavior is one of the key requirements of explainable agency. Prior work on explanation generation has been focused on supporting the rationale behind the r...
-
This study proposes a technique to estimate the output state of twisted string actuators (TSAs) based on payload's acceleration measurements. We outline differential kinematics relationships of the actuator, re-formulate these into a nonlinear parameter identification problem and then apply linearization techniques to efficiently solve it as a quadratic program. Using accurate estimates of string ...
-
The simulation of static friction, and especially the effect of stiction, is cumbersome to perform in discrete-time due to its discontinuity at zero velocity and its switching behavior. However, it is essential to achieve reliable simulations of friction to develop compliant torque control algorithms, as they are much disturbed by this phenomenon. This paper takes as a base an elastoplastic model ...
-
Synthetic fibre ropes have high tensile strength, a lower friction coefficient and are more flexible than steel ropes, and are therefore increasingly used in robotics. However, their characteristics are not well studied. In particular, previous work investigated the long-term behaviour only under static loading. In this paper, we investigate the elongation behaviour of synthetic fibre ropes under ...
-
This work demonstrates a novel approach to steering a magnetic swimming robot in two dimensions with a single pair of Maxwell coils. By leveraging the curvature of the magnetic field gradient, we achieve motion along two axes. This method allows us to control medical magnetic robots using only existing MRI technology, without requiring additional hardware or posing any additional risk to the patie...
-
Research towards (compliant) actuators, especially redundant ones like the Series Parallel Elastic Actuator (SPEA), has led to the development of drive trains, which have demonstrated to increase efficiency, torque-to-mass-ratio, power-to-mass ratio, etc. In the field of robotics such drive trains can be implemented, enabling technological improvements like safe, adaptable and energy-efficient rob...
-
This paper presents a novel design concept of a miniaturized Magneto-Rheological (MR) clutch. The design uses a set of spur gears as a means to control the torque. MR clutches with various configurations such as disk-, drum-, and armature-based have in the past been reported in the literature. However, to the best of our knowledge, the design of a clutch with spur gears to use MR fluid in squeeze ...
-
Future robots are expected to share their workspace with humans. Controlling and limiting the forces that such robots exert on their environment is crucial. While force control can be achieved actively with the help of force sensing, passive mechanisms have no time delay in their response to external forces, and would therefore be preferable. Series clutch actuators can be used to achieve high lev...
-
In this work, we present and explain in detail the design of a new type of pneumatic actuator made of fire hose, the fire hose actuator (FHA), see Fig. 1. We model the force output of this type of actuator and we compare the theoretic results to the data measured on the laboratory test stand.Furthermore, we present the design of a pneumatic rotary drive that is actuated by four of the above-mentio...
-
To reduce the risk of radiation leakages similar to the incident at the Fukushima Daiichi Nuclear Power Station, robots have been employed to remove fuel debris from reactors. To perform this process safely, it is important to monitor the interior of a reactor. A camera and neutron sensors are attached to the end of a robotic arm to monitor the interior of the reactor. The basic design requirement...
-
The recent development of in-pipe robots (IPR) with locomotion and inspection functions provides a new possibility to water distribution pipe maintenance - to rehabilitate pipe defects internally. Yet only a limited number of Rehabilitation in-pipe robots (R-IPR) have been proposed. One primary concern that impedes the development of Rehabilitation in-pipe robots is the excessive amount of contami...
-
In this paper, we updated an inspection robot with passive adaptation ability, which is used to detect small size water supply pipeline. By geometric calculation and kinematic verification, static model of the robot is checked for flexible movement in the pipeline. Besides, inertial measurement unit is leveraged to simultaneously detect the attitude of robot, and different algorithm is tested to c...
-
This paper details the development, modeling and performance of AmphiSTAR, a novel high-speed amphibious robot. The palm size AmphiSTAR, which belongs to the family of STAR robots, is a "wheeled" robot fitted with propellers at its bottom that allow it to crawl on the ground and run (i.e. hover) on water at high speeds. The AmphiSTAR is inspired by two members of the animal kingdom. It possesses a...
-
This paper presents an innovative robotic mechanism for generating peristaltic motion for robotic locomotion systems. The designed underactuated peristaltic robot utilizes a minimum amount of electromechanical hardware. Such a minimal electromechanical design not only reduces the number of potential failure modes but also provides the robot design with great potential for scaling to larger and sma...
-
This paper investigates the design of a three-degree-of-freedom rotational inertia generator using the gyroscopic effect to provide ungrounded torque feedback. It uses a rotating mass in order to influence the torques needed to move the device, creating a perceived inertia. The dynamic model and the control law of the device are derived, along with those of a comparable concept using three flywhee...
-
A spherical two-degree-of- freedom wrist adapting the structure of the spatial parallelogram is proposed. A U type extended link out of three UU type limbs of the spatial parallelogram is selected as an output link. As a result, the wrist can be interpreted as being formed by combination of a U type limb and a (2-UU)+U type hybrid limb. Screw theory is employed to analyze its first-order kinematic...
-
Most existing mechanical gravity compensators have been developed for revolute joints that are found in majority of articulated robot arms. However, robots such as patient transport robots use prismatic joints, which need to handle a heavy payload. In this study, a high-capacity linear gravity compensator (LGC), which comprises pure mechanical components, such as coil springs, a rack-pinion gear, ...
-
To access rough terrain and enhance the mobility in sandy terrain, a configurable paddle-wheel robot was pro-posed. This report addresses the paddle terradynamics, and the experimental verification of the locomotion performance of the robot over dry sandy terrain. To study the interactive forces between the paddle and the media, a terradynamic model is built and verified through experiments. To ex...
-
In this paper, a new remote center of motion (RCM) mechanism is presented whose end-effector is able to move through an entire hemisphere. In general, minimally invasive surgery (MIS) applications, an elliptic cone workspace with vertex angles of 60° and 90° gives the surgeon enough freedom to operate. Therefore, the majority of the developed RCM mechanisms have such a cone as the workspace. Howev...
-
We consider the design of under-actuated articulated mechanism that are able to maintain stable static balance. Our method augments an user-provided design with counter-weights whose mass and attachment locations are automatically computed. The optimized counterweights adjust the center of gravity such that, for bounded external perturbations, the mechanism returns to its original configuration. U...
-
AIRo-5.1 an in-pipe inspection robot comprised of two passive compliant joints and a single active compliant joint that is driven by a series elastic actuator (SEA) is presented in the course of this study. As an aid in pipeline maintenance, AIRo-5.1 controls joint angles and the torque of middle joints, to enable them to adapt to bend, branch, vertical pipes, and slippery surfaces. To sense the j...
-
This study proposes a novel 7-DOF robotic manipulator called CoSMo-Arm for high torque density multi-link robotic platform based on a concentrically stacked modular actuator (CoSMoA) and an extended coaxial spherical joint module (E-CoSMo) introduced in previous researches. The CoSMoA is an actuator module designed to improve thermal characteristics by stacking the motor actuator parts to share th...
-
This paper proposes a novel modular selfreconfigurable robot (MSRR) "FreeBOT", which can be connected freely at any point on other robots. FreeBOT is mainly composed of two parts: a spherical ferromagnetic shell and an internal magnet. The connection between the modules is genderless and instant, since the internal magnet can freely attract other FreeBOT spherical ferromagnetic shells, and not nee...
-
Movement in bio-inspired robots typically relies on the use of a series of actuators and transmissions with one or more degrees of freedom (DOF), allowing asymmetrical ellipsoidal gaits for use in walking, running, swimming, and crawling. In an effort to simplify these multi-component systems, we present a novel, modular, soft, bi-stable, one DOF dome actuator platform that is capable of complex g...
-
Parallel robots with configurable platforms (PRCPs) combine the benefits of parallel robots with additional functionalities such as grasping and cutting. However, some of the theoretical tools used to study classical parallel robots do not apply to parallel robots with configurable platforms. This paper uses screw theory to study the transferable wrenches from the robot's limbs to the configurable...
-
This paper deals with continuous tension validation for Cable-Driven Parallel Robots (CDPRs). The proposed method aims at determining whether or not a quasi-static path is feasible regarding cable tension limits. The available wrench set (AWS) is the set of wrenches that can be generated with cable tensions within given minimum and maximum limits. A pose of the robot is considered valid regarding ...
-
This work studies redundant actuation for both trajectory tracking and disturbance rejection on flexible cable-driven parallel robots (CDPR). High dynamics/bandwidth unidirectional force generators, like air propellers, are used in combination with conventional but slower cable winding winches. To optimally balance the action of the two types of actuation within their saturation constraints, a mod...
-
We propose a method of subdividing robot tasks into new lower abstract tasks. The description of robot tasks in an abstract manner is effective for motion planning for complex tasks and teaching robot movements in various environments. However, a more efficient task description may be obtained by using a lower abstraction according to the work environment. We argue that a higher abstract task can ...
-
Given an articulated robot arm, we present a method to identify two regions with non-empty interiors. The first region is a subset of the configuration space where every point in the region is manipulable. The second region is a subset of the workspace where every point in the region is reachable by the end-effector. Our method expresses the kinematic state of the robot arm using the maximal coord...
-
Providing explanations of chosen robotic actions can help to increase the transparency of robotic planning and improve users' trust. Social sciences suggest that the best explanations are contrastive, explaining not just why one action is taken, but why one action is taken instead of another. We formalize the notion of contrastive explanations for robotic planning policies based on Markov decision...
-
We investigate a multi-agent planning problem, where each agent aims to achieve an individual task while avoiding collisions with others. We assume that each agent's task is expressed as a Time-Window Temporal Logic (TWTL) specification defined over a 3D environment. We propose a decentralized receding horizon algorithm for online planning of trajectories. We show that when the environment is suff...
-
In this paper, we propose a method for automatically generating object handling actions based on simple action definitions. The need to replace workers by robots is increasing, and, in fact, many research projects on robots have worked with simple motion definitions. Many applications are for mobile robots such as drones, however, and if such methods are applied directly to object handling, like a...
-
Modern robots act in dynamic and partially unknown environments where path replanning can be mandatory if changes in the environment are observed. Task-prioritized control strategies are well known and effective solutions to ensure local adaptation of robot behaviour. The highest priority in a stack of tasks is typically given to the management of correct robot operation or safe interaction with t...
-
In this paper, we consider a temporal logic planning problem in which the objective is to find an infinite trajectory that satisfies an optimal selection from a set of soft specifications expressed in linear temporal logic (LTL) while nevertheless satisfying a hard specification expressed in LTL. Our previous work considered a similar problem in which linear dynamic logic for finite traces (LDLf),...
-
Learning-enabled components have been widely deployed in autonomous systems. However, due to the weak interpretability and the prohibitively high complexity of large-scale machine learning models such as neural networks, reliability has been a crucial concern for safety-critical autonomous systems. This work proposes an online monitor called Reach-Flow for fault prevention of waypoint-following ta...
-
This paper introduces the competitive coverage problem, a new variant of the robotic coverage problem in which a robot R competes with another robot O in order to be the first to cover an area. In the variant discussed in this paper, the asymmetric competitive coverage, O is unaware of the existence of R, which attempts to take that fact into consideration in order to succeed in being the first to...
-
This paper addresses the problem of planning for visibility-based pursuit evasion, in contexts where the pursuer robot may experience some positioning errors as it moves in search of the evader. Specifically, we consider the case in which a pursuer with an omnidirectional sensor searches a known environment to locate an evader that may move arbitrarily quickly. Known algorithms for this problem ar...
-
Computation of the volume of space required for a robot to execute a sweeping motion from a start to a goal has long been identified as a critical primitive operation in both task and motion planning. However, swept volume computation is particularly challenging for multi-link robots with geometric complexity, e.g., manipulators, due to the non-linear geometry. While earlier work has shown that de...
-
Discrete graphs are commonly used to approximately represent configuration spaces used in robot motion planning. This paper explores a representation in which the costs of crossing local regions of the configuration space are represented using piecewise linear regression (PLR). We explore a few simple motion planning problems, and show that for these problems, the memory required to store the repr...
-
Asymptotically optimal sampling-based planners require an intelligent exploration strategy to accelerate convergence. After an initial solution is found, a necessary condition for improvement is to generate new samples in the so-called "Informed Set". However, Informed Sampling can be ineffective in focusing search if the chosen heuristic fails to provide a good estimate of the solution cost. This...
-
Chinese calligraphy is a unique art form with great artistic value but difficult to master. In this paper, we formulate the calligraphy writing problem as a trajectory optimization problem, and propose an improved virtual brush model for simulating the real writing process. Our approach is inspired by pseudospectral optimal control in that we parameterize the actuator trajectory for each stroke as...
-
We present a general approach for constructing proofs of motion planning infeasibility. Effective high-dimensional motion planners, such as sampling-based methods, are limited to probabilistic completeness, so when no plan exists, these planners either do not terminate or can only run until a timeout. We address this completeness challenge by augmenting a sampling-based planner with a method to cr...
-
Determining a feasible path for nonholonomic mobile manipulators operating in congested environments is challenging. Sampling-based methods, especially bi-directional tree search-based approaches, are amongst the most promising candidates for quickly finding feasible paths. However, sampling uniformly when using these methods may result in high computation time. This paper introduces two technique...
-
Mobile manipulators consist of a mobile platform equipped with one or more robot arms and are of interest for a wide array of challenging tasks because of their extended workspace and dexterity. Typically, mobile manipulators are deployed in slow-motion collaborative robot scenarios. In this paper, we consider scenarios where accurate high-speed motions are required. We introduce a framework for t...
-
Mobile robotic gas distribution mapping (GDM) is a useful tool for hazardous scene assessment where a quick and accurate representation of gas concentration levels is required throughout a staging area. However, research in robotic path planning for GDM has primarily focused on mapping in open spaces or estimating the source term in dispersion models. Whilst this may be appropriate for environment...
-
Realization of industry-scale, goal-driven, autonomous systems with AI planning technology faces several challenges: flexibly specifying planning goal states in varying situations, synthesizing plans in large state spaces, re-planning in dynamic situations, and facilitating humans to supervise, give feedback and intervene. In this paper, we present Intent-driven Strategic Tactical Planning (ISTP) ...
-
The computational difficulty of planning search paths that seek to maximize a general deterministic value function increases dramatically as desired path lengths increase. Mobile search agents with limited computational resources often utilize receding horizon methods to address the path planning problem. Unfortunately, receding horizon planners may perform poorly due to myopic planning horizons. ...
-
This paper presents a path planner, which enables a nonholonomic mobile manipulator to move its end-effector on an observed surface with a constrained orientation, given start and destination points. A partial point cloud of the environment is captured using a vision-based sensor, but no prior knowledge of the surface shape is assumed. We consider the multi-objective optimisation problem of findin...
-
Multi-goal curvature-constrained planning such as the Dubins Traveling Salesman Problem (DTSP) combines NP-hard combinatorial routing with continuous optimization to determine the optimal vehicle heading angle for each target location. The problem can be addressed as combinatorial routing using a finite set of heading samples at target locations. In such a case, optimal heading samples can be dete...
-
Autonomous navigation missions require online decision making abilities, in order to choose from a given set of candidate actions an action that will lead to the best outcome. In a partially observable setting, decision making under uncertainty, also known as belief space planning (BSP), involves reasoning about belief evolution considering realizations of future observations. Yet, when candidate ...
-
Many kinodynamic motion planners have been developed that guarantee probabilistic completeness and asymptotic optimality for systems for which steering functions are available. Recently, some planners have been developed that achieve these properties of completeness and optimality without requiring a steering function. However, these planners have not taken strong advantage of heuristic guidance t...
-
Robust and safe feedback motion planning and navigation is a critical task for autonomous mobile robotic systems considering the highly dynamic and uncertain nature scenarios of modern applications. For these reasons motion planning and navigation algorithms that have deep roots in feedback control theory has been at the center stage of this domain recently. However, the vast majority of such poli...
-
This paper studies the adaptive reliable shortest path (RSP) planning problem in a Gaussian process (GP) regulated environment. With the reasonable assumption that the travel times of the underlying transportation network follow a multi-variate Gaussian distribution, we propose two algorithms namely, Gaussian process reactive path planning (GPRPP), and Gaussian process proactive path planning (GP4...
-
This paper presents a tangle-free frontier based exploration algorithm for planar mobile robots equipped with limited length and anchored tethers. After planning a path to the closest point in the frontier between free and unknown space, the robot computes an estimate of the future length of its tether and decides, by comparing the anticipated length with the minimum possible tether length, whethe...
-
This paper develops a coverage-centric adaptive path planner to visually survey a planar environment. This is achieved by modifying an existing path planning architecture to use a novel coverage estimation approach called convolved coverage estimation (CCE). The planner maximizes the probability of terrain coverage and exploits terrain features for loop closure to keep path uncertainty in check. T...
-
Most path planning algorithms for covering a complex 3D object ignore physical limitations or constraints on a robot's motion. Adhering to such constraints for a given path can slow down the time to cover the path because the motion may need to be adjusted. This work considers a scenario in computer numerical control (CNC) milling applications, where the robot is a cutting tool that needs to cover...
-
In this paper, we propose Belief Behavior Trees (BBTs), an extension to Behavior Trees (BTs) that allows to automatically create a policy that controls a robot in partially observable environments. We extend the semantic of BTs to account for the uncertainty that affects both the conditions and action nodes of the BT. The tree gets synthesized following a planning strategy for BTs proposed recentl...
-
In order to improve the operation ability of cleaning robots, this paper proposes a decision method for cleaning robot’s operation mode. Firstly, we use the hierarchical expression ability of deep network to obtain the attributes of garbage such as state, shape, distribution, size and so on. Then the causal relationship between the attributes and the operation modes can be built by using joint lea...
-
Integrating robotic systems in architectural and construction processes is of core interest to increase the efficiency of the building industry. Automated planning for such systems enables design analysis tools and facilitates faster design iteration cycles for designers and engineers. However, generic task-and-motion planning (TAMP) for long-horizon construction processes is beyond the capabiliti...
-
As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, i...
-
Learning is usually performed by observing real robot executions. Physics-based simulators are a good alternative for providing highly valuable information while avoiding costly and potentially destructive robot executions. We present a novel approach for learning the probabilities of symbolic robot action outcomes. This is done leveraging different environments, such as physics-based simulators, ...
-
The multi-robot task allocation problem comprises task assignment, coalition formation, task scheduling, and routing. We extend the distributed constraint optimization problem (DCOP) formalism to allocate tasks to a team of robots. The tasks have time window and ordering constraints. Each robot creates a simple temporal network to maintain the tasks in its schedule. The proposed layered framework,...
-
In this work, a novel solution to the optimal motion planning problem is proposed, through a continuous, deterministic and provably correct approach, with guaranteed safety and which is based on a parametrized Artificial Potential Field (APF). In particular, Reinforcement Learning (RL) is applied to adjust appropriately the parameters of the underlying potential field towards minimizing the Hamilt...
-
Heuristic-based graph search algorithms like A* are frequently used to solve motion planning problems in many domains. For most practical applications, it is infeasible and unnecessary to pre-compute the graph representing the whole search space. Instead, these algorithms generate the graph incrementally by applying a fixed set of actions (frequently called motion primitives) to find the successor...
-
Local navigation is an essential ability of any mobile robot working in a real-world environment. One of the most commonly used methods for local navigation is the Dynamic Window Approach (DWA), which heavily depends on the settings of the parameters in its cost function. Since the optimal choice of the parameters depends on the environment that may significantly vary and change at any time, the p...
-
Reliable real-time planning for robots is essential in today's rapidly expanding automated ecosystem. In such environments, traditional methods that plan by relaxing constraints become unreliable or slow-down for kinematically constrained robots. This paper describes the algorithm Dynamic Motion Planning Networks (Dynamic MPNet), an extension to Motion Planning Networks, for non-holonomic robots t...
-
We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning (DRL). Our approach uses local and global information for each robot from motion information maps. We use a three-layer CNN that takes these maps as input to generate a suitable action to drive each robot to its goal position. Our approach is general, learns an optima...
-
Accurately tracking dynamic targets relies on robots accounting for uncertainties in their own states to share information and maintain safety. The problem becomes even more challenging when there is an unknown and time-varying number of targets in the environment. In this paper we address this problem by introducing four new distributed algorithms that allow large teams of robots to: i) run the p...
-
This work proposes a novel method of incorporating calls to a motion planner inside a potential field control policy for safe multi-robot navigation with uncertain dynamics. The proposed framework can handle more general scenes than the control policy and has low computational costs. Our work is robust to uncertain dynamics and quickly finds high-quality paths in scenarios generated from real-worl...
-
We present a Lloyd-based navigation solution for robots that are required to move in a dynamic environment, where static obstacles (e.g, furnitures, parked cars) and unpredicted moving obstacles (e.g., humans, other robots) have to be detected and avoided on the fly. The algorithm can be computed in real-time and falls in the category of the reactive methods. Moreover, we propose an extension to t...
-
We consider the problem of enhanced security of multi-robot systems to prevent cyber-attackers from taking control of one or more robots in the group. We build upon a recently proposed solution that utilizes the physical measurement capabilities of the robots to perform introspection, i.e., detect the malicious actions of compromised agents using other members of the group. In particular, the prop...
-
Modeling the stochastic behavior of interacting agents is key for safe motion planning. In this paper, we study the interaction of risk-aware agents in a game-theoretical framework. Under the entropic risk measure, we derive an iterative algorithm for approximating the intractable feedback Nash equilibria of a risk-sensitive dynamic game. We use an iteratively linearized approximation of the syste...
-
One of the key factors for extended autonomy and resilience of multi-robot systems, especially when robots operate on batteries, is their ability to maintain energy sufficiency by recharging when needed. In situations with limited access to charging facilities, robots need to be able to share and coordinate recharging activities, with guarantees that no robot will run out of energy. In this work, ...
-
Autonomous round trip missions arise as an interesting topic since ESA and NASA agreed to bring soil samples from Mars. This work proposes a new method to improve autonomous rover guidance for this kind of missions. It is focused on the use of dynamically updated cost maps that are used to plan the rover path for a round-trip. The main advantage of the proposed method is the use of gathered inform...
-
This paper presents a trajectory planning algorithm for person following that is more comprehensive than existing algorithms. This algorithm is tailored for a front-wheel-steered vehicle, is designed to follow a person while avoiding collisions with both static and moving obstacles, simultaneously optimizing speed and steering, and minimizing control effort. This algorithm uses nonlinear model pre...
-
Hybrid locomotion, which combines multiple modalities of locomotion within a single robot, enables robots to carry out complex tasks in diverse environments. This paper presents a novel method for planning multi-modal locomotion trajectories using approximate dynamic programming. We formulate this problem as a shortest-path search through a state-space graph, where the edge cost is assigned as opt...
-
Many areas of scientific interest in planetary exploration, such as lunar pits, icy-moon crevasses, and Martian craters, are inaccessible to current wheeled rovers. Rappelling rovers can safely traverse these steep surfaces, but require techniques to navigate their complex terrain. This dynamic navigation is inherently time-critical and communication constraints (e.g. delays and small communicatio...
-
The Next Best View (NBV) problem is important in the active robotic reconstruction. It enables the robot system to perform scanning actions in a reasonable view sequence, and fulfil the reconstruction task in an effective way. Previous works mainly follow the volumetric methods, which convert the point cloud information collected by sensors into a voxel representation space and evaluate candidate ...
-
The paper presents a receding horizon planning strategy for a quadrotor-type mav to navigate through an unknown cluttered environment at high speed. Utilizing a lightweight on-board short-range sensor that generates point-clouds within a narrow Field of View (FOV), the reported approach generates safe and dynamically feasible trajectories within the fov of the sensor, which the mav uses to navigat...
-
Collision avoidance in unknown obstacle-cluttered environments may not always be feasible. This paper focuses on an emerging paradigm shift in which potential collisions with the environment can be harnessed instead of being avoided altogether. To this end, we introduce a new sampling-based online planning algorithm that can explicitly handle the risk of colliding with the environment and can swit...
-
Modern localization techniques allow ground vehicle robots to determine their position with centimeter-level accuracy under nominal conditions, enabling them to utilize fixed maps to navigate their environments. However, when localization measurements become unavailable, the position accuracy will drop and uncertainty will increase. While research and development on localization estimation seeks t...
-
This paper introduces a constraint-based skill framework for programming robot applications. Existing skill frameworks allow application developers to reuse skills and compose them sequentially or in parallel. However, they typically assume that the skills are running independently and in a nominal condition. This limitation hinders their applications for more involved and realistic scenarios e.g....
-
We consider a scenario in which an autonomous vehicle equipped with a downward facing camera operates in a 3D environment and is tasked with searching for an unknown number of stationary targets on the 2D floor of the environment. The key challenge is to minimize the search time while ensuring a high detection accuracy. We model the sensing field using a multi-fidelity Gaussian process that system...
-
Dnamic System Identification approaches usually heavily rely on evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system identif...
-
The snake-like robot without wheels is a bio-inspired robot whose high degree of freedom results in a challenge in autonomous locomotion control. The use of a Spiking Neural Network (SNN) which is a biologically plausible artificial neural network can help to achieve the autonomous locomotion behavior of snake robots in an energy-efficient manner. Approaches that use an SNN without hidden layers h...
-
Learning accurate models of the physical world is required for a lot of robotic manipulation tasks. However, during manipulation, robots are expected to interact with un-known workpieces so that building predictive models which can generalize over a number of these objects is highly desirable. In this paper, we study the problem of designing deep learning agents which can generalize their models o...
-
Control barrier functions are mathematical constructs used to guarantee safety for robotic systems. When integrated as constraints in a quadratic programming optimization problem, instantaneous control synthesis with real-time performance demands can be achieved for robotics applications. Prevailing use has assumed full knowledge of the safety barrier functions, however there are cases where the s...
-
Variable impedance control is advantageous for physical human-robot interaction to improve safety, adaptability and many other aspects. This paper presents a gain-scheduled variable stiffness control approach under strict frequency-domain constraints. Firstly, to reduce conservativeness, we characterize and constrain the impedance rendering, actuator saturation, disturbance/noise rejection and pas...
-
To this day, most robots are installed behind safety fences, separated from the human. New use-case scenarios demand for collaborative robots, e.g. to assist the human with physically challenging tasks. These robots are mainly installed in work-environments with limited space, e.g. existing production lines. This brings certain challenges for the control of such robots. The presented work addresse...
-
While impedance control is one of the most commonly used strategies for robot interaction control, variable impedance control is a more recent preoccupation. If designing impedance control with varying parameters allows increasing the system flexibility and dexterity, it is still a challenging issue, as it may result in a loss of passivity of the control system. This has an important impact on the...
-
This paper presents an alternative impedance controller for a macro-mini robotic system in which the mini robot is unactuated. The approach is verified experimentally on a simple two-degree-of-freedom macro-mini robot. The dynamic analysis of the robot is first presented. Then, a standard impedance controller is derived and analysed. Such a controller is shown to be experimentally unstable when us...
-
Aerial robotic manipulation is an emergent trend that poses several challenges. To overcome some of these, the DLR cable-Suspended Aerial Manipulator (SAM) has been envisioned. SAM is composed of a fully actuated multi-rotor anchored to a main carrier through a cable and a KUKA LWR attached below the multi-rotor. This work presents a control method to allow SAM, which is a holonomically constraine...
-
Magnetically actuated soft robots have recently been identified for application in medicine, due to their potential to perform minimally invasive exploration of human cavities. Magnetic solutions permit further miniaturization when compared to other actuation techniques, without loss in functionalities. Our long-term goal is to propose a novel actuation method for magnetically actuated soft robots...
-
In this paper, we propose a formal language to specify robot skills, i.e. the elementary behaviours or functions provided by the robot platform in order to perform an autonomous mission. The advantage of the language we propose is that it integrates a wide range of elements that allows to define and provide automatic translation both to operational models, used online to control the skill executio...
-
Screw theory is a powerful mathematical tool for the kinematic analysis of mechanisms and has become a cornerstone of modern kinematics. Although screw theory has rooted itself as a core concept, there is a lack of generic software tools for visualization of the geometric pattern of the screw elements. This paper presents STORM, an educational and research oriented framework for analysis and visua...
-
In recent years the need for manipulation tasks in the industrial as well as in the service robotics domain that require compliant interaction with the environment rose. Since then, an increased number of publications use a model-driven approach to describe these tasks. High-level tasks and sequences of skills are coordinated to achieve a desired motion for e.g., screwing, polishing, or snap mount...
-
Robots are currently deployed in safety-critical domains but proper techniques to assess the functional safety of their software are yet to be adopted. This is particularly critical in ROS, where highly configurable robots are built by composing third-party modules. To promote adoption, we advocate the use of lightweight formal methods, automatic techniques with minimal user input and intuitive fe...
-
This paper describes basic implementation of an embedded controller board based on a hybrid architecture equipped with an Intel FPGA SoC and an NVIDIA GPU SoC. Embedded distributed network involving motor-drivers or other embedded boards is constructed with low-latency optical transmission link. The central controller for high-level motion planning is connected via Gigabit Ethernet. The controller...
-
In this paper an approach for the identification of the dynamic parameters, i.e. base parameters, of rigid robots is presented. By using the polynomial approximation operator, an equation is obtained for the identification of the parameters which solely depends on measurable signals and thereby contains no equation error. The resulting expressions can be evaluated online or offline by filtering th...
-
In this paper, nonlinear control techniques are exploited to balance an unmanned bicycle with enlarged stability domain. We consider two cases. For the first case when the autonomous bicycle is balanced by the flywheel, the steering angle is set to zero, and the torque of the flywheel is used as the control input. The controller is designed based on the Interconnection and Damping Assignment Passi...
-
Cable-driven joints proved to be an effective solution in a wide variety of applications ranging from medical to industrial fields where light structures, interaction with unstructured and constrained environments and precise motion are required. These requirements are achieved by moving the actuators from joints to the robot chassis. Despite these positive properties a cable-driven robotic arm re...
-
This work addresses on the following problem: given a set of unsynchronized history observations of two scenes that are correlative on their dynamic changes, the purpose is to learn a cross-scene predictor, so that with the observation of one scene, a robot can onlinely predict the dynamic state of the other. A method is proposed to solve the problem via modeling dynamic correlation using latent s...
-
We suggest a procedure for dynamic parameter estimation of serial robot manipulators. Its basic idea relies on the synthesis of an optimal manipulation trajectory, which is based on properly introduced parameter aggregates to ensure a collection of numerically well-conditioned data-sets, yielding an accurate computation of parameter estimates. The optimal trajectory itself is computed by using a m...
-
This paper presents a method for performing free-fall penetrometer tests for soft soils using an instrumented dart deployed by a quadcopter. Tests were performed with three soil types and used to examine the effect of drop height on the penetration depth and the deceleration profile. Further tests analyzed the force required to remove a dart from the soil and the effect of pulling at different spe...
-
In haptic time delayed teleoperation as the time delay from the communication channel increases, teleoperation system stability and performance degrade. To increase performance and provide better stability margins, various estimation methods and observers have been implemented in literature to more accurately capture the force exerted by the remote system. Previously, solutions focused on environm...
-
We study for the first time the verification problem on learning-enabled state estimation systems for robotics, which use Bayes filter for localisation, and use deep neural network to process sensory input into observations for the Bayes filter. Specifically, we are interested in a robustness property of the systems: given a certain ability to an adversary for it to attack the neural network witho...
-
When exploring an unknown environment, a mobile robot must decide where to observe next. It must do this whilst minimising the risk of failure, by only exploring areas that it expects to be safe. In this context, safety refers to the robot remaining in regions where critical environment features (e.g. terrain steepness, radiation levels) are within ranges the robot is able to tolerate. More specif...
-
Localizing contacts and collisions is an important aspect of failure detection and recovery for robots and can aid perception and exploration of the environment. Contrary to state-of-the-art methods that rely on forces and torques measured on the robot, this paper proposes a kinematic method for proprioceptive contact localization on compliant robots using velocity measurements. The method is vali...
-
Having the merits of chemical inertness and immunity to electromagnetic interference, light weight, small size, and softness, optical waveguides have attracted much attention in making tactile sensors recently. This paper presents a new design of waveguide using two layers of cores, one of which has an uniform width and the other has an incremental width. It is deduced and verified that the contac...
-
The ability to measure multi-axis contact forces and contact surface normals in real time is critical to allow robots to improve their dexterous manipulation and locomotion abilities. This paper presents a new fingertip sensor for 3-axis contact force and contact location detection, as well as improvements on an existing footpad sensor through use of a new artificial neural network estimator. The ...
-
The development of modern sensitive lightweight robots allows the use of robot arms in numerous new scenarios. Especially in applications where interaction between the robot and an object is desired, e.g. in assembly, conventional purely position-controlled robots fail. Former research has focused, among others, on control methods that center on robot-environment interaction. However, these method...
-
In this paper, an approach for automatic peg-in-hole assembly is proposed. The task is divided into two main steps: searching phase and inserting phase. First, a multilayer perceptron network is designed to address the hole search problem and a hybrid force position controller is introduced to ensure a safe and stable interaction with the external environment. Then, for the inserting phase, a vari...
-
This paper presents a novel control strategy for the compensation of the slippage effect during non-rigidly grasped object manipulation. A detailed dynamic model of the interconnected system composed of the robotic manipulator, the object and the internal forces and torques induced by the slippage effect is provided. Next, we design a model-based variable impedance control scheme, in order to achi...
-
A high-quality free-motion rendering is one of the most vital traits to achieve an immersive human-robot interaction. Rendering free-motion is notably challenging for rehabilitation exoskeletons due to their relatively high weight and powerful actuators required for strength training and support. In the presence of dynamic human movements, accurate feedback linearization of the robot's dynamics is...
-
The increasing interest in autonomous robots with a high number of degrees of freedom for industrial applications and service robotics have also increased the demand for efficient control algorithms. The unstructured environment these robots operate in often impose constraints on the joint motion, an important type being the joint limits of the robot itself. These circumstances demand control algo...
-
We present a hierarchical framework for trajectory optimization and optimal feedback whole-body control of wheeled inverted pendulum (WIP) humanoid robot. The framework extends rapidly exponentially stabilizing control Lyapunov functions (RES-CLF) to operational space for controlling WIP humanoid robots while utilizing a hierarchical framework to compute an optimal policy. The upper level of the h...
-
This paper focuses on robustness to disturbance forces and uncertain payloads. We present a novel formulation to optimize the robustness of dynamic trajectories. A straightforward transcription of this formulation into a nonlinear programming problem is not tractable for state-of-the-art solvers, but it is possible to overcome this complication by exploiting the structure induced by the kinematics...
-
Sampling-based motion planners (SBMP) are commonly used to generate motion plans by incrementally constructing a search tree through a robot's configuration space. For high degree-of-freedom systems, sampling is often done in a lower-dimensional space, with a steering function responsible for local planning in the higher-dimensional configuration space. However, for highly-redundant systems with c...
-
We present a path planning strategy for a magnetic millirobot where the nonlinearities in the external magnetic force field (MFF) are encoded in the graph used for planning. The strategy creates a library of candidate MFFs and characterizes their topologies by identifying the unstable manifolds in the workspace. The path planning problem is then posed as a graph search problem where the computed p...
-
In a geometric mechanics framework, the configuration space is decomposed into a shape space and a position space. The internal motion of the system is prescribed by a closed loop in the shape space, which causes net motion in the position space. If the shape space is a simply connected domain in an Euclidean space, then with an optimal choice of the body frame, the displacement in the position sp...
-
Autonomous Navigation and Obstacle Avoidance of a Snake Robot with Combined Velocity-Heading Control
This paper presents combined velocity-heading control of a planar snake robot for the autonomous navigation and obstacle avoidance in a simulation environment. The kinematics and dynamics of the snake robot were derived using the articulated-body algorithm without considering the non-holonomic constraints. A double-layer controller was designed to control both heading direction and average velocit...
-
The feet of robots are typically used to design locomotion strategies, such as balancing, walking, and running. However, they also have great potential to perform manipulation tasks. In this paper, we propose a model predictive control (MPC) framework for a quadrupedal robot to dynamically balance on a ball and simultaneously manipulate it to follow various trajectories such as straight lines, sin...
-
This paper addressed a challenging problem of wheeled-legged robots with high degrees of freedom exploring in unknown rough environments. The proposed method works as a pipeline to achieve prioritized exploration comprising three primary modules: traversability analysis, frontier-based exploration and hybrid locomotion planning. Traversability analysis provides robots an evaluation about surroundi...
-
We present KOVIS, a novel learning-based, calibration-free visual servoing method for fine robotic manipulation tasks with eye-in-hand stereo camera system. We train the deep neural network only in the simulated environment; and the trained model could be directly used for real-world visual servoing tasks. KOVIS consists of two networks. The first keypoint network learns the keypoint representatio...
-
One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a ...
-
We address the monocular visual shape servoing problem. This pushes the challenging visual servoing problem one step further from rigid object manipulation towards deformable object manipulation. Explicitly, it implies deforming the object towards a desired shape in 3D space by robots using monocular 2D vision. We specifically concentrate on a scheme capable of controlling large isometric deformat...
-
This paper studies the problem of Image-Based Visual Servo Control (IBVS) for quadrotors. Although the control of quadrotors has been extensively studied in the last decades, combining the IBVS module with the quadrotor's dynamics is still hard, mainly due to the under-actuation issues related to the quadrotor control as opposed to the 6 DoF control outputs generated by the IBVS modules. We propos...
-
Electric motors have been widely used as the actuators of robot and automation systems. This paper aims at achieving the high-precision position control of motor drive systems. For this purpose, a robust control scheme is presented by combining the internal model principle, the sliding mode technique and the extended state observer (ESO). The PID-type controller is firstly designed by using the in...
-
Trajectory tracking of flexible link robots is a classical control problem. Historically, the link elasticity was considered as something to be removed. Hence, the control performance was guaranteed by adopting high-gain feedback loops and, possibly, a dynamic compensation with the result to stiffen up the dynamic behavior of the robot. Nowadays, robots are pushed more and more towards a safe phys...
-
In this paper, we present an optimal control approach using Linear Matrix Inequalities (LMIs) for trajectory tracking control of a three-wheeled omnidirectional mobile robot in the presence of external disturbances on the robot's actuators and noise in the robot's sensor measurements. First, a state-space representation of the omnidirectional robot dynamics is derived using a point-mass dynamic mo...
-
In this paper, the gain scheduling technique is applied to design a balance controller for an autonomous bicycle with an inertia wheel. Previously, two different balance controllers are needed depending on whether the bicycle is stationary or dynamic. The switch between the two different controllers may cause the instability of the autonomous bicycle. Our proposed gain scheduled controller can bal...
-
Humanoid robots are expected to play a big role at distress sites and disaster sites. There is a variety of multi-contact locomotion forms other than bipedal walking such as crawling through tightly, getting on the rubble by using its knees and elbows, or jumping in and rolling over the obstacles. If such multi-contact locomotion forms can be achieved, robots can reach environments that are curren...
-
Dynamic control for robotic automation tasks is traditionally designed and optimized with a model-based approach, and the performance relies heavily upon accurate system modeling. However, modeling the true dynamics of increasingly complex robotic systems is an extremely challenging task and it often renders the automation system to operate in a non-optimal condition. Notably, many industrial robo...
-
This work uses quadratic programming to perform torque control on an industrial collaborative robot, while keeping defined constraints. Limits for rotational and translational coordinates are considered at position, velocity and acceleration level. Although the problem of having hardware and safe limitations has been considered before. Solutions usually rely on functions that need a proper tuning....
-
The combination of policy search and deep neural networks holds the promise of automating a variety of decision-making tasks. Model Predictive Control (MPC) provides robust solutions to robot control tasks by making use of a dynamical model of the system and solving an optimization problem online over a short planning horizon. In this work, we leverage probabilistic decision-making approaches and ...
-
Recently, Differential Dynamic Programming (DDP) and other similar algorithms have become the solvers of choice when performing non-linear Model Predictive Control (nMPC) with modern robotic devices. The reason is that they have a lower computational cost per iteration when compared with off-the-shelf Non-Linear Programming (NLP) solvers, which enables its online operation. However, they cannot ha...
-
In fields such as minimally invasive surgery, effective control strategies are needed to guarantee safety and accuracy of the surgical task. Mechanical designs and actuation schemes have inevitable limitations such as backlash and joint limits. Moreover, surgical robots need to operate in narrow pathways, which may give rise to additional environmental constraints. Therefore, the control strategie...
-
This paper presents a multirotor control architecture, where Model Predictive Path Integral Control (MPPI) and ℒ1 adaptive control are combined to achieve both fast model predictive trajectory planning and robust trajectory tracking. MPPI provides a framework to solve nonlinear MPC with complex cost functions in real-time. However, it often lacks robustness, especially when the simulated dynamics ...
-
Safety is a critical issue in learning-based robotic and autonomous systems as learned information about their environments is often unreliable and inaccurate. In this paper, we propose a risk-aware motion control tool that is robust against errors in learned distributional information about obstacles moving with unknown dynamics. The salient feature of our model predictive control (MPC) method is...
-
A hierarchical algorithm involving two-layer optimization-based control policies with varying degrees of abstraction is proposed, including upper layer task scheduling and lower layer local path planning. A scenario with two robot arms performing cooperative pick-and-place tasks for moving objects is specifically addressed. The main focus of the paper lies on the bottom layer of the hierarchical c...
-
This paper proposes a new approach for online control law gains adaptation, through the use of neural networks and the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm, in order to optimize the behavior of the robot with respect to an objective function. The neural network considered takes as input the current observed state as well as its uncertainty, and provides as output the ...
-
This paper presents a neural-network based adaptive feedback control structure to regulate the velocity of 3D bipedal robots under dynamics uncertainties. Existing Hybrid Zero Dynamics (HZD)-based controllers regulate velocity through the implementation of heuristic regulators that do not consider model and environmental uncertainties, which may significantly affect the tracking performance of the...
-
With the advent of intelligent transport, quadrotors are becoming an attractive aerial transport solution during emergency evacuations, construction works etc. During such operations, dynamic variations in (possibly unknown) payload and unknown external disturbances cause considerable control challenges for path tracking algorithms. In fact, the state-dependent nature of the resulting uncertaintie...
-
Robotic systems are increasingly required not only to generate precise motions to complete their tasks but also to handle the interactions with the environment or human. Significantly, soft interaction brings great challenges on the force control due to the nonlinear, viscoelastic and inhomogeneous properties of the soft environment. In this paper, a robust impedance control scheme utilizing integ...
-
Rigid-terrain unmanned ground vehicles(UGV) can run under the field environment by the advanced adaptive ability. This paper presents a novel horse inspired rigid-terrain eight-wheel vehicle with four-swing arms. This unmanned ground vehicle is drived by distributed hydraulic motors. By cooperating with four-swing arms and eight wheels, the vehicle has the ability to work like a horse climbs an ob...
-
In this paper, we propose a set-inversion approach to validate the controller of a nonlinear system that should satisfy some state constraints. We introduce the notion of follow set which corresponds to the set of all output vectors such that the desired dynamics can be followed without violating the state-constraints. This follow set can then be used to choose feasible trajectories that a mobile ...
-
Traditionally, controllers and state estimators in robotic systems are designed independently. Controllers are often designed assuming perfect state estimation. However, state estimation methods such as Visual Inertial Odometry (VIO) drift over time and can cause the system to misbehave. While state estimation error can be corrected with the aid of GPS or motion capture, these complementary sensor...
-
Control of wire-borne underactuated brachiating robots requires a robust feedback control design that can deal with dynamic uncertainties, actuator constraints and unmeasurable states. In this paper, we develop a robust feedback control for brachiating on flexible cables, building on previous work on optimal trajectory generation and time-varying LQR controller design. We propose a novel simplifie...
-
Measuring the forces that humans exert with their fingers could have many potential applications, such as skill transfer from human experts to robots or monitoring humans. In this paper we introduce the "Exo-Glove" system, which can measure the joint angles and forces acting on the human finger without covering the skin that is in contact with the manipulated object. In particular, 3-axis sensors ...
-
The wheelchair is the major means of transport for elderly and physically disabled people in their daily lives. However it cannot overcome architectural barriers such as curbs and stairs. In this study, we developed an inverted-pendulum-type robotic wheelchair for climbing stairs. The wheelchair has a seat slider and two rotary links between the front and rear wheels on each side. When climbing up...
-
One of the main challenges of the active assistive devices is how to estimate the motion of the missing/impaired limbs and joints in line with the remaining limbs. To do so, a motion planner is required. This study proposes a motion planner that can be used for active prosthetic/orthotic knees. The aim is to continuously estimate the knee joint positions based on the thigh motion, using as few inp...
-
This paper proposes a data-driven method for powered prosthesis control that achieves stable walking without the need for additional sensors on the human. The key idea is to extract the nominal gait and the human interaction information from motion capture data, and reconstruct the walking behavior with a dynamic model of the human-prosthesis system. The walking behavior of a human wearing a power...
-
This paper focuses on the design and comparison of different deep neural networks for the real-time prediction of locomotor intentions by using data from inertial measurement units. The deep neural network architectures are convolutional neural networks, recurrent neural networks, and convolutional recurrent neural networks. The input to the architectures are features in the time domain, which hav...
-
This study develops a training system for a multimodal comprehensive care methodology for dementia patients called Humanitude. Humanitude has attracted much attention as a gentle and effective care technique. It consists of four main techniques, namely, eye contact, verbal communication, touch, and standing up, and more than 150 care elements. Learning Humanitude thus requires much time. To provid...
-
In gait training for walking rehabilitation of patients with stroke hemiplegia or bone joint conditions such as fractures, it is important to recognize the load of the affected lower limbs for improving gait ability and avoiding risks such as re-fractures. A weight scale is used at the actual rehabilitation site to recognize the load. However, in this situation, the trainee must look down to verif...
-
A Mixed-Integer Model Predictive Control Approach to Motion Cueing in Immersive Wheelchair Simulator
To allow wheelchair (electronic or manual) users to practice driving in different safe, repeatable and controlled scenarios, the use of simulator as a training tool is considered here. In this context, the capabilities of providing high fidelity motions for users of the simulator is highlighted as one of the most important aspects for the effectiveness of the tool. For this purpose, the motion cue...
-
Injuries, accidents, strokes, and other diseases can significantly degrade the capabilities to perform even the most simple activities in daily life. A large share of these cases involves neuromuscular diseases, which lead to severely reduced muscle function. However, even though affected people are no longer able to move their limbs, residual muscle function can still be existent. Previous work h...
-
With the development of robotic technology, the demand for state-of-the-art technology in the field of rehabilitation is rapidly increasing for the elderly and people with disabilities. In this paper, we propose a real-time virtual coach to assist physical therapists with the end-effector-based robot-assisted gait training for stroke survivors using Long Short-Term Memory (LSTM) networks. Our prop...
-
Force perturbation is used in this paper to study cervical neuromuscular responses which can be used in the future to assess impairments in patients with neurological diseases. Current literature on this topic is limited to applying forces on the head in the anterior-posterior direction, perhaps due to technological limitations. In this paper, we propose to use a robotic neck brace to address thes...
-
The process of planning views to observe a scene is known as the Next Best View (NBV) problem. Approaches often aim to obtain high-quality scene observations while reducing the number of views, travel distance and computational cost. Considering occlusions and scene coverage can significantly reduce the number of views and travel distance required to obtain an observation. Structured representatio...
-
We present a robotic grasping system that uses a single external monocular RGB camera as input. The object-to-robot pose is computed indirectly by combining the output of two neural networks: one that estimates the object-to-camera pose, and another that estimates the robot-to-camera pose. Both networks are trained entirely on synthetic data, relying on domain randomization to bridge the sim-to-re...
-
This paper presents a method to cope with autonomous assembly tasks in the presence of uncertainties. To this aim, a Peg-in-Hole operation is considered, where the target workpiece position is unknown and the peg-hole clearance is small. Deep learning based hole detection and 3D surface reconstruction techniques are combined for accurate workpiece localization. In detail, the hole is detected by u...
-
This paper presents a model-free approach to automate folding of a deformable object with robot manipulators, where its surface was labelled with markers to facilitate vision-based control and alignment. While performing the task involves solving nonconvex or nonlinear terms, in this paper, linearization was first performed to approximate the problem. By using the Levenberg-Marquardt algorithm, th...
-
Spatial mapping of surface roughness is a critical enabling technology for automating adaptive sanding operations. We leverage GelSight sensors to convert the problem of surface roughness measurement into a vision classification problem. By combining GelSight sensors with Optitrack positioning systems we attempt to develop an accurate spatial mapping of surface roughness that can compare to human ...
-
A motion taxonomy can encode manipulations as a binary-encoded representation, which we refer to as motion codes. These motion codes innately represent a manipulation action in an embedded space that describes the motion’s mechanical features, including contact and trajectory type. The key advantage of using motion codes for embedding is that motions can be more appropriately defined with robotic-...
-
Industrial robots play potential and important roles on labor-intensive and high-risk jobs. For example, typical industrial robots have been used in grinding process. However, the automatic grinding process by robots is a complex process because it still relies on skillful engineers to adaptively adjust several key parameters. Moreover, it might take a lot of time and effort to yield better grindi...
-
Because of their large workspace, robot manipulators have the potential to be used for high precision non-contact manufacturing processes, such as laser cutting or welding, on large complex work pieces. However, most industrial manipulators are not able to provide the necessary accuracy requirements. Mainly because of their flexible structures, they are subject to point to point positioning errors...
-
This paper presents a novel dataset targeting three types of challenging environments for autonomous driving, i.e., the industrial logistics environment, the undulating hill environment and the mixed complex urban environment. To the best of the author’s knowledge, similar dataset has not been published in the existing public datasets, especially for the logistics environment collected in the func...
-
This paper explores the heterogeneous task allocation problem in Multi-robot systems. A game-theoretic formulation of the problem is proposed to align the goal of individual robots with the system objective. The concept of Nash equilibrium is applied to define a desired solution for the task allocation problem in which each robot can allocate itself to an appropriate task group. We also introduce ...
-
For robot swarms operating on complex missions in an uncertain environment, it is important that the decision-making algorithm considers both heterogeneity and uncertainty. This paper presents a stochastic programming framework for the vehicle routing problem with stochastic travel energy costs and heterogeneous vehicles and tasks. We represent the heterogeneity as linear constraints, estimate the...
-
This paper presents an approach for multi-robot long-term planning under uncertainty over the duration of actions. The proposed methodology takes advantage of generalized stochastic Petri nets with rewards (GSPNR) to model multi-robot problems. A GSPNR allows for unified modeling of action selection, uncertainty on the duration of action execution, and for goal specification through the use of tra...
-
We present a novel algorithm for the multi-robot generalized assignment problem (GAP) with stochastic resource consumption. In this problem, each robot has a resource (e.g., battery life) constraint and it consumes a certain amount of resource to perform a task. In practice, the resource consumed for performing a task can be uncertain. Therefore, we assume that the resource consumption is a random...
-
We present the Pluggable Distributed Resource Allocator (PDRA), a middleware for distributed computing in heterogeneous mobile robotic networks. PDRA enables autonomous robotic agents to share computational resources for computationally expensive tasks such as localization and path planning. It sits between an existing single-agent planner/executor and existing computational resources (e.g. ROS pa...
-
In this paper, we present a large dataset with a variety of mobile mapping sensors collected using a handheld device carried at typical walking speeds for nearly 2.2 km around New College, Oxford as well as a series of supplementary datasets with much more aggressive motion and lighting contrast. The datasets include data from two commercially available devices - a stereoscopic-inertial camera and...
-
The recent introduction of powerful embedded graphics processing units (GPUs) has allowed for unforeseen improvements in real-time computer vision applications. It has enabled algorithms to run onboard, well above the standard video rates, yielding not only higher information processing capability, but also reduced latency. This work focuses on the applicability of efficient low-level, GPU hardwar...
-
Sampling-based planning has become a de facto standard for complex robots given its superior ability to rapidly explore high-dimensional configuration spaces. Most existing optimal sampling-based planning algorithms are sequential in nature and cannot take advantage of wide parallelism available on modern computer hardware. Further, tight synchronization of exploration and exploitation phases in t...
-
This paper proposes ROS-lite, a robot operating system (ROS) development framework for embedded many- core platforms based on network-on-chip (NoC) technology. Many-core platforms support the high processing capacity and low power consumption requirement of embedded systems. In this study, a self-driving software platform module is parallelized to run on many-core processors to demonstrate the pra...
-
We show how to use the Operational Space Control framework (OSC) under joint and Cartesian constraints for reinforcement learning in Cartesian space. Our method is able to learn fast and with adjustable degrees of freedom, while we are able to transfer policies without additional dynamics randomizations on a KUKA LBR iiwa peg-in-hole task. Before learning in simulation starts, we perform a system ...
-
Data-driven approaches to tactile sensing aim to overcome the complexity of accurately modeling contact with soft materials. However, their widespread adoption is impaired by concerns about data efficiency and the capability to generalize when applied to various tasks. This paper focuses on both these aspects with regard to a vision-based tactile sensor, which aims to reconstruct the distribution ...
-
Robots can learn to do complex tasks in simulation, but often, learned behaviors fail to transfer well to the real world due to simulator imperfections (the "reality gap"). Some existing solutions to this sim-to-real problem, such as Grounded Action Transformation (gat), use a small amount of real-world experience to minimize the reality gap by "grounding" the simulator. While very effective in ce...
-
In sim-to-real transfer of Reinforcement Learning (RL) policies for robot tasks, Domain Randomization (DR) is a widely used technique for improving adaptability. However, in DR there is a conflict between adaptability and training stability, and heavy DR tends to result in instability or even failure in training. To relieve this conflict, we propose a new algorithm named Domain Decomposition (DD) ...
-
Tumbling locomotion allows for small robots to traverse comparatively rough terrain, however, their motion is complex and difficult to control. Existing tumbling robot control methods involve manual control or the assumption of at terrain. Reinforcement learning allows for the exploration and exploitation of diverse environments. By utilizing reinforcement learning with domain randomization, a rob...
-
Inertial Measurement Unit (IMU) has long been a dream for stable and reliable motion estimation, especially in indoor environments where GPS strength limits. In this paper, we propose a novel method for position and orientation estimation of a moving object only from a sequence of IMU signals collected from the phone. Our main observation is that human motion is monotonous and periodic. We adopt t...
-
The main contribution of this paper is a probabilistic estimator that assists a mobile robot to locate a gas source in an indoor environment. The scenario is that a robot equipped with a gas sensor enters a building after the gas is released due to a leak or explosion. The problem is discretized by dividing the environment into a set of regions and time into a set of time intervals. Likelihood fun...
-
This paper presents a method for processing sparse, non-Gaussian multimodal data in a simultaneous localization and mapping (SLAM) framework using factor graphs. Our approach demonstrates the feasibility of using a sum-product inference strategy to recover functional belief marginals from highly non-Gaussian situations, relaxing the prolific unimodal Gaussian assumption. The method is more focused...
-
The deployment of a mobile service robot in domestic settings is a challenging task due to the dynamic and unstructured nature of such environments. Successful operation of the robot requires continuous human supervision to update its spatial knowledge about the dynamic environment. Thus, it is essential to develop a human-robot interaction (HRI) strategy that is suitable for novice end users to e...
-
We are interested in understanding whether retrieval-based localization approaches are good enough in the context of self-driving vehicles. Towards this goal, we introduce Pit30M, a new image and LiDAR dataset with over 30 million frames, which is 10 to 100 times larger than those used in previous work. Pit30M is captured under diverse conditions (i.e., season, weather, time of the day, traffic), ...
-
In this paper, we propose SolarSLAM, a batteryfree loop closure method for indoor localisation. Inertial Measurement Unit (IMU) based indoor localisation method has been widely used due to its ubiquity in mobile devices, such as mobile phones, smartwatches and wearable bands. However, it suffers from the unavoidable long term drift. To mitigate the localisation error, many loop closure solutions h...
-
In this paper, the 2D robot-to-robot relative pose (position and orientation) estimation problem based on ego-motion and noisy distance measurements is considered. We address this problem using an optimization-based method, which does not require complicated numerical analysis while yields no inferior relative localization (RL) results compared to existing approaches. In particular, we start from ...
-
Constructing a spatial map of an indoor environment, e.g., a typical office environment with glass surfaces, is a difficult and challenging task. Current state-of-the-art, e.g., camera- and laser-based approaches are unsuitable for detecting transparent surfaces. Hence, the spatial map generated with these approaches are often inaccurate. In this paper, a method that utilizes echolocation with sou...
-
To achieve robust motion estimation in visually degraded environments, thermal odometry has been an attraction in the robotics community. However, most thermal odometry methods are purely based on classical feature extractors, which is difficult to establish robust correspondences in successive frames due to sudden photometric changes and large thermal noise. To solve this problem, we propose Ther...
-
Aiming for a lightweight and robust localization solution for low-cost, low-power autonomous robot platforms, such as educational or industrial ground vehicles, under challenging conditions (e.g., poor sensor calibration, low lighting and dynamic objects), we propose a two-stage localization system which incorporates both offline prior map building and online multi-modal localization. In particula...
-
Small unmanned aerial vehicles (UAV) have penetrated multiple domains over the past years. In GNSS-denied or indoor environments, aerial robots require a robust and stable localization system, often with external feedback, in order to fly safely. Motion capture systems are typically utilized indoors when accurate localization is needed. However, these systems are expensive and most require a fixed...
-
In this paper, we present a novel smoothing approach for ultra-wideband (UWB) aided unmanned aerial vehicle (UAV) positioning. Existing works based on smoothing or filtering estimate 3D position of UAV by updating a solution for each single 1D low-dimensional UWB range measurement. However, a low-dimensional single range measurement merely acts as a weak constraint in a solution space for UAV posi...
-
Localization is one of the most important technologies needed to use Unmanned Aerial Vehicles (UAVs) in actual fields. Currently, most UAVs use GNSS to estimate their position. Recently, there have been attacks that target the weaknesses of UAVs that use GNSS, such as interrupting GNSS signal to crash the UAVs or sending fake GNSS signals to hijack the UAVs. To avoid this kind of situation, this p...
-
This paper presents a novel method to improve the inertial velocity estimation of a mobile body, for indoor navigation, using solely raw data from a triad of inertial sensors (accelerometer and gyroscope), as well as a determined arrangement of magnetometers array. The key idea of the method is the use of deep neural networks to dynamically tune the measurement covariance matrix of an Extended Kal...
-
In this paper, we investigate the viability of implementing machine learning (ML) algorithms to solve the odor source localization (OSL) problem. The primary objective is to obtain an ML model that guides and navigates a mobile robot to find an odor source without explicating searching algorithms. To achieve this goal, the model of an adaptive neuro-fuzzy inference system (ANFIS) is employed to ge...
-
In this paper, we introduce a novel visual-inertial-wheel odometry (VIWO) system for ground vehicles, which efficiently fuses multi-modal visual, inertial and 2D wheel odometry measurements in a sliding-window filtering fashion. As multi-sensor fusion requires both intrinsic and extrinsic (spatiotemproal) calibration parameters which may vary over time during terrain navigation, we propose to perf...
-
This paper presents a novel localisation framework based on an omnidirectional camera, targeted at outdoor urban environments. Bearing only information to persistent and easily observable high-level semantic landmarks (such as lamp-posts, street-signs and trees) are perceived using a Convolutional Neural Network (CNN). The framework utilises an information theoretic strategy to decide the best vie...
-
Localization using ground texture images recorded with a downward-facing camera is a promising approach to achieve reliable high-accuracy vehicle positioning. A common way to accomplish the task is to focus on prominent features of the ground texture such as stones and cracks. Our results indicate that with an approximately known camera pose it is sufficient to use arbitrary ground regions, i.e. e...
-
In this work, we present a vision-only global localization architecture for autonomous vehicle applications, and achieves centimeter-level accuracy and high robustness in various scenarios. We first apply pixel-wise segmentation to the front-view mono camera and extract the semantic features, e.g. pole-like objects, lane markings, and curbs, which are robust to illumination, viewing angles and sea...
-
Light-weight camera localization in existing maps is essential for vision-based navigation. Currently, visual and visual-inertial odometry (VO&VIO) techniques are well-developed for state estimation but with inevitable accumulated drifts and pose jumps upon loop closure. To overcome these problems, we propose an efficient monocular camera localization method in prior LiDAR maps using direct 2D-3D ...
-
Easy, yet robust long-term localization is still an open topic in research. Existing approaches require either dense maps, expensive sensors, specialized map features or proprietary detectors.We propose using semantic segmentation on a monocular camera to localize directly in a HD map as used for automated driving. This combines lightweight, yet powerful HD maps with the simplicity of monocular vi...
-
Localization is a crucial capability for mobile robots and autonomous cars. In this paper, we address learning an observation model for Monte-Carlo localization using 3D LiDAR data. We propose a novel, neural network-based observation model that computes the expected overlap of two 3D LiDAR scans. The model predicts the overlap and yaw angle offset between the current sensor reading and virtual fr...
-
Many applications with mobile robots require self-localization in indoor maps. While such maps can be previously generated by SLAM strategies, there are various localization approaches that use 2D floor plans as reference input. In this paper, we present a localization strategy using floor plan as map, which is based on spatial density information computed from dense depth data of RGB-D cameras. W...
-
Point cloud registration is a means of achieving loop closure correction within a simultaneous localization and mapping (SLAM) algorithm. Data association is a critical component in point cloud registration, and can be very challenging in feature-depleted environments such as seabed. This paper presents a point cloud registration pipeline for performing loop closure correction in feature-depleted ...
-
In this paper we present a novel multi-stereo visual-inertial odometry (VIO) framework which aims to improve the robustness of a robot's state estimate during aggressive motion and in visually challenging environments. Our system uses a fixed-lag smoother which jointly optimizes for poses and landmarks across all stereo pairs. We propose a 1-point RANdom SAmple Consensus (RANSAC) algorithm which i...
-
Map based visual inertial localization is a crucial step to reduce the drift in state estimation of mobile robots. The underlying problem for localization is to estimate the pose from a set of 3D-2D feature correspondences, of which the main challenge is the presence of outliers, especially in changing environment. In this paper, we propose a robust solution based on efficient global optimization ...
-
This paper describes a method that corrects errors of a VSLAM-estimated trajectory for cars driving in GPS-denied environments, by applying constraints from public databases of geo-tagged images (Google Street View, Mapillary, etc). The method, dubbed Appearance-based Geo-Alignment for Simultaneous Localisation and Mapping (AGA-SLAM), encodes the available image database as an appearance map, whic...
-
In this paper, we propose a novel probabilistic variant of iterative closest point (ICP) dubbed as CoBigICP. The method leverages both local geometrical information and global noise characteristics. Locally, the 3D structure of both target and source clouds are incorporated into the objective function through bidirectional correspondence. Globally, error metric of correntropy is introduced as nois...
-
In this paper, we propose a flexible mapping scheme that uses a masking function (mask) to focus the attention of a pose graph SLAM (Simultaneous Localization and Mapping) system. The masking function takes the robot's observations and returns true if the robot is in an important location. State-of-the-art methods in SLAM generate dense metric lidar maps, creating precise maps at a high computatio...
-
This work addresses the problem of learning a model of a dynamic environment using many independent Hidden Markov Models (HMMs) with a limited number of observations available per iteration. Many techniques exist to model dynamic environments, but do not consider how to deploy robots to build this model. Additionally, there are many techniques for exploring environments that do not consider how to...
-
In this paper, we present a new approach, AKIMap, that uses an adaptive kernel inference for dense and sharp occupancy grid representations. Our approach is based on the multivariate kernel estimation, and we propose a simple, two-stage based method that selects an adaptive bandwidth matrix for an efficient and accurate occupancy estimation. To utilize correlations of occupancy observations given ...
-
Awareness of the environment is essential for mobile robots. Perception for legged robots requires high levels of reliability and accuracy in order to walk stably in the types of complex, cluttered environments we are interested in. In this paper, we present a usable environmental perception algorithm designed to detect steppable areas and obstacles for the autonomous generation of desired foothol...
-
This paper presents the perception, mapping, and planning pipeline implemented on an autonomous race car. It was developed by the 2019 AMZ driverless team for the Formula Student Germany (FSG) 2019 driverless competition, where it won 1st place overall. The presented solution combines early fusion of camera and LiDAR data, a layered mapping approach, and a planning approach that uses Bayesian filt...
-
The ability to efficiently utilize crowd-sourced visual data carries immense potential for the domains of large scale dynamic mapping and autonomous driving. However, state-of-the-art methods for crowdsourced 3D mapping assume prior knowledge of camera intrinsics. In this work we propose a framework that estimates the 3D positions of semantically meaningful landmarks such as traffic signs without ...
-
Fast, aerial navigation in cluttered environments requires a suitable map representation for path planning. In this paper, we propose the use of an efficient, structured multiresolution representation that expands the sensor range of dense local grids for memory-constrained platforms. While similar data structures have been proposed, we avoid processing redundant occupancy information and use the ...
-
With the rapidly developing unmanned aerial vehicles, the requirements of generating maps efficiently and quickly are increasing. To realize online mapping, we develop a real-time dense mapping framework named DenseFusion which can incrementally generates dense geo-referenced 3D point cloud, digital orthophoto map (DOM) and digital surface model (DSM) from sequential aerial images with optional GP...
-
Searching for a lost teammate is an important task for multirobot systems. We present a variant of rapidly-expanding random trees (RRT) for generating search paths based on a probabilistic belief of the target teammate's position. The belief is updated using a hidden Markov model built from knowledge of the target's planned or historic behavior. For any candidate search path, this belief is used t...
-
This paper presents a new coordination algorithm for decentralised multi-robot information gathering. We consider planning for an online variant of the multi-agent orienteering problem with neighbourhoods. This formulation closely aligns with a number of important tasks in robotics, including inspection, surveillance, and reconnaissance. We propose a decentralised variant of the self-organising ma...
-
This work presents an asynchronous multi-robot adaptive sampling strategy through the synthesis of an intermittently connected mobile robot communication network. The objective is to enable a team of robots to adaptively sample and model a nonlinear dynamic spatiotemporal process. By employing an intermittently connected communication network, the team is not required to maintain an all-time conne...
-
For multiple robots performing exploration in a previously unmapped environment, such as planetary exploration, maintaining accurate localization and building a consistent map are vital. If the robots do not have a map to localize against and do not explore the same area, they may not be able to find visual loop closures to constrain their relative poses, making traditional SLAM impossible. This p...
-
In the context of a multi-agent system that uses a Gaussian process to estimate a spatial field of interest, we propose an approach that enables an agent to reduce the amount of data it shares with other agents. The main idea of the strategy is to rigorously assign a novelty metric to each measurement as it is collected, and only measurements that are sufficiently novel are communicated. We consid...
-
In this paper, we propose π-map, a tightly coupled fusion mechanism that dynamically consumes LiDAR and sonar data to generate reliable and scalable indoor maps for autonomous robot navigation. The key novelty of π-map over previous attempts is the utilization of a fusion mechanism that works in three stages: the first LiDAR scan matching stage efficiently generates initial key localization poses;...
-
Visually poor scenarios are one of the main sources of failure in visual localization systems in outdoor environments. To address this challenge, we present MOZARD, a multi-modal localization system for urban outdoor environments using vision and LiDAR. By fusing key point based visual multi-session information with semantic data, an improved localization recall can be achieved across vastly diffe...
-
Sensor fusion systems merging (multiple) delayed sensor signals through a statistical approach are challenging setups, particularly for resource constrained platforms. For statistical consistency, one would be required to keep an appropriate history, apply the correcting signal at the given time stamp in the past, and re-apply all information received until the present time. This re-calculation be...
-
This paper describes a new method for synchronizing microphones based on spectral warping in an asynchronous microphone array. In an audio signal observed by an asynchronous microphone array, two factors are involved: the time lag caused by a mismatch of the sampling rate and offset between microphones, and the modulation caused by differences in spatial transfer function between the sound source ...
-
Detecting sound source objects within visual observation is important for autonomous robots to comprehend surrounding environments. Since sounding objects have a large variety with different appearances in our living environments, labeling all sounding objects is impossible in practice. This calls for self-supervised learning which does not require manual labeling. Most of conventional self-superv...
-
In this paper, we propose an autonomous exploration and a tapping mechanism-based material mapping system for a mobile robot in unknown environments. The goal of the proposed system is to integrate simultaneous localization and mapping (SLAM) modules and sound-based material classification to enable a mobile robot to explore an unknown environment autonomously and at the same time identify the var...
-
This article introduces a decentralized multi-robot algorithm for Simultaneous Localization And Mapping (SLAM) inspired from previous work on collaborative mapping [1]. This method makes robots jointly build and exchange i) a collection of 3D dense locally consistent submaps, based on a Truncated Signed Distance Field (TSDF) representation of the environment, and ii) a pose-graph representation wh...
-
LiDAR is playing a more and more essential role in autonomous driving vehicles for objection detection, self localization and mapping. A single LiDAR frequently suffers from hardware failure (e.g., temporary loss of connection) due to the harsh vehicle environment (e.g., temperature, vibration, etc.), or performance degradation due to the lack of sufficient geometry features, especially for solid-...
-
Many modern simultaneous localization and mapping (SLAM) techniques rely on sparse landmark-based maps due to their real-time performance. However, these techniques frequently assert that these landmarks are fixed in position over time, known as the static-world assumption. This is rarely, if ever, the case in most real-world environments. Even worse, over long deployments, robots are bound to obs...
-
This paper introduces a Wearable SLAM system that performs indoor and outdoor SLAM in real time. The related project is part of the MALIN challenge which aims at creating a system to track emergency response agents in complex scenarios (such as dark environments, smoked rooms, repetitive patterns, building floor transitions and doorway crossing problems), where GPS technology is insufficient or in...
-
We present a challenging dataset, the TartanAir, for robot navigation tasks and more. The data is collected in photo-realistic simulation environments with the presence of moving objects, changing light and various weather conditions. By collecting data in simulations, we are able to obtain multi-modal sensor data and precise ground truth labels such as the stereo RGB image, depth image, segmentat...
-
Planar structures are common in man-made environments. Their addition to monocular SLAM algorithms is of relevance in order to achieve more complete and higher- level scene representations. Also, the additional constraints they introduce might reduce the estimation errors in certain situations. In this paper we present a novel formulation to incorporate plane landmarks and planar constraints to fe...
-
This work describes a monocular visual odometry framework, which exploits the best attributes of edge features for illumination-robust camera tracking, while at the same time ameliorating the performance degradation of edge mapping. In the front-end, an ICP-based edge registration provides robust motion estimation and coarse data association under lighting changes. In the back-end, a novel edge-gu...
-
Simultaneous Localization and Mapping (SLAM) is considered significant for intelligent mobile robot autonomous pathfinding. Over the past years, many successful SLAM systems have been developed and works satisfactorily in static environments. However, in some dynamic scenes with moving objects, the camera pose estimation error would be unacceptable, or the systems even lose their locations. In thi...
-
In this paper, we propose a method to use semantic information to improve the use of map priors in a sparse, feature-based MonoSLAM system. To incorporate the priors, the features in the prior and SLAM maps must be associated with one another. Most existing systems build a map using SLAM and then align it with the prior map. However, this approach assumes that the local map is accurate, and the ma...
-
SLAM (Simultaneous Localization And Mapping) seeks to provide a moving agent with real-time self-localization. To achieve real-time speed, SLAM incrementally propagates position estimates. This makes SLAM fast but also makes it vulnerable to local pose estimation failures. As local pose estimation is ill-conditioned, local pose estimation failures happen regularly, making the overall SLAM system b...
-
Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry (VO) and simultaneous localization and mapping (SLAM), where classic methods consisting of hand-crafted features and sampling-based outlier rejection have been a dominant choice for over a decade. Although multiple works propose to replace these modules with learning-based counterparts, most have n...
-
A robust and efficient Simultaneous Localization and Mapping (SLAM) system is essential for robot autonomy. For visual SLAM algorithms, though the theoretical framework has been well established for most aspects, feature extraction and association is still empirically designed in most cases, and can be vulnerable in complex environments. This paper shows that feature extraction with deep convoluti...
-
Object-level data association and pose estimation play a fundamental role in semantic SLAM, which remain unsolved due to the lack of robust and accurate algorithms. In this work, we propose an ensemble data associate strategy for integrating the parametric and nonparametric statistic tests. By exploiting the nature of different statistics, our method can effectively aggregate the information of di...
-
In dynamic environments, performance of visual SLAM techniques can be impaired by visual features taken from moving objects. One solution is to identify those objects so that their visual features can be removed for localization and mapping. This paper presents a simple and fast pipeline that uses deep neural networks, extended Kalman filters and visual SLAM to improve both localization and mappin...
-
This paper presents a new open-source dataset with ground truth position in a simulated colon environment to promote development of real-time feedback systems for physicians performing colonoscopies. Four systems (DSO, LSD-SLAM, SfMLearner, ORB-SLAM2) are tested on this dataset and their failures are analyzed. A data collection platform was fabricated and used to take the dataset in a colonoscopy ...
-
Real-time dense 3D localization and mapping systems are required to enable robotics platforms to interact in and with their environments. Several solutions have used surfel representations to model the world. While they produce impressive results, they require heavy and costly hardware to operate properly. Many of them are also limited to static environments and small inter-frame motions. Whereas ...
-
This paper presents a new visual-inertial odometry, term DUI-VIO, for estimating the motion state of an RGB-D camera. First, a Gaussian mixture model (GMM) to is employed to model the uncertainty of the depth data for each pixel on the camera's color image. Second, the uncertainties are incorporated into the VIO's initialization and optimization processes to make the state estimate more accurate. ...
-
Simultaneous localization and mapping (SLAM) is essential in numerous robotics applications such as autonomous navigation. Traditional SLAM approaches infer the metric state of the robot along with a metric map of the environment. While existing algorithms exhibit good results, they are still sensitive to measurement noise, sensors quality, data association and are still computationally expensive....
-
The problem of tracking self-motion as well as motion of objects in the scene using information from a camera is known as multi-body visual odometry and is a challenging task. This paper proposes a robust solution to achieve accurate estimation and consistent track-ability for dynamic multi-body visual odometry. A compact and effective framework is proposed leveraging recent advances in semantic i...
-
This paper presents a novel Bayesian-based controller for snake robots in cluttered environment. It extends the conventional shape-based compliant control into statistical field providing an explicit mathematical formulation with Bayesian network. A sequential density propagation rule is derived by introducing several probability densities in a unified framework. Specifically, two input influence ...
-
In this paper, we present a new locomotion control method for soft robot snakes. Inspired by biological snakes, our control architecture is composed of two key modules: A reinforcement learning (RL) module for achieving adaptive goal-tracking behaviors with changing goals, and a central pattern generator (CPG) system with Matsuoka oscillators for generating stable and diverse locomotion patterns. ...
-
Snake robots have the potential to locomote through tightly packed spaces, but turning effectively within unmodelled and unsensed environments remains challenging. Inspired by a behavior observed in the tiny nematode worm C. elegans, we propose a novel in-place turning gait for elongated limbless robots. To simplify the control of the robots' many internal degrees-of-freedom, we introduce a biolog...
-
Landing upside down on a ceiling is challenging as it requires a flier to invert its body and land against the gravity, a process that demands a stringent spatiotemporal coordination of body translational and rotational motion. Although such an aerobatic feat is routinely performed by biological fliers such as flies, it is not yet achieved in aerial robots using onboard sensors. This work describe...
-
Estimates of limb posture are critical for the control of robotic systems. This is generally accomplished by utilizing on-location joint angle encoders which may complicate the design, increase limb inertia, and add noise to the system. Conversely, some innovative or smaller robotic morphologies can benefit from non-collocated sensors when encoder size becomes prohibitively larger or the joints ar...
-
In this research, we have taken a biomimetic approach to the control of musculoskeletal humanoids. A controller was designed based on the motor directional tuning phenomenon seen in the motor cortex of primates. Despite the simple implementation of the control scheme, complex coordinated movements such as reaching for target objects with its upper body was achieved, and is demonstrated in the acco...
-
Interests in exploration of new energy resources are increasing due to the exhaustion of existing resources. To explore new energy sources, various studies have been conducted to improve the drilling performance of drilling equipment for deep and strong ground. However, with better performance, the modern drilling equipment is bulky and, furthermore, has become inconvenient in both installation an...
-
While most modern-day quadruped robots crouch their limbs during the stance phase to stabilize the trunk, mammals exploit the inverted-pendulum motions of their limbs and realize both efficient and stable walking. Although the flexibility of the shoulder region of mammals is expected to contribute to reconciling the discrepancy between the forelimbs and hindlimbs for natural walking, the complex b...
-
Earthworm-like soft robots have been widely studied for various applications, such as medical endoscopy and pipeline inspection. Many actuation modes have been chosen to drive the soft robots, including pneumatic actuators, dielectric elastomeric actuators, and shape memory actuators. Pneumatic actuators stand out since the soft robots with pneumatic actuation can produce relatively large forces a...
-
In this paper, the design, fabrication, numerical studies, and preliminary characterization of a multi-fin soft robot are presented. The design is simple, robust, and fully autonomous. The robot has a 216mm body length and displays great potential to achieve uncoupled surge (forwards and backwards), sway, and heave motions. Computational fluid dynamic (CFD) studies are employed to evaluate appropr...
-
The bodies of quadrupeds have very complex muscle-tendon structure. In particular, it is known that in the horse hindlimb, multiple joints in the leg are remarkably interlocked due to the muscle-tendon structure. Although the function of these interlocking mechanisms during standing has been investigated in the field of anatomy, the function related to the emergence of limb trajectory during dynam...
-
Gait emergence and adaptation in animals is unmatched in robotic systems. Animals can create and recover locomotive functions "on-the-fly" after an injury whereas locomotion controllers for robots lack robustness to morphological changes. In this work, we extend previous research on emergent interlimb coordination of legged robots based on coupled phase oscillators with force feedback terms. We in...
-
In this paper we present the design, fabrication, testing, and control of a 0.4 g milliscale robot employing a soft polymer flexure transmission for rapid ground movement. The robot was constructed through a combination of two methods: smart-composite-manufacturing (SCM) process to fabricate the actuators and robot chassis, and silicone elastomer molding and casting to fabricate a soft flexure tra...
-
We present the first demonstration of a battery-free untethered wirelessly powered sub-gram jumping robot on an insect-scale. In order to operate the insect-sized robot autonomously, the limitation in battery use emphasizes the need for a wireless power transmission system as an onboard power solution. We designed a wireless power transmission system based on inductive coupling to power the Shape ...
-
This work investigates the problem of simultaneous tracking and jamming of a rogue drone in 3D space with a team of cooperative unmanned aerial vehicles (UAVs). We propose a decentralized estimation, decision and control framework in which a team of UAVs cooperate in order to a) optimally choose their mobility control actions that result in accurate target tracking and b) select the desired transm...
-
In semantic mapping, which connects semantic information to an environment map, it is a challenging task for robots to deal with both local and global information of environments. In addition, it is important to estimate semantic information of unobserved areas from already acquired partial observations in a newly visited environment. On the other hand, previous studies on spatial concept formatio...
-
In this paper we investigate how social robots can efficiently gather user preferences without exceeding the allowed user annoyance threshold. To do so, we use a Gazebo based simulated office environment with a TIAGo Steel robot. We then formulate the user annoyance aware preference elicitation problem as a combination of tensor completion and knapsack problems. We then test our approach on the af...
-
One of the challenges of full autonomy is to have robots capable of manipulating its current environment to achieve another environment configuration. This paper is a step towards this challenge, focusing on the visual understanding of the task. Our approach trains a deep neural network to represent images as measurable features that are useful to estimate the progress (or phase) of a task. The tr...
-
Accurately forecasting the future movements of surrounding vehicles is essential for safe and efficient operations of autonomous driving cars. This task is difficult because a vehicle's moving trajectory is greatly determined by its driver's intention, which is often hard to estimate. By leveraging attention mechanisms along with long short-term memory (LSTM) networks, this work learns the relatio...
-
This paper proposes a multiple stakeholder perspective model (MSPM) which predicts the future pedestrian trajectory observed from vehicle's point of view. For the vehicle-pedestrian interaction, the estimation of the pedestrian's intention is a key factor. However, even if this interaction is commonly initiated by both the human (pedestrian) and the agent (driver), current research focuses on deve...
-
We investigate the task and motion planning problem of clearing clutter from a workspace with limited ingress/egress access for multiple robots. We call the problem multi-robot clutter removal (MRCR). Targeting practical applications where motion planning is non-trivial but is not a bottle-neck, we focus on finding high-quality solutions for feasible MRCR instances, which depends on the ability to...
-
Multi-Robot Systems have been recently employed in different applications and have advantages over single-robot systems, such as increased robustness and task performance efficiency. We consider such assemblies specifically in the scenario of perimeter defense, where the task is to defend a circular perimeter by intercepting radially approaching targets. Possible intruders appear randomly at a fix...
-
To realize effective heterogeneous multi-agent teams, we must be able to leverage individual agents' relative strengths. Recent work has addressed this challenge by introducing trait-based task assignment approaches that exploit the agents' relative advantages. These approaches, however, assume that the agents' traits remain static. Indeed, in real-world scenarios, traits are likely to vary as age...
-
Multi-robot cooperation requires agents to make decisions that are consistent with the shared goal without disregarding action-specific preferences that might arise from asymmetry in capabilities and individual objectives. To accomplish this goal, we propose a method named SLiCC: Stackelberg Learning in Cooperative Control. SLiCC models the problem as a partially observable stochastic game compose...
-
Recent approaches to multi-agent reinforcement learning (MARL) with inter-agent communication have often overlooked important considerations of real-world communication networks, such as limits on bandwidth. In this paper, we propose an approach to learning sparse discrete communication through backpropagation in the context of MARL, in which agents are incentivized to communicate as little as pos...
-
We investigate the problem of multi-robot coordinated planning in environments where the robots may have to operate in close proximity to each other. We seek computationally efficient planners that ensure safe paths and adherence to kinematic constraints. We extend the central planner dRRT* with our variant, fast-dRRT (fdRRT), with the intention being to use in tight environments that lead to a hi...
-
Among the available solutions for drone swarm simulations, we identified a lack of simulation frameworks that allow easy algorithms prototyping, tuning, debugging and performance analysis. Moreover, users who want to dive in the research field of drone swarms often need to interface with multiple programming languages. We present SwarmLab, a software entirely written in MATLAB, that aims at the cr...
-
Programming cooperative tasks for autonomous swarm robotic systems has always been challenging. In this paper, we introduce a concept ‘Actor’, as a virtualization for robot platforms. Every robot platform in the swarm robotic system carries out the task and interacts with others as an Actor. We designed an Actor-based framework for the management of autonomous swarm robotic systems including modul...
-
The challenges of developing low-cost, large-scale multi-robot navigation systems include noisy measurements, a large number of robots, and computing efficiency for collision avoidance. This paper presents a distributed motion planning framework for a large number of robots to navigate with robust collision avoidance using low-cost range only measurements. The novelty of this work is threefold. (1...
-
In this paper, we propose an abstraction that captures high-level formation and location-based swarm behaviors, and an automated control synthesis framework to generate correct-by-construction behaviors. Our abstraction includes symbols representing both possible formations and physical locations in the workspace. We allow users to write linear temporal logic (LTL) specifications over the symbols ...
-
Rapid progress in embedded computing hardware increasingly enables on-board image processing on small robots. This development opens the path to replacing costly sensors with sophisticated computer vision techniques. A case in point is the prediction of scene depth information from a monocular camera for autonomous navigation. Motivated by the aim to develop a robot swarm suitable for sensing, mon...
-
Recognising in what type of environment one is located is an important perception task. For instance, for a robot operating indoors it is helpful to be aware whether it is in a kitchen, a hallway or a bedroom. Existing approaches attempt to classify the scene based on 2D images or 2.5D range images. Here, we study scene recognition from 3D point cloud (or voxel) data, and show that it greatly outp...
-
Humans perceive and describe their surroundings with qualitative statements (e.g., "Alice's hand is in contact with a bottle."), rather than quantitative values (e.g., 6-D poses of Alice's hand and a bottle). Qualitative spatial representation (QSR) is a framework that represents the spatial information of objects in a qualitative manner. Region connection calculus (RCC), qualitative trajectory ca...
-
We consider the problem of acquiring mechanical knowledge through visual cues to help robots use objects in new situations. In this work, we propose a novel deep learning approach that allows a robot to acquire mechanical knowledge from 3D point clouds. This presents two main challenges. The first challenge is that a robot needs to infer novel objects' functions from its experience. Secondly, the ...
-
Manipulation and assembly tasks require non-trivial planning of actions depending on the environment and the final goal. Previous work in this domain often assembles particular instances of objects from known sets of primitives. In contrast, we aim to handle varying sets of primitives and to construct different objects of a shape category. Given a single object instance of a category, e.g. an arch...
-
To act effectively in its environment, a cognitive robot needs to understand the causal dependencies of all intermediate actions leading up to its goal. For example, the system has to infer that it is instrumental to open a cupboard door before trying to grasp an object inside the cupboard. In this paper, we introduce a novel learning method for extracting instrumental dependencies by following th...
-
This paper introduces an algorithm for discovering implicit and delayed causal relations between events observed by a robot at arbitrary times, with the objective of improving data-efficiency and interpretability of model- based reinforcement learning (RL) techniques. The proposed algorithm initially predicts observations with the Markov assumption, and incrementally introduces new hidden variable...
-
It has been claimed that a main advantage of cognitive architectures (compared to other types of specialized robotic architectures) is that they are task-general and can thus learn to perform any task as long as they have the right perceptual and action primitives. In this paper, we provide empirical evidence for this claim by directly comparing a high-performing custom robotic architecture develo...
-
3D object trackers usually require training on large amounts of annotated data that is expensive and time-consuming to collect. Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association. Large scale annotations for unlabeled data are cheaply obtained by automatic object detection and association across frames. ...
-
This paper presents F-Siamese Tracker, a novel approach for single object tracking prominently characterized by more robustly integrating 2D and 3D information to reduce redundant search space. A main challenge in 3D single object tracking is how to reduce search space for generating appropriate 3D candidates. Instead of solely relying on 3D proposals, firstly, our method leverages the Siamese net...
-
As warehouses, storage facilities and factories become more expanded and equipped with smart devices, there is a substantial need for rapid, intelligent and autonomous detection of unusual and potentially hazardous situations, also called anomalies. In particular for Autonomous Guided Vehicles (AGVs) that drive around these premises independently, unforeseen obstructions along their path-e.g. a ca...
-
This paper deals with predicting future 3D motions of 3D object scans from the previous two consecutive frames. Previous methods mostly focus on sparse motion prediction in the form of skeletons. While in this paper, we focus on predicting dense 3D motions in the form of 3D point clouds. To approach this problem, we propose a self-supervised approach that leverages the power of the deep neural net...
-
Robots operating in human environments must be careful, when executing their manipulation skills, not to disturb nearby objects. This requires robots to reason about the effect of their manipulation choices by accounting for the support relationships among objects in the scene. Humans do this in part by visually assessing their surroundings and using physics intuition for how likely it is that a p...
-
Point-clouds are a popular choice for robotics and computer vision tasks due to their accurate shape description and direct acquisition from range-scanners. This demands the ability to synthesize and reconstruct high-quality point-clouds. Current deep generative models for 3D data generally work on simplified representations (e.g., voxelized objects) and cannot deal with the inherent redundancy an...
-
In classical computer vision, rectification is an integral part of multi-view depth estimation. It typically includes epipolar rectification and lens distortion correction. This process simplifies the depth estimation significantly, and thus it has been adopted in CNN approaches. However, rectification has several side effects, including a reduced field of view (FOV), resampling distortion, and se...
-
Given an image or a video captured from a monocular camera, amodal layout estimation is the task of predicting semantics and occupancy in bird's eye view. The term amodal implies we also reason about entities in the scene that are occluded or truncated in image space. While several recent efforts have tackled this problem, there is a lack of standardization in task specification, datasets, and eva...
-
Nowadays, the prevalence of sensor networks has enabled tracking of the states of dynamic objects for a wide spectrum of applications from autonomous driving to environmental monitoring and urban planning. However, tracking realworld objects often faces two key challenges: First, due to the limitation of individual sensors, state estimation needs to be solved in a collaborative and distributed man...
-
We present ProxEmo, a novel end-to-end emotion prediction algorithm for socially aware robot navigation among pedestrians. Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints. To classify emotions, we propose a multi-view skeleton graph convolution-based model that work...
-
Due to the difficulty in generating the effective descriptors which are robust to occlusion and viewpoint changes, place recognition for 3D point cloud remains an open issue. Unlike most of the existing methods that focus on extracting local, global, and statistical features of raw point clouds, our method aims at the semantic level that can be superior in terms of robustness to environmental chan...
-
For high-level human-robot interaction tasks, 3D scene understanding is important and non-trivial for autonomous robots. However, parsing and utilizing effective environment information of the 3D scene is not trivial due to the complexity of the 3D environment and the limited ability for reasoning about our visual world. Although there have been great efforts on semantic detection and scene analys...
-
Current approaches to physical security suffer from high false alarm rates and frequent human operator involvement, despite the relative rarity of real-world threats. We present a novel architecture for autonomous adaptive physical security called autonomous detection and assessment with moving sensors (ADAMS). ADAMS is a framework for reducing nuisance and false alarms by placing mobile robotic p...
-
Neural networks have recently achieved impressive success in semantic and instance segmentation on 2D images. However, their capabilities have not been fully explored to address semantic instance segmentation on unstructured 3D point cloud data. Digging into the regional feature representation to boost point cloud comprehension, we propose a region-feature-enhanced structure consisting of adaptive...
-
Inferring semantic information towards an understanding of the surrounding environment is crucial for autonomous vehicles to drive safely. Deep learning-based segmentation methods can infer semantic information directly from laser range data, even in the absence of other sensor modalities such as cameras. In this paper, we address improving the generalization capabilities of such deep learning mod...
-
In this paper, we propose a camera control system towards occasionally videoing preassigned objects. Based on the technique of real-time visual detection and tracking, using the Kalman filter and re-identification (ReID), we propose continuous composition of lens, based on the atomic rules of shots, and give the trajectory planning of the camera, to generate the PID controller to the pan-tilt. By ...
-
We present a novel Multi-Relational Graph Convolutional Network (MRGCN) based framework to model on-road vehicle behaviors from a sequence of temporally ordered frames as grabbed by a moving monocular camera. The input to MRGCN is a multi-relational graph where the graph's nodes represent the active and passive agents/objects in the scene, and the bidirectional edges that connect every pair of nod...
-
Large special-events parking involves various parking scenarios, e.g., temporary parking and on-street parking. Their occupancy detection is challenging as it is unrealistic to construct gates/stations for temporary parking areas or build a sensor-based detection system to cover every single street. To address this issue, this study develops a quadrotor-enabled autonomous parking occupancy detecti...
-
Correlation filter (CF)-based methods have demonstrated exceptional performance in visual object tracking for unmanned aerial vehicle (UAV) applications, but suffer from the undesirable boundary effect. To solve this issue, spatially regularized correlation filters (SRDCF) proposes the spatial regularization to penalize filter coefficients, thereby significantly improving the tracking performance....
-
Physically soft robots are promising for robotic assembly tasks as they allow stable contacts with the environment. In this study, we propose a novel learning system for soft robotic assembly strategies. We formulate this problem as a reinforcement learning task and design the reward function from human demonstrations. Our key insight is that the failed demonstrations can be used as constraints to...
-
A robot in the future may initially has a good learning capability but an empty library of movements. It gradually enriches its library of movements through human demonstrations. Dynamic Movement Primitives (DMPs) has been proved to be an effective way to represent trajectories. Trajectories are classified into discrete and rhythmic ones, and parameters are set for each demonstrated trajectory. Ho...
-
Deploying robot learning frameworks in unconstrained environments requires robustness and tractability. We must not only equip the robot with a sufficient range of sensing capabilities, but also provide training data in a sample-efficient manner. To this end, we identify and address a need specifically in robot learning from demonstration (LfD) literature to account for not only end-effector pose ...
-
This paper proposes a dynamic system based learning from demonstration approach to teach a robot activities of daily living. The approach takes inspiration from human movement literature to formulate trajectory learning as an optimal control problem. We assume a weighted combination of basis objective functions is the true objective function for a demonstrated motion. We derive basis objective fun...
-
For many applications, robots will need to be incrementally trained to recognize the specific objects needed for an application. This paper presents a practical system for incrementally training a robot to recognize different object categories using only a small set of visual examples provided by a human. The paper uses a recently developed state-of-the-art method for few-shot incremental learning...
-
Place recognition is a critical component towards addressing the key problem of Simultaneous Localization and Mapping (SLAM). Most existing methods use visual images; whereas, place recognition using 3D point clouds, especially based on the voxel representations, has not been well addressed yet. In this paper, we introduce the novel approach of voxel-based representation learning (VBRL) that uses ...
-
Understanding spatial relations of objects is critical in many robotic applications such as grasping, manipulation, and obstacle avoidance. Humans can simply reason object's spatial relations from a glimpse of a scene based on prior knowledge of spatial constraints. The proposed method enables a robot to comprehend spatial relationships among objects from RGB-D data. This paper proposed a neural-l...
-
Manipulation tasks in daily life, such as pouring water, unfold through human intentions. Being able to process contextual knowledge from these Activities of Daily Living (ADLs) over time can help us understand manipulation intentions, which are essential for an intelligent robot to transition smoothly between various manipulation actions. In this paper, to model the intended concepts of manipulat...
-
Understanding spatial relations is a key element for natural human-robot interaction. Especially, a robot must be able to manipulate a given scene according to a human verbal command specifying desired spatial relations between objects. To endow robots with this ability, a suitable representation of spatial relations is necessary, which should be derivable from human demonstrations. We claim that ...
-
The ability to synchronize expectations among human-robot teams and understand discrepancies between expectations and reality is essential for human-robot collaboration scenarios. To ensure this, human activities and intentions must be interpreted quickly and reliably by the robot using various modalities. In this paper we propose a multimodal recognition system designed to detect physical interac...
-
When executing a certain task, human beings can choose or make an appropriate tool to achieve the task. This research especially addresses the optimization of tool shape for robotic tool-use. We propose a method in which a robot obtains an optimized tool shape, tool trajectory, or both, depending on a given task. The feature of our method is that a transition of the task state when the robot moves...
-
While abstract knowledge like cause-and-effect relations enables robots to problem-solve in new environments, acquiring such knowledge remains out of reach for many traditional machine learning techniques. In this work, we introduce a method for a robot to learn an explicit model of cause-and-effect by constructing a structural causal model through a mix of observation and self-supervised experime...
-
While grounded language learning, or learning the meaning of language with respect to the physical world in which a robot operates, is a major area in human-robot interaction studies, most research occurs in closed worlds or domain-constrained settings. We present a system in which language is grounded in visual percepts without using categorical constraints by combining CNN-based visual featuriza...
-
When searching for objects in cluttered environments, it is often necessary to perform complex interactions in order to move occluding objects out of the way and fully reveal the object of interest and make it graspable. Due to the complexity of the physics involved and the lack of accurate models of the clutter, planning and controlling precise predefined interactions with accurate outcome is ext...
-
Learning-based approaches to grasp planning are preferred over analytical methods due to their ability to better generalize to new, partially observed objects. However, data collection remains one of the biggest bottlenecks for grasp learning methods, particularly for multi-fingered hands. The relatively high dimensional configuration space of the hands coupled with the diversity of objects common...
-
We propose a method to annotate segmentation masks accurately and automatically using invisible marker for object manipulation. Invisible marker is invisible under visible (regular) light conditions, but becomes visible under invisible light, such as ultraviolet (UV) light. By painting objects with the invisible marker, and by capturing images while alternately switching between regular and UV lig...
-
Video object segmentation plays a vital role to many robotic tasks, beyond the satisfied accuracy, quickly adapt to the new scenario with very limited annotations and conduct a quick inference are also important. In this paper, we are specifically concerned with the task of fast segmenting all pixels of a target object in all frames, given the annotation mask in the first frame. Even when such ann...
-
In this paper, we propose a cascaded non-local neural network for point cloud segmentation. The proposed network aims to build the long-range dependencies of point clouds for the accurate segmentation. Specifically, we develop a novel cascaded non-local module, which consists of the neighborhood-level, superpoint-level and global-level non-local blocks. First, in the neighborhood-level block, we e...
-
Identifying new, moved or missing objects is an important capability for robot tasks such as surveillance or maintaining order in homes, offices and industrial settings. However, current approaches do not distinguish between novel objects or simple scene readjustments nor do they sufficiently deal with localization error and sensor noise. To overcome these limitations, we combine the strengths of ...
-
The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions. Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches. In this work, we propose a multimodal semantic segmentation model that can be applied during daytim...
-
Following considerable development in 3D scanning technologies, many studies have recently been proposed with various approaches for 3D vision tasks, including some methods that utilize 2D convolutional neural networks (CNNs). However, even though 2D CNNs have achieved high performance in many 2D vision tasks, existing works have not effectively applied them onto 3D vision tasks. In particular, se...
-
We present a novel end-to-end single-shot method that segments countable object instances (things) as well as background regions (stuff) into a non-overlapping panoptic segmentation at almost video frame rate. Current state-of-the-art methods are far from reaching video frame rate and mostly rely on merging instance segmentation with semantic background segmentation, making them impractical to use...
-
Personal robots and driverless cars need to be able to operate in novel environments and thus quickly and efficiently learn to recognise new object classes. We address this problem by considering the task of video object segmentation. Previous accurate methods for this task finetune a model using the first annotated frame, and/or use additional inputs such as optical flow and complex post-processi...
-
This paper focuses on pose registration of different object instances from the same category. This is required in online object mapping because object instances detected at test time usually differ from the training instances. Our approach transforms instances of the same category to a normalized canonical coordinate frame and uses metric learning to train fully convolutional geometric features. T...
-
Surgical tool segmentation in endoscopic images is an important problem: it is a crucial step towards full instrument pose estimation and it is used for integration of pre- and intra-operative images into the endoscopic view. While many recent approaches based on convolutional neural networks have shown great results, a key barrier to progress lies in the acquisition of a large number of manually-...
-
Truly autonomous driving without the need for human intervention can only be attained when self-driving cars fully understand their surroundings. Most of these vehicles rely on a suite of active and passive sensors. LiDAR sensors are a cornerstone in most of these hardware stacks, and leveraging them as a complement to other passive sensors such as RGB cameras is an enticing goal. Understanding th...
-
Detecting small obstacles on the road is critical for autonomous driving. In this paper, we present a method to reliably detect such obstacles through a multi-modal framework of sparse LiDAR(VLP-16) and Monocular vision. LiDAR is employed to provide additional context in the form of confidence maps to monocular segmentation networks. We show significant performance gains when the context is fed as...
-
We propose a method to facilitate robot navigation relative to sketched maps of human environments. Our main contribution centers around using thin plate splines for registering the robot's LIDAR observation with the hand-drawn maps. Thin plate splines are particularly effective for this task because they are able to handle many of the nonrigid deformations commonly seen in sketches of maps, which...
-
In this paper, we propose a dependable visual kidnap recovery (KR) framework that pinpoints a unique pose in a given 3D map when a device is turned on. For this framework, we first develop indoor-GeM (i-GeM), which is an extension of GeM [1] but considerably more robust than other global descriptors [2]-[4], including GeM itself. Then, we propose a convolutional neural network (CNN)-based system c...
-
In this paper, we study the localization problem under non-Gaussian noise. In particular, we consider systems that can be represented by a state transition and a measurement component. The state transition indicates how the system evolves given a control variable. The measurement component compares, for a given state, the received and predicted measurements. Here we consider a radio based range se...
-
SpoxelNet: Spherical Voxel-based Deep Place Recognition for 3D Point Clouds of Crowded Indoor Spaces
With its essential role in achieving full autonomy of robot navigation, place recognition has been widely studied with various approaches. Recently, numerous point cloud-based methods with deep learning implementation have been proposed with promising results for their application in outdoor environments. However, their performances are not as promising in indoor spaces because of the high level o...
-
Many indoor spaces have constantly changing layouts and may not be mapped by an autonomous vehicle, yet maps such as floor plans or evacuation maps of these places are common. We propose a method for an autonomous robot to localize itself on such maps with inconsistent scale using Stochastic Gradient Descent (SGD) with scan matching using a 2D LiDAR. We also introduce a new scale state in 2D local...
-
Focal-plane Sensor-processor (FPSP) is a next-generation camera technology which enables every pixel on the sensor chip to perform computation in parallel, on the focal plane where the light intensity is captured. SCAMP-5 is a general-purpose FPSP used in this work and it carries out computations in the analog domain before analog to digital conversion. By extracting features from the image on the...
-
The field of soft robotics has evolved as a domain for developing light, compliant and safe actuators. However, one of the challenges in the field is the lack of repeatable fabrication techniques as well as customizability that restricts the application of soft robots. We present a fabrication technique using sacrificial molding to fabricate pneumatic channels that are repeatable and less prone to...
-
Super-coiled polymer (SCP) artificial muscles have many interesting properties that show potentials for making high performance bionic devices. To realize human-like robotic devices from this type of actuator, it is important for the SCP driven mechanisms to achieve human-like performance, such as compliant behaviors through antagonistic mechanisms. This paper presents the simultaneous position-st...
-
Soft materials with embedded magnetic properties can be actuated in a contactless manner for dexterous motion in restricted and unstructured environments. Magnetic soft robots have been demonstrated to be capable of versatile and programmable untethered motion. However, magnetic soft robots reported in literature are typically actuated by utilizing magnetic fields to generate torques that produce ...
-
Soft links and actuators are nowadays emerging technologies aiming to overcome some problems in robotics such as weight, cost or human interaction. However, the nonlinear nature of their elements can make their characterization challenging and hinder the use of standard control engineering tools. In this paper, we explore different state-of-the-art identification methods for the soft neck, in orde...
-
In the last decade, soft robots have been at the forefront of a robotic revolution. Due to the flexibility of the soft materials employed, soft robots are equipped with a capability to execute new tasks in new application areas -beyond what can be achieved using classical rigid-link robots. Despite these promising properties, many soft robots nowadays lack the capability to exert sufficient force ...
-
Choosing a kinematic model for a continuum robot typically involves making a tradeoff between accuracy and computational complexity. One common modeling approach is to use the Cosserat rod equations, which have been shown to be accurate for many types of continuum robots. This approach, however, still presents significant computational cost, particularly when many Cosserat rods are coupled via kin...
-
Soft actuators have been widely studied in recent years because of their ability to adapt to diverse environments and safely interact with humans. Their softness broadens their potential range of medical applications since they can provide inherent safety. Among the various motions a soft robot can perform, "torsion" can maximize the efficiency of motion in confined spaces like the human abdominal...
-
This paper presents the design of a new soft pneumatic actuator whose direction and magnitude of bending may be precisely controlled via activation of different shape memory alloy (SMA) springs within the actuator, in conjunction with pneumatic actuation. This design is inspired by examples seen in nature such as the human tongue, where the combination of hydrostatic pressure and contraction of in...
-
Soft pneumatic actuators are commonly used in robotics for creating single-axis compression, extension, or bending motions. If these actuators are composed of compliant materials, they can also have low off-axis stiffnesses, making it difficult to restrict off-axis motions. In this work, we exploit the low off-axis stiffnesses of pneumatic actuators to design a modular actuator system that is capa...
-
This paper presents the design of the Multi-material Actuator for Variable Stiffness (MAVS), which consists of an inflatable soft fabric actuator fixed between two layers of rigid retainer pieces. The MAVS is designed to be integrated with a soft robotic ankle-foot orthosis (SR-AFO) exosuit to aid in supporting the human ankle in the inversion/eversion directions. This design aims to assist indivi...
-
Robotic insertion tasks are characterized by contact and friction mechanics, making them challenging for conventional feedback control methods due to unmodeled physical effects. Reinforcement learning (RL) is a promising approach for learning control policies in such settings. However, RL can be unsafe during exploration and might require a large amount of real-world training data, which is expens...
-
In this paper, we present an approach and an implemented pipeline for transferring data acquired from observing humans in virtual environments onto robots acting in the real world, and adapting the data accordingly to achieve successful task execution. We demonstrate our pipeline by inferring seven different symbolic and subsymbolic motion parameters of mobile pick and place actions, which allows ...
-
Over the last decades, Learning from Demonstration (LfD) has become a widely accepted solution for the problem of robot programming. According to LfD, the kinematic behavior is "taught" to the robot, based on a set of motion demonstrations performed by the human-teacher. The demonstrations can be either captured via kinesthetic teaching or external sensors, e.g., a camera. In this work, a controll...
-
In this paper we address variable admittance control for human-robot physical interaction in manual guidance applications. In the proposed solution, the parameters of the admittance filter can change not only as a function of the current state of motion (i.e. whether the human guiding the robot ia accelerating or decelerating) but also with reference to a predefined goal position. The human is in ...
-
During robot-assisted therapy of hemiplegic patients, interaction with the patient must be intrinsically safe. Straight-forward collision avoidance solutions can provide this safety requirement with conservative margins. These margins heavily reduce the robot's workspace and make interaction with the patient's unguided body parts impossible. However, interaction with the own body is highly benefic...
-
This paper presents a concept for haptic-based human-in-the-loop aerial manipulation for drilling. The concept serves as a case study for designing the human-drone interface to remotely drill with a mobile-manipulating drone. The notion of the work stems from using drones to perform dangerous tasks like material assembly, sensor insertion while being vertically elevated from bridge, wind turbine, ...
-
Although research indicates that telepresence robots offer a more socially telepresent alternative to conventional forms of remote communication, the lack of touch-based interactions presents challenges for both remote and local users. In order to address these challenges, we have designed and implemented a robotic manipulator emulating a human arm. However, contact interactions like handshakes wi...
-
Haptic feedback in teleoperation of flying robots can enable safe flight in unknown and densely cluttered environments. It is typically part of the robot's control scheme and used to aid navigation and collision avoidance via artificial force fields displayed to the operator. However, to achieve fully immersive embodiment in this context, high fidelity force feedback is needed. In this paper we pr...
-
We present our second generation tactile sensor for the Shadow Dexterous Hand's palm. We were able to significantly improve the tactile sensor characteristics by utilizing our latest barometer-based tactile sensing technology with linear (R2 ≥ 0.9996) sensor output and no noticeable hysteresis. The sensitivity threshold of the tactile cells and the spatial density were both dramatically increased....
-
A tactile sensor with high spatio-temporal resolution will greatly contribute to improving the performance of object recognition and human interaction in robots. In addition, being able to switch between higher spatial and higher temporal resolution will allow for more versatile sensing. To realize such a sensor, this paper introduces a method of increasing the sensing electrodes and adaptively se...
-
We present a modified TacTip biomimetic optical tactile sensor design which demonstrates the ability to induce and detect incipient slip, as confirmed by recording the movement of markers on the sensor's external surface. Incipient slip is defined as slippage of part, but not all, of the contact surface between the sensor and object. The addition of ridges - which mimic the friction ridges in the ...
-
In this study, we propose a method of noncontact elastography, which allows us to investigate stiffness of soft structures by combining optical and acoustic modalities. We use optical coherence tomography (OCT) as a means of detecting internal deformation of a sample appearing in response to a mechanical force applied by acoustic radiation pressure. Unlike most of other stiffness sensing, this met...
-
Tactile sensing is inherently contact based. To use tactile data, robots need to make contact with the surface of an object. This is inefficient in applications where an agent needs to make a decision between multiple alternatives that depend the physical properties of the contact location. We propose a method to get tactile data in a non-invasive manner. The proposed method estimates the output o...
-
Tactile sensing has been used for a variety of robotic exploration and manipulation tasks but a common constraint is a requirement for a large amount of training data. This paper addresses the issue of data-efficiency by proposing a novel method for online learning based on a Gaussian Process Latent Variable Model (GP-LVM), whereby the robot learns from tactile data whilst performing a contour fol...
-
In this paper, we present a novel approach for classification of unseen object instances from interactive tactile feedback. Furthermore, we demonstrate the utility of a low resolution tactile sensor array for tactile perception that can potentially close the gap between vision and physical contact for manipulation. We contrast our sensor to high-resolution camera-based tactile sensors. Our propose...
-
Little research into tactile feet has been done for walking robots despite the benefits such feedback could give when walking on uneven terrain. This paper describes the development of a simple, robust and inexpensive tactile foot for legged robots based on a high-resolution biomimetic TacTip tactile sensor. Several design improvements were made to facilitate tactile sensing while walking, includi...
-
Tactile perception is crucial for a variety of robot tasks including grasping and in-hand manipulation. New advances in flexible, event-driven, electronic skins may soon endow robots with touch perception capabilities similar to humans. These electronic skins respond asynchronously to changes (e.g., in pressure, temperature), and can be laid out irregularly on the robot’s body or end-effector. How...
-
Restoring tactile sensation is essential to enable in-hand manipulation and the smooth, natural control of upper-limb prosthetic devices. Here we present a platform to contribute to that long-term vision, combining an anthropomorphic robot hand (QB SoftHand) with a neuromorphic optical tactile sensor (neuroTac). Neuromorphic sensors aim to produce efficient, spike-based representations of informat...
-
Touch is arguably the most important sensing modality in physical interactions. However, tactile sensing has been largely under-explored in robotics applications owing to the complexity in making perceptual inferences until the recent advancements in machine learning or deep learning in particular. Touch perception is strongly influenced by both its temporal dimension similar to audition and its s...
-
Recently, tactile sensing has attracted great interest in robotics, especially for facilitating exploration of unstructured environments and effective manipulation. A detailed understanding of the surface textures via tactile sensing is essential for many of these tasks. Previous works on texture recognition using camera based tactile sensors have been limited to treating all regions in one tactil...
-
Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either only have a flat sensing surface or a compliant tip with a limited sensing area. In this paper, we propose a novel optical tactile sensor, the GelTip, that is shaped as a finger and can sense contacts on any location of its...
-
Grasping can be conceptualized as the ability of an end-effector to temporarily attach or fixture an object to a manipulator-constraining all motion of the workpiece with respect to the end-effector's base frame. This seemingly simplistic action often requires excessive sensing, computation, or control to achieve with multi-fingered hands, which can be mitigated with underactuated mechanisms. In t...
-
Manipulation in cluttered environments like homes requires stable grasps, precise placement and robustness against external contact. Towards addressing these challenges, we present the Soft-bubble gripper system that combines highly compliant gripping surfaces with dense-geometry visuotactile sensing and facilitates multiple kinds of tactile perception. We first present several mechanical design a...
-
Grasping and manipulating objects is an integral part of many robotic systems. Both soft and rigid grippers have been investigated for manipulating objects in a multitude of different roles. Rigid grippers can hold heavy objects and apply large amounts of force, while soft grippers can conform to the size and shape of objects as well as protect fragile objects from excess stress. However, grippers...
-
Mechanical systems are typically composed of a number of contacting surfaces that move against each other. Such surfaces are subject to friction forces. These dissipate part of the actuation energy and cause an undesired effect on the overall system functioning. Therefore, a suitable model of friction is needed to elide its action. The choice of such a model is not always straightforward, as it is...
-
Coating rollers are widely popular in structural painting, in comparison with brushes and sprayers, due to thicker paint layer, better color consistency, and effortless customizability of holder frame and naps. In this paper, we introduce a cost-effective method to employ a general purpose robot (Sawyer, Rethink Robotics) for autonomous coating. To sense the position and the shape of the target ob...
-
Multi-modal estimation systems have the advantage of increased accuracy and robustness. To achieve accurate sensor fusion with these types of systems, a reliable extrinsic calibration between each sensor pair is critical. This paper presents a novel self-calibration framework for lidar-inertial systems. The key idea of this work is to use an informative path planner to find the admissible path tha...
-
Sensor calibration is the fundamental block for a multi-sensor fusion system. This paper presents an accurate and repeatable LiDAR-IMU calibration method (termed LI-Calib), to calibrate the 6-DOF extrinsic transformation between the 3D LiDAR and the Inertial Measurement Unit (IMU). Regarding the high data capture rate for LiDAR and IMU sensors, LI-Calib adopts a continuous-time trajectory formulat...
-
While automotive radars are widely used in most assisted and autonomous driving systems, only a few works were proposed to tackle the calibration problems of automotive radars with other perception sensors. One of the key calibration challenges of automotive planar radars with other sensors is the missing elevation angle in 3D space. In this paper, extrinsic calibration is accomplished based on th...
-
In this paper, we address the issue of pedestrian tracking in crowd scenarios. People in close social relationships tend to act as a group which is a great challenge to individually discriminate and track pedestrians on a LiDAR system. In this paper, we integrally model groups of people and track them in a recursive framework based on Gaussian Mixture Model (GMM). The model is optimized by an exte...
-
Multi-human tracking in the crowded environment is a challenging problem due to occlusions, pose change, viewpoint variation and cluttered background. In this work, we propose a robust feature learning for tracking-by-detection methods based on second-order attention network that can capture higher-order relationships between salient features at the early stages of Convolutional Neural Network (CN...
-
We present a relational graph learning approach for robotic crowd navigation using model-based deep reinforcement learning that plans actions by looking into the future. Our approach reasons about the relations between all agents based on their latent features and uses a Graph Convolutional Network to encode higher-order interactions in each agent’s state representation, which is subsequently leve...
-
Being able to estimate the traversability of the area surrounding a mobile robot is a fundamental task in the design of a navigation algorithm. However, the task is often complex, since it requires evaluating distances from obstacles, type and slope of terrain, and dealing with non-obvious discontinuities in detected distances due to perspective. In this paper, we present an approach based on deep...
-
In this paper, we introduce a new large-scale sidewalk dataset called SideGuide that could potentially help impaired people. Unlike most previous datasets, which are focused on road environments, we paid attention to sidewalks, where understanding the environment could provide the potential for improved walking of humans, especially impaired people. Concretely, we interviewed impaired people and c...
-
This paper addresses the problem of learning to complete a scene's depth from sparse depth points and images of indoor scenes. Specifically, we study the case in which the sparse depth is computed from a visual-inertial simultaneous localization and mapping (VI-SLAM) system. The resulting point cloud has low density, it is noisy, and has nonuniform spatial distribution, as compared to the input fr...
-
We address the problem of depth and ego-motion estimation from image sequences. Recent advances in the domain propose to train a deep learning model for both tasks using image reconstruction in a self-supervised manner. We revise the assumptions and the limitations of the current approaches and propose two improvements to boost the performance of the depth and ego-motion estimation. We first use L...
-
Unsupervised learning of depth and ego-motion from unlabelled monocular videos has recently drawn great attention, which avoids the use of expensive ground truth in the supervised one. It achieves this by using the photometric errors between the target view and the synthesized views from its adjacent source views as the loss. Despite significant progress, the learning still suffers from occlusion ...
-
We present a benchmark for online, video-based depth estimation, a problem that is not covered by the current set of benchmarks for evaluating 3D reconstruction, which focus on offline, batch reconstruction. Online depth estimation from video captured by a moving camera is a key enabling technology for compelling applications in robotics and augmented reality. Inspired by progress in many aspects ...
-
As a critical sensor for high-level autonomous vehicles, LiDAR's limitations in adverse weather (e.g. rain, fog, snow, etc.) impede the deployment of self-driving cars in all weather conditions. In this paper, we model the performance of a popular 903nm ToF LiDAR under various fog conditions based on a LiDAR dataset collected in a well-controlled artificial fog chamber. Specifically, a two-stage d...
-
In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of ...
-
Depth from a monocular video can enable billions of devices and robots with a single camera to see the world in 3D. In this paper, we present a model for video depth estimation, which consists of a flow-to-depth layer, a camera pose refinement module, and a depth fusion network. Given optical flow and camera poses, our flow-to-depth layer generates depth proposals and their corresponding confidenc...
-
In this paper, we propose an unsupervised deep learning framework with Bayesian inference for improving the accuracy of per-pixel depth prediction from monocular RGB images. The proposed framework predicts confidence map along with depth and pose information for a given input image. The depth hypotheses from previous frames are propagated forward and fused with the depth hypothesis of the current ...
-
In this paper we apply the low-rank Tensor Train decomposition for compression and operations on 3D objects and scenes represented by volumetric distance functions. Our study shows that not only it allows for a very efficient compression of the high-resolution TSDF maps (up to three orders of magnitude of the original memory footprint at resolution of 5123), but also allows to perform TSDF-Fusion ...
-
We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based o...
-
Wireless capsule endoscopy (WCE) is a novel imaging tool that allows the noninvasive visualization of the entire gastrointestinal (GI) tract without causing discomfort to the patients. Although convolutional neural networks (CNNs) have obtained promising performance for the automatic lesion recognition, the results of the current approaches are still limited due to the small lesions and the backgr...
-
We propose a deep learning model for a robot to wipe 3D-objects. Wiping of 3D-objects requires recognizing the shapes of objects and planning the motor angle adjustments for tracing the objects. Unlike previous research, our learning model does not require pre-designed computational models of target objects. The robot is able to wipe the objects to be placed by using image, force, and arm joint in...
-
In this paper, we present a novel deep learning and direct method based monocular visual odometry system named D2VO. Our system reconstructs the dense depth map of each keyframe and tracks camera poses based on these keyframes. Combining direct method and deep learning, both tracking and mapping of the system could benefit from the geometric measurement and semantic information. For each input fra...
-
In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss...
-
Training deep neural networks at the edge on light computational devices, embedded systems and robotic platforms is nowadays very challenging. Continual learning techniques, where complex models are incrementally trained on small batches of new data, can make the learning problem tractable even for CPU-only embedded devices enabling remarkable levels of adaptiveness and autonomy. However, a number...
-
Convolutional Neural Networks (CNNs) have been successfully applied for relative camera pose estimation from labeled image-pair data, without requiring any handengineered features, camera intrinsic parameters or depth information. The trained CNN can be utilized for performing pose based visual servo control (PBVS). One of the ways to improve the quality of visual servo output is to improve the ac...
-
High-definition (HD) maps are emerging as an essential tool for autonomous driving since they provide high-precision semantic information about the physical environment. To function as a reliable source of map information, HD maps must be constantly updated with changes that occur to the state of the road. In this paper, we propose a novel framework for HD map change detection that can be used to ...
-
In this paper, we consider a problem of foothold selection for the quadrupedal robots equipped with compliant adaptive feet. Starting from a model of the foot we compute the quality of the potential footholds considering also kinematic constraints and collisions during evaluation. Since terrain assessment and constraints checking are computationally expensive we applied a Convolutional Neural Netw...
-
In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations. We find that the noise existing in Radar measurements is one of the mai...
-
Learning computational models for visual attention (saliency estimation) is an effort to inch machines/robots closer to human visual cognitive abilities. Data-driven efforts have dominated the landscape since the introduction of deep neural network architectures. In deep learning research, the choices in architecture design are often empirical and frequently lead to more complex models than necess...
-
This paper discusses the application of video motion capturing technology (VMocap) to a competitive team sports game. The setting introduces a specific set of constraints: large scale markerless motion capturing, big recording volume, transmitting and processing gigabytes of data, operation without interfering with players or distracting spectators and staff, etc... In this paper, we present how w...
-
In a collaborative scenario, robots working side by side with humans might rely on vision sensors to monitor the activity of the other agent. When occlusions of the human body occur, both the safety of the cooperation and the performance of the team can be penalized, since the robot could receive incorrect information about the ongoing cooperation. In this work, we propose a novel particle filter ...
-
Detecting persons using a 2D LiDAR is a challenging task due to the low information content of 2D range data. To alleviate the problem caused by the sparsity of the LiDAR points, current state-of-the-art methods fuse multiple previous scans and perform detection using the combined scans. The downside of such a backward looking fusion is that all the scans need to be aligned explicitly, and the nec...
-
Building successful collaboration between humans and robots requires efficient, effective, and natural communication. Here we study a RGB-based deep learning approach for controlling robots through gestures (e.g., "follow me"). To address the challenge of collecting high-quality annotated data from human subjects, synthetic data is considered for this domain. We contribute a dataset of gestures th...
-
To fluently collaborate with people, robots need the ability to recognize human activities accurately. Although modern robots are equipped with various sensors, robust human activity recognition (HAR) still remains a challenging task for robots due to difficulties related to multimodal data fusion. To address these challenges, in this work, we introduce a deep neural network-based multimodal HAR a...
-
Human-Robot Interaction (HRI) is a largely ad-dressed subject today. Collision avoidance is one of main strategies that allow space sharing and interaction without contact between human and robot. It is thus usual to use a 3D depth camera sensor which may involves issues related to occluded robot in camera view. While several works overcame this issue by applying infinite depth principle or increa...
-
Analysis of human daily living activities, particularly walking activity, is essential for health-care applications such as fall prevention, physical rehabilitation exercises, and gait monitoring. Studying the evolution of the gait cycle using wearable sensors is beneficial for the detection of any abnormal walking pattern. This paper proposes a novel discrete/continuous unsupervised Hidden Markov...
-
In recent years, human pose estimation has seen great improvements by the use of neural networks. However, these approaches are unsuitable for safety-critical applications such as human-robot interaction (HRI), as no guarantees are given whether a produced detection is correct or not and false detections with high confidence scores are produced on a regular basis. In this work, we propose a method...
-
We propose to leverage recent advances in reliable 2D pose estimation with Convolutional Neural Networks (CNN) to estimate the 3D pose of people from depth images in multi-person Human-Robot Interaction (HRI) scenarios. Our method is based on the observation that using the depth information to obtain 3D lifted points from 2D body landmark detections provides a rough estimate of the true 3D human p...
-
We present a simple, fast, and light-weight RNN based framework for forecasting future locations of humans in first person monocular videos. The primary motivation for this work was to design a network which could accurately predict future trajectories at a very high rate on a CPU. Typical applications of such a system would be a social robot or a visual assistance system "for all", as both cannot...
-
Robots navigating autonomously need to perceive and track the motion of objects and other agents in its surroundings. This information enables planning and executing robust and safe trajectories. To facilitate these processes, the motion should be perceived in 3D Cartesian space. However, most recent multi-object tracking (MOT) research has focused on tracking people and moving objects in 2D RGB v...
-
Accurate and reliable tracking of multiple moving objects in 3D space is an essential component of urban scene understanding. This is a challenging task because it requires the assignment of detections in the current frame to the predicted objects from the previous one. Existing filter-based approaches tend to struggle if this initial assignment is not correct, which can happen easily.We propose a...
-
Self-supervised learning for visual object tracking possesses valuable advantages compared to supervised learning, such as the non-necessity of laborious human annotations and online training. In this work, we exploit an end-to-end Siamese network in a cycle-consistent self-supervised framework for object tracking. Self-supervision can be performed by taking advantage of the cycle consistency in t...
-
3D multi-object tracking (MOT) is an essential component for many applications such as autonomous driving and assistive robotics. Recent work on 3D MOT focuses on developing accurate systems giving less attention to practical considerations such as computational cost and system complexity. In contrast, this work proposes a simple real-time 3D MOT system. Our system first obtains 3D detections from...
-
Tracking the 6D pose of objects in video sequences is important for robot manipulation. This task, however, introduces multiple challenges: (i) robot manipulation involves significant occlusions; (ii) data and annotations are troublesome and difficult to collect for 6D poses, which complicates machine learning solutions, and (iii) incremental error drift often accumulates in long term tracking to ...
-
Visual object tracking (VOT) is an essential component for many applications, such as autonomous driving or assistive robotics. However, recent works tend to develop accurate systems based on more computationally expensive feature extractors for better instance matching. In contrast, this work addresses the importance of motion prediction in VOT. We use an off-the-shelf object detector to obtain i...
-
The novelty of this study consists in a multi-modality approach to scene classification, where image and audio complement each other in a process of deep late fusion. The approach is demonstrated on a difficult classification problem, consisting of two synchronised and balanced datasets of 16,000 data objects, encompassing 4.4 hours of video of 8 environments with varying degrees of similarity. We...
-
There have been significant advances in neural networks for both 3D object detection using LiDAR and 2D object detection using video. However, it has been surprisingly difficult to train networks to effectively use both modalities in a way that demonstrates gain over single-modality networks. In this paper, we propose a novel Camera-LiDAR Object Candidates (CLOCs) fusion network. CLOCs fusion prov...
-
We present a simple, yet effective and flexible method for action recognition supporting multiple sensor modalities. Multivariate signal sequences are encoded in an image and are then classified using a recently proposed EfficientNet CNN architecture. Our focus was to find an approach that generalizes well across different sensor modalities without specific adaptions while still achieving good res...
-
A microphone array can provide a mobile robot with the capability of localizing, tracking and separating distant sound sources in 2D, i.e., estimating their relative elevation and azimuth. To combine acoustic data with visual information in real world settings, spatial correlation must be established. The approach explored in this paper consists of having two robots, each equipped with a microphon...
-
We aim to enable robots to visually localize a target person through the aid of an additional sensing modality - the target person's 3D inertial measurements. The need for such technology may arise when a robot is to meet a person in a crowd for the first time or when an autonomous vehicle must rendezvous with a rider amongst a crowd without knowing the appearance of the person in advance. A perso...
-
The ability of detecting materials plays an important role in robotic applications. The robot can incorporate the information from contactless material detection and adapt its behavior in how it grasps an object or how it walks on specific surfaces. In this, paper we apply machine learning on impedance spectra from capacitive proximity sensors for material detection. The unique spectra of certain ...
-
Humans perform grasping by breaking down the task into a series of action phases, where the transitions between the action phases are based on the comparison between the predicted tactile events and the actual tactile events. The dependency on tactile sensation in grasping allows humans to grasp objects without the need to locate the object precisely, which is a feature desirable in robot grasping...
-
Leveraging multimodal information with recursive Bayesian filters improves performance and robustness of state estimation, as recursive filters can combine different modalities according to their uncertainties. Prior work has studied how to optimally fuse different sensor modalities with analytical state estimation algorithms. However, deriving the dynamics and measurement models along with their ...
-
Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging
Material recognition can help inform robots about how to properly interact with and manipulate real-world objects. In this paper, we present a multimodal sensing technique, leveraging near-infrared spectroscopy and close-range high resolution texture imaging, that enables robots to estimate the materials of household objects. We release a dataset of high resolution texture images and spectral meas...
-
Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforem...
-
Estimating a dense and accurate depth map is the key requirement for autonomous driving and robotics. Recent advances in deep learning have allowed depth estimation in full resolution from a single image. Despite this impressive result, many deep-learning-based monocular depth estimation (MDE) algorithms have failed to keep their accuracy yielding a meter-level estimation error. In many robotics a...
-
Geometric primitives are a compact and versatile representation of the environment and the objects within. From a motion planning perspective, the geometric structure can be leveraged in order to implement potentially faster and smoother motion control algorithms than it has been possible with grid-based occupancy maps so far. In this paper, we introduce a novel perception pipeline that efficientl...
-
Real-time 3D perception of the environment is crucial for the adoption and deployment of reliable autonomous harvesting robots in agriculture. Using data collected with RGB-D cameras under farm field conditions, we present two methods for processing 3D data that reliably detect mature broccoli heads. The proposed systems are efficient and enable real-time detection on depth data of broccoli crops ...
-
Depth estimation plays a crucial role in robotic applications that require environment perception. With the introduction of convolutional neural networks, monocular depth estimation (MDE) methods have become viable alternatives to LiDAR and stereo reconstruction-based solutions. Such methods require less equipment, fewer resources and do not need additional sensor alignment requirements. However, ...
-
Efficient and robust person perception is one of the most basic skills a mobile robot must have to ensure intuitive human-machine interaction. In addition to person detection, this also includes estimating various attributes, like posture or body orientation, in order to achieve user-adaptive behavior. However, given limited computing and battery capabilities on a mobile robot, it is inefficient t...
-
The capability to detect objects is a core part of autonomous driving. Due to sensor noise and incomplete data, perfectly detecting and localizing every object is infeasible. Therefore, it is important for a detector to provide the amount of uncertainty in each prediction. Providing the autonomous system with reliable uncertainties enables the vehicle to react differently based on the level of unc...
-
Dynamic obstacle avoidance is one crucial component for compliant navigation in crowded environments. In this paper we present a system for accurate and reliable detection and tracking of dynamic objects using noisy point cloud data generated by stereo cameras. Our solution is real-time capable and specifically designed for the deployment on computationally-constrained unmanned ground vehicles. Th...
-
Object recognition in video is an important task for plenty of applications, including autonomous driving perception, surveillance tasks, wearable devices or IoT networks. Object recognition using video data is more challenging than using still images due to blur, occlusions or rare object poses. Specific video detectors with high computational cost or standard image detectors together with a fast...
-
Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks. Detectors using different modalities and with varying computational complexities offer different trade-offs. One option is to have a very lightweight model that can predict from all modalities at once for each frame. However, in some situations (e.g., in static scenes) it might be better t...
-
Mistakes/uncertainties in object detection could lead to catastrophes when deploying robots in the real world. In this paper, we measure the uncertainties of object localization to minimize this kind of risk. Uncertainties emerge upon challenging cases like occlusion. The bounding box borders of an occluded object can have multiple plausible configurations. We propose a deep multivariate mixture o...
-
Extrinsic perturbation always exists in multiple sensors. In this paper, we focus on the extrinsic uncertainty in multi-LiDAR systems for 3D object detection. We first analyze the influence of extrinsic perturbation on geometric tasks with two basic examples. To minimize the detrimental effect of extrinsic perturbation, we propagate an uncertainty prior on each point of input point clouds, and use...
-
In this work, we explore how a strategic selection of camera movements can facilitate the task of 6D multi-object pose estimation in cluttered scenarios while respecting real-world constraints such as time and distance travelled, important in robotics and augmented reality applications. In the proposed framework, multiple object hypotheses inferred by an object pose estimator are accumulated both ...
-
We propose a surface-to-surface (S2S) point registration algorithm by exploiting the Gaussian Process Implicit Surfaces for partially overlapping 3D surfaces to estimate the 6D pose transformation. Unlike traditional approaches, that separate the corresponding search and update steps in the inner loop, we formulate the point registration as a nonlinear non-constraints optimization problem which do...
-
For robots to operate robustly in the real world, they should be aware of their uncertainty. However, most methods for object pose estimation return a single point estimate of the object's pose. In this work, we propose two learned methods for estimating a distribution over an object's orientation. Our methods take into account both the inaccuracies in the pose estimation as well as the object sym...
-
In this research, we propose a method of estimating object class and orientation given multiple input images assuming the relative camera orientations are known. Input images are transformed to descriptors on 2-D manifolds defined for each class of object through a CNN, and the object class and orientation that minimize the distance between input descriptors and the descriptors associated with the...
-
Robots working in human environments must be able to perceive and act on challenging objects with articulations, such as a pile of tools. Articulated objects increase the dimensionality of the pose estimation problem, and partial observations under clutter create additional challenges. To address this problem, we present a generative-discriminative parts-based recognition and localization method f...
-
Compared with the maturity of 2D gaze tracking technology, 3D gaze tracking has gradually become a research hotspot in recent years. The head-mounted gaze tracker has shown great potential for gaze estimation in 3D space due to its appealing flexibility and portability. The general challenge for 3D gaze tracking algorithms is that calibration is necessary before the usage, and calibration targets ...
-
In this paper, we propose a novel approach to solve the 3D non-rigid registration problem from RGB images using Convolutional Neural Networks (CNNs). Our objective is to find a deformation field (typically used for transferring knowledge between instances, e.g., grasping skills) that warps a given 3D canonical model into a novel instance observed by a single-view RGB image. This is done by trainin...
-
This paper proposes a novel method to simultaneously perform relative camera pose estimation and planar reconstruction of a scene from two RGB images. We start by extracting and matching superpixel information from both images and rely on a novel multi-model RANSAC approach to estimate multiple homographies from superpixels and identify matching planes. Ambiguity issues when performing homography ...
-
Pose estimation of known objects is fundamental to tasks such as robotic grasping and manipulation. The need for reliable grasping imposes stringent accuracy requirements on pose estimation in cluttered, occluded scenes in dynamic environments. Modern methods employ large sets of training data to learn features in order to find correspondence between 3D models and observed data. However these meth...
-
Actuation means for soft robotic structures are manifold: despite actuation mechanisms such as tendon-driven manipulators or shape memory alloys, the majority of soft robotic actuators are fluidically actuated - either purely by positive or negative air pressure or by hydraulic actuation only. This paper presents the novel idea of employing hybrid fluidic - hydraulic and pneumatic - actuation for ...
-
Although soft robots are a good alternative to rigid, traditional robots due to their intrinsic compliance and environmental adaptability, there are several drawbacks that limit their impact, such as low force exertion capability and low resistance to deformation. For this reason, soft structures of variable stiffness have become a popular solution in the field to combine the benefits of both soft...
-
This paper presents a novel approach for developing robotic grippers with variable stiffness hinges for dexterous grasps. This approach for the first time uses pneumatically actuated pouch actuators to fold and unfold morphable flaps of flexure hinges thus change stiffness of the hinge. By varying the air pressure in pouch actuators, the flexure hinge morphs into a beam with various open sections ...
-
We present a novel underactued humanoid five finger soft hand, the KIT Finger-Vision Soft Hand, which is equipped with cameras in the fingertips and integrates a high performance embedded system for visual processing and control. We describe the actuation mechanism of the hand and the tendon-driven soft finger design with internally routed high-bandwidth flat-flex cables. For efficient on-board pa...
-
Physical softness has been proposed to absorb impacts when establishing contact with a robot or its workpiece, to relax control requirements and improve performance in assembly and insertion tasks. Previous work has focused on special end effector solutions for isolated tasks, such as the peg-in-hole task. However, as many robot tasks require the precision of rigid robots, and their performance wo...
-
Soft robots are capable of inherently safer interactions with their environment than rigid robots since they can mechanically deform in response to unanticipated stimuli. However, their complex mechanics can make planning and control difficult, particularly with tasks such as locomotion. In this work, we present a mobile and untethered underwater crawling soft robot, PATRICK, paired with a testbed...
-
The exploration of spatially limited terrestrial or aquatic environments requires miniature and lightweight robots. Soft-bodied robot research is paving ways for a new class of small-scale robots that can navigate a variety of environments with minimum influence on the environment itself. However, it is generally challenging to design miniature soft-bodied robots that efficiently adapt to the chan...
-
In this study, a new type of sling for assisting bedridden patients is developed using a pneumatic growing mechanism. Growing Sling focuses on minimizing the labor input of the caregivers by automating the sling insertion and retraction process while maintaining safety and comfort. Improvements over the typical growing mechanism were made by reinforcing the sling with shafts and filament tape for ...
-
Pneumatically operated soft growing robots that extend via tip eversion are well-suited for navigation in confined spaces. Adding the ability to interact with the environment using sensors and tools attached to the robot tip would greatly enhance the usefulness of these robots for exploration in the field. However, because the material at the tip of the robot body continually changes as the robot ...
-
The widespread use of fluidic actuation for soft robots creates a high demand for soft pumps and compressors. However, current off-the-shelf pumps are usually rigid, noisy, and cumbersome. As a result, it is hard to integrate most commercial pumps into soft robotic systems, which restricts the autonomy and portability of soft robots. This paper presents the novel design of a soft pump based on bel...
-
Traditional soft robots require separate sensors and actuators to precisely control their motion. A twisted-and-coiled actuator (TCA) is a new artificial muscle with both actuation and self-sensing capability that can simultaneously serve both as a sensor and an actuator allowing to control the motion of TCAs without external sensors. This paper investigates the integrated sensing and actuation fo...
-
Tendrils are common stable structures in nature and are used for sensing, actuation, and geometrical stiffness modulation. In this paper, for the first time we exploit the helical geometry of a shape memory alloy (SMA) tendril as a simple to fabricate highly dexterous robotic continuum tentacle that we called Active Tendril-Backbone Robot (ATBR). This is achieved via partial (120 deg) activation o...
-
Taking inspiration from nature, the work presented in this paper aims to develop bio-inspired claws to be used for grasping and perching in flapping-wing aerial systems. These claws can be 3D printed out of two different materials and will be capable of adapt to any shape. Also, they will be soft for avoiding undesired damages on the objects when performing manipulation. These claws will be actuat...
-
Ionic polymer-metal composites (IPMC) actuators are popular because they can be driven at a low voltage, possess excellent responsiveness, and can perform soft motions similar to that of living creatures. Conventional IPMC soft robots are manufactured by cutting and assembling IPMC sheets. However, using this conventional process to stably manufacture three-dimensional (3D)-shaped soft robots is d...
-
In this paper, we present an analytical modeling approach to address the problem of tension loss in a generic variable curvature tendon-driven continuum manipulators (TD-CM) occurring due to the tendon-sheath distributed friction force. Despite the previous approaches in the literature, our presented model and the iterative solution algorithm do not rely on a priori known curvature/shape of the TD...
-
The properties and applications of auxetics have been widely explored in the past years. Through proper utilization of auxetic structures, designs with unprecedented mechanical and structural behaviors can be produced. Taking advantage of this, we present the development of novel and low-cost 3D structures inspired by a simple auxetic unit. The core part, which we call the body in this paper, is a...
-
Tuning the stiffness of soft robots is essential in order to extend usability and control the maneuverability of soft robots. In this paper, we propose a novel mechanism that can reconfigure the stiffness of tubular structures, using pinching to induce highly directional changes in stiffness. When pinched, these tubes can be then utilized as flexure hinges to create virtual joints on demand; the o...
-
Tactile sensors are used in robot manipulation to reduce uncertainty regarding hand-object pose estimation. However, existing sensor technologies tend to be bulky and provide signals that are difficult to interpret into actionable changes. Here, we achieve wireless tactile sensing with soft and conformable magnetic stickers that can be easily placed on objects within the robot's workspace. We embe...
-
The growing consumer demand for large volumes of high quality fruit has generated an increasing need for auto-mated fruit quality control during production. Optical methods have been proved successful in a few cases, but with limitations related to the variability of fruit colors and lighting conditions during tests. Tactile sensing provides a valuable alternative, although it comes with the need ...
-
Electronic skins and tactile sensors can provide the sense of touch to robotic manipulators. These sensing modalities complement existing long range optical sensors and can provide detailed information before and after contact. However, integration with existing systems can be challenging due to size constraints, the interface geometry, and restrictions of external wiring used to interface with th...
-
This paper presents a vision-based sensing approach for a soft linear actuator, which is equipped with an internal camera. The proposed vision-based sensing pipeline predicts the three-dimensional tip position of the actuator. To train and evaluate the algorithm, predictions are compared to ground truth data from an external motion capture system. An off-the-shelf distance sensor is integrated in ...
-
Robotic manipulators can be found today in most industries, from autonomous warehouses to advanced assembly lines in factories. Most of these industrial robots are characterized by having non-flexible and highly rigid links. In dense and complex environments these manipulators require many degrees of freedom (DOFs) which complicates the mechanical structure of the manipulator, as well as the contr...
-
Lightweight robots are known to be intrinsically elastic in their joints. The established classical approaches to control such systems are mostly based on motor-side coordinates since the joints are comparatively stiff. However, that inevitably introduces errors in the coordinates that actually matter: the ones on the link side. Here we present a new joint-torque controller that uses feedback of t...
-
We used cells, which are the units that make up a living body, as building blocks to design a biomachine hybrid system and develop a tactile sensor that uses living cells as sensor receptors. We fabricated a novel cell tactile sensor with the electrodes formed using printed electronics technology. This sensor comprises elastic electrodes mounted on a soft material to acquire tactile information; s...
-
In this paper, we report a self-sensing soft tactile actuator based on Dielectric elastomer actuator (DEA) for wearable haptic interface. DEAs are one of electroactive polymer actuators, which are reported to have large area strain and fast response speed. A soft tactile actuator is constructed of a multi-layered DEA membrane layer, a passive membrane layer, and an inner circular pillar. The soft ...
-
With reduction in the scale of unmanned air vehicles, there is an increasing need for lightweight, compact, low-power sensors and alternate sensing modalities to facilitate flight control and navigation. This paper presents a novel method to fabricate a micro-scale artificial hair sensor that is capable of directional airflow sensing. The sensor consists of a high-aspect ratio hair structure attac...
-
Thin and imperceptible soft skins that can detect internal deformations as well as external forces, can go a long way to address perception and control challenges in soft robots. However, decoupling proprioceptive and exteroceptive stimuli is a challenging task. In this paper, we present a silicone-based, capacitive E-skin for exteroception and proprioception (SCEEP). This soft and stretchable sen...
-
CCD camera-based tactile sensors provide high-resolution information about the deformation of soft and elastic interfaces. However, they have poor scalibility as it is difficult to sense a large surface area without increasing the distance between the camera and the interface or using multiple processing chips. For example, using such tactile sensors for a whole robotic arm is not yet possible. In...
-
Personalized online machine learning allows a very accurate modelling of individual behavior and demands. In particular, a system that dynamically adapts during runtime can initiate a continuous collaboration with its user where both alternately adjust to each other to maximize the system's utility. However, in application scenarios based on supervised learning it is often unclear how to obtain th...
-
We present the sequential correlation network (SCN) to improve concurrent activity detection. SCN combines a recurrent neural network and a correlation model hierarchically to model the complex correlations and temporal dynamics of concurrent activities. SCN has several advantages that enable effective learning even from a small dataset for real-world deployment. Unlike the majority of approaches ...
-
Electronic Health Record (EHR) and healthcare claim data provide rich clinical information for time series analysis. In this work, we provide a different angle of solving healthcare multivariate time series classification by turning it into a computer vision problem. We propose a Convolutional Feature Engineering (CFE) methodology, that can effectively extract long sequence dependency time series ...
-
This work studies the problem of predicting the sequence of future actions for surrounding vehicles in real-world driving scenarios. To this aim, we make three main contributions. The first contribution is an automatic method to convert the trajectories recorded in real-world driving scenarios to action sequences with the help of HD maps. The method enables automatic dataset creation for this task...
-
Sensor-based activity recognition for construction vehicles is useful for evaluating the skills of the operator, measuring work efficiency, and many other use cases. Therefore, many researches have explored robust activity-recognition models. However, it remains a challenge to apply the model to many construction sites because of the imbalance of the dataset. While it is natural to employ multi-la...
-
The Everyday Activities Science and Engineering (EASE) Collaborative Research Consortium’s mission to enhance the performance of cognition-enabled robots establishes its foundation in the EASE Human Activities Data Analysis Pipeline. Through collection of diverse human activity information resources, enrichment with contextually relevant annotations, and subsequent multimodal analysis of the combi...
-
This paper describes a calibration method for RGB-D camera networks consisting of not only static overlapping, but also dynamic and non-overlapping cameras. The proposed method consists of two steps: online visual odometry-based calibration and depth image-based calibration refinement. It first estimates the transformations between overlapping cameras using fiducial tags, and bridges non-overlappi...
-
To fuse information from a 3D Light Detection and Ranging (LiDAR) sensor and a camera, the extrinsic transformation between the sensor coordinate systems needs to be known. Therefore, an extrinsic calibration must be performed, which is usually based on features extracted from sensor data. Naturally, sensor errors can affect the feature extraction process, and thus distort the calibration result. ...
-
In this paper we perform an extensive experimental evaluation of three planar target based 3D-LIDAR camera calibration algorithms, on a sensor suite consisting multiple 3D-LIDARs and cameras, assessing their robustness to random initialization and by using metrics like Mean Line Re-projection Error (MLRE) and Factory Stereo Calibration Error. We briefly describe each method and provide insights in...
-
This paper presents the development of a Kalman filter-based range estimation technique to precisely calculate the inter-node ranges of Ultra Wide Band (UWB) modules. Relative clock tracking filters running between every anchor pair tracks relative clock dynamics while estimating the time of flight as a filter state. Both inbound and outbound message timestamps are used to update the filter to mak...
-
In this paper, we propose a unified calibration method for multi-camera multi-LiDAR systems. Only using a single planar checkerboard, the captured checkerboard frames by each sensor are classified as either global frames if they are observed by at least two sensors, or a local frame if observed by a single camera. Both global and local frames of each camera are used to estimate its intrinsic param...
-
A practical robotic bin-picking system requires a high grasp success rate for various objects. Also, the system must be capable of coping with various constraints and their changes flexibly. To resolve these issues, this study proposes a novel deep learning-based method that exploits a simulator to generate desired grasping actions. The features of this method are as follows: (1) Grasping conditio...
-
In this paper, we propose a novel sim-to-real framework to solve bolting tasks with tight tolerance and complex contact geometry which are hard to be modeled. The sim-to-real has desirable features in terms of cost and safety, however, that of the assembly task is rare due to the lack of simulator, which can robustly render multi-contact assembly. We implement the sim-to-real transfer of nut tight...
-
Current trends in industrial automation favor agile systems that allow adaptation to rapidly changing task requirements and facilitate customized production in smaller batches. This work presents a flexible manufacturing system relying on compliance control, CAD based localization, and a multi-modal gripper to enable fast and efficient task programming for assembly operations. CAD file processing ...
-
Enabling robots to quickly learn manipulation skills is an important, yet challenging problem. Such manipulation skills should be flexible, e.g., be able adapt to the current workspace configuration. Furthermore, to accomplish complex manipulation tasks, robots should be able to sequence several skills and adapt them to changing situations. In this work, we propose a rapid robot skill-sequencing a...
-
Recent progress in deep reinforcement learning has enabled agents to autonomously learn complex control strategies from scratch. Model-free approaches like Deep Deterministic Policy Gradients (DDPG) seem promising for applications with intricate dynamics, such as contact-rich manipulation tasks. However, these methods typically require large amounts of training data or meticulous hyperparameter tu...
-
Multi-functional cells with cooperating teams of robots promise to be flexible, robust, and efficient and, thus, are a key to future factories. However, their programming is tedious and AI-based planning for multiple robots is computationally expensive. In this work, we present a modular and efficient two-layer planning approach for multi-robot assembly. The goal is to generate the program for coo...
-
In this paper an experimental study of set-based task-priority kinematic control for a dual-arm mobile robot is developed. The control strategy for the coordination of the two manipulators and the mobile base relies on the definition of a set of elementary tasks to be properly handled depending on their functional role. In particular, the tasks have been grouped into three categories: safety, oper...
-
Cooperative manipulation offers many advantages over single-arm manipulation. However, this comes at a cost of added complexity, both in modeling and control of multi-arm systems. Much research has been focused on determining optimal load distribution strategies based on several objective functions, some of which include manipulability, energy consumption and joint torque minimization. This paper ...
-
We present a distributed algorithm to enable a group of robots to collaboratively manipulate an object to a desired configuration while avoiding obstacles. Each robot solves a local optimization problem iteratively and communicates with its local neighbors, ultimately converging to the optimal trajectory of the object over a receding horizon. The algorithm scales efficiently to large groups, with ...
-
In this paper, we propose the "multi-vision hand", in which a number of small high-speed cameras are mounted on the robot hand of a common 7 degrees-of-freedom robot. Also, we propose visual-servoing control by using a multi-vision system that combines the multi-vision hand and external fixed high-speed cameras. The target task was ball catching motion, which requires high-speed operation. In the ...
-
In this study, Magripper, a highly backdrivable gripper, is developed to achieve high-speed hitting grasping executed seamlessly from reaching. The gripper is designed to achieve both high speed and environmental adaptability. The key element is backdrivability in terms of both hardware and control. In Magripper, a magnetic gear is introduced to passively absorb shock in the moment of contact as a...
-
We present the design of an active two-phase finger for mechanically mediated dexterous manipulation. The finger enables re-orientation of a grasped object by using a pneumatic braking mechanism to transition between free-rotating and fixed (i.e., braked) phases. Our design allows controlled high-bandwidth (5 Hz) phase transitions independent of the grasping force for manipulation of a variety of ...
-
The ability to perform in-hand manipulation still remains an unsolved problem; having this capability would allow robots to perform sophisticated tasks requiring repositioning and reorienting of grasped objects. In this work, we present a novel non-anthropomorphic robot grasper with the ability to manipulate objects by means of active surfaces at the fingertips. Active surfaces are achieved by sph...
-
Robotic hands with anthropomorphism considerations are of prominent popularity in human-centered environment. Existing anthropomorphic robotic hands achieving part or most of human hand comparable dexterity have been applied as various robotic end-effectors and prosthetics. However, two deficiencies are evident that the design for a dexterous anthropomorphic hand is largely based on the intuition ...
-
The use of tactile information is one of the most important factors for achieving stable in-grasp manipulation. Especially with low-cost robotic hands that provide low-precision control, robust in-grasp manipulation is challenging. Abundant tactile information could provide the required feed-back to achieve reliable in-grasp manipulation also in such cases. In this research, soft distributed 3-axi...
-
Juggling manipulation is one of difficult manipulation to acquire since some of such manipulation is unstable and also its physical model is unknown due to the complex non-prehensile manipulation. To acquire these unstable unknown-dynamics juggling manipulation, we propose a method for designing the predictive model of manipulation with a deep neural network, and a real-time optimal control law wi...
-
In this paper, we study the problem of computing grasping forces for quasi-static manipulation of large and heavy objects, by exploiting object-environment contacts. We present a general formulation of this problem as a Second-Order Cone Program (SOCP) that considers (i) contact friction constraints at the object-manipulator contacts and object-environment contacts, (ii) force/moment equilibrium c...
-
Synergy provides a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform in-hand manipulation, e.g., tasks that involve simultaneously grasping and manipulating an object. Locking the position of particular fingers of a multi-fingered hand is essential for in-hand manipulation tasks eith...
-
One of the key advantages of robots is the high speeds at which they can operate. In industrial settings, increased velocities can lead to higher throughputs and improved efficiency. Some manipulation tasks might require the robot to perform highly dynamic operations such as shaking, or swinging while grasping an object. These fast movements may produce high accelerations and thus give rise to ine...
-
Stable grip of wet, deformable objects is a challenging task for robotic grasping and manipulation, especially for food products' handling. The wet, slippery interfaces between the object and robotic fingers may require larger gripping force, resulting in higher risk of damaging the grasped object. This research aims to evaluate the role of micro-patterned soft pad on enhancement of wet adhesion i...
-
A method to identify the kinematics model of a human hand that less suffers from the skin artifact is proposed based on a fact that the movements of nails with respect to the corresponding fingertip bones are much smaller than that of skin. It consists of two stages. In the first (individual) stage, the most likely combination of joint assignments and angles of each finger is identified through a ...
-
Despite more than three decades of grasping research, many tools in our everyday life still pose a serious challenge for a robotic hand to grip. The level of dexterity for such a maneuver is surprisingly "high" that its execution may require a combination of closed loop controls and finger gaits. This paper studies the task of an anthropomorphic hand driven by a robotic arm to pick up and firmly h...
-
Localizing and tracking the pose of robotic grippers are necessary skills for manipulation tasks. However, the manipulators with imprecise kinematic models (e.g. low-cost arms) or manipulators with unknown world coordinates (e.g. poor camera-arm calibration) cannot locate the gripper with respect to the world. In these circumstances, we can leverage tactile feedback between the gripper and the env...
-
Novel robotic grippers have captured increasing interests recently because of their abilities to adapt to varieties of circumstances and their powerful functionalities. Differing from traditional gripper with mechanical components-made fingers, novel robotic grippers are typically made of novel structures and materials, using a novel manufacturing process. In this paper, a novel robotic gripper wi...
-
Robots should be able to learn and perform a manipulation task across different settings. This paper presents an approach that learns an RNN-based manipulation skill model from demonstrations and then generalizes the learned skill in new settings. The manipulation skill model learned from demonstrations in an initial set of setting performs well in those settings and similar ones. However, the mod...
-
Previous work has shown that interacting with a human adversary can significantly improve the efficiency of the learning process in robot grasping. However, people are not consistent in applying adversarial forces; instead they may alternate between acting antagonistically with the robot or helping the robot achieve its tasks. We propose a physical framework for robot learning in a mixed adversari...
-
Although picking up objects a few centimeters in size is a common task, achieving such ability in a robot manipulator remains challenging. We take a step toward solving this problem by focusing on the task of picking a 1.0-cm screw from a bulk bin using only tactile information to achieve the task. Inspired by how humans pick up small objects from a bin, we propose a "grasp-separate" strategy for ...
-
Deep Gated Multi-modal Learning: In-hand Object Pose Changes Estimation using Tactile and Image Data
For in-hand manipulation, estimation of the object pose inside the hand is one of the important functions to manipulate objects to the target pose. Since in-hand manipulation tends to cause occlusions by the hand or the object itself, image information only is not sufficient for in-hand object pose estimation. Multiple modalities can be used in this case, the advantage is that other modalities can...
-
Category-based methods for task-specified grasp planning have recently been proposed in the literature. Such methods, however, are normally time consuming in both training and grasp determination process and lack capabilities to improve grasping skills due to the fixed training data set. This paper presents an improved approach for knowledge-based grasp planning by developing a multi-layer network...
-
This paper deals with the problem of planning grasp strategies on constrained and cluttered scenarios. The planner sequences the objects for grasping by considering multiple factors: (i) possible environmental constraints that can be exploited to grasp an object, (ii) object neighborhood, (iii) capability of the arm, and (iv) confidence score of the vision algorithm. To successfully exploit the en...
-
Reasoning about object grasp affordances allows an autonomous agent to estimate the most suitable grasp to execute a task. While current approaches for estimating grasp affordances are effective, their prediction is driven by hypotheses on visual features rather than an indicator of a proposal's suitability for an affordance task. Consequently, these works cannot guarantee any level of performance...
-
A redundant manipulator has multiple inverse kinematics solutions per end-effector pose. Accordingly, there can be many trajectories for joints that follow a given end-effector path in the Cartesian space. In this paper, we present a trajectory optimization of a redundant manipulator (TORM) to synthesize a trajectory that follows a given end-effector path accurately, while achieving smoothness and...
-
The transition from free motion to contact is a challenging problem in robotics, in part due to its hybrid nature. Additionally, disregarding the effects of impacts at the motion planning level often results in intractable impulsive contact forces. In this paper, we introduce an impact-aware multi-mode trajectory optimization (TO) method that combines hybrid dynamics and hybrid control in a cohere...
-
In this work, we address a planar non-prehensile sorting task. Here, a robot needs to push many densely packed objects belonging to different classes into a configuration where these classes are clearly separated from each other. To achieve this, we propose to employ Monte Carlo tree search equipped with a task-specific heuristic function. We evaluate the algorithm on various simulated and real-wo...
-
Planners using accurate models can be effective for accomplishing manipulation tasks in the real world, but are typically highly specialized and require significant fine-tuning to be reliable. Meanwhile, learning is useful for adaptation, but can require a substantial amount of data collection. In this paper, we propose a method that improves the efficiency of sub-optimal planners with approximate...
-
In this paper, we approach the challenging problem of motion planning for knot tying. We propose a hierarchical approach in which the top layer produces a topological plan and the bottom layer translates this plan into continuous robot motion. The top layer decomposes a knotting task into sequences of abstract topological actions based on knot theory. The bottom layer translates each of these abst...
-
In this paper, we propose objective functions to estimate the principal contact between a unknown manipulated target object and its unknown surroundings from the motion of the object. We derived the objective functions based on the fact that contact involves a pair of geometrical primitives (a point of vertex, a line of edge, and a plane of face) for the singular condition of the calculation for t...
-
Performing various in-hand manipulation tasks, without learning each individual task, would enable robots to act more versatile, while reducing the effort for training. However, in general it is difficult to achieve stable in-hand manipulation, because the contact state between the fingertips becomes difficult to model, especially for a robot hand with anthropomorphically shaped fingertips. Rich t...
-
Robot motion is controlled in the joint space whereas the robots have to perform tasks in their task space. Many tasks like carrying a glass of liquid, pouring liquid, opening a drawer requires constraints on the end-effector during the motion. The forward and inverse kinematic mappings between joint space and task space are highly nonlinear and multi-valued (for IK). Consequently, modeling task s...
-
Robots and intelligent industrial systems that focus on sorting or inspection of products require end-effectors that can grasp and manipulate the objects surrounding them. The capability of such systems largely depends on their ability to efficiently identify the objects and estimate the forces exerted on them. This paper presents an underactuated, compliant, and lightweight hyper-adaptive robot g...
-
Collision detection is critical for safe robot operation in the presence of humans. Acoustic information originating from collisions between robots and objects provides opportunities for fast collision detection and localization; however, audio information from microphones on robot manipulators needs to be robustly differentiated from motors and external noise sources. In this paper, we present Pa...
-
Estimating the inertial properties of an object can make robotic manipulations more efficient, especially in extreme environments. This paper presents a novel method of estimating the 2D inertial parameters of an object, by having a robot applying a push on it. We draw inspiration from previous analyses on quasi-static pushing mechanics, and introduce a data-driven model that can accurately repres...
-
Control and localization of deformable linear objects (DLOs) require models to handle their deformation. This paper proposes an approach to automatically generate a model from available visual sensor information. Based on point cloud data obtained from a 3D stereo camera, the kinematics of a multibody model formulation are derived. The approach aims to balance the tradeoff between computational co...
-
Cloth detection and manipulation is a common task in domestic and industrial settings, yet such tasks remain a challenge for robots due to cloth deformability. Furthermore, in many cloth-related tasks like laundry folding and bed making, it is crucial to manipulate specific regions like edges and corners, as opposed to folds. In this work, we focus on the problem of segmenting and grasping these k...
-
Dexterous manipulation of objects in virtual environments with our bare hands, by using only a depth sensor and a state-of-the-art 3D hand pose estimator (HPE), is challenging. While virtual environments are ruled by physics, e.g. object weights and surface frictions, the absence of force feedback makes the task challenging, as even slight inaccuracies on finger tips or contact points from HPE may...
-
In real world applications, robotic solutions remain impractical due to the challenges that arise in unknown and unstructured environments. To perform complex manipulation tasks in complex and cluttered situations, robots need to be able to identify the interaction possibilities with the scene, i.e. the affordances of the objects encountered. In unstructured environments with noisy perception, ins...
-
For applications in e-commerce, warehouses, healthcare, and home service, robots are often required to search through heaps of objects to grasp a specific target object. For mechanical search, we introduce X-Ray, an algorithm based on learned occupancy distributions. We train a neural network using a synthetic dataset of RGBD heap images labeled for a set of standard bounding box targets with vary...
-
Significant progress has been made with artistic robots. However, existing robots fail to produce high-quality portraits in a short time. In this work, we present a drawing robot, which can automatically transfer a facial picture to a vivid portrait, and then draw it on paper within two minutes averagely. At the heart of our system is a novel portrait synthesis algorithm based on deep learning. In...
-
Data-efficient domain adaptation with only a few labelled data is desired for many robotic applications, e.g., in grasping detection, the inference skill learned from a grasping dataset is not universal enough to directly apply on various other daily/industrial applications. This paper presents an approach enabling the easy domain adaptation through a novel grasping detection network with confiden...
-
To improve the accuracy of the grasping detection, this paper proposes a novel detector with batch normalization masked evaluation model. It is designed with a two-layer sparse autoencoder, and a Batch Normalization based mask is incorporated into the second layer of the model to effectively reduce the features with weak correlation. The extracted features from such model are more distinctive, whi...
-
In this paper, we propose a novel contextual bandit algorithm that employs a neural network as a reward estimator and utilizes Shannon entropy regularization to encourage exploration, which is called Shannon entropy regularized neural contextual bandits (SERN). In many learning-based algorithms for robotic grasping, the lack of the real-world data hampers the generalization performance of a model ...
-
In this paper, we present a modular robotic system to tackle the problem of generating and performing antipodal robotic grasps for unknown objects from the n-channel image of the scene. We propose a novel Generative Residual Convolutional Neural Network (GR-ConvNet) model that can generate robust antipodal grasps from n-channel input at real-time speeds (~20ms). We evaluate the proposed model arch...
-
Sequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull ...
-
A successful robot-assisted feeding system requires bite acquisition of a wide variety of food items. It must adapt to changing user food preferences under uncertain visual and physical environments. Different food items in different environmental conditions require different manipulation strategies for successful bite acquisition. Therefore, a key challenge is how to handle previously unseen food...
-
Visual robot perception has been challenging to successful robot manipulation in noisy, cluttered and dynamic environments. While some perception systems fail to provide an adequate semantics of the scene, others fail to present appropriate learning models and training data. Another major issue encountered in some robot perception systems is their inability to promptly respond to robot control pro...
-
Despite recent progress in robot learning, it still remains a challenge to program a robot to deal with open-ended object manipulation tasks. One approach that was recently used to autonomously generate a repertoire of diverse skills is a novelty based Quality-Diversity (QD) algorithm. However, as most evolutionary algorithms, QD suffers from sample- inefficiency and, thus, it is challenging to ap...
-
In this paper, we introduce a novel learning-based approach for grasping known rigid objects in highly cluttered scenes and precisely placing them based on depth images. Our Placement Quality Network (PQ-Net) estimates the object pose and the quality for each automatically generated grasp pose for multiple objects simultaneously at 92 fps in a single forward pass of a neural network. All grasping ...
-
There is a high cost associated to the time and expertise required to program complex robot applications with high variability. This is one of the main barriers that inhibit the entry of robotic automation in small and medium-sized enterprises. To tackle the high level of task uncertainty associated with changing conditions of the environment, we propose a framework that leverages a combination be...
-
Container loading by a picking robot is an important challenge in the logistics industry. When designing such a robotic system, item picking and placing have been planned individually thus far. However, since the condition of picking an item affects the possible candidates for placing, it is preferable to plan picking and placing simultaneously. In this paper, we propose a deep reinforcement learn...
-
Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn comple...
-
Recent advances in deep reinforcement learning (RL) have demonstrated its potential to learn complex robotic manipulation tasks. However, RL still requires the robot to collect a large amount of real-world experience. To address this problem, recent works have proposed learning from expert demonstrations (LfD), particularly via inverse reinforcement learning (IRL), given its ability to achieve rob...
Introduction
Conference IROS2020 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 1000