DeepNLP IROS2020 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is ...
-
Anomaly detection is a key goal of autonomous surveillance systems that should be able to alert unusual observations. In this paper, we propose a holistic anomaly detection system using deep neural networks for surveillance of critical infrastructures (e.g., airports, harbors, warehouses) using an unmanned aerial vehicle (UAV). First, we present a heuristic method for the explicit representation o...
-
ROSflight is a lean, open-source autopilot system developed with the primary goal of supporting the needs of researchers working with micro aerial vehicle systems. The project consists of firmware designed to run on low-cost, readily available flight controller boards, as well as ROS packages for interfacing between the flight controller and application code and for simulation. The core objectives...
-
Nonlinear Model Predictive Control (NMPC) is a powerful and widely used technique for nonlinear dynamic process control under constraints. In NMPC, the state and control weights of the corresponding state and control costs are commonly selected based on human-expert knowledge, which usually reflects the acceptable stability in practice. Although broadly used, this approach might not be optimal for...
-
Unmanned Aerial Vehicles (UAVs) are becoming increasingly popular in the film and entertainment industries, in part because of their maneuverability and perspectives they enable. While there exists methods for controlling the position and orientation of the drones for visibility, other artistic elements of the filming process, such as focal blur, remain unexplored in the robotics community. The la...
-
With a growing number of applications in the world for UAVs, there is a clear limitation regarding the need for extended battery life. With the current flight times, many users would benefit greatly with an innovative option of field charging these devices. The objective of this project is to investigate feasibility of inductively harvesting energy from a power line cable for applications such as ...
-
We introduce SplitFlyer-a novel quadcopter with an ability to disassemble into two self-contained bicopters through human assistance. As a subunit, the bicopter is a severely underactuated aerial vehicle equipped with only two propellers. Still, each bicopter is capable of independent flight. To achieve this, we provide an analysis of the system dynamics by relaxing the control over the yaw rotati...
-
This paper addresses the problem of cooperative transport of a point mass hoisted by two aerial robots. Treating the robots as a leader and a follower, the follower stabilizes the system with respect to the leader using only feedback from its Inertial Measurement Units (IMU). This is accomplished by neglecting the acceleration of the leader, analyzing the system through the generalized coordinates...
-
Currently, drone research and development has received significant attention worldwide. Particularly, delivery services employ drones as it is a viable method to improve delivery efficiency by using a several unmanned drones. Research has been conducted to realize complete automation of drone control for such services. However, regarding the takeoff and landing port of the drones, conventional met...
-
A high-quality estimate of wind fields can potentially improve the safety and performance of Unmanned Aerial Vehicles (UAVs) operating in dense urban areas. Computational Fluid Dynamics (CFD) simulations can help provide a wind field estimate, but their accuracy depends on the knowledge of the distribution of the inlet boundary conditions. This paper provides a real-time methodology using a Partic...
-
Mobile robots of various types have been proposed for infrastructure inspection and disaster investigation. For such mobile robot applications, accessing the areas is of primary importance for missions. Therefore, various locomotive mechanisms have been studied. We introduce a novel mobile robot system, named DIR-3, combining a crawler robot and a microdrone. By rotating its arm back and forth, DI...
-
In this paper, we propose a new concept of robot which is hybrid, including aerial and crawling subsystems and an arm, and also modular with interchangeable crawling subsystems for different pipe configurations, since it has been designed to cover most industrial oil & gas end-users' requirements. The robot has the same ability than aerial robots to reach otherwise inaccessible locations, but make...
-
Geomorphological Analysis Using Unpiloted Aircraft Systems, Structure from Motion, and Deep Learning
We present a pipeline for geomorphological analysis that uses structure from motion (SfM) and deep learning on close-range aerial imagery to estimate spatial distributions of rock traits (size, roundness, and orientation) along a tectonic fault scarp. The properties of the rocks on the fault scarp derive from the combination of initial volcanic fracturing and subsequent tectonic and geomorphic fra...
-
In the paper, a pair of auto-tuning methods for fixed-parameter controllers is presented, in application to multirotor unmanned aerial vehicles (UAVs) control. In both cases, the automatized process of searching the best altitude controller parameters is carried out with the use of a modified golden-search method, for a selected cost function, during the flight of a pair of UAVs. All the calculati...
-
Trajectory optimization problems under affine motion model and convex cost function are often solved through the convex-concave procedure (CCP), wherein the non-convex collision avoidance constraints are replaced with its affine approximation. Although mathematically rigorous, CCP has some critical limitations. First, it requires a collision-free initial guess of solution trajectory which is diffi...
-
Landing is an essential part of multicopter task operations. A multicopter has relatively stringent requirements for landing, particularly for achieving flatness. Currently, landing on rough terrain with normal skids is difficult. Therefore, research is being conducted to obtain skids capable of landing on rough terrain. In this paper, a passive skid for multicopter landing on rough terrain is pro...
-
We present a novel paradigm and algorithm for optimal design of underactuated robot platforms in highly-constrained nonconvex parameter spaces. We apply this algorithm to two variants of the mature RoboBee platform, numerically demonstrating predicted performance improvements of over 10% in some cases by algorithmically reasoning about variable effective-mechanical-advantage (EMA) transmissions, h...
-
This work presents the design, fabrication, and characterization of an airflow sensor inspired by the whiskers of animals. The body of the whisker was replaced with a fin structure in order to increase the air resistance. The fin was suspended by a micro-fabricated spring system at the bottom. A permanent magnet was attached beneath the spring, and the motion of fin was captured by a readily acces...
-
We present PufferBot, an aerial robot with an expandable structure that may expand to protect a drone's propellers when the robot is close to obstacles or collocated humans. PufferBot is made of a custom 3D-printed expandable scissor structure, which utilizes a one degree of freedom actuator with rack and pinion mechanism. We propose four designs for the expandable structure, each with unique char...
-
Rotary-wing flying machines draw attention within the UAV community for their in-place hovering capability, and recently, holonomic motion over fixed-wings. In this paper, we investigate about the power-optimality in a mono-spinner, i.e., a class of rotary-wing UAVs with one rotor only, whose main body has a streamlined shape for producing additional lift when counter-spinning the rotor. We provid...
-
The conceptual design and flight controller of a novel kind of quadcopter are presented. This design is capable of morphing the shape of the UAV during flight to achieve position and attitude control. We consider a dynamic center of gravity (CoG) which causes continuous variation in a moment of inertia (MoI) parameters of the UAV. These dynamic structural parameters play a vital role in the stabil...
-
This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation driven by external forces. We derive the dynamic model of this variable geomet...
-
Exploiting contacts with environment structures provides extra force support to a UAV, often reducing the power consumption and hence extending the mission time. This paper investigates one such way to exploit flat surfaces in the environment by a novel aerial-ground hybrid locomotion. Our design is a single passive wheel integrated at the UAV bottom, serving a minimal design to date. We present t...
-
This work presents a model-free nonlinear controller for an ornithopter prototype with bioinspired wings and tail. The size and power requirements have been thought to allocate a customized autopilot on board. To assess the functionality and performance of the full mechatronic design, a controller has been designed and implemented to execute a prescribed perching 2D trajectory. Although functional...
-
Aerial vehicles with collision resilience can operate with more confidence in environments with obstacles that are hard to detect and avoid. This paper presents the methodology used to design a collision resilient aerial vehicle with icosahedron tensegrity structure. A simplified stress analysis of the tensegrity frame under impact forces is performed to guide the selection of its components. In a...
-
This work presents the deployment of UAVs for the exploration of clouds, from the system architecture and simulation tests to a real-flight campaign and trajectory analyzes. Thanks to their small size and low altitude, light UAVs have proven to be adapted for in-situ cloud data collection. The short life time of the clouds and limited endurance of the planes require to focus on the area of maximum...
-
In micro aerial vehicle (MAV) operations, the success of a mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and the designated mission are often different in unknown, unstructured environments, and only sub-optimality may be obtained in each aspect...
-
Coverage path planning (CPP) is the task of designing a trajectory that enables a mobile agent to travel over every point of an area of interest. We propose a new method to control an unmanned aerial vehicle (UAV) carrying a camera on a CPP mission with random start positions and multiple options for landing positions in an environment containing no-fly zones. While numerous approaches have been p...
-
This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasi...
-
For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, its limitations on flight altitude prevent the UAVs from observing the global map. Thus motion planning in the local map becomes a Partially Observa...
-
We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicl...
-
We propose an algorithm for generating minimum-snap trajectories for quadrotors with linear computational complexity with respect to the number of segments in the spline trajectory. Our algorithm is numerically stable for large numbers of segments and is able to generate trajectories of more than 500, 000 segments. The computational speed and numerical stability of our algorithm makes it suitable ...
-
Persistent surveillance with aerial vehicles (drones) subject to connectivity and power constraints is a relatively uncharted domain of research. To reduce the complexity of multi-drone motion planning, most state-of-the-art solutions ignore network connectivity and assume unlimited battery power. Motivated by this and advances in optimization and constraint satisfaction techniques, we introduce a...
-
This paper proposes a planning algorithm for autonomous media production with multiple Unmanned Aerial Vehicles (UAVs) in outdoor events. Given filming tasks specified by a media Director, we formulate an optimization problem to maximize the filming time considering battery constraints. As we conjecture that the problem is NP-hard, we consider a discretization version, and propose a graph-based al...
-
In this work, we present a system architecture to enable autonomous navigation of multiple agents across user-selected global interest points in a partially unknown environment. The system is composed of a server and a team of agents, here small aircrafts. Leveraging this architecture, computation-ally demanding tasks, such as global dense mapping and global path planning can be outsourced to a po...
-
Cooperative robotics is a trending topic nowadays as it makes possible a number of tasks that cannot be performed by individual robots, such as heavy payload transportation and agile manipulation. In this work, we address the problem of cooperative transportation by heterogeneous, manipulator- endowed robots. Specifically, we consider a generic number of robotic agents simultaneously grasping an o...
-
Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt...
-
There has been a considerable amount of recent work on high-speed micro-aerial vehicle flight in unknown and unstructured environments. Generally these approaches either use active sensing or fly slowly enough to ensure a safe braking distance with the relatively short sensing range of passive sensors. The former generally requires carrying large and heavy LIDARs and the latter only allows flight ...
-
The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements. Previous appearances are also gradually forgotten due to the exponential decay of historical views in traditional appearance updating scheme of DCF framework, reducing the model's robustness. In this work, a novel tracker based on DCF framework is proposed to augmen...
-
Next-Best-View planning for surface reconstruction of large-scale 3D environments with multiple UAVs
In this paper, we propose a novel cluster-based Next-Best-View path planning algorithm to simultaneously explore and inspect large-scale unknown environments with multiple Unmanned Aerial Vehicles (UAVs). In the majority of existing informative path-planning methods, a volumetric criterion is used for the exploration of unknown areas, and the presence of surfaces is only taken into account indirec...
-
Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i...
-
Inspection for structural properties (surface stiffness and coefficient of restitution) is crucial for understanding and performing aerial manipulations in unknown environments, with little to no prior knowledge on their state. Inspection-on-the-fly is the uncanny ability of humans to infer states during manipulation, reducing the necessity to perform inspection and manipulation separately. This p...
-
Visual tracking has yielded promising applications with unmanned aerial vehicle (UAV). In literature, the advanced discriminative correlation filter (DCF) type trackers generally distinguish the foreground from the background with a learned regressor which regresses the implicit circulated samples into a fixed target label. However, the predefined and unchanged regression target results in low rob...
-
The integration of computer vision techniques for the accomplishment of autonomous interaction tasks represents a challenging research direction in the context of aerial robotics. In this paper, we consider the problem of contact-based inspection of a textured target of unknown geometry and pose. Exploiting state of the art techniques in computer graphics, tuned and improved for the task at hand, ...
-
In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computation...
-
We present a method to reconstruct the 3D trajectory of an airborne robotic system only from videos recorded with cameras that are unsynchronized, may feature rolling shutter distortion, and whose viewpoints are unknown. Our approach enables robust and accurate outside-in tracking of dynamically flying targets, with cheap and easy-to-deploy equipment. We show that, in spite of the weakly constrain...
-
This paper presents a method for target localization and tracking in clutter using Bayesian fusion of vision and Radio Frequency (RF) sensors used aboard a small Unmanned Aircraft System (sUAS). Sensor fusion is used to ensure tracking robustness and reliability in case of camera occlusion or RF signal interference. Camera data is processed using an off-the-shelf algorithm that detects possible ob...
-
Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to- end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real- world data. Simulated data, on the other hand, is easy to...
-
Disturbance estimation for Micro Aerial Vehicles (MAVs) is crucial for robustness and safety. In this paper, we use novel, bio-inspired airflow sensors to measure the airflow acting on a MAV, and we fuse this information in an Unscented Kalman filter (UKF) to simultaneously estimate the three-dimensional wind vector, the drag force, and other interaction forces (e.g. due to collisions, interaction...
-
We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate th...
-
Automated driving requires highly precise and robust vehicle state estimation for its environmental perception, motion planning and control functions. Using GPS and environmental sensors can compensate for the deficits of the estimation based on traditional vehicle dynamics sensors. However, each type of sensor has specific strengths and limitations in accuracy and robustness due to their differen...
-
We present a probabilistic framework for detailed 3-D shape estimation and tracking using only vision measurements. Vision detections are processed via a bird's eye view representation, creating accurate detections at far ranges. A probabilistic model of the vision based point cloud measurements is learned and used in the framework. A 3-D shape model is developed by fusing a set of point cloud det...
-
Achieving and maintaining line-of-sight (LOS) is challenging for underwater optical communication systems, especially when the underlying platforms are mobile. In this work, we propose and demonstrate an active alignment controlbased LED-communication system that uses the DC value of the communication signal as feedback for LOS maintenance. Utilizing the uni-modal nature of the dependence of the l...
-
Underwater communication is extremely challenging for small underwater robots which typically have stringent power and size constraints. In our previous work, we developed an artificial electrocommunication system which could be an alternative for the communication of small underwater robots. This paper further presents a new electrocommunication system that utilizes Binary Frequency Shift Keying ...
-
Underwater roll rotation is a basic but essential maneuver that allows many biological swimmers to achieve high maneuverability and complex locomotion patterns. In particular, sea mammals (e.g., sea otter) with flexible vertebra structures have a unique mechanism to efficiently achieve roll rotation, not propelled mainly by inter-digital webbing or fin, but by bending and twisting their body.In th...
-
In this paper, we present a novel algorithm for constructing a maximally informative path for a robot in an information gathering task. We use a Self-Organizing Map (SOM) framework to discover important topological features in the information function. Using these features, we identify a set of distinct classes of trajectories, each of which has improved convexity compared with the original functi...
-
The SPIR, Submersible Pylon Inspection Robot, is developed to provide an innovative and practical solution to keep workers safe during maintenance of underwater structures in shallow waters, which involves working in dangerous water currents, and high-pressure water-jet cleaning. More advanced than work-class Remotely Operated Vehicles technology, the SPIR is automated and required minimum involve...
-
This paper presents a novel autonomous surface vessel (ASV), called Roboat II for urban transportation. Roboat II is capable of accurate simultaneous localization and mapping (SLAM), receding horizon tracking control and estimation, and path planning. Roboat II is designed to maximize the internal space for transport, and can carry payloads several times of its own weight. Moreover, it is capable ...
-
Automatic latching for charging in a disturbed environment for Unmanned Surface Vehicle (USVs) is always a challenging problem. In this paper, we propose a two-stage automatic latching system for USVs charging in berth. In Stage I, a vision-guided algorithm is developed to calculate an optimal latching position for charging. In Stage II, a novel latching mechanism is designed to compensate the mov...
-
This paper presents the results of the experimental tests performed to validate the functionality of a variable pitch system (VPS), designed for pitch attitude control of the novel underwater robotic vehicle explorer UX-1. The VPS is composed of a mass suspended from a central rod mounted across the hull. This mass is rotated around the transverse axis of the vehicle in order to perform a change i...
-
In this paper we present the LoCO AUV, a Low-Cost, Open Autonomous Underwater Vehicle. LoCO is a general-purpose, single-person-deployable, vision-guided AUV, rated to a depth of 100 meters. We discuss the open and expandable design of this underwater robot, as well as the design of a simulator in Gazebo. Additionally, we explore the platform's preliminary local motion control and state estimation...
-
In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborati...
-
In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to th...
-
This paper proposes a methodology for real-time depth estimation of underwater monocular camera images, fusing measurements from a single-beam echosounder. Our system exploits the echosounder's detection cone to match its measurements with the detected feature points from a monocular SLAM system. Such measurements are integrated in a monocular SLAM system to adjust the visible map points and the s...
-
This paper presents a novel risk vector-based near miss prediction and obstacle avoidance method. The proposed method uses the sensor readings about the pose of the other obstacles to infer their motion model (velocity and heading) and, accordingly, adapt the risk assessment and take corrective actions if necessary. Relative vector calculations allow the method to perform in real-time. The algorit...
-
This work presents a practical method of obtaining a dynamic system model for small omnidirectional aquatic vehicles. The models produced can be used to improve vehicle localisation, aid in the design or tuning of control systems and facilitate the development of simulated environments. The use of a dynamic model for onboard real-time velocity prediction is of particular importance for aquatic veh...
-
Despite the recent progress, guidance, navigation, and control (GNC) are largely unsolved for agile micro autonomous underwater vehicles (μAUVs). Hereby, robust and accurate self-localization systems which fit μAUVs play a key role and their absence constitutes a severe bottleneck in micro underwater robotics research. In this work we present, first, a small-size low-cost high performance vision-b...
-
Space manipulator systems in orbit are subject to link flexibilities since they are designed to be lightweight and long reaching. Often, their joints are driven by harmonic gear-motor units, which introduce joint flexibility. Both of these types of flexibility may cause structural vibrations. To improve endpoint tracking, advanced control strategies that benefit from the knowledge of system parame...
-
In-space assembly (ISA) is the next step to building larger and more permanent structures in orbit. The use of a robotic in-space assembler saves on costly and potentially risky EVAs. Determining the best robot for ISA is difficult as it will depend on the structure being assembled. A comparison between two categories of robots is presented: a stationary robot and robot which crawls along the trus...
-
Ground-based teleoperation of robot manipulators for on-orbit servicing of spacecraft represents an example of high-payoff, high-risk operations that are challenging to perform due to high latency communications, with telemetry time delays of several seconds. In these scenarios, confidence of operating without failure is paramount. We report the development of an Interactive Planning and Supervise...
-
A novel identification method is developed which identifies the accumulated angular momentum (AAM) of spinning reaction wheels (RWs) of an uncooperative satellite captured by a robotic servicer. In contrast to other methods that treat captured satellite's RWs as non-spinning, the developed method provides simultaneously accurate estimates of the AAM of the captured satellite's RWs and of the inert...
-
This paper presents the modeling and analysis of a novel moving mechanism "tumbling" for asteroid exploration. The system actuation is provided by an internal motor and torque wheel; elastic spring-mounted spikes are attached to the perimeter of a circular-shaped robot, protruding normal to the surface and distributed uniformly. Compared with the conventional motion mechanisms, this simple layout ...
-
In this paper, we propose three novel Hardware-in-the-loop simulation (HLS) methods for a fully-actuated orbital robot in the presence of external interactions using On-Ground Facility Manipulators (OGFM). In particular, a fixed-base and a vehicle-driven manipulator are considered in the analyses. The key idea is to describe the orbital robot's dynamics using the Lagrange-Poincaré(LP) equations, w...
-
With the booming development in aerospace technology, the video satellite which observes the live phenomena on the ground by video shooting has gradually emerged as a new Earth observation method. And remote sensing comes into a "dynamic" era with the demand for new processing techniques, especially the near-real-time tracking and geo-positioning algorithm for ground moving targets. However, many ...
-
The ability to recognize previously mapped locations is an essential feature for autonomous systems. Unstructured planetary-like environments pose a major challenge to these systems due to the similarity of the terrain. As a result, the ambiguity of the visual appearance makes state-of-the-art visual place recognition approaches less effective than in urban or man-made environments. This paper pre...
-
We propose a system for visually monitoring and servoing the cutting of a multi-layer insulation (MLI) blanket that covers the envelope of satellites and spacecraft. The main contributions of this paper are: 1) to propose a model for relating visual features describing the engagement depth of the blade to the force exerted on the MLI blanket by the cutting tool, 2) a blade design and algorithm to ...
-
This paper presents a novel approach to sampling subsurface asteroidal regolith under severe time constraints. Sampling operations that must be completed within a few hours require techniques that can manage subsurface obstructions that may be encountered. The large uncertainties due to our lack of knowledge of regolith properties also make sampling difficult. To aid in managing these challenges, ...
-
Modular robotic systems with self-repair or self-replication capabilities have been presented as a robust, low cost solution to extraterrestrial or Arctic exploration. This paper explores using ice as the sole structure element to build robots. The ice allows for increased flexibility in the system design, enabling the robotic structure to be designed and built post deployment, after tasks and ter...
-
The ocean world Europa is a prime target for exploration given its potential habitability [1]. We propose a mobile robotic system that is capable of autonomously traversing tens of meters to visit multiple sites of interest on a Europan analogue surface. Due to the topology of Europan terrain being largely unknown, it is desired that this mobility system traverse a large variety of terrain types. ...
-
To allow for the construction of large space structures to support future space endeavors, autonomous robotic solutions would serve to reduce cost and risk of human extravehicular activity (EVA). Practicality of autonomous assembly requires both theoretical and algorithmic advances, and hardware experimentation across a spectrum of technological readiness levels. Analysis of hardware experiments p...
-
Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which...
-
Recent works have proved that combining spatial and temporal visual cues can significantly improve the performance of various vision-based robotic systems. However, for the ultrasonic sensors used in most robotic tasks (e.g. collision avoidance, localization and navigation), there is a lack of benchmark ultrasonic datasets that consist of spatial and temporal data to verify the usability of spatia...
-
Drivers and other road users often encounter situations (e.g., arriving at an intersection simultaneously) where priority is ambiguous or unclear but must be resolved via communication to reach agreement. This poses a challenge for autonomous vehicles, for which no direct means for expressing intent and acknowledgment has yet been established. This paper contributes a minimal model to manage ambig...
-
Automated vehicles need to not only perceive their environment, but also predict the possible future behavior of all detected traffic participants in order to safely navigate in complex scenarios and avoid critical situations, ranging from merging on highways to crossing urban intersections. Due to the availability of datasets with large numbers of recorded trajectories of traffic participants, de...
-
Current autonomous driving systems are composed of a perception system and a decision system. Both of them are divided into multiple subsystems built up with lots of human heuristics. An end-to-end approach might clean up the system and avoid huge efforts of human engineering, as well as obtain better performance with increasing data and computation resources. Compared to the decision system, the ...
-
In autonomous driving, accurately estimating the state of surrounding obstacles is critical for safe and robust path planning. However, this perception task is difficult, particularly for generic obstacles/objects, due to appearance and occlusion changes. To tackle this problem, we propose an end-to-end deep learning framework for LIDAR-based flow estimation in bird's eye view (BeV). Our method ta...
-
Distracted driving is one of the main contributors to traffic accidents. This paper proposes a deep learning approach to detecting multiple distracted driving behaviors. In order to obtain more accurate detection results, a synchronized image recognition system based on two cameras is designed, by which the body movements and face of the driver are monitored respectively. The images captured from ...
-
In human-robot interaction (HRI) systems, such as autonomous vehicles, understanding and representing human behavior are important. Human behavior is naturally rich and diverse. Cost/reward learning, as an efficient way to learn and represent human behavior, has been successfully applied in many domains. Most of traditional inverse reinforcement learning (IRL) algorithms, however, cannot adequatel...
-
Understanding the kinematics of a ground robot is essential for efficient navigation. Based on the kinematic model of a robot, its full motion capabilities can be represented by theoretical motion primitives. However, depending on the environment and/or human preferences, not all of those theoretical motion primitives are desirable and/or achievable. This work presents a method to identify effecti...
-
We present a new measure, CMetric, to classify driver behaviors using centrality functions. Our formulation combines concepts from computational graph theory and social traffic psychology to quantify and classify the behavior of human drivers. CMetric is used to compute the probability of a vehicle executing a driving style, as well as the intensity used to execute the style. Our approach is desig...
-
We propose an integrated approach to active exploration by exploiting the Cartographer method as the base SLAM module for submap creation and performing efficient frontier detection in the geometrically co-aligned submaps induced by graph optimization. We also carry out analysis on the reachability of frontiers and their clusters to ensure that the detected frontier can be reached by robot. Our me...
-
Recent advancements in statistical learning and computational abilities have enabled autonomous vehicle technology to develop at a much faster rate. While many of the architectures previously introduced are capable of operating under highly dynamic environments, many of these are constrained to smaller-scale deployments, require constant maintenance due to the associated scalability cost with high...
-
Extensive city navigation remains an unresolved problem for autonomous mobile robots that share space with pedestrians. This paper proposes a configuration for a navigation map that expresses urban structures and an autonomous navigation scheme that uses the configuration. The proposed map configuration is a hybrid structure of multiple 2D grid maps and a topological graph. The occupancy grids for...
-
Predicting the future trajectory of surrounding vehicles in a randomly varying traffic level is one of the most challenging problems in developing an autonomous vehicle. Since there is no pre-defined number of interacting vehicles participated in, the prediction network has to be scalable with respect to the number of vehicles in order to guarantee consistent performance in terms of both accuracy ...
-
Traffic simulators are important tools in autonomous driving development. While continuous progress has been made to provide developers more options for modeling various traffic participants, tuning these models to increase their behavioral diversity while maintaining quality is often very challenging. This paper introduces an easily-tunable policy generation algorithm for autonomous driving agent...
-
We present a predictive runtime monitoring technique for estimating future vehicle positions and the probability of collisions with obstacles. Vehicle dynamics model how the position and velocity change over time as a function of external inputs. They are commonly described by discrete-time stochastic models. Whereas positions and velocities can be measured, the inputs (steering and throttle) are ...
-
Autonomous vehicles need to plan at the task level to compute a sequence of symbolic actions, such as merging left and turning right, to fulfill people's service requests, where efficiency is the main concern. At the same time, the vehicles must compute continuous trajectories to perform actions at the motion level, where safety is the most important. Task-motion planning in autonomous driving fac...
-
In this work, we address the motion planning problem for autonomous vehicles through a new lattice planning approach, called Feedback Enhanced Lattice Planner (FELP). Existing lattice planners have two major limitations, namely the high dimensionality of the lattice and the lack of modeling of agent vehicle behaviors. We propose to apply the Intelligent Driver Model (IDM) [1] as a speed feedback p...
-
Ground plane estimation and ground point segmentation is a crucial precursor for many applications in robotics and intelligent vehicles like navigable space detection and occupancy grid generation, 3D object detection, point cloud matching for localization and registration for mapping. In this paper, we present GndNet, a novel end-to-end approach that estimates the ground plane elevation informati...
-
Autonomous navigation and exploration in confined spaces are currently setting new challenges for robots. The presence of narrow passages, flammable atmosphere, dust, smoke, and other hazards makes the mapping and navigation tasks extremely difficult. To tackle these challenges, robots need to make intelligent decisions, maximising information while maintaining the safety of the system and their s...
-
As electric vehicle (EV) technologies become mature, EV has been rapidly adopted in modern transportation systems, and is expected to provide future autonomous mobility-on-demand (AMoD) service with economic and societal benefits. However, EVs require frequent recharges due to their limited and unpredictable cruising ranges, and they have to be managed efficiently given the dynamic charging proces...
-
Autonomous vehicles are typically developed and trained to work under certain system and environmental conditions defined at design time and can fail or perform poorly if unforeseen conditions such as disturbances or changes in model dynamics appear at runtime. In this work, we present a fast online planning, learning, and recovery approach for safe autonomous operations under unknown runtime dist...
-
Monocular 3D object detection aims to detect objects in a 3D physical world from a single camera. However, recent approaches either rely on expensive LiDAR devices, or resort to dense pixel-wise depth estimation that causes prohibitive computational cost. In this paper, we propose an end-to-end trainable monocular 3D object detector without learning the dense depth. Specifically, the grid coordina...
-
The Crossing or Not-Crossing (C/NC) problem is important to autonomous vehicles (AVs) for safe vehicle/pedestrian interactions. However, this problem setup often ignores pedestrians walking along the direction of the vehicles' movement (LONG). To enhance the AVs' awareness of pedestrian behavior, we make the first step towards extending the C/NC to the C/NC/LONG problem and recognize them based on...
-
We propose a robust solution to future trajectory forecast, which can be practically applicable to autonomous agents in highly crowded environments. For this, three aspects are particularly addressed in this paper. First, we use composite fields to predict future locations of all road agents in a singleshot, which results in a constant time complexity, regardless of the number of agents in the sce...
-
Learning-based driving solution, a new branch for autonomous driving, is expected to simplify the modeling of driving by learning the underlying mechanisms from data. To improve the tactical decision-making for learning-based driving solution, we introduce hierarchical behavior and motion planning (HBMP) to explicitly model the behavior in learning-based solution. Due to the coupled action space o...
-
Long-tail and rare event problems become crucial when autonomous driving algorithms are applied in the real world. For the purpose of evaluating systems in challenging settings, we propose a generative framework to create safety-critical scenarios for evaluating specific task algorithms. We first represent the traffic scenarios with a series of autoregressive building blocks and generate diverse s...
-
This contribution is concerned with the topic of using simulation to understand the behavior of groups of mutually interacting autonomous vehicles (AVs) or robots engaged in traffic/maneuvers that involve coordinated operation. We outline the structure of a multi-agent simulator called SYN-CHRONO and provide results pertaining to its scalability and ability to run real-time scenarios with humans i...
-
An autonomous vehicle platoon is a network of autonomous vehicles that communicate together to move in a desired way. One of the greatest threats to the operation of an autonomous vehicle platoon is the failure of either a physical component of a vehicle or a communication link between two vehicles. This failure affects the safety and stability of the autonomous vehicle platoon. Transmissibility-b...
-
The Go-CHART is a four-wheel, skid-steer robot that resembles a 1:28 scale standard commercial sedan. It is equipped with an onboard sensor suite and both onboard and external computers that replicate many of the sensing and computation capabilities of a full-size autonomous vehicle. The Go-CHART can autonomously navigate a small-scale traffic testbed, responding to its sensor input wiwithth progr...
-
Radar sensors have become an important part of the perception sensor suite due to their long range and their ability to work in adverse weather conditions. However, several shortcomings such as large amounts of noise and extreme sparsity of the point cloud result in them not being used to their full potential. In this paper, we present a novel Recursive Least Squares (RLS) based approach to estima...
-
While Lidar beams are often represented as rays, they actually have finite beam width and this width impacts the measured shape and size of objects in the scene. Here we investigate the effects of beam width on measurements of thin objects such as vertical poles. We propose a model for beam divergence and show how this can explain both object dilation and erosion. We develop a calibration method t...
-
Autonomous driving requires the inference of actionable information such as detecting and classifying objects, and determining the drivable space. To this end, we present Multi-View LidarNet (MVLidarNet), a two-stage deep neural network for multi-class object detection and drivable space segmentation using multiple views of a single LiDAR point cloud. The first stage processes the point cloud proj...
-
Roads have well defined geometries, topologies, and traffic rules. While this has been widely exploited in motion planning methods to produce maneuvers that obey the law, little work has been devoted to utilize these priors in perception and motion forecasting methods. In this paper we propose to incorporate these structured priors as a loss function. In contrast to imposing hard constraints, this...
-
Automobiles or robots with advanced autonomous systems are equipped with multiple types of sensors to overcome different weather and geographical conditions. These sensors generally have various data delays and sampling rates. Additionally, the communication delays or time synchronization errors between the onboard computers significantly affect the robustness and accuracy of localization for auto...
-
We consider the problem of localization and navigation of Autonomous Underwater Vehicles (AUV) in the context of high performance subsea asset inspection missions in deep water. We propose a solution based on the recently introduced Unscented Kalman Filter on Manifolds (UKF-M) for onboard navigation to estimate the robot's location, attitude and velocity, using a precise round and rotating Earth n...
-
Lane mark detection is one of the key tasks for autonomous driving systems. Accurate detection of lane marks under complex urban environments remains a challenge. In this paper, an end-to-end lane mark detection network named DSSF-net, which is capable of directly outputting the accurate fitted lane curves, is proposed. First, a dual-task segmentation framework for jointing lane category predictio...
-
Autonomous vehicles often rely on high-definition (HD) maps to navigate around. However, lane markings (LMs) are not necessarily static objects due to wear & tear from usage and road reconstruction & maintenance. Therefore, the wrong matching between LMs in the HD map and sensor readings may lead to erroneous localization or even cause traffic accidents. It is imperative to keep LMs up-to-date. Ho...
-
In recent years, self-supervised methods for monocular depth estimation has rapidly become an significant branch of depth estimation task, especially for autonomous driving applications. Despite the high overall precision achieved, current methods still suffer from a) imprecise object-level depth inference and b) uncertain scale factor. The former problem would cause texture copy or provide inaccu...
-
It is well known that semantic segmentation can be used as an effective intermediate representation for learning driving policies. However, the task of street scene semantic segmentation requires expensive annotations. Furthermore, segmentation algorithms are often trained irrespective of the actual driving task, using auxiliary image-space loss functions which are not guaranteed to maximize drivi...
-
This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. To tackle the first issue we exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. The maps are used in an attention mechanism that promotes segmentation confidence masks, thus focusing the network on semantic classes in the image tha...
-
Directionality in path planning is essential for efficient autonomous navigation in a number of real-world environments. In many map-based navigation scenarios, the viable path from a given point A to point B is not the same as the viable path from B to A. We present a method that automatically incorporates preferred navigation directionality into a path planning costmap. This `preference' is repr...
-
Trajectory prediction is crucial for autonomous vehicles. The planning system not only needs to know the current state of the surrounding objects but also their possible states in the future. As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest. Most of the existing works use rasterized maps to e...
-
Constrained Iterative Linear Quadratic Regulator (CILQR), a variant of ILQR, has been recently proposed for motion planning problems of autonomous vehicles to deal with constraints such as obstacle avoidance and reference tracking. However, the previous work considers either deterministic trajectories or persistent prediction for target dynamical obstacles. The other drawback is lack of generality...
-
The lane change maneuver is one of the typical maneuvers in various driving situations. Therefore the automatic lane change function is one of the key functions for autonomous vehicles. Many researches have been conducted in this field. Most existing work focused on the solutions for the static environment and assume that the surrounding vehicles are running at constant speeds. However, in reality...
-
In this paper, we present a real-time non-linear model-predictive control (NMPC) framework to perform minimum-time motion planning for autonomous racing cars. We introduce an innovative kineto-dynamical vehicle model, able to accurately predict non-linear longitudinal and lateral vehicle dynamics. The main parameters of this vehicle model can be tuned with only experimental or simulated maneuvers,...
-
In this paper we propose a hierarchical controller for autonomous racing where the same vehicle model is used in a two level optimization framework for motion planning. The high-level controller computes a trajectory that minimizes the lap time, and the low-level nonlinear model predictive path following controller tracks the computed trajectory online. Following a computed optimal trajectory avoi...
-
This paper studies a motion planning problem over a roadmap in which a vehicle aims to travel from a start to a destination in presence of an attacker who can launch a cyber-attack on the vehicle over any one edge of the roadmap. The vehicle (defender) has the capability to switch on/off a countermeasure that can detect and permanently disable the attack if it occurs concurrently. We first model t...
-
We investigate the autonomous navigation of a mobile robot in the presence of other moving vehicles under time-varying uncertain environmental disturbances. We first predict the future state distributions of other vehicles to account for their uncertain behaviors affected by the time-varying disturbances. We then construct a dynamic-obstacle-aware reachable space that contains states with high pro...
-
Plant phenotyping, i.e., the task of measuring plant traits to describe the anatomy and physiology of plants, is a central task in crop science and plant breeding. Standard methods often require intrusive or time-consuming operations involving a lot of manual labor. Cameras or range sensors, paired with 3D reconstructions methods, can support phenotyping but the task yields several challenges in p...
-
Global navigation satellite system (GNSS) has been considered as a panacea for positioning and tracking since the last decade. However, it suffers from severe limitations in terms of accuracy, particularly in highly cluttered and indoor environments. Though real-time kinematics (RTK) supported GNSS promises extremely accurate localisation, employing such services are expensive, fail in occluded en...
-
Continuous representations of objects have always been used in robotics in the form of geometric primitives and surface models. Recently, learning techniques have emerged which allow more complex continuous representations to be learned from data, but these learning techniques require training data in the form of watertight meshes which restricts their application as meshes of this form are diffic...
-
In this paper we consider the stochastic cost orienteering problem, i.e., a version of the classic orienteering problem where the cost associated with each edge is a random variable with known distribution. Such a model is relevant when travel costs are variable, e.g., when a robot moves in uncertain terrain conditions. We model this problem using a composite state space tracking both how much pro...
-
Precision livestock farming uses artificial intelligence to individually monitor livestock activity and health. Tracking individuals over time can reveal health indicators that correlate with productivity and longevity. For instance, locomotion patterns observed in lame pigs have been shown to correlate with poor animal welfare and productivity. Kinematic analysis of pigs using pose estimates prov...
-
In this paper, our motivating application lies in precision agriculture where accurate modeling of forage is essential for informing rotational grazing strategies. Unfortunately, a major difficulty arises in modeling forage processes as they evolve on large scales according to complex ecological influences. As robots can collect data over large scales in a forage environment, they act as a promisi...
-
Intuitive human robot interfaces like speech or gesture recognition are essential for gaining acceptance for robots in daily life. However, such interaction requires that the robot detects the human's intention to interact, tracks his position and keeps its sensor systems in an optimal configuration. Audio is a suitable modality for such task as it allows for detecting a speaker in arbitrary posit...
-
Towards long range and high endurance sailing, energy is of utmost importance. Moreover, benefiting from the dominance of the sailboat itself, it is energy-saving and environment-friendly. Thus, the sailboat with energy planning problem is meaningful. However, until now, the sailboat energy optimization problem has rarely been considered. In this paper, we focus on the energy consumption optimizat...
-
Tracking and locating radio-tagged wildlife is a labor-intensive and time-consuming task necessary in wildlife conservation. In this article, we focus on the problem of achieving embedded autonomy for a resource-limited aerial robot for the task capable of avoiding undesirable disturbances to wildlife. We employ a lightweight sensor system capable of simultaneous (noisy) measurements of radio sign...
-
The isoline tracking of this work is concerned with the control design for a sensing robot to track a given isoline of an unknown 2-D scalar filed. To this end, we propose a coordinate-free controller with a simple PI-like form using only the concentration feedback for a Dubins robot, which is particularly useful in GPS-denied environments. The key idea lies in the novel design of a sliding surfac...
-
Quadrotor flight in cluttered, unknown environments is challenging due to the limited range of perception sensors, challenging obstacles, and limited onboard computation. In this work, we directly address these challenges by proposing an efficient, reactive planning approach. We introduce the Bitwise Trajectory Elimination (BiTE) algorithm for efficiently filtering out in-collision trajectories fr...
-
This paper serves as one of the first efforts to enable large-scale and long-duration autonomy using the Boston Dynamics Spot robot. Motivated by exploring extreme environments, particularly those involved in the DARPA Subterranean Challenge, this paper pushes the boundaries of the state-of-practice in enabling legged robotic systems to accomplish real-world complex missions in relevant scenarios....
-
In this paper, we study a wheeled robot with a prismatic extension joint. This allows the robot to build up momentum to perform jumps over obstacles and to swing up to the upright position after the loss of balance. We propose a template model for the class of such two-wheeled jumping robots. This model can be considered as the simplest wheeled-legged system. We provide an analytical derivation of...
-
The holonomic and omnidirectional capabilities imparted to the mobile base platform depends mainly on two factors, i.e., the wheel design and its various arrangements in the platform chassis. This paper reports on the development of a novel omnidirectional spherical sectioned wheel named Ospheel. It is modular, and the spherical sectioned geometry of the wheel is driven using two actuators placed ...
-
In this work we present a Four-Wheeled Independent Drive and Steering (4WIDS) robot named AGRO and a method of controlling its orientation while airborne using wheel reaction torques. This is the first documented use of independently steerable wheels to both drive on the ground and achieve aerial attitude control when thrown. Inspired by a cat's self-righting reflex, this capability was developed ...
-
Autonomous navigation of steel bridge inspection robots are essential for proper maintenance. Majority of existing robotic solutions for bridge inspection require human intervention to assist in the control and navigation. In this paper, a control system framework has been proposed for a previously designed ARA robot [1], which facilitates autonomous real-time navigation and minimizes human involv...
-
A mobile robot that can achieve a stable attitude and locomotion on steep slopes is needed to overcome the problems of slipping and falling for automation of works on steep slopes. The conventional approaches to achieve a stable attitude and locomotion have been researched by adopting tracked wheels and multi-legged mechanisms instead of wheel mechanisms. However, these robots have limitations in ...
-
Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cases, the robot needs to safely push unripe fruits t...
-
Robotic packaging of fresh leafy produce such as herbs and salads generally involves picking out a target mass from a pile or crate of plant material. Typically, for low-complexity parallel grippers, the weight picked can be controlled by varying the opening aperture. However, often individual strands of plant material get entangled with each other, causing more to be picked out than desired. This...
-
Many robotic coverage applications involve detection of spatially distributed targets, followed by path planning to visit them for service. In these applications, the performance of the detection algorithm can have profound effect on planning decisions and costs. Range of operation, in both space and time, for robots is typically finite over a single mission and is a common constraint that needs t...
-
Crops are an important source of food and other products. In conventional farming, tractors apply large amounts of agrochemicals uniformly across fields for weed control and plant protection. Autonomous farming robots have the potential to provide environment-friendly weed control on a per plant basis. A system that reliably distinguishes crops, weeds, and soil under varying environment conditions...
-
We present techniques to measure crop heights using a 3D Light Detection and Ranging (LiDAR) sensor mounted on an Unmanned Aerial Vehicle (UAV). Knowing the height of plants is crucial to monitor their overall health and growth cycles, especially for high-throughput plant phenotyping. We present a methodology for extracting plant heights from 3D LiDAR point clouds, specifically focusing on plot-ba...
-
Creative design of grippers on their configurations, mechatronics control system, and multi-component collaborative algorithms is often utilized to realize complex operations in industrial applications, due to the environmental constraints or specific task requirements. Firstly, this paper introduces the background problems. As the main automatic equipment -- the shield machine -- in the field of ...
-
We propose a novel excavation (i.e., digging) trajectory planning framework for industrial autonomous robotic excavators, which emulates the strategies of human expert operators to optimize the excavation of (complex/unmodellable) soils while also upholding robustness and safety in practice. First, we encode the trajectory with dynamic movement primitives (DMP), which is known to robustly preserve...
-
Backhoe loads sediment onto the bed of dump trucks during earthmoving work. The prediction of backhoe loading time is essential for ensuring safe cooperation between the backhoe and dump trucks. However, it is difficult to predict the instant at which the backhoe is ready to load sediment, because of the similarity in motions observed during gathering sediment. Moreover, since operators have diffe...
-
In this paper, we propose a dynamic state estimation framework for lateral control of a heavy tractor-trailers system using only mass-produced low-cost sensors. This issue is challenging since the lateral velocity of the lead tractor is difficult to measure directly. The performance of existing dynamic model-based estimation methods will also be degraded, as different trailers and payloads cause t...
-
3D Point cloud registration is still a very challenging topic due to the difficulty in finding the rigid transformation between two point clouds with partial correspondences, and it's even harder in the absence of any initial estimation information. In this paper, we present an end-to-end deep-learning based approach to resolve the point cloud registration problem. Firstly, the revised LPD-Net is ...
-
This article presents a planning and control pipeline for legged-wheeled (hybrid) machines. It consists of a Trajectory Optimization based planner that computes references for end-effectors and joints. The references are tracked using a whole-body controller based on a hierarchical optimization approach. Our controller is capable of performing terrain adaptive whole-body control. Furthermore, it c...
-
In this article we present a data-driven approach for automated arm control of a hydraulic excavator. Except for the link lengths of the excavator, our method does not require machine-specific knowledge nor gain tuning. Using data collected during operation of the excavator, we train a general purpose model to effectively represent the highly non-linear dynamics of the hydraulic actuation and join...
-
Automation in construction continues to be a topic of interest for many in industry and academia. However, the dynamic environments presented in construction sites prove these tasks to be difficult to automate reliably. This paper proposes a novel method of teleoperation for multiple heterogeneous robots within a construction environment. The system is achieved by creating a virtual reality interf...
-
The joint detection of drivable areas and road anomalies is a crucial task for ground mobile robots. In recent years, many impressive semantic segmentation networks, which can be used for pixel-level drivable area and road anomaly detection, have been developed. However, the detection accuracy still needs improvement. Therefore, we develop a novel module named the Normal Inference Module (NIM), wh...
-
This paper characterizes an algorithm that estimates searcher skill level to support planning for search activities involving heterogeneous robot and human/robot teams. Specifically, we use Monte-Carlo simulations to determine the empirical accuracy of the estimator, to assess the quality of its predicted distribution (nonparametric) of agent skill levels, and the convergence rate of the estimate....
-
Developments in mobile robot navigation have enabled robots to operate in warehouses, retail stores, and on sidewalks around pedestrians. Various navigation solutions have been proposed, though few as widely adopted as ROS (Robot Operating System) Navigation. 10 years on, it is still one of the most popular navigation solutions1. Yet, ROS Navigation has failed to keep up with modern trends. We pro...
-
In this paper, we propose a novel path planning algorithm for the nonholonomic multiple mobile robot system with applications to a robotic autonomous luggage trolley collection system at airports. We consider this path planning algorithm as a Multiple Traveling Salesman Problem (MTSP). Our path planning algorithm consists of three parts. First, we use the Minimum Spanning Tree (MSP) algorithm to d...
-
Avoiding obstacles in the perceived world has been the classical approach to autonomous mobile robot navigation. However, this usually leads to unnatural and inefficient motions that significantly differ from the way humans move in tight and dynamic spaces, as we do not refrain interacting with the environment around us when necessary. Inspired by this observation, we propose a framework for auton...
-
Tissue engineering is trying to use modular tissue micro-rings to construct artificial biological microtubes as substitute of autologous tissue tubes to alleviate the shortage of donor sources. However, because of the lack of effective assembly strategies, it is still challenging to achieve high-speed fabrication of biological microtubes with high cell density. In this paper, we proposed a robotic...
-
This paper proposed a technique to introduce the membrane protein into the lab-on-chip analysis system having a planar lipid bilayer. The proposed technique utilized a dielectrophoretic(DEP) force generated by the asymmetric configuration of the solid electrodes on the aqueous buffer separator. By applying the alternating current to the separator and the counter electrode, we manipulated liposomes...
-
Under the excitation of acoustic radiation, the amount of energy absorbed and rebounded by cells have the relationship with mechanical properties, e.g. stiffness, shape, weight and so on. In this paper, a femtosecond laser-activated micro-detector is designed to convert this relationship into an electrical signal. First, the acoustic radiation is generated by a femtosecond laser pulse in a microch...
-
Sonoporation, which typically employs acoustic cavitation microbubbles, can enhance the permeability of the cell membrane, allowing foreign matter to enter cells across the natural barriers. However, the diameter nonuniformity and random distribution of microbubbles make it difficult to achieve controllable and high-efficiency sonoporation, while complex extern acoustic driving system also limits ...
-
Cell manipulation is a critical procedure in related biological applications such as embryo biopsy and intracytoplasmic sperm injection (ICSI), where the biological cell is required to be oriented to the desired position. To bridge the gap between the techniques and the clinical applications, a robotic micromanipulation method, which utilizes friction forces to rotate the cell with standard microp...
-
In this paper, we have constructed actively perfusable multiple hepatic lobule-like vascular networks in a 3D cellular structure by using magnetic tweezers. Without well-organized channel networks, cells in a large 3D tissue cannot receive nutrients and oxygen from the channel, and therefore, the cells will be dead after few days. To construct well-organized channel networks, we fabricated a hepat...
-
Magnetic microrobots can be propelled precisely and wirelessly in vivo using magnetic field for targeted drug delivery and early detection. They are promising for clinical trials since magnetic fields are capable of penetrating most materials with minimal interaction, and are nearly harmless to human beings. However, challenges like the biocompatibility, biodegradation and therapeutic effects of t...
-
In this paper, we present a magnetized cell-robot using macrophages as templates, which can be controlled under a strong gradient magnetic field, to approach and kill cancer cells in both vitro and vivo environment. Firstly, we establish a magnetic control system using only four coils which can generate gradient field up to 4.14 T/m utilizing the coupled field contributed by multiple electromagnet...
-
Magnetically-driven screws operating in soft-tissue environments could be used to deploy localized therapy or achieve minimally invasive interventions. In this work, we characterize the closed-loop behavior of magnetic screws in an agar gel tissue phantom using a permanent magnet-based robotic system with an open-configuration. Our closed-loop control strategy capitalizes on an analytical calculat...
-
The presence of any type of defect on the glass screen of smart devices has a great impact on their quality. We present a robust semi-supervised learning framework for intelligent micro-scaled localization and classification of defects on a 16K pixel image of smartphone glass. Our model features the efficient recognition and labeling of three types of defects: scratches, light leakage due to crack...
-
Functional nanomaterials possess exceptional multi-physical (e.g., mechanical, electrical and optical) properties compared with their bulk counterparts. To facilitate both synthesis and device applications of these nanomaterials, it is highly desired to characterize their multi-physical properties with high accuracy and efficiency. The nanomanipulation techniques under scanning electron microscopy...
-
Collaborative robotics allows merging the best capabilities of humans and robots to perform complex tasks. This allows the user to interact with remote and directly inaccessible environments such as the micro-scale world. This interaction is made possible by the bidirectional exchange of information (displacement - force) between the user and the environment through a haptic interface. The effecti...
-
Robust particle manipulation is a challenging but essential technique for single-cell analysis and processing of microfluidic devices. This paper proposes a micro-particle manipulation system with a microfluidic channel network. We built gravity-induced pressure actuators, which can generate high-resolution output pressure with a wide range so that the multiple particles can be delivered from the ...
-
By virtue of their ultra high resolution, scanning electron microscopes (SEMs) are essential to study topography, morphology, composition, and crystallography of materials, and thus are widely used for advanced researches in physics, chemistry, pharmacy, geology, etc. The major hindrance of using SEMs is that obtaining high quality images from SEMs requires a professional control of many control p...
-
In order to improve the convenience of operation for the medical assistive devices and reduce the use and maintenance cost, the Aruco recognition technology is applied to the navigation and positioning of visual guided electric assistive devices. Firstly, the differential control kinematic model of the electric wheelchair is analyzed. We discuss the feasibility of Aruco recognition technology in t...
-
Surgical navigation is challenging on complicated multi-branch structures such as intrarenal collecting systems or bronchi. The objective of this work is to help surgeons quickly establish the corresponding relationship between intraoperative endoscopic images and preoperative CT data. An endoscopic navigation method is proposed based on three-dimensional structure registration. It mainly includes...
-
Accurate volume segmentation from the Computed Tomography (CT) scan is a common prerequisite for pre-operative planning, intra-operative guidance and quantitative assessment of therapeutic outcomes in robot-assisted Minimally Invasive Surgery (MIS). 3D Deep Convolutional Neural Network (DCNN) is a viable solution for this task, but is memory intensive. Small isotropic patches are cropped from the ...
-
Intelligent vision is appealing in computer-assisted and robotic surgeries. Vision-based analysis with deep learning usually requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. We investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model live-cadaver GAN (LC-GAN) ...
-
This paper presents a technique to concurrently and jointly predict the future trajectories of surgical instruments and the future state(s) of surgical subtasks in robot-assisted surgeries (RAS) using multiple input sources. Such predictions are a necessary first step towards shared control and supervised autonomy of surgical subtasks. Minute-long surgical subtasks, such as suturing or ultrasound ...
-
For the time varying optimization problem, the tracking error cannot converge to zero at the finite time because of the optimal solution changing over time. This paper proposes a novel varying parameter recurrent neural network (VPRNN) based hierarchical optimization of a 7-DoF surgical manipulator for Robot-Assisted Minimally Invasive Surgery (RAMIS), which guarantees task tracking, Remote Center...
-
This paper proposes a magnetic needle steering controller to manipulate mesoscale magnetic suture needles for executing planned suturing motion. This is an initial step towards our research objective: enabling autonomous control of magnetic suture needles for suturing tasks in minimally invasive surgery. To demonstrate the feasibility of accurate motion control, we employ a cardinally-arranged fou...
-
The recent development of Robot-Assisted Minimally Invasive Surgery (RAMIS) has brought much benefit to ease the performance of complex Minimally Invasive Surgery (MIS) tasks and lead to more clinical outcomes. Compared to direct master-slave manipulation, semi-autonomous control for the surgical robot can enhance the efficiency of the operation, particularly for repetitive tasks. However, operati...
-
In magnetic resonance imaging (MRI) guided robotic catheter ablation procedures, reliable tracking of the catheter within the MRI scanner is needed to safely navigate the catheter. This requires accurate registration of the catheter to the scanner. This paper presents a differential, multi-slice image-based registration approach utilizing active fiducial coils. The proposed method would be used to...
-
One out of eight women will get breast cancer during their lifetime. A biopsy, a procedure in which a tissue sample is acquired from the lesion, is required to confirm the diagnosis. A biopsy is preferably executed under ultrasound (US) guidance because it is simple, fast, and cheap, gives real-time image feedback and causes little patient discomfort. However, Magnetic Resonance (MR)-detected lesi...
-
Robot-Assisted systems for Minimally Invasive Surgery enhance the surgeon capability, however, direct control over both the surgical tools and the endoscope results in an increased workload that leads to longer operation times. This work investigates the introduction of SCAN (System for Camera Autonomous Navigation) to overcome this limitation. An experimental study involving 12 participants was c...
-
One approach to control drug delivery in the cochlea is to use a magnetic microrobot powered by externally applied magnetic fields. However, it is necessary to integrate a localization system to ensure the precise navigation of the microrobot in the cochlear canal. To avoid integrating a clinical imaging modality for the navigation of microrobots in the cochlea, we propose in this work the applica...
-
As a novel therapy for peritoneal dissemination, it is desired to actualize an endoscopic photothermal therapy, which is minimally invasive and is highly therapeutically effective. However, since the endoscopic tumor temperature control has not been actualized, conventional therapies could damage healthy tissues by overhearing. In this paper, we develop a thermal endoscope system that controls the...
-
During the last two decades, minimally invasive surgery (MIS) has become popular because it offers advantages such as less pain, faster recovery, improved cosmesis, and reduced complications. Single-port laparoscopic surgery is a form of MIS where surgeons operate exclusively through a single entry. However, the view from the rigid endoscope is often obscured by the instruments which pass through ...
-
Hyper-redundant manipulators driven by cables are used in minimally invasive surgery because of their flexibility and small diameters. In particular, manipulators composed of many rigid links and joints have the advantages of high stiffness and payload. However, these manipulators have difficulty in estimating their positions and shapes using calculations based only on the kinematics model that as...
-
Automated surgical gestures classification and recognition are important precursors for achieving the goal of objective evaluation of surgical skills. Many works have been done to discover and validate metrics based on the motion of instruments that can be used as features for automatic classification of surgical gestures. In this work, we present a series of angular metrics that can be used toget...
-
Surgical knot tying is one of the most fundamental and important procedures in surgery, and a high-quality knot can significantly benefit the postoperative recovery of the patient. However, a longtime operation may easily cause fatigue to surgeons, especially during the tedious wound closure task. In this paper, we present a vision-based method to automate the suture thread grasping, which is a su...
-
There are many design trade-offs when building a magnetic manipulator to control millimeter-scale rotating magnetic swimmers for surgical applications.For example, increasing the magnitude of the flux density generated by the magnetic manipulator increases the torque applied to the swimmer, which could enable performing a wider variety of surgical tasks in the future. However, producing stronger m...
-
Circulating tumor cells (CTCs) are the primary cause of tumor metastasis after surgery. Metastatic tumor recurrence is the leading reason of cancer death. It is prerequisite to develop a platform for CTCs separation to predict the cancer cell transfer in important organs. Herein, a novel acoustic microfluidic device was designed to capture the "true" CTCs from the whole blood sample. The blood got...
-
Robot-assisted vitreoretinal surgery can filter surgeons' hand tremors and provide safe, accurate tool manipulation. In this paper, we report the design, optimization, and evaluation of a novel tilt mechanism for a new Steady-Hand Eye Robot (SHER). The new tilt mechanism features a four-bar linkage design and has a compact structure. Its kinematic configuration is optimized to minimize the require...
-
Currently, laparoscopic surgery systems are adapted for a large number of indications and patients and are therefore not optimized for one specific case. The challenge to create systems with an optimized kinematic structure for a specific patient regarding reachability and manipulability in the needed workspace is the automated design and construction process. We have developed an automated design...
-
Gastrointestinal (GI) endoscopy is a conventional and prevalent procedure used to diagnose and treat diseases in the digestive tract. This procedure requires inserting an endoscope equipped with a camera and instruments inside a patient to the target of interest. To manoeuvre the endoscope, an endoscopist would rotate the knob at the handle to change the direction of the distal tip and apply the f...
-
In this study, we developed a selective driving joint forceps (SDJF) for laparoscopic surgery. The SDJF has a mechanism that the driving joints can be selected arbitrarily, therefore each joint doesn't require an individual actuator for operating. The developed SDJF has six joints that can be operated using only four actuators. Each joint has 2-degrees-of-freedom (DOF) of flexion. Therefore, the S...
-
Many surgical robots with steerable surgical instruments have been proposed for endoscopic surgery. Surgical instruments should be small in size for insertion into the body and be able to handle large payloads such as tissue. Because the overall diameter and payload parameters are a trade-off, it is difficult to design an instrument with a large payload while reducing its diameter. In this paper, ...
-
The use of robots in minimally invasive surgery has improved the quality of standard surgical procedures. So far, only the automation of simple surgical actions has been investigated by researchers, while the execution of structured tasks requiring reasoning on the environment and the choice among multiple actions is still managed by human surgeons. In this paper, we propose a framework to impleme...
-
In this paper, an improved motion planning scheme is proposed for surgical robot control with multiple active constraints, including joint constraints, joint velocity constraints and remote center of motion constraints. It introduces an improved recurrent neural network (RNN) to optimize the online motion planning respect to multiple constraints. The demonstrated surgical operation trajectory is d...
-
In this paper we present a novel strategy for motion planning of autonomous robotic arms in Robotic Minimally Invasive Surgery (R-MIS). We consider a scenario where several laparoscopic tools must move and coordinate in a shared environment. The motion planner is based on a Model Predictive Controller (MPC) that predicts the future behavior of the robots and allows to move them avoiding collisions...
-
Force modulation of robotic manipulators has been extensively studied for several decades but is not yet commonly used in safety-critical applications due to a lack of accurate interaction contact modeling and weak performance guarantees - a large proportion of them concerning the modulation of interaction forces. This study presents a high-level framework for simultaneous trajectory optimization ...
-
Mitral regurgitation is one of the most common heart diseases caused by ventricular dysfunction or anatomic abnormality of the mitral valve. The fundamental treatment for mitral regurgitation is to repair/replace the mitral valve through open-heart surgery which is risky and requires more time to recover or through minimally invasive approaches, which have significant challenges and limitations. T...
-
Endometrial regeneration surgery is a new therapy for intrauterine adhesion (IUA). However, existing instruments lacking dexterity and compliance are with difficulty to successfully perform the tasks of generating transplant wounds and transplanting stem cells during endometrial regeneration surgery. This paper presents a novel shifted-routing continuum manipulator which is driven by only two cabl...
-
Fine needle insertions into a lung are challenging in terms of the needle deflection due to the breathing motion. Although previous related works neglected the effect for the needle deflection due to the breathing motion by patients stopping the breath during the insertion, they have to suffer from the discomfort. This paper proposes the intermittent insertion control method to decrease needle def...
-
Steerable needles have been widely researched in recent years, and they have multiple potential roles in the medical area. The flexibility and capability of avoiding obstacles allow the steerable needles to be applied in the biopsy, drug delivery and other medical applications that require a high degree of freedom and control accuracy. Radius of Curvature (ROC) of the needle while inserting in the...
-
This paper presents a smart continuum actuator based on a promising class of materials: ElectroActive polymer (EAP). Indeed these polymers undergo dimensional change in response to an applied electrical field and could be integrated directly in an endoscopic robot structure. We focuses on one of such materials, an electrostrictive polymer, for its valuable strain performances. An analytical model ...
-
This paper studies the contact stability and contact safety of a robotic intravascular cardiac catheter under blood flow disturbances while in contact with tissue surface. A probabilistic blood flow disturbance model, where the blood flow drag forces on the catheter body are approximated using a quasi-static model, is introduced. Using this blood flow disturbance model, probabilistic contact stabi...
-
Needle pose tracking is fundamental to achieve a precise and safe insertion in minimally-invasive percutaneous interventions. In this work, a method for estimating the full pose of steerable needles is presented, considering a four-segment Programmable Bevel-Tip Needle (PBN) as a case study. The method estimates also the torsion of the needle that can arise during the insertion because of the inte...
-
Haptic simulators can help medical students to train and improve their skills before practicing with a real patient. However, the vast majority of needle insertion haptic simulators are based on sophisticated models that are accurate but highly demanding in computing resources. Most of them do not provide haptic feedback and/or are not suitable for haptic control due to their computing time. In th...
-
Reinforcement Learning (RL) methods have demonstrated promising results for the automation of subtasks in surgical robotic systems. Since many trial and error attempts are required to learn the optimal control policy, RL agent training can be performed in simulation and the learned behavior can be then deployed in real environments. In this work, we introduce an open-source simulation environment ...
-
This paper presents a dynamic constraint formulation to provide protective virtual fixtures of 3D anatomical structures from polygon mesh representations. The proposed approach can anisotropically limit the tool motion of surgical robots without any assumption of the local anatomical shape close to the tool. Using a bounded search strategy and Principle Directed tree, the proposed system can run e...
-
Robot-assisted retinal surgery has become increasingly prevalent in recent years in part due to the potential for robots to help surgeons improve the safety of an immensely delicate and difficult set of tasks. The integration of robots into retinal surgery has resulted in diminished surgeon perception of tool-to-tissue interaction forces due to robot's stiffness. The tactile perception of these in...
-
A flexible endoscope introduces more dexterity to the image capturing in endoscopic surgery. However, manual control or automatic control based on instrument tracking does not handle the misorientation between the endoscopic video and the surgeon. We propose an automatic flexible endoscope control method that tracks the surgeon's head with respect to the object in the surgical scene. The robotic f...
-
This paper introduces an obstacle-crossing strategy, and the self-reconfiguration algorithm for a new class of modular robots called the rolling sphere, which can fit obstacles represented by cubes of different sizes due to the chain connection of multiple spheres. For the self-reconfiguration of the rolling spheres, a large gradient is obtained by classifying its action types and hierarchically m...
-
Modular robots are autonomous systems with variable morphology, composed of independent connected computational elements, called particles or modules. Due to critical resource constraints and limited capabilities, globally unique identifier (ID) assignment to each particle is a very challenging task in modular robots. However, having a unique ID in each one remains essential for various operations...
-
In this paper, we present a novel description for the configuration space of adaptive modular robots with a triangular structure based on extended binary trees. In general, binary trees can serve as a representation of kinematic trees with a maximum of two immediate descendants per element. Kinematic loops are incorporated in the tree structure by an ingenious extension of the binary tree indices....
-
Modular robots are defined as autonomous kinematic machines with variable morphology. They are composed of several thousands or even millions of modules which are able to coordinate in order to behave intelligently. Clustering the modules in modular robots has many benefits, including scalability, energy-efficiency, reducing communication delay and improving the self-configuration processes that f...
-
The modular assembly and actuation of 3D printed milliscale cuboid robots using a globally applied magnetic field is presented. Cuboids are composed of a rectangular resin shell embedded with two spherical permanent magnets that can independently align with any applied magnetic field. Placing cuboids within short distances of each other allows for modular assembly and disassembly by changing magne...
-
A fundamental challenge in the field of modular and collective robots is balancing the trade-off between unit-level simplicity, which allows scalability, and unit-level functionality, which allows meaningful behaviors of the collective. At the same time, a challenge in the field of soft robotics is creating untethered systems, especially at a large scale with many controlled degrees of freedom (DO...
-
Robotic self-assembly allows robots to join to form useful, on-demand structures. Unfortunately, the methods employed by most self-assembling robotic swarms compromise this promise of adaptability through their use of fixed docking locations, which impair a swarm's ability to handle imperfections in the structural lattice resulting from load deflection or imperfect robot manufacture; these concern...
-
We consider the problem of designing synthetic cells to achieve a complex goal (e.g., mimicking the immune system by seeking invaders) in a complex environment (e.g., the circulatory system), where they might have to change their control policy, communicate with each other, and deal with stochasticity including false positives and negatives-all with minimal capabilities and only a few bits of memo...
-
In this study, a passive arm-support exoskeleton was designed to provide assistive aid for manufacturing workers. The exoskeleton has two operating states which can be altered using an unique ratchet bar mechanism with two blocks fixed on the ratchet bar. When the upper arm is elevated to the highest poiont, the pawl module will touch the lower block to allow the pawl separated, so that the arm ca...
-
This paper presents electro-hydrostatic actuation as a valid substitute of electro-mechanical devices for powered knee prostheses. The work covers the design of a test rig exploiting linear electro-hydrostatic actuation. Typical control laws for prosthesis actuators are discussed, implemented and validated experimentally. Particularly, this work focuses on position and admittance control syntheses...
-
New challenges arise when investigating the use of active prostheses for lower limb replacement, such as high motor power requirements, leading to increased weight and reduced autonomy. Series and parallel elasticity are often explored to reduce the necessary motor power but often the effect on the energy consumption of the prosthesis is not directly investigated, as the mechanical power propertie...
-
The operational space of an 8-axis arm exoskeleton is partitioned into "tasks" based on the human arm motion, and a task priority approach is implemented to perform the inverse kinematics. The tasks are prioritized in the event that singularities or other constraints such as joint limits render the full desired operational space infeasible. The task reconstruction method is used to circumvent sing...
-
The human hand serves as an inspiration for robotic grippers. However, the dimensions of the human hand evolved under a different set of constraints and requirements than that of robots today. This paper discusses a method of kinematically optimizing the design of an anthropomorphic robotic hand. We focus on maximizing the workspace intersection of the thumb and the other fingers as well as maximi...
-
In this study, we address the inverse kinematics problem for an upper-limb exoskeleton by presenting a novel method that guarantees the satisfaction of joint-space constraints, and solves closed-chain mechanisms in a serial robot configuration. Starting from the conventional differential kinematics method based on the inversion of the Jacobian matrix, we describe and test two improved algorithms b...
-
Several powered exoskeletons have been developed and commercialized to assist people with complete spinal cord injury. For motion control of a powered exoskeleton, a normal gait pattern is often applied as a reference. However, the physical ability of paraplegics and the degrees of freedom of powered exoskeletons are totally different from those of people without disabilities. Therefore, this pape...
-
Gait Training Robot with Intermittent Force Application based on Prediction of Minimum Toe Clearance
Adaptive assistance of gait training robots has been determined to improve gait performance through motion assistance. An important control role during walking is to avoid tripping by controlling minimum toe clearance (MTC), which is an indicator of tripping risk, to avoid its decrease among gait cycles. No conventional gait training robots can adjust assistance timing based on MTC. In this paper,...
-
Optimizing lower-body exoskeleton walking gaits for user comfort requires understanding users' preferences over a high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LINECOSPAR, a human-in-the-loop preference-based fram...
-
Evidence suggests that the metabolic cost associated with the locomotive activity of walking is dependent upon ankle stiffness. This stiffness can be a control parameter in an ankle-foot prosthesis. Considering unique physical interaction between each individual with below-knee amputation and robotic ankle-foot prosthesis, individually tuned stiffness in a robotic ankle-foot prosthesis may improve...
-
This paper describes a novel low-level controller for the lower-limb exoskeleton Atalante. The controller implemented on the commercialized product Atalante works under the assumption of full rigidity, performing position control through decentralized joint PIDs. However, this controller is unable to tackle the presence of flexibilities in the system, which cause static errors and undesired oscill...
-
This paper outlines steps toward a framework for model-based user intent detection to enable fluent human-robot interaction in assistive exoskeletons. An interacting multi-model (IMM) estimation scheme is presented to address state estimation for lower-extremity exoskeletons and to handle their hybrid dynamics. The proposed IMM scheme includes new approaches that enable it to estimate states of hy...
-
To obtain synchronized gait assistance, this paper presents a new delayless adaptive dual-oscillator (ADO) scheme to address the inherent delay issue. In the ADO structure, a new oscillator is coupled with the primitive one but the phase is adaptively feed-forward compensated. It’s remarkable that the compensated phase is determined by the proposed extended phase lag observer, in which both the ph...
-
In order to fluidly perform complex tasks in collaboration with a human being, such as table handling, a humanoid robot has to recognize and adapt to human movements. To achieve such goals, a realistic model of the human locomotion that is computable on a robot is needed. In this paper, we focus on making a humanoid robot follow a human-like locomotion path. We mainly present two models of human w...
-
Despite plenty of motion planning strategies have been proposed for bipedal locomotion, enhancing the walking robustness in real-world environments is still an open question. This paper focuses on robust body and leg trajectories synthesis through integrating constrained optimization with imitation learning. Specifically, we first propose a Quadratically Constrained Quadratic Programming (QCQP) al...
-
In this paper we examine a novel method of core-located actuation that we believe can be used to vary gaits in a compass-gait walker, using critical analysis of a ball-in-tray mechanism to apply forces at the robot's "pelvis". The dynamic equations of motion of a tilting ball-tray system with several design parameters are developed and simulated for various tray designs. Results show that changes ...
-
Most state-of-the-art bipedal robots are designed to be anthropomorphic and therefore possess legs with knees. Whilst this facilitates more human-like locomotion, there are implementation issues that make walking with straight or near-straight legs difficult. Most bipedal robots have to move with a constant bend in the legs to avoid singularities at the knee joints, and to keep the centre of mass ...
-
A control algorithm that allows a human model to crawl using a pair of supernumerary robotic limbs (SuperLimbs) is presented. The human model and SuperLimbs are coupled by a compliant harness. This work is inspired by the need for wearable robotic systems that can support workers engaged in fatiguing tasks. The walking policy is developed based on Lyapunov analysis. The volume of the region of att...
-
This paper addresses reactive generation of step time and location of biped robots for balance recovery against a severe push. Key idea is to reformulate the balance recovery problem into a tracking problem for "hybrid" inverted pendulum model of the biped, where taking a new step implicitly yields a discrete jump of the tracking error. This interpretation offers a Lyapunov-based approach to react...
-
In this paper, we present a sequential motion planning and control method for generating somersaults on bipedal robots. The somersault (backflip or frontflip) is considered as a coupling between an axile hopping motion and a rotational motion about the center of mass of the robot; these are encoded by a hopping Spring-loaded Inverted Pendulum (SLIP) model and the rotation of a Flywheel, respective...
-
Compliance is important for humanoid robots, especially a position-controlled one, to perform tasks in complicated environments where unexpected or sudden contacts will result in large impacts which may cause instability or destroy the hardware of robots. This paper presents a compliance control method based on viscoelastic model for humanoid robots to survive on these conditions. The viscoelastic...
-
In this paper, we propose the stabilization strategy for a soft landing in a biped walking using impedance control and the optimization-based whole-body control framework. Even though proper contact forces and desired trajectories of the robot are given, the robot can be unstable easily if unexpected forces are applied to the robot or impulsive contact force is produced in the landing state while ...
-
Deformable objects are very common around us in our daily life. Because they have infinitely many degrees of freedom, they present a challenging problem in robotics. Inspired by practical industrial applications, we present in this paper our research on using a humanoid robot to take a long, thin and flexible belt out of a bobbin and pick up the bending part of the belt from the ground. By proposi...
-
Real-world applications require light-weight, energy-efficient, fully autonomous robots. Yet, increasing autonomy is oftentimes synonymous with escalating computational requirements. It might thus be desirable to offload intensive computation—not only sensing and planning, but also low-level whole-body control—to remote servers in order to reduce on-board computational needs. Fifth Generation (5G)...
-
The musculoskeletal humanoid has various biomimetic benefits, and the redundant muscle arrangement is one of its most important characteristics. This redundancy can achieve fail-safe redundant actuation and variable stiffness control. However, there is a problem that the maximum joint angle velocity is limited by the slowest muscle among the redundant muscles. In this study, we propose two methods...
-
The recent progress of an ongoing project utilizing a ducted-fan propulsion system to improve a humanoid robot's ability to step over large ditches is reported. A novel method (GAS) based on the genetic algorithm with smoothness constraint can effectively minimize the thrust by optimizing the robot's posture during 3D stepping. The significant advantage of the method is that it can realize the con...
-
The musculoskeletal humanoid has various biomimetic benefits, and it is important that we can embed and evaluate human reflexes in the actual robot. Although stretch reflex has been implemented in lower limbs of musculoskeletal humanoids, we apply it to the upper limb to discover its useful applications. We consider the implementation of stretch reflex in the actual robot, its active/passive appli...
-
In this work, we propose a system for humanoid robot fast motions. When a humanoid robot performs a motion such as a tennis forehand stroke motion, a whole-body fast motion in reaction to visual information is required. There are three problems to tackle. (1) Motion is desired to be quick. (2) Real-time visual processing considering visual noises is needed. (3) Real-time joint angle modification w...
-
A long-standing argument in model-based control of locomotion is about the level of complexity that a model should have to define a behavior such as running. Even though a goldilocks model based on biomechanical evidence is often sought, it is unclear what level of complexity qualifies to be such a model. This dilemma deepens further for bipedal robotic running with point feet, since these robots ...
-
We consider the motion planning problem of a hopper navigating a terrain comprising stepping stones while optimizing an energy metric. The most widely used approach of discrete searches (e.g., A-star) cannot handle boundary conditions (e.g., end path constraints on position, velocity). However, continuous optimizations can easily deal with the boundary value problem but are not widely used in moti...
-
Robots encounter many risks that threaten the success of practical locomotion tasks. Legs break, electrical components overheat, and feet can unexpectedly slip. When all risks cannot be completely avoided, how does a robot decide its best action? We present a method for planning robot motions by reasoning about risk-of-failure probabilities instead of applying cost-penalty functions or inflexible ...
-
Although they take many forms, legged robots rely upon springs to achieve high speed, dynamic locomotion. In this paper we examine the effect of adding parallel springs to robots that rely on virtual compliance. Specifically, we consider the trade-off between energetic efficiency and leg versatility that comes while using Parallel Elastic Actuators (PEAs). To do this, we vary the ratio of physical...
-
The ability of legged systems to traverse highly- constrained environments depends by and large on the performance of their motion and balance controllers. This paper presents a controller that excels in a scenario that most state- of-the-art balance controllers have not yet addressed: line walking, or walking on nearly null support regions. Our approach uses a low-dimensional virtual model (2-DoF...
-
Continuous robot operation in extreme scenarios such as underground mines or sewers is difficult because exteroceptive sensors may fail due to fog, darkness, dirt or malfunction. So as to enable autonomous navigation in these kinds of situations, we have developed a type of proprioceptive localization which exploits the foot contacts made by a quadruped robot to localize against a prior map of an ...
-
Animal-level agility and robustness in robots cannot be accomplished by solely relying on blind locomotion controllers. A significant portion of a robot’s ability to traverse terrain comes from reacting to the external world through visual sensing. However, embedding the sensors and compute that provide sufficient accuracy at high speeds is challenging, especially if the robot has significant spac...
-
This paper shows how CasADi’s state-of-the-art implementation of algorithmic differentiation can be leveraged to formulate and efficiently solve gait optimization problems, enabling rapid gait design for high-dimensional biped robots. Comparative studies on a 7-DOF planar biped show that CasADi generates optimal gaits 4 times faster than another existing advanced optimization package. The framewor...
-
Legged robots have great potential to perform complex loco-manipulation tasks, yet it is challenging to keep the robot balanced while it interacts with the environment. In this paper we investigated the use of additional contact points for maximising the robustness of loco-manipulation motions. Specifically, body-ground contact was studied for its ability to enhance robustness and manipulation cap...
-
This article introduces an innovative model-based strategy for designing a legged robot to generate animal-like running dynamics with differentiated leg braking and thrusting force patterns. Linear springs were utilized as legs, but instead of having one end of each spring connected directly to the hip joint, one extra bar was added to offset the spring's direction. The robot's front and hind legs...
-
Current techniques in robot design and fabrication are time consuming and costly. Robot designs are needed that facilitate low-cost fabrication techniques and reduce the design to production timeline. Here we present an axial-rotational coupled metastructure that can serve as the functional core of a low-cost 3D printed walking robot. Using an origami-inspired assembly technique, the axial-rotatio...
-
Underactuated designs of robot limbs can enable these systems to passively adapt their joint configuration in response to external forces. Passive adaptation and reconfiguration can be extremely beneficial in situations where manipulation or locomotion with complex substrates is required. A common design for underactuated systems often involves a single tendon that actuates multiple rotational joi...
-
Creating robots with emotional personalities will transform the usability of robots in the real-world. As previous emotive social robots are mostly based on statically stable robots whose mobility is limited, this paper develops an animation to real-world pipeline that enables dynamic bipedal robots that can twist, wiggle, and walk to behave with emotions. First, an animation method is introduced ...
-
For mitigating joint impact torques, researchers have reduced joint stiffness by series elastic actuators, reflected inertia by low gear ratios, and friction torque from drive-trains. However, these impact mitigation methods may impair the control performance of contact forces or may increase motor and robot mass. This paper proposes a design method for achieving a balance between impact mitigatio...
-
Truss structures can be found in many buildings and civil infrastructure such as bridges and towers. But as these architectures age, their maintenance is required to keep them structurally sound. A legged robotic solution capable of climbing these structures for maintenance is sought, but determining the size and shape of such a robot to maximise structure coverage is a challenging task. This pape...
-
Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation ope...
-
Quadrupedal locomotion skills are challenging to develop. In recent years, deep Reinforcement Learning promises to automate the development of locomotion controllers and map sensory observations to low-level actions. Moreover, the full robot dynamics model can be exploited, but no model-based simplifications are to be made. In this work, a method for developing controllers for the Laelaps II robot...
-
In the field of quadruped locomotion, dynamic locomotion behavior, and rich integration with sensory feedback represents a significant development. In this paper, we present an efficient neural model, which includes CPG and its sensorimotor coordination, and demonstrate its implementation in a quadruped robot to show how efficient integration of motor and sensory feedback can generate dynamic beha...
-
Spiking neurons might play a larger role than simply as an efficient signal transmitter. Several studies have demonstrated how movements can be generated using networks of spiking neurons. However, the complexity of spiking neural networks makes their implementation difficult, and the use of spiking neurons in robotics has remained largely impractical. In this paper, we show that the addition of a...
-
The purpose of this study is to make life-sized humanoid robots acquire tool manipulation skills that require complicated force adjustment. The difficulty in acquisition of tool manipulation skills comes from the hardship in physical modeling. Recent research have revealed that deep reinforcement learning (DRL), a model-free approach, performs superior in such tasks. However, DRL in general has a ...
-
This paper presents novel control techniques for passivation and stabilisation of floating-base systems with contacts, whose dynamical models comprise both joint-space, and Cartesian floating-base coordinates. The aforementioned results are achieved using both minimally model-based, and completely model-free controllers that employ power-shaping signals. Model-free control is permitted through usa...
-
Human jumping involves not only lower limbs but also whole-body coordination. During jumping, the effect of sinking the center of mass for recoil and arm swing are significant, and they can cause changes in the jump height. However, upper body movements during jumping movements of humanoid robots have not been studied adequately. When jumping involves only the lower limbs, the burden on the lower ...
-
This work presents a motion planning algorithm for legged robots capable of constructing long-horizon dynamic plans in real-time. Many existing methods use models that prohibit flight phases or even require static stability, while those that permit these dynamics often plan over short horizons or take minutes to compute. The algorithm presented here resolves these issues through a reduced-order dy...
-
This paper presents a framework providing a full pipeline to execute a complex physical interaction behaviour of a humanoid bipedal robot, both from a theoretical and a practical standpoint. Building from a multi-contact control architecture that combines contact planning and reactive force distribution capabilities, the main contribution of this work consists in the integration of a sample-based ...
-
A robot cannot lift up an object if it is not feasible to do so. However, in most research on robot lifting, "feasibility" is usually presumed to exist a priori. This paper proposes a three-step method for a humanoid robot to reason about the feasibility of lifting a heavy box with physical properties that are unknown to the robot. Since feasibility of lifting is directly related to the physical p...
-
Performing large step-ups is a challenging task for a humanoid robot. It requires the robot to perform motions at the limit of its reachable workspace while straining to move its body upon the obstacle. This paper presents a non-linear trajectory optimization method for generating step-up motions. We adopt a simplified model of the centroidal dynamics to generate feasible Center of Mass trajectori...
-
Wheeled-legged robots combine the efficiency of wheeled robots when driving on suitably flat surfaces and versatility of legged robots when stepping over or around obstacles. This paper introduces a planning and control framework to realise dynamic locomotion for wheeled biped robots. We propose the Cart-Linear Inverted Pendulum Model (Cart-LIPM) as a template model for the rolling motion and the ...
-
Most current bipedal robots were modeled with an assumption that there is no slip between the stance foot and ground. This paper relaxes that assumption and undertakes a comprehensive study of a compass gait biped with foot slipping. It is found that slips are most likely to happen near impact for a broad range of gaits. Among these gaits, ones with a backward swing foot velocity relative to the g...
-
Passive dynamic walking is a model that walks down a shallow slope without any control or input. This model has been widely used to investigate how stable walking is generated from a dynamic viewpoint, which is useful to provide design principles for developing energy-efficient biped robots. However, the basin of attraction is very small and thin, and it has a fractal-like complicated shape. This ...
-
This paper aims to develop time-varying virtual constraint controllers that allow stable and agile bounding gaits for full-order hybrid dynamical models of quadrupedal locomotion. As opposed to state-based nonlinear controllers, time-varying controllers can initiate locomotion from zero velocity. Motivated by this property, we investigate the stability guarantees that can be provided by the time-v...
-
The authors have investigated underactuated locomotion robots with an inner wobbling mass, it is discovered that the wobbling mass controls the gait speed by entrainment. Supplying the wobbling from outside, outer wobbling entrains load objects and controls the transferring speed. In this paper, we propose a vibratory conveyor system based on the frequency entrainment of a limit cycle walker. The ...
-
This paper proposes a multi-task control strategy for a quadruped robot named THU-QUAD II. The mechanical design of the robot ensures a wide range of motion for all joints, which allows it to stand and walk like a mammal as well as sprawl to the ground and crawl like a reptile. Five basic leg configurations are defined for the robot, including four mammal-type configurations with bidirectional kne...
-
This paper describes the design, control and initial experimental results of the quadruped robot LLAMA. Designed to operate in a human-scale world, this 67kg-class, all-electric robot is capable of rapid motion over a variety of terrains. Thanks to a unique leg configuration and custom high-torque, low gear-ratio motors, it can move omnidirectionally at speeds over 1 m/s. A hierarchical reactive c...
-
A floating base system is inevitably to contact the environment while it is moving. This paper explores the contact force estimation and regulation algorithm for a position-controlled floating base system without joint torque information. First, the joint space dynamic model of the system is presented and transformed into the contact space. Then, the inverse dynamics method is employed to estimate...
-
Although legged robots are becoming more nonlinear with higher degrees of freedom (DOFs), the centralized nonlinear control methods required to achieve stable locomotion cannot scale with the dimensionality of these robots. This paper investigates time-varying decentralized feedback control architectures based on hybrid zero dynamics (HZD) that stabilize dynamic legged locomotion with high degrees...
-
This paper presents a constrained nonlinear model predictive control (NMPC) framework for legged locomotion. The framework assumes a legged robot as a floating base single rigid body with contact forces being applied to the body as external forces. With consideration of orientation dynamics evolving on the rotation manifold SO(3), analytic Jacobians which are necessary for constructing the gradien...
-
An important issue when synthesizing legged locomotion plans is the combinatorial complexity that arises from gait pattern selection. Though it can be defined manually, the gait pattern plays an important role in the feasibility and optimality of a motion with respect to a task. Replacing human intuition with an automatic and efficient approach for gait pattern selection would allow for more auton...
-
This paper proposes a kinodynamic motion plan-ning framework for multi-legged robot jumping based on the mixed-integer convex program (MICP), which simultaneously reasons about centroidal motion, contact points, wrench, and gait sequences. This method uniquely combines configuration space discretization and the construction of feasible wrench polytope (FWP) to encode kinematic constraints, actuato...
-
Can we design motion primitives for complex legged systems uniformly for different terrain types without neglecting modeling details? This paper presents a method for rapidly generating quadrupedal locomotion on sloped terrains-from modeling to gait generation, to hardware demonstration. At the core of this approach is the observation that a quadrupedal robot can be exactly decomposed into coupled...
-
Elaborate trajectory optimization models with many degrees of freedom can be a useful locomotion-planning tool, as they provide rich solutions that take advantage of the robot's specific morphology. They are, however, prone to falling into local minima. Depending on the seed that initializes the solver, the trajectories themselves and the extent to which they minimize the cost function can vary wi...
-
As a strategy to address the difficulties encountered when modeling and controlling a musculoskeletal system, we present a straightforward implementation of an autonomous decentralized motion control system in this paper; the system is inspired by the spinal reflex system of animals. We developed an artificial receptor, a muscle, and a neuron to mechanically implement the reflex mechanisms of anim...
-
In this paper, we present the second version of a reconfigurable modular legged robot, Snapbot V2. The mechanical design of Snapbot V2 is enhanced for better dynamic performance and robust connection with modular legs. A motion generator for locomotion is developed to achieve various locomotion skills in one to six-leg configurations. The locomotion is tested on a multi-body dynamic simulation mod...
-
Passive sensing with ambient WiFi signals is a promising technique that will enable new types of human-robot interactions while preserving users' privacy. Here, we present PresSense, a system for human respiration sensing in noisy environments. Unlike existing WiFi-based respiration sensors, we employ a human presence detector, improving the robustness in scenarios where no human is present in an ...
-
Humans and robots are increasingly sharing their workspaces to benefit from the precision, endurance, and strength of machines and the universal capabilities of humans. Instead of performing time-consuming real experiments, computer simulations of humans could help to optimally orchestrate human and robotic tasks—either for setting up new production cells or by optimizing the motion planning of al...
-
A reduced balance ability can lead to falls and critical injuries. To prevent falls, humans use reaction forces and torques generated by swinging their arms. In animals, we can find that a similar strategy is taken using tails. Inspired by these strategies, we propose an approach that utilizes a robotic appendage as a human balance supporter without assistance from environmental contact. As a proo...
-
To function effectively in real-world environments, powered wearable robots such as exoskeletons and robotic prostheses must recognize the user's motion intent by detecting the user's locomotion modes such as walking, stair ascent and descent or ramp ascent and descent. Traditionally, intent detection is achieved using rule based methods such as state machines or fuzzy logic using data from wearab...
-
Human beings and animals are capable of recognizing places from a previous journey when viewing them under different environmental conditions (e.g., illuminations and weathers). This paper seeks to provide robots with a human-like place recognition ability using a new point cloud feature learning method. This is a challenging problem due to the difficulty of extracting invariant local descriptors ...
-
As an essential component of visual simultaneous localization and mapping (SLAM), place recognition is crucial for robot navigation and autonomous driving. Existing methods often formulate visual place recognition as feature matching, which is computationally expensive for many robotic applications with limited computing power, e.g., autonomous driving and cleaning robot. Inspired by the fact that...
-
In this paper, we introduce an ambiguity-aware robust active SLAM (ARAS) framework that makes use of multi-hypothesis state and map estimations to achieve better robustness. Ambiguous measurements can result in multiple probable solutions in a multi-hypothesis SLAM (MH-SLAM) system if they are temporarily unsolvable (due to insufficient information), our ARAS aims at taking all these probable esti...
-
This paper presents a proof-of-concept for a localization and mapping system for magnetic crawlers performing inspection tasks on structures made of large metal plates. By relying on ultrasonic guided waves reflected from the plate edges, we show that it is possible to recover the plate geometry and robot trajectory to a precision comparable to the signal wavelength. The approach is tested using r...
-
Simultaneous Localization and Mapping (SLAM) is considered a mature research field with numerous applications and publicly available open-source systems. Despite this maturity, existing SLAM systems often rely on ad-hoc implementations or are tailored to predefined sensor setups. In this work, we tackle these issues, proposing a novel unified SLAM architecture specifically designed to standardize ...
-
In this paper, we consider the problem of distributed pose graph optimization (PGO) that has extensive applications in multi-robot simultaneous localization and mapping (SLAM). We propose majorization minimization methods for distributed PGO and show that our methods are guaranteed to converge to first-order critical points under mild conditions. Furthermore, since our methods rely a proximal oper...
-
The ability to infer map variables and estimate pose is crucial to the operation of autonomous mobile robots. In most cases the shared dependency between these variables is modeled through a multivariate Gaussian distribution, but there are many situations where that assumption is unrealistic. Our paper shows how it is possible to relax this assumption and perform simultaneous localization and map...
-
In this work, we present a novel method for reconstructing particular 3D surface points using an imaging sonar sensor. We derive the two-dimensional Fermat flow equation, which may be applied to the planes defined by each discrete azimuth angle in the sonar image. We show that the Fermat flow equation applies to boundary points and surface points which correspond to specular reflections within the...
-
Motivated by the goal of achieving robust, drift-free pose estimation in long-term autonomous navigation, in this work we propose a methodology to fuse global positional information with visual and inertial measurements in a tightly-coupled nonlinear-optimization-based estimator. Differently from previous works, which are loosely-coupled, the use of a tightly-coupled approach allows exploiting the...
-
In recent years, many excellent SLAM methods based on cameras, especially the camera-IMU fusion (VIO), have emerged, which has greatly improved the accuracy and robustness of SLAM. However, we find through experiments that most of the existing VIO methods perform well on drones or drone datasets, but for ground robots on complex terrain, they cannot continuously provide accurate and robust localiz...
-
Introducing object-level semantic information into simultaneous localization and mapping (SLAM) system is critical. It not only improves the performance but also enables tasks specified in terms of meaningful objects. This work presents OrcVIO, for visual-inertial odometry tightly coupled with tracking and optimization over structured object models. OrcVIO differentiates through semantic feature a...
-
Multi-sensor fusion of multi-modal measurements from commodity inertial, visual and LiDAR sensors to provide robust and accurate 6DOF pose estimation holds great potential in robotics and beyond. In this paper, building upon our prior work (i.e., LIC-Fusion), we develop a sliding-window filter based LiDAR-Inertial-Camera odometry with online spatiotemporal calibration (i.e., LIC-Fusion 2.0), which...
-
With monocular Visual-Inertial Odometry (VIO) system, 3D point cloud and camera motion can be estimated simultaneously. Because pure sparse 3D points provide a structureless representation of the environment, generating 3D mesh from sparse points can further model the environment topology and produce dense mapping. To improve the accuracy of 3D mesh generation and localization, we propose a tightl...
-
We present SplitFusion, a novel dense RGB-D SLAM framework that simultaneously performs tracking and dense reconstruction for both rigid and non-rigid components of the scene. SplitFusion first adopts deep learning based semantic instant segmentation technique to split the scene into rigid or non-rigid surfaces. The split surfaces are independently tracked via rigid or non-rigid ICP and reconstruc...
-
We propose a framework for tightly-coupled lidar inertial odometry via smoothing and mapping, LIO-SAM, that achieves highly accurate, real-time mobile robot trajectory estimation and map-building. LIO-SAM formulates lidar-inertial odometry atop a factor graph, allowing a multitude of relative and absolute measurements, including loop closures, to be incorporated from different sources as factors i...
-
This paper proposes a 3D LiDAR simultaneous localization and mapping (SLAM) method that improves accuracy, robustness, and computational efficiency for an iterative closest point (ICP) algorithm employing a locally approximated geometry with clusters of normal distributions. In comparison with previous normal distribution-based ICP methods, such as normal distribution transformation and generalize...
-
Detecting loop closures in 3D Light Detection and Ranging (LiDAR) data is a challenging task since point-level methods always suffer from instability. This paper presents a semantic-level approach named GOSMatch to perform reliable place recognition. Our method leverages novel descriptors, which are generated from the spatial relationship between semantics, to perform frame description and data as...
-
Place recognition is essential for SLAM system since it is critical for loop closure and can help to correct the accumulated drift and result in a globally consistent map. Unlike the visual slam which can use diverse feature detection methods to describe the scene, there are limited works reported to represent a place using single LiDAR scan. In this paper, we propose a segmentation-based egocentr...
-
Numerous Simultaneous Localization and Mapping (SLAM) algorithms have been presented in last decade using different sensor modalities. However, robust SLAM in extreme weather conditions is still an open research problem. In this paper, RadarSLAM, a full radar based graph SLAM system, is proposed for reliable localization and mapping in large-scale environments. It is composed of pose tracking, loc...
-
GP-SLAM+: real-time 3D lidar SLAM based on improved regionalized Gaussian process map reconstruction
This paper presents a 3D lidar SLAM system based on improved regionalized Gaussian process (GP) map reconstruction to provide both low-drift state estimation and mapping in real-time for robotics applications. We utilize spatial GP regression to model the environment. This tool enables us to recover surfaces including those in sparsely scanned areas and obtain uniform samples with uncertainty. Tho...
-
State representation learning (SRL) in partially observable Markov decision processes has been studied to learn abstract features of data useful for robot control tasks. For SRL, acquiring domain-agnostic states is essential for achieving efficient imitation learning. Without these states, imitation learning is hampered by domain-dependent information useless for control. However, existing methods...
-
General-purpose trajectory planning algorithms for automated driving utilize complex reward functions to perform a combined optimization of strategic, behavioral, and kinematic features. The specification and tuning of a single reward function is a tedious task and does not generalize over a large set of traffic situations. Deep learning approaches based on path integral inverse reinforcement lear...
-
We consider the problem of visual imitation learning without human kinesthetic teaching or teleoperation, nor access to an interactive reinforcement learning training environment. We present a geometric perspective to this problem where geometric feature correspondences are learned from one training video and used to execute tasks via visual servoing. Specifically, we propose VGS-IL (Visual Geomet...
-
We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less ma...
-
Existing architectures for imitation learning using image-to-action policy networks perform poorly when presented with an input image containing multiple instances of the object of interest, especially when the number of expert demonstrations available for training are limited. We show that end-to-end policy networks can be trained in a sample efficient manner by (a) appending the feature map outp...
-
We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, stochastic, nonlinear dynamics. Our approach extends the Normalizing Flows framework to learn stable Stochastic Differential Equations. We prove the Lyapunov stability for a class of Stochastic Differential Equations and we propose a learning algorithm to learn them from a set of demonstrated tr...
-
Deep Generative Models such as Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE) have found multiple applications in Robotics, with recent works suggesting the potential use of these methods as a generic solution for the estimation of sampling distributions for motion planning in parameterized sets of environments. In this work we provide a first empirical study of challenge...
-
This paper presents a teaching by demonstration method for contact tasks with periodic movement on planar surfaces of unknown pose. To learn the motion on the plane, we utilize frequency oscillators with periodic movement primitives and we propose modified adaptation rules along with an extraction method of the task’s fundamental frequency by automatically discarding near-zero frequency components...
-
Sudden changes in the dynamics of robotic tasks, such as contact with an object or the latching of a door, are often viewed as inconvenient discontinuities that make manipulation difficult. However, when these transitions are well-understood, they can be leveraged to reduce uncertainty or aid manipulation-for example, wiggling a screw to determine if it is fully inserted or not. Current model-free...
-
Being able to quickly adapt to changes in dynamics is paramount in model-based control for object manipulation tasks. In order to influence fast adaptation of the inverse dynamics model's parameters, data efficiency is crucial. Given observed data, a key element to how an optimizer updates model parameters is the loss function. In this work, we propose to apply meta-learning to learn structured, s...
-
Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain ...
-
Planar pushing remains a challenging research topic, where building the dynamic model of the interaction is the core issue. Even an accurate analytical dynamic model is inherently unstable because physics parameters such as inertia and friction can only be approximated. Data-driven models usually rely on large amounts of training data, but data collection is time consuming when working with real r...
-
This paper introduces a new technique for learning probabilistic models of mass and friction distributions of unknown objects, and performing robust sliding actions by using the learned models. The proposed method is executed in two consecutive phases. In the exploration phase, a table-top object is poked by a robot from different angles. The observed motions of the object are compared against sim...
-
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces. In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own performance from real-world experience without requiring human supervision. To this end, we propose a...
-
A key challenge with controlling complex dynamical systems is to accurately model them. However, this requirement is very hard to satisfy in practice. Data-driven approaches such as Gaussian processes (GPs) have proved quite effective by employing regression based methods to capture the unmodeled dynamical effects. However, GPs scale cubically with number of data points n, and it is often a challe...
-
Locomotion is a prime example for adaptive behavior in animals and biological control principles have inspired control architectures for legged robots. While machine learning has been successfully applied to many tasks in recent years, Deep Reinforcement Learning approaches still appear to struggle when applied to real world robots in continuous control tasks and in particular do not appear as rob...
-
Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In addition, kinodynamic constraints are often non-differentiable and difficult to implement in an optimisation approach. In this work, these challenges are addressed by framing quadruped control as o...
-
Humans use simple probing actions to develop intuition about the physical behavior of common objects. Such intuition is particularly useful for adaptive estimation of favorable manipulation strategies of those objects in novel contexts. For example, observing the effect of tilt on a transparent bottle containing an unknown liquid provides clues on how the liquid might be poured. It is desirable to...
-
Humans learn about object properties using multiple modes of perception. Recent advances show that robots can use non-visual sensory modalities (i.e., haptic and tactile sensory data) coupled with exploratory behaviors (i.e., grasping, lifting, pushing, dropping, etc.) for learning objects' properties such as shape, weight, material and affordances. However, non-visual sensory representations cann...
-
Applying Deep Reinforcement Learning (DRL) to complex tasks in the field of robotics has proven to be very successful in the recent years. However, most of the publications focus either on applying it to a task in simulation or to a task in a real world setup. Although there are great examples of combining the two worlds with the help of transfer learning, it often requires a lot of additional wor...
-
Zero-shot sim-to-real transfer of tasks with complex dynamics is a highly challenging and unsolved problem. A number of solutions have been proposed in recent years, but we have found that many works do not present a thorough evaluation in the real world, or underplay the significant engineering effort and task-specific fine tuning that is required to achieve the published results. In this paper, ...
-
We explore using reinforcement learning on single and multi-agent systems such that after learning is finished we can apply a policy zero-shot to new environment sizes, as well as different number of agents and entities. Building off previous work, we show how to map back and forth between the state and action space of a standard Markov Decision Process (MDP) and multi-dimensional tensors such tha...
-
We present TrueÆdapt, a model-free method to learn online adaptations of robot trajectories based on their effects on the environment. Given sensory feedback and future waypoints of the original trajectory, a neural network is trained to predict joint accelerations at regular intervals. The adapted trajectory is generated by linear interpolation of the predicted accelerations, leading to continuou...
-
Learning from demonstration (LfD) is an intuitive framework allowing non-expert users to easily (re-)program robots. However, the quality and quantity of demonstrations have a great influence on the generalization performances of LfD approaches. In this paper, we introduce a novel active learning framework in order to improve the generalization capabilities of control policies. The proposed approa...
-
Conventional robot programming methods are not suited for non-experts to intuitively teach robots new tasks. For this reason, the potential of collaborative robots for production cannot yet be fully exploited. In this work, we propose an active learning framework, in which the robot and the user collaborate to incrementally program a complex task. Starting with a basic model, the robot's task know...
-
How can we learn representations for planning that are both efficient and flexible? Task and motion planning models are a good candidate, having been very successful in long-horizon planning tasks-however, they've proved challenging for learning, relying mostly on hand-coded representations. We present a framework for learning constraint-based task and motion planning models using gradient descent...
-
We present a novel learning and control framework that combines artificial neural networks with online trajectory optimization to learn dexterous manipulation skills from human demonstration and to transfer the learned behaviors to real robots. Humans can perform the demonstrations with their own hands and with real objects. An instrumented glove is used to record motions and tactile data. Our sys...
-
In this paper, we consider autonomous driving of a vehicle using imitation learning. Generative adversarial imitation learning (GAIL) is a widely used algorithm for imitation learning. This algorithm leverages positive demonstrations to imitate the behavior of an expert. In this paper, we propose a novel method, called mixed generative adversarial imitation learning (MixGAIL), which incorporates b...
-
Safe autonomous driving requires robust detection of other traffic participants. However, robust does not mean perfect, and safe systems typically minimize missed detections at the expense of a higher false positive rate. This results in conservative and yet potentially dangerous behavior such as avoiding imaginary obstacles. In the context of behavioral cloning, perceptual errors at training time...
-
This paper introduces two simple techniques to improve off-policy Reinforcement Learning (RL) algorithms. First, we formulate off-policy RL as a stochastic proximal point iteration. The target network plays the role of the variable of optimization and the value network computes the proximal operator. Second, we exploits the two value functions commonly employed in state-of-the-art off-policy algor...
-
Recent advancements in Bayesian likelihood-free inference enables a probabilistic treatment for the problem of estimating simulation parameters and their uncertainty given sequences of observations. Domain randomization can be performed much more effectively when a posterior distribution provides the correct uncertainty over parameters in a simulated environment. In this paper, we study the integr...
-
Recent breakthroughs both in reinforcement learning and trajectory optimization have made significant advances towards real world robotic system deployment. Reinforcement learning (RL) can be applied to many problems without needing any modeling or intuition about the system, at the cost of high sample complexity and the inability to prove any metrics about the learned policies. Trajectory optimiz...
-
Recently, there has been a lot of excitement surrounding the use of reinforcement learning for robot control and navigation. However, many of these algorithms encounter difficulty navigating long or complex trajectories. This paper presents a new mobile robot control system called Stochastic Neural Control (SNC), that uses a stochastic policy gradient algorithm for local control and a modified pro...
-
Model-free reinforcement learning (RL) is a powerful approach for learning control policies directly from high-dimensional state and observation. However, it tends to be data-inefficient, which is especially costly in robotic learning tasks. On the other hand, optimal control does not require data if the system model is known, but cannot scale to models with high-dimensional states and observation...
-
Learning control policies in robotic tasks requires a large number of interactions due to small learning rates, bounds on the updates or unknown constraints. In contrast humans can infer protective and safe solutions after a single failure or unexpected observation. In order to reach similar performance, we developed a hierarchical Bayesian optimization algorithm that replicates the cognitive infe...
-
One problem in real-world applications of reinforcement learning is the high dimensionality of the action search spaces, which comes from the combination of actions over time. To reduce the dimensionality of action sequence search spaces, macro actions have been studied, which are sequences of primitive actions to solve tasks. However, previous studies relied on humans to define macro actions or a...
-
Deep learning in combination with improved training techniques and high computational power has led to recent advances in the field of reinforcement learning (RL) and to successful robotic RL applications such as in-hand manipulation. However, most robotic RL relies on a well known initial state distribution. In real-world tasks, this information is however often not available. For example, when d...
-
In the next few years, the amount and variety of context-aware robotic manipulator applications is expected to increase significantly, especially in household environments. In such spaces, thanks to programming by demonstration, non-expert people will be able to teach robots how to perform specific tasks, for which the adaptation to the environment is imperative, for the sake of effectiveness and ...
-
Traffic signal controllers play an essential role in today's traffic system. However, the majority of them currently is not sufficiently flexible or adaptive to generate optimal traffic schedules. In this paper we present an approach to learn policies for signal controllers using deep reinforcement learning aiming for optimized traffic flow. Our method uses a novel formulation of the reward functi...
-
In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task.Our method is trained and evaluated on an in-house collected data-set of 34 vol...
-
Coverage path planning is a well-studied problem in robotics in which a robot must plan a path that passes through every point in a given area repeatedly, usually with a uniform frequency. To address the scenario in which some points need to be visited more frequently than others, this problem has been extended to non-uniform coverage planning. This paper considers the variant of non-uniform cover...
-
Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be c...
-
We propose a model-free algorithm for learning efficient policies capable of returning table tennis balls by controlling robot joints at a rate of 100Hz. We demonstrate that evolutionary search (ES) methods acting on CNN-based policy architectures for non-visual inputs and convolving across time learn compact controllers leading to smooth motions. Furthermore, we show that with appropriately tuned...
-
Continuum robots have long held a great potential for applications in inspection of remote, hard-to-reach environments. In future environments such as the Deep Space Gateway, remote deployment of robotic solutions will require a high level of autonomy due to communication delays and unavailability of human crews. In this work, we explore the application of policy optimization methods through Actor...
-
Deep reinforcement learning (DRL) is capable of learning high-performing policies on a variety of complex high-dimensional tasks, ranging from video games to robotic manipulation. However, standard DRL methods often suffer from poor sample efficiency, partially because they aim to be entirely problem-agnostic. In this work, we introduce a novel approach to exploration and hierarchical skill learni...
-
We explore the interpretation of sound for robot decision making, inspired by human speech comprehension. While previous methods separate sound processing unit and robot controller, we propose an end-to-end deep neural network which directly interprets sound commands for visual-based decision making. The network is trained using reinforcement learning with auxiliary losses on the sight and sound n...
-
We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions. The input LTL formulas have never been seen before, yet the network performs zero-shot generalization to satisfy them. This is a novel form of multi-task learning for RL agents where agents learn from one diverse set of tasks and ge...
-
In the present paper, we propose an extension of the Deep Planning Network (PlaNet), also referred to as PlaNet of the Bayesians (PlaNet-Bayes). There has been a growing demand in model predictive control (MPC) in partially observable environments in which complete information is unavailable because of, for example, lack of expensive sensors. PlaNet is a promising solution to realize such latent M...
-
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects. Planning is performed in a low-dimensional latent state space that embeds images. We define and implement a Latent Space Roadmap (LSR) which is a graph-based structure that globally captures the latent system dynamics. Our framework consists...
-
Utilization of latent space to capture a lower-dimensional representation of a complex dynamics model is explored in this work. The targeted application is of a robotic manipulator executing a complex environment interaction task, in particular, cutting a wooden object. We train two flavours of Variational Autoencoders-standard and Vector-Quantised-to learn the latent space which is then used to i...
-
Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of an held object through tactile exploration. ...
-
For robots acting in human-centered environments, the ability to improve based on experience is essential for reliable and adaptive operation; however, particularly in the context of robot failure analysis, experience-based improvement is practically useful only if robots are also able to reason about and explain the decisions they make during execution. In this paper, we describe and analyse a re...
-
Robots perceive their environment using various sensor modalities, e.g., vision, depth, sound or touch. Each modality provides complementary information for perception. However, while it can be assumed that all modalities are available for training, when deploying the robot in real-world scenarios the sensor setup often varies. In order to gain flexibility with respect to the deployed sensor setup...
-
Precomputed roadmaps can enable effective multi-query motion planning: a roadmap can be built for a robot as if no obstacles were present, and then after edges invalidated by obstacles observed at query time are deleted, path search through the remaining roadmap returns a collision-free plan. However, large roadmaps are memory intensive to store, and can be too slow for practical use. We present a...
-
Mobile robots operating in public environments require the ability to navigate among humans and other obstacles in a socially compliant and safe manner. This work presents a combined imitation learning and deep reinforcement learning approach for motion planning in such crowded and cluttered environments. By separately processing information related to static and dynamic objects, we enable our net...
-
Motion planning for robots of high degrees-of-freedom (DOFs) is an important problem in robotics with sampling-based methods in configuration space $\mathcal{C}$ as one popular solution. Recently, machine learning methods have been introduced into sampling-based motion planning methods, which train a classifier to distinguish collision free subspace from in-collision subspace in $\mathcal{C}$. In ...
-
UAVs or Unmanned Aerial Vehicles are an upcoming technology which has eased human lifestyles in many ways. Due to this trend future skies have a risk of getting congested. In such a situation time optimal collision avoidance would be extremely vital to travel in a shortest possible time by avoiding collisions. The paper proposes a novel method for time optimal collision avoidance for UAVs. The pro...
-
For accomplishing a variety of missions in challenging environments, the capability of navigating with full autonomy while avoiding unexpected obstacles is the most crucial requirement for UAVs in real applications. In this paper, we proposed such a computationally efficient obstacle avoidance trajectory planner that can be used in unknown cluttered environments. Because of the narrow view field o...
-
Robots have been successfully used in well-structured and deterministic environments, but they are still unable to function in unstructured environments mainly because of missing reliable real-time systems that integrate perception and control. In this paper, we close the loop between perception and control for real-time obstacle avoidance by introducing a new robust perception algorithm and a new...
-
Ensuring a safe online motion planning despite a large number of moving agents is the problem addressed in this paper. Collision avoidance is achieved without communication between the agents and without global localization system. The proposed solution is a modification of the Hybrid Reciprocal Velocity Obstacles (HRVO) combined with a tracking error estimation, in order to adapt the Velocity Obs...
-
Picking an item in the presence of other objects can be challenging as it involves occlusions and partial views. Given object models, one approach is to perform object pose estimation and use the most likely candidate pose per object to pick the target without collisions. This approach, however, ignores the uncertainty of the perception process both regarding the target's and the surrounding objec...
-
In this paper, we propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties for safer navigation through cluttered environments. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates through each step of our MPC's finite time ho...
-
For safe and efficient human-robot interaction, a robot needs to predict and understand the intentions of humans who share the same space. Mobile robots are traditionally built to be reactive, moving in unnatural ways without following social protocol, hence forcing people to behave very differently from human-human interaction rules, which can be overcome if robots instead were proactive. In this...
-
This paper proposes a dynamic attention-based visual odometry framework (DAVO), a learning-based VO method, for estimating the ego-motion of a monocular camera. DAVO dynamically adjusts the attention weights on different semantic categories for different motion scenarios based on optical flow maps. These weighted semantic categories can then be used to generate attention maps that highlight the re...
-
Recent CNN-based optical flow approaches have a separated structure of feature extraction and flow estimation. The core task of optical flow is finding the corresponding points while rich representation is just the key part of such matching problems. However, the prior work usually pays more attention to the design of flow decoder than the feature extraction. In this paper, we present a novel opti...
-
In this paper, a global descriptor for a LiDAR point cloud, called LiDAR Iris, is proposed for fast and accurate loop-closure detection. A binary signature image can be obtained for each point cloud after several LoG-Gabor filtering and thresholding operations on the LiDAR-Iris image representation. Given two point clouds, their similarities can be calculated as the Hamming distance of two corresp...
-
Accurate and reliable 3D object detection is vital to safe autonomous driving. Despite recent developments, the performance gap between stereo-based methods and LiDAR-based methods is still considerable. Accurate depth estimation is crucial to the performance of stereo-based 3D object detection methods, particularly for those pixels associated with objects in the foreground. Moreover, stereo-based...
-
In this paper, we tackle the problem of detecting objects in 3D and forecasting their future motion in the context of self-driving. Towards this goal, we design a novel approach that explicitly takes into account the interactions between actors. To capture their spatial-temporal dependencies, we propose a recurrent neural network with a novel Transformer [1] architecture, which we call the Interac...
-
The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different ob...
-
We consider the task of underwater robot navigation for the purpose of collecting scientifically relevant video data for environmental monitoring. The majority of field robots that currently perform monitoring tasks in unstructured natural environments navigate via path-tracking a pre-specified sequence of waypoints. Although this navigation method is often necessary, it is limiting because the ro...
-
In this work, we present a perception-aware path-planning pipeline for Unmanned Aerial Vehicles (UAVs) for navigation in challenging environments. The objective is to reach a given destination safely and accurately by relying on monocular camera-based state estimators, such as Keyframe-based Visual-Inertial Odometry (VIO) systems. Motivated by the recent advances in semantic segmentation using dee...
-
We present a robot navigation system that uses an imitation learning framework to successfully navigate in complex environments. Our framework takes a pre-built 3D scan of a real environment and trains an agent from pre-generated expert trajectories to navigate to any position given a panoramic view of the goal and the current visual input without relying on map, compass, odometry, or relative pos...
-
Autonomous navigation in complex environments is a crucial task in time-sensitive scenarios such as disaster response or search and rescue. However, complex environments pose significant challenges for autonomous platforms to navigate due to their challenging properties: constrained narrow passages, unstable pathway with debris and obstacles, or irregular geological structures and poor lighting co...
-
We present an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor (DVS) as input. The backbone of our pipeline is a bioinspired encoder-decoder neural network architecture - ECN. To train the pipeline, we introduce a covariance normalization technique which resembles the...
-
As one of the most promising areas, mobile robots draw much attention these years. Current work in this field is often evaluated in a few manually designed scenarios, due to the lack of a common experimental platform. Meanwhile, with the recent development of deep learning techniques, some researchers attempt to apply learning-based methods to mobile robot tasks, which requires a substantial amoun...
-
Vision and voice are two vital keys for agents’ interaction and learning. In this paper, we present a novel indoor navigation model called Memory Vision-Voice Indoor Navigation (MVV-IN), which receives voice commands and analyzes multimodal information of visual observation in order to enhance robots’ environment understanding. We make use of single RGB images taken by a rst-view monocular camera....
-
Visual motion estimation is an integral and well-studied challenge in autonomous navigation. Recent work has focused on addressing multimotion estimation, which is especially challenging in highly dynamic environments. Such environments not only comprise multiple, complex motions but also tend to exhibit significant occlusion.Previous work in object tracking focuses on maintaining the integrity of...
-
In this paper, we introduce IDOL, an optimization-based framework for IMU-DVS Odometry using Lines. Event cameras, also called Dynamic Vision Sensors (DVSs), generate highly asynchronous streams of events triggered upon illumination changes for each individual pixel. This novel paradigm presents advantages in low illumination conditions and high-speed motions. Nonetheless, this unconventional sens...
-
Reinforcement Learning (RL), among other learning-based methods, represents powerful tools to solve complex robotic tasks (e.g., actuation, manipulation, navigation, etc.), with the need for real-world data to train these systems as one of its most important limitations. The use of simulators is one way to address this issue, yet knowledge acquired in simulations does not work directly in the real...
-
In this paper, we propose an autonomous method for robot navigation based on a multi-camera setup that takes advantage of a wide field of view. A new multi-task network is designed for handling the visual information supplied by the left, central and right cameras to find the passable area, detect the intersection and infer the steering. Based on the outputs of the network, three navigation indica...
-
Robust estimation of camera motion under the presence of outlier noisevision. Despite existing efforts that focus on detecting motion and scene degeneracies, the best existing approach that builds on Random Consensus Sampling (RANSAC) still has non-negligible failure rate. Since a single failure can lead to the failure of the entire visual simultaneous localization and mapping, it is important to ...
-
Place recognition is a core component of Simultaneous Localization and Mapping (SLAM) algorithms. Particularly in visual SLAM systems, previously-visited places are recognized by measuring the appearance similarity between images representing these locations. However, such approaches are sensitive to visual appearance change and also can be computationally expensive. In this paper, we propose an a...
-
Synthetic data has been applied in many deep learning based computer vision tasks. Limited performance of algorithms trained solely on synthetic data has been approached with domain adaptation techniques such as the ones based on generative adversarial framework. We demonstrate how adversarial training alone can introduce semantic inconsistencies in translated images. To tackle this issue we propo...
-
We present an algorithm for the realtime matching of wireframe extractions in pairs of images. Here we treat extracted wireframes as graphs and propose a simplified Graduated Assignment algorithm to use with this problem. Using this algorithm we achieve a 30% accuracy improvement over the baseline method. We show that, for this problem, the simplified Graduated Assignment algorithm can achieve rea...
-
We propose an efficient edge-based stereo visual odometry (VO) using multiple quadtrees created according to image gradient orientations. To characterize edges, we classify them into eight orientation groups according to their image gradient directions. Using the edge groups, we construct eight quadtrees and set overlapping areas belonging to adjacent quadtrees for robust and efficient matching. F...
-
In this paper, we investigate the perception-aware path finding, planning and following for a class of snake robots autonomously serpentining in an unmodeled and unknown environment. In the work, the onboard LiDAR sensor mounted on the head of the snake robot is utilized to reconstruct the local environment, by which and the modified rapidly-exploring random tree method, a feasible path from the c...
-
Joint Feature Selection and Time Optimal Path Parametrization for High Speed Vision-Aided Navigation
We study a problem in vision-aided navigation in which an autonomous agent has to traverse a specified path in minimal time while ensuring extraction of a steady stream of visual percepts with low latency. Vision-aided robots extract motion estimates from the sequence of images of their on-board cameras by registering the change in bearing to landmarks in their environment. The computational burde...
-
Autonomous valet parking is a specific application for autonomous vehicles. In this task, vehicles need to navigate in narrow, crowded and GPS-denied parking lots. Accurate localization ability is of great importance. Traditional visual-based methods suffer from tracking lost due to texture-less regions, repeated structures, and appearance changes. In this paper, we exploit robust semantic feature...
-
Driver gaze mapping is crucial to estimate driver attention and determine which objects the driver is focusing on while driving. We introduce DGAZE, the first large-scale driver gaze mapping dataset. Unlike previous works, our dataset does not require expensive wearable eye-gaze trackers and instead relies on mobile phone cameras for data collection. The data was collected in a lab setting designe...
-
UAVs face many challenges in autonomous obstacle avoidance in large outdoor scenarios, specifically the long communication distance from ground stations. The computing power of onboard computers is limited, and the unknown obstacles cannot be accurately detected. In this paper, an autonomous obstacle avoidance scheme based on the fusion of millimeter wave radar and monocular camera is proposed. Th...
-
Trajectory prediction has always been a challenging problem for autonomous driving, since it needs to infer the latent intention from the behaviors and interactions from traffic participants. This problem is intrinsically hard, because each participant may behave differently under different environments and interactions. This key is to effectively model the interlaced influence from both spatial c...
-
Current unmanned aerial vehicle (UAV) visual tracking algorithms are primarily limited with respect to: (i) the kind of size variation they can deal with, (ii) the implementation speed which hardly meets the real-time requirement. In this work, a real-time UAV tracking algorithm with powerful size estimation ability is proposed. Specifically, the overall tracking task is allocated to two 2D filter...
-
This paper presents an bio-inspired event-based perception scheme for agile aerial robot maneuvering. It tries to mimic birds, which perform purposeful maneuvers by closing the separation in the retinal image (w.r.t. the goal) to follow time-to-contact trajectories. The proposed approach is based on event cameras, also called artificial retinas, which provide fast response and robustness against m...
-
Simulation data can be utilized to extend real-world driving data in order to cover edge cases, such as vehicle accidents. The importance of handling edge cases can be observed in the high societal costs in handling car accidents, as well as potential dangers to human drivers. In order to cover a wide and diverse range of all edge cases, we systemically parameterize and simulate the most common ac...
-
Safety is crucial for deploying robots in the real world. One way of reasoning about safety of robots is by building safe sets through Hamilton-Jacobi (HJ) reachability. However, safe sets are often computed offline, assuming perfect knowledge of the dynamics, due to high compute time. In the presence of uncertainty, the safe set computed offline becomes inaccurate online, potentially leading to d...
-
LiDAR odometry is a fundamental task for various areas such as robotics, autonomous driving. This problem is difficult since it requires the systems to be highly robust running in noisy real-world data. Existing methods are mostly local iterative methods. Feature-based global registration methods are not preferred since extracting accurate matching pairs in the nonuniform and sparse LiDAR data rem...
-
We propose a novel teach-and-repeat navigation system, SSM-Nav, which is based on the output of the recently introduced SSM visual place recognition methodology. During the teach phase, a teleoperated wheeled robot stores in a database features of images taken along an arbitrary route. During the repeat phase or navigation, a CNN-based comparison of each captured image is performed against the dat...
-
This study proposes a self-supervised method for detecting scene changes from an image pair. For mobile cameras such as drive recorders, to alleviate the camera viewpoints' difference, image alignment and change detection must be optimized simultaneously because they depend on each other. Moreover, lighting condition makes the scene change detection more difficult because it widely varies in image...
-
With the recent boost in autonomous driving, increased attention has been paid on radars as an input for occupancy mapping. Besides their many benefits, the inference of occupied space based on radar detections is notoriously difficult because of the data sparsity and the environment dependent noise (e.g. multipath reflections). Recently, deep learning-based inverse sensor models, from here on cal...
-
Robotic exploration requires adaptively selecting navigation goals that result in the rapid discovery and mapping of an unknown world. In many real-world environments, subtle structural cues can provide insight about the unexplored world, which may be exploited by a decision maker to improve the speed of exploration. In sparse subterranean tunnel networks, these cues come in the form of topologica...
-
Established indoor robot navigation frameworks build on the separation between global and local planners. Whereas global planners rely on traditional graph search algorithms, local planners are expected to handle driving dynamics and resolve minor conflicts. We present a system to train neural-network policies for such a local planner component, explicitly accounting for humans navigating the spac...
-
In this paper, we consider the problem of building learning agents that can efficiently learn to navigate in constrained environments. The main goal is to design agents that can efficiently learn to understand and generalize to different environments using high-dimensional inputs (a 2D map), while following feasible paths that avoid obstacles in obstacle-cluttered environment. To achieve this, we ...
-
Learning-based approaches often outperform hand-coded algorithmic solutions for many problems in robotics. However, learning long-horizon tasks on real robot hardware can be intractable, and transferring a learned policy from simulation to reality is still extremely challenging. We present a novel approach to model-free reinforcement learning that can leverage existing sub-optimal solutions as an ...
-
Path following is a challenging task for legged robots. In this paper, we present a hierarchical control architecture for path following of a quadruped salamander-like robot, in which, the tracking problem is decomposed into two sub-tasks: high-level policy learning based on the framework of reinforcement learning (RL) and low-level traditional controller design. More specifically, the high-level ...
-
Behavioral decision making is an important aspect of autonomous vehicles (AV). In this work, we propose a behavior planning structure based on hierarchical reinforcement learning (HRL) which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-go...
-
Energy-efficient mapless navigation is crucial for mobile robots as they explore unknown environments with limited on-board resources. Although the recent deep rein-forcement learning (DRL) approaches have been successfully applied to navigation, their high energy consumption limits their use in several robotic applications. Here, we propose a neuromorphic approach that combines the energy-efficie...
-
Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadr...
-
Robot control policies learned in simulation do not often transfer well to the real world. Many existing solutions to this sim-to-real problem, such as the Grounded Action Transformation (GAT) algorithm, seek to correct for- or ground-these differences by matching the simulator to the real world. However, the efficacy of these approaches is limited if they do not explicitly account for stochastici...
-
This paper considers the problem of learning behaviors in simulation without knowledge of the precise dynamical properties of the target robot platform(s). In this context, our learning goal is to mutually maximize task efficacy on each environment considered and generalization across the widest possible range of environmental conditions. The physical parameters of the simulator are modified by a ...
-
How to explore environments is one of the most critical factors for the performance of an agent in reinforcement learning. Conventional exploration strategies such as ε-greedy algorithm and Gaussian exploration noise simply depend on pure randomness. However, it is required for an agent to consider its training progress and long-term usefulness of actions to efficiently explore complex environment...
-
We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time; it must choose sensing actions that both curb localization uncertainty and achieve information gain. For this problem, belief space planning methods that forward- simulate robot sensing and estimation may of...
-
Robot navigation in human spaces today largely relies on the construction of precise geometric maps and a global motion plan. In this work, we navigate with only local sensing by using available signage - as designed for humans - in human-made environments such as airports. We propose a formalization of "signage" and define 4 levels of signage that we call complete, fully-specified, consistent and...
-
A semantic understanding of the environment is needed to enable high level autonomy in robotic systems. Recent results have demonstrated rapid progress in underlying technology areas, but few results have been reported on end-to-end systems that enable effective autonomous perception in complex environments. In this paper, we describe an approach for rapidly and autonomously mapping unknown enviro...
-
A robot understands its world through the raw information it senses from its surroundings. This raw information is not suitable as a shared representation between the robot and its user. A semantic map, containing high-level information that both the robot and user understand, is better suited to be a shared representation. We use the semantic map as the user-facing interface on our fleet of floor...
-
The ability to search for objects is a precondition for various robotic tasks. In this paper, we address the problem of finding objects in partially known indoor environments. Using the knowledge of the floor plan and the mapped objects, we consider object-object and object-room co-occurrences as prior information for identifying promising locations where an unmapped object can be present. We prop...
-
We develop an online probabilistic metric-semantic mapping approach for autonomous robots relying on streaming RGB-D observations. We cast this problem as a Bayesian inference task, requiring encoding both the geometric surfaces and semantic labels (e.g., chair, table, wall) of the unknown environment. We propose an online Gaussian Process (GP) training and inference approach, which avoids the com...
-
In order to enable a team of robots to operate successfully, retrieving accurate relative transformation between robots is the fundamental requirement. So far, most research on relative localization mainly focus on geometry features such as points, lines and planes. To address this problem, collaborative semantic map matching is proposed to perform semantic perception and relative localization. Th...
-
Finding chemical compounds in the air has applications when situations such as gas leaks, environmental emergencies and toxic chemical dispersion occur. Enabling robots to undertake this task would provide a powerful tool to prevent dangerous situations and assist humans when emergencies arise. While the dispersion of chemical compounds in the air is intrinsically a three-dimensional (3D) phenomen...
-
Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is...
-
This paper presents VCU-RVI, a new visual inertial odometry (VIO) benchmark with a set of diverse data sequences in different indoor scenarios. The benchmark was captured using an Structure Core (SC) sensor, consisting of an RGB-D camera and an IMU. It provides aligned color and depth images with 640×480 resolution at 30 Hz. The camera's data is synchronized with the IMU's data at 100 Hz. Thirty-n...
-
Human-Robot Interaction (HRI) user studies are challenging to evaluate and compare due to a lack of standardization and the infrastructure required to implement each study. The lack of experimental infrastructure also makes it difficult to systematically evaluate the impact of individual components (e.g., the quality of perception software) on overall system performance. This work proposes a frame...
-
With the recent development of autonomous vehicle technology, there have been active efforts on the deployment of this technology at different scales that include urban and highway driving. While many of the prototypes showcased have been shown to operate under specific cases, little effort has been made to better understand their shortcomings and generalizability to new areas. Distance, uptime an...
-
As robotics matures and increases in complexity, it is more necessary than ever that robot autonomy research be reproducible. Compared to other sciences, there are specific challenges to benchmarking autonomy, such as the complexity of the software stacks, the variability of the hardware and the reliance on data-driven techniques, amongst others. In this paper, we describe a new concept for reprod...
-
Real-world environments are inherently uncertain, and to operate safely in these environments robots must be able to plan around this uncertainty. In the context of motion planning, we desire systems that can maintain an acceptable level of safety as the robot moves, even when the exact locations of nearby obstacles are not known. In this paper, we solve this chance-constrained motion planning pro...
-
Recent advances in Deep Machine Learning have shown promise in solving complex perception and control loops via methods such as reinforcement and imitation learning. However, guaranteeing safety for such learned deep policies has been a challenge due to issues such as partial observability and difficulties in characterizing the behavior of the neural networks. While a lot of emphasis in safe learn...
-
Within a robot autonomy stack, the planner and controller are typically designed separately, and serve different purposes. As such, there is often a diffusion of responsibilities when it comes to ensuring safety for the robot. We propose that a planner and controller should share the same interpretation of safety but apply this knowledge in a different yet complementary way. To achieve this, we us...
-
Multi-agent safe systems have become an increasingly important area of study as we can now easily have multiple AI-powered systems operating together. In such settings, we need to ensure the safety of not only each individual agent, but also the overall system. In this paper, we introduce a novel multi-agent safe learning algorithm that enables decentralized safe navigation when there are multiple...
-
This paper investigates the safe path planning problem for an autonomous vehicle operating in unstructured, cluttered environments. While some objects may be accurately with canonical perception algorithms, other objects and clutter may be harder to track. We present an approach that combines two methods of risk assessment: for objects with reliable tracking, we use a Gaussian Process (GP) regulat...
-
This paper presents a method to validate localization safety for a preplanned trajectory in a given environment. Localization safety is defined as integrity risk and quantified as the probability of an undetected localization failure. Integrity risk differs from previously used metrics in robotics in that it accounts for unmodeled faults and evaluates safety under the worst possible combination of...
-
In this study we set out to design a computer vision-based system to assess human-robot trust in real time during close-proximity human-robot collaboration. This paper presents the setup and hardware for an augmented reality-enabled human-robot collaboration cell as well as a method of measuring operator proximity using an infrared camera. We tested this setup as a tool for assessing trust through...
-
As robots and robotic things become to have more agency, IoRT which consists of robots and robotic things can be considered as a social organization. Accordingly, social organization structure of IoRT could affect users' behavior and perception of IoRT. In this study, in order to examine the effect of social organization structure on people's acceptance of IoRT, we conducted a 2 (social organizati...
-
Common experience suggests that agents who know each other well are better able to work together. In this work, we address the problem of calibrating intention and capabilities in human-robot collaboration. In particular, we focus on scenarios where the robot is attempting to assist a human who is unable to directly communicate her intent. Moreover, both agents may have differing capabilities that...
-
As AI becomes an integral part of our lives, the development of explainable AI, embodied in the decision-making process of an AI or robotic agent, becomes imperative. For a robotic teammate, the ability to generate explanations to justify its behavior is one of the key requirements of explainable agency. Prior work on explanation generation has been focused on supporting the rationale behind the r...
-
This study proposes a technique to estimate the output state of twisted string actuators (TSAs) based on payload's acceleration measurements. We outline differential kinematics relationships of the actuator, re-formulate these into a nonlinear parameter identification problem and then apply linearization techniques to efficiently solve it as a quadratic program. Using accurate estimates of string ...
-
The simulation of static friction, and especially the effect of stiction, is cumbersome to perform in discrete-time due to its discontinuity at zero velocity and its switching behavior. However, it is essential to achieve reliable simulations of friction to develop compliant torque control algorithms, as they are much disturbed by this phenomenon. This paper takes as a base an elastoplastic model ...
-
Synthetic fibre ropes have high tensile strength, a lower friction coefficient and are more flexible than steel ropes, and are therefore increasingly used in robotics. However, their characteristics are not well studied. In particular, previous work investigated the long-term behaviour only under static loading. In this paper, we investigate the elongation behaviour of synthetic fibre ropes under ...
-
This work demonstrates a novel approach to steering a magnetic swimming robot in two dimensions with a single pair of Maxwell coils. By leveraging the curvature of the magnetic field gradient, we achieve motion along two axes. This method allows us to control medical magnetic robots using only existing MRI technology, without requiring additional hardware or posing any additional risk to the patie...
-
Research towards (compliant) actuators, especially redundant ones like the Series Parallel Elastic Actuator (SPEA), has led to the development of drive trains, which have demonstrated to increase efficiency, torque-to-mass-ratio, power-to-mass ratio, etc. In the field of robotics such drive trains can be implemented, enabling technological improvements like safe, adaptable and energy-efficient rob...
-
This paper presents a novel design concept of a miniaturized Magneto-Rheological (MR) clutch. The design uses a set of spur gears as a means to control the torque. MR clutches with various configurations such as disk-, drum-, and armature-based have in the past been reported in the literature. However, to the best of our knowledge, the design of a clutch with spur gears to use MR fluid in squeeze ...
-
Future robots are expected to share their workspace with humans. Controlling and limiting the forces that such robots exert on their environment is crucial. While force control can be achieved actively with the help of force sensing, passive mechanisms have no time delay in their response to external forces, and would therefore be preferable. Series clutch actuators can be used to achieve high lev...
-
In this work, we present and explain in detail the design of a new type of pneumatic actuator made of fire hose, the fire hose actuator (FHA), see Fig. 1. We model the force output of this type of actuator and we compare the theoretic results to the data measured on the laboratory test stand.Furthermore, we present the design of a pneumatic rotary drive that is actuated by four of the above-mentio...
-
To reduce the risk of radiation leakages similar to the incident at the Fukushima Daiichi Nuclear Power Station, robots have been employed to remove fuel debris from reactors. To perform this process safely, it is important to monitor the interior of a reactor. A camera and neutron sensors are attached to the end of a robotic arm to monitor the interior of the reactor. The basic design requirement...
-
The recent development of in-pipe robots (IPR) with locomotion and inspection functions provides a new possibility to water distribution pipe maintenance - to rehabilitate pipe defects internally. Yet only a limited number of Rehabilitation in-pipe robots (R-IPR) have been proposed. One primary concern that impedes the development of Rehabilitation in-pipe robots is the excessive amount of contami...
-
In this paper, we updated an inspection robot with passive adaptation ability, which is used to detect small size water supply pipeline. By geometric calculation and kinematic verification, static model of the robot is checked for flexible movement in the pipeline. Besides, inertial measurement unit is leveraged to simultaneously detect the attitude of robot, and different algorithm is tested to c...
-
This paper details the development, modeling and performance of AmphiSTAR, a novel high-speed amphibious robot. The palm size AmphiSTAR, which belongs to the family of STAR robots, is a "wheeled" robot fitted with propellers at its bottom that allow it to crawl on the ground and run (i.e. hover) on water at high speeds. The AmphiSTAR is inspired by two members of the animal kingdom. It possesses a...
-
This paper presents an innovative robotic mechanism for generating peristaltic motion for robotic locomotion systems. The designed underactuated peristaltic robot utilizes a minimum amount of electromechanical hardware. Such a minimal electromechanical design not only reduces the number of potential failure modes but also provides the robot design with great potential for scaling to larger and sma...
-
This paper investigates the design of a three-degree-of-freedom rotational inertia generator using the gyroscopic effect to provide ungrounded torque feedback. It uses a rotating mass in order to influence the torques needed to move the device, creating a perceived inertia. The dynamic model and the control law of the device are derived, along with those of a comparable concept using three flywhee...
-
A spherical two-degree-of- freedom wrist adapting the structure of the spatial parallelogram is proposed. A U type extended link out of three UU type limbs of the spatial parallelogram is selected as an output link. As a result, the wrist can be interpreted as being formed by combination of a U type limb and a (2-UU)+U type hybrid limb. Screw theory is employed to analyze its first-order kinematic...
-
Most existing mechanical gravity compensators have been developed for revolute joints that are found in majority of articulated robot arms. However, robots such as patient transport robots use prismatic joints, which need to handle a heavy payload. In this study, a high-capacity linear gravity compensator (LGC), which comprises pure mechanical components, such as coil springs, a rack-pinion gear, ...
-
To access rough terrain and enhance the mobility in sandy terrain, a configurable paddle-wheel robot was pro-posed. This report addresses the paddle terradynamics, and the experimental verification of the locomotion performance of the robot over dry sandy terrain. To study the interactive forces between the paddle and the media, a terradynamic model is built and verified through experiments. To ex...
-
In this paper, a new remote center of motion (RCM) mechanism is presented whose end-effector is able to move through an entire hemisphere. In general, minimally invasive surgery (MIS) applications, an elliptic cone workspace with vertex angles of 60° and 90° gives the surgeon enough freedom to operate. Therefore, the majority of the developed RCM mechanisms have such a cone as the workspace. Howev...
-
We consider the design of under-actuated articulated mechanism that are able to maintain stable static balance. Our method augments an user-provided design with counter-weights whose mass and attachment locations are automatically computed. The optimized counterweights adjust the center of gravity such that, for bounded external perturbations, the mechanism returns to its original configuration. U...
-
AIRo-5.1 an in-pipe inspection robot comprised of two passive compliant joints and a single active compliant joint that is driven by a series elastic actuator (SEA) is presented in the course of this study. As an aid in pipeline maintenance, AIRo-5.1 controls joint angles and the torque of middle joints, to enable them to adapt to bend, branch, vertical pipes, and slippery surfaces. To sense the j...
-
This study proposes a novel 7-DOF robotic manipulator called CoSMo-Arm for high torque density multi-link robotic platform based on a concentrically stacked modular actuator (CoSMoA) and an extended coaxial spherical joint module (E-CoSMo) introduced in previous researches. The CoSMoA is an actuator module designed to improve thermal characteristics by stacking the motor actuator parts to share th...
-
This paper proposes a novel modular selfreconfigurable robot (MSRR) "FreeBOT", which can be connected freely at any point on other robots. FreeBOT is mainly composed of two parts: a spherical ferromagnetic shell and an internal magnet. The connection between the modules is genderless and instant, since the internal magnet can freely attract other FreeBOT spherical ferromagnetic shells, and not nee...
-
Movement in bio-inspired robots typically relies on the use of a series of actuators and transmissions with one or more degrees of freedom (DOF), allowing asymmetrical ellipsoidal gaits for use in walking, running, swimming, and crawling. In an effort to simplify these multi-component systems, we present a novel, modular, soft, bi-stable, one DOF dome actuator platform that is capable of complex g...
-
Parallel robots with configurable platforms (PRCPs) combine the benefits of parallel robots with additional functionalities such as grasping and cutting. However, some of the theoretical tools used to study classical parallel robots do not apply to parallel robots with configurable platforms. This paper uses screw theory to study the transferable wrenches from the robot's limbs to the configurable...
-
This paper deals with continuous tension validation for Cable-Driven Parallel Robots (CDPRs). The proposed method aims at determining whether or not a quasi-static path is feasible regarding cable tension limits. The available wrench set (AWS) is the set of wrenches that can be generated with cable tensions within given minimum and maximum limits. A pose of the robot is considered valid regarding ...
-
This work studies redundant actuation for both trajectory tracking and disturbance rejection on flexible cable-driven parallel robots (CDPR). High dynamics/bandwidth unidirectional force generators, like air propellers, are used in combination with conventional but slower cable winding winches. To optimally balance the action of the two types of actuation within their saturation constraints, a mod...
-
We propose a method of subdividing robot tasks into new lower abstract tasks. The description of robot tasks in an abstract manner is effective for motion planning for complex tasks and teaching robot movements in various environments. However, a more efficient task description may be obtained by using a lower abstraction according to the work environment. We argue that a higher abstract task can ...
-
Given an articulated robot arm, we present a method to identify two regions with non-empty interiors. The first region is a subset of the configuration space where every point in the region is manipulable. The second region is a subset of the workspace where every point in the region is reachable by the end-effector. Our method expresses the kinematic state of the robot arm using the maximal coord...
-
Providing explanations of chosen robotic actions can help to increase the transparency of robotic planning and improve users' trust. Social sciences suggest that the best explanations are contrastive, explaining not just why one action is taken, but why one action is taken instead of another. We formalize the notion of contrastive explanations for robotic planning policies based on Markov decision...
-
We investigate a multi-agent planning problem, where each agent aims to achieve an individual task while avoiding collisions with others. We assume that each agent's task is expressed as a Time-Window Temporal Logic (TWTL) specification defined over a 3D environment. We propose a decentralized receding horizon algorithm for online planning of trajectories. We show that when the environment is suff...
-
In this paper, we propose a method for automatically generating object handling actions based on simple action definitions. The need to replace workers by robots is increasing, and, in fact, many research projects on robots have worked with simple motion definitions. Many applications are for mobile robots such as drones, however, and if such methods are applied directly to object handling, like a...
-
Modern robots act in dynamic and partially unknown environments where path replanning can be mandatory if changes in the environment are observed. Task-prioritized control strategies are well known and effective solutions to ensure local adaptation of robot behaviour. The highest priority in a stack of tasks is typically given to the management of correct robot operation or safe interaction with t...
-
In this paper, we consider a temporal logic planning problem in which the objective is to find an infinite trajectory that satisfies an optimal selection from a set of soft specifications expressed in linear temporal logic (LTL) while nevertheless satisfying a hard specification expressed in LTL. Our previous work considered a similar problem in which linear dynamic logic for finite traces (LDLf),...
-
Learning-enabled components have been widely deployed in autonomous systems. However, due to the weak interpretability and the prohibitively high complexity of large-scale machine learning models such as neural networks, reliability has been a crucial concern for safety-critical autonomous systems. This work proposes an online monitor called Reach-Flow for fault prevention of waypoint-following ta...
-
This paper introduces the competitive coverage problem, a new variant of the robotic coverage problem in which a robot R competes with another robot O in order to be the first to cover an area. In the variant discussed in this paper, the asymmetric competitive coverage, O is unaware of the existence of R, which attempts to take that fact into consideration in order to succeed in being the first to...
-
This paper addresses the problem of planning for visibility-based pursuit evasion, in contexts where the pursuer robot may experience some positioning errors as it moves in search of the evader. Specifically, we consider the case in which a pursuer with an omnidirectional sensor searches a known environment to locate an evader that may move arbitrarily quickly. Known algorithms for this problem ar...
-
Computation of the volume of space required for a robot to execute a sweeping motion from a start to a goal has long been identified as a critical primitive operation in both task and motion planning. However, swept volume computation is particularly challenging for multi-link robots with geometric complexity, e.g., manipulators, due to the non-linear geometry. While earlier work has shown that de...
-
Discrete graphs are commonly used to approximately represent configuration spaces used in robot motion planning. This paper explores a representation in which the costs of crossing local regions of the configuration space are represented using piecewise linear regression (PLR). We explore a few simple motion planning problems, and show that for these problems, the memory required to store the repr...
-
Asymptotically optimal sampling-based planners require an intelligent exploration strategy to accelerate convergence. After an initial solution is found, a necessary condition for improvement is to generate new samples in the so-called "Informed Set". However, Informed Sampling can be ineffective in focusing search if the chosen heuristic fails to provide a good estimate of the solution cost. This...
-
Chinese calligraphy is a unique art form with great artistic value but difficult to master. In this paper, we formulate the calligraphy writing problem as a trajectory optimization problem, and propose an improved virtual brush model for simulating the real writing process. Our approach is inspired by pseudospectral optimal control in that we parameterize the actuator trajectory for each stroke as...
-
We present a general approach for constructing proofs of motion planning infeasibility. Effective high-dimensional motion planners, such as sampling-based methods, are limited to probabilistic completeness, so when no plan exists, these planners either do not terminate or can only run until a timeout. We address this completeness challenge by augmenting a sampling-based planner with a method to cr...
-
Determining a feasible path for nonholonomic mobile manipulators operating in congested environments is challenging. Sampling-based methods, especially bi-directional tree search-based approaches, are amongst the most promising candidates for quickly finding feasible paths. However, sampling uniformly when using these methods may result in high computation time. This paper introduces two technique...
-
Mobile manipulators consist of a mobile platform equipped with one or more robot arms and are of interest for a wide array of challenging tasks because of their extended workspace and dexterity. Typically, mobile manipulators are deployed in slow-motion collaborative robot scenarios. In this paper, we consider scenarios where accurate high-speed motions are required. We introduce a framework for t...
-
Mobile robotic gas distribution mapping (GDM) is a useful tool for hazardous scene assessment where a quick and accurate representation of gas concentration levels is required throughout a staging area. However, research in robotic path planning for GDM has primarily focused on mapping in open spaces or estimating the source term in dispersion models. Whilst this may be appropriate for environment...
-
Realization of industry-scale, goal-driven, autonomous systems with AI planning technology faces several challenges: flexibly specifying planning goal states in varying situations, synthesizing plans in large state spaces, re-planning in dynamic situations, and facilitating humans to supervise, give feedback and intervene. In this paper, we present Intent-driven Strategic Tactical Planning (ISTP) ...
-
The computational difficulty of planning search paths that seek to maximize a general deterministic value function increases dramatically as desired path lengths increase. Mobile search agents with limited computational resources often utilize receding horizon methods to address the path planning problem. Unfortunately, receding horizon planners may perform poorly due to myopic planning horizons. ...
-
This paper presents a path planner, which enables a nonholonomic mobile manipulator to move its end-effector on an observed surface with a constrained orientation, given start and destination points. A partial point cloud of the environment is captured using a vision-based sensor, but no prior knowledge of the surface shape is assumed. We consider the multi-objective optimisation problem of findin...
-
Multi-goal curvature-constrained planning such as the Dubins Traveling Salesman Problem (DTSP) combines NP-hard combinatorial routing with continuous optimization to determine the optimal vehicle heading angle for each target location. The problem can be addressed as combinatorial routing using a finite set of heading samples at target locations. In such a case, optimal heading samples can be dete...
-
Autonomous navigation missions require online decision making abilities, in order to choose from a given set of candidate actions an action that will lead to the best outcome. In a partially observable setting, decision making under uncertainty, also known as belief space planning (BSP), involves reasoning about belief evolution considering realizations of future observations. Yet, when candidate ...
-
Many kinodynamic motion planners have been developed that guarantee probabilistic completeness and asymptotic optimality for systems for which steering functions are available. Recently, some planners have been developed that achieve these properties of completeness and optimality without requiring a steering function. However, these planners have not taken strong advantage of heuristic guidance t...
-
Robust and safe feedback motion planning and navigation is a critical task for autonomous mobile robotic systems considering the highly dynamic and uncertain nature scenarios of modern applications. For these reasons motion planning and navigation algorithms that have deep roots in feedback control theory has been at the center stage of this domain recently. However, the vast majority of such poli...
-
This paper studies the adaptive reliable shortest path (RSP) planning problem in a Gaussian process (GP) regulated environment. With the reasonable assumption that the travel times of the underlying transportation network follow a multi-variate Gaussian distribution, we propose two algorithms namely, Gaussian process reactive path planning (GPRPP), and Gaussian process proactive path planning (GP4...
-
This paper presents a tangle-free frontier based exploration algorithm for planar mobile robots equipped with limited length and anchored tethers. After planning a path to the closest point in the frontier between free and unknown space, the robot computes an estimate of the future length of its tether and decides, by comparing the anticipated length with the minimum possible tether length, whethe...
-
This paper develops a coverage-centric adaptive path planner to visually survey a planar environment. This is achieved by modifying an existing path planning architecture to use a novel coverage estimation approach called convolved coverage estimation (CCE). The planner maximizes the probability of terrain coverage and exploits terrain features for loop closure to keep path uncertainty in check. T...
-
Most path planning algorithms for covering a complex 3D object ignore physical limitations or constraints on a robot's motion. Adhering to such constraints for a given path can slow down the time to cover the path because the motion may need to be adjusted. This work considers a scenario in computer numerical control (CNC) milling applications, where the robot is a cutting tool that needs to cover...
-
In this paper, we propose Belief Behavior Trees (BBTs), an extension to Behavior Trees (BTs) that allows to automatically create a policy that controls a robot in partially observable environments. We extend the semantic of BTs to account for the uncertainty that affects both the conditions and action nodes of the BT. The tree gets synthesized following a planning strategy for BTs proposed recentl...
-
In order to improve the operation ability of cleaning robots, this paper proposes a decision method for cleaning robot’s operation mode. Firstly, we use the hierarchical expression ability of deep network to obtain the attributes of garbage such as state, shape, distribution, size and so on. Then the causal relationship between the attributes and the operation modes can be built by using joint lea...
-
Integrating robotic systems in architectural and construction processes is of core interest to increase the efficiency of the building industry. Automated planning for such systems enables design analysis tools and facilitates faster design iteration cycles for designers and engineers. However, generic task-and-motion planning (TAMP) for long-horizon construction processes is beyond the capabiliti...
-
As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, i...
-
Learning is usually performed by observing real robot executions. Physics-based simulators are a good alternative for providing highly valuable information while avoiding costly and potentially destructive robot executions. We present a novel approach for learning the probabilities of symbolic robot action outcomes. This is done leveraging different environments, such as physics-based simulators, ...
-
The multi-robot task allocation problem comprises task assignment, coalition formation, task scheduling, and routing. We extend the distributed constraint optimization problem (DCOP) formalism to allocate tasks to a team of robots. The tasks have time window and ordering constraints. Each robot creates a simple temporal network to maintain the tasks in its schedule. The proposed layered framework,...
-
In this work, a novel solution to the optimal motion planning problem is proposed, through a continuous, deterministic and provably correct approach, with guaranteed safety and which is based on a parametrized Artificial Potential Field (APF). In particular, Reinforcement Learning (RL) is applied to adjust appropriately the parameters of the underlying potential field towards minimizing the Hamilt...
-
Heuristic-based graph search algorithms like A* are frequently used to solve motion planning problems in many domains. For most practical applications, it is infeasible and unnecessary to pre-compute the graph representing the whole search space. Instead, these algorithms generate the graph incrementally by applying a fixed set of actions (frequently called motion primitives) to find the successor...
-
Local navigation is an essential ability of any mobile robot working in a real-world environment. One of the most commonly used methods for local navigation is the Dynamic Window Approach (DWA), which heavily depends on the settings of the parameters in its cost function. Since the optimal choice of the parameters depends on the environment that may significantly vary and change at any time, the p...
-
Reliable real-time planning for robots is essential in today's rapidly expanding automated ecosystem. In such environments, traditional methods that plan by relaxing constraints become unreliable or slow-down for kinematically constrained robots. This paper describes the algorithm Dynamic Motion Planning Networks (Dynamic MPNet), an extension to Motion Planning Networks, for non-holonomic robots t...
-
We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning (DRL). Our approach uses local and global information for each robot from motion information maps. We use a three-layer CNN that takes these maps as input to generate a suitable action to drive each robot to its goal position. Our approach is general, learns an optima...
-
Accurately tracking dynamic targets relies on robots accounting for uncertainties in their own states to share information and maintain safety. The problem becomes even more challenging when there is an unknown and time-varying number of targets in the environment. In this paper we address this problem by introducing four new distributed algorithms that allow large teams of robots to: i) run the p...
-
This work proposes a novel method of incorporating calls to a motion planner inside a potential field control policy for safe multi-robot navigation with uncertain dynamics. The proposed framework can handle more general scenes than the control policy and has low computational costs. Our work is robust to uncertain dynamics and quickly finds high-quality paths in scenarios generated from real-worl...
-
We present a Lloyd-based navigation solution for robots that are required to move in a dynamic environment, where static obstacles (e.g, furnitures, parked cars) and unpredicted moving obstacles (e.g., humans, other robots) have to be detected and avoided on the fly. The algorithm can be computed in real-time and falls in the category of the reactive methods. Moreover, we propose an extension to t...
-
We consider the problem of enhanced security of multi-robot systems to prevent cyber-attackers from taking control of one or more robots in the group. We build upon a recently proposed solution that utilizes the physical measurement capabilities of the robots to perform introspection, i.e., detect the malicious actions of compromised agents using other members of the group. In particular, the prop...
-
Modeling the stochastic behavior of interacting agents is key for safe motion planning. In this paper, we study the interaction of risk-aware agents in a game-theoretical framework. Under the entropic risk measure, we derive an iterative algorithm for approximating the intractable feedback Nash equilibria of a risk-sensitive dynamic game. We use an iteratively linearized approximation of the syste...
-
One of the key factors for extended autonomy and resilience of multi-robot systems, especially when robots operate on batteries, is their ability to maintain energy sufficiency by recharging when needed. In situations with limited access to charging facilities, robots need to be able to share and coordinate recharging activities, with guarantees that no robot will run out of energy. In this work, ...
-
Autonomous round trip missions arise as an interesting topic since ESA and NASA agreed to bring soil samples from Mars. This work proposes a new method to improve autonomous rover guidance for this kind of missions. It is focused on the use of dynamically updated cost maps that are used to plan the rover path for a round-trip. The main advantage of the proposed method is the use of gathered inform...
-
This paper presents a trajectory planning algorithm for person following that is more comprehensive than existing algorithms. This algorithm is tailored for a front-wheel-steered vehicle, is designed to follow a person while avoiding collisions with both static and moving obstacles, simultaneously optimizing speed and steering, and minimizing control effort. This algorithm uses nonlinear model pre...
-
Hybrid locomotion, which combines multiple modalities of locomotion within a single robot, enables robots to carry out complex tasks in diverse environments. This paper presents a novel method for planning multi-modal locomotion trajectories using approximate dynamic programming. We formulate this problem as a shortest-path search through a state-space graph, where the edge cost is assigned as opt...
-
Many areas of scientific interest in planetary exploration, such as lunar pits, icy-moon crevasses, and Martian craters, are inaccessible to current wheeled rovers. Rappelling rovers can safely traverse these steep surfaces, but require techniques to navigate their complex terrain. This dynamic navigation is inherently time-critical and communication constraints (e.g. delays and small communicatio...
-
The Next Best View (NBV) problem is important in the active robotic reconstruction. It enables the robot system to perform scanning actions in a reasonable view sequence, and fulfil the reconstruction task in an effective way. Previous works mainly follow the volumetric methods, which convert the point cloud information collected by sensors into a voxel representation space and evaluate candidate ...
-
The paper presents a receding horizon planning strategy for a quadrotor-type mav to navigate through an unknown cluttered environment at high speed. Utilizing a lightweight on-board short-range sensor that generates point-clouds within a narrow Field of View (FOV), the reported approach generates safe and dynamically feasible trajectories within the fov of the sensor, which the mav uses to navigat...
-
Collision avoidance in unknown obstacle-cluttered environments may not always be feasible. This paper focuses on an emerging paradigm shift in which potential collisions with the environment can be harnessed instead of being avoided altogether. To this end, we introduce a new sampling-based online planning algorithm that can explicitly handle the risk of colliding with the environment and can swit...
-
Modern localization techniques allow ground vehicle robots to determine their position with centimeter-level accuracy under nominal conditions, enabling them to utilize fixed maps to navigate their environments. However, when localization measurements become unavailable, the position accuracy will drop and uncertainty will increase. While research and development on localization estimation seeks t...
-
This paper introduces a constraint-based skill framework for programming robot applications. Existing skill frameworks allow application developers to reuse skills and compose them sequentially or in parallel. However, they typically assume that the skills are running independently and in a nominal condition. This limitation hinders their applications for more involved and realistic scenarios e.g....
-
We consider a scenario in which an autonomous vehicle equipped with a downward facing camera operates in a 3D environment and is tasked with searching for an unknown number of stationary targets on the 2D floor of the environment. The key challenge is to minimize the search time while ensuring a high detection accuracy. We model the sensing field using a multi-fidelity Gaussian process that system...
-
Dnamic System Identification approaches usually heavily rely on evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system identif...
-
The snake-like robot without wheels is a bio-inspired robot whose high degree of freedom results in a challenge in autonomous locomotion control. The use of a Spiking Neural Network (SNN) which is a biologically plausible artificial neural network can help to achieve the autonomous locomotion behavior of snake robots in an energy-efficient manner. Approaches that use an SNN without hidden layers h...
-
Learning accurate models of the physical world is required for a lot of robotic manipulation tasks. However, during manipulation, robots are expected to interact with un-known workpieces so that building predictive models which can generalize over a number of these objects is highly desirable. In this paper, we study the problem of designing deep learning agents which can generalize their models o...
-
Control barrier functions are mathematical constructs used to guarantee safety for robotic systems. When integrated as constraints in a quadratic programming optimization problem, instantaneous control synthesis with real-time performance demands can be achieved for robotics applications. Prevailing use has assumed full knowledge of the safety barrier functions, however there are cases where the s...
-
Variable impedance control is advantageous for physical human-robot interaction to improve safety, adaptability and many other aspects. This paper presents a gain-scheduled variable stiffness control approach under strict frequency-domain constraints. Firstly, to reduce conservativeness, we characterize and constrain the impedance rendering, actuator saturation, disturbance/noise rejection and pas...
-
To this day, most robots are installed behind safety fences, separated from the human. New use-case scenarios demand for collaborative robots, e.g. to assist the human with physically challenging tasks. These robots are mainly installed in work-environments with limited space, e.g. existing production lines. This brings certain challenges for the control of such robots. The presented work addresse...
-
While impedance control is one of the most commonly used strategies for robot interaction control, variable impedance control is a more recent preoccupation. If designing impedance control with varying parameters allows increasing the system flexibility and dexterity, it is still a challenging issue, as it may result in a loss of passivity of the control system. This has an important impact on the...
-
This paper presents an alternative impedance controller for a macro-mini robotic system in which the mini robot is unactuated. The approach is verified experimentally on a simple two-degree-of-freedom macro-mini robot. The dynamic analysis of the robot is first presented. Then, a standard impedance controller is derived and analysed. Such a controller is shown to be experimentally unstable when us...
-
Aerial robotic manipulation is an emergent trend that poses several challenges. To overcome some of these, the DLR cable-Suspended Aerial Manipulator (SAM) has been envisioned. SAM is composed of a fully actuated multi-rotor anchored to a main carrier through a cable and a KUKA LWR attached below the multi-rotor. This work presents a control method to allow SAM, which is a holonomically constraine...
-
Magnetically actuated soft robots have recently been identified for application in medicine, due to their potential to perform minimally invasive exploration of human cavities. Magnetic solutions permit further miniaturization when compared to other actuation techniques, without loss in functionalities. Our long-term goal is to propose a novel actuation method for magnetically actuated soft robots...
-
In this paper, we propose a formal language to specify robot skills, i.e. the elementary behaviours or functions provided by the robot platform in order to perform an autonomous mission. The advantage of the language we propose is that it integrates a wide range of elements that allows to define and provide automatic translation both to operational models, used online to control the skill executio...
-
Screw theory is a powerful mathematical tool for the kinematic analysis of mechanisms and has become a cornerstone of modern kinematics. Although screw theory has rooted itself as a core concept, there is a lack of generic software tools for visualization of the geometric pattern of the screw elements. This paper presents STORM, an educational and research oriented framework for analysis and visua...
-
In recent years the need for manipulation tasks in the industrial as well as in the service robotics domain that require compliant interaction with the environment rose. Since then, an increased number of publications use a model-driven approach to describe these tasks. High-level tasks and sequences of skills are coordinated to achieve a desired motion for e.g., screwing, polishing, or snap mount...
-
Robots are currently deployed in safety-critical domains but proper techniques to assess the functional safety of their software are yet to be adopted. This is particularly critical in ROS, where highly configurable robots are built by composing third-party modules. To promote adoption, we advocate the use of lightweight formal methods, automatic techniques with minimal user input and intuitive fe...
-
This paper describes basic implementation of an embedded controller board based on a hybrid architecture equipped with an Intel FPGA SoC and an NVIDIA GPU SoC. Embedded distributed network involving motor-drivers or other embedded boards is constructed with low-latency optical transmission link. The central controller for high-level motion planning is connected via Gigabit Ethernet. The controller...
-
In this paper an approach for the identification of the dynamic parameters, i.e. base parameters, of rigid robots is presented. By using the polynomial approximation operator, an equation is obtained for the identification of the parameters which solely depends on measurable signals and thereby contains no equation error. The resulting expressions can be evaluated online or offline by filtering th...
-
In this paper, nonlinear control techniques are exploited to balance an unmanned bicycle with enlarged stability domain. We consider two cases. For the first case when the autonomous bicycle is balanced by the flywheel, the steering angle is set to zero, and the torque of the flywheel is used as the control input. The controller is designed based on the Interconnection and Damping Assignment Passi...
-
Cable-driven joints proved to be an effective solution in a wide variety of applications ranging from medical to industrial fields where light structures, interaction with unstructured and constrained environments and precise motion are required. These requirements are achieved by moving the actuators from joints to the robot chassis. Despite these positive properties a cable-driven robotic arm re...
-
This work addresses on the following problem: given a set of unsynchronized history observations of two scenes that are correlative on their dynamic changes, the purpose is to learn a cross-scene predictor, so that with the observation of one scene, a robot can onlinely predict the dynamic state of the other. A method is proposed to solve the problem via modeling dynamic correlation using latent s...
-
We suggest a procedure for dynamic parameter estimation of serial robot manipulators. Its basic idea relies on the synthesis of an optimal manipulation trajectory, which is based on properly introduced parameter aggregates to ensure a collection of numerically well-conditioned data-sets, yielding an accurate computation of parameter estimates. The optimal trajectory itself is computed by using a m...
-
This paper presents a method for performing free-fall penetrometer tests for soft soils using an instrumented dart deployed by a quadcopter. Tests were performed with three soil types and used to examine the effect of drop height on the penetration depth and the deceleration profile. Further tests analyzed the force required to remove a dart from the soil and the effect of pulling at different spe...
-
In haptic time delayed teleoperation as the time delay from the communication channel increases, teleoperation system stability and performance degrade. To increase performance and provide better stability margins, various estimation methods and observers have been implemented in literature to more accurately capture the force exerted by the remote system. Previously, solutions focused on environm...
-
We study for the first time the verification problem on learning-enabled state estimation systems for robotics, which use Bayes filter for localisation, and use deep neural network to process sensory input into observations for the Bayes filter. Specifically, we are interested in a robustness property of the systems: given a certain ability to an adversary for it to attack the neural network witho...
-
When exploring an unknown environment, a mobile robot must decide where to observe next. It must do this whilst minimising the risk of failure, by only exploring areas that it expects to be safe. In this context, safety refers to the robot remaining in regions where critical environment features (e.g. terrain steepness, radiation levels) are within ranges the robot is able to tolerate. More specif...
-
Localizing contacts and collisions is an important aspect of failure detection and recovery for robots and can aid perception and exploration of the environment. Contrary to state-of-the-art methods that rely on forces and torques measured on the robot, this paper proposes a kinematic method for proprioceptive contact localization on compliant robots using velocity measurements. The method is vali...
-
Having the merits of chemical inertness and immunity to electromagnetic interference, light weight, small size, and softness, optical waveguides have attracted much attention in making tactile sensors recently. This paper presents a new design of waveguide using two layers of cores, one of which has an uniform width and the other has an incremental width. It is deduced and verified that the contac...
-
The ability to measure multi-axis contact forces and contact surface normals in real time is critical to allow robots to improve their dexterous manipulation and locomotion abilities. This paper presents a new fingertip sensor for 3-axis contact force and contact location detection, as well as improvements on an existing footpad sensor through use of a new artificial neural network estimator. The ...
-
The development of modern sensitive lightweight robots allows the use of robot arms in numerous new scenarios. Especially in applications where interaction between the robot and an object is desired, e.g. in assembly, conventional purely position-controlled robots fail. Former research has focused, among others, on control methods that center on robot-environment interaction. However, these method...
-
In this paper, an approach for automatic peg-in-hole assembly is proposed. The task is divided into two main steps: searching phase and inserting phase. First, a multilayer perceptron network is designed to address the hole search problem and a hybrid force position controller is introduced to ensure a safe and stable interaction with the external environment. Then, for the inserting phase, a vari...
-
This paper presents a novel control strategy for the compensation of the slippage effect during non-rigidly grasped object manipulation. A detailed dynamic model of the interconnected system composed of the robotic manipulator, the object and the internal forces and torques induced by the slippage effect is provided. Next, we design a model-based variable impedance control scheme, in order to achi...
-
A high-quality free-motion rendering is one of the most vital traits to achieve an immersive human-robot interaction. Rendering free-motion is notably challenging for rehabilitation exoskeletons due to their relatively high weight and powerful actuators required for strength training and support. In the presence of dynamic human movements, accurate feedback linearization of the robot's dynamics is...
-
The increasing interest in autonomous robots with a high number of degrees of freedom for industrial applications and service robotics have also increased the demand for efficient control algorithms. The unstructured environment these robots operate in often impose constraints on the joint motion, an important type being the joint limits of the robot itself. These circumstances demand control algo...
-
We present a hierarchical framework for trajectory optimization and optimal feedback whole-body control of wheeled inverted pendulum (WIP) humanoid robot. The framework extends rapidly exponentially stabilizing control Lyapunov functions (RES-CLF) to operational space for controlling WIP humanoid robots while utilizing a hierarchical framework to compute an optimal policy. The upper level of the h...
-
This paper focuses on robustness to disturbance forces and uncertain payloads. We present a novel formulation to optimize the robustness of dynamic trajectories. A straightforward transcription of this formulation into a nonlinear programming problem is not tractable for state-of-the-art solvers, but it is possible to overcome this complication by exploiting the structure induced by the kinematics...
-
Sampling-based motion planners (SBMP) are commonly used to generate motion plans by incrementally constructing a search tree through a robot's configuration space. For high degree-of-freedom systems, sampling is often done in a lower-dimensional space, with a steering function responsible for local planning in the higher-dimensional configuration space. However, for highly-redundant systems with c...
-
We present a path planning strategy for a magnetic millirobot where the nonlinearities in the external magnetic force field (MFF) are encoded in the graph used for planning. The strategy creates a library of candidate MFFs and characterizes their topologies by identifying the unstable manifolds in the workspace. The path planning problem is then posed as a graph search problem where the computed p...
-
In a geometric mechanics framework, the configuration space is decomposed into a shape space and a position space. The internal motion of the system is prescribed by a closed loop in the shape space, which causes net motion in the position space. If the shape space is a simply connected domain in an Euclidean space, then with an optimal choice of the body frame, the displacement in the position sp...
-
Autonomous Navigation and Obstacle Avoidance of a Snake Robot with Combined Velocity-Heading Control
This paper presents combined velocity-heading control of a planar snake robot for the autonomous navigation and obstacle avoidance in a simulation environment. The kinematics and dynamics of the snake robot were derived using the articulated-body algorithm without considering the non-holonomic constraints. A double-layer controller was designed to control both heading direction and average velocit...
-
The feet of robots are typically used to design locomotion strategies, such as balancing, walking, and running. However, they also have great potential to perform manipulation tasks. In this paper, we propose a model predictive control (MPC) framework for a quadrupedal robot to dynamically balance on a ball and simultaneously manipulate it to follow various trajectories such as straight lines, sin...
-
This paper addressed a challenging problem of wheeled-legged robots with high degrees of freedom exploring in unknown rough environments. The proposed method works as a pipeline to achieve prioritized exploration comprising three primary modules: traversability analysis, frontier-based exploration and hybrid locomotion planning. Traversability analysis provides robots an evaluation about surroundi...
-
We present KOVIS, a novel learning-based, calibration-free visual servoing method for fine robotic manipulation tasks with eye-in-hand stereo camera system. We train the deep neural network only in the simulated environment; and the trained model could be directly used for real-world visual servoing tasks. KOVIS consists of two networks. The first keypoint network learns the keypoint representatio...
-
One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a ...
-
We address the monocular visual shape servoing problem. This pushes the challenging visual servoing problem one step further from rigid object manipulation towards deformable object manipulation. Explicitly, it implies deforming the object towards a desired shape in 3D space by robots using monocular 2D vision. We specifically concentrate on a scheme capable of controlling large isometric deformat...
-
This paper studies the problem of Image-Based Visual Servo Control (IBVS) for quadrotors. Although the control of quadrotors has been extensively studied in the last decades, combining the IBVS module with the quadrotor's dynamics is still hard, mainly due to the under-actuation issues related to the quadrotor control as opposed to the 6 DoF control outputs generated by the IBVS modules. We propos...
-
Electric motors have been widely used as the actuators of robot and automation systems. This paper aims at achieving the high-precision position control of motor drive systems. For this purpose, a robust control scheme is presented by combining the internal model principle, the sliding mode technique and the extended state observer (ESO). The PID-type controller is firstly designed by using the in...
-
Trajectory tracking of flexible link robots is a classical control problem. Historically, the link elasticity was considered as something to be removed. Hence, the control performance was guaranteed by adopting high-gain feedback loops and, possibly, a dynamic compensation with the result to stiffen up the dynamic behavior of the robot. Nowadays, robots are pushed more and more towards a safe phys...
-
In this paper, we present an optimal control approach using Linear Matrix Inequalities (LMIs) for trajectory tracking control of a three-wheeled omnidirectional mobile robot in the presence of external disturbances on the robot's actuators and noise in the robot's sensor measurements. First, a state-space representation of the omnidirectional robot dynamics is derived using a point-mass dynamic mo...
-
In this paper, the gain scheduling technique is applied to design a balance controller for an autonomous bicycle with an inertia wheel. Previously, two different balance controllers are needed depending on whether the bicycle is stationary or dynamic. The switch between the two different controllers may cause the instability of the autonomous bicycle. Our proposed gain scheduled controller can bal...
-
Humanoid robots are expected to play a big role at distress sites and disaster sites. There is a variety of multi-contact locomotion forms other than bipedal walking such as crawling through tightly, getting on the rubble by using its knees and elbows, or jumping in and rolling over the obstacles. If such multi-contact locomotion forms can be achieved, robots can reach environments that are curren...
-
Dynamic control for robotic automation tasks is traditionally designed and optimized with a model-based approach, and the performance relies heavily upon accurate system modeling. However, modeling the true dynamics of increasingly complex robotic systems is an extremely challenging task and it often renders the automation system to operate in a non-optimal condition. Notably, many industrial robo...
-
This work uses quadratic programming to perform torque control on an industrial collaborative robot, while keeping defined constraints. Limits for rotational and translational coordinates are considered at position, velocity and acceleration level. Although the problem of having hardware and safe limitations has been considered before. Solutions usually rely on functions that need a proper tuning....
-
The combination of policy search and deep neural networks holds the promise of automating a variety of decision-making tasks. Model Predictive Control (MPC) provides robust solutions to robot control tasks by making use of a dynamical model of the system and solving an optimization problem online over a short planning horizon. In this work, we leverage probabilistic decision-making approaches and ...
-
Recently, Differential Dynamic Programming (DDP) and other similar algorithms have become the solvers of choice when performing non-linear Model Predictive Control (nMPC) with modern robotic devices. The reason is that they have a lower computational cost per iteration when compared with off-the-shelf Non-Linear Programming (NLP) solvers, which enables its online operation. However, they cannot ha...
-
In fields such as minimally invasive surgery, effective control strategies are needed to guarantee safety and accuracy of the surgical task. Mechanical designs and actuation schemes have inevitable limitations such as backlash and joint limits. Moreover, surgical robots need to operate in narrow pathways, which may give rise to additional environmental constraints. Therefore, the control strategie...
-
This paper presents a multirotor control architecture, where Model Predictive Path Integral Control (MPPI) and ℒ1 adaptive control are combined to achieve both fast model predictive trajectory planning and robust trajectory tracking. MPPI provides a framework to solve nonlinear MPC with complex cost functions in real-time. However, it often lacks robustness, especially when the simulated dynamics ...
-
Safety is a critical issue in learning-based robotic and autonomous systems as learned information about their environments is often unreliable and inaccurate. In this paper, we propose a risk-aware motion control tool that is robust against errors in learned distributional information about obstacles moving with unknown dynamics. The salient feature of our model predictive control (MPC) method is...
-
A hierarchical algorithm involving two-layer optimization-based control policies with varying degrees of abstraction is proposed, including upper layer task scheduling and lower layer local path planning. A scenario with two robot arms performing cooperative pick-and-place tasks for moving objects is specifically addressed. The main focus of the paper lies on the bottom layer of the hierarchical c...
-
This paper proposes a new approach for online control law gains adaptation, through the use of neural networks and the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm, in order to optimize the behavior of the robot with respect to an objective function. The neural network considered takes as input the current observed state as well as its uncertainty, and provides as output the ...
-
This paper presents a neural-network based adaptive feedback control structure to regulate the velocity of 3D bipedal robots under dynamics uncertainties. Existing Hybrid Zero Dynamics (HZD)-based controllers regulate velocity through the implementation of heuristic regulators that do not consider model and environmental uncertainties, which may significantly affect the tracking performance of the...
-
With the advent of intelligent transport, quadrotors are becoming an attractive aerial transport solution during emergency evacuations, construction works etc. During such operations, dynamic variations in (possibly unknown) payload and unknown external disturbances cause considerable control challenges for path tracking algorithms. In fact, the state-dependent nature of the resulting uncertaintie...
Introduction
Conference IROS2020 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 1000