DeepNLP IROS2023 Accepted Paper List AI Robotic and STEM Top Conference & Journal Papers
-
Although domestic service robots are expected to assist individuals who require support, they cannot currently interact smoothly with people through natural language. For example, given the instruction “Bring me a bottle from the kitchen,” it is difficult for such robots to specify the bottle in an indoor environment. Most conventional models have been trained on real-world datasets that are labor...
-
Deep neural networks using for real-world classification task require high reliability and robustness. However, the Softmax output by the last layer of network is often over-confident. We propose a novel confidence estimation method by considering model quality for deep classification models. Two metrics, MQ-Repres and MQ-Discri are developed accordingly to evaluate the model quality, and also pro...
-
Drivable area segmentation is an essential component of the visual perception system for autonomous driving vehicles. Recent efforts in deep neural networks have sig-nificantly improved semantic segmentation performance for autonomous driving. However, most DNN-based methods need a large amount of data to train the models, and collecting large-scale datasets with manually labeled ground truth is c...
-
Motion prediction is a challenging task for autonomous vehicles due to uncertainty in the sensor data, the non-deterministic nature of future, and complex behavior of agents. In this paper, we tackle this problem by representing the scene as dynamic occupancy grid maps (DOGMs), associating semantic labels to the occupied cells and incorporating map information. We propose a novel framework that co...
-
Recently, many research studies have been carried out on using deep learning methods for 3D point cloud understanding. However, there is still no remarkable result on 3D point cloud semantic segmentation compared to those of 2D research. One important reason is that 3D data has higher dimensionality but lacks large datasets, which means that the deep learning model is difficult to optimize and eas...
-
Semantic scene understanding is beneficial for mobile robots. Semantic information obtained through onboard cameras can improve robots' navigation performance. However, obtaining semantic information on small mobile robots with constrained power and computation resources is challenging. We propose a new lightweight convolution neural network comparable to previous semantic segmentation algorithms ...
-
Most of the existing moving object segmentation (MOS) methods regard MOS as an independent task, in this paper, we associate the MOS task with semantic segmentation, and propose a semantics-guided network for moving object segmentation (LiDAR-SGMOS). We first transform the range image and semantic features of the past scan into the range view of current scan based on the relative pose between scan...
-
The integration of semantic information in a map allows robots to understand better their environment and make high-level decisions. In the last few years, neural networks have shown enormous progress in their perception capabilities. However, when fusing multiple observations from a neural network in a semantic map, its inherent overconfidence with unknown data gives too much weight to the outlie...
-
Object rearrangement is the problem of enabling a robot to identify the correct object placement in a complex environment. Prior work on object rearrangement has explored a diverse set of techniques for following user instructions to achieve some desired goal state. Logical predicates, images of the goal scene, and natural language descriptions have all been used to instruct a robot in how to arra...
-
Mixup-based data augmentation has been validated to be a critical stage in the self-training framework for unsupervised domain adaptive semantic segmentation (UDASS), which aims to transfer knowledge from a well-annotated (source) domain to an unlabeled (target) domain. Existing self-training methods usually adopt the popular region-based mixup techniques with a random sampling strategy, which unf...
-
Age-related loss of mobility and an increased risk of falling remain major obstacles for older adults to live independently. Many elderly people lack the coordination and strength necessary to perform activities of daily living, such as getting out of bed or stepping into a bathtub. A traditional solution is to install grab bars around the home. For assisting in bathtub transitions, grab bars are ...
-
The trunk movements of an individual paralyzed by spinal cord injury (SCI) can be restored by Functional Neuromuscular Stimulation (FNS), a technique that applies low-level current to motor nerves to activate the muscles generating torques, and thus, produce trunk motions. FNS can be modulated to control trunk movements. However, a stabilizing modulation policy (i.e., control law) is difficult to ...
-
In recent years, the aging of society has become a serious problem, especially in developed countries. Walking is an important element in extending healthy life expectancy in old age. In particular, induction of proper ankle joint alignment at heel contact is important during the gait cycle from the perspective of smooth weight transfer and reduction of burden on the knees and hip. In this study, ...
-
Numerous attempts have been made to reduce metabolic energy while running with the help of assistive devices. A majority of studies on the assistive devices have focused on the assisting torque in the sagittal plane. In the case of running, however, the abduction torque in the frontal plane at the hip joint is greater than the flexion/extension torque in the sagittal plane. During running, as does...
-
Continuous enhancement in wearable technologies has led to several innovations in the healthcare, virtual reality, and robotics sectors. One form of wearable technology is wear-able sensors for kinematic measurements of human motion. However, measuring the kinematics of human movement is a challenging problem as wearable sensors need to conform to complex curvatures and deform without limiting the...
-
There is increasing evidence of the role of compromised mediolateral balance in falls and the need for rehabilitation specifically focused on mediolateral direction for various populations with motor deficits. To address this need, we have developed a neurorehabilitation platform by integrating a wearable robotic hip abduction-adduction exoskeleton with a visual interface. The platform is expected...
-
A strap is a frequently utilized component for securing wearable robots to their users in order to facilitate force transmission between humans and the devices. For the appropriate function of the wearable robot, the pressure between the strap and the skin should be maintained appropriately. Due to muscle contraction, the cross-section area of the human limb changes according to the movement of th...
-
There has been increasing awareness of the difficulties in reaching and extracting people from mass casualty scenarios, such as those arising from natural disasters. While platforms have been designed to consider reaching casualties and even carrying them out of harm's way, the challenge of repositioning a casualty from its found configuration to one suitable for extraction has not been explicitly...
-
Planning robot motions in complex environments is a fundamental research challenge and central to the autonomy, efficiency, and ultimately adoption of robots. While often the environment is assumed to be static, real-world settings, such as assembly lines, contain complex shaped, moving obstacles and changing target states. Therein robots must perform safe and efficient motions to achieve their ta...
-
We propose a co-navigation algorithm that enables a human and a robot to work together to navigate to a common goal. In this system, the human is responsible for making high-level steering decisions, and the robot, in turn, provides haptic feedback for collision avoidance and path suggestions while reacting to changes in the environment. Our algorithm uses optimized Rapidly-exploring Random Trees ...
-
We propose a novel methodology for robotic follow-ahead applications that address the critical challenge of obstacle and occlusion avoidance. Our approach effectively navigates the robot while ensuring avoidance of collisions and occlusions caused by surrounding objects. To achieve this, we developed a high-level decision-making algorithm that generates short-term navigational goals for the mobile...
-
For robots navigating in dynamic environments, exploiting and understanding uncertain human motion prediction is key to generate efficient, safe and legible actions. The robot may perform poorly and cause hindrances if it does not reason over possible, multi-modal future social interactions. With the goal of enhancing autonomous navigation in cluttered environments, we propose a novel formulation ...
-
Obstacle avoidance (OA) and joint-limit avoidance (JLA) are essential for redundant manipulators to ensure safe and reliable robotic operations. One solution to OA and JLA is to incorporate the involved constraints into a quadratic programming (QP), by solving which OA and JLA can be achieved. There exist a few non-iterative solvers such as zeroing neural networks (ZNNs), which can solve each samp...
-
Vision-based collision prediction for autonomous driving is a challenging task due to the dynamic movement of vehicles and diverse types of obstacles. Most existing methods rely on object detection algorithms, which only predict predefined collision targets, such as vehicles and pedestrians, and cannot anticipate emergencies caused by unknown obstacles. To address this limitation, we propose a nov...
-
DAMON leverages manifold learning and variational autoencoding to achieve obstacle avoidance, allowing for motion planning through adaptive graph traversal in a pre-learned low-dimensional hierarchically-structured manifold graph that captures intricate motion dynamics between a robotic arm and its obstacles. This versatile and reusable approach is applicable to various collaboration scenarios. Th...
-
This paper presents the gatekeeper algorithm, a real-time and computationally-lightweight method to ensure that nonlinear systems can operate safely in dynamic environments despite limited perception. gatekeeper integrates with existing path planners and feedback controllers by introducing an additional verification step that ensures that proposed trajectories can be executed safely, despite nonli...
-
We present a new approach for modeling avoidance constraints in 2D environments, in which waypoints are assigned to obstacle-free polyhedral regions. Constraints of this form are often formulated as mixed-integer programming (MIP) problems employing big-M techniques-however, these are generally not the strongest formulations possible with respect to the MIP's convex relaxation (so called ideal for...
-
This paper proposes a reachability-aware model predictive control with a discrete control barrier function for backward obstacle avoidance for a tractor-trailer system. The framework incorporates the state-variant reachable set obtained through sampling-based reachability analysis and symbolic regression into the objective function of model predictive control. By optimizing the intersection of the...
-
Optimization-based trajectory generation methods are widely used in whole-body planning for robots. However, existing work either oversimplifies the robot's geometry and environment representation, resulting in a conservative trajectory or suffers from a huge overhead in maintaining additional information such as the Signed Distance Field (SDF). To bridge the gap, we consider the robot as an impli...
-
For letting mobile robots travel flexibly through complicated environments, increasing attention has been paid to the whole-body collision evaluation. Most existing works either opt for the conservative corridor-based methods that impose strict requirements on the corridor generation, or ESDF-based methods that suffer from high computational overhead. It is still a great challenge to achieve fast ...
-
State-of-the-art multi-agent collision avoidance algorithms face limitations when applied to cluttered public environments, where obstacles may have a variety of shapes and structures. The issue arises because most of these algorithms are agent-level methods. They concentrate solely on preventing collisions between the agents while the obstacles are handled merely out-of-policy. Obstacle-aware pol...
-
This paper presents a novel modular robot system that can self-reconfigure to achieve omnidirectional movements for collaborative object transportation. Each robotic module is equipped with a steerable omni-wheel for navigation and is shaped as a regular icositetragon with a permanent magnet installed on each corner for stable docking. After aggregating multiple modules and forming a structure tha...
-
The aim of this research is to investigate the relationship between spinal flexion and quadruped locomotion in a rat robot equipped with a compliant spine, controlled by a central pattern generator (CPG). The study reveals that spinal flexion can enhance limb stride length, but it may also cause significant and unexpected motion disturbances during stride length variations. To address this issue, ...
-
In this paper, we present a general learning framework for controlling a quadruped robot that can mimic the behavior of real animals and traverse challenging terrains. Our method consists of two steps: an imitation learning step to learn from motions of real animals, and a terrain adaptation step to enable generalization to unseen terrains. We capture motions from a Labrador on various terrains to...
-
One can estimate the velocity and acceleration of robot manipulators by utilizing nonlinear observers. This involves combining inertial measurement units (IMUs) with the motor encoders of the robot through a model-based sensor fusion technique. This approach is lightweight, versatile (suitable for a wide range of trajectories and applications), and straightforward to implement. In order to further...
-
This paper develops a provably stable sensor-driven controller for path-following applications of robots with unicycle kinematics, one specific class of which is the wheeled mobile robot (WMR). The sensor measurement is converted to a scalar value (the score) through some mapping (the score function); the latter may be designed or learned. The score is then mapped to forward and angular velocities...
-
In sim-to-real Reinforcement Learning (RL), a policy is trained in a simulated environment and then deployed on the physical system. The main challenge of sim-to-real RL is to overcome the reality gap - the discrepancies between the real world and its simulated counterpart. Using generic geometric representations, such as convex decomposition, triangular mesh, signed distance field can improve sim...
-
Animals and insects showcase remarkably robust and adept navigational abilities, up to literally circumnavigating the globe. Primary progress in robotics inspired by these natural systems has occurred in two areas: highly theoretical computational neuroscience models, and handcrafted systems like RatSLAM and NeuroSLAM. In this research, we present work bridging the gap between the two, in the form...
-
This paper presents the design and control of a ballbot drivetrain that aims to achieve high agility, minimal footprint, and high payload capacity while maintaining dynamic stability. Two hardware platforms and analytical models were developed to test design and control methodologies. The full-scale ballbot prototype (MiaPURE) was constructed using off-the-shelf components and designed to have agi...
-
This paper addresses the lack of research on periodic reinforcement learning for physical robot control by presenting a 3-phase periodic Bayesian reinforcement learning method for uncertain environments. Drawing on cognition theory, the proposed approach achieves effective convergence with fewer training episodes. The coach-based demonstration phase narrows the search space and establishes a found...
-
The light and soft characteristics of Buoyancy Assisted Lightweight Legged Unit (BALLU) robots have a great potential to provide intrinsically safe interactions in environments involving humans, unlike many heavy and rigid robots. However, their unique and sensitive dynamics impose challenges to obtaining robust control policies in the real world. In this work, we demonstrate robust sim-to-real tr...
-
Differentiable Simulations have recently proven useful for various robotic manipulation tasks, including cloth manipulation. In robotic cloth simulation, it is crucial to maintain intersection-free properties. We present DiffClothAI, a differentiable cloth simulation with intersection-free friction contact and two-way coupling with articulated rigid bodies. DiffClothAI integrates the Project Dynam...
-
Modular Reconfigurable Robots (MRRs) represent an exciting path forward for industrial robotics, opening up new possibilities for robot design. Compared to monolithic manipulators, they promise greater flexibility, improved maintainability, and cost-efficiency. However, there is no tool or standardized way to model and simulate assemblies of modules in the same way it has been done for robotic man...
-
As robotic intelligence increases, so does the im-portance of agents that collect data from real-world environments. When learning in contact with the environment, one must consider how to minimize the impact on the environment and maintain reproducibility. To achieve this, the contact force with the environment must be reduced. One way to achieve this is to reduce the inertia of the arm. In this ...
-
The axes of robots and robot-like systems (RLS) usually include e-motor-gearbox-arrangements for optimal connection of the elements. The characteristics of the drive system and thus also of the robot depend strongly on the gears. Different gearbox designs are available which differ in stiffness, efficiency and further properties. For an application-optimal design of RLS a uniform documentation and...
-
Deployable structures provide adaptability and versatility for applications such as temporary architectures, space structures, and biomedical devices. Jamming is a mechanical phenomenon with which dramatic changes in stiffness can be achieved by increasing the frictional and kinematic coupling between constituents in a structure by applying an external pressure. This study applies jamming, which h...
-
Optimizing the morphologies and the controllers that adapt to various tasks is a critical issue in the field of robot design, aka. embodied intelligence. Previous works typically model it as a joint optimization problem and use search-based methods to find the optimal solution in the morphology space. However, they ignore the implicit knowledge of task-to-morphology mapping which can directly insp...
-
Effective design automation for building robots would make development faster and easier while also less prone to design errors. However, complex multi-domain constraints make creating such tools difficult. One persistent challenge in achieving this goal of design automation is the fundamental problem of component selection, an optimization problem where, given a general robot model, components mu...
-
This paper presents a haptic device with a simple architecture of only two limbs that can provide translational motion in three degrees of freedom (DOF) and one-DOF rotational motion. Actuation redundancy eliminates all forward-kinematic singularities and improves the motion-force transmission property. Thanks to the special structure of the kinematic chains, all actuators are close to the base an...
-
In this paper, we present a method to engage measurement uncertainties with the probabilistic robustness to one system uncertainty measure. Providing a metric indicating the potential occurrence of dangerous situations is highly essential for safety-critical robot applications. Due to the difficulty of finding a quantifiable, unambiguous representation however, such a metric has not been derived t...
-
Designing robots is a multiphase process aimed at solving a multi-criteria optimization problem to find the best possible detailed design. Generative design (GD) aims to accelerate the design process compared to manual design, since GD allows exploring and exploiting the vast design space more efficiently. In the field of robotics, however, relevant research focuses mostly on the generation of ful...
-
As robots become more prevalent, optimizing their design for better performance and efficiency is becoming increasingly important. However, current robot design practices overlook the impact of perception and design choices on a robot's learning capabilities. To address this gap, we propose a comprehensive methodology that accounts for the interplay between the robot's perception, hardware charact...
-
To facilitate the study of how passive leg stiffness influences locomotion dynamics and performance, we have developed an affordable and accessible 400 g quadruped robot driven by tunable compliant laminate legs, whose series and parallel stiffness can be easily adjusted; fabrication only takes 2.5 hours for all four legs. The robot can trot at 0.52 m/s or 4.4 body lengths per second with a 3.2 co...
-
In this paper we present a novel kinematic representation of a soft continuum robot to enable full shape estimation using a purely geometric solution. The kinematic representation involves using length varying piecewise constant curvature segments to describe the deformed shape of the robot. Based on this kinematic representation, we can use overlapping length sensors to estimate the shape of cont...
-
Soft robotics hold promise in the development of safe yet powered assistive wearable devices for infants. Key to this is the development of closed-loop controllers that can help regulate pneumatic pressure in the device's actuators in an effort to induce controlled motion at the user's limbs and be able to track different types of trajectories. This work develops a controller for soft pneumatic ac...
-
We present an implementation of an online op-timization algorithm for hitting a predefined target when returning ping-pong balls with a table tennis robot. The online algorithm optimizes over so-called interception policies, which define the manner in which the robot arm intercepts the ball. In our case, these are composed of the state of the robot arm (position and velocity) at interception time....
-
Deformable robots are notoriously difficult to model or control due to its high-dimensional configuration spaces. Direct trajectory optimization suffers from the curse-of-dimensionality and incurs a high computational cost, while learning-based controller optimization methods are sensitive to hyper-parameter tuning. To overcome these limitations, we hypothesize that high fidelity soft robots can b...
-
Soft fluidic actuators are becoming popular for their backdrivability, potential for high power density, and their support for power supply through flexible tubes. Control and design of such actuators requires serviceable models that describe how they relate fluid pressure and flow to mechanical force and motion. We present a simple 2-port model of a bellows actuator that accounts for the relation...
-
Continuum robots suffer large deflections due to internal and external forces. Accurate modeling of their passive compliance is necessary for accurate environmental interaction, especially in scenarios where direct force sensing is not practical. This paper focuses on deriving analytic formulations for the compliance of continuum robots that can be modeled as Kirchhoff rods. Compared to prior work...
-
Soft robots possess unique capabilities for adapting to the environment and interacting with it safely. However, their deformable nature also poses challenges for controlling their movement. In particular, the large deformations of a soft robot make it difficult to localize its individual body parts, which in turn impedes effective control. This paper introduces a novel localization framework desi...
-
Soft robots present unique capabilities, but have been limited by the lack of scalable technologies for construction and the complexity of algorithms for efficient control and motion. These depend on soft-body dynamics, high-dimensional actuation patterns, and external/onboard forces. This paper presents scalable methods and platforms to study the impact of weight distribution and actuation patter...
-
Soft robots are gaining popularity thanks to their intrinsic safety to contacts and adaptability. However, the potentially infinite number of Degrees of Freedom makes their modeling a daunting task, and in many cases only an approximated description is available. This challenge makes reinforcement learning (RL) based approaches inefficient when deployed on a realistic scenario, due to the large do...
-
Controlling soft continuum robotic arms is challenging due to their hyper-redundancy and dexterity. In this paper we experimentally demonstrate, for the first time, closed-loop control of the configuration space variables of a soft robotic arm, composed of independently controllable segments, using a Cosserat rod model of the robot and the distributed sensing and actuation capabilities of the segm...
-
In this paper, we tackle the control and trajectory planning problems for the cooperative transportation system of cable-suspended payload with multi Unmanned Aerial Vehicles (UAVs). Firstly, a payload controller is presented considering the dynamic coupling between the UAV and the payload to accomplish the active suppression of payload swing and the complex payload trajectory tracking. Secondly, ...
-
This paper presents a new control scheme for cooperative dual-arm robots manipulating heavy objects. The proposed method uses the full dynamical model of the kinematically coupled robot system and builds on a hierarchical quadratic programming (HQP) formulation to enforce dynamical inequality constraints such as joint torques or internal loads. This ensures optimal tracking of an object trajectory...
-
Efficient aerial data collection is important in many remote sensing applications. In large-scale monitoring scenarios, deploying a team of unmanned aerial vehicles (UAVs) offers improved spatial coverage and robustness against individual failures. However, a key challenge is cooperative path planning for the UAVs to efficiently achieve a joint mission goal. We propose a novel multi-agent informat...
-
Tracking multiple moving objects of interest (OOI) with multiple robot systems (MRS) has been addressed by active sensing that maintains a shared belief of OOIs and plans the motion of robots to maximize the information quality. Mobility of robots enables the behavior of pursuing better visibility, which is constrained by sensor field of view (FoV) and occlusion objects. We first extend prior work...
-
We consider a search problem where a robot has one or more types of sensors, each suited to detecting different types of targets or target information. Often, information in the form of a distribution of possible target locations, or locations of interest, may be available to guide the search. When multiple types of information exist, then a distribution for each type of information must also exis...
-
This paper studies a team coordination problem in a graph environment. Specifically, we incorporate “support” action which an agent can take to reduce the cost for its teammate to traverse some high cost edges. Due to this added feature, the graph traversal is no longer a standard multi-agent path planning problem. To solve this new problem, we propose a novel formulation that poses it as a planni...
-
Modeling a human-driven vehicle is a difficult subject since human drivers have a variety of stochastic behavioral components that influence their driving styles. We develop a cooperative driving framework to incorporate dif-ferent human behavior aspects, including the attentiveness of a driver and the tendency of the driver following advising commands. To demonstrate the framework, we consider th...
-
In applications such as search and rescue or disaster relief, heterogeneous multi-robot systems (MRS) can provide significant advantages for complex objectives that require a suite of capabilities. However, within these application spaces, communication is often unreliable, causing inefficiencies or outright failures to arise in most MRS algorithms. Many researchers tackle this problem by requirin...
-
Motion planning is challenging for multiple robots in cluttered environments without communication, especially in view of real-time efficiency, motion safety, distributed computation, and trajectory optimality, etc. In this paper, a reinforced potential field method is developed for distributed multi-robot motion planning, which is a synthesized design of reinforcement learning and artificial pote...
-
Using robots to collect data is an effective way to obtain information from the environment and communicate it to a static base station. Furthermore, robots have the capability to communicate with one another, potentially decreasing the time for data to reach the base station. We present a Mixed Integer Linear Program that reasons about discrete routing choices, continuous robot paths, and their e...
-
We present a coordination method for multiple mobile manipulators to sort objects in clutter. We consider the object rearrangement problem in which the objects must be sorted into different groups in a particular order. In clutter, the order constraints could not be easily satisfied since some objects occlude other objects so the occluded ones are not directly accessible to the robots. Those objec...
-
We present MOTLEE, a distributed mobile multi-object tracking algorithm that enables a team of robots to collaboratively track moving objects in the presence of localization error. Existing approaches to distributed tracking make limiting assumptions regarding the relative spatial relationship of sensors, including assuming a static sensor network or that perfect localization is available. Instead...
-
Exactly estimating and tracking the motion of surrounding dynamic objects is one of important tasks for the autonomy of a quadruped manipulator. However, with only an onboard RGB camera, it is still a challenging work for a quadruped manipulator to track the motion of a dynamic object moving with unknown and changing velocities. To address this problem, this manuscript proposes a novel image-based...
-
Legged robots leverage ground contacts and the reaction forces they provide to achieve agile locomotion. However, uncertainty coupled with contact discontinuities can lead to failure, especially in real-world environments with unexpected height variations such as rocky hills or curbs. To enable dynamic traversal of extreme terrain, this work introduces 1) a proprioception-based gait planner for es...
-
Quadrupedal robots are performing increasingly more real-world capabilities, but are primarily limited to locomotion tasks. To expand their task-level abilities of object acquisition, i.e., run-to-catch as frisbee catching for dogs, this paper developed a control pipeline using stereo vision for legged robots which allows for dynamic catching balls while the robot is in motion. To achieve high-fra...
-
Locomotion in the wild requires the quadruped robot to have strong capabilities in adaptation and robustness. The deep reinforcement learning (DRL) exhibits the huge potential in environmental adaptability, while its stability issues remain open. On the other hand, the quadruped robot dynamic model contains a lot of useful information that is beneficial to the robust control. The combination of DR...
-
Heavy quadrupedal drives have great potential for overcoming obstacles, showing great possibilities for transportation industries in complex environments. Ground reaction force (GRF) is a crucial state variable for quadrupedal control. Most GRF observations are implemented in lightweight quadrupeds, with little consideration of the loading being static or slippery on the body. However, the load in...
-
Offline evolutionary-based methodologies have supplied a successful motion planning framework for the quadrupedal jump. However, the time-consuming computation caused by massive population evolution in offline evolutionary-based jumping framework significantly limits the popularity in the quadrupedal field. This paper presents a time-friendly online motion planning framework based on meta-heuristi...
-
This paper presents a novel, low-cost proprioceptive sensing solution for legged robots with point feet to achieve accurate low-drift long-term position and velocity estimation. In addition to conventional sensors, including one body Inertial Measurement Unit (IMU) and joint encoders, we attach an additional IMU to each calf link of the robot just above the foot. An extended Kalman filter is used ...
-
Analytical expressions are derived for actuator demands in quadrupedal locomotion of constant speed and height by using a reduction from a trot/ pace 6-bar model to a single-legged model and employing two widely used two-segmented leg architectures, the serial and the parallel. A method is developed that outputs optimal gait characteristics and leg designs for a robot to move with maximum efficien...
-
Scientific exploration of planetary bodies is an activity well-suited for robots. Unfortunately, the regions that are richer in potential discoveries, such as impact craters, caves, and volcanic terraces, are hard to access with wheeled robots. Recent advances in legged-based approaches have shown the potential of the technology to overcome difficult terrains such as slopes and slippery surfaces. ...
-
Hybrid wheeled-legged quadrupeds have the potential to navigate challenging terrain with agility and speed and over long distances. However, obstacles can impede their progress by requiring the robots to either slow down to step over obstacles or modify their path to circumvent the obstacles. We propose a motion optimization framework for quadruped robots that incorporates non-steerable wheels and...
-
Legged robots are increasingly entering new domains and applications, including search and rescue, inspection, and logistics. However, for such a systems to be valuable in real-world scenarios, they must be able to autonomously and robustly navigate irregular terrains. In many cases, robots that are sold on the market do not provide such abilities, being able to perform only blind locomotion. Furt...
-
This paper proposes a new locomotion algorithm for truss robots on irregular terrain, in particular, for the Variable Topology Truss (VTT) system. The previous Polygon-based Random Tree (PRT) search algorithm for support polygon generation is extended to irregular terrain while considering friction and internal force limitations. By characterizing terrain, unreachable areas are excluded from searc...
-
Current motion planning approaches for autonomous mobile robots often assume that the low level controller of the system is able to track the planned motion with very high accuracy. In practice, however, tracking error can be affected by many factors, and could lead to potential collisions when the robot must traverse a cluttered environment. To address this problem, this paper proposes a novel re...
-
Autonomous Marine Vehicles (AMVs) have gained interest for scientific and commercial applications, including pipeline and algae bloom monitoring, contaminant tracking, and ocean debris removal. The Team Orienteering Problem (TOP) is relevant in this context as Multi-Robot Systems (MRSs) allow for better coverage of the area of interest, simultaneous data collection at different locations, and an i...
-
Search and rescue applications often need multiple agents to complete a set of conflicting tasks. This paper studies a Multi-Agent Multi-Objective Ergodic Search (MA-MO-ES) approach to this problem where each objective or task is to cover a domain subject to an information map. The goal is to allocate coverage tasks to agents so that all maps are explored ergodically. The combinatorial nature of t...
-
Multi-goal motion planning requires a robot to plan collision-free and dynamically-feasible motions to reach multiple goals, often in unstructured, obstacle-rich environments. This is challenging due to the complex dependencies between navigation and high-level reasoning, requiring the robot to explore a vast space of feasible motions and goal sequences. Our approach combines machine learning and ...
-
Neural networks have already demonstrated attractive performance for solving motion planning problems, especially in static and predictable environments. However, efficient neural planners that can adapt to unpredictable dynamic environments, a highly demanded scenario in many practical applications, are still under-explored. To fill this research gap and enrich the existing motion planning approa...
-
In existing task and motion planning (TAMP) research, it is a common assumption that experts manually specify the state space for task-level planning. A well-developed state space enables the desirable distribution of limited computational resources between task planning and motion planning. However, developing such task-level state spaces can be non-trivial in practice. In this paper, we consider...
-
To handle the two shortcomings of existing methods, (i) nearly all models rely on high-definition (HD) maps, yet the map information is not always available in real traffic scenes and HD map-building is expensive and time-consuming and (ii) existing models usually focus on improving prediction accuracy at the expense of reducing computing efficiency, yet the efficiency is crucial for various real ...
-
Game-theoretic motion planners are a powerful tool for the control of interactive multi-agent robot systems. Indeed, contrary to predict-then-plan paradigms, game-theoretic planners do not ignore the interactive nature of the problem, and simultaneously predict the behaviour of other agents while considering change in one's policy. This, however, comes at the expense of computational complexity, e...
-
In this paper, a comprehensive solution for enabling unmanned aerial vehicle (UAV) to autonomously fly through complex and dynamic environments is proposed. Moving objects all have unique property information, we propose a method that utilizes deep learning for 3D dynamic environment perception, while taking into account limitations in computing resources. For safer dynamic avoidance, we first mod...
-
6-DoF object-agnostic grasping in unstructured environments is a critical yet challenging task in robotics. Most current works use non-optimized approaches to sample grasp locations and learn spatial features without concerning the grasping task. This paper proposes GraNet, a graph-based grasp pose generation framework that translates a point cloud scene into multi-level graphs and propagates feat...
-
We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects, a task that requires fast, reactive, and accurately-timed robot motions. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii...
-
Language-Guided Robotic Manipulation (LGRM) is a challenging task as it requires a robot to understand human instructions to manipulate everyday objects. Recent approaches in LGRM rely on pre-trained Visual Grounding (VG) models to detect objects without adapting to manipulation environments. This results in a performance drop due to a substantial domain gap between the pre-training and real-world...
-
We introduce a practical robotics solution for the task of heterogeneous bagging, requiring the placement of multiple rigid and deformable objects into a deformable bag. This is a difficult task as it features complex interactions between multiple highly deformable objects under limited observability. To tackle these challenges, we propose a robotic system consisting of two learned policies: a rea...
-
In this work, we tackle the problem of 7-DoF grasping pose estimation(6-DoF with the opening width of parallel-jaw gripper) from point cloud data, which is a fundamental task in robotic manipulation. Most existing methods adopt 3D voxel CNNs as the backbone for their efficiency in handling unordered point cloud data. However, we found that these approaches overlook detailed information of the poin...
-
Robotic grasping faces new challenges in human-robot-interaction scenarios. We consider the task that the robot grasps a target object designated by human's language directives. The robot not only needs to locate a target based on vision-and-language information, but also needs to predict the reasonable grasp pose candidate at various views and postures. In this work, we propose a novel interactiv...
-
Pre-defined manipulation primitives are widely used for cloth manipulation. However, cloth properties such as its stiffness or density can highly impact the performance of these primitives. Although existing solutions have tackled the parameterisation of pick and place locations, the effect of factors such as the velocity or trajectory of quasi-static and dynamic manipulation primitives has been n...
-
Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR). While previous work mainly evaluates DR for disembodied tasks, such as pose...
-
Manipulating deformable objects, such as ropes (1D), fabrics (2D), and bags (3D), poses a significant challenge in robotics research due to their high degree of freedom in physical state and nonlinear dynamics. Compared with single-dimensional deformable objects, multi-dimensional object manipulation suffers from the difficulty in recognizing the characteristics of the object correctly and making ...
-
Adversarial object rearrangement in the real world (e.g., previously unseen or oversized items in kitchens and stores) could benefit from understanding task scenes, which inherently entail heterogeneous components such as current objects, goal objects, and environmental constraints. The semantic relationships among these components are distinct from each other and crucial for multi-skilled robots ...
-
To safely and efficiently extract an object from the clutter, this paper presents a bimanual manipulation planner in which one hand of the robot is used to slide the target object out of the clutter while the other hand is used to support the surrounding objects to prevent the clutter from collapsing. Our method uses a neural network to predict the physical phenomena of the clutter when the target...
-
In this work, we first formulate the problem of robotic water scooping using goal-conditioned reinforcement learning. This task is particularly challenging due to the complex dynamics of fluid and the need to achieve multi-modal goals. The policy is required to successfully reach both position goals and water amount goals, which leads to a large convoluted goal state space. To overcome these chall...
-
Operating unmanned aerial vehicles (UAVs) in complex environments that feature dynamic obstacles and external disturbances poses significant challenges, primarily due to the inherent uncertainty in such scenarios. Additionally, inaccurate robot localization and modeling errors further exacerbate these challenges. Recent research on UAV motion planning in static environments has been unable to cope...
-
It is crucial for hybrid unmanned aerial vehicles, such as lifting-wing multicopters, to plan a continuous, smooth, and collision-free trajectory to avoid obstacles. Unlike quad-copters, which typically work in indoor environments, lifting-wing multicopters typically fly at a high altitude with a high cruising speed, requiring higher maneuverability in the vertical direction. Inspired by birds, li...
-
Terrestrial and aerial bimodal vehicles have gained widespread attention due to their cross-domain maneuverability. Nevertheless, their bimodal dynamics significantly increase the complexity of motion planning and control, thus hindering robust and efficient autonomous navigation in unknown environments. To resolve this issue, we develop a model-based planning and control framework for terrestrial...
-
In recent years, there is a noteworthy advance-ment in autonomous drone racing. However, the primary focus is on attaining execution times, while scant attention is given to the challenges of dynamic environments. The high-speed nature of racing scenarios, coupled with the potential for unforeseeable environmental alterations, present stringent requirements for online replanning and its timeliness...
-
Drones have the potential to revolutionize power line inspection by increasing productivity, reducing inspection time, improving data quality, and eliminating the risks for human operators. Current state-of-the-art systems for power line inspection have two shortcomings: (i) control is decoupled from perception and needs accurate information about the location of the power lines and masts; (ii) ob...
-
Drilling, grinding, and setting anchors on vertical walls are fundamental processes in everyday construction work. Manually doing these works is error-prone, potentially dangerous, and elaborate at height. Today, heavy mobile ground robots can perform automatic power tool work. However, aerial vehicles could be deployed in untraversable environments and reach inaccessible places. Existing drone de...
-
High altitude balloons have proved useful for ecological aerial surveys, atmospheric monitoring, and communication relays. However, due to weight and power constraints, there is a need to investigate alternate modes of propulsion to navigate in the stratosphere. Very recently, reinforcement learning has been proposed as a control scheme to maintain balloons in the region of a fixed location, facil...
-
This paper proposes a sustainable planning system for small-sized unmanned aerial vehicles (UAVs). Our mapping module of the system uses a voxel array as data structure with an introduced feature which is local map origin update. This approach has clear advantages that the planning system can sustainably plan trajectories regardless of operating radius and flight distance, and it shows fastest inv...
-
Autonomous exploration is a crucial aspect of robotics that has numerous applications. Most of the existing methods greedily choose goals that maximize immediate reward. This strategy is computationally efficient but insufficient for overall exploration efficiency. In recent years, some state-of-the-art methods are proposed, which generate a global coverage path and significantly improve overall e...
-
In this paper, a new demonstration-based path-planning framework for the visual inspection of large structures using UAVs is proposed. We introduce UPPLIED: UAV Path PLanning for InspEction through Demonstration, which utilizes a demonstrated trajectory to generate a new trajectory to inspect other structures of the same kind. The demonstrated trajectory can inspect specific regions of the structu...
-
Instance segmentation is a fundamental skill for many robotic applications. We propose a self-supervised method that uses grasp interactions to collect segmentation supervision for an instance segmentation model. When a robot grasps an item, the mask of that grasped item can be inferred from the images of the scene before and after the grasp. Leveraging this insight, we learn a grasp segmentation ...
-
In many applications of advanced robotic manipulation, six degrees of freedom (6DoF) object pose estimates are continuously required. In this work, we develop a multi-modality tracker that fuses information from visual appearance and geometry to estimate object poses. The algorithm extends our previous method ICG, which uses geometry, to additionally consider surface appearance. In general, object...
-
Viewpoint planning is an important task in any application where objects or scenes need to be viewed from different angles to achieve sufficient coverage. The mapping of confined spaces such as shelves is an especially challenging task since objects occlude each other and the scene can only be observed from the front, posing limitations on the possible viewpoints. In this paper, we propose a deep ...
-
Accurately estimating the 6D pose of objects is crucial for many applications, such as robotic grasping, autonomous driving, and augmented reality. However, this task becomes more challenging in poor lighting conditions or when dealing with textureless objects. To address this issue, depth images are becoming an increasingly popular choice due to their invariance to a scene's appearance and the im...
-
Category-level object pose estimation involves estimating the 6D pose and the 3D metric size of objects from predetermined categories. While recent approaches take categorical shape prior information as reference to improve pose estimation accuracy, the single-stage network design and training manner lead to sub-optimal performance since there are two distinct tasks in the pipeline. In this paper,...
-
In reinforcement learning (RL), sparse rewards can present a significant challenge. Fortunately, expert actions can be utilized to overcome this issue. However, acquiring explicit expert actions can be costly, and expert observations are often more readily available. This paper presents a new approach that uses expert observations for learning in robot manipulation tasks with sparse rewards from p...
-
Reinforcement learning is used to tackle complex tasks with high-dimensional sensory inputs. Over the past decade, a wide range of reinforcement learning algorithms have been developed, with recent progress benefiting from deep learning for raw sensory signal representation. This raises a natural question: how well do these algorithms perform across different robotic manipulation tasks? To objecti...
-
Shape completion, i.e., predicting the complete geometry of an object from a partial observation, is highly relevant for several downstream tasks, most notably robotic manipulation. When basing planning or prediction of real grasps on object shape reconstruction, an indication of severe geometric uncertainty is indispensable. In particular, there can be an irreducible uncertainty in extended regio...
-
We introduce Structure from Action (SfA), a framework to discover 3D part geometry and joint parameters of unseen articulated objects via a sequence of inferred interactions. Our key insight is that 3D interaction and perception should be considered in conjunction to construct 3D articulated CAD models, especially for categories not seen during training. By selecting informative interactions, Sf A...
-
Domestic service robots are becoming increasingly popular due to their ability to help people with household tasks. These robots often encounter the challenge of manipulating objects in cluttered environments (MoC), which is difficult due to the complexity of effective planning and control. Previous solutions involved designing specific action primitives and planning paradigms. However, the pre-co...
-
Weakly supervised referring expression grounding (WREG) is an attractive and challenging task for grounding target regions in images by understanding given referring expressions. WREG learns to ground target objects without the manual annotations between image regions and referring expressions during the model training phase. Different from the predominant grounding pattern of existing models, whi...
-
Recognizing and comprehending human actions and gestures is a crucial perception requirement for robots to interact with humans and carry out tasks in diverse domains, including service robotics, healthcare, and manufacturing. Event cameras, with their ability to capture fast-moving objects at a high temporal resolution, offer new opportunities compared to standard action recognition in RGB videos...
-
This paper proposes a ski training system using VR (Virtual Reality) that enables beginners to acquire skiing skills without going to an actual ski ground. The proposed system obtains the speed of skiing based on the center of pressure (COP) of each player's foot. The first-person perspective of skiing at the obtained speed down a ski slope is fed back to the player as a VR image. Experiments were...
-
Facial animation in virtual reality environments is essential for applications that necessitate clear visibility of the user's face and the ability to convey emotional signals. In our scenario, we animate the face of an operator who controls a robotic Avatar system. The use of facial animation is particularly valuable when the perception of interacting with a specific individual, rather than just ...
-
Affordable 3D scanners often produce sparse and non-uniform point clouds that negatively impact downstream applications in robotic systems. While existing point cloud upsampling architectures have demonstrated promising results on standard benchmarks, they tend to experience significant performance drops when the test data have different distributions from the training data. To address this issue,...
-
Dynamic vision sensors or event cameras provide rich complementary information for video frame interpolation. Existing state-of-the-art methods follow the paradigm of combining both synthesis-based and warping networks. However, few of those methods fully respect the intrinsic characteristics of events streams. Given that event cameras only encode intensity changes and polarity rather than color i...
-
Depth completion, which aims to generate high-quality dense depth maps from sparse depth maps, has attracted increasing attention in recent years. Previous work usually employs RGB images as guidance, and introduces iterative spatial propagation to refine estimated coarse depth maps. However, most of the propagation refinement methods require several iterations and suffer from a fixed receptive fi...
-
Gestures form an important medium of communication between humans and machines. An overwhelming majority of existing gesture recognition methods are tailored to a scenario where humans and machines are located very close to each other. This short-distance assumption does not hold true for several types of interactions, for example gesture-based interactions with a floor cleaning robot or with a dr...
-
Recently, groundbreaking results have been presented on open-vocabulary semantic image segmentation. Such methods segment each pixel in an image into arbitrary categories provided at run-time in the form of text prompts, as opposed to a fixed set of classes defined at training time. In this work, we present a zero-shot volumetric open-vocabulary semantic scene segmentation method. Our method build...
-
Given a single RGB image and the attribute-rich language instructions, this paper investigates the novel problem of using Fine-grained instructions for the Language guided robotic Grasping (FLarG). This problem is made challenging by learning fine-grained language descriptions to ground target objects. Recent advances have been made in visually grounding the objects simply by several coarse attrib...
-
This study presents a method for estimating the three-dimensional (3D) shapes of transparent objects from an RGB-D image using a statistical shape model. Statistical shape models compress the dimensions of multiple shapes to represent shape variations using fewer parameters. It is difficult to measure the depth of a transparent object using sensors. Therefore, the statistical shape model is deform...
-
Localisation with Frequency-Modulated Continuous-Wave (FMCW) radar has gained increasing interest due to its inherent resistance to challenging environments. However, complex artefacts of the radar measurement process require appropriate uncertainty estimation - to ensure the safe and reliable application of this promising sensor modality. In this work, we propose a multi-session map management sy...
-
We present a novel framework for global localization and guided relocalization of a vehicle in an unstructured environment. Compared to existing methods, our pipeline does not rely on cues from urban fixtures (e.g., lane markings, buildings), nor does it make assumptions that require the vehicle to be navigating on a road network. Instead, we achieve localization in both urban and non-urban enviro...
-
Object-based maps are relevant for scene under-standing since they integrate geometric and semantic information of the environment, allowing autonomous robots to robustly localize and interact with on objects. In this paper, we address the task of constructing a metric-semantic map for the purpose of long-term object-based localization. We exploit 3D object detections from monocular RGB frames for...
-
Can knowing where you are assist in perceiving objects in your surroundings, especially under adverse weather and lighting conditions? In this work we investigate whether a prior map can be leveraged to aid in the detection of dynamic objects in a scene without the need for a 3D map or pixel-level map-query correspondences. We contribute an algorithm which refines an initial set of candidate objec...
-
Visual place recognition is essential for vision-based robot localization and SLAM. Despite the tremendous progress made in recent years, place recognition in changing environments remains challenging. A promising approach to cope with appearance variations is to leverage high-level semantic features like objects or place categories. In this paper, we propose FM-Loc which is a novel image-based lo...
-
Aligning a robot's trajectory or map to the inertial frame is a critical capability that is often difficult to do accurately even though inertial measurement units (IMUs) can observe absolute roll and pitch with respect to gravity. Accelerometer biases and scale factor errors from the IMU's initial calibration are often the major source of inaccuracies when aligning the robot's odometry frame with...
-
Place recognition is an important technique for autonomous cars to achieve full autonomy since it can provide an initial guess to online localization algorithms. Although current methods based on images or point clouds have achieved satisfactory performance, localizing the images on a large-scale point cloud map remains a fairly unexplored problem. This cross-modal matching task is challenging due...
-
As a crucial infrastructure of intelligent mobile robots, LiDAR-Inertial odometry (LIO) provides the basic capability of state estimation by tracking LiDAR scans. The high-accuracy tracking generally involves the $k\text{NN}$ search, which is used with minimizing the point-to-plane distance. The cost for this, however, is maintaining a large local map and performing $k\text{NN}$ plane fit for each...
-
Visual-inertial initialization can be classified into joint and disjoint approaches. Joint approaches tackle both the visual and the inertial parameters together by aligning observations from feature-bearing points based on IMU integration then use a closed-form solution with visual and acceleration observations to find initial velocity and gravity. In contrast, disjoint approaches independently s...
-
In the face of complex external environment, single sensor information can no longer meet the accuracy requirements of low-drift SLAM. In this paper, we focus on the fusion scheme of cameras and lidar, and explore the gain of semantic information to SLAM system. A Semantic-Enhanced Lidar-Visual Odometry (SELVO) is proposed to achieve pose estimation with high accuracy and robustness by applying se...
-
LiDAR-inertial odometry (LIO), which fuses complementary information of a LiDAR and an Inertial Measurement Unit (IMU), is an attractive solution for state estimation. In LIO, both pose and velocity are regarded as state variables that need to be solved. However, the widely-used Iterative Closest Point (ICP) algorithm can only provide constraint for pose, while the velocity can only be constrained...
-
The data fusion of camera, IMU, and wheel encoder measurements has proved its effectiveness in localizing ground robots, and obtaining accurate sensor extrinsic parameters is its premise. We propose an extrinsic parameter calibration algorithm and a multi-sensor-based pose estimation algorithm for the camera-IMU-wheel encoder system. First, we propose a joint calibration algorithm for the extrinsi...
-
This paper proposes a LiDAR-Inertial SLAM with efficiently extracted planes, which couples explicit planes in the odometry to improve accuracy and in the mapping for consistency. The proposed method consists of three parts: an efficient Point $\boldsymbol{\rightarrow\text{Line}\rightarrow \text{Plane}}$ extraction algorithm, a LiDAR-Inertial-Plane tightly coupled odometry, and a global plane-aided...
-
Using visual SLAM to map new environments requires time-consuming visits to all regions for data collection. We propose an approach to estimate maps of areas beyond the visible regions using a cheap and readily available modality of data-sound. We introduce the idea of an active audio-visual mapping agent. Besides collecting visual data, the proposed agent emits sounds during navigation, captures ...
-
As today's smartphone integrates various imaging sensors and Inertial Measurement Units (IMU) and becomes computationally powerful, there is a growing interest in developing smartphone-based visual-inertial (VI) SLAM methods for robotics and computer vision applications. In this paper, we introduce a new SLAM method, called Visual-LiDAR-Inertial Odometry (VLIO), based on an iPhone 12 Pro. VLIO for...
-
In this work, we present a comprehensive analysis of the application of the First-estimates Jacobian (FEJ) design methodology in nonlinear optimization-based Visual-Inertial Navigation Systems (VINS). The FEJ approach fixes system linearization points to preserve proper observability properties of VINS and has been shown to significantly improve the estimation performance of state-of-the-art filte...
-
Robotic solutions for pipeline inspection promise enhancement of human labor by automating data acquisition for pipe condition assessments, which are vital for the early detection of pipe anomalies and the prevention of hazardous leakages and explosions. Through simultaneous localization and mapping (SLAM), colorized 3D reconstructions of the pipe's inner surface can be generated, providing a more...
-
Modern autonomous driving systems face substantial challenges when navigating dense intersections due to the high uncertainty introduced by other road users. Due to the complexity of the task, the autonomous vehicle needs to generate policies at multiple levels of abstraction. However, previous deep imitation learning methods focused on learning control policies while using simple rule-based behav...
-
In this paper we show how to improve the performance of backward chained behavior trees (BTs) that include policies trained with reinforcement learning (RL). BTs represent a hierarchical and modular way of combining control policies into higher level control policies. Backward chaining is a design principle for the construction of BTs that combines reactivity with goal directed actions in a struct...
-
Urban Air Mobility (UAM) promises a new dimension to decongested, safe, and fast travel in urban and suburban hubs. These UAM aircraft are conceived to operate from small airports called vertiports each comprising multiple take-offllanding and battery-recharging spots. Since they might be situated in dense urban areas and need to handle many aircraft landings and take-offs each hour, managing this...
-
On end-to-end driving, human driving demonstrations are used to train perception-based driving models by imitation learning. This process is supervised on vehicle signals (e.g., steering angle, acceleration) but does not require extra costly supervision (human labeling of sensor data). As a representative of such vision-based end-to-end driving models, CILRS is commonly used as a baseline to compa...
-
Mobile robotic agents often suffer from localization uncertainty which grows with time and with the agents' movement. This can hinder their ability to accomplish their task. In some settings, it may be possible to perform assistive actions that reduce uncertainty about a robot's location. For example, in a collaborative multi-robot system, a wheeled robot can request assistance from a drone that c...
-
Trajectory prediction of neighboring agents is a critical task for high-speed robotics such as autonomous vehicles. In order to obtain fine-grained and robust scene representations, existing works attempt to consider abundant information that is deemed relevant. The cost, however, is the heavy computational burden and more importantly the inevitable interference brought by redundant information. I...
-
Autonomous vehicle (AV) evaluation has been the subject of increased interest in recent years both in industry and in academia. This paper focuses on the development of a novel framework for generating adversarial driving behavior of background vehicle interfering against the AV to expose effective and rational risky events. Specifically, the adversarial behavior is learned by a reinforcement lear...
-
Robotic manipulation tasks rely on a plethora of environmental and payload information. One critical piece of information for accurate manipulation is the center of mass (CoM) of the object, which is essential for estimating the dynamic response of the system and determining the payload placement. Traditionally, the CoM of a payload is provided prior to manipulation. In order to create a more robu...
-
In this study, we propose a novel approach for investigating optimization performance by flexible robot co-ordination in automated warehouses with multi-agent rein-forcement learning (MARL)-based control. Automated systems using robots are expected to achieve efficient operations compared with manual systems in terms of overall optimization performance. However, the impact of overall optimization ...
-
In autonomous racing, accurately tracking the race line at the limits of handling is essential to guarantee competitiveness. In this study, we show the effectiveness of Differential Flatness based control for high-speed trajectory tracking for car-like robots. We compare the tracking performance of our controller against Nonlinear Model Predictive Control and resource use while running on embedded...
-
We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings into the early stages of a 3D object detector. This feature fusion strategy enables the model to better capture the shapes and poses for challenging objects, compared with learning from raw points directly. Our method con...
-
Sequence models in reinforcement learning require task knowledge to estimate the task policy. This paper presents the hierarchical decision transformer (HDT). HDT is a hierarchical behavior cloning algorithm that improves the performance of transformer methods in imitation learning, improving their robustness to tasks with longer episodes and/or sparse rewards, without requiring task knowledge or ...
-
Learning policies from multiple demonstrators is often difficult because different individuals perform the same task differently due to hidden factors such as preferences. In the context of policy learning, this leads to multimodal policies. Existing policy learning methods often converge to a single solution mode, failing to capture the diversity in the solution space. In this paper, we introduce...
-
Reinforcement learning (RL) can potentially be applied to real-world robot control in complex and uncertain environments. However, it is difficult or even unpractical to design an efficient reward function for various tasks, especially those large and high-dimensional environments. Generative adversarial imitation learning (GAIL) - a general model-free imitation learning method, allows robots to d...
-
Learning-based approaches have achieved remarkable performance in the domain of autonomous driving. Leveraging the impressive ability of neural networks and large amounts of human driving data, complex patterns and rules of driving behavior can be encoded as a model to benefit the autonomous driving system. Besides, an increasing number of data-driven works have been studied in the decision-making...
-
Many applications of imitation learning require the agent to generate the full distribution of behaviour observed in the training data. For example, to evaluate the safety of autonomous vehicles in simulation, accurate and diverse behaviour models of other road users are paramount. Existing methods that improve this distributional realism typically rely on hierarchical policies. These condition th...
-
The deployment of agile autonomous systems in challenging, unstructured environments requires adaptation capabilities and robustness to uncertainties. Existing robust and adaptive controllers, such as those based on model predictive control (MPC), can achieve impressive performance at the cost of heavy online onboard computations. Strategies that efficiently learn robust and onboard-deployable pol...
-
Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
Multimodal demonstrations provide robots with an abundance of information to make sense of the world. However, such abundance may not always lead to good performance when it comes to learning sensorimotor control policies from human demonstrations. Extraneous data modalities can lead to state over-specification, where the state contains modalities that are not only useless for decision-making but ...
-
In this paper we investigate the effect of the unpredictability of surrounding cars on an ego-car performing a driving maneuver. We use Maximum Entropy Inverse reinforcement Learning to model reward functions for an ego-car conducting a lane change in a highway setting. We define a new feature based on the unpredictability of surrounding cars and use it in the reward function. We learn two reward ...
-
The current learning pipelines for robotics manipulation infer movement primitives sequentially along the temporal-evolving axis, which can result in an accumulation of prediction errors and subsequently cause the visual observations to fall out of the training distribution. This paper proposes a novel hierarchical behavior cloning approach which tries to dissociate standard behaviour cloning (BC)...
-
Visual-Inertial (VI) sensors are popular in robotics, self-driving vehicles, and augmented and virtual reality applications. In order to use them for any computer vision or state-estimation task, a good calibration is essential. However, collecting informative calibration data in order to render the calibration parameters observable is not trivial for a non-expert. In this work, we introduce a nov...
-
The use of serial robots for industrial and research purposes is often limited by a flawed positioning accuracy, caused by the differences between the robot nominal model, and the real one. Such an issue can be solved by means of kinematic calibration, which is usually a tedious and intricate task. In this paper, we propose a complete kinematic calibration procedure relying on established geometri...
-
Modern unmanned aerial vehicles (UAVs) with sophisticated mechanics ask for extended online system identification to aid model-based controls in task execution. In addition, UAVs in adverse environmental conditions require a more detailed environmental disturbance understanding. The necessary combination of online system identification, sensor suite self-calibration, and external disturbance analy...
-
Calibration is the first and foremost step in dealing with sensor displacement errors that can appear during extended operation and off-time periods to enable robot object manipulation with precision. In this paper, we present a novel multiplanar self-calibration between the camera system and the robot's end-effector for 3D object manipulation. Our approach first takes the robot end-effector as gr...
-
Compliance with global guidelines for sustainable and responsible production in modern industry requires a comparative analysis of consumer devices' energy consumption (EC). This also holds true for the newly established generation of lightweight industrial robots (LIRs). To identify potential strategies for energy optimization, standardized benchmarking procedures are required. However, to the be...
-
Robotic manipulators deal with serious issues due to their absolute positioning error. This error is usually compensated by an operator in classical robot programming using the teach-and-play method. However, it has a significant effect on accuracy of robotic guidance systems (RGS) that automatically generate process tool trajectory based on the measured data from a sensor. In this paper, we first...
-
Kinematic calibration is crucial to improve the positioning accuracy of serial robots. This paper proposes a novel algorithm for robotic kinematic calibration based on an augmented product of exponentials (POE)-based kinematic model using Gaussian mixture models (GMMs) with only position data. In this algorithm, non-geometric errors that cannot be fitted by varying the parameters within the tradit...
-
With the recent advances in autonomous driving and the decreasing cost of LiDARs, the use of multimodal sensor systems is on the rise. However, in order to make use of the information provided by a variety of complimentary sensors, it is necessary to accurately calibrate them. We take advantage of recent advances in computer graphics and implicit volumetric scene representation to tackle the probl...
-
Many intelligent robots use a combination of radar and camera sensors to capture environmental information. Robust and accurate perception highly relies on the result of multi-sensor calibration. Most current spatial calibration methods require a calibration board or a special marker as the target. In this paper, we provide a novel calibration method for RGBD camera and millimeter-wave radar, whic...
-
Multi-modal sensing often involves determining correspondences between each domain's signals, which in turn depends on the accurate extrinsic calibration of the sensors. Challengingly, the camera-LIDAR sensor modalities are quite dissimilar and the narrow field of view of most commercial LIDARs means that they observe only a partial view of the camera frustum. We present a framework for extrinsic ...
-
Ease of calibration and high-accuracy task-space state-estimation purely based on onboard sensors is a key requirement for enabling easily deployable cable robots in real-world applications. In this work, we incorporate the onboard camera and kinematic sensors to drive a statistical fusion framework that presents a unified localization and calibration system which requires no initial values for th...
-
The accurate and robust calibration result of sensors is considered as an important building block to the follow-up research in the autonomous driving and robotics domain. The current works involving extrinsic calibration between 3D LiDARs and monocular cameras mainly focus on target-based and target-less methods. The target-based methods are often utilized offline because of restrictions, such as...
-
This paper proposes an external wrench estimation method for modular manipulators, where each link module is driven with external actuation (e.g., rotors, thrusters) and inter-module joints can be locked to increase end-effector stiffness or workforce of the manipulator. For such systems, the commonly-used momentum-based observer (MBO [1]) is not suitable due to the presence of unknown joint locki...
-
In this study, we introduce AV-PedAware, a self-supervised audio-visual fusion system designed to improve dynamic pedestrian awareness for robotics applications. Pedestrian awareness is a critical requirement in many robotics applications. However, traditional approaches that rely on cameras and LIDARs to cover multiple views can be expensive and susceptible to issues such as changes in illuminati...
-
Pushing is an essential motor skill involved in several manipulation tasks, and has been an important research topic in robotics. Recent works have shown that Deep Q-Networks (DQNs) can learn pushing policies (when, where to push, and how) to solve manipulation tasks, potentially in synergy with other skills (e.g. grasping). Nevertheless, DQNs often assume a fixed setting and task, which may limit...
-
The performance of local feature descriptors degrades in the presence of large rotation variations. To address this issue, we present an efficient approach to learning rotation invariant descriptors. Specifically, we propose Rotated Kernel Fusion (RKF) which imposes rotations on the convolution kernel to improve the inherent nature of CNN. Since RKF can be processed by the subsequent re-parameteri...
-
The robot exploration task has been widely studied with applications spanning from novel environment mapping to item delivery. For some time-critical tasks, such as rescue catastrophes, the agent is required to explore as efficiently as possible. Recently, Visit Frequency-based map representation achieved great success in such scenarios by discouraging repetitive visits with a frequency-based pena...
-
Social group detection is a crucial aspect of various robotic applications, including robot navigation and human-robot interactions. To date, a range of model-based techniques have been employed to address this challenge, such as the F-formation and trajectory similarity frameworks. However, these approaches often fail to provide reliable results in crowded and dynamic scenarios. Recent advancemen...
-
The irregularity and permutation invariance of point cloud data pose challenges for effective learning. Conventional methods for addressing this issue involve converting raw point clouds to intermediate representations such as 3D voxel grids or range images. While such intermediate representations solve the problem of permutation invariance, they can result in significant loss of information. Appr...
-
Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior d...
-
Solving real-world manipulation tasks requires robots to be equipped with a repertoire of skills that can be applied to diverse scenarios. While learning-based methods can enable robots to acquire skills from interaction data, their success relies on collecting training data that covers the diverse range of tasks that the robot may encounter during the test time. However, creating diverse and feas...
-
Autonomous vehicles and robots need to operate over a wide variety of scenarios in order to complete tasks efficiently and safely. Multi-camera self-supervised monocular depth estimation from videos is a promising way to reason about the environment, as it generates metrically scaled geometric predictions from visual data without requiring additional sensors. However, most works assume well-calibr...
-
Deep learning methods are widely used in robotic applications. By learning from prior experience, the robot can abstract knowledge of the environment, and use this knowledge to accomplish different goals, such as object search, frontier exploration, or scene understanding, with a smaller amount of resources than might be needed without that knowledge. Most existing methods typically require a sign...
-
This paper introduces a novel state estimation framework for robots using differentiable ensemble Kalman filters (DEnKF). DEnKF is a reformulation of the traditional ensemble Kalman filter that employs stochastic neural networks to model the process noise implicitly. Our work is an extension of previous research on differentiable filters, which has provided a strong foundation for our modular and ...
-
Cutting objects into desired fragments is challenging for robots due to the spatially unstructured nature of fragments and the complex one-to-many object fragmentation caused by actions. We present a novel approach to model object fragmentation using an attributed stochastic grammar. This grammar abstracts fragment states as node variables and captures causal transitions in object fragmentation th...
-
In this paper, we perform an object rearrangement task for target retrieval in an environment with a confined space and limited observation directions. The agent must create a collision-free path to bring out the target object by relocating the surrounding objects using the prehensile action, i.e., pick-and-place. Object rearrangement in a confined space is a non-monotone problem, and finding a va...
-
Symbolic planning is a powerful technique to solve complex tasks that require long sequences of actions and can equip an intelligent agent with complex behavior. The downside of this approach is the necessity for suitable symbolic representations describing the state of the environment as well as the actions that can change it. Traditionally such representations are carefully hand-designed by expe...
-
Robots operating in real-world environments must reason about possible outcomes of stochastic actions and make decisions based on partial observations of the true world state. A major challenge for making accurate and robust action predictions is the problem of confounding, which if left untreated can lead to prediction errors. The partially observable Markov decision process (POMDP) is a widely-u...
-
Many planning problems in robotics require long planning horizon and uncertain in nature. The Par-tially Observable Markov Descision Process (POMDP) is a mathematically principled framework for planning under uncertainty. To alleviate the difficulties of computing good approximate POMDP solutions for long horizon problems, one often plans using macro actions, where each macro action is a chain of ...
-
This paper proposes a task planning and motion control framework that generates task plans for a linear temporal logic specification (LTL), which are then executed using a task-space constrained motion controller and a local task planner that overcomes local minima. We propose a new encoding for task specifications, directly in the task-space, as constraints of a mixed-integer linear program that ...
-
In this paper, we address task and motion plan-ning (TAMP) which is an important yet challenging robotics problem. It is known to suffer from the high combinatorial complexity of discrete search, often requiring a large number of geometric planning calls. We build upon recent works in TAMP by taking advantage of learning methods to provide action feasibility information as a heuristic to the symbo...
-
Task and motion planning is one of the key problems in robotics today. It is often formulated as a discrete task allocation problem combined with continuous motion planning. Many existing approaches to TAMP involve explicit descriptions of task primitives that cause discrete changes in the kinematic relationship between the actor and the objects. In this work we propose an alternative, fully diffe...
-
Effectively rearranging heterogeneous objects constitutes a high-utility skill that an intelligent robot should master. Whereas significant work has been devoted to the grasp synthesis of heterogeneous objects, little attention has been given to the planning for sequentially manipulating such objects. In this work, we examine the long-horizon sequential rearrangement of heterogeneous objects in a ...
-
Robust mission planning is an essential component for mission autonomy to perform complicated tasks in extreme environments. In this paper, we are interested in the role of semantic abstractions for guiding autonomous mission planning. In particular, we focus on how semantics can be leveraged to transition, at the mission level, in between individually robust task plans. We present a mission auton...
-
Autonomous robots are increasingly utilized in realistic scenarios with multiple complex tasks. In these scenarios, there may be a preferred way of completing the tasks, but it is often in conflict with optimal execution. Recent work studies preference-based planning, however, they have yet to extend the notion of preference to the behavior of the robot with respect to each task. In this work, we ...
-
Object rearrangement is a fundamental sub-task in accomplishing a great many physical tasks. As such, effectively executing rearrangement is an important skill for intelligent robots to master. In this study, we conduct the first algorithmic study on optimally solving the problem of Multi-layer Object Rearrangement on a Tabletop (MORT), in which one object may be relocated at a time, and an object...
-
Multi-object rearrangement is a crucial skill for service robots, and commonsense reasoning is frequently needed in this process. However, achieving commonsense arrangements requires knowledge about objects, which is hard to transfer to robots. Large language models (LLMs) are one potential source of this knowledge, but they do not naively capture information about plausible physical arrangements ...
-
Robotic knee-ankle prostheses have often fallen short relative to passive microprocessor prostheses in time-based clinical outcome tests. User ambulation endurance is an alternative clinical outcome metric that may better highlight the benefits of robotic prostheses. However, previous studies were unable to show endurance benefits due to inaccurate high-level classification, discretized mid-level ...
-
One of the primary benefits of emerging powered prosthetic legs is their ability to facilitate step-over-step stair ascent by providing positive mechanical work. Existing control methods typically have distinct steady-state activity modes for walking and stair ascent, where activity transitions involve discretely switching between controllers and often must be initiated with a particular leg. Howe...
-
Phase variable control based on global tibia kinematics holds promise for predicting gait cycle progression to continuously control robotic transtibial prostheses. Calibration of the phase variable is critical to ensure its monotonic behavior, to approach a linear relationship with gait percentage, and to accurately predict the percentage of gait. This paper compares four calibration approaches us...
-
Substantial research and development on the design and control of robotic ankle-foot prostheses have aimed to restore normal function and movement capacity for people with gait impairments and lower limb amputations. However, prostheses controllers usually fail to incorporate information pertaining to the properties of the walking terrain, such as ground stiffness. There is therefore a need for a ...
-
Ambulation in everyday life requires walking at variable speeds, variable inclines, and variable terrains. Powered prostheses aim to provide this adaptability through control of the actuated joints. Some powered prosthesis controllers can adapt to discrete changes in speed and incline but require manual tuning to determine the control parameters, leading to poor clinical viability. Other data-driv...
-
While motion classification architectures have improved in accuracy and robustness in recent years, computationally expensive approaches and sophisticated hardware dependencies limit their real-world applicability. To overcome these challenges, we have designed a lightweight, realtime architecture for classifying motions of the arm & hand using features derived from motor unit action potentials wi...
-
Despite significant advances in the design of robotic lower-limb prostheses for individuals with impaired mobility, there is a need for further progress in improving the robustness, safety, and stability of these devices in a wide range of activities of daily living. Although powered prostheses have been able to adapt to different speeds, conditions, and rigid terrains, no control strategies have ...
-
Powered lower-limb prostheses have the potential to improve amputee mobility by closely imitating the biomechanical function of the missing biological leg. To accomplish this goal, powered prostheses need controllers that can seamlessly adapt to the ambulation activity intended by the user. Most powered prosthesis control architectures address this issue by switching between specific controllers f...
-
Existing controllers for robotic powered prostheses regulate the prosthesis speed, timing, and energy generation using predefined position or torque trajectories. This approach enables climbing stairs step-over-step. However, it does not provide amputees with direct volitional control of the robotic prosthesis, a functionality necessary to restore full mobility to the user. Here we show that propo...
-
Robotic prosthetic legs have the potential to significantly improve the quality of life for lower limb amputees to perform locomotion in various environments and task conditions. However, these devices lack the capability to recover from internal intrinsic control faults, which can lead to harmful consequences affecting the user's gait performance and eroding trust in these robotic devices. Theref...
-
We introduce a multi-modal reconfiguration planner for the Variable Topology Truss (VTT) modular robot system. The VTT system is a truss-architecture modular self-reconfigurable robot. When a VTT is restricted to a single topology, the collision constraints between the truss members divide the configuration space into many connected components, which makes collision-free planning difficult. This n...
-
Fast and accurate path planning is important for ground robots to achieve safe and efficient autonomous navigation in unstructured outdoor environments. However, most existing methods exploiting either 2D or 2.5D maps struggle to balance the efficiency and safety for ground robots navigating in such challenging scenarios. In this paper, we propose a novel hybrid map representation by fusing a 2D g...
-
The Dijkstra algorithm is a classic path planning method, which in a discrete graph space, can start from a specified source node and find the shortest path between the source node and all other nodes in the graph. However, to the best of our knowledge, there is no effective method that achieves a function similar to that of the Dijkstra's algorithm in a continuous space. In this study, an optimal...
-
This paper examines a pursuit-evasion game (PEG) involving multiple pursuers and evaders. The decentralized pursuers aim to collaborate to capture the faster evaders while avoiding collisions. The policies of all agents are learning-based and are subjected to kinematic constraints that are specific to unicycles. To address the challenge of high dimensionality encountered in large-scale scenarios, ...
-
This work proposes a novel approach to social robot navigation by learning to generate robot controls from a social motion latent space. By leveraging this social motion latent space, the proposed method achieves significant improvements in social navigation metrics such as success rate, navigation time, and trajectory length while producing smoother (less jerk and angular deviations) and more ant...
-
We present an algorithm for safe robot navigation in complex dynamic environments using a variant of model predictive equilibrium point control. We use an optimization formulation to navigate robots gracefully in dynamic environments by optimizing over a trajectory cost function at each timestep. We present a novel trajectory cost formulation that significantly reduces conservative and deadlocking...
-
Robots need to be as simple to use as tools in a workshop and allow non-experts to program, modify and execute tasks. In particular for repetitive tasks in high-mix/low-volume production, robotic support and physical human-robot interaction (pHRI) help to significantly increase productivity. In path-following control (PFC), the geometric description of the path is decoupled from the time evolution...
-
This paper presents a design and characterization of a micropositioning stage driven by a reluctance actuator. The stage is constructed with a C-core reluctance actuator and four compression springs. The design of the stage is presented using a CAD model, followed by the fabrication process of the prototype. The mathematical model is formulated to present the interaction among the stage's electric...
-
Terrestrial mobile robots face diverse topographies while in field missions. Rough terrains cause the platform to oscillate, which is undesirable for some tasks. Robotic platforms with active tracked flippers can use such mechanisms to reach and maintain a leveled configuration while halted or moving. Thus, this work presents a posture controller that regulates the robot's orientation and contact ...
-
The study of motion control for the fish-like robots in complex fluid fields is of great importance in improving the performance of underwater vehicles, due to its strong maneuverability, propulsion efficiency, and deceptive visual appearance. In this article, a novel learning-based control framework is first proposed to autonomously explore efficient control policies that are capable of performin...
-
The generation of energy-efficient and dynamic-aware robot motions that satisfy constraints such as joint limits, self-collisions, and collisions with the environment remains a challenge. In this context, Riemannian geometry offers promising solutions by identifying robot motions with geodesics on the so-called configuration space manifold. While this manifold naturally considers the intrinsic rob...
-
One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Al-though several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully handcrafted obstacle descriptions. In this work,...
-
The work in this paper delves into the challenge of whole elongated body's obstacle avoidance during path following for a class of bionic snake robots. Currently, most studies focus solely on preventing the robot's head from colliding with obstacles through designed controllers. However, due to the unique elongated structure and biomimetic locomotion modes of snake robots, it is unavoidable that t...
-
This paper presents a safety-critical approach to the coordinated control of cooperative robots locomoting in the presence of fixed (holonomic) constraints. To this end, we leverage control barrier functions (CBFs) to ensure the safe cooperation of the robots while maintaining a desired formation and avoiding obstacles. The top-level planner generates a set of feasible trajectories, accounting for...
-
Trajectory optimization problems for legged robots are commonly formulated with fixed contact schedules. These multi-phase Hybrid Trajectory Optimization (HTO) methods result in locally optimal trajectories, but the result depends heavily upon the predefined contact mode sequence. Contact-Implicit Optimization (CIO) offers a potential solution to this issue by allowing the contact mode to be deter...
-
The safety-critical control of robotic systems often must account for multiple, potentially conflicting, safety constraints. This paper proposes novel relaxation techniques to address safety-critical control problems in the presence of conflicting safety conditions. In particular, Control Barrier Functions (CBFs) provide a means to encode safety as constraints in a Quadratic Program (QP), wherein ...
-
The rotation axis of human knee joint varies according to knee flexion angles. That is, human knee movement is spatial with dominant flexional rotation. Knee joints of most lower-limb exoskeletons were, however, realized with a simple revolute pair for design simplicity. Wearing the knee joint with a simple revolute pair constrains inevitably natural parasitic motion of human knee joint. Rigid con...
-
Operating in narrow spaces is an important challenge in the development of robots. Redundant manipulators are one way to solve this problem, but their mechanism design and control method still have much room for improvement. In this paper, we propose a coiled cable-conduit-driven hyper-redundant manipulator (C-CDHRM) with great slenderness and flexibility. In terms of mechanism design, it consider...
-
The flexure-based XYZ micropositioner with hybrid configuration has become more prevalent due to the characteristics of less mechanism decoupling and high motion precision. However, traditional mechanism design suffers from a large plane occupation with Z stage stacking, which leads to a low space-utilization efficiency. To address this issue, a novel conceptual design is proposed in this paper by...
-
Pipelines have become one of the most important infrastructures in the city. Over time, they are prone to aging, cracks, corrosion, and the demand for regular inspection is gradually increasing. Robotic solutions are effective methods for in-pipe inspection. However, existing In-pipe Inspection Robots (IPIR) require that the inner diameter of the pipe is fixed in the application scenarios, and nee...
-
Hands are one of the most intricate elements of a humanoid due to their role as end-effectors interacting with their non-linear surrounding environment. This paper aims to present the design of a bioinspired underactuated robotic hand with an improved dexterity that is capable of adaptive grasping and manipulation of a wide-range of objects using a dual-tendon mechanism. The proposed design is foc...
-
We propose a framework for optimizing a planar parallel-jaw gripper for use with multiple objects. While optimizing general-purpose grippers and contact locations for grasps are both well studied, co-optimizing grasps and the gripper geometry to execute them receives less attention. As such, our framework synthesizes grippers optimized to stably grasp sets of polygonal objects. Given a fixed numbe...
-
We have been investigating a crawling-like loco-motion robot to make it efficiently slide forward based on a simple system and control mechanisms on a slippery level surface, where the motion of the center of mass plays an important role. In this paper, we induce an effective motion of the center of mass considering a streamlined body shape of a locomotion robot in which a pendulum is installed. F...
-
The advantages of cable-driven exoskeleton robots with series elastic actuators can be summarized in twofold: 1) the inertia of the robot joint is relatively low, which is more friendly for human-robot interaction; 2) the elastic element is tolerant to impacts and hence provides structural safety. As trade-offs, the overall dynamic model of such a system is of high order and subject to both unmode...
-
Soft-growing robots are emerging with numerous potential applications because of their superior capability of frictionless navigation. However, their success is hindered by their tendency to buckle under the tension required to retract them via inversion. In this paper, we propose a simple and scalable tubular backbone to facilitate retracting the robot body without buckling. With this backbone, c...
-
We present CurveQuad, a miniature curved origami quadruped that is able to self-fold and unfold, crawl, and steer, all using a single actuator. CurveQuad is designed for planar manufacturing, with parts that attach and stack sequentially on a flat body. The design uses 4 curved creases pulled by 2 pairs of tendons from opposite ends of a link on a 270°servo. It is 8 cm in the longest direction and...
-
In this paper, we present a novel rolling, jumping robot. The robot consists of a driven pendulum mounted to a wheel in a compact, lightweight, 3D printed design. We show that by driving the pendulum to shift the robot's weight distribution, the robot is able to obtain significant rolling speed, achieve jumps of up to 2.5 body lengths vertically, and clear horizontal distances of over 6 body lengt...
-
Localization of robots is a complex task that is often hindered by the sensors these systems use. Due to the majority of field robots being rigid, most of these sensing modalities have the same common faults, such as performance being hindered when their camera vision is obscured. In addition, rigid systems lack flexibility when traversing multiple environments: especially when traversing uneven a...
-
When faced with an unstructured environment filled with an unknown number and size of obstacles on a chaotic terrain, it can be a challenge to determine the best method for navigating and mapping the space. This problem, known as Simultaneous Localization and Mapping (SLAM), has typically been approached using vision-based solutions, but these solutions require clear visual conditions in order to ...
-
Soft actuators offer compliant and safe interaction with an unstructured environment compared to their rigid counterparts. However, control of these systems is often challenging because they are inherently under-actuated, have infinite degrees of freedom (DoF), and their mechanical properties can change by unknown external loads. Existing works mainly relied on discretization and reduction, suffer...
-
Tensegrity robots, composed of rigid rods and flexible cables, exhibit high strength-to-weight ratios and significant deformations, which enable them to navigate unstructured terrains and survive harsh impacts. They are hard to control, however, due to high dimensionality, complex dynamics, and a coupled architecture. Physics-based simulation is a promising avenue for developing locomotion policie...
-
The adaptability of soft robots makes them ideal candidates to maneuver through unstructured environments. However, locomotion challenges arise due to complexities in modeling the body mechanics, actuation, and robot-environment dynamics. These factors contribute to the gap between their potential and actual autonomous field deployment. A closed-loop path planning framework for soft robot locomoti...
-
This paper introduces a novel approach for modeling the dynamics of soft robots, utilizing a differentiable filter architecture. The proposed approach enables end-to-end training to learn system dynamics, noise characteristics, and temporal behavior of the robot. A novel spatio-temporal embedding process is discussed to handle observations with varying sensor placements and sampling frequencies. T...
-
In this paper, we present a new approach to the control of continuum robot sections using IMU quaternion feedback. We use a discrete time root finding algorithm to steer a continuum section in the desired shape space direction. We found that the approach lacks end effector positioning accuracy when used by itself, however, when used in conjunction with a feedforward model it actively counters the ...
-
Machine learning-based approaches for soft robot proprioception have recently gained popularity, in part due to the difficulties in modeling the relationship between sensor signals and robot shape. However, to date, there exists no systematic analysis of the required design choices to set up a machine learning pipeline for soft robot proprioception. Here, we present the first study examining how d...
-
Recently, active research has been conducted on the therapeutic functions of capsule endoscopes. Here, we propose an active capsule system that captures images of the interior of the gastrointestinal tract (GI) and actively delivers therapeutic patches. The active capsule system mainly comprises therapeutic patches, an active capsule equipped with a camera, and a robot-assisted magnetic actuator. ...
-
This work presents a novel electrical method, implemented in the form of a microfluidic device, for cell arraying and target cell lysis. The microfluidic device contains a micro-well array on the photoconductive layer based on the optoelectronic tweezers (OET) method, where parallel cell manipulation is performed. As cell suspension flows over the micro-wells, cells can be actively captured in the...
-
The untethered microrobots driven by multiple external physics fields have promising ability in minimally invasive disease treatments. One common type of the driving fields is gradient magnetic field, which can provide microrobots with adequate driving force in complicated environment. In this study, a control method of microrobot through gradient magnetic field system is presented, which is reali...
-
One approach to the wireless actuation and gravity compensation of untethered helical magnetic devices (UHMD) is through swimming with a non-zero angle of attack (AoA). This configuration allows us to counteract gravity, so that for a given desired path, we can move the UHMD controllably without drifting downward under its own weight. This study seeks to investigate the use a reduced-order model o...
-
Magnetic nanoparticles can be electrostatically assembled around sperm cells to form biohybrid micro robots. These biohybrid microrobots possess sufficient magnetic material to potentially allow for pulse-echo localization and wireless actuation. Alternatively, magnetic excitation of these nanoparticles can be used for localization based on Faraday's law of induction using a detection coil. Here, ...
-
Mainly, the integration of fine positioning piezo-actuated stages in precision motion systems is considered, which results in multi-stage configurations. Mostly, in such configurations, the fine stages are attached to the coarse positioning stages- that do not meet required precision- by mechanical means. Once the motion is synchronized, the fine stages enhance the overall precision of the multi-s...
-
We propose a mechanism for low Reynolds num-ber walking (e.g., legged microscale robots). Whereas locomotion for legged robots has traditionally been classified as dynamic (where inertia plays a role) or static (where the system is always statically stable), we introduce a new locomotion modality we call buoyancy enabled non-inertial dynamic walking in which inertia plays no role, yet the robot is...
-
Ultrafast acoustic holography (AH) enabling dynamic contactless micro-nano robotic manipulation has recently attracted wide attention. As an advanced technique, AH encodes specific three-dimensional (3D) acoustic field on a two-dimensional (2D) hologram whereby realizing holographic reconstruction with high fidelity. However, current approaches face the limitation of encoding time, accuracy and fl...
-
Alginate hydrogels are widely researched in phar-maceutical applications for their abilities to encapsulate and dis-perse therapeutics in response to stimuli. While effective, their utility can be greatly improved once converted into artificial cell soft-microrobots, allowing them to actively navigate through complex in vivo environments and facilitate targeted drug deliv-ery. In this study, artif...
-
This paper presents the design and control of a novel microrobot that utilizes two distinct magnetic locomotion methods, a combination of rotating and gradient field control, for precise micro-object manipulation using multiple end-effectors. Rotating magnetic fields induce a tumbling locomotion mode to increase the movement speed and decrease issues associated with stiction and locomotion over ro...
-
We present a new unimorph actuator for micro-robotics, which is driven by thin shape-memory alloy (SMA) wires. Using a passive-capillary-alignment technique and existing SMA-microsystem fabrication methods, we developed an actuator that is 7 mm long, has a volume of 0.45 mm3, weighs 0.96 mg, and can achieve operation frequencies of up to 40 Hz as well as lift 155 times its own weight. To demonstra...
-
Sub-gram flying robots have transformative potential in applications from search and rescue to precision agriculture to environmental monitoring. However, a key gap in achieving autonomous flight for these applications is the low lift to weight ratio of flapping wing and quadrotor designs around 1 g or less. To close this gap, we propose a helictoper-style design that minimizes size and weight by ...
-
Colloidal particles trapped by a focused laser at the air-liquid interface provide an interesting assembly dynamic. In this study, we demonstrated manipulating optical force-induced swarms via dynamic locomotion of assemblies built with holographic optical tweezers. This manipulation approach builds the foundation for autonomous control of building assemblies at the air-liquid interface, which is ...
-
We present a reinforcement learning (RL) framework that enables quadrupedal robots to perform soccer goalkeeping tasks in the real world. Soccer goalkeeping with quadrupeds is a challenging problem, that combines highly dynamic locomotion with precise and fast non-prehensile object (ball) manipulation. The robot needs to react to and intercept a potentially flying ball using dynamic locomotion man...
-
This paper presents a safety-critical locomotion control framework for quadrupedal robots. Our goal is to enable quadrupedal robots to safely navigate in cluttered environments. To tackle this, we introduce exponential Discrete Control Barrier Functions (exponential DCBFs) with duality-based obstacle avoidance constraints into a Non-linear Model Predictive Control (NMPC) with Whole-Body Control (W...
-
Generating natural and physically feasible motions for legged robots has been a challenging problem due to its complex dynamics. In this work, we introduce a novel learning-based framework of autoregressive motion planner (ARMP) for quadruped locomotion and navigation. Our method can generate motion plans with an arbitrary length in an autore-gressive fashion, unlike most offline trajectory optimi...
-
Attics are one of the largest sources of energy loss in residential homes, but they are uncomfortable and dangerous for human workers to conduct air sealing and insulation. Hexapod robots are potentially suitable for carrying out those tasks in tight attic spaces since they are stable, compact, and lightweight. For hexapods to succeed in these tasks, they must be able to navigate inside tight atti...
-
Legged animals have developed a variety of modes of locomotion to adapt to the diverse and unknown terrain challenges posed in the natural world. Legged robots, however, have been largely limited to specializing in one domain, with few that have endeavored to bridge the gap between two. In this work we present the Scansorial, Terrestrial, and Aquatic Robot Quadruped (STARQ), a novel legged robot c...
-
Hierarchical Adaptive Control for Collaborative Manipulation of a Rigid Object by Quadrupedal Robots
Despite the potential benefits of collaborative robots, effective manipulation tasks with quadruped robots remain difficult to realize. In this paper, we propose a hierarchical control system that can handle real-world collaborative manipulation tasks, including uncertainties arising from object properties, shape, and terrain. Our approach consists of three levels of controllers. Firstly, an adapt...
-
Entanglements like vines and branches in natural settings or cords and pipes in human spaces prevent mobile robots from accessing many environments. Legged robots should be effective in these settings, and more so than wheeled or tracked platforms, but naive controllers quickly become entangled and stuck. In this paper we present a method for proprioception aimed specifically at the task of sensin...
-
Learning various motor skills for quadrupedal robots is a challenging problem that requires careful design of task-specific mathematical models or reward descriptions. In this work, we propose to learn a single capable policy using deep reinforcement learning by imitating a large number of reference motions, including walking, turning, pacing, jumping, sitting, and lying. On top of the existing mo...
-
This paper introduces a way to systematically investigate the effect of compliant prismatic spines in quadrupedal robot locomotion. We develop a novel spring-loaded lockable spine module, together with a new Spinal Compliance-Integrated Quadruped (SCIQ) platform for both empirical and numerical research. Individual spine tests reveal beneficial spinal characteristics like a degressive spring, and ...
-
The path planning problems arising in manipulation planning and in task and motion planning settings are typically repetitive: the same manipulator moves in a space that only changes slightly. Despite this potential for reuse of information, few planners fully exploit the available information. To better enable this reuse, we decompose the collision checking into reusable, and non-reusable parts. ...
-
We improve reliable, long-horizon, goal-directed navigation in partially-mapped environments by using nonlocally available information to predict the goodness of temporally-extended actions that enter unseen space. Making predictions about where to navigate in general requires nonlocal information: any observations the robot has seen so far may provide information about the goodness of a particula...
-
In the latest research for unmanned aerial vehicles (UAVs), time-optimal trajectory planning of a point-mass with acceleration as control input and constrained maximum velocity (TOT-PMAV) has proved to be very promising for UAV behavior planning. They can be calculated within microseconds and tracked with high precision by modern trajectory tracking controllers like model predictive control (MPC)....
-
This paper presents Fast Adaptive Tree (FAT), an asymptotically-optimal sampling-based path planner for dynamic and uncertain scenarios. Namely, the solution extracted converges to the optimal solution given the sensor information as the number of samples approaches infinity. The planner maintains an underlying graph, which increasingly approximates the search domain, and a dynamic spanning tree o...
-
Autonomous navigation of ground robots on uneven terrain is being considered in more and more tasks. However, uneven terrain will bring two problems to motion planning: how to assess the traversability of the terrain and how to cope with the dynamics model of the robot associated with the terrain. The trajectories generated by existing methods are often too conservative or cannot be tracked well b...
-
Robotics and automation are poised to change the landscape of home and work in the near future. Robots are adept at deliberately moving, sensing, and interacting with their environments. The pervasive use of robotics promises societal and economic payoffs due to its capabilities—conversely, the capabilities of robots to move within and sense the world around them is susceptible to abuse. Robots, u...
-
This paper presents methods for dramatically improving the performance of sampling-based kinodynamic planners. The key component is a complete, exact steering method that produces a time-optimal trajectory between any states for a vector of synchronized double integrators. This method is applied in three ways: 1) to generate RRT edges that quickly solve the two-point boundary-value problems, 2) to...
-
Inertia-dominated mechanical systems can achieve net displacement by 1) periodically changing their shape (known as kinematic gait) and 2) adjusting their inertia distribution to utilize the existing nonzero net momentum (known as momentum gait). Therefore, finding the gait that most effectively utilizes the two types of locomotion in terms of the magnitude of the net momentum is a significant top...
-
We consider the motion planning problem for stochastic nonlinear systems in uncertain environments. More precisely, in this problem the robot has stochastic nonlinear dynamics and uncertain initial locations, and the environment contains multiple dynamic uncertain obstacles. Obstacles can be of arbitrary shape, can deform, and can move. All uncertainties do not necessarily have Gaussian distributi...
-
Both goal-agnostic and goal-oriented tasks have practical value for robotic grasping: goal-agnostic tasks target all objects in the workspace, while goal-oriented tasks aim at grasping pre-assigned goal objects. However, most current grasping methods are only better at coping with one task. In this work, we propose a bifunctional push-grasping synergistic strategy for goal-agnostic and goal-orient...
-
Training with simulated data is a common approach in pose estimation research. However, a sim-to-real gap between clean simulated data and noisy real data will seriously weaken the generalization ability of the algorithm, especially for point clouds. To address this problem, this paper proposes a domain adaptive pose estimation network (DAPE-Net). For the feature extracted from the backbone, the n...
-
This study focuses on a robotic powder weighing task used in laboratory automation. In this task, a robot weighs a certain amount of powder with a milligram-level target mass using a dispensing spoon. The complex dynamics of the powder, the variations in the materials being weighed, and the need to balance conservative and aggressive actions are significant challenges in the robotics field. Theref...
-
Most state-of-the-art data-driven grasp sampling methods propose stable and collision-free grasps uniformly on the target object. For bin-picking, executing any of those reachable grasps is sufficient. However, for completing specific tasks, such as squeezing out liquid from a bottle, we want the grasp to be on a specific part of the object's body while avoiding other locations, such as the cap. T...
-
This paper proposes a novel approach to address the technical challenges of stable object grasping, particularly in the context of handling tableware in a home environment. Handling tableware is particularly important, yet challenging, due to the flat nature of most tableware objects and the need to maintain a stable posture to prevent spills. To address these challenges, we present three key cont...
-
We present One-Shot Affordance Learning (OSAL): a unified pipeline that learns manipulation for articulated objects by observing human demonstration only once. The key idea of our method is to embody affordance of articulated objects with an open-loop trajectory conditioned on a certain area of the object's surface. It serves as a simplified object-centric manipulation representation, which can be...
-
In this paper, we explore the dynamic grasping of moving objects through active pose tracking and reinforcement learning for hand-eye coordination systems. Most existing vision-based robotic grasping methods implicitly assume target objects are stationary or moving predictably. Performing grasping of unpredictably moving objects presents a unique set of challenges. For example, a pre-computed robu...
-
We propose an improved keypoint approach for 6-DoF grasp pose synthesis from RGB-D input. Keypoint-based grasp detection from image input demonstrated promising results in a previous study, where the visual information provided by color imagery compensates for noisy or imprecise depth measurements. However, it relies heavily on accurate keypoint prediction in image space. We devise a new grasp gen...
-
When grasping objects with a multi-finger hand, it is crucial for the grasp stability to apply the correct torques at each joint so that external forces are countered. Most current systems use simple heuristics instead of modeling the required torque correctly. Instead, we propose a learning-based approach that is able to predict torques for grasps on unknown objects in real-time. The neural netwo...
-
Multi-fingered robotic hands have potential to enable robots to perform sophisticated manipulation tasks. However, teaching a robot to grasp objects with an anthropomorphic hand is an arduous problem due to the high dimensionality of state and action spaces. Deep Reinforcement Learning (DRL) offers techniques to design control policies for this kind of problems without explicit environment or hand...
-
Screw-driving is an important operation in numerous applications. In many situations, hole pose cannot be estimated very accurately. Autonomous screw-driving cannot be performed by traditional industrial manipulators in position control mode when the hole pose uncertainty is high. This paper presents a mobile manipulator system for performing autonomous screw-driving in the presence of uncertainti...
-
We propose a lightweight, effective, and efficient anti-drone network, namely ADMNet, for visually detecting and monitoring unfriendly drones with a constrained view field, flying against a complex environment. We merge an SPP module to the first head of YOLOv4 to improve accuracy and perform network compression to reduce inference latency and model size. To compensate for the accuracy loss caused...
-
In this paper, we present LCM-MVSNet, a novel multi-view stereo (MVS) network with learnable cost metric (LCM) for more accurate and complete depth estimation and dense point cloud reconstruction. To adapt to the scene variation and improve the reconstruction quality in non-Lambertian low-textured scenes, we propose LCM to adaptively aggregate multi-view matching similarity into the 3D cost volume...
-
Perception systems for ornithopters face severe challenges. The harsh vibrations and abrupt movements caused during flapping are prone to produce motion blur and strong lighting condition changes. Their strict restrictions in weight, size, and energy consumption also limit the type and number of sensors to mount onboard. Lightweight traditional cameras have become a standard off-the-shelf solution...
-
The future of 3D printing utilizing unmanned aerial vehicles (UAVs) presents a promising capability to revolutionize manufacturing and to enable the creation of large-scale structures in remote and hard-to-reach areas e.g. in other planetary systems. Nevertheless, the limited payload capacity of UAVs and the complexity in the 3D printing of large objects pose significant challenges. In this articl...
-
This paper introduces a novel approach to video object detection detection and tracking on Unmanned Aerial Vehicles (UAVs). By incorporating metadata, the proposed approach creates a memory map of object locations in actual world coordinates, providing a more robust and interpretable representation of object locations in both, image space and the real world. We use this representation to boost con...
-
This paper addresses the problem of active collaborative localization in heterogeneous robot teams with unknown data association. It involves positioning a small number of identical unmanned ground vehicles (UGVs) at desired positions so that an unmanned aerial vehicle (UAV) can, through unlabelled measurements of UGVs, uniquely determine its global pose. We model the problem as a sequential two p...
-
This paper contributes a novel and modularized learning-based method for aerial robots navigating cluttered environments containing hard-to-perceive thin obstacles without assuming access to a map or the full pose estimation of the robot. The proposed solution builds upon a semantically-enhanced Variational Autoencoder that is trained with both real-world and simulated depth images to compress the...
-
Some animals exhibit multi-modal locomotion capability to traverse a wide range of terrains and environments, such as amphibians that can swim and walk or birds that can fly and walk. This capability is extremely beneficial for expanding the animal's habitat range and they can choose the most energy efficient mode of locomotion in a given environment. The robotic biomimicry of this multi-modal loc...
-
The perception-aware motion planning method based on the localization uncertainty has the potential to improve the localization accuracy for robot navigation. How-ever, most of the existing perception-aware methods pre-build a global feature map and can not generate the perception- aware trajectory in real time. This paper proposes a topology- guided perception-aware receding horizon trajectory ge...
-
Navigation in Global Positioning Systems (GPS)-denied environments requires robust estimators reliant on fusion of inertial sensors able to estimate rigid-body's orientation, position, and linear velocity. Ultra-wideband (UWB) and Inertial Measurement Unit (IMU) represent low-cost measurement technology that can be utilized for successful Inertial Navigation. This paper presents a nonlinear determ...
-
This paper presents a novel method for model-free prediction of grasp poses for suction grippers with multiple suction cups. Our approach is agnostic to the design of the gripper and does not require gripper-specific training data. In particular, we propose a two-step approach, where first, a neural network predicts pixel-wise grasp quality for an input image to indicate areas that are generally g...
-
This paper deals with the challenging task of picking semi-deformable linear objects (SDLOs) from a bin. SDLOs are deformable elements, such as cables, joined to a rigid part as a connector. We propose a vision-based strategy to detect, classify and estimate the pose and the state (free or occluded) of connectors belonging to an unspecified number of SDLOs, arranged in an unknown configuration in ...
-
For robot manipulation, a complete and accurate object shape is desirable. Here, we present a method that combines visual and haptic reconstruction in a closed-loop pipeline. From an initial viewpoint, the object shape is reconstructed using an implicit surface deep neural network. The location with highest uncertainty is selected for haptic exploration, the object is touched, the new information ...
-
When humans see a scene, they can roughly imagine the forces applied to objects based on their expe-rience and use them to handle the objects properly. This paper considers transferring this “force-visualization” ability to robots. We hypothesize that a rough force distribution (named “force map”) can be utilized for object manipulation strategies even if accurate force estimation is impossible. B...
-
For robotic systems to interact with objects in dynamic environments, it is essential to perceive the physical properties of the objects such as shape, friction coefficient, mass, center of mass, and inertia. This not only eases selecting manipulation action but also ensures the task is performed as desired. However, estimating the physical properties of especially novel objects is a challenging p...
-
When robots retrieve specific objects from cluttered scenes, such as home and warehouse environments, the target objects are often partially occluded or completely hidden. Robots are thus required to search, identify a target object, and successfully grasp it. Preceding works have relied on pre-trained object recognition or segmentation models to find the target object. However, such methods requi...
-
Robotic dexterous grasping is a challenging problem due to the high degree of freedom (DoF) and complex contacts of multi-fingered robotic hands. Existing deep re-inforcement learning (DRL) based methods leverage human demonstrations to reduce sample complexity due to the high dimensional action space with dexterous grasping. However, less attention has been paid to hand-object interaction represe...
-
Perception in robot manipulation has been actively explored with the goal of advancing and integrating vision and touch for global and local feature extraction. However, it is difficult to perceive certain object internal states, and the integration of visual and haptic perception is not compact and is easily biased. We propose to address these limitations by developing an active acoustic sensing ...
-
Robotic grasping is a fundamental skill for robots, but it is quite challenging in cluttered scenes. In cluttered scenes, the precise prediction of high-quality grasp configurations such as rotation and grasping width while avoiding collisions is essential. To accomplish this, the grasp detection models require the capabilities of stronger fine-grained information extracted around the grasp points...
-
Many fabric handling and 2D deformable material tasks in homes and industries require singulating layers of material such as opening a bag or arranging garments for sewing. In contrast to methods requiring specialized sensing or end effectors, we use only visual observations with ordinary parallel jaw grippers. We propose SLIP: Singulating Layers using Interactive Perception, and apply SLIP to the...
-
In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is ...
-
In recent years, there has been a significant effort dedicated to developing efficient, robust, and general human-to-robot handover systems. However, the area of flexible handover in the context of complex and continuous objects' motion remains relatively unexplored. In this work, we propose an approach for effective and robust flexible handover, which enables the robot to grasp moving objects wit...
-
An important task in trajectory prediction is to model the uncertainty of agents' motions, which requires the system to propose multiple plausible future trajectories for agents based on their past movements. Recently, many approaches have been developed following an endpointconditioned deep learning framework by firstly predicting the distribution of endpoints, then sampling endpoints from it and...
-
In robotic cockpit inspection scenarios, the 6D pose of highly-variable panel objects is necessary. However, the buttons with different states on the panel cause the variable texture and point cloud, which confuses the traditional invariable object pose estimation method. The bottleneck is the variable texture and point cloud. To address this issue, we propose a simple yet effective method denoted...
-
The exposed cameras of UAVs can shake, shift, or even malfunction under the influence of harsh weather, while the add-on devices (Dupont lines) are very vulnerable to dam-age. Although we can place a low-cost transparent film overlay around the camera to protect it, this would also introduce image degradation issues (such as oversaturation, astigmatism, etc). To tackle the image degradation proble...
-
It is important for daily life support robots to detect changes in their environment and perform tasks. In the field of anomaly detection in computer vision, probabilistic and deep learning methods have been used to calculate the image distance. These methods calculate distances by focusing on image pixels. In contrast, this study aims to detect semantic changes in the daily life environment using...
-
Aotearoa New Zealand has a strong and growing apple industry but struggles to access workers to complete skilled, seasonal tasks such as thinning. To ensure effective thinning and make informed decisions on a per-tree basis, it is crucial to accurately measure the crop load of individual apple trees. However, this task poses challenges due to the dense foliage that hides the fruitlets within the t...
-
Addressing domain shifts for complex perception tasks in autonomous driving has long been a challenging problem. In this paper, we show that existing domain adaptation methods pay little attention to the content mismatch issue between source and target domains, thus weakening the domain adaptation per-formance and the decoupling of domain-invariant and domain-specific representations. To solve the...
-
Text detection is frequently used in vision-based mobile robots when they need to interpret texts in their surroundings to perform a given task. For instance, delivery robots in multilingual cities need to be capable of doing multilingual text detection so that the robots can read traffic signs and road markings. Moreover, the target languages change from region to region, implying the need of eff...
-
Object detection techniques for autonomous Un-manned Aerial Vehicles (UAV) are built upon Deep Neural Networks (DNN), which are known to be vulnerable to adversarial patch perturbation attacks that lead to object detection evasion. Yet, current adversarial patch generation schemes are not designed for UAV imagery settings. This paper proposes a new robust adversarial patch generation attack agains...
-
Computing the distance from a point to a triangle mesh is a key computational step in robotics pipelines such as registration and collision detection, with applications to path planning, SLAM, and RGB-D vision. Numerous techniques to accelerate this computation have been developed, many of which use a cheap pre-processing step to construct a hierarchical decomposition of the mesh. If the mesh is f...
-
Line detection is widely used in many robotic tasks such as scene recognition, 3D reconstruction, and simultaneous localization and mapping (SLAM). Compared to points, lines can provide both low-level and high-level geometrical information for downstream tasks. In this paper, we propose a novel learnable edge-based line detection algorithm, AirLine, which can be applied to various tasks. In contra...
-
Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonizati...
-
Visual Place Recognition (VPR) is essential for autonomous robots and unmanned vehicles, as an accurate identification of visited places can trigger a loop closure to optimize the built map. The most prevalent methods tackle VPR as a single-frame retrieval task, which uses a CNN-based encoder to describe and compare each individual frame. These methods, however, overlook the temporal information b...
-
In a structural collapse, debris piles up in a chaotic and unstable manner, creating pockets and void spaces that are difficult to see or access. Often, these regions have the highest chances of concealing survivors and identifying such regions can increase the success of a search and rescue (SAR) operation while ensuring the safety of both survivors and rescue teams. In this paper, we present an ...
-
This paper proposes a multi-map based visual localization method for image sequences. Given multiple single-map based localization results, we combine them with SLAM to estimate robust and accurate camera poses under challenging conditions. Our method comprises three modules connected in a sequence. First, we reconstruct multiple reference maps using the Structure-from-Motion technique, one map fo...
-
This paper presents a novel approach called the Interacting Multiple Model (IMM)-based Maximum Correntropy Student's T Filter (MCStF), which addresses the challenges posed by non-Gaussian measurement noises. The MCStF demonstrates superior performance compared to the IMM algorithm based on Kalman Filters (KFs) in both simulation environments and real-time systems. The Crazyflie 2.0 nano Unmanned A...
-
The success of re-localisation has crucial implications for the practical deployment of robots operating within a prior map or relative to one another in real-world scenarios. Using single-modality, place recognition and localisation can be compromised in challenging environments such as forests. To address this, we propose a strategy to prevent lidar-based re-localisation failure using lidar-imag...
-
In recent years, Neural Radiation Fields (NeRF) have shown tremendous potential in encoding highly-detailed 3D geometry and environmental appearance, thus making it a promising alternative to traditional explicit maps for robot localization. However, current NeRF localization methods suffer from significant computational overheads, primarily resulting from the large number of iterations or particl...
-
Camera localization is a fundamental problem for many applications in computer vision, robotics, and autonomy. Despite recent deep learning-based approaches, the lack of robustness in challenging conditions persists due to changes in appearance caused by texture-less planes, repeating structures, reflective surfaces, motion blur, and illumination changes. Data augmentation is an attractive solutio...
-
This article proposes a novel indoor magnetic field-based place recognition algorithm that is accurate and fast to compute. For that, we modified the generalized “Hough Transform” to process magnetic data (MagHT). It takes as input a sequence of magnetic measures whose relative positions are recovered by an odometry system and recognizes the places in the magnetic map where they were acquired. It ...
-
Long-term visual localization is an essential problem in robotics and computer vision, but remains challenging due to the environmental appearance changes caused by lighting and seasons. While many existing works have attempted to solve it by directly learning invariant sparse keypoints and descriptors to match scenes, these approaches still struggle with adverse appearance changes. Recent develop...
-
Re-Iocalizing a camera from a single image in a previously mapped area is vital for many computer vision applications in robotics and augmented/virtual reality. In this work, we address the problem of estimating the 6 DoF camera pose relative to a global frame from a single image. We propose to leverage a novel network of relative spatial and temporal geometric constraints to guide the training of...
-
State-of-the-art lidar place recognition models exhibit unreliable performance when tested on environments different from their training dataset, which limits their use in complex and evolving environments. To address this issue, we investigate the task of uncertainty-aware lidar place recognition, where each predicted place must have an associated uncertainty that can be used to identify and reje...
-
Recently, data-driven approaches have brought both opportunities and challenges for Inertial Navigation Systems. In this paper, we propose a novel data-driven method which is composed of cascading orientation and translation estimation with IMU-only measurements. For robust orientation estimation, we combine a CNN-based neural network with an EKF to eliminate orientation errors caused by sensor no...
-
In recent years, depth sensors have become more and more affordable and have found their way into a growing amount of robotic systems. However, mono- or multi-modal sensor registration, often a necessary step for further pro-cessing, faces many challenges on raw depth images or point clouds. This paper presents a method of converting depth data into images capable of visualizing spatial details th...
-
This paper proposes an illumination-robust visual odometry (VO) system that incorporates both accelerated learning-based corner point algorithms and an extended line feature algorithm. To be robust to dynamic illumination, the proposed system employs the convolutional neural network (CNN) and graph neural network (GNN) to detect and match reliable and informative corner points. Then point feature ...
-
We propose a novel geometric and photometric 3D mapping pipeline for accurate and real-time scene reconstruction from casually taken monocular images. To achieve this, we leverage recent advances in dense monocular SLAM and real-time hierarchical volumetric neural radiance fields. Our insight is that dense monocular SLAM provides the right information to fit a neural radiance field of the scene in...
-
Pose graph relaxation has become an indispensable addition to SLAM enabling efficient global registration of sensor reference frames under the objective of satisfying pair-wise relative transformation constraints. The latter may be given by incremental motion estimation or global place recognition. While the latter case enables loop closures and drift compensation, care has to be taken in the mono...
-
With the increasing application of robots, stable and efficient Visual Odometry (VO) algorithms are becoming more and more important. Based on the Fourier Mellin Transformation (FMT) algorithm, the extended Fourier Mellin Transformation (eFMT) is an image registration approach that can be applied to downward-looking cameras, for example on aerial and underwater vehicles. eFMT extends FMT to multi-...
-
Fiducial markers can encode rich information about the environment and aid Visual SLAM (VSLAM) approaches in reconstructing maps with practical semantic information. Current marker-based VSLAM approaches mainly utilize markers for improving feature detections in low-feature environments and/or incorporating loop closure constraints, generating only low-level geometric maps of the environment prone...
-
This paper presents RVWO, a system designed to provide robust localization and mapping for wheeled mobile robots in challenging scenarios. The proposed approach leverages a probabilistic framework that incorporates semantic prior information about landmarks and visual re-projection error to create a landmark reliability model, which acts as an adaptive kernel for the visual residuals in optimizati...
-
Our paper proposes a direct sparse visual odometry method that combines event and RGBD data to estimate the pose of agile-legged robots during dynamic locomotion and acrobatic behaviors. Event cameras offer high temporal resolution and dynamic range, which can eliminate the issue of blurred RGB images during fast movements. This unique strength holds a potential for accurate pose estimation of agi...
-
Line information from urban structures can be exploited as an additional geometrical feature to achieve robust vision-based simultaneous localization and mapping (SLAM) systems in textureless scenes. Sometimes, however, conventional line tracking methods fail to track caused by image blur or occlusion. Even though these lost line features are just a subset of plenty of features, the failure in fea...
-
Robust feature matching forms the backbone for most Visual Simultaneous Localization and Mapping (vSLAM), visual odometry, 3D reconstruction, and Structure from Motion (SfM) algorithms. However, recovering feature matches from texture-poor scenes is a major challenge and still remains an open area of research. In this paper, we present a Stereo Visual Odometry (StereoVO) technique based on point a...
-
Transparency in decision-making is an important factor for AI-driven autonomous systems to be trusted and relied on by users. Studies in the field of visual information processing typically attempt to make an AI system's behavior transparent by showing bounding boxes or heatmaps as explanations. However, it has also been found that an excessive amount of explanations sometimes causes information o...
-
The performance of speech and events recognition systems significantly improved recently thanks to deep learning methods. However, some of these tasks remain challenging when algorithms are deployed on robots due to the unseen mechanical noise and electrical interference generated by their actuators while training the neural networks. Ego-noise reduction as a preprocessing step therefore can help ...
-
Effective access to relevant environmental changes in large human environments is critical for service robots to perform tasks. Since the position of a dynamic goal such as a human is variable, it will be difficult for the robot to locate him accurately. It is worth noting that humans can obtain information through social software, and deal with daily affairs. The current robots search for targets...
-
For a robot to personalize physical assistance effectively, it must learn user preferences that can be generally reapplied to future scenarios. In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people's preferences can vary greatly d...
-
Visual target navigation in unknown environments is a crucial problem in robotics. Despite extensive investigation of classical and learning-based approaches in the past, robots lack common-sense knowledge about household objects and layouts. Prior state-of-the-art approaches to this task rely on learning the priors during the training and typically require significant expensive resources and time...
-
Autonomous mobile agents (e.g., mobile ground robots and UAVs) typically require low-power/energy-efficient machine learning (ML) algorithms to complete their ML-based tasks (e.g., object recognition) while adapting to diverse environments, as mobile agents are usually powered by batteries. These requirements can be fulfilled by Spiking Neural Networks (SNNs) as they offer low power/energy process...
-
Large Language Models (LLMs) trained using massive text datasets have recently shown promise in generating action plans for robotic agents from high-level text queries. However, these models typically do not consider the robot's environment, resulting in generated plans that may not actually be executable, due to ambiguities in the planned actions or environmental constraints. In this paper, we pr...
-
Understanding and learning the actor-to-X inter-actions (AXIs), such as those between the focal vehicles (actor) and other traffic participants (e.g., other vehicles, pedestrians) as well as traffic environments (e.g., city/road map), is essential for the development of a decision-making model and simulation of autonomous driving (AD). Existing practices on imitation learning (IL) for AD simulatio...
-
We consider the detection of faults in robotic manipulators, with particular emphasis on faults that have not been observed or identified in advance, which naturally includes those that occur very infrequently. Recent studies indicate that the reward function obtained through Inverse Reinforcement Learning (IRL) can help detect anomalies caused by faults in a control system (i.e. fault detection)....
-
Programming robot behavior in a complex world faces challenges on multiple levels, from dextrous low-level skills to high-level planning and reasoning. Recent pre-trained Large Language Models (LLMs) have shown remarkable reasoning ability in few-shot robotic planning. However, it remains challenging to ground LLMs in multimodal sensory input and continuous action output, while enabling a robot to...
-
Deep reinforcement learning (DRL) algorithms have proven effective in robot navigation, especially in unknown environments, by directly mapping perception inputs into robot control commands. However, most existing methods ignore the local minimum problem in navigation and thereby cannot handle complex unknown environments. In this paper, we propose the first DRL-based navigation method modeled by ...
-
Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise and prior knowledge. Inspired by large pre-trained language models, this work introduces a paradigm for pretraining a general purpose representation that can serve as a starting point for multiple tasks on a given robot....
-
This paper presents a novel Learning from Demonstration (LfD) method that uses neural fields to learn new skills efficiently and accurately. It achieves this by utilizing a shared embedding to learn both scene and motion representations in a generative way. Our method smoothly maps each expert demonstration to a scene-motion embedding and learns to model them without requiring hand-crafted task pa...
-
We introduce a Learning from Demonstration (LID) approach for contact-rich manipulation tasks, i.e., tasks in which the manipulandum's motion is constrained by contact with the environment. Our approach is motivated by the insight that even a large number of demonstrations will often not contain sufficient information to obtain a general policy for the task. To obtain general policies, our approac...
-
Performing a manipulation contact task in an unknown and unstructured environment is still a challenge. Learning from Demonstration (LfD) techniques provide an intuitive means to define difficult-to-model contact tasks, but have attributes that make them undesirable for novice users in uncertain environments. We present a novel end-to-end system that captures a single manipulation task demonstrati...
-
Dynamic movement primitives are widely used for learning skills that can be demonstrated to a robot by a skilled human or controller. While their generalization capabilities and simple formulation make them very appealing to use, they possess no strong guarantees to satisfy operational safety constraints for a task. We present constrained dynamic movement primitives (CDMPs), which can allow for po...
-
Learning from feedback is a common paradigm to acquire information that is hard to specify a priori. In this work, we consider an agent with a known nominal reward model that captures its high-level task objective. Furthermore, the agent operates subject to constraints that are unknown a priori and must be inferred from human interventions. Unlike existing methods, our approach does not rely on fu...
-
The need for opponent modeling and tracking arises in several real-world scenarios, such as professional sports, video game design, and drug-trafficking interdiction. In this work, we present Graph based Adversarial Modeling with Mutual Information (GrAMMI) for modeling the behavior of an adversarial opponent agent. GrAMMI is a novel graph neural network (GNN) based approach that uses mutual infor...
-
Human-robot motion retargeting is a crucial approach for fast learning motion skills. Achieving real-time retargeting demands high levels of synchronization and accuracy. Even though existing retargeting methods have swift calculation, they still cause time-delay effect on the synchronous retargeting. To mitigate this issue, this paper proposes a motion retargeting method guided by prediction, whi...
-
Movement primitives are trainable parametric models that reproduce robotic movements starting from a limited set of demonstrations. Previous works proposed simple linear models that exhibited high sample efficiency and generalization power by allowing temporal modulation of move-ments (reproducing movements faster or slower), blending (merging two movements into one), via-point conditioning (const...
-
In this paper, we propose a novel network architecture for visual imitation learning that exploits neural radiance fields (NeRFs) and key-point correspondence for self-supervised visual motor policy learning. The proposed network architecture incorporates a dynamic system output layer for policy learning. Combining the stability and goal adaption properties of dynamic systems with the robustness o...
-
Learning from demonstrations is the paradigm where robots acquire new skills demonstrated by an expert and alleviate the physical burden on experts to perform repetitive tasks. Ultrasound scanning is one of the ways to view the anatomical structures of soft tissues, but it is repetitive for some tissue scanning tasks. In this study, an autonomous ultrasound scanning towards a standard plane framew...
-
Pelvic fractures are one of the most serious traumas in orthopedics, and the technical proficiency and expertise of the surgical team strongly influence the quality of reduction results. With the advancement of information technology and robotics, robot-assisted pelvic fracture reduction surgery is expected to reduce the impact caused by inexperienced doctors and improve the accuracy and stability...
-
Human-centric visual understanding is an important desideratum for effective human-robot interaction. In order to navigate crowded public places, social robots must be able to interpret the activity of the surrounding humans. This paper addresses one key aspect of human-centric visual understanding, multi-person pose estimation. Achieving good performance on multi-person pose estimation in crowded...
-
Gait analysis can provide relevant information about the physical and neurological conditions of individuals. For this reason, several studies have recently been carried out in an attempt to monitor people's gait and automatically detect gait anomalies. Among the various monitoring systems available for gait analysis, wearable sensors are considered the gold standard due to their wide capture rang...
-
Thanks to the development of 2D keypoint detectors, monocular 3D human pose estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable improvements. Still, monocular 3D HPE is a challenging problem due to the inherent depth ambiguities and occlusions. To handle this problem, many previous works exploit temporal information to mitigate such difficulties. However, there are many rea...
-
Robust, fast, and accurate human state - 6D pose and posture - estimation remains a challenging problem. For real-world applications, the ability to estimate the human state in realtime is highly desirable. In this paper, we present BodySLAM++, a fast, efficient, and accurate human and camera state estimation framework relying on visual-inertial data. BodySLAM++ extends an existing visual-inertial...
-
Two distinct technologies have gained attention lately due to their prospects for motor rehabilitation: robotics and brain-machine interfaces (BMIs). Harnessing their combined efforts is a largely uncharted and promising direction that has immense clinical potential. However, a significant challenge is whether motor intentions from the user can be accurately detected using non-invasive BMIs in the...
-
Human motion prediction is important for mobile service robots and intelligent vehicles to operate safely and smoothly around people. The more accurate predictions are, particularly over extended periods of time, the better a system can, e.g., assess collision risks and plan ahead. In this paper, we propose to exploit maps of dynamics (MoDs, a class of general representations of place-dependent sp...
-
An accurate and uncertainty-aware 3D human body pose estimation is key to enabling truly safe but efficient human-robot interactions. Current uncertainty-aware methods in 3D human pose estimation are limited to predicting the uncertainty of the body posture, while effectively neglecting the body shape and root pose. In this work, we present GloPro, which to the best of our knowledge the first fram...
-
This work devises an optimized machine learning approach for human arm pose estimation from a single smart-watch. Our approach results in a distribution of possible wrist and elbow positions, which allows for a measure of uncertainty and the detection of multiple possible arm posture solutions, i.e., multimodal pose distributions. Combining estimated arm postures with speech recognition, we turn t...
-
Identifying intentions is a critical task for comprehending the actions of others, anticipating their future behavior, and making informed decisions. However, it is challenging to recognize intentions due to the uncertainty of future human activities and the complex influence factors. In this work, we explore the method of recognizing intentions alluded under human behaviors in the real world, aim...
-
Human motion prediction is an essential component for enabling close-proximity human-robot collaboration. The task of accurately predicting human motion is non-trivial and is compounded by the variability of human motion and the presence of multiple humans in proximity. To address some of the open challenges in motion prediction, in this work, we propose VADER, a novel sequence learning algorithm ...
-
As personal robots become increasingly accessible and affordable, their applications extend beyond large corporate warehouses and factories to operate in diverse, less controlled environments, where they interact with larger groups of people. In such contexts, ensuring not only safety and efficiency but also mitigating potential adverse psychological impacts on humans and adhering to unwritten soc...
-
Vision systems mounted on home robots need to interact with unseen classes in changing environments. Robots have limited computational resources, labelled data and storage capability. These requirements pose some unique challenges: models should adapt without forgetting past knowledge in a data- and parameter-efficient way. We characterize the problem as few-shot (FS) online continual learning (OC...
-
Popular industrial robotic problems such as spray painting and welding require (i) conditioning on free-shape 3D objects and (ii) planning of multiple trajectories to solve the task. Yet, existing solutions make strong assumptions on the form of input surfaces and the nature of output paths, resulting in limited approaches unable to cope with real-data variability. By leveraging on recent advances...
-
This paper describes a domestic service robot (DSR) that fetches everyday objects and carries them to specified destinations according to free-form natural language instructions. Given an instruction such as “Move the bottle on the left side of the plate to the empty chair,” the DSR is expected to identify the bottle and the chair from multiple candidates in the environment and carry the target ob...
-
Unsupervised domain adaptation (UDA) is proposed to better adapt the network trained on labeled synthetic data to unlabeled real-world data for addressing the annotation cost. However, most of these methods pay more attention to domain distributions in input and output stages while ignoring the important differences in semantic expressions and local details in middle feature stages. Therefore, a n...
-
Event cameras provide high temporal precision, low data rates, and high dynamic range visual perception, which are well-suited for optical flow estimation. While data-driven optical flow estimation has obtained great success in RGB cameras, its generalization performance is seriously hindered in event cameras mainly due to the limited and biased training data. In this paper, we present a novel sim...
-
Autonomous robots deployed in the real world will need control policies that rapidly adapt to environmental changes. To this end, we propose AutoRobotics-Zero (ARZ), a method based on AutoML-Zero that discovers zero-shot adaptable policies from scratch. In contrast to neural network adaption policies, where only model parameters are optimized, ARZ can build control algorithms with the full express...
-
One powerful paradigm in visual navigation is to predict actions from observations directly. Training such an end-to-end system allows representations useful for downstream tasks to emerge automatically. However, the lack of inductive bias makes this system data inefficient. We hypothesize a sufficient representation of the current view and the goal view for a navigation policy can be learned by p...
-
This paper introduces and solves a visibility-based escort planning problem. This novel problem, which is closely related to the well-researched family of visibility-based pursuit-evasion problems in robotics, entails an escort agent tasked with escorting a vulnerable agent, called the VIP, in a 2-dimensional environment. The escort protects the VIP from adversaries that pose line-of-sight threats...
-
Motion planning tasks are often innately invariant to certain geometric transformations, or in other words, symmetric. This property, however, is not always reflected in learned policies that are trained on these tasks. Although this asymmetry can be addressed through data augmentation or additional training samples, doing so comes at a cost of increased training time. Instead of trying to remedy ...
-
An in-flight loss of thrust poses a risk to the aircraft, its passengers, and people on the ground. When a loss of thrust happens, the (auto)pilot is forced to perform an emergency landing, possibly toward one of the reachable airports. If none of the airports is reachable, the aircraft is forced to land at another location, which can be risky in urban environments. In this work, we present a gene...
-
This work focuses on spatial time-optimal motion planning, a generalization of the exact time-optimal path following problem that allows a system to plan within a predefined space. In contrast to state-of-the-art methods, we drop the assumption of a given collision-free geometric reference. Instead, we present a three-stage motion planning method that solely relies on start and goal locations and ...
-
Inertia drift is an aggressive transitional driving maneuver, which is challenging due to the high nonlinearity of the system and the stringent requirement on control and planning performance. This paper presents a solution for the consecutive inertia drift of an autonomous RC car based on primitive-based planning and data-driven control. The planner generates complex paths via the concatenation o...
-
We present a hybrid-state path planner for autonomous surface vehicles (ASVs) constrained by a min-imum turning radius. The work is motivated by the future Norwegian naval mine countermeasures (NMCM) concept, which includes mine sweeping operations with ASVs that operate alone or in a formation of two, with and without mine sweeping equipment attached. Our path-planning approach is a variant of hy...
-
Current motion planning approaches rely on binary collision checking to evaluate the validity of a state and thereby dictate where the robot is allowed to move. This approach leaves little room for robots to engage in contact with an object, as is often necessary when operating in densely cluttered spaces. In this work, we propose an alternative method that considers contact states as high-cost st...
-
A convenient situation can be realized if home robots replace housework. However, tasks in an actual home environment are challenging for robots. Particularly, cleaning the table after eating is challenging because of the cluttered environments and various tableware shapes. This study presents a new type of gripper appropriate for picking and placing various tableware in narrow and cluttered envir...
-
In scenarios involving grasping multiple targets, the learning of stacking relationships between objects is fundamental for robots to execute safely and efficiently. However, current methods lack subdivision for the hierarchy of stacking relationship types. In scenes where objects are mostly stacked in an orderly manner, they are incapable of performing human-like and high-efficient grasping decis...
-
To perform household tasks, assistive robots receive commands in the form of user language instructions for tool manipulation. The initial stage involves selecting the intended tool (i.e., object grounding) and grasping it in a task-oriented manner (i.e., task grounding). Nevertheless, prior researches on visual-language grasping (VLG) focus on object grounding, while disregarding the fine-grained...
-
When performing cloth-related tasks, such as garment hanging, it is often important to identify and grasp certain structural regions—a shirt's collar as opposed to its sleeve, for instance. However, due to cloth deformability, these manipulation activities, which are essential in domestic, health care, and industrial contexts, remain challenging for robots. In this paper, we focus on how to segmen...
-
Dexterous manipulation using an under-actuated hand has been a challenging task due to its non-linear dynamical characteristics. For a linkage-based under-actuated hand designed to be used to grasp and manipulate large, heavy, and rigid objects stably, precision grasping is necessary, which makes the task even more difficult to deal with. While approaches based on external sensors have been introd...
-
Studying the manipulation of deformable linear objects has significant practical applications in industry, including car manufacturing, textile production, and electronics automation. However, deformable linear object manipulation poses a significant challenge in developing planning and control algorithms, due to the precise and continuous control required to effectively manipulate the deformable ...
-
Humans naturally execute many everyday manipulation actions with both arms simultaneously. Similarly, endowing robots with bimanual manipulation task models is key to efficiently perform complex manipulation tasks. To do so, a promising approach is to learn a library of task models from human demonstrations. However, this requires human motions to be meaningfully segmented. In this paper, we propo...
-
Soft object manipulation tasks in domestic scenes pose a significant challenge for existing robotic skill learning techniques due to their complex dynamics and variable shape characteristics. Since learning new manipulation skills from human demonstration is an effective way for robot applications, developing prior knowledge of the representation and dynamics of soft objects is necessary. In this ...
-
The ability of aerial robots to operate in the presence of failures is crucial in various applications that demand continuous operations, such as surveillance, monitoring, and inspection. In this paper, we propose a fault-tolerant control strategy for quadrotors that can adapt to single and dual complete rotor failures. Our approach augments a classic geometric tracking controller on $S{O}(3)\time...
-
As the interest in ecosystem protection increases, many researchers are focusing on tracking flying insects to preserve biodiversity. Invasive alien species (IAS) such as Vespa velutina nigrithorax require extensive consideration in this regard owing to size and weight limitations. In this paper, we propose and experimentally validate an unmanned aerial vehicle (UAV)-based trilateration system for...
-
The development of autonomous rotary-wing UAVs has shown an evident tendency in miniaturization. However, the side effects brought by miniaturization, such as decreased load capability, shorter flight duration and reduced autonomous ability, seriously hinder its process. In this paper, we first investigate the configurations of different rotary-wing aircraft and optimize the configuration selectio...
-
This paper studies a novel application of an aerial manipulator (AM)-the contact-based landing on a mobile platform. An AM is inherently unstable, under-actuated, and usually loses some DOFs while contacting environments. Meanwhile, the AM's flight state is susceptible to uncertain movements of the mobile platform, such as acceleration, sudden stopping, and reversing. To accomplish the contact-bas...
-
Utilizing wind hovering techniques of soaring birds can save energy expenditure and improve the flight endurance of micro air vehicles (MAVs). Here, we present a novel method for fully autonomous orographic soaring without a priori knowledge of the wind field. Specifically, we devise an Incremental Nonlinear Dynamic Inversion (INDI) controller with control allocation, adapting it for autonomous so...
-
In many applications such as aerial transportation, delivery, and manipulation, it is essential to know the external wrench exerted on multirotor aerial vehicles precisely. This paper presents an algorithm to estimate external wrench using a rotor speed measurement unit, an inertial measurement unit and a motion capture system. Under a cascaded architecture containing two sub-systems, one error-st...
-
This study presents a new hardware design and control of a minimally actuated 5 control degrees of freedom (CDoF) quadrotor-based tiltrotor. The proposed tiltrotor possesses several characteristics distinct from those found in existing works, including: 1) minimal number of actuators for 5 CDoF, 2) large margin to generate interaction force during aerial physical interaction (APhI), and 3) no mech...
-
Autonomous Micro Aerial Vehicles (MAVs) such as quadrotors equipped with manipulation mechanisms have the potential to assist humans in tasks such as construction and package delivery. Cables are a promising option for manipulation mechanisms due to their low weight, low cost, and simple design. However, designing control and planning strategies for cable mechanisms presents challenges due to indi...
-
Using Unmanned Aerial Vehicles (UAVs) to per-form high-altitude manipulation tasks beyond just passive visual application can reduce the time, cost, and risk of human workers. Prior research on aerial manipulation has relied on either ground truth state estimate or GPS/total station with some Simultaneous Localization and Mapping (SLAM) algorithms, which may not be practical for many applications ...
-
We present a novel optimization algorithm called DroNeRF for the autonomous positioning of monocular camera drones around an object for real-time 3D reconstruction using only a few images. Neural Radiance Fields, or NeRF, is a novel view synthesis technique used to generate new views of an object or scene from a set of input images. Using drones in conjunction with NeRF provides a unique and dynam...
-
The safety-critical nature of vehicle steering is one of the main motivations for exploring the space of possible cyber-physical attacks against the steering systems of modern vehicles. This paper investigates the adversarial capabilities for destabilizing the interaction dynamics between human drivers and vehicle haptic shared control (HSC) steering systems. In contrast to the conventional roboti...
-
This paper proposes a novel Generative Adversarial Network (GAN)-based strategy to augment subjective haptic Quality of Experience (QoE) datasets for bilateral teleoperation with haptic feedback without conducting time-consuming subjective experiments. In our previous work, we proposed a multi-assessment fusion approach to predict subjective haptic quality using a collection of objective metrics. ...
-
Surgical simulators have been under development for years, formerly intended for surgical training and recently applied for training machine learning models. These systems often employ computationally efficient methods such as mass-spring models or position-based dynamics that prioritize real-time execution over physical accuracy, while the use of the finite element method (FEM) has been limited d...
-
Robotic object manipulation requires knowledge of the environment's state. In particular, the object poses of fixed elements in the environment relative to the robot and the in-hand poses of grasped objects are of interest. For insertion tasks with tight tolerances, the accuracy of vision systems to estimate the object and in-hand pose is not high enough. This work proposes a state estimation syst...
-
In this work, we evaluated the suitability of using a 7 degrees of freedom robotic manipulator as a planar haptic interface for studies in motor neuroscience. In particular, we assessed to what extent it can measure human movement and forces without applying undesired perturbations. To this aim, we evaluated the amount of perturbation exerted by the robot during planar reaching movements when cont...
-
Incorporating compliance into shape-changing displays can improve their wearability and actuation modalities. While recent advances in soft actuators highlight promising paths for soft shape-changing displays, these displays currently face some practical challenges of device failure and limited actuator displacement. A monolithic fabrication processes means the device is challenging to repair, for...
-
Robotic palpation shows significant potential to improve the accuracy and speed of tumor identification. How-ever, robotic palpation mechanisms often lack haptic feedback, making it difficult for the surgeon to identify variations in tissue stiffness. This paper presents a soft optical sensor integrated with a wearable haptic glove for tumor detection during robotic palpation. The sensor contains ...
-
Microsurgery involves the dexterous manipulation of delicate tissue or fragile structures, such as small blood vessels and nerves, under a microscope. To address the limitations of imprecise manipulation of human hands, robotic systems have been developed to assist surgeons in performing complex microsurgical tasks with greater precision and safety. However, the steep learning curve for robot-assi...
-
Real-time magnetic resonance imaging (MRI) in-terventions are significantly impacted by material compatibility problems and size constraints in the MRI bore. Limited physi-cian access to patients within the bore of the MRI necessitates iterative positioning and imaging, which prolongs the duration of the procedure and increases patient risk. We present a passive MR-safe haptic teleoperation device...
-
We propose novel symmetry-based modeling and hybrid orientation-force control frameworks for cutaneous haptic device (CHD) to generate precise three degree-of-freedom (DoF) contact force on the fingertip robustly against user variability. The CHD hardware is designed in a form of an underactuated cable-driven parallel mechanism, with springs placed along the tendon to stabilize the pose. We analyz...
-
We consider the problem of time-limited robotic exploration in previously unseen environments where exploration is limited by a predefined amount of time. We propose a novel exploration approach using learning-augmented model-based planning. We generate a set of sub goals associated with frontiers on the current map and derive a Bellman Equation for exploration with these subgoals. Visual sensing ...
-
Achieving high performance in deep reinforcement learning relies heavily on the ability to obtain good state representations from pixel inputs. However, learning an observation-space-to-action-space mapping from high-dimensional inputs is challenging in reinforcement learning, particularly when dealing with consecutive depth images as input states. In addition, we observe that the consecutive inpu...
-
Deep reinforcement learning has achieved signif-icant results in low-level controlling tasks. However, for some applications like autonomous driving and drone flying, it is difficult to control behavior stably since the agent may suddenly change its actions which often lowers the controlling sys-tem's efficiency, induces excessive mechanical wear, and causes uncontrollable, dangerous behavior to t...
-
Localization is a fundamental task in robotics for autonomous navigation. Existing localization methods rely on a single input data modality or train several computational models to process different modalities. This leads to stringent computational requirements and sub-optimal results that fail to capitalize on the complementary information in other data streams. This paper proposes UnLoc, a nove...
-
Recently several fusion and switching based approaches have been presented to solve the problem of Visual Place Recognition. In spite of these systems demonstrating significant boost in VPR performance they each have their own set of limitations. The multi-process fusion systems usually involve employing brute force and running all available VPR techniques simultaneously while the switching method...
-
Household robots operate in the same space for years. Such robots incrementally build dynamic maps that can be used for tasks requiring remote object localization. However, benchmarks in robot learning often test generalization through inference on tasks in unobserved environments. In an observed environment, locating an object is reduced to choosing from among all object proposals in the environm...
-
PLPL-VIO: A Novel Probabilistic Line Measurement Model for Point-Line-Based Visual-Inertial Odometry
Point and line features are complementary in Visual-Inertial Odometry (VIO) or Visual-Inertial Simultaneous Localization And Mapping (VI-SLAM) systems. The advantage of combining these two types of features relies on their proper weighting in the cost function, usually set by their uncertainty. Compared with point features, setting line segment endpoints' uncertainty with isotropic distribution is...
-
Over the past few years, there has been a great deal of research on navigation tasks in indoor environments using deep reinforcement learning agents. Most of these tasks use only visual information in the form of first-person images to navigate to a single goal. More recently, tasks that simultaneously use visual and auditory information to navigate to the sound source and even navigation tasks wi...
-
Automated Guided Vehicles (AGV) are omnipresent, and are able to carry out various kind of preprogrammed tasks. Unfortunately, a lot of manual configuration is still required in order to make these systems operational, and configuration needs to be re-done when the environment or task is changed. As an alternative to current inflexible methods, we employ a learning based method in order to perform...
-
Navigation of terrestrial robots is typically addressed either with localization and mapping (SLAM) followed by classical planning on the dynamically created maps, or by machine learning (ML), often through end-to-end training with reinforcement learning (RL) or imitation learning (IL). Recently, modular designs have achieved promising results, and hybrid algorithms that combine ML with classical ...
-
The development of effective vision-based algorithms has been a significant challenge in achieving autonomous drones, which promise to offer immense potential for many real-world applications. This paper investigates learning deep sensorimotor policies for vision-based drone racing, which is a particularly demanding setting for testing the limits of an algorithm. Our method combines feature repres...
-
Indoor magnetic fields are a combination of Earth's magnetic field and disruptions induced by ferromag-netic objects, such as steel structural components in buildings. As a result of these disruptions, pervasive in indoor spaces, mag-netic field data is often omitted from navigation algorithms in indoor environments. This paper leverages the spatially-varying disruptions to Earth's magnetic field ...
-
Most 6-DoF localization and SLAM systems use static landmarks but ignore dynamic objects because they cannot be usefully incorporated into a typical pipeline. Where dynamic objects have been incorporated, typical approaches have attempted relatively sophisticated identification and localization of these objects, limiting their robustness or general utility. In this research, we propose a middle gr...
-
Ultra-wideband (UWB) time difference of arrival (TDOA)-based localization has emerged as a low-cost and scalable indoor positioning solution. However, in cluttered environments, the performance of UWB TDOA-based localization deteriorates due to the biased and non-Gaussian noise distributions induced by obstacles. In this work, we present a bi-level optimization-based joint localization and noise m...
-
Mutual localization plays a crucial role in multi-robot cooperation. CREPES, a novel system that focuses on six degrees of freedom (DOF) relative pose estimation for multi-robot systems, is proposed in this paper. CREPES has a compact hardware design using active infrared (IR) LEDs, an IR fish-eye camera, an ultra-wideband (UWB) module and an inertial measurement unit (IMU). By leveraging IR light...
-
The ability to rapidly test a variety of algorithms for an arbitrary state estimation task is valuable in the prototyping phase of navigation systems. Lie group theory is now mainstream in the robotics community, and hence estimation prototyping tools should allow state definitions that belong to manifolds. A new package, called navlie, provides a framework that allows a user to model a large clas...
-
Decentralized and autonomous control of Unmanned Aerial Vehicle (UAV) swarms is a key enabler for cooperative systems and infrastructure-less formation flights. However, UAVs often lack reliable heading angle measurements, especially in indoor scenarios, space, and GNSS-denied environments, posing an additional observability challenge on range-based relative localization. We tackle this problem by...
-
This paper presents a new information source for supporting robot localisation: material composition. The proposed method complements the existing visual, structural, and semantic cues utilized in the literature. However, it has a distinct advantage in its ability to differentiate structurally [23], visually [25] or categorically [1] similar objects such as different doors, by using Raman spectrom...
-
In this paper, we present a fast, lightweight odometry method that uses the Doppler velocity measurements from a Frequency-Modulated Continuous-Wave (FMCW) lidar without data association. FMCW lidar is a recently emerging technology that enables per-return relative radial velocity measurements via the Doppler effect. Since the Doppler measurement model is linear with respect to the 6-degrees-of-fr...
-
In the context of robotics, accurate ground truth positioning is essential for the development of Simultaneous Localization and Mapping (SLAM) and control algorithms. Robotic Total Stations (RTSs) provide accurate and precise reference positions in different types of outdoor environments, especially when compared to the limited accuracy of Global Navigation Satellite System (GNSS) in cluttered are...
-
Point Cloud Registration is a fundamental and challenging problem in 3D computer vision. Recent works often utilize geometric structure features in downsampled points (patches) to seek correspondences, then propagate these sparse patch correspondences to the dense level in the corresponding patches' neighborhood. However, they neglect the explicit global scale rigid constraint at the dense level p...
-
The availability of accurate localization is critical for multi-robot exploration strategies; noisy or inconsistent localization causes failure in meeting exploration objectives. We aim to achieve high localization accuracy with contemporary exploration map belief and vice versa without needing global localization information. This paper proposes a novel simultaneous exploration and localization (...
-
Discrete-time architectures have an advantage over their continuous counterparts as they can be directly executed on embedded hardware without the need for dis-cretization, and discretization can result in a loss of stability margin. This paper presents a discrete-time adaptive control architecture for uncertain scalar multi agent systems with coupled dynamics. Our strategy includes observer dynam...
-
Effective coordination of heterogeneous multi-robot teams requires optimizing allocations, schedules, and motion plans in order to satisfy complex multi-dimensional task requirements. This challenge is exacerbated by the fact that real-world applications inevitably introduce uncertainties into robot capabilities and task requirements. In this paper, we extend our previous work on trait-based time-...
-
We focus on the problem of rearranging a set of objects with a team of car-like robot pushers built using off-the-shelf components. Maintaining control of pushed objects while avoiding collisions in a tight space demands highly coordinated motion that is challenging to execute on constrained hardware. Centralized replanning approaches become intractable even for small-sized problems whereas decent...
-
This paper introduces a game-theoretical approach to the multi-robot task allocation problem, where each robot is considered as self-interested and cannot share its personal utility functions. We consider the case where each robot can execute multiple tasks and each task requires only one robot. For real-world applications with mobile robots, we design a utility function that includes both assignm...
-
Planning for multi-robot teams in complex environments is a challenging problem, especially when these teams must coordinate to accomplish a common objective. In general, optimal solutions to these planning problems are computationally intractable, since the decision space grows exponentially with the number of robots. In this paper, we present a novel approach for multi-robot planning on topologi...
-
We present an approach to task scheduling in heterogeneous multi-robot systems. In our setting, the tasks to complete require diverse skills. We assume that each robot is multi-skilled, i.e., each robot offers a subset of the possible skills. This makes the formation of heterogeneous teams (coalitions) a requirement for task completion. We present two centralized algorithms to schedule robots acro...
-
During a natural disaster such as hurricane, earthquake, or fire, robots have the potential to explore vast areas and provide valuable aid in search & rescue efforts. These scenarios are often high-pressure and time-critical with dynamically-changing task goals. One limitation to these large scale deployments is effective human-robot interaction. Prior work shows that collaboration between one hum...
-
Electric autonomous vehicles (EAVs) are getting attention in future autonomous mobility-on-demand (AMoD) systems due to their economic and societal benefits. However, EAVs' unique charging patterns (long charging time, high charging frequency, unpredictable charging behaviors, etc.) make it challenging to accurately predict the EAVs supply in E-AMoD systems. Furthermore, the mobility demand's pred...
-
Deep learning-based models are at the top of most driver observation benchmarks due to their remarkable accuracies but come with a high computational cost, while the resources are often limited in real-world driving scenarios. This paper presents a lightweight framework for resource- efficient driver activity recognition. We enhance 3D MobileNet, a speed-optimized neural architecture for video cla...
-
Despite the significant research efforts on trajectory prediction for automated driving, limited work exists on assessing the prediction reliability. To address this limitation we propose an approach that covers two sources of error, namely novel situations with out-of-distribution (OOD) detection and the complexity in in-distribution (ID) situations with uncertainty estimation. We introduce two m...
-
Nowadays, transportation networks face the challenge of sub-optimal control policies that can have adverse effects on human health, the environment, and contribute to traffic congestion. Increased levels of air pollution and extended commute times caused by traffic bottlenecks make intersection traffic signal controllers a crucial component of modern transportation infrastructure. Despite several ...
-
The accurate prediction of the neighboring vehicles' trajectories affects the security of autonomous driving vehicles. However, it is challenging for existing methods to anticipating the trajectories of vehicles in the vicinity due to the uncertainty of driving behaviors and the complex interaction patterns of traffic flows. In this study, incorporating the planning information of the ego vehicle,...
-
Multi-Agent Reinforcement Learning (MARL) has become a promising solution for constructing a multi-agent autonomous driving system (MADS) in complex and dense scenarios. But most methods consider agents acting selfishly, which leads to conflict behaviors. Some existing works incorporate the concept of social value orientation (SVO) to promote coordination, but they lack the knowledge of other agen...
-
A platoon refers to a group of vehicles traveling together in very close proximity using automated driving technology. Owing to its immense capacity to improve fuel efficiency, driving safety, and driver comfort, platooning technology has garnered substantial attention from the autonomous vehicle research community. Although highly advantageous, recent research has uncovered that an excessively sm...
-
Motion prediction is crucial in enabling safe motion planning for autonomous vehicles in interactive scenarios. It allows the planner to identify potential conflicts with other traffic agents and generate safe plans. Existing motion predictors often focus on reducing prediction errors, yet it remains an open question on how well they help identify conflicts for the planner, which are critical to t...
-
Is mobile robot navigation a solved problem? We asked this question to 14 professional robot software engineers who work with navigation stacks of mobile, wheeled robots on a daily basis. They unanimously report that it remains challenging to ensure the performance of their mobile robots. We find that the method of choice to verify a robot's performance is to expose it to different environments un...
-
A key aspect of driving a road vehicle is to interact with other road users, assess their intentions and make riskaware tactical decisions. An intuitive approach to enabling an intelligent automated driving system would be incorporating some aspects of human driving behavior. To this end, we propose a novel driving framework for egocentric views based on spatio-temporal traffic graphs. The traffic...
-
Predicting the future trajectories of other agents in the scene fast and effectively is crucial for autonomous driving systems. We note that high-quality predictions require us to take into account the subjective initiative of the target agents, which is reflected by the fact that they themselves make decisions based on their own predictions about the future, just like our ego vehicle's prediction...
-
The safety of autonomous vehicles (AVs) depends on their ability to perform complex computations on high-volume sensor data in a timely manner. Their ability to run these computations with state-of-the-art models is limited by the processing power and slow update cycles of their onboard hardware. In contrast, cloud computing offers the ability to burst computation to vast amounts of the latest gen...
-
Reinforcement learning (RL) has emerged as a promising approach for optimizing traffic signal control (TSC) to ensure the efficient operation of transportation networks. However, the traditional trial-and-error technique in RL is usually impractical in real-world applications. Offline RL, which trains models using pre-collected datasets, is a more practical approach. However, this presents challen...
-
Meta-reinforcement learning (meta-RL) is a promising approach that enables the agent to learn new tasks quickly. However, most meta-RL algorithms show poor generalization in multi-task scenarios due to the insufficient task information provided only by rewards. Language-conditioned meta-RL improves the generalization capability by matching language instructions with the agent's behaviors. While bo...
-
An emerging field of sequential decision problems is safe Reinforcement Learning (RL), where the objective is to maximize the reward while obeying safety constraints. Being able to handle constraints is essential for deploying RL agents in real-world environments, where constraint violations can harm the agent and the environment. To this end, we propose a safe model-free RL algorithm with a novel...
-
For multi-agent area coverage path planning problem, existing researches regard it as a combination of Traveling Salesman Problem (TSP) and Coverage Path Planning (CPP). However, these approaches have disadvantages of poor observation ability in online phase and high computational cost in offline phase, making it difficult to be applied to energy-constrained Unmanned Aerial Vehicles (UAVs) and adj...
-
Reinforcement learning for swarms of flying robots is a challenging task that requires a large number of data samples. Moreover, the problem of sim-to-real transfer has long been a challenge in robotics algorithm deployment. To address these issues, we propose Air-M, a platform that facilitates large-scale drone swarm learning in a distributed docker container environment and deployment in a virtu...
-
Developing robot controllers capable of achieving dexterous nonprehensile manipulation, such as pushing an object on a table, is challenging. The underactuated and hybrid-dynamics nature of the problem, further complicated by the uncertainty resulting from the frictional interactions, requires sophisticated control behaviors. Reinforcement Learning (RL) is a powerful framework for developing such ...
-
Collaborative autonomous multi-agent systems covering a specified area have many potential applications. Traditional approaches for such problems involve designing model-based control policies; however, state-of-the-art classical control policy still exhibits a large degree of sub-optimality. We present a combined reinforcement learning (RL) and control approach for the multi-agent coverage proble...
-
Domain randomization (DR) is a powerful framework that has allowed the transfer of policies from randomized domain (a.k.a. simulation) to real robots with little to no retraining requirement. However, because the policy has to perform well for many different domain conditions, DR tends to produce sub-optimal policies that can be too conservative on the target real system. This problem is further e...
-
Recent research has demonstrated the potential of reinforcement learning (RL) in enabling effective multi-robot collaboration, particularly in social dilemmas where robots face a trade-off between self-interests and collective benefits. However, environmental factors such as miscommunication and adversarial robots can impact cooperation, making it crucial to explore how multi-robot communication c...
-
Electric vehicles (EVs) play critical roles in autonomous mobility-on-demand (AMoD) systems, but their unique charging patterns increase the model uncertainties in AMoD systems (e.g. state transition probability). Since there usually exists a mismatch between the training and test/true environments, incorporating model uncertainty into system design is of critical importance in real-world applicat...
-
Many state-of-the-art robotic applications utilize series elastic actuators (SEAs) with closed-loop force control to achieve complex tasks such as walking, lifting, and manipulation. Model-free PID control methods are more prone to instability due to nonlinearities in the SEA where cascaded model-based robust controllers can remove these effects to achieve stable force control. However, these mode...
-
This paper presents an approach to design a reward function by adopting both control theoretic and biomechanical perspectives. In reinforcement learning (RL), a reward function plays a crucial role for an RL agent training; especially, a task learning time and a task performance. Accordingly, designing a reward function becomes a key issue to train an RL agent generating human-like policy/strategy...
-
Recent works in object detection in LiDAR point clouds mostly focus on predicting bounding boxes around objects. This prediction is commonly achieved using anchor-based or anchor-free detectors that predict bounding boxes, requiring significant explicit prior knowledge about the objects to work properly. To remedy these limitations, we propose MaskBEV, a bird's-eye view (BEV) mask-based object det...
-
Object detection plays an important role in computer vision tasks such as autonomous driving, robotics, etc. Typically, a detection model is firstly trained on collected data and then deployed in real world. However, the discrepancy exists between training (source) and testing (target) data, which degrades the detection model's performance in the real world. To mitigate the negative effects, Unsup...
-
Affordance detection is a challenging problem with a wide variety of robotic applications. Traditional affordance detection methods are limited to a predefined set of affordance labels, hence potentially restricting the adaptability of intelligent robots in complex and dynamic environments. In this paper, we present the Open-Vocabulary Affordance Detection (OpenAD) method, which is capable of dete...
-
Uncertainty estimation is crucial in safety-critical settings such as automated driving as it provides valuable information for several downstream tasks including high-level decision making and path planning. In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framework using evidential learning to directly estimate both classification and regression uncertainties. ...
-
LiDAR and cameras are two essential sensors for 3D object detection in autonomous driving. LiDAR provides accurate and reliable 3D geometry information while the camera provides rich texture with color. Despite the increasing popularity of fusing these two complementary sensors, the challenge remains in how to effectively fuse 3D LiDAR point cloud with 2D camera images. Recent methods focus on poi...
-
High-performance real-time 3D object detection is crucial in autonomous driving perception systems. Voxel-or point-based 3D object detectors are highly accurate but inefficient and difficult to deploy, while other methods use 2D projection views to improve efficiency, but information loss usually degrades performance. To balance effectiveness and efficiency, we propose a scheme called RFDNet that ...
-
This paper presents a novel method for object-level unknown obstacle detection in driving scenes that reduces false positives. The proposed method combines existing anomaly detectors, depth estimation, and object detection techniques to achieve object-level predictions. Our method can predict anomalies as bound-box instance detections. These bounding boxes can then be used to refine anomaly detect...
-
The progress of LiDAR-based 3D object detection has significantly enhanced developments in autonomous driving and robotics. However, due to the limitations of LiDAR sensors, object shapes suffer from deterioration in occluded and distant areas, which creates a fundamental challenge to 3D perception. Existing methods estimate specific 3D shapes and achieve remarkable performance. However, these met...
-
3D object detection have achieved significant performance in many fields, e.g., robotics system, autonomous driving, and augmented reality. However, most existing methods could cause catastrophic forgetting of old classes when performing on the class-incremental scenarios. Meanwhile, the current class-incremental 3D object detection methods neglect the relationships between the object localization...
-
Spin plays a considerable role in table tennis, making a shot's trajectory harder to read and predict. However, the spin is challenging to measure because of the ball's high velocity and the magnitude of the spin values. Existing methods either require extremely high framerate cameras or are unreliable because they use the ball's logo, which may not always be visible. Because of this, many table t...
-
Robots operating in human-centered environments, such as retail stores, restaurants, and households, are often required to distinguish between similar objects in different contexts with a high degree of accuracy. However, fine-grained object recognition remains a challenge in robotics due to the high intra-category and low inter-category dissimilarities. In addition, the limited number of fine-gra...
-
The adversarial robustness of a model is its ability to resist adversarial attacks in the form of small perturbations to input data. Universal adversarial attack methods such as Fast Sign Gradient Method (FSGM) [1] and Projected Gradient Descend (PGD) [2] are popular for LiDAR object detection, but they are often deficient compared to task-specific adversarial attacks. Additionally, these universa...
-
Supervised 3D Object Detection models have been displaying increasingly better performance in single-domain cases where the training data comes from the same environment and sensor as the testing data. However, in real-world scenarios data from the target domain may not be available for finetuning or for domain adaptation methods. Indeed, 3D object detection models trained on a source dataset with...
-
Binocular depth estimation is a fundamental problem in computer vision. Learning-based models have achieved significant performance improvements on public datasets in recent years. Our study finds that the performance of the current state-of-the-art deep learning-based models deteriorates significantly in distant areas. We point out that these deep learning-based models suffer from a scale shrinka...
-
Current parking slot detection in advanced driver-assistance systems (ADAS) primarily relies on ultrasonic sen-sors. This method has several limitations such as the need to scan the entire parking slot before detecting it, the incapacity of detecting multiple slots in a row, and the difficulty of classifying them. Due to the complex visual environment, vehicles are equipped with surround view came...
-
Point cloud registration (PCR) aims to recover the rigid transformation between two noisy, unordered point sets. This task is typically tackled by establishing point-wise correspondences, and solving the rigid transformation between the two sets. Since descriptor-based methods find correspondences by matching the feature space distance, a powerful and rotation-robust point feature extractor is cri...
-
Vision research showed remarkable success in understanding our world, propelled by datasets of images and videos. Sensor data from radar, LiDAR and cameras supports research in robotics and autonomous driving for at least a decade. However, while visual sensors may fail in some conditions, sound has recently shown potential to complement sensor data. Simulated room impulse responses (RIR) in 3D ap...
-
Onboard sensors, such as cameras and thermal sensors, have emerged as effective alternatives to Global Positioning System (GPS) for geo-Iocalization in Unmanned Aerial Vehicle (UAV) navigation. Since GPS can suffer from signal loss and spoofing problems, researchers have explored camera-based techniques such as Visual Geo-Iocalization (VG) using satellite RGB imagery. Additionally, thermal geo-Ioc...
-
This work proposes the use of conditional Generative Adversarial Networks (cGANs) for acoustic-based 3D reconstruction. Acoustics being the most reliable sensor modality in underwater domains is accompanied with the loss of elevation angle in its images. The challenge of recovering the missing dimension in acoustic images have pushed researchers to try various methods and approaches over the past ...
-
In this paper, we tackle the problem of generating a novel image from an arbitrary viewpoint given a single frame as input. While existing methods operating in this setup aim at predicting the target view depth map to guide the synthesis, without explicit supervision over such a task, we jointly optimize our framework for both novel view synthesis and depth estimation to unleash the synergy betwee...
-
It is imperative that robots can understand natural language commands issued by humans. Such commands typically contain verbs that signify what action should be performed on a given object and that are applicable to many objects. We propose a method for generalizing manipulation skills to novel objects using verbs. Our method learns a probabilistic classifier that determines whether a given object...
-
This paper describes a method of domain adap-tive training for semantic segmentation using multiple source datasets that are not necessarily relevant to the target dataset. We propose a soft pseudo-label generation method by integrating predicted object probabilities from multiple source models. The prediction of each source model is weighted based on the estimated domain similarity between the so...
-
To automatically localize a target object in an image is crucial for many computer vision applications. To represent the 2D object, ellipse labels have recently been identified as a promising alternative to axis-aligned bounding boxes. This paper further considers 3D-aware ellipse labels, i.e., ellipses which are projections of a 3D ellipsoidal approximation of the object, for 2D target localizati...
-
Common image-based LiDAR point cloud semantic segmentation (LiDAR PCSS) approaches have bottlenecks resulting from the boundary-blurring problem of convolution neural networks (CNNs) and quantitation loss of spherical projection. In this work, we propose a transformer-based plug-and-play uncertain point refiner, i.e., TransUPR, to refine selected uncertain points in a learnable manner, which leads...
-
In this paper, Safety Control Barrier Functions (SCBFs) were used to adjust the null-space compliant behavior of a redundant robot to improve safety in Human-Robot Collaboration (HRC) without modifying the robot behavior with respect to its main Cartesian task. A Lyapunov function was included in an energy storage formulation compatible with strict passivity to provide global asymptotic stability ...
-
Parallel robots (PRs) allow for higher speeds in human-robot collaboration due to their lower moving masses but are more prone to unintended contact. For a safe reaction, knowledge of the location and force of a collision is useful. A novel algorithm for collision isolation and identification with proprioceptive information for a real PR is the scope of this work. To classify the collided body, th...
-
The ability to anticipate surrounding agents' behaviors is critical to enable safe and seamless autonomous vehicles (AVs). While phenomenological methods have successfully predicted future trajectories from scene context, these predictions lack interpretability. On the other hand, ontological approaches assume an underlying structure able to describe the interaction dynamics or agents' internal de...
-
One way of ensuring operator's safety during human-robot collaboration is through Speed and Separation Monitoring (SSM), as defined in ISO standard ISO/TS 15066. In general, it is impossible to avoid all human-robot collisions: consider for instance the case when the robot does not move at all, a human operator can still collide with it by hitting it of her own voluntary motion. In the SSM framewo...
-
Obtaining reliable detections of a human is crucial for many safety-related robotic tasks. This can be done by human pose estimation methods, which predict the position of several different keypoints of the human body. In most cases, recent approaches based on neural networks produce ‘good’ results, i.e. predictions with small localization errors, however, large errors do also occur. For an indivi...
-
This paper presents an effective approach to enable performance improvement in human-robot collaboration scenarios. The problem is tackled from the perspective of speed and separation monitoring principle, which stems from the recently instituted safety standard. The proposed approach attempts to seek for performance gains, measured by the speed-up of the production cycle, without compromising the...
-
This paper describes the hardware implementation and characterization of a capacitive sensor designed to support detection and localization of nearby objects. The sensor can be mounted on the exterior of any given robotic system. The technology is particularly well-suited to detection of capacitive material, such as living tissue. As such, it offers perspectives of facilitating human-robot interac...
-
Two regimes permitting safe physical human-robot interaction, speed and separation monitoring and safety-rated monitored stop, depend on reliable perception of the space surrounding the robot. This can be accomplished by visual sensors (like cameras, RGB-D cameras, LIDARs), proximity sensors, or dedicated devices used in industrial settings like pads that are activated by the presence of the opera...
-
Parallel robots (PRs) offer the potential for safe human-robot collaboration because of their low moving masses. Due to the in-parallel kinematic chains, the risk of contact in the form of collisions and clamping at a chain increases. Ensuring safety is investigated in this work through various contact reactions on a real planar PR. External forces are estimated based on proprioceptive information...
-
In this paper, we present AmbiSense, an acoustic field based sensing system that performs proximity detection and bearing estimation for safer physical human-robot interactions. A single low cost piezoelectric transducer is used to setup this novel acoustic sensing modality to create a blindspot-free sound field engulfing a robot arm. Two detection algorithms leveraging spectral information from r...
-
Since modern robots are supposed to work closely together with humans, physical human-robot interaction is gaining importance. One crucial aspect for safe collaboration is a robust collision reaction strategy that is triggered after an unintentional physical contact. In this work, we propose a dynamically-consistent collision reaction controller, where the reactive motion is performed in one parti...
-
Mobile robots functioning in human environments should behave with a secure and socially-compliant manner. Although many studies have revealed the effectiveness of Deep Reinforcement Learning (DRL) in robot navigation, most of them can only handle the presence of human as independent individuals. Failing to consider groups may lead to the robot getting stuck or behaving rudely, and omitting to sep...
-
The effectiveness of robotic exoskeletons for post-stroke gait rehabilitation might be limited as the control parameters of these devices do not adapt to key biomechanical descriptors. The main contribution of this study is to examine post-stroke gait with the aim of finding relationships between exoskeleton control parameters and a comprehensive set of biomechanical metrics. Five stroke survivors...
-
Robot-assisted rehabilitation is expected to reduce locomotor limitations of children and young adults with Cerebral Palsy (CP). However, to achieve this result, it is essential that the robot is transparent, allowing the user to move freely, and generate joint torques only when the exoskeleton joints significantly deviate from physiological gait patterns. Nevertheless, the development of transpar...
-
Emerging partial-assistance exoskeletons can enhance able-bodied performance and aid people with patho-logical gait or age-related immobility. However, every person walks differently, which makes it difficult to directly compute assistance torques from joint kinematics. Gait-state estimation-based controllers use phase (normalized stride time) and task variables (e.g., stride length and ground inc...
-
Upright locomotion has many health benefits for patients with spinal cord injury. Passive gait orthoses, such as an isocentric reciprocal gait orthosis (IRGO), allow patients to walk by pushing themselves forward with forearm supports. To move the legs, the IRGO physically couples the motion of stance and swing leg through a linkage. Unfortunately, this locomotion is associated with high metabolic...
-
Robotic ankle exoskeletons have been shown to reduce human effort during walking. However, existing ankle exoskeleton control approaches are limited in their ability to apply biomimetic torque across diverse tasks outside of the controlled lab environment. Energy shaping control can provide task-invariant assistance without estimating the user's state, classifying task, or reproducing pre-defined ...
-
The assistive profile of an active back support exoskeleton is strongly dependent on the manual tuning of controller gains based on previous experience and trial-and-error. Human-in-the-loop (HIL) optimization allows for automatic tuning of assistive profiles to different subjects. Most HIL methods make use of intrusive sensors that could affect out-of-the-lab exoskeleton adoption. Therefore, we p...
-
Maintaining balance during gait in the mediolateral direction requires more active motor control than in the anteroposterior direction. Step width modulation is a key strategy used by healthy individuals to achieve mediolateral walking balance, but it can be disrupted in populations with poor sensorimotor integration and weak hip abductors, such as the elderly, stroke patients, and people with low...
-
Manual materials handling occupations often involve repetitive lifting, lowering, and carrying motions, which can lead to muscular fatigue and/or injury. The risk increases when loads must be worn on the body for the entirety of a job shift. Exoskeletons have been developed to assist these types of motions, but require the user to bear the weight of a load through their body. Load carriage exoskel...
-
Exoskeleton robots are capable of safe torque-controlled interactions with a wearer while moving their limbs through predefined trajectories. However, affecting and assisting the wearer's movements while incorporating their inputs (effort and movements) effectively during an interaction re-mains an open problem due to the complex and variable nature of human motion. In this paper, we present a con...
-
Exoskeletons have shown great potential to enhance locomotion by augmenting the lower limb. While most research has focused on steady-state ambulatory activities, the ability to assist transient, ballistic tasks is also important for understanding the potential of exoskeletons in mobility enhancement. In this preliminary study (N = 5), we developed an individually-customized control strategy to as...
-
Healthy human locomotion functions with good gait symmetry depend on rhythmic coordination of the left and right legs, which can be deteriorated by neurological disorders like stroke and spinal cord injury. Powered exoskeletons are promising devices to improve impaired people's locomotion functions, like gait symmetry. However, given higher uncertainties and the time-varying nature of human-robot ...
-
2D forward-looking sonar is a crucial sensor for underwater robotic perception. A well-known problem in this field is estimating missing information in the elevation direction during sonar imaging. There are demands to estimate 3D information per image for 3D mapping and robot navigation during fly-through missions. Recent learning-based methods have demonstrated their strengths, but there are sti...
-
This paper presents MIMIR-UW, a multipurpose underwater synthetic dataset for SLAM, depth estimation, and object segmentation to bridge the gap between theory and application in underwater environments. MIMIR-UW integrates three camera sensors, inertial measurements, and ground truth for robot pose, image depth, and object segmentation. The underwater robot is deployed within a pipe exploration sc...
-
Reliable monitoring of vessel motions is crucial for safe and efficient operation of marine vessels. Pitching and rolling motions are commonly monitored using high-grade inertial measurement units (IMUs). However, such sensors become unreliable in presence of long-lasting accelerations. In this work, we propose a method for attitude estimation of marine vessels relying on an image stream and known...
-
The air-sea interface is vital in studying heat and energy exchange between the sea and air. The field observation technology of the air-sea interface is an effective way to explore the nature of the air-sea interface. This paper presents an observation system called DANDELION for the air-sea interface environment. The system includes an automatic surface vehicle (ASV), a launching device using a ...
-
In-situ observation of the deep-sea floor is a fundamental need for marine sciences and ecosystem monitoring. This work proposes a novel robotic approach for benthic observations in the deep sea using underwater gliders. The glider is equipped with a downward looking camera system to acquire high resolution optical images of the seafloor. The system works fully autonomous and has the potential for...
-
Operating in the near-vicinity of marine energy devices poses significant challenges to the control of underwater vehicles, predominantly due to the presence of large magnitude wave disturbances causing hazardous state perturbations. Approaches to tackle this problem have varied, but one promising solution is to adopt predictive control methods. Given the predictable nature of ocean waves, the pot...
-
In this paper we report an experimental evaluation of three popular methods for online system identification of unmanned surface vehicles (USVs) which were implemented as an ensemble: certifiably stable shallow recurrent neural network (RNN), adaptive identification (AID), and recursive least squares (RLS). The algorithms were deployed on eight USVs for a total of 30 hours of online estimation. Du...
-
Autonomous navigation of Unmanned Surface Vehicles (USV) in marine environments with current flows is challenging, and few prior works have addressed the sensor-based navigation problem in such environments under no prior knowledge of the current flow and obstacles. We propose a Distributional Reinforcement Learning (RL) based local path planner that learns return distributions which capture the u...
-
Exploration and monitoring of hazardous environments, such as legacy nuclear storage ponds, constitute safety-critical missions to be performed by small-scale underwater robots. These monitoring tasks require fully actuated robot platforms in order to allow for hovering while inspecting objects of interest in detail. A severe bottleneck arises from the restricted access points commonly encountered...
-
This paper presents a water walking robot with 6 feet which consists of a rimless wheel and a flywheel, and has a foot attached to the tip of each leg. First, in order to make the robot walk on water, we propose a foot control method by imitating the legs of some animals that can walk on water. Second, in order to increase the robot's forward speed, we improved the control method. In addition we h...
-
We present the Perseus autonomous underwater vehicle (AUV) - an A-sized aaA-size stands for the standard sonobuoy [2] form factor, with a maximum diameter of 124 mm and a length of around 0.9 m, ensuring the ability to launch from standard sonobuoy launchers onboard a wide array of fixed wing and rotary wing air crafts, surface ships and submarines [3] micro AUV, outfitted with a low-cost passive ...
-
Consumer robots can accompany children growing up, improving their abilities while playing and entertaining. This paper presents an open-source, practical, low-cost robotic Chinese chess player. The proposed system includes an elaborate mechanical structure, a simple kinematic solution, a novel robot operating system, real-time and accurate chess recognition. Regarding its mechanical design, it co...
-
This work presents the ETAUS, an Edge and Trustworthy AI UAV System, as a mobile sensing platform for air quality monitoring. ETAUS employs an FPGA device as the main hardware computing architecture rather than relying solely on a microprocessor or integrating with GPUs to meet real-time processing demands and achieve adaptivity and scalability. ETAUS contains a neural engine that can execute our ...
-
Modern computing platforms for robotics applications comprise a set of heterogeneous elements, e.g., multi-core CPUs, embedded GPUs, and FPGAs. FPGAs are reprogrammable hardware devices that allow for fast and energy-efficient computation of many relevant tasks in robotics. ROS is the de-facto programming standard for robotics and decomposes an application into a set of communicating nodes. ReconR...
-
Future robotics systems aiming for true autonomy must be robust against dynamic and unstructured environments. The 5th generation (5G) mobile network is expected to provide ubiquitous, reliable and low-latency wireless communications to ground robots, especially in outdoor scenarios. Empowered by 5G, the digital transformation of robotics is emerging, enabled by the cloud-native paradigm and the a...
-
The need for autonomous robot systems in both the service and the industrial domain is larger than ever. In the latter, the transition to small batches or even “batch size 1” in production created a need for robot control system architectures that can provide the required flexibility. Such architectures must not only have a sufficient knowledge integration framework. It must also support autonomou...
-
Piecewise-affine (PWA) systems are widely used for modeling and control of robotics problems including modeling contact dynamics. A common approach is to encode the control problem of the PWA system as a Mixed-Integer Convex Program (MICP), which can be solved by general-purpose off-the-shelf MICP solvers. To mitigate the scalability challenge of solving these MICP problems, existing work focuses ...
-
This paper introduces a novel, small form-factor, aerial vehicle research platform for agile object detection, classification, tracking, and interaction tasks. General-purpose hardware components were designed to augment a given aerial vehicle and enable it to perform safe and reliable grasping. These components include a custom collision tolerant cage and low-cost Gripper Extension Package, which...
-
We present Ungar, an open-source library to aid the implementation of high-dimensional optimal control problems (OCPs). We adopt modern template metaprogramming techniques to enable the compile-time modeling of complex systems while retaining maximum runtime efficiency. Our framework provides syntactic sugar to allow for expressive formulations of a rich set of structured dynamical systems. While ...
-
High-dimensional motion generation requires nu-merical precision for smooth, collision-free solutions. Typically, double-precision or single-precision floating-point (FP) formats are utilized. Using these for big tensors imposes a strain on the memory bandwidth provided by the devices and alters the memory footprint, hence limiting their applicability to low-power edge devices needed for mobile ro...
-
The advent of collaborative industrial and house-hold robotics has blurred the demarcation between the human and robot workspace. The capability of robots to function efficiently alongside humans requires new research to be conducted in dynamic environments as opposed to the traditional well-structured laboratory. In this work, we propose an efficient skill transfer methodology comprising intuitiv...
-
The cooperation of a human pilot with an autonomous agent during flight control realizes parallel autonomy. We propose an air-guardian system that facilitates cooperation between a pilot with eye tracking and a parallel end-to-end neural control system. Our vision-based air-guardian system combines a causal continuous-depth neural network model with a cooperation layer to enable parallel autonomy ...
-
Visual servoing represents a control strategy capable of driving dynamical systems from the current to the desired pose, when the only available information is the images generated at both poses. In this work, we analyze vulnerability of such systems and introduce two types of attacks to deceive visual servoing controller within a wheeled mobile robotic system. The attack goal is to alter the visu...
-
In this paper, we present an event-based control framework for the efficient tracking of contour-based areas, such as road pavements, using a multirotor aerial vehicle equipped with a bio-inspired Dynamic Vision Sensor (DVS). Concerning the detection part, the DVS camera captures events, which are asynchronously fed into a Neuromorphic Hough Transform algorithm running on a SpiNN-3 board and imple...
-
In this paper, we propose a physics-based robot controller to deform a soft object toward a desired 3D shape using a limited number of handling points. For this purpose, the shape of the deformable object is represented using Fourier descriptors. We derive the analytical relation that provides the variation of the Fourier coefficients as a function of the movements of the handling points by consid...
-
This work proposes a fast deployment pipeline for visually-servoed robots which does not assume anything about either the robot - e.g. sizes, colour or the presence of markers - or the deployment environment. Specifically, we apply a learning based approach to reliably estimate the pose of a robot in the image frame of a 2D camera upon which a visual servoing control system can be deployed. To all...
-
Vision-based perception has become prevalent in robotic applications, especially in those where the control loop relies on visual data, such as visual servoing. For those applications, ensuring that the features or target object remain visible to the camera is critical, necessitating visibility-aware control. In this paper, we propose a method to guarantee the visibility of a dynamic object using ...
-
In this paper, we present a novel clothoid-based visual servoing method for controlling the shape of a variable length continuum manipulator. Clothoids are curves with linearly changing curvature. They allow us to obtain a smooth representation of a continuum manipulator's shape in a compact form with few parameters. Using this curve model, we generate image features that are used in an adaptive v...
-
This paper presents a visual servoing method for controlling a robot in the configuration space by purely using its natural features. We first created a data collection pipeline that uses camera intrinsics, extrinsics, and forward kinematics to generate 2D projections of a robot's joint locations (keypoints) in image space. Using this pipeline, we are able to collect large sets of real-robot data,...
-
In a future with autonomous robots, visual and spatial perception is of utmost importance for robotic systems. Particularly for aerial robotics, there are many applications where utilizing visual perception is necessary for any real-world scenarios. Robotic aerial grasping using drones promises fast pick-and-place solutions with a large increase in mobility over other robotic solutions. Utilizing ...
-
The automation of key processes in metal cutting would substantially benefit many industries such as manufacturing and metal recycling. We present a vision-based control scheme for automated metal cutting with oxy-fuel torches, an established cutting medium in industry. The system consists of a robot equipped with a cutting torch and an eye-in-hand camera observing the scene behind a tinted visor....
-
We propose a control pipeline for SAG (Searching, Approaching, and Grasping) of objects, based on a decoupled arm kinematic chain and impedance control, which integrates image-based visual servoing (IBVS). The kinematic decoupling allows for fast end-effector motions and recovery that leads to robust visual servoing. The whole approach and pipeline can be generalized for any mobile platform (wheel...
-
Traditional methods based on Lyapunov analysis and learning-based approaches such as reinforcement learning (RL) are two powerful tools in visual servo tasks. Traditional methods are interpretable and their stability can be guar-anteed by Lyapunov analysis. However, they tend to have a high dependency on an accurate system dynamic model. RL approaches learn to act based on past experiences and thu...
-
Soft robotic devices have been popular in handling intricate grasping and dexterous manipulation tasks, serving as an alternative to conventional, rigid end-effectors. These devices are relatively simple, lightweight, and cost-effective. Recently, kirigami based structures have been used to create low-cost and disposable soft robotic grippers and hands. These grippers undergo a complex post-contac...
-
This paper presents the modeling and closed loop control of the shape-memory-alloy (SMA)-actuated hip joint of a flapping-wing flying robot (FWFR). Despite the lightweight legs/claw mechanism, a strong force of grasping is needed. The SMAs show high force delivery; however, it is difficult to control (position and temperature) the actuation due to the necessity of high currents for warming up, and...
-
Robot grasping is subject to an inherent tradeoff: Grippers with a large span typically take a longer time to close, and fast grippers usually cover a small span. However, many practical applications of grippers require the ability to close a large distance rapidly. For example, grasping cloth typically requires pressing a wide span of fabric into a graspable cusp. Besides, the ability to perform ...
-
To provide an alternative to conventional large-scale fitness equipment, we previously developed a soft passive wearable device for upper limb resistance exercises that utilized elastic exercise bands. However, the user was required to manually adjust the level of strength. In this paper, we introduce a novel wearable fitness device for upper limb exercise that constitutes cable-driven actuation t...
-
Growing robots based on the eversion principle are known for their ability to extend rapidly, from within, along their longitudinal axis, and, in doing so, reach deep into hitherto inaccessible, remote spaces. Despite many advantages, vine robots also present significant challenges, one of which is maintaining sensory payload at the tip without restricting the eversion process. A variety of tip me...
-
This study aims to achieve autonomous robotic assembly under uncertain conditions arising from imprecise goal positioning and variations in the angle of the grasped part. Soft robots are suitable for such uncertain and contact-rich environments and are capable of insertion tasks with imprecise goal positions. However, we may also struggle to handle further uncertainty, such as variations in graspi...
-
Over the years, soft growing robots that allow the feeding of new materials at their tips have attracted considerable attention owing to their unique locomotion characteristics. However, accessing targets over highly curved passages by steering compliant continuum bodies remains challenging. To this end, this study proposes a new tip steering mechanism that imparts soft growing robots with consecu...
-
Soft Robotics has established itself as an integral field in the broader discipline of general robotics through multiple advantages like inherent safety, adaptable morphology, and energy- and weight efficiency. Especially in environments hostile to humans and classical robots like the deep sea, soft robotic structures made out of silicone and actuated by seawater have numerous advantages. An appli...
-
This paper presents a soft earthworm robot that is capable of both efficient locomotion and obstacle avoidance. The robot is designed to replicate the unique locomotion mechanisms of earthworms, which enable them to move through narrow and complex environments with ease. The robot consists of multiple segments, each with its own set of actuators, that are connected through rigid plastic joints, al...
-
Eye gaze can convey rich information of human intentions, which enables the social robots to comprehend the cognition and behavior of human targets. However, the existing 3D gaze estimation methods generally have high requirements either on the dedicated hardware or the quantity and quality of training databases, which largely limits their practical application values. This paper proposes EasyGaze...
-
Convolutional neural networks trained using supervised learning can improve visual perception for human-robot walking. These advances have been possible due to largescale datasets like ExoNet and StairNet - the largest open-source image datasets of real-world walking environments. However, these datasets require vast amounts of manually annotated data, the development of which is time consuming an...
-
Deploying humans in a high-risk environment to extract casualties in order to provide medical attention is an inherently dangerous endeavor. To minimize this risk, Robotics and Autonomous Systems can be deployed in hazardous areas in place of human personnel to limit the exposure of first responders to various life-threatening conditions. The success of robotic extraction of injured persons depend...
-
Bronchoscopy is a medical procedure that involves the insertion of a flexible tube with a camera into the airways to survey, diagnose and treat lung diseases. Due to the complex branching anatomical structure of the bronchial tree and the similarity of the inner surfaces of the segmental airways, navigation systems are now being routinely used to guide the operator during procedures to access the ...
-
Ultrasound (US) imaging is widely used for diagnosing and monitoring arterial diseases, mainly due to the advantages of being non-invasive, radiation-free, and real-time. In order to provide additional information to assist clinicians in diagnosis, the tubular structures are often segmented from US images. To improve the artery segmentation accuracy and stability during scans, this work presents a...
-
The goal of this work is to develop a fast, compact, accurate, and robust neural network for automatic thyroid gland segmentation in medical ultrasound images when the computational resources are limited, such as portable intelligent medical diagnostic devices or robots. An adaptive width radial basis function neural network model with discriminative features is proposed in this work. The model co...
-
This paper presents a deep-learning model for deformable registration of ultrasound images at online rates, which we call U-RAFT. As its name suggests, U-RAFT is based on RAFT, a convolutional neural network for estimating optical flow. U-RAFT, however, can be trained in an unsupervised manner and can generate synthetic images for training vessel segmentation models. We propose and compare the reg...
-
Autonomous ultrasound (US) imaging has gained increased interest recently, and it has been seen as a potential solution to overcome the limitations of free-hand US exami-nations, such as inter-operator variations. However, it is still challenging to accurately map planned paths from a generic atlas to individual patients, particularly for thoracic applications with high acoustic-impedance bone str...
-
Bronchoscopy is currently the least invasive method for definitively diagnosing lung cancer, which kills more people in the United States than any other form of cancer. Successfully diagnosing suspicious lung nodules requires accurate localization of the bronchoscope relative to a planned biopsy site in the airways. This task is challenging because the lung deforms intraoperatively due to respirat...
-
Surgical context inference has recently garnered significant attention in robot-assisted surgery as it can facilitate workflow analysis, skill assessment, and error detection. However, runtime context inference is challenging since it requires timely and accurate detection of the interactions among the tools and objects in the surgical scene based on the segmentation of video data. On the other ha...
-
Multi-flagellated bacteria utilize the hydrodynamic interaction between their filamentary tails, known as flagella, to swim and change their swimming direction in low Reynolds number flow. Simplified hydrodynamics model, like Resistive Force Theories (RFT), lacks the capability to capture the dynamics of certain interactions known as bundling and tumbling. However, for the development of efficient...
-
The capability of a robot to perform tasks depends not only on precise motion control, but also on a well-suited body morphology. Adapting both morphology and control of robots to improve their task performance has been a widely studied and long-standing issue. While the bio-inspired bi-level optimization framework has gained popularity in recent years, it suffers from high computation complexity ...
-
Robot hands that imitate the shape of the human body have been actively studied, and various materials and mechanisms have been proposed to imitate the human body. Although the use of soft materials is advantageous in that it can imitate the characteristics of the human body's epidermis, it increases the number of parts and makes assembly difficult in order to perform complex movements. In this st...
-
In this paper, we focus on the kangaroo, which has powerful legs capable of jumping and a soft and strong tail. To incorporate these unique structure into a robot for utilization, we propose a design method that takes into account both the feasibility as a robot and the kangaroo-mimetic structure. Based on the kangaroo's musculoskeletal structure, we determine the structure of the robot that enabl...
-
This work briefly covers our efforts to stabilize the flight dynamics of Northeatern's tailless bat-inspired micro aerial vehicle, Aerobat. Flapping robots are not new. A plethora of examples is mainly dominated by insect-style design paradigms that are passively stable. However, Aerobat, in addition for being tailless, possesses morphing wings that add to the inherent complexity of flight control...
-
Here we present the development of an autonomous modular swimming robot. This robot, named µBot 2.0, was upgraded from our previous robot platform µBot and features onboard computing, sensing, and power. Its compact size and modularity render the robot an ideal platform for studying bio-inspired robot swimming. The robot is equipped with a micro controller in its head that communicates with extern...
-
Typical drones with multi rotors are generally less maneuverable due to unidirectional thrust, which may be unfavorable to agile flight in very narrow and confined spaces. This paper suggests a new bio-inspired drone that is empowered with high maneuverability in a lightweight and easy-to-carry way. The proposed flying squirrel inspired drone has controllable foldable wings to cover a wider range ...
-
In the last decade, researchers have been trying to develop many microrobots that mimic the extraordinary abilities of bionts in complex environments. How to fabricate the biomimetic microrobot with satisfying deformability and complex shapes to realize desired precise motion is the key issue. In this paper, we proposed an efficient programable fabrication method of the magnetic soft micro-robot t...
-
This paper uses a data-driven approach to model a highly redundantly driven tensegrity manipulator's forward and inverse kinematics. The tensegrity manipulator is based on a class-1 tensegrity with 20 struts and bends by 40 pneumatic actuators whose internal pressures are independently controlled. Based on the data obtained through random trials with the robot, a VAE-based kinematics model is trai...
-
Cable driven redundant manipulator (CDRM) can provide complex movements with high dexterity and singularity reduction. However, traditional CDRMs with universal joints have the disadvantages of requiring a high number of motors and having a narrow joint workspace. Furthermore, there is a limitation in terms of stiffness and payload. Recently, CDRMs composed of Quaternion joints have been developed...
-
Our study focuses on mapless navigation in robotics, which involves navigating without an established obstacle map of the environment. Spiking Neural Networks (SNNs) have recently been applied to this task using Deep Reinforcement Learning (DRL), but face challenges in dynamic and partially observable environments, as well as inaccuracies in transmitted data. To overcome these issues, we propose a...
-
Learning-based path planning is becoming a promising robot navigation methodology due to its adaptability to various environments. However, the expensive computing and storage associated with networks impose significant challenges for their deployment on low-cost robots. Motivated by this practical challenge, we develop a lightweight neural path planning architecture with a dual input network and ...
-
In recent years, Action Detection has become an active research topic in various fields such as human-robot interaction and assistive robots. Most of the previous methods in this field focus on temporally processing the action representation, without considering the dependencies among the action classes. However, actions that occur in a video are constantly related, and this correlation could offe...
-
Robotic systems are often composed of modular algorithms that each perform a specific function within a larger architecture, ranging from state estimation and task planning to trajectory optimization and object recognition. Existing work for specifying these systems as a formal composition of contract algorithms has limited expressiveness compared to the variety of sophisticated architectures that...
-
This paper studies the problem of Competitive Ant Coverage, in which two ant-like robots with very limited capabilities in terms of sensing range, computational power, and knowledge of the world compete in an area coverage task. We examine two variants of the problem that differ in the robot's objective: either being the First to Cover a Cell (FCC), or being the Last to Cover a Cell (LCC). Each ro...
-
Navigation is an essential task for social robots. However, certain rules must be followed to allow them to move without causing distraction or discomfort to people. Considering that the context surrounding robots and persons affects the expected behavior, this work defines a social area around a person that adapts to the real situation. In addition, the social context of a person is extended to i...
-
Optimal Multi-Robot Path Planning (MRPP) has garnered significant attention due to its many applications in domains including warehouse automation, transportation, and swarm robotics. Current MRPP solvers can be divided into reduction-based, search-based, and rule-based categories, each with their strengths and limitations. Regardless of the methodology, however, the issue of handling dense MRPP i...
-
For the best human-robot interaction experience, the robot's navigation policy should take into account personal preferences of the user. In this paper, we present a learning framework complemented by a perception pipeline to train a depth vision-based, personalized navigation controller from user demonstrations. Our virtual reality interface enables the demonstration of robot navigation trajector...
-
We focus on robot navigation in crowded environments. To navigate safely and efficiently within crowds, robots need models for crowd motion prediction. Building such models is hard due to the high dimensionality of multiagent domains and the challenge of collecting or simulating interaction-rich crowd-robot demonstrations. While there has been important progress on models for offline pedestrian mo...
-
Planning safe robot motions in the presence of humans requires reliable forecasts of future human motion. However, simply predicting the most likely motion from prior interactions does not guarantee safety. Such forecasts fail to model the long tail of possible events, which are rarely observed in limited datasets. On the other hand, planning for worst-case motions leads to overtly conservative be...
-
Biological agents, such as humans and animals, are capable of making decisions out of a very large number of choices in a limited time. They can do so because they use their prior knowledge to find a solution that is not necessarily optimal but good enough for the given task. In this work, we study the motion coordination of multiple drones under the above-mentioned paradigm, Bounded Rationality (...
-
Navigating effectively and safely in unknown urban environments is a crucial ability for service robot applications such as last-mile package delivery. To reach the entrance of its target destination, the robot must make informed local and global path planning decisions. We present a mapless global planning strategy based on pedestrian following. Our method allows the robot to exploit natural rout...
-
This work focuses on the capability of Mobile Manipulators to effectively control and maneuver cart-like non-holonomic systems. These cart-like systems are passive-wheeled objects with nonholonomic constraints with varying inertial parameters. We derive the dynamic equations of the cart-like system using a constrained Euler-Lagrange formulation and propose a Linear Quadratic Regulator controller t...
-
Many approaches to grasp synthesis optimize analytic quality metrics that measure grasp robustness based on finger placements and local surface geometry. However, generating feasible dexterous grasps by optimizing these metrics is slow, often taking minutes. To address this issue, this paper presents FRoGGeR: a method that quickly generates robust precision grasps using the min-weight metric, a no...
-
Grasping and moving objects in a large cluster is a common real scenario. In such scenarios, tens of objects are adjacent to each other, even stacked layer by layer, so that simple grasp would not work due to obstruction. In this paper, we propose a well-designed strategy to use synergy of pushing and grasping to automatically push and grasp objects in a large tightly packed cluster of objects. Ou...
-
Automation in the life science research laboratory is a paradigm that has gained increasing relevance in recent years. Current robotic solutions often have a limited scope, which reduces their acceptance and prevents the realization of complex workflows. The transport and manipulation of laboratory supplies with a robot is a particular case where this limitation manifests. In this paper, we deduce...
-
We report a new material and structure mapping (MSM) algorithm to assist robotic grasping and manipulation. Building on our new sensor development, the algorithm has four main components: 1) detection of time-of-flight (ToF) durations for the dual modalities of optoacoustic (OA) and pulse-echo ultrasound (US), 2) contour reconstruction by fusing OA and US signals, 3) local noise filtering by check...
-
Accurate estimation of the relative pose between an object and a robot hand is critical for many manipulation tasks. However, most of the existing object-in-hand pose datasets use two-finger grippers and also assume that the object remains fixed in the hand without any relative movements, which is not representative of real-world scenarios. To address this issue, a 6D object-in-hand pose dataset i...
-
In this paper, we present a novel method for mobile manipulators to perform multiple contact-rich manipulation tasks. While learning-based methods have the potential to generate actions in an end-to-end manner, they often suffer from insufficient action accuracy and robustness against noise. On the other hand, classical control-based methods can enhance system robustness, but at the cost of extens...
-
In this paper, we study the problem of task-oriented grasp synthesis from partial point cloud data using an eye-in-hand camera configuration. In task-oriented grasp synthesis, a grasp has to be selected so that the object is not lost during manipulation, and it is also ensured that adequate force/moment can be applied to perform the task. We formalize the notion of a gross manipulation task as a c...
-
In this work, we present GraspFlow, a refinement approach for generating context-specific grasps. We formulate the problem of grasp synthesis as a sampling problem: we seek to sample from a context-conditioned probability distribution of successful grasps. However, this target distribution is unknown. As a solution, we devise a discriminator gradient-flow method to evolve grasps obtained from a si...
-
We investigate the sequential manipulation planning problem for unmanned aerial manipulators (UAMs). Unlike prior work that primarily focuses on one-step manipulation tasks, sequential manipulations require coordinated motions of a UAM's floating base, the manipulator, and the object being manipulated, entailing a unified kinematics and dynamics model for motion planning under designated constrain...
-
This work studies how force measurement/estimation biases affect the force-based cooperative manipulation of a beam-like load suspended with cables by two aerial robots. Indeed, force biases are especially relevant in a force-based manipulation scenario in which direct communication is not relied upon. First, we compute the equilibrium configurations of the system. Then, we show that inducing an i...
-
The Roller-Quadrotor is a novel quadrotor that combines the maneuverability of aerial drones with the endurance of ground vehicles. This work focuses on the design, modeling, and experimental validation of the Roller-Quadrotor. Flight capabilities are achieved through a quadrotor config-uration, with four thrust-providing actuators. Additionally, rolling motion is facilitated by a unicycle-driven ...
-
This paper presents a novel uncrewed aerial vehicle (UAV) design named HALO, which stands for “harmless aerial limber robot”. HALO uses a swashplateless mechanism to generate a moment for pitch and roll control without requiring additional actuators such as servo, reducing the number of components needed for control and enhancing reliability. Its reduced weight and number of actuators improve payl...
-
The field of aerial manipulation has seen rapid advances, transitioning from push-and-slide tasks to interaction with articulated objects. The motion trajectory of these complex actions is usually hand-crafted or a result of online optimization methods like Model Predictive Control (MPC) or Model Predictive Path Integral (MPPI) control. However, these methods rely on heuristics or model simplifica...
-
Achieving precise, highly-dynamic maneuvers with Unmanned Aerial Vehicles (UAVs) is a major challenge due to the complexity of the associated aerodynamics. In particular, unsteady effectsas might be experienced in post-stall regimes or during sudden vehicle morphing-can have an adverse impact on the performance of modern flight control systems. In this paper, we present a vortex particle model and...
-
Inspection and maintenance work at heights car-ries significant risks and is time consuming for human workers. Therefore, aerial manipulators are expected to replace these tasks. This paper presents a multirotor UAV equipped with a single horizontal thruster. This minimal configuration en-ables physical contact while keeping the airframe's attitude horizontal for non-destructive inspection work on...
-
In this paper, we present the dynamic model and control of DoubleBee, a novel hybrid aerial-ground vehicle consisting of two propellers mounted on tilting servo motors and two motor-driven wheels. DoubleBee exploits the high energy efficiency of a bicopter configuration in aerial mode, and enjoys the low power consumption of a two-wheel self-balancing robot on the ground. Furthermore, the propelle...
-
In this paper, we propose a novel swashplateless-elevon actuation (SEA) for dual-rotor tail-sitter vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs). In contrast to the conventional elevon actuation (CEA) which controls both pitch and yaw using elevons, the SEA adopts swash-plateless mechanisms to generate an extra moment through motor speed modulation to control pitch and uses e...
-
We present an aerial vehicle composed of a custom quadrotor with tilted rotors and a helium balloon, called SBlimp. We propose a novel control strategy that takes advantage of the natural stable attitude of the blimp to control translational motion. Different from cascade controllers in the literature that controls attitude to achieve desired translational motion, our approach directly controls th...
-
Telemanipulation is widely used in robotics applications, ranging from maintenance of various industrial systems to search and rescue response in remote and/or hazardous environments. Human operators are often responsible for the control of such robotic systems. However, these remote interactions require highly trained and experienced operators owing to their complex nature. Semi-autonomous system...
-
Over the last decades, significant research effort has been put into creating Electromyography (EMG) based controllers for intuitive, hands-free control of robotic arms and hands. To achieve this, machine learning models have been employed to decode human motion and intention using EMG signals as input and to deliver several applications, such as prosthesis control using gesture classification. De...
-
Robotic teleoperation is used in various applications, including the nuclear industry, where the experience and intelligence of a human operator are necessary for making complex decisions that are beyond the autonomy of robots. Human-robot interfaces that help strengthen an operators situational awareness without inducing excessive cognitive load are crucial to the success of teleoperation. This p...
-
We explore task tolerances, i.e., allowable position or rotation inaccuracy, as an important resource to facilitate smooth and effective telemanipulation. Task tolerances provide a robot flexibility to generate smooth and feasible motions; however, in teleoperation, this flexibility may make the user's control less direct. In this work, we implemented a telema-nipulation system that allows a robot...
-
Robotic automation is expected to be applicable in various fields. The utilization of robots requires human-robot interaction (HRI) for prolonged direct manipulation or learning. Recently, numerous studies on HRI have been conducted in virtual space using virtual, augmented, and mixed reality (VAM-HRI). In the future, VAM-HRI applications are expected to involve users wearing head-mounted displays...
-
This paper presents a comparative performance evaluation of three different teleoperation interfaces for very low weight (<3 kg) anthropomorphic dual arms intended to conduct complex manipulation tasks involving a certain level of dexterity, accuracy and agility, either in ground service or in aerial manipulation applications. A visual human pose estimation system is developed to obtain the Cartes...
-
When humans control or supervise remote robot manipulation, augmented reality (AR) visual cues overlaid on the remote camera video stream can effectively enhance human's remote perception of task and robot states, and comprehension of the robot autonomy's capability and intent. In this work, we conducted a user study (N=18) to investigate: (RQ1) what AR cues humans prefer when controlling the robo...
-
A Shared Autonomous Nursing Robot Assistant with Dynamic Workspace for Versatile Mobile Manipulation
This paper presents a novel integration of a shared autonomous mobile humanoid robot for remote nursing assistance. The proposed nursing robot has a motorized versatile supporting structure to allow flexible integration of the system components, autonomously adjust its mobile manipulation workspace and improve its reachability and manipulability to operate in a cluttered environment. The robot als...
-
Local-remote systems allow robots to execute complex tasks in hazardous environments such as space and nuclear power stations. However, establishing accurate positional mapping between local and remote devices can be difficult due to time delays that can compromise system performance and stability. Enhancing the synchronicity and stability of localremote systems is vital for enabling robots to int...
-
This paper reports on Team Northeastern's Avatar system for telepresence, and our holistic approach to meet the ANA Avatar XPRIZE Final testing task requirements. The system features a dual-arm configuration with hydraulically actuated glove-gripper pair for haptic force feedback. Our proposed Avatar system was evaluated in the ANA Avatar XPRIZE Finals and completed all 10 tasks, scored 14.5 point...
-
Although telepresence assistive robots have made significant progress, they still lack the sense of realism and physical presence of the remote operator. This results in a lack of trust and adoption of such robots. In this paper, we introduce an Avatar Robot System which is a mixed real/virtual robotic system that physically interacts with a person in proximity of the robot. The robot structure is...
-
In this work, we use recent developments in the field of adaptive robust Model Predictive Control (MPC) to build a controller for bilateral teleoperation systems. To guarantee robust constraint satisfaction, we incorporate polytopic tube controllers in the MPC design. In addition, we use online learning methods to learn the environment model. Namely, we use set membership learning to learn the par...
-
A human being can communicate while working at the same time. However, teleoperated humanoid robots that can work and communicate simultaneously are currently too complex and expensive. We propose an “Augmented Avatar” that can perform both work and communication simultaneously at a low cost. The authors have developed two types of Avatars: a Manipulation Avatar with minimal functions tailored for...
-
Simulating camera sensors is a crucial task in autonomous driving. Although neural radiance fields are exceptional at synthesizing photorealistic views in driving simulations, they still fail to generate extrapolated views. This paper proposes to incorporate map priors into neural radiance fields to synthesize out-of-trajectory driving views with semantic road consistency. The key insight is that ...
-
Object goal navigation aims to navigate an agent to locations of a given object category in unseen environments. Classical methods explicitly build maps of environments and require extensive engineering while lacking semantic information for object-oriented exploration. On the other hand, end-to-end learning methods alleviate manual map design and predict actions using implicit representations. Su...
-
Visual perception is an important component for autonomous navigation of unmanned surface vessels (USV), particularly for the tasks related to autonomous inspection and tracking. These tasks involve vision-based navigation techniques to identify the target for navigation. Reduced visibility under extreme weather conditions in marine environments makes it difficult for vision-based approaches to wo...
-
Despite recent progress in Reinforcement Learning for robotics applications, many tasks remain prohibitively difficult to solve because of the expensive interaction cost. Transfer learning helps reduce the training time in the target domain by transferring knowledge learned in a source domain. Sim2Real transfer helps transfer knowledge from a simulated robotic domain to a physical target domain. K...
-
This paper introduces UVIO, a multi-sensor framework that leverages Ultra Wide Band (UWB) technology and Visual-Inertial Odometry (VIO) to provide robust and low-drift localization. In order to include range measurements in state estimation, the position of the UWB anchors must be known. This study proposes a multi-step initialization procedure to map multiple unknown anchors by an Unmanned Aerial...
-
A household robot should be able to navigate to target objects without requiring users to first annotate everything in their home. Most current approaches to object navigation do not test on real robots and rely solely on reconstructed scans of houses and their expensively labeled semantic 3D meshes. In this work, our goal is to build an agent that builds self-supervised models of the world via ex...
-
Underwater robots, like Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs), are promising tools for the exploration and study of the under-ice environment and the ecosystems that thrive there. However, state estimation is a well-known problem for robotic systems, especially, for the ones that travel underwater. In this paper, $w$e present a tightly-coupled multi-sensors fu...
-
Planning a path for a mobile robot typically requires building a map (e.g., an occupancy grid) of the environment as the robot moves around. While navigating in an unknown environment, the map built by the robot online may have many as-yet-unknown regions. A conservative planner may avoid such regions taking a longer time to navigate to the goal. Instead, if a robot is able to correctly predict th...
-
In order for robots to follow open-ended instructions like “go open the brown cabinet over the sink,” they require an understanding of both the scene geometry and the semantics of their environment. Robotic systems often handle these through separate pipelines, sometimes using very different representation spaces, which can be suboptimal when the two objectives conflict. In this work, we present U...
-
Prediction-based active perception has shown the potential to improve the navigation efficiency and safety of the robot by anticipating the uncertainty in the unknown environment. The existing works for 3D shape prediction make an implicit assumption about the partial observations and therefore cannot be used for real-world planning and do not consider the control effort for next-best-view plannin...
-
We present AVS-HRL, a modular Active visual SLAM system based on hierarchical reinforcement learning. The reward function explicitly considers the efficiency of exploration and the accuracy of mapping by utilizing the internal variables of SLAM, such as feature points distribution and loop-closure signal. Compared to end-to-end active SLAM methods, we designed a map reconstruction module that can ...
-
This work presents a novel method for point registration in 3D space. The proposed algorithm utilizes transformation-invariant geometry information to estimate the pose of objects based on correspondences between points in two sets. Conventional methods use geometry descriptors to find these correspondences, which can result in a large number of outliers. Most existing algorithms are error-prone w...
-
In this study, we introduce an online monocular lane mapping approach that solely relies on a single camera and odometry for generating spline-based maps. Our proposed technique models the lane association process as an assignment issue utilizing a bipartite graph, and assigns weights to the edges by incorporating Chamfer distance, pose uncertainty, and lateral sequence consistency. Furthermore, w...
-
Highly accurate and efficient map in unknown and complex environments is essential for robotics navigation. Traditionally, mobile robot platforms are often computationally constrained when using multiple sensors to process large amounts of input data. In previous works, some of them have been deployed to embedded platforms in real-time. However, how to balance accuracy and efficiency while reducin...
-
In this study, we propose a novel system for real-time coloring LiDAR point clouds with a low-cost RS camera. The main challenges are dealing with the motion distortion of the RS camera and the multi-sensor time synchronization. To tackle these challenges, we carefully design a hardware synchronizer to ensure the strict alignment of the LiDAR, inertial measurement unit, and RS camera. With accurat...
-
Based on the neural radiance fields (NeRF), we present a pipeline for generating novel views from a single 360° panoramic image. Prior research relied on the neighborhood interpolation capability of multi-layer perceptions to complete missing regions caused by occlusion. This resulted in artifacts in their predictions. We propose 360FusionNeRF, a semi-supervised learning framework that employs geo...
-
In this paper, we present a centralized framework for multi-session LiDAR mapping in urban environments, by utilizing lightweight line and plane map representations instead of widely used point clouds. The proposed framework achieves consistent mapping in a coarse-to-fine manner. Global place recognition is achieved by associating lines and planes on the Grassmannian manifold, followed by an outli...
-
Unmanned ground vehicles (UGVs) require effective perception and analysis of their surrounding terrain for safe operation. This paper presents a novel approach to their local elevation mapping and traversability analysis using sparse data from a single LiDAR sensor, which can generate a dense local traversability map in real-time. By preserving ground height information, our method can differentia...
-
3D mapping is crucial for many applications in robotics and related industries. To build dense high-quality point clouds accurate depth estimation or completion is needed. This paper presents the development of a metric-semantic mapping pipeline based on Deep Neural Networks (DNN) which assures geometrical consistency with enhancements for chal-lenging environments with transparent and reflecting ...
-
As the application scenarios of mobile robots are getting more complex and challenging, scene understanding becomes increasingly crucial. A mobile robot that is supposed to operate autonomously in indoor environments must have precise knowledge about what objects are present, where they are, what their spatial extent is, and how they can be reached; i.e., information about free space is also cruci...
-
This work introduces range-based GP maps, which directly represent terrain by modeling the range from a LiDAR sensor as a Gaussian process (GP) in spherical space. Such a model aligns the predicted uncertainty from the GP regression with the uncertainty in the underlying sensor observations. Experimental evaluation on simulated natural terrain indicates that local range-based GP maps perform compa...
-
Robots reason about the environment through dedicated representations. Despite the fact that Gaussian Process (GP)-based representations are appealing due to their probabilistic and continuous nature, the cubic computational complexity is a concern. In this paper, we present a novel efficient GP-based representation that has the ability to produce accurate distance fields and is parameterised by t...
-
We investigate motion planning algorithms for the assembly of shapes in the tilt model in which unit-square tiles move in a grid world under the influence of uniform external forces and self-assemble according to certain rules. We provide several heuristics and experimental evaluation of their success rate, solution length, and runtime.
-
Although dynamic games provide a rich paradigm for modeling agents' interactions, solving these games for real-world applications is often challenging. Many real-world interactive settings involve general nonlinear state and input constraints that couple agents' decisions with one another. In this work, we develop an efficient and fast planner for interactive trajectory optimization in constrained...
-
Planning a collision-free path efficiently among obstacles is crucial in robotics. Conventional one-shot unidirectional path planning algorithms work well in the static environment, but cannot respond to the environment changes timely in the dynamic environment. To tackle this issue and improve the search efficiency, we propose a bidirectional incremental search method, Bidirectional Lifelong Plan...
-
The Multi-Agent Path Finding (MAPF) problem is a critical challenge in dynamic multi-robot systems. Recent studies have revealed that multi-agent reinforcement learning (MARL) is a promising approach to solving MAPF problems in a fully decentralized manner. However, as the size of the multi-robot system increases, sample inefficiency becomes a major impediment to learning-based methods. This paper...
-
In this work, we consider the problem of allocating tasks and planning paths for multiple robots to operate cooper-atively. We formulate the problem as a bi-level optimization that involves optimizing the scheduling of robots and the collision-free path for each robot. To address the complexity of the environment with obstacles, we introduce a congestion-aware state representation technique with t...
-
We consider the problem of navigating a mobile robot towards a target in an unknown environment that is endowed with visual sensors, where neither the robot nor the sensors have access to global positioning information and only use first-person- view images. In order to overcome the need for positioning, we train the sensors to encode and communicate relevant viewpoint information to the mobile ro...
-
We address the problem of multi-goal multi robot path planning (MG-MRPP) via counterexample guided abstraction refinement (CEGAR) framework. MG-MRPP generalizes the standard discrete multi-robot path planning (MRPP) problem. While the task in MRPP is to navigate robots in an undirected graph from their starting vertices to one individual goal vertex per robot, MG-MRPP assigns each robot multiple g...
-
Cooperative multi-agent problems often require coordination between agents, which can be achieved through a centralized policy that considers the global state. Multi-agent policy gradient (MAPG) methods are commonly used to learn such policies, but they are often limited to problems with low-level action spaces. In complex problems with large state and action spaces, it is advantageous to extend M...
-
Autonomous robotic systems are useful in automating tasks such as inspection and surveying of unknown areas, where speed is often an important factor. In order to effectively reduce the time required to complete missions, an efficient exploration and coordination strategy is needed. In this spirit, this work proposes an approach based on the Monte Carlo Tree Search (MCTS) algorithm to guide robots...
-
We address the Warehouse Servicing Problem (WSP) in automated warehouses, which use teams of mobile robots to move products from shelves to packaging stations. Given a list of products, the WSP amounts to finding a motion plan which brings every product on the list from a shelf to a packaging station within a given time limit. The WSP consists of four subproblems, namely, deciding where to source ...
-
Autonomous driving in rain remains challenging. Rain causes sensor performance degradation that can affect sensor measurement quality. During the rain, lasers may suffer from energy loss due to raindrop absorption. As a result, some laser measurements reflected from obstacles may not be recognized by the LiDAR sensor, thus raising potential risks for autonomous vehicles. This work investigates a n...
-
Sensor degradation is one of the major challenges for autonomous driving. During the rain, the interference from raindrops can negatively influence LiDAR measurements. For example, valid measurements could be reduced during the rain, and some measurements may become noisy. Unreliable measurements can lead to potential safety issues if autonomous driving systems are unaware of these changes. In thi...
-
Trajectory forecasting is a widely-studied problem for autonomous navigation. However, existing benchmarks evaluate forecasting based on independent snapshots of trajectories, which are not representative of real-world applications that operate on a continuous stream of data. To bridge this gap, we introduce a benchmark that continuously queries future trajectories on streaming data and we refer t...
-
Autonomous vehicle navigation in complex and unpredictable outdoor environments requires extensive and detailed understanding of the surrounding area and compliance with the traffic rules. In this paper, we attempt to imitate human driver behavior towards autonomous navigation that is suitable for diverse, challenging environments, whether urban, semi-structured or rural-like. Our approach starts ...
-
Autonomous mobile robot navigation in a human populated and encumbered environment is recognized as a hard problem to be solved in real-time. Most of the time, robots face the so-called ‘Freezing Robot Problem’, that occurs when the robot stops because no feasible and safe motion can be found. In order to provide to the robot the capability of proactive navigation, in this work we generalize the c...
-
Recently, the feature-level generation has demonstrated the effectiveness of pseudo-stereo synthesis in Monocular 3D Detection (M3D). In this paper, we aim to further bridge the gap between the stereo and the monocular 3D object detectors in autonomous driving through direct image-level pseudo-stereo generation. We propose a novel Cycled Generative Pseudo-Stereo (CGPS) architecture to generate the...
-
Autonomous drone racing is becoming an excellent platform to challenge quadrotors' autonomy techniques including planning, navigation and control technologies. However, most research on this topic mainly focuses on single drone scenarios. In this paper, we describe a novel time-optimal trajectory generation method for generating time-optimal trajectories for a swarm of quadrotors to fly through pr...
-
Humans are well-adept at navigating public spaces shared with others, where current autonomous mobile robots still struggle: while safely and efficiently reaching their goals, humans communicate their intentions and conform to unwritten social norms on a daily basis; conversely, robots become clumsy in those daily social scenarios, getting stuck in dense crowds, surprising nearby pedestrians, or e...
-
Safe autonomous driving critically depends on how well the ego-vehicle can predict the trajectories of neighboring vehicles. To this end, several trajectory prediction algorithms have been presented in the existing literature. Many of these approaches output a multimodal distribution of obstacle trajectories instead of a single deterministic prediction to account for the underlying uncertainty. Ho...
-
Mobile robot navigation in a human-populated environment has been of great interest to the research community in recent years, referred to as crowd navigation. Currently, offline reinforcement learning (RL)-based method has been introduced to this domain, for its ability to alleviate the sim2real gap brought by online RL which relies on simulators to execute training, and its scalability to use th...
-
Robotic navigation in unknown, cluttered environ-ments with limited sensing capabilities poses significant chal-lenges in robotics. Local trajectory optimization methods, such as Model Predictive Path Intergal (MPPI), are a promising solution to this challenge. However, global guidance is required to ensure effective navigation, especially when encountering challenging environmental conditions or ...
-
This paper presents a LiDAR-based end-to-end autonomous driving method with Vehicle-to-Everything (V2X) communication integration, termed V2X-Lead, to address the challenges of navigating unregulated urban scenarios under mixed-autonomy traffic conditions. The proposed method aims to handle imperfect partial observations by fusing the onboard LiDAR sensor and V2X communication data. A model-free a...
-
With the increasing computing power, using data-driven approaches to co-design a robot's morphology and controller has become a promising way. However, most existing data-driven methods require training the controller for each morphology to calculate fitness, which is time-consuming. In contrast, the dual-network framework utilizes data collected by individual networks under a specific morphology ...
-
Deep reinforcement learning (DRL) has been widely applied in autonomous exploration and mapping tasks, but often struggles with the challenges of sampling efficiency, poor adaptability to unknown map sizes, and slow simulation speed. To speed up convergence, we combine curriculum learning (CL) with DRL, and first propose a Cumulative Curriculum Reinforcement Learning (CCRL) training framework to a...
-
Demonstrations are widely used in Deep Reinforcement Learning (DRL) for facilitating solving tasks with sparse rewards. However, the tasks in real-world scenarios can often have varied initial conditions from the demonstration, which would require additional prior behaviours. For example, consider we are given the demonstration for the task of picking up an object from an open drawer, but the draw...
-
To alleviate the sample complexity of reinforcement learning algorithms, simulations are a common approach to train control policies before deploying the policy on a real-world robot. However, a gap between simulation and reality generally persists, which endorses the aim to train robust policies already in simulation such that those can be transferred to a real robot at a high success rate. In th...
-
Deep Reinforcement Learning (RL) has shown promise in addressing complex robotic challenges. In real-world applications, RL is often accompanied by failsafe controllers as a last resort to avoid catastrophic events. While necessary for safety, these interventions can result in undesirable behaviors, such as abrupt braking or aggressive steering. This paper proposes two safety intervention reductio...
-
Adaptive interfaces can help users perform sequential decision-making tasks like robotic teleoperation given noisy, high-dimensional command signals (e.g., from a brain-computer interface). Recent advances in human-in-the-loop machine learning enable such systems to improve by interacting with users, but tend to be limited by the amount of data that they can collect from individual users in practi...
-
Legged locomotion is arguably the most suited and versatile mode to deal with natural or unstructured terrains. Intensive research into dynamic walking and running controllers has recently yielded great advances, both in the optimal control and reinforcement learning (RL) literature. Hopping is a challenging dynamic task involving a flight phase and has the potential to increase the traversability...
-
Applying Reinforcement Learning to solve real-world optimization problems presents significant challenges because of the large amount of data normally required. A popular solution is to train the algorithms in a simulation and transfer the weights to the real system. However, sim-to-real approaches are prone to fail when the Reality Gap is too big, e.g. in robotic systems with complex and non-line...
-
In classic reinforcement learning algorithms, agents make decisions at discrete and fixed time intervals. The duration between decisions becomes a crucial hyperparameter, as setting it too short may increase the problem's difficulty by requiring the agent to make numerous decisions to achieve its goal while setting it too long can result in the agent losing control over the system. However, physic...
-
Imitation learning (IL) is a simple and powerful way to use high-quality human driving data, which can be collected at scale, to produce human-like behavior. However, policies based on imitation learning alone often fail to sufficiently account for safety and reliability concerns. In this paper, we show how imitation learning combined with reinforcement learning using simple rewards can substan-ti...
-
Interactive reinforcement learning has shown promise in learning complex robotic tasks. However, the process can be human-intensive due to the requirement of a large amount of interactive feedback. This paper presents a new method that uses scores provided by humans instead of pairwise preferences to improve the feedback efficiency of interactive reinforcement learning. Our key insight is that sco...
-
Satisfying safety constraints in reinforcement learning (RL) is an important issue, especially in real-world applications. Many studies have approached safe RL with the Lagrangian method, which introduces dual variables. However, applying a trained policy with the optimal dual variable to a new environment can be hazardous since the optimal value of the dual variable, which represents a level of s...
-
In this study, we aim to develop a model that comprehends a natural language instruction (e.g., “Go to the living room and get the nearest pillow to the radio art on the wall”) and generates a segmentation mask for the target everyday object. The task is challenging because it requires (1) the understanding of the referring expressions for multiple objects in the instruction, (2) the prediction of...
-
Identifying moving objects is a crucial capability for autonomous navigation, consistent map generation, and future trajectory prediction of objects. In this paper, we propose a novel network that addresses the challenge of segmenting moving objects in 3D LiDAR scans. Our approach not only predicts point-wise moving labels but also detects instance information of main traffic participants. Such a ...
-
Open-world Instance Segmentation (OIS) is a challenging task that aims to accurately segment every object instance appearing in the current observation, regardless of whether these instances have been labeled in the training set. This is important for safety-critical applications such as robust autonomous navigation. In this paper, we present a flexible and effective OIS framework for LiDAR point ...
-
The integration of semantic information can effectively enhance the performance of 3D object detection based on lidar point cloud. Most of previous researches utilize camera-lidar fusion to improve detection accuracy for distant or small objects. However, this approach is typically unsuitable for real-time applications due to the large amount of input data. Recently, a multi-task framework using o...
-
Semantic segmentation of LiDAR point clouds can provide assistance for precise perception in autonomous driving, but traditional segmentation methods face challenges such as unbalanced class distribution and insufficient labeling. Generalized few-shot learning has been researched on image data, but these methods are difficult to apply directly to LiDAR point clouds. To tackle these challenges, we ...
-
Methods have recently been proposed that densely segment 3D volumes into classes using only color images and expert supervision in the form of sparse semantically annotated pixels. While impressive, these methods still require a relatively large amount of supervision and segmenting an object can take several minutes in practice. Such systems typically only optimize the representation on the scene ...
-
Unsupervised localization and segmentation are long-standing robot vision challenges that describe the critical ability for an autonomous robot to learn to decompose images into individual objects without labeled data. These tasks are important because of the limited availability of dense image manual annotation and the promising vision of adapting to an evolving set of object categories in lifelo...
-
Deep perception models have to reliably cope with an open-world setting of domain shifts induced by different geographic regions, sensor properties, mounting positions, and several other reasons. Since covering all domains with annotated data is technically intractable due to the endless possible variations, researchers focus on unsupervised domain adaptation (UDA) methods that adapt models traine...
-
Semantic segmentation, which aims to classify every pixel in an image, is a key task in machine perception, with many applications across robotics and autonomous driving. Due to the high dimensionality of this task, most existing approaches use local operations, such as convolutions, to generate per-pixel features. However, these methods are typically unable to effectively leverage global context ...
-
Segmenting and recognizing surgical operation trajectories into distinct, meaningful gestures is a critical preliminary step in surgical workflow analysis for robot-assisted surgery. This step is necessary for facilitating learning from demonstrations for autonomous robotic surgery, evaluating surgical skills, and so on. In this work, we develop a hierarchical semi-supervised learning framework fo...
-
State-of-the-art lidar panoptic segmentation (LPS) methods follow “bottom-up” segmentation-centric fashion wherein they build upon semantic segmentation networks by utilizing clustering to obtain object instances. In this paper, we re-think this approach and propose a surprisingly simple yet effective detection-centric network for both LPS and tracking. Our network is modular by design and optimiz...
-
In autonomous driving, performing robust semantic segmentation under adverse weather conditions is a long-standing challenge. Imperfect camera observations under adverse conditions result in images with reduced visibility, which hinders label annotation and semantic scene understanding based on these images. A common solution is to adopt semantic segmentation models trained in a source domain with...
-
Accurate tracking of transparent objects, such as glasses, plays a critical role in many robotic tasks such as robot-assisted living. Due to the adaptive and often reflective texture of such objects, traditional tracking algorithms that rely on general-purpose learned features suffer from reduced performance. Recent research has proposed to instill trans-parency awareness into existing general obj...
-
An event camera is a novel vision sensor that can capture per-pixel brightness changes and output a stream of asynchronous “events”. It has advantages over conventional cameras in those scenes with high-speed motions and challenging lighting conditions because of the high temporal resolution, high dynamic range, low bandwidth, low power consumption, and no motion blur. Therefore, several supervise...
-
Nano-quadcopters are versatile platforms attracting the interest of both academia and industry. Their tiny form factor, i.e., ~ 10 cm diameter, makes them particularly useful in narrow scenarios and harmless in human proximity. However, these advantages come at the price of ultra-constrained onboard computational and sensorial resources for autonomous operations. This work addresses the task of es...
-
Semantic scene completion (SSC) jointly predicts the semantics and geometry of the entire 3D scene, which plays an essential role in 3D scene understanding for autonomous driving systems. SSC has achieved rapid progress with the help of semantic context in segmentation. However, how to effectively exploit the relationships between the semantic context in semantic segmentation and geometric structu...
-
Reliable LiDAR panoptic segmentation (LPS), including both semantic and instance segmentation, is vital for many robotic applications, such as autonomous driving. This work proposes a new LPS framework named PANet to eliminate the dependency on the offset branch and improve the performance on large objects, which are always over-segmented by clustering algorithms. Firstly, we propose a non-learnin...
-
We present a new method to adapt an RGB-trained water segmentation network to target-domain aerial thermal imagery using online self-supervision by leveraging texture and motion cues as supervisory signals. This new thermal capability enables current autonomous aerial robots operating in near-shore environments to perform tasks such as visual navigation, bathymetry, and flow tracking at night. Our...
-
Data-driven visual odometry (VO) is a critical subroutine for autonomous edge robotics, and recent progress in the field has produced highly accurate point predictions in complex environments. However, emerging autonomous edge robotics devices like insect-scale drones and surgical robots lack a computationally efficient framework to estimate VO's predictive uncertainties. Meanwhile, as edge roboti...
-
Depth estimation is one of the essential tasks to be addressed when creating mobile autonomous systems. While monocular depth estimation methods have improved in recent times, depth completion provides more accurate and reliable depth maps by additionally using sparse depth information from other sensors such as LiDAR. However, current methods are specifically trained for a single LiDAR sensor. As...
-
Reinforcement learning (RL) algorithms face significant challenges when dealing with long-horizon robot manipulation tasks in real-world environments due to sample inefficiency and safety issues. To overcome these challenges, we propose a novel framework, SEED, which leverages two approaches: reinforcement learning from human feedback (RLHF) and primitive skill-based reinforcement learning. Both a...
-
Tele-operating aerial vehicles without any automated assistance is challenging due to various limitations, especially for inexperienced users. Autocomplete addresses this problem by automatically identifying and completing the user's intended motion. Such a framework uses machine learning to recognize and classify human inputs as one of a set of motion primitives, and then, if the human operator a...
-
Human action recognition and motion forecasting is becoming increasingly successful, in particular with utilizing graphs. We aim to transfer this success into the context of industrial Human-Robot Collaboration (HRC), where humans work closely with robots and interact with workpieces in defined workspaces. For this purpose, it is necessary to use all the available information extractable in such a...
-
In this work, we analyse the use of a prediction of the human's force in a Human-Robot collaborative object transportation task at a middle distance. We check that this force prediction can improve multiple parameters associated with effective Human-Robot Interaction (HRI) such as perception of the robot's contribution to the task, comfort or trust in the robot in a physical Human Robot Interactio...
-
As robots become increasingly prevalent amidst diverse environments, their ability to adapt to novel scenarios and objects is essential. Advances in modern object detection have also paved the way for robots to identify interaction entities within their immediate vicinity. One drawback is that the robot's operational domain must be known at the time of training, which hinders the robot's ability t...
-
As robots operate alongside humans in shared spaces, such as homes and offices, it is essential to have an effective mechanism for interacting with them. Natural language offers an intuitive interface for communicating with robots, but most of the recent approaches to grounded language understanding reason only in the context of an instantaneous state of the world. Though this allows for interpret...
-
In recent years, studies on Socially Assistive Robots (SARs) examine how to improve the quality of life of people living with dementia and older adults (OAs) in general. However, most SARs have somewhat limited perception capabilities or interact using simple pre-programmed responses, providing limited or repetitive interaction modalities. Integrating more advanced perceptual capabilities with dee...
-
This work addresses human intention identification during physical Human-Robot Interaction (pHRI) tasks to include this information in an assistive controller. To this purpose, human intention is defined as the desired trajectory that the human wants to follow over a finite rolling prediction horizon so that the robot can assist in pursuing it. This work investigates a Recurrent Neural Network (RN...
-
Human-in-the-loop reinforcement learning (RL) methods actively integrate human knowledge to create reward functions for various robotic tasks. Learning from preferences shows promise as alleviates the requirement of demonstrations by querying humans on state-action sequences. However, the limited granularity of sequence-based approaches complicates temporal credit assignment. The amount of human q...
-
Recognizing interactive action plays an important role in human-robot interaction and collaboration. Previous methods use late fusion and co-attention mechanism to capture interactive relations, which have limited learning capability or inefficiency to adapt to more interacting entities. With assumption that priors of each entity are already known, they also lack evaluations on a more general sett...
-
Service robots need language capabilities for communicating with people, and navigation skills for beyond-proximity interaction in the real world. When the robot explores the real world with people side by side, there is the compound problem of human-robot dialog and co-navigation. The human-robot team uses dialog to decide where to go, and their shared spatial awareness affects the dialog state. ...
-
Multi-human multi-robot teams have great potential for complex and large-scale tasks through the collaboration of humans and robots with diverse capabilities and expertise. To efficiently operate such highly heterogeneous teams and maximize team performance timely, sophisticated initial task allocation strategies that consider individual differences across team members and tasks are required. Whil...
-
Contact tooling operations like sanding and polishing have been high in demand for robotics and automation, as manual operations are labour-intensive with inconsistent quality. However, automating these operations remains a challenge since they are highly dependent on prior knowledge about the geometry of the workpiece. While several methods have been developed in existing research to automate the...
-
Augmented and mixed-reality techniques harbor a great potential for improving human-robot collaboration. Visual signals and cues may be projected to a human partner in order to explicitly communicate robot intentions and goals. However, it is unclear what type of signals support such a process and whether signals can be combined without adding additional cognitive stress to the partner. This paper...
-
Children around the world are growing more sedentary over time, which leads to considerable accompanying wellness challenges. Pilot results from our research group have shown that robots may offer something different or better than other developmentally appropriate toys when it comes to motivating physical activity. However, the foundations of this work involved larger-group interactions in which ...
-
Fluent teaming is characterized by tacit interaction without explicit communication. Such interaction requires team situation awareness (TSA) to facilitate. However, existing approaches often rely on explicit communication (such as visual projection) to support TSA, resulting in a paradox. In this paper, we consider implicit projection (IP) to improve TSA for tacit human-robot interaction. IP mini...
-
Enabling a robot to perform new tasks is a complex endeavor, usually beyond the reach of non-technical users. For this reason, research efforts that aim at empowering end-users to teach robots new abilities using intuitive modes of interaction are valuable. In this article, we present INtuitive PROgramming 2 (INPRO2), a learning framework that allows inferring planning actions from demonstrations ...
-
Human models play a crucial role in human-robot interaction (HRI), enabling robots to consider the impact of their actions on people and plan their behavior accordingly. However, crafting good human models is challenging; capturing context-dependent human behavior requires significant prior knowledge and/or large amounts of interaction data, both of which are difficult to obtain. In this work, we ...
-
Social robots have gained widespread attention for their potential to assist people in diverse domains, such as living assistance and logistics transportation. Human-accompanying, i.e., walking side-by-side with a person, is an expected and essential capability for social robots. However, due to the complexity of motion coordination between the target person and the mobile robot, the accompanying ...
-
Conversational assistive robots can aid people, especially those with cognitive impairments, to accomplish various tasks such as cooking meals, performing exercises, or operating machines. However, to interact with people effectively, robots must recognize human plans and goals from noisy observations of human actions, even when the user acts sub-optimally. Previous works on Plan and Goal Recognit...
-
Service robots are expected to assist users in a constantly growing range of environments and tasks. People may be unique in many ways, and online adaptation of robots is central to personalized assistance. We focus on collaborative tasks in which the human collaborator may not be fully ablebodied, with the aim for the robot to automatically determine the best level of support. We propose a method...
-
In an efficient and flexible human-robot collaborative work environment, a robot team member must be able to recognize both explicit requests and implied actions from human users. Identifying “what to do” in such cases requires an agent to have the ability to construct associations between objects, their actions, and the effect of actions on the environment. In this regard, semantic memory is bein...
-
Trust-aware human-robot interaction (HRI) has received increasing research attention, as trust has been shown to be a crucial factor for effective HRI. Research in trust-aware HRI discovered a dilemma - maximizing task rewards often leads to decreased human trust, while maximizing human trust would compromise task performance. In this work, we address this dilemma by formulating the HRI process as...
-
Perspective-taking is the ability to perceive or understand a situation or concept from another individual's point of view, and is crucial in daily human interactions. Enabling robots to perform perspective-taking remains an unsolved problem; existing approaches that use deterministic or handcrafted methods are unable to accurately account for uncertainty in partially-observable settings. This wor...
-
The use of mobile robots in unstructured environments like the agricultural field is becoming increasingly common. The ability for such field robots to proactively identify and avoid failures is thus crucial for ensuring efficiency and avoiding damage. However, the cluttered field environment introduces various sources of noise (such as sensor occlusions) that make proactive anomaly detection diff...
-
Cooperative object transportation using multiple robots has been intensively studied in the control and robotics literature, but most approaches are either only applicable to omnidirectional robots or lack a complete navigation and decision-making framework that operates in real time. This paper presents an autonomous nonholonomic multi-robot system and an end-to-end hierarchical autonomy framewor...
-
Due to the high complexity of a football match, the opponents' strategies are variable and unknown. Thus predicting the opponents' future intentions accurately based on current situation is crucial for football players' decision-making. To better anticipate the opponents and learn more effective strategies, a deconfounded opponent intention inference (DOII) method for football multi-player policy ...
-
We present a new robotic drawing system based on stroke-based rendering (SBR). Our motivation is the artistic quality of the whole performance. Not only should the generated strokes in the final drawing resemble the input image, but the stroke sequence should also exhibit a human artist's planning process. Thus, when a robot executes the drawing task, both the drawing results and the way the robot...
-
Isolation wards operate in quarantine rooms to prevent cross-contamination caused by infectious diseases. Behind the benefits, medical personnel can have the infection risk from patients and the heavy workload due to the isolation. This work proposes a robot-assisted system to alleviate these problems in isolation wards. We conducted a survey about the medical staff's difficulties and envisioning ...
-
Change detection and irregular object extraction in 3D point clouds is a challenging task that is of high importance not only for autonomous navigation but also for updating existing digital twin models of various industrial environments. This article proposes an innovative approach for change detection in 3D point clouds using deep learned place recognition descriptors and irregular object extrac...
-
Magnetic microrobots exhibit enormous potential in targeted drug delivery owing to the remote wireless manipulation and minimum invasion for medical treatment. High degree of freedom offers the magnetic propelled robots extraordinary application prospect since they can be controlled precisely when different magnetic fields sources working cooperatively. However, the biocompatibility of microrobots...
-
This paper proposes a tightly-coupled visual-Doppler-Velocity-Log (visual-DVL) fusion method for underwater robot localization through integrating the velocity measurements from a DVL into a visual odometry (VO). Considering that employing the DVL measurements in dead-reckoning systems easily leads to error accumulation and suboptimal results in previous works, we directly integrate them into the ...
-
This paper develops a novel slip estimator using the invariant observer design theory and Disturbance Observer (DOB). The proposed state estimator for mobile robots is fully proprioceptive and combines data from an inertial measurement unit and body velocity within a Right Invariant Extended Kalman Filter (RI-EKF). By embedding the slip velocity into SE3 (3) matrix Lie group, the developed DOB-bas...
-
The outdoor navigation capabilities of ground robots have improved significantly in recent years, opening up new potential applications in a variety of settings. Cost-based representations of the environment are frequently used in the path planning domain to obtain an optimized path based on various objectives, such as traversal time or energy consumption. However, obtaining such cost representati...
-
The proliferation of unmanned vehicles offers many opportunities for solving environmental sampling tasks with applications in resource monitoring and precision agriculture. Informative path planning (IPP) includes a family of methods which offer improvements over traditional surveying techniques for suggesting locations for observation collection. In this work, we present a novel solution to the ...
-
Leveraging recent developments in black-box risk-aware verification, we provide three algorithms that generate probabilistic guarantees on (1) optimality of solutions, (2) recursive feasibility, and (3) maximum controller runtimes for general nonlinear safety-critical finite-time optimal controllers. These methods forego the usual (perhaps) restrictive assumptions required for typical theoretical ...
-
In this paper, we consider the closed-loop control problem of nonlinear robotic systems in the presence of probabilistic uncertainties and disturbances. More precisely, we design a state feedback controller that minimizes deviations of the states of the system from the nominal state trajectories due to uncertainties and disturbances. Existing approaches to address the control problem of probabilis...
-
Rather than traditional position control, impedance control is preferred to ensure the safe operation of industrial robots programmed from demonstrations. However, variable stiffness learning studies have focused on task performance rather than safety (or compliance). Thus, this paper proposes a novel stiffness learning method to satisfy both task performance and compliance requirements. The propo...
-
Reinforcement learning (RL) has achieved promising results on most robotic control tasks. Safety of learning-based controllers is an essential notion of ensuring the effectiveness of the controllers. Current methods adopt whole consistency constraints during the training, thus resulting in inefficient exploration in the early stage. In this paper, we propose an algorithm named Constrained Policy O...
-
Robotic cloth manipulation is an increasingly relevant area of research, challenging classic control algorithms due to the deformable nature of cloth. While it is possible to apply linear model predictive control to make the robot move the cloth according to a given reference, this approach suffers from a large dimensionality of the state-space representation of the cloth models. To address this i...
-
In head-to-head racing, performing tightly con-strained, but highly rewarding maneuvers, such as overtaking, require an accurate model of interactive behavior of the opposing target vehicle (TV). We propose to construct a prediction model given data of the TV from previous races. In particular, a one-step Gaussian process (GP) model is trained on closed-loop interaction data to learn the behavior ...
-
Energy tanks have gained popularity inside the robotics and control communities over the last years, since they represent a formidable tool to enforce passivity (and, thus, input/output stability) of a controlled robot, possibly interacting with uncertain environments. One weak point of passification strategies based on energy tanks concerns, however, their initialization. Indeed, a too large init...
-
A Data-Driven Approach to Synthesizing Dynamics-Aware Trajectories for Underactuated Robotic Systems
We consider joint trajectory generation and tracking control for under-actuated robotic systems. A common solution is to use a layered control architecture, where the top layer uses a simplified model of system dynamics for trajectory generation, and the low layer ensures approximate tracking of this trajectory via feedback control. While such layered control architectures are standard and work we...
-
Least-squares programming is a popular tool in robotics due to its simplicity and availability of open-source solvers. However, certain problems like sparse programming in the $\ell_{0}$- or $\ell_{0}-\mathbf{norm}$ for time-optimal control are not equivalently solvable. In this work, we propose a non-linear hierarchical least-squares programming (NL-HLSP) for time-optimal control of non-linear di...
-
Pneumatically operated soft actuators are increasingly researched due to their fabrication simplicity, actuation capabilities, and low production cost. Depending on the Soft Pneumatic Actuator (SPA) objective, its design can be modified to reach new bending angles or increase its actuation strength. However, increasing the abilities of Soft Pneumatic Actuators (SPAs) requires increasing the comple...
-
The existing method of detecting defects in train components, which relies on visual identification, requires extensive involvement from inspectors and presents certain limitations. In this study, a two-stage defect detection based on prior knowledge was developed, which first detects the types and positions of components, and then conducts targeted detection of possible existing defect types. The...
-
Autonomous mobile robots (AMRs) face challenges in efficiently covering complex environments. To navigate narrow and expansive areas, AMRs must have two essential attributes: compact size for confined spaces and larger size with omnidirectional locomotion for broader spaces. This study utilizes omnidirectional expand and collapse robots (OECRs) to demonstrate efficient area coverage. OECRs can col...
-
Symbolic task planning for robots is computationally challenging due to the combinatorial complexity of the possible action space. This fact is amplified if there are several sub-goals to be achieved due to the increased length of the action sequences. In this work, we propose a multi-goal symbolic task planner for deterministic decision processes based on Monte Carlo Tree Search. We augment the a...
-
Automatic Robotic Assembly Sequence Planning (RASP) can significantly improve productivity and resilience in modern manufacturing along with the growing need for greater product customization. One of the main challenges in realizing such automation resides in efficiently finding solutions from a growing number of potential sequences for increasingly complex assemblies. Besides, costly feasibility ...
-
Stretching the definition of the standard Sine profile allows building a generalized symmetric frequency-aware basis function that can be used to generate reference motion trajectories. Other profiles such as polynomials, sigmoid, and harmonic-based models can be equally used under the proposed technique. Despite being suitable at the level of any higher-order time derivative, in this study, the g...
-
This paper presents a vision-guided two-stage approach with force feedback to achieve high-precision and flexible gear assembly. The proposed approach integrates YOLO to coarsely localize the target workpiece in a searching phase and deep reinforcement learning (DRL) to complete the insertion. Specifically, DRL addresses the challenge of partial visibility when the on-wrist camera is too close to ...
-
This study focuses on the powder grinding process, which is a necessary step for material synthesis in materials science experiments. In material science, powder grinding is a time-consuming process that is typically executed by hand, as commercial grinding machines are unsuitable for samples of small size. Robotic powder grinding would solve this problem, but it is a challenging task for robots, ...
-
Automating leaf detection and physical leaf sample collection using Internet of Things (IoT) technologies is a crucial task in precision agriculture. In this paper, we present a deep learning-based approach for detecting and segmenting crop leaves for robotic physical sampling. We discuss a method for generating a physical dataset of agricultural crops. Our proposed pipeline incorporates using an ...
-
Measuring geometry of the printing road is key for detection of anomalies in 3D printing processes. Although commercial 3D printers can measure the extrusion height using various distance sensors, measuring of the width in real-time remains a challenge. This paper presents a visual in-situ monitoring system to measure width of the printing filament road in 2D patterns. The proposed system is compo...
-
In semiconductor manufacturing, lithography machines are becoming more and more sophisticated system of systems. As an example, a TWINSCAN wafer scanner machine is composed of a wafer, and reticle handlers, reticle, optics, and two wafer chains or systems. In previous studies, we covered the interactions between the reticle, optics, and wafer chains during the step-and-scan cycle. In this study, w...
-
For various automated palletizing tasks, the detection of packaging units is a crucial step preceding the actual handling of the packaging units by an industrial robot. We propose an approach to this challenging problem that is fully trained on synthetically generated data and can be robustly applied to arbitrary real world packaging units without further training or setup effort. The proposed app...
-
In this article, we present a mechanism and related path planning algorithm to construct light-duty barriers out of extruded, inflated tubes weaved around existing environmental features. Our extruded tubes are based on everted vine-robots and in this context, we present a new method to steer their growth. We characterize the mechanism in terms of accuracy resilience, and, towards their use as bar...
-
This study presents a bistable tensegrity robot that can perform repetitive jumps using one motor. This robot is based on a tensegrity structure that uses rigid plate-shaped compressors. To achieve bistability in this structure, we optimized the position of additional springs using a physics simulator that considers geometric constraints attributed to the collision between compression materials. A...
-
Soft robotics requires effective tools to educate the next generation of engineers and researchers. Stemming from a lack of universally accepted principles for education and with high barriers to entry in terms of fabrication and hardware, education to date has been highly ad hoc. We present a low-cost toolkit based on re-configurable balloon which allows rapid development of soft yet functional r...
-
We explore the idea of robotic mechanisms that can shift between soft and rigid states, with the long-term goal of creating robots that marry the flexibility and robustness of soft robots with the strength and precision of rigid robots. We present a simple yet effective method to achieve large and rapid stiffness variations by compressing and relaxing a flexure using cables. Next, we provide a dif...
-
Soft-Continuum Manipulators are of increasing interest to researchers for various non-destructive applications (minimally invasive surgery, fibroscopy, oncology, pipe exploration and many others). They are made with soft material or special arrangement of actuators allowing them to exhibit resilience and dexterity. The concept of Proprioceptive Soft-Continuum Manipulators still remains a major cha...
-
A robot must exhibit softness so as not to accidentally damage its environment. However, stiffness is also necessary, so that the robot can transmit forces and perform tasks. In soft robotics, it is desirable to be able to switch between two states, namely a flexible state for adapt ion to the environment and a rigid state for the transmission of forces. String jamming mechanisms, which comprise m...
-
Soft actuators have several advantages, including large deformation, safety and adaptability to the environment, and shock absorbance. However, they are weak against sharp objects owing to their soft bodies. This paper proposes a novel protective skin mechanism with an exhaustive arrangement of tiny rigid bodies. Small pieces were sewed on an elastic sheet using Kevlar strings. We conducted some m...
-
Soft robots made of flexible materials are highly adaptive, easy to fabricate, and safer to interact with. One of the ways for soft robots to interact with the surrounding environment is through their deformable bodily characteristics including internal body stiffness and external body friction. Though the flexibility of soft-bodied robots has been rigorously studied, the frictional skin of such s...
-
Soft compliant microrobots have the potential to deliver significant societal impact when deployed in applications such as search and rescue. In this research we present mCLARI, a body compliant quadrupedal microrobot of 20mm neutral body length and 0.97g, improving on its larger predecessor, CLARI. This robot has four independently actuated leg modules with 2 degrees of freedom, each driven by pi...
-
Due to the advantages of high flexibility, large workspace, and good human-body compatibility, flexible tendon-driven surgical continuum robots have attracted a lot of attention in robot-assisted minimally invasive surgery. However, due to the coupling of the position and angle of the continuum robot, and the easy deformation of the external force, its inverse kinematics solution has always been a...
-
This paper presents the design of a new multi-arm robotic system with mobile magnetic actuation and extracorpo-real ultrasound guidance dedicated to magnetic catheterization. The kinematic model of the external mobile actuation arm (EMAA) and extracorporeal ultrasound-integrated tracking arm (EUTA) are derived based on Denavit-Hartenberg (DH) parameters, including specially designed end-effectors....
-
The rigid and straight nature of conventional surgical drills and screwdrivers makes it difficult to access the posterior mandible for fracture reduction without the creation of facial incisions. To assist transoral mandibular fracture reduction in hard-to-reach areas, we propose a shared-control dexterous robotic system. The end effector of this system is an articulated drilling/screwing tool to ...
-
This paper is concerned with the issue of the kinematic model-based bending control for the magnetically actuated robotic endoscope and its application for automatic retroflexion. By the utilization of the Cosserat rod theory and the transformation in the magnetic tip of the endoscope, the comprehensive kinematic model of the magnetically-actuated robotic endoscope is established. Afterward, a mag...
-
In robotic laser surgery, shape prediction of an one-shot ablation crater is an important problem for minimizing errant overcutting of healthy tissue during the course of pathological tissue resection and precise tumor removal. Since it is difficult to physically model the laser-tissue interaction due to the variety of optical tissue properties, complicated process of heat transfer, and uncertaint...
-
Miniature Magnetic Rotating Swimmers (MMRSs) are untethered machines containing magnetic materials. An external rotating magnetic field produces a torque on the swimmers to make them rotate. MMRSs have propeller fins that convert the rotating motion into forward propulsion. This type of robot has been shown to have potential applications in the medical realm. This paper presents new MMRS designs w...
-
A hybrid continuum robot design is introduced that combines a proximal tendon-actuated section with a distal telescoping section comprised of permanent-magnet spheres actuated using an external magnet. While, individually, each section can approach a point in its workspace from one or at most several orientations, the two-section combination possesses a dexterous workspace. The paper describes kin...
-
Telesurgery has a clear potential for providing high-quality surgery to medically underserved areas like rural areas, battlefields, and spacecraft; nevertheless, effective methods to overcome unreliable communication systems are still lacking. Furthermore, it is not well understood how users react at the moment of communication loss and also during the loss. In this paper, we aim to analyze human ...
-
The introduction of image-guided surgical navigation (IGSN) has greatly benefited technically demanding surgical procedures by providing real-time support and guidance to the surgeon during surgery. To develop effective IGSN, a careful selection of the surgical information and the medium to present this information to the surgeon is needed. However, this is not a trivial task due to the broad arra...
-
This article introduces a novel suture managing device (SMD) and new suture management controller to enable single-arm suture management during autonomous suturing with the Smart Tissue Autonomous Robot (STAR). The primary function of the SMD is to tension and manage the suture thread, a task that was previously carried out by a second manipulator or a human assistant. The SMD and its controller a...
-
Needle picking is a challenging manipulation task in robot-assisted surgery due to the characteristics of small slender shapes of needles, needles' variations in shapes and sizes, and demands for millimeter-level control. Prior works, heavily relying on the prior of needles (e.g., geometric models), are hard to scale to unseen needles' variations. In this paper, we present the first end- to-end le...
-
Reinforcement learning is still struggling with solving long-horizon surgical robot tasks which involve multiple steps over an extended duration of time due to the policy exploration challenge. Recent methods try to tackle this problem by skill chaining, in which the long-horizon task is decomposed into multiple subtasks for easing the exploration burden and subtask policies are temporally connect...
-
Generating dynamic jumping motions on legged robots remains a challenging control problem as the full flight phase and large landing impact are expected. Compared to quadrupedal robots or other multi-legged robots, bipedal robots place higher requirements for the control strategy given a much smaller support polygon. To solve this problem, a novel heuristic landing planner is proposed in this pape...
-
The estimation of external joint torque and contact wrench is essential for achieving stable locomotion of humanoids and safety-oriented robots. Although the contact wrench on the foot of humanoids can be measured using a force-torque sensor (FTS), FTS increases the cost, inertia, complexity, and failure possibility of the system. This paper introduces a method for learning external joint torque s...
-
A biped robot walking in a crowd must avoid falls and collisions at the same time. The latter is usually addressed through Passive Safety (PS), which guarantees that the robot is at rest when a collision is inevitable. Since PS may limit the robot's mobility, the purpose of this work is to introduce and explore the novel concept of Time To Danger (TTD) as an alternative. For a given robot motion, ...
-
In order for a humanoid robot to balance on the movable ground, balance feedback control in response to its unpredictable movement is required. However, feedback control in response to ground movement has the following two issues, (A) Interaction between the ground dynamics and the balance control may cause vibration. (B) The balance control may rather deteriorate the stability due to the response...
-
To implement successful bipedal locomotion in a robot, its center of pressure (CoP) is placed on a supporting area. To achieve locomotion, many studies have focused on the lower body. Given that the human upper body has a large mass and its behavior influences locomotion even in the case of the robot, this study investigates the effect of the upper body, which contains moving arms and a twisting t...
-
Mobility, a critical factor in quality of life, is often rehabilitated using simplistic solutions, such as walkers. Exoskeletons (wearable robotics) offer a more sophisticated rehabilitation approach. However, non-adherence to externally worn mobility aids limits their efficacy. Here, we present the concept of a fully implantable assistive limb actuator that overcomes non-adherence constraints, an...
-
This work focuses on the persistent monitoring problem, where a set of targets moving based on an unknown model must be monitored by an autonomous mobile robot with a limited sensing range. To keep each target's position estimate as accurate as possible, the robot needs to adaptively plan its path to (re-)visit all the targets and update its belief from measurements collected along the way. In doi...
-
The goal of intelligent embodied agents is to learn how to explore within the environment, interact with objects, and understand the environment in order to achieve task objectives. There are two main approaches to training such agents: one is to train an action policy that performs the task goal through end-to-end learning, and the other is to construct a policy by implementing the necessary abil...
-
Does the norm of first-come-first-serve (FCFS) equally apply to those piloting a Mobile Remote Presence (MRP) system as to those who are physically present with it? While telepresence robots could make social interactions more accessible and enjoyable for geographically-constrained individuals, such an outcome requires both pilots and local users of MRPs to share the same social norm expectations ...
-
Customer satisfaction is not only determined by how workers treat customers but also by how they treat their coworkers. In this sense, we examined whether the dynamics of robot workers influence user satisfaction. To optimize the robot team's atmosphere for Korean culture, we adopted the Korean honorific language system to express hierarchy. We set four types of relationships between two robots: C...
-
The purpose of this research is to study the impact of robot participation in group conversations and assess the effectiveness of different addressing policies. The study involved a total of 300 participants, who were divided into groups of four and engaged in a dialogue with a humanoid robot. The robot acted as a moderator, using information obtained during the conversation to determine which spe...
-
How should a robot speak in a formal, quiet and dark, or a bright, lively and noisy environment? By designing robots to speak in a more social and ambient-appropriate manner we can improve perceived awareness and intelligence for these agents. We describe a process and results toward selecting robot voice styles for perceived social appropriateness and ambiance awareness. Understanding how humans ...
-
Politeness is at the core of the common set of behavioral norms that regulate human communication and is therefore of significant interest in the design of Human-Robot Interactions. In this paper, we investigate how the politeness behaviors of a humanoid robot impact human decisions about where to join a group of two robots. We also evaluate the resulting impact on the perception of the robot's po...
-
Spatial formations can give many social cues, such as illustrating a group of people are having a conversation (social affiliation), or that they are trying to move swiftly through a space (functional goal). This work explored how people perceive varied robots formations while navigating through a space and approaching people. Evaluation occurred across four different geometric formations: wedge, ...
-
Conveying a robot's target mood is crucial to successful social interactions. The robot's expressive performance must be appropriate, persuasive, and consistent. However, this is challenging when interactions contain a mixture of scripted and improvised content, such as those generated by language models. In this paper, we take on the task of teaching robots to stay in character, that is to say, e...
-
In this paper, we propose a system that supports operators who provide services to customers using teleoperated robots. We observed that unprofessional or lazy operators of teleoperated robots are a risk for businesses as they are likely to speak in ways that are inappropriate for customer services. The proposed system lets competent operators talk freely to customers and thus provide high quality...
-
Predicting pedestrian motion is essential for developing socially-aware robots that interact in a crowded environment. While the natural visual perspective for a social interaction setting is an egocentric view, the majority of existing work in trajectory prediction therein has been investigated purely in the top-down trajectory space. To support first-person view trajectory prediction research, w...
-
For socially assistive robots to achieve widespread adoption, the ability to learn new tasks in the wild is critical. Learning from Demonstration (LfD) approaches are a popular method for learning in the wild, but current methods require significant amounts of data and can be difficult to interpret. Interactive Task Learning (ITL) is an emerging learning paradigm that aims to teach tasks in a stru...
-
We propose, analyze, and experimentally verify a new proactive approach for robot social navigation driven by the robot's “opinion” for which way and by how much to pass human movers crossing its path. The robot forms an opinion over time according to nonlinear dynamics that depend on the robot's observations of human movers and its level of attention to these social cues. For these dynamics, it i...
-
Backchanneling models, designed to enhance the interactive capabilities of robots, have primarily been trained on human-human interaction data. However, applying such data directly to social robots raises concerns due to dissimilarities in the way humans and robots exhibit verbal and nonverbal behaviors, particularly in the domain of emotional backchannels. This research aims to address this gap b...
-
Lower-limb exoskeletons may experience errors in operational settings, where an expected assistive torque is missing. These errors may affect user's gait strategies and perception of the exoskeleton's performance, leading to impacted human-exoskeleton fluency and user trust in the system. In this study, we introduced five different exoskeleton control algorithms with fixed error rates up to 10% er...
-
Quadruped robots have the potential to guide blind and low vision (BLV) people due to their highly flexible locomotion and emotional value provided by their bionic forms. However, the development of quadruped guide robots rarely involves BLV users' participatory designs and evaluations. In this paper, we conducted two empirical experiments both in indoor controlled and outdoor field scenarios, exp...
-
This paper proposes a novel operation for controlling a mobile robot using a head-mounted device. Conventionally, robots are operated using computers or a joystick, which creates limitations in usability and flexibility because control equipment has to be carried by hand. This lack of flexibility may prevent workers from multitasking or carrying objects while operating the robot. To address this l...
-
This paper introduces a novel interactive approach —Exploratory Experiences— that aims to improve the ability of people to reason about the capabilities and limitations of robotic technology. We focus on two areas: robot navigation and object detection. We evaluate the Exploratory Experiences with a novel approach that measures the participant's ability to predict when the robot will fail, followi...
-
The evaluation of systems under non-ideal conditions is a research problem, particularly in robotic applications for the rehabilitation of people with disabilities. Accordingly, the evaluation of algorithmic strategies for robustness validation under different non-ideal conditions is a current challenge for the scientific community. Therefore, in this study, a computational methodology based on Ex...
-
Learning from human feedback is an effective way to improve robotic learning in exploration-heavy tasks. Compared to the wide application of binary human feedback, scalar human feedback has been used less because it is believed to be noisy and unstable. In this paper, we compare scalar and binary feedback, and demonstrate that scalar feedback benefits learning when properly handled. We collected b...
-
The identification of human joint impedance is necessary for various applications, such as improving rehabilitation efficiency or monitoring the human operator's state (fatigue, stress). To this end, in this paper we combine robot's payload identification methods with sliding window recursive least squares algorithm allowing a continuous identification of the varying human joint model without the ...
-
The human limb possesses a remarkable capacity to absorb energy during physical human-robot interaction (pHRI), which can be quantified as the biomechanical “Excess of Passivity” (EoP) using non-linear control theory. This biome-chanical passivity index can be used to reduce conservatism and increase the transparency of pHRI stabilizers. Previous work on EoP has used system identification techniqu...
-
In the context of physical human-(tele)robot interaction, passivity-based stabilizers have been used to guarantee the physical or (tele) physical stability. In most of these examples, human biomechanics is considered an inherently passive system that dissipates energy. This assumption may not hold true when the interaction is implemented in the force-position domain, even though such a setting wou...
-
Legged robots companions may one day assist humans with everyday tasks, but their possible impact on human gait is unknown. While previous studies have shown that humans adjust their gait when walking with other humans, it is uncertain whether walking with legged robots would yield similar results. In this study, we measured the gait of healthy participants (N = 14) while they walked alone and wit...
-
This paper explores how the personality traits of robot operators can influence their task performance during remote control of robots. It is essential to explore the impact of personal dispositions on information processing, both directly and indirectly, when working with robots on specific tasks. To investigate this relationship, we utilize the open-access multi-modal dataset MOCAS to examine th...
-
Typical black-box optimization approaches in robotics focus on learning from metric scores. However, that is not always possible, as not all developers have ground truth available. Learning appropriate robot behavior in human-centric contexts often requires querying users, who typically cannot provide precise metric scores. Existing approaches leverage human feedback in an attempt to model an impl...
-
Modern exoskeletons are typically developed to optimize for a single, physiological objective, the “gold standard” of which is a reduction of the wearer's metabolic rate. However, recent research suggests that these changes in metabolic rate are not yet perceivable on average. To address this gap, this study explores a novel economic value metric to quantify the value of exoskeleton assistance. Th...
-
The emergence of harvesting robotics offers a promising solution to the issue of limited agricultural labor resources and the increasing demand for fruits. Despite notable advancements in the field of harvesting robotics, the utilization of such technology in orchards is still limited. The key challenge for harvesting robots is to improve the operational efficiency. Taking into account inner-arm c...
-
Tree fruit growers around the world are facing labor shortages for critical operations, including harvest and pruning. There is a great interest in developing robotic solutions for these labor-intensive tasks, but current efforts have been prohibitively costly, slow, or require a reconfiguration of the orchard in order to function. In this paper, we introduce an alternative approach to robotics us...
-
Animal behaviour can reflect the health and physiological stage of the animal. Animal behaviour recognition is a vital part of automated farming systems. Although image-based deep learning algorithms can accurately identify animal behaviour, the lack of data on animal abnormal behaviour makes the practical deployment of models of limited significance. At the same time, the ageing of farm monitorin...
-
Active perception for fruit mapping and harvesting is a difficult task since occlusions occur frequently and the location as well as size of fruits change over time. State-of-the-art viewpoint planning approaches utilize computationally expensive ray casting operations to find good viewpoints aiming at maximizing information gain and covering the fruits in the scene. In this paper, we present a no...
-
Panoptic segmentation provides both holistic and detailed image parsing information at both the pixel and the instance level. However, the computational burdens restrict its applications in real-time scenarios. A potential approach to learn more efficient models is to employ knowledge distillation. However, previous knowledge distillation schemes have focused mainly on classification with limited ...
-
Our world is non-static, and robots should be able to track its changing geometry. For tracking changes, data asso-ciations between 3D points over time are key. In this paper, we investigate the problem of associating 3D points on plant organs from different mapping runs over time while the plants grow. We achieve a high spatial-temporal matching performance by combining 3D RGB-D SLAM, visual plac...
-
Crop monitoring is crucial for maximizing agricultural productivity and efficiency. However, monitoring large and complex structures such as sweet pepper plants presents significant challenges, especially due to frequent occlusions of the fruits. Traditional next-best view planning can lead to unstructured and inefficient coverage of the crops. To address this, we propose a novel view motion plann...
-
Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. Obtaining such 3D information is non-trivial as agricultural environments are often repetitive and cluttered, and one has to account for the p...
-
Autonomous navigation is crucial for achieving the full automation of agricultural research and production management using agricultural robots. In this paper, we present a vision-based autonomous navigation approach for agriculture robots in trellised cropping systems, which stands out for its remarkable performance achieved entirely without human annotation. We propose a novel learning-based met...
-
Modern robotics has enabled the advancement in yield estimation for precision agriculture. However, when applied to the olive industry, the high variation of olive colors and their similarity to the background leaf canopy presents a challenge. Labeling several thousands of very dense olive grove images for segmentation is a labor-intensive task. This paper presents a novel approach to detecting ol...
-
The agricultural setting poses additional challenges for robotic manipulation, as fruit is firmly attached to plants and the environment is cluttered and occluded. Therefore, accurate feedback about the grasp state is essential for effective harvesting. This study examines the different states involved in fruit picking by a robot, such as successful grasp, slip, and failed grasp, and develops a le...
-
Automating the process of fleece contaminant removal has the potential to drastically improve the quality of wool leaving the farm gate. Towards this goal, we present a method to automatically extract skirting lines, i.e., the separations between clean and contaminated wool of a fleece using RGB images. We propose a learning-based sparse-to-dense approach for estimating the non-rigid deformation o...
-
Large-scale in-situ 3D reconstruction of crop fields presents a challenging task, as the 3D crop structures play a crucial role in plant phenotyping and significantly influence crop growth and yield. While existing efforts focus on close-range plants, only a limited number of deep learning-based methods have been developed explicitly for large-scale 3D crop reconstruction, mainly due to the scarci...
-
Acquiring dynamics is critical for robot learning and is fundamental to planning and control. This paper concerns two fundamental questions: How can we learn a model that covers massive, diverse robot dynamics? Can we construct a model that lifts the data-collection pain and domain expertise required for building specific robot models? We learn the dynamics involved in a dataset containing a large...
-
Micro Aerial Vehicles (MAVs) often face a high risk of collision during autonomous flight, particularly in cluttered and unstructured environments. To mitigate the collision impact on sensitive onboard devices, resilient MAVs with mechanical protective cages and reinforced frames are commonly used. However, compliant and impact-resilient MAVs offer a promising alternative by reducing the potential...
-
Rescue robots require versatility and the capability to operate in various environments to carry out a diverse set of tasks effectively. The eccentric paddle (ePaddle) mechanism stands out for its high efficiency and adaptability. Generally, it is designed as a quadruped robot with a combined structure for fully-actuated control, this approach is often both inefficient and inflexible due to the re...
-
The safe deployment of autonomous vehicles relies on their ability to effectively react to environmental changes. This can require maneuvering on varying surfaces which is still a difficult problem, especially for slippery terrains. To address this issue we propose a new approach that learns a surface-aware dynamics model by conditioning it on a latent variable vector storing surface information a...
-
This paper develops a data-based moving horizon estimation (MHE) method for agile quadrotors. Accurate state estimation of the system is paramount for precise trajectory control for agile quadrotors; however, the high level of aerodynamic forces experienced by the quadrotors during high-speed flights make this task extremely challenging. These complex turbulent effects are difficult to model and t...
-
The Spring-Loaded Inverted Pendulum (SLIP) is one of the simplest models of robot locomotion. SLIP is commonly used to predict the center of mass motion and derive simple control laws for stable locomotion. However, the SLIP model is not integrable, which means that no closed-form relation can be derived to understand how the design and control parameters of the SLIP model affect stable locomotion...
-
Soft vibrational bristlebots are robots with deformable bristles on their outside that propel the robot through the stick slip motion of the bristle tips interacting with the ground when a vibration is induced in the robot. Experimental results and theoretical analysis of the dynamics for this style of robot have been investigated on flat surfaces. However, for the soft vibrational bristlebots tra...
-
We propose an orthogonal collocation method (CM) for solving Cosserat rod Dirichlet-Neumann boundary value problems in static and dynamic modes. We interpolate the internal loading and collocate the strong form of the differential equations. The method uses Chebyshev polynomials in order to minimize Runge's phenomenon. The time derivatives are implicitly discretized using the backward differentiat...
-
Flying robots can exploit perching abilities to position themselves on strategically-chosen locations and monitor the areas of interest from a critical vantage point. Moreover, they can significantly extend their battery life by turning off the propulsion systems when carrying out a surveillance mission. However, unknown disturbances arise from the physical interactions between the robot and the o...
-
Most land-air amphibious UAVs feature a four-wheel design that limits their adaptability in narrow and uneven spaces. This study proposes the rotor flywheel as a new land-air design that integrates a one-wheel structure and eight-rotor wings for more flexible motion. The dynamics model is then conducted with the Kane method, finding two power-saving self-balance state while rolling. Its controller...
-
System Identification and Control of Front-Steered Ackermann Vehicles Through Differentiable Physics
In this paper, we address the problem of system identification and control of a front-steered vehicle which abides by the Ackermann geometry constraints. This problem arises naturally for on-road and off-road vehicles that require reliable system identification and basic feedback controllers for various applications such as lane keeping and way-point navigation. Traditional system identification r...
-
Hardware-in-the-loop simulation (HILS) allows a more realistic evaluation of control approaches than what is possible with pure software simulations, but without the actual complexity of the complete system. This is important for some complex systems such as orbital robots, where testing of the system is typically not possible after its launch, and an on-ground replica is used to validate the perf...
-
This work introduces a new design paradigm for robotic legs. Our concept extends upon classical series elastic actuation and directly integrates the series compliance into the structure of the leg. This integration reduces the mechanical design complexity and can potentially reduce the overall weight of the leg. In this paper, we introduce a prototype leg with a continuously compliant shank and de...
-
This paper outlines a novel versatile geometric method for the estimation of the efficiency of multistage compound planetary gearboxes. The approach is based on a virtual pitch point modeling that allows for accurate representation of the gear interaction with forces applied at the pitch point. When this modeling is applied to all gears in a multistage compound planetary gearbox, the gearbox effic...
-
This paper proposes a combined optimization and learning method for impact-friendly, non-prehensile catching of objects at non-zero velocity. Through a constrained Quadratic Programming problem, the method generates optimal trajectories up to the contact point between the robot and the object to minimize their relative velocity and reduce the impact forces. Next, the generated trajectories are upd...
-
This paper proposes an online estimation model for 2D arm stiffness in humans. The proposed model is based on recent physiological findings which suggest that: (1) joint stiffness is linearly related to the magnitude of joint torque and increases to compensate for environmental disturbances; and (2) the endpoint stiffness of the arm is proportional to grasp force. To validate the proposed model, p...
-
The use of impedance control has become widespread in applications requiring simultaneous position tracking and compliance in contact. However, disturbances such as friction and model uncertainties can adversely affect the performance of impedance-based motion control. The disturbance observer (DOB) has been proposed to address this issue, which is a widely-utilized robust controller that eliminat...
-
Recently, several approaches have attempted to combine motion generation and control in one loop to equip robots with reactive behaviors, that cannot be achieved with traditional time-indexed tracking controllers. These approaches however mainly focused on positions, neglecting the orientation part which can be crucial to many tasks e.g. screwing. In this work, we propose a control algorithm that ...
-
Robot teleoperation has been studied for the past 70 years and is relevant in many contexts, such as in the handling of hazardous materials and telesurgery. The COVID19 pandemic has rekindled interest in this topic, but the existing robotic education kits fall short of being suitable for teleoperated robotic manipulator learning. In addition, the global restrictions of motion motivated large inves...
-
Sophisticated manipulation requires both compliance and accuracy. While tactile robots excel at being compliant, their accuracy is often inadequate for complex manipulation. Contact-rich assembly tasks, such as the insertion and manipulation of objects with small tolerances pose an enormous challenge. Complex, highly integrated assemblies, especially in high-tech areas such as robotics, sensors, o...
-
Simulation systems of robots can facilitate the prediction, development, and debugging of robotic systems. However, they seldom applied in robotics education for primary and secondary school students. In this paper, we present a sim-to-real robotic system that enables students to optimize their algorithms in a simulated environment and validate them in a remote physical laboratory with data logs a...
-
Over the last decade, a plethora of soft robotic devices have been proposed for the execution of complex grasping and dexterous manipulation tasks. Tasks requiring such increased dexterity are typically executed using fully-actuated, rigid end-effectors equipped with sophisticated sensing and controlled with complex control laws. The new class of soft robotic devices offers an alternative to the t...
-
Manual oropharyngeal (OP) swab sampling is an intensive and risky task. In this article, a novel OP swab sampling device of low cost and high compliance is designed by combining the visuotactile sensor and the pneumatic actuator-based gripper. Here, a concave visuotactile sensor called CoTac is first proposed to address the problems of high cost and poor reliability of traditional multi-axis force...
-
This paper presents the underactuated vacuum gripper named Polypus. What is unique in Polypus is that it combines under-actuation and vacuum grasping to apply both power and unilateral grasp to objects of various shape and geometry. In addition, the gripper features modularity, i.e., single phalanges can be added or removed based on the application. The high flexibility also comes with a cost-effe...
-
In this paper, we propose a lip-inspired passive jamming gripper by mimicking teeth structures from a dog's oral structure. Animal lips are hydrostatic structures. To mimic the features, which are usually soft but rigid when contacted, we used the passive particle jamming effect. To increase the adaptability of our previous gripper to grasp various shaped objects, we focused on the dogs' oral stru...
-
A problem that plagues robotic grasping is the misalignment of the object and gripper due to difficulties in precise localization, actuation, etc. Under-actuated robotic hands with compliant mechanisms are used to adapt and compensate for these inaccuracies. However, these mechanisms come at the cost of controllability and coordination. For instance, adaptive functions that let the fingers of a tw...
-
We present an inflatable finger pad that allows regular parallel-jaw grippers to vary their grasp stiffness while maintaining a contact force and contact to non-planar surfaces. An eversible radial bellows structure made of silicone rubber allows the pad to extend to four times its original height and to retract into a rigid pod when not needed. The bellows act as passive universal joints when eve...
-
Robot grippers are widely used in industrial automation for pick-and-place tasks on a variety of objects. Whilst the majority of commercial grippers are capable of establishing stable grasps, few can perform in-hand-manipulation (IHM). IHM is has the potential to increase robotic motion efficiency, yet most IHM-capable manipulation platforms are anthropomorphic in nature and cost over $10,000, pos...
-
Drone grasping on power lines for recharging is challenging since it requires the gripper to be lightweight, carried by a drone, and efficient for a firm grasp. A deep understanding of the power line nature and its magnetic characteristic helps ease such challenges and bring new knowledge to gripper design. In this work, a novel adaptive, lightweight, and fail-safe magnetic gripper with a rechargi...
-
InstaGrasp: An Entirely 3D Printed Adaptive Gripper with TPU Soft Elements and Minimal Assembly Time
Fabricating existing and popular open-source adaptive robotic grippers commonly involves using multiple professional machines, purchasing a wide range of parts, and tedious, time-consuming assembly processes. This poses a significant barrier to entry for some robotics researchers and drives others to opt for expensive commercial alternatives. To provide both parties with an easier and cheaper (und...
-
A robotic finger is successfully designed, fabricated, analyzed, and examined. The finger consists of bones, joint ligaments, and an extensor hood. Driven by two tendons, it is two degrees on the freedom finger. Although the behavior of this design is not uniform, it provides useful dexterity, sensitivity, and versatility. The artificial bone is lightweight and compact. The actuation is backdrivab...
-
Autonomous Robotic Drilling System for Mice Cranial Window Creation: An Evaluation with an Egg Model
Robotic assistance for experimental manipulation in the life sciences is expected to enable precise manipulation of valuable samples, regardless of the skill of the scientist. Experimental specimens in the life sciences are subject to individual variability and deformation, and therefore require autonomous robotic control. As an example, we are studying the installation of a cranial window in a mo...
-
This paper presents a novel augmented reality (AR)-assisted orthopedic surgical robotic system based on Head-Mounted Display (HMD) devices. The proposed system can overlay the preoperative plans over the patient's anatomy and provide useful guidance for surgeons during interventions, with integrated calibration and registration components. A novel bi-directional generalised point set registration ...
-
Despite their impressive performance in various surgical scene understanding tasks, deep learning-based methods are frequently hindered from deploying to real-world surgical applications for various causes. Particularly, data collection, annotation, and domain shift in-between sites and patients are the most common obstacles. In this work, we mitigate data-related issues by efficiently leveraging ...
-
In this paper, a position-based visual-servoing control approach is introduced for a robotic camera holder to improve ergonomics and reduce mental stress during brain surgery. The visual tracking system controls and moves the robotic camera holder by following a selected surgical instrument without the need for artificial markers. The system was validated using a 7 Degree-of-Freedoms (DoFs) redund...
-
Augmented reality (AR) is considered one of the most promising solutions for safer procedures in several surgical specialities. Fusing patient-specific pre-operative information, typically 3D models extracted from CT scans or MRI, with real-time surgical images allows the surgeon to have detailed information on the anatomical structure of the surgical target intra-operatively. The coupling of AR a...
-
Ocular mobility disorders such as strabismus af-fect millions of people. Patients' descriptions of their symptoms, such as what they see and how their vision has changed, are important for ophthalmologists to diagnose, monitor pro-gression, and evaluate treatment effectiveness. However, such verbal depiction may be vague and Subjective. A data-driven simulator that visualizes abnormal vision exper...
-
Accurate segmentation of surgical instrument tip is an important task for enabling downstream applications in robotic surgery, such as surgical skill assessment, tool-tissue interaction and deformation modeling, as well as surgical autonomy. However, this task is very challenging due to the small sizes of surgical instrument tips, and significant variance of surgical scenes across different proced...
-
Pericardiocentesis is an important surgical intervention to treat a medical condition called pericardial effusion, during which excessive fluid accumulates around the heart, potentially leading to life-threatening situation. It involves the insertion of a needle and catheter towards the heart into the pericardial space to drain the excessive fluid under ultrasound (US) guidance. The risky procedur...
-
Studies of the human brain during natural activities, such as locomotion, would benefit from the ability to image deep brain structures during these activities. While Positron Emission Tomography (PET) can image these structures, the bulk and weight of current scanners are not compatible with the desire for a wearable device. This has motivated the design of a robotic system to support a PET imagi...
-
This study proposes a novel diagnosis framework to decrease the early detection miss rate of colorectal cancer (CRC) polyps by using a hypersensitive vision-based tactile sensor (HySenSe) and a deep residual neural network. The HySenSe generates high-resolution 3D textural images of 160 realistic polyp phantoms for accurate classification via the proposed deep learning (DL) architecture. The DL mo...
-
This paper proposes a smart handheld textural sensing medical device with complementary Machine Learning (ML) algorithms to enable on-site Colorectal Cancer (CRC) polyp diagnosis and pathology of excised tumors. The proposed unique handheld edge device benefits from a unique tactile sensing module and a dual-stage machine learning algorithms (composed of a dilated residual network and a t-SNE engi...
-
All-around, real-time navigation and sensing across the water environments by miniature soft robotics are promising, for their merits of small size, high agility and good compliance to the unstructured surroundings. In this paper, we propose and demonstrate a mantas-like soft aquatic robot which propels itself by flapping-fins using rolled dielectric elastomer actuators (DEAs) with bending motions...
-
Unlike most human-engineered systems, biological systems are emergent from low-level interactions, allowing much broader diversity and superior adaptation to complex environments. Inspired by the process of morphogenesis in nature, a bottom-up design approach for robot morphology is proposed to treat a robot's body as an emergent response to underlying processes rather than a predefined shape. Thi...
-
In this paper, we propose a system for real-time rat pose estimation based on stereo vision. The system is dedicated to robot-rat interaction research. First, we design a lightweight, high-resolution network (RRKDNet) for keypoint detection of the rat. The network is trained on a dataset of rat images, which are captured by the robotic rat in first-person view. Second, based on the keypoint detect...
-
Existing in-pipe robots have insufficient adaptability when dealing with accidents in unfamiliar pipe environments. Developing a pipe robot that can be designed and manufactured quickly is one solution. The tensegrity structure is a self-stressing spatial structure formed by the interaction of rigid members and flexible cables, which has the advantages of simple structure, good flexibility, deform...
-
The majority of current research on reinforcement learning (RL) for snake robot control do not sufficiently account for the spatial and temporal dependencies within the robot or its interaction with its environment during movement. To address this issue, we propose an RL based multi-layer Bayesian method for autonomous snake robot control, which handles challenging scenarios and improves navigatio...
-
Inspired by human proteins that are synthesized from only 20 types of amino acids, the development of self-assembly methods that allow robots to be built simply by randomly stirring the parts has been explored for many years. The key challenges include how to synthesize parts in pieces into a three-dimensional functional structure in a practical time, and subsequently, achieve a controlled robotic...
-
This paper presents an estimation and control framework to stabilize the parallel motion of a school of robotic fish using sensory feedback. Each robot is modeled as a constant speed, planar, self-propelled particle that produces a flowfield according to a dipole potential flow model. An artificial lateral line system senses pressure fluctuations at several locations along each robot's body. The e...
-
A small-scale jumping robot can reach obstacles much larger than its size. It is important for a jumping robot to perform intermittent jumps to cross through rough terrains. However, the limitations of conventional structures hinder the further integration of functions to a miniature (sub-50 g) jumping robot. No sub-50 g jumpers could perform intermittent jumps with adjustable jumping trajectories...
-
Insects have extraordinary navigational abilities. Monarch butterflies migrate every year to the same forest over hundreds of kilometers, desert ants find their way back to the nest tens of meters away and dung beetles maintain the same heading direction over meters. The performance of these agents has been optimized by evolution over the last 500 million years leading to power-efficient, low-late...
-
Animals' anatomies have control systems combined with multi-motors and high-bandwidth sensors. Their complicated mechanisms give them high maneuverability with sufficient inertial stabilization performance during walking, jumping, and flying. From the point of aerial locomotion, flying insects use abdomen reflexes to stabilize their head positions. Articulation of the thoracic-abdominal joint cont...
-
Unmanned aerial vehicles (UAVs) have revolutionised various industries, such as agriculture, remote sensing, and infrastructure inspection. To explore new designs and improve UAV flight performance, roboticists are seeking inspiration from nature. In this paper, we present a bioinspired tailsitter UAV utilizing shape-morphing wings with aerofoil-shaped artificial feathers. The design of the UAV is...
-
Biomimetic robotics can help support underwater exploration and monitoring while minimizing ecosystem distur-bance. It also has potential applications in sustainable aqua-farming management, biodiversity preservation, and animal-robot interaction studies. This study proposes a bio-inspired control strategy for an underactuated robotic fish, which utilizes a single DC motor to drive a mechanism tha...
-
Lane keeping, as a fundamental functionality of autonomous navigation, remains a challenging task for autonomous robots and vehicles. Recently, spiking neural networks (SNNs) have gained attention and research interest due to their biological plausibility and application potential on neuromorphic processors. SNNs have also been successfully deployed on robots to solve autonomous navigation problem...
-
Optimal placement of charging stations in a workspace is a crucial problem to address, for efficient operation of battery-driven mobile robots. When the battery charge of a robot reaches a certain threshold, the robot must be able to reach a nearby charging station to recharge its battery. In this paper, we deal with two different versions of the optimization problem related to the optimal placeme...
-
This paper introduces an extension of the LQR-tree algorithm, which is a feedback-motion-planning algorithm for stabilizing a system of ordinary differential equations from a bounded set of initial conditions to a goal. The constructed policies are represented by a tree of exemplary system trajec-tories, so called demonstrations, and linear-quadratic regulator (LQR) feedback controllers. Consequen...
-
As the future of autonomous underwater vehicle (AUV) deployments tends to multi-vehicle systems, new approaches in coordination and control are needed. In this work, we consider the problem of simultaneous survey and inspection where one vehicle dynamically discovers objects while another vehicle must inspect as many of the objects as possible over the course of the mission. This requires a fully ...
-
Relative Roughness Measurement Based Real-Time Speed Planning for Autonomous Vehicles on Rugged Road
In order to guarantee autonomous vehicles' autonomy, mobility, and ride quality in rugged environments, a real-time speed planning method based on the time-frequency transformation of terrain characteristics is designed to achieve adaptive speed planning of autonomous vehicles in rough ground. On the one hand, the vertical profile of the lidar's point cloud data is converted from the time domain t...
-
This study proposes a realtime motion planning framework that leverages the prediction of a portion of the optimal trajectory for sampling-based anytime planning algorithms. Existing algorithms predict the entire optimal path and bias random samples toward it for fast path planning. However, these algorithms may not be suitable for realtime frameworks because the bias-sampling strategy should cons...
-
Humanoids operate in repeated contact and non-contact with their environment and so the motion of humanoids such as walking on uneven terrain or in a narrow space requires the accurate force and position control. Joint torque control systems are suitable for position and force control, but are prone to friction and other modeling errors. To solve this problem, methods have been proposed to realize...
-
For a class of biped robots with impulsive dynamics and a non-empty set of passive gaits (unactuated, periodic motions of the biped model), we present a method for computing continuous families of locally optimal gaits with respect to a class of commonly used energetic cost functions (e.g., the integral of torque-squared). We compute these families using only the passive gaits of the biped, which ...
-
A new control paradigm using angular momentum and foot placement as state variables in the linear inverted pendulum model has expanded the realm of possibilities for the control of bipedal robots. This new paradigm, known as the ALIP model, has shown effectiveness in cases where a robot's center of mass height can be assumed to be constant or near constant as well as in cases where there are no no...
-
In real-world settings, bipedal robots must avoid collisions with people and their environment. Further, a biped can choose between modes of avoidance: (1) adjust its pose while standing or (2) step to gain maneuverability. We present a real-time motion planner and multibody control framework for dynamic bipedal robots that avoids multiple moving obstacles and automatically switches between standi...
-
This paper presents an online framework for synthesizing agile locomotion for bipedal robots that adapts to unknown environments, modeling errors, and external disturbances. To this end, we leverage step-to-step (S2S) dynamics which has proven effective in realizing dynamic walking on underactuated robots-assuming known dynamics and environments. This paper considers the case of uncertain models a...
-
This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentu...
-
Humanoid robots are expected to navigate in changing environments and perform a variety of tasks. Frequently, these tasks require the robot to make decisions online regarding the speed and precision of following a reference path. For example, a robot may want to decide to temporarily deviate from its path to overtake a slowly moving obstacle that shares the same path and is ahead. In this case, pa...
-
Much of the recent work developing formal methods techniques to specify or learn the behavior of autonomous systems is predicated on a belief that formal specifications are interpretable and useful for humans when checking systems. Though frequently asserted, this assumption is rarely tested. We performed a human experiment $(\mathbf{N}=62)$ with a mix of people who were and were not familiar with...
-
Signal Temporal Logic (STL) is a rigorous specification language that allows one to express various spatio-temporal requirements and preferences. Its semantics (called robustness) allows quantifying to what extent are the STL specifications met. In this work, we focus on enabling STL constraints and preferences in the Real-Time Rapidly Exploring Random Tree (RT-RRT*) motion planning algorithm in a...
-
A useful capability is that of classifying some agent's behavior using data from a sequence, or trace, of sensor measurements. The sensor selection problem involves choosing a subset of available sensors to ensure that, when generated, observation traces will contain enough information to determine whether the agent's activities match some pattern. In generalizing prior work, this paper studies a ...
-
Beyond programming robots to accomplish a single high-level task at a time, people also hope robots follow instructions and complete a series of tasks while meeting their requirements. This paper presents an interactive software system that consists of a multiple-task linear temporal logic (LTL) path planner and a human-machine interface (HMI). The HMI transforms human oral instructions into task ...
-
We propose a framework that uses temporal logic specifications to predict and monitor the intent of a robotic agent through passive observations of its actions over time. Our approach uses a set of possible hypothesized intents specified as Büchi automata, obtained from translating temporal logic formulae. Based on observing the actions of the robot, we update the probabilities of each hypothesis ...
-
This paper proposes two metrics for evaluating learned object detection models: the proposition-labeled and distance-parametrized confusion matrices. These metrics are leveraged to quantitatively analyze the system with respect to its system-level formal specifications via probabilistic model checking. In particular, we derive transition probabilities from these confusion matrices to compute the p...
-
We address a coordination problem for a team of heterogeneous and energy-limited agents to achieve cooperative tasks given as team-level spatio-temporal specifications. We assume that agents have stochastic energy dynamics and do not have identical capabilities. We define the team-level specification using Signal Temporal Logic (STL) with integral predicates, which can express tasks that can be co...
-
This work introduces efficient symbolic algorithms for quantitative reactive synthesis. We consider resource-constrained robotic manipulators that need to interact with a human to achieve a complex task expressed in linear temporal logic. Our framework generates reactive strategies that not only guarantee task completion but also seek cooperation with the human when possible. We model the interact...
-
Failure of any component in a robotic system during operation is a critical concern, and it is essential to address such incidents promptly. This work investigates a novel technique to recover from failures or changes in the configuration space while avoiding expensive re-computation or re-planning. We propose the Minimal Path Violation (MPV) concept to find the best feasible path with minimal re-...
-
We propose an automata-theoretic approach for reinforcement learning (RL) under complex spatio-temporal constraints with time windows. The problem is formulated using a Markov decision process under a bounded temporal logic constraint. Different from existing RL methods that can eventually learn optimal policies satisfying such constraints, our proposed approach enforces a desired probability of c...
-
This paper proposes a novel efficient multi-phase trajectory generation algorithm for dynamic dexterous manipulation tasks, such as throwing, catching, dynamic regrasping, and dynamic handover, which can be decomposed into multiple manipulation primitives, including sticking, rolling, approaching, separating, colliding, and grasping. Each manipulation primitive is formulate as a free-terminal opti...
-
This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifin-gered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the ...
-
In-hand manipulation (IHM) is an important ability for robotic hands. This ability refers to changing the position and orientation of a grasped object without dropping it from the hand workspace. One major challenge of IHM is to achieve a large range of manipulation (especially rotation), regardless of the shape, size, and the orientation during manipulation of the grasped object. There are two ma...
-
This paper presents a feedback-control framework for in-hand manipulation (IHM) with dexterous soft hands that enables the acquisition of manipulation skills in the real-world within minutes. We choose the deformation state of the soft hand as the control variable. To control for a desired deformation state, we use coarsley approximated Jacobians of the actuation-deformation dynamics. These Jacobi...
-
Shoe lacing (SL) is a challenging sensorimotor task in daily life and a complex engineering problem in the shoe-making industry. In this paper, we propose a system for autonomous SL. It contains a mathematical definition of the SL task and searches for the best lacing pattern corresponding to the shoe configuration and the user preferences. We propose a set of action primitives and generate plans ...
-
For over three decades, finger gaiting has remained largely a subject for theoretical inquiries. Successful execution of a sequence of finger gaits does not simply reduce to planning collision-free paths for the involved fingers. A major issue is how to move the gaiting finger without losing the finger contacts with the object, which will most likely undergo a motion as the contact forces need to ...
-
We propose a robotic manipulation method that can pivot objects on a surface using vision, wrist force and tactile sensing. We aim to control the rotation of an object around the grip point of a parallel gripper by allowing rotational slip, while maintaining a desired wrist force profile. Our approach runs an end-effector position controller and a gripper width controller concurrently in a closed ...
-
Swarm drones flying is a very attractive field of robotics research, motivated by natural bird flocking or other animal collective behaviors. In this paper, we propose and develop an open-source11https://github.com/micros-uav/CoFlyers universal platform CoFlyers for end-to-end whole-chain development from flocking-inspired models to real-drone swarm flying. In particular, CoFlyers is more user-fri...
-
Solving complex problems collectively with simple entities is a challenging task for swarm robotics. For the task of collective decision-making, robots decide based on local observations on the microscopic level to achieve consensus on the macroscopic level. We study this problem for a common benchmark of classifying distributed features in a binary dynamic environment. Our special focus is on env...
-
The learning approaches of designing a controller to guide the collective behavior of swarm robots have gained significant attention in recent years. However, the scalability of swarm robots and their inherent stochasticity complicate the control problem due to increasing complexity, unpredictability, and non-linearity. Despite considerable progress made in swarm robotics, addressing these challen...
-
Relying only on behaviors that emerge from simple responsive controllers; swarms of robots have been shown capable of autonomously aggregate themselves or objects into clusters without any form of communication. We push these controllers to the limit, requiring robots to sort themselves or objects into different clusters. Based on a responsive controller that maps the current reading of a line-of-...
-
This article introduces a bio-inspired 3D flocking algorithm for a drone swarm, built upon a previously established 2D model, which has proven to be effective in promoting stability, alignment, and distance variation between agents within large groups of agents. The study highlights how the incorporation of a vertical interaction between agents and the acquisition by each agent of a minimal amount...
-
Recent studies show that some security features that blockchains grant to decentralized networks on the internet can be ported to swarm robotics. Although the integration of blockchain technology and swarm robotics shows great promise, thus far, research has been limited to proof-of-concept scenarios where the blockchain-based mechanisms are tailored to a particular swarm task and operating enviro...
Introduction
Conference IROS2023 accepted paper complete List. Top ranking conferences for AI and Robotics communities. Total Accepted Paper Count 930