United States

Nowadays, the advancement of the mobile robot is also factored in the development of a world surrounded by technologies. Reinforcement Learning is used to enable the system to operate automatically, thus the robot will learn the next movement based on the interaction between the agent and the environment. The main contribution of this paper is to implement Soft Actor-Critic (SAC) to control mobile robot navigation in a 2D simulation. To evaluate the system developed using a 2D simulation with a trained SAC agent. According to this simulation, the process shows that SAC is successful to navigate the mobile robot from an initial to the target location and also can avoid obstacles in the environment.

RoboCup 2021

Soft Actor Critic Reinforcement Learning for Autonomous Mobile Robot Navigation

soft actor critic (sac)

autonomous mobile robot navigation

reinforcement learning

**General Chair Welcome**

![](https://assets.underline.io/markdown_image/1/image/d6e6dddc08bf2371b9136ea9c18dc5ab.png)              

Welcome to ICALEO® 2023!
	
We were excited to host the world’s leading international conference on Lasers and Electro-Optics in Chicago. The Laser has been shown to have the potential to revolutionize solutions for major global challenges like energy usage and supply, global warming, and CO2 footprint. Even the fusion energy breakthrough is based on laser technology. The conference, with peer-reviewed scientific and technical presentations, focused on the newest results in Micro Applications, Macro Applications, Additive Manufacturing, Beam Shaping and Frontiers in Laser Applications. Additionally, the Battery-Manufacturing Track gave an overview of the process combined with the newest advances in laser application results. 

The Business Session brought the business aspect and environment of the laser to the audience and the possibility to discuss with seasoned industry leaders from around the world. 

Whether this is the first time you are hearing about ICALEO or you are a longtime attendee, you will find something of interest, new ideas, and an excellent opportunity to network with other laser specialists worldwide. 

**Klaus Löffler **  
ICALEO General Chair   
Managing Director, Precitec GmbH

**What is ICALEO?**

ICALEO® brings together the brightest minds, seasoned professionals, and pioneering researchers from around the world. With a laser-focused agenda, ICALEO offers an exceptional platform to delve into the cutting-edge advancements, diverse applications, and transformative research that define lasers and electro-optics.

Participants will be immersed in a dynamic environment that fosters collaborative learning, networking, and idea exchange. Through engaging plenary presentations, technical sessions, an industry business panel, and networking events, attendees gain insights into the forefront of laser technology. From industrial applications to medical breakthroughs, additive manufacturing to materials processing, ICALEO covers the entire spectrum of laser-related disciplines. Moreover, the event serves as a launching pad for synergistic partnerships, paving the way for professionals to forge connections that shape the future of our industry.

Join us in Chicago and be part of an unparalleled gathering that challenges the boundaries of possibility within lasers and electro-optics. As we convene to share knowledge, explore innovations, and cultivate collaborations, ICALEO stands as a pivotal force driving the evolution of this transformative technology.

**Who attended?**

ICALEO is tailor-made for professionals, researchers, academics, and enthusiasts who are deeply passionate about the world of lasers and electro-optics. If you're an industry trailblazer, an academic luminary, an aspiring expert, or a postgraduate student looking to dive into the latest developments and trends, this is an event you won't want to miss.

Industrial leaders seeking to stay ahead of the curve in their fields, engineers working on laser applications, and manufacturing experts interested in additive manufacturing and materials processing will find ICALEO to be an invaluable platform. Researchers and scientists exploring the forefront of laser technology, from fundamental principles to groundbreaking applications, will also discover a wealth of knowledge and networking opportunities.

Medical professionals investigating laser-based medical advancements, entrepreneurs seeking to innovate with lasers, educators aiming to stay updated with the latest in laser education, and policy makers shaping the industry's landscape are all essential attendees. Whether you're an established expert or a rising star, the 42nd annual ICALEO promises to offer a diverse ecosystem of expertise, connections, and knowledge sharing that caters to the laser and electro-optics communities.

**Fields Represented:**

* Entrepreneurs and Innovators
* Manufacturing Experts
* Students and Graduates
* Policy Makers and Regulators
* Consultants and Advisors
* Technology Enthusiasts
* Suppliers and Manufacturers
* Startups and Innovators
* Researchers in Laser Education

**[View All Proceedings Here](https://icaleo.conferencespot.org/event-data)**

### **Program Tracks & Chairs**

To access the site please register [**here**](https://icaleo.org/attend). 

Please register for the ICALEO 2023 Conference to join the event.

ICALEO 2023

ICALEO® brings together the brightest minds, seasoned professionals, and pioneering researchers from around the world. With a laser-focused agenda, ICALEO offers an exceptional platform to delve into the cutting-edge advancements, diverse applications, and transformative research that define lasers and electro-optics.

poster

[![](https://assets.underline.io/uploads/markdown_image/1/image/21d13e03491d7e9acf6b8c46c7979c25.png)](https://docs.google.com/spreadsheets/d/1SCDDfkQk7es4J6RULKHVx1RzbB0Mutla--Sle5SVfb0/edit?usp=sharing)

**RoboCup 2021 will be Worldwide!**

We are excited to announce that RoboCup 2021 is a fully remote event with RoboCup competitions and activities taking place all over the world. Everyone is invited to participate in, or observe, the [competition leagues](https://2021.robocup.org/leagues) and the [symposium](https://2021.robocup.org/symposium)!

**RoboCup Symposium**

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

**RoboCup Introduction Video**
<iframe width="995" height="516" src="https://www.youtube.com/embed/Cnh43cCsDKU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>


Registration is required for entrance into RoboCup 2021. <br><br>
**Fees:** 
* Major Faculty or Symposium Faculty - US$50
* Major Student, Junior Mentor, Junior Student, or Symposium Student - US$20 

***Members of qualified teams:** Please contact your league for your registration password and proceed to register with your team. *<br>
***All non-team members:** Please register as "Symposium Only".*

You need to be registered to explore this event!

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

Human support robots (mobile robots able to perform useful domestic manipulative tasks) might be better accepted by people if they can communicate in ways they naturally understand: e.g. speech, but also facial expressions, postures, among others. Subtle (unconscious) mirroring of nonverbal cues during conversations promotes rapport building, essential for good communication. We investigate whether, as in human-human communication, the ability of a robot to mirror its user's head movements and facial expressions in real time can improve the user's experience with it. We describe the technical integration of a Toyota Human Support Robot (HSR) with a facially expressive 3D embodied conversational agent (ECA) (named ECA-HSR). The HSR and the ECA are aware of the user's head movements and facial emotions, and can mirror them, in real time. We then discuss a user study we designed in which participants interacted with ECA-HSR in a simple social dialog task with three conditions: mirroring of user's head movements, mirroring of user's facial emotions, and mirroring of both user's head movements and facial emotions. Our results suggest that interacting with an ECA-HSR that mirrors both the user's head movements and the facial expressions is preferred over the other conditions. Among other insights, the study revealed that the accuracy of open source, real-time recognition of facial expressions of emotion needs improvement for the best user's acceptance.

Towards building rapport with a Human Support Robot

Fan input and support is an important component in many individual and team sports, ranging from athletics to basketball. Audience interaction provides a consistent impact on the athletes’ performance. The analysis of the crowd noise can provide a global indication on the ongoing game situation, less conditioned by subjective factors that can influence a single fan. In this work, we exploit the collective intelligence of the audience of a robot soccer match to improve the performance of the robot players. In particular, audio features extracted from the crowd noise are used in a Reinforcement Learning process to possibly modify the game strategy. The effectiveness of the proposed approach is demonstrated by experiments on registered crowd noise samples from several past RoboCup SPL matches.

Learning from the Crowd: Improving the Decision Making Process in Robot Soccer using the Audience Noise

A stable and fast walk for humanoid robots is a challenging problem. In particular for multiple robots with different degrees of wear, it is necessary to tune several control parameters, which consumes a lot of time and wears down the hardware even more. In this paper, we present the latest improvements of our current walk, which enable a significant boost in the walking speed and stability, basically independent of the robot's hardware state. Based on a linear inverted pendulum model and thresholds determined statistically, the feet positions are adapted to follow the center of mass' direction of movement and let the robot walk in the direction of a potential fall if necessary. In addition, we ensure that the movement of the swing foot is parallel to the ground with a closed-loop feedback controller over the measured rotation errors of the support foot, without the direct use of the inertial measurement unit. The approaches presented allow for a up to 40% faster walk on different robots of the same type, without the need for a manual calibration.

Step Adjustment for a Robust Humanoid Walk

Detecting the field boundary is often one of the first steps in the vision pipeline of soccer robots. Conventional methods make use of a (possibly adaptive) green classifier, selection of boundary points and possibly model fitting. We present an approach to predict the coordinates of the field boundary column-wise in the image using a convolutional neural network. This is combined with a method to let the network predict the uncertainty of its output, which allows to fit a line model in which columns are weighted according to the network's confidence. Experiments show that the resulting models are accurate enough in different lighting conditions as well as real-time capable.

Soccer Field Boundary Detection Using Convolutional Neural Networks

A long term goal in robotics is lifelong learning, in which a robot learns to perform new tasks without human intervention. For this a promising alternative is reinforcement learning (RL), in which an agent learns how to solve sequential decision-making problems by interacting with its environment. Despite the effectiveness of RL, learning in a trial-and-error way can be infeasible in robotics applications, in which long training periods can significantly deteriorate hardware. For robots that face long sequences of tasks, transfer learning represents an alternative to reduce training time, by transferring knowledge from previously learned tasks. However, in domains such as service robotics, identifying reusable pieces of knowledge and transferring them across tasks can be challenging, due to the mismatch between the state-action spaces (heterogeneous tasks). Thus, in this article we present an inter-task similarity measure and a transfer learning method for heterogeneous tasks. Through experimental evaluations, we show that the proposed measure is able to rank tasks in a way that prioritizes those that cause a larger increment of the learning performance in a target task. Additionally, results show that for some tasks the similarity measure can be computed with few data, enabling significant speedups in the learning process by transferring knowledge at an early stage.

Inter-Task Similarity Measure for Heterogeneous Tasks

The task of 6D object pose estimation from RGB images is an important requirement for autonomous service robots to be able to interact with the real world.
In this work, we present a two-step pipeline for estimating the 6 DoF translation and orientation of known objects. Keypoints and Part Affinity Fields (PAFs) are predicted from the input image adopting the OpenPose CNN architecture from human pose estimation.
Object poses are then calculated from 2D-3D correspondences between detected and model keypoints via the PnP-RANSAC algorithm.
The proposed approach is evaluated on the YCB-Video dataset and achieves accuracy on par with recent methods from the literature.
Using PAFs to assemble detected keypoints into object instances proves advantageous over only using heatmaps. Models trained to predict keypoints of a single object class perform significantly better than models trained for several classes.

6D Object Pose Estimation using Keypoints and Part Affinity Fields

Pose estimation commonly refers to computer vision methods that recognize people's body postures in images or videos. With recent advancements in deep learning, we now have compelling models to tackle the problem in real-time. Since these models are usually designed for human images, one needs to adapt existing models to work on other creatures, including robots. This paper examines different state-of-the-art pose estimation models and proposes a lightweight model that can work in real-time on humanoid robots in the RoboCup Humanoid League environment. Additionally, we present a novel dataset called the HumanoidRobotPose dataset. The results of this work have the potential to enable many advanced behaviors for soccer-playing robots.

Real-time Pose Estimation from Images for Multiple Humanoid Robots

The design and implementation of behaviors for robots operating in dynamic and complex environments are becoming mandatory in nowadays applications. Reinforcement learning is consistently showing remarkable results in learning effective action policies and in achieving super-human performance in various tasks -- without exploiting prior knowledge. However, in robotics, the use of purely learning-based techniques is still subject to strong limitations. Foremost, sample efficiency. Such techniques, in fact, are known to require large training datasets, and long training sessions, in order to develop effective action policies. Hence in this paper, to alleviate such constraint, and to allow learning in such robotic scenarios, we introduce SErP (Sample Efficient robot Policies), an iterative algorithm to improve the sample-efficiency of learning algorithms. SErP exploits a sub-optimal planner (here implemented with a monitor-replanning algorithm) to lead the exploration of the learning agent through its initial iterations. Intuitively, SErP exploits the planner as an expert in order to enable focused exploration and to avoid portions of the search space that are not effective to solve the task of the robot. Finally, to confirm our insights and to show the improvements that SErP carries with, we report the results obtained in two different robotic scenarios: (1) a cartpole scenario and (2) a soccer-robots scenario within the RoboCup@Soccer SPL environment.


Improving Sample Efficiency in Behavior Learning by Using Sub-optimal Planners for Robots

This proposal addresses the creation of educational tools to help
students with the concept of ‘localization’ of a mobile robot using
the extended Kalman filter (EKF) and landmarks in the
environment.

Learning tools for mobile robots localization using visual landmarks and the extended Kalman filter

Multi-robot coordination is one of the fields where RoboCup has consistently provided conditions that promote state-of-the-art advances. Still, it is common to see many teams achieving autonomy by applying hand-crafted rule-based plans. Formal approaches have produced optimistic results when addressing this issue but currently lack available software tools. In this project, we developed a MATLAB® toolbox that provides, through a user-friendly API, methods to model multi-robot coordination problems as generalized stochastic Petri nets with rewards (GSPNRs). This formalism provides a compact way of capturing: action selection, uncertainty on the duration of action execution, and team goals. This toolbox also implements algorithms that allow extracting optimal policies for the GSPNR model, while reasoning over uncertainty and optimizing team-level objectives. Furthermore, it integrates with ROS middleware enabling the execution of real multi-robot systems. We strongly believe that this could significantly benefit the RoboCup initiative as it facilitates the use of state-of-the-art formal methods that enable the improvement of autonomous behaviors in aspects such as efficiency, robustness, reliability, safety, and predictability.

Premium content

Downloads

Next from RoboCup 2021

Towards building rapport with a Human Support Robot

Similar lecture

An Approach Towards the Application of Mobile Robots in Laser Materials Processing