Projects:2021s2-63311 Robots playing capture the flag
Contents
Abstract
Robots have been proven useful in many aspects of our lives. From easy tasks such as the auto-cleaning bot in homes, to more precise and complex tasks such as surgical robotic arm in hospitals, or systems of multi drone swarms used in defence. Most robotic system are known for their excellence in handling tasks that require a high level of responsiveness, repetition, and accurate execution. However, the more complicated task, the more inclination for robots to suffer functionally, especially with tasks that involves critical planning and making strategies in advance. To research such tactical tasks, the game capturing the flag is chosen. Capturing the flag is a team-based game that requires the player to capture their opponent’s flag while protecting their own. The team with better strategy and formation will be more likely to achieve victory. With that in mind, this project aims to simulate the game CTF in a software environment to study the ways in which complex and strategic behaviours can be implemented into robots using Python programming. Hopefully, it may contribute to future development of complex tasks integration for other strategical, tactical robots.
Acknowledgements
The author wishes to acknowledge the sponsorship of the Defence Science and Technology Group of Australia (DST Group). The credit for great support, helpful feedback and on-going consultation goes to David Hubczenko, Jijoong Kim, Anna Dostovalova and Bradley Fraser. Moreover, the author would also like to express his deepest gratitude towards invaluable guidance and support from project advisors Prof. Cheng-Chew Lim and Dr. Duong D. Nguyen. This project would not have reached this far without their expert recommendations.
Introduction
Motivation
The game Capturing the Flag (CTF) is a very popular game. It consists of two teams; each normally consists of five players. Each team will be required to capture their opponent’s player flag, and run back to the base without getting caught, while defending their own flag. The level of complexity of the game is mild, which suits the purpose of the project. The CTF game requires the players to have winning characteristics such as good teamwork, communication, planning strategy, tactical execution, and situational response. These characteristics are also traits that is needed to study for later implementation into robots. There has been work done in the past with similar objectives that was carried out before, that can be an example for our project to advance. The other inspiration comes from the successful implementation of AlphaGo [1, 2], the Artificial Intelligence (AI) that has been developed by Google Deep Mind. It achieved many victories (4/5 games won) against the 18th time world champion Lee Sedol in the game of Go in 2016. This project’s ambition is to write a self-learning, or at least self-guiding algorithm so that the robot players can play with themselves in the scope of the game CTF. Moreover, if time constraints are reasonable, it would be stimulating to implement real world robots that can play the CTF game, using the simulation built before. However, since to the level of high ambition implementation exceeds the ability of our group members, it would be reasonable to remain the project’s scope of making simulation of robots playing the game on computers only.
Project Aims
The deliverable of this project is a simulation program, which visualize robots playing the game together, with or without control from human observer. The robots were programmed so that they must obey fundamental physics rules, such as velocity, acceleration. Next, it is essential to define attacking, and defending algorithms so that they can be implemented into the program. The execution of algorithm will need to depend on the game situation. Additionally, the virtual environment in which the game takes place must also be considered. In the world of robotic simulation, basic physics constraints apply to the robot’s movements. For instance, the parameters with maximum velocity, acceleration and stopping acceleration will be considered. Since this report focus more on the art of how the game played, other physics elements, such as gravity, friction, robot’s inertia, mass, are neglected. Nevertheless, these elements can always be implemented in the future when it’s required. The focus of this project is algorithms building, which will be covered in Chapter 3 of this report. The algorithms will take in the input of other robot’s position, and situational information (such as user’s input instructions, or the state whether the flag is captured, and so on). After processing provided input, the robots will act accordingly. If an algorithm is implemented successfully, it is still required for implemented algorithms to be tested for their functionality, stability, and reliability. The result to each algorithm performance will be further discussed in Chapter 4. The following step would be algorithm analysis, and directions for further implementations in the future, if the project is to be continued.
Prior Arts
This game challenges the need to build a smart algorithm. In this case, it’s the tactical planning and execution for robots to play capture the flag. The goal is to integration these algorithms into physical robots that play capture the flag, but also where they can be used in a variety of other/different applications. The essence of this project is the strategic algorithm, which will determine how the robots act according to real-time events.
Aquaticus
One of the closest research projects that have done similar research on the same objective is Aquaticus [3]. The project tested the game of CTF with real human and robots on the water surface. The coordinates and movement of the game is recorded and made available on Aquaticus website. However, data is limited, as there had not been many game set to analyse. Meanwhile, a set of large number of simulations is needed to study the behaviour of the robotic players in the game.
Hamilton Jacobi reachability
Another group of researchers have been trying to implement different approaches to game planning on the game of CTF. The research had been successful at that time for the implementation of Hamilton-Jacobi reachability into one of the scenarios in the game CTF. The advantage of this research is to apply Hamilton Jacobi reachability to calculate the possibilities to successfully offend in the game of CTF, depending on different scenarios and achieve victory. However, the idea was abstract, on the paper research without actually simulate a full course game of CTF.
Results
Results of simulation carried out by 20th September 2021: Firstly games such as space invader, space shooter and pong were created so that the project team can be familiar with the simulation environment (python, pycharm & pygame). Then the game pong was used as a platform to create a player vs computer capture the flag game, where the player has to get past robots moving up and down in order to capture the flag. Next, for the ease of gameplay, a code was added to connect a controller to python, and a simple game of moving a box around the screen was created to show the success of this code. This code was then extended to a player vs robot capture the flag game, where the player is controlling a defender, while the robot has an algorithm which calculates and moves in the most direct path to the flag (flag appears randomly in the map on the defenders side).
Conclusion and Future work
Conclusion
. .
Future work
. .
References
NGUYEN, DUONG D., et al. "Dynamic Multi-Target Assignment with Decentralised Online Learning to Achieve Multiple Synchronised Goals." 2020 International Conference on Machine Learning and Cybernetics (ICMLC). IEEE, 2020.
Simpson, T., Hull, R., Dadgar, A., Rajagopalan, A., Kim, J.,&Chen, L. (2019). On-board/off-board smooth trajectory control for quadcopters through multi-loop PID tuning. Australasian Conference on Robotics and Automation, ACRA, 2019-December, [159894].
Hongjun Yu, Peng Shi, Cheng-Chew Lim, Dongzhe Wang. (2019) Formation control for multi-robot systems with collision avoidance. International Journal of Control 92:10, pages 2223-2234.
Yutong Liu, Hongjun Yu, Cheng-Chew Lim&Peng Shi (2017) Steady formation analysis on multi-robot systems, Journal of Control and Decision, 4:1, 12-31, DOI: 10.1080/23307706.2016.1254073
https://www.roboticoncology.com/history-of-robotic-surgery/
https://online.officetimeline.com/app/#/file/2fd73c1d-0f25-439a-87f6-d138e7b8cdf0
Project Team
Student researchers
Kevin Perera
Vu Quang Truong
Supervisors
Dr. Cheng-Chew Lim
Dr. Duong Duc Nguyen
David Hubczenko (DTSG)
