Projects:2015s1-17 Analysis of Electrical and Software Design in the Effectiveness of Robotics STEM Outreach Programs

Note: Due to the participation of Department of Education and Child Development (DECD) schools in this project, the names of any robotics kits cannot be mentioned; as such all kits will be referred to by a letter, e.g. Kit A.

The aim of this project was to analyse a number of different robotics kits, to determine which is the most effective as a learning tool for students and to assess what kit features influenced the effectiveness of each robotics kit.

Project Information

To test each robotics kit a series of workshops were run in schools with eighth grade students, having the students attempt to complete ten tasks with a robotics kit in 60 minutes. We determined that we required approximately 80 students to test each different kit to achieve a 90% confidence interval. The tasks were designed by the project team having researched lesson plans and robotics kits tasks currently available. The ten tasks were aimed to test the basic functions of the kits utilising key features such as motor movement, the use of sounds and obstacle and line detection; with the tasks to become progressively more difficult as the students progress. The time taken for the students to complete each task was recorded as well as the total number of tasks the students could complete in the 60 minutes. Once the students had finished the workshop with the robotics kits, they were asked to complete a survey we designed to gather their opinions on the robotics kit.

Project Team

Supervisors:

- Dr Braden Phillips

- Dr Hong Gunn Chew

Researchers:

- James Cadzow

- Benjamin Hayton

Project Background

In 2005-2006 and 2011-2012, Australia experienced engineering skill shortages that were described as ‘persistent and widespread across engineering specialisations’ [3]. This, combined with a decline in student interest in Science, Technology, Engineering and Mathematics (STEM) fields [2], has left Australia needing ‘long-term growth in the number of secondary students studying mathematics and science’ [4]. STEM outreach programs are a way to get school students interested in STEM fields and potentially having future careers in STEM. The robotics industry is seen by some economic forecasters as becoming the next big boom industry [1]; as such STEM outreach programs have used the availability of robotics kits to run programs with an aim of getting more children interested and involved in STEM fields. With robotics having ‘been shown to be a superb tool for hands-on learning’ [2], outreach programs such as Robogals are using robotics kits [5], to help students become more interested and involved in STEM fields in a hands-on manner. With backing from Engineers Australia, the Governments Chief Scientist and with international interest in the field [6], a large number of robotics kits have become available for use in STEM teaching. Despite this, no research has been made publicly available to aid teachers and STEM outreach programs in choosing the most effective robotics kit for their needs. This project attempted to fill this void that exists in the STEM education industry.

Project Method

Before we could explore the technical aspects of this project we first defined what effectiveness, as a learning tool would refer to for this project. We established that a robotics kit would be considered an effective learning tool if it met the following criteria:

• The kit and software challenged the students to think and problem solve when attempting tasks without being too easy or too difficult

• The kit would be affordable, relative to its level of performance. For example, if there were two kits, one that was cheap and one that was expensive, if the students were found to perform tasks with the cheaper kit at a similar but slightly decreased level to the expensive kit, then the cheaper kit would be considered more effective as more students are likely to be able to have access to them. Alternatively, if the more expensive kit clearly out-performs the cheaper kit, then it is obviously more effective even though it costs more.

• Finally, the students had to find the kits engaging and interesting; most importantly the students had to have fun with the robotics kit and want to use them.

Kit Selection

The kits we selected for use in our project had to be similar so that we could develop standardised experiments and tasks for the students to put the kits through. However, the kits also had to have a diverse range of features so that we could draw conclusions on if these features impacted the overall kit effectiveness. To select the three kits we compiled an initial list of 45 different robotics kits, by using terms such as ‘robotics kits’, ‘educational robotics kits’ and ‘programmable robotics kits’ in internet search engines. To further refine the list, we defined the following criteria the kits had to meet to be considered suitable for our project:

• Be Autonomous – i.e. not remote controlled

• Require Programming – the kit needed to require the students to create a program with a form of programming language to complete the different tasks. A number of kits in the initial list used options such as barcode scanning, to program the kit to perform different actions, without the user needing to actually create and develop the program.

• Not Require a Smart Device – a number of the kits required a smart device such as an iPad tablet or smart phone to use. The kits required the smart devices for both app-based programming and remote control, or were the actual computer for the kit.

• Meet Kit Intangibles - this included having to be a land-based kit with wheels or tracks; excluding kits such as mechanical arms, walker kits and drones or water devices. The kits also had to meet the appropriate skill range, i.e. they could not be too basic or too complex, such as being intended for use by children aged 5-10 years or requiring previous programming experience.

We were fortunate that a local STEM outreach program, Robogals, were generous enough to provide us free access to the ten-robotics kits they use for their programs (Kit N). These kits have a recommended retail price of around $500 and met all of the above criteria. To select the remaining two kits to use, we also had to factor in our project budget constraints. We were provided a budget of $1000 and as such used a final kit criterion of a maximum purchase price of $150 per kit. Implementing these criteria, we filtered down the list to a final 12 kits that seem suitable for our project. Over half of the kits were deemed unsuitable due to being too expensive for our project budget, with a further 19% requiring the use of a smart device. 9% of the initial kits were found to be remote controlled and the remaining 19% were excluded due to various kit intangibles. The final three selected kits were Kit N, available to us through Robogals, Kit E and Kit F, seen in the figures below. These kits were selected as they had various similarities that enabled us to create standardised testing methods, yet had various unique features and differences so that we would be able to draw conclusions on the impact of these features on the overall kit effectiveness.

Survey

The students were asked to complete a survey we developed after their 60-minute workshop with the robotics kit. The survey was compiled of two types of questions- open questions, where the participant responds using their own words- and closed questions, where the participant chooses from a list of predetermined responses [18]. Our survey had a majority of closed questions, as responses to closed questions provide data that can be statistically analysed that helped us in determining any trends that were present in student responses between the different kits. The closed questions were aimed at determining the student’s prior experience with robotics kits and to find out their opinions on their experience with the robotics kit they used. These questions were aimed at determining what aspects of the robotics kits they found particularly hard or easy and if the robotics kit they used had any issues or areas for improvement. The open questions we had in our survey were to potentially provide responses that we had not anticipated [20]. These questions were aimed at determining what aspects of the robotics kits they found particularly hard or easy and if the robotics kit they used had any issues or ideas for improvement.

The ratio of open to closed questions was aimed at making the survey quick and simple for the students to complete and upon request by the University of Adelaide’s Faculty of Engineering, Computer and Mathematic Sciences (ECMS), a further question was included in the survey asking for the students’ gender, so that we could assess if there were any differences between task performances and interest levels for male and female students.

References

[1] P. Turner.(2009).“Why Use Robots in Education...” Educational Technology Solutions. [On-line]. Available at: http://www.tribotix.com/EducationInfo/WhyRobotics.htm [9/4/2015]

[2] M.J. Mataric, N. Koenig, D Feil-Seifer. (2007). “Materials for Enabling Hands-On Robotics and STEM Education” [On-Line] Available at: http://robotics.usc.edu/publications/media/uploads/pubs/536.pdf [8/5/2015]

[3] Department of Employment, Labour Market Research and Analysis branch. (2014,Aug.).“Labour Market Research - Engineering Professions", [On-line]. Available at: https://docs.employment.gov.au/system/files/doc/other/engineeringprofessionalswa.pdf [14/4/2015]

[4] K. Condon. (2012, Aug.) “New study shows depth of engineering skills shortage”, Engineers Australia Media Release”[On-line]. Available at: https://www.engineersaustralia.org.au/sites/default/files/shado/News%20and%20Media/Media%20Statements/2012MediaStatements/new_study_shows_depth_of_engineering_skills_shortage.pdf [14/4/2015]

[5] Robogals School Visits, Robogals Website. Available at: http://adelaide.robogals.org.au/activities/school-visits [ 14/10/2015]

[6] Australian Government Chief Scientist. Science, Technology, Engineering and Mathematics: Australia's Future, 2014. pp. 8-10, 20-22. [14/4/2015]

[18] N. Thayer-Hart, J. Dykema, N.C. Schaeffer, J. Stevenson. (2010, Dec.) “Survey Fundamentals”, Office of Quality Improvements, University of Wisconsin-Madison, USA.

[20] A. Fink. (2010). “Survey Research Methods”. Education Research Methodology: Quantitative Methods and Research. University of California, USA. pp.152-160