Projects:2016s2-235 Personal Radar for Safer Walk & Text

Project team

Team member

Zou Chenlu

Zhu Yifei

Supervisor

Dr Withawat Withayachumnankul

Dr Brian Ng

Project introduction

Many of us have become highly reliant on the internet, whether for web surfing, social networking, watching video, etc. For some, it is to the point that they need to stay connected as soon as they open their eyes in the morning, and it lasts till they do the opposite at night. It is known that walking and texting on the phone is potentially harmful, with real risks of tripping over or hitting an obstacle. This behaviour is also annoying to other fellow pedestrians. We propose to come up with a smart solution, based on sound engineering, to alleviate such issues. In this project, you will build an ultrasonic echolocation device (similar idea to a radar) that is readily attached to your mobile phone. The device will periodically probe the walking path, and sending a warning signal to your mobile phone if it detects imminent obstacles, be it an object or a pedestrian. You will learn the skills in mobile programming and electronic circuitry to interface between the ultrasonic modules and the mobile phone.

Motivation & Significance

With the internet developed rapidly, the number of internet users is raised dramatically. People are drawn to the internet nowadays. Depending on the study of Lennon and Williamson, up to 40% of observed pedestrians are distracted by other things such as mobile phone. When crossing the road [1]. This research shows that if pedestrians focus on the mobile phone. The risk of crashing may increase [1]. It is well known that concentrate on mobile phones when pedestrians walking on the road are potentially harmful. The accident may occur because of this unsafety act. Also, this act may be annoying other pedestrians around the user. According to the study of Henderson, the number of pedestrians injured when they were talking or texting is increasing every year [2]. From the studies above, we notice that it is extremely urgent to use all group members’ knowledge to find a smart solution to decrease the number of people get injured when they talking or texting with a mobile phone. We are intended to design and build a device to help pedestrians who use the mobile phone frequently, and decrease the injured chance when pedestrians texting or talking on mobile phones.

Technical background

Ultrasonic sensor works as rada

Principle of ultrasonic sensor

The basic principle of the ultrasonic sensor is the reflected sound, or “echo”, the transducer ultrasonic sensor will send sound waves, when the waves reach the obstacle, the return of the waves can be detected by the receiver of the ultrasonic sensor, the ultrasonic sensor will analyse the time delay. And users can use the formula for the speed of sound to calculate the distance between the ultrasonic sensor and the obstacle [4].

The advantages of ultrasonic sensor

The accuracy of ultrasonic sensor cannot be affected by the color, sharp, size or other exterior condition of obstacles [4]. The ultrasonic sensor has large measuring angle which can detect obstacles multi-direction, instead only detect obstacles in front of the ultrasonic sensor. Compare to other sensors such as an infrared sensor. The ultrasonic sensor is influenced less by the weather condition, especially rains. Even transparent objects also can be detected. The group chose ultrasonic sensor HC-SR04, the maximum range that HC-SR04 can detect is 4 meters and the measuring angle is 15 degrees, after calculation, the group found the area that the sensor can detect is a circle with a 1-meter diameter. Those numbers match our project desired value. Also, the size of HC-SR04 is compact and exquisite, those features are matching with our desired equipment. The other advantage of HC-SR04 ultrasonic sensor is that unlike other sensors the result original data is current(A) or other value, and those values need to be translated to get the desired value. The original data of HC-SR04 is the time between the sending wave and the return wave. It helps group to simplify the original data.

The calculation about the measuring angle and distance

As for measuring distance, according to the study, the reaction time of human touch stimulus is 0.15 seconds, to ensure that users have enough time to judge and avoid the obstacles in front of users. The group increased the reaction time to 0.3 seconds, and add another additional 0.1 seconds for users to avoiding the obstacles. So the total time for users to detecting and avoiding is 0.4 seconds, also according to another study of Queensland university, the normal walking speed of human is 1.34 meters per second, and when people reading and texting, the walking speed of human decreased to 0.8 meters per second, so we can image from the user detected the obstacle, until user avoid the obstacle, the total time cost is 0.4 second and the total walking distance of user is 0.24 meters (because the 0.1 seconds is for avoiding the obstacle, user cannot move forward). During this time, the group also image another obstacle moving forward to user with speed of 30 kilometers per hour (8.33 meter per second), within 0.4 second, the obstacle moved 3.32 meters. That means users can avoided the 30-kilometer-per-hour obstacle successful and unharmful. According to the calculation above, the 4 meters detecting distance is enough for the project.

As for measuring angle, the group consider that the shoulder width of human is about 50 ceilometers, and in order to ensure that user may swing when they are walking, the group doubled this value, so the desired measuring width when the obstacle is four meters away from user is one meter. And using the graph above we can calculate that the desired measuring angle is 32 degrees, and according to the description of HC-SR04 ultrasonic sensor, the actually angle is 30 degrees, which is matching our desired value.

Introduction of Arduino

Arduino is an open-source electronics platform, which is based on hardware and software. Arduino also supports Bluetooth module, HC-SR04 ultrasonic sensor, and android system, the group chose Arduino Uno board as a test board. Arduino board is inexpensive compared to other microcontroller platforms. This feature decreased group budget significantly, also Arduino Uno equips 14 digital and analog pins, which support this project compatible with an ultrasonic sensor and the Bluetooth module.

Introduction of App invent

App inventor is an open-source web application for users to programming software application for Android operating system. The advantages of App inventor are that it allows newcomers to create simple software applications. The two group numbers are both electronic engineering student. They both have limited knowledge about mobile phone software design. Compared with other mobile phone development software. App inventor is easy to learn and implement. Also, consider the mobile phone application for this project is simple. And app inventor can reduce project budget, the group decide to choose app inventor as software programming language.

Introduction of Android Studio for the final type

Android Studio is the official integrated development environment for android platform development. The reason that group decide to choose android studio in the second type product is because that the limited function of app inventor, android studio not only can help group to solve the background working problem, the limitation of App inventor limited the function of project, the other reason is that android studio also help group to fix the problem that the display glitch for another android phone except Nexus 5. Although app inventor is a convenient for group member to learn, there still several problems that app inventor cannot handle, the main problem that App inventor cannot solve is that App inventor does not support working background. During the first type of product, App inventor works without problems, but during the second type of product, group members focus on the applicability of device. Group decide to change the mobile phone programming language to Android Studio.

Method

The group decides to divide the whole project into 3 stages. The first stage focuses on hardware design, and the second stage aim to design and build a mobile phone application which can display critical information such as alert and warning message for users. The final stage is for mixing the product of the first two stage, and ensures the product is functional.

Hardware stage

During this stage, the group is intended to design an ultrasonic sensor which can detect obstacles in front of users. The group decides to use a 9-volt battery as the power supply of hardware, and using the Arduino programming language, to ensure that the ultrasonic sensor can detect the obstacles. And record the analytical information of the obstacles, such as location, distance. All diagnostic information will be sent to an application designed by the group through a Bluetooth module. Depending on the study of the university of Queensland, the people reading or texting when they are walking, the speed is almost one meter per second which is slower than the normal speed. For hardware detecting part, the basic principle is that Arduino control the voltage of ultrasonic sensor to send and receive ultrasonic signal then return the delay time. Then Bluetooth will transfer the data. The connection shows below.

Software stage for the first type

The purpose of first type product

Because App inventor is much easier to learn than android studio, also because at that time, team members do not know the accessibility of the hardware, group decide to develop the first type project, which is built by Arduino hardware part and an application built by App inventor, the App inventor application helps team to measure the accuracy of the ultrasonic sensor and the time delay of the Bluetooth, but It is for test use only. After the first type, team decide to develop another application which is more practical and neat.

The principle of the Android application

During this stage, the group concentrates on analyses the critical information received from the previous stage which is the hardware stage. Frist, the Bluetooth receiver of android phone will receive the information from hardware stage. Then, the group decides to design an application which can analyses and filters the critical information, and record the useful information. Then pop up alert and warning message to the screen of users’ phone. Group members divide this stage into three parts.

Bluetooth connection part, group members use two listpicker functions to ensure that the android phone can find the Bluetooth before picking and also can connect with Bluetooth after picking. Choose the desired detecting distance is the second part, group members use a listpicker and a list function to display the options.

Alert is the other important part, group members use a clock function to achieve the alert function, in order to ensure that the alert massage always correct. Group members use a if function to ensure that the warming massage only can be sent if the Bluetooth is connected.

Software stage for the second type

In the second part, group members are not only satisfied the limited function of App inventor, and during the study time of android studio, team members also create several functions which is for testing.

The basic working process of the android studio application is similar with the app inventor. So only the most important function will be introduced.

Bluetooth detection

Different from the app inventor, in the new application, we can find the desired Bluetooth first, and set the Bluetooth address into our code, this step decreases the risk that user select the wrong Bluetooth, it also increases the performance of the application, the code shows below. The figure 10, shows the initialization of the Bluetooth address and the auto-select of the application.

Bluetooth connection

After found the desired Bluetooth, the application will ensure the status of the default Bluetooth adapter, if the Bluetooth is connected, application will run the data transfer process automatically, or if it is the first time connect the Bluetooth, the application will ask user to input the access key of the Bluetooth. In the original plan, the app inventor made application, group decide to list the all available Bluetooth device around the phone, and let user to choose which Bluetooth to connect, but after searching information and useful material, group decide to delete the ListView and connect the Bluetooth directly.

Bluetooth data transfer

This is the most important part of the project, it makes the communication between the Bluetooth and the cell phone alive. It is also the most difficult part of the project, after the research, group decide use socket to open the Bluetooth transfer connection. It is interesting that a connected Bluetooth with application is different in the data transfer process, it can be explained in the principle of cell phone: an active cell phone must connect with carrier, this is like the Bluetooth connection process. Pair with device, and wait for order, but Bluetooth data transfer connection is more like dialing number process of cell phone. It relays on the carriers’ connection, but it is unique because the data can be transferred only in this process. Socket is the dialed number in the Bluetooth data transfer process. The way Bluetooth transferring data is similar with Network data transfer. Socket is a port to monitor the desired channel, in our project, the desired channel is selected by the UUID After the initialisation of the socket, the information of the Bluetooth can be transferred into the application.

receive data from Bluetooth

This part is the essential part of the whole project, and it also need to break down into several steps, the first step is get the Thread, and ensure that input stream is available. Then receive the raw byte stream, the raw stream is untranslated stream, and it is also the Bluetooth translated stream. It looks like gibberish. So, the received part is finished, the program is going to the translate part.

Translation is the second part. Because the gibberish is built by UTF-8, so using UTF-8 to translate the received stream is required. Now the stream is readable, but in order to let the machine to read the UTF-8 code, we still need to improve the stream

First, we need to cut the stream into several parts. The update rate of the Arduino is 1000ms once, but the Bluetooth update rate is unknown, and to ensure that each time we cut, there must be a competed decimal inside, the team did a great number of research and experience, and in the end, team decide to cut the stream into every six bytes, because of 6-byte cut will ensure that the completed decimal will appear in every cut, and the normal format of the decimal is _A.BC_. _ means space, and A.BC means the decimal. There are also some other formats, such as __A.BC, no matter what format is, if a completed decimal appears in the cut, we can translate them.

Second, translate the string to the int is required, because machine only read int number, so we need to translate each cut into int, the basic principle is using equal function instead “==”. The difference between the two symbol is that “==” means absolute equal, not only the message is the same, the address, the type also need to be same, but equal only require the message be the same. After this step, now the detected, comparable int type distance is get. The only functional thing is comparing with the default alert distance.

the final part of receiving is display, in this part, the most important is the connection between the listview and the number. It is the connection between the interface and the program. And in order to make sure we can display every number on time, we need handler to post the number.

Alert distance setting

This function is use for sending the desired alert distance, and in the interface, we need edittext widget, and a button for this action. Because the comparison is inside the cell phone, so there will be no need to send the data to the Bluetooth. And we only need to add this function into the button’s action.

The alert function

After discussion about the alert function, group decide that the alert system need to apply several implementations. Although sound is a more direct way to notify user, but considered the using condition of the device, group decide not to include the sound notification.

It is inserting that alert is just another interface of the application, but this interface cannot be shown to user. Once application calls the notification function, application just pick that interface and show to the user. The two main notification ways shown below:

Head up notification

Head up notification means that the notification will pop up from the statue bar and show the alert detail. In real life, in the user need our project, that means user must focus on the phone instead of the safety condition of the street. So, the alert success rate must high.

The coding process of the head up notification is not difficult, the condition to ensure that the notification will pop up instead just show the icon is that the priority of the head up notification is max, that means the only thing we need to add is that at the bottom of the code of the normal notification, add the priority. Code shows below.

Vibration

Vibration is another way to notify user, it is more direct than the head up notification. We add vibration in case that user ignore or cannot see the head up notification. The coding process is simple too, include the managing code, only few lines need to add.

Final stage

During this stage, the group focuses on the mix the products of previous stages. And ensure the product can realize the function of detecting and warming. Then the group focus on simplifying the exterior of the products. Ensure the products is more convenient for users to use. In order to fix the size problem of the product, group decide to improve the board.

Remove pins from Arduino board.

Because the power supply group used is 9V battery. So, it is hard to put inside the Arduino board, group decide to remove 12 useless pins.

A case that contain all the components

Group also designed a case to put all the components inside, the software group used is solidwork, the designed case shows below:

Concept Methods

Test statue bar color and distance display in the statue bar

Group decide to set up another way to notify user, the approach need to be more conspicuous, and group decide to change the color of the status bar from black to orange. The code shows below

But quickly we found that it is not working, it will change the color of the statue bar, but unknown glitch will stop the program. After research lots of material, we found that Google changed the security level of the statue bar android system, and system which above android 5.0 cannot change the status bar color in in the thread function anymore. And then group member abandon this idea.

Speed test

Group member decide to calculate the speed of the approaching object, in order to give more detail for user to know what is in front of the user. The formula is simple. because the update rate of the sensor is two times per second, and the first detected distance minus the second, then divide by 0.5 will get the speed of the approaching object. But several tests prove that this function is not practical.

The first problem is that the speed of human is unknown, although from the recourse group find that the human walking seed is 0.8 meter per second average but the range is from 0.5 to 1.1 meters per second.

The second problem is that because of the delay of the Bluetooth, the calculated speed is not the actual speed, with group test , it is normal 0.5 second delay.

Test and result

Test of accuracy of ultrasonic sensor

To test the accuracy of ultrasonic sensor, a 20cm2 Perspex sheet is used as standardized obstacles. And to test as real as possible, hardware is putted between 1.3 to 1.4 meters to meet the normal height of mobile’s position when it is being used. Figures below demonstrate the set up for this test.

During the real test, to minimize the influence of the person who holds Perspex sheet, the person stands at side and holds sheet. The figure below demonstrates how the test runs.

For the test, we set several distance to test. For each distance, we recorded detected distance ten times to increase the sample rate to get more trustworthy database.

As it can be observed from the result, the ultrasonic sensor can detect distance pretty accurate between 0.02m to 3.0m. When detect distance shorter than 0.02m, it always read 0.02 or 0.03m. When detect distance longer than 3.5 meters, it has very messed up reading, sometimes it read accurate distance, however, most time, it read no-sense distance. After done using Perspex sheet, we use actual person as objective to be detected. The process is same as before just switch Perspex sheet to real person.

As it can be observed from the result, the ultrasonic sensor can detect distance pretty accurate between 0.02m to 3.5m. When detect distance shorter than 0.02m, it always read 0.02 or 0.03m. When detect distance longer than 3.5 meters, it has very messed up reading, sometimes it read accurate distance, however, most time, it read no-sense distance.

Test of angles that ultrasonic sensor covered

To test the covered angle for ultrasonic sensor, we use same process as test for accuracy. We set a real person at different point to let sensor detect. The person stays at a half circle which has 3 meters radius. A figure demonstrates how the test runs.

From The figure it is observe that ultrasonic sensor covers accurately from 75-105 degrees (red mark) in front. However, it has unstable detecting range between 70-75 and 105-110(blue mark) meaning the reading of distance jumps from accurate distance to some random numbers. Any ranger out 60-120 becomes undetectable

Test of power consumption

To test the power consumption, a multimeter is used to measure current drain. And a one ohm resistor is connected in series with hardware to let the multimeter measure current.

From the measurement, ultrasonic sensor drains 18.9 mA. Bluetooth drains 39.7 mA when pairing but only drains 20.3 when paired. Hence, it gives total current drain of 39.2mA. So this gives battery life of:

1200mAH / 39.2 mA = 30.69 hours

Alert distance based on reaction time

This is not an actual test but a theoretical calculation. So based on this, we can ignore the approaching objective speed and user speed as these two speed will be included in Vrelative.

We define:

-S is distance between user and objectives

-T is the device update rate

-Vrelative is speed combined with speed of user and speed of approaching objective.

-Salert is the alert distance

-t is human reaction time

Hence we have:

Vrelative = S/T

Then alert distance is:

Salert = Vrelative * t