Projects:2017s1-176 Smart Mirror with Raspberry Pi

Abstract

The future is here! Imagine yourself reading news feed or checking today weather while dressing or shaving in front of the mirror. The aim of the project is to achieve such a device, and incorporate cutting edge features. The Pi will connect with the internet to retrieve real-time information to display on a monitor. By hacking the monitor open, a one-way mirror can be placed on top to achieve a smart mirror!

Background

There have been many smart mirror projects conducted by others. They have predominantly used the Python and C programming languages to develop the software for their mirrors, and used operating systems such as Raspbian and Ubuntu. These mirrors are able to show the basic GUI and other information updated from website, such as weather and links of latest news. In addition, some smart mirror projects have voice control as well, which is supported by Amazon Alexa.

Our primary aim is to be incorporate face recognition technology into a smart mirror. We do not believe this has been done before as part of previous smart mirror projects. The secondary goal is to incorporate voice control. We also intend on using Android OS on the Raspberry Pi, and utilise user usage behaviour to customise their home screen.

Motivation

Such a device is expected to have a beneficial effect on society. It will streamline the lives of the users by allowing them to view information important to them and provide easy access to important services they may require. Furthermore, being able to control various household electronic systems from a single device will remove the need for a separate control for each device.

System Description

Both voice and image streams are obtained from the webcam. For face recognition, the image stream is processed by the face recognition algorithm. If the face is identified, the device is unlocked, otherwise access to smart mirror is denied.

Voice commands are processed once the device is unlocked. After being processed by the application program interface (API), the voice stream is converted to text. If the text matches existing commands, that specific command is executed.

Face Recognition

Several face detection & recognition algorithms were tested using Matlab and Python. The test algorithms followed the flow diagram in Fig 4. The Local Binary Pattern (LBP) Histogram and Viola-Jones algorithms performed best, and were selected for the Android implementation. The LBP algorithm achieved a recognition accuracy rate of 93%.

Voice Recognition

Two voice recognition algorithms were implemented and tested on Matlab. Dynamic Time Warping was found to be more accurate and simple when applied to isolated word recognition, while Hidden Markov Model was more reliable for sentence recognition. The DTW algorithm implemented on Matlab has an accuracy rate of 95%.

Android Development

Google Voice Search function was used to convert voice into text. Java class android.speech.RecognizerIntent simply activates Google Voice Search in Android Application and receive text commands returned from API.

Open source libraries were used to access the video stream from the USB webcam. However, the OpenCV libraries required for face recognition did not interface correctly with the video stream, hence face recognition was not successful on the Pi. A demo was however developed in an Unix environment

Problems Encountered

HMM Matlab...

Future Work and Recommendations

Successful implementation of voice control was achieved. The progress made can be leveraged to incorporate more variety of voice commands.

Similarly, the research, tests and development made for the face recognition aspect can be used to to incorporate face recognition, and other functionality such as multiuser capability into the system, and to integrate the device into a smart home system.

Project Team

Team Members

• Anbang Huang

• Syed Maruful Aziz

Project Supervisors

• Dr Withawat Withayachumnankul

• Dr Hong Gunn Chew