Projects:2016s1-197 Sound Triangulation for Invisible Keyboards - Revision history

A1681925: /* Microcontroller */

2016-10-26T05:45:54Z

‎Microcontroller

A1681925: /* Microphones */

2016-10-26T05:45:27Z

‎Microphones

A1681925: /* Capturing entire sound signals */

2016-10-26T05:40:59Z

‎Capturing entire sound signals

A1681925: /* Hardware System */

2016-10-26T05:40:07Z

‎Hardware System

A1681925: /* Keyboard */

2016-10-26T05:38:33Z

‎Keyboard

A1681925: /* Time Difference of Arrival (TDOA) */

2016-10-26T05:36:26Z

‎Time Difference of Arrival (TDOA)

A1681925: /* Trilateration Theory */

2016-10-26T05:30:07Z

‎Trilateration Theory

A1681925: /* Trilateration Theory */

2016-10-26T05:28:39Z

‎Trilateration Theory

A1681925 at 05:24, 26 October 2016

2016-10-26T05:24:45Z

A1681925: /* Project Instruction */

2016-10-26T05:21:43Z

‎Project Instruction

@@ Line 78: / Line 78: @@
 ChipKit Max32 will be used in this project which consists of a microcontroller PIC32MX795F512L and a development board. ChipKit Max32 is a prototyping platform that adds the performance of the Microchip PIC32 microcontroller.
 The Max32 board takes advantage of the powerful PIC32MX795F512L microcontroller, which features a 32-bit MIPS processor core running at 80 MHz and ADC speed of 1Mbps. This Chipkit can be programmed using Multi-Platform IDE (MPIDE)
 ==Theoretical Measurements==

@@ Line 72: / Line 72: @@
 ===Microphones===
 The Sparkfun Electret Microphone Breakout bob-09964 will be used in this project which consists of an omni-directional electret microphone and an amplifier. The operating voltage of it is from 2.7 V to 5.5 V. As mentioned earlier, a quantity of 3 microphones will be used.
 ===Microcontroller===

@@ Line 41: / Line 41: @@
 The first method is to capture the entire sound signals for three microphones by using analog input of PIC, and transfer the data to Matlab software, which can plot figure and detect the peak of the initial pressure pulse for each sound signal in time domain. Afterwards find the time corresponding to the peak for each one. As a result, the TDOA can be calculated by subtracting the two times, namely, t2-t1, as shown in figure 4.
 This method is able to track and inspect the data of sound signals received by PIC. However, using this idea has a couple of weaknesses, which cannot achieve a desired deliverable. Firstly, the speed of the keystrokes has been limited, because it is time-consuming to capture and transfer the entire sound signals for three channels, and the time of that is much longer than the time of one keystroke. Secondly, this idea is not able to achieve real-time monitoring of keystrokes, because analog reading and transferring data to Matlab are asynchronous communication. In addition, transferring data to Matlab can extend the time of analog reading indirectly. As a result, transferring data to Matlab must be at the end of analog reading of entire sound signals for three channels.
 ===Timer===

@@ Line 59: / Line 59: @@
 The hardware system consists of four main parts, namely, a keyboard, three microphones, three amplifiers and a PIC as figure 5 shown. Taping key engraved on keyboard will make a sound, which is picked by three fixed and identical microphones. Afterwards, the outputs of three microphones go though amplifiers respectively to amplify sound signals. Finally, PIC will receive signals and process data, and the results are shown on the screen by using serial monitor.
 ===Keyboard===
@@ Line 71: / Line 73: @@
 ChipKit Max32 will be used in this project which consists of a microcontroller PIC32MX795F512L and a development board. ChipKit Max32 is a prototyping platform that adds the performance of the Microchip PIC32 microcontroller.
 The Max32 board takes advantage of the powerful PIC32MX795F512L microcontroller, which features a 32-bit MIPS processor core running at 80 MHz and ADC speed of 1Mbps. This Chipkit can be programmed using Multi-Platform IDE (MPIDE)
 ==Theoretical Measurements==

← Older revision		Revision as of 05:38, 26 October 2016
Line 63:		Line 63:
	A flat plate is used as a keyboard engraved 12 “buttons” as shown in figure 6, which can be made by plastic or wood and both of them can be tapped to make a loud sound. The size of button is 1cm x 1cm, because if the button is too large, it will not save space and will trigger a big range of TDOA for each key, which may lead to trouble to distinguish the adjacent button. If the button is too small, it will be hard for users to tap button even though it reduces the difficulty for locating. Considering the points mentioned above, the 1cm x 1cm key can be an appropriate choice. The spacing between buttons is 2cm, which decides that the time of sampling to each sound signal must be less than 57.74 microseconds (us) according to the calculation in section 4.2.1. That will be a challenge for PIC. The three microphones will be placed and mounted at the edges as shown below, whose specification and design is described in the next section.		A flat plate is used as a keyboard engraved 12 “buttons” as shown in figure 6, which can be made by plastic or wood and both of them can be tapped to make a loud sound. The size of button is 1cm x 1cm, because if the button is too large, it will not save space and will trigger a big range of TDOA for each key, which may lead to trouble to distinguish the adjacent button. If the button is too small, it will be hard for users to tap button even though it reduces the difficulty for locating. Considering the points mentioned above, the 1cm x 1cm key can be an appropriate choice. The spacing between buttons is 2cm, which decides that the time of sampling to each sound signal must be less than 57.74 microseconds (us) according to the calculation in section 4.2.1. That will be a challenge for PIC. The three microphones will be placed and mounted at the edges as shown below, whose specification and design is described in the next section.

		+	[[File:keyboard.jpg]]

	===Microphones===		===Microphones===

@@ Line 30: / Line 30: @@
                                       Δt2 = Tku – Tiu = Dku / V – Diu / V	                                     (2)
 For our project case as the figure 3 shown, the minimum number of microphones is three according to the sound trilateration principle in two-dimensional geometry, and the reference points are the location of microphones; the subject point is the sound source. The t1 is the propagation time spent from sound source to Microphone 1; t2 is the propagation time spent from sound source to Microphone 2. Therefore, a TDOA can be acquired as Δt1 (Δt1 = t2- t1). Likewise, another TDOA can be calculated by subtracting times from sound source to Microphone 1 and Microphone 3 (Δt2 = t3- t1). It is not difficult to find that there are many points with the same TDOA (Δt1) on a curve as shown in figure 3, which is same as a hyperbola [3]. Another curve can be drawn by another TDOA (Δt2), which is also a hyperbola. The intersection of the two different hyperbolas provides a unique position of subject point (sound source) [3]. Consequently, two different TDOA can position an exclusive location in 2D case.
 ==Proposed Approaches==

@@ Line 19: / Line 19: @@
 Trilateration is a method to be used to locate the absolute or relative positions of points by measuring the distances and using the geometry of circles, spheres or triangles [2]. Compared with triangulation, it does not measure angles, but distances. More specifically, trilateration applies the known positions of two or more reference points to gauge the distance between a subject and each reference point [2]. These reference points are anchored and known locations.
 As to a two-dimensional (2D) plane shown in figure 2, at least three reference points are required to determine the position of a subject accurately and uniquely, which are (Xi, Yi), (Xj, Yj) and (Xk, Yk). The point of a subject is (Xu, Yu). Therefore, three distances between the subject and each reference point can be measured as Diu, Dju and Dku.
 ===Time Difference of Arrival (TDOA)===

@@ Line 2: / Line 2: @@
 With the rapid innovation of Human Interface Devices (HID), a common goal is to make more benefits, such as to be more portable, energy-saving and secure [1]. And the keyboard is the one of the most important HID, which is used universally in electronic devices. Therefore, improving the keyboard becomes significant. This project aims to design and implement an “invisible keyboard” by using three microphones to locate the position of keys based on the sound trilateration principle. There are three main parts to our project in terms of proposed method, system design and project management.
 The overall structure of system is shown in figure 1. Firstly, the sound source will be obtained by three identical microphones and then go through amplifiers respectively to amplify sound signals for three channels, which is to ensure the signals can be recognized by the PIC. After using three pin of PIC to receipt three channel signals, the PIC is able to process data, which can be transferred to the computer by a USB connector. Finally, the buttons pressed by users will display on the computer screen by using serial monitor.
@@ Line 17: / Line 19: @@
 Trilateration is a method to be used to locate the absolute or relative positions of points by measuring the distances and using the geometry of circles, spheres or triangles [2]. Compared with triangulation, it does not measure angles, but distances. More specifically, trilateration applies the known positions of two or more reference points to gauge the distance between a subject and each reference point [2]. These reference points are anchored and known locations.
 As to a two-dimensional (2D) plane shown in figure 2, at least three reference points are required to determine the position of a subject accurately and uniquely, which are (Xi, Yi), (Xj, Yj) and (Xk, Yk). The point of a subject is (Xu, Yu). Therefore, three distances between the subject and each reference point can be measured as Diu, Dju and Dku.
-[[File:2222.jpg]]
 ===Time Difference of Arrival (TDOA)===