Difference between revisions of "Projects:2017s1-165 Forensic Investigation of Fitness Devices"

From Projects
Jump to: navigation, search
(→‎Appendices)
 
(17 intermediate revisions by the same user not shown)
Line 54: Line 54:
  
  
 +
 +
'''Biometrics'''
 +
 +
Biometrics is defined as the ‘the science and technology of analyzing biological data’ [15]. For the purpose of this project, the following biometrics are considered:
 +
 +
1. Heartrate (beats/min)
 +
 +
2. Steps (steps/min)
 +
 +
3. Total Energy (kJ) = Resting Energy (kJ) + Active Energy (kJ)
  
 
== Apple Watch Series 1 ==
 
== Apple Watch Series 1 ==
Line 80: Line 90:
  
 
The Apple Watch uses the concept of photoplethysomography (PPG) to measure the user’s heart rate [6]. The technology uses a simple principle of light absorption. The red color of the blood is due to the reflection of the red light, and the absorption of the green light. The Watch has infrared and green LED lights which are paired with light sensitive photodiodes (Fig 2.1). These lights are flashed at a high frequency (>400 Hz), to measure the blood flow in the user’s wrist. When the heart beats, there is an increase in the blood flow in the wrist, thus resulting in an increase in the rate of green light absorption, which is then measured by the photodiodes. The LED brightness and sampling rate can be adjusted automatically by the Watch in low signal level conditions.The heart rate data is transmitted to the Health app every 10 minutes on average through a stable Bluetooth or Wi-Fi connection. The data can then be compiled in a graph for users to study.
 
The Apple Watch uses the concept of photoplethysomography (PPG) to measure the user’s heart rate [6]. The technology uses a simple principle of light absorption. The red color of the blood is due to the reflection of the red light, and the absorption of the green light. The Watch has infrared and green LED lights which are paired with light sensitive photodiodes (Fig 2.1). These lights are flashed at a high frequency (>400 Hz), to measure the blood flow in the user’s wrist. When the heart beats, there is an increase in the blood flow in the wrist, thus resulting in an increase in the rate of green light absorption, which is then measured by the photodiodes. The LED brightness and sampling rate can be adjusted automatically by the Watch in low signal level conditions.The heart rate data is transmitted to the Health app every 10 minutes on average through a stable Bluetooth or Wi-Fi connection. The data can then be compiled in a graph for users to study.
 +
 +
[[File:Apple watch heart rate sensor.jpg]]
 +
 +
[[File:Apple watch sensor teardown.jpg]]
  
  
Line 121: Line 135:
  
 
The PurePulse [10] is the continuous and automatic heart rate tracker used by FitBit. The sensor uses the principle of photoplethysomography, like the Apple Watch (Section 2.2.2). However, unlike the Apple Watch, the PurePulse sensor is capable of continuously monitoring the user’s heart rate during an activity and the resting heart rate too.  
 
The PurePulse [10] is the continuous and automatic heart rate tracker used by FitBit. The sensor uses the principle of photoplethysomography, like the Apple Watch (Section 2.2.2). However, unlike the Apple Watch, the PurePulse sensor is capable of continuously monitoring the user’s heart rate during an activity and the resting heart rate too.  
 +
 +
[[File:FitBit Alta HR PurePulse sensor.jpg]]
 +
 +
[[File:FitBit Alta teardown.jpg]]
  
  
Line 141: Line 159:
  
 
The Alta HR uses Bluetooth Low Energy (LE) wireless technology and an internet connection (Wi-Fi/mobile data) to sync with mobile devices and computers.  The device syncing range is up to 6.1 metres [9]. The device can also be synced with a computer using the dongle. If ‘All-day’ syncing is turned on, the devices should automatically sync every 15-30 minutes. Manual syncing is also available [14].
 
The Alta HR uses Bluetooth Low Energy (LE) wireless technology and an internet connection (Wi-Fi/mobile data) to sync with mobile devices and computers.  The device syncing range is up to 6.1 metres [9]. The device can also be synced with a computer using the dongle. If ‘All-day’ syncing is turned on, the devices should automatically sync every 15-30 minutes. Manual syncing is also available [14].
 +
 +
 +
 +
== Related Work ==
 +
 +
There is no decisive method to estimate the time of death in a murder case. The project aims to devise a technique to accurately calculate the victim’s time of death by analysing the user’s heart rate data logs acquired by the Apple Watch. There have been several studies and research for using wearable and mobile devices for forensic investigations, however these are mostly limited to device imaging and data examination of call, text, and location logs. No known research has been undertaken earlier to estimate a murder victim’s time of death using a fitness device.
 +
 +
 +
 +
== Methodology ==
 +
 +
'''Data Extraction – Apple Watch'''
 +
 +
Data extraction from the Apple Watch can be done by 4 ways:
 +
 +
1. Apple Health App (requires user’s phone password)
 +
 +
2. iCloud backup (requires user’s Apple ID)
 +
 +
3. iTunes backup (requires user’s phone password)
 +
 +
4. Chip-off
 +
 +
 +
'''Health App'''
 +
 +
The Apple Health App is a built-in application on all Apple Devices which allows the user to keep track of its activity and heart rate data.
 +
 +
[[File:Apple health app dashboard.png]]
 +
 +
The health data recorded by the Apple Watch and other Apple Devices (including the iPhone) can be accessed through the Health App.
 +
 +
[[File:Steps data from iPhone and Apple watch.jpg]]
 +
 +
The details of a particular activity log can also be accessed. The precise start and end time of the activity can be logged.  There is also a log time for when the data from the Apple Watch is synced and added to the Health App.
 +
 +
[[File:Details of data log in health app.jpg]]
 +
 +
 +
For a detailed analysis, the data can be exported from the Health App to a computer in .xml file format.
 +
 +
The steps to extract the health data from the Apple Health App are as follows:
 +
 +
 +
1. Open the Health App on the paired iPhone.
 +
 +
 +
[[File:Health app interface.png]]
 +
 +
 +
2. Tap on the‘User’ icon on the top right corner, and then click on ‘Export Health Data’. The data can then be emailed as a .zip file.
 +
 +
 +
[[File:Health app user details.png]]
 +
 +
 +
3. The exported health data can be accessed from the export.xml file in the above-mentioned .zip folder.
 +
 +
 +
The user’s personal details like date of birth, sex, blood group, skin type, height (in cm), and mass (in kg) is displayed.
 +
 +
 +
[[File:Excerpt from .xml file with user details.png]]
 +
 +
 +
The .xml file also shows the heart rate data (in count/min) or beats-per-minute (BPM), steps, distance covered (in km), basal and active energy burned (in kJ), and exercise time (in mins).  The exact activity log time (creationDate), and activity start and end times (startDate, endDate) are also displayed. Towards the end of the file, an activity summary log is also included.
 +
 +
 +
[[File:.xml file showing step count.png]]
 +
 +
[[File:.xml file showing heart rate data.png]]
 +
 +
 +
The exported data in the .xml file is hard to read and analyze. Thus, it needs to be converted in to a .csv (comma separated file), which can be opened using Microsoft Excel. This allows easy data visualization and analysis. 
 +
To convert the data to a .csv file, an open-source Python code from GitHub [12] was used (Code in Appendix). Ensure that the export.xml file is in the same directory as the code file.
 +
 +
[[File:.csv file showing heart rate data.png]]
 +
 +
[[File:.csv file showing step count data.png]]
 +
 +
For the purpose of this project, all the health data was extracted using the Apple Health App.
 +
 +
 +
'''iCloud Backup'''
 +
 +
If the paired iPhone is unavailable, the health data can be accessed through iCloud and iTunes backup. Data from all iOS devices is automatically backed up in iCloud when a stable Wi-Fi connection is available. However, iCloud can only be accessed if the Apple ID username and password is known. In the case where the user’s credentials are unknown, a third-party software like the Elcomsoft Phone Breaker (EPPB) can be used to gain access. This software can retrieve all data backed up in the cloud by using a ‘binary authentication token’ [17] formulated by the iCloud Control Panel to acquire the account log-in credentials.
 +
 +
 +
'''iTunes Backup'''
 +
 +
The Apple iTunes software can also be used to create a backup for all iOS devices. However, the iTunes backup is stored on an Apple Mac or a PC, and not online. If the log-in details are known, data can be directly acquired from iTunes. If not, data can be extracted by using forensic tools like UFED Touch/UFED Physical Analyser.
 +
 +
 +
'''Chip-off'''
 +
 +
Chip-off is the least preferred method to extract data. Chip-off is the removal of the flash memory chip which stores the device’s data [18]. This invasive method allows the user to decode data in the absolute raw form from the memory, thus decoding it is quite challenging. Chip-off also requires the use of special tools to safely isolate the data chips. To read the data from the flash memory chips, small wires must be connected to certain ‘contact points on the monolithic package’s hidden ball grid array’. This process is called ‘spiderwebbing’.
 +
 +
 +
 +
'''Data Extraction – FitBit Alta HR'''
 +
 +
 +
'''FitBit Mobile App'''
 +
 +
The FitBit Alta HR is primarily paired with an android phone for this project using the FitBit mobile app. The app dashboard displays the number of steps, heart rate, distance travelled, calories burned, and active minutes as recorded by the wearable. The app also displays the daily summary of the data in a graphical format for easy understanding.
 +
 +
[[File:FitBit mobile app dashboard.png]]
 +
 +
The step data for the daily summary is calculated for a 15-minute interval, whereas the heart rate data is averaged for every 5 minutes.
 +
 +
[[File:Daily step summary.png]]
 +
 +
[[File:Daily summary of heart rate data.png]]
 +
 +
[[File:Detailed log of heart rate data.png]]
 +
 +
 +
'''FitBit Computer Interface'''
 +
 +
The same data can be accessed from a Windows/Mac computer using the FitBit computer interface, which is accessible through https://www.FitBit.com/ .
 +
 +
[[File:FitBit computer dashboard.png]]
 +
 +
A more detailed activity log averaged over 5 minutes can also be accessed for a particular day along with the respective heart rate and energy data for the ‘Active’ minutes. Active minutes are awarded by FitBit after at least ‘10 minutes of moderate to intense activity’ [15].
 +
 +
[[File:Active minutes data.png]]
 +
 +
 +
[[File:Detailed activity log (step count).png]]
 +
 +
 +
FitBit allows users to extract daily activity summary in the form of .csv or .xls files which can be opened through Microsoft Excel. However, no intraday data is provided.
 +
 +
 +
[[File:.csv file for daily summary of exported data.png]]
 +
 +
 +
 +
'''Intraday Data Extraction using FitBit API'''
 +
 +
FitBit does not allow its users to extract second-level intraday data. The computer interface only displays steps data averaged over every 15 minutes, and heart rate data averaged over every 5 minutes.
 +
To extract intraday data, an external app was created using the web based FitBit API (Application Programming Interface). The FitBit API allows developers to register a personal app which authorises them to connect to the FitBit web app server and access the stored data. This process allowed the steps and energy data to be extracted for every minute, and the heart rate data to be extracted for every few seconds.
 +
 +
The process for the data extraction using the FitBit API are as follows:
 +
 +
1. Register a personal app on http://dev.fitbit.com.
 +
 +
 +
[[File:FitBit API - personal app registration.png]]
 +
 +
 +
2. Click on ‘OAuth 2.0 tutorial page’.
 +
 +
 +
[[File:FitBit API - application.png]]
 +
 +
 +
3. Type a ‘Redirect URL’. For this project, https://www.google.com.au was used as the Redirect URL. Select the scope of the data that needs to be accessed.
 +
 +
 +
[[File:FitBit API - redirect url.png]]
 +
 +
 +
4. Click on the authorisation URL provided and then copy the access token in the field provided.
 +
 +
 +
[[File:FitBit API - authorisation url.png]]
 +
 +
 +
 +
5. To make a call-back request, click on ‘Send to Hurl.it’.
 +
 +
 +
[[File:FitBit API - data request.png]]
 +
 +
 +
6. According to the FitBit Web API Reference document, there are 2 request formats for retrieving data.
 +
 +
https://api.fitbit.com/1/user/[user-id]/activities/steps/date/[date]/[period].json
 +
 +
https://api.fitbit.com/1/user/[user-id]/activities/steps/date/[date]/[end-date]/[detail-level]/time/[start-time]/[end-time].json
 +
 +
 +
[[File:FitBit API data parameters.JPG]]
 +
 +
 +
Example request:
 +
 +
https://api.fitbit.com/1/user/-/activities/steps/date/today/1d/1min.json
 +
 +
[[File:FitBit API - data request url.png]]
 +
 +
 +
Copy the request URL in the ‘Destination’ field and launch request.
 +
 +
 +
The extracted data is available in a easy-to-read format, and also available in raw-data format. 
 +
 +
 +
[[File:Minute level data for step count.png]]
 +
 +
 +
Other Resource Path options:
 +
 +
activities/calories
 +
 +
activities/distance
 +
 +
activities/floors
 +
 +
activities/elevation
 +
 +
activities/heart
 +
 +
 +
7. To convert the data into a .xls or .csv file, copy the raw data into Notepad and save as a .txt file. The .txt file can then be imported to Microsoft Excel and saved as either.xls or .csv file.
 +
 +
[[File:FitBit - .csv file showing step count data.png]]
 +
 +
 +
 +
'''Experimentation – Apple Watch'''
 +
 +
 +
'''Effect of arm movement on number of steps recorded'''
 +
 +
It was observed that steps were recorded by the Apple Watch when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. This indicates that only arm movements at a relatively higher acceleration can be registered as steps.
 +
 +
 +
'''Minimum number of steps to register a pedometer event'''
 +
 +
To determine the minimum number of steps to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed. The steps taken by the user were manually counted and the Health app on the paired iPhone was regularly checked to see whether the activity was recorded or not. The user walked 30, 20, 15, 10, 5, and 3 steps. It was observed that a minimum of 10 steps were required to register a pedometer event in the Apple Watch, and therefore the Health App.
 +
 +
 +
'''Minimum activity time to register a pedometer event'''
 +
 +
To determine the minimum time to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed for 20s, 10s, 5s, 3s, and 1s. The user was timed using a stop watch to an accuracy of 1ms and steps taken by the user were counted manually. It was observed that no data was recorded for 1s. However, 10 steps were recorded when the user walked for 3 seconds, thus suggesting that the user took an average of 0.33s for each step.
 +
 +
 +
'''Effect of walking speed on the step count'''
 +
 +
It was observed that when the walking speed of the user was significantly reduced, the Watch only recorded 5 steps, while the manual step count was 10.
 +
 +
 +
'''Pedometer interval time'''
 +
 +
The following table show the details of the data log for a pedometer event. To determine the meaning of the respective ‘Start Time’ and ‘End Time’, 5 sets of readings were taken (24hrs format) and the activity start and end times were manually recorded, and then compared to the data logged on the Health App.
 +
 +
[[File:Apple watch - measured and logged pedometer times.JPG]]
 +
 +
[[File:Apple watch - average lag times.JPG]]
 +
 +
From the previous table, it can be observed that on an average, there is a lag of 4.6 seconds between the measured and the logged start times. Similarly, there is lag of 17.2 seconds between the measures and the logged end times. On an average, the data is logged into the health app 43.8 seconds after the logged end time.
 +
 +
 +
 +
'''Effect on step count when Apple Watch is worn loosely'''
 +
 +
When the Apple Watch is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.
 +
 +
 +
'''Effect on step count when Apple Watch is placed in user’s bag'''
 +
 +
The Watch was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via local tram. It was observed that false data was recorded by the Watch. The Watch recorded a total of 7576 steps, suggesting that readings were also taken when the user was in the tram and relatively stationary. The built-in pedometer in the paired iPhone recorded 7892 steps.
 +
 +
 +
'''Minimum height and number of steps to record a ‘Flight of Stairs’'''
 +
 +
A flight of stairs is counted when the user gains at least 10 feet (3 metres) of elevation. On an average, 16 steps formulate 3 metres elevation gain. This is equivalent to climbing 1 floor.
 +
The Apple Watch Series 1 does not have an altimeter to measure the elevation gain. The iPhone consists of an altimeter sensor.
 +
 +
 +
'''Forced and continuous heart rate logs'''
 +
 +
In its normal state, the Apple watch does not measure the user’s heart rate continuously. However, a heart rate measurement can be forced using the ‘Heart Rate Glance’ feature in the Watch. On an average, it takes approximately 5 seconds to obtain an initial heart rate reading.
 +
 +
Theoretically, the Watch is supposed to measure the user’s heart rate every 10 minutes. However, the Watch can continuously monitor and record the user’s heartbeat in ‘Workout’ mode.
 +
 +
[[File:Apple watch heart rate logs.JPG]]
 +
 +
The table demonstrates the heart rate logs over a period of approximately 2 hours when the user was sitting quietly. The activity start time, the data logtime, and the heart rate count are shown. The difference between the subsequent start times, and the difference between the activity start time and data log time is also calculated. Their respective average, maximum, and minimum values are also calculated.
 +
It can be observed that the average time difference between subsequent start times is approximately 7 minutes 39 seconds, whereas the average time difference between start and logging times is 8 minutes 18 seconds.
 +
 +
 +
 +
'''Effect of different skin colours on heart rate sensor'''
 +
 +
When the Apple Watch was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes. 
 +
 +
 +
'''Effect of tattoos on heart rate sensor'''
 +
 +
The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.
 +
 +
 +
'''Effect of loose fitting on heart rate sensor'''
 +
 +
Stable skin-contact and a good Watch fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.
 +
 +
 +
 +
 +
 +
'''Experimentation – FitBit Alta HR'''
 +
 +
 +
'''Accuracy of steps recorded'''
 +
 +
The FitBit device used continuously measures the step taken by the user. The user manually calculated the steps taken, and then compared the number with the step count recorded by the device. It was observed that the device’s step count is extensively accurate. This holds true when the user was walking briskly and running.
 +
 +
 +
'''Effect of arm movement on number of steps recorded'''
 +
 +
It was observed that steps were continuously recorded by the FitBit device even when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. False steps were also recorded while the user was doing chores such as washing dishes, eating, typing on a computer, etc. Hence, it can be stated that any significant arm movement can be misreported by the device as physical steps taken by the user.
 +
 +
 +
'''Minimum number of steps to register a pedometer event'''
 +
 +
The FitBit device continuously records steps. Hence, each step taken by the user is logged into the device in real time.
 +
 +
 +
'''Minimum time between subsequent steps to register a pedometer event'''
 +
 +
The FitBit device can record a solitary step, thus, it does not need continuous movement to be able to record steps.
 +
 +
 +
'''Effect on step count when FitBit is worn loosely'''
 +
 +
When the FitBit device is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.
 +
 +
 +
'''Effect on step count when FitBit device is placed in user’s bag'''
 +
 +
The device was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via train. It was observed that false data was recorded by the device. The device recorded a total of 2385 steps. However, unlike the Apple Watch, no readings were recorded while the user was on the train.
 +
 +
 +
'''False heart rate readings when the device is not worn'''
 +
 +
It was observed that several false heart rate readings were recorded even when the user was not wearing the device. If the device is completely stationary, the heart rate sensor stops after approximately 5 seconds. However, even the slightest movement near the device can trigger the sensor. If an unusually high heart rate is recorded only once, the measurement is not logged into the FitBit App.
 +
 +
 +
'''Effect of different skin colours on heart rate sensor'''
 +
 +
When the FitBit device was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes. 
 +
 +
 +
'''Effect of tattoos on heart rate sensor'''
 +
 +
The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.
 +
 +
 +
'''Effect of loose fitting on heart rate sensor'''
 +
 +
Stable skin-contact and a good fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.
 +
 +
 +
'''Time required by the heart rate sensor to establish beats per minute (BPM)'''
 +
 +
To determine the time taken by the FitBit heart rate sensor to acquire a reading, the user put on the device 4 times, and recorded the time required for the device to display the heart rate using a stopwatch.
 +
 +
[[File:FitBit - .csv file showing step count data.png]]
 +
 +
From the table , it can be observed that it takes an average of 3.2 seconds for the FitBit device to record and display the user’s heart rate measurement.
 +
 +
 +
 +
'''Logging of non-periodic heartbeat measurements'''
 +
 +
FitBit’s heart rate sensor is equipped with the PurePulse technology which allows for the continuous measurement of the user’s heart rate, even when the user is sitting extremely still.
 +
 +
 +
'''FitBit Device Syncing Logs'''
 +
 +
Unlike Apple, FitBit does not support a sync log. Only the ‘last synced’ time is displayed in both the mobile and computer apps. Although, per the FitBit database [11], the device should sync with the paired phone every 15 minutes. However, it was observed that it is not true. The sync intervals were usually between 30-90 minutes unless manually synced.
 +
 +
 +
 +
'''Data Records - FitBit Alta HR'''
 +
 +
[[File:FitBit - step count data.JPG]]
 +
 +
 +
[[File:FitBit - heart rate data.JPG]]
 +
 +
 +
[[File:FitBit - active energy data.JPG]]
 +
 +
 +
From the figures, it can be observed that data acquired from a FitBit device is more consistent as FitBit is capable of continuously recording the user’s steps and heart rate. It can also be observed that the user’s heart rate is higher when the step count increases, and thus, the amount of active energy expenditure is also higher at that time. When the user is at rest (0 steps recorded), the active energy decreases to a nominal basal value of 4.387kJ. This value is based on the user’s BMR.
 +
 +
 +
 +
 +
'''Data Records - Apple Watch'''
 +
 +
[[File:Apple watch - step count data.JPG]]
 +
 +
 +
[[File:Apple watch - heart rate data.JPG]]
 +
 +
 +
[[File:Apple watch - active energy data.JPG]]
 +
 +
 +
From the table, it can be observed that despite the user’s active movement, the heart rate records are not continuous and periodic. Over a period of 10 minutes, the user’s heart rate was measured only 2 times. The user’s energy expenditure readings seem to be based on the step count rather than the heart rate of the user. It can also be observed that the number of data points is significantly less than what is observed in the graphs obtained from the FitBit data. However, the biometrics tend to follow a similar trend as observed in the data acquired from FitBit. The peak heart rate measurement observed at 16:07:00 (130 bpm) could be considered a false value.
 +
 +
 +
 +
'''Post-mortem Analysis'''
 +
 +
Based on the data sets acquired from the FitBit and Apple devices, a synthetic data set was created to depict a possible murder scenario.
 +
The timeline follows a period of 1 hour, and the victim’s steps count, heart rate, and active energy data is considered.
 +
 +
[[File:Victim - step count data.JPG]]
 +
 +
 +
[[File:Victim - heart rate data.JPG]]
 +
 +
 +
[[File:Victim - active energy data.JPG]]
 +
 +
 +
 +
In the figures, 5 specific incidents have been highlighted which are crucial for the timeline of the case.
 +
 +
1. Defence (16:45:00)
 +
 +
• 9 false steps recorded due to defensive arm movement. As mentioned in Section 5.4.2, significant arm movement can be recorded as false steps by the device.
 +
 +
• Slightly elevated heart rate.
 +
 +
• Average level of active energy burned.
 +
 +
 +
2. Attack (16:46:00)
 +
 +
• High heart rate following defensive action and stress
 +
 +
• High level of active energy burned.
 +
 +
• Subsequent variability in heart rate (from 16:46:00 to 16:57:00) due to possible cardiac arrest.
 +
 +
 +
3. Trauma (16:47:00)
 +
 +
• Fluctuating heart rate.
 +
 +
• Low energy – immediate effect of the traumatic injury. As the victim loses blood, the body tends to cool down. There is a subsequent increase in the active energy as the body is attempting to maintain a constant core body temperature.
 +
 +
 +
4. Last recorded heart rate (16:57:00)
 +
 +
• Low heart rate measurement.
 +
 +
• Low active energy. Active energy continuously decreases in the following minutes.
 +
 +
 +
5. Death (17:01:00)
 +
 +
• No heart rate measurement.
 +
 +
• Lowest active energy measurement.
 +
 +
== Conclusion ==
 +
 +
In majority of criminal investigations, it is challenging to establish an incident timeline, and to estimate the victim’s time of death. The project aimed to determine the extent to which fitness devices can be used as legitimate digital evidence in criminal investigations. The personal health data recorded on these devices can be used to perform post-mortem analysis of exercise-induced death or murder. The Apple Series 1 Watch and FitBit Alta HR were used for analysis. Phase 1 of the project focused on the techniques utilised to extract the user’s health data from the devices under consideration and their respective cloud based servers. To establish the accuracy of the measurements, and limitations of the data logs, certain experiments were conducted in Phase 2 of the project. For the scope of the project, the biometrics considered primarily were the user’s step count, heart rate, and active energy expenditure.
 +
 +
The following conclusions were derived:
 +
 +
1. Certain data logging delays were observed for the Apple Watch. The timestamps on the data logs can possibly be used to establish whether the user’s phone was turned on and had a wireless connection with the fitness device.
 +
 +
2. The heart rate data acquired from dark-skinned and tattooed users cannot be used as a reliable source of information for criminal investigations.
 +
 +
3. Several false steps can be recorded when there is significant arm movement, even when the user is not physically taking any steps, for instance, defensive arm movement of the victim. False steps can also be recorded when the user is in a moving vehicle, or when the device is in the user’s bag or pocket.
 +
 +
4. An incident timeline can be created by analysing the trends in the user’s step count, heart rate and energy expenditure. 
 +
 +
5. The last heart rate record does not necessarily signify death. A more accurate time of death can be established using the active energy burned by the user.
 +
 +
 +
 +
== Future Work ==
 +
 +
The scope of the project can be further extended to explore and relate the health data records with the device’s GPS data to establish a more accurate timeline and whereabouts of the victim or potential suspects.
 +
Presently, certain smartphones are equipped with advanced health apps that can record the user’s step count, heart rate, blood oxygen levels, and even stress levels. Further research can be undertaken to establish the accuracy of those records. Other wearable fitness devices like chest straps, headbands, and ‘smart’ shoes can also be investigated to determine their reliability as forensic evidence.
 +
In cases where access to the device’s cloud servers is limited, physical chip-off analysis can also be conducted to determine which data can be recovered and analysed for investigative purposes.
 +
The detailed comparison of the features of top fitness devices in the market is included in the Appendix.
 +
 +
 +
 +
 +
== Appendices ==
 +
 +
'''Code to convert raw Apple Health app data to .csv file'''
 +
 +
As stated in previous sections, an open-source Python code was used from GitHub to convert the raw data extracted from the Apple Health app to a .csv file. The code is as follows:
 +
 +
[[File:Applehealthdata-001.jpg]]
 +
 +
 +
[[File:Applehealthdata-002.jpg]]
 +
 +
 +
[[File:Applehealthdata-003.jpg]]
 +
 +
 +
[[File:Applehealthdata-004.jpg]]
 +
 +
 +
[[File:Applehealthdata-005.jpg]]
 +
 +
== References ==
 +
 +
[1] J. Stables, "Best fitness trackers 2017: FitBit, Garmin, Misfit, Withings and more", Wearable, 2017. [Online]. Available: https://www.wareable.com/fitness-trackers/the-best-fitness-tracker. [Accessed: 15- Apr- 2017].
 +
 +
[2] "Articles", Practicalhomicide.com, 2017. [Online]. Available: http://www.practicalhomicide.com/Research/LOmar2007.htm. [Accessed: 16- Apr- 2017].
 +
 +
[3] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed:21-Oct- 2017].
 +
 +
[4] A. Robinson, "Heart Rate Vs. Calories Burned", LIVESTRONG.COM,2017. [Online]. Available: https://www.livestrong.com/article/39443-heart-rate-vs.-calories-burned/. [Accessed: 21- Oct- 2017].
 +
 +
[5] "Apple Watch Teardown - iFixit", Ifixit.com, 2017. [Online]. Available: https://www.ifixit.com/Teardown/Apple+Watch+Teardown/40655. [Accessed: 15- Apr- 2017].
 +
 +
[6] "Your heart rate. What it means, and where on Apple Watch you’ll find it.", Apple Support, 2017. [Online]. Available: https://support.apple.com/en-au/HT204666. [Accessed: 18- Apr- 2017].
 +
 +
[7]"Fitzpatrick scale", En.wikipedia.org, 2017. [Online]. Available: https://en.wikipedia.org/wiki/Fitzpatrick_scale. [Accessed: 31- May- 2017].
 +
 +
[8]"Apple engineer explains how Apple decides which HealthKit data types to add", MobiHealthNews, 2017. [Online]. Available: http://www.mobihealthnews.com/44413/apple-engineer-explains-how-apple-decides-which-healthkit-data-types-to-add. [Accessed: 31- May- 2017].
 +
 +
[9]"Shop FitBit Alta HR", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/shop/altahr. [Accessed: 20- May- 2017].
 +
 +
[10]"FitBit PurePulse™ Continuous Wrist-Based Heart Rate", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/purepulse. [Accessed: 20- May- 2017].
 +
 +
[11]"FitBit's new Alta HR activity tracker brings a heart rate monitoring update to the original", The Verge, 2017. [Online]. Available: https://www.theverge.com/2017/3/6/14823688/FitBit-new-alta-hr-fitness-tracker-heart-rate-sleep-tracking. [Accessed: 22- May- 2017].
 +
 +
[12]"FitBit Alta Teardown | Chipworks", Chipworks.com, 2017. [Online]. Available: http://www.chipworks.com/about-chipworks/overview/blog/FitBit-alta-teardown. [Accessed: 01- Jun- 2017].
 +
 +
[13] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed: 27- Oct- 2017].
 +
 +
[14]"How do FitBit trackers sync their data?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1877/?l=en_US&c=Topics%3ASyncing&fs=Search&pn=1. [Accessed: 23- May- 2017].
 +
 +
[15] Guthrie, R. (2017). The future of biometric tracking will make step counters look like antiques. [online] Digital Trends. Available at: https://www.digitaltrends.com/health-fitness/future-of-biometrics-beyond-the-wrist-tracker/.
 +
 +
[16] "tdda/applehealthdata", GitHub, 2017. [Online]. Available: https://github.com/tdda/applehealthdata. [Accessed: 29- May- 2017].
 +
 +
[17] "Advanced mobile forensics: iOS (iPhone and iPad), Windows Phone and BlackBerry 10", Elcomsoft.com, 2017. [Online]. Available: https://www.elcomsoft.com/eppb.html. [Accessed: 19- Apr- 2017].
 +
 +
[18] "Chip-Off Forensics Services - Gillware Digital Forensics", Gillware Digital Forensics, 2017. [Online]. Available: https://www.gillware.com/forensics/chip-off-forensics-services. [Accessed: 18- Apr- 2017].
 +
 +
[19] "What are active minutes?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1379/?l=en_US&c=Topics%3ADashboard&fs=Search&pn=1. [Accessed: 01- Jun- 2017].
 +
 +
[20] "FitBit Web API Basics — FitBit Web API Docs", Dev.FitBit.com, 2017. [Online]. Available: https://dev.FitBit.com/docs/basics/. [Accessed: 30- May- 2017].

Latest revision as of 21:33, 29 October 2017

Project Group Members

Sanjam Kohli

Yuan Li


Project Supervisor

Dr. Matthew Sorell


Introduction

High-tech wearable devices have always been objects of interest in science fiction. From cheap plastic activity bands or rudimentary watches, these gadgets have evolved into elegantly designed devices that can provide greater connectivity, location services, and more importantly, incredible insights into people’s health. These nifty instruments have the capability to monitor a consumer’s heart rate, sleep patterns, and even one’s blood oxygen levels. The smartwatch and fitness band market is dominated by global technology leaders Apple, Samsung, and FitBit. More than 50% of the Australians now own a smartwatch [1]. As these devices are being adopted by a growing number of users, there is an increasing potential for them to become a haven for digital evidence in criminal investigations.


Aim

The project aims to explore the use of wearable fitness devices as forensic evidence, and to establish movement and activities of victims or suspects involved in a homicide investigation. This was achieved by analysing the heart beat and activity records which can be extracted and analysed from the Apple Watch and a FitBit device or their respective paired phones.


Motivation and Significance

A victim’s time of death is crucial to every criminal investigation. Currently, it is extremely challenging to determine the time of death in a homicide investigation using conventional methods. The project attempts to develop a mechanism to establish a more accurate timeline of the incident and a precise time of death using the heart beat and activity logs extracted from fitness devices. The method devised could further assist the South Australian Police (SAPOL), and other law enforcing agencies in future investigations.


Technical Background

Determining the time of death

Estimating the time of death is very crucial to a homicide investigation. It is a critical element of the case timeline. A specific and accurate time of death can corroborate any statements given by suspects in a crime. Despite years of research by forensic experts, no conclusive method has been devised to estimate a victim’s time of death. Presently, the estimation is based on various case specific factors and pathological changes in the human body like changes in body temperature, muscle stiffness, and lividity. In the absence of any witnesses, the complexity of this process increases. By using the current methods, the time of death is usually placed within a range of hours. These processes are highly subjective to errors, and thus it is ‘utterly impossible’ to fix an exact time [2].


Resting and Active Calories

The Basal Metabolic Rate (BMR) is defined as the amount of energy (measured in kilojoules, kJ) burned at rest [3]. BMR is calculated using one’s biometrics like weight, height, age, and sex. Resting calories signify the caloric base burn rate, and are calculated by using the individual’s BMR. Active calories are the calories that are burned due to additional activity.

1 kilocalorie (kCal) = 4.184 kilojoules (kJ)

Total Energy (kJ) = Basal Energy (kJ) + Active Energy (kJ)

Calorie expenditure is relatively linear to heart rate for an average individual, provided that the individual’s heart rate remains within the safe range of 90-150bpm [4].


Biometrics

Biometrics is defined as the ‘the science and technology of analyzing biological data’ [15]. For the purpose of this project, the following biometrics are considered:

1. Heartrate (beats/min)

2. Steps (steps/min)

3. Total Energy (kJ) = Resting Energy (kJ) + Active Energy (kJ)

Apple Watch Series 1

Device Specifications

The specifications [5] for the Apple Watch used in this project are as follows:

1. 38mm (vertically)

2. 290ppi screen

3. Custom designed Apple S1 SiP (system in package) chip.

4. NFC +WiFi 802.11b’g’n + Bluetooth 4.0

5. 8 GB onboard storage

6. Sensors: Heart rate monitor, gyroscope, accelerometer

Apple watch.png


Photoplethysomography

The Apple Watch uses the concept of photoplethysomography (PPG) to measure the user’s heart rate [6]. The technology uses a simple principle of light absorption. The red color of the blood is due to the reflection of the red light, and the absorption of the green light. The Watch has infrared and green LED lights which are paired with light sensitive photodiodes (Fig 2.1). These lights are flashed at a high frequency (>400 Hz), to measure the blood flow in the user’s wrist. When the heart beats, there is an increase in the blood flow in the wrist, thus resulting in an increase in the rate of green light absorption, which is then measured by the photodiodes. The LED brightness and sampling rate can be adjusted automatically by the Watch in low signal level conditions.The heart rate data is transmitted to the Health app every 10 minutes on average through a stable Bluetooth or Wi-Fi connection. The data can then be compiled in a graph for users to study.

Apple watch heart rate sensor.jpg

Apple watch sensor teardown.jpg


Steps Count

The Apple Watch has an accelerometer sensor which acts as a built-in step counter or pedometer. The steps are counted based on the height and stride length of the user.


Calorie Count

The Apple Watch measures the basal and active calories burnt using the biometrics (sex, weight, height) entered by the user, the user’s heart rate, and average human statistics. The activity being performed by the user is identified by the accelerometer, and is also considered for calculating energy expenditure.


Apple Watch memory and storage

The device consists of 512 MB of dynamic RAM, and 8 GB of flash memory. The Watch uses an HFS+ (hierarchical file system) created by Apple Inc., which has limited storage capacity than a device using removable SD cards


Device Syncing

The Apple Watch does not consist of a physical diagnostic port for users to transmit their data between devices. Thus, all the data is transferred and backed up in a companion iPhone by using either a Wi-Fi or Bluetooth connection. Once both the devices are in range, a stable ‘data stream’ is established. All the data is also backed up automatically in iCloud. The heart beat logs acquired by the Watch are sent to the paired iPhone. Using the built-in Health app, the user can access this data

FitBit Alta HR

Device Specifications

The hardware specifications of the device used for the project are as follow [9]:

1. 15mm wide

2. OLED tap display

3. Bluetooth 4.0

4. Sensors: Optical heart rate tracker, 3-axis accelerometer, vibration motor

5. Memory: 7 days of detailed data storage (minute by minute), daily summaries for 30 days


PurePulse

The PurePulse [10] is the continuous and automatic heart rate tracker used by FitBit. The sensor uses the principle of photoplethysomography, like the Apple Watch (Section 2.2.2). However, unlike the Apple Watch, the PurePulse sensor is capable of continuously monitoring the user’s heart rate during an activity and the resting heart rate too.

FitBit Alta HR PurePulse sensor.jpg

FitBit Alta teardown.jpg


Step Count

FitBit consists of a 3-axis accelerometer which is used to detect motion. The sensor calculates the step count, distance, and calories burnt based on the duration, frequency, and intensity of the activity [13].


Calorie Count

FitBit considers the user’s biometrics like weight, age, height, and sex to determine the user’s BMR, and thereby calculate the energy expenditure based on the user’s activity.


Device Memory and Storage

The Alta HR can store detailed minute-by-minute data for up to 7 days in the device. Daily summaries can be stored for up to 30 days in the device if the data is not synced to cloud [9]. For investigative purposes, the device data needs to be exported within 4-5 days to ensure the details are preserved and data is not lost.


Device Syncing

The Alta HR uses Bluetooth Low Energy (LE) wireless technology and an internet connection (Wi-Fi/mobile data) to sync with mobile devices and computers. The device syncing range is up to 6.1 metres [9]. The device can also be synced with a computer using the dongle. If ‘All-day’ syncing is turned on, the devices should automatically sync every 15-30 minutes. Manual syncing is also available [14].


Related Work

There is no decisive method to estimate the time of death in a murder case. The project aims to devise a technique to accurately calculate the victim’s time of death by analysing the user’s heart rate data logs acquired by the Apple Watch. There have been several studies and research for using wearable and mobile devices for forensic investigations, however these are mostly limited to device imaging and data examination of call, text, and location logs. No known research has been undertaken earlier to estimate a murder victim’s time of death using a fitness device.


Methodology

Data Extraction – Apple Watch

Data extraction from the Apple Watch can be done by 4 ways:

1. Apple Health App (requires user’s phone password)

2. iCloud backup (requires user’s Apple ID)

3. iTunes backup (requires user’s phone password)

4. Chip-off


Health App

The Apple Health App is a built-in application on all Apple Devices which allows the user to keep track of its activity and heart rate data.

Apple health app dashboard.png

The health data recorded by the Apple Watch and other Apple Devices (including the iPhone) can be accessed through the Health App.

Steps data from iPhone and Apple watch.jpg

The details of a particular activity log can also be accessed. The precise start and end time of the activity can be logged. There is also a log time for when the data from the Apple Watch is synced and added to the Health App.

Details of data log in health app.jpg


For a detailed analysis, the data can be exported from the Health App to a computer in .xml file format.

The steps to extract the health data from the Apple Health App are as follows:


1. Open the Health App on the paired iPhone.


Health app interface.png


2. Tap on the‘User’ icon on the top right corner, and then click on ‘Export Health Data’. The data can then be emailed as a .zip file.


Health app user details.png


3. The exported health data can be accessed from the export.xml file in the above-mentioned .zip folder.


The user’s personal details like date of birth, sex, blood group, skin type, height (in cm), and mass (in kg) is displayed.


Excerpt from .xml file with user details.png


The .xml file also shows the heart rate data (in count/min) or beats-per-minute (BPM), steps, distance covered (in km), basal and active energy burned (in kJ), and exercise time (in mins). The exact activity log time (creationDate), and activity start and end times (startDate, endDate) are also displayed. Towards the end of the file, an activity summary log is also included.


.xml file showing step count.png

.xml file showing heart rate data.png


The exported data in the .xml file is hard to read and analyze. Thus, it needs to be converted in to a .csv (comma separated file), which can be opened using Microsoft Excel. This allows easy data visualization and analysis. To convert the data to a .csv file, an open-source Python code from GitHub [12] was used (Code in Appendix). Ensure that the export.xml file is in the same directory as the code file.

.csv file showing heart rate data.png

.csv file showing step count data.png

For the purpose of this project, all the health data was extracted using the Apple Health App.


iCloud Backup

If the paired iPhone is unavailable, the health data can be accessed through iCloud and iTunes backup. Data from all iOS devices is automatically backed up in iCloud when a stable Wi-Fi connection is available. However, iCloud can only be accessed if the Apple ID username and password is known. In the case where the user’s credentials are unknown, a third-party software like the Elcomsoft Phone Breaker (EPPB) can be used to gain access. This software can retrieve all data backed up in the cloud by using a ‘binary authentication token’ [17] formulated by the iCloud Control Panel to acquire the account log-in credentials.


iTunes Backup

The Apple iTunes software can also be used to create a backup for all iOS devices. However, the iTunes backup is stored on an Apple Mac or a PC, and not online. If the log-in details are known, data can be directly acquired from iTunes. If not, data can be extracted by using forensic tools like UFED Touch/UFED Physical Analyser.


Chip-off

Chip-off is the least preferred method to extract data. Chip-off is the removal of the flash memory chip which stores the device’s data [18]. This invasive method allows the user to decode data in the absolute raw form from the memory, thus decoding it is quite challenging. Chip-off also requires the use of special tools to safely isolate the data chips. To read the data from the flash memory chips, small wires must be connected to certain ‘contact points on the monolithic package’s hidden ball grid array’. This process is called ‘spiderwebbing’.


Data Extraction – FitBit Alta HR


FitBit Mobile App

The FitBit Alta HR is primarily paired with an android phone for this project using the FitBit mobile app. The app dashboard displays the number of steps, heart rate, distance travelled, calories burned, and active minutes as recorded by the wearable. The app also displays the daily summary of the data in a graphical format for easy understanding.

FitBit mobile app dashboard.png

The step data for the daily summary is calculated for a 15-minute interval, whereas the heart rate data is averaged for every 5 minutes.

Daily step summary.png

Daily summary of heart rate data.png

Detailed log of heart rate data.png


FitBit Computer Interface

The same data can be accessed from a Windows/Mac computer using the FitBit computer interface, which is accessible through https://www.FitBit.com/ .

FitBit computer dashboard.png

A more detailed activity log averaged over 5 minutes can also be accessed for a particular day along with the respective heart rate and energy data for the ‘Active’ minutes. Active minutes are awarded by FitBit after at least ‘10 minutes of moderate to intense activity’ [15].

Active minutes data.png


Detailed activity log (step count).png


FitBit allows users to extract daily activity summary in the form of .csv or .xls files which can be opened through Microsoft Excel. However, no intraday data is provided.


.csv file for daily summary of exported data.png


Intraday Data Extraction using FitBit API

FitBit does not allow its users to extract second-level intraday data. The computer interface only displays steps data averaged over every 15 minutes, and heart rate data averaged over every 5 minutes. To extract intraday data, an external app was created using the web based FitBit API (Application Programming Interface). The FitBit API allows developers to register a personal app which authorises them to connect to the FitBit web app server and access the stored data. This process allowed the steps and energy data to be extracted for every minute, and the heart rate data to be extracted for every few seconds.

The process for the data extraction using the FitBit API are as follows:

1. Register a personal app on http://dev.fitbit.com.


FitBit API - personal app registration.png


2. Click on ‘OAuth 2.0 tutorial page’.


FitBit API - application.png


3. Type a ‘Redirect URL’. For this project, https://www.google.com.au was used as the Redirect URL. Select the scope of the data that needs to be accessed.


FitBit API - redirect url.png


4. Click on the authorisation URL provided and then copy the access token in the field provided.


FitBit API - authorisation url.png


5. To make a call-back request, click on ‘Send to Hurl.it’.


FitBit API - data request.png


6. According to the FitBit Web API Reference document, there are 2 request formats for retrieving data.

https://api.fitbit.com/1/user/[user-id]/activities/steps/date/[date]/[period].json

https://api.fitbit.com/1/user/[user-id]/activities/steps/date/[date]/[end-date]/[detail-level]/time/[start-time]/[end-time].json


FitBit API data parameters.JPG


Example request:

https://api.fitbit.com/1/user/-/activities/steps/date/today/1d/1min.json

FitBit API - data request url.png


Copy the request URL in the ‘Destination’ field and launch request.


The extracted data is available in a easy-to-read format, and also available in raw-data format.


Minute level data for step count.png


Other Resource Path options:

activities/calories

activities/distance

activities/floors

activities/elevation

activities/heart


7. To convert the data into a .xls or .csv file, copy the raw data into Notepad and save as a .txt file. The .txt file can then be imported to Microsoft Excel and saved as either.xls or .csv file.

FitBit - .csv file showing step count data.png


Experimentation – Apple Watch


Effect of arm movement on number of steps recorded

It was observed that steps were recorded by the Apple Watch when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. This indicates that only arm movements at a relatively higher acceleration can be registered as steps.


Minimum number of steps to register a pedometer event

To determine the minimum number of steps to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed. The steps taken by the user were manually counted and the Health app on the paired iPhone was regularly checked to see whether the activity was recorded or not. The user walked 30, 20, 15, 10, 5, and 3 steps. It was observed that a minimum of 10 steps were required to register a pedometer event in the Apple Watch, and therefore the Health App.


Minimum activity time to register a pedometer event

To determine the minimum time to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed for 20s, 10s, 5s, 3s, and 1s. The user was timed using a stop watch to an accuracy of 1ms and steps taken by the user were counted manually. It was observed that no data was recorded for 1s. However, 10 steps were recorded when the user walked for 3 seconds, thus suggesting that the user took an average of 0.33s for each step.


Effect of walking speed on the step count

It was observed that when the walking speed of the user was significantly reduced, the Watch only recorded 5 steps, while the manual step count was 10.


Pedometer interval time

The following table show the details of the data log for a pedometer event. To determine the meaning of the respective ‘Start Time’ and ‘End Time’, 5 sets of readings were taken (24hrs format) and the activity start and end times were manually recorded, and then compared to the data logged on the Health App.

Apple watch - measured and logged pedometer times.JPG

Apple watch - average lag times.JPG

From the previous table, it can be observed that on an average, there is a lag of 4.6 seconds between the measured and the logged start times. Similarly, there is lag of 17.2 seconds between the measures and the logged end times. On an average, the data is logged into the health app 43.8 seconds after the logged end time.


Effect on step count when Apple Watch is worn loosely

When the Apple Watch is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.


Effect on step count when Apple Watch is placed in user’s bag

The Watch was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via local tram. It was observed that false data was recorded by the Watch. The Watch recorded a total of 7576 steps, suggesting that readings were also taken when the user was in the tram and relatively stationary. The built-in pedometer in the paired iPhone recorded 7892 steps.


Minimum height and number of steps to record a ‘Flight of Stairs’

A flight of stairs is counted when the user gains at least 10 feet (3 metres) of elevation. On an average, 16 steps formulate 3 metres elevation gain. This is equivalent to climbing 1 floor. The Apple Watch Series 1 does not have an altimeter to measure the elevation gain. The iPhone consists of an altimeter sensor.


Forced and continuous heart rate logs

In its normal state, the Apple watch does not measure the user’s heart rate continuously. However, a heart rate measurement can be forced using the ‘Heart Rate Glance’ feature in the Watch. On an average, it takes approximately 5 seconds to obtain an initial heart rate reading.

Theoretically, the Watch is supposed to measure the user’s heart rate every 10 minutes. However, the Watch can continuously monitor and record the user’s heartbeat in ‘Workout’ mode.

Apple watch heart rate logs.JPG

The table demonstrates the heart rate logs over a period of approximately 2 hours when the user was sitting quietly. The activity start time, the data logtime, and the heart rate count are shown. The difference between the subsequent start times, and the difference between the activity start time and data log time is also calculated. Their respective average, maximum, and minimum values are also calculated. It can be observed that the average time difference between subsequent start times is approximately 7 minutes 39 seconds, whereas the average time difference between start and logging times is 8 minutes 18 seconds.


Effect of different skin colours on heart rate sensor

When the Apple Watch was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes.


Effect of tattoos on heart rate sensor

The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.


Effect of loose fitting on heart rate sensor

Stable skin-contact and a good Watch fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.



Experimentation – FitBit Alta HR


Accuracy of steps recorded

The FitBit device used continuously measures the step taken by the user. The user manually calculated the steps taken, and then compared the number with the step count recorded by the device. It was observed that the device’s step count is extensively accurate. This holds true when the user was walking briskly and running.


Effect of arm movement on number of steps recorded

It was observed that steps were continuously recorded by the FitBit device even when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. False steps were also recorded while the user was doing chores such as washing dishes, eating, typing on a computer, etc. Hence, it can be stated that any significant arm movement can be misreported by the device as physical steps taken by the user.


Minimum number of steps to register a pedometer event

The FitBit device continuously records steps. Hence, each step taken by the user is logged into the device in real time.


Minimum time between subsequent steps to register a pedometer event

The FitBit device can record a solitary step, thus, it does not need continuous movement to be able to record steps.


Effect on step count when FitBit is worn loosely

When the FitBit device is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.


Effect on step count when FitBit device is placed in user’s bag

The device was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via train. It was observed that false data was recorded by the device. The device recorded a total of 2385 steps. However, unlike the Apple Watch, no readings were recorded while the user was on the train.


False heart rate readings when the device is not worn

It was observed that several false heart rate readings were recorded even when the user was not wearing the device. If the device is completely stationary, the heart rate sensor stops after approximately 5 seconds. However, even the slightest movement near the device can trigger the sensor. If an unusually high heart rate is recorded only once, the measurement is not logged into the FitBit App.


Effect of different skin colours on heart rate sensor

When the FitBit device was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes.


Effect of tattoos on heart rate sensor

The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.


Effect of loose fitting on heart rate sensor

Stable skin-contact and a good fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.


Time required by the heart rate sensor to establish beats per minute (BPM)

To determine the time taken by the FitBit heart rate sensor to acquire a reading, the user put on the device 4 times, and recorded the time required for the device to display the heart rate using a stopwatch.

FitBit - .csv file showing step count data.png

From the table , it can be observed that it takes an average of 3.2 seconds for the FitBit device to record and display the user’s heart rate measurement.


Logging of non-periodic heartbeat measurements

FitBit’s heart rate sensor is equipped with the PurePulse technology which allows for the continuous measurement of the user’s heart rate, even when the user is sitting extremely still.


FitBit Device Syncing Logs

Unlike Apple, FitBit does not support a sync log. Only the ‘last synced’ time is displayed in both the mobile and computer apps. Although, per the FitBit database [11], the device should sync with the paired phone every 15 minutes. However, it was observed that it is not true. The sync intervals were usually between 30-90 minutes unless manually synced.


Data Records - FitBit Alta HR

FitBit - step count data.JPG


FitBit - heart rate data.JPG


FitBit - active energy data.JPG


From the figures, it can be observed that data acquired from a FitBit device is more consistent as FitBit is capable of continuously recording the user’s steps and heart rate. It can also be observed that the user’s heart rate is higher when the step count increases, and thus, the amount of active energy expenditure is also higher at that time. When the user is at rest (0 steps recorded), the active energy decreases to a nominal basal value of 4.387kJ. This value is based on the user’s BMR.



Data Records - Apple Watch

Apple watch - step count data.JPG


Apple watch - heart rate data.JPG


Apple watch - active energy data.JPG


From the table, it can be observed that despite the user’s active movement, the heart rate records are not continuous and periodic. Over a period of 10 minutes, the user’s heart rate was measured only 2 times. The user’s energy expenditure readings seem to be based on the step count rather than the heart rate of the user. It can also be observed that the number of data points is significantly less than what is observed in the graphs obtained from the FitBit data. However, the biometrics tend to follow a similar trend as observed in the data acquired from FitBit. The peak heart rate measurement observed at 16:07:00 (130 bpm) could be considered a false value.


Post-mortem Analysis

Based on the data sets acquired from the FitBit and Apple devices, a synthetic data set was created to depict a possible murder scenario. The timeline follows a period of 1 hour, and the victim’s steps count, heart rate, and active energy data is considered.

Victim - step count data.JPG


Victim - heart rate data.JPG


Victim - active energy data.JPG


In the figures, 5 specific incidents have been highlighted which are crucial for the timeline of the case.

1. Defence (16:45:00)

• 9 false steps recorded due to defensive arm movement. As mentioned in Section 5.4.2, significant arm movement can be recorded as false steps by the device.

• Slightly elevated heart rate.

• Average level of active energy burned.


2. Attack (16:46:00)

• High heart rate following defensive action and stress

• High level of active energy burned.

• Subsequent variability in heart rate (from 16:46:00 to 16:57:00) due to possible cardiac arrest.


3. Trauma (16:47:00)

• Fluctuating heart rate.

• Low energy – immediate effect of the traumatic injury. As the victim loses blood, the body tends to cool down. There is a subsequent increase in the active energy as the body is attempting to maintain a constant core body temperature.


4. Last recorded heart rate (16:57:00)

• Low heart rate measurement.

• Low active energy. Active energy continuously decreases in the following minutes.


5. Death (17:01:00)

• No heart rate measurement.

• Lowest active energy measurement.

Conclusion

In majority of criminal investigations, it is challenging to establish an incident timeline, and to estimate the victim’s time of death. The project aimed to determine the extent to which fitness devices can be used as legitimate digital evidence in criminal investigations. The personal health data recorded on these devices can be used to perform post-mortem analysis of exercise-induced death or murder. The Apple Series 1 Watch and FitBit Alta HR were used for analysis. Phase 1 of the project focused on the techniques utilised to extract the user’s health data from the devices under consideration and their respective cloud based servers. To establish the accuracy of the measurements, and limitations of the data logs, certain experiments were conducted in Phase 2 of the project. For the scope of the project, the biometrics considered primarily were the user’s step count, heart rate, and active energy expenditure.

The following conclusions were derived:

1. Certain data logging delays were observed for the Apple Watch. The timestamps on the data logs can possibly be used to establish whether the user’s phone was turned on and had a wireless connection with the fitness device.

2. The heart rate data acquired from dark-skinned and tattooed users cannot be used as a reliable source of information for criminal investigations.

3. Several false steps can be recorded when there is significant arm movement, even when the user is not physically taking any steps, for instance, defensive arm movement of the victim. False steps can also be recorded when the user is in a moving vehicle, or when the device is in the user’s bag or pocket.

4. An incident timeline can be created by analysing the trends in the user’s step count, heart rate and energy expenditure.

5. The last heart rate record does not necessarily signify death. A more accurate time of death can be established using the active energy burned by the user.


Future Work

The scope of the project can be further extended to explore and relate the health data records with the device’s GPS data to establish a more accurate timeline and whereabouts of the victim or potential suspects. Presently, certain smartphones are equipped with advanced health apps that can record the user’s step count, heart rate, blood oxygen levels, and even stress levels. Further research can be undertaken to establish the accuracy of those records. Other wearable fitness devices like chest straps, headbands, and ‘smart’ shoes can also be investigated to determine their reliability as forensic evidence. In cases where access to the device’s cloud servers is limited, physical chip-off analysis can also be conducted to determine which data can be recovered and analysed for investigative purposes. The detailed comparison of the features of top fitness devices in the market is included in the Appendix.



Appendices

Code to convert raw Apple Health app data to .csv file

As stated in previous sections, an open-source Python code was used from GitHub to convert the raw data extracted from the Apple Health app to a .csv file. The code is as follows:

Applehealthdata-001.jpg


Applehealthdata-002.jpg


Applehealthdata-003.jpg


Applehealthdata-004.jpg


Applehealthdata-005.jpg

References

[1] J. Stables, "Best fitness trackers 2017: FitBit, Garmin, Misfit, Withings and more", Wearable, 2017. [Online]. Available: https://www.wareable.com/fitness-trackers/the-best-fitness-tracker. [Accessed: 15- Apr- 2017].

[2] "Articles", Practicalhomicide.com, 2017. [Online]. Available: http://www.practicalhomicide.com/Research/LOmar2007.htm. [Accessed: 16- Apr- 2017].

[3] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed:21-Oct- 2017].

[4] A. Robinson, "Heart Rate Vs. Calories Burned", LIVESTRONG.COM,2017. [Online]. Available: https://www.livestrong.com/article/39443-heart-rate-vs.-calories-burned/. [Accessed: 21- Oct- 2017].

[5] "Apple Watch Teardown - iFixit", Ifixit.com, 2017. [Online]. Available: https://www.ifixit.com/Teardown/Apple+Watch+Teardown/40655. [Accessed: 15- Apr- 2017].

[6] "Your heart rate. What it means, and where on Apple Watch you’ll find it.", Apple Support, 2017. [Online]. Available: https://support.apple.com/en-au/HT204666. [Accessed: 18- Apr- 2017].

[7]"Fitzpatrick scale", En.wikipedia.org, 2017. [Online]. Available: https://en.wikipedia.org/wiki/Fitzpatrick_scale. [Accessed: 31- May- 2017].

[8]"Apple engineer explains how Apple decides which HealthKit data types to add", MobiHealthNews, 2017. [Online]. Available: http://www.mobihealthnews.com/44413/apple-engineer-explains-how-apple-decides-which-healthkit-data-types-to-add. [Accessed: 31- May- 2017].

[9]"Shop FitBit Alta HR", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/shop/altahr. [Accessed: 20- May- 2017].

[10]"FitBit PurePulse™ Continuous Wrist-Based Heart Rate", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/purepulse. [Accessed: 20- May- 2017].

[11]"FitBit's new Alta HR activity tracker brings a heart rate monitoring update to the original", The Verge, 2017. [Online]. Available: https://www.theverge.com/2017/3/6/14823688/FitBit-new-alta-hr-fitness-tracker-heart-rate-sleep-tracking. [Accessed: 22- May- 2017].

[12]"FitBit Alta Teardown | Chipworks", Chipworks.com, 2017. [Online]. Available: http://www.chipworks.com/about-chipworks/overview/blog/FitBit-alta-teardown. [Accessed: 01- Jun- 2017].

[13] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed: 27- Oct- 2017].

[14]"How do FitBit trackers sync their data?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1877/?l=en_US&c=Topics%3ASyncing&fs=Search&pn=1. [Accessed: 23- May- 2017].

[15] Guthrie, R. (2017). The future of biometric tracking will make step counters look like antiques. [online] Digital Trends. Available at: https://www.digitaltrends.com/health-fitness/future-of-biometrics-beyond-the-wrist-tracker/.

[16] "tdda/applehealthdata", GitHub, 2017. [Online]. Available: https://github.com/tdda/applehealthdata. [Accessed: 29- May- 2017].

[17] "Advanced mobile forensics: iOS (iPhone and iPad), Windows Phone and BlackBerry 10", Elcomsoft.com, 2017. [Online]. Available: https://www.elcomsoft.com/eppb.html. [Accessed: 19- Apr- 2017].

[18] "Chip-Off Forensics Services - Gillware Digital Forensics", Gillware Digital Forensics, 2017. [Online]. Available: https://www.gillware.com/forensics/chip-off-forensics-services. [Accessed: 18- Apr- 2017].

[19] "What are active minutes?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1379/?l=en_US&c=Topics%3ADashboard&fs=Search&pn=1. [Accessed: 01- Jun- 2017].

[20] "FitBit Web API Basics — FitBit Web API Docs", Dev.FitBit.com, 2017. [Online]. Available: https://dev.FitBit.com/docs/basics/. [Accessed: 30- May- 2017].

Retrieved from "https://projectswiki.eleceng.adelaide.edu.au/projects/index.php?title=Projects:2017s1-165_Forensic_Investigation_of_Fitness_Devices&oldid=9303"