Projects:2017s1-165 Forensic Investigation of Fitness Devices
Contents
Project Group Members
Sanjam Kohli
Yuan Li
Project Supervisor
Dr. Matthew Sorell
Introduction
High-tech wearable devices have always been objects of interest in science fiction. From cheap plastic activity bands or rudimentary watches, these gadgets have evolved into elegantly designed devices that can provide greater connectivity, location services, and more importantly, incredible insights into people’s health. These nifty instruments have the capability to monitor a consumer’s heart rate, sleep patterns, and even one’s blood oxygen levels. The smartwatch and fitness band market is dominated by global technology leaders Apple, Samsung, and FitBit. More than 50% of the Australians now own a smartwatch [1]. As these devices are being adopted by a growing number of users, there is an increasing potential for them to become a haven for digital evidence in criminal investigations.
Aim
The project aims to explore the use of wearable fitness devices as forensic evidence, and to establish movement and activities of victims or suspects involved in a homicide investigation. This was achieved by analysing the heart beat and activity records which can be extracted and analysed from the Apple Watch and a FitBit device or their respective paired phones.
Motivation and Significance
A victim’s time of death is crucial to every criminal investigation. Currently, it is extremely challenging to determine the time of death in a homicide investigation using conventional methods. The project attempts to develop a mechanism to establish a more accurate timeline of the incident and a precise time of death using the heart beat and activity logs extracted from fitness devices. The method devised could further assist the South Australian Police (SAPOL), and other law enforcing agencies in future investigations.
Technical Background
Determining the time of death
Estimating the time of death is very crucial to a homicide investigation. It is a critical element of the case timeline. A specific and accurate time of death can corroborate any statements given by suspects in a crime. Despite years of research by forensic experts, no conclusive method has been devised to estimate a victim’s time of death. Presently, the estimation is based on various case specific factors and pathological changes in the human body like changes in body temperature, muscle stiffness, and lividity. In the absence of any witnesses, the complexity of this process increases. By using the current methods, the time of death is usually placed within a range of hours. These processes are highly subjective to errors, and thus it is ‘utterly impossible’ to fix an exact time [2].
Resting and Active Calories
The Basal Metabolic Rate (BMR) is defined as the amount of energy (measured in kilojoules, kJ) burned at rest [3]. BMR is calculated using one’s biometrics like weight, height, age, and sex. Resting calories signify the caloric base burn rate, and are calculated by using the individual’s BMR. Active calories are the calories that are burned due to additional activity.
1 kilocalorie (kCal) = 4.184 kilojoules (kJ)
Total Energy (kJ) = Basal Energy (kJ) + Active Energy (kJ)
Calorie expenditure is relatively linear to heart rate for an average individual, provided that the individual’s heart rate remains within the safe range of 90-150bpm [4].
Biometrics
Biometrics is defined as the ‘the science and technology of analyzing biological data’ [15]. For the purpose of this project, the following biometrics are considered:
1. Heartrate (beats/min)
2. Steps (steps/min)
3. Total Energy (kJ) = Resting Energy (kJ) + Active Energy (kJ)
Apple Watch Series 1
Device Specifications
The specifications [5] for the Apple Watch used in this project are as follows:
1. 38mm (vertically)
2. 290ppi screen
3. Custom designed Apple S1 SiP (system in package) chip.
4. NFC +WiFi 802.11b’g’n + Bluetooth 4.0
5. 8 GB onboard storage
6. Sensors: Heart rate monitor, gyroscope, accelerometer
Photoplethysomography
The Apple Watch uses the concept of photoplethysomography (PPG) to measure the user’s heart rate [6]. The technology uses a simple principle of light absorption. The red color of the blood is due to the reflection of the red light, and the absorption of the green light. The Watch has infrared and green LED lights which are paired with light sensitive photodiodes (Fig 2.1). These lights are flashed at a high frequency (>400 Hz), to measure the blood flow in the user’s wrist. When the heart beats, there is an increase in the blood flow in the wrist, thus resulting in an increase in the rate of green light absorption, which is then measured by the photodiodes. The LED brightness and sampling rate can be adjusted automatically by the Watch in low signal level conditions.The heart rate data is transmitted to the Health app every 10 minutes on average through a stable Bluetooth or Wi-Fi connection. The data can then be compiled in a graph for users to study.
Steps Count
The Apple Watch has an accelerometer sensor which acts as a built-in step counter or pedometer. The steps are counted based on the height and stride length of the user.
Calorie Count
The Apple Watch measures the basal and active calories burnt using the biometrics (sex, weight, height) entered by the user, the user’s heart rate, and average human statistics. The activity being performed by the user is identified by the accelerometer, and is also considered for calculating energy expenditure.
Apple Watch memory and storage
The device consists of 512 MB of dynamic RAM, and 8 GB of flash memory. The Watch uses an HFS+ (hierarchical file system) created by Apple Inc., which has limited storage capacity than a device using removable SD cards
Device Syncing
The Apple Watch does not consist of a physical diagnostic port for users to transmit their data between devices. Thus, all the data is transferred and backed up in a companion iPhone by using either a Wi-Fi or Bluetooth connection. Once both the devices are in range, a stable ‘data stream’ is established. All the data is also backed up automatically in iCloud. The heart beat logs acquired by the Watch are sent to the paired iPhone. Using the built-in Health app, the user can access this data
FitBit Alta HR
Device Specifications
The hardware specifications of the device used for the project are as follow [9]:
1. 15mm wide
2. OLED tap display
3. Bluetooth 4.0
4. Sensors: Optical heart rate tracker, 3-axis accelerometer, vibration motor
5. Memory: 7 days of detailed data storage (minute by minute), daily summaries for 30 days
PurePulse
The PurePulse [10] is the continuous and automatic heart rate tracker used by FitBit. The sensor uses the principle of photoplethysomography, like the Apple Watch (Section 2.2.2). However, unlike the Apple Watch, the PurePulse sensor is capable of continuously monitoring the user’s heart rate during an activity and the resting heart rate too.
Step Count
FitBit consists of a 3-axis accelerometer which is used to detect motion. The sensor calculates the step count, distance, and calories burnt based on the duration, frequency, and intensity of the activity [13].
Calorie Count
FitBit considers the user’s biometrics like weight, age, height, and sex to determine the user’s BMR, and thereby calculate the energy expenditure based on the user’s activity.
Device Memory and Storage
The Alta HR can store detailed minute-by-minute data for up to 7 days in the device. Daily summaries can be stored for up to 30 days in the device if the data is not synced to cloud [9]. For investigative purposes, the device data needs to be exported within 4-5 days to ensure the details are preserved and data is not lost.
Device Syncing
The Alta HR uses Bluetooth Low Energy (LE) wireless technology and an internet connection (Wi-Fi/mobile data) to sync with mobile devices and computers. The device syncing range is up to 6.1 metres [9]. The device can also be synced with a computer using the dongle. If ‘All-day’ syncing is turned on, the devices should automatically sync every 15-30 minutes. Manual syncing is also available [14].
Related Work
There is no decisive method to estimate the time of death in a murder case. The project aims to devise a technique to accurately calculate the victim’s time of death by analysing the user’s heart rate data logs acquired by the Apple Watch. There have been several studies and research for using wearable and mobile devices for forensic investigations, however these are mostly limited to device imaging and data examination of call, text, and location logs. No known research has been undertaken earlier to estimate a murder victim’s time of death using a fitness device.
Methodology
Data Extraction – Apple Watch
Data extraction from the Apple Watch can be done by 4 ways:
1. Apple Health App (requires user’s phone password)
2. iCloud backup (requires user’s Apple ID)
3. iTunes backup (requires user’s phone password)
4. Chip-off
Health App
iCloud Backup
If the paired iPhone is unavailable, the health data can be accessed through iCloud and iTunes backup. Data from all iOS devices is automatically backed up in iCloud when a stable Wi-Fi connection is available. However, iCloud can only be accessed if the Apple ID username and password is known. In the case where the user’s credentials are unknown, a third-party software like the Elcomsoft Phone Breaker (EPPB) can be used to gain access. This software can retrieve all data backed up in the cloud by using a ‘binary authentication token’ [17] formulated by the iCloud Control Panel to acquire the account log-in credentials.
iTunes Backup
The Apple iTunes software can also be used to create a backup for all iOS devices. However, the iTunes backup is stored on an Apple Mac or a PC, and not online. If the log-in details are known, data can be directly acquired from iTunes. If not, data can be extracted by using forensic tools like UFED Touch/UFED Physical Analyser.
Chip-off
Chip-off is the least preferred method to extract data. Chip-off is the removal of the flash memory chip which stores the device’s data [18]. This invasive method allows the user to decode data in the absolute raw form from the memory, thus decoding it is quite challenging. Chip-off also requires the use of special tools to safely isolate the data chips. To read the data from the flash memory chips, small wires must be connected to certain ‘contact points on the monolithic package’s hidden ball grid array’. This process is called ‘spiderwebbing’.
Data Extraction – FitBit Alta HR
FitBit Mobile App
The FitBit Alta HR is primarily paired with an android phone for this project using the FitBit mobile app. The app dashboard displays the number of steps, heart rate, distance travelled, calories burned, and active minutes as recorded by the wearable. The app also displays the daily summary of the data in a graphical format for easy understanding.
FitBit Computer Interface
Intraday Data Extraction using FitBit API
Experimentation – Apple Watch
Effect of arm movement on number of steps recorded
It was observed that steps were recorded by the Apple Watch when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. This indicates that only arm movements at a relatively higher acceleration can be registered as steps.
Minimum number of steps to register a pedometer event
To determine the minimum number of steps to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed. The steps taken by the user were manually counted and the Health app on the paired iPhone was regularly checked to see whether the activity was recorded or not. The user walked 30, 20, 15, 10, 5, and 3 steps. It was observed that a minimum of 10 steps were required to register a pedometer event in the Apple Watch, and therefore the Health App.
Minimum activity time to register a pedometer event
To determine the minimum time to register a pedometer event, the user was made to walk on a smooth carpeted surface with a normal walking speed for 20s, 10s, 5s, 3s, and 1s. The user was timed using a stop watch to an accuracy of 1ms and steps taken by the user were counted manually. It was observed that no data was recorded for 1s. However, 10 steps were recorded when the user walked for 3 seconds, thus suggesting that the user took an average of 0.33s for each step.
Effect of walking speed on the step count
It was observed that when the walking speed of the user was significantly reduced, the Watch only recorded 5 steps, while the manual step count was 10.
Pedometer interval time
Effect on step count when Apple Watch is worn loosely
When the Apple Watch is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.
Effect on step count when Apple Watch is placed in user’s bag
The Watch was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via local tram. It was observed that false data was recorded by the Watch. The Watch recorded a total of 7576 steps, suggesting that readings were also taken when the user was in the tram and relatively stationary. The built-in pedometer in the paired iPhone recorded 7892 steps.
Minimum height and number of steps to record a ‘Flight of Stairs’
A flight of stairs is counted when the user gains at least 10 feet (3 metres) of elevation. On an average, 16 steps formulate 3 metres elevation gain. This is equivalent to climbing 1 floor. The Apple Watch Series 1 does not have an altimeter to measure the elevation gain. The iPhone consists of an altimeter sensor.
Forced and continuous heart rate logs
Effect of different skin colours on heart rate sensor
When the Apple Watch was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes.
Effect of tattoos on heart rate sensor
The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.
Effect of loose fitting on heart rate sensor
Stable skin-contact and a good Watch fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.
Experimentation – FitBit Alta HR
Accuracy of steps recorded
The FitBit device used continuously measures the step taken by the user. The user manually calculated the steps taken, and then compared the number with the step count recorded by the device. It was observed that the device’s step count is extensively accurate. This holds true when the user was walking briskly and running.
Effect of arm movement on number of steps recorded
It was observed that steps were continuously recorded by the FitBit device even when the user held its arm in a horizontal position and restricted its movement completely. This suggests that the steps are recorded when the device records certain acceleration. It was also observed that when the user was stationary, and only swinging its arms, false steps were recorded. False steps were also recorded while the user was doing chores such as washing dishes, eating, typing on a computer, etc. Hence, it can be stated that any significant arm movement can be misreported by the device as physical steps taken by the user.
Minimum number of steps to register a pedometer event
The FitBit device continuously records steps. Hence, each step taken by the user is logged into the device in real time.
Minimum time between subsequent steps to register a pedometer event
The FitBit device can record a solitary step, thus, it does not need continuous movement to be able to record steps.
Effect on step count when FitBit is worn loosely
When the FitBit device is worn loosely on the wrist, no discrepancies were recorded in the number of steps counted.
Effect on step count when FitBit device is placed in user’s bag
The device was removed from the user’s wrist and placed in a bag pack. The user then proceeded to walk, and commute via train. It was observed that false data was recorded by the device. The device recorded a total of 2385 steps. However, unlike the Apple Watch, no readings were recorded while the user was on the train.
False heart rate readings when the device is not worn
It was observed that several false heart rate readings were recorded even when the user was not wearing the device. If the device is completely stationary, the heart rate sensor stops after approximately 5 seconds. However, even the slightest movement near the device can trigger the sensor. If an unusually high heart rate is recorded only once, the measurement is not logged into the FitBit App.
Effect of different skin colours on heart rate sensor
When the FitBit device was tested on a dark-skinned user, it was observed that the heart rate readings were lower than average. This could be because of the high absorption, and low reflection of the green light emitted from the heart rate sensor. Thus, it can be stated that the data recorded for dark skinned users is not accurate, and hence, not reliable for investigative purposes.
Effect of tattoos on heart rate sensor
The extent of the effect of tattoos on the data recorded by the heart rate sensor depends on the type of ink and colour of the tattoo. Dark tattoo ink tends to absorb more green light than light coloured ink. As the light reflection rate is less for darker colours, the heart rate recorded is lower than average. Hence, it can be stated that the heart rate data recorded for tattooed subjects is not accurate, and is unreliable for investigative purposes.
Effect of loose fitting on heart rate sensor
Stable skin-contact and a good fit is required for the device to measure the user’s heart rate accurately. It was observed that when the device was worn loosely, the heart rate readings of the user were higher than average as the rate of light absorption by the user’s skin is significantly reduced.
Time required by the heart rate sensor to establish beats per minute (BPM)
Logging of non-periodic heartbeat measurements
FitBit’s heart rate sensor is equipped with the PurePulse technology which allows for the continuous measurement of the user’s heart rate, even when the user is sitting extremely still.
FitBit Device Syncing Logs
Unlike Apple, FitBit does not support a sync log. Only the ‘last synced’ time is displayed in both the mobile and computer apps. Although, per the FitBit database [11], the device should sync with the paired phone every 15 minutes. However, it was observed that it is not true. The sync intervals were usually between 30-90 minutes unless manually synced.
Data Records - FitBit Alta HR
Data Records - Apple Watch
Post-mortem Analysis
Based on the data sets acquired from the FitBit and Apple devices, a synthetic data set was created to depict a possible murder scenario. The timeline follows a period of 1 hour, and the victim’s steps count, heart rate, and active energy data is considered.
In figure ????, 5 specific incidents have been highlighted which are crucial for the timeline of the case.
1. Defence (16:45:00)
• 9 false steps recorded due to defensive arm movement. As mentioned in Section 5.4.2, significant arm movement can be recorded as false steps by the device.
• Slightly elevated heart rate.
• Average level of active energy burned.
2. Attack (16:46:00)
• High heart rate following defensive action and stress
• High level of active energy burned.
• Subsequent variability in heart rate (from 16:46:00 to 16:57:00) due to possible cardiac arrest.
3. Trauma (16:47:00)
• Fluctuating heart rate.
• Low energy – immediate effect of the traumatic injury. As the victim loses blood, the body tends to cool down. There is a subsequent increase in the active energy as the body is attempting to maintain a constant core body temperature.
4. Last recorded heart rate (16:57:00)
• Low heart rate measurement.
• Low active energy. Active energy continuously decreases in the following minutes.
5. Death (17:01:00)
• No heart rate measurement.
• Lowest active energy measurement.
Conclusion
In majority of criminal investigations, it is challenging to establish an incident timeline, and to estimate the victim’s time of death. The project aimed to determine the extent to which fitness devices can be used as legitimate digital evidence in criminal investigations. The personal health data recorded on these devices can be used to perform post-mortem analysis of exercise-induced death or murder. The Apple Series 1 Watch and FitBit Alta HR were used for analysis. Phase 1 of the project focused on the techniques utilised to extract the user’s health data from the devices under consideration and their respective cloud based servers. To establish the accuracy of the measurements, and limitations of the data logs, certain experiments were conducted in Phase 2 of the project. For the scope of the project, the biometrics considered primarily were the user’s step count, heart rate, and active energy expenditure.
The following conclusions were derived:
1. Certain data logging delays were observed for the Apple Watch. The timestamps on the data logs can possibly be used to establish whether the user’s phone was turned on and had a wireless connection with the fitness device.
2. The heart rate data acquired from dark-skinned and tattooed users cannot be used as a reliable source of information for criminal investigations.
3. Several false steps can be recorded when there is significant arm movement, even when the user is not physically taking any steps, for instance, defensive arm movement of the victim. False steps can also be recorded when the user is in a moving vehicle, or when the device is in the user’s bag or pocket.
4. An incident timeline can be created by analysing the trends in the user’s step count, heart rate and energy expenditure.
5. The last heart rate record does not necessarily signify death. A more accurate time of death can be established using the active energy burned by the user.
Future Work
The scope of the project can be further extended to explore and relate the health data records with the device’s GPS data to establish a more accurate timeline and whereabouts of the victim or potential suspects. Presently, certain smartphones are equipped with advanced health apps that can record the user’s step count, heart rate, blood oxygen levels, and even stress levels. Further research can be undertaken to establish the accuracy of those records. Other wearable fitness devices like chest straps, headbands, and ‘smart’ shoes can also be investigated to determine their reliability as forensic evidence. In cases where access to the device’s cloud servers is limited, physical chip-off analysis can also be conducted to determine which data can be recovered and analysed for investigative purposes. The detailed comparison of the features of top fitness devices in the market is included in the Appendix.
Appendices
References
[1] J. Stables, "Best fitness trackers 2017: FitBit, Garmin, Misfit, Withings and more", Wearable, 2017. [Online]. Available: https://www.wareable.com/fitness-trackers/the-best-fitness-tracker. [Accessed: 15- Apr- 2017].
[2] "Articles", Practicalhomicide.com, 2017. [Online]. Available: http://www.practicalhomicide.com/Research/LOmar2007.htm. [Accessed: 16- Apr- 2017].
[3] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed:21-Oct- 2017].
[4] A. Robinson, "Heart Rate Vs. Calories Burned", LIVESTRONG.COM,2017. [Online]. Available: https://www.livestrong.com/article/39443-heart-rate-vs.-calories-burned/. [Accessed: 21- Oct- 2017].
[5] "Apple Watch Teardown - iFixit", Ifixit.com, 2017. [Online]. Available: https://www.ifixit.com/Teardown/Apple+Watch+Teardown/40655. [Accessed: 15- Apr- 2017].
[6] "Your heart rate. What it means, and where on Apple Watch you’ll find it.", Apple Support, 2017. [Online]. Available: https://support.apple.com/en-au/HT204666. [Accessed: 18- Apr- 2017].
[7]"Fitzpatrick scale", En.wikipedia.org, 2017. [Online]. Available: https://en.wikipedia.org/wiki/Fitzpatrick_scale. [Accessed: 31- May- 2017].
[8]"Apple engineer explains how Apple decides which HealthKit data types to add", MobiHealthNews, 2017. [Online]. Available: http://www.mobihealthnews.com/44413/apple-engineer-explains-how-apple-decides-which-healthkit-data-types-to-add. [Accessed: 31- May- 2017].
[9]"Shop FitBit Alta HR", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/shop/altahr. [Accessed: 20- May- 2017].
[10]"FitBit PurePulse™ Continuous Wrist-Based Heart Rate", FitBit.com, 2017. [Online]. Available: https://www.FitBit.com/au/purepulse. [Accessed: 20- May- 2017].
[11]"FitBit's new Alta HR activity tracker brings a heart rate monitoring update to the original", The Verge, 2017. [Online]. Available: https://www.theverge.com/2017/3/6/14823688/FitBit-new-alta-hr-fitness-tracker-heart-rate-sleep-tracking. [Accessed: 22- May- 2017].
[12]"FitBit Alta Teardown | Chipworks", Chipworks.com, 2017. [Online]. Available: http://www.chipworks.com/about-chipworks/overview/blog/FitBit-alta-teardown. [Accessed: 01- Jun- 2017].
[13] "Metabolism", Betterhealth.vic.gov.au, 2017. [Online]. Available: https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/metabolism. [Accessed: 27- Oct- 2017].
[14]"How do FitBit trackers sync their data?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1877/?l=en_US&c=Topics%3ASyncing&fs=Search&pn=1. [Accessed: 23- May- 2017].
[15] Guthrie, R. (2017). The future of biometric tracking will make step counters look like antiques. [online] Digital Trends. Available at: https://www.digitaltrends.com/health-fitness/future-of-biometrics-beyond-the-wrist-tracker/.
[16] "tdda/applehealthdata", GitHub, 2017. [Online]. Available: https://github.com/tdda/applehealthdata. [Accessed: 29- May- 2017].
[17] "Advanced mobile forensics: iOS (iPhone and iPad), Windows Phone and BlackBerry 10", Elcomsoft.com, 2017. [Online]. Available: https://www.elcomsoft.com/eppb.html. [Accessed: 19- Apr- 2017].
[18] "Chip-Off Forensics Services - Gillware Digital Forensics", Gillware Digital Forensics, 2017. [Online]. Available: https://www.gillware.com/forensics/chip-off-forensics-services. [Accessed: 18- Apr- 2017].
[19] "What are active minutes?", FitBit Help, 2017. [Online]. Available: https://help.FitBit.com/articles/en_US/Help_article/1379/?l=en_US&c=Topics%3ADashboard&fs=Search&pn=1. [Accessed: 01- Jun- 2017].
[20] "FitBit Web API Basics — FitBit Web API Docs", Dev.FitBit.com, 2017. [Online]. Available: https://dev.FitBit.com/docs/basics/. [Accessed: 30- May- 2017].
