Main Body

Chapter 11: The smart home

The homes in which we live are being transformed into residences that incorporate the latest technology. As time evolves, there are few appliances or functions involving the operation and maintenance of a home that do not now have a technology component available. Lighting, appliances, temperature, security, and home functionality are capable of being fully automated and designed to be connected to the internet, making the home a vastly different environment to that which existed in previous decades. With the advent of Artificial Intelligence, homes are becoming even more connected with more functionality.

Example:

Whereas once a person would check if the doors are locked, close the windows, and turn off lights and ceiling fans before they went to bed, this can now all be accomplished by a single voice command such as “Goodnight Google”.

Technology is being integrated into many aspects of our lives, and consumers are embracing technology in return. We are approaching the point in our lives where there is little a connected consumer does in society, or their homes that is not captured and stored by technology. Whether or not this is a positive feature of our technologically advanced lifestyle is up to each individual to assess the convenience and functionality of the technology against their privacy desires and concerns.

To a detective or civilian investigator, the residence of a person of interest may be a particularly important location for examination and evidence capture. For example, a person who has unexpectedly gone missing from their home may have left evidence of their disappearance on computers or other connected devices including showing the exact time of their leaving their home and whether it was with another person, who may also be identifiable. The smart home is likely to be of major assistance to the investigator who finds this place to be a crime scene or is seeking evidence of activity within it. As technology evolves, devices that have historically been stand-alone devices such as thermostats, windows, stoves, toasters and refrigerators have become glorified computers with the original function being operated by technology. Smart appliances record and store data so an investigator may see these devices as relevant to being included in the scene investigation as the laptop or desktop computer.

The smart home may provide a full audit of activities within a crime scene where the detective’s task of recreating events leading up to, during and after the event is recorded and stored by highly accurate digital devices. This may provide a forensic trail of activities from which the detective may build their timeline of events and schedule of suspects. Items in the smart home take the guesswork out of understanding the crime scene and the person of interest’s involvement.

An AI generated image of a smart house connected to cars and multiple devices.
A smart house connected to cars and multiple devices. Image created by author using DALL-E 2 AI.

So, what is a smart home?

“A smart home is a contemporary application of ubiquitous computing that incorporates intelligence into dwellings management and operation for comfort, healthcare, safety security and energy conservation.” [1]

A second interpretation is:

“The integration of technology and services through home networking for a better quality of living.” [2]

To be a smart home, the premises must do more than merely record data as it is generated. The smart home must analyse the data generated and collected and be capable of making decisions without the interaction of a user. It must have a form of intelligence where the combined technology may predict a user’s intent and act pre-emptively such as identifying that a person is walking through a house and turning the required lights on.[3] This means the smart home is not a passive environment but is continually learning about the people and activities within it and attempting to build the knowledge to anticipate the activities before they occur. In other words, it can learn about the environment it is operating in, build up a database of knowledge and make decisions without a user’s direct intervention.

Devices within a smart home can communicate with each other and make predictions based on acquired knowledge of the premises, occupants and activities.[4] Many devices need to be paired to a companion client, such as a laptop computer or home assistant such as Amazon Alexa to manage the smart home environment.[5]

It is worth noting that not all devices may be connected to a single manager/controller device as connectiveness can depend on the skill set or desires of the owner. The controller device could be a source of identifying smart devices within the location and also be its own source of evidence.

Smart homes are made up of devices and sensors, appliances and metering services. [6] They may track behaviour, and activities within the premises, detect an event that occurs in real time, make predictions based on accumulated knowledge, operate within designated rules and adhere to privacy and security policies.[7]

There are many reasons why a person may want a smart home full of the latest technology that monitors and records many aspects of their lives. Some reasons include safety, security, convenience, the management of electricity, entertainment, personal lifestyle and the ability for interaction between the user and the home.[8] People also assess the items and determine their ease of use and how useful the device may be.[9]

Another benefit of a smart home is for those who have physical disabilities or health issues. The smart home can anticipate their requirements and build a picture of their daily routine, identifying deviations from the occupant’s routine that may indicate illness or injury. This can lead to automated medical monitoring of alerts and triage where required.[10] It also helps people living with health or mobility issues to be as independent as possible whilst retaining the ability to communicate with their family and support networks as required.

Smart homes use a controller such as a laptop computer to integrate the multitude of home automation systems. This is commonly called the Companion Device. The devices have the capability to be interacted with by the residents of the premises during set up and ongoing maintenance. Once set up the controller is left to perform their duties autonomously and can also be voice-controlled.[11] Some devices within a smart home will be connected to the controller, however, some may be connected via a smart device such as an app or a person’s phone or smart speaker.[12]

Many homes today are being built with the intention of being a smart, fully connected home.[13] Existing homes are also being retrofitted to incorporate smart technology. With the wireless technology available, there is no need for many devices to be connected to the controller by cables, meaning installation is easy, quick and convenient for the user.[14] Also, devices can be replaced and upgraded with a minimum of inconvenience.

Homes can also be controlled through a smart speaker or a smart hub. Although there are similarities in these products, they are different in nature.

Lee, Zimmerman and Schaub define a smart speaker as:

“Voice-controlled mobile devices which use artificial intelligence and natural language processing to facilitate a variety of functional and hedonic tasks, such as playing music, setting reminders, and retrieving information.” [15]

The technology website TechTarget[16] defines smart hubs as:

“The smart home hub is a hardware device that acts as the central point of the smart home system and is able to sense, process data and communicate wirelessly. It combines all of the disparate apps into a single smart home app that can be controlled remotely by homeowners.”[17]

Technology is becoming prevalent within the home where it is acknowledged that criminal activity may take place. The detective investigating a suspect crime within the smart home has all this technology available to assist them in identifying the required facts of the matter under investigation.

United Kingdom lawyer Clive Halperin states:

“I think IoT (Internet of Things) devices will help generate a lot more evidence. When things are constantly connected to the internet, it’s going to provide a huge amount of data for crime solving. People will be constantly leaving electronic trails wherever they go.” [18]

One area of consideration any investigation must address is attribution.[19] That is, smart devices in homes collect a huge amount of data, and the investigator needs to attribute which company device within the environment creates the activity to generate the smart device activity and subsequent logs, and also, which individual within the house generated the recorded activity.

What smart devices may be found in a home?

Smart technology for a domestic residence is expanding with new devices and features becoming available at regular intervals.

Examples of smart home technology include:

  • Air conditioning.
  • Coffee machines.
  • Climate control.
  • Curtains, blinds and drapes.
  • Door locks including garage doors.
  • Dryers.
  • Electricity.
  • Floor heating.
  • Furniture.
  • Garage door.
  • Gas heating and cooking.
  • Home assistants.
  • Hub.
  • Lighting.
  • Locks.
  • Medical devices.
  • Motion detectors.
  • Ovens and cookers.
  • Power plugs.
  • Refrigerators.
  • Security including cameras.
  • Smoke alarm.
  • Speakers.
  • Television.
  • Thermostats.
  • Washing machines.
  • Windows.

Numerous devices fit within these categories, and some may cross over into different categories. Within these categories are many sub-categories, such is the nature of the numerous smart appliances being presented in the marketplace.

To the investigator, it is important to extensively search for the many items of technology in and near a crime scene and understand they are designed to collect data on the activities of people within the boundaries of their operation. In some instances, such as with the home hub, data is obtained from multiple sources. That is within the hub itself, the connected cloud account and the technology that connects to the hub.

This data becomes available to the investigator who has learnt where to look, how to recognise certain data as potential evidence and what it can tell them about the scene and its activities.

Analysis of smart devices

In this section, we shall examine a selection of smart devices which may be located within a residence. It is to be once again noted that technology is constantly changing, and the evidence located within this section may expand over time as technology is upgraded by the manufacturer. Alternatively, the evidence that was available today may be unavailable when the next update of that technology is pushed onto devices or vice versa.

Smart assistants/speakers

Home assistants have become the smart home device of choice for many consumers. They provide the home user with the potential to control aspects of their home environment, control music choices and obtain information online via voice command. They can act as a hub or link to compatible smart devices within the house.  Major suppliers of smart speakers include Apple, Amazon and Google.

For example, Michael Palermo of Amazon identifies an Alexa user can set a voice command to execute a series of pre-designated actions. For example, the command “Turn on bedtime” changes the status of lights, ceiling fan and room temperature.[20] This technology has become more sophisticated and has been accepted and appreciated by many as an essential part of the home environment.

Smart speakers are designed to be always listening and waiting for a 'Wake' word. It has previously been identified that these devices are sometimes activated by incorrect wake words and commence recording.[21] For example, the difference between the words ‘Alexa’ and ‘Alex’ may be very small depending upon the pronunciation of the speaker or other factors such as noise near the person or device. Also, a person may use a wake word in general conversation near the device which activates the device to record the conversation that follows within the cloud computing service connected to the smart speaker.

The user commands are processed in the cloud environment and not within the device itself although snippets of data may reside there such as logs, details of the connected Wi-Fi network, small caches of data that are overwritten and technology to detect device activation words.[22] As with many forms of technology, different manufacturers present different characteristics in their devices, and it is the role of the investigator to understand these.

In summary, the device is always listening for the spoken activation or wake word. When it recognises the word, the device starts recording the commands and sends them to the cloud where they are processed and activated. Once activated, the spoken command is processed and stored in the cloud service, with the request of the user delivered through the home speaker.

To the investigator, this means that the most complete evidence you seek from the home speaker is likely to be located in cloud computing services which may be accessible through the controller (if you have the account holder username and password) subject to the laws of your jurisdiction and where the cloud service is based. It is recommended that the investigator seek qualified legal advice regarding the legality of accessing evidence from a cloud computing service because the actions in your jurisdiction may not be legal in the jurisdiction where the cloud computing service and subsequent evidence is based.[23]

Some digital evidence may be located within the device such as the wake words as previously mentioned. The Amazon Echo Dot for example stores log files, however, these are not accessible to the user.[24] The companion device (computer, phone) may also hold valuable evidence including account authentication details which may be compared to the authentication details located in the accompanying cloud service.[25]

The following section provides a brief introduction to the main home assistants/speakers and hubs with some of the features that may be of interest to the investigator who locates these devices within a crime scene.

Amazon Echo and Alexa

Amazon has a range of products[26] that operate within a home environment. The speakers connecting to the environment operate under a variety of names including Echo Dot and Show.[27] When a user sets up an Amazon device, they need a unique identifier linked to an Amazon account. This can be set up using a companion device such as a laptop computer or mobile device.

Alexa is the name of the Amazon Artificial Intelligence function resident in the Amazon cloud computing service. Echo is the name of the speaker device itself. The smart speaker, such as the Echo Dot, is connected to Alexa which is where the commands from the user are processed, and results delivered. The speaker is the interface for the user to access the functionality of Alexa in the cloud. The conversations recorded are stored in the Alexa cloud.[28] In effect, the user speaks to the speaker which transfers the request to the Alexa cloud where it is received, processed and results actioned back to the speaker and user.

Alexa can manage the tasks that are common across the range of home speakers produced by the major manufacturers. It can manage music requests, answer questions, and manage a range of smart devices within the home by voice command.[29] Alexa can control the complete smart home ecosystem. [30]

The Amazon speakers also work with third-party developers and can function with devices apart from those that are directly associated with Amazon. It has voice control to music services such as Amazon Music, Apple Music, Spotify, Pandora and more. The third-party applications may link back to their own cloud computing service to assist in providing the service. The Amazon cloud is also extending from in-home devices to include cars.[31] In effect, a user asking for music through their Amazon speaker from Spotify may interact with several cloud services. Firstly, Alexa in Amazon’s cloud interacts with the Spotify cloud which undertakes the request for a specific song or playlist to be played and then sends the required action back to the speaker through the Alexa cloud.

When searching for evidence from the Amazon speakers, commands may be resident within the device, however, most of the evidence such as conversations will be resident in the linked Alexa cloud account. Configuration evidence may also be located from the companion device such as a computer, laptop or mobile device that is used in the initial set-up of the Amazon speaker. This device can also be used to access the cloud-stored data of the registered user.[32] However, once a user deletes data from the cloud account, it cannot be recovered.

There is limited information available within the device itself, however, Privacy International has identified that the following data may be accessible from the device:[33]

      • Alarms the customer has set.
      • Device logs.
      • Wake words.
      • Wi-Fi network connected to.

Research has identified that there is obtainable evidence from the Amazon Alexa cloud environment using a companion device to access the cloud. [34], [35], [36], [37], [38], [39] 

Types of evidence in the Alexa cloud environment accessible with a companion device.

A-L M-W
Account information Music playlist
Alarms set Notifications
Alexa enabled devices including Dot, Echo and Fire TV Password
Bluetooth devices paired Preferences on registered Alexa devices.
Calls and messages Primary customer information
Calendars Region
Compatible devices Registered details of the account owner
Connection address Server address
Contacts Shopping list
Conversations between users and Alexa including history Third party services
Device details Time zone
Device group Timer and alarm set
Device Media Access Control address To-do list
Device name Timestamps
Device serial number User activity
Devices connected to Alexa User created lists
Device logs Username
Email Voice interactions between a user and Alexa
Gateway Internet Protocol address Wake word list for Alexa devices
Google calendars Wi-Fi
Hardware data Wi-Fi configurations
Household accounts Wi-Fi network
Language Wi-Fi settings which may including unencrypted passwords
Location information for traffic update

As this section shows, there is a lot of information available to the investigator from the Alexa ecosystem although having the user credentials to access the user’s account is required, particularly for cloud-based evidence.    

Apple HomePod

The Apple HomePod operates similarly to the Amazon product where speakers interact with the user and Siri in the cloud processes the commands and delivers the result for the speaker to action.

The HomePod is very sensitive and can pick up the spoken word clearly even when there is other and louder noise in the environment. Apple notes that communication between the device and the Siri servers is encrypted and is not tied to a user’s Apple Id[40] which is different to the way the Amazon Echo range of products operate. Each Apple device uses an anonymous unique identifier and is not linked to a specific user account such as an email address which increases privacy for the user.[41] This anonymity complicates the investigation strategy of law enforcement who now will be looking for this unique identifier to access the cloud account along with other Apple identifiers to conduct further inquiries such as email addresses.

Users who attach the Smart Home Accessories[42] to the HomePod can set up home scenes which are predetermined action events such as ‘turn off all the lights’ and ‘close all the windows’ when the user says “Good night, Siri” or wants a series of events undertaken when the front door is opened such as putting on the coffee machine and turning on the television.[43] This is similar to the functionality of the Amazon Echo devices.

Actionable events are driven by the user and can be set up on a Mac, or Apple mobile device such as a tablet or iPhone. An important point to note is they can be created and operated based on who is in the location at a specific time. [44] Events are specific to the user and will differ between homes.

HomePod operates as a hub to the HomeKit [45] so attached devices can be controlled whether this is accessed within the house or remotely via an app on a user’s iPhone. HomeKit is an app on Apple mobile devices and can be located on an Apple mobile device within the Privacy & Security tab. [46]

As with other smart speakers, the HomePod only commences recording when the command words are spoken. In the Apple environment, these are “Hey Siri”. [47]

For the investigator, the evidence that may advance an investigation is held in the Apple cloud servers and has been encrypted and anonymised, with the only possible identifier originating from the device’s unique identifier. The conversations and commands recorded are not linked to an identifiable user account as with the Amazon products, meaning it may be difficult for the investigator to locate evidence to support their investigation as Apple are most likely unable to locate it.[48]

The HomePod also accesses Apple services such as Apple Music[49] which is a subscription service. Identifying the payment credentials for the music subscription will be valuable evidence to identify the owner/controller of the device.[50]

There will be evidence linking a device such as an iPhone or Mac to a device that is required when setting up scenes. This will most likely be the companion device or mobile device of the owner/operator.

The Apple HomeKit is designed to act as a connection between a user and all the connected devices throughout the home or zones within the home.[51] The HomeKit is compatible with many manufacturers, not only those selling goods through Apple shops. To use the HomeKit, a user is required to input user credentials which are stored in the app. Remember, the HomePod is the physical speaker and the HomeKit is the application stored on Apple devices that connects to the HomePod speaker.

Access to the HomeKit can be remote to the home location, for example, a user can use their phone to alert their home to be ready for them in 10 minutes. When receiving this alert, the smart home can prepare a predetermined scene such as windows closed, the stove turned on, the coffee machine prepared, and the television turned on to a favourite channel.[52]

Compatible devices may hold user-generated data on their own clouds rather than with Apple. This may result in separate lines of inquiry for investigators who need to identify the specific devices of interest and where the data is stored.

Facebook Portal

Facebook Portal comes in a range of products; these being Portal TV, Portal Mini, Portal and Portal+.[53]

The Portal products are a series of screens with a camera that follows the user as they move around a dwelling. When a person walks into a room that Facebook Portal monitors it widens the screen to include that person in the screen. The device contains a cover that can be pulled over the camera and a sliding switch to disable recording. Calls from the Portal products can be made to other Portal devices as well as mobile phones and tablets through WhatsApp and Messenger which are both owned by Meta. The wake word of Portal is “Hey Portal”.[54] This is when the recording commences.

Facebook Portal can be linked to Amazon Alexa,[55] it has the same functionality as the Amazon range of home assistants as discussed previously. A user will need a separate Amazon account to link Portal to Alexa.[56] Using Alexa is interactive with Portal when using the phrase “Hey Alexa” which enables the features of Alexa. Also within the Amazon environment users can access Amazon Prime video and music subscription services.

The Portal collects user data that may be reviewed by Facebook under agreed terms and conditions, for example, the length and frequency of calls which can be used in building a user’s profile across the range of Facebook products.

Subscription services linked to Facebook and Amazon may be valuable evidence to an investigator should the circumstances dictate.

Google Home

Google Home[57]is a voice-activated assistant [58] and contains the main functions that Apple and Amazon speakers present. It can be used to control connected devices such as lighting, air conditioning, door locks as well as connect to online shopping and news.[59] A Google account is required for set-up through a companion device. As with the other speakers reviewed in this section, Google Home can receive commands, process them within their cloud and provide answers. It can also provide information and play movies if the device is connected to a Google Chromecast device.

Since the device is linked to a Google account, there may be examples of evidence being recorded in the MyActivity tab of the user’s account which is based in the cloud. There may be text transcripts of the comments of a user to Google which may be recovered as well as the time and location of that command. As well as the text transcript of the commands, the original voice commands may also be preserved.[60]

A Google Nest Hub is a voice-activated hub that a person can use to control their smart home. The hub presents a screen from which photos can be displayed, as well as connecting to multimedia such as YouTube.[61] Kim et al identify that the Google Nest Hub contains details about the smart home including information about each room and the devices installed.[62] This can prove to be valuable data indicating locations of further evidence.  

The app on a mobile device can contain relevant information obtained under forensic analysis including the current account name and email address of the user.[63] There is otherwise little digital forensic evidence that can be obtained from the Google app. Google advises that their device listens for the hot word to activate. If a hot word is not heard, recordings are deleted from the device and not sent to their cloud storage. The Google hot words are “Ok Google” or “Hey Google”.[64]

An examination may locate the following data from the speaker, associated account and companion device/application:[65], [66],[67],[68]

      • Account details.
      • Activity log.
      • Application usage history (YouTube, Google Maps, Chrome browser).
      • Call history.
      • Conversations with Google Assistant.
      • Deleted conversations including voice searches.
      • Detected sensor motion.
      • Device details.
      • Email account.
      • Homegroup.
      • Identify companion device.
      • Location.
      • Notifications.
      • Preferences from Amazon Alexa.
      • Registered devices.
      • Reminders including text transcript.
      • User information.
      • Voice commands.
      • Wi-Fi Service Set Identifier.

Smart Hub

A smart hub is a physical device or software that connects the many smart devices within a location and allows a user control of them.[69] There is an intersection between the physical hardware hubs and the smart speakers of Apple, Google and Amazon where apps and companion devices such as a computer that allows connection and control as well as operating as a smart speaker. [70] In effect, the hub is a central control where connected devices can be interacted with and activities such as setting air conditioning temperatures can be applied. A hub is controlled through an app or central console which is different to a smart speaker which can also be voice-controlled.

As well as a product of great convenience, a smart hub, like the smart speaker, is also of great benefit to those in the community who are elderly or disabled. Like with the home assistants, being able to tell the device they are going to bed and to shut down the home saves them the physical burden of going around the home and shutting windows, closing curtains, checking all the doors are locked etc which for some people, may involve extensive physical pain. With Amazon products, creating a series of commands is called a scene.

The technology behind the smart speaker and hub is very complex as one would imagine. The more devices connected to the hub the greater its ability to provide services and benefits to the residents. Examples of smart hubs include the Samsung SmartThings hub, Wink Hub and the Samsung Connect Home. [71]

The following section introduces examples of smart devices that may be found in a home. They are examples only as there are numerous smart devices now available to consumers with each category having many suppliers. This section provides examples to identify how these items may be of value to you in your investigations.

Smart television

Internet-connected televisions are not new technology and have been accepted into the home with minimal consideration. Televisions have evolved from passive devices which provide only a selection of programs to multi-functional devices which are capable of many of the functions of computers including providing access to internet browsing. Smart televisions also have a wide range of installed apps and optional apps that can be added by the user.

Examples of evidence that may be obtained from a television include:[72],[73]

  • Apps such as Facebook, Twitter and YouTube – Should an investigator be unable to access a suspect Facebook account through their computer or mobile device, access may be able to be obtained via the television. This may be the case for any app installed.
  • Bluetooth devices – Useful to identify devices that have been paired to the television.
  • Browser history including search engines.
  • Channels viewed.
  • Cloud services such as Dropbox and One Drive – As with many other smart devices, be aware of the relationship between the television and cloud services where extra evidence may be located. Locating access to a cloud storage account through a smart television which is not accessible on any other device at the location such as a phone, tablet, laptop or desktop may cause the investigator to consider why access to this account is accessible only through one of the more inconvenient devices at the premises.
  • Connected devices including external storage.
  • Content watched, purchased, downloaded or streamed.
  • Device identifiers.
  • Device name.
  • Emails and appointments – Using the television to connect to hidden email addresses is a strategy that many investigators may not be aware of. A suspect would expect their computers and mobile devices to be seized should they ever be the subject of a police search warrant, so using the television to connect to an email account they want to be hidden may be a successful strategy for them.
  • External storage device – With a smart television, external storage may be located and attached to the back of the television. The unique storage identifier may be stored on the television as well as the unique identifiers of other external devices which may be of interest to the investigator in the circumstances.
  • Gateway Internet Protocol address – This is the Internet Protocol (IP) address the television uses to connect to the Internet. It may not necessarily be the same IP address that other items within the residence connect to, indicating there may be another modem/router within the residence or a mobile device the television is connected to.
  • Location.
  • Media Access Control (MAC) address – This can be linked against the router to identify web traffic should the router store such detail.
  • Network information.
  • Operating System version – Smart devices often do not have their OS or firmware updated meaning the OS version will be the same as when purchased.
  • Phone numbers.
  • Pictures and multimedia.
  • Port numbers – This is the technical pathway the television uses when connected to the internet. Televisions use specific ports and hackers can identify vulnerable televisions using port numbers.
  • Smart functions.
  • Text messages.
  • Viewing history including networks, channels, websites, and programs viewed.
  • Voice service interactions.
  • Web browsing history and bookmarks.

Nest Devices

Google makes nest devices to suit smart home users and operate within the Google, as well as other developers’, environments. Nest products[74] can be found among many of the devices which are commonly found in smart homes including speakers, displays, streaming services, Wi-Fi, smoke, and CO (Carbon Monoxide) alarms, cameras, and doorbells.[75],[76]

Examples of data that can be located from Nest devices and connected cloud services include:[77],[78]

  • Camera video data including the start/end time.
  • Chrome browser cache details.
  • Connected devices.
  • Connection details.
  • Country code.
  • Device information including backup status and IMEI number.
  • Device setup information.
  • Email id of the user.
  • GPS details.
  • Identifier used for updating user actions on the device.
  • Installation time of the device.
  • Last time when the device communicated with the cloud.
  • Installation time of the app.
  • Light.
  • Local IP address used.
  • Location to which the device was pinned (latitude/longitude).
  • Media Access Control address.
  • Name associated with the user.
  • Network details.
  • Postal code entered during device activation.
  • Primary phone number of the user.
  • Profile picture of the user.
  • Temperature.
  • Time stamps of events.
  • Ultrasound.
  • User Id.
  • Various event timestamps.
  • Unique identifier of the user account associated with the device.

Refrigerator

Like any other smart device, a fridge can connect to a hub and have internet connectivity. For example, the Samsung range of smart fridges contains cameras placed inside the fridge so the user can identify what is in the fridge and what needs to be replenished. The Samsung Family Hub fridge allows remote access via an app and allows the television to be played on the screen as well as show photos, post notes and draw pictures. This fridge also has a calendar and can mirror what is on a compatible smartphone. Samsung also has a separate smart hub which is a physical device that can be wall-mounted. [79]

The Samsung Family Hub fridge can also connect to and control other compatible smart devices within the environment. The screen on the front of the fridge can display content such as to-do lists and reminders and provide access to numerous recipes. It also has a feature where a double knock turns the fridge door transparent so a user can see what is in the fridge without opening it.[80] Whilst at first glance a fridge is not the source of evidence many detectives would look at it to solve a homicide or any other serious crime as in an everything-always-connected environment there may be small traces of evidence from the fridge that can be used. This can include notes, schedules or digital evidence of calendar entries.

Features of smart fridges that may be useful in obtaining evidence include:[81]

      • Apps.
      • Calendars.
      • Draw pictures.
      • Family and home smart hub.
      • Internet access.
      • Memos and notes.
      • Messages.
      • Music such as Spotify.
      • Phone and TV mirroring.
      • Phone calls.
      • Photos.
      • Recipe planner.
      • Shopping lists.
      • Television.
      • Transparent view into the fridge.
      • Videos.
      • Schedules.
      • Video camera monitoring.
      • Voice recognition.
      • Weather forecast.
      • Whiteboard.
The smart fridge also stores food and beverages.  

Robot vacuum cleaners

Sensors and cameras control robot vacuum cleaners and the camera assesses a room as the cleaner navigates the environment. They can be programmed to operate to a schedule and can detect obstacles. The robot cleaner may also be able to map a house and its contents that are within the vision of the smart cleaner. A robot vacuum cleaner may even identify brands of objects within the environment.[82] Some models can map the size of the home with the location and the amount of furniture in the home.[83]

Wall Socket

A smart wall plug is installed into a standard wall socket. Communication is made to the controller which can record the amount of electricity the device connected to the socket is using. The device has an on/off function and commands can be sent remotely directly to the plug cutting or allowing power to the connected device.[84]

Key Takeaways

Key Takeaways

In this chapter, we have discussed the various forms of data that may originate from the multitude of smart devices in a home. You will note the most consistent evidence originates from the cloud services linked to the smart device directly or through the smart hub. To access this data, you will need the username and password credentials. As you examine the devices, you will learn there is a lot of data accumulated and stored in or through smart devices. The selection referenced in this chapter shows a lot of similar data is available in full or as artefacts through digital examination, and this can be used as a template to direct your thoughts at the many other smart devices you will locate during your investigations.

Digital evidence can open new lines of inquiry which were previously unknown to investigators. With technology being introduced into the home at an increasing rate, the potential for the smart home to solve crimes committed within its environment is increasing. Smart technology generates and stores large volumes of data, and the investigator can use this to advance their investigation should they be aware of this and know what to look for. Of note, the detective entering a smart home crime scene needs to be aware the house will be recording their activity, and they should turn off mobile phone wireless and Bluetooth connectivity before entering any crime scene.

Awareness of technology advancements and what data is captured is important. The astute investigator is aware that innovative technology is being created every day and existing technology mentioned in this book is being frequently upgraded. It is the detective’s job, once they become aware of a smart house’s potential, to provide evidence to ensure they are identifying all potential sources of evidence and to update their knowledge as to what is being recorded in a residence. Most police departments will have digital forensic experts on their staff, and these people are a valuable addition to the smart home crime scene investigation team.

Of additional benefit to the detective, is that all but the most technically savvy suspects will be unaware of the invasive nature of smart home data capture as the rest of the community. Those who commit spontaneous crimes will have little regard for Alexa in the background or a thermostat recording the change of temperature in the room once they leave the crime scene. They will also be unaware that their presence may be recorded on the many smart devices within a location or on their person.

Major crimes are solved by detectives being thorough, finding new lines of inquiry, running them out and seeing where they lead. Thinking of a house telling the detective who killed the victim, what time it occurred and how long the interaction lasted sounds unusual, however using the technology that is in homes today and downloading digital data as evidence via devices within a house is as strange to detectives today as the experience of the first detectives being told they could use fingerprints or photographs to identify a suspect decades ago.

It is in the interests of device manufacturers to have their products placed in smart houses and to optimise their devices’ real-time capacity to identify how their products are used. Also, the data captured may be sold to generate a further return on investment. Ed Thomas, the principal analyst at Global Data told the newspaper The Guardian that he believes Amazon and Google lose money on the production of the physical home assistants they sell; however, they make up for this with the extremely valuable data they collect from users.[85] This should tell the investigator that home assistants are designed to collect data and if it is being collected, they should try and locate and use it. If this is correct, these devices collect enormous amounts of extremely sensitive data about the environment within which they operate and the people occupying the home. The investigator interested in identifying what data can be accessed through communication with these giant technology companies is entitled to ask for such data via a search warrant or other applicable court order.

Scenario

Alex’s smart home is the location of the most serious crime, being the scene of his murder. There is a lot of technology within the home as Alex was an early adopter of technology as the following list indicates:

  1. External Close Circuit Television (CCTV). The CCTV camera caught Sledge as he walked to the front door. Unfortunately, as Sledge was wearing a baseball cap pulled down over the front of his face there was no clear image available to verify his identification. The CCTV images can however be matched against the home modem/router logs to identify the person walking to the house at that specific time was carrying the mobile phone of Sledge and was more likely than not to be Sledge himself.  The baseball cap can be seen and has a unique identifying symbol of a yacht club Sledge had once visited and may be searched for by police should they need to execute a search warrant.
  2. The front door Ring lock contains a camera that captures Sledge’s approach. As it is situated at a lower height than the CCTV cameras, it captures an image of Sledge’s face. This image complements the evidence from the CCTV camera as the cap Sledge is wearing is identifiable on the Ring camera as well as the CCTV cameras. Both cameras show that Sledge is carrying a bag that has no specific identifying particulars. It is confirmed to be black in colour.
  3. Internal CCTV cameras within the house record both men in some areas of the house. There is no camera where the actual crime took place or the pathway to the garage. The cameras later record Sledge walking around the house, with no sign of Alex.
  4. The Home Assistant hears the name “Alex” shouted and interprets it as Alexa. The command and noise from the subsequent fight is recorded by Alexa and stored in Alex’s cloud account. The home assistant contains a lot of data from Alex’s phone, but little from Sledge’s. Access to the home assistant may be sought from the provider by a search warrant, however, some data may be accessible if the username and password is available.
  5. The fridge records the door opened and a bottle of water was removed. The timestamps of these activities are recorded after Alex’s smartwatch records no heart activity indicating it was the killer accessing the fridge.
  6. The smart water meter records the increase in water usage as Sledge has a shower. It also records increased water usage as Sledge washes his clothes in the smart washing machine.
  7. The smart washing machine corroborates the water usage as it records a wash containing a small quantity of clothing set on a heavy-duty wash. The timeline in the log records can record the time of the wash and this can be accurately recorded within the overall operation timeline.
  8. The smart light sensors throughout the house record a single person walking throughout the house. The timestamps record the time the sensors were activated, turning on lights as Sledge walked throughout the house.
  9. Air conditioning. The smart home identifies the changes in the temperature of the house and adjusts the temperature setting of the air conditioner accordingly.
  10. The smart garage door records the time that the door is opened and where the command originated from. In this instance, it was from the phone of Alex in Sledge’s possession.
  11. The home router records the time of disconnection, reconnection and subsequent disconnection of Alex’s car as Sledge leaves the address, returns and leaves again with the body of Alex.

There is other physical evidence for investigators to examine such as the immediate scene of death, damage to the house, remnants of drugs and the stolen property of Alex’s girlfriend. These are lines of inquiry beyond the scope of this book, however, the technical evidence presented in this book corroborates and supports these significant areas of investigation.

 

  1. Mocrii,  D. ,Chen, Y. , & Musilek., P. (2018) IoT-based smart homes: A review of system architecture, software, communications, privacy, and security. Internet of Things (1-2 ), pp. 81-98. ↵
  2. Robels, R. J. & Kim, T. (2010) Applications, systems, and methods in Smart home technology. International Journal of Advanced Science and Technology. Vol. 15, February. ↵
  3. Mocrii,  D. ,Chen, Y. , & Musilek., P. (2018) IoT-based smart homes: A review of system architecture, software, communications, privacy, and security. Internet of Things (1-2 ), pp. 81-98. ↵
  4. Ibid. ↵
  5. Chung, H. Park, J. & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital investigation 22 (2017) S15-25. ↵
  6. Yassine, A., Singh, S., Hossain, M.S. & Muhammad, G. (2018, September) IoT big data analytics for smart homes with fog and cloud computing. Future Generations Computer Systems. ↵
  7. Ibid. ↵
  8. India Telecommunication Centre (2017, March). Technical Report M2M enablement in smart homes. Telecommunications Engineering Centre, Department of Telecommunications, Ministry of Communications, Government of India. ↵
  9. Nikou, S. (2019). Factors driving the adoption of smart home technology: An empirical assessment. Telematics and Informatics 45 (2019). 101283. ↵
  10. CSIRO. (n.d.). Living at home longer. www.csiro.au. https://www.csiro.au/en/research/health-medical/Smarter-safer-homes ↵
  11. Robels, R. J. & Kim, T. (2010) Applications, systems, and methods in Smart home technology. International Journal of Advanced Science and Technology. Vol. 15, February. ↵
  12. Kim, S., Park, M., Lee, S., & Kim, J. (2020). Smart Home Forensics—Data Analysis of IoT Devices. Electronics, 9(8), 1215. https://doi.org/10.3390/electronics9081215. ↵
  13. Hirt, M. & Allen, S. (2022, September 26). 10 Smart Home Trends This Year. Forbes Home. https://www.forbes.com/home-improvement/internet/smart-home-tech-trends/ ↵
  14. Smart Home Retrofit | Your Smart Life. (2015). Your Smart Life. https://www.yoursmartlife.com.au/smart-home-retrofit#:~:text=What%20is%20a%20retrofitted%20smart%20home%3F%20This%20refers ↵
  15. Lau, J., B. Zimmerman, and F. Schaub. (2018) Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviours with smart speakers. In Proceedings of the ACM on Human-Computer Interaction, 102.1–.31. Accessed February 14, 2021. https://dl-acm-org.proxyiub. uits.iu.edu/doi/pdf/10.1145/3274371. ↵
  16. TechTarget. (2024) What is? Retrieved Feb 2024. https://www.techtarget.com/whatis/ ↵
  17. Chai, W., Shea, S., Wigmore, I. (Aug, 2020) Smart home hub (home automation hub). TechTarget. Retrieved May 1, 2023. https://www.techtarget.com/iotagenda/definition/smart-home-hub-home-automation-hub. ↵
  18. McCloskey, J. (2018). CSI Alexa: The smart home has become the new crime scene witness. The Ambient. https://www.the-ambient.com/features/smart-home-crime-scene-264. ↵
  19. With my fridge as my witness?! (2019). Privacy International. https://www.privacyinternational.org/long-read/3026/my-fridge-my-witness ↵
  20. Palermo, M. (2016). Announcing Scenes for Smart Home Skills: Alexa Blogs. (n.d.). Developer.amazon.com. Retrieved May 1, 2023, from https://developer.amazon.com/blogs/alexa/post/Tx38PSX7O9YKIK1/announcing-scenes-for-smart-home-skills ↵
  21. Edwards, G. (2019) Cybercrime Investigators Handbook, Wiley, New Jersey USA. ↵
  22. Crawford, K. & Joler, V. (2019). The mystery of the Amazon Echo data. (2019). Privacy International. https://privacyinternational.org/news-analysis/2819/mystery-amazon-echo-data ↵
  23. Edwards, G. (2019) Cybercrime Investigators Handbook, Wiley, New Jersey USA. ↵
  24. Crawford, K. & Joler, V. (2019). The mystery of the Amazon Echo data. (2019). Privacy International. https://privacyinternational.org/news-analysis/2819/mystery-amazon-echo-data ↵
  25. Chung, H. Park, J. & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital investigation 22 (2017) S15-25. ↵
  26. Amazon.com.au: Echo & Alexa devices: Amazon Devices & Accessories. (n.d.). Www.amazon.com.au. Retrieved May 10, 2024, from https://www.amazon.com.au/Alexa-Amazon-Echo-Smart-Speakers/b?node=5425434051 ‌ ↵
  27. Ibid. ↵
  28. Chung, H. Park, J. & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital investigation (22) S15-25. ↵
  29. Li, S., Choo, K.-K. R., Sun, Q., Buchanan, W. J., & Cao, J. (2019). IoT Forensics: Amazon Echo as a Use Case. IEEE Internet of Things Journal, 6(4), 6487–6497. https://doi.org/10.1109/jiot.2019.2906946 ↵
  30. Johnson, J. (2019, August 28). What is Alexa? How it works and what it can do. Android Central; Android Central. https://www.androidcentral.com/what-alexa-how-it-works-and-what-it-can-do ↵
  31. Chung, H. Park, J. & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital investigation 22 (2017) S15-25. ↵
  32. Ibid. ↵
  33. Crawford, K. & Joler, V. (2019). The mystery of the Amazon Echo data. Privacy International. https://privacyinternational.org/news-analysis/2819/mystery-amazon-echo-data ↵
  34. Chung, H. Park, J. & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital investigation 22 (2017) S15-25. ↵
  35. With my fridge as my witness?! (2019). Privacy International. https://www.privacyinternational.org/long-read/3026/my-fridge-my-witness ↵
  36. Crawford, K. & Joler, V. (2019). The mystery of the Amazon Echo data. (2019). Privacy International. https://privacyinternational.org/news-analysis/2819/mystery-amazon-echo-data ↵
  37. Li, S., Choo, K.-K. R., Sun, Q., Buchanan, W. J., & Cao, J. (2019). IoT Forensics: Amazon Echo as a Use Case. IEEE Internet of Things Journal, 6(4), 6487–6497. https://doi.org/10.1109/jiot.2019.2906946 ↵
  38. Shafi, H. (2020, February 17). Alexa is the new target in Digital Forensics investigation. The Startup. https://medium.com/swlh/alexa-is-the-new-target-in-digital-forensics-investigation-e5eccba32e9f ↵
  39. Engelhardt, S. (2019). Smart speaker forensics. Business/Business Administration. 56. ↵
  40. Legal - Ask Siri, Dictation & Privacy - Apple. (n.d.). Apple Legal. Retrieved July 12, 2023, from https://www.apple.com/au/legal/privacy/data/en/ask-siri-dictation/ ↵
  41. Ibid. ↵
  42. HomeKit - All Accessories. (n.d.). Apple (AU). https://www.apple.com/au/shop/accessories/all/homekit ↵
  43. HomePod (2nd generation). (n.d.). Apple (Australia). https://www.apple.com/au/homepod-2nd-generation/ ↵
  44. Legal - Ask Siri, Dictation & Privacy - Apple. (n.d.). Apple Legal. Retrieved July 12, 2023, from https://www.apple.com/au/legal/privacy/data/en/ask-siri-dictation/ ↵
  45. HomePod. (n.d.). Apple (Australia). Retrieved July 16, 2023, from https://www.apple.com/au/homepod/ ↵
  46. HomePod (2nd generation). (n.d.). Apple (Australia). https://www.apple.com/au/homepod-2nd-generation/ ↵
  47. Ibid. ↵
  48. Legal - Ask Siri, Dictation & Privacy - Apple. (n.d.). Apple Legal. Retrieved July 12, 2023, from https://www.apple.com/au/legal/privacy/data/en/ask-siri-dictation/ ↵
  49. Apple. (2019).  Apple Music. Apple (Australia). https://www.apple.com/au/apple-music/ ↵
  50. HomePod (2nd generation). (n.d.). Apple (Australia). https://www.apple.com/au/homepod-2nd-generation/ ↵
  51. HomePod. (n.d.). Apple (Australia). Retrieved July 16, 2023, from https://www.apple.com/au/homepod/ ↵
  52. HomePod (2nd generation). (n.d.). Apple (Australia). https://www.apple.com/au/homepod-2nd-generation/ ↵
  53. Meta (2024) Meta Portal. Meta. https://www.meta.com/au/portal/ ↵
  54. Ibid. ↵
  55. Facebook 2020. How do I enable Alexa on my Portal? portal.facebook.com/au/help/573803526408538/ ↵
  56. Ibid. ↵
  57. Google Home (n.d.) A home that knows how to help. Google. https://home.google.com/welcome/. ↵
  58. Ibid. ↵
  59. Introducing Google Nest Hub - Google Nest Help. (2023). Support.google.com. https://support.google.com/googlenest/answer/9136909?hl=en ↵
  60. Engelhardt, S. (2019). Smart speaker forensics. Business/Business Administration. 56. ↵
  61. Introducing Google Nest Hub - Google Nest Help. (2023). Support.google.com. https://support.google.com/googlenest/answer/9136909?hl=en ↵
  62. Kim, S., Park, M., Lee, S., & Kim, J. (2020). Smart Home Forensics—Data Analysis of IoT Devices. Electronics, 9(8), 1215. https://doi.org/10.3390/electronics9081215. ↵
  63. Engelhardt, S. (2019). Smart speaker forensics. Business/Business Administration. 56. ↵
  64. With my fridge as my witness?! (2019). Privacy International. https://www.privacyinternational.org/long-read/3026/my-fridge-my-witness ↵
  65. Kim, S., Park, M., Lee, S., & Kim, J. (2020). Smart Home Forensics—Data Analysis of IoT Devices. Electronics, 9(8), 1215. https://doi.org/10.3390/electronics9081215. ↵
  66. Engelhardt, S. (2019). Smart speaker forensics. Business/Business Administration. 56. ↵
  67. Akinbi, A., & Berry, T. (2020). Forensic Investigation of Google Assistant. SN Computer Science, 1(5). https://doi.org/10.1007/s42979-020-00285-x ↵
  68. Kim, S., Park, M., Lee, S., & Kim, J. (2020). Smart Home Forensics—Data Analysis of IoT Devices. Electronics, 9(8), 1215. https://doi.org/10.3390/electronics9081215 ↵
  69. Menachem Domb. (2019, February 28). Smart Home Systems Based on the Internet of Things. In Internet of Things (IoT) for Automated and Smart Applications. Editor, Ismail, Yasser. DOI: 10.5772/intechopen.84894 ↵
  70. Engelhardt, S. (2019). Smart speaker forensics. Business/Business Administration. 56. https://scholarsarchive.library.albany.edu/honorscollege_business/56. ↵
  71. What Is a Smart Home Hub and Do You Need One? (n.d.). IoT Agenda. https://www.techtarget.com/iotagenda/definition/smart-home-hub-home-automation-hub ↵
  72. Boztas, A., Riethoven, A. R. J., & Roeloffs, M. (2015). Smart TV forensics: Digital traces on televisions. Digital Investigation, 12, S72–S80. https://doi.org/10.1016/j.diin.2015.01.012 ↵
  73. Popescul, Daniela, Smart . (July 27, 2018) TVs: What Do They Know (and Tell) about Us? IACSS 2018-IACLPM 2018 Joint Conference Proceedings. PDF. Available at SSRN: https://ssrn.com/abstract=3294943 ↵
  74. Google Nest smart home. (n.d.). Google Store. https://store.google.com/au/category/connected_home?hl=en-GB ↵
  75. Introducing Google Nest Hub - Google Nest Help. (2023). Support.google.com. https://support.google.com/googlenest/answer/9136909?hl=en ↵
  76. Dorai, Gokila; Houshmand, Shiva; and Baggili, Ibrahim. (2018) "I Know What You Did Last Summer: Your Smart Home Internet of Things and Your iPhone Forensically Ratting You Out". Electrical & Computer Engineering and Computer Science Faculty Publications. 78. https://digitalcommons.newhaven.edu/electricalcomputerengineering-facpubs/78 ↵
  77. Ibid. ↵
  78. Crawford, K. & Joler, V. (2019). The mystery of the Amazon Echo data. (2019). Privacy International. https://privacyinternational.org/news-analysis/2819/mystery-amazon-echo-data ↵
  79. SRF9300BFH 637L Family HubTM French Door Smart Fridge | Samsung Australia. (n.d.). Samsung Au. https://www.samsung.com/au/refrigerators/french-door/rf9000ac-family-hub-smart-fridge-637l-black-rf65a9770b1-sa/ ↵
  80. Ibid. ↵
  81. Ibid. ↵
  82. Knobloch, Carley. (March 17, 2018) Cleaning up: How robot vacuums work and what’s next. International Home and Houseware Show. https://blog.housewares.org/2018/03/17/cleaning-robot-vacuums-work-whats-next/ ↵
  83. Dreame (2024) How to Robot Vacuums Navigate? Dreame Global. https://global.dreametech.com/blogs/blog/how-do-robot-vacuums-navigate ↵
  84. Li, M., Gu, W., Chen, W., He, Y., Wu, Y., & Zhang, Y. (2018). Smart Home: Architecture, Technologies and Systems. Procedia Computer Science, 131, 393–400. https://doi.org/10.1016/j.procs.2018.04.219 ↵
  85. Gibbs, S. (2018, January 6). How smart speakers stole the show from smartphones. The Guardian. https://www.theguardian.com/technology/2018/jan/06/how-smart-speakers-stole-the-show-from-smartphones[ ↵
definition

Licence

Icon for the Creative Commons Attribution-NonCommercial 4.0 International License

Digital Evidence Manual Copyright © 2024 by Graeme Edwards is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, except where otherwise noted.

Share This Book