Unobtrusive sensors and pervasive computing technology incorporated into the daily home life of older adults enables meaningful health and activity changes to be recorded continuously for months to years, providing ecologically valid, high frequency, multi-domain data for research or clinical use.
An end-to-end suite of technologies has been established for the unobtrusive and continuous monitoring of health and activity changes occurring in the daily life of older adults over extended periods of time. The technology is aggregated into a system that incorporates the principles of being minimally obtrusive, while generating secure, privacy protected, continuous objective data in real-world (home-based) settings for months to years. The system includes passive infrared presence sensors placed throughout the home, door contact sensors installed on exterior doors, connected physiological monitoring devices (such as scales), medication boxes, and wearable actigraphs. Driving sensors are also installed in participants' cars and computer (PC, tablet or smartphone) use is tracked. Data is annotated via frequent online self-report options that provide vital information with regard to the data that is difficult to infer via sensors such as internal states (e.g., pain, mood, loneliness), as well as data referent to activity pattern interpretation (e.g., visitors, rearranged furniture). Algorithms have been developed using the data obtained to identify functional domains key to health or disease activity monitoring, including mobility (e.g., room transitions, steps, gait speed), physiologic function (e.g., weight, body mass index, pulse), sleep behaviors (e.g., sleep time, trips to the bathroom at night), medication adherence (e.g., missed doses), social engagement (e.g., time spent out of home, time couples spend together), and cognitive function (e.g., time on computer, mouse movements, characteristics of online form completion, driving ability). Change detection of these functions provides a sensitive marker for the application in health surveillance of acute illnesses (e.g., viral epidemic) to the early detection of prodromal dementia syndromes. The system is particularly suitable for monitoring the efficacy of clinical interventions in natural history studies of geriatric syndromes and in clinical trials.
Prevailing clinical research is fraught with limitations in the reliability and validity of data captured because of inherent shortcomings of assessment methodology. Interviews are constrained by the times when the clinician and patient can coordinate schedules. Time allotted for examinations is limited by what the volunteer can reasonably be asked to do in a single session. These brief, widely spaced sessions - even if augmented by occasional telephone calls or Internet queries - severely limit the potential to detect meaningful change in function or wellbeing over time. Current test sessions are largely composed of requests for information that can be difficult to recall and verify (e.g., "do you remember to take your medications?") or performance of artificial tasks (e.g., "stand up and sit down as fast as you can"; "remember these ten words"). The assessments are often designed to restrict test-to-test variability when in fact variability in performance itself may be a key diagnostic feature. Further, these brief biopsies of time are conducted under artificial conditions rather than within the normal flow of day-to-day life. Therefore, they are of limited ecological validity. Finally, the current paradigm inherently cannot provide direct linkage of interdependent key events or outcomes (e.g., sleep, socialization, physical activity) because the data are not time-stamped other than as recalled.
An approach to overcoming these shortcomings lies in the development of systems that can be embedded in home or community that take advantage of advances in pervasive computing and sensing technology, wireless communications, and high frequency multi-domain data analytics. The technology and experience in this area are growing and a number of systems have been developed, but have been limited in deployment, features or longitudinal experience1,2,3,4. In this manuscript, we describe a protocol developed as a means to provide real-time, continuous and longitudinal home-based assessment of health-relevant data to improve upon the limitations of the current health assessment paradigm. Oregon Center for Aging & Technology (ORCATECH) has developed a home-based system based on pervasive computing and sensing technology to provide continuous, real-time assessment of health-relevant activity and behavior. Bringing the assessment into home to enable largely unobtrusive and continuous monitoring of real-world activity substantially overcomes current limitations. First, since the core system is embedded in the life-space of the participants as part of their ambient environment, it is inherently convenient. Assessments requiring discrete responses can be collected when a person is most at ease and, in the case of passive collection methods, as frequently as necessary without burdening a participant. Second, being in the person's normal life-space affords the opportunity to collect data that are immediately ecologically relevant, not simply testing contrived measures of function, but everyday cognition. For example, prospective memory failure, a common complaint difficult to naturalistically test in the clinic, can be assessed at home by automatic daily tracking of medication taking behavior, thus tapping both everyday cognition, as well as a key performance metric known to be sensitive to cognitive change. Third, because the data are digital and time-stamped, measurement of multiple interrelated measures aligned in time is facilitated. For example, time on the telephone and time out of home (measures of social engagement or withdrawal), computer use (measuring initiation, psychomotor activity and cognitive function), and other measures that have been shown to change with functional decline (sleep behavior, weight, walking speed) can add to the sensitivity of the sensor net to distinguish subtle changes that may not be otherwise apparent. Importantly, the effect of health and life events on cognition and function (e.g., weekly reports of pain, medication change, low mood) can also be linked to this data stream as they occur. Finally, conventional testing and queries can be presented via computer or related interfaces (e.g., tablet, smartphone), providing unparalleled opportunities to simultaneously compare legacy test performance to novel digitally derived measures from the same tests, such as response or pause times, learning curves and intra-test variability. This new approach thus transforms current assessment to be more convenient, unobtrusive, continuous, multi-domain and naturalistic. Ultimately, the basic platform of in-home sensor based assessment technology and methodology, provides a system that can be tuned and scaled to address a wide range of specific research questions related to health and wellbeing with noted advantages over the current accepted practice of infrequent clinic or telephone-based assessments.
The following protocol outlines the process of deploying this platform for unobtrusive in-home behavioral and health-related data collection. In developing this platform, a key goal has been to provide a basic suite of assessment functions that can provide the data necessary to infer both general domains of health and well-being (physical, cognitive, social, emotive), as well as more specific behaviors (e.g., medication taking, walking, sleep-related activities, physiologic activity). The development of the platform has been guided by several principles including using the most passive unobtrusive sensing approaches, minimizing direct user engagement with technology, being technology 'agnostic' (i.e., employing the best devices or technical solutions rather than requiring a particular approach or product), being durable (for long-term assessment) and scalable, and minimizing hands-on maintenance.
The platform described has evolved over the past twelve years, importantly informed by a range of end-users, from "digitally naive" to early adopters. Periodic surveys and focus groups have been key to informing this development5,6,7. Hundreds of volunteers have allowed the systems to be continuously deployed in their homes for up to eleven years with iterative modifications being introduced based on the advances in technology, new functional capacities requested by the research community, and the key constant input of individuals living in home where the technology has been deployed. Collectively, these volunteers have formed a "living" laboratory in the community which we call the "Life Laboratory" where their homes and the continuous data collected throughout the day provide a unique level of details about health, activity, and the life course.
A basic platform of sensing technology forms the backbone of the overall system for capturing continuous home-based data. The elements of this platform are described subsequently. The core platform is modified (elements may be added or removed) based on the information obtained during the process of gathering user attitudes, and beliefs and outcome measures of interest for the study using the research platform. Because data communication protocols are standardized, the system has been designed to allow any device that follows these protocols to be incorporated into the network.
The basic platform described here is based on the use case of volunteers in the Life Laboratory (LL) who consent to have the platform deployed within their homes to collect naturalistic activity and behavior data of their normal life activities for many years (longest current continuous deployment = 11 years).
The hub computer and Ethernet/WiFi connection allow data collection from system devices and transmission back to secure servers at ORCATECH without participant interference. The hub computer is configured to the specific participant and home set-up upon system installation using a laptop or tablet and a control panel that connects to a centralized digital participant management system. Additional data collection devices (such as sensors, MedTracker, and scale) can be configured by communicating with the hub computer in the same way.
The ORCATECH Console and Remote Technology Management System is a custom digital technology and data management system called "Console" that allows participant home technology configuration and system set-up, as well as ongoing remote technology management of homes including secure data collection and monitoring. In addition, to facilitate deployment of the system in the community where each home may have a unique layout, a graphing tool based on a tablet interface is used to automatically record where various sensors are located and their valid physical adjacencies to other sensors (Figure 2). This is important for reference during remote monitoring of the system at the home level.
Passive infrared (PIR) Motion Sensors are digitally assigned to a given home during system installation, communicating with the hub computer via a wireless USB Dongle. One sensor is placed per room to sense motion within the room and participant transitions from room to room. A straight "Sensor Line" of four sensors is placed on the ceiling of a hallway or other area where the participant walks regularly at a consistent pace. This sensor line allows unobtrusive gathering of walking speed many times per day. Other metrics can be derived from these activity sensors such as dwell time or number of room transitions. Door contact sensors are placed around the home at all external doors to detect participants' coming and going from the home, and on the refrigerator to determine general frequency of food access.
On-Line Weekly Health and Activity Self-Reports are required to make optimal sense of the data from the passive system of collection devices. These data are critical to the analysis of participant report of events in the home relative to the sensor collected data. The online weekly self-report survey can be completed on any computing device (e.g., laptop, tablet, smartphone) with an Internet connection to query participants about trips out of the home, visitors in the home, health changes, space changes within the home, loneliness, depression, and pain level. Weekly data collection relies on a relatively short window of recollection, which provides much higher resolution of data and likelihood of accuracy than, for instance, annual or semi-annual check-ups. Furthermore, this self-report process also allows investigators to examine passive indicators of potential cognitive impairment, such as variation in the time to complete the survey, variation in number of clicks, increased difficulties reporting accurate dates, or impairment markers in free text responses. As part of the basic platform, we install a seven day electronic pill box that records whether or not the designated day's compartment was opened and the time(s) that it was opened each day. This provides information about medication adherence as well as a potential indication of cognitive decline if consistency of medication-taking decreases.
A wireless digital bioimpedence scale that also collects pulse, body composition metrics, pulse wave velocity, environmental temperature, and ambient carbon dioxide level is installed in the bathroom, providing data on participants' daily weight. This data can then be correlated with other reported events (e.g., health status, medications), as well as other passive indicators of behavior, such as protocol adherence and frequency of use over time.
In cases where our participants drive, we install a driving sensor in their vehicles. This sensor provides information about driving habits such as frequency, timing, duration, and distance of trips, as well as frequency of hard stops or hard accelerations.
A wrist worn wearable device collects physical activity data both in and out of the home. Several brands and models of wearables have been used in Life Laboratory homes.
Depending on the project, an investigator using the ORCATECH platform may choose to supplement the basic sensor set with additional data-collection components. Examples tested in the past include a phone sensor to monitor socialization through landline phone activity, the development and implementation of a digital balance-board for balance testing, a tablet with periodic cognitive tasks for the participant to complete in their own home, and an automated texting system to evaluate the efficacy of medication reminders via phone.
To handle the diverse data generated by the ORCATECH Life Lab, a tailored information and data system is used for collecting, annotating, maintaining, and analyzing copious activity and health data. ORCATECH has developed a custom system for participant management, self-report data collection and processing, and continuous data collection from all system devices and sensors. The system relies on a distributed NoSQL Cassandra server cluster to store the sensor data and a lambda architecture using Kafka and Spark which allows our data processing capabilities to move closer to real-time processing. Using a REST API, data is transferred into standard data analysis platforms and statistical software programs for data analysis.
All participants provided written informed consent. Life Laboratory participants are asked to live their lives as they normally would to allow longitudinal observational study of their life activities and patterns for the rest of their lives. They may withdraw at any time if they wish. The study protocol was approved by the Oregon Health & Science University (OHSU) Institutional Review Board (Life Laboratory OHSU IRB #2765).
- Prior to deployment, add all equipment to the Console inventory system online. Assign a name to each device or sensor along with its serial number and MAC address, allowing it to be stored within the Console inventory system.
- Place a QR code tag on each piece of equipment, enabling specific assignments of home location for the sensors and devices when deployed in home.
- Before deployment, all sensors and devices that are to be installed will be assigned to the home by scanning the QR code on the sensor or device. This will bring up a website that allows sensor or device assignment to that specific home.
- Install the hub computer with a SD card which contains the hub configuration management tool.
- Pack all the now inventoried sensors and equipment, hub computer with installed SD card into an installation kit (box) for home deployment.
- Verify that the participant's home has an Internet service provider.
2. Home Deployment
- Set up the hub computer by inserting the wireless dongle and the primary ZigBee coordinator dongle into the hub computer. Insert the Ethernet cable into the hub computer. Lastly, plug the hub computer's power cord into a centrally located room's power socket and connect the Ethernet cable to the home's Internet connection.
Note: The configuration management tool will ensure that it is using the most current version of the software.
- Connect an Internet enabled device (laptop, tablet, cell phone) to the hub computer'swireless network to access the local control panel website. The control panel website will show the status of the hub computer, as well as any sensors installed in the home (Figure 3).
- Run the software configuration tool, ensuring the appropriate software is installed. Do this by navigating to the control panel and running Update.
- Navigate to the control panel to check that the hub computer is communicating with the main servers. Make sure that the services that allow data to be collected from each installed device and sensor are up and running.
- Add the sensors to the home, beginning with the motion sensors. Start by opening the sensor placement website from the control panel.
NOTE: If the home requires more than 16 motion sensors, connect a router dongle into the hub computer and add it to the home or personal area network, also called the "PAN." Once the extension dongles (if needed) are added to the PAN, remove them from the hub computer and plug them into outlets spread throughout the home, creating a network around the home that will send the motion sensor data to the hub computer.
- In the sensor placement website, create a virtual floor plan of the home, including all the rooms and egress doors. Make sure to select the Sensor Line as one of the areas that is added to the floor plan. Add virtual representations of sensors to the floor plan. Lastly, link the virtual representations of home areas to other each other - in a way that reflects the physical layout of the home - and to the virtual representations of the sensors.
- Add each successive sensor to the PAN - known as the Personal Area Network - by using the sensor placement tool and physically pressing a button near the sensor's battery. Then, start attaching each sensor to the room or area in the home that is represented in the virtual floor plan.
- Continue attaching the physical sensors to the walls of the home. Place each wall sensor at head height in each room (kitchen, bedrooms, bathrooms, living rooms) ensuring that the sensor only captures the activity in that room and does not pick up the activity from another area (for example, avoid someone walking down a hallway being picked up by the sensor in a room next to the hallway).
NOTE: The sensor placement tool allows you to identify and create pathways between rooms.
- Install a row of four restricted field (ceiling) sensors in a straight walkway (hallways or other areas where the participants must walk past each of the four sensors without a change in pace) on the ceiling to capture walking speed.
- Space these walking speed sensors 61 cm (2 feet) apart.
- Record the exact distance between the restricted field sensors in the sensor placement website.
- Install door sensors on each egress door, again using the floor plan on the sensor placement website to indicate their physical location.
- Add the pillbox to the PAN, confirming that the device has been assigned to the home inventory. Then activate the device by opening one of its lids. Because the pillbox communicates with the hub computer, ensure it is close enough to the hub computer for its signal to be detected.
Note: The pillbox is frequently kept in the kitchen or bathroom based on participant preference.
- To setup the scale, navigate to the scale page, located within the Devices tab in the control panel.
- On the scale, press the side power button for 10 seconds. The scale should display a confirmation message.
- Once the scale shows up in the list of devices, click the Setup button on the right side of the control panel to start the setup process.
- Enter the participant's height and weight when prompted in the control panel.
- If the participant does not have a pacemaker, toggle the Pacemaker button in the control panel notifying the scale that it can collect bioimpedence data.
- Place the scale in a location that has a flat, solid surface easily accessible to the participant (typically in a bathroom).
- Have the participant weigh themselves, confirming the scale is recording their initial weight which is entered into the control panel.
- Set up the wrist worn wearable device by opening the wearable control panel setup page and pressing the reset button located on the back side of the device ten times.
- After the device shows up in the list of devices in the control panel and click the Setup button on the right side of the control panel to start the setup process.
- Once the account is set up, calibrate the time by using the wheel tool on the setup page.
- Finalize the setup by synchronizing the wearable with the hub computer. Click the Sync button in the control panel to confirm that the device is connecting properly and the time is set to the same time as the hub computer.
- Indicate in Console on which wrist the wearable is intended to be worn by the participant.
NOTE: Different devices may require different procedures depending on the manufacturer. Additional sensors and devices may also be deployed and integrated into the data stream such as computer use software and driving sensors. Procedures for adding these are given next.
- Install the commercial computer use monitoring software on the participant's computer and record their e-mail address. These e-mail addresses are used to send and receive weekly online health and activity surveys.
- Verify that the participant's computer operating system is compatible with the commercial computer use monitoring software.
- Install the software on the participant's computer using the installation program hosted on a USB flash drive.
- Verify that the software is operational on the computer by opening the Task Manager and checking that the software is in the list of Applications.
- In the Console inventory system, associate the software program with the participant's profile.
NOTE: See the Table of Materials for the specific computer use software used (other commercially available monitoring software can be substituted).
- Setup a driving sensor for participants
- Verify that the participant's car was made after 1996 and that the car is supported by the driving sensor device software.
- Install the driving monitoring device's app on a mobile device and use the app to setup the adapter.
- With the car turned off, plug the adapter into the car's on-board diagnostic (ODB) port.
- Wait for the app to recognize and connect to the adapter. This should take 2 - 4 min.
- Insert the car key into the ignition. (If the car has keyless ignition, press the car's start button). Turn the key to the position where it switches on the electric power without starting the engine.
- Wait for the app to finish setting up the adapter.
- In the Console inventory system, add the participant's account information from the app in order to allow the adapter's data to be transferred to the ORCATECH servers using the commercial software's application programming interface (API).
NOTE: See the Table of Materials for the specific driving monitoring device used.
3. System Confirmation
- Once all devices are in their final places in the home, confirm that the hub computer is working properly by navigating to the control panel. Check that the hub computer can communicate with the main servers to transfer data and the services to collect the data for each device type are running.
- Review if data is streaming from each device by navigating to the data collection page on the control panel.
- Walk near the motion sensors installed in each room in the home in order to confirm each sensor is collecting data about recent movements. Check the motion sensors by viewing the live graph of the motion sensor data activated by walking through the home.
- Check the pillbox by opening and closing each of the compartment doors of the pillbox a few times. Review the data collection page on the control panel to see if this recent activity was measured and collected.
- Check the scale by weighing yourself or the participant. Confirm this data is properly synched and transmitted by navigating to the Synched column within the scale Devices page in the control panel.
- Check if the wearable device is properly synched and transmitting data by navigating to the Synched column within the wearable Devices page in the control panel.
The ORCATECH suite of technology makes it possible to collect a uniquely rich data set about the life patterns of people as they go about their usual activities. The sensor system allows unobtrusive and continuous monitoring of the volunteers in their own homes. The system has been used in dozens of studies involving hundreds of volunteers in research examining key domains of health and function such as walking speed and mobility, medication-taking behavior, mood, time in or out of home, sleep, and computer use8,9,10,11,12,13,14,15,16,17,18. Representative results are presented here.
Walking speed is an example of a function that is challenging to conventionally assess frequently over time in natural settings. It is well recognized as an important indicator of general health and a useful component of clinical testing observed during office visits. In clinic assessment of gait speed has several drawbacks. Observations are made only at most a few times a year. The patient being observed may also walk with an atypical pace due to being aware of the assessment. With the ORCATECH sensor suite, four restricted field passive infrared sensors are placed a known distance apart typically in a hallway of the home. An algorithm estimates the walking speed of a person walking beneath the sensors over time8. This unobtrusive monitoring method made it possible to collect 39,474 walking episodes during a single month of the observation9. Walking speeds collected in-home were found to correlate well with motor measures assessed during an in-person examination, as well as cognitive function (Figure 2). This technique makes it possible to observe walking function on a regular basis over long periods of time, months or years. This then provides the opportunity to observe changes in gait speed that may be characteristic of impending pathological states such as mild cognitive decline or dementia (Figure 4)16.
The continuous activity data by location combined with self-report and other known environmental data provides the opportunity to observe changes in activity associated with many salient life events. The data can be visualized or displayed in a number of ways. Spiral plots have been found to be useful in this regard for identifying long-term trends in the data stream. An example is provided in Figure 5.
Figure 1. ORCATECH platform. Multiple sensors and devices are deployed throughout homes, tuned to particular outcome measures of interest. The box designated Device/Sensor "X" represents the ability of the system to plug in any number of new devices or technologies as needed over time. The data generated follows standard security protocols and is available with appropriate review for use by many interested parties (Data Scientists, University Collaborators, PHARMA, Health Industry, etc.). The homes on the left represent the cohorts enrolled in various studies using the platform. Life Laboratory - BC is a small cohort located in British Columbia, Canada; AIMS Transitions is a study of the use of remote monitoring data by care teams to reduce unwanted transitions to dependent levels of care; EVALUATE-AD is the Ecologically Valid, Ambient, Longitudinal, Unbiased Assessment of Treatment Efficacy in Alzheimer's Disease; iCONECT is the Conversational Engagement as a Means to Delay Alzheimer's Disease Onset study conducted in Oregon and Michigan; CART (Collaborative Aging Research using Technology Initiative) has four sites (Low income elders in Portland, veterans in the rural northwest, African American elders in Chicago and Latino elders in Miami); "Studies XYZ" denotes a number of other studies (not listed here) that also use this ORCATECH infrastructure. Please click here to view a larger version of this figure.
Figure 2. Each room in a home is linked to the others based on possible room transitions. Sensors are linked to the rooms in which they are located. The walking sensor line in a home consists of four restricted field passive infrared sensors linked together in the order in which they are placed. The walking sensor line links room(s) in which the sensors are located. Green (or red dots, not shown here) on the sensor nodes indicate if the sensor is currently reporting in to the sensor network. Please click here to view a larger version of this figure.
Figure 3. The control panel displays the status of the hub computer and all sensors installed in the home. Please click here to view a larger version of this figure.
Figure 4. The plot shows the trajectory of change in walking speed variability defined as the Coefficient of Variation (COV) in a cohort of 93 older adults followed in their homes for over three years. Time 0 is the first week of monitored walking speed for each participant. Latent trajectory models were used to identify four distinct groups: : Group 1, the highest baseline and increasing COV followed by a sharply declining COV; Groups 2 and 3, relatively stable COV; and Group 4, the lowest baseline and decreasing COV. Participants with non-amnestic MCI (naMCI) were more likely to be members of either highest or lowest baseline COV groups (groups 1 or 4), possibly representing the trajectory of walking speed variability for early- and late-stage MCI, respectively. The figure is modified from Dodge H.H., et al.19. Please click here to view a larger version of this figure.
Figure 5. Spiral plot of six months of home activity data for a Life Laboratory volunteer. The data are plotted as a 24 h clock, with midnight at the top and noon at the bottom. Each day forms one circle. The solid blue circles mark one-month boundaries. Colored points indicate sensor activity by room location: red = bathroom, green = bedroom, pink = kitchen, blue = living room. Note the three-week period in which the volunteer had a live-in guest who slept in the living room (night-time activity in the living room is apparent; second month). The 2-week period when the volunteer did not leave the apartment for meals, is due to a Noro virus epidemic that resulted in a congregation restriction in the community (fourth month). Finally, note the consistent patterns of the behavior: bedtime at 22:30, up once most nights between 2 am and 4 am, rise time 06:00, a visit from the housekeeper at 5 pm every 2 weeks. Please click here to view a larger version of this figure.
We have described a basic system or platform enabling home- and community-based remote sensing and reporting of salient health and wellbeing measures on a continuous basis. The system is intended to be used primarily in research at this time.
Where possible, the system uses open source tools and sensors or devices taking advantage of available APIs and software development kits (SDK). The system is designed to be technology "agnostic" such that a wide variety of sensors or devices can be "plugged-in" or incorporated as needed. Sensed domains chosen (e.g., motion-mobility measures, medication taking activity, computer use, etc.) will depend on the clinically relevant areas of function that may be considered most informative for a particular indication. Thus, a sparse array or more fully featured array of sensors and devices can be deployed. Data generated grows continuously and thus the data system has required modification to meet this demand. Accordingly, we have transitioned to a more distributed data architecture to allow for more processing capacity and scalability. The system is designed to run in the community with daily remote assessment of the technical health of the home system. Although autonomous, low-touch operation has been the key, the system requires occasional home visits as sensors or devices can fail and need to be fixed or replaced.
There are a number of critical steps to successfully deploying this system in research. The first and most important is not 'technical.' It is participant-based - ensuring that the participants understand the protocol and are comfortable with the longitudinal nature of the research. This ensures adherence, for example, to answering the weekly questionnaire or using a medication-tracking device. Ensuring not only proper documentation of the initial set-up of the home (e.g., where the sensors are located), but also any changes that may be made by the resident over time is also a key to proper data collection and management. Similarly, on-going surveillance of the functionality of the system once installed ensures that the longitudinal data is collected with high fidelity over time.
The ORCATECH platform makes possible continuous, long term, and unobtrusive data collection with a base in the home environment. Populations are heterogeneous with regard to their comfort and acceptance of technology in their home. Data shows that a major reason motivating the acceptance of using this technology in research studies is participant perception that the data collected will ultimately be useful (e.g., can address ways to successfully age in place). Older, retired adults tend to spend most of their time at home (on average about 21.5 h a day as assessed with the ORCATECH platform), making it an ideal location for observations that reflect the true state of a person's health as opposed to the clinic environment20. Younger or working populations may spend considerably less time at home. However, even younger individuals spend at least a third of their day at home (usually in evenings and at night). In addition, the system is designed to synch with wearable technologies and mobile devices (e.g., smartphones) and other remote sensed data (e.g., driving data), so many types of out-of-home behaviors and activities can be captured as well. Technology changes rapidly, adding a challenge to the task of maintaining a stable platform of the devices and software used to collect data and ensure that it is comparable across epochs. Further, the remote data is captured unsupervised and ultimately requires validation as to its inferred real activity. To this end, it is necessary to validate observations made or derived using ORCATECH data with "gold standard" or "ground truth" techniques. For instance, algorithms used to estimate sleep parameters such as total time asleep were verified using pressure mats placed under the mattress11. Walking speeds calculated from the firings of a line of passive infrared sensors were validated using a gait mat8. These validations not only provide observable connection to the real world sensed events, but a standard for comparing new sensors or devices that may be used to putatively measure the same function in the future. Nevertheless, there are few accepted standards for real-world validation at this time. Some of the instruments that have been used in the ORCATECH system, such as commercial actigraphs or fitness trackers, may use proprietary specifications or algorithms that are not shared with researchers. The proprietary nature of such commercial technology and their related algorithms makes the use of many commercial sensing devices or technologies a current relative limitation of research in the mobile and digital biomarker field in general.
In the introduction, we emphasized that the ability to collect continuous, unobtrusive, and ecologically valid data in home and community settings may be game changing for clinical research that has primarily relied on sparse self-report and brief exams. The system described in this paper provides a pathway for the research community to achieve this goal. Many individual assessment technologies (wearables, smartphone reporting apps, etc.) for monitoring people in situ have been described21. Far fewer, more integrated 'smart home' systems have been reported that have many of the features described in this paper22 . This remains a nascent field and few have successfully developed and deployed these technologies as an end-to-end system for research, deployed in wider community settings. Wider adoption of these approaches is a major focus for future work.
There are many potential applications for this platform. For example, the system might be used for public health surveillance such as monitoring communities for infectious disease outbreaks where relatively acute changes in room transitions indicate individuals who developed viral symptoms23. At the same time, information from this approach may allow understanding as to the effectiveness of a quarantine or activity restriction at the community level by determining how many people remained in their homes over time.
At a more basic level, the system may be used to inform researchers about fundamental aspects of human behavior in the real world. Thus, it has been employed to examine the predictability of human activity in personal space, an important aspect for the generalizability of all predictive modeling of research applied to people while in their home or indoor environments24,25. The results suggested that human mobility in personal space is highly stereotyped, and that monitoring discontinuities in routine room-level mobility patterns may provide an opportunity to predict individual human health and functional status or detect adverse events and trends.
Other applications may more directly impact diagnostic and disease progression determinations that are highly relevant for patients, families and health providers. In this context, the ORCATECH system has been frequently used to examine relevant activities and behaviors that indicate the relative preservation of cognition and health in aging populations. The system is particularly attuned to address age-related changes in cognitive and physical health which is challenging to assess with traditional tools and methods where the current assessment and care paradigm consists of brief clinic-based visits a few times a year, often using qualitative inventories to assess physical, cognitive, and neurobehavioral symptoms. These inventories are based on the subjective memories and/or impressions of a caregiver26,27 or self-report by the patient, which can be unreliable. One study comparing self-reported activity with objective activity data gathered from the ORCATECH sensors at home found the agreement only of 25% at the time28. Continuous objective observation may improve the detection of changes in health and the assessment of treatments especially for progressive or chronic conditions such as mood changes, mobility disorders or dementia where it is challenging for individuals or their families to accurately observe and recall slow as well as fluctuating changes over time.
The platform's diverse variety of potential data channels offers many windows of observation and resulting assessments of meaningful change to interrogate how individual or multiple functions may change across a particular condition. For example, multiple platform functions have been used to interrogate change associated with mild cognitive impairment (MCI) over time. Thus, walking speed, as calculated from passive infrared sensor firings, was found to decrease with mild MCI, while the variability in walking speed increased over time in this population16. Rest and activity patterns were inferred from bedroom sensor firings with the finding that sleep becomes more disrupted as cognitive impairment progresses11. Diminished medication adherence, as followed using the MedTracker pillbox, was also found to be a marker for early cognitive change12,17. Computer usage tracking software was used to discover that computer use decreases with MCI and patterns in the use of a computer mouse may also reflect cognitive decline13,14.
In summary, the ORCATECH paradigm of data collection has many strengths compared to traditional clinical research data acquisition methods. The data is largely unobtrusively collected in the familiar home environment. Information is gathered at high frequency, on a minute or daily level, over a long period of time, making ecologically valid longitudinal studies of health and wellbeing possible. The ORCATECH system is well-suited to a wide variety of research domains: basic human activity and behavior discovery, public health surveillance, and disease or condition-specific tracking. It may be especially useful as applied to interventions and clinical trials research for the sensitive detection of meaningful real-world change. In this application, there are multiple potential benefits including: 1) allowing researchers to improve study outcomes' objectivity and reliability; 2) reducing sample sizes and decreasing time to an answer due to the high temporal resolution of data; 3) uncovering hidden adverse events that can be subtly detected, but might have gone unreported by the participant; 4) facilitating long-term continuous post-trial or post-market assessment of a target intervention. It is anticipated that this methodology will substantially advance clinical research as it becomes more widely deployed and continually improves with inevitable advances in technology and its applications.
The variety of data channels and domains produced by the system offers a unique window into the everyday lives of participants to a number of research settings that can take advantage of high frequency, long-term, and ecologically valid data capture. It is anticipated that this approach will be of value to many research teams. Future plans include expansion of this platform to enable a large-scale collaborative deployment to thousands of homes facilitated by the recent Collaborative Aging Research Using Technology (CART, www.carthome.org) initiative of the NIH and VA. This new data will be combined with the more than ten years’ worth of information already collected to create a major resource for understanding cognitive and functional change with aging across diverse populations.
The authors have nothing to disclose.
The research described here was supported by grants from the National Institutes of Health, National Institute on Aging (U2CAG054397, P30 AG024978, P30 AG008017, R01 AG042191, R01 AG024059), Intel, the Foundation for the National Institutes of Health and the Robert Wood Johnson Foundation.
|Raspberry Pi 3 Model B||Raspberry Pi Foundation||Raspberry Pi 3 Model B|
|Motion Sensor||NYCE Sensors Inc||NCZ-3041-HA|
|Door/Window Sensor||NYCE Sensors Inc||NCZ-3011-HA|
|Curtain Motion Sensor||NYCE Sensors Inc||NCZ-3045-HA|
|Home Stealth USB Phone Recorder||Fiho||Fi3001B|
|Body Cardio Scale||Nokia||WBS04|
|Activite/Steel Activity Monitor||Nokia||HWA01 STEEL|
|Zigbee USB Stick||Silicon Labs||ETRX3USB|
- Peetoom, K. K., Lexis, M. A., Joore, M., Dirksen, C. D., De Witte, L. P. Literature review on monitoring technologies and their outcomes in independently living elderly people. Disabil Rehabil Assist Technol. 10, (4), 271-294 (2015).
- Liu, L., Stroulia, E., Nikolaidis, I., Miguel-Cruz, A., Rios Rincon, A. Smart homes and home health monitoring technologies for older adults: A systematic review. Int J Med Inform. 91, 44-59 (2016).
- Kim, K. I., Gollamudi, S. S., Steinhubl, S. Digital technology to enable aging in place. Exp Gerontol. 88, 25-31 (2017).
- Kaye, J. Making pervasive computing technology pervasive for health & wellness in aging. Public Policy & Aging Report. 27, (2), 53-61 (2017).
- Wild, K., Boise, L., Lundell, J., Foucek, A. Unobtrusive in-home monitoring of cognitive and physical health: Reactions and perceptions of older adults. Journal of Applied Gerontology. 27, (2), 181-200 (2008).
- Wild, K., Boise, L. In-Home Monitoring Technologies: Perspectives and Priorities of Older Adults. (2012).
- Boise, L., et al. Willingness of older adults to share data and privacy concerns after exposure to unobtrusive in-home monitoring. Gerontechnology: international journal on the fundamental aspects of technology to serve the ageing society. 11, (3), 428 (2013).
- Hayes, T. L., Hagler, S., Austin, D., Kaye, J., Pavel, M. Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE. 7248-7251 (2009).
- Kaye, J., et al. One walk a year to 1000 within a year: Continuous in-home unobtrusive gait assessment of older adults. Gait & posture. 35, (2), 197-202 (2012).
- Petersen, J., Austin, D., Mattek, N., Kaye, J. Time out-of-home and cognitive, physical, and emotional wellbeing of older adults: A longitudinal mixed effects model. PloS one. 10, (10), 0139643 (2015).
- Hayes, T. L., Riley, T., Mattek, N., Pavel, M., Kaye, J. A. Sleep habits in mild cognitive impairment. Alzheimer disease and associated disorders. 28, (2), 145 (2014).
- Hayes, T. L., Larimer, N., Adami, A., Kaye, J. A. Medication adherence in healthy elders: small cognitive changes make a big difference. Journal of aging and health. (2009).
- Kaye, J., et al. Unobtrusive measurement of daily computer use to detect mild cognitive impairment. Alzheimer's & Dementia. 10, (1), 10-17 (2014).
- Seelye, A., et al. Computer mouse movement patterns: A potential marker of mild cognitive impairment. Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring. 1, (4), 472-480 (2015).
- Seelye, A., et al. Embedded online questionnaire measures are sensitive to identifying mild cognitive impairment. Alzheimer Dis Assoc Disord. 30, (2), 152-159 (2016).
- Dodge, H., Mattek, N., Austin, D., Hayes, T., Kaye, J. In-home walking speeds and variability trajectories associated with mild cognitive impairment. Neurology. 78, (24), 1946-1952 (2012).
- Austin, J., Klein, K., Mattek, N., Kaye, J. Variability in medication taking is associated with cognitive performance in nondemented older adults. Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring. 6, 210-213 (2017).
- Austin, J., et al. A smart-home system to unobtrusively and continuously assess loneliness in older adults. IEEE Journal of Translational Engineering in Health and Medicine. 4, 1-11 (2016).
- Dodge, H. H., Mattek, N. C., Austin, D., Hayes, T. L., Kaye, J. A. In-home walking speeds and variability trajectories associated with Mild Cognitive Impairment. Neurology. 78, (24), 1946-1952 (2012).
- Kaye, J. A., et al. Intelligent systems for assessing aging changes: home-based, unobtrusive, and continuous assessment of aging. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 66, suppl 1 180-190 (2011).
- Wang, Z., Yang, Z., Dong, T. A Review of Wearable Technologies for Elderly Care that Can Accurately Track Indoor Position, Recognize Physical Activities and Monitor Vital Signs in Real Time. Sensors. 17, (341), (2017).
- Skubic, M., Alexander, G., Popescu, M., Rantz, M., Keller, J. A smart home application to eldercare: Current status and lessons. Technol. Health Care. 17, (3), 183-201 (2009).
- Campbell, I. H., et al. Engineering in Medicine and Biology Society, EMBC, 2011. Annual International Conference of the IEEE. 6793-6796 (2011).
- Austin, D., Cross, R. M., Hayes, T., Kaye, J. Regularity and predictability of human mobility in personal space. PloS one. 9, (2), 90256 (2014).
- Buysse, D. J., Reynolds, C. F., Monk, T. H., Berman, S. R., Kupfer, D. J. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry research. 28, (2), 193-213 (1989).
- Cummings, J. L. The Neuropsychiatric Inventory Assessing psychopathology in dementia patients. Neurology. 48, 5 Suppl 6 10-16 (1997).
- Teng, E., et al. Utility of the Functional Activities Questionnaire for distinguishing mild cognitive impairment from very mild Alzheimer's disease. Alzheimer disease and associated disorders. 24, (4), 348 (2010).
- Wild, K. V., Mattek, N., Austin, D., Kaye, J. A. "Are You Sure?" Lapses in Self-Reported Activities Among Healthy Older Adults Reporting Online. Journal of Applied Gerontology. (2015).