Concept Development and Use of an Automated Food Intake and Eating Behavior Assessment Method

Summary

This protocol shows and explains a new technology-based dietary assessment method. The method consists of a dining tray with multiple built-in weighing scales and a video camera. The device is unique in the sense that it incorporates automated measures of food and drink intake and eating behavior over the course of a meal.

Abstract

The vast majority of dietary and eating behavior assessment methods are based on self-reports. They are burdensome and also prone to measurement errors. Recent technological innovations allow for the development of more accurate and precise dietary and eating behavior assessment tools that require less effort for both the user and the researcher. Therefore, a new sensor-based device to assess food intake and eating behavior was developed. The device is a regular dining tray equipped with a video camera and three separate built-in weighing stations. The weighing stations measure the weight of the bowl, plate, and drinking cup continuously over the course of a meal. The video camera positioned to the face records eating behavior characteristics (chews, bites), which are analyzed using artificial intelligence (AI)-based automatic facial expression software. The tray weight and the video data are transported at real-time to a personal computer (PC) using a wireless receiver. The outcomes of interest, such as the amount eaten, eating rate and bite size, can be calculated by subtracting the data of these measures at the timepoints of interest. The information obtained by the current version of the tray can be used for research purposes, an upgraded version of the device would also facilitate the provision of more personalized advice on dietary intake and eating behavior. Contrary to the conventional dietary assessment methods, this dietary assessment device measures food intake directly within a meal and is not dependent on memory or the portion size estimation. Ultimately, this device is therefore suited for daily main meal food intake and eating behavior measures. In the future, this technology based dietary assessment method can be linked to health applications or smart watches to obtain a complete overview of exercise, energy intake, and eating behavior.

Introduction

In nutrition research and dietary practice, it is key to have good measures of what, how much, and how people eat, to find solutions to the overweight and obesity problems. To assess dietary intake, often conventional self-report questionnaires are used such as food diaries, 24 h recalls or food frequency questionnaires¹. These methods rely on self-report and are therefore time-consuming and prone to bias due to social-desirable answers, memory inadequacy, and difficulties in estimating portion sizes^2,3. In addition to measures of the diet quality (food type and amount eaten), it is also important to know how the food is eaten, as eating behaviors that slow down food intake have been shown to prevent overconsumption within a meal⁴. To assess eating behavior the golden standard is to have two observers annotate video recordings of people eating a meal⁵. This method is rather labor intensive and time consuming and does not allow for immediate feedback on the behavior.

Recent technological advances now provide the opportunity to combine automated measures of food intake with automated measures of the eating behavior over the course of a meal. In response to these developments, a new sensor-based dietary assessment method was developed, called the mEETr, the acronym of the two Dutch words 'Meter' (translated: measuring device), and 'eet' (translated: to eat). The mEETr is a regular dining tray with three built-in weighing stations (Figure 1 demonstrates the design of the tray and the sensor plates) and a camera holder. Each weighing station consists of three triangularly positioned measurement points to distribute the weight. The weighing stations measure the weight of the bowl, plate, and drinking cup or glass continuously over the meal. The mEETr also includes a video-camera holder. Currently, the camera holder is separate from the tray, but for standardization purposes an integrated camera after the next upgrade of mEETr (a folding video camera stick) would be ideal. The camera facilitates automated real-time analysis of the number of bites and chews, and eating duration, which allows for the generation of information on the eating rate and the bite size. Automated analysis of eating behavior is done with the use of a newly developed algorithm. Various research groups have developed devices to provide people real-time feedback on the acceleration of eating and the quantity people eat⁶. Also, augmented forks have been developed to provide real-time feedback on the number of bites and their frequency within a meal⁷. Additionally, an ear sensor was developed to measure the microstructure of eating in free living conditions^8,9. Similar to this device is the set-up used by Ioakimidis et al.¹⁰, where video measures were combined with a weighing plate to determine the food intake, number of bites, and chewing behavior.

Compared to these devices the novelty of the mEETr is that it combines automated measures of food intake of two plates and a drinking cup (n = 3) and eating behavior (e.g., eating rate, number of bites, bite size, and chewing behavior) in one device. The mEETr, as demonstrated, is suited for within meal measures of food intake and eating behavior within a controlled (eating lab) environment, but eventually the aim is to use the mEETr in less controlled environments where re-occurring meal plans are used such as daycares, elderly-homes, and hospitals.

Ultimately, the mEETr will provide a more objective, and as such, more accurate and precise measure of food intake and eating behavior than conventional dietary assessment methods and manual coding of videos. Better measures of the food intake would benefit nutrition and health research, but also the health professionals in their challenge to combat the increase in food-related non-communicable diseases¹¹. Ultimately the mEETr can be used in research and health-care settings as well as by health-conscious users at home by linking the mEETr to existing technologies and software, such as other health apps or smart watches. Overall, these health measures provide the user or the health-care professional with a rather diverse and complete overview of a variety of health-behavior patterns (e.g., food intake, eating behavior, energy expenditure based on real-life measures, sleep, stress) enabling the user to optimize their diet and create a healthy lifestyle.

Protocol

This pilot study was approved by the METC of Wageningen University prior to starting the project.

CAUTION: All the participants contributing to this project provided an informed consent, including the approval of video images showing visible and recognizable faces.

1. Sample preparation and participant consent

1. Prepare a juice (glass or cup), fruit yoghurt (bowl), and fruit pieces (plate).
   NOTE: These foods are selected for demonstration purposes only (Figure 2).
2. Recruit a participant or a volunteer who agrees to participate in the study.
3. Exclude the participants wearing glasses (who cannot use contact lenses) and/or having facial hair (beard or mustache) to avoid measurement errors.
4. Inform the participants about the study and the data collection (data storage, accessibility). Obtain separate permissions in case of non-anonymous video recordings. Get the signature of the participant on the informed consent before collecting data.

2. Device and measurement location set-up

NOTE: This protocol is suited for data collection in a controlled (eating laboratory) setting.

1. Make sure that the light in the room is evenly distributed-avoid shadows on participants' faces.
2. Avoid the background noise on the video recordings due to the presence of individuals other than the participant.
3. Seat the participant on a chair before a table; with tabletop located just below the participant's chest.
4. Connect the wireless receiver of the tray and the webcam to a laptop.
5. Start up the laptop. Ensure that the laptop has the following specifications: CPU i7-10750H, SSD M.2 512 GB, Memory 1x 16 GB, DDR4 2933 MHz non-ECC-memory, Operating system 64 bit.
6. Switch on the tray and ensure that there is charge in the tray (green light).
7. Open the connector program (dos), the receiver, and the processor software program together with the dashboard, respectively.
8. Check the incoming image quality in the processor program (Figure 3).
   NOTE: To detect the eating behavior, the image quality should be within the last quarter of the task bar (green); as close to 100% green as possible. Shadow forming may lead to low image quality.
9. Make sure the image frame is correct to prevent poor image quality. Ensure that the participant's head (above the cranium) till chest, including arms and shoulders are clearly visible.

3. Weighing system and data transport

1. Validate the measures prior to the use of the mEETr for the first time.
   NOTE: The mEETr device consists of a regular commercially available dining tray (fiber enforced epoxy dinner tray) with three built-in weighing stations (Figure 2).
2. To validate the set-up, ensure that the tray continuously measures the weight of a plate, bowl, and drinking glass.
   NOTE: The precision of the weighing scale across the whole range should be 0.3%.
3. Do not place too much weight on each weighing platform. The maximum weight for the largest platform (dinner plate) is 1.5 kg. The maximum weight of the two smaller weighing platforms (bowl and glass) is 800 g. The minimum weight that can be accurately measured is 1 g for each weighing station.
4. Make sure that the plate, the cup, and the bowl are not resting on the platform or the surrounding tray. Make use of the center ring to avoid this.
   NOTE: Each weighing station consists of three triangle positioned force sensors that act together as one scale. A triangle position was chosen to balance weight.
5. Make sure to keep the tray dry. The tray includes a 50 mm thin base panel (central circuit board) below the tray that contains the electronics.
6. For data transfer, make sure that the tray connects to a wireless receiver.
   NOTE: Transfer the weighing data at a 1 s interval via a short-range radio signal (about 1 m distance). Connect the receiver to a personal computer (PC) via an USB port.
7. The three force sensors measure the forces (or weights), sum them up, and convert them to a calibrated weight value.
8. Recharge the tray after each use.
   NOTE: The tray is powered by an internal battery pack and can be charged with a USB charger. An on/off slide switch is located near to the USB socket. A full battery charge provides for about 20 h of use.
9. Do not clean the tray in a dishwasher; the tray is not dishwasher proof. Clean the tray using a cleaning spray. Ensure that the tray is kept clean and dry. Leak channels along the platforms drain liquid spills.

4. Participant explanations and start of observation

1. Place the mEETr in front of the participant.
2. Instruct the participant to 1) eat as much or as little as he/she wants, 2) look straight into the webcam while eating, and 3) do not put hands in front of the face while eating.
3. Start a new observation in the receiver software. Log the date, participant number, participant's gender, age, and anthropometric data, such as weight and height. Include additional information such as the study condition and the study visit in the observation name.
4. Press Record in the receiver software to record the observation.
5. Activate the dashboard in order to check the video recordings and the incoming data during data collection (Figure 4).
6. Prior to the recording, ask the participant to 1) raise the card with the participant number, and 2) raise their hand at the start and end of the meal.
7. End the observation when the participant finishes eating. It takes 2 min to transfer all the data to a spreadsheet.
8. This is the end of the session for the participant.
9. Disconnect the webcam and the tray-receiver from the laptop and clean it with a cleaning tissue or cleaning spray.

5. Evaluation and transfer of data

1. Open the last observation in the receiver software. Automated measures of eating behavior are stored under the heading Data. Click on Export Data to extract the raw data. The subsequent output file contains data on the participant number, real time, relative (to start) time, and eating behavior variables (number of bites, number of chews, chewing duration) per test participant.
   NOTE: All behaviors are time stamped. In the receiver software, additional external data of the tray can be extracted, for example, the data on the weight of each of the three trays. The data is recorded 10 times per second and transferred. The tray data collection time is synchronized with the eating behavior recordings.
2. Summarize and visualize the results in different bar charts within the program itself. Export the results as raw data in log files (.xsl) (Figure 5).
3. Export the log files to a spreadsheet and perform the data analysis using the statistical program of preference.
4. Clean the data before data analysis.
   ​NOTE: Due to the distortions of pressing with cutlery on the plates (causing an increase in weight), weighing data of the tray needs to be cleaned to a Kaplan Meier curve with step size indicating bite size, the step length indicating time between bites. The beginning of the curve indicates start-weight, the last step indicates end-weight) as follows.
   1. Smoothen over the timepoints to measurements per second to filter out extreme values.
   2. Set 5 g boundary, detect weight plateaus (i.e., no change within +/- 5 g), and weight changes (changes over time larger than 5 g) to indicate bite sizes and portion changes.
   3. Exclude weight increase due to cutlery remaining on the plate.
      NOTE: The output is the total weight changes per weight station begin and end of meal (= meal size), average bite size, and bites per min.
5. To determine eating rate (g/sec) and bite size (g/bite) changes over the course of the meal, manually integrate the tray weight data and the eating behavior (Figure 6, Figure 7, Figure 8, Figure 9, Figure 10).

Results

A slower ingestion rate (Figure 7), smaller sip/bite sizes (Figure 8), and more chews (Figure 9) led to lower intake of the salad compared to the yoghurt and juice (Figure 6) as measured by the mEETr tray. The participants ate 17% less of the fruit salad compared to the fruit juice. All the eating behavior characteristics differed between the juice, yoghurt, and salad (Figure 7

Discussion

A healthy diet and a healthy eating behavior have shown to play a key role in the prevention of and solution to overweight and obesity¹¹. However, many of the methods used to measure the dietary intake and the eating behavior are burdensome for users, researchers, and health-care professionals and may be biased as they are dependent on memory and portion size estimations. Using the mEETr, independently or alongside conventional video and dietary assessment methods, would decrease the effort and th...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Battery	na	na	Battery pack (LiPo) and charge electronics via an USB port connector. No data from this port.
Connector program	Noldus		Noldus Information technology software dashboard nview
Dinner tray	na	na	Standard dinner tray from glass inforced epoxy
Larger scale	na	na	One high range custom made scale based on a triple force sensor method.
Mainboard	na	na	A mainboard converting the three scale measurements to calibrated weight numbers. This board also contains the low power short range RF transmitter.
OS Windows	Microsoft		windows 10 Pro 64 bit
Processor program	Noldus		Noldus Information technology software FaceReader
Receiver program	Noldus		Noldus Information technology software Observer
RF receiver	na	na	Custom build USB converter connected to a RF receiver. This receiver has a squelch setting for making it low range sensitive.
Small scales	na	na	Two low range custom made scales based on a triple force sensor method.