2.1. Microcontroller Subsystem

The core of the smartwatch is a microcontroller, which is used to collect and process the sensor data. Additionally, it is responsible for power management to reduce the energy consumption of the whole device. For this purpose a Texas Instruments MSP430FR5969 MCU was used due to its low active power consumption of only 103µA/MHz and fast, non-volatile and highly energy efficient 64KB Ferroelectric Random Access Memory (FRAM). The various Low Power Modes (LPM) allow to turn off the CPU if only a peripheral is working or to shut down the MCU to ultra-low power saving mode with only 20nA current consumption. Finally, this MCU features a broad range of peripherals like: internal oscillator, analog-to-digital converters (ADCs), timers, PWM, Universal Serial Communication Interfaces (USCI), and brownout reset circuitry. This allows the acquisition and processing of data from three analog sensors (camera, microphone and temperature) via analog-to-digital converter ADC input and digital output of the accelerometer sensor is via SPI serial port. Also the digital interfaces allow to communicate with the NFC communication chip and to update the contents of the e-paper display.

2.2. Multi Sensor Subsystem

This subsection presents the sensor subsystem and the low power mechanisms to increase the energy efficiency of the sensor data acquisition. As shown in the block diagram in Fig. 3, the microphone and the accelerometer MEMS sensors and the temperature sensor are integrated on the core board of the device. The image sensor is attached to the core board with a connector.

ACCELEROMETER

The accelerometer ADXL362 from Analog Devices, was selected as it is the lowest power consumption on the market, especially in sleep mode (40nW@2V). Its motion triggered wake up feature allows the implementation of aggressive power management, which is of special importance in an energy harvesting supplied system. In fact, this sensor is an ultra-low power, 3-axis accelerometer that consumes less than 2µA at a 100Hz output data rate and 270nA when in motion triggered wake-up mode, which allows entering a lower LPM of the microcontroller, which is woken up again when a new event happens that triggers a motion wakeup. The internal data buffer and interrupt generation also allow fast and energy efficient burst transfers of sensor data.

MICROPHONE

The second on-board sensor is an INMP801 MEMS microphone by InvenSense, which was selected for its low power current consumption of only 17µA at 1V. Due to the low output swing, an operational amplifier (Op-Amp) is used to adjust the swing and offset of the microphone output signal to obtain high quality data when sampling the signal with the ADC of the microcontroller. A Texas Instruments OPA344 Op-Amp was selected for this task, because it features good performance for audio signal at a very low power consumption. As the design has been driven with low power consumption as the primary constraint, a DC-DC converter (LTC2406) and a Texas Instruments TPS22960 load switch controlled by the microcontroller can switch on/off the microphone circuit, the MEMS sensor, the DC-DC and the Op-Amp. This results in only the small quiescent current of 250nA of the load switch when the microphone system is not used for audio data acquisition. The schematic of this subsystem is shown in Fig. 4.

IMAGE SENSOR

This sensor is placed on a separate breakout board that is directly connected to the core board, which controls the sensor and supplies it with power. This allows the power management to stay on the core board, since a controllable DC-DC converter is used to convert the voltage from 3V to the 3.3V supplied to the image sensor. The Texas Instruments TPS61097-33 converter was chosen not only because of high conversion efficiency but especially to be used as switch for the peripherals with only 5nA in shut down, and the presence of the enable pin on the DC-DC. The selected camera is the ultra-low power Centeye Stonyman sensor. With less than 2mW at 3V, the power consumption of this image sensor is a few orders of magnitude less than a digital camera and perfectly fits in our power-constrained, wearable scenario. The chip features a 112×112 pixel, grayscale image using a simple interface with 5 digital inputs and one analog output. The digital input allows to configure the image sensor and to select the pixel which is connected to the analog output for analog-to-digital conversion with the microcontroller. The image acquisition was optimized using hardware peripherals of the microcontroller: PWM for digital control pulse generation, ADC for analog output sampling, and DMA for transferring the data to FRAM. Exploiting the hardware modules allows turning off the CPU core during the image data acquisition and speeds up the image acquisition process. The image acquisition time and energy of 468ms and 2.22mJ for a software implementation is reduced down to only 47ms and 226µJ for a single image acquisition. This results in an energy saving of 89.9% for the acquisition of one image.

TEMPERATURE SENSOR

A Negative Temperature Coefficient (NTC) thermistor with resistance that decreases while temperature increases has been integrated directly in the smart watch. We used a Vishay 10kΩ 1% thermistor connected to the ADC of the microcontroller to monitor the ambient temperature.

2.3. Energy Harvesting Subsystem

The design includes both a solar and a thermal harvester subsystem to exploit the combination for the two sources. Both are used to supply a single battery, thus it is important to manage them to do not waste energy when one of the two sources is not harvesting.

SOLAR ENERGY

The solar energy harvesting subsystem is based on the bq25570 ultra-low power IC from Texas Instruments. It exploits a high efficiency boost converter to harvest energy from sub-milliwatt low-voltage sources such as small solar cells under indoor lighting and integrates a configurable battery management circuit to recharge all battery chemistries or supercapacitors. For high efficient energy harvesting, the bq25570 features maximum power point tracking (MPPT) capabilities. Moreover, it integrates an ultra-low power buck converter with programmable output voltage. The bq25570 consumes less than 500nA in active mode and about 5nA when it is switched off using its chip enable pin. The power source of our smartwatch is a wrist strap, which embeds one flexible solar cell, MP3-25 from PowerFilm. The bq25570 always adapts itself to work at the maximum power point (configured at 80% of the open-circuit voltage) providing a maximum power of 126µW @ 2.2V under low office light conditions (250lx) and 4.45mW @ 3.0V under sunlight (10000lx). The adopted storage element is a 40mAh – 3.7V lithium-polymer (LiPo) rechargeable battery. To reduce the energy consumption of the bq25570 when no solar input power is available, we use a Texas Instruments nano-power TLV3691 comparator and disable the bq25570 harvester chip if the solar input voltage drops below 300mV. This reduces the power consumption of the solar harvester from 500nA to 5nA when no input energy is available. Even if the comparator circuit consumes 110 nA, it reduces the overall power consumption of the harvesting circuit by 28% when considering an average day to night cycle with 50% solar energy unavailability. To still supply the smartwatch under solar energy unavailability, this shutdown circuit features an override capability that allows the microcontroller to enable the chip once its buffer capacitance runs out of energy. For this feature, the microcontroller uses its supply voltage supervisor to wake up and re-enable the bq25570 output voltage before running out of energy.

THERMAL ENERGY

The thermal energy harvesting subsystem is built around the LTC3108 from Linear Technology. This DC-DC converter can start conversion from only 20mV and boosts the voltage using a fly back converter with a 1:100 transformer at the input and was optimized to work with input between 20mV and 150mV. Because our TEGs should be wearable around the wrist, we are severely constrained in size. The Quick-Cool QC-32-0.6-1.2 has a size of only 8×10mm and a thickness of 2.6mm, allowing to fit them on the bottom of the wrist together with a heat sink of 14×14mm and a height of 6 mm glued on top of each of them. We found that using 7 TEGs in series was a good compromise, requiring a temperature difference of only 1.75K to obtain the required startup voltage for the step-up converter and being still wearable.

The solar panel and TEG modules are combined together in one wristband as illustrated in Fig. 4. To combine the harvested power of both harvesting circuits in one storage, we use power OR-ing at the output of both harvesting sub circuits, as power OR-ing before the harvesting stage does not work because of the highly internal resistances of the used sources. Both passive power OR-ing using a diode and active power OR-ing using a controlled MOSFET circuit were evaluated. An active solution requires additional power for control and is only useful if, on average, the losses in the diodes exceed that power. In our setup, this is not the case. Only adding a diode to the output of the LTC3108 is sufficient, since the bq25570 already has such a diode built-in at the battery charging output. The power harvesting circuit is shown in Fig. 5.

2.4. User Interface

DISPLAY MODULE

The graphical interface of the smartwatch is an e-paper display (EPD). This technology is thought to emulate the behavior of the ink on a paper sheet and, unlike LCD or OLED technologies, does not need any background illumination because they reflect light similar to traditional paper. This makes them not only readable under direct sunlight, but they also consume power only during the display update and can hold the image for days without a power supply. For this project, a 1.44" TFT EPD EK014AS015 from Pervasive Displays was used. It features a matrix of 96x128 pixels and can display images in bitmap one-bit black/white format. It communicates with the microcontroller by means of SPI protocol, the supply voltage is within the range 3-3.6V. A display update operation requires a maximum time interval of 1.2s, consuming only 12mJ of energy. At the end of each update the EPD is switched-off to save power.

COMMUNICATION MODULE

The wireless connectivity is provided by the M24LR16E-R NFC-EEPROM from ST Microelectronics. It is a NFC/RFID tag IC with 16 kbit EEPROM and I2C bus interface. The M24LR16E-R is characterized by a maximum write time of 5ms in I2C mode and 5.75ms in wireless mode. The I2C mode supports a 400 kHz communication frequency while in RF mode the maximum data rate is 6.6 kbit/s to 26 kbit/s. A power supply of 1.8-5.5V is needed only for I2C operations because this chip behaves as a passive tag, thus the wireless communication is powered by the initiator device, namely a smartphone or other devices with NFC capabilities.

3.1. Low-Power Sensor Data Acquisition

In the previous section we have already highlighted some of the software optimizations that go together with the low power hardware design. To speed up the image acquisition and lower the power consumption, PWM, continuous ADC conversion and automatic DMA transfer to FRAM was used. This implementation is not only 90% faster due to hardware support, it also allows to turn off the CPU during acquisition, which results in a reduction of the power consumption by 19%. Similar to the image acquisition, the microphone recording also uses continuous ADC conversion and DMA to store the data in FRAM. Here again, this allows to shut down the CPU of the microcontroller. Because of external analog circuitry, the data stored in FRAM can directly be used later for processing, because the external circuit already takes care to properly scale the audio signal and apply the necessary offset.

For the digital sensors and interfaces, the SPI and I²C communication interfaces are already implemented as hardware peripherals. This means that these data transfers only need to be initiated by the CPU, which then can be turned off for the rest of the transfer. The data for transmission and reception can then simply be read or written to the FRAM using the DMA module. For sensors with a local data buffer, like the ADXL362 accelerometer, we exploit this feature to send the microcontroller to deep sleep mode and only wake up to transfer the buffered data in larger blocks.

3.2. Power-Aware Context Recognition

Context awareness provides completely new use cases for a smart watch and makes it much more user-friendly. Our context awareness system tries to classify the action that is being performed by the wearer of the smart watch. It does so based on the data available from the many different sensors.

In order to train a classifier we need to collect and label a dataset. We chose 5 classes: relaxing, walking, public transport, office and cafeteria, and acquired data from the temperature sensor, accelerometer, camera and microphone. Each data item lasts 5s and contains 8kHz audio data, 100Hz 3-axis accelerometer measurements, one temperature read-out and one image of 112x112 pixel. Each of the classes has several hours of labeled data.

We perform the classification using a decision tree which is constructed using the continuous C4.5 algorithm and our acquired dataset. This algorithm was chosen because of its energy efficiency and low computational complexity during classification (as opposed to during learning). For performing the classification, there is only the decision tree, which has to be descended until arriving at a leaf.

Performing the classification on the acquired samples directly yields very poor results, because they usually represent the information in an unfavorable way, e.g. such that very little noise or small variations of the environment yield completely different results. This is overcome by extracting features from this data. For the different types of sensors, there are different suitable features:

• For the audio data, we use the number of zero crossing with a 1% hysteresis, the average energy of the signal, the maximum absolute value and dispersion. We also use features from the frequency domain, such as the spectral centroid, the bandwidth, and the well-known Mel-frequency cepstral coefficients (MFCCs).
• For the accelerometer data, we compute mean, variance, energy, covariance between the axes, the dynamic range and the frequency-domain entropy.
• For the temperature we calculate the average rate of change, mean, variance, and dynamic range.
• For the camera data we computed the mean (avg. brightness), variance and contrast.

Many of these features have an intuitive meaning, like the energy of a segment of the audio stream, which provides an indication of the loudness. However, to make use of the large number of features we obtain this way, we would need an immense amount of labelled training data to prevent overfitting the classifier. Instead, we chose to perform feature selection, selecting only a fixed number of features. We chose those which jointly have most information relevant for classification, i.e. those which maximize the mutual information with the classification. Among the most important features there were the mean of the temperature, the mean of the camera image, the spectral energy and entropy of the accelerometer axes, followed by a long list of audio features.

3.3. Feature Extraction Cost

Due to the limited energy budget in our wearable device, it is important to understand how energy is consumed in the sensor data extraction and processing and classification process. Fig. 7 shows the estimation for the time needed and energy consumed to acquire and extract features from different sensors. It should be noted that these values were calculated after performing our system wide optimizations, both in the hardware and software.

3.4. Context Recognition Accuracy Analysis

With the low-complexity classification system presented here, we were able to achieve a mean accuracy of 65.24%. To give more insight into the limitations, we present the confusion matrix in Fig. 8. While some classes can be distinguished very well, some are often confused. This is the case for 'relaxing' and 'office', which is not surprising since the test subject were often in front of the computer at work as well as at home.

The various sensors require a substantial amount of power, considering the targeted energy neutrality. Clearly there is a trade-off between which sensors are used and what accuracy can be achieved. We visualize this trade-off in Fig. 9, considering only the data of some of the mentioned sensors.