Manuals+

A Practical Guide to Home Automation Using Open Source Tools

By Steve Ovens

Published by Opensource.com

About Opensource.com

Opensource.com publishes stories about creating, adopting, and sharing open source solutions. Visit Opensource.com to learn more about how the open source way is improving technologies, education, business, government, health, law, entertainment, humanitarian efforts, and more.

Submit a story idea: opensource.com/story

Email us: open@opensource.com

Supported by Red Hat ?

About the Author

Steve Ovens is a dedicated IT professional and Linux advocate. Prior to joining Red Hat, he spent several years in financial, automotive, and movie industries. Steve currently works for Red Hat as an OpenShift consultant and has certifications ranging from the RHCA (in DevOps), to Ansible, to Containerized Applications and more. He spends a lot of time discussing technology and writing tutorials on various technical subjects with friends, family, and anyone who is interested in listening.

Follow him at @linuxovens.

Why I Use Home Assistant for Open Source Home Automation

Home automation can be a slippery slope. The right open source tools can get you on firmer footing.

Home automation is a slippery slope; you have been warned! This multipart series discusses home automation using the open source project Home Assistant. This introductory article covers the author's journey to Home Assistant, what the application does, and why it's important.

How My Journey Began

The author's initial goal was to solve a need related to his homelab setup, which involved a Netgear 10G Ethernet switch that would periodically crash when its state table filled up. The only solution was a reboot, which was inconvenient when the author was away from home. The solution involved using a smart plug on a separate, firewalled network to remotely reboot the switch.

My Journey Away From the Cloud

Being privacy-conscious, the author is uncomfortable with "cloud"-connected anything. While not "anti-cloud," the author prefers to host services locally. This led to discovering alternative firmware projects like Tasmota and ESPHome for wireless chipsets, allowing for local control of devices. Stores like CloudFree sell devices pre-flashed with Tasmota, and companies like Shelly offer products with optional cloud components but maintain local control.

The author now has 43 Internet of Things (IoT) devices, ranging from custom sensors to smart bulbs and infrared blasters. Despite having minimal experience with soldering or electronics, the author found Home Assistant accessible.

Centralized Local Control

With many IoT devices, managing them individually became untenable. The author explored open-source projects like openHAB and Home Assistant. Home Assistant was chosen due to the author's investment in it and its focus on local control.

Home Assistant aims to create a house that reacts to its inhabitants. For example, lights can turn on softly when entering a room at night, rather than brightly. While voice assistants like Alexa or Google can control lights, they rely on cloud services and raise privacy concerns. Home Assistant allows for automation based on sensor inputs, such as time of day, room occupancy, or temperature, enabling sophisticated automations like turning on a fan when the room exceeds 26°C but not if a person is in bed.

Sustainable and Open Source

Home Assistant benefits from critical mass adoption and has a sustainable income model through Nabu Casa, which supports the community and developers. Unlike "open core" models, Home Assistant is fully open source, allowing the community to fork the project if they disagree with its direction.

Future articles will cover wireless protocols (Zigbee, Z-Wave, WiFi), installing Home Assistant, setting up add-ons like MQTT and Node-RED, and making backups.

The author invites comments or tweets for specific topics to explore.

Links:

Cloud Control vs. Local Control: What to Choose for Your Home Automation

Cloud may be more convenient, but local control gives you more privacy and other options in your Home Assistant ecosystem.

This article discusses foundational issues and technologies in home automation, influencing how users approach and configure their Internet of Things (IoT) devices.

Cloud Connectivity

Most devices today rely on cloud services, offering convenience but also privacy issues related to companies accessing personal habits. Cloud access also creates reliance on third parties. Examples include Sonos remotely bricking older speakers and Automatic shutting down its cloud-based car tracker service. IFTTT, a popular service for event programming, recently altered its free plan, severely limiting events. Some companies have blocked users' access to devices based on user feedback. Cloud connectivity can also introduce delays due to signal travel time to servers and back, and is vulnerable to internet outages.

Image Description: A screenshot showing voice control options for smart home devices, including compatibility with Amazon Alexa, Google Assistant, and IFTTT.

Local Control

Regaining control over smart devices can be achieved through proprietary platforms like Hubitat, or by using open-source solutions like Home Assistant. The author uses Home Assistant and standardized on WiFi, with plans to branch out to Zigbee. Devices are either bought or made based on compatibility with open-source firmware like Tasmota or ESPHome. While not always "plug-and-play," options like Shelly and CloudFree offer pre-flashed devices.

Achieving Local Control with MQTT

Local control devices often use direct API calls or Message Queuing Telemetry Transport (MQTT). MQTT is a widely used protocol for local IoT communication. It involves three components: the sender (triggers action), the broker (message hub), and the device (performs action) using a publish-subscribe model.

A message is sent to a topic on the broker, and devices subscribed to that topic receive the message. MQTT supports Quality of Service (QoS) states:

MQTT also has a "retain flag" which, when enabled, means the last message on a topic is stored by the broker, ensuring new subscribers receive the latest information.

Diagram Description: MQTT QoS 0 - A client publishes a message to a broker.

Diagram Description: MQTT QoS 1 - A client publishes a message, and the broker acknowledges receipt (PUBACK).

Diagram Description: MQTT QoS 2 - A client publishes a message, the broker acknowledges (PUBREC), the client confirms receipt (PUBREL), and the broker confirms completion (PUBCOMP).

Wrapping Up

Home Assistant is powerful software that requires understanding fundamental technologies for troubleshooting and coordination. Future articles will cover wireless protocols (Zigbee, Z-Wave, WiFi) and installing Home Assistant.

Links:

How to Choose a Wireless Protocol for Home Automation

This article discusses the three dominant wireless protocols for smart devices: WiFi, Z-Wave, and Zigbee, and helps users choose the right one for their home automation setup.

Connecting Devices to Home Assistant

Devices connect to Home Assistant via cloud services, central hubs, or directly over the local network. Wireless connectivity is key for device usefulness. The three dominant protocols are WiFi, Z-Wave, and Zigbee.

Note on Wireless Spectra: Higher frequencies (GHz) transmit more data faster but are more susceptible to interference and have poor penetration through solid objects. Lower frequencies travel further and penetrate better but transmit less data.

WiFi

WiFi is the most widely known standard, leveraging existing home routers. It requires no additional hardware for IoT devices. Setup is typically done via a vendor app or web browser.

Most low-to-moderate priced IoT devices use the 2.4GHz spectrum, which is crowded with many devices, leading to potential interference and reduced performance. WiFi access points can only communicate with one client at a time, meaning more devices can lead to longer wait times.

Pros: Ubiquitous, inexpensive, easy setup, easy range extension, uses existing network, requires no hub.

Cons: Interference from neighboring devices/networks, uses crowded 2.4GHz spectrum, router limits device count, higher power consumption (less battery-friendly), potential latency for sensitive activities, most products require internet connection.

Z-Wave

Z-Wave is a closed wireless protocol ensuring guaranteed interoperability between devices from different manufacturers. It operates in the 0.9GHz spectrum, offering a larger range. A central hub is required. Z-Wave uses a mesh network topology where devices act as repeaters, potentially increasing coverage. It limits signals to 4 hops to reduce latency but requires strategic placement for optimal network paths. Lower frequencies limit speed and data capacity.

Pros: Guaranteed compatibility, mesh network, low power/battery-friendly, reliable mesh with more devices, uses 0.9GHz (longer range), least susceptible to interference from solid objects.

Cons: Closed protocol, higher cost, maximum of four hops in mesh, supports up to 230 devices per network, uses slowest 0.9GHz spectrum.

Zigbee

Zigbee is an open standard, allowing manufacturers flexibility. However, different implementations can lead to interoperability issues where devices may not "understand" each other, potentially creating multiple interfering networks. Like Z-Wave, it uses a mesh network topology but has no hop limit, theoretically supporting over 65,000 devices. It can communicate faster than Z-Wave.

Pros: Open standard, mesh network, low power/battery-friendly, supports over 65,000 devices, faster communication than Z-Wave.

Cons: No guaranteed compatibility, can form separate interfering mesh networks, uses oversaturated 2.4GHz spectrum, transmits only 10-30 meters.

Pick Your Protocol

Choosing the right protocol (WiFi, Z-Wave, or Zigbee) is more important than focusing on specific device types. The next article will cover the initial installation and configuration of a Home Assistant virtual machine.

Links:

Set Up Home Assistant to Manage Your Open Source Smart Home

This article guides users through installing and configuring Home Assistant.

Setting Up the VM

The author uses a virtualized environment and the QCOW2 image for Home Assistant. When creating the Virtual Machine (VM), it's crucial to set the boot type to UEFI. The package edk2-ovmf (or ovmf on Ubuntu) is required. Ensure the "Customize configuration before install" box is checked.

Image Description: A screenshot of the VM creation dialog in Virt-Manager, showing options like Name, OS, Install Media, Memory, CPUs, Storage, and a checkbox for "Customize configuration before install".

Image Description: A screenshot of the VM customization dialog, highlighting the "Firmware" setting set to "UEFI x86_64".

Important Note: The firmware type cannot be changed after the VM is created. Choosing BIOS will prevent the VM from booting.

To expand disk space, shut down the VM and use qemu-img resize hassos_ova-4.13.qcow2 +40G. The space will be allocated upon boot.

The first boot can take time as HA downloads and configures software. Allow at least 10 minutes for the system to start and settle.

First-Time Setup

Access Home Assistant via http://homeassistant.local:8123 (if your router supports automatic DNS) or its IP address (e.g., http://192.168.122.90:8123).

Create a user account by entering a username and password.

Image Description: A screenshot of the Home Assistant welcome screen asking to create a user account with fields for Name, Username, and Password.

Next, select a location for your home. This information is used for sun-based automations (sunrise/sunset times) and is not shared outside your network. The graphical map is used to set the location.

Image Description: A screenshot of the Home Assistant location setup screen, showing a map and fields for Time Zone, Unit System, and Elevation.

After completing this, a confirmation screen appears.

Integrations (devices and services) can be set up now or later from the configuration screen.

Image Description: A screenshot of the Home Assistant integrations setup screen, with options to set up devices and services, and a "More" button leading to "FINISH".

The author prefers to click "Finish" to configure devices using the full UI. The default interface is called Lovelace, which is YAML-described. The Lovelace default view shows weather information.

Clicking the username in the bottom-left panel accesses user settings.

Image Description: A screenshot of the Home Assistant Overview page showing weather information (Sunny, 18°C, 61% Humidity) and a user profile menu in the bottom left.

Advanced Mode and Configuration Validation

Enable "Advanced Mode" in the user settings to unlock more features, including syntax checking for HA configuration files.

Navigate to "Server Controls" under "Configuration" to perform a configuration validation. A "Configuration valid!" message appears if there are no errors.

Image Description: A screenshot of the Home Assistant configuration screen, showing various options like Overview, Map, Logbook, Developer Tools, Supervisor, Configuration, and Server Controls.

Image Description: A screenshot of the Home Assistant Server Controls screen, showing a "Configuration validation" section with a "CHECK CONFIGURATION" button.

Looking Ahead

Future articles will cover installing add-ons, creating snapshots, updating HA, using the Home Assistant Community Store (HACS), configuring entities, working with MQTT, and creating automation flows with Node-RED.

Links:

Integrate Devices and Add-ons into Your Home Automation Setup

This article guides users through setting up initial integrations and installing add-ons in Home Assistant.

Set Up Initial Integrations

Home Assistant coordinates devices from various manufacturers. The author demonstrates adding a Sonoff Zigbee Bridge, which had its stock firmware replaced with Tasmota for decoupling from the cloud.

Navigate to the "Configuration" menu, then select "Integrations". Click "Add Integration" and search for "Zigbee".

Image Description: A screenshot of the Home Assistant Integrations screen, showing a search bar and the "Zigbee Home Automation" integration.

The author used DigiBlur's Sonoff Guide for firmware replacement. For integrating the Zigbee Bridge, the serial port needs to be specified. The author also mentions using a CC2531 USB stick for other Zigbee devices, which appears as a Serial Device.

Image Description: A screenshot showing the configuration for the Zigbee Home Automation integration, with options to select the serial port and enter manual details.

The next step is to choose the radio type (e.g., EZSP, Legacy TI_CC, XBee, ZNP, ZiGate, deCONZ). The author uses EZSP for the Sonoff Bridge.

Image Description: A screenshot showing the "Radio Type" selection for Zigbee Home Automation, listing various protocols.

Finally, the IP address, port, and speed of the Sonoff Bridge are needed. The author uses DHCP to ensure a consistent IP address for the device.

Image Description: A screenshot showing the settings for the Zigbee Home Automation integration, including "Serial device path" and "port speed".

After adding the Bridge, devices can be paired by ensuring they are in pairing mode. The Bridge will find them.

Image Description: A screenshot showing the "Initialization Complete" status for a Door Sensor 1, with options to change device name and assign an area.

Devices are displayed as cards on the Integrations tab, grouping information related to specific entities.

Image Description: A screenshot showing the "Zigbee Home Automation" card, indicating "2 devices and 2 entities".

Add Functionality with Add-ons

Home Assistant offers many add-ons for extended functionality. While commercial products might not need them, custom sensor creation often does.

Add-ons are accessed via the "Supervisor" menu, then the "Add-on store" tab.

Image Description: A screenshot of the Home Assistant Supervisor menu, highlighting the "Add-on store" tab.

The author lists several useful add-ons:

The author highlights File Editor for convenient configuration file management, Samba share for accessing backups, and Mosquitto broker for MQTT client management.

Grafana and InfluxDB are optional for visualizing data. ESPHome is for creating custom sensors.

Image Description: A screenshot of the Home Assistant Add-on store, showing various official and community add-ons like File Editor, Mosquitto broker, Node-RED, Grafana, and ESPHome.

Image Description: A screenshot showing system usage statistics from Grafana, including disk usage, memory usage, and load average.

Node-RED

Node-RED is preferred for automation flow handling due to its visual flow editor. Most add-ons require configuration, such as setting a password for Node-RED.

Image Description: A screenshot of the Node-RED add-on configuration screen, showing fields for setting a password.

Important: Using secrets.yaml for passwords is an option, but the official documentation should be consulted.

Add-ons with web UIs often default to ssl: true. The author sets this to false for local networks.

After configuration, enable and start the add-on.

Image Description: A screenshot of the Node-RED add-on main screen, showing options to Start, Watchdog, Auto update, and Show in sidebar.

Wrapping Up

The UI makes configuration steps relatively painless. With HA installed and basic add-ons configured, the next step is integrating custom IoT devices and exploring Node-RED flows.

Links:

How to Set Up Custom Sensors in Home Assistant

This article dives into YAML files to set up custom sensors in Home Assistant.

Examining the Configuration Files

Several configuration files are important. While configuration.yaml can hold everything, splitting into dedicated files is recommended for large installations. The author uses the File Editor add-on to manage these files.

Image Description: A screenshot of the File Editor add-on in Home Assistant, showing the file structure within the /config directory, including configuration.yaml, automations.yaml, secrets.yaml, etc.

The notation script: !include scripts.yaml indicates that Home Assistant should reference the contents of scripts.yaml for script objects. The /config folder contains key files.

The author added lines to configuration.yaml to include separate files for input_boolean, binary_sensor, and sensor definitions.

Image Description: A screenshot of the Home Assistant File Editor, showing the /config folder structure and highlighting the inclusion of binary_sensor.yaml in configuration.yaml.

MQTT settings are configured in configuration.yaml.

Image Description: A screenshot showing MQTT configuration in configuration.yaml, including discovery, prefix, broker IP, username, and password.

Remember to save edits using the disk icon.

The /config/binary_sensor.yaml File

This file defines binary sensors, which have two states (on/off). The author uses binary sensors for door, motion, and power detection, sending data to MQTT.

Example binary sensor definitions include:

state_topic is where the device's state is published. The path is arbitrary, often following location/state/object convention.

Image Description: A screenshot showing the creation of a new file in the Home Assistant File Editor, with a dialog to name the file (e.g., binary_sensor.yaml).

Image Description: A screenshot showing the configuration.yaml file with lines including binary_sensor: !include binary_sensor.yaml.

The name field is used to reference the device within Home Assistant, as seen in Lovelace interface cards.

Image Description: A screenshot of a Home Assistant Lovelace entity card configuration for binary_sensor.brmotion.

qos (Quality of Service) refers to how an MQTT client communicates with the broker. payload_on and payload_off are determined by firmware.

The /config/sensor.yaml File

Sensors in this file can have more complex data payloads. The author uses value_template to extract specific data points from JSON payloads.

Example for a DHT temperature sensor:

The same sensor publishes multiple statistics (temperature, humidity, dewpoint) and can include data from different sensors (e.g., DHT11 and BH1750 light sensor) on the same chip.

value_json.<component>.<data point> is used to retrieve specific data from the JSON payload.

The author also includes system monitoring resources in sensor.yaml (disk usage, memory free/used, processor use).

Image Description: A screenshot showing the definition of a DHT temperature sensor in sensor.yaml, including value_template to extract temperature data from a JSON payload.

Image Description: A screenshot showing the configuration.yaml file with lines including sensor: !include sensor.yaml.

The /config/input_boolean File

Input booleans track states (on/off, home/away) and are used extensively in automations. The author defines several input booleans for presence detection and device states.

Image Description: A screenshot showing a list of input boolean definitions in a YAML file, including steve_home, steve_in_bed, guest_home, etc.

These can be used to create badges in the Lovelace UI to toggle states and inform automations.

Image Description: A screenshot of the Lovelace UI showing various badges representing input booleans like "steve", "guest", "TV Power Status", "It's Bedtime".

Wrapping Up

By examining YAML configuration files and adding custom sensors, users can create functioning automations with Home Assistant and Node-RED. Future articles will cover Node-RED flows and basic automations.

Links:

Protect Your Home Assistant with These Backups

This article covers snapshots, backups, and backup strategies for Home Assistant to ensure quick recovery from failures.

Backups vs. Copies

A copy is a representation of a point in time, whereas a backup is designed to retrieve protected files, often creating an exact replica. "One is none" applies to backups; having multiple files over time is crucial for identifying issues.

Backups and Snapshots

Home Assistant uses the term "snapshot" for what is traditionally considered backups. These are created by Home Assistant itself, but users must store them securely.

Set a Good Backup Strategy

A good backup strategy involves more than just creating backups; it requires testing them. The author shares an anecdote about a client who performed backups correctly but neglected testing, leading to missed data due to a software upgrade. This highlights the importance of testing to ensure backups are valid and to identify when changes occurred.

A typical strategy includes daily, weekly, and monthly backups. The author recommends taking a backup before making any system changes.

Create Snapshots

Snapshots are created via the "Supervisor" tab, then the "Snapshots" menu. Users can choose between a "Full snapshot" or a "Partial snapshot," which allows selecting specific folders (e.g., Home Assistant configuration, SSL, Share, Media, Local add-ons) or add-ons.

Image Description: A screenshot of the Home Assistant Supervisor menu, highlighting the "Snapshots" tab.

Image Description: A screenshot showing the "Create snapshot" interface, with options for Name, Type (Full/Partial), Folders, and Add-ons.

Image Description: A screenshot showing available snapshots in Home Assistant, with names, types, and timestamps.

After creating a snapshot, it's crucial to save these files securely in a different location, as file names are randomly generated. The author suggests renaming them for easier management.

Image Description: A screenshot showing files being accessed via Samba share on a GNOME desktop, with Nextcloud integration for cloud backup.

My Homelab Strategy

The author's homelab setup involves VMs on a Linux host using KVM, with a network-attached storage (NAS) for dual 10GB network cards. While overkill for Home Assistant, this infrastructure is used for other services. Home Assistant runs well on less powerful hardware like a Raspberry Pi 4.

The author's backup approach includes using the Home Assistant snapshotting function, storing four weeks of data on the VM, and then using Samba to copy snapshots to a GNOME desktop. Nextcloud is used for cloud backup, and SpiderOak One Backup clients link backups across multiple hosts for privacy and resilience. A cold copy is kept on an external hard drive, and occasional replication to an offsite location is also performed.

Image Description: A diagram illustrating the author's homelab backup strategy, showing VM Hosts, NAS, Nextcloud, Desktop, and SpiderOak.

The author also notes that while the process is extensive, the built-in versioning and multiple copies make recovery highly probable. Automation of this process is something the author plans to improve.

Image Description: A diagram showing the network-attached storage (NAS) setup with dual 10GB network cards and VM hosts.

The author also keeps important setup information on a private wiki (Wiki.js).

Wrapping Up

Backups are essential for Home Assistant as it's a critical component of the home automation infrastructure. While Home Assistant is generally stable, external factors can cause issues. A good backup strategy is recommended for making HA a central part of the house.

Image Description: A screenshot of a Node-RED flow, illustrating automations for a "Bedtime" sequence.

Links:

PDF preview unavailable. Download the PDF instead.

a practical guide to home automation using open source tools Adobe PDF Library 10.0.1 Adobe InDesign CS6 (Macintosh)