openHAB + Heartbeat + Pacemaker + GlusterFS = clustered home automation nirvana!

Posted on Sat 25 July 2015 in Automation

image

I've been steadily building my home automation system/network. It's begun as a simple "Insteon Automation Starter Kit"Insteon Logo (Blue) and evolved from there.

One thing I realized from the gate was that the nucleus needed to be open source. With tech that comes and goes faster then you can change your underwear, it was important to me to build my abode efforts on solid foundation. Open Source brings that to the table with it's very nature. I was hoping to find a pure Python setup and did find some great ones! Ultimately I settled on openHAB (which is a self-contained Java stack and web serve) because it has been pushed further and wider then most of the other offerings. Here is an overview of the architecture:

image

Since I'm making my environment very modular, I can hopefully adapt a different platform later on if need be. Once I played around with openHAB and kicked the tires enough to satisfy my questions, it was time to begin assembling the brick shell of my technological house. While this will very much be a work in progress, I've gotten off to a good start I think. I am trying to make things low-powered, ultra-dense, and reliable. With these project goals in mind, I decided on a 3-node Raspberry Pi Cluster utilizing GlusterFS for shared storage.

Using three nodes will allow for proper "Quorum" to be established and buy "lazy time", should I suffer a hardware failure. Long-term the cluster will mate with 3x UPS's, and will contain a network bond of the ethernet and wireless ports for each node. Since the Pi's are so lite on the power consumption side, standard office ups's should provide a few days of backup power.

I still need to sort the network bonding out, so like stated before, this is very much a work in progress. I did really want to get this out into the wild though, so I can capture feedback and input from others with similar projects. I've created a rough-draft build sheet as I go along below. I'm also attaching the txt file since my site doesn't render all exactly as they should be executed.

:::bash ##Configure your wifi adapter on the mgmt node ## wlan0 iface wlan0 inet static address x.x.x.x< netmask xxx.xxx.xxx.x gateway 172.16.x.xxx wpa-ssid "my wireless ssid" wpa-psk "my wireless password" ##Configure static ip’s for each node in cluster ## sudo nano /etc/network/interfaces iface lo inet loopback iface eth0 inet static address xxx.xxx.xxx.xxx netmask xxx.xxx.xxx.xxx network xxx.xxx.xxx.xxx broadcast xxx.xxx.xxx.xxx gateway xxx.xxx.xxx.xxx ##Now add DNS servers## sudo nano /etc/resolv.conf nameserver 8.8.8.8 nameserver 8.8.4.4 ##Restart networking to reflect changes## sudo ./etc/init.d/networking restart ##Change hostnames for each node ## ##Edit the file /etc/hostname and change the name from the default (raspberrypi)to “clusnodeX” .## sudo nano /etc/hostname ##Edit the file /etc/hosts and replace the original name ##raspberrypi with the new name hostname ##and also the ip’s of other cluster nodes: sudo nano /etc/hosts 172.16.x.xxx clusnode1 172.16.x.xxx clusnode2 172.16.x.xxx clusnode3 ##Config SSH keys ##Create dirs on each node: mkdir ~/.ssh ##Run on clusmgmt node: ssh-keygen ##Copy the files to each node now: scp -p /home/pi/.ssh/id_rsa.pub clusnode1:/home/pi/.ssh/authorized_keys scp -p /home/pi/.ssh/id_rsa.pub clusnode2:/home/pi/.ssh/authorized_keys scp -p /home/pi/.ssh/id_rsa.pub clusnode3:/home/pi/.ssh/authorized_keys ##Install Heartbeat and Pacemaker ##(below will install both Heartbeat and Pacemaker ##with all dependancies) ##Make sure your system is up to date sudo apt-get update sudo apt-get upgrade sudo apt-get install heartbeat gawk-doc lm-sensors snmp-mibs-downloader ##Then we need to create the HA config files sudo nano /etc/ha.d/ha.cf debugfile /var/log/ha-debug logfile /var/log/ha-log logfacility local0 keepalive 2 deadtime 30 warntime 10 initdead 120 udpport 694 bcast eth0 node clusnode1 clusnode2 clusnode3 ping 172.16.x.xxx crm respawn ##Save the HA auth keys sudo nano /etc/ha.d/authkeys auth 1 1 crc ##Change the file permission for the authkeys file sudo chmod 600 /etc/ha.d/authkeys ##Restart cluster services on each node sudo service heartbeat restart ##Test changes sudo crm status ##Configure GlusterFS shared storage for openHAB ##Install GlusterFS server ## apt-get install glusterfs-server ##Create GlusterFS directory/brick location (DO NOT WORK INSIDE THIS DIR DIRECTLY or you'll corrupt your Gluster volume !!!) sudo mkdir /srv/glusexport1 ##Probe for each gluster peer. Establish's nodes in gluster cluster. ##Only needs to be run from first node, but probe all nodes in cluster) sudo gluster peer probe 172.16.x.xxx ## Create Gluster policy. define shared storage strategy. ## Include the primary node or one this is being run from. sudo gluster volume create glusexport1 replica 3 172.16.x.xxx:/srv/glusexport1 172.16.x.xx1:/srv/glusexport1 172.16.x.xx2:/srv/glusexport1 ## Now that vol is created, let's start it ## sudo gluster volume start glusexport1 ## Verify it was created correctly ## sudo gluster volume info ## Create local mounts for gluster drive ## sudo mkdir -p /opt/openhab sudo mount -t glusterfs 172.16.x.xxx:glusexport1 /opt/openhab ## Make mount permanent by editing /etc/fstab ## sudo nano /etc/fstab 172.16.x.xxx:glusexport1 /opt/openhab glusterfs defaults,_netdev 0 0 ##Install openHAB and a few other things # ##Check java version ## java -version ##Download and unzip openHAB distro to newly created directory wget openhab-runtime-.zip unzip openhab-runtime-.zip -d /opt/openhab ##Install Bindings ## http-binding-.jar ##Copy the default config template to useable starting config/state sudo cp /opt/openhab/configurations/openhab_default.cfg /opt/openhab/configurations/openhab.cfg ##Set openhab to be owned by root ## sudo chown -hR root:root /opt/openhab/ ##Make init.d script executable ## sudo chmod a+x /etc/init.d/openhab ## Test starting and stopping openhab before pushing config into Cluster Resource Manager## /etc/init.d/openhab start /etc/init.d/openhab stop ## !!! DO NOT CONFIGURE openHAB TO START AT BOOT, CLUSTER SOFTWARE WILL HANDLE THIS !!! ## ##Configure cluster parameters sudo crm configure property stonith-enabled=false sudo crm configure property expected-quorum-votes="2" sudo crm configure property no-quorum-policy=ignore sudo crm configure rsc_defaults resource-stickiness=100 sudo crm_attribute --type rsc_defaults --attr-name migration-threshold --attr-value 2 sudo crm_attribute --type rsc_defaults --attr-name failure-timeout --attr-value 30s sudo crm configure primitive clusterIP1 ocf:IPaddr2 params ip=172.16.0.xxx cidr_netmask=24 nic=eth0 op monitor interval=30s sudo crm configure primitive openhab lsb:openhab op monitor interval=20s ## Test changes sudo crm status ##Create cluster Group for resources sudo crm configure group HA-GROUP clusterIP1 openhab ## Show cluster status (will reflect new group after few mins) root@clusnode1:/etc/ha.d/resource.d# crm_mon -1 Last updated: Tue Jul 7 10:21:29 2015 Last change: Tue Jul 7 10:21:14 2015 via cibadmin on clusnode1 Stack: Heartbeat Current DC: clusnode1 (d45fed8b-8e47-4cdf-a580-3588c68b05bc) - partition with quorum Version: 1.1.7-ee0730e13d124c3d58f00016c3376a1de5323cff 3 Nodes configured, 2 expected votes 2 Resources configured. Online: [ clusnode2 clusnode3 clusnode1 ] Resource Group: HA-Group clusterIP1 (ocf::heartbeat:IPaddr2): Started clusnode1 openhab (lsb:openhab): Started clusnode2 root@clusnode1:/etc/ha.d/resource.d# crm_mon -1 Last updated: Tue Jul 7 10:21:54 2015 Last change: Tue Jul 7 10:21:14 2015 via cibadmin on clusnode1 Stack: Heartbeat Current DC: clusnode1 (d45fed8b-8e47-4cdf-a580-3588c68b05bc) - partition with quorum Version: 1.1.7-ee0730e13d124c3d58f00016c3376a1de5323cff 3 Nodes configured, 2 expected votes 2 Resources configured. Online: [ clusnode2 clusnode3 clusnode1 ] Resource Group: HA-Group clusterIP1 (ocf::heartbeat:IPaddr2): Started clusnode1 openhab (lsb:openhab): Started clusnode1 root@clusnode1:/etc/ha.d/resource.d# ```bash ## That's about it ## test failover and such. When you pull the plug on a node, ## it takes a few mins for the services ## to failover and come up fully on the next node