Home-automation follow-along

I’m experimenting with Home Automation using Python. For those of you curious about how it works and how confused you’ll need to get, I wanted to provide this little tutorial/follow-along. You don’t have to participate and you can skip bits you don’t care about.

NOTE: The purpose of this post is to show you the workings behind the workings, you aren’t going to have to get your hands this dirty to work with most home automation systems.

Python-Hue and Python-Homeassistant

I’ll be demonstrating talking to a Hue hub and my local Home Assistant install. If you’re using some other hub (Wink, etc) it is left as an exercise for the reader to find the appropriate ways to talk to that API. You should still be able to follow along in spirit.

Dumb vs Smart?

The prefix, “Smart”, for a light bulb, motion sensor or pretzel mulcher is generally an allusion to a device that can participate in smart home automation.

The minimum bar for “Smart” is providing information to a controller, usually wirelessly. The next step is being able to be controlled the same way.

Motion sensor: A “dumb” sensor sits between the light it controls and the electricity supply. It acts like a physical switch. No motion? No power to the light. When it senses motion, it closes the switch and current flows to the light.

Whereas a “Smart” sensor detects motion and turns it into information that it sends to an intermediate device. Which will be called a “controller”, “gateway” or “hub” depending on precisely what that thing does.

In order for the sensor to turn a light on or off, a virtual connection is made by configuring or programming something to say “When the sensor sees motion, take this action”.

The fundamental value in “Smart” is the ability (and perhaps desire) to make decisions such as “if the sensor sees motion and it is dark then turn on the patio light”.

iPoke it with iPython

I’m going to be using Python to do all of this, iPython Notebook in particular: if you’re running Win/Lin/Mac it’s pretty easy to get set up.

Install iPython[notebook]

Windows:

  • Install a version of Python selecting to add it to the path as you go,
    Choose 3.x if you know nothing about Python,
    or if you have to have Python 2.x make sure you choose 2.7.5 or higher,
  • Open a command prompt (Press the window key and type: cmd then hit enter),

Mac:

  • Congratulations, you already have Python. You just need the package installer, so you’ll need to open a terminal. (⌘+space and then type term)
  • If you’re using a “brew”ed python, type “brew install –upgrade pip” — if you’re not sure what “brew”ed means, pass this step,
  • Type “pip” to see if it’s installed. If not, try:
    sudo easy_install pip

Linux:

  • If you don’t already have pip, try one of
    sudo easy_install pip
    
    sudo apt-get install python-pip
    
    sudo yum install python-pip

Once you have pip, all you need to do is (at the command prompt):

pip install --upgrade ipython[notebook]

You may need to try one or other of

pip install --user --upgrade ipython[notebook]

or mac/linux:

sudo -H pip install --upgrade ipython[notebook]

Make it go, already

To start an iPython Notebook session, run the command

ipython notebook

and after a few moments a web-browser should open pointed at the iPython instance, which looks a lot like a directory listing at first:

ipy1

Click on ‘New’ to get started…

ipy2.png

If you’ve used iPython before, you’ll be semi-familiar with what happens next, which is a web-based presentation of an interactive python prompt…

ipy3

Notebooks are based on blocks of code called “cells”. They can be a single line or an entire program. Enter some simple code in here for the lulz:

ipy4

Here I’ve typed three lines that make the variable a have a value of 2 and the variable equal to 3, then multiplied them.

Click the run icon (see image) to execute the current “cell”.

ipy5

Now change “b = 3” to “b = 7” and run again. You can edit and re-run a cell at any point, and when you’re ready save the whole thing or export it as a script.

You don’t have to use iPython to follow along, but it’s easier to understand what we’re doing if you do.

Give it a REST

Your basic Smart Home system is going to be something like Ikea’s light bulb and “gateway”. You don’t actually talk to the bulbs, you talk to their controller.

This all happens via “REST”, which means using the same fundamental technology that makes your web-browser work. The hub/gateway pretends to be a website and assigns meaning to parts of the URL.

http://hub/lights/on/bedroom/window

When the hub’s fake-webserver gets this request, it doesn’t look for a file, instead it recognizes the first ‘directory’ is “lights” and treats the rest of the URL as a light-related request. The next directory is “on” so it treats it as a command to turn it on, “bedroom” specifies in which room and “window” which light.

The other half of REST is that the response you get is in JSON which a human can read but is designed to be used by machines.

Practice what you’re preaching already

If you already know about REST and JSON, you can skip or skim the next block down to “Talk to the … bulb, already!”

This is harder to explain than it is to understand once you see it.

We’re going to install the “requests” module and leverage the “JSON PlaceHolder” service; no home automation hardware required.

Install/upgrade the python ‘requests’ module:

import pip
pip.install("install --upgrade requests".split())

Hint: Pressing Ctrl+Enter runs the current cell, Shift+Enter runs the cell and adds a new empty one.

ipy6

A quick [ab]use of JSON PlaceHolder:

import requests
requests.get("https://jsonplaceholder.typicode.com/posts")

ipy7

If we did this with our browser: https://jsonplaceholder.typicode.com/posts (go ahead, I’ll wait) we’d get some odd-looking text.

But in python we get a “<Response [200]>”… Wat?

This is just iPython telling us the return value of the last statement it executed, with the ‘<…>’ indicating that the object is describing itself the way we see it. In order to make use of this, we’ll need to RTFM, but with python we can do that here and now. Edit that last cell:

import requests
result = requests.get("https://jsonplaceholder.typicode.com/posts")

hit Shift+Enter to run the cell and create a new one. Because we captured the result to a variable, there’s no return status, so our output line goes away:

ipy8

There are a number of ways we can find out more about this; since we’re using iPython we can ask:

result?    # will give us some more information about what 'result' is,
result??  # will give us more information and a view of the source code...

Both a bit advanced, lets use Python’s handy “help” function in the next cell to dump information from “docstrings” in the source code to tell us what properties and functions the object has:

help(result)  # hit ctrl+enter

ipy9

Here I’ve scrolled down to what looks pertinent: text! Add a new cell and type:

result.text

ipy10.png

This is the same text you would see in your web-browser if you went to the same URL (https://jsonplaceholder.typicode.com/posts).

It’s actually not text, though. If you query result.headers[‘Content-Type’] it’ll tell you that this is “application/json”. If you scroll back in the “help” for “result”, you’ll find that there’s a ‘json()’ method. But at first glance, all this appears to have done is remove some quotes from what we saw before?

ipy11

The result of the “json()” method is not text, it’s actually an object we can interact with. JSON itself happens to look a lot like the way dictionaries/maps are populated in various languages, including Python. Let’s use the “pprint” module to get a better look.

from pprint import pprint
pprint(result.json(), depth=1)

this imports the ‘pprint’ function from the ‘pprint’ module so we can use it as ‘pprint’ instead of ‘pprint.pprint’, and then it ‘pprint’s the first level of the json data. What we find is that it is a list (“[…]”) of dictionaries (“{…}”).

ipy12

Lets take a look at what the first (0th) element of this list looks like:

from pprint import pprint
json = result.json()
pprint(json[0])

ipy13

A dictionary maps names to values, in this case the names are strings (‘body’, ‘id’, ‘title’, ‘userId’). We can print just the title any of these ways:

pprint(json['title'])
pprint(json["title"])
field_we_want_to_see = "title"  # use any name you like
pprint(json[field_we_want_to_see])

Or we can print a list of the first 5 titles (remember, the top level of this particular json data is a list)

for entry in json[0:5]:  # remember, json is a list
  pprint(entry["title"])  # " and ' are interchangeable.

ipy14.png

What makes this REST is that the API we’re talking to is organized based on the type of data. In this case, “/posts” gives us a list of posts. But “/posts/3” will give us the post with ID#3:

import requests
# We can get post 3 by querying the list and getting entry #3
postsJson = requests.get("https://jsonplaceholder.typicode.com/posts").json()
viaPosts = postsJson[2] # remember: lists are 0-based, so [0] is id 1.
# Or we can use the API to fetch a single item...
viaId = requests.get("https://jsonplaceholder.typicode.com/posts/3").json()
print(viaPosts["id"], viaPosts["title"])
print(viaId["id"], viaId["title"])

ipy15.png

Talk to the … bulb, already!

Rather than talking directly to bulbs, we talk to the gateway, bridge or hub they are connected to.

In my case, I’m going to talk to my Philips Hue hub, which my network knows as “philips.lan”. Philips handily provide a little tool for testing json queries to it directly, called “CLIP”. For me, that’s http://philips.lan/debug/clip.html

hue1

Philips provide a great introduction to their API online, here.

If we try requesting “/api/newdeveloper”, we should get back information – because of the way Hue works – telling us that we’re not authorized. In the case of the Hue, you have to press the button to authorize a new connection.

hue2

This, believe it or not, will be our starting point, with the use of the ‘requests’ library and a simple piece of helper code:

import requests

hub = "philips.lan" # or provide the ip address

def query(query_path):
 result = requests.get("http://" + hub + query_path)
 return result.json()

And we use it like this:

query("/api/newdeveloper")

NOTE: Dodgy code alert! We’re tinkering right now, so this code doesn’t perform any checks.

hue3

The “REST” interface of path names only gets us so far. Beyond a certain point, we need to provide arguments (how bright do you want the light?).

Lets amp-up our query function and pass the Hue an important keyword for it’s API, “devicetype”:

def query(query_path, body=None):
    if body is None:
        result = requests.get("http://" + hub + query_path)
    else:
        result = requests.get("http://" + hub + query_path, json=body)
    return result.json()

query("/api", body={'devicetype': 'testapp'})

Which gives us

hue5.png

The description tells us what we need to know: we haven’t registered our app with the Hue’s security. Go ahead and press it and run the cell again. You’ll get a different answer.

hue6

Oops – we should probably have captured that username. Never mind, iPython to the rescue. Those numbers in []s are actually meaningful. You can refer to a previous output by using it’s number after an underscore; the number above is #14, and the its in []s so it’s a list, of one element.

hue7.png

A better way to do this would have been to capture it directly from the query:

result = query("/api", body={'devicetype': 'testapp'})
status = result[0] # why it's a list, I don't know...
if 'error' in status:
 print("ERROR: ", status['error']['description'])
 print(status)
elif 'success' in status:
 hue_username = status['success']['username']
 print("Success! Registered as", hue_username)

Run the cell, it’ll complain we’re unregistered:

huge8

Press the button and run it again.

hue9

Lets go ahead and leverage ipython to quickly make this a helper (code here).

hue10.png

Can we turn on a light yet?

First we need to find it, which means using the “/api/{username}/lights” call. This should give us a dump of the lights and allow us to search through it to find the “Desk Lamp”, shown here to be id 14 (unrelated to the 14 above):

hue11

But that’s going to get old quick, lets make that a function (full code here)

def hue(query_path, body=None):
    register()
    full_path = "/api/" + username + query_path
    return query(full_path)

And lets use that to short-cut to “/api/<username>/lights/{light_id}”

hue12

Great – the light is off. I could have told you that:

WIN_20170611_23_24_49_Pro

To issue a command via this particular REST API is simple: we make a request to “/api/<username>/lights/<light id>/state”, and we send a body with a description of what we want done.

But lets start with “On”, shall we?

hue("/lights/14/state", body={"on": True})

This gives us an error “method, POST, not available for resource, /lights/14”. Oops. It turns out that “GET” is for requests, “POST” is for creating things and “PUT” is for updates.

A little more code tinkering.

The first block is now this, centered on simplifying “query” and adding “update”.

def query(query_path, body=None):
    if body is None:
        result = requests.get("http://" + hub + query_path)
    else:
        result = requests.post("http://" + hub + query_path, json=body)
    return result.json()

def update(query_path, body=None):
    result = requests.put("http://" + hub + query_path, json=body)
    return result.json()

And we’ll have to change the ‘hue’ function.

def hue_query(query_path, body=None):
    register()
    full_path = "/api/" + username + query_path
    return query(full_path, body)

def hue_update(query_path, body=None):
    register()
    full_path = "/api/" + username + query_path
    return update(full_path, body)

But we are ready:

from pprint import pprint
pprint(hue_query("/lights/14")['state'])
pprint(hue_update("/lights/14/state", body={'on': True}))

hue13.png

Result?

WIN_20170611_23_49_52_Pro

Lets be fancy.

import time
# quick on and off
hue_update("/lights/14/state", body={"on": True, "bri":254, "sat":0, "transitiontime":0})
time.sleep(3)
hue_update("/lights/14/state", body={"on": False, "transitiontime":0})
time.sleep(4)

# slow on and off
hue_update("/lights/14/state", body={"on": True, "bri":254, "sat":0, "transitiontime":15})
time.sleep(3)
hue_update("/lights/14/state", body={"on": False, "bri":0, "sat":0, "hue":1000, "transitiontime":15})
time.sleep(4)

# some color effects
hue_update("/lights/14/state", body={"hue":30000, "transitiontime":0}) # set the initial hue
hue_update("/lights/14/state", body={"on": True, "bri":254, "sat":254, "hue":60000, "transitiontime":50})
time.sleep(5)
hue_update("/lights/14/state", body={"hue":1000, "transitiontime":50})
time.sleep(5)
hue_update("/lights/14/state", body={"ct": 153, "transitiontime":50})

And this is how it looks:

You can find the complete ipython notebook as a gist here (I modified it to ask you which light id you want to change): https://gist.github.com/kfsone/0c923f62672ed258319095c5bb468b38

So we’ve successfully talked to a hub, and we have a good idea of what’s going on under the hood. There are actually existing libraries for talking to something like the Hue, which make it much easier than this shonky works I’ve shown you, but again the goal was to learn how and why.

Once you understand that almost all home-automation systems are based on this same set of technologies, it becomes easier to think in terms of conducting the activities of your smart gear and what you should be able to expect…

 

Leave a Reply

Name and email address are required. Your email address will not be published.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

You may use these HTML tags and attributes:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <pre> <q cite=""> <s> <strike> <strong> 

%d bloggers like this: