The Force is Strong with Sphero (and Apple Watch)

R&D engineer at ELEKS

The saga continues. Remember when we awoke the Force with Kinect and Sphero? Right after that we received a similar request. We’re not gonna disclose the previous customer (don’t worry, Darth, you’re fine), but this time the order came from Master Yoda himself. When he saw Sphero’s new Star Wars watch, Master reasoned: why use an additional device when you already have Apple Watch on your wrist?

Apple Watch need you. Force feel you must…
Why not Android Wear?
Argue with the Master must you not. Use Apple Watch!

Really, who are we to argue with the powerful old green man? So, we started developing an app that will allow controlling Sphero with Apple Watch.

The_Force_is_Strong_with_Sphero_Apple_Watch_2

WatchOS 2 and How to Track the Power

Fortunately for us, starting from WatchOS 2.0 developers have access to raw accelerometer data. Which is great because we will use it to track the changes in wrist movement.

The basic idea is to:

  1. Extract the accelerometer data from Apple Watch.
  2. From that data get the states (positions of the watch relative to the ground) that could be used as the control units (control movements).
  3. Send the corresponding movement action to an iPhone.
  4. Use Sphero SDK to send the command from an iPhone to Sphero.

Implementation

So, we have some data and now we need to extract a few patterns from it to create some basic movements. There are several ways it can be implemented, one of the easiest being to use the g-force vector. Because raw accelerometer data is returned in the format g + user acceleration (acceleration performed by the user), we can get the basic movements which we are interested in.

  • abs(acceleration.z) == g. Wrist is parallel to the ground (watch screen is parallel to the ground). Sphero moves forward.
  • acceleration.x == g. Wrist is pointing to the ground. Sphero moves backward.
  • acceleration.x == -g. Wrist is pointing to the sky. Sphero stops moving.
  • acceleration.y == g. Wrist is parallel to the ground and rotated 90 degrees clockwise. Sphero moves right.
  • acceleration.y == -g. Wrist is parallel to the ground and rotated 90 degrees counterclockwise. Sphero moves left.

Using these four states we can build a simple state machine which will manage the behavior of Sphero.

Basically, the list of technologies we used is simple: Swift, CoreMotion, Sphero SDK, WatchKit and WatchConnectivity.

What We Couldn’t Do…

Unfortunately, we can’t use bluetooth to send the signal directly from the smartwatch to Sphero. That is why we used the basic approach for the iPhone – Apple Watch communication. Maybe the next release of Apple Watch OS will give us better opportunities. By the way, Android has no such issue :)

… and What Can Be Done

We used only 4 states to manage Sphero, but using simple machine learning can help create a wider set of patterns, like the guys did here.

How to Use the Force

The_Force_is_Strong_with_Sphero_Apple_Watch_3

  1. Turn Sphero on. Aim Sphero
  2. Start the application on your iPhone. Check if Sphero is connected.
  3. Start the corresponding app on your Apple Watch.
  4. Move your wrist to control Sphero.

Click here to see the code on GitHub.

tags

Comments