Tangible interface designer and inventor Andrea Bianchi, along with his colleague, Ian Oakley (University of Madeira / Carnegie Mellon Europe), have come with a novel approach to interacting with a mobile device. Using the magnetometer built into most modern smartphones, Bianchi and Oakley have created a series of tangible user interface demonstrations that go beyond what’s achievable with capacitive touch displays.
We caught up with Andrea over the weekend as he prepares to deliver a presentation at the upcoming ACM TEI 2013 conference in Barcelona to ask him a few questions about his technique.
MAKE: How does this differ from capacitive touch tokens?
Bianchi: First of all, capacitive tokens need to occupy (often relatively large) portions of the screen, while magnetic tokens can be located anywhere around the screen. In the video many tokens are located on the screen to simplify the calibration process: since the location of the token is know in advance (there is a “put the token here” marker on the GUI), then it is trivial to detect the position/orientation of the magnet. However, all the techniques shown in the video can work equally well off screen (for example assuming that we put all the tokens on the left or right of the screen).
Moreover, the capacitive tokens cannot be passively sensed, requiring either human contact or active electrical components to simulate finger touches (here an example). However, magnetic tokens do not require the user to keep touching them, nor they are active components (not batteries, just magnets…)
MAKE: Assuming you don’t have to touch the device for this technique to work, what is the practical distance it can be used?
Bianchi: It depends on the strength of the magnets and the interferences of other magnetic fields. Assuming there are not other strong magnets around, usually the device will sense the Earth magnetic field (0.25 to 0.65 gauss). The 3 magnets we used were very strong in comparison (Small: thickness 2mm, diameter 0.5 cm, 400 gauss; Medium: thickness 2mm, diameter 1 cm, 1000 gauss; Large: thickness 2mm, diameter 2 cm, 1500 gauss) so we had no problem detecting them, and you can imagine that we could have used a much richer set of identifiable magnets. To avoid noise and keep detection reliable, I would say that empirically we found that 10cm from the device border is about as much as you want to get, but this can be probably improved.
MAKE: What is the smallest magnet you have used? How about the largest? Does physical size matter?
Bianchi: Magnetic fields have two properties: strength and direction. Strength is the intensity of a magnetic field and it varies from magnet to magnet and with distance. Direction reflects the fact that magnets have a north and south pole. Flipping a magnet inverts the poles, causing substantial changes to the magnetic field. Both strength and direction can be measured with a gauss-meter or via magneto-meters and compass sensors. Hence, the magnet physical size (see previous question) matters. The strength of a magnetic field is affected by the size of a magnet. Our software simply sense measurable changes on the magnetic field and try to use them to build novel interactions for exploring the design space. We could have used more than three magnets, but we just adopted a simple approach for the demo.
MAKE: How many magnets can you use at once?
Bianchi: Our techniques leverage on the detection of the magnetic field strength (e.g., detecting position, or size, linear movement), or orientation (e.g, flipping, rotational movement), or both (e.g., orientation). Since the software only reads a cumulative value of the magnetic field strength and direction, it cannot know if for instance a “small intensity” is due to a small magnet or a strong magnet that is far away. Generally though you can use multiple magnets if you measure different (orthogonal) properties (e.g., one magnet will be used for the strength, one for the direction) or if they are used together to achieve a combined effect (e.g, snapping two magnets together makes a stronger field, so we can identify this action). I attach a table that show you how these techniques can be combined together. So for instance, “flipping and position” or “flipping and identification” leverages on orthogonal properties, so we can use 2 magnets at the same time. “Position and identification” leverages both on the magnetic strenghts so only one token at a time can be used. This problem can be solved either introducing a more complex calibration, or constraints on the movement (e.g., only few targets) or using actively powered magnets (e.g, solenoid) which could pulse at different identifiable frequencies.
MAKE: Can this break my device?
Bianchi: It is usually good to keep (strong) magnets away from electronic devices. In practice though, I did not find any damage or malfunctioning of my devices (phone and tablets) during or after the usage of magnetic appcessories.
MAKE: Are you using a particular platform to develop and if so, why?
Bianchi: We used Android on a Samsung Galaxy Tab simply because developing a prototype for Android is extremely simple. All this work was basically built in few days.
MAKE: How can I get started using this in my app?
Bianchi: We have not released an app yet, but we are considering working on a open source toolbox to help other developers working with magnets. This work is however still very young and require some few iterations. Extensions of this work will investigate better ways to calibrate magnets, tokens that snap together (creating recognizably stronger magnets), explore the potential of active magnetic tokens (e.g., electromagnets pulsing at different frequencies) to create larger sets of uniquely identifiable tokens and combine magnetic sensing with capacitive sensing.
MAKE: What are some of your favorite examples of this technique in practice?
Bianchi: When I was designing these tokens I was thinking to use them for DJing. So, that’s the inspiration for sliders (faders) and wheels (volume and gain controllers) or even menu selections. The main idea is that we could use tangible interaction with commonplaces devices for some activities (e.g., DJing) that seem to work much better with physical widgets than not “beyond-the-glass” graphical interfaces. I also can imagine how magnetic appcessories could be used for making toys.
I make cool stuff and write about other people making cool stuff on makezine.com. If you have something you think I should see, send me a tip.
