Manipulation with Whole-Arm Tactile Sensing
We are attempting to create a new foundation for robot manipulation that encourages contact between the robot's arm and the world. A key aspect of our approach is that we assume the robot has compliant joints and whole-arm tactile sensing. To help promote the development of this approach, we have released the following open source code to control robot arms and open hardware that can cover robot arms with touch sensors. The code and hardware are products of our research funded by DARPA and the National Science Foundation (NSF).
Open Source Software: Control Robot Arms with Whole-Arm Tactile Sensing
Our open source software includes code to control robot arms to reach in clutter while keeping contact forces low, as described in our 2013 IJRR paper, "Reaching in Clutter with Whole-Arm Tactile Sensing"
. It also includes code to interface with tactile sensor arrays and visualize their output, including all the code required to use our open hardware tactile sensors that we describe below. The code can be found via the following link:
open source code to control robot arms with whole-arm tactile sensing
Documentation for the code can be found via the following link:
documentation for the code
Open Hardware: Stretchable Fabric-Based Tactile Sensing
Our approach assumes that the robot arm has compliant joints and whole-arm tactile sensing (robot skin). Robots with compliant joints have become common in research and commercial products exist. Yet, relative to compliant joints, whole-arm tactile sensing for robots is rare and the technology is less mature. To support our research, we have developed tactile sensor arrays that are easy to fabricate, easy to use, and inexpensive relative to other large tactile sensing arrays. This has enabled us to quickly prototype tactile sensor arrays and give multiple robots whole-arm tactile sensing.
Our main focus is not tactile sensor development and our sensors have limitations. However, we have found that they are sufficient to support our research and enable robots to perform useful tasks, such as reaching near the body of a user. We have released our sensors as open hardware with the intention of supporting researchers and hobbyists, although anyone is welcome to use our designs. We hope that the sensors and accompanying software will make it easier for people to build on our research.
If you use our open-hardware tactile sensor design or wish to refer to it, please cite the following paper:
The main contributors to the development of this hardware were
Dr. Advait Jain, Sarvagya Vaish, Jeff Hawke, and Prof. Charles
Dr. Advait Jain was the lead student for the first
versions of the hardware for Cody and the PR2. Sarvagya Vaish was also
a student contributor to the first versions. Sarvagya Vaish and Jeff
Hawke were the main student developers of the advanced tactile array.
1. Basic Principle
A single sensing element or taxel (short for tactile pixel) consists
of five layers of fabric. The layer in the middle is resistive fabric, which is
sandwiched between two layers of conductive fabric. The taxel exhibits
piezo-resistive properties. Specifically, the resistance between the
two electrodes made of conductive fabric decreases when we apply a
force on the sensor.
We use this taxel as part of a resistive voltage divider that we
connect to the analog input pin of an Arduino board.
2. Steps to Make a Single Taxel
Having a picture or wiring diagram associated with each step would be geat.
- Stick a patch of iron-on adhesive to the conductive fabric
- Cut two pieces of conductive fabric to serve as the two
electrodes; one for the ground plane and the other for sensing area of the taxel
- Stick the conductive fabric to the non-conductive fabric using iron-on adhesive
- Sew the edges using a zig-zag stitch. The glue will no longer hold the conductive fabric in
place when the sensor stretches, but makes sewing a lot easier.
- Add female snap buttons to the two layers of conductive fabric. Connect wires to the conductive fabric using male snap buttons
- Put a layer of resistive fabric between the conductive fabric
- Wire up an Arduino as shown in the schematic below and upload this code to it.
- Run this python script after starting roscore. Visualize using rxplot with the following command:
rxplot /arduino/ADC/data -b 15 -r100 -p5
3. Making a Tactile Sensor Array
To make an array of tactile sensors, we change one of the layers of
conductive fabric to have multiple discrete electrodes of the desired
shapes and sizes. For example, 15 taxels that are rectangles of size
3cm x 5cm.
The second conductive fabric serves as a common ground for all the
We also use a single sheet of resistive fabric on top of the common
3.1 Simple Tactile Sensing Array
To create a simple array of tactile sensors, we connect the taxels to
snap buttons using conductive thread (as shown in the picture below),
or put the snap buttons directly on the taxel and use normal wires (method
that we used for the PR2). We then connect them to an ADC port on an Arduino board.
3.2 Advanced Tactile Sensing Array
To create an advanced array of tactile sensors, we attach snap buttons on the taxels,
but instead of connecting them to an ADC port on an Arduino board, they are connected to
an 8-channel ADC chip mounted on the skin. The I2C communication protocol is used to
connect the ADC chips to the Arduino. Up to four ADC chips can be daisy-chained and
then a single set of Power, Ground, SCL, and SDL wires connect the ADC to the arduino.
This helps reduce the wiring from the sleeves to the Arduinos.
The ADC chip, bought from Sparkfun, is mounted on a custom PCB board. The PCB board also
houses in-line pullup resistors and connectors. The entire PCB board is stitched on to the
sleeve to prevent it from moving, as shown below. Since each ADC chip has 8-channels, eight
wires branch out from the PCB, connecting to the snap buttons on eight different taxels.
4. Cody Specific Design
4.1 Simple Taxel Layout
To cover Cody's arm with our tactile sensors, we used one sleeve with
12 taxels in the forearm and 12 taxels in the end-effector region.
for all SolidWorks files.
The following are the images showing the taxel arrangement for Cody's sleeve.
5. PR2 Specific Design
5.1 Simple Taxel Layout
To cover the PR2 arm with our tactile sensors, we have different
designs for the gripper (10 taxels), forearm (14 taxels), and upper
arm (3 taxels).
- The picture on the left shows how the taxels are arranged
- The picture on the right shows how the cutouts for the ground plane, resistive fabric and outer and inner coverings are arraged.
for all SolidWorks files
for all SolidWorks files
for all SolidWorks files
5.2 Advanced Taxel Layout
For the advanced taxel layout, the number of taxels on the gripper and the upper arm are not changed.
The number of taxels on the forearm were increased to 22 taxels. This was done by adding two taxels
on each of the four edges, thus adding a total of eight taxels.
Forearm Design Files
for all SolidWorks files.
The following are the images showing the taxel arrangement for different sides of the forearm.
6. Raw Materials and Equipment
Here are the places where we purchased the different raw materials and
This work was support by DARPA Maximum Mobility and Manipulation (M3) Contract W911NF-11-1-603 and National Science Foundation (NSF) Award IIS-1150157.
Our design is inspired by the rSkin
project. We thank Hannah Perner-Wilson and Ian Danforth for helpful discussions.
We also thank Matt Carney and Aaron Edsinger from Meka Robotics for
pointing us to the rSkin project.
10. Useful Links