Hybrid Sensors in Robotics
by Bennett Brumson
, Contributing Editor
Robotic Industries Association Posted 07/19/2007
Robotic hybrid sensors, those sensors that have multiple types of detection systems, are not part of the mainstream in flexible automation. However, the use of hybrids, while small, is on the rise. More and more end-users of robotics believe hybrid sensors have a role in manufacturing. Hybrid sensors are especially useful for delivering more than one type of part information in a robotic manufacturing process.
A hybrid sensor is defined by Dr. John R. Wright, Jr., PhD as having ‘‘multiple sensors and multiple processing techniques to obtain more information than one could achieve from independent sensors.’‘ Dr. Wright, an Associate Professor of Automation and Electronics Technologies at the University of Pennsylvania, Millersville, uses a simple analogy to describe hybrid sensors: ‘‘1+1=3.’‘
And The Survey Said...
In the Fall of 2006, the Robotic Industries Association (RIA), Ann Arbor, Michigan, with the help of Dr. Wright, undertook a survey of member companies on the potential market for hybrid sensors. The data shows that almost 75% of respondents believe that hybrid sensors would be useful in their applications.
RIA, the only North American trade association solely dedicated to robotics, offers results of the survey free to its members. Others may purchase it for $1,395.
According to the survey, the top three points of interest for hybrid sensors are vision, infrared/laser and photoelectric applications. The survey also asked what the benefits of deploying hybrid sensors would be. Adding functionality to a robotic work cell and ease of integration topped the list of perceived benefits.
The potential market for hybrid sensors will come from developing vision systems with supplemental functionality to undertake applications that are currently problematical for current vision technology. Nearly 75% of those answering RIA’s survey indicate that vision has the most potential to be used in a hybrid sensor system. The survey also found that material handling, assembly, welding and machine tending are applications where hybrid sensors would be most useful.
Why Go Hybrid?
Jeffery Noruk, President of Servo-Robot Corp., Mequon, Wisconsin, gives two reasons that end-users of robotics should consider investing in hybrid sensors. ‘‘Hybrid sensors can do things that traditional two-dimensional cameras cannot. Two-dimensional cameras can take a picture and send the data to the robot,’‘ Noruk says. ‘‘(But,) two-dimensional cameras cannot operate in all conditions, such as when lighting changes or depth information changes.’‘ These scenarios are where a hybrid sensor with laser and two-dimensional capabilities would come in handy, Noruk claims.
The second motive given by Noruk to invest in hybrid sensors is related to the limitations of stand-alone laser vision systems. ‘‘Laser vision has a small depth of field and can only work in a relatively small envelope. In a larger envelope, we use an ultrasonic sensor to access a part with precision.’‘ Noruk adds hybrid sensors are less expensive in the long run because less equipment is used overall to accomplish a range of tasks.
Dr. Wright also gives two big cost benefits for using hybrids. ‘‘Justifications for investing in hybrid sensors are the cost per performance argument, where hybrid sensors might compete with low-end vision systems. Also, a pure performance argument, where the addition of other sensory perception combined with existing vision systems may add value for complex applications.’‘
Hybrid Sensor Apps
Hybrid sensors are useful for a variety of applications. Jeff Noruk says hybrid sensors are used in material handling applications. ‘‘Hybrid sensors used in material handling applications combine a camera and an ultrasonic sensor. The ultrasonic sensor is utilized to get the gripper to a part that is to be picked up at a high speed,’‘ Noruk says. ‘‘At a certain distance from the part, the laser sensor, which has a smaller range or depth of field, takes over and guides the robot to the precise location.’‘
Welding is another application where hybrid sensors could be useful. Again, Jeff Noruk: ‘‘One of our end-users deployed a laser system for weld seam-tracking, where we added a microphone and a web camera. The laser sensor is used in real-time robot guidance and the microphone was added to monitor the process by listening if the process sounds right,’‘ Noruk says. ‘‘While the robot is in teach mode, end-users have seen the advantages of using hybrid sensors for welding applications.’‘
Noruk went on to say that hybrid sensors are beneficial in pick and place, depalletizing, and auto-racking applications when combining ultrasonic sensors with laser vision. ‘‘Hybrid sensors allow manufacturers to perform some material handling applications that were not possible or efficient before.’‘
FANUC Robotics America, Inc., Rochester Hills, Michigan, also believes hybrid sensors have a role in material handling applications. Claude Dinsmoor, General Manager for Controller Product Development at FANUC says, ‘‘Hybrid sensors are used for part location with two-dimensional vision fused with three-dimensional position and orientation information.’‘
Dinsmoor adds that using hybrid sensors for part location also helps in palletizing and machine loading applications. Hybrid sensors are useful in assembly applications, where two-dimensional and three-dimensional vision sensors are utilized with force sensing for complex assembly, says Dinsmoor. ‘‘Hybrid sensors are used for spline assemblies as in clutches or gears, where neither a person nor a vision system alone can see.’‘
In packaging applications, Dinsmoor says, ‘‘Hybrid sensors are also used for mixed unit palletizing, especially for partial layers as well as finding edges of layers.’‘ Furthermore, Dinsmoor says that hybrid sensors are used in sealant and dispensing applications. ‘‘Hybrids are used for finding the location of complex joint and surfaces, such as ditch rail or hem flange areas in automotive manufacturing and for other complex features.’‘
Dinsmoor goes on to say that hybrid sensors work in three dimensions and have six degrees of freedom to find the location of parts. Hybrid sensors add a sense of touch when force sensing is added. ‘‘Just looking for the part is not good enough,’‘ says Dinsmoor.
Dr. Wright says, ‘‘The role of hybrids in material handling, assembly, packaging and dispensing applications may be done today with existing sensors, but it is a cost/performance competition role.’‘ Dr. Wright goes on to add, ‘‘The ease of implementation of hybrid sensors would likely be a secondary benefit for automation professionals who might opt to go with a hybrid.’‘
Getting Hybrids Going
Jeff Noruk of Servo-Robot says the extra sensors are worth the effort of integrating them. ‘‘If end-users have more than one sensor in a work cell, integrators need to know how to program all the sensor technologies involved.’‘ Noruk adds that the same computer and the same software package are used for all sensors in the hybrid system.
Dr. Wright also speaks about implementing hybrid sensors. ‘‘The trick of using hybrid sensors is to allow the user to configure the technology to the desired application. A major hurdle will be the interface - balancing the power of Artificial Intelligence with the development of an easy-to-use, intuitive application software package.’‘
Not all sensors can be combined with each other. ‘‘Compatibility is an issue with some optics, as laser vision cannot have a field of view over 120 cm. Laser sensors have a limited field of view,’‘ Noruk says.
In another example, if a Web camera got too big, it would not be practical to fit it into a work cell. Some compromises have to be made on the depth of field and the field of view as well as resolution.
‘‘Servo-Robot’s hybrid sensor for material handling has a 100 mm field of view. Ultrasonic sensors have a field of view of four meters,’‘ Noruk says. ‘‘If the work cell called for a six meter field of view on an ultrasonic sensor and twice as much on a Web camera, that combination may not work well together.’‘
How many sensors can be combined together? Dr. Wright says, ‘‘There will be a point of no returns where adding more sensors will not yield any more significant gains. The number is only limited by cost and performance.’‘ Jeff Noruk says, ‘‘Servo-Robot has put three sensors in a material handling work cell, ultrasonic, a laser vision and a Web camera viewer. In a welding application, we have combined laser vision with sound and a Web camera.’‘ Noruk asserts that it is feasible to add more than three sensors, but he has not yet come across that as of yet.
FANUC’s Claude Dinsmoor believes that hybrid sensors are often less of a challenge to integrate than a series of external sensors. ‘‘If designed as part of the robot, hybrids can actually be simpler and easier to setup and program than conventional sensors. The additional data makes the application more robust and less sensitive to part or environmental variance,’‘ Dinsmoor says. ‘‘If the sensors are not integrated, they tend to be more complex to design and to apply.’‘
When will hybrid sensors be the norm in robotic-based manufacturing? Dr. Wright believes hybrids will require more investment before they take off in the market. ‘‘Hybrid sensors becoming the norm in robotics would depend on the amount of investment that might occur into this research line. I foresee hybrids becoming more commonplace in robotics-based manufacturing within ten years.’‘
Claude Dinsmoor of FANUC believes that hybrid sensors are coming. ‘‘As hybrid sensors become reliable, more applications will use them to add value in robotic-based manufacturing. I think it will be three to five years before hybrid sensors cut across most robotic applications,’‘ Dinsmoor predicts. ‘‘Hybrid sensors will make the use of robotics even more attractive to manufacturers over hard automation because reliability and flexibility are key.’‘
Servo-Robot’s Jeff Noruk sees steady growth of hybrid sensors. ‘‘Hybrid sensors will creep up in the robotics market rather than explode. In welding applications, hybrids have been offered for the past three years’‘ says Jeff Noruk. ‘‘Hybrid sensors in material handling will grow because some applications cannot be done without them. It took three years to get to five percent of the material handling market,’‘ Noruk says. ‘‘Hybrids will not be half the market in three years, that is for sure. It will take more success stories told about hybrids to get there.’‘
RIA members can download the full Material Handling Hybrid Sensor Market Research Report free from the Members Only section at www.roboticsonline.com. Non-members can purchase at the RIA Online Bookstore at http://www.roboticsonline.com/store/ or by calling 734/994-6088.
For more information, you may contact any of the experts listed in this article.
Claude Dinsmoor, General Manager of Controller Product Development, FANUC Robotics America, Inc., 248-377-7174, Claude.Dinsmoor@fanucrobotics.com
Jeffery Noruk, President, Servo-Robot Corporation, 262-238-4625, firstname.lastname@example.org
Dr. John Wright, Jr., Ph.D., CSIT, Associate Professor of Automation and Electronics Technologies, University of Pennsylvania, Millersville, 717-871-2499, email@example.com