Ultrasonic Inspection with Twin Industrial Robots

University of Strathclyde, Department of Electronic & Electrical Engineering Glasgow, G1 1XW, UK

P. Ian Nicholson, Ian Cooper and Ben Wright
TWI Ltd, Granta Park Great Abington, Cambridge, CB21 6AL, UK

Paper presented at NDT 2016. The 55th annual conference of the British Institute of Non-Destructive Testing. 12-14 Sept. 2015, Nottingham, UK.

Abstract

The uptake of composite materials in the aerospace sector has led to the development of a number of automated inspections systems based on industrial robots, including the IntACom project at TWI Technology Centre Wales. Robotic inspections are designed using Off-Line Programing (OLP) software to describe a path on a Computer-Aided Design (CAD) model of the object to be inspected using the pulse-echo ultrasonic method. By synchronising the movements of two robots, a second robot can be used to follow the path of the first, allowing for ultrasonic through transmission inspections. Investigations carried out at TWI have identified key challenges encountered in alignment and synchronisation when carrying out through transmission inspections of various components. Reasons behind these challenges include inherent latency in the communication between cooperating robots as well as the non-trivial problem of matching positional and orientational velocities.

This paper discusses the above mentioned challenges and presents ongoing work at TWI to tackle these issues. The effects of misalignment on the received ultrasonic signal are discussed and experimentally verified. The used configuration is shown to be tolerant to misalignments of up to 3mm. Different robot synchronisation methods are presented along with their relevance to through transmission inspections. Twin robot pose misalignment during synchronised motion is shown to vary from an average of 0.186mm to 1.108mm depending on the type of synchronisation used. Robot velocity and acceleration effects are also taken into account for geometries with high curvature and their effects on through-transmission inspections are discussed. A constrained version of combined synchronous operation is shown to produce the best through transmission inspection results.

1. Introduction

The increased use of composite materials such as Carbon Fibre Reinforced Plastics (CFRP) in the aerospace industry is well known. This uptake has in turn increased the number of components with complex geometries which are difficult and time consuming to inspect using manual methods, which has caused the Non-Destructive Testing (NDT) process to become an expensive bottleneck in production.  The use of industrial robots to perform inspections has been demonstrated through several projects ^(1)–(4). To achieve a reliable and accurate scan, several factors have been overcome: fast and accurate off-line path planning (OLP), the fusion of ultrasonic and positional data, design and calibration of the NDT delivery payload, human-to-machine interface, data visualisation and robot accuracy issues.

This paper discusses the methods and challenges associated with ultrasonic through transmission inspections using two cooperating industrial robots. The through transmission method is defined to be an inspection wherein one robot carries a transmitting piezoelectric probe (or array thereof) while the cooperating robot carries a similar receiving probe. Also known as a “pitch-catch” inspection, this approach has several benefits over the pulse-echo inspection. The through transmission inspection is used to monitor the attenuation of the ultrasonic signal as it passes through a component, and can be presented as a C-scan when encoded with positional data. This method can detect flaws in composite materials common to the manufacturing stage: cracks (orientation dependent), delaminations, porosity, inclusions and fibre volume fraction changes ⁽⁵⁾. The main advantages of the through transmission method, compared to the pulse echo method, are the full depth coverage due to the lack of a front-wall echo and less attenuation of the ultrasonic signal. On the other hand, the method cannot be used for gauging the depth of a defect and requires good alignment between the transmitting and receiving probes.

Few previous research papers have been published which discuss through transmission inspections with industrial robots. Lu et al. ⁽⁶⁾ highlight the importance of probe alignment during robotic through transmission inspections and  demonstrate a method for obtaining synchronised motion with two industrial robots using a custom built server connection. The authors highlight the importance of maintaining control of the orientation of the receiving probe such that Snell’s law can be used to obtain the optimal signal.

Schwabe et al. ⁽²⁾ discuss the use of water jet nozzles for through transmission inspections using both gantry-type inspection systems and industrial robots. The authors mention that achieving less than 2dB of signal fluctuation is challenging with standard industrial robots. Nieto et al. ⁽⁷⁾ discuss the challenges of designing a water jet nozzle for phased array ultrasonic through transmission inspections. The authors demonstrate a design which enables robotic through transmission inspections with a 20mm diameter water jet and scan path speed at 750mm/s.  

1.2 Robotic Inspection Systems

The use of robotics has greatly increased in recent years ⁽⁸⁾ and political roadmaps for the future of manufacturing in high-wage economies identify the use of robots as a key driver for growth [9]-[10]. Traditionally industrial robots have been used in the automotive industry where their high repeatability has meant that standardised tasks such as welding, pick-and-place and surface spraying could be automated easily. For these conveyor-type operations, the synchronisation between cooperating robots has been a main concern to increase throughput speeds and reduce the risk of collisions. In recent years automated inspection systems have begun to use robots due to their low cost, high flexibility and ease of programming.

A robot can consist of a number of different joints, each of which usually adds a degree of freedom to manipulate the payload, commonly referred to as the Tool Centre Point (TCP). The location of the TCP is calculated through forward kinematics; a set of equations where an encoder in each joint coupled with a mathematical model of the robot is used to find the 3D location and orientation of the TCP. A typical industrial robot has 6 joints providing six degrees of freedom.  Further degrees of freedom can be obtained using external axes such as linear tracks and turntables.

1.3 The IntACom project

To meet the demand for a faster inspection method of complex geometry composite components, the IntACom project was developed by TWI Technology Centre Wales in a joint partnership with a number of aerospace manufacturers and the University of Strathclyde ⁽¹⁾. The system consists of two KUKA KR16 L6-2 six axis industrial robots which can inspect throughout a 3m x 1m x 1m volume using either phased array or single element ultrasonic probes. The TWI developed software encodes the ultrasonic data with the positional feedback from the robots to provide a seamless user experience for the NDT inspector. The ultrasonic signal is coupled to the components via water jet nozzles which are designed and prototyped using additive manufacturing techniques at TWI’s facility. Details of a phased array water jet probe and an image of the probe in action is shown in Figure 1.

Phased Array Water Jet Probe
Elements	64
Pitch	0.6 mm
Central Frequency	5 MHz
Water Column Width	5 mm
Water Column Length	50 mm
Pump Capacity	3600 L/hr

2. Twin Robot Coordinated Motion

For ultrasonic inspections using a twin robot solution, robot cooperation can be utilised for through transmission inspections where one robot carries the transmitting ultrasonic probe and the other robot carries the receiving probe.

Each robot contains its own internal reference system which usually has the origin located inside the base of the robot. The physical location of this origin is not accessible from the outside but is calculated through a calibration procedure using the encoder values and the kinematic model of the robot. An illustration of the different reference frames which are relevant to synchronised motion is shown in Figure 2. The transformation between the internal reference frame of the robot and the last joint (“wrist”) of the robot is provided by the kinematic model of the robot, usually supplied by the robot manufacturer. A calibration procedure is then needed to establish the relationship between the TCP and the wrist which is discussed in more detail in Section 3.

To perform coordinated motions, the two robots will need to establish a common reference frame. This is necessary to be able to calculate the position of each end effector with respect to the other. A single base frame is typically defined on the part to be inspected or the base reference frame of one robot is shared by both. To share a single robotic base coordinate system, the transformation between robot bases must be found.

2.1 Types of Cooperation Motion

Following the work of ⁽¹¹⁾, three different types of cooperative robot motion can be identified. These are concurrent cooperation, coupled synchronous cooperation and combined synchronous operation.

2.1.1 Concurrent Cooperation

Concurrent cooperation is achieved when two robots are triggered to start their paths at the same time, T₀ . This is the simplest type of synchronisation behaviour whereby a simultaneous motion start time is enforced for both robots. This is often used in conveyor type operations where one robot will wait for another to finish parts of a task, (such as spot welding), before both robots then proceed. No orientation, position or velocity constraints are applied. From a through-transmission inspection perspective, this method is only useful if the component is flat such that two robots can cover the same distance in the same period of time.

2.1.2 Coupled Synchronous Cooperation

Coupled synchronous cooperation occurs when the relative pose between the end effectors (transformation H1 in Figure 2) of each robot remains fixed in a master-slave relationship. This motion is begun at a synchronisation time, T₀ .  To achieve the slave motion, the relative transform between each base reference needs to be known as well as the pose of the master as described by Equation 1.  Equation 1 describes the motion of TCP2 at time t as a function of the relative transform between the two robots, the position of TCP1 and the initial pose each was in at T₀ . This motion effectively renders the path planning of the slave robot irrelevant as the only path available to both robots is the one planned for the master robot. Communication latency between the two robots can lead to a situation where the velocities are not matched due to the time it takes for the slave to update its position which has to be taken into account during path planning.

2.1.3 Combined Synchronous Operation

A second type of synchronised motion allows both robots to move on independent paths but synchronises the speed between the two end effectors. This type of motion is started at T₀ after which the two robots will constrain their motions to complete a number of commands in a specified interval, ∆T. This method requires that each robot can follow a planned path along with a method for allowing each robot to communicate its speed and position to the other. The path behaviour is described by Equation 41 in ⁽¹¹⁾ where a complete derivation of the motion constraints can also be found.

One of the main difficulties of using this approach is efficiently and accurately calculating the constraint relationship between the two robot end effector velocities, each of which is given by Equation 2, wherein v is the positional velocity and ω is the orientational velocity. q = (q₁, q₂, q₃ ...q_n) is the joint displacement vector for a n-degrees-of-freedom robot. J_ρ and J_α are the positional and orientational parts of the Jacobian matrix, respectively.

2.2 Synchronised Motion in Through Transmission Inspections

Each of the above mentioned cooperative motion methods can be applied to ultrasonic inspections. The simplest method is achieved when the relative pose between robots is constant (coupled synchronous cooperation). This method is however only useful for components with a uniform or slightly varying thickness. As the ultrasound is transmitted through a water path, some variation in thickness can be tolerated as long as the water path is long enough to avoid collision between the part and the end-effector. A longer water path often results in turbulent water flow when using water jet nozzles which must also be taken into account.

Combined synchronous operation allows for the inspection of components with large variations in thickness, but is more difficult to program as two paths have to be designed, which synchronise the velocities of the robots. Discussed in section 2.1.3, velocity synchronisations of the end effectors include both velocities along the path but also the velocity with which the robot changes orientation, resulting in a non-trivial constrained motion calculation.

3. Calibration

A calibration must be undertaken to relate the kinematic model of a robot to its physical counterpart. Calibration typically involves driving the TCP of the end-effector to the same position in a number of different orientations. By solving the system of equations in the form AX = XB (where X is the unknown wrist-to-TCP transform) the missing transformation can be found. A detailed description of calibration procedures can be found in ⁽¹²⁾. Each of the above mentioned cooperation methods requires TCP and base calibration to be done previously to executing any robotic motion, ensuring that the intended off-line path planning motions are performed and minimising the risk of collisions.

One of the main challenges encountered when calibrating ultrasonic probes is the fact that the surface of the probes is typically not accessible as it sits within the water jet nozzle. An intermediate calibration procedure can be performed using a point on the casing of the water jet nozzle. The ultrasonic signal from an array probe can then be used to optimise this calibration by monitoring the real-time B-scan, a feature available in the IntACom software package. The optimal calibration can be found through monitoring the strength of the ultrasonic signal as the robot rotates the probe around each axis of the TCP reference frame. The stand-off distance (the water path between the component and the end of the nozzle) can then be adjusted mathematically by projecting a point along the forward pointing vector of the TCP reference system.

4. Validation Experiments

A series of validation experiments were carried out which measured the distance between the robots using a laser distance meter (Acuity AR200-100 ⁽¹³⁾). This was implemented by mounting the laser distance meter as the end-effect or on one robot and measuring the distance to the second robot’s end-effect, as shown in Figure 3. A path was designed which would encompass the typical working envelope of the two robots on a non-linear path to measure any difference in the relative distance between the robots when inspecting curved surfaces. The laser distance meter measures distance with a resolution of 30.5µm using a 650nm wavelength laser, at a measurement frequency of up to 1250Hz.

Two different methods were utilised to compare any variation in the separation of the two robots. For the first experiment, the second robot was instructed to follow the first in the coupled synchronous cooperation mode. As described by Equation 1, this mode depends on an accurately known transformation between the two robot base coordinate systems (transformation H3H2 in Figure 2). The second experiment used the combined synchronous operation motion method wherein both robots follow the cylindrical path used previously, but with synchronised starting points and different velocities depending on the distance each will need to travel. Hence the second robot travels faster than the first to maintain alignment.

4.1 Robot Separation

The results from a semi-cylindrical path are shown in Figure 4 and 5. Each figure shows the variation in the distance from the initial separation which was set to approximately 70mm. Figure 4 shows the results from using coupled synchronous cooperation. Initial investigations showed an average variation in separation of 0.531 ±0.281mm which utilised a previous calibration. After recalibrating the robot-to-robot transform the average variation was 0.186 ± 0.138mm. This illustrates the importance of an accurate calibration prior to conducting ultrasonic scans as the stand-off distance affects the results as shown in Section 4.2. Further improvements could be gained with the use of an external metrology system to perform the robot-to-robot calibration as well as giving the possibility of monitoring the position of the two robots during the scan.

Figure 4 exhibits a periodic change of separation as the robots move right to left and vice-versa. This feature is believed to be due to communication lag between each robot which causes the second robot to trail behind the other. Combined with a slight angle in the reflective surface this could cause an apparent change in separation distance.

Figure 5 shows the results of using the combined synchronous operation method. The average variation in separation during this motion was 1.108 ±0.740mm with a maximum separation of 2.770mm. For this type of motion the same base reference system has to be defined for both robots. This is typically done by physically moving each robot to a number of fixed points in space to define a coordinate reference system, which is a potential source of error. The main reason for the larger deviation is believed to be due to differences in the speed with which the orientation of each robot changes.

The two experiments illustrate the non-trivial velocity matching which is needed for synchronous behaviour between two industrial robots. Although it is possible to control the orientation velocity and acceleration of each robot, the calculation of different velocities for arbitrarily shaped surfaces is not supported in most OLP software packages. There is however potential for incorporating such features in custom-made robot control toolboxes such as RoboNDT ⁽¹⁴⁾. To get a better understanding of the misalignment between cooperating robots, another type of sensor is needed that can monitor the position of both end-effectors with six degrees of freedom simultaneously. Although the current experiment is limited to investigate one dimensional variations, the data still illustrates some of the challenges which are encountered during robotic through-transmission inspections.      

4.2 Beam Refraction and Misalignment

To obtain the best possible ultrasonic signal in through transmission inspections, several factors have to be taken into account. Refraction occurs at interfaces when sound travels through different media. The sound beam will also disperse in a medium depending on the distance travelled and be attenuated by the material it travels through. Finally, for the purposes of water jet inspections, the effects of turbulence have to be considered. Figure 6 illustrates a general situation wherein Snell’s law can be used to calculate the angle of the outgoing sound, O. The incoming sound, I, is refracted at both interfaces such that α1 ≠ α2. To avoid the effects of refraction, robotic paths are designed to be normal to the incident surface such that refraction only occurs at the rear surface for components with non-parallel surfaces. Due to the large impedance mismatch between water and air, the water jet also acts as an ultrasonic waveguide, making it easier to receive the sound even with some refraction.

An experiment was carried out to investigate which effects the observed differences in robot separation have on ultrasonic through transmission inspections. This experiment consisted in making one robot raster a receiving probe along the back surface of an acrylic component while a second robot placed the transmitting probe on the opposite side of the acrylic at various distances from the surface. Both surfaces of the acrylic were slightly curved but exhibited the same curvature at the point of inspection. This particular sample was used to illustrate the change in intensity as a function of misalignment between the transmitting and receiving probe. 

4.2.1 Raster Data

Ultrasonic data was gathered using two water jet nozzles, shown in Figure 7, each containing a 64 element phased array probe with a central frequency of 5MHz. The pitch of each array probe was 0.6mm and the central 15 elements were used such that the overall active area was an approximately 9mm x 9mm square, the idea being to produce a nearly symmetrical Gaussian beam profile. The raster step was 1.2mm and data was recorded at every 0.6mm in the scanning direction.

The receiving nozzle was kept at 5 ±0.6mm offset from the surface of the component which was verified by monitoring the front wall signal in a separate pulse-echo scan. The second robot was moved backwards in steps of 5mm while staying perpendicular to the surface of the component. A KUKA Robot Language (KRL) file was written to produce this path behaviour while recording data through the IntACom software. The results of the experiment are show in Figure 8 which illustrates the loss of signal as the transmitting probe is moved farther from the surface. The effects of turbulence as the water path increases are clearly visible from offsets larger than 25mm.

4.2.2 Beam Profile

To assess the effect of different standoff distances on the ultrasonic signal, the ultrasonic intensity was plotted across each column and row in the C-scan images shown in Figure 8. The intensity across the C-scans in the horizontal direction is shown in Figure 9. A similar profile was seen across the C-scans in the vertical direction confirming the Gaussian beam profile. The beam profiles for different offsets shown in Figure 9 clearly show the drop in intensity as the transmitter moves further away but also illustrate how the energy is spread out across a larger area. To quantify the expected loss of signal as a function of misalignment, a Gaussian model was fitted to each data series in MATLAB. The resulting characteristics are shown in Table 1.

The data in Table 1 clearly shows that the peak position only shifts very slightly with a varying standoff distance, indicating that the wave guide formed by the column of water counteracts any major refractive effects. The Full Width at Max Height (FWMH) range indicates where a 6dB drop in amplitude occurs which is a useful measure of the spread of sound intensity. To get a better understanding of the degree to which misalignment will show pronounceable drops in intensity a contour map can be created as shown in Figure 10A.

Table 1.  Results from Gaussian fitting for horizontal profiles

Standoff (mm)	Peak Position (mm)	FWMH (mm)	Amplitude (arb units)
5	35.86	6.94	106.03
10	36.37	7.53	98.38
15	36.37	8.30	92.50
20	36.51	8.72	85.42
25	36.70	9.14	80.37
30	36.41	9.32	75.72
35	36.21	9.76	65.93
40	N/A	N/A	0

Generating a contour plot for each stand-off position allows for the analysis of a region of less than 2dB of intensity loss as a function of misalignment. The raster experiment was replicated but with a 2mm stand-off increment instead of 5mm. Figure 10B shows how this area drops as the transmitting probe is moved away from the surface. The peak at close to 7mm is caused by the natural focal point of the beam hitting the surface which was confirmed through CIVA simulations.

The resulting graph shows that an area of nearly 30mm² of good signal intensity is received through the acrylic component for stand-off distances between 5mm and 10mm. Assuming a circular focal spot, this area provides roughly 3mm of misalignment tolerances in all directions wherein the signal will fluctuate less than 2dB. Such tolerances allow for small misalignments arising from robot positional inaccuracies as well as the separation distance found during synchronised movement, discussed in Section 4.1.

5. Through Transmission Experiments

As it has been demonstrated that the water jet coupled approach is tolerable to minor misalignments, two through transmission inspections of a component were carried out. The component used is shown in Figure 11 and has been designed to mimic shapes found in industry. The challenging geometry offers very few parallel surfaces and exhibits curvatures in all three dimensions. A series of flat-bottomed holes parallel to the front and back surfaces were placed throughout the part to represent defects.

The component is made from an acrylic material and was scanned in through transmission mode using two 5MHz, 64 element phased array probes with a 0.6mm pitch. The full array was used with a sub-aperture of 15 elements which was electronically stepped across the array at 2 element increments to speed up the inspection. The raster step was 29.3mm and the component was scanned at 400mm/s. Ultrasonic data was gathered using the IntACom software and a Micropulse 5PA from Peak NDT⁽¹⁵⁾ using 8-bit dynamic range.

5.1 Results

To avoid the large variations in robot separation, shown in Figure 5, while maintain the ability to inspect varying thicknesses, a combination of synchronisation methods for through transmission inspections was used. Due to the large variations in component thickness the concurrent cooperation and coupled synchronous cooperation motions are unfeasible. Combined synchronous operation remained the only option. To maintain alignment, a variation in which the angular velocities remained fixed was programmed as well as the more unconstrained approach wherein the angular velocities are calculated as a function of geometry. The resulting ultrasonic C-scans are presented in Figures 12 and 13.

The differences in signal intensity between Figure 12 and 13 show the clear benefit of constraining the angular velocity during synchronised motion for components with large variations in geometry. The lower intensity seen in Figure 13 is believed to be a result of a misalignment between the two probes as one robot needs to vary the orientation of its end-effector significantly more than the first robot. Further work is still needed to assess how this problem can overcome using external control of the robot through different path planning approaches.

6. Conclusion

Industry has shown a clear trend in moving towards automated inspection systems using industrial robots. This paper has shown different ways in which two cooperating robots can be used for ultrasonic through transmission inspections. The effects of robot-to-robot pose misalignment on the ultrasonic signal are discussed and experimentally verified. Variation in robot separation during synchronised motion has been found to vary from an average of 0.186mm to 1.108mm, depending on the synchronisation method used. Positional tolerances when using water jet nozzles are discussed and it was found that an area of nearly 30mm2 with high signal intensity is achievable with a 5-10mm stand-off distance. 

7. Acknowledgements

The authors wish to thank Carmelo Mineo, Benjamin Knight-Gregson and Daniel Aylett for technical support and helpful discussions. The authors would also like to thank the UK Research Centre for Non-Destructive Evaluation (RCNDE) for continued support.

8. Bibliography

 

1. I. Cooper, I. Nicholson, D. Yan, B. Wright, D. Liaptsis, and C. Mineo, “DEVELOPMENT OF A FAST INSPECTION SYSTEM FOR COMPLEX COMPOSITE STRUCTURE - THE INTACOM PROJECT,” in 5th International Symposium on NDT in Aerospace, 2013, no. November, pp. 13–15.
2. D. M. Schwabe, A. Maurer, and R. Koch, “Ultrasonic Testing Machines with Robot Mechanics – A New Approach to CFRP Component Testing,” in 2nd International Symposium on NDT in Aerospace, 2010, pp. 1–5.
3. S. A. Tecnatom, “Ultrasonic Techniques and Industrial Robots : Natural Evolution of Inspection Systems,” pp. 1–12, 2012.
4. The Manufacturer, “USL partner up for new automated ultrasonic testing,” The Manufacturer (Online) - Hennik Research, Oct-2014.
5. J. Summerscales, Non-destructive testing of fibre-reinforced plastics composites. Springer Netherlands, 1990.
6. Z. Lu, C. Xu, X. Zhao, L. Zhang, H. Wang, and Q. Pan, “Ultrasonic transmission testing of twin-robots coordinated control,” 2013 IEEE Int. Conf. Mechatronics Autom. IEEE ICMA 2013, pp. 1256–1260, 2013.
7. M. Á. Nieto, Z. Re, and E. Cuevas, “TTU Phased Array : Quality and Productivity,” in 6th International Symposium on NDT in Aerospace, 2014, no. November, pp. 12–14.
8. International Federation of Robotics, “World Robotic Report - Executive Summary 2013,” 2013.
9. D. Lane and R. Buckingham, “RAS 2020 - Robotics and Autonomous Systems,” no. July, 2014.
10. European Union, “Factory in a Day,” 2013. [Online]. Available: http://www.factory-in-a-day.eu/.[Accessed: 07-May-2016].
11. Y. Gan, X. Dai, and J. Li, “Cooperative path planning and constraints analysis for master-slave industrial robots,” Int. J. Adv. Robot. Syst., vol. 9, 2012.
12. Y. C. Shiu and S. Ahmad, “Calibration of Wrist-Mounted Robotic Sensors by Solving Homogeneous Transform Equations the Form A X = X B,” IEEE Trans. Autom., vol. 5, no. 1, pp. 16–29, 1989.
13. Acuity, “AR200 LASER MEASUREMENT SENSOR.” [Online]. Available: http://www.acuitylaser.com/products/item/ar200-laser-measurement-sensor. [Accessed: 16-Apr-2016].
14. C. Mineo, S. G. Pierce, P. I. Nicholson, and I. Cooper, “Robotic path planning for non-destructive testing – A custom MATLAB toolbox approach,” Robot. Comput. Integr. Manuf., vol. 37, pp. 1–12, 2016.
15. Peak NDT, “Micropulse 5PA T/R.” [Online]. Available: http://peakndt.com/products/micropulse-5pa-tr/. [Accessed: 26-Apr-2016].