A clinical experience with a hierarchically controlled myoelectric hand prosthesis with vibro-tactile feedback
P. J. Kyberd *
N. Mustapha *
F. Carnegie *
P. H. Chappell *
Abstract
Improved performance of externally powered myoelectric hands is possible when the direct control of the digit flexion and grip force are given over to an electronic controller which frees the operator to concentrate on other demands.
Design: A commercial myoelectric hand was modified to take the new touch and slip sensors and novel control method.
Subject: An adult male with a traumatic mid-forearm amputation.
Outcome measure: The range and ease of use of the prosthetics system.
Result: The hand was easily and usefully operated in the home and work environment.
Conclusion: Hierarchical control of a hand is possible using sensory feedback to a sophisticated electronic controller. Such a control method reduces the demands on the user's concentration and enhances the hand's range.
Introduction
The survival rate of an individual following amputation, prior to the development of successful anaesthesia, was poor. History records a few hardy individuals who survived (Pliny; Herodotus; Childress, 1985). The replacement limbs were often simple. The more sophisticated hands were often based on the techniques developed by armourers in building articulated gloves. Once the survival rate improved, the opportunities for commerical exploitation also grew and companies formed, the oldest in the UK being over a hundred years old.
All practical prostheses were body powered and this continued to be the major form of actuation until the Seventies when electric sources became practical: such devices are fitted to a small proportion of the population.
In the research arena, other forms of power sources have been used as far back as 1916 (Childress, 1985), for example carbon dioxide gas under pressure. However, none of these have achieved clinical significance, though small numbers of people continue to use gas powered arms. This is due to a variety of reasons, from the limited capabilities of the power supply, to the availability of power sources (Millstein et al., 1986; Simpson, 1972).
The increase in the levels of complexity and the integration of electronic circuits and some improvements in the technology of electrical storage, have encouraged experimental designs of hand and controller that provide better performances or longer periods between recharging than current designs (Gow and Douglas, 1990; Chappell and Kyberd, 1991),
Control of prosthetic hands
In the clinical setting there are still only two widely used means of control of prostheses. The first is in body powered terminal devices which usually are in the form of a split hook where control is by body movement. The second form is the electrically driven cosmetic hand controlled by myoelectric signals derived from muscles (usually antagonistic pairs) on the wearer's stump. Both types of prostheses have a single driven action and a passive wrist.
Other hand geometries and prosthesis forms have been suggested (LeBlanc et al., 1987; Childress, 1972). Few examples have achieved commercial exploitation, the exceptions including the NU-VA Synergetic prehensor, the Steeper powered hook and the CAPP-II-TD (Patton, 1988; Bennett Wilson, 1989).
The level of acceptance and use of a particular hand depends on a large number of factors. However ease of use is an important aspect. The body powered hook is successful in part due to its geometry, allowing a wide range of objects to be held. In addition its control is based on simple body movements. This allows a high degree of control and so a high level of confidence to be developed by the operator. The system has a number of drawbacks including poor cosmesis. In cases where the stump is poorly padded, following traumatic amputation, the force required to open the hook must be transferred through the stump end causing pain.
The action of an electrically operated hand is commonly voluntary opening and voluntary closing. The commands are obtained from the electrical signals generated when the muscles contract (known as myoelectric signals or EMG signals). The strength of the signal is dependent on the muscle tension. Extensor tension which exceeds a set threshold opens the hand, while sufficient flexor tension closes it. Relaxation disables the motor (Fig. 1a). This representation in Fig. 1 of muscle activity is used throughout (Scott, 1988). The signals due to the flexor and extensor muscles are plotted on the horizontal axis. Extensor tension increases to the right, flexor tension to the left. For antagonistic pairs of muscles there is a single range from maximum flexion to maximum extension, through a central region where both muscles are relaxed. The drive mechanism leaves the hand locked open at that point. If the hand is left fully extended the result looks unnatural, so during training users are taught to leave the hand closed if it is to remain idle. Alternatively, a single muscle can be used to command opening and closing of the hand (Fig. 1b) but the operator must pass through one direction command to reach the other.
To hold an object the hand is opened wide enough to admit the object and is then closed around it. The wearer either judges when to stop closing around the object by eye, or, when the sight-line is obscured, allows the hand to stop when the controlling circuitry stops the hand. This latter option is easier on mental effort but provides very coarse control of the grip force; a delicate grip is difficult to achieve using this method. To enhance their control the users may utilise other information that is available from the prosthesis but this accidental path is not designed into the mechanism.
The geometry of the hand limits its functional range and the coarse grip force control ensures some operations cannot be performed successfully by such a hand. The hands are thus most often used in cosmetic and support roles.
An additional limitation of the hand's use is that myoelectric channels are inherently noisy. In setting up a working prosthesis the users must set the threshold of action for the EMG pickups so the hand is useful to them. A large amplification sets the threshold low so the hand is easily operated. This may lead to inadvertent opening of the hand when an object is being gripped. To ensure a more secure grip flexor tension must be periodically reapplied thus losing any finesse in the management of the grip tension. Alternatively the actuating threshold can be set higher making the grip more secure but the hand more tiring to use.
Another form of arm control has been proposed (Gow et al., 1983; Simpson and Kenworthy, 1973). This is known as Extended Physiological Proprioception (EPP), developed by D. C. Simpson at the Bioengineering Unit of Princess Margaret Rose Hospital in Edinburgh, Scotland, for use on the Simpson arm. The key to this form of control is the appropriate nature of the input and return signals. A force command is returned to the input lever as a change in position in proportion to the position of the end of a multi-degree of freedom arm. This mimics the body's own system where the effect of a muscular force is detected as a positional change in the arm. Thus the EPP driven arm extends the users awareness along the length of the arm. The result is accurate and easily controlled (Gow et al., 1983). It is essentially a method of accurately and easily positioning a terminal device in space. However, recent studies suggest that it can be modified to transfer touch information, (Meek et al., 1989). This suggestion ignores the central aspect of EPP, the appropriate nature of the feedback, It is the position of the lever that reflects the position of the arm.
A widely functional hand requires a greater number of degrees of freedom than the conventional number available. When an arm prosthesis has multiple degrees of freedom it requires separate myoelectric channels for the control of each independent axis of motion. This technique is not easy to learn or control for hand or arms.
A practical multi-degree of freedom hand is potentially capable of a wide range of actions providing its control is of a sufficient standard yet it must also be easy to use. Thus it is necessary to devolve the detailed control of the hand to a computer and allow the supervisory actions to be taken by the operator, much as the human Central Nervous System (CNS) separates the positional reflexes from the gross intentions of the person. The Southampton Adaptive Manipulation Scheme (SAMS) performs this task.
SAMS
SAMS was initially conceived to be used in a multi-degree of freedom hand (Chappell and Kyberd, 1991; Baits et al., 1969; Kyberd, 1990; Senski, 1980) but the system's virtues can be applied to simpler hands as well. The hand described here is of that form.
Although there are forms of proportional voluntary control available, these are crude and require constant attention by the user. Good control is achievable by exceptional operators. SAMS achieves enhanced performance by feeding information about the grip force and any movement of the held object back to the controller using a vibro-tactile sensor (Chappell et al., 1987). Using this method the mode of operation becomes simpler.
The control input is via two single channel EMG amplifiers. Fig. 2 shows the two signals from the flexor and extensor muscles plotted on the horizontal axis. The central region is where both muscles are relaxed. Extensor tension increases to the right, flexor tension to the left. The vertical axis shows the control states, (OPEN, HOLD, POSITION, SQUEEZE and RELEASE) the boxes show the degree of hysteresis available to each command. Fig. 3 shows the regions of proportional response. The scale represents the degree of flexion of the finger, which is in direct proportion to the extensor tension. Grip force in response to the SQUEEZE demand is in proportion to the flexor tension, and is shown as increasing in the negative direction.
Extensor tension opens the hand by an amount in proportion to the tension. Relaxation of the muscle allows the hand to close once more. This is voluntary opening, involuntary closing, in a similar form to the body powered split hook. Extensor tension below a set threshold is taken by the controller to be a relaxed state. If the sensor makes contact with an object while the hand is closing, the movement is stopped so only the lightest touch is applied. When the touch sensor is activated sufficient flexor tension enables the HOLD mode. Once in HOLD the automatic force control is activated. Input from the vibro-tactile sensor detects a slipping object and the controller increases the tension accordingly. Muscle tension less than the SQUEEZE and RELEASE thresholds does not affect the controller.
Whilst is the HOLD mode, should the wearer wish to increase tension and override the slip control, a SQUEEZE mode is available. The motor force increases in proportion to flexor tension.
To prevent inadvertent squeezing when the HOLD mode is initially invoked, the EMG must return to within the relax range before an excursion beyond the SQUEEZE threshold is recognised as a squeeze demand. At any time when in HOLD or SQUEEZE extensor tension beyond the release threshold will cause the hand to release the object by opening the hand slowly to the position of the first contact, or (if the hand still touches the object) beyond to the point where contact is broken. Relaxing the user demand to the RELAX range returns the control to the position mode.
It can be seen that the result of such action is to free the operator from the detailed control tasks. The hand only applies the minimum force to maintain a stable grip. In addition the hand automatically closes when empty and not in use and the knowledge of whether the hand is empty or holding an object means the different EMG thresholds can be set widely differently making prehension automatic and object release very deliberate.
Although a bipolar command channel was employed, the controlling programme can simply be modified to allow a single site channel operation.
Similarly, the EMG input could easily be changed to a number of different forms. By converting the input to an electrical signal, the contoller can then be reprogrammed to use the data accordingly. Thus, for example, a purely mechanical input via a pull cord or acoustic myography (AMG) could be used, (Barry et al., 1986).
It is the autonomous nature of the control of the hand which implies that the use of the system would be broadly subconscious. Thus the only effective test of the system is its long term use in the field. To assess the potential of the concept a portable controller was devised and fitted to an individual to allow a limited field test to be carried out (Chappell et al., 1987; Kyberd, 1990).
Materials and methods
A modified commercial single degree of freedom myoelectric hand was adapted to carry sensors controlled by an electronic controller. The controller was built upon a single printed circuit board within a case (220mm x 150mm x 25mm). The EMG amplifier was mounted in a separate metal enclosure to shield it from external interference.
The hand was an MM3 produced by Viennatone, driven by two conventional 12V batteries, the electronics by a further two PP6 batteries. The hand was mounted on a standard self-suspending socket with a passive wrist. The wiring led to a pouch worn over the shoulder of the subject. The basic elements of the system are shown in Fig. 4.
Although this arrangement is far from optimal the aim of the experiment was to assess the potential of the system prior to a more extended trial. The electronics and software had undergone extensive testing within the laboratory. It was therefore a test of the control philosophy and not the hardware. Similarly, as the design was made independently of the prosthesis it could be applied to any electric hand. The choice of the Viennatone device was made on the basis of availability. No assessment was made on the comparative speed or lightness of devices as the factors are dependent on the vehicle rather than the control strategy.
A display of lights was used for training purposes but was removed when the hand was used continuously. The individual lights showed the status of the controller in HOLD or POSITION states.
In the initial experiments the EMG amplifier/ processor was mounted near to the electrodes, but it was necessary to move it to a more distant site for the field trial, with a subsequent decrease in its reliability.
Patient trials and results
Initial tests were carried out by one of the authors (PJK) using a right hand prosthesis mounted upon a splint. The splint was made of the thermoplastic Plastazote. The splint extended from the elbow to the knuckles, which immobilised the natural hand, and so encouraged the use of the prosthesis. It also allowed the muscles to contract principally in an isometric fashion similar to the manner of control by an amputee.
The hand was mounted with its centre line 10cm from the centre of the wearer's arm, and was positioned parallel to the natural hand. Though this made the hand off-centre to the arm, which changed the natural hand/eye relationship, most problems were overcome with practice. The one exception was that heavy objects caused the splint to rotate on the arm and disturb the EMG pickup.
The tasks performed ranged from manipulation of abstract objects, to the performance of everyday actions.
During the tests the natural right hand was not used and attempts were made to use the prosthesis in the dominant role. Certain tasks, such as rewiring a plug required the prosthesis to be used as a vice with the more dexterous tasks being performed by the left hand. The tests were also performed in parallel with the experiment in the field, providing insights into the systems's capabilities.
The system was then used in the field by a single adult male who had lost his left hand at the wrist in an industrial accident. He normally used a Steeper myoelectric hand for daily activities. His myoelectric control was good, thus he would derive less advantage with the new controller compared with that enjoyed by a less adept user. It can therefore be argued that any advantage experienced by the subject represents a significant improvement in control.
The hand was fitted at the limb fitting centre at Queen Mary's Hospital, Roehampton, London. The left hand was fitted to a standard arm socket copied from his normal prosthesis. The outer moulding was made to accommodate the wires to the controller and the EMG amplifier could be held in a pouch slung over the shoulder (Fig. 5). Five two-hour sessions were conducted with the subject accustoming himself to the different manner of control of the device. In the interval between the later sessions he took the system home and used it in the domestic and work environments. A video recording was made of the test.
The training consisted of performing the standard manipulative tasks conducted at the Arm Training Unit.
The tests included picking up a range of standard objects from a board and placing them in a box. The shapes were cones, cylinders and blocks of varying sizes as well as a cup and ball. The time taken for the board to be cleared was recorded. The varying sizes and shapes of the test objects ensured the need for the subject to reorient the hand and to adjust the grip and approach for each object. Less abstract tasks were also used, such as rewiring a plug, cutting and manipulating paper.
Results
The subject rapidly became adept at using the system.
The initial control was inaccurate as he attempted to use it as if it were his normal prosthesis (voluntary closure as well as opening). This tendency was reduced as his confidence grew and the advantages of automatic grip tension became appreciated. Pick and place tasks were performed with skill and precision.
The speeds of operation of his conventional hand and the SAMS hand were very similar, so that the time taken for the pick and place operation was broadly the same. However the measure of the utility of the hands was the level of concentration required for the tasks.
System operation
Problems were encountered in four areas:
Socket: The subject suffered discomfort when wearing the socket as it was too tight and this limited the time the hand could be worn over one session. Each time the arm was put on it required talcum powder or a water based cream to ease the insertion. These problems were unrelated to the electronic system itself.
Wiring: The principle causes of failure of the system were due to the wiring to the EMG amplifier. This alone was responsible for 80% of all failures while on the subject. The signal from the muscles is small and presents a high impedance to the wires conveying the signal. Pre-amplification close to the site would alleviate the problem. Very few failures were encountered when the splint was used. For this the EMG amplifier and signal processor were mounted close to the muscles and wires were short. This arrangement would have been inconvenient with the field tests. The construction of a new compact amplifier/ processor is possible, but a more logical path would be to use a commercial analogue amplifier and perform the additional processing in software (Kyberd, 1985).
Electronics: The electronic controller itself suffered no failures. The sensor ultimately fatigued. The sensor had been used over a period of 18 months of testing. Once the worn component was replaced the sensor operated satisfactorily.
Batteries: Two rechargeable 12V batteries were used. A single unit normally powers the unmodified hand, thus they were adopted as a matter of course. The batteries themselves had poor reliability and charge retention qualities. Three cells burst during operation or recharging. The batteries used on the other commercial system the subject normally wears, were much more reliable but were only 6V. The 12V cells supplied power for 1 hour's continuous use but the modifications of the system to 6V from 12V was not feasible.
Discussion
Minor alterations were made to the system as a result of these tests. One such change was suggested specifically by the subject. It was an alteration to the invocation of the HOLD command. The user must pause in the relaxed state before proceeding to HOLD to ensure that an accidental HOLD command is avoided. Also HOLD is only invoked by flexor tension once an object is in contact with the hand. The point of contact occurs when the hand is partly extended so the user may still have some extensor tension, thus there is a delay in waiting for the HOLD state to be asserted after contact and the EMG must be relaxed and then applied. This was felt to be inconvenient.
The provision of a pre-HOLD state, when the hand is touching the object prevents the hand from gripping tighter inadvertently despite object movement. This was provided to allow objects to be manoeuvred before being held. This especially caters for bilateral users who do not have fine control with either hand and so must manoeuvre the object while in the hand. In the present subject's case, he felt the if he wished to change the grasp of the object or its attitude in the hand, he would relax and re-grasp it using his natural hand to assist the task. This option is not present for the bilateral user. For the subject this feature was removed. In any future trials such customising could be made available depending on user taste and requirements. This is the advantage of a computer controlled hand, as the expensive structural modifications to the system can be kept to a minimum. In addition the adaptable nature of the control philosophy means it can be applied to any form of externally powered prosthesis, mechanical or anthropomorphic. It is important that such choices are available to the user population at large.
One aspect of the user control of standard prostheses not widely reported concerns the use of motor vibration feedback to the wearer. Discussions with the subject showed he utilised this a great deal. Since he possessed a long stump, his arm was very close to the motor and so could easily feel it vibrating. Anecdotal evidence suggests this is a commonly used feedback path with many patients, whether the loss is congenital or traumatic. This includes those with much shorter stumps. This use of other signals depends on the current limit circuitry on the motor drive. Once the motor stalls the drive power is cut and the user feels the change in vibrations. A second application of closure demand indicates resilience of the object and the stability of grip, based on the length of time the motor runs before it stops once more. Greater compliance or a slipping grip will allow the motor to turn for longer before it cuts out. Thus the user can 'feel' this without recourse to visual cues. This unexpected use of feed-in demonstrates the adaptability of the human subject. However it is a less sensitive form of grip tension control than that realised using SAMS control.
A single touch/force sensor was used and was mounted on the tip of the index finger. The prosthesis is designed to hold objects in a three jaw chuck grip between the thumb and the first two fingers. The thumb is therefore a better location for the primary sensor. In addition all objects must be held in contact with this sensor, so great care had to be exercised when gripping certain objects to ensure this was the case. When additional touch/no touch sensors were mounted on the palmar surface of the hand these objects became easier to grip and manipulate (Kyberd, 1990).
Conclusions
The limited field test of the single degree of freedom variant of the Southampton hand can be considered to be a success, though limited in nature. Five conclusions can be drawn;
- It is possible to modify conventional prostheses to take the Southampton control scheme.
- The control scheme is easy to learn and use. Its utilisation enhances the function of a prosthesis.
- The use of adaptive thresholds on a myoelectric prosthesis eases the use of the device.
- Microprocessor control of a prosthesis allows for easy adaptation to different user requirements.
- The additional cost in power consumption is negligible and far outweighed by the benefits.
Compared to the 'subjects' prescribed hand, for the test hand the standard pick and place operations were very similar in length of time to execute. To provide qualitative data a broader range of tasks would have to be devised to explore the differences, not merely to exploit the modified device while hiding its flaws.
It is possible to construct a much more compact device that would reduce the bulk, weight and problems of unreliability of the current system. This would use more recently developed electronics such as microcontroller devices. The use of a Very Large Scale Integration technology would also allow far better recording of the use of the hand whilst in the field, although this facility and the application of the information gained must be used with caution to avoid arousing the fears of the user population. This was revealed at a United Kingdom meeting of the International Society of Prosthetics and Orthotics where representatives of the British Limbless Ex-Servicemans Association expressed concern over the use of such data to justify the withdrawal of appliances when a similar data logger for a leg prosthesis was demonstrated, (Pearson et al., 1990).
It is important to appreciate the comparison of the prototype system described herein with a standard single degree freedom device is based on the control aspects and not on the mechanical function. The mechanical attributes were similar, as were many of their drawbacks. It is these drawbacks that a four degree of freedom hand addresses (Kyberd, 1990).
Many of the problems encountered with the hardware were related to the highly experimental nature of the equipment used. What is required is a longer trial on a device developed specifically for the application. This task is now in progress under the TIDE (Technology for the socio-economic Integration of the Disabled and Elderly) initiative of the European Community. The experiment is designed to place six units based on the SAMS technology in two centres (UK and Italy) for an extended trial.
Acknowledgements
The authors would like to thank J. E. Hanger and Co. for support of the project, John Cooper and Julian Hawkings of the company for their help. In addition we acknowledge and thank Robin Cooper and Barry Fazakerly of Hugh Steeper Ltd. for their help in the trial and finally to George Lawson for his help and patience.
References:
- Baits JC, Gatto E (1969). The feasibility of an adaptive control scheme for artificial prehension. In: Proceedings of a symposium at the Institute of Mechanical Engineers Czechoslovakia 31st October - 1st November 1968. - Czechoslovakia: I Mech Eng, 1969.
- Barry DT, Leonard JA, Gitter AJ (1986). Acoustic myography as a control signal for an externally powered prosthesis. Arch Phys Med Rehabil 67,267-269.
- Bennet Wilson A (1989). Limb Prosthetics./6th edition New York: Demos.
- Chappell PH, Nightingale JM, Kyberd PJ (1987). Control of a single degree of freedom artificial hand. J Biomed Eng 9, 273-277.
- Chappell PH, Kyberd PJ (1991). The prehensile control of a hand prosthesis by a microcontroller. J Biomed Eng 13, 363-369.
- Childress D (1985). Historical aspects of powered limbs prostheses. Clin Prosthet Orthot 9(l), 2-13.
- Childress DS (1973). An approach to powered grasp. In: Advances in External Control of Human Extremities; proceedings of the Fourth International Symposium, Dubrovnik, August 28 - September 2, 1972. - Belgrade, Yugoslav: Committee for Electronics and Automation.
- Gow DJ, Dick TD, Draper ERC (1982). The phsiologically appropriate control of an electrically powered hand prosthesis. In: Abstracts; International Society of Prosthetics and Orthotics 4th world congress, London, 5-9 September 1983. -Copenhagen: ISPO.
- Gow DJ, Douglas WB. The REACH electrical hand prosthesis for transcarpal amputees, a progress report. In: Abstract; UK annual scientific meeting, 4-6 April 1990, Edinburgh. International Society for Prosthetics and Orthotics - ISPO.
- Herodotus, Calliope IX./Translated by H. Cary. - London: George Routledge and Sons Ltd, 1987.
- Kyberd PJ (1990). Algorithmic control of a multifunction hand prosthesis. PhD. Thesis. - Southampton: University of Southampton.
- Kyberd PJ (1985). Processing of electromyograms for the control of a multifunctional hand prosthesis. MSc Thesis. - Southampton: University of Southampton.
- LeBlanc M, Parker D, Nelson C (1987). Do we need a prosthetic prehensor which is neither a hook nor a hand? In: RESNA '87 Meeting the challenge; proceedings of the 10th annual conference on Rehabilitation Technology, San Jose, California, June 19-23 1987. - Washington, DC: RESNA.
- Meek SG, Jacobsen SC, Gould PP (1987). Extended physiological traction: design and evaluation of a proportional force feedback system. J Rehabil Res Dev 26(3), 53-62.
- Millstein HG, Heger H, Hunter GA (1986). Prosthetic use in adult upper-limb amputees: a comparison of body powered and electrically powered prostheses. Prosthet Orthot Int 10, 27-34.
- Patton JG (1988). Upper-limb prosthetic components for children and teenagers. In: The comprehensive management of the upper-limb amputee./edited by D. J. Atkins, R. H. Meier - New York: Springer-Verlag.
- Pearson T, Convery P, Simpson D (1990). Development of miniaturised equipment to monitor the use of prescribed prosthetic and orthotic equipment. In: Abstract; UK Annual Scientific Meeting, 4-6 April 1990. Edinburgh. International Society for Prosthetics and Orthotics - ISPO.
- Pliny the younger: Natural histories. 2(7) xxvii./ Translated by H. Rackham. - London: W Heinemann 1942.104-106.
- Scott RM (1988). Biomedical engineering in upper-limb prosthetics. In: The comprehensive management of the upper-limb amputee./edited by D. J. Atkins, R. H. Meier - New York: Springer-Verlag.
- Senski T A consumer's guide to "bionic arms". Br Med J 12 July, 126-127.
- Simpson DC (1973). The control and supply of multimovement externally powered upper limb prosthesis. In: Advances in External Control of Human Extremities; proceedings of the Fourth International Symposium. Dubrovnik, August 28 - September 2, 1972. - Belgrade Yugoslav: Committee for Electronics, and Automation.
- Simpson DC, Kenworthy G (1973). The design of a complete arm prosthesis. Biomed Eng 8, 56-59.
|
ISPO Home
