Evaluation Revalued
Robert E. Stewart, D.D.S. *
In any sound program of research and development, whatever the intended goal,
there must inevitably come a time when extensive evaluation of the product is
indicated. Less than fifty years ago, systematic tests of new concepts were
performed more or less routinely by private inventors dedicated to proper
self-appraisal as occasion warranted. In a period less sophisticated
technologically, this fashion in science served its purpose adequately and well.
But with the growth in a more modern era of the large and vastly more
complicated system of scientific inquiry, such as we know it today in government
and industry alike, the requirement for periodic assessment of experimental
results has led to the development of the independent testing laboratory, either
as a part of the basic organization or as a separate contracting institution. So
indispensable has this phase of technical investigation become that now large
sums of money are spent annually in support of evaluation groups who themselves
commonly engage at least in part in research aimed at improving their own
methods and techniques.
With respect to these matters, the Artificial Limb Program has exhibited
ostensibly no basic deviation from the general pattern now characteristic of
other broad exploratory projects involving the cooperation of various
specialists in otherwise distinct disciplines. But because of the peculiar
nature of the amputee problem, the particular state of the art of limb
prosthetics, especially in the upper extremity, and the demands of rather
unusual external influences of one kind or another, the approach to systematic
evaluation has in this case evolved out of a unique history and has,
consequently, given rise to some valuable results in research and education of
which the influence was not fully anticipated in the beginning.
Although in that portion of ALP devoted to the upper extremity much of the
initial investigation was directed toward all-purpose, or "ideal," prostheses
for selected levels of arm amputation, it was soon recognized that the desired
objectives would be served more effectively were a variety of components made
available for assembly into various combinations the better to provide for the
particular needs of the individual patient. As these components were developed, prototypes and, later, production units were subjected
to systematic testing by the Prosthetic Devices Study, an organization
established for this specific purpose within the Research Division of the
College of Engineering of New York University.
At this point, evaluation generally furnished much needed data concerning the
usefulness and reliability of individual units in direct comparison with
previous similar parts but without regard for the influence of socket fit, type
of harness and harness adjustment, type and extent of training, individual
amputee preference, and other factors. Because methods suitable for the
evaluation of techniques had yet to be introduced, early evaluations of
components brought with them the subtle dangers of misinterpretation owing to
the indirect effects of pre-existing errors in socket or harness, to say nothing
of the possibility of the influence of one component upon the performance of
another used in conjunction. In these circumstances, a great deal was left to be
desired in reference to the over-all problem of upper-extremity prosthetics.
To fill the gap, there was initiated in 1950, in the Department of
Engineering at the University of California at Los Angeles, the so-called "Case
Study," with the purpose of bringing together all available information, of
viewing systematically the results obtained by use of various combinations of
devices and techniques, and thus of developing a set of general principles of
management for the upper-extremity amputee. As the Case Study progressed, there
arose an increasing awareness of the necessity for teamwork in the proper
application of such knowledge as there was, and by 1952 the Prosthetic Devices
Study was called upon to conduct an evaluation of the results of the UCLA Case
Study.
It was obvious that, if such an evaluation were to be conclusive, large
numbers of cases under varying geographical conditions would be needed for
observation and that therefore the services of a number of clinic teams
throughout the country would be required. Although the Prosthetic and Sensory
Aids Service of the Veterans Administration, long the chief sponsor of the
Artificial Limb Program, had already established some thirty prosthetic clinic
teams, and although these groups were readily available for participation, it
was patently mandatory that they be trained in the latest methods before any
reliable program of evaluation could be initiated. Accordingly, short-term
courses for clinic team members physicians, therapists, and prosthetists were
organized and conducted at UCLA beginning in 1953. The formation of new clinic
teams outside the VA framework was encouraged, and these, along with a few
private clinic teams already in existence, were invited to participate.
The education program leading to the Upper-Extremity Field Studies, the name
applied to this part of the NYU evaluation work, proved to be a pioneering
effort in its own right. While results of research were being made available to
clinic teams for general use in a remarkably short time after the initiation of
laboratory work, the continued association of clinic personnel with the research program through participation in the Field Studies had a
definite impact on those responsible for amputee care. Thus the Field Studies
came to be a series of complex investigations designed not only to evaluate the
usefulness of available upper-extremity prostheses but also to determine the
effectiveness of the management procedures elucidated by the UCLA Case Study.
Simultaneously, and almost unavoidably, the process of accumulating voluminous
clinical data on one segment of the population led to a general upgrading of
industry practices in amputee service and furnished the basis for further
research into the needs, physical and mental, of the armless.
Because the NYU Field Studies represent the first, and thus far the only,
attempt in the United States to appraise the status of upper-extremity
prosthetics directly and on such a broad scale, and because the results present
such a wealth of information not available elsewhere, this and the following
issue of Artificial Limbs are given over to presentation of a series of summary
articles divided into two parts the first (this number) concerned with the
educative aspects of the work, the second (Autumn 1958, Vol. 5, No. 2) with the
research implications. For those who would undertake further study and
interpretation in the interest of scholarship, the original data, far too
detailed for thorough analysis by other than biostatisticians, are available in
the College of Engineering of New York University, New York City.
In reviewing the material offered here, it is appropriate to keep in mind
that the Field Studies constituted a new voyage into an area in which both
subject matter and method of approach were uncharted and unexplored.
Understandably beset by all the problems of design, organization, and execution
typical of adventures into the unknown, they now reveal certain deficiencies
most readily viewed with benefit of hindsight. In all probability, the true
value of the Field Studies remains to be had in the further application of the
principles not only in the field of limb prosthetics but in other, more general
areas of physical handicap as well.
|
