MS-*- a ^ *£! ,/"-^ *fB*»* aas/^ ' *•♦***$ TV? QJortwll Hmweraitg Eibranj 3tl)ara. Nrtn fork PROM ..CflTObii...toiver»i.tj...Xiil3[rftrjr in excbaage Date Due \r 8 'Si JAN? r 1 r i52(J ^SSHD ^H&Ug ^-^4^ f Cornell University Library LC1043 .T67 Trade tests in education olin 3 1924 030 614 428 »'j4 • Trade Tests in Education By Herbert Anderson Toops Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, in the Faculty of Philosophy, Columbia University Published by QCeacgettf College, Columbia 33mbersttp New York City I921 Cornell University Library The original of this book is in the Cornell University Library. There are no known copyright restrictions in the United States on the use of the text. http://www.archive.org/details/cu31924030614428 Trade Tests in Education By Herbert Anderson Toops Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, in the Faculty of Philosophy, Columbia University Published by GDeacherg College, Columbia ©m'betjsitp New York City IQ2I \ ■4 O/ v/ Copyright, ig2i, by Herbert Anderson Toops y*^ U: f\5\o&3.+ ACKNOWLEDGMENTS In a work of this kind, reporting as it does in large part the methods developed by an army, group, the author is evidently indebted to many persons, both directly and indirectly. To Dr. E. L. Thorndike, whose direction and inspiration has been such as to lead to a greater undertaking in this work than originally planned, much special credit is due for such merit as this work may possess. Special mention should be made of others whose guidance and aid have directly been very helpful in this work: Dr. Truman L. Kelly, whose statistical guidance has made pos- sible the analysis methods developed herein ; Dr. Arthur D. Dean, whose vocational philosophy is reflected from many pages of this book; Miss Lucy L. Brown, teacher in charge of the vocational testing classes at the Manhattan Trade School for Girls, whose cooperation and interest have made possible the analysis of vo- cational guidance data; Dr. A. H. Ryan, director of the Indus- trial Hygiene Department of The Scovill Manufacturing Co., for industrial data bearing on the evaluation of talents for the job of youths newly entering industry; Dr. Howard G. Burgess, of the Vocational Training Department of the New York Military Train- ing Commission, who has supplied statistics on the grades completed in school by working boys recently leaving school in the state in New York; Mr. James D. Jackson who spent many weary hours in most helpful statistical aid; Drs. J. Crosby Chapman, William McCall, and Arthur S. Otis who gave valuable aid in special aspects of this work. Acknowledgment is made to a host of others, whose contributions, although not specially mentioned, are none the less valuable. Herbert A. Toops CONTENTS CHAPTER PAGE I. The Value and Uses of Trade Tests ... i 1. The value of tjrade school instruction a debatable iss,ue. 2. Lack of units in which to measure trade proficiency. 3. The value of trade tests in vocational education. A. In testing progress, and maintaining interest in trade in- struction. B. In making proper allowances for individual differences in rates of acquiring trade proficiency. C. In educational and vocational guidance and school place- ment work. 4. The broadness of scope of "trade test methods.'' 5. The classification of trade tests. A. Tests of proficiency or success in a trade. a. Description of the oral, picture, and performance trade test methods, and reproduction of a typical test of each. B. Tests of trade capacity, potentiality or promise. a. The varied nature of tests of trade capacity. b. Description of a typical performance test of ability to learn a new trade operation. C. The fundamental basis of all trade tests, a statistical tech- nique common to all test methods. II. Tests of Trade Proficiency or Success, and their Adaptation to School Work 17 1. Comparison of the tests of the interview and "hiring on trial" with the oral trade test method. 2. The technique of trade test construction. A. Selection and revision of oral trade test questions. B. Selection and revision of picture trade test questions. C. Standardization of trade tests. D. Selection of tasks for performance trade tests. E. The multiple choice written trade test. a. An experiment, with results obtained in the use of a writ- ten bricklayers' and a farmers' test. F. An experiment, with the results obtained from a written infor- mation test made up in three different examination methods. 3. The factors determinative of the best examination method for school use. A. Adaptation of the one-word-answer question to trade and non-trade school examinations. B. The merits and possible criticisms of trade test methods in school examinations. vi Contents III. Tests of Trade Capacity and Results of the Eval- uation of their Use in a Trade School ... 63 1. The evolution of present-day test philosophy. 2. The problem of industrial placement. A. Results from tests of vocational capacity secured by other investigators. B. Use of job analysis in vocational placement. 3. An experiment, with results from the evaluation of trade school tests of trade capacity. A. The testing department. B. The tests. C. Methods of, and results from evaluation of qualification card and supplementary data. a. Intercorrelations of the tests. b. Follow-up report on tested pupils who later entered com- mercial high school. c. The value of the tests. IV. General Considerations in Testing and Training for Proficiency and Promise in the Trades . . 96 1. Trade school graduates not the tradesmen of industry. 2. Specialization in industry and its effects upon tradesmen's talents for the job. 3. The intellectual narrowness of industrial trade training. 4. Distributions of talents of tradesmen, and their significance for training and selection of tradesmen. A. Tradesmen's intelligence. B. Tradesmen's general education. 5. The contribution of general education to vocational proficiency. A. Results in the case of general clerks. B. Results in the case of eyelet machine operators. 6. Changes in trade school administration suggested by trade test methods. A. Adaptation of trade school instruction to the intelligence level of the pupils. B. Use of trade examinations as learning incentives. C. The standard job method. D. The merits of the self-administrative recall examination. Appendix: Statistical Methods II2 Bibliography . ... . . 116 CHAPTER I THE VALUE AND USES OF TRADE TESTS Vocational schools, as opposed to schools for general education, have as their avowed aim the preparation of youth to earn a living in vocations. The extent to which their aim is achieved by such schools as are called vocational schools is at present a debatable issue. One reason for this is the lack of means with which to measure the human products of such schools. Inability to know the product of the school at once predetermines an inability to better the school administration on a basis of the needs of students. Any betterment in present methods of measuring the product of a school at once produces the chance for bettering the school administration on the basis of the needs of the students. Only until very re- cently has trade skill and knowledge been measured in more refined terms than subjective personal judgments of "skilled man" and "helper," "successful" and "unsuccessful," "competent" and "incompetent." Army trade tests solved this problem for the army; adaptations of the method may solve the problem for our vocational schools. Graduates of our vocational schools are at present of equal merit, so far as measure of their merit goes, for they all re- ceive diplomas. Industry wants a more accurate measure of their hiring worth ; and the school desires to recognize the varying merit of its product. Unless the prospective employer can successfully rate the human product of the vocational school he is at a loss to know in placing a graduate of a vocational school in industry whether he should adopt a course different from that followed in the case of any other ap- plicant for the job. The average employer is skeptical of the value of the "book learned" tradesman. His skepticism, we must believe, is not directed so much toward vocational school graduates as against particular exceptional individuals in the past who have happened to fail when given the test of industry, the test of doing the job itself. Could graduating students be rated for general trade proficiency against men in the trade itself, much would be gained in the way of proving just what is the value of any specific vocational training program. 2 Trade Tests in Education The percentage of a graduating vocational school group which equals or exceeds in general trade skill the median tradesman in industry is a theoretical measure of the efficiency of the vocational school instruction. The army trade tests, standardized on random sampling of men in industry, fulfill these requirements of measures of final trade proficiency after a period of training. They may be used equally well to measure trade proficiency at any point of its acquirement. Vocational school instruction has so generally meant trade school instruction that the author will not attempt here to make a distinction between the two. It is evident that there is no dis- tinction between the two in the field of tests, which have a common statistical technique and hence make no distinction, save in content, between intelligence tests, educational tests, vocational tests, trade tests, or any other variety of test which has as its aim the pre- diction of peoples' abilities to do something, or to act in a certain way. Accordingly, the terms "trade test" and "vocational test" will be used interchangeably. VALUE OF TRADE TESTS IN EDUCATIONAL ADMINISTRATION The values of trade tests in education are such as pertain to any objective, standardized trade examination method. Standard- ized tests, or tests designed by taking into account the testing princi- ples thus far discovered, will have a better chance of fulfilling the demands made upon them than tests not so constructed. For argument's sake, one may take for granted the assumption that a vocational education course under consideration is a desir- able course, and that it is as good a course as may be expected to be produced. Not all students will apply themselves equally to the tasks of learning the trade instructions given them in such a course, nor profit equally from those received. Tests of progress, given at periodical intervals, are needed to keep pupils up to a reasonable standard of interest in their courses and for purposes of grading students at the end of the year, or at the end of the course. If such tests can be standardized, then so much the better. But if not, as will often be the case, then vocational instructors may be able to profit greatly by attention to the methods of question making and test lormulation used in army procedure. Value and Uses 3 Of two students of equal general trade or vocational ability, one may be unusually good in operation A and be unusually poor in B; the other unusually poor in A but good in B. If, during the course of instruction, the strengths and weaknesses of each student could be made known, much valuable time of the student might be saved, and a much better allround ability gained by the teacher's adapting his teaching to individual needs. Diagnostic tests of an- alyzed or subdivided elements of general trade ability are needed. These will be developed in the future. A knowledge of what the army was able to do in measuring trade ability should help the prospective test maker in making up such tests. Where such tests have been developed to the point where they are sufficiently accurate for the purpose, we may expect trade instruction to be done largely by the job or operation sheet method, and the pupil to be advanced from job to job or operation to operation just as fast as he shows a test measured proficiency of X in each job or operation. Thus will result economy of time required in learning a trade, since each pupil will progress just as fast as he is capable of doing. This is the method found successful by the S. A. T. C. during the war, and since adopted by the E. and R. Schools of the Army in teaching trades to thousands of boys. In the past the test of whether or not a given pupil was ready for promotion to a new job has been a subjective estimate by the instructor, aided by performance on the job, it is true, but nevertheless a test which could not be duplicated by an outside person without a personal knowledge of the pupils being rated. Tests of promise or capacity have been used in the field of general education for a number of years; they have not been so generally used in trade education. Much research work, greatly encouraged by the success of the army intelligence tests, has demonstrated the feasibility of classifying school children on the basis of their mental ability, or on the basis of a composite score of a number of tests designed to predict pupils' abilities to profit by the instruction given them. The E. and R. Schools of the Army have thus classified many thousands of soldiers, drawn from every part of the country and from foreign countries, with varied and non-comparable pre- vious school experiences. The value of such educational guidance has been already proved. Trade test methods, making use of refined tests of trade capacity, 4 Trade Tests in Education promise to throw some light upon the age-old problem of "What vocation should a boy enter?" Whether a boy should take up general education, trade education, or professional education, is a problem which we may hope to solve only by development of a vocational philosophy formulated with the aid of pertinent facts of human abilities and vocational opportunities made available by the statistical methods common to all survey methods. In con- junction with psychological tests, physical tests and questionnaires, tests of trade capacity may be expected to become a valuable aid to the vocational counsellor of the future. The best large vocational schools have their own placement or em- ployment bureaus. The problem of such bureaus is to evaluate into one single fitness score, by a subjective process, the diversified talents of boys and girls as a basis for a twofold categorical divi- sion of resulting action into "recommended" and "not recommended" for the prospective position, whether job in industry, or school instruction in an advanced school. Such estimates, when based upon proficiency in performing trade operations, may be greatly enhanced by more accurate methods of measuring trade skill. The two aspects of such placement are: (a) knowledge of the pupil's abilities, and (b) the requirements of the job, in the way of human abilities and acquirements. Text-books on vocational guidance give many facts about trades, tradesmen and trade conditions. They generally fail to specify how diverse human talents for the job, or diverse demands of the job upon the worker, may be evaluated. These problems will be ultimately solved by methods available to, or to be developed by, the test statistician. This larger aspect of "fitting the job and the man" is even more important than the more restricted problem of measuring the talents for a job, present or future, measurable by trade tests. All make use of a common statistical technique. Accordingly, we should mean by "trade test methods" this larger aspect of evaluat- ing measurable human talents in terms of some independent criterion of job proficiency. Inasmuch as tests of present trade proficiency are promises of future performances (immediate or remote) on the job, all trade tests in the last analysis are tests of promise or capacity. The common statistical problem of all such methods is the problem of measuring mutual dependence of related measures of trade capacity. Value and Uses 5 CLASSIFICATION OF TRADE TESTS AND EXAMPLES OF TYPICAL FORMS The term "trade test" is used to refer to any test involving test material taken from the trades. Trade tests differ, then, from any other forms of tests mainly in a difference in content. They may be categorically divided into two classes according to the purpose of the test, and further sub-classified according to form of test material and method of administration, as follows: I. Tests of proficiency or success in a trade. 1. Verbal. a. Oral Administration. (1) Army Oral. (a) Oral Answers. (&) Written Answers. (2) Picture. b. Written Administration. (1) One-word-answer. (2) Multiple Choice. 2. Performance Tests. II. Tests of trade capacity, potentiality, or promise. Sub-classes of this group: Practically every known variety of test, including the above verbal forms, has been used by some investigator at one time or another. Trade tests were used in the army to determine which men, of those claiming specific trade experience, were sufficiently competent to be detailed for special duty in the trades maintained in the army. An example of an army oral form of test, used for measuring trade proficiency, is reproduced herewith in an eyelet machine operator's test. This test may be given individually and orally by an examiner, the answers being oral; it may be given orally to a group, the answers being written on blank paper; or it may be administered by the use of printed or mimeographed questions, the answers being in the one-word-answer form. EYELET MACHINE OPERATOR (Tender) (Army oral form of test with answer in the one-word-answer form.) 1. What do you call the large main shaft at the top of the machine? Ans. Cam. 2. What rubs on the cams all the time to oil them? Ans. Tassel. 6 Trade Tests in Education 3. What does the scrap wind up on? Ans. Reel. 4. What drives the reel from the bottom cam shaft? Ans. Belt. 5. What do you call the die which does the first operation? Ans. Blanking. 6. What is the name of the large part which moves back and forth and carries the work from one die to another? Ans. Slide. 7. What do you call the part which moves up and down and carries the punch at its lower end? Ans. Plunger. 8. What do you call the pulley on the drive shaft which is not fastened to the shaft? Ans. Loose. 9. What is the shaft underneath the machine called? Ans. Cam. 10. What are the small springlike parts called which grab the work and carry it from one die to another? Ans. Finger, n. What do, you use to measure the thickness of the metal? Ans. Micrometer. 12. What do you use to cut off the strip metal? Ans. Shears (snips). 13. What is used to tell if the finished work is the correct size? Ans. Gage. 14. What do you call a gage which shows the largest and smallest size that can be allowed on a finished piece of work? Ans. Limit (maximum and minimum) ("max and min"). 15. Of what metal are the reels made? Ans. Brass. 16. What is the shape of the plungers? Ans. Square. 17. When five rows of blanks are'to be punched from the strip metal, which row is punched first? Ans. Middle (center). 1 8. What is there on the end of the air pipe to let the air out at just the right time? Ans. Valve. 19. How many holes are there in the drive pulley to turn it by hand? Ans. 2 (4). 20. What do you call the tool used to turn the fly wheel by hand? Ans. Bar. 21. What part of the cam shaft regulates the air? Ans. Cam. 22. If the top cam shaft turns at 60 revolutions per minute how many revolu- tions per minute does the bottom cam shaft turn? Ans. 60 (same). Value and Uses 7 23. What guides the finished work into the pan? Ans. Spout (pipe). 24. What is the sheet iron around the gears called? Ans. Guard. 25. What does the pan for the work set on? Ans. Box. 26. What do the dies set in? Ans. Holder. 27. What tool do you use to turn the roll feed? Ans. Wrench. 28. What material is used to line a brake on machines which have a brake on the drive shaft? Ans. Leather (wood). 29. How is the punch fastened to the plunger? Ans. Screws in (wrench) (thread). 30. What is the washer made of that is sometimes used on the reel shaft? Ans. Leather. 31. What do you call the attachment sometimes used around a punch to make the work let loose from the punch? Ans. Thimble. 32. How are the bottom cams made so that they can be timed? Ans. Split (halves) (screw). 33. What tool do you use to set the bottom cams on the bottom cam shaft? Ans. Screw-driver. 34. Of what material is the part of the slide friction made that rubs on the slide? Ans. Wood (maple). Army picture tests were administered orally and the answer? given orally. They were used as a check on the ratings made on the oral tests, which are somewhat subject to language difficulties and to coaching. The method is essentially the army oral method, but with pictures used to describe the trade situation in the place of the descriptive phrases or clauses which must be used in the oral method. The lathe hand test, herewith reproduced, pages 8 to 14, is typical of this form of examination. The performance test aims to duplicate a shop situation, the test subject being rated on the quantity and quality of work done on a job or problem assigned. These have the merit, not possessed by ordinary shop jobs, of being so chosen as to require a maximum amount of manipulation of tools, use of trade knowledge and trade skill. A sample performance test, used in testing ability to use machinists' measuring tools, is shown on page 15. This particular test has as its products a series of dimensions entered on a form blank, which are scorable by stencil. 8 Trade Tests in Education Tests of trade proficiency were developed later than tests of trade capacity. Tests of trade proficiency were developed to a high point of efficiency during the War. The possibilities of large scale re- search in developing tests of trade capacity have not yet been tested. It is the special function of this research to point out the value of such work by statistical inquiry into particular selected vocations and trade school methods. LATHE HAND TEST Picture i i. What is the part at G called? Ans. Steady-rest. 2. What letter shows the live center? Ans. D. 3. What is the lever at K used for? Ans. Screws (threads). 4. What do you call the part at C? Ans. Face-plate. 5. What do you call the lever at A? Ans. Back-gear. 6. What is the part at L used for? Ans. Gears (change the gears). 7. What letter shows the cone? Ans. B. 8. What letter marks the tail stock? Ans. H. 9. What is the part at F called? Ans. Follower-rest (follower). 10. What is the part at J called? Ans. Rack. 11. What is the part at I called? Ans. Lead-screw. 12. What is the part at E called? Ans. Tool-post. Picture 2 13. What does that micrometer read? Ans. .125. 14. What is the part at A called? Ans. Ratchet. Picture 3 15. What do you call that lathe tool? Ans. Dog. 16. What does the part at A fit into on the lathe? Ans. Face-plate (slot on face-plate). ,A ,B FiS 8 Fif I Plate i. Page i of Lathe Hand Picture Trade Test io Trade Tests in Education Picture 4 17. What do you call that tool? Ans. Boring. Picture 5 18. What is that tool called? Ans. Dividers. Picture 6 19. What do you call that lathe tool? Ans. Center. 20. How many degrees in the angle of the point of that center? Ans. 60. Picture 7 21. What do you call that tool? Ans. Center-punch. Picture 8 22. What is that tool used for? Ans. Threads (screws). Picture 9 23. In what operation is that tool used? Ans. Drilling. Picture 10 24. What letter marks the apron of that lathe? Ans. G. 25. What letter marks the bed of that lathe? Ans. H. 26. What letter shows the part which you turn to throw the longitudinal feed in gear? Ans. C. 27. What letter shows the part which you turn to throw the automatic cross-feed in gear? Ans. D. 28. What letter marks the part which works the compound-feed? Ans. I. 29. What letter shows the swing of that lathe? Ans. A. Picture ii 30. What operation is being done in that picture? Ans. Boring. 31. What is the part at A called? Ans. Face-plate. 32. What is the part at B called? Ans. Dog. F/^ II. 4 @ 'GJ 3 FlJ.IZ. ' FlJ.I3. FI9.I4-. Fit) 16 Flj-17. Fi<). 18. Plate II. Page 2 of Lathe Hand Picture Trade Test 12 Trade Tests in Education 33. What is the part at C called? Ans. Boring-bar. Picture 12 34. If the lead screw on that lathe has 8-threads per inch, how many threads per inch will that lathe cut with those gears? Ans. 16. 35. What is the gear at A called? Ans. Idler (intermediate). Picture 13 36. What is the tool in that picture called? Ans. Morphadite (hermaphrodite). Picture 14 37. What tool is being squared-up in that picture? Ans. Threading. Picture 15 38. What operation is being done in that picture? Ans. Taper. 39. If the stock in that lathe is 2-feet long and you want a taper of ,%-inch per foot, how far do you set-over the screw G? Ans. yi (>£-inch). Picture 16 40. What tool is being squared-up in that picture? Ans. Inside-threading. Picture 17 41. What do you call that tool? Ans. Vernier-caliper. 42. What is the smallest fraction of an inch which that will measure? Ans. 1 /iooo. Picture 18 43. What do you call the part at B? Ans. Collet (collar) (washer). 44. What do you call a slot like that at A? Ans. T-slot. 45. How far does one revolution of the handle at C advance the point of the thread tool? Ans. yi (J^-inch) (.125). Picture 19 46. What is the name of the tool at 1? Ans. Left-hand-facing. 47. What is the name of the tool at 2? Ans. Right-hand-facing. 48. What is the name of the'tool at 4? Ans. Diamond-point. ~ on the average, cor- relates highest with the other two tests. With this criterion, recall ranks first, recognition second, and true- false last. 48 Trade Tests in Education The recall form, with fifty questions used in all three forms, will yield the highest reliability coefficients of all three forms. For further computations, all persons taking recall are grouped together, indiscriminately, irrespective of method ; and likewise for the recognition and recall. From these total distributions the total reliability coefficients and total average times of the table are com- puted. Now, although for fifty questions, i.e., the same number of ques- tions, done by subjects on recall, recognition and true-false forms respectively in an average number of minutes of 6.9, 5.6, and 3.6, the recall method is the most reliable, it does not necessarily follow that the method is the most reliable for the same average amount of time spent in examination. The average times show that, in the same time required to give one recall question, 1.23 recognition questions may be given and 1.92 true-false questions. The question then is, "Will the added nu nber of questions which may be given in the same time be sufficient to in- crease the reliability of either recognition or true-false to a point exceeding the reliability of the recall?" To answer this question theoretically, we make use of the usual formula for reliability coeffi- cients of n forms of a test with n forms, r nrn (1) _ "" i + („-i) ru in which, r„ n is the reliability to be calculated, r n is the reliability of the halves of the test, and n is the number of multiples of the halves considered. Table II gives the m values, correlations of one 25-question form with another 25-question form. When n = 2, we get the reliability of a 50-question form of the test with another 50-question set; i.e., Form A with Form B of the same test. The formula then becomes (2) „ 2r n r-n (recall) x 4. Tn Solving formula (2) directly for recall, and formulae (3) and (4) below, respectively for recognition and true-false; i.e., taking n of such value that equal times will be spent in examination, such time as is required to give 50 recall questions, or 6.9 minutes, we obtain the last horizontal row of Table II. 2(i.23>m (3) r n (recognition) t 4. [2(1.23) — l]-r u Adaptation to School Work Table II 49 Comparison of Reliability Coefficients of the Recall, Recognition and True-False Tests (All Methods Grouped Together) Recall Recognition True-false Reliability (rn) of halves. 124 cases. Two forms of 25 each. .448 t •385 •34° Reliability of two 50-question sets. Formula (2) .618 •556 ■507 Average time in minutes to do 50 questions 6.9 5-6 3-6 Number of questions per unit of recall time 1. 00 1-23 1.92 Number of sets of 25 ques- tions to get equal reliability of .618 2.000 2.602 3.140 Reliability of Form A with Form B when 6.9 minutes examination time is i^sed .618 .607 .664 (4) 2 (1.92)- m Hi. (true-false) 1 + [2(1.92) — l]-r u This table yields the interesting result that, with all methods grouped together, with equal examination time, the true-false test is the most reliable, the recall less reliable, and recognition least reliable. In order to obtain an idea of the dependence of reliability upon the amount of time taken in an examination, we may plot the three equations involving reliability coefficients and examination time. This has been done in Fig. 28, where the ordinates are r nn , and abscissae n, or rather, n reduced to time in units of the recall time, which is used as a standard of comparison for the other two, using the scale, 1 n' = yi. (6.9) = 3.45 minutes, in which n' is the average time in minutes required to do 1 multiple of 25 recall information questions. The numbers of questions, Q, done in t minutes, are given respectively by the formulae : (5) Recall, Q = 7-25 t. (6) Recognition, Q = 8.83 t. (7) True-false, Q = 13.89 t. 5o Trade Tests in Education These equations may be conveniently plotted as straight-line equa- tions on the same graph, using coordinates on the right-hand margin- of the graph. From the graph one can thus determine both the num- ber of questions and the examination time required to yield any pre- determined reliability coefficient. This graph holds good only for the range of ability represented by the test group of subjects and for the conditions of the experiment above outlined. %l.00 Fig. 28. Theoretical Relationship of Reliability Coefficients to Examination Time. There is some question whether grouping all persons regardless of method or order of giving the test may not obscure some of the differences which exist in the reliabilities. The average times, for instance, show that if a group of persons given the true-false or the recall or recognition first, followed by either of the other two, the group tested tends to take more time on the test taken first than any other group of persons given any other test first. Accordingly, the reliabilities have been computed, grouping together all the 76 people who took recall first, the 26 who took recognition first, and the 22 who took true-false first. Evidently the results on these tests are not influenced by previous consideration of any other form of test, and so correspond as closely as may be to an actual recall or other Adaptation to School Work 5i test as given alone and without any other. The following table results : Recall Recognition True-falst m ( 25 questions •743 .689 .561 f 50 questions r& { Form A with •853 .816 .719 [ Form B Average time in minutes 7.80 7.24 4.24 j Equal examination time r " n \oi 7.80 minutes ■853 .826 .824 The reliabilities here are much higher. The added "conscientious- ness" when persons take the true-false first seems to make their aver- age time, which is not increased proportionately, such that the true- false is only about on a par wkh the recognition form when equal amounts of examination time are taken. The conclusions to be drawn from this experiment are: (1) When equal numbers of in- formation questions are given on the three forms of information test, the recall is always the most reliable, followed in order by recognition and true-false forms, but that (2) where equal amounts of examination time are taken on the three forms of test, the relia- bilities do not differ greatly. RELIABILITY OF VARIOUS TRADE TESTS The above technique allows us to compare the reliability coeffi- cients of various trade tests based on different numbers of ques- tions. Reliability coefficients of the same magnitude are equal only when the range of ability is the same in the two cases being com- pared. Accordingly in Table III there is given a brief qualitative description of the group tested and their range of ability, as well as a brief qualitative description of the various tests, the number of questions in the test, the reliability of the halves of the test com- puted between the odd and even numbered question scores of the test, the reliability of Form A with Form B of the same test, — 1 + r n , and the reliability of Form A with Form B if fifty questions had been used in each test. The last column is thus the comparative measure 52 Trade Tests in Education of the different tests, since it reduces all tests to the same number of questions. It is impossible with the data at hand to compute the reliabilities with equal amounts of examination time taken. The author wishes to emphasize the point that all correlation coefficients, of which reliability coefficients are but one variety, must be inter- preted in terms of the range of ability; concretely, that since the range is limited to eighth grade students only, .795, the reliability of the girls' trade school extension students on the one-word-answer written switchboard test, is as high a reliability as would be repre- sented by a much larger figure if several grades were to be included and the tests given to a larger range. Again, the best measure of an examination method is not the length of time spent in examination, but rather a composite of time spent in administration plus the time required for scoring. The time per person required for adminis- tration is often negligible as compared with the time required for scoring a test. From Table III it will be noticed that the reliability of all trade tests, measures of acquired learning, is uniformly high save in the switchboard operators' test above mentioned, which is probably affected by the fact that only twenty questions were used in the examination, and in the test of progress in the university class in mental measurements likewise. The general information tests, or- dinarily rated as tests of intelligence, are in a distinctly lower class as regards reliability. This table presents the reliability coefficients of all the trade tests which the author has been able to collect up to the present time. Trade test methods give high reliability coefficients, provided enough questions, and sufficiently difficult or rather well scaled questions, are used. Forty selected difficult questions on the farm- ers' recognition test gave a higher reliability coefficient than one hundred and ten unrevised questions. Selected, or revised ques- tions on the soldiers' mechanical test gave a higher reliability than the unrevised questions. The one-word-answer form of test gen- erally gives higher reliabilities than the recognition or true-false forms. SCHOOL EXAMINATIONS Because of its high reliability and ease of administration and because it embodies all the superiority of the recall form of exam- ination over the recognition form, the one-word-answer method is Adaptation to School Work 53 TABLE III Reliability Coefficients of Various Trade Tests 73 ■ft. a o 8 >> •S 1 -a 5 C .O "«? to a O - -* -0 + Mi basis of so questions. Form A with Form B Eyelet 20 Three weeks One - word - answer. 50 .850 .919 .919 Machine to one year Eyelet machine. Tenders of experience Oral trade test. Bricklayers 14 Apprentice to Army oral. One-word- 38 .929 .964 .985 Standardiza- expert inclu- answer. Army oral tion Group sive 4 A, 5 J, SE. bricklayers test. Girls' Trade 14 All eighth One - word - answer 20 .436 .607 • 795 School Exten- grade pupils. written. S w i t ch- sion Students board oral test. Farmers and 30 8 women Multiple choice (4 "Easy" .926 .962 ■947 University novices. 12 choices) written 70 Students men novices. 10 farmers. questions. Farmers' test. 30 "Hard" 40 ■899 •947 •958 30 Al no •9S7 •979 •953 Men and 124 "Above col- General information Women lege entrance test. All methods Summer require- combined. School ments." Recall. One ■ word - 50 •448 .618 .618 College answer. Students Recognition of cor- rect answer. 5 al- ternatives. SO .385 -556 • 556 True-false . SO .340 ■ S07 • 507 Men and 6l Graduate stu- School test in mental 20 .25 .40 .625 Women dents in col- measurements. True- College lege. false. Students 61 Same in one-word- answer form. 20 .50 .667 .833 Soldiers 271 Vocational courses E. & R. School. Use of tools unrevised picture mechanical test. 46 .796 .886 ■894 223 Reyised picture me- chanical interest 45 .849 .920 .928 test. 54 Trade Tests in Education desirable in school examina tions. The one-word-answer method can easily be adapted to school examinations, especially in those subjects which have a high information content : all types of science, physics, chemistry, botany, zoology, vocational courses, and technical subjects such as mathematics, statistics, mechanical drawing, logic. Any study which possesses a large number of terms peculiar to such studies, paralleled by "trade terms" in the trade, lends itself admirably to this method. A physics examination, designed to illustrate the ease with which such questions may be written in a science subject, is herewith reproduced. GENERAL PHYSICS EXAMINATION To illustrate the trade test form of question i. What do you call a minute particle of matter which is made up of atoms? Ans. Molecule. 2. What Physical property of iron allows it to be drawn into a wire? Ans. Ductility. 3. What do you call the product of the mass of a body multiplied by its accel- eration? Ans. Force. 4. What physical property of a body tends to keep it in motion whence once set in motion? Ans. Inertia (momentum). 5. What do you call one-half the product of the mass of a body multiplied by the square of its velocity? Ans. Kinetic-energy. 6. What is the C-G-S unit of force? Ans. Dyne. 7. What is the diffusion of a fluid through a membrane called? Ans. Osmosis. 8. Who discovered the law of falling bodies? Ans. Newton. 9. How many feet per second of acceleration is given to a falling body by gravity? Ans. From 32.00 to 32.16. 10. What do you call a glass tube bent so as to drain water over a point higher than the surface of the water? Ans. Syphon. 11. How much is the weight of one cubic centimeter of water at its greatest density? Ans. 1 -gram. 12. If the volume of a given weight of gas at constant temperature is doubled, how many per cent is the pressure decreased? Ans. 50. Adaptation to School Work 55 13. What property of a tone is determined by the vibration rate? Ans. Pitch. 14. What do you call it when rays of light are bent by going through a prism? Ans. Refraction. 15. What shape of mirror reflects parallel rays of light to a focus? Ans. Concave. 16. What are the first three colors of the primary spectrum, in order, at the long wave length end? Ans. Red-orange-yellow. 17. At what Centigrade temperature does water freeze? Ans. Zero (o). 18. What do you call the variation of a magnetic needle from a true north and south line? Ans. Declination. 19. What kind of electricity is produced by friction? Ans. Static. 20. What is the unit of electrical current called? Ans. Ampere. 21. What other temperature scale is there besides the Fahrenheit and Centigrade scales? Ans. Reaumur. 22. What do you call the force which makes a body fall to the earth when it is dropped? Ans. Gravity. 23. What do you call a space from which most of the air has been pumped? Ans. Vacuum. 24. What class of substances is generally easily volatilized? Ans. Liquids (fluids). 25. What liquid is generally used in a thermometer for measuring temperature of more than forty degrees below zero? Ans. Alcohol. 26. What do you call the ratio of the weight of a substance to the weight of the same volume of water? Ans. Specific-gravity. 27. What is the unit of electrical power called? Ans. Watt. 28. What is the unit of mechanical power in F.P.S. units called? Ans. Foot-pound. 29. How many watt-hours in a kilo-watt-hour? Ans. 1000. 30. What do you call the product of a force multiplied by the distance through which it moves? Ans. Work. 31. What physical property has a body which causes it to rebound when dropped from a height upon a solid surface? Ans. Elasticity. 56 Trade Tests in Education 32. What do you call it when a sound wave has been reflected from a building and comes back to you as a distinct word? Ans. Echo. 33. What is the name of the hypothesis which says that no force is ever created anew or ever destroyed? Ans. Conservation-of -energy. 34. What do you call the force which holds together two molecules? Ans. Cohesion (adhesion). 35. What is the force called which holds a soap bubble together? Ans. Surface tension (capillary attraction). 36. What do you call it when water takes up a quantity of gas? Ans. Absorption. 37. What do you call that part of a body about which, if balanced, it would be fiee to revolve in any direction with equal ease? Ans. Center-of-gravity. 38. What do you call half the swing of a pendulum's arc? Ans. Amplitude. 39. How many foot-pounds in a horse-power? Ans. 33000. 40. Who formulated the hydrostatic law which says that liquids transmit pres- sures equally in all directions and at right angles to the surface? Ans. Pascal. 41. If the pressure on a block of wood two feet below the surface of water in a tank is one pound, how many pounds will it be if placed four feet below? Ans. 2. 42. A block 2 cubic centimeters in volume weighs 8 grams in air; how many grams will it weigh in water at 4 degrees Centigrade? Ans. 6. 43. How many feet will an ordinary suction pump lift the water? Ans. From 27 to 32. 44. How many feet per second is the velocity of sound in air? Ans. From 1080 to 1150. 45. If a string four feet long vibrates twenty times per second, how many times per second will a string eight feet long vibrate if under the same general other conditions? Ans. 10. 46. What is the instrument called which is used with a prism to measure the wave length of light? Ans. Spectroscope (spectrometer). 47. What do you call two antagonistic colors which produce gray when mixed on a color wheel? Ans. Complementary (supplementary). 48. What do you call the force which makes the governor on a steam engine act? Ans. Centrifugal. 49. What is the phenomenon called when one electric current in a wire sets up a current in a neighboring wire? Ans. Induction. Adaptation to School Work 57 50. What do you call it when hydrogen collects on the positive plate of a battery and thus stops the flow of current? Ans. Polarization. 51 . What is the decomposition of water by the electric current called? Ans. Electrolysis. 52. What do you call the galvanometer which is used with an ordinary magnetic compass needle to measure electric current? Ans. Tangent. 53. What kind of electricity is it called when electricity is generated by heating two different metals connected in series? Ans. Thermo (thermo-electricity). 54. What do you call the ratio of the amount of heat required to raise a given weight of metal one degree Centigrade to that required to raise the same weight of water one degree? Ans. Specific-heat. 55. What do you call the amount of heat required to raise the temperature of one pound of water one degree Fahrenheit? Ans. British thermal unit (B. T. U.) 56. What, besides the volume of gas in a container, determines its pressure? Ans. Temperature (heat) (warmth). One may note that not only informational questions, but ques- tions involving the application of principles can be easily adapted to this form of test. In physics the use of formulae as a means of expressing physical laws is very common. One aim of most physics courses is to teach the students to be able to solve these equations by the simple process of substitution in the equation of values deter- mined from experiment, obtaining therefrom one unknown term of the equation. In an examination there is little merit in requiring the solution of equations which will involve decimal numbers. By using data which are in integral numbers and which are multiples of quan- tities which it may be necessary to divide, a test can be quickly made of the pupil's ability to apply the physical principles involved. Take, for instance, the problem, "What Fahrenheit temperature is 20 Centigrade equal to?" In order to transform 20° Centigrade into Fahrenheit temperature, the pupil has merely mentally to perform the multiplication of c x 20 and then add 32°, giving an even quan- tity as the answer. It seems reasonable to suppose that the majority of students who can mentally perform that operation would also be able to perform the operation with the aid of a paper and pencil if 22° rather than 20 were the Centigrade temperature. Whether such is generally the case is not known. The relative value of the two. 58 Trade Tests in Education types of tests is one which can be solved only by the method of cor- relation. When evaluating the relative merits of the two kinds of problem one must also take into account the fact that possibly two or three questions of the trade test form can be given in the same time as one of the other form. The problem in the last analysis is then "Will n questions of the trade-test form, in the same amount of examination time, give us more or less knowledge of a pupil's ability in physics, than will -^ questions requiring extended arithmetical operations to be performed with the necessary aid of paper and pencil?" Of course the use of paper and pencil in computing 9 x 20 should be allowed if the pupil needs them. It is also a question whether this mental type of computation should not be taught in physics. In many of our courses in science and mathematics it is possible that a great many problems which can be solved mentally or with little effort would result in clearer insight into the principles and the uses of the formulae involved, than would a few lengthy prob- lems involving complicated arithmetical operations. Possibly this point is one of the main differences between ordinary school teach- ing and the training in the trade which a boy gets in industry itself. In industry a boy learns to do fractions by using a chart which con- verts the fractions into decimals. The commercial statistician, rec- ognizing the fallibility of "long-hand" methods, uses a calculating machine. Another point to be noticed in such examinations is the fact that in numerous instances the formula itself can be given. This again is the practice followed in industry. Undoubtedly the most valuable commercial statistician is the one who can at a moment's notice write correctly any formula which he needs. The expert statistician, however, whenever complicated formulae are involved, resorts, as does the machinist in the machine shop, to his published handbooks of formulae, tables, and the like. A commercial physicist who could remember all his formulae would undoubtedly be able to do a larger amount of work in a day than one who had continually to resort to books, especially if the formulae were to be found in numerous books rather than in one hand-book. Such attempts to justify imposing routine upon a pupil in the study of any science are unworthy attempts at an outworn theory of discipline. They belong to the old puritanic philosophy of "discipline for discipline's sake, work for Adaptation to School Work 59 work's sake," and do not fit in with the modern philosophy of letting a machine do the work and reserving the more costly human machine for work which requires planning and exec- utive ability. One of the great barriers to securing large amounts of data on school practice, as a basis for revising school administration and school procedure, is the large amount of time required to give, score, and evaluate the tests. The trade test form of question, if it can be effectively applied to school subjects, has much to be said in its favor on this score. The above physics test clearly indicates the possibility of quickly examining a student on a very great many elements of the course pursued. Testing on many elements is a much more reliable method of securing an accurate evaluation of the indi- vidual's ability than testing on only a very limited number, as is done where a composition is written in response to each question of the examination. The value of the one-word-answer method in edu- cational work remains of course to be proved. On the point of time taken to give the test and to score the results, we may state at the outset that a very great advantage can be secured, since the tests can be quickly scored by stencil. The use of the underscoring, or multiple choice method, of giving an examination may be considered on the basis of ease of scoring. This method is even easier to score than the one-word-answer method. The value of a test, however, should not be sacrificed merely for the ease of scoring. Using the underscore method, a pupil cannot make use of the independent thought which he must use when answering the recall or trade test form of question. Many more items of information can be touched upon by using the under- score method than can be tested in the same time with the one-word- answer form of question. Certain types of information may be easier to present with the one type of examination than with the other. Another point of practical consideration is that the one-word- answer form of question cannot be used very often in the recitation itself. It is difficult to form good trade test questions merely on the spur of the moment, and the same difficulty as is encountered in the trades applies here also. If only seventy-five good trade questions can ordinarily be discovered in a trade, then certainly not that num- ber will be found in the ordinary amount of subject matter covered by a class in physics in a single recitation. It is possible to multiply 6o Trade Tests in Education examples of trade test applications of formulae, for instance, and so occasionally to obviate this difficulty. Care should be take in using the trade test form of question that the teaching be not made to conform to the arbitrary standard set up by the trade test. Teaching would become rather artificial if the teacher always stressed the child's giving the short answer form of replies which the trade test demands. Teaching would become life- less if the aim of the teacher were directed towards merely preparing the pupils to answer trade test questions. This applies, however, with equal force to any measure of an educational product; the teaching always will become very lifeless if the teaching is directed towards preparing pupils merely to pass tests. There are some obvious exceptions, however; in the case of arithmetic or any such mathematics as the formulae of physics, the educational aim should certainly be to enable the pupil to use such fundamental operations, to solve equations with ease; in other words, to use his acquired skill in exactly the same manner as is required in the trade test question. The one-word-answer form of question can be used to stimulate the pupil to definiteness of expression. Both the composition type of examination question and the trade test form should be used, per- haps alternating the one with the other. The former encourages clarity of expression in composition; the latter, definiteness. The latter type of ability is perhaps demanded in industry to a greater extent than the former. The leading question, a type of question which indefinitely suggests the line of approach of the student, will continue to be used by the teacher as a means to stimulate the child's curiosity and lead him on to rely on his own ability. The picture trade test method, essentially an oral method, is probably sugges- tive of some possible school examination methods such as have hith- erto not been attempted. The picture lathe hand test shown on pages 8-14, if made easier than it is now, could be adapted to written examinations in that subject in trade schools. The picture form of examination will also readily adapt itself to the teaching of school subjects. Among the specific criticisms of the use of the one-word-answer, the picture, or the multiple choice tests in examining in school trade subjects is the often recurring one that such tests are tests of infor- mation mainly and do not require independent thought and organ- ization of subject matter. It is true that, in any trade examination Adaptation to School Work 61 made up on these methods, a casual inspection will reveal a seeming preponderance of "informational" questions. This objection is mini- mized by the consideration that the importance of "thinking" in doing many of the routine tasks of industry has probably been over- estimated. The most successful army oral questions were found to be the questions which seemingly best sampled the everyday habits and knowledge of the tradesman. In present-day industry, the workman's task and the directions for performing it are worked out for him in advance ; he relies largely on charts and mechanical devices for such "figuring" as is required; moreover, methods of scientific management and specialization in industry are continually breaking up trades into series of jobs, each of which is so simple as to be per- formed adequately by persons of only fair intellectual or mechanical ability. Finally, the whole question of whether or not many informational questions are desirable is one which will ultimately be settled only by the correlation resulting from experiments in which both "informa- tional" and "thought provoking tests" are tried with the same people. It must be remembered that thinking takes much time, while writing immediately recalled information takes much less time per question. The question then is, "For a given amount of examination time, will X thought provoking questions or (X + Y ) informational questions give higher correlations with demonstrated ability in performance on the job?" These methods are very apparently of limited value in the recitation, upon which should be thrown the chief burden of teaching pupils to think clearly. Psychologically, there is no reason why the results of a com- plicated thought process cannot be indicated by a simple one-word- answer response on the part of the subject. The mental arithmetic "reasoning" problems, to be found in great abundance in any ele- mentary arithmetic text, generally fulfill the major requirements of a good one-word-answer question. Few people would maintain that these are not thought provoking. It merely requires more skill on the part of the question maker to write "thought provoking" ques- tions than to write "informational" questions. All these methods are subject to the danger of preparation by coaching. The one-word-answer form of test is probably more sub- ject to this difficulty than the picture form. Certain forms of psy- chological tests are equally subject to this danger. The obvious 62 Trade Tests in Education solution to the difficulty lies in having so many questions of each of the different forms that alternative tests may be provided. The tradesman in industry has seldom had an incentive for coach- ing up on any form of trade examination ; in trade school procedure, examinations are expected as a matter of course. In the trade school the same material may readily be, formulated in time into four forms of test, the recall one-word-answer, recognition or multiple choice, true-false, and picture, in order to overcome undue influence of "coaching for examination." The coachability of a test is probably in inverse proportion to the number of thought provoking questions included, obeying thus the same law as applies in general to all ex- amination methods. Finally we may dismiss with but little discussion the possible objection that the acquisition of trade knowledge is not the aim of the vocational school. The present avowed aim is the preparation of youth to earn a livelihood in the vocations. We have before indi- cated the growing need in industry of the workman possessed of a goodly number of specific habits, skills and knowledges, and the growing tendency to simplify trades to such an extent that less in- telligence and less "mechanical ability" is required for success on the job by the majority of workers. CHAPTER III TESTS OF TRADE CAPACITY OR PROMISE The possibility of predicting a person's probable success at a job depends upon a relatively fixed condition of the job, a knowledge of the requirements of the job in the way of qualifications of its workers, and methods of measuring human qualifications and evaluating these in terms of degrees of success. Historically there have been three stages in the methods of attacking the problem, each of which has been characterized by developments in the methods of measur- ing the job on the one hand, and the worker on the other. In the first stage, which we may call the "mental type" stage, it was believed that success at a given job was possible only for per- sons of a given mental type. Efforts were accordingly directed towards determining the mental types best suited to given jobs, and toward determining to what mental type a given person belonged. This stage gave way to a second stage in which was attempted an analysis of the lives of successful and unsuccessful men, in order to discover what psychological or physiological trait or traits might be responsible for their success or lack of success. These methods were largely a failure. Statistically, the difficulty with them is that any one such single trait is very likely to have only a very low correla- tion with job success. On the side of measuring the job, efforts were made to classify jobs according to the predominant traits of char- acter and mental ability necessary for success on the job. This second stage then merged into the third, in which, not single traits, but combinations of traits, are considered responsible for the degree of success on the job. The development of tests of general intelligence marks the turning point in this stage. The general intelligence tests were found to be good predicters of general school success of school children, and to be of value in detecting children of abnormal mentality — sub-normal or super-normal ability to profit by the instruction received. From this it was but a step to infer that general intelligence scores would be found very significant in placing persons properly in industry. We are at present in this third stage but rapidly entering a fourth, in which the emphasis will be placed upon the development of statistical methods for the 64 Trade Tests in Education evaluating into one composite score of fitness for the job all va- rieties of facts about an individual: his intelligence, parentage, physical capacity, formal education, informal trade training in mechanical manipulation of his environment and so on. Evidence has been accumulating rapidly in the last few years to point to the fact that individual traits may singly correlate but slightly with job success, but that a composite of such traits may correlate rather highly with job success. The method of partial correlation enables one to make the best possible mathematical prediction (assuming linearity of regression lines) of job success from a number of vari- able measures or tests. The statistical requirements for a composite test of high predictive value are that the correlations of the tests with the criterion of success be high in relation to the intercorrela- tions among the tests themselves. Furthermore, there is coming to be stressed the fact that the reported correlation of the composite test score with the independent measure of job success can only be interpreted in the light of the range of ability covered by the test subjects; or, concretely, that a correlation of .60, where only eighth grade children are involved, indicates as good a composite test as would be shown by a much higher correlation if children from all the grades were included. On the side of measuring the job, there has arisen, just at the transition point from the third to the fourth stage of the develop- ment of human measurement, the method of job analysis. In its present best developed stage, the job is described in terms of con- crete job processes or equally concrete and measurable human quali- fications. Already we have heard hints of the possibilities of develop- ment of methods of measuring jobs by a scale of composite com- parable measures for all jobs, similar to the scales already being developed for human measurement. Fitting the worker to the job is only half of the larger social problem involved in vocational guid- ance. In many cases, jobs should be modified to suit workers. Such modification will tend to come about automatically as soon as the job analysis has caught up, in its process of development, with human measurement and entered likewise into the fourth stage. The names of many experimenters are connected with the develop- ment of present methods of predicting school success by the use of composite scales. The application of like methods in predicting industrial success has followed the former, and hence has fewer Tests of Trade Capacity or Promise 65 names associated with it. Mention might be made of the names of Thorndike, Scott, Thurstone, Link, Burt, Marcus, and Otis. Some six years ago Dr. Thorndike devised and put into use a series of intelligence tests for the selection of clerical workers for the Metropolitan Life Insurance Company. Among the tests were tests not only of general intelligence but of abilities similar to those employed by clerical workers in their work. The examination, con- structed in alternative forms in order as far as possible to overcome coaching difficulties, is reported to be giving successful results after six years of trial, and yielding correlations with job success of probably over .65. Dr. Scott has devised for a number of firms composite scales for the selection of salesmen and clerical workers. His tests have often included the use of some form of group judgment or rating scale of important character qualities presumably not tested by the psy- chological tests. Dr. Thurstone has developed a widely used test for predicting clerical ability. During the war, his researches into the qualifications required for telegraphers involved physiological or psycho-physical tests as well as psychological tests. Dr. Link 1 reports many correlations sufficiently high to justify the belief that composite scales of various tests, selected on a "trial and error" basis, may be used to predict the chance of success in many industrial jobs, chiefly of a repetitive motor or manual type. Dr. Burt 2 reports on the success of weighting tests by the method of partial correlation, using different weightings of the same tests for different jobs, by means of which the probable degree of success of a given worker in the rubber industry may be predicted for a number of jobs. Essentially the same technique was developed by Dr. Thorndike during the war, for rating members of the S. A. T. C. for different branches of the army. Mr. Marcus 3 has shown that a composite of psychological tests may be more efficient in predicting the probable success of certain 1 Link, H. C, Employment Psychology, 1919. 2 Burt, H. E., "Employment Psychology in the Rubber Industry," Journal of Applied Psychology, Vol. 4, No. 1, pp. 1-7. 3 Marcus, L., "Vocational Selection for Specialized Tasks," Journal of Applied Psychology, Vol 4, Nos. 2 and 3, pp. 186-201. 66 Trade Tests in Education manual clerical workers than the ordinary civil service examinations given these workers. Dr. Otis 4 has devised a clerical test which he reports as being val- uable in predicting success of clerical workers in a large silk mill. He also reports "zero" correlations between a composite score of "intel- ligence" tests and performance on the job of manual workers in the same silk mill. He thus concludes that intelligence is not an impor- tant consideration in success in such jobs. A number of attenuating factors enter into his results. With all the above developments in the methods of measuring human fitness for jobs, more time should be devoted in the future to methods of measuring jobs. The lines of development in job analysis will be towards : i. Securing greater definitness of descriptions, as measures of the job which can be verified by other competent persons. 2. Determining methods of comparing the relative merits of one job with another, with the practical point of view of eliminating the discrepancies in wages and working conditions of jobs which require approximately the same composite of talents in their workers. 3. Rating jobs on scales of physiological, psychological, and socio- logical qualifications of workers. 4. Providing for a series of gradations of preferences for jobs by workers on the grounds of monotony, cleanliness, and all those job conditions which affect the emotional life of the worker. 5. Providing for formal methods of training talent for the job among the workers, and of rewarding periodically and automatically any acquisition of better qualifications for the job by job promotion, either in wages, responsibility or social recognition. JOB ANALYSIS The term "job analysis" is used rather loosely. By it is meant, as we have more or less implied above, the entire system of industrial and personnel research, sometimes referred to as a "labor audit." In its narrowest sense, job analysis, made primarily for the purpose of hiring and training, transferring or promoting employees, is the analytical process of acquiring industrial facts. This is to be care- fully distinguished from the products of the analytical process, such 4 Otis, A. S., "The Selection of Mill Workers with Mental Tests," Journal of Applied Psychology, Vol 4, No. 4, pp. 339-341- Tests of Trade Capacity or Promise 67 as job specifications and personnel specifications, which are written reports of the work done by such analytical methods. 1 Job specifications are formulated groups of facts which relate to the nature and conditions of the work, the duties of the worker and certain conditions of service, such as pay, hours and promotions. The personnel specifications present the same type of facts for the worker, for example, his qualifications, allowable handicaps, and required training. Only as an abstract logical analysis are the per- sonnel specifications to be thought of independently of the job specifications. Both must be developed at the same time; both are logically used together in an attempt to fit the job and the worker to each other. Every job description should contain at least a minimum amount of description of the various operations, tools, problems and the like, expected of the worker in the job. In addition, a condensed personnel classification should be included if there are any special requirements on the part of the worker which he himself can easily and objectively measure to see if he fits up with the demands of the job. Such, for instance, is the amount of education required. As a part of the job specifications, whether or not included in the final typewritten or printed job specifications, or whether given on an independent new worker initiation sheet, there should be included all the information which the new applicant will want to have about the job outside of the duties of the job itself: the minimum and maximum wage to be expected, the piece work rates, the average length of time required by workers of average intelligence to learn the job, the average length of time spent at the job before promo- tion, the working hours, statements in regard to holidays and vaca- tions, and social, educational, industrial or other privileges which may be granted the worker. There should be collected other items which have a bearing upon the applicant's contentedness while on the job, such, for instance, as whether or not the worker is required to sit or stand most of the time on the job, whether the work is dusty, hot, damp or otherwise disagreeable, by what type of men the job is manned, referring to such things as nationality and color. 1 An excellent summary of all job analysis work to date is given in Meine, F., Job Specifications, Federal Board for Vocational Education, Employment Management Series, No. 3, Bulletin 45, 1910. See also Tead, Ordway, The Labor Audit, Employ- ment Management Series No. 8, Bulletin 43, 1920. 68 Trade Tests in Education In the personnel specifications should be included the degree of intelligence and the amount of education, experience or technical knowledge required on the part of the applicant, and factors regard- ing the work itself, translated into human qualities, such as the height, the amount of strength, the quality of sight and hearing, and other measurable physical characteristics that may be required on the part of the applicant. The utmost care should be taken to see that whatever goes into the job and personnel specifications, should consist of items, which can be measured with some reasonable degree of accuracy by the people who will have to use them. Too frequently have there been long, wordy descriptions of the job as being one which required "speed, endurance, industry, skillful manipulation, etc., etc." The analysis can thus be worded in such skillful manner that even a sophisticated job analysis expert will be misled by the fluency of the phrases. The principle always to be followed is simple: Whenever tempted to use a term like "skillful manipulation," the job analyst should always ask himself, "How much skillful manipulation is necessary"; and "Can the person who is to use the job specifications in any way measure this same trait." The following questionnaire, only a beginning in this field rather than a completed product, will suggest some of the minimum items of information which it may be worth while to collect in the machine- like trades. This questionnaire is designed for a preliminary general job analysis with the expectation of later following it with special- ized surveys of sanitation, training and psychological investigation. Note that as many of the items as possible are in the form of specific answers to specific questions for underlining and that space is always left for necessary unforseen classifications. Such items as have no claim to objectivity may be generally recognized by having lines in place of descriptive answers to be underscored. Notice also the emphasis upon learning just where and how all available records of the job and workers are kept. GENERAL JOB ANALYSIS QUESTIONNAIRE SHEET Present Name of Trade Symbol Code General Specific Number Reported by Date Dept Foreman Tests of Trade Capacity or Promise 69 I. THE JOB. A. Analysis of the Job in the scheme of: 1 . Classification of jobs : 2. Products produced by worker: 3. Machine (Pictures of all types of machine) Types are: 4. Number of workers required per production unit: Unit is: 5. Number of workers hired in this occupation in this department on this date: B. Similar Occupations: C. Substitute Occupations: D. Physical Conditions of the Job: 1 . Place where work is done: 2. Posture: 3. Activity: MACHINE BENCH . . . DESK FLOOR STANDING CLIMBING PULLING SITTING WALKING SHOVING . STOOPING LIFTING WALKING LIFTING SHOVING CLIMBING FINGER-MOVEMENT WRIST-MOVEMENT PULLING KICKING 4. Physical Condi- WET DRY DUSTY tions: OILY Fumes of VAPOR. LIVE-STEAM HOT COLD NOISY DIRTY STICKY CLAMMY ELECTRICAL 5. Lighting: Source Location Intensity Glare Shadows. 6. Cutting or Lubri- What? When used? eating fluids : 7. Feeds: MACHINE-FEED HAND-FEED SEMI- AUTOMATIC a. Is speed independent of the operator? YES NO 6. What is mechanical maximum? (Explain) jo i Trade Tests in Education 8. Rythm: REPETITIVE NON-REPETITIVE o. Is raw material uniform? YES NO (Explain) . b. Is each operation uniform? YES NO (Explain) . 9. Monotony:. 10. Motion Study (chronological sequence of operations. Describe briefly) 11. Fatigue: a. Degree Type 6. No. of cycle repetitions per HOUR . • c. Size of unit of production d. Amount of lifting done Weight of piece handled. e. Estimated number of footpounds of work done per day. 12. Members used: a. BOTH HANDS LEFT HAND RIGHT HAND b. BOTH LEGS LEFT LEG RIGHT LEG E. Miscellaneous Conditions of Job: 1. Does worker have to supply any tools, clothing, etc? YES NO Specify: 2. Chairs, foot rests, back rests, arm rests, coat racks, eating places: 3. Impediments of hair, clothing, rings, jewelry, etc. : 4. Vacations: 5. Holidays: FOUR REGULAR 6. Night work: Amount Rate Hours worked . 7. Overtime: Amount Rate : 8. Sunday work: 9. Rest periods: 10. Lunch interval: a. What do workers do during lunch interval? 11. Deductions, deposits, fines: 12. Tardiness: Amt Records of, (where and how kept?) . 13. Absences: Amt Records of, (where and how kept?). Tests of Trade Capacity or Promise 71 14. Are workers allowed to talk to each other during work? (describe): 15. Do mechanical factors of job preclude possibility of same production with shorter hours? Foreman's opinion Interviewer's opinion F. Accident and Health Hazards: 1. Accident risk (nature and causes) : 2. Percentage of accidents to workers: (Foreman's opinion). 3. Safety devices and use: 4. Attention to job required to prevent accident: 5. Liability to accident through carelessness of others: 6. Health hazards (nature and causes) : 7. Occupational diseases: Inspection : 1. How often is product inspected? By whom? 2. Type of inspection: LIMIT-GAGE MICROMETER 3. Responsibility for inspection : 4. Are finished products turned back on account of inspection? (Describe) 5. How often are machine and tools inspected? By whom? 6. Spoiled work: Amount Cost Cause: HUMAN MACHINE 7. Are records of spoiled work kept? YES NO (Describe) . . . 8. Can spoiled work be traced to the operator producing it? YES NO (Describe) H. Production, Turnover, Security of Job: 1. Regularity of work: seasonal lay-offs 2. What are grounds for discharge of worker? 3. Production: a. Average production of workers on job: (Foreman's estimate) b. Minimum: Maximum: 4. Foreman's opinion as to whether greater production possible and desirable: 72 Trade Tests in Education 5. Production asked by Planning Dept. : 6. Interchangeability of tools and machines, idle machines: 7. Rush orders (nature) : 8. a. Is worker idle while setting up machinery, tool setting, in- spection, repairs? (Describe) b. Does he rest in such intervals? (Describe) 9. Labor turnover on this job (Foreman's estimate) : a. Average length of employment of worker on this job. (Fore- man's estimate) b. If men and women are both employed, which stay longer? (Foreman's estimate) 10. a. Is production adaptable to count YES NO (Describe) .... 6. Are production records kept? YES NO Where? How accurate? 11. Average length of training for average worker to become profi- cient on job 12. Cost of labor turnover: I. Labor Supply: 1. Any apprentices (understudies) being trained? 2. Deficiencies in qualifications of workers (Foreman): 3. Will supply meet demand in future? (Foreman) YES NO (Describe) 4. Source of supply: 5. Transfers : Amount Per cent formed of turnover 6. How are workers selected (function of foreman) II. THE WORKER: J. Description of Duties: General duties (Describe briefly): Tests of Trade Capacity or Promise 73 Special duties (if sub-group of larger group, e.g., dial press tool setter vs. blanking gang press tool setter) : Miscellaneous: 1. Set tools: 2. Oiling: 3. Belts: 4. Brakes: 5. Sharpen tools:. 6. Repairing:. . . . 7. Inspecting: . . . 8. Teaching:. . . . 9. Safety: 10. Responsibility: K. Qualifications : Previous trade training required: 1. Blue prints: Drawings: 2. Order tools: 3. Measuring tools: 4. Names of tools, machines, parts: 5. Knowledge of materials (qualities and quantities):. . . . 6. Teaching: 7. Inspection : Requirements of schooling, intelligence, training on the job. 1. Is the job: SKILLED UNSKILLED 2. SCHOOLING: 3. Technical training: 4. Deficiencies in TRAINING of workers (Interviewer):. 5. Necessary previous experience in same or allied job: What job? 6. Average length of time for average green hand to come up to average piece rate earnings : 7. Are motions, methods, feeds standardized? 8. Qualifications for a worker to be trained (reasons): 9. Intelligence: 10. Training on job: amount By whom given? . 74 Trade Tests in Education 1 1. How introduced into new job? 12. Is new worker followed up in ten days? YES NO 13. Any written or oral instructions to new employee? YES NO By whom? What? 14. Required ability in mathematics: COPY FIGURES, ADD, SUB- TRACT, MULTIPLY, DIVIDE, DECIMALS, SQUARE ROOT, SOLVE FORMULAE, CALCULUS. 15. Nationality preferred (foreman's reasons) : 16. Literacy: GIVE-ORDERS READ ORDERS UNDERSTAND-VERBAL-ORDERS Physical: 1. Eyesight: acuity glasses an objection color 2. Hearing: acuity 3. Sex: M F (Reasons) 4. Height: : 5. Weight: 6. Arms: length strength 7. Fingers: length dexterity size 8. Fingernails: toughness Wear fingerstalls? YES NO 9. General strength: 10. Strength of back: Lifting 11. Endurance: 12. Agility in moving about : : . . . 13. Sensitiveness: touch pressure weight 14. Attention required: 15. Personal appearance: 16. Physical defects disqualifying: 17. Physical defects allowable: 18. Can women do the job? YES NO Ever tried? YES NO Foreman's opinion of results: 19. Can cripple, aged, or physical defective do the job? YES NO Remarks: 20. Can mental defective do the job? YES NO Remarks: 21. Can left-handed person do the job? YES NO Remarks: L. Wages and Promotion: 1. Line of promotion: 2. Do workers know line of promotion? 3. Do promotions come unasked for? Tests of Trade Capacity or Promise 75 4. Any definite plan of promotion? 5. Wages : Basis of Pay Lowest Highest Average: Day rate Piece rate Contingent rate 6. Nature of contingent rate : a. On day rate : b. On piece rate standard: c. On piece rate earnings : 7. Average daily earnings: 8. Average weekly earnings: 9. Average yearly earnings: 10. Number of all on this job receiving piece rates: 11. How long before green hand put on piece rates? 12. Stereotyped wages in production: 13. Can operator trained on one type of machine operate another 14. Does this job prepare operator for higher job? 15. Are wages dependent upon: WORKER'S-OWN-ABILITY GROUP-SPIRIT It will be noted that much information is of the purely subjective type, as, for instance, "attention required, degree of fatigue, etc., etc." At the present time but little better can be done than these subjective judgments which approach dangerously close to the type of character analysis which we have criticized so severely. It is desir- able to find objective methods for expressing these items of a job. It would seem possible to devise a very simple objective rating scale for monotony, for instance: Degree Definition 1 as monotonous as a dial press job. 2 as monotonous as a production lathe job. 3 as monotonous as a criminal lawyer's job. 76 Trade Tests in Education Such a scale with intermediate degrees, carefully and thought- 1 fully worked out, would yield results by means of which various jobs might be semi-objectively compared for monotony. Outside of being valuable in turnover investigations, such information is of little or no value in hiring, since no one is competent and no test method is available which will predict with surety in advance whether an appli- cant will work contentedly or not at monotonous work. If, however, an applicant should report to the interviewing examiner that he had worked steadily for X years at a job Y, which the examiner knows is a job with degree of monotony i}4, then he would have some assurance that the applicant would be satisfied on a similar job Z with the same degree of monotony. Men do not fall into two clearly defined groups, the honest or the dishonest, but are of many degrees of honesty ; jobs likewise are not monotonous or varied, but of all degrees of monotony. Furthermore, the job analyst must not fail to make the subtle distinction that a job, monotonous to himself to the extent T, may have a lesser monotony to the worker who works at that job as represented by the expression, T — K. There is unlimited room for improvement in the methods of measuring jobs. This statement does not imply that there is not also room for improvement of tests used in measuring human capaci- ties. One line of development, as yet but little used, is that of meas- uring by standardized tests a worker's performance on the job. In the training school of industry this aspect of human measurement would take the line of measurement of product turned out after a given amount of trade instruction on standard test jobs of produc- tion; in the trade school, it will mean performance tests, of a stand- ardized nature, given to the students in trade courses to measure their ability in the course taken; while in the vocational guidance bureau, it will mean subjecting boys and girls to trade instruction with the sole purpose of testing their ability to learn the trade proc- esses taught. It has been the privilege of the author to become acquainted with an educational experiment which combines both the latter purposes in a "trade extension school for vocational gui- dance." EVALUATION OF VOCATIONAL GUIDANCE TESTS At the Manhattan Trade School for Girls, New York City, there is being conducted one of the most interesting experiments to be found anywhere in this country in extension education and vocational Tests of Trade Capacity or Promise 77 guidance by means of tests. The uniqueness of the plan, the numbers of pupils given advice, and the fact that tests are being used actually in basing action upon them, rather than as mere interesting but unused researches, entitles this extension department to an enviable position among the foremost vocational guidance programs to be found to-day. Every second week throughout the school year, a new group of from 1 00 to 120 eighth grade public school girls from various public schools of New York City come to this extension department for a two-weeks course. The aim of the department is both to in- struct the pupils on various vocational opportunities and to test by means of performance tests their ability to learn a few funda- mental things about a limited number of these trades or vocations. The teachers of this extension department make occasional visits to the industries, working on follow-up reports of girls placed in industry by the school, and thus become acquainted with the facts of the various industries touched upon in the extension school course. Thus information becomes available in regard to average wages, range of wages, trade opportunities, active and dull seasons sani- tary and other working conditions of the various trades. These facts are imparted to the girls by the respective commercial and indus- trial teachers. The information given on commercial work covers information on the many subdivisions of commercial work; for in- stance, the positions of typist, stenographer, secretary, telephone operator, computing machine operator, filing clerk, bookkeeper, etc. In addition to this work, educative and highly desirable as a part of the vocational guidance program, the larger part of the two weeks time is given over to the performance tests. The general plan of the tests is first to give oral instructions in the method of doing a funda- mental operation, and then to judge subjectively the excellence of the objective product produced. The pupils are rated by the teachers at approximately the end of every hour's work during the fifty hours spent in this extension school. In some cases it is possible to secure independent ratings of two different teachers of the same industrial subject, hand sewing, for instance. This makes for more reliable ratings when the double ratings are possible. The great amount of rating required neces- sitates that the teacher become acquainted as rapidly as possible with the pupils. What might to the outside observer appear as an exces- sive amount of rating is therefore very valuable in insuring that 78 Trade Tests in Education judgments will be made in as few cases as possible without knowing pupils. The large amount of rating required means that the time that could be devoted to objective scoring of products is very limited ; in fact, to the extent that at the present time no objective scoring is done. Objective scoring, using scales, tests scored by stencil or otherwise, could be adopted only by devoting to such work time now spent in subjective scoring. As a final record of each pupil, a qualification card is made out in duplicate. One copy of this card is retained in the files of the exten- sion school for possible later reference or follow-up work; the other is sent to the principal of the school from which the pupils came, to to be used by him in supplementing his own guidance given to grad- uating pupils. An evaluation of the items of this qualification card is here undertaken in an effort to determine the value of the several tests, both theoretically and practically, with the hope of suggesting improvements in the present tests or test technique, with the prac- tical point in view of securing (a) better predictions of after-school sucess in the vocations undertaken, and (b) greater ease and objec- tiveness of scoring the various tests in reporting the results. The scores on the qualification card are not the scores received on the several tests ; the hand sewing scores of the qualification cards, for instance, represent the results of the first subjective summation of scores in the four abilities of hand sewing, "accuracy, speed, neat- ness, handling," in making a simple hem handkerchief and a child's cap ; the pasting scores of the qualification card represent the first subjective summation of a series of scores of the four abilities of pasting "accuracy, speed, neatness, handling" in doing sample mounting of colored cloth triangles upon a cardboard backing and making a paper covered pasteboard box with cloth reinforced corners. The test scores of qualification cards are subdivided into com- mercial tests, ten in number, and industrial tests, nine in number. To evaluate these, a celluloid stencil was constructed, on which each test and each score or rating on a given test was arbitrarily given a numerical value. It is only by assigning numerical values to "high average" position of check mark on a line opposite the English test, that such scores can be effectively dealt with statistically. The commercial tests scores were all reported as "good, fair," and "poor," finer differentiations of ability being secured by significance Tests of Trade Capacity or Promise 79 being attached to the position of the check mark, indicating the score in the test, upon a ruled line approximately 2.6 inches in length. On the celluloid scoring stencil, this length was subdivided into nine equal parts, numbered from 1 to 9, in increasing order of merit of score obtained. The position of the check mark as falling in one of these nine vertical columns of the stencil is the score in each of the commercial tests, the variables being numbered 5-14, inclusive. These check mark estimates were subjectively evaluated and a com- mercial "stamp," or recommendation of the director, placed on the card as variable No. 15. The various stamps and their evaluation, as well as the similar industrial "stamps," are explained below. The industrial tests, in their various subdivisions, had their scores reported as "good, fair'' and "poor,'' arbitrarily given numerical values respectively of 3, 2, and 1, in variables numbered 16 to 34, inclusive. The several tests, briefly described, are as follows: I. Commercial Tests. Variable 5. English. Three compositions and a final test are required in the course: A. Composition: Skilled and Unskilled Labor. B. Composition: Vocations for Women. C. Composition: Effect of the Introduction of Machinery. D. Final test. a. Dictation : Discourse between two people. b. Composition: What These Courses in the Extension Classes have Meant to Me. Time: Lectures, preliminary to compositions, 30 minutes each. Compositions, 30 minutes each. Final test, 60 minutes. Variable 6. Arithmetic. Single problems in arithmetical operations. Variable 7. Penmanship. Based on penmanship on written work of compositions. Variable 8. Switchboard. A series of demonstration performance lessons on the use of a private branch exchange switchboard, the various pupils taking turns in being, "private branch operator," "lady in a residence," "desk clerk," etc., being rated by the teacher in each capacity in turn. Variable 9. Filing. A series of three lessons with tests on each. 80 Trade Tests in Education A. Alphabetical filing. Test of arranging 50 cards in alphabetical order and copying the order on paper. B. Geographical filing. Test on same. C. Telephone Directory and its use. Test of using directory in locating desig- nated names. Variable 10. Stenography. A series of lessons, beginning with the single strokes and leading up to the final construction of simple words and sentences. Ability scored on the basis of reactions to simple dictation involving beginning principles of stenography. Variable 11. Typewriting. A series of five lessons leading up to the final formation of simple letter groups, or syllables, by the touch system. Practice sheets are preserved and sub- jectively scored by the teacher. Variable 12. General Adaptability. A subjective judgment of presumable general intelligence. Variable 13. Personality. A subjective judgment. Variable 14. Appearance. A subjective judgment. Stressing neatness rather than personal beauty. Variable 15. Commercial Estimate, or "stamp." This is the director's subjective summation of all the check marks of the above tests. This is the recommendation of action to the principal of the school from which the pupil comes. A series of rubber stamps are used, an imprint of one of which is affixed to each card. As arbitrarily evaluated, after comparison of meanings of the various ratings of the various teachers by drawing on a ten- inch line the division points of the various meanings, the stamps were given numerical scores ranging in ascending order from 1 to 7. II. Industrial Tests Variable 16. Teacher's subjective evaluation of hand sewing ability, based on the card entries of the four following sewing variables; accuracy, speed, neatness and handling, variables 17, 18, 19, 20. For the 97 persons of the experiment this correlates with the sum of the gross scores of the four variables, of which it is a subjective summation, to the extent of .662. There was a fair spread of the summation scores. Variable 21. Teacher's subjective evaluation of power machine operating, based on the card entries of the four following power machine sewing variables; accuracy, speed, neatness, and handling, variables 22, 23, 24, 25. For the 97 this correlates with the sum of the gross scores of the variables, of which it is a subjective summation, to the extent of .881. Variable 26. Teacher's subjective evaluation of pasting, based on the card entries of the four following pasting variables; accuracy, speed, neatness, handling, variables 27, 28, 29, 30. For the 97 this correlates with the sum of the gross Tests of Trade Capacity or Promise 8 1 scores of the four variables, of which it is a subjective summation, to the extent of .868. Variable 31. Grasp of Directions. A subjective judgment, presumably "ability to imitate" demonstrated per- formance. Variable 32. Persistence. A subjective judgment. Variable 33. Interest. A subjective judgment. Variable 34. Originality. A subjective judgment, "designing"' ability presumably predominating. Variable 35. Sewing estimate, or "stamp." The stamps were given numerical scores in ascending order of merit from o to 4. The power machine and hand sewing evaluations of variables 16 and 21 are here subjectively combined in the one stamp, making impossible sep- arate evaluation of these two items. Variable 36. Pasting estimate or "stamp." The stamps were given numerical scores in ascending order of merit from o to 4. The above descriptions are but meager and inadequate repre- sentations of the tests used and the numbers of ratings made. This research is devoted to the evaluation of the qualification card entries, and not to the original ratings from which these are made. As such, the results reported logically fall into two headings : A. Intercorrelations of various additional tests administered to 97 of one such section of pupils, on whom qualification card records were complete, together with similar intercorrelations of the quali- fication card data and test scores. B. Evaluation of the qualification card test scores against aver- age semester school marks in commercial high school subjects of 30 pupils who had passed through this extension school previously, and were found to be attending commercial high school at the time of the investigation. A . Results of Intercorrelations Based on gy Pupils With the aim of investigating the possibilities of standard objec-' tive tests in conjunction with the subjective performance tests, addi- tional verbal and non-verbal intelligence tests were given to one of the bi-weekly classes of the extension department. There were 97 girls who had complete test records in all tests. 82 Trade Tests in Education In the results which follow, it is to be remembered that the reli- ability of the gross scores is somewhat greater than is ordinarily the case with subjective judgments, since each gross score of the qualifi- cation card is normally the result of a subjective summation of several personal judgments, made in some cases by two or more teachers independently. In order to evaluate the qualification card variables against one another, combinations of the scores of the variables were made by adding the gross scores and computing the intercor relations. These intercorrelations are shown in Table IV. Of the commercial tests, switchboard, stenography and type- writing (8 + 10 + n) combined have the highest correlation with the director's estimate (15), r = .676. Next in value in predicting the estimate (15) are English and arithmetic combined (5 + 6), r = .595; next, general adaptability, personality and appearance combined (12 -f 13 + 14),?" = -575; and finally penmanship and filing (7 + 9), r = .506. In view of the magnitude of these correla- tions, the average of which is .588, it would seem that too much of personal judgment of the director is being injected into the final estimates, over and beyond what the tests themselves show. The weighting of these variables in determining the final "stamp" applied to the card is a subjective matter. The correlations would lead us to believe that much might be gained from using a series of objective weights in determining what is to be the final stamp. The majority of the industrial tests, as combined, correlate nega- tively with the commercial estimate. The average of the nine com- binations of correlations of industrial tests of the table with the com- mercial estimate is — .003. The final commercial stamp (15) corre- lates with sewing estimate to the extent of — .118; and with pasting to the extent of — .008. Taking the tests at their face value, one would conclude that commercial ability and industrial ability in sew- ing and pasting are correlated slightly negatively. Grasp of direc- tions, persistence, interest and originality in industrial work (31 +32 + 33 + 34) combined correlate positively with commercial estimate; r = .306. This suggests that these subjective tests of character qualities re influenced by the same considerations which make for a correlation of .186 with general adaptability, personality and appear- ance (12 + 13 + 14) of the commercial tests, and a positive corre- lation with all other commercial tests. These four subjective judg- Tests of Trade Capacity or Promise 83 ments could well be discarded for a non-verbal intelligence test, for which they are a very inferior substitute. All industrial tests intercorrelate among themselves positively. The average of nine correlations of the table with the final sewing stamp gives r = .483, and with the pasting stamp, r = .516. Combining ''accuracy, speed, neatness and handling'' by adding the scores, gives lower correlations with the final industrial stamp than the subjective evaluation of these items; in the case of sewing evaluation and sewing stamp, .618 vs. .920; sewing estimate and pasting stamp, .543 vs. .647; power machine operating and pasting stamp, .718 vs. .797. In the other cases the combined score corre- lates higher with the final stamps; power machine operating and sewing stamp .474 for the estimate vs. .569 for the combined score; pasting evaluation with sewing stamp, .305 vs. .308, or equal so far as all practical considerations are concerned ; and pasting evaluation with pasting stamp, .374 vs. .377, again equal for all practical pur- poses. Inasmuch as the scores in hand sewing have little dispersion, it seems true that subjective bias has entered into these hand sewing scores, and consequently into their final evaluation and is reflected almost perfectly from the evaluation in the sewing stamp applied. Hand sewing evaluation and sewing stamp correlate to the extent of .920. Statistically, the difficulty lies in not making use in the final stamp of the differentiation already secured in the tests. Judgments of ''accuracy, speed, neatness, handling" are based on more than one judgment and so have some reliability in themselves. The evaluations can go only by full steps ; the summation can add up by partial credits. That is, the evaluation of A's ability must be either 1, 2, or 3; while by summation it may be any score from 4 to 12 in- clusive. The correlations afforded by the evaluations of the stamps are such as to prove the feasibility of using the sum of the four. Alpha (Form 6) and three Thorndike Non-Verbal Intelligence Tests (Forms: I-A, I-L, II-K) were given to the 97 subjects. The intercorrelations are shown in Table V. It is to be noted here, as in the case of all correlations based on these 97 subjects, that while the correlations apparently are low, these correlations are based on a very limited range of talent, the eighth grade of school to which all belonged. Since all are of the same grade, presumably the older pupils are the less intelligent; this is brought out by the four out of five negative intercorrelations. The Toops-Pintner revised direc- 8 4 Trade Tests in Education w 03 < II 3 z H 0) -4-> H c H * w t— t b/1 e a u ioi> to to « . 0> to M n fO ^* - ^" ^ IO o *o sO OE + J5s + 8 5 + is •O N Oi Oi Ot u) u) O Oi O O m o o o 1 1 r> Oi N m co ,mooo> MMlOCtO , w o *"- n io n floo ■? ( ? r ? « 6. CO It- IO M 9S r- oo to m oo n d ifl w o O O O tO «-< ' \ ' \ \ lo O O - 00 fO "1 * ** o o> to . o a o t- to ^ M CO . CO to fO to fO csi to l-l Ss + frs + Ez + zz O -t t- r- m « r- w o ■* O O O in o 1 1 1 1 «J CI H H M (1) OOO OOOO . ro « io vO m n lo co . to to io io o> »o to c* w IS O Tt M H O o o o o o f i' * r r-O m o r* i> ^- o> O to ' oo r» io ^- r- Oi »* ■•* , CO N N V Tf N c* 1" H 00 00 IO os + 6i + 8i + ii Q, m DO H ro O to to i> io M M H H O i r r r r o . to O » oo co to . . Tj- Ifl ■* ( 00«0"*mmoo^- «£5 ^f to fO fO « a o O M o IO hi 'S I o U Si Oi O l> •> • Ch fO O H 00 QivOOOCO sOiOw i «*fOOiO>-«o MOOOworOwO II II II SO IO O to N fri + Ei + si o a n .to I- ■* 00 . t- « ro O -to 00HMI>MO\>OsOp 0> r- •* O lOOiOO OiO MHOnfOOHCMI-t 00 ^« *o IO CO r- 1 1 1 1 1 1 II II + 01 + 8 il - .NO to so . oo r» ^- to ■ <0 moo to o> to Oi to n o ^- ro >o m to O io to O mhOOOOmwo III 1 sO to 6+4 oo . m a «o o . ** o IO . (1) fl U) O>iO-*Tj-00 « (OO l> O to ^ c- co m w. w ih MMOOOHOMW i r i r f r * r r o o a H 9+ S . co n o >n . o to o> a coo»OOo>soi>esi **■ O O h « c<) loco to «* whOOOOOhO 1 1 1 1 II «o to CO IO N CsJ 9jqvtWA O e> + + .O t*_£ + O to , ^- « csi O co + + ?+ a ** ^ io M W Ol IO O_L-H_l_>0 N _i_lOO M~MI^M_1_I^(0(0 CO 1*1 N M « CO CO + + ++ t- ei h h n r- ro > < o » 5 ID 5 b Tests of Trade Capacity or Promise 85 tions test was also given. This test is a poor measure of whatever abilities are measured by the three non-verbal tests. As a short, easily administered, and perfectly objective, scorable-by-stencil test of intelligence, the test has promise; its correlation with Alpha is higher than the average intercorrelations of the three non-verbal tests. Of the three non-verbal tests, Form I-L, which is most diffi- cult of the three forms, has the highest average intercorrelation. This seems to suggest that more difficult non-verbal tests might prove to be of more value than the present ones. The correlations of the two intelligence tests and the combined score in the three non-verbal tests, found by adding together the separate gross scores in the three non-verbal tests, are shown in Table VI. The striking contrast between the value of Alpha and the value of non-verbal tests in predicting respectively the commercial test scores and the industrial test scores is shown by the plus sign which indicates which of the two "intelligence'' tests is the better predicter of the respective test scores. If we say that the Alpha is TABLE V Intercorrelation of Measures of Intelligence All Eighth Grade Pupils. N = 97 Intercorrelation With: Variable A Re Directions N. V.- I-A N. V.- I-L N. V.- II-K Alpha Average with other intelli- gence measures Age —.242 -.038 -OSI —.III —.276 -.123 Directions — .242 — .019 -0-iO ■034 .398 .113 N. V.-I-A -.038 —.019 •473 .287 .227 .242 N. V.-I-L .051 .040 • 473 • 375 ■ 319 .302 N. V.-II-K —.III • 034 .287 •375 .411 ■ 277 Alpha —.276 • 398 .227 • 319 .411 ■339 Variable Average 13-45 15-65 43.02 39-45 45-30 65.80 a .824 3-395 12.081 9.258 9-5 19 14.322 86 Trade Tests in Education o a > « > I !S o W m gfe; g 3 B g S3 o z o S H « O U o}t q CO CO + -mtitfuy 'Sutatag puvfj 1 S9tjtjtq y mog fo uotj *C CO + -vnpag '3utm.9S puvji q « u tfUtV}S n JVWA91UU10J 1 CO, Ok + 9QUV -J,V9$$y x f,%%\VU0SA9^ M to + fyiJtqVjGvp y JOA9U9Q Sui -juaigtffij; puv 6u4v* 0\ o» + -3ou9ts l pM>oqi{oitais 3u tlfd P uv 4fHSUvtuu9j + 9tJ9Ult(fUy puv ustfiug p- ■* + u o Z c o 55 - T3 a) S S o u •a c 2 E o U ft < $ a 3 s H 5.3 B 8 * " C 05 73 •a ** ** to « 2 5 -s •£ a « . o > •" c5 t5 fi ■" at M v to ro rt (U to > >< 3 ? S a oJ O . to to ^j * S «; a M *3 n ■« .2 t» o g o S p > g - M O H = "■; CD Tests of Trade Capacity or Promise 87 "typical" of commercial test scores and the non-verbal test of indus- trial test scores, and if into the subjective evaluation of the four "abilities" in each of the three industrial tests of hand sewing, power machine operating, and pasting there enter largely those personal subjective qualities otherwise designated as "personality," "general adaptability," etc, — then we should expect to find the following: 1. Alpha correlating higher with the subjective evaluation of the four abilities than the objective arithmetical summation of these four abilities. This it does (Table VI) in two cases out of three, the exception being in power machine operating, in which it may be noted, the test product is more easily scored objectively than the hand sewing or pasting products. 2. Non-verbal tests correlating higher with the objective arith- metical summation of the four abilities than with the subjective evaluation. This the combined non-verbal score does in all three cases. The argument thus seems strongly in favor of scoring the indus- trial tests solely on the basis of the objective tests, and not allowing for a subjective judgment "over and beyond" what the objective arithmetical summation of the tests alone would give. B. Follow-up of Students who Entered Commercial High School From the records of the extension school was found a list of 134 students who had signified their intention of entering the commercial course in Washington Irving High School. From this list only thirty- nine records could be located at the high school (29.1 per cent). This fact is mentioned as showing one of the difficulties encountered in checking up the value of any selection methods. Undoubtedly more than thirty-nine students had entered high school, but had dropped out, since there were a few of the persons who were at the time in their fifth semester at the high school. Our records therefore are lacking in one important element, — knowing the test record and high school record of previous failures. From the thirty-nine records, there were available only the thirty records sufficiently complete in all respects to be used in partial correlation. The thirty-nine students had been in school for the following lengths of time : 1 semester, o ; 2 semesters, 2 ; 3 semesters, 4 ; 4 sem- esters, 31 ; 5 semesters, 2; total, 39. The average per semester percentage school marks were com- puted in the following subjects: English; mathematics; drawing; stenography; typing; bookeeping. 88 Trade Tests in Education A few unmentioned electives were disregarded in obtaining the average per semester percentage school mark in all subjects. This average school mark is inaccurate through the rating of different per- sons by different teachers, through changes in the standard of grading from year to year, through unequal lengths of time in school and con- sequent different numbers of semester courses, together with all the errors inherent in a percentage system of grading. It is a rough crite- rion of ability to do commercial high school work, but is the best one may obtain without excessive statistical labor spent on only a few cases. Criteria for evaluating test methods are generally subject to such difficulties. A check on the value of the extension school tests in predicting the probable commercial high school success of these pupils is the corre- lation between their extension school tests and commercial high school grades in the same or similar work. Table VII shows the results. It is seen that the typing and stenography tests, approaches to an objective trade test method, are fair predicters of high school marks in those subjects. Extension department arithmetic tests are fair predicters of bookkeeping marks, but not of the marks in algebra, TABLE VII Table of Correlations of Extension School Tests with Same or Similar School Course Average Per Semester High School Marks Correlation of Number of Cases r = School Mark in Typing with Typewriting Test Score 24 •43° School Mark in English with English Test Score 3° .205 School Mark in Stenography with Stenography Test Score 23 •452 School Mark in Mathematics (Algebra) with Arithmetic Test Score 14 +.135 School Mark in Bookkeeping with Arithmetic Test Score 26 .478 Tests of Trade Capacity or Promise 89 a more abstract mathematical subject. The English test is a poor predicter of high school marks in English. Probably all these corre- lations, seemingly not very high, would be higher if the high school group were not of so restricted arrange. Furthermore, the group studied consisted of only the "survivors" and not the "entrants." Again, the extension school estimate must be one which will predict the average success in a subject for as long period of time as five semesters in high school. The results reported above in the case of the English test, a composition test, which in common with most composition tests probably has a high P. E. of the individual score, suggests the need for a revision of this test. The relationships of the high school commercial course marks to the various "stamps" in commercial tests, sewing tests, pasting tests, are shown in Table VIII. Commercial "stamps" correlate positively to the extent of .403 with commercial high school grades. It is this figure which we hope later to raise by the partial correlation weight- ing of the various commercial tests. High school marks correlate positively with sewing "stamps" to the extent of .242 ; and with past- ing "stamps" to the extent of .063. There is evidently little relation- ship between ability to get along well in a commercial high school, as measured, and ability in sewing or pasting, as measured by the TABLE VIII Intercorrelations of "Stamps" in Commercial, Sewing and Pasting Tests with Average Per Semester High School Marks. N=30 Correlation with Variable: Variable 1 2 3 4 1 1. 000 •403 .242 .063 2 •4°3 1. 000 •346 •336 3 .242 •346 1. 000 •597 4 .063 •336 •597 1. 000 „ 6.320 I-585 .829 •539 Average 67.20 3-47 2.67 3.10 Variable 1 — Average per semester school mark in commercial high school courses. Variable 2 — Estimate "stamp" of commercial ability. Variable 3 — Estimate "stamp" of sewing ability. Variable 4 — Estimate "stamp" of pasting ability. go Trade Tests in Education extension school "stamps." The fact that, where we have objective evidence in regard to the pupils' commercial abilities, the correla- tions of commercial success with the industrial stamps are positive in every case, would lead us to doubt whether the low negative correla- tions of Table IV really indicate a fundamental negative relation- ship between "commercial ability" and "industrial ability," or are only the result of the teachers' possible adherence to a "type" philosophy of tests, — the ever popular belief that a person "preeminently" fitted to be a commercial worker will not be a good industrial worker and vice versa. Such a chance for bias, not possible with objective trade tests, may impose a very real obstacle to the success of the subjective method of judging the results of objective tests. A previous tabulation of the commercial "stamps" and subjective composite of the sewing and pasting "stamps," called "industrial test rating" in the case of 346 successively accumulating qualification cards, yielded a positive correlation between commercial "stamps" and industrial "stamps" of .071. Commercial ability, as measured by these subjective stamp evaluations of commercial tests, is related but slightly to industrial ability similarly determined. The intercorrelations of the various commercial tests, the corre- lations with average per semester school marks, the average scores and standard deviations are shown in Table IX. These tests, excluding the estimate, variable 15, were evaluated by partial correlation yielding the formulae : In terms of deviations : English Arithmetic Penmanship Switchboard ^ = -OI 5 -* +.173 -^ +.I84 -5- +.036 -* \°) "A Of> ff6 ai a ™,. _ , General Filing Stenography Typewriting Adaptability + . II0 ^L+. I2 o^i +.280 *L +. I37 J^L °9 010 (Tn <7l2 Personality Appearance -.008 — -.047 —2- The total correlation of the composite and criterion is, ru = .712. Tests of Trade Capacity or Promise 9i « o «! a. a > < a a < o -1 : c 3 M m H 5 in fl W a 2 c- y W a O B h ° 5 fc 2 en H 2 > H W °3 erf 1 lfl m W N O O O1O1OO f*5 fO < ^i»/Y looeys ^J S9 MNiOMTj-roO>OOwO tJ- ■"* m m T \n m O m t* 1 1 O M C* t*} «0 >> V « 2 1 1- > < Mark English Arithmetic Penmanship Switchboard Filing Stenography Typewriting General Ada Personality Appearance Estimate or* M .2 3 > < Q b < loor^oooiOMNffs-^ui 92 Trade Tests in Education Or, in terms of gross scores : (9) *a = - -040 X t + .508 X 6 + .509 X-, + .102 X 8 + .350 X B + .307 X10 + -943 -Xn + 424 X n - .034 Z13 - .161 X u + 53-238- We are now in a position to state the value of the several tests. We shall do this by variable numbers. Variable 5. English Test. As shown by the very low negative weighting, the test is, for these follow-up cases, worse than useless as now scored. As pre- viously shown, the English test is of almost no value in predicting high school marks in English. A composition test has too high a probable error of the indi- vidual score to be of much value as an English test. A Trabue Completion-Test Language Scale, requiring not more than ten minutes, would probably give much better measure of school English marks, and besides is a good intelligence test. Variable 6. Arithmetic Test. A highly valuable test, exceeded in value only by the penmanship and typewriting tests. Variable 7. Penmanship Test. Exceeded in value only by the typewriting test. Variable 8. Switchboard Test. Of little value in predicting high school marks in a commercial course. The test may prove to be of value as a vocational test for switchboard operators. For predicting high school commercial course success, the time spent on this test could well be devoted to other tests of promise, a good intelligence test, for instance. Variable 9. Filing Test. A desirable test, exceeded in value only by five others. Variable 10. Stenography Test. A desirable test, about on a par with the filing test. Variable 11. Typewriting Test. The most valuable test of the group. The spread of scores (standard deviation) is less than in any of the other objective tests: if a greater spread of scores could be obtained, the test might prove to be an even better test than at present. Variable 12. General Adaptability. This is a subjective judgment test and is about on a par with the filing and stenography tests. The trait probably cor- responds most nearly to general intelligence of any of the personal traits judged. In spite of its being a subjective test, it receives a fair positive weighting, suffi- cient to suggest greater possibilities from an objective intelligence test. The administrative demands upon such an intelligence test are: (1) Short adminis- tration time. (2) Short scoring time. (3) High reliability. Some form of omnibus intelligence test will meet these requirements. If the present English test should be abandoned in favor of a Trabue Language Scale, there would be less need of such an intelligence test. Again, the time now spent on the switchboard test would give more than ample time for such an intelligence test, if the switch- board test were abandoned. Variable 13. Personality. This is another personal judgment test. It receives a negative weighting, which might suggest that "personality" is "bad personality" that; remains after a pupil has been judged for intelligence. The time taken for this useless judgment might well be spent in scoring a good intelligence test. The spread of scores is least of all the objective and subjective tests. Tests of Trade Capacity or Promise 93 Variable 14. Appearance. Although the spread of scores is higher than the pre- vious test, it would seem that "personal appearance" is of no concern in com- mercial high school success! One might, of course, argue that the trait determines to some extent one's success after leaving school and entering business. This seems doubtful, if one may judge from popular opinion, for the homely girl is often preferred in many clerical positions as it is commonly believed that "her chances of being married are less than those of a pretty girl." At any rate, it seems doubtful whether a public school should ever bar anyone from taking a commercial course on such grounds. The results of this investigation are convincing to the effect that objective tests of the type used can be made a very worthwhile part of the educational machinery involved in preparing children for life's struggles. The results are promising enough to justify the army trade test procedure being adopted in the case of one or two hundred high school freshman students whose abilities will become known. It would be but a simple matter to have the same tests, as are now administered to the eighth grade girls, administered to a class of high school students who would attend the extension school for two weeks for the purpose. During this time, an additional number of intelli- gence tests, reading tests and other tests of promise should be tried. By standardization of the tests on large numbers of pupils who will be certain to attend the high school, we will include the "to-be-fail- ures" in our group as well as the "survivors." This procedure con- forms to the procedure of trade tests, adapted to vocational guid- ance. It should be evident that testing students who have already had extended high school training in stenography, typewriting, and the like, may not be expected a priori to yield the desired norms; what is needed is to test a high school freshman class upon entering the high school or soon thereafter. One should also not fail to make the distinction that not all persons in grade school who made a 50 percentile score in the distribution of extension department test scores) will attain that degree of success in high school; rather, if standardized in the manner suggested, "of those people interested enough to enter high school, the tests so standardized will predict such-and-such success, with a given correlation X." Thus, interest, logically kindled through the talks given the extension students by the teachers, may lead the child to make the decision to go to high school, after which the tests will reveal with a rather high degree of accuracy his probable success in the high school course. A correla- tion of .71 between tests and high school success is undoubtedly 94 Trade Tests in Education Table X Original Qualification Card and Follow-up Data of High School Commercial Students ft "J Score in Tests: 6 a 8 a S B O E a ■8 § I 00 a 5 ■ft. ! 55 00 ,a a «. "3 fe a a p ft. a 3 "3 5 w i 72 8 8 8 8 8 8 5 8 8 8 5 2 83 8 8 8 5 5 9 8 8 8 5 5 3 68 5 2 8 2 8 9 2 5 5 5 5 4 58 8 2 2 8 5 2 2 2 8 8 2 5 79 8 8 8 5 8 8 5 8 8 8 6 6 73 8 5 5 8 8 2 5 5 8 8 5 7 65 8 5 5 2 2 5 8 5 8 8 5 8 66 5 2 5 5 8 2 2 5 5 8 2 9 67 2 5 8 8 5 5 5 5 8 2 5 10 72 2 5 8 8 5 5 5 5 S 5 2 II 74 2 5 8 2 8 S 5 8 5 5 1 12 66 8 5 8 5 5 2 2 5 5 5 2 13 70 2 5 8 5 8 5 5 5 8 8 5 14 62 8 5 2 5 5 5 5 5 8 8 5 IS 64 5 2 8 1 8 5 2 2 8 5 2 16 62 5 8 5 5 8 5 2 2 5 5 2 17 58 5 2 5 2 8 2 2 2 5 2 2 18 64 8 2 2 5 8 2 5 5 8 8 3 19 70 5 5 5 2 8 8 5 8 5 5 5 20 65 5 5 5 5 5 8 5 5 8 8 5 21 53 5 5 5 5 5 2 5 5 8 5 2 22 66 2 2 8 2 2 2 5 5 5 5 2 23 76 5 8 5 5 8 2 5 5 S 8 2 24 73 8 2 5 8 5 5 8 2 8 8 5 25 67 2 8 2 2 2 5 2 5 5 5 2 , 26 58 8 5 2 5 8 2 2 2 S 8 I 27 65 8 5 8 8 5 8 5 5 8 8 5 28 65 5 2 5 5 5 8 2 2 8 5 4 29 68 2 2 2 5 8 2 5 2 S 5 2 30 67 8 s 8 2 5 5 5 8 8 8 5 Average 67.20 5.60 4.60 5.70 4-77 6.20 4-77 4-30 4.8O 6.70 6.30 3-47 a 6.319 2.375 2.154 2.282 2.232 1.990 2.474 1.847 2.04O 1.487 1.847 1.586 Tests of Trade Capacity or Promise 95 much better than a consensus j udgment of many interviewers with- out tests would be. Proper additional selection of promising objec- tive tests may better this figure considerably. One should consider the possibility of rating pupils by their suc- cess in industry. This is impossible at present. The most that can be expected of such a vocational guidance department is to make a good prediction of probable high school success. Table X gives, for reference, the original data used in calculating the correlations of Table IX involving the thirty commercial high school students. CHAPTER IV GENERAL CONSIDERATIONS IN TESTING AND TRAIN- ING FOR PROFICIENCY AND PROMISE IN THE TRADES Some of the observations made on the trades, on tradesmen and on trade practices, during the assembly and standardization of ninety army oral trade tests may be of value to anyone who would compare army methods of testing trade proficiency with school methods. The army tests were standardized on men in industry so that the following remarks apply to men in industry and not to soldiers. Vocational schools may not have been "successful" in the past owing to the fact that they were not, in spite of their claims, aiming to train workers of the kind which they claimed to be training. The vocational schools have claimed to be training tradesmen; they really were attempting to train foremen. Industry has been looking to the trade schools for tradesmen, and necessarily has been disap- pointed in some cases. A tradesman's training in industry is intellectually narrow in comparison with the training which the average vocational school would give him. We have already mentioned the specialization which is entering practically all industries and all trades. Trades are being split up into jobs, and jobs into operations, in order that inexperi- enced and unintelligent persons can quickly become competent and productive, and in order to obtain the profit accruing to the increased production made practicable by a person's being allowed to become highly skilled in one simple operation. No longer does the typical tradesman have to do much figuring for himself. He is "sent out on a job," the standard time for doing which has often been figured out in advance. On the job, a foreman directs his work at every step. If his tools get dull, he hands them in many cases to a toolgrinder ; if his belts break, he calls the millwright whose sole business it is to keep the machinery running; if his work is not turning out properly, he may call the toolsetter; and finally, an inspector tests his products. In consequence of this tendency to specialization, in but few of Testing and Training for Proficiency 97 the trades are there many pertinent items about which trade test questions may be formed. Even in the highly skilled mechanical field of the toolmaker, piece-work wages are being given to some workers. Again, we have mentioned the use of charts, tables, and mechan- ical devices used in industry to eliminate arithmetical calculations on the part of the worker. The typical workman is disdainful of studying books. Along with this goes a lack of realization of just how dependent he really is upon the directions of others and upon such book-made aids to trade proficiency. As a result of this atti- tude, perhaps, he reads trade magazines but occasionally. These remarks do not apply to the foreman, who is not the typical, or jour- neyman, tradesman. Having but little responsibility for the job, once his toolchest is locked up, the average journeyman's interest in trade matters largely ceases at five p. m. on week days and at twelve o'clock on Saturdays. But few tradesmen study books on their trades after hours. Wherever, in army standardization of oral trade tests, a boy was found who was pursuing a correspondence course in his trade, he generally made a higher trade test score than other boys of the same length of experience but without the correspondence school course. The general consensus of opinion of foremen in such cases was to the effect that such boys were of more value on the job than boys who did not have the "ambition" to take such courses. Boys who attend night school seem generally superior to boys who do not, but less markedly so than the boys who take correspondence school courses. Possibly one may correctly look upon correspondence school adver- tisements as an excellent sieve for picking out these boys who have "enough ambition to save up X dollars for self-improvement," where- as the night school appeals to a lesser ability, "enough ambition to go to a free school for self-improvement." The average journeyman, and the expert to a lesser degree, is lim- ited in the difficulty of trade questions which he is able to answer. It has been found, for instance, that "technical" questions, of the type which one might propound to a superintendent of a machine shop or to a mechanical engineer, were seldom retained for the final examina- tion set in the army oral questions ; they were usually discarded be- cause there were so few passes on a question as to make it almost worthless in being able to differentiate between different classes of trade ability. As a striking example, we might cite the case of some 98 Trade Tests in Education forty or fifty millwrights, journeymen and experts, to whom three such "difficult" questions were administered. A millwright is in charge of setting up machinery in a factory and keeping the belts, shafts, etc., in constant repair. One of the easiest of the three questions was : "If a drive pulley, 24 inches in diameter, is running at 200 revolutions per minute, how many revolutions per minute will an 8-inch pulley revolve, which is belted to it?" Only one journeyman and five experts were able to answer the question correctly. Many of them would not even attempt it. The question, it must be observed, is an oral ques- tion and some intelligence is needed to keep all the factors in mind at one time ; it is not certain that the same results would be secured from using a blue-print of the same situation and asking the ques- tion with reference to the blue-print. The question was repeated in its entirety, as often as required by the tradesmen, and yet many of them seemed unable to comprehend the situation clearly enough even to attempt to answer it. Oral trade test questions must, for tradesmen in the trade, be limited to "practical shop kinks and everyday knowledge." In few trades are there to be found as many as seventy-five questions, of graded difficulty as is necessary, which will fulfill all the require- ments of standardization questions ; and of these, many will be dis- carded in the standardization process if rating tests are being made. The majority of questions must not be technical questions, and on the other hand must be such questions as will sample the daily habits of the workman. For "difficult" questions one must search for the type of question ordinarily regarded as "trade secrets." These trade secrets, when analyzed, prove in most cases to be merely principles of physics, chemistry, general science, or measurement, such as any high school senior should know. Such, for example, is the "rule" that "to get the circumference of an iron ring you multiply the diameter by 3 1 h-" ^ seems at least plausible that the high repute for "skill and accuracy" generally conceded the general machinist, or "mechan- ic," is due to a misjudgment of the public as to the difficulties in- volved in measuring to the one-thousandth part of an inch. Using the mechanical measuring equipment which is available in any machine shop, any boy of the intelligence level of a high school graduate should be able to learn to measure to that degree of accu- racy in about fifteen minutes. Removing one thousandth of an inch of stock from a piece of iron being turned in a lathe is an even Testing and Training for Proficiency 99 simpler process, with fewer chances for error. "Expert" workman- ship, such as is required on but few of the production jobs of indus- try, means ability to measure to the ten-thousandth part of an inch. For a nominal sum one may now buy sets of "size" blocks, turned out in quantitites, which are "guaranteed not to vary over one hun- dred-thousandth part of an inch from the stated dimension." Con- fronted by such statements in the papers, the general public is in- clined to overestimate the amount of skill, knowledge, and "trade secrets" possessed by the typical journeyman. Consistent with the overestimation of trade skill possessed by the typical journeyman, the general public has likewise been inclined to overestimate or misplace the emphasis which should be placed upon intelligence in the acquisition of trade proficiency. A chart, en- titled "Occupational Intelligence Standards" published by the Sur- geon General's Office, in a pamphlet, "Army Mental Tests," clearly illustrates this fact. The chart shows the range in intelligence cov- ered by the middle 50 per cent of each of the trades indicated. Only a very few workmen, such as laborers, have an average intelligence very decidedly below the average intelligence of the whole popula- tion; in perhaps three fourths of the trades of modern industry, "average" intelligence is quite capable of doing work in the manner in which it is now done; in only a very few occupations, such as the clerical vocations and the professions, is an unusually high average amount of intelligence necessary for competency in the trade. If one talks to a foreman who is a leader in his trade, he is rather sure to state the fact that an unusually high degree of intelligence is de- manded in order to be a satisfactory worker in his trade. The man prides himself on the fact that his trade is a very difficult one to learn, requiring years of patience and effort in order to become a very skilled worker. As a matter of fact, no such condition prevails in at least three fourths of the occupations available to boys entering industry. Such men are prone to overrate the ability required in the present day to learn the trade which they may have learned them- selves only by dint of many "hard knocks." Much of the trade information which the tradesman of thirty years ago learned only through experience can now be acquired by a few minutes search in standard reference books on the trade. Informal tests of tradesmen's ability to recall the questions asked, performed immediately after an examination was finished, have dis- ioo Trade Tests in Education closed the fact that few but the very best and most intelligent trades- men can remember even half a dozen questions an hour later. The examination should proceed with dispatch from one question to another, thus giving the subject but little time to consider each question and thus fix it in his mind for later recall. Questions involving complicated mental arithmetic are generally beyond the ability of even the better tradesmen. Questions involv- ing the use of fractions are generally useless in oral trade tests. Mental computation, more elaborate than simple addition of easy numbers is generally too difficult for a tradesman of lower standing than a foreman. Questions involving many limiting conditions, or clauses, are unsatisfactory, owing to the inability of the tradesman mentally to grasp so complicated a situation. By conditions, we mean questions similar to the following: "(i) If you were filing brass, (2) on a speed lathe, (3) using a leather belt, (4) the spindle running at a speed of 1500 revolutions per minute, (5) what ?" As a practical resulting principle for trade test formulation, a ques- tion should not include many conditions if it is to be satisfactory. Essential elements of knowledge may generally be found in such questions, and the question may then be split up into a number of simpler questions. For the same reason, those questions which con- tain but few words are the surer of being correctly comprehended by the workman. If possible, oral questions should be limited to less than twenty-five words in length, and the shorter the better. Short questions have the additional value of being quickly administered. If one were to use twenty questions of twenty- five words each, or forty questions of ten words each in approximately the same exam- ination time the latter procedure is much more likely to give a selec- tion of tradesmen in the correct order of their abilities. Language ability seems more necessary for a foreman than for an average journeyman ; consequently, the longer, more involved questions will be a better measure of foremanship than journeymanship ability. The general education possessed by tradesmen is such as to make one doubtful as to whether success is due to such education, or to intel- ligence and other qualities of character. THE EDUCATION OF TRADESMEN The belief has been fairly common that the tradesman is an "educated" man, and the laborer an "uneducated" man. A tabula- Testing and Training for Proficiency 101 tion, by trades, of the number of school grades completed by 924 tradesmen distributed among thirty trades in the city of Pittsburgh shows that: 1. The median grade at which the various groups of tradesmen left school does not vary greatly, the range being from 7.0 in the case of horseshoers to 10.0 (second year High School) in the case of car- buretor man. 2. In the case of six trades, samplings of tradesmen from Pitts- burgh and Newark (N. J.) the median grades of the men in the same trades varied in the two cities less than an average of .3 grade. 3. The trades were given a trade rank order depending upon the median grade of education of their tradesmen. The correlation, p = .79, between the grades and the average trade rank per grade, indi- cates that, in general, poorly educated men tend to gravitate into trades of which poor education is the more typical, and vice versa. 4. Of all the tradesmen, only 26.4 per cent had persisted through a part of high school or more; and 2.8 per cent had completed a part of a college course or more. 5. Experts are generally the best educated of the three classes of tradesmen, apprentices next best, and journeymen least well edu- cated. This seems to indicate that the journeyman stage is a sifting- out stage, possibly a stage of adaptation of mentality and mechanical aptitude or interest to the job, during which time the better educated (highly intelligent) men become experts, foremen, owners, and the like, leaving behind a group of less capable men, "doomed to be jour- neymen." 6. White laborers, from the Camp Dix personnel office files, were found to be less well educated than colored laborers, while both were very inferior in education to tradesmen. 7. General clerks in New York City were found to have a higher median education than any of the trades. The educational chart on page 104 (Fig. 29) shows comparable results for the above trades and occupations in regard to educa- tion. The results here presented were compiled as the result of interviews with tradesmen in the shop. The length of the bars of this chart indicates the range of the middle 80 per cent in regard to educa- tion. The cross-hatched area of the bar indicates the range of the middle 50 per cent. These data are based on a rather small number of cases, the numbers upon which each distribution is based being 102 Trade Tests in Education indicated on the left of the chart. It will be noticed that such trades of this list as are included in the army intelligence chart parallel re- markably the intelligence of those groups. A correlation between the median ranking in education and intelligence of fourteen trades, for which data are available gives p = .78 =*= .10. TABLE XI Percentage Distribution of Men, According to Number of Grades of School Completed. General Summary of !74.857 Men, Variously Classified Grade Per Cent Leaving School to Enter Succeeding Grade: A B c D £s A B C D E None 2.2 ■ 1.0 4-7 0.3 97-8 99.0 95-3 99-7 1 0.7 0.3 0.5 s 97.1 98.7 94-8 99-7 2 IS ■ 4.8 0.8 0.9 ■ 95-6 97-9 93-9 99-7 3 2.8 1.2 1.8 0.2 92.8 96.7 92.1 99-5 4 5.0 2.8 3-6 0.5 87.8 95-2 93-9 88.S 99.O S 8.0 4.0 4-7 3-4 1.9 798 91.2 89-2 85.I 97.1 6 11.6 12.7 11.8 7-6 2.9 68.2 78.5 77-4 77-5 94-2 7 10.6 22.3 9.8 10.3 10.0 57-6 56.2 67.6 67.2 84.2 8 31.5 3S-0 45-1 34-6 48.3 26.1 21.2 22.5 32.6 35-9 IH. S. 9-7 11.3 6.1 6.0 6.9 16.4 9.9 16.4 26.6 29.0 II H. S. 6.2 6.6 6.4 6.1 8.9 10.2 3-3 10.0 20.5 20.1 Ill H. S. 2.9 2.2 2.7 4.2 51 7-3 l.l 7-3 16.3 15.0 IV H. S. 4-7 1.1 S.o 7-1 • 10.3 2.6 2 2.3 9.2 4-7 A College 0.9 2 0.5 ' 1.2 1-7 2 1.8 3.5 B College 0.6 2 0.3 I.I I.I 2 1-5 2.4 C College 0.3 2 0.4 \ 9-2 0.6 0.8 2 I.I 1.8 D College 0.7 2 1.1 1.7 0.1 2 0.0 0.1 Post-Grad. 0.1 2 0.0 0.1 0.0 0.0 0.0 0.0 0.0 Total 100.0 100.0 100.0 100.0 100.0 Per 26.1 Cent with Part H 21.2 22.5 . S. or Better: 32.6 35-9 No. of Men 18,266 I20.289 3 924 30,583* 4.795 PerC ~«tf with Part Col ege or Better: Av. Grade 7-43 1 7.53 7-83 8.09 8.86 2.6 1 2.3 9.2 4.7 ' Post-graduate college counted as seventeenth grade. 2 College not tabulated. Hence all percentages slightly too high. 3 Only 43,358 cards selected at random from the 120,289 were tabulated. 4 Excludes "aliens, both friendly and enemy." ■Samplings only of three groups taken: (o) Non-referred men, (b) referred and placed, (c) referred and not placed. Weighted for column E according to frequency of occurrence: E = (a x 17) + (6 X 7) + (c x 5). ( Application card did not provide for entry of second and third grades. Testing and Training for Proficiency 103 Table XI. Key to Columns A. Unemployed men at Columbus and Dayton, Ohio, Free Employment Offices 1916-1917 (Toops, H. A. and Pintner, R., "Mentality in Its Relation to Elimination from School," School and Society, Vol. 7, No. 173 and No. 174, 1918, pp. 507-510 and 534-539). B. All Working Boys 16, 17 and 18 years of Age subject to Military Duty, resid- ing in villages or towns of 5000 population or over, 1919. Survey of the Military Training Commission, State of New York. Data kindly furnished by Mr. H. G. Burdge, Director of Vocational Training Bureau. C. Tradesmen, all degrees of skill distributed among 31 trades, Pittsburgh, Pa. Examined in War Department standardization of Army Oral Trade Tests. (Toops, H. A. and Pintner, R., "Educational Differences among Tradesmen,'' Journal of Applied Psychology, Vol. 3, No. 2, 1919, pp. 33-49.) D. Minneapolis Draftees. "Aliens excluded." Malone, T. J., "When Boys Leave School," American Review of Reviews, Vol. 60, No. 6, Dec. 1919, pp. 627- 630.) E. Returned Soldiers, applying for work at the Reemployment Bureau of New York City, 1919. (Toops, H. A., Unsigned news note in the World, New York, Sept. 21, 1919.) A somewhat selected group, containing many clerical workers. Table XI is a distribution table, compiled from all data available to the author, of the grades at which various groups of working men left school. It shows that tradesmen are not, in general, either a superior or an inferior group of people so far as education is con- cerned. This table serves to emphasize the fact that the typical tradesman is not a high school man. Many of our vocational schools attempt work of high school grade. Where such is done, it obviously will fail to affect the lives of the three fourths of working people who have left school before formal vocational instruction is begun. We are interested also in the relationship of the amount of educa- tion possessed, as measured by the grade at leaving school, to the degree of proficiency in the trade. THE RELATIONSHIP OF GENERAL EDUCATION TO PROFICIENCY IN A TRADE Many articles have been written, and much discussion has been devoted to the problem, "Does education pay?" Groups of educa- tors, intent on proving the worth of some specialty in education, or of education in general, have used the argument that such education pays because it may be statistically demonstrated that people with such education receive more wages, rise to higher positions of influ- ence, and the like, than their unlearned brothers. That these argu- 104 Trade Tests in Education Ho- of Men. SO 50 18 9 (LI 18 S3 4Z 3+ 53 30 10 17 43 45 Z0 16 38 47 SO 43 40 &0 47 38 18 40 17 17 IS 19 31 9 130 TRADE Laborers-white. Laborers - colored. Brass Me Iters. Horseshoers. Cobblers. B/acksmiths-shop. Bricklayers. Carpenters -house. Linemen- Telegraph Locomotive Rep'rs. Typewriter Rep'rs. Harness Makers. Steamfitters. Bench Hands. Auto Truck Mech J s. Struc Steel Erec's. Sheet Metal 'Workers. Powerhouse -D.C. Powerhouse -AC- Too/makers. Inside Wire men. General Auto Repr. E/ec Inst. Reprs. Plumbers. Motor Repairers. Vulcanizers. Patternmaker*. Ammonia Pipefrt'rs. Motorcycle Rep'rs. Lathe Hands. Storaae Batterjmm Switchboard Rep'rs. Carburetor Men. General Clerks. i t 3 TH*? t r t s io ii it is f if i* i Egg » I W/){\'!\ 1 fea 1 3 C C L wbh»mm-\ ' H" 1 ' c r~m i viph/w* 1 W/klM ~nm *H ■''''■• 1 r_ O t £ J * 5" * 7 8 9 to II I*. IB n> l*~ Fig. 29. Distributions of Grade at Leaving School of Occupational Groups. The shaded area of each bar shows the range of the middle 50 per cent; the whole bar shows the range of the middle 80 per cent. The vertical line indicates the median of each trade. Testing and Training for Proficiency 105 ments may be fallacious is evident if one but considers the possibility that the group in question may have been a selected group who would have succeeded equally well in industry without the educa- tion. The author was granted permission to evaluate the qualification cards of one hundred and thirty soldiers, just returned from overseas, and applying for jobs as general clerks. Using as a cri- terion of occupational worth whether or not a man was hired when referred to a prospective employer for a job, and evaluating against this, by partial correlation method, the talents for the job, available from the qualification card, we obtain the following equation: in which, X\ is the predicted job value (criterion) (not hired = score of 1 hired = score of 2) X2 is the age in years. x 3 is the grade at leaving school. X4 is the marital condition, (single = score of 1 ; married = score of 2) Xf, is the number of dependents. The composite score, compiled by use of the equation, yields a correlation of .405 with the criterion. The fact that, other things being equal in this composite of talents, the more education a pros- pective employee has the less desirable was he considered by the prospective employer, may be merely a reflection of the economic situation at the time (spring of 1918). Again, one may say that the criterion is too rough for accurate results. The criterion is admittedly rough and defective. The important consideration is that grade at leaving school receives a negative weighting in the composite of talents for the job. The author was also privileged to secure intensive data on a group of nineteen eyelet machine tenders in a large brass factory. This job is semi-automatic in that the tender merely has to keep a number of machines fed with strip metal, and to watch for defects in the prod- 106 Trade Tests in Education uct, and detect imperfect working of the machine. The following measures of job worth were obtained: Variable i. (Criterion). Average daily piece-rate wage over a period of two weeks, reduced to a comparable io-hour day basis, exclusive of overtime and contingent wage. Variable 2. Average of three rankings in trade ability of the workers made by the foreman, one week apart, using the slip arrangement order of merit method. Variable 3. Length of experience on the job to the nearest .01 part of a year. Variable 4. Oral trade test score. One-word-answer form of question, adminis- tered orally and individually by the author. Variable 5. Grade completed in school. Using wages as our criterion, it is possible to determine the money value contributed to daily wage by one question answered on a trade test, by one grade in school completed, etc., all other talents for the job of the composite being constant. The formulae resulting are : In terms of deviations, (il) X, Xi , „ X 3 , Xi Xi — = -3305 - + -0890- + .5204- - .1449- °"l "i a Z °i a i Or, in terms of gross measures, (12) Xi = .0508X2 + .1727X3 + .0456X4 - .1408X5 + K. The correlation of the composite with wages is .687. Again we notice that, although it is an advantage in wages received for a work- man to be rated highly by the foreman, to have long experience at the trade, and to receive a high trade test score, grade at leaving school is a disadvantage, when other talents are equal, to the extent of fourteen and a fraction cents per day per grade of school attained, in this composite; that is, grade at school receives a negative weight- ing in spite of its positive correlation with the criterion. When our educators are willing to evaluate their products by the same methods which they would have industry use in evaluating theirs, both will profit by the renewed interest in education. One should not general- ize from two isolated instances such as these. ' The roughness and defectiveness of any criterion by means of which to evaluate the worth of any educational program, any selective method, makes this form of analysis difficult. The method of analysis suggests prob- lems rather than solves them. Testing and Training for Proficiency 107 It is interesting to note, in the first of the above equations, that "trade test ability" is the most important contributor to wages, fore- man's judgments of his men next best, while mere experience in years is least important of all four, as judged by the magnitude of the partial correlations. School surveys, designed to test the needs for vocational schools, have always been made with the point of view that such education would pay. Hence such surveys have literally been searches for facts to "prove the point" that vocational education would pay. This has resulted oftentimes in misplaced emphasis, to the detriment of voca- tional education. The problem should be "what kind of vocational education will pay?" rather than a search for facts to justify voca- tional education in general. In such surveys, job analyses are commonly made of a number of trades. The personal requirements, personnel specifications, of workers in the trades (for vocational education has commonly meant trade education) are commonly set forth. These have often been set down, possibly on the basis of a few interviews with foremen, or worse still, on the basis of the opinion of some one or two authorities in vocational education as to what ought to be the personal require- ments of the worker. As contrasted with such points of view, we have that developed in the trade-test work of the army, in which the requirements of the worker are any requirements above the abilities of the x-percent- ile man, his standing in the trade being derived independently of the talents, or tests, for the job, — preferably being some comparative measure of production. When requirements for the job are viewed in this light, and sub- jected to statistical treatment, it will probably appear that many of the traits now considered important will prove to have low partial correlations with demonstrated ability on the job. It is instructive to think of the total sum of any industrial popu- lation's talents, at any one given time, as being a definitely limited quantity of talent; thus 1,000 men together will have had 8,000 years of school attendance, and so on. When education is considered in this light, almost all large groups of workmen will be found to have a very large percentage of "successful" men who fall far short of the educational standards set up by ordinary job analyses. Table XI gives a summary of the distribution of grade at leaving school of 108 Trade Tests in Education 174.857 workmen, as tabulated from the men's own statements recorded on questionnaires. A casual inspection of this table will prove that about half of our working population, with less than eighth grade education, find jobs somewhere. When our vocational educators find vocational education for this half of our population, then such education will more nearly meet the needs of those not "predestined" to success even without the education. As a partial solution of the problem of vocational placement, we need to recognize traits as being of variable worth, between definite limits. The present tendency is to set up merely a minimum limit (usually an "ideal" limit) which, because of a definite limitation of the total sum of human talents, it is ordinarily impossible to main- tain. We should be willing to admit that possibly a sixth grade edu- cation is a minimum limit for general clerks, that possibly not more than one year of college education is desirable, and that within the aforementioned minimum and maximum limits, a seventh grade education fits better for the job than a sixth grade, an eighth than a seventh, and so on. This view is at total variance with the economic philosophy which would attempt to educate only the few in order that the world might be well supplied with "workers." Instead, it means conservation of our human resources by reserving to the unintelligent those jobs, any harder than which they are incapable of performing with accept- able speed and accuracy, in short, the highest type of vocational guidance philosophy. THE USE OF TRADE TESTS AS INCENTIVES TO LEARNING It is even more difficult for the student of trade subjects to know when he has done his work well than for the academic or scientific student. The academic student, with whom learning is largely con- tent learning, may usually easily classify categorically the amount and quality of his lesson learned as "the amount learned to the point of recall." His test of knowledge and skill gained is the simple test of recall. The scientific student may easily apply the same test to his content matter; the student working on experimental work usually has the self-applicable test of whether his figures agree with the expected figures of the textbook, whether his chemical analysis reveals the same chemical elements which the chemistry instructor put into the test solution, and so on. Trade instruction usually con- Testing and Training for Proficiency 109 tains a minimum amount of content instruction, and much manual manipulation. It is difficult to measure the product of manual ma- nipulation subjectively. The use of self-administrative performance trade tests for meas- uring mechanical products may be made of great use as incentives to better trade performance in the case of students sufficiently advanced to apply the tests. Substantially this plan is already in operation in two engineering colleges known to the writer. In the first of these schools, a simple time standard of performance quality being judged ''passable" by the student foreman, has been set up by the best average records of former classes. Those students who beat this time standard are allowed extra credit in grades for the superior performance. There is great rivalry among the boys to turn out product as rapidly as possible, and a keen group cooperative spirit of determination to beat the best record of the previous classes. The method is lacking in the important feature of failing to suffi- ciently stress "good" work as well as quantity of work, now stressed. In the second school, percentile norms of performance have been set up. This has obvious advantages over a categorical "standard time," as the student can see for himself not only that he is better or poorer than the standard, but just how much better or poorer. For school purposes the merit of a timed production of work obviously depends upon a measure of dispersion of the distribution of times of former students as well as upon the absolute values of the respective times taken. In industry, the boy who turns out a product in half the time of another is worth twice as much wages as the latter. In vocational school work, the primary emphasis should be upon the acquirement of a certain amount of trade skill, the time required being a matter of secondary importance. If percentile norms (or index numbers derived from an assump- tion of a normal curve of distribution of trade ability measured in cr's) were set up for time of performance of all vocational school projects, and if a similar, or preferably an absolute standard of quality of performance were likewise set up for all vocational school projects, then every task performed by the vocational student would be a cumulative test of the degree of his skill at the trade. These tests should be self-administrative. In this simple manner the student may be continually faced by the two generalized measures or objectives of trade work, quality and quantity of product. no Trade Tests in Education One of the big advantages of such tests of daily production is the psychological advantage of interest in the work which is created by receiving the rating immediately. A keen interest is usually kindled by any formal examination. If the ratings are not received for sev- eral days or weeks, during the time required by the teacher for scor- ing the papers, much of the original interest in the ratings has been lost by the time the ratings are received. The value of a series of daily grades, determined by the student himself, in keeping him interested in his work can be illustrated by a few percentile ratings which presumably might be received: 75, 83, 70, 86, 83, 20. Were the student required to plot his grades as a daily, or job require- ment, the graph afforded by the above percentile ratings would be sufficient incentive to most boys to redouble their efforts on the seventh day. And, inasmuch as the percentiles are based on the work of past classes, the poor pupil has, in effect, an absolute stand- ard to beat and not the "unbeatable" record of relative standing in his class. "Working for grades" under such conditions of objective measurement, free from the instructor's bias, is a highly laudable performance. For rating quality of product, a series of "limit" or ''snap" gages can be easily provided for many products. Where judgments of quality are needed, uniform blanks of analyzed "traits" similar to the army performance test scoring points might be provided and the student be required as part of his task to rate his own product, being graded on his ability to rate his product as well as on the excellence of the product itself. Formal tests of performance on analyzed important aspects of trade proficiency may also be made self-administrative and self- scorable. The performance measurement test of Chapter I, designed to test the student's ability to use measuring instruments, is scorable by the use of a stencil in but a minute of the student's time. It would seem possible to devise such diagnostic tests of individual weak- nesses and strengths in a large number of the operations and pro- cesses found in the machine or woodworking shop. The self-scorable feature of the one-word-answer form of test has been found to have a big appeal to the interest of students in classes in content subjects, where the method has been given a trial. Examinations given weekly or oftener, taking only a half hour or so for the complete examination including the scoring, may be found Testing and Training for Proficiency 1 1 1 to be of much more value, both as incentive and as rating method, than much longer examinations given at longer intervals and scor- able only by the teacher at a great expense of time and effort. Using such tests, it is not even necessary that the questions be mimeo- graphed. The teacher, standing before the class, merely reads the questions and waits a short time for the students to record their answers before proceeding to the next question. All students may then exchange papers, the answers being scored either right or wrong by the pupils as the teacher slowly reads the correct answer list. The results thus far show that this method may be applied successfully with students as low in ability as the third semester of high school. The method will be found applicable, in some subjects, to the upper grades of the elementary school as well. APPENDIX STATISTICAL METHODS The Pearson correlations herein reported, of all variables of n classes with n classes insofar as it was practicable to plot them, were computed by means of stencil and tables of deviation squares by fre- quencies, using the following specific formula : - I (SX 2 + SF 2 ) - 2(X - Y)A - (SX)-(SF) (13) r = — === VN (XX 2 ) - (SX) 2 VN (SF 2 ) - (SF) 2 in which X and F are stencil steps. For the fourfold Pearson correlations entering into partial correla- tion regression computations, such cases as the correlation between single — married and not-hired — hired, the formula used is, (14) r w = ad — be V(a + c) (ft + d) (a + 6) (c + d) in which ad represents the product of the (single — not-hired) by the (married — hired) frequencies. For the Pearson correlations between a variable of n classes and a categorical variable of two classes, the following formula was used : (,,\ N ?(b -F) - B (SF) . ,. , 05) r = K ' — , in which, VA-B ViV(SF 2 ) - (SF) 2 A is the total frequency of the lesser degree of the cate- gories of the .X-variable, B is the total frequency of the greater degree of the cate- gories of the X-variable, Sft-F is the sum of the ft (greater degree categorical frequen- cies) by their respective F-gross scores. Appendix 113 For intercorrelations of n variables, using a calculating machine, it has been found very advantageous : (1) To arrange the crude scores in parallel columns, X lt X 2l . . . Xn, (2) subtract from each gross score the lowest gross score of the col- umn yielding the remainders, Xi — c u X 2 — c 2 , X n — c„, and (3) then compute the correlation in parallel columns headed, (X, - Cl )\ (X, - c 2 ) 2 . ...(*„- c n )\ (Xr - *)■ (X t - c), (X, - cJ-(X 3 - c$,....(X„ - £,,)•(!, - c„). Most of these multiplications may be done mentally (the squares of all numbers from 1 to 40 may be readily memorized) or by means of a simple multiplication table. The method will be found to be par- ticularly efficient when N is not over 50 persons. The formula used is, (16) VNXX 2 - (SX) 2 ViVSF 2 - (SF) 2 The general arbitrary scoring formula for tests given by the mul- tiple choice method, and applicable either to work-limit or time- limit methods of scoring, where n is the number of choices, R the number of attempted questions answered correctly, and W is the number of attempted questions answered wrongly (or omitted), and A is the number of questions attempted, is, , tV > s=r- — ^- -w. (17) n - 1 The probable error of an individual score is given by the formula, t, ^ . ( » ) / R-W If a criterion is available, so as to use the partial regression equa- tion of (19) S = R + C ■ W, wherein C has the value determined by Thurstone, 1 1 Thurstone, L. L., "A Scoring Method for Mental Tests." Psych. Bui., Vol. 16, No. 7, 1019, pp. 233-240. H4 (20) Trade Tests in Education . A. T. C.) War Department, Washington, 1919, 179 pp. 48. War Department, Division of Psychology, Surgeon General's Office. Army Mental Tests, Washington, 1918, 23 pp. (a pamphlet). 49. Watson, M. S. Trade Test Principles in Industry. Industrial Management, Vol. 59, No. 3, 1920, pp. 247-250. 50. Wardner, C. A. Applying the Army Trade Tests to Vocational Schools. Industrial Arts Magazine, Vol. 8, 1919, pp. 402-403. vrTA Herbert Anderson Toops was born at Kiousville, Ohio, Septem- ber 18, 1895. After six years of attendance at country schools he entered Midway High School, Sedalia, Ohio, graduating therefrom in 1912. From 1912-1914 he attended Ohio Wesleyan University; and from 1914 to 1917, Ohio State University in the College of Engi- neering, and later in the College of Education. He received the degrees of B. A. and B.Sc. in Education in 1916 and M. A. in 1917. During the war, he entered the Trade Test Division of the Com- mittee on Classification of Personnel in the Army, where he served in the various capacities of standardizer, assembler and statistician on oral trade tests until January, 19 19. At that time he entered the newly formed Technical Interview Methods Section of the United States Department of Labor for the purpose of introducing army trade test methods into the U. S. Employment Service. Later, he served as statistical assistant in the statistical evaluation of the National Research Council group intelligence tests. During 1919- 1920 he was a research scholar at Teachers College, Columbia Uni- versity. Since August, 1920, he has been engaged as specialist in the construction of vocational tests for the E. and R. Schools of the Army. His publications, to date, in full or in joint authorship, are as follows : 1. Pintner, R., and Toops, H. A. A Chart for Rapid Computation of Point Scale Scores. Journal of Delinquency, Vol. 2, No. 4, 1917, pp. 209-210 plus chart. 2. Pintner, R., and Toops, H. A. A Mental Survey of the Population of a Workhouse. Journal of Delinquency, Vol. 2, No. 5, 1917, pp. 278-287. 3. Pintner, R., and Toops, H. A. Mental Tests of Unemployed Men. Journal of Applied Psychology, Part I, Vol. I, No. 4, 1917, pp. 325-341; Part 2, Vol. 2, No. 1, 1918, pp. 15-25- 4. Toops, H. A., and Pintner, R. Mentality in its Relation to Elimination from School. School and Society, Vol. 7, No. 174, 1918, pp. 507-510; and No. 175, 1918, pp. 534-539- 5. Pintner, R., and Toops, H. A. A Revised Directions Test. Journal of Edu- cational Psychology, Vol. 9, No. 3, 1918, pp. 123-142. 6. Pintner, R„ and Toops, H. A. A Drawing Completion Test. Journal of Applied Psychology, Vol. 2, No. 2, 1918, pp. 164-173. 7. Toops, H. A., and Pintner, R. Variability of the Education of Unemployed Men. Journal of Applied Psychology, Vol. 2, No. 3, 1918, pp. 207-218. 8. Toops, H. A., and Pintner, R. A Chart for the Determination of I. Q. Values. Journal of Delinquency, Vol. 3, No. 6, 1918, p. 272 plus chart. 9. Toops, H. A., and Pintner, R. Educational Differences Among Tradesmen. Journal of Applied Psychology, Vol. 3, No. 1, 1919, pp. 33-49. 10. Toops, H. A. Plotting Equations of Three Variables in Mental Measure- ments. Psychological Review, Vol. 26, No. 4, 1919, pp. 317-326. 11. Toops, H. A., and Pintner, R. Mentality and School Progress. Journal of Educational Psychology, Vol. 10, No. 5-6, 1919, pp. 253-262. 12. Chapman, J. C, and Toops, H. A. A Written Trade Test: Multiple Choice Method. Journal of Applied Psychology, Vol. 3, No. 4, 1919, pp. 358-365. 13. Toops, H.A.,and Pintner, R. Curves of Growth of Intelligence. Journal of Experimental Psychology, Vol. 3, No. 3, 1920, pp. 231-242. •»- iff* ff." fc*T' .? $**< l^-^ ^Bfc <&**