CT OM EEG TM MIU 7/14

One consideration should be kept in mind concerning follow-up studies using the WGCTA. Welch and Walberg (1970) found no evidence for inflated posttest scores caused by the experience of a pretest and conclude that

:..it might be speculated that pretest and sensitization effects are not serious problems when the treatment occurs in a normal classroom situation over the academic year and when test taking is part of the normal routine. (p.613)

Contrary to this assumption of Welch and Walberg, Lucus (1972) did find a pretest effect with the WGCTA in an experiment with South Australian high school biology students.

The authors of the WGCTA tried to insure unquestionably right answers as the standard for the scoring key. To this end they submitted all the items to a jury of 15 persons trained in logic and the scientific method who apparently showed perfect agreement as to the correct responses.

The WGCTA received good 'reviews" in Burns' Mental Measurements Yearbooks. Robert H. Thouless of Cambridge University (Buros, 19 . p.§ ) wrote "No one who has tried to construct a test of this type will underestimate its difficulties. The authors have succeeded in making a test which should prove useful for measurement and diagnosis and ...there are ingenious methods of assessing qualities of opinions and...the ability to think critically on particular problems."

Harold Fawcett (1945,p. 139) cited the WGCTA as “a helpful and effective means of measuring abilities which are today so essential to competent citizenship" but "probably best suited for students on the senior-high school level."
 

Walker H. Hill of Michigan State University (Buros, 19 , p.796) wrote that ”If, as this reviewer believes, critical thinking is a central goal of education, serious efforts to understand it and appraise it must be encouraged...and the Watson-Glaser Critical Thinking Appraisal  is one of the useful instruments for this purpose.” But he urged caution in the use of subtest scores. He cited as a "loophole" in the inference subtest the required use of ”certain commonly accepted knowledge or information which practically every person knows" as stated in the WGCTA Manual. Hill also found the norms to be based on a ”regrettably limited college population. "

Carl I. Hovland of Yale University (Buros, 19 , p.797) wrote “The Watson-Glaser test is a conscientious, imaginative effort to provide appraisal in a most difficult area--that of 'critical thinking'."  Hovland also found the norms in the manual ”quite sketchy". He mentioned that it is critical to take into account which form of the WGCTA one uses in a study because of differences ranging up to six points. Hovland also said that since the theoretical relationship between critical thinking I and other measures of intelligence is not established, it is difficult to asses whether the correlation of .70 reported between this test and the Terman-McNemar Test of Mental Ability means that the Watson-Glaser test is measuring a single major aspect of intelligence or is just another form of  intelligence test."

John P. Crites (1965, p.328) found the WGCTA to be a "novel" and "laudable approach" but urged caution in using the subtest scores and suggested using only the total score until more research is done on the WGCTA.  Crites also wrote that "caution should be exercised in using the combined grades percentiles" which this study will do. He said ”the two new forms of the Watson-Glaser, YM and ZM, are supposedly equivalent, but they differ as much as six raw score points at the same percentile, particularly in the middle of the distribution.” Because "there appears to be insufficient range on the test...for college students, particularly those in their last year" the “data raise a question about whether the Watson-Glaser is appropriate for use at the higher educational levels, as the Manual implies."  Crites concludes that "little is known about its stability over time and its usefulness in prediction, but its internal consistency is high and its concurrent validity is acceptable" and suggests "a more difficult form could be devised for the selection of graduate students."

G.C. Helmstadter (1965, p.254) however, found that "in general, the test seems to be an excellent one..." and wrote "this instrument could provide an excellent criterion measure for those who claim their instruction results in the 'abi1ity to think' rather than simply the acquisition of subject matter."  This seems to be what MIU seeks to achieve in its students.
 

Helmstadter agreed with other reviewers that ”some users might welcome a wider variety of normative groups".  In comparing the two WGCTA forms “it does seem strange that when two forms of a test are available. and an equivalent raw score table is presented, no correlation between the two form is given... an examination of both the median item discrimination indexes and the reliabilities for the various subtests suggests that form YM is likely to be slightly superior to form ZM."

This study will use form YM. Helmstadter concludes ”while there may be some flaws in the test, it is doubtful whether a significantly better measure will be found until there is a major breakthrough either in test technology or in our understanding of the 'thinking' process."

The Rokeach Adult Dogmatism Scale was used to measure open-mindedness. "The primary purpose of this scale is to measure openness or closedness of belief systems” (Rokeach. 1966, p.71). There have been five editions of the Rokeach Scale, Forms A through E, each building upon the previous one. Form E consists of no statements which Rokeach and his researchers felt to be typical of dogmatic beliefs, and takes about 12 minutes.

Subjects indicate disagreement or agreement with each item on a scale ranging from -3 (strongly disagree) to +3 (strongly agree), with the 0 point excluded in order to force responses toward disagreement or agreement. For scoring purposes, a constant of 4 is added to each item score to give a 1 to 7 point scale. The total score is the sum of
scores obtained on each of the no items on the test.

"By virtue of the way open and closed are defined, this scale also purports to measure general authoritarianism and intolerance" (Rokeach, 1960, p.96). with such a wide variety of possible belief systems, constructing a short scale with construct and content validity was a
challenge. Rokeach constructed the scale by interviewing a wide variety of people "we intuitively thought to be closed-minded. Above all, each statement in the scale had to be designed to transcend specific ideological positions in order to penetrate to the formal and structural characteristics of all positions” (p.72).
 

The last revision of the Rokeach Scale, the 40-item Form E, was found to have a corrected reliability of .81 for students in English colleges, and .78 for English workers in an automobile factory (Vauxhall Motors) in 1959 (Rokeach, 1960, p.89).` Form E was found to have reliabilities (odd-even) ranging from .68 to .85 in students at Ohio State University
and Michigan State University.  "These reliabilities are considered to be quite satisfactory, especially when we remember that the Dogmatism Scale contains quite a strange collection of items that cover a lot of territory and appear on the surface to be unrelated to each other" (Rokeach, 1960, p.90).

Other open-mindedness/dogmatism scales considered were:

Dommert's adaptation of the Rokeach Scale (1967)

Figart's version of the Dogmatism Scale (1965)

A Short-form Dogmatism Scale for Use in Field Studies (Troldahl and Powell (1965)

A Shortened Version of the Rokeach Dogmatism Scale (Schulze, 1962)

A balanced "positive statement” and "negative statement" dogmatism scale developed by Ray (1970)

Ray's scale was seriously considered because it was felt desirable to make the test instrument being given to the MIU students sampled as positive and uplifting as possible. Yet little research and comparative statistics are currently available for Ray's scale, designed in Australia.

A sample of the Rokeach Adult Dogmatism Scale} Form E, may be found in the Appendix of this paper.