This article is based on a paper presented at the 11th Annual
Conference on Research in Medical Education at the 83rd Annual
Meeting of the AAMC, Miami Beach, Florida, November 6, 1972.
Dr. Naftulin is associate professor and director of the Division of Continuing Education in Psychiatry, University of Southern California School of Medicine; Mr. Ware is assistant professor of medical education and health care planning and director of research and evaluation, Southern Illinois University School of Medicine; Mr. Donnelly is an instructor in psychiatry (psychology) in the USC Division of Continuing Education in Psychiatry.
|Abstract - On the basis of publications supporting the hypothesis that student ratings of educators depend largely on personality variables and not educational content, the authors programmed an actor to teach charismatically and non substantively on a topic about which he knew nothing. The authors hypothesized that given a sufficiently impressive lecture paradigm, even experienced educators participating in a new learning experience can be seduced into feeling satisfied that they have learned despite irrelevant, conflicting, and meaningless content conveyed by the lecturer. The hypothesis was supported when 55 subjects responded favorably at the significant level to an eight-item questionnaire concerning their attitudes toward the lecture. The study serves as an example to educators that their effectiveness must be evaluated beyond the satisfaction with which students view them and raises the possibility of training actors to give "legitimate" lectures as an innovative approach toward effective education. The authors conclude by emphasizing that student satisfaction with learning may represent little more than the illusion of having learned.|
Teaching effectiveness is difficult to study since so many variables must be considered in its evaluation. Among the obvious are the education, social background, knowledge of subject matter, experience, and personality of the educator. It would seem that an educator with the proper combination of these and other variables would be effective. However, such a combination may result in little more than the educator's ability to satisfy students, but not necessarily educate them.
Getzels and Jackson (1) have stated that the personality of the teacher might be the most significant variable in the evaluation of teaching effectiveness. Wallen and Travers (1) also supported this concept in stating that "we have tried to demonstrate that patterns of teacher behavior and the teaching methods they represent are mainly the products of forces which have little to do with scientific knowledge of learning."
Similarly, Goffman (2) viewed audience receptivity to a lecturer as highly influenced by the person introducing him as well as by the quality of the introduction. In addition, Goffman described an audience as influenced by the speaker's "involuntary expressive behavior" as much as by the expressed information he wished to convey. This is especially so if the audience has had little time to evaluate the information. Consequently, the learner's impression of the information conveyer becomes a decisive factor in how he responds to the information conveyed.
Rogers (3) stressed the importance of humanizing our educational institutions by bringing "together the cognitive and the affective-experiential" aspects of learning. He also discussed the significance of the educator's genuineness. He feels that the educator who does not present a facade is more likely to be effective. The educator, states Rogers, must have a "direct personal encounter with the learner."
In one study (4) in which student perceptions of educators in 1,427 seventh through 12th grade classes were factor analyzed, it was reported that the students regarded "teacher charisma or popularity" as the most important characteristic when rating teachers. The article further states that "students do not respond directly to specific questions regarding teacher effectiveness. Rather a kind of halo effect on teacher charisma or popularity determines to a large extent how students react to questions about their teacher."
If charisma or popularity have such an effect on the rating of teachers by junior high and high school students, the authors wondered whether the ratings of a highly trained group of professional educators in a learning situation might be similarly influenced. If that were the case, a demonstration of the personality factor in perceived learning might serve to arouse the group members' concern about the proper combination of style and substance in their own teaching.
The hypothesis for this study was as follows. Given a sufficiently impressive lecture paradigm, an experienced group of educators participating in a new learning situation can feel satisfied that they have learned despite irrelevant, conflicting, and meaningless content conveyed by the lecturer.
To test the hypothesis, the authors selected a professional actor who looked distinguished and sounded authoritative; provided him with a sufficiently ambiguous title, Dr. Myron L. Fox, an authority on the application of mathematics to human behavior; dressed him up with a fictitious but impressive curriculum vitae, and presented him to a group of highly trained educators.
The lecture method was the teaching format selected since it is one used extensively in the professional educational setting. It has been described as the one teaching method during which most of the time the instructor talks to the students (1). Its acceptance as an effective teaching tool is attributable mainly to its time-testedness.
Dr. Fox's topic was to be "Mathematical Game Theory as Applied to Physician Education." His source material was derived from a complex but sufficiently understandable scientific article geared to lay readers (5). One of the authors, on two separate occasions, coached the lecturer to present his topic and conduct his question and answer period with an excessive use of double talk, neologisms, non sequiturs, and contradictory statements. All this was to be interspersed with parenthetical humor and meaningless references to unrelated topics.
Eleven psychiatrists, psychologists, and social-worker educators who were gathered for a teacher training conference in continuing education comprised the learner group. The purpose of the conference was to help this group be more effective educators of other health professionals by providing them various instructional goals, media, and experiences. Dr. Fox was introduced as "the real McCoy" to this unsuspecting group; and he presented his one-hour lecture in the manner described, followed by a half hour discussion period which was hardly more substantive.
At the end of his performance an authentic looking satisfaction questionnaire was distributed to which all 11 mental health educators were asked to respond anonymously (Table 1). The introduction of the lecturer as well as his lecture and discussion were videotaped for use with other groups.
Significantly, more favorable than unfavorable responses to the questionnaire were obtained (chi-square = 35.96, p < .001). The one item with most favorable responses was the first, "Did he dwell upon the obvious?" It was the feeling of half the group that he did. The remaining items received a majority of favorable responses. No respondent reported having read Dr. Fox's publications. Subjective responses included the following:
Excellent presentation, enjoyed listening. Has warm manner. Good flow, seems enthusiastic. What about the two types of games, zero-sum and non-zero sum? Too intellectual a presentation. My orientation is more pragmatic.
Because the first group was few in number and quite select, the authors sought other subjects with similar experience and professional identity who might provide further data to test the hypothesis.
The second group consisted of 11 subjects who were psychiatrists, psychologists, and psychiatric social workers, all identified as mental health educators. A videotape of the previously described lecture and discussion period as well as the preparatory introduction was shown to the group. After the presentation group members responded to it using the same questionnaire as did the first group (Table 1). Favorable responses far outweighed unfavorable responses, and the difference between the two was (chi-square = 64.53, p < .001). All responded favorably to the first item, which means that they felt he did not "dwell upon the obvious." There were also significantly more favorable than unfavorable responses to the other items and one respondent reported having read the lecturer's publications. Some subjective statements were:
Did not carry it far enough. Lack of visual materials to relate it to psychiatry. Too much gesturing. Left out relevant examples. He misses the last few phrases which I believe would have tied together his ideas for me.
Still more subjects were sought to further test the hypothesis.
The third group was different in that it consisted of 33 educators and administrators enrolled in a graduate level university educational philosophy course. Of the 33 subjects in this group, 21 held master's degrees, eight had bachelor's degrees, and four had other degrees which were not specified. Most of these educators were not specifically mental health professionals but had been identified as having counseling experience in their respective schools. The videotape of the lecture was again presented to this group, after which the educators responded to it by using the same questionnaire as the first two groups (Table I).
Again the number of favorable responses was significantly greater than the number of unfavorable responses (chisquare = 102.83, p < .001). The majority of respondents from Group III also did not feel the lecturer dwelt upon the obvious, and they also responded favorably for the most part to the other items. Subjective responses, when given, were again interesting. Some were:
Lively examples. His relaxed manner of presentation was a large factor in holding my interest. Extremely articulate. Interesting, wish he dwelled more on background. Good analysis of subject that has been personally studied before. Very dramatic presentation. He was certainly captivating. Somewhat disorganized. Frustratingly boring. Unorganized and ineffective. Articulate. Knowledgeable.
Given the responses of these three groups of educators to the lecture paradigm, the authors believe that the study hypothesis has been supported.
EXAMPLES OF QUESTIONS USED AND PERCENTAGE OF RESPONSES* FOR THREE GROUPS
|Questions||Group I||Group II||Group III|
|Did he dwell upon the obvious?||50||50||0||100||28||72|
|Did he seem interested in his subject?||100||0||91||9||97||3|
|Did he use enough examples to clarify his material?||90||10||64||36||91||9|
|Did he present his material in a well organized form?||90||10||82||18||70||30|
|Did he stimulate your thinking?||100||0||91||9||87||13|
|Did he put his material across in an interesting way?||90||10||82||18||81||19|
|Have you read any of this speaker's publications?||0||100||9||91||0||100|
|Specify any other important characteristics of his presentation.|
|* "Yes" responses to all but item one are considered favorable.|
The notion that students, even if they are professional educators, can be effectively "seduced" into an illusion of having learned if the lecturer simulates a style of authority and wit is certainly not new. In a terse but appropriate statement on educators, Postman and Weingartner (6) emphasized that "it is the sign of a competent crap detector that he is not completely captivated by the arbitrary abstractions of the community in which he happened to grow up." The three groups of learners in this study, all of whom had grown up in the academic community and were experienced educators, obviously failed as "competent crap detectors" and were seduced by the style of Dr. Fox's presentation. Considering the educational sophistication of the subjects, it is striking that none of them detected the lecture for what it was.
In addition to testing the hypothesis, the paradigm was to provide these professional educators with an example of being educationally seduced and to demonstrate that there is much more to teaching than making students happy, A balanced combination of knowledge and personality are needed for effective teaching even if the student does not require the former to sustain the illusion that he has learned. It is hoped that this experience has helped respondents from these three groups to question their educational effectiveness more meaningfully.
To the authors' knowledge a simulated teaching paradigm such as this with student responses to subsequently perceived learning has not been reported. Despite the usual reservations about generalizing data from only 55 subjects, the results of the study raise some interesting questions. The first involves the content of the lecture. Does a topic seemingly short on content and long on ambiguity or abstraction lend itself more readily to such a lecture paradigm than a content-based factual presentation from a more concrete topic area? The answer is an equivocal "yes," as a subject in Group I noted after being told of the study's design. He said he felt that the lecturer might have had a tougher time talking nonsense about a more concrete topic but even under those circumstances a fake lecture could be "pulled off" with an unsuspecting group. This raises the next question.
If the group were more sophisticated about a more concrete aspect of the lecturer's subject matter, in this case mathematics, would he have been as successful in seducing the respondents into an illusion of having learned? Probably not. Or at least the lecturer would have to be extremely skillful to be successful. The study also raises the larger issue of what mix of style and substance in the lecture method is optimal for not just integrating information in a meaningful way but for providing learning motivation as well. Although the study was not specifically addressed to this question, the fact that no respondents saw through the hoax of the lecture, that all respondents had significantly more favorable than unfavorable responses, and that one even believed he read Dr. Fox's publications suggests that for these learners "style" was more influential than "content" in providing learner satisfaction.
A more ideal assessment of the relative value of content and style in determining learner-reporter satisfaction might consist of programming the same "lecturer" to systematically alter the content of his presentation before three equivalent groups of learners. Simultaneously, his "involuntary expressive behavior" would remain constant for each of the three groups; for example, Group A would receive sufficient content conveyed with sufficient "involuntary expressive behavior," Group B moderately insufficient content accompanied by the same "involuntary expressive behavior" as was displayed with Group A, and Group C totally inadequate content delivered in the same manner as to the first two groups; the three groups of learners could then be more systematically compared as to learner perceived satisfaction.
After the respondents *in the actual study were informed of its purpose, numerous subjects from each group requested the article from which the lecturer was programmed. Reported intent of these requests ranged from curiosity to disbelief, but the authors were told by some respondents that Dr. Fox did stimulate interest in the subject area even after the respondents were told of the study's purpose. Despite having been misinformed, the motivation of some respondents to learn more about the subject matter persisted. Consequently it is the authors' impression that the "arbitrary abstractions" suggested by Postman and Weingartner have some initial pump-priming effect on educational motivation.
The relationship of the illusion of having learned to motivation for learning more has not been fully addressed here, but should a positive relationship exist, this study supports the possibility of training actors to give legitimate lecture as an innovative educational approach toward student-perceived satisfaction wit the learning process. The corollary would be to provide the scholar-educator with more dramatic stage presence to enhance student satisfaction with the learning process. Either extreme has a soap-selling quality not likely to lather the enthusiasm of the pure scholar. However, this paper is not addressed to him but rather to student-perceived satisfaction with how well he has shared his information. More important, as has been noted, it suggests to the educator that the extent to which his students are satisfied with his teaching, and even the degree to which they feel they have learned, reflects little more than their illusions of having learned.
1. Gage, N. L. (Ed.). Handbook of Research on Teaching.
New York: Rand McNally, 1963, pp. 506, 464, and 481.
2. Goffman, E. The Presentation of Self in Everyday Life. New York: Doubleday, 1959.
3. Rogers, C. R. Bringing Together Idea and Feelings in Learning. Learning Today, 5:32-43, Spring, 1972.
4. Coats, W. D., and Swierenga, L. Student Perceptions of Teachers. A Factor Analytic Study. J. Educ. Res., 65:357-36, April, 1972.
5. Rapoport, A. The Use and Misuse of Game Theory. Scientific American, 207: 108-114, December, 1962.
6. Postman, N., and Weingartner, C Teaching as a Subversive Activity. New York: Delacorte Press, 1969, pp. 1-15.