The following appeared in Volume 98, Number 1 (Fall, 1998) of the APA Newsletters

Newsletter on Philosophy and Computers

Web-Assisted Critical Thinking
George Rainbolt and Steven Rieber
Georgia State University

There is presently a great deal of discussion concerning the use of computer technology in the classroom. Some foresee a sea-change while others predict that computer technology, like the advent of television, will have no substantial impact. Obviously, the only way to answer this question is to try using computers in the classroom and see if they help students learn. One cannot answer this question with one project. It may well be that some uses of computers in the classroom are effective and others are not. But one must begin somewhere and at Georgia State we have begun to develop and test one use of computers in the classroom.

The project applies an innovative teaching technique developed for physics courses at Harvard University to critical thinking courses at Georgia State University. The technique was developed by Dr. Eric Mazur. Mazur abandons the traditional lecture format and breaks his class into 10-minute mini-lectures. After each 10-minute period, the instructor asks a question designed to see if the students have understood the basic concept of the mini-lecture. Each student has 30 seconds to answer. There is then a one-minute period in which students can talk to those around them. They then reanswer the same question. The professor can then move on (if most students get the correct answer) or do another mini-lecture on the same concept with another question (if most students do not). Mazur found that students taught in this way show statistically better results on standardized physics tests than do students taught in the traditional way.

Like introductory physics, Logic and Critical Thinking (Phil 241) is an introductory class in which one hopes that the students will grasp a set of basic concepts (e.g., sound argumentation, the identification of standard flaws in reasoning) which will be used in other courses and later in life. The project adapts Mazur’s basic design to apply it to critical thinking and to use the World Wide Web. We have made many changes to Mazur’s basic design. The questions are presented to the students using the web and their answers are reported to the instructor via the web in real time. In addition, at the end of the class, each student receives a report as to how well he or she did on that day’s questions. The professor also receives a report on each student.

The course is given in a classroom equipped with a local area network (LAN). Each student and the instructor sit at a client workstation. The design of the course is platform independent—it will run on any system that can run Netscape 3 or better. The technology does not require that students and/or the instructor be in the same room (or even same continent ). The course uses a set of programs (written in Perl 5 and running on a UNIX mainframe) to process the information, a set of over 200 questions written in the HyperText Markup Language (HTML) and a few Javascripts for some security tasks.

The course, once set up, is very easy to run. Students need only know how to start Netscape, how to type in one initial Uniform Resource Locator (URL), how to click, and how to scroll. We cover these techniques the first day of class and no student has had any trouble navigating through the course. The instructor need only know what the students know and how to type in a six-number code. The Perl 5 programs, however, must be installed on a mainframe and this requires a fair amount of computer knowledge. (On the other hand, any university computer center would have qualified individuals.) The installer must know Perl 5 well, know HTML well, know basic UNIX commands (e.g., ls, cp, rm, chmod), and have a rudimentary knowledge of Javascripts.

To assess the effectiveness of the technique, we use the California Critical Thinking Skills Test (CCTST). This test has been employed at over 500 colleges and universities. We use it in a pre- and post-test format. We did initial shake-down testing during the summer quarter of 1997. We did beta-testing during the fall quarter of 1997. The CCTST was given to four classes—one taught by Dr. Rainbolt using the web technique, and three taught by three of the department’s part-time instructors who did not use the web. (We chose experienced part-time instructors with especially good student evaluations.) Below you will find the average increase in test scores between the pre and the post-test for each class. (The test is scored on a scale of 1-36.)

Class Avg. change in test score
Control Class 1: 0.63
Web Class: 3.34
Control Class 2: +0.85
Control Class 3: +0.48

As you can see, the students in the web class did significantly better than those in the control classes. However, this is a very imperfect control. In particular, the difference in scores may reflect the difference in instructors, not in computer use. Dr. Rainbolt may also have unconsciously worked extra hard because he wanted the web project to succeed. In addition, different texts, tests, and writing assignments were given in each class.

To achieve a better control group, both of us taught two sections of Critical Thinking during the winter quarter of 1998. Each of us taught one section with the computers and one without. The text, the tests, papers, and all other assignments were identical. Here are the results of the pre- and post- testing.

Instructor and Format Avg. change in test score
Rieber with web 2.42
Rieber without web 2.00
Rainbolt with web 3.77
Rainbolt without web 2.80

Once again, the students in the web class did better than those in the control classes. There is, however, a problem with our data. In Dr. Rieber’s classes, the person who gave the test forgot to tell the students to put their names on the test. Therefore, all that is reported here is the difference between the average of those students who were in the class on the first day of the course and the average of those students who were in the class on the last day of the course. These are not the same groups because some students added or dropped the course. (This was, on the other hand, done in both the web and non-web classes.) There is, therefore, the possibility that the increase in scores reflects the fact that poor students tend to drop. The data for Dr. Rainbolt reflects only those students who were in the class on the first day and the last day, i.e. one consistent group of students. It is interesting to note that the improvement which seems to have resulted from use of the web is much smaller than the improvement that seems to have resulted from using regular (tenure-track) faculty instead of part-time instructors. The difference between the Rainbolt or Rieber with and without the web is much smaller than the difference between the average improvement found when part-time instructors teach the course and when regular faculty teach the course. Much more testing needs to be done but the initial results of this method appear promising. We will continue testing in the fall of 1998.

There is a demonstration of this project available to all at: http://www.gsu.edu/~wwwphl/241web/demostart.html. Comments or questions about Critical Thinking on the Web are warmly requested. Please contact George Rainbolt (grainbolt@gsu.edu) or Steve Rieber (phlsdr@panther.gsu.edu)

Notes

*We gratefully acknowledge the support that the development of this project received from two grants—one from the Board of Regents of the University System of Georgia and another from Georgia State University. We would like to thank Mike Palumbo, Philip Mitchem, Dale Richards, and the contributors to the Perl and Javascript newsgroups for their assistance in ridding the program of many nasty bugs.

Table of Contents

apa5.gif (1212 bytes)Return to the Index for Volume 98 Number 1 of the APA Newsletters