The following appeared in Volume 98, Number 1 (Fall, 1998) of the APA Newsletters

In March 1998 Paul Allen Virtual Education Foundation announced a competition for the Outstanding Online Course Award. The competition received 182 entries from around the world. Five judges developed the competition criteria and assessed the entries. The results were announced on May 15 with the $25,000 award going to Professor Brad Cox of George Mason University for his course, Taming the Electronic Frontier. Eight courses were given honorable mentions. Results, details, and course links are available at <www.paulallen.com/foundations>.

This editor was honored to be a participant on the development committee and the judging panel for the award. The perspective I gained from examining and critiquing 182 internet-based courses raised my awareness of the nature of on-line teaching and the issues involved in judging course quality generally.

The submitted courses came from a wide range of the curriculum. Humanities courses were well represented as were science, health, math, business, and languages. Among the entries were five philosophy courses:

Like most of the courses submitted to the competition, the philosophy courses are well produced and innovative uses of the internet media. Some of these have been reviewed in Philosophy and Computers and elsewhere. It is important to see such achievements, given the all too common assumption that humanities, especially philosophy, education is not suitable for the internet.¹

Producing a competition to select outstanding online courses requires criteria for educational quality. Developing and applying these criteria proved to be the central challenge of this effort. The judges were in no position to be making assessments of the relevance and quality of the content of 182 courses from such a wide curricular sample. The effort came to focus on the success of the course characteristics relative to the objectives set out by the developers. Thus, the criterion appropriateness of the media used was judged by whether those uses significantly advanced some aspect of the learning sought by the course. For example, numerous courses use animated graphics to grace their pages. Only a few used animated graphics in ways that contributed significantly to the subject matter. It is possible decorative uses of moving graphics may contribute to appeal and motivation; they are not pedagogically useless. Still, an animation that shows the path of flow in a process or traces a journey across a map, has used the medium in ways to advance specific aspects of the learning objectives. Where the course characteristics correlate to course aims, we have evidence of strong instructional design.

As a result of the award process, I have continued to work on grounds for judging course quality, resulting in six categories of criteria (with examples of specific measures):

1) Objectives

• Are the objectives clearly stated?

• Are the objectives complete?

• Are performative objectives provided?

2) Appropriate Use of Technology

• Are uses of technology appropriate relative to the course objectives?

• Are uses of technology made to enhance the course content?

3) Self-Assessment

• Are opportunities for student-directed self-assessment available?

• Are students guided in revision of their performance based on self-assessment?

4) Instructor-Student Interaction

• Is student-to-instructor interaction readily available?

• Do the students receive frequent instructor feedback?

5) Student-Student Interaction

• Are opportunities for student-to-student interaction provided?

• Is student-to-student interaction integral to the Learning

• Are aspects of the student-to-student interaction directed by the instructor?

6) Course Evaluation

• Are students provided opportunities to evaluate the course?

• Are a variety of instruments used to evaluate course effectiveness?

• Is formative evaluation used throughout the course?

• Is anecdotal reporting a part of the course evaluation?

The Outstanding Online Course Award recipient, Taming the Electronic Frontier by Professor Brad Cox <www.virtualschool.edu/98al>, is remarkable as an exemplar of these criteria. The course uses a database to track each student and deliver adaptive pages customized to the individual student. This approach allows students to be involved in several group projects simultaneously while always having clear indicators as to what they have completed and what needs to be done. Cox gives students ongoing assessments of the quality of their work (graphically and verbally) in relation to both the work of other students and the goals of the course. A crucial move in this process is peer review among the students, for that is the richest form of qualitative exchange and judgement. Professor Cox uses Zen and the Art of Motorcycle Maintenance by Robert Pirsig to introduce reflections about the nature of "quality." It is fitting that the course which is acknowledged with top honors is one that takes quality seriously enough to make it part of the subject matter.

Awards provide touchstones for further reflection and practice. A broad effort such as the Allen Foundation award is useful in setting a direction. It is up to the individual disciplines to fill in the details and test those directions. Academic philosophers should take on this task by developing criteria and guidelines for the highest quality online philosophy courses. This task is challenging, due in part to the opposition some academics hold generally to online courses. Yet, that very challenge renders the task crucial. Curriculum development and promotion and tenure are already affected by attitudes toward teaching online. Online course developers typically face demands to decisively demonstrate the quality of their products. Such demands are justified when rigorous criteria for course quality are in place and the on-line teaching is held to criteria relevant to the media. In the effort to verify the quality of on-line courses, some teachers have run up against the ambiguities of traditional course assessment. It is not clear, for instance, how classroom lectures and class discussions are qualitatively assessed, other than the anecdotal "prof. review" in which a colleague attends and reports on a session. Still, online teachers are frequently expected to prove that the online tutorial is as effective as a lecture and the online discussion of a quality equivalent to in-class discussion. In order to make comparative assessments, we need comparable assessment methods.

The foundations of sound course quality assessment are the learning objectives set by the discipline. It is by this measure that uses of various methods and media can be effectively judged. Academic philosophy needs to develop clear statements of such objectives for general use. I believe the APA should undertake a survey and open discussions over the possible objectives that philosophy courses may hold themselves to. Wide variations in style and content are possible within common criteria of excellence. A process focused on general philosophy course quality, inclusive of online methods, will be an aid to innovative educators and will do the discipline good.

Notes

1. Consider, for example, this claim made in a report widely distributed to educational administrators; "Some fields are not suited to extensive computer mediation, especially those concerned with questions of meaning and value, of culture and philosophy." William F. Massey and Robert Zemsky, Using IT to Enhance Academic Productivity, (NY: Educom, 1995), p.5.

Table of Contents

• From the Editor
• From the Chair
• Event Handlers
  • Selmer Bringsjord, Bill Uzgalis
• Internet Resources
  • Using Audio and Video in World Wide Web Ethics Conferences and Lectures, Lawrence M. Hinman
• Web Site Search
  • William James, Reviewed by Larry Hickman
  • Philosophy Pages from Garth Kemerling, Reviewed by Steven K. Strange
  • Electronic Journals: Number 3, Reviewed by Robert L. Causey
  • Web Site Search Reviews Index
• Teaching in Cyberspace
  • Teaching Philosophy of Art On-Line, Julie Van Camp
  • Web-Assisted Critical Thinking, George Rainbolt and Steven Rieber
  • Using the Internet in the Philosophy Classroom, Andrew N. Carpenter
  • From the Blackboard to the Screen: Reflections on Using a Computer in the Classroom, Lawrence M. Hinman
  • The Southern Regional Electronic Campus, Ron Barnette
• Announcements
• Platform
  • Judging Courses of Quality

Return to the Index for Volume 98 Number 1 of the APA Newsletters