APA Statements on the Profession

Outcomes Assessment

The following statement was prepared by the APA Committee on the Teaching of Philosophy, Randall Curren, Chair, and approved by the Board of Officers at its 2006 Meeting.

The purpose of this statement is threefold: to clarify the concept of Outcomes Assessment, to explain how it is used, and to address concerns regarding its implementation.

Outcomes Assessment

The concept of Outcomes Assessment (OA) reflects an increasingly widespread desire to evaluate educational programs in terms of clear and objective criteria. Its chief intent is to develop instruments measuring the correspondence between the claims institutions make for their programs and what they actually achieve. It has come to be applied internationally at all levels of teaching and learning, from primary and secondary through higher education, and to entire degree programs and curricula, as well as to individual courses. OA typically focuses upon three factors: the learning outcomes of a given program, course, or curriculum; the means by which these outcomes are pursued; and measures of the degree to which these outcomes have been achieved by those who complete the program.

While assessment is not new, what is new is that assessment is now associated with accountability. In pursuit of accountability, accrediting organizations around the country have required that colleges and universities create assessment plans for their academic programs. Since accreditation is a precondition for the provision of federally guaranteed student loans, administrators have directed their institutions to develop comprehensive plans for assessing student learning in ways that go beyond assigning grades for performance in courses. Other factors that have contributed to the pressures for accountability are loss of confidence in conventional grading systems due to grade inflation, doubts about the effectiveness of K-12 public education, and demands by state governments, businesses, and professional sectors that graduates exhibit greater readiness for the world of work. Institutions are expected to produce assessment results that reflect both students' mastery of disciplinary content and demonstration of ability. Thus, departments may be asked to demonstrate "objectively" the differences their degree programs make in the development of students' abilities through their work in the discipline. In turn, instructors may be asked to formulate specific outcomes for each of their courses, and to develop instruments that measure the degree to which students attain those outcomes.

(On the history of the outcomes assessment movement in the United States, see Barbara Wright's article "More Art Than Science: The Postsecondary Assessment Movement Today" (http://www.udel.edu/apa/governance/committees/teaching/assessing.html).)

OA in Practice

To some extent, philosophy courses and programs, as traditionally conceived and practiced, have defined learning outcomes and means of assessing these outcomes. There are specific skills that students and majors in philosophy should acquire and refine. These include critical reading and writing skills, as well as the critical thinking standards of clarity, accuracy, relevance, depth, breadth, and coherence. There is specific knowledge that students and majors should acquire, including knowledge of the history of various philosophical debates, the main trends, traditions, concepts, terminology, etc. Syllabi and course outlines typically contain descriptions of what students are expected to learn in a course, of the kinds of skills and competencies they are expected to demonstrate, and of the nature of the instruments that will be used in assessing their demonstration of knowledge, skill, and competence. Outcomes normally are measured in philosophy courses by examinations and other written assignments that test whether students have mastered the objectives appropriate for the course and its level. Departments frequently establish curricula based on a consensus of their instructors regarding minimal standards of competence appropriate to given degree programs; and comprehensive examinations, written and oral, are sometimes used to decide whether students have attained that level of competence.

Currently, however, most philosophy courses and programs do not formulate learning outcomes specific enough to satisfy typical OA requirements. A course description tailored to meet the demands of OA would have the kind of specificity and detail illustrated by the following example:

INTRODUCTION TO ETHICS
In this course we will read classical and contemporary writings on such matters as good and evil, relativism, happiness, virtue, egoism, moral education, abortion, and social policy. We will seek to answer, using critical reasoning, a series of questions about these issues as raised by the course readings. In addition, we will engage each other in sustained discussion of these issues. Learning is a complex process, and philosophical learning is no exception. There are cognitive, affective, and social dimensions, for learning involves not only knowledge and understanding, but also values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Listed are the outcomes a successful student will attain by the end of this course in ethics:

COGNITIVE (Knowledge and understanding)
1. Demonstrate knowledge of the views of some historically important moral philosophers (e.g., Plato, Aristotle, Kant, Nietzsche, Singer).
2. Demonstrate knowledge of the main concepts and theories of ethics (e.g., egoism, altruism, rights, duties, utilitarianism, Kantianism, virtue ethics).
3. Apply these concepts and theories to case studies and contemporary moral issues.
4. Articulate an understanding of connections between reason and feeling and between cultural and intellectual traditions.
5. Express conclusions with awareness of the degree to which these conclusions are supported by evidence.

AFFECTIVE (Skills)
6. Demonstrate imaginative, creative, and reflective abilities by articulating philosophical insights.
7. Present effectively in writing an extended argument on a topic of ethical importance.
8. Articulate counter-arguments to one's own position.
9. Ask questions to clarify problems further.

SOCIAL (Values)
10. Demonstrate openness and intellectual humility by approaching situations involving a conflict of views in a spirit of inquiry.
11. Identify and reflect on values through analysis of case studies in such areas as justice, abortion, and the impact of humans on the environment.
12. Reflect on one's intellectual and intuitive responses to issues concerning ethical values.
13. Demonstrate increasing awareness of the complexity of issues and of the necessity of examining issues from many different perspectives.

(Outcomes adapted from the Ontario Academic Course in Philosophy, Canada: http://www.edu.gov.on.ca/eng/document/curricul/philoso/philoso.html)

Notice how each of the outcomes specified here takes an active verb: demonstrate, apply, reflect, etc. Such a philosophy course description in the OA mold will go on to specify what assessment techniques will be used to measure whether students have in fact matured sufficiently along these cognitive, affective, and social dimensions. These might include exams (written and/or oral), papers, quizzes, group work, participation, and self-assessment.

(More information on OA techniques and expectations can be found at http://www.aacu.org/issues/assessment/index.cfm)

Concerns regarding OA

1. OA threatens to be an exercise in measuring what's easy, rather than a process of improving what philosophy instructors (and presumably even students) really care about. If philosophy courses and programs do satisfy the enormous pressure from various sources to find objective measures of learning outcomes, then there is a real danger that OA imperatives will create pressures to tailor the teaching of philosophy to things that admit of "before and after" measurement, to its serious detriment. So, for example, students who take philosophy courses dealing with different ways of thinking about such problems as the idea and existence of God, the relation of our minds to our bodies, the nature of truth, the conditions and limits of human knowledge, or the status of moral principles and concepts, should be more sophisticated in their thinking about these issues after taking the courses than they were at the outset. This should be discernible in both discussion and written work. It is only on the most superficial level of treatment of any such topics, however, that one can find specific matters admitting of before-and-after measurement (e.g., being able to identify, define and distinguish different arguments for the existence of God, conceptions of truth, types of knowledge, or different moral theories, or knowing who said what about them in the history of philosophy). And to make instruction in such matters the focus of philosophical education (in order to yield dramatic before-and-after results) would be to reduce it to a caricature of the development of any real sophistication in students with respect to these issues. The basic aim of education in philosophy is not and should not be primarily to impart information. Rather it is to help students learn to understand various kinds of deeply difficult intellectual problems, to interpret texts that address these problems, to analyze and criticize the arguments found in them, and to express themselves in ways that clarify and carry forward reflection upon them. The worry is that these kinds of abilities are not amenable (though others might be) to patterns of outcomes measurement typical of OA. It is not to be expected that student progress in philosophy can either be specified to a degree beyond what is already possible by means of an essay examination or a term paper, or given a purely quantitative expression. It is essential that those values inherent in and specific to the process of teaching and learning in philosophy not be lost. In short, the adoption of OA in philosophy might seem to undermine, rather than improve, the quality of instruction.

Yet those who use OA do not find this to be the case. First, the extent to which an outcome in philosophy is easy to measure seems to have little to do with the degree to which it is worthy of measurement. Learning outcomes in traditional symbolic logic courses are often in the "easy to measure" category but are certainly worth caring about. Second, there may not be such a large gap between the easy to measure and the difficult (some would say impossible) to measure outcomes. The sample list of thirteen desired learning outcomes for an ethics course can be used to illustrate this. At first glance, most would undoubtedly consider outcome 2 ("demonstrate knowledge of the main concepts and theories of ethics") to be amenable to "easy," before-and-after measurement, since it is content and technique specific. Two straightforward exams, a beginning of term exam and an end of term exam would seem to suffice. Contrast this learning outcome with outcome 12 ("reflect on one's intellectual and intuitive responses to issues concerning ethical values"). Surely most would initially register this as difficult (perhaps impossible) to measure in a way helpful to OA. But this is to exaggerate the differences between outcomes 2 and 12. For in either case a rote answer might be given; there are certainly instructors who discuss in class "different intellectual and intuitive responses concerning ethical values." The fact that a student holds an idea that others have held before her surely cannot be used against her when grading. In this sense, outcome 12 is similar to outcome 2, for how can the instructor tell in grading an exam or essay whether or not a student truly understands "the main concepts and theories of ethics" or is just recapitulating them on paper? This, however, suggests a solution that is often considered integral to the proper use of OA. Careful practitioners of OA use the student's self-assessment along with evidence from her performance in essays and exams to measure such things as attitudinal changes, for instance, a commitment to using philosophical methods and ethical concepts in resolving issues of personal and professional importance to the student. Such an approach greatly increases the chances of measuring accurately both outcomes 2 and 12. It also serves to avoid easily quantifiable measures of assessment that do not adequately reflect the complexity of student learning.

None of this is to say that all outcomes are or should be measured in similar ways. It is not obvious, for instance, that self-assessment is needed in logic. In fact, different outcomes may require different kinds of instruments of measurement. But this is surely not equivalent to saying that some outcomes are easy to measure and others practically impossible.

2. At the time of this writing, there does not seem to be any rigorous research comparing different kinds of instruments for observing and measuring learning outcomes peculiar to philosophy. Controlled studies, where, for example, the same philosophy instructor articulates outcomes and regularly performs assessments in one of her ethics classes and not in another ethics class, do not seem to exist. However, it appears that there is much anecdotal evidence that outcomes like the ones expressed above can be achieved and demonstrated in a wide variety of learning activities. Learning activities range from written exams administered throughout the term, class discussions and quizzes, questions solicited by the instructor, group work on pre-selected or limited topics, essay assignments graded with a departmental rubric, and student self-assessment. That being said, the APA sees a need for further empirical research into the usefulness of different kinds of assessment instruments for measuring the outcomes of concern to philosophy courses and programs.

Recommendations

Note the above use of the term "careful practitioners of OA." Certainly OA can be used in a careless and damaging fashion, for instance, where only one kind of measurement is used or where the outcomes are entirely along the cognitive dimension while ignoring the affective and social dimensions. OA must not be treated as an end in itself, but rather as one (albeit important) means for educational improvement. Educational values should guide not only what instructors choose to assess but also how they do so, and those values can be made clear to students through the methods of OA. Assessment should be an ongoing process and not episodic, especially for majors continuing beyond the term. In the spirit of continual development, student progress toward the intended outcomes should be monitored. And, importantly, the assessment process itself should be regularly assessed. OA must also take into account the peculiarities of each discipline to which it is applied. The APA calls upon administrators to recognize that philosophy is fundamentally a matter of the cultivation and employment of analytic, interpretive, normative and critical abilities. Learning outcomes and assessment methods must be devised accordingly. It is recommended that special consideration should be given to the means of assessment already in place at an institution.

The APA recognizes the interest of public agencies in establishing ways of assessing the success of colleges and universities in carrying out their educational missions, and accomplishing their objectives. It seems possible to create assessment instruments for both students and programs that satisfy administrators yet at the same time avoid easy measures that do not sufficiently mirror the complexity and special nature of student learning in philosophy. The concept of Outcomes Assessment may be of some help in achieving these ends, but it must be applied carefully.

(2006 revision of a statement originally published in Proceedings and Addresses of the American Philosophical Association, Vol. 69, No. 2, pp. 94-95.)

6/16/06