Program Assessment I

Michelle D. Deardorff, Jackson State University
David M. Brodsky, University of Tennessee-Chattanooga
Russell Mayer, Merrimack College
Maria Rost Rublee, University of Tampa

The sessions in our track reflected the broader diversity of the discipline in how we addressed the issue of program assessment. Our members included four graduate students, three community college faculty, seven liberal arts college professors, and six state university faculty; our interests in participating in this track were as diverse. We expressed concerns ranging from how we assess skills, evaluate conceptual understanding and knowledge, and review general education courses to the specific problems surrounding program review. The participants explored various classroom and pedagogical tool assessment, means of evaluating the success of active learning techniques, the value of pre-test/post-test approach, and endeavored to find ways to engage large classes more compellingly. Despite, and maybe because, of our wide scope of interests, our discussions furthered the conversation on assessment that has evolved at the Teaching and Learning Conference over the last four years.

As our essay in the July edition of PS will demonstrate, we came up with three primary findings and several questions that need to continue to be addressed at future TLC conferences. First, we explored the significant issues arising from the Politics of Assessment on our campuses, in our accrediting agencies, and now with the federal and state governments. We recognize the very real disciplinary concerns and fears surrounding assessment: possible standardization that impacts both academic freedom and pedagogical approaches; the perpetual wasting of faculty time; and, the fact that the means of evaluation actually may determine the intellectual content and pedagogical approaches of a program. To assuage these fears departments and the discipline must be clear on why we assess our teaching and programs, ensure that there is adequate financial and temporal resources, guarantee that our departmental and course goals relate substantially to institutional goals, learn how to leverage our assessment plans for departmental and faculty gains, and find ways to motivate our students to engage and participate in assessment.

Secondly, we queried how are we Deciding What We Need to Assess. Such a determination should consider the following: assessment should begin with smaller elements of programs, be selective in what you assess, consider what faculty are interested in discovering, assess what can be embedded in courses and in current faculty tasks, seek partnerships for assessment both internal and external to the institution, and consider both independent and dependent variables in developing plans. We found it may be a better expenditure of time to assess skills, attitudes, and values as opposed to simply measuring the accumulation of knowledge. As the weekend progressed we began to wrestle with how the purpose of the assessment--is it for developmental purposes (formative assessment) or to draw conclusions (summative assessment--drives the means of assessment. We ended on some cautionary notes, declaring that a department or program should only assess elements on which the feedback loop can be closed; in other words, if you can’t or won’t change something, why assess it? We also noted it is essential for a department to carefully consider the costs and consequences of what it chooses to assess before beginning the process of assessment.

Finally, the track participants considered the necessity of Closing the Feedback Loop to the assessment process. This is the process of guaranteeing that the findings of assessment techniques actually impact the daily work of the department and its faculty. This section of our discussions reinforces the previous two conclusions. We recognize that this process may be very difficult and that it definitely requires deliberateness on the part of faculty. Questions of sustainability need to be considered and incremental changes may at times be the most logical to implement. Issues of formative changes, as opposed to summative changes, may be different. We realize that resources do matter, but we returned continually to two key points. This process requires intentionality and if you can’t close the loop, don’t assess it.

We found ourselves asking several reoccurring questions:

How do we assess critical thinking?
- What do we mean by critical thinking?
How do we respond to the push to assess knowledge from external pressures
- Are we measuring knowledge or conceptual understanding?
- What concepts and how do we measure them?
How can we effectively measure and use attitudinal data for assessment?
Is peer review appropriately valued as an assessment instrument?

We believe that as these conversations at the TLC continue to push the issue of assessment forward, we will be able to more effectively answer these questions.