Avoiding assessment pitfalls

What Are the Most Common Problems in Assessment Planning?

Selecting inappropriate assessment methods

Assessment activities are too time- and labor-intensive to be squandered in poorly designed plans. Faculty must scrutinize assessment proposals carefully from the standpoint of how well the results will provide evidence consistent with the department's mission and goals.

Selecting only one or two assessment measures

Relying on just one or two measures is bound to produce an incomplete picture of what students are accomplishing. Departments should consider the array of possibilities and then select the strategies that will capture their achievements and distinctiveness.

Limiting assessment strategies to productivity, viability measures

Some departments may opt to limit their strategies merely to archival data to reduce the intrusion of assessment in faculty lives. Graduation rates, enrollment figures, and faculty productivity measures simply will not contribute to the development of a vigorous culture of evidence.

Failing to interpret assessment data adequately

A common error involves departments' presenting assessment data as if the data stand on their own merit. In nearly all cases, appropriate context must be established if the department is to take full advantage of what they have achieved. Setting context provides for a more complete explanation of situational factors that should be considered in interpreting the results of assessment.

Failing to use assessment results to implement change

The primary purpose of assessment is promote continuous improvement. Well designed assessment strategies produce results that hold the key to strengthening the department. The department needs to commit to a careful review of the implications of what the data suggest about program improvement.

Failing to exploit positive results

Some departments file the results of their assessment activities upon completion. The faculty may expect that administrators and other stakeholders will be advocates for the department based on the department's reputation. Positive assessment results provide a great opportunity to remind stakeholders of the quality of the department.

Misusing assessment data

Assessment should emphasize the improvement of student learning. Data generated from the assessment of student learning should not be used for individual faculty evaluation. Administrators need to distinguish appropriate productivity/viability measures for faculty from those that assess student learning. Faculty need to be vigorous in protesting misappropriated data.

Emphasizing compliance with the process more than the results

Some departments demonstrate greater enthusiasm for enacting the assessment strategy as a means of giving evidence to the vigor their campus allegiance. One consequence may be paying less attention to the actual results of the assessment. In the best case, they will lose the opportunity to tout their achievements. In the worst case, they may neglect important feedback that should prompt change. Department members should evaluate the results purposefully from the standpoint of what directions the data suggest for improvement.

Getting swept away by winning the assessment "game"

The assessment challenge sometimes appeals to departments as a way of generating proof of their superiority on campus. Not only will such a competitive stance misdirect faculty energies, it will potentially alienate campus partners. Department members must concentrate on collecting data that will help them with the collective goal of improving the curriculum and the quality of student experience.

Making inappropriate comparisons within or across institutions

Some assessment strategies lend themselves to comparisons that may not be appropriate. For example, the use of standardized commercial tests offer performance norms that do not take into account the actual course preparation students will have had prior to the testing. Alternatively, departments may not emphasize in their own curriculum requirements some dimensions of the test. Comparisons of assessment results may mask significant differences in program philosophy, mission, and curriculum.

Adopting a defensive posture

Faculty can adopt multiple rationales for resisting involvement in assessment and express their resistance with attitudes and behaviors ranging from hostility through apathy. Assessment can be viewed as a threat to successful programs (status quo). Assessment activities that increase in relation to accreditation demands can be seen as an externally mandated, periodic bother. Assessment may be seen as "add on" work of little importance. Assessment may also be seen as a vehicle that leads to funding reallocation or program discontinuation.

Source: American Psychological Association Assessment CyberGuide (2009). http://www.apa.org/ed/governance/bea/assess.aspx