«The Plea Our accrediting association liaison was on campus to inform our faculty, in no uncertain terms, that we needed more direct assessment of ...»
A Simple and (almost) painless way to document assessment of student learning.
(2014), Assessment Annotations: A Simple and (almost) painless way to document assessment of
student learning. Assessment Update, 26: 1,2,14–16. doi: 10.1002/au.20007
Our accrediting association liaison was on campus to inform our faculty, in no uncertain terms,
that we needed more direct assessment of student learning. Right in the middle of the meeting, one
member of the faculty from the other side of the room turned towards me and said “Just tell us what to do and we will do it!” The frustration was evident in his voice and on his face. Most of the heads in the room were now turned towards me, many nodding in agreement with their wrought colleague. My first thought was “FINALLY, faculty buy-in!” But not exactly the way we would have preferred.
The Need Three years earlier, as a result of our self-study process, we were asked to produce a progress letter “documenting implementation of an organized and sustained process to assess the achievement of student learning outcomes.” (Sound familiar to anyone?) The progress letter was accepted but with the stipulation of “a prompt staff visit to discuss the Commission’s expectations” regarding further direct evidence of student outcomes for our next Periodic Review Report. After some email correspondence with our liaison, she indicated the ultimate purpose of her visit. “I really have only one message, and that’s to the faculty: Grades are not direct evidence of student learning!” Despite the three workshops we conducted with the faculty, the numerous resources made available to them, and our own attempts to instill in them a sense of urgency in this matter, we were still struggling with documenting the assessment of student learning without using grades as the main criteria. We were confident that embedded classroom assessment techniques were being used by most of the faculty, but documenting the results of those activities was still slipping through our grasp.
The Response My initial response to the passionate plea from the faculty member in the meeting was to remind him and the others that they are the content experts and should know best how to assess student learning in their disciplines. But in the days that followed, I realized it was probably not so much a matter of the actual assessment itself, it was a matter of documenting the various types of classroom assessments being conducted. A simple and painless way of providing documentation of assessment activities might be the answer.
Our desire was to come up with a brief electronic form that could be easily completed immediately after a class session which included an embedded assessment activity. We entitled the form an Assessment Annotation.
The Form The form is short and deceptively simple (see a completed example in Figure 1). It starts by asking for the usual demographic information. Drop-down menus are provided for some of this information. The selection of the appropriate program is important for two reasons; (1) it will populate the form with the appropriate sets of learning outcomes, and (2) it will determine which program coordinator will receive a copy of the completed form. The rationale for forwarding a copy of the form to the program coordinator is three-fold. First, it guarantees accountability. If the instructor indicates some action steps need to be taken as a result of this assessment, then submitting this form to an immediate supervisor provides the first step in making sure that someone is expecting follow-through.
Secondly, this form is not to be simply filed away, but the documentation of student learning assessment is to be used by the department chair as evidence of the fulfillment of program and department objectives, or at least that actions are being taken to improve the achievement of those objectives. Thirdly, as the department chair reviews the assessment annotations submitted, examples of best practices can be recognized and possibly rewarded.
The form asks for a brief description of the Assessment Activity including any appropriate documents. The instructor describes the assessment technique utilized, whether it be a one-minute paper, muddiest point, RSQC2, double-entry journals, or any other embedded assessment activity.1 In our Figure 1 example, students in the course Organizational Social Work were to use a local community newspaper to locate informal social opportunities which did not come with any stigma of being involved with social services.
Of course, the assessment activity should have some relevance to a course objective. So in the next section of the form instructors make that connection by identifying learning outcomes this activity assesses. They start by filling in the specific course objective this activity is meant to assess.
Then, through the use of the drop-down menus mentioned earlier, the instructor indicates to which higher level objectives this course objective is tied (program outcomes, general education outcomes, information literacy outcomes, core knowledge and skills, etc.).
The Results of Activity portion of the form allows the instructor to briefly describe how the activity went. Did the students understand the assessment activity? Did they actively participate? How many were able to complete the activity satisfactorily? What didn’t go well? What aspects of the activity could be improved?
Now that we have a description of the results of the activity, we come to the critical but often over-looked task of analyzing the results. If the assessment activity found that all students were meeting the desired outcome perfectly or at least to the desired level, then very little analysis is needed.
But if not all students evidenced the desire outcome, then an explanation should be sought.
The final Actions to be Taken section prompts the instructor to close the loop. Based upon the Analysis of Results, instructors are to briefly describe what steps need to be taken. Here is where the instructor is able to definitely show how assessment drives the instructional process. When a gap exists between the desired results and the actual results, changes need to be made to close that gap.
Action plans will usually be two-fold: one immediate and one future. First, when it is evident that some students have not grasped the objective in question, an action should take place which will provide them with an opportunity to fully achieve the learning outcome. Most of the time that will mean planning some additional activity for the next class period or later in the semester. Or it may mean assigning those specific students an additional assignment or suggesting some one-on-one time.
However, as we see in our Figure 1 example, it may be possible to incorporate an action right then and there, before the class period is even over, to make sure that all students have achieved the learning objective.
The future aspect of the Actions to be Taken portion of our form addresses changes that should be made so the next time this course is taught there will be a greater chance all students will achieve the learning outcome the first time around. Note how the instructor in our example did just that by making a note that in future offerings of this course she would provide the students with specific examples of informal versus formal resources along with additional fictitious case scenarios.
The Key Be brief! The secret to the success of Assessment Annotations is to be brief. Notice in the form that the word ‘briefly’ is used three times. We are not looking for a treatise. Ideally, instructors will complete the first part of the form during their preparation time prior to the class. Then, after the class is over, with the assessment activity and its results still fresh in their minds, they would immediately return to their office and spend a mere 15 minutes or so to record the results, do a little analysis, and then come up with one or more action plans. The key to making this system work is in being as brief as possible in describing the activity, the results, the analysis, and the action to be taken.
The Results Our social work instructor took the concept of doing Assessment Annotations and ran with it!
She turned in two almost immediately. She was excited about the fact that she finally had something tangible to work with in order to document the assessment of student learning in the classroom. Plus, the process of documenting her successes stimulated her to brainstorm additional classroom assessment activities.
Other faculty members jumped on board almost as quickly. Yet, others still need a little ‘encouragement.’ Therefore, the Vice President of Academic Affairs asked faculty members to submit at least one Assessment Annotation each semester. Some program coordinators are requiring even more. Specifically, the coordinators are asking instructors in their programs to submit Assessment Annotations which focus on the two or three programmatic outcomes being assessed that academic year.
When the electronic form is submitted, a copy also goes to our Office for Institutional Effectiveness. We are able to plug these evidences directly into our assessment database. Collections of Assessment Annotations tied to specific programmatic outcomes provide us with direct evidence that those learning outcomes are being met at the course level. These are then combined with other evidences, both direct and indirect, as part of our triangulation approach to documenting assessment.
A final copy of each annotation is sent to our Director of Teaching & Learning. She is then able to provide guidance and encouragement to individual faculty members as needed.
The assessment of student learning outcomes ultimately rests with faculty members, the content experts. While we can assist them in learning some effective means by which to assess student learning, there is no magic formula we can share with them in answer to their plea of “Just tell us what to do and we’ll do it!” Each program, each course, and each instructor is different. However, for us, this simple Assessment Annotation form has proven to be effective in helping them to document the assessment that is taking place in their classrooms, which in turn, helps to document the achievement of higher level outcomes and objectives as well.
_____ Angelo, Thomas A., and A. Patricia Cross, Classroom Assessment Techniques, second edition. John Wiley & Sons, 1993.
San Francisco, CA.
Dale L. Mort, Ph.D.
Associate Vice President Institutional Effectiveness Lancaster Bible College