Report


Home
Report
Project Members
Articles
Bibliography

Authentic Assessment:  Effects Upon Personal Goals

Timothy B. Giesbers and Bret Hamilton

 

Abstract

This research project compared high school (grades 9-12) students’ self-chosen goals with authentic assessments, to attempt to determine whether the use of authentic assessments (rather than traditional assessments methods) would help students achieve self-chosen goals.  Our findings indicate that they can, but that students’ perceptions of the assessments are a critical component in the achievement of the goals of any class.

An authentic assessment, alone, was found to be insufficient to engage students and encourage them to use the assessments to reach their goals.  There were three primary components to this insufficiency:  (1) Students generally focused on short-term goals; a final grade for a class is, to them, a long-term goal, (2) students needed help to create measurable, achievable goals and tracking their progress toward achieving them, and (3) though our assessments were authentic, they were not engaging enough to encourage students to participate with their full effort and track their own progress toward their goals.

     Keywords:  authentic assessment; student perception, teacher perception, goals, engagement

 

Introduction

        Mr. Hamilton recently earned a Bachelor’s degree in Health and Human Performance, and a Minor in Spanish from George Fox University.  Mr. Giesbers’ initial degree was a Bachelor’s degree in Computer Systems Engineering Technology.  After almost 20 years as a hardware and software engineer, he earned a Master of Business Administration degree and worked for a decade as a manager of software engineers. Mr. Hamilton will be endorsed to teach both physical education and Spanish while Mr. Giesbers will be endorsed for middle school mathematics and business.  Both men will be certified to teach at both the middle and high school levels.

        Both men are graduate students in the Master of Arts in Teaching program at George Fox University, expecting their degrees to be awarded on April 30, 2011.

        Mr. Hamilton and Mr. Giesbers are pre-service teachers performing full-time practicums at different schools.  Mr. Hamilton currently teaches three different physical education courses at the high school level.  One is a Strength and Conditioning course, which focuses primarily on weight training, flexibility, and explosive plyometric techniques.  The other two are Fitness courses, which have a broader focus than the strength training courses. They emphasize speed, agility, quickness, flexibility and stamina.  The assessments in these courses are fully authentic.

        Mr. Giesbers currently teaches four courses at the high school level; two Algebra Support classes, an Introduction to Accounting course and a Keyboarding course.  The data used in our study is from the Introduction to Accounting course, which uses authentic assessments throughout the course.

In our practicum experiences, we have each seen that teachers use a wide range of assessment techniques.  Of these, the concept of “authentic assessments” seemed the poorest understood and the least frequently used.  We wanted to see whether authentic assessments could supplement or replace existing assessment methods.  If they could do so, we also wanted to see whether they should do so.  To answer the latter question, we believed that the students’ input was necessary.  Our MAT courses and our Cooperating Teachers also made clear the fact that course material must be perceived by students to be relevant to them, be interesting to them, and allow them freedom to discover methods of learning which work for them.  We decided that students’ goals for a course would be an appropriate measure for our research; if students can reach their goals by using authentic assessment, then we would be able to answer our initial question:  Can, and if so should, authentic assessments replace or at least supplement existing assessments?

       To answer this question, we need to determine whether a correlation exists between personal goal setting and authentic assessments.  Additionally, we need to compare our classes to examine correlations between the disparate subjects of physical education and business to see whether our data answers the question consistently or answers the question for only one area.

       We began by wondering about the effect of different assessments methods upon student cheating.  Although this topic was interesting, and we have seen instances of cheating within our classes, we decided that our limited opportunities precluded the adoption of this topic.  For example, our time in our classes was limited by our MAT program and practicum.  Also, the methods for observing and documenting instances of cheating are, of necessity, somewhat vague and would be best performed by individuals with more classroom experience, and more practice with classroom management.  Also, “cheating” as a topic did not fit well with Mr. Hamilton’s Physical Education classes; generally, a student is participating, or not.  In an authentically assessed physical education course, students have limited opportunities to cheat via paperwork; the only way they can legitimately cheat is if they cheat themselves on their workouts. This simply was not the type of cheating we had initially desired to measure.  Finally, cheating is a “negative” topic and we wished to focus on something positive.  We decided to focus upon a positive aspect of the assessment/student equation; their goals vs. authentic assessment methods.  This topic lets us learn about our students, their goals, and about authentic assessments versus standard testing methods.

       The value of traditional assessment methods, such as multiple-choice tests and essays, are being re-evaluated given the current climate of dissatisfaction with the quality of secondary students' education and their corresponding lack of readiness for the "real world," whether that means college studies or employment.  Authentic assessment methodology is therefore becoming a popular research topic, since it is believed by many teachers, academics and researchers that examining students' knowledge in a realistic fashion will better assess their readiness for their future endeavors.

Authentic assessment is “a form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills” (Mueller, p. 1).

The basic idea behind authentic assessment is to have students use realistic situations and data in the class setting, and assess their learning based upon their manipulation of that data.  These assessments are, therefore, essentially formative in nature rather than summative, because the requirements generally lead to experiments, projects, and work that is more in-depth than a typical lecture/test model.  Authentic assessments can include many steps, and an instructor may assess the students' progress at any point or at multiple times during an assignment or project.  Authentic assessments may also be brief, such as a simple laboratory experiment, essay, or physical activity.

Traditional assessment is essentially summative in nature; instructors give tests, quizzes, or specific homework problems.  This work may not be a good indication of students' learning (or lack of learning), because students may pass or fail due to factors other than the level of knowledge they have learned.  For example, students may be able to pass a mathematics tests if they have learned the steps necessary to solve a particular type of problem, but they may not be able to apply that knowledge outside the limited scope of the classroom.  Or, they may fail the test, even though they have learned the topic well, because they do not feel well on the day of the test or they are simply very nervous when taking tests.

Mueller (2011, pp. 2-3) gives a simple, comparison of traditional versus authentic assessment techniques:

Traditional

 

Authentic

Selecting a Response

 

Performing a Task

Contrived

 

Real-life

Recall/Recognition

 

Construction/Application

Teacher-structured

 

Student-structured

Indirect Evidence

 

Direct Evidence

“Authentic assessment...encourages the integration of teaching, learning and assessing” (Mueller, 2011, p. 1). If done well, students need never feel the stress of an assessment, and will have multiple opportunities to show genuine progress in their learning.

        Although authentic assessments have been well-studied, we found no research that directly studied the effects of authentic assessment upon students’ goals.  We want to know what the benefits are, to students’ personal goals in high school courses, when authentic assessments are used.

 

Literature Review

        This study focuses on students’ goals and whether authentic assessment helps students meet those goals.  Existing literature, however, does not mention this specific point.  While the literature often discusses student involvement, students’ feelings about the assessments, and students’ academic responses to the assessments, no studies address the specific combination of student goals and authentic assessments.  The only definition found which addressed students’ feedback was written by Chung & Behan (2010):

 

Authentic assessment exercises are similar to real-world tasks that would be expected by a professional.  An authentic assessment rehearses students for their future roles, whether as scientists or informed citizens.  Ideally, students will become self-directed learners, capable of reflection and self-critique and able to give constructive criticism to others.  This requires faculty to behave as coaches, to provide framework for learning, feedback, and opportunity for revision.  Students should be involved in improving the assessment, and their input can be collected by survey (p. 24).

 

Available literature addressed three key factors which were directly relevant to this study:

1.  Why are/should authentic assessments be used, and what are they?

2.  How are authentic assessments implemented?

3.  What are students’ reactions to, and opinions of, authentic assessment?

        These factors were important to this study because they enabled the authors to create and use authentic assessments which were consistent with current educational practices, gave a basis of comparison between this study’s assessments and its implementation and other studies’ assessments, and helped the authors differentiate between assessments which were genuinely authentic versus those which merely seemed authentic.

Why are/should authentic assessments be used, and what are they?

        Traditional testing methods, both summative and formative, have become recognized as tools with limited usefulness (DeCastro-Ambrosetti & Cho, 2005, DeMartino & Casteneda, 2007, Suurtamm, 2004).  Although useful and valid for fact-checking, these methods provide little or no insight into students’ acquisition of “soft” skills such as time management, the ability to work in groups, thinking creatively, and learning to be self-learners.  Authentic assessments, with their ability to meld facts with creativity, provide a way to meet both sets of needs and therefore have been extensively studied and have become popular throughout the education community.

        “’The future workforce is here – and it is woefully ill-prepared for the demands of today’s (and tomorrow’s) workplace’ (DiMartino & Casteneda, 2007, p. 38).  This was the stark conclusion of a study conducted by the partnership for 21st Century Skills and three other organizations that surveyed 400 employers across the United States about the workforce readiness of recent high school and college graduates.  The respondents indicated that the skills new job entrants most need for success in the workplace – oral and written communication, time management, critical thinking, problem solving, personal accountability, and the ability to work effectively with others – are the areas in which graduates are the least prepared” (DiMartino & Castaneda, 2007, p. 42).

        The review of literature provides many definitions for “authentic assessment,” but the basic idea is clear:  Students are assigned work which closely imitates real-world activities, and provides them with opportunities to perform their own research or investigations, encourages group work, and encourages the development of the “soft” skills which are so desired by employers.  Other reasons for authentic assessments include:

  • Students “felt more relaxed with an authentic assessment activity because it alleviated the stress of a paper-and-pencil test, where resources or communication are prohibited” (Suurtamm, 2004, p. 507).

  • “Assessment and instruction are intrinsically linked.  The teachers’ need to use authentic assessment emerged as their instructional practices moved from a more traditional style to the use of such things as contextual problems, cooperative learning, and an emphasis on communication” (Suurtamm, 2004, p. 505).

  • “Characteristics of (today’s) students are that they are team oriented, confident, structured, and technologically savvy; work well together in groups; and like a customized, flexible, fun class” (Chung & Behan, 2010, p. 28).

  • “(With authentic assessment, performance assessment, etc.)…the goal of assessment is the acquisition of higher-order thinking processes and competencies instead of factual knowledge and basic skills” (Gulikers, Bastiaens & Kirschner, 2004, p. 67).

  • “A major advantage of having students work together in a group-inquiry-based project is that it helps students learn from themselves and from each other.  …it helps them develop into self-learners.  The instructor’s role here is to facilitate students as they develop their expertise in critical thinking, and not to over-instruct.  The instructor provides guidelines and technical specifications for framing and presenting the work” (Chung & Behan, 2010, p. 28).

  • “…the variety of projects (generated by students in response to general guidelines) spoke to incorporation of higher order thinking skills by students.  This project allowed for students to utilize their individual learning styles which they came to realize were strengths” (DeCastro-Ambrosetti & Cho, 2005, p. 59).

How are authentic assessments implemented?

As Mueller (2011) stated, authentic assessment asks students to “…perform real-world tasks that demonstrate meaningful application of essential knowledge and skills” (p. 1).  In a physical education environment, assessing student performance is very simple because, for example, running a mile in a P.E. class is essentially the same as running a mile in a non-academic environment. The translation between performing skills in a P.E. class and performing them elsewhere is seamless.

        Authentic assessments have other attributes, some of which could perhaps constitute “best practices” for using authentic assessments.  For example, some researchers recommend the use of rubrics (DeCastro-Ambrosetti & Cho, 2005, DiMartino & Castaneda , 2007, Suurtaam, 2004, and Chung & Behan, 2010).  Others point out the benefits of being able to scaffold and structure authentic (especially project-based) assessments (Meyers & Nulty, 2009, DeCastro-Ambrosetti & Cho, 2005). The ability to create “higher-order thinking processes,” (Gulikers, Bastiaens & Kirschner, 2004, p. 67) as opposed to the repetition of basic skills, is also frequently discussed by researchers (Meyers & Nulty, Chung & Behan, DeCastro-Ambrosetti & Cho).

There is a large amount of literature describing authentic assessments, their definition, use, pedagogy, rubrics and successes.  However, we found no literature tying students’ goals to authentic assessments. Chung & Behan (2010) noted that “…we did not formally assess the ways in which students benefited from this process (communication, organization skills, critical thinking, or project management)” (p. 26).

        Student feedback was, we assume, a component of the many classrooms and students who were party to these studies, but we found that the literature does not address student feedback as a primary focus of the studies.  Instead, although the attitudes and feelings of students were often mentioned (DeCastro-Ambrosetti & Cho, 2005, Thompson, 2009, DiMartino & Castaneda, 2007, Chung & Behan, 2010), the focus was upon teachers’ ability to assess their students.

Feedback from assessments, as well as the studies, was intended more for the teachers’ benefit rather than creating feedback for students.  In these studies, students benefited relatively passively, as teachers learned about their new assessments’ successes and failures and therefore redesigned the assessments and/or lessons.  Success, for students, is gauged by the teachers’ perceptions of students’ learning at higher levels of thinking, learning “soft” skills such as organization and planning, thinking creatively, or learning to be self-learners or learn in groups.

Scoring and grading were rarely discussed within these studies.  If they were, items such as feedback within proficiency grading would have been part of the literature.  Instead, we found this information absent and decided to study it ourselves.

What are students’ reactions to, and opinions of, authentic assessment?

        Although students’ goals and direct student feedback have not been a part of the literature, students’ feedback regarding authentic assessments, in general, has certainly been noted.  Such feedback has been positive and enthusiastic (DeCastro-Ambrosetti & Cho, 2005, Chung & Behan, 2010, Thompson, 2009).

        Perhaps one of the most valuable common opinions is in regard to students’ perception of teachers’ interest in students’ opinions.  When students can, within some general guidelines (such as a rubric), create their own project and have teachers assess their work with enthusiasm, and especially modify their plans according to students’ feedback and reactions, students feel valued as a part of the process, rather than its subjects (Thompson, 2009).  Students who felt that they were treated respectfully, and who felt their opinions were important, improved their attitudes and behavior and achieved a much more significant attitude of “buy-in” to their class work. Students who feel that their teachers express vulnerability (e.g. admitting that they did not know all the answers) and a willingness to change are much more likely to participate actively in their own learning (Thompson, 2009, Chung & Behan, 2010).

Gulikers, Bastiaens, Kirschner & Kester (2008) noted that “for authentic assessments to reach their potential in positively stimulating student learning and better preparing them for the labor market, it is imperative that students perceive their assessments as authentic” (p. 501). Students who knew that their in-class work truly represented the real world were much more interested and invested in their projects.  This was particularly true when the students were allowed to do research on genuine issues, express their opinions (especially when they were in opposition to the teacher’s opinions), visit and research relevant sites, interview other people, and choose their own projects and their own methods for researching and completing their projects.

        67% to 89% of students felt that project-based authentic learning helped them in the areas of evaluating others’ work, their ability to “envision a project from start to finish,” communicating within their group, and organizational skills (Chung & Behan, 2010, p. 26).  Working in groups, particularly when all group members are enthusiastic and fully engaged in the process, clearly helped students develop “soft” skills in the areas which are, as discussed previously, important to employers and others—communication skills, working in groups, time management, planning a project, and preparing a report (or other method of disseminating information).

 

Data Collection and Methodology

Our objective was to determine the benefits of combining personal goal setting with authentic assessment in high school Business/Accounting and Physical Education courses.  A secondary goal was to compare the results of the Accounting vs. the Physical Education courses in order to determine whether there are correlations between the two data sets.  To complete our research, we obviously needed to implement the use of authentic assessments within our practicums.  Therefore, we read the literature with an eye toward gleaning this sort of “how-to” information.

         One common theme in the literature, though rarely stated directly, is that “…assessment can and should take the central role in curriculum design because it is one of the first things students look at and because it defines the curriculum for them. Consequently, assessment drives activities that students engage in” (Meyers & Nulty, 2009, p. 574). Although our curriculums were largely defined for us by our Cooperating Teachers, we were each allowed considerable latitude in developing additional features for our Action Research project.  Mr. Hamilton, in particular, built his classes around students’ goals, making sure that students had a chance, weekly, to show improvement and progress toward their goals.  Mr. Giesbers’ assessments were primarily simulations used by the Cooperating Teacher.  Mr. Giesbers also continued to circulate through the class during students’ work time and gave formative assessments during direct instruction, and both of these activities provided good information about students’ progress in addition to their performance on the work projects and tests (all of which were essentially accounting simulations).

         Suurtamm (2004) described how teachers felt about their subject matter—that teachers practicing authentic assessments felt that they needed “to be very confident and knowledgeable about” their subject in order to “provide suitable prompts to students as they move through investigations and explorations” (p. 504). We also perceived this need.  In Mr. Hamilton’s case, he used his extensive knowledge of the human anatomy, muscular system, cardiorespiratory system, weight training technique, nutrition and motivational skills to teach his students all of the necessary strategies they needed to know in order to improve and, ultimately, succeed in meeting their goals in their Strength and Conditioning courses.  Mr. Giesbers supplemented his existing knowledge by working through all of the student’s work prior to beginning his practicum.

DeCastro-Ambrosetti & Cho (2005) strongly recommended that teachers provide a rubric for authentic assessments.  “Rubrics establish a clear channel of communication between teachers and their students. It takes the guesswork out of the assessment process on both the teachers’ and students’ parts” (p. 60).  Though we did not write rubrics, we did make our expectations clear, we communicated them regularly and frequently, and the scoring methodology was well-known to the students.

         Authentic assessment appears as a catch-all phrase, one that means different things to each teacher and student.  Frey & Schmitt (2007) noted this explicitly:  “…it is the assessors who should determine whether a given assessment is authentic, using the criteria that seem most crucial to them.  Criteria of authenticity could include, among other aspects, the nature of the stimuli, the complexity of the task, conditions, resources, consequences, and whether the specific tasks of activities are determined by the student or the assessor” (p. 410).  Because we have taken the time and effort to know our subjects well—by which we mean the technical material, its use in the “real world,” and the students—we know that our assessments are genuine and authentic, and this gives us confidence in our results.

         We each derived questionnaires which students completed, in order to learn their goals for our classes.  We focused on trying to get students to identify specific, measurable goals.  For example, when Mr. Hamilton received general statements such as “I want to have more stamina,” he asked students to clarify and focus these statements so that they could be qualitatively and quantitatively measurable for data collection purposes. For example, “I want to have more stamina” turned in to “I want to increase my stamina so that I can successfully complete a one-mile run in under seven minutes and 30 seconds.”

         The study period was February 1 through March 30.  We each collected data from our students’ performances on the authentic assessments which were part of our classes.  The data gathering methods used for our project are described below in two sections; one for the Physical Education classes and one for the Introduction to Accounting class.  The plan and methods were submitted to George Fox University for Human Subject Review approval (see Appendix E).

 

Data Gathering – Mr. Hamilton’s Physical Education Classes

Mr. Hamilton’s practicum was at a high school in northern Oregon located about 30 minutes southwest of Portland.  This school has approximately 1,700 students enrolled. Roughly 75 percent of the student population are Caucasian, 25 percent are Hispanic, with the remaining demographics, such as Asian/Pacific Islander, constituting only fractions of a percent. These demographics were accurately represented in all of Mr. Hamilton’s Fitness and Strength and Conditioning and classes.  The students in the physical education classes ranged from grades 9-12.  Students’ academic backgrounds varied considerably in terms of overall academic performance. However, the one common tie that bonded the groups was that all of them had the goal of becoming stronger, better athletes.

The Strength and Conditioning courses were taught primarily in the school’s weight room, which is one of the best-equipped in Oregon.  The gym was remodeled in the late 1990s, and the equipment remains in immaculate condition despite its growing age.  The weight room is quite spacious, with enough room for four power cleans boxes, four bench press racks, four incline press racks, four squat racks, two sets of dumbbells ranging from 5 to 100 pounds, as well as multiple auxiliary lifts, such as two Power Runner machines, back extension, leg press, hack squat, in addition to various other machines.  As a whole, there were more than enough lifts and machines to accommodate 50 students comfortably.

In addition to the weight room, Mr. Hamilton had access to two separate gymnasiums.  The first was the Red gym—the larger of the two. It was considered the main gym where school assemblies, basketball games, and other P.E. classes all took place.  The Strength and Conditioning class was occasionally rotated out of the weight room and into the Red gym for a short shot of badminton or dodge ball at the end of a period if time allowed.  The smaller White gymnasium could also be used for various games and activities at the end of a period.

Mr. Hamilton initially began his action research in his Fitness 2 class. This class of 38 students was open to sophomores through seniors. The class focused on a various subjects within the physical education realm, meaning that for the students, variety was one of its strongest draws. However, for research purposes, this made it more difficult for Mr. Hamilton to draw meaningful conclusions between authentic assessment and student goal setting. Student goals ultimately reflected the variety of the class because they ranged from goals dealing with strength gains, speed gains, weight loss—all of which were potentially measurable in the class, but were made more difficult to do by the lack of consistency in the class curriculum. This made the starting and ending conditions, as well as the measurability of achievements almost impossible to record. 

Thus, Mr. Hamilton decided to re-focus upon his Strength and Conditioning class, which was offered to students in all grade levels. In order to take the class, though, students must have already taken and passed Fitness 1, a freshman level class.  Strength and Conditioning classes primarily focused on lifting weights and plyometric explosion activities.  To prepare his students for success in meeting their goals, Mr. Hamilton first primed his students with a brief questionnaire that dealt with student experiences with goal setting (see Fitness 2 – Initial Goal Analysis in Appendix A). This data had no bearing upon student performance, but it served well to give Mr. Hamilton a better idea on past student experiences with authentic assessment type grading.  The general consensus from these questionnaires suggested that the majority of the individuals who participated in the study agreed that a concrete, objective way of grading was preferable over one that allowed for more subjectivity.  Every student also suggested that the creation and personalization of goals held more weight and accountability than if those goals were created by the teacher. The results of this questionnaire prepared the students to create their own goals and to be assessed in a way that was just, objective, and ultimately, authentic in nature.

        Mr. Hamilton then taught a mini-lesson on how to set quality goals. He walked them through, step by step, on how to create a goal based upon the “S.M.A.R.T. principle” (Quinn, 2004, p. 1) (Specific, Measurable, Attainable, Relevant, and Timely). To help prevent the problem with the goals in the Fitness 2 class, the students were asked to create goals that not only aligned with the SMART principle but could easily be measured for progress on a weekly basis. Upon completing their initial goal sheet, “Adapted Weekly Goal Setting Template” was used for subsequent weeks. This adapted worksheet contained two additional sections, one for planning how to best reach the goal and the other for reflecting on the goal after completion or failure in meeting it.

All student goals were related specifically to the Strength and Conditioning class, meaning that they had to be something that students could achieve within the confines of the Strength and Conditioning class setting only. For example, a goal of “I want to get A’s in all of my classes” would have been an excellent goal, but it would also have been considered inappropriate for the Strength and Conditioning class. A more appropriate goal would have been “I want to increase my four core lift maxes by 20 percent at the end of the term.”  After setting their initial goals, students were asked, on a bi-weekly basis, to formulate specific, process-oriented short term goals which were intended to help students stay focused on achieving their long term goals. These short term goal sheets had blank spaces for students to write in their desired repetition and weight amount for each of their core lifts.  Students then answered three prompts that dealt with their preparation for meeting the goal.  These prompts—one for how best to prepare in terms of hydration, nutrition, and rest, respectively—helped keep students accountable for their actions outside of class, thus better helping them to meet their goals.  The final section of the goal sheet functioned as a review.  At the end of each week, students completed this section of the goal sheet to state what they actually lifted in each of their core lifts as well as their prospective reasons why or why not they met their goal.

        Although students created their initial long term goal in the beginning of February and consistently set new goals on a weekly basis, a total of three data collections were used for research purposes.  The initial data collection came in the form of the questionnaire and a long term goal. The second data collection came midway through the research process, and the third came at the end. It was fitting that the final data collection fell on a week when students performed maxes on all of their lifts, meaning that it was very easy to measure whether or not students had been able to meet their goals from the beginning of the data collection process.

 

Data Gathering – Mr. Giesbers’ Introduction to Accounting Class

Mr. Giesbers prepared for his class by working through all eight of the student work packets (one for each chapter of the text), eight tests and two culminating projects in the Introduction to Accounting course, and found that this was a very effective way to learn the material so that he could answer students’ questions quickly and correctly.  Mr. Giesbers also created computerized copies of the tests for chapters 3-8 and both of the large, post-chapter-8 projects; in addition to helping Mr. Giesbers learn the material, he was able to provide CDs to students who chose to use them, and to create copies of the tests which were of higher copy quality than those being used (copies were sometimes too dark, where numbers were expected to be written).  Mr. Giesbers’ Cooperating Teacher asked for copies of all of these files (there were a total of 28 documents, including teacher’s answer keys).  Mr. Giesbers and his Cooperating Teacher found that Mr. Giesbers’ life experiences and additional research also had the benefit of projecting authority in the subject, because students readily perceived that they the material thoroughly.

        Mr. Giesbers’ data consisted of a form which asked students to define Accounting terms, plus the tests and work packets for the material in class (the work packets and tests are all simulations of accounting problems one might experience in a real business).

Students’ goals were self-chosen, without adult guidance, at the beginning of Mr. Giesbers’ work sample period; these primarily focused on the students’ grades, but also often described their desire for basic financial information.  The latter goals showed that students didn’t understand the nature of the course they’d chosen, but a few also chose the class because a parent worked in the accounting profession and the students wanted to see whether the profession was a good opportunity for them.  Students’ goals are detailed in Appendix D.  At the end of the research period, the goals were used to gauge students’ achievements vs. their performance on the assessments.  The students in this elective class were an interesting mix; approximately half chose the class due to their own interests, and the other half were placed in the class by the school administration simply because it fit their schedule and the class was not considered to be too difficult.

The definitions paper was a one-time task, and there were eight separate chapters (and therefore eight separate work packets and tests) comprising the class, plus a “reinforcement” simulation which combined chapters 5-8 and a final business simulation which tied in concepts from all eight chapters.

In addition to the realistic, authentic assignments and accounting simulations used in the course, the students were also asked to complete a two-page survey of their knowledge of Accounting topics and definitions, to see whether this knowledge is retained throughout the course’s term (see Appendix B).

 

Results

        Before reviewing the data, we reacquainted ourselves with two key findings from our literature review.  First, we remembered that “it is assessors who should determine whether a given assessment is authentic, using the criteria that seem most crucial to them” (Frey & Schmitt, 2007, p. 410).  For this reason, data from the Physical Education and Accounting classes were interpreted by their teachers; Mr. Hamilton and Mr. Giesbers, respectively.  Second, we recognized, during the gathering of our data, another key point:  “…for authentic assessments to reach their potential in positively stimulating student learning and better preparing them for the labour market, it is imperative that students perceive their assessments as authentic” (Bulikers, Bastiaens, Kirschner & Kester, 2008, p. 401, italics in the original).   As we reviewed our data, we found that both of these statements are accurate and critical to the findings of any study of authentic assessment.

 

Physical Education Data

Eighteen students participated in the research study in Mr. Hamilton’s Strength and Conditioning course.  These students created their initial goals, all of which were measurable through the use of authentic assessments.  During the research process, students filled out a goal sheet to provide Mr. Hamilton with a progress report regarding their progress toward meeting goals, and to help hold students accountable to meet their long-term fitness goals.  During the final week of the research period, students were asked to analyze their goal performance by completing a final goal analysis.  Students exceeded, met, or did not meet their goals.  Students’ goal results were correlated with attendance rates and their grade in the class.  The results are shown in the graph and table, below.  Detailed results for each student are listed in Appendix B.

 

Student

Goal Met?
1 = No
2 = Yes
3 = Exceeded

Grade %

Attendance
0.5 = Poor
0.8 = Good

JA

2

100%

0.8

AB

1

85%

0.5

AC

3

100%

0.8

KD

1

90%

0.5

RG

2

95%

0.5

AM

2

100%

0.8

BP

2

100%

0.8

TP

2

100%

0.8

KP

1

80%

0.5

AR

2

100%

0.8

JR

1

90%

0.5

RS

2

100%

0.8

ES1

1

85%

0.5

ES2

2

100%

0.8

MS

2

100%

0.8

AS

2

100%

0.8

DS

2

95%

0.8

JV

2

100%

0.8

The data shows a strong correlation between student grades and whether they met their goals (meeting these goals was not part of the student’s grades for the course).  The five students who did not meet their goals had the weakest grades in the class, and these five students had statistically poorer attendance when compared to the students who met or exceeded their goals.

The weight room provided the ideal environment in which to authentically assess students.  Students were able to set personal performance goals and then work, under supervision and with help, to achieve them.  The assessments were authentic because students were able to perform exercises in the same way they would be performed in, for example, a commercial gym or as a member of a professional sports team.

Poor attendance was the largest factor for those students who did not meet their goal.  Consistent training is important when striving to gain strength in weight training, and a lack of consistency destroys one’s ability to maintain gains.  The body loses strength and mass at an exponentially greater rate than it can gain strength; therefore, each time a student missed an opportunity to lift they were hurting their own chances of reaching their goals. The students who remained consistent had less difficulty achieving their objective.

Other than attendance, student grades were calculated by wearing the proper clothing and footwear, participation, accurate recording of weight lifting manuals, and overall effort.  Student effort was the only category that allowed for subjectivity.  However, the students who did not follow directions, talked, rested excessively, and took shortcuts on their workouts were the ones who also struggled with meeting their goals.  Student KD was a prime example of this.  He had excellent strength to begin with, but he struggled to meet his goals because he was consistently off task.  Despite Mr. Hamilton’s reminders, KD still cheated himself by not finishing his entire workout on a consistent basis.

As noted above, Appendix B contains detailed information for each student in this study.

 

Introduction to Accounting Data

Students in this class could be divided into two types; those who took the class to fulfill their own interest and those who took the class because they had no choice (usually due to the lack of other available electives during this period).  Students’ achievements – good and poor, respectively – were generally divided among these lines and according to their GPA.  This is shown clearly in the graph below:

        The graph shows that the relationship between students meeting their own goals, their GPA, and their final grade in this class are direct relationships.  The sole exception is student AD, who was absent for 10 days during the course, hospitalized once, and diagnosed with a serious illness; though allowances were made for the student (“forgiving” some work and focusing on other material), the student was unmotivated to continue the class (the student had also previously indicated that the class was “boring,” so the student’s motivation was low initially).

Essentially, those students who were unmotivated remained unmotivated by the authentic assessments or the promise of a good grade, and received poor grades due to their inactivity.  This was clearly partially an expression of the levels of their motivation and personal discipline, as well as their enjoyment of the subject matter.

The Oregon Skill Set goals were communicated to students, but students did not refer to them during the remainder of the class.  Those who performed well on the assessments achieved these skills, and the other students did not.  Though these are certainly authentic goals (relevant to many professions and practices, as well as required by the state), they were not relevant to the students.

Students were generally weak in the area of checking their work before submitting it for scoring.  In this course the standard practice was to have the answer key on the teacher’s desk, available to all students (except for tests and the final project).  With the answers available, most students finished their work and came to the desk to check their work immediately.  “Checking for errors” became an exercise in detecting the errors, erasing them, and writing in the correct values.

There were several other ongoing issues which were a part of the original set-up of the class, for example:

  • Work was often turned in late (all work was given full credit regardless of when it was turned in, so there was no incentive to complete work on time).

  • Tests taken late were also given full credit; this tended to encourage absenteeism and cheating.

  • Absenteeism was high, particularly for several of the unmotivated students.

  • As noted above, students rarely checked their own work for errors in the fashion of an accountant.

  • Some students attempted to cheat by waiting for students to throw away their graded work, then retrieve the papers from the trash and copy them (we ceased returning students’ papers, but reported their grades to them instead).

         Though all of the above factors are relevant, the most important finding was clear:  the assessments for this class, though authentic, failed to garner the students’ attention and failed to inspire them to perform the work.

Although the assessments used were authentic assessments – accurately representing work an accounting intern might perform, for example – it was essentially “paperwork,” and therefore not likely to induce students to fully engage with the material.  It seems obvious that an accounting class which is more dynamic, perhaps requiring group participation and role-playing activities, for example, and/or genuinely simulating some of the activities which generate accounting transactions, would have more fully engaged all of the students.  Such a class would also reduce or eliminate several other problems which existed during this class, mentioned above, such as absenteeism and work turned in late.

To truly assess the effectiveness of students’ goals vs. authentic assessments, this study would need to be repeated using authentic assessments which engaged and interested students.

 

Conclusions

        We have drawn several conclusions from our research and our practicums:

  1. Authentic assessments must be more than simply an example of assessments using a real-world scenario.

  2. Students must be interested in, and enthused by, their subject and their teachers.

  3. Students can develop measurable, attainable goals for themselves.

  4. Teachers must help students track their goals and guide them with ways to achieve those goals.

  5. Authentic assessments can help students achieve their goals, if those assessments are ones which create enthusiasm and participation among students.

         Assessments that are authentic but uninteresting to students will not help students achieve their goals.  Mere relevance to the real world is not sufficient; relevance to the students, including components which interest and excite them, is required.  Similarly, students can achieve their goals, but need guidance to select goals which will be measurable and attainable within the scope and time period of the course.  If students’ goals are too generalized or are unattainable in the short term, students will be disinterested and, again, the teacher’s tool of “goals” will not be useful within the course.

Students must be guided, regularly, to focus upon their goals within the scope of a course.  A minimum of weekly goal-setting and goal-checking activities must be performed, or the student will see no value in the exercise and will forget to focus on his/her goals.

Authentic assessments require both students and teachers to agree (not necessarily verbally or with deliberation) that the assessments are valid expressions of the course material, have relevance in students’ lives, and are able to be sub-divided into portions which allow teachers to evaluate students’ progress regularly, and to give students feedback regarding their progress (e.g., a month-long project would require a teacher to give student feedback on a weekly basis, at least).

We have learned that our future teaching must be focused upon drawing students into a class, creating enthusiasm among the students, and making them see the benefits of participation.  Part of our methodology can be the use of authentic assessments, but those assessments must be created with the students’ interests and desire for personal relevance in mind.

 

Appendices

Appendix A - Data Gathering Documents used in Physical Education Classes

Appendix B - Data Gathering Documents used in the Introduction to Accounting Class

Appendix C - Raw Data from the Physical Education Classes

Appendix D - Raw Data from the Introduction to Accounting Class

Appendix E - Human Subjects Approval Form

Appendix F - Authentic Assessments in the Introduction to Accounting Course


Home | Report | Project Members | Articles | Bibliography

 Copyright 2010-2011 by Bret Hamilton and Tim Giesbers.
For problems or questions regarding this Web site contact Tim Giesbers.
Last updated: 04/26/11.