eMedici Voltaire ct
Facebook

Module: Psychology

Choose Another Module

Assessing Honours Psychology theses - how to avoid problems and pitfalls

Author(s):

Professor Anna Chur-Hansen
School of Psychology
University of Adelaide

Overview:

This is a case focussed on the assessment of Honours Psychology theses at the University of Adelaide.  After reading the following background information you will be asked to complete 4 Multiple Choice items.  If you are a Registered Psychologist you can count the time that you spend on this activity as Active CPD.

Diversity

The Honours thesis requires students to work with a high degree of independence on a research project. However, the independence in deciding upon the topic and the approach will range from students being expected by their supervisor to take a substantive role in shaping the research question and chosen methodology through to a more directive approach, where the student becomes part of a wider research project. Both scenarios are equally valid, and regardless, the objective is for the student to demonstrate competency in conducting research in psychology.

At the University of Adelaide School of Psychology there are a range of different project types – from conversation analysis and other qualitative approaches, through to experimental and mathematical methodologies.  All methodologies are equally valid.

Importance

The Honours thesis grade is extremely important to students’ future opportunities. An overall First or High Second Class (2A) Honours grade allows for entry into Masters programs and PhDs. Scholarships are usually awarded on the basis of the Honours grade – usually, but not exclusively, a First Class is necessary for Scholarships.

Students are highly selected for Honours and are high achievers. Their thesis will reflect this fact, with the majority gaining a First or High Second Class (2A) grade.

Because of the importance of the Honours thesis grade, which comprises 50% of the year’s work, it is essential that the assessment of the thesis is fair and transparent. The mark should reflect learning attributes, and not factors outside of the student’s control.

Criterion-based assessment versus norm-based assessment

In criterion-based assessment students are able to clearly identify the standards by which their performance is assessed. Thus, marks are assigned based on performance relative to these standards. The University of Adelaide Honours theses are marked according to 7 criteria on a 1-10 scale. These are:

1. Originality and/or independent thought

2. Mastery of the content area

3. The ability to develop clear arguments that justify the research aims and cogently discuss the extent to which the aims have been borne out in the findings

4. Clarity and quality of written expression including appropriate figures and tables

5. A sound research methodology

6. The ability to apply appropriate quantitative and/or qualitative data analysis

7. A critical awareness of the implications of the findings as well as the strengths and liEmitations of the study

In norm-referenced, or norm-based assessment the assessor marks by considering the thesis in relation to the performance of peers. This type of assessment is less transparent than criterion-based assessment. It is also problematic because it relies on the assessor’s knowledge of other theses and their experience in assessment. It allows for bias – so it is easier to be swayed by a topic area that appeals to the assessor, for example.

Global assessments

In assessment it is important to assess according to criteria. It is also important to consider the thesis as a whole. Thus, in assessment it is not appropriate to “add up” ratings to come to a final grade. Rather, the criteria are considered in relation to one another and to the thesis as an entity. There will likely be high consistency between the assessment of the 7 criteria and the global assessment. However, it is possible that despite the thesis rating highly on a number of criteria, the global assessment is less high. This will require the assessor to make a judgement.

Writing detailed feedback in relation to the criteria will assist in coming to the overall, global assessment. 

The role of experience in assessment

The role of experience in assessing Honours Psychology theses is related to making an overall, global judgement.  This is not the same as norm-referenced assessment.  Rather, this is related to stepping back and asking if the thesis is consistent with a First Class or Second Class or other, in relation to the criteria.  For assessors who are not experienced, it would be wise to ask the Honours Co-ordinator to provide exemplars of theses judged by experienced assessors to be outstanding (or otherwise).

Competency-based assessment

In competency-based assessment the question is not about whether a student has particular knowledge, but whether they have a particular competency – can they “do” something? An Honours thesis in psychology requires demonstrated competency in the ability to formulate, design, plan and execute research. So a global assessment question the marker should be asking is: “Has this student demonstrated an ability to conduct research in psychology?” The answer to this question is part of the assessor’s global assessment of the thesis.

Reliability and validity – trustworthiness and rigour

The written feedback in relation to the criteria assists in maintaining reliability and validity – or more correctly, trustworthiness and rigour. The assessor must be mindful of why they see aspects of the thesis as strengths or weaknesses and be able to articulate these. Similarly, another assessor must be able to clearly understand the reasoning behind decisions. The student also must be able to understand why they have achieved the grade they have been awarded.

As with any high stakes assessment, it is important that more than one assessment is made. At the University of Adelaide two independent markers provide their assessment. The markers are “independent” in that they are not supervisors of the project, and that they arrive at their initial recommended marks independently, without consultation. The two markers give a range as the first step (e.g. Low First Class 80-84). The two markers then confer and moderate their scores, arriving at a final grade. If they cannot arrive on an exact mark, they can agree on the range e.g. 80-84. The Honours Co-ordinator will then deliberate as to what the final grade should be (and this will generally be the midpoint of the range).

Where there is consistency between the two assessors, the process is fairly straightforward.

Where one marker has given a different range of scores – for example one arrives at 80-84 and the other gives 70-75, then discussion is required to ascertain whether there can be agreement. This might mean that the two assessors meet in the middle.

If there is no agreement, a third assessor is called. In this event, the third assessor, who acts as a moderator, reads the thesis in light of the comments and feedback provided by the two assessors and then makes the final recommendation.

Assessors are always graduates of Psychology, with an Honours degree of their own. Research shows that reliability in assessment of Honours theses is greater when the assessors are from the same discipline (Henderson et al., 2009). It is also important that the assessor has experience as a supervisor of Honours theses, as they thus better understand the scope and expectations for students at this level.

Biases in assessment

There is very little research on assessment of Honours theses. There is research on assessment of Masters and PhD theses and thus the following is based on that information. This is not an exhaustive list. The list is in no particular order. These are factors for individuals to consider, reflect upon and control for in the assessment of an Honours thesis.

*Influence of the student’s supervisor/supervisory panel

*Influence of the topic – either strong feeling in favour or against

*Influence of the topic – familiarity with it or little understanding of the area

*Influence of the methodology – either strongly for or against

*Rushing to complete the task without reading the thesis carefully

*Comparing the thesis against other previous or current theses

*Allowing one single criterion to colour the overall, global judgement

*Expecting PhD level work from the student

*Being unforgiving of factors outside of the student’s control e.g. recruitment

*Balancing grades by awarding one thesis a lower grade than another

*Assuming that the grade has to be high because the student is a high achiever

*Combing the thesis carefully seeking out even minor mistakes and errors

*Lack of awareness of the expectations of a Psychology Honours thesis


Start Case Study

Ajax Loader