‘Online Examination Design in HAS’ (updated August 2021)

UWE position;

  • No return to mass on campus invigilated exams/no ECC
  • Online exams – minimum 24 hour period

Points to consider

Multiple Choice Questions (MCQs) and short answer questions; if MCQs/short answer questions are to be used as a summative assessment then the design should consider how these can adequately and appropriately assess the relevant learning outcomes. For example students are permitted to access resources and whilst they are not permitted to collude, consider how you will deter this, or identify/evidence if it has occurred. The UWE position is that even for MCQ/short answer questions the 24 hour window should be in place to allow for RA’s and IT issues.

  • Time constraint; 24 hours permitted to complete the assessment;
    – It may be helpful to advise students that they are not expected to spend more than X hours undertaking the assessment – this cannot be enforced but may help students gauge the amount of effort, you may wish to monitor as part of your assessment evaluation to inform future years
    – Do not set a timer constraint reducing the completion time within the 24 hours.
  • Word limit; yes set a maximum word limit. As we moved our traditional exams to online exams a guide of 1000 words per hour was used. However module teams are now designing online assessments for an online platform so consider the task and the word count related to the task rather than necessarily following previous guidance. If you are expecting students to read around and reference then this takes time. Be clear what happens if word count is exceeded – align to word count policy; over the word count will not be marked.
  • Do you want students to upload different files for different questions (may help marking) – if so make this clear in the rubric
  • Avoid/decrease student choice e.g. one or a small number of long answer questions all of which are compulsory
  • Students have access to resources;
    – design questions that require students to demonstrate how they use information rather than recall – replicate the ‘real world’ i.e. ordinarily we do look things up when we are not sure and this can be a skill in itself, – rarely are we in an exam situation
    – free text, longer contextual answers reduce the likelihood of finding an answer on ‘google’
    – be clear in the assessment brief; what is your expectation in terms of use of resources, level of referencing required/presentation (are you expecting spell checking, reference list etc.)
    – consider your marking scheme taking into account resources are available; will you be rewarding/penalising communication/presentation – ensure consistency
    – do not turn off Bb resources for the duration of the assessment; whilst this may seem useful it does not prevent students from accessing these resources as they could have material downloaded prior to the assessment which in turn could create inconsistency
    – Be clear on how students may use lecture and teaching materials/resources and how you would expect these to be referenced
  • Remember inclusivity still applies – check the format, use the word accessibility checker, and ensure you put your exam paper through Blackboard Ally
  • Design out assessment offences
    – Do not repeat exam paper/assignment questions
    – Create individualised assessment tasks
    – Ask for student opinions and analysis rather than descriptive essays
    – Once an assignment is designed, ‘Google’ the questions and see what information is identified
    – Require students to identify sources they have used – where possible a hyperlink so you can check if you wish to
    – Consider releasing resource(s) at the start of the assessment that they must read prior to the assessment and base the assessment on this, e.g. critical review of a research paper where the research paper is made available at the start
    – Clear deterrent in the assessment brief – detection tool process/penalty

Recording online assessments in webinar tools

Recording online assessments in Collaborate; set up the assessment using the Collaborate Student Assessments system (i.e. not using a regular Collaborate Classroom). This is to ensure that assessment recordings are not viewable by other students. This assessment system was developed and released last summer to allow assessment in a compliant way, in regards to our data protection responsibilities. There are Blackboard Support pages that support the Collaborate Student Assessment System. In particular, some great videos on how to use the Collaborate Student Assessment system from FET.

Example of an Online Examination brief

Online Examination identify which module/component/element

Exam paper will be released at ******************** and you will have 24 hours to complete the paper (identify if BST or GMT). Submission will be via Blackboard.

There is no late submission window; please do not leave your upload to the last minute.

Include any specific instructions e.g.

  • Submit one Word document containing all answers/submit a separate Word document for each answer clearly labelling each file with the question it relates to e.g. Question 1
  • Clearly indicate your student number as a header in your answer script.
  • Indicate which question(s) you have chosen to answer before you start answering the question. If students have a choice.
  • You have a XXX word limit for each question. This is an absolute limit – anything over the limit will not be marked. The total word limit should reflect what students are being asked to do, this should assist students to write an appropriate amount and also assist markers.
    – Be specific about what is/is not included e.g. in text citations/tables/hyperlinks to sources are all included (easy to check/count)
    – Please give the word count at the end of the answer
  • Resources can be accessed during this assessment, consider what message you want to convey, some examples below;
    – Resources can be accessed during this assessment; where you access a resource you should provide the link to that resource or make it clear what the source is – a full Harvard referencing system and reference list is not required however it is essential that the sources you have used can be checked by the marker. Marks will not be lost for presentation of sources but marks may be lost if sources cannot be followed/found, or are not cited, the latter may constitute an assessment offence. Where your answer is based on lecture notes or teaching materials then please make this clear – you should be conveying information in your own words.
    – The expectation is that you should be accessing relevant resources to complete this assessment. Resources should be referenced within the body of your work and you should provide a reference list; consider familiarising yourself with a package to help you easily create a reference list during the assessment. Referencing should be presented in UWE Harvard format. Marks will reflect the range/appropriateness of the resources you have used as well as the way in which you have used them to support your assessment.
    – The assessment is designed to encourage you to reflect on your learning, resources can be accessed during this assessment and where you access a resource you should provide the link to that resource or make it clear what the source is – a full Harvard referencing system and reference list is not required however it is essential that the sources you have used can be checked by the marker. There is no maximum number of resources that can be accessed; between six and eight references is recommended as a guide.
  • Do not copy, copy and paste from sources, including lecture and teaching materials; ensure that you understand and practice good academic writing. Markers are looking for evidence of application and understanding, not just factual content.

*Submissions will be put through plagiarism detection software; this compares your work with that of other students, as well as external sources, so do not be tempted to contact others for this assessment, sharing work or colluding is an assessment offence. This should be you own independent piece of work. Students found to have committed an assessment offence will be managed according to the assessment offence policy.

Marks will be returned via Blackboard.

Identify how, when and where feedback will be given e.g. generic or individual. Some thoughts, where your assessment is still considered to be an ‘exam’;

  • Feedback is a requirement but does not have to be at individual scripts level and can be at cohort level
    – Be mindful how students may perceive a lack of parity if for some online exams they are getting individual feedback and for some they are getting cohort level generic feedback; the rationale for the mixed approaches may be useful to share
    – If you are annotating student scripts please check if the annotations are to be external examiner facing only or if comments will be student facing as this may impact on what annotations are added
  • If you are providing cohort level generic feedback then it may be useful to consider gathering this proactively from your markers particularly where you are using the ‘Team Second Sample Marking’ process, trying to gather this retrospectively may be more problematic. Consider;
    – Asking markers to identify things students generally did well and things they could improve on in the course of the marking;
    – Providing markers with some key questions/areas/points to report back on as to how well students addressed these in the exam
  • When you have the generic feedback collated from markers then consider how you might deliver this to the cohort – a short video may be useful to be able to convey more feedback and appropriate tone rather than written feedback

Staff availability:

Identify when the module leader (or nominee) will be available during the 24 hour period and how they might be contacted. Be clear this is for queries about the exam paper…. UWE guidance indicates that ML will be available for the first hour but you may wish to apply some flexibility for this to capture early and later starters. E.g.

‘Dear Students

As Module Leader I (or nominee) will be available by email/telephone (please specify) between *** and *** (two 45 minute sessions might be ideal to capture those early/later starters but appreciate there will be constraints on individuals availability) for exam related questions. Please note that it is not appropriate for me to respond to questions seeking clarification on aspects such as question interpretation. If a query is raised that warrants whole cohort clarification then this will be communicated by ************* (Bb announcement?).

If you experience IT issues during your examination then please contact IT services, the IT helpline number to call is; +44 (0)117 328 3612; Email; itonline@uwe.ac.uk ; their core hours are Monday-Thursday: 08:30-17:00, Friday: 08:30-16:30. Calls outside of these hours will automatically go to an ‘out of hours’ team (non UWE). ‘

For operational guidance on setting up an online examination; https://fetliu.net/has/2020/09/10/online-examination-design/ Or https://fetliu.net/has/methods-of-assessing-online/

Student facing pages;
https://www.uwe.ac.uk/study/academic-information/assessments/exams/exam-guidance#a39c59048-39fc-4ce7-85da-184cd29efbf5