Dr Kirsten Way
Peter Faber Business School
OHSE602 OHSE Management Systems
A mix of those who are very experienced, and those who have no experience
Kirsten’s team received a grant to create an 'Authentic Work-Based Assessment for Online Students’. The team wanted to focus on using case-based learning to embed a realistic, applied context for the unit structure and assessment using a sustainable model that could be modified for future iterations of the unit.
In the past students analysed an industrial accident from history based on written materials e.g. the Esso Gas Plant Explosion at Longford. The plan was to improve this approach by creating an online mimetic simulation that unfolds over 9-weeks of the unit, with related assessments.
Here is a short 2 min interview exploring Kirsten’s plans before she embarked on the project….
ACU Case study: Creating authentic online simulations using branching scenarios I
Length: 2 mins.
Intended project outcomes
- Build higher quality student interaction through discussion forums
- Develop critical thinking skills
- Improve learning behaviours through inclusion of emotionally-charged content
- Prepare students for the realities of the workplace
The team developed a 9-week simulation around a workplace industrial accident. The simulation was built in LEO lesson, to allow a ‘choose your own adventure’ style structure, in which students are posed realistic questions and then participate in activities about next steps in unpacking the incident and responding to unexpected events as they occur throughout the period.
ACU Case study: Creating authentic online simulations using branching scenarios II
Setting the scene: Overview
The team split development into three parts:
- Develop the 'big picture' storyboard for the LEO lesson, including where to place key decision points and where to link to resources and activities external to the lesson
- Develop the resources (e.g. authentic materials relevant to investigating the incident, video and audio stimuli, online lecture-based resources) and activities (e.g. forums for discussing issues related to decision-making) that students could link to from within the lesson
- Build the simulation in LEO
Structure of the simulation
There are three stages to the simulation, each one lasting 3 weeks. Each week a different step in the stage is played out, and students choose how they will respond. There is an authentic assessment associated with each stage.
1 - Out of the blue
2 - Mind the gap
3 - What's really driving behaviour
|The incident occurs, and students are in the role of first responders and have to undertake a range of actions upon finding out about the incident.||Students do an analysis, and have some interpersonal issues that they have to navigate within the organisation, with the CEO, HR and the grieving family members.||The final stage is around the leadership of the board, and safety culture. Students learn about the different safety leadership styles of board members and attempt to influence their safety behaviours.|
|Business report about the interim findings||Report to the board about the failures of the safety management system that led to the incident||Video pitch to the senior managers and board to implement Safety II measures to improve the safety culture|
Planning the scenario
The team created a detailed plan of the flow of the scenario and how it would run. This was developed in an iterative way once building began.
For future iterations it provides a solid blueprint for making and discussing updates offline before updating the lesson activity itself.
For other units, it provides a good practice example of how to plan and prepare for a lesson simulation offline before building it in LEO.
Videographers and actors
In order to make the simulation more authentic, the team employed an external videographer to film clips. They utilised the acting students from Brisbane campus to perform scripted vignettes, displaying the interpersonal, emotional and real aspects of dealing with workplace industrial incidents.
The clips allowed students to consider the sorts of situations they will have to deal with and explore the sorts of decisions they will need to make as a practitioner.
Example video: Your CEO, who you rarely see, is suddenly standing at your office door.
- Your phone rings [ audio sample of call from a very distressed person on a work site, saying an incident has happened, there’s blood everywhere, the ambulance are here, the police is here, please can they come and help].
- Students are asked – What do you do? Do you go to the worksite?
- Students are encouraged to go to the discussion forum to discuss it.
- What do you take with you? What is your purpose? What are you going to do there? [students are linked to relevant lecture material and industry experts talking about how to do an accident investigation etc.]
- They arrive on site, talk to the hotel manager, and are asked to prioritise the site tasks.
- Choose where to go next.
Actual project outcomes
While data collection is still ongoing, initial findings indicate:
- Higher quality student interaction through discussion forums
- There was more than double the number of posts on the discussion forum
- The quality of posts increased and students shared personal perspectives in how they’ve dealt with these kind of problems in workplaces.
- Most of the lecture videos have been viewed twice as often as in the non-simulation version of the unit. Indicating a more critical engagement with the content and topics covered.
This is what was learnt from the experience.
- Expect to need to iron out LEO issues in the first iteration. For example, in the first iteration we discovered there is a limit of a maximum of 10 attempts students can make at questions in a LEO Lesson before they automatically get progressed to the next page and can no longer return to the question page. As students tended to go down different paths, they would reach the maximum. The solution was to not use the question pages in this way.
- Getting the complexity and challenge right to cater for students with experience and with no experience
- Even with rigorous planning, the first time the unit runs, with many students going through it and talking about it, errors will surface. Have a strategy in place to deal with these issues as they arise.
- Kirsten plans on using templates for assignments to communicate clear expectations on assessment requirements as some students created assessment responses that were too extensive
- In the first iteration, the team discovered students were perturbed by the intentional inconsistencies in the scenario. The team had embedded gaps and inconsistencies purposely, as they were trying to create a realistic scenario because in real life when you’re doing workplace investigation, people tell you different things and there is information you don’t yet know.To meet the students need for reassurance whilst keeping pedagogic intent, the team added to the second iteration:
- Introduction modules to give students an overview of the pedagogical approach (e.g. transparent pedagogy)
- Within the introduction "design warnings" about purposeful inconsistencies and the reason they exist
ACU Case study: Creating authentic online simulations using branching scenarios IV
Length: 2 minutes.
...the practical simulation was one of the best features of the unit
...what was more important than that was the whole process filled me with a confidence. Do you know what I mean? I now have a greater degree of confidence if I’m to approach a real life situation that was complex …...
There is a lot more confidence in my approach mentally, I mean I haven’t physically approached a similar problem to the simulation but I do feel confident if that came up I could handle it quite adequately
- Some students perceived the workload associated with the simulation as high, as it involved sustained involvement across the semester
- Some students were perturbed by the ‘authenticity’ and unpredictability of case-based simulation
Academic colleague feedback:
I was very, very impressed with the scope of tools that were able to be deployed in order to simulate a real-life experience for an OHS practitioner. In fact, I after about an hour and a half to two hours engaging with the material I found myself almost stressed [laughter] because it took me back to a work role and all these competing things coming at you and trying to prioritize them and dealing with so many different people. Maybe because I was dealing with it in a compressed timeframe and trying to simulate as much as I could within two hours. But I did. I didn't have to go and get a valium or anything but I had to walk away and get a coffee [laughter]. That's a [tick]. I think it's a tick in terms of simulating the real-life activity.
.... the time sequencing is real-life and time spacing is real-time if I can put it that way. So I think it does create a sense of urgency, which you can relate to a work experience rather than just an academic experience. So if I don't do this, then the boss will be onto me, or if I don't do this, well, then I can't do that. So I think there's a lot of time management sequencing and prioritizing attributes in built, which may be incidental, but they're coming to the fore the whole way through.
Ideas for the future
Collecting further data to further examine:
- improvements in student engagement and learning outcomes
- measureable ‘authenticity’ (Ashford-Rowe et al., 2013) of the simulation and assessment
- mechanisms for efficacy, focusing on students’ ability to recognise emotion in the online simulation, as well as their ability to recognise their own emotional response.
Kirsten was also supported in the design and development by:
- FLB Educational Designer – Lisa Burrell.
- FLB eLearning Technical Support Officer – Meghan Brennan
- Academic collaboration – Kevin Ashford Rowe
- Ashford-Rowe, K., Herrington, J. & Brown, C. 2013. Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39, 205-222.
Page last updated: 2018-09-11
Short url: https://www.acu.edu.au/1700715