Reflection feedback

Audio feedback loops

When marking assessments, you put a lot of effort going in to marking identifying and correcting errors, explaining misunderstandings, praising good practice and providing suggestions for better approaches.

So does providing assessment feedback feel like that something is getting lost in translation?  In some extreme circumstances, maybe on you are spending time defending your feedback that you feel it was balanced and informed?

The cause may lie in part with inherent limitations of the feedback method. The solution might be in an alternative approach: audio feedback.

Jane Jones, Nick Purkis, Sandy Stockwell and Dr Ellie Woodacre from University of Winchester and University of Portsmouth faced the same issues and were looking for new ways to increase feedback’s effectiveness (see webinar https://bit.ly/2EP0Jwj).  This post is a summary of this webinar.

After some early experiments in using audio-based feedback, they wanted to move from anecdotal experience to a more formal research-based evidence.

The Study

After a small study, they are now sharing their initial research findings that explores the usefulness of audio feedback, specifically:

1. Student feedback preference: written vs audio
2. Interpretation of content
3. Student interpretation of given feedback
4. How are they giving their messages & impact on feedback.

To compare the value of audio vs written feedback, the researchers required a better understanding of how student perceived and used written feedback. The analysis used 6 comment categories:

1. Identifying errors
2. Correcting errors / explaining misunderstandings and misconceptions
3. Explaining misunderstandings
4. Praising good practice and providing suggestions for better approaches
5. Justifying marks
6. Other ‘personal’ comments spontaneous suggestions or reactions.
Each of these could be further classified as either reflecting subject knowledge, academic practice, written standards.

Speaking with students, they found written feedback was commonly interpreted as negative – sometimes, tutors spent time defending feedback comments that were intended as positive. Students disliked comments like “good” as they were ambiguous as to what specifically was good.

Tutors had concerns:

• What about clumsy expression while talking off-the-cuff.
Students saw this positively, hearing the authentic voice.

• How do I sound? What about my accent?
Building experience built confidence and these issues disappeared.

• What about accessibility?
This was not considered for the small-scale trial – but can always revert to written feedback instead.

• What about time?
The tool imposed a technical limit of 5 minutes. It does take longer but tutors felt it’s value outweighed the cost.

Structure of Audio Feedback

So what was happening in the process of written feedback? The researchers found a basic structure emerged:
Step 1: Set context
Hi StudentX. I’m TutorX and I’m providing feedback on your essay.

Step 2: Deliver feedback
Use the student name a lot (“StudentX, I really like…”) provide feedback using the 6 following comment categories:

1. Identifying errors
2. Correcting errors / explaining misunderstandings and misconceptions
3. Explaining misunderstandings
4. Praising good practice and providing suggestions for better approaches
5. Justifying marks
6. Other ‘personal’ comments spontaneous suggestions or reactions.

Step 3: Conclude
Provide the summarise feedback, include the mark and conclude.

Step 4: Post feedback
Release feedback to student.

For weaker assessments, sometimes the tutor required two audio files. Students found it took more time to read

The Tool

The tool used was Techsmith’s Jing screen capture tool. Turnitin also provides an audio comments tool. On a practical level, the free account would allow only feedback for about 50 to 80 students before server space for saving audio feedback runs out.

[Author’s note: Please be aware the Jing’s Screencast is a third-party service and your institution may not be covered by a contractual agreements which would cover issues of personally identifiable information and recourse to malicious use (eg improper sharing on a social network).]

The reactions

Students had positive reactions:

  • Increased satisfaction with audio.
  • Audio feedback engaging and used.
  • Feedback was more personal with frequent uses of their name
  • Positive feedback was easier to understand and motivating.
  • They felt that their tutors really read their submission.

Tutors had positive reactions as they could:

  • Be more engaged in the feedback process.
  • The tone of voice could express what took too long in written feedback.
  • More in-depth compared to written feedback.
  • Personalise the feedback more, giving feedback a similar feel as a direct conversation with the student.
  • Better balance between positive and corrective feedback.

The analysis

Two standout differences between written and audio feedback was a change of emphasis: less on identification of errors and more explaining misunderstandings.

At an overall level, the distribution of comments across subject knowledge, academic practice, written standard English were similar.

quote

The standout item was the other spontaneous discussions or emotions that fall in the ‘other’ category. The conjecture is this reflects the personalisation.
WcMgcq

In the analysis of audio feedback transcripts revealed how tutors ‘softened’ the tone of feedback with phrases like “you might consider…” and “try and avoid…”. In this way tutors made the necessary critical feedback, less impactful through their careful choice of words. From this point of view, audio feedback empowered tutors to be more caring for their students.

Interestingly the transcript analysis found that students interpreted the nature of feedback very differently from tutors. In a research exercise, students and tutors classified the audio feedback comments into the same 6 comment categories above. The students classified them very differently from the tutors. Regardless of the variation, it was the tone of voice that expressed the intention of feedback that was an essential advantage for both tutors and students compared to written feedback.

The outcomes

Did the audio feedback improve grades?
No (a finding consistent with other research).

Did the audio feedback improve the student experience?
Yes – audio feedback was valued more and improved student motivation. As a bonus, it also improved the tutor experience.

Resources

The researchers provided a list of publications that informed their own research:

Bauer, Sara. “When I Stopped Writing on Their Papers: Accommodating the Needs of Student Writers with Audio Comments.” English Journal 101, no. 2 (2011): 64–67.

Butler, Des. “Closing the Loop 21st Century Style: Providing Feedback on Written Assessment via MP3 Recordings.” Journal of the Australasian Law Teachers Association 9 (2011): 10.

Cavaleri, M., Di Biase, B., & Kawaguchi, S. (2014). “Academic literacy development: does video commentary feedback lead to greater engagement and response than conventional written feedback?” International Journal of Literacies (2014).

Cavanaugh, Andrew, and Liyan Song. “Audio Feedback versus Written Feedback: Instructors’ and Students’ Perspectives.” Journal of Online Learning and Teaching 10, no. 1 (2014): 122.

Chalmers, Charlotte, Janis MacCallum, Elaine Mowat, and Norma Fulton. “Audio Feedback: Richer Language but No Measurable Impact on Student Performance.” Practitioner Research in Higher Education 8, no. 1 (2014): 64–73.

Hope, Sheila. “Giving Audio – Visual Feedback Using Jing and GradeMark.” Higher Education Academy, July 1, 2010. https://www.heacademy.ac.uk/knowledge-hub/giving-audio-visual-feedback-using-jing-and-grademark.

Ice, Philip. “Using Asynchronous Audio Feedback to Enhance Teaching Presence and Students Sense of Community.” Journal of Asynchronous Learning Networks 11, no. 2 (July 1, 2007): 23.

Lunt, Tom, and John Curran. “‘Are You Listening Please?’ The Advantages of Electronic Audio Feedback Compared to Written Feedback.” Assessment & Evaluation in Higher Education 35, no. 7 (2010): 759–69. https://doi.org/10.1080/02602930902977772.

Macgregor, George, Alex Spiers, and Chris Taylor. “Exploratory Evaluation of Audio Email Technology in Formative Assessment Feedback.” Research in Learning Technology 19, no. 1 (2011): 39–59. https://doi.org/10.1080/09687769.2010.547930.

Merry, Stephen, and Paul Orsmond. “Students’ Attitudes to and Usage of Academic Feedback Provided via Audio Files.” Bioscience Education E-Journal 11 (2008): Journal, 2008, Vol.11(). https://doi.org/10.3108/beej.11.3.

Morra, Ana María, and María Inés Asís. “The Effect of Audio and Written Teacher Responses on EFL Student Revision.” Journal of College Reading and Learning 39, no. 2 (2009): 68–81. https://doi.org/10.1080/10790195.2009.10850319.

Sipple, Susan. “Ideas in Practice: Developmental Writers’ Attitudes toward Audio and Written Feedback.” Journal of Developmental Education 30, no. 3 (2007): 22–24.

Leave a reply

Your email adress will not be published. Required fields are marked*