Checkpoint analytics and process analytics

7649183772_a98c86b4b9_z

James Royal – Lawson: Flickr Capture creative commons

This activity applies checkpoint analytics and process analytics to the learning design of part of the H817 module at the Open University . I have chosen to review Block 1.

Getting to know each other Examining innovation Innovation and eLearning Evaluating innovation Horizon scanning
Where checkpoint analytics might be used

 

Introducing self

 

Setting up a blog

 

Setting up and using OU Live

Are OER both open and innovative –websites visited

Investigating a learning theory – group wiki completed and links visited Choosing and evaluating a specific technology – group report production and links visited Significant new technologies – creation of a table and links visited
Why checkpoint analytics might be used Is the learner engaging with the resources and activities required within the module design.
Where process analytics might be used

 

How to document reflection

Reading an article and searching for relevant references

Application of learning through  blog/forum debate and comments

Are OER both open and innovative – thoughts on questions posed posted to the TGF

Innovation in your context – application of concepts to own world described in TGF

A theory for elearning – stating reasons for agreement/disagreement in TGF

Investigating a learning theory working in a team to develop knowledge

Connectivism – a summary of personal analysis of connectivism completed

Choosing and evaluating a specific technology – group report content Application of thinking about new technologies to learner’s  own environment. Recorded in blog/TGF
Why process analytics might be used Are the learning objectives of the module being realised?

Is the learner understanding the module material – evidenced through analysis, discussion and application to own environment

Who would benefit Checkpoints analytics will help the Tutor ensure the learner is on track and prompt for progress if appropriate. There is also the opportunity early in this module to assess individual’s competence with technology and have insight into where additional support may be required later.

Process analytics initially help learners and the tutor get to know each other, allow the tutor to explore the ways in which learners are engaging and give insight into learner understanding.

Impact Learners understand each others’ environments, values and perspectives very quickly through collaboration which set the scene for the rest of the module

The tutor has an early view on learner’s study patterns, values and perspectives

The tutor can prompt thinking and debate around topics to ensure learning objectives are being achieved.

I would prioritise checkpoint analytics at key points in this block

  1. Introductions – to monitor early engagement and as an early warning sign that help may be needed.
  2. Group Wiki created in week 3 – to monitor engagement in collaborative tasks and to inform group choice for week 4 activity
  3. Group report created in week 4 – again to evaluate engagement (thinking ahead as early warning sign for Block 3) and to identify where the tutor may need to check in with individual learners

I would prioritise process analytics at the following points

  1. Week 1 – are learners debating and commenting?
  2. Week 2 – are learners applying their research and reading to their own environment?
  3. Week 3 – do learners understand the key debates around learning theories?
  4. Weeks 4 & 5- can learners evaluate the relevance of a specific technology

It strikes me that my process analytics priorities are focused on the  assessment requirements which are tested in an assignment at the end of the block.

Learning analytics and learning design

This blog is based on reading ‘Informing pedagogical action: aligning learning analytics with learning design’ (Lockyer et al., 2013)

The authors claim that data collected is underused or even unused due to a lack of an underlying framework. They propose a framework called checkpoint and process analytics. They argue that this framework can be applied to provide information on the impact of learning activities.

Checkpoint analytics

The student has accessed the relevant resources of the learning design e.g shown through log-ins and pages visited. Checkpoint analytics measure which files diagrams etc the learner has accessed (these are considered to be pre-requisites to the learning).

The value of checkpoint analytics lies in providing teachers with insight into whether learners are progressing through the planned learning sequence.

Some pedagogical actions

  • reports of student log-ins can be used to offer prompts for late starters
  • teacher can initiate action
  • student participation levels can be reviewed to see whether all are particpating in activities where they are all required to.

Process analytics

The student has processed the learning and applies information and knowledge e.g. shown through the tasks completed, forum postings and discussions. Process analytics measures whether the learner carries out the tasks they are expected to, using the provided learning resources to do this.

The value of process analytics lies in providing teachers with insight into engagement levels of individual learners, which networks they have built and therefore whether they have support structure or not. They also have value in determining the level of understanding.

Some pedagogical actions

  • ideas are shared and discussed – teacher can monitor the level of understanding
  • social network analysis allows identification of the effectiveness of each groups’ interaction process

I feel these categories are more useful than previously explored (data driven and pedagogical driven) This is a practical and pragmatic framework and feels  more user friendly.

References:

Lockyer, L., Heathcote, E. and Dawson, S. (2013) ‘Informing pedagogical action: aligning learning analytics with learning design’, American Behavioral Scientist, vol. 57, no. 10; also available online at http://libezproxy.open.ac.uk/ login?url=http://dx.doi.org/ 10.1177/ 0002764213479367 (accessed 14 July 2016).

MAODE, H817, Open University, Block 4 Activity 11