Trying to analyze marks obtained by students in exams and getting something meaningful out of it has been something that I had planned on working for a long time.
The reason for not jumping in was mainly around what tools I would use to do the analysis. Though conventional statistics could provide most of what I was looking to, I wanted to go a little further and look into ways of categorizing students to find topics that should be repeated to them.
Having gone though a bit of Decision trees and K Means clustering I decided to use these and see how they work in small datasets. *A sample dataset is added at the end of the article.
The dataset is for a particular exam paper (this being Physics). It consists of marks obtained by students in each of the questions, the total and a grade that is given for a range of totals.
To make it simple for analysis I normalized the scores to 0 and 1. This had to be done as the highest score for individual questions varied.
First off I created a box and whisker plot of the marks.
This should variation in marks for Q4, Q8 are very wide while those for Q0, Q5, A13 and Q14 are narrow — meaning students scored near each other for Q0, Q5, A13 and Q14.
To find out questions that really changed the scores for the students, I created a decision tree.
This Decision tree gives the questions that groups of students either scored high in or scored low in. If we follow the tree along the left most path, one finds that many students (12 out of 34) have done badly in question number 7. Also ones who have done well in questions Q7 have also done well in question 5. At the top of the tree the students who have not done well in Q6, seem have not done well in most of the questions.
Having learned this I set out to build a K mean cluster to see the students group based on the marks they scored. Below is the cluster visualization.
The cluster with the red dots show students that are faring along well compared to others in all questions. The ones in yellow are not faring along well and need attention in all questions. If one goes back to the Decision tree the yellow dot group did really badly in Q6, Q1 and Q7 thereby moving towards the left end of the K means cluster.
This is starting of data analysis tool for students data. The thought behind this is that in Nepal and countries around it, large scale exams will happen and we need to find ways to analyze this data to provide education opportunity to students based on how they are faring in them.
Zip file of code and sample data — http://www.prjoshi.com/wp-content/uploads/2019/12/BVS_marksAnalyzer.zip