You can view analytics about scores and responses as well as question and distractor quality on any assessment that uses Crowdmark’s multiple choice bubble sheet. After booklets have been matched to students and the assessment has been graded you can view the analytics by navigating to the Results page of your assessment and clicking the Multiple choice analytics button.
Scores
The Scores page displays statistics for the total scores on the multiple choice component of your assessment. You can also download the data as CSV files to analyze question and student responses.
 Number of students: The total number of students matched to a graded assessment.
 Average score: Mean total score
 Median score: Middle score when the total scores are sorted from lowest to highest
 Lower quartile: Middle score of the lower half of scores when sorted from lowest to highest
 Upper quartile: Middle score of the upper half of scores when sorted from lowest to highest
 Highest score: Highest total score
 Lowest score: Lowest total score
 Standard deviation: Measures the amount of variation in the total scores
where
n
is the number of students,xi
is the total score for studenti
andx
is the average total score on the test
Question data export
From the Scores page, you can download a CSV file of data for each question. The following statistics are available in the export:
 Percent answered correctly: Percent of students who answered the question correctly
 Class: Percent of all students who answered the question correctly
 Top 27%: Percent of students who answered the question correctly in the top 27% of scores when total scores are sorted from lowest to highest
 Bottom 27%: Percent of students who answered the question correctly in the bottom 27% of scores when total scores are sorted from lowest to highest
 Average Score: Mean score on question
 Standard Deviation: Measures the amount of variation in the question score
 Point biserial: Measures how well the question discriminates between high and low scoring students. Point biserial less than 0.2 should be rejected or improved by revision
 Response Frequencies: Percent of students who selected each response (A, B, C, D, E, or x for no response)
 Answer Key: Correct response or responses to question
 NonDistractors: Response(s) to question not selected by any students.
Student data export
From the Scores page, you can download a CSV file of data for each student. The following statistics are available in the export:
 Scores: Score on each multiple choice question
 Total Score: Total score on all multiple choice questions. Total score has a minimum value of 0. If the sum of the scores produces a negative value, Crowdmark will record it as 0.
 Responses: Selected response for each question (A,B,C,D,E, or x for no response)
Questions
For each multiple choice question on your test, Crowdmark displays a bar chart showing the frequencies of each response and of no response selected (denoted “x”) on the Questions page. Hovering over one of the green bars will show you the percentage of students who selected this response. The correct response for the question is typed in bold and the corresponding bar is highlighted in darker green.
Diagnostics
The Diagnostics page summarizes your multiple choice questions in terms of difficulty, discrimination and distractors. Often referred to as item analysis, these diagnostics give you insight into your individual multiple choice questions as well as some testlevel statistics to help you determine the quality of your test. Crowdmark highlights your most difficult and easiest questions, questions that are most and least effective at discriminating between your best and worst performing students and questions that include functional and nonfunctional distractors.

Difficulty: The difficulty of a question is determined by computing the percentage of students who answered the question correctly.
Crowdmark classifies questions as easy when more than 70% of students answer correctly and difficult when less than 30% of students answer correctly.

Discrimination: The discrimination of a question refers to the correlation between student responses to a particular question and total scores on the test. Crowdmark uses the Point Biserial coefficient, denoted by
rpb
, to calculate the discrimination of the correct response to each question using the formula:where
M1
is the mean test score for the group who answered the question correctly,M0
is the mean test score for the group who answered the question incorrectly,sn
is the test standard deviation,p
is the proportion of students who answered the question correctly andq
is the proportion of students who answered the question incorrectly.A positive point biserial coefficient for a correct response indicates that those who scored well on the whole test answered the question correctly. Generally, the correct response for a point biserial greater than 0.40 is a very good discriminator and less than 0.2 should be rejected or improved by revision.
Point biserial correlation coefficients can also be calculated for each incorrect response (distractor). A question should have a negative pointbiserial discrimination for each of its distractors. This means that the incorrect responses were selected by the weaker students.

Distractors: Distractors are the incorrect responses to a question.
Crowdmark classifies questions as poor distractors poor when an incorrect response to the question was selected by less than 5% of students or the incorrect response was selected by top performing students.
Crowdmark classifies questions as having good distractors when all incorrect responses are chosen by at least 5% of the students and incorrect responses are selected by weak students.
Highlevel statistics

Distractor quality: Distractors are the incorrect responses to a question. For example, if the correct response to a question is B, then the distractors would be A, C, D and E. A good distractor is one that was selected by students with low achievement. Crowdmark measures the distractor quality of the overall test by calculating the percentage of good distractors on the test. Crowdmark classifies a good distractor as having a frequency of response of at least 5% and discriminating towards students with a lower total score. Crowdmark uses the point biserial coefficient to calculate the discrimination correlation of each distractor.

Reliability coefficient: Measures whether the test as a whole discriminates between higher and lower performing students. Crowdmark uses the KuderRichardson 20 coefficient (KR20) to calculate the test reliability, which we will denote by
r
in the formula below.where
n
is the number of questions,pi
is the proportion of students who answered questioni
correctly,qi
is the proportion of students who answered questioni
incorrectly andsn
is the test standard deviation.The KR20 coefficient typically ranges between 0 and 1 but it is possible to obtain a negative value with small sample sizes. The closer to 1 the more reliable an exam is considered because its questions do a good job consistently discriminating among higher and lower performing students.
The interpretation of the KR20 depends on the purpose of the test. Tests intended to distinguish between students who have mastered the material and those who have not should have a reliability coefficient of 0.5 or higher. A test with a KR20 value less than 0.3 is considered poor. If the purpose of the test is to ensure that all students have mastered essential concepts, then the KR20 value should be close to 0.
Sources:
 Distractor Efficiency in an Item Pool for a Statistics Classroom Exam: Assessing Its Relation With Item Cognitive Level Classified According to Bloom’s Taxonomy
 Kuder–Richardson Formula 20 on Wikipedia
 Interpreting Exam Performance: What Do Those Stats Mean Anyway?
 Statistical Analysis of Multiple Choice Exams
 Item Analysis for CriterionReferenced Tests