Item Analysis: Evaluating Multiple Choice Questions for Quality

Question 1

What statistic measures the likelihood that a test taker will answer an item correctly?

Accepted Answer

Item difficulty index (p)

Answer

Discrimination index

Answer

Point-biserial correlation

Answer

Reliability coefficient

Question 2

Which limitation of classical test theory (CTT) item analysis pertains to its sensitivity to sample characteristics?

Accepted Answer

Is sensitive to sample characteristics

Answer

Assumes unidimensionality

Answer

Requires large sample sizes

Answer

Cannot estimate item parameters

Question 3

What is the primary function of logistic regression in item analysis?

Accepted Answer

Predicting item responses based on examinee characteristics

Answer

Identifying potential biases

Answer

Estimating test reliability

Answer

Calibrating test items

Question 4

Which technique is used to pinpoint biased items within a test?

Accepted Answer

Differential item functioning (DIF) analysis

Answer

Discrimination index

Answer

Item difficulty index

Answer

Point-biserial correlation

Question 5

What is the purpose of examining the standard error of measurement (SEM) in item analysis?

Accepted Answer

To estimate the degree of uncertainty in item responses

Answer

To detect potential biases

Answer

To enhance test reliability

Answer

To calibrate test items

Question 6

For multiple-choice items, what is considered an optimal difficulty index?

Accepted Answer

0.50

Answer

0.25

Answer

0.75

Answer

1.00

Question 7

Which item analysis method is specifically designed to detect items that may be biased against certain groups of test takers?

Accepted Answer

Differential Item Functioning (DIF) analysis

Answer

Item difficulty analysis

Answer

Item discrimination analysis

Question 8

What is the primary purpose of the discrimination index?

Accepted Answer

To measure item discrimination power

Answer

To determine item difficulty

Answer

To assess item reliability

Answer

To identify item bias

Question 9

Which of the following is NOT a type of item analysis software?

Accepted Answer

All of the above are types of item analysis software

Answer

RStudio

Answer

JASP

Answer

Excel

Question 10

To conduct a distractor analysis, what is the correct sequence of steps?

Accepted Answer

Identify the correct answer, analyze the distractors, calculate the discrimination index, and revise the item.

Answer

Calculate the difficulty index, identify the correct answer, analyze the distractors, and revise the item.

Answer

Analyze the distractors, identify the correct answer, revise the item, and calculate the discrimination index.

Answer

Revise the item, analyze the distractors, calculate the difficulty index, and identify the correct answer.

Question 11

Which of the following is NOT a purpose of item analysis?

Accepted Answer

Determining the content validity of a test

Answer

Identifying items that are too easy or too difficult

Answer

Estimating the reliability of a test

Answer

Detecting items that are biased

Question 12

What does the item discrimination index indicate?

Accepted Answer

The ability of an item to differentiate between high-performing and low-performing test takers

Answer

The difficulty level of an item

Answer

The amount of guessing that occurred on an item

Answer

The bias of an item

Question 13

What is the main purpose of conducting a difficulty index analysis?

Accepted Answer

To identify items that are too easy or too difficult

Answer

To assess the overall difficulty level of a test

Answer

To detect items that are biased

Answer

To estimate the reliability of a test

Question 14

Which of the following is an example of a qualitative approach to item analysis?

Accepted Answer

Examining the content and wording of an item

Answer

Calculating the item discrimination index

Answer

Conducting a factor analysis

Answer

Estimating the item difficulty level

Question 15

What is the intended role of distractors in multiple-choice items?

Accepted Answer

To provide students with incorrect but plausible answer choices

Answer

To increase the difficulty of the item

Answer

To ensure that all students have an equal chance of answering the item correctly

Answer

To reduce the amount of guessing

Question 16

What is the initial step in the item analysis process?

Accepted Answer

Identifying the purpose of the item analysis and selecting the appropriate data

Answer

Collecting data from test takers

Answer

Analyzing the data

Answer

Reporting the results

Question 17

What does the difficulty index of an item measure?

Accepted Answer

The percentage of test takers who choose the correct answer

Answer

The percentage of test takers who answer an item incorrectly

Question 18

Which of the following does NOT affect item difficulty?

Accepted Answer

Test-taker's age

Answer

Item length

Answer

Item complexity

Answer

Item familiarity

Question 19

What is the primary purpose of using distractors in multiple-choice items?

Accepted Answer

To provide incorrect options for test takers to choose from

Answer

To make the item more challenging

Answer

To measure test takers' ability to identify correct and incorrect answers

Question 20

What does a high item discrimination index indicate?

Accepted Answer

The item is better able to differentiate between high- and low-performing students

Answer

The item is easier for all students

Answer

The item is more difficult for all students

Answer

The item is not able to differentiate between students

Question 21

What is the purpose of calculating the item-test correlation?

Accepted Answer

To determine how well an item correlates with the overall test score

Answer

To determine the difficulty of the item

Answer

To determine the discrimination index of the item

Answer

To identify items that are biased

Question 22

How can item analysis help improve student learning?

Accepted Answer

Both of the above

Answer

By providing information about student strengths and weaknesses

Answer

By identifying areas where instruction can be improved

Answer

Neither of the above

Question 23

To identify test items that might unfairly disadvantage certain groups, which item analysis method is considered the most effective?

Accepted Answer

Differential Item Functioning (DIF) analysis

Answer

Factor analysis

Answer

Item response theory (IRT)

Answer

Classical test theory (CTT)

Question 24

How does item analysis contribute to the creation of effective assessments?

Accepted Answer

Improves assessment quality by identifying problematic items and guiding revisions

Answer

Facilitates assessment development by reusing high-quality items

Answer

Eliminates the need for external assessment validation

Answer

Ensures assessments are challenging for all students

Question 25

The difficulty index of an item is a measure of the proportion of students who answered the item correctly. Which of the following is the formula for calculating the difficulty index of an item?

Accepted Answer

(Number of students who answered the item correctly) / (Total number of students who took the test)

Answer

(Number of students who answered the item incorrectly) / (Total number of students who took the test)

Answer

(Total number of students who took the test) / (Number of students who answered the item correctly)

Answer

(Total number of students who answered the item incorrectly) / (Number of students who answered the item correctly)

Question 26

Item discrimination is a measure of how well an item distinguishes between high- and low-achieving students. What is the purpose of item discrimination?

Accepted Answer

Identify items that distinguish between high- and low-achieving students

Answer

Estimate the reliability of each item

Answer

Calculate the validity of the test

Answer

Create a table of specifications

Question 27

A table of specifications is a document that outlines the content and skills that will be covered on a test. What is the importance of using a table of specifications when conducting item analysis?

Accepted Answer

It ensures that the items on the test are aligned with the learning objectives

Answer

It helps to identify areas where the test needs to be revised

Answer

It provides a way to track the performance of individual students on the test

Answer

It allows researchers to compare the results of different tests

Question 28

Item difficulty can be interpreted in different ways. Which of the following is an example of a criterion-referenced interpretation of item difficulty?

Accepted Answer

60% of students were able to answer the item correctly

Answer

The item was more difficult than the other items on the test

Answer

The item discriminated between high- and low-achieving students

Answer

The item was biased against a particular group of students

Question 29

What is the overarching purpose of conducting item analysis?

Accepted Answer

To improve the overall quality of multiple-choice assessments

Answer

To identify and eliminate overly easy items

Answer

To identify and eliminate overly difficult items

Answer

To identify and eliminate biased items against specific test-taker groups

Question 30

Item analysis can be a valuable tool, but what is a significant drawback?

Accepted Answer

It's often time-consuming and costly.

Answer

It doesn't reveal how difficult test items are.

Answer

It doesn't highlight the test's strengths and weaknesses.

Answer

It can't identify biased test items.

Question 31

Which index measures the difficulty of an item?

Accepted Answer

Item Difficulty Index

Answer

Item Validity Index

Answer

Item Reliability Index

Answer

Item Discrimination Index

Question 32

What is the purpose of item analysis?

Accepted Answer

To evaluate the quality of test items

Answer

To create new test items

Answer

To calculate the overall test difficulty

Answer

To identify students with learning difficulties

Question 33

Which type of bias occurs when an item favors test takers from a particular cultural or ethnic group?

Accepted Answer

Cultural bias

Answer

Gender bias

Answer

Difficulty bias

Answer

Reliability bias

Question 34

What is the purpose of a distractor?

Accepted Answer

To make the item more challenging and to discriminate between high and low performers

Answer

To provide additional information about the topic being tested

Answer

To confuse the test taker

Question 35

Which of the following is NOT a benefit of item analysis?

Accepted Answer

Improved test security

Answer

Identification of bias

Answer

Improved test validity

Answer

Improved test reliability

Question 36

Which of the following is a limitation of item analysis?

Accepted Answer

It can be time-consuming and requires specialized knowledge.

Answer

It can only identify items that are too easy or too difficult, not items that are biased.

Answer

It is only useful for multiple-choice items.

Question 37

Which of the following is a type of index used to measure item discrimination?

Accepted Answer

Point-biserial correlation

Answer

Pearson correlation

Answer

Spearman correlation

Question 38

Which of the following is a step in the item analysis process?

Accepted Answer

Calculate item difficulty and discrimination indices

Answer

Administer the test

Answer

Write test items

Question 39

Which of the following is a potential benefit of using item analysis?

Accepted Answer

Improved test validity and reliability

Answer

Faster test scoring

Answer

Increased student motivation

Answer

Reduced test anxiety

Question 40

What is a typical value for an item difficulty index?

Accepted Answer

0.5

Answer

0.75

Answer

0.25

Answer

1.0

Question 41

Which of the following is NOT a primary purpose of item analysis?

Accepted Answer

To assess the validity of individual test items

Answer

To identify items that are biased against certain groups

Answer

To identify items that are too easy or too difficult

Answer

To improve the overall quality of a test

Question 42

Which of the following is a commonly used measure of item discrimination?

Accepted Answer

Point-biserial correlation

Answer

Item validity

Answer

Item difficulty index

Answer

Item reliability

Question 43

Which of the following is an example of a type of item bias?

Accepted Answer

Gender bias

Answer

Item validity

Answer

Item discrimination

Answer

Item difficulty

Question 44

What is the primary function of distractors in multiple-choice items?

Accepted Answer

To provide plausible incorrect options for test takers to choose from

Answer

To simplify the item for test takers

Answer

To increase the validity of the item

Answer

To increase the reliability of the test

Question 45

How is item difficulty typically related to item discrimination?

Accepted Answer

Items with moderate difficulty levels tend to have higher discrimination values

Answer

Items with low difficulty levels are often more discriminating

Answer

Items with high difficulty levels are generally more discriminating

Question 46

When interpreting item analysis data, what is the most essential factor to consider?

Accepted Answer

The intended purpose of the assessment

Answer

The total number of items on the assessment

Answer

The overall difficulty level of the assessment

Answer

The time limit allocated for the assessment

Question 47

What is a commonly recommended cut-off value for interpreting the point-biserial correlation (rpb) in item analysis?

Accepted Answer

|rpb| > 0.30

Answer

|rpb| > 0.10

Answer

|rpb| > 0.70

Answer

|rpb| > 0.50

Question 48

What is the primary index used to represent item difficulty?

Accepted Answer

P-value

Answer

Reliability coefficient

Answer

Discrimination index

Answer

Validity coefficient

Question 49

Which of the following discrimination index values generally indicates a good item?

Accepted Answer

0.3 or higher

Answer

0.1 or higher

Answer

-0.1 or lower

Answer

-0.3 or lower

Question 50

What is the main goal of conducting a Differential Item Functioning (DIF) analysis?

Accepted Answer

To identify items that function differently for various subgroups of test takers

Answer

To establish the overall difficulty of a test

Answer

To enhance the reliability of a test

Question 51

Which of the following is NOT a method for conducting a DIF analysis?

Accepted Answer

Cronbach's alpha

Answer

Mantel-Haenszel method

Answer

Logistic regression

Question 52

What is the recommended P-value threshold for flagging items as potentially biased in a DIF analysis?

Accepted Answer

0.05 or less

Answer

0.01 or less

Answer

0.10 or less

Answer

0.15 or less

Question 53

Which of the following is a limitation of item analysis?

Accepted Answer

It can be time-consuming and resource-intensive

Answer

It can provide precise and reliable information about item quality

Answer

It can be used to improve the validity of assessments

Question 54

What is the best way to use the results of item analysis?

Accepted Answer

To make informed decisions about item revisions and assessment improvement

Answer

To ignore items that are flagged as problematic

Answer

To punish students who perform poorly on biased items