Diagnostic Map

From BEARWiki
Jump to: navigation, search

Like the Item Tracking Map, this map also provides a convenient way for teachers to quickly identify unexpected responses, or in measurement terms, misfitting responses. One difference between the Diagnostic Map and the Item Tracking Map is that the Diagnostic Map shows the respondent's current proficiency location plus or minus one standard deviation, providing a somewhat wider range of expected performance than shown on the Item Tracking Map.

  1. To see the Diagnostic Map for a student, first select the student, then select Reports – Individual Reports - Diagnostic Map from the menu. You will be presented with a dialog window of the display options listed in the table below.
  2. Figure 1. Display options for the Diagnostic Map.
    Table 1. Display options for a Diagnostic Map.
    Option Description
    Title The title you provide will appear at the top of the report, below the Project Title.
    File Name Use the Browse... button to select the folder you want and then enter the filename you want the map to be stored in. ConstructMap will automatically store the report as a text file, so you do not need to add the .txt to your filename.
    Item Set Browse... to the Item Set that you want included. If you want to see all of the items, select the “base” Item Set.
    Step Labels Select Numerical to show step numbers (e.g., 1, 2, 3, etc.) or select Short Labels to show user-defined labels for each step (e.g., Incorr, Incompl, Corr, etc.)
    Display Item Names Select Yes if you want to see item names in the report; select No if you want to see item numbers.
    Display All Active Items Select Yes if you want to see all active items including those the student did not attempt, select No to see only items the student attempted.
    Max. Range Set the maximum logit value to display.
    Min. Range Set the minimum logit value to display.
    Rows/Interval Select Interval to set the logit interval for each row, or select Row to set the number of rows per page.
    Left Column Heading Title for the left column, which shows the respondent’s non-zero responses on each item.
    Right Column Heading Title for the right column, which shows one level above the respondent’s response level on each item.

    -

  3. The Diagnostic Map display options shown in Figure 1 were used to produce the examples in this section. Note that only items the student completed are displayed.
  4. Figure 2. Diagnostic Map heading area.
    Figure 3. Diagnostic Map excerpt for Anne Jackson on the DCI variable after completing Item Set "29-38."

    Figure 2 shows the Diagnostic Map heading section for Anne Jackson on the DCI variable, after completing Item Set "29-38". For multidimensional projects, a Map is generated for each dimension. The current dimension is shown in the heading, along with the respondent’s proficiency estimate on that dimension, labeled “Ability,” and person fit statistics (see the Full Ability Estimates section for a description of the person fit statistics).

    The left side of the map segment displayed in Figure 2, in the column labeled "Level Responded,” shows the responses the person attained for each item completed, while the right side, labeled “Next Level,” shows one step above the level of those responses. For example, the student shown here achieved a score of 3 on item “DCI-SRP”, so the location of “DCI-SRP.3” is shown in the left column and the location of “DCI-SRP.4” is shown in the right column. The notation “i.k”, is used to indicate the item (i) and score (k), respectively. “DCI-SRP.3” means a score of 3 on item “DCI-SRP.”

    The horizontal “XXX” in the center column indicates the respondent’s location, or proficiency level, on this dimension. The two horizontal dashed lines are located one standard deviation above and below the respondent’s location to delineate the range of expected responses. Entries above this range in the left-hand column or below this range in the right-hand column indicate “surprising responses” by the student. These are areas of misfit, where the person’s response did not match expectations.

    The interpretation of the relative locations of persons and items on the map is that when a respondent’s location is above an item threshold (i.e., i.k), then the probability of responding at that level or higher on the item is greater than 0.5; when a respondent’s location is below an item threshold, then the probability of responding at that level or higher is less than 0.5. Thus, the response levels of 3 on items “DCI-SRP,” "DCI-OD" and "DCI-AID" are we would expect of this respondent.

    All of the items below the top dashed line, on either side of the chart, are considered within the respondent’s proficiency range. The items above the top line are considered more challenging than the respondent’s current proficiency. In this example, the respondent’s proficiency level indicates that we would expect her to achieve a score of 3 on the "DCI-DI" item (located in the right column) also. She actually achieved a score of 2 on that item (not shown in the excerpt); but the proficiency required to achieve a score of 3 or higher is less than the proficiency level she was exhibiting overall. This is a surprising response. A teacher may want to take a closer look at why that occurred, and whether other students exhibited similar patterns on that item.

  5. Resize or change the shape of the map by dragging the corners of the map window, or the window edges. Some actions will crop the window, but dragging back will restore the window contents.
  6. Since this report produces text file output as well as an output screen, you can print this report using Word or Notepad.
  7. Currently, this map cannot be saved. Close the display by clicking on the close box, CloseX.jpg, in the upper right-hand corner.

Links