TCMtests.com principle type of multiple choice item formats is the conventional multiple choice test with four distractors; rarely will we use matching, extending matching, pictoral items sets, problem solving item sets, vignette or item scenario item sets. There are four areas of item writing guidelines that TCMtests.com uses.

I. Content Guidelines

Every item reflects specific content and a single specific cognitive process, as called for in the test specifications.

1. The content should come from a topic outline or list of major topics, books, etc. as dictated by the particular board examiners or college course. All content can essentially be reduced to facts, concepts, principles, and procedures. Generally topics subsume this distinction. The cognitive demand is usually recalled or understanding but if the intent of the item is to infer status to the ability such as problem solving, the application of knowledge and skills is assumed.

2. Base each item on something important to learn; avoid trivial content. The judgment of the importance of content and the cognitive demand is subjective and is appropriately done by experts in deciding a particular item for test.

3. Use novel material to test for understanding and other forms of learning. The testing of understanding is that of recall and can be done using some strategies or a concept principle or procedure is tested in a novel way. To achieve this novelty, content presented at the textbook is not reproduced in the test item; instead one asks the students to identify an example of something or re-paraphrase a definition to see if a student can link the paraphrase definition to a concept principle or procedure. This type of concept learning usually is presented in a scenario or vignettes that ask for critical thinking and problem solving. Below are two examples. The first item provides dictionary definitions. The second item provides examples of writing as the student understands a metaphor is supposed to select the correct example.

   Which is the best definition of a metaphor?

   1. Metaphor describes something as if it were something else.
   2. Metaphors make comparisons to other things.
   3. Metaphors are trite, stereotype expressions.

   Which of the following is a metaphor?

   1. At the breakfast buffet, I ate like a pig.
   2. My cat has fur like knotted wool.
   3. She is like a rose full of thorns and smelly.
   4. The mother was a bear protecting her cubs.

4. Keep the content of the item independent of the content of other items on the test. A tendency when writing sets of items is to provide information in one item that helps the test taker answer another item. Do not do that.

5. Over-specific or over-general content. Avoid over-specific or over-general content. The concept of specificity of knowledge refers to continual knowledge from specifics to general. Most items should probably be written with this in mind. We should avoid the extremes. Over-specific knowledge tends to reveal to the domain of knowledge intended. General knowledge may have many exceptions or are ambiguous. The judgment of specificity and generality of the items should be reviewed by others who can help judge the specificity and generality of each item.

6. Avoid opinion based items. Items should reflect well-known publicly supported facts, concepts, principles, and procedures. One should qualify each opinion such as according to the Nei Jing...?

7. Avoid trick items. Trick items are intended to deceive the test taker into choosing a distractor instead of the right answer. Trick items are difficult to illustrate. Trick items can be deliberately intended by the item writer or sometimes accidentally become trick items to test takers. There are seven types of items that students perceive as tricky.

   These are

   1. The item writers intention appear to be deceive, confuse, or mislead test takers;
   2. Trivial content was represented;
   3. The discrimination among opinions was to find;
   4. Items had window dressing that was irrelevant to the problem;
   5. Multiple correct answers were possible;
   6. Principles were presented in ways that were not learned, thus, deceiving the students;
   7. Items were so highly ambiguous, even the best students, had no idea about the right answer.

   The following are open-ended trick items.

   1. Is there a 4th of July in England?
   2. Some months have 31 days. How many have 28?

II. Style and Format Concerns

8. Format items vertically instead of horizontally. Horizontal formatting is harder to read thus confusing students and lowering test scores.

9. Edit items for clarity. Early in the development that item should be subject to scrutiny by a qualified editor to determine if the central ideas presented as clearly and concisely as possible.

10. Added items for correct grammar, punctuation, capitalization, and spelling. Acronyms may be used but there use should be done carefully. Generally acronyms are explained in the test before being reused.

11. Simplify vocabulary. The purpose of most multiple choice achievement test is to measure knowledge and skills that were supposed to be learned. Research shows limited English proficient students performed better when the language was simplified.

12. Minimize reading time. Items may be unnecessarily wordy. Verbosity is also an enemy to clarity. The time passed for verbose items limit the number of items that can be asked in this time and has a negative effect upon the adequacy of sampling content and the reliability of test scores.

13. Proofread each item. A highly recommended procedure is in the production of any test needs to proofread it. A good rule of thumb from expert editors is that if you spot three errors in the final proofreading phase of final test development, you have probably missed one error.

III. Writing the Stem

14. Make directions as clear as possible. The stem should be written in a way that the test taker knows what the focus of the item is. For example, the two following questions, are examples of unclear first and clear directions in the stem as demonstrated in the second example.

    Bad example:
    Plant in a flower pot fell over. What happened?

    Clear example:
    A plant growing in a flower pot was turned on its side. A week later, what would you expect to see?

15. Make the stem as brief as possible.

16. Place the main idea in the stem not in the choices. Item's stems should always contain the main idea. The test taker should always know what is being asked in the item after reading the stem. When an item fails to perform as intended with a group of students who have received appropriate instruction, there are often many reasons. One reason is that the stem did not present the main idea.

17. Avoid irrelevant information (window dressing). Some items contain entire sentences that contain that have nothing to do with the problem stated in the stem. One reason is to make the item look more life like or realistic. For many good reasons, listed in guidelines 9, 11, 12, 14, and 15 window dressing is not needed. However, there are times when verbage in the stem may be appropriate in problems with test takers sorts through information and distinguishes between relevant and irrelevant information to solve a problem. In this case, excessive information may be necessary. In this instance, the purpose of excessive information is to see if the examinee can separate useless from useful information.

18. Avoid negative words in the stem. Researchers in the field have shown that negative words in the stem have negative effects on students and their responses to such items. They also have found that students have difficulty with negatively phrased items. Negatively phrased items require twice as much working memory as positive phrase forms of such an item. Negative words appearing both in the stem and in one or more options might require four times as much working memory as a positively phrased equivalent item. Other experts have noted that the human brain cannot think in the reverse of an idea with any ease at all. If a negative term is used (such as not, except), it should be emphasized by placing it in bold type, capitalizing it, underlining it, or all of the above.

IV. Writing the Choices

19. Use as many choices as possible but three seems to be the natural limit. In most of TCMtests.com tests we require four choices or four distractors. Unless otherwise requested, please provide four.

20. Vary the location of the right answer according to the number of options. Assign the position of the correct answer randomly. At TCMtests.com, our computer technology randomly assigns the options. For your purposes as an item writer, please make 'A' the first option ALWAYS the correct answer. For our purposes at TCMtests.com, please list the source underneath 'A,' the book, author, date, publisher, and page number.

21. Place options in logical or numerical order. The answers should always be arranged in ascending or descending numerical order. Always place the decimal points in quantitative answers. Decimal points should be aligned for easy reading. Always with a zero in front of the decimal point such as 0.250 as opposed to .250.

22. Keep choices independent. Choices should not be overlapping. If options are overlapping, these options are likely to give a clue to the test taker about the correct answer and the distractors. If values are given, do not use overlapping options such as:

    What age range represents the physical peak of life?

    1. 11-15 years
    2. 13-19 years
    3. 18-25 years
    4. 24-32 years

    Avoid the overlapping options in all cases.

23. Keep choices homogenous in content and grammatical structure. If the correct answer is shorter or more specific or stated in other language perhaps more technical or less technical, this might make the items easier to identify and correctly answer.

24. Keep the length of choices about the same. One common fault in item writing is to make the correct answer the longest. Please do not do that.

25. None of the above should be used sparingly. As the last option, 'none of the above' is easy to construct. Research has increased controversery over these guidelines and has split authors over this option as a distractor.

26. Avoid using 'all of the above.' The use of 'all of the above' is a good device for capturing information where one, two, or even three right answers exist. However, the use of this choice may help test wise test takers.

27. Avoid options that give clues to the right answer. These include the following, i.e., specific determiners. Specific determiners are so extreme that seldom are they the right answer. These include such terms as 'always,' 'never,' 'totally,' 'absolute,' or 'completely.' A specific determiner may occasionally be the right answer. In these instances, their use is justified if the distractors also contain other specific determiners.

28. Avoid clang associations. Sometimes a word or phrase that appears in the item's stem will also appear in the list of choices and the word or phrase will be the correct answer. If a clang association exists and a word or phrase is not the correct answer, the item may be a trick question.

29. Options should be homogenous with respect to grammar. Sometimes a grammatically error in writing may lead a test taker to the right answer.

30. Options should be homogenous with respect to content. A wise test taker student is likely to choose the heterogenous option.

31. Blatantly absurd and ridiculous options. When writing that third or fourth option, there is a temptation to develop a ridiculous choice either as humor or out of desperation. In either case, the ridiculous case will seldom be chosen and in taking board examinations such as TCMtests.com preparation, we do not use humor or wish to have trick questions. Please avoid this at all costs.

32. Make all distractors plausible. Multiple choice questions are used to measure knowledge and cognitive skills. Therefore, the right answer must be right and wrong answer must be wrong. The key developing wrong answer is plausibility. Plausibility refers to the idea that an item should be correctly answered but should be correctly answered by those who possess a high degree of knowledge and incorrectly answered by those who possess a low degree of knowledge. The plausible distractor will look like a right answer to those that lack this knowledge. The effectiveness of a distractor can be statistically analyzed where only 3% of the students choose that option. One will conclude that this option is very implausible. Writing plausible distractors comes from hard work and is the most difficult part of a multiple choice item writing.

33. Use typical errors of students when writing distractors. The good plausible distractor comes from a common understanding of common student errors and writing them as a distractor.

34. Use humor if it is compatible with the teacher. Avoid humor in formal testing situations. TCMtests.com is a formal testing situation, therefore, please avoid humor.

35. Specific guidelines for writing matching items. The 31 general guidelines enumerated above should be used, however, for writing matching items. The set of choices for matching items that is homogenous as to content. The benefit of a matching format is the measurement of understanding of a single learner's outcome; the homogeneity of content is a characteristic of a set of matching items. Also, the number of choices should not equal the number of items.

    The seven rules for the matching format:

    1. Provide clear directions to the students about how to select an option for each stem.
    2. Provide more stems than choices.
    3. Make choices homogenous.
    4. Put choices in logical order or numerical order.
    5. Keep the stems longer than the options.
    6. Number stems and use letters for options (A, B, C).
    7. Keep all items on a single page or bordered section of the page.