Validation is done and the questionnaire is ready to field. What gets decided at this point determines the quality of the data that comes back: the order the items appear in, how reverse-coded items are marked, and what documentation the analyst will have.
What makes this stage distinct is that none of it can be undone. Once responses start arriving you cannot reorder items, and data collected without a codebook becomes a block of numbers nobody can interpret a few months later.
Does Item Order Affect Responses?
It does. Earlier items setting the frame for later ones is a long-documented feature of survey response. Ask about job satisfaction and then life satisfaction, and the distribution differs from what you get in the reverse order.
There are two approaches to arrangement. Grouping items by factor keeps the respondent in one context and lowers cognitive load, but adjacent items pick up artificially inflated correlations, so the factor structure can look cleaner than it is. Randomizing reduces that artifact at the cost of making respondents switch context repeatedly, which raises fatigue and dropout.
Podsakoff et al. (2003), writing on common method bias, set out procedural remedies built on separating measurement contexts: putting distance between predictor and criterion items, or varying the response format. Whichever arrangement you choose, state the choice and the reason in the methods section. A manuscript that does not report its arrangement has no answer when a reviewer asks whether the factor structure is an artifact of item placement.
Reverse-Coded Items — Separate the Respondent View From the Analyst View
Keep reverse-item markings out of the respondent's questionnaire and in the codebook only. A label announcing that an item is reverse-scored changes how people answer it.
Reverse coding is computed against the scale range: subtract the raw score from 6 on a five-point scale, from 8 on a seven-point scale. Write that rule into the codebook, or you will be trying to remember whether the scale was five or seven points.
It is also worth knowing that reverse items can distort the factor structure themselves. Even in widely used scales, reverse-worded items have been reported loading as a separate factor. The same pattern appeared in the RSES reproduction analysis covered on this blog. If you keep reverse items, check separately how they behave in the factor analysis.
What Belongs in a Codebook
A codebook links questionnaire items to the variables in the data file. Without it, an analyst opening the raw data cannot tell what any column is.
| Element | Content |
|---|---|
| Variable name | Short identifier used in the statistical software |
| Item number and wording | Which questionnaire item the variable corresponds to |
| Scale and response range | Five-point Likert, 1 = strongly disagree, and so on |
| Reverse coding rule | Which items are reversed and how they are transformed |
| Factor composition and formula | Which items average or sum into each factor score |
| Missing value code | What value records a non-response |
The missing value code is the one most often left out. Record non-responses as 99, omit that from the codebook, and the means come out wrong. If you left cells blank instead, state that too.
The codebook is also a document for yourself. Requests to reanalyze during review usually arrive months after the analysis was finished.
Final Pre-Field Checks
The list shortens to a few items immediately before fielding.
Time the questionnaire with a real respondent first. Beyond about fifteen minutes, response quality drops in the later sections. For online instruments, whether a progress indicator is shown also affects dropout.
Decide required-response settings carefully. Making every item mandatory reduces missingness but raises abandonment, and respondents who abandon leave no data at all. That can introduce more bias than the missingness would have.
Confirm that item numbers and codebook variable names match what the collection tool actually produces. Survey platforms sometimes renumber items when they are moved.
What modidoc Does at the Fielding Stage
modidoc's fielding stage builds an arrangement from the finalized items and generates a codebook that includes reverse coding rules and factor scoring formulas. Reverse markings are stripped from the respondent-facing questionnaire and retained only in the codebook, and the record of decisions accumulated during development is compiled and passed to the methods section draft. This stage is implemented internally as the C9 fielding engine.
Frequently Asked Questions
What has to be in a survey codebook?
Variable names, item numbers and wording, the scale and response range, reverse coding rules, factor scoring formulas, and missing value codes. With those six, someone holding only the raw data can reproduce the analysis. Adding the collection period and sample characteristics helps when you return to the data later.
How do I handle reverse coding?
Subtract the raw score from the scale maximum plus one: from 6 on a five-point scale, from 8 on a seven-point scale. Do it before computing factor scores, and keep both the original and recoded values so the transformation can be checked later. Do not mark reverse items on the respondent's questionnaire.
Does item order affect results?
Yes. Earlier items shape the frame for later judgments, and placing items from the same factor adjacent to each other can inflate inter-item correlations. Reporting the arrangement and the reason for it is standard, and designs where common method bias is a concern should also apply procedural remedies that separate measurement contexts (Podsakoff et al., 2003).
The next article turns the development record into a methods section: which decisions to report in what order, and how to present statistical results in APA format.
Previous: SEM Sample Size and Model Identification