You prove validity by presenting evidence, not by reporting a coefficient. Four kinds of evidence do almost all the work in a South African dissertation: expert review of the items, the factor structure of the responses, whether items that should agree do, and whether constructs that should differ actually do. Reliability is a separate question, and a reliable instrument can be reliably wrong.
What is the difference between reliability and validity?
Reliability is consistency: do the items in your scale move together, and would the instrument give a similar answer if you administered it again? Validity is accuracy: is the instrument measuring the thing you say it measures?
The two come apart in one direction. A scale can be highly consistent and measure the wrong construct entirely — five items that all reliably measure job satisfaction while you claim they measure organisational commitment. This is why reporting Cronbach’s alpha and stopping is the most common measurement error in South African dissertations. Alpha answers the reliability question. Nobody has asked the validity question yet.
The alpha computation itself, inside the wider analysis workflow, is covered in our step-by-step guide to analysing questionnaire data in SPSS. This article is about the argument you build around it.
What is content validity, and how do I show it?
Content validity asks whether your items cover the construct properly — nothing important missing, nothing irrelevant included. It is established before you collect data, by expert judgement, and it is the one form of validity available to every student regardless of sample size.
The standard procedure is a content validity index. Give a panel of subject experts each item and ask them to rate its relevance on a four-point scale from irrelevant to very strongly relevant. Then:
- Item-level (I-CVI) = the number of experts rating an item 3 or 4, divided by the total number of experts. An I-CVI of 0,78 or above is treated as acceptable; items below it get revised on the panel’s feedback or dropped.
- Scale-level (S-CVI/Ave) = the average of the I-CVI values across all items. 0,90 or above indicates acceptable content validity for the scale.
Those two thresholds are the ones applied in current published instrument-validation work, and they are what an examiner will recognise. Other approaches exist — Lawshe’s content validity ratio and modified kappa among them — so if your supervisor prefers one, use it and say which you used.
Panels in published studies commonly run to around ten experts. At honours or master’s level five to eight is usual and defensible; what matters is that you name them by role, not by name, and state how many rated each item.
Practical note for South African projects: your expert panel is also the cheapest way to catch items that do not travel. An item written for a North American workplace that assumes annual performance bonuses, private medical insurance or a commute by car will read as irrelevant to a South African sample, and an expert panel will say so before your respondents do it silently by skipping the question.
What is construct validity, and do I need factor analysis?
Construct validity asks whether the pattern in your data matches the structure your theory predicts. If your instrument claims to measure three dimensions, the responses should group into three dimensions.
The usual evidence is factor analysis. Exploratory factor analysis (EFA) asks what structure is in the data; confirmatory factor analysis (CFA) tests whether a specified structure fits. Published validation studies frequently split the sample and run both — EFA on one half, CFA on the other — and check that the data suit factoring at all using the Kaiser–Meyer–Olkin measure and Bartlett’s test of sphericity before proceeding. Common fit criteria reported for a CFA include CFI, TLI and IFI values above 0,900.
Do you need it? If you developed or substantially adapted the instrument, yes, and your sample must be large enough to support it — sample size is a design decision made before collection, as our answer to how many participants you need for a dissertation sets out. If you are using an established, previously validated instrument unchanged, on a population similar to the one it was validated on, citing that validation evidence is normally sufficient at master’s level. Ask your supervisor which of those two situations you are in, in writing, before you collect.
What are convergent and discriminant validity?
These are the two halves of the same test, and they matter most in the structural equation modelling that South African management and industrial psychology dissertations use heavily.
Convergent validity asks whether items claiming to measure one construct actually converge on it. The conventional evidence, and the thresholds published validation studies apply:
| Statistic | Threshold treated as satisfactory | What it tells you |
|---|---|---|
| Standardised factor loading | ≥ 0,50 | Each item is meaningfully related to its own construct |
| Average variance extracted (AVE) | ≥ 0,50 | The construct explains more of its items’ variance than error does |
| Composite reliability (CR) | ≥ 0,70 | The construct’s items are jointly reliable indicators of it |
Discriminant validity asks the opposite: that constructs which are supposed to be different are empirically distinguishable, rather than two names for one thing. If two of your constructs correlate at 0,92, you do not have two constructs.
Note what the AVE and CR row implies for your alpha. CR and Cronbach’s alpha both speak to reliability; AVE speaks to validity. Reporting AVE alongside CR is the cheapest way to show an examiner you understand that they answer different questions.
What if my questionnaire was written in another country?
Most instruments a South African postgraduate will use were developed elsewhere, usually in English, usually on a Western sample. Two things follow, and both belong in Chapter 3.
First, get permission. Published scales are frequently copyrighted, and some are licensed commercially. Check the original article for a use statement and write to the corresponding author if there is none. Keep the reply — some faculties want it attached to the ethics application, alongside the other documents our guide to ethics clearance at a South African university covers.
Second, adaptation is not a synonym for translation. Changing “principal” to “school principal”, replacing a currency, or adjusting an item about health insurance to fit a South African medical-aid context are all adaptations, and each one means the original validation evidence no longer transfers unchanged. Say what you changed and why, then re-establish content validity on the adapted version.
How do I translate a questionnaire for a South African sample?
South Africa has twelve official languages, and a questionnaire administered in a respondent’s second or third language measures reading comprehension as well as the construct. If you translate, use the forward–backward method that published cross-language validation studies use:
- Forward translation into the target language by at least one translator fluent in both languages and familiar with the subject matter.
- Independent back-translation into the original language by a different translator who has not seen the original instrument.
- Reconciliation: compare the back-translation against the original item by item, and resolve every discrepancy in a documented decision.
- Expert panel review of the translated version, producing I-CVI and S-CVI/Ave on the translated items rather than the originals.
- Pilot with a small group of speakers from the target population and record what they found ambiguous.
Expected output: a translation paragraph in Chapter 3 naming each step, each translator’s qualification in general terms, and the CVI values obtained on the translated version.
Two South African cautions. Translating into one language does not solve a multilingual sample — administering an English version to some respondents and a translated version to others introduces a difference you must test for, not ignore. And if translation is beyond your project’s scope, say so in your limitations and explain what you did instead, such as piloting for comprehension in plain English. A documented limitation costs far less than an undocumented translation.
What should I actually write in Chapter 3?
One paragraph per evidence type, in this order, each naming the statistic, the threshold you applied and the value you obtained:
- Instrument source. Where the scale came from, whether it was adapted, what changed and why, and that permission was obtained.
- Content validity. Panel size and composition, the rating scale used, I-CVI and S-CVI/Ave values, and what happened to items below threshold.
- Pilot. How many respondents, what changed as a result.
- Construct validity. KMO and Bartlett’s test, the factor structure obtained, and fit indices if you ran a CFA.
- Convergent and discriminant validity, if applicable: loadings, AVE, CR, and how you established that constructs are distinguishable.
- Reliability. Cronbach’s alpha per subscale, not one alpha for the whole instrument, because an alpha computed across dimensions that are not supposed to be one dimension is meaningless.
That last point catches more students than any threshold. If your instrument has four subscales, report four alphas.
What does an examiner actually check?
Three things, quickly, and in this order. Whether you distinguished reliability from validity at all. Whether the evidence you present matches the claim you make in your abstract. And whether the thresholds you applied are named and cited rather than asserted.
A questionnaire with modest values that are honestly reported and discussed survives examination. A questionnaire with excellent values and no stated thresholds invites the question of where the standard came from — and that question is harder to answer at a viva than it is to answer now, in writing, in Chapter 3.
Which test you then run on the resulting scores is a separate decision, set out in our decision table for which statistical test to use for a dissertation. And the validity argument should already have been sketched in your proposal — our guide to writing a research proposal for a South African university shows where it belongs.
Keeping the argument together
Validity evidence is collected across months — the expert panel in March, the pilot in May, the factor analysis in September — and it is usually assembled into one section the week before submission, from memory. Tesify keeps the methodological sources you relied on attached to the chapter as you draft, so the threshold you applied in March is still citable in September. The judgements stay yours.
Build your methodology chapter in Tesify
Frequently asked questions
Does a high Cronbach’s alpha mean my questionnaire is valid?
No. Alpha measures internal consistency — whether the items agree with one another. An instrument can be highly consistent and still measure the wrong construct. Validity requires separate evidence.
What is an acceptable content validity index?
An item-level index (I-CVI) of 0,78 or above is treated as acceptable in published validation work, and a scale-level average (S-CVI/Ave) of 0,90 or above indicates acceptable content validity for the scale as a whole.
How many experts do I need on a content validity panel?
Published studies commonly use around ten. At honours or master’s level five to eight subject experts is usual and defensible. Report the number and their roles, and state how the ratings were collected.
What AVE and CR values should I report?
An average variance extracted of 0,50 or more and a composite reliability of 0,70 or more, with standardised factor loadings of 0,50 or more, are the conventional indicators of satisfactory convergent validity.
Do I have to run factor analysis?
If you developed or substantially adapted the instrument, yes, and you need a sample large enough to support it. If you are using an established instrument unchanged on a comparable population, citing its published validation is normally accepted at master’s level — confirm which case applies with your supervisor.
Do I need permission to use someone else’s questionnaire?
Usually yes. Check the original article for a use statement and write to the corresponding author if there is none. Keep the reply; some faculties require it with the ethics application.
Does changing a few words invalidate a published scale?
It does not invalidate it, but it does mean the original validation evidence no longer transfers unchanged. Document what you changed and why, and re-establish content validity on your adapted version.
How do I translate a questionnaire properly?
Forward translation, independent back-translation by someone who has not seen the original, item-by-item reconciliation of discrepancies, expert panel review of the translated version, and a comprehension pilot. Report each step.
Can I administer English and translated versions to the same sample?
You can, but the language of administration then becomes a variable you must report and, where your design allows, test for rather than ignore. Say in Chapter 3 how many respondents completed each version.
Should I report one alpha or one per subscale?
One per subscale. An alpha computed across dimensions that theory says are distinct is not interpretable, and reporting a single figure for a multidimensional instrument is a standard examiner query.
What if my validity values come out low?
Report them, diagnose them and discuss them. Low values with a clear account of why — a small pilot, a heterogeneous sample, an adapted instrument — read far better than values that appear without a stated threshold.
Is face validity enough for an honours project?
Face validity alone is weak evidence, but a documented expert review producing CVI values is achievable within an honours timeline and is substantially stronger. Aim for content validity at minimum, and be explicit about what you did not establish.