Results Doctor: what to do when alpha, KMO, HTMT or a hypothesis fails

Worked example: low Cronbach's alpha on real data

What should you do when Cronbach's alpha is low? Check for data errors first (reverse-worded items, missing-value codes, careless responses), then consider dropping one item for a stated reason, and if alpha is still below the threshold, report it honestly as a limitation. The example uses the Openness scale (O1–O5, 6-point) of the public bfi data set (Revelle, 2026), the same data as on our verification page, so you can reproduce it in the Cronbach's alpha calculator.

A model Results paragraph in APA 7 style:

The five-item Openness scale had a Cronbach's alpha of .603, 95% CI [.578, .626]. Item O4 had a corrected item-total correlation of .220, below .30, and its content concerns reflection rather than intellectual curiosity, so it was removed. After removing O4, Cronbach's alpha was .614, which is acceptable only in exploratory research (Hair et al., 2010); this is a limitation of the study.

How to read the diagnosis

Each issue gets a severity (Serious, Needs action, Note, Info), the numbers it is based on, what it means, and a numbered list of options in the order to try them. Every option states what to do, when it is legitimate, what follows from it, how to write it up and the source. Tick the steps you actually carry out and type in the result after re-running: the change log and the procedure paragraph update themselves, ready for your supervisor and your appendix. Everything runs in your browser.

The order matters because many "failed" results come from undetected data errors:

1. Reverse-worded items not recoded. A negative item-total correlation or a negative loading almost always means this. Dropping the item hides an error you could fix.
2. Data entry and missing codes. A code of 99 not declared under Missing Values, or out-of-range values, distorts every index.
3. Low-quality responses. Screen only with rules written down in advance (attention checks, straight-lining, very short completion time), applied once to all cases.
4. Wording and translation. Double-barrelled or mistranslated items explain why an item performs poorly.
5. Dropping an item for a reason. Only with a pre-specified rule (for example item-total correlation below .30), at least 3 items left, the construct's content still covered, one item at a time, reporting before and after.
6. Reporting a limitation. When nothing justifies a change, keep the number and explain it.
7. Collecting more data, only when valid responses fall below the planned or minimum sample size. Collect the shortfall once and re-analyse; never check p after each batch.

HTMT above .85: what to do

HTMT above .85 (or .90 for conceptually close constructs) means respondents did not clearly separate the two constructs (Henseler et al., 2015). In the SmartPLS example, HTMT between HL and YD is .913, so the tool flags it as serious. Put the two definitions and their items side by side, bootstrap to see whether the HTMT confidence interval includes 1, and merge the constructs or drop an overlapping item only for a theoretical reason, reported as a departure from your proposal. Dropping items one by one until HTMT dips below .85 is not a legitimate fix.

Thresholds the tool uses

• Cronbach's alpha: below .60 serious, .60–.70 needs action, above .95 suggests redundant items; item-total correlation below .30.
• KMO following Kaiser (1974): below .50 unacceptable, .50–.60 miserable. Bartlett's test with Sig. of .05 or more fails.
• Rotated matrix: Howard's (2016) .40–.30–.20 rule for low loadings and cross-loadings.
• AVE at least .50, CR at least .70, outer loadings at least .708, HTMT below .85 or .90.
• VIF of 5 or more (PLS-SEM: 3.3), |skewness| above 2, |kurtosis| above 7.
• Sample size: 5 cases per item for EFA, Green (1991) for regression, 155 for PLS-SEM (Kock & Hadaya, 2018).

These are common conventions, not laws. If your department uses another threshold (for example loadings of .50), follow it and state it in your Methods chapter. Full references with DOIs appear at the end of each diagnosis.

An unsupported hypothesis is not a failure

p > .05 does not ruin a thesis. It is a valid result: the data do not provide enough evidence for that effect in your context. Mark it "Not supported" in the hypothesis table with the actual β and p, then write a separate Discussion paragraph comparing it with prior studies. Note that p ≥ .05 does not prove there is no effect; that conclusion needs an equivalence test.

What you must never do

• Edit, add, delete or "balance" responses, fill in questionnaires yourself or buy ready-made responses. That is fabrication (Steneck, 2006); in Fanelli's (2009) meta-analysis about 2% of scientists admitted to it.
• Try dropping items or cases one by one and keep the combination that works. Simmons, Nelson and Simonsohn (2011) showed that four such flexible choices together raise the false-positive rate from 5% to 60.7%.
• Switch methods (regression to PLS-SEM, two-tailed to one-tailed) just because the new one gives p < .05, or rewrite hypotheses after seeing the results (HARKing).

Frequently asked questions

Will I fail if Cronbach's alpha is below .70?

Not automatically. Examiners judge how you handled it: whether you checked reverse-worded items, missing codes and careless responses, dropped items only for a reason, and reported honestly. Alpha from .60 to just under .70 is accepted in exploratory research (Hair et al., 2010) if you say so.

Why is my Cronbach's alpha low?

Most often: reverse-worded items not recoded, too few items, mistranslated items, careless responses, or a scale that mixes several facets. Alpha also rises with the number of items (Cortina, 1993), so a 3-item scale reaches .70 less easily than an 8-item scale of the same quality.

Will collecting more responses raise alpha?

Not systematically. More cases narrow alpha's confidence interval; they do not push alpha up. Collect more only when valid responses are below your planned sample size, and do it once.

Can I delete cases to get alpha to .70?

No, if the reason is the alpha value. Deleting cases is legitimate only when they break screening rules written beforehand (failed attention check, straight-lining, too-short completion time), applied to every case, with the number removed reported.

References for this guide

• Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. https://doi.org/10.1037/0021-9010.78.1.98
• Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE, 4(5), e5738. https://doi.org/10.1371/journal.pone.0005738
• Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Pearson Prentice Hall.
• Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
• Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36. https://doi.org/10.1007/BF02291575
• Revelle, W. (2026). psych: Procedures for psychological, psychometric, and personality research (R package version 2.6.9). https://CRAN.R-project.org/package=psych
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
• Steneck, N. H. (2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics, 12(1), 53–74. https://doi.org/10.1007/PL00022268