User Tools

Site Tools


stats:diagnostic_testing

diagnostic tests - sens, spec, PPV, NPV, etc

see also:

mathematical risk = when we don’t know what the outcome is, but we do know the distribution of the outcomes

mathematical uncertainty = when we don’t know what the outcome is, and we don’t know the distribution of outcomes

“ There are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know. ”

United States Secretary of Defence, Donald Rumsfeld

David Newman's lecture - "Health Care's Most Critical Reform: The Doctor" - highlighting how the medical profession continues with disproven and even harmful therapies - a must watch for all clinicians

Assessing a new diagnostic test in a journal:

is the paper worth reading?

  • is this test relevant to my practice?

are the results of the study valid?

  • was there an independent, blind comparison with a reference (“gold”) standard?
  • was the diagnostic test evaluated in an appropriate spectrum of patients (like those to whom it would be offered in practice)?
  • was the reference standard applied regardless of the result?

are the valid results of this study important?

  • what were the results?
    • see below for sensitivity, specificity, etc.
  • how precise were the results?
    • were the results expressed with a range or confidence interval?
  • were likelihood ratios presented or data provided to calculate them?

can I apply these important, valid results to my patients?

  • will the test be available, affordable, accurate & reliable in my setting?
  • if the test is not available, are there sufficient details to enable it to be replicated?
  • can you generate a clinically sensible estimate of your patient's pre-test probability?
  • will the resulting post-test probabilities affect your management & help your patient?
  • would the consequences of this test help your patient?

Diagnostic test results

Test resultDisease presentDisease absentTotal
test positive a b a+b
true positive false positive
test negative c d c+d
false negative true negative
all tests a+c b+d

Using the Diagnostic Test

  • For each diagnostic test, we need to know the following characteristics:

1. Sensitivity:

  • ie. % of all +ve cases picked up by the test
  • ie. 100 - (% false negatives);
  • thus, = (case positives + test positives) x 100 / (case positives)
  • ie. = true positives x 100/ case positives

2. Specificity:

  • ie. % of all +ve tests that were case +ve;
  • ie. 100 - (% false positive);
  • thus, = (case negatives + test negatives) x 100 / (case negatives);
  • ie = true negatives x 100 / case negatives
  • NB. in determining the cut-off point in a test, there is always a trade off between sensitivity & specificity (see below);

3. Positive predictive value (varies with disease % incidence!):

  • ie. chance of a +ve test reflecting true presence of dis.;
  • ie. (case positives + test positives) x 100 / (test positives)
  • ie. a / (a + b)
  • or using sensitivities (s), specificity (sp) and pre-test probability of disease (y) in percentages:
    • = (s x y/100) / [(s x y/100)+(100-y)-(sp(100-y)/100)]

4. Negative predictive value (varies with disease % incidence!):

  • ie. chance of a -ve test reflecting true absence of dis.;
  • ie. (case negatives + test negatives) x 100 / (test negatives);
  • ie. d / (c + d)
  • or using sensitivities (s), specificity (sp) and pre-test probability of disease (y) in percentages:
    • = [(sp(100-y)/100] / [(y-(s x y/100)) + (sp(100-y)/100)]

5. Likelihood ratios:

Baye's nomogram for determining post-test probability of a disease:
  • This nomogram can also be used to determine the range of pre-test probabilities between which performance of the test may be useful, below which, wait & see is best, & above which, immediate treatment may be warranted. To use this need to know the threshold of post-test probability of disease at which you will decide to treat the disease;
  • Nomogram for using Likelihood Ratios (LRs) to convert pre-test probabilities into post-test probabilities for diagnostic test results with a known LR
  • Drag the blue arrows to the values you have for your patient's pre-test probability (%) and the test result's LR, and read off the post test probability from the red arrow on the right.

6. Positive likelihood ratio (LR+):

  • = probability of +ve test in those with disease / probability of +ve test in those without disease
  • ie. true positive rate / false positive rate
  • ie. = (a/[a+c]) / (b/[b+d])
  • ie. = sensitivity / (1 - specificity)
  • thus stress ECG for IHD with sens.71% spec.73% has LR+ of 2.6 and LR- of 0.4
  • for CT-PA for PE, assuming sens 80% & spec. 90% then has LR+ of 8 and LR- of 0.2

7. Negative likelihood ratio (LR-):

  • = probability of -ve test in those with disease / probability of -ve test in those without disease
  • = false negative rate / true negative rate
  • = (a/[a+c]) / (b/[b+d])
  • = (1- sensitivity) / specificity

8. Will the result make a difference to our management plan?

  • Will a +ve result increase our pretest prob. of disease presence significantly or will a neg. result increase our pre-test prob. of absence significantly?
  • Would we prefer to over-treat rather than under-treat?
    • eg. if we know a pt with “typical” ischaemic PIC has 90% likelihood of having CAD, then performing a stress ECG test (eg. sens.71% spec.73%) on this pt. may not significantly increase our diagnostic accuracy to change our management:
      • ie. stress ECG +ve → PPV 96% → likely to have CAD;
      • -ve → NPV 21% → no CAD unlikely!!
      • however, if we had not known the pre-test prob .of disease (90%), then +ve result would have helped but a -ve result would not have helped!
  • THUS we need to have an idea of pre-test probability of the presence of disease as well to interpret the result.

9. other factors requiring consideration:

  • availability of, & time duration to perform test;
  • cost of test;
  • invasiveness of test & possible complications;

10. Compute the likelihood of whether doing the test will cause more harm than good

Other references

stats/diagnostic_testing.txt · Last modified: 2014/01/11 12:05 (external edit)