Advantages and Disadvantages of Observational and Experimental Studies For Diabetes Research
Advantages and Disadvantages of Observational and Experimental Studies For Diabetes Research
of observational and
experimental studies for diabetes
research
Sarah Wild, University of Edinburgh
BIRO Academy 2nd Residential
Course
January 2011
Outline
• Hierarchy of research evidence
• Advantages of trials
• Limitations of trials
• Advantages of observational studies
• Limitations of observational studies
• Summary
Levels of evidence
for interventions
Evidence obtained from a systematic review of all
relevant randomised trials.
Evidence obtained from at least one properly-
designed randomised controlled trial.
Evidence from well-controlled trials that are not
randomised; or well-designed cohort or case-control
studies; or multiple time series (with or without the
intervention).
Opinions of respected authorities; based on clinical
experience; descriptive studies; or reports of expert
committees.
Levels of evidence for
anecdote-based medicine
BUT
• Non-blinded RCTs may overestimate
treatment effects eg estimates of effect
from trials with inadequately concealed
allocation have been 40% larger than
clinical trials with adequately concealed
random allocation
Disadvantages of RCTs
• More difficult to design and conduct than
observational studies
– ethical issues
– feasibility
– costs
• Still some risk of bias and generalisibility
often limited
• Not suitable for all research questions
Limitations of trial design
Trials may be
• Unnecessary eg very effective intervention and
confounding unlikely to explain effects (eg
insulin for T1DM)
• Inappropriate eg measurement of infrequent
adverse outcomes, distant events
• Impossible eg ethical issues if outcome harmful,
widespread use of intervention, size of task
• Inadequate eg limited generalisibility – patients,
staff , care not representative
HR compared to MF monotherapy
• 1.08 (0.96-1.21) for MF+SU
• 1.36 (1.19-1.54) for SU monotherapy
• 1.42 (1.27-1.60) for insulin
HR compared to untreated DM
• 0.90 (0.79-1.03) for MF
Source: Currie et al Diabetologia 2009;52:1766-1777
Diabetes Rx and cancer mortality
• 10,309 new users for >1 year of metformin (MF) or
sulfonylureas (SU) 1991-1996 with an average follow-up
of 5.4 ± 1.9 years (means ± SD) identified from
Saskatchewan Health administrative databases. Mean
age 63.4 ± 13.3 years, 55% men.
• Cancer mortality over follow-up was 4.9% (162 of 3,340)
for SU monotherapy users, 3.5% (245 of 6,969) for MF
users, and 5.8% (84 of 1,443) for insulin users
After adjustment for age, sex, insulin use, co-morbidity HR
for cancer mortality compared with the MF cohort
• 1.3 [95% CI 1.1–1.6]; P = 0.012) for SU users
• 1.9 (95% CI 1.5–2.4; P < 0.0001) for insulin users
Sources: http://clinicaltrials.gov/ct2/show/NCT01112839
http://clinicaltrials.gov/ct2/show/NCT01101438
Received: 29 Aug 2008
Accepted: 26 May 2009
Published online: 30 Jun 2009
• Smoking
• Diet including alcohol
• Physical activity
• Socio-economic status
• Ethnicity
• Reproductive history
• Cancer treatment (surgery, chemotherapy, radiotherapy)
Incident Cancers in Large Randomized
Trials of Glucose Lowering.
Obesity Hyperinsulinaemia
Poor control
Treatment
Treatment Cancer Death
Insulin Diabetes
resistance
Good control
Beta cell failure
Genes
Summary
• Well conducted RCTs are the optimum study
design to test beneficial effects of treatment in a
selected populations because they have the
lowest risk of bias and confounding
• Observational studies have a role to play in
– generating hypotheses
– investigating drug effectiveness in real world
– describing rare, adverse outcomes in large
populations
BUT role of bias and confounding should be
considered in the interpretation of findings
Further reading
• A proposed method of bias adjustment for meta-analyses of
published observational studies Thompson S et al Int. J. Epidemiol.
(2010) doi: 10.1093/ije/dyq248