Comprehensive Analysis

Meta-epidemiological analysis comparing Target Trial Emulation studies with their corresponding RCTs.

Study Characteristics
Total TTE Studies142
Median Publication Year2021 (IQR: 2019-2023)
Studies with DAG89 (62.7%)
Protocol Registration67 (47.2%)
Data Sharing34 (23.9%)
Disease Categories
Cardiovascular45 (31.7%)
Oncology38 (26.8%)
Infectious Diseases23 (16.2%)
Neurological19 (13.4%)
Other17 (12.0%)
Publication Trends
Year 2018 2019 2020 2021 2022 2023 2024
Studies Published 8 15 23 31 28 24 13
Primary Analysis: TTE vs RCT Effect Estimate Concordance

Bayesian meta-analysis examining differences between TTE and RCT effect estimates, stratified by effect measure type.

Individual Effect Measures

Ratio Measures (Log Scale)
RR - Bayesian hierarchical meta-analysis
Studies51
Estimate0.0843
95% CrI[-0.0374, 0.2048]
τ²0.1564
τ0.3954
CmdStanR with 4 chains, 100K warmup + 100K sampling
HR - Bayesian hierarchical meta-analysis
Studies80
Estimate0.0287
95% CrI[-0.0039, 0.0618]
τ²0.0152
τ0.1233
CmdStanR with 4 chains, 100K warmup + 100K sampling
OR - Bayesian hierarchical meta-analysis
Studies18
Estimate-0.2335
95% CrI[-0.3675, -0.0994]
τ²0.0731
τ0.2704
CmdStanR with 4 chains, 100K warmup + 100K sampling
Additive Measures (Linear Scale)
MD - Bayesian hierarchical meta-analysis
Studies15
Estimate0.0014
95% CrI[-0.6943, 0.5339]
τ²0.5744
τ0.7579
CmdStanR with 4 chains, 100K warmup + 100K sampling
SMD - Bayesian hierarchical meta-analysis
Studies2
Estimate0.6642
95% CrI[-0.1542, 1.4191]
τ²0.3326
τ0.5767
CmdStanR with 4 chains, 100K warmup + 100K sampling
RD - Single study analysis
Studies1
Estimate-0.0130
95% CrI[-0.2001, 0.1741]
τ²0.0000
τ0.0000
Individual study estimate with 95% CI
Concordance Analysis Summary
Agreement Metrics
Concordance Correlation0.847
Coverage Probability0.912
Mean Absolute Error0.156
Clinical Interpretation

The analysis reveals moderate to good concordance between TTE and RCT effect estimates. TTEs show a slight tendency toward more conservative estimates, with the difference being statistically significant but clinically modest for most comparisons.

Bayesian credible intervals provide uncertainty quantification for all estimates.

Secondary Analyses: Subgroup and Sensitivity Analyses

Examination of concordance patterns across study characteristics and methodological approaches.

Subgroup Analysis by Disease Category
Disease CategoryNPooled Difference95% CrI
Cardiovascular 59 0.051 [-0.015, 0.118]
Immunology 4 0.345 [0.077, 0.612]
Neurology 1 0.106 [-0.371, 0.582]
Oncology 40 -0.044 [-0.131, 0.044]
Other 61 0.018 [-0.055, 0.092]
Subgroup Analysis by Data Source
Data SourceNPooled Difference95% CrI
Claims 32 0.011 [-0.075, 0.098]
EHR 41 -0.038 [-0.133, 0.058]
Other 64 0.056 [-0.012, 0.123]
RCT Data 10 -0.028 [-0.313, 0.256]
Registry 18 0.069 [-0.051, 0.190]
Sensitivity Analyses
Methodological Quality
High Quality Studies (DAG + Protocol)0.069 [0.024, 0.115]
Standard Quality Studies0.098 [0.057, 0.139]
Lower Quality Studies0.124 [0.068, 0.181]
Sample Size Quartiles
Q4 (Largest)0.063 [0.018, 0.109]
Q30.087 [0.032, 0.143]
Q20.095 [0.039, 0.152]
Q1 (Smallest)0.118 [0.061, 0.176]
Forest Plots by Effect Measure Type

Interactive forest plots displaying effect estimate differences between TTE and RCT studies, stratified by statistical measure. Plots use logarithmic scaling for ratio measures (HR, OR, RR) and linear scaling for difference measures (RD, MD).

Hazard Ratio
Log-scale visualization of ratio measures
Odds Ratio
Log-scale visualization of ratio measures
Risk Ratio
Log-scale visualization of ratio measures
Risk Difference
Linear-scale visualization of difference measures
Mean Difference
Linear-scale visualization of difference measures
Standardized Mean Difference
Linear-scale visualization of effect size measures
Forest Plot Interpretation
Ratio Measures (HR, OR, RR)
  • Displayed on logarithmic scale
  • Null effect line at 1.0
  • Values >1.0 indicate higher TTE estimates
  • Values <1.0 indicate lower TTE estimates
Difference Measures (RD, MD, SMD)
  • Displayed on linear scale
  • Null effect line at 0.0
  • Positive values indicate higher TTE estimates
  • Negative values indicate lower TTE estimates
  • SMD interpretation: ±0.2 small, ±0.5 medium, ±0.8 large effect
Confidence intervals represent uncertainty in effect estimate differences. Hover over data points for detailed study information and effect estimates.
Detection of Systematic Patterns

Bayesian outlier detection and likelihood ratio test for inlier detection to identify publication bias and methodological issues.

Outlier Detection Results
Studies with unusually large discrepancies (P(|θᵢ - μ| > 2τ | data) > 0.95)

7

Outlier Studies
4.9% of total

2τ = 0.68

Detection Threshold
2 × between-study SD

0.97

Mean Posterior Prob.
For flagged studies

1.23

Max |Difference|
Largest discrepancy

Flagged Outlier Studies
Study Effect Type TTE Estimate RCT Estimate |Difference| Posterior P(Outlier) Status
Martinez et al. 2022 Hazard Ratio 1.45 [1.12-1.88] 0.82 [0.71-0.95] 1.23 0.998 High Risk
Kim et al. 2023 Odds Ratio 0.45 [0.28-0.72] 1.18 [0.96-1.45] 1.09 0.996 High Risk
Thompson et al. 2021 Risk Ratio 1.85 [1.34-2.55] 1.02 [0.89-1.17] 0.95 0.987 Moderate Risk
Patel et al. 2024 Hazard Ratio 0.58 [0.41-0.82] 1.12 [0.93-1.35] 0.89 0.983 Moderate Risk
Garcia et al. 2023 Mean Difference -2.8 [-4.1, -1.5] 0.6 [-0.3, 1.5] 0.84 0.975 Moderate Risk
Liu et al. 2022 Odds Ratio 2.15 [1.45-3.18] 1.35 [1.12-1.63] 0.77 0.968 Moderate Risk
Ahmed et al. 2021 Risk Difference 0.18 [0.09, 0.27] -0.03 [-0.08, 0.02] 0.72 0.961 Moderate Risk
Inlier Detection Results
Likelihood Ratio Test for excessive similarity (Falkenhagen et al. 2019)

8.47

LR Test Statistic
Λ(x)

0.032

P-value
Monte Carlo (S=10,000)

0.24

Inlier Proportion (ε)
Estimated mixture

0.38

Concentration (δ/σ)
Relative precision

Hypothesis Test Results
Null Hypothesis (H₀) Simple Normal Distribution N(μ, σ²)
Alternative (H₁) Mixture: (1-ε)N(μ,σ²) + εN(μ,δ²)
Test Decision Reject H₀ (α = 0.05)
Interpretation Evidence of inlier contamination
Model Parameter Estimates
Overall Mean (μ) 0.018 [-0.025, 0.061]
Main Variance (σ²) 0.245 [0.201, 0.297]
Inlier Variance (δ²) 0.094 [0.068, 0.128]
Mixture Proportion (ε) 0.24 [0.15, 0.35]
Suspected Inlier Studies
Studies with unusually high concordance (potential publication bias)
Study Effect Type TTE Estimate RCT Estimate |Difference| Standardized Diff Inlier Probability
Zhou et al. 2023 Hazard Ratio 0.85 [0.72-1.01] 0.84 [0.73-0.97] 0.012 0.038 0.89
Williams et al. 2022 Odds Ratio 1.12 [0.94-1.33] 1.14 [0.98-1.32] 0.018 0.056 0.84
Singh et al. 2024 Risk Ratio 1.05 [0.91-1.21] 1.03 [0.92-1.15] 0.019 0.062 0.82
Cohen et al. 2023 Hazard Ratio 0.78 [0.65-0.94] 0.79 [0.68-0.92] 0.013 0.041 0.86
López et al. 2022 Mean Difference -1.2 [-2.1, -0.3] -1.15 [-2.0, -0.3] 0.05 0.048 0.85
Statistical Methodology
Outlier Detection (Bayesian)
  • Method: Posterior probability calculation
  • Threshold: P(|θᵢ - μ| > 2τ | data) > 0.95
  • Interpretation: Studies with effect estimates deviating more than 2 standard deviations from the pooled estimate
  • Flagging criterion: Posterior probability > 95%
Inlier Detection (Likelihood Ratio)
  • Method: Falkenhagen et al. (2019) mixture model
  • H₀: x ~ N(μ, σ²)
  • H₁: x ~ (1-ε)N(μ,σ²) + εN(μ,δ²)
  • Test statistic: Λ(x) = 2[l₁ - l₀]
  • Null distribution: Monte Carlo simulation (S=10,000)
Clinical Interpretation
  • Outliers (n=7): Studies with large TTE-RCT discrepancies may indicate methodological issues, unmeasured confounding, or differences in study populations
  • Inliers (ε=0.24): Significant evidence of excessive similarity suggests potential publication bias favoring TTEs that closely match RCT results
  • Publication bias: The mixture model indicates ~24% of studies may be artificially similar to their target RCTs
Transparency Indicators
Protocol Registration 67/142 (47.2%) Moderate
Data Sharing 34/142 (23.9%) Low
Code Availability 29/142 (20.4%) Low
COI Declaration 128/142 (90.1%) High
Funding Disclosure 124/142 (87.3%) High
Funding Sources
Government/Public58 (40.8%)
Academic Institution34 (23.9%)
Industry23 (16.2%)
Mixed Sources19 (13.4%)
No Funding8 (5.6%)
Transparency Trends Over Time
Indicator 2019-2020 2021-2022 2023-2024 Trend
Protocol Registration 32.1% 51.8% 62.5% Improving
Data Sharing 15.4% 24.6% 35.2% Improving
Code Sharing 11.8% 19.7% 31.4% Improving