Replication Analysis

Definition

Replication Analysis is the process of running the same simulation model multiple times using different random number seeds to obtain a statistically significant estimate of system performance.

Why It Matters

A ‘sample of one’ is a dangerous gamble in stochastic systems. If you don’t run multiple replications, you are making decisions based on luck rather than law. In high-stakes fields like aerospace, this leads to catastrophic ‘Black Swan’ failures.

Core Concepts

• Stochastic Variation: Because simulations use random distributions, a single run is only one possible outcome (a “sample of size one”).

• Independence: Each replication must be independent (different random seeds) to ensure the results are not biased.

• Confidence Intervals: Used to calculate the range within which the “true” system mean likely lies (e.g., $95\%$ Confidence Interval).

• How to read: “The ninety-five percent confidence interval.”

  • Meaning: If you repeated the experiment many times, 95% of such intervals would contain the true mean. Wider interval = less precision.

• Determining Number of Replications ($n$): Calculated based on the desired precision (half-width) and the observed variance ($s^2$):

  • $n = (\frac{z \cdot s}{h})^2$, where $h$ is the desired half-width.

• How to read: “The N equals the quantity z times s divided by h squared.”

  • Meaning / when to use: $z$ is the z-score for your confidence level (1.96 for 95%), $s$ is sample standard deviation, $h$ is desired margin of error. More replications needed when variance is high or precision demands are tight.

• Terminating Systems: Replication is the primary method of analysis for systems that start empty and idle and have a natural end.

Connected Concepts

• Terminating Systems
• Nonterminating Systems
• Central Limit Theorem
• Statistical Validity