Normal distribution

Assumptions

Although a few generating processes are provably normal, we mostly use the normal distribution in contexts where it is “close enough”, and do not require any particular assumptions.

However, keep in mind that the normal distribution, at least in theory, is:

  • Unbounded
  • Continuous
  • Symmetrical

If the process you are modeling is bounded, discrete, or asymmetrical, then the normal distribution may be a poor fit. Two common exceptions would be:

  • When the distribution is naturally bounded, but most values are observed very far from the bounds (such as the weights of passenger jets, bounded below by 0, or the returns on a stock index, bounded below at -100%)
  • When the distribution is discrete, but most values are very large or very finely subdivided (such as stadium attendance, or the current value of your bank account)

Definition

\[\begin{array}{ll} \text{Support:} & \mathbb{R} \\ \text{Parameter(s):} & \mu,\text{ the mean }(\mu \in \mathbb{R}) \\ & \sigma,\text{ the standard deviation }(\sigma \gt 0) \\ \text{PDF:} & f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \\ \text{CDF:} & F_X(x) = \Phi(\frac{x-\mu}{\sigma})\quad (\text{No closed form expression}) \\ \text{Mean:} & \mathbb{E}[X]=\mu \\ \text{Variance:} & \mathbb{V}[X]=\sigma^2 \\ \end{array}\]

Visualizer

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| viewerHeight: 650

library(shiny)
library(bslib)

ui <- page_fluid(
      tags$head(tags$style(HTML("body {overflow-x: hidden;}"))),
  title = "Normal distribution PDF",
  fluidRow(plotOutput("distPlot")),
  fluidRow(column(width=6,sliderInput("mu", "Mean (mu)", min=-10, max=10, value=0)),
           column(width=6,sliderInput("sigma", "Std Dev (sigma)", min=0.01, max=10, value=1))))

server <- function(input, output) {
  output$distPlot <- renderPlot({
    x <- seq(input$mu-3*input$sigma,input$mu+3*input$sigma,input$sigma/100)
    y <- dnorm(x,input$mu,input$sigma)
    xlims <- c(mean(c(-3,x[1])),mean(c(3,x[601])))
    ylims <- c(0,mean(c(dnorm(0),y[301])))
    plot(x=x,y=y,main=NULL,xlab='x',ylab='Density',type='l',lwd=2,
         xlim=xlims,ylim=ylims)
  })
}

shinyApp(ui = ui, server = server)

Properties

  • The normal distribution with mean \(\mu=0\) and variance \(\sigma^2=1\) is said to be the standard normal distribution and often written as \(Z \sim \mathrm{Norm}(0,1)\). The CDF of the standard normal distribution and its inverse are often abbreviated as \(F_X(x)=\Phi(x)\) and \(F_X^{-1}(x)=\Phi^{-1}(x)\), respectively.
  • If \(X\) is a normal random variable with mean \(\mu\) and variance \(\sigma^2\), then for any constants \(a,b \in \mathbb{R}\) the transformation \(aX + b\) is also a normal random variable with mean \(a\mu + b\) and variance \(a^2\sigma^2\).
  • If \(X\) and \(Y\) are two independent normal random variables with means \(\mu_X, \mu_Y\) and variances \(\sigma^2_X, \sigma^2_Y\), then their sum \(X+Y\) is also a normal random variable with mean \(\mu_X+\mu_Y\) and variance \(\sigma^2_X+\sigma^2_Y\).
  • More generally, any linear combination of any number of independent normal random variables is itself a normal random variable!

Relations to other distributions

  • The sum of the squares of \(n\) independent standard normal variates is chi-squared distributed with \(df=n\): \[\sum_{i=1}^n Z_i^2 \sim \chi_{(n)}^2\]
  • The ratio of two standard normal variates has the standard Cauchy distribution, i.e. \[\mathrm{For\ }Z_1,Z_2 \sim \mathrm{Norm}(0,1),\quad \frac{Z_1}{Z_2} \sim \mathrm{Cauchy}(0,1)\]
  • The standard normal distribution is the limit case for the Student’s t-distribution (as \(df \rightarrow \infty\)). The standard normal can be used in place of the t-distribution with little loss of accuracy for large \(df\).
  • The normal distribution with mean \(df\) and standard deviation \(\sqrt{2df}\) closely approximates the chi-squared distribution for large \(df\).
  • The Poisson distribution and binomial distribution both form discrete approximations to the normal distribution when either \(\lambda\) is very large (Poisson) or \(np\) is very large and \(p\) is not near 0 or 1.