Exponential distribution

The Exponential distribution describes the interarrival times between events from a Poisson process. The most common examples are the time it takes to be seen by a teller at a bank, or the time spent waiting for a bus at a bus stop.1

Assumptions

A Poisson process is a type of number-generating process in which “events” of some kind happen irregularly, but with a long-term average rate.

Poisson processes are most often observed over time (such as the number of large meteor strikes observed over 500 years), but they can also be observed over space (such as the number of craters observed across 500 square kilometers of the moon’s surface), or combinations of the two, or even more exotic concepts (such as observations across the electromagnetic spectrum). In these notes, we will continue to assume the observation windows are time-based.

The formal definition of a Poisson process requires some time to unpack. For now, let’s say the following:

  • Let \(N(t)\) be a counting process, that is, a non-negative, integer-valued, non-decreasing function with the property that for all \(t \ge s\) we can say that \(N(t) - N(s)\) represents the number of events which happen in the window \([s,t]\)

  • Let \(\lambda\) represent the long-term average rate of events which occur per unit of observation: \(\mathbb{E}[N(1)] = \lambda\) and \(\mathbb{E}[N(t)] = t\lambda\)

    • As a corollary, notice that \(N(0) = 0\), meaning that events must always arrive one at a time, and never two or more simultaneously

  • Let the event counts in mutually exclusive observation windows be independent of each other, which we could write several ways, e.g. for all \(t \ge s\), then \(N(t - s) \perp N(s)\)

Then we say that \(N\) is a Poisson process, that \(P(N(t) \le k)\) has a Poisson CDF with rate \(\lambda\), and that if \(T\) is the arrival time of the next event (i.e., \(T = \min(t: \,N(t)=1)\)) then \(P(T \le t)\) has an exponential distribtuion with rate \(\lambda\).

A key concept to the Poisson and the Exponential distributions is that they describe the same counting process. The Poisson (discrete) counts the number of events for a fixed window of time, and the Exponential (continuous) counts the time elapsed until the next event.

Definition

\[\begin{array}{ll} \text{Support:} & \mathbb{R}^+ \\ \text{Parameter(s):} & \lambda,\text{ the rate in events per unit of time }(\lambda \gt 0) \\ \text{PMF:} & f_X(x) = \lambda e^{-\lambda x} \\ \text{CDF:} & F_X(x) = \left\{\begin{array}{cl} 0, & \quad x \lt 0 \\ 1 - \lambda e^{-\lambda x}, & \quad x \ge 0 \end{array}\right\} \\ \text{Mean:} & \mathbb{E}[X] = \frac{1}{\lambda} \\ \text{Variance:} & \mathbb{V}[X] = \frac{1}{\lambda^2} \\ \end{array}\]

Visualizer

In the visualizer below, notice that the Exponential distribution is self-similar — it displays a fractal-like property of keeping the same curve for every choice of \(\lambda\). If you do not fix the x-axis, then only the scale of the axes will change, not the shape of the PDF. However, once we fix the x-axis, you’ll better notice how the shape changes along a specific stretch from \(x=0\) to \(x=2\).

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| viewerHeight: 650

library(shiny)
library(bslib)

ui <- page_fluid(
      tags$head(tags$style(HTML("body {overflow-x: hidden;}"))),
  title = "Exponential distribution PDF",
  fluidRow(plotOutput("distPlot")),
  fluidRow(column(width=8,sliderInput("lambda", "Rate (lambda)", min=0, max=20, step=0.1, value=2)),
           column(width=4,checkboxInput("fax", "Fix x-axis"))))

server <- function(input, output) {
  xseq <- reactive(seq(0,list(4/input$lambda,2)[[input$fax+1]],length.out=200))
  output$distPlot <- renderPlot({
    plot(x=xseq(),y=dexp(xseq(),input$lambda),
         xlim=list(NULL,c(0,2))[[input$fax+1]],ylim=NULL,
         type='l',main=NULL,xlab='x (Arrival time)',ylab='Density',lwd=3)})
}

shinyApp(ui = ui, server = server)

Relations to other distributions

  • The Geometric distribution (specifically type (ii) mentioned above which also counts the last success) forms a discrete approximation to the Exponential distribution with \(\lambda = p\), and the quality of this approximation grows for \(p\) near 0. For example, compare:
pgeom(q=20,prob=0.05)
[1] 0.6594384
pexp(q=21,rate=0.05)
[1] 0.6500623

  1. Though in real life, neither of these are well-approximated by the Exponential distribution!↩︎