Beta-Delta (Quasi-Hyperbolic) Discounting

This notebook shows how to implement naive beta-delta discounting in PyLCM using a custom aggregation function $H$ and SolveSimulateFunctionPair. It covers:

1. The beta-delta discount function

2. Why naive agents need different $H$ for solve vs. simulate

3. A 3-period consumption-savings model with analytical solutions

4. Verifying numerical results against closed-form solutions

Background: Beta-Delta Discounting¶

Standard exponential discounting uses a single discount factor $\delta$ to weight future utility:

Quasi-hyperbolic (beta-delta) discounting (Laibson, 1997) introduces a present-bias parameter $\beta \in (0, 1]$ that discounts all future periods relative to the present:

The discount weights are $\{1,\; \beta\delta,\; \beta\delta^2,\; \ldots\}$. When $\beta = 1$ this reduces to exponential discounting. When $\beta < 1$ the agent is present-biased — they overweight current utility relative to any future period.

Note the key structure: $\beta$ appears once (not compounded). The weight $n$ periods ahead is $\beta\delta^n$, not $(\beta\delta)^n$.

PyLCM’s $H$ Function and the Naive Agent¶

PyLCM defines the value function recursively via an aggregation function $H$:

The default $H$ is $H(u, \mathbb{E}[V']) = u + \delta \cdot \mathbb{E}[V']$.

A naive beta-delta agent believes their future selves will be exponential discounters, but at each decision point applies present bias $\beta\delta$ to the continuation value. This requires two different $H$ implementations:

• Solve phase (backward induction): use exponential discounting with $\delta$ to compute the perceived continuation values $V^E$

• Simulate phase (action selection): use $\beta\delta$ to choose actions, applying present bias to the perceived $V^E$

SolveSimulateFunctionPair wraps these two implementations so you can call simulate once with a single params dict.

Why not just use $H(u, V') = u + \beta\delta V'$ for both phases? Because the recursion would compound $\beta$ at every step, giving effective weights $(\beta\delta)^n$ instead of the correct $\beta\delta^n$. Splitting the phases ensures $\beta$ is applied exactly once — at the action selection step.

Setup: A 3-Period Model¶

We use a minimal consumption-savings model:

• 3 periods: $t = 0, 1$ (decisions), $t = 2$ (terminal)

• 1 state: wealth $w$

• 1 action: consumption $c$

• Utility: $u(c) = \log(c)$

• Budget: $w' = w - c$ (interest rate $R = 1$)

• Constraint: $c \le w$

• Terminal: $V_2(w) = \log(w)$ (consume everything)

We build two models:

• One with exponential_H for the standard exponential agent

• One with SolveSimulateFunctionPair for the naive beta-delta agent — solving with exponential $H$ and simulating with beta-delta $H$

Analytical Solution¶

With $\log$ utility, the optimal consumption rule is $c_t = w_t / D_t$, where $D_t$ is a “marginal propensity to save” denominator.

At $t = 1$ (one period before terminal), the agent maximizes:

First-order condition: $1/c = \beta\delta / (w - c)$, giving $c_1 = w / (1 + \beta\delta)$.

At $t = 0$, the naive agent uses the perceived exponential continuation value $V^E_1$, which has coefficient $(1 + \delta)$ on $\log w$. So the agent maximizes:

giving $c_0 = w / (1 + \beta\delta(1 + \delta))$.

The naive agent consumes more at $t = 0$ than the exponential agent — present bias pulls consumption forward. At $t = 1$, the naive agent also consumes more because $\beta\delta < \delta$.

Numerical Results¶

Exponential Agent ($\beta = 1$)¶

Naive Agent¶

The naive agent uses model_naive, built with a SolveSimulateFunctionPair wrapping $H$. During backward induction, exponential_H computes the perceived value function $V^E$; during simulation, beta_delta_H applies present bias for action selection.

The params dict is the union of both variants’ parameters — discount_factor for exponential_H, plus beta and delta for beta_delta_H. Each variant receives only the kwargs it expects.

Comparison¶

The small differences between numerical and analytical solutions are due to grid discretization.

Summary¶

Beta-delta discounting in PyLCM requires no core modifications:

1. Define two $H$ functions — one exponential, one with present bias:

   def exponential_H(utility, E_next_V, discount_factor):
       return utility + discount_factor * E_next_V
   
   def beta_delta_H(utility, E_next_V, beta, delta):
       return utility + beta * delta * E_next_V

2. Wrap them in SolveSimulateFunctionPair so backward induction uses exponential $H$ while action selection uses beta-delta $H$:

   from lcm import SolveSimulateFunctionPair
   
   functions={
       "utility": utility,
       "H": SolveSimulateFunctionPair(
           solve=exponential_H,
           simulate=beta_delta_H,
       ),
   }

3. Provide the union of both variants’ parameters:

   params = {"working": {"H": {"discount_factor": delta, "beta": beta, "delta": delta}}}

This split is essential: using $H(u, V') = u + \beta\delta V'$ for both phases would compound $\beta$ at every recursive step, giving effective weights $(\beta\delta)^n$ instead of the correct $\beta\delta^n$.