The Power of Bayes in Industry

Your Business Model is Your Data Generating Process

Dante Gates, PyMCon Web Series 2023

About Me

Husband and father

Philly data scientist

Wrote a haiku once
dantegates.github.io

Outline

Getting Started

What is a DGP?


Disambiguating an overloaded term:

  • The True Data Generating Process
  • The True DGP as modeled

Modeling from first principles



  • The True Data Generating Process
  • The True DGP as modeled

}

Goal: Align these

Motivating Example

Stan Case Study: Golf putting

Golf puts, a first principles model


x n y
0 2 1443 1346
1 3 694 577
2 4 455 337
3 5 353 208
4 6 272 149

The true DGP
Photo Credit: insidescience.org

The DGP as Modeled


Image Source: Stan Development Team

First principles fit

Image Source: Stan Development Team

First principles fit

Summary
  • Began with basic assumptions
  • Sketched mental model
  • Let the math play out
  • Fit the model

Image Source: Stan Development Team

First principles fit

Modeled outcome

Will the ball go in the hole?

-not-

y=0/1?

Image Source: Stan Development Team

First principles fit

Black Box Model?

Image Source: Stan Development Team

Useful, or cute party trick?



👶

Credit building through E-Commerce

Sketching

Modeling loan maturation

with pm.Model():
    ...

    D = ...
    
    ...

First Principles TODO

  • \(P(L) = 1 - P(D)\)

  • \(P(D_{t})\leq P(D_{t+1})\), \(P(L_{t})\leq P(L_{t+1})\)
  • \(P(L_{0})=P(D_{t}\vert t<T)=0\)
  • \(P(D_{t})=D\) as \(t\to\infty\)
  • \(P(L_{t})=L\) as \(t\to\infty\)

  • \(P(I_{t})=1-P(L_{t})-P(D_{t})\)
  • \(P(I_{0})=1\)
  • \(P(I_{t})=0\) as \(t\to\infty\)

D: Learned parameter of the mature default rate as \(t\rightarrow\infty\), e.g. pm.Beta()

Modeling loan maturation

with pm.Model():
    ...

    D = ...
    L = 1 - D
    
    ...

First Principles TODO

  • \(P(L) = 1 - P(D)\)

  • \(P(D_{t})\leq P(D_{t+1})\), \(P(L_{t})\leq P(L_{t+1})\)
  • \(P(L_{0})=P(D_{t}\vert t<T)=0\)
  • \(P(D_{t})=D\) as \(t\to\infty\)
  • \(P(L_{t})=L\) as \(t\to\infty\)

  • \(P(I_{t})=1-P(L_{t})-P(D_{t})\)
  • \(P(I_{0})=1\)
  • \(P(I_{t})=0\) as \(t\to\infty\)

L: The mature liquidation rate as \(t\to\infty\)

Modeling loan maturation

with pm.Model():
    ...

    D = ...
    L = 1 - D

    D_t = D * some_cdf(t-T)
    L_t = L * another_cdf(t)
    
    ...

First Principles TODO

  • \(P(L) = 1 - P(D)\)

  • \(P(D_{t})\leq P(D_{t+1})\), \(P(L_{t})\leq P(L_{t+1})\)
  • \(P(L_{0})=P(D_{t}\vert t<T)=0\)
  • \(P(D_{t})=D\) as \(t\to\infty\)
  • \(P(L_{t})=L\) as \(t\to\infty\)

  • \(P(I_{t})=1-P(L_{t})-P(D_{t})\)
  • \(P(I_{0})=1\)
  • \(P(I_{t})=0\) as \(t\to\infty\)
\(D\times\text{some_cdf}: R^{+}\to[0,D]\)

\(L\times\text{another_cdf}: R^{+}\to[0,L]\)

Modeling loan maturation

with pm.Model():
    ...

    D = ...
    L = 1 - D

    D_t = D * some_cdf(t-T)
    L_t = L * another_cdf(t)
    I_t = 1 - D_t - L_t
    
    ...

First Principles TODO

  • \(P(L) = 1 - P(D)\)

  • \(P(D_{t})\leq P(D_{t+1})\), \(P(L_{t})\leq P(L_{t+1})\)
  • \(P(L_{0})=P(D_{t}\vert t<T)=0\)
  • \(P(D_{t})=D\) as \(t\to\infty\)
  • \(P(L_{t})=L\) as \(t\to\infty\)

  • \(P(I_{t})=1-P(L_{t})-P(D_{t})\)
  • \(P(I_{0})=1\)
  • \(P(I_{t})=0\) as \(t\to\infty\)

I_t: The percentage of loans in-flight at time \(t\)

Modeling Loan Maturation

with pm.Model():
    ...

    likelihood = pm.Multinomial(
        'likelihood',
        n=N,
        p=[
            D_t,
            L_t,
            I_t
        ],
        observed=[
            default_at_t,
            liquidation_at_t,
            inflight_at_t,
        ]
    )

First Principles TODO

  • \(P(L) = 1 - P(D)\)

  • \(P(D_{t})\leq P(D_{t+1})\), \(P(L_{t})\leq P(L_{t+1})\)
  • \(P(L_{0})=P(D_{t}\vert t<T)=0\)
  • \(P(D_{t})=D\) as \(t\to\infty\)
  • \(P(L_{t})=L\) as \(t\to\infty\)

  • \(P(I_{t})=1-P(L_{t})-P(D_{t})\)
  • \(P(I_{0})=1\)
  • \(P(I_{t})=0\) as \(t\to\infty\)

Technically we need to do pt.Stack(...).T for p and observed=, but this makes the slide less readable.

Modeling Loan Maturation


...

likelihood = pm.Multinomial(
    'likelihood',
    n=N,
    p=[
        D_t,
        L_t,
        I_t
    ],
    observed=[
        default_at_t,
        liquidation_at_t,
        inflight_at_t,
    ]
)

Advantages over vanilla ML

  • Robust to overfitting
  • Model maps cleanly to the business model, not just business outcomes
  • Easy to explain, easy to troubleshoot
  • Estimating 5 outcomes in one shot, all estimates are self-consistent
  • Suitable for modeling data when outcome is not directly observed

Modeling Loan Maturation


...

likelihood = pm.Multinomial(
    'likelihood',
    n=N,
    p=[
        D_t,
        L_t,
        I_t
    ],
    observed=[
        default_at_t,
        liquidation_at_t,
        inflight_at_t,
    ]
)

Critiques

  • Models loan states, not balances
  • Models loans, not accounts: i.e. assumes independence
  • Assumes all loans are of similar risk
  • Does not account for growth
    • e.g. N changing over time
    • product improvements
  • No features!?

Awesome 2.0

Modeling with recovery


with pm.Model as model:
    ...

    # R: learned parameter of recovery, e.g. pm.Beta()
    # D_T: number of loans that defaulted `t-T` days ago
    # PD_T: number of loans in a past-due state `t-T` days ago
    # N: total number of loans
    R = ...
    D_t = (D_T + (1-R) * PD_T) / N
    D = D_t / some_cdf(t-T)
    
    # ↓↓↓ everything else same as before ↓↓↓

    ...

Results

Discussion

⚠️ Disclaimer

👋

Why industry?


From Wikipedia,

Why Bayesian?

Priors! All Bayesian models are DGPs!

Why Bayesian?

Priors! All Bayesian models are DGPs!

Why PyMCon?





Note: this code was simplified for the purpose of illustration and not quite correct. A better version can be found here.

Why should I care?

Humanz win


🧍 1
🤖 0

Perpay is Hiring

Appendix

Tips

Resources