Problems with Prior and Posterior Checks

[jgzga]

I have this code and it was working well until the last library update. I'm using the library by installing it like this: !pip install git+https://github.com/pymc-labs/pymc-marketing.git. I'm including the graphs of "Prior Predictive Check" and "Posterior Predictive Check". In the "Posterior Predictive Check" graph, it's as if the "Observed" line is adding a trend without me defining it.

Can someone help me with this?

Plots before the update

Plots after the update

The code is exactly the same and it's as follows. I also tried installing version 0.8.0 but it still gives me the same result.

import warnings
import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from pymc_marketing.mmm import MMM
from pymc_marketing.prior import Prior
from pymc_marketing.mmm import (
    GeometricAdstock,
    LogisticSaturation
)
from sklearn.preprocessing import StandardScaler

warnings.filterwarnings("ignore", category=FutureWarning)
az.style.use("arviz-darkgrid")
plt.rcParams["figure.figsize"] = [12, 7]
plt.rcParams["figure.dpi"] = 100

# Load data
data_path = "Data.xlsx"
data_df = pd.read_excel(data_path)

# Normalize column names: remove spaces and convert to lowercase
data_df.columns = [col.strip().replace(' ', '_').lower() for col in data_df.columns]

# Define specific columns
date_column = 'fecha'
target_column = 'total_adts'
media_columns = ['cine', 'olv', 'ooh', 'prensa', 'programatic', 'radio', 'redes_sociales', 'revistas', 'search', 'tv_abierta', 'tv_local', 'tv_paga']
other_columns = ['campaña', 'precipitación_nacional', 'temperatura_nacional', 'promociones_frappy', 'quincenas_dummy', 'vacaciones_dummy', 'quincena', 'festivo', 'aumento_precio_dummy']

all_columns = [date_column, target_column] + media_columns + other_columns

# Prepare data
df = data_df[all_columns].copy()
df = df.rename(columns={date_column: 'date_week', target_column: 'y'})

# Convert 'date_week' to datetime
df['date_week'] = pd.to_datetime(df['date_week'])

# Clean media columns: replace non-numeric values with NaN and then fill with 0
df[media_columns] = df[media_columns].replace(' - ', np.nan).apply(pd.to_numeric, errors='coerce').fillna(0)

# Handle numeric variables
numeric_columns = other_columns
df[numeric_columns] = df[numeric_columns].apply(pd.to_numeric, errors='coerce').fillna(0)

# Standardize numeric variables
scaler = StandardScaler()
df[numeric_columns] = scaler.fit_transform(df[numeric_columns])

# Handle NaN values in the target variable
df['y'] = df['y'].replace(' - ', np.nan).apply(pd.to_numeric, errors='coerce')
df = df.dropna(subset=['y'])

# Prepare final X and y
X = df[['date_week'] + media_columns + numeric_columns]
y = df['y']

# Define custom priors using Prior from pymc_marketing
my_model_config = {
    "intercept": Prior("Normal", mu=0, sigma=.5).constrain(lower=0, upper=0.85),
    "gamma_fourier": Prior("Laplace", mu=0, b=2, dims="fourier_mode"),
    "beta_channel": Prior("HalfNormal", sigma=2, dims="channel"),
    "beta_control": Prior("Normal", mu=0, sigma=2, dims="control"),
    "likelihood": {
        "dist": "StudentT",
        "kwargs": {
            "sigma": Prior("HalfNormal", sigma=0.5),
            "nu": Prior("Gamma", alpha=25, beta=2)
        }
    }
}

mmm = MMM(
    model_config=my_model_config,
    date_column="date_week",
    channel_columns=media_columns,
    control_columns=numeric_columns,
    yearly_seasonality=7,
    adstock=GeometricAdstock(l_max=8),
    saturation=LogisticSaturation(),
)

# Build the model
mmm.build_model(X, y)

# Prior predictive check
mmm.sample_prior_predictive(X, y, samples=2_000)
mmm.plot_prior_predictive()
plt.show()

# Model fitting
mmm.fit(X=X, y=y, target_accept=0.999, chains=4, nuts_sampler="numpyro", random_seed=np.random.default_rng(seed=sum(map(ord, "mmm"))))

# Sample posterior predictive
posterior_pred = mmm.sample_posterior_predictive(X)

# Diagnostics and posterior predictive checks
mmm.plot_posterior_predictive(original_scale=True)
plt.show()

mmm.plot_errors(original_scale=True)
plt.show()

mmm.plot_waterfall_components_decomposition()
plt.show()

# Save the model
mmm.save("model_specific_variables.nc")

print("Modelo MMM ajustado y guardado exitosamente.")

[juanitorduz]

@jgzga Thanks for reporting! Can you try running the example notebook (https://www.pymc-marketing.io/en/stable/notebooks/mmm/mmm_example.html) and see if these errors still happen 🙏 ?

For me the example notebook runs as expected

[juanitorduz]

Can you try removing

# Build the model
mmm.build_model(X, y)

[wd60622]

I think it has something to do with the standard scaler being used. Could you try without that scaling?

The model already doesn't scaling of the variables via the maxabs scaler. The media variables end up always being positive which wouldn't be the case after standard scaling

EDIT: Seeing that it is not the media being scaled in your example. Might be good to try with out your processing though

[wd60622]

Can you share what is the result of this?

Prior("Normal", mu=0, sigma=.5).constrain(lower=0, upper=0.85),

[wd60622]

Does running this with the default priors make any difference?

[wd60622]

When you run,

df['y'] = df['y'].replace(' - ', np.nan).apply(pd.to_numeric, errors='coerce')
df = df.dropna(subset=['y'])

are you dropping values in the middle of the time series?