[Template Fitting] What if sample sizes for data and templates are very different

[zjkjsd]

Aloha community and experts,

I am studying Template-fit-with-nuisance-parameters section and come up with a question in calculating the cost function. Below is the definition of the BB cost function from the tutorial.

class BB:
    def __init__(self, n, t):
        self.data = n, t

    def __call__(self, par):
        n, t = self.data
        bins = len(n)
        yields = par[:2]
        nuisances = par[2:]
        b = nuisances[:bins]
        s = nuisances[bins:]
        mu = 0
        for y, c in zip(yields, (b, s)):
            mu += y * c / np.sum(c)
        r = poisson_chi2(n, mu) + poisson_chi2(t[0], b) + poisson_chi2(t[1], s)
        return r

    @property
    def ndata(self):
        n, t = self.data
        return np.prod(n.shape) + np.prod(t.shape)

In my practice using iminuit template fitting, a few test samples (to which I am fitting) are generated and their sample sizes are small compared to the templates. So when I first looked at the line r = poisson_chi2(n, mu) + poisson_chi2(t[0], b) + poisson_chi2(t[1], s), I interpreted it in this way:

The last two terms are only modeling the shape difference between the templates and fitted components. t[0] and t[1] should be normalized templates, otherwise, the poisson_chi2 would be too large in the case where the test samples and templates have very different sample size.

However, I shortly realized that my understanding is incorrect because that way the uncertainties in the template are not well estimated. Can cost.Template handle very different sample sizes between test samples and templates?

Mahalo,
Boyang

[MarkusPrim]

Hi Boyang,

I assume your test samples correspond to the "data" you would later see in your real application, and your templates are derived from MC. As in the real application, can you now just reweight your templates to the sample size expected in your test (data) sample?

Cheers
Markus

[zjkjsd]

Hi Markus,

Thank you so much for your answer! Your assumption is correct and yes I can reweight templates, I wasn't sure if this reweighting was necessary. If I want to do a bootstrapping or toy MC study, do I need to reweight the template every time if the test sample sizes are different?

Thank you!
Boyang

[HDembinski]

Yes, the Template cost function handles the case you describe correctly. To see how the math works, I recommend our paper https://inspirehep.net/literature/2512593