Implement RooGradMinimizer using exact MINUIT gradient algorithm #10

egpbos · 2017-10-19T14:21:23Z

We will build a variant of RooMinimizer called RooGradMinimizer that handles gradient calculation internally, instead of letting MINUIT do it. The end goal in this is to then parallelize gradient calculation from the RooFit side.

We will use Lorenzo's example as the basis of our numerical derivation scheme: https://github.com/lmoneta/root/blob/lvmini/math/mathcore/src/NumericalDerivator.cxx
This example mimics MINUIT to a great degree, but some details are different, like the initial guess for the gradient. These we will modify to be as much as MINUIT as possible, preferably exactly identical, so that we can guarantee that any new code based on RooGradMinimizer will yield the same results as RooMinimizer did, but also faster eventually.

The text was updated successfully, but these errors were encountered:

egpbos · 2017-10-19T14:23:56Z

Added RooGradMinimizer branch off of gradient_testing branch in which we implemented a first proof-of-concept of separating the gradient out of MINUIT (#6, #8).

egpbos · 2017-10-19T14:50:34Z

Have to somehow pass parameter information to the derivator to determine the initial guess gradient, i.e. in NumericalDerivatorMinuit2::SetInitialGradient.

In the ROOT minimizer, in Math/Minimizer.h, SetVariables shows that different calls are used to set different types of parameters: bound vs fixed vs "regular" parameters. Something like that should be used here as well. SetVariables is fed a std::vector of ROOT::Fit::ParameterSettings objects, see Fit/FitConfig.h. The vector is called fSettings in FitConfig objects.

In RooMinimizerFcn, this kind of information is normally synchronized with MINUIT before each minimization call through RooMinimizerFcn::Synchronize. To use this kind of information in this derivator, it also has to be synchronized to here.

egpbos · 2017-10-20T14:58:10Z

Implemented the gradient initialization, only one detail missing from the gradient calculation itself, search for DIFFERS in the code.

egpbos · 2017-10-24T14:03:01Z

Integrated the gradient calculator in the code, now testing using a modified version of the GaussMinimizer.cpp script.

egpbos · 2017-10-25T12:45:14Z

Ok, so the gradient calculator seems to be working. However, it is a lot slower than the one from MINUIT, which is odd. From the GradMinimizer.cpp script, I now get the following output:


Processing projects/apcocsm/code/GradMinimizer.cpp...

�[1mRooFit v3.61a -- Developed by Wouter Verkerke and David Kirkby�[0m 
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt



  1) 0x7f96e0799a60 RooRealVar::          mu = -2.9  L(-3 - 3)  "mu"
  2) 0x7f96e1342800 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 0.0149208  "g"
  3) 0x7f96e0b43600 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 56225.6  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes: 
  Address: 0x7f96e0799a60
  Clients: 
    (0x7f96e1342800,V-) RooGaussian::g "g"
    (0x7f96e0b43600,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f96e0c4c600,V-) RooGaussian::g "g"
    (0x7f96e0a77600,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers: 
  Proxies: 
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0


trying nominal calculation
[#0] WARNING:Minization -- RooMinimizerFcn::synchronize: WARNING: no initial error estimate available for mu: using 0.05
 **********
 **    1 **SET PRINT           3
 **********
 **********
 **    2 **SET NOGRAD
 **********
 PARAMETER DEFINITIONS:
    NO.   NAME         VALUE      STEP SIZE      LIMITS
     1 mu          -2.90000e+00  5.00000e-02   -3.00000e+00  3.00000e+00
 **********
 **    3 **SET ERR         0.5
 **********
 **********
 **    4 **SET PRINT           3
 **********
 **********
 **    5 **SET STR           1
 **********
 NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
 **********
 **    6 **MIGRAD         500           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.

prevFCN = 56225.63481   START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
mu=-2.894, 
prevFCN = 56051.82271  mu=-2.906, 
prevFCN = 56394.72247  mu=-2.899, 
prevFCN = 56208.46555  mu=-2.901, 
prevFCN = 56242.75683  mu=-2.899, 
prevFCN = 56208.46555  mu=-2.897, 
prevFCN = 56139.31684  mu=-2.891, 
prevFCN = 55963.15745  mu=-2.871, 
prevFCN = 55406.8632  mu=-2.803, 
prevFCN = 53498.17016  mu=-2.518, 
prevFCN = 45963.74229  mu=-1.522, 
prevFCN = 25836.90417  mu=-1.52, 
prevFCN = 25821.18002  mu=-1.523, 
prevFCN = 25852.63521   FCN=25836.9 FROM MIGRAD    STATUS=INITIATE       13 CALLS          14 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  mu          -1.52152e+00   5.00000e-02   7.79972e-01  -3.94870e+04
                               ERR DEF= 0.5
NO ERROR MATRIX       
mu=1.107, 
prevFCN = 20220.42576  mu=0.02213, 
prevFCN = 14164.11274  mu=0.02331, 
prevFCN = 14164.30297  mu=0.02094, 
prevFCN = 14163.93652  mu=0.02295, 
prevFCN = 14164.24335  mu=0.02131, 
prevFCN = 14163.98888   FCN=14164.1 FROM MIGRAD    STATUS=PROGRESS       19 CALLS          20 TOTAL
                     EDM=9.94102    STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  mu           2.21282e-02   5.00000e-02   5.39279e-01   4.64423e+02
                               ERR DEF= 0.5
 RELATIVE CHANGE IN COV. MATRIX= 67.7 per cent
mu=0.003321, 
prevFCN = 14162.96967  mu=0.006647, 
prevFCN = 14162.91439  mu=0.007468, 
prevFCN = 14162.91776  mu=0.005825, 
prevFCN = 14162.91777   FCN=14162.9 FROM MIGRAD    STATUS=PROGRESS       23 CALLS          24 TOTAL
                     EDM=3.41696e-10    STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  mu           6.64662e-03   5.00000e-02  -5.16058e-03  -4.08736e-03
                               ERR DEF= 0.5
 RELATIVE CHANGE IN COV. MATRIX= 39.2 per cent
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
   START COVARIANCE MATRIX CALCULATION.
mu=0.006647, 
prevFCN = 14162.91439  mu=0.007468, 
prevFCN = 14162.91776  mu=0.005825, 
prevFCN = 14162.91777  mu=0.006811, 
prevFCN = 14162.91452  mu=0.006482, 
prevFCN = 14162.91452   EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
         1.0000e+00
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=14162.9 FROM MIGRAD    STATUS=CONVERGED      28 CALLS          29 TOTAL
                     EDM=1.84978e-10    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                   STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  mu           6.64662e-03   1.00001e-02   2.73950e-04  -4.08016e-03
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  1    ERR DEF=0.5
  1.000e-04 
 EXTERNAL ERROR MATRIX.    NDIM=   1    NPAR=  1    ERR DEF=0.5
  1.000e-04 
mu=0.006647, 

  1) 0x7f96e0799a60 RooRealVar::          mu = 0.006647 +/- 0.01  L(-3 - 3)  "mu"
  2) 0x7f96e1342800 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f96e0b43600 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes: 
  Address: 0x7f96e0799a60
  Clients: 
    (0x7f96e1342800,V-) RooGaussian::g "g"
    (0x7f96e0b43600,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f96e0c4c600,V-) RooGaussian::g "g"
    (0x7f96e0a77600,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers: 
  Proxies: 
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01


nominal mu fit is,0.006647


  1) 0x7f96e0799a60 RooRealVar::          mu = 0.006647 +/- 0.01  L(-3 - 3)  "mu"
  2) 0x7f96e1342800 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f96e0b43600 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes: 
  Address: 0x7f96e0799a60
  Clients: 
    (0x7f96e1342800,V-) RooGaussian::g "g"
    (0x7f96e0b43600,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f96e0c4c600,V-) RooGaussian::g "g"
    (0x7f96e0a77600,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers: 
  Proxies: 
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01

 === reset initial values === 

  1) 0x7f96e0799a60 RooRealVar::          mu = -2.9  L(-3 - 3)  "mu"
  2) 0x7f96e1342800 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 0.01492  "g"
  3) 0x7f96e0b43600 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 5.623e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes: 
  Address: 0x7f96e0799a60
  Clients: 
    (0x7f96e1342800,V-) RooGaussian::g "g"
    (0x7f96e0b43600,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f96e0c4c600,V-) RooGaussian::g "g"
    (0x7f96e0a77600,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers: 
  Proxies: 
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0


trying GradMinimizer
RooGradMinimizer created
[#0] WARNING:Minization -- RooGradMinimizerFcn::synchronize: WARNING: no initial error estimate available for mu: using 0.05

prevFCN = 56225.63481  func eval #1, value: 5.623e+04

prevFCN = 56225.63481  func eval #2, value: 5.623e+04
RooGradMinimizerFcn using strategy 1

prevFCN = 56225.63481  func eval #3, value: 5.623e+04
mu=-2.897, 
prevFCN = 56142.83462  func eval #4, value: 5.614e+04
mu=-2.903, 
prevFCN = 56308.51074  func eval #5, value: 5.631e+04
mu=-2.899, 
prevFCN = 56200.99462  func eval #6, value: 5.62e+04
mu=-2.901, 
prevFCN = 56250.2817  func eval #7, value: 5.625e+04
grad value -2.862e+04
mu=-2.615, 
prevFCN = 48431.53426  func eval #8, value: 4.843e+04
mu=-2.996, 
prevFCN = 59016.76446  func eval #9, value: 5.902e+04
mu=1.009, 
prevFCN = 19181.80833  func eval #10, value: 1.918e+04
mu=-2.752, 
prevFCN = 52079.30276  func eval #11, value: 5.208e+04
mu=-0.3291, 
prevFCN = 14726.55759  func eval #12, value: 1.473e+04
mu=-1.899, 
prevFCN = 32308.93052  func eval #13, value: 3.231e+04
mu=-0.2742, 
prevFCN = 14557.32562  func eval #14, value: 1.456e+04
mu=-3, 
prevFCN = 59128.19534  func eval #15, value: 5.913e+04
mu=-2.226, 
prevFCN = 39058.54506  func eval #16, value: 3.906e+04
mu=-0.2742, 
prevFCN = 14557.32562  func eval #17, value: 1.456e+04

prevFCN = 14557.32562  func eval #18, value: 1.456e+04
mu=-0.2738, 
prevFCN = 14556.09596  func eval #19, value: 1.456e+04
mu=-0.2747, 
prevFCN = 14558.5572  func eval #20, value: 1.456e+04
grad value -2809
mu=-2.999, 
prevFCN = 59088.7761  func eval #21, value: 5.909e+04
mu=0.7988, 
prevFCN = 17300.49434  func eval #22, value: 1.73e+04
mu=2.825, 
prevFCN = 53721.02926  func eval #23, value: 5.372e+04
mu=2.999, 
prevFCN = 58714.22721  func eval #24, value: 5.871e+04
mu=-2.985, 
prevFCN = 58683.81856  func eval #25, value: 5.868e+04
mu=-1.624, 
prevFCN = 27457.6029  func eval #26, value: 2.746e+04
mu=2.86, 
prevFCN = 54694.37927  func eval #27, value: 5.469e+04
mu=0.174, 
prevFCN = 14302.93905  func eval #28, value: 1.43e+04

prevFCN = 14302.93905  func eval #29, value: 1.43e+04

prevFCN = 14302.93905  func eval #30, value: 1.43e+04
mu=0.1744, 
prevFCN = 14303.63094  func eval #31, value: 1.43e+04
mu=0.1736, 
prevFCN = 14302.24886  func eval #32, value: 1.43e+04
grad value 1673
mu=0.006301, 
prevFCN = 14162.91498  func eval #33, value: 1.416e+04

prevFCN = 14162.91498  func eval #34, value: 1.416e+04

prevFCN = 14162.91498  func eval #35, value: 1.416e+04
mu=0.006712, 
prevFCN = 14162.91441  func eval #36, value: 1.416e+04
mu=0.00589, 
prevFCN = 14162.91725  func eval #37, value: 1.416e+04
grad value -3.458
mu=0.006648, 
prevFCN = 14162.91439  func eval #38, value: 1.416e+04

prevFCN = 14162.91439  func eval #39, value: 1.416e+04

prevFCN = 14162.91439  func eval #40, value: 1.416e+04
mu=0.007058, 
prevFCN = 14162.91523  func eval #41, value: 1.416e+04
mu=0.006237, 
prevFCN = 14162.91523  func eval #42, value: 1.416e+04
grad value 0.007826
mu=0.006647, 
prevFCN = 14162.91439  func eval #43, value: 1.416e+04

prevFCN = 14162.91439  func eval #44, value: 1.416e+04

prevFCN = 14162.91439  func eval #45, value: 1.416e+04
mu=0.007058, 
prevFCN = 14162.91523  func eval #46, value: 1.416e+04
mu=0.006236, 
prevFCN = 14162.91523  func eval #47, value: 1.416e+04
grad value 2.213e-09
mu=0.006647, 
prevFCN = 14162.91439  func eval #48, value: 1.416e+04
mu=-0.01356, 
prevFCN = 14164.95652  func eval #49, value: 1.416e+04
mu=0.02686, 
prevFCN = 14164.95646  func eval #50, value: 1.416e+04
mu=0.006033, 
prevFCN = 14162.91627  func eval #51, value: 1.416e+04
mu=0.00726, 
prevFCN = 14162.91627  func eval #52, value: 1.416e+04
mu=0.006524, 
prevFCN = 14162.91446  func eval #53, value: 1.416e+04
mu=0.006769, 
prevFCN = 14162.91446  func eval #54, value: 1.416e+04
mu=0.006647, 

  1) 0x7f96e0799a60 RooRealVar::          mu = 0.006647 +/- 0.01414  L(-3 - 3)  "mu"
  2) 0x7f96e1342800 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f96e0b43600 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes: 
  Address: 0x7f96e0799a60
  Clients: 
    (0x7f96e1342800,V-) RooGaussian::g "g"
    (0x7f96e0b43600,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f96e0c4c600,V-) RooGaussian::g "g"
    (0x7f96e0a77600,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers: 
  Proxies: 
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01414

egpbos · 2017-10-25T14:43:22Z

Set random seed constant, so that created data should be the same every time (right?), but it seems like MINUIT also does some random things, since not every run goes the same way.

Also tried splitting the GradMinimizer and normal Minimizer tests into separate files, to make sure they don't influence each other. The result is the same: GradMinimizer takes longer.
https://github.com/roofit-dev/parallel-roofit-scripts/blob/master/onlyGradMinimizer.cpp
https://github.com/roofit-dev/parallel-roofit-scripts/blob/master/only_normal_Minimizer.cpp

egpbos · 2017-10-26T08:57:44Z

Added debug prints to Numerical2PGradientCalculator, but they don't print anything... Is Numerical2PGradientCalculator actually the gradient calculator that Minuit2 uses? It seems like that's not the case!

egpbos · 2017-10-26T09:14:33Z

Oops! We were using Minuit1 in our tests, not Minuit2...

egpbos · 2017-10-26T09:53:25Z

Now correctly using Minuit2 in the nominal minimizer and in the GradMinimizer. Bizarre result using some extra diagnostic output from inside Minuit2: for some reason, after function call number 29 in the GradMinimizer, Minuit2 again calls its own Numerical2PDerivative!

Full output of a GradMinimizer.cpp test run:

root [0]
Processing projects/apcocsm/code/GradMinimizer.cpp...

RooFit v3.61a -- Developed by Wouter Verkerke and David Kirkby
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt



  1) 0x7f883adb6830 RooRealVar::          mu = -2.9  L(-3 - 3)  "mu"
  2) 0x7f883c3e1e00 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 0.0149208  "g"
  3) 0x7f883cc80000 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 56225.6  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes:
  Address: 0x7f883adb6830
  Clients:
    (0x7f883c3e1e00,V-) RooGaussian::g "g"
    (0x7f883cc80000,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f883caa2600,V-) RooGaussian::g "g"
    (0x7f883ccade00,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers:
  Proxies:
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0


trying nominal calculation
[#0] WARNING:Minization -- RooMinimizerFcn::synchronize: WARNING: no initial error estimate available for mu: using 0.05
Minuit2Minimizer: Minimize with max-calls 500 convergence for edm < 1 strategy 1

prevFCN = 56225.63481  MnSeedGenerator: for initial parameters FCN = 56225.63481053


########### Numerical2PDerivative : START
fGrd[0] = 14.84	fG2[0] = 220.4	fGstep[0] = 0.006737	position[0] = -1.312

mu=-2.891,
prevFCN = 55978.45301  mu=-2.908,
prevFCN = 56463.36791  mu=-2.899,
prevFCN = 56201.33998  mu=-2.901,
prevFCN = 56249.83515

########### Numerical2PDerivative : END
fGrd[0] = 14.84	fG2[0] = 220.4	fGstep[0] = 0.006737	position[0] = -1.312

MnSeedGenerator: Initial state:   - FCN =   56225.63481053 Edm =        -3111 NCalls =      5
mu=-2.899,
prevFCN = 56201.33998  mu=-2.896,
prevFCN = 56103.21799  mu=-2.887,
prevFCN = 55851.35425  mu=-2.858,
prevFCN = 55040.59222  mu=-2.754,
prevFCN = 52143.04858  mu=-2.284,
prevFCN = 40355.09733  mu=-0.6219,
prevFCN = 16138.46645

########### Numerical2PDerivative : START
fGrd[0] = -2.198e+04	fG2[0] = -7.766e+04	fGstep[0] = 0.001103	position[0] = -0.2088

mu=-0.621,
prevFCN = 16132.66354  mu=-0.6229,
prevFCN = 16144.27751

########### Numerical2PDerivative : END
fGrd[0] = -2.198e+04	fG2[0] = -7.766e+04	fGstep[0] = 0.001103	position[0] = -0.2088

MnSeedGenerator: Negative G2 found - new state:   - FCN =   16138.46645139 Edm =         2070 NCalls =     14
VariableMetric: start iterating until Edm is < 0.001
VariableMetric: Initial state   - FCN =   16138.46645139 Edm =         2070 NCalls =     14
VariableMetric: Iteration #   0 - FCN =   16138.46645139 Edm =         2070 NCalls =     14
mu=0.04667,
prevFCN = 14170.92208

########### Numerical2PDerivative : START
fGrd[0] = -1.845e+04	fG2[0] = 8.221e+04	fGstep[0] = 0.0003148	position[0] = 0.01556

mu=0.04753,
prevFCN = 14171.26993  mu=0.04581,
prevFCN = 14170.58162

########### Numerical2PDerivative : END
fGrd[0] = -1.845e+04	fG2[0] = 8.221e+04	fGstep[0] = 0.0003148	position[0] = 0.01556

VariableMetric: Iteration #   1 - FCN =   14170.92207567 Edm =        8.764 NCalls =     17
mu=0.00554,
prevFCN = 14162.92051

########### Numerical2PDerivative : START
fGrd[0] = 1200	fG2[0] = 8.996e+04	fGstep[0] = 0.0002867	position[0] = 0.001847

mu=0.006362,
prevFCN = 14162.91479  mu=0.004718,
prevFCN = 14162.93299

########### Numerical2PDerivative : END
fGrd[0] = 1200	fG2[0] = 8.996e+04	fGstep[0] = 0.0002867	position[0] = 0.001847

VariableMetric: Iteration #   2 - FCN =   14162.92051115 Edm =     0.006295 NCalls =     20
mu=0.006647,
prevFCN = 14162.91439

########### Numerical2PDerivative : START
fGrd[0] = -33.2	fG2[0] = 9e+04	fGstep[0] = 0.000274	position[0] = 0.002216

mu=0.007469,
prevFCN = 14162.91777  mu=0.005825,
prevFCN = 14162.91776

########### Numerical2PDerivative : END
fGrd[0] = -33.2	fG2[0] = 9e+04	fGstep[0] = 0.000274	position[0] = 0.002216

VariableMetric: Iteration #   3 - FCN =   14162.91438689 Edm =    1.565e-10 NCalls =     23
mu=0.006647,
prevFCN = 14162.91439  mu=0.007469,
prevFCN = 14162.91777  mu=0.005825,
prevFCN = 14162.91776  mu=0.006811,
prevFCN = 14162.91452  mu=0.006483,
prevFCN = 14162.91452  VariableMetric: After Hessian   - FCN =   14162.91438689 Edm =    1.569e-10 NCalls =     28
VariableMetric: Iteration #   4 - FCN =   14162.91438689 Edm =    1.569e-10 NCalls =     28
Number of iterations 5
----------> Iteration 0
            FVAL = 16138.4664514 Edm = 2069.61683815 Nfcn = 14
            Error matrix change = 1
            Parameters :  p0 = -0.6219
----------> Iteration 1
            FVAL = 14170.9220757 Edm = 8.76398094324 Nfcn = 17
            Error matrix change = 0.5325
            Parameters :  p0 = 0.04667
----------> Iteration 2
            FVAL = 14162.9205112 Edm = 0.00629465324698 Nfcn = 20
            Error matrix change = 0.2801
            Parameters :  p0 = 0.00554
----------> Iteration 3
            FVAL = 14162.9143869 Edm = 1.56514910404e-10 Nfcn = 23
            Error matrix change = 0.1401
            Parameters :  p0 = 0.006647
----------> Iteration 4
            FVAL = 14162.9143869 Edm = 1.56914829289e-10 Nfcn = 28
            Error matrix change = 0
            Parameters :  p0 = 0.006647
Minuit2Minimizer : Valid minimum - status = 0
FVAL  = 14162.9143868946248
Edm   = 1.56914829289240187e-10
Nfcn  = 28
mu	  = 0.006647	 +/-  0.01	(limited)
mu=0.006647,

  1) 0x7f883adb6830 RooRealVar::          mu = 0.006647 +/- 0.01  L(-3 - 3)  "mu"
  2) 0x7f883c3e1e00 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f883cc80000 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes:
  Address: 0x7f883adb6830
  Clients:
    (0x7f883c3e1e00,V-) RooGaussian::g "g"
    (0x7f883cc80000,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f883caa2600,V-) RooGaussian::g "g"
    (0x7f883ccade00,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers:
  Proxies:
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01


nominal mu fit is,0.006647


  1) 0x7f883adb6830 RooRealVar::          mu = 0.006647 +/- 0.01  L(-3 - 3)  "mu"
  2) 0x7f883c3e1e00 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f883cc80000 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes:
  Address: 0x7f883adb6830
  Clients:
    (0x7f883c3e1e00,V-) RooGaussian::g "g"
    (0x7f883cc80000,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f883caa2600,V-) RooGaussian::g "g"
    (0x7f883ccade00,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers:
  Proxies:
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01

 === reset initial values ===

  1) 0x7f883adb6830 RooRealVar::          mu = -2.9  L(-3 - 3)  "mu"
  2) 0x7f883c3e1e00 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 0.01492  "g"
  3) 0x7f883cc80000 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 5.623e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes:
  Address: 0x7f883adb6830
  Clients:
    (0x7f883c3e1e00,V-) RooGaussian::g "g"
    (0x7f883cc80000,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f883caa2600,V-) RooGaussian::g "g"
    (0x7f883ccade00,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers:
  Proxies:
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0


trying GradMinimizer
RooGradMinimizer created
[#0] WARNING:Minization -- RooGradMinimizerFcn::synchronize: WARNING: no initial error estimate available for mu: using 0.05

prevFCN = 56225.63481  func eval #1, value: 5.623e+04
RooGradMinimizerFcn using strategy 1

Start:
fGrd[0] = 40	fG2[0] = 800	fGstep[0] = 0.005

prevFCN = 56225.63481  func eval #2, value: 5.623e+04
mu=-2.897,
prevFCN = 56142.83462  func eval #3, value: 5.614e+04
mu=-2.903,
prevFCN = 56308.51074  func eval #4, value: 5.631e+04
mu=-2.899,
prevFCN = 56200.99462  func eval #5, value: 5.62e+04
mu=-2.901,
prevFCN = 56250.2817  func eval #6, value: 5.625e+04

End:
fGrd[0] = -2.862e+04	fG2[0] = 9039	fGstep[0] = 0.0008611
grad value -2.862e+04
mu=-2.976,
prevFCN = 58424.33998  func eval #7, value: 5.842e+04
mu=-3,
prevFCN = 59120.85241  func eval #8, value: 5.912e+04
mu=-2.992,
prevFCN = 58903.06242  func eval #9, value: 5.89e+04
mu=-2.983,
prevFCN = 58644.39715  func eval #10, value: 5.864e+04
mu=2.975,
prevFCN = 58015.19319  func eval #11, value: 5.802e+04
mu=0.4597,
prevFCN = 15189.03194  func eval #12, value: 1.519e+04

Start:
fGrd[0] = -2.862e+04	fG2[0] = 9039	fGstep[0] = 0.0008611

prevFCN = 15189.03194  func eval #13, value: 1.519e+04
mu=0.4601,
prevFCN = 15191.06047  func eval #14, value: 1.519e+04
mu=0.4592,
prevFCN = 15187.00541  func eval #15, value: 1.519e+04

End:
fGrd[0] = 4530	fG2[0] = 9999	fGstep[0] = 0.0004476
grad value 4530
mu=-1.174,
prevFCN = 21130.17688  func eval #16, value: 2.113e+04
mu=-0.002794,
prevFCN = 14163.36  func eval #17, value: 1.416e+04

Start:
fGrd[0] = 4530	fG2[0] = 9999	fGstep[0] = 0.0004476

prevFCN = 14163.36  func eval #18, value: 1.416e+04
mu=-0.002383,
prevFCN = 14163.32205  func eval #19, value: 1.416e+04
mu=-0.003205,
prevFCN = 14163.39963  func eval #20, value: 1.416e+04

End:
fGrd[0] = -94.4	fG2[0] = 1e+04	fGstep[0] = 0.0004109
grad value -94.4
mu=0.006795,
prevFCN = 14162.9145  func eval #21, value: 1.416e+04

Start:
fGrd[0] = -94.4	fG2[0] = 1e+04	fGstep[0] = 0.0004109

prevFCN = 14162.9145  func eval #22, value: 1.416e+04
mu=0.007206,
prevFCN = 14162.91595  func eval #23, value: 1.416e+04
mu=0.006384,
prevFCN = 14162.91473  func eval #24, value: 1.416e+04

End:
fGrd[0] = 1.481	fG2[0] = 1e+04	fGstep[0] = 0.0004109
grad value 1.481
mu=0.006647,
prevFCN = 14162.91439  func eval #25, value: 1.416e+04

Start:
fGrd[0] = 1.481	fG2[0] = 1e+04	fGstep[0] = 0.0004109

prevFCN = 14162.91439  func eval #26, value: 1.416e+04
mu=0.007058,
prevFCN = 14162.91523  func eval #27, value: 1.416e+04
mu=0.006236,
prevFCN = 14162.91523  func eval #28, value: 1.416e+04

End:
fGrd[0] = 5.874e-06	fG2[0] = 1e+04	fGstep[0] = 0.0004109
grad value 5.874e-06
mu=0.006647,
prevFCN = 14162.91439  func eval #29, value: 1.416e+04


########### Numerical2PDerivative : START
fGrd[0] = 120	fG2[0] = 7199	fGstep[0] = 0.001667	position[0] = 0.002216

mu=0.009552,
prevFCN = 14162.9566  func eval #30, value: 1.416e+04
mu=0.003741,
prevFCN = 14162.9566  func eval #31, value: 1.416e+04
mu=0.007469,
prevFCN = 14162.91776  func eval #32, value: 1.416e+04
mu=0.005825,
prevFCN = 14162.91776  func eval #33, value: 1.416e+04


########### Numerical2PDerivative : END
fGrd[0] = 120	fG2[0] = 7199	fGstep[0] = 0.001667	position[0] = 0.002216

mu=0.007469,
prevFCN = 14162.91776  func eval #34, value: 1.416e+04
mu=0.005825,
prevFCN = 14162.91776  func eval #35, value: 1.416e+04
mu=0.006811,
prevFCN = 14162.91452  func eval #36, value: 1.416e+04
mu=0.006482,
prevFCN = 14162.91452  func eval #37, value: 1.416e+04
mu=0.00668,
prevFCN = 14162.91439  func eval #38, value: 1.416e+04
mu=0.006614,
prevFCN = 14162.91439  func eval #39, value: 1.416e+04
mu=0.006647,

  1) 0x7f883adb6830 RooRealVar::          mu = 0.006647 +/- 0.01414  L(-3 - 3)  "mu"
  2) 0x7f883c3e1e00 RooGaussian::           g[ x=x mean=mu sigma=sigma ] = 1  "g"
  3) 0x7f883cc80000 RooNLLVar:: nll_g_gData[ paramSet=(mu,sigma) ] = 1.416e+04  "-log(likelihood)"
--- RooAbsArg ---
  Value State: clean
  Shape State: clean
  Attributes:
  Address: 0x7f883adb6830
  Clients:
    (0x7f883c3e1e00,V-) RooGaussian::g "g"
    (0x7f883cc80000,V-) RooNLLVar::nll_g_gData "-log(likelihood)"
    (0x7f883caa2600,V-) RooGaussian::g "g"
    (0x7f883ccade00,V-) RooRealIntegral::g_Int[x] "Integral of g"
  Servers:
  Proxies:
--- RooAbsReal ---

  Plot label is "mu"
--- RooAbsRealLValue ---
  Fit range is [ -3 , 3 ]
--- RooRealVar ---
  Error = 0.01414

egpbos · 2017-10-26T09:55:45Z

Also: the fit already converged to the exact same result that Minuit converged to at call 29... seems then that our algorithm is working on this test!

Todo:

Figure out how to stop Minuit from calling its own derivator again at the "end" of the fit.
~~Confirm that our Derivator is working with some more, perhaps multi-dimensional and/or otherwise more complicated tests.~~ --> Test RooGradMinimizer #11

egpbos · 2017-10-26T10:01:17Z

Not quite the same actually. Our algo does two identical function calls just before and just after the start of the derivative. Minuit somehow reuses the function call from before the derivative inside the derivative. Eliminating these duplicate calls will reduce the number of calls by 5 in this example, yielding a total of 24, already getting closer to Minuit :)

Eliminate duplicate FCN calls before/after GRAD calls

egpbos · 2017-10-30T07:46:59Z

The calling of Minuit2's 2P derivator was done from the MnHesse call inside the iteration loop of VariableMetricBuilder::Minimum:

MinimumState st = MnHesse(strategy)(fcn, min.State(), min.Seed().Trafo(),maxfcn);

This can be omitted by setting the minimizer strategy to zero. This reduces the number of function calls in the nominal minimization to 23 and in our minimizer to 28.

I also tried implementing RooGradMinimizer::hesse(), but this is apparently not called by MnHesse.

Find out how to get the internal Minuit2 MnHesse thing to call our RooGradMinimizer::hesse() and use our derivator.

egpbos · 2017-10-30T07:53:20Z

Find out why RooGradMinimizer::hesse() also calls the Numerical2PDerivative instead of our own.
~~Find out why RooGradMinimizer::hesse() only takes 5 function calls, whereas MnHesse called from VariableMetricBuilder::Minimum takes 11~~ --> Find out why RooGradMinimizer::hesse() only takes 5 function calls, whereas MnHesse called from VariableMetricBuilder::Minimum takes 11 #12

egpbos · 2017-10-30T08:38:04Z

All Hesse calls end up in

MnHesse::operator()(const FCNBase& fcn, const MnUserParameterState& state, unsigned int maxcalls)

This is not overloaded for functions with their own gradient, i.e. there is no

MnHesse::operator()(const FCNGradientBase& fcn, const MnUserParameterState& state, unsigned int maxcalls)

and assorted higher/lower level versions. There are currently 8 MnHesse:: operator()s, all of which would have to be reimplemented to take FCNGradientBase.

~~Implement MnHesse::operator()(const FCNGradientBase&, ...) versions that take a function and a gradient instead of only a function with the current interface.~~ --> Implement MnHesse::operator()(const FCNGradientBase&, ...) versions #13

egpbos · 2017-10-30T10:00:54Z

Eliminated duplicate FCN calls before/after GRAD calls by putting a memoization mechanism inside DoEval.

Now the number of function calls is 22, actually one less than Minuit2 😃

egpbos · 2017-10-30T15:07:48Z

After running the test a couple of times, I believe that we are getting more or less the right result, though not exactly what Minuit2 gives. The likelihood values are typically equal in the first 14 digits. The parameter of interest mu is usually equal in its first 4 digits (rounded). However, the error is always different:

trying nominal calculation
Minuit2Minimizer : Valid minimum - status = 0
FVAL  = 14089.1157832817262
Edm   = 6.71561080824408584e-11
Nfcn  = 23
mu	  = -0.00327251	 +/-  0.0100001	(limited)

 === reset initial values ===

trying GradMinimizer
Minuit2Minimizer : Valid minimum - status = 0
FVAL  = 14089.1157832818699
Edm   = 2.10626254990635505e-10
Nfcn  = 14
mu	  = -0.00327284	 +/-  0.0141422	(limited)

I get this same result of a factor 1.41422 (~ sqrt(2)...) in the error on mu in every run.

The number of recorded function calls that the minimizer reports is without the gradient function calls in the GradMinimizer case; oddly I couldn't find where this number is actually updated in the code, so I can't correct it for GradMinimizer.

Finally, the EDM is different, though always very small in both cases.

Basically, I'm already testing the code, so we can close this and move on to #11 (and #12 and #13 for follow up issues from above discussion).

…transactions. This fixes https://sft.its.cern.ch/jira/browse/ROOT-9672 by having cling::Interpreter::DeclareCFunction return the transaction containing the compiled code. With the previous code, cling::Interpreter::compileFunction will get confused by transaction created during the callbacks executed during the cling::IncrementalParser::commitTransaction of the main transaction. Reproducer: With a main composed of 'only': int main(int argc, char ** argv) { char const * class_string = (argc == 2) ? argv[1] : "std::vector<int>"; auto const result [[gnu::unused]] = TClass::GetClass(class_string); return 0; } which is a representation of real use case (in a more complex setup) in ART. We were getting: Error in <TClingCallFunc::make_wrapper>: Failed to compile ==== SOURCE BEGIN ==== #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wformat-security" __attribute__((used)) extern "C" void __cf_0(void* obj, int nargs, void** args, void* ret) { if (ret) { (*(TStreamerInfo**)ret) = new TStreamerInfo(); return; } else { new TStreamerInfo(); return; } } #pragma clang diagnostic pop ==== SOURCE END ==== Error in <TClingCallFunc::ExecT>: Called with no wrapper, not implemented! Error in <TVirtualStreamerInfo::Factory>: The plugin handler for TVirtualStreamerInfo was found but failed to create the factory object! The reason is that during TClingCallFunc::make_wrapper, the call to cling::Interpreter::compileFunction ends with: if (const llvm::GlobalValue* GV = getLastTransaction()->getModule()->getNamedValue(name)) However in the 'broken' case, the getLastTransaction does not return the transaction for the code being compiled by DeclareCFunction but instead the one used/created at: #0 cling::IncrementalParser::endTransaction (this=0x4a2980, T=0x8c0fb0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/IncrementalParser.cpp:345 #1 0x00007fffeebc7899 in cling::Interpreter::PushTransactionRAII::pop (this=0x7fffffffcb00) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:111 #2 0x00007fffeebc785e in cling::Interpreter::PushTransactionRAII::~PushTransactionRAII (this=0x7fffffffcb00, __in_chrg=<optimized out>) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:106 #3 0x00007fffeebeb659 in cling::LookupHelper::findScope (this=0x4a9dd0, className=..., diagOnOff=cling::LookupHelper::NoDiagnostics, resultType=0x7fffffffcd08, instantiateTemplate=false) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/LookupHelper.cpp:466 #4 0x00007fffeeabe0df in TCling::CheckClassInfo (this=0x4a0550, name=<optimized out>, autoload=<optimized out>, isClassOrNamespaceOnly=<optimized out>) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TCling.cxx:3630 #5 0x00007ffff7c3040d in TClass::Init (this=this@entry=0xdafd20, name=name@entry=0x7ffff7cb7638 "TGlobal", cversion=cversion@entry=2, typeinfo=typeinfo@entry=0x7ffff7d8b6d8 <typeinfo for TGlobal>, isa=isa@entry=0x477430, dfil=dfil@entry=0x7ffff7cb8cab "TGlobal.h", ifil=<optimized out>, dl=<optimized out>, il=<optimized out>, givenInfo=<optimized out>, silent=<optimized out>) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TClass.cxx:1431 #6 0x00007ffff7c3a1b8 in TClass::TClass (this=0xdafd20, name=0x7ffff7cb7638 "TGlobal", cversion=<optimized out>, info=..., isa=0x477430, dfil=0x7ffff7cb8cab "TGlobal.h", ifil=0x7ffff7cccf88 "/local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TGlobal.cxx", dl=27, il=25, silent=false) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TClass.cxx:1273 #7 0x00007ffff7c3a72a in ROOT::CreateClass (cname=0x7ffff7cb7638 "TGlobal", id=id@entry=2, info=..., isa=isa@entry=0x477430, dfil=dfil@entry=0x7ffff7cb8cab "TGlobal.h", ifil=ifil@entry=0x7ffff7cccf88 "/local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TGlobal.cxx", dl=27, il=25) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TClass.cxx:5607 #8 0x00007ffff7c4b552 in ROOT::Internal::TDefaultInitBehavior::CreateClass (il=25, dl=27, ifil=0x7ffff7cccf88 "/local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TGlobal.cxx", dfil=0x7ffff7cb8cab "TGlobal.h", isa=0x477430, info=..., id=2, cname=<optimized out>, this=0x7ffff7da7508 <ROOT::Internal::DefineBehavior(void*, void*)::theDefault>) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/Rtypes.h:176 #9 ROOT::TGenericClassInfo::GetClass (this=0x7ffff7dab660 <ROOT::GenerateInitInstanceLocal(TGlobal const*)::instance>) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TGenericClassInfo.cxx:250 #10 0x00007ffff7b1a2d8 in TGlobal::Class () at /home/pcanal/root_builds/v6-14-00-patches/opt/core/base/G__Core.cxx:17156 #11 0x00007ffff7ac01de in TGlobal::IsA (this=0xee3bc0) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/TGlobal.h:48 #12 TGlobal::CheckTObjectHashConsistency (this=0xee3bc0) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/TGlobal.h:48 #13 0x00007ffff7be9dcd in TObject::CheckedHash (this=0xee3bc0) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/TObject.h:314 #14 THashTable::GetCheckedHashValue (this=0xe65a20, obj=0xee3bc0) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/THashTable.h:94 #15 THashTable::Add (this=0xe65a20, obj=0xee3bc0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/cont/src/THashTable.cxx:96 #16 0x00007ffff7be6bf1 in THashList::AddLast (this=this@entry=0x5be690, obj=obj@entry=0xee3bc0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/cont/src/THashList.cxx:100 #17 0x00007ffff7c4e0d1 in TListOfDataMembers::AddLast (this=0x5be690, obj=0xee3bc0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TListOfDataMembers.cxx:103 #18 0x00007ffff7ab8785 in TList::Add (obj=0xee3bc0, this=0x5be690) at /home/pcanal/root_builds/v6-14-00-patches/opt/include/TList.h:87 #19 TROOT::GetListOfGlobals (this=0x7ffff7da7a60 <ROOT::Internal::GetROOT1()::alloc>, load=load@entry=false) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/base/src/TROOT.cxx:1767 #20 0x00007fffeeab1058 in TCling::HandleNewDecl (this=0x4a0550, DV=0xedf238, isDeserialized=isDeserialized@entry=true, modifiedTClasses=...) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TCling.cxx:555 #21 0x00007fffeeabb785 in TCling::UpdateListsOnCommitted (this=0x4a0550, T=...) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TCling.cxx:6115 #22 0x00007fffeebd0103 in cling::MultiplexInterpreterCallbacks::TransactionCommitted (this=0x57fe20, T=...) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/MultiplexInterpreterCallbacks.h:76 #23 0x00007fffeed05d71 in cling::IncrementalParser::commitTransaction (this=0x4a2980, PRT=..., ClearDiagClient=true) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/IncrementalParser.cpp:532 #24 0x00007fffeed06399 in cling::IncrementalParser::Compile (this=0x4a2980, input=..., Opts=...) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/IncrementalParser.cpp:663 #25 0x00007fffeebcbc4e in cling::Interpreter::DeclareInternal (this=0x4a0f30, input=..., CO=..., T=0x7fffffffd680) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:1195 root-project#26 0x00007fffeebca8e8 in cling::Interpreter::declare (this=0x4a0f30, input=..., T=0x7fffffffd680) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:823 root-project#27 0x00007fffeebcb560 in cling::Interpreter::DeclareCFunction (this=0x4a0f30, name=..., code=..., withAccessControl=true) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:1096 root-project#28 0x00007fffeebcb862 in cling::Interpreter::compileFunction (this=0x4a0f30, name=..., code=..., ifUnique=false, withAccessControl=true) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/interpreter/cling/lib/Interpreter/Interpreter.cpp:1140 root-project#29 0x00007fffeeafb83c in TClingCallFunc::compile_wrapper (withAccessControl=true, wrapper=..., wrapper_name=..., this=0xcf3c10) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TClingCallFunc.cxx:270 root-project#30 TClingCallFunc::make_wrapper (this=this@entry=0xcf3c10) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TClingCallFunc.cxx:1096 root-project#31 0x00007fffeeafbcb8 in TClingCallFunc::IFacePtr (this=this@entry=0xcf3c10) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TClingCallFunc.cxx:2233 root-project#32 0x00007fffeeafbe83 in TClingCallFunc::ExecT<long> (address=0x0, this=0xcf3c10) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TClingCallFunc.cxx:2045 root-project#33 TClingCallFunc::ExecInt (this=0xcf3c10, address=0x0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/metacling/src/TClingCallFunc.cxx:2065 root-project#34 0x00007ffff7c56e8d in TMethodCall::Execute (this=0xd97710, object=<optimized out>, retLong=@0x7fffffffd958: 0) at /local2/pcanal/cint_working/rootcling/v6-14-00-patches/core/meta/src/TMethodCall.cxx:457 root-project#35 0x0000000000401009 in TMethodCall::Execute(long&) () root-project#36 0x00000000004010ea in long TPluginHandler::ExecPluginImpl<>() () root-project#37 0x000000000040106d in long TPluginHandler::ExecPlugin<>(int) () root-project#38 0x0000000000400e21 in mytest() () root-project#39 0x0000000000400e92 in main ()

Before, MetaParser might have pointed to a StringRef whose storage was gone, see asan failure in roottest/cling/other/runfileClose.C below. This was caused by recursive uses of MetaParser; see stack trace below: the inner recursion returned, but as the same MetaParser object was used by both frames, the objects cursor now pointed to freed memory. Instead, create a MetaParser (and MetaLexer) object per input. That way, their lifetime corresponds to the lifetime of their input. ================================================================= ==529104==ERROR: AddressSanitizer: stack-use-after-return on address 0x7ffff3afd82a at pc 0x7fffea18df6d bp 0x7fffffff8170 sp 0x7fffffff8168 READ of size 1 at 0x7ffff3afd82a thread T0 [Detaching after fork from child process 529183] #0 0x7fffea18df6c in cling::MetaLexer::Lex(cling::Token&) src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:58:11 #1 0x7fffea190d7c in cling::MetaParser::lookAhead(unsigned int) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:89:15 #2 0x7fffea190bd5 in cling::MetaParser::consumeToken() src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:49:5 #3 0x7fffea191d4d in cling::MetaParser::isLCommand(cling::MetaSema::ActionResult&) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:147:9 #4 0x7fffea1914dd in cling::MetaParser::isCommand(cling::MetaSema::ActionResult&, cling::Value*) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:123:12 #5 0x7fffea191216 in cling::MetaParser::isMetaCommand(cling::MetaSema::ActionResult&, cling::Value*) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:101:33 #6 0x7fffea14e5aa in cling::MetaProcessor::process(llvm::StringRef, cling::Interpreter::CompilationResult&, cling::Value*, bool) src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:317:24 #7 0x7fffe99b67b7 in HandleInterpreterException(cling::MetaProcessor*, char const*, cling::Interpreter::CompilationResult&, cling::Value*) src/core/metacling/src/TCling.cxx:2431:29 #8 0x7fffe99bde30 in TCling::Load(char const*, bool) src/core/metacling/src/TCling.cxx:3454:10 #9 0x7ffff7865f11 in TSystem::Load(char const*, char const*, bool) src/core/base/src/TSystem.cxx:1941:27 #10 0x7ffff7b8a0e3 in TUnixSystem::Load(char const*, char const*, bool) src/core/unix/src/TUnixSystem.cxx:2789:20 #11 0x7fffd78dd08b (<unknown module>) #12 0x7fffe9f8a5d9 in cling::IncrementalExecutor::executeWrapper(llvm::StringRef, cling::Value*) const src/interpreter/cling/lib/Interpreter/IncrementalExecutor.cpp:376:3 #13 0x7fffe9d73dc2 in cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1141:20 #14 0x7fffe9d6e317 in cling::Interpreter::EvaluateInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1391:29 #15 0x7fffe9d6c1fe in cling::Interpreter::process(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::Value*, cling::Transaction**, bool) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:819:9 #16 0x7fffea151826 in cling::MetaProcessor::readInputFromFile(llvm::StringRef, cling::Value*, unsigned long, bool) src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:507:22 #17 0x7fffe99b585b in TCling::ProcessLine(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:2570:39 #18 0x7fffe99bbfee in TCling::ProcessLineSynch(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:3496:17 #19 0x7ffff77203d3 in TApplication::ExecuteFile(char const*, int*, bool) src/core/base/src/TApplication.cxx:1608:30 #20 0x7ffff771ebdf in TApplication::ProcessFile(char const*, int*, bool) src/core/base/src/TApplication.cxx:1480:11 #21 0x7ffff771e385 in TApplication::ProcessLine(char const*, bool, int*) src/core/base/src/TApplication.cxx:1453:14 #22 0x7ffff7f8157a in TRint::ProcessLineNr(char const*, char const*, int*) src/core/rint/src/TRint.cxx:766:11 #23 0x7ffff7f802f0 in TRint::Run(bool) src/core/rint/src/TRint.cxx:424:22 #24 0x4ff96d in main src/main/src/rmain.cxx:30:12 #25 0x7ffff6e040b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 root-project#26 0x41f35d in _start (asan/bin/root.exe+0x41f35d) Address 0x7ffff3afd82a is located in stack of thread T0 at offset 42 in frame #0 0x7fffe99b3d8f in TCling::ProcessLine(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:2456 This frame has 21 object(s): [32, 56) 'sLine' (line 2462) <== Memory access at offset 42 is inside this variable [96, 104) 'R__guard2471' (line 2471) [128, 136) 'R__guard2488' (line 2488) [160, 176) 'interpreterFlagsRAII' (line 2491) [192, 240) 'result' (line 2511) [272, 276) 'compRes' (line 2512) [288, 312) 'mod_line' (line 2517) [352, 376) 'aclicMode' (line 2518) [416, 440) 'arguments' (line 2519) [480, 504) 'io' (line 2520) [544, 568) 'fname' (line 2521) [608, 632) 'ref.tmp' (line 2547) [672, 696) 'ref.tmp145' (line 2547) [736, 768) 'code' (line 2555) [800, 832) 'codeline' (line 2556) [864, 1384) 'in' (line 2559) [1520, 1552) 'ref.tmp176' (line 2562) [1584, 1600) 'agg.tmp' [1616, 1624) 'ref.tmp198' (line 2568) [1648, 1664) 'agg.tmp207' [1680, 1696) 'autoParseRaii' (line 2588) HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-use-after-return src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:58:11 in cling::MetaLexer::Lex(cling::Token&) Shadow bytes around the buggy address: 0x10007e757ab0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757ac0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757ad0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757ae0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757af0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 =>0x10007e757b00: f5 f5 f5 f5 f5[f5]f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757b10: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757b20: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757b30: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757b40: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x10007e757b50: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==529104==ABORTING at src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:49 at src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:41 compRes=@0x7ffff3afd910: cling::Interpreter::kSuccess, result=0x7ffff3afd8c0, disableValuePrinting=false) at src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:314 input_line=0x7ffff3afd829 "#define XYZ 21", compRes=@0x7ffff3afd910: cling::Interpreter::kSuccess, result=0x7ffff3afd8c0) at src/core/metacling/src/TCling.cxx:2431 error=0x7fffd78cb0f4 <x>) at src/core/metacling/src/TCling.cxx:2591 sync=false, err=0x7fffd78cb0f4 <x>) at src/core/base/src/TApplication.cxx:1472 line=0x7fffd78c9000 "#define XYZ 21", error=0x7fffd78cb0f4 <x>) at src/core/base/src/TROOT.cxx:2328 from asan/roottest/cling/other/fileClose_C.so filename=0x6070000f0fd0 "asan/roottest/cling/other/fileClose_C.so", flag=257) at /home/axel/build/llvm/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:6270 at src/interpreter/cling/lib/Utils/PlatformPosix.cpp:118 permanent=false, resolved=true) at src/interpreter/cling/lib/Interpreter/DynamicLibraryManager.cpp:184 at src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1444 T=0x0) at src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1560 at src/interpreter/cling/lib/MetaProcessor/MetaSema.cpp:57 actionResult=@0x7ffff39532b0: cling::MetaSema::AR_Success)

This tutorial crashed with the following stacktrace when run interactively. Avoiding to register a canvas with the same name multiple times fixes the crash: ``` #8 0x00007f5b7876967d in TCanvas::Resize(char const*) (this=0x55e768e126c0) at ../graf2d/gpad/src/TCanvas.cxx:1740 #9 0x00007f5b3e90d668 in TRootCanvas::HandleContainerConfigure(Event_t*) (this=0x55e76852b460) at ../gui/gui/src/TRootCanvas.cxx:1789 #10 0x00007f5b3e8464fd in TGFrame::HandleEvent(Event_t*) (this=0x55e767938e70, event=0x7f5b79adff40) at ../gui/gui/src/TGFrame.cxx:476 #11 0x00007f5b3e7f4c9a in TGClient::HandleEvent(Event_t*) (this=0x55e768de2290, event=0x7f5b79adff40) at ../gui/gui/src/TGClient.cxx:846 #12 0x00007f5b3e7f531d in TGClient::ProcessOneEvent() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:656 #13 TGClient::ProcessOneEvent() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:648 #14 0x00007f5b3e7f536b in TGClient::HandleInput() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:703 #15 0x00007f5b8dcb0ff8 in TUnixSystem::DispatchOneEvent(bool) (this=0x55e75ccfd080, pendingOnly=<optimized out>) at ../core/unix/src/TUnixSystem.cxx:1067 #16 0x00007f5b8dbd0dca in TSystem::ProcessEvents() (this=0x55e75ccfd080) at ../core/base/src/TSystem.cxx:424 #17 0x00007f5b8130600d in () #18 0x00007f5b79ae0450 in () #19 0x00007f5b8de5215f in WrapperCall(Cppyy::TCppMethod_t, size_t, void*, void*, void*) (method=94452242807424, nargs=0, args_=0x7f5b79ae01d7, self=0x55e75ccfd080, result=0x7f5b79ae01d7) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:778 #20 0x00007f5b8de527cf in CallT<unsigned char> (args=<optimized out>, nargs=<optimized out>, self=<optimized out>, method=<optimized out>) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:816 #21 Cppyy::CallB(long, void*, unsigned long, void*) (method=<optimized out>, self=<optimized out>, nargs=<optimized out>, args=<optimized out>) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:833 #22 0x00007f5b8decdc0f in GILCallB (ctxt=0x7f5b79ae0430, self=<optimized out>, method=<optimized out>) at ../bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:69 #23 CPyCppyy::(anonymous namespace)::BoolExecutor::Execute(Cppyy::TCppMethod_t, Cppyy::TCppObject_t, CPyCppyy::CallContext*) (this=<optimized out>, method=<optimized out>, self=<optimized out>, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:148 #24 0x00007f5b8deba4c9 in CPyCppyy::CPPMethod::ExecuteFast(void*, long, CPyCppyy::CallContext*) (self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>, this=<optimized out>, this=<optimized out>) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:74 #25 0x00007f5b8debd3a8 in CPyCppyy::CPPMethod::ExecuteProtected(void*, long, CPyCppyy::CallContext*) (this=this entry=0x55e760617f50, self=0x55e75ccfd080, offset=0, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:149 root-project#26 0x00007f5b8debb6fa in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) (this=this entry=0x55e760617f50, self=self entry=0x55e75ccfd080, offset=<optimized out>, ctxt=ctxt entry=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:728 root-project#27 0x00007f5b8debc46c in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object*, _object*, CPyCppyy::CallContext*) (this=0x55e760617f50, self= 0x7f5b8080ef50: 0x7f5b808043c0, args=0x7f5b8e1ab040, kwds=<optimized out>, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:783 root-project#28 0x00007f5b8dec09fe in CPyCppyy::(anonymous namespace)::mp_call(CPyCppyy::CPPOverload*, PyObject*, PyObject*) (pymeth=0x7f5b8080ef40, args=0x7f5b8e1ab040, kwds=0x0) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPOverload.cxx:566 root-project#29 0x00007f5b8e941333 in _PyObject_MakeTpCall () at /usr/lib/libpython3.9.so.1.0 root-project#30 0x00007f5b8e93d218 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0 root-project#31 0x00007f5b8e936fd9 in () at /usr/lib/libpython3.9.so.1.0 root-project#32 0x00007f5b8e948b8e in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0 root-project#33 0x00007f5b8e93aec9 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0 root-project#34 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0 root-project#35 0x00007f5b8e93858e in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0 root-project#36 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0 root-project#37 0x00007f5b8e93858e in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0 root-project#38 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0 root-project#39 0x00007f5b8e95795b in () at /usr/lib/libpython3.9.so.1.0 root-project#40 0x00007f5b8ea3cac6 in () at /usr/lib/libpython3.9.so.1.0 root-project#41 0x00007f5b8ea17554 in () at /usr/lib/libpython3.9.so.1.0 root-project#42 0x00007f5b8e62c259 in start_thread () at /usr/lib/libpthread.so.0 root-project#43 0x00007f5b8e7425e3 in clone () at /usr/lib/libc.so.6 ```

The test was dynamically allocating the array data members of the `Data` struct, but never deallocating them. This commit polishes the `Data` struct definition and ensures proper management of the data members. The previous way of writing data to the TTree was leading to a bad memory access in the ReadBasicPointer inlined function in TStreamerInfoReadBuffer.cxx while reading the `double*` array. In particular, the issue arises when accessing and then deallocating the array at the current index provided by the `TCompInfo` object. ``` Target 0: (repro.out) stopped. (lldb) Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf140 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] 920 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong: ReadBasicPointer(Long_t); continue; 921 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong64: ReadBasicPointer(Long64_t); continue; 922 case TStreamerInfo::kOffsetP + TStreamerInfo::kFloat: ReadBasicPointer(Float_t); continue; -> 923 case TStreamerInfo::kOffsetP + TStreamerInfo::kDouble: ReadBasicPointer(Double_t); continue; 924 case TStreamerInfo::kOffsetP + TStreamerInfo::kUChar: ReadBasicPointer(UChar_t); continue; 925 case TStreamerInfo::kOffsetP + TStreamerInfo::kUShort: ReadBasicPointer(UShort_t); continue; 926 case TStreamerInfo::kOffsetP + TStreamerInfo::kUInt: ReadBasicPointer(UInt_t); continue; Target 0: (repro.out) stopped. (lldb) Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf184 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) [inlined] TBuffer::BufferSize(this=0x000060e00010ef00) const at TBuffer.h:98:41 [opt] 95 TObject *GetParent() const; 96 char *Buffer() const { return fBuffer; } 97 char *GetCurrent() const { return fBufCur; } -> 98 Int_t BufferSize() const { return fBufSize; } 99 void DetachBuffer() { fBuffer = nullptr; } 100 Int_t Length() const { return (Int_t)(fBufCur - fBuffer); } 101 void Expand(Int_t newsize, Bool_t copy = kTRUE); // expand buffer to newsize Target 0: (repro.out) stopped. (lldb) p fBufSize (Int_t) 32008 (lldb) s Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf194 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] 920 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong: ReadBasicPointer(Long_t); continue; 921 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong64: ReadBasicPointer(Long64_t); continue; 922 case TStreamerInfo::kOffsetP + TStreamerInfo::kFloat: ReadBasicPointer(Float_t); continue; -> 923 case TStreamerInfo::kOffsetP + TStreamerInfo::kDouble: ReadBasicPointer(Double_t); continue; 924 case TStreamerInfo::kOffsetP + TStreamerInfo::kUChar: ReadBasicPointer(UChar_t); continue; 925 case TStreamerInfo::kOffsetP + TStreamerInfo::kUShort: ReadBasicPointer(UShort_t); continue; 926 case TStreamerInfo::kOffsetP + TStreamerInfo::kUInt: ReadBasicPointer(UInt_t); continue; Target 0: (repro.out) stopped. (lldb) s Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xbebebebebebebeae) frame #0: 0x0000000107bac674 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) + 76 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate: -> 0x107bac674 <+76>: casalb w8, w9, [x22] 0x107bac678 <+80>: cmp w8, #0x2 0x107bac67c <+84>: b.ne 0x107bac6f4 ; <+204> 0x107bac680 <+88>: mov x8, #-0x100000000 ; =-4294967296 Target 0: (repro.out) stopped. (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xbebebebebebebeae) * frame #0: 0x0000000107bac674 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) + 76 frame #1: 0x0000000107c0c444 libclang_rt.asan_osx_dynamic.dylib`wrap__ZdaPv + 232 frame #2: 0x00000001044d4a60 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] frame #3: 0x0000000103ffc888 libRIO.so`TStreamerInfoActions::GenericReadAction(buf=0x000060e00010ef00, addr=0x0000602000056bd0, config=0x0000604000149910) at TStreamerInfoActions.cxx:195:45 frame #4: 0x0000000103caa5ec libRIO.so`TStreamerInfoActions::TConfiguredAction::operator()(this=0x00006030001693f0, buffer=0x000060e00010ef00, object=0x0000602000056bd0) const at TStreamerInfoActions.h:123:17 frame #5: 0x0000000103ca9ef8 libRIO.so`TBufferFile::ApplySequence(this=0x000060e00010ef00, sequence=0x000060600011ac20, obj=0x0000602000056bd0) at TBufferFile.cxx:3702:10 frame #6: 0x00000001064bc570 libTree.so`TBranchElement::ReadLeavesMemberBranchCount(this=0x0000619000566380, b=0x000060e00010ef00) at TBranchElement.cxx:4603:6 frame #7: 0x0000000106455ce4 libTree.so`TBranch::GetEntry(this=0x0000619000566380, entry=0, getall=0) at TBranch.cxx:1753:4 frame #8: 0x00000001064a1764 libTree.so`TBranchElement::GetEntry(this=0x0000619000566380, entry=0, getall=0) at TBranchElement.cxx:2783:27 frame #9: 0x000000010739915c libTreePlayer.so`ROOT::Detail::TBranchProxy::Read(this=0x00006110000c9580) at TBranchProxy.h:163:42 frame #10: 0x0000000107649ba8 libTreePlayer.so`(anonymous namespace)::TObjectArrayReader::At(this=0x0000603000169900, proxy=0x00006110000c9580, idx=1) at TTreeReaderArray.cxx:176:22 frame #11: 0x000000010000c2e4 repro.out`ROOT::Internal::TTreeReaderArrayBase::UntypedAt(this=0x000000016fdfe740, idx=1) const at TTreeReaderArray.h:41:62 frame #12: 0x000000010000c200 repro.out`TTreeReaderArray<double>::At(this=0x000000016fdfe740, idx=1) at TTreeReaderArray.h:205:54 frame #13: 0x00000001000065e0 repro.out`TTreeReaderArray<double>::operator[](this=0x000000016fdfe740, idx=1) at TTreeReaderArray.h:207:44 frame #14: 0x0000000100007b48 repro.out`simpleTest() at repro.cpp:123:26 frame #15: 0x0000000100007e10 repro.out`main at repro.cpp:128:5 frame #16: 0x000000018c718274 dyld`start + 2840 ```

egpbos self-assigned this Oct 19, 2017

egpbos mentioned this issue Oct 30, 2017

Test RooGradMinimizer #11

Open

egpbos mentioned this issue Oct 30, 2017

Find out why RooGradMinimizer::hesse() only takes 5 function calls, whereas MnHesse called from VariableMetricBuilder::Minimum takes 11 #12

Open

egpbos closed this as completed Oct 30, 2017

egpbos mentioned this issue Oct 30, 2017

Implement MnHesse::operator()(const FCNGradientBase&, ...) versions #13

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement RooGradMinimizer using exact MINUIT gradient algorithm #10

Implement RooGradMinimizer using exact MINUIT gradient algorithm #10

egpbos commented Oct 19, 2017

egpbos commented Oct 19, 2017

egpbos commented Oct 19, 2017

egpbos commented Oct 20, 2017

egpbos commented Oct 24, 2017

egpbos commented Oct 25, 2017

egpbos commented Oct 25, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017 •

edited

Loading

egpbos commented Oct 26, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading

egpbos commented Oct 30, 2017

egpbos commented Oct 30, 2017

Implement RooGradMinimizer using exact MINUIT gradient algorithm #10

Implement RooGradMinimizer using exact MINUIT gradient algorithm #10

Comments

egpbos commented Oct 19, 2017

egpbos commented Oct 19, 2017

egpbos commented Oct 19, 2017

egpbos commented Oct 20, 2017

egpbos commented Oct 24, 2017

egpbos commented Oct 25, 2017

egpbos commented Oct 25, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017

egpbos commented Oct 26, 2017 • edited Loading

egpbos commented Oct 26, 2017 • edited Loading

egpbos commented Oct 30, 2017 • edited Loading

egpbos commented Oct 30, 2017 • edited Loading

egpbos commented Oct 30, 2017 • edited Loading

egpbos commented Oct 30, 2017

egpbos commented Oct 30, 2017

egpbos commented Oct 26, 2017 •

edited

Loading

egpbos commented Oct 26, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading

egpbos commented Oct 30, 2017 •

edited

Loading