not able to create pandas dataframe

[sv3048]

Hi developers !

I am creating a pandas dataframe but issue is its giving me tuple each time.

AttributeError Traceback (most recent call last)
/tmp/ipykernel_557/1180174254.py in
2 import pandas as pd
3
----> 4 dfall.columns

AttributeError: 'tuple' object has no attribute 'columns'

Here is the code that I run for making pandas dataframe.

filename = "/eos/cms/store/group/phys_higgs/cmshww/amassiro/HWWNano/Summer20UL18_106x_nAODv9_Full2018v9/MCl1loose2018v9__MCCorr2018v9NoJERInHorn__MCCombJJLNu2018/nanoLatino_GluGluToWWToQQ_Sig_private__part9.root"
file = uproot.open(filename)
 

# show what is inside the root file loaded from uproot
print(file.classnames())
print(file.keys())

tree = file["Events"]  # select the TTree inside the root file
tree.show()  # show all the branches inside the TTree
dfall = tree.arrays(library="pd")  # convert uproot TTree into pandas dataframe
#dfall.columns
print("type of dfall", type(dfall))
print("============================================")
print("File loaded with ", len(dfall), " events ")

Thanks,
Sadhana

[jpivarski]

Uproot 4.x tries to "explode" ragged data, so that an array of variable numbers of particles per event are turned into a DataFrame with numbers in the cells and MultiIndex rows, indicating the nesting, similar to ak.to_dataframe.

But this isn't always possible. If you are trying to read, for instance, both muons and electrons, the numbers of particles in these two collections are not in general (or even usually) equal to each other, so there's no single MultiIndex that they can both expand to. In that case, Uproot 4.x produces a tuple of DataFrames, one for each particle type.

Uproot 5.x, however, uses Akimbo to put lists of numbers into each cell (instead of individual numbers) with a normal index. That's because dataframe libraries are starting to use Arrow format, rather than Python lists, and it's not a big performance loss to do so.

So you have two options: 1. without installing any new packages, select (with expressions or filter_names) a single type of particle from the ROOT file. You can do additional calls to get the other particle types into other DataFrames. Or 2. upgrade to the latest version of Uproot.

[sv3048]

Hi Jim !

I would prefer for now method 2.

!pip install uproot==5.4.1 awkward==2.6.9

Defaulting to user installation because normal site-packages is not writeable
Collecting uproot==5.4.1
Using cached uproot-5.4.1-py3-none-any.whl.metadata (32 kB)
Collecting awkward==2.6.9
Using cached awkward-2.6.9-py3-none-any.whl.metadata (7.0 kB)
Requirement already satisfied: cramjam>=2.5.0 in /eos/home-i02/s/sverma/.local/lib/python3.9/site-packages (from uproot==5.4.1) (2.8.2)
Requirement already satisfied: fsspec in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from uproot==5.4.1) (2023.10.0)
Requirement already satisfied: numpy in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from uproot==5.4.1) (1.23.5)
Requirement already satisfied: packaging in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from uproot==5.4.1) (23.1)
Requirement already satisfied: typing-extensions>=4.1.0 in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from uproot==5.4.1) (4.5.0)
Requirement already satisfied: xxhash in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from uproot==5.4.1) (3.4.1)
Collecting awkward-cpp==39 (from awkward==2.6.9)
Using cached awkward_cpp-39-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.2 kB)
Requirement already satisfied: importlib-metadata>=4.13.0 in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from awkward==2.6.9) (6.6.0)
Requirement already satisfied: zipp>=0.5 in /cvmfs/sft.cern.ch/lcg/views/LCG_105a_swan/x86_64-el9-gcc13-opt/lib/python3.9/site-packages (from importlib-metadata>=4.13.0->awkward==2.6.9) (3.15.0)
Using cached uproot-5.4.1-py3-none-any.whl (357 kB)
Using cached awkward-2.6.9-py3-none-any.whl (835 kB)
Using cached awkward_cpp-39-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (636 kB)
DEPRECATION: gosam 2.1.1-4b98559 has a non-standard version number. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of gosam or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at pypa/pip#12063
DEPRECATION: professor2 X.Y.Z has a non-standard version number. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of professor2 or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at pypa/pip#12063
Installing collected packages: awkward-cpp, awkward, uproot
Attempting uninstall: awkward-cpp
Found existing installation: awkward-cpp 30
Uninstalling awkward-cpp-30:
Successfully uninstalled awkward-cpp-30
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
zfit 0.16.0 requires tensorflow<2.14,>=2.13, which is not installed.
zfit 0.16.0 requires tensorflow-probability<0.22,>=0.20, which is not installed.
coffea 0.7.21 requires awkward<2,>=1.10.3, but you have awkward 2.6.9 which is incompatible.
coffea 0.7.21 requires uproot!=4.2.4,!=4.3.0,!=4.3.1,==4.*,>=4.1.6, but you have uproot 5.4.1 which is incompatible.
servicex 2.6.2 requires awkward<2,>=1.0.1, but you have awkward 2.6.9 which is incompatible.
servicex 2.6.2 requires idna==2.10, but you have idna 3.2 which is incompatible.
servicex 2.6.2 requires uproot<5,>=4.0.1, but you have uproot 5.4.1 which is incompatible.
pylhe 0.8.0 requires vector>=0.8.1, but you have vector 0.0.0 which is incompatible.
scikit-hep 2024.3.1 requires decaylanguage~=0.18.0, but you have decaylanguage 0.15.4 which is incompatible.
scikit-hep 2024.3.1 requires hepunits~=2.3.0, but you have hepunits 2.2.0 which is incompatible.
scikit-hep 2024.3.1 requires hist~=2.7.0, but you have hist 2.6.2 which is incompatible.
scikit-hep 2024.3.1 requires histoprint~=2.4.0, but you have histoprint 2.2.0 which is incompatible.
scikit-hep 2024.3.1 requires iminuit~=2.25.0, but you have iminuit 2.24.0 which is incompatible.
scikit-hep 2024.3.1 requires uproot~=5.3.0, but you have uproot 5.4.1 which is incompatible.
scikit-hep 2024.3.1 requires vector~=1.2.0, but you have vector 0.0.0 which is incompatible.
Successfully installed awkward-2.6.2 awkward-cpp-39 uproot-5.3.0

when I try to install any newer version of uproot. I am getting a lot of warnings for other packages. I am working inside swan. I don't know is it solely related now to uproot issue or I should ask somewhere else.

Thanks,
Sadhana