aggregations to dataframe always misses the first pagination

[Andy7475]

Hi,
This is a really useful package, thank you! I noticed though that scan_composite_agg has a bug. It misses the first page of aggregations. I think it is because you declare variable 'buckets', enter while loop, re-declare it, then iterate over it. So the first version of buckets never has a chance to be iterated over, which is why I presume we always miss the 1st pagination. . Hope that makes sense, think you just need to move a line

YOU DECLARE BUCKETS HERE
buckets: List[BucketDict] = r.aggregations.data[a_name][ # type: ignore
"buckets"
]
after_key: AfterKey = r.aggregations.data[a_name]["after_key"] # type: ignore

    init: bool = True
    while init or len(buckets) == size:
        init = False
        s._aggs = s._aggs.as_composite(size=size, after=after_key)
        r = s.execute()
        agg_clause_response = r.aggregations.data[a_name]

THEN CHANGE IT HERE, BEFORE YOU HAVE HAD A CHANCE TO ITERATE OVER THE OLD ONE
buckets = agg_clause_response["buckets"] # type: ignore ****MOVE THIS LINE TO LATER
for bucket in buckets:

[leonardbinet]

Hi @Andy7475 , thanks for finding this bug 👍 I'll merge the fix as soon as I regain admin rights on this repo.

[leonardbinet]

@Andy7475 fixes/features are available on pandagg 0.2.4 version (#115)

[Andy7475]

Hi Leonard, I can see you have modified the code, but doing 'pip install pandagg' is still pulling the old code with the error for some reason (even with 0.2.4); I am modifying it locally for the time being. Just thought you would like to know :) Andy

…

On Mon, 14 Feb 2022 at 16:52, Léonard Binet ***@***.***> wrote: Hi @Andy7475 <https://github.com/Andy7475> , thanks for finding this bug 👍 I'll merge the fix as soon as I regain admin rights on this repo. — Reply to this email directly, view it on GitHub <#101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADBY27FMYZLJMR7SMCVOHCLU3EXL7ANCNFSM5KJD6MAA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Andy7475]

Subject: Request for a new release of Pandagg on PyPI

Hi Leonard,
I hope this email finds you well. I was trying to install the latest version of the package via pip, but I noticed that the version on PyPI (0.2.4) is not the same as the latest release on GitHub master branch (0.2.1). 0.2.4 [dev branch] has a bug in search.py file with the line starting raw_data =..., but the master branch looks good.

I was wondering if it would be possible for you to publish a new version of the package from the master branch to PyPI, so that users can easily install the latest version using pip.

I understand that this may not be a priority for you, and I would be happy to assist in any way that I can. If there is anything I can do to help, please let me know.

Thank you for your time and for maintaining such a valuable package. I look forward to your response.

All the best,

Andy