Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug with get_filtered_records function #15

Open
mgrobaker opened this issue Oct 19, 2020 · 3 comments
Open

bug with get_filtered_records function #15

mgrobaker opened this issue Oct 19, 2020 · 3 comments

Comments

@mgrobaker
Copy link

I am not able to download data using the python library. It could be an issue on my end, but I don't think so

When I follow instructions here, I get this error

Traceback (most recent call last):
  File "test_openeew.py", line 18, in <module>
    eew_data = eew_aws_client.get_filtered_records(start_time, end_time)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/openeew/data/aws.py", line 350, in get_filtered_records
    key_records = loop.run_until_complete(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/openeew/data/aws.py", line 305, in _download_keys
    key_records = await asyncio.gather(*coros)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/openeew/data/aws.py", line 276, in _get_records_from_key
    await async_s3_client.download_fileobj(
AttributeError: 'ClientCreatorContext' object has no attribute 'download_fileobj'
sys:1: RuntimeWarning: coroutine 'AioSession._create_client' was never awaited
@pdornian
Copy link

pdornian commented Nov 8, 2020

I'm having the exact same issue.

Trying to execute code as written in https://openeew.com/docs/analyze-historic gives

RuntimeError                              Traceback (most recent call last)
<ipython-input-10-5ac17f81d3ef> in <module>
----> 1 records = data_client.get_filtered_records(start_date_utc, end_date_utc)

D:\Applications\Anaconda3\lib\site-packages\openeew\data\aws.py in get_filtered_records(self, start_date_utc, end_date_utc, device_ids)
    348         loop = asyncio.get_event_loop()
    349 
--> 350         key_records = loop.run_until_complete(
    351                 self._download_keys(keys_to_download)
    352                 )

D:\Applications\Anaconda3\lib\asyncio\base_events.py in run_until_complete(self, future)
    590         """
    591         self._check_closed()
--> 592         self._check_running()
    593 
    594         new_task = not futures.isfuture(future)

D:\Applications\Anaconda3\lib\asyncio\base_events.py in _check_running(self)
    550     def _check_running(self):
    551         if self.is_running():
--> 552             raise RuntimeError('This event loop is already running')
    553         if events._get_running_loop() is not None:
    554             raise RuntimeError(

RuntimeError: This event loop is already running

Executing the code with the nested asynchio fix demonstrated in this repo's jupyter notebook

# Allow nested asyncio event loop
# See https://github.com/erdewit/nest_asyncio
import nest_asyncio
nest_asyncio.apply()

then produces the same error as @mgrobaker .

@krook
Copy link
Member

krook commented Nov 11, 2020

Thoughts on this @andygrillo?

@mallmanj
Copy link
Contributor

Thanks for reporting this error. It looks like it's caused by a breaking change in aiobotocore to do with how clients are created: https://github.com/aio-libs/aiobotocore/blob/master/CHANGES.rst#100-2020-03-31.

As mentioned in https://github.com/terrycain/aioboto3#async-aws-sdk-for-python, clients must now be created as async context managers. As it happens, this change has already been made as part of the unreleased #10. Until a new version of the library has been released, using the current master branch should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants