Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot read R lists #8

Open
DachuanZhao opened this issue Feb 28, 2019 · 10 comments
Open

Cannot read R lists #8

DachuanZhao opened this issue Feb 28, 2019 · 10 comments
Labels
enhancement New feature or request waiting for librdata changes the issue needs some fixes to the C library librdata before it can be solved

Comments

@DachuanZhao
Copy link

It gets wrong results when I load a RData.Here is the file
FILE

@ofajardo
Copy link
Owner

Would you be so kind of indicating what exactly is wrong?
Also, I do not have access to the file, I have requested it through dropbox.

@DachuanZhao
Copy link
Author

image

When I load it in python,it only have one key.I think it should like this:

{"historicalCongestion":pandas.DataFrame,
"testData":list,
"trainingData":list}

@ofajardo
Copy link
Owner

From the preadr readme, known limitations section :

Lists are not read

The limitation comes from librdata. I have filed an issue but it will take time to eventually get fixed.

@ofajardo ofajardo reopened this Mar 30, 2019
@ofajardo ofajardo added enhancement New feature or request waiting for librdata changes the issue needs some fixes to the C library librdata before it can be solved labels Mar 30, 2019
@ofajardo ofajardo changed the title Wrong results when I load a RData.Here is the file Cannot read R lists Mar 30, 2019
@JoaoCarabetta
Copy link

Hey @ofajardo,

I noticed that your issue was recently solved by this pull request.

I am working on a project that could use this update.

Are you already working on fixing this issue? Do you have a time horizon on when to fix this issue?

Thanks :)

@ofajardo
Copy link
Owner

Hi Joao
I am going to work on that next week, as this week I am attending EuroPython (Do you happen to be here?)
But notice that pull request solves only reading a R vector with type DATE ( issue #9) it will not solve reading lists (this issue). Is this what you are expecting?

@JoaoCarabetta
Copy link

Otto,

Unfortunately I am not attending the EuroPython, but I'll definitely check out the talks online.

About the problem, I actually need more than dates. My current .rds has shapefiles in it. I posted the problem in StackOverflow if you want more details.

Do you know any other solution to read those besides rpy2?

@ofajardo
Copy link
Owner

ofajardo commented Jul 10, 2019

interesting. Actually that shape file is an S3 object of class "sf" that is just on top of a normal data frame (which in turn is an object on top of a list). You can check this in R by doing class(shapeobject).
The interesting thing is that pyreadr can read for example tibbles which are also S3 objects on top of data frames. I tried to change the class of this shape file to just data.frame and now pyreadr gives a different error: the file has unsupported features. So there are more things in this object that pyreadr/librdata cannot read.

As far as my knowledge goes you are out of luck and you will have to rely on rpy2. R interoperability is awful and most of the times you will need R in order to read R objects. The only other choice is to save them in some more interoperable format. =(

You can also submit an issue to librdata directly asking for support to read this objects. As they are on top of a data frame maybe it is not too hard for them to implement it, but no idea. If they implement the support then I can take that and expose it from pyreadr.

@ofajardo
Copy link
Owner

ofajardo commented Jul 10, 2019

Oh, now I see that the geometry this dataframe contains nested objects which seems to be built on top of a list of matrices (?). In addition of not reading lists, Librdata can also not read nested objects in dataframes, and in addition it does not read the dimensions of matrices, so supporting correctly this looks far fetched.
Anyway you can raise the issue in librdata if you like, to raise the awareness on them that these missing featured are required by the community.

@JoaoCarabetta
Copy link

Hey,

thanks for the attention.

I think we are changing the format to something more interpretable. Then we don't need to rely on several libraries to be updated.

But, if you think that the problem is interesting, this may have great value to future geo projects :)

@ofajardo
Copy link
Owner

Issue tracked here: WizardMac/librdata#32

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request waiting for librdata changes the issue needs some fixes to the C library librdata before it can be solved
Projects
None yet
Development

No branches or pull requests

3 participants