Method for exposing the raw data

[paalge]

This is a method for exposing the raw data as a numpy array.
Not sure if this is exactly the way you want to do it, but it looks like it is working for me

Fixes #47

[SamWhited]

We definitely don't want a hard dependency on numpy. Can we use a regular array for now, and we'll figure out a good general API to support numpy (hopefully w/o copying data to Python land first) later?

[paalge]

I normally use numpy, so I have very little experience with the regular array. As long as you can write to a normal uint16 array from an iterator numpy isn't needed.

[SamWhited]

Yah, I just meant a normal Python array. It won't be very efficient, but I've also opened an issue (#106) for discussing the best way to handle numpy support, which is something we want to do.

[paalge]

II understand. Can you read from an iterator into such an array?

[SamWhited]

Yup, that should be fine. You won't be able to use the numpy methods to do it though; just regular iteration will work fine.

[paalge]

I've removed the numpy dependency and changed the error handling

[SamWhited]

Ah, this makes sense, thanks! Let's catch the actual exception here though; bare excepts can have unintended side effects :)

[SamWhited]

This might be cleaner, no idea if there's a "standard" way to do this in Python:

data = [None] * iheight * iwidth

It might be a pre-mature optimization either way though. Or I may be completely wrong. Just thinking out loud. @campaul?

[paalge]

It needs to be 2D, or you'll have to pass height and width on, and let the calling function do the reshape.
From what I understood, list compression is the normal way of initialising this.

[SamWhited]

Sounds good then. I think the above will make a 2d list too, but I could be mistaken. If we're going to have to iterate anyways though I think we might as well not preallocate.

[paalge]

A quick quick test gives it as a 1D.
As for the preallocate, I like it due to the large lists being created, though I don't know what is fastest.

[SamWhited]

If we're going to loop and append, 0's though we might as well just loop and append the data. I'm not sure this is actually an optimization, unless Python does some sort of optimization on simple list comprehensions that I don't know about.

[campaul]

This would probably be more efficient. I'll do some timings later and see for sure.

data = [[] for j in range(iwidth)]

for ii in range(iheight):
    for jj in range(iwidth):
        data[jj].append(data_pointer[ii * iwidth + jj])

[SamWhited]

This still appears to be subscripting a 2D array; won't this be broken?

[paalge]

In my test run this works as intended.
Though I'm not sure what you think would be broken?
Line 221 makes a 2D list, which one can index with [][] in line 225

[SamWhited]

Oh right, yah, I was misunderstanding a large chunk of our discussion, sorry. I think it's best to make this a 1D list to match the data that we get from LibRaw / elsewhere in rawkit. Not sure though; will consider the API implications of doing this and get back to you. It's not super important right now.

[campaul]

Can we change ii and jj to y and x respectively so we're working with Cartesian coordinates?

[SamWhited]

These aren't actually coordinates on the image though since there are multiple indexes per pixel.

[paalge]

They will be coordinates, though you have to interpolate the image first, as in that loop they change between different colour channels.

[campaul]

@paalge if you go ahead and add a get_3_col_raw function, update the docs to say the functions are experimental, and then squash your commits I'm happy merging this.

We'll be making some changes to the interface as time goes on, but I want to get the core functionality merged in so I can start playing with it.

[paalge]

@campaul I've now added the requested method. I also fixed a lot of error in the original code. The returned image is the image excluding the calibration pixels (it could be possible to switch between this in an input flag). I've only tested the code one three types of images (ARW, NEF and three colour CR2). The three colour image is strange, as it includes a zero pixel.
Can you have a look at the code? I'll squash the git when you've taken a look

[campaul]

In general the code looks good, but the two functions are almost identical. Can you extract the shared code into a helper function so we aren't duplicating everything?

[paalge]

I'm sorry for the late update. I've now moved the common code into a new function, though I'm unsure of how much better it is with respect to readability, as there where some subtle differences in the code.

[campaul]

Oops, I didn't see the email from github for your last comment. I had already taken your commits and made the changes I wanted, as well as updated the tests, in #108. Could you take a look at that branch and let me know if you're still ok with how the code looks? If you're happy with things I'll merge that branch in and we should be good to go.

[campaul]

This was included as part of #108.