refactor: move IO responsibility out of Processor trait

[garyttierney]

De-duplicates the event reading/data management code contained within each processor and delegates it to the Backtester to handle.

Standalone version of the changes in #124, but without any of the parallel reader code.

[garyttierney]

Oops, few todo!()s left in there. Will fix those.

[nkaz001]

Can you write test cases? I will look into this separately. But since the code is growing and refactored widely, it becomes hard to check if works as expected (the backtesting result is the same as the previous).

[garyttierney]

Yes, I think we can start by testing that the ProcessorState is correctly doing what process_data() was doing previously. From there I think we can write some more tests now that it's a bit easier to setup the backtester with a given mock state.

[nkaz001]

It looks good. It can be merged after the tests have been completed.

[garyttierney]

@nkaz001 added a simple unit test that runs the actual Backtester and will create a couplee more. What do you think about this testing strategy, and is there anything specific you think we should check?

[nkaz001]

IMO, the original implementation needs refactoring to make it more unit testable at the function level. Currently, I am testing it by running the tutorials. Similarly, it would be better to run a testing strategy, which generates a high turnover, with test data and verifying results, such as the final P&L and turnover, for a holistic test.

[garyttierney]

IMO, the original implementation needs refactoring to make it more unit testable at the function level. Currently, I am testing it by running the tutorials.

I agree. I've started to move handling of event intents out of Backtester and into the processor state, but I'm not sure it's the correct way to go. For now it does allow easily checking that the correct flow is followed when end-of-data is reached, etc. I'm exploring a few avenues for breaking it up into easily testable units, but keen to here if you had any ideas in mind.