service vm needs better support for handling commit conflicts

[dalehenrich]

https://code.google.com/p/glassdb/issues/detail?id=326

[dalehenrich]

@jbrichau, was this covered by https://code.google.com/p/glassdb/issues/detail?id=355

[jbrichau]

@dalehenrich I think that issue was more about 'what if commit conflicts happen, how can we handle them inside the service vm'. I think you created the issue because I said it was quite hard to discover what was going on inside the service vm. I'm not sure what can be done about that and if it is even necessary?

[dalehenrich]

Okay that sounds fine ... so better support is stiil needed ... BTW, you
are using service vms, right? could you share some information about what
you are doing?

Obviously I'm interested in blowing the dust of this (and other projects)
but I don't want to break what you are doing ... are you using Nick's
futures work or are you using the basic service ...

There are several paths ... one is to just get everything pulled together
in one place with proper updated scripts (and an install script) ... a nice
working example using Zinc and the tODE scripts needed to setup/configurat
the vm, launch service vm, launch zinc and run the example .... in a nice
neat package so if you have fixes etc. that are easily sharable (or if you
just threaten to share, I will wait for you since I don't want to duplicate
effort or head off in wildly different directions ...)

the other is to take the futures work from Nick and bring it into the mix

the third is to take some of the Promises done in squeak by Frank Shearar
and hook it up ...

At this moment I am not an expert on Futures/Promises ... I'm just a
plumber hooking things up and patching the walls, but we could push into
new ground if that is called fo...

Dale

On Fri, Jun 6, 2014 at 11:32 AM, Johan Brichau [email protected]
wrote:

@dalehenrich https://github.com/dalehenrich I think that issue was more
about 'what if commit conflicts happen, how can we handle them inside the
service vm'. I think you created the issue because I said it was quite hard
to discover what was going on inside the service vm. I'm not sure what can
be done about that and if it is even necessary?

—
Reply to this email directly or view it on GitHub
https://github.com/glassdb/ServiceVM/issues/3#issuecomment-45369871.

[jbrichau]

We're using the basic service (i.e. the packages included with Seaside in GemStone) but have written an integration with our DALi framework that pretty much cross-cuts the code. Never used the Futures work either.

Our use is straightforward: in a (Seaside) request, we fork off a task to execute, which we store in the database and can poll them regularly when/if they are finished. Something we found useful is a separation between the code that should run inside a transaction and the code that should run outside of the transaction. After fixing the bug with doTransaction, the service vm basically executes the tasks in sequence (because they each should run in their own transaction). We improved that by splitting a task into two blocks to execute. The first one executes outside of the transaction and is intended for code that does not have anything to commit (so when other tasks commit their transaction, it's never made persistent). The results of the block is then passed to the second block, which runs inside a transaction and does the persistent work.

We do have a bugfix for the serviceVMTaskServiceExample which we should share. I should dig up what it exactly fixes but I think the servicevm got stuck when more than 100 tasks were queued.v It was not published because it dated before we fixed the doTransaction bug and we originally thought that was the bug that blocked our vms. Since we continued seeing it, we thought we did not fix anything, and eventually hit the fix for doTransaction. But I'll dig up that bug in this context and get the fix merged in here, for sure.

[dalehenrich]

Thanks very much ... this is very helpful and I want to make sure that I don't collide interfere with your work ... I like the two-block idea separation is real good and not having to explicitly write transaction code is a very good thing ...

as with all of my projects I'm sure the work will go slowly, but I've always thought that the server vm was a valuable thing to have and I've always thought that the server vm did receive enough love from the documentation aspect and that I will clean up in the short term ...