Skip to content

Commit

Permalink
Renamed QueueMgr class to Channel. Renamed WorkQueueBroker class to W…
Browse files Browse the repository at this point in the history
…orkQueueMgr.

Updated test and demo code to use the new class names. Hopefully, the new class names
make better sense. The README.md file has been split up into several
separate files to improve readability. No functionality has been changed
since the previous version.
  • Loading branch information
cwjohan committed Nov 5, 2014
1 parent eb13450 commit 144bf5b
Show file tree
Hide file tree
Showing 8 changed files with 963 additions and 0 deletions.
61 changes: 61 additions & 0 deletions CHANGE_LOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
##Change Log

**v0.0.0**: Initial version.

**v0.0.1-3**: Changes to README.md. Implementation of jasmine-node tests.

**v0.1.2**: Refactored to implement new QueueMgr and WorkQueueBroker interfaces; Implementation of connection strategies.

**v0.1.3**: Further Refactoring to implement new QueueMgr and WorkQueueBroker interfaces;
Merged v0.1.3 from flex branch into master.

**v0.1.4**: Fix for issue #1 - Where to find redis-queue-config file is too restrictive - Now uses
QUEUE_CONFIG_FILE environment variable to find the config file;
An alternative is to specify the config file path in the queueMgr or workQueueBroker constructor.
Changed testing from Jasmine to Mocha.; Implemented Mocha tests;
Introduced interleaved tests.

**v0.1.5**: Various comment and README.md changes;
Corrected error in provision of Redis Cloud hostname;

**v0.1.6**: Compiled to capture latest mod to .coffee source.

**v0.1.7**: Fix for issue #4 - Stall occurs when one of two work queues on same connection becomes empty.

**v0.1.8**: Fix for REDISCLOUD_URL having no auth field; Created change log in README.md.

**v0.1.9**: Added usage examples to README.md for WorkQueueBroker. Added commandQueueLength function
to permit some rudimentary control of backpressure. Documented 'drain' event.

**v0.1.10**: Changed `grunt test` to use mocha rather than jasmine-node. Improved usage documentation.

**v0.1.11**: Added shutdownSoon function to QueueMgr and WorkQueueBroker. Improved README.md and demos. Made test suite
use unique queue names to prevent interference from demos.

**v0.1.12**: Modified WorkQueueMgr to preserve the order of
queue names used when calling popAny. ECMA-262 5.1 (15.2.3.14 Object.keys and 12.6.4 The for-in Statement) does not
specify enumeration order, so an array should be used. Also, see: https://code.google.com/p/v8/issues/detail?id=164

**v0.1.13**: Modified connStrategyBlueMixRedisCloud to use a configured redis version. Added config info to README.md.

**v0.1.14**: Added 'clean' task to Gruntfile. Fixed some potential problems found by jshint. Tidied Gruntfile.
Replaced some URLs in the demo source that were no longer working (404 not found).

**v0.1.15**: Send now checks that queue has not been destroyed.
Added 'compile-test' task to Gruntfile. Fixed
incorrect calls to isValidQueueName. Added tests for WorkQueue
exceptions. Grunt now uses grunt-mocha-test plugin for better
reporting.

**v0.1.16**: Reverted WorkQueue behaviour back to previous version since v0.1.15 change was too restrictive.
Added destroy function to WorkQueue. Updated README.md with info about the new destroy function. Also, added
some architecture notes.

**v0.1.17**: Added ability to schedule parallel jobs in the consume function via an optional arity parameter.
Added @outstanding to queueMgr class. worker04 example uses a second WorkQueueBroker instance when arity is
greater than 1 to send result back to provider04.

**v0.2.0**: Renamed QueueMgr class to Channel. Renamed WorkQueueBroker class to WorkQueueMgr. Updated test
and demo code to use the new class names, which have been adopted to correspond better to what they represent.
The README.md file has been split up into several separate files to improve readability.

159 changes: 159 additions & 0 deletions COFFEESCRIPT_USAGE_EXAMPLES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
###Channel Coffescript Usage Example

1. Ensure you have a Redis server installed and running. For example, once installed, you can run it locally by

redis-server &

1. Require `node-redis-queue` Channel

Channel = require('node-redis-queue').Channel

1. Create a Channel instance and connect to Redis

channel = new Channel()
channel.connect ->
console.log 'ready'
myMainLogic()

Alternatively, you can provide an existing Redis connection (i.e., a redis client instance)

channel.attach redisConn

1. Optionally, handle error events

channel.on 'error', (error) ->
console.log 'Stopping due to: ' + error
process.exit()

1. Optionally, handle lost connection events

channel.on 'end', ->
console.log 'Connection lost'

1. Optionally, clear previous data from the queue, providing a callback
to handle the data.

channel.clear queueName, ->
console.log 'cleared'
doImportantStuff()

1. Optionally, push data to your queue

channel.push queueName, myData

1. Optionally, pop data off your queue

channel.pop queueName, (myData) ->
console.log 'data = ' + myData

or, alternatively, pop off any of multiple queues

channel.popAny queueName1, queueName2, (myData) ->
console.log 'data = ' + myData

Once popping data from a queue, avoid pushing data to the same queue from the same connection, since
a hang could result. This appears to be a Redis limitation when using blocking reads.

1. When done, quit the Channel instance

channel.disconnect()

or, alternatively, if consuming data from the queue, end the connection

channel.end()

or, if there may be a number of redis commands queued,

channel.shutdownSoon()

###WorkQueueMgr Coffeescript Usage Example

1. Ensure you have a Redis server installed and running. For example, once installed, you can run it locally by

redis-server &

1. Require `node-redis-queue` WorkQueueMgr

WorkQueueMgr = require('node-redis-queue').WorkQueueMgr

1. Create a WorkQueueMgr instance and connect to Redis

mgr = new WorkQueueMgr()
mgr.connect ->
console.log 'ready'
myMainLogic()

Alternatively, you can provide an existing Redis connection (i.e., a redis client instance)

mgr.attach redisConn

1. Optionally, handle error events

mgr.on 'error', (error) ->
console.log 'Stopping due to: ' + error
process.exit()

1. Optionally, handle lost connection events

mgr.on 'end', ->
console.log 'Connection lost'

1. Create a work queue instance

queue = mgr.createQueue queueName

1. Optionally, clear previous data from the queue, providing a callback
to handle the data.

queue.clear ->
console.log 'cleared'
doImportantStuff()

1. Optionally, send data to your queue

queue.send myData

1. Optionally, consume data from your queue and call ack when ready to consume another data item

queue.consume (myData, ack) ->
console.log 'data = ' + myData
...
ack()

or, alternatively,

queue.consume (myData, ack) ->
console.log 'data = ' + myData
...
ack()
, arity

where arity is an integer indicating the number of async callbacks to schedule in parallel. See demo 04 for example usage.

If multiple queues are being consumed, they are consumed with highest priority given to the queues consumed first
(i.e., in the order in which the consume statements are executed).

Note that ack(true) may be used to indicate that no further data is expected from the given work queue.
This is useful, for example, in testing, when a clean exit from a test case is desired.

Once consuming from a queue, avoid sending data to the same queue from the same connection
(i.e., the same mgr instance), since a hang could result. This appears to be a Redis limitation when using
blocking reads. You can test `mgr.channel.outstanding` for zero to determine if it is OK to send on the same connection.

1. Optionally, destroy a work queue if it no longer is needed. Assign null to the queue variable to free up memory.

queue.destroy()
queue = null

1. When done, quit the WorkQueueMgr instance

mgr.disconnect()

or, alternatively, if consuming data from the queue, end the connection

mgr.end()

or, if there may be a number of redis commands queued,

mgr.shutdownSoon()

30 changes: 30 additions & 0 deletions DEVELOPER_INFO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
##Running the test suite

Use either `grunt test` or `npm test` to run the suite of tests using Mocha. The test cases reside in the `test` directory.

##Running grunt for development tasks

`grunt` runs coffeelint and then coffee.

`grunt coffeelint` runs coffeelint on all the .coffee source code in the src directory.

`grunt coffee` runs coffee on all the .coffee source code in the src directory, converting it to .js code in the
corresponding lib directory.

`grunt jshint` runs jshint on all the .js code except one in the demo/lib/helpers directory. Note that jshint may
have a lot of complaints about the generated .js code, but is useful to spot potential problems.

`grunt clean` runs a script that removes vim backup files (i.e., files ending with '~' and .js files in the test directory).

`grunt test` runs the suite of tests using Mocha. It looks for .coffee files in the test directory.

`grunt bump` bumps the patch number up in the package.json file.

`grunt git-tag` commits the latest staged code changes and tags the code with the version obtained from the package.json file.

`grunt release` runs coffee on all the .coffee source code in the src directory, converting it to .js code, and
then runs the git-tag task to commit the latest staged code changes and tag the code with the version obtained from the
package.json file.

`grunt compile-test` runs coffee on the test .coffee code. This is only for debugging of test cases when you need to see the generated javascript code.

Loading

0 comments on commit 144bf5b

Please sign in to comment.