-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Renamed QueueMgr class to Channel. Renamed WorkQueueBroker class to W…
…orkQueueMgr. Updated test and demo code to use the new class names. Hopefully, the new class names make better sense. The README.md file has been split up into several separate files to improve readability. No functionality has been changed since the previous version.
- Loading branch information
Showing
8 changed files
with
963 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
##Change Log | ||
|
||
**v0.0.0**: Initial version. | ||
|
||
**v0.0.1-3**: Changes to README.md. Implementation of jasmine-node tests. | ||
|
||
**v0.1.2**: Refactored to implement new QueueMgr and WorkQueueBroker interfaces; Implementation of connection strategies. | ||
|
||
**v0.1.3**: Further Refactoring to implement new QueueMgr and WorkQueueBroker interfaces; | ||
Merged v0.1.3 from flex branch into master. | ||
|
||
**v0.1.4**: Fix for issue #1 - Where to find redis-queue-config file is too restrictive - Now uses | ||
QUEUE_CONFIG_FILE environment variable to find the config file; | ||
An alternative is to specify the config file path in the queueMgr or workQueueBroker constructor. | ||
Changed testing from Jasmine to Mocha.; Implemented Mocha tests; | ||
Introduced interleaved tests. | ||
|
||
**v0.1.5**: Various comment and README.md changes; | ||
Corrected error in provision of Redis Cloud hostname; | ||
|
||
**v0.1.6**: Compiled to capture latest mod to .coffee source. | ||
|
||
**v0.1.7**: Fix for issue #4 - Stall occurs when one of two work queues on same connection becomes empty. | ||
|
||
**v0.1.8**: Fix for REDISCLOUD_URL having no auth field; Created change log in README.md. | ||
|
||
**v0.1.9**: Added usage examples to README.md for WorkQueueBroker. Added commandQueueLength function | ||
to permit some rudimentary control of backpressure. Documented 'drain' event. | ||
|
||
**v0.1.10**: Changed `grunt test` to use mocha rather than jasmine-node. Improved usage documentation. | ||
|
||
**v0.1.11**: Added shutdownSoon function to QueueMgr and WorkQueueBroker. Improved README.md and demos. Made test suite | ||
use unique queue names to prevent interference from demos. | ||
|
||
**v0.1.12**: Modified WorkQueueMgr to preserve the order of | ||
queue names used when calling popAny. ECMA-262 5.1 (15.2.3.14 Object.keys and 12.6.4 The for-in Statement) does not | ||
specify enumeration order, so an array should be used. Also, see: https://code.google.com/p/v8/issues/detail?id=164 | ||
|
||
**v0.1.13**: Modified connStrategyBlueMixRedisCloud to use a configured redis version. Added config info to README.md. | ||
|
||
**v0.1.14**: Added 'clean' task to Gruntfile. Fixed some potential problems found by jshint. Tidied Gruntfile. | ||
Replaced some URLs in the demo source that were no longer working (404 not found). | ||
|
||
**v0.1.15**: Send now checks that queue has not been destroyed. | ||
Added 'compile-test' task to Gruntfile. Fixed | ||
incorrect calls to isValidQueueName. Added tests for WorkQueue | ||
exceptions. Grunt now uses grunt-mocha-test plugin for better | ||
reporting. | ||
|
||
**v0.1.16**: Reverted WorkQueue behaviour back to previous version since v0.1.15 change was too restrictive. | ||
Added destroy function to WorkQueue. Updated README.md with info about the new destroy function. Also, added | ||
some architecture notes. | ||
|
||
**v0.1.17**: Added ability to schedule parallel jobs in the consume function via an optional arity parameter. | ||
Added @outstanding to queueMgr class. worker04 example uses a second WorkQueueBroker instance when arity is | ||
greater than 1 to send result back to provider04. | ||
|
||
**v0.2.0**: Renamed QueueMgr class to Channel. Renamed WorkQueueBroker class to WorkQueueMgr. Updated test | ||
and demo code to use the new class names, which have been adopted to correspond better to what they represent. | ||
The README.md file has been split up into several separate files to improve readability. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
###Channel Coffescript Usage Example | ||
|
||
1. Ensure you have a Redis server installed and running. For example, once installed, you can run it locally by | ||
|
||
redis-server & | ||
|
||
1. Require `node-redis-queue` Channel | ||
|
||
Channel = require('node-redis-queue').Channel | ||
|
||
1. Create a Channel instance and connect to Redis | ||
|
||
channel = new Channel() | ||
channel.connect -> | ||
console.log 'ready' | ||
myMainLogic() | ||
|
||
Alternatively, you can provide an existing Redis connection (i.e., a redis client instance) | ||
|
||
channel.attach redisConn | ||
|
||
1. Optionally, handle error events | ||
|
||
channel.on 'error', (error) -> | ||
console.log 'Stopping due to: ' + error | ||
process.exit() | ||
|
||
1. Optionally, handle lost connection events | ||
|
||
channel.on 'end', -> | ||
console.log 'Connection lost' | ||
|
||
1. Optionally, clear previous data from the queue, providing a callback | ||
to handle the data. | ||
|
||
channel.clear queueName, -> | ||
console.log 'cleared' | ||
doImportantStuff() | ||
|
||
1. Optionally, push data to your queue | ||
|
||
channel.push queueName, myData | ||
|
||
1. Optionally, pop data off your queue | ||
|
||
channel.pop queueName, (myData) -> | ||
console.log 'data = ' + myData | ||
|
||
or, alternatively, pop off any of multiple queues | ||
|
||
channel.popAny queueName1, queueName2, (myData) -> | ||
console.log 'data = ' + myData | ||
|
||
Once popping data from a queue, avoid pushing data to the same queue from the same connection, since | ||
a hang could result. This appears to be a Redis limitation when using blocking reads. | ||
|
||
1. When done, quit the Channel instance | ||
|
||
channel.disconnect() | ||
|
||
or, alternatively, if consuming data from the queue, end the connection | ||
|
||
channel.end() | ||
|
||
or, if there may be a number of redis commands queued, | ||
|
||
channel.shutdownSoon() | ||
|
||
###WorkQueueMgr Coffeescript Usage Example | ||
|
||
1. Ensure you have a Redis server installed and running. For example, once installed, you can run it locally by | ||
|
||
redis-server & | ||
|
||
1. Require `node-redis-queue` WorkQueueMgr | ||
|
||
WorkQueueMgr = require('node-redis-queue').WorkQueueMgr | ||
|
||
1. Create a WorkQueueMgr instance and connect to Redis | ||
|
||
mgr = new WorkQueueMgr() | ||
mgr.connect -> | ||
console.log 'ready' | ||
myMainLogic() | ||
|
||
Alternatively, you can provide an existing Redis connection (i.e., a redis client instance) | ||
|
||
mgr.attach redisConn | ||
|
||
1. Optionally, handle error events | ||
|
||
mgr.on 'error', (error) -> | ||
console.log 'Stopping due to: ' + error | ||
process.exit() | ||
|
||
1. Optionally, handle lost connection events | ||
|
||
mgr.on 'end', -> | ||
console.log 'Connection lost' | ||
|
||
1. Create a work queue instance | ||
|
||
queue = mgr.createQueue queueName | ||
|
||
1. Optionally, clear previous data from the queue, providing a callback | ||
to handle the data. | ||
|
||
queue.clear -> | ||
console.log 'cleared' | ||
doImportantStuff() | ||
|
||
1. Optionally, send data to your queue | ||
|
||
queue.send myData | ||
|
||
1. Optionally, consume data from your queue and call ack when ready to consume another data item | ||
|
||
queue.consume (myData, ack) -> | ||
console.log 'data = ' + myData | ||
... | ||
ack() | ||
|
||
or, alternatively, | ||
|
||
queue.consume (myData, ack) -> | ||
console.log 'data = ' + myData | ||
... | ||
ack() | ||
, arity | ||
|
||
where arity is an integer indicating the number of async callbacks to schedule in parallel. See demo 04 for example usage. | ||
|
||
If multiple queues are being consumed, they are consumed with highest priority given to the queues consumed first | ||
(i.e., in the order in which the consume statements are executed). | ||
|
||
Note that ack(true) may be used to indicate that no further data is expected from the given work queue. | ||
This is useful, for example, in testing, when a clean exit from a test case is desired. | ||
|
||
Once consuming from a queue, avoid sending data to the same queue from the same connection | ||
(i.e., the same mgr instance), since a hang could result. This appears to be a Redis limitation when using | ||
blocking reads. You can test `mgr.channel.outstanding` for zero to determine if it is OK to send on the same connection. | ||
|
||
1. Optionally, destroy a work queue if it no longer is needed. Assign null to the queue variable to free up memory. | ||
|
||
queue.destroy() | ||
queue = null | ||
|
||
1. When done, quit the WorkQueueMgr instance | ||
|
||
mgr.disconnect() | ||
|
||
or, alternatively, if consuming data from the queue, end the connection | ||
|
||
mgr.end() | ||
|
||
or, if there may be a number of redis commands queued, | ||
|
||
mgr.shutdownSoon() | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
##Running the test suite | ||
|
||
Use either `grunt test` or `npm test` to run the suite of tests using Mocha. The test cases reside in the `test` directory. | ||
|
||
##Running grunt for development tasks | ||
|
||
`grunt` runs coffeelint and then coffee. | ||
|
||
`grunt coffeelint` runs coffeelint on all the .coffee source code in the src directory. | ||
|
||
`grunt coffee` runs coffee on all the .coffee source code in the src directory, converting it to .js code in the | ||
corresponding lib directory. | ||
|
||
`grunt jshint` runs jshint on all the .js code except one in the demo/lib/helpers directory. Note that jshint may | ||
have a lot of complaints about the generated .js code, but is useful to spot potential problems. | ||
|
||
`grunt clean` runs a script that removes vim backup files (i.e., files ending with '~' and .js files in the test directory). | ||
|
||
`grunt test` runs the suite of tests using Mocha. It looks for .coffee files in the test directory. | ||
|
||
`grunt bump` bumps the patch number up in the package.json file. | ||
|
||
`grunt git-tag` commits the latest staged code changes and tags the code with the version obtained from the package.json file. | ||
|
||
`grunt release` runs coffee on all the .coffee source code in the src directory, converting it to .js code, and | ||
then runs the git-tag task to commit the latest staged code changes and tag the code with the version obtained from the | ||
package.json file. | ||
|
||
`grunt compile-test` runs coffee on the test .coffee code. This is only for debugging of test cases when you need to see the generated javascript code. | ||
|
Oops, something went wrong.