pykit: A collection of python libs those are used in project s2: Storage-Service at baishancloud.com
This library is in beta phase.
It has been used heavily in our object storage service, as a foundamental library of our devops platform.
There is a README.md for each module.
| name | description |
|---|---|
| argchecker | Validates arguments against the schema |
| aws | AWS error codes and so on |
| awssign | Sign a request by using aws signature version 4 |
| cacheable | Cache data which access frequently |
| cachepool | Reusable object cache in process |
| cgrouparch | Build cgroup directory tree, add set cgroup pid |
| cluster | Some helper function for the server in a cluster |
| csvutil | Utility functions for CSV file loading and conversion |
| daemonize | Start, stop or restart a daemon process |
| dictutil | Dictionary helper utility |
| ectypes | Utility functions for the server |
| etcd | etcd client |
| fsutil | File-system Utilities |
| heap | Min heap |
| http | HTTP/1.1 client |
| httpmultipart | Utility functions to make headers or make body reader about multipart |
| humannum | Convert number to human readable number string |
| jobq | Process serial of input elements with several functions concurrently and sequentially |
| jobscheduler | Run jobs at certain time |
| logcollector | Collect logs of interest locally |
| logutil | Utility functions to create logger or make log message |
| mime | Utility functions to handle mime type |
| modutil | Submodule Utilities |
| mysqlconnpool | Mysql connection pool with MySQLdb in python |
| mysqlutil | Mysql related datatype, operations |
| net | Network utility |
| portlock | cross process lock |
| priorityqueue | Priority queue |
| proc | Utility to create sub process |
| rangeset | Segmented range. |
| ratelimiter | rate limiter |
| redisutil | For using redis more easily. |
| shell | Set different command arguments to execute different functions |
| strutil | A collection of helper functions used to manipulate string |
| threadutil | Utility functions for better management of threads |
| timeutil | Support specify time format output and get current ts, ms, us api etc |
| utfjson | Force json.dump and json.load in utf-8 encoding |
| utfyaml | Force yaml.dump and yaml.load in utf-8 encoding |
| wsjobd | Job daemon based on websocket protocol |
| zktx | Transaction implementation on Zookeeper |
| zkutil | Utility functions for zookeeper |
Just clone it and copy it into your project source folder.
cd your_project_folder
git clone https://github.com/baishancloud/pykit.git
from pykit import jobq
def add1(args):
return args + 1
def printarg(args):
print args
jobq.run([0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3
See config.md
Run one of following to test all, a module, a TestCase or a function.
./script/t
./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id
See Details
There are several scripts for developers. See script.
Zhang Yanpo (张炎泼) [email protected]
The MIT License (MIT)
Copyright (c) 2015 Zhang Yanpo (张炎泼) [email protected]