Selwin Ong
8bfbeac67d
Running rqworker without specifiying --exception-handler shouldn't override default behavior.
10 years ago
mozillazg
7198b4568a
Fixed a format error
10 years ago
Vincent Driessen
0de225ec62
Emphasize private nature of `stopped` property
10 years ago
Vincent Driessen
ec0e04727b
Rename `maintenance_date` -> `last_cleaned_at`
10 years ago
Vincent Driessen
891be55b87
Clean up Worker.__init__ logic a little
10 years ago
Vincent Driessen
4d0ae5da98
Clean up type checking of Worker constructor
10 years ago
Vincent Driessen
94258761ae
Make string formatting consistent
10 years ago
Selwin Ong
8f9c507f12
Bump version to 0.5.3.
10 years ago
Marcus Martins
513f6310d2
Change default TTL to None insted of -1
10 years ago
Marcus Martins
5b8726ad2d
Fixes #502
...
Fixes some broken tests and misbehaviour with ttls. There was a temporal
coupling between saving the job and setting its expires parameter.
10 years ago
Selwin Ong
f370f79819
Merge pull request #534 from selwin/registry-maintenance
...
Worker automatically cleans job registries every hour
10 years ago
Neal Todd
bac6699ea4
Allow non-ASCII characters in keyword arguments.
10 years ago
Selwin Ong
c3767e28e2
Worker now runs maintenance tasks every hour and on startup.
10 years ago
Selwin Ong
5782ac10c4
Added worker.clean_registries().
10 years ago
Selwin Ong
faf9d3e668
Added clean_registries(queue) function to clean job registries related to that queue.
10 years ago
RyanMTB
9a00b0eca6
Updated Worker API
10 years ago
Selwin Ong
d51f0200d7
Merge pull request #513 from trevorprater/master
...
Improving logging.
10 years ago
Selwin Ong
70d5f971bd
Jobs from FinishedJobRegistry that are moved to FailedQueue should have "failed" as status.
10 years ago
Cosmin Stefan
260f7caf66
Enable proper setup of signale handlers for SimpleWorker as well. fixes #523
...
This allows a SIGTERM to make the worker perform a warm shutdown and cleanly break out of the loop and finish the current job, if any.
10 years ago
Vincent Driessen
8f7322ed10
This is 0.5.2
10 years ago
Trevor Prater
5674edad61
Changed rqworker shutdown message.
...
- As requested by @nvie and @selwin.
10 years ago
Trevor Prater
fd1dca40b9
Improving logging.
...
- Include worker key in worker startup log statement.
- Added a notification to make it more clear when a 'burst' worker dies.
10 years ago
Selwin Ong
cff6fbff41
Merge pull request #477 from glaslos/ttl_tests_fixes
...
TTL tests and fixes
10 years ago
glaslos
071c47dacb
hard-coded TTL to -1 on job.perform()
...
Removed merge artifacts
10 years ago
Serhii Maltsev
91f263d8e0
change try/except in python2/3 compatibility to to_text()
10 years ago
Tamir Duberstein
985a2664a4
Prevent `Queue#dequeue` from blowing the stack
...
In the case of many sequential jobs having been deleted, a recursive
implementation of `Queue#dequeue` is prone to blowing the stack in the
absence of tail-recursion support. Change the implementation from
recursive to iterative to work around this issue in CPython.
10 years ago
Vincent Driessen
5cb873b438
Fix PEP8 complaint.
10 years ago
Vincent Driessen
cd155299a4
Fix PEP8 complaint.
10 years ago
Vincent Driessen
6c6e53542f
Don't require redis-py 2.10 just for the SSL option.
10 years ago
Vincent Driessen
8505198298
Bump to 0.5.1.
10 years ago
Vincent Driessen
5990fa9d73
Merge pull request #499 from vladkens/issue/498
...
ZADD bugfix
10 years ago
Eric Bustarret
9fba806662
Add REDIS_SSL option
...
Allow the worker to connect to a Redis instance through SSL (ex: Azure
Redis Cache use SSL only by default)
10 years ago
Vlad Pronsky
933e6697cd
Fixed redis drivers bug
10 years ago
glaslos
3d4d6a86d5
persist the job right before execution
10 years ago
glaslos
24d5e08992
Merge branch 'master' into ttl_tests_fixes
...
Conflicts:
tests/test_job.py
10 years ago
Selwin Ong
30b21ef06e
Merge pull request #465 from marklap/workerbirthdeath
...
add birth_date and death_date properties to Worker
10 years ago
Mark LaPerriere
636a537fa7
updates addressing @selwin comments for PR #465
10 years ago
Taras Semenenko
3dfd044767
Add missed `multi` command after `watch`
...
`watch` command should be used in conjunction with `multi` command
which was missed in enqueuing of job with dependencies.
Fix #487
10 years ago
Bradley Young
04e403e1a3
dread comma, I adjure thee.
10 years ago
Bradley Young
d56b5424c8
merging master and fixing conflicts
10 years ago
Bradley Young
c428f955e4
per @selwin, adding a deprecated handler for exc_handler
10 years ago
alternativshik
219f21b637
Allow non-ASCII characters in arguments
...
get_call_string() failed if any arguments contained non-ASCII strings.
Fixes #406
10 years ago
Selwin Ong
1005222371
Restored the ability to specify connection params in config
10 years ago
glaslos
c155918d6d
lrem only first occurrence of the job id (we expect only one)
10 years ago
Mark LaPerriere
a4ca4704ac
convert redis returned bytes into text/string
10 years ago
Mark LaPerriere
cfe3da287a
updated branch to latest
10 years ago
Vincent Driessen
b8d425b318
Various PEP8 fixes.
10 years ago
Vincent Driessen
fac2b10309
Auto-sort imports using isort.
10 years ago
Vincent Driessen
e8c3b96a5a
Bump to 0.5.0.
10 years ago
glaslos
703ab0e355
removed merged comment
10 years ago
glaslos
646f1cda22
Merge branch 'master' into ttl_tests_fixes
...
Conflicts:
rq/queue.py
10 years ago
glaslos
5b5ab4860f
passing on the ttl
10 years ago
Bradley Young
f05d77b92b
changing "default" for exception_handlers to None; same exact logic can be implemented, but cleaner
10 years ago
Bradley Young
d3e4fb567b
merge && updates to add multiple exception handlers.
10 years ago
Bradley Young
f575137612
updating worker to have exception_handlers
10 years ago
Selwin Ong
dac0be6cc7
Added DeferredJobsRegistry to keep track of deferred jobs.
10 years ago
Selwin Ong
3e674fbe6a
queue.enqueue() should set job.origin.
10 years ago
Selwin Ong
9320496402
Simplify FailedQueue.quarantine and ensure that a deferred job's status is set to Queued when enqueued.
10 years ago
Selwin Ong
7fd2ac8ca6
Added "DEFERRED" Job status for jobs that have unsatisfied dependencies.
10 years ago
Vincent Driessen
dc5cd514ee
Explicitly cast map() result to a list, for Python 3 compat.
10 years ago
Selwin Ong
dc09676ee0
Minor copywriting fixes.
10 years ago
Selwin Ong
01ab2f20dd
Minor copywriting fixes.
10 years ago
Vincent Driessen
543bcbc60b
Explicitly cast map() result to a list, for Python 3 compat.
10 years ago
Selwin Ong
eae2c40ea0
Merge pull request #457 from jlopex/block_until_horse_finishes
...
Make parent worker process wait until horse process finishes
10 years ago
Selwin Ong
0a0a81da6d
Merge pull request #463 from conslo/skip-queue
...
Skip queue
10 years ago
Mark LaPerriere
e6c32edad9
add birth_date and death_date properties to Worker
10 years ago
Jonathan Tushman
1ae5a12a81
merging master
10 years ago
Jonathan Tushman
ccd41396f6
adding suspend with duration test
10 years ago
Travis Johnson
add4d625b6
Merge branch 'nvie' into skip-queue
...
Conflicts:
rq/queue.py
10 years ago
Vincent Driessen
cd0c3c9c1e
Merge pull request #462 from conslo/worker-hash
...
Worker hash
10 years ago
Jonathan Tushman
82333d2ad5
triggering shutdown by setting a redis flag
10 years ago
Travis Johnson
db75958ad2
use 'at_front' instead of 'skip_queue'
10 years ago
Selwin Ong
e1801c5c44
Merge pull request #453 from glaslos/job_ttl
...
TTL for jobs
10 years ago
Travis Johnson
ac61f502a1
skip_queue functionality
10 years ago
Travis Johnson
f60e4884df
a comment typo
10 years ago
Travis Johnson
0ad4cb3410
comments
10 years ago
Travis Johnson
c301369c2e
implement __eq__ and __hash__ for workers
10 years ago
Nic Cope
09cab7a90d
Allow maximum job age to be specified when cleaning up or getting expired jobs from a registry.
...
This ensures that all jobs cleaned from a started registry end up in the failed queue.
10 years ago
Javier Lopez
fc5295edfc
Set busy state on parent while waiting for horse
10 years ago
Nic Cope
6ab7070a92
Rename registry timeout to registry ttl
...
This matches job.cleanup() better and (I think) makes the purpose slightly clearer.
10 years ago
Nic Cope
9f699d758d
Any negative number should cause a job to live forever in a registry.
...
While the documentation explicitly mentions ttl=-1, this matches better the behaviour of the job.cleanup() method.
10 years ago
glaslos
c6a83eaa82
calling self.cleanup to set job expiration
10 years ago
glaslos
8a3fd91e2e
renaming job.job_ttl to job.ttl
10 years ago
Selwin Ong
2091c601b1
Merge pull request #451 from conslo/depend-on-unsaved-job-ids
...
Depend on unsaved job ids
10 years ago
Nic Cope
8fa184b86b
Ensure the FinishedJobRegistry honors an 'infinite' result_ttl of -1
10 years ago
glaslos
55c541bc59
added job ttl to queue.enqueue()
10 years ago
Travis Johnson
af4b0436c1
instantiate a job instead of fetch
...
this removes the need for the depended on job.id already being saved
10 years ago
Travis Johnson
14d1186241
use internal job_class for check
10 years ago
Bradley Young
5caccaabfe
Adding optional list handling to the exc_handler option in Worker.
...
Adding command line --exception_handler option (with multiple entries allowed) to `rq worker`
Added tests for command line options.
10 years ago
Travis Johnson
3467868f1f
allow depends_on to be a job id or a job itself
...
Other parts of the code (i.e.: the `.create()` method) allow the `depends_on` kwarg to be a `Job` object *or* a job id. This is an attempt to allow that same idea within the `.enqueue_call()` method for a queue. Since this part of the code requires actually knowing the precise redis key for the job that's depended on, my intuition is that a `.fetch()` is the only way to solve this.
10 years ago
Selwin Ong
0dbe68527c
Merge pull request #431 from negz/worker-redis-bug
...
Pass the default worker connection into the started and finished registries
10 years ago
Adam Wen
f5001a0e36
fix decode error in py3.
10 years ago
Adam Wen
1795e1ee53
fix decode
10 years ago
Adam Wen
cd0230cae3
make job description unicode friendly
10 years ago
Nic Cope
de0b501823
Pass the default worker connection into the started and finished registries
...
In practice all the registry methods are called with pipelines, but if no
connection is passed in when instantiating the registry and the connection
stack is empty resolve_connection will raise NoRedisConnectionException.
10 years ago
zhangliyong
6621105bde
Use type instead of default to specify interval option type
10 years ago
zhangliyong
3d49784bcc
Merge branch 'master' of github.com:nvie/rq into cli-rq-worker
10 years ago
Selwin Ong
2307bc8253
Added FinishedJobRegistry to monitor finished jobs.
10 years ago
Selwin Ong
74a9982ecb
Fixed a bug where expired jobs aren't deleted from JobStartedRegistry.
10 years ago
Selwin Ong
18c6ea64ef
Merge pull request #419 from selwin/registry-len
...
Added JobStartedRegistry.count
10 years ago
zhangliyong
f9511664c8
Make flake8 happy :)
10 years ago
zhangliyong
1b39996bf7
Make pep8
10 years ago
zhangliyong
c6033b312e
Cleanup
10 years ago
zhangliyong
6510bb46f3
Fix interval type bug, it should be float
10 years ago
zhangliyong
21c14c0e21
Cleanup
10 years ago
zhangliyong
4d91f136f8
Move redis url default value to connect function
...
So worker subcommand can retrieve redis url from config file if url
option value is None, otherwise url option value will never be None.
10 years ago
zhangliyong
4df076f3aa
Retrieve url from config file
10 years ago
zhangliyong
229a00fee5
Cleanup
10 years ago
zhangliyong
88fe5124d8
Make pythonic
10 years ago
zhangliyong
907e71dc86
Make pythonic
10 years ago
zhangliyong
f10c749b7c
Use click's evvar to retrieve SENTRY_DSN environment value
10 years ago
zhangliyong
842f27294a
Convert rqworker to 'rq worker' subcommand
10 years ago
zhangliyong
7b434a32eb
Let cleanup_ghosts accept conn argument
10 years ago
Selwin Ong
202be75b21
Merge pull request #415 from foxx/feature/custom-job-ids
...
Allow job ID to be set on enqueue/enqueue_call() - fixes #412
10 years ago
Selwin Ong
c1dc30eae3
Added __len__ method to Queue.
10 years ago
Selwin Ong
fae7df5aa7
Renamed StartedJobRegistry.get_job_count to StartedJobRegistry.count for consistency.
10 years ago
Selwin Ong
41ae1ce8a7
Added a registry.get_job_count().
10 years ago
foxx
30ea76ddd6
Fixed silly typo
10 years ago
foxx
0466562a13
Fixes from comments in #415
...
Added tox/vagrant tmp dir to gitignore
Removed unused import
Renamed id to job_id in enqueue_call() as per comments in #415
10 years ago
Vincent Driessen
783b6f9ec7
Fix PEP8 complaint.
10 years ago
Vincent Driessen
7686785d61
Merge branch 'selwin-working-queue'
...
Conflicts:
tests/test_worker.py
10 years ago
Vincent Driessen
4b7c59d68d
Fix PEP8 complaints.
10 years ago
Selwin Ong
6d79082b62
Call move_expired_jobs_to_failed_queue before returning job_ids.
10 years ago
Selwin Ong
9341a4a33d
Renamed WorkingQueue to StartedJobRegistry.
10 years ago
Selwin Ong
2e96148b31
Fixed Python 3 tests for "WorkingQueue".
10 years ago
Selwin Ong
60c7a3cc6e
working_queue.remove call should be pipelined.
10 years ago
Selwin Ong
4d90cc062e
Add job to WorkingQueue before execution and remove from WorkingQueue after.
10 years ago
Selwin Ong
b0c0a84ab0
Moved some logic into worker.prepare_job_execution to make things testable.
10 years ago
Selwin Ong
a28575088b
Implemented WorkingQueue.cleanup().
10 years ago
Selwin Ong
3dc008d090
Added WorkingQueue class.
10 years ago
foxx
b6499ce71c
Fixed another py3 bug - my bad. Also moved type checking further down
10 years ago
foxx
f5779c194f
Renamed job_id to id, and fixed py3 bug, per #415
10 years ago
Vincent Driessen
9ab46804c6
Merge pull request #417 from foxx/feature/testworker
...
Added test worker for unit tests - fixes #413
10 years ago
foxx
52c3ad0fbf
Renamed to SimpleWorker
10 years ago
foxx
16ee71f26d
Use class for test worker, and performs PID check in unit tests, #412
10 years ago
Vincent Driessen
b5fbc3992b
Restructure new CLI modules.
...
A few things have changed. First of all, there is no separate copy of
the argparse-based `rqinfo` anymore. It now fully utilizes the new
Click subcommand. In other words: `rqinfo` and `rq info` both invoke
the same function under the hood.
In order to support this, the main command group now does NOT take
a `url` option and initializes the connection. Besides supporting this
alias pattern, this change was useful for two more reasons: (1) it
allows us to add subcommands that don't need the Redis server running in
the future, and (2) it makes the `--url` option an option underneath
each subcommand. This avoids command invocations that look like this:
$ rq --url <url> info --more --flags
And instead allows us to pass the URL to each subcommand where it's
deemed necessary:
$ rq info --url <url> --more --flags
Which is much friendlier to use/remember.
10 years ago
Vincent Driessen
652cd71d2b
Merge branch 'zhangliyong-cli-rq'
10 years ago
Vincent Driessen
5c96c61255
Change `rq requeue` subtly, mostly docs and vars.
10 years ago
Vincent Driessen
07dda74710
Change options of the `rq empty` command.
...
* Does not ask confirmation
* Does not need `--yes` option anymore
* Added --all, to clear all queues
* No implicit emptying of the failed queue. If you want to empty the
failed queue, just use `$ rq empty failed`
10 years ago
Vincent Driessen
8f1e066f15
Various little style fixes.
10 years ago
Vincent Driessen
fc803f5d0d
Fix various flake8 complaints.
10 years ago
foxx
2753f17e8e
Fixed bug with format() in py3
10 years ago
foxx
72bc9e37b7
Forced job_id to only allow str/unicode
10 years ago
foxx
6bb8b26114
Allow job ID to be set on enqueue/enqueue_call() - fixes #412
10 years ago
Selwin Ong
21620e98ca
Merge pull request #411 from selwin/fix-exception-handler
...
Don't pass "args" into logging call
10 years ago
zhangliyong
3eabe76690
Move back rqinfo command for backward compatibility
10 years ago
zhangliyong
4248c934ab
Remove unused function 'gettersize'
10 years ago
Selwin Ong
7cf8ef42d1
Don't pass "args" into logging call.
10 years ago
zhangliyong
ae1681e7a0
Change doc of rq cli
10 years ago
zhangliyong
de471b70b6
Merge branch 'master' of github.com:nvie/rq into cli-rq
10 years ago