Marko Mrdjenovic
3da3eab11d
moved job run to separate method to make async jobs easier
8 years ago
Benjamin Root
efb30b45a3
Take a different tack, and enable the exclusion of meta from saves
8 years ago
Benjamin Root
56c89c1334
Trigger a refresh after job execution
...
* Prevents clobbering of user-supplied metadata
modified during the job execution
8 years ago
Jannis Leidel
c019662430
Allow passing backend classes (job, queue, worker, connection) from CLI and other APIs
...
This includes:
- a partial refactor of the CLI to organize the shared options
- extends the tests in areas where passing custom backend classes makes sense
- allow setting the core CLI options as env vars
- minor cosmetic changes here and there
8 years ago
Jannis Leidel
27e4f3a768
Small refactor to simplify registry key setup.
8 years ago
Selwin Ong
b241d50c10
Merge pull request #774 from samuelcolvin/heroku-warm-shutdown-fix
...
allow for already dead horse on heroku worker termination
8 years ago
Atala
544f985d41
fix: set HTTP transport as default for sentry
8 years ago
Samuel Colvin
b4b99f30b0
allow for already dead horse on heroku worker termination
8 years ago
Michael DeWulf
9e8922a099
Don't run lrem directly if we have a pipeline
8 years ago
Selwin Ong
6721463fdf
queue.enqueue_call should call Job.exists with connection parameters.
8 years ago
Selwin Ong
1ae10dd529
Bump version to 0.7.1.
8 years ago
Julien Surloppe
dc3bba9362
Another check on failed status and test
8 years ago
Julien Surloppe
88a3c60177
Fix get job on failed queue
8 years ago
Stefan Hammer
f9bff3d12b
Merge branch 'master' into 739_fix_race_condition
8 years ago
Michael DeWulf
f3f61bbc15
Pass pipeline to push_job_id from enqueue_job
8 years ago
Vincent Driessen
20e258f610
Bump to 0.7.0
8 years ago
Vincent Driessen
af6ce54ea4
Fix PEP8 complaints
8 years ago
Vincent Driessen
5945c28ce3
Remove legacy script
8 years ago
Stefan Hammer
a0cee2d2a0
refactored worker code
...
Moved code into a new handle_job_success() method and reduced context of used
pipelines.
8 years ago
Samuel Colvin
afc7469c27
fetch_job - check correct queue, fix #728
8 years ago
Selwin Ong
e9a6c2e329
Merge pull request #747 from th3hamm0r/handle_missing_dependency
...
Raise an exception if a given dependency does not exist
8 years ago
Stefan Hammer
301e5c927b
Raise an exception if a given dependency does not exist
...
Adapted some tests to the change: the dependency has to be saved first.
8 years ago
Stefan Hammer
67ae9277e5
completely pipelined enqueue_dependents
8 years ago
Stefan Hammer
7d267cf7ef
implemented correct watch/multi/exec
8 years ago
任晓磊
ee4cf6e3ee
fix get_redis_from_config()
8 years ago
任晓磊
83f81b351d
fix default argument
8 years ago
任晓磊
191cc28543
custom connection class
8 years ago
Stefan Hammer
7b8d4e075e
protecting dependency enqueuing with redis WATCH
8 years ago
Samuel Colvin
bbf50f4893
use compat for PY2 test and add tests
9 years ago
Selwin Ong
a0497a2bfb
Merge pull request #727 from spiliopoulos/fix_failing_heroku_signal_handler
...
Fix request_force_stop_sigrtmin failing for python3
9 years ago
Yannis Spiliopoulos
7d0895ecf3
Small bugfix when checking python version
9 years ago
Yannis Spiliopoulos
2e30c4016b
Fix request_force_stop_sigrtmin failing for python3
...
request_force_stop_sigrtmin would fail for python3 because it would
try to read frame attributes that have been removed in python3
This patch fixes that by reading more fram attributes only for
python2
9 years ago
Yannis Spiliopoulos
1a9b36b19e
Merge remote-tracking branch 'upstream/master' into fix_issue_702
9 years ago
Yannis Spiliopoulos
b38aaab8a7
Inline unhandled failure handler
9 years ago
Yannis Spiliopoulos
14d8783212
Rename function
9 years ago
Yannis Spiliopoulos
9b774771db
Fix style of conditional
9 years ago
Tom Wallis
89a380ef4b
Added better signal name lookup
9 years ago
Yannis Spiliopoulos
08de4190e7
Dry the code. Export handling failed current job to a method
9 years ago
Yannis Spiliopoulos
c4fd1659ea
Fix docstring for monitor_work_horse
9 years ago
Yannis Spiliopoulos
f9d5897922
Solves issue 702
...
In order to solve issue 702 we have to check whether a work-horse
terminated unexpectedly (by inspecting the exit code of the work-horse
process). If it exited unexpectedly we check if the job has either been
marked as finished, failed or other valid states. If it's not in any
valid state we mark it as failed and move it to the failed queue.
Since the process was terminated unexpectedly (think OOM) we do not
have any exception context and we can't run any custom exception handlers.
There is still a chance that the job will finish successfully but the
work-horse process will be killed before the job is marked as finished
and we will erroneously mark it as failed. The users should take care
to write idempotent jobs.
9 years ago
Yannis Spiliopoulos
93d286a6c7
Split execute job to expose issue 702
...
In order to create a test for issue 702 we had to split execute_job
to a fork_work_horse function and a monitor_work_horse function.
9 years ago
Vincent Driessen
07adca6ca3
Bump dev version
9 years ago
Vincent Driessen
46c39654d0
Merge old tag 0.6.0 so it becomes part of master again
...
Otherwise, there's no linear history between this old tag and any new,
future, tags. This is likely been a manual mistake when I released
0.6.0, so this is corrected now.
9 years ago
Samuel Colvin
7efd036a2c
move ShutDownImminentException into exceptions.py
9 years ago
Samuel Colvin
0e26db9e08
correct wording in docstring and tests
9 years ago
Samuel Colvin
9f9c887645
better function names and process double SIGRTMIN
9 years ago
Samuel Colvin
2b544e5b17
add tests to HerokuWorker
9 years ago
Samuel Colvin
e2f89b3171
fix for python 2.6
9 years ago
Samuel Colvin
18ba4658a4
adding heroku worker as per #584
9 years ago
Arnold Krille
acbcea0c66
Add the workers connection to _connection_stack
...
This allows jobs to use get_current_connection() with a resolvable connection.
And then these jobs can schedule new jobs for example (my use-case). Or attach
information to their job-object.
Also pop the pushed connection after running the jobs. This is needed for some
tests that check the _connection_stack afterwards;-) And also for use-cases
where the workers are used multiple times.
fixes nvie/rq#479
9 years ago
Javier Lopez
8e24d3a92d
job.py: Fix cancel failure on failed queue
9 years ago
Vincent Driessen
6a57d321f2
Ignore flake8 checks in local.py
9 years ago
Selwin Ong
7b1df1e878
Bump version to 0.6.0.
9 years ago
Selwin Ong
766bb60006
Merge branch 'dependent-jobs-with-result-ttl'
9 years ago
Selwin Ong
e9d227c3df
Dependent jobs are now correctly enqueued even if their parent jobs have result_ttl=0.
9 years ago
Selwin Ong
8b7b0e5584
Merge pull request #701 from jlopex/javi/fix_issue_700
...
Fixes issue #700
9 years ago
Javier Lopez
c5e62c70bb
job.py: Use fetch to get a complete Job instance
9 years ago
amyangfei
724c844378
Don't call job.cancel if job has finished
9 years ago
Javier Lopez
3668c6f5d5
worker.py: Fix default logging_level
9 years ago
Selwin Ong
94dfb943b1
Merge pull request #690 from liorsbg/master
...
Fix for Issue #644
9 years ago
liorsbg
4c5d04105a
Fix for #644
9 years ago
Selwin Ong
cd0414361c
Merge pull request #687 from jlopex/javi/allow_quiet_worker_class
...
Allow Worker work method to specify the log verbosity
9 years ago
Javier Lopez
43cb51064a
Allow Worker work method to specify the log verbosity
9 years ago
Chris Curvey
4326e19140
have worker use the passed-in connection if provided
9 years ago
zfz
c2374c8035
Correct the comment of job's requeue function.
9 years ago
zfz
0492eaef01
Fix the mistypo of logger str for procline.
9 years ago
Selwin Ong
8158a180f3
Merge pull request #666 from redbaron4/python_2_6_log
...
Fix logging regression for Python-2.6
9 years ago
Selwin Ong
af0b19e707
Merge pull request #642 from butla/master
...
Worker only creates log handlers if there aren't any.
9 years ago
Dheeraj
0a49e247ea
Fix logging regression for Python-2.6
...
logger.setLevel() doesn't work when a string is passed in Python-2.6.
This patch checks if the level has been set and if not (in Python-2.6),
tries to set it again using appropriate level constants
Also unused dictConfig import is removed
9 years ago
Selwin Ong
e1e4c00a67
Merge pull request #659 from javimb/javimb/decorator-ttl
...
Add ttl argument to decorator
9 years ago
Antoine Leclair
81679a35d9
Merge branch 'master' into custom-queue-class
...
Conflicts:
tests/test_worker.py
9 years ago
VicarEscaped
01bd0cebd6
Update job status with set_status
9 years ago
Javi Imbernon
1d6c2b98b7
Add ttl argument to decorator
9 years ago
Antoine Leclair
05ed85804c
Worker accepts custom queue class
9 years ago
Selwin Ong
640b2e31a0
Merge pull request #633 from olingerc/warmshutdown
...
Save date on which a busy worker receives a warm shutdown request
9 years ago
orangain
e8165fdddf
Accept byte strings as the first argument of Worker() in Python 2
...
This make it easy to write Python 2/3 compatible code.
e.g.
Worker(sys.argv[1:])
# Without from __future__ import unicode_literals
Worker(['high', 'normal', 'low'])
9 years ago
Michal Bultrowicz
fe122ac799
Worker only creates log handlers if there aren't any.
9 years ago
Eduard Carreras
c27a6434a8
Set job as finished when not working in async mode
9 years ago
Nick Snell
0e532f3ec4
Correction to worker command, cast results-ttl to int
9 years ago
Christophe Olinger
2be468a404
- Rename paramter both in redis and python to shutdown_requested_date
...
- Use connection instead of pipeline
9 years ago
Christophe Olinger
ae27ce9871
Merge branch 'master' into warmshutdown
9 years ago
Christophe Olinger
3e586bd6d6
Add remove current_job based on the correct upstream master branch
9 years ago
Christophe Olinger
d622f47aff
Save date in redis on which worker receives a warm shutdown request while busy
9 years ago
Selwin Ong
2485334100
Merge pull request #609 from tornstrom/master
...
Allow meta when enqueing
9 years ago
SkyLothar
70b5554566
make sure exc_info is text type
9 years ago
SkyLothar
f9978bc27c
make sure state text_type
9 years ago
Tornstrom
50a114a0a8
Allow meta when enqueing
9 years ago
Selwin Ong
45addbcaf2
Deleted job.set_started_at_now.
9 years ago
Selwin Ong
364919c7c8
Moved job.hset_value to connection._hset.
9 years ago
Selwin Ong
961e443611
Merge pull request #619 from foxx/patch-1
...
Fixes #613
9 years ago
Selwin Ong
5afd1a90e5
Merge pull request #589 from samuelcolvin/job-started_at
...
add job.started_at
9 years ago
Samuel Colvin
6fc34dc6a6
moving started_at to prepare_job_execution
9 years ago
Cal Leeming
d80f9f8ba0
Fixes #613
...
This has been discussed in #514 , #282 and #88 .
Using an explicit type check via `isinstance`, rather than duck typing, is typically considered unpythonic and breaks compatibility with mock objects such as FakeRedis. This patches removes the type check, and instead looks for a common method that should be present on the object as a hint on whether it's compatible or not.
9 years ago
Selwin Ong
8bbd833855
Merge pull request #600 from glaslos/cancel_remove
...
Cancel and Delete differences
9 years ago
Selwin Ong
702dfd2c31
Merge pull request #611 from ahxxm/delay-task
...
depreciation fixes
9 years ago
Michael Keirnan
14723ecc1f
Merge remote-tracking branch 'upstream/master'
9 years ago
ahxxm
b06f112cb0
fix tests
...
syntax: assertEquals -> assertEqual, assertNotEquals -> assertNotEqual
usage: status of worker and job now will use get/set method instead of property method
9 years ago
Selwin Ong
91177b7317
Merge pull request #601 from glaslos/registry_cleanup_log_msg
...
log.info added when cleaning registries
9 years ago
glaslos
a424d320a8
log.info added when cleaning registries
9 years ago
Karoly Frendrich
ca069e5112
rqinfo config parameter
9 years ago
glaslos
0a6df13d9d
delete dependents and delete in cleanup. Fixed tests.
9 years ago
glaslos
d2eeb3b3db
job.cancel before delete and doc string fixes
9 years ago
glaslos
02844034d7
Merge branch 'master' into cancel_remove
9 years ago
glaslos
5a3bebf85b
only remove from queue on job.cancel
9 years ago
horida
04987a80da
mark as busy before forking horse
9 years ago
Selwin Ong
95f740299a
Merge pull request #588 from samuelcolvin/logging-extras
...
more info in 'Job OK' log entry
9 years ago
Samuel Colvin
082e61a212
add job.started_at
9 years ago
Samuel Colvin
30c29614c6
more info in 'Job OK' log entry
9 years ago
Samuel Colvin
e750134e8a
move request_(force_)stop out of _install_signal_handlers
9 years ago
Vincent Driessen
ca235aee19
This is 0.5.6
9 years ago
Selwin Ong
544f2e6fb2
Fixed result logging call .
9 years ago
Selwin Ong
2140cc4fab
Job result should be converted to text before being logged.
9 years ago
Selwin Ong
bb4336f612
Merge pull request #578 from tbaugis/master
...
log job result in DEBUG level and truncate to 2k symbols
9 years ago
Toms Bauģis
5a4f297134
do not truncate result output as that can be done in the log formatter
9 years ago
Michael Keirnan
da7d59e1a9
Merge remote-tracking branch 'upstream/master'
...
* upstream/master:
Modified patch_connection to allow Redis connection to be easily mocked.
9 years ago
Michael Keirnan
bad054989d
export worker id and job id to env of work horse
...
For tracing job execution in a distributed system it is useful to tag log
entries with the worker id and job id. The current job is accessible
via get_current_job(), but that requires an extra redis connection. And
the current worker id (the worker id of the parent process) does not appear
to be available.
Rather than introducing an `rqworker` alternative or subclassing Worker, it
feels simple and efficient to make these contextual ids available as
environment variables. This should have no performance cost and no
API compatibility issues.
Some useful things to do with these values in the worker horse process:
+ include them in log messages
+ include them as 'x-' headers in HTTP requests made by workers
9 years ago
Selwin Ong
e538512c79
Merge branch 'master' of github.com:nvie/rq
...
Conflicts:
tests/test_job.py
9 years ago
Selwin Ong
0e4112da7a
Modified patch_connection to allow Redis connection to be easily mocked.
9 years ago
Toms Bauģis
ade154575e
fix travis error - python 2.6 or whatevs wants positions spelled out for .format
9 years ago
Selwin Ong
1a089887c0
Merge pull request #564 from jlopex/javi/fix_dependent_jobs_on_different_queues
...
support multiple queues on dependent jobs
9 years ago
Toms Bauģis
682d4adb40
log result in DEBUG level and truncate to 2k symbols
9 years ago
Javier Lopez
d3d9a20680
Do not call pipeline.execute() if pipeline is given as argument
9 years ago
Selwin Ong
cf20f1d9b2
Merge pull request #576 from jlopex/ensure_custom_handlers_are_called
...
Ensure custom exception handlers call when Redis is down
9 years ago
Vincent Driessen
02c6df6a45
This is 0.5.5
9 years ago
Vincent Driessen
c1a4780d10
Fix PEP8 complaints
9 years ago
Javier Lopez
4fb59a4ceb
Ensure custom exception handlers calls when Redis is down
9 years ago
Javier Lopez
8a6bc08e47
Added pipeline named argument to enqueue_job
9 years ago
Javier Lopez
b4f157516e
enqueue_dependents now uses origin queue to enqueue
...
Current implementation is using the first job queue
to insert the dependent jobs. This makes RQ to ignore
the original queue where the jobs were enqueued.
With this change, we will use job origin attribute to
fetch the original queue, then insert the dependent
job to the fetched queue.
fixes issue #512
9 years ago
Selwin Ong
e6a15c57b3
Disable click >= 5.0's unicode_literals warnings.
9 years ago
Selwin Ong
f09be523dd
Merge pull request #573 from jcsaaddupuy/feature/dequeue_any_not_recursive
...
dequeue_any not recursive
9 years ago
Jean-Christophe SAAD-DUPUY
8fbf5452fa
fix: return (None, None) for dequeue_any if no result
10 years ago
Jean-Christophe SAAD-DUPUY
59d193590c
fix: remove recursive call to dequeue_any
10 years ago
Damnever
43616ea696
_Colorizer().notty is more clear
10 years ago
Dustin Falgout
91a3aa7493
make args and kwargs explicit for enqueue_call function
...
closes #466
10 years ago
likeon
fc4124e39c
Fixes Job.__eq__ crash when other object doesn't have id attribute
10 years ago
Selwin Ong
e6a499ada8
Merge pull request #561 from selwin/async-bug
...
Sync jobs should be cleaned up after execution.
10 years ago
Selwin Ong
edd139d86f
Sync jobs should be cleaned up after execution.
10 years ago
Selwin Ong
cdcea84105
Merge pull request #557 from selwin/exception_handling
...
Exception handling
10 years ago
Vincent Driessen
e88525a649
This is 0.5.4
10 years ago
Vincent Driessen
28a19ee717
Remove warning that's now breaking since the Raven API changed
10 years ago
Selwin Ong
3c3646cf91
Merge branch 'master' into exception_handling
...
Conflicts:
tests/fixtures.py
10 years ago
Selwin Ong
8bfbeac67d
Running rqworker without specifiying --exception-handler shouldn't override default behavior.
10 years ago
mozillazg
7198b4568a
Fixed a format error
10 years ago
Vincent Driessen
0de225ec62
Emphasize private nature of `stopped` property
10 years ago
Vincent Driessen
ec0e04727b
Rename `maintenance_date` -> `last_cleaned_at`
10 years ago
Vincent Driessen
891be55b87
Clean up Worker.__init__ logic a little
10 years ago
Vincent Driessen
4d0ae5da98
Clean up type checking of Worker constructor
10 years ago
Vincent Driessen
94258761ae
Make string formatting consistent
10 years ago
Selwin Ong
8f9c507f12
Bump version to 0.5.3.
10 years ago
Marcus Martins
513f6310d2
Change default TTL to None insted of -1
10 years ago
Marcus Martins
5b8726ad2d
Fixes #502
...
Fixes some broken tests and misbehaviour with ttls. There was a temporal
coupling between saving the job and setting its expires parameter.
10 years ago
Selwin Ong
f370f79819
Merge pull request #534 from selwin/registry-maintenance
...
Worker automatically cleans job registries every hour
10 years ago
Neal Todd
bac6699ea4
Allow non-ASCII characters in keyword arguments.
10 years ago
Selwin Ong
c3767e28e2
Worker now runs maintenance tasks every hour and on startup.
10 years ago
Selwin Ong
5782ac10c4
Added worker.clean_registries().
10 years ago
Selwin Ong
faf9d3e668
Added clean_registries(queue) function to clean job registries related to that queue.
10 years ago
RyanMTB
9a00b0eca6
Updated Worker API
10 years ago
Selwin Ong
d51f0200d7
Merge pull request #513 from trevorprater/master
...
Improving logging.
10 years ago
Selwin Ong
70d5f971bd
Jobs from FinishedJobRegistry that are moved to FailedQueue should have "failed" as status.
10 years ago
Cosmin Stefan
260f7caf66
Enable proper setup of signale handlers for SimpleWorker as well. fixes #523
...
This allows a SIGTERM to make the worker perform a warm shutdown and cleanly break out of the loop and finish the current job, if any.
10 years ago
Vincent Driessen
8f7322ed10
This is 0.5.2
10 years ago
Trevor Prater
5674edad61
Changed rqworker shutdown message.
...
- As requested by @nvie and @selwin.
10 years ago
Trevor Prater
fd1dca40b9
Improving logging.
...
- Include worker key in worker startup log statement.
- Added a notification to make it more clear when a 'burst' worker dies.
10 years ago
Selwin Ong
cff6fbff41
Merge pull request #477 from glaslos/ttl_tests_fixes
...
TTL tests and fixes
10 years ago
glaslos
071c47dacb
hard-coded TTL to -1 on job.perform()
...
Removed merge artifacts
10 years ago
Serhii Maltsev
91f263d8e0
change try/except in python2/3 compatibility to to_text()
10 years ago
Tamir Duberstein
985a2664a4
Prevent `Queue#dequeue` from blowing the stack
...
In the case of many sequential jobs having been deleted, a recursive
implementation of `Queue#dequeue` is prone to blowing the stack in the
absence of tail-recursion support. Change the implementation from
recursive to iterative to work around this issue in CPython.
10 years ago
Vincent Driessen
5cb873b438
Fix PEP8 complaint.
10 years ago
Vincent Driessen
cd155299a4
Fix PEP8 complaint.
10 years ago
Vincent Driessen
6c6e53542f
Don't require redis-py 2.10 just for the SSL option.
10 years ago
Vincent Driessen
8505198298
Bump to 0.5.1.
10 years ago
Vincent Driessen
5990fa9d73
Merge pull request #499 from vladkens/issue/498
...
ZADD bugfix
10 years ago
Eric Bustarret
9fba806662
Add REDIS_SSL option
...
Allow the worker to connect to a Redis instance through SSL (ex: Azure
Redis Cache use SSL only by default)
10 years ago
Vlad Pronsky
933e6697cd
Fixed redis drivers bug
10 years ago
glaslos
3d4d6a86d5
persist the job right before execution
10 years ago
glaslos
24d5e08992
Merge branch 'master' into ttl_tests_fixes
...
Conflicts:
tests/test_job.py
10 years ago
Selwin Ong
30b21ef06e
Merge pull request #465 from marklap/workerbirthdeath
...
add birth_date and death_date properties to Worker
10 years ago
Mark LaPerriere
636a537fa7
updates addressing @selwin comments for PR #465
10 years ago
Taras Semenenko
3dfd044767
Add missed `multi` command after `watch`
...
`watch` command should be used in conjunction with `multi` command
which was missed in enqueuing of job with dependencies.
Fix #487
10 years ago
Bradley Young
04e403e1a3
dread comma, I adjure thee.
10 years ago
Bradley Young
d56b5424c8
merging master and fixing conflicts
10 years ago
Bradley Young
c428f955e4
per @selwin, adding a deprecated handler for exc_handler
10 years ago
alternativshik
219f21b637
Allow non-ASCII characters in arguments
...
get_call_string() failed if any arguments contained non-ASCII strings.
Fixes #406
10 years ago
Selwin Ong
1005222371
Restored the ability to specify connection params in config
10 years ago
glaslos
c155918d6d
lrem only first occurrence of the job id (we expect only one)
10 years ago
Mark LaPerriere
a4ca4704ac
convert redis returned bytes into text/string
10 years ago
Mark LaPerriere
cfe3da287a
updated branch to latest
10 years ago
Vincent Driessen
b8d425b318
Various PEP8 fixes.
10 years ago
Vincent Driessen
fac2b10309
Auto-sort imports using isort.
10 years ago
Vincent Driessen
e8c3b96a5a
Bump to 0.5.0.
10 years ago
glaslos
703ab0e355
removed merged comment
10 years ago
glaslos
646f1cda22
Merge branch 'master' into ttl_tests_fixes
...
Conflicts:
rq/queue.py
10 years ago
glaslos
5b5ab4860f
passing on the ttl
10 years ago
Bradley Young
f05d77b92b
changing "default" for exception_handlers to None; same exact logic can be implemented, but cleaner
10 years ago
Bradley Young
d3e4fb567b
merge && updates to add multiple exception handlers.
10 years ago
Bradley Young
f575137612
updating worker to have exception_handlers
10 years ago
Selwin Ong
dac0be6cc7
Added DeferredJobsRegistry to keep track of deferred jobs.
10 years ago
Selwin Ong
3e674fbe6a
queue.enqueue() should set job.origin.
10 years ago
Selwin Ong
9320496402
Simplify FailedQueue.quarantine and ensure that a deferred job's status is set to Queued when enqueued.
10 years ago
Selwin Ong
7fd2ac8ca6
Added "DEFERRED" Job status for jobs that have unsatisfied dependencies.
10 years ago
Vincent Driessen
dc5cd514ee
Explicitly cast map() result to a list, for Python 3 compat.
10 years ago
Selwin Ong
dc09676ee0
Minor copywriting fixes.
10 years ago
Selwin Ong
01ab2f20dd
Minor copywriting fixes.
10 years ago
Vincent Driessen
543bcbc60b
Explicitly cast map() result to a list, for Python 3 compat.
10 years ago
Selwin Ong
eae2c40ea0
Merge pull request #457 from jlopex/block_until_horse_finishes
...
Make parent worker process wait until horse process finishes
10 years ago
Selwin Ong
0a0a81da6d
Merge pull request #463 from conslo/skip-queue
...
Skip queue
10 years ago
Mark LaPerriere
e6c32edad9
add birth_date and death_date properties to Worker
10 years ago
Jonathan Tushman
1ae5a12a81
merging master
10 years ago
Jonathan Tushman
ccd41396f6
adding suspend with duration test
10 years ago
Travis Johnson
add4d625b6
Merge branch 'nvie' into skip-queue
...
Conflicts:
rq/queue.py
10 years ago
Vincent Driessen
cd0c3c9c1e
Merge pull request #462 from conslo/worker-hash
...
Worker hash
10 years ago
Jonathan Tushman
82333d2ad5
triggering shutdown by setting a redis flag
10 years ago
Travis Johnson
db75958ad2
use 'at_front' instead of 'skip_queue'
10 years ago
Selwin Ong
e1801c5c44
Merge pull request #453 from glaslos/job_ttl
...
TTL for jobs
10 years ago
Travis Johnson
ac61f502a1
skip_queue functionality
10 years ago
Travis Johnson
f60e4884df
a comment typo
10 years ago
Travis Johnson
0ad4cb3410
comments
10 years ago
Travis Johnson
c301369c2e
implement __eq__ and __hash__ for workers
10 years ago
Nic Cope
09cab7a90d
Allow maximum job age to be specified when cleaning up or getting expired jobs from a registry.
...
This ensures that all jobs cleaned from a started registry end up in the failed queue.
10 years ago
Javier Lopez
fc5295edfc
Set busy state on parent while waiting for horse
10 years ago
Nic Cope
6ab7070a92
Rename registry timeout to registry ttl
...
This matches job.cleanup() better and (I think) makes the purpose slightly clearer.
10 years ago
Nic Cope
9f699d758d
Any negative number should cause a job to live forever in a registry.
...
While the documentation explicitly mentions ttl=-1, this matches better the behaviour of the job.cleanup() method.
10 years ago
glaslos
c6a83eaa82
calling self.cleanup to set job expiration
10 years ago
glaslos
8a3fd91e2e
renaming job.job_ttl to job.ttl
10 years ago
Selwin Ong
2091c601b1
Merge pull request #451 from conslo/depend-on-unsaved-job-ids
...
Depend on unsaved job ids
10 years ago
Nic Cope
8fa184b86b
Ensure the FinishedJobRegistry honors an 'infinite' result_ttl of -1
10 years ago
glaslos
55c541bc59
added job ttl to queue.enqueue()
10 years ago
Travis Johnson
af4b0436c1
instantiate a job instead of fetch
...
this removes the need for the depended on job.id already being saved
10 years ago
Travis Johnson
14d1186241
use internal job_class for check
10 years ago
Bradley Young
5caccaabfe
Adding optional list handling to the exc_handler option in Worker.
...
Adding command line --exception_handler option (with multiple entries allowed) to `rq worker`
Added tests for command line options.
10 years ago
Travis Johnson
3467868f1f
allow depends_on to be a job id or a job itself
...
Other parts of the code (i.e.: the `.create()` method) allow the `depends_on` kwarg to be a `Job` object *or* a job id. This is an attempt to allow that same idea within the `.enqueue_call()` method for a queue. Since this part of the code requires actually knowing the precise redis key for the job that's depended on, my intuition is that a `.fetch()` is the only way to solve this.
10 years ago
Selwin Ong
0dbe68527c
Merge pull request #431 from negz/worker-redis-bug
...
Pass the default worker connection into the started and finished registries
10 years ago
Adam Wen
f5001a0e36
fix decode error in py3.
10 years ago
Adam Wen
1795e1ee53
fix decode
10 years ago
Adam Wen
cd0230cae3
make job description unicode friendly
10 years ago
Nic Cope
de0b501823
Pass the default worker connection into the started and finished registries
...
In practice all the registry methods are called with pipelines, but if no
connection is passed in when instantiating the registry and the connection
stack is empty resolve_connection will raise NoRedisConnectionException.
10 years ago
zhangliyong
6621105bde
Use type instead of default to specify interval option type
10 years ago
zhangliyong
3d49784bcc
Merge branch 'master' of github.com:nvie/rq into cli-rq-worker
10 years ago
Selwin Ong
2307bc8253
Added FinishedJobRegistry to monitor finished jobs.
10 years ago
Selwin Ong
74a9982ecb
Fixed a bug where expired jobs aren't deleted from JobStartedRegistry.
10 years ago
Selwin Ong
18c6ea64ef
Merge pull request #419 from selwin/registry-len
...
Added JobStartedRegistry.count
10 years ago
zhangliyong
f9511664c8
Make flake8 happy :)
10 years ago
zhangliyong
1b39996bf7
Make pep8
10 years ago
zhangliyong
c6033b312e
Cleanup
10 years ago
zhangliyong
6510bb46f3
Fix interval type bug, it should be float
10 years ago
zhangliyong
21c14c0e21
Cleanup
10 years ago
zhangliyong
4d91f136f8
Move redis url default value to connect function
...
So worker subcommand can retrieve redis url from config file if url
option value is None, otherwise url option value will never be None.
10 years ago
zhangliyong
4df076f3aa
Retrieve url from config file
10 years ago
zhangliyong
229a00fee5
Cleanup
10 years ago
zhangliyong
88fe5124d8
Make pythonic
10 years ago
zhangliyong
907e71dc86
Make pythonic
10 years ago
zhangliyong
f10c749b7c
Use click's evvar to retrieve SENTRY_DSN environment value
10 years ago
zhangliyong
842f27294a
Convert rqworker to 'rq worker' subcommand
10 years ago
zhangliyong
7b434a32eb
Let cleanup_ghosts accept conn argument
10 years ago
Selwin Ong
202be75b21
Merge pull request #415 from foxx/feature/custom-job-ids
...
Allow job ID to be set on enqueue/enqueue_call() - fixes #412
10 years ago
Selwin Ong
c1dc30eae3
Added __len__ method to Queue.
10 years ago
Selwin Ong
fae7df5aa7
Renamed StartedJobRegistry.get_job_count to StartedJobRegistry.count for consistency.
10 years ago
Selwin Ong
41ae1ce8a7
Added a registry.get_job_count().
10 years ago
foxx
30ea76ddd6
Fixed silly typo
10 years ago
foxx
0466562a13
Fixes from comments in #415
...
Added tox/vagrant tmp dir to gitignore
Removed unused import
Renamed id to job_id in enqueue_call() as per comments in #415
10 years ago
Vincent Driessen
783b6f9ec7
Fix PEP8 complaint.
10 years ago
Vincent Driessen
7686785d61
Merge branch 'selwin-working-queue'
...
Conflicts:
tests/test_worker.py
10 years ago
Vincent Driessen
4b7c59d68d
Fix PEP8 complaints.
10 years ago
Selwin Ong
6d79082b62
Call move_expired_jobs_to_failed_queue before returning job_ids.
10 years ago
Selwin Ong
9341a4a33d
Renamed WorkingQueue to StartedJobRegistry.
10 years ago
Selwin Ong
2e96148b31
Fixed Python 3 tests for "WorkingQueue".
10 years ago
Selwin Ong
60c7a3cc6e
working_queue.remove call should be pipelined.
10 years ago
Selwin Ong
4d90cc062e
Add job to WorkingQueue before execution and remove from WorkingQueue after.
10 years ago
Selwin Ong
b0c0a84ab0
Moved some logic into worker.prepare_job_execution to make things testable.
10 years ago
Selwin Ong
a28575088b
Implemented WorkingQueue.cleanup().
10 years ago
Selwin Ong
3dc008d090
Added WorkingQueue class.
10 years ago
foxx
b6499ce71c
Fixed another py3 bug - my bad. Also moved type checking further down
10 years ago
foxx
f5779c194f
Renamed job_id to id, and fixed py3 bug, per #415
10 years ago
Vincent Driessen
9ab46804c6
Merge pull request #417 from foxx/feature/testworker
...
Added test worker for unit tests - fixes #413
10 years ago
foxx
52c3ad0fbf
Renamed to SimpleWorker
10 years ago
foxx
16ee71f26d
Use class for test worker, and performs PID check in unit tests, #412
10 years ago
Vincent Driessen
b5fbc3992b
Restructure new CLI modules.
...
A few things have changed. First of all, there is no separate copy of
the argparse-based `rqinfo` anymore. It now fully utilizes the new
Click subcommand. In other words: `rqinfo` and `rq info` both invoke
the same function under the hood.
In order to support this, the main command group now does NOT take
a `url` option and initializes the connection. Besides supporting this
alias pattern, this change was useful for two more reasons: (1) it
allows us to add subcommands that don't need the Redis server running in
the future, and (2) it makes the `--url` option an option underneath
each subcommand. This avoids command invocations that look like this:
$ rq --url <url> info --more --flags
And instead allows us to pass the URL to each subcommand where it's
deemed necessary:
$ rq info --url <url> --more --flags
Which is much friendlier to use/remember.
10 years ago
Vincent Driessen
652cd71d2b
Merge branch 'zhangliyong-cli-rq'
10 years ago
Vincent Driessen
5c96c61255
Change `rq requeue` subtly, mostly docs and vars.
10 years ago
Vincent Driessen
07dda74710
Change options of the `rq empty` command.
...
* Does not ask confirmation
* Does not need `--yes` option anymore
* Added --all, to clear all queues
* No implicit emptying of the failed queue. If you want to empty the
failed queue, just use `$ rq empty failed`
10 years ago
Vincent Driessen
8f1e066f15
Various little style fixes.
10 years ago
Vincent Driessen
fc803f5d0d
Fix various flake8 complaints.
10 years ago
foxx
2753f17e8e
Fixed bug with format() in py3
10 years ago
foxx
72bc9e37b7
Forced job_id to only allow str/unicode
10 years ago
foxx
6bb8b26114
Allow job ID to be set on enqueue/enqueue_call() - fixes #412
10 years ago
Selwin Ong
21620e98ca
Merge pull request #411 from selwin/fix-exception-handler
...
Don't pass "args" into logging call
10 years ago
zhangliyong
3eabe76690
Move back rqinfo command for backward compatibility
10 years ago
zhangliyong
4248c934ab
Remove unused function 'gettersize'
10 years ago
Selwin Ong
7cf8ef42d1
Don't pass "args" into logging call.
10 years ago
zhangliyong
ae1681e7a0
Change doc of rq cli
10 years ago
zhangliyong
de471b70b6
Merge branch 'master' of github.com:nvie/rq into cli-rq
10 years ago
zhangliyong
f734708ad4
Make pythonic
10 years ago
zhangliyong
5670f251d8
Add '--all' option and [JOB_IDS] argument to specify what to requeue
10 years ago
Dave Bonner
6eea986012
allow queue class to be overriden from command line
...
use the same mechanism as the override for worker class.
10 years ago
zhangliyong
37748e5ace
Make pep8
10 years ago
zhangliyong
1c97a55064
compat str.format syntax with py26
10 years ago
zhangliyong
0c4e28bd62
Use 'with Connection' instead of 'use_connection'
10 years ago
zhangliyong
1e5a8b58a1
Use -y option to confirm emptying failed queue by default
10 years ago
zhangliyong
c90da5336e
Avoid using use_connection, initial queues directly using redis connection
10 years ago
zhangliyong
adc24b6fe0
Fix import error
10 years ago
zhangliyong
551483b477
Add rq info command
10 years ago
zhangliyong
81febc6dc5
Format empty help message
11 years ago
zhangliyong
f845ae25a2
Make command empty help message friendly
11 years ago
zhangliyong
36880db872
Show progressbar when requeuing
11 years ago
zhangliyong
e1d236ec02
Use click.echo instead of print to output
11 years ago
zhangliyong
7f3cb8e785
Merge branch 'master' into cli-rq
11 years ago
zhangliyong
07aea5e5c1
Add CLI `rq` to empty queues and requeue failed jobs
11 years ago
INADA Naoki
4fb2914c6f
Easy refactoring on decorators.py
11 years ago
Selwin Ong
b05c2052e1
Capture more data when logging errors.
11 years ago
Vincent Driessen
3de8a47f06
Merge pull request #390 from selwin/pipeline-dependency-registration
...
job.register_dependency() should support pipelining
11 years ago
Selwin Ong
fc19a64670
job.register_dependency() should support pipelining.
11 years ago
Vincent Driessen
b91b89a183
Use highest pickle protocol version to serialize job data.
11 years ago
Vincent Driessen
79db282879
Support enqueueing callable classes.
...
Fixes #388 .
11 years ago
Vincent Driessen
865efd6e8c
Leave setting the state to the worker, not the Job itself.
...
Should fix #389 .
11 years ago
Vincent Driessen
eb6fb27281
Change confusing method name.
11 years ago
Vincent Driessen
6299af15fc
Merge branch 'master' of github.com:nvie/rq
11 years ago
Vincent Driessen
d37f49f643
Merge branch 'andrewsg-warnings'
11 years ago
Vincent Driessen
12932a433b
Merge pull request #382 from selwin/cancel-should-remove-from-queue
...
job.cancel() should remove itself from queue
11 years ago
Selwin Ong
2ab7276230
Merge pull request #377 from selwin/logging-fix
...
Only add logging handler if nothing is configured
11 years ago
Selwin Ong
638211df20
job.cancel() should remove itself from queue.
11 years ago
Selwin Ong
b828078dde
Fixed default logging level in setup_loghandlers.
11 years ago
Selwin Ong
92fb787a69
Only add logging handler if nothing is configured.
11 years ago
Scott Vitale
b464c37468
Use as_text to fix syntax error discovered in Python 3.2 (raspberry pi)
11 years ago
David Bonner
e2f398be80
fix --worker-ttl command line arg
...
Make sure it's converted to an int, since the rest of the code expects it to be. Without this patch, trying to use that argument gets you something like this:
(cogo)[dbonner@bonnervm18 syseng]$ rqworker --worker-ttl 86400
21:52:07 RQ worker started, version 0.4.6
Traceback (most recent call last):
File "/opt/cogo/bin/rqworker", line 9, in <module>
load_entry_point('rq==0.4.6', 'console_scripts', 'rqworker')()
File "/home/dbonner/src/github/rq/rq/scripts/rqworker.py", line 102, in main
w.work(burst=args.burst)
File "/home/dbonner/src/github/rq/rq/worker.py", line 354, in work
timeout = None if burst else max(1, self.default_worker_ttl - 60)
TypeError: unsupported operand type(s) for -: 'str' and 'int'
11 years ago
Selwin Ong
8095a565ac
rqworker script should accept "--job-class" parameter.
11 years ago
Selwin Ong
7ac1c3500a
Worker.__init__ should accept custom job class.
11 years ago
Selwin Ong
141278bb42
Queue.__init__ supports job_class argument.
11 years ago
Andrew Gorcester
17a4ed056d
use warnings.warn() instead of raise for DeprecationWarning
11 years ago
Vincent Driessen
6644872273
Raise warning when using async Sentry transport.
11 years ago
Vincent Driessen
2314b62d8c
Start release 0.4.6.
11 years ago
Vincent Driessen
f0e02996bc
Use single quotes for string literals.
11 years ago
Vincent Driessen
0fbc9e4851
Release 0.4.5.
11 years ago
Marko Mrdjenovic
78b8d2a6ef
kwargs must be strings, unicode fails with "keywords must be strings"
11 years ago
Vincent Driessen
972b367af4
Release 0.4.4.
11 years ago
Vincent Driessen
83382f51d0
PEP8ify.
11 years ago
Vincent Driessen
c263f8f600
Fix regression introduced in 0.4.3.
11 years ago
Vincent Driessen
c0567be69a
Release 0.4.3.
11 years ago
Vincent Driessen
fba11aac91
Merge branch 'SanyerMyasoedov-master'
11 years ago
Vincent Driessen
c5939479b5
Merge remote-tracking branch 'origin/master' into SanyerMyasoedov-master
...
Conflicts:
rq/decorators.py
tests/__init__.py
tests/test_decorator.py
11 years ago
Vincent Driessen
b048e23875
Merge pull request #342 from friedcell/master
...
make job class dynamic
11 years ago
Vincent Driessen
ab9e6b852e
Fix PEP8 complaints.
11 years ago
Vincent Driessen
38ec259b6e
Enable the most modern Python syntax.
11 years ago
Vincent Driessen
4746602c57
Explicit string check in Job constructor.
11 years ago
Vincent Driessen
9def988a85
Flip conditional sides of helper definitions (no semantic change).
11 years ago
Marko Mrdjenovic
5cba260cca
Job and Queue classes as variables (compatible with existing naming)
11 years ago
msoedov
e76020d894
Removed @resolve_connection from job decorator.
...
-- resolve_connection already presence inside Queue __init__.
11 years ago
msoedov
b4bccb2d31
Moved @resolve_connection call into @delay wrapper
11 years ago
Marko Mrdjenovic
67dd1cbf34
make job class dynamic for more hackability
11 years ago
Vincent Driessen
ef9456ddbd
Bump version to 0.4.2.
11 years ago
Vincent Driessen
5fe82bd543
Merge pull request #341 from SanyerMyasoedov/master
...
Added depends_on to job decorator
11 years ago
msoedov
46c96f9481
Added depends_on to job decorator
11 years ago
Vincent Driessen
415662d42c
Bump version.
11 years ago
Vincent Driessen
e60584ef76
Use single quotes for string literals.
11 years ago
Vincent Driessen
dfd23d6fb8
Fix bug where RQ 0.4 could not unpickle jobs from RQ 0.3.x.
11 years ago
Vincent Driessen
39258e2c15
Release 0.4.0.
11 years ago
Vincent Driessen
aef7af9c77
Merge pull request #334 from nvie/emptying-failed-queue
...
Emptying failed queue
11 years ago
Vincent Driessen
7ea106e50e
Merge pull request #333 from selwin/explicit-args
...
Rearranged how explicit kwargs are passed into queue.enqueue(). Fixes #322
11 years ago
Vincent Driessen
cb34acc279
Add rqinfo -X command.
...
This can be used to empty the failed queue right from the command line.
11 years ago
Vincent Driessen
4d9c20d5d9
Reimplement Queue.empty() in a Lua script.
...
This makes the .empty() function perform all the computing in Redis
itself, rather than in Python. This is both atomic, and faster.
11 years ago
Vincent Driessen
057c4657ef
Fix whitespace.
11 years ago
Vincent Driessen
fdf4abcf69
Fix pipeline call.
11 years ago
Selwin Ong
4f918041e3
Rearranged how explicit kwargs are passed into queue.enqueue(). Fixes #322
11 years ago
Tomas Hanacek
6494233058
rqworker default arguments of socket, worker_ttl, results_ttl bugfix
11 years ago
Vincent Driessen
35d839f4e2
Merge pull request #330 from selwin/death-penalty
...
Job timeouts are now handled by "worker.death_penalty_class"
11 years ago
Vincent Driessen
3649ff863b
Don't rely on external package `first`.
11 years ago
Selwin Ong
7eb6c2ab9f
Job timeouts are now handled by "worker.death_penalty_class".
11 years ago
Vincent Driessen
a55be12864
Merge branch 'malthe-default-to-environment-args'
11 years ago
Vincent Driessen
3667e37ed3
Don't expose the envvar values through `rqinfo -h`.
11 years ago
Vincent Driessen
1ada3462dd
Merge pull request #329 from olingerc/workonrqinfo
...
Use get_state() instead of .state in rqinfo
11 years ago
Christophe Olinger
e16b89de3b
Use get_state() instead of .state in rqinfo
11 years ago
Malthe Borch
e77b3e8a21
Pull argument parser defaults from environment variables.
11 years ago
Selwin Ong
835c353380
Job status should be updated during job.perform().
11 years ago
Selwin Ong
57990fa052
Simplify worker.work() by moving some functionalities to relevant methods.
11 years ago
Selwin Ong
f5c3c9a6dd
Deprecate job.status.
11 years ago
Selwin Ong
72457d2286
Added DeprecationWarning to worker.state.
11 years ago
Selwin Ong
2fe5d9e25e
Python 3 compatibility with worker.get_current_job().
11 years ago
Selwin Ong
802ecb5ccb
Renamed worker.set_job_id() and worker.get_job_id() for consistency.
11 years ago
Selwin Ong
93bb9ec5f4
Added tests for worker.get_current_job().
11 years ago
Selwin Ong
1b3832adb5
Merge branch 'master' of https://github.com/mbr/rq into worker-get_job_id
...
Conflicts:
rq/worker.py
11 years ago
Selwin Ong
37376cfe42
Deprecate worker.state.
11 years ago
Vincent Driessen
383ffa464f
Merge branch 'rdooley-master'
11 years ago
Vincent Driessen
d2bbd31c59
Fix some flake8 complaints.
11 years ago
Vincent Driessen
f7e2375b36
Remove two-char short args.
...
In UNIX, this would mean the shorthand notation for `-w -t`, which is
confusing and conflicts with the existing `-w` and `-t` flags. Instead,
if you need this, you'll have to spell the flag name out because we've
run out of chars that make sense.
11 years ago
Vincent Driessen
c7498b2066
Merge pull request #312 from selwin/rename-fork_and_perform
...
Renamed Worker.fork_and_perform_job to Worker.execute_job.
11 years ago
Malthe Borch
ad529e2609
Add repr for (showing id and enqueue time).
11 years ago
Selwin Ong
2044e790f5
Renamed Worker.fork_and_perform_job to Worker.execute_job.
11 years ago
Rees Dooley
fb65e321ed
Changed timeout to results_ttl. Added similar worker_ttl param
11 years ago
Rees Dooley
f8e775a784
Results timeout arg added to rqworker
11 years ago
Marc Brinkmann
e81ee5b672
Update worker.py
...
Use docstring instead of comment. Sorry.
11 years ago
Vincent Driessen
9ad7da684c
Merge branch 'KanbanSolutions-use-pipeline-context'
11 years ago
Vincent Driessen
d0381e98ee
Fix .pipeline -> ._pipeline.
...
This makes sure it's compatible with both Redis and StrictRedis back
ends.
11 years ago
Sylvain Zimmer
f0a3dd262e
Added connection argument to get_current_job()
11 years ago
Dj Gilcrease
fd036538f7
Wrap the perform_job try/except in a pipeline context so the pipeline connection is properly refrshed after the work is done
11 years ago
Vincent Driessen
44233709ed
Allow two-way setting of job data properties.
...
As a side-effect, this also enables us to lazily load the data.
11 years ago
Vincent Driessen
66a554bc29
Change AssertionError to proper TypeError.
11 years ago
Malthe Borch
ce3924c901
Just use a regular private method along with 'property'.
11 years ago
Malthe Borch
c898fe618e
Move seemingly general purpose decorator into class scope, use and delete.
...
Also, reduce repeat attribute declarations, at the cost of a
requirement that decorated functions appear in the right order.
11 years ago
Malthe Borch
ba0b39a43b
Unpickle job data lazily. This fixes issue #294 .
11 years ago
Vincent Driessen
888d771d4d
Merge pull request #287 from malthe/shed-times-dependency
...
Remove dependency on 'times' library (issue #286 ).
11 years ago
Selwin Ong
c02bf3d6aa
Merge pull request #289 from acmerfight/master
...
delte Unused variable 'prefix'
11 years ago