util
/
rq
mirror of https://github.com/peter4431/rq.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,456 Commits
2 Branches
0 Tags
3.3 MiB
main
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '52e426f40f'
${ noResults }
Commit Graph

930 Commits (52e426f40ffee046f06269fc2e7593707230e9f6)

Author SHA1 Message Date
Selwin Ong 549648bd1b
rq info management command now cleans up registries when first run (#1107) 
* rq info management command now cleans up registries when first run

* Deleted print statement

* Improve CLI test coverage

* Fixed CLI test on Linux
 6 years ago
Selwin Ong 905824e8d8
Improve worker start/stop logging messages (#1105) 6 years ago
Paul Robertson e1c135d4de add the ability to have the worker stop executing after a max amount of jobs (#1094) 
* add the ability to have the worker stop executing after a max amount of jobs

* rename to max-jobs

* updated logging messages
 6 years ago
janierdavila d191adbeb9 Fixes disable-job-desc-logging flag on CLI (#1096) 
PR #991 introduced this useful flag but it seems to have missed
passing this flag to the worker. This simple commit fixes that
issue.
 6 years ago
秋葉 e40fc393b2 remove unused code (#1092) 6 years ago
Ted Summer 79a6fd7999 Fix timeout adding job to StartedJobRegistry (#1086) 
* Fix timeout adding job to StartedJobRegistry

* Fix prepare_job_execution handling neg timeout

* Add test for inf job timeout in StartedJobRegistry

* refactor(worker): simplify checking neg timeout
 6 years ago
Selwin Ong 7021cedaf9
Implemented Job.fetch_many (#1072) 6 years ago
Selwin Ong c4cbb3af2f
RQ v1.0! (#1059) 
* Added FailedJobRegistry.

* Added job.failure_ttl.

* queue.enqueue() now supports failure_ttl

* Added registry.get_queue().

* FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL.

* StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry.

* Failed jobs are now added to FailedJobRegistry.

* Added FailedJobRegistry.requeue()

* Document the new `FailedJobRegistry` and changes in custom exception handler behavior.

* Added worker.disable_default_exception_handler.

* Document --disable-default-exception-handler option.

* Deleted worker.failed_queue.

* Deleted "move_to_failed_queue" exception handler.

* StartedJobRegistry should no longer move jobs to FailedQueue.

* Deleted requeue_job

* Fixed test error.

* Make requeue cli command work with FailedJobRegistry

* Added .pytest_cache to gitignore.

* Custom exception handlers are no longer run in reverse

* Restored requeue_job function

* Removed get_failed_queue

* Deleted FailedQueue

* Updated changelog.

* Document `failure_ttl`

* Updated docs.

* Remove job.status

* Fixed typo in test_registry.py

* Replaced _pipeline() with pipeline()

* FailedJobRegistry no longer fails on redis-py>=3

* Fixes test_clean_registries

* Worker names are now randomized

* Added a note about random worker names in CHANGES.md

* Worker will now stop working when encountering an unhandled exception.

* Worker should reraise SystemExit on cold shutdowns

* Added anchor.js to docs

* Support for Sentry-SDK (#1045)

* Updated RQ to support sentry-sdk

* Document Sentry integration

* Install sentry-sdk before running tests

* Improved rq info CLI command to be more efficient when displaying lar… (#1046)

* Improved rq info CLI command to be more efficient when displaying large number of workers

* Fixed an rq info --by-queue bug

* Fixed worker.total_working_time bug (#1047)

* queue.enqueue() no longer accepts `timeout` argument (#1055)

* Clean worker registry (#1056)

* queue.enqueue() no longer accepts `timeout` argument

* Added clean_worker_registry()

* Show worker hostname and PID on cli (#1058)

* Show worker hostname and PID on cli

* Improve test coverage

* Remove Redis version check when SSL is used

* Bump version to 1.0

* Removed pytest_cache/README.md

* Changed worker logging to use exc_info=True

* Removed unused queue.dequeue()

* Fixed typo in CHANGES.md

* setup_loghandlers() should always call logger.setLevel() if specified
 6 years ago
Wolfgang Langner abf6881114 Fix #1040 queue default timeout bug. (#1042) 
Add test for queue default_timeout.
 6 years ago
Wolfgang Langner 8fc987dc68 Make logging in worker consitent. (#1030) 
Switch some messages from warn to info because it is normal requested bahavior.
 7 years ago
Chyroc d9798fd64f refactor: job status check (#1035) 7 years ago
Chyroc 7eb95bf405 refactor: use try ImportError instead of py-version check (#1034) 7 years ago
Wolfgang Langner 7a6abb4738 Fix collections abc import deprecation warning. (#1032) 
* Fix collections abc import.
Prepare for 3.8 when it is deprecated.

* Fix Pyton 2.7 build.
 7 years ago
Finnci 14db0ecd26 Update/add flag for description logging (#991) 
* test workers

* indent

* add docs and add option to the cli

* rename flag for cli

* logging
 7 years ago
Selwin Ong bf0b4308fd Bump version to 0.13.0 7 years ago
Darshan Rai ada2ad03ca modify zadd calls for redis-py 3.0 (#1016) 
* modify zadd calls for redis-py 3.0

redis-py 3.0 changes the zadd interface that accepts a single
mapping argument that is expected to be a dict.
https://github.com/andymccurdy/redis-py#mset-msetnx-and-zadd

* change FailedQueue.push_job_id to always push a str

redis-py 3.0 does not attempt to cast values to str and is left
to the user.

* remove Redis connection patching

Since in redis-py 3.0, Redis == StrictRedis class, we no longer
need to patch _zadd and other methods.
Ref: https://github.com/rq/rq/pull/1016#issuecomment-441010847
 7 years ago
Selwin Ong 6559b0ffd7
Replace "timeout" argument in queue.enqueue() with "job_timeout" (#1010) 7 years ago
John Stowers eaf598d73c Pass job_id to death penalty class (#936) 
This allows custom workers to use associated
custom Timeout classes and apply custom timeouts
or less messy death methods
 7 years ago
Selwin Ong 47d291771f
SimpleWorker's ttl must always be longer than jobs. (#1002) 7 years ago
Qingping Hou 051104b0ff support workder name override from config file (#947) 
* support workder name override from config file

* update docs for NAME config

* Update workers.md
 7 years ago
Selwin Ong e00ff97867
Fix heroku worker (#993) 
* Attempt to fix HerokuWorker test.

* Fixed test_shutdown_double_sigrtmin

* Another attempt to fix test

* Another attempt to fix test

* Another attempt to fix test

* When work horse runs into an exception, raise that exception before calling os._exit()
 7 years ago
Selwin Ong 25f2b75b1f Horse should exit with return code 1 if it raises an exception 7 years ago
Selwin Ong 4759c43067 horse should always call os._exit(0) 7 years ago
shikharsg cc19d1a89e added logging formatting options (#979) 
* added logging formatting options

* added docs for log formatting options

* fixed naming format
 7 years ago
Paul Robertson e86fb57366 add is_async property to queue (#982) 7 years ago
Selwin Ong 6a376191d9 Bump version to 0.12.0 7 years ago
chevell c2b939d2df Replace 'async' keyword with 'is_async' for Queue objects (#977) 
* Replaced async keyword with is_async in the Queue class to fix reserved keyword syntax errors in Python 3.7

* Updated tests to use is_async keyword when instantiating Queue objects

* Updated docs to reference is_async keyword for Queue objects

* Updated tox.ini, setup.py and .travis.yml with references to Python 3.7
 7 years ago
Selwin Ong a5596caf42 Bump version to 0.11.0 7 years ago
Theofanis Despoudis 875cc27c2f #908 Using a timeout string value for job works (#955) 
Fixes https://github.com/rq/rq/issues/908
 7 years ago
Theofanis Despoudis d6b12c2402 Issue 872 (#954) 
* Fixes #872 - Use -1 to indicate infinite ttl

* Fixes #872 Restored comma

* #872 Code review fix
 7 years ago
Selwin Ong 531fde8e3c worker.main_work_horse should always return 0 7 years ago
Thomas Kriechbaumer 3133d94b58 add periodic worker heartbeats (#945) 
* add periodic worker heartbeats

fixes #944

* improve worker default option handling
 7 years ago
Selwin Ong c639018fb9 Registry objects can be instantiated by passing a queue object. 7 years ago
stj 487ef72f21 Define redis key prefix as class variable (#939) 
* Define redis key prefix as class variable

Some prefixes were hardcoded in several places. This made it hard to
use custom prefixes via subclasses.

Resolves #920

* fixup! Define redis key prefix as class variable
 7 years ago
Christophe Olinger a6eb5d37ee Delete dependents of job explicitely (#916) 
* Initial take on delete_dependents

* Add tests including corner cases

* No need to canel dependents since they are not in a queue yet anyway

* The dependents keys can be deleted in all cases

* Update tests to included saved jobs in the deletion tests

* Correctly use pipeline in cancel method

* Unused connection

* Include dependents into dict format of job

* Add TODO

* Address comments from selwin

* Delete dependents key in redis if delete_dependents is called on its own

* Address recent comments from selwin

* Small change to trigger travis

* Remove TODO referring to canceled job state

* Remove dependent_ids from to_dict

* Address recent comments from selwin
 8 years ago
Selwin Ong ca0d7ef9f1 Bump version to 0.10.0 8 years ago
Nikita Lyubchich a3eb6475dc Add description param to @job decorator (#912) 
After using `@job` decorator for a function that takes a long string, in RQ worker I got printed all the args/kwargs via `job.get_call_string()`.

To get this overridden, I added `description` argument to the decorator.

I decided not to put this in `delay` method because it's may be currently be used by end user.
 8 years ago
Selwin Ong 7a3c85f185
Added the ability to fetch workers by queue (#911) 
* job.exc_info is now compressed.

* job.data is now stored in compressed format.

* Added worker_registration.unregister.

* Added worker_registration.get_keys().

* Modified Worker.all(), Worker.all_keys() and Worker.count() to accept "connection" and "queue" arguments.
 8 years ago
John Lucas 34c403ec8d Add meta to decorator, move depends_on + at_front to decorator (#892) 8 years ago
Samuel Colvin df571e14fd improve logging in worker.py (#902) 
* improve logging in worker

* tests for log_result_lifespan
 8 years ago
Selwin Ong f500186f3d
Job compression (#907) 
job.exc_info and job.data is now stored in compressed format in Redis.

* job.data is now stored in compressed format.
 8 years ago
Selwin Ong 44a0a7b972 Bump version to 0.9.2. 8 years ago
vanife ff36e0656e Fixed an issue where `birth` not present in Redis (#901) 
* Fixed an issue where `birth` not present in Redis

Fixed an issue where worker.refresh() may fail if `birth` is not present in Redis

* added test coverage
 8 years ago
Selwin Ong e5de3dfbb5 Bump version to 0.9.1. 8 years ago
Selwin Ong 7b9c3b6b66 Fixed an issue where worker.refresh() may fail if last_heartbeat is not present in Redis. 8 years ago
Selwin Ong 25ced943e1 Merge branch 'master' of github.com:nvie/rq 8 years ago
Selwin Ong 1d7b5e834b
Worker statistics (#897) 
* First stab at implementing worker statistics.

* Moved worker data restoration logic to worker.refresh().

* Failed and successfull job counts are now properly incremented.

* Worker now keeps track of total_working_time

* Ensure job.ended_at is set in the case of unhandled job failure.

* handle_job_failure shouldn't crash if job.started_at is not present.
 8 years ago
Selwin Ong 3c98175a4a Bump version to 0.9.0 8 years ago
Theo f226d38603 Fixed #812 - Send heartbeat during suspension check 8 years ago
Selwin Ong 92c88d3f4d Merge pull request #878 from theodesp/Issue-731 
Fixed #731 - Support for deleting Queues
 8 years ago
Selwin Ong a4be98dc87 Merge pull request #895 from stylight/master 
try latin-1 if utf-8 doesn't work, when decoding exception strings
 8 years ago
ndparker 911c5e0ca3 try latin-1 if utf-8 doesn't work, when decoding exception strings 8 years ago
Dmitriy Sorokin d6e875119a Add at_front option for decorated functions 8 years ago
Theo c095fe1825 Fixed #731 - Code review issues. Added delete_jobs parameter and pipelining. 8 years ago
Selwin Ong 43c9279c8d Merge pull request #882 from jiajunhuang/specify_logging_level 
expose set logging level in cmd
 8 years ago
Jiajun Huang 7408c23bc5 expose set logging level in cmd 8 years ago
Samuel Colvin 260fd84f51 add milliseconds into timestamps, fix #721 8 years ago
Theo 160fe99323 Fixed #731 - Support for deleting Queues 8 years ago
Theo 096c5ad3c2 Fixed #866 - Flak8 errors 8 years ago
Samuel Colvin 423da3683c remove python 2.6 support 8 years ago
Selwin Ong cc41bdfa65 Bump version to 0.8.2. 8 years ago
Selwin Ong 801b3b7293 Added non zero check for Queue class. 8 years ago
Selwin Ong 4a9bc6e76a Merge branch 'master' of github.com:nvie/rq 8 years ago
Selwin Ong 03c3f66ca7 Bump version to 0.8.1. 8 years ago
Selwin Ong 54bc04bb45 job.save() shouldn't crash on unpickleable return value. 8 years ago
Simon Thompson fab545c193 path option needs to be a shared command to ensure that settings file can be loaded from subcommands such as requeue, resume etc otherwise sentinel config cannot be passed 8 years ago
Selwin Ong 2bc2f447d3 Merge pull request #841 from strawposter/master 
accept multiple path
 8 years ago
Selwin Ong 3666f4987e Merge pull request #855 from floqqi/master 
Windows: Use SIGTERM instead of SIGKILL
 8 years ago
Selwin Ong 5f5e113790 Merge pull request #831 from katichev/explicit_cleanup 
Replace job id with job instance in the job stack. Remove implicit "cleanup" call from "job.save"
 8 years ago
Selwin Ong 165b7ae6b9 Merge pull request #845 from liangsijian/master 
use enum WorkerStatus instead of str
 8 years ago
Florian 61f27215b3 Windows: Use SIGTERM instead of SIGKILL 8 years ago
Amjith Ramanujam 3cf5850afc Honor the job_class for failed_queue. 8 years ago
liangsijian 4556f39c34 use enum WorkerStatus instead of str 8 years ago
liangsijian 4875331b60 Optimize job.dependency 
job.refresh has been call in job.fetch
 8 years ago
Alexey Katichev 09697e567f revert back job.cleanup changes 8 years ago
kirill f2ff1b05f7 accept multiple path 8 years ago
Selwin Ong 8ebebc9f7c Merge pull request #833 from yaniv-g/extract_move_to_failed_queue 
Expose move_to_failed_queue from Worker
 8 years ago
Yaniv Greenberg 7dc5d4a936 local commit to avoid circular imports. 8 years ago
Yaniv Greenberg 2da4ccd48d Moved move_to_failed_queue to separate file. 8 years ago
Selwin Ong d71a6c2ece Merge pull request #832 from yaniv-g/master 
Expand timeout string feature to both result_ttl and ttl, and add days possibility.
 8 years ago
fate0 1ebf880402 fix uncaught StopRequested during send heartbeat 8 years ago
Yaniv Greenberg b49e039d63 Separate move_to_failed_queue from the worker to allow it's use in cli exception handler. 8 years ago
Yaniv Greenberg 39e43207a6 * Added support to use strings to specify both result_ttl and ttl. 
* Added Days as possible timeout string (as 'd')
 8 years ago
Alexey Katichev b7d50cedc4 replace job.id with job instance in local _job_stack 8 years ago
Alexey Katichev 3596449cc0 remove implicit cleanup call from job.save 8 years ago
Alexey Katichev a0113c83cf introduce job.update_meta() to store updated meta to Redis (#823) 
* introduce job.update_meta() to store updated meta to Redis

This closes nvie/rq#811

* rename update_meta to save_meta
 8 years ago
Selwin Ong fe84828d72 Bump up version to 0.8.0 and updated changelog. 8 years ago
Selwin Ong dc45ab8799 Worker.find_by_key should use hmget instead of repeated hget calls. (#826) 8 years ago
luojiebin cd529d0ce1 Fixed issue#72 (#818) 
* Added a custom exception for timeout transfer

* Added a util to transfer timeout to a united format

* Transfer timeout format when creating a queue or enqueue jobs

* Fixed typos

* Fixed bug in transfer_timeout function

* Added test for function transfer_timeout

* Updated transfer_timeout to allow uppercase unit

* Renamed function in utils
 8 years ago
Aarni Koskela 92cf6f4696 Add waitpid() return value to workhorse error/log messages (#819) 
* Refactor the inner part of the monitor_work_horse loop into a separate function

* Add waitpid() return value to workhorse error/log messages
 8 years ago
Peng Liu b7d4b4ec1b Solve the UnicodeDecodeError while decode literal things. (#817) 
* Solve the UnicodeDecodeError while decode literal things.

* Add test case for when worker result is a unicode or str object that other than
pure ascii content.
 8 years ago
Felipe Lacerda cab89254b5 Make `Queue.enqueue_job()` execute immediately if `async=False` (#798) 
Currently, the job is being performed inside `enqueue_call()`, which
means that `async=False` has no effect if `enqueue_job()` is called
directly. This commit fixes that.
 8 years ago
strawposter 1029adaf1b add sentinel support (#808) 
* add sentinel support

* add sentinel support

* add comment
 8 years ago
Peter Featherstone a99d79187b Minor spelling mistake 
Minor spelling update of `proccess` to `process`
 8 years ago
Selwin Ong 5660f925ae Merge pull request #788 from friedcell/master 
Better support for async job payloads
 8 years ago
Selwin Ong f6b4c286c9 Merge pull request #757 from jaywink/fix-unicode-decode-error 
Fix UnicodeDecodeError when failing jobs
 8 years ago
Selwin Ong f760fcb20f job.delete() should cleans itself from FailedQueue and various registries. 8 years ago
Samuel Colvin fd9babe8ce correct heroku worker exit logic 
as per @Chronial's comment on b4b99f3
 8 years ago
Marko Mrdjenovic c8a955fbb1 Merge commit '83007b20740a97f085a14d3840d1e434350c6600' 
# Conflicts:
#	rq/queue.py
 9 years ago
Jason Robinson 213969742e Fix UnicodeDecodeError when failing jobs 
Worker handle_exception and move_to_failed_queue couldn't handle a situation where the exception raised had non-ascii characters. This caused a UnicodeDecodeError when trying to format the exception strings.

If on Python 2, ensure strings get decoded before building the exception string.

Closes #482
 9 years ago