387 Commits (a33ed3a01959b028ad6394702071c8b8078e9d46)

Author SHA1 Message Date
Selwin Ong 7021cedaf9
Implemented Job.fetch_many (#1072) 7 years ago
Selwin Ong c4cbb3af2f
RQ v1.0! (#1059)
* Added FailedJobRegistry.

* Added job.failure_ttl.

* queue.enqueue() now supports failure_ttl

* Added registry.get_queue().

* FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL.

* StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry.

* Failed jobs are now added to FailedJobRegistry.

* Added FailedJobRegistry.requeue()

* Document the new `FailedJobRegistry` and changes in custom exception handler behavior.

* Added worker.disable_default_exception_handler.

* Document --disable-default-exception-handler option.

* Deleted worker.failed_queue.

* Deleted "move_to_failed_queue" exception handler.

* StartedJobRegistry should no longer move jobs to FailedQueue.

* Deleted requeue_job

* Fixed test error.

* Make requeue cli command work with FailedJobRegistry

* Added .pytest_cache to gitignore.

* Custom exception handlers are no longer run in reverse

* Restored requeue_job function

* Removed get_failed_queue

* Deleted FailedQueue

* Updated changelog.

* Document `failure_ttl`

* Updated docs.

* Remove job.status

* Fixed typo in test_registry.py

* Replaced _pipeline() with pipeline()

* FailedJobRegistry no longer fails on redis-py>=3

* Fixes test_clean_registries

* Worker names are now randomized

* Added a note about random worker names in CHANGES.md

* Worker will now stop working when encountering an unhandled exception.

* Worker should reraise SystemExit on cold shutdowns

* Added anchor.js to docs

* Support for Sentry-SDK (#1045)

* Updated RQ to support sentry-sdk

* Document Sentry integration

* Install sentry-sdk before running tests

* Improved rq info CLI command to be more efficient when displaying lar… (#1046)

* Improved rq info CLI command to be more efficient when displaying large number of workers

* Fixed an rq info --by-queue bug

* Fixed worker.total_working_time bug (#1047)

* queue.enqueue() no longer accepts `timeout` argument (#1055)

* Clean worker registry (#1056)

* queue.enqueue() no longer accepts `timeout` argument

* Added clean_worker_registry()

* Show worker hostname and PID on cli (#1058)

* Show worker hostname and PID on cli

* Improve test coverage

* Remove Redis version check when SSL is used

* Bump version to 1.0

* Removed pytest_cache/README.md

* Changed worker logging to use exc_info=True

* Removed unused queue.dequeue()

* Fixed typo in CHANGES.md

* setup_loghandlers() should always call logger.setLevel() if specified
7 years ago
Wolfgang Langner abf6881114 Fix #1040 queue default timeout bug. (#1042)
Add test for queue default_timeout.
8 years ago
Wolfgang Langner 8fc987dc68 Make logging in worker consitent. (#1030)
Switch some messages from warn to info because it is normal requested bahavior.
8 years ago
Chyroc 7eb95bf405 refactor: use try ImportError instead of py-version check (#1034) 8 years ago
Finnci 14db0ecd26 Update/add flag for description logging (#991)
* test workers

* indent

* add docs and add option to the cli

* rename flag for cli

* logging
8 years ago
Samuel Colvin 2f35222ddb skip test_1_sec_shutdown with pypy (#1020)
* skip test_1_sec_shutdown with pypy, fix #1019

* skip all HerokuWorkerShutdownTestCase with pypy
8 years ago
Darshan Rai ada2ad03ca modify zadd calls for redis-py 3.0 (#1016)
* modify zadd calls for redis-py 3.0

redis-py 3.0 changes the zadd interface that accepts a single
mapping argument that is expected to be a dict.
https://github.com/andymccurdy/redis-py#mset-msetnx-and-zadd

* change FailedQueue.push_job_id to always push a str

redis-py 3.0 does not attempt to cast values to str and is left
to the user.

* remove Redis connection patching

Since in redis-py 3.0, Redis == StrictRedis class, we no longer
need to patch _zadd and other methods.
Ref: https://github.com/rq/rq/pull/1016#issuecomment-441010847
8 years ago
Selwin Ong 6559b0ffd7
Replace "timeout" argument in queue.enqueue() with "job_timeout" (#1010) 8 years ago
Selwin Ong ad66d872f0 Fixed a unicode test. 8 years ago
Selwin Ong 47d291771f
SimpleWorker's ttl must always be longer than jobs. (#1002) 8 years ago
Paul Robertson e86fb57366 add is_async property to queue (#982) 8 years ago
chevell c2b939d2df Replace 'async' keyword with 'is_async' for Queue objects (#977)
* Replaced async keyword with is_async in the Queue class to fix reserved keyword syntax errors in Python 3.7

* Updated tests to use is_async keyword when instantiating Queue objects

* Updated docs to reference is_async keyword for Queue objects

* Updated tox.ini, setup.py and .travis.yml with references to Python 3.7
8 years ago
Theofanis Despoudis 875cc27c2f #908 Using a timeout string value for job works (#955)
Fixes https://github.com/rq/rq/issues/908
8 years ago
Theofanis Despoudis d6b12c2402 Issue 872 (#954)
* Fixes #872 - Use -1 to indicate infinite ttl

* Fixes #872 Restored comma

* #872 Code review fix
8 years ago
Selwin Ong 531fde8e3c worker.main_work_horse should always return 0 8 years ago
Thomas Kriechbaumer 3133d94b58 add periodic worker heartbeats (#945)
* add periodic worker heartbeats

fixes #944

* improve worker default option handling
8 years ago
Selwin Ong 63a04d275e Use dbsize() to test for empty Redis database 8 years ago
Selwin Ong c639018fb9 Registry objects can be instantiated by passing a queue object. 8 years ago
stj 487ef72f21 Define redis key prefix as class variable (#939)
* Define redis key prefix as class variable

Some prefixes were hardcoded in several places. This made it hard to
use custom prefixes via subclasses.

Resolves #920

* fixup! Define redis key prefix as class variable
8 years ago
Christophe Olinger a6eb5d37ee Delete dependents of job explicitely (#916)
* Initial take on delete_dependents

* Add tests including corner cases

* No need to canel dependents since they are not in a queue yet anyway

* The dependents keys can be deleted in all cases

* Update tests to included saved jobs in the deletion tests

* Correctly use pipeline in cancel method

* Unused connection

* Include dependents into dict format of job

* Add TODO

* Address comments from selwin

* Delete dependents key in redis if delete_dependents is called on its own

* Address recent comments from selwin

* Small change to trigger travis

* Remove TODO referring to canceled job state

* Remove dependent_ids from to_dict

* Address recent comments from selwin
9 years ago
Selwin Ong 7a3c85f185
Added the ability to fetch workers by queue (#911)
* job.exc_info is now compressed.

* job.data is now stored in compressed format.

* Added worker_registration.unregister.

* Added worker_registration.get_keys().

* Modified Worker.all(), Worker.all_keys() and Worker.count() to accept "connection" and "queue" arguments.
9 years ago
John Lucas 34c403ec8d Add meta to decorator, move depends_on + at_front to decorator (#892) 9 years ago
Samuel Colvin df571e14fd improve logging in worker.py (#902)
* improve logging in worker

* tests for log_result_lifespan
9 years ago
Selwin Ong f500186f3d
Job compression (#907)
job.exc_info and job.data is now stored in compressed format in Redis.

* job.data is now stored in compressed format.
9 years ago
vanife ff36e0656e Fixed an issue where `birth` not present in Redis (#901)
* Fixed an issue where `birth` not present in Redis

Fixed an issue where worker.refresh() may fail if `birth` is not present in Redis

* added test coverage
9 years ago
Selwin Ong 7b9c3b6b66 Fixed an issue where worker.refresh() may fail if last_heartbeat is not present in Redis. 9 years ago
Selwin Ong 1d7b5e834b
Worker statistics (#897)
* First stab at implementing worker statistics.

* Moved worker data restoration logic to worker.refresh().

* Failed and successfull job counts are now properly incremented.

* Worker now keeps track of total_working_time

* Ensure job.ended_at is set in the case of unhandled job failure.

* handle_job_failure shouldn't crash if job.started_at is not present.
9 years ago
Selwin Ong 92c88d3f4d Merge pull request #878 from theodesp/Issue-731
Fixed #731 - Support for deleting Queues
9 years ago
Theo c095fe1825 Fixed #731 - Code review issues. Added delete_jobs parameter and pipelining. 9 years ago
Samuel Colvin 260fd84f51 add milliseconds into timestamps, fix #721 9 years ago
Selwin Ong 19bc288378 Merge pull request #877 from theodesp/Issue-809
Fixed #809 - Added tests for various cli config parameters
9 years ago
Theo 160fe99323 Fixed #731 - Support for deleting Queues 9 years ago
Theo 0fab93d683 Fixed #809 - Added tests for various cli config parameters 9 years ago
Theo 261f4ac3d5 Fixed #866 - Flak8 errors 9 years ago
Theo 096c5ad3c2 Fixed #866 - Flak8 errors 9 years ago
Samuel Colvin 423da3683c remove python 2.6 support 9 years ago
Theo ee64114e6e Fixed #870 Improved test coverage for connections.py and utils.py 9 years ago
Selwin Ong 0efb87a46b Fixed test error in Python 3. 9 years ago
Selwin Ong 54bc04bb45 job.save() shouldn't crash on unpickleable return value. 9 years ago
Alexey Katichev 09697e567f revert back job.cleanup changes 9 years ago
Alexey Katichev 3596449cc0 remove implicit cleanup call from job.save 9 years ago
Alexey Katichev a0113c83cf introduce job.update_meta() to store updated meta to Redis (#823)
* introduce job.update_meta() to store updated meta to Redis

This closes nvie/rq#811

* rename update_meta to save_meta
9 years ago
Selwin Ong dc45ab8799 Worker.find_by_key should use hmget instead of repeated hget calls. (#826) 9 years ago
luojiebin cd529d0ce1 Fixed issue#72 (#818)
* Added a custom exception for timeout transfer

* Added a util to transfer timeout to a united format

* Transfer timeout format when creating a queue or enqueue jobs

* Fixed typos

* Fixed bug in transfer_timeout function

* Added test for function transfer_timeout

* Updated transfer_timeout to allow uppercase unit

* Renamed function in utils
9 years ago
Peng Liu b7d4b4ec1b Solve the UnicodeDecodeError while decode literal things. (#817)
* Solve the UnicodeDecodeError while decode literal things.

* Add test case for when worker result is a unicode or str object that other than
pure ascii content.
9 years ago
Felipe Lacerda cab89254b5 Make `Queue.enqueue_job()` execute immediately if `async=False` (#798)
Currently, the job is being performed inside `enqueue_call()`, which
means that `async=False` has no effect if `enqueue_job()` is called
directly. This commit fixes that.
9 years ago
Selwin Ong f6b4c286c9 Merge pull request #757 from jaywink/fix-unicode-decode-error
Fix UnicodeDecodeError when failing jobs
10 years ago
Selwin Ong f760fcb20f job.delete() should cleans itself from FailedQueue and various registries. 10 years ago
Samuel Colvin fd9babe8ce correct heroku worker exit logic
as per @Chronial's comment on b4b99f3
10 years ago