369 Commits (c639018fb9572cc1b9fc1cd371d53853df2afdc8)

Author SHA1 Message Date
Selwin Ong c639018fb9 Registry objects can be instantiated by passing a queue object. 7 years ago
stj 487ef72f21 Define redis key prefix as class variable (#939)
* Define redis key prefix as class variable

Some prefixes were hardcoded in several places. This made it hard to
use custom prefixes via subclasses.

Resolves #920

* fixup! Define redis key prefix as class variable
7 years ago
Christophe Olinger a6eb5d37ee Delete dependents of job explicitely (#916)
* Initial take on delete_dependents

* Add tests including corner cases

* No need to canel dependents since they are not in a queue yet anyway

* The dependents keys can be deleted in all cases

* Update tests to included saved jobs in the deletion tests

* Correctly use pipeline in cancel method

* Unused connection

* Include dependents into dict format of job

* Add TODO

* Address comments from selwin

* Delete dependents key in redis if delete_dependents is called on its own

* Address recent comments from selwin

* Small change to trigger travis

* Remove TODO referring to canceled job state

* Remove dependent_ids from to_dict

* Address recent comments from selwin
7 years ago
Selwin Ong 7a3c85f185
Added the ability to fetch workers by queue (#911)
* job.exc_info is now compressed.

* job.data is now stored in compressed format.

* Added worker_registration.unregister.

* Added worker_registration.get_keys().

* Modified Worker.all(), Worker.all_keys() and Worker.count() to accept "connection" and "queue" arguments.
7 years ago
John Lucas 34c403ec8d Add meta to decorator, move depends_on + at_front to decorator (#892) 7 years ago
Samuel Colvin df571e14fd improve logging in worker.py (#902)
* improve logging in worker

* tests for log_result_lifespan
7 years ago
Selwin Ong f500186f3d
Job compression (#907)
job.exc_info and job.data is now stored in compressed format in Redis.

* job.data is now stored in compressed format.
7 years ago
vanife ff36e0656e Fixed an issue where `birth` not present in Redis (#901)
* Fixed an issue where `birth` not present in Redis

Fixed an issue where worker.refresh() may fail if `birth` is not present in Redis

* added test coverage
7 years ago
Selwin Ong 7b9c3b6b66 Fixed an issue where worker.refresh() may fail if last_heartbeat is not present in Redis. 7 years ago
Selwin Ong 1d7b5e834b
Worker statistics (#897)
* First stab at implementing worker statistics.

* Moved worker data restoration logic to worker.refresh().

* Failed and successfull job counts are now properly incremented.

* Worker now keeps track of total_working_time

* Ensure job.ended_at is set in the case of unhandled job failure.

* handle_job_failure shouldn't crash if job.started_at is not present.
7 years ago
Selwin Ong 92c88d3f4d Merge pull request #878 from theodesp/Issue-731
Fixed #731 - Support for deleting Queues
7 years ago
Theo c095fe1825 Fixed #731 - Code review issues. Added delete_jobs parameter and pipelining. 7 years ago
Samuel Colvin 260fd84f51 add milliseconds into timestamps, fix #721 7 years ago
Selwin Ong 19bc288378 Merge pull request #877 from theodesp/Issue-809
Fixed #809 - Added tests for various cli config parameters
7 years ago
Theo 160fe99323 Fixed #731 - Support for deleting Queues 7 years ago
Theo 0fab93d683 Fixed #809 - Added tests for various cli config parameters 7 years ago
Theo 261f4ac3d5 Fixed #866 - Flak8 errors 7 years ago
Theo 096c5ad3c2 Fixed #866 - Flak8 errors 7 years ago
Samuel Colvin 423da3683c remove python 2.6 support 7 years ago
Theo ee64114e6e Fixed #870 Improved test coverage for connections.py and utils.py 7 years ago
Selwin Ong 0efb87a46b Fixed test error in Python 3. 8 years ago
Selwin Ong 54bc04bb45 job.save() shouldn't crash on unpickleable return value. 8 years ago
Alexey Katichev 09697e567f revert back job.cleanup changes 8 years ago
Alexey Katichev 3596449cc0 remove implicit cleanup call from job.save 8 years ago
Alexey Katichev a0113c83cf introduce job.update_meta() to store updated meta to Redis (#823)
* introduce job.update_meta() to store updated meta to Redis

This closes nvie/rq#811

* rename update_meta to save_meta
8 years ago
Selwin Ong dc45ab8799 Worker.find_by_key should use hmget instead of repeated hget calls. (#826) 8 years ago
luojiebin cd529d0ce1 Fixed issue#72 (#818)
* Added a custom exception for timeout transfer

* Added a util to transfer timeout to a united format

* Transfer timeout format when creating a queue or enqueue jobs

* Fixed typos

* Fixed bug in transfer_timeout function

* Added test for function transfer_timeout

* Updated transfer_timeout to allow uppercase unit

* Renamed function in utils
8 years ago
Peng Liu b7d4b4ec1b Solve the UnicodeDecodeError while decode literal things. (#817)
* Solve the UnicodeDecodeError while decode literal things.

* Add test case for when worker result is a unicode or str object that other than
pure ascii content.
8 years ago
Felipe Lacerda cab89254b5 Make `Queue.enqueue_job()` execute immediately if `async=False` (#798)
Currently, the job is being performed inside `enqueue_call()`, which
means that `async=False` has no effect if `enqueue_job()` is called
directly. This commit fixes that.
8 years ago
Selwin Ong f6b4c286c9 Merge pull request #757 from jaywink/fix-unicode-decode-error
Fix UnicodeDecodeError when failing jobs
8 years ago
Selwin Ong f760fcb20f job.delete() should cleans itself from FailedQueue and various registries. 8 years ago
Samuel Colvin fd9babe8ce correct heroku worker exit logic
as per @Chronial's comment on b4b99f3
8 years ago
Jason Robinson 213969742e Fix UnicodeDecodeError when failing jobs
Worker handle_exception and move_to_failed_queue couldn't handle a situation where the exception raised had non-ascii characters. This caused a UnicodeDecodeError when trying to format the exception strings.

If on Python 2, ensure strings get decoded before building the exception string.

Closes #482
8 years ago
Selwin Ong 83007b2074 Merge pull request #786 from jezdez/backend-class-overrides
Allow passing backend classes from CLI and other APIs
8 years ago
Benjamin Root 30a7ab4899 Add similar test for when the job fails 8 years ago
Benjamin Root f68aeff481 Added integration test for the metadata persistence feature 8 years ago
Jannis Leidel c019662430
Allow passing backend classes (job, queue, worker, connection) from CLI and other APIs
This includes:

- a partial refactor of the CLI to organize the shared options
- extends the tests in areas where passing custom backend classes makes sense
- allow setting the core CLI options as env vars
- minor cosmetic changes here and there
8 years ago
Jannis Leidel 27e4f3a768
Small refactor to simplify registry key setup. 8 years ago
Samuel Colvin 763a304ec6 cope with python 2.6 yawwwwwwn 8 years ago
Samuel Colvin f50bdb31bf skip test_run_scheduled_access_self on pypy 8 years ago
Samuel Colvin b4b99f30b0 allow for already dead horse on heroku worker termination 8 years ago
Julien Surloppe dc3bba9362 Another check on failed status and test 8 years ago
Selwin Ong 2428370dfd Merge pull request #763 from jezdez/skip-heroku-macos
Stop running some Heroku tests on macOS since it’s missing the appropriate signals
8 years ago
Jannis Leidel fcec17c8e1
Stop running some Heroku tests on macOS since it’s missing the appropriate signals
Refs issue #584.
8 years ago
Stefan Hammer f9bff3d12b Merge branch 'master' into 739_fix_race_condition 8 years ago
Vincent Driessen af6ce54ea4 Fix PEP8 complaints 8 years ago
Stefan Hammer a0cee2d2a0 refactored worker code
Moved code into a new handle_job_success() method and reduced context of used
pipelines.
8 years ago
Samuel Colvin afc7469c27 fetch_job - check correct queue, fix #728 8 years ago
Stefan Hammer 301e5c927b Raise an exception if a given dependency does not exist
Adapted some tests to the change: the dependency has to be saved first.
8 years ago
Stefan Hammer 44f98693c7 added a test for the race condition 8 years ago