Commit Graph

1455 Commits (4d450ec94d4c09f28723af20aa1ca88bc109738a)
 

Author SHA1 Message Date
Selwin Ong 4d450ec94d Bump version to 1.4.1
Bo Bayles f0846a7645
Use pickle.HIGHEST_PROTOCOL by default ()
Bo Bayles 5859339a51
Avoid deprecation warnings on redis-py 3.5.0 hmset ()
Selwin Ong e6e5703f28 Merge branch 'master' of github.com:rq/rq
Selwin Ong 02eb983e90 Bump version to 1.4.0
Michael Angeletti 6ab6a0a573
Remove extraneous try/except ()
The exception handling block was raising the caught exception in-place, which caused the original traceback info to be lost. Rather than replace `raise e` with `raise`, I simply removed the whole try / except, since no action was being taken in the except block.
Selwin Ong 9d6f38df0e Slightly increase job key timeout in monitor_work_horse()
grayshirt d8dea02081
Parse job_id as keyword argument to delay() () ()
Vincent Jacques 9fb80424ef
Fix typo in scheduling doc ()
rmartin48 a922a553cb
Always set job.started_at in monitor_work_horse ()
Co-authored-by: Russell Martin <russell@divipay.io>
Prajjwal Nijhara b2be17417f
Fix some code quality issues ()
Pierre Mdawar eb92d688a8
Add the queue to the Redis queues set when scheduling a job ()
* Add the queue to the queues set when scheduling a job

* Fix the registry properties docstrings
Pierre Mdawar 0dd9ff0ec9
Accept lowercase logging level names and accept tuples when setting exception handlers ()
* Accept lowercase logging level names

* Accept both lists and tuples when setting Worker exception_handlers
Babatunde Olusola e1cbc3736c
Implement Customizable Serializer Support ()
* Implement Customizable Serializer Support

* Refractor serializer instance methods

* Update tests with other serializers

* Edit function description

* Edit function description

* Raise appropriate exception

* Update tests for better code coverage

* Remove un-used imports and un-necessary code

* Refractor resolve_serializer

* Remove un-necessary alias from imports

* Add documentation

* Refractor tests, improve documentation
Paweł Bąk d914343e7c
Add sentry_debug and sentry_ca_certs params ()
Co-authored-by: pawel bak <p.bak@inteliclinic.com>
Selwin Ong cfe389bd65
FailedJobRegistry.requeue() resets job.started_at and job.ended_at ()
Selwin Ong 636d6d2f54
registry.cleanup() now writes information to job.exc_info ()
Selwin Ong a090e9e840
Remove unused code ()
Selwin Ong add0b171ec Bump version to 1.3.0
Samuel Colvin 4036471203
fixing HerokuWorkerShutdownTestCase after ()
Selwin Ong 8a0153efc8 Properly decode hostname in job.refresh()
Selwin Ong d8bd455c12
enqueue_at should support explicit args and kwargs ()
Seamus Mac Conaonaigh 8f7dbf1b1d
Pass job ID to error handlers ()
The worker handles exceptions in the job outside of the job's own context, so an exception handler / logger cannot call `get_current_job()` to obtain the job ID. The job ID can be used to locate the job in the failed job registry, which allows useful behaviors such as linking to a failed job on a dashboard in an error report.

Closes .
wevsty dc299bc7ec
fix kill_horse will cause zombie processes ()
* fix kill_horse will cause zombie processes

fix issue 

* Update tips message
Selwin Ong bf93e11f2f Merge branch 'master' of github.com:rq/rq
Selwin Ong 57f8fe6a0c Updated job.result docs.
Bo Bayles 42457eed9b
Add rq.cli.__main__ ()
Tim Gates a49680f789
Fix simple typo: constrast -> contrast ()
Closes 
Ryan Febriansyah 71b01ba1b6
fix code tag in connections.md ()
* Update connections.md

* Update testing.md
Zach Kanzler 2e245a7b4e
Respect timeout -1 in worker hard kill ()
Selwin Ong 43fb21b827 Show job scheduling section on docs website
Levin Rickert 0080b223fe
Fix typos in job_registries.md ()
Selwin Ong f18a951eec Bump version to 1.2.2
Selwin Ong fda4b35f46
Fixes Job.fetch when return value is unpickleable ()
* Fixes Job.fetch when return value is unpickleable

* Fixed connection test in newer versions of Redis
Selwin Ong 90e032cd64 Bump version to 1.2.1
Ivan Kiryanov ed67de22c6 Add job status setting in enqueue_at (and in enqueue_in) methods ()
* Add job status setting in enqueue_at (and in enqueue_in) methods

Update tests for this change
Closes: 

* Add status param to create_job func, rework enqueue_at status setting
Selwin Ong ccfd4a02cb
Failed jobs will now auto expire ()
Selwin Ong 927fb5a3ed Bump version to 1.2.0
mr-trouble 5f949f4cef Add a hard kill from the parent process with a 10% increased timeout … ()
* Add a hard kill from the parent process with a 10% increased timeout in case the forked process gets stuck and cannot stop itself.

* Added test for the force kill of the parent process.

* Changed 10% to +1 second, and other misc changes based on review comments.
Florian Perucki 37a6304a4f fix: use correct logging handler for scheduler ()
Selwin Ong baa0cc268a
Job scheduling ()
* First RQScheduler prototype

* WIP job scheduling

* Fixed Python 2.7 tests

* Added ScheduledJobRegistry.get_scheduled_time(job)

* WIP on scheduler's threading mechanism

* Fixed test errors

* Changed scheduler.acquire_locks() to instance method

* Added scheduler.prepare_registries()

* Somewhat working implementation of RQ scheduler

* Only call stop_scheduler if there's a scheduler present

* Use OSError rather than ProcessLookupError for PyPy compatibility

* Added `auto_start` argument to scheduler.acquire_locks()

* Make RQScheduler play better with timezone

* Fixed test error

* Added --with-scheduler flag to rq worker CLI

* Fix tests on Python 2.x

* More Python 2 fixes

* Only call `scheduler.start` if worker is run in non burst mode

* Fixed an issue where running worker with scheduler would fail sometimes

* Make `worker.stop_scheduler()` more resilient to errors

* worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks

* Scheduler can now work with worker in both burst and non burst mode

* Fixed scheduler logging message

* Always log scheduler errors when running

* Improve scheduler error logging message

* Removed testing code

* Scheduler should periodically try to acquire locks for other queues it doesn't have

* Added tests for scheduler.should_reacquire_locks

* Added queue.enqueue_in()

* Fixes queue.enqueue_in() in Python 2.7

* First stab at documenting job scheduling

* Remove unused methods

* Remove Python 2.6 logging compatibility code

* Remove more unused imports

* Added convenience methods to access job registries from queue

* Added test for worker.run_maintenance_tasks()

* Simplify worker.queue_names() and worker.queue_keys()

* Updated changelog to mention RQ's new job scheduling mechanism.
Christian Clauss f09d4db080 Travis CI: sudo: deprecated and Xenial is default ()
goldstar611 b960ad53e6 Add example for unix:// ()
Selwin Ong a06421e125 Updated changelog
Selwin Ong a407bcfe36 Updated changelog
Yongtao Zhang 5bb5fda1a6 fix log level parameter override, Fixes ()
Selwin Ong cf50151853 Added a note to indicate that `registry.remove(job, delete_job=True)` is new
Thomas Matecki 80c82f731f Multi Dependency Support - Registration & Enqueue Call ()
* Multi Dependency Support - Registration & Enqueue Call

Internal API changes to support multiple dependencies.
* Store all of a job's _dependencies_ in a redis set. Delete that set when a job is deleted.
* Add Job#fetch_dependencies method - which return all jobs a job is dependent upon and optionally _WATCHES_ all dependency ids.
* Use Job#fetch_dependencies in Queue#call_enqueue. `fetch_dependencies` now sets WATCH and raises InvalidJobDependency, rather than call_enqueue.

`Queue` and `Job` public APIs still expect single ids of jobs for `depends_on` but internally register them in a way that could support multiple jobs being passed as dependencies.

Next up: need to update Queue#enqueue_dependents

* Use existing fetch_many method to get dependencies.

Modify fetch_dependencies to use fetch_many.

* Remove default value for fetch_many's connection parameter

* PR review housekeeping

* Remove a duplicate test
* Oneline something
* Fix missing colon in dependencies key
* Delete job key, dependents and dependencies at once

* More Fixes From Code Review

Updates to Job, Queue and associated tests.

* When Checking dependencies Avoid, trip to Redis

* When checking the status of a job, we have a 'clean' status of all dependencies(returned from Job#fetch_dependencies) and the job keys are WATCHed, so there's no reason to go back to Redis to get the status _again_.
* Looks as though, the `_status` set in `Job#restore` was bytes while it was converted to text(`as_text`) in `Job#get_status` - for consistency(and tests) converting to text in `restore` as well.
* In `Queue#enqueue_call`, moved WATCH of dependencies_key to before fetching dependencies. This doesn't really matter but seems more _correct_ - one can imagine some rogue API adding a dependency after they've been fetched but before they've been WATCHEed.

* Update Job#get_status to get _local_ status

* If refresh=False is passed, don't get status from Redis; return the value of _status. This is to avoid a trip to Redis if the caller can guarantee that the value of `_status` is _clean_.

* More Fixups

* Expire dependency keys in Job#cleanup
* Consistency in Job#fetch_dependencies
Selwin Ong af678243e1
Added `delete_job` argument to registry.remove()` ()
Yongtao Zhang 5bb03b9c2c Fix rq info not found workers information error, Fixes ()
* Fix rq info not found workers information error, Fixes 

* Add test()