6 Commits (bbfeb8c8ed2913e980412ee7606c171fdf37e713)

Author SHA1 Message Date
Simó Albert i Beltran 64e202ea19
Check dependencies when enqueue via Queue.enqueue_job() (#1837)
* test: check dependencies when enqueue via Queue.enqueue_job()

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>

* fix: check dependencies when enqueue via Queue.enqueue_job()

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
Co-authored-by: Selwin Ong <selwin.ong@gmail.com>

---------

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
2 years ago
Simó Albert i Beltran ed59b9248a
fix: Dependency list in depends_on (#1843)
* test: Dependency list in depends_on

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>

* fix: Dependency list in depends_on

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>

---------

Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
2 years ago
lowercase00 375ace1747
Typing (#1698)
* Gitignore Venv + VScode

* Add Typings, Add Test to Makefile

* Fix, More typing, Redis Pipeline specific type

* More types

* Fix 3.7- Typing Compat, Add Tox Envs, Tests Dockerfile

* fix listindex error (#1700)

* More docstrings

* More Types

* Fix Typo on Dependency

* Last Types

Co-authored-by: Burak Yılmaz <46003469+yilmaz-burak@users.noreply.github.com>
2 years ago
Jahn Thomas Fidje 8404385592
Add feature to enqueue dependents at the front of queues (#1696)
* Add feature to enqueue dependents at the front of queues

* Add documentation for the Dependency(enqueue_at_front=...) parameter

* docs: Add `enqueue_at_front` to list of parameters for Dependency

* test: Update dependency test to not rely on Redis ordering

* refactor: Save enqueue_at_front boolean in job.meta instead of separate instance attr

* fix: Made enqueue_at_front an instance attribute instead of putting it inside meta
2 years ago
Selwin Ong 8e3283dab3
Job should not be enqueued if dependency is canceled (#1695)
* Fix job.dependencies_are_met() if dependency is canceled

* Slightly better test coverage on dependencies_are_met()

* Fixed job.cancel(enqueue_dependent=True)
2 years ago
Selwin Ong 5b95725dc4
Dependency with failures (#1681)
* added Dependency class with allow_failures

* Requested changes

* Check type before setting `job.dependency_allow_fail` within `Job.create`

* Set `job.dependency_allow_fail` within `Job.create`

* Added test to ensure persistence of `dependency_allow_fail`

* Removed typing and allow mixed list of ints and Job objects

* Convert dependency_allow_fail boolean to integer during serialization to avoid redis DataError

* Updated `test_multiple_dependencies_are_accepted_and_persisted` test to include `Dependency` cases

* Adding placeholder test to test actual behavior of new `Dependency` usage in `depends_on`

* Updated `test_job_dependency` to include cases using `Dependency`

* Added dependency_allow_fail logic to `Job.restore`

* Renamed `dependency_allow_fail` to a simpler `allow_failure`

* Update docs to add section about the new `Dependency` class and use-case

* Updated `Job.dependencies_are_met` logic to take `FAILED` and `STOPPED` jobs into account when `allow_failure=True`

* Updated `test_job_dependency` test. Still failing with `Dependency` case.

* Fix `allow_failure` type coercion in `Job.restore`

* Re-arrange tests, so default `Dependency.allow_failure` is before explicit `allow_failure=True`

* Fixed Dependency, so it works correctly when allow_failure=True

* Attempt to execute pipeline prior to queueing a failed job's dependents. test_create_and_cancel_job_enqueue_dependents_in_registry test now passes.

* Added `Depedency` test utilizing multiple dependencies

* Removed irrelevant on_success and on_failure keyword arguments in example

* Replaced use of long_running_job

* Add test to verify `Dependency.jobs` contraints

* Suppress connection error in handle_job_failure

* test_dependencies have passed

* All tests pass if enqueue_dependents called without pipeline.watch()

* All tests now pass

* Removed print statements

* Cleanup Dependency implementation

* Renamed job.allow_failure to job.allow_dependency_failures

Co-authored-by: mattchan <mattchan@tencent.com>
Co-authored-by: Mike Hill <mhilluniversal@gmail.com>
2 years ago