1734 Commits (64e202ea1919ce6a3e356b0aa2c88383dd1ac430)
 

Author SHA1 Message Date
lowercase00 0f5f18bec7
Allow for configurable scheduler fallback period (#1759)
* Allow for configurable scheduler fallback period

This adds a parameter to the Scheduler called
`fallback_period`, which determines the period
before work fallsback to a new scheduler.

* Update defaults.py

Reduce Scheduler Fallback Period to 120 seconds.

* Update scheduler.py

Remove fallback period parameter.
2 years ago
lowercase00 9c2d353640
Remove unused dummy tasks (#1760)
Dummy tasks were probably used for load testing
in the past, but currently are not being imported from anywhere.
Also, similar task utilities are already present on the tests fixtures,
making the `dummy` module irrelevant.
2 years ago
lowercase00 e163e224e2
Bugfix: Python & RQ Version (#1757)
* Fix min version on `setup.py`

* Fix rq version
2 years ago
Selwin Ong 8f4e51b7a4 Bump version to 1.12.0 2 years ago
dependabot[bot] fa7a635aa4
Bump actions/setup-python from 4.4.0 to 4.5.0 (#1750)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.4.0 to 4.5.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.4.0...v4.5.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Selwin Ong d0e832181f
Don't log job exceptions twice (#1746)
* Don't log job exceptions twice

* Preserve traceback while encountering deserialization error
2 years ago
dependabot[bot] 62acd56c91
Bump actions/setup-python from 4.3.0 to 4.4.0 (#1745)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.3.0 to 4.4.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.3.0...v4.4.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Oscar Arbeláez-Echeverri 9aaceb22e6
Cache redis version in the connection object (#1742) 2 years ago
gabriels1234 50d8d72928
Add at_front for scheduled/enqueue_at jobs (#1743)
* Add at_front for scheduled/enqueue_at jobs

* Add test to at_front
2 years ago
Ankush Menat 5f7ed6970d
Move lock aquire log from INFO to DEBUG level (#1737)
closes https://github.com/rq/rq/issues/1717
2 years ago
eswolinsky3241 5119716911
Persist worker_name after job is finished (#1730)
* Persist worker_name after job is finished

Persisting the worker_name on the job object in Redis would allow for debugging and
analyzing logs from the worker

* Remove redundent job.save() method call

* Remove check for null worker

Now that worker name is persisted after job finishes or fails,
no need to assert that worker name is None

* Change github runner to Ubuntu 20.04

* Change github runner to Ubuntu 20.04
2 years ago
Selwin Ong f39554dbe4
Make test_commands pass on MacOS (#1735) 2 years ago
Krzysztof Jeziorny d108eaf235
#1733 Typography (#1734)
html font size removed, body font size set in rem w/o weight;
font stacks defined;
code formatted
2 years ago
Selwin Ong 0691b4d46e
Multiple results using Redis Streams (#1725)
* WIP job results

* Result can now be saved

* Successfully saved and restored result

* result.save() should accept pipeline

* Successful results are saved

* Failures are now saved properly too.

* Added test for Result.get_latest()

* Checkpoint

* Got Result.all() to work

* Added Result.count(), Result.delete()

* Backward compatibility for job.result and job.exc_info

* Added some typing

* More typing stuff

* Fixed typing in job.py

* More typing updates

* Only keep the last 10 results

* Documented job.results()

* Got results test to pass

* Don't run test_results.py on Redis server < 5.0

* Fixed mock import on some Python versions

* Remove Redis 3 from test matrix

* Jobs should never use the new Result implementation if server is < 5.0

* Results should only be created is Redis stream is supported.

* Added back Redis 3 to test matrix

* Fixed job.supports_redis_streams

* Fixed worker test

* Updated docs.
2 years ago
Jan Rüegg 09856f9924
pre-warm redis server version cache of job from queue (#1727)
Co-authored-by: Jan Rüegg <jrueegg@apple.com>
2 years ago
dependabot[bot] 67432ec8be
Bump actions/setup-python from 4.2.0 to 4.3.0 (#1718)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Jan Rüegg 4750eb4316
Fix broken pipeline / transaction (#1715)
Co-authored-by: Jan Rüegg <jrueegg@apple.com>
2 years ago
lowercase00 375ace1747
Typing (#1698)
* Gitignore Venv + VScode

* Add Typings, Add Test to Makefile

* Fix, More typing, Redis Pipeline specific type

* More types

* Fix 3.7- Typing Compat, Add Tox Envs, Tests Dockerfile

* fix listindex error (#1700)

* More docstrings

* More Types

* Fix Typo on Dependency

* Last Types

Co-authored-by: Burak Yılmaz <46003469+yilmaz-burak@users.noreply.github.com>
2 years ago
Selwin Ong 48e821ecd0 Add packages write permission to docker workflow 2 years ago
Selwin Ong 7b6250c7ef
Remove Python 3.5 from CI. (#1712) 2 years ago
Alex 02d5cf297b
GitHub Workflows security hardening (#1711)
* build: harden docker.yml permissions

Signed-off-by: Alex <aleksandrosansan@gmail.com>

* build: harden workflow.yml permissions

Signed-off-by: Alex <aleksandrosansan@gmail.com>

Signed-off-by: Alex <aleksandrosansan@gmail.com>
2 years ago
Selwin Ong 80c0edc232 Bump version to 1.11.1 2 years ago
Jahn Thomas Fidje 8404385592
Add feature to enqueue dependents at the front of queues (#1696)
* Add feature to enqueue dependents at the front of queues

* Add documentation for the Dependency(enqueue_at_front=...) parameter

* docs: Add `enqueue_at_front` to list of parameters for Dependency

* test: Update dependency test to not rely on Redis ordering

* refactor: Save enqueue_at_front boolean in job.meta instead of separate instance attr

* fix: Made enqueue_at_front an instance attribute instead of putting it inside meta
2 years ago
Miles Winter 108c2ea666
scheduler heart for acquired locks only (#1710) 2 years ago
Alex Garel fd116b33db
doc: default queues order is a priority order (#1704)
* doc: default queues order is a priority order


In introduction, we say that queue order matters, but we are not very specific.
Added a paragraph to explicit that.

* doc: fix typo
2 years ago
Rony Lutsky 0c47a8edf4
fix log (#1709) 2 years ago
Burak Yılmaz 153d29cce7
fix listindex error (#1700) 2 years ago
Selwin Ong 8e3283dab3
Job should not be enqueued if dependency is canceled (#1695)
* Fix job.dependencies_are_met() if dependency is canceled

* Slightly better test coverage on dependencies_are_met()

* Fixed job.cancel(enqueue_dependent=True)
2 years ago
Yang Yang 9db728921d
Improve the lint situation (#1688)
* Move common flake8 options into config file

Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.

Move --count and --statistics to config file as well.

* Fix some lints
2 years ago
dependabot[bot] 01635dc809
Bump actions/setup-python from 4.1.0 to 4.2.0 (#1687)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.1.0...v4.2.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Yang Yang 3f59a7e6e3
Support callbacks in enqueue_many (#1686)
The on_success and on_failure callbacks are currently supported in
Queue.enqueue but not Queue.enqueue_many, despite documentary indicating
the contrary.

This commit brings the bulk enqueuing API back into parity.

Fixes #1680
2 years ago
Selwin Ong 98c200d07a Bump version to 1.11.0 2 years ago
Selwin Ong c0bb0ba4e6 Merge branch 'master' of github.com:rq/rq 2 years ago
Selwin Ong d08cc72842
Added Redis 7 to test matrix (#1683)
* Updated changelog.

* Added Redis 7 and redis-py 4.3 to test matrix

* Addd Redis 7 to test matrix

* Addd Redis 7 to test matrix
2 years ago
Selwin Ong 08377698ab Updated changelog. 2 years ago
Selwin Ong bc5bb1fba2
Revert "Jobs that are run synchronously should always raise an exception" (#1682)
This reverts commit 0d21e714c33cfefba51a8c2b714a735bcad1264d.
2 years ago
Selwin Ong 5b95725dc4
Dependency with failures (#1681)
* added Dependency class with allow_failures

* Requested changes

* Check type before setting `job.dependency_allow_fail` within `Job.create`

* Set `job.dependency_allow_fail` within `Job.create`

* Added test to ensure persistence of `dependency_allow_fail`

* Removed typing and allow mixed list of ints and Job objects

* Convert dependency_allow_fail boolean to integer during serialization to avoid redis DataError

* Updated `test_multiple_dependencies_are_accepted_and_persisted` test to include `Dependency` cases

* Adding placeholder test to test actual behavior of new `Dependency` usage in `depends_on`

* Updated `test_job_dependency` to include cases using `Dependency`

* Added dependency_allow_fail logic to `Job.restore`

* Renamed `dependency_allow_fail` to a simpler `allow_failure`

* Update docs to add section about the new `Dependency` class and use-case

* Updated `Job.dependencies_are_met` logic to take `FAILED` and `STOPPED` jobs into account when `allow_failure=True`

* Updated `test_job_dependency` test. Still failing with `Dependency` case.

* Fix `allow_failure` type coercion in `Job.restore`

* Re-arrange tests, so default `Dependency.allow_failure` is before explicit `allow_failure=True`

* Fixed Dependency, so it works correctly when allow_failure=True

* Attempt to execute pipeline prior to queueing a failed job's dependents. test_create_and_cancel_job_enqueue_dependents_in_registry test now passes.

* Added `Depedency` test utilizing multiple dependencies

* Removed irrelevant on_success and on_failure keyword arguments in example

* Replaced use of long_running_job

* Add test to verify `Dependency.jobs` contraints

* Suppress connection error in handle_job_failure

* test_dependencies have passed

* All tests pass if enqueue_dependents called without pipeline.watch()

* All tests now pass

* Removed print statements

* Cleanup Dependency implementation

* Renamed job.allow_failure to job.allow_dependency_failures

Co-authored-by: mattchan <mattchan@tencent.com>
Co-authored-by: Mike Hill <mhilluniversal@gmail.com>
2 years ago
Tim Gates d82af1469f
docs: Fix a few typos (#1679)
There are small typos in:
- docs/docs/exceptions.md
- docs/docs/jobs.md
- rq/queue.py
- tests/fixtures.py
- tests/test_job.py

Fixes:
- Should read `slightly` rather than `slighty`.
- Should read `requeuing` rather than `requeueing`.
- Should read `implementers` rather than `implementors`.
- Should read `definition` rather than `defition`.
- Should read `canceled` rather than `canceld`.

Signed-off-by: Tim Gates <tim.gates@iress.com>
2 years ago
Selwin Ong 145884fcd5
When stopped jobs are deleted, they should also be removed from FailedJobRegistry. (#1677) 2 years ago
dependabot[bot] 468eb00c13
Bump actions/setup-python from 3 to 4.1.0 (#1675)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.1.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4.1.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Hugo d5175c38da
Drop python2-specific syntax (#1674)
* Drop syntax required only for Python 2

* Drop python2-style super() calls

Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
2 years ago
Hugo 61a4a1720b
Use unittest.mock instead of mock (#1673)
This module has been included in Python itself since 3.3.

Fixes: https://github.com/rq/rq/issues/1646
2 years ago
Selwin Ong 87bacb650e
Jobs that are run synchronously should always raise an exception (#1671) 3 years ago
Naveen aff877cfcf
chore: Enable codeql action (#1666)
This action runs GitHub's industry-leading semantic code analysis engine,
CodeQL, against a repository's source code to find security vulnerabilities.

https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning-with-codeql

https://github.com/ossf/scorecard/blob/main/docs/checks.md#sast
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
3 years ago
Pavel Levchuk aff862dd6a
Fix syntax error in docs example (#1664)
Adds a missing colon in function arguments
3 years ago
dependabot[bot] a69498e185
Bump actions/setup-python from 2 to 3 (#1658)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Steve Kreitzer ecf0d06090
Support at_front requeue support (#1657)
Co-authored-by: Steve Kreitzer <skreitzer@cloudflare.com>
3 years ago
dependabot[bot] 48df759879
Bump codecov/codecov-action from 1 to 3 (#1660)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] a2ada2d1ec
Bump supercharge/redis-github-action from 1.1.0 to 1.4.0 (#1661)
Bumps [supercharge/redis-github-action](https://github.com/supercharge/redis-github-action) from 1.1.0 to 1.4.0.
- [Release notes](https://github.com/supercharge/redis-github-action/releases)
- [Changelog](https://github.com/supercharge/redis-github-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/supercharge/redis-github-action/compare/1.1.0...1.4.0)

---
updated-dependencies:
- dependency-name: supercharge/redis-github-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] ae0611b967
Bump actions/upload-artifact from 2 to 3 (#1663)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago