1772 Commits (933860ee8cab780df406e53e412bbe306a9b0255)
 

Author SHA1 Message Date
Krzysztof Jeziorny d108eaf235
#1733 Typography (#1734)
html font size removed, body font size set in rem w/o weight;
font stacks defined;
code formatted
2 years ago
Selwin Ong 0691b4d46e
Multiple results using Redis Streams (#1725)
* WIP job results

* Result can now be saved

* Successfully saved and restored result

* result.save() should accept pipeline

* Successful results are saved

* Failures are now saved properly too.

* Added test for Result.get_latest()

* Checkpoint

* Got Result.all() to work

* Added Result.count(), Result.delete()

* Backward compatibility for job.result and job.exc_info

* Added some typing

* More typing stuff

* Fixed typing in job.py

* More typing updates

* Only keep the last 10 results

* Documented job.results()

* Got results test to pass

* Don't run test_results.py on Redis server < 5.0

* Fixed mock import on some Python versions

* Remove Redis 3 from test matrix

* Jobs should never use the new Result implementation if server is < 5.0

* Results should only be created is Redis stream is supported.

* Added back Redis 3 to test matrix

* Fixed job.supports_redis_streams

* Fixed worker test

* Updated docs.
2 years ago
Jan Rüegg 09856f9924
pre-warm redis server version cache of job from queue (#1727)
Co-authored-by: Jan Rüegg <jrueegg@apple.com>
2 years ago
dependabot[bot] 67432ec8be
Bump actions/setup-python from 4.2.0 to 4.3.0 (#1718)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Jan Rüegg 4750eb4316
Fix broken pipeline / transaction (#1715)
Co-authored-by: Jan Rüegg <jrueegg@apple.com>
2 years ago
lowercase00 375ace1747
Typing (#1698)
* Gitignore Venv + VScode

* Add Typings, Add Test to Makefile

* Fix, More typing, Redis Pipeline specific type

* More types

* Fix 3.7- Typing Compat, Add Tox Envs, Tests Dockerfile

* fix listindex error (#1700)

* More docstrings

* More Types

* Fix Typo on Dependency

* Last Types

Co-authored-by: Burak Yılmaz <46003469+yilmaz-burak@users.noreply.github.com>
2 years ago
Selwin Ong 48e821ecd0 Add packages write permission to docker workflow 2 years ago
Selwin Ong 7b6250c7ef
Remove Python 3.5 from CI. (#1712) 2 years ago
Alex 02d5cf297b
GitHub Workflows security hardening (#1711)
* build: harden docker.yml permissions

Signed-off-by: Alex <aleksandrosansan@gmail.com>

* build: harden workflow.yml permissions

Signed-off-by: Alex <aleksandrosansan@gmail.com>

Signed-off-by: Alex <aleksandrosansan@gmail.com>
2 years ago
Selwin Ong 80c0edc232 Bump version to 1.11.1 2 years ago
Jahn Thomas Fidje 8404385592
Add feature to enqueue dependents at the front of queues (#1696)
* Add feature to enqueue dependents at the front of queues

* Add documentation for the Dependency(enqueue_at_front=...) parameter

* docs: Add `enqueue_at_front` to list of parameters for Dependency

* test: Update dependency test to not rely on Redis ordering

* refactor: Save enqueue_at_front boolean in job.meta instead of separate instance attr

* fix: Made enqueue_at_front an instance attribute instead of putting it inside meta
2 years ago
Miles Winter 108c2ea666
scheduler heart for acquired locks only (#1710) 2 years ago
Alex Garel fd116b33db
doc: default queues order is a priority order (#1704)
* doc: default queues order is a priority order


In introduction, we say that queue order matters, but we are not very specific.
Added a paragraph to explicit that.

* doc: fix typo
2 years ago
Rony Lutsky 0c47a8edf4
fix log (#1709) 2 years ago
Burak Yılmaz 153d29cce7
fix listindex error (#1700) 2 years ago
Selwin Ong 8e3283dab3
Job should not be enqueued if dependency is canceled (#1695)
* Fix job.dependencies_are_met() if dependency is canceled

* Slightly better test coverage on dependencies_are_met()

* Fixed job.cancel(enqueue_dependent=True)
2 years ago
Yang Yang 9db728921d
Improve the lint situation (#1688)
* Move common flake8 options into config file

Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.

Move --count and --statistics to config file as well.

* Fix some lints
2 years ago
dependabot[bot] 01635dc809
Bump actions/setup-python from 4.1.0 to 4.2.0 (#1687)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4.1.0...v4.2.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Yang Yang 3f59a7e6e3
Support callbacks in enqueue_many (#1686)
The on_success and on_failure callbacks are currently supported in
Queue.enqueue but not Queue.enqueue_many, despite documentary indicating
the contrary.

This commit brings the bulk enqueuing API back into parity.

Fixes #1680
2 years ago
Selwin Ong 98c200d07a Bump version to 1.11.0 2 years ago
Selwin Ong c0bb0ba4e6 Merge branch 'master' of github.com:rq/rq 2 years ago
Selwin Ong d08cc72842
Added Redis 7 to test matrix (#1683)
* Updated changelog.

* Added Redis 7 and redis-py 4.3 to test matrix

* Addd Redis 7 to test matrix

* Addd Redis 7 to test matrix
2 years ago
Selwin Ong 08377698ab Updated changelog. 2 years ago
Selwin Ong bc5bb1fba2
Revert "Jobs that are run synchronously should always raise an exception" (#1682)
This reverts commit 0d21e714c33cfefba51a8c2b714a735bcad1264d.
2 years ago
Selwin Ong 5b95725dc4
Dependency with failures (#1681)
* added Dependency class with allow_failures

* Requested changes

* Check type before setting `job.dependency_allow_fail` within `Job.create`

* Set `job.dependency_allow_fail` within `Job.create`

* Added test to ensure persistence of `dependency_allow_fail`

* Removed typing and allow mixed list of ints and Job objects

* Convert dependency_allow_fail boolean to integer during serialization to avoid redis DataError

* Updated `test_multiple_dependencies_are_accepted_and_persisted` test to include `Dependency` cases

* Adding placeholder test to test actual behavior of new `Dependency` usage in `depends_on`

* Updated `test_job_dependency` to include cases using `Dependency`

* Added dependency_allow_fail logic to `Job.restore`

* Renamed `dependency_allow_fail` to a simpler `allow_failure`

* Update docs to add section about the new `Dependency` class and use-case

* Updated `Job.dependencies_are_met` logic to take `FAILED` and `STOPPED` jobs into account when `allow_failure=True`

* Updated `test_job_dependency` test. Still failing with `Dependency` case.

* Fix `allow_failure` type coercion in `Job.restore`

* Re-arrange tests, so default `Dependency.allow_failure` is before explicit `allow_failure=True`

* Fixed Dependency, so it works correctly when allow_failure=True

* Attempt to execute pipeline prior to queueing a failed job's dependents. test_create_and_cancel_job_enqueue_dependents_in_registry test now passes.

* Added `Depedency` test utilizing multiple dependencies

* Removed irrelevant on_success and on_failure keyword arguments in example

* Replaced use of long_running_job

* Add test to verify `Dependency.jobs` contraints

* Suppress connection error in handle_job_failure

* test_dependencies have passed

* All tests pass if enqueue_dependents called without pipeline.watch()

* All tests now pass

* Removed print statements

* Cleanup Dependency implementation

* Renamed job.allow_failure to job.allow_dependency_failures

Co-authored-by: mattchan <mattchan@tencent.com>
Co-authored-by: Mike Hill <mhilluniversal@gmail.com>
2 years ago
Tim Gates d82af1469f
docs: Fix a few typos (#1679)
There are small typos in:
- docs/docs/exceptions.md
- docs/docs/jobs.md
- rq/queue.py
- tests/fixtures.py
- tests/test_job.py

Fixes:
- Should read `slightly` rather than `slighty`.
- Should read `requeuing` rather than `requeueing`.
- Should read `implementers` rather than `implementors`.
- Should read `definition` rather than `defition`.
- Should read `canceled` rather than `canceld`.

Signed-off-by: Tim Gates <tim.gates@iress.com>
2 years ago
Selwin Ong 145884fcd5
When stopped jobs are deleted, they should also be removed from FailedJobRegistry. (#1677) 2 years ago
dependabot[bot] 468eb00c13
Bump actions/setup-python from 3 to 4.1.0 (#1675)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.1.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4.1.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Hugo d5175c38da
Drop python2-specific syntax (#1674)
* Drop syntax required only for Python 2

* Drop python2-style super() calls

Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
2 years ago
Hugo 61a4a1720b
Use unittest.mock instead of mock (#1673)
This module has been included in Python itself since 3.3.

Fixes: https://github.com/rq/rq/issues/1646
2 years ago
Selwin Ong 87bacb650e
Jobs that are run synchronously should always raise an exception (#1671) 3 years ago
Naveen aff877cfcf
chore: Enable codeql action (#1666)
This action runs GitHub's industry-leading semantic code analysis engine,
CodeQL, against a repository's source code to find security vulnerabilities.

https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning-with-codeql

https://github.com/ossf/scorecard/blob/main/docs/checks.md#sast
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
3 years ago
Pavel Levchuk aff862dd6a
Fix syntax error in docs example (#1664)
Adds a missing colon in function arguments
3 years ago
dependabot[bot] a69498e185
Bump actions/setup-python from 2 to 3 (#1658)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Steve Kreitzer ecf0d06090
Support at_front requeue support (#1657)
Co-authored-by: Steve Kreitzer <skreitzer@cloudflare.com>
3 years ago
dependabot[bot] 48df759879
Bump codecov/codecov-action from 1 to 3 (#1660)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] a2ada2d1ec
Bump supercharge/redis-github-action from 1.1.0 to 1.4.0 (#1661)
Bumps [supercharge/redis-github-action](https://github.com/supercharge/redis-github-action) from 1.1.0 to 1.4.0.
- [Release notes](https://github.com/supercharge/redis-github-action/releases)
- [Changelog](https://github.com/supercharge/redis-github-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/supercharge/redis-github-action/compare/1.1.0...1.4.0)

---
updated-dependencies:
- dependency-name: supercharge/redis-github-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] ae0611b967
Bump actions/upload-artifact from 2 to 3 (#1663)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] c503823ff8
Bump actions/download-artifact from 2 to 3 (#1662)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] 395fd1c51a
Bump actions/checkout from 2 to 3 (#1659)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Naveen 4875db7a9c
chore: Included githubactions in the dependabot config (#1656)
This should help with keeping the GitHub actions updated on new releases. This will also help with keeping it secure.

Dependabot helps in keeping the supply chain secure https://docs.github.com/en/code-security/dependabot

GitHub actions up to date https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot

https://github.com/ossf/scorecard/blob/main/docs/checks.md#dependency-update-tool
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
3 years ago
Naveen adfb355abe
chore: Set permissions for GitHub actions (#1648)
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
3 years ago
nevious 5eb2f0b9a4
added ssl support for sentinel connections (#1649)
Co-authored-by: Christian Schläppi <cschlaeppi@hosttech.ch>
3 years ago
urmikab f2a3e8d346
Update exceptions.md (#1655)
Added the import statement for Job (line 18)
3 years ago
Rajesh Khadka d2dcc2fb6b
fix typo (#1640) 3 years ago
Jannis Vajen 8877d0ff52
Update link to Celery website (#1638)
The old domain has been compromised. See https://github.com/celery/celeryproject/issues/52 for more information.
3 years ago
Mateusz Soszyński f23c134bfd
Docs: don't break links at the header (#1636) 3 years ago
Michael Hill f4602d30d5
Cross platform simple worker (#1629)
* Added CrossPlatformDeathPenalty that doesn't rely on signals

* Updated `SimpleWorker`'s `death_penalty_class` to utilize `CrossPlatformDeathPenalty` to allow use on Windows machines

* Changed `CrossPlatformDeathPenalty` to `TimerDeathPenalty`

* Removed overridden `death_penalty_class` in `SimpleWorker` until feature matures

* Added section in testing.md explaining how to utilize `SimpleWorker` on Windows OS

* Replaced usage of chatting with .format for python 3.5 compatibility

* Add tests for new timeout feature

* Explicitly set defaults.CALLBACK_TIMEOUT

* Implemented cross-thread method of raising errors by using ctypes

* Finished writing tests for new TimerDeathPenalty
3 years ago
Selwin Ong 609c068ef8
Add codecov.yml to ignore tests directory 3 years ago
Neptalí Piña 34e2427f67
job decorator allow on_failure and on_success parameters (#1626) (#1628)
Co-authored-by: neptali <neptali@plerk.io>
3 years ago