* Update linting configuration
This removes flake8 in favor of ruff, which also provides isort support, and
updates all files to be black, isort, and ruff compliant. This also adds black
and ruff checks to the tox and Github linting workflow.
* Tweak the code coverage config and calls
* Don't fail CI if codecov fails
* Remove codecov from dev-requirements.txt
* Pin packaging dependency to 21.3
* Pin coverage to 6.5.0
* Pin coverage to 6.2
* Downgrade packaging
* Drop Python 3.6 from test matrix
* Reenable Python 3.6 in tests
* Downgrade coverage req
* Make a different dev-requirements file for Python 3.6
* Moved dev-requirements-36 to root dir
* First stab at implementating worker pool
* Use process.is_alive() to check whether a process is still live
* Handle shutdown signal
* Check worker loop done
* First working version of `WorkerPool`.
* Added test for check_workers()
* Added test for pool.start()
* Better shutdown process
* Comment out test_start() to see if it fixes CI
* Make tests pass
* Make CI pass
* Comment out some tests
* Comment out more tests
* Re-enable a test
* Re-enable another test
* Uncomment check_workers test
* Added run_worker test
* Minor modification to dead worker detection
* More test cases
* Better process name for workers
* Added back pool.stop_workers() when signal is received
* Cleaned up cli.py
* WIP on worker-pool command
* Fix test
* Test that worker pool ignores consecutive shutdown signals
* Added test for worker-pool CLI command.
* Added timeout to CI jobs
* Fix worker pool test
* Comment out test_scheduler.py
* Fixed worker-pool in burst mode
* Increase test coverage
* Exclude tests directory from coverage.py
* Improve test coverage
* Renamed `Pool(num_workers=2) to `Pool(size=2)`
* Revert "Renamed `Pool(num_workers=2) to `Pool(size=2)`"
This reverts commit a1306f89ad0d8686c6bde447bff75e2f71f0733b.
* Renamed Pool to WorkerPool
* Added a new TestCase that doesn't use LocalStack
* Added job_class, worker_class and serializer arguments to WorkerPool
* Use parse_connection() in WorkerPool.__init__
* Added CLI arguments for worker-pool
* Minor WorkerPool and test fixes
* Fixed failing CLI test
* Document WorkerPool
* treewide: apply black style
This PR applied the black code style, adds it to the CI and README. The
changes look big, however no functional changed are applied at all.
The line length is set to 120, which is different from black
recommendation of 88. I would suggest to stick to the 88 recommendation
and adapt the flake8 check.
Add the `-S` parameter to `black` to keep single quotes.
Signed-off-by: Paul Spooren <mail@aparcar.org>
* README: add black badge
Help people to find the used code style.
Signed-off-by: Paul Spooren <mail@aparcar.org>
* CI: check codestyle via black
Automatically run the CI and check that the code style is still black.
Signed-off-by: Paul Spooren <mail@aparcar.org>
---------
Signed-off-by: Paul Spooren <mail@aparcar.org>
* Persist worker_name after job is finished
Persisting the worker_name on the job object in Redis would allow for debugging and
analyzing logs from the worker
* Remove redundent job.save() method call
* Remove check for null worker
Now that worker name is persisted after job finishes or fails,
no need to assert that worker name is None
* Change github runner to Ubuntu 20.04
* Change github runner to Ubuntu 20.04
* Move common flake8 options into config file
Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.
Move --count and --statistics to config file as well.
* Fix some lints
* use shutil.get_terminal_size instead of click.get_terminal_size()
resolves warning:
rq/cli/helpers.py:107: DeprecationWarning: 'click.get_terminal_size()' is deprecated and will be removed in Click 8.1. Use 'shutil.get_terminal_size()' instead.
termwidth, _ = click.get_terminal_size()
* remove StrictVersion from rq
* asyncio.get_event_loop -> asyncio.new_event_loop()
resolves warning:
tests/test_job.py::TestJob::test_create_job_with_async
rq/rq/job.py:839: DeprecationWarning: There is no current event loop
loop = asyncio.get_event_loop()
* Add python 3.10 to matrix
Co-authored-by: rpkak <rpkak@users.noreply.github.com>
* Make new workflow
Descriped in https://github.com/rq/rq/pull/1465#issuecomment-842464560
* For testing
* failing tests
For testing
* log file
For testing
* log
For testing
* no newlines
For testing
* no "
For testing
* no only one issue
For testing
* as job
For testing
* use artifacts
For testing
* use artifacts2
For testing
* use artifacts3
For testing
* use artifacts4
For testing
* finish
* name
* only if "normal" workflow doesn't fail
https://github.com/rq/rq/pull/1470#discussion_r641343532
* tests: updated github worklow for tests to use requirements.txt and dev-requirements.txt
* build: updated dev-requirements.txt
Co-authored-by: Ruslan Mullakhmetov <ruslan@twentythree.net>
* feat: avoided "zombie" processes after killing work horse by setting work horse process group and killing this group
* fixed tests
* tests: added test to check that all workhorse subprocesses are killed
* tests: updated guthub run tests dependencies since they are not using (dev-)requirements.txt
Co-authored-by: Ruslan Mullakhmetov <ruslan@twentythree.net>
* CI: replace travis with GitHub action matrix
The GitHub action runs the same tests as travis however using a more
complex matrix:
Python versions: 3.5, 3.6, 3.7, 3.8
Python 3.4 is dropped by redis-py, therefore not required in the tests.
Redis versions: 3, 4, 5, 6
The different Redis versions offer different features and this allows
checks for compatibility.
redis-py versions: 3.5.0, 3.5.3
3.5.0 is the oldest supported version, 3.5.3 is the latest upstream
verison
Signed-off-by: Paul Spooren <mail@aparcar.org>
* CI: Add flake8 lint action
This actions runs `flake8` and shows style problems of the code. It uses
th GitHub default options which handle most problems as warnings. These
rules could become slowly more strict.
Signed-off-by: Paul Spooren <mail@aparcar.org>
wyang
dependabot[bot]
Cyril Chapellier
Rob Hudson
Selwin Ong
Paul Spooren
lowercase00
eswolinsky3241
Alex
Yang Yang
Naveen
rpkak
dependabot-preview[bot]
Ruslan Mullakhmetov