* Update cli.py to support custom loggers
Allows a config.py file (via rq worker --config) to support things like this for logfmt logging:
DICT_CONFIG = {
"version": 1,
"formatters": {"logfmt": {
"()": "logfmter.Logfmter",
"keys": ["level","when","pid"],
"mapping": {"level":"levelname","when":"asctime","pid":"process"},
"datefmt": "%Y-%m-%dT%H:%M:%S%z"
}},
"handlers": {"console": {"class": "logging.StreamHandler","formatter": "logfmt" }},
"loggers": {
"root": {"handlers":["console"], "level": "INFO"},
"rq": {"handlers":["console"], "level": "INFO", "propagate": False},
}
}
* added simple test and documentation for DICT_CONFIG
* further attempt to get testing right for dictConfig
* move import to correct location
* fix
* remove meaningless options.get() usage
* linting checks and added missing test config file
* Scheduler should release and heartbeat only acquired locks.
* Added tests for heartbeat and release only acquired locks.
* Changed test description to correct one.
* Update linting configuration
This removes flake8 in favor of ruff, which also provides isort support, and
updates all files to be black, isort, and ruff compliant. This also adds black
and ruff checks to the tox and Github linting workflow.
* Tweak the code coverage config and calls
* Don't fail CI if codecov fails
* Remove codecov from dev-requirements.txt
* Pin packaging dependency to 21.3
* Pin coverage to 6.5.0
* Pin coverage to 6.2
* Downgrade packaging
* Drop Python 3.6 from test matrix
* Reenable Python 3.6 in tests
* Downgrade coverage req
* Make a different dev-requirements file for Python 3.6
* Moved dev-requirements-36 to root dir
* First stab at implementating worker pool
* Use process.is_alive() to check whether a process is still live
* Handle shutdown signal
* Check worker loop done
* First working version of `WorkerPool`.
* Added test for check_workers()
* Added test for pool.start()
* Better shutdown process
* Comment out test_start() to see if it fixes CI
* Make tests pass
* Make CI pass
* Comment out some tests
* Comment out more tests
* Re-enable a test
* Re-enable another test
* Uncomment check_workers test
* Added run_worker test
* Minor modification to dead worker detection
* More test cases
* Better process name for workers
* Added back pool.stop_workers() when signal is received
* Cleaned up cli.py
* WIP on worker-pool command
* Fix test
* Test that worker pool ignores consecutive shutdown signals
* Added test for worker-pool CLI command.
* Added timeout to CI jobs
* Fix worker pool test
* Comment out test_scheduler.py
* Fixed worker-pool in burst mode
* Increase test coverage
* Exclude tests directory from coverage.py
* Improve test coverage
* Renamed `Pool(num_workers=2) to `Pool(size=2)`
* Revert "Renamed `Pool(num_workers=2) to `Pool(size=2)`"
This reverts commit a1306f89ad0d8686c6bde447bff75e2f71f0733b.
* Renamed Pool to WorkerPool
* Added a new TestCase that doesn't use LocalStack
* Added job_class, worker_class and serializer arguments to WorkerPool
* Use parse_connection() in WorkerPool.__init__
* Added CLI arguments for worker-pool
* Minor WorkerPool and test fixes
* Fixed failing CLI test
* Document WorkerPool
* Added BaseWorker class
* Moved logging related functions to logutils
* Remove uneeded colorize function
* Updated worker.get_current_job_id() to not fail when job ID is None
* job.restore() shouldn't crash if is not present
* Fix various as_text() related crashes
* Added parse_connection function
* feat: allow custom connection pool class (#1885)
* Added test for SSL
---------
Co-authored-by: Cyril Chapellier <tchapi@users.noreply.github.com>
* improve the docs on in-flight jobs of killed/lost workers
* improve the docs on in-flight jobs of killed/lost workers
* document abandoned job behavior before 0.14
* consolidate job failure callback execution
* fix
* success callback as well
* merge fix
* create Callback class and change how callbacks serde
* deprecation
* add tests
* string format
* pr fix
* revert serialization changes
* fix timeout typing
* add documentation
* add test
* fix a bug
* fix tests
* move job heartbeat call to worker and make sure there is always a callback timeout
* fix test
* fix test
* test: check dependencies when enqueue via Queue.enqueue_job()
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix: check dependencies when enqueue via Queue.enqueue_job()
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
---------
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
* Updated arguments passed to the Sentinel Object when created from the settings
- added `USERNAME` key
- added `CONNECTION_KWARGS` key to allow passing additionals arguments to the Redis connections
- updated the documentation
* added missing comma
* tests(helpers): Added tests for Sentinel
---------
Co-authored-by: Joachim Burket <joachim.burket@hopitalvs.ch>
* Add started job registry cleanup job failure callback call
* WIP - need to fix test
* fix test
* rename, tests and docs
* better log message
* use class name
* Update registry.py
* Fix TimerDeathPenalty not properly handling negative/infinite timeout
* revert back to using exc_info
---------
Co-authored-by: Marcus <marcus@us2.ai>
* test: Dependency list in depends_on
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix: Dependency list in depends_on
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
---------
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix accessing None when dequeued result is None (burst=True, or timeout=None)
* add a test
* implement + tests
* fix if
* adjust test
* merge
* test
* test
* merge master
* take max_idle_time into account for dequeue_timeout
* refactor a bit
* potential bug fix
* tests
* math.ceil
* buffer tests
juur
xzander
Rob Hudson
dependabot[bot]
Tom Hamilton Stubber
Cyril Chapellier
Selwin Ong
gabriels1234
nhairs
luto
Tianci Hu Marrero
Joris
Rony Lutsky
Jochen Kupperschmidt
Simó Albert i Beltran
Jesse P. Johnson
lowercase00
Joachim Burket
Marcus Ong
amks1