* Fix TimerDeathPenalty not properly handling negative/infinite timeout
* revert back to using exc_info
---------
Co-authored-by: Marcus <marcus@us2.ai>
* test: Dependency list in depends_on
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix: Dependency list in depends_on
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
---------
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix accessing None when dequeued result is None (burst=True, or timeout=None)
* add a test
* implement + tests
* fix if
* adjust test
* merge
* test
* test
* merge master
* take max_idle_time into account for dequeue_timeout
* refactor a bit
* potential bug fix
* tests
* math.ceil
* buffer tests
* test: queue.result_ttl=-1
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* test: queue.result_ttl=0
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* fix: queue.result_ttl=-1
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
---------
Signed-off-by: Simó Albert i Beltran <sim6@bona.gent>
* treewide: apply black style
This PR applied the black code style, adds it to the CI and README. The
changes look big, however no functional changed are applied at all.
The line length is set to 120, which is different from black
recommendation of 88. I would suggest to stick to the 88 recommendation
and adapt the flake8 check.
Add the `-S` parameter to `black` to keep single quotes.
Signed-off-by: Paul Spooren <mail@aparcar.org>
* README: add black badge
Help people to find the used code style.
Signed-off-by: Paul Spooren <mail@aparcar.org>
* CI: check codestyle via black
Automatically run the CI and check that the code style is still black.
Signed-off-by: Paul Spooren <mail@aparcar.org>
---------
Signed-off-by: Paul Spooren <mail@aparcar.org>
When running on debug mode, some debug logs were
using colorizers on lists, which would raise an unhandled exception
on the worker. The same happened for a debug log that was
using colors on a response from Redis (bytes).
* Enhanced Redis Connection Reliability
The Redis connection may fail for several reasons. As the connection can be
(1) explicitly passed to the worker or (2) implicity set, this will improve the
Connection configuration by setting a timeout to the socket, and adding
an ExponentialBackoff Retry logic.
* Simpler Connection logic
* Add simple retry logic to Redis Connection Error
* Make retry exponential, add keepalive & socket_connect_timeout
* Handles configuration on Redis' connection pool
* Simplifies timeout exception logic
* Fix burst bug, add test
* Add docs related to `socket_timeout`, improve compatibility with older RedisPy versions
* Fixes
* New timeout private method
* Fix timeout
* Improve docstrings on `connections`
* Enhanced Job methods docstrings & Serialization Protocol
This adds docstrings to all Job methods in a standard format.
It also implements a `serializer` protocol.
* Excludes `Protocol` (keeping compatibility with < 3.8)
* Add docstrings & type annotation to the `job` decorator
* Docstrings for the `defaults` vars
* Add deprecation warning to Connection context manager
* Fix Types
* Moves the `compact` helper to `utils`
There is a helper funcion that excludes `None` values
from a list. This was being declared both in the Queue
and in the worker. This centralizes this helper in the `utils`
Importing
* Fix Type Annotation
* Remove unused code from compat module
* Remove unused dictconfig
* Remove total_ordering compat layer
* Remove compatibility layer
This completely removes the compat module. It moves utilities
functions (`as_text` and `decode_redis_hash`) to the `utils`
module, is eliminates the use of the proxies `text_type` and
`string_types`, using the `str` construct directly.
* Remove compat module
Finishes the cleaning of the compatibility module.
The last function being the `is_python_version` function
which was being used internally.
* Fix old import
* Fix Imports
* Remove Dummy (Force GH Actions)
* Fix Imports
* Organize Imports
* Add debug logs to dequeue
There were reports where a worker wouldn't fetch new jobs.
Since one possible cause is the infinite hanging of the `BLPOP` command,
this adds a couple of logs surrounding the `dequeue_any` function,
that interacts with Redis using `BLPOP`,
* Update worker.py
Fix Typo
* More logs
* Logs
* Update worker.py
* Add debug logs to the queue
* Reviewed logs
* Remove queue count before job push
* New log wording
* Remove logs
* Allow for configurable scheduler fallback period
This adds a parameter to the Scheduler called
`fallback_period`, which determines the period
before work fallsback to a new scheduler.
* Update defaults.py
Reduce Scheduler Fallback Period to 120 seconds.
* Update scheduler.py
Remove fallback period parameter.
Dummy tasks were probably used for load testing
in the past, but currently are not being imported from anywhere.
Also, similar task utilities are already present on the tests fixtures,
making the `dummy` module irrelevant.
* Persist worker_name after job is finished
Persisting the worker_name on the job object in Redis would allow for debugging and
analyzing logs from the worker
* Remove redundent job.save() method call
* Remove check for null worker
Now that worker name is persisted after job finishes or fails,
no need to assert that worker name is None
* Change github runner to Ubuntu 20.04
* Change github runner to Ubuntu 20.04
Marcus Ong
Simó Albert i Beltran
Tianci Hu Marrero
amks1
lowercase00
Selwin Ong
Rony Lutsky
dependabot[bot]
Daniel M
Rony Lutsky
Paul Spooren
Oleg
Oscar Arbeláez-Echeverri
gabriels1234
Ankush Menat
eswolinsky3241