25 Commits (62acd56c917137e8096d3f73514364c07994d9e4)

Author SHA1 Message Date
Yang Yang 9db728921d
Improve the lint situation (#1688)
* Move common flake8 options into config file

Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.

Move --count and --statistics to config file as well.

* Fix some lints
2 years ago
Hugo d5175c38da
Drop python2-specific syntax (#1674)
* Drop syntax required only for Python 2

* Drop python2-style super() calls

Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
2 years ago
Selwin Ong c556106a38
Main worker should use zadd(xx=True) to update heartbeat. (#1550) 3 years ago
Josh Cohen b80045d615
Respect serializer (#1538)
* Add serializer where missing in code

* Fix cli

* Pass option to command

* Add tests for serializer option

* Merge branch 'master' into respect-serializer
- Update enqueue cli to resp. serializer

* Address @selwin's review
3 years ago
Selwin Ong f9ddf7dca5
Requeueing job should clear job.exc_info (#1543) 3 years ago
rpkak d41f60b906
Allows enqueueing by the cli (#1466)
* Allows enqueueing by the cli

#372

* schedule support

* `_` to `-`

* fix flake8

* echo job-id

* Some improvements

 - Description as in python jobs
 - return result
 - quiet mode
 - allows `--boolean` and `--integer`
 - raises errors if not used correctly

* added tests

* add schedule tests

* add retry test

* use click exceptions

* add error test

* add job_func test

* change messages

https://github.com/rq/rq/pull/1466#discussion_r640211128
https://github.com/rq/rq/pull/1466#discussion_r640210850

* Use different format for arguments

View https://github.com/rq/rq/pull/1466#discussion_r650510889

* Add file support

Usage: @filename

* ast.literal_eval support with `#` instead of `:`

* func -> function

Makes error messages more readable

* click Error

* print function string

* add docs

* increase seconds in test

* Update `parse_function_arg`

Add `ParsingMode` enum (https://github.com/rq/rq/pull/1466#discussion_r656676114)
Change error messages (https://github.com/rq/rq/pull/1466#discussion_r656676800, https://github.com/rq/rq/pull/1466#discussion_r656677082)

* `#` to `%`

`#` is the letter for a comment in bash

* Add some tests

(https://github.com/rq/rq/pull/1466#discussion_r656674539, https://github.com/rq/rq/pull/1466#discussion_r656676543)

* Add some tests

* docs: Add some examples

* catch all literal_eval exceptions

There are some edge cases with other exceptions

* remove job_func
(https://github.com/rq/rq/pull/1466#pullrequestreview-690110118)

* edit docs

https://github.com/rq/rq/pull/1466#pullrequestreview-695758691

* format examples

* format examples

`queue.enqueue(path.to.func, args=['abc'])` to `queue.enqueue(path.to.func, 'abc')`

https://github.com/rq/rq/pull/1466#discussion_r673615464

* add examples

https://github.com/rq/rq/pull/1466#discussion_r673658933

* add doc test

https://github.com/rq/rq/pull/1466#discussion_r673659124

* Update index.md

* Update test_cli.py

* Update test_cli.py

* Add version info

Co-authored-by: rpkak <rpkak@users.noreply.github.com>
3 years ago
rpkak 73d0210d65
Add environment variables for connection (#1472)
* Add environment variables for connection

Use same args as in config file

#1342

* add test

* add suggestions

https://github.com/rq/rq/pull/1472#discussion_r640205865
https://github.com/rq/rq/pull/1472#discussion_r640206565

* remove unused import
4 years ago
f0cker efe703214e
Added --serializer option to cli, finishing off PR #1381 and fix #1357 (#1395)
* Added --serializer option to cli, finishing off PR #1381 and fix #1357

* Update documentation

* Update documentation

* Modified help message

Co-authored-by: f0cker <dturner@trustwave.com>
4 years ago
Selwin Ong f3e924cdd1
Added job.worker_name (#1375)
* Added job.worker_name

* Fix compatibility with Redis server 3.x

* Document job.worker_name

* Removed some Python 2 compatibility stuff.

* Remove unused codes
4 years ago
Selwin Ong baa0cc268a
Job scheduling (#1163)
* First RQScheduler prototype

* WIP job scheduling

* Fixed Python 2.7 tests

* Added ScheduledJobRegistry.get_scheduled_time(job)

* WIP on scheduler's threading mechanism

* Fixed test errors

* Changed scheduler.acquire_locks() to instance method

* Added scheduler.prepare_registries()

* Somewhat working implementation of RQ scheduler

* Only call stop_scheduler if there's a scheduler present

* Use OSError rather than ProcessLookupError for PyPy compatibility

* Added `auto_start` argument to scheduler.acquire_locks()

* Make RQScheduler play better with timezone

* Fixed test error

* Added --with-scheduler flag to rq worker CLI

* Fix tests on Python 2.x

* More Python 2 fixes

* Only call `scheduler.start` if worker is run in non burst mode

* Fixed an issue where running worker with scheduler would fail sometimes

* Make `worker.stop_scheduler()` more resilient to errors

* worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks

* Scheduler can now work with worker in both burst and non burst mode

* Fixed scheduler logging message

* Always log scheduler errors when running

* Improve scheduler error logging message

* Removed testing code

* Scheduler should periodically try to acquire locks for other queues it doesn't have

* Added tests for scheduler.should_reacquire_locks

* Added queue.enqueue_in()

* Fixes queue.enqueue_in() in Python 2.7

* First stab at documenting job scheduling

* Remove unused methods

* Remove Python 2.6 logging compatibility code

* Remove more unused imports

* Added convenience methods to access job registries from queue

* Added test for worker.run_maintenance_tasks()

* Simplify worker.queue_names() and worker.queue_keys()

* Updated changelog to mention RQ's new job scheduling mechanism.
5 years ago
Yongtao Zhang 5bb03b9c2c Fix rq info not found workers information error, Fixes #1139 (#1149)
* Fix rq info not found workers information error, Fixes #1139

* Add test(#1149)
5 years ago
Selwin Ong 549648bd1b
rq info management command now cleans up registries when first run (#1107)
* rq info management command now cleans up registries when first run

* Deleted print statement

* Improve CLI test coverage

* Fixed CLI test on Linux
6 years ago
Selwin Ong c4cbb3af2f
RQ v1.0! (#1059)
* Added FailedJobRegistry.

* Added job.failure_ttl.

* queue.enqueue() now supports failure_ttl

* Added registry.get_queue().

* FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL.

* StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry.

* Failed jobs are now added to FailedJobRegistry.

* Added FailedJobRegistry.requeue()

* Document the new `FailedJobRegistry` and changes in custom exception handler behavior.

* Added worker.disable_default_exception_handler.

* Document --disable-default-exception-handler option.

* Deleted worker.failed_queue.

* Deleted "move_to_failed_queue" exception handler.

* StartedJobRegistry should no longer move jobs to FailedQueue.

* Deleted requeue_job

* Fixed test error.

* Make requeue cli command work with FailedJobRegistry

* Added .pytest_cache to gitignore.

* Custom exception handlers are no longer run in reverse

* Restored requeue_job function

* Removed get_failed_queue

* Deleted FailedQueue

* Updated changelog.

* Document `failure_ttl`

* Updated docs.

* Remove job.status

* Fixed typo in test_registry.py

* Replaced _pipeline() with pipeline()

* FailedJobRegistry no longer fails on redis-py>=3

* Fixes test_clean_registries

* Worker names are now randomized

* Added a note about random worker names in CHANGES.md

* Worker will now stop working when encountering an unhandled exception.

* Worker should reraise SystemExit on cold shutdowns

* Added anchor.js to docs

* Support for Sentry-SDK (#1045)

* Updated RQ to support sentry-sdk

* Document Sentry integration

* Install sentry-sdk before running tests

* Improved rq info CLI command to be more efficient when displaying lar… (#1046)

* Improved rq info CLI command to be more efficient when displaying large number of workers

* Fixed an rq info --by-queue bug

* Fixed worker.total_working_time bug (#1047)

* queue.enqueue() no longer accepts `timeout` argument (#1055)

* Clean worker registry (#1056)

* queue.enqueue() no longer accepts `timeout` argument

* Added clean_worker_registry()

* Show worker hostname and PID on cli (#1058)

* Show worker hostname and PID on cli

* Improve test coverage

* Remove Redis version check when SSL is used

* Bump version to 1.0

* Removed pytest_cache/README.md

* Changed worker logging to use exc_info=True

* Removed unused queue.dequeue()

* Fixed typo in CHANGES.md

* setup_loghandlers() should always call logger.setLevel() if specified
6 years ago
Chyroc 7eb95bf405 refactor: use try ImportError instead of py-version check (#1034) 6 years ago
Theo 0fab93d683 Fixed #809 - Added tests for various cli config parameters 7 years ago
Jannis Leidel c019662430
Allow passing backend classes (job, queue, worker, connection) from CLI and other APIs
This includes:

- a partial refactor of the CLI to organize the shared options
- extends the tests in areas where passing custom backend classes makes sense
- allow setting the core CLI options as env vars
- minor cosmetic changes here and there
8 years ago
ahxxm b06f112cb0 fix tests
syntax: assertEquals -> assertEqual, assertNotEquals -> assertNotEqual
usage: status of worker and job now will use get/set method instead of property method
9 years ago
Vincent Driessen c1a4780d10 Fix PEP8 complaints 9 years ago
Selwin Ong 8bfbeac67d Running rqworker without specifiying --exception-handler shouldn't override default behavior. 10 years ago
Bradley Young d3e4fb567b merge && updates to add multiple exception handlers. 10 years ago
Jonathan Tushman 82333d2ad5 triggering shutdown by setting a redis flag 10 years ago
Bradley Young 5caccaabfe Adding optional list handling to the exc_handler option in Worker.
Adding command line --exception_handler option (with multiple entries allowed) to `rq worker`
Added tests for command line options.
10 years ago
zhangliyong 94e494e645 Add testcase for rq worker command 10 years ago
zhangliyong 918403e371 Update test_cli 10 years ago
Vincent Driessen dab8373311 Fix all tests. 10 years ago