85 Commits (d41f60b9065717d0b69e405d7f7461ed48c54305)

Author SHA1 Message Date
rpkak d41f60b906
Allows enqueueing by the cli (#1466)
* Allows enqueueing by the cli

#372

* schedule support

* `_` to `-`

* fix flake8

* echo job-id

* Some improvements

 - Description as in python jobs
 - return result
 - quiet mode
 - allows `--boolean` and `--integer`
 - raises errors if not used correctly

* added tests

* add schedule tests

* add retry test

* use click exceptions

* add error test

* add job_func test

* change messages

https://github.com/rq/rq/pull/1466#discussion_r640211128
https://github.com/rq/rq/pull/1466#discussion_r640210850

* Use different format for arguments

View https://github.com/rq/rq/pull/1466#discussion_r650510889

* Add file support

Usage: @filename

* ast.literal_eval support with `#` instead of `:`

* func -> function

Makes error messages more readable

* click Error

* print function string

* add docs

* increase seconds in test

* Update `parse_function_arg`

Add `ParsingMode` enum (https://github.com/rq/rq/pull/1466#discussion_r656676114)
Change error messages (https://github.com/rq/rq/pull/1466#discussion_r656676800, https://github.com/rq/rq/pull/1466#discussion_r656677082)

* `#` to `%`

`#` is the letter for a comment in bash

* Add some tests

(https://github.com/rq/rq/pull/1466#discussion_r656674539, https://github.com/rq/rq/pull/1466#discussion_r656676543)

* Add some tests

* docs: Add some examples

* catch all literal_eval exceptions

There are some edge cases with other exceptions

* remove job_func
(https://github.com/rq/rq/pull/1466#pullrequestreview-690110118)

* edit docs

https://github.com/rq/rq/pull/1466#pullrequestreview-695758691

* format examples

* format examples

`queue.enqueue(path.to.func, args=['abc'])` to `queue.enqueue(path.to.func, 'abc')`

https://github.com/rq/rq/pull/1466#discussion_r673615464

* add examples

https://github.com/rq/rq/pull/1466#discussion_r673658933

* add doc test

https://github.com/rq/rq/pull/1466#discussion_r673659124

* Update index.md

* Update test_cli.py

* Update test_cli.py

* Add version info

Co-authored-by: rpkak <rpkak@users.noreply.github.com>
3 years ago
Selwin Ong 5590aab458
Success and failure callbacks (#1480)
* Support enqueueing with on_success_callback

* Got success callback to execute properly

* Added on_failure callback

* Document success and failure callbacks

* More Flake8 fixes

* Mention that options can also be passed in via environment variables

* Increase coverage on test_callbacks.py
4 years ago
Josh Cohen 456743b225
Make `Queue.enqueue`, `Queue.enqueue_call`, `Queue.enqueue_at``Queue.parse_args` accept `pipeline` arg, add `Queue.enqueue_many` method (#1485)
* Make `enqueue_*` and `Queue.parse_args` accept `pipeline` arg

* undo bad docs

* Fix lints in new code

* Implement enqueue_many, refactor dependency setup out to method

* Make name consistant

* Refactor enqueue_many, update tests,  add docs

* Refactor out enqueueing from dependency setup

* Move new docs to queue page

* Fix section header

* Fix section header again

* Update rq version to 1.9.0
4 years ago
rpkak 29d02c0798
Docs: Add scheduled to job.get_status() (#1493) 4 years ago
rpkak 28e7ab10df
Add chat page (#1474)
* Add chat page

* no js / iframe
4 years ago
Joseph ba915508a3
Typo: Than -> Then (#1479)
Co-authored-by: mgjo5899 <mgjo5899@gmail.com>
4 years ago
rpkak d42947fb75
add `/` (#1475) 4 years ago
Antonino Sabetta 1680f49264
Fix mistake in doc of Random dequeuing strategy (#1443) 4 years ago
James Cobb 6bfd47f735
Newer pip install command from git (#1439)
* Newer pip install command from git

Updated cutting edge pip install command to use HTTPS rather than insecure git+git protocol, as recommended by pip documentation. This allows the command to work with pip >= 21.0.1

* Updated pip command in README
4 years ago
Biel Cardona 08ef54dcf4
Workers dequeuing jobs from queues using both Round-Robin and Random strategies (#1425)
* implemented round-robin and random access to queues

* added tests for RoundRobinQueue

* reverted change in gitignore

* removed linebreak

* added tests for random queues

* added documentation for round robin and random queues

* moved round robin strategy to worker

* reverted changes to queue.py

* reverted changes to workers.md

* reverted changes to test_queue

* added tests for RoundRobinWorker and RandomWorker

* added doc for round robin and random workers

* removed f-strings for backward compatibility

* corrected a mistake

* minor changes (code style)

* now using _ordered_queues instead of queues for reordering queues
4 years ago
Daniel Alley fc7940c77b
Add a "stopped" JobStatus and the machinery to properly handle it (#1394)
* Add a "stopped" JobStatus and the machinery to properly handle it

fixes #1389

* Apply requested changes
4 years ago
Selwin Ong aa5dbf4af3 Document that --serializer CLI argument is only available in 1.8.0 4 years ago
f0cker efe703214e
Added --serializer option to cli, finishing off PR #1381 and fix #1357 (#1395)
* Added --serializer option to cli, finishing off PR #1381 and fix #1357

* Update documentation

* Update documentation

* Modified help message

Co-authored-by: f0cker <dturner@trustwave.com>
4 years ago
Selwin Ong d3b07fba47 Bump version to 1.7.0 4 years ago
Selwin Ong ab19ad9798 Improve docs 4 years ago
C. Mangla 8b9e2188e4
Add `job.refresh()` to docs (#1384) 4 years ago
Kyle Verhoog 3aaa1c1209
fix snippet link in docs (#1380)
The former link is broken since http://flask.pocoo.org/snippets has been
taken down. See https://github.com/pallets/website/issues/41 for more
info.
4 years ago
Arturo Bracero 2ece5d0fba
fixed typo in send_kill_horse_command() example (#1378) 4 years ago
Selwin Ong 492e77d86d
send_stop_job_command (#1376)
* Added send_stop_job_command().

* send_stop_job_command now accepts just connection and job_id

* Document send_job_job_command

* Updated test coverage
4 years ago
Selwin Ong f3e924cdd1
Added job.worker_name (#1375)
* Added job.worker_name

* Fix compatibility with Redis server 3.x

* Document job.worker_name

* Removed some Python 2 compatibility stuff.

* Remove unused codes
4 years ago
Ruslan Mullakhmetov ed264f08bb
feat: added job heartbeat to track whether job is actually executing (#1349)
* feat: added job heartbeat to track whether job is actually executing

heartbeat might be needed in cases when worker was hardkilled or the whole VM/docker was forcibly rebooted.

* fixed tests

* fixed test coverage issue

* chore: renamed job.heartbeat stuff according to review feedback

* chore: pipelined worker heartbeat and job heartbeat

* docs: documented job.heartbeat property

* fixes after review

* docs: updated last_heartbeat description

* chore: review

Co-authored-by: Ruslan Mullakhmetov <ruslan@twentythree.net>
4 years ago
Selwin Ong a721db34b1
Workers can listen to external commands via pubsub (#1363)
* Added a way to send shutdown command via pubsub

* Added kill-horse command

* Added kill horse command

* Added send_kill_horse_command() and send_shutdown_command()

* Document worker commands
4 years ago
Selwin Ong f5ce159b60 Remove alpha warning from scheduler docs 4 years ago
maces 48f5a68013
fixed minor import error in jobs docs (#1334) 4 years ago
Selwin Ong 57f286eac4 Minor doc fix 4 years ago
Selwin Ong 167ab96070 Slightly tweaked docs CSS 4 years ago
Selwin Ong dff7a09a1d Fixed Github Pages build 4 years ago
Selwin Ong f3c86c02c6 Bump version to 1.5.0 4 years ago
Zhicheng Cai a62b762200
Update index.md (#1306) 4 years ago
Selwin Ong 49b156ecc7
Job retry feature. Docs WIP (#1299)
* Initial implementation of Retry class

* Fixes job.refresh() under Python 3.5

* Remove the use of text_type in job.py

* Retry can be scheduled

* monitor_work_horse() should call handle_job_failure() with queue argument.

* Flake8 fixes

* Added docs for job retries
4 years ago
Paul Spooren 73506b26fc
Add get_job_position and get_position feature (#1271)
Fix #1197

Signed-off-by: Paul Spooren <mail@aparcar.org>
5 years ago
Vincent Jacques 9fb80424ef
Fix typo in scheduling doc (#1245) 5 years ago
Babatunde Olusola e1cbc3736c
Implement Customizable Serializer Support (#1219)
* Implement Customizable Serializer Support

* Refractor serializer instance methods

* Update tests with other serializers

* Edit function description

* Edit function description

* Raise appropriate exception

* Update tests for better code coverage

* Remove un-used imports and un-necessary code

* Refractor resolve_serializer

* Remove un-necessary alias from imports

* Add documentation

* Refractor tests, improve documentation
5 years ago
Selwin Ong 57f8fe6a0c Updated job.result docs. 5 years ago
Ryan Febriansyah 71b01ba1b6
fix code tag in connections.md (#1189)
* Update connections.md

* Update testing.md
5 years ago
Selwin Ong 43fb21b827 Show job scheduling section on docs website 5 years ago
Levin Rickert 0080b223fe
Fix typos in job_registries.md (#1185) 5 years ago
Selwin Ong baa0cc268a
Job scheduling (#1163)
* First RQScheduler prototype

* WIP job scheduling

* Fixed Python 2.7 tests

* Added ScheduledJobRegistry.get_scheduled_time(job)

* WIP on scheduler's threading mechanism

* Fixed test errors

* Changed scheduler.acquire_locks() to instance method

* Added scheduler.prepare_registries()

* Somewhat working implementation of RQ scheduler

* Only call stop_scheduler if there's a scheduler present

* Use OSError rather than ProcessLookupError for PyPy compatibility

* Added `auto_start` argument to scheduler.acquire_locks()

* Make RQScheduler play better with timezone

* Fixed test error

* Added --with-scheduler flag to rq worker CLI

* Fix tests on Python 2.x

* More Python 2 fixes

* Only call `scheduler.start` if worker is run in non burst mode

* Fixed an issue where running worker with scheduler would fail sometimes

* Make `worker.stop_scheduler()` more resilient to errors

* worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks

* Scheduler can now work with worker in both burst and non burst mode

* Fixed scheduler logging message

* Always log scheduler errors when running

* Improve scheduler error logging message

* Removed testing code

* Scheduler should periodically try to acquire locks for other queues it doesn't have

* Added tests for scheduler.should_reacquire_locks

* Added queue.enqueue_in()

* Fixes queue.enqueue_in() in Python 2.7

* First stab at documenting job scheduling

* Remove unused methods

* Remove Python 2.6 logging compatibility code

* Remove more unused imports

* Added convenience methods to access job registries from queue

* Added test for worker.run_maintenance_tasks()

* Simplify worker.queue_names() and worker.queue_keys()

* Updated changelog to mention RQ's new job scheduling mechanism.
5 years ago
goldstar611 b960ad53e6 Add example for unix:// (#1171) 5 years ago
Selwin Ong cf50151853 Added a note to indicate that `registry.remove(job, delete_job=True)` is new 5 years ago
Selwin Ong af678243e1
Added `delete_job` argument to registry.remove()` (#1161) 5 years ago
joncros 68276e7252 add doc page for job registries (#1140)
* add doc page for job registries

* Use consistent language in registry type descriptions

* Correct usage of 'connection' parameter in Job Registry doc
5 years ago
joncros 5328a6252b Edit jobs doc for clarity (#1136)
* add section "Job Creation" to jobs doc

* edit jobs doc for clarity
5 years ago
Selwin Ong ed15b9e031 Updated ttl argument docs 5 years ago
William Woodruff b020c7fa34 docs/jobs: Fix minor typo (#1128) 5 years ago
Joe 8df4e8ecfa Update worker document for setproctitle feature (#1109)
* Replace procname with setproctitle

* Document process title feature

* Fix typo
6 years ago
Paul Robertson e1c135d4de add the ability to have the worker stop executing after a max amount of jobs (#1094)
* add the ability to have the worker stop executing after a max amount of jobs

* rename to max-jobs

* updated logging messages
6 years ago
Christopher Pickering d2e06e4785 Update index.md (#1089)
added "import time"
6 years ago
Ted Summer eed51df14a Fix usage example of queue.enqueue() (#1082)
* Fix usage example of queue.enqueue()

* Refactor enqueue example
6 years ago
Selwin Ong f467e708f2 Updated leftover references to FailedQueue 6 years ago