Changed docs to use Github compatible code block markup

main
Selwin Ong 6 years ago
parent fc8dd95377
commit 4b8e615644

@ -23,20 +23,20 @@ pass in Redis connection references to queues directly.
In development mode, to connect to a default, local Redis server: In development mode, to connect to a default, local Redis server:
{% highlight python %} ```python
from rq import use_connection from rq import use_connection
use_connection() use_connection()
{% endhighlight %} ```
In production, to connect to a specific Redis server: In production, to connect to a specific Redis server:
{% highlight python %} ```python
from redis import Redis from redis import Redis
from rq import use_connection from rq import use_connection
redis = Redis('my.host.org', 6789, password='secret') redis = Redis('my.host.org', 6789, password='secret')
use_connection(redis) use_connection(redis)
{% endhighlight %} ```
Be aware of the fact that `use_connection` pollutes the global namespace. It Be aware of the fact that `use_connection` pollutes the global namespace. It
also implies that you can only ever use a single connection. also implies that you can only ever use a single connection.
@ -59,7 +59,7 @@ Each RQ object instance (queues, workers, jobs) has a `connection` keyword
argument that can be passed to the constructor. Using this, you don't need to argument that can be passed to the constructor. Using this, you don't need to
use `use_connection()`. Instead, you can create your queues like this: use `use_connection()`. Instead, you can create your queues like this:
{% highlight python %} ```python
from rq import Queue from rq import Queue
from redis import Redis from redis import Redis
@ -68,7 +68,7 @@ conn2 = Redis('remote.host.org', 9836)
q1 = Queue('foo', connection=conn1) q1 = Queue('foo', connection=conn1)
q2 = Queue('bar', connection=conn2) q2 = Queue('bar', connection=conn2)
{% endhighlight %} ```
Every job that is enqueued on a queue will know what connection it belongs to. Every job that is enqueued on a queue will know what connection it belongs to.
The same goes for the workers. The same goes for the workers.
@ -85,7 +85,7 @@ default connection to be used.
An example will help to understand it: An example will help to understand it:
{% highlight python %} ```python
from rq import Queue, Connection from rq import Queue, Connection
from redis import Redis from redis import Redis
@ -98,7 +98,7 @@ with Connection(Redis('localhost', 6379)):
assert q1.connection != q2.connection assert q1.connection != q2.connection
assert q2.connection != q3.connection assert q2.connection != q3.connection
assert q1.connection == q3.connection assert q1.connection == q3.connection
{% endhighlight %} ```
You can think of this as if, within the `Connection` context, every newly You can think of this as if, within the `Connection` context, every newly
created RQ object instance will have the `connection` argument set implicitly. created RQ object instance will have the `connection` argument set implicitly.
@ -112,7 +112,7 @@ If your code does not allow you to use a `with` statement, for example, if you
want to use this to set up a unit test, you can use the `push_connection()` and want to use this to set up a unit test, you can use the `push_connection()` and
`pop_connection()` methods instead of using the context manager. `pop_connection()` methods instead of using the context manager.
{% highlight python %} ```python
import unittest import unittest
from rq import Queue from rq import Queue
from rq import push_connection, pop_connection from rq import push_connection, pop_connection
@ -127,8 +127,7 @@ class MyTest(unittest.TestCase):
def test_foo(self): def test_foo(self):
"""Any queues created here use local Redis.""" """Any queues created here use local Redis."""
q = Queue() q = Queue()
... ```
{% endhighlight %}
### Sentinel support ### Sentinel support
@ -136,10 +135,10 @@ To use redis sentinel, you must specify a dictionary in the configuration file.
Using this setting in conjunction with the systemd or docker containers with the Using this setting in conjunction with the systemd or docker containers with the
automatic restart option allows workers and RQ to have a fault-tolerant connection to the redis. automatic restart option allows workers and RQ to have a fault-tolerant connection to the redis.
{% highlight python %} ```python
SENTINEL: {'INSTANCES':[('remote.host1.org', 26379), ('remote.host2.org', 26379), ('remote.host3.org', 26379)], SENTINEL: {'INSTANCES':[('remote.host1.org', 26379), ('remote.host2.org', 26379), ('remote.host3.org', 26379)],
'SOCKET_TIMEOUT': None, 'SOCKET_TIMEOUT': None,
'PASSWORD': 'secret', 'PASSWORD': 'secret',
'DB': 2, 'DB': 2,
'MASTER_NAME': 'master'} 'MASTER_NAME': 'master'}
{% endhighlight %} ```

@ -23,29 +23,27 @@ exception occurs.
This is how you register custom exception handler(s) to an RQ worker: This is how you register custom exception handler(s) to an RQ worker:
{% highlight python %} ```python
from rq.handlers import move_to_failed_queue # RQ's default exception handler from rq.handlers import move_to_failed_queue # RQ's default exception handler
w = Worker([q], exception_handlers=[my_handler, move_to_failed_queue]) w = Worker([q], exception_handlers=[my_handler, move_to_failed_queue])
... ```
{% endhighlight %}
The handler itself is a function that takes the following parameters: `job`, The handler itself is a function that takes the following parameters: `job`,
`exc_type`, `exc_value` and `traceback`: `exc_type`, `exc_value` and `traceback`:
{% highlight python %} ```python
def my_handler(job, exc_type, exc_value, traceback): def my_handler(job, exc_type, exc_value, traceback):
# do custom things here # do custom things here
# for example, write the exception info to a DB # for example, write the exception info to a DB
...
{% endhighlight %} ```
You might also see the three exception arguments encoded as: You might also see the three exception arguments encoded as:
```python ```python
def my_handler(job, *exc_info): def my_handler(job, *exc_info):
# do custom things here # do custom things here
...
``` ```
## Chaining exception handlers ## Chaining exception handlers
@ -63,7 +61,7 @@ as `True` (i.e. continue with the next handler).
To replace the default behaviour (i.e. moving the job to the `failed` queue), To replace the default behaviour (i.e. moving the job to the `failed` queue),
use a custom exception handler that doesn't fall through, for example: use a custom exception handler that doesn't fall through, for example:
{% highlight python %} ```python
def black_hole(job, *exc_info): def black_hole(job, *exc_info):
return False return False
{% endhighlight %} ```

@ -13,13 +13,13 @@ arguments onto a queue. This is called _enqueueing_.
To put jobs on queues, first declare a function: To put jobs on queues, first declare a function:
{% highlight python %} ```python
import requests import requests
def count_words_at_url(url): def count_words_at_url(url):
resp = requests.get(url) resp = requests.get(url)
return len(resp.text.split()) return len(resp.text.split())
{% endhighlight %} ```
Noticed anything? There's nothing special about this function! Any Python Noticed anything? There's nothing special about this function! Any Python
function call can be put on an RQ queue. function call can be put on an RQ queue.
@ -27,7 +27,7 @@ function call can be put on an RQ queue.
To put this potentially expensive word count for a given URL in the background, To put this potentially expensive word count for a given URL in the background,
simply do this: simply do this:
{% highlight python %} ```python
from rq import Queue from rq import Queue
from redis import Redis from redis import Redis
from somewhere import count_words_at_url from somewhere import count_words_at_url
@ -43,14 +43,14 @@ print(job.result) # => None
# Now, wait a while, until the worker is finished # Now, wait a while, until the worker is finished
time.sleep(2) time.sleep(2)
print(job.result) # => 889 print(job.result) # => 889
{% endhighlight %} ```
If you want to put the work on a specific queue, simply specify its name: If you want to put the work on a specific queue, simply specify its name:
{% highlight python %} ```python
q = Queue('low', connection=redis_conn) q = Queue('low', connection=redis_conn)
q.enqueue(count_words_at_url, 'http://nvie.com') q.enqueue(count_words_at_url, 'http://nvie.com')
{% endhighlight %} ```
Notice the `Queue('low')` in the example above? You can use any queue name, so Notice the `Queue('low')` in the example above? You can use any queue name, so
you can quite flexibly distribute work to your own desire. A common naming you can quite flexibly distribute work to your own desire. A common naming
@ -78,28 +78,28 @@ job function.
In the last case, it may be advantageous to instead use the explicit version of In the last case, it may be advantageous to instead use the explicit version of
`.enqueue()`, `.enqueue_call()`: `.enqueue()`, `.enqueue_call()`:
{% highlight python %} ```python
q = Queue('low', connection=redis_conn) q = Queue('low', connection=redis_conn)
q.enqueue_call(func=count_words_at_url, q.enqueue_call(func=count_words_at_url,
args=('http://nvie.com',), args=('http://nvie.com',),
timeout=30) timeout=30)
{% endhighlight %} ```
For cases where the web process doesn't have access to the source code running For cases where the web process doesn't have access to the source code running
in the worker (i.e. code base X invokes a delayed function from code base Y), in the worker (i.e. code base X invokes a delayed function from code base Y),
you can pass the function as a string reference, too. you can pass the function as a string reference, too.
{% highlight python %} ```python
q = Queue('low', connection=redis_conn) q = Queue('low', connection=redis_conn)
q.enqueue('my_package.my_module.my_func', 3, 4) q.enqueue('my_package.my_module.my_func', 3, 4)
{% endhighlight %} ```
## Working with Queues ## Working with Queues
Besides enqueuing jobs, Queues have a few useful methods: Besides enqueuing jobs, Queues have a few useful methods:
{% highlight python %} ```python
from rq import Queue from rq import Queue
from redis import Redis from redis import Redis
@ -117,7 +117,7 @@ job = q.fetch_job('my_id') # Returns job having ID "my_id"
# Deleting the queue # Deleting the queue
q.delete(delete_jobs=True) # Passing in `True` will remove all jobs in the queue q.delete(delete_jobs=True) # Passing in `True` will remove all jobs in the queue
# queue is unusable now unless re-instantiated # queue is unusable now unless re-instantiated
{% endhighlight %} ```
### On the Design ### On the Design
@ -151,7 +151,7 @@ of course).
If you're familiar with Celery, you might be used to its `@task` decorator. If you're familiar with Celery, you might be used to its `@task` decorator.
Starting from RQ >= 0.3, there exists a similar decorator: Starting from RQ >= 0.3, there exists a similar decorator:
{% highlight python %} ```python
from rq.decorators import job from rq.decorators import job
@job('low', connection=my_redis_conn, timeout=5) @job('low', connection=my_redis_conn, timeout=5)
@ -161,7 +161,7 @@ def add(x, y):
job = add.delay(3, 4) job = add.delay(3, 4)
time.sleep(1) time.sleep(1)
print(job.result) print(job.result)
{% endhighlight %} ```
## Bypassing workers ## Bypassing workers
@ -170,12 +170,12 @@ For testing purposes, you can enqueue jobs without delegating the actual
execution to a worker (available since version 0.3.1). To do this, pass the execution to a worker (available since version 0.3.1). To do this, pass the
`is_async=False` argument into the Queue constructor: `is_async=False` argument into the Queue constructor:
{% highlight pycon %} ```python
>>> q = Queue('low', is_async=False, connection=my_redis_conn) >>> q = Queue('low', is_async=False, connection=my_redis_conn)
>>> job = q.enqueue(fib, 8) >>> job = q.enqueue(fib, 8)
>>> job.result >>> job.result
21 21
{% endhighlight %} ```
The above code runs without an active worker and executes `fib(8)` The above code runs without an active worker and executes `fib(8)`
synchronously within the same process. You may know this behaviour from Celery synchronously within the same process. You may know this behaviour from Celery
@ -188,11 +188,11 @@ a redis instance for storing states related to job execution and completion.
New in RQ 0.4.0 is the ability to chain the execution of multiple jobs. New in RQ 0.4.0 is the ability to chain the execution of multiple jobs.
To execute a job that depends on another job, use the `depends_on` argument: To execute a job that depends on another job, use the `depends_on` argument:
{% highlight python %} ```python
q = Queue('low', connection=my_redis_conn) q = Queue('low', connection=my_redis_conn)
report_job = q.enqueue(generate_report) report_job = q.enqueue(generate_report)
q.enqueue(send_report, depends_on=report_job) q.enqueue(send_report, depends_on=report_job)
{% endhighlight %} ```
The ability to handle job dependencies allows you to split a big job into The ability to handle job dependencies allows you to split a big job into
several smaller ones. A job that is dependent on another is enqueued only when several smaller ones. A job that is dependent on another is enqueued only when

@ -13,14 +13,14 @@ jobs.
All job information is stored in Redis. You can inspect a job and its attributes All job information is stored in Redis. You can inspect a job and its attributes
by using `Job.fetch()`. by using `Job.fetch()`.
{% highlight python %} ```python
from redis import Redis from redis import Redis
from rq.job import Job from rq.job import Job
connection = Redis() connection = Redis()
job = Job.fetch('my_job_id', connection=redis) job = Job.fetch('my_job_id', connection=redis)
print('Status: %s' $ job.get_status()) print('Status: %s' $ job.get_status())
{% endhighlight %} ```
Some interesting job attributes include: Some interesting job attributes include:
* `job.status` * `job.status`
@ -38,14 +38,14 @@ Some interesting job attributes include:
Since job functions are regular Python functions, you have to ask RQ for the Since job functions are regular Python functions, you have to ask RQ for the
current job ID, if any. To do this, you can use: current job ID, if any. To do this, you can use:
{% highlight python %} ```python
from rq import get_current_job from rq import get_current_job
def add(x, y): def add(x, y):
job = get_current_job() job = get_current_job()
print('Current job: %s' % (job.id,)) print('Current job: %s' % (job.id,))
return x + y return x + y
{% endhighlight %} ```
## Storing arbitrary data on jobs ## Storing arbitrary data on jobs
@ -56,7 +56,7 @@ To add/update custom status information on this job, you have access to the
`meta` property, which allows you to store arbitrary pickleable data on the job `meta` property, which allows you to store arbitrary pickleable data on the job
itself: itself:
{% highlight python %} ```python
import socket import socket
def add(x, y): def add(x, y):
@ -67,7 +67,7 @@ def add(x, y):
# do more work # do more work
time.sleep(1) time.sleep(1)
return x + y return x + y
{% endhighlight %} ```
## Time to live for job in queue ## Time to live for job in queue
@ -77,13 +77,13 @@ _New in version 0.4.7._
A job has two TTLs, one for the job result and one for the job itself. This means that if you have A job has two TTLs, one for the job result and one for the job itself. This means that if you have
job that shouldn't be executed after a certain amount of time, you can define a TTL as such: job that shouldn't be executed after a certain amount of time, you can define a TTL as such:
{% highlight python %} ```python
# When creating the job: # When creating the job:
job = Job.create(func=say_hello, ttl=43) job = Job.create(func=say_hello, ttl=43)
# or when queueing a new job: # or when queueing a new job:
job = q.enqueue(count_words_at_url, 'http://nvie.com', ttl=43) job = q.enqueue(count_words_at_url, 'http://nvie.com', ttl=43)
{% endhighlight %} ```
## Failed Jobs ## Failed Jobs
@ -92,7 +92,7 @@ If a job fails and raises an exception, the worker will put the job in a failed
On the Job instance, the `is_failed` property will be true. To fetch all failed jobs, scan On the Job instance, the `is_failed` property will be true. To fetch all failed jobs, scan
through the `get_failed_queue()` queue. through the `get_failed_queue()` queue.
{% highlight python %} ```python
from redis import StrictRedis from redis import StrictRedis
from rq import push_connection, get_failed_queue, Queue from rq import push_connection, get_failed_queue, Queue
from rq.job import Job from rq.job import Job
@ -115,4 +115,4 @@ fq.requeue(job.id)
assert fq.count == 0 assert fq.count == 0
assert Queue('fake').count == 1 assert Queue('fake').count == 1
{% endhighlight %} ```

@ -13,10 +13,10 @@ which looks like this:
To install, just do: To install, just do:
{% highlight console %} ```console
$ pip install rq-dashboard $ pip install rq-dashboard
$ rq-dashboard $ rq-dashboard
{% endhighlight %} ```
It can also be integrated easily in your Flask app. It can also be integrated easily in your Flask app.
@ -25,7 +25,7 @@ It can also be integrated easily in your Flask app.
To see what queues exist and what workers are active, just type `rq info`: To see what queues exist and what workers are active, just type `rq info`:
{% highlight console %} ```console
$ rq info $ rq info
high |██████████████████████████ 20 high |██████████████████████████ 20
low |██████████████ 12 low |██████████████ 12
@ -36,14 +36,14 @@ Bricktop.19233 idle: low
Bricktop.19232 idle: high, default, low Bricktop.19232 idle: high, default, low
Bricktop.18349 idle: default Bricktop.18349 idle: default
3 workers, 3 queues 3 workers, 3 queues
{% endhighlight %} ```
## Querying by queue names ## Querying by queue names
You can also query for a subset of queues, if you're looking for specific ones: You can also query for a subset of queues, if you're looking for specific ones:
{% highlight console %} ```console
$ rq info high default $ rq info high default
high |██████████████████████████ 20 high |██████████████████████████ 20
default |█████████ 8 default |█████████ 8
@ -52,7 +52,7 @@ default |█████████ 8
Bricktop.19232 idle: high, default Bricktop.19232 idle: high, default
Bricktop.18349 idle: default Bricktop.18349 idle: default
2 workers, 2 queues 2 workers, 2 queues
{% endhighlight %} ```
## Organising workers by queue ## Organising workers by queue
@ -60,7 +60,7 @@ Bricktop.18349 idle: default
By default, `rq info` prints the workers that are currently active, and the By default, `rq info` prints the workers that are currently active, and the
queues that they are listening on, like this: queues that they are listening on, like this:
{% highlight console %} ```console
$ rq info $ rq info
... ...
@ -68,12 +68,12 @@ Mickey.26421 idle: high, default
Bricktop.25458 busy: high, default, low Bricktop.25458 busy: high, default, low
Turkish.25812 busy: high, default Turkish.25812 busy: high, default
3 workers, 3 queues 3 workers, 3 queues
{% endhighlight %} ```
To see the same data, but organised by queue, use the `-R` (or `--by-queue`) To see the same data, but organised by queue, use the `-R` (or `--by-queue`)
flag: flag:
{% highlight console %} ```console
$ rq info -R $ rq info -R
... ...
@ -82,7 +82,7 @@ low: Bricktop.25458 (busy)
default: Bricktop.25458 (busy), Mickey.26421 (idle), Turkish.25812 (busy) default: Bricktop.25458 (busy), Mickey.26421 (idle), Turkish.25812 (busy)
failed: failed:
3 workers, 4 queues 3 workers, 4 queues
{% endhighlight %} ```
## Interval polling ## Interval polling
@ -90,16 +90,16 @@ failed:
By default, `rq info` will print stats and exit. By default, `rq info` will print stats and exit.
You can specify a poll interval, by using the `--interval` flag. You can specify a poll interval, by using the `--interval` flag.
{% highlight console %} ```console
$ rq info --interval 1 $ rq info --interval 1
{% endhighlight %} ```
`rq info` will now update the screen every second. You may specify a float `rq info` will now update the screen every second. You may specify a float
value to indicate fractions of seconds. Be aware that low interval values will value to indicate fractions of seconds. Be aware that low interval values will
increase the load on Redis, of course. increase the load on Redis, of course.
{% highlight console %} ```console
$ rq info --interval 0.5 $ rq info --interval 0.5
{% endhighlight %} ```
[dashboard]: https://github.com/nvie/rq-dashboard [dashboard]: https://github.com/nvie/rq-dashboard

@ -93,15 +93,15 @@ If a job requires more (or less) time to complete, the default timeout period
can be loosened (or tightened), by specifying it as a keyword argument to the can be loosened (or tightened), by specifying it as a keyword argument to the
`enqueue()` call, like so: `enqueue()` call, like so:
{% highlight python %} ```python
q = Queue() q = Queue()
q.enqueue(mytask, args=(foo,), kwargs={'bar': qux}, timeout=600) # 10 mins q.enqueue(mytask, args=(foo,), kwargs={'bar': qux}, timeout=600) # 10 mins
{% endhighlight %} ```
You can also change the default timeout for jobs that are enqueued via specific You can also change the default timeout for jobs that are enqueued via specific
queue instances at once, which can be useful for patterns like this: queue instances at once, which can be useful for patterns like this:
{% highlight python %} ```python
# High prio jobs should end in 8 secs, while low prio # High prio jobs should end in 8 secs, while low prio
# work may take up to 10 mins # work may take up to 10 mins
high = Queue('high', default_timeout=8) # 8 secs high = Queue('high', default_timeout=8) # 8 secs
@ -109,7 +109,7 @@ low = Queue('low', default_timeout=600) # 10 mins
# Individual jobs can still override these defaults # Individual jobs can still override these defaults
low.enqueue(really_really_slow, timeout=3600) # 1 hr low.enqueue(really_really_slow, timeout=3600) # 1 hr
{% endhighlight %} ```
Individual jobs can still specify an alternative timeout, as workers will Individual jobs can still specify an alternative timeout, as workers will
respect these. respect these.

@ -9,7 +9,7 @@ You may wish to include your RQ tasks inside unit tests. However many frameworks
Therefore, you must use the SimpleWorker class to avoid fork(); Therefore, you must use the SimpleWorker class to avoid fork();
{% highlight python %} ```python
from redis import Redis from redis import Redis
from rq import SimpleWorker, Queue from rq import SimpleWorker, Queue
@ -18,7 +18,7 @@ queue.enqueue(my_long_running_job)
worker = SimpleWorker([queue], connection=queue.connection) worker = SimpleWorker([queue], connection=queue.connection)
worker.work(burst=True) # Runs enqueued job worker.work(burst=True) # Runs enqueued job
# Check for result... # Check for result...
{% endhighlight %} ```
## Running Jobs in unit tests ## Running Jobs in unit tests
@ -31,11 +31,11 @@ Additionally, we can use fakeredis to mock a redis instance, so we don't have to
run a redis server separately. The instance of the fake redis server can run a redis server separately. The instance of the fake redis server can
be directly passed as the connection argument to the queue: be directly passed as the connection argument to the queue:
{% highlight python %} ```python
from fakeredis import FakeStrictRedis from fakeredis import FakeStrictRedis
from rq import Queue from rq import Queue
queue = Queue(is_async=False, connection=FakeStrictRedis()) queue = Queue(is_async=False, connection=FakeStrictRedis())
job = queue.enqueue(my_long_running_job) job = queue.enqueue(my_long_running_job)
assert job.is_finished assert job.is_finished
{% endhighlight %} ```

@ -13,14 +13,14 @@ to perform inside web processes.
To start crunching work, simply start a worker from the root of your project To start crunching work, simply start a worker from the root of your project
directory: directory:
{% highlight console %} ```console
$ rq worker high normal low $ rq worker high normal low
*** Listening for work on high, normal, low *** Listening for work on high, normal, low
Got send_newsletter('me@nvie.com') from default Got send_newsletter('me@nvie.com') from default
Job ended normally without result Job ended normally without result
*** Listening for work on high, normal, low *** Listening for work on high, normal, low
... ...
{% endhighlight %} ```
Workers will read jobs from the given queues (the order is important) in an Workers will read jobs from the given queues (the order is important) in an
endless loop, waiting for new work to arrive when all jobs are done. endless loop, waiting for new work to arrive when all jobs are done.
@ -37,14 +37,14 @@ new work when they run out of work. Workers can also be started in _burst
mode_ to finish all currently available work and quit as soon as all given mode_ to finish all currently available work and quit as soon as all given
queues are emptied. queues are emptied.
{% highlight console %} ```console
$ rq worker --burst high normal low $ rq worker --burst high normal low
*** Listening for work on high, normal, low *** Listening for work on high, normal, low
Got send_newsletter('me@nvie.com') from default Got send_newsletter('me@nvie.com') from default
Job ended normally without result Job ended normally without result
No more work, burst finished. No more work, burst finished.
Registering death. Registering death.
{% endhighlight %} ```
This can be useful for batch work that needs to be processed periodically, or This can be useful for batch work that needs to be processed periodically, or
just to scale up your workers temporarily during peak periods. just to scale up your workers temporarily during peak periods.
@ -106,7 +106,7 @@ yourself before starting the work loop.
To do this, provide your own worker script (instead of using `rq worker`). To do this, provide your own worker script (instead of using `rq worker`).
A simple implementation example: A simple implementation example:
{% highlight python %} ```python
#!/usr/bin/env python #!/usr/bin/env python
import sys import sys
from rq import Connection, Worker from rq import Connection, Worker
@ -121,7 +121,7 @@ with Connection():
w = Worker(qs) w = Worker(qs)
w.work() w.work()
{% endhighlight %} ```
### Worker names ### Worker names
@ -139,7 +139,7 @@ starting the worker, using the `--name` option.
`Worker` instances store their runtime information in Redis. Here's how to `Worker` instances store their runtime information in Redis. Here's how to
retrieve them: retrieve them:
{% highlight python %} ```python
from redis import Redis from redis import Redis
from rq import Queue, Worker from rq import Queue, Worker
@ -150,14 +150,14 @@ workers = Worker.all(connection=redis)
# Returns all workers in this queue (new in version 0.10.0) # Returns all workers in this queue (new in version 0.10.0)
queue = Queue('queue_name') queue = Queue('queue_name')
workers = Worker.all(queue=queue) workers = Worker.all(queue=queue)
{% endhighlight %} ```
_New in version 0.10.0._ _New in version 0.10.0._
If you only want to know the number of workers for monitoring purposes, using If you only want to know the number of workers for monitoring purposes, using
`Worker.count()` is much more performant. `Worker.count()` is much more performant.
{% highlight python %} ```python
from redis import Redis from redis import Redis
from rq import Worker from rq import Worker
@ -169,8 +169,7 @@ workers = Worker.count(connection=redis)
# Count the number of workers for a specific queue # Count the number of workers for a specific queue
queue = Queue('queue_name', connection=redis) queue = Queue('queue_name', connection=redis)
workers = Worker.all(queue=queue) workers = Worker.all(queue=queue)
```
{% endhighlight %}
### Worker statistics ### Worker statistics
@ -180,14 +179,14 @@ _New in version 0.9.0._
If you want to check the utilization of your queues, `Worker` instances If you want to check the utilization of your queues, `Worker` instances
store a few useful information: store a few useful information:
{% highlight python %} ```python
from rq.worker import Worker from rq.worker import Worker
worker = Worker.find_by_key('rq:worker:name') worker = Worker.find_by_key('rq:worker:name')
worker.successful_job_count # Number of jobs finished successfully worker.successful_job_count # Number of jobs finished successfully
worker.failed_job_count. # Number of failed jobs processed by this worker worker.failed_job_count. # Number of failed jobs processed by this worker
worker.total_working_time # Number of time spent executing jobs worker.total_working_time # Number of time spent executing jobs
{% endhighlight %} ```
## Taking down workers ## Taking down workers
@ -209,7 +208,7 @@ If you'd like to configure `rq worker` via a configuration file instead of
through command line arguments, you can do this by creating a Python file like through command line arguments, you can do this by creating a Python file like
`settings.py`: `settings.py`:
{% highlight python %} ```python
REDIS_URL = 'redis://localhost:6379/1' REDIS_URL = 'redis://localhost:6379/1'
# You can also specify the Redis DB to use # You can also specify the Redis DB to use
@ -228,7 +227,7 @@ SENTRY_DSN = 'sync+http://public:secret@example.com/1'
# If you want custom worker name # If you want custom worker name
# NAME = 'worker-1024' # NAME = 'worker-1024'
{% endhighlight %} ```
The example above shows all the options that are currently supported. The example above shows all the options that are currently supported.
@ -236,9 +235,9 @@ _Note: The_ `QUEUES` _and_ `REDIS_PASSWORD` _settings are new since 0.3.3._
To specify which module to read settings from, use the `-c` option: To specify which module to read settings from, use the `-c` option:
{% highlight console %} ```console
$ rq worker -c settings $ rq worker -c settings
{% endhighlight %} ```
## Custom worker classes ## Custom worker classes
@ -255,9 +254,9 @@ more common requests so far are:
You can use the `-w` option to specify a different worker class to use: You can use the `-w` option to specify a different worker class to use:
{% highlight console %} ```console
$ rq worker -w 'path.to.GeventWorker' $ rq worker -w 'path.to.GeventWorker'
{% endhighlight %} ```
## Custom Job and Queue classes ## Custom Job and Queue classes
@ -267,15 +266,15 @@ _Will be available in next release._
You can tell the worker to use a custom class for jobs and queues using You can tell the worker to use a custom class for jobs and queues using
`--job-class` and/or `--queue-class`. `--job-class` and/or `--queue-class`.
{% highlight console %} ```console
$ rq worker --job-class 'custom.JobClass' --queue-class 'custom.QueueClass' $ rq worker --job-class 'custom.JobClass' --queue-class 'custom.QueueClass'
{% endhighlight %} ```
Don't forget to use those same classes when enqueueing the jobs. Don't forget to use those same classes when enqueueing the jobs.
For example: For example:
{% highlight python %} ```python
from rq import Queue from rq import Queue
from rq.job import Job from rq.job import Job
@ -287,7 +286,7 @@ class CustomQueue(Queue):
queue = CustomQueue('default', connection=redis_conn) queue = CustomQueue('default', connection=redis_conn)
queue.enqueue(some_func) queue.enqueue(some_func)
{% endhighlight %} ```
## Custom DeathPenalty classes ## Custom DeathPenalty classes
@ -307,9 +306,9 @@ _New in version 0.5.5._
If you need to handle errors differently for different types of jobs, or simply want to customize If you need to handle errors differently for different types of jobs, or simply want to customize
RQ's default error handling behavior, run `rq worker` using the `--exception-handler` option: RQ's default error handling behavior, run `rq worker` using the `--exception-handler` option:
{% highlight console %} ```console
$ rq worker --exception-handler 'path.to.my.ErrorHandler' $ rq worker --exception-handler 'path.to.my.ErrorHandler'
# Multiple exception handlers is also supported # Multiple exception handlers is also supported
$ rq worker --exception-handler 'path.to.my.ErrorHandler' --exception-handler 'another.ErrorHandler' $ rq worker --exception-handler 'path.to.my.ErrorHandler' --exception-handler 'another.ErrorHandler'
{% endhighlight %} ```

Loading…
Cancel
Save