You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Ruslan Mullakhmetov 676ec9b0ea
tests: updated github worklow for tests to use requirements.txt and d… (#1364)
* tests: updated github worklow for tests to use requirements.txt and dev-requirements.txt

* build: updated dev-requirements.txt

Co-authored-by: Ruslan Mullakhmetov <ruslan@twentythree.net>
4 years ago
.github tests: updated github worklow for tests to use requirements.txt and d… (#1364) 4 years ago
docker Fix #1340 (#1341) 4 years ago
docs feat: added job heartbeat to track whether job is actually executing (#1349) 4 years ago
examples fix print in example 11 years ago
rq feat: added job heartbeat to track whether job is actually executing (#1349) 4 years ago
tests feat: added job heartbeat to track whether job is actually executing (#1349) 4 years ago
.coveragerc Ignore local.py (it's tested in werkzeug instead). 11 years ago
.deepsource.toml Fix some code quality issues (#1235) 5 years ago
.gitignore RQ v1.0! (#1059) 6 years ago
.mailmap Add .mailmap 9 years ago
CHANGES.md Bump version to 1.5.2 4 years ago
Dockerfile tests: added ability to run tests in Docker (#1361) 4 years ago
LICENSE Fix year. 13 years ago
MANIFEST.in include requirements.txt in sdist (#1335) 4 years ago
Makefile Clean dist+build folders before releasing 10 years ago
README.md Bump version to 1.5.0 4 years ago
dev-requirements.txt tests: updated github worklow for tests to use requirements.txt and d… (#1364) 4 years ago
requirements.txt Improve requirements handling (#1287) 5 years ago
run_tests_in_docker.sh tests: added ability to run tests in Docker (#1361) 4 years ago
setup.cfg modify zadd calls for redis-py 3.0 (#1016) 6 years ago
setup.py Exclude tests directory from wheel builds 4 years ago
tox.ini Fix run_tests to use pytest. (#1033) 6 years ago

README.md

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

RQ requires Redis >= 3.0.0.

Build status PyPI Coverage

Full documentation can be found here.

Support RQ

If you find RQ useful, please consider supporting this project via Tidelift.

Getting started

First, run a Redis server, of course:

$ redis-server

To put jobs on queues, you don't have to do anything special, just define your typically lengthy or blocking function:

import requests

def count_words_at_url(url):
    """Just an example function that's called async."""
    resp = requests.get(url)
    return len(resp.text.split())

You do use the excellent requests package, don't you?

Then, create an RQ queue:

from redis import Redis
from rq import Queue

queue = Queue(connection=Redis())

And enqueue the function call:

from my_module import count_words_at_url
job = queue.enqueue(count_words_at_url, 'http://nvie.com')

Scheduling jobs are also similarly easy:

# Schedule job to run at 9:15, October 10th
job = queue.enqueue_at(datetime(2019, 10, 8, 9, 15), say_hello)

# Schedule job to run in 10 seconds
job = queue.enqueue_in(timedelta(seconds=10), say_hello)

Retrying failed jobs is also supported:

from rq import Retry

# Retry up to 3 times, failed job will be requeued immediately
queue.enqueue(say_hello, retry=Retry(max=3))

# Retry up to 3 times, with configurable intervals between retries
queue.enqueue(say_hello, retry=Retry(max=3, interval=[10, 30, 60]))

For a more complete example, refer to the docs. But this is the essence.

The worker

To start executing enqueued function calls in the background, start a worker from your project's directory:

$ rq worker --with-scheduler
*** Listening for work on default
Got count_words_at_url('http://nvie.com') from default
Job result = 818
*** Listening for work on default

That's about it.

Installation

Simply use the following command to install the latest released version:

pip install rq

If you want the cutting edge version (that may well be broken), use this:

pip install -e git+https://github.com/nvie/rq.git@master#egg=rq

Check out these below repos which might be useful in your rq based project.

Project history

This project has been inspired by the good parts of Celery, Resque and this snippet, and has been created as a lightweight alternative to the heaviness of Celery or other AMQP-based queueing implementations.