Somebody needed a job queue in Python: Multiple writers insert
into it in random order, and the jobs are written into the MySQL
table jobs. From the jobs table,
multiple consumers claim jobs in batches of n or
smaller (n=100), and process them. After processing, the
consumers delete the jobs. We need concurrent job generation and
consumption, with proper and efficient locking.
The full source for this example can be seen in mysql-dev-examples in mysql-claim-jobs.py.
Base Program
Using our usual includes and setup,
from time import sleep
from random import randint
from sys import exit
from multiprocessing import Process
import click
import MySQLdb
import MySQLdb.cursors …