SkySQL just blogged about a tool to schedule long running MySQL
jobs, prevent too many queries from running simultaneously, and
stores the results in tables. It even uses Gearman.
You will note that the article says that it uses PAQU,
which uses Shard-Query.
I think PAQU was created for two reasons. A) Shard-Query
lacked support for fast aggregation of STDDEV and VARIANCE (this
has been fixed), and B) their data set requires “cross-shard
queries”. From what I can see though, their type of
cross-shard queries can be solved using subqueries in the FROM
clause using Shard-Query, instead of using a customized (forked)
version of Shard-Query. It is unfortunate, because my
recent improvements to Shard-Query have to be ported into PAQU by
the PAQU authors.
I’d like to encourage you to look at Shard-Query if you need to
run complex jobs in the background and get the results later.
As a bonus, you …
[Read more]