Commit Graph

1859 Commits

Author SHA1 Message Date
Raul Ochoa
2822b68198 onJobHandler receives host with job
Queue seeker only returns hosts, not mixing responsibilities
2016-10-11 19:45:26 +02:00
Raul Ochoa
01cf6f244f Share redis pool for pubsub 2016-10-11 19:41:58 +02:00
Raul Ochoa
ecc6bf0400 Use real on message handler 2016-10-11 19:04:12 +02:00
Raul Ochoa
611508c654 Hide queue seeker behind job subscriber 2016-10-11 19:01:39 +02:00
Raul Ochoa
e7c4ee32df Share redis channel config 2016-10-11 18:41:59 +02:00
Raul Ochoa
d15c7ab0de Always return client to pool 2016-10-11 18:30:35 +02:00
Raul Ochoa
e4b1711e8e pub/sub package 2016-10-11 18:28:46 +02:00
Raul Ochoa
6f30ff95d8 Stubs next version 2016-10-11 08:59:00 +02:00
Raul Ochoa
e7068b8468 Release 1.38.0 2016-10-11 08:57:36 +02:00
Raul Ochoa
d50fd89df1 Bump version 2016-10-11 08:54:33 +02:00
Raul Ochoa
b6c2d60b71 Merge pull request #373 from CartoDB/batch-queue-leader
Batch queue leader
2016-10-10 20:01:12 +02:00
Raul Ochoa
2c064041a1 Add dist lock to run all jobs by host in order
It uses http://redis.io/topics/distlock
Which is not perfect: http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
2016-10-10 19:54:59 +02:00
Raul Ochoa
0de5d94617 Use debug with same params, no considering job status 2016-10-10 19:53:59 +02:00
Raul Ochoa
90c489119b Add distributed lock implementation with redis distlock 2016-10-10 19:51:11 +02:00
Raul Ochoa
56a632347b Inject publisher 2016-10-10 19:47:50 +02:00
Raul Ochoa
f036e29b8f Make test stable 2016-10-10 19:47:18 +02:00
Raul Ochoa
41d7823227 Drop table if exists 2016-10-10 19:46:53 +02:00
Raul Ochoa
a0ace8d034 Use name in test 2016-10-10 19:46:39 +02:00
Raul Ochoa
66820a67bb Make possible to specify a name for batch 2016-10-10 19:46:07 +02:00
Raul Ochoa
f4950f73c0 Add test to validate multiple consumers query order scenario 2016-10-10 15:56:04 +02:00
Raul Ochoa
ece95afc95 Add test client to run plain queries 2016-10-10 15:37:46 +02:00
Raul Ochoa
46579093dd Fix test scenario description 2016-10-10 15:37:19 +02:00
Raul Ochoa
8ac656492f Move to support directory 2016-10-10 15:22:50 +02:00
Raul Ochoa
faa02a79b9 Test for 578c43b1a8 2016-10-10 15:17:05 +02:00
Raul Ochoa
fd6eb49fe2 Merge pull request #372 from CartoDB/batch-queries-limits
Batch queries limits
2016-10-10 12:25:26 +02:00
Raul Ochoa
d8c3181054 Update news 2016-10-10 12:11:21 +02:00
Raul Ochoa
deb1ccf876 DRY job final statuses 2016-10-10 12:09:13 +02:00
Raul Ochoa
8a4f54bb87 Allow users to set max statement_timeout for their queries 2016-10-10 12:01:36 +02:00
Raul Ochoa
5401a7edff Timeout is passed into query runner 2016-10-10 12:00:54 +02:00
Raul Ochoa
51d4ff0698 Differentiate between statement timeout and user cancelled query 2016-10-10 11:58:44 +02:00
Raul Ochoa
e23edadcf1 Fix and use a batch timeout good for tests 2016-10-10 11:10:49 +02:00
Raul Ochoa
595d4f0918 Fix sample config for batch timeout 2016-10-10 11:10:28 +02:00
Raul Ochoa
1004131333 Merge pull request #371 from CartoDB/query-order
Query order
2016-10-06 18:50:12 +02:00
Raul Ochoa
1d20f11f0c Remove unused var 2016-10-06 18:44:21 +02:00
Raul Ochoa
578c43b1a8 Multiple queries jobs pushed as first job between queries 2016-10-06 18:27:38 +02:00
Raul Ochoa
e108d0df57 Debug query to run 2016-10-06 18:24:28 +02:00
Raul Ochoa
aa19120495 Merge pull request #357 from CartoDB/limit-batch-queries
set limit to batch queries to 12h
2016-10-06 13:04:51 +02:00
Raul Ochoa
7c7320061f Merge branch 'master' into limit-batch-queries 2016-10-06 12:46:34 +02:00
Raul Ochoa
5d3f4f06cb Merge pull request #370 from CartoDB/batch-queries-limits
Move all batch tests to their own directory
2016-10-06 12:41:31 +02:00
Raul Ochoa
26fe6a1626 Add readme for batch queries feature 2016-10-06 12:27:38 +02:00
Raul Ochoa
2ac39b5748 Move all batch tests to their own directory 2016-10-06 11:53:56 +02:00
Raul Ochoa
eb2768c197 Add dbhost attribute to batch queries logs 2016-10-05 19:09:10 +02:00
Raul Ochoa
2bba1934ed Stubs next version 2016-10-05 17:33:56 +02:00
Raul Ochoa
2014036ba7 Release 1.37.1 2016-10-05 17:33:05 +02:00
Raul Ochoa
14a8a99191 Update news 2016-10-05 17:32:45 +02:00
Raul Ochoa
42a1f3ad4c Accept multipart requests 2016-10-05 17:29:16 +02:00
Raul Ochoa
6309318534 Use body-parser from old connect module 2016-10-05 14:22:44 +02:00
Raul Ochoa
025b3f3cc7 Increase body limit to 20mb 2016-10-05 11:14:06 +02:00
Raul Ochoa
86b2b2bb81 Merge pull request #369 from CartoDB/batch-queries-limits
Job controller DRY
2016-10-04 17:12:24 +02:00
Raul Ochoa
2a2a54a073 DRY in job response handler 2016-10-04 16:07:13 +02:00