cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,047 Commits 1 Branch 118 Tags 9.7 MiB
97e898feed
Commit Graph

2047 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Raul Ochoa
f6dc991ab5 Allow to override configuration per API call 2016-10-17 10:52:35 +02:00
Raul Ochoa
cb23b7f46c Make test finish quicker 2016-10-17 10:52:07 +02:00
Raul Ochoa
c62fe29160 Load config on object creation 2016-10-17 10:51:50 +02:00
Raul Ochoa
6179327486 Rename 2016-10-14 13:10:27 +02:00
Raul Ochoa
b8c63f5ffc Rename 2016-10-14 12:56:41 +02:00
Raul Ochoa
5bb7d8fa1c Merge branch 'master' into batch-user-queues 2016-10-14 12:33:37 +02:00
Raul Ochoa
b510a3684d Add option to skip script downloading 2016-10-14 12:28:43 +02:00
Raul Ochoa
9d952a8050 Stubs next version 2016-10-13 15:20:38 +02:00
Raul Ochoa
39b4baed5f Release 1.38.2 2016-10-13 15:19:11 +02:00
Raul Ochoa
a6cc4444d2 Update news 2016-10-13 15:18:48 +02:00
Raul Ochoa
a8802d1163 redis-distlock acquires and releases redis clients by operation 2016-10-13 13:48:06 +02:00
Raul Ochoa
05eda290be Create one client for queue-seeker and share per seek cycle 2016-10-13 13:09:56 +02:00
Raul Ochoa
df32c1a2bb Stubs next version 2016-10-13 10:17:35 +02:00
Raul Ochoa
24eb1ad10e Release 1.38.1 2016-10-13 10:16:08 +02:00
Raul Ochoa
734a147dc3 Update news 2016-10-13 10:08:44 +02:00
Raul Ochoa
32b4d66d1f Merge pull request #375 from CartoDB/batch-subscriber-improvements 
Batch subscriber improvements
 2016-10-13 09:59:43 +02:00
Raul Ochoa
1e442b37ab Allow to set a max number of queued jobs per user 2016-10-12 22:40:35 +02:00
Raul Ochoa
1f038ac1f4 Moves from host queues to user queues 
- Existing jobs are moved before start processing them.
 - Uses a new queue prefix to avoid collisions.
 - Pub/Sub also changes communication channel.
 - Job subscriber emits user+host on new jobs.
 - Batch processor is faulty. See TODO in batch.js.
 2016-10-12 21:32:29 +02:00
Raul Ochoa
aa68dff24b Use constants for tests 2016-10-12 18:44:46 +02:00
Raul Ochoa
f7d1f9426c Use constants for queues 2016-10-12 17:53:03 +02:00
Raul Ochoa
189aff2aa9 Only log message on empty queue 2016-10-12 17:42:46 +02:00
Raul Ochoa
6bb2abde0d Only start lock renewal on lock acquisition 2016-10-12 17:01:24 +02:00
Raul Ochoa
b86f82d3ca Batch.stop removes all listeners 2016-10-12 16:43:18 +02:00
Raul Ochoa
782caaf125 Run acceptance tests at the end 2016-10-12 16:41:33 +02:00
Raul Ochoa
c21f373291 Tests for locker 2016-10-12 13:15:55 +02:00
Raul Ochoa
75fc21241f Locker TTL is configured 2016-10-12 13:11:20 +02:00
Raul Ochoa
88f6d46d00 Reuse existing redlock 
Return not connected clients to pool
 2016-10-12 13:10:18 +02:00
Raul Ochoa
3f1b67993c Locker keep refreshing lock by itself 2016-10-12 12:30:13 +02:00
Raul Ochoa
67566c1d0e Callback in subscriber unsubscribe errors 2016-10-12 12:29:54 +02:00
Raul Ochoa
c74f9bcce0 More aggressive on seek interval 2016-10-12 12:29:18 +02:00
Raul Ochoa
98185e55cf Remove Job Queue Pool and use internal structure 
- We don't need to create a different job queue per host.
- Batch locks on message instead of dequeue.
 2016-10-12 12:26:50 +02:00
Raul Ochoa
167ecc6965 Set NODE_ENV environment var to test 2016-10-12 01:41:01 +02:00
Raul Ochoa
e1d0ffc7dd Logger set to fatal on test environment 2016-10-12 01:40:35 +02:00
Raul Ochoa
e401c01d78 Only log on non-test environments 2016-10-12 01:40:14 +02:00
Raul Ochoa
22d8e48f53 Only lock on dequeue 2016-10-12 00:10:40 +02:00
Raul Ochoa
81393190f7 Add callback to jobseeker result from initial load 2016-10-11 19:59:11 +02:00
Raul Ochoa
8bc52b09cf Remove console call 2016-10-11 19:46:27 +02:00
Raul Ochoa
dc1a23e886 Add error handler for channel subscriber 2016-10-11 19:45:43 +02:00
Raul Ochoa
2822b68198 onJobHandler receives host with job 
Queue seeker only returns hosts, not mixing responsibilities
 2016-10-11 19:45:26 +02:00
Raul Ochoa
01cf6f244f Share redis pool for pubsub 2016-10-11 19:41:58 +02:00
Raul Ochoa
ecc6bf0400 Use real on message handler 2016-10-11 19:04:12 +02:00
Raul Ochoa
611508c654 Hide queue seeker behind job subscriber 2016-10-11 19:01:39 +02:00
Raul Ochoa
e7c4ee32df Share redis channel config 2016-10-11 18:41:59 +02:00
Raul Ochoa
d15c7ab0de Always return client to pool 2016-10-11 18:30:35 +02:00
Raul Ochoa
e4b1711e8e pub/sub package 2016-10-11 18:28:46 +02:00
Raul Ochoa
6f30ff95d8 Stubs next version 2016-10-11 08:59:00 +02:00
Raul Ochoa
e7068b8468 Release 1.38.0 2016-10-11 08:57:36 +02:00
Raul Ochoa
d50fd89df1 Bump version 2016-10-11 08:54:33 +02:00
Raul Ochoa
b6c2d60b71 Merge pull request #373 from CartoDB/batch-queue-leader 
Batch queue leader
 2016-10-10 20:01:12 +02:00
Raul Ochoa
2c064041a1 Add dist lock to run all jobs by host in order 
It uses http://redis.io/topics/distlock
Which is not perfect: http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
 2016-10-10 19:54:59 +02:00