Commit Graph

24 Commits

Author SHA1 Message Date
Daniel García Aubert
5eaad4d5d9 Uses redis-mpool for pubsub in Batch API 2016-07-07 14:14:46 +02:00
Daniel García Aubert
a1f31df92e Now Batch API broadcast to other APIs everytime that re-enqueues a multiple-query job 2016-06-29 18:29:53 +02:00
Daniel García Aubert
0873b6fcaa Merge branch 'master' into batch-add-profile 2016-06-03 12:03:17 +02:00
Daniel García Aubert
976bf5b039 Implemented profiling for job-runner and job-controller 2016-05-26 19:44:59 +02:00
Daniel García Aubert
64ad284c9c WIP: adding metrics to Batch API 2016-05-24 11:19:00 +02:00
Daniel García Aubert
636686386e Moved userDatabaseMetadataService from job_runner to query_runner 2016-05-23 17:51:56 +02:00
Daniel García Aubert
d2d3ba8159 Passed tests 2016-05-16 01:22:47 +02:00
Daniel García Aubert
cc7dd7a0d2 Job model refactor 2016-05-13 18:50:55 +02:00
Daniel
423e729e11 Merge pull request #283 from CartoDB/batch-improve-startup
Improved initialization in Batch API
2016-04-05 15:46:21 +02:00
Daniel García Aubert
b62edcb64f Refactored batch-api startup. Isolated code related to queue discovering, api much clear and concise. 2016-04-04 16:05:33 +02:00
dgaubert
e8a44136c5 Now Batch API looks for queues with jobs to consume at service initialization time 2016-03-31 17:37:35 +02:00
dgaubert
fd9bfe277e Implemented multiquery jobs for Batch API 2016-03-31 12:39:03 +02:00
Daniel García Aubert
e6f714a51b Implemented some new unit test 2016-01-22 19:31:25 +01:00
Daniel García Aubert
c308551d05 Implemented drain to stop batch service gracefully.
Split job acceptance test
2016-01-13 16:25:25 +01:00
Daniel García Aubert
20f00d58d9 Refactored batch service to avoid event noise, doing in callback way 2016-01-08 15:47:59 +01:00
Daniel García Aubert
f9f52d2bd1 Improved object creation in batch service 2016-01-08 11:32:01 +01:00
Daniel García Aubert
3c23bf12e7 Fixed bug whether a job consumer is empty. And refactored batch module, now is a common object intead of module function 2015-12-22 20:12:10 +01:00
Daniel García Aubert
3762ad7a39 Now jobs are stored in redis instead of user's database 2015-12-21 19:57:10 +01:00
Daniel García Aubert
fb8feeb964 Now Job is an event emitter instance and we are able to know what happen wwith any job. Implemented test without a silly timeout 2015-12-16 18:13:48 +01:00
Daniel García Aubert
43f759e96a Refactored Batch API using streams instead a interval to consume the job queue. Limited one job running at the same time per queue instead of using a job's counter to limit it. 2015-12-16 15:57:58 +01:00
Daniel García Aubert
d0787d03f7 Reused matadata backend instance for batch service 2015-12-14 10:36:16 +01:00
Daniel García Aubert
6cc48bf9dd Implemented batch service 2015-12-09 20:17:45 +01:00
Daniel García Aubert
0c6edc573a Used cartodb-redis instead redis-mpool 2015-12-09 12:18:33 +01:00
Daniel García Aubert
00721bcd02 Implementing batch service 2015-12-09 00:02:08 +01:00