Raul Ochoa
|
98185e55cf
|
Remove Job Queue Pool and use internal structure
- We don't need to create a different job queue per host.
- Batch locks on message instead of dequeue.
|
2016-10-12 12:26:50 +02:00 |
|
Raul Ochoa
|
01cf6f244f
|
Share redis pool for pubsub
|
2016-10-11 19:41:58 +02:00 |
|
Raul Ochoa
|
611508c654
|
Hide queue seeker behind job subscriber
|
2016-10-11 19:01:39 +02:00 |
|
Raul Ochoa
|
e4b1711e8e
|
pub/sub package
|
2016-10-11 18:28:46 +02:00 |
|
Raul Ochoa
|
56a632347b
|
Inject publisher
|
2016-10-10 19:47:50 +02:00 |
|
Raul Ochoa
|
66820a67bb
|
Make possible to specify a name for batch
|
2016-10-10 19:46:07 +02:00 |
|
Daniel García Aubert
|
aa0ce62a85
|
Implement batch logger to log query times when queries are defined with id
|
2016-09-29 15:09:36 +02:00 |
|
Raul Ochoa
|
d33fe5ac21
|
Stop indexing jobs per user
Removes .list() from job backend
|
2016-08-30 19:01:23 +02:00 |
|
Daniel García Aubert
|
5eaad4d5d9
|
Uses redis-mpool for pubsub in Batch API
|
2016-07-07 14:14:46 +02:00 |
|
Daniel García Aubert
|
a1f31df92e
|
Now Batch API broadcast to other APIs everytime that re-enqueues a multiple-query job
|
2016-06-29 18:29:53 +02:00 |
|
Daniel García Aubert
|
0873b6fcaa
|
Merge branch 'master' into batch-add-profile
|
2016-06-03 12:03:17 +02:00 |
|
Daniel García Aubert
|
976bf5b039
|
Implemented profiling for job-runner and job-controller
|
2016-05-26 19:44:59 +02:00 |
|
Daniel García Aubert
|
64ad284c9c
|
WIP: adding metrics to Batch API
|
2016-05-24 11:19:00 +02:00 |
|
Daniel García Aubert
|
636686386e
|
Moved userDatabaseMetadataService from job_runner to query_runner
|
2016-05-23 17:51:56 +02:00 |
|
Daniel García Aubert
|
d2d3ba8159
|
Passed tests
|
2016-05-16 01:22:47 +02:00 |
|
Daniel García Aubert
|
cc7dd7a0d2
|
Job model refactor
|
2016-05-13 18:50:55 +02:00 |
|
Daniel
|
423e729e11
|
Merge pull request #283 from CartoDB/batch-improve-startup
Improved initialization in Batch API
|
2016-04-05 15:46:21 +02:00 |
|
Daniel García Aubert
|
b62edcb64f
|
Refactored batch-api startup. Isolated code related to queue discovering, api much clear and concise.
|
2016-04-04 16:05:33 +02:00 |
|
dgaubert
|
e8a44136c5
|
Now Batch API looks for queues with jobs to consume at service initialization time
|
2016-03-31 17:37:35 +02:00 |
|
dgaubert
|
fd9bfe277e
|
Implemented multiquery jobs for Batch API
|
2016-03-31 12:39:03 +02:00 |
|
Daniel García Aubert
|
e6f714a51b
|
Implemented some new unit test
|
2016-01-22 19:31:25 +01:00 |
|
Daniel García Aubert
|
c308551d05
|
Implemented drain to stop batch service gracefully.
Split job acceptance test
|
2016-01-13 16:25:25 +01:00 |
|
Daniel García Aubert
|
20f00d58d9
|
Refactored batch service to avoid event noise, doing in callback way
|
2016-01-08 15:47:59 +01:00 |
|
Daniel García Aubert
|
f9f52d2bd1
|
Improved object creation in batch service
|
2016-01-08 11:32:01 +01:00 |
|
Daniel García Aubert
|
3c23bf12e7
|
Fixed bug whether a job consumer is empty. And refactored batch module, now is a common object intead of module function
|
2015-12-22 20:12:10 +01:00 |
|
Daniel García Aubert
|
3762ad7a39
|
Now jobs are stored in redis instead of user's database
|
2015-12-21 19:57:10 +01:00 |
|
Daniel García Aubert
|
fb8feeb964
|
Now Job is an event emitter instance and we are able to know what happen wwith any job. Implemented test without a silly timeout
|
2015-12-16 18:13:48 +01:00 |
|
Daniel García Aubert
|
43f759e96a
|
Refactored Batch API using streams instead a interval to consume the job queue. Limited one job running at the same time per queue instead of using a job's counter to limit it.
|
2015-12-16 15:57:58 +01:00 |
|
Daniel García Aubert
|
d0787d03f7
|
Reused matadata backend instance for batch service
|
2015-12-14 10:36:16 +01:00 |
|
Daniel García Aubert
|
6cc48bf9dd
|
Implemented batch service
|
2015-12-09 20:17:45 +01:00 |
|
Daniel García Aubert
|
0c6edc573a
|
Used cartodb-redis instead redis-mpool
|
2015-12-09 12:18:33 +01:00 |
|
Daniel García Aubert
|
00721bcd02
|
Implementing batch service
|
2015-12-09 00:02:08 +01:00 |
|