Raul Ochoa
16e9e709b8
Cancel with user statement_timeout limit from redis
2016-10-21 14:35:24 +02:00
Raul Ochoa
26e4cb3196
Get timeout from async function
2016-10-21 13:09:17 +02:00
Raul Ochoa
1f038ac1f4
Moves from host queues to user queues
...
- Existing jobs are moved before start processing them.
- Uses a new queue prefix to avoid collisions.
- Pub/Sub also changes communication channel.
- Job subscriber emits user+host on new jobs.
- Batch processor is faulty. See TODO in batch.js.
2016-10-12 21:32:29 +02:00
Raul Ochoa
8a4f54bb87
Allow users to set max statement_timeout for their queries
2016-10-10 12:01:36 +02:00
Raul Ochoa
5401a7edff
Timeout is passed into query runner
2016-10-10 12:00:54 +02:00
Raul Ochoa
51d4ff0698
Differentiate between statement timeout and user cancelled query
2016-10-10 11:58:44 +02:00
Raul Ochoa
578c43b1a8
Multiple queries jobs pushed as first job between queries
2016-10-06 18:27:38 +02:00
Daniel García Aubert
e387458801
Now profiler is passed as argument instead to be a member property of job runner
2016-06-22 16:10:42 +02:00
Daniel
39ceba6707
Merge pull request #312 from CartoDB/batch-add-profile
...
Adds profiling to batch and job controller
2016-06-08 19:31:36 +02:00
Daniel García Aubert
0873b6fcaa
Merge branch 'master' into batch-add-profile
2016-06-03 12:03:17 +02:00
Daniel García Aubert
cc1a5641ea
Fixed conflicts in merge
2016-06-03 11:43:21 +02:00
Daniel García Aubert
b6967f98f2
Fixed error handling in job query runner
2016-06-03 10:44:16 +02:00
Daniel García Aubert
7de1e323b0
Avoid to use job_id to profile job timing
2016-05-30 12:43:42 +02:00
Daniel García Aubert
cf1e072c17
Improved steps in profiling
2016-05-27 12:41:24 +02:00
Daniel García Aubert
976bf5b039
Implemented profiling for job-runner and job-controller
2016-05-26 19:44:59 +02:00
Daniel García Aubert
64ad284c9c
WIP: adding metrics to Batch API
2016-05-24 11:19:00 +02:00
Daniel García Aubert
636686386e
Moved userDatabaseMetadataService from job_runner to query_runner
2016-05-23 17:51:56 +02:00
Daniel García Aubert
e04787a070
Fixed missing failed_reason in failing jobs
2016-05-19 17:02:48 +02:00
Daniel García Aubert
b2eac6b880
Implemented test for fallback jobs
2016-05-17 01:00:27 +02:00
Daniel García Aubert
d2d3ba8159
Passed tests
2016-05-16 01:22:47 +02:00
Daniel García Aubert
cc7dd7a0d2
Job model refactor
2016-05-13 18:50:55 +02:00
Daniel García Aubert
4a83de09af
Added job status enumerable to avoid hardcodes strings
2016-04-05 18:50:04 +02:00
dgaubert
270cdf8969
Set pending to multiquery job when draining
2016-03-31 15:11:35 +02:00
dgaubert
f95e8ef7f2
Fixed job status when failing
2016-03-31 13:36:58 +02:00
dgaubert
fd9bfe277e
Implemented multiquery jobs for Batch API
2016-03-31 12:39:03 +02:00
Daniel García Aubert
aefd32ee06
Improved message in failed_error property for multi-query jobs
2016-03-21 15:10:05 +01:00
Daniel García Aubert
ee0821a518
Changed error message in batch api
2016-03-21 12:29:43 +01:00
Daniel García Aubert
689f97c1c7
Improved function naming and query error message.
2016-03-21 12:09:54 +01:00
Daniel García Aubert
705e649b80
Fixed style typos
...
Please enter the commit message for your changes. Lines starting
2016-03-18 18:19:00 +01:00
Daniel García Aubert
5e0b86b181
Improved error message in multi-jobs
2016-03-18 16:23:40 +01:00
Daniel García Aubert
dcab147e90
Improved error handling for multi-jobs and cancelation.
2016-03-18 16:05:36 +01:00
Daniel García Aubert
056f22b156
Implemented multi-jobs, user is able to send an array of jobs and batch service will run them in series
2016-03-18 14:57:18 +01:00
Daniel García Aubert
cb638bb0fe
Fixed issue with Postgres error codes, implemented some test for job controller
2016-01-22 18:22:21 +01:00
Daniel García Aubert
5c4d239467
Fixed unit test for job queue and moved isJobFound function as private method in job backend
2016-01-22 12:43:41 +01:00
Daniel García Aubert
d07af1740a
Fixed bug after cancel a job by user the job runner's callback is never called
2016-01-17 19:28:15 +01:00
Daniel García Aubert
c308551d05
Implemented drain to stop batch service gracefully.
...
Split job acceptance test
2016-01-13 16:25:25 +01:00
Daniel García Aubert
20f00d58d9
Refactored batch service to avoid event noise, doing in callback way
2016-01-08 15:47:59 +01:00
Daniel García Aubert
f9f52d2bd1
Improved object creation in batch service
2016-01-08 11:32:01 +01:00
Daniel García Aubert
44d4fd1bb8
Minor improvement in job runner creation
2016-01-07 19:16:39 +01:00
Daniel García Aubert
f89c44586c
Fixed bug related to growing file descriptors when batch api processes jobs
2016-01-07 18:36:32 +01:00
Daniel García Aubert
246655de94
Changed redis data structure for users jobs
2016-01-04 19:08:13 +01:00
Daniel García Aubert
13ffaab0d5
Resolved minor bug when user tries to cancel a job and included a ttl to jobs after its resolution
2016-01-04 15:20:06 +01:00
Daniel García Aubert
35650985db
Implemented job modification whether job is pending
2015-12-31 15:42:31 +01:00
Daniel García Aubert
ada39d84b8
Implemented job cancelation in Batch API
2015-12-30 20:16:18 +01:00
Daniel García Aubert
851c393f75
Refactored batch manager avoiding concurrence issues
2015-12-29 10:19:10 +01:00
Daniel García Aubert
c4f96b4a59
Implemented support for listing jobs for one user
2015-12-24 17:42:49 +01:00
Daniel García Aubert
ecb76ca1de
Improved batch API being consistent with naming, humam readability for job dates, hide redis key details for jobs.
2015-12-23 17:29:11 +01:00
Daniel García Aubert
f41c43aa38
Disabled statement timeout bypassing pgbouncer connection
2015-12-22 15:43:00 +01:00
Daniel García Aubert
3762ad7a39
Now jobs are stored in redis instead of user's database
2015-12-21 19:57:10 +01:00
