Raul Ochoa
ea9454d87c
Add test to validate old jobs will get discovered
2017-04-04 15:42:34 +02:00
Daniel García Aubert
25a291bb38
Improve waek assertion
2017-04-04 09:50:45 +02:00
Daniel García Aubert
36d5dc1719
Queue seeker refactor:
...
- Extract from job subcriber everything related to queue dicovering
- Job Queue is responsible of queue dicovering and exposes functions to scan user queues.
- Batch creates an interval to consume queues
- Now Batch.start() does:
* Finds existing queues from metadata, adding to queue index the ones that are not indexed yet
* Binds callback to process jobs for every queue found.
* Inits queue consumer interval
* Subscribes to PUB/SUB channel
2017-04-03 19:26:21 +02:00
Daniel García Aubert
8012fe26aa
Avoid scan behaviour to discover active queues of user's jobs, now keeps an index to know which queues are available
2017-03-31 14:30:33 +02:00
Daniel García Aubert
7faa78c3f4
skip state dependant test
2016-11-21 15:31:43 +01:00
Daniel García Aubert
78f5706726
Avoid to use scan command to search work in progress queues
2016-11-03 14:22:43 +01:00
Daniel García Aubert
af75bbda50
Rename
2016-10-28 12:50:33 +02:00
Daniel García Aubert
1f657a4f94
Ensure test is isolated
2016-10-28 12:24:23 +02:00
Daniel García Aubert
0085b8ee3d
List users with work in progress jobs
2016-10-27 20:42:49 +02:00
Daniel García Aubert
f65208ba0d
Add listWorkInProgressJobByUser function
2016-10-27 18:43:28 +02:00
Daniel García Aubert
8f65e6b16c
Add test
2016-10-27 18:24:39 +02:00
Raul Ochoa
16e9e709b8
Cancel with user statement_timeout limit from redis
2016-10-21 14:35:24 +02:00
Raul Ochoa
7563868514
Re-insert into the tree if there was a user in done state that gets a new task
2016-10-21 11:42:27 +02:00
Raul Ochoa
62cfe974b7
Use constants for users
2016-10-21 11:10:17 +02:00
Raul Ochoa
66a1c33f96
Simplify listener subscription logic
...
Always remove pending listeners on acquire call.
Always register add and release listeners on acquire.
2016-10-20 23:47:39 +02:00
Raul Ochoa
e4d54e9ab7
Fix condition to pick next candidate
2016-10-20 20:16:57 +02:00
Raul Ochoa
80d2e190ad
Fix test, use fixed to replace one and infinity
2016-10-19 18:47:55 +02:00
Raul Ochoa
1ee0878631
Scheduler uses a red–black tree to decide on next job candidate
2016-10-19 16:55:49 +02:00
Raul Ochoa
1e490be0a1
Don't rely on batch.on(job:status)
2016-10-18 20:18:49 +02:00
Raul Ochoa
cdde1be29e
Re-use redis pool as much as possible
2016-10-17 15:02:34 +02:00
Raul Ochoa
8b9a30eb75
Queue seeker was not _finding_ queues when only one present
2016-10-17 12:27:06 +02:00
Raul Ochoa
1f038ac1f4
Moves from host queues to user queues
...
- Existing jobs are moved before start processing them.
- Uses a new queue prefix to avoid collisions.
- Pub/Sub also changes communication channel.
- Job subscriber emits user+host on new jobs.
- Batch processor is faulty. See TODO in batch.js.
2016-10-12 21:32:29 +02:00
Raul Ochoa
b86f82d3ca
Batch.stop removes all listeners
2016-10-12 16:43:18 +02:00
Raul Ochoa
c21f373291
Tests for locker
2016-10-12 13:15:55 +02:00
Raul Ochoa
e4b1711e8e
pub/sub package
2016-10-11 18:28:46 +02:00
Raul Ochoa
b07e2d3de5
Merge pull request #346 from CartoDB/fix-batch-api-test
...
Removed duplicated config
2016-10-04 13:01:46 +02:00
Raul Ochoa
20edddc721
Remove user indexer dependency from tests
2016-08-30 19:06:59 +02:00
Raul Ochoa
d33fe5ac21
Stop indexing jobs per user
...
Removes .list() from job backend
2016-08-30 19:01:23 +02:00
Raul Ochoa
05ada98124
Remove .list() from job service
2016-08-30 18:51:51 +02:00
Raul Ochoa
4c8d734bbf
Remove update method from job service
2016-08-30 17:46:32 +02:00
Daniel García Aubert
085abb3446
Removed duplicated config
2016-08-01 11:48:11 +02:00
Daniel García Aubert
587ab6a5d9
Extratcs redis clean method for batch keys
2016-07-22 17:05:01 +02:00
Daniel García Aubert
2e8382eeff
Fixed issues in batch tests:
...
- Wait for batch is ready before testing it
- Remove batch keys in redis after testing every suit
- Fixed issue creating jobs with same id in job service integration test
2016-07-22 14:30:55 +02:00
Daniel García Aubert
027e7b8768
Removed unused dependency
2016-07-20 19:42:53 +02:00
Daniel García Aubert
1625d8a556
Fixed timeout in job canceller test
2016-07-20 19:41:55 +02:00
Daniel García Aubert
457414c2fe
Skipping integration test
2016-07-20 18:27:54 +02:00
Daniel García Aubert
39a9f950b0
Added integration test for multiquery jobs
2016-07-20 16:38:57 +02:00
Daniel García Aubert
b33caf6126
Moved batch integration test to a separated folder
2016-07-19 19:56:12 +02:00
