Daniel García Aubert
36d5dc1719
Queue seeker refactor:
...
- Extract from job subcriber everything related to queue dicovering
- Job Queue is responsible of queue dicovering and exposes functions to scan user queues.
- Batch creates an interval to consume queues
- Now Batch.start() does:
* Finds existing queues from metadata, adding to queue index the ones that are not indexed yet
* Binds callback to process jobs for every queue found.
* Inits queue consumer interval
* Subscribes to PUB/SUB channel
2017-04-03 19:26:21 +02:00
Daniel García Aubert
24ff1cf808
Discover and add to index queues of users with jobs on batch service startup
2017-04-03 12:34:30 +02:00
Daniel García Aubert
8012fe26aa
Avoid scan behaviour to discover active queues of user's jobs, now keeps an index to know which queues are available
2017-03-31 14:30:33 +02:00
Daniel García Aubert
6592435e87
Use COUNT option for scan command
2017-03-29 20:40:40 +02:00
Daniel García Aubert
09d5a14ade
Check data from redis scan is an array
2017-03-29 20:04:19 +02:00
Daniel García Aubert
56429ffebf
Use traditional for loop instead of .forEach()
2017-03-29 19:54:08 +02:00
Daniel García Aubert
2edfd1fad5
Compare interger instead of strings
2017-03-29 19:51:19 +02:00
Daniel García Aubert
34b352a0a6
Use max number of attempts to scan user queues
2017-03-29 19:38:36 +02:00
Raul Ochoa
8b9a30eb75
Queue seeker was not _finding_ queues when only one present
2016-10-17 12:27:06 +02:00
Raul Ochoa
5bb7d8fa1c
Merge branch 'master' into batch-user-queues
2016-10-14 12:33:37 +02:00
Raul Ochoa
05eda290be
Create one client for queue-seeker and share per seek cycle
2016-10-13 13:09:56 +02:00
Raul Ochoa
1f038ac1f4
Moves from host queues to user queues
...
- Existing jobs are moved before start processing them.
- Uses a new queue prefix to avoid collisions.
- Pub/Sub also changes communication channel.
- Job subscriber emits user+host on new jobs.
- Batch processor is faulty. See TODO in batch.js.
2016-10-12 21:32:29 +02:00
Raul Ochoa
f7d1f9426c
Use constants for queues
2016-10-12 17:53:03 +02:00
Raul Ochoa
67566c1d0e
Callback in subscriber unsubscribe errors
2016-10-12 12:29:54 +02:00
Raul Ochoa
c74f9bcce0
More aggressive on seek interval
2016-10-12 12:29:18 +02:00
Raul Ochoa
81393190f7
Add callback to jobseeker result from initial load
2016-10-11 19:59:11 +02:00
Raul Ochoa
8bc52b09cf
Remove console call
2016-10-11 19:46:27 +02:00
Raul Ochoa
dc1a23e886
Add error handler for channel subscriber
2016-10-11 19:45:43 +02:00
Raul Ochoa
2822b68198
onJobHandler receives host with job
...
Queue seeker only returns hosts, not mixing responsibilities
2016-10-11 19:45:26 +02:00
Raul Ochoa
ecc6bf0400
Use real on message handler
2016-10-11 19:04:12 +02:00
Raul Ochoa
611508c654
Hide queue seeker behind job subscriber
2016-10-11 19:01:39 +02:00
Raul Ochoa
e7c4ee32df
Share redis channel config
2016-10-11 18:41:59 +02:00
Raul Ochoa
d15c7ab0de
Always return client to pool
2016-10-11 18:30:35 +02:00
Raul Ochoa
e4b1711e8e
pub/sub package
2016-10-11 18:28:46 +02:00
