Raul Ochoa
1ee0878631
Scheduler uses a red–black tree to decide on next job candidate
2016-10-19 16:55:49 +02:00
Raul Ochoa
71d32e003b
Better debug
2016-10-19 11:46:02 +02:00
Raul Ochoa
4daa39bd2c
Start scheduler from host-scheduler
2016-10-19 11:45:48 +02:00
Raul Ochoa
e26bed2e66
Move status close to entity
2016-10-19 10:45:37 +02:00
Raul Ochoa
ca3d71ea48
Tasks with their own entity
...
- Use a list of tasks and keep an index per user.
- Removes WAITING status.
TODO: improve candidate selection.
2016-10-19 10:43:24 +02:00
Raul Ochoa
6c232a1fd0
Discard numeric status
2016-10-19 10:40:03 +02:00
Raul Ochoa
51ac1a3ab7
Remove TODO as it is already done
2016-10-19 10:38:39 +02:00
Raul Ochoa
ac65c1c39a
Rename
2016-10-19 10:36:13 +02:00
Raul Ochoa
3a57331a54
Delegate job scheduling
...
There is a host scheduler managing the host locking.
When it can acquire a lock over the host it will delegate
all the tasks related to that host to the same scheduler.
This scheduler will take care of how many jobs it will submit,
and in which order. It's also responsible for guaranteeing the
execution order per user.
Capacity planner dictates how many jobs can be run at the
same time in a given host. There are two simple strategies:
1. Infinity: it will attempt to run as many jobs as different users.
2. One: it will run just one job at the same time.
Missing things:
- Handle lock renewal failures.
- Fair scheduling for pending/waiting users.
- Capacity based on real resources.
2016-10-18 20:43:15 +02:00
Raul Ochoa
dce051d52b
Make leader locker to emit on renewal errors
2016-10-18 20:34:22 +02:00
Raul Ochoa
d1e3be2e22
Do not emit job:status from batch
2016-10-18 20:19:44 +02:00
Raul Ochoa
ef6cd24bf3
Correct debug
2016-10-18 11:18:11 +02:00
Raul Ochoa
ac7bad43a5
Lock by host instead of host + user
...
- Host lock only released if there are no pending jobs.
- Will allow to schedule jobs by host.
2016-10-17 19:03:55 +02:00
Raul Ochoa
761fbe5205
Separate job draining from processing
2016-10-17 18:44:47 +02:00
Raul Ochoa
a8e03f01c9
Add debug information in Jobs Queue
2016-10-17 18:44:37 +02:00
Raul Ochoa
c6e906d3ef
Use same debug group
2016-10-17 18:44:28 +02:00
Raul Ochoa
3772b1c896
Log created at time and waiting time for fallback jobs
2016-10-17 16:12:02 +02:00
Raul Ochoa
803a4b533f
Add some notes about redis data structures for batch queries
2016-10-17 16:00:30 +02:00
Raul Ochoa
66d1c18941
Default to 64 queued jobs as max
2016-10-17 15:23:53 +02:00
Raul Ochoa
cdde1be29e
Re-use redis pool as much as possible
2016-10-17 15:02:34 +02:00
Raul Ochoa
431f72873a
250 queued jobs as default limit
2016-10-17 13:00:23 +02:00
Raul Ochoa
180ba19df5
Fix host queue seeking
2016-10-17 12:51:01 +02:00
Raul Ochoa
39bb7e6249
Lock resources by host+user
...
This allows to run multiple jobs in parallel but guarantees order by user
2016-10-17 12:34:52 +02:00
Raul Ochoa
8b9a30eb75
Queue seeker was not _finding_ queues when only one present
2016-10-17 12:27:06 +02:00
Raul Ochoa
c62fe29160
Load config on object creation
2016-10-17 10:51:50 +02:00
Raul Ochoa
6179327486
Rename
2016-10-14 13:10:27 +02:00
Raul Ochoa
b8c63f5ffc
Rename
2016-10-14 12:56:41 +02:00
Raul Ochoa
5bb7d8fa1c
Merge branch 'master' into batch-user-queues
2016-10-14 12:33:37 +02:00
Raul Ochoa
a8802d1163
redis-distlock acquires and releases redis clients by operation
2016-10-13 13:48:06 +02:00
Raul Ochoa
05eda290be
Create one client for queue-seeker and share per seek cycle
2016-10-13 13:09:56 +02:00
Raul Ochoa
1e442b37ab
Allow to set a max number of queued jobs per user
2016-10-12 22:40:35 +02:00
Raul Ochoa
1f038ac1f4
Moves from host queues to user queues
...
- Existing jobs are moved before start processing them.
- Uses a new queue prefix to avoid collisions.
- Pub/Sub also changes communication channel.
- Job subscriber emits user+host on new jobs.
- Batch processor is faulty. See TODO in batch.js.
2016-10-12 21:32:29 +02:00
Raul Ochoa
f7d1f9426c
Use constants for queues
2016-10-12 17:53:03 +02:00
Raul Ochoa
189aff2aa9
Only log message on empty queue
2016-10-12 17:42:46 +02:00
Raul Ochoa
6bb2abde0d
Only start lock renewal on lock acquisition
2016-10-12 17:01:24 +02:00
Raul Ochoa
b86f82d3ca
Batch.stop removes all listeners
2016-10-12 16:43:18 +02:00
Raul Ochoa
75fc21241f
Locker TTL is configured
2016-10-12 13:11:20 +02:00
Raul Ochoa
88f6d46d00
Reuse existing redlock
...
Return not connected clients to pool
2016-10-12 13:10:18 +02:00
Raul Ochoa
3f1b67993c
Locker keep refreshing lock by itself
2016-10-12 12:30:13 +02:00
Raul Ochoa
67566c1d0e
Callback in subscriber unsubscribe errors
2016-10-12 12:29:54 +02:00
Raul Ochoa
c74f9bcce0
More aggressive on seek interval
2016-10-12 12:29:18 +02:00
Raul Ochoa
98185e55cf
Remove Job Queue Pool and use internal structure
...
- We don't need to create a different job queue per host.
- Batch locks on message instead of dequeue.
2016-10-12 12:26:50 +02:00
Raul Ochoa
e1d0ffc7dd
Logger set to fatal on test environment
2016-10-12 01:40:35 +02:00
Raul Ochoa
22d8e48f53
Only lock on dequeue
2016-10-12 00:10:40 +02:00
Raul Ochoa
81393190f7
Add callback to jobseeker result from initial load
2016-10-11 19:59:11 +02:00
Raul Ochoa
8bc52b09cf
Remove console call
2016-10-11 19:46:27 +02:00
Raul Ochoa
dc1a23e886
Add error handler for channel subscriber
2016-10-11 19:45:43 +02:00
Raul Ochoa
2822b68198
onJobHandler receives host with job
...
Queue seeker only returns hosts, not mixing responsibilities
2016-10-11 19:45:26 +02:00
Raul Ochoa
01cf6f244f
Share redis pool for pubsub
2016-10-11 19:41:58 +02:00
Raul Ochoa
ecc6bf0400
Use real on message handler
2016-10-11 19:04:12 +02:00
