Daniel García Aubert
44ff21b7c0
Fix eslint errors
2019-12-26 16:10:41 +01:00
Daniel García Aubert
cc535b13d2
Use asert.strict mode
2019-12-26 14:23:21 +01:00
Daniel García Aubert
5a96dbb59c
Run eslint --fix
2019-12-23 18:19:08 +01:00
Daniel García Aubert
06a47a8afc
Changed folder structure to reflect application functionallity. Renamed files using hyphens instead of underscore to have a more consistent naming across the whole project
2019-10-03 18:24:39 +02:00
Daniel García Aubert
46190008f5
Remove table cache
2019-07-24 17:44:04 +02:00
Daniel García Aubert
2208fb9764
Implement a cancel backend through query instead of node-postgres protocol
2019-05-14 15:39:54 +02:00
Daniel García Aubert
2e2e233b68
Use 'bunyan' logger instead of 'debug' to log debug messages
2019-04-04 14:31:41 +02:00
Daniel García Aubert
a616982ec3
Use strict mode
2018-10-24 15:42:33 +02:00
Rafa de la Torre
632e6818fa
Test that COPY uses db_batch_port
2018-07-20 12:04:06 +02:00
Eneko Lakasta
3891d93b8d
remove unnecessary require
2018-06-05 17:59:46 +02:00
Eneko Lakasta
4993f8a956
use job configuration instead of user metadata service in test
2018-06-05 17:56:29 +02:00
Eneko Lakasta
03e484c5da
FIX tests. Create jobs should return 403 if auth fails
2018-06-05 15:49:15 +02:00
Eneko Lakasta
8515c2cc31
use job configuration instead of user metadata service
2018-06-04 11:28:59 +02:00
Eneko Lakasta
46e3a87f41
add some auth TODOs
2018-05-30 18:15:35 +02:00
Eneko Lakasta
cf8bf6e5e6
get DB configuration from job instead from metadata in Job Canceller
2018-05-30 17:36:55 +02:00
Eneko Lakasta
017dc69c02
add auth params to test. Is this OK?
2018-05-28 17:38:04 +02:00
Simon Martín
c351360085
ensuring job_id on jobBackend.create and updating test
2018-02-12 10:39:58 +01:00
Simon Martín
dd07a21c98
unsubscribe on batch publisher test
2018-02-08 17:59:28 +01:00
Raul Ochoa
3e536b822e
Remove unnecessary destroyOnError option
2017-08-08 18:21:10 +02:00
Rafa de la Torre
ae304999c7
Add a few integration tests #422
2017-06-27 16:38:07 +02:00
Raul Ochoa
ea9454d87c
Add test to validate old jobs will get discovered
2017-04-04 15:42:34 +02:00
Daniel García Aubert
25a291bb38
Improve waek assertion
2017-04-04 09:50:45 +02:00
Daniel García Aubert
36d5dc1719
Queue seeker refactor:
...
- Extract from job subcriber everything related to queue dicovering
- Job Queue is responsible of queue dicovering and exposes functions to scan user queues.
- Batch creates an interval to consume queues
- Now Batch.start() does:
* Finds existing queues from metadata, adding to queue index the ones that are not indexed yet
* Binds callback to process jobs for every queue found.
* Inits queue consumer interval
* Subscribes to PUB/SUB channel
2017-04-03 19:26:21 +02:00
Daniel García Aubert
8012fe26aa
Avoid scan behaviour to discover active queues of user's jobs, now keeps an index to know which queues are available
2017-03-31 14:30:33 +02:00
Daniel García Aubert
7faa78c3f4
skip state dependant test
2016-11-21 15:31:43 +01:00
Daniel García Aubert
78f5706726
Avoid to use scan command to search work in progress queues
2016-11-03 14:22:43 +01:00
Daniel García Aubert
af75bbda50
Rename
2016-10-28 12:50:33 +02:00
Daniel García Aubert
1f657a4f94
Ensure test is isolated
2016-10-28 12:24:23 +02:00
Daniel García Aubert
0085b8ee3d
List users with work in progress jobs
2016-10-27 20:42:49 +02:00
Daniel García Aubert
f65208ba0d
Add listWorkInProgressJobByUser function
2016-10-27 18:43:28 +02:00
Daniel García Aubert
8f65e6b16c
Add test
2016-10-27 18:24:39 +02:00
Raul Ochoa
16e9e709b8
Cancel with user statement_timeout limit from redis
2016-10-21 14:35:24 +02:00
Raul Ochoa
7563868514
Re-insert into the tree if there was a user in done state that gets a new task
2016-10-21 11:42:27 +02:00
Raul Ochoa
62cfe974b7
Use constants for users
2016-10-21 11:10:17 +02:00
Raul Ochoa
66a1c33f96
Simplify listener subscription logic
...
Always remove pending listeners on acquire call.
Always register add and release listeners on acquire.
2016-10-20 23:47:39 +02:00
Raul Ochoa
e4d54e9ab7
Fix condition to pick next candidate
2016-10-20 20:16:57 +02:00
Raul Ochoa
80d2e190ad
Fix test, use fixed to replace one and infinity
2016-10-19 18:47:55 +02:00
Raul Ochoa
1ee0878631
Scheduler uses a red–black tree to decide on next job candidate
2016-10-19 16:55:49 +02:00
Raul Ochoa
1e490be0a1
Don't rely on batch.on(job:status)
2016-10-18 20:18:49 +02:00
Raul Ochoa
cdde1be29e
Re-use redis pool as much as possible
2016-10-17 15:02:34 +02:00
Raul Ochoa
8b9a30eb75
Queue seeker was not _finding_ queues when only one present
2016-10-17 12:27:06 +02:00
Raul Ochoa
1f038ac1f4
Moves from host queues to user queues
...
- Existing jobs are moved before start processing them.
- Uses a new queue prefix to avoid collisions.
- Pub/Sub also changes communication channel.
- Job subscriber emits user+host on new jobs.
- Batch processor is faulty. See TODO in batch.js.
2016-10-12 21:32:29 +02:00
Raul Ochoa
b86f82d3ca
Batch.stop removes all listeners
2016-10-12 16:43:18 +02:00
Raul Ochoa
c21f373291
Tests for locker
2016-10-12 13:15:55 +02:00
Raul Ochoa
e4b1711e8e
pub/sub package
2016-10-11 18:28:46 +02:00
Raul Ochoa
b07e2d3de5
Merge pull request #346 from CartoDB/fix-batch-api-test
...
Removed duplicated config
2016-10-04 13:01:46 +02:00
Raul Ochoa
20edddc721
Remove user indexer dependency from tests
2016-08-30 19:06:59 +02:00
Raul Ochoa
d33fe5ac21
Stop indexing jobs per user
...
Removes .list() from job backend
2016-08-30 19:01:23 +02:00
Raul Ochoa
05ada98124
Remove .list() from job service
2016-08-30 18:51:51 +02:00
Raul Ochoa
4c8d734bbf
Remove update method from job service
2016-08-30 17:46:32 +02:00
