cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,745 Commits 1 Branch 118 Tags 9.7 MiB
020c64af26
Commit Graph

826 Commits

Author SHA1 Message Date
Raul Ochoa
ba4499c196 Don't use extra function 2016-12-19 15:13:12 +01:00
Raul Ochoa
29e6c347d8 Test all 2016-12-19 14:31:39 +01:00
Raul Ochoa
f62ba29ac7 Dont debug that much 2016-12-19 14:31:25 +01:00
Raul Ochoa
59fba7c676 Only test regressions suite for travis debugging 2016-12-19 12:05:56 +01:00
Raul Ochoa
c0e51c4c1e Explicit callback 2016-12-19 11:11:40 +01:00
Raul Ochoa
7dc78e7c5a More debug 2016-12-19 11:11:31 +01:00
Raul Ochoa
6c8929079c Add debugging info in assert.response 2016-12-19 10:50:05 +01:00
Raul Ochoa
32aaccb33e Merge branch 'master' into node-v6 2016-12-16 15:24:26 +01:00
Daniel García Aubert
05cfdae7f8 Fix "after" hook in acceptance test to not throw uncaught exception 2016-12-13 17:21:57 +01:00
Daniel García Aubert
acafd84acf Reverts: fix backend crash to not throw an uncaught exception 2016-12-13 17:17:53 +01:00
Daniel García Aubert
ac65b8a68d Fix backend crash test behavior 2016-12-13 11:10:14 +01:00
Raul Ochoa
779bbc4ddb Drop if exists 2016-12-09 12:05:49 +01:00
Raul Ochoa
3747fed78f query-returning suite 2016-12-09 11:26:45 +01:00
Raul Ochoa
b85ad66288 app-configuration suite 2016-12-09 11:19:01 +01:00
Raul Ochoa
67ba027ad5 COPY statements suite 2016-12-09 11:10:08 +01:00
Raul Ochoa
a473389f0c Skipfields suite 2016-12-09 11:02:37 +01:00
Raul Ochoa
15df082486 System queries 2016-12-09 10:56:50 +01:00
Raul Ochoa
b9b58ccd1a Pagination suite 2016-12-09 10:56:44 +01:00
Raul Ochoa
7a666a8f57 Rename untitle_table_4 dependant objects 2016-12-09 10:28:51 +01:00
Daniel García Aubert
c65124fc37 Fix backend crash to not throw an uncaught exception 2016-12-07 16:16:03 +01:00
Joshua Chamberlain
a1427c26bf Respect host and port settings for postgres and redis config 2016-11-30 11:46:47 -05:00
Daniel García Aubert
b0f39d16ab Remove commented code 2016-11-24 17:39:14 +01:00
Daniel García Aubert
4d97f77e14 Remove console.log 2016-11-24 17:38:54 +01:00
Daniel García Aubert
2bc46ca034 Revert change 2016-11-24 17:28:59 +01:00
Daniel García Aubert
48469aaf0d Make some test less demanding 2016-11-24 16:53:15 +01:00
Daniel García Aubert
b2a6b319e4 Unskip test 2016-11-24 16:45:00 +01:00
Daniel García Aubert
99a8b794dc Dont use setInterval in work in progress test suite 2016-11-24 16:21:28 +01:00
Daniel García Aubert
9a63006719 Validate job response using the internal reference 2016-11-24 15:40:09 +01:00
Daniel García Aubert
14dabcafa2 Use batch-test-client for job timing suite 2016-11-24 14:37:03 +01:00
Daniel García Aubert
b54c9dce43 Use custom assert 2016-11-24 14:11:43 +01:00
Daniel García Aubert
a0eea98940 Implement custom assert for batch test client 2016-11-24 13:25:14 +01:00
Daniel García Aubert
04ba1279da Use test-client and batch-test-client for callback template suite 2016-11-24 13:10:29 +01:00
Daniel García Aubert
363a550633 Fix error message 2016-11-24 13:09:33 +01:00
Daniel García Aubert
ce51fb8860 Set tiemout for test client to 5s 2016-11-24 00:15:17 +01:00
Daniel García Aubert
3268dcd43c Put into same file batch use cases 2016-11-23 23:48:23 +01:00
Daniel García Aubert
4b04ccc222 Remove test that its functionality is no longer supported 2016-11-23 23:42:03 +01:00
Daniel García Aubert
4c3b56c529 Use batch test client for use case 8 2016-11-23 23:39:15 +01:00
Daniel García Aubert
c459303748 Use batch test client for use case 7 2016-11-23 23:33:30 +01:00
Daniel García Aubert
f89fb0e683 Remove test that its functionality is no longer supported 2016-11-23 23:30:01 +01:00
Daniel García Aubert
0176db54ac Use batch test client for use case 3 2016-11-23 23:23:34 +01:00
Daniel García Aubert
7be92d02d5 Use batch test client for use case 2 2016-11-23 23:23:24 +01:00
Daniel García Aubert
0e56942507 Use batch test client for use case 1 2016-11-23 23:23:14 +01:00
Daniel García Aubert
b76120c41b Use test bacth client 2016-11-23 20:47:18 +01:00
Daniel García Aubert
f7a41d3a0d Try to cancel a job 2016-11-23 20:46:19 +01:00
Daniel García Aubert
e29b35a98d Assert if error 2016-11-23 20:09:38 +01:00
Daniel García Aubert
e6bc22ee5d Fix jshint typo 2016-11-23 19:29:03 +01:00
Daniel García Aubert
86e9a3f122 Use batch client test for fallback suit test 2016-11-23 19:22:24 +01:00
Daniel García Aubert
5eca803fc8 Add support to get some specific job status 2016-11-23 19:20:44 +01:00
Daniel García Aubert
93a8fcc2f9 Make customizable timeout value for request 2016-11-23 12:30:16 +01:00
Daniel García Aubert
37b9eda6c3 Fix typo 2016-11-22 18:33:46 +01:00
Daniel García Aubert
23054034b4 Add mechanism to avoid infiniy request to check job status 2016-11-22 14:26:21 +01:00
Daniel García Aubert
bbe80f0810 Remove useless bacth service 2016-11-22 14:09:16 +01:00
Daniel García Aubert
687e6218b4 Make polling to check work-in-progress job 2016-11-22 14:06:32 +01:00
Daniel García Aubert
80798e2231 Skip test that use queue with assert.response 2016-11-22 12:52:26 +01:00
Daniel García Aubert
172c9d91e8 Set request timeout to 4sec 2016-11-22 11:54:22 +01:00
Daniel García Aubert
d152aa7ec9 Set request timeout to 5sec 2016-11-22 11:41:35 +01:00
Daniel García Aubert
549d7dcdf7 Remove skip 2016-11-22 11:37:35 +01:00
Daniel García Aubert
0f5a67833f Set default timeout for assert response to 4 sec 2016-11-21 18:29:46 +01:00
Daniel García Aubert
eb67994ba8 skip state dependant test 2016-11-21 18:17:22 +01:00
Daniel García Aubert
670633655e skip state dependant test 2016-11-21 17:44:27 +01:00
Daniel García Aubert
dba3fd24ec skip state dependant test 2016-11-21 17:27:57 +01:00
Daniel García Aubert
7faa78c3f4 skip state dependant test 2016-11-21 15:31:43 +01:00
Daniel García Aubert
78f5706726 Avoid to use scan command to search work in progress queues 2016-11-03 14:22:43 +01:00
Daniel García Aubert
802c93dcee Moved work in progress test 2016-11-02 13:03:22 +01:00
Daniel García Aubert
bd716867fa Revert last env test changes 2016-11-02 13:02:29 +01:00
Daniel García Aubert
041b69a1f6 Isolate failing test in CI 2016-11-02 12:47:29 +01:00
Daniel García Aubert
e459494123 Be more accurate on query sleep times 2016-11-02 11:39:05 +01:00
Daniel García Aubert
aa1527b371 Use clear work-in-progress after finishing jobs 2016-10-28 15:58:41 +02:00
Daniel García Aubert
4a64d37c6c Add work in progrees list endpoint 2016-10-28 15:08:42 +02:00
Daniel García Aubert
af75bbda50 Rename 2016-10-28 12:50:33 +02:00
Daniel García Aubert
1f657a4f94 Ensure test is isolated 2016-10-28 12:24:23 +02:00
Daniel García Aubert
0085b8ee3d List users with work in progress jobs 2016-10-27 20:42:49 +02:00
Daniel García Aubert
f65208ba0d Add listWorkInProgressJobByUser function 2016-10-27 18:43:28 +02:00
Daniel García Aubert
8f65e6b16c Add test 2016-10-27 18:24:39 +02:00
Raul Ochoa
16e9e709b8 Cancel with user statement_timeout limit from redis 2016-10-21 14:35:24 +02:00
Raul Ochoa
7563868514 Re-insert into the tree if there was a user in done state that gets a new task 2016-10-21 11:42:27 +02:00
Raul Ochoa
62cfe974b7 Use constants for users 2016-10-21 11:10:17 +02:00
Raul Ochoa
66a1c33f96 Simplify listener subscription logic 
Always remove pending listeners on acquire call.
Always register add and release listeners on acquire.
 2016-10-20 23:47:39 +02:00
Raul Ochoa
e4d54e9ab7 Fix condition to pick next candidate 2016-10-20 20:16:57 +02:00
Raul Ochoa
80d2e190ad Fix test, use fixed to replace one and infinity 2016-10-19 18:47:55 +02:00
Raul Ochoa
604e28533c Fix and improve test 2016-10-19 16:58:57 +02:00
Raul Ochoa
372c9f5511 Basic test to test scheduler happy case 2016-10-19 16:57:10 +02:00
Raul Ochoa
5030fddc9c Allow to override test client 2016-10-19 16:56:43 +02:00
Raul Ochoa
1ee0878631 Scheduler uses a red–black tree to decide on next job candidate 2016-10-19 16:55:49 +02:00
Raul Ochoa
2853c7b0a7 Fix status check 2016-10-19 10:36:27 +02:00
Raul Ochoa
1e490be0a1 Don't rely on batch.on(job:status) 2016-10-18 20:18:49 +02:00
Raul Ochoa
a29f847767 Don't rely on batch.on(job:status) 2016-10-18 20:05:57 +02:00
Raul Ochoa
a1400e956d Don't rely on batch.on(job:status) 2016-10-18 20:01:11 +02:00
Raul Ochoa
66d1c18941 Default to 64 queued jobs as max 2016-10-17 15:23:53 +02:00
Raul Ochoa
cdde1be29e Re-use redis pool as much as possible 2016-10-17 15:02:34 +02:00
Raul Ochoa
4dad54d004 Add test to validate query order by user 2016-10-17 12:33:49 +02:00
Raul Ochoa
b8a57460dc Handle stop gracefully 2016-10-17 12:28:01 +02:00
Raul Ochoa
8b9a30eb75 Queue seeker was not _finding_ queues when only one present 2016-10-17 12:27:06 +02:00
Raul Ochoa
f6dc991ab5 Allow to override configuration per API call 2016-10-17 10:52:35 +02:00
Raul Ochoa
cb23b7f46c Make test finish quicker 2016-10-17 10:52:07 +02:00
Raul Ochoa
5bb7d8fa1c Merge branch 'master' into batch-user-queues 2016-10-14 12:33:37 +02:00
Raul Ochoa
b510a3684d Add option to skip script downloading 2016-10-14 12:28:43 +02:00
Raul Ochoa
1e442b37ab Allow to set a max number of queued jobs per user 2016-10-12 22:40:35 +02:00
Raul Ochoa
1f038ac1f4 Moves from host queues to user queues 
- Existing jobs are moved before start processing them.
 - Uses a new queue prefix to avoid collisions.
 - Pub/Sub also changes communication channel.
 - Job subscriber emits user+host on new jobs.
 - Batch processor is faulty. See TODO in batch.js.
 2016-10-12 21:32:29 +02:00
Raul Ochoa
aa68dff24b Use constants for tests 2016-10-12 18:44:46 +02:00
Raul Ochoa
b86f82d3ca Batch.stop removes all listeners 2016-10-12 16:43:18 +02:00
Raul Ochoa
c21f373291 Tests for locker 2016-10-12 13:15:55 +02:00
Raul Ochoa
167ecc6965 Set NODE_ENV environment var to test 2016-10-12 01:41:01 +02:00
Raul Ochoa
611508c654 Hide queue seeker behind job subscriber 2016-10-11 19:01:39 +02:00
Raul Ochoa
e4b1711e8e pub/sub package 2016-10-11 18:28:46 +02:00
Raul Ochoa
f036e29b8f Make test stable 2016-10-10 19:47:18 +02:00
Raul Ochoa
41d7823227 Drop table if exists 2016-10-10 19:46:53 +02:00
Raul Ochoa
a0ace8d034 Use name in test 2016-10-10 19:46:39 +02:00
Raul Ochoa
66820a67bb Make possible to specify a name for batch 2016-10-10 19:46:07 +02:00
Raul Ochoa
f4950f73c0 Add test to validate multiple consumers query order scenario 2016-10-10 15:56:04 +02:00
Raul Ochoa
ece95afc95 Add test client to run plain queries 2016-10-10 15:37:46 +02:00
Raul Ochoa
46579093dd Fix test scenario description 2016-10-10 15:37:19 +02:00
Raul Ochoa
8ac656492f Move to support directory 2016-10-10 15:22:50 +02:00
Raul Ochoa
faa02a79b9 Test for 578c43b1a8 2016-10-10 15:17:05 +02:00
Raul Ochoa
8a4f54bb87 Allow users to set max statement_timeout for their queries 2016-10-10 12:01:36 +02:00
Raul Ochoa
2ac39b5748 Move all batch tests to their own directory 2016-10-06 11:53:56 +02:00
Raul Ochoa
b07e2d3de5 Merge pull request #346 from CartoDB/fix-batch-api-test 
Removed duplicated config
 2016-10-04 13:01:46 +02:00
Raul Ochoa
0e03bcd1d3 Delete table so suite ends clear 2016-09-26 18:28:15 +02:00
Raul Ochoa
2712d81d50 Remove unused server utils 2016-09-26 18:13:08 +02:00
Raul Ochoa
b29358a0ed Remove old response from assert 2016-09-26 18:11:40 +02:00
Raul Ochoa
96fc32b24a Concurrency test with assert.response 2016-09-26 18:10:09 +02:00
Raul Ochoa
1a20ca0bc3 Use listener for test 2016-09-26 18:09:50 +02:00
Raul Ochoa
abc2f130c9 Migrate to express 4.x series 
- Remove express logger
 - Error handler responds with application/[json|javascript]
 - Fix all tests relying on res.headers
 - assert.response based on request module
 2016-09-26 18:09:27 +02:00
Raul Ochoa
a4aba62e9c Fix name clash for err vars 2016-09-26 14:51:41 +02:00
Raul Ochoa
90e4600a6c Rename vars to avoid name clash 2016-09-26 14:42:50 +02:00
Raul Ochoa
3dce6af20f Remove unused param 2016-09-26 14:42:35 +02:00
Raul Ochoa
9bd240ef6c Remove unused params 2016-09-26 14:42:24 +02:00
Raul Ochoa
26400342f3 Remove unused param 2016-09-26 14:42:14 +02:00
Raul Ochoa
c47dde1a29 Short hand method 2016-09-26 14:42:02 +02:00
Raul Ochoa
c9846a2437 Change callback signature in assert.response 2016-09-26 14:37:40 +02:00
Raul Ochoa
0d54604428 Remove setMaxListeners from tests 2016-09-14 21:24:22 +02:00
Raul Ochoa
84b80547c2 Remove settings' app_root 2016-09-14 21:02:56 +02:00
Raul Ochoa
f92d50cccf Rename from app to server 
Removes app_root dependency in requires
 2016-09-14 20:54:53 +02:00
Daniel García Aubert
17411f2bb6 Avoids to delete empty keys 2016-09-12 16:00:33 +02:00
Raul Ochoa
461728d3e2 Remove user indexer 2016-08-30 19:08:06 +02:00
Raul Ochoa
20edddc721 Remove user indexer dependency from tests 2016-08-30 19:06:59 +02:00
Raul Ochoa
d33fe5ac21 Stop indexing jobs per user 
Removes .list() from job backend
 2016-08-30 19:01:23 +02:00
Raul Ochoa
05ada98124 Remove .list() from job service 2016-08-30 18:51:51 +02:00
Raul Ochoa
ba0f2f1066 Remove endpoint to retrieve jobs list 2016-08-30 18:49:01 +02:00
Raul Ochoa
4c8d734bbf Remove update method from job service 2016-08-30 17:46:32 +02:00
Raul Ochoa
ad0d101bfd Remove patch/put endpoints for jobs 2016-08-30 17:43:34 +02:00
Daniel García Aubert
085abb3446 Removed duplicated config 2016-08-01 11:48:11 +02:00
Daniel García Aubert
f3f45bb4e2 Fixed lint issues 2016-07-22 17:11:21 +02:00
Daniel García Aubert
587ab6a5d9 Extratcs redis clean method for batch keys 2016-07-22 17:05:01 +02:00
Daniel García Aubert
7fd5e42213 Removed timeouts for batch tests and skipped again a failing one in job callback template 2016-07-22 14:41:26 +02:00
Daniel García Aubert
2e8382eeff Fixed issues in batch tests: 
- Wait for batch is ready before testing it
 - Remove batch keys in redis after testing every suit
 - Fixed issue creating jobs with same id in job service integration test
 2016-07-22 14:30:55 +02:00
Daniel García Aubert
438d61fb70 Set timeout to 30s 2016-07-20 19:50:28 +02:00
Daniel García Aubert
027e7b8768 Removed unused dependency 2016-07-20 19:42:53 +02:00
Daniel García Aubert
1625d8a556 Fixed timeout in job canceller test 2016-07-20 19:41:55 +02:00
Daniel García Aubert
457414c2fe Skipping integration test 2016-07-20 18:27:54 +02:00
Daniel García Aubert
39a9f950b0 Added integration test for multiquery jobs 2016-07-20 16:38:57 +02:00
Daniel García Aubert
b33caf6126 Moved batch integration test to a separated folder 2016-07-19 19:56:12 +02:00
Daniel García Aubert
8d4dbd85df Implmented integration tests for publisher, runner and canceller of batch service. 2016-07-19 19:42:49 +02:00
Daniel García Aubert
fe11828046 Implemented integration test for job service 2016-07-19 16:20:53 +02:00
Daniel García Aubert
2a4364142d Added integration test for job backend 2016-07-19 13:08:52 +02:00
Daniel García Aubert
ddd5a0bd19 Added integration folder to tests 2016-07-18 14:49:57 +02:00
Daniel García Aubert
395635d246 Merge branch 'master' into fix-publisher-connection 2016-07-11 17:49:11 +02:00
Raul Ochoa
3ebbd9f7c4 Skip tables with no updated_at registered in cdb_tablemetadata 2016-07-11 16:39:12 +02:00
Daniel García Aubert
ccff602bbf Merge branch 'master' into fix-publisher-connection 2016-07-07 16:07:41 +02:00
Raul Ochoa
368fe2403e Allow to setup more than one domain to validate oauth against 2016-07-07 14:20:36 +02:00
Daniel García Aubert
5eaad4d5d9 Uses redis-mpool for pubsub in Batch API 2016-07-07 14:14:46 +02:00
Daniel García Aubert
74d83a457e Now batch publisher sends a ping to server before publishing and create a new connection if error. 
Batch publisher and subscriber logs (if debug enabled) both outcoming and incoming messages to give more visibility.
 2016-07-07 10:44:17 +02:00
Raul Ochoa
63d3cd59e2 Skip test for travis 2016-06-30 20:02:43 +02:00
Raul Ochoa
b4ee2effbd Fix line too long error 2016-06-30 19:49:34 +02:00
Raul Ochoa
b9af5e7846 Create table in before 2016-06-30 19:40:36 +02:00
Raul Ochoa
8a57a64961 Output actual response when it fails 2016-06-30 19:14:21 +02:00
Raul Ochoa
1f8f5e2c57 Use another name for test table 2016-06-30 19:12:45 +02:00
Raul Ochoa
dc5807b790 Do not validate expected output 2016-06-30 18:55:19 +02:00
Raul Ochoa
74964bc696 Be more clear about what is not matching 2016-06-30 18:31:58 +02:00
Raul Ochoa
51747879d6 Reorder tests 2016-06-30 18:16:52 +02:00
Raul Ochoa
be0f059f01 Add <%= job_id %> template support for onerror and onsuccess fallback queries 2016-06-30 17:41:02 +02:00
Daniel García Aubert
a1f31df92e Now Batch API broadcast to other APIs everytime that re-enqueues a multiple-query job 2016-06-29 18:29:53 +02:00
Raul Ochoa
85fa9d3c2b Merge branch 'master' into batch-onerror-template 
Conflicts:
	NEWS.md
 2016-06-29 16:25:05 +02:00
Raul Ochoa
a3117a2f01 Add <%= error_message %> template support for onerror fallback queries 2016-06-29 14:22:23 +02:00
Raul Ochoa
1d8f5539a7 Adds start and end time for batch queries with fallback 2016-06-29 13:56:45 +02:00
Daniel García Aubert
c7a9f59320 Added CDB_OverviewsSupport as remote sql script in fixtrues 2016-06-22 16:38:30 +02:00
Daniel
58055080c9 Merge pull request #311 from CartoDB/309-skipped-status 
Fixes #309, added skipped status to fallback-jobs
 2016-06-08 11:12:59 +02:00
Daniel García Aubert
2a2127f4e1 Improved falback job to get the status of last finished query 2016-06-02 18:20:53 +02:00
Raul Ochoa
299490f46f Put all remote sql script together 2016-06-01 10:52:54 +02:00
Daniel García Aubert
481a82500b Refactored fallback jobs 2016-06-01 10:39:01 +02:00
Raul Ochoa
5710d83908 Output postgresql server version 2016-06-01 10:25:16 +02:00
Daniel García Aubert
1a0e2b681b Fixes #309, added skipped status to fallback-jobs 2016-05-26 17:37:37 +02:00
Daniel García Aubert
e079491a7e Fixed issue with status transition fallback jobs 2016-05-25 17:00:27 +02:00
Raul Ochoa
23228b2d73 Payload size validates multiple queries and fallback queries 
It uses a middleware to check the body size
 2016-05-24 14:28:00 +02:00
Daniel García Aubert
2d86fc2ce3 In fallback jobs, used fallback_status property instead of an array of statuses 2016-05-23 15:05:14 +02:00
Daniel García Aubert
052cb0796d Modified fallback tests in order to use our custom assert.response 2016-05-20 14:42:53 +02:00
Daniel García Aubert
e04787a070 Fixed missing failed_reason in failing jobs 2016-05-19 17:02:48 +02:00
Daniel García Aubert
cf7dff4696 Removed commented code 2016-05-19 14:13:54 +02:00
Daniel García Aubert
62c8a9d315 Refactor and fixed issue, when cancelling a fallback job, it is not cancelled properly 2016-05-19 14:10:18 +02:00
Daniel García Aubert
5c4ced34e6 Added acceptance test for job cancelling 2016-05-18 13:25:08 +02:00
Daniel García Aubert
b94a9c7c0a Fixed conflicts 2016-05-18 11:48:39 +02:00
Daniel
8c8da8e39c Merge pull request #301 from CartoDB/job-model-refactor 
Job model refactor
 2016-05-18 11:38:41 +02:00
Daniel García Aubert
6acc02e998 Improved fallback setStatus method and added acceptance test 2016-05-17 18:51:02 +02:00
Daniel García Aubert
b2eac6b880 Implemented test for fallback jobs 2016-05-17 01:00:27 +02:00
Daniel García Aubert
be3d9d2316 Fallback jobs WIP 2016-05-16 13:49:56 +02:00
Daniel García Aubert
d2d3ba8159 Passed tests 2016-05-16 01:22:47 +02:00
Raul Ochoa
4e3238c2e8 Fix control flow when using OGR with _needSRS=true and empty queries 
Continue on empty results as srid and geom type are not critical
when there are no results

Fixes #299
 2016-05-12 17:21:07 +02:00
Juan Ignacio Sánchez Lara
e979b79f49 gpkg format #291 2016-05-10 18:42:23 +02:00
Juan Ignacio Sánchez Lara
7b9911416b geopackage export test #291 2016-05-10 18:01:50 +02:00
Juan Ignacio Sánchez Lara
c82cafb1ed Geopackage format support #291 2016-04-27 16:56:06 +02:00
Daniel García Aubert
5de931daa5 Improver error message in job max payload limit. 2016-04-25 11:18:30 +02:00
Daniel García Aubert
832695031b Improved tests for query size limit 2016-04-18 18:05:03 +02:00
Daniel García Aubert
1dcbb1afdf Improved error message 2016-04-18 16:24:52 +02:00
Daniel García Aubert
790b9c6124 Fixed typos 2016-04-18 15:44:48 +02:00
Daniel García Aubert
7cc7482944 Implemented limit to query size in Batch API. 2016-04-18 15:30:16 +02:00
Daniel
423e729e11 Merge pull request #283 from CartoDB/batch-improve-startup 
Improved initialization in Batch API
 2016-04-05 15:46:21 +02:00
Daniel García Aubert
7121b3b800 Passed test 2016-04-04 20:05:58 +02:00
Daniel García Aubert
b62edcb64f Refactored batch-api startup. Isolated code related to queue discovering, api much clear and concise. 2016-04-04 16:05:33 +02:00
dgaubert
e8a44136c5 Now Batch API looks for queues with jobs to consume at service initialization time 2016-03-31 17:37:35 +02:00
dgaubert
fd9bfe277e Implemented multiquery jobs for Batch API 2016-03-31 12:39:03 +02:00
Daniel García Aubert
a32fa7b108 Implemented new test for multiquery jobs 2016-03-18 18:12:01 +01:00
Daniel García Aubert
056f22b156 Implemented multi-jobs, user is able to send an array of jobs and batch service will run them in series 2016-03-18 14:57:18 +01:00
Raul Ochoa
66cd1f400f Skip query tables cache for authenticated requests 2016-03-10 19:20:56 +01:00
Alejandro Martínez
e100cfb584 Merge pull request #273 from CartoDB/new_querytables 
Use node-cartodb-querytables library and add Surrogate-Keys
 2016-03-08 15:33:04 +01:00
Raul Ochoa
5ebc8e43d5 Start hits=0 as first time is not a hit 2016-03-08 14:50:08 +01:00
Raul Ochoa
69fa13ab50 Do not track CDB_ sql function files 2016-03-08 14:09:15 +01:00
Daniel García Aubert
9e67d21a05 Passed test 2016-03-02 10:55:35 +01:00
Alejandro Martínez
9262223de4 Add Surrogate-key specs 2016-02-24 11:44:27 +01:00
Alejandro Martínez
5e06711b4b Use node-cartodb-querytables library 2016-02-22 12:24:05 +01:00
Raul Ochoa
15f90c1a78 QueryTablesApi only caches affected tables and always retrieve last modification 2016-02-02 01:16:24 +01:00
Raul Ochoa
c41d4550ce Fix style 2016-02-01 17:22:51 +01:00
Raul Ochoa
9deab814c9 Skip query-tables-api for authenticated requests 2016-02-01 16:53:28 +01:00
Raul Ochoa
7774c10c66 Remove commented out code 2016-01-28 14:44:50 +01:00
Raul Ochoa
98f697849a Remove select permission from CDB_TableMetadata on publicuser 
Some tests will start to fail
 2016-01-28 12:36:06 +01:00
Daniel García Aubert
c8dc2b3dc0 Removed possible reenqueued jobs in tests because of draining. 2016-01-26 12:29:04 +01:00
Daniel García Aubert
bd2ec0e77f Set pg_sleep to 3 seconds in test. 2016-01-26 12:13:04 +01:00
Daniel García Aubert
1857d5d456 Made all test ready to run 2016-01-26 11:53:36 +01:00
Daniel García Aubert
458d762f9f Revernting last changes: 
- Short option in mocha
  - Using batch event emmit to check job status in test
 2016-01-26 11:43:30 +01:00
Daniel García Aubert
aed83ed6b0 Added short option to mocha test runner 2016-01-26 11:33:37 +01:00
Daniel García Aubert
276bc679e6 Changed test to be responsive instead pulling every 50 milliseconds 2016-01-26 11:04:10 +01:00
Daniel García Aubert
a960bd18f1 Dranied jobs in hook 'after' 2016-01-25 17:30:07 +01:00
Daniel García Aubert
7971bcf86b Improved acceptance test for batch api 2016-01-25 16:44:00 +01:00
Daniel García Aubert
20fd536659 Now batch stop is synchronous, changed signature in test. 
Fixed minor issue when draining the last job, queue is got before cancel the job.
 2016-01-25 14:51:37 +01:00
Daniel García Aubert
c540899580 Implemented unit test for user indexer in batch service 2016-01-25 10:47:21 +01:00
Daniel García Aubert
e6f714a51b Implemented some new unit test 2016-01-22 19:31:25 +01:00
Daniel García Aubert
cb638bb0fe Fixed issue with Postgres error codes, implemented some test for job controller 2016-01-22 18:22:21 +01:00
Daniel García Aubert
2da35ac968 Merge branch 'master' of github.com:CartoDB/CartoDB-SQL-API into batch-api 2016-01-22 13:09:20 +01:00
Daniel García Aubert
5c4d239467 Fixed unit test for job queue and moved isJobFound function as private method in job backend 2016-01-22 12:43:41 +01:00
Raul Ochoa
c096d843a2 Adds coverage tool 2016-01-22 12:18:20 +01:00
Daniel García Aubert
2913aed697 Placed job controller endpoit under /api/vX/sql 2016-01-18 20:12:44 +01:00
Daniel García Aubert
a89e3776ec Removed unused callback in before hook in test and added proper configuration from user database config 2016-01-14 14:08:34 +01:00
Daniel García Aubert
c308551d05 Implemented drain to stop batch service gracefully. 
Split job acceptance test
 2016-01-13 16:25:25 +01:00
Daniel García Aubert
6e820b4573 Improved test's messages 2016-01-12 10:24:09 +01:00
Daniel García Aubert
565fb6e0da Add a comment 2016-01-12 10:08:56 +01:00
Daniel García Aubert
6c849f2bef Fixed issue with job acceptance test 2016-01-12 10:06:31 +01:00
Daniel García Aubert
2d13bde301 Implemented uses cases test for Batch API 2016-01-11 20:14:15 +01:00
Daniel García Aubert
078eb7706b Now SQL API is able to disable Batch Service using argv param --no-batch 2016-01-08 18:29:36 +01:00
Daniel García Aubert
20f00d58d9 Refactored batch service to avoid event noise, doing in callback way 2016-01-08 15:47:59 +01:00
Daniel García Aubert
f9f52d2bd1 Improved object creation in batch service 2016-01-08 11:32:01 +01:00
Daniel García Aubert
e6a4e0f7eb Returned 201 HTTP code in job creation 
Fixed bug listing jobs, now user job index is cleaned before respond to client
 2016-01-07 12:07:30 +01:00
Daniel García Aubert
681772bf37 Implemented test for Job Controller and implemented batch stop service 2016-01-05 17:42:28 +01:00
Daniel García Aubert
8c42019641 Improved namespace in redis for batch's stuff 2015-12-29 15:46:04 +01:00
Daniel García Aubert
851c393f75 Refactored batch manager avoiding concurrence issues 2015-12-29 10:19:10 +01:00
Daniel García Aubert
c4f96b4a59 Implemented support for listing jobs for one user 2015-12-24 17:42:49 +01:00
Daniel García Aubert
ecb76ca1de Improved batch API being consistent with naming, humam readability for job dates, hide redis key details for jobs. 2015-12-23 17:29:11 +01:00
Daniel García Aubert
c54f274bd4 Fixed callback called multiple times in tests 2015-12-23 14:54:54 +01:00
Daniel García Aubert
b16c0983c6 Implemented tests for batch API 2015-12-22 23:13:33 +01:00
Daniel García Aubert
3c23bf12e7 Fixed bug whether a job consumer is empty. And refactored batch module, now is a common object intead of module function 2015-12-22 20:12:10 +01:00
Daniel García Aubert
f41c43aa38 Disabled statement timeout bypassing pgbouncer connection 2015-12-22 15:43:00 +01:00
Daniel García Aubert
30d0b2906b Implemented GET for jobs API 2015-12-22 11:45:25 +01:00
Daniel García Aubert
ef59a1d942 Fixed typo in batch test 2015-12-22 11:06:46 +01:00
Daniel García Aubert
328d4c6bfe Fixed bad assertion in batch acceptance test 2015-12-22 11:02:16 +01:00
Daniel García Aubert
3762ad7a39 Now jobs are stored in redis instead of user's database 2015-12-21 19:57:10 +01:00
Daniel García Aubert
fb8feeb964 Now Job is an event emitter instance and we are able to know what happen wwith any job. Implemented test without a silly timeout 2015-12-16 18:13:48 +01:00
Daniel García Aubert
43f759e96a Refactored Batch API using streams instead a interval to consume the job queue. Limited one job running at the same time per queue instead of using a job's counter to limit it. 2015-12-16 15:57:58 +01:00
Daniel García Aubert
d0787d03f7 Reused matadata backend instance for batch service 2015-12-14 10:36:16 +01:00
Daniel García Aubert
6cc48bf9dd Implemented batch service 2015-12-09 20:17:45 +01:00
Daniel García Aubert
da66703ce3 Skipped undone test 2015-12-09 12:23:49 +01:00
Daniel García Aubert
56cc5b439a Removed test filter for runner 2015-12-09 12:22:00 +01:00
Daniel García Aubert
0c6edc573a Used cartodb-redis instead redis-mpool 2015-12-09 12:18:33 +01:00
Daniel García Aubert
00721bcd02 Implementing batch service 2015-12-09 00:02:08 +01:00
Daniel García Aubert
f6632d2373 Removed filter test 2015-12-07 12:11:46 +01:00
vagrant
35d6600f36 Implemented test for job manegement 2015-12-07 10:29:55 +00:00
Daniel García Aubert
a715879e37 Removed mocha test filter 2015-12-07 09:42:33 +01:00
Daniel García Aubert
a1243ad64d Implemented job controller to enqueue jobs 2015-12-07 09:40:51 +01:00
Daniel García Aubert
c6bae6d951 app module moved to right place 2015-12-03 18:33:17 +01:00
Raul Ochoa
b160f4051a Merge pull request #250 from CartoDB/issue-238 
Stop adding X-Cache-Channel header when no tables involved
 2015-09-08 12:30:26 +02:00
Raul Ochoa
5ec6d7c77b Remove no longer needed health check params 2015-09-07 18:34:50 +02:00
Raul Ochoa
875eccba62 Stop adding X-Cache-Channel header when no tables were identified in SQL query 
Fixes #238 and fixes #157
 2015-09-07 18:22:24 +02:00
Raul Ochoa
920cb350cc x-cache-channel tests 2015-09-07 18:09:42 +02:00
Raul Ochoa
a5d1d1d3bd Re-enable regression test for tables with (.) dots in names 
Closes #224 after CDB_QueryTables returns named properly quoted
 2015-09-07 17:10:24 +02:00
Raul Ochoa
0068322fe2 Upgrade CDB_QueryTables to use latest version 2015-09-07 17:10:24 +02:00
Raul Ochoa
74619b38f5 Adds scenario to validate log is limited to 2000 chars 2015-09-04 12:45:51 +02:00
Raul Ochoa
335d71bc04 Adds a custom log4js format function to log sql query from POST requests 
Tokens in log4js are not provided with request/response for context,
using a custom format function allow to access them

In the case of express logger token functions receive request/response
for context so it uses a custom token

Closes #206
 2015-09-04 12:29:20 +02:00
Raul Ochoa
bb884b51a6 Disable regression test until CDB_QueryTablesText is fixed 2015-09-03 11:24:38 +02:00
Raul Ochoa
df589b5e85 Merge pull request #247 from CartoDB/last-modified-header 
Set Last-Modified header based on affected tables
 2015-09-03 09:30:50 +02:00
Raul Ochoa
0e07e39efc Adds regression test for issue #224 2015-09-02 17:09:22 +02:00
Raul Ochoa
f816093036 Set Last-Modified header based on affected tables 
Closes #101
 2015-09-02 16:25:56 +02:00
Raul Ochoa
d42e5a6b75 Do not expose remove from pool after query error as a config option 2015-08-04 16:23:36 +02:00
Raul Ochoa
7c696317d0 New option to remove client from pool after error happens 
This help to avoid issues with transactions.

Closes #241
 2015-08-04 15:53:50 +02:00
Raul Ochoa
8db6b88853 Update CDB_QueryTables to match current spec 2015-08-04 14:45:14 +02:00
Raul Ochoa
62e7e1634f Use in memory sqlite databases for spatialite tests 2015-06-16 17:33:04 +02:00
Raul Ochoa
e9dda12ad3 Adds tests for big results in ogr formats 2015-06-16 15:54:13 +02:00
Francisco Dans
59d93aeb57 removes only from test call 2015-06-01 17:17:05 +02:00
Francisco Dans
fc30771879 adds couple more tests 2015-06-01 15:16:05 +02:00
Francisco Dans
2363b87b61 adds acc test for sqlite 2015-06-01 13:20:51 +02:00
Raul Ochoa
b29933f96e Closes stream responses on error 
fixes #219
 2015-05-13 18:15:38 +02:00
Raul Ochoa
469bd0b696 use describe+it 2015-05-13 12:23:40 +02:00
Raul Ochoa
33731fc315 jshint fixes 2015-05-13 11:53:14 +02:00
Raul Ochoa
9486643f0a jshint fixes 2015-05-13 11:21:44 +02:00