cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,026 Commits 1 Branch 118 Tags 9.7 MiB
a383785e7c
Commit Graph

949 Commits

Author SHA1 Message Date
Daniel García Aubert
06cb514ece Remove unused reference 2019-05-30 11:36:14 +02:00
Daniel García Aubert
33e659c142 Please, linter 2019-05-29 11:38:25 +02:00
Daniel García Aubert
e6b9325b77 Extract class 2019-05-29 11:20:18 +02:00
Daniel García Aubert
6676260b17 Implement a throttler stream to check input data speed 2019-05-28 19:24:29 +02:00
Daniel García Aubert
5a116c69e9 Draft for copy from throttle 2019-05-27 19:32:43 +02:00
Daniel García Aubert
7a3e4d617d Encapsulate cancel 2019-05-27 19:24:48 +02:00
Daniel García Aubert
73eb9656cd Removing unpipe calls, regarding documentation, unpipe is emitted when error. 2019-05-27 15:26:44 +02:00
Daniel García Aubert
c0b5997745 Terminate backend when cancel did not work 2019-05-27 15:11:52 +02:00
Daniel García Aubert
19b9aab042 Do not resume internal connection stream (handled properly in the pg-copy-stream module) 2019-05-24 17:31:56 +02:00
Daniel García Aubert
c52dbb7a40 Release connection to the pool depending on the action 2019-05-24 15:53:09 +02:00
Daniel García Aubert
9c0e76bb4a decouple cancel from the stream 2019-05-24 15:39:56 +02:00
Daniel García Aubert
7519037333 de-tab 2019-05-24 13:07:34 +02:00
Daniel García Aubert
b297005ebb Do not pass unecessary parameter 2019-05-24 12:45:27 +02:00
Daniel García Aubert
e743fefe37 Don't use console.log to log errors 2019-05-16 17:46:59 +02:00
Daniel García Aubert
17de6e8d4f Unpipe streams when error 2019-05-16 17:24:21 +02:00
Daniel García Aubert
9b21b813be Do not pass error for emitting cancel events 2019-05-16 15:07:26 +02:00
Daniel García Aubert
62dc063677 Create stream at the begining 2019-05-16 15:06:27 +02:00
Daniel García Aubert
f7f96f004a Be more specific with arguments 2019-05-16 12:19:03 +02:00
Daniel García Aubert
2208fb9764 Implement a cancel backend through query instead of node-postgres protocol 2019-05-14 15:39:54 +02:00
Raul Marin
915d86ab50 Update doc ref 2019-04-10 12:03:26 +02:00
Raul Marin
6be8f591bf Add missing error codes 2019-04-10 12:01:40 +02:00
Daniel García Aubert
ce04cbcb45 Use callback properly 2019-04-04 18:21:49 +02:00
Daniel García Aubert
f2b5921ade Flush batch logger to not lose info 2019-04-04 18:15:21 +02:00
Daniel García Aubert
2e2e233b68 Use 'bunyan' logger instead of 'debug' to log debug messages 2019-04-04 14:31:41 +02:00
Simon Martín
3b06161c04 jshint 2019-02-28 16:57:23 +01:00
Simon Martín
6c0d040b9b improving logic with Batch API fallback query 2019-02-28 16:42:24 +01:00
Simon Martín
52e4dfa728 using the new TYPES 2019-02-28 11:49:05 +01:00
Simon Martín
a97c25e914 adding TYPES in middleware 2019-02-28 11:48:51 +01:00
Simon Martín
68aebe6b3f Merge branch 'master' into log-queries 2019-02-27 16:14:12 +01:00
Simon Martín
27dbf65372 copy_from and copy_to params defaults 2019-02-27 13:14:16 +01:00
Simon Martín
c8eb265bd9 using config parametes in middleware 2019-02-27 12:51:01 +01:00
Simon Martín
25e54dc267 jshint 2019-02-27 11:59:04 +01:00
Simon Martín
0848b5e759 refactor to clone object and not modify the original 2019-02-27 11:57:28 +01:00
Simon Martín
e829df6367 using stringifyForLogs 2019-02-27 10:40:49 +01:00
Simon Martín
932420e21b extracting stringifyForLogs function 2019-02-27 10:40:39 +01:00
Simon Martín
9e25cc9501 accept multiple values types in queries log 2019-02-27 10:07:56 +01:00
Simon Martín
feb3a21dfd remove log code from query endpoints 2019-02-27 10:07:19 +01:00
Simon Martín
254d804f0c fix log middleware folder 2019-02-27 09:04:43 +01:00
Simon Martín
09dcb6f2fa log copy endpoint 2019-02-27 09:02:49 +01:00
Simon Martín
2066115002 log query and job endpoints 2019-02-27 09:02:31 +01:00
Simon Martín
2c1ce33e34 log middleware 2019-02-27 09:01:51 +01:00
Simon Martín
6768baa0ae ensuring req.body 2019-02-26 17:11:11 +01:00
Simon Martín
b618691af9 splitting behavior batch & not batch handling query 2019-02-26 17:09:36 +01:00
Simon Martín
f925863534 using handle query middleware in copy endpoint 2019-02-26 16:43:18 +01:00
Simon Martín
90b693f050 handling job queries with middleware 2019-02-26 16:40:20 +01:00
Simon Martín
7b5a118c7e renaming from q to sql in middleware 2019-02-26 16:34:26 +01:00
Simon Martín
a6eddef252 same things together 2019-02-26 15:05:14 +01:00
Simon Martín
69e2aaf748 using handleQueryMiddleware in query controller 2019-02-26 14:47:23 +01:00
Simon Martín
143af82b8b semicolon 2019-02-26 14:40:41 +01:00
Simon Martín
0a5fc471c1 handleQueryMiddleware 2019-02-26 14:36:11 +01:00
Simon Martín
799b767505 remove old query log system 2019-02-26 13:58:01 +01:00
Daniel García Aubert
23d76393cd Lint 2019-01-16 15:58:11 +01:00
Simon Martín
60702faa57 adding Authorization to Access-Control-Allow-Headers 2019-01-10 15:55:08 +01:00
Simon Martín
f947fd7cd5 Merge branch 'master' into unique-query-ratelimit 2019-01-09 08:50:41 +01:00
Raul Marin
df4732ff94 Consider cancelled queries as platform limits 2019-01-03 15:17:47 +01:00
Raul Marin
03701ae695 Set platform limits message on streaming responses too 2019-01-02 16:21:33 +01:00
Simon Martín
254896e666 remove query_format rate limit 2018-12-10 18:20:10 +01:00
Simon Martín
498ddc58b7 unique rate limit fot both query endpoints 2018-12-10 18:19:56 +01:00
Rafa de la Torre
aeec1922ee Fix test and add const #543 2018-12-05 19:15:57 +01:00
Simon Martín
1293acca48 cancel query when Quota exceeded or max POST size 2018-11-29 18:15:14 +01:00
Daniel García Aubert
e0e011e806 Do not use assert to throw erros as in Node.js > 6 wraps the original error, the keyword 'throw' does the trick and it's backwards compatible 2018-11-07 18:05:39 +01:00
Daniel García Aubert
121a19eafb Prevent: Octal literals are not allowed in strict mode. 2018-10-24 15:42:54 +02:00
Daniel García Aubert
a616982ec3 Use strict mode 2018-10-24 15:42:33 +02:00
Rafa de la Torre
d5a7f6d2d6 Fix paused stream (DB socket) after copy to #590 
Ideally, the CopyToQueryStream would take care o piping, unpiping and
resuming the connection with the DB.

For some reason it does not always work when trying that from the
_transform callback.

Since there's no better callback (at least in node6), this patch is
required to make sure that such connection can be returned to the pool
and reused by some other query.
 2018-09-17 16:54:51 +02:00
Simon Martín
32ff1e7a8e ensure tmpDir folder 2018-08-30 12:27:34 +02:00
Simon Martín
9365986272 better message 2018-08-14 15:30:51 +02:00
Simon Martín
0a1f7c6ba5 copyto adding gzip to log 2018-08-14 15:14:22 +02:00
Simon Martín
735033faee jshint 2018-08-10 14:39:32 +02:00
Simon Martín
c0ea5be5d3 Improve gzip error message 2018-08-10 14:38:50 +02:00
Simon Martín
9eeeadbc07 maintain req error handler 2018-08-10 14:38:24 +02:00
Rafa de la Torre
8a76df8884 Fix for unhandled gunzip/zlib errors #528 2018-08-10 12:37:49 +02:00
Rafa de la Torre
a709e8f26f Set the default copy_from_max_post_size to 2 GB 
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
 2018-07-26 12:52:20 +02:00
Rafa de la Torre
82348a79a5 Remove unnecessary metrics.end(null, err) 
As per comment in PR
 2018-07-24 14:36:39 +02:00
Rafa de la Torre
19c9c15969 Replace let by const in pseudo-constant conditional assignment 2018-07-23 18:41:27 +02:00
Rafa de la Torre
151bffaad4 return to avoid further errors 
As per a PR comment: `return done(err);` should prevent further errors
and make sure the request is over.
 2018-07-23 18:39:30 +02:00
Rafa de la Torre
21295a2d93 Use pgstream.emit('error', myError) 
As suggested in PR, emit our own error to deal with the quota and max
POST size checks, to do proper error handling.
 2018-07-23 18:29:41 +02:00
Rafa de la Torre
a35a519702 Use CAPITALS for constants (as suggested in PR) 2018-07-23 18:24:48 +02:00
Rafa de la Torre
bac9e94428 Please jshint 2018-07-23 17:15:21 +02:00
Rafa de la Torre
774c144837 Implementation of the max POST size check 2018-07-23 16:59:05 +02:00
Rafa de la Torre
8ec7e0e309 Fix for "Can't set headers after they are sent" 2018-07-23 16:15:11 +02:00
Rafa de la Torre
a5046f58de Add dbRemainingQuota check to COPY FROM 2018-07-23 15:05:22 +02:00
Rafa de la Torre
84d88f91fb Better implementation of dbQuotaMiddleware 2018-07-23 15:02:20 +02:00
Rafa de la Torre
7e54912ee4 First version of the dbQuotaMiddleware 2018-07-23 14:38:56 +02:00
Rafa de la Torre
ea46db5923 Remove uneeded timeout middleware 2018-07-20 17:48:56 +02:00
Rafa de la Torre
cc348f3725 Make the SET statement_timeout async 2018-07-20 16:25:47 +02:00
Rafa de la Torre
17c8cbc723 Add a timeout setting for COPY commands 2018-07-20 15:32:32 +02:00
Rafa de la Torre
763134437b Use the db_batch_port for copy 2018-07-20 10:22:54 +02:00
Eneko Lakasta
1bf8a06ec2 Rephrase error message 2018-06-29 13:19:32 +02:00
Eneko Lakasta
99058e68af add space to message 2018-06-27 15:53:51 +02:00
Eneko Lakasta
2bb28b5578 remove line break in the error message 2018-06-27 15:43:06 +02:00
Eneko Lakasta
879e85d07f add 'SQL query timeout expired error.' to DB timeout limit error message. 2018-06-27 15:17:38 +02:00
Simon Martín
59dd495a87 small style details 2018-06-22 10:50:39 +02:00
Simon Martín
06c40a396b copy from: events is series 2018-06-21 15:58:23 +02:00
Simon Martín
195a4866c3 managing gzip pipe with PassThrough 2018-06-21 15:54:42 +02:00
Simon Martín
f5641a1873 remove unneeded this.connectionClosedByClient 2018-06-21 15:18:26 +02:00
Simon Martín
fae18fb242 using getPGStream 2018-06-21 15:17:32 +02:00
Simon Martín
823e3abcfe unify to and from method in only one getPGStream 2018-06-21 15:16:47 +02:00
Simon Martín
3ede1ea9f1 unify streams 2018-06-21 14:53:07 +02:00
Simon Martín
f4651cadae using this.copyFromStream 2018-06-21 14:49:38 +02:00
Simon Martín
9970052e1b copy from: removing copyFromStream dependency 2018-06-21 14:38:22 +02:00