cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,999 Commits 1 Branch 118 Tags 9.7 MiB
db6002c267
Commit Graph

935 Commits

Author SHA1 Message Date
Daniel García Aubert
17de6e8d4f Unpipe streams when error 2019-05-16 17:24:21 +02:00
Daniel García Aubert
9b21b813be Do not pass error for emitting cancel events 2019-05-16 15:07:26 +02:00
Daniel García Aubert
62dc063677 Create stream at the begining 2019-05-16 15:06:27 +02:00
Daniel García Aubert
f7f96f004a Be more specific with arguments 2019-05-16 12:19:03 +02:00
Daniel García Aubert
2208fb9764 Implement a cancel backend through query instead of node-postgres protocol 2019-05-14 15:39:54 +02:00
Raul Marin
915d86ab50 Update doc ref 2019-04-10 12:03:26 +02:00
Raul Marin
6be8f591bf Add missing error codes 2019-04-10 12:01:40 +02:00
Daniel García Aubert
ce04cbcb45 Use callback properly 2019-04-04 18:21:49 +02:00
Daniel García Aubert
f2b5921ade Flush batch logger to not lose info 2019-04-04 18:15:21 +02:00
Daniel García Aubert
2e2e233b68 Use 'bunyan' logger instead of 'debug' to log debug messages 2019-04-04 14:31:41 +02:00
Simon Martín
3b06161c04 jshint 2019-02-28 16:57:23 +01:00
Simon Martín
6c0d040b9b improving logic with Batch API fallback query 2019-02-28 16:42:24 +01:00
Simon Martín
52e4dfa728 using the new TYPES 2019-02-28 11:49:05 +01:00
Simon Martín
a97c25e914 adding TYPES in middleware 2019-02-28 11:48:51 +01:00
Simon Martín
68aebe6b3f Merge branch 'master' into log-queries 2019-02-27 16:14:12 +01:00
Simon Martín
27dbf65372 copy_from and copy_to params defaults 2019-02-27 13:14:16 +01:00
Simon Martín
c8eb265bd9 using config parametes in middleware 2019-02-27 12:51:01 +01:00
Simon Martín
25e54dc267 jshint 2019-02-27 11:59:04 +01:00
Simon Martín
0848b5e759 refactor to clone object and not modify the original 2019-02-27 11:57:28 +01:00
Simon Martín
e829df6367 using stringifyForLogs 2019-02-27 10:40:49 +01:00
Simon Martín
932420e21b extracting stringifyForLogs function 2019-02-27 10:40:39 +01:00
Simon Martín
9e25cc9501 accept multiple values types in queries log 2019-02-27 10:07:56 +01:00
Simon Martín
feb3a21dfd remove log code from query endpoints 2019-02-27 10:07:19 +01:00
Simon Martín
254d804f0c fix log middleware folder 2019-02-27 09:04:43 +01:00
Simon Martín
09dcb6f2fa log copy endpoint 2019-02-27 09:02:49 +01:00
Simon Martín
2066115002 log query and job endpoints 2019-02-27 09:02:31 +01:00
Simon Martín
2c1ce33e34 log middleware 2019-02-27 09:01:51 +01:00
Simon Martín
6768baa0ae ensuring req.body 2019-02-26 17:11:11 +01:00
Simon Martín
b618691af9 splitting behavior batch & not batch handling query 2019-02-26 17:09:36 +01:00
Simon Martín
f925863534 using handle query middleware in copy endpoint 2019-02-26 16:43:18 +01:00
Simon Martín
90b693f050 handling job queries with middleware 2019-02-26 16:40:20 +01:00
Simon Martín
7b5a118c7e renaming from q to sql in middleware 2019-02-26 16:34:26 +01:00
Simon Martín
a6eddef252 same things together 2019-02-26 15:05:14 +01:00
Simon Martín
69e2aaf748 using handleQueryMiddleware in query controller 2019-02-26 14:47:23 +01:00
Simon Martín
143af82b8b semicolon 2019-02-26 14:40:41 +01:00
Simon Martín
0a5fc471c1 handleQueryMiddleware 2019-02-26 14:36:11 +01:00
Simon Martín
799b767505 remove old query log system 2019-02-26 13:58:01 +01:00
Daniel García Aubert
23d76393cd Lint 2019-01-16 15:58:11 +01:00
Simon Martín
60702faa57 adding Authorization to Access-Control-Allow-Headers 2019-01-10 15:55:08 +01:00
Simon Martín
f947fd7cd5 Merge branch 'master' into unique-query-ratelimit 2019-01-09 08:50:41 +01:00
Raul Marin
df4732ff94 Consider cancelled queries as platform limits 2019-01-03 15:17:47 +01:00
Raul Marin
03701ae695 Set platform limits message on streaming responses too 2019-01-02 16:21:33 +01:00
Simon Martín
254896e666 remove query_format rate limit 2018-12-10 18:20:10 +01:00
Simon Martín
498ddc58b7 unique rate limit fot both query endpoints 2018-12-10 18:19:56 +01:00
Rafa de la Torre
aeec1922ee Fix test and add const #543 2018-12-05 19:15:57 +01:00
Simon Martín
1293acca48 cancel query when Quota exceeded or max POST size 2018-11-29 18:15:14 +01:00
Daniel García Aubert
e0e011e806 Do not use assert to throw erros as in Node.js > 6 wraps the original error, the keyword 'throw' does the trick and it's backwards compatible 2018-11-07 18:05:39 +01:00
Daniel García Aubert
121a19eafb Prevent: Octal literals are not allowed in strict mode. 2018-10-24 15:42:54 +02:00
Daniel García Aubert
a616982ec3 Use strict mode 2018-10-24 15:42:33 +02:00
Rafa de la Torre
d5a7f6d2d6 Fix paused stream (DB socket) after copy to #590 
Ideally, the CopyToQueryStream would take care o piping, unpiping and
resuming the connection with the DB.

For some reason it does not always work when trying that from the
_transform callback.

Since there's no better callback (at least in node6), this patch is
required to make sure that such connection can be returned to the pool
and reused by some other query.
 2018-09-17 16:54:51 +02:00
Simon Martín
32ff1e7a8e ensure tmpDir folder 2018-08-30 12:27:34 +02:00
Simon Martín
9365986272 better message 2018-08-14 15:30:51 +02:00
Simon Martín
0a1f7c6ba5 copyto adding gzip to log 2018-08-14 15:14:22 +02:00
Simon Martín
735033faee jshint 2018-08-10 14:39:32 +02:00
Simon Martín
c0ea5be5d3 Improve gzip error message 2018-08-10 14:38:50 +02:00
Simon Martín
9eeeadbc07 maintain req error handler 2018-08-10 14:38:24 +02:00
Rafa de la Torre
8a76df8884 Fix for unhandled gunzip/zlib errors #528 2018-08-10 12:37:49 +02:00
Rafa de la Torre
a709e8f26f Set the default copy_from_max_post_size to 2 GB 
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
 2018-07-26 12:52:20 +02:00
Rafa de la Torre
82348a79a5 Remove unnecessary metrics.end(null, err) 
As per comment in PR
 2018-07-24 14:36:39 +02:00
Rafa de la Torre
19c9c15969 Replace let by const in pseudo-constant conditional assignment 2018-07-23 18:41:27 +02:00
Rafa de la Torre
151bffaad4 return to avoid further errors 
As per a PR comment: `return done(err);` should prevent further errors
and make sure the request is over.
 2018-07-23 18:39:30 +02:00
Rafa de la Torre
21295a2d93 Use pgstream.emit('error', myError) 
As suggested in PR, emit our own error to deal with the quota and max
POST size checks, to do proper error handling.
 2018-07-23 18:29:41 +02:00
Rafa de la Torre
a35a519702 Use CAPITALS for constants (as suggested in PR) 2018-07-23 18:24:48 +02:00
Rafa de la Torre
bac9e94428 Please jshint 2018-07-23 17:15:21 +02:00
Rafa de la Torre
774c144837 Implementation of the max POST size check 2018-07-23 16:59:05 +02:00
Rafa de la Torre
8ec7e0e309 Fix for "Can't set headers after they are sent" 2018-07-23 16:15:11 +02:00
Rafa de la Torre
a5046f58de Add dbRemainingQuota check to COPY FROM 2018-07-23 15:05:22 +02:00
Rafa de la Torre
84d88f91fb Better implementation of dbQuotaMiddleware 2018-07-23 15:02:20 +02:00
Rafa de la Torre
7e54912ee4 First version of the dbQuotaMiddleware 2018-07-23 14:38:56 +02:00
Rafa de la Torre
ea46db5923 Remove uneeded timeout middleware 2018-07-20 17:48:56 +02:00
Rafa de la Torre
cc348f3725 Make the SET statement_timeout async 2018-07-20 16:25:47 +02:00
Rafa de la Torre
17c8cbc723 Add a timeout setting for COPY commands 2018-07-20 15:32:32 +02:00
Rafa de la Torre
763134437b Use the db_batch_port for copy 2018-07-20 10:22:54 +02:00
Eneko Lakasta
1bf8a06ec2 Rephrase error message 2018-06-29 13:19:32 +02:00
Eneko Lakasta
99058e68af add space to message 2018-06-27 15:53:51 +02:00
Eneko Lakasta
2bb28b5578 remove line break in the error message 2018-06-27 15:43:06 +02:00
Eneko Lakasta
879e85d07f add 'SQL query timeout expired error.' to DB timeout limit error message. 2018-06-27 15:17:38 +02:00
Simon Martín
59dd495a87 small style details 2018-06-22 10:50:39 +02:00
Simon Martín
06c40a396b copy from: events is series 2018-06-21 15:58:23 +02:00
Simon Martín
195a4866c3 managing gzip pipe with PassThrough 2018-06-21 15:54:42 +02:00
Simon Martín
f5641a1873 remove unneeded this.connectionClosedByClient 2018-06-21 15:18:26 +02:00
Simon Martín
fae18fb242 using getPGStream 2018-06-21 15:17:32 +02:00
Simon Martín
823e3abcfe unify to and from method in only one getPGStream 2018-06-21 15:16:47 +02:00
Simon Martín
3ede1ea9f1 unify streams 2018-06-21 14:53:07 +02:00
Simon Martín
f4651cadae using this.copyFromStream 2018-06-21 14:49:38 +02:00
Simon Martín
9970052e1b copy from: removing copyFromStream dependency 2018-06-21 14:38:22 +02:00
Simon Martín
01eff0a5a2 copy from: removing done dependecy 2018-06-21 14:36:40 +02:00
Simon Martín
a15c2e58aa copy from: manage error in 1 point 2018-06-21 14:36:03 +02:00
Simon Martín
d17a5608fb copy from: unneeded pgstream.end in req error 2018-06-21 14:32:36 +02:00
Simon Martín
5a28d36d69 copy from: pgstream events in series 2018-06-21 14:29:19 +02:00
Simon Martín
925115e8c1 copy from: remove client dependecy 2018-06-21 14:25:54 +02:00
Simon Martín
327aaf3450 jshint happy 2018-06-21 13:41:13 +02:00
Simon Martín
bafcb8051f StreamCopy getResults refactor 
- changing get Result to getRowCount
- ensuring to and from independecy in StreamCopy
 2018-06-21 13:39:33 +02:00
Simon Martín
2cecc54cb2 copy to: removing copyToStream dependecy 2018-06-21 13:21:23 +02:00
Simon Martín
e39a9b2bf7 copy to: manage error in 1 point 2018-06-21 13:13:37 +02:00
Simon Martín
1dac5355d1 copy to: remove unneeded done and removing it as a dependency 2018-06-21 13:13:16 +02:00
Simon Martín
42a94a3b6c copy to: removing unnedeed client dependecy 2018-06-21 13:01:59 +02:00
Simon Martín
e6448a3255 copy to: cancelQuery to streamCopy 2018-06-21 13:00:57 +02:00
Simon Martín
39dfcb1f9b copy to events is series 2018-06-21 12:50:50 +02:00
Simon Martín
5fbd6e8e3a req on error is neccesary when connection is canceled 2018-06-21 12:49:06 +02:00