cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,016 Commits 1 Branch 118 Tags 9.7 MiB
16a8187269
Commit Graph

542 Commits

Author SHA1 Message Date
Simon Martín
1293acca48 cancel query when Quota exceeded or max POST size 2018-11-29 18:15:14 +01:00
Daniel García Aubert
e0e011e806 Do not use assert to throw erros as in Node.js > 6 wraps the original error, the keyword 'throw' does the trick and it's backwards compatible 2018-11-07 18:05:39 +01:00
Daniel García Aubert
a616982ec3 Use strict mode 2018-10-24 15:42:33 +02:00
Simon Martín
9365986272 better message 2018-08-14 15:30:51 +02:00
Simon Martín
0a1f7c6ba5 copyto adding gzip to log 2018-08-14 15:14:22 +02:00
Simon Martín
735033faee jshint 2018-08-10 14:39:32 +02:00
Simon Martín
c0ea5be5d3 Improve gzip error message 2018-08-10 14:38:50 +02:00
Simon Martín
9eeeadbc07 maintain req error handler 2018-08-10 14:38:24 +02:00
Rafa de la Torre
8a76df8884 Fix for unhandled gunzip/zlib errors #528 2018-08-10 12:37:49 +02:00
Rafa de la Torre
a709e8f26f Set the default copy_from_max_post_size to 2 GB 
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
 2018-07-26 12:52:20 +02:00
Rafa de la Torre
82348a79a5 Remove unnecessary metrics.end(null, err) 
As per comment in PR
 2018-07-24 14:36:39 +02:00
Rafa de la Torre
21295a2d93 Use pgstream.emit('error', myError) 
As suggested in PR, emit our own error to deal with the quota and max
POST size checks, to do proper error handling.
 2018-07-23 18:29:41 +02:00
Rafa de la Torre
a35a519702 Use CAPITALS for constants (as suggested in PR) 2018-07-23 18:24:48 +02:00
Rafa de la Torre
bac9e94428 Please jshint 2018-07-23 17:15:21 +02:00
Rafa de la Torre
774c144837 Implementation of the max POST size check 2018-07-23 16:59:05 +02:00
Rafa de la Torre
8ec7e0e309 Fix for "Can't set headers after they are sent" 2018-07-23 16:15:11 +02:00
Rafa de la Torre
a5046f58de Add dbRemainingQuota check to COPY FROM 2018-07-23 15:05:22 +02:00
Rafa de la Torre
7e54912ee4 First version of the dbQuotaMiddleware 2018-07-23 14:38:56 +02:00
Rafa de la Torre
ea46db5923 Remove uneeded timeout middleware 2018-07-20 17:48:56 +02:00
Simon Martín
59dd495a87 small style details 2018-06-22 10:50:39 +02:00
Simon Martín
06c40a396b copy from: events is series 2018-06-21 15:58:23 +02:00
Simon Martín
195a4866c3 managing gzip pipe with PassThrough 2018-06-21 15:54:42 +02:00
Simon Martín
fae18fb242 using getPGStream 2018-06-21 15:17:32 +02:00
Simon Martín
9970052e1b copy from: removing copyFromStream dependency 2018-06-21 14:38:22 +02:00
Simon Martín
01eff0a5a2 copy from: removing done dependecy 2018-06-21 14:36:40 +02:00
Simon Martín
a15c2e58aa copy from: manage error in 1 point 2018-06-21 14:36:03 +02:00
Simon Martín
d17a5608fb copy from: unneeded pgstream.end in req error 2018-06-21 14:32:36 +02:00
Simon Martín
5a28d36d69 copy from: pgstream events in series 2018-06-21 14:29:19 +02:00
Simon Martín
925115e8c1 copy from: remove client dependecy 2018-06-21 14:25:54 +02:00
Simon Martín
bafcb8051f StreamCopy getResults refactor 
- changing get Result to getRowCount
- ensuring to and from independecy in StreamCopy
 2018-06-21 13:39:33 +02:00
Simon Martín
2cecc54cb2 copy to: removing copyToStream dependecy 2018-06-21 13:21:23 +02:00
Simon Martín
e39a9b2bf7 copy to: manage error in 1 point 2018-06-21 13:13:37 +02:00
Simon Martín
1dac5355d1 copy to: remove unneeded done and removing it as a dependency 2018-06-21 13:13:16 +02:00
Simon Martín
42a94a3b6c copy to: removing unnedeed client dependecy 2018-06-21 13:01:59 +02:00
Simon Martín
e6448a3255 copy to: cancelQuery to streamCopy 2018-06-21 13:00:57 +02:00
Simon Martín
39dfcb1f9b copy to events is series 2018-06-21 12:50:50 +02:00
Simon Martín
5fbd6e8e3a req on error is neccesary when connection is canceled 2018-06-21 12:49:06 +02:00
Simon Martín
3da32308ce jshint happy 2018-06-21 12:30:49 +02:00
Simon Martín
6955d254f8 Merge branch 'master' into pgcopy-stream-refactor2-dgaubert 2018-06-21 12:19:16 +02:00
Simon Martín
3d30f1f762 trailing white spaces 2018-06-19 10:34:52 +02:00
Simon Martín
30402f2e89 remove unused statsClient 2018-06-19 10:04:17 +02:00
Simon Martín
7727a9d506 data ingestion logger created in controller and added to app 2018-06-18 18:48:11 +02:00
Simon Martín
8a450a862c recover the metrics header (needed for tests purposes) 2018-06-15 13:25:47 +02:00
Simon Martín
9097cbe16e adding log of the error when headers are already sent 2018-06-14 18:29:50 +02:00
Simon Martín
b091a375fc fix ensuring right validation error 2018-06-14 10:49:36 +02:00
Simon Martín
1f31b8e2ae removing old copy logs 2018-06-14 10:45:23 +02:00
Daniel García Aubert
9299fe51ce Add todo 2018-06-14 01:01:05 +02:00
Daniel García Aubert
9c8d802f75 Chain response event callbacks after .pipe() in order to improve readability in copyto controller 2018-06-14 00:31:08 +02:00
Daniel García Aubert
508d688b9d Use request 'close' event when the request is closed unexpectedly by the client 2018-06-14 00:16:06 +02:00
Daniel García Aubert
bd8e4be596 Do not emit to regist the number of rows processed in copyfrom 2018-06-13 20:11:39 +02:00
Daniel García Aubert
5655f26fbd Do not emit to regist the number of rows processed in copyto 2018-06-13 20:06:35 +02:00
Daniel García Aubert
e6a331433a Handle error when client disconnects in copyto requests 2018-06-13 19:54:34 +02:00
Daniel García Aubert
6aba1ea307 Handle error when sql injected in pgstream fails (e.g: unexisting table) 2018-06-13 19:52:53 +02:00
Daniel García Aubert
2792d95c41 Do not handle stream unpiping when it wasn't even created 2018-06-13 18:30:05 +02:00
Simon Martín
270daae579 removing forgotten callback 2018-06-12 18:40:51 +02:00
Simon Martín
409273bafe removing 2nd calback using events 2018-06-12 18:39:50 +02:00
Simon Martín
19aae3c40f manage pgstrem error on stream copy class 2018-06-12 17:04:44 +02:00
Simon Martín
ba2f85421d StreamCopy to class 2018-06-12 16:56:18 +02:00
Simon Martín
8a15b2f8d1 Merge branch 'master' into pgcopy-stream-refactor2 2018-06-12 15:21:28 +02:00
Simon Martín
4d99872975 return in next err 2018-06-12 15:18:28 +02:00
Simon Martín
b9a0fa78d2 adding metadataBackend to user middleware in copycontroller 2018-06-11 14:55:37 +02:00
Simon Martín
1174499dcf Merge branch 'master' into pgcopy-stream 2018-06-11 14:38:52 +02:00
Simon Martín
df63e0eedb adding log of the error when headers are already sent 2018-06-11 13:19:12 +02:00
Simon Martín
2995676adb let to const 2018-06-11 12:56:16 +02:00
Simon Martín
4fab96cb3a rename gzip to isGzip 2018-06-11 12:55:30 +02:00
Simon Martín
87fd9db1fe remove uneeded comment 2018-06-11 12:43:35 +02:00
Simon Martín
2581ddbfb4 better formatting 2018-06-08 17:09:28 +02:00
Simon Martín
6a9e7d7c14 forgotten ) 2018-06-08 17:04:10 +02:00
Simon Martín
7266b3c708 one logger 2018-06-08 17:02:31 +02:00
Simon Martín
014f0a1491 refactor of copy to PoC 2018-06-08 16:58:32 +02:00
Simon Martín
f7454228c6 refactor of copy from PoC 2018-06-08 16:50:12 +02:00
Simon Martín
e259a51623 unifiyng to see 'the forest' 2018-06-08 15:03:21 +02:00
Simon Martín
8e8ccc9c67 removing old metrics in copy 2018-06-08 13:33:17 +02:00
Simon Martín
c62ef15dda using StreamCopy in controller 2018-06-08 11:15:23 +02:00
Simon Martín
2bcea0484a rename BunyanLogger to Logger 2018-06-08 10:59:34 +02:00
Eneko Lakasta
eab3d289b6 refactor authenticated to authorizationLevel 2018-06-05 13:21:56 +02:00
Eneko Lakasta
0207b67d5f refactor forceToBeAuthenticated to forceToBeMaster 2018-06-05 13:14:50 +02:00
Simon Martín
1cf7032c9a adding user and date in copy logs 2018-06-04 18:08:34 +02:00
Eneko Lakasta
fa5a99211c check user exists in user middleware 
This way, we keep sending a 404 error if the user does not exist.
 2018-05-29 13:23:50 +02:00
Simon Martín
008fad3d13 undo 'ensuring data ingestion log' 2018-05-29 12:45:57 +02:00
Simon Martín
e092647246 ensuring data ingestion log 2018-05-28 11:25:09 +02:00
Simon Martín
7d32ae293f rename buyan_logger file name 2018-05-28 11:18:30 +02:00
Simon Martín
3cf28bb7ff jshint happy 2018-05-25 18:50:56 +02:00
Simon Martín
22caa049ee StreamCopyMetrics 2018-05-25 18:47:41 +02:00
Simon Martín
b31984cbc6 simplify response 2018-05-25 17:57:40 +02:00
Simon Martín
1fa5afd759 unify error handler 2018-05-25 17:50:59 +02:00
Simon Martín
6c3f9cf1d3 simplify controllers 2018-05-25 17:42:30 +02:00
Simon Martín
fd70673d88 more details 2018-05-25 16:46:25 +02:00
Simon Martín
8a2c777577 details 2018-05-25 16:00:27 +02:00
Simon Martín
9b0b92fb6b removing try catch 2018-05-25 15:46:12 +02:00
Simon Martín
5bed04a38c fix gzip problem with req close event 2018-05-25 15:34:30 +02:00
Simon Martín
fe3bd4fd37 handling errors from request in COPYfrom 2018-05-25 14:33:54 +02:00
Simon Martín
e2bba967f9 handling copyto errors 2018-05-24 19:48:24 +02:00
Simon Martín
fca6ee8232 changing name of dataIngestionLogPath 2018-05-23 17:32:44 +02:00
Simon Martín
b2a36eb556 copy metrics with BunyanLogger 2018-05-23 17:25:46 +02:00
Simon Martín
5ba7dca79c copyfrom metrics to kibana 2018-05-23 10:30:37 +02:00
Paul Ramsey
f77707f6da Fix reponse copy from 2018-05-22 10:48:11 -04:00
Paul Ramsey
a4654f82b2 Merge branch 'pgcopy-stream' of github.com:CartoDB/CartoDB-SQL-API into pgcopy-stream-q 2018-05-22 10:37:50 -04:00
Paul Ramsey
ed696a96ec Merge branch 'pgcopy-stream' of github.com:CartoDB/CartoDB-SQL-API into pgcopy-stream-q 2018-05-22 10:23:04 -04:00
Simon Martín
6d73d97ae2 copyFrom metrics to statsd 2018-05-22 16:02:14 +02:00