cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,627 Commits 1 Branch 118 Tags 9.7 MiB
8a76df8884
Commit Graph

2627 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rafa de la Torre
8a76df8884 Fix for unhandled gunzip/zlib errors #528 2018-08-10 12:37:49 +02:00
Rafa de la Torre
6193f58dd2 Test for faulty gzip headers scenario 2018-08-10 12:15:31 +02:00
Rafa de la Torre
d40cdd8d37 Modify python code example to really stream data 
And add a remark about headers and query/schema inference.
 2018-08-02 17:43:07 +02:00
Rafa de la Torre
a9dbde7877
Merge pull request #526 from CartoDB/fix-copy-docs 
Fix copy docs
 2018-07-31 16:52:40 +02:00
Rafa de la Torre
f2b38bd661 Use https in all examples 2018-07-31 15:59:15 +02:00
Rafa de la Torre
1037241e0a Clarification about table creation 
We got some internal feedback about the difficulties in cartodbfy'ing a
table. We also got some feedback telling us about the ambiguity of the
table creation need.

We want to make it clear and straightforward so that people don't find
roadblocks when following the examples.
 2018-07-31 15:56:17 +02:00
Rafa de la Torre
bdb3b9fbaa
Merge pull request #522 from CartoDB/521-test-robustness-pgbouncer 
521 test robustness pgbouncer
 2018-07-30 12:47:41 +02:00
Rafa de la Torre
942b0073d8 Please jshint 
As suggested by jshint:
- Remove the unused `res` return value
- Add a semicolon at the end of the file
 2018-07-30 12:36:16 +02:00
Rafa de la Torre
2afe845d78 More descriptive comment about the intent of the code 
As suggested in review comment.
 2018-07-30 12:34:29 +02:00
Rafa de la Torre
bf1d5bf3b4 Early return instead of if/else 
As suggested in review comment.
 2018-07-30 12:31:57 +02:00
Rafa de la Torre
de1157354c
Merge pull request #525 from CartoDB/copy-configs-tweak 
Copy configs tweak
 2018-07-26 18:10:46 +02:00
Rafa de la Torre
a709e8f26f Set the default copy_from_max_post_size to 2 GB 
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
 2018-07-26 12:52:20 +02:00
Rafa de la Torre
0a7edc2e11 Add missing COPY config parameters to staging.js.example 
I forgot to add these to the staging example file and missed them when
testing config changes there.

Related to https://github.com/CartoDB/CartoDB-SQL-API/pull/523
 2018-07-26 12:46:05 +02:00
Rafa de la Torre
7277143c27 Merge remote-tracking branch 'origin/master' into 521-test-robustness-pgbouncer 2018-07-25 15:07:55 +02:00
Rafa de la Torre
097900f00d Stub next version 2018-07-25 10:00:13 +02:00
Rafa de la Torre
d53b7bb5f9 Release version 2.2.0 2018-07-25 09:55:56 +02:00
Rafa de la Torre
e00ab734cd
Merge pull request #523 from CartoDB/copy-limits-rework 
Copy limits rework
 2018-07-25 09:45:22 +02:00
Rafa de la Torre
82348a79a5 Remove unnecessary metrics.end(null, err) 
As per comment in PR
 2018-07-24 14:36:39 +02:00
Rafa de la Torre
19c9c15969 Replace let by const in pseudo-constant conditional assignment 2018-07-23 18:41:27 +02:00
Rafa de la Torre
151bffaad4 return to avoid further errors 
As per a PR comment: `return done(err);` should prevent further errors
and make sure the request is over.
 2018-07-23 18:39:30 +02:00
Rafa de la Torre
21295a2d93 Use pgstream.emit('error', myError) 
As suggested in PR, emit our own error to deal with the quota and max
POST size checks, to do proper error handling.
 2018-07-23 18:29:41 +02:00
Rafa de la Torre
a35a519702 Use CAPITALS for constants (as suggested in PR) 2018-07-23 18:24:48 +02:00
Rafa de la Torre
772b42f639 Add limits information 
To be moved to the Support section once officially released.
 2018-07-23 17:52:56 +02:00
Rafa de la Torre
bac9e94428 Please jshint 2018-07-23 17:15:21 +02:00
Rafa de la Torre
1a56b0af26 Update NEWS with the COPY limits 2018-07-23 17:03:41 +02:00
Rafa de la Torre
6053940d51 Add config for COPY FROM max POST size 
You may be wondering: why it checks for 1.99 GB but it says the limit is
at 2 GB? The actual hard-limit is set in nginx, but we want the app to
catch this error condition and report it properly to the client,
hence the 0.5 % license.
 2018-07-23 16:59:27 +02:00
Rafa de la Torre
774c144837 Implementation of the max POST size check 2018-07-23 16:59:05 +02:00
Rafa de la Torre
75f25a6c02 Test for the max POST size 2018-07-23 16:58:05 +02:00
Rafa de la Torre
a8e538aba4 Test for COPY TO and DB quota 2018-07-23 16:20:15 +02:00
Rafa de la Torre
8ec7e0e309 Fix for "Can't set headers after they are sent" 2018-07-23 16:15:11 +02:00
Rafa de la Torre
7b405cee5d Add test for DB quota exceeded 2018-07-23 16:14:10 +02:00
Rafa de la Torre
153ae3ea3d Fix test: increase timeout for DB quota middleware 
In order to give the DB quota middleware a chance of getting executed,
increase the 1 ms timeout to 10 ms. This is short but enough, as the
quota mocks are constant functions.
 2018-07-23 15:36:13 +02:00
Rafa de la Torre
44858df713 Add a mock for DB quota functions 2018-07-23 15:29:53 +02:00
Rafa de la Torre
a5046f58de Add dbRemainingQuota check to COPY FROM 2018-07-23 15:05:22 +02:00
Rafa de la Torre
84d88f91fb Better implementation of dbQuotaMiddleware 2018-07-23 15:02:20 +02:00
Rafa de la Torre
7e54912ee4 First version of the dbQuotaMiddleware 2018-07-23 14:38:56 +02:00
Rafa de la Torre
ea46db5923 Remove uneeded timeout middleware 2018-07-20 17:48:56 +02:00
Rafa de la Torre
cc348f3725 Make the SET statement_timeout async 2018-07-20 16:25:47 +02:00
Rafa de la Torre
fdab0136e6 Fix copy/paste test 2018-07-20 16:17:44 +02:00
Rafa de la Torre
ce409b4925 Fix typo in test describe 2018-07-20 16:09:10 +02:00
Rafa de la Torre
0f8e1f43c4 Add more tests for copy_timeout 2018-07-20 15:57:36 +02:00
Rafa de la Torre
682d7e6812 Simplify timeout tests 2018-07-20 15:38:20 +02:00
Rafa de la Torre
f437df9045 Fix timeout tests 2018-07-20 15:33:12 +02:00
Rafa de la Torre
17c8cbc723 Add a timeout setting for COPY commands 2018-07-20 15:32:32 +02:00
Rafa de la Torre
632e6818fa Test that COPY uses db_batch_port 2018-07-20 12:04:06 +02:00
Rafa de la Torre
763134437b Use the db_batch_port for copy 2018-07-20 10:22:54 +02:00
Rafa de la Torre
5b65252c4a Ensure no connection pgbouncer -> test_db #521 
This is needed to make the setup a little bit more robust: when trying
to delete the test database, it won't be able if there are "idle
sessions" in the db (that is, connections from pgbouncer to the test
database).

Otherwise it fails when trying to create the database, because there's
already one with the same name.
 2018-07-19 18:46:28 +02:00
Rafa de la Torre
20d1db78dc Helper to reset pgbouncer connections #521 
This sends a PAUSE and RESUME to pgbouncer (in case there's one) before
and after executing tests, to make sure new connections are established
in the tests.

This may be especially important when role or session settings are
modified in the DB (same happens in prod, BTW).
 2018-07-19 18:43:17 +02:00
Rafa de la Torre
6eeb949583 This is a bash script #521 
And using sh can make some things fail. E.g:

```
[[: not found
```
 2018-07-19 18:40:31 +02:00
Rafa de la Torre
b88a45889f
Merge pull request #520 from CartoDB/allow-tests-on-pgbouncer 
Allow tests to be executed against pgbouncer
 2018-07-18 17:12:46 +02:00