cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,767 Commits 1 Branch 118 Tags 9.7 MiB
0f3cdd8532
Commit Graph

2767 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rafa de la Torre
de1157354c
Merge pull request #525 from CartoDB/copy-configs-tweak 
Copy configs tweak
 2018-07-26 18:10:46 +02:00
Rafa de la Torre
a709e8f26f Set the default copy_from_max_post_size to 2 GB 
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
 2018-07-26 12:52:20 +02:00
Rafa de la Torre
0a7edc2e11 Add missing COPY config parameters to staging.js.example 
I forgot to add these to the staging example file and missed them when
testing config changes there.

Related to https://github.com/CartoDB/CartoDB-SQL-API/pull/523
 2018-07-26 12:46:05 +02:00
Rafa de la Torre
7277143c27 Merge remote-tracking branch 'origin/master' into 521-test-robustness-pgbouncer 2018-07-25 15:07:55 +02:00
Rafa de la Torre
097900f00d Stub next version 2018-07-25 10:00:13 +02:00
Rafa de la Torre
d53b7bb5f9 Release version 2.2.0 2018-07-25 09:55:56 +02:00
Rafa de la Torre
e00ab734cd
Merge pull request #523 from CartoDB/copy-limits-rework 
Copy limits rework
 2018-07-25 09:45:22 +02:00
Rafa de la Torre
82348a79a5 Remove unnecessary metrics.end(null, err) 
As per comment in PR
 2018-07-24 14:36:39 +02:00
Rafa de la Torre
19c9c15969 Replace let by const in pseudo-constant conditional assignment 2018-07-23 18:41:27 +02:00
Rafa de la Torre
151bffaad4 return to avoid further errors 
As per a PR comment: `return done(err);` should prevent further errors
and make sure the request is over.
 2018-07-23 18:39:30 +02:00
Rafa de la Torre
21295a2d93 Use pgstream.emit('error', myError) 
As suggested in PR, emit our own error to deal with the quota and max
POST size checks, to do proper error handling.
 2018-07-23 18:29:41 +02:00
Rafa de la Torre
a35a519702 Use CAPITALS for constants (as suggested in PR) 2018-07-23 18:24:48 +02:00
Rafa de la Torre
772b42f639 Add limits information 
To be moved to the Support section once officially released.
 2018-07-23 17:52:56 +02:00
Rafa de la Torre
bac9e94428 Please jshint 2018-07-23 17:15:21 +02:00
Rafa de la Torre
1a56b0af26 Update NEWS with the COPY limits 2018-07-23 17:03:41 +02:00
Rafa de la Torre
6053940d51 Add config for COPY FROM max POST size 
You may be wondering: why it checks for 1.99 GB but it says the limit is
at 2 GB? The actual hard-limit is set in nginx, but we want the app to
catch this error condition and report it properly to the client,
hence the 0.5 % license.
 2018-07-23 16:59:27 +02:00
Rafa de la Torre
774c144837 Implementation of the max POST size check 2018-07-23 16:59:05 +02:00
Rafa de la Torre
75f25a6c02 Test for the max POST size 2018-07-23 16:58:05 +02:00
Rafa de la Torre
a8e538aba4 Test for COPY TO and DB quota 2018-07-23 16:20:15 +02:00
Rafa de la Torre
8ec7e0e309 Fix for "Can't set headers after they are sent" 2018-07-23 16:15:11 +02:00
Rafa de la Torre
7b405cee5d Add test for DB quota exceeded 2018-07-23 16:14:10 +02:00
Rafa de la Torre
153ae3ea3d Fix test: increase timeout for DB quota middleware 
In order to give the DB quota middleware a chance of getting executed,
increase the 1 ms timeout to 10 ms. This is short but enough, as the
quota mocks are constant functions.
 2018-07-23 15:36:13 +02:00
Rafa de la Torre
44858df713 Add a mock for DB quota functions 2018-07-23 15:29:53 +02:00
Rafa de la Torre
a5046f58de Add dbRemainingQuota check to COPY FROM 2018-07-23 15:05:22 +02:00
Rafa de la Torre
84d88f91fb Better implementation of dbQuotaMiddleware 2018-07-23 15:02:20 +02:00
Rafa de la Torre
7e54912ee4 First version of the dbQuotaMiddleware 2018-07-23 14:38:56 +02:00
Rafa de la Torre
ea46db5923 Remove uneeded timeout middleware 2018-07-20 17:48:56 +02:00
Rafa de la Torre
cc348f3725 Make the SET statement_timeout async 2018-07-20 16:25:47 +02:00
Rafa de la Torre
fdab0136e6 Fix copy/paste test 2018-07-20 16:17:44 +02:00
Rafa de la Torre
ce409b4925 Fix typo in test describe 2018-07-20 16:09:10 +02:00
Rafa de la Torre
0f8e1f43c4 Add more tests for copy_timeout 2018-07-20 15:57:36 +02:00
Rafa de la Torre
682d7e6812 Simplify timeout tests 2018-07-20 15:38:20 +02:00
Rafa de la Torre
f437df9045 Fix timeout tests 2018-07-20 15:33:12 +02:00
Rafa de la Torre
17c8cbc723 Add a timeout setting for COPY commands 2018-07-20 15:32:32 +02:00
Rafa de la Torre
632e6818fa Test that COPY uses db_batch_port 2018-07-20 12:04:06 +02:00
Rafa de la Torre
763134437b Use the db_batch_port for copy 2018-07-20 10:22:54 +02:00
Rafa de la Torre
5b65252c4a Ensure no connection pgbouncer -> test_db #521 
This is needed to make the setup a little bit more robust: when trying
to delete the test database, it won't be able if there are "idle
sessions" in the db (that is, connections from pgbouncer to the test
database).

Otherwise it fails when trying to create the database, because there's
already one with the same name.
 2018-07-19 18:46:28 +02:00
Rafa de la Torre
20d1db78dc Helper to reset pgbouncer connections #521 
This sends a PAUSE and RESUME to pgbouncer (in case there's one) before
and after executing tests, to make sure new connections are established
in the tests.

This may be especially important when role or session settings are
modified in the DB (same happens in prod, BTW).
 2018-07-19 18:43:17 +02:00
Rafa de la Torre
6eeb949583 This is a bash script #521 
And using sh can make some things fail. E.g:

```
[[: not found
```
 2018-07-19 18:40:31 +02:00
Rafa de la Torre
b88a45889f
Merge pull request #520 from CartoDB/allow-tests-on-pgbouncer 
Allow tests to be executed against pgbouncer
 2018-07-18 17:12:46 +02:00
Rafa de la Torre
d3e31a9d6c Allow tests to be executed against pgbouncer 
pgbouncer does not support the `--client-min-messages` option. Actually
it fails connections if used like that with this somewhat cryptic
message:

```
psql: ERROR:  Unsupported startup parameter: options
```

In order to be able to execute tests against pgbouncer port (which is
desirable IMO), we either need to remove that option (with little to no
impact) or change the lines above to choose the batch API port from the
config.

Mind that this affects just test setup.
 2018-07-18 15:50:57 +02:00
Rafa de la Torre
dce803d04e
Merge pull request #518 from CartoDB/copy-doc-improvements 
Copy doc improvements
 2018-07-18 10:50:12 +02:00
Rafa de la Torre
c86f49e468 Remove confusing usage of filename request param 2018-07-17 18:41:46 +02:00
Rafa de la Torre
d853350135 Address the CDB_CartodbfyTable doc issue 2018-07-17 18:19:58 +02:00
Rafa de la Torre
c9d560e585 Note about (HEADER true) in CSV files 2018-07-17 18:11:57 +02:00
Simon Martín
a8ef19d318 updating NEWS 2018-07-06 11:46:30 +02:00
Simon Martín
b93e67ce01
Merge pull request #517 from CartoDB/upgrade-cartodb-redis-v2 
Upgrading cartodb-redis
 2018-07-06 11:41:18 +02:00
Simon Martín
b32338983d Merge branch 'master' into upgrade-cartodb-redis-v2 2018-07-06 11:05:21 +02:00
Simon Martín
cef024cf70 fixed version of cartodb-redis 2018-07-06 10:35:33 +02:00
Simon Martín
f11b87a07d upgrading cartodb-redis to 2.0.1 2018-07-05 16:37:30 +02:00