Simon Martín
4322ccdd09
adding new endpoints to rate limits
2018-05-03 18:50:13 +02:00
Simon Martín
af4c5906c8
adding /sql tro the endpoints
2018-05-03 18:31:49 +02:00
Paul Ramsey
36bdf84f71
Merge branch 'master' of github.com:CartoDB/CartoDB-SQL-API into pgcopy-stream
2018-04-27 15:56:13 +02:00
Paul Ramsey
e2d8896307
Typo in curl example
2018-04-27 12:32:47 +02:00
Paul Ramsey
5f9cc37dba
Jshint change, DRY the content-disposition handler
2018-04-27 12:19:53 +02:00
Paul Ramsey
fe52af71ac
Add documentation SQL API copyfrom/copyto end points,
...
and make small modifications arising from that task
2018-04-27 11:55:06 +02:00
Paul Ramsey
c78cac9bd6
Use the correct database credentials, so we obey security
2018-04-26 10:45:52 +02:00
Paul Ramsey
f13028b497
More jshint changes
2018-04-26 10:20:21 +02:00
Simon Martín
5f8e1d8f53
fix validatePGEntitiesAccess access
2018-04-26 10:13:03 +02:00
Paul Ramsey
cc5fa5c6ce
Changes suggested by jshint
2018-04-26 10:04:38 +02:00
Paul Ramsey
ea66076255
Remove 'console.debug', try and get travis tests clean
2018-04-26 09:40:13 +02:00
Paul Ramsey
a98781d335
Working code for /copyfrom (streaming) and /copyto (streaming)
2018-04-25 18:37:04 +02:00
Paul Ramsey
b59bda5780
Get working copyto implementation in place. Many test cases
...
needed still and thoughts on how to communicate errors back
that are maybe nicer than stack messages?
2018-04-24 15:55:20 +02:00
Simon Martín
e12e16507a
create object and export
2018-04-24 13:21:43 +02:00
Paul Ramsey
4914100205
CopyFrom works, but still needs a decent return payload and
...
a lot of work on returning useful information for error cases (post empty input,
db errors returned more nicely? etc)
2018-04-24 13:07:57 +02:00
Paul Ramsey
72bce5732b
WIP on /copyfrom
2018-04-24 11:26:15 +02:00
Simon Martín
3b228ac17b
solving jshint crazy error
2018-04-24 10:10:37 +02:00
Simon Martín
985f9ef7f5
pgEntitiesAccessValidator
2018-04-23 18:17:44 +02:00
Rafa de la Torre
2229d0ee57
Fix jshint issues
2018-04-13 16:36:43 +02:00
Paul Ramsey
0161696627
Update copy_controller.js
2018-04-13 05:43:23 -07:00
Paul Ramsey
d54e2f5a07
Implementation including multer, custom storage engine, and pg-copy,
...
but without turning over pg-copy, and demonstrating the missing 'sql'
parameter in the custom storage engine.
2018-04-12 12:25:28 -07:00
Simon Martín
2a729b7da8
destructuring error_handler parameters
2018-04-03 13:43:17 +02:00
Simon Martín
3f6b6d3cff
middleware naming
2018-04-03 13:41:18 +02:00
Simon Martín
db6ab4a591
let & const vars
2018-03-28 13:52:45 +02:00
Simon Martín
adab7f21d8
setErrorHeader refactor
2018-03-28 13:50:45 +02:00
Simon Martín
720c3eb8cb
changing msg var name for errorResponse
2018-03-28 13:35:19 +02:00
Simon Martín
4301b63189
fix error
2018-03-28 13:31:31 +02:00
Simon Martín
616f3f3c07
errorHandlerFactory
2018-03-28 13:06:39 +02:00
Simon Martín
16ea727fc3
move ErrorHandler logic from error middleware
2018-03-28 12:58:10 +02:00
Simon Martín
5b6f876cd6
change pgErrorHandler for more generic errorHandler
2018-03-28 12:50:07 +02:00
Simon Martín
b45f5fe31e
move ErrorHandler to services folder
2018-03-28 12:49:33 +02:00
Simon Martín
287d4634c6
rename createPgError to createGenericError
2018-03-28 12:43:34 +02:00
Simon Martín
e611d21029
getName refactor
2018-03-28 12:38:45 +02:00
Simon Martín
2bfa00ab57
getStatus refactor
2018-03-28 12:35:01 +02:00
Simon Martín
d727a50407
jshint
2018-03-28 12:17:42 +02:00
Simon Martín
96435d28e0
removing underscore dependency
2018-03-28 12:15:55 +02:00
Simon Martín
e2641200f7
extracting error type check
2018-03-28 12:13:31 +02:00
Simon Martín
575f1b3a05
simplify error response on models
2018-03-28 11:55:52 +02:00
Simon Martín
050c598e90
using new error handler as a Error
2018-03-28 11:30:11 +02:00
Simon Martín
2722302a89
using error handler as a Error
2018-03-28 11:29:38 +02:00
Simon Martín
60021a70dc
extends Error
2018-03-28 10:56:06 +02:00
Simon Martín
522363c419
removing semicolons
2018-03-28 10:49:50 +02:00
Simon Martín
eefea03b80
error handler to class
2018-03-28 10:48:25 +02:00
Simon Martín
67e1c04efd
indentation details
2018-03-28 10:48:08 +02:00
Simon Martín
c6eb0ad4c7
error middleware name
2018-03-28 09:58:05 +02:00
Simon Martín
0998e50014
refactor getFields method
2018-03-27 15:56:53 +02:00
Simon Martín
b75334f1e7
Merge branch 'master' into limits-error-refactor
2018-03-27 15:32:24 +02:00
Simon Martín
483c263f61
adding and testing limits error response with params
2018-03-26 19:10:23 +02:00
Simon Martín
54876fa203
addind properties to limits errors
2018-03-26 15:56:50 +02:00
Simon Martín
50b8a9ed17
remove unused parameter
2018-03-23 18:02:44 +01:00
Simon Martín
0b708165b3
simplify timeout error catch
2018-03-23 17:48:21 +01:00
Simon Martín
8c86c2b3f8
correct error message in rate limit
2018-03-23 15:35:06 +01:00
Simon Martín
26baaf6081
removing retry after when no necessary
2018-03-23 15:09:16 +01:00
Simon Martín
80818c3cbb
fix userLimits reference
2018-03-20 16:20:56 +01:00
Simon Martín
581bd0dd89
Merge branch 'master' into rateLimits
2018-03-14 12:47:45 +01:00
Simon Martín
1ffeb456e8
Headers following rfc6648
2018-03-14 12:17:26 +01:00
Daniel García Aubert
8dea7615c4
Use user's port instead of the generic one
2018-03-13 19:08:51 +01:00
Daniel García Aubert
328d6f253e
Honor batch api port
2018-03-13 18:49:01 +01:00
Eneko Lakasta
2ec767c1fc
Set Vary Header to honor Authorization header when caching (fastly)
2018-03-13 11:59:07 +01:00
Simon Martín
9cd4e85768
using cartodb-redis in rate limits
2018-03-03 15:01:45 +01:00
Simon Martín
b6ff37d33f
changing param name
2018-03-02 19:57:33 +01:00
Simon Martín
967cdf8a06
refactoring user_limits
2018-03-02 19:54:22 +01:00
Simon Martín
5cd7c24776
refactoring middleware and updating tests
2018-03-02 16:42:45 +01:00
Simon Martín
a11d9870a3
exporting RATE_LIMIT_REDIS_DB
2018-03-02 13:17:50 +01:00
Simon Martín
25d1e5442c
new redis keys
2018-03-01 15:51:19 +01:00
Simon Martín
7a6ef637ae
make jshint happy
2018-03-01 14:47:34 +01:00
Simon Martín
f3adad57aa
fix assignment
2018-03-01 13:15:32 +01:00
Simon Martín
779ab3b83f
adding rate limit middleware to controllers
2018-03-01 12:19:57 +01:00
Simon Martín
a3dfd2767b
rate limit middleware
2018-03-01 12:19:28 +01:00
Simon Martín
2834975d14
controllers using user limits service
2018-03-01 11:31:35 +01:00
Simon Martín
3fdb7abeaf
user limits service
2018-03-01 11:30:34 +01:00
Daniel García Aubert
5802b2182e
Improve profiling
2018-02-27 13:56:00 +01:00
Daniel García Aubert
78f8c430ea
Merge branch 'master' into auth-api
2018-02-27 12:24:40 +01:00
Daniel García Aubert
6f4c79eaef
Fix common interface for auth-backends
2018-02-26 19:02:05 +01:00
Daniel García Aubert
418ec1304f
Fix issue with auth fallback, it wasn't authenticated when apikey master was provided
2018-02-26 18:13:49 +01:00
Daniel García Aubert
cf0214f5c3
Respond with 403 intead of 401 when the request has no permissions to the specific resource
2018-02-23 15:50:23 +01:00
Daniel García Aubert
f85bdf53a5
Missing middleware
2018-02-23 13:19:26 +01:00
Daniel García Aubert
897d38c3f4
Do not set undefined step
2018-02-23 13:05:41 +01:00
Daniel García Aubert
47b54612c7
Place send-response-middleware to the very end of middleware stack
2018-02-23 13:03:56 +01:00
Rafa de la Torre
90b0ec61c2
Add RESIZE=yes param to shapefiles export
...
Add the parameter `RESIZE=yes` to gdal shapefile driver. This resizes
fields to their optimal size. See
http://www.gdal.org/drv_shapefile.html for details.
This is needed because some customers are having trouble exporting
shapefiles, then reimporting into ArcGIS.
2018-02-23 10:34:44 +01:00
Daniel García Aubert
3807d9f94d
Template for User not found error
2018-02-22 17:49:02 +01:00
Daniel García Aubert
ec24019cc0
Apply some ES6 goodies
2018-02-22 17:35:03 +01:00
Daniel García Aubert
868dc798b2
Cosmetic line break
2018-02-22 15:59:13 +01:00
Daniel García Aubert
095c34358e
Remove step
2018-02-22 15:58:52 +01:00
Daniel García Aubert
8c3b9fa05b
Remove bad assignment
2018-02-22 13:27:50 +01:00
Daniel García Aubert
e4067132b0
Add profile to user limit middleware
2018-02-22 12:48:39 +01:00
Daniel García Aubert
700c64bba3
Extract to a middleware user timeout limit from user-database-services
2018-02-22 12:45:55 +01:00
Daniel García Aubert
afceac7369
Improve profile step name
2018-02-22 12:23:37 +01:00
Daniel García Aubert
94c5bd11df
Split authorization middleware, it was actually doing two things: authorize and get database connection params
2018-02-22 12:22:39 +01:00
Daniel García Aubert
1252f32579
Get api_key token from res.locals
2018-02-22 12:07:27 +01:00
Daniel García Aubert
8730b5c517
Read if the request is authenticated through res.locals
2018-02-22 12:04:05 +01:00
Daniel García Aubert
b399abee18
Move authorization to auth-api and extract it from user-database-service
2018-02-22 11:46:34 +01:00
Daniel García Aubert
6c92781600
Remove empty line
2018-02-21 16:19:12 +01:00
Daniel García Aubert
9088cbf150
Improve naming
2018-02-21 13:51:28 +01:00
Daniel García Aubert
28a7cf9259
Don't authenticate when default_public api-key is provided
2018-02-21 11:14:31 +01:00
Daniel García Aubert
ba113d8628
Improve naming
2018-02-20 16:43:43 +01:00
Daniel García Aubert
33baa399ae
Create function to compose middlewares to perfom job operations
2018-02-20 16:25:16 +01:00
Daniel García Aubert
864ec30058
Remove duplication
2018-02-20 13:22:33 +01:00
Daniel García Aubert
d4d54648d4
Improve naming
2018-02-20 13:14:28 +01:00
Daniel García Aubert
03e9b8006b
Actually pass just apikey instead of the entire "res" object
2018-02-19 19:16:33 +01:00
Daniel García Aubert
d6776189bd
Remove unused param
2018-02-19 16:04:57 +01:00
Daniel García Aubert
4af751ff62
Rename param
2018-02-19 15:58:48 +01:00
Daniel García Aubert
b6a666a8b4
Miss rename middleware
2018-02-19 15:54:05 +01:00
Daniel García Aubert
8c13add7a8
Rename middleware
2018-02-19 15:49:17 +01:00
Daniel García Aubert
939443ef81
Unify credentials and authenticated request midllewares
2018-02-19 15:44:28 +01:00
Daniel García Aubert
7898b49e59
Extract profiler middleware to used in query and job controllers
2018-02-19 15:13:36 +01:00
Daniel García Aubert
85fbd7c6b2
Create middlewares to initialize and finish profiling
2018-02-19 14:42:52 +01:00
Daniel García Aubert
633d06bf2d
Do not use underscore
2018-02-19 14:41:06 +01:00
Daniel García Aubert
e04529c501
Rename middleware
2018-02-19 14:20:09 +01:00
Daniel García Aubert
75c2d85dbb
Use authenticated middleware in query controller
2018-02-19 13:24:44 +01:00
Daniel García Aubert
114070ef96
Rename middleware
2018-02-19 12:37:19 +01:00
Daniel García Aubert
adfe3dd52b
Remove comment
2018-02-19 11:10:39 +01:00
Daniel García Aubert
ea310db62c
Cosmetic chenges
2018-02-19 11:04:28 +01:00
Daniel García Aubert
edd382fb8f
Check user is the same user that sends the request when basic-auth is provided
2018-02-16 18:21:06 +01:00
Daniel García Aubert
bcf88093e9
Support basic-auth for batch api
2018-02-16 11:52:57 +01:00
Daniel García Aubert
2cadfe6f18
Add support for Basic Auth
2018-02-16 10:46:58 +01:00
Daniel García Aubert
756fbe42c9
Use apikey midlleware
2018-02-15 17:24:04 +01:00
Daniel García Aubert
ad772246d0
Get api_key from specific middleware and save it into res.locals
2018-02-15 17:23:35 +01:00
Daniel García Aubert
67f2a85abd
Add database credentials to jobs
2018-02-15 12:30:04 +01:00
Daniel García Aubert
ea6e8b5315
Implement fallback mechanism to be able to authenticate as usual in case of apikey is not found
2018-02-14 16:22:36 +01:00
Daniel García Aubert
7c859f0d7b
Use actual apikey properties
2018-02-12 17:49:50 +01:00
Daniel García Aubert
c003a491c0
Use current implementation of cartodb-redis
2018-02-12 16:41:35 +01:00
Daniel García Aubert
5c7f5fbaa5
Merge branch 'master' into auth-api
2018-02-12 16:30:09 +01:00
Simon Martín
bd410fb5f6
using same way to search a string
2018-01-09 17:16:38 +01:00
Simon Martín
df86cc16e4
method to detect timeout error
2018-01-09 17:14:11 +01:00
Simon Martín
6e45c39b1f
Merge branch 'master' into pythonTimeout
2018-01-08 11:29:17 +01:00
Simon Martín
ee511d3762
stringifyForLogs more usual case first
2017-12-18 14:54:23 +01:00
Simon Martín
b531922459
add line at EOF
2017-12-18 12:59:20 +01:00
Simon Martín
485d9adddd
escape chars function
2017-12-18 12:45:44 +01:00
Simon Martín
3fa09a007e
line at EOF and better comment
2017-12-12 17:16:40 +01:00
Simon Martín
6481d14192
escape quotes for logs
2017-12-12 12:58:31 +01:00
Simon Martín
c478e8e62e
removing default error log value
2017-12-12 10:55:28 +01:00
Simon Martín
61b351535c
change funcion name
2017-11-30 15:06:25 +01:00
Simon Martín
0ab87df644
extract error message from array
2017-11-28 17:19:10 +01:00
Simon Martín
4d598eacd7
dont modify err, works with errorsLog
2017-11-28 16:11:04 +01:00
Simon Martín
a17e1fc5ec
logErrors function
2017-11-28 15:58:14 +01:00
Simon Martín
3e23be2087
add errors header with default value
2017-11-28 15:57:58 +01:00
Daniel García Aubert
88015aaf0f
Implement new auth system based on api-key tokens with scoped permissions
2017-11-24 17:57:54 +01:00
Simon Martín
670cb6eb8b
Merge branch 'master' into pythonTimeout
2017-11-24 16:02:13 +01:00
Daniel García Aubert
15949d2fc2
Fix wrong function name
2017-11-24 15:55:16 +01:00
Daniel García Aubert
b10937a8fb
Use res.locals instead of req.context to share object among middlewares
2017-11-24 15:49:25 +01:00
Daniel García Aubert
c2d1a4653e
Improve naming for connection database params holder
2017-11-24 15:45:49 +01:00
Simon Martín
11e0ac89d2
undo test of postgres9.5 error
2017-11-24 12:22:30 +01:00
Rafa de la Torre
296d1878fd
Trying regexp instead of exact match
2017-11-24 10:55:02 +01:00
Daniel García Aubert
833b7b962c
Order geo-columns to export 'the_geom' by default while exporting SHP format
2017-11-17 14:45:58 +01:00
Simon Martín
27e43c8d91
handling python timeout error
2017-11-17 13:30:58 +01:00
Daniel García Aubert
b1112e2bb1
Remove unneeded header
2017-09-20 16:48:22 +02:00
Daniel García Aubert
980e762ab7
Skip user auth middlewares
2017-09-20 16:35:33 +02:00
Daniel García Aubert
0597a3b8a6
Change job work in progress endpoint
2017-09-20 15:46:29 +02:00
Raul Ochoa
32154b67c6
Merge branch 'master' into upgrade-cartodb-psql
2017-08-11 11:57:16 +02:00
Daniel García Aubert
f13c16e523
Typo
2017-08-10 11:22:08 +02:00
Daniel García Aubert
01a71ee60e
Apply user timeout to ogr2ogr command
2017-08-09 12:50:16 +02:00
Raul Ochoa
3e536b822e
Remove unnecessary destroyOnError option
2017-08-08 18:21:10 +02:00
Daniel García Aubert
fb477260ec
Respond with 429 and proper message when database fires a timeout error
2017-08-03 17:19:08 +02:00
Rafa de la Torre
07890ec608
Change tableCache.get for tableCache.peek #244
...
This will make it consider age of the key when it was initially set,
instead of renewing it with the time of the last access.
2017-07-03 11:13:21 +02:00
Rafa de la Torre
1db935a24b
Merge pull request #423 from CartoDB/422-disable-query-lru-cache
...
422 disable query lru cache
2017-06-27 17:40:33 +02:00
Rafa de la Torre
2d9a5e9655
Move cyclomatic complexity to the factory #422
2017-06-27 16:05:05 +02:00
Rafa de la Torre
c8e42f561d
Reduce cyclomatic complexity (jshint 2/2) #422
2017-06-27 15:35:51 +02:00
Rafa de la Torre
34ccfc10df
Please jshint (1/2) #422
2017-06-27 13:26:14 +02:00
Rafa de la Torre
bf478cabb2
Fix initialization of NoCache #422
2017-06-27 12:44:11 +02:00
Rafa de la Torre
bc71889f93
First attempt at removing tableCache #422
2017-06-27 12:10:24 +02:00
Daniel García Aubert
aa66000848
Going green: use replacer parameter in JSON.stringify method to not cast special number values to null
2017-06-08 12:42:37 +02:00
Mario de Frutos
3a99020632
Added regexp for the test to matches both 0.10 and 6.X node version
2017-04-18 12:15:47 +02:00
Mario de Frutos
da42daa536
Improved error message for zip spawn command
2017-04-18 12:15:47 +02:00
Daniel García Aubert
e4eeb9e1f6
Clean commented code
2017-04-18 12:15:47 +02:00
Daniel García Aubert
e0f9209017
Attach error handler while spawning zip command
2017-04-18 12:15:47 +02:00
Daniel García Aubert
884d3aea0a
Dont call 'next' callback twice, include error message if zip don't exit normally
2017-04-18 12:15:47 +02:00
Daniel García Aubert
4b98b7056a
Make zip command path configurable
2017-04-18 12:15:47 +02:00
Daniel García Aubert
6dba06130b
Retur a proper error message when ogr2ogr command fails
2017-04-18 11:40:37 +02:00
Daniel García Aubert
d60e2107af
Add error callback to ogr command while spawning
2017-04-11 09:43:25 +02:00
Raul Ochoa
db9bfacf07
Extract stats client creation
2017-03-30 16:13:17 +02:00
Raul Ochoa
d48a901533
Merge branch 'master' into node-v6
2017-01-12 19:49:56 +01:00
Javier Torres
62a67f148c
Correctly accept custom options for formats in ogr.js
2017-01-11 19:07:50 +01:00
Javier Torres
c67a2e7d09
Use cartodb_id as FID column in gpkg
2017-01-11 19:07:50 +01:00
Daniel García Aubert
ef124d689e
Handle query end in error listener since pg no longer emits end if and error occurs
2016-12-07 16:22:21 +01:00
Daniel García Aubert
99c7a6e4f9
Raise job query size to 16kb
2016-11-07 16:50:44 +01:00
Daniel García Aubert
4a64d37c6c
Add work in progrees list endpoint
2016-10-28 15:08:42 +02:00
Raul Ochoa
cdde1be29e
Re-use redis pool as much as possible
2016-10-17 15:02:34 +02:00
Raul Ochoa
e401c01d78
Only log on non-test environments
2016-10-12 01:40:14 +02:00
Raul Ochoa
e4b1711e8e
pub/sub package
2016-10-11 18:28:46 +02:00
Raul Ochoa
66820a67bb
Make possible to specify a name for batch
2016-10-10 19:46:07 +02:00
Raul Ochoa
42a1f3ad4c
Accept multipart requests
2016-10-05 17:29:16 +02:00
Raul Ochoa
6309318534
Use body-parser from old connect module
2016-10-05 14:22:44 +02:00
Raul Ochoa
025b3f3cc7
Increase body limit to 20mb
2016-10-05 11:14:06 +02:00
Raul Ochoa
2a2a54a073
DRY in job response handler
2016-10-04 16:07:13 +02:00
Raul Ochoa
19c9bec633
Callback with job
2016-10-04 15:57:13 +02:00
Raul Ochoa
05cbd55b95
Use db host from request's context
2016-10-04 15:50:39 +02:00
Raul Ochoa
c32a2199fa
Use request bootstrapper to add host header
2016-10-04 15:43:19 +02:00
Raul Ochoa
7b7d651d8f
DRY while authenticating requests
2016-10-04 15:40:56 +02:00
Raul Ochoa
20f50d988e
Use user middleware in job controller
2016-10-04 15:19:31 +02:00
Raul Ochoa
b139b9ab21
Add context object in all requests
2016-10-04 15:12:46 +02:00
Raul Ochoa
2edc7505e7
Do not condition req.profiler
2016-10-04 15:08:31 +02:00
Raul Ochoa
cb9db0c4cb
Make request.profiler always available
2016-10-04 14:40:04 +02:00
Raul Ochoa
3b6bc14d17
Increment errors on err
2016-10-04 13:19:29 +02:00
Raul Ochoa
bf1a67780d
Merge branch 'master' into express-4.x
2016-09-30 17:37:30 +02:00
Daniel García Aubert
59e5c10f85
Set batch queries log path
2016-09-29 15:37:14 +02:00
Raul Ochoa
2aa1f045ff
Use .status() API
2016-09-26 18:10:20 +02:00
Raul Ochoa
abc2f130c9
Migrate to express 4.x series
...
- Remove express logger
- Error handler responds with application/[json|javascript]
- Fix all tests relying on res.headers
- assert.response based on request module
2016-09-26 18:09:27 +02:00
Raul Ochoa
f92d50cccf
Rename from app to server
...
Removes app_root dependency in requires
2016-09-14 20:54:53 +02:00
Raul Ochoa
0626d80e24
Removes support for optional rollbar logging
2016-09-14 19:22:31 +02:00
