Commit Graph

1845 Commits

Author SHA1 Message Date
Raul Marin
089be35b5d Aggregation count: Do not return null categories 2018-03-08 18:13:20 +01:00
Daniel García Aubert
bbcb335d60 Merge branch 'master' into refactor-map-controler 2018-03-08 13:18:43 +01:00
Daniel García Aubert
aae814a156 Use template strings 2018-03-08 12:35:54 +01:00
Daniel García Aubert
49bcc5368d Use base number as radix to pare intergers 2018-03-08 12:30:27 +01:00
Daniel García Aubert
555e04f9e7 Use ternary operator 2018-03-08 12:27:49 +01:00
Daniel García Aubert
3f6f2e4e23 Use template string 2018-03-08 12:23:43 +01:00
Daniel García Aubert
abffc4b067 Uppercase for actual constants 2018-03-08 12:23:00 +01:00
Daniel García Aubert
363cb0b679 Extract middlewares from map-controller class 2018-03-08 12:16:24 +01:00
Daniel García Aubert
d26910ba9c Extract checkJsonContentType middleware from MapController class 2018-03-07 19:11:03 +01:00
Daniel García Aubert
74b2f305ea Extract initProfiler middleware from map-controller 2018-03-07 19:09:52 +01:00
Daniel García Aubert
6c2f893651 Rename map-store-map-config-provider middleware 2018-03-07 18:53:20 +01:00
Daniel García Aubert
faaf121eb6 Rename center and bbox middlewares 2018-03-07 18:51:43 +01:00
Daniel García Aubert
83ab65163d Rename attributes middleware 2018-03-07 18:43:35 +01:00
Daniel García Aubert
9dcd5ff332 Impreve naming 2018-03-07 15:56:16 +01:00
Daniel García Aubert
c6635f63c1 Unify layer and tile middlewares 2018-03-07 15:39:59 +01:00
Daniel García Aubert
56213219e4 Rename middleware 2018-03-07 15:25:30 +01:00
Daniel García Aubert
7c2dc20dbe Merge branch 'master' into middlewarify-layergroup-controller 2018-03-07 15:24:34 +01:00
Daniel García Aubert
c8e8317ea4 Do not attach middleware to LayergroupController classs 2018-03-07 15:20:47 +01:00
Daniel
8509796743
Merge pull request #882 from CartoDB/middleware-refactor
Middleware refactor
2018-03-07 15:19:04 +01:00
Daniel García Aubert
90aaed0f2c Typo 2018-03-07 15:05:36 +01:00
Daniel García Aubert
48be15b742 Use const in favour of var 2018-03-07 15:01:04 +01:00
Daniel García Aubert
a95b3f2f99 Fix comment 2018-03-07 14:54:09 +01:00
Daniel García Aubert
b2cc7ab84f Move functions to improve readablity 2018-03-07 14:53:13 +01:00
Daniel García Aubert
eb3414f07f Follow middleware pattern 2018-03-07 14:48:21 +01:00
Daniel García Aubert
292dad130d Move middlewares to the right place 2018-03-07 14:42:21 +01:00
Daniel García Aubert
ec41cddb19 Do not pass the whole res.locals to backends 2018-03-07 12:52:44 +01:00
Daniel García Aubert
5871f8290d Use default param values 2018-03-07 12:46:18 +01:00
Daniel García Aubert
33089be2cd Do not attach header middlewares to node status endpoint 2018-03-07 12:30:59 +01:00
Daniel García Aubert
d351c8d14c Define var as const 2018-03-07 12:09:41 +01:00
Daniel García Aubert
82446e5ffa Use template string to define routes 2018-03-07 12:05:53 +01:00
Daniel García Aubert
b786164e8a Middlewarify metrics increment whether success or error 2018-03-07 11:56:57 +01:00
Daniel García Aubert
a66c19c6c7 Do not bind context when unneeded 2018-03-06 20:05:55 +01:00
Daniel García Aubert
94d1667d70 Refactor affected tables 2018-03-06 20:01:43 +01:00
Daniel García Aubert
3399db1cff Add comment 2018-03-06 18:58:09 +01:00
Daniel García Aubert
874ea99d19 Remove step 2018-03-06 18:43:23 +01:00
Daniel García Aubert
7022fb87b4 Extract header, affected-tables and response middlewares 2018-03-06 18:28:40 +01:00
Daniel García Aubert
7c1e2a6af0 Avoid nested steps 2018-03-06 17:08:39 +01:00
Daniel García Aubert
2f011c3266 Remove nested steps 2018-03-06 17:01:51 +01:00
Daniel García Aubert
4762aa0897 Remove step from sendResponse function 2018-03-06 16:55:27 +01:00
Daniel García Aubert
f30f83331f Extract tile error middleware 2018-03-06 16:44:37 +01:00
Daniel García Aubert
3695e1e3e5 Place function closer to where is called 2018-03-06 16:21:46 +01:00
Daniel García Aubert
585b5929aa Middlewarify tile and layer endpoints 2018-03-06 16:19:53 +01:00
Eneko Lakasta
b0eacb2a79 add vary header to honor authorization header when caching 2018-03-06 12:46:38 +01:00
Daniel García Aubert
9b40370794 Now that mapConfigProvider is linked to 'res.locals' do not pass the whole 'res.locals' to map-config-provider to avoid converting circular structure to JSON 2018-03-06 12:44:17 +01:00
Daniel García Aubert
95f3d58383 Make jshint happy 2018-03-05 19:33:46 +01:00
Daniel García Aubert
0f0cde1093 Middlewarify static-api (bbox/center) endpoints 2018-03-05 19:26:26 +01:00
Daniel García Aubert
ca56df5cfe Middlewarify attributes endpoint 2018-03-05 18:28:52 +01:00
Daniel García Aubert
d8a4209768 Middlewarify analysis-node-status endpoint 2018-03-05 18:13:19 +01:00
Daniel García Aubert
40712a2e62 Middlewarify search dataview endpoint 2018-03-05 18:05:42 +01:00
Daniel García Aubert
acb9ce33b1 Pass dataview-backend as middleware option 2018-03-05 18:04:50 +01:00
Daniel García Aubert
5e43a7145a Middlewarify dataview endpoint 2018-03-05 17:44:04 +01:00
Daniel García Aubert
5de8c4f9c3 Make explicit that forceFormat is optional 2018-03-05 11:19:17 +01:00
Simon Martín
9bc9fc46ff using cartodb-redis for rate limit 2018-03-03 14:46:39 +01:00
Simon Martín
a2bf235553 Merge branch 'master' into rateLimits 2018-03-02 21:21:10 +01:00
Simon Martín
1b18b2b188 changing endpointGroup var name 2018-03-02 20:55:50 +01:00
Daniel García Aubert
6331bebb30 Going green: be able to request defaul named tiles in vector format 2018-03-02 18:25:23 +01:00
Simon Martín
4dd404771e refactoring user_limits_api 2018-03-02 16:07:42 +01:00
Simon Martín
bf267e9c95 fix checking rate limit enabled 2018-03-02 15:48:31 +01:00
Simon Martín
843f70cdba interchange var and middlewware names 2018-03-02 15:35:38 +01:00
Simon Martín
42e0e07c14 refactoring rate limit middleware 2018-03-02 15:33:50 +01:00
Simon Martín
dfdd2b9043 adding middlewares to composer in maps controller 2018-03-02 15:24:18 +01:00
Daniel García Aubert
8656fcd8d1 Use 'const' 2018-03-02 14:04:29 +01:00
Daniel García Aubert
f2f6b9d49c ES6 goodies 2018-03-02 13:29:30 +01:00
Daniel García Aubert
82f1e6753b Remove unreachable code 2018-03-02 13:14:02 +01:00
Daniel García Aubert
7ed717607a Missing space before paramenter list 2018-03-02 13:08:57 +01:00
Daniel García Aubert
416970c819 Remove empty line 2018-03-01 19:10:35 +01:00
Daniel García Aubert
ccc28f3617 Add profiler step to lzma 2018-03-01 19:09:11 +01:00
Daniel García Aubert
5bac36b30f Remove bad profiler usage 2018-03-01 18:53:05 +01:00
Daniel García Aubert
ef3ffddec7 Cosmetic changes 2018-03-01 18:49:44 +01:00
Daniel García Aubert
e6ba467d98 ES6 goodies 2018-03-01 18:47:07 +01:00
Daniel García Aubert
314508bcd8 Middleware naming convention 2018-03-01 18:46:04 +01:00
Daniel García Aubert
da18506e41 Follow middleware factory pattern 2018-03-01 18:45:04 +01:00
Daniel García Aubert
5eaee0b71e Follow middleware naming convention 2018-03-01 18:12:07 +01:00
Daniel García Aubert
bd93e7dc7e Follow middleware pattern 2018-03-01 18:09:49 +01:00
Daniel García Aubert
2c762813ba Follow middleware pattern, return a function as the actual middleware 2018-03-01 15:52:48 +01:00
Simon Martín
136c6fa70b new redis keys 2018-03-01 15:50:40 +01:00
Simon Martín
67b2343571 fix existing var name 2018-03-01 15:50:20 +01:00
Daniel García Aubert
3caa1d9c4a ES6 cosmetics 2018-03-01 15:42:46 +01:00
Daniel García Aubert
b0c924ca03 Follow middleware pattern, should return a function as the actual middleware 2018-03-01 15:42:03 +01:00
Daniel García Aubert
f6f59023b4 Ungroup middlewares 2018-02-28 19:46:46 +01:00
Daniel García Aubert
9dc4e7c955 Use the right step name for profiling 2018-02-28 19:29:53 +01:00
Daniel García Aubert
faa44e54ae Cosmetic changes 2018-02-28 19:29:10 +01:00
Daniel García Aubert
bfb743b851 Improve profiling steps 2018-02-28 19:27:49 +01:00
Daniel García Aubert
dad2e92dd3 Follow middleware naming convention 2018-02-28 19:26:47 +01:00
Daniel García Aubert
59c312ea40 Require modules at the beginning of module 2018-02-28 19:25:50 +01:00
Daniel García Aubert
48c5a458f3 Remove bad use of profiling step 2018-02-28 19:22:22 +01:00
Daniel García Aubert
c0830862c8 Follow middleware naming convention 2018-02-28 19:21:44 +01:00
Daniel García Aubert
42deb7abbe Rename middleware 2018-02-28 19:20:51 +01:00
Daniel García Aubert
62deda6470 Improve naming 2018-02-28 19:13:49 +01:00
Daniel García Aubert
5c2248d419 Merge branch 'master' into project-auth-api 2018-02-28 14:43:12 +01:00
Simon Martín
335d91b42d separate rate limit logic from middleware and EVALSHA 2018-02-28 13:11:18 +01:00
Daniel García Aubert
102b11b1b5 Follow middleware naming convention 2018-02-28 13:10:46 +01:00
Eneko Lakasta
26df09b13f require debug at the top of file 2018-02-28 11:42:44 +01:00
Simon Martín
254991c56c changing endpoint groups constant keys 2018-02-27 17:57:25 +01:00
Simon Martín
a492ab0143 improving naming 2018-02-27 17:36:03 +01:00
Simon Martín
b0d63b2ec0 changing module exports and middleware name 2018-02-27 16:52:27 +01:00
Simon Martín
85b0c63eb0 destructuring array assignment 2018-02-27 16:24:30 +01:00
Simon Martín
98a92f51e6 make jshint happy 2018-02-26 17:17:01 +01:00
Eneko Lakasta
8867cdbc02 use anonymous function instead of arrow function in middleware export to don't bind this 2018-02-26 15:57:42 +01:00
Simon Martín
f03ee4b836 removing hardcoded user 2018-02-26 11:20:31 +01:00
Simon Martín
90418b204e getLowerRateLimit tests 2018-02-26 10:51:51 +01:00
Simon Martín
918674e01a changing endpoint groups names 2018-02-23 21:09:27 +01:00
Simon Martín
9b01a05727 rate limit supporting multiple limits 2018-02-23 17:42:24 +01:00
Simon Martín
2937b6a804 updating redis keys 2018-02-21 17:25:26 +01:00
Simon Martín
08a5e57180 new config params and using them in middleware 2018-02-20 18:18:15 +01:00
Simon Martín
faab174a79 redis keys in easier way 2018-02-20 17:19:50 +01:00
Eneko Lakasta
521b441da5 default apikey is returned by metadata module if no apikey found, remove this code because is never going to be run 2018-02-20 12:53:33 +01:00
Eneko Lakasta
59ca00b33b move apikey credentials getter to middleware file 2018-02-20 12:31:36 +01:00
Simon Martín
80918f5b9b adding rate limit middlewware to named maps controller 2018-02-20 11:26:24 +01:00
Simon Martín
e061b3e631 adding rate limit middlewware to maps controller 2018-02-20 11:26:09 +01:00
Simon Martín
06ec3f80b9 adding rate limit middlewware to named maps admin controller 2018-02-20 11:25:56 +01:00
Simon Martín
e6011287f4 adding middleware to layergroup controller 2018-02-20 11:25:16 +01:00
Daniel García Aubert
7c7d606aa7 Remove trailing spaces 2018-02-19 19:05:13 +01:00
Daniel García Aubert
603ef4044c Reduce cyclomatic complexity 2018-02-19 18:48:02 +01:00
Daniel García Aubert
2e3abfb2cd Catch "name not found" errors from metadata backend and set http code status 404 2018-02-19 18:28:58 +01:00
Simon Martín
98907a886c using a real endpoint in test 2018-02-19 17:08:26 +01:00
Eneko Lakasta
7e14247ea9 remove cause of unreachable code/dead code. Not necessary because carto redis assures at least the default api key 2018-02-19 17:06:59 +01:00
Simon Martín
e519984790 changing const names and exporting them 2018-02-19 17:01:41 +01:00
Eneko Lakasta
7b21bd26d0 Merge branch 'project-auth-api' into project-auth-api-fallback 2018-02-16 11:29:36 +01:00
Eneko Lakasta
4ac224688c in fallback mode, use default api key if api key token doesnt exist 2018-02-16 11:20:04 +01:00
Eneko Lakasta
cda2616a8a get and check api key credentials from api key: username and token 2018-02-15 17:49:47 +01:00
Eneko Lakasta
18dbeea003 get apikey token from request in named maps admin middleware 2018-02-15 15:20:52 +01:00
Eneko Lakasta
3e916c6054 check if req.query exist before getting req.query.api_key/map_key 2018-02-15 15:20:05 +01:00
Eneko Lakasta
fc420c2c0f use for compatibility res.locals.api_key instead of res.locals.apikeyToken 2018-02-15 15:19:09 +01:00
Eneko Lakasta
5db0e9c8d8 add middleware for apikeyToken 2018-02-15 12:50:42 +01:00
Simon Martín
63d1c19263 validation in lua script 2018-02-15 12:44:27 +01:00
Eneko Lakasta
963737d3fb create lib for getting api key token from request 2018-02-15 11:36:42 +01:00
Simon Martín
c059f44bf1 lua script and rate limit process 2018-02-14 18:39:57 +01:00
Eneko Lakasta
890f0d1ef6 add fallback for using metadata fallback 2018-02-14 17:31:05 +01:00
Simon Martín
5fca005a3f Merge branch 'master' into rateLimits 2018-02-14 15:40:50 +01:00
Simon Martín
86d4f8e219 lua script v1 2018-02-14 15:40:12 +01:00
Rafa de la Torre
251fe96509 Cosmetic fix, as suggested in PR 2018-02-12 19:24:53 +01:00
Simon Martín
15bf74f770 starting rate limit middleware 2018-02-12 17:19:08 +01:00
Eneko Lakasta
e1a2ee2381 control API access grants 2018-02-08 13:07:25 +01:00
Eneko Lakasta
b82d26527a remove comment 2018-02-08 12:35:44 +01:00
Eneko Lakasta
b0e9df1400 add pgConnection.getDatabaseParams 2018-02-08 12:04:03 +01:00
Eneko Lakasta
6ebf51ce45 let select apikey type in setDBAuth: regular, default, master 2018-02-08 11:29:17 +01:00
Eneko Lakasta
41f3606572 return unauthorized error when api key not found 2018-02-07 19:12:14 +01:00
Eneko Lakasta
ea0542dcb1 remove use of step 2018-02-07 18:48:59 +01:00
Eneko Lakasta
a4dbc1bac2 remove step and check existance of proper api key 2018-02-07 18:20:56 +01:00
Rafa de la Torre
065f56e161 Improve the speed of the aggregation dataview
Improve the performance of the aggregation dataview.

Instead of using a CTE (WITH) for filtered_source, which is only used in
one place to calculate ranks, inject it as a subquery.

This way the planner has a chance to ignore uneeded columns as well as
to parallelize the exectution of the window function (WindowAgg in the
query plan).

That is the part that takes most of the time of the query.

The improvement is about 20-40% in speed on PG10 with 4 cores.
2018-02-07 18:10:13 +01:00
Eneko Lakasta
95538707c9 add parameter asMaster to setDBAuth 2018-02-07 17:14:13 +01:00
Eneko Lakasta
4c76a921b1 use res.locals instead of req.params 2018-02-07 16:02:13 +01:00
Eneko Lakasta
85c1c987af refactor setDBConn to not use step 2018-02-07 15:49:32 +01:00
Eneko Lakasta
bde86323fd use master api key in setDBAuth 2018-02-07 15:36:24 +01:00
Eneko Lakasta
880e3f388d remove use of _.extend calls 2018-02-07 12:46:10 +01:00
Eneko Lakasta
c1535b1a12 refactor setDBAuth to not use step 2018-02-07 12:40:36 +01:00
Raul Marin
f9fe3ace37 Add mapnik metrics option 2018-02-06 16:07:22 +01:00
Javier Goizueta
2e7784ddf2 Add comment to clafify aggregation resolution limit 2018-02-01 10:26:52 +01:00
Javier Goizueta
d88fbbaa87 Use camelCase 2018-01-31 18:55:28 +01:00