Commit Graph

1897 Commits

Author SHA1 Message Date
Daniel García Aubert
02f93f3a14 Extract layergroup-metadata class 2018-03-09 12:58:05 +01:00
Raúl Marín
06604cd738
Merge pull request #888 from Algunenano/support_1368
Aggregation count: Do not return null categories
2018-03-09 11:06:40 +01:00
Raul Marin
089be35b5d Aggregation count: Do not return null categories 2018-03-08 18:13:20 +01:00
Daniel García Aubert
bbcb335d60 Merge branch 'master' into refactor-map-controler 2018-03-08 13:18:43 +01:00
Daniel García Aubert
aae814a156 Use template strings 2018-03-08 12:35:54 +01:00
Daniel García Aubert
49bcc5368d Use base number as radix to pare intergers 2018-03-08 12:30:27 +01:00
Daniel García Aubert
555e04f9e7 Use ternary operator 2018-03-08 12:27:49 +01:00
Daniel García Aubert
3f6f2e4e23 Use template string 2018-03-08 12:23:43 +01:00
Daniel García Aubert
abffc4b067 Uppercase for actual constants 2018-03-08 12:23:00 +01:00
Daniel García Aubert
363cb0b679 Extract middlewares from map-controller class 2018-03-08 12:16:24 +01:00
Daniel García Aubert
d26910ba9c Extract checkJsonContentType middleware from MapController class 2018-03-07 19:11:03 +01:00
Daniel García Aubert
74b2f305ea Extract initProfiler middleware from map-controller 2018-03-07 19:09:52 +01:00
Daniel García Aubert
6c2f893651 Rename map-store-map-config-provider middleware 2018-03-07 18:53:20 +01:00
Daniel García Aubert
faaf121eb6 Rename center and bbox middlewares 2018-03-07 18:51:43 +01:00
Daniel García Aubert
83ab65163d Rename attributes middleware 2018-03-07 18:43:35 +01:00
Daniel García Aubert
9dcd5ff332 Impreve naming 2018-03-07 15:56:16 +01:00
Daniel García Aubert
c6635f63c1 Unify layer and tile middlewares 2018-03-07 15:39:59 +01:00
Daniel García Aubert
56213219e4 Rename middleware 2018-03-07 15:25:30 +01:00
Daniel García Aubert
7c2dc20dbe Merge branch 'master' into middlewarify-layergroup-controller 2018-03-07 15:24:34 +01:00
Daniel García Aubert
c8e8317ea4 Do not attach middleware to LayergroupController classs 2018-03-07 15:20:47 +01:00
Daniel
8509796743
Merge pull request #882 from CartoDB/middleware-refactor
Middleware refactor
2018-03-07 15:19:04 +01:00
Daniel García Aubert
90aaed0f2c Typo 2018-03-07 15:05:36 +01:00
Daniel García Aubert
48be15b742 Use const in favour of var 2018-03-07 15:01:04 +01:00
Daniel García Aubert
a95b3f2f99 Fix comment 2018-03-07 14:54:09 +01:00
Daniel García Aubert
b2cc7ab84f Move functions to improve readablity 2018-03-07 14:53:13 +01:00
Daniel García Aubert
eb3414f07f Follow middleware pattern 2018-03-07 14:48:21 +01:00
Daniel García Aubert
292dad130d Move middlewares to the right place 2018-03-07 14:42:21 +01:00
Daniel García Aubert
ec41cddb19 Do not pass the whole res.locals to backends 2018-03-07 12:52:44 +01:00
Daniel García Aubert
5871f8290d Use default param values 2018-03-07 12:46:18 +01:00
Daniel García Aubert
33089be2cd Do not attach header middlewares to node status endpoint 2018-03-07 12:30:59 +01:00
Daniel García Aubert
d351c8d14c Define var as const 2018-03-07 12:09:41 +01:00
Daniel García Aubert
82446e5ffa Use template string to define routes 2018-03-07 12:05:53 +01:00
Daniel García Aubert
b786164e8a Middlewarify metrics increment whether success or error 2018-03-07 11:56:57 +01:00
Daniel García Aubert
a66c19c6c7 Do not bind context when unneeded 2018-03-06 20:05:55 +01:00
Daniel García Aubert
94d1667d70 Refactor affected tables 2018-03-06 20:01:43 +01:00
Daniel García Aubert
3399db1cff Add comment 2018-03-06 18:58:09 +01:00
Daniel García Aubert
874ea99d19 Remove step 2018-03-06 18:43:23 +01:00
Daniel García Aubert
7022fb87b4 Extract header, affected-tables and response middlewares 2018-03-06 18:28:40 +01:00
Daniel García Aubert
7c1e2a6af0 Avoid nested steps 2018-03-06 17:08:39 +01:00
Daniel García Aubert
2f011c3266 Remove nested steps 2018-03-06 17:01:51 +01:00
Daniel García Aubert
4762aa0897 Remove step from sendResponse function 2018-03-06 16:55:27 +01:00
Daniel García Aubert
f30f83331f Extract tile error middleware 2018-03-06 16:44:37 +01:00
Daniel García Aubert
3695e1e3e5 Place function closer to where is called 2018-03-06 16:21:46 +01:00
Daniel García Aubert
585b5929aa Middlewarify tile and layer endpoints 2018-03-06 16:19:53 +01:00
Eneko Lakasta
b0eacb2a79 add vary header to honor authorization header when caching 2018-03-06 12:46:38 +01:00
Daniel García Aubert
9b40370794 Now that mapConfigProvider is linked to 'res.locals' do not pass the whole 'res.locals' to map-config-provider to avoid converting circular structure to JSON 2018-03-06 12:44:17 +01:00
Daniel García Aubert
95f3d58383 Make jshint happy 2018-03-05 19:33:46 +01:00
Daniel García Aubert
0f0cde1093 Middlewarify static-api (bbox/center) endpoints 2018-03-05 19:26:26 +01:00
Daniel García Aubert
ca56df5cfe Middlewarify attributes endpoint 2018-03-05 18:28:52 +01:00
Daniel García Aubert
d8a4209768 Middlewarify analysis-node-status endpoint 2018-03-05 18:13:19 +01:00
Daniel García Aubert
40712a2e62 Middlewarify search dataview endpoint 2018-03-05 18:05:42 +01:00
Daniel García Aubert
acb9ce33b1 Pass dataview-backend as middleware option 2018-03-05 18:04:50 +01:00
Daniel García Aubert
5e43a7145a Middlewarify dataview endpoint 2018-03-05 17:44:04 +01:00
Daniel García Aubert
5de8c4f9c3 Make explicit that forceFormat is optional 2018-03-05 11:19:17 +01:00
Simon Martín
9bc9fc46ff using cartodb-redis for rate limit 2018-03-03 14:46:39 +01:00
Simon Martín
a2bf235553 Merge branch 'master' into rateLimits 2018-03-02 21:21:10 +01:00
Simon Martín
1b18b2b188 changing endpointGroup var name 2018-03-02 20:55:50 +01:00
Daniel García Aubert
6331bebb30 Going green: be able to request defaul named tiles in vector format 2018-03-02 18:25:23 +01:00
Simon Martín
4dd404771e refactoring user_limits_api 2018-03-02 16:07:42 +01:00
Simon Martín
bf267e9c95 fix checking rate limit enabled 2018-03-02 15:48:31 +01:00
Simon Martín
843f70cdba interchange var and middlewware names 2018-03-02 15:35:38 +01:00
Simon Martín
42e0e07c14 refactoring rate limit middleware 2018-03-02 15:33:50 +01:00
Simon Martín
dfdd2b9043 adding middlewares to composer in maps controller 2018-03-02 15:24:18 +01:00
Daniel García Aubert
8656fcd8d1 Use 'const' 2018-03-02 14:04:29 +01:00
Daniel García Aubert
f2f6b9d49c ES6 goodies 2018-03-02 13:29:30 +01:00
Daniel García Aubert
82f1e6753b Remove unreachable code 2018-03-02 13:14:02 +01:00
Daniel García Aubert
7ed717607a Missing space before paramenter list 2018-03-02 13:08:57 +01:00
Daniel García Aubert
416970c819 Remove empty line 2018-03-01 19:10:35 +01:00
Daniel García Aubert
ccc28f3617 Add profiler step to lzma 2018-03-01 19:09:11 +01:00
Daniel García Aubert
5bac36b30f Remove bad profiler usage 2018-03-01 18:53:05 +01:00
Daniel García Aubert
ef3ffddec7 Cosmetic changes 2018-03-01 18:49:44 +01:00
Daniel García Aubert
e6ba467d98 ES6 goodies 2018-03-01 18:47:07 +01:00
Daniel García Aubert
314508bcd8 Middleware naming convention 2018-03-01 18:46:04 +01:00
Daniel García Aubert
da18506e41 Follow middleware factory pattern 2018-03-01 18:45:04 +01:00
Daniel García Aubert
5eaee0b71e Follow middleware naming convention 2018-03-01 18:12:07 +01:00
Daniel García Aubert
bd93e7dc7e Follow middleware pattern 2018-03-01 18:09:49 +01:00
Daniel García Aubert
2c762813ba Follow middleware pattern, return a function as the actual middleware 2018-03-01 15:52:48 +01:00
Simon Martín
136c6fa70b new redis keys 2018-03-01 15:50:40 +01:00
Simon Martín
67b2343571 fix existing var name 2018-03-01 15:50:20 +01:00
Daniel García Aubert
3caa1d9c4a ES6 cosmetics 2018-03-01 15:42:46 +01:00
Daniel García Aubert
b0c924ca03 Follow middleware pattern, should return a function as the actual middleware 2018-03-01 15:42:03 +01:00
Daniel García Aubert
f6f59023b4 Ungroup middlewares 2018-02-28 19:46:46 +01:00
Daniel García Aubert
9dc4e7c955 Use the right step name for profiling 2018-02-28 19:29:53 +01:00
Daniel García Aubert
faa44e54ae Cosmetic changes 2018-02-28 19:29:10 +01:00
Daniel García Aubert
bfb743b851 Improve profiling steps 2018-02-28 19:27:49 +01:00
Daniel García Aubert
dad2e92dd3 Follow middleware naming convention 2018-02-28 19:26:47 +01:00
Daniel García Aubert
59c312ea40 Require modules at the beginning of module 2018-02-28 19:25:50 +01:00
Daniel García Aubert
48c5a458f3 Remove bad use of profiling step 2018-02-28 19:22:22 +01:00
Daniel García Aubert
c0830862c8 Follow middleware naming convention 2018-02-28 19:21:44 +01:00
Daniel García Aubert
42deb7abbe Rename middleware 2018-02-28 19:20:51 +01:00
Daniel García Aubert
62deda6470 Improve naming 2018-02-28 19:13:49 +01:00
Daniel García Aubert
5c2248d419 Merge branch 'master' into project-auth-api 2018-02-28 14:43:12 +01:00
Simon Martín
335d91b42d separate rate limit logic from middleware and EVALSHA 2018-02-28 13:11:18 +01:00
Daniel García Aubert
102b11b1b5 Follow middleware naming convention 2018-02-28 13:10:46 +01:00
Eneko Lakasta
26df09b13f require debug at the top of file 2018-02-28 11:42:44 +01:00
Simon Martín
254991c56c changing endpoint groups constant keys 2018-02-27 17:57:25 +01:00
Simon Martín
a492ab0143 improving naming 2018-02-27 17:36:03 +01:00
Simon Martín
b0d63b2ec0 changing module exports and middleware name 2018-02-27 16:52:27 +01:00
Simon Martín
85b0c63eb0 destructuring array assignment 2018-02-27 16:24:30 +01:00
Simon Martín
98a92f51e6 make jshint happy 2018-02-26 17:17:01 +01:00
Eneko Lakasta
8867cdbc02 use anonymous function instead of arrow function in middleware export to don't bind this 2018-02-26 15:57:42 +01:00
Simon Martín
f03ee4b836 removing hardcoded user 2018-02-26 11:20:31 +01:00
Simon Martín
90418b204e getLowerRateLimit tests 2018-02-26 10:51:51 +01:00
Simon Martín
918674e01a changing endpoint groups names 2018-02-23 21:09:27 +01:00
Simon Martín
9b01a05727 rate limit supporting multiple limits 2018-02-23 17:42:24 +01:00
Simon Martín
2937b6a804 updating redis keys 2018-02-21 17:25:26 +01:00
Simon Martín
08a5e57180 new config params and using them in middleware 2018-02-20 18:18:15 +01:00
Simon Martín
faab174a79 redis keys in easier way 2018-02-20 17:19:50 +01:00
Eneko Lakasta
521b441da5 default apikey is returned by metadata module if no apikey found, remove this code because is never going to be run 2018-02-20 12:53:33 +01:00
Eneko Lakasta
59ca00b33b move apikey credentials getter to middleware file 2018-02-20 12:31:36 +01:00
Simon Martín
80918f5b9b adding rate limit middlewware to named maps controller 2018-02-20 11:26:24 +01:00
Simon Martín
e061b3e631 adding rate limit middlewware to maps controller 2018-02-20 11:26:09 +01:00
Simon Martín
06ec3f80b9 adding rate limit middlewware to named maps admin controller 2018-02-20 11:25:56 +01:00
Simon Martín
e6011287f4 adding middleware to layergroup controller 2018-02-20 11:25:16 +01:00
Daniel García Aubert
7c7d606aa7 Remove trailing spaces 2018-02-19 19:05:13 +01:00
Daniel García Aubert
603ef4044c Reduce cyclomatic complexity 2018-02-19 18:48:02 +01:00
Daniel García Aubert
2e3abfb2cd Catch "name not found" errors from metadata backend and set http code status 404 2018-02-19 18:28:58 +01:00
Simon Martín
98907a886c using a real endpoint in test 2018-02-19 17:08:26 +01:00
Eneko Lakasta
7e14247ea9 remove cause of unreachable code/dead code. Not necessary because carto redis assures at least the default api key 2018-02-19 17:06:59 +01:00
Simon Martín
e519984790 changing const names and exporting them 2018-02-19 17:01:41 +01:00
Eneko Lakasta
7b21bd26d0 Merge branch 'project-auth-api' into project-auth-api-fallback 2018-02-16 11:29:36 +01:00
Eneko Lakasta
4ac224688c in fallback mode, use default api key if api key token doesnt exist 2018-02-16 11:20:04 +01:00
Eneko Lakasta
cda2616a8a get and check api key credentials from api key: username and token 2018-02-15 17:49:47 +01:00
Eneko Lakasta
18dbeea003 get apikey token from request in named maps admin middleware 2018-02-15 15:20:52 +01:00
Eneko Lakasta
3e916c6054 check if req.query exist before getting req.query.api_key/map_key 2018-02-15 15:20:05 +01:00
Eneko Lakasta
fc420c2c0f use for compatibility res.locals.api_key instead of res.locals.apikeyToken 2018-02-15 15:19:09 +01:00
Eneko Lakasta
5db0e9c8d8 add middleware for apikeyToken 2018-02-15 12:50:42 +01:00
Simon Martín
63d1c19263 validation in lua script 2018-02-15 12:44:27 +01:00
Eneko Lakasta
963737d3fb create lib for getting api key token from request 2018-02-15 11:36:42 +01:00
Simon Martín
c059f44bf1 lua script and rate limit process 2018-02-14 18:39:57 +01:00
Eneko Lakasta
890f0d1ef6 add fallback for using metadata fallback 2018-02-14 17:31:05 +01:00
Simon Martín
5fca005a3f Merge branch 'master' into rateLimits 2018-02-14 15:40:50 +01:00
Simon Martín
86d4f8e219 lua script v1 2018-02-14 15:40:12 +01:00
Rafa de la Torre
251fe96509 Cosmetic fix, as suggested in PR 2018-02-12 19:24:53 +01:00
Simon Martín
15bf74f770 starting rate limit middleware 2018-02-12 17:19:08 +01:00
Eneko Lakasta
e1a2ee2381 control API access grants 2018-02-08 13:07:25 +01:00
Eneko Lakasta
b82d26527a remove comment 2018-02-08 12:35:44 +01:00
Eneko Lakasta
b0e9df1400 add pgConnection.getDatabaseParams 2018-02-08 12:04:03 +01:00
Eneko Lakasta
6ebf51ce45 let select apikey type in setDBAuth: regular, default, master 2018-02-08 11:29:17 +01:00
Eneko Lakasta
41f3606572 return unauthorized error when api key not found 2018-02-07 19:12:14 +01:00
Eneko Lakasta
ea0542dcb1 remove use of step 2018-02-07 18:48:59 +01:00
Eneko Lakasta
a4dbc1bac2 remove step and check existance of proper api key 2018-02-07 18:20:56 +01:00
Rafa de la Torre
065f56e161 Improve the speed of the aggregation dataview
Improve the performance of the aggregation dataview.

Instead of using a CTE (WITH) for filtered_source, which is only used in
one place to calculate ranks, inject it as a subquery.

This way the planner has a chance to ignore uneeded columns as well as
to parallelize the exectution of the window function (WindowAgg in the
query plan).

That is the part that takes most of the time of the query.

The improvement is about 20-40% in speed on PG10 with 4 cores.
2018-02-07 18:10:13 +01:00
Eneko Lakasta
95538707c9 add parameter asMaster to setDBAuth 2018-02-07 17:14:13 +01:00
Eneko Lakasta
4c76a921b1 use res.locals instead of req.params 2018-02-07 16:02:13 +01:00
Eneko Lakasta
85c1c987af refactor setDBConn to not use step 2018-02-07 15:49:32 +01:00
Eneko Lakasta
bde86323fd use master api key in setDBAuth 2018-02-07 15:36:24 +01:00
Eneko Lakasta
880e3f388d remove use of _.extend calls 2018-02-07 12:46:10 +01:00
Eneko Lakasta
c1535b1a12 refactor setDBAuth to not use step 2018-02-07 12:40:36 +01:00
Raul Marin
f9fe3ace37 Add mapnik metrics option 2018-02-06 16:07:22 +01:00
Javier Goizueta
2e7784ddf2 Add comment to clafify aggregation resolution limit 2018-02-01 10:26:52 +01:00
Javier Goizueta
d88fbbaa87 Use camelCase 2018-01-31 18:55:28 +01:00
Javier Goizueta
7db0744f67 Simplify expression 2018-01-31 17:54:40 +01:00
Javier Goizueta
d1fcd797a3 Limit the minimum size of the the aggregation grid
Fixes #860
2018-01-31 17:46:13 +01:00
Javier Goizueta
27b5420358
Merge pull request #856 from CartoDB/841-the_geom_webmercator-type
Check the type of the_geom_webmercator for aggregation
2018-01-29 16:13:19 +01:00
Javier Goizueta
7641542e67 Check the type of the_geom_webmercator for aggregation
Fixes #841
2018-01-29 15:48:35 +01:00
Eneko Lakasta
2bd4c9e814
Merge pull request #851 from CartoDB/1259-category-widget-error-group-by-missing
use original column name in group by instead of alias
2018-01-29 15:38:07 +01:00
Eneko Lakasta
941ebf7d80 Merge branch 'master' into 1259-category-widget-error-group-by-missing 2018-01-29 14:51:59 +01:00
Simon Martín
c38bf6ade8 Merge branch 'master' into respect-types-aggreagation 2018-01-29 14:51:20 +01:00
Javier Goizueta
f644b3a226 Add cartodb_id to point-grid aggregation
Fixes #846
2018-01-29 12:49:27 +01:00
Simon Martín
8c839e214d changing the value of string 2018-01-26 15:44:21 +01:00
Simon Martín
99421b613c moving 'other' outside of the query allowing queries of different types 2018-01-26 15:24:21 +01:00
Simon Martín
bc7a556297 removing category cast to string in aggregation 2018-01-26 12:37:10 +01:00
Eneko Lakasta
220f1d6a73 use original column name in group by instead of alias 2018-01-18 15:27:54 +01:00
Daniel García Aubert
da32d96607 Fix regression: default to all layers if layer filter is not provided 2018-01-16 17:57:22 +01:00
Daniel García Aubert
76da828168 Use error label as middleware argument 2018-01-16 17:55:09 +01:00
Daniel García Aubert
ce4050e3e3 Extrac method to get only user params 2018-01-15 18:09:54 +01:00
Daniel García Aubert
b82767c60d Pass a copy of res.locals w/o new data boud to named-map-provider-cache 2018-01-15 17:40:34 +01:00
Eneko Lakasta
0236935212 please jshint 2018-01-11 12:22:51 +01:00
Eneko Lakasta
ae48a01e26 extract setICUEnvVariable() to it's own module 2018-01-11 11:57:11 +01:00
Daniel
fc82ca7490
Merge pull request #834 from CartoDB/middlewarify-named-maps-controller
Middlewarify named maps controller
2018-01-09 11:40:59 +01:00
Daniel García Aubert
183c8291bc Use arrow functions when it applies 2018-01-09 11:20:20 +01:00
Daniel García Aubert
d908ffdbca Don't use arrow functions when there is no needed 2018-01-09 11:17:07 +01:00
Raul Ochoa
d5b5ef584d Be explicit about requesting urlTemplate+subdomains format 2018-01-03 23:33:59 +00:00
Raul Ochoa
2cda43dc8d Promote https urls over http 2018-01-03 22:18:59 +00:00
Raul Ochoa
f7f513a61a Add urlTemplate URLs to metadata
This is useful when using client libraries like leaflet.
2018-01-03 20:53:03 +00:00
Raul Ochoa
f297374449
Merge pull request #837 from CartoDB/tilejson
Return tilejson in metadata
2018-01-03 20:13:44 +01:00
Raul Ochoa
060b93c314 Rename middleware fn name 2018-01-03 18:44:10 +00:00
Raul Ochoa
bf40b240d3 Return tilejson in metadata
It returns tilejson for each individual layer and also for all vector and raster layers.
2018-01-03 16:54:45 +01:00
Raul Ochoa
5d4d2bddd6 Implementation for getTilesUrls
This will be useful for generating the tilejson in the metadata
2018-01-03 16:05:19 +01:00
Daniel
788bc302a0
Merge pull request #833 from CartoDB/middlewarify-named-map-admin-controller
Middlewarify named map admin controller
2018-01-03 13:20:14 +01:00
Daniel García Aubert
1ba240d099 Rename middleware function 2018-01-03 13:15:11 +01:00
Daniel García Aubert
1c7da2c4b3 Going green: do not fail when map-config is vector-only and a layer doesn't have points 2018-01-03 12:00:25 +01:00
Daniel García Aubert
49c97e2cf2 Use default argument 2018-01-02 10:56:45 +01:00
Daniel García Aubert
41e65a9633 Remove max cyclomatic complexity 2018-01-01 18:06:56 +01:00
Daniel García Aubert
feae766e62 Create middleware to fetch named map template 2018-01-01 16:54:35 +01:00
Daniel García Aubert
e3bdeec8ca Simplify middleware 2018-01-01 16:21:22 +01:00
Daniel García Aubert
80c4207c74 Remove underscore dependencie 2017-12-30 18:18:37 +01:00
Daniel García Aubert
80e4306fbc Remove step and assert dependencies 2017-12-30 18:03:26 +01:00
Daniel García Aubert
543d257a20 Move sendResponse to a middleware 2017-12-30 17:18:12 +01:00
Daniel García Aubert
8a023e3d2f Keep error label 2017-12-30 16:08:46 +01:00
Daniel García Aubert
f13b45862d Move incrementMapViews to a middlewares 2017-12-30 16:04:24 +01:00
Daniel García Aubert
731fe4c00f Move getStaticImageOptions and getImage to a middlewares 2017-12-30 15:21:20 +01:00
Daniel García Aubert
500cbb959f Move method to a middleware 2017-12-30 14:13:23 +01:00
Daniel García Aubert
108a319143 Do not use step 2017-12-29 19:33:49 +01:00
Daniel García Aubert
ef5ea5b4cb Create and use getNamedMapProvider middleware 2017-12-29 19:31:02 +01:00
Raul Ochoa
10d1381e51 Merge remote-tracking branch 'origin/master' into pg-mvt-do-not-filter-columns 2017-12-29 17:50:09 +00:00
Daniel García Aubert
dfef7ff3c0 Use spread assignment 2017-12-29 18:45:45 +01:00
Daniel García Aubert
83d0ce4040 Rename method 2017-12-29 18:25:08 +01:00
Daniel García Aubert
75f72c4d07 Return empty aggregation configuration when the map-config is vector-only and the layer has no aggregation 2017-12-29 17:52:28 +01:00