cartodb4/Windshaft-cartodb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,950 Commits 1 Branch 311 Tags 39 MiB
ef9e9f8c78
Commit Graph

573 Commits

Author SHA1 Message Date
Raul Ochoa
b25bb03cdf Merge branch 'master' into private-cdb_tablemetadata 
Conflicts:
	lib/cartodb/server_options.js
	test/support/sql/windshaft.test.sql
 2015-02-16 11:41:45 +01:00
Raul Ochoa
597cb5286d No more before/after state change actions as there is no longer 
a style change
 2015-02-11 18:44:09 +01:00
Raul Ochoa
59dfd11e5b Remove geom_type retrieval 2015-02-10 16:57:43 +01:00
Raul Ochoa
636591ecbb Removes flush_cache endpoint 2015-02-10 00:13:50 +01:00
Raul Ochoa
a4eade31a2 Removes map_metadata endpoint 2015-02-10 00:08:08 +01:00
Raul Ochoa
ba0f394a48 Remove infowindow endpoint 2015-02-10 00:03:44 +01:00
Raul Ochoa
75c4153f9b No need to retrieve api key externally, QueryTablesApi takes care 2015-02-09 19:41:38 +01:00
Raul Ochoa
90b22b2718 QueryTables and last updated_at retrieved with user 
Move setDBAuth and setDBConn to PgConnection entity
 - It uses cartodb-redis to retrieve datasource configuration
Start using it in ServerOptions, TemplateMaps and QueryTablesApi
QueryTablesApi don't receive anymore the connection/credentials
 - It will always use an authenticated query to retrieve last update
 - That will allow to query affected private tables last update
 2015-02-09 14:46:52 +01:00
Raul Ochoa
6bd967e9fb Merge branch 'master' into medusa-improvements 
Conflicts:
	lib/cartodb/server_options.js
 2015-02-06 12:01:28 +01:00
Raul Ochoa
13f5fda1b8 Merge pull request #255 from CartoDB/empty-flush_cache-endpoint 
Remove per table flush cache endpoint
 2015-02-05 17:37:56 +01:00
Raul Ochoa
09527b6808 Remove no longer needed method from query_tables_api 2015-02-05 17:08:20 +01:00
Raul Ochoa
d065ace036 Remove per table flush cache endpoint 2015-02-05 17:05:50 +01:00
Raul Ochoa
fa8b27231c Removed unused/old template lock functionality 2015-02-04 19:36:16 +01:00
Raul Ochoa
c17af23a40 A non empty datasource from MapConfigNamedLayersAdapter.getLayers 
means the affected tables can have private tables involved.
That implies QueryTablesApi will need the proper user to use
CDB_QueryTables. So we store it in a request context to use it in
the afterLayergroupCreate call.

Tiles for these layergroups will fail to add a X-Cache-Channel
header because it won't be possible to use the proper user within
those tiles. Ok, they will fail if they are not requested through
the same tiler instance because if they are they most likely will
reuse the in memory cache.

See https://github.com/CartoDB/Windshaft-cartodb/issues/253
 2015-02-04 19:31:20 +01:00
Raul Ochoa
2c7bc6adde Datasource to give per-layer authentication in named layers 
Make beforeLayergroupCreate to return a datasource with different
 authentication for the different layers.
 - Named layers will get access to private tables in case it's needed

Changes in MapConfigNamedLayersAdapter:
  - It will retrieve the dbAuth params only if named layers are present so
  there is no extra overhead for normal layers
  - Rename queue function signature from `callback` to `done` so it is easier
  to follow the code

Add several tests to validate `named` layers authentication
 2015-02-04 11:30:36 +01:00
Raul Ochoa
4949616c4e Some acceptance tests, http_status = 403 should not happen in adapter, 
needs improvement
 2015-01-30 19:29:45 +01:00
Raul Ochoa
87eaeb0074 Some integration tests for different cases in named layers type 2015-01-30 18:57:01 +01:00
Raul Ochoa
8b07156a2d Make templateMaps available in cartodb windshaft 2015-01-30 16:51:09 +01:00
Raul Ochoa
d0ef87b0cf Add a before layergroup creation action to allow first level named 
maps layer type to be extended as other layers
 2015-01-30 15:31:49 +01:00
Raul Ochoa
e28fe1fdc0 Initialize template maps in server options 2015-01-30 15:30:13 +01:00
Raul Ochoa
aecb07b008 Create redis pool in server options when not supplied 2015-01-30 15:28:55 +01:00
Raul Ochoa
efe090f5b0 Accept 'open' string in templated auth as authorized 2015-01-28 17:29:50 +01:00
Raul Ochoa
6ab6fd91e4 Merge pull request #248 from CartoDB/196-validate-layergroup-in-named-maps 
Basic layergroup validation on named map creation/update
 2015-01-26 17:04:48 +01:00
Raul Ochoa
e690170689 More exhaustive layergroup validation: 
- layers is an array and it's not empty
- layers has at least options
 2015-01-26 15:51:10 +01:00
Raul Ochoa
11d9f5dd76 Basic layergroup validation on named map creation/update 2015-01-23 18:24:25 +01:00
Raul Ochoa
571a635fed Old style, avoid merge conflicts, missing history 2015-01-23 17:46:58 +01:00
Raul Ochoa
6e70518146 Split between old cache_enabled and new purge_enabled configuration 2015-01-23 17:46:16 +01:00
Raul Ochoa
fabb438cf0 Escape \b for the regex 2015-01-23 17:22:49 +01:00
Raul Ochoa
272e8cd221 Adds Surrogate Keys to named maps 2015-01-23 16:37:38 +01:00
Raul Ochoa
885accdadf Adds varnish http port to the default configurations 2015-01-23 16:36:45 +01:00
Raul Ochoa
f5a3b77737 Make TemplateMaps to emit messages when adding/updating/deleting templates 2015-01-23 16:35:47 +01:00
Raul Ochoa
20eb92a3b1 Remove signedmaps and locks functionality as it is no longer needed 2015-01-22 19:28:59 +01:00
Raul Ochoa
8d22ed7594 Tests to validate template instantiation returns new instances with 
default values if they are missing.
 2015-01-22 18:38:42 +01:00
Raul Ochoa
e8ab3a48c6 Removes TemplateMaps dependency on SignedMaps 
- Token validation is done against the template
 - Template is always extended with default values for auth and placeholders
 - MapConfig is extended, in order to validate auth_toknes, with template info:
    - template name
    - template auth
 - No more locks to create, update or delete templates
    - Trusting in redis' hash semantics
    - Some tradeoffs:
        * A client having more templates than allowed by a race condition
        between limit (HLEN) check and creation (HSET)
        * Updating a template could happen while the deleting it, resulting in
        in a new template
        * Templates already instantiated will be accessible thrught their
        layergroup so it is possible to continue requesting tiles/grids/etc.
 - Authorization is now handled by template maps
 2015-01-22 15:40:40 +01:00
Raul Ochoa
64b5a64e1b Add templateMaps to serveroptions for the time being 2015-01-21 11:44:06 +01:00
Raul Ochoa
8bc08d75b7 Separate signed maps instantiation 2015-01-20 18:40:56 +01:00
Raul Ochoa
c14157acc2 Moves template routing 2015-01-20 18:16:09 +01:00
Raul Ochoa
595dac57a0 Moves setDBParams into controller 2015-01-20 18:14:10 +01:00
Raul Ochoa
5632b19e16 Remove unused functionality from app 2015-01-20 18:13:36 +01:00
Raul Ochoa
007196555d Use userByReq from serverOptions 2015-01-20 18:12:24 +01:00
Raul Ochoa
62ffc05ef4 Move template map instantiation into controller 2015-01-20 17:57:53 +01:00
Raul Ochoa
5962141114 Moves template options to controller 2015-01-20 17:45:47 +01:00
Raul Ochoa
7901a05b55 List templates moved into controller 2015-01-20 17:39:33 +01:00
Raul Ochoa
4c2a0ca048 Delete moved to controller 2015-01-20 17:34:23 +01:00
Raul Ochoa
b40c8e6624 Retrieve template moved to controller 2015-01-20 17:17:06 +01:00
Raul Ochoa
97d3b1a03b Move update template to controller 2015-01-20 17:07:55 +01:00
Raul Ochoa
fcea0c9b83 Move template creation to controller 2015-01-20 16:58:12 +01:00
Raul Ochoa
7ce8737e75 Initial split template maps endpoint into its own controller 2015-01-20 16:56:06 +01:00
Raul Ochoa
caa05e779a Add scale_factor param as valid one 2015-01-14 18:11:13 +01:00
Raul Ochoa
845ebcac15 Merge branch 'master' into static-controllers 
Conflicts:
	npm-shrinkwrap.json
 2015-01-13 12:45:37 +01:00
Raul Ochoa
3f1aa9955b Remove query tables api dependency from health check 2015-01-13 12:09:02 +01:00
Alejandro Martínez
96bcd14bb8 Remove PostgreSQL from health checks 
This way the health checks will only check for Redis and Mapnik
initialization.
An empty tile without layers or datasources is generated.
 2015-01-13 11:29:19 +01:00
Raul Ochoa
48d60821a7 Exposes http renderer config 2014-12-01 18:43:40 +01:00
Raul Ochoa
97c55c1187 Removes console.log 2014-11-05 15:42:16 +01:00
Raul Ochoa
7c389a8010 Health check endpoint 2014-11-05 15:06:01 +01:00
Raul Ochoa
c88330f5f2 Allow a different cache-control max-age for layergroup responses 2014-10-24 16:05:41 +02:00
Raul Ochoa
cf5e34eae6 Upgrades Windshaft to start reporting redis/renderers/mapnik pool metrics 2014-10-15 16:45:49 +02:00
Raul Ochoa
9f5faf7cf8 Server options to instantiate cartodb-redis with redis configuration if pool is not provided 2014-10-14 21:19:44 +02:00
Raul Ochoa
711c1a89ee Merge branch 'master' into one-redis-mpool 2014-10-14 18:23:17 +02:00
Raul Ochoa
774104b34e Defaults resultSet to object if undefined in QueryTablesApi 2014-10-13 15:24:14 +02:00
Raul Ochoa
3dad6e96e3 Merge branch 'master' into one-redis-mpool 2014-10-07 12:47:10 +02:00
Raul Ochoa
b4bee864d2 Lock now considers the creation time and compares against a ttl so 
a lock is not keep forever in case of failure.

Pending: lazy removal of expired locks.
 2014-09-25 19:00:35 +02:00
Raul Ochoa
3e571b4ce8 Use object.keys to iterate over objects 2014-09-25 12:17:32 +02:00
Raul Ochoa
fb8fd5121e Do not expose internal implementation 2014-09-25 12:16:34 +02:00
Raul Ochoa
ac2a3243b5 Don't cache regexes and avoid the _re hack 2014-09-25 12:04:52 +02:00
Raul Ochoa
1c10b8193b Adds dot to compile templates 2014-09-24 19:17:51 +02:00
Raul Ochoa
abf0fa1b32 Remove unused var 2014-09-24 19:12:43 +02:00
Raul Ochoa
4c5bc13c7f Check style fixes 2014-09-24 19:11:53 +02:00
Raul Ochoa
7009eb20f8 Check style fixes 2014-09-24 11:42:53 +02:00
Raul Ochoa
24cbd192aa Share one redis-mpool across the application 2014-09-24 11:42:36 +02:00
Raul Ochoa
9496d83d1c Adds poolSize configuration for mapnik 2014-09-18 19:06:45 +02:00
Raul Ochoa
7faff8f887 Removes cors metric 2014-08-26 11:53:59 +02:00
Raul Ochoa
f406001315 Removes get_infowindow and get_map_metadata metrics 2014-08-25 19:15:31 +02:00
Raul Ochoa
2b2020b43b Removes getTableGeometryType metric 2014-08-25 19:09:54 +02:00
Raul Ochoa
65e0364d37 Removes duplicated code in query_tables_api 2014-08-22 12:52:05 +02:00
Raul Ochoa
965e1cd0c4 Supports !scale_denominator! dynamic param in SQL queries 2014-08-22 10:16:39 +02:00
Raul Ochoa
a3a5964926 Upgrades dependencies 2014-08-14 19:54:45 +02:00
Raul Ochoa
6a8cff6fcd Merge branch 'remove-mapnik-dependency' 
Conflicts:
	NEWS.md
 2014-08-14 19:26:52 +02:00
Raul Ochoa
23a7684208 Removes mapnik dependency as it now relies on Windshaft to check mapnik version 2014-08-14 18:27:54 +02:00
Raul Ochoa
f2541d8cae Merge branch 'master' into CDB-3686 
Conflicts:
	package.json
 2014-08-13 15:31:22 +02:00
Raul Ochoa
40acf533ae Specifies name in the redis pool 2014-08-13 15:12:46 +02:00
Raul Ochoa
e7ab71c606 Merge branch 'master' into CDB-3686 2014-08-11 12:19:11 +02:00
Raul Ochoa
2ed656ca0d Upgrades windshaft (and grainstore) to be able to specify the tile 
format, see: https://github.com/mapnik/mapnik/wiki/OutputFormats
 2014-08-07 01:57:21 +02:00
Raul Ochoa
5cf79c82bb Configurable QueryTablesAPI to call directly postgresql using cartodb-psql 
or to keep using a request to the SQL API
 2014-08-06 21:48:08 +02:00
Raul Ochoa
d1373bec66 Improves SQL query for affected tables and last updated time 2014-08-04 17:48:59 +02:00
Raul Ochoa
dabcba9f5f Merge branch 'master' into CDB-3686 2014-08-04 13:11:40 +02:00
Raul Ochoa
414673b347 CDB-3657 Adds profiler as JSON to the header 2014-08-04 12:53:15 +02:00
Raul Ochoa
507a6a8979 CDB-3686 Style changes 2014-08-04 01:32:49 +02:00
Raul Ochoa
9f8d73a1df Removes duplicated file 2014-07-30 18:17:14 +02:00
Raul Ochoa
3af45e1a32 Moves calls to SQL API to its own entity. 
Groups affected tables and last updated time for affected tables into one request.
 2014-07-30 13:46:46 +02:00
Raul Ochoa
75088c89d3 Style fixes 2014-07-30 13:45:53 +02:00
Raul Ochoa
67df6a4d73 Adds support for several auth tokens 2014-07-08 10:35:45 +02:00
Raul Ochoa
f756b9d77f Removes search_path param 2014-07-04 12:18:35 +02:00
Raul Ochoa
bfdcee3772 Retrieving db public user from redis. It uses a new multiget method from cartodb-redis 2014-07-03 21:39:47 +02:00
Raul Ochoa
470aea22d9 Sets full search_path 2014-07-03 10:24:37 +02:00
javi
3806ad8843 Merge remote-tracking branch 'origin/CDB-2891-search_path' into multiuser 2014-06-27 09:10:39 +02:00
Raul Ochoa
037ce2dc12 CDB-2891 Exposes username as search_path in params 2014-06-27 00:48:48 +01:00
javi
338c0bcdbe use regclass instead table name to look for last_updated in CDB_tablemetadata 2014-06-26 15:00:55 +02:00
Raul Ochoa
4cf1ddd6fc CDB-3256 Adds response and method references to fake request object 2014-06-24 15:52:47 +02:00
Raul Ochoa
5e73b12cf5 CDB-3256 adds headers based on affected tables when creating a layergroup via HTTP GET 2014-06-24 12:16:30 +02:00
Sandro Santilli
49829f8935 Set default PostgreSQL application name to "cartodb_tiler" 2014-05-07 16:19:22 +02:00
Sandro Santilli
882ec65ba0 Use signer's map_key when contacting sql-api 
Includes testcase.
Fixes #188
 2014-04-08 09:44:49 +02:00
Sandro Santilli
9e495b42ee Do not cache non-success jsonp responses 
Closes #186
Includes testcase
 2014-03-21 13:58:20 +01:00
Sandro Santilli
60b552027b Add optional support for rollbar 
Re-targets to 1.10.0
Also installs an uncaught exception handler

Closes #150
 2014-03-12 17:21:35 +01:00
javi
5bfc360856 added serverMetadata option for layer group, close #182 CDB-1940 2014-03-06 15:19:12 +01:00
Sandro Santilli
0afc9c154b Cleanly catch exceptions from sendResponse 
Closes #178
 2014-03-04 18:04:58 +01:00
Sandro Santilli
97e00fb47d Do not send duplicated stats on template instanciation 
Closes #179
 2014-03-04 17:51:50 +01:00
Sandro Santilli
ffda103d61 Do not UNWATCH on every redis client release 
Closes #161
 2014-03-04 15:36:08 +01:00
Sandro Santilli
ecc9ea1226 Use 403 for forbidden, not 401 
Includes upgrade of windshaft to 0.19.3
Includes upgrade of redis-mpool to 0.0.4
 2014-03-04 15:32:31 +01:00
Sandro Santilli
93345a19b2 Do not log an error on GET / 
Closes #177
 2014-03-04 14:26:41 +01:00
Sandro Santilli
1741a20575 Do not cache map creation responses 
Closes #176
CDB-1908 #resolve
CDB-1901 #resolve

Includes testcase
 2014-03-04 10:46:15 +01:00
Sandro Santilli
30eb939dc7 Fix error message on missing requested signature 
We don't really distinguish between missing or non-authorizing
signature. And that's fine. See #170
 2014-03-03 18:14:17 +01:00
Sandro Santilli
40a254922a Raise 403 forbidden on missing requested signature 
Closes #170
Includes testcase
 2014-03-03 18:06:39 +01:00
Sandro Santilli
7bc5bab432 Properly prefix statsd labels for all endpoints 
CDB-1861 #resolve
Will be 100% complete with update of Windshaft to 0.19.3+
 2014-03-03 16:24:20 +01:00
Sandro Santilli
bddc65a504 Forbid instanciating templates of foreign users 
Closes #173
Includes testcase
 2014-02-28 16:05:46 +01:00
Sandro Santilli
ddd2628c19 Fix database connection settings on template instanciation 
Closes #174
Enhances testsuite to ensure test.js settings are read
 2014-02-28 15:56:31 +01:00
Sandro Santilli
f46dc90035 Forbid using map signatures of foreign users 
Closes #173
Includes testcase
 2014-02-28 13:24:38 +01:00
Sandro Santilli
55f333c0b7 Call userByReq() only once in req2params 2014-02-27 16:40:59 +01:00
Sandro Santilli
f24e4f8a0a Really skip CDB_TableMetadata lookup for sql affected by no tables 
Closes #169
 2014-02-27 15:34:09 +01:00
Sandro Santilli
36632c762e Do not query CDB_TableMetadata for queries affected by no tables 
Closes #16
 2014-02-27 12:32:34 +01:00
Sandro Santilli
f284362988 Reduce sql-api communication timeout, and allow overriding it 
Introduces new sqlapi.timeout directive, defaults to 100 ms
Includes testcase.
Closes #167
 2014-02-27 10:33:32 +01:00
Sandro Santilli
a13c1f61af Do not log an error for a legit request requiring no X-Cache-Channel 2014-02-24 17:34:00 +01:00
Sandro Santilli
5c466c51a8 Revert order of hostname components for statsd.prefix 2014-02-21 17:25:10 +01:00
Sandro Santilli
09ea924eb2 Allow using GET with sql-api for queries shorter than configured len 
Introduces new sqlapi.max_get_sql_length directive, defaults to 2048.
Closes #155
Includes testcases.
 2014-02-20 10:17:48 +01:00
Sandro Santilli
c8a042abdd Expand "addCacheChannel" stats 2014-02-19 18:10:33 +01:00
Sandro Santilli
b4fc8ec4a5 Allow using ":host" as part of statsd.prefix 
It'll be replaced with hostname.
Closes #153
 2014-02-19 15:31:12 +01:00
Sandro Santilli
6c6f3d02f6 Always generate X-Cache-Channel for token-based tile responses 
Closes #152
 2014-02-19 10:09:54 +01:00
Sandro Santilli
36a135f02b Refactor addCacheChannel using Step 2014-02-19 07:19:41 +01:00
Sandro Santilli
1c3734fde7 Make server_option a callable function, to reduce globals 
Updates acceptance test for #152 to not mess wit internals
 2014-02-19 06:45:29 +01:00
Sandro Santilli
3c09be64ce Add pending test for X-Cache-Channel on tiler restart (#152) 2014-02-18 18:33:00 +01:00
Sandro Santilli
c14378ca5d Avoid checking for table privacy when not using table maps 
See #147
 2014-02-17 18:20:18 +01:00
Sandro Santilli
26b9c8123d Set maxSocket to allow more than 5 concurrent connections to sql-api 2014-02-17 18:03:11 +01:00
Sandro Santilli
1f2e4edd35 Comments cleanup 2014-02-17 11:10:08 +01:00
Sandro Santilli
212eec2ca6 Pass profiler back to windshaft on createLayergroup 2014-02-17 08:50:12 +01:00
Sandro Santilli
935826ed1a Integrate statsd in template instanciation endpoint 
NOTE: stats are only enabled using windshaft 0.18.2+
 2014-02-15 08:23:43 +01:00
Sandro Santilli
8f3c6c3c87 Add profiler calls in template instanciation endpoint 2014-02-15 08:06:57 +01:00
Sandro Santilli
63401ca3df Use a single redis client in SignedMap.isAuthorized 2014-02-14 17:07:52 +01:00
Sandro Santilli
def474c611 Skip getting geometry type if request has no table 2014-02-14 12:26:34 +01:00
Sandro Santilli
c1b2d16119 rename tablePrivacy_getUserDBName profile label 2014-02-14 11:47:43 +01:00
Sandro Santilli
678d653ee9 Allow configuring TTL of mapConfigs via "mapConfigTTL" 2014-02-13 15:44:54 +01:00
Sandro Santilli
4a6af108b4 Fix use of maxUserTemplate configuration variable 2014-02-13 15:01:58 +01:00
Sandro Santilli
e4cd37647e Allow limiting number of allowed user templates 
Adds maxUserTemplates directive.
Closes #136
 2014-02-13 14:55:31 +01:00
Sandro Santilli
f7cef9dcd8 Fix bogus reference in SignedMaps when globals.environment is not set 2014-02-13 10:57:41 +01:00
Sandro Santilli
b673cb2a1f Add more detailed profile info about the "authorize" step 
Closes #142
 2014-02-13 10:25:28 +01:00
Sandro Santilli
e88e49001a Do not retrive user's api key if no api key was provided 
Reduces redis interaction, see #142
 2014-02-13 10:16:11 +01:00
Sandro Santilli
bf45bbea56 Do not send multiple equal commands to Varnish on connect 
Closes #135
Also accept varnish "secret" in config
 2014-02-12 16:14:27 +01:00
javi
01feeae6f4 include state configuration for windshaft fixes #139 2014-02-12 15:27:42 +01:00
Sandro Santilli
a948038ff4 Disable debug logging unless "debug" config param evaluates to true 
Closes #137
 2014-02-11 16:34:43 +01:00
Sandro Santilli
8c013ed2d1 Rename Step function in setDBConn 2014-02-11 13:42:44 +01:00
Sandro Santilli
7a749631e8 Fix profiler labels 2014-02-11 13:40:17 +01:00
Sandro Santilli
747f4803ba Include hash of template in the maptoken returned from instanciation 
Doing so basically removes the need to include the template identifier
in the surrogate keys of the responses for resources fetched via
the instance whenever template is updated. See #105
 2014-02-10 15:30:35 +01:00
Sandro Santilli
399bed34ad Do not try to replace template variables in undefined elements 
See #133
 2014-02-10 11:26:21 +01:00
Sandro Santilli
6b41fef96c Fix sendError calls to receive the full Error instance 2014-02-10 11:11:35 +01:00
Sandro Santilli
fe6e915c0d Always set database access parameters from req2params 
Fixes privileged database access from unauthorized users while
fetching torque tiles or feature attributes (unreleased feature).
Closes #132.

Includes testcase, which closes #119
 2014-02-07 18:08:41 +01:00
javi
f5e0d06e2f fixed when default value in a template attribute is a number and type = number checking fails fixed #130 2014-02-06 17:45:48 +01:00
javi
78f69d5236 template variables with spaces are not replaced correctly fixed #129 2014-02-06 17:33:26 +01:00
Sandro Santilli
b4936ffafa Do not allow creating template with auth='token' and no valid tokens 
Closes #128
Includes acceptance test for both creation and update
 2014-02-06 12:24:14 +01:00
Sandro Santilli
752e9ec655 Add checkInvalidCertificate method for SignedMap class 
Includes unit test
 2014-02-06 12:05:01 +01:00
Sandro Santilli
9018e39762 Make endpoints configurable 
Closes #127
Uses /api/v1/maps* in the production and staging example configs,
keeps /maps* for development and test (they are examples...)
 2014-02-05 15:14:47 +01:00
Sandro Santilli
a964ed5fe6 Implement Unified Map API 
Closes #126
 2014-02-04 19:04:59 +01:00
Sandro Santilli
b862904506 Be explicit about the map output srid configuration 2014-02-04 16:26:26 +01:00
javi
7197cc2d62 added stack to response in development mode 2014-02-04 14:58:21 +01:00
Sandro Santilli
978ea9cd04 Fix sqlapi request header to be "Host", not "Hostname" 
Closes (better) #117 -- automated test included
 2014-01-30 16:46:26 +01:00
Sandro Santilli
ca4f3d2025 Re-introduce sqlapi.host directive, allowing DNS lookups drop 
For backward compatibility, sqlapi.host is only used if domain
is also defined and has a different value (empty string allowed).

Closes #117
 2014-01-30 16:12:37 +01:00
Sandro Santilli
1f0faba71c Stop processing XML on renderer creation 
Not needed anymore since 1.6.1 introduced on-demand XML generation.
 2014-01-30 11:14:52 +01:00
Sandro Santilli
d32278b227 Rename template instanciation function 2014-01-29 14:30:27 +01:00
javi
ad0b2ffc8e added support for template instanciation with jsonp closes #116 2014-01-29 13:11:37 +01:00
Sandro Santilli
67e4e7e99b Set api_key to signer's when instanciating a template map 
Closes #114
 2014-01-28 12:37:41 +01:00
javi
92ca447c06 fixed #91 2014-01-28 12:05:01 +01:00
javi
e4e08db0b4 Merge branch 'master' of github.com:Vizzuality/Windshaft-cartodb 2014-01-22 19:10:37 +01:00
javi
164d952e56 support CORS in template instanciation endpoint, fixes #113 2014-01-22 19:10:09 +01:00
Sandro Santilli
5772c81590 Fix support for long (>64k chars) queries in layergroup creation 
Closes #111. Includes testcase.
 2014-01-16 17:20:30 +01:00
Sandro Santilli
f89fd98ed7 Expect malformed response objects (#109) 
Include test for sql errors on layergroup creation
Closes #109
 2014-01-15 11:53:19 +01:00
Sandro Santilli
d6fe5339cf Do not choke on headers cleanup when response headers are not set 
Raise a WARNING instead.
See #107 (github) and CDB-1438 (JIRA)
 2014-01-13 18:56:09 +01:00
Sandro Santilli
2690ef3f05 Drop cache headers from error responses. 
Closes #107 (github), #resolve CDB-1423 (JIRA)
 2014-01-13 11:20:02 +01:00
Sandro Santilli
4ee4492490 Yet another username extraction fix. Thanks again @demimismo. 
Closes #100 (yet again)
 2014-01-09 16:46:47 +01:00
Sandro Santilli
fcd17692ee Fix username extraction in another two places. Thanks @demimismo. 
Closes #100 (again)
 2014-01-09 15:36:16 +01:00
Sandro Santilli
ef3733aebe Improve error on attempt to delete missing template 2013-12-20 12:54:38 +01:00
Sandro Santilli
c7465479a2 Improve error on a signature certificate with no or broken auth 2013-12-20 10:41:27 +01:00
Sandro Santilli
50a902a90b Fix english of error message for sql-api connection problems 2013-12-18 12:59:26 +01:00
Sandro Santilli
0f90d687c7 Implement signed teplate maps 
Closes #98

Raises minimum required redis version to 2.4.0+ (Debian stable has 2.4.14)
 2013-12-17 17:39:21 +01:00
Sandro Santilli
84b7d78ea4 Add an utility authorizedByAPIKey method for reuse 2013-12-17 17:17:17 +01:00
Sandro Santilli
73a065c1cc Make sure user from domain is always computed locally 
Involved upgrade of cartodb-redis to 0.3.0
Really closes #100
 2013-12-17 17:17:17 +01:00
Sandro Santilli
1f693c6c78 Add 'user_from_host' directive to generalize username extraction 
Closes #100
Default extractor is backward compatible
 2013-12-17 17:17:17 +01:00
Sandro Santilli
e9db535dd8 Drop the idea that we can distinguish a "dbowner" from the domain 
We only recognize "users"
 2013-12-17 17:17:17 +01:00
Sandro Santilli
7b7408dab7 Revert "Drop /map_metadata API entry point" 
This reverts commit b37b07a06a1dd3cf05d60f4aa613ab5c48b90700.

This was too light of a decision...
 2013-12-17 17:17:17 +01:00
Sandro Santilli
9c897a91a9 Drop /map_metadata API entry point 
Closes #101
 2013-12-17 17:17:17 +01:00
Sandro Santilli
38342a7f5f Refactor req2params to make setting db credential easier 2013-12-17 17:17:16 +01:00
Sandro Santilli
276422f4be Set grainstore's GC run probability, for documentation purpose 
It sets it to the current grainstore default, so nothing changes.
 2013-12-17 17:17:16 +01:00
Sandro Santilli
385022de80 Revert "fixed #91" -- the fix was for an unconfirmed bug 
This reverts commit 9155724082.
See #38 for further action
 2013-12-17 17:17:16 +01:00
javi
a378fc4e68 fixed #91 2013-12-17 17:17:16 +01:00
javi
01de288c35 fixed #96 2013-12-17 17:17:15 +01:00
Sandro Santilli
cc09a8b66f Update to cartodb-redis 0.1.0 2013-12-17 17:17:15 +01:00
Sandro Santilli
a60a3adc12 CartoDB redis interaction delegated to "cartodb-redis" module 2013-12-17 17:17:14 +01:00
Sandro Santilli
baa95a62d1 Add support for reading user-specific database_password from redis 
This commits adds support for CartoDB-2.5.0 model.
Closes #89.
Change is backward compatible.
 2013-12-17 17:17:14 +01:00
Sandro Santilli
12f0826d32 Do not force ending dot in SQL-API hostname, for easier testing 2013-12-17 17:17:14 +01:00
Sandro Santilli
07cb36ebc7 Read user's database_host from redis, when available (#88) 
Still lacks a testcase
 2013-12-17 17:17:14 +01:00
Sandro Santilli
d7c82e7a51 Indent fixes 2013-12-17 17:17:14 +01:00
Sandro Santilli
bf340e684a Tweak error messages on missing redis variables, update tests 2013-12-17 17:17:14 +01:00
Luis Bosque
8d1b394df1 Add function to read database host from redis 2013-12-17 17:17:14 +01:00
Sandro Santilli
eb51d18012 Add support for specifying database connection passwords 2013-12-17 17:17:13 +01:00
Sandro Santilli
a27cf1b41c Do not let anonymous requests use authorized renderer caches 
Puts dbuser in params, for correct use by Windshaft renderer cache.
Before this fix, and after commit 1c9f63c9, the renderer cache key
did not contain the db user.
 2013-12-17 17:17:12 +01:00
Sandro Santilli
f5c24cf252 Add more profile slots 2013-12-17 17:17:11 +01:00
Sandro Santilli
dbf6bb5fca Only use sqlapi configuration "host" if "domain" is undefined 
We'll consider an empty string domain as valid (it's actually used
for testsuite).
 2013-12-17 17:17:11 +01:00
Javier Arce
d4d5272bf2 Sets the sqlapi domain. Fixes #82 2013-12-17 17:17:11 +01:00
Sandro Santilli
633e8d164b Rename sqlapi.host configuration to sqlapi.domain. Closes #79. 
Support for "host" is retained for backward compatibility.
 2013-08-21 10:11:30 +02:00
Sandro Santilli
ec97381820 Add more timing in the profile, add useProfiler config variable 
Default to useProfiler:true in staging and development
 2013-07-16 16:33:03 +02:00
Sandro Santilli
381b9a9edf Take cache_buster value, if present, as a Last-Modified timestamp 
This makes the Last-Modified header consistent across requests
using the same cache_buster (embedded in the token for multilayer
API).
 2013-07-15 13:48:06 +02:00
Sandro Santilli
76c056c7a1 Revert "Use a constant Last-Modified time with cache_policy=persist" 
This reverts commit 4b5899ff1a.

The reason is that setting Last-Modified to a remote date in the past
triggers early expiration of cache (as max-age will be reached sooner)
 2013-07-15 13:14:06 +02:00
Sandro Santilli
4b5899ff1a Use a constant Last-Modified time with cache_policy=persist 
After all if the client is asking for persistance it doesn't make
sense to set a different Last-Modified for different incoming
requests (even if we don't expect any) ....
 2013-07-15 12:09:13 +02:00
Sandro Santilli
afd4c3b460 Set Last-Modified header to allow for 304 responses 2013-07-15 12:02:54 +02:00
Sandro Santilli
65c7c5fc9c Always serve multilayer tiles and grids with persisting cache request 2013-07-08 12:13:45 +02:00
Sandro Santilli
60242c80f4 Set default layergroup time to live in redis to 2 hours 2013-07-08 11:50:19 +02:00
Sandro Santilli
632d75a7c8 specify units for rendererConfig.cache_ttl 2013-06-28 17:58:11 +02:00
Sandro Santilli
d7b1ff9a80 Set default layergroup ttl locally 2013-06-26 16:26:02 +02:00
Sandro Santilli
eb83851bb7 Fix database authentication with multi-table layergroups 2013-06-17 17:24:09 +02:00
Sandro Santilli
70f535d13a Properly report error from unsuccessful source table fetching 
Report terse error to user, verbose to log
 2013-06-11 10:28:05 +02:00
Sandro Santilli
63296a87cb Do not increment undefined mapview stat tags 2013-06-06 13:26:59 +02:00
Sandro Santilli
d977f83bd1 Change stats format for multilayer map token request 
See https://github.com/Vizzuality/Windshaft-cartodb/wiki/Redis-stats-format

Target 1.3.0
 2013-06-04 13:30:28 +02:00
Sandro Santilli
5b6919e0c6 Fix unit of measure for lastUpdated info extraction 2013-05-30 16:48:40 +02:00
javi
efa79b243c fixed lzma decoding to fix browser requirements 2013-04-24 15:10:58 +02:00
Sandro Santilli
f0fc44aac9 Fix fetching of affected tables when mapnik tokens are used 
We'll replace !bbox! with an empty box and !pixel_width! and
!pixel_height! with 1 before passing the query to CDB_QueryTable
 2013-04-23 17:29:49 +02:00
Sandro Santilli
f85ca16c62 Change LZMA expected encoding from HEX to base64, reducing its size 2013-04-19 16:16:20 +02:00
Sandro Santilli
14953e992f Multilayer API changes, target 1.2.0 
- Layers passed by index in grid fetching url
 - Interactivity only specified in layergroup config
 - Encode cache_buster as part of the token
 2013-04-15 18:51:28 +02:00
Sandro Santilli
78a6f4de1b Keep a counter of layergroup created per user. 
The counter is in redis db 5, in a field "mapviews" of an hash
"tiler:users:USERNAME". It's incremented whenever the layergroup
token for a configuration is requested.
 2013-04-12 17:28:34 +02:00
Sandro Santilli
83e6e0d457 More verbose logging for SQL api connection errors 2013-04-09 18:07:53 +02:00
Sandro Santilli
e5af3b90f4 Revert "Require interactivity param in single-layer grid fetching request" 
This reverts commit 3383c44eb7.

Fixes regression with default interactivity parameter.
Closes #74. See #69.
 2013-04-05 18:11:36 +02:00
Sandro Santilli
113b70cf98 Add support for creating layergroups via GET 2013-04-02 13:30:49 +02:00
Sandro Santilli
3383c44eb7 Require interactivity param in single-layer grid fetching request 
Closes #69
 2013-03-29 18:25:28 +01:00
Sandro Santilli
001bf97d69 Add support for LZMA compressed GET parameters 
You can now replace the whole query string with a single `lzma`
parameter having as value an hex encoded LZMA compressed version
of the whole query string as a JSON object.
 2013-03-22 18:55:59 +01:00
javi
e53122de7e fixed last_update in laytergroup response 2013-03-21 11:39:55 +01:00
Sandro Santilli
3ff3dc2c97 Cleanup, handle error in req2param on flushCache 2013-03-15 19:25:13 +01:00
Sandro Santilli
4605bd1e1d Add last_modified field to POST layergroup response (#72) 
Includes testcases
 2013-03-13 18:41:37 +01:00
Sandro Santilli
dfc4a02398 Fix X-Cache-Channel for multilayer (by token) responses 
Required upgrading Windshaft to 0.9.2
Includes testcases
 2013-03-13 16:45:15 +01:00
Sandro Santilli
402fc90e63 Absence of X-Cache-Channel will be enough for Varnish to skip caching 
Do not override Cache-Control in this case, which means let the
clients or geographical proxies cache the response with usual TTL.
 2013-03-13 12:01:35 +01:00
Sandro Santilli
e8cbc666e2 Handle SQL API errors by logging them and requesting NO cache 
SQL api is used to determine the list of source tables affected
by a query. Before this commit, the X-Cache-Channel header set
on sql api error was an arbitrary 'table' string, now the header
is omitted, the error logged and Cache-Control and Pragma headers
are sent as an attempt to request no caching.

The code includes test for this mechanism.
 2013-03-13 10:39:00 +01:00
Sandro Santilli
f6d50fafb1 Expose renderer settings in the environment config files 
These are: metatile, bufferSize and cache_ttl
 2013-02-25 17:05:59 +01:00
Sandro Santilli
9dcf6a1acf Set 'base_url_notable' config for Windshaft-0.9 (multilayer) 2013-02-12 18:53:41 +01:00
Sandro Santilli
a79b999e7a Do not try to send commands to an unoconnected redis client 
This changes "Cannot read property 'HGET' of null" messages into
"Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED".
 2013-02-11 15:05:23 +01:00
Sandro Santilli
6b71cde56e Do not throw an Error embedding another Error 
Should fix #68, but doesn't come with an automated test
 2013-02-08 12:27:49 +01:00
Sandro Santilli
cb57dfb27d Fix async throws in getGeometryType, getInfoWindow and getMapMetadata 2013-02-08 12:14:53 +01:00
Sandro Santilli
8d4f033a56 Revert "getDatabase: properly handle redis connection failures" 
This reverts commit dd19d74149.

The code was already correct
 2013-01-28 17:39:50 +01:00
Sandro Santilli
dd19d74149 getDatabase: properly handle redis connection failures 2013-01-28 17:30:58 +01:00
Sandro Santilli
ac49abe750 Do not leak redis client connections on redis command error 2013-01-28 17:13:49 +01:00
Sandro Santilli
b130b67f24 Check redis connection at pool creation time 2013-01-28 17:12:21 +01:00
Sandro Santilli
19436a8b14 Let "style_convert" pass by, add tests for GET and POST with it 2012-11-14 15:28:58 +01:00
Sandro Santilli
52303e7821 Fix use of "style_version" with GET (inline styles) 
It took a lot of time to produce a testcase for this as the test
config was setting srid to 4326 but not changing geom column name
thus all tiles fetched by tests returned blank (ouch!)
 2012-11-06 12:45:04 +01:00
Sandro Santilli
2bc09a61cf Add support for cache_policy=persistent 
When cache_policy=persistent is given the response will contain
a Cache-Control header requesting for 1 year lifetime caching
 2012-10-24 09:40:05 +02:00
Sandro Santilli
d9e6aeb254 Fix crash on unknown user. Closes #55. 2012-10-22 15:30:16 +02:00
Sandro Santilli
ab8cb5bbb3 Add Windshaft-cartodb version to the /version route 2012-10-15 17:03:57 +02:00
Sandro Santilli
20dca2e8f8 Use windshaft-0.6.2 sendError function to send non-200 responses 
Ensures all errors are logged
 2012-10-11 16:48:41 +02:00
Sandro Santilli
6a1933bed9 Print a warning when configured mapnik version doesn't match installed 2012-10-09 11:45:57 +02:00
Sandro Santilli
961269fa1f Autodetect target mapnik version and let config override it 
Closes #40
 2012-10-08 17:45:03 +02:00
Sandro Santilli
8a759babf0 Add tests for getting metadata (#183) 
... and fix forbidden metadata response
 2012-10-05 17:08:24 +02:00
Sandro Santilli
7196c8c285 Only invalidate cache on del style when caching is enabled 2012-10-05 16:55:58 +02:00
Sandro Santilli
dc9286b610 Accept "api_key" as "map_key", in both query_string and POST body 
Closes #38
 2012-10-05 16:17:49 +02:00
Sandro Santilli
1ed65544e5 Send detailed error when user metadata are missing from redis 
Include tip on how to restore the redis db from cartodb.
 2012-10-05 16:05:32 +02:00
Sandro Santilli
a201888fde Make logging format configurable (closes #4) 
NOTE: the default format for the "test" environment is without
ansi colors, to be easier on remote terminal sessions
 2012-10-05 15:52:51 +02:00
Sandro Santilli
bc506784ca Add an X-Cache-Channel header to all GET requests. Closes #53. 2012-09-25 09:27:03 +02:00
Sandro Santilli
dd34fc507a Automated localization of external resources referenced in carto 2012-09-19 18:52:13 +02:00
Sandro Santilli
dcbe051654 Return 401 status for unauthorized requests (see #48) 2012-09-05 20:16:55 +02:00
Sandro Santilli
b469d5eabf Rename authenticated postgresql user configuration setting 
See issue #49
 2012-09-05 15:49:10 +02:00
Sandro Santilli
5ca84d9ffc Always replace the database user parameter in the XML style 
This provides for an automatic migration of styles to any new
authentication user.
 2012-09-05 15:41:22 +02:00
Sandro Santilli
87b0e07815 Expose Varnish cache TTL in configuration. Closes #46 2012-09-03 18:57:35 +02:00
Sandro Santilli
b9e29938e2 Compile XML processing regular expression only once. 
Requests per second from ~95 to ~98 with a "standard" test.
 2012-09-03 15:30:31 +02:00
Sandro Santilli
1c9f63c901 Add XML processor to change database user when authenticated 2012-09-03 14:54:23 +02:00
Sandro Santilli
9ba80acab7 Properly bubble getInfowindow errors up to response 2012-08-14 20:01:05 +02:00
Sandro Santilli
b30c8d9a98 Send jsQuery ready error message on GET /infowindow errors 2012-08-14 20:00:26 +02:00
Sandro Santilli
ca47fbd10b Fix unauthenticated access to table styles. Closes #43. 
Re-introduces use of the redis "privacy" hash key as an additional
security measure for requests that do not involve PostgreSQL access.

Accessing private table styles is tested with this commit.
Accessing private table metadata or infowindow is _not_ tested,
but should also be fixed now.
 2012-08-14 19:26:40 +02:00
Sandro Santilli
fb3f3a312e Really fix setting or deleting styles from unauth. request 
Closes #44
 2012-08-14 16:15:41 +02:00
Sandro Santilli
b5348db94c Add CORS headers to all requests 
In addition to the ones served by the base Windshaft server this commit
adds CORS headers to the GET /infowindow, GET /map_metadata and
DEL /flush_cache requests. Closes issue #42.
 2012-08-14 14:50:18 +02:00
Sandro Santilli
282ac94e29 Prevent unauthenticated requests from changing map styles 2012-08-07 17:10:15 +02:00
Sandro Santilli
6a92fd3170 Propagate style changes to caches for unauthenticated requests 
Closes #41, does it implementing the new afterStyleChange and
afterStyleDelete callbacks in Windshaft 0.4.10.

Adds automated testcases for the bug.
 2012-08-07 16:11:49 +02:00
Sandro Santilli
651c45bc16 Use the correct redis variable for checking map key (closes #39) 
Includes testcase
 2012-08-02 11:15:01 +02:00
Sandro Santilli
de275bfc50 Delegate user permission to PostgreSQL (closes #18) 
If the request is authenticated (with map_key) then we log as the
database owner, otherwise we log as the default user.
The default user is now "publicuser" by default.

Raises dependency on Windshaft to 0.4.9+, to get the grainstore
version allowing override of database username.

Add test for req2params function, particularly authentication,
Add test for authenticated / unauthenticated access
 2012-07-18 11:09:17 +02:00
Sandro Santilli
15162e8c38 Use global environment settings in carto_data (closes #27) 2012-07-09 19:33:17 +02:00
Simon Tokumine
98e5bab8e4 commenting and removal of bogus config. also disable cache in test environment 2012-06-06 15:24:44 +01:00
Simon Tokumine
a382b9f601 check for nulls. 2012-05-08 10:52:58 +01:00
Simon Tokumine
ac7593a743 check for nulls. out 2012-05-08 10:51:51 +01:00
Simon Tokumine
13b66c4e38 check for nulls 2012-05-08 10:38:01 +01:00
Simon Tokumine
4078098c3f enable cache clearing at table level granularity 2012-05-02 19:32:54 +01:00
Simon Tokumine
d1226992c3 clean up environment settings 2012-05-01 19:00:14 +01:00
Simon Tokumine
dcea209ca5 update log format 2011-12-14 00:56:10 +00:00
Simon Tokumine
ef15f4b48b migrated to node-varnish, fixed tests, refactor 2011-12-12 18:02:10 +00:00
Simon Tokumine
e810747a21 Merge branch 'master' of github.com:Vizzuality/Windshaft-cartodb 2011-12-09 00:32:41 +00:00
Simon Tokumine
78d415569d pin to mapnik 2.0, enable passing style via url 2011-12-09 00:32:32 +00:00
javi
b53bcceb6d lovely fixes 2011-12-07 13:06:23 +01:00
javi
5084e69aff added varnish tests 2011-12-07 11:54:24 +01:00
Simon Tokumine
5fbd3a376e cache age of 60 mins 2011-12-06 21:46:17 +00:00
Simon Tokumine
1bb5824cf7 spelling: 2011-12-06 21:21:19 +00:00
javi
2f9987beb8 Merged 2011-12-06 19:05:39 +01:00
Simon Tokumine
473a2d55a5 reinstate 304 2011-12-06 17:13:56 +00:00
Simon Tokumine
6d05c83631 add readycallback to reconnect 2011-12-06 16:39:21 +00:00
Simon Tokumine
74cd176eb5 reconnect if varnish is down 2011-12-06 16:32:23 +00:00
Simon Tokumine
37cf676063 set last modified and cache control for varnish 2011-12-06 01:46:02 +00:00
javi
e7784c9a65 manage reconnections 2011-12-05 18:28:37 +01:00
javi
db55c08be1 some varnish sanity checks 2011-12-05 15:50:47 +01:00
javi santana
f4f7f6300d fixed log 2011-11-30 20:00:41 +01:00
javi santana
8474425375 added varnish invalidation code 2011-11-30 19:59:28 +01:00
javi santana
54df010694 removed old cache code 2011-11-30 19:57:26 +01:00