cartodb/Windshaft-cartodb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,232 Commits 1 Branch 311 Tags 39 MiB
5f6185dd51
Commit Graph

343 Commits

Author SHA1 Message Date
Raul Ochoa
40acf533ae Specifies name in the redis pool 2014-08-13 15:12:46 +02:00
Raul Ochoa
e7ab71c606 Merge branch 'master' into CDB-3686 2014-08-11 12:19:11 +02:00
Raul Ochoa
2ed656ca0d Upgrades windshaft (and grainstore) to be able to specify the tile 
format, see: https://github.com/mapnik/mapnik/wiki/OutputFormats
 2014-08-07 01:57:21 +02:00
Raul Ochoa
5cf79c82bb Configurable QueryTablesAPI to call directly postgresql using cartodb-psql 
or to keep using a request to the SQL API
 2014-08-06 21:48:08 +02:00
Raul Ochoa
d1373bec66 Improves SQL query for affected tables and last updated time 2014-08-04 17:48:59 +02:00
Raul Ochoa
dabcba9f5f Merge branch 'master' into CDB-3686 2014-08-04 13:11:40 +02:00
Raul Ochoa
414673b347 CDB-3657 Adds profiler as JSON to the header 2014-08-04 12:53:15 +02:00
Raul Ochoa
507a6a8979 CDB-3686 Style changes 2014-08-04 01:32:49 +02:00
Raul Ochoa
9f8d73a1df Removes duplicated file 2014-07-30 18:17:14 +02:00
Raul Ochoa
3af45e1a32 Moves calls to SQL API to its own entity. 
Groups affected tables and last updated time for affected tables into one request.
 2014-07-30 13:46:46 +02:00
Raul Ochoa
75088c89d3 Style fixes 2014-07-30 13:45:53 +02:00
Raul Ochoa
67df6a4d73 Adds support for several auth tokens 2014-07-08 10:35:45 +02:00
Raul Ochoa
f756b9d77f Removes search_path param 2014-07-04 12:18:35 +02:00
Raul Ochoa
bfdcee3772 Retrieving db public user from redis. It uses a new multiget method from cartodb-redis 2014-07-03 21:39:47 +02:00
Raul Ochoa
470aea22d9 Sets full search_path 2014-07-03 10:24:37 +02:00
javi
3806ad8843 Merge remote-tracking branch 'origin/CDB-2891-search_path' into multiuser 2014-06-27 09:10:39 +02:00
Raul Ochoa
037ce2dc12 CDB-2891 Exposes username as search_path in params 2014-06-27 00:48:48 +01:00
javi
338c0bcdbe use regclass instead table name to look for last_updated in CDB_tablemetadata 2014-06-26 15:00:55 +02:00
Raul Ochoa
4cf1ddd6fc CDB-3256 Adds response and method references to fake request object 2014-06-24 15:52:47 +02:00
Raul Ochoa
5e73b12cf5 CDB-3256 adds headers based on affected tables when creating a layergroup via HTTP GET 2014-06-24 12:16:30 +02:00
Sandro Santilli
49829f8935 Set default PostgreSQL application name to "cartodb_tiler" 2014-05-07 16:19:22 +02:00
Sandro Santilli
882ec65ba0 Use signer's map_key when contacting sql-api 
Includes testcase.
Fixes #188
 2014-04-08 09:44:49 +02:00
Sandro Santilli
9e495b42ee Do not cache non-success jsonp responses 
Closes #186
Includes testcase
 2014-03-21 13:58:20 +01:00
Sandro Santilli
60b552027b Add optional support for rollbar 
Re-targets to 1.10.0
Also installs an uncaught exception handler

Closes #150
 2014-03-12 17:21:35 +01:00
javi
5bfc360856 added serverMetadata option for layer group, close #182 CDB-1940 2014-03-06 15:19:12 +01:00
Sandro Santilli
0afc9c154b Cleanly catch exceptions from sendResponse 
Closes #178
 2014-03-04 18:04:58 +01:00
Sandro Santilli
97e00fb47d Do not send duplicated stats on template instanciation 
Closes #179
 2014-03-04 17:51:50 +01:00
Sandro Santilli
ffda103d61 Do not UNWATCH on every redis client release 
Closes #161
 2014-03-04 15:36:08 +01:00
Sandro Santilli
ecc9ea1226 Use 403 for forbidden, not 401 
Includes upgrade of windshaft to 0.19.3
Includes upgrade of redis-mpool to 0.0.4
 2014-03-04 15:32:31 +01:00
Sandro Santilli
93345a19b2 Do not log an error on GET / 
Closes #177
 2014-03-04 14:26:41 +01:00
Sandro Santilli
1741a20575 Do not cache map creation responses 
Closes #176
CDB-1908 #resolve
CDB-1901 #resolve

Includes testcase
 2014-03-04 10:46:15 +01:00
Sandro Santilli
30eb939dc7 Fix error message on missing requested signature 
We don't really distinguish between missing or non-authorizing
signature. And that's fine. See #170
 2014-03-03 18:14:17 +01:00
Sandro Santilli
40a254922a Raise 403 forbidden on missing requested signature 
Closes #170
Includes testcase
 2014-03-03 18:06:39 +01:00
Sandro Santilli
7bc5bab432 Properly prefix statsd labels for all endpoints 
CDB-1861 #resolve
Will be 100% complete with update of Windshaft to 0.19.3+
 2014-03-03 16:24:20 +01:00
Sandro Santilli
bddc65a504 Forbid instanciating templates of foreign users 
Closes #173
Includes testcase
 2014-02-28 16:05:46 +01:00
Sandro Santilli
ddd2628c19 Fix database connection settings on template instanciation 
Closes #174
Enhances testsuite to ensure test.js settings are read
 2014-02-28 15:56:31 +01:00
Sandro Santilli
f46dc90035 Forbid using map signatures of foreign users 
Closes #173
Includes testcase
 2014-02-28 13:24:38 +01:00
Sandro Santilli
55f333c0b7 Call userByReq() only once in req2params 2014-02-27 16:40:59 +01:00
Sandro Santilli
f24e4f8a0a Really skip CDB_TableMetadata lookup for sql affected by no tables 
Closes #169
 2014-02-27 15:34:09 +01:00
Sandro Santilli
36632c762e Do not query CDB_TableMetadata for queries affected by no tables 
Closes #16
 2014-02-27 12:32:34 +01:00
Sandro Santilli
f284362988 Reduce sql-api communication timeout, and allow overriding it 
Introduces new sqlapi.timeout directive, defaults to 100 ms
Includes testcase.
Closes #167
 2014-02-27 10:33:32 +01:00
Sandro Santilli
a13c1f61af Do not log an error for a legit request requiring no X-Cache-Channel 2014-02-24 17:34:00 +01:00
Sandro Santilli
5c466c51a8 Revert order of hostname components for statsd.prefix 2014-02-21 17:25:10 +01:00
Sandro Santilli
09ea924eb2 Allow using GET with sql-api for queries shorter than configured len 
Introduces new sqlapi.max_get_sql_length directive, defaults to 2048.
Closes #155
Includes testcases.
 2014-02-20 10:17:48 +01:00
Sandro Santilli
c8a042abdd Expand "addCacheChannel" stats 2014-02-19 18:10:33 +01:00
Sandro Santilli
b4fc8ec4a5 Allow using ":host" as part of statsd.prefix 
It'll be replaced with hostname.
Closes #153
 2014-02-19 15:31:12 +01:00
Sandro Santilli
6c6f3d02f6 Always generate X-Cache-Channel for token-based tile responses 
Closes #152
 2014-02-19 10:09:54 +01:00
Sandro Santilli
36a135f02b Refactor addCacheChannel using Step 2014-02-19 07:19:41 +01:00
Sandro Santilli
1c3734fde7 Make server_option a callable function, to reduce globals 
Updates acceptance test for #152 to not mess wit internals
 2014-02-19 06:45:29 +01:00
Sandro Santilli
3c09be64ce Add pending test for X-Cache-Channel on tiler restart (#152) 2014-02-18 18:33:00 +01:00
Sandro Santilli
c14378ca5d Avoid checking for table privacy when not using table maps 
See #147
 2014-02-17 18:20:18 +01:00
Sandro Santilli
26b9c8123d Set maxSocket to allow more than 5 concurrent connections to sql-api 2014-02-17 18:03:11 +01:00
Sandro Santilli
1f2e4edd35 Comments cleanup 2014-02-17 11:10:08 +01:00
Sandro Santilli
212eec2ca6 Pass profiler back to windshaft on createLayergroup 2014-02-17 08:50:12 +01:00
Sandro Santilli
935826ed1a Integrate statsd in template instanciation endpoint 
NOTE: stats are only enabled using windshaft 0.18.2+
 2014-02-15 08:23:43 +01:00
Sandro Santilli
8f3c6c3c87 Add profiler calls in template instanciation endpoint 2014-02-15 08:06:57 +01:00
Sandro Santilli
63401ca3df Use a single redis client in SignedMap.isAuthorized 2014-02-14 17:07:52 +01:00
Sandro Santilli
def474c611 Skip getting geometry type if request has no table 2014-02-14 12:26:34 +01:00
Sandro Santilli
c1b2d16119 rename tablePrivacy_getUserDBName profile label 2014-02-14 11:47:43 +01:00
Sandro Santilli
678d653ee9 Allow configuring TTL of mapConfigs via "mapConfigTTL" 2014-02-13 15:44:54 +01:00
Sandro Santilli
4a6af108b4 Fix use of maxUserTemplate configuration variable 2014-02-13 15:01:58 +01:00
Sandro Santilli
e4cd37647e Allow limiting number of allowed user templates 
Adds maxUserTemplates directive.
Closes #136
 2014-02-13 14:55:31 +01:00
Sandro Santilli
f7cef9dcd8 Fix bogus reference in SignedMaps when globals.environment is not set 2014-02-13 10:57:41 +01:00
Sandro Santilli
b673cb2a1f Add more detailed profile info about the "authorize" step 
Closes #142
 2014-02-13 10:25:28 +01:00
Sandro Santilli
e88e49001a Do not retrive user's api key if no api key was provided 
Reduces redis interaction, see #142
 2014-02-13 10:16:11 +01:00
Sandro Santilli
bf45bbea56 Do not send multiple equal commands to Varnish on connect 
Closes #135
Also accept varnish "secret" in config
 2014-02-12 16:14:27 +01:00
javi
01feeae6f4 include state configuration for windshaft fixes #139 2014-02-12 15:27:42 +01:00
Sandro Santilli
a948038ff4 Disable debug logging unless "debug" config param evaluates to true 
Closes #137
 2014-02-11 16:34:43 +01:00
Sandro Santilli
8c013ed2d1 Rename Step function in setDBConn 2014-02-11 13:42:44 +01:00
Sandro Santilli
7a749631e8 Fix profiler labels 2014-02-11 13:40:17 +01:00
Sandro Santilli
747f4803ba Include hash of template in the maptoken returned from instanciation 
Doing so basically removes the need to include the template identifier
in the surrogate keys of the responses for resources fetched via
the instance whenever template is updated. See #105
 2014-02-10 15:30:35 +01:00
Sandro Santilli
399bed34ad Do not try to replace template variables in undefined elements 
See #133
 2014-02-10 11:26:21 +01:00
Sandro Santilli
6b41fef96c Fix sendError calls to receive the full Error instance 2014-02-10 11:11:35 +01:00
Sandro Santilli
fe6e915c0d Always set database access parameters from req2params 
Fixes privileged database access from unauthorized users while
fetching torque tiles or feature attributes (unreleased feature).
Closes #132.

Includes testcase, which closes #119
 2014-02-07 18:08:41 +01:00
javi
f5e0d06e2f fixed when default value in a template attribute is a number and type = number checking fails fixed #130 2014-02-06 17:45:48 +01:00
javi
78f69d5236 template variables with spaces are not replaced correctly fixed #129 2014-02-06 17:33:26 +01:00
Sandro Santilli
b4936ffafa Do not allow creating template with auth='token' and no valid tokens 
Closes #128
Includes acceptance test for both creation and update
 2014-02-06 12:24:14 +01:00
Sandro Santilli
752e9ec655 Add checkInvalidCertificate method for SignedMap class 
Includes unit test
 2014-02-06 12:05:01 +01:00
Sandro Santilli
9018e39762 Make endpoints configurable 
Closes #127
Uses /api/v1/maps* in the production and staging example configs,
keeps /maps* for development and test (they are examples...)
 2014-02-05 15:14:47 +01:00
Sandro Santilli
a964ed5fe6 Implement Unified Map API 
Closes #126
 2014-02-04 19:04:59 +01:00
Sandro Santilli
b862904506 Be explicit about the map output srid configuration 2014-02-04 16:26:26 +01:00
javi
7197cc2d62 added stack to response in development mode 2014-02-04 14:58:21 +01:00
Sandro Santilli
978ea9cd04 Fix sqlapi request header to be "Host", not "Hostname" 
Closes (better) #117 -- automated test included
 2014-01-30 16:46:26 +01:00
Sandro Santilli
ca4f3d2025 Re-introduce sqlapi.host directive, allowing DNS lookups drop 
For backward compatibility, sqlapi.host is only used if domain
is also defined and has a different value (empty string allowed).

Closes #117
 2014-01-30 16:12:37 +01:00
Sandro Santilli
1f0faba71c Stop processing XML on renderer creation 
Not needed anymore since 1.6.1 introduced on-demand XML generation.
 2014-01-30 11:14:52 +01:00
Sandro Santilli
d32278b227 Rename template instanciation function 2014-01-29 14:30:27 +01:00
javi
ad0b2ffc8e added support for template instanciation with jsonp closes #116 2014-01-29 13:11:37 +01:00
Sandro Santilli
67e4e7e99b Set api_key to signer's when instanciating a template map 
Closes #114
 2014-01-28 12:37:41 +01:00
javi
92ca447c06 fixed #91 2014-01-28 12:05:01 +01:00
javi
e4e08db0b4 Merge branch 'master' of github.com:Vizzuality/Windshaft-cartodb 2014-01-22 19:10:37 +01:00
javi
164d952e56 support CORS in template instanciation endpoint, fixes #113 2014-01-22 19:10:09 +01:00
Sandro Santilli
5772c81590 Fix support for long (>64k chars) queries in layergroup creation 
Closes #111. Includes testcase.
 2014-01-16 17:20:30 +01:00
Sandro Santilli
f89fd98ed7 Expect malformed response objects (#109) 
Include test for sql errors on layergroup creation
Closes #109
 2014-01-15 11:53:19 +01:00
Sandro Santilli
d6fe5339cf Do not choke on headers cleanup when response headers are not set 
Raise a WARNING instead.
See #107 (github) and CDB-1438 (JIRA)
 2014-01-13 18:56:09 +01:00
Sandro Santilli
2690ef3f05 Drop cache headers from error responses. 
Closes #107 (github), #resolve CDB-1423 (JIRA)
 2014-01-13 11:20:02 +01:00
Sandro Santilli
4ee4492490 Yet another username extraction fix. Thanks again @demimismo. 
Closes #100 (yet again)
 2014-01-09 16:46:47 +01:00
Sandro Santilli
fcd17692ee Fix username extraction in another two places. Thanks @demimismo. 
Closes #100 (again)
 2014-01-09 15:36:16 +01:00
Sandro Santilli
ef3733aebe Improve error on attempt to delete missing template 2013-12-20 12:54:38 +01:00
Sandro Santilli
c7465479a2 Improve error on a signature certificate with no or broken auth 2013-12-20 10:41:27 +01:00
Sandro Santilli
50a902a90b Fix english of error message for sql-api connection problems 2013-12-18 12:59:26 +01:00
Sandro Santilli
0f90d687c7 Implement signed teplate maps 
Closes #98

Raises minimum required redis version to 2.4.0+ (Debian stable has 2.4.14)
 2013-12-17 17:39:21 +01:00
Sandro Santilli
84b7d78ea4 Add an utility authorizedByAPIKey method for reuse 2013-12-17 17:17:17 +01:00
Sandro Santilli
73a065c1cc Make sure user from domain is always computed locally 
Involved upgrade of cartodb-redis to 0.3.0
Really closes #100
 2013-12-17 17:17:17 +01:00
Sandro Santilli
1f693c6c78 Add 'user_from_host' directive to generalize username extraction 
Closes #100
Default extractor is backward compatible
 2013-12-17 17:17:17 +01:00
Sandro Santilli
e9db535dd8 Drop the idea that we can distinguish a "dbowner" from the domain 
We only recognize "users"
 2013-12-17 17:17:17 +01:00
Sandro Santilli
7b7408dab7 Revert "Drop /map_metadata API entry point" 
This reverts commit b37b07a06a1dd3cf05d60f4aa613ab5c48b90700.

This was too light of a decision...
 2013-12-17 17:17:17 +01:00
Sandro Santilli
9c897a91a9 Drop /map_metadata API entry point 
Closes #101
 2013-12-17 17:17:17 +01:00
Sandro Santilli
38342a7f5f Refactor req2params to make setting db credential easier 2013-12-17 17:17:16 +01:00
Sandro Santilli
276422f4be Set grainstore's GC run probability, for documentation purpose 
It sets it to the current grainstore default, so nothing changes.
 2013-12-17 17:17:16 +01:00
Sandro Santilli
385022de80 Revert "fixed #91" -- the fix was for an unconfirmed bug 
This reverts commit 9155724082.
See #38 for further action
 2013-12-17 17:17:16 +01:00
javi
a378fc4e68 fixed #91 2013-12-17 17:17:16 +01:00
javi
01de288c35 fixed #96 2013-12-17 17:17:15 +01:00
Sandro Santilli
cc09a8b66f Update to cartodb-redis 0.1.0 2013-12-17 17:17:15 +01:00
Sandro Santilli
a60a3adc12 CartoDB redis interaction delegated to "cartodb-redis" module 2013-12-17 17:17:14 +01:00
Sandro Santilli
baa95a62d1 Add support for reading user-specific database_password from redis 
This commits adds support for CartoDB-2.5.0 model.
Closes #89.
Change is backward compatible.
 2013-12-17 17:17:14 +01:00
Sandro Santilli
12f0826d32 Do not force ending dot in SQL-API hostname, for easier testing 2013-12-17 17:17:14 +01:00
Sandro Santilli
07cb36ebc7 Read user's database_host from redis, when available (#88) 
Still lacks a testcase
 2013-12-17 17:17:14 +01:00
Sandro Santilli
d7c82e7a51 Indent fixes 2013-12-17 17:17:14 +01:00
Sandro Santilli
bf340e684a Tweak error messages on missing redis variables, update tests 2013-12-17 17:17:14 +01:00
Luis Bosque
8d1b394df1 Add function to read database host from redis 2013-12-17 17:17:14 +01:00
Sandro Santilli
eb51d18012 Add support for specifying database connection passwords 2013-12-17 17:17:13 +01:00
Sandro Santilli
a27cf1b41c Do not let anonymous requests use authorized renderer caches 
Puts dbuser in params, for correct use by Windshaft renderer cache.
Before this fix, and after commit 1c9f63c9, the renderer cache key
did not contain the db user.
 2013-12-17 17:17:12 +01:00
Sandro Santilli
f5c24cf252 Add more profile slots 2013-12-17 17:17:11 +01:00
Sandro Santilli
dbf6bb5fca Only use sqlapi configuration "host" if "domain" is undefined 
We'll consider an empty string domain as valid (it's actually used
for testsuite).
 2013-12-17 17:17:11 +01:00
Javier Arce
d4d5272bf2 Sets the sqlapi domain. Fixes #82 2013-12-17 17:17:11 +01:00
Sandro Santilli
633e8d164b Rename sqlapi.host configuration to sqlapi.domain. Closes #79. 
Support for "host" is retained for backward compatibility.
 2013-08-21 10:11:30 +02:00
Sandro Santilli
ec97381820 Add more timing in the profile, add useProfiler config variable 
Default to useProfiler:true in staging and development
 2013-07-16 16:33:03 +02:00
Sandro Santilli
381b9a9edf Take cache_buster value, if present, as a Last-Modified timestamp 
This makes the Last-Modified header consistent across requests
using the same cache_buster (embedded in the token for multilayer
API).
 2013-07-15 13:48:06 +02:00
Sandro Santilli
76c056c7a1 Revert "Use a constant Last-Modified time with cache_policy=persist" 
This reverts commit 4b5899ff1a.

The reason is that setting Last-Modified to a remote date in the past
triggers early expiration of cache (as max-age will be reached sooner)
 2013-07-15 13:14:06 +02:00
Sandro Santilli
4b5899ff1a Use a constant Last-Modified time with cache_policy=persist 
After all if the client is asking for persistance it doesn't make
sense to set a different Last-Modified for different incoming
requests (even if we don't expect any) ....
 2013-07-15 12:09:13 +02:00
Sandro Santilli
afd4c3b460 Set Last-Modified header to allow for 304 responses 2013-07-15 12:02:54 +02:00
Sandro Santilli
65c7c5fc9c Always serve multilayer tiles and grids with persisting cache request 2013-07-08 12:13:45 +02:00
Sandro Santilli
60242c80f4 Set default layergroup time to live in redis to 2 hours 2013-07-08 11:50:19 +02:00
Sandro Santilli
632d75a7c8 specify units for rendererConfig.cache_ttl 2013-06-28 17:58:11 +02:00
Sandro Santilli
d7b1ff9a80 Set default layergroup ttl locally 2013-06-26 16:26:02 +02:00
Sandro Santilli
eb83851bb7 Fix database authentication with multi-table layergroups 2013-06-17 17:24:09 +02:00
Sandro Santilli
70f535d13a Properly report error from unsuccessful source table fetching 
Report terse error to user, verbose to log
 2013-06-11 10:28:05 +02:00
Sandro Santilli
63296a87cb Do not increment undefined mapview stat tags 2013-06-06 13:26:59 +02:00
Sandro Santilli
d977f83bd1 Change stats format for multilayer map token request 
See https://github.com/Vizzuality/Windshaft-cartodb/wiki/Redis-stats-format

Target 1.3.0
 2013-06-04 13:30:28 +02:00
Sandro Santilli
5b6919e0c6 Fix unit of measure for lastUpdated info extraction 2013-05-30 16:48:40 +02:00
javi
efa79b243c fixed lzma decoding to fix browser requirements 2013-04-24 15:10:58 +02:00
Sandro Santilli
f0fc44aac9 Fix fetching of affected tables when mapnik tokens are used 
We'll replace !bbox! with an empty box and !pixel_width! and
!pixel_height! with 1 before passing the query to CDB_QueryTable
 2013-04-23 17:29:49 +02:00
Sandro Santilli
f85ca16c62 Change LZMA expected encoding from HEX to base64, reducing its size 2013-04-19 16:16:20 +02:00
Sandro Santilli
14953e992f Multilayer API changes, target 1.2.0 
- Layers passed by index in grid fetching url
 - Interactivity only specified in layergroup config
 - Encode cache_buster as part of the token
 2013-04-15 18:51:28 +02:00
Sandro Santilli
78a6f4de1b Keep a counter of layergroup created per user. 
The counter is in redis db 5, in a field "mapviews" of an hash
"tiler:users:USERNAME". It's incremented whenever the layergroup
token for a configuration is requested.
 2013-04-12 17:28:34 +02:00
Sandro Santilli
83e6e0d457 More verbose logging for SQL api connection errors 2013-04-09 18:07:53 +02:00
Sandro Santilli
e5af3b90f4 Revert "Require interactivity param in single-layer grid fetching request" 
This reverts commit 3383c44eb7.

Fixes regression with default interactivity parameter.
Closes #74. See #69.
 2013-04-05 18:11:36 +02:00
Sandro Santilli
113b70cf98 Add support for creating layergroups via GET 2013-04-02 13:30:49 +02:00
Sandro Santilli
3383c44eb7 Require interactivity param in single-layer grid fetching request 
Closes #69
 2013-03-29 18:25:28 +01:00
Sandro Santilli
001bf97d69 Add support for LZMA compressed GET parameters 
You can now replace the whole query string with a single `lzma`
parameter having as value an hex encoded LZMA compressed version
of the whole query string as a JSON object.
 2013-03-22 18:55:59 +01:00