cartodb/Windshaft-cartodb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,401 Commits 1 Branch 311 Tags 39 MiB
2ac050501b
Commit Graph

2386 Commits

Author SHA1 Message Date
Sandro Santilli
38342a7f5f Refactor req2params to make setting db credential easier 2013-12-17 17:17:16 +01:00
Sandro Santilli
276422f4be Set grainstore's GC run probability, for documentation purpose 
It sets it to the current grainstore default, so nothing changes.
 2013-12-17 17:17:16 +01:00
Sandro Santilli
385022de80 Revert "fixed #91" -- the fix was for an unconfirmed bug 
This reverts commit 9155724082.
See #38 for further action
 2013-12-17 17:17:16 +01:00
javi
a378fc4e68 fixed #91 2013-12-17 17:17:16 +01:00
javi
01de288c35 fixed #96 2013-12-17 17:17:15 +01:00
Sandro Santilli
cc09a8b66f Update to cartodb-redis 0.1.0 2013-12-17 17:17:15 +01:00
Sandro Santilli
a60a3adc12 CartoDB redis interaction delegated to "cartodb-redis" module 2013-12-17 17:17:14 +01:00
Sandro Santilli
baa95a62d1 Add support for reading user-specific database_password from redis 
This commits adds support for CartoDB-2.5.0 model.
Closes #89.
Change is backward compatible.
 2013-12-17 17:17:14 +01:00
Sandro Santilli
12f0826d32 Do not force ending dot in SQL-API hostname, for easier testing 2013-12-17 17:17:14 +01:00
Sandro Santilli
07cb36ebc7 Read user's database_host from redis, when available (#88) 
Still lacks a testcase
 2013-12-17 17:17:14 +01:00
Sandro Santilli
d7c82e7a51 Indent fixes 2013-12-17 17:17:14 +01:00
Sandro Santilli
bf340e684a Tweak error messages on missing redis variables, update tests 2013-12-17 17:17:14 +01:00
Luis Bosque
8d1b394df1 Add function to read database host from redis 2013-12-17 17:17:14 +01:00
Sandro Santilli
eb51d18012 Add support for specifying database connection passwords 2013-12-17 17:17:13 +01:00
Sandro Santilli
a27cf1b41c Do not let anonymous requests use authorized renderer caches 
Puts dbuser in params, for correct use by Windshaft renderer cache.
Before this fix, and after commit 1c9f63c9, the renderer cache key
did not contain the db user.
 2013-12-17 17:17:12 +01:00
Sandro Santilli
f5c24cf252 Add more profile slots 2013-12-17 17:17:11 +01:00
Sandro Santilli
dbf6bb5fca Only use sqlapi configuration "host" if "domain" is undefined 
We'll consider an empty string domain as valid (it's actually used
for testsuite).
 2013-12-17 17:17:11 +01:00
Javier Arce
d4d5272bf2 Sets the sqlapi domain. Fixes #82 2013-12-17 17:17:11 +01:00
Sandro Santilli
633e8d164b Rename sqlapi.host configuration to sqlapi.domain. Closes #79. 
Support for "host" is retained for backward compatibility.
 2013-08-21 10:11:30 +02:00
Sandro Santilli
ec97381820 Add more timing in the profile, add useProfiler config variable 
Default to useProfiler:true in staging and development
 2013-07-16 16:33:03 +02:00
Sandro Santilli
381b9a9edf Take cache_buster value, if present, as a Last-Modified timestamp 
This makes the Last-Modified header consistent across requests
using the same cache_buster (embedded in the token for multilayer
API).
 2013-07-15 13:48:06 +02:00
Sandro Santilli
76c056c7a1 Revert "Use a constant Last-Modified time with cache_policy=persist" 
This reverts commit 4b5899ff1a.

The reason is that setting Last-Modified to a remote date in the past
triggers early expiration of cache (as max-age will be reached sooner)
 2013-07-15 13:14:06 +02:00
Sandro Santilli
4b5899ff1a Use a constant Last-Modified time with cache_policy=persist 
After all if the client is asking for persistance it doesn't make
sense to set a different Last-Modified for different incoming
requests (even if we don't expect any) ....
 2013-07-15 12:09:13 +02:00
Sandro Santilli
afd4c3b460 Set Last-Modified header to allow for 304 responses 2013-07-15 12:02:54 +02:00
Sandro Santilli
65c7c5fc9c Always serve multilayer tiles and grids with persisting cache request 2013-07-08 12:13:45 +02:00
Sandro Santilli
60242c80f4 Set default layergroup time to live in redis to 2 hours 2013-07-08 11:50:19 +02:00
Sandro Santilli
632d75a7c8 specify units for rendererConfig.cache_ttl 2013-06-28 17:58:11 +02:00
Sandro Santilli
d7b1ff9a80 Set default layergroup ttl locally 2013-06-26 16:26:02 +02:00
Sandro Santilli
eb83851bb7 Fix database authentication with multi-table layergroups 2013-06-17 17:24:09 +02:00
Sandro Santilli
70f535d13a Properly report error from unsuccessful source table fetching 
Report terse error to user, verbose to log
 2013-06-11 10:28:05 +02:00
Sandro Santilli
63296a87cb Do not increment undefined mapview stat tags 2013-06-06 13:26:59 +02:00
Sandro Santilli
d977f83bd1 Change stats format for multilayer map token request 
See https://github.com/Vizzuality/Windshaft-cartodb/wiki/Redis-stats-format

Target 1.3.0
 2013-06-04 13:30:28 +02:00
Sandro Santilli
5b6919e0c6 Fix unit of measure for lastUpdated info extraction 2013-05-30 16:48:40 +02:00
javi
efa79b243c fixed lzma decoding to fix browser requirements 2013-04-24 15:10:58 +02:00
Sandro Santilli
f0fc44aac9 Fix fetching of affected tables when mapnik tokens are used 
We'll replace !bbox! with an empty box and !pixel_width! and
!pixel_height! with 1 before passing the query to CDB_QueryTable
 2013-04-23 17:29:49 +02:00
Sandro Santilli
f85ca16c62 Change LZMA expected encoding from HEX to base64, reducing its size 2013-04-19 16:16:20 +02:00
Sandro Santilli
14953e992f Multilayer API changes, target 1.2.0 
- Layers passed by index in grid fetching url
 - Interactivity only specified in layergroup config
 - Encode cache_buster as part of the token
 2013-04-15 18:51:28 +02:00
Sandro Santilli
78a6f4de1b Keep a counter of layergroup created per user. 
The counter is in redis db 5, in a field "mapviews" of an hash
"tiler:users:USERNAME". It's incremented whenever the layergroup
token for a configuration is requested.
 2013-04-12 17:28:34 +02:00
Sandro Santilli
83e6e0d457 More verbose logging for SQL api connection errors 2013-04-09 18:07:53 +02:00
Sandro Santilli
e5af3b90f4 Revert "Require interactivity param in single-layer grid fetching request" 
This reverts commit 3383c44eb7.

Fixes regression with default interactivity parameter.
Closes #74. See #69.
 2013-04-05 18:11:36 +02:00
Sandro Santilli
113b70cf98 Add support for creating layergroups via GET 2013-04-02 13:30:49 +02:00
Sandro Santilli
3383c44eb7 Require interactivity param in single-layer grid fetching request 
Closes #69
 2013-03-29 18:25:28 +01:00
Sandro Santilli
001bf97d69 Add support for LZMA compressed GET parameters 
You can now replace the whole query string with a single `lzma`
parameter having as value an hex encoded LZMA compressed version
of the whole query string as a JSON object.
 2013-03-22 18:55:59 +01:00
javi
e53122de7e fixed last_update in laytergroup response 2013-03-21 11:39:55 +01:00
Sandro Santilli
3ff3dc2c97 Cleanup, handle error in req2param on flushCache 2013-03-15 19:25:13 +01:00
Sandro Santilli
4605bd1e1d Add last_modified field to POST layergroup response (#72) 
Includes testcases
 2013-03-13 18:41:37 +01:00
Sandro Santilli
dfc4a02398 Fix X-Cache-Channel for multilayer (by token) responses 
Required upgrading Windshaft to 0.9.2
Includes testcases
 2013-03-13 16:45:15 +01:00
Sandro Santilli
402fc90e63 Absence of X-Cache-Channel will be enough for Varnish to skip caching 
Do not override Cache-Control in this case, which means let the
clients or geographical proxies cache the response with usual TTL.
 2013-03-13 12:01:35 +01:00
Sandro Santilli
e8cbc666e2 Handle SQL API errors by logging them and requesting NO cache 
SQL api is used to determine the list of source tables affected
by a query. Before this commit, the X-Cache-Channel header set
on sql api error was an arbitrary 'table' string, now the header
is omitted, the error logged and Cache-Control and Pragma headers
are sent as an attempt to request no caching.

The code includes test for this mechanism.
 2013-03-13 10:39:00 +01:00
Sandro Santilli
f6d50fafb1 Expose renderer settings in the environment config files 
These are: metatile, bufferSize and cache_ttl
 2013-02-25 17:05:59 +01:00
Sandro Santilli
9dcf6a1acf Set 'base_url_notable' config for Windshaft-0.9 (multilayer) 2013-02-12 18:53:41 +01:00
Sandro Santilli
a79b999e7a Do not try to send commands to an unoconnected redis client 
This changes "Cannot read property 'HGET' of null" messages into
"Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED".
 2013-02-11 15:05:23 +01:00
Sandro Santilli
6b71cde56e Do not throw an Error embedding another Error 
Should fix #68, but doesn't come with an automated test
 2013-02-08 12:27:49 +01:00
Sandro Santilli
cb57dfb27d Fix async throws in getGeometryType, getInfoWindow and getMapMetadata 2013-02-08 12:14:53 +01:00
Sandro Santilli
8d4f033a56 Revert "getDatabase: properly handle redis connection failures" 
This reverts commit dd19d74149.

The code was already correct
 2013-01-28 17:39:50 +01:00
Sandro Santilli
dd19d74149 getDatabase: properly handle redis connection failures 2013-01-28 17:30:58 +01:00
Sandro Santilli
ac49abe750 Do not leak redis client connections on redis command error 2013-01-28 17:13:49 +01:00
Sandro Santilli
b130b67f24 Check redis connection at pool creation time 2013-01-28 17:12:21 +01:00
Sandro Santilli
19436a8b14 Let "style_convert" pass by, add tests for GET and POST with it 2012-11-14 15:28:58 +01:00
Sandro Santilli
52303e7821 Fix use of "style_version" with GET (inline styles) 
It took a lot of time to produce a testcase for this as the test
config was setting srid to 4326 but not changing geom column name
thus all tiles fetched by tests returned blank (ouch!)
 2012-11-06 12:45:04 +01:00
Sandro Santilli
2bc09a61cf Add support for cache_policy=persistent 
When cache_policy=persistent is given the response will contain
a Cache-Control header requesting for 1 year lifetime caching
 2012-10-24 09:40:05 +02:00
Sandro Santilli
d9e6aeb254 Fix crash on unknown user. Closes #55. 2012-10-22 15:30:16 +02:00
Sandro Santilli
ab8cb5bbb3 Add Windshaft-cartodb version to the /version route 2012-10-15 17:03:57 +02:00
Sandro Santilli
20dca2e8f8 Use windshaft-0.6.2 sendError function to send non-200 responses 
Ensures all errors are logged
 2012-10-11 16:48:41 +02:00
Sandro Santilli
6a1933bed9 Print a warning when configured mapnik version doesn't match installed 2012-10-09 11:45:57 +02:00
Sandro Santilli
961269fa1f Autodetect target mapnik version and let config override it 
Closes #40
 2012-10-08 17:45:03 +02:00
Sandro Santilli
8a759babf0 Add tests for getting metadata (#183) 
... and fix forbidden metadata response
 2012-10-05 17:08:24 +02:00
Sandro Santilli
7196c8c285 Only invalidate cache on del style when caching is enabled 2012-10-05 16:55:58 +02:00
Sandro Santilli
dc9286b610 Accept "api_key" as "map_key", in both query_string and POST body 
Closes #38
 2012-10-05 16:17:49 +02:00
Sandro Santilli
1ed65544e5 Send detailed error when user metadata are missing from redis 
Include tip on how to restore the redis db from cartodb.
 2012-10-05 16:05:32 +02:00
Sandro Santilli
a201888fde Make logging format configurable (closes #4) 
NOTE: the default format for the "test" environment is without
ansi colors, to be easier on remote terminal sessions
 2012-10-05 15:52:51 +02:00
Sandro Santilli
bc506784ca Add an X-Cache-Channel header to all GET requests. Closes #53. 2012-09-25 09:27:03 +02:00
Sandro Santilli
dd34fc507a Automated localization of external resources referenced in carto 2012-09-19 18:52:13 +02:00
Sandro Santilli
dcbe051654 Return 401 status for unauthorized requests (see #48) 2012-09-05 20:16:55 +02:00
Sandro Santilli
b469d5eabf Rename authenticated postgresql user configuration setting 
See issue #49
 2012-09-05 15:49:10 +02:00
Sandro Santilli
5ca84d9ffc Always replace the database user parameter in the XML style 
This provides for an automatic migration of styles to any new
authentication user.
 2012-09-05 15:41:22 +02:00
Sandro Santilli
87b0e07815 Expose Varnish cache TTL in configuration. Closes #46 2012-09-03 18:57:35 +02:00
Sandro Santilli
b9e29938e2 Compile XML processing regular expression only once. 
Requests per second from ~95 to ~98 with a "standard" test.
 2012-09-03 15:30:31 +02:00
Sandro Santilli
1c9f63c901 Add XML processor to change database user when authenticated 2012-09-03 14:54:23 +02:00
Sandro Santilli
9ba80acab7 Properly bubble getInfowindow errors up to response 2012-08-14 20:01:05 +02:00
Sandro Santilli
b30c8d9a98 Send jsQuery ready error message on GET /infowindow errors 2012-08-14 20:00:26 +02:00
Sandro Santilli
ca47fbd10b Fix unauthenticated access to table styles. Closes #43. 
Re-introduces use of the redis "privacy" hash key as an additional
security measure for requests that do not involve PostgreSQL access.

Accessing private table styles is tested with this commit.
Accessing private table metadata or infowindow is _not_ tested,
but should also be fixed now.
 2012-08-14 19:26:40 +02:00
Sandro Santilli
fb3f3a312e Really fix setting or deleting styles from unauth. request 
Closes #44
 2012-08-14 16:15:41 +02:00
Sandro Santilli
b5348db94c Add CORS headers to all requests 
In addition to the ones served by the base Windshaft server this commit
adds CORS headers to the GET /infowindow, GET /map_metadata and
DEL /flush_cache requests. Closes issue #42.
 2012-08-14 14:50:18 +02:00
Sandro Santilli
282ac94e29 Prevent unauthenticated requests from changing map styles 2012-08-07 17:10:15 +02:00
Sandro Santilli
6a92fd3170 Propagate style changes to caches for unauthenticated requests 
Closes #41, does it implementing the new afterStyleChange and
afterStyleDelete callbacks in Windshaft 0.4.10.

Adds automated testcases for the bug.
 2012-08-07 16:11:49 +02:00
Sandro Santilli
651c45bc16 Use the correct redis variable for checking map key (closes #39) 
Includes testcase
 2012-08-02 11:15:01 +02:00
Sandro Santilli
de275bfc50 Delegate user permission to PostgreSQL (closes #18) 
If the request is authenticated (with map_key) then we log as the
database owner, otherwise we log as the default user.
The default user is now "publicuser" by default.

Raises dependency on Windshaft to 0.4.9+, to get the grainstore
version allowing override of database username.

Add test for req2params function, particularly authentication,
Add test for authenticated / unauthenticated access
 2012-07-18 11:09:17 +02:00
Sandro Santilli
15162e8c38 Use global environment settings in carto_data (closes #27) 2012-07-09 19:33:17 +02:00
Simon Tokumine
98e5bab8e4 commenting and removal of bogus config. also disable cache in test environment 2012-06-06 15:24:44 +01:00
Simon Tokumine
a382b9f601 check for nulls. 2012-05-08 10:52:58 +01:00
Simon Tokumine
ac7593a743 check for nulls. out 2012-05-08 10:51:51 +01:00
Simon Tokumine
13b66c4e38 check for nulls 2012-05-08 10:38:01 +01:00
Simon Tokumine
4078098c3f enable cache clearing at table level granularity 2012-05-02 19:32:54 +01:00
Simon Tokumine
d1226992c3 clean up environment settings 2012-05-01 19:00:14 +01:00
Simon Tokumine
dcea209ca5 update log format 2011-12-14 00:56:10 +00:00
Simon Tokumine
ef15f4b48b migrated to node-varnish, fixed tests, refactor 2011-12-12 18:02:10 +00:00
Simon Tokumine
e810747a21 Merge branch 'master' of github.com:Vizzuality/Windshaft-cartodb 2011-12-09 00:32:41 +00:00
Simon Tokumine
78d415569d pin to mapnik 2.0, enable passing style via url 2011-12-09 00:32:32 +00:00
javi
b53bcceb6d lovely fixes 2011-12-07 13:06:23 +01:00
javi
5084e69aff added varnish tests 2011-12-07 11:54:24 +01:00
Simon Tokumine
5fbd3a376e cache age of 60 mins 2011-12-06 21:46:17 +00:00
Simon Tokumine
1bb5824cf7 spelling: 2011-12-06 21:21:19 +00:00
javi
2f9987beb8 Merged 2011-12-06 19:05:39 +01:00
Simon Tokumine
473a2d55a5 reinstate 304 2011-12-06 17:13:56 +00:00
Simon Tokumine
6d05c83631 add readycallback to reconnect 2011-12-06 16:39:21 +00:00
Simon Tokumine
74cd176eb5 reconnect if varnish is down 2011-12-06 16:32:23 +00:00
Simon Tokumine
37cf676063 set last modified and cache control for varnish 2011-12-06 01:46:02 +00:00
javi
e7784c9a65 manage reconnections 2011-12-05 18:28:37 +01:00
javi
db55c08be1 some varnish sanity checks 2011-12-05 15:50:47 +01:00
javi santana
f4f7f6300d fixed log 2011-11-30 20:00:41 +01:00
javi santana
8474425375 added varnish invalidation code 2011-11-30 19:59:28 +01:00
javi santana
54df010694 removed old cache code 2011-11-30 19:57:26 +01:00
javi
cb69faffc0 added X-Cache-Channel header so the tile cache could be invalidated using that key 
for the moment we're using the database name as 
cache key, it is not the best solution because all
the tiles for that database will be invalidated 
when a change is done in any table. Doing this we
avoid getting wrong data when user gets tiles which
come from a query with joins.
 2011-11-29 21:19:10 +01:00
Simon Tokumine
5cdadd863e mapkey checks a set, not a singular 2011-11-22 03:46:59 +00:00
Simon Tokumine
4347589826 update status codes 2011-11-21 22:29:35 +00:00
Simon Tokumine
431d4f867b explicitly set http status for 304 2011-10-27 15:11:51 +01:00
Simon Tokumine
7b40749d96 osx is not case sensitive, linux is 2011-10-27 13:15:02 +01:00
javi
102e17363f added ttl_settings 2011-10-21 17:42:57 +02:00
javi
e5da826b94 fixed lru cache 2011-10-21 17:41:41 +02:00
javi
9c5718a427 ttl options and tests 2011-10-21 16:52:07 +02:00
javi
f0b8c29b03 added ttl 2011-10-21 16:33:35 +02:00
javi
87b04ce80a added last-modified header to tiler 2011-10-20 18:52:11 +02:00
javi
91d3bc66c8 trying to avoid precision problems with small time measures 2011-10-20 16:46:39 +02:00
javi santana
b4a0732120 added more cache stats 2011-10-13 16:20:29 +02:00
javi santana
ab808b1dcb set cache settings in enviorments 2011-10-13 15:22:54 +02:00
javi santana
c1ba45cf5a included LRU cache with redis invalidation 2011-10-13 13:17:00 +02:00
Simon Tokumine
de8508ed57 add map metadata enpoind and move all responsiblity for infowindow and metadata to here 2011-10-07 15:38:28 +01:00
Simon Tokumine
2e1b36a1a4 configure tileuser and mapkey security 2011-09-21 16:33:25 -04:00
Simon Tokumine
306eb7aaf2 enable cors and bump version used 2011-09-19 21:27:23 -04:00
Simon Tokumine
0b5a799ac8 remove defaults and bump windshaft version 2011-09-14 00:17:02 -06:00
Simon Tokumine
4dd105cfb7 add multipoly styles 2011-09-13 23:58:08 -06:00
Simon Tokumine
c509cc0b6b add multipoly styles 2011-09-13 23:52:59 -06:00
Simon Tokumine
3d3ea9902e updated production settings 2011-09-13 21:51:51 -06:00
Simon Tokumine
5db227c26f update readme and make interactivity settable 2011-09-05 00:17:39 +01:00
Simon Tokumine
431fb56ad2 windshaft for cartodb 2011-09-05 00:00:41 +01:00