Merge remote-tracking branch 'origin/master' into pg-mvt-do-not-filter-columns

NEWS.md

@@ -4,7 +4,9 @@
 Released YYYY-MM-DD
 
 Announcements:
+ - Upgrades windshaft to [4.1.1](https://github.com/CartoDB/windshaft/releases/tag/4.1.1).
  - Validate aggregation input params
+ - Fix column names collisions in histograms [#828](https://github.com/CartoDB/Windshaft-cartodb/pull/828)
 
 
 ## 4.5.0

docs/anonymous_maps.md

@@ -1,6 +1,7 @@
 # Anonymous Maps
 
-Anonymous Maps allows you to instantiate a map given SQL and CartoCSS. It also allows you to add interaction capabilities using [UTF Grid.](https://github.com/mapbox/utfgrid-spec)
+Anonymous Maps allows you to instantiate a map given SQL and CartoCSS. It also allows you to add interaction capabilities using [UTF Grid.](https://github.com/mapbox/utfgrid-spec).
+Alternatively, you can get the data for the map (geometry and attributes for each layer) using vector tiles (in which case CartoCSS is not required).
 
 
 ## Instantiate
@@ -72,19 +73,19 @@ curl 'https://{username}.carto.com/api/v1/map' -H 'Content-Type: application/jso
 
 ## Map Tile Rendering
 
-Map tiles create the graphical representation of your map in a web browser. The performance rendering of map tiles is dependent on the type of geospatial data model (raster or vector) that you are using.
+Map tiles are used to create the graphic representation of your map in a web browser. Tiles can be requested either as pre-rendered *raster* tiles (images) or as *vector* map data to be rendered by the client (browser).
 
-- **Raster**: Generates map tiles based on a grid of pixels to represent your data. Each cell is a fixed size and contains values for particular map features. On the server-side, each request queries a dataset to retrieve data for each map tile. The grid size of map tiles can often lead to graphic quality issues.
+- **Raster**: If a tile is requested as a raster image format, like PNG, the map will be rendered on the server, using the CartoCSS styles defined in the layers of the map. It is necessary that all the layers of a map define CartoCSS styles in order to obtain raster tiles. Raster tiles are made up of 256x256 pixels; to avoid graphic quality issues tiles should be used unscaled to represent the zoom level (Z) for which they are requested. In order to render tiles, data will be retrieved from the database (in vector format) on the server-side.
 
-- **Vector**: Generates map tiles based on pre-defined coordinates to represent your data, similar to how basemap image tiles are rendered. On the client-side, map tiles represent real-world geometries of a map. Depending on the coordinates, vertices are used to connect the data and display points, lines, or polygons for the map tiles.
+- **Vector**: Tiles can also be requested as MVT (Mapbox Vector Tiles). In this case, only the geospatial vector data, without any styling, is returned. These tiles should be processed in the client-side to render the map.  In this case layers do not need to define CartoCSS, as any rendering and styling will be performed on the client side. The vector data of a tile represents real-world geometries by defining the vertices of points, lines or polygons in a tile-specific coordinate system.
 
 ## Retrieve resources from the layergroup
 
 When you have a layergroup, there are several resources for retrieving layergoup details such as, accessing Mapnik tiles, getting individual layers, accessing defined Attributes, and blending and layer selection.
 
-### Mapnik tiles
+### Raster tiles
 
-These raster tiles retrieve just the Mapnik layers. See [individual layers](#individual-layers) for details about how to retrieve other layers.
+These raster tiles are PNG images that represent only the Mapnik layers of a map. See [individual layers](#individual-layers) for details about how to retrieve other layers.
 
 ```bash
 https://{username}.carto.com/api/v1/map/{layergroupid}/{z}/{x}/{y}.png
@@ -92,9 +93,9 @@ https://{username}.carto.com/api/v1/map/{layergroupid}/{z}/{x}/{y}.png
 
 ### Mapbox Vector Tiles (MVT)
 
-[Mapbox Vector Tiles (MVT)](https://www.mapbox.com/vector-tiles/specification/) are map tiles that store geographic vector data on the client-side. Browser performance is fast since you can pan and zoom without having to query the server.
+[Mapbox Vector Tiles (MVT)](https://www.mapbox.com/vector-tiles/specification/) are map tiles that transfer geographic vector data to the client-side. Browser performance is fast since you can pan and zoom without having to query the server.
 
-CARTO uses a Web Graphics Library (WebGL) to process MVT files. This is useful since WebGL's are compatible with most web browsers, include support for multiple client-side mapping engines, and do not require additional information from the server; which makes it more efficient for rendering map tiles. However, you can use any implementation tool for processing MVT files.
+CARTO uses Web Graphics Library (WebGL) to process MVT files on the browser. This is useful since WebGL is compatible with most web browsers, include support for multiple client-side mapping engines, and do not require additional information from the server; which makes it more efficient for rendering map tiles. However, you can use any implementation tool for processing MVT files.
 
 The following examples describe how to fetch MVT tiles with a cURL request.
 

lib/cartodb/models/dataview/histograms/date-histogram.js

@@ -56,7 +56,7 @@ __wd_buckets AS
         ${ctx.query}
     ) __source, __wd_tz
     ${condition_str}
-    GROUP BY timestamp, __wd_tz.name
+    GROUP BY 1, __wd_tz.name
 ),`;
 }
 

lib/cartodb/models/dataview/histograms/numeric-histogram.js

@@ -28,7 +28,7 @@ const binsQueryTpl = ctx => `
             ) AS quartile
             FROM __cdb_filtered_source) _cdb_quartiles
             WHERE quartile = 1 or quartile = 3
-            GROUP BY quartile
+            GROUP BY 1
         ) __cdb_iqr
     ),
     __cdb_bins AS (
@@ -137,8 +137,8 @@ FROM
 (
     ${ctx.query}
 ) __cdb_filtered_source_query${extra_tables}
-GROUP BY bin${extra_groupby}
+GROUP BY 10${extra_groupby}
-ORDER BY bin;`;
+ORDER BY 10;`;
     }
 
     _hasOverridenBins (override) {

test/acceptance/layers-filters.js

@@ -0,0 +1,54 @@
+require('../support/test_helper');
+var TestClient = require('../support/test-client');
+
+describe('layers filters', function() {
+    const type = 'mapnik';
+    const sql = 'select * from populated_places_simple_reduced';
+    const cartocss = `#points {
+        marker-fill-opacity: 1.0;
+        marker-line-color: #FFF;
+        marker-line-width: 0.5;
+        marker-line-opacity: 1.0;
+        marker-placement: point;
+        marker-type: ellipse;
+        marker-width: 8;
+        marker-fill: red;
+        marker-allow-overlap: true;
+    }`;
+    const cartocss_version = '3.0.12';
+    const options = {
+        sql,
+        cartocss,
+        cartocss_version
+    };
+
+    const mapConfig = {
+        version: '1.6.0',
+        layers: [
+            {
+                type,
+                id: 'layerA',
+                options
+            },
+            {
+                type,
+                id: 'layerB',
+                options
+            }
+        ]
+    };
+
+    afterEach(function(done) {
+        if (this.testClient) {
+            this.testClient.drain(done);
+        }
+    });
+
+    ['layerA', 'layerB'].forEach(layer => {
+        it(`should work for individual layer ids: ${layer}`, function (done) {
+            this.testClient = new TestClient(mapConfig);
+            this.testClient.getTile(0, 0, 0, { layers: layer }, done);
+        });
+    });
+
+});

test/support/test_helper.js

@@ -75,6 +75,22 @@ function checkSurrogateKey(res, expectedKey) {
     assert.deepEqual(keys, expectedKeys);
 }
 
+var uncaughtExceptions = [];
+process.on('uncaughtException', function(err) {
+    uncaughtExceptions.push(err);
+});
+beforeEach(function() {
+    uncaughtExceptions = [];
+});
+//global afterEach to capture uncaught exceptions
+afterEach(function() {
+    assert.equal(
+        uncaughtExceptions.length,
+        0,
+        'uncaughtException:\n\n' + uncaughtExceptions.map(err => err.stack).join('\n\n'));
+});
+
+
 var redisClient;
 
 beforeEach(function() {

test/unit/cartodb/ported/tile_stats.test.js

@@ -11,7 +11,7 @@ describe('tile stats', function() {
     });
 
     afterEach(function() {
-        global.statsClient  = this.statsClient;
+        global.statsClient = this.statsClient;
     });
 
     it('finalizeGetTileOrGrid does not call statsClient when format is not supported', function() {
@@ -84,7 +84,7 @@ describe('tile stats', function() {
     });
 
     function mockStatsClientGetInstance(instance) {
-        global.statsClient = instance;
+        global.statsClient = Object.assign(global.statsClient, instance);
     }
 
 });

yarn.lock

@@ -33,8 +33,8 @@ ajv@^4.9.1:
     json-stable-stringify "^1.0.1"
 
 ajv@^5.1.0:
-  version "5.5.1"
+  version "5.5.2"
-  resolved "https://registry.yarnpkg.com/ajv/-/ajv-5.5.1.tgz#b38bb8876d9e86bee994956a04e721e88b248eb2"
+  resolved "https://registry.yarnpkg.com/ajv/-/ajv-5.5.2.tgz#73b5eeca3fab653e3d3f9422b341ad42205dc965"
   dependencies:
     co "^4.6.0"
     fast-deep-equal "^1.0.0"
@@ -1380,8 +1380,8 @@ mocha@~3.4.1:
     supports-color "3.1.2"
 
 moment@^2.10.6:
-  version "2.19.4"
+  version "2.20.1"
-  resolved "https://registry.yarnpkg.com/moment/-/moment-2.19.4.tgz#17e5e2c6ead8819c8ecfad83a0acccb312e94682"
+  resolved "https://registry.yarnpkg.com/moment/-/moment-2.20.1.tgz#d6eb1a46cbcc14a2b2f9434112c1ff8907f313fd"
 
 moment@~2.18.1:
   version "2.18.1"
@@ -1633,7 +1633,7 @@ pg-types@1.*:
     postgres-date "~1.0.0"
     postgres-interval "^1.1.0"
 
-pg@cartodb/node-postgres#6.1.6-cdb1:
+"pg@github:cartodb/node-postgres#6.1.6-cdb1":
   version "6.1.6"
   resolved "https://codeload.github.com/cartodb/node-postgres/tar.gz/3eef52dd1e655f658a4ee8ac5697688b3ecfed44"
   dependencies:
@@ -2393,8 +2393,8 @@ window-size@^0.2.0:
   resolved "https://registry.yarnpkg.com/window-size/-/window-size-0.2.0.tgz#b4315bb4214a3d7058ebeee892e13fa24d98b075"
 
 windshaft@cartodb/windshaft#pg-mvt-do-not-filter-columns:
-  version "4.1.1"
+  version "4.1.2"
-  resolved "https://codeload.github.com/cartodb/windshaft/tar.gz/04df77330ea983193345ef65601ee131bc1ce796"
+  resolved "https://codeload.github.com/cartodb/windshaft/tar.gz/3a62a4e9f5fdbcb3e462adf047f97b1c8433971c"
   dependencies:
     abaculus cartodb/abaculus#2.0.3-cdb1
     canvas cartodb/node-canvas#1.6.2-cdb2