cartodb4/cartodb-postgresql
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,059 Commits 1 Branch 92 Tags 1.5 MiB
f461faf0b6
Commit Graph

102 Commits

Author SHA1 Message Date
Javier Goizueta
5c3c0f5fc9 Fix bug in CDB_DropOverviews 
Fixes #223
 2016-04-06 18:57:52 +02:00
Javier Goizueta
06bb669d4c Add comment 
Clarify the reason why some functions are dropped at the
end of the file and not where their new definitions are.
 2016-04-05 13:22:07 +02:00
Javier Goizueta
00a3d6e650 Fix upgrades from prior versions 2016-04-05 13:10:00 +02:00
Javier Goizueta
f0ff197c56 Merge branch 'master' into 219-overviews-tolerance 
# Conflicts:
#	scripts-available/CDB_Overviews.sql
#	test/CDB_OverviewsTest_expect
 2016-04-05 13:08:22 +02:00
Javier Goizueta
57c32332e2 New function CDB_CreateOverviewsWithToleranceInPixels 
This function allows defining the previously fixed parameter grid_px.
The lim parameter used to define the reference Z level is also
correlated to this value.

Fixes #219
 2016-04-04 19:21:10 +02:00
Javier Goizueta
3c71eecbae Fix cases that require explicit schema name 
This allows using overview functions in situations where
the base tables require being qualified with the schema name.
 2016-04-04 17:45:20 +02:00
Javier Goizueta
c0262a05eb Fix CDB_Overviews for cases that require explicit schema name 
Fixes #218
 2016-04-04 17:02:59 +02:00
Javier Goizueta
4074173c05 Add support for boolean columns in overview creation 2016-03-21 12:58:29 +01:00
Javier Goizueta
5285943dbf Fix function declaration for create_from_unpackaged.sh use 
The script create_from_unpackaged.sh does not support
function declarations where the signature (name and parameter list
of the function, including parentheses) is not on a single line.
 2016-02-01 18:56:55 +01:00
Javier Goizueta
00bd302f01 Avoid creating GridCluster overviews for non-point datasets 2016-01-29 16:19:35 +01:00
Javier Goizueta
93d4a6ead0 Restrict SECURITY DEFINER to overview registration 
And check it is applied only overview tables with valid names
 2016-01-28 17:04:06 +01:00
Javier Goizueta
66387c2d44 Make CDB_CreateOverviews a SECURITY DEFINER function 
So that it can change the permissions of overview tables
(by updating pg_class) even when executed by non-priviledeged users.
 2016-01-28 16:32:40 +01:00
Javier Goizueta
b8d50204dd Avoid aggregation which causes out-of-memory crashes in PostgreSQL 
The use of multiple string_agg functions, even if applied to groups
of one single record causes out of memory crashes in PG 9.3.4 for
some (large) tables.
 2016-01-27 15:24:04 +01:00
Javier Goizueta
474de01757 Change the overview table naming scheme 
The scheme is changed from table_ovN to _vovw_N_table for
lower collision probability.
Also future naming changes will be easier by using the functions
_CDB_OverviewTableDiscriminator, _CDB_OverviewTableName,
_CDB_IsOverviewTableOf, etc.
 2016-01-26 13:20:28 +01:00
Javier Goizueta
1ebaeb76ac Overload CDB_Overviews to accept either a single table or an array 
The result of CDB_Overviews has been expandend with a base_table column
 2016-01-26 10:39:26 +01:00
Javier Goizueta
7033a8d9ac Fixes for table names that require quotes or which include a schema name 
Some cases left unsolved, with FIXME comments
 2016-01-13 18:49:27 +01:00
Javier Goizueta
46bc774d38 Fix CDB_Overviews for the case that the table name requires quoting 2016-01-13 18:24:06 +01:00
Javier Goizueta
a9e2d19918 Document CDB_DropOverviews 2016-01-12 16:20:11 +01:00
Javier Goizueta
5b47c51221 Add function to drop overviews 2016-01-12 15:35:50 +01:00
Javier Goizueta
ccdf8de59e Enhance documentation, update comments 2016-01-12 12:08:41 +01:00
Javier Goizueta
02f386be33 Regard the GridCluster reduction strategy as private 
The *strategy* parameteriation will be considered an
implementation detail for the time being.
 2016-01-12 11:59:42 +01:00
Javier Goizueta
0a066e0126 Change the default overview reduction strategy to GridCluster 2016-01-11 15:35:04 +01:00
Javier Goizueta
913640e2dc Preserve the column ordering of the base table in overviews 2016-01-11 15:34:10 +01:00
Javier Goizueta
fd7a8cff71 Change regular expression for consistency 2016-01-11 15:27:05 +01:00
Javier Goizueta
6ea63af974 Add function to obtain existing overviews of a table 2016-01-11 14:50:10 +01:00
Javier Goizueta
496f079b1c Copy dataset privileges to overview tables 2016-01-04 18:06:56 +01:00
Javier Goizueta
4580c9cd5c Change the cartodb_ids of aggregated overviews 
Instead of arbitrary ids, the id of one of the aggregated records
is used, so that if it is used in the UI to query point
the information of one of the records grouped at the point will be
retrieved.
 2016-01-04 10:33:39 +01:00
Javier Goizueta
08828b8b7d Set owner of overlays to the owner of the base table 2015-12-29 15:22:12 +01:00
Javier Goizueta
cecba655eb Preserve aggregated attributes of singleton groups 2015-12-29 14:56:47 +01:00
Javier Goizueta
b34a752172 Generate overviews for all Z levels 
Skipping levels produces visually inferior results
 2015-12-29 14:56:04 +01:00
Javier Goizueta
8ba9e74c4a Order columns of aggregated overlays as in the base table 2015-12-29 13:38:07 +01:00
Javier Goizueta
6a6a5bc96a Fix Sampling reduction to avoid RandomTids problems 
The fixed cases will not be common but do occur in tests.
This is an interim fix which should be reverted if CDB_randomTids changes.
 2015-12-28 19:41:14 +01:00
Javier Goizueta
2ff686de27 Clean up: remove spurious comment 2015-12-28 19:30:20 +01:00
Javier Goizueta
07326626b7 More efficient sampling strategy 
It is also renamed properly.
 2015-12-28 19:28:20 +01:00
Javier Goizueta
4afc427008 Rename Ref. Z strategy function 
It's not that *dummy* anymore, so choose a more descriptive name
 2015-12-28 17:47:57 +01:00
Javier Goizueta
a8a2c04d71 Remove invalid comment 2015-12-23 14:20:57 +01:00
Javier Goizueta
a5bca7d715 Add function for post-processing new overview tables 2015-12-23 14:17:50 +01:00
Javier Goizueta
552206464e Make strategy function public 2015-12-23 12:50:04 +01:00
Javier Goizueta
2af0b9a57f Add function comments 2015-12-23 12:42:40 +01:00
Javier Goizueta
1b5de84c9a Add missing attribute-aggregationto the point reduction strategy 2015-12-23 12:17:37 +01:00
Javier Goizueta
5a78ee2896 Optimize the gridded clustering strategy 
The internal grid_px parameter is adjusted for best results with default symbol size
 2015-12-22 17:59:49 +01:00
Javier Goizueta
c8a1ef6f68 Slight optimization of gridded clustering 2015-12-22 15:25:21 +01:00
Rafa de la Torre
564ab75d2d Use _cdb_estimated_extent instead of ST_Extent 
With a 3.8M points table, this was a ~30% gain in my local env.
 2015-12-21 18:41:50 +01:00
Rafa de la Torre
5010109c7d Add _cdb_estimated_extent to get the extent from stats 2015-12-21 18:41:45 +01:00
Rafa de la Torre
e60f73a31b A bit of internal documentation 2015-12-21 13:16:57 +01:00
Javier Goizueta
415a09392f Gridded clustering aggregation strategy for overviews 2015-12-16 17:45:36 +01:00
Javier Goizueta
e5cc9ef0bd Fix: typo 2015-12-16 17:26:17 +01:00
Javier Goizueta
f7857945c1 Change feature density computation algorithm 
Now parameterize by the number of levels to traverse and
start with the level that covers the extent of the table
with at least N*N tiles (N=4).
 2015-12-16 16:39:38 +01:00
Javier Goizueta
554464e43e Use tile-recursive computation of feature density 
This is a more adaptive way of estimating the feature density to
determine the base Z level.
Applying technique from http://javisantana.com/2014/10/22/traversing-quadtree.html
 2015-12-16 12:14:37 +01:00
Javier Goizueta
d7c8f3d7e8 Fix: overlay generation was not using the proper scale 2015-12-15 19:13:39 +01:00
Javier Goizueta
4c85d7f3ad Compute the reference Z level for a table 2015-12-15 19:12:48 +01:00
Javier Goizueta
d0e66910a0 Sketch for new Overview-creation functionality 2015-12-15 17:36:27 +01:00