Javier Goizueta
7bdee5c13e
Avoid double-quoting
...
Since dst_schema is a REGNAMESPACE, it is automatically quoted when casted to TEXT
2019-07-03 18:34:54 +02:00
Javier Goizueta
2e1fe2933c
Fix whitespace
2019-07-03 16:56:29 +02:00
Rafa de la Torre
446f4113d9
Return NULL instead of NOW()
...
The absence of foreign CDB_TableMetadata actually means that we cannot
really tell when a remote table was modified.
Therefore we're using NULL with the meaning of "I don't know when it
was last modified".
To be taken in other caching layers, to adjust headers accordingly.
2019-07-03 16:19:46 +02:00
Javier Goizueta
5963c67376
Order the columns of a cartodbfied table consistently
...
The final order of the columns of a cartodbfied table wasn't uniquely specified, so could vary across PG versions.
This was a problem in particular for having deterministic test results.
2019-07-03 16:16:37 +02:00
Javier Goizueta
f5f18ca57c
Fix the order of the columns
2019-07-02 18:14:18 +02:00
Javier Goizueta
d8c840d126
Quote table and column names when necessary
...
Also use type NAME when appropriate. Note that quoted column names are not NAMES (may be longer).
2019-07-02 18:14:04 +02:00
Javier Goizueta
65483743b4
Remove unused vars
2019-07-02 17:44:06 +02:00
Javier Goizueta
470bae6268
Merge branch 'master' into table-syncer-notmps
...
# Conflicts:
# scripts-available/CDB_SyncTable.sql
2019-07-02 12:21:49 +02:00
Javier Goizueta
69cc56b589
Remove schema from public function examples
2019-07-01 15:13:48 +02:00
Javier Goizueta
1028b24333
Fix cartodb schema references
2019-07-01 14:48:42 +02:00
Rafa de la Torre
c06d24aa19
Make CDB_Get_Foreign_Updated_At robust to missing CDB_TableMetadata
...
This may happen with non-carto DB's, when checking the updated_at
times and not finding the corresponding remote.cdb_tablemetadata
imported from the foreign non-carto DB.
Instead of failing, return a NOW() timestampt, so that caching logic
just assumes there may have been changes.
This makes it work today, and leaves open the possibility of adding
the required carto metadata for homogeneous caching in the future.
2019-06-28 16:25:52 +02:00
Javier Goizueta
4f4df2de8d
Merge branch 'table-syncer' into table-syncer-notmps
2019-06-28 15:28:06 +02:00
Javier Goizueta
b5f36902c5
Merge branch 'master' into table-syncer
2019-06-28 13:50:46 +02:00
Javier Goizueta
0bcbf6708a
Remove usage of temporary tables
2019-06-28 13:49:15 +02:00
Gonzalo Riestra
e6ecde4346
add CREATE/ALTER/DROP FOREIGN TABLE to ddl commands
2019-06-20 11:20:48 +02:00
Gonzalo Riestra
2d42e6197a
use bigint to store values from txid_current()
2019-06-20 11:20:03 +02:00
Gonzalo Riestra
5605fdd9b2
fix event trigger
2019-06-19 15:27:28 +02:00
Raul Marin
a43abb37e0
Add missing qualifications
2019-06-03 15:22:52 +02:00
Raul Marin
c45ef6c540
Fully qualify function calls
2019-05-31 15:37:15 +02:00
Rafa de la Torre
42dc03d77b
Qualify functions with extension schema
...
This avoids some issues with search_path and scripting black magic.
2019-05-30 16:56:12 +02:00
Rafa de la Torre
9254723719
Make format expression more readable
...
Make an EXECUTE format('', param1, ..., paramN) more readable by
adding the index of the argument to print (`position`) in the format
specifier.
2019-05-29 13:13:30 +02:00
Rafa de la Torre
26ad966ab6
Add some timing info
2019-05-28 16:40:01 +02:00
Rafa de la Torre
a2723a3c90
Exclude certain columns from sync if instructed to do so
...
For the Geocoding (and in general for LDS use cases) it may come in
handy to exclude geometry columns from the list of stuff to
syncrhonize. Otherwise they may be lost, overwritten with NULL values.
2019-05-28 16:11:56 +02:00
Rafa de la Torre
2f8ea7e4ea
Avoid tables name clashing when executing within same transaction
...
Generate more unique temp table names when the CDB_SyncTable function
is executed multiple times within the same transaction.
When executed in isolation, there will be always an implicit
surrounding transaction.
But when executed several times within the same transaction it can
give an `ERROR: relation "src_sync_718794" already exists`.
E.g:
```
BEGIN;
SELECT cartodb.CDB_SyncTable('source1', 'public', 'dest1');
SELECT cartodb.CDB_SyncTable('source12, 'public', 'dest2');
COMMIT;
```
2019-05-28 15:39:02 +02:00
Rafa de la Torre
7606585672
Perf optimization: use EXCEPT instead of NOT IN
...
With javitonino's help, greatly reduce the processing time by using
EXCEPT instead of NOT IN, which causes it to use a `HashSetOp Except`
plan on the subqueries rather than a `Seq Scan` on `Materialize`'d
subtables.
2019-05-27 15:20:06 +02:00
Rafa de la Torre
81d0f338cf
Create HASH indices on the temp tables
2019-05-27 15:18:13 +02:00
Rafa de la Torre
951f257654
Simplify code by using ON COMMIT DROP
...
Simplify code by relying on automatic removal of temp tables with ON
COMMIT DROP, so that we avoid the messy EXCEPTION management.
2019-05-27 12:02:12 +02:00
Rafa de la Torre
f461faf0b6
Simplify udpate: use * instead of column list
2019-05-27 11:56:41 +02:00
Rafa de la Torre
a8d57abda6
Update changed rows
2019-05-27 11:54:38 +02:00
Rafa de la Torre
982ddfdeff
Helper function to generate UPDATE SET clause
2019-05-27 11:31:07 +02:00
Rafa de la Torre
da4331ac78
WIP: A function to sync tables
...
It assumes there's a cartodb_id column in both source and target. It
does not perform unnecessary actions. It respects augmented columns in
target table, if they exist. It is meant to be efficient.
2019-05-24 18:38:28 +02:00
Gonzalo Riestra
85d6164956
add log when there is no tig config
2019-03-14 11:39:10 +01:00
Gonzalo Riestra
3faa389860
improve logs
2019-03-13 18:16:08 +01:00
Gonzalo Riestra
bc5d532735
remove default TIS config
2019-03-13 18:12:48 +01:00
Gonzalo Riestra
45fed9cf1b
more explicit schemas
2019-03-13 10:29:50 +01:00
Gonzalo Riestra
e19489144c
simplify queries
2019-03-13 09:08:13 +01:00
Gonzalo Riestra
83707297de
add explicit schema for all functions
2019-03-12 18:39:48 +01:00
Gonzalo Riestra
65d51fd8bd
move cdb_ddl_execution table creation/drop to functions
2019-03-11 14:02:45 +01:00
Gonzalo Riestra
9c6294d95b
move trigger drop/creation inside functions
2019-03-11 11:28:45 +01:00
Gonzalo Riestra
c7bba14e9a
simplify code
2019-03-11 09:20:05 +01:00
Gonzalo Riestra
667f896cfb
add again views to the trigger
2019-03-08 13:21:11 +01:00
Gonzalo Riestra
a794fb3d31
read tis config from cdb_conf
2019-03-08 12:03:55 +01:00
Gonzalo Riestra
e3138cd56a
make enable/disable trigger idempotent
2019-03-08 08:43:22 +01:00
Gonzalo Riestra
ab6720ad32
add create/drop/alter view to the trigger
2019-03-08 08:28:24 +01:00
Gonzalo Riestra
5f154a5859
use TIS instead of Redis
2019-03-08 08:28:06 +01:00
Gonzalo Riestra
99dd7cefc7
ghost tables functions using redis
2019-03-06 18:20:04 +01:00
Raul Marin
5d4f1d98d7
Update ST_Shift_Longitude to ST_ShiftLongitude
2019-03-01 11:27:09 +01:00
Gonzalo Riestra
0568b36a90
use session_user instead of current_user
2019-02-20 14:45:57 +01:00
Gonzalo Riestra
2a4ecd4850
Add CDB_Username() function
2019-02-20 10:38:00 +01:00
Raul Marin
6c57751579
CDB_TransformToWebmercator: Yet another hack over a function that shouldn't exist
2019-01-02 19:33:28 +01:00
Raul Marin
80fdd00541
Drop functions removed in 0.11.1
2019-01-02 18:56:24 +01:00
Raul Marin
d2a00852a8
Jenks: Iterate increasing the number of moved elements exponentially
2018-09-11 19:22:48 +02:00
Raul Marin
e605234d38
Jenks: Remove log messages
2018-09-10 15:53:02 +02:00
Raul Marin
0e7d797400
Jenks: Fix multiple bugs
2018-09-10 15:42:35 +02:00
Raul Marin
7e131143f9
_cdb_estimated_extent: Adapt to ST_EstimatedExtent change
...
The behaviour was changed in Postgis 1.5.4 to return NULL instead of throwing
2018-09-04 18:21:28 +02:00
Eneko Lakasta
b209726b1c
Merge pull request #320 from bloomberg/v0_18_5/cdb_tablemetadata_text_hotfix
...
Fix incorrect timestamps in CDB_TableMetadata_Text
2018-07-23 16:00:19 +02:00
Javier Torres
5659275c0c
Don't rely on regexp to identify non explainable queries
2018-07-20 13:01:51 +02:00
Javier Goizueta
2766bbc83a
Fix PARALLEL tags
...
Fixes #333
2018-07-18 12:13:31 +02:00
Rafa de la Torre
61d2024eb5
Make the code nicer by avoiding IF/THEN/ELSE
...
As suggested by Algunenano.
2018-07-03 15:35:05 +02:00
Rafa de la Torre
af142306aa
Mark _CDB_Table_Exists() as PARALLEL UNSAFE
...
As pointed out by Algunenano, PL/pgSQL function which establishes an
EXCEPTION block to catch errors must be qualified with it.
2018-07-03 15:26:48 +02:00
Rafa de la Torre
82f90e618c
Use CREATE OR REPLACE FUNCTION
2018-07-03 13:04:39 +02:00
Rafa de la Torre
55a77b0ef0
Add a new helper function _CDB_Table_Exists
2018-07-03 13:00:24 +02:00
Javier Torres
ccbabaa3b4
Missing quote idents
2018-05-31 17:06:39 +02:00
Javier Torres
7f5bef1203
Escape regclass call with quote_ident
2018-05-29 14:04:53 +02:00
Javier Torres
625d62c448
Remove unneeded variable
2018-05-28 20:29:36 +02:00
Javier Torres
f06418c99b
Extract sequence-listing function
2018-05-28 20:14:57 +02:00
Javier Torres
1958f2de5b
Working code to add permission to all table sequences
2018-05-28 20:07:16 +02:00
Mario de Frutos
c9f4685e5d
Fix error with hyphens, moving logic to the Format part
2018-03-21 16:33:05 +01:00
Mario de Frutos
f96c334f48
Fixed a trucate problem with the seq rename. See #325
2018-03-21 12:05:56 +01:00
Andy Eschbacher
697a0a3a36
Merge branch 'master' into quantiles-bins-updates
2018-03-12 09:50:01 -04:00
Paul Ramsey
62c27ab140
Merge branch 'master' of github.com:CartoDB/cartodb-postgresql into seq-drop
2018-03-09 09:58:25 -08:00
Andy Eschbacher
b6b0ef704a
Merge branch 'master' into quantiles-bins-updates
2018-03-08 10:33:01 -05:00
Andy Eschbacher
39998a9c88
fix syntax error oof oof
2018-03-08 10:21:43 -05:00
Andy Eschbacher
cf38d4cf25
oof include SELECT
2018-03-08 10:14:08 -05:00
Andy Eschbacher
0467c075f7
opts for array version of percentile_disc
2018-03-08 10:00:52 -05:00
Javier Goizueta
0a58c05049
Merge pull request #323 from CartoDB/321-grant-sequence
...
grant usage on cartodb_id sequence when sharing read write
2018-02-15 16:29:25 +01:00
Alberto Romeu
cd4653ecc0
grant usage on cartodb_id sequence when sharing read write
2018-02-15 08:40:17 +01:00
Javier Goizueta
7ea428913a
Add limit to the grid-generating functions
...
This adds an optinal maxcells parameter to the rectangle and hexagonal grids,
with a default value of 262144 (2^18).
An error occurs when the grids would need more cells.
2018-02-14 17:50:06 +01:00
Tyler Parsons
d811a71da0
Fix incorrect timestamps in CDB_TableMetadata_Text
...
Instead of performing a proper join on tabname, CDB_TableMetadata_Text joins
cdb_tablemetadata against pg_catalog.pg_class (i.e. All postgres tables,
views, indices, etc.) and gives a record for every possible tabname and
updated_at combination. This results in the latest updated timestamp being
chosen for any table in CDB_Get_Foreign_Updated_At, which leads to unnecessary
and incorrect cache invalidation.
2018-01-25 12:27:38 -05:00
Paul Ramsey
4050555801
Quiet errors where there's no existing sequence
2018-01-18 10:26:01 -08:00
Paul Ramsey
e4b0e7ea7a
Make user final seq has the right desired name
2018-01-18 10:06:38 -08:00
Paul Ramsey
a79c2cb7a8
draft tweek on seq handling
2018-01-18 09:27:46 -08:00
Andy Eschbacher
691b9a8312
moves quantile bins to use postgres precentile functions
2017-12-05 16:16:39 -05:00
Raul Marin
9d980ab17c
Mark CDB_JenksBins as VOLATILE as it uses random
2017-11-13 09:35:20 +01:00
Raul Marin
72c214a8e4
Fix regex for PARALLEL aggregations
...
Worked in pg10, but it didn't in pg9.5
2017-11-13 09:35:20 +01:00
Raul Marin
0abc2ba250
Add PARALLEL parameter to functions
2017-11-13 09:35:20 +01:00
Raul Marin
f24d15f6ca
Fix issues when running under pg10
2017-10-25 14:38:06 +02:00
Rafa de la Torre
ffb779eb74
Increase search space of ids by 100x #305
2017-06-29 17:54:42 +02:00
Mario de Frutos
8a3d506a53
Add some debug logs to be used in the future
2017-06-02 15:41:18 +02:00
Mario de Frutos
4908bacc4b
If existing overviews we regenerate them
...
Instead of make a DROP/CREATE always, in case we have existing
overviews we have to make DELETE/INSERT because DDL operations require
AccessExclusiveLock which is not compatible with AccessShareLock that
is used by the queries used to gather data for the tiler.
This incompatibility above mentioned leads to have deadlocks
2017-06-02 15:23:47 +02:00
Mario de Frutos
38fa3b485c
Remove unused and old _CDB_GridCluster_Reduce_Strategy function for overviews
2017-06-02 13:05:05 +02:00
Javier Goizueta
234373df11
Replace unnecessary count
2017-04-10 08:08:59 +02:00
Javier Goizueta
a486eed2e3
Add CDB_EstimateRowCount function
...
See #295
2017-04-07 15:35:48 +02:00
Javier Goizueta
bbadcc838e
Merge pull request #244 from CartoDB/equalbins
...
Convert CDB_EqualIntervalBins to a single SQL statement and add float version
2016-11-30 16:09:05 +01:00
Javier Goizueta
b1a0904c07
Merge pull request #181 from CartoDB/update_to_cdb_stats
...
Fix for division by zero error on empty or homogenous array
2016-11-30 16:08:40 +01:00
Javier Goizueta
7c0636c5f9
Merge pull request #290 from CartoDB/286-overview-strategies
...
Add point overview strategies
2016-11-30 11:46:29 +01:00
Javier Goizueta
f58f870457
Remove use of first aggregator in sample-cluster overviews strategy
...
This is not more efficient but the geometry now corresponds to the cartodb_id
and the dependency with custom aggregators (firt) is removed.
2016-11-29 14:08:08 +01:00
Javier Goizueta
7d7ecc06f5
Merge branch 'master' into fix-281-merge
2016-11-03 15:57:24 +01:00
Javier Goizueta
5992304b47
Add a couple of overview clustering strategies
2016-11-03 13:31:04 +01:00
Alejandro Martínez
0bfdeae147
Move "username" column to the last position on analysis_catalog - fixes #276
...
Due to the way it was first implemented, the "username" column would be
on a different position depending on if it was an extension upgrade or
a fresh install.
This caused problems with pg_dumping databases and restoring them.
pg_dump does not include the extension source (so this table's schema is
not included on the dump) but does include this contents, using a COPY
without column names by default and failing due to the order difference.
After this has run, all tables (whether updated or not) will have the
"username" column on the last position.
2016-10-20 14:30:02 +02:00
