From 29c719354a15d904da2ebd91c3f0a8ae3db42c47 Mon Sep 17 00:00:00 2001 From: csobier Date: Thu, 26 May 2016 11:00:13 -0400 Subject: [PATCH 01/49] applied docs 781 to dataservices-api doc --- doc/API.md | 2 +- doc/demographic_functions.md | 14 +++---- doc/geocoding_functions.md | 78 +++++++++++++++++------------------ doc/isoline_functions.md | 24 +++++------ doc/overview.md | 20 ++++----- doc/quota_information.md | 2 +- doc/routing_functions.md | 16 +++---- doc/segmentation_functions.md | 16 +++---- 8 files changed, 86 insertions(+), 86 deletions(-) diff --git a/doc/API.md b/doc/API.md index 9e3b92d..275f635 100644 --- a/doc/API.md +++ b/doc/API.md @@ -1,6 +1,6 @@ # Data Services API -The CartoDB Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications. +The Carto Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications. ## Documentation diff --git a/doc/demographic_functions.md b/doc/demographic_functions.md index 73084a6..c61a2a5 100644 --- a/doc/demographic_functions.md +++ b/doc/demographic_functions.md @@ -1,6 +1,6 @@ # Demographic Functions -The Demographic Snapshot enables you to collect demographic reports around a point location. For example, you can take the coordinates of a coffee shop and find the average population characteristics, such as total population, educational attainment, housing and income information around that location. You can use raw street addresses by combining the Demographic Snapshot with CartoDB's geocoding features. If you need help creating coordinates from addresses, see the [Geocoding Functions](/cartodb-platform/dataservices-api/geocoding-functions/) documentation. +The Demographic Snapshot enables you to collect demographic reports around a point location. For example, you can take the coordinates of a coffee shop and find the average population characteristics, such as total population, educational attainment, housing and income information around that location. You can use raw street addresses by combining the Demographic Snapshot with Carto's geocoding features. If you need help creating coordinates from addresses, see the [Geocoding Functions](/carto-engine/dataservices-api/geocoding-functions/) documentation. _**Note:** The Demographic Snapshot functions are only available for the United States._ @@ -12,7 +12,7 @@ Fields returned include information about income, education, transportation, rac Name | Description | Example Values --- | --- | --- -point geometry | A point geometry. You can use the helper function, `CDB_LatLng` to quickly generate one from latitude and longitude | `CDB_LatLng(40.760410,-73.964242)` +point geometry | A point geometry. You can use the helper function, `carto_LatLng` to quickly generate one from latitude and longitude | `carto_LatLng(40.760410,-73.964242)` ### Returns @@ -40,7 +40,7 @@ obs_getdemographicsnapshot: { ### Examples ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetDemographicSnapshot({{point geometry}}) ``` @@ -49,15 +49,15 @@ OBS_GetDemographicSnapshot({{point geometry}}) __Get the Demographic Snapshot at Camp David__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM -OBS_GetDemographicSnapshot(CDB_LatLng(39.648333, -77.465)) +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +OBS_GetDemographicSnapshot(carto_LatLng(39.648333, -77.465)) ``` __Get the Demographic Snapshot in the Upper West Side__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM -OBS_GetDemographicSnapshot(CDB_LatLng(40.80, -73.960)) +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +OBS_GetDemographicSnapshot(carto_LatLng(40.80, -73.960)) ``` ## Glossary of Demographic Measures diff --git a/doc/geocoding_functions.md b/doc/geocoding_functions.md index ef5dc2f..af29225 100644 --- a/doc/geocoding_functions.md +++ b/doc/geocoding_functions.md @@ -1,25 +1,25 @@ # Geocoding Functions -The [geocoder](https://cartodb.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CartoDB SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CartoDB provides functions for several different categories of geocoding through the Data Services API. +The [geocoder](https://carto.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the Carto SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. Carto provides functions for several different categories of geocoding through the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ Here is an example of how to geocode a single country: ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=SELECT carto_geocode_admin0_polygon('USA')&api_key={api_key} ``` -In order to geocode an existent CartoDB dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset: +In order to geocode an existent Carto dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset: ```bash -https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = carto_geocode_admin0_polygon('USA')&api_key={api_key} ``` Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function: ```bash -https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(carto_geocode_admin0_polygon('USA'))&api_key={api_key} ``` @@ -29,7 +29,7 @@ The following geocoding functions are available, grouped by categories. This function geocodes your data into country border geometries. It recognizes the names of the different countries either by different synonyms (such as their English name or their endonym), or by ISO (ISO2 or ISO3) codes. -### cdb_geocode_admin0_polygon(_country_name text_) +### carto_geocode_admin0_polygon(_country_name text_) Geocodes the text name of a country into a country_name geometry, displayed as polygon data. @@ -48,20 +48,20 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column}) +UPDATE {tablename} SET the_geom = carto_geocode_admin0_polygon({country_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin0_polygon('France') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin0_polygon('France') ``` ## Level-1 Administrative Regions Geocoder This function geocodes your data into polygon geometries for [Level 1](https://en.wikipedia.org/wiki/Table_of_administrative_divisions_by_country), or [NUTS-1](https://en.wikipedia.org/wiki/NUTS_1_statistical_regions_of_England), administrative divisions (or units) of countries. For example, a "state" in the United States, "départements" in France, or an autonomous community in Spain. -### cdb_geocode_admin1_polygon(_admin1_name text_) +### carto_geocode_admin1_polygon(_admin1_name text_) Geocodes the name of the province/state into a Level-1 administrative region, displayed as a polygon geometry. @@ -80,17 +80,17 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column}) +UPDATE {tablename} SET the_geom = carto_geocode_admin1_polygon({province_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin1_polygon('Alicante') ``` -### cdb_geocode_admin1_polygon(_admin1_name text, country_name text_) +### carto_geocode_admin1_polygon(_admin1_name text, country_name text_) Geocodes the name of the province/state for a specified country into a Level-1 administrative region, displayed as a polygon geometry. @@ -110,12 +110,12 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column}, {country_column}) +UPDATE {tablename} SET the_geom = carto_geocode_admin1_polygon({province_column}, {country_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante', 'Spain') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin1_polygon('Alicante', 'Spain') ``` @@ -123,7 +123,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante', This function geocodes your data into point geometries for names of cities. It is recommended to use geocoding functions that require more defined parameters — this returns more accurate results when several cities have the same name. _If there are duplicate results for a city name, the city name with the highest population will be returned._ -### cdb_geocode_namedplace_point(_city_name text_) +### carto_geocode_namedplace_point(_city_name text_) Geocodes the text name of a city into a named place geometry, displayed as point data. @@ -144,17 +144,17 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}) +UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('Barcelona') ``` -### cdb_geocode_namedplace_point(_city_name text, country_name text_) +### carto_geocode_namedplace_point(_city_name text, country_name text_) Geocodes the text name of a city for a specified country into a named place point geometry. @@ -174,17 +174,17 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, 'Spain') +UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}, 'Spain') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona', 'Spain') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('Barcelona', 'Spain') ``` -### cdb_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_) +### carto_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_) Geocodes your data into a named place point geometry, containing the text name of a city, for a specified province/state and country. This is recommended for the most accurate geocoding of city data. #### Arguments @@ -204,22 +204,22 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, {province_column}, 'USA') +UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}, {province_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('New York', 'New York', 'USA') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('New York', 'New York', 'USA') ``` ## Postal Code Geocoder This function geocodes your data into point, or polygon, geometries for postal codes. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response. -**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.cartodb.com/faqs/datasets-and-data/#why-does-cartodb-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. +**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.carto.com/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. -### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_) +### carto_geocode_postalcode_polygon(_postal_code text, country_name text_) Goecodes the postal code for a specified country into a **polygon** geometry. @@ -239,16 +239,16 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_polygon({postal_code_column}, 'USA') +UPDATE {tablename} SET the_geom = carto_geocode_postalcode_polygon({postal_code_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_polygon('11211', 'USA') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_postalcode_polygon('11211', 'USA') ``` -### cdb_geocode_postalcode_point(_code text, country_name text_) +### carto_geocode_postalcode_point(_code text, country_name text_) Goecodes the postal code for a specified country into a **point** geometry. @@ -268,13 +268,13 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_point({postal_code_column}, 'USA') +UPDATE {tablename} SET the_geom = carto_geocode_postalcode_point({postal_code_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_point('11211', 'USA') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_postalcode_point('11211', 'USA') ``` @@ -282,7 +282,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_point('11211', This function geocodes your data into point geometries for IP addresses. This is useful if you are analyzing location based data, based on a set of user's IP addresses. -### cdb_geocode_ipaddress_point(_ip_address text_) +### carto_geocode_ipaddress_point(_ip_address text_) Geocodes a postal code from a specified country into an IP address, displayed as a point geometry. @@ -302,22 +302,22 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_ipaddress_point('102.23.34.1') +UPDATE {tablename} SET the_geom = carto_geocode_ipaddress_point('102.23.34.1') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_ipaddress_point('102.23.34.1') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_ipaddress_point('102.23.34.1') ``` ## Street-Level Geocoder -This function geocodes your data into a point geometry for a street address. CartoDB uses several different service providers for street-level geocoding, depending on your platform. If you access CartoDB on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. +This function geocodes your data into a point geometry for a street address. Carto uses several different service providers for street-level geocoding, depending on your platform. If you access Carto on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. -**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) for details and recommendations about quota consumption. +**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. -### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_) +### carto_geocode_street_point(_search_text text, [city text], [state text], [country text]_) Geocodes a complete address into a single street geometry, displayed as point data. @@ -339,11 +339,11 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = cdb_geocode_street_point({street_name_column}) +UPDATE {tablename} SET the_geom = carto_geocode_street_point({street_name_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States') +INSERT INTO {tablename} (the_geom) SELECT carto_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States') ``` diff --git a/doc/isoline_functions.md b/doc/isoline_functions.md index 0c54efb..250022a 100644 --- a/doc/isoline_functions.md +++ b/doc/isoline_functions.md @@ -1,18 +1,18 @@ # Isoline Functions -[Isolines](https://cartodb.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CartoDB are available by requesting a single function in the Data Services API. +[Isolines](https://carto.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through Carto are available by requesting a single function in the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table. ```bash -https://{username}.cartodb.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} ``` The following functions provide an isoline generator service, based on time or distance. This service uses the isolines service defined for your account. The default service limits the usage of displayed polygons represented on top of [HERE](https://developer.here.com/coverage-info) maps. -## cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_) +## carto_isodistance(_source geometry, mode text, range integer[], [options text[]]_) Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of distance (in meters). @@ -39,29 +39,29 @@ Name | Type | Description ##### Calculate and insert isodistance polygons from a point into another table ```bash -INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[]) +INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[]) ``` or equivalently: ```bash -INSERT INTO {table} (the_geom) SELECT (cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])).the_geom +INSERT INTO {table} (the_geom) SELECT (carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])).the_geom ``` ##### Calculate and insert the generated isolines from `points_table` table to another table ```bash -INSERT INTO {table} (the_geom) SELECT (cdb_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table} +INSERT INTO {table} (the_geom) SELECT (carto_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table} ``` -## cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_) +## carto_isochrone(_source geometry, mode text, range integer[], [options text[]]_) Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of time (in seconds). #### Arguments -This function uses the same parameters and information as the `cdb_isodistance` function, with the exception that the range is measured in seconds instead of meters. +This function uses the same parameters and information as the `carto_isodistance` function, with the exception that the range is measured in seconds instead of meters. Name | Type | Description | Accepted values --- | --- | --- | --- @@ -75,19 +75,19 @@ Name | Type | Description | Accepted values ##### Calculate and insert isochrone polygons from a point into another table ```bash -INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]) +INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]) ``` or equivalently: ```bash -INSERT INTO {table} (the_geom) SELECT (cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])).the_geom +INSERT INTO {table} (the_geom) SELECT (carto_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])).the_geom ``` ##### Calculate and insert the generated isolines from `points_table` table into another table ```bash -INSERT INTO {table} (the_geom) SELECT (cdb_isochrone(the_geom, 'walk', string_to_array(time_distance, ',')::integer[])).the_geom FROM {points_table} +INSERT INTO {table} (the_geom) SELECT (carto_isochrone(the_geom, 'walk', string_to_array(time_distance, ',')::integer[])).the_geom FROM {points_table} ``` ### Optional isoline parameters diff --git a/doc/overview.md b/doc/overview.md index ed9d307..d1bf299 100644 --- a/doc/overview.md +++ b/doc/overview.md @@ -1,32 +1,32 @@ # Overview -By using CartoDB libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. +By using Carto libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. -**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/#quota-information). +**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.carto.com/carto-engine/dataservices-api/quota-information/#quota-information). -_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CartoDB Platform._ +_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our Carto Engine._ ## Data Services Integration By using the SQL API to query the Data Services API functions, you can manage specific operations and the corresponding geometries (a `polygon` or a `point`), according to the input information. -The Data Services API decouples the geocoding and isoline services from the CartoDB Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests. +The Data Services API decouples the geocoding and isoline services from the Carto Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests. The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/). -**Note:** The Data Services API [geocoding functions](http://docs.cartodb.com/cartodb-platform/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CartoDB Platform does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. +**Note:** The Data Services API [geocoding functions](http://docs.carto.com/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The Carto Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. ### Best Practices _Be mindful of the following usage notes when using the Data Services functions with the SQL API:_ -It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://docs.cartodb.com/cartodb-platform/dataservices-api/quota-information/#quota-consumption). +It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://docs.carto.com/carto-engine/dataservices-api/quota-information/#quota-consumption). The Data Services API is **recommended** to be used with INSERT or UPDATE operations, for applying location data to your tables. While SELECT (retrieve) is standard for SQL API requests, be mindful of quota consumption and use INSERT (to insert a new record) or UPDATE (to update an existing record), for best practices. ## Authentication -All requests performed to the CartoDB Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/cartodb-platform/sql-api/authentication/) documentation. +All requests performed to the Carto Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/carto-engine/sql-api/authentication/) documentation. ## Errors @@ -40,10 +40,10 @@ Errors are described in the response of the request. An example is as follows: } ``` -Since the Data Services API is used on top of the CartoDB SQL API, you can refer to the [Making calls to the SQL API](/cartodb-platform/sql-api/making-calls/) documentation for help debugging your SQL errors. +Since the Data Services API is used on top of the Carto SQL API, you can refer to the [Making calls to the SQL API](/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. -If the requested information is not in the CartoDB geocoding database, or if CartoDB is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. +If the requested information is not in the Carto geocoding database, or if Carto is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. ## Limits -Usage of the Data Services API is subject to the CartoDB SQL API limits, stated in our [Terms of Service](https://cartodb.com/terms/#excessive). +Usage of the Data Services API is subject to the Carto SQL API limits, stated in our [Terms of Service](https://carto.com/terms/#excessive). diff --git a/doc/quota_information.md b/doc/quota_information.md index e87dca1..9f9581b 100644 --- a/doc/quota_information.md +++ b/doc/quota_information.md @@ -1,6 +1,6 @@ # Quota Information -**Based on your account plan, some of the Data Services API functions are subject to quota limitations and extra fees may apply.** View our [terms and conditions](https://cartodb.com/terms/), or [contact us](mailto:sales@cartodb.com) for details about which functions require service credits to your account. +**Based on your account plan, some of the Data Services API functions are subject to quota limitations and extra fees may apply.** View our [terms and conditions](https://carto.com/terms/), or [contact us](mailto:sales@carto.com) for details about which functions require service credits to your account. ## Quota Consumption diff --git a/doc/routing_functions.md b/doc/routing_functions.md index 49aa69b..adc67c4 100644 --- a/doc/routing_functions.md +++ b/doc/routing_functions.md @@ -1,8 +1,8 @@ # Routing Functions -Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through CartoDB are available by using the available functions in the Data Services API. +Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through Carto are available by using the available functions in the Data Services API. -### cdb_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_) +### carto_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_) Returns a route from origin to destination. @@ -30,15 +30,15 @@ Name | Type | Description ##### Insert the values from the calculated route in your table ```bash -INSERT INTO (duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car') +INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM carto_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car') ``` ##### Update the geometry field with the calculated route shape ```bash -UPDATE
SET the_geom = (SELECT shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[])) +UPDATE
SET the_geom = (SELECT shape FROM carto_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[])) ``` -### cdb_route_with_waypoints(_waypoints geometry(Point)[], mode text, [options text[], units text]_) +### carto_route_with_waypoints(_waypoints geometry(Point)[], mode text, [options text[], units text]_) Returns a route that goes from origin to destination and whose path travels through the defined locations. @@ -60,19 +60,19 @@ Name | Type | Description `length` | `real` | Length in the defined unit in the `units` field. `kilometers` by default . `the_geom` | `geometry(LineString)` | LineString geometry of the calculated route in the 4326 projection. -*Note*: A request to the function _cdb\_route\_with\_waypoints(waypoints geometry(Point)[], mode text, [options text[], units text])_ with only two points in the geometry array are automatically defined as origin and destination. It is equivalent to performing the following request with these two locations as parameters: _cdb\_route\_point\_to\_point(origin geometry(Point), destination geometry(Point), mode text, [options text[], units text])_. +*Note*: A request to the function _carto\_route\_with\_waypoints(waypoints geometry(Point)[], mode text, [options text[], units text])_ with only two points in the geometry array are automatically defined as origin and destination. It is equivalent to performing the following request with these two locations as parameters: _carto\_route\_point\_to\_point(origin geometry(Point), destination geometry(Point), mode text, [options text[], units text])_. #### Examples ##### Insert the values from the calculated route in your table ```bash -INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'walk') +INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM carto_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'walk') ``` ##### Update the geometry field with the calculated route shape ```bash -UPDATE
SET the_geom = (SELECT shape FROM cdb_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'car', ARRAY['mode_type=shortest']::text[])) +UPDATE
SET the_geom = (SELECT shape FROM carto_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'car', ARRAY['mode_type=shortest']::text[])) ``` diff --git a/doc/segmentation_functions.md b/doc/segmentation_functions.md index cc71054..b84e5bf 100644 --- a/doc/segmentation_functions.md +++ b/doc/segmentation_functions.md @@ -1,6 +1,6 @@ # Segmentation Functions -The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](/cartodb-platform/dataservices-api/geocoding-functions/) documentation. +The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](/carto-engine/dataservices-api/geocoding-functions/) documentation. _**Note:** The Segmentation Snapshot functions are only available for the United States. Our first release (May 18, 2016) is derived from Census 2010 variables. Our next release will be based on Census 2014 data. For the latest information, see the [Open Segments](https://github.com/CartoDB/open-segments) project repository._ @@ -10,7 +10,7 @@ _**Note:** The Segmentation Snapshot functions are only available for the United Name | Description | Example Values --- | --- | --- -point geometry | A point geometry. You can use the helper function, `CDB_LatLng` to quickly generate one from latitude and longitude | `CDB_LatLng(40.760410,-73.964242)` +point geometry | A point geometry. You can use the helper function, `carto_LatLng` to quickly generate one from latitude and longitude | `carto_LatLng(40.760410,-73.964242)` ### Returns @@ -158,7 +158,7 @@ The possible segments are: ### Examples ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetSegmentSnapshot({{point geometry}}) ``` @@ -168,14 +168,14 @@ __Get the Segmentation Snapshot around the MGM Grand__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM -OBS_GetSegmentSnapshot(CDB_LatLng(36.10222, -115.169516)) +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +OBS_GetSegmentSnapshot(carto_LatLng(36.10222, -115.169516)) ``` -__Get the Segmentation Snapshot at CartoDB's NYC HQ__ +__Get the Segmentation Snapshot at Carto's NYC HQ__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM -OBS_GetSegmentSnapshot(CDB_LatLng(40.704512, -73.936669)) +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +OBS_GetSegmentSnapshot(carto_LatLng(40.704512, -73.936669)) ``` From 14ba8c6b84309784564930d41fcb4a5fd2fc85e2 Mon Sep 17 00:00:00 2001 From: csobier Date: Tue, 31 May 2016 13:20:30 -0400 Subject: [PATCH 02/49] reverted rebranded code, as instructed. Legacy cartodb code appears instead. Also applied ALL CAPS for rebranded name, as instruted --- doc/API.md | 2 +- doc/demographic_functions.md | 14 +++---- doc/geocoding_functions.md | 72 +++++++++++++++++------------------ doc/isoline_functions.md | 22 +++++------ doc/overview.md | 16 ++++---- doc/routing_functions.md | 16 ++++---- doc/segmentation_functions.md | 14 +++---- 7 files changed, 78 insertions(+), 78 deletions(-) diff --git a/doc/API.md b/doc/API.md index 275f635..816c561 100644 --- a/doc/API.md +++ b/doc/API.md @@ -1,6 +1,6 @@ # Data Services API -The Carto Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications. +The CARTO Data Services API offers a set of location based services that can be used programatically to empower your geospatial applications. ## Documentation diff --git a/doc/demographic_functions.md b/doc/demographic_functions.md index c61a2a5..3e698bb 100644 --- a/doc/demographic_functions.md +++ b/doc/demographic_functions.md @@ -1,6 +1,6 @@ # Demographic Functions -The Demographic Snapshot enables you to collect demographic reports around a point location. For example, you can take the coordinates of a coffee shop and find the average population characteristics, such as total population, educational attainment, housing and income information around that location. You can use raw street addresses by combining the Demographic Snapshot with Carto's geocoding features. If you need help creating coordinates from addresses, see the [Geocoding Functions](/carto-engine/dataservices-api/geocoding-functions/) documentation. +The Demographic Snapshot enables you to collect demographic reports around a point location. For example, you can take the coordinates of a coffee shop and find the average population characteristics, such as total population, educational attainment, housing and income information around that location. You can use raw street addresses by combining the Demographic Snapshot with CARTO's geocoding features. If you need help creating coordinates from addresses, see the [Geocoding Functions](/carto-engine/dataservices-api/geocoding-functions/) documentation. _**Note:** The Demographic Snapshot functions are only available for the United States._ @@ -12,7 +12,7 @@ Fields returned include information about income, education, transportation, rac Name | Description | Example Values --- | --- | --- -point geometry | A point geometry. You can use the helper function, `carto_LatLng` to quickly generate one from latitude and longitude | `carto_LatLng(40.760410,-73.964242)` +point geometry | A point geometry. You can use the helper function, `CDB_LatLng` to quickly generate one from latitude and longitude | `CDB_LatLng(40.760410,-73.964242)` ### Returns @@ -40,7 +40,7 @@ obs_getdemographicsnapshot: { ### Examples ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM OBS_GetDemographicSnapshot({{point geometry}}) ``` @@ -49,15 +49,15 @@ OBS_GetDemographicSnapshot({{point geometry}}) __Get the Demographic Snapshot at Camp David__ ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM -OBS_GetDemographicSnapshot(carto_LatLng(39.648333, -77.465)) +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +OBS_GetDemographicSnapshot(CDB_LatLng(39.648333, -77.465)) ``` __Get the Demographic Snapshot in the Upper West Side__ ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM -OBS_GetDemographicSnapshot(carto_LatLng(40.80, -73.960)) +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +OBS_GetDemographicSnapshot(CDB_LatLng(40.80, -73.960)) ``` ## Glossary of Demographic Measures diff --git a/doc/geocoding_functions.md b/doc/geocoding_functions.md index af29225..36fe3ba 100644 --- a/doc/geocoding_functions.md +++ b/doc/geocoding_functions.md @@ -1,25 +1,25 @@ # Geocoding Functions -The [geocoder](https://carto.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the Carto SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. Carto provides functions for several different categories of geocoding through the Data Services API. +The [geocoder](https://carto.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CARTO SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CARTO provides functions for several different categories of geocoding through the Data Services API. _**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ Here is an example of how to geocode a single country: ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT carto_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key} ``` -In order to geocode an existent Carto dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset: +In order to geocode an existent CARTO dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset: ```bash -https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = carto_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon('USA')&api_key={api_key} ``` Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function: ```bash -https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(carto_geocode_admin0_polygon('USA'))&api_key={api_key} +https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key} ``` @@ -29,7 +29,7 @@ The following geocoding functions are available, grouped by categories. This function geocodes your data into country border geometries. It recognizes the names of the different countries either by different synonyms (such as their English name or their endonym), or by ISO (ISO2 or ISO3) codes. -### carto_geocode_admin0_polygon(_country_name text_) +### cdb_geocode_admin0_polygon(_country_name text_) Geocodes the text name of a country into a country_name geometry, displayed as polygon data. @@ -48,20 +48,20 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_admin0_polygon({country_column}) +UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin0_polygon('France') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin0_polygon('France') ``` ## Level-1 Administrative Regions Geocoder This function geocodes your data into polygon geometries for [Level 1](https://en.wikipedia.org/wiki/Table_of_administrative_divisions_by_country), or [NUTS-1](https://en.wikipedia.org/wiki/NUTS_1_statistical_regions_of_England), administrative divisions (or units) of countries. For example, a "state" in the United States, "départements" in France, or an autonomous community in Spain. -### carto_geocode_admin1_polygon(_admin1_name text_) +### cdb_geocode_admin1_polygon(_admin1_name text_) Geocodes the name of the province/state into a Level-1 administrative region, displayed as a polygon geometry. @@ -80,17 +80,17 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_admin1_polygon({province_column}) +UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin1_polygon('Alicante') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante') ``` -### carto_geocode_admin1_polygon(_admin1_name text, country_name text_) +### cdb_geocode_admin1_polygon(_admin1_name text, country_name text_) Geocodes the name of the province/state for a specified country into a Level-1 administrative region, displayed as a polygon geometry. @@ -110,12 +110,12 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_admin1_polygon({province_column}, {country_column}) +UPDATE {tablename} SET the_geom = cdb_geocode_admin1_polygon({province_column}, {country_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin1_polygon('Alicante', 'Spain') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_admin1_polygon('Alicante', 'Spain') ``` @@ -123,7 +123,7 @@ INSERT INTO {tablename} (the_geom) SELECT carto_geocode_admin1_polygon('Alicante This function geocodes your data into point geometries for names of cities. It is recommended to use geocoding functions that require more defined parameters — this returns more accurate results when several cities have the same name. _If there are duplicate results for a city name, the city name with the highest population will be returned._ -### carto_geocode_namedplace_point(_city_name text_) +### cdb_geocode_namedplace_point(_city_name text_) Geocodes the text name of a city into a named place geometry, displayed as point data. @@ -144,17 +144,17 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}) +UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('Barcelona') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona') ``` -### carto_geocode_namedplace_point(_city_name text, country_name text_) +### cdb_geocode_namedplace_point(_city_name text, country_name text_) Geocodes the text name of a city for a specified country into a named place point geometry. @@ -174,17 +174,17 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}, 'Spain') +UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, 'Spain') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('Barcelona', 'Spain') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('Barcelona', 'Spain') ``` -### carto_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_) +### cdb_geocode_namedplace_point(_city_name text, admin1_name text, country_name text_) Geocodes your data into a named place point geometry, containing the text name of a city, for a specified province/state and country. This is recommended for the most accurate geocoding of city data. #### Arguments @@ -204,13 +204,13 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_namedplace_point({city_column}, {province_column}, 'USA') +UPDATE {tablename} SET the_geom = cdb_geocode_namedplace_point({city_column}, {province_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_namedplace_point('New York', 'New York', 'USA') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('New York', 'New York', 'USA') ``` ## Postal Code Geocoder @@ -219,7 +219,7 @@ This function geocodes your data into point, or polygon, geometries for postal c **Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.carto.com/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. -### carto_geocode_postalcode_polygon(_postal_code text, country_name text_) +### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_) Goecodes the postal code for a specified country into a **polygon** geometry. @@ -239,16 +239,16 @@ Geometry (polygon, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_postalcode_polygon({postal_code_column}, 'USA') +UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_polygon({postal_code_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_postalcode_polygon('11211', 'USA') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_polygon('11211', 'USA') ``` -### carto_geocode_postalcode_point(_code text, country_name text_) +### cdb_geocode_postalcode_point(_code text, country_name text_) Goecodes the postal code for a specified country into a **point** geometry. @@ -268,13 +268,13 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_postalcode_point({postal_code_column}, 'USA') +UPDATE {tablename} SET the_geom = cdb_geocode_postalcode_point({postal_code_column}, 'USA') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_postalcode_point('11211', 'USA') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_postalcode_point('11211', 'USA') ``` @@ -282,7 +282,7 @@ INSERT INTO {tablename} (the_geom) SELECT carto_geocode_postalcode_point('11211' This function geocodes your data into point geometries for IP addresses. This is useful if you are analyzing location based data, based on a set of user's IP addresses. -### carto_geocode_ipaddress_point(_ip_address text_) +### cdb_geocode_ipaddress_point(_ip_address text_) Geocodes a postal code from a specified country into an IP address, displayed as a point geometry. @@ -302,22 +302,22 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_ipaddress_point('102.23.34.1') +UPDATE {tablename} SET the_geom = cdb_geocode_ipaddress_point('102.23.34.1') ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_ipaddress_point('102.23.34.1') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_ipaddress_point('102.23.34.1') ``` ## Street-Level Geocoder -This function geocodes your data into a point geometry for a street address. Carto uses several different service providers for street-level geocoding, depending on your platform. If you access Carto on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. +This function geocodes your data into a point geometry for a street address. CARTO uses several different service providers for street-level geocoding, depending on your platform. If you access CARTO on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. **This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. -### carto_geocode_street_point(_search_text text, [city text], [state text], [country text]_) +### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_) Geocodes a complete address into a single street geometry, displayed as point data. @@ -339,11 +339,11 @@ Geometry (point, EPSG 4326) or null ##### Update the geometry of a table to geocode it ```bash -UPDATE {tablename} SET the_geom = carto_geocode_street_point({street_name_column}) +UPDATE {tablename} SET the_geom = cdb_geocode_street_point({street_name_column}) ``` ##### Insert a geocoded row into a table ```bash -INSERT INTO {tablename} (the_geom) SELECT carto_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States') +INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_street_point('651 Lombard Street', 'San Francisco', 'California', 'United States') ``` diff --git a/doc/isoline_functions.md b/doc/isoline_functions.md index 250022a..63660f2 100644 --- a/doc/isoline_functions.md +++ b/doc/isoline_functions.md @@ -1,18 +1,18 @@ # Isoline Functions -[Isolines](https://carto.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through Carto are available by requesting a single function in the Data Services API. +[Isolines](https://carto.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CARTO are available by requesting a single function in the Data Services API. _**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table. ```bash -https://{username}.carto.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} +https://{username}.cartodb.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} ``` The following functions provide an isoline generator service, based on time or distance. This service uses the isolines service defined for your account. The default service limits the usage of displayed polygons represented on top of [HERE](https://developer.here.com/coverage-info) maps. -## carto_isodistance(_source geometry, mode text, range integer[], [options text[]]_) +## cdb_isodistance(_source geometry, mode text, range integer[], [options text[]]_) Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of distance (in meters). @@ -39,29 +39,29 @@ Name | Type | Description ##### Calculate and insert isodistance polygons from a point into another table ```bash -INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[]) +INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[]) ``` or equivalently: ```bash -INSERT INTO {table} (the_geom) SELECT (carto_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])).the_geom +INSERT INTO {table} (the_geom) SELECT (cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'walk', ARRAY[300, 600, 900]::integer[])).the_geom ``` ##### Calculate and insert the generated isolines from `points_table` table to another table ```bash -INSERT INTO {table} (the_geom) SELECT (carto_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table} +INSERT INTO {table} (the_geom) SELECT (cdb_isodistance(the_geom, 'walk', string_to_array(distance, ',')::integer[])).the_geom FROM {points_table} ``` -## carto_isochrone(_source geometry, mode text, range integer[], [options text[]]_) +## cdb_isochrone(_source geometry, mode text, range integer[], [options text[]]_) Displays a contoured line on a map, connecting geometries to a defined area, measured by an equal range of time (in seconds). #### Arguments -This function uses the same parameters and information as the `carto_isodistance` function, with the exception that the range is measured in seconds instead of meters. +This function uses the same parameters and information as the `cdb_isodistance` function, with the exception that the range is measured in seconds instead of meters. Name | Type | Description | Accepted values --- | --- | --- | --- @@ -75,19 +75,19 @@ Name | Type | Description | Accepted values ##### Calculate and insert isochrone polygons from a point into another table ```bash -INSERT INTO {table} (the_geom) SELECT the_geom FROM carto_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]) +INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[]) ``` or equivalently: ```bash -INSERT INTO {table} (the_geom) SELECT (carto_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])).the_geom +INSERT INTO {table} (the_geom) SELECT (cdb_isochrone('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[], ARRAY['mode_traffic=enabled','quality=3']::text[])).the_geom ``` ##### Calculate and insert the generated isolines from `points_table` table into another table ```bash -INSERT INTO {table} (the_geom) SELECT (carto_isochrone(the_geom, 'walk', string_to_array(time_distance, ',')::integer[])).the_geom FROM {points_table} +INSERT INTO {table} (the_geom) SELECT (cdb_isochrone(the_geom, 'walk', string_to_array(time_distance, ',')::integer[])).the_geom FROM {points_table} ``` ### Optional isoline parameters diff --git a/doc/overview.md b/doc/overview.md index d1bf299..f4830c8 100644 --- a/doc/overview.md +++ b/doc/overview.md @@ -1,20 +1,20 @@ # Overview -By using Carto libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. +By using CARTO libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. **Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.carto.com/carto-engine/dataservices-api/quota-information/#quota-information). -_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our Carto Engine._ +_The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CARTO Engine._ ## Data Services Integration By using the SQL API to query the Data Services API functions, you can manage specific operations and the corresponding geometries (a `polygon` or a `point`), according to the input information. -The Data Services API decouples the geocoding and isoline services from the Carto Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests. +The Data Services API decouples the geocoding and isoline services from the CARTO Editor. The API allows you to geocode data (from single rows, complete datasets, or simple inputs) and to perform trade areas analysis (computing isodistances or isochrones) programatically, through authenticated requests. The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/). -**Note:** The Data Services API [geocoding functions](http://docs.carto.com/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The Carto Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. +**Note:** The Data Services API [geocoding functions](http://docs.carto.com/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CARTO Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. ### Best Practices @@ -26,7 +26,7 @@ The Data Services API is **recommended** to be used with INSERT or UPDATE operat ## Authentication -All requests performed to the Carto Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/carto-engine/sql-api/authentication/) documentation. +All requests performed to the CARTO Data Services API must be authenticated with the user API Key. For more information about where to find your API Key, and how to authenticate your SQL API requests, view the [SQL API authentication](/carto-engine/sql-api/authentication/) documentation. ## Errors @@ -40,10 +40,10 @@ Errors are described in the response of the request. An example is as follows: } ``` -Since the Data Services API is used on top of the Carto SQL API, you can refer to the [Making calls to the SQL API](/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. +Since the Data Services API is used on top of the CARTO SQL API, you can refer to the [Making calls to the SQL API](/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. -If the requested information is not in the Carto geocoding database, or if Carto is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. +If the requested information is not in the CARTO geocoding database, or if CARTO is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. ## Limits -Usage of the Data Services API is subject to the Carto SQL API limits, stated in our [Terms of Service](https://carto.com/terms/#excessive). +Usage of the Data Services API is subject to the CARTO SQL API limits, stated in our [Terms of Service](https://carto.com/terms/#excessive). diff --git a/doc/routing_functions.md b/doc/routing_functions.md index adc67c4..2f2e345 100644 --- a/doc/routing_functions.md +++ b/doc/routing_functions.md @@ -1,8 +1,8 @@ # Routing Functions -Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through Carto are available by using the available functions in the Data Services API. +Routing is the navigation from a defined start location to a defined end location. The calculated results are displayed as turn-by-turn directions on your map, based on the transportation mode that you specified. Routing services through CARTO are available by using the available functions in the Data Services API. -### carto_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_) +### cdb_route_point_to_point(_origin geometry(Point), destination geometry(Point), mode text, [options text[], units text]_) Returns a route from origin to destination. @@ -30,15 +30,15 @@ Name | Type | Description ##### Insert the values from the calculated route in your table ```bash -INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM carto_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car') +INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car') ``` ##### Update the geometry field with the calculated route shape ```bash -UPDATE
SET the_geom = (SELECT shape FROM carto_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[])) +UPDATE
SET the_geom = (SELECT shape FROM cdb_route_point_to_point('POINT(-3.70237112 40.41706163)'::geometry,'POINT(-3.69909883 40.41236875)'::geometry, 'car', ARRAY['mode_type=shortest']::text[])) ``` -### carto_route_with_waypoints(_waypoints geometry(Point)[], mode text, [options text[], units text]_) +### cdb_route_with_waypoints(_waypoints geometry(Point)[], mode text, [options text[], units text]_) Returns a route that goes from origin to destination and whose path travels through the defined locations. @@ -60,19 +60,19 @@ Name | Type | Description `length` | `real` | Length in the defined unit in the `units` field. `kilometers` by default . `the_geom` | `geometry(LineString)` | LineString geometry of the calculated route in the 4326 projection. -*Note*: A request to the function _carto\_route\_with\_waypoints(waypoints geometry(Point)[], mode text, [options text[], units text])_ with only two points in the geometry array are automatically defined as origin and destination. It is equivalent to performing the following request with these two locations as parameters: _carto\_route\_point\_to\_point(origin geometry(Point), destination geometry(Point), mode text, [options text[], units text])_. +*Note*: A request to the function _cdb\_route\_with\_waypoints(waypoints geometry(Point)[], mode text, [options text[], units text])_ with only two points in the geometry array are automatically defined as origin and destination. It is equivalent to performing the following request with these two locations as parameters: _cdb\_route\_point\_to\_point(origin geometry(Point), destination geometry(Point), mode text, [options text[], units text])_. #### Examples ##### Insert the values from the calculated route in your table ```bash -INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM carto_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'walk') +INSERT INTO
(duration, length, the_geom) SELECT duration, length, shape FROM cdb_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'walk') ``` ##### Update the geometry field with the calculated route shape ```bash -UPDATE
SET the_geom = (SELECT shape FROM carto_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'car', ARRAY['mode_type=shortest']::text[])) +UPDATE
SET the_geom = (SELECT shape FROM cdb_route_with_waypoints(Array['POINT(-3.7109 40.4234)'::GEOMETRY, 'POINT(-3.7059 40.4203)'::geometry, 'POINT(-3.7046 40.4180)'::geometry]::geometry[], 'car', ARRAY['mode_type=shortest']::text[])) ``` diff --git a/doc/segmentation_functions.md b/doc/segmentation_functions.md index b84e5bf..47b6d91 100644 --- a/doc/segmentation_functions.md +++ b/doc/segmentation_functions.md @@ -10,7 +10,7 @@ _**Note:** The Segmentation Snapshot functions are only available for the United Name | Description | Example Values --- | --- | --- -point geometry | A point geometry. You can use the helper function, `carto_LatLng` to quickly generate one from latitude and longitude | `carto_LatLng(40.760410,-73.964242)` +point geometry | A point geometry. You can use the helper function, `CDB_LatLng` to quickly generate one from latitude and longitude | `CDB_LatLng(40.760410,-73.964242)` ### Returns @@ -158,7 +158,7 @@ The possible segments are: ### Examples ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM OBS_GetSegmentSnapshot({{point geometry}}) ``` @@ -168,14 +168,14 @@ __Get the Segmentation Snapshot around the MGM Grand__ ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM -OBS_GetSegmentSnapshot(carto_LatLng(36.10222, -115.169516)) +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +OBS_GetSegmentSnapshot(CDB_LatLng(36.10222, -115.169516)) ``` -__Get the Segmentation Snapshot at Carto's NYC HQ__ +__Get the Segmentation Snapshot at CARTO's NYC HQ__ ```bash -https://{username}.carto.com/api/v2/sql?q=SELECT * FROM -OBS_GetSegmentSnapshot(carto_LatLng(40.704512, -73.936669)) +https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +OBS_GetSegmentSnapshot(CDB_LatLng(40.704512, -73.936669)) ``` From f46b2f68f40a2734313fa74007aa46467062bbc6 Mon Sep 17 00:00:00 2001 From: csobier Date: Mon, 4 Jul 2016 17:01:20 -0400 Subject: [PATCH 03/49] updated all api examples with rebranded account url- confirmed by luis --- doc/demographic_functions.md | 6 +++--- doc/geocoding_functions.md | 6 +++--- doc/isoline_functions.md | 2 +- doc/segmentation_functions.md | 6 +++--- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/demographic_functions.md b/doc/demographic_functions.md index 3e698bb..8df164d 100644 --- a/doc/demographic_functions.md +++ b/doc/demographic_functions.md @@ -40,7 +40,7 @@ obs_getdemographicsnapshot: { ### Examples ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetDemographicSnapshot({{point geometry}}) ``` @@ -49,14 +49,14 @@ OBS_GetDemographicSnapshot({{point geometry}}) __Get the Demographic Snapshot at Camp David__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetDemographicSnapshot(CDB_LatLng(39.648333, -77.465)) ``` __Get the Demographic Snapshot in the Upper West Side__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetDemographicSnapshot(CDB_LatLng(40.80, -73.960)) ``` diff --git a/doc/geocoding_functions.md b/doc/geocoding_functions.md index 36fe3ba..0450ff1 100644 --- a/doc/geocoding_functions.md +++ b/doc/geocoding_functions.md @@ -7,19 +7,19 @@ _**This service is subject to quota limitations and extra fees may apply**. View Here is an example of how to geocode a single country: ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=SELECT cdb_geocode_admin0_polygon('USA')&api_key={api_key} ``` In order to geocode an existent CARTO dataset, an SQL UPDATE statement must be used to populate the geometry column in the dataset with the results of the Data Services API. For example, if the column where you are storing the country names for each one of our rows is called `country_column`, run the following statement in order to geocode the dataset: ```bash -https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon('USA')&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon('USA')&api_key={api_key} ``` Notice that you can make use of Postgres or PostGIS functions in your Data Services API requests, as the result is a geometry that can be handled by the system. For example, suppose you need to retrieve the centroid of a specific country, you can wrap the resulting geometry from the geocoder functions inside the PostGIS `ST_Centroid` function: ```bash -https://{username}.cartodb.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=UPDATE {tablename} SET the_geom = ST_Centroid(cdb_geocode_admin0_polygon('USA'))&api_key={api_key} ``` diff --git a/doc/isoline_functions.md b/doc/isoline_functions.md index 63660f2..791f73d 100644 --- a/doc/isoline_functions.md +++ b/doc/isoline_functions.md @@ -7,7 +7,7 @@ _**This service is subject to quota limitations and extra fees may apply**. View You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table. ```bash -https://{username}.cartodb.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} +https://{username}.carto.com/api/v2/sql?q=INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isodistance('POINT(-3.70568 40.42028)'::geometry, 'car', ARRAY[300, 600, 900]::integer[])&api_key={api_key} ``` The following functions provide an isoline generator service, based on time or distance. This service uses the isolines service defined for your account. The default service limits the usage of displayed polygons represented on top of [HERE](https://developer.here.com/coverage-info) maps. diff --git a/doc/segmentation_functions.md b/doc/segmentation_functions.md index 47b6d91..7a92e00 100644 --- a/doc/segmentation_functions.md +++ b/doc/segmentation_functions.md @@ -158,7 +158,7 @@ The possible segments are: ### Examples ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetSegmentSnapshot({{point geometry}}) ``` @@ -168,7 +168,7 @@ __Get the Segmentation Snapshot around the MGM Grand__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetSegmentSnapshot(CDB_LatLng(36.10222, -115.169516)) ``` @@ -176,6 +176,6 @@ __Get the Segmentation Snapshot at CARTO's NYC HQ__ ```bash -https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM +https://{username}.carto.com/api/v2/sql?q=SELECT * FROM OBS_GetSegmentSnapshot(CDB_LatLng(40.704512, -73.936669)) ``` From d6a3d26cf7589c5d3f9b0525b728fa396a5e67fa Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Tue, 5 Jul 2016 10:22:55 +0200 Subject: [PATCH 04/49] Use new URLs for docs instead of docs subdomain --- doc/geocoding_functions.md | 6 +++--- doc/isoline_functions.md | 2 +- doc/overview.md | 8 ++++---- doc/segmentation_functions.md | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/doc/geocoding_functions.md b/doc/geocoding_functions.md index 0450ff1..bf96f19 100644 --- a/doc/geocoding_functions.md +++ b/doc/geocoding_functions.md @@ -2,7 +2,7 @@ The [geocoder](https://carto.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CARTO SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CARTO provides functions for several different categories of geocoding through the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ Here is an example of how to geocode a single country: @@ -217,7 +217,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('New York This function geocodes your data into point, or polygon, geometries for postal codes. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response. -**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://docs.carto.com/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. +**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://carto.com/docs/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. ### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_) @@ -315,7 +315,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_ipaddress_point('102.23.34 This function geocodes your data into a point geometry for a street address. CARTO uses several different service providers for street-level geocoding, depending on your platform. If you access CARTO on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. -**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. +**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. ### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_) diff --git a/doc/isoline_functions.md b/doc/isoline_functions.md index 791f73d..2ee6949 100644 --- a/doc/isoline_functions.md +++ b/doc/isoline_functions.md @@ -2,7 +2,7 @@ [Isolines](https://carto.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CARTO are available by requesting a single function in the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://docs.carto.com/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table. diff --git a/doc/overview.md b/doc/overview.md index f4830c8..effde63 100644 --- a/doc/overview.md +++ b/doc/overview.md @@ -2,7 +2,7 @@ By using CARTO libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. -**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://docs.carto.com/carto-engine/dataservices-api/quota-information/#quota-information). +**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-information). _The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CARTO Engine._ @@ -14,13 +14,13 @@ The Data Services API decouples the geocoding and isoline services from the CART The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/). -**Note:** The Data Services API [geocoding functions](http://docs.carto.com/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CARTO Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. +**Note:** The Data Services API [geocoding functions](http://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CARTO Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. ### Best Practices _Be mindful of the following usage notes when using the Data Services functions with the SQL API:_ -It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://docs.carto.com/carto-engine/dataservices-api/quota-information/#quota-consumption). +It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-consumption). The Data Services API is **recommended** to be used with INSERT or UPDATE operations, for applying location data to your tables. While SELECT (retrieve) is standard for SQL API requests, be mindful of quota consumption and use INSERT (to insert a new record) or UPDATE (to update an existing record), for best practices. @@ -40,7 +40,7 @@ Errors are described in the response of the request. An example is as follows: } ``` -Since the Data Services API is used on top of the CARTO SQL API, you can refer to the [Making calls to the SQL API](/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. +Since the Data Services API is used on top of the CARTO SQL API, you can refer to the [Making calls to the SQL API](http://carto.com/docs/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. If the requested information is not in the CARTO geocoding database, or if CARTO is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. diff --git a/doc/segmentation_functions.md b/doc/segmentation_functions.md index 7a92e00..f5cec4c 100644 --- a/doc/segmentation_functions.md +++ b/doc/segmentation_functions.md @@ -1,6 +1,6 @@ # Segmentation Functions -The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](/carto-engine/dataservices-api/geocoding-functions/) documentation. +The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](http://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/) documentation. _**Note:** The Segmentation Snapshot functions are only available for the United States. Our first release (May 18, 2016) is derived from Census 2010 variables. Our next release will be based on Census 2014 data. For the latest information, see the [Open Segments](https://github.com/CartoDB/open-segments) project repository._ From 3a43cf209450eb1e5323a4c811ecc63fe5ad2447 Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Tue, 5 Jul 2016 10:45:49 +0200 Subject: [PATCH 05/49] Use HTTPS URLs for docs --- doc/geocoding_functions.md | 6 +++--- doc/isoline_functions.md | 2 +- doc/overview.md | 8 ++++---- doc/segmentation_functions.md | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/doc/geocoding_functions.md b/doc/geocoding_functions.md index bf96f19..55fc56b 100644 --- a/doc/geocoding_functions.md +++ b/doc/geocoding_functions.md @@ -2,7 +2,7 @@ The [geocoder](https://carto.com/data/geocoder-api/) functions allow you to match your data with geometries on your map. This geocoding service can be used programatically to geocode datasets via the CARTO SQL API. It is fed from _Open Data_ and it serves geometries for countries, provinces, states, cities, postal codes, IP addresses and street addresses. CARTO provides functions for several different categories of geocoding through the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](https://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ Here is an example of how to geocode a single country: @@ -217,7 +217,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_namedplace_point('New York This function geocodes your data into point, or polygon, geometries for postal codes. The postal code polygon geocoder covers the United States, France, Australia and Canada; a request for a different country will return an empty response. -**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](http://carto.com/docs/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. +**Note:** For the USA, US Census [Zip Code Tabulation Areas](https://www.census.gov/geo/reference/zctas.html) (ZCTA) are used to reference geocodes for USPS postal codes service areas. See the [FAQs](https://carto.com/docs/faqs/datasets-and-data/#why-does-carto-use-census-bureau-zctas-and-not-usps-zip-codes-for-postal-codes) about datasets and data for details. ### cdb_geocode_postalcode_polygon(_postal_code text, country_name text_) @@ -315,7 +315,7 @@ INSERT INTO {tablename} (the_geom) SELECT cdb_geocode_ipaddress_point('102.23.34 This function geocodes your data into a point geometry for a street address. CARTO uses several different service providers for street-level geocoding, depending on your platform. If you access CARTO on a Google Cloud Platform, [Google Maps geocoding](https://developers.google.com/maps/documentation/geocoding/intro) is applied. All other platform users are provided with [HERE geocoding services](https://developer.here.com/rest-apis/documentation/geocoder/topics/quick-start.html). Additional service providers will be implemented in the future. -**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. +**This service is subject to quota limitations, and extra fees may apply**. View the [Quota information](https://carto.com/docs/carto-engine/dataservices-api/quota-information/) for details and recommendations about quota consumption. ### cdb_geocode_street_point(_search_text text, [city text], [state text], [country text]_) diff --git a/doc/isoline_functions.md b/doc/isoline_functions.md index 2ee6949..6cc47a4 100644 --- a/doc/isoline_functions.md +++ b/doc/isoline_functions.md @@ -2,7 +2,7 @@ [Isolines](https://carto.com/data/isolines/) are contoured lines that display equally calculated levels over a given surface area. This enables you to view polygon dimensions by forward or reverse measurements. Isoline functions are calculated as the intersection of areas from the origin point, measured by distance (isodistance) or time (isochrone). For example, the distance of a road from a sidewalk. Isoline services through CARTO are available by requesting a single function in the Data Services API. -_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](http://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ +_**This service is subject to quota limitations and extra fees may apply**. View the [Quota Information](https://carto.com/docs/carto-engine/dataservices-api/quota-information/) section for details and recommendations about to quota consumption._ You can use the isoline functions to retrieve, for example, isochrone lines from a certain location, specifying the mode and the ranges that will define each of the isolines. The following query calculates isolines for areas that are 5, 10 and 15 minutes (300, 600 and 900 seconds, respectively) away from the location by following a path defined by car routing and inserts them into a table. diff --git a/doc/overview.md b/doc/overview.md index effde63..b83fb57 100644 --- a/doc/overview.md +++ b/doc/overview.md @@ -2,7 +2,7 @@ By using CARTO libraries and the SQL API, you can apply location data services to your maps with unique data services functions. These functions are integrated with a number of internal and external services, enabling you to programatically customize subsets of data for your visualizations. These features are useful for geospatial analysis and the results can be saved, and stored, for additional location data service operations. -**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](http://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-information). +**Note:** Based on your account plan, some of these data services are subject to different [quota limitations](https://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-information). _The Data Services API is collaborating with [Mapzen](https://mapzen.com/), and several other geospatial service providers, in order to supply the best location data services from within our CARTO Engine._ @@ -14,13 +14,13 @@ The Data Services API decouples the geocoding and isoline services from the CART The geometries provided by this API are projected in the projection [WGS 84 SRID 4326](http://spatialreference.org/ref/epsg/wgs-84/). -**Note:** The Data Services API [geocoding functions](http://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CARTO Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. +**Note:** The Data Services API [geocoding functions](https://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/#geocoding-functions) return different types of geometries (points or polygons) as result of different geocoding processes. The CARTO Engine does not support multi-geometry layers or datasets, therefore you must confirm that you are using consistent geometry types inside a table, to avoid future conflicts in your map visualization. ### Best Practices _Be mindful of the following usage notes when using the Data Services functions with the SQL API:_ -It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](http://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-consumption). +It is discouraged to use the SELECT operation with the Data Services API functions in your map layers, as these type of queries consume quota when rendering tiles for your live map views. It may also result in sync performance issues, due to executing multiple requests to the API each time your map is viewed. See details about [Quota Consumption](https://carto.com/docs/carto-engine/dataservices-api/quota-information/#quota-consumption). The Data Services API is **recommended** to be used with INSERT or UPDATE operations, for applying location data to your tables. While SELECT (retrieve) is standard for SQL API requests, be mindful of quota consumption and use INSERT (to insert a new record) or UPDATE (to update an existing record), for best practices. @@ -40,7 +40,7 @@ Errors are described in the response of the request. An example is as follows: } ``` -Since the Data Services API is used on top of the CARTO SQL API, you can refer to the [Making calls to the SQL API](http://carto.com/docs/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. +Since the Data Services API is used on top of the CARTO SQL API, you can refer to the [Making calls to the SQL API](https://carto.com/docs/carto-engine/sql-api/making-calls/) documentation for help debugging your SQL errors. If the requested information is not in the CARTO geocoding database, or if CARTO is unable to recognize your input and match it with a result, the geocoding function returns `null` as a result. diff --git a/doc/segmentation_functions.md b/doc/segmentation_functions.md index f5cec4c..ec11c59 100644 --- a/doc/segmentation_functions.md +++ b/doc/segmentation_functions.md @@ -1,6 +1,6 @@ # Segmentation Functions -The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](http://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/) documentation. +The Segmentation Snapshot functions enable you to determine the pre-calculated population segment for a location. Segmentation is a method that divides a populations into subclassifications based on common traits. For example, you can take the a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, see the [Geocoding Functions](https://carto.com/docs/carto-engine/dataservices-api/geocoding-functions/) documentation. _**Note:** The Segmentation Snapshot functions are only available for the United States. Our first release (May 18, 2016) is derived from Census 2010 variables. Our next release will be based on Census 2014 data. For the latest information, see the [Open Segments](https://github.com/CartoDB/open-segments) project repository._ From 893b8db374877ad29b45dfbccfa00a18ea1b8bca Mon Sep 17 00:00:00 2001 From: Mario de Frutos Date: Fri, 3 Jun 2016 19:34:55 +0200 Subject: [PATCH 06/49] First stage is calculating the matrix of points --- .../cartodb_services/mapzen/__init__.py | 1 + .../cartodb_services/mapzen/isolines.py | 160 ++++++++++++++++++ 2 files changed, 161 insertions(+) create mode 100644 server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py index ee7552c..cc651c2 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py @@ -1,2 +1,3 @@ from routing import MapzenRouting, MapzenRoutingResponse +from isolines import MapzenIsolines from geocoder import MapzenGeocoder diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py new file mode 100644 index 0000000..2aaa585 --- /dev/null +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -0,0 +1,160 @@ +import requests +import json +import re + +from math import cos, sin, tan, sqrt, pi, radians, degrees, asin, atan2 +from exceptions import WrongParams, MalformedResult +from qps import qps_retry +from cartodb_services.tools import Coordinate, PolyLine + + +class MapzenIsolines: + + 'A Mapzen Isochrones feature using the mapzen distance matrix' + + MATRIX_API_URL = 'https://matrix.mapzen.com/one_to_many' + + ACCEPTED_MODES = { + "walk": "pedestrian", + "car": "auto", + } + + ACCEPTED_TYPES = ['distance', 'time'] + + AUTO_SHORTEST = 'auto_shortest' + + OPTIONAL_PARAMS = [ + 'mode_type', + ] + + METRICS_UNITS = 'kilometers' + IMPERIAL_UNITS = 'miles' + + EARTH_RADIUS_METERS = 6371000 + EARTH_RADIUS_MILES = 3959 + + DISTANCE_MULTIPLIER = [0.8, 0.9, 1, 1.10, 1.20] # From 80% to 120% of range + METERS_PER_SECOND = { + "walk": 1.38889, #Based on 5Km/h + "car": 22.3 #Based on 80Km/h + } + UNIT_MULTIPLIER = { + "kilometers": 1, + "miles": 0.3048 + } + + def __init__(self, app_key, base_url=MATRIX_API_URL): + self._app_key = app_key + self._url = base_url + + def calculate_isochrone(self, origin, mode, mode_range=[], units=METRICS_UNITS): + return self._calculate_isolines(origin, mode, 'time', mode_range, units) + + def calculate_isodistance(self, origin, mode, mode_range=[], units=METRICS_UNITS): + return self._calculate_isolines(origin, mode, 'distance', mode_range, units) + + def _calculate_isolines(self, origin, mode, mode_type, mode_range=[], units=METRICS_UNITS): + for r in mode_range: + radius = self._calculate_radius(r, mode, mode_type, units) + destination_points = self._calculate_destination_points(origin, radius) + destination_matrix = self._calculate_destination_matrix(origin, destination_points, mode, units) + + def _calculate_radius(self, init_radius, mode, mode_type, units): + if mode_type is 'time': + radius_meters = init_radius * self.METERS_PER_SECOND[mode] * self.UNIT_MULTIPLIER[units] + else: + radius_meters = init_radius + + return [init_radius*multiplier for multiplier in self.DISTANCE_MULTIPLIER] + + def _calculate_destination_points(self, origin, radius): + destinations = [] + angles = [i*36 for i in range(10)] + for angle in angles: + d = [self._calculate_destination_point(origin, r, angle) for r in radius] + destinations.extend(d) + return destinations + + def _calculate_destination_point(self, origin, radius, angle): + bearing = radians(angle) + origin_lat_radians = radians(origin.latitude) + origin_long_radians = radians(origin.longitude) + dest_lat_radians = asin(sin(origin_lat_radians) * cos(radius / self.EARTH_RADIUS_METERS) + cos(origin_lat_radians) * sin(radius / self.EARTH_RADIUS_METERS) * cos(bearing)) + dest_lng_radians = origin_long_radians + atan2(sin(bearing) * sin(radius / self.EARTH_RADIUS_METERS) * cos(origin_lat_radians), cos(radius / self.EARTH_RADIUS_METERS) - sin(origin_lat_radians) * sin(dest_lat_radians)) + + return Coordinate(degrees(dest_lng_radians), degrees(dest_lat_radians)) + + def _calculate_destination_matrix(self, origin, destination_points, mode, units): + json_request_params = self.__parse_json_parameters(destination_points, mode, units) + request_params = self.__parse_request_parameters(json_request_params) + response = requests.get(self._url, params=request_params) + import ipdb; ipdb.set_trace() # breakpoint 2b65ce71 // + if response.status_code == requests.codes.ok: + return self.__parse_routing_response(response.text) + elif response.status_code == requests.codes.bad_request: + return MapzenIsochronesResponse(None, None, None) + else: + response.raise_for_status() + + def __parse_request_parameters(self, json_request): + request_options = {"json": json_request} + request_options.update({'api_key': self._app_key}) + + return request_options + + def __parse_json_parameters(self, destination_points, mode, units): + import ipdb; ipdb.set_trace() # breakpoint 2b65ce71 // + json_options = {"locations": self._parse_destination_points(destination_points)} + json_options.update({'costing': self.ACCEPTED_MODES[mode]}) + #json_options.update({"directions_options": {'units': units, + # 'narrative': False}}) + + return json.dumps(json_options) + + def _parse_destination_points(self, destination_points): + destinations = [] + for dest in destination_points: + destinations.append({"lat": dest.latitude, "lon": dest.longitude}) + + return destinations + + + def __parse_matrix_response(self, response): + try: + parsed_json_response = json.loads(response) + except IndexError: + return [] + except KeyError: + raise MalformedResult() + + def __parse_mode_param(self, mode, options): + if mode in self.ACCEPTED_MODES: + mode_source = self.ACCEPTED_MODES[mode] + else: + raise WrongParams("{0} is not an accepted mode type".format(mode)) + + if mode == self.ACCEPTED_MODES['car'] and 'mode_type' in options and \ + options['mode_type'] == 'shortest': + mode = self.AUTO_SHORTEST + + return mode + + +class MapzenIsochronesResponse: + + def __init__(self, shape, length, duration): + self._shape = shape + self._length = length + self._duration = duration + + @property + def shape(self): + return self._shape + + @property + def length(self): + return self._length + + @property + def duration(self): + return self._duration From 40cacd99dc2fec21673009f05ef1a91a5b250c1c Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Mon, 4 Jul 2016 20:07:58 +0200 Subject: [PATCH 07/49] Some code trying to pull everything together (WIP) --- server/extension/sql/80_isolines_helper.sql | 68 +++++++++++++++++++++ server/extension/sql/90_isochrone.sql | 25 ++++++++ 2 files changed, 93 insertions(+) diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index b598cf6..f0e4e57 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -53,3 +53,71 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ finally: quota_service.increment_total_service_use() $$ LANGUAGE plpythonu SECURITY DEFINER; + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_routing_isolines( + username TEXT, + orgname TEXT, + isotype TEXT, + source geometry(Geometry, 4326), + mode TEXT, + data_range integer[], + options text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + import json + from cartodb_services.mapzen import MapzenIsolines + from cartodb_services.metrics import QuotaService + from cartodb_services.here.types import geo_polyline_to_multipolygon # TODO do we use the same types? + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)] + + # -- Check the quota + #quota_service = QuotaService(user_isolines_routing_config, redis_conn) + #if not quota_service.check_user_quota(): + # plpy.error('You have reached the limit of your quota') + + try: + # TODO: encapsulate or refactor this ugly code + mapzen_conf_str = plpy.execute("SELECT * FROM CDB_Conf_Getconf('mapzen_conf') AS mapzen_conf")[0]['mapzen_conf'] + mapzen_conf = json.loads(mapzen_conf_str) + + client = MapzenIsolines(mapzen_conf['routing']['api_key']) + + if source: + lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat'] + lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon'] + source_str = 'geo!%f,%f' % (lat, lon) + else: + source_str = None + + if isotype == 'isodistance': + resp = client.calculate_isodistance(source_str, mode, data_range, options) + elif isotype == 'isochrone': + resp = client.calculate_isochrone(source_str, mode, data_range, options) + + if resp: + result = [] + for isoline in resp: + data_range_n = isoline['range'] + polyline = isoline['geom'] + multipolygon = geo_polyline_to_multipolygon(polyline) + result.append([source, data_range_n, multipolygon]) + #quota_service.increment_success_service_use() + #quota_service.increment_isolines_service_use(len(resp)) + return result + else: + #quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + #quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to obtain isolines using mapzen: {0}'.format(e) + #plpy.notice(traceback.format_tb(traceback_)) + raise e + #plpy.error(error_msg) + finally: + pass + #quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu SECURITY DEFINER; diff --git a/server/extension/sql/90_isochrone.sql b/server/extension/sql/90_isochrone.sql index 6237bd7..a75ec67 100644 --- a/server/extension/sql/90_isochrone.sql +++ b/server/extension/sql/90_isochrone.sql @@ -19,3 +19,28 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ return isolines $$ LANGUAGE plpythonu; + + +-- mapzen isochrones +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)] + type = 'isochrone' + + # if we were to add a config check, it'll go here + #if user_isolines_config.google_services_user: + # plpy.error('This service is not available for google service users.') + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; From 53fe4ce21d5f55d4d2ce4a68a708b798b0dfd925 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 10:32:38 +0200 Subject: [PATCH 08/49] An attempt to adapt paremetrs (WIP) --- server/extension/sql/80_isolines_helper.sql | 7 ++++--- .../cartodb_services/cartodb_services/mapzen/isolines.py | 7 +++++++ 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index f0e4e57..43ff444 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -91,10 +91,11 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ else: source_str = None + # -- TODO Support options properly if isotype == 'isodistance': - resp = client.calculate_isodistance(source_str, mode, data_range, options) + resp = client.calculate_isodistance(source_str, mode, data_range) elif isotype == 'isochrone': - resp = client.calculate_isochrone(source_str, mode, data_range, options) + resp = client.calculate_isochrone(source_str, mode, data_range) if resp: result = [] @@ -114,7 +115,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ type_, value_, traceback_ = sys.exc_info() #quota_service.increment_failed_service_use() error_msg = 'There was an error trying to obtain isolines using mapzen: {0}'.format(e) - #plpy.notice(traceback.format_tb(traceback_)) + plpy.debug(traceback.format_tb(traceback_)) raise e #plpy.error(error_msg) finally: diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 2aaa585..1d2a814 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -60,7 +60,14 @@ class MapzenIsolines: destination_matrix = self._calculate_destination_matrix(origin, destination_points, mode, units) def _calculate_radius(self, init_radius, mode, mode_type, units): + import logging + #logging.basicConfig(filename='/tmp/isolines.log',level=logging.DEBUG) + logging.debug(mode_type) if mode_type is 'time': + logging.debug(init_radius) + logging.debug(self.METERS_PER_SECOND[mode]) + logging.debug("units = %s", units) + logging.debug(self.UNIT_MULTIPLIER[units]) radius_meters = init_radius * self.METERS_PER_SECOND[mode] * self.UNIT_MULTIPLIER[units] else: radius_meters = init_radius From a70560e566941445cce469b1821314b33fc8fa8f Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 11:12:15 +0200 Subject: [PATCH 09/49] Minimal Mapzen Time-Distance Matrix client --- .../cartodb_services/mapzen/__init__.py | 1 + .../cartodb_services/mapzen/matrix_client.py | 29 +++++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py index cc651c2..e3dca35 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/__init__.py @@ -1,3 +1,4 @@ from routing import MapzenRouting, MapzenRoutingResponse from isolines import MapzenIsolines from geocoder import MapzenGeocoder +from matrix_client import MatrixClient diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py new file mode 100644 index 0000000..0c2d1a8 --- /dev/null +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py @@ -0,0 +1,29 @@ +import requests +import json + +class MatrixClient: + + ONE_TO_MANY_URL = 'https://matrix.mapzen.com/one_to_many' + + def __init__(self, matrix_key): + self._matrix_key = matrix_key + + """Get distances and times to a set of locations. + See https://mapzen.com/documentation/matrix/api-reference/ + + Args: + locations Array of {lat: y, lon: x} + costing Costing model to use + + Returns: + A dict with one_to_many, units and locations + """ + def one_to_many(self, locations, costing): + request_params = { + 'json': json.dumps({'locations': locations}), + 'costing': costing, + 'api_key': self._matrix_key + } + response = requests.get(self.ONE_TO_MANY_URL, params=request_params) + + return response.json() From 96199b0d6dfea835d6bb23bc87060e5732ef4094 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 16:18:59 +0200 Subject: [PATCH 10/49] Add example to code doc --- .../cartodb_services/mapzen/matrix_client.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py index 0c2d1a8..de9b5cc 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py @@ -3,6 +3,18 @@ import json class MatrixClient: + """ + A minimal client for Mapzen Time-Distance Matrix Service + + Example: + + client = MatrixClient('your_api_key') + locations = [{"lat":40.744014,"lon":-73.990508},{"lat":40.739735,"lon":-73.979713},{"lat":40.752522,"lon":-73.985015},{"lat":40.750117,"lon":-73.983704},{"lat":40.750552,"lon":-73.993519}] + costing = 'pedestrian' + + print client.one_to_many(locations, costing) + """ + ONE_TO_MANY_URL = 'https://matrix.mapzen.com/one_to_many' def __init__(self, matrix_key): @@ -17,7 +29,7 @@ class MatrixClient: Returns: A dict with one_to_many, units and locations - """ + """ def one_to_many(self, locations, costing): request_params = { 'json': json.dumps({'locations': locations}), From 46971fe96f45db7aa5da7a0e8e0ac63f74d375f4 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 16:19:19 +0200 Subject: [PATCH 11/49] Raise error when response not OK --- .../cartodb_services/cartodb_services/mapzen/matrix_client.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py index de9b5cc..9b4df53 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/matrix_client.py @@ -38,4 +38,6 @@ class MatrixClient: } response = requests.get(self.ONE_TO_MANY_URL, params=request_params) + response.raise_for_status() # raise exception if not 200 OK + return response.json() From 87413255af97f71fb735c68387dcfbeb6da03cb5 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 16:19:54 +0200 Subject: [PATCH 12/49] Major rewrite of MapzenIsolines (WIP) --- .../cartodb_services/mapzen/isolines.py | 194 +++++------------- 1 file changed, 47 insertions(+), 147 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 1d2a814..76e5b96 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -1,167 +1,67 @@ -import requests -import json -import re - from math import cos, sin, tan, sqrt, pi, radians, degrees, asin, atan2 -from exceptions import WrongParams, MalformedResult -from qps import qps_retry -from cartodb_services.tools import Coordinate, PolyLine - class MapzenIsolines: - 'A Mapzen Isochrones feature using the mapzen distance matrix' - - MATRIX_API_URL = 'https://matrix.mapzen.com/one_to_many' - - ACCEPTED_MODES = { - "walk": "pedestrian", - "car": "auto", - } - - ACCEPTED_TYPES = ['distance', 'time'] - - AUTO_SHORTEST = 'auto_shortest' - - OPTIONAL_PARAMS = [ - 'mode_type', - ] - - METRICS_UNITS = 'kilometers' - IMPERIAL_UNITS = 'miles' + NUMBER_OF_ANGLES = 12 + MAX_ITERS = 5 + TOLERANCE = 0.1 EARTH_RADIUS_METERS = 6371000 - EARTH_RADIUS_MILES = 3959 - DISTANCE_MULTIPLIER = [0.8, 0.9, 1, 1.10, 1.20] # From 80% to 120% of range - METERS_PER_SECOND = { - "walk": 1.38889, #Based on 5Km/h - "car": 22.3 #Based on 80Km/h - } - UNIT_MULTIPLIER = { - "kilometers": 1, - "miles": 0.3048 - } + def __init__(self, matrix_client): + self._matrix_client = matrix_client - def __init__(self, app_key, base_url=MATRIX_API_URL): - self._app_key = app_key - self._url = base_url + """Get an isochrone using mapzen API. - def calculate_isochrone(self, origin, mode, mode_range=[], units=METRICS_UNITS): - return self._calculate_isolines(origin, mode, 'time', mode_range, units) + The implementation tries to sick close to the SQL API: + cdb_isochrone(source geometry, mode text, range integer[], [options text[]]) -> SETOF isoline - def calculate_isodistance(self, origin, mode, mode_range=[], units=METRICS_UNITS): - return self._calculate_isolines(origin, mode, 'distance', mode_range, units) + But this calculates just one isoline. - def _calculate_isolines(self, origin, mode, mode_type, mode_range=[], units=METRICS_UNITS): - for r in mode_range: - radius = self._calculate_radius(r, mode, mode_type, units) - destination_points = self._calculate_destination_points(origin, radius) - destination_matrix = self._calculate_destination_matrix(origin, destination_points, mode, units) + Args: + origin dict containing {lat: y, lon: x} + transport_mode string, for the moment just "car" or "walk" + isorange int range of the isoline in seconds - def _calculate_radius(self, init_radius, mode, mode_type, units): - import logging - #logging.basicConfig(filename='/tmp/isolines.log',level=logging.DEBUG) - logging.debug(mode_type) - if mode_type is 'time': - logging.debug(init_radius) - logging.debug(self.METERS_PER_SECOND[mode]) - logging.debug("units = %s", units) - logging.debug(self.UNIT_MULTIPLIER[units]) - radius_meters = init_radius * self.METERS_PER_SECOND[mode] * self.UNIT_MULTIPLIER[units] - else: - radius_meters = init_radius + Returns: + Array of {lon: x, lat: y} as a representation of the isoline + """ + def calculate_isochrone(self, origin, transport_mode, isorange): + if transport_mode != 'walk': + # TODO move this restriction to the appropriate place + raise NotImplementedError('walk is the only supported mode for the moment') - return [init_radius*multiplier for multiplier in self.DISTANCE_MULTIPLIER] + bearings = self._get_bearings(self.NUMBER_OF_ANGLES) + location_estimates = [self._get_dest_location_estimate(origin, b, isorange) for b in bearings] - def _calculate_destination_points(self, origin, radius): - destinations = [] - angles = [i*36 for i in range(10)] - for angle in angles: - d = [self._calculate_destination_point(origin, r, angle) for r in radius] - destinations.extend(d) - return destinations - - def _calculate_destination_point(self, origin, radius, angle): - bearing = radians(angle) - origin_lat_radians = radians(origin.latitude) - origin_long_radians = radians(origin.longitude) - dest_lat_radians = asin(sin(origin_lat_radians) * cos(radius / self.EARTH_RADIUS_METERS) + cos(origin_lat_radians) * sin(radius / self.EARTH_RADIUS_METERS) * cos(bearing)) - dest_lng_radians = origin_long_radians + atan2(sin(bearing) * sin(radius / self.EARTH_RADIUS_METERS) * cos(origin_lat_radians), cos(radius / self.EARTH_RADIUS_METERS) - sin(origin_lat_radians) * sin(dest_lat_radians)) - - return Coordinate(degrees(dest_lng_radians), degrees(dest_lat_radians)) - - def _calculate_destination_matrix(self, origin, destination_points, mode, units): - json_request_params = self.__parse_json_parameters(destination_points, mode, units) - request_params = self.__parse_request_parameters(json_request_params) - response = requests.get(self._url, params=request_params) - import ipdb; ipdb.set_trace() # breakpoint 2b65ce71 // - if response.status_code == requests.codes.ok: - return self.__parse_routing_response(response.text) - elif response.status_code == requests.codes.bad_request: - return MapzenIsochronesResponse(None, None, None) - else: - response.raise_for_status() - - def __parse_request_parameters(self, json_request): - request_options = {"json": json_request} - request_options.update({'api_key': self._app_key}) - - return request_options - - def __parse_json_parameters(self, destination_points, mode, units): - import ipdb; ipdb.set_trace() # breakpoint 2b65ce71 // - json_options = {"locations": self._parse_destination_points(destination_points)} - json_options.update({'costing': self.ACCEPTED_MODES[mode]}) - #json_options.update({"directions_options": {'units': units, - # 'narrative': False}}) - - return json.dumps(json_options) - - def _parse_destination_points(self, destination_points): - destinations = [] - for dest in destination_points: - destinations.append({"lat": dest.latitude, "lon": dest.longitude}) - - return destinations + # calculate the "actual" cost for each location estimate as first iteration + resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') + costs = resp['one_to_many'][0][1:] + #import pdb; pdb.set_trace() - def __parse_matrix_response(self, response): - try: - parsed_json_response = json.loads(response) - except IndexError: - return [] - except KeyError: - raise MalformedResult() + # NOTE: all angles in calculations are in radians + def _get_bearings(self, number_of_angles): + step = (2.0 * pi) / number_of_angles + return [(x * step) for x in xrange(0, number_of_angles)] - def __parse_mode_param(self, mode, options): - if mode in self.ACCEPTED_MODES: - mode_source = self.ACCEPTED_MODES[mode] - else: - raise WrongParams("{0} is not an accepted mode type".format(mode)) + # TODO: this just works for walk isochrone + # TODO: split this into two + def _get_dest_location_estimate(self, origin, bearing, trange): + # my rule of thumb: normal walk speed is about 1km in 10 minutes = 6 km/h + # use 12 km/h as an upper bound + speed = 3.333333 # in m/s + distance = speed * trange - if mode == self.ACCEPTED_MODES['car'] and 'mode_type' in options and \ - options['mode_type'] == 'shortest': - mode = self.AUTO_SHORTEST + return self._calculate_dest_location(origin, bearing, distance) - return mode + def _calculate_dest_location(self, origin, angle, radius): + origin_lat_radians = radians(origin['lat']) + origin_long_radians = radians(origin['lon']) + dest_lat_radians = asin(sin(origin_lat_radians) * cos(radius / self.EARTH_RADIUS_METERS) + cos(origin_lat_radians) * sin(radius / self.EARTH_RADIUS_METERS) * cos(angle)) + dest_lng_radians = origin_long_radians + atan2(sin(angle) * sin(radius / self.EARTH_RADIUS_METERS) * cos(origin_lat_radians), cos(radius / self.EARTH_RADIUS_METERS) - sin(origin_lat_radians) * sin(dest_lat_radians)) - -class MapzenIsochronesResponse: - - def __init__(self, shape, length, duration): - self._shape = shape - self._length = length - self._duration = duration - - @property - def shape(self): - return self._shape - - @property - def length(self): - return self._length - - @property - def duration(self): - return self._duration + return { + 'lon': degrees(dest_lng_radians), + 'lat': degrees(dest_lat_radians) + } From 9becf1adb460bf5e07f6eb168aa2ba7fa7a999b3 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 17:07:43 +0200 Subject: [PATCH 13/49] Iterative part of the algorithm (WIP) --- .../cartodb_services/mapzen/isolines.py | 48 +++++++++++++------ 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 76e5b96..5ecfa79 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -31,30 +31,50 @@ class MapzenIsolines: # TODO move this restriction to the appropriate place raise NotImplementedError('walk is the only supported mode for the moment') - bearings = self._get_bearings(self.NUMBER_OF_ANGLES) - location_estimates = [self._get_dest_location_estimate(origin, b, isorange) for b in bearings] + # Formally, a solution is an array of {angle, radius, lat, lon, cost} with cardinality NUMBER_OF_ANGLES + # we're looking for a solution in which abs(cost - isorange) / isorange <= TOLERANCE + + # Initial setup + angles = self._get_angles(self.NUMBER_OF_ANGLES) # array of angles + upper_rmax = 3.3333333 * isorange # an upper bound for the radius, assuming 12km/h walking speed + rmax = [upper_rmax] * self.NUMBER_OF_ANGLES + rmin = [0.0] * self.NUMBER_OF_ANGLES + location_estimates = [self._calculate_dest_location(origin, a, upper_rmax / 2.0) for a in angles] # calculate the "actual" cost for each location estimate as first iteration resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') - costs = resp['one_to_many'][0][1:] + costs = [c['time'] for c in resp['one_to_many'][0][1:]] #import pdb; pdb.set_trace() + # iterate to refine the first solution, if needed + for i in xrange(0, self.MAX_ITERS): + errors = [(cost - isorange) / float(isorange) for cost in costs] + max_abs_error = [abs(e) for e in errors] + if max_abs_error <= self.TOLERANCE: + # good enough, stop there + break + + # let's refine the solution, binary search + for j in xrange(0, self.NUMBER_OF_ANGLES): + if errors[j] > 0: + rmax[j] = (rmax[j] - rmin[j]) / 2.0 + else: + rmin[j] = (rmax[j] - rmin[j]) / 2.0 + location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]-rmin[j])/2.0) + + # and check "actual" costs again + resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') + costs = [c['time'] for c in resp['one_to_many'][0][1:]] + + return location_estimates + + # NOTE: all angles in calculations are in radians - def _get_bearings(self, number_of_angles): + def _get_angles(self, number_of_angles): step = (2.0 * pi) / number_of_angles return [(x * step) for x in xrange(0, number_of_angles)] - # TODO: this just works for walk isochrone - # TODO: split this into two - def _get_dest_location_estimate(self, origin, bearing, trange): - # my rule of thumb: normal walk speed is about 1km in 10 minutes = 6 km/h - # use 12 km/h as an upper bound - speed = 3.333333 # in m/s - distance = speed * trange - - return self._calculate_dest_location(origin, bearing, distance) - def _calculate_dest_location(self, origin, angle, radius): origin_lat_radians = radians(origin['lat']) origin_long_radians = radians(origin['lon']) From 9a9f35d9c25e48349e97ce16773cf3a5bf358cfc Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 17:41:15 +0200 Subject: [PATCH 14/49] Fix silly typos spotted by jgoizueta (WIP) --- .../cartodb_services/cartodb_services/mapzen/isolines.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 5ecfa79..0e89ef4 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -57,10 +57,10 @@ class MapzenIsolines: # let's refine the solution, binary search for j in xrange(0, self.NUMBER_OF_ANGLES): if errors[j] > 0: - rmax[j] = (rmax[j] - rmin[j]) / 2.0 + rmax[j] = (rmax[j] + rmin[j]) / 2.0 else: - rmin[j] = (rmax[j] - rmin[j]) / 2.0 - location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]-rmin[j])/2.0) + rmin[j] = (rmax[j] + rmin[j]) / 2.0 + location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) # and check "actual" costs again resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') From 2d95601c5a853605c2f4d10e1c724dd6281a1f91 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 18:35:40 +0200 Subject: [PATCH 15/49] Fix: max_abs_error should be a scalar --- .../python/cartodb_services/cartodb_services/mapzen/isolines.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 0e89ef4..f6f50aa 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -49,7 +49,7 @@ class MapzenIsolines: # iterate to refine the first solution, if needed for i in xrange(0, self.MAX_ITERS): errors = [(cost - isorange) / float(isorange) for cost in costs] - max_abs_error = [abs(e) for e in errors] + max_abs_error = max([abs(e) for e in errors]) if max_abs_error <= self.TOLERANCE: # good enough, stop there break From 77cdc3d8ffc81470968fafeb77fa2455bbffd1f1 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 18:36:28 +0200 Subject: [PATCH 16/49] Only refine individual solutions when error > TOLERANCE --- .../cartodb_services/mapzen/isolines.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index f6f50aa..2feb6ed 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -56,11 +56,13 @@ class MapzenIsolines: # let's refine the solution, binary search for j in xrange(0, self.NUMBER_OF_ANGLES): - if errors[j] > 0: - rmax[j] = (rmax[j] + rmin[j]) / 2.0 - else: - rmin[j] = (rmax[j] + rmin[j]) / 2.0 - location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) + + if abs(errors[j]) > self.TOLERANCE: + if errors[j] > 0: + rmax[j] = (rmax[j] + rmin[j]) / 2.0 + else: + rmin[j] = (rmax[j] + rmin[j]) / 2.0 + location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) # and check "actual" costs again resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') From b78bd057548219500639c4adb472547f3f198def Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 20:52:41 +0200 Subject: [PATCH 17/49] Be resilient to None cost estimation --- .../cartodb_services/mapzen/isolines.py | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 2feb6ed..31b7c01 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -1,4 +1,5 @@ from math import cos, sin, tan, sqrt, pi, radians, degrees, asin, atan2 +import logging class MapzenIsolines: @@ -27,6 +28,14 @@ class MapzenIsolines: Array of {lon: x, lat: y} as a representation of the isoline """ def calculate_isochrone(self, origin, transport_mode, isorange): + + # NOTE: not for production + #logging.basicConfig(level=logging.DEBUG, filename='/tmp/isolines.log') + logging.debug('origin = %s' % origin) + logging.debug('transport_mode = %s' % transport_mode) + logging.debug('isorange = %d' % isorange) + + if transport_mode != 'walk': # TODO move this restriction to the appropriate place raise NotImplementedError('walk is the only supported mode for the moment') @@ -42,12 +51,14 @@ class MapzenIsolines: location_estimates = [self._calculate_dest_location(origin, a, upper_rmax / 2.0) for a in angles] # calculate the "actual" cost for each location estimate as first iteration + # NOTE: sometimes it cannot calculate the cost and returns None. Just assume isorange and stop the calculations there resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') - costs = [c['time'] for c in resp['one_to_many'][0][1:]] + costs = [(c['time'] or isorange) for c in resp['one_to_many'][0][1:]] #import pdb; pdb.set_trace() # iterate to refine the first solution, if needed for i in xrange(0, self.MAX_ITERS): + logging.debug('costs = %s' % costs) errors = [(cost - isorange) / float(isorange) for cost in costs] max_abs_error = max([abs(e) for e in errors]) if max_abs_error <= self.TOLERANCE: @@ -66,7 +77,7 @@ class MapzenIsolines: # and check "actual" costs again resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') - costs = [c['time'] for c in resp['one_to_many'][0][1:]] + costs = [(c['time'] or isorange) for c in resp['one_to_many'][0][1:]] return location_estimates From 6810dc0ff0eae489b155eb12cb8c648b962214f2 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Tue, 5 Jul 2016 20:56:15 +0200 Subject: [PATCH 18/49] Code to glue together pg and python (WIP) --- server/extension/sql/80_isolines_helper.sql | 27 +++++++++++++++------ server/extension/sql/90_isochrone.sql | 2 +- 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index 43ff444..bcb30a5 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -55,7 +55,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ $$ LANGUAGE plpythonu SECURITY DEFINER; -CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_routing_isolines( +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_isolines( username TEXT, orgname TEXT, isotype TEXT, @@ -65,6 +65,7 @@ CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_routing_isolines( options text[]) RETURNS SETOF cdb_dataservices_server.isoline AS $$ import json + from cartodb_services.mapzen import MatrixClient from cartodb_services.mapzen import MapzenIsolines from cartodb_services.metrics import QuotaService from cartodb_services.here.types import geo_polyline_to_multipolygon # TODO do we use the same types? @@ -82,24 +83,34 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ mapzen_conf_str = plpy.execute("SELECT * FROM CDB_Conf_Getconf('mapzen_conf') AS mapzen_conf")[0]['mapzen_conf'] mapzen_conf = json.loads(mapzen_conf_str) - client = MapzenIsolines(mapzen_conf['routing']['api_key']) + + client = MatrixClient(mapzen_conf['matrix']['api_key']) + mapzen_isolines = MapzenIsolines(client) if source: lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat'] lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon'] - source_str = 'geo!%f,%f' % (lat, lon) + origin = {'lat': lat, 'lon': lon} else: - source_str = None + raise Exception('source is NULL') # -- TODO Support options properly + isolines = [] if isotype == 'isodistance': - resp = client.calculate_isodistance(source_str, mode, data_range) + # -- TODO implement + raise 'not implemented' + resp = mapzen_isolines.calculate_isodistance(origin, mode, data_range) elif isotype == 'isochrone': - resp = client.calculate_isochrone(source_str, mode, data_range) + for r in data_range: + isoline = mapzen_isolines.calculate_isochrone(origin, mode, r) + isolines.append(isoline) - if resp: + return [] # -- TODO delete this + + # -- TODO rewrite this block + if isolines: result = [] - for isoline in resp: + for isoline in isolines: data_range_n = isoline['range'] polyline = isoline['geom'] multipolygon = geo_polyline_to_multipolygon(polyline) diff --git a/server/extension/sql/90_isochrone.sql b/server/extension/sql/90_isochrone.sql index a75ec67..83af69b 100644 --- a/server/extension/sql/90_isochrone.sql +++ b/server/extension/sql/90_isochrone.sql @@ -34,7 +34,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ #if user_isolines_config.google_services_user: # plpy.error('This service is not available for google service users.') - mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) isolines = [] for element in result: From e9346faf42274319085f4e77f8c79b17bbb79f87 Mon Sep 17 00:00:00 2001 From: Carla Date: Wed, 6 Jul 2016 12:33:59 +0200 Subject: [PATCH 19/49] Fix bug on exception raise --- .../python/cartodb_services/cartodb_services/metrics/config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py index 340e5f6..d1b4a35 100644 --- a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py +++ b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py @@ -144,7 +144,7 @@ class MapzenGeocoderConfig(ServiceConfig): self._period_end_date = date_parse(self._redis_config[self.PERIOD_END_DATE]) self._cost_per_hit = 0 except Exception as e: - raise ConfigException("Malformed config for Mapzen geocoder: {1}".format(key, e)) + raise ConfigException("Malformed config for Mapzen geocoder: {0}".format(e)) @property def service_type(self): From eb906fae352e4d20c2a062f7739ac35e56493019 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 11:20:39 +0200 Subject: [PATCH 20/49] Convert to multipolygon and return isolines --- server/extension/sql/80_isolines_helper.sql | 34 ++++++++++----------- 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index bcb30a5..98b5091 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -79,7 +79,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ # plpy.error('You have reached the limit of your quota') try: - # TODO: encapsulate or refactor this ugly code + # --TODO: encapsulate or refactor this ugly code mapzen_conf_str = plpy.execute("SELECT * FROM CDB_Conf_Getconf('mapzen_conf') AS mapzen_conf")[0]['mapzen_conf'] mapzen_conf = json.loads(mapzen_conf_str) @@ -95,7 +95,7 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ raise Exception('source is NULL') # -- TODO Support options properly - isolines = [] + isolines = {} if isotype == 'isodistance': # -- TODO implement raise 'not implemented' @@ -103,24 +103,22 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ elif isotype == 'isochrone': for r in data_range: isoline = mapzen_isolines.calculate_isochrone(origin, mode, r) - isolines.append(isoline) + isolines[r] = (isoline) - return [] # -- TODO delete this + result = [] + for r in data_range: - # -- TODO rewrite this block - if isolines: - result = [] - for isoline in isolines: - data_range_n = isoline['range'] - polyline = isoline['geom'] - multipolygon = geo_polyline_to_multipolygon(polyline) - result.append([source, data_range_n, multipolygon]) - #quota_service.increment_success_service_use() - #quota_service.increment_isolines_service_use(len(resp)) - return result - else: - #quota_service.increment_empty_service_use() - return [] + # -- TODO encapsulate this block into a func/method + locations = isolines[r] + [ isolines[r][0] ] # close the polygon repeating the first point + wkt_coordinates = ','.join(["%f %f" % (l['lon'], l['lat']) for l in locations]) + sql = "SELECT ST_MPolyFromText('MULTIPOLYGON((({0})))', 4326) as geom".format(wkt_coordinates) + multipolygon = plpy.execute(sql, 1)[0]['geom'] + + result.append([source, r, multipolygon]) + # --TODO take care of this quota/usage stuff + #quota_service.increment_success_service_use() + #quota_service.increment_isolines_service_use(len(resp)) + return result except BaseException as e: import sys, traceback type_, value_, traceback_ = sys.exc_info() From 3dacb43a9a02dfa7f0e93e2d0040d8ec168d046e Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 12:10:43 +0200 Subject: [PATCH 21/49] Add cdb_mapzen_isochrone to client On behalf of @iriberri. --- client/renderer/interface.yaml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/client/renderer/interface.yaml b/client/renderer/interface.yaml index b2917ca..0f3eb6c 100644 --- a/client/renderer/interface.yaml +++ b/client/renderer/interface.yaml @@ -103,6 +103,16 @@ - { name: range, type: "integer[]" } - { name: options, type: "text[]", default: 'ARRAY[]::text[]' } +- name: cdb_mapzen_isochrone + return_type: SETOF cdb_dataservices_client.isoline + multi_row: true + multi_field: true + params: + - { name: source, type: "geometry(Geometry, 4326)" } + - { name: mode, type: text } + - { name: range, type: "integer[]" } + - { name: options, type: "text[]", default: 'ARRAY[]::text[]' } + - name: cdb_route_point_to_point return_type: cdb_dataservices_client.simple_route multi_field: true From cdcac2dc1f78922fee4bdf365ca643c90efec397 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 12:18:52 +0200 Subject: [PATCH 22/49] Fix typo in test case --- server/lib/python/cartodb_services/test/test_mapzengeocoder.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/test/test_mapzengeocoder.py b/server/lib/python/cartodb_services/test/test_mapzengeocoder.py index 0c7be54..7eee6c0 100644 --- a/server/lib/python/cartodb_services/test/test_mapzengeocoder.py +++ b/server/lib/python/cartodb_services/test/test_mapzengeocoder.py @@ -11,7 +11,7 @@ requests_mock.Mocker.TEST_PREFIX = 'test_' @requests_mock.Mocker() -class GoogleGeocoderTestCase(unittest.TestCase): +class MapzenGeocoderTestCase(unittest.TestCase): MAPZEN_GEOCODER_URL = 'https://search.mapzen.com/v1/search' EMPTY_RESPONSE = """{ From a046d3ce977cec08b7106fc266325f0c99d2700f Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Wed, 6 Jul 2016 12:40:31 +0200 Subject: [PATCH 23/49] Add Mapzen Matrix to config and metrics services --- server/extension/sql/15_config_helper.sql | 14 ++++++ server/extension/sql/80_isolines_helper.sql | 26 +++++------ server/extension/sql/90_isochrone.sql | 6 +-- .../cartodb_services/metrics/__init__.py | 2 +- .../cartodb_services/metrics/config.py | 45 +++++++++++++++++++ .../cartodb_services/metrics/quota.py | 3 ++ 6 files changed, 76 insertions(+), 20 deletions(-) diff --git a/server/extension/sql/15_config_helper.sql b/server/extension/sql/15_config_helper.sql index 3d6fd72..118bbec 100644 --- a/server/extension/sql/15_config_helper.sql +++ b/server/extension/sql/15_config_helper.sql @@ -26,6 +26,20 @@ RETURNS boolean AS $$ return True $$ LANGUAGE plpythonu SECURITY DEFINER; +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_mapzen_isolines_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_mapzen_isolines_routing_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import MapzenIsolinesRoutingConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + mapzen_isolines_config = MapzenIsolinesRoutingConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = mapzen_isolines_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_internal_geocoder_config(username text, orgname text) RETURNS boolean AS $$ cache_key = "user_internal_geocoder_config_{0}".format(username) diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index 98b5091..874d8d1 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -71,20 +71,15 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ from cartodb_services.here.types import geo_polyline_to_multipolygon # TODO do we use the same types? redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] - user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)] + user_mapzen_isolines_routing_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] # -- Check the quota - #quota_service = QuotaService(user_isolines_routing_config, redis_conn) - #if not quota_service.check_user_quota(): - # plpy.error('You have reached the limit of your quota') + quota_service = QuotaService(user_mapzen_isolines_routing_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') try: - # --TODO: encapsulate or refactor this ugly code - mapzen_conf_str = plpy.execute("SELECT * FROM CDB_Conf_Getconf('mapzen_conf') AS mapzen_conf")[0]['mapzen_conf'] - mapzen_conf = json.loads(mapzen_conf_str) - - - client = MatrixClient(mapzen_conf['matrix']['api_key']) + client = MatrixClient(user_mapzen_isolines_routing_config.mapzen_matrix_api_key) mapzen_isolines = MapzenIsolines(client) if source: @@ -115,19 +110,18 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ multipolygon = plpy.execute(sql, 1)[0]['geom'] result.append([source, r, multipolygon]) - # --TODO take care of this quota/usage stuff - #quota_service.increment_success_service_use() - #quota_service.increment_isolines_service_use(len(resp)) + + quota_service.increment_success_service_use() + quota_service.increment_isolines_service_use(len(isolines)) return result except BaseException as e: import sys, traceback type_, value_, traceback_ = sys.exc_info() - #quota_service.increment_failed_service_use() + quota_service.increment_failed_service_use() error_msg = 'There was an error trying to obtain isolines using mapzen: {0}'.format(e) plpy.debug(traceback.format_tb(traceback_)) raise e #plpy.error(error_msg) finally: - pass - #quota_service.increment_total_service_use() + quota_service.increment_total_service_use() $$ LANGUAGE plpythonu SECURITY DEFINER; diff --git a/server/extension/sql/90_isochrone.sql b/server/extension/sql/90_isochrone.sql index 83af69b..8e24c94 100644 --- a/server/extension/sql/90_isochrone.sql +++ b/server/extension/sql/90_isochrone.sql @@ -26,11 +26,11 @@ CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isochrone(username RETURNS SETOF cdb_dataservices_server.isoline AS $$ plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] - plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) - user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] type = 'isochrone' - # if we were to add a config check, it'll go here + # If we were to add a config check, it'll go here #if user_isolines_config.google_services_user: # plpy.error('This service is not available for google service users.') diff --git a/server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py b/server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py index 9bba57c..78d09e7 100644 --- a/server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py +++ b/server/lib/python/cartodb_services/cartodb_services/metrics/__init__.py @@ -1,3 +1,3 @@ -from config import GeocoderConfig, MapzenGeocoderConfig, IsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, ObservatorySnapshotConfig, ObservatoryConfig +from config import GeocoderConfig, MapzenGeocoderConfig, IsolinesRoutingConfig, MapzenIsolinesRoutingConfig, InternalGeocoderConfig, RoutingConfig, ConfigException, ObservatorySnapshotConfig, ObservatoryConfig from quota import QuotaService from user import UserMetricsService diff --git a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py index 340e5f6..e4d6597 100644 --- a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py +++ b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py @@ -241,6 +241,41 @@ class IsolinesRoutingConfig(ServiceConfig): return self._geocoder_type == self.GOOGLE_GEOCODER +class MapzenIsolinesRoutingConfig(ServiceConfig): + + PERIOD_END_DATE = 'period_end_date' + + def __init__(self, redis_connection, db_conn, username, orgname=None): + super(MapzenIsolinesRoutingConfig, self).__init__(redis_connection, db_conn, + username, orgname) + try: + self._mapzen_matrix_api_key = self._db_config.mapzen_matrix_api_key + self._isolines_quota = self._db_config.mapzen_matrix_monthly_quota + self._period_end_date = date_parse(self._redis_config[self.PERIOD_END_DATE]) + self._cost_per_hit = 0 + except Exception as e: + raise ConfigException("Malformed config for Mapzen isolines: {0}".format(e)) + + @property + def service_type(self): + return 'mapzen_isolines' + + @property + def isolines_quota(self): + return self._isolines_quota + + @property + def soft_isolines_limit(self): + return False + + @property + def period_end_date(self): + return self._period_end_date + + @property + def mapzen_matrix_api_key(self): + return self._mapzen_matrix_api_key + class InternalGeocoderConfig(ServiceConfig): def __init__(self, redis_connection, db_conn, username, orgname=None): @@ -438,6 +473,8 @@ class ServicesDBConfig: raise ConfigException('Mapzen configuration missing') else: mapzen_conf = json.loads(mapzen_conf_json) + self._mapzen_matrix_api_key = mapzen_conf['matrix']['api_key'] + self._mapzen_matrix_quota = mapzen_conf['matrix']['monthly_quota'] self._mapzen_routing_api_key = mapzen_conf['routing']['api_key'] self._mapzen_routing_quota = mapzen_conf['routing']['monthly_quota'] self._mapzen_geocoder_api_key = mapzen_conf['geocoder']['api_key'] @@ -492,6 +529,14 @@ class ServicesDBConfig: def heremaps_geocoder_cost_per_hit(self): return self._heremaps_geocoder_cost_per_hit + @property + def mapzen_matrix_api_key(self): + return self._mapzen_matrix_api_key + + @property + def mapzen_matrix_monthly_quota(self): + return self._mapzen_matrix_quota + @property def mapzen_routing_api_key(self): return self._mapzen_routing_api_key diff --git a/server/lib/python/cartodb_services/cartodb_services/metrics/quota.py b/server/lib/python/cartodb_services/cartodb_services/metrics/quota.py index c627769..4496e3c 100644 --- a/server/lib/python/cartodb_services/cartodb_services/metrics/quota.py +++ b/server/lib/python/cartodb_services/cartodb_services/metrics/quota.py @@ -70,6 +70,9 @@ class QuotaChecker: elif re.match('here_isolines', self._user_service_config.service_type) is not None: return self.__check_isolines_quota() + elif re.match('mapzen_isolines', + self._user_service_config.service_type) is not None: + return self.__check_isolines_quota() elif re.match('routing_mapzen', self._user_service_config.service_type) is not None: return self.__check_routing_quota() From 2147d190a12b1791249c070db83ccd728a21ea46 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 13:13:10 +0200 Subject: [PATCH 24/49] Unit test for the mapzen isolines --- .../python/cartodb_services/test/__init__.py | 0 .../test/test_mapzenisolines.py | 76 +++++++++++++++++++ 2 files changed, 76 insertions(+) create mode 100644 server/lib/python/cartodb_services/test/__init__.py create mode 100644 server/lib/python/cartodb_services/test/test_mapzenisolines.py diff --git a/server/lib/python/cartodb_services/test/__init__.py b/server/lib/python/cartodb_services/test/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/server/lib/python/cartodb_services/test/test_mapzenisolines.py b/server/lib/python/cartodb_services/test/test_mapzenisolines.py new file mode 100644 index 0000000..69f5597 --- /dev/null +++ b/server/lib/python/cartodb_services/test/test_mapzenisolines.py @@ -0,0 +1,76 @@ +import unittest +from cartodb_services.mapzen import MapzenIsolines +from math import radians, cos, sin, asin, sqrt + +""" +This file is basically a sanity test on the algorithm. + + +It uses a mocked client, which returns the cost based on a very simple model: +just proportional to the distance from origin to the target point. +""" + +class MatrixClientMock(): + + def __init__(self, speed): + """ + Sets up the mock with a speed in km/h + """ + self._speed = speed + + def one_to_many(self, locations, costing): + origin = locations[0] + distances = [self._distance(origin, l) for l in locations] + response = { + 'one_to_many': [ + [ + { + 'distance': distances[i] * self._speed, + 'time': distances[i] / self._speed * 3600, + 'to_index': i, + 'from_index': 0 + } + for i in xrange(0, len(distances)) + ] + ], + 'units': 'km', + 'locations': [ + locations + ] + } + return response + + def _distance(self, a, b): + """ + Calculate the great circle distance between two points + on the earth (specified in decimal degrees) + http://stackoverflow.com/questions/4913349/haversine-formula-in-python-bearing-and-distance-between-two-gps-points + + Returns: + distance in meters + """ + + # convert decimal degrees to radians + lon1, lat1, lon2, lat2 = map(radians, [a['lon'], a['lat'], b['lon'], b['lat']]) + + # haversine formula + dlon = lon2 - lon1 + dlat = lat2 - lat1 + a = sin(dlat/2)**2 + cos(lat1) * cos(lat2) * sin(dlon/2)**2 + c = 2 * asin(sqrt(a)) + r = 6371 # Radius of earth in kilometers. Use 3956 for miles + return c * r + + +class MapzenIsolinesTestCase(unittest.TestCase): + + def setUp(self): + speed = 6 # in km/h + matrix_client = MatrixClientMock(speed) + self.mapzen_isolines = MapzenIsolines(matrix_client) + + def test_calculate_isochrone(self): + origin = {"lat":40.744014,"lon":-73.990508} + transport_mode = 'walk' + isorange = 10 * 60 # 10 minutes + solution = self.mapzen_isolines.calculate_isochrone(origin, transport_mode, isorange) From ff4eb5b3480b6d6d8fd6dedbf6c57ccd2dd1149c Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Wed, 6 Jul 2016 14:21:32 +0200 Subject: [PATCH 25/49] Mock mapzen matrix config --- server/lib/python/cartodb_services/test/test_helper.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/test/test_helper.py b/server/lib/python/cartodb_services/test/test_helper.py index b0ffaf0..fe6fd8f 100644 --- a/server/lib/python/cartodb_services/test/test_helper.py +++ b/server/lib/python/cartodb_services/test/test_helper.py @@ -65,7 +65,7 @@ def _plpy_execute_side_effect(*args, **kwargs): if args[0] == "SELECT cartodb.CDB_Conf_GetConf('heremaps_conf') as conf": return [{'conf': '{"geocoder": {"app_id": "app_id", "app_code": "code", "geocoder_cost_per_hit": 1}, "isolines": {"app_id": "app_id", "app_code": "code"}}'}] elif args[0] == "SELECT cartodb.CDB_Conf_GetConf('mapzen_conf') as conf": - return [{'conf': '{"routing": {"api_key": "api_key_rou", "monthly_quota": 1500000}, "geocoder": {"api_key": "api_key_geo", "monthly_quota": 1500000}}'}] + return [{'conf': '{"routing": {"api_key": "api_key_rou", "monthly_quota": 1500000}, "geocoder": {"api_key": "api_key_geo", "monthly_quota": 1500000}, "matrix": {"api_key": "api_key_mat", "monthly_quota": 1500000}}'}] elif args[0] == "SELECT cartodb.CDB_Conf_GetConf('logger_conf') as conf": return [{'conf': '{"geocoder_log_path": "/dev/null"}'}] elif args[0] == "SELECT cartodb.CDB_Conf_GetConf('data_observatory_conf') as conf": From 7ddb3da60d5567c3e52f27cfe65e187e411bbea9 Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Wed, 6 Jul 2016 15:43:26 +0200 Subject: [PATCH 26/49] Remove useless cost_per_hit line --- .../python/cartodb_services/cartodb_services/metrics/config.py | 1 - 1 file changed, 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py index e4d6597..a5b80fa 100644 --- a/server/lib/python/cartodb_services/cartodb_services/metrics/config.py +++ b/server/lib/python/cartodb_services/cartodb_services/metrics/config.py @@ -252,7 +252,6 @@ class MapzenIsolinesRoutingConfig(ServiceConfig): self._mapzen_matrix_api_key = self._db_config.mapzen_matrix_api_key self._isolines_quota = self._db_config.mapzen_matrix_monthly_quota self._period_end_date = date_parse(self._redis_config[self.PERIOD_END_DATE]) - self._cost_per_hit = 0 except Exception as e: raise ConfigException("Malformed config for Mapzen isolines: {0}".format(e)) From 6c4829df015278fced84f86fda0c1fd5d0a63d1e Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 16:04:41 +0200 Subject: [PATCH 27/49] Small refactor for sanity --- .../cartodb_services/mapzen/isolines.py | 22 +++++++++---------- .../test/test_mapzenisolines.py | 2 +- 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 31b7c01..9a75075 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -31,6 +31,7 @@ class MapzenIsolines: # NOTE: not for production #logging.basicConfig(level=logging.DEBUG, filename='/tmp/isolines.log') + #logging.basicConfig(level=logging.DEBUG) logging.debug('origin = %s' % origin) logging.debug('transport_mode = %s' % transport_mode) logging.debug('isorange = %d' % isorange) @@ -50,15 +51,15 @@ class MapzenIsolines: rmin = [0.0] * self.NUMBER_OF_ANGLES location_estimates = [self._calculate_dest_location(origin, a, upper_rmax / 2.0) for a in angles] - # calculate the "actual" cost for each location estimate as first iteration - # NOTE: sometimes it cannot calculate the cost and returns None. Just assume isorange and stop the calculations there - resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') - costs = [(c['time'] or isorange) for c in resp['one_to_many'][0][1:]] - #import pdb; pdb.set_trace() - - # iterate to refine the first solution, if needed + # Iterate to refine the first solution for i in xrange(0, self.MAX_ITERS): - logging.debug('costs = %s' % costs) + # Calculate the "actual" cost for each location estimate. + # NOTE: sometimes it cannot calculate the cost and returns None. + # Just assume isorange and stop the calculations there + response = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') + costs = [(c['time'] or isorange) for c in response['one_to_many'][0][1:]] + logging.debug('i = %d, costs = %s' % (i, costs)) + errors = [(cost - isorange) / float(isorange) for cost in costs] max_abs_error = max([abs(e) for e in errors]) if max_abs_error <= self.TOLERANCE: @@ -73,11 +74,8 @@ class MapzenIsolines: rmax[j] = (rmax[j] + rmin[j]) / 2.0 else: rmin[j] = (rmax[j] + rmin[j]) / 2.0 - location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) - # and check "actual" costs again - resp = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') - costs = [(c['time'] or isorange) for c in resp['one_to_many'][0][1:]] + location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) return location_estimates diff --git a/server/lib/python/cartodb_services/test/test_mapzenisolines.py b/server/lib/python/cartodb_services/test/test_mapzenisolines.py index 69f5597..4fb4e96 100644 --- a/server/lib/python/cartodb_services/test/test_mapzenisolines.py +++ b/server/lib/python/cartodb_services/test/test_mapzenisolines.py @@ -65,7 +65,7 @@ class MatrixClientMock(): class MapzenIsolinesTestCase(unittest.TestCase): def setUp(self): - speed = 6 # in km/h + speed = 4 # in km/h matrix_client = MatrixClientMock(speed) self.mapzen_isolines = MapzenIsolines(matrix_client) From 075edf0e0d595d44deb3d485ffc7ef18cbf326f7 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 18:01:33 +0200 Subject: [PATCH 28/49] More precission for earth's radius --- .../python/cartodb_services/cartodb_services/mapzen/isolines.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 9a75075..95294ad 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -7,7 +7,7 @@ class MapzenIsolines: MAX_ITERS = 5 TOLERANCE = 0.1 - EARTH_RADIUS_METERS = 6371000 + EARTH_RADIUS_METERS = 6367444 def __init__(self, matrix_client): self._matrix_client = matrix_client From 6d888a7a625fcf5b9b24761afdb973cef4d8147b Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 18:03:20 +0200 Subject: [PATCH 29/49] Fix for points getting None cost Sometimes there's no route information for the point in a particular angle we're interested in. In this case it is better to use more points/angles and discard the ones we're not interested in. --- .../cartodb_services/mapzen/isolines.py | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 95294ad..a78cf4a 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -3,7 +3,7 @@ import logging class MapzenIsolines: - NUMBER_OF_ANGLES = 12 + NUMBER_OF_ANGLES = 24 MAX_ITERS = 5 TOLERANCE = 0.1 @@ -77,7 +77,13 @@ class MapzenIsolines: location_estimates[j] = self._calculate_dest_location(origin, angles[j], (rmax[j]+rmin[j])/2.0) - return location_estimates + # delete points that got None + location_estimates_filtered = [] + for i, c in enumerate(costs): + if c <> isorange: + location_estimates_filtered.append(location_estimates[i]) + + return location_estimates_filtered From 54221fa67185892d3421304c6ed0406dc3873b19 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 18:05:51 +0200 Subject: [PATCH 30/49] Add transport mode car --- .../cartodb_services/mapzen/isolines.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index a78cf4a..63aa2b3 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -36,17 +36,21 @@ class MapzenIsolines: logging.debug('transport_mode = %s' % transport_mode) logging.debug('isorange = %d' % isorange) + if transport_mode == 'walk': + upper_rmax = 3.3333333 * isorange # an upper bound for the radius, assuming 12km/h walking speed + costing_model = 'pedestrian' + elif transport_mode == 'car': + upper_rmax = 41.67 * isorange # assuming 140km/h max speed + costing_model = 'auto' + else: + raise NotImplementedError('car and walk are the only supported modes for the moment') - if transport_mode != 'walk': - # TODO move this restriction to the appropriate place - raise NotImplementedError('walk is the only supported mode for the moment') # Formally, a solution is an array of {angle, radius, lat, lon, cost} with cardinality NUMBER_OF_ANGLES # we're looking for a solution in which abs(cost - isorange) / isorange <= TOLERANCE # Initial setup angles = self._get_angles(self.NUMBER_OF_ANGLES) # array of angles - upper_rmax = 3.3333333 * isorange # an upper bound for the radius, assuming 12km/h walking speed rmax = [upper_rmax] * self.NUMBER_OF_ANGLES rmin = [0.0] * self.NUMBER_OF_ANGLES location_estimates = [self._calculate_dest_location(origin, a, upper_rmax / 2.0) for a in angles] @@ -56,8 +60,10 @@ class MapzenIsolines: # Calculate the "actual" cost for each location estimate. # NOTE: sometimes it cannot calculate the cost and returns None. # Just assume isorange and stop the calculations there - response = self._matrix_client.one_to_many([origin] + location_estimates, 'pedestrian') + + response = self._matrix_client.one_to_many([origin] + location_estimates, costing_model) costs = [(c['time'] or isorange) for c in response['one_to_many'][0][1:]] + logging.debug('i = %d, costs = %s' % (i, costs)) errors = [(cost - isorange) / float(isorange) for cost in costs] From 523eda2cc797dbe97f25b170071becd78a4d1555 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 18:43:09 +0200 Subject: [PATCH 31/49] Generalize calculate_isochrone to calculate_isoline --- .../cartodb_services/mapzen/isolines.py | 47 ++++++++++++++----- 1 file changed, 34 insertions(+), 13 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 63aa2b3..6ad7464 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -27,25 +27,46 @@ class MapzenIsolines: Returns: Array of {lon: x, lat: y} as a representation of the isoline """ - def calculate_isochrone(self, origin, transport_mode, isorange): + def calculate_isochrone(self, origin, transport_mode, time_range): + if transport_mode == 'walk': + max_speed = 3.3333333 # In m/s, assuming 12km/h walking speed + costing_model = 'pedestrian' + elif transport_mode == 'car': + max_speed = 41.67 # In m/s, assuming 140km/h max speed + costing_model = 'auto' + else: + raise NotImplementedError('car and walk are the only supported modes for the moment') + + upper_rmax = max_speed * time_range # an upper bound for the radius + + return self.calculate_isoline(origin, costing_model, time_range, upper_rmax, 'time') + + """Get an isoline using mapzen API. + + The implementation tries to sick close to the SQL API: + cdb_isochrone(source geometry, mode text, range integer[], [options text[]]) -> SETOF isoline + + But this calculates just one isoline. + + Args: + origin dict containing {lat: y, lon: x} + costing_model string "auto" or "pedestrian" + isorange int Range of the isoline in seconds + upper_rmax float An upper bound for the binary search + cost_variable string Variable to optimize "time" or "distance" + + Returns: + Array of {lon: x, lat: y} as a representation of the isoline + """ + def calculate_isoline(self, origin, costing_model, isorange, upper_rmax, cost_variable): # NOTE: not for production #logging.basicConfig(level=logging.DEBUG, filename='/tmp/isolines.log') #logging.basicConfig(level=logging.DEBUG) logging.debug('origin = %s' % origin) - logging.debug('transport_mode = %s' % transport_mode) + logging.debug('costing_model = %s' % costing_model) logging.debug('isorange = %d' % isorange) - if transport_mode == 'walk': - upper_rmax = 3.3333333 * isorange # an upper bound for the radius, assuming 12km/h walking speed - costing_model = 'pedestrian' - elif transport_mode == 'car': - upper_rmax = 41.67 * isorange # assuming 140km/h max speed - costing_model = 'auto' - else: - raise NotImplementedError('car and walk are the only supported modes for the moment') - - # Formally, a solution is an array of {angle, radius, lat, lon, cost} with cardinality NUMBER_OF_ANGLES # we're looking for a solution in which abs(cost - isorange) / isorange <= TOLERANCE @@ -62,7 +83,7 @@ class MapzenIsolines: # Just assume isorange and stop the calculations there response = self._matrix_client.one_to_many([origin] + location_estimates, costing_model) - costs = [(c['time'] or isorange) for c in response['one_to_many'][0][1:]] + costs = [(c[cost_variable] or isorange) for c in response['one_to_many'][0][1:]] logging.debug('i = %d, costs = %s' % (i, costs)) From 230112b7e5657ccd0d2e540625a63d0682fb70ef Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 19:20:21 +0200 Subject: [PATCH 32/49] Add calculate_isodistance function --- .../cartodb_services/mapzen/isolines.py | 28 +++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index 6ad7464..cd521c0 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -41,6 +41,29 @@ class MapzenIsolines: return self.calculate_isoline(origin, costing_model, time_range, upper_rmax, 'time') + """Get an isodistance using mapzen API. + + Args: + origin dict containing {lat: y, lon: x} + transport_mode string, for the moment just "car" or "walk" + isorange int range of the isoline in seconds + + Returns: + Array of {lon: x, lat: y} as a representation of the isoline + """ + def calculate_isodistance(self, origin, transport_mode, distance_range): + if transport_mode == 'walk': + costing_model = 'pedestrian' + elif transport_mode == 'car': + costing_model = 'auto' + else: + raise NotImplementedError('car and walk are the only supported modes for the moment') + + upper_rmax = distance_range # an upper bound for the radius, going in a straight line + + return self.calculate_isoline(origin, costing_model, time_range, upper_rmax, 'distance', 1000.0) + + """Get an isoline using mapzen API. The implementation tries to sick close to the SQL API: @@ -54,11 +77,12 @@ class MapzenIsolines: isorange int Range of the isoline in seconds upper_rmax float An upper bound for the binary search cost_variable string Variable to optimize "time" or "distance" + unit_factor float A factor to adapt units of isorange (meters) and units of distance (km) Returns: Array of {lon: x, lat: y} as a representation of the isoline """ - def calculate_isoline(self, origin, costing_model, isorange, upper_rmax, cost_variable): + def calculate_isoline(self, origin, costing_model, isorange, upper_rmax, cost_variable, unit_factor=1.0): # NOTE: not for production #logging.basicConfig(level=logging.DEBUG, filename='/tmp/isolines.log') @@ -83,7 +107,7 @@ class MapzenIsolines: # Just assume isorange and stop the calculations there response = self._matrix_client.one_to_many([origin] + location_estimates, costing_model) - costs = [(c[cost_variable] or isorange) for c in response['one_to_many'][0][1:]] + costs = [(c[cost_variable]*unit_factor or isorange) for c in response['one_to_many'][0][1:]] logging.debug('i = %d, costs = %s' % (i, costs)) From 99798f26180f7a3c4220e4ff5151372cffc4c230 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 19:40:40 +0200 Subject: [PATCH 33/49] Integrate isodistance into SQL API --- client/renderer/interface.yaml | 10 ++++++++++ server/extension/sql/80_isolines_helper.sql | 7 +++---- server/extension/sql/85_isodistance.sql | 15 +++++++++++++++ server/extension/sql/90_isochrone.sql | 11 +---------- 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/client/renderer/interface.yaml b/client/renderer/interface.yaml index 0f3eb6c..10b4ef2 100644 --- a/client/renderer/interface.yaml +++ b/client/renderer/interface.yaml @@ -113,6 +113,16 @@ - { name: range, type: "integer[]" } - { name: options, type: "text[]", default: 'ARRAY[]::text[]' } +- name: cdb_mapzen_distance + return_type: SETOF cdb_dataservices_client.isoline + multi_row: true + multi_field: true + params: + - { name: source, type: "geometry(Geometry, 4326)" } + - { name: mode, type: text } + - { name: range, type: "integer[]" } + - { name: options, type: "text[]", default: 'ARRAY[]::text[]' } + - name: cdb_route_point_to_point return_type: cdb_dataservices_client.simple_route multi_field: true diff --git a/server/extension/sql/80_isolines_helper.sql b/server/extension/sql/80_isolines_helper.sql index 874d8d1..f04e62c 100644 --- a/server/extension/sql/80_isolines_helper.sql +++ b/server/extension/sql/80_isolines_helper.sql @@ -68,7 +68,6 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ from cartodb_services.mapzen import MatrixClient from cartodb_services.mapzen import MapzenIsolines from cartodb_services.metrics import QuotaService - from cartodb_services.here.types import geo_polyline_to_multipolygon # TODO do we use the same types? redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] user_mapzen_isolines_routing_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] @@ -92,9 +91,9 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ # -- TODO Support options properly isolines = {} if isotype == 'isodistance': - # -- TODO implement - raise 'not implemented' - resp = mapzen_isolines.calculate_isodistance(origin, mode, data_range) + for r in data_range: + isoline = mapzen_isolines.calculate_isodistance(origin, mode, r) + isolines[r] = (isoline) elif isotype == 'isochrone': for r in data_range: isoline = mapzen_isolines.calculate_isochrone(origin, mode, r) diff --git a/server/extension/sql/85_isodistance.sql b/server/extension/sql/85_isodistance.sql index 3c3ac0e..29beddb 100644 --- a/server/extension/sql/85_isodistance.sql +++ b/server/extension/sql/85_isodistance.sql @@ -19,3 +19,18 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ return isolines $$ LANGUAGE plpythonu; + +-- mapzen isodistance +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + type = 'isodistance' + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + + return result +$$ LANGUAGE plpythonu; diff --git a/server/extension/sql/90_isochrone.sql b/server/extension/sql/90_isochrone.sql index 8e24c94..409cf39 100644 --- a/server/extension/sql/90_isochrone.sql +++ b/server/extension/sql/90_isochrone.sql @@ -30,17 +30,8 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] type = 'isochrone' - # If we were to add a config check, it'll go here - #if user_isolines_config.google_services_user: - # plpy.error('This service is not available for google service users.') - mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) - isolines = [] - for element in result: - isoline = element['isoline'] - isoline = isoline.translate(None, "()").split(',') - isolines.append(isoline) - return isolines + return result $$ LANGUAGE plpythonu; From bcc6bc35d3973ccfdaa80c7859cdddfd63a4a085 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 19:48:20 +0200 Subject: [PATCH 34/49] Fix None*unit_factor error Also make the code more explicit about what happens when getting cost == None. --- .../cartodb_services/cartodb_services/mapzen/isolines.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index cd521c0..d686407 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -107,7 +107,12 @@ class MapzenIsolines: # Just assume isorange and stop the calculations there response = self._matrix_client.one_to_many([origin] + location_estimates, costing_model) - costs = [(c[cost_variable]*unit_factor or isorange) for c in response['one_to_many'][0][1:]] + costs = [None] * self.NUMBER_OF_ANGLES + for idx, c in enumerate(response['one_to_many'][0][1:]): + if c[cost_variable]: + costs[idx] = c[cost_variable]*unit_factor + else: + costs[idx] = isorange logging.debug('i = %d, costs = %s' % (i, costs)) From 9b7a2d491f2b3ada4a065f139b64f389ed488c1f Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 19:58:04 +0200 Subject: [PATCH 35/49] Fix bug adapting types passing through plpython --- server/extension/sql/85_isodistance.sql | 7 ++++++- server/extension/sql/90_isochrone.sql | 7 ++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/server/extension/sql/85_isodistance.sql b/server/extension/sql/85_isodistance.sql index 29beddb..6349015 100644 --- a/server/extension/sql/85_isodistance.sql +++ b/server/extension/sql/85_isodistance.sql @@ -31,6 +31,11 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) - return result + return isolines $$ LANGUAGE plpythonu; diff --git a/server/extension/sql/90_isochrone.sql b/server/extension/sql/90_isochrone.sql index 409cf39..11f11cb 100644 --- a/server/extension/sql/90_isochrone.sql +++ b/server/extension/sql/90_isochrone.sql @@ -32,6 +32,11 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') #--TODO what is this for? + isolines.append(isoline) - return result + return isolines $$ LANGUAGE plpythonu; From 9c87762b8bfeaf22f7a1c39c6e51a41a11c088f8 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 20:03:17 +0200 Subject: [PATCH 36/49] Fix typo in client interface --- client/renderer/interface.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/client/renderer/interface.yaml b/client/renderer/interface.yaml index 10b4ef2..97c7675 100644 --- a/client/renderer/interface.yaml +++ b/client/renderer/interface.yaml @@ -113,7 +113,7 @@ - { name: range, type: "integer[]" } - { name: options, type: "text[]", default: 'ARRAY[]::text[]' } -- name: cdb_mapzen_distance +- name: cdb_mapzen_isodistance return_type: SETOF cdb_dataservices_client.isoline multi_row: true multi_field: true From f5d51da673e1facba35c5bd31a0f26e97d23e827 Mon Sep 17 00:00:00 2001 From: Rafa de la Torre Date: Wed, 6 Jul 2016 20:05:37 +0200 Subject: [PATCH 37/49] Fix another typo (hello Carla!!) This feature is dedicated to you. Keep rocking. --- .../python/cartodb_services/cartodb_services/mapzen/isolines.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py index d686407..de75459 100644 --- a/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py +++ b/server/lib/python/cartodb_services/cartodb_services/mapzen/isolines.py @@ -61,7 +61,7 @@ class MapzenIsolines: upper_rmax = distance_range # an upper bound for the radius, going in a straight line - return self.calculate_isoline(origin, costing_model, time_range, upper_rmax, 'distance', 1000.0) + return self.calculate_isoline(origin, costing_model, distance_range, upper_rmax, 'distance', 1000.0) """Get an isoline using mapzen API. From 442a9a8433e13af5781828ff1f01d9eed3ee085c Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 09:51:52 +0200 Subject: [PATCH 38/49] Bump client version to 0.9.0 --- client/Makefile | 4 +- .../cdb_dataservices_client--0.8.0--0.9.0.sql | 78 + .../cdb_dataservices_client--0.9.0--0.8.0.sql | 7 + client/cdb_dataservices_client--0.9.0.sql | 1478 +++++++++++++++++ client/cdb_dataservices_client.control | 2 +- .../cdb_dataservices_client--0.7.0--0.8.0.sql | 0 .../cdb_dataservices_client--0.8.0--0.7.0.sql | 0 .../cdb_dataservices_client--0.8.0.sql | 0 8 files changed, 1566 insertions(+), 3 deletions(-) create mode 100644 client/cdb_dataservices_client--0.8.0--0.9.0.sql create mode 100644 client/cdb_dataservices_client--0.9.0--0.8.0.sql create mode 100644 client/cdb_dataservices_client--0.9.0.sql rename client/{ => old_versions}/cdb_dataservices_client--0.7.0--0.8.0.sql (100%) rename client/{ => old_versions}/cdb_dataservices_client--0.8.0--0.7.0.sql (100%) rename client/{ => old_versions}/cdb_dataservices_client--0.8.0.sql (100%) diff --git a/client/Makefile b/client/Makefile index e3d3d8a..b0ca79d 100644 --- a/client/Makefile +++ b/client/Makefile @@ -13,8 +13,8 @@ OLD_VERSIONS = $(wildcard old_versions/*.sql) # @see http://www.postgresql.org/docs/current/static/extend-pgxs.html DATA = $(NEW_EXTENSION_ARTIFACT) \ $(OLD_VERSIONS) \ - cdb_dataservices_client--0.7.0--0.8.0.sql \ - cdb_dataservices_client--0.8.0--0.7.0.sql + cdb_dataservices_client--0.8.0--0.9.0.sql \ + cdb_dataservices_client--0.9.0--0.8.0.sql REGRESS = $(notdir $(basename $(wildcard test/sql/*test.sql))) diff --git a/client/cdb_dataservices_client--0.8.0--0.9.0.sql b/client/cdb_dataservices_client--0.8.0--0.9.0.sql new file mode 100644 index 0000000..515ae97 --- /dev/null +++ b/client/cdb_dataservices_client--0.8.0--0.9.0.sql @@ -0,0 +1,78 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "ALTER EXTENSION cdb_dataservices_client UPDATE TO '0.9.0'" to load this file. \quit + + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_mapzen_isochrone (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_mapzen_isochrone(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_mapzen_isodistance (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_mapzen_isodistance(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + + + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_mapzen_isochrone (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_mapzen_isochrone (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_mapzen_isodistance (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_mapzen_isodistance (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; \ No newline at end of file diff --git a/client/cdb_dataservices_client--0.9.0--0.8.0.sql b/client/cdb_dataservices_client--0.9.0--0.8.0.sql new file mode 100644 index 0000000..ceeebac --- /dev/null +++ b/client/cdb_dataservices_client--0.9.0--0.8.0.sql @@ -0,0 +1,7 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "ALTER EXTENSION cdb_dataservices_client UPDATE TO '0.8.0'" to load this file. \quit +DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_mapzen_isochrone(geometry(Geometry, 4326), text, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_mapzen_isodistance(geometry(Geometry, 4326), text, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isochrone(text, text, geometry(Geometry, 4326), text, integer[], text); +DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isodistance(text, text, geometry(Geometry, 4326), text, integer[], text); diff --git a/client/cdb_dataservices_client--0.9.0.sql b/client/cdb_dataservices_client--0.9.0.sql new file mode 100644 index 0000000..c885002 --- /dev/null +++ b/client/cdb_dataservices_client--0.9.0.sql @@ -0,0 +1,1478 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "CREATE EXTENSION cdb_dataservices_client" to load this file. \quit +-- +-- Geocoder server connection config +-- +-- The purpose of this function is provide to the PL/Proxy functions +-- the connection string needed to connect with the server + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._server_conn_str() +RETURNS text AS $$ +DECLARE + db_connection_str text; +BEGIN + SELECT cartodb.cdb_conf_getconf('geocoder_server_config')->'connection_str' INTO db_connection_str; + SELECT trim(both '"' FROM db_connection_str) INTO db_connection_str; + RETURN db_connection_str; +END; +$$ LANGUAGE 'plpgsql';CREATE TYPE cdb_dataservices_client._entity_config AS ( + username text, + organization_name text +); + +-- +-- Get entity config function +-- +-- The purpose of this function is to retrieve the username and organization name from +-- a) schema where he/her is the owner in case is an organization user +-- b) entity_name from the cdb_conf database in case is a non organization user +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_entity_config() +RETURNS record AS $$ +DECLARE + result cdb_dataservices_client._entity_config; + is_organization boolean; + username text; + organization_name text; +BEGIN + SELECT cartodb.cdb_conf_getconf('user_config')->'is_organization' INTO is_organization; + IF is_organization IS NULL THEN + RAISE EXCEPTION 'User must have user configuration in the config table'; + ELSIF is_organization = TRUE THEN + SELECT nspname + FROM pg_namespace s + LEFT JOIN pg_roles r ON s.nspowner = r.oid + WHERE r.rolname = session_user INTO username; + SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO organization_name; + ELSE + SELECT cartodb.cdb_conf_getconf('user_config')->>'entity_name' INTO username; + organization_name = NULL; + END IF; + result.username = username; + result.organization_name = organization_name; + RETURN result; +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER;CREATE TYPE cdb_dataservices_client.isoline AS ( + center geometry(Geometry,4326), + data_range integer, + the_geom geometry(Multipolygon,4326) +); + +CREATE TYPE cdb_dataservices_client.simple_route AS ( + shape geometry(LineString,4326), + length real, + duration integer +);-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin0_polygon (country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_admin0_polygon(username, orgname, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon (admin1_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon (admin1_name text, country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_admin1_polygon(username, orgname, admin1_name, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text, country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point (city_name text, admin1_name text, country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_namedplace_point(username, orgname, city_name, admin1_name, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_polygon (postal_code text, country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_postalcode_polygon(username, orgname, postal_code, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_point (postal_code text, country_name text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_postalcode_point(username, orgname, postal_code, country_name) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_ipaddress_point (ip_address text) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_ipaddress_point(username, orgname, ip_address) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_geocode_street_point (searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_geocode_street_point(username, orgname, searchtext, city, state_province, country) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_here_geocode_street_point (searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_here_geocode_street_point(username, orgname, searchtext, city, state_province, country) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_google_geocode_street_point (searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_google_geocode_street_point(username, orgname, searchtext, city, state_province, country) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_mapzen_geocode_street_point (searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._cdb_mapzen_geocode_street_point(username, orgname, searchtext, city, state_province, country) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isodistance (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_isodistance(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_isochrone (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_isochrone(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_mapzen_isochrone (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_mapzen_isochrone(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_mapzen_isodistance (source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._cdb_mapzen_isodistance(username, orgname, source, mode, range, options); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_route_point_to_point (origin geometry(Point, 4326), destination geometry(Point, 4326), mode text, options text[] DEFAULT ARRAY[]::text[], units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_client.simple_route AS $$ +DECLARE + ret cdb_dataservices_client.simple_route; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT * FROM cdb_dataservices_client._cdb_route_point_to_point(username, orgname, origin, destination, mode, options, units) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.cdb_route_with_waypoints (waypoints geometry(Point, 4326)[], mode text, options text[] DEFAULT ARRAY[]::text[], units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_client.simple_route AS $$ +DECLARE + ret cdb_dataservices_client.simple_route; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT * FROM cdb_dataservices_client._cdb_route_with_waypoints(username, orgname, waypoints, mode, options, units) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_get_demographic_snapshot (geom geometry(Geometry, 4326), time_span text DEFAULT '2009 - 2013'::text, geometry_level text DEFAULT '"us.census.tiger".block_group'::text) +RETURNS json AS $$ +DECLARE + ret json; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_get_demographic_snapshot(username, orgname, geom, time_span, geometry_level) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_get_segment_snapshot (geom geometry(Geometry, 4326), geometry_level text DEFAULT '"us.census.tiger".census_tract'::text) +RETURNS json AS $$ +DECLARE + ret json; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_get_segment_snapshot(username, orgname, geom, geometry_level) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getdemographicsnapshot (geom geometry(Geometry, 4326), time_span text DEFAULT NULL, geometry_level text DEFAULT NULL) +RETURNS SETOF JSON AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getdemographicsnapshot(username, orgname, geom, time_span, geometry_level); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getsegmentsnapshot (geom geometry(Geometry, 4326), geometry_level text DEFAULT NULL) +RETURNS SETOF JSON AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getsegmentsnapshot(username, orgname, geom, geometry_level); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getboundary (geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getboundary(username, orgname, geom, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getboundaryid (geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL) +RETURNS text AS $$ +DECLARE + ret text; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getboundaryid(username, orgname, geom, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getboundarybyid (geometry_id text, boundary_id text, time_span text DEFAULT NULL) +RETURNS Geometry AS $$ +DECLARE + ret Geometry; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getboundarybyid(username, orgname, geometry_id, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getboundariesbygeometry (geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getboundariesbygeometry(username, orgname, geom, boundary_id, time_span, overlap_type); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getboundariesbypointandradius (geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getboundariesbypointandradius(username, orgname, geom, radius, boundary_id, time_span, overlap_type); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getpointsbygeometry (geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getpointsbygeometry(username, orgname, geom, boundary_id, time_span, overlap_type); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getpointsbypointandradius (geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getpointsbypointandradius(username, orgname, geom, radius, boundary_id, time_span, overlap_type); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getmeasure (geom Geometry, measure_id text, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ +DECLARE + ret numeric; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getmeasure(username, orgname, geom, measure_id, normalize, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getmeasurebyid (geom_ref text, measure_id text, boundary_id text, time_span text DEFAULT NULL) +RETURNS numeric AS $$ +DECLARE + ret numeric; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getmeasurebyid(username, orgname, geom_ref, measure_id, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getcategory (geom Geometry, category_id text, boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS text AS $$ +DECLARE + ret text; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getcategory(username, orgname, geom, category_id, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getuscensusmeasure (geom Geometry, name text, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ +DECLARE + ret numeric; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getuscensusmeasure(username, orgname, geom, name, normalize, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getuscensuscategory (geom Geometry, name text, boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS text AS $$ +DECLARE + ret text; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getuscensuscategory(username, orgname, geom, name, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getpopulation (geom Geometry, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ +DECLARE + ret numeric; + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + SELECT cdb_dataservices_client._obs_getpopulation(username, orgname, geom, normalize, boundary_id, time_span) INTO ret; + RETURN ret; + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_search (search_term text, relevant_boundary text DEFAULT NULL) +RETURNS TABLE(id text, description text, name text, aggregate text, source text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_search(username, orgname, search_term, relevant_boundary); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +-- +-- Public dataservices API function +-- +-- These are the only ones with permissions to publicuser role +-- and should also be the only ones with SECURITY DEFINER + +CREATE OR REPLACE FUNCTION cdb_dataservices_client.obs_getavailableboundaries (geom Geometry, timespan text DEFAULT NULL) +RETURNS TABLE(boundary_id text, description text, time_span text, tablename text) AS $$ +DECLARE + + username text; + orgname text; +BEGIN + IF session_user = 'publicuser' OR session_user ~ 'cartodb_publicuser_*' THEN + RAISE EXCEPTION 'The api_key must be provided'; + END IF; + SELECT u, o INTO username, orgname FROM cdb_dataservices_client._cdb_entity_config() AS (u text, o text); + -- JSON value stored "" is taken as literal + IF username IS NULL OR username = '' OR username = '""' THEN + RAISE EXCEPTION 'Username is a mandatory argument, check it out'; + END IF; + + RETURN QUERY + SELECT * FROM cdb_dataservices_client._obs_getavailableboundaries(username, orgname, geom, timespan); + +END; +$$ LANGUAGE 'plpgsql' SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin0_polygon (username text, organization_name text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_admin0_polygon (username, organization_name, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin1_polygon (username text, organization_name text, admin1_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon (username, organization_name, admin1_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_admin1_polygon (username text, organization_name text, admin1_name text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_admin1_polygon (username, organization_name, admin1_name, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_namedplace_point (username text, organization_name text, city_name text, admin1_name text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_namedplace_point (username, organization_name, city_name, admin1_name, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_postalcode_polygon (username text, organization_name text, postal_code text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_postalcode_polygon (username, organization_name, postal_code, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_postalcode_point (username text, organization_name text, postal_code text, country_name text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_postalcode_point (username, organization_name, postal_code, country_name); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_ipaddress_point (username text, organization_name text, ip_address text) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_ipaddress_point (username, organization_name, ip_address); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_geocode_street_point (username text, organization_name text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_geocode_street_point (username, organization_name, searchtext, city, state_province, country); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_here_geocode_street_point (username text, organization_name text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_here_geocode_street_point (username, organization_name, searchtext, city, state_province, country); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_google_geocode_street_point (username text, organization_name text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_google_geocode_street_point (username, organization_name, searchtext, city, state_province, country); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_mapzen_geocode_street_point (username text, organization_name text, searchtext text, city text DEFAULT NULL, state_province text DEFAULT NULL, country text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.cdb_mapzen_geocode_street_point (username, organization_name, searchtext, city, state_province, country); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isodistance (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_isodistance (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_isochrone (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_isochrone (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_mapzen_isochrone (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_mapzen_isochrone (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_mapzen_isodistance (username text, organization_name text, source geometry(Geometry, 4326), mode text, range integer[], options text[] DEFAULT ARRAY[]::text[]) +RETURNS SETOF cdb_dataservices_client.isoline AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_mapzen_isodistance (username, organization_name, source, mode, range, options); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_route_point_to_point (username text, organization_name text, origin geometry(Point, 4326), destination geometry(Point, 4326), mode text, options text[] DEFAULT ARRAY[]::text[], units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_client.simple_route AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_route_point_to_point (username, organization_name, origin, destination, mode, options, units); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._cdb_route_with_waypoints (username text, organization_name text, waypoints geometry(Point, 4326)[], mode text, options text[] DEFAULT ARRAY[]::text[], units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_client.simple_route AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.cdb_route_with_waypoints (username, organization_name, waypoints, mode, options, units); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_get_demographic_snapshot (username text, organization_name text, geom geometry(Geometry, 4326), time_span text DEFAULT '2009 - 2013'::text, geometry_level text DEFAULT '"us.census.tiger".block_group'::text) +RETURNS json AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_get_demographic_snapshot (username, organization_name, geom, time_span, geometry_level); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_get_segment_snapshot (username text, organization_name text, geom geometry(Geometry, 4326), geometry_level text DEFAULT '"us.census.tiger".census_tract'::text) +RETURNS json AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_get_segment_snapshot (username, organization_name, geom, geometry_level); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getdemographicsnapshot (username text, organization_name text, geom geometry(Geometry, 4326), time_span text DEFAULT NULL, geometry_level text DEFAULT NULL) +RETURNS SETOF JSON AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getdemographicsnapshot (username, organization_name, geom, time_span, geometry_level); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getsegmentsnapshot (username text, organization_name text, geom geometry(Geometry, 4326), geometry_level text DEFAULT NULL) +RETURNS SETOF JSON AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getsegmentsnapshot (username, organization_name, geom, geometry_level); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getboundary (username text, organization_name text, geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getboundary (username, organization_name, geom, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getboundaryid (username text, organization_name text, geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL) +RETURNS text AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getboundaryid (username, organization_name, geom, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getboundarybyid (username text, organization_name text, geometry_id text, boundary_id text, time_span text DEFAULT NULL) +RETURNS Geometry AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getboundarybyid (username, organization_name, geometry_id, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getboundariesbygeometry (username text, organization_name text, geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_getboundariesbygeometry (username, organization_name, geom, boundary_id, time_span, overlap_type); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getboundariesbypointandradius (username text, organization_name text, geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_getboundariesbypointandradius (username, organization_name, geom, radius, boundary_id, time_span, overlap_type); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getpointsbygeometry (username text, organization_name text, geom geometry(Geometry, 4326), boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_getpointsbygeometry (username, organization_name, geom, boundary_id, time_span, overlap_type); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getpointsbypointandradius (username text, organization_name text, geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text DEFAULT NULL, overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_getpointsbypointandradius (username, organization_name, geom, radius, boundary_id, time_span, overlap_type); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getmeasure (username text, organization_name text, geom Geometry, measure_id text, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getmeasure (username, organization_name, geom, measure_id, normalize, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getmeasurebyid (username text, organization_name text, geom_ref text, measure_id text, boundary_id text, time_span text DEFAULT NULL) +RETURNS numeric AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getmeasurebyid (username, organization_name, geom_ref, measure_id, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getcategory (username text, organization_name text, geom Geometry, category_id text, boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS text AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getcategory (username, organization_name, geom, category_id, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getuscensusmeasure (username text, organization_name text, geom Geometry, name text, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getuscensusmeasure (username, organization_name, geom, name, normalize, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getuscensuscategory (username text, organization_name text, geom Geometry, name text, boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS text AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getuscensuscategory (username, organization_name, geom, name, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getpopulation (username text, organization_name text, geom Geometry, normalize text DEFAULT 'area', boundary_id text DEFAULT NULL, time_span text DEFAULT NULL) +RETURNS numeric AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT cdb_dataservices_server.obs_getpopulation (username, organization_name, geom, normalize, boundary_id, time_span); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_search (username text, organization_name text, search_term text, relevant_boundary text DEFAULT NULL) +RETURNS TABLE(id text, description text, name text, aggregate text, source text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_search (username, organization_name, search_term, relevant_boundary); + +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_client._obs_getavailableboundaries (username text, organization_name text, geom Geometry, timespan text DEFAULT NULL) +RETURNS TABLE(boundary_id text, description text, time_span text, tablename text) AS $$ + CONNECT cdb_dataservices_client._server_conn_str(); + + SELECT * FROM cdb_dataservices_server.obs_getavailableboundaries (username, organization_name, geom, timespan); + +$$ LANGUAGE plproxy; + +-- Make sure by default there are no permissions for publicuser +-- NOTE: this happens at extension creation time, as part of an implicit transaction. +REVOKE ALL PRIVILEGES ON SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser CASCADE; + +-- Grant permissions on the schema to publicuser (but just the schema) +GRANT USAGE ON SCHEMA cdb_dataservices_client TO publicuser; + +-- Revoke execute permissions on all functions in the schema by default +REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_client FROM PUBLIC, publicuser;GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin0_polygon(country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon(admin1_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_admin1_polygon(admin1_name text, country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text, country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_polygon(postal_code text, country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_postalcode_point(postal_code text, country_name text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_ipaddress_point(ip_address text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_here_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_google_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isodistance(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_isochrone(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_isochrone(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_isodistance(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_route_point_to_point(origin geometry(Point, 4326), destination geometry(Point, 4326), mode text, options text[], units text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_route_with_waypoints(waypoints geometry(Point, 4326)[], mode text, options text[], units text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_get_demographic_snapshot(geom geometry(Geometry, 4326), time_span text, geometry_level text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_get_segment_snapshot(geom geometry(Geometry, 4326), geometry_level text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getdemographicsnapshot(geom geometry(Geometry, 4326), time_span text, geometry_level text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getsegmentsnapshot(geom geometry(Geometry, 4326), geometry_level text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getboundary(geom geometry(Geometry, 4326), boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getboundaryid(geom geometry(Geometry, 4326), boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getboundarybyid(geometry_id text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getboundariesbygeometry(geom geometry(Geometry, 4326), boundary_id text, time_span text, overlap_type text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getboundariesbypointandradius(geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text, overlap_type text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getpointsbygeometry(geom geometry(Geometry, 4326), boundary_id text, time_span text, overlap_type text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getpointsbypointandradius(geom geometry(Geometry, 4326), radius numeric, boundary_id text, time_span text, overlap_type text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getmeasure(geom Geometry, measure_id text, normalize text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getmeasurebyid(geom_ref text, measure_id text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getcategory(geom Geometry, category_id text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getuscensusmeasure(geom Geometry, name text, normalize text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getuscensuscategory(geom Geometry, name text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getpopulation(geom Geometry, normalize text, boundary_id text, time_span text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_search(search_term text, relevant_boundary text) TO publicuser; +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.obs_getavailableboundaries(geom Geometry, timespan text) TO publicuser; diff --git a/client/cdb_dataservices_client.control b/client/cdb_dataservices_client.control index a594f0a..1289a30 100644 --- a/client/cdb_dataservices_client.control +++ b/client/cdb_dataservices_client.control @@ -1,5 +1,5 @@ comment = 'CartoDB dataservices client API extension' -default_version = '0.8.0' +default_version = '0.9.0' requires = 'plproxy, cartodb' superuser = true schema = cdb_dataservices_client diff --git a/client/cdb_dataservices_client--0.7.0--0.8.0.sql b/client/old_versions/cdb_dataservices_client--0.7.0--0.8.0.sql similarity index 100% rename from client/cdb_dataservices_client--0.7.0--0.8.0.sql rename to client/old_versions/cdb_dataservices_client--0.7.0--0.8.0.sql diff --git a/client/cdb_dataservices_client--0.8.0--0.7.0.sql b/client/old_versions/cdb_dataservices_client--0.8.0--0.7.0.sql similarity index 100% rename from client/cdb_dataservices_client--0.8.0--0.7.0.sql rename to client/old_versions/cdb_dataservices_client--0.8.0--0.7.0.sql diff --git a/client/cdb_dataservices_client--0.8.0.sql b/client/old_versions/cdb_dataservices_client--0.8.0.sql similarity index 100% rename from client/cdb_dataservices_client--0.8.0.sql rename to client/old_versions/cdb_dataservices_client--0.8.0.sql From 102209730095fbcf606899247700dccc16f37e2c Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 09:58:51 +0200 Subject: [PATCH 39/49] Bump server to 0.12.0 --- ...db_dataservices_server--0.11.0--0.12.0.sql | 74 + ...db_dataservices_server--0.12.0--0.11.0.sql | 7 + .../cdb_dataservices_server--0.12.0.sql | 2260 +++++++++++++++++ .../extension/cdb_dataservices_server.control | 2 +- ...db_dataservices_server--0.10.0--0.11.0.sql | 0 ...db_dataservices_server--0.11.0--0.10.0.sql | 0 .../cdb_dataservices_server--0.11.0.sql | 0 server/lib/python/cartodb_services/setup.py | 2 +- 8 files changed, 2343 insertions(+), 2 deletions(-) create mode 100644 server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql create mode 100644 server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql create mode 100644 server/extension/cdb_dataservices_server--0.12.0.sql rename server/extension/{ => old_versions}/cdb_dataservices_server--0.10.0--0.11.0.sql (100%) rename server/extension/{ => old_versions}/cdb_dataservices_server--0.11.0--0.10.0.sql (100%) rename server/extension/{ => old_versions}/cdb_dataservices_server--0.11.0.sql (100%) diff --git a/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql new file mode 100644 index 0000000..d9eddf1 --- /dev/null +++ b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql @@ -0,0 +1,74 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.12.0'" to load this file. \quit + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_isolines( + username TEXT, + orgname TEXT, + isotype TEXT, + source geometry(Geometry, 4326), + mode TEXT, + data_range integer[], + options text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + import json + from cartodb_services.mapzen import MatrixClient + from cartodb_services.mapzen import MapzenIsolines + from cartodb_services.metrics import QuotaService + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_mapzen_isolines_routing_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + + # -- Check the quota + quota_service = QuotaService(user_mapzen_isolines_routing_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + client = MatrixClient(user_mapzen_isolines_routing_config.mapzen_matrix_api_key) + mapzen_isolines = MapzenIsolines(client) + + if source: + lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat'] + lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon'] + origin = {'lat': lat, 'lon': lon} + else: + raise Exception('source is NULL') + + # -- TODO Support options properly + isolines = {} + if isotype == 'isodistance': + for r in data_range: + isoline = mapzen_isolines.calculate_isodistance(origin, mode, r) + isolines[r] = (isoline) + elif isotype == 'isochrone': + for r in data_range: + isoline = mapzen_isolines.calculate_isochrone(origin, mode, r) + isolines[r] = (isoline) + + result = [] + for r in data_range: + + # -- TODO encapsulate this block into a func/method + locations = isolines[r] + [ isolines[r][0] ] # close the polygon repeating the first point + wkt_coordinates = ','.join(["%f %f" % (l['lon'], l['lat']) for l in locations]) + sql = "SELECT ST_MPolyFromText('MULTIPOLYGON((({0})))', 4326) as geom".format(wkt_coordinates) + multipolygon = plpy.execute(sql, 1)[0]['geom'] + + result.append([source, r, multipolygon]) + + quota_service.increment_success_service_use() + quota_service.increment_isolines_service_use(len(isolines)) + return result + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to obtain isolines using mapzen: {0}'.format(e) + plpy.debug(traceback.format_tb(traceback_)) + raise e + #plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu SECURITY DEFINER; \ No newline at end of file diff --git a/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql b/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql new file mode 100644 index 0000000..b917b03 --- /dev/null +++ b/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql @@ -0,0 +1,7 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "ALTER EXTENSION cdb_dataservices_server UPDATE TO '0.11.0'" to load this file. \quit + +DROP FUNCTION IF EXISTS cdb_dataservices_server._cdb_mapzen_isolines(text, text, text, geometry(Geometry, 4326), text, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_mapzen_isodistance(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_mapzen_isochrone(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]); diff --git a/server/extension/cdb_dataservices_server--0.12.0.sql b/server/extension/cdb_dataservices_server--0.12.0.sql new file mode 100644 index 0000000..b26ceef --- /dev/null +++ b/server/extension/cdb_dataservices_server--0.12.0.sql @@ -0,0 +1,2260 @@ +--DO NOT MODIFY THIS FILE, IT IS GENERATED AUTOMATICALLY FROM SOURCES +-- Complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "CREATE EXTENSION cdb_dataservices_server" to load this file. \quit +CREATE TYPE cdb_dataservices_server.simple_route AS ( + shape geometry(LineString,4326), + length real, + duration integer +); + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_route_with_waypoints( + username TEXT, + orgname TEXT, + waypoints geometry(Point, 4326)[], + mode TEXT, + options text[] DEFAULT ARRAY[]::text[], + units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_server.simple_route AS $$ + import json + from cartodb_services.mapzen import MapzenRouting, MapzenRoutingResponse + from cartodb_services.mapzen.types import polyline_to_linestring + from cartodb_services.metrics import QuotaService + from cartodb_services.tools import Coordinate + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_routing_config = GD["user_routing_config_{0}".format(username)] + + quota_service = QuotaService(user_routing_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + client = MapzenRouting(user_routing_config.mapzen_api_key) + + if not waypoints or len(waypoints) < 2: + plpy.notice("Empty origin or destination") + quota_service.increment_empty_service_use() + return [None, None, None] + + waypoint_coords = [] + for waypoint in waypoints: + lat = plpy.execute("SELECT ST_Y('%s') AS lat" % waypoint)[0]['lat'] + lon = plpy.execute("SELECT ST_X('%s') AS lon" % waypoint)[0]['lon'] + waypoint_coords.append(Coordinate(lon,lat)) + + resp = client.calculate_route_point_to_point(waypoint_coords, mode, options, units) + if resp and resp.shape: + shape_linestring = polyline_to_linestring(resp.shape) + if shape_linestring: + quota_service.increment_success_service_use() + return [shape_linestring, resp.length, resp.duration] + else: + quota_service.increment_empty_service_use() + return [None, None, None] + else: + quota_service.increment_empty_service_use() + return [None, None, None] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to obtain route using mapzen provider: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu SECURITY DEFINER; +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_route_point_to_point( + username TEXT, + orgname TEXT, + origin geometry(Point, 4326), + destination geometry(Point, 4326), + mode TEXT, + options text[] DEFAULT ARRAY[]::text[], + units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_server.simple_route AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_routing_config = GD["user_routing_config_{0}".format(username)] + + waypoints = [origin, destination] + mapzen_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._cdb_mapzen_route_with_waypoints($1, $2, $3, $4, $5, $6) as route;", ["text", "text", "geometry(Point, 4326)[]", "text", "text[]", "text"]) + result = plpy.execute(mapzen_plan, [username, orgname, waypoints, mode, options, units]) + return [result[0]['shape'],result[0]['length'], result[0]['duration']] +$$ LANGUAGE plpythonu; + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_route_with_waypoints( + username TEXT, + orgname TEXT, + waypoints geometry(Point, 4326)[], + mode TEXT, + options text[] DEFAULT ARRAY[]::text[], + units text DEFAULT 'kilometers') +RETURNS cdb_dataservices_server.simple_route AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_routing_config = GD["user_routing_config_{0}".format(username)] + + mapzen_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._cdb_mapzen_route_with_waypoints($1, $2, $3, $4, $5, $6) as route;", ["text", "text", "geometry(Point, 4326)[]", "text", "text[]", "text"]) + result = plpy.execute(mapzen_plan, [username, orgname, waypoints, mode, options, units]) + return [result[0]['shape'],result[0]['length'], result[0]['duration']] +$$ LANGUAGE plpythonu; +-- Get the connection to redis from cache or create a new one +CREATE OR REPLACE FUNCTION cdb_dataservices_server._connect_to_redis(user_id text) +RETURNS boolean AS $$ + cache_key = "redis_connection_{0}".format(user_id) + if cache_key in GD: + return False + else: + from cartodb_services.tools import RedisConnection, RedisDBConfig + metadata_config = RedisDBConfig('redis_metadata_config', plpy) + metrics_config = RedisDBConfig('redis_metrics_config', plpy) + redis_metadata_connection = RedisConnection(metadata_config).redis_connection() + redis_metrics_connection = RedisConnection(metrics_config).redis_connection() + GD[cache_key] = { + 'redis_metadata_connection': redis_metadata_connection, + 'redis_metrics_connection': redis_metrics_connection, + } + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; +-- +-- Observatory connection config +-- +-- The purpose of this function is provide to the PL/Proxy functions +-- the connection string needed to connect with the current production database + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._obs_server_conn_str( + username TEXT, + orgname TEXT) +RETURNS text AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)] + + return user_obs_snapshot_config.connection_str +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetDemographicSnapshotJSON( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL, + geometry_level TEXT DEFAULT NULL) +RETURNS json AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetDemographicSnapshot(geom, time_span, geometry_level); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_demographic_snapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL, + geometry_level TEXT DEFAULT NULL) +RETURNS json AS $$ + from cartodb_services.metrics import QuotaService + import json + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_snapshot_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetDemographicSnapshotJSON($1, $2, $3, $4, $5) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, time_span, geometry_level]) + if result: + quota_service.increment_success_service_use() + return result[0]['snapshot'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetDemographicSnapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL, + geometry_level TEXT DEFAULT NULL) +RETURNS SETOF json AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetDemographicSnapshot(geom, time_span, geometry_level); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetDemographicSnapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL, + geometry_level TEXT DEFAULT NULL) +RETURNS SETOF JSON AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_snapshot_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetDemographicSnapshot($1, $2, $3, $4, $5) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, time_span, geometry_level]) + if result: + resp = [] + for element in result: + value = element['snapshot'] + resp.append(value) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use get_geographic_snapshot: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetSegmentSnapshotJSON( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + geometry_level TEXT DEFAULT NULL) +RETURNS json AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetSegmentSnapshot(geom, geometry_level); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.obs_get_segment_snapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + geometry_level TEXT DEFAULT NULL) +RETURNS json AS $$ + from cartodb_services.metrics import QuotaService + import json + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_snapshot_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetSegmentSnapshotJSON($1, $2, $3, $4) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, geometry_level]) + if result: + quota_service.increment_success_service_use() + return result[0]['snapshot'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetSegmentSnapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + geometry_level TEXT DEFAULT NULL) +RETURNS SETOF json AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetSegmentSnapshot(geom, geometry_level); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetSegmentSnapshot( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + geometry_level TEXT DEFAULT NULL) +RETURNS SETOF JSON AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_snapshot_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_snapshot_config = GD["user_obs_snapshot_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_snapshot_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetSegmentSnapshot($1, $2, $3, $4) as snapshot;", ["text", "text", "geometry(Geometry, 4326)", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, geometry_level]) + if result: + resp = [] + for element in result: + value = element['snapshot'] + resp.append(value) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use get_segment_snapshot: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetMeasure( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + measure_id TEXT, + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetMeasure(geom, measure_id, normalize, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetMeasure( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + measure_id TEXT, + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetMeasure($1, $2, $3, $4, $5, $6, $7) as measure;", ["text", "text", "geometry(Geometry, 4326)", "text", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, measure_id, normalize, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['measure'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetMeasure: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetCategory( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + category_id TEXT, + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetCategory(geom, category_id, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetCategory( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + category_id TEXT, + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetCategory($1, $2, $3, $4, $5, $6) as category;", ["text", "text", "geometry(Geometry, 4326)", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, category_id, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['category'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetCategory: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetUSCensusMeasure( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + name TEXT, + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetUSCensusMeasure(geom, name, normalize, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetUSCensusMeasure( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + name TEXT, + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetUSCensusMeasure($1, $2, $3, $4, $5, $6, $7) as census_measure;", ["text", "text", "geometry(Geometry, 4326)", "text", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, name, normalize, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['census_measure'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetUSCensusMeasure: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetUSCensusCategory( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + name TEXT, + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetUSCensusCategory(geom, name, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetUSCensusCategory( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + name TEXT, + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetUSCensusCategory($1, $2, $3, $4, $5, $6) as census_category;", ["text", "text", "geometry(Geometry, 4326)", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, name, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['census_category'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetUSCensusCategory: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetPopulation( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetPopulation(geom, normalize, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetPopulation( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + normalize TEXT DEFAULT 'area', + boundary_id TEXT DEFAULT NULL, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetPopulation($1, $2, $3, $4, $5, $6) as population;", ["text", "text", "geometry(Geometry, 4326)", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, normalize, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['population'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetPopulation: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetMeasureById( + username TEXT, + orgname TEXT, + geom_ref TEXT, + measure_id TEXT, + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetMeasureById(geom_ref, measure_id, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetMeasureById( + username TEXT, + orgname TEXT, + geom_ref TEXT, + measure_id TEXT, + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS NUMERIC AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetMeasureById($1, $2, $3, $4, $5, $6) as measure;", ["text", "text", "text", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom_ref, measure_id, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['measure'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetMeasureById: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_Search( + username TEXT, + orgname TEXT, + search_term TEXT, + relevant_boundary TEXT DEFAULT NULL) +RETURNS TABLE(id text, description text, name text, aggregate text, source text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_Search(search_term, relevant_boundary); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_Search( + username TEXT, + orgname TEXT, + search_term TEXT, + relevant_boundary TEXT DEFAULT NULL) +RETURNS TABLE(id text, description text, name text, aggregate text, source text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_Search($1, $2, $3, $4);", ["text", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, search_term, relevant_boundary]) + if result: + resp = [] + for element in result: + id = element['id'] + description = element['description'] + name = element['name'] + aggregate = element['aggregate'] + source = element['source'] + resp.append([id, description, name, aggregate, source]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [None, None, None, None, None] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_Search: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetAvailableBoundaries( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL) +RETURNS TABLE(boundary_id text, description text, time_span text, tablename text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetAvailableBoundaries(geom, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetAvailableBoundaries( + username TEXT, + orgname TEXT, + geom geometry(Geometry, 4326), + time_span TEXT DEFAULT NULL) +RETURNS TABLE(boundary_id text, description text, time_span text, tablename text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetAvailableBoundaries($1, $2, $3, $4) as available_boundaries;", ["text", "text", "geometry(Geometry, 4326)", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, time_span]) + if result: + resp = [] + for element in result: + id = element['boundary_id'] + description = element['description'] + tspan = element['time_span'] + tablename = element['tablename'] + resp.append([id, description, tspan, tablename]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetAvailableBoundaries: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetBoundary( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS geometry(Geometry, 4326) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetBoundary(geom, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetBoundary( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS geometry(Geometry, 4326) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetBoundary($1, $2, $3, $4) as boundary;", ["text", "text", "geometry(Point, 4326)", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['boundary'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetBoundary: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetBoundaryId( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetBoundaryId(geom, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetBoundaryId( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS TEXT AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetBoundaryId($1, $2, $3, $4, $5) as boundary;", ["text", "text", "geometry(Point, 4326)", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['boundary'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use obs_search: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetBoundaryById( + username TEXT, + orgname TEXT, + geometry_id TEXT, + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS geometry(Geometry, 4326) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT cdb_observatory.OBS_GetBoundaryById(geometry_id, boundary_id, time_span); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetBoundaryById( + username TEXT, + orgname TEXT, + geometry_id TEXT, + boundary_id TEXT, + time_span TEXT DEFAULT NULL) +RETURNS geometry(Geometry, 4326) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT cdb_dataservices_server._OBS_GetBoundaryById($1, $2, $3, $4, $5) as boundary;", ["text", "text", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geometry_id, boundary_id, time_span]) + if result: + quota_service.increment_success_service_use() + return result[0]['boundary'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetBoundaryById: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetBoundariesByGeometry( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type text DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetBoundariesByGeometry(geom, boundary_id, time_span, overlap_type); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetBoundariesByGeometry( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetBoundariesByGeometry($1, $2, $3, $4, $5, $6) as boundary;", ["text", "text", "geometry(Point, 4326)", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, boundary_id, time_span, overlap_type]) + if result: + resp = [] + for element in result: + the_geom = element['the_geom'] + geom_refs = element['geom_refs'] + resp.append([the_geom, geom_refs]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetBoundariesByGeometry: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetBoundariesByPointAndRadius( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + radius NUMERIC, + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetBoundariesByPointAndRadius(geom, radius, boundary_id, time_span, overlap_type); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetBoundariesByPointAndRadius( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + radius NUMERIC, + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetBoundariesByPointAndRadius($1, $2, $3, $4, $5, $6, $7) as boundary;", ["text", "text", "geometry(Point, 4326)", "numeric", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, radius, boundary_id, time_span, overlap_type]) + if result: + resp = [] + for element in result: + the_geom = element['the_geom'] + geom_refs = element['geom_refs'] + resp.append([the_geom, geom_refs]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetBoundariesByPointAndRadius: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetPointsByGeometry( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetPointsByGeometry(geom, boundary_id, time_span, overlap_type); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetPointsByGeometry( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetPointsByGeometry($1, $2, $3, $4, $5, $6) as boundary;", ["text", "text", "geometry(Point, 4326)", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, boundary_id, time_span, overlap_type]) + if result: + resp = [] + for element in result: + the_geom = element['the_geom'] + geom_refs = element['geom_refs'] + resp.append([the_geom, geom_refs]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetPointsByGeometry: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._OBS_GetPointsByPointAndRadius( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + radius NUMERIC, + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + CONNECT cdb_dataservices_server._obs_server_conn_str(username, orgname); + SELECT * FROM cdb_observatory.OBS_GetPointsByPointAndRadius(geom, radius, boundary_id, time_span, overlap_type); +$$ LANGUAGE plproxy; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.OBS_GetPointsByPointAndRadius( + username TEXT, + orgname TEXT, + geom geometry(Point, 4326), + radius NUMERIC, + boundary_id TEXT, + time_span TEXT DEFAULT NULL, + overlap_type TEXT DEFAULT 'intersects') +RETURNS TABLE(the_geom geometry, geom_refs text) AS $$ + from cartodb_services.metrics import QuotaService + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_obs_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_obs_config = GD["user_obs_config_{0}".format(username)] + + quota_service = QuotaService(user_obs_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + obs_plan = plpy.prepare("SELECT * FROM cdb_dataservices_server._OBS_GetPointsByPointAndRadius($1, $2, $3, $4, $5, $6, $7) as boundary;", ["text", "text", "geometry(Point, 4326)", "numeric", "text", "text", "text"]) + result = plpy.execute(obs_plan, [username, orgname, geom, radius, boundary_id, time_span, overlap_type]) + if result: + resp = [] + for element in result: + the_geom = element['the_geom'] + geom_refs = element['geom_refs'] + resp.append([the_geom, geom_refs]) + quota_service.increment_success_service_use() + return resp + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to use OBS_GetPointsByPointAndRadius: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_geocoder_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_geocoder_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import GeocoderConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + geocoder_config = GeocoderConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = geocoder_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_mapzen_geocoder_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_mapzen_geocoder_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import MapzenGeocoderConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + mapzen_geocoder_config = MapzenGeocoderConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = mapzen_geocoder_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_mapzen_isolines_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_mapzen_isolines_routing_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import MapzenIsolinesRoutingConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + mapzen_isolines_config = MapzenIsolinesRoutingConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = mapzen_isolines_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_internal_geocoder_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_internal_geocoder_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import InternalGeocoderConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + geocoder_config = InternalGeocoderConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = geocoder_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_isolines_routing_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_isolines_routing_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import IsolinesRoutingConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + isolines_routing_config = IsolinesRoutingConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = isolines_routing_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_routing_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_routing_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import RoutingConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + routing_config = RoutingConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = routing_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_snapshot_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_obs_snapshot_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import ObservatorySnapshotConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + obs_snapshot_config = ObservatorySnapshotConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = obs_snapshot_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_obs_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_obs_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import ObservatoryConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + obs_config = ObservatoryConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = obs_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; +-- Geocodes a street address given a searchtext and a state and/or country +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_geocoder_config_{0}".format(username)] + + if user_geocoder_config.heremaps_geocoder: + here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(here_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + elif user_geocoder_config.google_geocoder: + google_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_google_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(google_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + elif user_geocoder_config.mapzen_geocoder: + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(mapzen_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + else: + plpy.error('Requested geocoder is not available') + +$$ LANGUAGE plpythonu; + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_here_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_geocoder_config_{0}".format(username)] + + if user_geocoder_config.heremaps_geocoder: + here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(here_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + else: + plpy.error('Here geocoder is not available for your account.') + +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_google_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_geocoder_config_{0}".format(username)] + + if user_geocoder_config.google_geocoder: + google_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_google_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(google_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + else: + plpy.error('Google geocoder is not available for your account.') + +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + # The configuration is retrieved but no checks are performed on it + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_mapzen_geocoder_config_{0}".format(username)] + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_geocode_street_point($1, $2, $3, $4, $5, $6) as point; ", ["text", "text", "text", "text", "text", "text"]) + return plpy.execute(mapzen_plan, [username, orgname, searchtext, city, state_province, country], 1)[0]['point'] + +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + from cartodb_services.here import HereMapsGeocoder + from cartodb_services.metrics import QuotaService + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_geocoder_config = GD["user_geocoder_config_{0}".format(username)] + + # -- Check the quota + quota_service = QuotaService(user_geocoder_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + geocoder = HereMapsGeocoder(user_geocoder_config.heremaps_app_id, user_geocoder_config.heremaps_app_code) + coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country) + if coordinates: + quota_service.increment_success_service_use() + plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"]) + point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0] + return point['st_setsrid'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using here maps geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_google_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + from cartodb_services.google import GoogleMapsGeocoder + from cartodb_services.metrics import QuotaService + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_geocoder_config = GD["user_geocoder_config_{0}".format(username)] + quota_service = QuotaService(user_geocoder_config, redis_conn) + + try: + geocoder = GoogleMapsGeocoder(user_geocoder_config.google_client_id, user_geocoder_config.google_api_key) + coordinates = geocoder.geocode(searchtext=searchtext, city=city, state=state_province, country=country) + if coordinates: + quota_service.increment_success_service_use() + plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"]) + point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0] + return point['st_setsrid'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using google maps geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_geocode_street_point(username TEXT, orgname TEXT, searchtext TEXT, city TEXT DEFAULT NULL, state_province TEXT DEFAULT NULL, country TEXT DEFAULT NULL) +RETURNS Geometry AS $$ + from cartodb_services.mapzen import MapzenGeocoder + from cartodb_services.mapzen.types import country_to_iso3 + from cartodb_services.metrics import QuotaService + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_mapzen_geocoder_config = GD["user_mapzen_geocoder_config_{0}".format(username)] + quota_service = QuotaService(user_mapzen_geocoder_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + geocoder = MapzenGeocoder(user_mapzen_geocoder_config.mapzen_api_key) + country_iso3 = None + if country: + country_iso3 = country_to_iso3(country) + coordinates = geocoder.geocode(searchtext=searchtext, city=city, + state_province=state_province, + country=country_iso3) + if coordinates: + quota_service.increment_success_service_use() + plan = plpy.prepare("SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326); ", ["double precision", "double precision"]) + point = plpy.execute(plan, [coordinates[0], coordinates[1]], 1)[0] + return point['st_setsrid'] + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using mapzen geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin0_polygon(username text, orgname text, country_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin0_polygon(trim($1)) AS mypolygon", ["text"]) + rv = plpy.execute(plan, [country_name], 1) + result = rv[0]["mypolygon"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + + +-------------------------------------------------------------------------------- + +-- Implementation of the server extension +-- Note: these functions depend on the cdb_geocoder extension +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin0_polygon(country_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT n.the_geom as geom INTO ret + FROM (SELECT q, lower(regexp_replace(q, '[^a-zA-Z\u00C0-\u00ff]+', '', 'g'))::text x + FROM (SELECT country_name q) g) d + LEFT OUTER JOIN admin0_synonyms s ON name_ = d.x + LEFT OUTER JOIN ne_admin0_v3 n ON s.adm0_a3 = n.adm0_a3 GROUP BY d.q, n.the_geom, s.adm0_a3; + + RETURN ret; + END +$$ LANGUAGE plpgsql; +---- cdb_geocode_admin1_polygon(admin1_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1)) AS mypolygon", ["text"]) + rv = plpy.execute(plan, [admin1_name], 1) + result = rv[0]["mypolygon"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +---- cdb_geocode_admin1_polygon(admin1_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_admin1_polygon(username text, orgname text, admin1_name text, country_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_admin1_polygon(trim($1), trim($2)) AS mypolygon", ["text", "text"]) + rv = plpy.execute(plan, [admin1_name, country_name], 1) + result = rv[0]["mypolygon"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +-------------------------------------------------------------------------------- + +-- Implementation of the server extension +-- Note: these functions depend on the cdb_geocoder extension + +---- cdb_geocode_admin1_polygon(admin1_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + SELECT q, ( + SELECT the_geom + FROM global_province_polygons + WHERE d.c = ANY (synonyms) + ORDER BY frequency DESC LIMIT 1 + ) geom + FROM ( + SELECT + trim(replace(lower(admin1_name),'.',' ')) c, admin1_name q + ) d + ) v; + + RETURN ret; + END +$$ LANGUAGE plpgsql; + +---- cdb_geocode_admin1_polygon(admin1_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_admin1_polygon(admin1_name text, country_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + WITH p AS (SELECT r.c, r.q, (SELECT iso3 FROM country_decoder WHERE lower(country_name) = ANY (synonyms)) i FROM (SELECT trim(replace(lower(admin1_name),'.',' ')) c, country_name q) r) + SELECT + geom INTO ret + FROM ( + SELECT + q, ( + SELECT the_geom + FROM global_province_polygons + WHERE p.c = ANY (synonyms) + AND iso3 = p.i + ORDER BY frequency DESC LIMIT 1 + ) geom + FROM p) n; + + RETURN ret; + END +$$ LANGUAGE plpgsql; + +---- cdb_geocode_namedplace_point(city_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1)) AS mypoint", ["text"]) + rv = plpy.execute(plan, [city_name], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +---- cdb_geocode_namedplace_point(city_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, country_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2)) AS mypoint", ["text", "text"]) + rv = plpy.execute(plan, [city_name, country_name], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_namedplace_point(username text, orgname text, city_name text, admin1_name text, country_name text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_namedplace_point(trim($1), trim($2), trim($3)) AS mypoint", ["text", "text", "text"]) + rv = plpy.execute(plan, [city_name, admin1_name, country_name], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +-------------------------------------------------------------------------------- + +-- Implementation of the server extension +-- Note: these functions depend on the cdb_geocoder extension + +---- cdb_geocode_namedplace_point(city_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + WITH best AS (SELECT s AS q, (SELECT the_geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) ORDER BY population DESC LIMIT 1) AS geom FROM (SELECT city_name as s) p), + next AS (SELECT p.s AS q, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM (SELECT city_name as s) p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL)) + SELECT q, geom, TRUE AS success FROM best WHERE geom IS NOT NULL + UNION ALL + SELECT q, geom, CASE WHEN geom IS NULL THEN FALSE ELSE TRUE END AS success FROM next + ) v; + + RETURN ret; + END +$$ LANGUAGE plpgsql; + +---- cdb_geocode_namedplace_point(city_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, country_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + WITH p AS (SELECT r.s, r.c, (SELECT iso2 FROM country_decoder WHERE lower(r.c) = ANY (synonyms)) i FROM (SELECT city_name AS s, country_name::text AS c) r), + best AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.iso2 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p), + next AS (SELECT p.s AS q, p.c AS c, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND gp.iso2 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE c = p.c AND geom IS NOT NULL)) + SELECT geom FROM best WHERE geom IS NOT NULL + UNION ALL + SELECT geom FROM next + ) v; + + RETURN ret; + END +$$ LANGUAGE plpgsql; + +---- cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text) +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_namedplace_point(city_name text, admin1_name text, country_name text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + WITH inputcountry AS ( + SELECT iso2 as isoTwo FROM country_decoder WHERE lower(country_name) = ANY (synonyms) LIMIT 1 + ), + p AS ( + SELECT r.s, r.a1, (SELECT admin1 FROM admin1_decoder, inputcountry WHERE lower(r.a1) = ANY (synonyms) AND admin1_decoder.iso2 = inputcountry.isoTwo LIMIT 1) i FROM (SELECT city_name AS s, admin1_name::text AS a1) r), + best AS (SELECT p.s AS q, p.a1 as a1, (SELECT gp.the_geom AS geom FROM global_cities_points_limited gp WHERE gp.lowername = lower(p.s) AND gp.admin1 = p.i ORDER BY population DESC LIMIT 1) AS geom FROM p), + next AS (SELECT p.s AS q, p.a1 AS a1, (SELECT gp.the_geom FROM global_cities_points_limited gp, global_cities_alternates_limited ga WHERE lower(p.s) = ga.lowername AND ga.admin1 = p.i AND ga.geoname_id = gp.geoname_id ORDER BY preferred DESC LIMIT 1) geom FROM p WHERE p.s NOT IN (SELECT q FROM best WHERE geom IS NOT NULL)) + SELECT geom FROM best WHERE geom IS NOT NULL + UNION ALL + SELECT geom FROM next + ) v; + + RETURN ret; + END +$$ LANGUAGE plpgsql; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1)) AS mypoint", ["text"]) + rv = plpy.execute(plan, [code], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_point(username text, orgname text, code text, country text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_point(trim($1), trim($2)) AS mypoint", ["TEXT", "TEXT"]) + rv = plpy.execute(plan, [code, country], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1)) AS mypolygon", ["text"]) + rv = plpy.execute(plan, [code], 1) + result = rv[0]["mypolygon"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_postalcode_polygon(username text, orgname text, code text, country text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_postalcode_polygon(trim($1), trim($2)) AS mypolygon", ["TEXT", "TEXT"]) + rv = plpy.execute(plan, [code, country], 1) + result = rv[0]["mypolygon"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +-------------------------------------------------------------------------------- + +-- Implementation of the server extension +-- Note: these functions depend on the cdb_geocoder extension +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + SELECT + q, ( + SELECT the_geom + FROM global_postal_code_points + WHERE postal_code = upper(d.q) + LIMIT 1 + ) geom + FROM (SELECT code q) d + ) v; + + RETURN ret; +END +$$ LANGUAGE plpgsql; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_point(code text, country text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + SELECT + q, ( + SELECT the_geom + FROM global_postal_code_points + WHERE postal_code = upper(d.q) + AND iso3 = ( + SELECT iso3 FROM country_decoder WHERE + lower(country) = ANY (synonyms) LIMIT 1 + ) + LIMIT 1 + ) geom + FROM (SELECT code q) d + ) v; + + RETURN ret; +END +$$ LANGUAGE plpgsql; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + SELECT + q, ( + SELECT the_geom + FROM global_postal_code_polygons + WHERE postal_code = upper(d.q) + LIMIT 1 + ) geom + FROM (SELECT code q) d + ) v; + + RETURN ret; +END +$$ LANGUAGE plpgsql; + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_postalcode_polygon(code text, country text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + BEGIN + SELECT geom INTO ret + FROM ( + SELECT + q, ( + SELECT the_geom + FROM global_postal_code_polygons + WHERE postal_code = upper(d.q) + AND iso3 = ( + SELECT iso3 FROM country_decoder WHERE + lower(country) = ANY (synonyms) LIMIT 1 + ) + LIMIT 1 + ) geom + FROM (SELECT code q) d + ) v; + + RETURN ret; +END +$$ LANGUAGE plpgsql; +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_geocode_ipaddress_point(username text, orgname text, ip text) +RETURNS Geometry AS $$ + from cartodb_services.metrics import QuotaService + from cartodb_services.metrics import InternalGeocoderConfig + + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_internal_geocoder_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_geocoder_config = GD["user_internal_geocoder_config_{0}".format(username)] + + quota_service = QuotaService(user_geocoder_config, redis_conn) + try: + plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_geocode_ipaddress_point(trim($1)) AS mypoint", ["TEXT"]) + rv = plpy.execute(plan, [ip], 1) + result = rv[0]["mypoint"] + if result: + quota_service.increment_success_service_use() + return result + else: + quota_service.increment_empty_service_use() + return None + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to geocode using admin0 geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu; + +-------------------------------------------------------------------------------- + +-- Implementation of the server extension +-- Note: these functions depend on the cdb_geocoder extension +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_geocode_ipaddress_point(ip text) +RETURNS Geometry AS $$ + DECLARE + ret Geometry; + + new_ip INET; + BEGIN + BEGIN + IF family(ip::inet) = 6 THEN + new_ip := ip::inet; + ELSE + new_ip := ('::ffff:' || ip)::inet; + END IF; + EXCEPTION WHEN OTHERS THEN + SELECT NULL as geom INTO ret; + RETURN ret; + END; + + WITH + ips AS (SELECT ip s, new_ip net), + matches AS (SELECT s, (SELECT the_geom FROM ip_address_locations WHERE network_start_ip <= ips.net ORDER BY network_start_ip DESC LIMIT 1) geom FROM ips) + SELECT geom INTO ret + FROM matches; + RETURN ret; +END +$$ LANGUAGE plpgsql; +CREATE TYPE cdb_dataservices_server.isoline AS (center geometry(Geometry,4326), data_range integer, the_geom geometry(Multipolygon,4326)); + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_here_routing_isolines(username TEXT, orgname TEXT, type TEXT, source geometry(Geometry, 4326), mode TEXT, data_range integer[], options text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + import json + from cartodb_services.here import HereMapsRoutingIsoline + from cartodb_services.metrics import QuotaService + from cartodb_services.here.types import geo_polyline_to_multipolygon + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_isolines_routing_config = GD["user_isolines_routing_config_{0}".format(username)] + + # -- Check the quota + quota_service = QuotaService(user_isolines_routing_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + client = HereMapsRoutingIsoline(user_isolines_routing_config.heremaps_app_id, user_isolines_routing_config.heremaps_app_code, base_url = HereMapsRoutingIsoline.PRODUCTION_ROUTING_BASE_URL) + + if source: + lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat'] + lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon'] + source_str = 'geo!%f,%f' % (lat, lon) + else: + source_str = None + + if type == 'isodistance': + resp = client.calculate_isodistance(source_str, mode, data_range, options) + elif type == 'isochrone': + resp = client.calculate_isochrone(source_str, mode, data_range, options) + + if resp: + result = [] + for isoline in resp: + data_range_n = isoline['range'] + polyline = isoline['geom'] + multipolygon = geo_polyline_to_multipolygon(polyline) + result.append([source, data_range_n, multipolygon]) + quota_service.increment_success_service_use() + quota_service.increment_isolines_service_use(len(resp)) + return result + else: + quota_service.increment_empty_service_use() + return [] + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to obtain isodistances using here maps geocoder: {0}'.format(e) + plpy.notice(traceback.format_tb(traceback_)) + plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu SECURITY DEFINER; + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._cdb_mapzen_isolines( + username TEXT, + orgname TEXT, + isotype TEXT, + source geometry(Geometry, 4326), + mode TEXT, + data_range integer[], + options text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + import json + from cartodb_services.mapzen import MatrixClient + from cartodb_services.mapzen import MapzenIsolines + from cartodb_services.metrics import QuotaService + + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + user_mapzen_isolines_routing_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + + # -- Check the quota + quota_service = QuotaService(user_mapzen_isolines_routing_config, redis_conn) + if not quota_service.check_user_quota(): + plpy.error('You have reached the limit of your quota') + + try: + client = MatrixClient(user_mapzen_isolines_routing_config.mapzen_matrix_api_key) + mapzen_isolines = MapzenIsolines(client) + + if source: + lat = plpy.execute("SELECT ST_Y('%s') AS lat" % source)[0]['lat'] + lon = plpy.execute("SELECT ST_X('%s') AS lon" % source)[0]['lon'] + origin = {'lat': lat, 'lon': lon} + else: + raise Exception('source is NULL') + + # -- TODO Support options properly + isolines = {} + if isotype == 'isodistance': + for r in data_range: + isoline = mapzen_isolines.calculate_isodistance(origin, mode, r) + isolines[r] = (isoline) + elif isotype == 'isochrone': + for r in data_range: + isoline = mapzen_isolines.calculate_isochrone(origin, mode, r) + isolines[r] = (isoline) + + result = [] + for r in data_range: + + # -- TODO encapsulate this block into a func/method + locations = isolines[r] + [ isolines[r][0] ] # close the polygon repeating the first point + wkt_coordinates = ','.join(["%f %f" % (l['lon'], l['lat']) for l in locations]) + sql = "SELECT ST_MPolyFromText('MULTIPOLYGON((({0})))', 4326) as geom".format(wkt_coordinates) + multipolygon = plpy.execute(sql, 1)[0]['geom'] + + result.append([source, r, multipolygon]) + + quota_service.increment_success_service_use() + quota_service.increment_isolines_service_use(len(isolines)) + return result + except BaseException as e: + import sys, traceback + type_, value_, traceback_ = sys.exc_info() + quota_service.increment_failed_service_use() + error_msg = 'There was an error trying to obtain isolines using mapzen: {0}'.format(e) + plpy.debug(traceback.format_tb(traceback_)) + raise e + #plpy.error(error_msg) + finally: + quota_service.increment_total_service_use() +$$ LANGUAGE plpythonu SECURITY DEFINER; +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)] + type = 'isodistance' + + if user_isolines_config.google_services_user: + plpy.error('This service is not available for google service users.') + + here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; + +-- mapzen isodistance +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + type = 'isodistance' + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_isolines_routing_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_isolines_routing_config_{0}".format(username)] + type = 'isochrone' + + if user_isolines_config.google_services_user: + plpy.error('This service is not available for google service users.') + + here_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_here_routing_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(here_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; + + +-- mapzen isochrones +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + type = 'isochrone' + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') #--TODO what is this for? + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; +DO $$ +BEGIN + IF NOT EXISTS ( + SELECT * + FROM pg_catalog.pg_user + WHERE usename = 'geocoder_api') THEN + + CREATE USER geocoder_api; + END IF; + GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA cdb_dataservices_server TO geocoder_api; + GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO geocoder_api; + GRANT USAGE ON SCHEMA cdb_dataservices_server TO geocoder_api; + GRANT USAGE ON SCHEMA public TO geocoder_api; + GRANT SELECT ON ALL TABLES IN SCHEMA public TO geocoder_api; +END$$; diff --git a/server/extension/cdb_dataservices_server.control b/server/extension/cdb_dataservices_server.control index 5352099..929b898 100644 --- a/server/extension/cdb_dataservices_server.control +++ b/server/extension/cdb_dataservices_server.control @@ -1,5 +1,5 @@ comment = 'CartoDB dataservices server extension' -default_version = '0.11.0' +default_version = '0.12.0' requires = 'plpythonu, plproxy, postgis, cdb_geocoder' superuser = true schema = cdb_dataservices_server diff --git a/server/extension/cdb_dataservices_server--0.10.0--0.11.0.sql b/server/extension/old_versions/cdb_dataservices_server--0.10.0--0.11.0.sql similarity index 100% rename from server/extension/cdb_dataservices_server--0.10.0--0.11.0.sql rename to server/extension/old_versions/cdb_dataservices_server--0.10.0--0.11.0.sql diff --git a/server/extension/cdb_dataservices_server--0.11.0--0.10.0.sql b/server/extension/old_versions/cdb_dataservices_server--0.11.0--0.10.0.sql similarity index 100% rename from server/extension/cdb_dataservices_server--0.11.0--0.10.0.sql rename to server/extension/old_versions/cdb_dataservices_server--0.11.0--0.10.0.sql diff --git a/server/extension/cdb_dataservices_server--0.11.0.sql b/server/extension/old_versions/cdb_dataservices_server--0.11.0.sql similarity index 100% rename from server/extension/cdb_dataservices_server--0.11.0.sql rename to server/extension/old_versions/cdb_dataservices_server--0.11.0.sql diff --git a/server/lib/python/cartodb_services/setup.py b/server/lib/python/cartodb_services/setup.py index 663c5ec..869dcc3 100644 --- a/server/lib/python/cartodb_services/setup.py +++ b/server/lib/python/cartodb_services/setup.py @@ -10,7 +10,7 @@ from setuptools import setup, find_packages setup( name='cartodb_services', - version='0.6.5', + version='0.6.6', description='CartoDB Services API Python Library', From 97f4a8228b43cb968450817699b69f135f4f07e0 Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 10:01:48 +0200 Subject: [PATCH 40/49] Missing file upload --- ...db_dataservices_server--0.11.0--0.12.0.sql | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql index d9eddf1..5682ba0 100644 --- a/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql +++ b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql @@ -71,4 +71,48 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ #plpy.error(error_msg) finally: quota_service.increment_total_service_use() -$$ LANGUAGE plpythonu SECURITY DEFINER; \ No newline at end of file +$$ LANGUAGE plpythonu SECURITY DEFINER; + + + +-- mapzen isodistance +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isodistance(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + type = 'isodistance' + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; + + + +-- mapzen isochrones +CREATE OR REPLACE FUNCTION cdb_dataservices_server.cdb_mapzen_isochrone(username TEXT, orgname TEXT, source geometry(Geometry, 4326), mode TEXT, range integer[], options text[] DEFAULT array[]::text[]) +RETURNS SETOF cdb_dataservices_server.isoline AS $$ + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metrics_connection'] + plpy.execute("SELECT cdb_dataservices_server._get_mapzen_isolines_config({0}, {1})".format(plpy.quote_nullable(username), plpy.quote_nullable(orgname))) + user_isolines_config = GD["user_mapzen_isolines_routing_config_{0}".format(username)] + type = 'isochrone' + + mapzen_plan = plpy.prepare("SELECT cdb_dataservices_server._cdb_mapzen_isolines($1, $2, $3, $4, $5, $6, $7) as isoline; ", ["text", "text", "text", "geometry(Geometry, 4326)", "text", "integer[]", "text[]"]) + result = plpy.execute(mapzen_plan, [username, orgname, type, source, mode, range, options]) + isolines = [] + for element in result: + isoline = element['isoline'] + isoline = isoline.translate(None, "()").split(',') #--TODO what is this for? + isolines.append(isoline) + + return isolines +$$ LANGUAGE plpythonu; \ No newline at end of file From aff5dd13d72fa7ff789fc29bd851d06cbb2699eb Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 10:04:54 +0200 Subject: [PATCH 41/49] Bump python library to 0.7.0 --- server/lib/python/cartodb_services/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/setup.py b/server/lib/python/cartodb_services/setup.py index 869dcc3..a0a3f66 100644 --- a/server/lib/python/cartodb_services/setup.py +++ b/server/lib/python/cartodb_services/setup.py @@ -10,7 +10,7 @@ from setuptools import setup, find_packages setup( name='cartodb_services', - version='0.6.6', + version='0.7.0', description='CartoDB Services API Python Library', From 8f7e94e7d66f42b5adbb6c3eb074e669bb1dfe1a Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 10:12:11 +0200 Subject: [PATCH 42/49] Add grants to client functions --- client/cdb_dataservices_client--0.8.0--0.9.0.sql | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/client/cdb_dataservices_client--0.8.0--0.9.0.sql b/client/cdb_dataservices_client--0.8.0--0.9.0.sql index 515ae97..0b12c9f 100644 --- a/client/cdb_dataservices_client--0.8.0--0.9.0.sql +++ b/client/cdb_dataservices_client--0.8.0--0.9.0.sql @@ -75,4 +75,16 @@ RETURNS SETOF cdb_dataservices_client.isoline AS $$ SELECT * FROM cdb_dataservices_server.cdb_mapzen_isodistance (username, organization_name, source, mode, range, options); -$$ LANGUAGE plproxy; \ No newline at end of file +$$ LANGUAGE plproxy; + + +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_isochrone(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; + +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_isodistance(source geometry(Geometry, 4326), mode text, range integer[], options text[]) TO publicuser; + +-- Grants missing in previous version +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_here_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; + +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_google_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; + +GRANT EXECUTE ON FUNCTION cdb_dataservices_client.cdb_mapzen_geocode_street_point(searchtext text, city text, state_province text, country text) TO publicuser; From d8aa64625119738c7db3cfddfe6ff88cc628ee68 Mon Sep 17 00:00:00 2001 From: Carla Iriberri Date: Thu, 7 Jul 2016 10:41:15 +0200 Subject: [PATCH 43/49] Add mapzen config helper function --- .../cdb_dataservices_server--0.11.0--0.12.0.sql | 17 ++++++++++++++++- .../cdb_dataservices_server--0.12.0--0.11.0.sql | 1 + 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql index 5682ba0..d09ba67 100644 --- a/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql +++ b/server/extension/cdb_dataservices_server--0.11.0--0.12.0.sql @@ -115,4 +115,19 @@ RETURNS SETOF cdb_dataservices_server.isoline AS $$ isolines.append(isoline) return isolines -$$ LANGUAGE plpythonu; \ No newline at end of file +$$ LANGUAGE plpythonu; + + +CREATE OR REPLACE FUNCTION cdb_dataservices_server._get_mapzen_isolines_config(username text, orgname text) +RETURNS boolean AS $$ + cache_key = "user_mapzen_isolines_routing_config_{0}".format(username) + if cache_key in GD: + return False + else: + from cartodb_services.metrics import MapzenIsolinesRoutingConfig + plpy.execute("SELECT cdb_dataservices_server._connect_to_redis('{0}')".format(username)) + redis_conn = GD["redis_connection_{0}".format(username)]['redis_metadata_connection'] + mapzen_isolines_config = MapzenIsolinesRoutingConfig(redis_conn, plpy, username, orgname) + GD[cache_key] = mapzen_isolines_config + return True +$$ LANGUAGE plpythonu SECURITY DEFINER; diff --git a/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql b/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql index b917b03..1acf9ed 100644 --- a/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql +++ b/server/extension/cdb_dataservices_server--0.12.0--0.11.0.sql @@ -5,3 +5,4 @@ DROP FUNCTION IF EXISTS cdb_dataservices_server._cdb_mapzen_isolines(text, text, text, geometry(Geometry, 4326), text, integer[], text[]); DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_mapzen_isodistance(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]); DROP FUNCTION IF EXISTS cdb_dataservices_server.cdb_mapzen_isochrone(TEXT, TEXT, geometry(Geometry, 4326), TEXT, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_server._get_mapzen_isolines_config(text, text); \ No newline at end of file From d22bf661e184f3eabfae61b9d0a7784e33124ab5 Mon Sep 17 00:00:00 2001 From: Carla Date: Thu, 7 Jul 2016 11:27:02 +0200 Subject: [PATCH 44/49] typo in function signature in downgrade --- client/cdb_dataservices_client--0.9.0--0.8.0.sql | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/client/cdb_dataservices_client--0.9.0--0.8.0.sql b/client/cdb_dataservices_client--0.9.0--0.8.0.sql index ceeebac..3703182 100644 --- a/client/cdb_dataservices_client--0.9.0--0.8.0.sql +++ b/client/cdb_dataservices_client--0.9.0--0.8.0.sql @@ -3,5 +3,5 @@ \echo Use "ALTER EXTENSION cdb_dataservices_client UPDATE TO '0.8.0'" to load this file. \quit DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_mapzen_isochrone(geometry(Geometry, 4326), text, integer[], text[]); DROP FUNCTION IF EXISTS cdb_dataservices_client.cdb_mapzen_isodistance(geometry(Geometry, 4326), text, integer[], text[]); -DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isochrone(text, text, geometry(Geometry, 4326), text, integer[], text); -DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isodistance(text, text, geometry(Geometry, 4326), text, integer[], text); +DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isochrone(text, text, geometry(Geometry, 4326), text, integer[], text[]); +DROP FUNCTION IF EXISTS cdb_dataservices_client._cdb_mapzen_isodistance(text, text, geometry(Geometry, 4326), text, integer[], text[]); From 4e6aa0721d89f06456803633797ec718f7a1affe Mon Sep 17 00:00:00 2001 From: Carla Date: Mon, 11 Jul 2016 10:26:53 +0200 Subject: [PATCH 45/49] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5a9b232..6fea0fa 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # Data Services API -The CartoDB Data Services SQL API (server and client FTM) +The CARTO Data Services SQL API (server and client FTM) ### Deploy instructions Steps to deploy a new Data Services API version : From 5721e10520568c2e58935b42622e848fd436d5aa Mon Sep 17 00:00:00 2001 From: Carla Date: Mon, 11 Jul 2016 10:47:07 +0200 Subject: [PATCH 46/49] Update README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6fea0fa..2f8af0a 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# Data Services API -The CARTO Data Services SQL API (server and client FTM) +# CARTO Data Services API +The CARTO Data Services SQL API ### Deploy instructions Steps to deploy a new Data Services API version : From 6fb9f67e64c8600d4fbdff800ee66bd466226210 Mon Sep 17 00:00:00 2001 From: Carla Date: Mon, 11 Jul 2016 10:50:19 +0200 Subject: [PATCH 47/49] Update README.md --- server/lib/python/cartodb_services/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/lib/python/cartodb_services/README.md b/server/lib/python/cartodb_services/README.md index 3703c82..1af84ce 100644 --- a/server/lib/python/cartodb_services/README.md +++ b/server/lib/python/cartodb_services/README.md @@ -1,6 +1,6 @@ # CartoDB dataservices API python module -This directory contains the python library used by the server side of CartoDB LDS (Location Data Services). +This directory contains the python library used by the server side of CARTO LDS (Location Data Services). It is used from pl/python functions contained in the `cdb_dataservices_server` extension. It goes hand in hand with the extension so please consider running the integration tests. From 2aace4bf7d77cc7075aecd144979fe7b9293a29f Mon Sep 17 00:00:00 2001 From: Carla Date: Mon, 11 Jul 2016 10:51:16 +0200 Subject: [PATCH 48/49] Update README.md --- server/extension/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/server/extension/README.md b/server/extension/README.md index 5b8f75b..e45e3fc 100644 --- a/server/extension/README.md +++ b/server/extension/README.md @@ -1,15 +1,15 @@ -# CartoDB dataservices API server extension -Postgres extension for the CartoDB dataservices API, server side. +# CARTO Data Services API server extension +Postgres extension for the CARTO Data Services API, server side. ## Dependencies -This extension is thought to be used on top of CartoDB geocoder extension, for the internal geocoder. +This extension is thought to be used on top of CARTO geocoder extension, for the internal geocoder. The following is a non-comprehensive list of dependencies: - Postgres 9.3+ - Postgis extension - Schema triggers extension -- CartoDB extension +- cartodb-postgresql CARTO extension ## Installation into the db cluster This requires root privileges @@ -28,7 +28,7 @@ One-liner: sudo PGUSER=postgres make all install installcheck ``` -## Install onto a cartodb user's database +## Install onto a CARTO user's database Remember that **is mandatory to install it on top of cdb_geocoder** From ca1461c020b1b248b77e45289ed8f27db90bdfd2 Mon Sep 17 00:00:00 2001 From: Carla Date: Mon, 11 Jul 2016 10:51:51 +0200 Subject: [PATCH 49/49] Update README.md --- client/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/client/README.md b/client/README.md index df533f4..9b26f0e 100644 --- a/client/README.md +++ b/client/README.md @@ -1,5 +1,5 @@ -# CartoDB dataservices API client extension -Postgres extension for the CartoDB dataservices API, client side. +# CARTO Data Services API client extension +Postgres extension for the CARTO Data Services API, client side. ## Dependencies This extension is thought to be used on top of CartoDB geocoder extension, for the multiples available geocoders (internal, nokia, etc). @@ -9,7 +9,7 @@ The following is a non-comprehensive list of dependencies: - Postgres 9.3+ - Postgis extension - Schema triggers extension -- CartoDB extension +- cartodb-postgresql CARTO extension ## Installation into the db cluster This requires root privileges @@ -28,7 +28,7 @@ One-liner: sudo PGUSER=postgres make all install installcheck ``` -## Install onto a cartodb user's database +## Install onto a CARTO user's database ``` psql -U postgres cartodb_dev_user_fe3b850a-01c0-48f9-8a26-a82f09e9b53f_db