cartodb4/dataservices-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,944 Commits 1 Branch 183 Tags 3.3 MiB
d920d72b0b
Commit Graph

288 Commits

Author SHA1 Message Date
Rafa de la Torre
2e6f37af56 Add observatory #302 2016-11-11 11:27:57 +01:00
Mario de Frutos
b5f4c507b3 Fixed some typos in SQL artifact v0.18 and v0.17 2016-11-11 11:13:08 +01:00
Rafa de la Torre
95dda082e7 Add routing #302 2016-11-11 10:24:03 +01:00
Rafa de la Torre
9f9034f4f1 Make cdb_enough_quota check the table got from cdb_service_params #302 2016-11-10 19:23:39 +01:00
Rafa de la Torre
603fbbbc3f Make cdb_service_params return a table for all services #302 2016-11-10 19:23:01 +01:00
Rafa de la Torre
3fa686ec65 Remove the class Service in favor of service_type ENUM #302 2016-11-10 19:21:29 +01:00
Rafa de la Torre
429a71ef37 Add a function to check for enough quota #302 2016-11-10 18:56:50 +01:00
Rafa de la Torre
b2cdb1c74b Move all into one func returning a tuple #302 2016-11-10 18:40:18 +01:00
Rafa de la Torre
84061dec4f Move everything to cdb_service_params (WIP) #302 2016-11-10 18:12:24 +01:00
Mario de Frutos
08debb83ed Server side change JSONB to JSON in the legacybuildermedatada function 2016-11-10 18:08:35 +01:00
Rafa de la Torre
7a32ace1ac Add the hires_geocoder to cdb_monthly_quota #302 2016-11-10 17:00:25 +01:00
Mario de Frutos
040b094b01 New observatory meta functions for server side 2016-11-10 11:29:19 +01:00
Rafa de la Torre
eb7188235d Provider function #302 2016-11-10 11:14:15 +01:00
Rafa de la Torre
3e059003b6 Add soft limit function #302 2016-11-10 11:09:40 +01:00
Rafa de la Torre
0e38af6ecd Add cdb_remaining_quota to server #302 2016-11-08 18:47:22 +01:00
Rafa de la Torre
6f9feb07a0 Add cdb_monthly_quota (WIP) #302 2016-11-08 18:29:06 +01:00
Mario de Frutos
352d4217bc Fixed typo in the config cache for observatory snapshot functions 2016-11-07 10:31:53 +01:00
Mario de Frutos
49fd51f6da Added logger to metrics context manager 2016-11-04 13:39:10 +01:00
Mario de Frutos
708bee2912 Added metrics context manager to public functions 2016-11-04 13:39:10 +01:00
Mario de Frutos
cf3c6f2ce5 New metrics logger system 
- It's not mandatory to have the log_path defined for all the services
- Now we add metrics to a MetricsGatherer using the add method in order
  to gather data from all the function scope, not just in one defined
  point
- There is a `metrics` context manager to wrap the function block. This
  context manager get the execution time for the block among other
  things
 2016-11-02 15:26:39 +01:00
Rafa de la Torre
35da7e48fd Implement a ServerConfigBackendFactory 
mostly to keep layers separated.
 2016-10-03 17:42:19 +02:00
Rafa de la Torre
0d87a95270 Inject the environment into the LoggerConfigBuilder 2016-10-03 17:06:57 +02:00
Rafa de la Torre
fcca5da302 Add a RedisMetricsConnectionFactory class 2016-10-03 16:47:45 +02:00
Rafa de la Torre
1aec541906 Add ServerEnvironment and ServerEnvironmentBuilder 2016-10-03 16:21:28 +02:00
Rafa de la Torre
9e98e0794d Add backends for user and org configs 2016-10-03 16:05:08 +02:00
Rafa de la Torre
8fbb41742c Rename redis conf factories to builders 2016-10-03 15:25:01 +02:00
Rafa de la Torre
275a6dc27f Move redis_config related classes to separate file 2016-10-03 13:33:46 +02:00
Rafa de la Torre
073163eb1a Rename redis_config.py to redis_connection_config.py 2016-10-03 13:20:30 +02:00
Rafa de la Torre
86ab3abc53 Rename mapzen_geocoder module to mapzen_geocoder_config 2016-10-03 12:58:15 +02:00
Rafa de la Torre
b1f3405cd0 Rename variable s/user_geocoder_config/mapzen_geocoder_config 2016-10-03 12:52:18 +02:00
Carla Iriberri
6c61626214 Remove RedisMock TODO 2016-09-30 12:29:00 +02:00
Carla Iriberri
74d2fba763 Add Redis Mock -- tested and working 2016-09-30 12:28:27 +02:00
Rafa de la Torre
e24819f193 Take the environment into account 2016-09-30 11:29:44 +02:00
Rafa de la Torre
3a6cc4c364 Add mapzen config and integrate into legacy code 2016-09-29 18:41:16 +02:00
Rafa de la Torre
8ad2434b1d Add instantiation of configs to mapzen geocoder 2016-09-29 17:47:36 +02:00
Rafa de la Torre
02a2619b45 Fix instantiation of redis metadata connection 2016-09-29 17:19:18 +02:00
Rafa de la Torre
1f3a655ae5 Add all the code to instantiate a user_config_storage (WIP) 2016-09-29 16:57:31 +02:00
Rafa de la Torre
efdc151282 Add things to get logger config from storage 2016-09-28 18:22:51 +02:00
Carla
fc99f7aba9 Merge pull request #267 from CartoDB/augment_revamp_analysis 
Table-level OBS_GetMeasue revamp
 2016-09-01 17:21:56 +02:00
Carla Iriberri
1e9b551160 Add populate table function and several fixes 2016-08-30 16:38:44 +02:00
Carla Iriberri
fc291a7c63 First steps 2016-08-29 18:07:58 +02:00
Mario de Frutos
18ae2525b6 Add search type filter for Mapzen geocoder 2016-08-29 14:32:49 +02:00
Mario de Frutos
06462fdf7a Control file for version 0.15 and empty upgrade/downgrade 2016-08-29 14:04:08 +02:00
Mario de Frutos
71d5ce951a Use mapzen as first option for the namedplace geocoding 2016-08-26 16:39:25 +02:00
Carla Iriberri
19b3e05661 Remove default intersect for OBS functions 2016-08-10 16:25:26 +02:00
John Krauss
d63f2ba4d5 Replace all 'area' defaults for with NULL. Fixes #235 2016-08-10 16:25:26 +02:00
Mario de Frutos
7f2f6e7d99 Change notice for info in the logger method 2016-08-04 17:54:57 +02:00
Mario de Frutos
5f8416d166 Add logger config function with proper permission to get logger config 2016-08-03 17:37:40 +02:00
Mario de Frutos
a33999ed04 Dont re-raise the exception to avoid leak sensitive information 2016-08-03 12:08:23 +02:00
Mario de Frutos
9186eb5d5b Log part decoupled from the main config class 2016-08-02 18:40:01 +02:00
Mario de Frutos
ed5cf25e9c Added new logger to all the functions 2016-08-02 16:44:03 +02:00
Mario de Frutos
6349be476d Remove 1 row limit for isolines functions 2016-07-29 17:00:08 +02:00
Mario de Frutos
fc399c12c1 Change notices for debug 2016-07-25 13:58:01 +02:00
Mario de Frutos
79d0b5ba7c Add service providers that come from the user configuration 2016-07-25 12:25:52 +02:00
Mario de Frutos
69548eaa25 Fixed return error in case NULL value is returned for isolines shape 2016-07-22 14:55:01 +02:00
Carla Iriberri
39a7cc955a Move IP address reading to DS server 2016-07-15 11:54:38 +02:00
Carla Iriberri
5bc2d974ac Adding OBS_DumpVersion function 2016-07-11 18:25:39 +02:00
Rafa de la Torre
554af3075e Avoid unneeded mapping/translation of rows 2016-07-11 16:57:05 +02:00
Rafa de la Torre
9f4df6fa7d Return None/null instead of crashing 
When there's no enough information to produce an isoline (less than 3
points) return a NULL multipolygon to the upper layer. This usually
happens when the matrix API returns null cost for most points in the
request.
 2016-07-11 16:57:05 +02:00
Carla Iriberri
7b1132b4d2 Fix conflicts take 4 2016-07-11 12:18:07 +02:00
Carla Iriberri
99d21f9a84 bring master to fix versioning issues 2016-07-11 11:29:06 +02:00
Rafa de la Torre
9b7a2d491f Fix bug adapting types passing through plpython 2016-07-06 19:58:04 +02:00
Rafa de la Torre
99798f2618 Integrate isodistance into SQL API 2016-07-06 19:40:40 +02:00
Carla Iriberri
a046d3ce97 Add Mapzen Matrix to config and metrics services 2016-07-06 13:19:28 +02:00
Rafa de la Torre
eb906fae35 Convert to multipolygon and return isolines 2016-07-06 13:18:54 +02:00
Rafa de la Torre
6810dc0ff0 Code to glue together pg and python (WIP) 2016-07-05 20:56:15 +02:00
Rafa de la Torre
53fe4ce21d An attempt to adapt paremetrs (WIP) 2016-07-05 18:48:21 +02:00
Rafa de la Torre
40cacd99dc Some code trying to pull everything together (WIP) 2016-07-05 18:48:21 +02:00
Carla Iriberri
4075e7349b Add new provider functions in geocode street 2016-07-05 11:53:46 +02:00
Carla Iriberri
6d35cff9c7 Exposes geocoder providers in public geocoder functions 
Add config function in postgres explicitly to get MapzenGeocoderConfig.
Bump versions for client and server APIs. New MapzenGeocoderConfig
included to be able to use current QuotaServices with non-configured
users.
 2016-07-05 11:13:43 +02:00
Carla Iriberri
ffe44ce94e Fully qualify, several fixes and variable renaming 2016-07-01 12:23:57 +02:00
Rafa de la Torre
4b72af34ec Fixes to be able to install the extension 2016-07-01 12:18:49 +02:00
Rafa de la Torre
d517c62e6f Make cdb_dataservices_server.OBS_GetProcessedData "public" 2016-07-01 12:18:49 +02:00
Rafa de la Torre
1ed02c69bc Qualify types with the schema name 2016-07-01 12:18:49 +02:00
Carla
6c627fb207 Add augment functions 2016-07-01 12:18:11 +02:00
Mario de Frutos
770ff0eb95 Server side part for function obs_getmeasurebyid 2016-05-30 18:13:45 +02:00
Carla Iriberri
21aac960a6 Add function cdb_routing_with_waypoints 
Add routing with waypoints functions to client and server. Includes
signature checks tests for the Postgresql functions and unit and
integration tests for the Python library.

Add client v0.6.0 and server v0.9.0
 2016-05-25 11:38:37 +02:00
Mario de Frutos
5d6c3d7b11 Maintain old snapshot functions to avoid break compatibility 2016-05-16 10:25:27 +02:00
Mario de Frutos
8680c9cbd0 We let the old qlik snapshots functions but we make new ones with the new snapshot part from observatory' 2016-05-16 10:25:27 +02:00
Mario de Frutos
f59217779e New integration tests for data observatory functions 2016-05-16 10:25:27 +02:00
Mario de Frutos
c7c9e6e284 Server side new data observatory functions (v0.8.0) 2016-05-16 10:25:26 +02:00
Mario de Frutos
3e09605757 On 4xx we have to return an empty response instead of an exception 2016-05-10 16:46:38 +02:00
Mario de Frutos
0735bbcb41 Change loopback to observatory just to call the observatory extension functions 2016-05-09 18:25:44 +02:00
Mario de Frutos
e81dadaf2e Green/Blue deploy system for the observatory functions 2016-05-03 13:49:14 +02:00
Mario de Frutos
4407ff630e Change data observatory config file to segragate more in order to prepare for future observatory elements 2016-04-25 10:03:39 +02:00
Mario de Frutos
1daf78e41a Integration tests for the data observatory 2016-04-20 15:58:39 +02:00
Mario de Frutos
3c5325c1d7 Server data observatory functions 2016-04-20 15:58:38 +02:00
Mario de Frutos
c9ddc37051 Mapzen geocoder and routing quota check 2016-04-15 10:21:40 +02:00
Mario de Frutos
b4c1991c90 Mapzen geocoder checks for the iso3 for the country passed 2016-04-13 18:49:13 +02:00
Mario de Frutos
89ec3dcfa6 Mapzen geocoder integrated 2016-03-23 13:37:30 +01:00
Mario de Frutos
357458f44d Check for some issues like empty input, recoverable exceptions, etc 2016-03-22 15:19:03 +01:00
Mario de Frutos
4061009df0 4xx generated empty responses 2016-03-17 22:54:19 +01:00
Mario de Frutos
47f2eba4e4 Removed old versioning system 2016-03-17 22:49:55 +01:00
Mario de Frutos
0f262510b2 User config helper for internal geocoder too 2016-03-17 11:48:25 +01:00
Mario de Frutos
bd2e380225 Add logger for the internal geocoder too 2016-03-08 16:47:49 +01:00
Mario de Frutos
b97e838416 Refactor to move logic from SQL functions to Python 
- Moved the logic the retrieve the redis connection params to
  RedisDBConfig class
- Moved the logic that retrieve the services configuration to
  ServicesDBConfig
 2016-03-08 16:47:49 +01:00
Mario de Frutos
7237e01601 Change metrics increment naming to reflect service instead of geocoder 2016-03-07 16:03:28 +01:00
Mario de Frutos
fbd42e3db0 Scaffolding for the 0.5.1 version 2016-03-07 16:03:23 +01:00
Mario de Frutos
9a36445661 Sentinel id could be removed safely from the config 
Removing the sentinel_master_id property from the cdb_conf
redis config, the extension is going to use a single redis instance
instead of Sentinel
 2016-02-26 10:49:10 +01:00
Mario de Frutos
e6b0e3794b Mapzen integration for the server side 2016-02-26 10:49:10 +01:00
Luis Bosque
6c5b1b4e99 Use strictredis if sentinel_master_id is not null 
Also, use only a redis_host and redis_port attributes either if the
connection is to redis or to sentinel
 2016-02-26 10:49:09 +01:00
Luis Bosque
93463b3032 Support using a common redis connection 
If sentinel_host or sentinel_cluster_id is not provided it will try to
connect with a redis_host parameter
 2016-02-26 10:49:08 +01:00
Mario de Frutos
d27b2a2b78 Block google services users to use isolines functions 
Due some service restrictions, google services users couldn't use
the HERE services which includes the isolines functions.
 2016-02-26 10:42:05 +01:00
Mario de Frutos
448f47589a Renamed from routing to isolines helper 2016-02-22 10:28:19 +01:00
Mario de Frutos
e764b9036d Quota check for the routing feature 
We have refactor all the quota logic and extracted to a new QuotaChecker
class in order to have it ready to create a factory when needed

Added the logic for the routing quota check
 2016-02-18 15:40:55 +01:00
Mario de Frutos
32c146336d Change the url to production in here routing functions 2016-02-12 14:52:00 +01:00
Mario de Frutos
5c1b320555 Routing server side working properly 2016-02-10 22:26:32 +01:00
Carla Iriberri
d0f7ac4d49 Fix conflict 2016-02-10 20:00:48 +01:00
Carla Iriberri
d4a2184ce0 Multiple fixes 2016-02-10 19:59:00 +01:00
Carla
42d0812383 Fix indents 2016-02-10 17:58:48 +01:00
Carla Iriberri
e996d3796a Comment quota code 2016-02-10 17:42:11 +01:00
Carla Iriberri
bd3f6678c1 Upload server functions for routing 2016-02-10 17:17:53 +01:00
Mario de Frutos
910381e4a6 Dataservices server 0.4.0 scaffold 2016-02-09 10:51:12 +01:00
Mario de Frutos
142affc95e Removed legacy name for street function 2016-02-08 13:27:19 +01:00
Mario de Frutos
2994a1b402 Geocoder server renamed to dataservices server extension 2016-02-05 17:57:22 +01:00
Mario de Frutos
2a9d6c14bd Added metrics storage for internal functions and trim for all the params 2016-02-05 16:00:46 +01:00
Mario de Frutos
1665cf8069 Fixed some python import calls due the refactor made 2016-02-04 20:04:46 +01:00
Mario de Frutos
db715d6d54 Added security definer permission to the sql part too 2016-02-04 15:42:22 +01:00
Mario de Frutos
1367c46957 Added automatic generated files warning header 2016-02-04 10:38:12 +01:00
Mario de Frutos
3960c13484 Refactor python library to unify and rename as cdb_services 2016-02-03 10:21:57 +01:00
Mario de Frutos
e7c58b9a51 Google geocoder working 2016-02-03 10:21:34 +01:00
Mario de Frutos
bb629dfe4b Change the signature name for the geocoder street point function 2016-01-26 16:22:28 +01:00
Mario de Frutos
343bcf539a Add security definer to connect redis function too 2016-01-26 13:15:15 +01:00
Mario de Frutos
35b59c448f Add needed security definer to config function 2016-01-26 12:55:06 +01:00
Mario de Frutos
4e53397f1c Change heremaps module to support host as parameter 2016-01-25 17:36:18 +01:00
Mario de Frutos
1a8109ecdf Quote NULL values to avoid treat the NULL as a string 2016-01-22 19:17:16 +01:00
Mario de Frutos
db3eaf31ad Versioned the server tests and created the upgrade and downgrade files 2016-01-22 15:36:16 +01:00
Mario de Frutos
984519e7f1 Change to have functions for all the geocoder types but only one function to call them 2016-01-22 10:05:46 +01:00
Mario de Frutos
4d0cabb429 Add username and orgname to the config object instead of pass them to every object 2016-01-21 18:03:23 +01:00
Mario de Frutos
7c3ab87b78 When geocoder API calls heremaps increments the usage metrics 2016-01-21 13:45:29 +01:00
Mario de Frutos
ad9c16b4df Nokia geocoder config moved to the config helper 2016-01-21 08:40:25 +01:00
Mario de Frutos
fc35911b91 Geocoder street function with quota checking 2016-01-20 20:57:24 +01:00
Mario de Frutos
b2fa8739e2 User config now is retrieved from redis 2016-01-20 20:56:19 +01:00
Mario de Frutos
9ecabf9144 Create either metrics and metadata redis connections 2016-01-20 16:02:16 +01:00
Mario de Frutos
922c531313 Added orgname to the arguments we pass to the server 2015-12-03 11:03:10 +01:00
Mario de Frutos
941e7084e6 Change functions to user username and remove JSON config passed as argument 2015-12-02 18:54:49 +01:00
Mario de Frutos
082d876c8d Create and give permissions to the server user 2015-12-01 17:23:32 +01:00
Mario de Frutos
97cab091b6 Added cdb_* prefix to all the functions 2015-11-24 15:36:52 +01:00
Mario de Frutos
2b141452b4 Changed api key verifying to client instead of server 2015-11-24 12:44:52 +01:00
Mario de Frutos
8434a830b2 Changed server geocode ip address signature to match client 2015-11-24 10:14:34 +01:00
Mario de Frutos
fa1d7af410 Deleted unused test for config tables 2015-11-23 12:53:46 +01:00
Mario de Frutos
6722f64f23 Added server specification API tests 2015-11-23 11:48:24 +01:00
Mario de Frutos
9e30bf2223 New Redis structure for services 2015-11-23 09:40:10 +01:00
Mario de Frutos
e28aa9a3a2 Removed the _conf table functions and change to use cartodb.cdb_conf 2015-11-23 09:39:47 +01:00
Mario de Frutos
e8983283da Redis connection get from sentinel master 2015-11-23 09:39:47 +01:00
Mario de Frutos
b38a9b2fd3 Added redis config logic 2015-11-23 09:39:47 +01:00
Mario de Frutos
da12d6628d Extracted redis connection and quota check to be reusable 2015-11-23 09:39:47 +01:00
Rafa de la Torre
05d75c82ee Merge pull request #25 from CartoDB/add-config-table-and-funcs 
Config table and functions for server extension
 2015-11-12 17:04:26 +01:00
Rafa de la Torre
b45aba4278 Config table and functions for server extension 
Pretty much as done in cartodb and client extensions.
 2015-11-12 13:02:51 +01:00
Carla Iriberri
e3269af48f Removes double tests for namedplaces 2015-11-12 12:49:54 +01:00