cartodb4/dataservices-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,572 Commits 1 Branch 183 Tags 3.3 MiB
3a240bf6ad
Commit Graph

259 Commits

Author SHA1 Message Date
Javier Goizueta
6c5ca97468 Complete rate-limiting for Mapzen & Here gecoding 
ServiceManager class has been introduced to handle service configuration at
SQL level (with a LegacyServiceManager alternative for non-refactored services).
These new classes take the responsibility of rate limits and quota checking.

Tests have been added for ServiceManager and rate limits, but currently they
check only the limits configuration since Lua support would be needed
to use rratelimit with MockRedis.
 2017-03-14 18:51:18 +01:00
Javier Goizueta
250d384d06 Geocoder, per user, cofigurable rate limits 
WIP, specially the GeocoderConfig part is flaky and ugly
 2017-03-09 18:42:48 +01:00
Rafa de la Torre
25ba9866ae Small PoC with rratelimit #346 2017-03-07 13:31:06 +01:00
Javier Goizueta
89762c1a7f Use service params configuration in Mapzen services 2017-03-06 15:25:11 +01:00
Javier Goizueta
29cdd8099a Add service configuration (URL, etc) for Mapzen Geocoding 2017-03-06 10:20:20 +01:00
Javier Goizueta
677d2acead Add service configuration (URL, etc) for Here Geocoding 
See #343
 2017-03-02 17:15:14 +01:00
Mario de Frutos
435a84f294 OBS_GetData functions shall increase the DO quota usage 2017-02-06 17:47:15 +01:00
Mario de Frutos
061956f250 Remove quota usage for obs_meta functions 2017-02-02 10:59:21 +01:00
John Krauss
56de523498 add wrappers for getmeta, getdata(geoms) and getdata(ids) 2017-01-26 12:27:29 +01:00
Mario de Frutos
77f4f3e7ff Mapzen isochtrones integration 2016-11-29 12:57:23 +01:00
Rafa de la Torre
9c6eabc59e Do not enforce types for params #302 
As types are tied to a schema, they don't get along very well with
pl/proxy. Do not use them for the service_type.
 2016-11-24 18:38:33 +01:00
Rafa de la Torre
769b740ba4 Rename service_params to service_quota_info #302 2016-11-24 17:46:07 +01:00
Rafa de la Torre
c2ede37d75 Revert internal geocoder related stuff #302 2016-11-24 17:45:19 +01:00
Rafa de la Torre
4c75c5f261 Revert internal geocoder related stuff #302 
This reverts commits 579d11e, 4af5975 and a2da597
 2016-11-24 17:33:06 +01:00
Rafa de la Torre
efb10e43e4 Merge remote-tracking branch 'origin/development' into 302-expose-quotas 2016-11-24 12:36:23 +01:00
Rafa de la Torre
a2da597e00 Support null quota (internal_geocoder) #302 2016-11-23 18:10:44 +01:00
Rafa de la Torre
4af5975ac1 Add the internal geocoder to cdb_service_params #302 2016-11-23 17:07:23 +01:00
Rafa de la Torre
d70d149eed Fix bug: the service_type to used_quota #302 2016-11-23 16:40:00 +01:00
Rafa de la Torre
decc7626a8 Create types only if they don't exist #302 2016-11-23 16:38:47 +01:00
Rafa de la Torre
abbb227ad5 Fix typo: use obs config for obs service #302 2016-11-23 11:40:02 +01:00
Mario de Frutos
8396eae733 Server update to type obs_meta_geometry with new types 2016-11-11 18:13:01 +01:00
Rafa de la Torre
2e6f37af56 Add observatory #302 2016-11-11 11:27:57 +01:00
Mario de Frutos
b5f4c507b3 Fixed some typos in SQL artifact v0.18 and v0.17 2016-11-11 11:13:08 +01:00
Rafa de la Torre
95dda082e7 Add routing #302 2016-11-11 10:24:03 +01:00
Rafa de la Torre
9f9034f4f1 Make cdb_enough_quota check the table got from cdb_service_params #302 2016-11-10 19:23:39 +01:00
Rafa de la Torre
603fbbbc3f Make cdb_service_params return a table for all services #302 2016-11-10 19:23:01 +01:00
Rafa de la Torre
3fa686ec65 Remove the class Service in favor of service_type ENUM #302 2016-11-10 19:21:29 +01:00
Rafa de la Torre
429a71ef37 Add a function to check for enough quota #302 2016-11-10 18:56:50 +01:00
Rafa de la Torre
b2cdb1c74b Move all into one func returning a tuple #302 2016-11-10 18:40:18 +01:00
Rafa de la Torre
84061dec4f Move everything to cdb_service_params (WIP) #302 2016-11-10 18:12:24 +01:00
Mario de Frutos
08debb83ed Server side change JSONB to JSON in the legacybuildermedatada function 2016-11-10 18:08:35 +01:00
Rafa de la Torre
7a32ace1ac Add the hires_geocoder to cdb_monthly_quota #302 2016-11-10 17:00:25 +01:00
Mario de Frutos
040b094b01 New observatory meta functions for server side 2016-11-10 11:29:19 +01:00
Rafa de la Torre
eb7188235d Provider function #302 2016-11-10 11:14:15 +01:00
Rafa de la Torre
3e059003b6 Add soft limit function #302 2016-11-10 11:09:40 +01:00
Rafa de la Torre
0e38af6ecd Add cdb_remaining_quota to server #302 2016-11-08 18:47:22 +01:00
Rafa de la Torre
6f9feb07a0 Add cdb_monthly_quota (WIP) #302 2016-11-08 18:29:06 +01:00
Mario de Frutos
352d4217bc Fixed typo in the config cache for observatory snapshot functions 2016-11-07 10:31:53 +01:00
Mario de Frutos
49fd51f6da Added logger to metrics context manager 2016-11-04 13:39:10 +01:00
Mario de Frutos
708bee2912 Added metrics context manager to public functions 2016-11-04 13:39:10 +01:00
Mario de Frutos
cf3c6f2ce5 New metrics logger system 
- It's not mandatory to have the log_path defined for all the services
- Now we add metrics to a MetricsGatherer using the add method in order
  to gather data from all the function scope, not just in one defined
  point
- There is a `metrics` context manager to wrap the function block. This
  context manager get the execution time for the block among other
  things
 2016-11-02 15:26:39 +01:00
Rafa de la Torre
35da7e48fd Implement a ServerConfigBackendFactory 
mostly to keep layers separated.
 2016-10-03 17:42:19 +02:00
Rafa de la Torre
0d87a95270 Inject the environment into the LoggerConfigBuilder 2016-10-03 17:06:57 +02:00
Rafa de la Torre
fcca5da302 Add a RedisMetricsConnectionFactory class 2016-10-03 16:47:45 +02:00
Rafa de la Torre
1aec541906 Add ServerEnvironment and ServerEnvironmentBuilder 2016-10-03 16:21:28 +02:00
Rafa de la Torre
9e98e0794d Add backends for user and org configs 2016-10-03 16:05:08 +02:00
Rafa de la Torre
8fbb41742c Rename redis conf factories to builders 2016-10-03 15:25:01 +02:00
Rafa de la Torre
275a6dc27f Move redis_config related classes to separate file 2016-10-03 13:33:46 +02:00
Rafa de la Torre
073163eb1a Rename redis_config.py to redis_connection_config.py 2016-10-03 13:20:30 +02:00
Rafa de la Torre
86ab3abc53 Rename mapzen_geocoder module to mapzen_geocoder_config 2016-10-03 12:58:15 +02:00
Rafa de la Torre
b1f3405cd0 Rename variable s/user_geocoder_config/mapzen_geocoder_config 2016-10-03 12:52:18 +02:00
Carla Iriberri
6c61626214 Remove RedisMock TODO 2016-09-30 12:29:00 +02:00
Carla Iriberri
74d2fba763 Add Redis Mock -- tested and working 2016-09-30 12:28:27 +02:00
Rafa de la Torre
e24819f193 Take the environment into account 2016-09-30 11:29:44 +02:00
Rafa de la Torre
3a6cc4c364 Add mapzen config and integrate into legacy code 2016-09-29 18:41:16 +02:00
Rafa de la Torre
8ad2434b1d Add instantiation of configs to mapzen geocoder 2016-09-29 17:47:36 +02:00
Rafa de la Torre
02a2619b45 Fix instantiation of redis metadata connection 2016-09-29 17:19:18 +02:00
Rafa de la Torre
1f3a655ae5 Add all the code to instantiate a user_config_storage (WIP) 2016-09-29 16:57:31 +02:00
Rafa de la Torre
efdc151282 Add things to get logger config from storage 2016-09-28 18:22:51 +02:00
Carla
fc99f7aba9 Merge pull request #267 from CartoDB/augment_revamp_analysis 
Table-level OBS_GetMeasue revamp
 2016-09-01 17:21:56 +02:00
Carla Iriberri
1e9b551160 Add populate table function and several fixes 2016-08-30 16:38:44 +02:00
Carla Iriberri
fc291a7c63 First steps 2016-08-29 18:07:58 +02:00
Mario de Frutos
18ae2525b6 Add search type filter for Mapzen geocoder 2016-08-29 14:32:49 +02:00
Mario de Frutos
06462fdf7a Control file for version 0.15 and empty upgrade/downgrade 2016-08-29 14:04:08 +02:00
Mario de Frutos
71d5ce951a Use mapzen as first option for the namedplace geocoding 2016-08-26 16:39:25 +02:00
Carla Iriberri
19b3e05661 Remove default intersect for OBS functions 2016-08-10 16:25:26 +02:00
John Krauss
d63f2ba4d5 Replace all 'area' defaults for with NULL. Fixes #235 2016-08-10 16:25:26 +02:00
Mario de Frutos
7f2f6e7d99 Change notice for info in the logger method 2016-08-04 17:54:57 +02:00
Mario de Frutos
5f8416d166 Add logger config function with proper permission to get logger config 2016-08-03 17:37:40 +02:00
Mario de Frutos
a33999ed04 Dont re-raise the exception to avoid leak sensitive information 2016-08-03 12:08:23 +02:00
Mario de Frutos
9186eb5d5b Log part decoupled from the main config class 2016-08-02 18:40:01 +02:00
Mario de Frutos
ed5cf25e9c Added new logger to all the functions 2016-08-02 16:44:03 +02:00
Mario de Frutos
6349be476d Remove 1 row limit for isolines functions 2016-07-29 17:00:08 +02:00
Mario de Frutos
fc399c12c1 Change notices for debug 2016-07-25 13:58:01 +02:00
Mario de Frutos
79d0b5ba7c Add service providers that come from the user configuration 2016-07-25 12:25:52 +02:00
Mario de Frutos
69548eaa25 Fixed return error in case NULL value is returned for isolines shape 2016-07-22 14:55:01 +02:00
Carla Iriberri
39a7cc955a Move IP address reading to DS server 2016-07-15 11:54:38 +02:00
Carla Iriberri
5bc2d974ac Adding OBS_DumpVersion function 2016-07-11 18:25:39 +02:00
Rafa de la Torre
554af3075e Avoid unneeded mapping/translation of rows 2016-07-11 16:57:05 +02:00
Rafa de la Torre
9f4df6fa7d Return None/null instead of crashing 
When there's no enough information to produce an isoline (less than 3
points) return a NULL multipolygon to the upper layer. This usually
happens when the matrix API returns null cost for most points in the
request.
 2016-07-11 16:57:05 +02:00
Carla Iriberri
7b1132b4d2 Fix conflicts take 4 2016-07-11 12:18:07 +02:00
Carla Iriberri
99d21f9a84 bring master to fix versioning issues 2016-07-11 11:29:06 +02:00
Rafa de la Torre
9b7a2d491f Fix bug adapting types passing through plpython 2016-07-06 19:58:04 +02:00
Rafa de la Torre
99798f2618 Integrate isodistance into SQL API 2016-07-06 19:40:40 +02:00
Carla Iriberri
a046d3ce97 Add Mapzen Matrix to config and metrics services 2016-07-06 13:19:28 +02:00
Rafa de la Torre
eb906fae35 Convert to multipolygon and return isolines 2016-07-06 13:18:54 +02:00
Rafa de la Torre
6810dc0ff0 Code to glue together pg and python (WIP) 2016-07-05 20:56:15 +02:00
Rafa de la Torre
53fe4ce21d An attempt to adapt paremetrs (WIP) 2016-07-05 18:48:21 +02:00
Rafa de la Torre
40cacd99dc Some code trying to pull everything together (WIP) 2016-07-05 18:48:21 +02:00
Carla Iriberri
4075e7349b Add new provider functions in geocode street 2016-07-05 11:53:46 +02:00
Carla Iriberri
6d35cff9c7 Exposes geocoder providers in public geocoder functions 
Add config function in postgres explicitly to get MapzenGeocoderConfig.
Bump versions for client and server APIs. New MapzenGeocoderConfig
included to be able to use current QuotaServices with non-configured
users.
 2016-07-05 11:13:43 +02:00
Carla Iriberri
ffe44ce94e Fully qualify, several fixes and variable renaming 2016-07-01 12:23:57 +02:00
Rafa de la Torre
4b72af34ec Fixes to be able to install the extension 2016-07-01 12:18:49 +02:00
Rafa de la Torre
d517c62e6f Make cdb_dataservices_server.OBS_GetProcessedData "public" 2016-07-01 12:18:49 +02:00
Rafa de la Torre
1ed02c69bc Qualify types with the schema name 2016-07-01 12:18:49 +02:00
Carla
6c627fb207 Add augment functions 2016-07-01 12:18:11 +02:00
Mario de Frutos
770ff0eb95 Server side part for function obs_getmeasurebyid 2016-05-30 18:13:45 +02:00
Carla Iriberri
21aac960a6 Add function cdb_routing_with_waypoints 
Add routing with waypoints functions to client and server. Includes
signature checks tests for the Postgresql functions and unit and
integration tests for the Python library.

Add client v0.6.0 and server v0.9.0
 2016-05-25 11:38:37 +02:00
Mario de Frutos
5d6c3d7b11 Maintain old snapshot functions to avoid break compatibility 2016-05-16 10:25:27 +02:00
Mario de Frutos
8680c9cbd0 We let the old qlik snapshots functions but we make new ones with the new snapshot part from observatory' 2016-05-16 10:25:27 +02:00