cartodb4/dataservices-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,923 Commits 1 Branch 183 Tags 3.3 MiB
cd366d7589
Commit Graph

524 Commits

Author SHA1 Message Date
Rafa de la Torre
c1dd410201 Make RedisConnectionConfigBuilder abstract 
and also use custom ConfigException instead of generic one.
 2016-10-03 12:21:45 +02:00
Rafa de la Torre
34ddd28e6b Move ConfigException to a separate file 2016-10-03 12:20:17 +02:00
Carla
443fe88d5a Add newline to end fle 2016-09-30 13:08:27 +02:00
Carla Iriberri
74d2fba763 Add Redis Mock -- tested and working 2016-09-30 12:28:27 +02:00
Rafa de la Torre
1e6ee8d5c1 Add an Environment class 2016-09-30 11:21:35 +02:00
Rafa de la Torre
3a6cc4c364 Add mapzen config and integrate into legacy code 2016-09-29 18:41:16 +02:00
Rafa de la Torre
0b7b44d8a5 Add a couple of factories to abstract user/org configs 2016-09-29 17:46:17 +02:00
Rafa de la Torre
4b4a02905c Implement the RedisConfigStorage 2016-09-29 17:18:45 +02:00
Rafa de la Torre
1f3a655ae5 Add all the code to instantiate a user_config_storage (WIP) 2016-09-29 16:57:31 +02:00
Rafa de la Torre
9d60fde0b8 A stubb for RedisConfigStorage (WIP) 2016-09-29 13:07:40 +02:00
Rafa de la Torre
efdc151282 Add things to get logger config from storage 2016-09-28 18:22:51 +02:00
Rafa de la Torre
fd2cc21942 Add server config storage classes 2016-09-28 17:32:26 +02:00
Rafa de la Torre
18f05fbd4f User class with tests 2016-09-28 16:30:37 +02:00
Carla Iriberri
96a93e3c56 Python package version 0.8.1 2016-09-28 15:09:29 +02:00
Rafa de la Torre
64fc18b9e0 Fix and improve test speed 2016-09-28 13:43:50 +02:00
Mario de Frutos
9381d5644b Fixed QPS retry decorator 2016-09-16 13:46:02 +02:00
Carla
9f55f2ee3b Update README.md 2016-09-09 11:44:57 +02:00
Mario de Frutos
18ae2525b6 Add search type filter for Mapzen geocoder 2016-08-29 14:32:49 +02:00
Mario de Frutos
71d5ce951a Use mapzen as first option for the namedplace geocoding 2016-08-26 16:39:25 +02:00
Mario de Frutos
d602c42559 Strip spaces in the mapzen geocode parameters 2016-08-19 13:32:11 +02:00
Mario de Frutos
3cc4e06420 Add more logs for mapzen services 2016-08-19 11:44:59 +02:00
Mario de Frutos
0fb3cdd8b1 JSON response is not always present, better show the reponse object complete 2016-08-05 11:35:16 +02:00
Mario de Frutos
7cca8e95c3 Remove testing exception 2016-08-03 14:45:57 +02:00
Mario de Frutos
a33999ed04 Dont re-raise the exception to avoid leak sensitive information 2016-08-03 12:08:23 +02:00
Mario de Frutos
9186eb5d5b Log part decoupled from the main config class 2016-08-02 18:40:01 +02:00
Mario de Frutos
ed5cf25e9c Added new logger to all the functions 2016-08-02 16:44:03 +02:00
Mario de Frutos
5d5e8d6f9a Added logger with rollbar integration 2016-08-01 18:03:42 +02:00
Mario de Frutos
993059eafb Added server config with environment data 2016-08-01 14:00:32 +02:00
Mario de Frutos
f9f39dcf9c Refactor metrics logger 2016-08-01 13:17:06 +02:00
Mario de Frutos
4d47b905e8 Revert include the gateway error in QPS 2016-08-01 11:23:06 +02:00
Mario de Frutos
c23a57c03f Add QPS retry for mapzen isolines and add the 504 error to retry 2016-07-29 16:22:34 +02:00
Mario de Frutos
efde2ee1b7 Fix typo in the default value for service providers 2016-07-25 14:00:29 +02:00
Mario de Frutos
fc399c12c1 Change notices for debug 2016-07-25 13:58:01 +02:00
Mario de Frutos
f03897257e Better than lower, change to empty string if not provider setting in redis 2016-07-25 13:40:06 +02:00
Mario de Frutos
90998d4819 Fix users without provider setting setted 2016-07-25 13:35:59 +02:00
Mario de Frutos
79d0b5ba7c Add service providers that come from the user configuration 2016-07-25 12:25:52 +02:00
Mario de Frutos
ff8eed9750 Fix empty component return in isolines 2016-07-22 14:54:53 +02:00
Carla
6fb9f67e64 Update README.md 2016-07-11 10:50:19 +02:00
Carla Iriberri
aff5dd13d7 Bump python library to 0.7.0 2016-07-07 10:04:54 +02:00
Carla Iriberri
1022097300 Bump server to 0.12.0 2016-07-07 09:58:51 +02:00
Carla
5c20866277 Merge pull request #216 from CartoDB/mapzen-isolines-reloaded 
Add mapzen isolines
 2016-07-07 09:31:04 +02:00
Rafa de la Torre
f5d51da673 Fix another typo (hello Carla!!) 
This feature is dedicated to you. Keep rocking.
 2016-07-06 20:29:08 +02:00
Rafa de la Torre
bcc6bc35d3 Fix None*unit_factor error 
Also make the code more explicit about what happens when getting cost ==
None.
 2016-07-06 19:48:23 +02:00
Rafa de la Torre
230112b7e5 Add calculate_isodistance function 2016-07-06 19:20:21 +02:00
Rafa de la Torre
523eda2cc7 Generalize calculate_isochrone to calculate_isoline 2016-07-06 18:43:09 +02:00
Rafa de la Torre
54221fa671 Add transport mode car 2016-07-06 18:05:51 +02:00
Rafa de la Torre
6d888a7a62 Fix for points getting None cost 
Sometimes there's no route information for the point in a particular
angle we're interested in. In this case it is better to use more
points/angles and discard the ones we're not interested in.
 2016-07-06 18:03:24 +02:00
Rafa de la Torre
075edf0e0d More precission for earth's radius 2016-07-06 18:01:33 +02:00
Rafa de la Torre
6c4829df01 Small refactor for sanity 2016-07-06 16:05:51 +02:00
Carla Iriberri
7ddb3da60d Remove useless cost_per_hit line 2016-07-06 15:43:26 +02:00
Carla Iriberri
ff4eb5b348 Mock mapzen matrix config 2016-07-06 14:21:32 +02:00
Rafa de la Torre
2147d190a1 Unit test for the mapzen isolines 2016-07-06 13:19:28 +02:00
Carla Iriberri
a046d3ce97 Add Mapzen Matrix to config and metrics services 2016-07-06 13:19:28 +02:00
Rafa de la Torre
cdcac2dc1f Fix typo in test case 2016-07-06 13:19:28 +02:00
Carla
e9346faf42 Fix bug on exception raise 2016-07-06 12:33:59 +02:00
Rafa de la Torre
b78bd05754 Be resilient to None cost estimation 2016-07-05 20:52:41 +02:00
Rafa de la Torre
77cdc3d8ff Only refine individual solutions when error > TOLERANCE 2016-07-05 18:48:21 +02:00
Rafa de la Torre
2d95601c5a Fix: max_abs_error should be a scalar 2016-07-05 18:48:21 +02:00
Rafa de la Torre
9a9f35d9c2 Fix silly typos spotted by jgoizueta (WIP) 2016-07-05 18:48:21 +02:00
Rafa de la Torre
9becf1adb4 Iterative part of the algorithm (WIP) 2016-07-05 18:48:21 +02:00
Rafa de la Torre
87413255af Major rewrite of MapzenIsolines (WIP) 2016-07-05 18:48:21 +02:00
Rafa de la Torre
46971fe96f Raise error when response not OK 2016-07-05 18:48:21 +02:00
Rafa de la Torre
96199b0d6d Add example to code doc 2016-07-05 18:48:21 +02:00
Rafa de la Torre
a70560e566 Minimal Mapzen Time-Distance Matrix client 2016-07-05 18:48:21 +02:00
Rafa de la Torre
53fe4ce21d An attempt to adapt paremetrs (WIP) 2016-07-05 18:48:21 +02:00
Mario de Frutos
893b8db374 First stage is calculating the matrix of points 2016-07-05 18:48:21 +02:00
Carla Iriberri
6d35cff9c7 Exposes geocoder providers in public geocoder functions 
Add config function in postgres explicitly to get MapzenGeocoderConfig.
Bump versions for client and server APIs. New MapzenGeocoderConfig
included to be able to use current QuotaServices with non-configured
users.
 2016-07-05 11:13:43 +02:00
Rafa de la Torre
9ed059f4cc Bump server python lib version to 0.6.4 #204 2016-06-14 18:42:29 +02:00
Rafa de la Torre
972970a12d Use DAY_OF_MONTH_ZERO_PADDED elsewhere #204 2016-06-14 15:25:38 +02:00
Rafa de la Torre
37bcaeada3 Fix test_orgs_should_write_zero_padded_dates (WIP) #204 2016-06-14 13:18:42 +02:00
Rafa de la Torre
25bf9b6372 Add test_orgs_should_write_zero_padded_dates #204 2016-06-14 13:18:34 +02:00
Rafa de la Torre
f76a5cdfcc Fix test_should_write_zero_padded_dates #204 2016-06-14 10:13:14 +02:00
Rafa de la Torre
e046ca2c4d Add test_should_write_zero_padded_dates #204 2016-06-14 10:04:17 +02:00
Rafa de la Torre
c9f57259be Instructions on how to run integration tests #204 2016-06-13 19:05:31 +02:00
Rafa de la Torre
41b9db4144 Update python lib to 0.6.3 #201 2016-06-13 17:21:15 +02:00
Rafa de la Torre
25d42a75e4 test_should_not_request_redis_twice_when_unneeded #201 2016-06-13 15:45:56 +02:00
Rafa de la Torre
f9e9617d6f test_should_sum_amounts_from_both_key_formats #201 2016-06-13 13:10:07 +02:00
Rafa de la Torre
b0bb60736c Implement test and fix for that case #201 
Add test_should_account_for_zero_paddded_keys plus its corresponding fix
for it.
 2016-06-13 13:08:35 +02:00
Rafa de la Torre
a2863d7135 Allow for the manipulation of datetime.date.today() #201 2016-06-13 12:57:00 +02:00
Rafa de la Torre
a114f857fd Add the first test that makes sense #201 2016-06-13 12:21:06 +02:00
Rafa de la Torre
47962c9ccb Stub tests to be implemented (WIP) #201 2016-06-13 11:33:14 +02:00
Rafa de la Torre
901f5d7b8b Add a README.md file for dummies like me #201 2016-06-13 11:19:14 +02:00
Rafa de la Torre
c7bcbddaa9 Delete copy/pasted comment from requirements.txt #201 2016-06-13 11:16:49 +02:00
Rafa de la Torre
9ad55a4d53 Update url of pip package 2016-06-10 13:03:09 +02:00
Carla Iriberri
21aac960a6 Add function cdb_routing_with_waypoints 
Add routing with waypoints functions to client and server. Includes
signature checks tests for the Postgresql functions and unit and
integration tests for the Python library.

Add client v0.6.0 and server v0.9.0
 2016-05-25 11:38:37 +02:00
Carla
f46305de6f Update test_helper.py 2016-05-24 13:18:43 +02:00
Mario de Frutos
bb17b3bff7 Hotifix: obs_general_quota logic missing in the redis config 2016-05-16 14:19:08 +02:00
Mario de Frutos
c7c9e6e284 Server side new data observatory functions (v0.8.0) 2016-05-16 10:25:26 +02:00
Mario de Frutos
3e09605757 On 4xx we have to return an empty response instead of an exception 2016-05-10 16:46:38 +02:00
Mario de Frutos
e81dadaf2e Green/Blue deploy system for the observatory functions 2016-05-03 13:49:14 +02:00
Mario de Frutos
a9c420ba7b Use redis or db data observatory quota value 2016-04-25 09:33:18 +02:00
Mario de Frutos
3c5325c1d7 Server data observatory functions 2016-04-20 15:58:38 +02:00
Mario de Frutos
1e30b21d35 Fixed tests with the isolines changes 2016-04-18 18:37:46 +02:00
Mario de Frutos
15a7ed5a09 Fixed isolines overwrite for organizations 2016-04-18 11:49:11 +02:00
Mario de Frutos
8cd94973ff Version 0.6.2 2016-04-15 14:43:45 +02:00
Mario de Frutos
c9ddc37051 Mapzen geocoder and routing quota check 2016-04-15 10:21:40 +02:00
Mario de Frutos
b4c1991c90 Mapzen geocoder checks for the iso3 for the country passed 2016-04-13 18:49:13 +02:00
Mario de Frutos
a8dd211acf Use retry on timeout flag for Redis connections 
In order to avoid some fails due some high load peaks in our Redis
databases, we are going to active this flag that retries 1 more time
in case it receives a socket.Timeout error while reading or connecting
 2016-04-04 10:17:59 +02:00
Carla Iriberri
752d8317a2 Merge master 2016-04-01 10:18:46 +02:00
Carla
8c5a4d32de Update setup.py 2016-04-01 10:15:27 +02:00
Mario de Frutos
d625b26e57 Fixed timeout and random problems 2016-04-01 08:45:40 +02:00
Carla
62da0eff21 Update test_helper.py 2016-03-31 16:45:27 +02:00
Carla Iriberri
1505e0e6ea Make isoline credentials independent 2016-03-31 15:52:52 +02:00
Mario de Frutos
89ec3dcfa6 Mapzen geocoder integrated 2016-03-23 13:37:30 +01:00
Mario de Frutos
357458f44d Check for some issues like empty input, recoverable exceptions, etc 2016-03-22 15:19:03 +01:00
Mario de Frutos
4061009df0 4xx generated empty responses 2016-03-17 22:54:19 +01:00
Mario de Frutos
b9f18dda69 Added exception info for config retrieving 2016-03-17 11:36:52 +01:00
Mario de Frutos
7acf70131b Change db and timeout attributes too 2016-03-17 11:01:56 +01:00
Mario de Frutos
0e44910165 Change to sentinel_id 2016-03-17 10:58:49 +01:00
Mario de Frutos
1c33a72fdc Change sentinel attribute to use the config one 2016-03-17 10:56:25 +01:00
Mario de Frutos
5d57624fd5 Merge pull request #107 from CartoDB/rename_and_metrics 
Rename and metrics
 2016-03-17 10:23:14 +01:00
Mario de Frutos
f0a49d5fbc Deleted the used_credits field 2016-03-17 10:15:12 +01:00
Mario de Frutos
60f17fdb09 QPS manager for Mapzen 2016-03-14 11:21:26 +01:00
Mario de Frutos
bd2e380225 Add logger for the internal geocoder too 2016-03-08 16:47:49 +01:00
Mario de Frutos
c293b55c3c Added log and cost_per_hit config to the db config 2016-03-08 16:47:49 +01:00
Mario de Frutos
b97e838416 Refactor to move logic from SQL functions to Python 
- Moved the logic the retrieve the redis connection params to
  RedisDBConfig class
- Moved the logic that retrieve the services configuration to
  ServicesDBConfig
 2016-03-08 16:47:49 +01:00
Mario de Frutos
ac1627b5c5 Logger for geocoder metrics 2016-03-08 16:47:48 +01:00
Mario de Frutos
7237e01601 Change metrics increment naming to reflect service instead of geocoder 2016-03-07 16:03:28 +01:00
Mario de Frutos
df08b27068 Removed dependency with polyline and use our implementation 2016-03-02 10:49:02 +01:00
Mario de Frutos
4d04882c18 Polyline decoder 2016-03-02 10:43:27 +01:00
Mario de Frutos
d2e73a69fa Python library supports Mapzen routing 2016-02-26 10:49:10 +01:00
Luis Bosque
6c5b1b4e99 Use strictredis if sentinel_master_id is not null 
Also, use only a redis_host and redis_port attributes either if the
connection is to redis or to sentinel
 2016-02-26 10:49:09 +01:00
Luis Bosque
93463b3032 Support using a common redis connection 
If sentinel_host or sentinel_cluster_id is not provided it will try to
connect with a redis_host parameter
 2016-02-26 10:49:08 +01:00
Mario de Frutos
d27b2a2b78 Block google services users to use isolines functions 
Due some service restrictions, google services users couldn't use
the HERE services which includes the isolines functions.
 2016-02-26 10:42:05 +01:00
Mario de Frutos
e764b9036d Quota check for the routing feature 
We have refactor all the quota logic and extracted to a new QuotaChecker
class in order to have it ready to create a factory when needed

Added the logic for the routing quota check
 2016-02-18 15:40:55 +01:00
Mario de Frutos
3a4130282e Limit the mode types accepted 2016-02-12 10:01:42 +01:00
Mario de Frutos
f36a345db2 Updated python library version with routing part 2016-02-10 22:26:53 +01:00
Carla Iriberri
d4a2184ce0 Multiple fixes 2016-02-10 19:59:00 +01:00
Mario de Frutos
54622d5c81 Added the routing config and metrics 2016-02-10 17:01:41 +01:00
Mario de Frutos
75a915019f Helper to transform to Postgis multipolygon 2016-02-10 16:01:03 +01:00
Mario de Frutos
13cfee2d4c Fixed tests for the heremaps geocoder 2016-02-10 15:24:25 +01:00
Mario de Frutos
3b0f92d5e7 Routing isolines python library 2016-02-10 15:11:45 +01:00
Mario de Frutos
17b8b5942e Refactor to make the service config extensible to other services 2016-02-05 10:18:21 +01:00
Mario de Frutos
3960c13484 Refactor python library to unify and rename as cdb_services 2016-02-03 10:21:57 +01:00
Mario de Frutos
e7c58b9a51 Google geocoder working 2016-02-03 10:21:34 +01:00
Mario de Frutos
e73736cbec Restrict the python-dateutil library to avoid conflict with other projects that need that specific version 2016-01-26 19:57:18 +01:00
Mario de Frutos
4e53397f1c Change heremaps module to support host as parameter 2016-01-25 17:36:18 +01:00
Mario de Frutos
fffcef90a5 Check for the mandatories configurations in Redis before proceed 2016-01-25 08:17:03 +01:00
Mario de Frutos
e46600325b Use the real heremaps url 2016-01-22 16:39:55 +01:00
Mario de Frutos
984519e7f1 Change to have functions for all the geocoder types but only one function to call them 2016-01-22 10:05:46 +01:00
Mario de Frutos
c74947b0a5 Cartodb geocoder python module tests fixed 2016-01-22 10:05:40 +01:00
Mario de Frutos
aced68096b Heremaps tests fixed 2016-01-21 18:07:07 +01:00
Mario de Frutos
8a51558068 We need to have one date at least to check the quota 2016-01-21 18:06:04 +01:00
Mario de Frutos
3488d3f990 Pass the amount always 2016-01-21 18:05:34 +01:00
Mario de Frutos
111804dafc Refactor of the google geocoder keys in the config object 2016-01-21 18:05:08 +01:00
Mario de Frutos
af051e2ed5 Set the soft geocoding limit always 2016-01-21 18:04:08 +01:00
Mario de Frutos
4d0cabb429 Add username and orgname to the config object instead of pass them to every object 2016-01-21 18:03:23 +01:00
Mario de Frutos
7c3ab87b78 When geocoder API calls heremaps increments the usage metrics 2016-01-21 13:45:29 +01:00
Mario de Frutos
ad9c16b4df Nokia geocoder config moved to the config helper 2016-01-21 08:40:25 +01:00
Mario de Frutos
fc35911b91 Geocoder street function with quota checking 2016-01-20 20:57:24 +01:00
Mario de Frutos
b2fa8739e2 User config now is retrieved from redis 2016-01-20 20:56:19 +01:00
Mario de Frutos
d2a096907e Removed unused examples for python geocoder library 2015-11-23 09:50:10 +01:00
Mario de Frutos
9e30bf2223 New Redis structure for services 2015-11-23 09:40:10 +01:00
Mario de Frutos
928e33b489 Added config helper for user and geocoder config 2015-11-23 09:39:47 +01:00
Mario de Frutos
e8983283da Redis connection get from sentinel master 2015-11-23 09:39:47 +01:00
Mario de Frutos
b38a9b2fd3 Added redis config logic 2015-11-23 09:39:47 +01:00
Mario de Frutos
da12d6628d Extracted redis connection and quota check to be reusable 2015-11-23 09:39:47 +01:00
Mario de Frutos
6b9a8537c6 Added new requeriments 2015-11-11 18:46:46 +01:00
Mario de Frutos
6a75eae03c Added soft/hard geocoder limit 2015-11-11 13:17:15 +01:00
Mario de Frutos
a18bfa9554 Added quota service tests 2015-11-11 12:31:00 +01:00
Mario de Frutos
fbd48135b2 Added tests for user service 2015-11-11 12:31:00 +01:00
Mario de Frutos
f22807553f Removed logger 2015-11-11 12:30:59 +01:00
Mario de Frutos
29ec5da86d Added properties instead of getters 2015-11-11 12:30:59 +01:00
Mario de Frutos
7eab3e1f90 Refactor, extracted user_service 2015-11-11 12:30:59 +01:00
Mario de Frutos
bb3b2cc2da Change to use the installed package 2015-11-11 12:30:59 +01:00
Mario de Frutos
ab19c1b8ed Added setup.py for the package 2015-11-11 12:30:59 +01:00
Mario de Frutos
8ed26d4dd0 Client example using the cdb_conf table 2015-11-11 12:30:59 +01:00
Mario de Frutos
76c3b18dd6 Client function example 2015-11-11 12:30:59 +01:00
Mario de Frutos
a375e9dcfb Cached redis connections 2015-11-11 12:30:59 +01:00
Mario de Frutos
904336a298 Added plpythonu function example 2015-11-11 12:30:59 +01:00
Mario de Frutos
48052f8a70 Increment every time a row is georeferenced successfully 2015-11-11 12:30:59 +01:00
Mario de Frutos
1b347b7ad0 Checking from redis at localhost 2015-11-11 12:30:58 +01:00
Guido Fioravantti
e367456156 Make heremaps module pip installable 2015-11-06 12:07:41 +01:00
Mario de Frutos
a77b8f3686 Created new structure 2015-11-05 17:05:07 +01:00