cartodb4/dataservices-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,056 Commits 1 Branch 183 Tags 3.3 MiB
edbefad6b0
Commit Graph

549 Commits

Author SHA1 Message Date
Mario de Frutos
33f40bc945 TOMTOM uses 403 instead of 429 for rate limiting 
That has a great problem when we're dealing with legit 403 status for
example deactivated user, forbidden access, etc.

I've added a check for the HTTP header `X-Error-Detail-Header` in order
to distinguish between legit 403 and 429 error messages

Possible values for `X-Error-Detail-Header` in a 403 error:

  o Service Requires SSL : http is used instead of https (secure)

  o Invalid Referer : invalid 'Referer' header value is send
  to https://api.tomtom.com and allowed referer values are
  configured on specific API key

  o Account Over Queries Per Second Limit : rate limit exceeded

  o Account Inactive : incorrect API key/API key no longer valid
 2018-10-31 12:59:13 +01:00
Mario de Frutos
17c993f6ef Bump version 2018-10-24 12:00:44 +02:00
Mario de Frutos
6640909780 Add provider for QPS manger in tomtom services 2018-10-24 12:00:08 +02:00
Mario de Frutos
56a2ebb210 Fix qps timeout bug 2018-10-03 18:50:20 +02:00
Mario de Frutos
61a27a9f9d Tomtom QPS fix 
Tomtom return 403 status code in the following scenarios:

- Forbidden
- Not authorized
- Account inactive
- Account over queries per second limit
- Account over rate limit
- Rate limit exceeded

So when we reach the QPS limit we receive a 403 instead of 429 so we
have to include the provider in the qps service to differentiate between
providers that returns 429 and special cases like tomtom
 2018-10-03 16:35:38 +02:00
Mario de Frutos
a8e96366a5 Bump for the python library version 2018-08-21 18:40:22 +02:00
Mario de Frutos
6cc3cda6e0 Remove obs_snapshot_config from python library 2018-08-21 18:28:55 +02:00
Alejandro Guirao Rodríguez
9c8647ebd4
Changes in component names and versions 2018-08-17 10:57:52 +02:00
Juan Ignacio Sánchez Lara
d9f647504a Disable Mapbox Matrix API usage at tests 2018-08-14 09:39:42 +02:00
Juan Ignacio Sánchez Lara
ed2e87f4ca Fix test fixture 2018-08-14 09:27:59 +02:00
Juan Ignacio Sánchez Lara
c1c671755c Final carto-package.json format 2018-08-13 16:28:47 +02:00
Juan Ignacio Sánchez Lara
a32e90ea8a carto-package.json proposal 2018-08-02 12:59:45 +02:00
Juan Ignacio Sánchez Lara
075f602a7f WELL_KNOWN_SHAPE and WELL_KNOWN_LENGTH fixture update 2018-07-25 11:22:31 +02:00
Juan Ignacio Sánchez Lara
11ec6075c3 Python library version 0.19.1 2018-07-24 11:59:29 +02:00
Juan Ignacio Sánchez Lara
c6720bf689 Better debug message 2018-07-24 11:57:54 +02:00
Juan Ignacio Sánchez Lara
3524ee1e24 Handle postprocessing error 2018-07-24 11:31:05 +02:00
Juan Ignacio Sánchez Lara
80dcde2db0 Log Mapbox unknown status 2018-07-23 22:00:16 +02:00
Juan Ignacio Sánchez Lara
fa3d7db5f8 Fix Google geocoder error handling 2018-07-23 21:54:37 +02:00
Juan Ignacio Sánchez Lara
d060ab3d41 Empty vs missing count detail 2018-07-23 19:11:23 +02:00
Juan Ignacio Sánchez Lara
3a5360c96c Refactor and fix for actual searches type 2018-07-23 19:10:35 +02:00
Juan Ignacio Sánchez Lara
fc75f1afc8 Google batch geocoder error handling 2018-07-23 18:41:47 +02:00
Juan Ignacio Sánchez Lara
4be3aa88fd Constant extraction refactor 2018-07-23 18:30:33 +02:00
Juan Ignacio Sánchez Lara
8162bff204 Serial geocoding error handling 2018-07-23 18:27:02 +02:00
Juan Ignacio Sánchez Lara
1b31c089ce Global error handling for batched geocoding 2018-07-23 18:16:40 +02:00
Juan Ignacio Sánchez Lara
5d2303e1de Log a failed one if any 2018-07-23 17:23:08 +02:00
Juan Ignacio Sánchez Lara
07f5be9207 TomTom error handling 2018-07-23 17:17:38 +02:00
Juan Ignacio Sánchez Lara
c5d9db61e6 Mapbox error handling 2018-07-23 15:48:32 +02:00
Juan Ignacio Sánchez Lara
1ff512839d Fixes empty results count 2018-07-23 13:01:46 +02:00
Juan Ignacio Sánchez Lara
9a1b1e2832 Error count 2018-07-23 12:18:26 +02:00
Juan Ignacio Sánchez Lara
1cebbe7af0 Missing warning mock and fix for debug 2018-07-23 12:16:48 +02:00
Juan Ignacio Sánchez Lara
2862c80025 Proper empty count on bulk geocoding 2018-07-23 11:53:02 +02:00
Juan Ignacio Sánchez Lara
abbaf83e97 run_street_point_geocoder tests 2018-07-23 11:42:51 +02:00
Juan Ignacio Sánchez Lara
cd5e6510a6 In case of general error, failed service use should be incremented by searches length 2018-07-23 09:30:09 +02:00
Juan Ignacio Sánchez Lara
fd097724f1 In case of general error, total service use should be incremented by searches length 2018-07-19 21:17:09 +02:00
Juan Ignacio Sánchez Lara
96fbf3080a Base run_street_point_geocoder test 2018-07-19 21:08:07 +02:00
Juan Ignacio Sánchez Lara
0d490bbb19 Extract EMPTY_RESPONSE 2018-07-19 20:52:59 +02:00
Juan Ignacio Sánchez Lara
b0c1948c14 TL;DR: safer deployment and minor fixes 
- Instead of modifying cdb_service_quota_info to return max_batch_size,
a new type (service_quota_info_batch) and a new function
(cdb_service_quota_info_batch) are created. That makes deployment safe.
- Fixes geocoding with forced batch size 1.
- Improves namespacing for count_estimate (->
cdb_dataservices_client.cdb_count_estimate).
- Improves namespacing for jsonb_array_casttext (->
cdb_dataservices_client.cdb_jsonb_array_casttext).
 2018-07-18 14:57:40 +02:00
Juan Ignacio Sánchez Lara
5e34faefe5 Quota test 2018-07-17 14:39:56 +02:00
Juan Ignacio Sánchez Lara
b90d402fa9 Round relevance (plus refactor) 2018-07-17 13:56:01 +02:00
Juan Ignacio Sánchez Lara
c104f6f34b Metadata attributes constant extraction 2018-07-17 12:46:16 +02:00
Juan Ignacio Sánchez Lara
e9ed3bca18 Safer comparison 2018-07-17 12:27:42 +02:00
Juan Ignacio Sánchez Lara
e2762a6e03 Removed debug traces 2018-07-17 10:06:43 +02:00
Juan Ignacio Sánchez Lara
e82346e7f6 match_types for batched HERE metadata 2018-07-16 12:43:40 +02:00
Juan Ignacio Sánchez Lara
080de34163 match_types for Mapbox metadata 2018-07-16 12:11:40 +02:00
Juan Ignacio Sánchez Lara
0a92ae1445 match_types for TomTom metadata 2018-07-16 12:01:55 +02:00
Juan Ignacio Sánchez Lara
0b635377ef match_types for HERE metadata 2018-07-16 11:59:43 +02:00
Juan Ignacio Sánchez Lara
f2197d4b2a match_types for Google metadata 2018-07-16 11:34:33 +02:00
Juan Ignacio Sánchez Lara
6e78da55b2 Precision metadata for Mapbox 2018-07-11 19:28:16 +02:00
Juan Ignacio Sánchez Lara
4123a4c442 Precision metadata for TomTom 2018-07-11 19:09:02 +02:00
Juan Ignacio Sánchez Lara
dbb4f9204a Precision metadata for HERE 2018-07-11 15:30:51 +02:00
Juan Ignacio Sánchez Lara
67fee1cce8 Precision metadata for Google 2018-07-11 14:06:50 +02:00
Juan Ignacio Sánchez Lara
b779742585 Fix logging on error 2018-07-11 12:51:56 +02:00
Juan Ignacio Sánchez Lara
d46d51c3bb Relevance metadata for Google 2018-07-11 11:43:54 +02:00
Juan Ignacio Sánchez Lara
0b2ee85c11 TomTom normalization 2018-07-11 09:30:28 +02:00
Juan Ignacio Sánchez Lara
825e3b7ee8 Relevance metadata for Mapbox 2018-07-11 09:30:08 +02:00
Juan Ignacio Sánchez Lara
2af9204542 Relevance metadata for TomTom 2018-07-10 21:21:42 +02:00
Juan Ignacio Sánchez Lara
34e622b809 Relevance metadata for HERE 2018-07-10 20:30:01 +02:00
Juan Ignacio Sánchez Lara
531ad28158 Send optimal batch size 2018-07-10 19:31:58 +02:00
Juan Ignacio Sánchez Lara
286a75fa8e _bulk_geocode logic extraction 2018-07-10 15:17:14 +02:00
Juan Ignacio Sánchez Lara
a6c5c21131 Serial geocode for Google bulk 2018-07-10 13:45:01 +02:00
Juan Ignacio Sánchez Lara
f6b7c13dde GoogleMapsBulkGeocoder extraction 2018-07-10 13:38:51 +02:00
Juan Ignacio Sánchez Lara
1ffe3658fe Revert "maxresults depends on batch" 
This reverts commit bf8b76b5fe.
 2018-07-10 12:28:16 +02:00
Juan Ignacio Sánchez Lara
8e430ce1c1 Google geocoder works better concatenating all components 2018-07-10 11:17:21 +02:00
Juan Ignacio Sánchez Lara
cce5f92312 Escape semicolons at Mapbox batch 2018-07-09 18:17:45 +02:00
Juan Ignacio Sánchez Lara
40ace9cfaa Send _serial_geocode for single results 2018-07-09 17:42:08 +02:00
Juan Ignacio Sánchez Lara
f618e4aec3 Mapbox bulk geocoding 2018-07-09 17:35:56 +02:00
Juan Ignacio Sánchez Lara
ae84122c3d countries --> country 2018-07-09 15:35:37 +02:00
Juan Ignacio Sánchez Lara
b8475bac30 TomTom batch geocoding 2018-07-06 20:11:48 +02:00
Juan Ignacio Sánchez Lara
bf8b76b5fe maxresults depends on batch 2018-07-06 20:04:28 +02:00
Juan Ignacio Sánchez Lara
31afc82b56 TomTom bulk geocoding by bypassing to serial 2018-07-06 15:52:45 +02:00
Juan Ignacio Sánchez Lara
5be43e15c0 Fix error message on TomTom error 2018-07-06 15:52:45 +02:00
Juan Ignacio Sánchez Lara
6da70fd8ea Fix encoding of missing fields 2018-07-06 15:52:10 +02:00
Juan Ignacio Sánchez Lara
6c89ca8d70 Fix exception raising 2018-07-05 12:42:18 +02:00
Juan Ignacio Sánchez Lara
3c07133912 Improve logging of exceptions from sys.exc_info 2018-07-05 08:50:26 +02:00
Juan Ignacio Sánchez Lara
8968f0e6ec Fix message 2018-07-04 13:36:18 +02:00
Juan Ignacio Sánchez Lara
44744de73d Explicit check for result length 2018-07-04 13:29:37 +02:00
Juan Ignacio Sánchez Lara
9856adb7ce Explicit NotImplementedError 2018-07-04 12:33:29 +02:00
Juan Ignacio Sánchez Lara
e416a8a641 HERE batch support 2018-07-02 18:35:36 +02:00
Juan Ignacio Sánchez Lara
fc610313bf Test refactor and Here serial batch 2018-06-29 19:18:53 +02:00
Juan Ignacio Sánchez Lara
18e2349713 Bulk geocoding refactor 2018-06-29 14:59:07 +02:00
Juan Ignacio Sánchez Lara
d0b04a97b8 Better Google geocoding through concatenation 2018-06-27 13:47:57 +02:00
Juan Ignacio Sánchez Lara
8f4249ee24 Merge branch 'development' into geocoder_boost 2018-06-27 11:17:56 +02:00
Juan Ignacio Sánchez Lara
9c90c539f8 Fix Mapbox routing fixture shape 2018-06-27 11:09:32 +02:00
Juan Ignacio Sánchez Lara
bbbf70f3ac Street level columns for country, city and state 2018-06-22 12:19:51 +02:00
Juan Ignacio Sánchez Lara
58d70e252f checked, bulked cdb_bulk_geocode_street_point 2018-06-21 10:23:39 +02:00
Juan Ignacio Sánchez Lara
f3f2b213e7 Bump versions 2018-06-11 16:26:57 +02:00
Juan Ignacio Sánchez Lara
3f08d37ef7 Google bulk_geocoder 2018-06-11 12:56:07 +02:00
Juan Ignacio Sánchez Lara
be446c1bf2 exception logging at _send_to_plpy 2018-06-07 10:29:57 +02:00
Juan Ignacio Sánchez Lara
5251534283 Allow using non-Premium keys for Google Maps client 2018-06-04 18:07:16 +02:00
Juan Ignacio Sánchez Lara
7577936c33 Fixes isolines quota check 2018-04-11 15:24:17 +02:00
Juan Ignacio Sánchez Lara
c3c28dfd5e Simpler quota check matching 2018-04-11 12:14:11 +02:00
Juan Ignacio Sánchez Lara
33f23e0902 Minor instead of patch for TomTom support at Python lib 2018-04-10 11:36:34 +02:00
Juan Ignacio Sánchez Lara
94aeceb894 Fixes encoding 2018-04-10 09:32:23 +02:00
Juan Ignacio Sánchez Lara
e21d3e2e70 TomTom Python lib version 2018-04-09 16:29:00 +02:00
Juan Ignacio Sánchez Lara
3b121c8793 Merge branch 'master' into Added_TomTom_services 2018-04-09 13:07:04 +02:00
Mario de Frutos
50aa537a7a Bump python library version 2018-03-27 16:36:04 +02:00
Juan Ignacio Sánchez Lara
2a47000f32 coordinates_to_polygon should always return multipolygon to avoid inconsistencies 2018-03-27 16:36:04 +02:00
Antonio
552a7d4886 Fixing Mapbox geocoding validity checks 2018-03-16 16:55:08 +01:00
Antonio
e78063ae75 Fixed geocoding validity 2018-03-16 16:01:00 +01:00
Antonio
d66804b93d Fixed test 2018-03-16 14:47:56 +01:00