cartodb4/dataservices-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,067 Commits 1 Branch 183 Tags 3.3 MiB
d4dc60783e
Commit Graph

956 Commits

Author SHA1 Message Date
Mario de Frutos
2f6d067b4e Parse JSON parameters to avoid problems 
If we pass JSON as string, with scape characters and so on, we are
going to have problems parsing that metrics in ES
 2018-11-05 17:06:52 +01:00
Mario de Frutos
089d3e3f27 New version 0.34.1 2018-11-05 16:51:17 +01:00
Mario de Frutos
33f40bc945 TOMTOM uses 403 instead of 429 for rate limiting 
That has a great problem when we're dealing with legit 403 status for
example deactivated user, forbidden access, etc.

I've added a check for the HTTP header `X-Error-Detail-Header` in order
to distinguish between legit 403 and 429 error messages

Possible values for `X-Error-Detail-Header` in a 403 error:

  o Service Requires SSL : http is used instead of https (secure)

  o Invalid Referer : invalid 'Referer' header value is send
  to https://api.tomtom.com and allowed referer values are
  configured on specific API key

  o Account Over Queries Per Second Limit : rate limit exceeded

  o Account Inactive : incorrect API key/API key no longer valid
 2018-10-31 12:59:13 +01:00
Mario de Frutos
17c993f6ef Bump version 2018-10-24 12:00:44 +02:00
Mario de Frutos
6640909780 Add provider for QPS manger in tomtom services 2018-10-24 12:00:08 +02:00
Mario de Frutos
56a2ebb210 Fix qps timeout bug 2018-10-03 18:50:20 +02:00
Mario de Frutos
61a27a9f9d Tomtom QPS fix 
Tomtom return 403 status code in the following scenarios:

- Forbidden
- Not authorized
- Account inactive
- Account over queries per second limit
- Account over rate limit
- Rate limit exceeded

So when we reach the QPS limit we receive a 403 instead of 429 so we
have to include the provider in the qps service to differentiate between
providers that returns 429 and special cases like tomtom
 2018-10-03 16:35:38 +02:00
antoniocarlon
3990d13572 Server changes 2018-09-11 14:35:02 +02:00
antoniocarlon
a33dd4315c Fixe 2018-09-10 14:31:08 +02:00
antoniocarlon
49bcf31832 Removed appname parameter 2018-09-10 14:26:53 +02:00
antoniocarlon
b4123625f7 Fixed server tests 2018-09-10 13:15:19 +02:00
antoniocarlon
4b6b4e92e8 CR suggestions (remove appname) 2018-09-10 12:54:37 +02:00
antoniocarlon
603d691017 Added metrics 2018-09-06 13:47:50 +02:00
antoniocarlon
04a5a25d06 Fixed m04r tests 2018-09-06 12:55:31 +02:00
antoniocarlon
30e480e545 Fixed m04r tests 2018-09-06 12:46:12 +02:00
antoniocarlon
eca4b42a73 Fixed tests 2018-09-06 12:32:48 +02:00
antoniocarlon
7a7ccbbfcf Added appname parameter to the public server functions 2018-09-06 12:04:11 +02:00
Mario de Frutos
a8e96366a5 Bump for the python library version 2018-08-21 18:40:22 +02:00
Mario de Frutos
6cc3cda6e0 Remove obs_snapshot_config from python library 2018-08-21 18:28:55 +02:00
Mario de Frutos
0e95c5ff90 Now the snapshot functions use the obs_genral config 2018-08-21 18:25:12 +02:00
Mario de Frutos
fba933cc88 Remove obs_snapshot_config 2018-08-21 18:18:10 +02:00
Mario de Frutos
44da876b4c Version 0.33.0 initial commit 2018-08-21 18:15:29 +02:00
Alejandro Guirao Rodríguez
9c8647ebd4
Changes in component names and versions 2018-08-17 10:57:52 +02:00
Alejandro Guirao Rodríguez
bd391f4bf4
Component change and versioning using more numbers 2018-08-17 10:54:24 +02:00
Juan Ignacio Sánchez Lara
d9f647504a Disable Mapbox Matrix API usage at tests 2018-08-14 09:39:42 +02:00
Juan Ignacio Sánchez Lara
ed2e87f4ca Fix test fixture 2018-08-14 09:27:59 +02:00
Juan Ignacio Sánchez Lara
c1c671755c Final carto-package.json format 2018-08-13 16:28:47 +02:00
Juan Ignacio Sánchez Lara
a32e90ea8a carto-package.json proposal 2018-08-02 12:59:45 +02:00
Juan Ignacio Sánchez Lara
075f602a7f WELL_KNOWN_SHAPE and WELL_KNOWN_LENGTH fixture update 2018-07-25 11:22:31 +02:00
Juan Ignacio Sánchez Lara
11ec6075c3 Python library version 0.19.1 2018-07-24 11:59:29 +02:00
Juan Ignacio Sánchez Lara
c6720bf689 Better debug message 2018-07-24 11:57:54 +02:00
Juan Ignacio Sánchez Lara
3524ee1e24 Handle postprocessing error 2018-07-24 11:31:05 +02:00
Juan Ignacio Sánchez Lara
80dcde2db0 Log Mapbox unknown status 2018-07-23 22:00:16 +02:00
Juan Ignacio Sánchez Lara
fa3d7db5f8 Fix Google geocoder error handling 2018-07-23 21:54:37 +02:00
Juan Ignacio Sánchez Lara
d060ab3d41 Empty vs missing count detail 2018-07-23 19:11:23 +02:00
Juan Ignacio Sánchez Lara
3a5360c96c Refactor and fix for actual searches type 2018-07-23 19:10:35 +02:00
Juan Ignacio Sánchez Lara
fc75f1afc8 Google batch geocoder error handling 2018-07-23 18:41:47 +02:00
Juan Ignacio Sánchez Lara
4be3aa88fd Constant extraction refactor 2018-07-23 18:30:33 +02:00
Juan Ignacio Sánchez Lara
8162bff204 Serial geocoding error handling 2018-07-23 18:27:02 +02:00
Juan Ignacio Sánchez Lara
1b31c089ce Global error handling for batched geocoding 2018-07-23 18:16:40 +02:00
Juan Ignacio Sánchez Lara
5d2303e1de Log a failed one if any 2018-07-23 17:23:08 +02:00
Juan Ignacio Sánchez Lara
07f5be9207 TomTom error handling 2018-07-23 17:17:38 +02:00
Juan Ignacio Sánchez Lara
c5d9db61e6 Mapbox error handling 2018-07-23 15:48:32 +02:00
Juan Ignacio Sánchez Lara
1ff512839d Fixes empty results count 2018-07-23 13:01:46 +02:00
Juan Ignacio Sánchez Lara
9a1b1e2832 Error count 2018-07-23 12:18:26 +02:00
Juan Ignacio Sánchez Lara
1cebbe7af0 Missing warning mock and fix for debug 2018-07-23 12:16:48 +02:00
Juan Ignacio Sánchez Lara
2862c80025 Proper empty count on bulk geocoding 2018-07-23 11:53:02 +02:00
Juan Ignacio Sánchez Lara
abbaf83e97 run_street_point_geocoder tests 2018-07-23 11:42:51 +02:00
Juan Ignacio Sánchez Lara
cd5e6510a6 In case of general error, failed service use should be incremented by searches length 2018-07-23 09:30:09 +02:00
Juan Ignacio Sánchez Lara
fd097724f1 In case of general error, total service use should be incremented by searches length 2018-07-19 21:17:09 +02:00
Juan Ignacio Sánchez Lara
96fbf3080a Base run_street_point_geocoder test 2018-07-19 21:08:07 +02:00
Juan Ignacio Sánchez Lara
0d490bbb19 Extract EMPTY_RESPONSE 2018-07-19 20:52:59 +02:00
Juan Ignacio Sánchez Lara
b0c1948c14 TL;DR: safer deployment and minor fixes 
- Instead of modifying cdb_service_quota_info to return max_batch_size,
a new type (service_quota_info_batch) and a new function
(cdb_service_quota_info_batch) are created. That makes deployment safe.
- Fixes geocoding with forced batch size 1.
- Improves namespacing for count_estimate (->
cdb_dataservices_client.cdb_count_estimate).
- Improves namespacing for jsonb_array_casttext (->
cdb_dataservices_client.cdb_jsonb_array_casttext).
 2018-07-18 14:57:40 +02:00
Juan Ignacio Sánchez Lara
0c5e9da028 Upgrade/downgrade scripts for server 0.32.0 and client 0.25.0 2018-07-17 19:48:13 +02:00
Juan Ignacio Sánchez Lara
5e34faefe5 Quota test 2018-07-17 14:39:56 +02:00
Juan Ignacio Sánchez Lara
b90d402fa9 Round relevance (plus refactor) 2018-07-17 13:56:01 +02:00
Juan Ignacio Sánchez Lara
c104f6f34b Metadata attributes constant extraction 2018-07-17 12:46:16 +02:00
Juan Ignacio Sánchez Lara
e9ed3bca18 Safer comparison 2018-07-17 12:27:42 +02:00
Juan Ignacio Sánchez Lara
e2762a6e03 Removed debug traces 2018-07-17 10:06:43 +02:00
Juan Ignacio Sánchez Lara
e82346e7f6 match_types for batched HERE metadata 2018-07-16 12:43:40 +02:00
Juan Ignacio Sánchez Lara
080de34163 match_types for Mapbox metadata 2018-07-16 12:11:40 +02:00
Juan Ignacio Sánchez Lara
0a92ae1445 match_types for TomTom metadata 2018-07-16 12:01:55 +02:00
Juan Ignacio Sánchez Lara
0b635377ef match_types for HERE metadata 2018-07-16 11:59:43 +02:00
Juan Ignacio Sánchez Lara
f2197d4b2a match_types for Google metadata 2018-07-16 11:34:33 +02:00
Juan Ignacio Sánchez Lara
6e78da55b2 Precision metadata for Mapbox 2018-07-11 19:28:16 +02:00
Juan Ignacio Sánchez Lara
4123a4c442 Precision metadata for TomTom 2018-07-11 19:09:02 +02:00
Juan Ignacio Sánchez Lara
dbb4f9204a Precision metadata for HERE 2018-07-11 15:30:51 +02:00
Juan Ignacio Sánchez Lara
67fee1cce8 Precision metadata for Google 2018-07-11 14:06:50 +02:00
Juan Ignacio Sánchez Lara
b779742585 Fix logging on error 2018-07-11 12:51:56 +02:00
Juan Ignacio Sánchez Lara
d46d51c3bb Relevance metadata for Google 2018-07-11 11:43:54 +02:00
Juan Ignacio Sánchez Lara
0b2ee85c11 TomTom normalization 2018-07-11 09:30:28 +02:00
Juan Ignacio Sánchez Lara
825e3b7ee8 Relevance metadata for Mapbox 2018-07-11 09:30:08 +02:00
Juan Ignacio Sánchez Lara
2af9204542 Relevance metadata for TomTom 2018-07-10 21:21:42 +02:00
Juan Ignacio Sánchez Lara
34e622b809 Relevance metadata for HERE 2018-07-10 20:30:01 +02:00
Juan Ignacio Sánchez Lara
531ad28158 Send optimal batch size 2018-07-10 19:31:58 +02:00
Juan Ignacio Sánchez Lara
286a75fa8e _bulk_geocode logic extraction 2018-07-10 15:17:14 +02:00
Juan Ignacio Sánchez Lara
a6c5c21131 Serial geocode for Google bulk 2018-07-10 13:45:01 +02:00
Juan Ignacio Sánchez Lara
f6b7c13dde GoogleMapsBulkGeocoder extraction 2018-07-10 13:38:51 +02:00
Juan Ignacio Sánchez Lara
1ffe3658fe Revert "maxresults depends on batch" 
This reverts commit bf8b76b5fe.
 2018-07-10 12:28:16 +02:00
Juan Ignacio Sánchez Lara
8e430ce1c1 Google geocoder works better concatenating all components 2018-07-10 11:17:21 +02:00
Juan Ignacio Sánchez Lara
cce5f92312 Escape semicolons at Mapbox batch 2018-07-09 18:17:45 +02:00
Juan Ignacio Sánchez Lara
40ace9cfaa Send _serial_geocode for single results 2018-07-09 17:42:08 +02:00
Juan Ignacio Sánchez Lara
f618e4aec3 Mapbox bulk geocoding 2018-07-09 17:35:56 +02:00
Juan Ignacio Sánchez Lara
ae84122c3d countries --> country 2018-07-09 15:35:37 +02:00
Juan Ignacio Sánchez Lara
b8475bac30 TomTom batch geocoding 2018-07-06 20:11:48 +02:00
Juan Ignacio Sánchez Lara
bf8b76b5fe maxresults depends on batch 2018-07-06 20:04:28 +02:00
Juan Ignacio Sánchez Lara
31afc82b56 TomTom bulk geocoding by bypassing to serial 2018-07-06 15:52:45 +02:00
Juan Ignacio Sánchez Lara
5be43e15c0 Fix error message on TomTom error 2018-07-06 15:52:45 +02:00
Juan Ignacio Sánchez Lara
6da70fd8ea Fix encoding of missing fields 2018-07-06 15:52:10 +02:00
Juan Ignacio Sánchez Lara
6c89ca8d70 Fix exception raising 2018-07-05 12:42:18 +02:00
Juan Ignacio Sánchez Lara
3c07133912 Improve logging of exceptions from sys.exc_info 2018-07-05 08:50:26 +02:00
Juan Ignacio Sánchez Lara
5b46c1527e Revert "Revert expected warnings" 
This reverts commit 89e9bf1ed6.
 2018-07-05 08:23:33 +02:00
Juan Ignacio Sánchez Lara
89e9bf1ed6 Revert expected warnings 2018-07-04 15:07:54 +02:00
Juan Ignacio Sánchez Lara
8968f0e6ec Fix message 2018-07-04 13:36:18 +02:00
Juan Ignacio Sánchez Lara
44744de73d Explicit check for result length 2018-07-04 13:29:37 +02:00
Juan Ignacio Sánchez Lara
9856adb7ce Explicit NotImplementedError 2018-07-04 12:33:29 +02:00
Juan Ignacio Sánchez Lara
e416a8a641 HERE batch support 2018-07-02 18:35:36 +02:00
Juan Ignacio Sánchez Lara
fc610313bf Test refactor and Here serial batch 2018-06-29 19:18:53 +02:00
Juan Ignacio Sánchez Lara
18e2349713 Bulk geocoding refactor 2018-06-29 14:59:07 +02:00
Juan Ignacio Sánchez Lara
8fe9903e7a searchtext -> searches for bulk geocoding 2018-06-27 15:14:11 +02:00
Juan Ignacio Sánchez Lara
d0b04a97b8 Better Google geocoding through concatenation 2018-06-27 13:47:57 +02:00
Juan Ignacio Sánchez Lara
a931086e29 Log now also logs the exception, fixing fixtures 2018-06-27 13:13:08 +02:00
Juan Ignacio Sánchez Lara
8f4249ee24 Merge branch 'development' into geocoder_boost 2018-06-27 11:17:56 +02:00
Juan Ignacio Sánchez Lara
9c90c539f8 Fix Mapbox routing fixture shape 2018-06-27 11:09:32 +02:00
Juan Ignacio Sánchez Lara
bbbf70f3ac Street level columns for country, city and state 2018-06-22 12:19:51 +02:00
Juan Ignacio Sánchez Lara
58d70e252f checked, bulked cdb_bulk_geocode_street_point 2018-06-21 10:23:39 +02:00
Juan Ignacio Sánchez Lara
e85f43f1d1 cdb_bulk_geocode_street_point skeleton 2018-06-15 09:55:52 +02:00
Juan Ignacio Sánchez Lara
f3f2b213e7 Bump versions 2018-06-11 16:26:57 +02:00
Juan Ignacio Sánchez Lara
34fc6439d2 cdb_bulk_geocode_street_point functions 2018-06-11 16:12:41 +02:00
Juan Ignacio Sánchez Lara
3f08d37ef7 Google bulk_geocoder 2018-06-11 12:56:07 +02:00
Juan Ignacio Sánchez Lara
be446c1bf2 exception logging at _send_to_plpy 2018-06-07 10:29:57 +02:00
Juan Ignacio Sánchez Lara
5251534283 Allow using non-Premium keys for Google Maps client 2018-06-04 18:07:16 +02:00
Juan Ignacio Sánchez Lara
3a240bf6ad params at routing metrics 2018-04-27 12:43:29 +02:00
Juan Ignacio Sánchez Lara
7577936c33 Fixes isolines quota check 2018-04-11 15:24:17 +02:00
Juan Ignacio Sánchez Lara
193513bfea params at metrics 2018-04-11 14:28:15 +02:00
Juan Ignacio Sánchez Lara
c3c28dfd5e Simpler quota check matching 2018-04-11 12:14:11 +02:00
Juan Ignacio Sánchez Lara
e5d2182da3 Removed geocoder_api user creation from version upgrade script 2018-04-11 11:50:30 +02:00
Juan Ignacio Sánchez Lara
33f23e0902 Minor instead of patch for TomTom support at Python lib 2018-04-10 11:36:34 +02:00
Juan Ignacio Sánchez Lara
94aeceb894 Fixes encoding 2018-04-10 09:32:23 +02:00
Juan Ignacio Sánchez Lara
e21d3e2e70 TomTom Python lib version 2018-04-09 16:29:00 +02:00
Juan Ignacio Sánchez Lara
adde2d3a46 0.31.0 release scripts 2018-04-09 14:41:44 +02:00
Juan Ignacio Sánchez Lara
6143c04c82 Remove server extension versioned files of outdated version changes 2018-04-09 13:11:50 +02:00
Juan Ignacio Sánchez Lara
3b121c8793 Merge branch 'master' into Added_TomTom_services 2018-04-09 13:07:04 +02:00
Mario de Frutos
50aa537a7a Bump python library version 2018-03-27 16:36:04 +02:00
Mario de Frutos
e25e2c26a0 Remove the PARALLEL modifiers from the version SQL artifact 2018-03-27 16:36:04 +02:00
Juan Ignacio Sánchez Lara
8025f657b8 Upgrade version scripts 2018-03-27 16:36:04 +02:00
Juan Ignacio Sánchez Lara
2a47000f32 coordinates_to_polygon should always return multipolygon to avoid inconsistencies 2018-03-27 16:36:04 +02:00
Antonio
552a7d4886 Fixing Mapbox geocoding validity checks 2018-03-16 16:55:08 +01:00
Antonio
a34da0adb5 Fixed CR suggestions 2018-03-16 16:37:17 +01:00
Antonio
f0cef02dfc Fixed CR suggestions 2018-03-16 16:33:45 +01:00
Antonio
e78063ae75 Fixed geocoding validity 2018-03-16 16:01:00 +01:00
Antonio
d66804b93d Fixed test 2018-03-16 14:47:56 +01:00
Antonio
6572891036 Merging changes 2018-03-16 14:30:01 +01:00
Antonio
6017b53ea0 Server version 0.30.4 2018-03-16 12:07:20 +01:00
Antonio
e15c6127d3 Added ST_CollectionExtract 2018-03-16 12:01:12 +01:00
Ubuntu
44fae489a3 Update scripts 2018-03-15 15:09:21 +00:00
Alex Martín
71c93fe13a Add ST_MakeValid to sql 2018-03-15 15:58:15 +01:00
Antonio
c90859e58b Fixed error when checking quota 2018-03-14 16:26:00 +01:00
Antonio
f5cbc195cc Fixed tests 2018-03-14 13:31:00 +01:00
Mario de Frutos
0196292093 Bump version 2018-03-14 11:20:12 +01:00
Mario de Frutos
f652a52a8d Use routing quota instead of monthly quota 2018-03-14 11:19:48 +01:00
Antonio
1c38b33501 Updated server side scripts 2018-03-14 09:55:07 +01:00
Antonio
59751d7c9c Avoid reaching provider for empty geocodings 2018-03-13 16:01:05 +01:00
Antonio
ffa1080277 isochrones fix 2018-03-13 14:58:40 +01:00
Antonio
39800122b2 Merging 2018-03-02 17:25:32 +01:00
Antonio
12ac2c269d Modified expected result for test 2018-03-02 17:17:03 +01:00
Antonio
74c0bb6f26 Created new server version (0.31.0) 2018-03-02 17:09:25 +01:00
Antonio
c9d0f0447f Added server functions (untested) 2018-03-02 16:40:12 +01:00
Antonio
596189185f Added configuration for TomTom services 2018-03-02 16:01:22 +01:00
Antonio
35f743164e Keep deleting points that got None 2018-03-02 10:27:37 +01:00