Mario de Frutos
9e247685b8
Merge pull request #465 from CartoDB/457-Fix_Mapbox_Python_tests
...
457 fix mapbox python tests
2018-02-14 10:21:29 +01:00
Juan Ignacio Sánchez Lara
1fdb4d3b3a
Pythonic refactor, importing single method
2018-02-13 18:24:21 +01:00
Juan Ignacio Sánchez Lara
029541f298
api_key -> mapbox_api_key rename refactor
2018-02-13 16:16:13 +01:00
Juan Ignacio Sánchez Lara
45d9edbba6
Mapbox test API key is supplied now through a MAPBOX_API_KEY environment variable
2018-02-13 15:58:10 +01:00
Antonio
8674dabeb2
Version bumped
2018-02-13 08:38:54 +01:00
Antonio
080a386b8f
Fixed mapbox requests and responses
2018-02-12 18:02:37 +01:00
Antonio
b65d003742
Using Mapbox permanent geocoder by default
2018-02-09 09:07:31 +01:00
Mario de Frutos
2f54ef7e4e
Merge pull request #452 from CartoDB/fix_misplaced_coordinate_polyline
...
Multiple fixes
2018-02-05 09:30:24 +01:00
Mario de Frutos
a791d02dcc
422 errors from mapbox return empty response
2018-02-02 14:11:22 +01:00
Mario de Frutos
e2612645c3
Bump to version 0.16.5
2018-02-02 13:18:47 +01:00
Mario de Frutos
c5fed2cc80
Check for empty coordinates when we transform to polygon
2018-02-02 13:18:32 +01:00
Mario de Frutos
a1f339376e
By default the polyline comes with 6 decimals so we avoid to use 5 by default
2018-02-02 13:17:56 +01:00
Mario de Frutos
5c7d1e9ac1
Solve some dependencies bugs
2018-02-02 12:29:48 +01:00
Mario de Frutos
6e54507b5a
Move exceptions to a proper place
2018-02-02 12:06:43 +01:00
Mario de Frutos
14925ea448
Add missing imports for google gecode street
2018-02-02 11:18:56 +01:00
Mario de Frutos
3973483af2
Move limits check inside the try
2018-02-01 19:03:32 +01:00
Mario de Frutos
88edad9414
Release artifacts for version 0.30.2
2018-02-01 18:46:24 +01:00
Mario de Frutos
88ad551e4a
Bump python version to 0.16.4
2018-02-01 18:40:21 +01:00
Mario de Frutos
94eaf84f91
Treat quota exception as a failure
2018-02-01 18:27:32 +01:00
Mario de Frutos
0df0b0d49c
Avoid treating quota exception as bug exception
2018-02-01 18:21:15 +01:00
Mario de Frutos
6ae8aa45fd
CR changes
2018-01-31 16:13:20 +01:00
Mario de Frutos
07f00cc0ae
Added tests for the client/channel part of google
2018-01-31 13:17:36 +01:00
Mario de Frutos
544e0fa763
Going green with non empty request strings
2018-01-31 12:25:35 +01:00
Mario de Frutos
2bbd6bac91
Going red with non empty request string
2018-01-31 12:25:18 +01:00
Mario de Frutos
2f8dbbb5dc
Dont raise exception when empty params passed to HERE geocoder
2018-01-31 12:16:13 +01:00
Mario de Frutos
56c90cc541
Bump version to 0.16.3
2018-01-31 12:12:52 +01:00
Mario de Frutos
3583cc6f47
Going green with empty requests that leads to 404 responses
2018-01-31 12:11:25 +01:00
Mario de Frutos
850dc09a4f
Going green with empty responses
2018-01-31 12:11:25 +01:00
Mario de Frutos
876bae6b56
Going red with bad request parameters
2018-01-31 12:11:25 +01:00
Antonio
e0b1632fa8
Fixed tests
2018-01-30 09:13:43 +01:00
Antonio
51bf6c2a43
Fixed valid Mapbox types
2018-01-29 18:39:11 +01:00
Antonio
05718ce58c
Fixed KeyError Exception
2018-01-29 17:32:44 +01:00
Antonio
fed444ff6c
Version bumped
2018-01-29 17:18:18 +01:00
Antonio
d1c8f8ced0
Fixing encoding issues
2018-01-29 17:09:17 +01:00
Antonio
1a2785dff4
Version bumped
2018-01-18 15:21:06 +01:00
Antonio
d442fab9da
Fixed encoding problem with Mapbox geocoding
2018-01-18 11:55:25 +01:00
Mario de Frutos
7d8cc4ae87
Fix bug that checks only for name of the custom type and not uses the
...
namespace
2018-01-16 18:28:54 +01:00
Antonio
9921f0f394
Added iso3166 import
2018-01-16 17:25:12 +01:00
Antonio
6c53846e33
Fixed country code problem. Added test
2018-01-16 16:36:24 +01:00
Mario de Frutos
5781f78c7f
Revert "Removed suspect import"
...
This reverts commit 9c1ec1ddde
2018-01-16 11:04:29 +01:00
Antonio
9c1ec1ddde
Removed suspect import
2018-01-16 10:02:41 +01:00
Antonio
0ed477d264
Generated sql files
2018-01-15 11:18:04 +01:00
Antonio
6339623ad9
Added ServiceManager for Mapbox services
2018-01-12 12:24:17 +01:00
Antonio
a02a634e76
Added metrics for DO (related to #419 and removed for easy merging)
2018-01-11 11:13:37 +01:00
Antonio
7dc87ad38e
Rotating API keys for Mapbox
2018-01-11 10:18:01 +01:00
Antonio Carlón
b10f48166d
Merge branch 'development' into Clients_for_Mapbox_services
2018-01-10 12:50:20 +01:00
Mario de Frutos
6047300259
Move the new params parameter to the end to avoid breaking backward compatibility
2018-01-10 12:41:08 +01:00
Antonio
b282c7db5b
Fixed tests
2018-01-10 12:33:10 +01:00
Mario de Frutos
84749a2631
Revert SQL changes just leave the python ones
2018-01-10 12:31:49 +01:00
Antonio
c0cc5fe6a7
Logging GetMeta and GetData parameters
2018-01-10 12:31:49 +01:00
Antonio
7d99ede28d
Error fixed
2018-01-10 11:55:02 +01:00
Antonio
cd5dba920e
Using user's configured geocoder for namedplaces via street point
2018-01-10 10:34:18 +01:00
Antonio
6d978deb16
Using user's configured geocoder for namedplaces via street point
2018-01-10 10:16:59 +01:00
Antonio
d3e01b4d53
Reverted default providers to mapzen to allow dark deployment
2018-01-09 17:15:03 +01:00
Antonio
dd6ad0119c
Added CR suggestions
2018-01-09 16:21:55 +01:00
Antonio
3bbb3c6bcc
Autogenerated sql file
2018-01-04 18:02:49 +01:00
Antonio
fa3f2fbc3a
Modified SQL files
2018-01-04 17:58:46 +01:00
Antonio
dc5dbe8c0a
Fixing tests
2018-01-04 16:01:32 +01:00
Antonio
e7c0628751
Solving errors and refactor
2018-01-04 15:00:52 +01:00
Antonio
51d97228dc
Added CR suggestions
2018-01-04 12:30:58 +01:00
Antonio
801c5a3cee
Solved data conversion issues
2018-01-03 14:14:25 +01:00
Antonio
941af522ae
Bumped Python library version
2018-01-03 09:47:55 +01:00
Antonio
bb86f080fc
Added limits/quotas and tests for Mapbox services
2018-01-02 16:21:57 +01:00
Antonio
c46174210d
Added cdb_dataservices_server functions
2017-12-29 14:07:51 +01:00
Antonio
af07def7fc
Added parameters for PLPython functions
2017-12-28 11:36:02 +01:00
Antonio
bc9dbdbb72
Removed routing client from isolines
2017-12-27 16:31:29 +01:00
Antonio
dabd596271
Added QPS decorator
2017-12-27 14:55:50 +01:00
Antonio
8c0af7d51d
Pure Python implementation of clients for Mapbox services
2017-12-27 13:43:14 +01:00
Rafa de la Torre
580f6d9867
Make the install target depend on deploy
...
The deploy target strips the PARALLEL tag when needed. This is done for
versions not supporting it.
2017-12-01 10:47:30 +01:00
Rafa de la Torre
48942cbb03
Fix typo in PARALLEL stripping
2017-12-01 10:25:10 +01:00
Rafa de la Torre
048aebedd9
Remove permissions stuff from downgrade script
2017-11-30 13:08:54 +01:00
Rafa de la Torre
1bcfa30938
Remove type creation from downgrade script
2017-11-30 13:08:26 +01:00
Rafa de la Torre
24098a6d2c
Remove duplicated header
2017-11-30 13:07:31 +01:00
Rafa de la Torre
645818838b
Append old script to downgrade stub
2017-11-30 13:06:57 +01:00
Rafa de la Torre
a2929e88cc
Remove permissions stuff from upgrade script
2017-11-30 13:04:34 +01:00
Rafa de la Torre
7e7432b4ec
Remove type creation from upgrade script
2017-11-30 13:04:09 +01:00
Rafa de la Torre
82db791706
Delete duplicate header from upgrade script
2017-11-30 13:02:28 +01:00
Rafa de la Torre
d21aa07673
Apend contents of extension to upgrade stub
2017-11-30 13:01:32 +01:00
Rafa de la Torre
dec7842240
Stub new server v0.29.0
2017-11-30 13:00:23 +01:00
Rafa de la Torre
0cb359a9bc
Add some instructions to make release target
2017-11-30 12:59:11 +01:00
Rafa de la Torre
b5c0e135f9
Revert "PG release: Include all files in git upon release"
...
This reverts commit b44778c586
2017-11-30 12:19:11 +01:00
Raul Marin
b44778c586
PG release: Include all files in git upon release
2017-11-28 16:27:45 +01:00
Raul Marin
723a3d02ff
PG release: Strip PARALLEL on deploy instead of release
2017-11-28 15:44:30 +01:00
Raúl Marín
6180b00525
Merge pull request #408 from Algunenano/parallel
...
Client: Add PARALLEL and VOLATILE categories
2017-11-27 13:56:01 +01:00
Raul Marin
ff71a406ae
Server: Categorize functions
...
Redis/Service functions declared as STATIC to avoid unneded calls
but it's really questionable if they trully are
2017-11-14 16:48:48 +01:00
Raul Marin
98206a3296
Server: Mark all plproxy functions as VOLATILE PARALLEL UNSAFE
2017-11-14 16:48:48 +01:00
Raul Marin
a69da0084b
Server: Add PARALLEL support
2017-11-14 16:48:48 +01:00
Juan Ignacio Sánchez Lara
da8e5f6e36
Extract parsing and return fix
2017-11-14 13:15:11 +01:00
Juan Ignacio Sánchez Lara
f540a3d6b3
Bump version
2017-11-13 19:09:52 +01:00
Juan Ignacio Sánchez Lara
e26bf2a803
Proper parsing of client and channel parameters
2017-11-13 19:09:17 +01:00
Juan Ignacio Sánchez Lara
43dd9f6ada
Better regexp and wrong input handling
2017-11-08 17:33:00 +01:00
Juan Ignacio Sánchez Lara
2cf4072b21
Support for multiple parameters at client (such as channel)
2017-11-08 11:07:35 +01:00
Mario de Frutos
68e87ca477
Release 0.28.0 server extension artifact
2017-10-18 10:25:04 +02:00
Mario de Frutos
4c7a3a25ec
Remove duplicated functions after the new ones are added
2017-10-10 16:23:15 +02:00
Mario de Frutos
e68cc3e820
OBS_GetAvailableGeometries now has number of geometries parameter
...
We've add a new parameter, number of geometries, that would be used
by DO to compute the geometry that better fit with the input provided
2017-10-09 17:20:57 +02:00
Mario de Frutos
baf9a48dad
Stub version 0.28.0 for server
2017-10-09 17:05:07 +02:00
Rafa de la Torre
1936374124
Version 0.15.5 of the python library
2017-10-06 16:03:03 +02:00
Rafa de la Torre
3ed41b065a
Marginally improve tests for #382
2017-10-06 15:27:51 +02:00
Rafa de la Torre
6c3260b6ee
Move credentials tests to GoogleMapsClientFactoryTestCase
2017-10-06 15:23:44 +02:00
Rafa de la Torre
5005572f89
Add tests for GoogleMapsClientFactory
2017-10-06 15:18:51 +02:00
Rafa de la Torre
27fde5a910
Use "{id}:{secret}" as caching key
2017-10-06 15:12:05 +02:00
Rafa de la Torre
b8e1120169
Revert "Use a different client_id for the test"
...
This reverts commit 854fbb803c
2017-10-06 15:03:08 +02:00
Rafa de la Torre
8e33cf9723
Fix google services key validity check #382
2017-10-06 14:40:53 +02:00
Rafa de la Torre
7b988e3f78
Add tests for #382
...
The examples are not arbitrary: base64 by default ignores characters
that are not in its alphabet. When decoding, the dashes and underscores
are not b64-valid and therefore ignored, resulting in a sequence with
the wrong padding.
2017-10-06 14:37:26 +02:00
Rafa de la Torre
f79ac9297d
Add again the check for valid credentials
...
As our check is more strict that the one provided in googlemaps library.
2017-10-06 13:44:08 +02:00
Rafa de la Torre
854fbb803c
Use a different client_id for the test
2017-10-06 13:41:50 +02:00
Rafa de la Torre
d029ad7041
GoogleMapsClientFactory to persist clients #401
...
Add a GoogleMapsClientFactory and remove the check for valid key,
which is no longer needed, as it is done in the google library.
2017-10-06 12:05:34 +02:00
Rafa de la Torre
445d4cf97d
Upgrade googlemaps python library to v2.5.1
...
This closes #383
2017-10-04 16:22:33 +02:00
Rafa de la Torre
f0a3249b4e
Keep metrics keys consistent
...
See cartodb repo and our internal documentation:
https://github.com/CartoDB/cartodb/search?utf8=%E2%9C%93&q=failed_responses&type=
2017-09-07 11:24:38 +02:00
Mario de Frutos
475df918c7
Fix invalid geometries due to generalize option
...
They add a note saying that use their simplification option could
lead to a self-intersection (which is the problem we have) and it's
creating invalid geometries
See it here https://mapzen.com/documentation/mobility/isochrone/api-reference/#other-request-parameters
2017-08-30 17:35:08 +02:00
Mario de Frutos
c13dba71a4
Deactivated DO credits usage for users that have quota
...
See https://github.com/CartoDB/bigmetadata/issues/215
2017-08-24 14:35:20 +02:00
Mario de Frutos
a297504c63
Release artifacts for client (0.20.0) and server (0.27.0)
2017-08-10 14:34:42 +02:00
Mario de Frutos
3eee68c56e
_OBS_Precheck and OBS_MetadataValidator
...
New functions to perform a precheck for the user data and metadata
parameters that are going to be used to execute a OBS_GetData operation.
Doing this before the OBD_GetData we are able to check if we're going to
have problems with the current request. Is useful because right now
we're using the *_Exception_Safe functions which return NULL in case we
have an exception, this leads to execute the whole request even if we
know beforehand that it's going to fail.
2017-08-09 12:26:41 +02:00
Mario de Frutos
c982c6e4fb
Bump version of the server extension to 0.27.0
2017-08-08 10:56:48 +02:00
Daniel García Aubert
13a876b854
Fix typo while logging caught error in geocoder
2017-07-21 14:43:41 +02:00
Mario de Frutos
928d1111f3
Release artifacts for client 0.19.0 and server 0.26.0
2017-07-20 12:01:58 +02:00
Mario de Frutos
6d802c72be
Bump to version 0.19.0 and 0.26.0
2017-07-13 17:58:50 +02:00
Mario de Frutos
1ffbba2a03
New DO function _OBS_GetNumerators
2017-07-13 17:31:49 +02:00
Mario de Frutos
7281c4a0e3
Bump server version to 0.25.0
2017-07-13 17:23:23 +02:00
Daniel García Aubert
1503b5bb2b
Release artifacts:
...
- Server extension version 0.25.0
- Client extension version 0.18.0
- Python library version 0.15.2
2017-07-13 15:34:44 +02:00
Javier Goizueta
d3a1485a38
Fix tests
...
The cherry-picked commit to fix the tests included some changes related to other branch
2017-07-12 17:37:43 +02:00
Mario de Frutos
c5c182fcf0
Fixed tests
2017-07-10 10:26:58 +02:00
Rafa de la Torre
2f35bc5fe5
Swap the function creation order
...
I was creating the wrappers before the wrapped function and postgres
extensions do not support that.
I did not notice in my local environment cause the old functions were
already there.
2017-07-07 19:03:16 +02:00
Rafa de la Torre
e3aaff4413
Add missing trailing semicolons
2017-07-07 18:54:56 +02:00
Rafa de la Torre
49a53e4cd7
Fix non-zero-padded days in tests
...
See https://github.com/CartoDB/dataservices-api/issues/204 and related
tickets.
2017-07-07 15:14:27 +02:00
Rafa de la Torre
074234b8da
Use double precision as numeric type
...
As shown in the tests, that type has the flexibility of being able to be
cast to from any other numeric type. This gives us flexibility without
bloating the API with lots of type-specific functions.
2017-07-07 13:08:41 +02:00
Rafa de la Torre
847a7aef99
Add support for integer postal codes
2017-07-05 18:29:49 +02:00
Mario de Frutos
12ffc44efb
Server extension version 0.24.2 release artifacts
2017-05-26 12:46:57 +02:00
Mario de Frutos
ad6ed9a9bc
Bump version for DS
2017-05-26 12:41:42 +02:00
Mario de Frutos
ac854a94af
Fixed fallback to internal geocoder in nameplaces geocoding functions
...
- Fixed fallback becuase we have to use spiexceptions in this case
because we retrieve the configuration usig plpy.execute fuctions that
wrap any exception into a spiexception.
- In case we don't want to use Mapzen, we could leave the api_key empty
becuase we arise a ConfigException if the Mapzen api_key is empty so
we are able to make fallback. Right now we can't remove the mapzen
configuration because it'll fail when the InternalGeocoderConfig try to
load the ServiceDBConfig
This is a dirty hack, we should improve how the DB config is loaded. See
2017-05-26 12:41:41 +02:00
Mario de Frutos
61efb66aba
Add an extra check for google credentials
...
If the user has a wrong base64 padded secret key the googlemaps
python library is returning "TypeError: Incorrect padding" which
is very hard to understand. So now we check if the secret key is
a valid base64 string
2017-05-26 11:51:32 +02:00
Rafael Porres Molina
37eeab5b9e
past participle of set is set
2017-05-24 15:41:12 +02:00
Rafa de la Torre
d7fad6d8d3
Prepare release of server v0.24.1
2017-05-16 17:58:34 +02:00
Rafa de la Torre
59a8cafc74
Fix another empty table case #366
2017-05-16 17:20:16 +02:00
Rafa de la Torre
4ef6083344
Test for OBS_GetData(geomrefs TEXT[], params JSON) #366
2017-05-16 17:19:40 +02:00
Rafa de la Torre
b50637d36f
Fix return type of OBS_GetData #366
2017-05-16 16:23:13 +02:00
Rafa de la Torre
c653914694
Add test for empty table response #366
2017-05-16 16:18:42 +02:00
Rafa de la Torre
921ef46eb8
Use contrib_regression as obs backend #366
2017-05-16 15:59:34 +02:00
Rafa de la Torre
4c46effc9b
Add new params at the end #361
2017-05-09 16:36:03 +02:00
Rafa de la Torre
4c5183e9bd
Prepare new version of the server #361
...
Changes in TYPEs obs_meta_geometry and obs_meta_timespan
2017-05-04 17:54:49 +02:00
John Krauss
8b031a3016
add tags to getavailablegeometries, and a few additional columns to obs_getavailabletimespans
2017-04-10 18:15:19 +00:00
Javier Goizueta
524d15c1a9
Fix uses of cartodb cdb_conf functions
2017-03-30 18:10:53 +02:00
Javier Goizueta
c0b0a58d35
Missing changes in source
2017-03-30 18:03:45 +02:00
Javier Goizueta
7ec0a3ab66
Missing changes in v23 script
2017-03-30 18:03:25 +02:00
Javier Goizueta
e55338de90
Fix functions to write cdb_conf
...
Two problems fixe with this functions ported from the cartodb extension:
* There was an incorrect reference to the cartodb scchema
* They need to be SECURITY DEFINER to be usable with the geocoder_api role
2017-03-30 16:33:12 +02:00
Javier Goizueta
2197edb467
Fix bugs in geocoding server functions
2017-03-29 16:20:59 +02:00
Javier Goizueta
88c43bab2f
Add missing functions to migration scripts
2017-03-29 13:08:28 +02:00
Javier Goizueta
aac89e0236
New versions 0.16.0 (client), 0.23.0 (server), 0.15.0 (python)
2017-03-28 17:53:40 +02:00
Javier Goizueta
f0b0a9e7f2
Merge pull request #355 from CartoDB/346-user-rate-limits
...
Service rate limits
2017-03-28 17:48:46 +02:00
Rafa de la Torre
77dfda11d9
Merge pull request #353 from CartoDB/udpate-docs
...
Fix links and improve readability
2017-03-28 11:53:07 +02:00
Javier Goizueta
2c15110255
Rename constructor arguments for consistency
2017-03-28 10:53:16 +02:00
Javier Goizueta
6b86acfaa3
Fix indentation
2017-03-28 10:44:25 +02:00
Javier Goizueta
ef09840525
Implement server-side rate limits configuration interface
2017-03-21 17:58:33 +01:00
Javier Goizueta
cf7460c27d
Fix functions schema
2017-03-21 16:17:39 +01:00
Javier Goizueta
63c03894cc
Fix bugs
2017-03-21 15:45:05 +01:00
Javier Goizueta
942c6ac63d
Fix indentation
2017-03-21 15:44:50 +01:00
Javier Goizueta
208469f534
Add rate limits configuration writing tests
2017-03-21 15:43:33 +01:00
Javier Goizueta
73f97128ed
Reorganization of configuration modules, fix circular dependencies
...
The new module cartodb_services/config is intended for services configuration objects
Some legacy configuration objects remain under cartodb_services/metrics.
The refactored configuration backends are also not moved here
2017-03-21 15:42:53 +01:00
Javier Goizueta
945c6cd685
WIP: support for storing rate limits configuration
2017-03-16 19:12:39 +01:00
Javier Goizueta
7f6c19b292
Minor changes for clarity
...
* ServiceManager check method renamed as assert_within_limits
* Legacy classes moved to separate files
2017-03-16 17:59:22 +01:00
Javier Goizueta
87e37e1a26
Fix typo
2017-03-16 17:55:27 +01:00
Javier Goizueta
9a04ad06a0
Move rate limits out of Config
...
The (legacy) Config object rate limit-related modifications are reverted.
For the legacy case, configuration is handled in a specific RateLimits builder class.
2017-03-15 19:32:50 +01:00
Javier Goizueta
cebd7d2141
Apply ServiceManager to Google Geocoder
...
Note that the Goolge geocoder uses QuotaService to account for used quota,
but it doesn't have a quota limit.
2017-03-15 17:58:14 +01:00
Javier Goizueta
696bdb40b0
Fix incorrect import
2017-03-15 17:49:29 +01:00
Javier Goizueta
680206a437
Missing import
2017-03-15 12:51:56 +01:00
Javier Goizueta
548d6b08db
Debug SQL geocoding interface which uses ServiceManager
2017-03-15 12:47:11 +01:00
Javier Goizueta
77b5ff0b9e
Fix problem with db config default arguments
2017-03-15 12:46:25 +01:00
Javier Goizueta
05b29967c7
Merge branch 'development' into 346-user-rate-limits
...
# Conflicts:
# README.md
# server/extension/sql/20_geocode_street.sql
# server/lib/python/cartodb_services/cartodb_services/metrics/config.py
2017-03-14 19:00:53 +01:00
Javier Goizueta
6c5ca97468
Complete rate-limiting for Mapzen & Here gecoding
...
ServiceManager class has been introduced to handle service configuration at
SQL level (with a LegacyServiceManager alternative for non-refactored services).
These new classes take the responsibility of rate limits and quota checking.
Tests have been added for ServiceManager and rate limits, but currently they
check only the limits configuration since Lua support would be needed
to use rratelimit with MockRedis.
2017-03-14 18:51:18 +01:00
Rafa de la Torre
a64557b50e
New version 0.14.1 of python library
2017-03-13 15:10:03 +01:00
Rafa de la Torre
4e14e9b0c0
Fix links and improve readability
2017-03-13 13:12:06 +01:00
Rafa de la Torre
26d025a5d1
Remove tests that no longer apply #206
2017-03-10 13:15:56 +01:00
Rafa de la Torre
80e7ed90a8
Make get_metrics read just zero-padded dates #206
2017-03-10 13:14:36 +01:00
Javier Goizueta
250d384d06
Geocoder, per user, cofigurable rate limits
...
WIP, specially the GeocoderConfig part is flaky and ugly
2017-03-09 18:42:48 +01:00
Rafa de la Torre
c6c4255242
Server 0.22.0 version release artifacts
2017-03-08 15:22:58 +01:00
Rafa de la Torre
87a2e02721
Make more apparent what happens with make release
...
Instead of getting (deleted and untracked files):
```
$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 20 commits.
#
# Changes not staged for commit:
# (use "git add/rm <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# deleted: cdb_dataservices_server--0.20.0--0.21.0.sql
# deleted: cdb_dataservices_server--0.21.0--0.20.0.sql
# deleted: cdb_dataservices_server--0.21.0.sql
# modified: cdb_dataservices_server.control
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# cdb_dataservices_server--0.21.0--0.22.0.sql
# cdb_dataservices_server--0.22.0--0.21.0.sql
# cdb_dataservices_server--0.22.0.sql
# old_versions/cdb_dataservices_server--0.20.0--0.21.0.sql
# old_versions/cdb_dataservices_server--0.21.0--0.20.0.sql
# old_versions/cdb_dataservices_server--0.21.0.sql
no changes added to commit (use "git add" and/or "git commit -a")
```
you'd get something like:
```
$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 20 commits.
#
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# renamed: cdb_dataservices_server--0.20.0--0.21.0.sql -> old_versions/cdb_dataservices_server--0.20.0--0.21.0.sql
# renamed: cdb_dataservices_server--0.21.0--0.20.0.sql -> old_versions/cdb_dataservices_server--0.21.0--0.20.0.sql
# renamed: cdb_dataservices_server--0.21.0.sql -> old_versions/cdb_dataservices_server--0.21.0.sql
#
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: cdb_dataservices_server.control
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# cdb_dataservices_server--0.21.0--0.22.0.sql
# cdb_dataservices_server--0.22.0--0.21.0.sql
# cdb_dataservices_server--0.22.0.sql
```
which is nicer IMHO.
2017-03-08 13:22:28 +01:00
Rafa de la Torre
fc1e05658e
Bump version of python library to 0.14.0
2017-03-08 13:13:19 +01:00
Rafa de la Torre
b8ec3dc306
Merge pull request #342 from CartoDB/remove-mapzen-forcing-layers-equal-address
...
Remove the search_type if its address from the params sent to Mapzen
2017-03-08 13:07:07 +01:00
Rafa de la Torre
ffdb6524da
Fix bug: search_type can be None
...
Found via unit test:
```sh
$ nosetests -x test/
....................................................................................E
======================================================================
ERROR: test_geocode_address_with_valid_params (test.test_mapzengeocoder.MapzenGeocoderTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/requests_mock/mocker.py", line 177, in inner
return func(*args, **kwargs)
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/test/test_mapzengeocoder.py", line 100, in test_geocode_address_with_valid_params
country='ESP')
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/cartodb_services/mapzen/qps.py", line 28, in wrapped_function
return QPSService(retry_timeout=timeout, queries_per_second=qps).call(original_function, *args, **kwargs)
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/cartodb_services/mapzen/qps.py", line 53, in call
raise e
AttributeError: 'NoneType' object has no attribute 'lower'
----------------------------------------------------------------------
Ran 85 tests in 0.140s
FAILED (errors=1)
```
2017-03-08 10:19:32 +01:00
Rafa de la Torre
82a7477d37
Beautify code
2017-03-08 10:14:37 +01:00
Rafa de la Torre
049c8e8657
Merge branch 'development' into remove-mapzen-forcing-layers-equal-address
2017-03-07 14:56:55 +01:00
Rafa de la Torre
25ba9866ae
Small PoC with rratelimit #346
2017-03-07 13:31:06 +01:00
Javier Goizueta
6dca3f4e58
Change default arguent value for clarity
...
It is valie to pass None in service_params arguments, so make it the default.
2017-03-06 19:01:05 +01:00
Javier Goizueta
d01170471e
Missing @property
2017-03-06 18:58:00 +01:00
Javier Goizueta
89762c1a7f
Use service params configuration in Mapzen services
2017-03-06 15:25:11 +01:00
Javier Goizueta
1081e81047
Add Mapzen service params to configuration
2017-03-06 14:34:23 +01:00
Javier Goizueta
29cdd8099a
Add service configuration (URL, etc) for Mapzen Geocoding
2017-03-06 10:20:20 +01:00
Javier Goizueta
c0374f5800
Add service configuration (URL, etc) for Here Routing
2017-03-02 17:55:08 +01:00
Javier Goizueta
677d2acead
Add service configuration (URL, etc) for Here Geocoding
...
See #343
2017-03-02 17:15:14 +01:00
Mario de Frutos
435a84f294
OBS_GetData functions shall increase the DO quota usage
2017-02-06 17:47:15 +01:00
nygeog
a9e942a48b
remove the search_type if its address from the params sent to mapzen
...
remove the search_type if its address from the params sent to mapzen
2017-02-02 10:23:09 -05:00
Mario de Frutos
061956f250
Remove quota usage for obs_meta functions
2017-02-02 10:59:21 +01:00
Mario de Frutos
3d94d1dfaf
Server 0.21.0 version release artifacts
2017-02-01 10:19:32 +01:00
Mario de Frutos
c364c7f5ed
Merge pull request #339 from CartoDB/obs-getdata-getmeta-wrappers
...
Obs_GetData OBS_GetMeta wrappers
2017-02-01 10:05:52 +01:00
Mario de Frutos
c550a5d4e9
Merge pull request #321 from CartoDB/fix-default-isolines-provider
...
Change default isolines provider
2017-02-01 09:41:21 +01:00
John Krauss
1f225e0978
fix extraneous newline in expectation
2017-01-26 12:27:29 +01:00
John Krauss
e81f005ce9
adding server tests, interface to generate client sql code
2017-01-26 12:27:29 +01:00
John Krauss
8204bb3451
fix error-causing raise (which obscures real error)
2017-01-26 12:27:29 +01:00
John Krauss
56de523498
add wrappers for getmeta, getdata(geoms) and getdata(ids)
2017-01-26 12:27:29 +01:00
Rafa de la Torre
a0dd7c7497
Fix tests: zero-pad months in redis key #335
2017-01-04 11:11:27 +01:00
Rafa de la Torre
9700ae966c
Remove uneeded math for simplicity's sake
2016-12-28 14:44:48 +01:00
Javier Goizueta
bdf9627586
Fix typo
...
I needed to fix this; I felt some kitten was being killed somewhere
because of this. And it made baby Jesus cry.
2016-12-27 17:11:39 +01:00
Mario de Frutos
80b23c62c3
QPS timeout was badly calculated
...
timedelta microseconds is just the microseconds part of the timedelta
object not the elapsed time in microseconds.
I've change to use the total_seconds method to get all the elapsed time
in seconds and transform to miliseconds.
2016-12-23 11:40:30 +01:00
Mario de Frutos
23a2de0321
Add mapzen isolines to the quota checker
2016-12-22 17:38:10 +01:00
Javier Goizueta
9ee1d045c8
Fix tests
2016-12-21 09:54:08 +01:00
Rafa de la Torre
360b5bd57f
Merge remote-tracking branch 'origin/development' into mapzen_services_use_quotas
...
Solve merge conflict in setup.py (version of python lib).
2016-12-20 18:14:01 +01:00
Javier Goizueta
32010669e8
New version of the python lib
...
To fix bug with period end date; see #322
2016-12-20 16:39:19 +01:00
Mario de Frutos
38754fec26
Cover refactor mapzen geocoder config with tests
2016-12-20 16:17:42 +01:00
Mario de Frutos
356135672b
Sanitize quota get in the refactor part
2016-12-20 16:17:42 +01:00
Mario de Frutos
4d0abf9026
Bump python library version
2016-12-20 16:17:42 +01:00
Mario de Frutos
0672f2752b
Improve tests and add more unit tests for the quota functionality
2016-12-20 16:17:41 +01:00
Javier Goizueta
b5514aea60
Avoid invalid end of period dates
...
Fixes #322
2016-12-20 12:48:36 +01:00
Javier Goizueta
53acd4d30e
Add tests that reveal end-of-period date problem
...
See #322
2016-12-20 12:41:54 +01:00
Mario de Frutos
6c71d73498
Use the current quota for mapzen services
2016-12-20 11:52:06 +01:00
Rafa de la Torre
a00fca6d13
Update README.md
...
Add a bit about versioning stuff
2016-12-16 18:00:22 +01:00
Rafa de la Torre
29caaf9297
Update README.md
...
Remove misleading paragraph about `requirements.txt` and `setup.py` dependencies. Refer to https://packaging.python.org/requirements/ for an authoritative discussion.
2016-12-16 18:00:22 +01:00
Javier Goizueta
4fc90626ab
New version of the python lib
...
New release to change number of http connection retries to 1
2016-12-13 18:03:37 +01:00
Javier Goizueta
1f53af65b9
Set number of http connection retries to 1
...
Change the maximum number of retries for connection to external services to 1.
2016-12-13 11:32:22 +01:00
Javier Goizueta
96a147a242
The default isolines provider should be Mapzen
2016-12-13 10:19:27 +01:00
Javier Goizueta
79fb796180
Fix bug in retries usage; mantain behaviour
...
The number of retries was being passed as the pool size parameter.
This is corrected but number of retries changed to 1 to maintain previous behaviour.
2016-12-13 10:07:10 +01:00
Javier Goizueta
2953fda75c
Revert num of isochrone retries to 3
...
See #317
2016-12-12 12:14:10 +01:00
Rafa de la Torre
98fa248fff
New version of the python lib #317
2016-12-07 16:39:28 +01:00
Rafa de la Torre
7c348dee0f
Mapzen isochrones: tweak retry params #317
2016-12-07 16:38:47 +01:00
Rafa de la Torre
bdeaadf33f
Fix QPS: Compare everything in microseconds #317
2016-12-07 16:37:25 +01:00
Mario de Frutos
51face5593
Make the HTTP requests retry on timeout
2016-12-02 17:01:05 +01:00
Mario de Frutos
77b2ac9155
Bump python version
2016-11-29 16:34:59 +01:00
Mario de Frutos
8edbf33ff1
Added denoise and generalize options to make the ischrones less heavy
2016-11-29 16:10:59 +01:00
Mario de Frutos
bd439d8985
Car is auto in mapzen isochrones
2016-11-29 13:18:52 +01:00
Mario de Frutos
16259862dd
Version 0.20.0 artifact
2016-11-29 13:08:43 +01:00
Mario de Frutos
77f4f3e7ff
Mapzen isochtrones integration
2016-11-29 12:57:23 +01:00
Mario de Frutos
4b714b3845
New version 0.20.0
2016-11-29 12:25:35 +01:00
Rafa de la Torre
ab4584bed3
New server version 0.19.0 #302
2016-11-25 10:56:41 +01:00
Rafa de la Torre
2b2691d0b9
New minor version of the python library #302
2016-11-25 10:43:06 +01:00
Rafa de la Torre
9c6eabc59e
Do not enforce types for params #302
...
As types are tied to a schema, they don't get along very well with
pl/proxy. Do not use them for the service_type.
2016-11-24 18:38:33 +01:00
Rafa de la Torre
769b740ba4
Rename service_params to service_quota_info #302
2016-11-24 17:46:07 +01:00
Rafa de la Torre
c2ede37d75
Revert internal geocoder related stuff #302
2016-11-24 17:45:19 +01:00
Rafa de la Torre
4c75c5f261
Revert internal geocoder related stuff #302
...
This reverts commits 579d11e4af5975a2da597
2016-11-24 17:33:06 +01:00
Rafa de la Torre
efb10e43e4
Merge remote-tracking branch 'origin/development' into 302-expose-quotas
2016-11-24 12:36:23 +01:00
Rafa de la Torre
1dc6060570
Fix: deal with string values for soft limits #302
2016-11-23 18:51:13 +01:00
Rafa de la Torre
a2da597e00
Support null quota (internal_geocoder) #302
2016-11-23 18:10:44 +01:00
Rafa de la Torre
4af5975ac1
Add the internal geocoder to cdb_service_params #302
2016-11-23 17:07:23 +01:00
Rafa de la Torre
579d11ebd5
Add params to InternalGeocoderConfig #302
...
Add a bunch of params to InternalGeocoderConfig for consistency with
other services:
- monthly_quota
- period_end_date
- soft_limit
2016-11-23 17:04:50 +01:00
Rafa de la Torre
a1eae3934c
Add OBS to UserMetricsService #302
2016-11-23 16:47:35 +01:00
Rafa de la Torre
d70d149eed
Fix bug: the service_type to used_quota #302
2016-11-23 16:40:00 +01:00
Rafa de la Torre
decc7626a8
Create types only if they don't exist #302
2016-11-23 16:38:47 +01:00
Rafa de la Torre
abbb227ad5
Fix typo: use obs config for obs service #302
2016-11-23 11:40:02 +01:00
Mario de Frutos
f0474852f2
Client 0.12.1 and server 0.18.1 artifacts
2016-11-11 18:27:37 +01:00
Mario de Frutos
8396eae733
Server update to type obs_meta_geometry with new types
2016-11-11 18:13:01 +01:00
Mario de Frutos
dfaa037475
Server init new version 0.18.1
2016-11-11 17:53:20 +01:00
Mario de Frutos
d1b17da70b
Don't make mandatory to have a metrics file
2016-11-11 11:48:08 +01:00
Rafa de la Torre
2e6f37af56
Add observatory #302
2016-11-11 11:27:57 +01:00
Mario de Frutos
b5f4c507b3
Fixed some typos in SQL artifact v0.18 and v0.17
2016-11-11 11:13:08 +01:00
Rafa de la Torre
95dda082e7
Add routing #302
2016-11-11 10:24:03 +01:00
Rafa de la Torre
9f9034f4f1
Make cdb_enough_quota check the table got from cdb_service_params #302
2016-11-10 19:23:39 +01:00
Rafa de la Torre
603fbbbc3f
Make cdb_service_params return a table for all services #302
2016-11-10 19:23:01 +01:00
Rafa de la Torre
3fa686ec65
Remove the class Service in favor of service_type ENUM #302
2016-11-10 19:21:29 +01:00
Rafa de la Torre
429a71ef37
Add a function to check for enough quota #302
2016-11-10 18:56:50 +01:00
Rafa de la Torre
b2cdb1c74b
Move all into one func returning a tuple #302
2016-11-10 18:40:18 +01:00
Rafa de la Torre
84061dec4f
Move everything to cdb_service_params (WIP) #302
2016-11-10 18:12:24 +01:00
Mario de Frutos
08debb83ed
Server side change JSONB to JSON in the legacybuildermedatada function
2016-11-10 18:08:35 +01:00
Rafa de la Torre
7a32ace1ac
Add the hires_geocoder to cdb_monthly_quota #302
2016-11-10 17:00:25 +01:00
Rafa de la Torre
7a56dc698f
List of services bound to quota #302
2016-11-10 16:59:58 +01:00
Mario de Frutos
4945b4d0bf
Server version 0.18.0 artifact
2016-11-10 16:03:08 +01:00
Mario de Frutos
040b094b01
New observatory meta functions for server side
2016-11-10 11:29:19 +01:00
Mario de Frutos
a006f76285
New server version 0.18.0
2016-11-10 11:29:18 +01:00
Rafa de la Torre
eb7188235d
Provider function #302
2016-11-10 11:14:15 +01:00
Rafa de la Torre
3e059003b6
Add soft limit function #302
2016-11-10 11:09:40 +01:00
Rafa de la Torre
0e38af6ecd
Add cdb_remaining_quota to server #302
2016-11-08 18:47:22 +01:00
Rafa de la Torre
b00edf218a
Fix: quotas should be integers #302
2016-11-08 18:46:58 +01:00
Rafa de la Torre
6f9feb07a0
Add cdb_monthly_quota (WIP) #302
2016-11-08 18:29:06 +01:00
Rafa de la Torre
2c76fa4501
Fix: quotas should be integers #302
2016-11-08 18:28:28 +01:00
Rafa de la Torre
4169bc0f88
Fix doc: how to run python unit tests #302
2016-11-08 17:52:18 +01:00
Mario de Frutos
6177f0ac76
Remove info trace not needed in metrics
2016-11-07 11:02:09 +01:00
Mario de Frutos
352d4217bc
Fixed typo in the config cache for observatory snapshot functions
2016-11-07 10:31:53 +01:00
Mario de Frutos
2fd560f3cf
Release 0.17.0 artifact
2016-11-04 13:41:12 +01:00
Mario de Frutos
88ed777785
Added plpy_mock and check multiple instances for metricsdatagatherer
2016-11-04 13:39:11 +01:00
Mario de Frutos
73aa26590b
Make the data gather singleton support multiple instances
...
Due to we use pgbouncer, we could share the same instance of the metrics
gatherer singleton in multiples requests. Added support to have
multiple instances identified by transaction id
2016-11-04 13:39:11 +01:00
Mario de Frutos
c571edb21e
Split response and function data, added uuid and type of element
2016-11-04 13:39:11 +01:00
Mario de Frutos
e878031744
Added url to the response data
2016-11-04 13:39:11 +01:00
Mario de Frutos
49fd51f6da
Added logger to metrics context manager
2016-11-04 13:39:10 +01:00
Mario de Frutos
708bee2912
Added metrics context manager to public functions
2016-11-04 13:39:10 +01:00
Mario de Frutos
cf3c6f2ce5
New metrics logger system
...
- It's not mandatory to have the log_path defined for all the services
- Now we add metrics to a MetricsGatherer using the add method in order
to gather data from all the function scope, not just in one defined
point
- There is a `metrics` context manager to wrap the function block. This
context manager get the execution time for the block among other
things
2016-11-02 15:26:39 +01:00
Mario de Frutos
93579532e3
Move old versions
2016-10-28 10:42:25 +02:00
Mario de Frutos
6058960ec5
Added timeout for all the third-party connections
2016-10-27 16:17:37 +02:00
Mario de Frutos
336d8be977
Hotfix: Typo in status code check
2016-10-26 17:49:07 +02:00
Mario de Frutos
80963e2589
504 errors return empty data instead of raise exception
...
Due to some problems in Mapzen, we're receiving 504 errors from their
servers. To mitigate this problem, instead of raise an exception, we're
going to return empty data for that point
2016-10-26 16:39:35 +02:00
Mario de Frutos
19d6cacdb3
Fix empty response from matrix
...
https://github.com/CartoDB/dataservices-api/issues/293
2016-10-26 16:00:50 +02:00
Rafa de la Torre
0d22942a72
Update version of python lib server package to 0.9.2
2016-10-21 17:38:32 +02:00
Rafa de la Torre
d4ac2eb5e6
Tests for ServicesRedisConfig (mapzen quota) #281
2016-10-21 15:40:22 +02:00
Rafa de la Torre
db80d389e0
Do not require org geocoder and isolines quota to be present
2016-10-21 15:38:58 +02:00
Rafa de la Torre
8e02c64aeb
Tests for UserMetricsService, mapzen routing #281
2016-10-21 15:21:03 +02:00
Rafa de la Torre
cc2ab1bc0c
Fix test: make it independent of current date #281
2016-10-21 15:08:17 +02:00
Rafa de la Torre
e1a7d1751c
Fix corner case: used_quota == monthly_quota #281
2016-10-20 15:36:48 +02:00
Rafa de la Torre
0c49107f96
Some tests for the QuotaChecker #281
2016-10-20 15:31:21 +02:00
Rafa de la Torre
247034c21e
Some tests for RoutingConfig #281
2016-10-19 18:40:17 +02:00
Rafa de la Torre
2b1b1c981f
Add routing to UserMetricsService.used_quota
2016-10-19 17:41:43 +02:00
Rafa de la Torre
aaff5564ec
Use soft_limit for __check_routing_quota #281
2016-10-18 17:44:56 +02:00
Rafa de la Torre
72998c324a
Simplify the code for regularity's sake #281
2016-10-18 17:40:50 +02:00
Rafa de la Torre
bbd9b6b98e
Add soft limit to config object #281
2016-10-18 17:35:36 +02:00
Rafa de la Torre
27be704bd6
Changes needed to read quota from redis #281
...
Use per-user/org quota from redis, if present. Otherwise default to the
quota stored in server database (as it did to date).
2016-10-18 17:26:45 +02:00
