The examples are not arbitrary: base64 by default ignores characters
that are not in its alphabet. When decoding, the dashes and underscores
are not b64-valid and therefore ignored, resulting in a sequence with
the wrong padding.
New functions to perform a precheck for the user data and metadata
parameters that are going to be used to execute a OBS_GetData operation.
Doing this before the OBD_GetData we are able to check if we're going to
have problems with the current request. Is useful because right now
we're using the *_Exception_Safe functions which return NULL in case we
have an exception, this leads to execute the whole request even if we
know beforehand that it's going to fail.
I was creating the wrappers before the wrapped function and postgres
extensions do not support that.
I did not notice in my local environment cause the old functions were
already there.
As shown in the tests, that type has the flexibility of being able to be
cast to from any other numeric type. This gives us flexibility without
bloating the API with lots of type-specific functions.
- Fixed fallback becuase we have to use spiexceptions in this case
because we retrieve the configuration usig plpy.execute fuctions that
wrap any exception into a spiexception.
- In case we don't want to use Mapzen, we could leave the api_key empty
becuase we arise a ConfigException if the Mapzen api_key is empty so
we are able to make fallback. Right now we can't remove the mapzen
configuration because it'll fail when the InternalGeocoderConfig try to
load the ServiceDBConfig
This is a dirty hack, we should improve how the DB config is loaded. See
If the user has a wrong base64 padded secret key the googlemaps
python library is returning "TypeError: Incorrect padding" which
is very hard to understand. So now we check if the secret key is
a valid base64 string
Two problems fixe with this functions ported from the cartodb extension:
* There was an incorrect reference to the cartodb scchema
* They need to be SECURITY DEFINER to be usable with the geocoder_api role
The new module cartodb_services/config is intended for services configuration objects
Some legacy configuration objects remain under cartodb_services/metrics.
The refactored configuration backends are also not moved here
The (legacy) Config object rate limit-related modifications are reverted.
For the legacy case, configuration is handled in a specific RateLimits builder class.
ServiceManager class has been introduced to handle service configuration at
SQL level (with a LegacyServiceManager alternative for non-refactored services).
These new classes take the responsibility of rate limits and quota checking.
Tests have been added for ServiceManager and rate limits, but currently they
check only the limits configuration since Lua support would be needed
to use rratelimit with MockRedis.
Instead of getting (deleted and untracked files):
```
$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 20 commits.
#
# Changes not staged for commit:
# (use "git add/rm <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# deleted: cdb_dataservices_server--0.20.0--0.21.0.sql
# deleted: cdb_dataservices_server--0.21.0--0.20.0.sql
# deleted: cdb_dataservices_server--0.21.0.sql
# modified: cdb_dataservices_server.control
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# cdb_dataservices_server--0.21.0--0.22.0.sql
# cdb_dataservices_server--0.22.0--0.21.0.sql
# cdb_dataservices_server--0.22.0.sql
# old_versions/cdb_dataservices_server--0.20.0--0.21.0.sql
# old_versions/cdb_dataservices_server--0.21.0--0.20.0.sql
# old_versions/cdb_dataservices_server--0.21.0.sql
no changes added to commit (use "git add" and/or "git commit -a")
```
you'd get something like:
```
$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 20 commits.
#
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# renamed: cdb_dataservices_server--0.20.0--0.21.0.sql -> old_versions/cdb_dataservices_server--0.20.0--0.21.0.sql
# renamed: cdb_dataservices_server--0.21.0--0.20.0.sql -> old_versions/cdb_dataservices_server--0.21.0--0.20.0.sql
# renamed: cdb_dataservices_server--0.21.0.sql -> old_versions/cdb_dataservices_server--0.21.0.sql
#
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: cdb_dataservices_server.control
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# cdb_dataservices_server--0.21.0--0.22.0.sql
# cdb_dataservices_server--0.22.0--0.21.0.sql
# cdb_dataservices_server--0.22.0.sql
```
which is nicer IMHO.
Found via unit test:
```sh
$ nosetests -x test/
....................................................................................E
======================================================================
ERROR: test_geocode_address_with_valid_params (test.test_mapzengeocoder.MapzenGeocoderTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/requests_mock/mocker.py", line 177, in inner
return func(*args, **kwargs)
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/test/test_mapzengeocoder.py", line 100, in test_geocode_address_with_valid_params
country='ESP')
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/cartodb_services/mapzen/qps.py", line 28, in wrapped_function
return QPSService(retry_timeout=timeout, queries_per_second=qps).call(original_function, *args, **kwargs)
File "/home/ubuntu/src/dataservices-api/server/lib/python/cartodb_services/cartodb_services/mapzen/qps.py", line 53, in call
raise e
AttributeError: 'NoneType' object has no attribute 'lower'
----------------------------------------------------------------------
Ran 85 tests in 0.140s
FAILED (errors=1)
```
timedelta microseconds is just the microseconds part of the timedelta
object not the elapsed time in microseconds.
I've change to use the total_seconds method to get all the elapsed time
in seconds and transform to miliseconds.
The number of retries was being passed as the pool size parameter.
This is corrected but number of retries changed to 1 to maintain previous behaviour.
