Go to file
2016-07-06 15:43:26 +02:00
client Add cdb_mapzen_isochrone to client 2016-07-06 13:18:54 +02:00
doc Fixed titles 2016-05-25 18:19:05 +02:00
scripts Install cdb_geocoder extension with non-admin user 2016-02-26 10:49:09 +01:00
server Remove useless cost_per_hit line 2016-07-06 15:43:26 +02:00
test Add integration tests for mapzen provider 2016-07-05 12:09:22 +02:00
.gitignore Polyline decoder 2016-03-02 10:43:27 +01:00
LICENSE Add LICENSE file 2015-12-22 18:13:46 +01:00
README.md typo 2016-06-28 12:06:13 +02:00

Data Services API

The CartoDB Data Services SQL API (server and client FTM)

Deploy instructions

Steps to deploy a new Data Services API version :

  • Deploy new version of dataservices API to all servers
  • Update the server user using: ALTER EXTENSION cdb_dataservices_server UPDATE TO '<CURRENT_VERSION>';
  • Update the python dependencies if needed: cartodb_geocoder and heremaps
  • Add the needed config in the cdb_conf table:
    • redis_metadata_config and redis_metrics_conf
      • {"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}
    • heremaps_conf
      • {"app_id": "APP_ID", "app_code": "APP_CODE"}
  • Deploy the client to all the servers with the new version
  • Deploy the editor with the new dataservices api version changed (https://github.com/CartoDB/cartodb/blob/master/app/models/user/db_service.rb#L18)
  • Execute the rails task to update first the CartoDB team organizaton to test in production
    • RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_organizations['team']
  • Check if all works perfectly for our team. If so, execute the rake tasks to update all the users and organizations:
    • RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_organizations['', true]
    • RAILS_ENV=production bundle exec rake cartodb:db:configure_geocoder_extension_for_non_org_users['', true]
  • Freeze the generated SQL file for the version. Eg. cdb_dataservices_server--0.0.1.sql

Local install instructions

  • install data services geocoder extension

    git clone git@github.com:CartoDB/data-services.git
    cd data-services/geocoder/extension
    sudo make install
    
  • install observatory extension

    git clone git@github.com:CartoDB/observatory-extension.git
    cd observatory
    sudo make install
    
  • install server and client extensions

    cd client && sudo make install
    cd server/extension && sudo make install
    
  • install python library

    cd server/lib/python/cartodb_services && python setup.py install
    
  • install extensions in user database

    create extension cdb_geocoder;
    create extension plproxy;
    create extension observatory;
    create extension cdb_dataservices_server;
    create extension cdb_dataservices_client;
    
  • add configuration for different services in user database

    # If sentinel is used:
      SELECT CDB_Conf_SetConf('redis_metadata_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "mymaster", "timeout": 0.1, "redis_db": 5}');
      SELECT CDB_Conf_SetConf('redis_metrics_config', '{"sentinel_host": "localhost", "sentinel_port": 26379, "sentinel_master_id": "", "timeout": 0.1, "redis_db": 5}');
    
    # If sentinel is not used
      SELECT CDB_Conf_SetConf('redis_metadata_config', '{"redis_host": "localhost", "redis_port": 26379, "sentinel_master_id": "", "timeout": 0.1, "redis_db": 5}');
      SELECT CDB_Conf_SetConf('redis_metrics_config', '{"redis_host": "localhost", "redis_port": 6379, "sentinel_master_id": "", "timeout": 0.1, "redis_db": 5}');
    
    SELECT CDB_Conf_SetConf('heremaps_conf', '{"geocoder": {"app_id": "here_geocoder_app_id", "app_code": "here_geocoder_app_code", "geocoder_cost_per_hit": "1"}, "isolines" : {"app_id": "here_isolines_app_id", "app_code": "here_geocoder_app_code"}}');
    SELECT CDB_Conf_SetConf('user_config', '{"is_organization": false, "entity_name": "<YOUR_USERNAME>"}')
    SELECT CDB_Conf_SetConf('mapzen_conf', '{"routing": {"api_key": "valhalla_app_key", "monthly_quota": 999999}, "geocoder": {"api_key": "search_app_key", "monthly_quota": 999999}}');
    SELECT CDB_Conf_SetConf('logger_conf', '{"geocoder_log_path": "/tmp/geocodings.log"}')
    SELECT CDB_Conf_SetConf('data_observatory_conf', '{"connection": {"whitelist": [], "production": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api", "staging": "host=localhost port=5432 dbname=dataservices_db user=geocoder_api"}}')
    
  • configure plproxy to point to the a database (you can use a specific database for the server or your same user)

     SELECT CDB_Conf_SetConf('geocoder_server_config', '{ "connection_str": "host=localhost port=5432 dbname=<SERVER_DB_NAME> user=postgres"}');
    
  • configure the search path in order to be able to execute the functions without using the schema:

    ALTER ROLE "<USER_ROLE>" SET search_path="$user", public, cartodb, cdb_dataservices_client;