diff --git a/doc/manual/source/install.rst b/doc/manual/source/install.rst index 26fa824ba6..b65e70e211 100644 --- a/doc/manual/source/install.rst +++ b/doc/manual/source/install.rst @@ -3,10 +3,12 @@ Installation ============ .. warning:: - CartoDB works with Ubuntu 16.04 x64. This documentation describes the process to install CartoDB in this specific OS version. + CARTO works with Ubuntu 16.04 x64. This documentation describes the process to install CartoDB in this specific OS version. However this doesn't mean that it won't work with other Operating Systems or other Ubuntu. There are also many successful installations on Amazon EC2, Linode, dedicated instances and development machines running OS X and Ubuntu 12.04+. + You will find notes along this guide explaining some of the Ubuntu 16.04 specifics, and pointing to alternative solutions for other environments. + System requirements ------------------- Besides the OS version mentioned in the introduction, there are some system requirements needed before starting with the installation of the stack. Also this process assumes that you have enough permissions in the system to run successfully most part of the commands of this doc. @@ -36,17 +38,18 @@ Although we try to maintain packaged versions of almost every part of the stack, .. code-block:: bash - sudo apt-get install autoconf binutils-doc bison build-essential flex + sudo apt-get install make pkg-config + You will also need to install GCC 4.9 in order to build some Node modules later. You can install it doing this: .. code-block:: bash - sudo add-apt-repository ppa:cartodb/gcc && sudo apt-get update sudo apt-get install gcc-4.9 g++-4.9 export CC=/usr/bin/gcc-4.9 export CXX=/usr/bin/g++-4.9 + GIT ~~~ @@ -59,37 +62,27 @@ You will need git commands in order to handle some repositories and install some PostgreSQL ---------- -* Add PPA repository - - .. code-block:: bash - - sudo add-apt-repository ppa:cartodb/postgresql-9.5 && sudo apt-get update - +.. note:: + CARTO requires PostgreSQL 10+. The PPA packages also provide some additional patches, which are not needed but help improve the experience in production environments. -* Install client packages +* Add PPA repository .. code-block:: bash - sudo apt-get install libpq5 \ - libpq-dev \ - postgresql-client-9.5 \ - postgresql-client-common + sudo add-apt-repository ppa:cartodb/postgresql-10 && sudo apt-get update -* Install server packages +* Install packages .. code-block:: bash - sudo apt-get install postgresql-9.5 \ - postgresql-contrib-9.5 \ - postgresql-server-dev-9.5 \ - postgresql-plpython-9.5 - + sudo apt-get install postgresql-10 \ + postgresql-plpython-10 \ + postgresql-server-dev-10 +PostgreSQL access authorization is managed through pg_hba.conf configuration file, which is normally in ``/etc/postgresql/10/main/pg_hba.conf``. Here it's defined how the users created in postgresql cluster can access the server. This involves several aspects like type of authentication (md5, no password, etc..) or source IP of the connection. In order to simplify the process of the installation we are going to allow connections with postgres user from localhost without authentication. Of course this can be configured in a different way at any moment but changes here should imply changes in database access configuration of CARTO apps. -PostgreSQL access authorization is managed through pg_hba.conf configuration file, which is normally in /etc/postgresql/9.5/main/pg_hba.conf. Here it's defined how the users created in postgresql cluster can access the server. This involves several aspects like type of authentication (md5, no password, etc..) or source IP of the connection. In order to simplify the process of the installation we are going to allow connections with postgres user from localhost without authentication. Of course this can be configured in a different way at any moment but changes here should imply changes in database access configuration of CartoDB apps. - -This is the pg_hba.conf with the no password access from localhost: +Edit ``/etc/postgresql/10/main/pg_hba.conf``, modifying the existing lines to use ``trust`` authentication (no password access from localhost): .. code-block:: bash @@ -101,11 +94,10 @@ For these changes to take effect, you'll need to restart postgres: .. code-block:: bash - sudo service postgresql restart - + sudo systemctl restart postgresql -* Create some users in PostgreSQL. These users are used by some CartoDB apps internally +* Create some users in PostgreSQL. These users are used by some CARTO apps internally .. code-block:: bash @@ -130,52 +122,34 @@ GIS dependencies sudo add-apt-repository ppa:cartodb/gis && sudo apt-get update -* Install Proj - - .. code-block:: bash - - sudo apt-get install proj proj-bin proj-data libproj-dev - -* Install JSON - - .. code-block:: bash - - sudo apt-get install libjson0 libjson0-dev python-simplejson - -* Install GEOS - - .. code-block:: bash - - sudo apt-get install libgeos-c1v5 libgeos-dev * Install GDAL .. code-block:: bash - sudo apt-get install gdal-bin libgdal1-dev libgdal-dev - sudo apt-get install gdal2.1-static-bin - + sudo apt-get install gdal-bin libgdal-dev PostGIS ------- +.. note:: + CARTO requires PostGIS 2.4. The PPA just packages this version for Ubuntu 16.04. + * Install PostGIS .. code-block:: bash - sudo apt-get install libxml2-dev - sudo apt-get install liblwgeom-2.2.2 postgis postgresql-9.5-postgis-2.2 postgresql-9.5-postgis-scripts + sudo apt-get install postgis * Initialize template postgis database. We create a template database in postgresql that will contain the postgis extension. This way, every time CartoDB creates a new user database it just clones this template database .. code-block:: bash sudo createdb -T template0 -O postgres -U postgres -E UTF8 template_postgis - sudo createlang plpgsql -U postgres -d template_postgis psql -U postgres template_postgis -c 'CREATE EXTENSION postgis;CREATE EXTENSION postgis_topology;' sudo ldconfig -* Run an installcheck to verify the database has been installed properly +* (Optional) Run an installcheck to verify the database has been installed properly .. code-block:: bash @@ -183,41 +157,37 @@ PostGIS Check https://github.com/cartodb/cartodb-postgresql for further reference -* Restart PostgreSQL after all this process - - .. code-block:: bash - - sudo service postgresql restart - Redis ----- -Redis 3+ is needed. +.. note:: + CARTO requires Redis 4+. You can also optionally install redis-cell for rate limiting, which is not described by this guide. * Add redis PPA .. code-block:: bash - sudo add-apt-repository ppa:cartodb/redis && sudo apt-get update + sudo add-apt-repository ppa:cartodb/redis-next && sudo apt-get update * Install redis .. code-block:: bash - sudo apt-get install redis-server + sudo apt-get install redis .. warning:: - By default redis server is configured to not have any type of disk persistence. If stopped or restarted everything stored in redis will be lost. In CartoDB redis is not just a simple cache storage. It stores information that need to be persisted. + By default redis server is configured to only have periodic snapshotting to disk. If stopped or restarted some data stored in redis since the last snahpshot can be lost. In CARTO redis is not just a simple cache storage. It stores information that need to be persisted. - Make sure to have proper values of *save*, *appendonly* and *appendfsync* config attributes. For more information check `http://redis.io/topics/persistence` + For data safety, make sure to have proper values of *save*, *appendonly* and *appendfsync* config attributes. For more information check `http://redis.io/topics/persistence` NodeJS ------ NodeJS is required by different parts of the stack. The more significant are the Maps and SQL APIs. It's also used to install and execute some dependencies of the editor. + * Add the PPA .. code-block:: bash @@ -237,11 +207,8 @@ NodeJS is required by different parts of the stack. The more significant are the nodejs -v npm -v -If npm version is wrong you should update it: - - .. code-block:: bash - - npm install npm@3.10.9 -g +.. warning:: + CARTO uses Node JS v6.9.2 and npm 3.10.9. You must use these precise versions to avoid problems. We will also install some development libraries that will be necessary to build some Node modules: @@ -259,7 +226,6 @@ SQL API git clone git://github.com/CartoDB/CartoDB-SQL-API.git cd CartoDB-SQL-API - git checkout master * Install npm dependencies @@ -290,20 +256,25 @@ MAPS API git clone git://github.com/CartoDB/Windshaft-cartodb.git cd Windshaft-cartodb - git checkout master -* Install npm dependencies +* Maps API dependencies are managed using yarn. Install it: .. code-block:: bash - npm install + sudo npm install -g yarn@0.27.5 +* Install yarn dependencies + + .. code-block:: bash + + yarn install * Create configuration. The name of the filename of the configuration must be the same than the environment you are going to use to start the service. Let's assume it's development. .. code-block:: bash cp config/environments/development.js.example config/environments/development.js + mkdir logs * Start the service. The second parameter is always the environment of the service. Remember to use the same you used in the configuration. @@ -316,41 +287,29 @@ MAPS API Ruby ---- -* Download ruby-install. Ruby-install is a script that makes ruby install easier. It's not needed to get ruby installed but it helps in the process. - - .. code-block:: bash - - wget -O ruby-install-0.5.0.tar.gz https://github.com/postmodern/ruby-install/archive/v0.5.0.tar.gz - tar -xzvf ruby-install-0.5.0.tar.gz - cd ruby-install-0.5.0/ - sudo make install - -* Install some ruby dependencies - - .. code-block:: bash - - sudo apt-get install libreadline6-dev openssl +.. note:: + CARTO requires exactly Ruby 2.2.x. Older or newer versions won't work. -* Install ruby 2.2.3. CartoDB has been deeply tested with Ruby 2.2. +* Add brightbox ruby repositories .. code-block:: bash - sudo ruby-install ruby 2.2.3 + sudo apt-add-repository ppa:brightbox/ruby-ng && sudo apt-get update -* Ruby-install will leave everything in /opt/rubies/ruby-2.2.3/bin. To be able to run ruby and gem later on, you'll need to add the Ruby 2.2.3 bin folder to your PATH variable. It's also a good idea to include this line in your bashrc so that it gets loaded on restart +* Install ruby 2.2 .. code-block:: bash - export PATH=/opt/rubies/ruby-2.2.3/bin:$PATH + sudo apt-get install ruby2.2 ruby2.2-dev -* Install bundler. Bundler is an app used to manage ruby dependencies. It is needed by CartoDB's editor +* Install bundler. Bundler is an app used to manage ruby dependencies. It is needed by CARTO Builder .. code-block:: bash - sudo gem install bundler + sudo apt-get install ruby-bundler -* Install compass. It will be needed later on by CartoDB's editor +* Install compass. It will be needed later on by CARTO's editor .. code-block:: bash @@ -360,7 +319,10 @@ Ruby Editor ------ -* Download the editor code +.. note:: + CARTO users Python 2.7+. Python 3 will not work correctly. + +* Download Builder's code .. code-block:: bash @@ -378,14 +340,14 @@ Editor .. code-block:: bash - sudo apt-get install python-all-dev + sudo apt-get install python-pip * Install dependencies .. code-block:: bash - sudo apt-get install imagemagick unp zip + sudo apt-get install imagemagick unp zip libicu-dev RAILS_ENV=development bundle install npm install sudo pip install --no-use-wheel -r python_requirements.txt @@ -407,11 +369,6 @@ Editor If gdal keeps failing, see more information here: http://gis.stackexchange.com/questions/28966/python-gdal-package-missing-header-file-when-installing-via-pip -* Add the grunt command to the PATH - - .. code-block:: bash - - export PATH=$PATH:$PWD/node_modules/grunt-cli/bin * Install all necesary gems @@ -419,11 +376,17 @@ Editor bundle install +* Compile static assets + + .. code-block:: bash + + npm run carto-node && npm run build:static -* Precompile assets. Note that the last parameter is the environment used to run the application. It must be the same used in the Maps and SQL APIs +* (Optional) Precompile assets. Needed if you don't want to use CARTO's CDN for assets. .. code-block:: bash + export PATH=$PATH:$PWD/node_modules/grunt-cli/bin bundle exec grunt --environment=development @@ -434,18 +397,18 @@ Editor cp config/app_config.yml.sample config/app_config.yml cp config/database.yml.sample config/database.yml -* Initialize the metadata database +* Start the redis-server that allows access to the SQL and Maps APIs: .. code-block:: bash - RAILS_ENV=development bundle exec rake db:create - RAILS_ENV=development bundle exec rake db:migrate + sudo systemctl start redis-server -* Start the redis-server that allows access to the SQL and Maps APIs: +* Initialize the metadata database .. code-block:: bash - redis-server & + RAILS_ENV=development bundle exec rake db:create + RAILS_ENV=development bundle exec rake db:migrate * Start the editor HTTP server diff --git a/python_requirements.txt b/python_requirements.txt index 2d9f3a8f09..3f13183b5c 100644 --- a/python_requirements.txt +++ b/python_requirements.txt @@ -3,6 +3,6 @@ argparse==1.2.1 brewery==0.6 redis==2.4.9 python-varnish==0.1.2 -gdal==1.10.0 +gdal==2.2.2 csvkit==0.9.0 openpyxl==2.1.3