From 191313fc3c749b8192464e4a5166b0c639bb0eee Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Thu, 22 Oct 2015 11:43:14 +0200 Subject: [PATCH 01/10] docs: split API.md in different files --- doc/API.md | 334 +------------------------------- doc/authentication.md | 20 ++ doc/handling_geospatial_data.md | 56 ++++++ doc/libraries_support.md | 27 +++ doc/making_calls.md | 161 +++++++++++++++ doc/query_optimizations.md | 13 ++ doc/tips_and_tricks.md | 37 ++++ doc/version.md | 3 + 8 files changed, 327 insertions(+), 324 deletions(-) create mode 100644 doc/authentication.md create mode 100644 doc/handling_geospatial_data.md create mode 100644 doc/libraries_support.md create mode 100644 doc/making_calls.md create mode 100644 doc/query_optimizations.md create mode 100644 doc/tips_and_tricks.md create mode 100644 doc/version.md diff --git a/doc/API.md b/doc/API.md index d4ead1e8..dad50e1b 100644 --- a/doc/API.md +++ b/doc/API.md @@ -1,4 +1,4 @@ -## SQL API +# SQL API CartoDB's SQL API allows you to interact with your tables and data inside CartoDB as if you were running SQL statements against a normal database. The database behind CartoDB is PostgreSQL so if you need help with specific SQL statements or you want to learn more about it, visit the [official documentation](http://www.postgresql.org/docs/9.1/static/sql.html). @@ -10,326 +10,12 @@ There are two main situations in which you would want to use the SQL API: Remember that in order to access, read or modify data in private tables, you will need to authenticate your requests. When a table is public, you can do non-authenticated queries that read data, but you cannot write or modify data without authentication. -## Authentication - -For all access to private tables and for write access to public tables, CartoDB enforces secure API access that requires you to authorize your queries. In order to authorize queries, you can use an API key or a Consumer Key. - -### API Key - -The API key offers the simplest way to access private data or perform writes and updates to your public data. Remember that your API key protects access to your data, so keep it confidential and only share it if you want others to have this access. If necessary, you can reset your API key in your admin dashboard. - -To find your API key: - -- Go to your dashboard. -- Click on your username in the top right corner, and select "Your API keys." -- Here, you can copy your API key, see use examples, and reset your API key. - -To use your API key, pass it as a parameter in an URL call to the CartoDB API. For example, to perform an insert into your table, you would use the following URL structure. - -
Query example with the api_key parameter
-```bash -https://{account}.cartodb.com/api/v2/sql?q={SQL statement}&api_key={Your API key} -``` - -## Making calls to the SQL API - -CartoDB is based on the rock solid PostgreSQL database. All of your tables reside a single database, which means you can perform complex queries joining tables or carrying out geospatial operations. The best place to learn about PostgreSQL's SQL language is the [official documentation](http://www.postgresql.org/docs/9.1/static/). - -CartoDB is also based on PostGIS, so take a look at the [official PostGIS reference](http://postgis.refractions.net/docs/) to know what functionality we support in terms of geospatial operations. All of our tables include a column called *the_geom,* which is a geometry field that indexes geometries in the EPSG:4326 (WGS 1984) coordinate system. All tables also have an automatically generated and updated column called *the_geom_webmercator*. We use the column internally to quickly create tiles for maps. - -### URL endpoints - -All SQL API requests to your CartoDB account should follow this general pattern: - -
SQL QUERY EXAMPLE
-```bash -https://{account}.cartodb.com/api/v2/sql?q={SQL statement} -``` - -If you encounter errors, double-check that you are using the correct account name, and that your SQL statement is valid. A simple example of this pattern is conducting a count of all the records in your table: - -
SQL QUERY COUNT EXAMPLE
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name} -``` - -
RESULT
-```javascript -{ - time: 0.007, - total_rows: 1, - rows: [ - { - count: 4994 - } - ] -} -``` - -Finally, remember that in order to use the SQL API, either your table must be public, or you must be authenticated using API Keys, as discussed above. - - -### POST and GET - -The CartoDB SQL API is setup to handle both GET and POST requests. You can test the GET method directly in your browser. Below is an example of a JQuery SQL API request to CartoDB: - -
JQUERY
-```javascript -$.getJSON('https://'+your_account_name+'.cartodb.com/api/v2/sql/?q='+sql_statement, function(data) { - $.each(data.rows, function(key, val) { - // do something! - }); -}); -``` - -By default, GET requests work from anywhere. In CartoDB, POST requests work from any website as well. We achieve this by hosting a cross-domain policy file at the root of all of our servers. This allows you the greatest level of flexibility when developing your application. - -### Response formats - -The standard response from the CartoDB SQL API is JSON. If you are building a web-application, the lightweight JSON format allows you to quickly integrate data from the SQL API. - -
JSON
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 -``` - -
RESULT
-```javascript -{ - time: 0.006, - total_rows: 1, - rows: [ - { - year: " 2011", - month: 10, - day: "11", - the_geom: "0101000020E610...", - cartodb_id: 1, - the_geom_webmercator: "0101000020110F000..." - } - ] -} -``` - -Alternatively, you can use the [GeoJSON specification](http://www.geojson.org/geojson-spec.html) to return data from the API. To do so, simply supply the `format` parameter as GeoJSON: - -
GEOJSON
-```bash -https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_name} LIMIT 1 -``` - -
RESULT
-```javascript -{ - type: "FeatureCollection", - features: [ - { - type: "Feature", - properties: { - year: " 2011", - month: 10, - day: "11", - cartodb_id: 1 - }, - geometry: { - type: "Point", - coordinates: [ - -97.335, - 35.498 - ] - } - } - ] -} -``` - -The SQL API accepts other output formats that can be useful to export data. Right now you can use the following formats: CSV, SHP, SVG, KML, SpatiaLite and GeoJSON. - -### Output filename -To customize the output filename, add the `filename` parameter to your URL: - -
Customize filename
-```bash -https://{account}.cartodb.com/api/v2/sql?filename={custom_filename}&q=SELECT * FROM {table_name} LIMIT 1 -``` - -### Getting table information - -Currently, there is no public method to access your table schemas. The simplest way to retrieve table structure is to access the first row of the data, - -
COLUMNS
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 -``` - -### Response errors - -To help you debug your SQL queries, the CartoDB SQL API returns errors as part of the JSON response. Errors come back as follows, - -
RESULT
-```javascript -{ - error: [ - "syntax error at or near "LIMIT"" - ] -} -``` - -You can use these errors to help understand your SQL. For more complete documentation see the Error Codes and Solutions section of this Users Guide. - -### Write data to your CartoDB account - -Performing inserts or updates on your data is simple using your [API key](#authentication). All you need to do is supply a correct SQL [INSERT](http://www.postgresql.org/docs/9.1/static/sql-insert.html) or [UPDATE](http://www.postgresql.org/docs/9.1/static/sql-update.html) statement for your table along with the api_key parameter for your account. Be sure to keep these requests private, as anyone with your API key will be able to modify your tables. A correct SQL insert statement means that all the columns you want to insert into already exist in your table, and all the values for those columns are the right type (quoted string, unquoted string for geoms and dates, or numbers). - -
COLUMNS
-```bash -https://{account}.cartodb.com/api/v2/sql?q=INSERT INTO test_table (column_name, column_name_2, the_geom) VALUES ('this is a string', 11, ST_SetSRID(ST_Point(-110, 43),4326))&api_key={Your API key} -``` - -Updates are just as simple. Here is an example, updating a row based on the value of the cartodb_id column. - -
COLUMNS
-```bash -https://{account}.cartodb.com/api/v2/sql?q=UPDATE test_table SET column_name = 'my new string value' WHERE cartodb_id = 1 &api_key={Your API key} -``` - -## Handling geospatial data - -Handling geospatial data through the SQL API is easy! By default, *the_geom* is returned straight from the database, in a format called Well-Known Binary. There are a handful of ways you can transform your geometries into more useful formats. - - -The first, is to use the format=GeoJSON method described above. Others can be handled through your SQL statements directly. For example, enclosing your the_geom in a function called [ST_AsGeoJSON](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) will allow you to use JSON for your data but a GeoJSON string for your geometry column only. Alternatively, using a the [ST_AsText](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) function will return your geometry as Well-Known Text. - -
ASGEOJSON
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_geom) as the_geom FROM {table_name} LIMIT 1 -``` - -
RESULT
-```javascript -{ - time: 0.003, - total_rows: 1, - rows: [ - { - cartodb_id: 1, - the_geom: "{"type":"Point","coordinates":[-97.3349,35.4979]}" - } - ] -} -``` - - -
ASTEXT
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom) FROM {table_name} LIMIT 1 -``` - -
RESULT
-```javascript -{ - time: 0.003, - total_rows: 1, - rows: [ - { - cartodb_id: 1, - the_geom: "POINT(-74.0004162 40.6920918)", - } - ] -} -``` - -More advanced methods exist in the PostGIS library to extract meaningful data from your geometry. Explore the PostGIS documentation and get familiar with functions such as, [ST_XMin](http://www.postgis.org/docs/ST_XMin.html), [ST_XMax](http://www.postgis.org/docs/ST_XMax.html), [ST_AsText](http://www.postgis.org/docs/ST_AsText.html), and more. - -All data returned from *the_geom* column is in WGS 84 (EPSG:4326). You can change this quickly and easily on the fly using SQL. For example, if you desire geometries in the Hanoi 1972 (EPSG:4147) projection, you could [ST_Transform](http://www.postgis.org/docs/ST_Transform.html), - -
ASTEXT
-```bash -https://{account}.cartodb.com/api/v2/sql?q=SELECT ST_Transform(the_geom,4147) FROM {table_name} LIMIT 1 -``` - -CartoDB also stores a second geometry column, *the_geom_webmercator*. We use this internally to build your map tiles as fast as we can. In the user-interface it is hidden, but it is visible and available for use. In this column we store a reprojected version of all your geometries using Web Mercator (EPSG:3857). - -## Query optimizations - -There are some tricks to consider when using the SQL API that might make your application a little faster. - -* Only request the fields you need. Selecting all columns will return a full version of your geometry in *the_geom* as well as a reprojected version in *the_geom_webmercator*. - -* Use PostGIS functions to simplify and filter out unneeded geometries when possible. One very handy function is, [ST_Simplify](http://www.postgis.org/docs/ST_Simplify.html). - -* Remember to build indexes that will speed up some of your more common queries. - -* Use *cartodb_id* to retrieve specific rows of your data, this is the unique key column added to every CartoDB table. - - - -## API version number - -All CartoDB applications use **Version 2** of our APIs. All other APIs are deprecated and will not be maintained or supported. You can check that you are using **Version 2** of our APIs by looking at your request URLS. They should all begin contain **/v2/** in the URLs as follows `https://{account}.cartodb.com/api/v2/` - -## Libraries in different languages - -To make things easier for developers, we provide client libraries for different programming languages and caching functionalities. - -- **R** - To help more researchers use CartoDB to drive their geospatial data, we have released the R client library. [Fork it on GitHub!](https://github.com/Vizzuality/cartodb-r) - -- **NODE.js** - This demo app authenticates with your CartoDB and shows how to perform read and write queries using the SQL API. [Fork it on GitHub!](https://github.com/Vizzuality/cartodb-nodejs) - -- **PHP** - The PHP library provides a wrapper around the SQL API to get PHP objects straight from SQL calls to CartoDB. [Fork it on GitHub!](https://github.com/Vizzuality/cartodbclient-php) - -- **PYTHON** - Provides API Key access to SQL API. [Fork it on GitHub!](https://github.com/vizzuality/cartodb-python) - -- **JAVA** - Very basic example of how to access CartoDB SQL API. [Fork it on GitHub!](https://github.com/cartodb/cartodb-java-client) - -- **NET** - .NET library for authenticating with CartoDB using an API key, based on work started by [The Data Republic](http://www.thedatarepublic.com/). [Fork it on GitHub!](https://github.com/thedatarepublic/CartoDBClientDotNET) - -- **Clojure** - Clojure library for authenticating with CartoDB, maintained by [REDD Metrics](http://www.reddmetrics.com/). [Fork it on GitHub!](https://github.com/reddmetrics/cartodb-clj) - -- **iOS** - Objective-C library for interacting with CartoDB in native iOS applications. [Fork it on GitHub!](https://github.com/jmnavarro/cartodb-objectivec-client) - -## Other Tips and Questions - -### What does CartoDB do to prevent SQL injection? - -CartoDB uses the database access mechanism itself for security. Every writable connection is verified by an API key, and if you have the correct API key, you can write to anything the database allows you to write to. If you don’t have the correct API key, your client is "logged in" as a low privilege user, and you can read anything the database allows you to read. - -SQL injection works by tricking a database user that is only showing you certain parts of the database to show all of it, or by tricking the database into writing things it shouldn't. This happens when the database connection has perhaps more privileges than you would freely hand out to your API users. - -Because CartoDB enforces roles and access at the database level, the idea of a “SQL injection attack” is not possible with CartoDB. Injection is possible, but clients will still run into our security wall at the database level. Put another way, the SQL API already lets you _attempt_ to run any query you want. The database will reject your SQL API request if it finds your user/role doesn't have the requisite permissions. In other words, you can ask any question of the database you like; the CartoDB database doesn’t guarantee it will be answered. - -If a user's API key found its way out into the wild, then that would be a problem but is not something CartoDB can prevent. This is why it is very important for all CartoDB users to secure their API keys. In the event a user's API key is compromised, either the user or the CartoDB Enterprise administrator can regenerate the API key in their account settings. - -### What levels of database access can roles/users have? - -There are three levels of access with CartoDB: - -1. __API Key level:__ Do whatever you want in your account on the tables you own (or have been shared with you in Enterprise/multi-user accounts). -2. __"publicuser" level:__ Do whatever has been granted to you. The publicuser level is normally read-only, but you could GRANT INSERT/UPDATE/DELETE permissions to publicuser if needed for some reason - for API key-less write operations. Use with caution. -3. __postgres superadmin level:__ This third access level, the actual PostgreSQL system user, is only accessible from a direct database connection via the command line, which is only available currently via [CartoDB On-Premises](https://cartodb.com/on-premises/). - -### If a user has write access and makes a `DROP TABLE` query, is that data gone? - -Yes. Grant write access with caution and keep backups of your data elsewhere / as duplicate CartoDB tables. - -### Is there an in between where a user can write but not `DROP` or `DELETE`? - -Yes. Create the table, and GRANT INSERT/UPDATE to the user. - -### Is there an actual PostgreSQL account for each CartoDB login/username? - -Yes, there is. Unfortunately, the names are different - though there is a way to determine the name of the PostgreSQL user account. Every CartoDB user gets their own PostgreSQL database. But there’s a system database too, with the name mappings in `username` and `database_name` columns. `database_name` is the name of the database that user belongs to. It will be `cartodb_user_ID`. `id` holds long hashkey. The `database_name` is derived from this ID hash too, but in case of an Enterprise/multi-user account it will come from the user ID of the owner of the organization - and `database_name` will hold the same value for every user in an Enterprise/multi-user account. - -You can also just do `select user` using the SQL API (without an API key to get the publicuser name and with an API key to get the CartoDB user's PostgreSQL user name), to determine the name of the corresponding PostgreSQL user. - -### Could I configure my CartoDB database permissions exactly the same way I could on my own PostgreSQL instance? - -Yes, through using GRANT statements to the SQL API. There are a few caveats to be aware of, including the aforementioned naming differences. Also, you'll be limited to permissions a user has with their own tables. Users don’t have PostgreSQL superuser privileges. So they can’t be creating languages, or C functions, or anything that requires superuser or CREATEUSER privileges. +## Documentation + +* [Authentication](/CartoDB/CartoDB-SQL-API/blob/master/doc/authentication.md) +* [Making calls to the SQL API](/CartoDB/CartoDB-SQL-API/blob/master/doc/making_calls.md) +* [Handling geospatial data](/CartoDB/CartoDB-SQL-API/blob/master/doc/handling_geospatial_data.md) +* [Query optimizations](/CartoDB/CartoDB-SQL-API/blob/master/doc/query_optimizations.md) +* [API version number](/CartoDB/CartoDB-SQL-API/blob/master/doc/version.md) +* [Libraries in different languages](/CartoDB/CartoDB-SQL-API/blob/master/doc/libraries_support.md) +* [Other Tips and Questions](/CartoDB/CartoDB-SQL-API/blob/master/doc/tips_and_tricks.md) diff --git a/doc/authentication.md b/doc/authentication.md new file mode 100644 index 00000000..25bff58d --- /dev/null +++ b/doc/authentication.md @@ -0,0 +1,20 @@ +# Authentication + +For all access to private tables and for write access to public tables, CartoDB enforces secure API access that requires you to authorize your queries. In order to authorize queries, you can use an API key or a Consumer Key. + +## API Key + +The API key offers the simplest way to access private data or perform writes and updates to your public data. Remember that your API key protects access to your data, so keep it confidential and only share it if you want others to have this access. If necessary, you can reset your API key in your admin dashboard. + +To find your API key: + +- Go to your dashboard. +- Click on your username in the top right corner, and select "Your API keys." +- Here, you can copy your API key, see use examples, and reset your API key. + +To use your API key, pass it as a parameter in an URL call to the CartoDB API. For example, to perform an insert into your table, you would use the following URL structure. + +
Query example with the api_key parameter
+```bash +https://{account}.cartodb.com/api/v2/sql?q={SQL statement}&api_key={Your API key} +``` diff --git a/doc/handling_geospatial_data.md b/doc/handling_geospatial_data.md new file mode 100644 index 00000000..e19da01a --- /dev/null +++ b/doc/handling_geospatial_data.md @@ -0,0 +1,56 @@ +# Handling geospatial data + +Handling geospatial data through the SQL API is easy! By default, *the_geom* is returned straight from the database, in a format called Well-Known Binary. There are a handful of ways you can transform your geometries into more useful formats. + + +The first, is to use the format=GeoJSON method described above. Others can be handled through your SQL statements directly. For example, enclosing your the_geom in a function called [ST_AsGeoJSON](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) will allow you to use JSON for your data but a GeoJSON string for your geometry column only. Alternatively, using a the [ST_AsText](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) function will return your geometry as Well-Known Text. + +
ASGEOJSON
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_geom) as the_geom FROM {table_name} LIMIT 1 +``` + +
RESULT
+```javascript +{ + time: 0.003, + total_rows: 1, + rows: [ + { + cartodb_id: 1, + the_geom: "{"type":"Point","coordinates":[-97.3349,35.4979]}" + } + ] +} +``` + + +
ASTEXT
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom) FROM {table_name} LIMIT 1 +``` + +
RESULT
+```javascript +{ + time: 0.003, + total_rows: 1, + rows: [ + { + cartodb_id: 1, + the_geom: "POINT(-74.0004162 40.6920918)", + } + ] +} +``` + +More advanced methods exist in the PostGIS library to extract meaningful data from your geometry. Explore the PostGIS documentation and get familiar with functions such as, [ST_XMin](http://www.postgis.org/docs/ST_XMin.html), [ST_XMax](http://www.postgis.org/docs/ST_XMax.html), [ST_AsText](http://www.postgis.org/docs/ST_AsText.html), and more. + +All data returned from *the_geom* column is in WGS 84 (EPSG:4326). You can change this quickly and easily on the fly using SQL. For example, if you desire geometries in the Hanoi 1972 (EPSG:4147) projection, you could [ST_Transform](http://www.postgis.org/docs/ST_Transform.html), + +
ASTEXT
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT ST_Transform(the_geom,4147) FROM {table_name} LIMIT 1 +``` + +CartoDB also stores a second geometry column, *the_geom_webmercator*. We use this internally to build your map tiles as fast as we can. In the user-interface it is hidden, but it is visible and available for use. In this column we store a reprojected version of all your geometries using Web Mercator (EPSG:3857). diff --git a/doc/libraries_support.md b/doc/libraries_support.md new file mode 100644 index 00000000..1b436ce3 --- /dev/null +++ b/doc/libraries_support.md @@ -0,0 +1,27 @@ +# Libraries in different languages + +To make things easier for developers, we provide client libraries for different programming languages and caching functionalities. + +- **R** + To help more researchers use CartoDB to drive their geospatial data, we have released the R client library. [Fork it on GitHub!](https://github.com/Vizzuality/cartodb-r) + +- **NODE.js** + This demo app authenticates with your CartoDB and shows how to perform read and write queries using the SQL API. [Fork it on GitHub!](https://github.com/Vizzuality/cartodb-nodejs) + +- **PHP** + The PHP library provides a wrapper around the SQL API to get PHP objects straight from SQL calls to CartoDB. [Fork it on GitHub!](https://github.com/Vizzuality/cartodbclient-php) + +- **PYTHON** + Provides API Key access to SQL API. [Fork it on GitHub!](https://github.com/vizzuality/cartodb-python) + +- **JAVA** + Very basic example of how to access CartoDB SQL API. [Fork it on GitHub!](https://github.com/cartodb/cartodb-java-client) + +- **NET** + .NET library for authenticating with CartoDB using an API key, based on work started by [The Data Republic](http://www.thedatarepublic.com/). [Fork it on GitHub!](https://github.com/thedatarepublic/CartoDBClientDotNET) + +- **Clojure** + Clojure library for authenticating with CartoDB, maintained by [REDD Metrics](http://www.reddmetrics.com/). [Fork it on GitHub!](https://github.com/reddmetrics/cartodb-clj) + +- **iOS** + Objective-C library for interacting with CartoDB in native iOS applications. [Fork it on GitHub!](https://github.com/jmnavarro/cartodb-objectivec-client) diff --git a/doc/making_calls.md b/doc/making_calls.md new file mode 100644 index 00000000..8e56f762 --- /dev/null +++ b/doc/making_calls.md @@ -0,0 +1,161 @@ +# Making calls to the SQL API + +CartoDB is based on the rock solid PostgreSQL database. All of your tables reside a single database, which means you can perform complex queries joining tables or carrying out geospatial operations. The best place to learn about PostgreSQL's SQL language is the [official documentation](http://www.postgresql.org/docs/9.1/static/). + +CartoDB is also based on PostGIS, so take a look at the [official PostGIS reference](http://postgis.refractions.net/docs/) to know what functionality we support in terms of geospatial operations. All of our tables include a column called *the_geom,* which is a geometry field that indexes geometries in the EPSG:4326 (WGS 1984) coordinate system. All tables also have an automatically generated and updated column called *the_geom_webmercator*. We use the column internally to quickly create tiles for maps. + +## URL endpoints + +All SQL API requests to your CartoDB account should follow this general pattern: + +
SQL QUERY EXAMPLE
+```bash +https://{account}.cartodb.com/api/v2/sql?q={SQL statement} +``` + +If you encounter errors, double-check that you are using the correct account name, and that your SQL statement is valid. A simple example of this pattern is conducting a count of all the records in your table: + +
SQL QUERY COUNT EXAMPLE
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name} +``` + +
RESULT
+```javascript +{ + time: 0.007, + total_rows: 1, + rows: [ + { + count: 4994 + } + ] +} +``` + +Finally, remember that in order to use the SQL API, either your table must be public, or you must be authenticated using API Keys, as discussed above. + + +## POST and GET + +The CartoDB SQL API is setup to handle both GET and POST requests. You can test the GET method directly in your browser. Below is an example of a JQuery SQL API request to CartoDB: + +
JQUERY
+```javascript +$.getJSON('https://'+your_account_name+'.cartodb.com/api/v2/sql/?q='+sql_statement, function(data) { + $.each(data.rows, function(key, val) { + // do something! + }); +}); +``` + +By default, GET requests work from anywhere. In CartoDB, POST requests work from any website as well. We achieve this by hosting a cross-domain policy file at the root of all of our servers. This allows you the greatest level of flexibility when developing your application. + +## Response formats + +The standard response from the CartoDB SQL API is JSON. If you are building a web-application, the lightweight JSON format allows you to quickly integrate data from the SQL API. + +
JSON
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 +``` + +
RESULT
+```javascript +{ + time: 0.006, + total_rows: 1, + rows: [ + { + year: " 2011", + month: 10, + day: "11", + the_geom: "0101000020E610...", + cartodb_id: 1, + the_geom_webmercator: "0101000020110F000..." + } + ] +} +``` + +Alternatively, you can use the [GeoJSON specification](http://www.geojson.org/geojson-spec.html) to return data from the API. To do so, simply supply the `format` parameter as GeoJSON: + +
GEOJSON
+```bash +https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_name} LIMIT 1 +``` + +
RESULT
+```javascript +{ + type: "FeatureCollection", + features: [ + { + type: "Feature", + properties: { + year: " 2011", + month: 10, + day: "11", + cartodb_id: 1 + }, + geometry: { + type: "Point", + coordinates: [ + -97.335, + 35.498 + ] + } + } + ] +} +``` + +The SQL API accepts other output formats that can be useful to export data. Right now you can use the following formats: CSV, SHP, SVG, KML, SpatiaLite and GeoJSON. + +## Output filename +To customize the output filename, add the `filename` parameter to your URL: + +
Customize filename
+```bash +https://{account}.cartodb.com/api/v2/sql?filename={custom_filename}&q=SELECT * FROM {table_name} LIMIT 1 +``` + +## Getting table information + +Currently, there is no public method to access your table schemas. The simplest way to retrieve table structure is to access the first row of the data, + +
COLUMNS
+```bash +https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 +``` + +## Response errors + +To help you debug your SQL queries, the CartoDB SQL API returns errors as part of the JSON response. Errors come back as follows, + +
RESULT
+```javascript +{ + error: [ + "syntax error at or near "LIMIT"" + ] +} +``` + +You can use these errors to help understand your SQL. For more complete documentation see the Error Codes and Solutions section of this Users Guide. + +## Write data to your CartoDB account + +Performing inserts or updates on your data is simple using your [API key](#authentication). All you need to do is supply a correct SQL [INSERT](http://www.postgresql.org/docs/9.1/static/sql-insert.html) or [UPDATE](http://www.postgresql.org/docs/9.1/static/sql-update.html) statement for your table along with the api_key parameter for your account. Be sure to keep these requests private, as anyone with your API key will be able to modify your tables. A correct SQL insert statement means that all the columns you want to insert into already exist in your table, and all the values for those columns are the right type (quoted string, unquoted string for geoms and dates, or numbers). + +
COLUMNS
+```bash +https://{account}.cartodb.com/api/v2/sql?q=INSERT INTO test_table (column_name, column_name_2, the_geom) VALUES ('this is a string', 11, ST_SetSRID(ST_Point(-110, 43),4326))&api_key={Your API key} +``` + +Updates are just as simple. Here is an example, updating a row based on the value of the cartodb_id column. + +
COLUMNS
+```bash +https://{account}.cartodb.com/api/v2/sql?q=UPDATE test_table SET column_name = 'my new string value' WHERE cartodb_id = 1 &api_key={Your API key} +``` diff --git a/doc/query_optimizations.md b/doc/query_optimizations.md new file mode 100644 index 00000000..fab699cb --- /dev/null +++ b/doc/query_optimizations.md @@ -0,0 +1,13 @@ +# Query optimizations + +There are some tricks to consider when using the SQL API that might make your application a little faster. + +* Only request the fields you need. Selecting all columns will return a full version of your geometry in *the_geom* as well as a reprojected version in *the_geom_webmercator*. + +* Use PostGIS functions to simplify and filter out unneeded geometries when possible. One very handy function is, [ST_Simplify](http://www.postgis.org/docs/ST_Simplify.html). + +* Remember to build indexes that will speed up some of your more common queries. + +* Use *cartodb_id* to retrieve specific rows of your data, this is the unique key column added to every CartoDB table. + + diff --git a/doc/tips_and_tricks.md b/doc/tips_and_tricks.md new file mode 100644 index 00000000..540d228d --- /dev/null +++ b/doc/tips_and_tricks.md @@ -0,0 +1,37 @@ +# Other Tips and Questions + +## What does CartoDB do to prevent SQL injection? + +CartoDB uses the database access mechanism itself for security. Every writable connection is verified by an API key, and if you have the correct API key, you can write to anything the database allows you to write to. If you don’t have the correct API key, your client is "logged in" as a low privilege user, and you can read anything the database allows you to read. + +SQL injection works by tricking a database user that is only showing you certain parts of the database to show all of it, or by tricking the database into writing things it shouldn't. This happens when the database connection has perhaps more privileges than you would freely hand out to your API users. + +Because CartoDB enforces roles and access at the database level, the idea of a “SQL injection attack” is not possible with CartoDB. Injection is possible, but clients will still run into our security wall at the database level. Put another way, the SQL API already lets you _attempt_ to run any query you want. The database will reject your SQL API request if it finds your user/role doesn't have the requisite permissions. In other words, you can ask any question of the database you like; the CartoDB database doesn’t guarantee it will be answered. + +If a user's API key found its way out into the wild, then that would be a problem but is not something CartoDB can prevent. This is why it is very important for all CartoDB users to secure their API keys. In the event a user's API key is compromised, either the user or the CartoDB Enterprise administrator can regenerate the API key in their account settings. + +## What levels of database access can roles/users have? + +There are three levels of access with CartoDB: + +1. __API Key level:__ Do whatever you want in your account on the tables you own (or have been shared with you in Enterprise/multi-user accounts). +2. __"publicuser" level:__ Do whatever has been granted to you. The publicuser level is normally read-only, but you could GRANT INSERT/UPDATE/DELETE permissions to publicuser if needed for some reason - for API key-less write operations. Use with caution. +3. __postgres superadmin level:__ This third access level, the actual PostgreSQL system user, is only accessible from a direct database connection via the command line, which is only available currently via [CartoDB On-Premises](https://cartodb.com/on-premises/). + +## If a user has write access and makes a `DROP TABLE` query, is that data gone? + +Yes. Grant write access with caution and keep backups of your data elsewhere / as duplicate CartoDB tables. + +## Is there an in between where a user can write but not `DROP` or `DELETE`? + +Yes. Create the table, and GRANT INSERT/UPDATE to the user. + +## Is there an actual PostgreSQL account for each CartoDB login/username? + +Yes, there is. Unfortunately, the names are different - though there is a way to determine the name of the PostgreSQL user account. Every CartoDB user gets their own PostgreSQL database. But there’s a system database too, with the name mappings in `username` and `database_name` columns. `database_name` is the name of the database that user belongs to. It will be `cartodb_user_ID`. `id` holds long hashkey. The `database_name` is derived from this ID hash too, but in case of an Enterprise/multi-user account it will come from the user ID of the owner of the organization - and `database_name` will hold the same value for every user in an Enterprise/multi-user account. + +You can also just do `select user` using the SQL API (without an API key to get the publicuser name and with an API key to get the CartoDB user's PostgreSQL user name), to determine the name of the corresponding PostgreSQL user. + +## Could I configure my CartoDB database permissions exactly the same way I could on my own PostgreSQL instance? + +Yes, through using GRANT statements to the SQL API. There are a few caveats to be aware of, including the aforementioned naming differences. Also, you'll be limited to permissions a user has with their own tables. Users don’t have PostgreSQL superuser privileges. So they can’t be creating languages, or C functions, or anything that requires superuser or CREATEUSER privileges. diff --git a/doc/version.md b/doc/version.md new file mode 100644 index 00000000..784fbdad --- /dev/null +++ b/doc/version.md @@ -0,0 +1,3 @@ +# API version number + +All CartoDB applications use **Version 2** of our APIs. All other APIs are deprecated and will not be maintained or supported. You can check that you are using **Version 2** of our APIs by looking at your request URLS. They should all begin contain **/v2/** in the URLs as follows `https://{account}.cartodb.com/api/v2/` From 981f60d40f7c81f28f5fdbf2a8f3d0afe7c455f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Thu, 22 Oct 2015 13:25:47 +0200 Subject: [PATCH 02/10] newdocs: relative links --- doc/API.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/API.md b/doc/API.md index dad50e1b..b423a615 100644 --- a/doc/API.md +++ b/doc/API.md @@ -12,10 +12,10 @@ Remember that in order to access, read or modify data in private tables, you wil ## Documentation -* [Authentication](/CartoDB/CartoDB-SQL-API/blob/master/doc/authentication.md) -* [Making calls to the SQL API](/CartoDB/CartoDB-SQL-API/blob/master/doc/making_calls.md) -* [Handling geospatial data](/CartoDB/CartoDB-SQL-API/blob/master/doc/handling_geospatial_data.md) -* [Query optimizations](/CartoDB/CartoDB-SQL-API/blob/master/doc/query_optimizations.md) -* [API version number](/CartoDB/CartoDB-SQL-API/blob/master/doc/version.md) -* [Libraries in different languages](/CartoDB/CartoDB-SQL-API/blob/master/doc/libraries_support.md) -* [Other Tips and Questions](/CartoDB/CartoDB-SQL-API/blob/master/doc/tips_and_tricks.md) +* [Authentication](authentication.md) +* [Making calls to the SQL API](making_calls.md) +* [Handling geospatial data](handling_geospatial_data.md) +* [Query optimizations](query_optimizations.md) +* [API version number](version.md) +* [Libraries in different languages](libraries_support.md) +* [Other Tips and Questions](tips_and_tricks.md) From 450ea8f7e5408c15393ec25e1b5556d6293e86e1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Wed, 28 Oct 2015 12:35:07 +0100 Subject: [PATCH 03/10] doc: authentication --- doc/authentication.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/authentication.md b/doc/authentication.md index 25bff58d..369b5a58 100644 --- a/doc/authentication.md +++ b/doc/authentication.md @@ -14,7 +14,8 @@ To find your API key: To use your API key, pass it as a parameter in an URL call to the CartoDB API. For example, to perform an insert into your table, you would use the following URL structure. -
Query example with the api_key parameter
+#### Example + ```bash https://{account}.cartodb.com/api/v2/sql?q={SQL statement}&api_key={Your API key} ``` From 8bde217e7c24a4b61b57a71d0453f185bff76f71 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Wed, 28 Oct 2015 12:37:51 +0100 Subject: [PATCH 04/10] doc: handling_geospatial_data --- doc/handling_geospatial_data.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/doc/handling_geospatial_data.md b/doc/handling_geospatial_data.md index e19da01a..421978eb 100644 --- a/doc/handling_geospatial_data.md +++ b/doc/handling_geospatial_data.md @@ -2,15 +2,18 @@ Handling geospatial data through the SQL API is easy! By default, *the_geom* is returned straight from the database, in a format called Well-Known Binary. There are a handful of ways you can transform your geometries into more useful formats. - The first, is to use the format=GeoJSON method described above. Others can be handled through your SQL statements directly. For example, enclosing your the_geom in a function called [ST_AsGeoJSON](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) will allow you to use JSON for your data but a GeoJSON string for your geometry column only. Alternatively, using a the [ST_AsText](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) function will return your geometry as Well-Known Text. -
ASGEOJSON
+### ST_AsGeoJSON + +#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_geom) as the_geom FROM {table_name} LIMIT 1 ``` -
RESULT
+#### Result + ```javascript { time: 0.003, @@ -24,13 +27,16 @@ https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_ge } ``` +### ST_AsText + +#### Call -
ASTEXT
```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom) FROM {table_name} LIMIT 1 ``` -
RESULT
+#### Result + ```javascript { time: 0.003, @@ -48,7 +54,8 @@ More advanced methods exist in the PostGIS library to extract meaningful data fr All data returned from *the_geom* column is in WGS 84 (EPSG:4326). You can change this quickly and easily on the fly using SQL. For example, if you desire geometries in the Hanoi 1972 (EPSG:4147) projection, you could [ST_Transform](http://www.postgis.org/docs/ST_Transform.html), -
ASTEXT
+### ST_Transform + ```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT ST_Transform(the_geom,4147) FROM {table_name} LIMIT 1 ``` From 23c8749d03a6c913838e067f31ac945f217b7eff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Wed, 28 Oct 2015 14:31:42 +0100 Subject: [PATCH 05/10] docs: making calls --- doc/making_calls.md | 58 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 14 deletions(-) diff --git a/doc/making_calls.md b/doc/making_calls.md index 8e56f762..de616110 100644 --- a/doc/making_calls.md +++ b/doc/making_calls.md @@ -4,23 +4,27 @@ CartoDB is based on the rock solid PostgreSQL database. All of your tables resid CartoDB is also based on PostGIS, so take a look at the [official PostGIS reference](http://postgis.refractions.net/docs/) to know what functionality we support in terms of geospatial operations. All of our tables include a column called *the_geom,* which is a geometry field that indexes geometries in the EPSG:4326 (WGS 1984) coordinate system. All tables also have an automatically generated and updated column called *the_geom_webmercator*. We use the column internally to quickly create tiles for maps. + ## URL endpoints All SQL API requests to your CartoDB account should follow this general pattern: -
SQL QUERY EXAMPLE
+#### SQL query example + ```bash https://{account}.cartodb.com/api/v2/sql?q={SQL statement} ``` If you encounter errors, double-check that you are using the correct account name, and that your SQL statement is valid. A simple example of this pattern is conducting a count of all the records in your table: -
SQL QUERY COUNT EXAMPLE
+#### Count example + ```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name} ``` -
RESULT
+#### Result + ```javascript { time: 0.007, @@ -38,9 +42,12 @@ Finally, remember that in order to use the SQL API, either your table must be pu ## POST and GET -The CartoDB SQL API is setup to handle both GET and POST requests. You can test the GET method directly in your browser. Below is an example of a JQuery SQL API request to CartoDB: +The CartoDB SQL API is setup to handle both GET and POST requests. You can test the GET method directly in your browser. Below is an example of a jQuery SQL API request to CartoDB: + +### jQuery + +#### Call -
JQUERY
```javascript $.getJSON('https://'+your_account_name+'.cartodb.com/api/v2/sql/?q='+sql_statement, function(data) { $.each(data.rows, function(key, val) { @@ -51,16 +58,21 @@ $.getJSON('https://'+your_account_name+'.cartodb.com/api/v2/sql/?q='+sql_stateme By default, GET requests work from anywhere. In CartoDB, POST requests work from any website as well. We achieve this by hosting a cross-domain policy file at the root of all of our servers. This allows you the greatest level of flexibility when developing your application. + ## Response formats The standard response from the CartoDB SQL API is JSON. If you are building a web-application, the lightweight JSON format allows you to quickly integrate data from the SQL API. -
JSON
+### JSON + +#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 ``` -
RESULT
+#### Result + ```javascript { time: 0.006, @@ -80,12 +92,16 @@ https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 Alternatively, you can use the [GeoJSON specification](http://www.geojson.org/geojson-spec.html) to return data from the API. To do so, simply supply the `format` parameter as GeoJSON: -
GEOJSON
+### GeoJSON + +#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_name} LIMIT 1 ``` -
RESULT
+#### Result + ```javascript { type: "FeatureCollection", @@ -112,28 +128,35 @@ https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_n The SQL API accepts other output formats that can be useful to export data. Right now you can use the following formats: CSV, SHP, SVG, KML, SpatiaLite and GeoJSON. + ## Output filename + To customize the output filename, add the `filename` parameter to your URL: -
Customize filename
+#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?filename={custom_filename}&q=SELECT * FROM {table_name} LIMIT 1 ``` + ## Getting table information Currently, there is no public method to access your table schemas. The simplest way to retrieve table structure is to access the first row of the data, -
COLUMNS
+#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1 ``` + ## Response errors To help you debug your SQL queries, the CartoDB SQL API returns errors as part of the JSON response. Errors come back as follows, -
RESULT
+#### Result + ```javascript { error: [ @@ -144,18 +167,25 @@ To help you debug your SQL queries, the CartoDB SQL API returns errors as part o You can use these errors to help understand your SQL. For more complete documentation see the Error Codes and Solutions section of this Users Guide. + ## Write data to your CartoDB account Performing inserts or updates on your data is simple using your [API key](#authentication). All you need to do is supply a correct SQL [INSERT](http://www.postgresql.org/docs/9.1/static/sql-insert.html) or [UPDATE](http://www.postgresql.org/docs/9.1/static/sql-update.html) statement for your table along with the api_key parameter for your account. Be sure to keep these requests private, as anyone with your API key will be able to modify your tables. A correct SQL insert statement means that all the columns you want to insert into already exist in your table, and all the values for those columns are the right type (quoted string, unquoted string for geoms and dates, or numbers). -
COLUMNS
+### Insert + +#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?q=INSERT INTO test_table (column_name, column_name_2, the_geom) VALUES ('this is a string', 11, ST_SetSRID(ST_Point(-110, 43),4326))&api_key={Your API key} ``` Updates are just as simple. Here is an example, updating a row based on the value of the cartodb_id column. -
COLUMNS
+### Update + +#### Call + ```bash https://{account}.cartodb.com/api/v2/sql?q=UPDATE test_table SET column_name = 'my new string value' WHERE cartodb_id = 1 &api_key={Your API key} ``` From 6e3ea96c7e4a3cfe45028b4e8523f2ba6d579909 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20Matall=C3=ADn?= Date: Wed, 28 Oct 2015 14:44:26 +0100 Subject: [PATCH 06/10] docs: tips-and-tricks --- doc/query_optimizations.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/doc/query_optimizations.md b/doc/query_optimizations.md index fab699cb..6bac5a7f 100644 --- a/doc/query_optimizations.md +++ b/doc/query_optimizations.md @@ -3,11 +3,8 @@ There are some tricks to consider when using the SQL API that might make your application a little faster. * Only request the fields you need. Selecting all columns will return a full version of your geometry in *the_geom* as well as a reprojected version in *the_geom_webmercator*. - * Use PostGIS functions to simplify and filter out unneeded geometries when possible. One very handy function is, [ST_Simplify](http://www.postgis.org/docs/ST_Simplify.html). - * Remember to build indexes that will speed up some of your more common queries. - * Use *cartodb_id* to retrieve specific rows of your data, this is the unique key column added to every CartoDB table. From a7a8ddcf53b27726d98528c876cc4dcc19c62a2d Mon Sep 17 00:00:00 2001 From: Raul Ochoa Date: Fri, 22 Jan 2016 12:03:24 +0100 Subject: [PATCH 07/10] Makefile getting tests with find command --- Makefile | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 9ca74be3..3383fbc0 100644 --- a/Makefile +++ b/Makefile @@ -11,17 +11,21 @@ jshint: @echo "***jshint***" @./node_modules/.bin/jshint app/ test/ app.js +TEST_SUITE := $(shell find test/{acceptance,unit} -name "*.js") +TEST_SUITE_UNIT := $(shell find test/unit -name "*.js") +TEST_SUITE_ACCEPTANCE := $(shell find test/acceptance -name "*.js") + test: @echo "***tests***" - test/run_tests.sh ${RUNTESTFLAGS} test/unit/*.js test/unit/model/*.js test/acceptance/*.js test/acceptance/export/*.js + test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE) test-unit: @echo "***unit tests***" - test/run_tests.sh ${RUNTESTFLAGS} test/unit/*.js test/unit/model/*.js + test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_UNIT) test-acceptance: @echo "***acceptance tests***" - test/run_tests.sh ${RUNTESTFLAGS} test/acceptance/*.js test/acceptance/export/*.js + test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_ACCEPTANCE) test-all: jshint test From 99e7f1cdf248343c3b25920a3d077ce736a7b095 Mon Sep 17 00:00:00 2001 From: Raul Ochoa Date: Fri, 22 Jan 2016 12:13:01 +0100 Subject: [PATCH 08/10] Ignore npm debug log --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 55b7ef6f..5e734af8 100644 --- a/.gitignore +++ b/.gitignore @@ -9,3 +9,4 @@ tools/munin/cdbsqlapi.conf test/redis.pid test/test.log test/acceptance/oauth/venv/* +npm-debug.log From c096d843a26232628c9785a718c22e61fcae7715 Mon Sep 17 00:00:00 2001 From: Raul Ochoa Date: Fri, 22 Jan 2016 12:18:20 +0100 Subject: [PATCH 09/10] Adds coverage tool --- .gitignore | 1 + Makefile | 5 ++++- package.json | 1 + test/run_tests.sh | 15 +++++++++++++-- 4 files changed, 19 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index 5e734af8..40f9c0cf 100644 --- a/.gitignore +++ b/.gitignore @@ -9,4 +9,5 @@ tools/munin/cdbsqlapi.conf test/redis.pid test/test.log test/acceptance/oauth/venv/* +coverage/ npm-debug.log diff --git a/Makefile b/Makefile index 3383fbc0..8de488eb 100644 --- a/Makefile +++ b/Makefile @@ -29,4 +29,7 @@ test-acceptance: test-all: jshint test -.PHONY: test +coverage: + @RUNTESTFLAGS=--with-coverage make test + +.PHONY: test coverage diff --git a/package.json b/package.json index 8bb21efe..b803968e 100644 --- a/package.json +++ b/package.json @@ -31,6 +31,7 @@ }, "devDependencies": { "redis": "0.7.1", + "istanbul": "~0.4.2", "request": "~2.60.0", "shapefile": "0.3.0", "mocha": "~1.21.4", diff --git a/test/run_tests.sh b/test/run_tests.sh index a1a4b14a..c00c3c2a 100755 --- a/test/run_tests.sh +++ b/test/run_tests.sh @@ -16,6 +16,7 @@ OPT_CREATE_PGSQL=yes # create/prepare the postgresql test database OPT_CREATE_REDIS=yes # create/prepare the redis test databases OPT_DROP_PGSQL=yes # drop the postgreql test environment OPT_DROP_REDIS=yes # drop the redis test environment +OPT_COVERAGE=no # run tests with coverage cd $(dirname $0) BASEDIR=$(pwd) @@ -80,6 +81,10 @@ while [ -n "$1" ]; do OPT_CREATE_REDIS=no shift continue + elif test "$1" = "--with-coverage"; then + OPT_COVERAGE=yes + shift + continue else break fi @@ -94,6 +99,7 @@ if [ -z "$1" ]; then echo " --nodrop do not drop the test environment on exit" >&2 echo " --nodrop-pg do not drop the pgsql test environment" >&2 echo " --nodrop-redis do not drop the redis test environment" >&2 + echo " --with-coverage use istanbul to determine code coverage" >&2 exit 1 fi @@ -127,8 +133,13 @@ echo echo " ogr2ogr version: "`ogr2ogr --version` echo -echo "Running tests" -mocha -t 5000 -u tdd ${TESTS} +if test x"$OPT_COVERAGE" = xyes; then + echo "Running tests with coverage" + ./node_modules/.bin/istanbul cover node_modules/.bin/_mocha -- -u tdd -t 5000 ${TESTS} +else + echo "Running tests" + mocha -u tdd -t 5000 ${TESTS} +fi ret=$? cleanup || exit 1 From 9ef8179cb3031953a414464838301aa43d2434a8 Mon Sep 17 00:00:00 2001 From: Raul Ochoa Date: Fri, 22 Jan 2016 12:22:54 +0100 Subject: [PATCH 10/10] Define shell as bash --- Makefile | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 3383fbc0..e6de9c1e 100644 --- a/Makefile +++ b/Makefile @@ -1,3 +1,5 @@ +SHELL=/bin/bash + all: npm install @@ -17,15 +19,15 @@ TEST_SUITE_ACCEPTANCE := $(shell find test/acceptance -name "*.js") test: @echo "***tests***" - test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE) + @$(SHELL) test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE) test-unit: @echo "***unit tests***" - test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_UNIT) + @$(SHELL) test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_UNIT) test-acceptance: @echo "***acceptance tests***" - test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_ACCEPTANCE) + @$(SHELL) test/run_tests.sh ${RUNTESTFLAGS} $(TEST_SUITE_ACCEPTANCE) test-all: jshint test