Merge pull request #275 from CartoDB/docs-661-add-batch-api-to-sql-doc
Docs 661 add batch api to sql doc (Do not merge yet)
This commit is contained in:
Daniel
commit
7508d19ab5
@ -1,6 +1,6 @@
|
||||
# SQL API
|
||||
|
||||
CartoDB's SQL API allows you to interact with your tables and data inside CartoDB as if you were running SQL statements against a normal database. The database behind CartoDB is PostgreSQL so if you need help with specific SQL statements or you want to learn more about it, visit the [official documentation](http://www.postgresql.org/docs/9.1/static/sql.html).
|
||||
CartoDB's SQL API allows you to interact with your tables and data inside CartoDB, as if you were running SQL statements against a normal database. The database behind CartoDB is PostgreSQL so if you need help with specific SQL statements or you want to learn more about it, visit the [official documentation](http://www.postgresql.org/docs/9.1/static/sql.html).
|
||||
|
||||
There are two main situations in which you would want to use the SQL API:
|
||||
|
||||
@ -16,6 +16,7 @@ Remember that in order to access, read or modify data in private tables, you wil
|
||||
* [Making calls to the SQL API](making_calls.md)
|
||||
* [Handling geospatial data](handling_geospatial_data.md)
|
||||
* [Query optimizations](query_optimizations.md)
|
||||
* [SQL Batch API](sql_batch_api.md)
|
||||
* [API version number](version.md)
|
||||
* [Libraries in different languages](libraries_support.md)
|
||||
* [Other Tips and Questions](tips_and_tricks.md)
|
||||
|
||||
@ -1,21 +1,17 @@
|
||||
# Authentication
|
||||
|
||||
For all access to private tables and for write access to public tables, CartoDB enforces secure API access that requires you to authorize your queries. In order to authorize queries, you can use an API key or a Consumer Key.
|
||||
For all access to private tables, and for write access to public tables, CartoDB enforces secure API access that requires you to authorize your queries. In order to authorize queries, you can use an API Key or a Consumer Key.
|
||||
|
||||
## API Key
|
||||
|
||||
The API key offers the simplest way to access private data or perform writes and updates to your public data. Remember that your API key protects access to your data, so keep it confidential and only share it if you want others to have this access. If necessary, you can reset your API key in your admin dashboard.
|
||||
The API Key offers the simplest way to access private data, or perform writes and updates to your public data. Remember that your API Key protects access to your data, so keep it confidential and only share it if you want others to have this access. If necessary, you can reset your API Key from your CartoDB dashboard.
|
||||
|
||||
To find your API key:
|
||||
**Tip:** For details about how to access, or reset, your API Key, see [Your Account](http://docs.cartodb.com/cartodb-editor/your-account/#api-key) details.
|
||||
|
||||
- Go to your dashboard.
|
||||
- Click on your username in the top right corner, and select "Your API keys."
|
||||
- Here, you can copy your API key, see use examples, and reset your API key.
|
||||
|
||||
To use your API key, pass it as a parameter in an URL call to the CartoDB API. For example, to perform an insert into your table, you would use the following URL structure.
|
||||
To use your API Key, pass it as a parameter in an URL call to the CartoDB API. For example, to perform an insert into your table, you would use the following URL structure.
|
||||
|
||||
#### Example
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q={SQL statement}&api_key={Your API key}
|
||||
https://{username}.cartodb.com/api/v2/sql?q={SQL statement}&api_key={api_key}
|
||||
```
|
||||
|
||||
@ -1,15 +1,15 @@
|
||||
# Handling geospatial data
|
||||
# Handling Geospatial Data
|
||||
|
||||
Handling geospatial data through the SQL API is easy! By default, *the_geom* is returned straight from the database, in a format called Well-Known Binary. There are a handful of ways you can transform your geometries into more useful formats.
|
||||
Handling geospatial data through the SQL API is easy. By default, *the_geom* is returned straight from the database, in a format called Well-Known Binary. There are a handful of ways you can transform your geometries into more useful formats.
|
||||
|
||||
The first, is to use the format=GeoJSON method described above. Others can be handled through your SQL statements directly. For example, enclosing your the_geom in a function called [ST_AsGeoJSON](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) will allow you to use JSON for your data but a GeoJSON string for your geometry column only. Alternatively, using a the [ST_AsText](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) function will return your geometry as Well-Known Text.
|
||||
The first is to use the format=GeoJSON method described above. Others can be handled through your SQL statements directly. For example, enclosing your the_geom in a function called [ST_AsGeoJSON](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) will allow you to use JSON for your data but a GeoJSON string for your geometry column only. Alternatively, using a the [ST_AsText](http://www.postgis.org/documentation/manual-svn/ST_AsGeoJSON.html) function will return your geometry as Well-Known Text.
|
||||
|
||||
### ST_AsGeoJSON
|
||||
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_geom) as the_geom FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_geom) as the_geom FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
#### Result
|
||||
@ -32,7 +32,7 @@ https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsGeoJSON(the_ge
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom) FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom) FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
#### Result
|
||||
@ -50,14 +50,14 @@ https://{account}.cartodb.com/api/v2/sql?q=SELECT cartodb_id,ST_AsText(the_geom)
|
||||
}
|
||||
```
|
||||
|
||||
More advanced methods exist in the PostGIS library to extract meaningful data from your geometry. Explore the PostGIS documentation and get familiar with functions such as, [ST_XMin](http://www.postgis.org/docs/ST_XMin.html), [ST_XMax](http://www.postgis.org/docs/ST_XMax.html), [ST_AsText](http://www.postgis.org/docs/ST_AsText.html), and more.
|
||||
More advanced methods exist in the PostGIS library to extract meaningful data from your geometry. Explore the PostGIS documentation and get familiar with functions such as, [ST_XMin](http://www.postgis.org/docs/ST_XMin.html), [ST_XMax](http://www.postgis.org/docs/ST_XMax.html), [ST_AsText](http://www.postgis.org/docs/ST_AsText.html), and so on.
|
||||
|
||||
All data returned from *the_geom* column is in WGS 84 (EPSG:4326). You can change this quickly and easily on the fly using SQL. For example, if you desire geometries in the Hanoi 1972 (EPSG:4147) projection, you could [ST_Transform](http://www.postgis.org/docs/ST_Transform.html),
|
||||
All data returned from *the_geom* column is in WGS 84 (EPSG:4326). You can change this quickly on the fly, by using SQL. For example, if you prefer geometries using the Hanoi 1972 (EPSG:4147) projection, use [ST_Transform](http://www.postgis.org/docs/ST_Transform.html),
|
||||
|
||||
### ST_Transform
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT ST_Transform(the_geom,4147) FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT ST_Transform(the_geom,4147) FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
CartoDB also stores a second geometry column, *the_geom_webmercator*. We use this internally to build your map tiles as fast as we can. In the user-interface it is hidden, but it is visible and available for use. In this column we store a reprojected version of all your geometries using Web Mercator (EPSG:3857).
|
||||
CartoDB also stores a second geometry column, *the_geom_webmercator*. We use this internally to build your map tiles as fast as we can. In the user-interface it is hidden, but it is visible and available for use. In this column, we store a reprojected version of all your geometries using Web Mercator (EPSG:3857).
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Libraries in different languages
|
||||
# Libraries in Different Languages
|
||||
|
||||
To make things easier for developers, we provide client libraries for different programming languages and caching functionalities.
|
||||
|
||||
@ -18,7 +18,7 @@ To make things easier for developers, we provide client libraries for different
|
||||
Very basic example of how to access CartoDB SQL API. [Fork it on GitHub!](https://github.com/cartodb/cartodb-java-client)
|
||||
|
||||
- **NET**
|
||||
.NET library for authenticating with CartoDB using an API key, based on work started by [The Data Republic](http://www.thedatarepublic.com/). [Fork it on GitHub!](https://github.com/thedatarepublic/CartoDBClientDotNET)
|
||||
.NET library for authenticating with CartoDB using an API Key, based on work started by [The Data Republic](http://www.thedatarepublic.com/). [Fork it on GitHub!](https://github.com/thedatarepublic/CartoDBClientDotNET)
|
||||
|
||||
- **Clojure**
|
||||
Clojure library for authenticating with CartoDB, maintained by [REDD Metrics](http://www.reddmetrics.com/). [Fork it on GitHub!](https://github.com/reddmetrics/cartodb-clj)
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Making calls to the SQL API
|
||||
# Making Calls to the SQL API
|
||||
|
||||
CartoDB is based on the rock solid PostgreSQL database. All of your tables reside a single database, which means you can perform complex queries joining tables or carrying out geospatial operations. The best place to learn about PostgreSQL's SQL language is the [official documentation](http://www.postgresql.org/docs/9.1/static/).
|
||||
CartoDB is based on the rock solid PostgreSQL database. All of your tables reside a single database, which means you can perform complex queries joining tables, or carrying out geospatial operations. The best place to learn about PostgreSQL's SQL language is the [official documentation](http://www.postgresql.org/docs/9.1/static/).
|
||||
|
||||
CartoDB is also based on PostGIS, so take a look at the [official PostGIS reference](http://postgis.refractions.net/docs/) to know what functionality we support in terms of geospatial operations. All of our tables include a column called *the_geom,* which is a geometry field that indexes geometries in the EPSG:4326 (WGS 1984) coordinate system. All tables also have an automatically generated and updated column called *the_geom_webmercator*. We use the column internally to quickly create tiles for maps.
|
||||
|
||||
@ -12,7 +12,7 @@ All SQL API requests to your CartoDB account should follow this general pattern:
|
||||
#### SQL query example
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q={SQL statement}
|
||||
https://{username}.cartodb.com/api/v2/sql?q={SQL statement}
|
||||
```
|
||||
|
||||
If you encounter errors, double-check that you are using the correct account name, and that your SQL statement is valid. A simple example of this pattern is conducting a count of all the records in your table:
|
||||
@ -20,7 +20,7 @@ If you encounter errors, double-check that you are using the correct account nam
|
||||
#### Count example
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name}
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name}
|
||||
```
|
||||
|
||||
#### Result
|
||||
@ -37,7 +37,7 @@ https://{account}.cartodb.com/api/v2/sql?q=SELECT count(*) FROM {table_name}
|
||||
}
|
||||
```
|
||||
|
||||
Finally, remember that in order to use the SQL API, either your table must be public, or you must be authenticated using API Keys, as discussed above.
|
||||
Finally, remember that in order to use the SQL API, either your table must be public, or you must be [authenticated](http://docs.cartodb.com/cartodb-platform/sql-api/authentication/#authentication) using API Keys.
|
||||
|
||||
|
||||
## POST and GET
|
||||
@ -49,7 +49,7 @@ The CartoDB SQL API is setup to handle both GET and POST requests. You can test
|
||||
#### Call
|
||||
|
||||
```javascript
|
||||
$.getJSON('https://'+your_account_name+'.cartodb.com/api/v2/sql/?q='+sql_statement, function(data) {
|
||||
$.getJSON('https://{username}.cartodb.com/api/v2/sql/?q='+sql_statement, function(data) {
|
||||
$.each(data.rows, function(key, val) {
|
||||
// do something!
|
||||
});
|
||||
@ -68,7 +68,7 @@ The standard response from the CartoDB SQL API is JSON. If you are building a we
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
#### Result
|
||||
@ -97,7 +97,7 @@ Alternatively, you can use the [GeoJSON specification](http://www.geojson.org/ge
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
#### Result
|
||||
@ -126,7 +126,7 @@ https://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table_n
|
||||
}
|
||||
```
|
||||
|
||||
The SQL API accepts other output formats that can be useful to export data. Right now you can use the following formats: CSV, SHP, SVG, KML, SpatiaLite and GeoJSON.
|
||||
The SQL API accepts other output formats that can be useful to export data. You can use the following formats: CSV, SHP, SVG, KML, SpatiaLite and GeoJSON.
|
||||
|
||||
|
||||
## Output filename
|
||||
@ -136,7 +136,7 @@ To customize the output filename, add the `filename` parameter to your URL:
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?filename={custom_filename}&q=SELECT * FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?filename={custom_filename}&q=SELECT * FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
|
||||
@ -147,7 +147,7 @@ Currently, there is no public method to access your table schemas. The simplest
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1
|
||||
https://{username}.cartodb.com/api/v2/sql?q=SELECT * FROM {table_name} LIMIT 1
|
||||
```
|
||||
|
||||
|
||||
@ -167,25 +167,24 @@ To help you debug your SQL queries, the CartoDB SQL API returns the full error p
|
||||
|
||||
You can use these errors to help understand your SQL. If you encounter errors executing SQL, either through the CartoDB Editor, or through the SQL API, it is suggested to Google search the error for independent troubleshooting.
|
||||
|
||||
|
||||
## Write data to your CartoDB account
|
||||
|
||||
Performing inserts or updates on your data is simple using your [API key](#authentication). All you need to do is supply a correct SQL [INSERT](http://www.postgresql.org/docs/9.1/static/sql-insert.html) or [UPDATE](http://www.postgresql.org/docs/9.1/static/sql-update.html) statement for your table along with the api_key parameter for your account. Be sure to keep these requests private, as anyone with your API key will be able to modify your tables. A correct SQL insert statement means that all the columns you want to insert into already exist in your table, and all the values for those columns are the right type (quoted string, unquoted string for geoms and dates, or numbers).
|
||||
Performing inserts or updates on your data is simple using your [API Key](#authentication). All you need to do is supply a correct SQL [INSERT](http://www.postgresql.org/docs/9.1/static/sql-insert.html) or [UPDATE](http://www.postgresql.org/docs/9.1/static/sql-update.html) statement for your table along with the api_key parameter for your account. Be sure to keep these requests private, as anyone with your API Key will be able to modify your tables. A correct SQL insert statement means that all the columns you want to insert into already exist in your table, and all the values for those columns are the right type (quoted string, unquoted string for geoms and dates, or numbers).
|
||||
|
||||
### Insert
|
||||
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=INSERT INTO test_table (column_name, column_name_2, the_geom) VALUES ('this is a string', 11, ST_SetSRID(ST_Point(-110, 43),4326))&api_key={Your API key}
|
||||
https://{username}.cartodb.com/api/v2/sql?q=INSERT INTO test_table (column_name, column_name_2, the_geom) VALUES ('this is a string', 11, ST_SetSRID(ST_Point(-110, 43),4326))&api_key={api_key}
|
||||
```
|
||||
|
||||
Updates are just as simple. Here is an example, updating a row based on the value of the cartodb_id column.
|
||||
Updates are just as simple. Here is an example of updating a row based on the value of the cartodb_id column.
|
||||
|
||||
### Update
|
||||
|
||||
#### Call
|
||||
|
||||
```bash
|
||||
https://{account}.cartodb.com/api/v2/sql?q=UPDATE test_table SET column_name = 'my new string value' WHERE cartodb_id = 1 &api_key={Your API key}
|
||||
https://{username}.cartodb.com/api/v2/sql?q=UPDATE test_table SET column_name = 'my new string value' WHERE cartodb_id = 1 &api_key={api_key}
|
||||
```
|
||||
|
||||
@ -1,8 +1,8 @@
|
||||
# Query optimizations
|
||||
# Query Optimizations
|
||||
|
||||
There are some tricks to consider when using the SQL API that might make your application a little faster.
|
||||
|
||||
* Only request the fields you need. Selecting all columns will return a full version of your geometry in *the_geom* as well as a reprojected version in *the_geom_webmercator*.
|
||||
* Only request the fields you need. Selecting all columns will return a full version of your geometry in *the_geom*, as well as a reprojected version in *the_geom_webmercator*.
|
||||
* Use PostGIS functions to simplify and filter out unneeded geometries when possible. One very handy function is, [ST_Simplify](http://www.postgis.org/docs/ST_Simplify.html).
|
||||
* Remember to build indexes that will speed up some of your more common queries. For details, see [Creating Indexes](http://docs.cartodb.com/cartodb-editor/managing-your-data/#creating-indexes)
|
||||
* Use *cartodb_id* to retrieve specific rows of your data, this is the unique key column added to every CartoDB table.
|
||||
|
||||
235
doc/sql_batch_api.md
Normal file
235
doc/sql_batch_api.md
Normal file
@ -0,0 +1,235 @@
|
||||
# SQL Batch API
|
||||
|
||||
The SQL Batch API enables you to request queries with long-running processing times. Typically, these kind of requests raise timeout errors when using the SQL API. In order to avoid timeouts, you can use the SQL Batch API to [create](#create-a-job), [read](#read-a-job), [list](#list-jobs), [update](#update-a-job) and [cancel](#cancel-a-job) queries. You can also run [multiple](#multi-query-batch-jobs) SQL queries in one job. The SQL Batch API schedules the incoming jobs and allows you to request the job status for each query.
|
||||
|
||||
**Note:** In order to use the SQL Batch API, your table must be public, or you must be [authenticated](http://docs.cartodb.com/cartodb-platform/sql-api/authentication/#authentication) using API keys.
|
||||
|
||||
## SQL Batch API Job Schema
|
||||
|
||||
The SQL Batch API request to your CartoDB account includes the following job schema elements. _Only the `query` element can be modified._ All other elements of the job schema are defined by the SQL Batch API and are read-only.
|
||||
|
||||
Name | Description
|
||||
--- | ---
|
||||
`job_id` | a universally unique identifier (uuid).
|
||||
`user` | user identifier, as displayed by the username.
|
||||
`status` | displays the result of the long-running query. The possible status results are:
|
||||
--- | ---
|
||||
|_ `pending` | job waiting to be executed.
|
||||
|_ `running` | indicates that the job is currently running.
|
||||
|_ `done` | job executed successfully.
|
||||
|_ `failed` | job executed but failed, with errors.
|
||||
|_ `canceled` | job canceled by user request.
|
||||
|_ `unknown` | appears when it is not possible to determine what exactly happened with the job.
|
||||
`query` | the SQL statement to be executed in a database. _You can modify the select SQL statement to be used in the job schema._<br/><br/>**Tip:** In some scenarios, you may need to retrieve the query results from a finished job. If that is the case, wrap the query with SELECT * INTO, or CREATE TABLE AS. The results will be stored in a new table in your user database. For example:<br/><br/>1. A job query, `SELECT * FROM user_dataset;`<br/><br/>2. Wrap the query, `SELECT * INTO job_result FROM (SELECT * FROM user_dataset) AS job;`<br/><br/>3. Once the table is created, retrieve the results through the CartoDB SQL API, `SELECT * FROM job_result;`
|
||||
`created_at` | the date and time when the job schema was created.
|
||||
`updated_at` | the date and time of when the job schema was last updated, or modified.
|
||||
`failed_reason` | displays the database error message, if something went wrong.
|
||||
|
||||
#### Example
|
||||
|
||||
```bash
|
||||
HEADERS: 201 CREATED; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT * FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-15T07:36:25Z”
|
||||
}
|
||||
```
|
||||
|
||||
## Create a Job
|
||||
|
||||
To create an SQL Batch API job, make a POST request with the following parameters.
|
||||
|
||||
Creates an SQL Batch API job request.
|
||||
|
||||
```bash
|
||||
HEADERS: POST /api/v2/sql/job
|
||||
BODY: {
|
||||
query: ‘SELECT * FROM user_dataset’
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 201 CREATED; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT * FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-15T07:36:25Z”
|
||||
}
|
||||
```
|
||||
|
||||
## Read a Job
|
||||
|
||||
To read an SQL Batch API job, make a GET request with the following parameters.
|
||||
|
||||
```bash
|
||||
HEADERS: GET /api/v2/sql/job/de305d54-75b4-431b-adb2-eb6b9e546014
|
||||
BODY: {}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 200 OK; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT * FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-15T07:36:25Z”
|
||||
}
|
||||
```
|
||||
|
||||
## List Jobs
|
||||
|
||||
To list SQL Batch API jobs, make a GET request with the following parameters.
|
||||
|
||||
```bash
|
||||
HEADERS: GET /api/v2/sql/job
|
||||
BODY: {}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 200 OK; application/json
|
||||
BODY: [{
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT * FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-15T07:36:25Z”
|
||||
}, {
|
||||
“job_id”: “ba25ed54-75b4-431b-af27-eb6b9e5428ff”,
|
||||
“user”: “cartofante”
|
||||
“query”: “DELETE FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:43:12Z”,
|
||||
“updated_at”: “2015-12-15T07:43:12Z”
|
||||
}]
|
||||
```
|
||||
|
||||
## Update a Job
|
||||
|
||||
To update an SQL Batch API job, make a PUT request with the following parameters.
|
||||
|
||||
```bash
|
||||
HEADERS: PUT /api/v2/sql/job/de305d54-75b4-431b-adb2-eb6b9e546014
|
||||
BODY: {
|
||||
"query": “SELECT cartodb_id FROM user_dataset”
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 200 OK; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT cartodb_id FROM user_dataset”,
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-17T15:45:56Z”
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** Jobs can only be updated while the `status: "pending"`, otherwise the SQL Batch API Update operation is not allowed. You will receive an error if the job status is anything but "pending".
|
||||
|
||||
```bash
|
||||
errors: [
|
||||
“The job status is not pending, it cannot be updated”
|
||||
]
|
||||
```
|
||||
|
||||
## Cancel a Job
|
||||
|
||||
To cancel an SQL Batch API job, make a DELETE request with the following parameters.
|
||||
|
||||
```bash
|
||||
HEADERS: DELETE /api/v2/sql/job/de305d54-75b4-431b-adb2-eb6b9e546014
|
||||
BODY: {}
|
||||
```
|
||||
|
||||
**Note:** Be mindful when cancelling a job when the status: `pending` or `running`.
|
||||
|
||||
- If the job is `pending`, the job will never be executed
|
||||
- If the job is `running`, the job will be terminated immediately
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 200 OK; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: “SELECT * FROM user_dataset”,
|
||||
“status”: “cancelled”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-17T06:22:42Z”
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** Jobs can only be cancelled while the `status: "running"` or `status: "pending"`, otherwise the SQL Batch API Cancel operation is not allowed. You will receive an error if the job status is anything but "running" or "pending".
|
||||
|
||||
```bash
|
||||
errors: [
|
||||
“The job status is done, cancel is not allowed”
|
||||
]
|
||||
```
|
||||
|
||||
## Multi Query Batch Jobs
|
||||
|
||||
In some cases, you may need to run multiple SQL queries in one job. The Multi Query batch option enables you run an array of SQL statements, and define the order in which the queries are executed. You can use any of the operations (create, read, list, update, cancel) for the queries in a Multi Query batch job.
|
||||
|
||||
```bash
|
||||
HEADERS: POST /api/v2/sql/job
|
||||
BODY: {
|
||||
query: [
|
||||
‘SELECT * FROM user_dataset_0’,
|
||||
‘SELECT * FROM user_dataset_1’,
|
||||
‘SELECT * FROM user_dataset_2’
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```bash
|
||||
HEADERS: 201 CREATED; application/json
|
||||
BODY: {
|
||||
“job_id”: “de305d54-75b4-431b-adb2-eb6b9e546014”,
|
||||
“user”: “cartofante”
|
||||
“query”: [{
|
||||
“query”: “SELECT * FROM user_dataset_0”,
|
||||
“status”: “pending”
|
||||
}, {
|
||||
“query”: “SELECT * FROM user_dataset_1”,
|
||||
“status”: “pending”
|
||||
}, {
|
||||
“query”: “SELECT * FROM user_dataset_2”,
|
||||
“status”: “pending”
|
||||
}],
|
||||
“status”: “pending”,
|
||||
“created_at”: “2015-12-15T07:36:25Z”,
|
||||
“updated_at”: “2015-12-15T07:36:25Z”
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** The SQL Batch API returns a job status for both the parent Multi Query request, and for each child query within the request. The order in which each query is executed is guaranteed. Here are the possible status results for Multi Query batch jobs:
|
||||
|
||||
- If one query within the Multi Query batch fails, the `"status": "failed"` is returned for both the job and the query, and any "pending" queries will not be processed
|
||||
|
||||
- If you cancel the Multi Query batch job, the job status changes to `"status": "cancelled"`. Any running queries within the job will be stopped and changed to `"status": "pending"`, and will not be processed
|
||||
|
||||
- Suppose the first query job status is `"status": "done"`, the second query is `"status": "running"`, and the third query `"status": "pending"`. If the second query fails for some reason, the job status changes to `"status": "failed"` and the last query will not be processed. It is indicated which query failed in the Multi Query batch job
|
||||
@ -2,13 +2,13 @@
|
||||
|
||||
## What does CartoDB do to prevent SQL injection?
|
||||
|
||||
CartoDB uses the database access mechanism for security. Every writable connection is verified by an API key. If you have the correct API key, you can write-access to the database. If you do not have the correct API key, your client is "logged in" as a low privilege user, and you have read-only access to the database (if the database allows you to read).
|
||||
CartoDB uses the database access mechanism for security. Every writable connection is verified by an API Key. If you have the correct API Key, you can write-access to the database. If you do not have the correct API Key, your client is "logged in" as a low privilege user, and you have read-only access to the database (if the database allows you to read).
|
||||
|
||||
SQL injection works by tricking a database user, so that running a query retrieves database wide results, even though the database is protected.
|
||||
|
||||
Because CartoDB enforces roles and access at the database level, the idea of a "SQL injection attack" is not possible with CartoDB. Injection is possible, but clients will still run into our security wall at the database level. The SQL API already lets you _attempt_ to run any query you want. The database will reject your SQL API request if it finds your user/role does not have the requisite permissions. In other words, you can ask any question of the database you like; the CartoDB database does not guarantee it will be answered.
|
||||
|
||||
If a user's API key found its way out into the wild, that could be a problem, but it is not something CartoDB can prevent. _This is why it is very important for all CartoDB users to secure their API keys_. In the event a user's API key is compromised, the user (or the CartoDB Enterprise administrator), can regenerate the API key in their account settings.
|
||||
If a user's API Key found its way out into the wild, that could be a problem, but it is not something CartoDB can prevent. _This is why it is very important for all CartoDB users to secure their API Keys_. In the event a user's API Key is compromised, the user (or the CartoDB Enterprise administrator), can regenerate the API Key in their account settings.
|
||||
|
||||
**Note:** While the SQL API is SQL injection secure, if you build additional layers to allow another person to run queries (i.e., building a proxy so that others can indirectly perform authenticated queries through the SQL API), the security of those newly added layers are the responsibility of the creator.
|
||||
|
||||
@ -17,7 +17,7 @@ If a user's API key found its way out into the wild, that could be a problem, bu
|
||||
There are three levels of access with CartoDB:
|
||||
|
||||
1. __API Key level:__ Do whatever you want in your account on the tables you own (or have been shared with you in Enterprise/multi-user accounts).
|
||||
2. __"publicuser" level:__ Do whatever has been granted to you. The publicuser level is normally read-only, but you could GRANT INSERT/UPDATE/DELETE permissions to publicuser if needed for some reason - for API key-less write operations. Use with caution.
|
||||
2. __"publicuser" level:__ Do whatever has been granted to you. The publicuser level is normally read-only, but you could GRANT INSERT/UPDATE/DELETE permissions to publicuser if needed for some reason - for API Key-less write operations. Use with caution.
|
||||
3. __postgres superadmin level:__ This third access level, the actual PostgreSQL system user, is only accessible from a direct database connection via the command line, which is only available currently via [CartoDB On-Premises](https://cartodb.com/on-premises/).
|
||||
|
||||
## If a user has write access and makes a `DROP TABLE` query, is that data gone?
|
||||
@ -30,10 +30,10 @@ Yes. Create the table, and GRANT INSERT/UPDATE to the user.
|
||||
|
||||
## Is there an actual PostgreSQL account for each CartoDB login/username?
|
||||
|
||||
Yes, there is. Unfortunately, the names are different - though there is a way to determine the name of the PostgreSQL user account. Every CartoDB user gets their own PostgreSQL database. But there’s a system database too, with the name mappings in `username` and `database_name` columns. `database_name` is the name of the database that user belongs to. It will be `cartodb_user_ID`. `id` holds long hashkey. The `database_name` is derived from this ID hash too, but in case of an Enterprise/multi-user account it will come from the user ID of the owner of the organization - and `database_name` will hold the same value for every user in an Enterprise/multi-user account.
|
||||
Yes, there is. Unfortunately, the names are different - though there is a way to determine the name of the PostgreSQL user account. Every CartoDB user gets their own PostgreSQL database. But there is a system database too, with the name mappings in `username` and `database_name` columns. `database_name` is the name of the database that user belongs to. It will be `cartodb_user_ID`. `id` holds long hashkey. The `database_name` is derived from this ID hash too, but in case of an Enterprise/multi-user account it will come from the user ID of the owner of the organization - and `database_name` will hold the same value for every user in an Enterprise/multi-user account.
|
||||
|
||||
You can also just do `select user` using the SQL API (without an API key to get the publicuser name and with an API key to get the CartoDB user's PostgreSQL user name), to determine the name of the corresponding PostgreSQL user.
|
||||
You can also just do `select user` using the SQL API (without an API Key to get the publicuser name and with an API Key to get the CartoDB user's PostgreSQL user name), to determine the name of the corresponding PostgreSQL user.
|
||||
|
||||
## Could I configure my CartoDB database permissions exactly the same way I could on my own PostgreSQL instance?
|
||||
## Can I configure my CartoDB database permissions exactly the same way I do on my own PostgreSQL instance?
|
||||
|
||||
Yes, through using GRANT statements to the SQL API. There are a few caveats to be aware of, including the aforementioned naming differences. Also, you'll be limited to permissions a user has with their own tables. Users don’t have PostgreSQL superuser privileges. So they can’t be creating languages, or C functions, or anything that requires superuser or CREATEUSER privileges.
|
||||
Yes, through using GRANT statements to the SQL API. There are a few caveats to be aware of, including the aforementioned naming differences. Also, you will be limited to permissions a user has with their own tables. Users do not have PostgreSQL superuser privileges. So they cannot be creating languages, or C functions, or anything that requires superuser or CREATEUSER privileges.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# API version number
|
||||
# API Version Number
|
||||
|
||||
All CartoDB applications use **Version 2** of our APIs. All other APIs are deprecated and will not be maintained or supported. You can check that you are using **Version 2** of our APIs by looking at your request URLS. They should all begin contain **/v2/** in the URLs as follows `https://{account}.cartodb.com/api/v2/`
|
||||
All CartoDB applications use **Version 2** of our APIs. All other APIs are deprecated and will not be maintained or supported. You can check that you are using **Version 2** of our APIs by looking at your request URLS. They should all begin containing **/v2/** in the URLs as follows, `https://{username}.cartodb.com/api/v2/`
|
||||
|
||||
Loading…
Reference in New Issue
Block a user