commit
a9dbde7877
@ -4,8 +4,8 @@ Copy queries allow you to use the [PostgreSQL copy command](https://www.postgres
|
||||
|
||||
The support for copy is split across two API end points:
|
||||
|
||||
* `http://{username}.carto.com/api/v2/sql/copyfrom` for uploading data to CARTO
|
||||
* `http://{username}.carto.com/api/v2/sql/copyto` for exporting data out of CARTO
|
||||
* `https://{username}.carto.com/api/v2/sql/copyfrom` for uploading data to CARTO
|
||||
* `https://{username}.carto.com/api/v2/sql/copyto` for exporting data out of CARTO
|
||||
|
||||
## Copy From
|
||||
|
||||
@ -29,7 +29,9 @@ Composing a chunked POST is moderately complicated, so most developers will use
|
||||
|
||||
### Example
|
||||
|
||||
For a table to be readable by CARTO, it must have a minimum of three columns with specific data types:
|
||||
#### Table preparation
|
||||
|
||||
First of all, you'll need to have **a table with the right schema** to copy your data into. For a table to be readable by CARTO, it must have a minimum of three columns with specific data types:
|
||||
|
||||
* `cartodb_id`, a `serial` primary key
|
||||
* `the_geom`, a `geometry` in the ESPG:4326 projection (aka long/lat)
|
||||
@ -37,8 +39,8 @@ For a table to be readable by CARTO, it must have a minimum of three columns wit
|
||||
|
||||
Creating a new CARTO table with all the right triggers and columns can be tricky, so here is an example:
|
||||
|
||||
-- create the table using the *required* columns and a
|
||||
-- couple more
|
||||
```sql
|
||||
-- Create the table using the *required* columns and a couple more
|
||||
CREATE TABLE upload_example (
|
||||
the_geom geometry,
|
||||
name text,
|
||||
@ -47,10 +49,19 @@ Creating a new CARTO table with all the right triggers and columns can be tricky
|
||||
|
||||
-- adds the 'cartodb_id' and 'the_geom_webmercator'
|
||||
-- adds the required triggers and indexes
|
||||
SELECT CDB_CartodbfyTable('upload_example');
|
||||
SELECT CDB_CartodbfyTable(current_schema, 'upload_example');
|
||||
```
|
||||
|
||||
you can get these queries executed directly through the SQL API this way, just by replacing the `{username}` and `{api_key}` by yours:
|
||||
|
||||
```sh
|
||||
# Table creation
|
||||
curl -v "https://{user}.carto.com/api/v2/sql?api_key={api_key}&q=CREATE+TABLE+upload_example+(the_geom+geometry,name+text,age+integer)"
|
||||
|
||||
# Cartodbfy table (optional, but allows for table registartion in Builder dashboard)
|
||||
curl -v "http://{user}.carto.com/api/v2/sql?api_key={api_key}&q=SELECT+CDB_CartodbfyTable(current_schema,'upload_example')"
|
||||
```
|
||||
|
||||
-- Note that CDB_CartodbfyTable is called differently if you have an organization user
|
||||
-- SELECT CDB_CartodbfyTable('your_org_username', 'upload_example');
|
||||
|
||||
Now you are ready to upload your file. Suppose you have a CSV file like this:
|
||||
|
||||
@ -68,7 +79,7 @@ The `FROM STDIN` option tells the database that the input will come from a data
|
||||
|
||||
To actually run upload, you will need a tool or script that can generate a chunked POST, so here are a few examples in different languages.
|
||||
|
||||
### CURL Example
|
||||
#### CURL Example
|
||||
|
||||
The [curl](https://curl.haxx.se/) utility makes it easy to run web requests from the command-line, and supports chunked POST upload, so it can feed the `copyfrom` end point.
|
||||
|
||||
@ -78,7 +89,7 @@ Assuming that you have already created the table, and that the CSV file is named
|
||||
-H "Transfer-Encoding: chunked" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
--data-binary @upload_example.csv \
|
||||
"http://{username}.carto.com/api/v2/sql/copyfrom?api_key={api_key}&q=COPY+upload_example+(the_geom,+name,+age)+FROM+STDIN+WITH+(FORMAT+csv,+HEADER+true)"
|
||||
"https://{username}.carto.com/api/v2/sql/copyfrom?api_key={api_key}&q=COPY+upload_example+(the_geom,+name,+age)+FROM+STDIN+WITH+(FORMAT+csv,+HEADER+true)"
|
||||
|
||||
To upload a larger file, using compression for a faster transfer, first compress the file, and then upload it with the content encoding set:
|
||||
|
||||
@ -87,10 +98,10 @@ To upload a larger file, using compression for a faster transfer, first compress
|
||||
-H "Transfer-Encoding: chunked" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
--data-binary @upload_example.csv.gz \
|
||||
"http://{username}.carto.com/api/v2/sql/copyfrom?api_key={api_key}&q=COPY+upload_example+(the_geom,+name,+age)+FROM+STDIN+WITH+(FORMAT+csv,+HEADER+true)"
|
||||
"https://{username}.carto.com/api/v2/sql/copyfrom?api_key={api_key}&q=COPY+upload_example+(the_geom,+name,+age)+FROM+STDIN+WITH+(FORMAT+csv,+HEADER+true)"
|
||||
|
||||
|
||||
### Python Example
|
||||
#### Python Example
|
||||
|
||||
The [Requests](http://docs.python-requests.org/en/master/user/quickstart/) library for HTTP makes doing a file upload relatively terse.
|
||||
|
||||
@ -102,7 +113,7 @@ username = {api_key}
|
||||
upload_file = 'upload_example.csv'
|
||||
q = "COPY upload_example (the_geom, name, age) FROM STDIN WITH (FORMAT csv, HEADER true)"
|
||||
|
||||
url = "http://%s.carto.com/api/v2/sql/copyfrom" % username
|
||||
url = "https://%s.carto.com/api/v2/sql/copyfrom" % username
|
||||
with open(upload_file, 'rb') as f:
|
||||
r = requests.post(url, params={'api_key': api_key, 'q': q}, data=f, stream=True)
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user