CartoDB-SQL-API/NEWS.md

1.35.1 - 2016-mm-dd
-------------------


1.35.0 - 2016-09-15
-------------------

New features:
 * Allow to use `--config /path/to/config.js` to specify configuration file.
   - Environment will be loaded from config file if `environment` key is present, otherwise it keeps current behaviour.

Bug fixes:
 * Allow to use absolute paths for log files.

Announcements:
 * Removes support for optional rollbar logging.


1.34.2 - 2016-08-30
-------------------

Announcements:
 * Upgrades cartodb-redis to 0.13.1.
 * Set TTL of finished job to 2h


1.34.1 - 2016-07-11
-------------------

Bug fixes:
 * Fixed issue with redis connections in Batch API #326


1.34.0 - 2016-07-11
-------------------

New features:
 * Skip tables with no updated_at registered in cdb_tablemetadata.
 * Allow to setup more than one domain to validate oauth against.


1.33.0 - 2016-07-01
-------------------

New features:
 * Add `<%= job_id %>` template support for onerror and onsuccess fallback queries.


1.32.0 - 2016-06-30
-------------------

New features:
 * Broadcast after enqueueing jobs to improve query distribution load.
 * Batch pub-sub channel handles its connections using `redis-mpool`.


1.31.0 - 2016-06-29
-------------------

New features:
 * Adds start and end time for batch queries with fallback.
 * Add `<%= error_message %>` template support for onerror fallback queries.


1.30.1 - 2016-06-23
-------------------

Bug fixes:
 * Fixed issue with profiling in Batch API #318


1.30.0 - 2016-06-14
-------------------

Announcements:
 * Now Batch API sends stats metrics to statsd server #312
 * Now Batch API sets "skipped" instead of "pending" to queries that won't be performed #311

 Bug fixes:
  * Fixed issue with error handling in Batch API #316


1.29.2 - 2016-05-25
-------------------

Bug fixes:
 * Fixed issue with status transition in fallback jobs #308


1.29.1 - 2016-05-24
-------------------

Announcements:
 * Change Batch API size limit: 8kb per job.


1.29.0 - 2016-05-24
-------------------

New features:
 * Add support for fallback-jobs in Batch API #296

Bug fixes:
 * Fix issue in Batch API when a 'no longer running' job reports as 'running' before and after a job cancel #293


1.28.1 - 2016-05-12
-------------------

Bug fixes:
 * OGR with _needSRS=true_ fails for empty tables #299


1.28.0 - 2016-05-11
-------------------

Announcements:
 - Upgrades step-profiler to 0.3.0 to avoid dots in json keys #294

 New features:

  * Add support for geopackage format (`format=gpkg` at the URL) #291


1.27.1 - 2016-04-18
-------------------

Bug fixes:

  * Size of queries in Batch API is limited to 4kb per job


1.27.0 - 2016-04-05
-------------------

New features:

 * Add support for multiquery-jobs in Batch API #280
 * Add queue discovering for Batch processing at service startup #282

Bug fixes:

 * Fix issue in Batch API when after a period of inactivity it does not process jobs


1.26.0 - 2016-03-08
-------------------

New features:

 * Add [Surrogate-Key](https://github.com/CartoDB/cartodb/wiki/CartoDB-Surrogate-Keys) headers to responses

Enhancements:

 * Use new `node-cartodb-query-tables` library to obtain affected tables in queries


1.25.3 - 2016-02-02
-------------------

Enhancements:

 * QueryTablesApi caches affected tables and retrieves last modification #269


1.25.2 - 2016-02-01
-------------------

Bug fixes:

 * Skip query-tables-api for authenticated requests


1.25.1 - 2016-01-28
-------------------

Bug fixes:

 * Fix X-Cache-Channel generation when request are not authenticated #266


1.25.0 - 2016-01-26
-------------------

Bug fixes:

 * Stop adding X-Cache-Channel header when no tables involved #250

New features:

 * Set `Last-Modified` header based on affected tables (#101)
 * Batch API (#261):
   - New endpoint to create, read, update and delete long-running queries (jobs).
   - Batch service to process jobs.
 * Set Last-Modified header based on affected tables #247

Announcements:

 * Upgrades cartodb-psql to [0.6.1](https://github.com/CartoDB/node-cartodb-psql/releases/tag/0.6.1)


1.24.0 - 2015-08-04
-------------------

New features:

 * Client is removed from pool after error happens. This help to avoid issues with transactions (#241).

Announcements:

 * Upgrades cartodb-psql to [0.6.0](https://github.com/CartoDB/node-cartodb-psql/releases/tag/0.6.0)


1.23.0 - 2015-06-16
-------------------

Announcements:

 * Reverts tables=<fake> for ogr2ogr commands
   Ref https://github.com/CartoDB/CartoDB-SQL-API/commit/b6e53f732672369d7f9be26555ef412edc202727

Enhancements:

 * Makes ogr2ogr command configurable so it's possible to change path/bin
 * Uses :remote-addr instead of :req[X-Real-IP] \(#197)

New features:

 * Adds SpatiaLite as export format (#226)


1.22.2 - 2015-05-26
-------------------

Bug fixes:

 * Reintroduces tables= param in ogr2ogr exports (#204)
   This will avoid running a heavy ogr2ogr query when the pg catalog is big
   Ref https://github.com/CartoDB/CartoDB-SQL-API/commit/84c422c505391ef0e743aed2204214d4286d7e30


1.22.1 - 2015-05-14
-------------------

Bug fixes:

 * Close stream responses on error (#219)

Enhancements:

 * Format files split into pg and ogr directories


1.22.0 - 2015-04-09
-------------------

Announcements:

 * Now health check only validates against a disabling file
 * Supports user extraction from request params via base_url config


1.21.1 - 2015-03-02
-------------------

Enhancements:

 * Improve row size limit error message


1.21.0 - 2015-03-02
-------------------

New features:

 * Logs with console.error too large row errors


1.20.0 - 2015-02-26
-------------------

Announcements:

 * Upgrades cartodb-psql to 0.5.1 for keep alive configuration
 * Dependencies from npm registry when available


1.19.1 - 2014-12-15
-------------------

Bug fixes:

 * Closes stream responses on error (#188)
 * Closes fd for log files on `kill -HUP` (#187)


1.19.0 - 2014-11-21
-------------------

New features:

 * Add more fields to error responses with hint, detail and context for SQL errors.

Enhancements:

 * Don't loop twice over svg rows
 * Improve statement timeout error messages
 * Improve topojson output by streaming json


1.18.0 - 2014-10-14
-------------------

Announcements:

 * Dropping support for npm <1.2.1
   npm-shrinkwrap.json is incompatible when generated with npm >=1.2.1 and consumed by npm <1.2.1


1.17.1 - 2014-09-23
-------------------

Enhancements:

 * Removes tables=fake wadus param in ogr2ogr command so it can go to geometry
   columns view to retrieve the column data type. This requires to grant select
   permission on geometry_columns and geography_columns to the public user.
 * Removes query tables console.log

Bug fixes:

 * Fixes "make check" on systems with non-default PostgreSQL superuser (#152)


1.17.0 - 2014-09-17
-------------------

Bug fixes:
 * Returns 401 Unauthorized for queries without permission

New features:

 * New header for database host serving the request
 * Health check endpoint

Enhancements:

 * Upgrades dependencies:
    * cartodb-redis
    * cartodb-psql
    * log4js


1.16.0 - 2014-08-19
-------------------

Enhancements:

 * Metrics revamp: removes and adds some metrics

1.15.0 - 2014-08-18
-------------------

Enhancements:

 * Upgrades cartodb-redis
 * Upgrades underscore, removes underscore.string dependency
 * Uses https endpoints for dependencies


1.14.1 - 2014-08-08
-------------------

Other changes:

 * Constraint for pg_ queries if request is non authenticated

1.14.0 - 2014-08-07
-------------------

Other changes:

 * Removes sql statements restriction on pg_ queries


1.13.0 - 2014-08-07
-------------------

New features:

 * New authentication mechanism: checks in advance if credentials are provided
   in order to do a single request to redis to retrieve the required database
   connection parameters.
 * Retrieves OAuth hash values using new cartodb-redis method so it will reuse
   the redis pool from cartodb-redis instead of using just one pool for oauth.


1.12.1 - 2014-08-05
-------------------

Bug fixes:

 * Fixes GeoJSON stream error responses
 * Fixes GeoJSON stream empty responses
 * JSONP callbacks return with 200 status error code

Enhancements:

 * Re-enables tests

1.12.0 - 2014-08-04
-------------------

New features:

 * Add header for host serving the request
 * Stream JSON/GeoJSON responses

1.11.0 - 2014-07-30
-------------------

New features:

 * Support for different schemas and different public users

Enhancements:

 * Profiler header sent as JSON string

Other changes:

 * Revamped documentation

Bug fixes:

 * Pick redis pool configuration values

1.10.1 - 2014-06-05
-------------------
Bug fixes:

 * Backing out Stream JSON responses

1.10.0 - 2014-06-04
-------------------

New features:

 * Order by and sort order through http query params
 * Cancelling queries in Postgresql when HTTP request is aborted/closed

Enhancements:

 * Stream JSON responses
 * Pre-compiling may write regex
 * Set default PostgreSQL application name to "cartodb_sqlapi"

Bug fixes:

 * Support trailing semicolons (#147)

1.9.1 - 2014-03-27
------------------

Bug fixes:

 * Fix paging with queries starting with comments (#144)

1.9.0 - 2014-03-20
------------------

New features:

 * Add optional support for rollbar (#137)
 * Add '/version' endpoint (#138)
 * Add profiler support (#142)
 * Add statsd support (#133)

Enhancements:

 * Allow configuring log_format (#131)
 * Use log4js for logging (#136)
 * Include version in startup log
 * Allow passing environment configuration name via NODE_ENV to app.js
 * Print environment configuration name on app start
 * Upgrade node-zipfile to ~0.5.0
 * Add support for node-0.10 (#132)
 * Fix lack of response on backend crash (#135)
 * Reduce work on aborted requests (#129)

Other changes:

 * Switch to 3-clause BSD license (#143)

1.8.3 - 2014-02-10
------------------

Bug fixes:

 * Honour the 'node_socket_timeout' configuration directive (#128)

Enhancements:

 * Add support for error handling in assert.request
 * Stop using ANSI colors in the logs (#130)

1.8.2 - 2014-01-20
------------------

Bug fixes:

 * Restore compatibility with 1.6.x configuration
 * Use db_port in ogr2ogr

1.8.1 - 2014-01-10
------------------

Bug fixes:

 * Fix use of "SELECT .. INTO" with windowing params (#127)


1.8.0 - 2013-12-18
------------------

New features:

* Add 'user_from_host' directive to generalize username extraction (#124)

Improvements:

* Enhance error message on unknown cartodb username (#126)

1.7.1 - 2013-12-02
------------------

* Fix documentation for CSV export format: geoms are in hexewkb, not ewkt.
* Fix field types names lookup after PSQL model refactoring
  NOTE: fixes missing .prj in shapefile export regression (#122)

1.7.0 - 2013-11-19
------------------

New features:

* Optionally read user-specific database_host and database_password
  from redis, as per CartoDB-2.5.0 model (#120, #121)
* Add warnings and notices to JSON response (#104)

Other changes:

* CartoDB redis interaction delegated to "cartodb-redis" module

1.6.3 - 2013-11-10
------------------
* JSON format: correctly recognize "numeric" type columns (#119)

1.6.2 - 2013-11-07
------------------
* JSON format: correctly recognize "date" type columns (#117)
* Allow access to tables whose name contains (but does not start with)
  the "pg_" substring (#118)

1.6.1 - 2013-11-05
------------------
* Still set a meaningful X-Cache-Channel with cache_policy=persist (#105)
* Fix wrong projection in KML exports for manually altered tables (#116)
* Set KML folder name to the requested filename (#115)
* Make public PostgreSQL user name a configuration parameter (#56)

1.6.0 - 2013-10-02
------------------
* Fix shapefile export for non-linestring results starting with NULLs (#111)
* Fix missing .prj in shapefile export (#110)
* Improve recognition of non-standard field types names by db lookup (#112)
* Upgrade node-pg dependency to 2.6.2
* Drop support for cluster

1.5.4 - 2013-10-01
------------------
* Honour skipfields in JSON schema response (#109)

1.5.3
-----
* Set Last-Modified header with cache_policy=persist
* Raise max-age to one year for all cacheable queries
* Set max-age to 0 for uncacheable (mutating) queries
* Add REINDEX to the list of uncacheable queries
* Support all parameters with POST as well as GET
* Ensure testsuite passes with both GDAL-0.9 and GDAL-0.10
* JSON output: report boolean types as boolean, not string (#106)

1.5.2
-----
* Keep numbers as such in JSON output (#100)
* Revert max-age=0 in Cache-Control when using no-cache

1.5.1
-----
* Improve cacheability of queries selecting "updated_at" fields (#99)

1.5.0
-----
* Add "fields" member in JSON return (#97)
* Add --skipfields switch to cdbsql
* Fix windowing with CTE
* Retain UTC offset in JSON date output
* Set max-age=0 in Cache-Control when using no-cache

1.4.1
-----
* Fix windowing support for non-uppercased SELECT queries
* Fix oAuth testcase

1.4.0
-----
* Add arraybuffer format
* Fix filesystem access conflict among clustered processes
* Fix discard of queued export requests on error
* Really fix problem identifying OAuth requests

1.3.10
------
* Fixed problem identifying OAuth request protocol
* Make base url configurable
* Update underscore dependency
* Add munin plugin
* Make PostgreSQL client pooling settings configurable (#47)
* Do not execute queries on OPTIONS (#94)
* Survive postgresql connection losses (#95)

1.3.9
-----
* Do not choke on multiple `skipfields` parameter
* Do not request caching of TRUNCATE queries

1.3.8
-----
* Make using SET or querying system catalogues harder
* Allow sql queries to end with a semicolon (#90)
* Testsuite fixes, jenkins ready :)

1.3.7
-----
* Fix parsing of numeric arrays (#88)
* node-pool upgraded to 2.0.3
* Reduce memory use on KML export
* Fix concurrent request for KML and Shapefile exports
* Send an empty-like KML when exporting zero-rows queries
* Make temporary dir a configuration setting
* Use OGR for CSV output, reducing memory requirement

1.3.6 (DD/MM/YY)
-----
* Do not confuse warnings with errors on shapefile output (#87)

1.3.5 (19/02/13)
-----
* Fix skipfields use with SHP output format (#81)
* Fix Content-Disposition for error responses (#82)
* Add pid to /cachestatus (#83)
* Check CDB_QueryTable response before saving into cache (#83)
* Use an expiring LRU cache for query tables (#83)
* Fix X-Cache-Channel computation with paging parameters (#85)

1.3.4 (21/01/13)
-----
* Improve mixed-geometry export error message (#78)
* Remove NULL the_geom features from topojson output (#80)
* Fix crash when issuing SQL "COPY" command
* Return an error when "the_geom" is in skipfield for SVG output (#73)

1.3.3 (11/01/13)
-----
* Fix Date format in CSV output (#77)
* Add TopoJSON output format (#79)

1.3.2 (30/11/12)
-----
* Fix KML export truncation (#70)
* Fix UTF8 in shapefile export (#66)

1.3.1 (DD/MM/YY)
-----
* Support 'format' and 'filename' params in POST
* Fix oAuth bug introduced by 'skipfields' param in 1.3.0 (#69)

1.3.0 (DD/MM/YY)
-----
* Support for specifying a filename for exports (#64)
* Support for specifying a list of fields to skip from output (#63)
* Add 'cache_policy' parameter (#62)

1.2.1 (DD/MM/YY)
-----
* Added timeout default to 600 miliseconds in cluster.js

1.2.0 (DD/MM/YY)
-----
* New output formats:
  * ESRI Shapefile (format=shp)
  * SVG (format=svg)
  * KML (format=kml)
* Advertise header presence in CSV Content-Type
* Fix CSV output with no rows (#60)
* Use "attachment" Content-Disposition for all output formats (#61)
* Only use last format parameter when multiple are requested
* Return a 400 response on unsupported format request
* Added X-Prototype-Version, X-CSRF-Token to Access-Control-Allow-Headers

1.1.0 (30/10/12)
-----
* Fixed problem in cluster2 with pidfile name
* SVG output format
* Enhancement to the cdbsql tool:
  - New switches: --format, --key, --dp
  - Interactive mode
* API documentation
* ./configure script
* Restrict listening to a node host

1.0.0 (03/10/12)
-----
* Migrated to node 0.8 version

0.9.0 (18/09/12)
-----
* Fix INSERT and UPDATE with RETURNING clause