CartoDB-SQL-API/NEWS.md
2016-10-17 16:35:10 +02:00

810 lines
17 KiB
Markdown

1.39.2 - 2016-mm-dd
-------------------
1.39.1 - 2016-10-17
-------------------
Enhancements:
* Log creation and waiting time for fallback jobs' queries.
1.39.0 - 2016-10-17
-------------------
Enhancements:
* Use just one Redis pool across the whole application.
New features:
* Batch queries use per user-queues.
* Batch queries queues can limit the number of queued jobs per user.
- Default is 64 jobs.
- Configuration key `batch_max_queued_jobs` allows to modify the limit.
1.38.2 - 2016-10-13
-------------------
Bug fixes:
* Batch queries: release redis clients to pool from locker and seeker.
1.38.1 - 2016-10-13
-------------------
Enhancements:
* Batch queries: improvements over leader locking.
1.38.0 - 2016-10-11
-------------------
Announcements:
* Allow to set statement timeout per query in multi query batch queries.
* Batch queries default statement timeout set to 12 hours.
* Multiple queries jobs pushed as first job between queries.
1.37.1 - 2016-10-05
-------------------
Bug fixes:
* Body parser accepting multipart requests.
1.37.0 - 2016-10-04
-------------------
Enhancements:
* Migrate to Express.js 4.x series.
1.36.2 - 2016-10-03
-------------------
Bug fixes:
- Batch Queries logs: use path instead of stream to be able to reopen FD.
1.36.1 - 2016-09-30
-------------------
Enhancements:
* Tag fallback jobs logs.
1.36.0 - 2016-09-30
-------------------
New features:
* Log queries from batch fallback jobs.
Enhancements:
* assert.response following callback(err, obj) pattern.
1.35.0 - 2016-09-15
-------------------
New features:
* Allow to use `--config /path/to/config.js` to specify configuration file.
- Environment will be loaded from config file if `environment` key is present, otherwise it keeps current behaviour.
Bug fixes:
* Allow to use absolute paths for log files.
Announcements:
* Removes support for optional rollbar logging.
1.34.2 - 2016-08-30
-------------------
Announcements:
* Upgrades cartodb-redis to 0.13.1.
* Set TTL of finished job to 2h
1.34.1 - 2016-07-11
-------------------
Bug fixes:
* Fixed issue with redis connections in Batch API #326
1.34.0 - 2016-07-11
-------------------
New features:
* Skip tables with no updated_at registered in cdb_tablemetadata.
* Allow to setup more than one domain to validate oauth against.
1.33.0 - 2016-07-01
-------------------
New features:
* Add `<%= job_id %>` template support for onerror and onsuccess fallback queries.
1.32.0 - 2016-06-30
-------------------
New features:
* Broadcast after enqueueing jobs to improve query distribution load.
* Batch pub-sub channel handles its connections using `redis-mpool`.
1.31.0 - 2016-06-29
-------------------
New features:
* Adds start and end time for batch queries with fallback.
* Add `<%= error_message %>` template support for onerror fallback queries.
1.30.1 - 2016-06-23
-------------------
Bug fixes:
* Fixed issue with profiling in Batch API #318
1.30.0 - 2016-06-14
-------------------
Announcements:
* Now Batch API sends stats metrics to statsd server #312
* Now Batch API sets "skipped" instead of "pending" to queries that won't be performed #311
Bug fixes:
* Fixed issue with error handling in Batch API #316
1.29.2 - 2016-05-25
-------------------
Bug fixes:
* Fixed issue with status transition in fallback jobs #308
1.29.1 - 2016-05-24
-------------------
Announcements:
* Change Batch API size limit: 8kb per job.
1.29.0 - 2016-05-24
-------------------
New features:
* Add support for fallback-jobs in Batch API #296
Bug fixes:
* Fix issue in Batch API when a 'no longer running' job reports as 'running' before and after a job cancel #293
1.28.1 - 2016-05-12
-------------------
Bug fixes:
* OGR with _needSRS=true_ fails for empty tables #299
1.28.0 - 2016-05-11
-------------------
Announcements:
- Upgrades step-profiler to 0.3.0 to avoid dots in json keys #294
New features:
* Add support for geopackage format (`format=gpkg` at the URL) #291
1.27.1 - 2016-04-18
-------------------
Bug fixes:
* Size of queries in Batch API is limited to 4kb per job
1.27.0 - 2016-04-05
-------------------
New features:
* Add support for multiquery-jobs in Batch API #280
* Add queue discovering for Batch processing at service startup #282
Bug fixes:
* Fix issue in Batch API when after a period of inactivity it does not process jobs
1.26.0 - 2016-03-08
-------------------
New features:
* Add [Surrogate-Key](https://github.com/CartoDB/cartodb/wiki/CartoDB-Surrogate-Keys) headers to responses
Enhancements:
* Use new `node-cartodb-query-tables` library to obtain affected tables in queries
1.25.3 - 2016-02-02
-------------------
Enhancements:
* QueryTablesApi caches affected tables and retrieves last modification #269
1.25.2 - 2016-02-01
-------------------
Bug fixes:
* Skip query-tables-api for authenticated requests
1.25.1 - 2016-01-28
-------------------
Bug fixes:
* Fix X-Cache-Channel generation when request are not authenticated #266
1.25.0 - 2016-01-26
-------------------
Bug fixes:
* Stop adding X-Cache-Channel header when no tables involved #250
New features:
* Set `Last-Modified` header based on affected tables (#101)
* Batch API (#261):
- New endpoint to create, read, update and delete long-running queries (jobs).
- Batch service to process jobs.
* Set Last-Modified header based on affected tables #247
Announcements:
* Upgrades cartodb-psql to [0.6.1](https://github.com/CartoDB/node-cartodb-psql/releases/tag/0.6.1)
1.24.0 - 2015-08-04
-------------------
New features:
* Client is removed from pool after error happens. This help to avoid issues with transactions (#241).
Announcements:
* Upgrades cartodb-psql to [0.6.0](https://github.com/CartoDB/node-cartodb-psql/releases/tag/0.6.0)
1.23.0 - 2015-06-16
-------------------
Announcements:
* Reverts tables=<fake> for ogr2ogr commands
Ref https://github.com/CartoDB/CartoDB-SQL-API/commit/b6e53f732672369d7f9be26555ef412edc202727
Enhancements:
* Makes ogr2ogr command configurable so it's possible to change path/bin
* Uses :remote-addr instead of :req[X-Real-IP] \(#197)
New features:
* Adds SpatiaLite as export format (#226)
1.22.2 - 2015-05-26
-------------------
Bug fixes:
* Reintroduces tables= param in ogr2ogr exports (#204)
This will avoid running a heavy ogr2ogr query when the pg catalog is big
Ref https://github.com/CartoDB/CartoDB-SQL-API/commit/84c422c505391ef0e743aed2204214d4286d7e30
1.22.1 - 2015-05-14
-------------------
Bug fixes:
* Close stream responses on error (#219)
Enhancements:
* Format files split into pg and ogr directories
1.22.0 - 2015-04-09
-------------------
Announcements:
* Now health check only validates against a disabling file
* Supports user extraction from request params via base_url config
1.21.1 - 2015-03-02
-------------------
Enhancements:
* Improve row size limit error message
1.21.0 - 2015-03-02
-------------------
New features:
* Logs with console.error too large row errors
1.20.0 - 2015-02-26
-------------------
Announcements:
* Upgrades cartodb-psql to 0.5.1 for keep alive configuration
* Dependencies from npm registry when available
1.19.1 - 2014-12-15
-------------------
Bug fixes:
* Closes stream responses on error (#188)
* Closes fd for log files on `kill -HUP` (#187)
1.19.0 - 2014-11-21
-------------------
New features:
* Add more fields to error responses with hint, detail and context for SQL errors.
Enhancements:
* Don't loop twice over svg rows
* Improve statement timeout error messages
* Improve topojson output by streaming json
1.18.0 - 2014-10-14
-------------------
Announcements:
* Dropping support for npm <1.2.1
npm-shrinkwrap.json is incompatible when generated with npm >=1.2.1 and consumed by npm <1.2.1
1.17.1 - 2014-09-23
-------------------
Enhancements:
* Removes tables=fake wadus param in ogr2ogr command so it can go to geometry
columns view to retrieve the column data type. This requires to grant select
permission on geometry_columns and geography_columns to the public user.
* Removes query tables console.log
Bug fixes:
* Fixes "make check" on systems with non-default PostgreSQL superuser (#152)
1.17.0 - 2014-09-17
-------------------
Bug fixes:
* Returns 401 Unauthorized for queries without permission
New features:
* New header for database host serving the request
* Health check endpoint
Enhancements:
* Upgrades dependencies:
* cartodb-redis
* cartodb-psql
* log4js
1.16.0 - 2014-08-19
-------------------
Enhancements:
* Metrics revamp: removes and adds some metrics
1.15.0 - 2014-08-18
-------------------
Enhancements:
* Upgrades cartodb-redis
* Upgrades underscore, removes underscore.string dependency
* Uses https endpoints for dependencies
1.14.1 - 2014-08-08
-------------------
Other changes:
* Constraint for pg_ queries if request is non authenticated
1.14.0 - 2014-08-07
-------------------
Other changes:
* Removes sql statements restriction on pg_ queries
1.13.0 - 2014-08-07
-------------------
New features:
* New authentication mechanism: checks in advance if credentials are provided
in order to do a single request to redis to retrieve the required database
connection parameters.
* Retrieves OAuth hash values using new cartodb-redis method so it will reuse
the redis pool from cartodb-redis instead of using just one pool for oauth.
1.12.1 - 2014-08-05
-------------------
Bug fixes:
* Fixes GeoJSON stream error responses
* Fixes GeoJSON stream empty responses
* JSONP callbacks return with 200 status error code
Enhancements:
* Re-enables tests
1.12.0 - 2014-08-04
-------------------
New features:
* Add header for host serving the request
* Stream JSON/GeoJSON responses
1.11.0 - 2014-07-30
-------------------
New features:
* Support for different schemas and different public users
Enhancements:
* Profiler header sent as JSON string
Other changes:
* Revamped documentation
Bug fixes:
* Pick redis pool configuration values
1.10.1 - 2014-06-05
-------------------
Bug fixes:
* Backing out Stream JSON responses
1.10.0 - 2014-06-04
-------------------
New features:
* Order by and sort order through http query params
* Cancelling queries in Postgresql when HTTP request is aborted/closed
Enhancements:
* Stream JSON responses
* Pre-compiling may write regex
* Set default PostgreSQL application name to "cartodb_sqlapi"
Bug fixes:
* Support trailing semicolons (#147)
1.9.1 - 2014-03-27
------------------
Bug fixes:
* Fix paging with queries starting with comments (#144)
1.9.0 - 2014-03-20
------------------
New features:
* Add optional support for rollbar (#137)
* Add '/version' endpoint (#138)
* Add profiler support (#142)
* Add statsd support (#133)
Enhancements:
* Allow configuring log_format (#131)
* Use log4js for logging (#136)
* Include version in startup log
* Allow passing environment configuration name via NODE_ENV to app.js
* Print environment configuration name on app start
* Upgrade node-zipfile to ~0.5.0
* Add support for node-0.10 (#132)
* Fix lack of response on backend crash (#135)
* Reduce work on aborted requests (#129)
Other changes:
* Switch to 3-clause BSD license (#143)
1.8.3 - 2014-02-10
------------------
Bug fixes:
* Honour the 'node_socket_timeout' configuration directive (#128)
Enhancements:
* Add support for error handling in assert.request
* Stop using ANSI colors in the logs (#130)
1.8.2 - 2014-01-20
------------------
Bug fixes:
* Restore compatibility with 1.6.x configuration
* Use db_port in ogr2ogr
1.8.1 - 2014-01-10
------------------
Bug fixes:
* Fix use of "SELECT .. INTO" with windowing params (#127)
1.8.0 - 2013-12-18
------------------
New features:
* Add 'user_from_host' directive to generalize username extraction (#124)
Improvements:
* Enhance error message on unknown cartodb username (#126)
1.7.1 - 2013-12-02
------------------
* Fix documentation for CSV export format: geoms are in hexewkb, not ewkt.
* Fix field types names lookup after PSQL model refactoring
NOTE: fixes missing .prj in shapefile export regression (#122)
1.7.0 - 2013-11-19
------------------
New features:
* Optionally read user-specific database_host and database_password
from redis, as per CartoDB-2.5.0 model (#120, #121)
* Add warnings and notices to JSON response (#104)
Other changes:
* CartoDB redis interaction delegated to "cartodb-redis" module
1.6.3 - 2013-11-10
------------------
* JSON format: correctly recognize "numeric" type columns (#119)
1.6.2 - 2013-11-07
------------------
* JSON format: correctly recognize "date" type columns (#117)
* Allow access to tables whose name contains (but does not start with)
the "pg_" substring (#118)
1.6.1 - 2013-11-05
------------------
* Still set a meaningful X-Cache-Channel with cache_policy=persist (#105)
* Fix wrong projection in KML exports for manually altered tables (#116)
* Set KML folder name to the requested filename (#115)
* Make public PostgreSQL user name a configuration parameter (#56)
1.6.0 - 2013-10-02
------------------
* Fix shapefile export for non-linestring results starting with NULLs (#111)
* Fix missing .prj in shapefile export (#110)
* Improve recognition of non-standard field types names by db lookup (#112)
* Upgrade node-pg dependency to 2.6.2
* Drop support for cluster
1.5.4 - 2013-10-01
------------------
* Honour skipfields in JSON schema response (#109)
1.5.3
-----
* Set Last-Modified header with cache_policy=persist
* Raise max-age to one year for all cacheable queries
* Set max-age to 0 for uncacheable (mutating) queries
* Add REINDEX to the list of uncacheable queries
* Support all parameters with POST as well as GET
* Ensure testsuite passes with both GDAL-0.9 and GDAL-0.10
* JSON output: report boolean types as boolean, not string (#106)
1.5.2
-----
* Keep numbers as such in JSON output (#100)
* Revert max-age=0 in Cache-Control when using no-cache
1.5.1
-----
* Improve cacheability of queries selecting "updated_at" fields (#99)
1.5.0
-----
* Add "fields" member in JSON return (#97)
* Add --skipfields switch to cdbsql
* Fix windowing with CTE
* Retain UTC offset in JSON date output
* Set max-age=0 in Cache-Control when using no-cache
1.4.1
-----
* Fix windowing support for non-uppercased SELECT queries
* Fix oAuth testcase
1.4.0
-----
* Add arraybuffer format
* Fix filesystem access conflict among clustered processes
* Fix discard of queued export requests on error
* Really fix problem identifying OAuth requests
1.3.10
------
* Fixed problem identifying OAuth request protocol
* Make base url configurable
* Update underscore dependency
* Add munin plugin
* Make PostgreSQL client pooling settings configurable (#47)
* Do not execute queries on OPTIONS (#94)
* Survive postgresql connection losses (#95)
1.3.9
-----
* Do not choke on multiple `skipfields` parameter
* Do not request caching of TRUNCATE queries
1.3.8
-----
* Make using SET or querying system catalogues harder
* Allow sql queries to end with a semicolon (#90)
* Testsuite fixes, jenkins ready :)
1.3.7
-----
* Fix parsing of numeric arrays (#88)
* node-pool upgraded to 2.0.3
* Reduce memory use on KML export
* Fix concurrent request for KML and Shapefile exports
* Send an empty-like KML when exporting zero-rows queries
* Make temporary dir a configuration setting
* Use OGR for CSV output, reducing memory requirement
1.3.6 (DD/MM/YY)
-----
* Do not confuse warnings with errors on shapefile output (#87)
1.3.5 (19/02/13)
-----
* Fix skipfields use with SHP output format (#81)
* Fix Content-Disposition for error responses (#82)
* Add pid to /cachestatus (#83)
* Check CDB_QueryTable response before saving into cache (#83)
* Use an expiring LRU cache for query tables (#83)
* Fix X-Cache-Channel computation with paging parameters (#85)
1.3.4 (21/01/13)
-----
* Improve mixed-geometry export error message (#78)
* Remove NULL the_geom features from topojson output (#80)
* Fix crash when issuing SQL "COPY" command
* Return an error when "the_geom" is in skipfield for SVG output (#73)
1.3.3 (11/01/13)
-----
* Fix Date format in CSV output (#77)
* Add TopoJSON output format (#79)
1.3.2 (30/11/12)
-----
* Fix KML export truncation (#70)
* Fix UTF8 in shapefile export (#66)
1.3.1 (DD/MM/YY)
-----
* Support 'format' and 'filename' params in POST
* Fix oAuth bug introduced by 'skipfields' param in 1.3.0 (#69)
1.3.0 (DD/MM/YY)
-----
* Support for specifying a filename for exports (#64)
* Support for specifying a list of fields to skip from output (#63)
* Add 'cache_policy' parameter (#62)
1.2.1 (DD/MM/YY)
-----
* Added timeout default to 600 miliseconds in cluster.js
1.2.0 (DD/MM/YY)
-----
* New output formats:
* ESRI Shapefile (format=shp)
* SVG (format=svg)
* KML (format=kml)
* Advertise header presence in CSV Content-Type
* Fix CSV output with no rows (#60)
* Use "attachment" Content-Disposition for all output formats (#61)
* Only use last format parameter when multiple are requested
* Return a 400 response on unsupported format request
* Added X-Prototype-Version, X-CSRF-Token to Access-Control-Allow-Headers
1.1.0 (30/10/12)
-----
* Fixed problem in cluster2 with pidfile name
* SVG output format
* Enhancement to the cdbsql tool:
- New switches: --format, --key, --dp
- Interactive mode
* API documentation
* ./configure script
* Restrict listening to a node host
1.0.0 (03/10/12)
-----
* Migrated to node 0.8 version
0.9.0 (18/09/12)
-----
* Fix INSERT and UPDATE with RETURNING clause