CartoDB-SQL-API/NEWS.md at 6dba06130bbe09f6851b5bac4eff2450ee4cf268

cartodb/CartoDB-SQL-API

Fork 0

Daniel García Aubert 2276187c27 Release 1.44.2

2017-04-05 15:05:29 +02:00

19 KiB

Raw Blame History

#Changelog

1.44.2

Released 2017-04-05

Bug fixes:

Update queue index while enqueueing jobs to the top of queue.

1.44.1

Released 2017-04-04

Bug fixes:

Avoid to scan the whole meta-database to discover active job queues. Now Batch Queries uses a set as index to know what queues are being processed #415

1.44.0

Released 2017-03-30

Announcements:

Active GC interval for Node.js >=v6.

1.43.1

Released 2017-01-16

Announcements:

Upgrade cartodb-psql to 0.7.1.

1.43.0

Released 2017-01-16

Announcements:

Upgrade cartodb-psql to 0.7.0.

1.42.7

Released 2017-01-12

Enhancements:

Avoid gpkg fid column #404.

1.42.6

Released 2016-12-19

Announcements:

Upgrade cartodb-redis to 0.13.2.
Upgrade redis-mpool to 0.4.1.

1.42.5

Released 2016-12-12

Enhancements:

Improvements in testing environment/tests.

1.42.4

Released 2016-11-30

Enhancements:

Include query status in batch queries log entries.

1.42.3

Released 2016-11-07

Announcements:

Raise payload limit for batch-queries to 16kb.

1.42.2

Released 2016-11-07

Bug fixes:

Improve error handling while registering jobs to be tracked.

1.42.1

Released 2016-11-03

Bug fixes:

Avoid to use SCAN command to find work-in-progress queues.

1.42.0

Released 2016-11-02

Announcements:

Adds endpoint to check running batch queries

1.41.0

Released 2016-10-21

Announcements:

Stop migrating old queues by default.

Bug fixes:

Fix some scenarios where batch queries got stuck waiting for available slots.

1.40.0

Released 2016-10-20

New features:

Batch queries are handled per db host.
- There is an scheduler controlling how many queries and in what order they are run.
  - Priority is based on: number of queries already ran, and oldest user in queue.
Batch queries capacity: allow to configure how many jobs to run per db host.

1.39.1

Released 2016-10-17

Enhancements:

Log creation and waiting time for fallback jobs' queries.

1.39.0

Released 2016-10-17

Enhancements:

Use just one Redis pool across the whole application.

New features:

Batch queries use per user-queues.
Batch queries queues can limit the number of queued jobs per user.
- Default is 64 jobs.
- Configuration key batch_max_queued_jobs allows to modify the limit.

1.38.2

Released 2016-10-13

Bug fixes:

Batch queries: release redis clients to pool from locker and seeker.

1.38.1

Released 2016-10-13

Enhancements:

Batch queries: improvements over leader locking.

1.38.0

Released 2016-10-11

Announcements:

Allow to set statement timeout per query in multi query batch queries.
Batch queries default statement timeout set to 12 hours.
Multiple queries jobs pushed as first job between queries.

1.37.1

Released 2016-10-05

Bug fixes:

Body parser accepting multipart requests.

1.37.0

Released 2016-10-04

Enhancements:

Migrate to Express.js 4.x series.

1.36.2

Released 2016-10-03

Bug fixes:

Batch Queries logs: use path instead of stream to be able to reopen FD.

1.36.1

Released 2016-09-30

Enhancements:

Tag fallback jobs logs.

1.36.0

Released 2016-09-30

New features:

Log queries from batch fallback jobs.

Enhancements:

assert.response following callback(err, obj) pattern.

1.35.0

Released 2016-09-15

New features:

Allow to use --config /path/to/config.js to specify configuration file.
- Environment will be loaded from config file if environment key is present, otherwise it keeps current behaviour.

Bug fixes:

Allow to use absolute paths for log files.

Announcements:

Removes support for optional rollbar logging.

1.34.2

Released 2016-08-30

Announcements:

Upgrades cartodb-redis to 0.13.1.
Set TTL of finished job to 2h

1.34.1

Released 2016-07-11

Bug fixes:

Fixed issue with redis connections in Batch API #326

1.34.0

Released 2016-07-11

New features:

Skip tables with no updated_at registered in cdb_tablemetadata.
Allow to setup more than one domain to validate oauth against.

1.33.0

Released 2016-07-01

New features:

Add <%= job_id %> template support for onerror and onsuccess fallback queries.

1.32.0

Released 2016-06-30

New features:

Broadcast after enqueueing jobs to improve query distribution load.
Batch pub-sub channel handles its connections using redis-mpool.

1.31.0

Released 2016-06-29

New features:

Adds start and end time for batch queries with fallback.
Add <%= error_message %> template support for onerror fallback queries.

1.30.1

Released 2016-06-23

Bug fixes:

Fixed issue with profiling in Batch API #318

1.30.0

Released 2016-06-14

Announcements:

Now Batch API sends stats metrics to statsd server #312
Now Batch API sets "skipped" instead of "pending" to queries that won't be performed #311

Bug fixes:

Fixed issue with error handling in Batch API #316

1.29.2

Released 2016-05-25

Bug fixes:

Fixed issue with status transition in fallback jobs #308

1.29.1

Released 2016-05-24

Announcements:

Change Batch API size limit: 8kb per job.

1.29.0

Released 2016-05-24

New features:

Add support for fallback-jobs in Batch API #296

Bug fixes:

Fix issue in Batch API when a 'no longer running' job reports as 'running' before and after a job cancel #293

1.28.1

Released 2016-05-12

Bug fixes:

OGR with needSRS=true fails for empty tables #299

1.28.0

Released 2016-05-11

Announcements:

Upgrades step-profiler to 0.3.0 to avoid dots in json keys #294

New features:

Add support for geopackage format (format=gpkg at the URL) #291

1.27.1

Released 2016-04-18

Bug fixes:

Size of queries in Batch API is limited to 4kb per job

1.27.0

Released 2016-04-05

New features:

Add support for multiquery-jobs in Batch API #280
Add queue discovering for Batch processing at service startup #282

Bug fixes:

Fix issue in Batch API when after a period of inactivity it does not process jobs

1.26.0

Released 2016-03-08

New features:

Add Surrogate-Key headers to responses

Enhancements:

Use new node-cartodb-query-tables library to obtain affected tables in queries

1.25.3

Released 2016-02-02

Enhancements:

QueryTablesApi caches affected tables and retrieves last modification #269

1.25.2

Released 2016-02-01

Bug fixes:

Skip query-tables-api for authenticated requests

1.25.1

Released 2016-01-28

Bug fixes:

Fix X-Cache-Channel generation when request are not authenticated #266

1.25.0

Released 2016-01-26

Bug fixes:

Stop adding X-Cache-Channel header when no tables involved #250

New features:

Set Last-Modified header based on affected tables (#101)
Batch API (#261):
- New endpoint to create, read, update and delete long-running queries (jobs).
- Batch service to process jobs.
Set Last-Modified header based on affected tables #247

Announcements:

Upgrades cartodb-psql to 0.6.1

1.24.0

Released 2015-08-04

New features:

Client is removed from pool after error happens. This help to avoid issues with transactions (#241).

Announcements:

Upgrades cartodb-psql to 0.6.0

1.23.0

Released 2015-06-16

Announcements:

Reverts tables= for ogr2ogr commands Ref b6e53f7326

Enhancements:

Makes ogr2ogr command configurable so it's possible to change path/bin
Uses :remote-addr instead of :req[X-Real-IP] (#197)

New features:

Adds SpatiaLite as export format (#226)

1.22.2

Released 2015-05-26

Bug fixes:

Reintroduces tables= param in ogr2ogr exports (#204) This will avoid running a heavy ogr2ogr query when the pg catalog is big Ref 84c422c505

1.22.1

Released 2015-05-14

Bug fixes:

Close stream responses on error (#219)

Enhancements:

Format files split into pg and ogr directories

1.22.0

Released 2015-04-09

Announcements:

Now health check only validates against a disabling file
Supports user extraction from request params via base_url config

1.21.1

Released 2015-03-02

Enhancements:

Improve row size limit error message

1.21.0

Released 2015-03-02

New features:

Logs with console.error too large row errors

1.20.0

Released 2015-02-26

Announcements:

Upgrades cartodb-psql to 0.5.1 for keep alive configuration
Dependencies from npm registry when available

1.19.1

Released 2014-12-15

Bug fixes:

Closes stream responses on error (#188)
Closes fd for log files on kill -HUP (#187)

1.19.0

Released 2014-11-21

New features:

Add more fields to error responses with hint, detail and context for SQL errors.

Enhancements:

Don't loop twice over svg rows
Improve statement timeout error messages
Improve topojson output by streaming json

1.18.0

Released 2014-10-14

Announcements:

Dropping support for npm <1.2.1 npm-shrinkwrap.json is incompatible when generated with npm >=1.2.1 and consumed by npm <1.2.1

1.17.1

Released 2014-09-23

Enhancements:

Removes tables=fake wadus param in ogr2ogr command so it can go to geometry columns view to retrieve the column data type. This requires to grant select permission on geometry_columns and geography_columns to the public user.
Removes query tables console.log

Bug fixes:

Fixes "make check" on systems with non-default PostgreSQL superuser (#152)

1.17.0

Released 2014-09-17

Bug fixes:

Returns 401 Unauthorized for queries without permission

New features:

New header for database host serving the request
Health check endpoint

Enhancements:

Upgrades dependencies:
- cartodb-redis
- cartodb-psql
- log4js

1.16.0

Released 2014-08-19

Enhancements:

Metrics revamp: removes and adds some metrics

1.15.0

Released 2014-08-18

Enhancements:

Upgrades cartodb-redis
Upgrades underscore, removes underscore.string dependency
Uses https endpoints for dependencies

1.14.1

Released 2014-08-08

Other changes:

Constraint for pg_ queries if request is non authenticated

1.14.0

Released 2014-08-07

Other changes:

Removes sql statements restriction on pg_ queries

1.13.0

Released 2014-08-07

New features:

New authentication mechanism: checks in advance if credentials are provided in order to do a single request to redis to retrieve the required database connection parameters.
Retrieves OAuth hash values using new cartodb-redis method so it will reuse the redis pool from cartodb-redis instead of using just one pool for oauth.

1.12.1

Released 2014-08-05

Bug fixes:

Fixes GeoJSON stream error responses
Fixes GeoJSON stream empty responses
JSONP callbacks return with 200 status error code

Enhancements:

Re-enables tests

1.12.0

Released 2014-08-04

New features:

Add header for host serving the request
Stream JSON/GeoJSON responses

1.11.0

Released 2014-07-30

New features:

Support for different schemas and different public users

Enhancements:

Profiler header sent as JSON string

Other changes:

Revamped documentation

Bug fixes:

Pick redis pool configuration values

1.10.1

Released 2014-06-05

Bug fixes:

Backing out Stream JSON responses

1.10.0

Released 2014-06-04

New features:

Order by and sort order through http query params
Cancelling queries in Postgresql when HTTP request is aborted/closed

Enhancements:

Stream JSON responses
Pre-compiling may write regex
Set default PostgreSQL application name to "cartodb_sqlapi"

Bug fixes:

Support trailing semicolons (#147)

1.9.1

Released 2014-03-27

Bug fixes:

Fix paging with queries starting with comments (#144)

1.9.0

Released 2014-03-20

New features:

Add optional support for rollbar (#137)
Add '/version' endpoint (#138)
Add profiler support (#142)
Add statsd support (#133)

Enhancements:

Allow configuring log_format (#131)
Use log4js for logging (#136)
Include version in startup log
Allow passing environment configuration name via NODE_ENV to app.js
Print environment configuration name on app start
Upgrade node-zipfile to ~0.5.0
Add support for node-0.10 (#132)
Fix lack of response on backend crash (#135)
Reduce work on aborted requests (#129)

Other changes:

Switch to 3-clause BSD license (#143)

1.8.3

Released 2014-02-10

Bug fixes:

Honour the 'node_socket_timeout' configuration directive (#128)

Enhancements:

Add support for error handling in assert.request
Stop using ANSI colors in the logs (#130)

1.8.2

Released 2014-01-20

Bug fixes:

Restore compatibility with 1.6.x configuration
Use db_port in ogr2ogr

1.8.1

Released 2014-01-10

Bug fixes:

Fix use of "SELECT .. INTO" with windowing params (#127)

1.8.0

Released 2013-12-18

New features:

Add 'user_from_host' directive to generalize username extraction (#124)

Improvements:

Enhance error message on unknown cartodb username (#126)

1.7.1

Released 2013-12-02

Fix documentation for CSV export format: geoms are in hexewkb, not ewkt.
Fix field types names lookup after PSQL model refactoring NOTE: fixes missing .prj in shapefile export regression (#122)

1.7.0

Released 2013-11-19

New features:

Optionally read user-specific database_host and database_password from redis, as per CartoDB-2.5.0 model (#120, #121)
Add warnings and notices to JSON response (#104)

Other changes:

CartoDB redis interaction delegated to "cartodb-redis" module

1.6.3

Released 2013-11-10

JSON format: correctly recognize "numeric" type columns (#119)

1.6.2

Released 2013-11-07

JSON format: correctly recognize "date" type columns (#117)
Allow access to tables whose name contains (but does not start with) the "pg_" substring (#118)

1.6.1

Released 2013-11-05

Still set a meaningful X-Cache-Channel with cache_policy=persist (#105)
Fix wrong projection in KML exports for manually altered tables (#116)
Set KML folder name to the requested filename (#115)
Make public PostgreSQL user name a configuration parameter (#56)

1.6.0

Released 2013-10-02

Fix shapefile export for non-linestring results starting with NULLs (#111)
Fix missing .prj in shapefile export (#110)
Improve recognition of non-standard field types names by db lookup (#112)
Upgrade node-pg dependency to 2.6.2
Drop support for cluster

1.5.4

Released 2013-10-01

Honour skipfields in JSON schema response (#109)

1.5.3

Released yyyy-mm-dd

Set Last-Modified header with cache_policy=persist
Raise max-age to one year for all cacheable queries
Set max-age to 0 for uncacheable (mutating) queries
Add REINDEX to the list of uncacheable queries
Support all parameters with POST as well as GET
Ensure testsuite passes with both GDAL-0.9 and GDAL-0.10
JSON output: report boolean types as boolean, not string (#106)

1.5.2

Released yyyy-mm-dd

Keep numbers as such in JSON output (#100)
Revert max-age=0 in Cache-Control when using no-cache

1.5.1

Released yyyy-mm-dd

Improve cacheability of queries selecting "updated_at" fields (#99)

1.5.0

Released yyyy-mm-dd

Add "fields" member in JSON return (#97)
Add --skipfields switch to cdbsql
Fix windowing with CTE
Retain UTC offset in JSON date output
Set max-age=0 in Cache-Control when using no-cache

1.4.1

Released yyyy-mm-dd

Fix windowing support for non-uppercased SELECT queries
Fix oAuth testcase

1.4.0

Released yyyy-mm-dd

Add arraybuffer format
Fix filesystem access conflict among clustered processes
Fix discard of queued export requests on error
Really fix problem identifying OAuth requests

1.3.10

Released yyyy-mm-dd

Fixed problem identifying OAuth request protocol
Make base url configurable
Update underscore dependency
Add munin plugin
Make PostgreSQL client pooling settings configurable (#47)
Do not execute queries on OPTIONS (#94)
Survive postgresql connection losses (#95)

1.3.9

Released yyyy-mm-dd

Do not choke on multiple skipfields parameter
Do not request caching of TRUNCATE queries

1.3.8

Released yyyy-mm-dd

Make using SET or querying system catalogues harder
Allow sql queries to end with a semicolon (#90)
Testsuite fixes, jenkins ready :)

1.3.7

Released yyyy-mm-dd

Fix parsing of numeric arrays (#88)
node-pool upgraded to 2.0.3
Reduce memory use on KML export
Fix concurrent request for KML and Shapefile exports
Send an empty-like KML when exporting zero-rows queries
Make temporary dir a configuration setting
Use OGR for CSV output, reducing memory requirement

1.3.6

Released yyyy-mm-dd

Do not confuse warnings with errors on shapefile output (#87)

1.3.5

Released 2013-02-19

Fix skipfields use with SHP output format (#81)
Fix Content-Disposition for error responses (#82)
Add pid to /cachestatus (#83)
Check CDB_QueryTable response before saving into cache (#83)
Use an expiring LRU cache for query tables (#83)
Fix X-Cache-Channel computation with paging parameters (#85)

1.3.4

Released 2013-01-21

Improve mixed-geometry export error message (#78)
Remove NULL the_geom features from topojson output (#80)
Fix crash when issuing SQL "COPY" command
Return an error when "the_geom" is in skipfield for SVG output (#73)

1.3.3

Released 2013-01-11

Fix Date format in CSV output (#77)
Add TopoJSON output format (#79)

1.3.2

Released 2012-11-30

Fix KML export truncation (#70)
Fix UTF8 in shapefile export (#66)

1.3.1

Released yyyy-mm-dd

Support 'format' and 'filename' params in POST
Fix oAuth bug introduced by 'skipfields' param in 1.3.0 (#69)

1.3.0

Released yyyy-mm-dd

Support for specifying a filename for exports (#64)
Support for specifying a list of fields to skip from output (#63)
Add 'cache_policy' parameter (#62)

1.2.1

Released yyyy-mm-dd

Added timeout default to 600 miliseconds in cluster.js

1.2.0

Released yyyy-mm-dd

New output formats:
- ESRI Shapefile (format=shp)
- SVG (format=svg)
- KML (format=kml)
Advertise header presence in CSV Content-Type
Fix CSV output with no rows (#60)
Use "attachment" Content-Disposition for all output formats (#61)
Only use last format parameter when multiple are requested
Return a 400 response on unsupported format request
Added X-Prototype-Version, X-CSRF-Token to Access-Control-Allow-Headers

1.1.0

Released 2012-10-30

Fixed problem in cluster2 with pidfile name
SVG output format
Enhancement to the cdbsql tool:
- New switches: --format, --key, --dp
- Interactive mode
API documentation
./configure script
Restrict listening to a node host

1.0.0

Released 2012-10-03

Migrated to node 0.8 version

0.9.0

Released 2012-09-18

Fix INSERT and UPDATE with RETURNING clause

19 KiB Raw Blame History

1.44.2

1.44.1

1.44.0

1.43.1

1.43.0

1.42.7

1.42.6

1.42.5

1.42.4

1.42.3

1.42.2

1.42.1

1.42.0

1.41.0

1.40.0

1.39.1

1.39.0

1.38.2

1.38.1

1.38.0

1.37.1

1.37.0

1.36.2

1.36.1

1.36.0

1.35.0

1.34.2

1.34.1

1.34.0

1.33.0

1.32.0

1.31.0

1.30.1

1.30.0

1.29.2

1.29.1

1.29.0

1.28.1

1.28.0

1.27.1

1.27.0

1.26.0

1.25.3

1.25.2

1.25.1

1.25.0

1.24.0

1.23.0

1.22.2

1.22.1

1.22.0

1.21.1

1.21.0

1.20.0

1.19.1

1.19.0

1.18.0

1.17.1

1.17.0

1.16.0

1.15.0

1.14.1

1.14.0

1.13.0

1.12.1

1.12.0

1.11.0

1.10.1

1.10.0

1.9.1

1.9.0

1.8.3

1.8.2

1.8.1

1.8.0

1.7.1

1.7.0

1.6.3

1.6.2

19 KiB

Raw Blame History