- Log system revamp:
- Logs to stdout, disabled while testing
- Use header `X-Request-Id`, or create a new `uuid` when no present, to identyfy log entries
- Be able to set log level from env variable `LOG_LEVEL`, useful while testing: `LOG_LEVEL=info npm test`; even more human-readable: `LOG_LEVEL=info npm t | ./node_modules/.bin/pino-pretty`
- Be able to reduce the footprint in the final log file depending on the environment
- Use one logger for every service: Queries, Batch Queries (Jobs), and Data Ingestion (CopyTo/CopyFrom)
- Stop using headers such as: `X-SQL-API-Log`, `X-SQL-API-Profiler`, and `X-SQL-API-Errors` as a way to log info.
- Be able to tag requests with labels as an easier way to provide business metrics
- Metro: Add log-collector utility (`metro`), it will be moved to its own repository. Attaching it here fro development purposes. Try it with the following command `LOG_LEVEL=info npm t | node metro`
- Metro: Creates `metrics-collector.js` a stream to update Prometheus' counters and histograms and exposes them via Express' app (`:9145/metrics`). Use the ones defined in `grok_exporter`
Announcements:
- Profiler is always set. No need to check its existence anymore
- Unify profiler usage for every endpoint
Bug fixes:
- Avoid hung requests while fetching user identifier
- Update `gc-stats` to version 1.4.0
- Replace `zipfile` -> `adm-zip`
- Update `libxmljs` to version 0.19.7
- Update `sqlite` to version 4.2.0
- Adapted pool acquires to the new version of `cartodb-redis`
- Adapted test to use `adm-zip`
If the final name is going to be too big, use the hash over the proposed
path to create an unique name that fits the requirements of path length
under Linux
I think this was added for historical reasons when upstream plpython
was not interrumpible, but that's not the case anymore.
Right now, either we adapt it to use plpython2u or plpython3u depending
on what's available, or we just remove it since upstream was fixed
several years ago. It's simpler to remove it
This introduces a couple of changes in the parameter interpretation:
copy_from_minimum_input_speed is now the total number of bytes in the interval
(so we could multiply the currently configured values by the interval to preserve min. throughput.
the minimum speed is now inclusive (only smaller values will cause an exception).
* Tests: Work with re-runs
* Test: Adapt copyfrom test
- Reduce timeout from 10 to 1 ms.
- Increase the size of the file from 6 to 2016 rows.
* Use node-cartodb-psql 0.14.0
Ref: https://github.com/CartoDB/CartoDB-SQL-API/pull/612
