- Log system revamp:
- Logs to stdout, disabled while testing
- Use header `X-Request-Id`, or create a new `uuid` when no present, to identyfy log entries
- Be able to set log level from env variable `LOG_LEVEL`, useful while testing: `LOG_LEVEL=info npm test`; even more human-readable: `LOG_LEVEL=info npm t | ./node_modules/.bin/pino-pretty`
- Be able to reduce the footprint in the final log file depending on the environment
- Use one logger for every service: Queries, Batch Queries (Jobs), and Data Ingestion (CopyTo/CopyFrom)
- Stop using headers such as: `X-SQL-API-Log`, `X-SQL-API-Profiler`, and `X-SQL-API-Errors` as a way to log info.
- Be able to tag requests with labels as an easier way to provide business metrics
- Metro: Add log-collector utility (`metro`), it will be moved to its own repository. Attaching it here fro development purposes. Try it with the following command `LOG_LEVEL=info npm t | node metro`
- Metro: Creates `metrics-collector.js` a stream to update Prometheus' counters and histograms and exposes them via Express' app (`:9145/metrics`). Use the ones defined in `grok_exporter`
Announcements:
- Profiler is always set. No need to check its existence anymore
- Unify profiler usage for every endpoint
Bug fixes:
- Avoid hung requests while fetching user identifier
* Create middleware and service for pubsub metrics
* Use pubsub middleware for all request
* Replace isEnabled with isDisabled to avoid negation
* Use new headers names
* Add acceptance and unit tests
* Remove commented log calls
* Remove only filters in tests
Remove typo
* Refactor service to ease integration test
* Fix interaction with query controller
* Use middleware at api-router and test all controllers
* Rename user middleware function
* Use sinon latest version
* Create middleware and service for pubsub metrics
* Use pubsub middleware for all request
* Replace isEnabled with isDisabled to avoid negation
* Use new headers names
* Add acceptance and unit tests
* Remove commented log calls
* Remove only filters in tests
Remove typo
* Refactor service to ease integration test
* Fix interaction with query controller
* Use middleware at api-router and test all controllers
* Rename user middleware function
* Use sinon latest version
* Fix tests
* Fix typos
* Checks if pubsub config exists to enable the service
* Fix typo
* Normalize headers values for pubsub
* Trim fields when normalizing
* Trim fields when normalizing
Set the default copy_from_max_post_size to exactly 2 GB, as the nginx
limitation has been removed for that endpoint. So, the application is
now in control and this keeps the code clear and clean.
You may be wondering: why it checks for 1.99 GB but it says the limit is
at 2 GB? The actual hard-limit is set in nginx, but we want the app to
catch this error condition and report it properly to the client,
hence the 0.5 % license.
