cartodb/CartoDB-SQL-API
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,194 Commits 1 Branch 118 Tags 9.7 MiB
3384719ea9
Commit Graph

3194 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rafa de la Torre
3384719ea9 Add a markdown explaining how the logger works 2020-08-03 16:19:21 +02:00
Alberto Asuero
b28835ff56 Adding docker ressources to .gitignore 2020-07-28 09:00:27 +00:00
Daniel G. Aubert
ad2e37e89e
Merge pull request #666 from CartoDB/feature/ch91877/remove-log-aggregation-in-metro 
Adapt logger output to the new standard for the entire platform
 2020-07-23 13:44:24 +02:00
Daniel García Aubert
8e0366c1ed Missing new middleware 2020-07-23 12:19:40 +02:00
Daniel García Aubert
6857a892f7 Log request with the username bound 2020-07-23 12:14:39 +02:00
Daniel García Aubert
1eae4ee41f Log stats with the username bound 2020-07-23 11:58:52 +02:00
Daniel García Aubert
df604adc2a Add request duration in milliseconds 2020-07-23 11:37:00 +02:00
Daniel García Aubert
b7ecea3201 Rename 'message' key by 'event_message' 2020-07-23 10:32:58 +02:00
Daniel García Aubert
1ae9a66f85 Adapt logger output to the new standard for the entire platform 2020-07-22 18:29:05 +02:00
Daniel G. Aubert
36fd4a85c1
Merge pull request #663 from CartoDB/dgaubert/ch87185/log-system-revamp-sql-api 
Log system revamp
 2020-07-06 13:59:59 +02:00
Daniel García Aubert
b611b85604 Missing header 2020-07-06 11:48:10 +02:00
Daniel García Aubert
84a6432e42 Handle error coming from database, such as query statement timeout 2020-07-01 20:59:21 +02:00
Daniel García Aubert
5e0cbcb879 For copyto endpoint, log on custom event to be able able to log all the gathered info even when error 2020-07-01 20:23:16 +02:00
Daniel García Aubert
65640a4c77 Do not use logger in stream-copy module 2020-07-01 17:38:35 +02:00
Daniel García Aubert
dc6e28097f Use named params for controllers 2020-07-01 12:32:51 +02:00
Daniel García Aubert
c4db741028 Move down init-logger and profiler middlewares to controllers 2020-07-01 12:29:51 +02:00
Daniel García Aubert
72eb0379e3 Rename 2020-07-01 12:19:19 +02:00
Daniel García Aubert
c31f4c9c25 Use the right port for sql-api prometheus metrics 2020-07-01 10:54:11 +02:00
Daniel García Aubert
d3a1974569 Lint 2020-07-01 10:53:44 +02:00
Daniel García Aubert
c1e02c2c95 Use the latest version of metro 2020-06-30 19:41:36 +02:00
Daniel García Aubert
c1ed382949 Merge branch 'master' into dgaubert/ch87185/log-system-revamp-sql-api 2020-06-30 17:50:10 +02:00
Daniel García Aubert
762a240890 Breaking changes: 
- Log system revamp:
  - Logs to stdout, disabled while testing
  - Use header `X-Request-Id`, or create a new `uuid` when no present, to identyfy log entries
  - Be able to set log level from env variable `LOG_LEVEL`, useful while testing: `LOG_LEVEL=info npm test`; even more human-readable: `LOG_LEVEL=info npm t | ./node_modules/.bin/pino-pretty`
  - Be able to reduce the footprint in the final log file depending on the environment
  - Use one logger for every service: Queries, Batch Queries (Jobs), and Data Ingestion (CopyTo/CopyFrom)
  - Stop using headers such as: `X-SQL-API-Log`, `X-SQL-API-Profiler`, and `X-SQL-API-Errors` as a way to log info.
  - Be able to tag requests with labels as an easier way to provide business metrics
  - Metro: Add log-collector utility (`metro`), it will be moved to its own repository. Attaching it here fro development purposes. Try it with the following command `LOG_LEVEL=info npm t | node metro`
  - Metro: Creates `metrics-collector.js` a stream to update Prometheus' counters and histograms and exposes them via Express' app (`:9145/metrics`). Use the ones defined in `grok_exporter`

Announcements:
- Profiler is always set. No need to check its existence anymore
- Unify profiler usage for every endpoint

Bug fixes:
- Avoid hung requests while fetching user identifier
 2020-06-30 17:42:59 +02:00
Álvaro
5caba983b1
Merge pull request #660 from CartoDB/clientHeaders 
respond with user-id and client in the headers
 2020-06-23 17:56:20 +02:00
Álvaro Manera
3994deda1f change test description 2020-06-22 12:51:45 +02:00
Álvaro Manera
0046480231 move middleware to sql-router 2020-06-22 12:51:35 +02:00
Álvaro Manera
598e7615a5 missing return on error case 2020-06-22 12:50:48 +02:00
Álvaro Manera
2e914d5454 fix lint 2020-06-22 09:56:48 +02:00
Álvaro Manera
a4a100e65a respond with user-id and client in the headers 
added headers to the test-client callback to be able to check them
 2020-06-22 09:47:22 +02:00
Daniel García Aubert
29f5db47f8 Update carto-package.json 2020-06-05 14:16:00 +02:00
Daniel García Aubert
d2b8e629bf Release 6.0.0 2020-06-05 14:02:34 +02:00
Daniel G. Aubert
f3df9a60a8
Merge pull request #659 from CartoDB/update-deps 
Update deps
 2020-06-01 19:21:21 +02:00
Daniel García Aubert
c02d1ae5a2 Update NEWS 2020-06-01 12:23:53 +02:00
Daniel García Aubert
d2e4ac7bac Fix vulns 2020-06-01 12:07:14 +02:00
Daniel García Aubert
d45506554b Update yargs to version 15.3.1 2020-06-01 11:19:15 +02:00
Daniel García Aubert
e1f8e7fef1 Update mocha@7.2.0 2020-06-01 11:13:56 +02:00
Daniel García Aubert
882ac9c2bc Remove mkdirp as dep 2020-06-01 10:55:18 +02:00
Daniel García Aubert
6bd2460f56 Auto npm audit fixes 2020-06-01 10:54:01 +02:00
Daniel G. Aubert
e02079b99d
Merge pull request #656 from CartoDB/node-12 
Node 12
 2020-05-20 15:59:39 +02:00
Daniel García Aubert
1d320704ce Merge branch 'master' into node-12 2020-05-20 15:03:33 +02:00
Daniel García Aubert
89a2d2ce2f Update NEWS 2020-05-20 15:02:39 +02:00
Daniel García Aubert
57be7add68 Do not emit error as it closes the internal DB's connection stream and the Pool needs to keep it open to reuse the the connection 2020-05-19 09:23:48 +02:00
Daniel García Aubert
95a092bd83 Drop support Node.js 10 in CI 2020-05-18 11:41:50 +02:00
Daniel García Aubert
4c8b3e2c47 Support Node.js 12 in CI 2020-05-18 11:39:24 +02:00
Daniel García Aubert
0c92fcaf96 Node.js 12 support: 
- Update `gc-stats` to version 1.4.0
- Replace `zipfile` -> `adm-zip`
- Update `libxmljs` to version 0.19.7
- Update `sqlite` to version 4.2.0
- Adapted pool acquires to the new version of `cartodb-redis`
- Adapted test to use `adm-zip`
 2020-05-18 11:32:41 +02:00
Raúl Marín
4a5b0a589a
Merge pull request #655 from Algunenano/our_packages 
Travis: Give priority to CARTO custom packages
 2020-05-18 10:41:45 +02:00
Daniel García Aubert
7f1de39797 Upgrade redis-mpool to version 0.8.0 and cartodb-redis to version 3.0.0 2020-05-14 18:59:01 +02:00
Raúl Marín
a72bde1af7 Travis: Give priority to CARTO custom packages 2020-05-14 18:17:44 +02:00
Raúl Marín
5e611f8826
Merge pull request #654 from Algunenano/ogr_length 
Fix an ogr2ogr export issue when the temporal name is too big
 2020-05-14 12:47:17 +02:00
Raúl Marín
25f15dfc61 Fix an ogr2ogr export issue when the temporal name is too big 
If the final name is going to be too big, use the hash over the proposed
path to create an unique name that fits the requirements of path length
under Linux
 2020-05-11 18:35:35 +02:00
Raúl Marín
c529ac41ff
Merge pull request #652 from Algunenano/gdal23 
Gdal 2.3
 2020-04-17 11:04:12 +02:00