Simon Martín
bafcb8051f
StreamCopy getResults refactor
...
- changing get Result to getRowCount
- ensuring to and from independecy in StreamCopy
2018-06-21 13:39:33 +02:00
Simon Martín
2cecc54cb2
copy to: removing copyToStream dependecy
2018-06-21 13:21:23 +02:00
Simon Martín
e39a9b2bf7
copy to: manage error in 1 point
2018-06-21 13:13:37 +02:00
Simon Martín
1dac5355d1
copy to: remove unneeded done and removing it as a dependency
2018-06-21 13:13:16 +02:00
Simon Martín
42a94a3b6c
copy to: removing unnedeed client dependecy
2018-06-21 13:01:59 +02:00
Simon Martín
e6448a3255
copy to: cancelQuery to streamCopy
2018-06-21 13:00:57 +02:00
Simon Martín
39dfcb1f9b
copy to events is series
2018-06-21 12:50:50 +02:00
Simon Martín
5fbd6e8e3a
req on error is neccesary when connection is canceled
2018-06-21 12:49:06 +02:00
Simon Martín
3da32308ce
jshint happy
2018-06-21 12:30:49 +02:00
Simon Martín
6955d254f8
Merge branch 'master' into pgcopy-stream-refactor2-dgaubert
2018-06-21 12:19:16 +02:00
Simon Martín
3d30f1f762
trailing white spaces
2018-06-19 10:34:52 +02:00
Simon Martín
30402f2e89
remove unused statsClient
2018-06-19 10:04:17 +02:00
Simon Martín
7727a9d506
data ingestion logger created in controller and added to app
2018-06-18 18:48:11 +02:00
Simon Martín
32dca8dfdb
reopenFileStreams to parent class
2018-06-18 18:47:12 +02:00
Simon Martín
8f727fc15a
cdb-user in logs
2018-06-15 13:30:42 +02:00
Simon Martín
8a450a862c
recover the metrics header (needed for tests purposes)
2018-06-15 13:25:47 +02:00
Simon Martín
c7e01f2ed6
success in copy log
2018-06-14 18:40:06 +02:00
Simon Martín
9097cbe16e
adding log of the error when headers are already sent
2018-06-14 18:29:50 +02:00
Simon Martín
39e5395edc
change gzip to isGzip
2018-06-14 18:29:03 +02:00
Simon Martín
4cb7de7318
changing let for const
2018-06-14 10:51:05 +02:00
Simon Martín
b091a375fc
fix ensuring right validation error
2018-06-14 10:49:36 +02:00
Simon Martín
1f31b8e2ae
removing old copy logs
2018-06-14 10:45:23 +02:00
Daniel García Aubert
9299fe51ce
Add todo
2018-06-14 01:01:05 +02:00
Daniel García Aubert
8eab164e4e
Be more one liner
2018-06-14 00:54:03 +02:00
Daniel García Aubert
9c8d802f75
Chain response event callbacks after .pipe() in order to improve readability in copyto controller
2018-06-14 00:31:08 +02:00
Daniel García Aubert
508d688b9d
Use request 'close' event when the request is closed unexpectedly by the client
2018-06-14 00:16:06 +02:00
Daniel García Aubert
37a079363a
Do not extend from EventEmitter
2018-06-13 20:12:56 +02:00
Daniel García Aubert
bd8e4be596
Do not emit to regist the number of rows processed in copyfrom
2018-06-13 20:11:39 +02:00
Daniel García Aubert
5655f26fbd
Do not emit to regist the number of rows processed in copyto
2018-06-13 20:06:35 +02:00
Daniel García Aubert
eb4ba60ba0
Pass the error to the pg connection's callback
2018-06-13 19:56:56 +02:00
Daniel García Aubert
e6a331433a
Handle error when client disconnects in copyto requests
2018-06-13 19:54:34 +02:00
Daniel García Aubert
6aba1ea307
Handle error when sql injected in pgstream fails (e.g: unexisting table)
2018-06-13 19:52:53 +02:00
Daniel García Aubert
2792d95c41
Do not handle stream unpiping when it wasn't even created
2018-06-13 18:30:05 +02:00
Daniel García Aubert
b9971d94dd
Do not use the regular callback in streams events
2018-06-13 18:28:34 +02:00
Daniel García Aubert
7d1a0abadd
Preventing callbacks from accidentally being called twice
2018-06-13 18:26:58 +02:00
Eneko Lakasta
f5d56a2253
FIX add return to callback call
2018-06-13 13:17:01 +02:00
Simon Martín
270daae579
removing forgotten callback
2018-06-12 18:40:51 +02:00
Simon Martín
409273bafe
removing 2nd calback using events
2018-06-12 18:39:50 +02:00
Simon Martín
19aae3c40f
manage pgstrem error on stream copy class
2018-06-12 17:04:44 +02:00
Simon Martín
ba2f85421d
StreamCopy to class
2018-06-12 16:56:18 +02:00
Simon Martín
8a15b2f8d1
Merge branch 'master' into pgcopy-stream-refactor2
2018-06-12 15:21:28 +02:00
Simon Martín
4d99872975
return in next err
2018-06-12 15:18:28 +02:00
Simon Martín
b9a0fa78d2
adding metadataBackend to user middleware in copycontroller
2018-06-11 14:55:37 +02:00
Simon Martín
1174499dcf
Merge branch 'master' into pgcopy-stream
2018-06-11 14:38:52 +02:00
Simon Martín
df63e0eedb
adding log of the error when headers are already sent
2018-06-11 13:19:12 +02:00
Simon Martín
2995676adb
let to const
2018-06-11 12:56:16 +02:00
Simon Martín
4fab96cb3a
rename gzip to isGzip
2018-06-11 12:55:30 +02:00
Simon Martín
87fd9db1fe
remove uneeded comment
2018-06-11 12:43:35 +02:00
Simon Martín
2581ddbfb4
better formatting
2018-06-08 17:09:28 +02:00
Simon Martín
b63806f8de
cleaning stream copy
2018-06-08 17:04:28 +02:00
Simon Martín
6a9e7d7c14
forgotten )
2018-06-08 17:04:10 +02:00
Simon Martín
7266b3c708
one logger
2018-06-08 17:02:31 +02:00
Simon Martín
014f0a1491
refactor of copy to PoC
2018-06-08 16:58:32 +02:00
Simon Martín
f7454228c6
refactor of copy from PoC
2018-06-08 16:50:12 +02:00
Simon Martín
e259a51623
unifiyng to see 'the forest'
2018-06-08 15:03:21 +02:00
Simon Martín
8e8ccc9c67
removing old metrics in copy
2018-06-08 13:33:17 +02:00
Simon Martín
c62ef15dda
using StreamCopy in controller
2018-06-08 11:15:23 +02:00
Simon Martín
732c229e63
creating StreamCopy class
2018-06-08 11:09:51 +02:00
Simon Martín
2bcea0484a
rename BunyanLogger to Logger
2018-06-08 10:59:34 +02:00
Eneko Lakasta
440ba8c840
make function checkUserExists node callback pattern compilant
2018-06-06 15:48:22 +02:00
Eneko Lakasta
340e55ea46
reduce error info
2018-06-06 15:23:53 +02:00
Eneko Lakasta
ca8ae503ac
Merge branch 'master' into remove-auth-fallback
2018-06-05 16:19:36 +02:00
Eneko Lakasta
eab3d289b6
refactor authenticated to authorizationLevel
2018-06-05 13:21:56 +02:00
Eneko Lakasta
da08e42921
refactor forceToBeAuthenticated to forceToBeMaster
2018-06-05 13:16:36 +02:00
Simon Martín
66af518deb
jshint
2018-06-05 13:16:14 +02:00
Eneko Lakasta
0207b67d5f
refactor forceToBeAuthenticated to forceToBeMaster
2018-06-05 13:14:50 +02:00
Eneko Lakasta
a4292f08cf
refactor apikey to apikeyToken
2018-06-05 13:13:09 +02:00
Simon Martín
29d1fb1274
logging GzipSize
2018-06-05 13:02:14 +02:00
Simon Martín
02cc606be1
gzipSize support in metrics
2018-06-05 13:01:15 +02:00
Simon Martín
dd689d3568
log metrics only once
2018-06-05 12:36:36 +02:00
Simon Martín
550c736032
metrics in errors
2018-06-05 12:35:52 +02:00
Rafa de la Torre
d2c0e68a78
Merge pull request #497 from CartoDB/cancel-copy-to-client-disconnect
...
Cancel copy to upon client disconnect
2018-06-05 09:59:49 +02:00
Paul Ramsey
d6d9022c7f
Coerce format string to upper for better log consistency
2018-06-04 13:12:06 -07:00
Simon Martín
66f7ab45fe
release connection with error
2018-06-04 20:51:21 +02:00
Simon Martín
7b6056b799
using the correct errors in done
2018-06-04 20:36:16 +02:00
Rafa de la Torre
014ea8142b
A cleaner approach to the cancel command
2018-06-04 19:06:11 +02:00
Simon Martín
2f2dcfd762
fix copy format case
2018-06-04 18:15:28 +02:00
Simon Martín
a844b5d31d
jshint happy
2018-06-04 18:15:00 +02:00
Simon Martín
1cf7032c9a
adding user and date in copy logs
2018-06-04 18:08:34 +02:00
Rafa de la Torre
994e8a702b
Add callbacks to pg.connect
...
And call them to return connections to the pool.
2018-05-31 18:59:19 +02:00
Rafa de la Torre
b59ae1d057
Make sure the COPY TO query is cancelled
...
Issue a CancelRequest upon client disconnection, to make sure the COPY
TO query is cancelled and the connection/session is put back to the pg
pool.
2018-05-31 17:27:35 +02:00
Rafa de la Torre
332f7096d3
Listen to response events (on behalf of @oleurud)
2018-05-31 17:06:19 +02:00
Rafa de la Torre
0eab03a7e7
Add a more informative message
2018-05-31 16:41:22 +02:00
Eneko Lakasta
46e3a87f41
add some auth TODOs
2018-05-30 18:15:35 +02:00
Eneko Lakasta
cf8bf6e5e6
get DB configuration from job instead from metadata in Job Canceller
2018-05-30 17:36:55 +02:00
Simon Martín
310f652ae4
send CopyFail when user close connection
2018-05-30 12:59:49 +02:00
Eneko Lakasta
7764975c09
please jshint: remove unnecessary require (underscore)
2018-05-30 12:08:04 +02:00
Simon Martín
a083eb909c
fix req events
2018-05-29 16:19:53 +02:00
Eneko Lakasta
fa5a99211c
check user exists in user middleware
...
This way, we keep sending a 404 error if the user does not exist.
2018-05-29 13:23:50 +02:00
Simon Martín
008fad3d13
undo 'ensuring data ingestion log'
2018-05-29 12:45:57 +02:00
Eneko Lakasta
ebe04d04ad
refactor user database service. Get proper DB config based on auth type: oauth vs apikey
...
Oauth uses only master api key configuration
Api key uses master and the provided api key configurations
Also move default api key fallback to this service
2018-05-28 15:53:51 +02:00
Simon Martín
e092647246
ensuring data ingestion log
2018-05-28 11:25:09 +02:00
Simon Martín
7d32ae293f
rename buyan_logger file name
2018-05-28 11:18:30 +02:00
Simon Martín
3cf28bb7ff
jshint happy
2018-05-25 18:50:56 +02:00
Simon Martín
22caa049ee
StreamCopyMetrics
2018-05-25 18:47:41 +02:00
Simon Martín
b31984cbc6
simplify response
2018-05-25 17:57:40 +02:00
Simon Martín
1fa5afd759
unify error handler
2018-05-25 17:50:59 +02:00
Simon Martín
6c3f9cf1d3
simplify controllers
2018-05-25 17:42:30 +02:00
Eneko Lakasta
d3b6ebd260
Fallback to default api key if no api key is provided
...
What happens with oauth authorization? This way we will always have an api_key and oauth will never trigger....
2018-05-25 17:28:56 +02:00
Eneko Lakasta
ec75227aa2
use apikeys instead of metadata for the database connections parameters
2018-05-25 17:23:24 +02:00
