Commit Graph

97 Commits

Author SHA1 Message Date
Raul Ochoa
23254513e6 removes unneeded comma 2014-06-02 18:57:45 +02:00
Raul Ochoa
b3609696a3 requests associated with formats based on postgres expose a cancel method that will be called on client request abortion/cancelling so postgres can cancel ongoing queries 2014-06-02 14:48:38 +02:00
Raul Ochoa
537cb238c6 CDB-2081 Adds support for order_by through http query params. 2014-04-14 15:01:12 +02:00
Sandro Santilli
49ef1bc0c7 Stream JSON responses
Reduces memory usage for big datasets.
JIRA CDB-2600 #resolve
2014-04-03 15:44:53 +02:00
Sandro Santilli
8a953c116c Fix paging with queries starting with comments
Closes #144
Includes testcase
2014-03-27 12:48:39 +01:00
Sandro Santilli
2cbdc4c38c Add profiler support
Closes #142
2014-03-19 18:34:21 +01:00
Sandro Santilli
b0d0d4d073 Reduce work on aborted requests
Closes #129
Includes testcase
2014-03-19 13:30:29 +01:00
Sandro Santilli
65f31fd7c1 Add optional support for rollbar
Closes #150
Logs messages of severity ERROR or higher
2014-03-13 12:09:39 +01:00
Sandro Santilli
5c22b8ddaa Drop unused code 2014-01-31 15:47:33 +01:00
Luis Bosque
973b800fca Use db_port in ogr2ogr 2014-01-20 12:28:28 +01:00
Sandro Santilli
61cec80d4d Fix use of "SELECT .. INTO" with windowing params
Includes testcase. Closes #127
2014-01-09 17:56:09 +01:00
Sandro Santilli
87d35aa155 Generalize CartoDB username extraction, allowing for multiuser setups
Closes #124
2013-12-18 11:59:00 +01:00
Sandro Santilli
3397f2f3f9 Fix field types names lookup after PSQL model refactoring
Fixes missing .prj in shapefile export regression
Closes #122
2013-12-02 11:47:52 +01:00
Sandro Santilli
9389a04030 Add warnings and notices to JSON response. Closes #104. 2013-11-18 17:01:06 +01:00
Sandro Santilli
6ebeed04c2 Optionally read user-specific database_password from redis
Follows CartoDB-2.5.0 model. Includes testcase.
Closes #120 -- Jira ref CDB-870
2013-11-18 13:31:11 +01:00
Sandro Santilli
0d44139d8c Do not choke if ogr2ogr fails without on stderr 2013-11-18 11:54:00 +01:00
Sandro Santilli
7218c89b2d Refactor PSQL model to take full db config in the constructor
Closes #58, stops reading configuration variables itself
2013-11-18 11:42:43 +01:00
Sandro Santilli
03ea51a375 CartoDB redis interaction delegated to "cartodb-redis" module 2013-11-15 19:14:55 +01:00
Sandro Santilli
4a5c9b0eed Refactor OGR class interface to improve flexibility
The toOGR and toOGR_SingleFile function now take an "options"
object rather than a long list of arguments. This allows for
easier extension toward per-user database host and password
(see tickets #120 and #121)
2013-11-15 13:24:48 +01:00
javi
93da1c110e removed console.log refs #119 2013-11-09 16:56:58 +01:00
javi
370d45d8b6 fixed #119 2013-11-09 16:55:40 +01:00
Sandro Santilli
0139c8fb16 JSON format: correctly recognize "date" type columns
Closes #117 -- includes testcase
2013-11-06 11:43:56 +01:00
Sandro Santilli
45d54c47cc Add note about which GDAL version fixes the SRID detection bug 2013-11-06 08:06:29 +01:00
Sandro Santilli
8d2347af99 Make public PostgreSQL user name a configuration parameter
Closes #56.
Updates documentation and tests and example config files
2013-11-05 17:49:10 +01:00
Sandro Santilli
2abb136258 Set KML folder name to the requested filename.
Closes #115. Includes testcase.
2013-11-05 17:29:02 +01:00
Sandro Santilli
3c03099e8d Do not trust GDAL SRID detection for KML output.
Closes #116. Includes testcase.
2013-11-05 16:57:23 +01:00
Sandro Santilli
75bc36039d Remove done TODOs 2013-10-11 17:25:38 +02:00
Sandro Santilli
82f965fa57 Upgrade node-pg dependency to 2.6.2
Simplifies PSQL.quoteIdentifier and exposes PSQL.quoteLiteral
2013-10-02 12:32:24 +02:00
Sandro Santilli
e92530d6e5 Fix shapefile export for non-linestring results starting with NULL
Includes tests. Closes #111.
2013-10-02 12:12:09 +02:00
Sandro Santilli
0d84a704df Fix missing .prj file in shapefile format
Finds srid, when needed, with an additional query.
Closes #110.
Includes testcases.
2013-10-02 11:44:06 +02:00
Sandro Santilli
7f7ef682ac Improve recognition of non-standard field types names by db lookup
Closes #112.
Only looks up "geometry" and "raster" types for now, can be improved
over time.
2013-10-02 10:22:13 +02:00
Sandro Santilli
1ddec5fdd1 Add note about why we want to skip querying geometry_columns 2013-09-30 10:43:32 +02:00
Sandro Santilli
742936b2b8 Honour skipfields in JSON schema response. Closes #109 2013-09-26 13:26:45 +02:00
Sandro Santilli
a0ff7ee10c Remove last bit of hard-coded format names from app controller 2013-09-05 17:10:48 +02:00
Sandro Santilli
f7bee2345d Report boolean values as of boolean type, not string. Closes #106 2013-08-22 13:45:20 +02:00
Sandro Santilli
23d092f768 Use float parsing for int8 too
This makes us ready to upgrade to node-pg 2.x
2013-07-22 18:22:30 +02:00
Sandro Santilli
46b7f7a309 Parse all numbers as floats. Closes #100 2013-07-09 21:51:40 +02:00
Sandro Santilli
ae502335c3 Set "numeric" type to "number" 2013-07-09 21:51:08 +02:00
Sandro Santilli
ba28496127 Add support for "text" datatypes in json output format 2013-07-04 16:04:13 +02:00
javi
78a80e6c82 added string type 2013-07-04 15:47:09 +02:00
Sandro Santilli
5e379012a8 Add support for CTE in sql windowing, add unit tests
This is still an undocumented feature, but as long as it's present
and used (by cartodb UI) better tested than broken...

NOTE: more tests are needed for CTE and RETURNING queries
2013-06-17 16:36:36 +02:00
Sandro Santilli
88f1d33b42 Add "fields" member in JSON return. Closes #97 2013-06-14 18:36:05 +02:00
Luis Bosque
80452ef6f4 Merge branch 'release/staging' into develop 2013-05-30 11:17:10 +02:00
Luis Bosque
12afc3c88f In OAuth requests, get req.path
Using req.route.path breaks OAuth whtn used with regex routes. Instead
of this we need to use req.path because the path is already expanded
2013-05-29 17:36:41 +02:00
Sandro Santilli
ffafe814ef Do not hang up to queued request on ogr generation failure 2013-05-29 17:33:10 +02:00
Sandro Santilli
7f9463e7fd Fix filesystem access conflict among clustered processes 2013-05-29 16:48:20 +02:00
Sandro Santilli
510ac0533f Switch to using evented query model for postgresql
This enables formats for processing rows as they arrive from the
database, thus possibly reducing memory use. For a start the skip
fields are immediately removed from the result, rather than only
at the end.
2013-05-29 15:39:39 +02:00
javi
0be9e47930 Add arraybuffer format 2013-05-28 10:37:39 +02:00
Sandro Santilli
76705a3fd7 Refactor format objects to expose a common stream-able interface
See app/models/formats/README for info
2013-05-28 10:36:15 +02:00
Sandro Santilli
ab8494ab12 Survive postgresql connection losses. Closes #95. 2013-05-24 18:18:17 +02:00