cartodb4/observatory-extension
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into iss66-cast-geom-ids-to-text

Browse Source
This commit is contained in:
Andy Eschbacher 2016-05-17 11:09:10 -04:00
commit fd483b60f4
21 changed files with 15439 additions and 16223 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -2,3 +2,5 @@
src/pg/observatory--current--dev.sql src/pg/observatory--current--dev.sql
src/pg/observatory--dev--current.sql src/pg/observatory--dev--current.sql
src/pg/observatory--dev.sql src/pg/observatory--dev.sql
venv
*.pyc

112
doc/API.md
View File

@ -1,110 +1,10 @@
# Data Observatory Access # Data Observatory Documentation
This file is for reference purposes only. It is intended for tracking the Data Observatory API functions that should be displayed from the Docs site. Like all API doc, the golden source of the code will live in this repo. I will pull the list of files below into the docs for the output. This file is for reference purposes only. It is intended for tracking the Data Observatory functions that should be displayed from the live Docs site. Like all API doc, the golden source of this code will live in this observatory-extension repo, and will be edited in this repo.
## Documentation ## Documentation
## OBS_GetDemographicSnapshot * [Measures Functions](measures_functions.md)
* [Boundary Functions](boundary_functions.md)
The Demographic Snapshot API call enables you to collect demographic details around a point location. For example, you can take the coordinates of a bus stop and find the average population characteristics in that location. If you need help creating coordinates from addresses, [see our geocoding documentation]. * [Discovery Functions](discovery_functions.md)
* [Glossary](glossary.md)
Fields returned include information about income, education, transportation, race, and more. Not all fields will have information for every coordinate queried.
### API Syntax
```html
https://{{account name}}.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetDemographicSnapshot({{point geometry}})
```
#### Parameters
| Parameter | Description | Example |
|---|:-:|:-:|
| account name | The name of your CartoDB account where the Data Observatory has been enabled | example_account |
| point geometry | A WKB point geometry. You can use the helper function, CDB_LatLng to quickly generate one from latitude and longitude | CDB_LatLng(40.760410,-73.964242) |
#### Geographic Scope
The Demographic Snapshot API is available for the following countries:
* United States
### API Examples
__Get the Demographic Snapshot at Camp David__
```text
https://example_account.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetDemographicSnapshot(CDB_LatLng(39.648333, -77.465))
```
__Get the Demographic Snapshot in the Upper West Side__
```text
https://example_account.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetDemographicSnapshot(CDB_LatLng(40.80, -73.960))
```
### API Response
[Click to expand](https://gist.github.com/ohasselblad/c9e59a6e8da35728d0d81dfed131ed17)
### Available fields
The Demographic Snapshot contains a broad subset of demographic measures in the Data Observatory. Over 80 measurements are returned by a single API request.
__todo: turn this spreadsheet into a markdown table__
https://docs.google.com/spreadsheets/d/1U3Uajw_PsIy3_YgeujnJ7AiL2VREdT-ozdaulx07q2g/edit#gid=430723120
## OBS_GetSegmentationSnapshot
The Segmentation Snapshot API call enables you to determine the pre-calculated population segment for a location. For example, you can take the location of a store location and determine what classification of population exists around that location. If you need help creating coordinates from addresses, [see our geocoding documentation].
### API Syntax
```html
https://{{account name}}.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetSegmentationSnapshot({{point geometry}})
```
#### Parameters
| Parameter | Description | Example |
|---|:-:|:-:|
| account name | The name of your CartoDB account where the Data Observatory has been enabled | example_account |
| point geometry | A WKB point geometry. You can use the helper function, CDB_LatLng to quickly generate one from latitude and longitude | CDB_LatLng(40.760410,-73.964242) |
#### Geographic Scope
The Segmentation Snapshot API is available for the following countries:
* United States
### API Examples
__Get the Segmentation Snapshot around the MGM Grand__
```text
https://example_account.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetSegmentationSnapshot(CDB_LatLng(36.10222, -115.169516))
```
__Get the Segmentation Snapshot at CartoDB's NYC HQ__
```text
https://example_account.cartodb.com/api/v2/sql?q=SELECT * FROM
OBS_GetSegmentationSnapshot(CDB_LatLng(40.704512, -73.936669))
```
### API Response
__todo__
### Available segments
__todo__
### Methodology
Segmentation is a method that divides a target market into subgroups based on shared common traits. While we plan to make many different segmentation methods available, our first release includes a segmentation profile first defined in a paper, _Understanding America's Neighborhoods Using Uncertain Data from the American Community Survey: Output Data: US_tract_clusters_new_. [See here](http://www.tandfonline.com/doi/pdf/10.1080/00045608.2015.1052335) for further information on the work in that paper.

271
doc/boundary_functions.md Normal file
View File

@ -0,0 +1,271 @@
# Boundary Functions
Use the following functions to retrieve [Boundary](/cartodb-platform/dataobservatory/overview/#boundary-data) data. Data ranges from small areas (e.g. US Census Block Groups) to large areas (e.g. Countries). You can access boundaries by point location lookup, bounding box lookup, direct ID access and several other methods described below.
## OBS_GetBoundariesByGeometry(polygon geometry, geometry_id text)
The ```OBS_GetBoundariesByGeometry(geometry, geometry_id)``` method returns a set of boundary geometries that intersect a supplied geometry. This can be used to find all boundaries that are within or overlap a bounding box. You have the ability to choose whether to retrieve all boundaries that intersect your supplied bounding box or only those that fall entirely inside of your bounding box.
#### Arguments
Name |Description
--- | ---
polygon | a bounding box or other WGS84 geometry
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from ('NULL' (default) gives most recent)
overlap_type (optional) | one of '[intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html)' (default), '[contains](http://postgis.net/docs/manual-2.2/ST_Contains.html)', or '[within](http://postgis.net/docs/manual-2.2/ST_Within.html)'.
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a boundary geometry (e.g., US Census tract boundaries)
geom_refs | a string identifier for the geometry (e.g., geoids of US Census tracts)
If geometries are not found for the requested `polygon`, `geometry_id`, `timespan`, or `overlap_type`, then null values are returned.
#### Example
Insert all Census Tracts from Lower Manhattan and nearby areas within the supplied bounding box to a table named `manhattan_census_tracts` which has columns `the_geom` (geometry) and `geoid` (text).
```sql
INSERT INTO manhattan_census_tracts(the_geom, geoid)
SELECT *
FROM OBS_GetBoundariesByGeometry(
ST_MakeEnvelope(-74.0251922607,40.6945658517,
-73.9651107788,40.7377626342,
4326),
'us.census.tiger.census_tract')
```
#### Errors
* If a geometry other than a point is passed as the first argument, an error is thrown: `Invalid geometry type (ST_Polygon), expecting 'ST_Point'`
* If an `overlap_type` other than the valid ones listed above is entered, then an error is thrown
## OBS_GetPointsByGeometry(polygon geometry, geometry_id text)
The ```OBS_GetPointsByGeometry(polygon, geometry_id)``` method returns point geometries and their geographical identifiers that intersect (or are contained by) a bounding box polygon and lie on the surface of a boundary corresponding to the boundary with same geographical identifiers (e.g., a point that is on a census tract with the same geoid). This is a useful alternative to ```OBS_GetBoundariesByGeometry``` listed above because it returns much less data for each location.
#### Arguments
Name |Description
--- | ---
polygon | a bounding box or other geometry
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of '[intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html)' (default), '[contains](http://postgis.net/docs/manual-2.2/ST_Contains.html)', or '[within](http://postgis.net/docs/manual-2.2/ST_Within.html)'.
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a point geometry on a boundary (e.g., a point that lies on a US Census tract)
geom_refs| a string identifier for the geometry (e.g., the geoid of a US Census tract)
If geometries are not found for the requested geometry, `geometry_id`, `timespan`, or `overlap_type`, then null values are returned.
#### Example
Insert points that lie on Census Tracts from Lower Manhattan and nearby areas within the supplied bounding box to a table named `manhattan_census_tracts` which has columns `the_geom` (geometry) and `geoid` (text).
```sql
INSERT INTO manhattan_census_tract_points(the_geom, geoid)
SELECT *
FROM OBS_GetPointsByGeometry(
ST_MakeEnvelope(-74.0251922607,40.6945658517,
-73.9651107788,40.7377626342,
4326),
'us.census.tiger.census_tract')
```
#### Errors
* If a geometry other than a point is passed as the first argument, an error is thrown: `Invalid geometry type (ST_Point), expecting 'ST_MultiPolygon' or 'ST_Polygon'`
## OBS_GetBoundary(point_geometry, boundary_id)
The ```OBS_GetBoundary(point_geometry, boundary_id)``` method returns a boundary geometry defined as overlapping the point geometry and from the desired boundary set (e.g. Census Tracts). See the [Boundary ID glossary table below](below). This is a useful method for performing aggregations of points.
#### Arguments
Name | Description
--- | ---
point_geometry | a WGS84 polygon geometry (the_geom)
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
A boundary geometry. If no value is found at the requested `boundary_id` or `timespan`, a null value is returned.
Value | Description
--- | ---
geom | WKB geometry
#### Example
Overwrite a point geometry with a boundary geometry that contains it in your table
```SQL
UPDATE tablename
SET the_geom = OBS_GetBoundary(the_geom, 'us.census.tiger.block_group')
```
#### Errors
* If a geometry other than a point is passed, an error is thrown: `Invalid geometry type (ST_Line), expecting 'ST_Point'`
## OBS_GetBoundaryId(point_geometry, boundary_id)
The ```OBS_GetBoundaryId(point_geometry, boundary_id)``` returns a unique geometry_id for the boundary geometry that contains a given point geometry. See the [Boundary ID glossary table below](below). The method can be combined with ```OBS_GetBoundaryById(geometry_id)``` to create a point aggregation workflow.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
A TEXT boundary geometry id. If no value is found at the requested `boundary_id` or `timespan`, a null value is returned.
Value | Description
--- | ---
geometry_id | a string identifier of a geometry in the Boundaries
#### Example
Write the US Census block group geoid that contains the point geometry for every row as a new column in your table.
```SQL
UPDATE tablename
SET geometry_id = OBS_GetBoundaryId(the_geom, 'us.census.tiger.block_group')
```
#### Errors
* If a geometry other than a point is passed, an error is thrown: `Invalid geometry type (ST_Line), expecting 'ST_Point'`
## OBS_GetBoundaryById(geometry_id, boundary_id)
The ```OBS_GetBoundaryById(geometry_id, boundary_id)``` returns the boundary geometry for a unique geometry_id. A geometry_id can be found using the ```OBS_GetBoundaryId(point_geometry, boundary_id)``` method described above.
#### Arguments
Name | Description
--- | ---
geometry_id | a string identifier for a Boundary geometry
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
A boundary geometry. If a geometry is not found for the requested `geometry_id`, `boundary_id`, or `timespan`, then a null value is returned.
Key | Description
--- | ---
geom | a WGS84 polygon geometry
#### Example
Use a table of `geometry_id`s (e.g., geoid from the U.S. Census) to select the unique boundaries that they correspond to.
```SQL
SELECT
OBS_GetBoundaryById(geometry_id, 'us.census.tiger.county') As the_geom,
geometry_id,
count(*)
FROM tablename
GROUP BY geometry_id
```
## OBS_GetBoundariesByPointAndRadius(point geometry, radius numeric, boundary_id text)
The ```OBS_GetBoundariesByPointAndRadius(point, radius, boundary_id)``` method returns boundary geometries and their geographical identifiers that intersect (or are contained by) a circle centered on a point with a radius.
#### Arguments
Name |Description
--- | ---
point | a WGS84 point geometry
radius | a radius (in meters) from the center point
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of '[intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html)' (default), '[contains](http://postgis.net/docs/manual-2.2/ST_Contains.html)', or '[within](http://postgis.net/docs/manual-2.2/ST_Within.html)'.
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a boundary geometry (e.g., a US Census tract)
geom_refs| a string identifier for the geometry (e.g., the geoid of a US Census tract)
If geometries are not found for the requested point and radius, `geometry_id`, `timespan`, or `overlap_type`, then null values are returned.
#### Example
Insert into table `denver_census_tracts` the census tract boundaries and geoids of census tracts which intersect within 10 miles of downtown Denver, Colorado.
```sql
INSERT INTO denver_census_tracts(the_geom, geoid)
SELECT *
FROM OBS_GetBoundariesByPointAndRadius(
CDB_LatLng(39.7392, -104.9903), -- Denver, Colorado
10000 * 1.609, -- 10 miles (10km * conversion to miles)
'us.census.tiger.census_tract')
```
#### Errors
* If a geometry other than a point is passed, an error is thrown. E.g., `Invalid geometry type (ST_Line), expecting 'ST_Point'`
## OBS_GetPointsByPointAndRadius(point geometry, radius numeric, boundary_id text)
The ```OBS_GetPointsByPointAndRadius(point, radius, boundary_id)``` method returns point geometries on boundaries (e.g., a point that lies on a Census tract) and their geographical identifiers that intersect (or are contained by) a circle centered on a point with a radius.
#### Arguments
Name |Description
--- | ---
point | a WGS84 point geometry
radius | radius (in meters)
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of '[intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html)' (default), '[contains](http://postgis.net/docs/manual-2.2/ST_Contains.html)', or '[within](http://postgis.net/docs/manual-2.2/ST_Within.html)'.
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a point geometry (e.g., a point on a US Census tract)
geom_refs | a string identifier for the geometry (e.g., the geoid of a US Census tract)
If geometries are not found for the requested point and radius, `geometry_id`, `timespan`, or `overlap_type`, then null values are returned.
#### Example
Insert into table `denver_census_tracts` points on US census tracts and their corresponding geoids for census tracts which intersect within 10 miles of downtown Denver, Colorado, USA.
```sql
INSERT INTO denver_census_tracts(the_geom, geoid)
SELECT *
FROM OBS_GetPointsByPointAndRadius(
CDB_LatLng(39.7392, -104.9903), -- Denver, Colorado
10000 * 1.609, -- 10 miles (10km * conversion to miles)
'us.census.tiger.census_tract')
```
#### Errors
* If a geometry other than a point is passed, an error is thrown. E.g., `Invalid geometry type (ST_Line), expecting 'ST_Point'`

58
doc/discovery_functions.md Normal file
View File

@ -0,0 +1,58 @@
# Discovery Functions
If you are using the [discovery methods](/cartodb-platform/dataobservatory/overview/#discovery-methods) from the Data Observatory, use the following functions to retrieve [boundary](/cartodb-platform/dataobservatory/overview/#boundary-data) and [measures](/cartodb-platform/dataobservatory/overview/#measures-data) data.
## OBS_Search(search_term)
Use arbitrary text to search all available measures
#### Arguments
Name | Description
--- | ---
search_term | a string to search for available measures
boundary_id | a string identifier for a boundary geometry (optional)
#### Returns
A TABLE containing the following properties
Key | Description
--- | ---
measure_id | the unique id of the measure for use with the ```OBS_GetMeasure``` function
name | the human readable name of the measure
description | a brief description of the measure
aggregate_type | **sum** are raw count values, **median** are statistical medians, **average** are statistical averages, **undefined** other (e.g. an index value)
sources | where the data came from (e.g. US Census Bureau)
#### Example
```SQL
SELECT * FROM OBS_Search('inequality')
```
## OBS_GetAvailableBoundaries(point_geometry)
Returns available `boundary_id`s at a given point geometry.
#### Arguments
Name | Description
--- | ---
point_geometry | a WGS84 point geometry (e.g. the_geom)
#### Returns
A TABLE containing the following properties
Key | Description
--- | ---
boundary_id | a boundary identifier from the [boundary ID glossary](/cartodb-platform/dataobservatory/glossary/#boundary-ids)
description | a brief description of the boundary dataset
timespan | the timespan attached the boundary. this does not mean that the boundary is invalid outside of the timespan, but is the explicit timespan published with the geometry.
#### Example
```SQL
SELECT * FROM OBS_GetAvailableBoundaries(CDB_LatLng(40.7, -73.9))
```

126
doc/glossary.md Normal file
View File

@ -0,0 +1,126 @@
# Glossary
A list of boundary ids and measure_names for Data Observatory functions. For US based boundaries, the Shoreline Clipped version provides a high-quality shoreline clipping for mapping uses.
## Boundary IDs
Boundary name | Boundary ID | Shoreline Clipped Boundary ID
--------------------- | --------------------- | ---
US States | us.census.tiger.state | us.census.tiger.state_clipped
US County | us.census.tiger.county | us.census.tiger.county_clipped
US Census Zip Code Tabulation Areas | us.census.tiger.zcta5 | us.census.tiger.zcta5_clipped
US Census Tracts | us.census.tiger.census_tract | us.census.tiger.census_tract_clipped
US Elementary School District | us.census.tiger.school_district_elementary | us.census.tiger.school_district_elementary_clipped
US Secondary School District | us.census.tiger.school_district_secondary | us.census.tiger.school_district_secondary_clipped
US Unified School District | us.census.tiger.school_district_unified | us.census.tiger.school_district_unified_clipped
US Congressional Districts | us.census.tiger.congressional_district | us.census.tiger.congressional_district_clipped
US Census Blocks | us.census.tiger.block | us.census.tiger.block_clipped
US Census Block Groups | us.census.tiger.block_group | us.census.tiger.block_group_clipped
US Census PUMAs | us.census.tiger.puma | us.census.tiger.puma_clipped
US Incorporated Places | us.census.tiger.place | us.census.tiger.place_clipped
ES Sección Censal | es.ine.geom |
Regions (First-level Administrative) | whosonfirst.wof_region_geom |
Continents | whosonfirst.wof_continent_geom |
Countries | whosonfirst.wof_country_geom |
Marine Areas | whosonfirst.wof_marinearea_geom |
Disputed Areas | whosonfirst.wof_disputed_geom |
## OBS_GetUSCensusMeasure Names Table
This list contains human readable names accepted in the ```OBS_GetUSCensusMeasure``` function. For the more comprehensive list of columns available to the ```OBS_GetMeasure``` function, see the [Data Observatory Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf).
Measure name | Measure description
------------------------ | --------------------
Male Population | The number of people within each geography who are male.
Female Population | The number of people within each geography who are female.
Median Age | The median age of all people in a given geographic area.
Total Population | The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.
Population not Hispanic | The number of people not identifying as Hispanic or Latino in each geography.
White Population | The number of people identifying as white, non-Hispanic in each geography.
Black or African American Population | The number of people identifying as black or African American, non-Hispanic in each geography.
American Indian and Alaska Native Population | The number of people identifying as American Indian or Alaska native in each geography.
Asian Population | The number of people identifying as Asian, non-Hispanic in each geography.
Other Race population | The number of people identifying as another race in each geography.
Two or more races population | The number of people identifying as two or more races in each geography.
Hispanic Population | The number of people identifying as Hispanic or Latino in each geography.
Not a U.S. Citizen Population | The number of people within each geography who indicated that they are not U.S. citizens.
Workers over the Age of 16 | The number of people in each geography who work. Workers include those employed at private for-profit companies, the self-employed, government workers and non-profit employees.
Commuters by Car, Truck, or Van | The number of workers age 16 years and over within a geographic area who primarily traveled to work by car, truck or van. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters who drove alone | The number of workers age 16 years and over within a geographic area who primarily traveled by car driving alone. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters by Carpool | The number of workers age 16 years and over within a geographic area who primarily traveled to work by carpool. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters by Bus | The number of workers age 16 years and over within a geographic area who primarily traveled to work by bus. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work. This is a subset of workers who commuted by public transport.
Commuters by Subway or Elevated | The number of workers age 16 years and over within a geographic area who primarily traveled to work by subway or elevated train. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work. This is a subset of workers who commuted by public transport.
Walked to Work | The number of workers age 16 years and over within a geographic area who primarily walked to work. This would mean that of any way of getting to work, they travelled the most distance walking.
Worked at Home | The count within a geographical area of workers over the age of 16 who worked at home.
Workers age 16 and over who do not work from home | The number of workers over the age of 16 who do not work from home in a geographic area.
Number of workers with less than 10 minute commute | The number of workers over the age of 16 who do not work from home and commute in less than 10 minutes in a geographic area.
Number of workers with a commute between 35 and 44 minutes | The number of workers over the age of 16 who do not work from home and commute in between 35 and 44 minutes in a geographic area.
Number of workers with a commute of over 60 minutes | The number of workers over the age of 16 who do not work from home and commute in over 60 minutes in a geographic area.
Aggregate travel time to work | The total number of minutes every worker over the age of 16 who did not work from home spent spent commuting to work in one day in a geographic area.
Commuters by Public Transportation | The number of workers age 16 years and over within a geographic area who primarily traveled to work by public transportation. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Number of workers with a commute between 10 and 14 minutes | The number of workers over the age of 16 who do not work from home and commute in between 10 and 14 minutes in a geographic area.
Number of workers with a commute between 15 and 19 minutes | The number of workers over the age of 16 who do not work from home and commute in between 15 and 19 minutes in a geographic area.
Number of workers with a commute between 20 and 24 minutes | The number of workers over the age of 16 who do not work from home and commute in between 20 and 24 minutes in a geographic area.
Number of workers with a commute between 25 and 29 minutes | The number of workers over the age of 16 who do not work from home and commute in between 25 and 29 minutes in a geographic area.
Number of workers with a commute between 30 and 34 minutes | The number of workers over the age of 16 who do not work from home and commute in between 30 and 34 minutes in a geographic area.
Number of workers with a commute between 45 and 59 minutes | The number of workers over the age of 16 who do not work from home and commute in between 45 and 59 minutes in a geographic area.
Children under 18 Years of Age | The number of people within each geography who are under 18 years of age.
Households | A count of the number of households in each geography. A household consists of one or more people who live in the same dwelling and also share at meals or living accommodation, and may consist of a single family or some other grouping of people.
Population 15 Years and Over | The number of people in a geographic area who are over the age of 15. This is used mostly as a denominator of marital status.
Never Married | The number of people in a geographic area who have never been married.
Currently married | The number of people in a geographic area who are currently married.
Married but separated | The number of people in a geographic area who are married but separated.
Widowed | The number of people in a geographic area who are widowed.
Divorced | The number of people in a geographic area who are divorced.
Population 3 Years and Over | The total number of people in each geography age 3 years and over. This denominator is mostly used to calculate rates of school enrollment.
Students Enrolled in School | The total number of people in each geography currently enrolled at any level of school, from nursery or pre-school to advanced post-graduate education. Only includes those over the age of 3.
Students Enrolled in Grades 1 to 4 | The total number of people in each geography currently enrolled in grades 1 through 4 inclusive. This corresponds roughly to elementary school.
Students Enrolled in Grades 5 to 8 | The total number of people in each geography currently enrolled in grades 5 through 8 inclusive. This corresponds roughly to middle school.
Students Enrolled in Grades 9 to 12 | The total number of people in each geography currently enrolled in grades 9 through 12 inclusive. This corresponds roughly to high school.
Students Enrolled as Undergraduate in College | The number of people in a geographic area who are enrolled in college at the undergraduate level. Enrollment refers to being registered or listed as a student in an educational program leading to a college degree. This may be a public school or college, a private school or college.
Population 25 Years and Over | The number of people in a geographic area who are over the age of 25. This is used mostly as a denominator of educational attainment.
Population Completed High School | The number of people in a geographic area over the age of 25 who completed high school, and did not complete a more advanced degree.
Population completed less than one year of college, no degree | The number of people in a geographic area over the age of 25 who attended college for less than one year and no further.
Population completed more than one year of college, no degree | The number of people in a geographic area over the age of 25 who attended college for more than one year but did not obtain a degree.
Population Completed Associate's Degree | The number of people in a geographic area over the age of 25 who obtained a associate's degree, and did not complete a more advanced degree.
Population Completed Bachelor's Degree | The number of people in a geographic area over the age of 25 who obtained a bachelor's degree, and did not complete a more advanced degree.
Population Completed Master's Degree | The number of people in a geographic area over the age of 25 who obtained a master's degree, but did not complete a more advanced degree.
Population 5 Years and Over | The number of people in a geographic area who are over the age of 5. This is primarily used as a denominator of measures of language spoken at home.
Speaks only English at Home | The number of people in a geographic area over age 5 who speak only English at home.
Speaks Spanish at Home | The number of people in a geographic area over age 5 who speak Spanish at home, possibly in addition to other languages.
Population for Whom Poverty Status Determined | The number of people in each geography who could be identified as either living in poverty or not. This should be used as the denominator when calculating poverty rates, as it excludes people for whom it was not possible to determine poverty.
Income In The Past 12 Months Below Poverty Level | The number of people in a geographic area who are part of a family (which could be just them as an individual) determined to be in poverty following the Office of Management and Budget's Directive 14. (https://www.census.gov/hhes/povmeas/methodology/ombdir14.html)
Households with income less than $10,000 | The number of households in a geographic area whose annual income was less than $10,000.
Households with income of $10,000 to $14,999 | The number of households in a geographic area whose annual income was between $10,000 and $14,999.
Households with income of $15,000 to $19,999 | The number of households in a geographic area whose annual income was between $15,000 and $19,999.
Households with income of $20,000 To $24,999 | The number of households in a geographic area whose annual income was between $20,000 and $24,999.
Households with income of $25,000 To $29,999 | The number of households in a geographic area whose annual income was between $20,000 and $24,999.
Households with income of $30,000 To $34,999 | The number of households in a geographic area whose annual income was between $30,000 and $34,999.
Households with income of $35,000 To $39,999 | The number of households in a geographic area whose annual income was between $35,000 and $39,999.
Households with income of $40,000 To $44,999 | The number of households in a geographic area whose annual income was between $40,000 and $44,999.
Households with income of $45,000 To $49,999 | The number of households in a geographic area whose annual income was between $45,000 and $49,999.
Households with income of $50,000 To $59,999 | The number of households in a geographic area whose annual income was between $50,000 and $59,999.
Households with income of $60,000 To $74,999 | The number of households in a geographic area whose annual income was between $60,000 and $74,999.
Households with income of $75,000 To $99,999 | The number of households in a geographic area whose annual income was between $75,000 and $99,999.
Households with income of $100,000 To $124,999 | The number of households in a geographic area whose annual income was between $100,000 and $124,999.
Households with income of $125,000 To $149,999 | The number of households in a geographic area whose annual income was between $125,000 and $149,999.
Households with income of $150,000 To $199,999 | The number of households in a geographic area whose annual income was between $150,000 and $1999,999.
Households with income of $200,000 Or More | The number of households in a geographic area whose annual income was more than $200,000.
Median Household Income in the past 12 Months | Within a geographic area, the median income received by every household on a regular basis before payments for personal income taxes, social security, union dues, medicare deductions, etc. It includes income received from wages, salary, commissions, bonuses, and tips; self-employment income from own nonfarm or farm businesses, including proprietorships and partnerships; interest, dividends, net rental income, royalty income, or income from estates and trusts; Social Security or Railroad Retirement income; Supplemental Security Income (SSI); any cash public assistance or welfare payments from the state or local welfare office; retirement, survivor, or disability benefits; and any other sources of income received regularly such as Veterans' (VA) payments, unemployment and/or worker's compensation, child support, and alimony.
Population age 16 and over | The number of people in each geography who are age 16 or over.
Population in Labor Force | The number of people in each geography who are either in the civilian labor force or are members of the U.S. Armed Forces (people on active duty with the United States Army, Air Force, Navy, Marine Corps, or Coast Guard).
Population in Civilian Labor Force | The number of civilians 16 years and over in each geography who can be classified as either employed or unemployed below.
Employed Population | The number of civilians 16 years old and over in each geography who either (1) were at work, that is, those who did any work at all during the reference week as paid employees, worked in their own business or profession, worked on their own farm, or worked 15 hours or more as unpaid workers on a family farm or in a family business; or (2) were with a job but not at work, that is, those who did not work during the reference week but had jobs or businesses from which they were temporarily absent due to illness, bad weather, industrial dispute, vacation, or other personal reasons. Excluded from the employed are people whose only activity consisted of work around the house or unpaid volunteer work for religious, charitable, and similar organizations; also excluded are all institutionalized people and people on active duty in the United States Armed Forces.
Unemployed Population | The number of civilians in each geography who are 16 years old and over and are classified as unemployed.
Population in Armed Forces | The number of people in each geography who are members of the U.S. Armed Forces (people on active duty with the United States Army, Air Force, Navy, Marine Corps, or Coast Guard).
Population Not in Labor Force | The number of people in each geography who are 16 years old and over who are not classified as members of the labor force. This category consists mainly of students, homemakers, retired workers, seasonal workers interviewed in an off season who were not looking for work, institutionalized people, and people doing only incidental unpaid family work.
Housing Units | A count of housing units in each geography. A housing unit is a house, an apartment, a mobile home or trailer, a group of rooms, or a single room occupied as separate living quarters, or if vacant, intended for occupancy as separate living quarters.
Vacant Housing Units | The count of vacant housing units in a geographic area. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Vacant Housing Units for Rent | The count of vacant housing units in a geographic area that are for rent. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Vacant Housing Units for Sale | The count of vacant housing units in a geographic area that are for sale. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Median Rent | The median contract rent within a geographic area. The contract rent is the monthly rent agreed to or contracted for, regardless of any furnishings, utilities, fees, meals, or services that may be included. For vacant units, it is the monthly rent asked for the rental unit at the time of interview.
Percent of Household Income Spent on Rent | Within a geographic area, the median percentage of household income which was spent on gross rent. Gross rent is the amount of the contract rent plus the estimated average monthly cost of utilities (electricity, gas, water, sewer etc.) and fuels (oil, coal, wood, etc.) if these are paid by the renter. Household income is the sum of the income of all people 15 years and older living in the household.
Owner-occupied Housing Units valued at $1,000,000 or more. | The count of owner occupied housing units in a geographic area that are valued at $1,000,000 or more. Value is the respondent's estimate of how much the property (house and lot, mobile home and lot, or condominium unit) would sell for if it were for sale.
Owner-occupied Housing Units with a Mortgage | The count of housing units within a geographic area that are mortagaged. Mortgage refers to all forms of debt where the property is pledged as security for repayment of the debt, including deeds of trust, trust deed, contracts to purchase, land contracts, junior mortgages, and home equity loans.

163
doc/measures_functions.md Normal file
View File

@ -0,0 +1,163 @@
# Measures Functions
[Data Observatory Measures](/cartodb-platform/dataobservatory/overview/#measures-methods) are the numerical location data you can access. The measure Functions allow you to access individual measures to augment your own data or integrate in your analysis workflows. Measures are used by sending an identifier or a geometry (Point or Polygon) and receiving back a measure (an absolute value) for that location.
There are hundreds of Measures and the list is growing with each release. You can currently discover and learn about measures contained in the Data Observatory by downloading our [Data Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf).
We show here how you can access Measures through the CartoDB Editor. The same methods will work if you are using the CartoDB Platform to develop your application. We encourage you to use table modifying methods (UPDATE and INSERT) over dynamic methods (SELECT).
## OBS_GetUSCensusMeasure(point geometry, measure_name text);
The ```OBS_GetUSCensusMeasure(point, measure_name)``` function returns a measure based on a subset of the US Census variables at a point location. The ```OBS_GetUSCensusMeasure``` function is limited to only a subset of all Measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetMeasure``` function below.
#### Arguments
Name |Description
--- | ---
point | a WGS84 point geometry (the_geom)
measure_name | a human readable name of a US Census variable. The list of measure_names is [available in the glossary](/cartodb-platform/dataobservatory/glossary/#obsgetuscensusmeasure-names-table).
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'area' and response comes back as a rate per square kilometer. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](http://cartodb.github.io/bigmetadata/index.html) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a measure to an empty numeric column based on point locations in your table.
```SQL
UPDATE tablename
SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
## OBS_GetUSCensusMeasure(polygon geometry, measure_name text);
The ```OBS_GetUSCensusMeasure(point, measure_name)``` function returns a measure based on a subset of the US Census variables within a given polygon. The ```OBS_GetUSCensusMeasure``` function is limited to only a subset of all measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetUSCensusMeasure``` function below.
#### Arguments
Name |Description
--- | ---
polygon | a WGS84 polygon geometry (the_geom)
measure_name | a human readable string name of a US Census variable. The list of measure_names is [available in the glossary](/cartodb-platform/dataobservatory/glossary/#obsgetuscensusmeasure-names-table).
normalize | for measures that are **sums** (e.g. population) the default normalization is 'none' and response comes back as a raw value. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a measure to an empty numeric column based on polygons in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
## OBS_GetMeasure(point geometry, measure_id text);
The ```OBS_GetMeasure(point, measure_id)``` function returns any Data Observatory measure at a point location. You can browse all available Measures in the [Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf)).
#### Arguments
Name |Description
--- | ---
point | a WGS84 point geometry (the_geom)
measure_id | a measure identifier from the Data Observatory ([see available measures](https://cartodb.github.io/bigmetadata/observatory.pdf)). It is important to note that these are different than 'measure_name' used in the Census based functions above.
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'area' and response comes back as a rate per square kilometer. The other option is 'denominator', which will use the denominator specified in the [Data Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf). (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a measure to an empty numeric column based on point locations in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetMeasure(the_geom, 'us.census.acs.B08134006')
```
## OBS_GetMeasure(polygon geometry, measure_id text);
The ```OBS_GetMeasure(polygon, measure_id)``` function returns any Data Observatory measure calculated within a polygon.
#### Arguments
Name |Description
--- | ---
polygon_geometry | a WGS84 polygon geometry (the_geom)
measure_id | a measure identifier from the Data Observatory ([see available measures](https://cartodb.github.io/bigmetadata/observatory.pdf))
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'none' and response comes back as a raw value. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a measure to an empty column based on polygons in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetMeasure(the_geom, 'us.census.acs.B08134006')
```
#### Errors
* If an unrecognized normalization type is input, raise an error: `'Only valid inputs for "normalize" are "area" (default) and "denominator".`
## OBS_GetCategory(point geometry, category_id text);
The ```OBS_GetCategory(point, category_id)``` function returns any Data Observatory Category value at a point location. The Categories available are currently limited to Segmentation categories. See the Segmentation section of the [Catalog](https://cartodb.github.io/bigmetadata/observatory.pdf) for more detail.
#### Arguments
Name |Description
--- | ---
point | a WGS84 point geometry (the_geom)
category_id | a category identifier from the Data Observatory ([see available measures](https://cartodb.github.io/bigmetadata/observatory.pdf)).
#### Returns
A TEXT value
Key | Description
--- | ---
value | a text based category found at the supplied point
#### Example
Add the Category to an empty column text column based on point locations in your table
```SQL
UPDATE tablename
SET segmentation = OBS_GetCategory(the_geom, 'us.census.spielman_singleton_segments.X55')
```

702
doc/methods.md
View File

@ -1,702 +0,0 @@
# Measures
Measures Services allow users to access geospatial measures for analysis workflows. Measures are used by sending an identifier or a geometry (Point or Polygon) and receiving back a measure or absolute value for that location. Every measure contained in the Data Catalog can be accessed through the CartoDB Editor.
Below are the methods. For detailed information for accessing any measures, see the catalog here, [Catalog PDF](http://cartodb.github.io/bigmetadata/index.html)
## OBS_GetUSCensusMeasure(point_geometry, measure_name);
The ```OBS_GetUSCensusMeasure(point_geometry, measure_name)``` method returns a measure based on a subset of the US Census variables at a point location. The ```OBS_GetUSCensusMeasure``` method is limited to only a subset of all measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetMeasure``` method below.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
measure_name | a human readable string name of a US Census variable. The glossary of measure_names is [available below]('measure_name table').
normalize | for measures that are are **sums** (e.g., population) the default normalization is 'area' and response comes back as a rate per square kilometer. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](http://cartodb.github.io/bigmetadata/index.html) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value containing the following properties
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
Get a measure at a single point location
```SQL
SELECT OBS_GetUSCensusMeasure(CDB_LatLng(40.7, -73.9), 'Male Population')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetUSCensusMeasure(polygon_geometry, measure_name);
The ```OBS_GetUSCensusMeasure(point_geometry, measure_name)``` method returns a measure based on a subset of the US Census variables within a given polygon. The ```OBS_GetUSCensusMeasure``` method is limited to only a subset of all measures that are available in the Data Observatory, to access the full list, use the ```OBS_GetUSCensusMeasure``` method below.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 polygon geometry (the_geom)
measure_name | a human readable string name of a US Census variable. The glossary of measure_names is [available below]('measure_name table').
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'none' and response comes back as a raw value. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](http://cartodb.github.io/bigmetadata/index.html) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a Measure to an empty column based on polygons in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
```
Get a measure at a single polygon
```SQL
SELECT OBS_GetMeasure(
ST_Buffer(
CDB_LatLng(40.7, -73.9)::geography,
1000)::geometry,
'Male Population')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetUSCensusCategory(point_geometry, measure_name);
The ```OBS_GetUSCensusCategory(point_geometry, category_name)``` method returns a categorical measure based on a subset of the US Census variables at a point location. It requires a different function from ```OBS_GetUSCensusMeasure``` because this function will always return TEXT, whereas ```OBS_GetUSCensusMeasure``` will always returna NUMERIC value.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
measure_name | a human readable string name of a US Census variable. The glossary of measure_names is [available below]('measure_name table').
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A TEXT value containing the following properties
Key | Description
--- | ---
value | description of the segment at the location of the input geometry
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetUSCensusCategory(the_geom, 'Spielman Singleton Category 10')
```
Get a measure at a single point location
```SQL
SELECT OBS_GetUSCensusCategory(CDB_LatLng(40.7, -73.9), 'Spielman Singleton Category 10')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetMeasure(point_geometry, measure_id);
The ```OBS_GetMeasure(point_geometry, measure_id)``` method returns any Data Observatory measure at a point location.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
measure_id | a measure identifier from the Data Observatory ([see available measures](http://cartodb.github.io/bigmetadata/index.html))
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'area' and response comes back as a rate per square kilometer. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](http://cartodb.github.io/bigmetadata/index.html) and 'none' which will return a raw value. (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a Measure to an empty column based on point locations in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetMeasure(the_geom, 'us.census.acs.B08134006')
```
Get a measure at a single point location
```SQL
SELECT OBS_GetMeasure(
CDB_LatLng(40.7, -73.9),
'us.census.acs.B08134006')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetMeasure(polygon_geometry, measure_id);
The ```OBS_GetMeasure(polygon_geometry, measure_id)``` method returns any Data Observatory measure calculated within a polygon.
#### Arguments
Name |Description
--- | ---
polygon_geometry | a WGS84 polygon geometry (the_geom)
measure_id | a measure identifier from the Data Observatory ([see available measures](http://cartodb.github.io/bigmetadata/index.html))
normalize | for measures that are are **sums** (e.g. population) the default normalization is 'none' and response comes back as a raw value. Other options are 'denominator', which will use the denominator specified in the [Data Catalog](http://cartodb.github.io/bigmetadata/index.html) (optional)
boundary_id | source of geometries to pull measure from (e.g., 'us.census.tiger.census_tract')
time_span | time span of interest (e.g., 2010 - 2014)
#### Returns
A NUMERIC value
Key | Description
--- | ---
value | the raw or normalized measure
#### Example
Add a Measure to an empty column based on polygons in your table
```SQL
UPDATE tablename
SET local_male_population = OBS_GetMeasure(the_geom, 'us.census.acs.B08134006')
```
Get a measure within a polygon
```SQL
SELECT OBS_GetMeasure(
ST_Buffer(
CDB_LatLng(40.7, -73.9)::geography,
1000)::geometry,
'us.census.acs.B08134006')
```
<!--
Should add the SQL API call here too
-->
---
# Boundaries
## OBS_GetBoundary(point_geometry, boundary_id)
The ```OBS_GetBoundary(point_geometry, boundary_id)``` method returns a boundary geometry defined as overlapping the point geometry and from the desired boundary set (e.g. Census Tracts). See the [Boundary ID glossary table below](below). This is a useful method for performing aggregations of points.
#### Arguments
Name | Description
--- | ---
point_geometry | a WGS84 polygon geometry (the_geom)
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
Value | Description
--- | ---
geom | WKB geometry
#### Example
Overwrite a point geometry with a boundary geometry that contains it in your table
```SQL
UPDATE tablename
SET the_geom = OBS_GetBoundary(the_geom, 'us.census.tiger.block_group')
```
<!--
Should add the SQL API call here too
-->
## OBS_GetBoundaryId(point_geometry, boundary_id)
The ```OBS_GetBoundaryId(point_geometry, boundary_id)``` returns a unique geometry_id for the boundary geometry that contains a given point geometry. See the [Boundary ID glossary table below](below). The method can be combined with ```OBS_GetBoundaryById(geometry_id)``` to create a point aggregation workflow.
#### Arguments
Name |Description
--- | ---
point_geometry | a WGS84 point geometry (the_geom)
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
Value | Description
--- | ---
geometry_id | a string identifier of a geometry in the Boundaries
#### Example
Write the geometry_id that contains the point geometry for every row as a new column in your table
```SQL
UPDATE tablename
SET boundary_id = OBS_GetBoundaryId(the_geom, ' us.census.tiger.block_group')
```
## OBS_GetBoundaryById(geometry_id, boundary_id)
The ```OBS_GetBoundaryById(geometry_id, boundary_id)``` returns the boundary geometry for a unique geometry_id. A geometry_id can be found using the ```OBS_GetBoundaryId(point_geometry, boundary_id)``` method described above.
#### Arguments
Name | Description
--- | ---
geometry_id | a string identifier for a Boundary geometry
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
#### Returns
A table containing the following properties
Key | Description
--- | ---
geom | a WGS84 polygon geometry
#### Example
Use a table of geometry_id to select the unique boundaries. Useful with the ```Create Dataset from Query``` option in CartoDB.
```SQL
SELECT
OBS_GetBoundaryById(geometry_id, 'us.census.tiger.county') As the_geom,
geometry_id,
count(*)
FROM tablename
GROUP BY geometry_id
```
## OBS_GetBoundariesByGeometry(geometry, geometry_id)
The ```OBS_GetBoundariesByGeometry(geometry, geometry_id)``` method returns the boundary geometries and their geographical identifiers that intersect (or are contained by) a bounding box polygon.
#### Arguments
Name |Description
--- | ---
geometry | a bounding box
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of 'intersects' (default), 'contains', or 'within'. See [ST_Intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html), [ST_Contains](http://postgis.net/docs/manual-2.2/ST_Contains.html), or [ST_Within](http://postgis.net/docs/manual-2.2/ST_Within.html) for more
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a boundary geometry (e.g., US Census tracts)
geom_ref | a string identifier for the geometry (e.g., the geoid of a US Census tract)
#### Example
Get all Census Block Groups in Lower Manhattan (geoids beginning with `36061`) without getting Brooklyn or New Jersey into an existing table with a geometry column `the_geom` and text column `geoid`.
```sql
INSERT INTO lowermanhattan_census_tracts(the_geom, geoid)
SELECT the_geom, geom_ref
FROM OBS_GetBoundariesByGeometry(
ST_MakeEnvelope(-74.0251922607,40.6945658517,
-73.9651107788,40.7377626342,
4326),
'us.census.tiger.block_group')
WHERE geom_ref like '36061%'
```
#### API Example
Retrieve all Census tracts contained in a bounding box around Denver, CO as a JSON response:
```text
http://observatory.cartodb.com/api/v2/sql?q=SELECT%20*%20FROM%20OBS_GetBoundariesByGeometry(ST_MakeEnvelope(-105.4287704158,39.4600507935,-104.5089737248,40.0901569675,4326),%27%22us.census.tiger%22.census_tract%27,%272009%27,%27contains%27)
```
## OBS_GetBoundariesByPointAndRadius(geometry, radius, boundary_id)
The ```OBS_GetBoundariesByPointAndRadius(geometry, radius, boundary_id)``` method returns boundary geometries and their geographical identifiers that intersect (or are contained by) a circle centered on a point with a radius.
#### Arguments
Name |Description
--- | ---
geometry | a point geometry
radius | a radius (in meters) from the center point
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of 'intersects' (default), 'contains', or 'within'. See [ST_Intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html), [ST_Contains](http://postgis.net/docs/manual-2.2/ST_Contains.html), or [ST_Within](http://postgis.net/docs/manual-2.2/ST_Within.html) for more
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a boundary geometry (e.g., a US Census tract)
geom_ref | a string identifier for the geometry (e.g., the geoid of a US Census tract)
#### Example
Get Census tracts which intersect within 10 miles of Downtown, Colorado.
```sql
SELECT *
FROM OBS_GetBoundariesByPointAndRadius(
CDB_LatLng(39.7392, -104.9903), -- Denver, Colorado
10000 * 1.609, -- 10 miles (10km * conversion to miles)
'us.census.tiger.census_tract')
```
#### API Example
Retrieve all Census tracts contained in a bounding box around Denver, CO as a JSON response:
```text
http://observatory.cartodb.com/api/v2/sql?q=SELECT%20*%20FROM%20OBS_GetBoundariesByPointAndRadius(CDB_LatLng(39.7392,-104.9903),10000*1609),%27%22us.census.tiger%22.census_tract%27,%272009%27,%27contains%27)
```
## OBS_GetPointsByGeometry(geometry, geometry_id)
The ```OBS_GetPointsByGeometry(geometry, geometry_id)``` method returns point geometries and their geographical identifiers that intersect (or are contained by) a bounding box polygon and lie on the surface of a boundary corresponding to the boundary with same geographical identifiers (e.g., a point that is on a census tract with the same geoid).
#### Arguments
Name |Description
--- | ---
geometry | a bounding box
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of 'intersects' (default), 'contains', or 'within'. See [ST_Intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html), [ST_Contains](http://postgis.net/docs/manual-2.2/ST_Contains.html), or [ST_Within](http://postgis.net/docs/manual-2.2/ST_Within.html) for more
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a point geometry on a boundary (e.g., a point that lies on a US Census tract)
geom_ref | a string identifier for the geometry (e.g., the geoid of a US Census tract)
#### Example
Get points in all Census Tracts in Lower Manhattan (geoids beginning with `36061`) without getting Brooklyn or New Jersey
```sql
SELECT *
FROM OBS_GetPointsByGeometry(
ST_MakeEnvelope(-74.0251922607,40.6945658517,
-73.9651107788,40.7377626342,
4326),
'us.census.tiger.census_tract')
WHERE geom_ref like '36061%'
```
#### API Example
Retrieve all Census tracts contained in a bounding box around Denver, CO as a JSON response:
```text
http://observatory.cartodb.com/api/v2/sql?q=SELECT%20*%20FROM%20OBS_GetPointsByGeometry(ST_MakeEnvelope(-105.4287704158,39.4600507935,-104.5089737248,40.0901569675,4326),%27%22us.census.tiger%22.census_tract%27,%272009%27,%27contains%27)
```
## OBS_GetPointsByPointAndRadius(geometry, radius, boundary_id)
The ```OBS_GetPointsByPointAndRadius(geometry, radius, boundary_id)``` method returns point geometries on boundaries (e.g., a point that lies on a Census tract) and their geographical identifiers that intersect (or are contained by) a circle centered on a point with a radius.
#### Arguments
Name |Description
--- | ---
geometry | a point geometry
radius | radius (in meters)
geometry_id | a string identifier for a boundary geometry
timespan (optional) | year(s) to request from (`NULL` (default) gives most recent)
overlap_type (optional) | one of 'intersects' (default), 'contains', or 'within'. See [ST_Intersects](http://postgis.net/docs/manual-2.2/ST_Intersects.html), [ST_Contains](http://postgis.net/docs/manual-2.2/ST_Contains.html), or [ST_Within](http://postgis.net/docs/manual-2.2/ST_Within.html) for more
#### Returns
A table with the following columns:
Column Name | Description
--- | ---
the_geom | a point geometry (e.g., a point on a US Census tract)
geom_ref | a string identifier for the geometry (e.g., the geoid of a US Census tract)
#### Example
Get Census tracts which intersect within 10 miles of Downtown, Colorado.
```sql
SELECT *
FROM OBS_GetPointsByPointAndRadius(
CDB_LatLng(39.7392, -104.9903), -- Denver, Colorado
10000 * 1.609, -- 10 miles (10km * conversion to miles)
'us.census.tiger.census_tract')
```
#### API Example
Retrieve all Census tracts contained in a bounding box around Denver, CO as a JSON response:
```text
http://observatory.cartodb.com/api/v2/sql?q=SELECT%20*%20FROM%20OBS_GetPointsByPointAndRadius(CDB_LatLng(39.7392,-104.9903),10000*1609),%27%22us.census.tiger%22.census_tract%27,%272009%27,%27contains%27)
```
# Discovery
## OBS_Search(search_term)
Use arbitrary text to search all available Measures
#### Arguments
Name | Description
--- | ---
search_term | a string to search for available Measures
boundary_id | a string identifier for a Boundary geometry (optional)
#### Returns
Key | Description
--- | ---
measure_id | the unique id of the measure for use with the ```OBS_GetMeasure``` method
name | the human readable name of the measure
description | a brief description of the measure
aggregate_type | **sum** are raw count values, **median** are statistical medians, **average** are statistical averages, **undefined** other (e.g. an index value)
sources | where the data came from (e.g. US Census Bureau)
#### Example
```SQL
SELECT * FROM OBS_Search('inequality')
```
## OBS_GetAvailableBoundaries(point_geometry)
Returns available boundary_ids at a given point geometry.
#### Arguments
Name | Description
--- | ---
point_geometry | a WGS84 point geometry (e.g. the_geom)
#### Returns
Key | Description
--- | ---
boundary_id | a boundary identifier from the [Boundary ID glossary table below](below)
description | a brief description of the boundary dataset
timespan | the timespan attached the boundary. this does not mean that the boundary is invalid outside of the timespan, but is the explicit timespan published with the geometry.
#### Example
```SQL
SELECT * FROM OBS_GetAvailableBoundaries(CDB_LatLng(40.7, -73.9))
```
# Glossary
#### Boundary IDs
Boundary name | Boundary ID
--------------------- | ---
US Census Block Groups | us.census.tiger.block_group
US Census Tracts | us.census.tiger.census_tract
US States | us.census.tiger.state
US County | us.census.tiger.county
US Census Public Use Microdata Areas | us.census.tiger.puma
US Census Zip Code Tabulation Areas | us.census.tiger.zcta5
Unified School District | us.census.tiger.school_district_unified
US Congressional Districts | us.census.tiger.congressional_district
Elementary School District | us.census.tiger.school_district_elementary
Secondary School District | us.census.tiger.school_district_secondary
US Census Blocks | us.census.tiger.block
#### OBS_GetUSCensusMeasure names table
Below is a list of human readable names accepted in the ```OBS_GetUSCensusMeasure``` method. For the more comprehensive list of columns available to the ```OBS_GetMeasure``` method, see the [Data Catalog]
Measure name | Measure description
--------------------- | ---
Total Population | The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.
Male Population | The number of people within each geography who are male.
Female Population | The number of people within each geography who are female.
Median Age | The median age of all people in a given geographic area.
White Population | The number of people identifying as white, non-Hispanic in each geography.
Black or African American Population | The number of people identifying as black or African American, non-Hispanic in each geography.
Asian Population | The number of people identifying as Asian, non-Hispanic in each geography.
Hispanic Population | The number of people identifying as Hispanic or Latino in each geography.
American Indian and Alaska Native Population | The number of people identifying as American Indian or Alaska native in each geography.
Other Race population | The number of people identifying as another race in each geography
Two or more races population | The number of people identifying as two or more races in each geography
Population not Hispanic | The number of people not identifying as Hispanic or Latino in each geography.
Not a U.S. Citizen Population | The number of people within each geography who indicated that they are not U.S. citizens.
Workers over the Age of 16 | The number of people in each geography who work. Workers include those employed at private for-profit companies, the self-employed, government workers and non-profit employees.
Commuters by Car, Truck, or Van | The number of workers age 16 years and over within a geographic area who primarily traveled to work by car, truck or van. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters who drove alone | The number of workers age 16 years and over within a geographic area who primarily traveled by car driving alone. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters by Carpool | The number of workers age 16 years and over within a geographic area who primarily traveled to work by carpool. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters by Public Transportation | The number of workers age 16 years and over within a geographic area who primarily traveled to work by public transportation. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work.
Commuters by Bus | The number of workers age 16 years and over within a geographic area who primarily traveled to work by bus. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work. This is a subset of workers who commuted by public transport.
Commuters by Subway or Elevated | The number of workers age 16 years and over within a geographic area who primarily traveled to work by subway or elevated train. This is the principal mode of travel or type of conveyance, by distance rather than time, that the worker usually used to get from home to work. This is a subset of workers who commuted by public transport.
Walked to Work | The number of workers age 16 years and over within a geographic area who primarily walked to work. This would mean that of any way of getting to work, they travelled the most distance walking.
Worked at Home | The count within a geographical area of workers over the age of 16 who worked at home.
Children under 18 Years of Age | The number of people within each geography who are under 18 years of age.
Households | A count of the number of households in each geography. A household consists of one or more people who live in the same dwelling and also share at meals or living accommodation, and may consist of a single family or some other grouping of people.
Population 3 Years and Over | The total number of people in each geography age 3 years and over. This denominator is mostly used to calculate rates of school enrollment.
Students Enrolled in School | The total number of people in each geography currently enrolled at any level of school, from nursery or pre-school to advanced post-graduate education. Only includes those over the age of 3.
Students Enrolled in Grades 1 to 4 | The total number of people in each geography currently enrolled in grades 1 through 4 inclusive. This corresponds roughly to elementary school.
Students Enrolled in Grades 5 to 8 | The total number of people in each geography currently enrolled in grades 5 through 8 inclusive. This corresponds roughly to middle school.
Students Enrolled in Grades 9 to 12 | The total number of people in each geography currently enrolled in grades 9 through 12 inclusive. This corresponds roughly to high school.
Students Enrolled as Undergraduate in College | The number of people in a geographic area who are enrolled in college at the undergraduate level. Enrollment refers to being registered or listed as a student in an educational program leading to a college degree. This may be a public school or college, a private school or college.
Population 25 Years and Over | The number of people in a geographic area who are over the age of 25. This is used mostly as a denominator of educational attainment.
Population Completed High School | The number of people in a geographic area over the age of 25 who completed high school, and did not complete a more advanced degree.
Population completed less than one year of college, no degree | The number of people in a geographic area over the age of 25 who attended college for less than one year and no further.
Population completed more than one year of college, no degree | The number of people in a geographic area over the age of 25 who attended college for more than one year but did not obtain a degree
Population Completed Associates Degree | The number of people in a geographic area over the age of 25 who obtained a associates degree, and did not complete a more advanced degree.
Population Completed Bachelors Degree | The number of people in a geographic area over the age of 25 who obtained a bachelors degree, and did not complete a more advanced degree.
Population Completed Masters Degree | The number of people in a geographic area over the age of 25 who obtained a masters degree, but did not complete a more advanced degree.
Population 5 Years and Over | The number of people in a geographic area who are over the age of 5. This is primarily used as a denominator of measures of language spoken at home.
Speaks only English at Home | The number of people in a geographic area over age 5 who speak only English at home.
Speaks Spanish at Home | The number of people in a geographic area over age 5 who speak Spanish at home, possibly in addition to other languages.
Population for Whom Poverty Status Determined | The number of people in each geography who could be identified as either living in poverty or not. This should be used as the denominator when calculating poverty rates, as it excludes people for whom it was not possible to determine poverty.
Income In The Past 12 Months Below Poverty Level | The number of people in a geographic area who are part of a family (which could be just them as an individual) determined to be in poverty following the Office of Management and Budgets Directive 14. (https://www.census.gov/hhes/povmeas/methodology/ombdir14.html)
Median Household Income in the past 12 Months | Within a geographic area, the median income received by every household on a regular basis before payments for personal income taxes, social security, union dues, medicare deductions, etc. It includes income received from wages, salary, commissions, bonuses, and tips; self-employment income from own nonfarm or farm businesses, including proprietorships and partnerships; interest, dividends, net rental income, royalty income, or income from estates and trusts; Social Security or Railroad Retirement income; Supplemental Security Income (SSI); any cash public assistance or welfare payments from the state or local welfare office; retirement, survivor, or disability benefits; and any other sources of income received regularly such as Veterans (VA) payments, unemployment and/or workers compensation, child support, and alimony.
Gini Index |
Per Capita Income in the past 12 Months |
Housing Units | A count of housing units in each geography. A housing unit is a house, an apartment, a mobile home or trailer, a group of rooms, or a single room occupied as separate living quarters, or if vacant, intended for occupancy as separate living quarters.
Vacant Housing Units | The count of vacant housing units in a geographic area. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Vacant Housing Units for Rent | The count of vacant housing units in a geographic area that are for rent. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Vacant Housing Units for Sale | The count of vacant housing units in a geographic area that are for sale. A housing unit is vacant if no one is living in it at the time of enumeration, unless its occupants are only temporarily absent. Units temporarily occupied at the time of enumeration entirely by people who have a usual residence elsewhere are also classified as vacant.
Median Rent | The median contract rent within a geographic area. The contract rent is the monthly rent agreed to or contracted for, regardless of any furnishings, utilities, fees, meals, or services that may be included. For vacant units, it is the monthly rent asked for the rental unit at the time of interview.
Percent of Household Income Spent on Rent | Within a geographic area, the median percentage of household income which was spent on gross rent. Gross rent is the amount of the contract rent plus the estimated average monthly cost of utilities (electricity, gas, water, sewer etc.) and fuels (oil, coal, wood, etc.) if these are paid by the renter. Household income is the sum of the income of all people 15 years and older living in the household.
Owner-occupied Housing Units |
Owner-occupied Housing Units valued at $1,000,000 or more. | The count of owner occupied housing units in a geographic area that are valued at $1,000,000 or more. Value is the respondents estimate of how much the property (house and lot, mobile home and lot, or condominium unit) would sell for if it were for sale.
Owner-occupied Housing Units with a Mortgage | The count of housing units within a geographic area that are mortagaged. Mortgage refers to all forms of debt where the property is pledged as security for repayment of the debt, including deeds of trust, trust deed, contracts to purchase, land contracts, junior mortgages, and home equity loans.
Families with young children (under 6 years of age) |
Two-parent families with young children (under 6 years of age) |
Two-parent families, both parents in labor force with young children (under 6 years of age) |
Two-parent families, father only in labor force with young children (under 6 years of age) |
Two-parent families, mother only in labor force with young children (under 6 years of age) |
Two-parent families, neither parent in labor force with young children (under 6 years of age) |
One-parent families with young children (under 6 years of age) |
One-parent families, father, with young children (under 6 years of age) |
Men age 45 to 64 (middle aged) | 0
Men age 45 to 49 |
Men age 50 to 54 |
Men age 55 to 59 |
Men age 60 to 61 |
Men age 62 to 64 |
Black Men age 45 to 54 |
Black Men age 55 to 64 |
Hispanic Men age 45 to 54 |
Hispanic Men age 55 to 64 |
White Men age 45 to 54 |
White Men age 55 to 64 |
Asian Men age 45 to 54 |
Asian Men age 55 to 64 |
Men age 45 to 64 who attained less than a 9th grade education |
Men age 45 to 64 who attained between 9th and 12th grade, no diploma |
Men age 45 to 64 who completed high school or obtained GED |
Men age 45 to 64 who completed some college, no degree |
Men age 45 to 64 who obtained an associates degree |
Men age 45 to 64 who obtained a bachelors degree |
Men age 45 to 64 who obtained a graduate or professional degree |
One-parent families, father in labor force, with young children (under 6 years of age) |
Population 15 Years and Over | The number of people in a geographic area who are over the age of 15. This is used mostly as a denominator of marital status.
Never Married | The number of people in a geographic area who have never been married.
Currently married | The number of people in a geographic area who are currently married
Married but separated | The number of people in a geographic area who are married but separated
Widowed | The number of people in a geographic area who are widowed
Divorced | The number of people in a geographic area who are divorced
Workers age 16 and over who do not work from home | The number of workers over the age of 16 who do not work from home in a geographic area
Number of workers with less than 10 minute commute | The number of workers over the age of 16 who do not work from home and commute in less than 10 minutes in a geographic area
Number of workers with a commute between 10 and 14 minutes | The number of workers over the age of 16 who do not work from home and commute in between 10 and 14 minutes in a geographic area
Number of workers with a commute between 15 and 19 minutes | The number of workers over the age of 16 who do not work from home and commute in between 15 and 19 minutes in a geographic area
Number of workers with a commute between 20 and 24 minutes | The number of workers over the age of 16 who do not work from home and commute in between 20 and 24 minutes in a geographic area
Number of workers with a commute between 25 and 29 minutes | The number of workers over the age of 16 who do not work from home and commute in between 25 and 29 minutes in a geographic area
Number of workers with a commute between 30 and 34 minutes | The number of workers over the age of 16 who do not work from home and commute in between 30 and 34 minutes in a geographic area
Number of workers with a commute between 35 and 44 minutes | The number of workers over the age of 16 who do not work from home and commute in between 35 and 44 minutes in a geographic area
Number of workers with a commute between 45 and 59 minutes | The number of workers over the age of 16 who do not work from home and commute in between 45 and 59 minutes in a geographic area
Number of workers with a commute of over 60 minutes | The number of workers over the age of 16 who do not work from home and commute in over 60 minutes in a geographic area
Aggregate travel time to work | The total number of minutes every worker over the age of 16 who did not work from home spent spent commuting to work in one day in a geographic area
Households with income less than $10,000 | The number of households in a geographic area whose annual income was less than $10,000.
Households with income of $10,000 to $14,999 | The number of households in a geographic area whose annual income was between $10,000 and $14,999.
Households with income of $15,000 to $19,999 | The number of households in a geographic area whose annual income was between $15,000 and $19,999.
Households with income of $20,000 To $24,999 | The number of households in a geographic area whose annual income was between $20,000 and $24,999.
Households with income of $25,000 To $29,999 | The number of households in a geographic area whose annual income was between $20,000 and $24,999.
Households with income of $30,000 To $34,999 | The number of households in a geographic area whose annual income was between $30,000 and $34,999.
Households with income of $35,000 To $39,999 | The number of households in a geographic area whose annual income was between $35,000 and $39,999.
Households with income of $40,000 To $44,999 | The number of households in a geographic area whose annual income was between $40,000 and $44,999.
Households with income of $45,000 To $49,999 | The number of households in a geographic area whose annual income was between $45,000 and $49,999.
Households with income of $50,000 To $59,999 | The number of households in a geographic area whose annual income was between $50,000 and $59,999.
Households with income of $60,000 To $74,999 | The number of households in a geographic area whose annual income was between $60,000 and $74,999.
Households with income of $75,000 To $99,999 | The number of households in a geographic area whose annual income was between $75,000 and $99,999.
Households with income of $100,000 To $124,999 | The number of households in a geographic area whose annual income was between $100,000 and $124,999.
Households with income of $125,000 To $149,999 | The number of households in a geographic area whose annual income was between $125,000 and $149,999.
Households with income of $150,000 To $199,999 | The number of households in a geographic area whose annual income was between $150,000 and $1999,999.
Population age 16 and over | The number of people in each geography who are age 16 or over.
Population in Labor Force | The number of people in each geography who are either in the civilian labor force or are members of the U.S. Armed Forces (people on active duty with the United States Army, Air Force, Navy, Marine Corps, or Coast Guard).
Population in Civilian Labor Force | The number of civilians 16 years and over in each geography who can be classified as either employed or unemployed below.
Employed Population | The number of civilians 16 years old and over in each geography who either (1) were at work, that is, those who did any work at all during the reference week as paid employees, worked in their own business or profession, worked on their own farm, or worked 15 hours or more as unpaid workers on a family farm or in a family business; or (2) were with a job but not at work, that is, those who did not work during the reference week but had jobs or businesses from which they were temporarily absent due to illness, bad weather, industrial dispute, vacation, or other personal reasons. Excluded from the employed are people whose only activity consisted of work around the house or unpaid volunteer work for religious, charitable, and similar organizations; also excluded are all institutionalized people and people on active duty in the United States Armed Forces.
Unemployed Population | The number of civilians in each geography who are 16 years old and over are classified as unemployed if they (1) were neither at work nor with a job but not at work during the reference week, and (2) were actively looking for work during the last 4 weeks, and (3) were available to start a job. Also included as unemployed are civilians who did not work at all during the reference week, were waiting to be called back to a job from which they had been laid off, and were available for work except for temporary illness. Examples of job seeking activities are:
* Registering at a public or private employment office
* Meeting with prospective employers
* Investigating possibilities for starting a professional
practice or opening a business
* Placing or answering advertisements
* Writing letters of application
* Being on a union or professional register
Population in Armed Forces | The number of people in each geography who are members of the U.S. Armed Forces (people on active duty with the United States Army, Air Force, Navy, Marine Corps, or Coast Guard).
Population Not in Labor Force | The number of people in each geography who are 16 years old and over who are not classified as members of the labor force. This category consists mainly of students, homemakers, retired workers, seasonal workers interviewed in an off season who were not looking for work, institutionalized people, and people doing only incidental unpaid family work (less than 15 hours during the reference week).
Households with income of $200,000 Or More | The number of households in a geographic area whose annual income was more than $200,000.

15
scripts/README.md Normal file
View File

@ -0,0 +1,15 @@
## Automatic tests and utilities
### Installation
Python 2.7 should cover you. Virtualenv recommended.
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
### Execution
Run automated tests against a hostname:
(venv) OBS_HOSTNAME=<hostname.cartodb.com> OBS_API_KEY=foobar nosetests autotest.py

100
scripts/autotest.py Normal file
View File

@ -0,0 +1,100 @@
from nose.tools import assert_equal, assert_is_not_none
from nose_parameterized import parameterized
import os
import re
import requests
HOSTNAME = os.environ['OBS_HOSTNAME']
API_KEY = os.environ['OBS_API_KEY']
META_HOSTNAME = os.environ.get('OBS_META_HOSTNAME', HOSTNAME)
META_API_KEY = os.environ.get('OBS_META_API_KEY', API_KEY)
USE_SCHEMA = 'OBS_USE_SCHEMA' in os.environ
def query(q, is_meta=False, **options):
'''
Query the account. Returned is the response, wrapped by the requests
library.
'''
url = 'https://{hostname}/api/v2/sql'.format(
hostname=META_HOSTNAME if is_meta else HOSTNAME)
params = options.copy()
params['q'] = re.sub(r'\s+', ' ', q)
params['api_key'] = META_API_KEY if is_meta else API_KEY
return requests.get(url, params=params)
MEASURE_COLUMNS = [(r['id'], ) for r in query('''
SELECT id FROM observatory.obs_column
WHERE type ILIKE 'numeric'
AND weight > 0
''', is_meta=True).json()['rows']]
CATEGORY_COLUMNS = [(r['id'], ) for r in query('''
SELECT id FROM observatory.obs_column
WHERE type ILIKE 'text'
AND weight > 0
''', is_meta=True).json()['rows']]
BOUNDARY_COLUMNS = [(r['id'], ) for r in query('''
SELECT id FROM observatory.obs_column
WHERE type ILIKE 'geometry'
AND weight > 0
''', is_meta=True).json()['rows']]
def default_point(column_id):
'''
Returns default test point for the column_id.
'''
if column_id == 'whosonfirst.wof_disputed_geom':
return 'CDB_LatLng(33.78, 76.57)'
elif column_id == 'whosonfirst.wof_marinearea_geom':
return 'CDB_LatLng(43.33, -68.47)'
elif column_id in ('us.census.tiger.school_district_elementary',
'us.census.tiger.school_district_secondary',
'us.census.tiger.school_district_elementary_clipped',
'us.census.tiger.school_district_secondary_clipped'):
return 'CDB_LatLng(40.7025, -73.7067)'
elif column_id.startswith('es.ine'):
return 'CDB_LatLng(40.39, -3.7)'
elif column_id.startswith('us.zillow'):
return 'CDB_LatLng(28.3305906291771, -81.3544048197256)'
else:
return 'CDB_LatLng(40.7, -73.9)'
@parameterized(MEASURE_COLUMNS)
def test_measure_points(column_id):
resp = query('''
SELECT * FROM {schema}OBS_GetMeasure({point}, '{column_id}')
'''.format(column_id=column_id,
schema='cdb_observatory.' if USE_SCHEMA else '',
point=default_point(column_id)))
assert_equal(resp.status_code, 200)
rows = resp.json()['rows']
assert_equal(1, len(rows))
assert_is_not_none(rows[0].values()[0])
@parameterized(CATEGORY_COLUMNS)
def test_category_points(column_id):
resp = query('''
SELECT * FROM {schema}OBS_GetCategory({point}, '{column_id}')
'''.format(column_id=column_id,
schema='cdb_observatory.' if USE_SCHEMA else '',
point=default_point(column_id)))
assert_equal(resp.status_code, 200)
rows = resp.json()['rows']
assert_equal(1, len(rows))
assert_is_not_none(rows[0].values()[0])
@parameterized(BOUNDARY_COLUMNS)
def test_boundary_points(column_id):
resp = query('''
SELECT * FROM {schema}OBS_GetBoundary({point}, '{column_id}')
'''.format(column_id=column_id,
schema='cdb_observatory.' if USE_SCHEMA else '',
point=default_point(column_id)))
assert_equal(resp.status_code, 200)
rows = resp.json()['rows']
assert_equal(1, len(rows))
assert_is_not_none(rows[0].values()[0])

11
scripts/generate_fixtures.py
View File

@ -40,14 +40,15 @@ fixtures = [
('us.census.tiger.block_group', 'us.census.tiger.block_group', '2014'), ('us.census.tiger.block_group', 'us.census.tiger.block_group', '2014'),
('us.census.tiger.zcta5', 'us.census.tiger.zcta5', '2014'), ('us.census.tiger.zcta5', 'us.census.tiger.zcta5', '2014'),
('us.census.tiger.county', 'us.census.tiger.county', '2014'), ('us.census.tiger.county', 'us.census.tiger.county', '2014'),
('us.census.acs.B01001001', 'us.census.tiger.census_tract', '2009 - 2013'), ('us.census.acs.B01003001', 'us.census.tiger.census_tract', '2010 - 2014'),
('us.census.acs.B01001001_quantile', 'us.census.tiger.census_tract', '2009 - 2013'), ('us.census.acs.B01003001_quantile', 'us.census.tiger.census_tract', '2010 - 2014'),
('us.census.acs.B01001001', 'us.census.tiger.block_group', '2009 - 2013'), ('us.census.acs.B01003001', 'us.census.tiger.block_group', '2010 - 2014'),
('us.census.acs.B01001001', 'us.census.tiger.block_group', '2010 - 2014'), ('us.census.spielman_singleton_segments.X10', 'us.census.tiger.census_tract', '2010 - 2014'),
('us.census.spielman_singleton_segments.X10', 'us.census.tiger.census_tract', '2009 - 2013'),
('us.zillow.AllHomes_Zhvi', 'us.census.tiger.zcta5', '2014-01'), ('us.zillow.AllHomes_Zhvi', 'us.census.tiger.zcta5', '2014-01'),
('us.zillow.AllHomes_Zhvi', 'us.census.tiger.zcta5', '2016-03'), ('us.zillow.AllHomes_Zhvi', 'us.census.tiger.zcta5', '2016-03'),
('whosonfirst.wof_country_geom', 'whosonfirst.wof_country_geom', '2016'), ('whosonfirst.wof_country_geom', 'whosonfirst.wof_country_geom', '2016'),
('us.census.tiger.zcta5_clipped', 'us.census.tiger.zcta5_clipped', '2014'),
('us.census.tiger.block_group_clipped', 'us.census.tiger.block_group_clipped', '2014'),
] ]
unique_tables = set() unique_tables = set()

3
scripts/requirements.txt Normal file
View File

@ -0,0 +1,3 @@
requests
nose
nose_parameterized

21
src/pg/sql/41_observatory_augmentation.sql
View File

@ -39,7 +39,7 @@ AS $$
boundary_id = 'us.census.tiger.block_group'; boundary_id = 'us.census.tiger.block_group';
END IF; END IF;
target_cols := Array['us.census.acs.B01001001', target_cols := Array['us.census.acs.B01003001',
'us.census.acs.B01001002', 'us.census.acs.B01001002',
'us.census.acs.B01001026', 'us.census.acs.B01001026',
'us.census.acs.B01002001', 'us.census.acs.B01002001',
@ -172,6 +172,7 @@ BEGIN
RAISE NOTICE 'Point % is outside of the data region', ST_AsText(geom); RAISE NOTICE 'Point % is outside of the data region', ST_AsText(geom);
-- TODO this should return JSON -- TODO this should return JSON
RETURN QUERY SELECT '{}'::text[], '{}'::NUMERIC[]; RETURN QUERY SELECT '{}'::text[], '{}'::NUMERIC[];
RETURN;
END IF; END IF;
IF data_table_info IS NULL THEN IF data_table_info IS NULL THEN
@ -198,6 +199,7 @@ BEGIN
SELECT unnest($1) SELECT unnest($1)
$query$ $query$
USING results; USING results;
RETURN;
END; END;
$$ LANGUAGE plpgsql; $$ LANGUAGE plpgsql;
@ -356,11 +358,11 @@ BEGIN
' INTO denominator_id ' INTO denominator_id
USING measure_id; USING measure_id;
measure_ids := ARRAY[measure_id, denominator_id]; measure_ids := ARRAY[measure_id, denominator_id];
ELSIF normalize IS NULL OR normalize ILIKE 'none' THEN ELSIF normalize ILIKE 'none' THEN
-- TODO we need a switch on obs_get to disable area normalization -- TODO we need a switch on obs_get to disable area normalization
RAISE EXCEPTION 'No normalization not yet supported.'; RAISE EXCEPTION 'No normalization not yet supported.';
ELSE ELSE
RAISE EXCEPTION 'Only valid inputs for "normalize" are "area" (default), "denominator", or "none".'; RAISE EXCEPTION 'Only valid inputs for "normalize" are "area" (default) and "denominator".';
END IF; END IF;
EXECUTE ' EXECUTE '
@ -398,7 +400,7 @@ BEGIN
IF time_span IS NULL THEN IF time_span IS NULL THEN
-- TODO we should determine latest timespan for this measure -- TODO we should determine latest timespan for this measure
time_span := '2009 - 2013'; time_span := '2010 - 2014';
END IF; END IF;
EXECUTE ' EXECUTE '
@ -492,7 +494,7 @@ DECLARE
result NUMERIC; result NUMERIC;
BEGIN BEGIN
-- TODO use a super-column for global pop -- TODO use a super-column for global pop
population_measure_id := 'us.census.acs.B01001001'; population_measure_id := 'us.census.acs.B01003001';
EXECUTE format('SELECT cdb_observatory.OBS_GetMeasure( EXECUTE format('SELECT cdb_observatory.OBS_GetMeasure(
%L, %L, %L, %L, %L %L, %L, %L, %L, %L
@ -602,7 +604,7 @@ IF boundary_id IS NULL THEN
boundary_id = 'us.census.tiger.census_tract'; boundary_id = 'us.census.tiger.census_tract';
END IF; END IF;
target_cols := Array[ target_cols := Array[
'us.census.acs.B01001001_quantile', 'us.census.acs.B01003001_quantile',
'us.census.acs.B01001002_quantile', 'us.census.acs.B01001002_quantile',
'us.census.acs.B01001026_quantile', 'us.census.acs.B01001026_quantile',
'us.census.acs.B01002001_quantile', 'us.census.acs.B01002001_quantile',
@ -668,7 +670,7 @@ target_cols := Array[
array_agg(_OBS_GET->>'value') As vals array_agg(_OBS_GET->>'value') As vals
FROM cdb_observatory._OBS_Get($1, FROM cdb_observatory._OBS_Get($1,
$2, $2,
'2009 - 2013', '2010 - 2014',
$3) $3)
), percentiles As ( ), percentiles As (
@ -709,7 +711,7 @@ DECLARE
BEGIN BEGIN
IF time_span IS NULL THEN IF time_span IS NULL THEN
time_span = '2009 - 2013'; time_span = '2010 - 2014';
END IF; END IF;
IF boundary_id IS NULL THEN IF boundary_id IS NULL THEN
@ -722,6 +724,7 @@ BEGIN
THEN THEN
RAISE NOTICE 'Point % is outside of the data region', ST_AsText(geom); RAISE NOTICE 'Point % is outside of the data region', ST_AsText(geom);
RETURN QUERY SELECT '{}'::text[], '{}'::text[]; RETURN QUERY SELECT '{}'::text[], '{}'::text[];
RETURN;
END IF; END IF;
EXECUTE ' EXECUTE '
@ -735,6 +738,7 @@ BEGIN
THEN THEN
RAISE NOTICE 'No data table found for this location'; RAISE NOTICE 'No data table found for this location';
RETURN QUERY SELECT NULL::json; RETURN QUERY SELECT NULL::json;
RETURN;
END IF; END IF;
EXECUTE EXECUTE
@ -749,6 +753,7 @@ BEGIN
THEN THEN
RAISE NOTICE 'No geometry id for this location'; RAISE NOTICE 'No geometry id for this location';
RETURN QUERY SELECT NULL::json; RETURN QUERY SELECT NULL::json;
RETURN;
END IF; END IF;
query := 'SELECT ARRAY['; query := 'SELECT ARRAY[';

20
src/pg/sql/42_observatory_exploration.sql
View File

@ -15,7 +15,7 @@ BEGIN
THEN THEN
RETURN QUERY RETURN QUERY
EXECUTE EXECUTE
'SELECT tablename, timespan 'SELECT tablename::text, timespan::text
FROM observatory.obs_table t FROM observatory.obs_table t
JOIN observatory.obs_column_table ct JOIN observatory.obs_column_table ct
ON ct.table_id = t.id ON ct.table_id = t.id
@ -24,10 +24,11 @@ BEGIN
WHERE c.type ILIKE ''geometry'' WHERE c.type ILIKE ''geometry''
AND c.id = $1' AND c.id = $1'
USING search_term; USING search_term;
RETURN;
ELSE ELSE
RETURN QUERY RETURN QUERY
EXECUTE EXECUTE
'SELECT tablename, timespan 'SELECT tablename::text, timespan::text
FROM observatory.obs_table t FROM observatory.obs_table t
JOIN observatory.obs_column_table ct JOIN observatory.obs_column_table ct
ON ct.table_id = t.id ON ct.table_id = t.id
@ -37,6 +38,7 @@ BEGIN
AND c.id = $1 AND c.id = $1
AND t.timespan = $2' AND t.timespan = $2'
USING search_term, time_span; USING search_term, time_span;
RETURN;
END IF; END IF;
END; END;
@ -63,9 +65,9 @@ BEGIN
RETURN QUERY RETURN QUERY
EXECUTE format($string$ EXECUTE format($string$
SELECT id, description, SELECT id::text, description::text,
name, name::text,
aggregate, aggregate::text,
NULL::TEXT source -- TODO use tags NULL::TEXT source -- TODO use tags
FROM observatory.OBS_column FROM observatory.OBS_column
where name ilike '%%' || %L || '%%' where name ilike '%%' || %L || '%%'
@ -98,10 +100,10 @@ BEGIN
EXECUTE EXECUTE
$string$ $string$
SELECT SELECT
column_id, column_id::text As column_id,
obs_column.description, obs_column.description::text As description,
timespan, timespan::text As timespan,
tablename tablename::text As tablename
FROM FROM
observatory.OBS_table, observatory.OBS_table,
observatory.OBS_column_table, observatory.OBS_column_table,

11
src/pg/sql/44_observatory_geometries.sql
View File

@ -119,7 +119,7 @@ BEGIN
-- If not point, raise error -- If not point, raise error
IF ST_GeometryType(geom) != 'ST_Point' IF ST_GeometryType(geom) != 'ST_Point'
THEN THEN
RAISE EXCEPTION 'Error: Invalid geometry type (%), expecting ''ST_Point''', ST_GeometryType(geom); RAISE EXCEPTION 'Invalid geometry type (%), expecting ''ST_Point''', ST_GeometryType(geom);
END IF; END IF;
-- choose appropriate table based on time_span -- choose appropriate table based on time_span
@ -143,7 +143,7 @@ BEGIN
-- if no tables are found, raise notice and return null -- if no tables are found, raise notice and return null
IF target_table IS NULL IF target_table IS NULL
THEN THEN
RAISE NOTICE 'No boundaries found for ''%''', boundary_id; RAISE NOTICE 'Warning: No boundaries found for ''%''', boundary_id;
RETURN NULL::text; RETURN NULL::text;
END IF; END IF;
@ -545,9 +545,9 @@ BEGIN
RETURN QUERY RETURN QUERY
EXECUTE EXECUTE
format($string$ format($string$
SELECT geoid_ct.colname As geoid_colname, SELECT geoid_ct.colname::text As geoid_colname,
tablename, tablename::text,
geom_ct.colname As geom_colname geom_ct.colname::text As geom_colname
FROM observatory.obs_column_table As geoid_ct, FROM observatory.obs_column_table As geoid_ct,
observatory.obs_table As geom_t, observatory.obs_table As geom_t,
observatory.obs_column_table As geom_ct, observatory.obs_column_table As geom_ct,
@ -569,6 +569,7 @@ BEGIN
-- AND geom_t.timespan = '%s' <-- put in requested year -- AND geom_t.timespan = '%s' <-- put in requested year
-- TODO: filter by clipped vs. not so appropriate tablename are unique -- TODO: filter by clipped vs. not so appropriate tablename are unique
-- so the limit 1 can be removed -- so the limit 1 can be removed
RETURN;
END; END;
$$ LANGUAGE plpgsql; $$ LANGUAGE plpgsql;

24
src/pg/test/expected/40_observatory_utility_test.out
View File

@ -3,11 +3,11 @@
\i test/fixtures/load_fixtures.sql \i test/fixtures/load_fixtures.sql
SET client_min_messages TO WARNING; SET client_min_messages TO WARNING;
\set ECHO none \set ECHO none
_obs_geomtable _obs_geomtable_with_returned_table
obs_fc050f0b8673cfe3c6aa1040f749eb40975691b7 t
(1 row) (1 row)
_obs_geomtable _obs_geomtable_with_null_response
t
(1 row) (1 row)
test_get_obs_column_with_geoid_and_census_1|test_get_obs_column_with_geoid_and_census_2 test_get_obs_column_with_geoid_and_census_1|test_get_obs_column_with_geoid_and_census_2
t|t t|t
@ -15,15 +15,15 @@ t|t
obs_getcolumndata_missing_measure obs_getcolumndata_missing_measure
t t
(1 row) (1 row)
_obs_buildsnapshotquery _obs_buildsnapshotquery_test_1
SELECT vals[1] As total_pop, vals[2] As male_pop, vals[3] As female_pop, vals[4] As median_age t
(1 row) (1 row)
_obs_buildsnapshotquery _obs_buildsnapshotquery_test_2
SELECT vals[1] As mandarin_orange t
(1 row) (1 row)
_obs_getrelatedcolumn _obs_getrelatedcolumn_test
{es.ine.total_pop,NULL,us.census.acs.B01001001} t
(1 row) (1 row)
_obs_standardizemeasurename _obs_standardizemeasurename_test
test_343_2_qqq t
(1 row) (1 row)

41
src/pg/test/expected/42_observatory_exploration_test.out
View File

@ -2,17 +2,36 @@
SET client_min_messages TO WARNING; SET client_min_messages TO WARNING;
\set ECHO none \set ECHO none
_obs_searchtables_tables_match|_obs_searchtables_timespan_matches _obs_searchtables_tables_match|_obs_searchtables_timespan_matches
obs_1babf5a26a1ecda5fb74963e88408f71d0364b81|t t|t
(1 row) (1 row)
_obs_searchtables_timespan_does_not_match _obs_searchtables_timespan_does_not_match
t t
(1 row) (1 row)
obs_search id|description|name|aggregate|source
(es.ine.total_pop,"The total number of all people living in a geographic area.","Total Population",sum,) us.census.acs.B01003001_quantile|The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.|Quantile:Total Population|quantile|
(us.census.acs.B01001001,"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Total Population",sum,) es.ine.total_pop|The total number of all people living in a geographic area.|Total Population|sum|
(us.census.acs.B01001001_quantile,"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","Quantile:Total Population",quantile,) us.census.acs.B01003001|The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.|Total Population|sum|
(3 rows) us.census.acs.B19301001|Per capita income is the mean income computed for every man, woman, and child in a particular group. It is derived by dividing the total income of a particular group by the total population.|Per Capita Income in the past 12 Months|average|
us.census.acs.B01001001_quantile|The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.|Quantile:Total Population|quantile|
(5 rows)
boundary_id|description|time_span|tablename boundary_id|description|time_span|tablename
us.census.tiger.zcta5|ZCTAs are approximate area representations of U.S. Postal Service (USPS) five-digit ZIP Code service areas that the Census Bureau creates using whole blocks to present statistical data from censuses and surveys. The Census Bureau defines ZCTAs by allocating each block that contains addresses to a single ZCTA, usually to the ZCTA that reflects the most frequently occurring ZIP Code for the addresses within that tabulation block. Blocks that do not contain addresses but are completely surrounded by a single ZCTA (enclaves) are assigned to the surrounding ZCTA; those surrounded by multiple ZCTAs will be added to a single ZCTA based on limited buffering performed between multiple ZCTAs. The Census Bureau identifies five-digit ZCTAs using a five-character numeric code that represents the most frequently occurring USPS ZIP Code within that ZCTA, and this code may contain leading zeros.
There are significant changes to the 2010 ZCTA delineation from that used in 2000. Coverage was extended to include the Island Areas for 2010 so that the United States, Puerto Rico, and the Island Areas have ZCTAs. Unlike 2000, when areas that could not be assigned to a ZCTA were given a generic code ending in “XX” (land area) or “HH” (water area), for 2010 there is no universal coverage by ZCTAs, and only legitimate five-digit areas are defined. The 2010 ZCTAs will better represent the actual Zip Code service areas because the Census Bureau initiated a process before creation of 2010 blocks to add block boundaries that split polygons with large numbers of addresses using different Zip Codes.
Data users should not use ZCTAs to identify the official USPS ZIP Code for mail delivery. The USPS makes periodic changes to ZIP Codes to support more efficient mail delivery. The ZCTAs process used primarily residential addresses and was biased towards Zip Codes used for city-style mail delivery, thus there may be Zip Codes that are primarily nonresidential or boxes only that may not have a corresponding ZCTA.|2014|obs_144e8b4f906885b2e057ac4842644a553ae49c6e
whosonfirst.wof_continent_geom|Continents of the world.|2016|obs_9880042f935aab0d0e4b71fb6963d7726e64c534
whosonfirst.wof_country_geom| |2016|obs_1ea93bbc109c87c676b3270789dacf7a1430db6c
whosonfirst.wof_region_geom| |2016|obs_4fca4f060854cc3ae8c109999635a71bbde6964e
whosonfirst.wof_marinearea_geom| |2016|obs_5105018d57c69b8a7e064fc17a9485647b311a99
whosonfirst.wof_disputed_geom| |2016|obs_7dba9374b15fbab0c7bd7dca6dec6c4792fe86a3
us.census.tiger.congressional_district|Congressional districts are the 435 areas from which people are elected to the U.S. House of Representatives. After the apportionment of congressional seats among the states based on decennial census population counts, each state with multiple seats is responsible for establishing congressional districts for the purpose of electing representatives. Each congressional district is to be as equal in population to all other congressional districts in a state as practicable. For the District of Columbia, Puerto Rico, and each Island Area, a separate code is used to identify the entire areas of these state-equivalent entities as having a single nonvoting delegate.|2014|obs_01b608b3a4ca503ad7acc0b1f84817bc1da3e193
us.census.tiger.congressional_district_clipped|A cartography-ready version of US Congressional Districts|2014|obs_9d258128ff4288eb9a6d7e5c0adbef8ef0172a86
us.census.tiger.school_district_unified_clipped|A cartography-ready version of Unified School District|2014|obs_546285f2c636f5380e7bfbb0c0db67863f6ed41d
us.census.tiger.cbsa_clipped|A cartography-ready version of Core Based Statistical Area (CBSA)|2014|obs_3512a78ca8c7e9b5fbd5390c4ed1638b9938fcbf
us.census.tiger.block|Census blocks are statistical areas bounded by visible features, such as streets, roads, streams, and railroad tracks, and by nonvisible boundaries, such as selected property lines and city, township, school district, and county limits and short line-of-sight extensions of streets and roads. Generally, census blocks are small in area; for example, a block in a city bounded on all sides by streets. Census blocks in suburban and rural areas may be large, irregular, and bounded by a variety of features, such as roads, streams, and transmission lines. In remote areas, census blocks may encompass hundreds of square miles. Census blocks cover the entire territory of the United States, Puerto Rico, and the Island Areas. Census blocks nest within all other tabulated census geographic entities and are the basis for all tabulated data.|2014|obs_ffebc3eb689edab4faa757f75ca02c65d7db7327
us.census.tiger.school_district_elementary_clipped|A cartography-ready version of Elementary School District|2014|obs_41d99a86857c05f63320ea44419a68831e74ac3d
us.census.tiger.block_group_clipped|A cartography-ready version of US Census Block Groups|2014|obs_6c1309a64d8f3e6986061f4d1ca7b57743e75e74
us.census.tiger.census_tract_clipped|A cartography-ready version of US Census Tracts|2014|obs_fcd4e4f5610f6764973ef8c0c215b2e80bec8963
us.census.tiger.zcta5_clipped|A cartography-ready version of US Census Zip Code Tabulation Areas|2014|obs_7615e8622a68bfc5fe37c69c9880edfb40250103
us.census.tiger.place|Incorporated places are those reported to the Census Bureau as legally in existence as of January 1, 2010, as reported in the latest Boundary and Annexation Survey (BAS), under the laws of their respective states. An incorporated place is established to provide governmental functions for a concentration of people as opposed to a minor civil division, which generally is created to provide services or administer an area without regard, necessarily, to population. Places always are within a single state or equivalent entity, but may extend across county and county subdivision boundaries. An incorporated place usually is a city, town, village, or borough, but can have other legal descriptions. For Census Bureau data tabulation and presentation purposes, incorporated places exclude: us.census.tiger.place|Incorporated places are those reported to the Census Bureau as legally in existence as of January 1, 2010, as reported in the latest Boundary and Annexation Survey (BAS), under the laws of their respective states. An incorporated place is established to provide governmental functions for a concentration of people as opposed to a minor civil division, which generally is created to provide services or administer an area without regard, necessarily, to population. Places always are within a single state or equivalent entity, but may extend across county and county subdivision boundaries. An incorporated place usually is a city, town, village, or borough, but can have other legal descriptions. For Census Bureau data tabulation and presentation purposes, incorporated places exclude:
Boroughs in Alaska (treated as statistical equivalents of counties). Boroughs in Alaska (treated as statistical equivalents of counties).
Towns in the New England states, New York, and Wisconsin (treated as MCDs). Towns in the New England states, New York, and Wisconsin (treated as MCDs).
@ -20,9 +39,13 @@ Boroughs in New York (treated as MCDs).
Census Designated Places (CDPs) are the statistical counterparts of incorporated places, and are delineated to provide data for settled concentrations of population that are identifiable by name but are not legally incorporated under the laws of the state in which they are located. The boundaries usually are defined in cooperation with local or tribal officials and generally updated prior to each decennial census. These boundaries, which usually coincide with visible features or the boundary of an adjacent incorporated place or another legal entity boundary, have no legal status, nor do these places have officials elected to serve traditional municipal functions. CDP boundaries may change from one decennial census Census Designated Places (CDPs) are the statistical counterparts of incorporated places, and are delineated to provide data for settled concentrations of population that are identifiable by name but are not legally incorporated under the laws of the state in which they are located. The boundaries usually are defined in cooperation with local or tribal officials and generally updated prior to each decennial census. These boundaries, which usually coincide with visible features or the boundary of an adjacent incorporated place or another legal entity boundary, have no legal status, nor do these places have officials elected to serve traditional municipal functions. CDP boundaries may change from one decennial census
to the next with changes in the settlement pattern; a CDP with the same name as in an earlier census does not necessarily have the same boundary. CDPs must be contained within a single state and may not extend into an incorporated place. There are no population size requirements for CDPs. to the next with changes in the settlement pattern; a CDP with the same name as in an earlier census does not necessarily have the same boundary. CDPs must be contained within a single state and may not extend into an incorporated place. There are no population size requirements for CDPs.
Hawaii is the only state that has no incorporated places recognized by the Census Bureau. All places shown in decennial census data products for Hawaii are CDPs. By agreement with the state of Hawaii, the Census Bureau does not show data separately for the city of Honolulu, which is coextensive with Honolulu County. In Puerto Rico, which also does not have incorporated places, the Census Bureau recognizes only CDPs and refers to them as comunidades or zonas urbanas. Guam also has only CDPs.|2014|obs_76a52df2018de8d064f1a99f93544473927cb7ae Hawaii is the only state that has no incorporated places recognized by the Census Bureau. All places shown in decennial census data products for Hawaii are CDPs. By agreement with the state of Hawaii, the Census Bureau does not show data separately for the city of Honolulu, which is coextensive with Honolulu County. In Puerto Rico, which also does not have incorporated places, the Census Bureau recognizes only CDPs and refers to them as comunidades or zonas urbanas. Guam also has only CDPs.|2014|obs_76a52df2018de8d064f1a99f93544473927cb7ae
us.census.tiger.place_clipped|A cartography-ready version of Incorporated Places|2014|obs_db91d46d317a4ffcf509efca8e5e3a42d29e0792
us.census.tiger.school_district_secondary_clipped|A cartography-ready version of Secondary School District|2014|obs_63b15ba742ccea136ddb88427cc37fee218702d4
us.census.tiger.cbsa|Core Based Statistical Areas (CBSAs) consist of the county or counties or equivalent entities associated with at least one core (urbanized area or urban cluster) of at least 10,000 population, plus adjacent counties having a high degree of social and economic integration with the core as measured through commuting ties with the counties associated with the core. The general concept of a CBSA is that of a us.census.tiger.cbsa|Core Based Statistical Areas (CBSAs) consist of the county or counties or equivalent entities associated with at least one core (urbanized area or urban cluster) of at least 10,000 population, plus adjacent counties having a high degree of social and economic integration with the core as measured through commuting ties with the counties associated with the core. The general concept of a CBSA is that of a
core area containing a substantial population nucleus, together with adjacent communities having a high degree of economic and social integration with that core. The term “core based statistical area” became effective in 2003 and refers collectively to metropolitan statistical areas and micropolitan statistical areas. The U.S. Office of Management and Budget (OMB) defines CBSAs to provide a nationally consistent set of geographic entities for the United States and Puerto Rico for use in tabulating and presenting statistical data. Current CBSAs are based on application of the 2000 standards (published in the Federal Register of December 27, 2000) with Census 2000 data. The first set of areas defined based on the 2000 standards were announced on June 6, 2003; subsequent updates have been made to the universe of CBSAs and related statistical areas. No CBSAs are defined in the Island Areas. Statistical areas related to CBSAs include metropolitan divisions, combined statistical areas (CSAs), New England city and town areas (NECTAs), NECTA divisions, and combined NECTAs.|2014|obs_c75be9ef45e87c789c3607dd9aeef6094d5e5109 core area containing a substantial population nucleus, together with adjacent communities having a high degree of economic and social integration with that core. The term “core based statistical area” became effective in 2003 and refers collectively to metropolitan statistical areas and micropolitan statistical areas. The U.S. Office of Management and Budget (OMB) defines CBSAs to provide a nationally consistent set of geographic entities for the United States and Puerto Rico for use in tabulating and presenting statistical data. Current CBSAs are based on application of the 2000 standards (published in the Federal Register of December 27, 2000) with Census 2000 data. The first set of areas defined based on the 2000 standards were announced on June 6, 2003; subsequent updates have been made to the universe of CBSAs and related statistical areas. No CBSAs are defined in the Island Areas. Statistical areas related to CBSAs include metropolitan divisions, combined statistical areas (CSAs), New England city and town areas (NECTAs), NECTA divisions, and combined NECTAs.|2014|obs_c75be9ef45e87c789c3607dd9aeef6094d5e5109
us.census.tiger.puma_clipped|A cartography-ready version of US Census Public Use Microdata Areas|2014|obs_dc244bf520f62e4a09e290a02e55368fd0758f95
us.census.tiger.county|The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier.|2014|obs_1babf5a26a1ecda5fb74963e88408f71d0364b81 us.census.tiger.county|The primary legal divisions of most states are termed counties. In Louisiana, these divisions are known as parishes. In Alaska, which has no counties, the equivalent entities are the organized boroughs, city and boroughs, municipalities, and census areas; the latter of which are delineated cooperatively for statistical purposes by the state of Alaska and the Census Bureau. In four states (Maryland, Missouri, Nevada, and Virginia), there are one or more incorporated places that are independent of any county organization and thus constitute primary divisions of their states. These incorporated places are known as independent cities and are treated as equivalent entities for purposes of data presentation. The District of Columbia and Guam have no primary divisions, and each area is considered an equivalent entity for purposes of data presentation. All of the counties in Connecticut and Rhode Island and nine counties in Massachusetts were dissolved as functioning governmental entities; however, the Census Bureau continues to present data for these historical entities in order to provide comparable geographic units at the county level of the geographic hierarchy for these states and represents them as nonfunctioning legal entities in data products. The Census Bureau treats the following entities as equivalents of counties for purposes of data presentation: municipios in Puerto Rico, districts and islands in American Samoa, municipalities in the Commonwealth of the Northern Mariana Islands, and islands in the U.S. Virgin Islands. Each county or statistically equivalent entity is assigned a three-character numeric Federal Information Processing Series (FIPS) code based on alphabetical sequence that is unique within state and an eight-digit National Standard feature identifier.|2014|obs_1babf5a26a1ecda5fb74963e88408f71d0364b81
us.census.tiger.county_clipped|A cartography-ready version of US County|2014|obs_23cb5063486bd7cf36f17e89e5e65cd31b331f6e
us.census.tiger.school_district_unified|School Districts are geographic entities within which state, county, local officials, the Bureau of Indian Affairs, or the U.S. Department of Defense provide public educational services for the area’s residents. The Census Bureau obtains the boundaries, names, local education agency codes, and school district levels for school districts from state and local school officials for the primary purpose of providing the U.S. Department of Education with estimates of the number of children “at risk” within each school district, county, and state. This information serves as the basis for the Department of Education to determine the annual allocation of Title I funding to states and school districts. us.census.tiger.school_district_unified|School Districts are geographic entities within which state, county, local officials, the Bureau of Indian Affairs, or the U.S. Department of Defense provide public educational services for the area’s residents. The Census Bureau obtains the boundaries, names, local education agency codes, and school district levels for school districts from state and local school officials for the primary purpose of providing the U.S. Department of Education with estimates of the number of children “at risk” within each school district, county, and state. This information serves as the basis for the Department of Education to determine the annual allocation of Title I funding to states and school districts.
The Census Bureau tabulates data for three types of school districts: elementary, secondary, and unified. Each school district is assigned a five-digit code that is unique within state. School district codes are the local education agency number assigned by the Department of Education and are not necessarily in alphabetical order by school district name. The Census Bureau tabulates data for three types of school districts: elementary, secondary, and unified. Each school district is assigned a five-digit code that is unique within state. School district codes are the local education agency number assigned by the Department of Education and are not necessarily in alphabetical order by school district name.
The elementary school districts provide education to the lower grade/age levels and the secondary school districts provide education to the upper grade/age levels. Unified school districts provide education to children of all school ages in their service areas. In general, where there is a unified school district, no elementary or secondary school district exists; and where there is an elementary school district, the secondary school district may or may not exist. The elementary school districts provide education to the lower grade/age levels and the secondary school districts provide education to the upper grade/age levels. Unified school districts provide education to children of all school ages in their service areas. In general, where there is a unified school district, no elementary or secondary school district exists; and where there is an elementary school district, the secondary school district may or may not exist.
@ -41,9 +64,7 @@ The Census Bureau’s representation of school districts in various data product
us.census.tiger.block_group|Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau’s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate. us.census.tiger.block_group|Block groups (BGs) are statistical divisions of census tracts, are generally defined to contain between 600 and 3,000 people, and are used to present data and control block numbering. A block group consists of clusters of blocks within the same census tract that have the same first digit of their four-digit census block number. For example, blocks 3001, 3002, 3003, ..., 3999 in census tract 1210.02 belong to BG 3 in that census tract. Most BGs were delineated by local participants in the Census Bureau’s Participant Statistical Areas Program. The Census Bureau delineated BGs only where a local or tribal government declined to participate, and a regional organization or State Data Center was not available to participate.
A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see “Tribal Census Tract” and “Tribal Block Group”). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county.|2014|obs_c6fb99c47d61289fbb8e561ff7773799d3fcc308 A BG usually covers a contiguous area. Each census tract contains at least one BG, and BGs are uniquely numbered within the census tract. Within the standard census geographic hierarchy, BGs never cross state, county, or census tract boundaries but may cross the boundaries of any other geographic entity. Tribal census tracts and tribal BGs are separate and unique geographic areas defined within federally recognized American Indian reservations and can cross state and county boundaries (see “Tribal Census Tract” and “Tribal Block Group”). The tribal census tracts and tribal block groups may be completely different from the census tracts and block groups defined by state and county.|2014|obs_c6fb99c47d61289fbb8e561ff7773799d3fcc308
us.census.tiger.state|States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation.|2014|obs_624e5d2362e08aaa5463d7671e7748432262719c us.census.tiger.state|States and Equivalent Entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation.|2014|obs_624e5d2362e08aaa5463d7671e7748432262719c
us.census.tiger.zcta5|ZCTAs are approximate area representations of U.S. Postal Service (USPS) five-digit ZIP Code service areas that the Census Bureau creates using whole blocks to present statistical data from censuses and surveys. The Census Bureau defines ZCTAs by allocating each block that contains addresses to a single ZCTA, usually to the ZCTA that reflects the most frequently occurring ZIP Code for the addresses within that tabulation block. Blocks that do not contain addresses but are completely surrounded by a single ZCTA (enclaves) are assigned to the surrounding ZCTA; those surrounded by multiple ZCTAs will be added to a single ZCTA based on limited buffering performed between multiple ZCTAs. The Census Bureau identifies five-digit ZCTAs using a five-character numeric code that represents the most frequently occurring USPS ZIP Code within that ZCTA, and this code may contain leading zeros. us.census.tiger.state_clipped|A cartography-ready version of US States|2014|obs_f39f1d7cd5a22b87140860cbd58539f1591a1810
There are significant changes to the 2010 ZCTA delineation from that used in 2000. Coverage was extended to include the Island Areas for 2010 so that the United States, Puerto Rico, and the Island Areas have ZCTAs. Unlike 2000, when areas that could not be assigned to a ZCTA were given a generic code ending in “XX” (land area) or “HH” (water area), for 2010 there is no universal coverage by ZCTAs, and only legitimate five-digit areas are defined. The 2010 ZCTAs will better represent the actual Zip Code service areas because the Census Bureau initiated a process before creation of 2010 blocks to add block boundaries that split polygons with large numbers of addresses using different Zip Codes.
Data users should not use ZCTAs to identify the official USPS ZIP Code for mail delivery. The USPS makes periodic changes to ZIP Codes to support more efficient mail delivery. The ZCTAs process used primarily residential addresses and was biased towards Zip Codes used for city-style mail delivery, thus there may be Zip Codes that are primarily nonresidential or boxes only that may not have a corresponding ZCTA.|2014|obs_144e8b4f906885b2e057ac4842644a553ae49c6e
us.census.tiger.puma|PUMAs are geographic areas for which the Census Bureau provides selected extracts of raw data from a small sample of census records that are screened to protect confidentiality. These extracts are referred to as public use microdata sample (PUMS) files. us.census.tiger.puma|PUMAs are geographic areas for which the Census Bureau provides selected extracts of raw data from a small sample of census records that are screened to protect confidentiality. These extracts are referred to as public use microdata sample (PUMS) files.
For the 2010 Census, each state, the District of Columbia, Puerto Rico, and some Island Area participants delineated PUMAs for use in presenting PUMS data based on a 5 percent sample of decennial census or American Community Survey data. These areas are required to contain at least 100,000 people. This is different from Census 2000 when two types of PUMAs were defined: a 5 percent PUMA as for 2010 and an additional super-PUMA designed to provide a 1 percent sample. The PUMAs are identified by a five-digit census code unique within state.|2014|obs_7c9493c41fa8f4bd178ab993ea3d5891c1977667 For the 2010 Census, each state, the District of Columbia, Puerto Rico, and some Island Area participants delineated PUMAs for use in presenting PUMS data based on a 5 percent sample of decennial census or American Community Survey data. These areas are required to contain at least 100,000 people. This is different from Census 2000 when two types of PUMAs were defined: a 5 percent PUMA as for 2010 and an additional super-PUMA designed to provide a 1 percent sample. The PUMAs are identified by a five-digit census code unique within state.|2014|obs_7c9493c41fa8f4bd178ab993ea3d5891c1977667
(11 rows) (30 rows)

9
src/pg/test/fixtures/drop_fixtures.sql vendored
View File

@ -7,13 +7,14 @@ DROP TABLE IF EXISTS observatory.obs_column_tag;
DROP TABLE IF EXISTS observatory.obs_tag; DROP TABLE IF EXISTS observatory.obs_tag;
DROP TABLE IF EXISTS observatory.obs_column_to_column; DROP TABLE IF EXISTS observatory.obs_column_to_column;
DROP TABLE IF EXISTS observatory.obs_65f29658e096ca1485bf683f65fdbc9f05ec3c5d; DROP TABLE IF EXISTS observatory.obs_65f29658e096ca1485bf683f65fdbc9f05ec3c5d;
DROP TABLE IF EXISTS observatory.obs_1746e37b7cd28cb131971ea4187d42d71f09c5f3;
DROP TABLE IF EXISTS observatory.obs_1a098da56badf5f32e336002b0a81708c40d29cd; DROP TABLE IF EXISTS observatory.obs_1a098da56badf5f32e336002b0a81708c40d29cd;
DROP TABLE IF EXISTS observatory.obs_fc050f0b8673cfe3c6aa1040f749eb40975691b7; DROP TABLE IF EXISTS observatory.obs_7615e8622a68bfc5fe37c69c9880edfb40250103;
DROP TABLE IF EXISTS observatory.obs_1babf5a26a1ecda5fb74963e88408f71d0364b81; DROP TABLE IF EXISTS observatory.obs_1babf5a26a1ecda5fb74963e88408f71d0364b81;
DROP TABLE IF EXISTS observatory.obs_8764a6b439a4f8714f54d4b3a157bc5e36519066; DROP TABLE IF EXISTS observatory.obs_8764a6b439a4f8714f54d4b3a157bc5e36519066;
DROP TABLE IF EXISTS observatory.obs_3e7cc9cfd403b912c57b42d5f9195af9ce2f3cdb; DROP TABLE IF EXISTS observatory.obs_b393b5b88c6adda634b2071a8005b03c551b609a;
DROP TABLE IF EXISTS observatory.obs_d34555209878e8c4b37cf0b2b3d072ff129ec470; DROP TABLE IF EXISTS observatory.obs_fc050f0b8673cfe3c6aa1040f749eb40975691b7;
DROP TABLE IF EXISTS observatory.obs_6c1309a64d8f3e6986061f4d1ca7b57743e75e74;
DROP TABLE IF EXISTS observatory.obs_d39f7fe5959891c8296490d83c22ded31c54af13; DROP TABLE IF EXISTS observatory.obs_d39f7fe5959891c8296490d83c22ded31c54af13;
DROP TABLE IF EXISTS observatory.obs_144e8b4f906885b2e057ac4842644a553ae49c6e; DROP TABLE IF EXISTS observatory.obs_144e8b4f906885b2e057ac4842644a553ae49c6e;
DROP TABLE IF EXISTS observatory.obs_c6fb99c47d61289fbb8e561ff7773799d3fcc308; DROP TABLE IF EXISTS observatory.obs_c6fb99c47d61289fbb8e561ff7773799d3fcc308;
DROP TABLE IF EXISTS observatory.obs_ab038198aaab3f3cb055758638ee4de28ad70146;

29909
src/pg/test/fixtures/load_fixtures.sql vendored
View File

File diff suppressed because one or more lines are too long

31
src/pg/test/sql/40_observatory_utility_test.sql
View File

@ -10,7 +10,7 @@ SELECT
ST_SetSRID(ST_Point(-74.0059, 40.7128), 4326), ST_SetSRID(ST_Point(-74.0059, 40.7128), 4326),
'us.census.tiger.census_tract', 'us.census.tiger.census_tract',
'2014' '2014'
); ) = 'obs_fc050f0b8673cfe3c6aa1040f749eb40975691b7' As _obs_geomtable_with_returned_table;
-- get null for unknown geometry_id -- get null for unknown geometry_id
-- should give back null -- should give back null
@ -18,7 +18,7 @@ SELECT
cdb_observatory._OBS_GeomTable( cdb_observatory._OBS_GeomTable(
ST_SetSRID(ST_Point(-74.0059, 40.7128), 4326), ST_SetSRID(ST_Point(-74.0059, 40.7128), 4326),
'us.census.tiger.nonexistant_id' -- not in catalog 'us.census.tiger.nonexistant_id' -- not in catalog
); ) IS NULL _obs_geomtable_with_null_response;
-- future test: give back nulls when geometry doesn't intersect -- future test: give back nulls when geometry doesn't intersect
-- SELECT -- SELECT
@ -27,21 +27,16 @@ SELECT
-- 'us.census.tiger.census_tract' -- 'us.census.tiger.census_tract'
-- ); -- );
-- OBS_GetColumnData
-- should give back:
-- colname | tablename | aggregate
-- -----------|-----------------|-----------
-- geoid | obs_{hex table} | null
-- total_pop | obs_{hex table} | sum
WITH result as ( WITH result as (
SELECT SELECT
array_agg(a) expected from cdb_observatory._OBS_GetColumnData( array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'us.census.tiger.census_tract', 'us.census.tiger.census_tract',
Array['us.census.tiger.census_tract_geoid', 'us.census.acs.B01001001'], Array['us.census.spielman_singleton_segments.X55', 'us.census.acs.B01003001'],
'2009 - 2013') a '2010 - 2014') a
) )
select (expected)[1]::text = '{"colname":"geoid","tablename":"obs_d34555209878e8c4b37cf0b2b3d072ff129ec470","aggregate":null,"name":"US Census Tracts Geoids","type":"Text","description":null,"boundary_id":"us.census.tiger.census_tract"}' as test_get_obs_column_with_geoid_and_census_1, select
(expected)[2]::text = '{"colname":"geoid","tablename":"obs_ab038198aaab3f3cb055758638ee4de28ad70146","aggregate":null,"name":"US Census Tracts Geoids","type":"Text","description":null,"boundary_id":"us.census.tiger.census_tract"}' as test_get_obs_column_with_geoid_and_census_2 (expected)[1]::text = '{"colname":"x55","tablename":"obs_65f29658e096ca1485bf683f65fdbc9f05ec3c5d","aggregate":null,"name":"Spielman-Singleton Segments: 55 Clusters","type":"Text","description":"Sociodemographic classes from Spielman and Singleton 2015, 55 clusters","boundary_id":"us.census.tiger.census_tract"}' as test_get_obs_column_with_geoid_and_census_1,
(expected)[2]::text = '{"colname":"total_pop","tablename":"obs_b393b5b88c6adda634b2071a8005b03c551b609a","aggregate":"sum","name":"Total Population","type":"Numeric","description":"The total number of all people living in a given geographic area. This is a very useful catch-all denominator when calculating rates.","boundary_id":"us.census.tiger.census_tract"}' as test_get_obs_column_with_geoid_and_census_2
from result; from result;
-- should be null-valued -- should be null-valued
@ -50,7 +45,7 @@ SELECT
array_agg(a) expected from cdb_observatory._OBS_GetColumnData( array_agg(a) expected from cdb_observatory._OBS_GetColumnData(
'us.census.tiger.census_tract', 'us.census.tiger.census_tract',
Array['us.census.tiger.baloney'], Array['us.census.tiger.baloney'],
'2009 - 2013') a '2010 - 2014') a
) )
select expected is null as OBS_GetColumnData_missing_measure select expected is null as OBS_GetColumnData_missing_measure
from result; from result;
@ -60,24 +55,24 @@ from result;
SELECT SELECT
cdb_observatory._OBS_BuildSnapshotQuery( cdb_observatory._OBS_BuildSnapshotQuery(
Array['total_pop','male_pop','female_pop','median_age'] Array['total_pop','male_pop','female_pop','median_age']
); ) = 'SELECT vals[1] As total_pop, vals[2] As male_pop, vals[3] As female_pop, vals[4] As median_age' As _OBS_BuildSnapshotQuery_test_1;
-- should give back: SELECT vals[1] As mandarin_orange -- should give back: SELECT vals[1] As mandarin_orange
SELECT SELECT
cdb_observatory._OBS_BuildSnapshotQuery( cdb_observatory._OBS_BuildSnapshotQuery(
Array['mandarin_orange'] Array['mandarin_orange']
); ) = 'SELECT vals[1] As mandarin_orange' As _OBS_BuildSnapshotQuery_test_2;
SELECT cdb_observatory._OBS_GetRelatedColumn( SELECT cdb_observatory._OBS_GetRelatedColumn(
Array[ Array[
'es.ine.pop_0_4', 'es.ine.pop_0_4',
'us.census.acs.B01001001', 'us.census.acs.B01003001',
'us.census.acs.B01001002' 'us.census.acs.B01001002'
], ],
'denominator' 'denominator'
); ) = '{es.ine.total_pop,NULL,us.census.acs.B01003001}' As _OBS_GetRelatedColumn_test;
-- should give back a standardized measure name -- should give back a standardized measure name
SELECT cdb_observatory._OBS_StandardizeMeasureName('test 343 %% 2 qqq }}{{}}'); SELECT cdb_observatory._OBS_StandardizeMeasureName('test 343 %% 2 qqq }}{{}}') = 'test_343_2_qqq' As _OBS_StandardizeMeasureName_test;
\i test/fixtures/drop_fixtures.sql \i test/fixtures/drop_fixtures.sql

23
src/pg/test/sql/41_observatory_augmentation_test.sql
View File

@ -6,14 +6,9 @@
WITH result as( WITH result as(
Select count(coalesce(OBS_GetDemographicSnapshot->>'value', 'foo')) expected_columns Select count(coalesce(OBS_GetDemographicSnapshot->>'value', 'foo')) expected_columns
FROM cdb_observatory.OBS_GetDemographicSnapshot(cdb_observatory._TestPoint()) FROM cdb_observatory.OBS_GetDemographicSnapshot(cdb_observatory._TestPoint())
) select expected_columns ='59' as OBS_GetDemographicSnapshot_test_no_returns ) select expected_columns = 52 as OBS_GetDemographicSnapshot_test_no_returns
FROM result; FROM result;
--
-- names | vals
-- --------------|-------
-- median_income | 45122
WITH result as ( WITH result as (
SELECT _OBS_Get::text as expected FROM SELECT _OBS_Get::text as expected FROM
cdb_observatory._OBS_Get( cdb_observatory._OBS_Get(
@ -103,13 +98,13 @@ SELECT cdb_observatory.OBS_GetSegmentSnapshot(
cdb_observatory._TestPoint(), cdb_observatory._TestPoint(),
'us.census.tiger.census_tract' 'us.census.tiger.census_tract'
)::text = )::text =
'{"segment_name":"Spielman-Singleton Segments: 10 Clusters","us.census.acs.B01001001_quantile":"0.234783783783784","us.census.acs.B01001002_quantile":"0.422405405405405","us.census.acs.B01001026_quantile":"0.0987567567567568","us.census.acs.B01002001_quantile":"0.0715","us.census.acs.B03002003_quantile":"0.295310810810811","us.census.acs.B03002004_quantile":"0.407189189189189","us.census.acs.B03002006_quantile":"0.625608108108108","us.census.acs.B03002012_quantile":"0.795202702702703","us.census.acs.B05001006_quantile":"0.703797297297297","us.census.acs.B08006001_quantile":"0.59227027027027","us.census.acs.B08006002_quantile":"0.0180540540540541","us.census.acs.B08006008_quantile":"0.993756756756757","us.census.acs.B08006009_quantile":"0.728162162162162","us.census.acs.B08006011_quantile":"0.995972972972973","us.census.acs.B08006015_quantile":"0.929135135135135","us.census.acs.B08006017_quantile":"0.625432432432432","us.census.acs.B09001001_quantile":"0.0386081081081081","us.census.acs.B11001001_quantile":"0.157121621621622","us.census.acs.B14001001_quantile":"0.241878378378378","us.census.acs.B14001002_quantile":"0.173783783783784","us.census.acs.B14001005_quantile":"0.0380675675675676","us.census.acs.B14001006_quantile":"0.0308108108108108","us.census.acs.B14001007_quantile":"0.0486216216216216","us.census.acs.B14001008_quantile":"0.479743243243243","us.census.acs.B15003001_quantile":"0.297675675675676","us.census.acs.B15003017_quantile":"0.190351351351351","us.census.acs.B15003022_quantile":"0.802513513513514","us.census.acs.B15003023_quantile":"0.757148648648649","us.census.acs.B16001001_quantile":"0.255405405405405","us.census.acs.B16001002_quantile":"0.196094594594595","us.census.acs.B16001003_quantile":"0.816851351351351","us.census.acs.B17001001_quantile":"0.252513513513514","us.census.acs.B17001002_quantile":"0.560054054054054","us.census.acs.B19013001_quantile":"0.777472972972973","us.census.acs.B19083001_quantile":"0.336932432432432","us.census.acs.B19301001_quantile":"0.655378378378378","us.census.acs.B25001001_quantile":"0.141810810810811","us.census.acs.B25002003_quantile":"0.362824324324324","us.census.acs.B25004002_quantile":"0.463837837837838","us.census.acs.B25004004_quantile":"0","us.census.acs.B25058001_quantile":"0.939040540540541","us.census.acs.B25071001_quantile":"0.419445945945946","us.census.acs.B25075001_quantile":"0.0387972972972973","us.census.acs.B25075025_quantile":"0"}' as test_point_segmentation; '{"segment_name":"Spielman-Singleton Segments: 10 Clusters","us.census.acs.B01003001_quantile":"0.3235","us.census.acs.B01001002_quantile":"0.494716216216216","us.census.acs.B01001026_quantile":"0.183756756756757","us.census.acs.B01002001_quantile":"0.0752837837837838","us.census.acs.B03002003_quantile":"0.293162162162162","us.census.acs.B03002004_quantile":"0.455527027027027","us.census.acs.B03002006_quantile":"0.656405405405405","us.census.acs.B03002012_quantile":"0.840081081081081","us.census.acs.B05001006_quantile":"0.727135135135135","us.census.acs.B08006001_quantile":"0.688635135135135","us.census.acs.B08006002_quantile":"0.0204459459459459","us.census.acs.B08006008_quantile":"0.679324324324324","us.census.acs.B08006009_quantile":"0.996716216216216","us.census.acs.B08006011_quantile":"0.967418918918919","us.census.acs.B08006015_quantile":"0.512945945945946","us.census.acs.B08006017_quantile":"0.0504864864864865","us.census.acs.B09001001_quantile":"0.192405405405405","us.census.acs.B11001001_quantile":"0.331702702702703","us.census.acs.B14001001_quantile":"0.296283783783784","us.census.acs.B14001002_quantile":"0.045472972972973","us.census.acs.B14001005_quantile":"0.0442702702702703","us.census.acs.B14001006_quantile":"0.0829054054054054","us.census.acs.B14001007_quantile":"0.701135135135135","us.census.acs.B14001008_quantile":"0.404527027027027","us.census.acs.B15003001_quantile":"0.191824324324324","us.census.acs.B15003017_quantile":"0.864162162162162","us.census.acs.B15003022_quantile":"0.754297297297297","us.census.acs.B15003023_quantile":"0.350054054054054","us.census.acs.B16001001_quantile":"0.217635135135135","us.census.acs.B16001002_quantile":"0.85972972972973","us.census.acs.B16001003_quantile":"0.342851351351351","us.census.acs.B17001001_quantile":"0.51204054054054","us.census.acs.B17001002_quantile":"0.813540540540541","us.census.acs.B19013001_quantile":"0.0948648648648649","us.census.acs.B19083001_quantile":"0.678351351351351","us.census.acs.B19301001_quantile":"0.146108108108108","us.census.acs.B25001001_quantile":"0.149067567567568","us.census.acs.B25002003_quantile":"0","us.census.acs.B25004002_quantile":"0","us.census.acs.B25004004_quantile":"0.944554054054054","us.census.acs.B25058001_quantile":"0.398040540540541","us.census.acs.B25071001_quantile":"0.0596081081081081","us.census.acs.B25075001_quantile":"0","us.census.acs.B25075025_quantile":null}' as test_point_segmentation;
-- segmentation around null island -- segmentation around null island
SELECT cdb_observatory.OBS_GetSegmentSnapshot( SELECT cdb_observatory.OBS_GetSegmentSnapshot(
ST_SetSRID(ST_Point(0, 0), 4326), ST_SetSRID(ST_Point(0, 0), 4326),
'us.census.tiger.census_tract' 'us.census.tiger.census_tract'
)::text = '{"segment_name":null,"us.census.acs.B01001001_quantile":null,"us.census.acs.B01001002_quantile":null,"us.census.acs.B01001026_quantile":null,"us.census.acs.B01002001_quantile":null,"us.census.acs.B03002003_quantile":null,"us.census.acs.B03002004_quantile":null,"us.census.acs.B03002006_quantile":null,"us.census.acs.B03002012_quantile":null,"us.census.acs.B05001006_quantile":null,"us.census.acs.B08006001_quantile":null,"us.census.acs.B08006002_quantile":null,"us.census.acs.B08006008_quantile":null,"us.census.acs.B08006009_quantile":null,"us.census.acs.B08006011_quantile":null,"us.census.acs.B08006015_quantile":null,"us.census.acs.B08006017_quantile":null,"us.census.acs.B09001001_quantile":null,"us.census.acs.B11001001_quantile":null,"us.census.acs.B14001001_quantile":null,"us.census.acs.B14001002_quantile":null,"us.census.acs.B14001005_quantile":null,"us.census.acs.B14001006_quantile":null,"us.census.acs.B14001007_quantile":null,"us.census.acs.B14001008_quantile":null,"us.census.acs.B15003001_quantile":null,"us.census.acs.B15003017_quantile":null,"us.census.acs.B15003022_quantile":null,"us.census.acs.B15003023_quantile":null,"us.census.acs.B16001001_quantile":null,"us.census.acs.B16001002_quantile":null,"us.census.acs.B16001003_quantile":null,"us.census.acs.B17001001_quantile":null,"us.census.acs.B17001002_quantile":null,"us.census.acs.B19013001_quantile":null,"us.census.acs.B19083001_quantile":null,"us.census.acs.B19301001_quantile":null,"us.census.acs.B25001001_quantile":null,"us.census.acs.B25002003_quantile":null,"us.census.acs.B25004002_quantile":null,"us.census.acs.B25004004_quantile":null,"us.census.acs.B25058001_quantile":null,"us.census.acs.B25071001_quantile":null,"us.census.acs.B25075001_quantile":null,"us.census.acs.B25075025_quantile":null}' as null_island_segmentation; )::text = '{"segment_name":null,"us.census.acs.B01003001_quantile":null,"us.census.acs.B01001002_quantile":null,"us.census.acs.B01001026_quantile":null,"us.census.acs.B01002001_quantile":null,"us.census.acs.B03002003_quantile":null,"us.census.acs.B03002004_quantile":null,"us.census.acs.B03002006_quantile":null,"us.census.acs.B03002012_quantile":null,"us.census.acs.B05001006_quantile":null,"us.census.acs.B08006001_quantile":null,"us.census.acs.B08006002_quantile":null,"us.census.acs.B08006008_quantile":null,"us.census.acs.B08006009_quantile":null,"us.census.acs.B08006011_quantile":null,"us.census.acs.B08006015_quantile":null,"us.census.acs.B08006017_quantile":null,"us.census.acs.B09001001_quantile":null,"us.census.acs.B11001001_quantile":null,"us.census.acs.B14001001_quantile":null,"us.census.acs.B14001002_quantile":null,"us.census.acs.B14001005_quantile":null,"us.census.acs.B14001006_quantile":null,"us.census.acs.B14001007_quantile":null,"us.census.acs.B14001008_quantile":null,"us.census.acs.B15003001_quantile":null,"us.census.acs.B15003017_quantile":null,"us.census.acs.B15003022_quantile":null,"us.census.acs.B15003023_quantile":null,"us.census.acs.B16001001_quantile":null,"us.census.acs.B16001002_quantile":null,"us.census.acs.B16001003_quantile":null,"us.census.acs.B17001001_quantile":null,"us.census.acs.B17001002_quantile":null,"us.census.acs.B19013001_quantile":null,"us.census.acs.B19083001_quantile":null,"us.census.acs.B19301001_quantile":null,"us.census.acs.B25001001_quantile":null,"us.census.acs.B25002003_quantile":null,"us.census.acs.B25004002_quantile":null,"us.census.acs.B25004004_quantile":null,"us.census.acs.B25058001_quantile":null,"us.census.acs.B25071001_quantile":null,"us.census.acs.B25075001_quantile":null,"us.census.acs.B25075025_quantile":null}' as null_island_segmentation;
WITH result as ( WITH result as (
SELECT array_agg(_OBS_GetCategories) as expected FROM SELECT array_agg(_OBS_GetCategories) as expected FROM
@ -150,15 +145,15 @@ SELECT abs(OBS_GetMeasure_zhvi_point_default_latest - 972900) / 972900 < 0.001 A
SELECT abs(OBS_GetMeasure_total_pop_point - 10923.093200390833950) / 10923.093200390833950 < 0.001 As OBS_GetMeasure_total_pop_point_test FROM SELECT abs(OBS_GetMeasure_total_pop_point - 10923.093200390833950) / 10923.093200390833950 < 0.001 As OBS_GetMeasure_total_pop_point_test FROM
cdb_observatory.OBS_GetMeasure( cdb_observatory.OBS_GetMeasure(
cdb_observatory._TestPoint(), cdb_observatory._TestPoint(),
'us.census.acs.B01001001' 'us.census.acs.B01003001'
) As t(OBS_GetMeasure_total_pop_point); ) As t(OBS_GetMeasure_total_pop_point);
-- Poly-based OBS_GetMeasure, default normalization (none) -- Poly-based OBS_GetMeasure, default normalization (none)
-- is result within 0.1% of expected -- is result within 0.1% of expected
SELECT abs(OBS_GetMeasure_total_pop_polygon - 12327.3133495107) / 12327.3133495107 < 0.001 As OBS_GetMeasure_total_pop_polygon_test FROM SELECT abs(OBS_GetMeasure_total_pop_polygon - 9833.47316573952) / 9833.47316573952 < 0.001 As OBS_GetMeasure_total_pop_polygon_test FROM
cdb_observatory.OBS_GetMeasure( cdb_observatory.OBS_GetMeasure(
cdb_observatory._TestArea(), cdb_observatory._TestArea(),
'us.census.acs.B01001001' 'us.census.acs.B01003001'
) As t(OBS_GetMeasure_total_pop_polygon); ) As t(OBS_GetMeasure_total_pop_polygon);
-- Point-based OBS_GetMeasure with denominator normalization -- Point-based OBS_GetMeasure with denominator normalization
@ -169,7 +164,7 @@ SELECT (abs(cdb_observatory.OBS_GetMeasure(
-- Poly-based OBS_GetMeasure with denominator normalization -- Poly-based OBS_GetMeasure with denominator normalization
SELECT abs(cdb_observatory.OBS_GetMeasure( SELECT abs(cdb_observatory.OBS_GetMeasure(
cdb_observatory._TestArea(), cdb_observatory._TestArea(),
'us.census.acs.B01001002', 'denominator') - 0.49026340444793965457) / 0.49026340444793965457 < 0.001 As OBS_GetMeasure_total_male_poly_denominator; 'us.census.acs.B01001002', 'denominator') - 0.50597531462834994530) / 0.49026340444793965457 < 0.001 As OBS_GetMeasure_total_male_poly_denominator;
-- Point-based OBS_GetCategory -- Point-based OBS_GetCategory
SELECT cdb_observatory.OBS_GetCategory( SELECT cdb_observatory.OBS_GetCategory(
@ -186,7 +181,7 @@ SELECT (abs(OBS_GetPopulation - 10923.093200390833950) / 10923.093200390833950)
) As m(OBS_GetPopulation); ) As m(OBS_GetPopulation);
-- Poly-based OBS_GetPopulation, default normalization (none) -- Poly-based OBS_GetPopulation, default normalization (none)
SELECT (abs(obs_getpopulation_polygon - 12327.3133495107) / 12327.3133495107) < 0.001 As obs_getpopulation_polygon_test SELECT (abs(obs_getpopulation_polygon - 9833.47316573952) / 9833.47316573952) < 0.001 As obs_getpopulation_polygon_test
FROM FROM
cdb_observatory.OBS_GetPopulation( cdb_observatory.OBS_GetPopulation(
cdb_observatory._TestArea() cdb_observatory._TestArea()
@ -198,7 +193,7 @@ SELECT (abs(cdb_observatory.obs_getuscensusmeasure(
-- Poly-based OBS_GetUSCensusMeasure, default normalization (none) -- Poly-based OBS_GetUSCensusMeasure, default normalization (none)
SELECT (abs(cdb_observatory.obs_getuscensusmeasure( SELECT (abs(cdb_observatory.obs_getuscensusmeasure(
cdb_observatory._testarea(), 'male population') - 6043.63061042765) / 6043.63061042765) < 0.001 As obs_getuscensusmeasure; cdb_observatory._testarea(), 'male population') - 4975.49467892449) / 4975.49467892449) < 0.001 As obs_getuscensusmeasure;
-- Point-based OBS_GetUSCensusCategory -- Point-based OBS_GetUSCensusCategory
SELECT cdb_observatory.OBS_GetUSCensusCategory( SELECT cdb_observatory.OBS_GetUSCensusCategory(

8
src/pg/test/sql/42_observatory_exploration_test.sql
View File

@ -9,7 +9,7 @@
-- _OBS_SearchTables tests -- _OBS_SearchTables tests
SELECT SELECT
t.table_name As _OBS_SearchTables_tables_match, t.table_name = 'obs_1babf5a26a1ecda5fb74963e88408f71d0364b81' As _OBS_SearchTables_tables_match,
t.timespan = '2014' As _OBS_SearchTables_timespan_matches t.timespan = '2014' As _OBS_SearchTables_timespan_matches
FROM cdb_observatory._OBS_SearchTables( FROM cdb_observatory._OBS_SearchTables(
'us.census.tiger.county', 'us.census.tiger.county',
@ -24,8 +24,10 @@ FROM cdb_observatory._OBS_SearchTables(
'1988' -- year before first tiger data was collected '1988' -- year before first tiger data was collected
) As t(table_name, timespan); ) As t(table_name, timespan);
SELECT cdb_observatory.OBS_Search('total_pop'); SELECT *
FROM cdb_observatory.OBS_Search('total_pop');
SELECT * from cdb_observatory.OBS_GetAvailableBoundaries(cdb_observatory._TestPoint()); SELECT *
FROM cdb_observatory.OBS_GetAvailableBoundaries(cdb_observatory._TestPoint());
\i test/fixtures/drop_fixtures.sql \i test/fixtures/drop_fixtures.sql