FGFS-4.1/fgmeta
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

FGmeta/Nasa2FGearthview

Browse Source 
-----------------------
* merge contribution of eatdirt <chris@ringeval.com>
  add download, conversion and generation of normalmaps
  Big thanks Chris!
* update README and some minor updates to the script
This commit is contained in:
chris_blues 2019-03-25 18:20:43 +01:00
parent 1cf284bd6f
commit 7b51c99439
2 changed files with 734 additions and 241 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

165
Nasa2FGearthview/README.md
View File

@ -1,96 +1,123 @@
# Nasa2FGearthview
A bash-script to convert NASA satellite images to ready-to-use
textures for FG's EarthView using ImageMagick.
textures for FG's EarthView using ImageMagick and normalmap
For info about FGearthview, see the forum thread:
You can get "normalmap" there:
https://github.com/planrich/normalmap
For info about FGearthview, see the forum thread:\
https://forum.flightgear.org/viewtopic.php?f=6&t=15754
or this FG-wiki-page:
or this FG-wiki-page:\
http://wiki.flightgear.org/Earthview
### Caution!
Don't use this script on a server! It will most likely cause
Denial-of-service (DoS). When working on these huge images, the
harddisk throughput will cease occasionally and CPU / RAM usage will
spike tremendously! So, only use this script on your home desktop
computer, or if you don't mind several long server-outages...
------------------------------------
About:
## About:
This script runs on Linux (maybe Mac also?) in a Bash
(Bourne Again Shell) - Windows is not supported (by the nature of the
script). Maybe it works on windows as well, I don't know, feel free
to try, and please let me know! :)
This will download the raw images from http://visibleearth.nasa.gov -
their server is not very fast, so I provide an alternative download
location: https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z
This one is much quicker! If you really want the images directly from
NASA, then provide "nasa" to the script (see below)
In the end you will have 8 world-textures in .png and .dds format.
Generally .dds is better in performance, but it won't work on some
graphics cards. If this is the case for you, then try the .png files.
For further information see:
For further information see:\
http://wiki.flightgear.org/index.php?title=DDS_Textures_in_FlightGear&redirect=no
If you also converted the clouds, then you'll also find 8 cloud-
textures in the format .png. Because the .dds-format has trouble with
rendering heavy alpha images, which is because of it's compression
algorythm [1], I think it's useless to also build faulty files.
However, this is not entirely true! It is possible to switch off the
.dds/DXT compression. But this results in huge files and is rather
heavy on the GPU's RAM.
If you also converted the clouds and the height maps, then you'll also
find 8 cloud- and 8 height textures (as well as their conversion to
normal maps) in the format .png. Because the .dds-format has trouble
with rendering heavy alpha images, which is because of it's
compression algorythm [1], I think it's useless to also build faulty
files. However, this is not entirely true! It is possible to switch
off the .dds/DXT compression. But this results in huge files and is
rather heavy on the GPU's RAM.
Buckaroo has created a nice overview on dds-compression:
[1] http://www.buckarooshangar.com/flightgear/tut_dds.html
------------------------------------
Installation and usage:
## Installation and usage:
Simply copy "convert.sh" into a folder of your liking and run it:
$ ./convert.sh
```shell
./convert.sh
```
This will show a help text, since you didn't specify any target(s).
Possible targets are:
* world
* clouds
* heights
* all
Additionally, there are some options you could specify (further
explained below):
* 1k | 2k | 4k | 8k | 16k
* nasa
* no-download
* download | no-download
* world
* clouds
* heights
* cleanup
* rebuild
* check
So your call could look sth like this:
$ ./convert.sh world no-download cleanup 8k
```shell
./convert.sh world download alt cleanup 8k
```
------------------------------------
Requirements:
## Requirements:
WARNING!
This script uses a *lot* disk space! Make sure you have at least 90GB
available!
This script uses a *lot* disk space! In my last test run, which
generated all maps in all resolutions, the disk usage was about 330GB!
Beware!\
Also, I wouldn't recommend doing this on a SSD! While SSDs are
generally faster, they also get more wear-and-tear when write such
huge files. So this script might cause your SSD to die earlier as it
should. Generally speaking, this won't kill your SSD, but it might
cause it to die earlier. HDDs are much more robust in that respect.
Also, this script will run for a *very long* time! It might be best to
let it run over night - your computer might become unresponsive from
time to time, due to the heavy CPU and memory load, which tends to
occur, when converting 54000x27000 images. ;-)
I also recommend to deactivate swapping!
$ sudo swapoff -a
To reactivate swapping do:
$ sudo swapon -a
This script relies on wget and imagemagick. Both are easily installed
by your systems package-management-system.
(On Debian/Ubuntu this is "apt-get")
I also recommend to deactivate swapping!
```shell
sudo swapoff -a
```
To reactivate swapping do:
```shell
$ sudo swapon -a
```
This script relies on wget, ImageMagick and, for converting the height
maps to normal maps, on "normalmap". Some of these programs are easily
installed by your systems package-management-system.\ (On
Debian/Ubuntu this is "apt-get").
So, on Debian for instance, you only need to put the following into
the console:
$ sudo apt-get install wget imagemagick
```shell
sudo apt-get install wget imagemagick
```
Depending on your distro, the package names might differ slightly! Use
a search engine of your choice to find out, how the packages are named
@ -98,55 +125,77 @@ in your distro!
You may want to check:
$ apt search imagemagick
```shell
apt search imagemagick
```
### IMPORTANT!
Check out your ```/etc/ImageMagick-6/policy.xml```
On some distros, there are limits set, which will cause IM to abort
the conversion of images larger than
[rediculously small images](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860763).
Edit and set to our needs:
* width: at least 55000
* height: at least 55000
* area: less than your free RAM
### Normalmap
For normalmap, you can download and compile it from
https://github.com/planrich/normalmap
You can install the binary into your system, or just copy it next to
convert.sh - both should work.
------------------------------------
Targets:
## Targets:
world
**world**\
Generates the world tiles, needed to run FG with EarthView.
You will find the results in output/[$resolution]/*. Copy
these into $FGDATA/Models/Astro/*. More about the installation
You will find the results in output/[$resolution]/\*. Copy
these into $FGDATA/Models/Astro/\*. More about the installation
of these textures can be found here:
http://wiki.flightgear.org/Earthview#Customization
clouds
**clouds**\
Generates the cloud tiles, needed to run FG with EarthView.
The locations are the same as the other textures mentioned
above. Note that clouds are only available with up to 8k
resolution, due to the available data at NASA.
all
**heights**\
Generates the height tiles, which are then converted to the
normal maps needed to run FG with EarthView. The locations are
the same as the other textures mentioned above. Note that
heights are only available with up to 8k resolution, due to the
available data at NASA.
**all**\
Converts everything needed for a full-blown earthview texture
set. Does the same as:
$ ./convert.sh world clouds
```./convert.sh world clouds heights```
Options:
## Options:
1k | 2k | 4k | 8k | 16k
**1k | 2k | 4k | 8k | 16k**\
Lets you specify a desired resolution of the textures.
Possible values are 1k, 2k, 4k, 8k and 16k. If nothing is
specified, the script will generate all of the resolutions.
16k is only available for earth textures.
16k is recommended only for earth textures, it will induce
oversampling from clouds and height maps.
nasa
Causes the script to download directly from
http://visibleearth.nasa.gov . If omitted the script will
download from
https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z
which is much faster!
Uses wget either way.
**download**\
Causes the script to download the needed data, this is the
default behavior (and can be omitted).
no-download
**no-download**\
Causes the script to skip the download function. If you
already have the source images, then you don't need to
re-download them. (About 2.4GB!)
If omitted, the script will download the source images from
https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z
the default location.
cleanup
**cleanup**\
Deletes the temporary files created during texture generation.
These can be found in tmp/
Note: if for some reason you later want some other resolution,
@ -155,13 +204,13 @@ cleanup
Frees up a lot of disk-space! Which would have to be
regenerated if needed again.
rebuild
**rebuild**\
Deletes only the temporary files of the given target. So if
you call './convert.sh rebuild world' the script will delete
you call ```./convert.sh rebuild world``` the script will delete
all corresponding temp-files of the target world, which will
trigger a complete regeneration of the relevant (instead of
skipping existing files)
check
**check**\
Creates mosaics of the tiles, so you can look at them and see
if all went well.

592
Nasa2FGearthview/convert.sh
View File

@ -1,7 +1,8 @@
#!/bin/bash
# convert.sh - a script to convert NASA sattelite images into textures for use with FGearthview (orbital rendering)
# Copyright (C) 2016 Chris Schwarz <chris@musicchris.de>
# convert.sh - a script to convert NASA sattelite images into textures
# for use with FGearthview (orbital rendering)
# Copyright (C) 2016 chris_blues <chris@musicchris.de>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@ -16,9 +17,12 @@
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# v0.12: Chris Ringeval (eatdirt):
# -add normalmapping output + small improvements
VERSION="v0.11"
VERSION="v0.13"
# make sure the script halts on error
set -e
@ -29,21 +33,21 @@ function showHelp
echo "https://github.com/chris-blues/Nasa2FGearthview"
echo
echo "Usage:"
echo "./convert.sh [ download world clouds 8k cleanup rebuild ]"
echo "./convert.sh [ download no-download world clouds heights"
echo " 1k 2k 4k 8k 16k cleanup rebuild ]"
echo
echo "* Append \"nasa\" to download the needed images from NASA"
echo " -> This will download ca 2.4GB of data!"
echo " -> wget can continue interrupted downloads!"
echo " If omitted, it will download from my server, which is a lot"
echo " faster. See README for details."
echo "* Append \"no-download\" to the command to skip the download"
echo " process alltogether. Only makes sense if you already got"
echo " the necessary data."
echo "* Append \"world\" to the command to generate the world tiles"
echo "* Append \"clouds\" to the command to generate cloud tiles"
echo "* Append \"all\" to the command to generate both - world and"
echo " clouds"
echo "* Append the size of the tiles (1k, 2k, 4, 8k, 16k). If you"
echo "* Append \"heights\" to the command to generate height tiles"
echo " and the normalmaps needed by EarthView. Notice that you need"
echo " the normalmap binary to be installed. You can get it from:"
echo " https://github.com/planrich/normalmap"
echo "* Append \"all\" to the command to generate all - world, clouds"
echo " and heights"
echo "* Append the size of the tiles (1k, 2k, 4k, 8k, 16k). If you"
echo " don't pass a resolution, then all resolutions will be"
echo " generated."
echo "* Append \"cleanup\" to delete all temporary files in tmp/"
@ -59,7 +63,7 @@ function showHelp
echo " Useful if the source files have changed."
echo "* Append \"check\" to let check the results. This will create"
echo " mosaics of the existing tiles. If no target is specified,"
echo " all 3 layers will be built: clouds world and nightlights."
echo " all layers will be built: clouds, heights, world and nightlights."
echo
echo "If, for some reason, the script aborts, then it will try to"
echo "skip the already completed steps, so you don't have to wait"
@ -102,11 +106,11 @@ if [ $1 == "-h" ] ; then showHelp ; fi
################################
for ARG in "$@"
do
if [ $ARG == "nasa" ] ; then DOWNLOAD="true" ; DL_LOCATION="NASA" ; echo "Downloading from visibleearth.nasa.gov" ; fi
if [ $ARG == "no-download" ] ; then DOWNLOAD="false" ; echo "Skipping the download process" ; fi
if [ $ARG == "world" ] ; then WORLD="true" ; fi
if [ $ARG == "clouds" ] ; then CLOUDS="true" ; fi
if [ $ARG == "all" ] ; then WORLD="true" ; CLOUDS="true" ; fi
if [ $ARG == "heights" ] ; then HEIGHTS="true" ; fi
if [ $ARG == "all" ] ; then WORLD="true" ; CLOUDS="true" ; HEIGHTS="true" ; fi
if [ $ARG == "1k" ] ; then RESOLUTION="1024" ; fi
if [ $ARG == "2k" ] ; then RESOLUTION="2048" ; fi
if [ $ARG == "4k" ] ; then RESOLUTION="4096" ; fi
@ -119,17 +123,22 @@ done
if [ -z $DOWNLOAD ] ; then DOWNLOAD="true" ; fi
if [ -z $WORLD ] ; then WORLD="false" ; fi
if [ -z $CLOUDS ] ; then CLOUDS="false" ; fi
if [ -z $HEIGHTS ] ; then HEIGHTS="false" ; fi
if [ -z $CLEANUP ] ; then CLEANUP="false" ; fi
if [ -z $REBUILD ] ; then REBUILD="false" ; fi
if [ -z $BUILDCHECKS ] ; then BUILDCHECKS="false" ; fi
CHECKWORLD=$WORLD
CHECKCLOUDS=$CLOUDS
CHECKHEIGHTS=$HEIGHTS
########################
## Set some variables ##
########################
DL_LOCATION="NASA"
mkdir -p tmp
export MAGICK_TMPDIR=${PWD}/tmp
echo "tmp-dir: $MAGICK_TMPDIR"
@ -139,21 +148,50 @@ TIME=$(date +"%Y-%m-%d_%H:%M:%S")
LOGFILE_GENERAL="logs/${TIME}.log"
LOGFILE_TIME="logs/${TIME}.time.log"
URLS_WORLD="http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.A1.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.A2.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.B1.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.B2.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.C1.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.C2.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.D1.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.D2.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/79000/79765/dnb_land_ocean_ice.2012.54000x27000_geo.tif"
#command line gimp plugin from https://github.com/planrich/normalmap
#higher filters (5x5) create too sharp features (no rescaling, I
#assume earthview do its own normalization)
NORMALBIN="normalmap"
NORMALOPTS="-s 1 -f FILTER_PREWITT_3x3"
URLS_CLOUDS="http://eoimages.gsfc.nasa.gov/images/imagerecords/57000/57747/cloud.E.2001210.21600x21600.png
http://eoimages.gsfc.nasa.gov/images/imagerecords/57000/57747/cloud.W.2001210.21600x21600.png"
URLS_WORLD="https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.A1.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.A2.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.B1.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.B2.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.C1.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.C2.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.D1.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/74000/74117/world.200408.3x21600x21600.D2.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/79000/79765/dnb_land_ocean_ice.2012.54000x27000_geo.tif"
URLS_CLOUDS="https://eoimages.gsfc.nasa.gov/images/imagerecords/57000/57747/cloud.E.2001210.21600x21600.png
https://eoimages.gsfc.nasa.gov/images/imagerecords/57000/57747/cloud.W.2001210.21600x21600.png"
URLS_HEIGHTS="https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_A1_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_A2_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_B1_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_B2_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_C1_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_C2_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_D1_grey_geo.tif
https://eoimages.gsfc.nasa.gov/images/imagerecords/73000/73934/gebco_08_rev_elev_D2_grey_geo.tif"
if ! [ -x "$(command -v $NORMALBIN)" ]
then
if ! [ -x "./${NORMALBIN}" ]
then
echo ">>>>>>>>>>>> Error: $NORMALBIN binary not found! <<<<<<<<<<<<<"
echo "You can get it from: https://github.com/planrich/normalmap"
HEIGHTS="false"
else
NORMALBIN="./${NORMALBIN}"
fi
else
echo ">>>>>>>>>>>> Error: $NORMALBIN binary not found! <<<<<<<<<<<<<"
echo "You can get it from: https://github.com/planrich/normalmap"
HEIGHTS="false"
fi
ALTERNATE_URL="https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z"
ALTERNATE_FILENAME="raw-data-NASA.7z"
if [ -z $RESOLUTION ]
then
@ -162,7 +200,7 @@ if [ -z $RESOLUTION ]
4096
8192
16384"
NO_RESOLUTION_GIVEN="true"
NO_RESOLUTION_GIVEN="false"
RESOLUTION_MAX="16384"
fi
if [ -z $RESOLUTION_MAX ] ; then RESOLUTION_MAX=$RESOLUTION ; fi
@ -236,6 +274,18 @@ function rebuild
rm tmp/cloud*
}
fi
if [ $HEIGHTS == "true" ]
then
{
echo
echo "#########################################"
echo "## Removing tmp-files of target heights##"
echo "#########################################"
rm tmp/height*
}
fi
}
function cleanUp
@ -244,9 +294,10 @@ function cleanUp
echo "############################"
echo "## Removing all tmp-files ##"
echo "############################"
rm -rv tmp/night*
rm -rv tmp/world*
rm -rv tmp/clouds*
rm -rvf tmp/night*
rm -rvf tmp/world*
rm -rvf tmp/cloud*
rm -rvf tmp/height*
}
function prettyTime
@ -312,13 +363,18 @@ function downloadImages
{
echo | tee -a $LOGFILE_GENERAL
echo "###################################################" | tee -a $LOGFILE_GENERAL
if [ ! -z $DL_LOCATION ]
then echo "## Downloading images from visibleearth.nasa.gov ##" | tee -a $LOGFILE_GENERAL
else echo "## Downloading images from musicchris.de ##" | tee -a $LOGFILE_GENERAL
if [ -z $DL_LOCATION ]
then
DL_LOCATION="NASA"
fi
if [ $DL_LOCATION == "NASA" ]
then
echo "## Downloading images from visibleearth.nasa.gov ##" | tee -a $LOGFILE_GENERAL
fi
echo "###################################################" | tee -a $LOGFILE_GENERAL
if [ -z $DL_LOCATION ] ; then f=$ALTERNATE_URL ; DL_LOCATION="musicchris.de" ; fi
FILENAME=$(echo $f | sed 's@.*/@@')
if [ $WORLD == "true" ]
then
if [ $DL_LOCATION == "NASA" ]
@ -335,7 +391,13 @@ function downloadImages
fi
fi
if [ $DL_LOCATION == "musicchris.de" ] ; then downloadMusicchris ; fi
if [ $HEIGHTS == "true" ]
then
if [ $DL_LOCATION == "NASA" ]
then
downloadHeights
fi
fi
}
function downloadWorld
@ -345,10 +407,23 @@ function downloadWorld
for f in $URLS_WORLD
do
FILENAME=$(echo $f | sed 's@.*/@@')
wget --output-document=input/$FILENAME --continue --show-progress $f 2>> $LOGFILE_GENERAL
wget --output-document=input/$FILENAME --continue --show-progress $f | tee -a $LOGFILE_GENERAL 2>> $LOGFILE_GENERAL
done
}
function downloadHeights
{
mkdir -p input
echo "Downloading height tiles..." | tee -a $LOGFILE_GENERAL
for f in $URLS_HEIGHTS
do
FILENAME=$(echo $f | sed 's@.*/@@')
wget --output-document=input/$FILENAME --continue --show-progress $f | tee -a $LOGFILE_GENERAL 2>> $LOGFILE_GENERAL
done
}
function downloadClouds
{
mkdir -p input
@ -356,24 +431,10 @@ function downloadClouds
for f in $URLS_CLOUDS
do
FILENAME=$(echo $f | sed 's@.*/@@')
wget --output-document=input/$FILENAME --continue --show-progress $f 2>> $LOGFILE_GENERAL
wget --output-document=input/$FILENAME --continue --show-progress $f | tee -a $LOGFILE_GENERAL 2>> $LOGFILE_GENERAL
done
}
function downloadMusicchris
{
mkdir -p input
echo "Downloading raw images... (ca 2.2 GB)" | tee -a $LOGFILE_GENERAL
for p in 1 2 3 4 5
do
wget --output-document=input/${FILENAME}.00${p} --continue --show-progress $ALTERNATE_URL | tee -a $LOGFILE_GENERAL 2>> $LOGFILE_GENERAL
done
echo "Unpacking raw images..." | tee -a $LOGFILE_GENERAL
cd input
7z e -bt -y raw-data-NASA.7z 2>> $LOGFILE_GENERAL
cd ..
}
function generateWorld
{
STARTTIME=$(date +%s)
@ -419,7 +480,7 @@ function generateWorld
let "RESIZE_H = ( $RESOLUTION_MAX - ( 2 * $BORDER_WIDTH ) ) * 2"
if [ ! -s "tmp/nightlights_${RESIZE_W}x${RESIZE_H}.mpc" ]
then
for r in 16384 8192 4096 2048
for r in $RESOLUTION
do
if [ $r -le $RESOLUTION_MAX ]
then continue
@ -553,7 +614,7 @@ function generateWorld
do
NASA2FG $t
FOUND_BIGGER_WORLD_PICTURE="false"
unset $TIMESAVER_SIZE
unset TIMESAVER_SIZE
if [ ! -s "tmp/world_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc" ]
then
{
@ -825,6 +886,10 @@ function generateWorld
echo
echo "--> Writing output/${r}/world_${t}.dds @ ${r}x${r}"
# set -x
if [ ! -s "output/${r}/world_${t}.dds" ]
then
convert \
-monitor \
tmp/world_${RESOLUTION_MAX}_done_${t}.mpc \
@ -834,7 +899,15 @@ function generateWorld
output/${r}/world_${t}.dds
set +x
echo
else echo "=> Skipping existing file: output/${r}/world_${t}.dds" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
echo "--> Writing output/${r}/world_${t}.png @ ${r}x${r}"
if [ ! -s "output/${r}/world_${t}.png" ]
then
# set -x
convert \
-monitor \
@ -843,6 +916,10 @@ function generateWorld
output/${r}/world_${t}.png
set +x
echo
else echo "=> Skipping existing file: output/${r}/world_${t}.png" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
}
done
@ -874,7 +951,11 @@ function generateClouds
{
if [ -z $STARTTIME ] ; then STARTTIME=$(date +%s) ; fi
# maximum cloud-tile resolution is 8192, since we have no big enough source files...
if [[ $NO_RESOLUTION_GIVEN == "true" ]]; then
if [ $RESOLUTION_MAX -eq 16384 ] ; then RESOLUTION_MAX=8192 ; fi
fi
let "BORDER_WIDTH = $RESOLUTION_MAX / 128"
let "IMAGE_BORDERLESS = $RESOLUTION_MAX - ( 2 * $BORDER_WIDTH )"
let "IMAGE_WITH_BORDER = $RESOLUTION_MAX - $BORDER_WIDTH - 1"
@ -1071,7 +1152,9 @@ W"
echo "## propagate last pixels to the edge ##" | tee -a $LOGFILE_GENERAL
echo "#######################################" | tee -a $LOGFILE_GENERAL
if [[ $NO_RESOLUTION_GIVEN == "true" ]]; then
if [ $RESOLUTION_MAX -eq 16384 ] ; then RESOLUTION_MAX=8192 ; fi
fi
let "BORDER_WIDTH = $RESOLUTION_MAX / 128"
let "IMAGE_BORDERLESS = $RESOLUTION_MAX - ( 2 * $BORDER_WIDTH )"
let "IMAGE_WITH_BORDER = $RESOLUTION_MAX - $BORDER_WIDTH - 1"
@ -1190,23 +1273,22 @@ W"
for r in $RESOLUTION
do
{
if [ $r -eq 16384 ]
then
if [ $NO_RESOLUTION_GIVEN == "true" ]
then
continue
else
r=8192
fi
fi
mkdir -p output/$r
echo
if [ ! -s "output/${r}/clouds_${t}.png" ]
then
echo "--> Writing output/${r}/clouds_${t}.png @ ${r}x${r}" | tee -a $LOGFILE_GENERAL
convert \
-monitor \
tmp/clouds_${RESOLUTION_MAX}_${t}_done.mpc \
-resize ${r}x${r} \
output/${r}/clouds_${t}.png
echo
else echo "=> Skipping existing file: output/${r}/clouds_${t}.png" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
}
done
@ -1231,6 +1313,322 @@ W"
echo "Overall processing time: $DAYS d, $HOURS h, $MINUTES m, $SECS s" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
}
function generateHeights
{
if [ -z $STARTTIME ] ; then STARTTIME=$(date +%s) ; fi
echo | tee -a $LOGFILE_GENERAL
echo "################################" | tee -a $LOGFILE_GENERAL
echo "#### Processing Heights ####" | tee -a $LOGFILE_GENERAL
echo "################################" | tee -a $LOGFILE_GENERAL
echo | tee -a $LOGFILE_GENERAL
if [[ $NO_RESOLUTION_GIVEN == "true" ]]; then
if [ $RESOLUTION_MAX -eq 16384 ] ; then RESOLUTION_MAX=8192 ; fi
fi
let "BORDER_WIDTH = $RESOLUTION_MAX / 128"
let "IMAGE_BORDERLESS = $RESOLUTION_MAX - ( 2 * $BORDER_WIDTH )"
let "IMAGE_WITH_BORDER = $RESOLUTION_MAX - $BORDER_WIDTH - 1"
let "IMAGE_WITH_BORDER_POS = $RESOLUTION_MAX - $BORDER_WIDTH"
let "SIZE = 2 * $IMAGE_BORDERLESS"
mkdir -p tmp
mkdir -p output
echo "################################################" | tee -a $LOGFILE_GENERAL
echo "## Resize the NASA-Originals to ${RESOLUTION_MAX}-(2*${BORDER_WIDTH}) ##" | tee -a $LOGFILE_GENERAL
echo "################################################" | tee -a $LOGFILE_GENERAL
for t in $NASA
do
NASA2FG $t
FOUND_BIGGER_WORLD_PICTURE="false"
unset TIMESAVER_SIZE
if [ ! -s "tmp/heights_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc" ]
then
{
for r in $RESOLUTION
do
if [ $r -le $RESOLUTION_MAX ]
then
continue
fi
let "IMAGE_SIZE = $r - ( 2 * ( $r / 128 ) )"
let "I_W = $IMAGE_SIZE * 4"
let "I_H = $IMAGE_SIZE * 2"
if [ -s tmp/heights_seamless_${IMAGE_SIZE}_${DEST}.mpc ]
then
if [ $IMAGE_SIZE -ge $IMAGE_BORDERLESS ]
then
FOUND_BIGGER_WORLD_PICTURE="true"
TIMESAVER_SIZE="$IMAGE_SIZE"
fi
fi
done
if [ $FOUND_BIGGER_WORLD_PICTURE != "true" ]
then
# set -x
convert \
-monitor \
-limit memory 32 \
-limit map 32 \
input/gebco_08_rev_elev_${t}_grey_geo.tif \
-resize ${IMAGE_BORDERLESS}x${IMAGE_BORDERLESS} \
tmp/heights_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc
set +x
else
echo "==> Timesaver:) Using existing file: tmp/heights_seamless_${TIMESAVER_SIZE}_${DEST}.mpc -> tmp/heights_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc" | tee -a $LOGFILE_GENERAL
set -x
convert \
-monitor \
-limit memory 32 \
-limit map 32 \
tmp/heights_seamless_${TIMESAVER_SIZE}_${DEST}.mpc \
-resize ${IMAGE_BORDERLESS}x${IMAGE_BORDERLESS} \
tmp/heights_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc
set +x
fi
}
else echo "=> Skipping existing file: tmp/heights_seamless_${IMAGE_BORDERLESS}_${DEST}.mpc" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
done
# 3h, 12m, 9s
if [ -z $LASTTIME ] ; then LASTTIME=$STARTTIME ; fi
echo "input/gebco_08_rev_elev_[A-D][12]_grey_geo.tif -> tmp/heights_seamless_${IMAGE_BORDERLESS}_[NS][1-4].mpc" >> $LOGFILE_TIME
getProcessingTime
echo | tee -a $LOGFILE_GENERAL
echo "#####################################" | tee -a $LOGFILE_GENERAL
echo "## Put a ${BORDER_WIDTH}px border to each side ##" | tee -a $LOGFILE_GENERAL
echo "#####################################" | tee -a $LOGFILE_GENERAL
for t in $TILES
do
if [ ! -s "tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.mpc" ]
then
# set -x
convert \
-monitor \
tmp/heights_seamless_${IMAGE_BORDERLESS}_${t}.mpc \
-bordercolor none \
-border ${BORDER_WIDTH} \
tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.mpc
set +x
echo
fi
if [ ! -s "tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc" ]
then
# set -x
cp tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.mpc tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc
cp tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.cache tmp/heights_seams_${RESOLUTION_MAX}_${t}.cache
set +x
else echo "=> Skipping existing file: tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
done
# 11m, 24s
echo "-> tmp/heights_seams_${RESOLUTION_MAX}_[NS][1-4]_emptyBorder.mpc -> tmp/heights_seams_${RESOLUTION_MAX}_[NS][1-4].mpc" >> $LOGFILE_TIME
getProcessingTime
echo | tee -a $LOGFILE_GENERAL
echo "######################################################" | tee -a $LOGFILE_GENERAL
echo "## crop borderline pixels and propagate to the edge ##" | tee -a $LOGFILE_GENERAL
echo "######################################################" | tee -a $LOGFILE_GENERAL
if [[ $NO_RESOLUTION_GIVEN == "true" ]]; then
if [ $RESOLUTION_MAX -eq 16384 ] ; then RESOLUTION_MAX=8192 ; fi
fi
let "BORDER_WIDTH = $RESOLUTION_MAX / 128"
let "IMAGE_BORDERLESS = $RESOLUTION_MAX - ( 2 * $BORDER_WIDTH )"
let "IMAGE_WITH_BORDER = $RESOLUTION_MAX - $BORDER_WIDTH - 1"
let "IMAGE_WITH_BORDER_POS = $RESOLUTION_MAX - $BORDER_WIDTH"
let "SIZE = 2 * $IMAGE_BORDERLESS"
CROP_TOP="${IMAGE_BORDERLESS}x1+${BORDER_WIDTH}+${BORDER_WIDTH}"
CROP_RIGHT="1x${IMAGE_BORDERLESS}+${IMAGE_WITH_BORDER}+${BORDER_WIDTH}"
CROP_BOTTOM="${IMAGE_BORDERLESS}x1+${BORDER_WIDTH}+${IMAGE_WITH_BORDER}"
CROP_LEFT="1x${IMAGE_BORDERLESS}+${BORDER_WIDTH}+${BORDER_WIDTH}"
CROP_TOPLEFT="1x1+${BORDER_WIDTH}+${BORDER_WIDTH}"
CROP_TOPRIGHT="1x1+${IMAGE_WITH_BORDER}+${BORDER_WIDTH}"
CROP_BOTTOMRIGHT="1x1+${IMAGE_WITH_BORDER}+${IMAGE_WITH_BORDER}"
CROP_BOTTOMLEFT="1x1+${BORDER_WIDTH}+${IMAGE_WITH_BORDER}"
## HORIZ meaning a horizontal bar, like the one on top
HORIZ_RESIZE="${IMAGE_BORDERLESS}x${BORDER_WIDTH}"
VERT_RESIZE="${BORDER_WIDTH}x${IMAGE_BORDERLESS}"
POS_TOP="+${BORDER_WIDTH}+0"
POS_RIGHT="+${IMAGE_WITH_BORDER_POS}+${BORDER_WIDTH}"
POS_BOTTOM="+${BORDER_WIDTH}+${IMAGE_WITH_BORDER_POS}"
POS_LEFT="+0+${BORDER_WIDTH}"
for t in $TILES
do
if [ ! -s "tmp/heights_${RESOLUTION_MAX}_done_${t}.mpc" ]
then
for b in $BORDERS
do
{
if [ $b == "top" ]
then
CROP=$CROP_TOP
RESIZE=$HORIZ_RESIZE
POSITION=$POS_TOP
CROPCORNER=$CROP_TOPRIGHT
CORNER_POS="+${IMAGE_WITH_BORDER_POS}+0"
CORNER_NAME="topRight"
fi
if [ $b == "right" ]
then
CROP=$CROP_RIGHT
RESIZE=$VERT_RESIZE
POSITION=$POS_RIGHT
CROPCORNER=$CROP_BOTTOMRIGHT
CORNER_POS="+${IMAGE_WITH_BORDER_POS}+${IMAGE_WITH_BORDER_POS}"
CORNER_NAME="bottomRight"
fi
if [ $b == "bottom" ]
then
CROP=$CROP_BOTTOM
RESIZE=$HORIZ_RESIZE
POSITION=$POS_BOTTOM
CROPCORNER=$CROP_BOTTOMLEFT
CORNER_POS="+0+${IMAGE_WITH_BORDER_POS}"
CORNER_NAME="bottomLeft"
fi
if [ $b == "left" ]
then
CROP=$CROP_LEFT
RESIZE=$VERT_RESIZE
POSITION=$POS_LEFT
CROPCORNER=$CROP_TOPLEFT
CORNER_POS="+0+0"
CORNER_NAME="topLeft"
fi
echo
# set -x
convert \
-monitor \
tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.mpc \
-crop $CROP \
-resize $RESIZE\! \
tmp/heights_${RESOLUTION_MAX}_${t}_seam_${b}.mpc
convert \
-monitor \
tmp/heights_seams_${RESOLUTION_MAX}_${t}_emptyBorder.mpc \
-crop $CROPCORNER \
-resize ${BORDER_WIDTH}x${BORDER_WIDTH}\! \
tmp/heights_${RESOLUTION_MAX}_${t}_seam_${CORNER_NAME}.mpc
convert \
-monitor \
tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc \
tmp/heights_${RESOLUTION_MAX}_${t}_seam_${b}.mpc \
-geometry $POSITION \
-composite \
tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc
echo
convert \
-monitor \
tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc \
tmp/heights_${RESOLUTION_MAX}_${t}_seam_${CORNER_NAME}.mpc \
-geometry $CORNER_POS \
-composite \
tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc
set +x
echo
}
done
echo
# set -x
cp -v tmp/heights_seams_${RESOLUTION_MAX}_${t}.mpc tmp/heights_${RESOLUTION_MAX}_done_${t}.mpc | tee -a $LOGFILE_GENERAL
cp -v tmp/heights_seams_${RESOLUTION_MAX}_${t}.cache tmp/heights_${RESOLUTION_MAX}_done_${t}.cache | tee -a $LOGFILE_GENERAL
set +x
else echo "=> Skipping existing file: tmp/heights_${RESOLUTION_MAX}_done_${t}.mpc" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
done
# 37m, 6s
echo "-> tmp/heights_seams_${RESOLUTION_MAX}_[NS][1-4].mpc -> tmp/heights_${RESOLUTION_MAX}_done_[NS][1-4].mpc" >> $LOGFILE_TIME
getProcessingTime
for t in $TILES
do
echo | tee -a $LOGFILE_GENERAL
echo "#############################" | tee -a $LOGFILE_GENERAL
echo "## Final output of tile $t ##" | tee -a $LOGFILE_GENERAL
echo "## and normalmapping ##" | tee -a $LOGFILE_GENERAL
echo "#############################" | tee -a $LOGFILE_GENERAL
for r in $RESOLUTION
do
{
mkdir -p output/$r
set +x
echo
echo "--> Writing output/${r}/heights_${t}.png @ ${r}x${r}"
# set -x
if [ ! -s "output/${r}/heights_${t}.png" ]
then
convert \
-monitor \
tmp/heights_${RESOLUTION_MAX}_done_${t}.mpc \
-resize ${r}x${r} \
output/${r}/heights_${t}.png
echo
else echo "=> Skipping existing file: tmp/heights_${RESOLUTION_MAX}_done_${t}.mpc" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
echo "--> Writing output/${r}/normalmap_earth_${t}.png @ ${r}x${r}"
if [ ! -s "output/${r}/normalmap_earth_${t}.png" ]
then
$NORMALBIN $NORMALOPTS output/${r}/heights_${t}.png output/${r}/normalmap_earth_${t}.png
else echo "=> Skipping existing file: output/${r}/normalmap_earth_${t}.png" | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
fi
set +x
echo
}
done
echo | tee -a $LOGFILE_GENERAL
echo "Heights and Normal $t [ done ]" | tee -a $LOGFILE_GENERAL
echo | tee -a $LOGFILE_GENERAL
done
echo "###############################" | tee -a $LOGFILE_GENERAL
echo "#### Heights: [ done ] ####" | tee -a $LOGFILE_GENERAL
echo "###############################" | tee -a $LOGFILE_GENERAL
# 2h, 19m, 7s
# Overall processing time: 44089 s
# Overall processing time: 0 d, 2 h, 19 m, 7 s
echo "-> output/<\$RESOLUTIONS>/heights_[NS][1-4].png" >> $LOGFILE_TIME
getProcessingTime
echo
if [ $STARTTIME -eq $ENDTIME ]
then SECS=0
else let "SECS = $ENDTIME - $STARTTIME"
fi
echo "Overall processing time: $SECS s" | tee -a $LOGFILE_GENERAL
prettyTime
echo "Overall processing time: $DAYS d, $HOURS h, $MINUTES m, $SECS s" | tee -a $LOGFILE_GENERAL
}
function checkResults
{
echo | tee -a $LOGFILE_GENERAL
@ -1251,7 +1649,7 @@ function checkResults
let "HEIGHT = 2 * $RES"
echo "Lowest available resolution is: $RES" | tee -a $LOGFILE_GENERAL
if [ $CHECKCLOUDS == "true" ]
if [[ $CHECKCLOUDS == "true" ]]
then
{
echo "checking clouds..." | tee -a $LOGFILE_GENERAL
@ -1292,7 +1690,51 @@ function checkResults
}
fi
if [ $CHECKWORLD == "true" ]
if [[ $CHECKHEIGHTS == "true" ]]
then
{
echo "checking heights..." | tee -a $LOGFILE_GENERAL
echo | tee -a $LOGFILE_GENERAL
echo "Creating canvas ${WIDTH}x${HEIGHT}" | tee -a $LOGFILE_GENERAL
convert \
-size ${WIDTH}x${HEIGHT} \
xc:Black \
-alpha on \
check_heights.png
POS=0
for t in 1 2 3 4
do
convert \
-monitor \
check_heights.png \
output/${RES}/heights_N${t}.png \
-geometry +${POS}+0 \
-composite \
check_heights.png
echo
convert \
-monitor \
check_heights.png \
output/${RES}/heights_S${t}.png \
-geometry +${POS}+${RES} \
-composite \
check_heights.png
echo
let "POS += $RES"
done
mogrify \
-monitor \
-resize 4096x2048 \
check_heights.png
}
fi
if [[ $CHECKWORLD == "true" ]]
then
{
echo "checking world..." | tee -a $LOGFILE_GENERAL
@ -1366,6 +1808,7 @@ echo $TIME | tee -a $LOGFILE_TIME
echo | tee -a $LOGFILE_GENERAL
printf "Target: " | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
if [ $CLOUDS == "true" ] ; then printf "clouds " | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME ; fi
if [ $HEIGHTS == "true" ] ; then printf "heights " | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME ; fi
if [ $WORLD == "true" ] ; then printf "world " | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME ; fi
echo | tee -a $LOGFILE_GENERAL | tee -a $LOGFILE_TIME
echo "Will work in ${RESOLUTION_MAX}x${RESOLUTION_MAX} resolution and will output" | tee -a $LOGFILE_GENERAL
@ -1377,12 +1820,13 @@ echo "--------------------------------------------------------------" | tee -a $
echo | tee -a $LOGFILE_GENERAL
if [ $REBUILD == "true" ] ; then rebuild ; fi
if [ $DOWNLOAD == "true" ] ; then downloadImages ; fi
if [ $WORLD == "true" ] ; then generateWorld ; fi
if [ $CLOUDS == "true" ] ; then generateClouds ; fi
if [ $BUILDCHECKS == "true" ] ; then checkResults ; fi
if [ $CLEANUP == "true" ] ; then cleanUp ; fi
if [[ $REBUILD == "true" ]] ; then rebuild ; fi
if [[ $DOWNLOAD == "true" ]] ; then downloadImages ; fi
if [[ $WORLD == "true" ]] ; then generateWorld ; fi
if [[ $CLOUDS == "true" ]] ; then generateClouds ; fi
if [[ $HEIGHTS == "true" ]]; then generateHeights; fi
if [[ $BUILDCHECKS == "true" ]] ; then checkResults ; fi
if [[ $CLEANUP == "true" ]] ; then cleanUp ; fi