Mirror GHS-composite-S2¶
This section covers mirroring locally the UK section of the cloud-free composite Sentinel-2 mosaic created by the European Commission. Official website is over at:
https://ghsl.jrc.ec.europa.eu/ghs_s2composite.php
And paper for the dataset is:
Corbane, C., Politis, P., Kempeneers, P., Simonetti, D., Soille, P., Burger, A., … & Kemper, T. (2020). A global cloud free pixel-based image composite from Sentinel-2 data. Data in Brief, 105737.
import sys
sys.path.insert(0, "../")
import utils
import geopandas
from dask import dataframe as dd
from dask.system import cpu_count
Set up¶
Before accessing and downloading each GeoTIFF, let’s set up the target folder:
local_dir = "../../urbangrammar_samba/ghs_composite_s2/"
The set of UTM tiles we require for GB are the following:
gb_utm_tiles = ["30U", "31U", "29V", "30V"]
The metadata for the grid of tiles and their URLs is available as a GeoJSON. We read the file and exclude every tile that does not cover GB:
meta_p = "GHS-composite-S2.geojson"
meta = geopandas.read_file(meta_p)
meta = meta[meta["UTMtile"].isin(gb_utm_tiles)]
Download scenes¶
All of the scenes will be stored in the same folder (local_dir) in the OSGB 1936 British National Grid (EPSG:EPSG:27700). This aligns with the rest of data in the project and will also allow setting up a single virtual raster (see next section).
Generate a column with target file¶
meta["dst_path"] = meta["URL"].apply(lambda x: local_dir+x.split("/")[-1])
Parallel download/reprojection¶
In parallel:
Download each file on its each location
Reproject to OSGB grid
# Ship `meta` to Dask
dmeta = dd.from_pandas(meta[["dst_path", "UTMtile", "URL"]],
npartitions=10
)
# Apply in parallel
dout = dmeta.apply(utils.process_scene,
axis=1,
meta=("Output", None),
progressbar=False,
remove_intermediate=False,
)
_ = dout.compute()
09/11/2020 14:59:05 | Working on Tile 29V - File: S2_percentile_UTM_148-0000069888-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 31U - File: S2_percentile_UTM_210-0000069888-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 29V - File: S2_percentile_UTM_148-0000023296-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 30U - File: S2_percentile_UTM_209-0000069888-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 30U - File: S2_percentile_UTM_209-0000023296-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 31U - File: S2_percentile_UTM_210-0000023296-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 30V - File: S2_percentile_UTM_149-0000069888-0000023296.tif
09/11/2020 14:59:05 | Working on Tile 30V - File: S2_percentile_UTM_149-0000023296-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000023296-0000023296.tif (1.90 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000069888-0000023296.tif (2.29 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000069888-0000023296.tif (2.04 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000023296-0000023296.tif (1.09 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000069888-0000023296.tif (1.06 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000023296-0000023296.tif (2.55 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000023296-0000023296.tif (1.11 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000069888-0000023296.tif (980.5 MB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000023296.tif
09/11/2020 15:00:26 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000023296.tif
09/11/2020 15:00:31 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000069888-0000023296.tif
09/11/2020 15:00:57 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000069888-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000069888-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000023296.tif
09/11/2020 15:00:58 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000023296.tif
09/11/2020 15:00:59 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000023296.tif
09/11/2020 15:01:02 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000023296.tif
09/11/2020 15:01:09 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000023296.tif
09/11/2020 15:01:45 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
Split-opt.
Split-opt.
Split-opt.
Split-opt.
09/11/2020 15:09:09 | Optimising S2_percentile_UTM_148-0000069888-0000023296_wm_R.tif
09/11/2020 15:09:47 | Optimising S2_percentile_UTM_149-0000069888-0000023296_wm_R.tif
09/11/2020 15:10:07 | Optimising S2_percentile_UTM_149-0000023296-0000023296_wm_R.tif
Split-opt.
Split-opt.
Split-opt.
Split-opt.
09/11/2020 15:13:59 | Optimising S2_percentile_UTM_148-0000023296-0000023296_wm_R.tif
09/11/2020 15:16:08 | Optimising S2_percentile_UTM_149-0000069888-0000023296_wm_G.tif
09/11/2020 15:16:19 | Optimising S2_percentile_UTM_149-0000023296-0000023296_wm_G.tif
09/11/2020 15:16:35 | Optimising S2_percentile_UTM_148-0000069888-0000023296_wm_G.tif
09/11/2020 15:19:21 | Optimising S2_percentile_UTM_210-0000069888-0000023296_wm_R.tif 09/11/2020 15:19:21 | Optimising S2_percentile_UTM_209-0000069888-0000023296_wm_R.tif
09/11/2020 15:19:21 | Optimising S2_percentile_UTM_148-0000023296-0000023296_wm_G.tif
09/11/2020 15:22:25 | Optimising S2_percentile_UTM_149-0000023296-0000023296_wm_B.tif
09/11/2020 15:22:44 | Optimising S2_percentile_UTM_149-0000069888-0000023296_wm_B.tif
09/11/2020 15:23:03 | Optimising S2_percentile_UTM_209-0000023296-0000023296_wm_R.tif
09/11/2020 15:23:08 | Optimising S2_percentile_UTM_148-0000069888-0000023296_wm_B.tif
09/11/2020 15:24:43 | Optimising S2_percentile_UTM_210-0000023296-0000023296_wm_R.tif
09/11/2020 15:26:54 | Optimising S2_percentile_UTM_209-0000069888-0000023296_wm_G.tif 09/11/2020 15:26:54 | Optimising S2_percentile_UTM_148-0000023296-0000023296_wm_B.tif
09/11/2020 15:28:24 | Optimising S2_percentile_UTM_149-0000069888-0000023296_wm_I.tif
09/11/2020 15:31:13 | Optimising S2_percentile_UTM_149-0000023296-0000023296_wm_I.tif
09/11/2020 15:31:13 | Optimising S2_percentile_UTM_148-0000069888-0000023296_wm_I.tif
09/11/2020 15:33:37 | Working on Tile 30V - File: S2_percentile_UTM_149-0000069888-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000069888-0000000000.tif (2.38 GB)
09/11/2020 15:34:29 | Optimising S2_percentile_UTM_209-0000023296-0000023296_wm_G.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000000000.tif
09/11/2020 15:34:42 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000069888-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 15:36:42 | Optimising S2_percentile_UTM_210-0000069888-0000023296_wm_G.tif
09/11/2020 15:38:12 | Working on Tile 30V - File: S2_percentile_UTM_149-0000023296-0000000000.tif
09/11/2020 15:38:12 | Working on Tile 29V - File: S2_percentile_UTM_148-0000069888-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000023296-0000000000.tif (1.86 GB)
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000069888-0000000000.tif (1.76 GB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000000000.tif
09/11/2020 15:39:07 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000023296-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000000000.tif
09/11/2020 15:39:13 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000069888-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 15:42:31 | Optimising S2_percentile_UTM_148-0000023296-0000023296_wm_I.tif
09/11/2020 15:43:59 | Optimising S2_percentile_UTM_209-0000069888-0000023296_wm_B.tif
09/11/2020 15:45:48 | Optimising S2_percentile_UTM_210-0000069888-0000023296_wm_B.tif
Split-opt.
09/11/2020 15:47:12 | Optimising S2_percentile_UTM_210-0000023296-0000023296_wm_G.tif
09/11/2020 15:48:52 | Optimising S2_percentile_UTM_209-0000023296-0000023296_wm_B.tif
09/11/2020 15:48:52 | Working on Tile 29V - File: S2_percentile_UTM_148-0000023296-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000023296-0000000000.tif (1.47 GB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000000000.tif
09/11/2020 15:49:27 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000023296-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Split-opt.
Split-opt.
09/11/2020 15:53:20 | Optimising S2_percentile_UTM_149-0000069888-0000000000_wm_R.tif
09/11/2020 15:53:23 | Optimising S2_percentile_UTM_210-0000069888-0000023296_wm_I.tif
09/11/2020 15:53:23 | Optimising S2_percentile_UTM_209-0000069888-0000023296_wm_I.tif
09/11/2020 15:55:41 | Optimising S2_percentile_UTM_210-0000023296-0000023296_wm_B.tif
09/11/2020 15:56:37 | Optimising S2_percentile_UTM_209-0000023296-0000023296_wm_I.tif
09/11/2020 15:59:49 | Working on Tile 30U - File: S2_percentile_UTM_209-0000069888-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000069888-0000000000.tif (1.82 GB)
09/11/2020 16:04:26 | Optimising S2_percentile_UTM_149-0000023296-0000000000_wm_R.tif
09/11/2020 16:04:26 | Optimising S2_percentile_UTM_148-0000023296-0000000000_wm_R.tif
09/11/2020 16:04:26 | Optimising S2_percentile_UTM_149-0000069888-0000000000_wm_G.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000000000.tif
09/11/2020 16:04:44 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000069888-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 16:08:07 | Optimising S2_percentile_UTM_210-0000023296-0000023296_wm_I.tif
09/11/2020 16:09:55 | Working on Tile 30U - File: S2_percentile_UTM_209-0000023296-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000023296-0000000000.tif (2.16 GB)
09/11/2020 16:11:28 | Optimising S2_percentile_UTM_149-0000069888-0000000000_wm_B.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000000000.tif
09/11/2020 16:11:51 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000023296-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Split-opt.
09/11/2020 16:15:20 | Optimising S2_percentile_UTM_149-0000023296-0000000000_wm_G.tif
09/11/2020 16:15:21 | Optimising S2_percentile_UTM_148-0000069888-0000000000_wm_G.tif
09/11/2020 16:17:19 | Optimising S2_percentile_UTM_210-0000069888-0000000000_wm_R.tif
09/11/2020 16:17:22 | Working on Tile 31U - File: S2_percentile_UTM_210-0000023296-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000023296-0000000000.tif (2.12 GB)
Split-opt.
09/11/2020 16:19:50 | Optimising S2_percentile_UTM_148-0000023296-0000000000_wm_G.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000000000.tif
09/11/2020 16:19:50 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000023296-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 16:20:43 | Optimising S2_percentile_UTM_149-0000069888-0000000000_wm_I.tif
Split-opt.
09/11/2020 16:23:09 | Optimising S2_percentile_UTM_209-0000069888-0000000000_wm_R.tif
09/11/2020 16:27:00 | Optimising S2_percentile_UTM_149-0000023296-0000000000_wm_B.tif
09/11/2020 16:27:01 | Optimising S2_percentile_UTM_148-0000069888-0000000000_wm_B.tif
09/11/2020 16:28:10 | Optimising S2_percentile_UTM_210-0000069888-0000000000_wm_G.tif
09/11/2020 16:30:34 | Working on Tile 30V - File: S2_percentile_UTM_149-0000046592-0000023296.tif
Split-opt.
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000046592-0000023296.tif (1.13 GB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000046592-0000023296.tif
09/11/2020 16:31:00 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000046592-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000046592-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 16:32:24 | Optimising S2_percentile_UTM_148-0000023296-0000000000_wm_B.tif
09/11/2020 16:33:32 | Optimising S2_percentile_UTM_209-0000069888-0000000000_wm_G.tif
09/11/2020 16:36:20 | Optimising S2_percentile_UTM_209-0000023296-0000000000_wm_R.tif
Split-opt.
09/11/2020 16:40:36 | Optimising S2_percentile_UTM_210-0000023296-0000000000_wm_R.tif
09/11/2020 16:40:36 | Optimising S2_percentile_UTM_149-0000023296-0000000000_wm_I.tif
09/11/2020 16:44:00 | Optimising S2_percentile_UTM_148-0000069888-0000000000_wm_I.tif
09/11/2020 16:47:00 | Optimising S2_percentile_UTM_210-0000069888-0000000000_wm_B.tif
09/11/2020 16:47:00 | Optimising S2_percentile_UTM_149-0000046592-0000023296_wm_R.tif
09/11/2020 16:49:55 | Optimising S2_percentile_UTM_209-0000069888-0000000000_wm_B.tif
09/11/2020 16:52:25 | Optimising S2_percentile_UTM_148-0000023296-0000000000_wm_I.tif
09/11/2020 16:52:28 | Optimising S2_percentile_UTM_149-0000046592-0000023296_wm_G.tif
09/11/2020 16:55:22 | Optimising S2_percentile_UTM_210-0000069888-0000000000_wm_I.tif
09/11/2020 16:55:22 | Optimising S2_percentile_UTM_209-0000023296-0000000000_wm_G.tif
09/11/2020 16:58:23 | Working on Tile 29V - File: S2_percentile_UTM_148-0000046592-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000046592-0000023296.tif (1.20 GB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000023296.tif
09/11/2020 16:59:21 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:01:14 | Working on Tile 30V - File: S2_percentile_UTM_149-0000000000-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000000000-0000023296.tif (813.0 MB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000023296.tif
09/11/2020 17:01:36 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:03:40 | Optimising S2_percentile_UTM_210-0000023296-0000000000_wm_G.tif
09/11/2020 17:06:36 | Optimising S2_percentile_UTM_149-0000046592-0000023296_wm_B.tif
09/11/2020 17:06:37 | Working on Tile 31U - File: S2_percentile_UTM_210-0000046592-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000046592-0000023296.tif (2.62 GB)
Split-opt.
Split-opt.
09/11/2020 17:07:33 | Optimising S2_percentile_UTM_209-0000023296-0000000000_wm_B.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000023296.tif
09/11/2020 17:08:30 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:08:38 | Optimising S2_percentile_UTM_209-0000069888-0000000000_wm_I.tif
09/11/2020 17:12:29 | Working on Tile 29V - File: S2_percentile_UTM_148-0000000000-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000000000-0000023296.tif (407.2 MB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000000000-0000023296.tif
09/11/2020 17:12:39 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000000000-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000000000-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:13:45 | Optimising S2_percentile_UTM_149-0000000000-0000023296_wm_R.tif
09/11/2020 17:13:45 | Optimising S2_percentile_UTM_149-0000046592-0000023296_wm_I.tif
Split-opt.
09/11/2020 17:16:24 | Optimising S2_percentile_UTM_148-0000046592-0000023296_wm_R.tif
09/11/2020 17:16:25 | Working on Tile 30U - File: S2_percentile_UTM_209-0000046592-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000046592-0000023296.tif (2.20 GB)
09/11/2020 17:16:45 | Optimising S2_percentile_UTM_210-0000023296-0000000000_wm_B.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000023296.tif
09/11/2020 17:17:06 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:20:11 | Optimising S2_percentile_UTM_148-0000000000-0000023296_wm_R.tif
09/11/2020 17:22:37 | Optimising S2_percentile_UTM_149-0000000000-0000023296_wm_G.tif
09/11/2020 17:22:49 | Optimising S2_percentile_UTM_209-0000023296-0000000000_wm_I.tif
09/11/2020 17:24:09 | Optimising S2_percentile_UTM_148-0000000000-0000023296_wm_G.tif
09/11/2020 17:27:20 | Optimising S2_percentile_UTM_210-0000046592-0000023296_wm_R.tif
09/11/2020 17:29:07 | Optimising S2_percentile_UTM_149-0000000000-0000023296_wm_B.tif
09/11/2020 17:29:08 | Optimising S2_percentile_UTM_210-0000023296-0000000000_wm_I.tif
09/11/2020 17:30:20 | Optimising S2_percentile_UTM_148-0000046592-0000023296_wm_B.tif
09/11/2020 17:32:00 | Optimising S2_percentile_UTM_148-0000000000-0000023296_wm_B.tif
Split-opt.
Split-opt.
09/11/2020 17:33:40 | Working on Tile 30U - File: S2_percentile_UTM_209-0000000000-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000000000-0000023296.tif (2.00 GB)
09/11/2020 17:34:16 | Optimising S2_percentile_UTM_149-0000000000-0000023296_wm_I.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000023296.tif
09/11/2020 17:34:31 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:36:40 | Optimising S2_percentile_UTM_149-0000046592-0000000000_wm_R.tif
09/11/2020 17:38:25 | Optimising S2_percentile_UTM_210-0000046592-0000023296_wm_G.tif
09/11/2020 17:38:26 | Optimising S2_percentile_UTM_209-0000046592-0000023296_wm_R.tif
09/11/2020 17:40:28 | Optimising S2_percentile_UTM_148-0000046592-0000023296_wm_I.tif
09/11/2020 17:42:27 | Working on Tile 31U - File: S2_percentile_UTM_210-0000000000-0000023296.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000000000-0000023296.tif (1.50 GB)
09/11/2020 17:42:31 | Working on Tile 30V - File: S2_percentile_UTM_149-0000000000-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30V/S2_percentile_UTM_149-0000000000-0000000000.tif (678.9 MB)
Split-opt.
09/11/2020 17:42:49 | Optimising S2_percentile_UTM_148-0000000000-0000023296_wm_I.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000000000.tif
09/11/2020 17:43:06 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_149-0000000000-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000023296.tif
09/11/2020 17:43:07 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000023296.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000023296_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 17:46:35 | Optimising S2_percentile_UTM_209-0000046592-0000023296_wm_G.tif
09/11/2020 17:46:36 | Optimising S2_percentile_UTM_209-0000000000-0000023296_wm_R.tif
Split-opt.
09/11/2020 17:47:54 | Optimising S2_percentile_UTM_149-0000046592-0000000000_wm_G.tif
09/11/2020 17:50:01 | Optimising S2_percentile_UTM_149-0000000000-0000000000_wm_R.tif
09/11/2020 17:50:04 | Working on Tile 29V - File: S2_percentile_UTM_148-0000046592-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000046592-0000000000.tif (2.05 GB)
09/11/2020 17:51:02 | Optimising S2_percentile_UTM_210-0000046592-0000023296_wm_B.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000000000.tif
09/11/2020 17:51:12 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_148-0000046592-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Split-opt.
09/11/2020 17:52:33 | Working on Tile 29V - File: S2_percentile_UTM_148-0000000000-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/29V/S2_percentile_UTM_148-0000000000-0000000000.tif (1.03 GB)
09/11/2020 17:57:54 | Optimising S2_percentile_UTM_209-0000000000-0000023296_wm_G.tif
09/11/2020 17:59:45 | Optimising S2_percentile_UTM_149-0000046592-0000000000_wm_B.tif
09/11/2020 18:01:16 | Optimising S2_percentile_UTM_149-0000000000-0000000000_wm_G.tif
Split-opt.
09/11/2020 18:02:18 | Optimising S2_percentile_UTM_210-0000046592-0000023296_wm_I.tif
Split-opt.
09/11/2020 18:04:31 | Optimising S2_percentile_UTM_148-0000000000-0000000000_wm_R.tif
09/11/2020 18:06:24 | Optimising S2_percentile_UTM_209-0000046592-0000023296_wm_I.tif
09/11/2020 18:08:09 | Optimising S2_percentile_UTM_209-0000000000-0000023296_wm_B.tif
09/11/2020 18:08:09 | Optimising S2_percentile_UTM_148-0000046592-0000000000_wm_R.tif
09/11/2020 18:12:11 | Optimising S2_percentile_UTM_149-0000046592-0000000000_wm_I.tif
09/11/2020 18:12:11 | Optimising S2_percentile_UTM_210-0000000000-0000023296_wm_G.tif
09/11/2020 18:13:57 | Optimising S2_percentile_UTM_149-0000000000-0000000000_wm_B.tif
09/11/2020 18:16:04 | Working on Tile 31U - File: S2_percentile_UTM_210-0000046592-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000046592-0000000000.tif (2.40 GB)
09/11/2020 18:17:38 | Working on Tile 30U - File: S2_percentile_UTM_209-0000046592-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000046592-0000000000.tif (2.22 GB)
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000000000.tif
09/11/2020 18:18:02 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000046592-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000000000.tif
09/11/2020 18:18:30 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000046592-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 18:19:30 | Optimising S2_percentile_UTM_209-0000000000-0000023296_wm_I.tif
09/11/2020 18:21:02 | Optimising S2_percentile_UTM_148-0000000000-0000000000_wm_G.tif
09/11/2020 18:22:40 | Optimising S2_percentile_UTM_148-0000046592-0000000000_wm_G.tif
09/11/2020 18:25:33 | Optimising S2_percentile_UTM_210-0000000000-0000023296_wm_B.tif
09/11/2020 18:26:47 | Optimising S2_percentile_UTM_149-0000000000-0000000000_wm_I.tif
Split-opt.
09/11/2020 18:27:43 | Working on Tile 30U - File: S2_percentile_UTM_209-0000000000-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/30U/S2_percentile_UTM_209-0000000000-0000000000.tif (2.18 GB)
Split-opt.
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000000000.tif
09/11/2020 18:28:18 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_209-0000000000-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 18:32:32 | Optimising S2_percentile_UTM_148-0000046592-0000000000_wm_B.tif
09/11/2020 18:32:32 | Optimising S2_percentile_UTM_210-0000046592-0000000000_wm_R.tif
09/11/2020 18:32:32 | Optimising S2_percentile_UTM_209-0000046592-0000000000_wm_R.tif
09/11/2020 18:33:21 | Optimising S2_percentile_UTM_148-0000000000-0000000000_wm_B.tif
Split-opt.
09/11/2020 18:38:30 | Optimising S2_percentile_UTM_210-0000000000-0000023296_wm_I.tif
09/11/2020 18:41:42 | Optimising S2_percentile_UTM_209-0000046592-0000000000_wm_G.tif
09/11/2020 18:41:42 | Optimising S2_percentile_UTM_209-0000000000-0000000000_wm_R.tif
09/11/2020 18:43:48 | Optimising S2_percentile_UTM_210-0000046592-0000000000_wm_G.tif
09/11/2020 18:45:48 | Optimising S2_percentile_UTM_148-0000046592-0000000000_wm_I.tif
09/11/2020 18:47:07 | Working on Tile 31U - File: S2_percentile_UTM_210-0000000000-0000000000.tif
Downloading data from http://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A/GHS_composite_S2_L1C_2017-2018_GLOBE_R2020A_UTM_10/V1-0/31U/S2_percentile_UTM_210-0000000000-0000000000.tif (1.65 GB)
09/11/2020 18:47:51 | Optimising S2_percentile_UTM_148-0000000000-0000000000_wm_I.tif
Successfully downloaded file to ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000000000.tif
09/11/2020 18:47:57 | rio warp ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000000000.tif ../../urbangrammar_samba/ghs_composite_s2/S2_percentile_UTM_210-0000000000-0000000000_osgb.tif --threads 16 --dst-crs EPSG:27700
09/11/2020 18:50:51 | Optimising S2_percentile_UTM_209-0000046592-0000000000_wm_B.tif
09/11/2020 18:52:20 | Optimising S2_percentile_UTM_209-0000000000-0000000000_wm_G.tif
09/11/2020 18:54:38 | Optimising S2_percentile_UTM_210-0000046592-0000000000_wm_B.tif
Split-opt.
09/11/2020 18:58:34 | Optimising S2_percentile_UTM_209-0000046592-0000000000_wm_I.tif
09/11/2020 19:00:26 | Optimising S2_percentile_UTM_209-0000000000-0000000000_wm_B.tif
09/11/2020 19:00:26 | Optimising S2_percentile_UTM_210-0000000000-0000000000_wm_R.tif
09/11/2020 19:01:37 | Optimising S2_percentile_UTM_210-0000046592-0000000000_wm_I.tif
09/11/2020 19:06:32 | Optimising S2_percentile_UTM_209-0000000000-0000000000_wm_I.tif
09/11/2020 19:07:53 | Optimising S2_percentile_UTM_210-0000000000-0000000000_wm_G.tif
09/11/2020 19:12:23 | Optimising S2_percentile_UTM_210-0000000000-0000000000_wm_B.tif
09/11/2020 19:16:00 | Optimising S2_percentile_UTM_210-0000000000-0000000000_wm_I.tif
Generate .vrt file¶
from osgeo import gdal
from glob import glob
files = glob(local_dir + "S2_percentile_UTM_*_osgb.tif")
vrt = gdal.BuildVRT(local_dir + 'GHS-composite-S2.vrt', files)
vrt = None # needed to close and save file due to some bug
Alternatively, you can use a command line tool to do the same.
NOTE - The command below does not seem to work from the notebook and in some other situations. If you run the equivalent from the terminal without backtracking folders (../) it should work.
! gdalbuildvrt $local_dir"GHS-composite-S2.vrt" $local_dir"S2_percentile_UTM_*_osgb.tif"