issue when opening raster image in R, the pixel values a rescaled somehow

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

issue when opening raster image in R, the pixel values a rescaled somehow

Victor Rodriguez-Galiano
I opened a Geotiff image in R using the raster function. However, the image
seems to be rescaled. The minimum and maximum values should be 13607 and
15461 but are 275 and 305. The Geotiff image when opened in a GIS is
correct but not in R.

This is my code:

Script:

library(raster)

trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif",
datatype = "INT2U")

trial

plot(trial)

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

Roger Bivand
Administrator
Please make the file in question available for download (url), or the url you downloaded it from (no login), on this list. It looks as though the unsigned 16 bit integer is not right.

Roger Bivand
Norwegian School of Economics
Bergen, Norway

________________________________
Fra: R-sig-Geo <[hidden email]> p� vegne av Victor F. Rodriguez Galiano <[hidden email]>
Sendt: mandag 23. mars 2020, 21.02
Til: [hidden email]
Emne: [R-sig-Geo] issue when opening raster image in R, the pixel values a rescaled somehow

I opened a Geotiff image in R using the raster function. However, the image seems to be rescaled. The minimum and maximum values should be 13607 and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is correct but not in R. This is my code: Script: library(raster) trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif", datatype = "INT2U") trial plot(trial) [[alternative HTML version deleted]] _______________________________________________ R-sig-Geo mailing list [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo

        [[alternative HTML version deleted]]


_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

Victor Rodriguez-Galiano
Hi Roger, the link to he image is given below. This image was generated
from a .hdf file using the function gdal_translate.

gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1,
nchar(filename)-4) ,".tif", ))

https://wetransfer.com/downloads/4569e9a85c02faad5016b83d0b50217820200324084015/f2eafdfafaae74de9ac8e25017fc5cc520200324084015/e46e80


Thanks

Victor

El lun., 23 mar. 2020 a las 21:07, Roger Bivand (<[hidden email]>)
escribió:

> Please make the file in question available for download (url), or the url
> you downloaded it from (no login), on this list. It looks as though the
> unsigned 16 bit integer is not right.
>
> Roger Bivand
> Norwegian School of Economics
> Bergen, Norway
>
> ------------------------------
> *Fra:* R-sig-Geo <[hidden email]> på vegne av Victor F.
> Rodriguez Galiano <[hidden email]>
> *Sendt:* mandag 23. mars 2020, 21.02
> *Til:* [hidden email]
> *Emne:* [R-sig-Geo] issue when opening raster image in R, the pixel
> values a rescaled somehow
>
> I opened a Geotiff image in R using the raster function. However, the
> image seems to be rescaled. The minimum and maximum values should be 13607
> and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is
> correct but not in R. This is my code: Script: library(raster)
> trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif", datatype =
> "INT2U") trial plot(trial) [[alternative HTML version deleted]]
> _______________________________________________ R-sig-Geo mailing list
> [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

Roger Bivand
Administrator
Please put the original file on the link too, so that we can track where
the misconfiguration occurs, and provide more of the workflow code. I
think we'll also need sessionInfo() output and GDAL version, both as seen
by gdalUtils, raster and rgdal.

Roger

On Tue, 24 Mar 2020, Victor F. Rodriguez Galiano wrote:

> Hi Roger, the link to he image is given below. This image was generated
> from a .hdf file using the function gdal_translate.
>
> gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1,
> nchar(filename)-4) ,".tif", ))
>
> https://wetransfer.com/downloads/4569e9a85c02faad5016b83d0b50217820200324084015/f2eafdfafaae74de9ac8e25017fc5cc520200324084015/e46e80
>
>
> Thanks
>
> Victor
>
> El lun., 23 mar. 2020 a las 21:07, Roger Bivand (<[hidden email]>)
> escribió:
>
>> Please make the file in question available for download (url), or the url
>> you downloaded it from (no login), on this list. It looks as though the
>> unsigned 16 bit integer is not right.
>>
>> Roger Bivand
>> Norwegian School of Economics
>> Bergen, Norway
>>
>> ------------------------------
>> *Fra:* R-sig-Geo <[hidden email]> på vegne av Victor F.
>> Rodriguez Galiano <[hidden email]>
>> *Sendt:* mandag 23. mars 2020, 21.02
>> *Til:* [hidden email]
>> *Emne:* [R-sig-Geo] issue when opening raster image in R, the pixel
>> values a rescaled somehow
>>
>> I opened a Geotiff image in R using the raster function. However, the
>> image seems to be rescaled. The minimum and maximum values should be 13607
>> and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is
>> correct but not in R. This is my code: Script: library(raster)
>> trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif", datatype =
>> "INT2U") trial plot(trial) [[alternative HTML version deleted]]
>> _______________________________________________ R-sig-Geo mailing list
>> [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

Roger Bivand
Administrator
In reply to this post by Victor Rodriguez-Galiano
Opening in stars shows that the read data are interpreted as degrees
Kelvin:

> library(stars)
> ss <- read_stars("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
> print(ss, n=1e8)
stars object with 2 dimensions and 1 attribute
attribute(s):
  MOD11A2.A2000049.h17v05.006.2015058135048.tif [K]
  Min.   :272.1
  1st Qu.:293.3
  Median :296.5
  Mean   :296.1
  3rd Qu.:298.8
  Max.   :309.2
  NA's   :926190
dimension(s):
   from   to   offset    delta  refsys point values
x    1 1200 -1111951  926.625 unnamed FALSE   NULL [x]
y    1 1200  4447802 -926.625 unnamed FALSE   NULL [y]

The summary agrees with

> library(rgdal)
> o <- readGDAL("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
> summary(o)
Object of class SpatialGridDataFrame
Coordinates:
        min     max
x -1111951       0
y  3335852 4447802
Is projected: TRUE
proj4string :
[+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs]
Grid attributes:
   cellcentre.offset cellsize cells.dim
x          -1111487 926.6254      1200
y           3336315 926.6254      1200
Data attributes:
      band1
  Min.   :272.1
  1st Qu.:293.3
  Median :296.5
  Mean   :296.1
  3rd Qu.:298.8
  Max.   :309.2
  NA's   :926190

and

> library(raster)
> r <- raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
> summary(values(r))
    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's
   272.1   293.3   296.5   296.1   298.8   309.2  926190

So my feeling is that the representation is correct in degrees Kelvin.

Others please join in, I'm not familiar with MODIS HDF data. I think
something gives the metadata in the GeoTIFF the wrong data bounds - they
do seem to be UInt16. The output of gdalinfo (command line) includes:

...
   units=K
   valid_range=7500, 65535
...
Band 1 Block=1200x3 Type=UInt16, ColorInterp=Gray
   NoData Value=0
   Unit Type: K
...

Roger


On Tue, 24 Mar 2020, Victor F. Rodriguez Galiano wrote:

> Hi Roger, the link to he image is given below. This image was generated
> from a .hdf file using the function gdal_translate.
>
> gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1,
> nchar(filename)-4) ,".tif", ))
>
> https://wetransfer.com/downloads/4569e9a85c02faad5016b83d0b50217820200324084015/f2eafdfafaae74de9ac8e25017fc5cc520200324084015/e46e80
>
>
> Thanks
>
> Victor
>
> El lun., 23 mar. 2020 a las 21:07, Roger Bivand (<[hidden email]>)
> escribió:
>
>> Please make the file in question available for download (url), or the url
>> you downloaded it from (no login), on this list. It looks as though the
>> unsigned 16 bit integer is not right.
>>
>> Roger Bivand
>> Norwegian School of Economics
>> Bergen, Norway
>>
>> ------------------------------
>> *Fra:* R-sig-Geo <[hidden email]> på vegne av Victor F.
>> Rodriguez Galiano <[hidden email]>
>> *Sendt:* mandag 23. mars 2020, 21.02
>> *Til:* [hidden email]
>> *Emne:* [R-sig-Geo] issue when opening raster image in R, the pixel
>> values a rescaled somehow
>>
>> I opened a Geotiff image in R using the raster function. However, the
>> image seems to be rescaled. The minimum and maximum values should be 13607
>> and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is
>> correct but not in R. This is my code: Script: library(raster)
>> trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif", datatype =
>> "INT2U") trial plot(trial) [[alternative HTML version deleted]]
>> _______________________________________________ R-sig-Geo mailing list
>> [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

Roger Bivand
Administrator
On Tue, 24 Mar 2020, Roger Bivand wrote:

> Opening in stars shows that the read data are interpreted as degrees Kelvin:
>
>>  library(stars)
>>  ss <- read_stars("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>>  print(ss, n=1e8)
> stars object with 2 dimensions and 1 attribute
> attribute(s):
>  MOD11A2.A2000049.h17v05.006.2015058135048.tif [K]
>  Min.   :272.1
>  1st Qu.:293.3
>  Median :296.5
>  Mean   :296.1
>  3rd Qu.:298.8
>  Max.   :309.2
>  NA's   :926190
> dimension(s):
>  from   to   offset    delta  refsys point values
> x    1 1200 -1111951  926.625 unnamed FALSE   NULL [x]
> y    1 1200  4447802 -926.625 unnamed FALSE   NULL [y]
>
> The summary agrees with
>
>>  library(rgdal)
>>  o <- readGDAL("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>>  summary(o)
> Object of class SpatialGridDataFrame
> Coordinates:
>       min     max
> x -1111951       0
> y  3335852 4447802
> Is projected: TRUE
> proj4string :
> [+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs]
> Grid attributes:
>  cellcentre.offset cellsize cells.dim
> x          -1111487 926.6254      1200
> y           3336315 926.6254      1200
> Data attributes:
>      band1
>  Min.   :272.1
>  1st Qu.:293.3
>  Median :296.5
>  Mean   :296.1
>  3rd Qu.:298.8
>  Max.   :309.2
>  NA's   :926190
>
> and
>
>>  library(raster)
>>  r <- raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>>  summary(values(r))
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's
>   272.1   293.3   296.5   296.1   298.8   309.2  926190
>
> So my feeling is that the representation is correct in degrees Kelvin.
>
> Others please join in, I'm not familiar with MODIS HDF data. I think
> something gives the metadata in the GeoTIFF the wrong data bounds - they do
> seem to be UInt16. The output of gdalinfo (command line) includes:
>
> ...
>   units=K
>   valid_range=7500, 65535
> ...
> Band 1 Block=1200x3 Type=UInt16, ColorInterp=Gray
>   NoData Value=0
>   Unit Type: K
> ...
I missed the next line with the scale in it:

$gdalinfo -stats MOD11A2.A2000049.h17v05.006.2015058135048.tif
...
Band 1 Block=1200x3 Type=UInt16, ColorInterp=Gray
   Minimum=13607.000, Maximum=15461.000, Mean=14804.047, StdDev=191.673
   NoData Value=0
   Unit Type: K
   Offset: 0,   Scale:0.02
   Metadata:
     DESCRIPTION=8-day daytime 1km grid Land-surface Temperature
     STATISTICS_MAXIMUM=15461
     STATISTICS_MEAN=14804.046793562
     STATISTICS_MINIMUM=13607
     STATISTICS_STDDEV=191.67312038983
     STATISTICS_VALID_PERCENT=35.68

and 13607*0.02 = 272.14, 15461*0.02 = 309.22, so GDAL and the R interfaces
are using the scale factor to read the data as degrees Kelvin. Divide by
the scale factor if you need the input values.

Roger

>
> Roger
>
>
> On Tue, 24 Mar 2020, Victor F. Rodriguez Galiano wrote:
>
>>  Hi Roger, the link to he image is given below. This image was generated
>>  from a .hdf file using the function gdal_translate.
>>
>>  gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1,
>>  nchar(filename)-4) ,".tif", ))
>>
>>  https://wetransfer.com/downloads/4569e9a85c02faad5016b83d0b50217820200324084015/f2eafdfafaae74de9ac8e25017fc5cc520200324084015/e46e80
>>
>>
>>  Thanks
>>
>>  Victor
>>
>>  El lun., 23 mar. 2020 a las 21:07, Roger Bivand (<[hidden email]>)
>>  escribió:
>>
>>>  Please make the file in question available for download (url), or the
>>>  url
>>>  you downloaded it from (no login), on this list. It looks as though the
>>>  unsigned 16 bit integer is not right.
>>>
>>>  Roger Bivand
>>>  Norwegian School of Economics
>>>  Bergen, Norway
>>>
>>>  ------------------------------
>>>  *Fra:* R-sig-Geo <[hidden email]> på vegne av Victor F.
>>>  Rodriguez Galiano <[hidden email]>
>>>  *Sendt:* mandag 23. mars 2020, 21.02
>>>  *Til:* [hidden email]
>>>  *Emne:* [R-sig-Geo] issue when opening raster image in R, the pixel
>>>  values a rescaled somehow
>>>
>>>  I opened a Geotiff image in R using the raster function. However, the
>>>  image seems to be rescaled. The minimum and maximum values should be
>>>  13607
>>>  and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is
>>>  correct but not in R. This is my code: Script: library(raster)
>>>  trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif", datatype
>>>  =
>>>  "INT2U") trial plot(trial) [[alternative HTML version deleted]]
>>>  _______________________________________________ R-sig-Geo mailing list
>>>  [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: issue when opening raster image in R, the pixel values a rescaled somehow

edzer
In reply to this post by Roger Bivand
I see at the end of the output obtained with running gdalinfo on the file:

Band 1 Block=1200x3 Type=UInt16, ColorInterp=Gray
  NoData Value=0
  Unit Type: K
  Offset: 0,   Scale:0.02
  Metadata:
    DESCRIPTION=8-day daytime 1km grid Land-surface Temperature

This means that Kelving values are obtained by computing

0 (offset) + 0.02 (scale) * pixel_value

packages stars and readGDAL do this transformation for you, so you get
actual Kelvin values, and represent them with doubles so there is no
rounding problem. Other, lower level tools may give you INT2 values
where you need to take care of the offset & scale yourself.

On 3/24/20 10:28 AM, Roger Bivand wrote:

> Opening in stars shows that the read data are interpreted as degrees
> Kelvin:
>
>> library(stars)
>> ss <- read_stars("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>> print(ss, n=1e8)
> stars object with 2 dimensions and 1 attribute
> attribute(s):
>  MOD11A2.A2000049.h17v05.006.2015058135048.tif [K]
>  Min.   :272.1
>  1st Qu.:293.3
>  Median :296.5
>  Mean   :296.1
>  3rd Qu.:298.8
>  Max.   :309.2
>  NA's   :926190
> dimension(s):
>   from   to   offset    delta  refsys point values
> x    1 1200 -1111951  926.625 unnamed FALSE   NULL [x]
> y    1 1200  4447802 -926.625 unnamed FALSE   NULL [y]
>
> The summary agrees with
>
>> library(rgdal)
>> o <- readGDAL("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>> summary(o)
> Object of class SpatialGridDataFrame
> Coordinates:
>        min     max
> x -1111951       0
> y  3335852 4447802
> Is projected: TRUE
> proj4string :
> [+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs]
> Grid attributes:
>   cellcentre.offset cellsize cells.dim
> x          -1111487 926.6254      1200
> y           3336315 926.6254      1200
> Data attributes:
>      band1
>  Min.   :272.1
>  1st Qu.:293.3
>  Median :296.5
>  Mean   :296.1
>  3rd Qu.:298.8
>  Max.   :309.2
>  NA's   :926190
>
> and
>
>> library(raster)
>> r <- raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif")
>> summary(values(r))
>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's
>   272.1   293.3   296.5   296.1   298.8   309.2  926190
>
> So my feeling is that the representation is correct in degrees Kelvin.
>
> Others please join in, I'm not familiar with MODIS HDF data. I think
> something gives the metadata in the GeoTIFF the wrong data bounds - they
> do seem to be UInt16. The output of gdalinfo (command line) includes:
>
> ...
>   units=K
>   valid_range=7500, 65535
> ...
> Band 1 Block=1200x3 Type=UInt16, ColorInterp=Gray
>   NoData Value=0
>   Unit Type: K
> ...
>
> Roger
>
>
> On Tue, 24 Mar 2020, Victor F. Rodriguez Galiano wrote:
>
>> Hi Roger, the link to he image is given below. This image was generated
>> from a .hdf file using the function gdal_translate.
>>
>> gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1,
>> nchar(filename)-4) ,".tif", ))
>>
>> https://wetransfer.com/downloads/4569e9a85c02faad5016b83d0b50217820200324084015/f2eafdfafaae74de9ac8e25017fc5cc520200324084015/e46e80
>>
>>
>>
>> Thanks
>>
>> Victor
>>
>> El lun., 23 mar. 2020 a las 21:07, Roger Bivand (<[hidden email]>)
>> escribió:
>>
>>> Please make the file in question available for download (url), or the
>>> url
>>> you downloaded it from (no login), on this list. It looks as though the
>>> unsigned 16 bit integer is not right.
>>>
>>> Roger Bivand
>>> Norwegian School of Economics
>>> Bergen, Norway
>>>
>>> ------------------------------
>>> *Fra:* R-sig-Geo <[hidden email]> på vegne av Victor F.
>>> Rodriguez Galiano <[hidden email]>
>>> *Sendt:* mandag 23. mars 2020, 21.02
>>> *Til:* [hidden email]
>>> *Emne:* [R-sig-Geo] issue when opening raster image in R, the pixel
>>> values a rescaled somehow
>>>
>>> I opened a Geotiff image in R using the raster function. However, the
>>> image seems to be rescaled. The minimum and maximum values should be
>>> 13607
>>> and 15461 but are 275 and 305. The Geotiff image when opened in a GIS is
>>> correct but not in R. This is my code: Script: library(raster)
>>> trial<-raster("MOD11A2.A2000049.h17v05.006.2015058135048.tif",
>>> datatype =
>>> "INT2U") trial plot(trial) [[alternative HTML version deleted]]
>>> _______________________________________________ R-sig-Geo mailing list
>>> [hidden email] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Edzer Pebesma
Institute for Geoinformatics
Heisenbergstrasse 2, 48149 Muenster, Germany
Phone: +49 251 8333081

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

pEpkey.asc (3K) Download Attachment