Wrong reading of data type in readOGR ?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Wrong reading of data type in readOGR ?

Patrick Giraudoux
Dear Listers,

I meet a problem with readOGR on an ESRI shapefile which was correctly
read in the past (practical training with students). Now all integer
variables are read as "factors" although they should be read as integer.
With the argument stringsAsFactors=FALSE, the reading gives
"characters", but not the expected integers.

Reading the corresponding *.dbf file with foreign:read.dbf gives a
correct reading. This makes me suspect that something goes wrong with
the last version of readOGR (maybe earlier too). Those installed one
year ago in the computer room are OK.

Any suggestion/idea?

Best,

Patrick

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Wrong reading of data type in readOGR ?

Roger Bivand
Administrator
On Fri, 25 Nov 2016, Patrick Giraudoux wrote:

> Dear Listers,
>
> I meet a problem with readOGR on an ESRI shapefile which was correctly read
> in the past (practical training with students). Now all integer variables are
> read as "factors" although they should be read as integer. With the argument
> stringsAsFactors=FALSE, the reading gives "characters", but not the expected
> integers.

Please see:

https://stat.ethz.ch/pipermail/r-sig-geo/2016-October/025038.html

https://stat.ethz.ch/pipermail/r-sig-geo/2016-November/025104.html

This is the result of the antiquated way DBF handle numbers, and changes
made in GDAL2 to accommodate 64-bit integers, which most often are IDs,
not numbers. Maybe try the GDAL1_integer64_policy= argument, or re-write
the shapefiles with the offending integer written as a double, or stop
using shapefiles.

Roger

>
> Reading the corresponding *.dbf file with foreign:read.dbf gives a correct
> reading. This makes me suspect that something goes wrong with the last
> version of readOGR (maybe earlier too). Those installed one year ago in the
> computer room are OK.
>
> Any suggestion/idea?
>
> Best,
>
> Patrick
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: [hidden email]
http://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
http://depsy.org/person/434412

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Wrong reading of data type in readOGR ?

Patrick Giraudoux
Le 25/11/2016 à 16:18, Roger Bivand a écrit :

> On Fri, 25 Nov 2016, Patrick Giraudoux wrote:
>
>> Dear Listers,
>>
>> I meet a problem with readOGR on an ESRI shapefile which was
>> correctly read in the past (practical training with students). Now
>> all integer variables are read as "factors" although they should be
>> read as integer. With the argument stringsAsFactors=FALSE, the
>> reading gives "characters", but not the expected integers.
>
> Please see:
>
> https://stat.ethz.ch/pipermail/r-sig-geo/2016-October/025038.html
>
> https://stat.ethz.ch/pipermail/r-sig-geo/2016-November/025104.html
>
> This is the result of the antiquated way DBF handle numbers, and
> changes made in GDAL2 to accommodate 64-bit integers, which most often
> are IDs, not numbers. Maybe try the GDAL1_integer64_policy= argument,
> or re-write the shapefiles with the offending integer written as a
> double, or stop using shapefiles.
>
> Roger

OK. Clear enough. Thanks, Roger. Will modify the training guideline
accordingly.

Best,

Patrick

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Wrong reading of data type in readOGR ?

Patrick Giraudoux
In reply to this post by Roger Bivand
Le 25/11/2016 à 16:18, Roger Bivand a écrit :

> On Fri, 25 Nov 2016, Patrick Giraudoux wrote:
>
>> Dear Listers,
>>
>> I meet a problem with readOGR on an ESRI shapefile which was
>> correctly read in the past (practical training with students). Now
>> all integer variables are read as "factors" although they should be
>> read as integer. With the argument stringsAsFactors=FALSE, the
>> reading gives "characters", but not the expected integers.
>
> Please see:
>
> https://stat.ethz.ch/pipermail/r-sig-geo/2016-October/025038.html
>
> https://stat.ethz.ch/pipermail/r-sig-geo/2016-November/025104.html
>
> This is the result of the antiquated way DBF handle numbers, and
> changes made in GDAL2 to accommodate 64-bit integers, which most often
> are IDs, not numbers. Maybe try the GDAL1_integer64_policy= argument,
> or re-write the shapefiles with the offending integer written as a
> double, or stop using shapefiles.
>
> Roger

For info, a command line like this (with GDAL1_integer64=TRUE)

2539<-readOGR(".","Arvicola2539maj11au", GDAL1_integer64=TRUE)

...works perfect and meet the expectations. May be useful to keep in
mind for old shapefiles left on a shelter in the attic...

Before Roger's response, during the training course (may day, may day,
etc.), the other way round had been to import with no arguments and then
use a loop such:

for(i in 18:75) d2539@data[,i]<-as.numeric(as.character(d2539@data[,i]))

but it is (not recommandable) ruffian's way (:-)

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo