sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
(representations of coordinate reference systems) are handled, steps are
being taken to find ways to adapt sp/rgdal workflows. A current proposal
is to store the WKT2_2018 string as a comment to CRS objects as defined in
the sp package.

A draft development-in-progress version of rgdal is available at
https://r-forge.r-project.org/R/?group_id=884, and for sp at
https://github.com/rsbivand/sp (this version of sp requires rgdal >=
1.5-1). This adds the WKT comments to CRS objects on reading vector and
raster data sources, and uses WKT comments if found when writing vector
and raster objects (or at least does as far as I've checked, possibly
fragile).

An RFC with tersely worked cases for using CRS object comments to carry
WKT strings but maintaining full backward compatibility is online at
http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.

If you have other ideas or concerns about trying to use this mechanism for
sp CRS objects, please contribute at your earliest convenience.

http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows the
beginning of the next step, to query transformation operations to find
viable coordinate operation pipelines.

I'm assuming that the previous behaviour (transform without considering
accuracy with whatever is to hand) is not viable going forward, and that
we will need two steps: list coordinate operations between source and
target CRS (using the WKT comments as better specifications than the PROJ
strings), possibly intervene manually to install missing grids, then
undertake the coordinate operation.

The fallback may be simply to choose the least inaccurate available
coordinate operation, but this should be a fallback. This means that all
uses of spTransform() will require intervention.

Is this OK (it is tiresome but modernises workflows once), or is it not OK
(no user intervention is crucial)?

These behaviours may be set in an option, so that package maintainers and
users may delay modernisation, but all are undoubtedly served by rapid
adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
development versions all state that they list candidate coordinate
operations).

We cannot ship all the grids, they are very bulky, and probably nobody
needs sub-metre accuracy world-wide. Work in PROJ is starting to create a
content delivery network for trusted download and mechanisms for
registering downloaded grids on user platforms. We would for example not
want Windows users of rgdal and sf to have to download the same grid
twice.

Comments welcome here and at
https://github.com/r-spatial/discuss/issues/28 or
https://github.com/r-spatial/sf/issues/1187

Roger

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
And this link explains the CDN proposal for grid distribution:

https://www.spatialys.com/en/crowdfunding/

Roger

On Wed, 13 Nov 2019, Roger Bivand wrote:

> Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
> (representations of coordinate reference systems) are handled, steps are
> being taken to find ways to adapt sp/rgdal workflows. A current proposal is
> to store the WKT2_2018 string as a comment to CRS objects as defined in the
> sp package.
>
> A draft development-in-progress version of rgdal is available at
> https://r-forge.r-project.org/R/?group_id=884, and for sp at
> https://github.com/rsbivand/sp (this version of sp requires rgdal >= 1.5-1).
> This adds the WKT comments to CRS objects on reading vector and raster data
> sources, and uses WKT comments if found when writing vector and raster
> objects (or at least does as far as I've checked, possibly fragile).
>
> An RFC with tersely worked cases for using CRS object comments to carry WKT
> strings but maintaining full backward compatibility is online at
> http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
>
> If you have other ideas or concerns about trying to use this mechanism for sp
> CRS objects, please contribute at your earliest convenience.
>
> http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows the
> beginning of the next step, to query transformation operations to find viable
> coordinate operation pipelines.
>
> I'm assuming that the previous behaviour (transform without considering
> accuracy with whatever is to hand) is not viable going forward, and that we
> will need two steps: list coordinate operations between source and target CRS
> (using the WKT comments as better specifications than the PROJ strings),
> possibly intervene manually to install missing grids, then undertake the
> coordinate operation.
>
> The fallback may be simply to choose the least inaccurate available
> coordinate operation, but this should be a fallback. This means that all uses
> of spTransform() will require intervention.
>
> Is this OK (it is tiresome but modernises workflows once), or is it not OK
> (no user intervention is crucial)?
>
> These behaviours may be set in an option, so that package maintainers and
> users may delay modernisation, but all are undoubtedly served by rapid
> adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
> development versions all state that they list candidate coordinate
> operations).
>
> We cannot ship all the grids, they are very bulky, and probably nobody needs
> sub-metre accuracy world-wide. Work in PROJ is starting to create a content
> delivery network for trusted download and mechanisms for registering
> downloaded grids on user platforms. We would for example not want Windows
> users of rgdal and sf to have to download the same grid twice.
>
> Comments welcome here and at https://github.com/r-spatial/discuss/issues/28 
> or https://github.com/r-spatial/sf/issues/1187
>
> Roger
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
The development version of rgdal on R-Forge is now at rev 894, and is now
ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
migrate within 6 months to modern CRS representations. The motivating RFC
is also updated to cover coordinate operations, the use of prepared
(pre-searched) coordinate operations, and should be read carefully by
anyone using rgdal::spTransform(). Note further that rgdal::project() will
not be adapted for PROJ6, and is effectively deprecated.

I'll be running reverse dependency checks, and may be bugging package
maintainers. I would really prefer that mainainers of packages using
spTransform() checked themselves and joined this thread or the associated
twitter thread: https://twitter.com/RogerBivand/status/1194586193108914177

Be ready for modern PROJ and GDAL, they are already being deployed across
open source geospatial software, like GRASS, QGIS, pyproj, spatialite etc.

Waiting, hopefully not in vain, for contributions.

Roger

On Wed, 13 Nov 2019, Roger Bivand wrote:

> And this link explains the CDN proposal for grid distribution:
>
> https://www.spatialys.com/en/crowdfunding/
>
> Roger
>
> On Wed, 13 Nov 2019, Roger Bivand wrote:
>
>>  Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
>>  (representations of coordinate reference systems) are handled, steps are
>>  being taken to find ways to adapt sp/rgdal workflows. A current proposal
>>  is to store the WKT2_2018 string as a comment to CRS objects as defined in
>>  the sp package.
>>
>>  A draft development-in-progress version of rgdal is available at
>>  https://r-forge.r-project.org/R/?group_id=884, and for sp at
>>  https://github.com/rsbivand/sp (this version of sp requires rgdal >=
>>  1.5-1). This adds the WKT comments to CRS objects on reading vector and
>>  raster data sources, and uses WKT comments if found when writing vector
>>  and raster objects (or at least does as far as I've checked, possibly
>>  fragile).
>>
>>  An RFC with tersely worked cases for using CRS object comments to carry
>>  WKT strings but maintaining full backward compatibility is online at
>>  http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
>>
>>  If you have other ideas or concerns about trying to use this mechanism for
>>  sp CRS objects, please contribute at your earliest convenience.
>>
>>  http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows the
>>  beginning of the next step, to query transformation operations to find
>>  viable coordinate operation pipelines.
>>
>>  I'm assuming that the previous behaviour (transform without considering
>>  accuracy with whatever is to hand) is not viable going forward, and that
>>  we will need two steps: list coordinate operations between source and
>>  target CRS (using the WKT comments as better specifications than the PROJ
>>  strings), possibly intervene manually to install missing grids, then
>>  undertake the coordinate operation.
>>
>>  The fallback may be simply to choose the least inaccurate available
>>  coordinate operation, but this should be a fallback. This means that all
>>  uses of spTransform() will require intervention.
>>
>>  Is this OK (it is tiresome but modernises workflows once), or is it not OK
>>  (no user intervention is crucial)?
>>
>>  These behaviours may be set in an option, so that package maintainers and
>>  users may delay modernisation, but all are undoubtedly served by rapid
>>  adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
>>  development versions all state that they list candidate coordinate
>>  operations).
>>
>>  We cannot ship all the grids, they are very bulky, and probably nobody
>>  needs sub-metre accuracy world-wide. Work in PROJ is starting to create a
>>  content delivery network for trusted download and mechanisms for
>>  registering downloaded grids on user platforms. We would for example not
>>  want Windows users of rgdal and sf to have to download the same grid
>>  twice.
>>
>>  Comments welcome here and at
>>  https://github.com/r-spatial/discuss/issues/28 or
>>  https://github.com/r-spatial/sf/issues/1187
>>
>>  Roger
>>
>>
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
A description of the status now with regard to a prototype resolution is
online at:

https://rsbivand.github.io/ECS530_h19/ECS530_III.html

I'm planning to stream a talk about this at 09:15-11:00 CET on Tuesday 3
December. I need a volunteer to test the streaming link in advance during
next week. I'm unsure which technology to use for remote participants to
provide feedback.

Contributions/comments welcome!

Roger


On Fri, 15 Nov 2019, Roger Bivand wrote:

> The development version of rgdal on R-Forge is now at rev 894, and is now
> ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
> migrate within 6 months to modern CRS representations. The motivating RFC is
> also updated to cover coordinate operations, the use of prepared
> (pre-searched) coordinate operations, and should be read carefully by anyone
> using rgdal::spTransform(). Note further that rgdal::project() will not be
> adapted for PROJ6, and is effectively deprecated.
>
> I'll be running reverse dependency checks, and may be bugging package
> maintainers. I would really prefer that mainainers of packages using
> spTransform() checked themselves and joined this thread or the associated
> twitter thread: https://twitter.com/RogerBivand/status/1194586193108914177
>
> Be ready for modern PROJ and GDAL, they are already being deployed across
> open source geospatial software, like GRASS, QGIS, pyproj, spatialite etc.
>
> Waiting, hopefully not in vain, for contributions.
>
> Roger
>
> On Wed, 13 Nov 2019, Roger Bivand wrote:
>
>>  And this link explains the CDN proposal for grid distribution:
>>
>>  https://www.spatialys.com/en/crowdfunding/
>>
>>  Roger
>>
>>  On Wed, 13 Nov 2019, Roger Bivand wrote:
>>
>>>   Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
>>>   (representations of coordinate reference systems) are handled, steps are
>>>   being taken to find ways to adapt sp/rgdal workflows. A current proposal
>>>   is to store the WKT2_2018 string as a comment to CRS objects as defined
>>>   in
>>>   the sp package.
>>>
>>>   A draft development-in-progress version of rgdal is available at
>>>   https://r-forge.r-project.org/R/?group_id=884, and for sp at
>>>   https://github.com/rsbivand/sp (this version of sp requires rgdal >=
>>>   1.5-1). This adds the WKT comments to CRS objects on reading vector and
>>>   raster data sources, and uses WKT comments if found when writing vector
>>>   and raster objects (or at least does as far as I've checked, possibly
>>>   fragile).
>>>
>>>   An RFC with tersely worked cases for using CRS object comments to carry
>>>   WKT strings but maintaining full backward compatibility is online at
>>>   http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
>>>
>>>   If you have other ideas or concerns about trying to use this mechanism
>>>   for
>>>   sp CRS objects, please contribute at your earliest convenience.
>>>
>>>   http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows
>>>   the
>>>   beginning of the next step, to query transformation operations to find
>>>   viable coordinate operation pipelines.
>>>
>>>   I'm assuming that the previous behaviour (transform without considering
>>>   accuracy with whatever is to hand) is not viable going forward, and that
>>>   we will need two steps: list coordinate operations between source and
>>>   target CRS (using the WKT comments as better specifications than the
>>>   PROJ
>>>   strings), possibly intervene manually to install missing grids, then
>>>   undertake the coordinate operation.
>>>
>>>   The fallback may be simply to choose the least inaccurate available
>>>   coordinate operation, but this should be a fallback. This means that all
>>>   uses of spTransform() will require intervention.
>>>
>>>   Is this OK (it is tiresome but modernises workflows once), or is it not
>>>   OK
>>>   (no user intervention is crucial)?
>>>
>>>   These behaviours may be set in an option, so that package maintainers
>>>   and
>>>   users may delay modernisation, but all are undoubtedly served by rapid
>>>   adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
>>>   development versions all state that they list candidate coordinate
>>>   operations).
>>>
>>>   We cannot ship all the grids, they are very bulky, and probably nobody
>>>   needs sub-metre accuracy world-wide. Work in PROJ is starting to create
>>>   a
>>>   content delivery network for trusted download and mechanisms for
>>>   registering downloaded grids on user platforms. We would for example not
>>>   want Windows users of rgdal and sf to have to download the same grid
>>>   twice.
>>>
>>>   Comments welcome here and at
>>>   https://github.com/r-spatial/discuss/issues/28 or
>>>   https://github.com/r-spatial/sf/issues/1187
>>>
>>>   Roger
>>>
>>>
>>
>>
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Lorenzo Busetto
Dear Roger,

    I'd be very interested in participating to the stream talk. Is it
confirmed?

Concerning testing the connection, I'd be happy to do it but I do not know
how much I will be available online in the coming days. You can drop me a
mail/DM on twitter, and if I am available I'll gladly help.

Concerning technology, Zoom usually works quite well (https://zoom.us/
<https://zoom.us/ent?zcid=3172>)

   regards,

Lorenzo

On Sat, 23 Nov 2019 at 13:04, Roger Bivand <[hidden email]> wrote:

> A description of the status now with regard to a prototype resolution is
> online at:
>
> https://rsbivand.github.io/ECS530_h19/ECS530_III.html
>
> I'm planning to stream a talk about this at 09:15-11:00 CET on Tuesday 3
> December. I need a volunteer to test the streaming link in advance during
> next week. I'm unsure which technology to use for remote participants to
> provide feedback.
>
> Contributions/comments welcome!
>
> Roger
>
>
> On Fri, 15 Nov 2019, Roger Bivand wrote:
>
> > The development version of rgdal on R-Forge is now at rev 894, and is
> now
> > ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
> > migrate within 6 months to modern CRS representations. The motivating
> RFC is
> > also updated to cover coordinate operations, the use of prepared
> > (pre-searched) coordinate operations, and should be read carefully by
> anyone
> > using rgdal::spTransform(). Note further that rgdal::project() will not
> be
> > adapted for PROJ6, and is effectively deprecated.
> >
> > I'll be running reverse dependency checks, and may be bugging package
> > maintainers. I would really prefer that mainainers of packages using
> > spTransform() checked themselves and joined this thread or the
> associated
> > twitter thread:
> https://twitter.com/RogerBivand/status/1194586193108914177
> >
> > Be ready for modern PROJ and GDAL, they are already being deployed
> across
> > open source geospatial software, like GRASS, QGIS, pyproj, spatialite
> etc.
> >
> > Waiting, hopefully not in vain, for contributions.
> >
> > Roger
> >
> > On Wed, 13 Nov 2019, Roger Bivand wrote:
> >
> >>  And this link explains the CDN proposal for grid distribution:
> >>
> >>  https://www.spatialys.com/en/crowdfunding/
> >>
> >>  Roger
> >>
> >>  On Wed, 13 Nov 2019, Roger Bivand wrote:
> >>
> >>>   Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
> >>>   (representations of coordinate reference systems) are handled, steps
> are
> >>>   being taken to find ways to adapt sp/rgdal workflows. A current
> proposal
> >>>   is to store the WKT2_2018 string as a comment to CRS objects as
> defined
> >>>   in
> >>>   the sp package.
> >>>
> >>>   A draft development-in-progress version of rgdal is available at
> >>>   https://r-forge.r-project.org/R/?group_id=884, and for sp at
> >>>   https://github.com/rsbivand/sp (this version of sp requires rgdal >=
> >>>   1.5-1). This adds the WKT comments to CRS objects on reading vector
> and
> >>>   raster data sources, and uses WKT comments if found when writing
> vector
> >>>   and raster objects (or at least does as far as I've checked, possibly
> >>>   fragile).
> >>>
> >>>   An RFC with tersely worked cases for using CRS object comments to
> carry
> >>>   WKT strings but maintaining full backward compatibility is online at
> >>>   http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
> >>>
> >>>   If you have other ideas or concerns about trying to use this
> mechanism
> >>>   for
> >>>   sp CRS objects, please contribute at your earliest convenience.
> >>>
> >>>   http://rgdal.r-forge.r-project.org/reference/list_coordOps.html
> shows
> >>>   the
> >>>   beginning of the next step, to query transformation operations to
> find
> >>>   viable coordinate operation pipelines.
> >>>
> >>>   I'm assuming that the previous behaviour (transform without
> considering
> >>>   accuracy with whatever is to hand) is not viable going forward, and
> that
> >>>   we will need two steps: list coordinate operations between source and
> >>>   target CRS (using the WKT comments as better specifications than the
> >>>   PROJ
> >>>   strings), possibly intervene manually to install missing grids, then
> >>>   undertake the coordinate operation.
> >>>
> >>>   The fallback may be simply to choose the least inaccurate available
> >>>   coordinate operation, but this should be a fallback. This means that
> all
> >>>   uses of spTransform() will require intervention.
> >>>
> >>>   Is this OK (it is tiresome but modernises workflows once), or is it
> not
> >>>   OK
> >>>   (no user intervention is crucial)?
> >>>
> >>>   These behaviours may be set in an option, so that package maintainers
> >>>   and
> >>>   users may delay modernisation, but all are undoubtedly served by
> rapid
> >>>   adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj,
> QGIS
> >>>   development versions all state that they list candidate coordinate
> >>>   operations).
> >>>
> >>>   We cannot ship all the grids, they are very bulky, and probably
> nobody
> >>>   needs sub-metre accuracy world-wide. Work in PROJ is starting to
> create
> >>>   a
> >>>   content delivery network for trusted download and mechanisms for
> >>>   registering downloaded grids on user platforms. We would for example
> not
> >>>   want Windows users of rgdal and sf to have to download the same grid
> >>>   twice.
> >>>
> >>>   Comments welcome here and at
> >>>   https://github.com/r-spatial/discuss/issues/28 or
> >>>   https://github.com/r-spatial/sf/issues/1187
> >>>
> >>>   Roger
> >>>
> >>>
> >>
> >>
> >
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
Thanks, we use the local, institutional interface which provides the quality we delivered at the 2014 Geostat summer school. The link will be posted on https://rsbivand.github.io/ECS530_h19/streaming_ecs530_h19.html when available (delayed because I've been off work). We'll just have to use Monday's classes to check that things are working, feedback info in the link above.

Roger

--
Roger Bivand
Norwegian School of Economics
Helleveien 30, 5045 Bergen, Norway
[hidden email]


________________________________________
Fra: Lorenzo Busetto <[hidden email]>
Sendt: fredag 29. november 2019 18.30
Til: Roger Bivand
Kopi: [hidden email]
Emne: Re: [R-sig-Geo] sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Dear Roger,

    I'd be very interested in participating to the stream talk. Is it confirmed?

Concerning testing the connection, I'd be happy to do it but I do not know how much I will be available online in the coming days. You can drop me a mail/DM on twitter, and if I am available I'll gladly help.

Concerning technology, Zoom usually works quite well (https://zoom.us/<https://zoom.us/ent?zcid=3172>)

   regards,

Lorenzo

On Sat, 23 Nov 2019 at 13:04, Roger Bivand <[hidden email]<mailto:[hidden email]>> wrote:
A description of the status now with regard to a prototype resolution is
online at:

https://rsbivand.github.io/ECS530_h19/ECS530_III.html

I'm planning to stream a talk about this at 09:15-11:00 CET on Tuesday 3
December. I need a volunteer to test the streaming link in advance during
next week. I'm unsure which technology to use for remote participants to
provide feedback.

Contributions/comments welcome!

Roger


On Fri, 15 Nov 2019, Roger Bivand wrote:

> The development version of rgdal on R-Forge is now at rev 894, and is now
> ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
> migrate within 6 months to modern CRS representations. The motivating RFC is
> also updated to cover coordinate operations, the use of prepared
> (pre-searched) coordinate operations, and should be read carefully by anyone
> using rgdal::spTransform(). Note further that rgdal::project() will not be
> adapted for PROJ6, and is effectively deprecated.
>
> I'll be running reverse dependency checks, and may be bugging package
> maintainers. I would really prefer that mainainers of packages using
> spTransform() checked themselves and joined this thread or the associated
> twitter thread: https://twitter.com/RogerBivand/status/1194586193108914177
>
> Be ready for modern PROJ and GDAL, they are already being deployed across
> open source geospatial software, like GRASS, QGIS, pyproj, spatialite etc.
>
> Waiting, hopefully not in vain, for contributions.
>
> Roger
>
> On Wed, 13 Nov 2019, Roger Bivand wrote:
>
>>  And this link explains the CDN proposal for grid distribution:
>>
>>  https://www.spatialys.com/en/crowdfunding/
>>
>>  Roger
>>
>>  On Wed, 13 Nov 2019, Roger Bivand wrote:
>>
>>>   Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
>>>   (representations of coordinate reference systems) are handled, steps are
>>>   being taken to find ways to adapt sp/rgdal workflows. A current proposal
>>>   is to store the WKT2_2018 string as a comment to CRS objects as defined
>>>   in
>>>   the sp package.
>>>
>>>   A draft development-in-progress version of rgdal is available at
>>>   https://r-forge.r-project.org/R/?group_id=884, and for sp at
>>>   https://github.com/rsbivand/sp (this version of sp requires rgdal >=
>>>   1.5-1). This adds the WKT comments to CRS objects on reading vector and
>>>   raster data sources, and uses WKT comments if found when writing vector
>>>   and raster objects (or at least does as far as I've checked, possibly
>>>   fragile).
>>>
>>>   An RFC with tersely worked cases for using CRS object comments to carry
>>>   WKT strings but maintaining full backward compatibility is online at
>>>   http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
>>>
>>>   If you have other ideas or concerns about trying to use this mechanism
>>>   for
>>>   sp CRS objects, please contribute at your earliest convenience.
>>>
>>>   http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows
>>>   the
>>>   beginning of the next step, to query transformation operations to find
>>>   viable coordinate operation pipelines.
>>>
>>>   I'm assuming that the previous behaviour (transform without considering
>>>   accuracy with whatever is to hand) is not viable going forward, and that
>>>   we will need two steps: list coordinate operations between source and
>>>   target CRS (using the WKT comments as better specifications than the
>>>   PROJ
>>>   strings), possibly intervene manually to install missing grids, then
>>>   undertake the coordinate operation.
>>>
>>>   The fallback may be simply to choose the least inaccurate available
>>>   coordinate operation, but this should be a fallback. This means that all
>>>   uses of spTransform() will require intervention.
>>>
>>>   Is this OK (it is tiresome but modernises workflows once), or is it not
>>>   OK
>>>   (no user intervention is crucial)?
>>>
>>>   These behaviours may be set in an option, so that package maintainers
>>>   and
>>>   users may delay modernisation, but all are undoubtedly served by rapid
>>>   adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
>>>   development versions all state that they list candidate coordinate
>>>   operations).
>>>
>>>   We cannot ship all the grids, they are very bulky, and probably nobody
>>>   needs sub-metre accuracy world-wide. Work in PROJ is starting to create
>>>   a
>>>   content delivery network for trusted download and mechanisms for
>>>   registering downloaded grids on user platforms. We would for example not
>>>   want Windows users of rgdal and sf to have to download the same grid
>>>   twice.
>>>
>>>   Comments welcome here and at
>>>   https://github.com/r-spatial/discuss/issues/28 or
>>>   https://github.com/r-spatial/sf/issues/1187
>>>
>>>   Roger
>>>
>>>
>>
>>
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]<mailto:[hidden email]>
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]<mailto:[hidden email]>
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Roger Bivand
Administrator
Streaming link now published (thanks to Arild Schanke), only live when
transmission active.

Roger

On Fri, 29 Nov 2019, Roger Bivand wrote:

> Thanks, we use the local, institutional interface which provides the
> quality we delivered at the 2014 Geostat summer school. The link will be
> posted on
> https://rsbivand.github.io/ECS530_h19/streaming_ecs530_h19.html when
> available (delayed because I've been off work). We'll just have to use
> Monday's classes to check that things are working, feedback info in the
> link above.
>
> Roger
>
> --
> Roger Bivand
> Norwegian School of Economics
> Helleveien 30, 5045 Bergen, Norway
> [hidden email]
>
>
> ________________________________________
> Fra: Lorenzo Busetto <[hidden email]>
> Sendt: fredag 29. november 2019 18.30
> Til: Roger Bivand
> Kopi: [hidden email]
> Emne: Re: [R-sig-Geo] sp/rgdal workflows with PROJ >= 6 and GDAL >= 3
>
> Dear Roger,
>
>    I'd be very interested in participating to the stream talk. Is it confirmed?
>
> Concerning testing the connection, I'd be happy to do it but I do not know how much I will be available online in the coming days. You can drop me a mail/DM on twitter, and if I am available I'll gladly help.
>
> Concerning technology, Zoom usually works quite well (https://zoom.us/<https://zoom.us/ent?zcid=3172>)
>
>   regards,
>
> Lorenzo
>
> On Sat, 23 Nov 2019 at 13:04, Roger Bivand <[hidden email]<mailto:[hidden email]>> wrote:
> A description of the status now with regard to a prototype resolution is
> online at:
>
> https://rsbivand.github.io/ECS530_h19/ECS530_III.html
>
> I'm planning to stream a talk about this at 09:15-11:00 CET on Tuesday 3
> December. I need a volunteer to test the streaming link in advance during
> next week. I'm unsure which technology to use for remote participants to
> provide feedback.
>
> Contributions/comments welcome!
>
> Roger
>
>
> On Fri, 15 Nov 2019, Roger Bivand wrote:
>
>> The development version of rgdal on R-Forge is now at rev 894, and is now
>> ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
>> migrate within 6 months to modern CRS representations. The motivating RFC is
>> also updated to cover coordinate operations, the use of prepared
>> (pre-searched) coordinate operations, and should be read carefully by anyone
>> using rgdal::spTransform(). Note further that rgdal::project() will not be
>> adapted for PROJ6, and is effectively deprecated.
>>
>> I'll be running reverse dependency checks, and may be bugging package
>> maintainers. I would really prefer that mainainers of packages using
>> spTransform() checked themselves and joined this thread or the associated
>> twitter thread: https://twitter.com/RogerBivand/status/1194586193108914177
>>
>> Be ready for modern PROJ and GDAL, they are already being deployed across
>> open source geospatial software, like GRASS, QGIS, pyproj, spatialite etc.
>>
>> Waiting, hopefully not in vain, for contributions.
>>
>> Roger
>>
>> On Wed, 13 Nov 2019, Roger Bivand wrote:
>>
>>>  And this link explains the CDN proposal for grid distribution:
>>>
>>>  https://www.spatialys.com/en/crowdfunding/
>>>
>>>  Roger
>>>
>>>  On Wed, 13 Nov 2019, Roger Bivand wrote:
>>>
>>>>   Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
>>>>   (representations of coordinate reference systems) are handled, steps are
>>>>   being taken to find ways to adapt sp/rgdal workflows. A current proposal
>>>>   is to store the WKT2_2018 string as a comment to CRS objects as defined
>>>>   in
>>>>   the sp package.
>>>>
>>>>   A draft development-in-progress version of rgdal is available at
>>>>   https://r-forge.r-project.org/R/?group_id=884, and for sp at
>>>>   https://github.com/rsbivand/sp (this version of sp requires rgdal >=
>>>>   1.5-1). This adds the WKT comments to CRS objects on reading vector and
>>>>   raster data sources, and uses WKT comments if found when writing vector
>>>>   and raster objects (or at least does as far as I've checked, possibly
>>>>   fragile).
>>>>
>>>>   An RFC with tersely worked cases for using CRS object comments to carry
>>>>   WKT strings but maintaining full backward compatibility is online at
>>>>   http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
>>>>
>>>>   If you have other ideas or concerns about trying to use this mechanism
>>>>   for
>>>>   sp CRS objects, please contribute at your earliest convenience.
>>>>
>>>>   http://rgdal.r-forge.r-project.org/reference/list_coordOps.html shows
>>>>   the
>>>>   beginning of the next step, to query transformation operations to find
>>>>   viable coordinate operation pipelines.
>>>>
>>>>   I'm assuming that the previous behaviour (transform without considering
>>>>   accuracy with whatever is to hand) is not viable going forward, and that
>>>>   we will need two steps: list coordinate operations between source and
>>>>   target CRS (using the WKT comments as better specifications than the
>>>>   PROJ
>>>>   strings), possibly intervene manually to install missing grids, then
>>>>   undertake the coordinate operation.
>>>>
>>>>   The fallback may be simply to choose the least inaccurate available
>>>>   coordinate operation, but this should be a fallback. This means that all
>>>>   uses of spTransform() will require intervention.
>>>>
>>>>   Is this OK (it is tiresome but modernises workflows once), or is it not
>>>>   OK
>>>>   (no user intervention is crucial)?
>>>>
>>>>   These behaviours may be set in an option, so that package maintainers
>>>>   and
>>>>   users may delay modernisation, but all are undoubtedly served by rapid
>>>>   adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj, QGIS
>>>>   development versions all state that they list candidate coordinate
>>>>   operations).
>>>>
>>>>   We cannot ship all the grids, they are very bulky, and probably nobody
>>>>   needs sub-metre accuracy world-wide. Work in PROJ is starting to create
>>>>   a
>>>>   content delivery network for trusted download and mechanisms for
>>>>   registering downloaded grids on user platforms. We would for example not
>>>>   want Windows users of rgdal and sf to have to download the same grid
>>>>   twice.
>>>>
>>>>   Comments welcome here and at
>>>>   https://github.com/r-spatial/discuss/issues/28 or
>>>>   https://github.com/r-spatial/sf/issues/1187
>>>>
>>>>   Roger
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]<mailto:[hidden email]>
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]<mailto:[hidden email]>
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: sp/rgdal workflows with PROJ >= 6 and GDAL >= 3

Lorenzo Busetto
Thanks. I'll check it out on Monday and let you know if there are any
issues.

Lorenzo

On Sat, 30 Nov 2019, 11:53 Roger Bivand, <[hidden email]> wrote:

> Streaming link now published (thanks to Arild Schanke), only live when
> transmission active.
>
> Roger
>
> On Fri, 29 Nov 2019, Roger Bivand wrote:
>
> > Thanks, we use the local, institutional interface which provides the
> > quality we delivered at the 2014 Geostat summer school. The link will be
> > posted on
> > https://rsbivand.github.io/ECS530_h19/streaming_ecs530_h19.html when
> > available (delayed because I've been off work). We'll just have to use
> > Monday's classes to check that things are working, feedback info in the
> > link above.
> >
> > Roger
> >
> > --
> > Roger Bivand
> > Norwegian School of Economics
> > Helleveien 30, 5045 Bergen, Norway
> > [hidden email]
> >
> >
> > ________________________________________
> > Fra: Lorenzo Busetto <[hidden email]>
> > Sendt: fredag 29. november 2019 18.30
> > Til: Roger Bivand
> > Kopi: [hidden email]
> > Emne: Re: [R-sig-Geo] sp/rgdal workflows with PROJ >= 6 and GDAL >= 3
> >
> > Dear Roger,
> >
> >    I'd be very interested in participating to the stream talk. Is it
> confirmed?
> >
> > Concerning testing the connection, I'd be happy to do it but I do not
> know how much I will be available online in the coming days. You can drop
> me a mail/DM on twitter, and if I am available I'll gladly help.
> >
> > Concerning technology, Zoom usually works quite well (https://zoom.us/<
> https://zoom.us/ent?zcid=3172>)
> >
> >   regards,
> >
> > Lorenzo
> >
> > On Sat, 23 Nov 2019 at 13:04, Roger Bivand <[hidden email]<mailto:
> [hidden email]>> wrote:
> > A description of the status now with regard to a prototype resolution is
> > online at:
> >
> > https://rsbivand.github.io/ECS530_h19/ECS530_III.html
> >
> > I'm planning to stream a talk about this at 09:15-11:00 CET on Tuesday 3
> > December. I need a volunteer to test the streaming link in advance during
> > next week. I'm unsure which technology to use for remote participants to
> > provide feedback.
> >
> > Contributions/comments welcome!
> >
> > Roger
> >
> >
> > On Fri, 15 Nov 2019, Roger Bivand wrote:
> >
> >> The development version of rgdal on R-Forge is now at rev 894, and is
> now
> >> ready for trying out with PROJ6/GDAL3 workflows, and workflows that may
> >> migrate within 6 months to modern CRS representations. The motivating
> RFC is
> >> also updated to cover coordinate operations, the use of prepared
> >> (pre-searched) coordinate operations, and should be read carefully by
> anyone
> >> using rgdal::spTransform(). Note further that rgdal::project() will not
> be
> >> adapted for PROJ6, and is effectively deprecated.
> >>
> >> I'll be running reverse dependency checks, and may be bugging package
> >> maintainers. I would really prefer that mainainers of packages using
> >> spTransform() checked themselves and joined this thread or the
> associated
> >> twitter thread:
> https://twitter.com/RogerBivand/status/1194586193108914177
> >>
> >> Be ready for modern PROJ and GDAL, they are already being deployed
> across
> >> open source geospatial software, like GRASS, QGIS, pyproj, spatialite
> etc.
> >>
> >> Waiting, hopefully not in vain, for contributions.
> >>
> >> Roger
> >>
> >> On Wed, 13 Nov 2019, Roger Bivand wrote:
> >>
> >>>  And this link explains the CDN proposal for grid distribution:
> >>>
> >>>  https://www.spatialys.com/en/crowdfunding/
> >>>
> >>>  Roger
> >>>
> >>>  On Wed, 13 Nov 2019, Roger Bivand wrote:
> >>>
> >>>>   Because PROJ >= 6 and GDAL >= 3 change the way that PROJ strings
> >>>>   (representations of coordinate reference systems) are handled,
> steps are
> >>>>   being taken to find ways to adapt sp/rgdal workflows. A current
> proposal
> >>>>   is to store the WKT2_2018 string as a comment to CRS objects as
> defined
> >>>>   in
> >>>>   the sp package.
> >>>>
> >>>>   A draft development-in-progress version of rgdal is available at
> >>>>   https://r-forge.r-project.org/R/?group_id=884, and for sp at
> >>>>   https://github.com/rsbivand/sp (this version of sp requires rgdal
> >=
> >>>>   1.5-1). This adds the WKT comments to CRS objects on reading vector
> and
> >>>>   raster data sources, and uses WKT comments if found when writing
> vector
> >>>>   and raster objects (or at least does as far as I've checked,
> possibly
> >>>>   fragile).
> >>>>
> >>>>   An RFC with tersely worked cases for using CRS object comments to
> carry
> >>>>   WKT strings but maintaining full backward compatibility is online at
> >>>>   http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html.
> >>>>
> >>>>   If you have other ideas or concerns about trying to use this
> mechanism
> >>>>   for
> >>>>   sp CRS objects, please contribute at your earliest convenience.
> >>>>
> >>>>   http://rgdal.r-forge.r-project.org/reference/list_coordOps.html
> shows
> >>>>   the
> >>>>   beginning of the next step, to query transformation operations to
> find
> >>>>   viable coordinate operation pipelines.
> >>>>
> >>>>   I'm assuming that the previous behaviour (transform without
> considering
> >>>>   accuracy with whatever is to hand) is not viable going forward, and
> that
> >>>>   we will need two steps: list coordinate operations between source
> and
> >>>>   target CRS (using the WKT comments as better specifications than the
> >>>>   PROJ
> >>>>   strings), possibly intervene manually to install missing grids, then
> >>>>   undertake the coordinate operation.
> >>>>
> >>>>   The fallback may be simply to choose the least inaccurate available
> >>>>   coordinate operation, but this should be a fallback. This means
> that all
> >>>>   uses of spTransform() will require intervention.
> >>>>
> >>>>   Is this OK (it is tiresome but modernises workflows once), or is it
> not
> >>>>   OK
> >>>>   (no user intervention is crucial)?
> >>>>
> >>>>   These behaviours may be set in an option, so that package
> maintainers
> >>>>   and
> >>>>   users may delay modernisation, but all are undoubtedly served by
> rapid
> >>>>   adaptation (GRASS 7.8.1 released yesterday, libspatialite, pyproj,
> QGIS
> >>>>   development versions all state that they list candidate coordinate
> >>>>   operations).
> >>>>
> >>>>   We cannot ship all the grids, they are very bulky, and probably
> nobody
> >>>>   needs sub-metre accuracy world-wide. Work in PROJ is starting to
> create
> >>>>   a
> >>>>   content delivery network for trusted download and mechanisms for
> >>>>   registering downloaded grids on user platforms. We would for
> example not
> >>>>   want Windows users of rgdal and sf to have to download the same grid
> >>>>   twice.
> >>>>
> >>>>   Comments welcome here and at
> >>>>   https://github.com/r-spatial/discuss/issues/28 or
> >>>>   https://github.com/r-spatial/sf/issues/1187
> >>>>
> >>>>   Roger
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> > --
> > Roger Bivand
> > Department of Economics, Norwegian School of Economics,
> > Helleveien 30, N-5045 Bergen, Norway.
> > voice: +47 55 95 93 55; e-mail: [hidden email]<mailto:
> [hidden email]>
> > https://orcid.org/0000-0003-2392-6140
> > https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]<mailto:[hidden email]>
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo