Question spgwr package - running time gwr()

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Question spgwr package - running time gwr()

R-sig-geo mailing list
Dear all,

Is there an option to shorten the running time for the gwr() function, similar to the ‚LU‘ method for lagsarlm() in the spdep package? Because I have a model with roughly 500’000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.

Thank you for your help in advance.

Best regards,

Raphael Mesaric
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Question spgwr package - running time gwr()

Roger Bivand
Administrator
On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:

> Dear all,
>
> Is there an option to shorten the running time for the gwr() function,
> similar to the ‚LU‘ method for lagsarlm() in the spdep package? Because
> I have a model with roughly 500’000 observations, and the running time
> at the moment is quite long, respectively it has not yet terminated.

What are you actually doing? Why did you choose GWR? Are you fitting a GWR
with 500K observations, or have you fitted a GWR with many fewer
observations, and are now rendering that fitted model with 500K fit
points? GWR is only for detecting possible non-stationarity or similar
mis-specification in moderately sized data sets. Trying to fit with 500K
gives a dense hat matrix of 500K x 500K, which is imposssible (or were it
possible would be uninformative). Think of 5K as a sensible maximum if GWR
is condidered sensible at all. I would think that finding a bandwidth is
impossible too.

Roger

>
> Thank you for your help in advance.
>
> Best regards,
>
> Raphael Mesaric
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Question spgwr package - running time gwr()

R-sig-geo mailing list
Dear Roger,

Thank you very much for your reply.

My thesis supervisor told me to use GWR to explore small-scale differences and test the consistency of the SAR models.

So, I tried to fit the GWR model with all my observations. However, if I understood you correctly, I would have to choose smaller sections of the grid and try to fit a GWR there. What I am not sure about how to do this in order to get informative results.

As for the SAR model, the dependent variable consists n blocks where the m-th entry of each block corresponds to the m-th cell of my grid. So, if I have to reduce the grid to about 5K entries, I would need to take a subset of roughly 250 cells and then take all the 20 observations for each of these cells? And then try to fit a GWR there?

Best,

Raphael


> Am 17.05.2019 um 14:20 schrieb Roger Bivand <[hidden email]>:
>
> On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:
>
>> Dear all,
>>
>> Is there an option to shorten the running time for the gwr() function, similar to the ‚LU‘ method for lagsarlm() in the spdep package? Because I have a model with roughly 500’000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.
>
> What are you actually doing? Why did you choose GWR? Are you fitting a GWR with 500K observations, or have you fitted a GWR with many fewer observations, and are now rendering that fitted model with 500K fit points? GWR is only for detecting possible non-stationarity or similar mis-specification in moderately sized data sets. Trying to fit with 500K gives a dense hat matrix of 500K x 500K, which is imposssible (or were it possible would be uninformative). Think of 5K as a sensible maximum if GWR is condidered sensible at all. I would think that finding a bandwidth is impossible too.
>
> Roger
>
>>
>> Thank you for your help in advance.
>>
>> Best regards,
>>
>> Raphael Mesaric
>> _______________________________________________
>> R-sig-Geo mailing list
>> [hidden email]
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Question spgwr package - running time gwr()

Roger Bivand
Administrator
On Fri, 17 May 2019, Raphael Mesaric wrote:

> Dear Roger,
>
> Thank you very much for your reply.
>
> My thesis supervisor told me to use GWR to explore small-scale
> differences and test the consistency of the SAR models.

It happens. From your problem description, I'm unsure whether GWR or SAR
are appropriate methods, and would suggest the mixed-model and multilevel
literatures. Mapable IID and spatially structured random effects may be
more appropriate.

Roger

>
> So, I tried to fit the GWR model with all my observations. However, if I
> understood you correctly, I would have to choose smaller sections of the
> grid and try to fit a GWR there. What I am not sure about how to do this
> in order to get informative results.
>
> As for the SAR model, the dependent variable consists n blocks where the
> m-th entry of each block corresponds to the m-th cell of my grid. So, if
> I have to reduce the grid to about 5K entries, I would need to take a
> subset of roughly 250 cells and then take all the 20 observations for
> each of these cells? And then try to fit a GWR there?
>
> Best,
>
> Raphael
>
>
>> Am 17.05.2019 um 14:20 schrieb Roger Bivand <[hidden email]>:
>>
>> On Fri, 17 May 2019, Raphael Mesaric via R-sig-Geo wrote:
>>
>>> Dear all,
>>>
>>> Is there an option to shorten the running time for the gwr() function, similar to the ‚LU‘ method for lagsarlm() in the spdep package? Because I have a model with roughly 500’000 observations, and the running time at the moment is quite long, respectively it has not yet terminated.
>>
>> What are you actually doing? Why did you choose GWR? Are you fitting a GWR with 500K observations, or have you fitted a GWR with many fewer observations, and are now rendering that fitted model with 500K fit points? GWR is only for detecting possible non-stationarity or similar mis-specification in moderately sized data sets. Trying to fit with 500K gives a dense hat matrix of 500K x 500K, which is imposssible (or were it possible would be uninformative). Think of 5K as a sensible maximum if GWR is condidered sensible at all. I would think that finding a bandwidth is impossible too.
>>
>> Roger
>>
>>>
>>> Thank you for your help in advance.
>>>
>>> Best regards,
>>>
>>> Raphael Mesaric
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> [hidden email]
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway