Error in lag.listw(listw2, u) : Variable contains non-finite values - panel spatial model using splm

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Error in lag.listw(listw2, u) : Variable contains non-finite values - panel spatial model using splm

R-sig-geo mailing list
Dear community,

I am working on a panel spatial model in R using the package splm. In particular I am trying to estimate the following model

spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect = "twoways",zero.policy=TRUE)

but I get the following error and warnings:

Error in lag.listw(listw2, u) : Variable contains non-finite values
In addition: Warning messages:
1: In lag.listw(listw, u) : NAs in lagged values
2: In lag.listw(listw, u) : NAs in lagged values
3: In lag.listw(listw2, u) : NAs in lagged values
4: In lag.listw(listw2, u) : NAs in lagged values


Similarly, if I model the spatial correlation only for the error term,

spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect = "twoways",zero.policy=TRUE)

I get the following warnings

1: In lag.listw(listw, u) : NAs in lagged values

...

3: In lag.listw(listw, TT) : NAs in lagged values

...

35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
NA/Inf replaced by maximum positive value

...


In this case however the model turns out to be estimated, despite the standard errors are everywhere NAs and the spatial rho coefficient equal to 1 with weird t stats and p-values.

An identical problem to mine was already signalled in this mailing list, unfortunately without receiving any suggestion : http://r-sig-geo.2731867.n2.nabble.com/problems-using-spml-with-a-listw-where-not-everybody-has-a-neighbour-td7587857.html.

I guess that the issue could be due to the n by n spatial weighting matrix W30, which contains some neighbourless observations. However this feature should be taken into account by the zero.policy option. Moreover the same framework perfectly works in the cross-sectional case, by using the function sacsarlm from the package spdep.

Please, does anybody know what is causing this and how can I solve this issue? Any help would be really appreciated.

Kind Regards,

Marco Mello

 
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Error in lag.listw(listw2, u) : Variable contains non-finite values - panel spatial model using splm

Roger Bivand
Administrator
On Mon, 10 Jun 2019, Marco Mello via R-sig-Geo wrote:

> Dear community,

Please post plain text, not HTML, easier to copy and paste into an R
session.

>
> I am working on a panel spatial model in R using the package splm. In
> particular I am trying to estimate the following model
>
> spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
> "twoways",zero.policy=TRUE)
>
> but I get the following error and warnings:
>
> Error in lag.listw(listw2, u) : Variable contains non-finite values
> In addition: Warning messages:
> 1: In lag.listw(listw, u) : NAs in lagged values
> 2: In lag.listw(listw, u) : NAs in lagged values
> 3: In lag.listw(listw2, u) : NAs in lagged values
> 4: In lag.listw(listw2, u) : NAs in lagged values
>

This is not a reproducible example. Such an example is needed unless you
can run traceback() and debug() yourself to solve the problem, but since
you have posted, I assume you prefer that someone else runs debug() - and
someone will then need a reproducible example, preferably with an adapted
standard dataset (add NAs to Produc?).

I seem to recall that zero.policy= was not always passed through in some
model fitting functions, possibly in splm. If so, set the option with
spatialreg::set.ZeroPolicyOption(TRUE) and/or
spdep::set.ZeroPolicyOption(TRUE)
to avoid that issue if it is present and biting. Of course, had I had a
reproducible example, I could have checked and given clearer advice.

>
> Similarly, if I model the spatial correlation only for the error term,
>
> spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
> "twoways",zero.policy=TRUE)
>
> I get the following warnings
>
> 1: In lag.listw(listw, u) : NAs in lagged values
>
> ...
>
> 3: In lag.listw(listw, TT) : NAs in lagged values
>
> ...
>
> 35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
> NA/Inf replaced by maximum positive value
>
> ...
>
>
> In this case however the model turns out to be estimated, despite the
> standard errors are everywhere NAs and the spatial rho coefficient equal
> to 1 with weird t stats and p-values.
>
> An identical problem to mine was already signalled in this mailing list,
> unfortunately without receiving any suggestion :
> http://r-sig-geo.2731867.n2.nabble.com/problems-using-spml-with-a-listw-where-not-everybody-has-a-neighbour-td7587857.html.

Nabble is only an archive, the real link is:

https://stat.ethz.ch/pipermail/r-sig-geo/2015-March/022428.html

The posting also gave no reproducible example, so nobody had anything to
work on.

Hope this helps,

Roger

>
> I guess that the issue could be due to the n by n spatial weighting
> matrix W30, which contains some neighbourless observations. However this
> feature should be taken into account by the zero.policy option. Moreover
> the same framework perfectly works in the cross-sectional case, by using
> the function sacsarlm from the package spdep.
>
> Please, does anybody know what is causing this and how can I solve this
> issue? Any help would be really appreciated.
>
> Kind Regards,
>
> Marco Mello
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Error in lag.listw(listw2, u) : Variable contains non-finite values - panel spatial model using splm

R-sig-geo mailing list
Dear Roger,

many thanks for your help. Actually the issue was exactly the zero.policy option. I solved this by imposing spdep::set.ZeroPolicyOption(TRUE). Now the estimation works. This solution should apply to all weighting matrices having rows with all zeros, i.e. neighbourless observations.

I notice however that results are different between those obtained if we estimate the model with both individual and time fixed effects by setting effect="twoways" and those obtained by estimating the model with just one type of fixed effect (e.g. effect="individual") and adding the other ones by hand in the formula.

I will add a reproducible example using the Produc dataset to show this issue.





library(plm)
library(spdep)
library(splm)

data(Produc, package = "plm")
data(usaww)

##effect="twoways"

fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp, data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "twoways", spatial.error="b", Hess = FALSE)

summary(fespaterr)

##effect="individual" and time fixed effects by hand

fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(year), data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "individual", spatial.error="b", Hess = FALSE)

summary(fespaterr)

##effect="time" and individual fixed effects by hand

fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(state), data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "time", spatial.error="b", Hess = FALSE)

summary(fespaterr)





Is this correct?

In this example the coefficients are just slightly different between the three specifications. In my case however, with T=2, these three specifications provide completely different results.

Thank you again for your help,

Marco



>
>     Il 10 giugno 2019 alle 19.42 Roger Bivand <[hidden email]> ha scritto:
>
>     On Mon, 10 Jun 2019, Marco Mello via R-sig-Geo wrote:
>
>         > >
> >         Dear community,
> >
> >     >
>     Please post plain text, not HTML, easier to copy and paste into an R
>     session.
>
>     >
>
>         > >
> >         I am working on a panel spatial model in R using the package splm. In
> >         particular I am trying to estimate the following model
> >
> >         spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> >         = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
> >         "twoways",zero.policy=TRUE)
> >
> >         but I get the following error and warnings:
> >
> >         Error in lag.listw(listw2, u) : Variable contains non-finite values
> >         In addition: Warning messages:
> >         1: In lag.listw(listw, u) : NAs in lagged values
> >         2: In lag.listw(listw, u) : NAs in lagged values
> >         3: In lag.listw(listw2, u) : NAs in lagged values
> >         4: In lag.listw(listw2, u) : NAs in lagged values
> >
> >     >
>     This is not a reproducible example. Such an example is needed unless you
>     can run traceback() and debug() yourself to solve the problem, but since
>     you have posted, I assume you prefer that someone else runs debug() - and
>     someone will then need a reproducible example, preferably with an adapted
>     standard dataset (add NAs to Produc?).
>
>     I seem to recall that zero.policy= was not always passed through in some
>     model fitting functions, possibly in splm. If so, set the option with
>     spatialreg::set.ZeroPolicyOption(TRUE) and/or
>     spdep::set.ZeroPolicyOption(TRUE)
>     to avoid that issue if it is present and biting. Of course, had I had a
>     reproducible example, I could have checked and given clearer advice.
>
>     >
>
>         > >
> >         Similarly, if I model the spatial correlation only for the error term,
> >
> >         spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> >         = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
> >         "twoways",zero.policy=TRUE)
> >
> >         I get the following warnings
> >
> >         1: In lag.listw(listw, u) : NAs in lagged values
> >
> >         ...
> >
> >         3: In lag.listw(listw, TT) : NAs in lagged values
> >
> >         ...
> >
> >         35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
> >         NA/Inf replaced by maximum positive value
> >
> >         ...
> >
> >         In this case however the model turns out to be estimated, despite the
> >         standard errors are everywhere NAs and the spatial rho coefficient equal
> >         to 1 with weird t stats and p-values.
> >
> >         An identical problem to mine was already signalled in this mailing list,
> >         unfortunately without receiving any suggestion :
> >         http://r-sig-geo.2731867.n2.nabble.com/problems-using-spml-with-a-listw-where-not-everybody-has-a-neighbour-td7587857.html.
> >
> >     >
>     Nabble is only an archive, the real link is:
>
>     https://stat.ethz.ch/pipermail/r-sig-geo/2015-March/022428.html
>
>     The posting also gave no reproducible example, so nobody had anything to
>     work on.
>
>     Hope this helps,
>
>     Roger
>
>     >
>
>         > >
> >         I guess that the issue could be due to the n by n spatial weighting
> >         matrix W30, which contains some neighbourless observations. However this
> >         feature should be taken into account by the zero.policy option. Moreover
> >         the same framework perfectly works in the cross-sectional case, by using
> >         the function sacsarlm from the package spdep.
> >
> >         Please, does anybody know what is causing this and how can I solve this
> >         issue? Any help would be really appreciated.
> >
> >         Kind Regards,
> >
> >         Marco Mello
> >
> >         [[alternative HTML version deleted]]
> >
> >         _______________________________________________
> >         R-sig-Geo mailing list
> >         [hidden email]
> >         https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> >     >
>     --
>     Roger Bivand
>     Department of Economics, Norwegian School of Economics,
>     Helleveien 30, N-5045 Bergen, Norway.
>     voice: +47 55 95 93 55; e-mail: [hidden email]
>     https://orcid.org/0000-0003-2392-6140
>     https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Error in lag.listw(listw2, u) : Variable contains non-finite values - panel spatial model using splm

Roger Bivand
Administrator
On Tue, 11 Jun 2019, [hidden email] wrote:

> Dear Roger,
>
> many thanks for your help. Actually the issue was exactly the
> zero.policy option. I solved this by imposing
> spdep::set.ZeroPolicyOption(TRUE). Now the estimation works. This
> solution should apply to all weighting matrices having rows with all
> zeros, i.e. neighbourless observations.
>

OK, good, my memory isn't so bad after all ...

> I notice however that results are different between those obtained if we
> estimate the model with both individual and time fixed effects by
> setting effect="twoways" and those obtained by estimating the model with
> just one type of fixed effect (e.g. effect="individual") and adding the
> other ones by hand in the formula.
>
> I will add a reproducible example using the Produc dataset to show this
> issue.
>

Thanks for the example, I've CC'ed the splm maintainer to elicit more
insight.

Roger

>
>
>
>
> library(plm)
> library(spdep)
> library(splm)
>
> data(Produc, package = "plm")
> data(usaww)
>
> ##effect="twoways"
>
> fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp, data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "twoways", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
> ##effect="individual" and time fixed effects by hand
>
> fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(year), data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "individual", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
> ##effect="time" and individual fixed effects by hand
>
> fespaterr <- spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(state), data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "time", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
>
>
>
>
> Is this correct?
>
> In this example the coefficients are just slightly different between the three specifications. In my case however, with T=2, these three specifications provide completely different results.
>
> Thank you again for your help,
>
> Marco
>
>
>
>>
>>     Il 10 giugno 2019 alle 19.42 Roger Bivand <[hidden email]> ha scritto:
>>
>>     On Mon, 10 Jun 2019, Marco Mello via R-sig-Geo wrote:
>>
>>        >>
>>>         Dear community,
>>>
>>>    >
>>     Please post plain text, not HTML, easier to copy and paste into an R
>>     session.
>>
>>    >
>>
>>        >>
>>>         I am working on a panel spatial model in R using the package splm. In
>>>         particular I am trying to estimate the following model
>>>
>>>         spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
>>>         = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
>>>         "twoways",zero.policy=TRUE)
>>>
>>>         but I get the following error and warnings:
>>>
>>>         Error in lag.listw(listw2, u) : Variable contains non-finite values
>>>         In addition: Warning messages:
>>>         1: In lag.listw(listw, u) : NAs in lagged values
>>>         2: In lag.listw(listw, u) : NAs in lagged values
>>>         3: In lag.listw(listw2, u) : NAs in lagged values
>>>         4: In lag.listw(listw2, u) : NAs in lagged values
>>>
>>>    >
>>     This is not a reproducible example. Such an example is needed unless you
>>     can run traceback() and debug() yourself to solve the problem, but since
>>     you have posted, I assume you prefer that someone else runs debug() - and
>>     someone will then need a reproducible example, preferably with an adapted
>>     standard dataset (add NAs to Produc?).
>>
>>     I seem to recall that zero.policy= was not always passed through in some
>>     model fitting functions, possibly in splm. If so, set the option with
>>     spatialreg::set.ZeroPolicyOption(TRUE) and/or
>>     spdep::set.ZeroPolicyOption(TRUE)
>>     to avoid that issue if it is present and biting. Of course, had I had a
>>     reproducible example, I could have checked and given clearer advice.
>>
>>    >
>>
>>        >>
>>>         Similarly, if I model the spatial correlation only for the error term,
>>>
>>>         spatpan<-spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
>>>         = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
>>>         "twoways",zero.policy=TRUE)
>>>
>>>         I get the following warnings
>>>
>>>         1: In lag.listw(listw, u) : NAs in lagged values
>>>
>>>         ...
>>>
>>>         3: In lag.listw(listw, TT) : NAs in lagged values
>>>
>>>         ...
>>>
>>>         35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
>>>         NA/Inf replaced by maximum positive value
>>>
>>>         ...
>>>
>>>         In this case however the model turns out to be estimated, despite the
>>>         standard errors are everywhere NAs and the spatial rho coefficient equal
>>>         to 1 with weird t stats and p-values.
>>>
>>>         An identical problem to mine was already signalled in this mailing list,
>>>         unfortunately without receiving any suggestion :
>>>         http://r-sig-geo.2731867.n2.nabble.com/problems-using-spml-with-a-listw-where-not-everybody-has-a-neighbour-td7587857.html.
>>>
>>>    >
>>     Nabble is only an archive, the real link is:
>>
>>     https://stat.ethz.ch/pipermail/r-sig-geo/2015-March/022428.html
>>
>>     The posting also gave no reproducible example, so nobody had anything to
>>     work on.
>>
>>     Hope this helps,
>>
>>     Roger
>>
>>    >
>>
>>        >>
>>>         I guess that the issue could be due to the n by n spatial weighting
>>>         matrix W30, which contains some neighbourless observations. However this
>>>         feature should be taken into account by the zero.policy option. Moreover
>>>         the same framework perfectly works in the cross-sectional case, by using
>>>         the function sacsarlm from the package spdep.
>>>
>>>         Please, does anybody know what is causing this and how can I solve this
>>>         issue? Any help would be really appreciated.
>>>
>>>         Kind Regards,
>>>
>>>         Marco Mello
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>>         _______________________________________________
>>>         R-sig-Geo mailing list
>>>         [hidden email]
>>>         https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>>    >
>>     --
>>     Roger Bivand
>>     Department of Economics, Norwegian School of Economics,
>>     Helleveien 30, N-5045 Bergen, Norway.
>>     voice: +47 55 95 93 55; e-mail: [hidden email]
>>     https://orcid.org/0000-0003-2392-6140
>>     https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway