

Dear community,
I am working on a panel spatial model in R using the package splm. In particular I am trying to estimate the following model
spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect = "twoways",zero.policy=TRUE)
but I get the following error and warnings:
Error in lag.listw(listw2, u) : Variable contains nonfinite values
In addition: Warning messages:
1: In lag.listw(listw, u) : NAs in lagged values
2: In lag.listw(listw, u) : NAs in lagged values
3: In lag.listw(listw2, u) : NAs in lagged values
4: In lag.listw(listw2, u) : NAs in lagged values
Similarly, if I model the spatial correlation only for the error term,
spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect = "twoways",zero.policy=TRUE)
I get the following warnings
1: In lag.listw(listw, u) : NAs in lagged values
...
3: In lag.listw(listw, TT) : NAs in lagged values
...
35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
NA/Inf replaced by maximum positive value
...
In this case however the model turns out to be estimated, despite the standard errors are everywhere NAs and the spatial rho coefficient equal to 1 with weird t stats and pvalues.
An identical problem to mine was already signalled in this mailing list, unfortunately without receiving any suggestion : http://rsiggeo.2731867.n2.nabble.com/problemsusingspmlwithalistwwherenoteverybodyhasaneighbourtd7587857.html.
I guess that the issue could be due to the n by n spatial weighting matrix W30, which contains some neighbourless observations. However this feature should be taken into account by the zero.policy option. Moreover the same framework perfectly works in the crosssectional case, by using the function sacsarlm from the package spdep.
Please, does anybody know what is causing this and how can I solve this issue? Any help would be really appreciated.
Kind Regards,
Marco Mello
[[alternative HTML version deleted]]
_______________________________________________
RsigGeo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/rsiggeo

Administrator

On Mon, 10 Jun 2019, Marco Mello via RsigGeo wrote:
> Dear community,
Please post plain text, not HTML, easier to copy and paste into an R
session.
>
> I am working on a panel spatial model in R using the package splm. In
> particular I am trying to estimate the following model
>
> spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
> "twoways",zero.policy=TRUE)
>
> but I get the following error and warnings:
>
> Error in lag.listw(listw2, u) : Variable contains nonfinite values
> In addition: Warning messages:
> 1: In lag.listw(listw, u) : NAs in lagged values
> 2: In lag.listw(listw, u) : NAs in lagged values
> 3: In lag.listw(listw2, u) : NAs in lagged values
> 4: In lag.listw(listw2, u) : NAs in lagged values
>
This is not a reproducible example. Such an example is needed unless you
can run traceback() and debug() yourself to solve the problem, but since
you have posted, I assume you prefer that someone else runs debug()  and
someone will then need a reproducible example, preferably with an adapted
standard dataset (add NAs to Produc?).
I seem to recall that zero.policy= was not always passed through in some
model fitting functions, possibly in splm. If so, set the option with
spatialreg::set.ZeroPolicyOption(TRUE) and/or
spdep::set.ZeroPolicyOption(TRUE)
to avoid that issue if it is present and biting. Of course, had I had a
reproducible example, I could have checked and given clearer advice.
>
> Similarly, if I model the spatial correlation only for the error term,
>
> spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
> "twoways",zero.policy=TRUE)
>
> I get the following warnings
>
> 1: In lag.listw(listw, u) : NAs in lagged values
>
> ...
>
> 3: In lag.listw(listw, TT) : NAs in lagged values
>
> ...
>
> 35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
> NA/Inf replaced by maximum positive value
>
> ...
>
>
> In this case however the model turns out to be estimated, despite the
> standard errors are everywhere NAs and the spatial rho coefficient equal
> to 1 with weird t stats and pvalues.
>
> An identical problem to mine was already signalled in this mailing list,
> unfortunately without receiving any suggestion :
> http://rsiggeo.2731867.n2.nabble.com/problemsusingspmlwithalistwwherenoteverybodyhasaneighbourtd7587857.html.
Nabble is only an archive, the real link is:
https://stat.ethz.ch/pipermail/rsiggeo/2015March/022428.htmlThe posting also gave no reproducible example, so nobody had anything to
work on.
Hope this helps,
Roger
>
> I guess that the issue could be due to the n by n spatial weighting
> matrix W30, which contains some neighbourless observations. However this
> feature should be taken into account by the zero.policy option. Moreover
> the same framework perfectly works in the crosssectional case, by using
> the function sacsarlm from the package spdep.
>
> Please, does anybody know what is causing this and how can I solve this
> issue? Any help would be really appreciated.
>
> Kind Regards,
>
> Marco Mello
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> RsigGeo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/rsiggeo>

Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N5045 Bergen, Norway.
voice: +47 55 95 93 55; email: [hidden email]
https://orcid.org/0000000323926140https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en_______________________________________________
RsigGeo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/rsiggeo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N5045 Bergen, Norway


Dear Roger,
many thanks for your help. Actually the issue was exactly the zero.policy option. I solved this by imposing spdep::set.ZeroPolicyOption(TRUE). Now the estimation works. This solution should apply to all weighting matrices having rows with all zeros, i.e. neighbourless observations.
I notice however that results are different between those obtained if we estimate the model with both individual and time fixed effects by setting effect="twoways" and those obtained by estimating the model with just one type of fixed effect (e.g. effect="individual") and adding the other ones by hand in the formula.
I will add a reproducible example using the Produc dataset to show this issue.
library(plm)
library(spdep)
library(splm)
data(Produc, package = "plm")
data(usaww)
##effect="twoways"
fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp, data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "twoways", spatial.error="b", Hess = FALSE)
summary(fespaterr)
##effect="individual" and time fixed effects by hand
fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(year), data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "individual", spatial.error="b", Hess = FALSE)
summary(fespaterr)
##effect="time" and individual fixed effects by hand
fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(state), data = Produc, listw = mat2listw(usaww),
model="within",lag=T,effect = "time", spatial.error="b", Hess = FALSE)
summary(fespaterr)
Is this correct?
In this example the coefficients are just slightly different between the three specifications. In my case however, with T=2, these three specifications provide completely different results.
Thank you again for your help,
Marco
>
> Il 10 giugno 2019 alle 19.42 Roger Bivand < [hidden email]> ha scritto:
>
> On Mon, 10 Jun 2019, Marco Mello via RsigGeo wrote:
>
> > >
> > Dear community,
> >
> > >
> Please post plain text, not HTML, easier to copy and paste into an R
> session.
>
> >
>
> > >
> > I am working on a panel spatial model in R using the package splm. In
> > particular I am trying to estimate the following model
> >
> > spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> > = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
> > "twoways",zero.policy=TRUE)
> >
> > but I get the following error and warnings:
> >
> > Error in lag.listw(listw2, u) : Variable contains nonfinite values
> > In addition: Warning messages:
> > 1: In lag.listw(listw, u) : NAs in lagged values
> > 2: In lag.listw(listw, u) : NAs in lagged values
> > 3: In lag.listw(listw2, u) : NAs in lagged values
> > 4: In lag.listw(listw2, u) : NAs in lagged values
> >
> > >
> This is not a reproducible example. Such an example is needed unless you
> can run traceback() and debug() yourself to solve the problem, but since
> you have posted, I assume you prefer that someone else runs debug()  and
> someone will then need a reproducible example, preferably with an adapted
> standard dataset (add NAs to Produc?).
>
> I seem to recall that zero.policy= was not always passed through in some
> model fitting functions, possibly in splm. If so, set the option with
> spatialreg::set.ZeroPolicyOption(TRUE) and/or
> spdep::set.ZeroPolicyOption(TRUE)
> to avoid that issue if it is present and biting. Of course, had I had a
> reproducible example, I could have checked and given clearer advice.
>
> >
>
> > >
> > Similarly, if I model the spatial correlation only for the error term,
> >
> > spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
> > = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
> > "twoways",zero.policy=TRUE)
> >
> > I get the following warnings
> >
> > 1: In lag.listw(listw, u) : NAs in lagged values
> >
> > ...
> >
> > 3: In lag.listw(listw, TT) : NAs in lagged values
> >
> > ...
> >
> > 35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
> > NA/Inf replaced by maximum positive value
> >
> > ...
> >
> > In this case however the model turns out to be estimated, despite the
> > standard errors are everywhere NAs and the spatial rho coefficient equal
> > to 1 with weird t stats and pvalues.
> >
> > An identical problem to mine was already signalled in this mailing list,
> > unfortunately without receiving any suggestion :
> > http://rsiggeo.2731867.n2.nabble.com/problemsusingspmlwithalistwwherenoteverybodyhasaneighbourtd7587857.html.
> >
> > >
> Nabble is only an archive, the real link is:
>
> https://stat.ethz.ch/pipermail/rsiggeo/2015March/022428.html>
> The posting also gave no reproducible example, so nobody had anything to
> work on.
>
> Hope this helps,
>
> Roger
>
> >
>
> > >
> > I guess that the issue could be due to the n by n spatial weighting
> > matrix W30, which contains some neighbourless observations. However this
> > feature should be taken into account by the zero.policy option. Moreover
> > the same framework perfectly works in the crosssectional case, by using
> > the function sacsarlm from the package spdep.
> >
> > Please, does anybody know what is causing this and how can I solve this
> > issue? Any help would be really appreciated.
> >
> > Kind Regards,
> >
> > Marco Mello
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > RsigGeo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/rsiggeo> >
> > >
> 
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N5045 Bergen, Norway.
> voice: +47 55 95 93 55; email: [hidden email]
> https://orcid.org/0000000323926140> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en>
[[alternative HTML version deleted]]
_______________________________________________
RsigGeo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/rsiggeo

Administrator

On Tue, 11 Jun 2019, [hidden email] wrote:
> Dear Roger,
>
> many thanks for your help. Actually the issue was exactly the
> zero.policy option. I solved this by imposing
> spdep::set.ZeroPolicyOption(TRUE). Now the estimation works. This
> solution should apply to all weighting matrices having rows with all
> zeros, i.e. neighbourless observations.
>
OK, good, my memory isn't so bad after all ...
> I notice however that results are different between those obtained if we
> estimate the model with both individual and time fixed effects by
> setting effect="twoways" and those obtained by estimating the model with
> just one type of fixed effect (e.g. effect="individual") and adding the
> other ones by hand in the formula.
>
> I will add a reproducible example using the Produc dataset to show this
> issue.
>
Thanks for the example, I've CC'ed the splm maintainer to elicit more
insight.
Roger
>
>
>
>
> library(plm)
> library(spdep)
> library(splm)
>
> data(Produc, package = "plm")
> data(usaww)
>
> ##effect="twoways"
>
> fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp, data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "twoways", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
> ##effect="individual" and time fixed effects by hand
>
> fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(year), data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "individual", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
> ##effect="time" and individual fixed effects by hand
>
> fespaterr < spml(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp + factor(state), data = Produc, listw = mat2listw(usaww),
> model="within",lag=T,effect = "time", spatial.error="b", Hess = FALSE)
>
> summary(fespaterr)
>
>
>
>
>
> Is this correct?
>
> In this example the coefficients are just slightly different between the three specifications. In my case however, with T=2, these three specifications provide completely different results.
>
> Thank you again for your help,
>
> Marco
>
>
>
>>
>> Il 10 giugno 2019 alle 19.42 Roger Bivand < [hidden email]> ha scritto:
>>
>> On Mon, 10 Jun 2019, Marco Mello via RsigGeo wrote:
>>
>> >>
>>> Dear community,
>>>
>>> >
>> Please post plain text, not HTML, easier to copy and paste into an R
>> session.
>>
>> >
>>
>> >>
>>> I am working on a panel spatial model in R using the package splm. In
>>> particular I am trying to estimate the following model
>>>
>>> spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
>>> = na.fail, lag = TRUE, spatial.error = "b", model = "within", effect =
>>> "twoways",zero.policy=TRUE)
>>>
>>> but I get the following error and warnings:
>>>
>>> Error in lag.listw(listw2, u) : Variable contains nonfinite values
>>> In addition: Warning messages:
>>> 1: In lag.listw(listw, u) : NAs in lagged values
>>> 2: In lag.listw(listw, u) : NAs in lagged values
>>> 3: In lag.listw(listw2, u) : NAs in lagged values
>>> 4: In lag.listw(listw2, u) : NAs in lagged values
>>>
>>> >
>> This is not a reproducible example. Such an example is needed unless you
>> can run traceback() and debug() yourself to solve the problem, but since
>> you have posted, I assume you prefer that someone else runs debug()  and
>> someone will then need a reproducible example, preferably with an adapted
>> standard dataset (add NAs to Produc?).
>>
>> I seem to recall that zero.policy= was not always passed through in some
>> model fitting functions, possibly in splm. If so, set the option with
>> spatialreg::set.ZeroPolicyOption(TRUE) and/or
>> spdep::set.ZeroPolicyOption(TRUE)
>> to avoid that issue if it is present and biting. Of course, had I had a
>> reproducible example, I could have checked and given clearer advice.
>>
>> >
>>
>> >>
>>> Similarly, if I model the spatial correlation only for the error term,
>>>
>>> spatpan<spml(y ~ x, data = data_p, index = NULL, listw = W30, na.action
>>> = na.fail, lag = FALSE, spatial.error = "b", model = "within", effect =
>>> "twoways",zero.policy=TRUE)
>>>
>>> I get the following warnings
>>>
>>> 1: In lag.listw(listw, u) : NAs in lagged values
>>>
>>> ...
>>>
>>> 3: In lag.listw(listw, TT) : NAs in lagged values
>>>
>>> ...
>>>
>>> 35: In optimize(sarpanelerror, interval = interval, maximum = TRUE, ... :
>>> NA/Inf replaced by maximum positive value
>>>
>>> ...
>>>
>>> In this case however the model turns out to be estimated, despite the
>>> standard errors are everywhere NAs and the spatial rho coefficient equal
>>> to 1 with weird t stats and pvalues.
>>>
>>> An identical problem to mine was already signalled in this mailing list,
>>> unfortunately without receiving any suggestion :
>>> http://rsiggeo.2731867.n2.nabble.com/problemsusingspmlwithalistwwherenoteverybodyhasaneighbourtd7587857.html.
>>>
>>> >
>> Nabble is only an archive, the real link is:
>>
>> https://stat.ethz.ch/pipermail/rsiggeo/2015March/022428.html>>
>> The posting also gave no reproducible example, so nobody had anything to
>> work on.
>>
>> Hope this helps,
>>
>> Roger
>>
>> >
>>
>> >>
>>> I guess that the issue could be due to the n by n spatial weighting
>>> matrix W30, which contains some neighbourless observations. However this
>>> feature should be taken into account by the zero.policy option. Moreover
>>> the same framework perfectly works in the crosssectional case, by using
>>> the function sacsarlm from the package spdep.
>>>
>>> Please, does anybody know what is causing this and how can I solve this
>>> issue? Any help would be really appreciated.
>>>
>>> Kind Regards,
>>>
>>> Marco Mello
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> RsigGeo mailing list
>>> [hidden email]
>>> https://stat.ethz.ch/mailman/listinfo/rsiggeo>>>
>>> >
>> 
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N5045 Bergen, Norway.
>> voice: +47 55 95 93 55; email: [hidden email]
>> https://orcid.org/0000000323926140>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en>>
>

Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N5045 Bergen, Norway.
voice: +47 55 95 93 55; email: [hidden email]
https://orcid.org/0000000323926140https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en_______________________________________________
RsigGeo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/rsiggeo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N5045 Bergen, Norway

