Spatial data when the missing mechanism is MNAR (non-ignorable)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Spatial data when the missing mechanism is MNAR (non-ignorable)

AMITHA PURANIK
Is it possible to impute missing values in spatial data when the
missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
selection model be modified to incorporate autocorrelation property and
used in this context?
Any suggestion/opinion is appreciated.

Thanks in advance,
Amitha Puranik

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Spatial data when the missing mechanism is MNAR (non-ignorable)

Roger Bivand
Administrator
On Sun, 4 Oct 2020, Amitha Puranik wrote:

> Is it possible to impute missing values in spatial data when the
> missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
> selection model be modified to incorporate autocorrelation property and
> used in this context?
> Any suggestion/opinion is appreciated.

MNAR possibly means "missing not at random". I see
https://doi.org/10.1186/1476-072X-14-1 for point support data using INLA.
For lattice data, see perhaps https://doi.org/10.1007/s10109-019-00316-z 
and work by Thomas Suesse https://doi.org/10.1016/j.csda.2017.11.004 
https://doi.org/10.1080/00949655.2017.1286495. This might be relevant:
https://doi.org/10.1016/j.epsr.2020.106640, but be extremely careful of
imputation in training/test settings with spatial data, as the spatial (or
temporal or both) lead to information leaking between the training and
test data because they are no longer independent.

Hope this helps,

Roger

>
> Thanks in advance,
> Amitha Puranik
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Spatial data when the missing mechanism is MNAR (non-ignorable)

AMITHA PURANIK
Dear Roger,

Thank you for the quick response. I shall refer to the articles that
you recommended.
Thanks again!

Regards,
Amitha Puranik.

On Mon, Oct 5, 2020 at 1:48 PM Roger Bivand <[hidden email]> wrote:

> On Sun, 4 Oct 2020, Amitha Puranik wrote:
>
> > Is it possible to impute missing values in spatial data when the
> > missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
> > selection model be modified to incorporate autocorrelation property and
> > used in this context?
> > Any suggestion/opinion is appreciated.
>
> MNAR possibly means "missing not at random". I see
> https://doi.org/10.1186/1476-072X-14-1 for point support data using INLA.
> For lattice data, see perhaps https://doi.org/10.1007/s10109-019-00316-z
> and work by Thomas Suesse https://doi.org/10.1016/j.csda.2017.11.004
> https://doi.org/10.1080/00949655.2017.1286495. This might be relevant:
> https://doi.org/10.1016/j.epsr.2020.106640, but be extremely careful of
> imputation in training/test settings with spatial data, as the spatial (or
> temporal or both) lead to information leaking between the training and
> test data because they are no longer independent.
>
> Hope this helps,
>
> Roger
>
> >
> > Thanks in advance,
> > Amitha Puranik
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Spatial data when the missing mechanism is MNAR (non-ignorable)

AMITHA PURANIK
Dear Prof Roger,



This is in continuation to my previous query on spatial data imputation
with MNAR mechanism. I have gone through the references recommended by you
and have the following concerns which I request you to address.

1. The papers by Thomas Suesse, Takafumi Kato suggest likelihood based
approaches for predicting missing data in simultaneous autoregressive
models with an assumption of *missing at random* mechanism.

2. The additional reference provided by you, i.e. 'Missing Data in Wind
Farm Time Series: Properties and Effect on Forecasts' by Tawn et al.,
assume *missing not at random* mechanism in an autoregressive framework and
have applied mean imputation and multiple imputation methods.



I am presently looking for a technique to deal with MNAR in spatially
autocorrelated data. Would it be reasonable to apply the methods
recommended by Suesse or Kato in this scenario by ignoring the missing
mechanism?

From what I understand, using conventional methods that are effective for
MAR case would produce biased estimates when data is MNAR. Can the approach
applied by Tawn et al. (i.e. mean imputation or multiple imputation) be
used on spatial data with MNAR mechanism?



Any comment/ suggestion will be appreciated.


Thanks in advance,

Amitha Puranik.












On Mon, Oct 5, 2020 at 2:13 PM Amitha Puranik <[hidden email]>
wrote:

> Dear Roger,
>
> Thank you for the quick response. I shall refer to the articles that you recommended.
> Thanks again!
>
> Regards,
> Amitha Puranik.
>
> On Mon, Oct 5, 2020 at 1:48 PM Roger Bivand <[hidden email]> wrote:
>
>> On Sun, 4 Oct 2020, Amitha Puranik wrote:
>>
>> > Is it possible to impute missing values in spatial data when the
>> > missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
>> > selection model be modified to incorporate autocorrelation property and
>> > used in this context?
>> > Any suggestion/opinion is appreciated.
>>
>> MNAR possibly means "missing not at random". I see
>> https://doi.org/10.1186/1476-072X-14-1 for point support data using
>> INLA.
>> For lattice data, see perhaps https://doi.org/10.1007/s10109-019-00316-z
>> and work by Thomas Suesse https://doi.org/10.1016/j.csda.2017.11.004
>> https://doi.org/10.1080/00949655.2017.1286495. This might be relevant:
>> https://doi.org/10.1016/j.epsr.2020.106640, but be extremely careful of
>> imputation in training/test settings with spatial data, as the spatial
>> (or
>> temporal or both) lead to information leaking between the training and
>> test data because they are no longer independent.
>>
>> Hope this helps,
>>
>> Roger
>>
>> >
>> > Thanks in advance,
>> > Amitha Puranik
>> >
>> >       [[alternative HTML version deleted]]
>> >
>> > _______________________________________________
>> > R-sig-Geo mailing list
>> > [hidden email]
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>> >
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: Spatial data when the missing mechanism is MNAR (non-ignorable)

Roger Bivand
Administrator
On Tue, 6 Oct 2020, Amitha Puranik wrote:

> Dear Prof Roger,
>
> This is in continuation to my previous query on spatial data imputation
> with MNAR mechanism. I have gone through the references recommended by you
> and have the following concerns which I request you to address.
>
> 1. The papers by Thomas Suesse, Takafumi Kato suggest likelihood based
> approaches for predicting missing data in simultaneous autoregressive
> models with an assumption of *missing at random* mechanism.
>
> 2. The additional reference provided by you, i.e. 'Missing Data in Wind
> Farm Time Series: Properties and Effect on Forecasts' by Tawn et al.,
> assume *missing not at random* mechanism in an autoregressive framework and
> have applied mean imputation and multiple imputation methods.
>
>
>
> I am presently looking for a technique to deal with MNAR in spatially
> autocorrelated data. Would it be reasonable to apply the methods
> recommended by Suesse or Kato in this scenario by ignoring the missing
> mechanism?
>
> From what I understand, using conventional methods that are effective for
> MAR case would produce biased estimates when data is MNAR. Can the approach
> applied by Tawn et al. (i.e. mean imputation or multiple imputation) be
> used on spatial data with MNAR mechanism?
>

Since probably nobody knows, you may very well need to run simulations in
the settings you need to determine which outcomes correspond to reasonable
practice. Reviewers of you work would probably appreciate your having
explored the robustness of your choice of method. Quite a lot will depend
on the spatial support of your data too.

Roger

>
>
> Any comment/ suggestion will be appreciated.
>
>
> Thanks in advance,
>
> Amitha Puranik.
>
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Oct 5, 2020 at 2:13 PM Amitha Puranik <[hidden email]>
> wrote:
>
>> Dear Roger,
>>
>> Thank you for the quick response. I shall refer to the articles that you recommended.
>> Thanks again!
>>
>> Regards,
>> Amitha Puranik.
>>
>> On Mon, Oct 5, 2020 at 1:48 PM Roger Bivand <[hidden email]> wrote:
>>
>>> On Sun, 4 Oct 2020, Amitha Puranik wrote:
>>>
>>>> Is it possible to impute missing values in spatial data when the
>>>> missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
>>>> selection model be modified to incorporate autocorrelation property and
>>>> used in this context?
>>>> Any suggestion/opinion is appreciated.
>>>
>>> MNAR possibly means "missing not at random". I see
>>> https://doi.org/10.1186/1476-072X-14-1 for point support data using
>>> INLA.
>>> For lattice data, see perhaps https://doi.org/10.1007/s10109-019-00316-z
>>> and work by Thomas Suesse https://doi.org/10.1016/j.csda.2017.11.004
>>> https://doi.org/10.1080/00949655.2017.1286495. This might be relevant:
>>> https://doi.org/10.1016/j.epsr.2020.106640, but be extremely careful of
>>> imputation in training/test settings with spatial data, as the spatial
>>> (or
>>> temporal or both) lead to information leaking between the training and
>>> test data because they are no longer independent.
>>>
>>> Hope this helps,
>>>
>>> Roger
>>>
>>>>
>>>> Thanks in advance,
>>>> Amitha Puranik
>>>>
>>>>       [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> [hidden email]
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Spatial data when the missing mechanism is MNAR (non-ignorable)

AMITHA PURANIK
Dear Prof. Roger,

Thanks for that suggestion. I will go for a simulation approach then to
identify the appropriate method. Thanks again for your help.

Regards,
Amitha Puranik.




On Tue, 6 Oct, 2020, 6:53 PM Roger Bivand, <[hidden email]> wrote:

> On Tue, 6 Oct 2020, Amitha Puranik wrote:
>
> > Dear Prof Roger,
> >
> > This is in continuation to my previous query on spatial data imputation
> > with MNAR mechanism. I have gone through the references recommended by
> you
> > and have the following concerns which I request you to address.
> >
> > 1. The papers by Thomas Suesse, Takafumi Kato suggest likelihood based
> > approaches for predicting missing data in simultaneous autoregressive
> > models with an assumption of *missing at random* mechanism.
> >
> > 2. The additional reference provided by you, i.e. 'Missing Data in Wind
> > Farm Time Series: Properties and Effect on Forecasts' by Tawn et al.,
> > assume *missing not at random* mechanism in an autoregressive framework
> and
> > have applied mean imputation and multiple imputation methods.
> >
> >
> >
> > I am presently looking for a technique to deal with MNAR in spatially
> > autocorrelated data. Would it be reasonable to apply the methods
> > recommended by Suesse or Kato in this scenario by ignoring the missing
> > mechanism?
> >
> > From what I understand, using conventional methods that are effective for
> > MAR case would produce biased estimates when data is MNAR. Can the
> approach
> > applied by Tawn et al. (i.e. mean imputation or multiple imputation) be
> > used on spatial data with MNAR mechanism?
> >
>
> Since probably nobody knows, you may very well need to run simulations in
> the settings you need to determine which outcomes correspond to reasonable
> practice. Reviewers of you work would probably appreciate your having
> explored the robustness of your choice of method. Quite a lot will depend
> on the spatial support of your data too.
>
> Roger
>
> >
> >
> > Any comment/ suggestion will be appreciated.
> >
> >
> > Thanks in advance,
> >
> > Amitha Puranik.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 5, 2020 at 2:13 PM Amitha Puranik <[hidden email]>
> > wrote:
> >
> >> Dear Roger,
> >>
> >> Thank you for the quick response. I shall refer to the articles that
> you recommended.
> >> Thanks again!
> >>
> >> Regards,
> >> Amitha Puranik.
> >>
> >> On Mon, Oct 5, 2020 at 1:48 PM Roger Bivand <[hidden email]>
> wrote:
> >>
> >>> On Sun, 4 Oct 2020, Amitha Puranik wrote:
> >>>
> >>>> Is it possible to impute missing values in spatial data when the
> >>>> missingness is *MNAR (non-ignorable)*? Can pattern mixture model or
> >>>> selection model be modified to incorporate autocorrelation property
> and
> >>>> used in this context?
> >>>> Any suggestion/opinion is appreciated.
> >>>
> >>> MNAR possibly means "missing not at random". I see
> >>> https://doi.org/10.1186/1476-072X-14-1 for point support data using
> >>> INLA.
> >>> For lattice data, see perhaps
> https://doi.org/10.1007/s10109-019-00316-z
> >>> and work by Thomas Suesse https://doi.org/10.1016/j.csda.2017.11.004
> >>> https://doi.org/10.1080/00949655.2017.1286495. This might be relevant:
> >>> https://doi.org/10.1016/j.epsr.2020.106640, but be extremely careful
> of
> >>> imputation in training/test settings with spatial data, as the spatial
> >>> (or
> >>> temporal or both) lead to information leaking between the training and
> >>> test data because they are no longer independent.
> >>>
> >>> Hope this helps,
> >>>
> >>> Roger
> >>>
> >>>>
> >>>> Thanks in advance,
> >>>> Amitha Puranik
> >>>>
> >>>>       [[alternative HTML version deleted]]
> >>>>
> >>>> _______________________________________________
> >>>> R-sig-Geo mailing list
> >>>> [hidden email]
> >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>
> >>>
> >>> --
> >>> Roger Bivand
> >>> Department of Economics, Norwegian School of Economics,
> >>> Helleveien 30, N-5045 Bergen, Norway.
> >>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>> https://orcid.org/0000-0003-2392-6140
> >>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo