modis download authentication

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

modis download authentication

luca candeloro
I used succesfully the function getBinaryURL from RCurl library to download
.hdf file, as in:

content <- getBinaryURL("
http://e4ftl01.cr.usgs.gov/MOLT/MOD11A2.005/2016.01.01/MOD11A2.A2016001.h18v04.005.2016012041335.hdf
")

Since http://e4ftl01.cr.usgs.gov/ has recently applied a login and pass
authentication to retrieve data, the previous code doesn't work.
It seems (from
https://lpdaac.usgs.gov/sites/default/files/public/get_data/docs/Command%20Line%20Access%20Tips%20for%20Utilizing%20Earthdata%20Login.docx
)
that curl options should be necessarily  set.
Does someone know how to correctly specify RCurl options in R, so that
getBinaryURL works again?
thanks,
Luca

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: modis download authentication

Ben Tupper
Hi,

RCurl's getBinaryURL() passes the '...' and '.opts' argument to RCurl's getURL().  The help page for getURL() shows an example that passes the username and password.

Will that do the trick?

Ben


> On Sep 8, 2016, at 7:16 AM, Luca Candeloro <[hidden email]> wrote:
>
> I used succesfully the function getBinaryURL from RCurl library to download
> .hdf file, as in:
>
> content <- getBinaryURL("
> http://e4ftl01.cr.usgs.gov/MOLT/MOD11A2.005/2016.01.01/MOD11A2.A2016001.h18v04.005.2016012041335.hdf
> ")
>
> Since http://e4ftl01.cr.usgs.gov/ has recently applied a login and pass
> authentication to retrieve data, the previous code doesn't work.
> It seems (from
> https://lpdaac.usgs.gov/sites/default/files/public/get_data/docs/Command%20Line%20Access%20Tips%20for%20Utilizing%20Earthdata%20Login.docx
> )
> that curl options should be necessarily  set.
> Does someone know how to correctly specify RCurl options in R, so that
> getBinaryURL works again?
> thanks,
> Luca
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Report Gulf of Maine jellyfish sightings to [hidden email] or tweet them to #MaineJellies -- include date, time, and location, as well as any descriptive information such as size or type.  Learn more at https://www.bigelow.org/research/srs/nick-record/nick-record-laboratory/mainejellies/

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Reply | Threaded
Open this post in threaded view
|

Re: modis download authentication

R-sig-geo mailing list
In reply to this post by luca candeloro
This is what worked for me, on a Mac:

myopts <- RCurl::curlOptions(netrc=TRUE, netrc.file=path.expand("~/.netrc"),
                             cookiefile=path.expand("~/.urs_cookies"),
                             followlocation=TRUE)


getBinaryURL(your_url, .opts=myopts)

where the .netrc file should look like this:

machine urs.earthdata.nasa.gov login your_login password your_pass
machine e4ftl01.cr.usgs.gov login your_login password your_pass

(note that both lines are required, and your_login and your_pass are your credentials to the EarthData system)

and the .urs_cookies is just an empty file where the access information will be saved to.

You can use a simple command that explicitly requires your credentials, but I don't think revealing your credentials on a R script is a good idea.

There are some helpful tips here as well:
http://disc.sci.gsfc.nasa.gov/registration/registration-for-data-access
 Greetings,
 -- Thiago V. dos Santos

PhD student
Land and Atmospheric Science
University of Minnesota



On Thursday, September 8, 2016 6:19 AM, Luca Candeloro <[hidden email]> wrote:
I used succesfully the function getBinaryURL from RCurl library to download
.hdf file, as in:

content <- getBinaryURL("
http://e4ftl01.cr.usgs.gov/MOLT/MOD11A2.005/2016.01.01/MOD11A2.A2016001.h18v04.005.2016012041335.hdf
")

Since http://e4ftl01.cr.usgs.gov/ has recently applied a login and pass
authentication to retrieve data, the previous code doesn't work.
It seems (from
https://lpdaac.usgs.gov/sites/default/files/public/get_data/docs/Command%20Line%20Access%20Tips%20for%20Utilizing%20Earthdata%20Login.docx
)
that curl options should be necessarily  set.
Does someone know how to correctly specify RCurl options in R, so that
getBinaryURL works again?
thanks,
Luca

    [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo