Error in basename(x) : path too long

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Error in basename(x) : path too long

John Wasige
​Dear all,

I am running the script below & I get the following error:
Error in basename(x) : path too long

What could be the problem?
Thanks for your help
John

​### Script​

setwd("I:/Mauritius2001_2015")  # directory of data
newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)

refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
refr[!is.na(refr)] <- 0
for(i in seq(1,345,by=23)){
  rsum <- refr
  for(p in 0:22){
    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
    rsum <- rsum + r
  }
  #   rsum <- rsum
  writeRaster(rsum,
filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep=''),
format="GTiff", overwrite=TRUE)
}

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Error in basename(x) : path too long

Ben Tupper
Hi,

It's not possible to know but I have a suspicion and a suggestion.

suspicion:  You are accessing the elements of the data frame "newlist" different ways ...

> refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))

and

> r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))

and

> filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep='')


The first likely works.  The latter two return the (i+p)'th and i'th columns each as a new data frame so the filenames may indeed be quite long.  So, it looks like you need to work on your data frame indexing.

suggestion:  Instead of using paste() to make file paths I suggest that you give file.path() a try.  It works across all platforms and is easier than using paste.

CHeers,
Ben

> On Jul 29, 2017, at 7:56 AM, John Wasige <[hidden email]> wrote:
>
> ​Dear all,
>
> I am running the script below & I get the following error:
> Error in basename(x) : path too long
> ​
> What could be the problem?
> Thanks for your help
> John
>
> ​### Script​
>
> setwd("I:/Mauritius2001_2015")  # directory of data
> newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)
>
> refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
> refr[!is.na(refr)] <- 0
> for(i in seq(1,345,by=23)){
>  rsum <- refr
>  for(p in 0:22){
>    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
>    rsum <- rsum + r
>  }
>  #   rsum <- rsum
>  writeRaster(rsum,
> filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep=''),
> format="GTiff", overwrite=TRUE)
> }
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Ecocast Reports: http://seascapemodeling.org/ecocast.html
Tick Reports: https://report.bigelow.org/tick/
Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Error in basename(x) : path too long

Ben Tupper
In reply to this post by John Wasige
Hi again,

A late thought - I'm still on the first cups of coffee.

It looks to me like you are iterating over a stack to select certain layers to sum.  You could achieve the same outcome with possibly much less work.  The following example will create a sum of 24-layer blocks along a stack of rasters.

# from https://gist.github.com/btupper/20879e0b46e5ed63d402d7cff424dbb7
#' Split a vector into groups of MAX (or possibly fewer)
#'
#' @param v vector or list to split
#' @param MAX numeric the maximum size per group
#' @return a list of the vector split into groups
split_vector <- function(v, MAX = 200){
    nv <- length(v)
    if (nv <= MAX) return(list('1' = v))
    split(v, findInterval(1:nv, seq(from = 1, to = nv, by = MAX)))
}


library(raster)

N <- 345
n <- 24
nc <- 4
nr <- 3

R <- raster(matrix(runif(nc*nr), ncol = nc, nrow = nr))

RR <- stack(lapply(seq_len(N), function(i) R))

ix <- split_vector(seq_len(N), MAX = n)

SS <- lapply(ix, function(index) sum(RR[[index]]))

So, S[[1]], which looks like this...

$`1`
class       : RasterLayer
dimensions  : 3, 4, 12  (nrow, ncol, ncell)
resolution  : 0.25, 0.3333333  (x, y)
extent      : 0, 1, 0, 1  (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : in memory
names       : layer
values      : 0.9451534, 20.0503  (min, max)

... is the sum of the first 24 layers of RR. SS[[2]] will be the sum of the next 24, and so on.

Is that what you are trying to do?

Cheers,
Ben




> On Jul 29, 2017, at 7:56 AM, John Wasige <[hidden email]> wrote:
>
> ​Dear all,
>
> I am running the script below & I get the following error:
> Error in basename(x) : path too long
> ​
> What could be the problem?
> Thanks for your help
> John
>
> ​### Script​
>
> setwd("I:/Mauritius2001_2015")  # directory of data
> newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)
>
> refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
> refr[!is.na(refr)] <- 0
> for(i in seq(1,345,by=23)){
>  rsum <- refr
>  for(p in 0:22){
>    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
>    rsum <- rsum + r
>  }
>  #   rsum <- rsum
>  writeRaster(rsum,
> filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep=''),
> format="GTiff", overwrite=TRUE)
> }
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Ecocast Reports: http://seascapemodeling.org/ecocast.html
Tick Reports: https://report.bigelow.org/tick/
Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Error in basename(x) : path too long

John Wasige
Thank you Ben,
I would like to generate annual sums & means for each year from a list
rasters (not stack or brick because of memory issues with R). For 15 years
I have 345 individual rasters and frequency of 23 per year.

I will be very glad for an idea on how to do that.

Thanks

John





On Sat, Jul 29, 2017 at 5:37 PM, Ben Tupper <[hidden email]> wrote:

> Hi again,
>
> A late thought - I'm still on the first cups of coffee.
>
> It looks to me like you are iterating over a stack to select certain
> layers to sum.  You could achieve the same outcome with possibly much less
> work.  The following example will create a sum of 24-layer blocks along a
> stack of rasters.
>
> # from https://gist.github.com/btupper/20879e0b46e5ed63d402d7cff424dbb7
> #' Split a vector into groups of MAX (or possibly fewer)
> #'
> #' @param v vector or list to split
> #' @param MAX numeric the maximum size per group
> #' @return a list of the vector split into groups
> split_vector <- function(v, MAX = 200){
>     nv <- length(v)
>     if (nv <= MAX) return(list('1' = v))
>     split(v, findInterval(1:nv, seq(from = 1, to = nv, by = MAX)))
> }
>
>
> library(raster)
>
> N <- 345
> n <- 24
> nc <- 4
> nr <- 3
>
> R <- raster(matrix(runif(nc*nr), ncol = nc, nrow = nr))
>
> RR <- stack(lapply(seq_len(N), function(i) R))
>
> ix <- split_vector(seq_len(N), MAX = n)
>
> SS <- lapply(ix, function(index) sum(RR[[index]]))
>
> So, S[[1]], which looks like this...
>
> $`1`
> class       : RasterLayer
> dimensions  : 3, 4, 12  (nrow, ncol, ncell)
> resolution  : 0.25, 0.3333333  (x, y)
> extent      : 0, 1, 0, 1  (xmin, xmax, ymin, ymax)
> coord. ref. : NA
> data source : in memory
> names       : layer
> values      : 0.9451534, 20.0503  (min, max)
>
> ... is the sum of the first 24 layers of RR. SS[[2]] will be the sum of
> the next 24, and so on.
>
> Is that what you are trying to do?
>
> Cheers,
> Ben
>
>
>
>
> > On Jul 29, 2017, at 7:56 AM, John Wasige <[hidden email]> wrote:
> >
> > ​Dear all,
> >
> > I am running the script below & I get the following error:
> > Error in basename(x) : path too long
> > ​
> > What could be the problem?
> > Thanks for your help
> > John
> >
> > ​### Script​
> >
> > setwd("I:/Mauritius2001_2015")  # directory of data
> > newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)
> >
> > refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
> > refr[!is.na(refr)] <- 0
> > for(i in seq(1,345,by=23)){
> >  rsum <- refr
> >  for(p in 0:22){
> >    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
> >    rsum <- rsum + r
> >  }
> >  #   rsum <- rsum
> >  writeRaster(rsum,
> > filename=paste("D:/Mauritius2001_2015/Annual/",
> substr(newlist[i],1,6),".tif",sep=''),
> > format="GTiff", overwrite=TRUE)
> > }
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> Ben Tupper
> Bigelow Laboratory for Ocean Sciences
> 60 Bigelow Drive, P.O. Box 380
> East Boothbay, Maine 04544
> http://www.bigelow.org
>
> Ecocast Reports: http://seascapemodeling.org/ecocast.html
> Tick Reports: https://report.bigelow.org/tick/
> Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/
>
>
>
>


--
John Wasige
"There are no REGRATES in LIFE, just lessons (Jennifer Aniston)”

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Error in basename(x) : path too long

Ben Tupper
Hi John,

Using a raster::stack() should the perfect solution for you; I work with very large raster stacks all of the time.  From the introductory vignette https://cran.r-project.org/web/packages/raster/vignettes/Raster.pdf

"A notable feature of the raster package is that it can work with raster
datasets that are stored on disk and are too large to be loaded into memory
(RAM). The package can work with large files because the objects it creates
from these files only contain information about the structure of the data, such
as the number of rows and columns, the spatial extent, and the filename, but it
does not attempt to read all the cell values in memory."

Have you tried something like...

library(raster)
S <- stack(vector_of_filenames)

Ben




> On Jul 31, 2017, at 5:19 AM, John Wasige <[hidden email]> wrote:
>
> Thank you Ben,
> I would like to generate annual sums & means for each year from a list rasters (not stack or brick because of memory issues with R). For 15 years I have 345 individual rasters and frequency of 23 per year.
>
> I will be very glad for an idea on how to do that.
>
> Thanks
>
> John
>
>
>
>
>
> On Sat, Jul 29, 2017 at 5:37 PM, Ben Tupper <[hidden email]> wrote:
> Hi again,
>
> A late thought - I'm still on the first cups of coffee.
>
> It looks to me like you are iterating over a stack to select certain layers to sum.  You could achieve the same outcome with possibly much less work.  The following example will create a sum of 24-layer blocks along a stack of rasters.
>
> # from https://gist.github.com/btupper/20879e0b46e5ed63d402d7cff424dbb7
> #' Split a vector into groups of MAX (or possibly fewer)
> #'
> #' @param v vector or list to split
> #' @param MAX numeric the maximum size per group
> #' @return a list of the vector split into groups
> split_vector <- function(v, MAX = 200){
>     nv <- length(v)
>     if (nv <= MAX) return(list('1' = v))
>     split(v, findInterval(1:nv, seq(from = 1, to = nv, by = MAX)))
> }
>
>
> library(raster)
>
> N <- 345
> n <- 24
> nc <- 4
> nr <- 3
>
> R <- raster(matrix(runif(nc*nr), ncol = nc, nrow = nr))
>
> RR <- stack(lapply(seq_len(N), function(i) R))
>
> ix <- split_vector(seq_len(N), MAX = n)
>
> SS <- lapply(ix, function(index) sum(RR[[index]]))
>
> So, S[[1]], which looks like this...
>
> $`1`
> class       : RasterLayer
> dimensions  : 3, 4, 12  (nrow, ncol, ncell)
> resolution  : 0.25, 0.3333333  (x, y)
> extent      : 0, 1, 0, 1  (xmin, xmax, ymin, ymax)
> coord. ref. : NA
> data source : in memory
> names       : layer
> values      : 0.9451534, 20.0503  (min, max)
>
> ... is the sum of the first 24 layers of RR. SS[[2]] will be the sum of the next 24, and so on.
>
> Is that what you are trying to do?
>
> Cheers,
> Ben
>
>
>
>
> > On Jul 29, 2017, at 7:56 AM, John Wasige <[hidden email]> wrote:
> >
> > ​Dear all,
> >
> > I am running the script below & I get the following error:
> > Error in basename(x) : path too long
> > ​
> > What could be the problem?
> > Thanks for your help
> > John
> >
> > ​### Script​
> >
> > setwd("I:/Mauritius2001_2015")  # directory of data
> > newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)
> >
> > refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
> > refr[!is.na(refr)] <- 0
> > for(i in seq(1,345,by=23)){
> >  rsum <- refr
> >  for(p in 0:22){
> >    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
> >    rsum <- rsum + r
> >  }
> >  #   rsum <- rsum
> >  writeRaster(rsum,
> > filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep=''),
> > format="GTiff", overwrite=TRUE)
> > }
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> Ben Tupper
> Bigelow Laboratory for Ocean Sciences
> 60 Bigelow Drive, P.O. Box 380
> East Boothbay, Maine 04544
> http://www.bigelow.org
>
> Ecocast Reports: http://seascapemodeling.org/ecocast.html
> Tick Reports: https://report.bigelow.org/tick/
> Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/
>
>
>
>
>
>
> --
> John Wasige
> "There are no REGRATES in LIFE, just lessons (Jennifer Aniston)”

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Ecocast Reports: http://seascapemodeling.org/ecocast.html
Tick Reports: https://report.bigelow.org/tick/
Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Loading...