Southern Ocean State Estimate

[mdsumner]

Direct links! So easy:

http://sose.ucsd.edu/

tester in PRIVATE dev/sose.ucsd.edu/SO6/ITER122

[mdsumner]

Got this with wget (only want ITER122 and montly for now).

##Example file: http://sose.ucsd.edu/SO6/ITER122/bsose_i122_2013to2017_monthly_MLD.nc
library(bowerbird)

sose <- bb_source(
  name="Biogeochemical Southern Ocean State Estimate solution (Iteration 122)",
  id="122, doi:10.1002/2016JC012650",
  description="B-SOSE is a contribution of the Southern Ocean Carbon and Climate Observations and Modeling project (SOCCOM) program. SOCCOM is an NSF-sponsored program focused on unlocking the mysteries of the Southern Ocean and determining its influence on climate.",
  doc_url="http://sose.ucsd.edu/BSOSE6_iter122_solution.html/",
  citation="A. Verdy and M. Mazloff, 2017, A data assimilating model for estimating Southern Ocean biogeochemistry. J. Geophys. Res. Oceans.,  122, doi:10.1002/2016JC012650.",
  source_url = "http://sose.ucsd.edu/BSOSE6_iter122_solution.html",
  license="Please cite",
  method = list("bb_handler_wget", accept_regex = "monthly.*\\.nc$", robots_off = TRUE),
  collection_size=341,
  comment="",
  data_group="Oceanography")

root <- "<secrets>/data"

cf <- bb_config(local_file_root= root )
cf <- cf %>% bb_add(sose)
sync_status <- bb_sync(cf,  verbose = TRUE)

Couldn't get it to work with rget ...

[raymondben]

Just fyi for rget:

sose <- bb_source(
  ...
  method = list("bb_handler_rget", accept_download = "monthly.*\\.nc$", level = 1),
  ...
)

[mdsumner]

cool thanks will try that. There are some broken links on the ITER122 page, particularly

http://sose.ucsd.edu/SO6/ITER122/bsose_i122_2013to2017_monthly_Phyto.nc

http://sose.ucsd.edu/SO6/ITER122/bsose_i122_2013to2017_5day_Phyto.nc

I've emailed them about it. All other monthly files worked with the config above.

[mdsumner]

Looks good with rget: (BEWARE this is 341 Gb)

##Example file: http://sose.ucsd.edu/SO6/ITER122/bsose_i122_2013to2017_monthly_MLD.nc
library(bowerbird)

sose <- bb_source(
  name="Biogeochemical Southern Ocean State Estimate solution (Iteration 122)",
  id="122, doi:10.1002/2016JC012650",
  description="B-SOSE is a contribution of the Southern Ocean Carbon and Climate Observations and Modeling project (SOCCOM) program. SOCCOM is an NSF-sponsored program focused on unlocking the mysteries of the Southern Ocean and determining its influence on climate.",
  doc_url="http://sose.ucsd.edu/BSOSE6_iter122_solution.html/",
  citation="A. Verdy and M. Mazloff, 2017, A data assimilating model for estimating Southern Ocean biogeochemistry. J. Geophys. Res. Oceans.,  122, doi:10.1002/2016JC012650.",
  source_url = "http://sose.ucsd.edu/BSOSE6_iter122_solution.html",
  license="Please cite",
  method = list("bb_handler_rget", accept_download = "monthly.*\\.nc$", level = 1),
  collection_size=341,
  comment="",
  data_group="Oceanography")

root <- "<secrets>/data"

cf <- bb_config(local_file_root= root )
cf <- cf %>% bb_add(sose)
sync_status <- bb_sync(cf,  verbose = TRUE)

[mdsumner]

Uses Mercator, here's how to treat it natively

f <- "//rdsi/PUBLIC/raad/data/sose.ucsd.edu/SO6/ITER122/bsose_i122_2013to2017_monthly_Chl.nc"
library(raster)
br <- raster::brick(f, stopIfNotEqualSpaced = FALSE, lvar = 4)  ## x, y, depth

## trust me, see below
extent(br) <- extent(c(0, 2 * 20037508 , -14317550, -3445832))
merc <- "+proj=merc +datum=WGS84"
## let's think in longlat
cropper <- projectExtent(raster::raster(extent(120, 180, -60, -30), crs = "+proj=longlat +datum=WGS84"), 
                         merc)

plot(crop(readAll(br[[1]]), cropper))

data("wrld_simpl", package = "maptools")
w <- sp::spTransform(subset(wrld_simpl, NAME %in% c("New Zealand", "Australia")), merc)
plot(w, add = TRUE)

# ncmeta::nc_dims(f)
# # A tibble: 4 x 4
# id name  length unlim
# <int> <chr>  <dbl> <lgl>
#   1     0 time      60 FALSE
# 2     1 XC      2160 FALSE
# 3     2 YC       588 FALSE
# 4     3 Z         52 FALSE
#range(RNetCDF::var.get.nc(RNetCDF::open.nc(f), "YC"))
## -77.98265 -29.71532
# amin <- reproj::reproj(cbind(0, c(-77.98265, -29.71532)), source = "+proj=longlat +datum=WGS84", target = "+proj=merc")
# amax <- reproj::reproj(cbind(180, c(-77.98265, -29.71532)), source = "+proj=longlat +datum=WGS84", target = "+proj=merc")
# 
# extent(br) <- extent(amin[1, 1], amax[1, 1] * 2, 
#                      amin[1, 2], amin[2, 2])

[mdsumner]

We have monthly files for ITER122 and ITER106 here, each is ~300Gb:

data/sose.ucsd.edu/SO6/ITER122
data/sose.ucsd.edu/SO6/ITER106

So, we could

• remove ITER106 and get ITER133 (created February 2020).
• consider remove ITER122 (but for now I'll write code against that)
• remove tester in PRIVATE dev/sose.ucsd.edu/SO6/ITER122
• find out what higher temporal resolution is of interest

More examples, crux is that the data is stored with rectilinear latitude (it's a Mercator projection) so we subvert raster a bit to make that work:

https://gist.github.com/mdsumner/e0e81b46830f5525fc1a01707a762af5