init Archiver module

[Polkas]

closes #81

Simply run the Previewer app from the vignette and test it yourself.

TODO

• test with different bootstrap versions (3,4, and 5)
• input validation
• separate shiny app - a new vignette
• load back, a way to validate the archiver zip
• UI design
• docs
• test the module, testServer
• Should we record the Archiver creator?
• ...

teal_report_archiver_demo.mov

[lcd2yyz]

Pretty cool feature, very nice work!

Question: When the JSON file is downloaded, what's the actual content? Is it just the code/setting or the actual visualization? Asking because what if the input datasets changed between the download/upload?
Comment 1: This feature make more sense to live in the Previewer tab, as it's restoring the contents to be shown in the Previewer.
Comment 2: It's not clear from just looking at the name/label "Archiver" what the functionality actually do. Perhaps should consider a more self-explanatory naming or adding description label to indicate it's saving/restoring the analysis cards?

[Polkas]

TL;DR; There are many possible improvements to this feature.

0. Question. That is a good comment. We already talked about the validation of datasets. Without additional protection, we will risk using different data in the same report (END USER responsibility). The idea of a dataset hash for each report card was proposed, which is automatic but adds complexity. Then Reporter (not card as then will be added once) (singleton instance - unique one for the whole session) requires an additional method like set_hash which will be optionally specified by the app developer when the reporter instance is created (the main unique element shared across the session). This hash have to be transferred to each Card when saved.
   I have an even more straightforward idea like Archiver ID_ARCHIVER (server function argument) set by the shiny app developer, and later each archiver zip file name will have this ID_ARCHIVER. It is simple but not perfect, as the responsibility is entirely on the developer of the shiny app (update the ID when data in the app changed). This functionality should be optional for developer UX when the END USER responsibility is enough for the assumptions. PS. the downloaded file is the zip file with one JSON file (Reporter representation) and static files for images and tables.
1. This is for sure to be done. I already added it to the list too. However, for TEAL I recommend the save button on the direct level, as the save functionality should be at hand for the end users (We can think of a notification each 20 minutes to save the Archive). We can have four buttons on the level of each TEAL module, so then we will add only the save button to the three that already exist and change the title. In the general case, the functionality is modular so that somebody can build any setup.
2. @insightsengineering/nest-core-dev, your naming ideas are wanted here. I appreciate your help. In the past, we name it Storager/Storage; currently, it is Archiver/Archive.

[Polkas]

After some talks with @mhallal1 I want to leave a small comment.
@pawelru and the team decided we want teal.reporter tool to be as general as possible, and this way it was designed.
The priority is to make it working for any shiny app, NOT a teal one.
Because of its generality later can be easily implemented in the teal app.
This generality profits in bigger target group as can be used by the whole shiny community.
More than that code is more clear and easier to maintain, because of this generality.
@lcd2yyz for sure can change this strategy.

[Polkas]

format name ?
do we need anything more than JSON?

[Polkas]

I remove the archiver class after our discussions. We think the YAML implementation where we use yaml file instead of json is not needed. Moreover we see a little probability of implementing Data Base (e.g. remote MySQL) solution with login and password so end users do not have to manage the zip files. @nikolas-burkoff pointed a security issues for such too.

Main difference in the code is only in 2 functions.

Without Archiver:

#' @keywords internal
load_json_archiver <- function(reporter, zip_path, filename) {
  tmp_dir <- tempdir()
  output_dir <- file.path(tmp_dir, sprintf("archiver_load_%s", gsub("[.]", "", format(Sys.time(), "%Y%m%d%H%M%OS4"))))
  dir.create(path = output_dir)
  if (!is.null(zip_path) && grepl("archiver_", filename)) {
    tryCatch(
      expr = zip::unzip(zip_path, exdir = output_dir, junkpaths = TRUE),
 ...
    )
    reporter$from_jsondir(output_dir)
  } else {
    shiny::showNotification("Failed to load the archiver file.", type = "error")
  }
}

#' @keywords internal
archiver_download_handler_engine <- function(reporter, type, file) {
  switch(type,
    JSON = {
      tmp_dir <- tempdir()
      output_dir <- file.path(
        tmp_dir,
        sprintf("archiver_%s", gsub("[.]", "", format(Sys.time(), "%Y%m%d%H%M%OS4")))
      )
      dir.create(path = output_dir)
      archiver_dir <- reporter$to_jsondir(output_dir)
      temp_zip_file <- tempfile(fileext = ".zip")
      tryCatch(
        expr = zip::zipr(temp_zip_file, archiver_dir),
       ...
      )

      tryCatch(
        expr = file.copy(temp_zip_file, file),
        ...
      )
    }
  )
}

With Archiver:

#' @keywords internal
load_json_archiver <- function(reporter, zip_path, filename) {
  archiver <- JSONArchiver$new()
  tmp_dir <- tempdir()
  output_dir <- file.path(tmp_dir, sprintf("archive_%s", gsub("[.]", "", format(Sys.time(), "%Y%m%d%H%M%OS4"))))
  dir.create(output_dir)
  tryCatch(
    expr = zip::unzip(zip_path, exdir = output_dir, junkpaths = TRUE),
...
  )
  reporter_new <- archiver$read(output_dir)
  reporter$from_reporter(reporter_new)
}

#' @keywords internal
archiver_download_handler_engine <- function(reporter, type, file) {
  switch(type,
    JSON = {
      archiver_dir <- JSONArchiver$new()$write(reporter)
      temp_zip_file <- tempfile(fileext = ".zip")
      tryCatch(
        expr = zip::zipr(temp_zip_file, archiver_dir$get_output_dir()),
...
      )

      tryCatch(
        expr = file.copy(temp_zip_file, file),
...
      )
    }
  )
}

[Polkas]

I can recall essential things now.

The from_jsondir and to_jsondir seems to be suited for Archiver (removed in this PR) class, not Reporter. There were two reasons It was decided this way. As the whole process assumes a singleton design for the Reporter, the method on the level of the Reporter makes it very easy to update it. On the other hand, when these two methods were in Archiver, we needed the additional from_reporter method, and the process was less optimal.
Another thing is that I was thinking of the Data Base solution based on a solution like AWS S3, then we could use the from_jsondir and to_jsondir methods for it too. We can send a directory directly to AWS S3 buckets. What class will manage such DB functionality? We need some additional encapsulation, then.

To sum up, the whole problem of perceiving this item is how we think about other solutions for saving the report/previewer. Why the current could be better as the end user could end up with hundreds of files in his local Downloads folder. More than that, the end user is responsible for managing the files himself.

@nikolas-burkoff I think you will be interested.

[gogonzo]

Closing along with #81