JIPipe Help

R integration

JIPipe provides nodes that allow to execute native R code and use it to process data. All nodes that are relevant to R processing can be found in the Miscellaneous | R script category. The following nodes are available:

R script (iterating) ⭐

(Most commonly used node)

Executes the R script for each iteration step. Each slot receives exactly one data item per step.

R script (merging)

Executes the R script for each iteration step. Each slot receives multiple data items per step if they cannot be separated by annotations.

Background

The JIPipe R integration basically works by applying the following steps:

  1. Create temporary directories for inputs and outputs

  2. Write all input data to its temporary directory in JIPipe's data table format.

  3. Run a modified R script that has additional variables and functions provided (injected) by JIPipe

  4. Let JIPipe read all data from the output directory. The data must be in JIPipe's data table format.

Writing R scripts

Generally, you can use all R libraries that are available to you in the currently configured R environment. But to facilitate the data transfer between JIPipe and R, you will need to use the functionality provided by the JIPipe-specific R code.

JIPipe-generated variables

On executing the R script, JIPipe will automatically inject the following global variables into the R script:

Name

Description

JIPipe.InputSlotRowCounts

Named list of the number of input data provided by the given input (always 1 for R script (iterating)).

count <- JIPipe.InputSlotRowCounts$Input

JIPipe.TextAnnotations

Named list of the text annotations associated to the current iteration step.

value <- JIPipe.TextAnnotations$key

JIPipe.InputSlotRowTextAnnotations

Named list of the text annotations associated to the input slot row. This may be helpful if you want to access the annotations before merging them.

value <- JIPipe.InputSlotRowTextAnnotations$key

JIPipe.Variables

Named list of variables from the Script variables parameter of the node.

JIPipe-generated functions

The R script will also have the following functions:

Signature

Description

JIPipe.GetInputFolder(slot, row=0)

Returns the row storage folder where the data of the given slot and row are stored.

slot

(String) The slot name

row

(Integer) The row index, defaults to 0

<Returns>

(String) The path to the storage directory

JIPipe.GetInputAsDataFrame(slot, row=0)

Returns the data of the given input slot and row as data frame. Please note that the input should be a Results table.

slot

(String) The slot name

row

(Integer) The row index, defaults to 0

<Returns>

(data.frame) The data

JIPipe.AddOutputFolder(slot, annotations=list())

Adds a new output folder into the specified output slot and returns the folder path. Optionally, you can assign annotations to add as list of named strings. Please note that the folder must contain data according to the slot's data type (an image file or a CSV file respectively)

slot

(String) The slot name

annotations

(Named list) The text annotations

<Returns>

(String) The output directory path. You will need to store supported data into it.

JIPipe.AddOutputDataFrame(data, slot, annotations=list())

Adds the specified data frame into the specified output slot (must be a Results table). Optionally, you can add annotations to the data.

data

(data.frame) The data

slot

(String) The slot name

annotations

(Named list) The text annotations

JIPipe.AddOutputPNGImagePath(ignored, slot, annotations=list())

Generates an image file path to be added as output and return the path. Please note that you must use png() or other functions to actually write this file. Optionally, you can add annotations to the data.

ignored

Can be ignored. Set it to NULL.

slot

(String) The slot name

annotations

(Named list) The text annotations

<Returns>

(String) The output file path. Please export a PNG file to this path.

JIPipe.AddOutputTIFFImagePath(ignored, slot, annotations=list())

Generates an image file path to be added as output and return the path. Please note that you must use tiff() or other functions to actually write this file. Optionally, you can add annotations to the data.

ignored

Can be ignored. Set it to NULL.

slot

(String) The slot name

annotations

(Named list) The text annotations

<Returns>

(String) The output file path. Please export a TIFF file to this path.

Importing data from JIPipe into R

To copy data from the JIPipe into an R variable you will need to use the JIPipe-provided functions to either directly import a data.frame or manually load data from the row storage path.

The JIPipe adapter code provides a convenient function to read a data.frame from a Results table input:

# Import as from input "Input" df <- JIPipe.GetInputAsDataFrame("Input")

If the input is not a data frame, you can obtain the row folder that contains the file(s) that represent the data.

# Get the path where the data is stored folder <- JIPipe.GetInputFolder("Input", row=0) # You can use for example Sys.glob(file.path(folder, "*.tif")) to find all TIFF files # Please adapt this for your data

Exporting data from R into JIPipe

JIPipe expects that data is placed at very specific locations, which can be achieved by utilizing the JIPipe-provided functions and variables.

Depending on which kind of data you want to export, you will have the following options:

You can use a JIPipe-provided function to conveniently output a data.frame back into JIPipe:

# Assume we have a data.frame df # Output it into the slot "Output" JIPipe.AddOutputDataFrame(slot="Output", data=df)

For images to be detected by JIPipe, you will need to export an image file (PNG/TIFF/JPG/BMP) into a specific directory. You can obtain the path via the JIPipe-provided functions.

# For example we have a ggplot plot # Get the output directory folder <- JIPipe.AddOutputFolder("Output", annotations=list()) # Save the file # The file name is not important, just the extension ggsave(folder + "/data.png")

If you are working with other data types, please read about how JIPipe stores this particular type by navigating to Help | Data type documentation and going to the Data storage section.

# Get the output directory folder <- JIPipe.AddOutputFolder("Output", annotations=list()) # Export everything into folder

Debugging R scripts

While JIPipe currently does not have any advanced debugging features, you can utilize the log entry generated by JIPipe to identify errors and even run your pipeline outside JIPipe.

Full script printout

As JIPipe applies modifications to your script, it will print the final script into the log window:

Screenshot of the log window
Commandline run command

JIPipe will also log all environment variables and the full run command:

Screenshot of the log window
R stdout/stderr output

JIPipe will also log all standard out and standard error messages directly after the run command.

JIPipe-provided python

JIPipe will automatically take care of setting up R for you and select the newest available artifact for you. The following R versions with the specified libraries are available:

abind askpass backports base64enc bit bit64 blob brio broom bslib cachem callr car carData caret cellranger cli clipr clock colorspace conflicted cpp11 crayon curl data.table DBI dbplyr desc diagram dials DiceDesign diffobj digest doFuture dplyr dtplyr e1071 ellipsis evaluate fansi farver fastmap fontawesome forcats foreach fs furrr future future.apply gargle generics ggforce ggplot2 ggraph ggrepel globals glue googledrive googlesheets4 gower GPfit graphlayouts gridExtra gtable hardhat haven highr hms htmltools httr ids igraph infer ipred isoband iterators jquerylib jsonlite knitr labeling lava lhs lifecycle listenv lme4 lubridate magrittr MatrixModels memoise mime minqa modeldata modelenv ModelMetrics modelr multcomp munsell mvtnorm nloptr numDeriv openssl parallelly parsnip patchwork pbkrtest pillar pkgbuild pkgconfig pkgload plyr polyclip praise prettyunits pROC processx prodlim progress progressr proxy ps purrr quantreg R6 ragg rappdirs RColorBrewer Rcpp RcppArmadillo RcppEigen readr readxl recipes rematch rematch2 reprex reshape2 rjson rlang rmarkdown rprojroot rsample rstudioapi rvest sandwich sass scales selectr shape slider SparseM SQUAREM stringi stringr sys systemfonts testthat textshaping TH.data tibble tidygraph tidymodels tidyr tidyselect tidyverse timechange timeDate tinytex tune tweenr tzdb utf8 uuid vctrs viridis viridisLite vroom waldo warp withr workflows workflowsets xfun xml2 yaml yardstick zoo

Same packages as 4.4.0.1000.

Overriding the R version

You can always override the utilized R version on a node, project, and application level.

  1. Click the Infos & Settings button at the top right.

  2. Go to the Settings tab on the right-hand side and select Plugins | R integration.

  3. Ensure that Project default environment is enabled.

  4. Click the Configure ... button and select one of the options.

  1. Select the node you want to change

  2. Go to the Parameters panel and activate the parameter Override R environment.

  3. Click the Configure ... button located next to the Override R environment parameter and select one of the options.

  1. Navigate to Project | Application settings

  2. Select Plugins | R integration

  3. Click the Configure ... button located next to Default R environment and select one of the options.

Last modified: 19 September 2024
Python integrationFrequently asked questions