Exporting data I: Machine-readable
Explains how to export data into a machine-readable formats
Tutorial: Exporting data I (Machine-readable)
Step 1
Open the example project Tutorial_B22-1_Part1.jip and navigate to the Export compartment (red arrow 1).
Update the cache using the output node’s Cache intermediate results option (red arrow 2).
Step 2
In the Cache browser switch to the Import/Export tab (red arrow 1). Here you can manually export data into a JIPipe-compatible format. You have two options:
- Exporting the the currently viewed table of data (red arrow 2) as ZIP or directory
- Exporting the cache of the whole node (red arrow 3) as directory
The difference between these options is that the cache export ensures that you can later load in the result back into the current node via the Import function in the Cache section. To enable this, JIPipe saves multiple tables and additional metadata.
👉 Try to export the table as ZIP and directory, as well as exporting the cache.
Step 3
JIPipe comes with a node that allows the automated export of data into the standardized format.
This can be achieved by adding (red arrow 1) the node Export data table and connecting it to the output of any node (red arrow 2).
For example, we connected the node to the AVG density output of the Analysis output.
Step 4
By default, the Export data table node will store its output inside a automatically generated directory relative to the current output path (for cache runs it is in a temporary directory.
Alternatively, you can provide a custom path or one that is relative to the project directory:
To do this, select the Export data table node and navigate to the Parameters tab (red arrow 1).
Provide a relative output directory (i.e. does not start with / on macOS/Linux or with a drive letter on Windows; red arrow 2). Then enable the setting Output relative to project directory (red arrow 3).
Output relative to project directory only works if you loaded the project from a *.jip file or have saved it at least once.
Step 5
Running the Export data table node now generates a directory next to the project file that follows the standardized JIPipe output format.
The actual data is contained in the data.csv file in the 0 directory (because it is the first row of the data table; red arrow 1). All other files contain the metadata.
Step 6
A benefit of the machine-readable JIPipe format is that all contained data and metadata can be conveniently restored by JIPipe.
Open the Tutorial_B22-1_Part1.jip project that comes pre-loaded with nodes that cover various import scenarios:
The node Import data table (archive) (red rectangle 1) can import a ZIP file generated by the cache browsers' ZIP-exporter function.
If the data is stored inside a directory (or if you just extract the ZIP), use the Import data table (directory) node (red rectangle 2 and 3).
In the example pipeline, we import both the All outputs export generated by the cache browser, and the AVG-Density output that was exported via the node.
You can easily identify valid ZIP files or directories for the import by checking whether a data-table.json file is located directly within the directory or ZIP file.