BPCells fragments can be read/written in compressed (bitpacked) or uncompressed form in a variety of storage locations: in memory (as an R object), in an hdf5 file, or in a directory on disk (containing binary files).
Usage
write_fragments_memory(fragments, compress = TRUE)
write_fragments_dir(
fragments,
dir,
compress = TRUE,
buffer_size = 1024L,
overwrite = FALSE
)
open_fragments_dir(dir, buffer_size = 1024L)
write_fragments_hdf5(
fragments,
path,
group = "fragments",
compress = TRUE,
buffer_size = 8192L,
chunk_size = 1024L,
overwrite = FALSE,
gzip_level = 0L
)
open_fragments_hdf5(path, group = "fragments", buffer_size = 16384L)Arguments
- fragments
Input fragments object
- compress
Whether or not to compress the data. With compression, storage size is be about half the size of a gzip-compressed 10x fragments file.
- dir
Directory to read/write the data from
- buffer_size
For performance tuning only. The number of items to be bufferred in memory before calling writes to disk.
- overwrite
If
TRUE, write to a temp dir then overwrite existing data. Alternatively, pass a temp path as a string to customize the temp dir location.- path
Path to the hdf5 file on disk
- group
The group within the hdf5 file to write the data to. If writing to an existing hdf5 file this group must not already be in use
- chunk_size
For performance tuning only. The chunk size used for the HDF5 array storage.
- gzip_level
Gzip compression level. Default is 0 (no compression). This is recommended when both compression and compatibility with outside programs is required. Otherwise, using compress=TRUE is recommended as it is >10x faster with often similar compression levels.
Details
Saving in a directory on disk is a good default for local analysis, as it provides the best I/O performance and lowest memory usage. The HDF5 format allows saving within existing hdf5 files to group data together, and the in memory format provides the fastest performance in the event memory usage is unimportant.
