Download Fast5 File
Angelo Dearring
Imagine that I have a fast5 file created by MinION which contains information for 4000 reads. I have a list of titles for, say, 1000 reads, and I want to create a new fast5 file which contains information only for those 1000 reads. How to do this?
I will be grateful for possible help.
My project involves sequencing cDNA (comparative transcriptomics). The PI's in my lab basecall only the fast5_pass data but could I basecall both the pass and fail fast5 data and combine the passe reads in each group?
Concrete implementation of the fast5 file schema using the generic h5py library
- Plain-english-named methods to interact with and reflect the fast5 file schema
- Tools to convert between multi_read and single_read formats
- Tools to compress/decompress raw data in files
h5py: 2.6 or higher
- NumPy: 1.11 or higher
- six: 1.10 or higher
- progressbar33: 2.3.1 or higher
The ont_fast5_api provides terminal/command-line console_scripts forconverting between files in the Oxford Nanopore single_read andmulti_read .fast5 file formats. These are provided to ensure compatibility betweentools which expect either the single_read or multi_read .fast5 fileformats.
Where /data/reads and/or its subfolders contain single_read .fast5files. The output will be multi_read fast5 files each containing 100 reads,in the folder: /data/multi_reads with the names: batch_output_0.fast5,batch_output_1.fast5 etc.
Where /data/multi_reads and/or its subfolders contain multi_read .fast5files and read_id_list.txt is a text file either containing 1 read_id per lineor a tsv file with a column named read_id.The output will be multi_read .fast5 files each containing 100 reads,in the folder: /data/multi_reads with the names: batch_output_0.fast5,batch_output_1.fast5 etc.
Where /data/multi_reads and/or its subfolders contain fast5 files from multiplexed experiment,barcoding_summary.txt is the output of guppy_barcoder. /data/demultiplexed_reads will contain a directory perbarcode, containing multi_read .fast5 files with names: /data/demultiplexed_reads/barcode01/batch_0.fast5,/data/demultiplexed_reads/barcode02/batch_0.fast5 etc. Directories are named by values in demultiplex column.
Where /data/uncompressed_reads and/or its subfolders contain .fast5 files. The output will be a copy of the inputfolder structure containing compressed reads preserving both the folder structure and file type.
Benchmarking the performance of compression within the ont_fast5_api against a normal file copy showedcompressing from gzip to vbz was approximately 2x slower than copying files. In other words, if it would take twohours to copy a set of files from an input folder to an output folder then it should take four hours to compress thosefiles with VBZ. Running the script without compressing (i.e. the same type of compression in and out; gzip->gzip)was approximately 2x faster than a file copy since it can utilise mutiple threads.
Single read fast5 - A fast5 file containing all the data pertaining to asingle Oxford Nanopore read. This may include raw signal data, run metadata,fastq-basecalls and any other additional analyses
This tutorial uses the ont_fast5_api software; this is not installed in the default EPI2ME Labs environment. We will install this now in an isolated manner so as to not interfere with the existing environment.
Files output by the MinKNOW instrument software and the Guppy basecalling software using the .fast5 file extension are a container file using the HDF5 format. As such they are a self-describing file with all the necessary information to correctly interpret the data they contain.
A Fast5 file differs from a generic HDF5 file in containing only a fixed, defined structure of data. This structure is elucidated in the ont_h5_validator repository on Github, specifically in the file multi_read_fast5.yaml.
For files generated by MinKNOW without live basecalling enabled, the Analyses section will be absent (or contain no subgroups). The sample file is one that has been created by Guppy using the --fast5_out option to produce Fast5 files in addition to .fastq.gz files containing solely the basecalls. The Analyses section listed above therefore contains two subgroups: Segmentation_000 and Basecall_1D_000. The first of these contains information regarding how a read as been trimmed by the basecaller into seqeuncing adapter and, barcode and insert regions. The second contains the basecaller outputs, primarily the Fastq dataset but also two additional groups Move and Trace which contain advanced basecaller outputs. Again see the documentation in the Nanopore Community for a full description of these.
Oxford Nanopore Technologies provides a Python-based software for accessing data stored within a set of Fast5 files: ont_fast5_api. For the most part this set of tools hides from the user the need to understand anything about the nature of Fast5 files. Here we will show how to perform some common tasks that might be required when dealing with Fast5 files. For a guide in using ont_fast5_api programmatically please see the documention.
760c119bf3