Dear GCP life sciences team
i have a question as I had run deep variant instance
it generated report in ubuntu gcp environment
- HG003.output.g.vcf.gz
- HG003.output.g.vcf.gz.tbi
- HG003.output.vcf.gz
- HG003.output.vcf.gz.tbi
- HG003.output.visual_report.html
- in addition,I did not find "GRCh38_no_alt_analysis_set.fasta" in any bucket gs:// deepvariant
- BIN_VERSION="1.1.0"
BASE="${HOME}/deepvariant-run"
INPUT_DIR="${BASE}/input"
REF="GRCh38_no_alt_analysis_set.fasta"
BAM="HG003.novaseq.pcr-free.35x.dedup.grch38_no_alt.chr20.bam"
OUTPUT_DIR="${BASE}/output"
DATA_DIR="${INPUT_DIR}/data"
OUTPUT_VCF="HG003.output.vcf.gz"
OUTPUT_GVCF="HG003.output.g.vcf.gz"
2nd question about variant transforms
what are the main steps of variant transforms
storing raw vcf files
validation of vcf files
loading of vcf files from cloud storage to big query
it displays table
i) what about ti/tv ratio
i have difficulty to find it right
ii) also shed light on flattening of bigquery variants
iii) parsing and annotation of vcf files
i ,ii, iii are optional steps for variant transforms
3rd question is
i am uploading file about 1 gb ,500 mb from my pc to cloud storage bucket but it fails and returns error in contrast i copy large files from one bucket to other it copies fast and successful
4th question
bq ls --format=pretty GOOGLE_CLOUD_PROJECT
:BIGQUERY_DATASET
this command work
but this does not
bq show --format=pretty GOOGLE_CLOUD_PROJECT:BIGQUERY_DATASET.BIGQUERY_TABLE
is this is the final result of Variant Transforms or i have to calculate ti/tv ratio ,flattening of biqquery variants nad parsing and annotations
-----------------------------------+-------+--------+-------------------+------------------------------+
| tableId | Type | Labels | Time Partitioning | Clustered Fields |
+-----------------------------------+-------+--------+-------------------+------------------------------+
| variant_uk_covid2020__chr1 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr10 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr11 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr12 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr13 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr14 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr15 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr16 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr17 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr18 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr19 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr2 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr20 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr21 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr22 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr3 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr4 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr5 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr6 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr7 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr8 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chr9 | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chrX | TABLE | | | start_position, end_position |
| variant_uk_covid2020__chrY | TABLE | | | start_position, end_position |
| variant_uk_covid2020__residual | TABLE | | | start_position, end_position |
| variant_uk_covid2020__sample_info | TABLE | | | |
+-----------------------------------+-------+--------+-------------------+------------------------------+
in the end i will paste out the error
as well
(genomic-analysis-255208)$ bq show --format=pretty genomic-analysis-255208:lineageB_1_1_7.variant_uk_covid2020
BigQuery error in show operation: Not found: Table genomic-analysis-255208:lineageB_1_1_7.variant_uk_covid2020
AND
bq show --format=pretty genomic-analysis-255208:lineageB_1_1_7.variant_uk_covid2020__residual
+-----------------+---------------------------------------+------------+-------------+------------+-------------------+------------------------------+--------+
| Last modified | Schema | Total Rows | Total Bytes | Expiration | Time Partitioning | Clustered Fields | Labels |
+-----------------+---------------------------------------+------------+-------------+------------+-------------------+------------------------------+--------+
| 14 Feb 01:34:56 | |- reference_name: string | 24 | 996 | | | start_position, end_position | |
| | |- start_position: integer | | | | | | |
| | |- end_position: integer | | | | | | |
| | |- reference_bases: string | | | | | | |
| | +- alternate_bases: record (repeated) | | | | | | |
| | | |- alt: string | | | | | | |
| | |- names: string (repeated) | | | | | | |
| | |- quality: float | | | | | | |
| | |- filter: string (repeated) | | | | | | |
| | +- call: record (repeated) | | | | | | |
| | | |- sample_id: integer | | | | | | |
| | | |- genotype: integer (repeated) | | | | | | |
| | | |- phaseset: string | | | | | | |
+-----------------+---------------------------------------+------------+-------------+------------+-------------------+------------------------------+--------+
-----------------+----------------------------------+------------+-------------+------------+-------------------+------------------+--------+
| Last modified | Schema | Total Rows | Total Bytes | Expiration | Time Partitioning | Clustered Fields | Labels |
+-----------------+----------------------------------+------------+-------------+------------+-------------------+------------------+--------+
| 14 Feb 01:34:53 | |- sample_id: integer | 0 | 0 | | | | |
| | |- sample_name: string | | | | | | |
| | |- file_path: string | | | | | | |
| | |- ingestion_datetime: timestamp | | | | | | |
+-----------------+----------------------------------+------------+-------------+------------+-------------------+------------------+--------+
thanks in advance
haroon zeb