GenBank Release 227.0 Available : August 16 2018 : 7:22pm

[Cavanaugh, Mark (NIH/NLM/NCBI) [E]]

Greetings GenBank Users,

GenBank Release 227.0 is now available via FTP from the National Center
for Biotechnology Information (NCBI):

Ftp Site Directory Contents
---------------- --------- ---------------------------------------
ftp.ncbi.nih.gov genbank GenBank Release 227.0 flatfiles
ncbi-asn1 ASN.1 data used to create Release 227.0

Close-of-data for GenBank 227.0 occurred on 08/13/2018. Uncompressed,
the Release 227.0 flatfiles require roughly 894 GB (sequence files only).
The ASN.1 data require approximately 738 GB.

Recent statistics for 'traditional' sequences (including non-bulk-oriented
TSA, and excluding WGS, bulk-oriented TSA, TLS, and the CON-division):

Release Date Base Pairs Entries

226 Jun 2018 263957884539 209775348
227 Aug 2018 260806936411 208831050

Recent statistics for WGS sequencing projects:

Release Date Base Pairs Entries

226 Jun 2018 2944617324086 639804105
227 Aug 2018 3204855013281 665309765

Recent statistics for bulk-oriented TSA sequencing projects:

Release Date Base Pairs Entries

226 Jun 2018 216556686631 238788334
227 Aug 2018 225520004678 249295386

Recent statistics for bulk-oriented TLS sequencing projects:

Release Date Base Pairs Entries

226 Jun 2018 5896511468 15393041
227 Aug 2018 6077824493 15822538

During the 57 days between the close dates for GenBank Releases 226.0
and 227.0, the 'traditional' portion of GenBank declined by 3,150,948,128
basepairs and 944,298 sequence records. See Section 1.3.1 of the
release notes for further information about this net decrease.
During that same period, 477,074 records were updated and 1,520,363 new
records were added. An average of 35,042 'traditional' records were
added and/or updated per day.

Between releases 226.0 and 227.0, the WGS component of GenBank grew by
260,237,689,195 basepairs and by 25,505,660 sequence records.

Between releases 226.0 and 227.0, the TSA component of GenBank grew by
8,963,318,047 basepairs and by 10,507,052 sequence records.

Between releases 226.0 and 227.0, the TLS component of GenBank grew by
181,313,025 basepairs and by 429,497 sequence records.

For additional release information, see the README files in either of
the directories mentioned above, and the release notes (gbrel.txt) in
the genbank directory. Sections 1.3 and 1.4 of the release notes
(Changes in Release 227.0 and Upcoming Changes) have been appended
below for your convenience.

* * * Important Notice * * *

Section 1.4.1 of the GenBank release notes describes future accession
format changes for traditional nucleotide sequences, WGS/TSA/TLS
sequences, and for protein sequences. These important changes are likely
to be of interest to many GenBank users, and we encourage a review of
the section.

Release 227.0 data, and subsequent updates, are available now via
NCBI's Entrez and Blast services.

As a general guideline, we suggest first transferring the GenBank
release notes (gbrel.txt) whenever a release is being obtained. Check
to make sure that the date and release number in the header of the
release notes are current (eg: August 15 2018, 227.0). If they are
not, interrupt the remaining transfers and then request assistance from
the NCBI Service Desk.

A comprehensive check of the headers of all release files after your
transfers are complete is also suggested. Here's how one might go about
this on a Unix or Linux platform, using csh/tcsh :

set files = `ls gb*.*`
foreach i ($files)
head -10 $i | grep Release
end

Or, if the files are compressed, perhaps:

gzcat $i | head -10 | grep Release

If you encounter problems while ftp'ing or uncompressing Release
227.0, please send email outlining your difficulties to:

in...@ncbi.nlm.nih.gov

Mark Cavanaugh, Michael Kimelman, Ilya Dondoshansky
GenBank
NCBI/NLM/NIH/HHS


1.3 Important Changes in Release 227.0

1.3.1 Suppression of nearly two million sequence records for 36 genomes

A submission of 36 unannotated nematode-related genomes to our colleagues
at the European Nucleotide Archive was recently suppressed by their submitter.
The genomes in question will be annotated and resubmitted by the Wellcome
Trust Sanger Institute, as WGS projects rather than traditional sequences
utilizing nucleotide accessions of the "2+6" format.

This large-scale suppression has resulted in a net *decrease* in the
number of sequences and basepairs for the non-WGS/TSA/TLS portion of GenBank.
Since this is the first-ever such decrease, providing some details about
the genomes involved seems appropriate. A total of 36 distinct accession
number ranges are involved, with 1,962,764 accessions (see below for the
specific ranges and organism names). Although well over one million sequence
records are new for GenBank 227.0, the overall number of records declined
by 944,000, and the overall number of basepairs declined by 3.15 Gbp. These
decreases can be seen in Table 1, Section 2.2.8, "Growth of GenBank".

Suppressed accession ranges:

LK871857-LK903566
LK903567-LK927529
LK939141-LK945524
LK945525-LK964159
LK964160-LK978188
LK978189-LK979538
LK979539-LK985316
LK996042-LK999920
LL000001-LL188369
LL188370-LL233095
LL233096-LL319178
LL319179-LL361183
LL361184-LL515692
LL569982-LL710275
LL710276-LL718353
LL718354-LL748444
LL748445-LL876845
LL876846-LL900200
LL900201-LL956978
LL956979-LL999047
LM000001-LM013520
LM013521-LM035856
LM035857-LM065041
LM065042-LM125181
LM125182-LM149318
LM149319-LM211379
LM211380-LM378689
LM383430-LM397339
LM397340-LM413551
LM413552-LM433383
LM433384-LM462758
LM462759-LM523157
LM530258-LM537325
LM537584-LM583055
LM583056-LM607978
LM676446-LM992843

Suppressed organism names:

Angiostrongylus cantonensis
Angiostrongylus costaricensis
Anisakis simplex
Ascaris lumbricoides
Brugia pahangi
Brugia timori
Cylicostephanus goldi
Dibothriocephalus latus
Dracunculus medinensis
Echinostoma caproni
Elaeophora elaphi
Enterobius vermicularis
Gongylonema pulchrum
Haemonchus placei
Heligmosomoides polygyrus
Hydatigera taeniaeformis
Hymenolepis diminuta
Hymenolepis nana
Mesocestoides corti
Nippostrongylus brasiliensis
Onchocerca flexuosa
Parascaris equorum
Protopolystoma xenopodis
Schistocephalus solidus
Schistosoma curassoni
Schistosoma margrebowiei
Schistosoma mattheei
Schistosoma rodhaini
Soboliphyme baturini
Strongylus vulgaris
Syphacia muris
Taenia asiatica
Thelazia callipaeda
Toxocara canis
Trichobilharzia regenti
Wuchereria bancrofti

Suppression of genomes leads to their removal from BLAST databases at the
NCBI. The new, annotated versions of these genomes are expected to be
distributed by the ENA fairly soon, at which point they will become BLAST-able
once again, and available via our standard WGS mechanisms.

1.3.2 Organizational changes

The total number of sequence data files increased by 2 with this release:

- the BCT division is now composed of 520 files (+28)
- the CON division is now composed of 369 files (+4)
- the GSS division is now composed of 308 files (+2)
- the INV division is now composed of 108 files (-60)
- the PAT division is now composed of 337 files (+2)
- the PLN division is now composed of 198 files (+24)
- the PRI division is now composed of 59 files (+1)
- the VRL division is now composed of 57 files (+1)

The decrease in the number of INV-division files is a consequence of the
suppression of the thirty-six genomes described in Section 1.3.1 of these
release notes.

1.3.3 GSS File Header Problem

GSS sequences at GenBank are maintained in two different systems, depending
on their origin, and the dumps from those systems occur in parallel. Because
the second dump (for example) has no prior knowledge of exactly how many GSS
files will be dumped by the first, it does not know how to number its own
output files.

There is thus a discrepancy between the filenames and file headers for 130
of the GSS flatfiles in Release 227.0. Consider gbgss179.seq :

GBGSS1.SEQ Genetic Sequence Data Bank
August 15 2018

NCBI-GenBank Flat File Release 227.0

GSS Sequences (Part 1)

87375 loci, 64103840 bases, from 87375 reported sequences

Here, the filename and part number in the header is "1", though the file
has been renamed as "179" based on the number of files dumped from the other
system. Files gbgss179.seq.gz through gbgss308.seq.gz are affected. We hope
to resolve this discrepancy at some point, but the priority is certainly
much lower than many other tasks.

1.4 Upcoming Changes

1.4.1 Changes to nucleotide and protein accession formats

The accession format used for Whole Genome Shotgun (WGS), Transcriptome
Shotgun Assembly (TSA), and Targeted Locus Study (TLS) sequencing projects
consists of a four-letter Project Code prefix, a two-digit Assembly-Version
number, followed by 6, 7, or 8 digits (depending on the number of sequences
in the project).

By the end of 2018 the INSDC members plan to expand this format, using
a six-letter Project Code prefix, two-digit Assembly-Version number, followed
by 7, 8, or 9 digits. An example of such an accession is AAAAAA020000001 .

Non-WGS/TLS/TSA nucleotide sequences currently make use of a "2+6"
accession format, consisting of a two-letter prefix followed by six digits.
This format will be expanded to make use of eight digits, allowing accession
ranges that have been exhausted (eg: JG000001-JG999999) to be "re-opened"
(eg: JG00000001-JG99999999).

Protein sequences currently make use of a "3+5" accession format, consisting
of a three-letter prefix followed by five digits. This format will be expanded
to make use of seven digits, allowing exhausted protein accession ranges such
as EZZ00001-EZZ99999 to be re-opened as EZZ0000001-EZZ9999999.

The earliest date on which these new accessions will appear for GenBank
sequences is December 15th 2018.