SequenceServer 2.0 (release candidate)

[Anurag Priyam]

Dear colleagues,

We are pleased to announce a candidate release of SequenceServer 2.0. This completes the rearchitecting of SequenceServer 1.0 that we announced almost two years ago: https://groups.google.com/d/msg/sequenceserver/2G4_jdAr4-k/2yxQKixVBQAJ

We have made many changes since version 1.0. The key features are:

1. Support for BLAST 2.10.0 and the new BLAST database format.

2. Results now open in a new page. You can share the link with colleagues or bookmark the page and return to it at a later date. Results are saved for 30 days.

3. We have integrated three new visualisations that facilitate interpretation of BLAST results:

Length distribution (histogram) of all hits of a query. This is inspired from our work with GeneValidator (https://academic.oup.com/bioinformatics/article/32/10/1559/1742817) and can, for example, help quality-assessment of gene predictions.

Kablammo visualisation (https://academic.oup.com/bioinformatics/article/31/8/1305/212772) for each query-hit pair indicates clearly which parts of the query match which part of the hits.

Circos-style plot of queries and their top hits. This is similar to the circoletto tool (https://academic.oup.com/bioinformatics/article/26/20/2620/194655). For example, it can help visually testing for conserved synteny or gene duplication.

4. We provide a command-line mechanism for importing a BLAST XML file and visualising it in SequenceServer. This works for DIAMOND as well.

5. The 30 hit limit for FASTA download has been removed. In addition, you can now download raw pairwise alignments.

6. Better support for long-running BLAST searches, for rendering large BLAST results (thousands of hits), and for integrating as part of other websites.

7. Improved error handling, security enhancements, and a new look!

Compared to the beta releases, the biggest remaining limitations were SequenceServer's handling of large BLAST results and back button of browsers. We have addressed those.

We request you to try out the "candidate" release of version 2.0 and report any issues you encounter with your setup (or give us a thumbs up). We will update the documentation at http://sequenceserver.com over the next few weeks. In the meantime, please feel free to ask any question here or on GitHub.

Upgrading is simple. If you have Ruby 2.3 or later, run:

   gem install --pre sequenceserver

This does not remove your current sequenceserver installation. Instead, it changes

the default. If something breaks, you can rollback by running:

   gem uninstall sequenceserver

Please report any issues here: https://github.com/wurmlab/sequenceserver/issues.

You can also send us pull-requests. We welcome any help you may be able to provide in terms of improving test coverage and making code style checkers happy.

Kind regards,

Priyam & Yannick

-------------------------

https://wurmlab.github.io

Evolutionary genomics @ Queen Mary U London

[Anurag Priyam]

Dear colleagues,

We are pleased to announce the second candidate release of SequenceServer 2.0 addressing reported bugs and containing a few other enhancements.

Change and upgrade notes below. We would very much appreciate it if you could try out the "candidate" release of version 2.0 and report any issues you encounter with your setup. 

Similarly, if you have 5 minutes, please help us by taking the following brief survey: 

https://forms.gle/yvDX9PTxSCgjZoqj7

# Release notes

1. A programming error prevented databases from being deselected when you go back to the search form after submitting a query. That's now fixed. Thanks to Norman Johnson for reporting the issue.

2. Database creation step could skip FASTA files containing too many Ns at the beginning of the file. Fixed that. Thanks to Michał T. Lorenc for reporting the issue.

3. During the initial setup SequenceServer would print the message 'config file not found' multiple times. Fixed that.

4. Results page would fail to load completely if the submitted queries resulted in no hits. Fixed that.

5. Give users an option to sign up for release and other important announcements during the initial setup.

6. Highlight the use of non-default parameters by changing the background of advanced parameters field to yellow when filled.

7. Include SequenceServer version and a link to our paper in the search summary at the top of the results page.

# New documentation

We have also been able to make many of the required changes to the website for the 2.0 release and are testing it locally: https://github.com/wurmlab/sequenceserver/tree/gh-pages2.0. 

# Upgrading

Upgrading is simple. If you have Ruby 2.3 or later, run:

   gem install --pre sequenceserver

This does not remove your current sequenceserver installation. Instead, it changes

the default. If something breaks, you can rollback by running:

   gem uninstall sequenceserver

# Issues / comments/ contributions

Please report any issues here: https://github.com/wurmlab/sequenceserver/issues.

You can also send us pull-requests. We welcome any help you may be able to provide in terms of improving test coverage and making code style checkers happy.

Kind regards,

Priyam & Yannick

-------------------------

https://wurmlab.github.io

Evolutionary genomics @ Queen Mary U London

[Anurag Priyam]

Dear colleagues,

Thank you for trying out the candidate releases, sharing your feedback, and trying out the fixes.

Based on your feedback we have fixed bugs affecting docker and sub-URI deployments, updated documentation for HPC-integration, and most importantly, we have created a path for migrating to NCBI’s newer V5 database format (see release notes below).

If you have 5 minutes, please help us by taking the following brief survey: https://forms.gle/yvDX9PTxSCgjZoqj7

# Release notes

1. Running 'sequenceserver -m' will now automatically detect older V4 databases, and those created without the '-parse_seqids' option of makeblastdb, and offer to rebuild them. This works even if you deleted the original FASTA files. Database titles are preserved when rebuilding, however, taxonomy information in the database is unfortunately not preserved.

2. If you do not wish to rebuild your databases yet, you can now run SequenceServer with BLAST+ 2.9.0 instead. The tradeoff is that you would have to manually download BLAST+ 2.9.0 and indicate its location in the configuration file. SequenceServer will only download 2.10 by default.

3. If building BLAST databases fails for some reason, the command that was tried and the resulting output is properly relayed to the user so that the problem can be diagnosed (#469). Thanks to mjcoynejr for testing the fix.

4. BLASTing a mix of older V4 and newer V5 databases causes error. SequenceServer now catches and informs the user of this error. Thanks to Massimiliano babbucci for reporting this issue.

5. On sub-URI deployments, clicking the back button would result in no databases to be displayed in the search form (#462). Fixed that. Thanks to Xiang Zhang for reporting the issue and testing the fix.

6. The list of databases in the search form should be alphabetically sorted. This behaviour was lost in the rewrite leading to version 2.0 and has now been fixed. Thanks to Loraine Guéguen for reporting the issue.

7. Docker-related optimisations: SequenceServer no longer prompts to join the mailing list when running under Docker (#468). Thanks to Matt for reporting the issue. Furthermore, we have updated the docker base image and reduced the image size by ~100 Mb.

8. Documentation for HPC integration was out of date with regard to how to invoke the wrapper script (#445). Thanks to Loraine Guéguen for reporting the issue, testing the fix.

Thanks to Niek Art, Matt Yoder, Massimiliano babbucci, Bjoernsen, and Eric Y for reporting database compatibility issues that prompted us to create a path for migrating to the new database format.

# Upgrading

Upgrading is simple. If you have Ruby 2.3 or later, run:

   gem install --pre sequenceserver

This does not remove your current sequenceserver installation. Instead, it changes

the default. If something breaks, you can rollback by running:

   gem uninstall sequenceserver

# Issues / comments/ contributions

Please report any issues here: https://github.com/wurmlab/sequenceserver/issues.

You can also send us pull-requests. We welcome any help you may be able to provide in terms of improving test coverage and making code style checkers happy.

Kind regards,

Priyam & Yannick

-------------------------

https://wurmlab.github.io

Evolutionary genomics @ Queen Mary U London

[Anurag Priyam]

Dear colleagues,

Thank you for trying out the candidate releases, sharing your feedback and sending bug fixes. We are pleased to announce the next candidate release of SequenceServer 2.0.

# Release notes

1. Automatically check for incompatible databases on startup and prompt users to reformat them.

2. Upgrading from the older (V4) to the newer (V5) database now preserves any taxonomy information embedded in the database.

3. Add ability to use -taxids_map of makeblastdb during database creation. To use it, add a '.taxids_map.txt' file next to the FASTA file (e.g. "/db/ants.taxids_map.txt" for "/db/ants.fa")

4. Add option in the search form to open BLAST results in a new tab.

5. Add 'Edit search' and 'New search' links to the report page. (thanks to Tomas-Pluskal & TomMD for the push)

6. Search form can now be cleared by reloading the page. Relevant if you use your web browser's back button or 'Edit search' link and want to clear the form to start over.

7. Make it easier to pass command line arguments to Docker image. For example, number of BLAST threads can now be set as "docker run ... wurmlab/sequenceserver sequenceserver -n 4" instead of "docker run ... wurmlab/sequenceserver bundle exec bin/sequenceserver -d /db -n4" (yeah!)

8. Reduce size of Docker image 3-fold. (credit: Nathan Weeks, Neel Kamath)

9. Reduce size of gem file 3-fold. (credit: Nathan Weeks)

10. Fix a bug that prevented query textarea from being resized veritcally. (credit: Iain-S)

11. Fix a bug that prevented search form from rendering query sequences sent to it via POST request. (credit: Sam Hokin)

12. Fix a bug that prevented the use of keyboard shortcut 'Ctrl-Enter' to submit the search form.

13. Fix a bug that would cause an infinite loop on search form if the user had only one database.

14. Fix a bug that could cause new users to be prompted more than once to join the announcements list.

15. Fix a bug that was causing Travis builds to fail.

Special thanks to Iain-S for introducing us to hacktoberfest. It was incredibly productive, leading to changes 8-10 above.

# Upgrading

Upgrading is simple. If you have Ruby 2.3 or later, run:

   gem install --pre sequenceserver

This does not remove your current sequenceserver installation. Instead, it changes the default. If something breaks, you can rollback by running:

   gem uninstall sequenceserver

# Issues / comments/ contributions

Please report any issues here: https://github.com/wurmlab/sequenceserver/issues.

You can also send us pull-requests. We welcome any help you may be able to provide, including improving test coverage and making code style checkers happy.

Kind regards,

Priyam & Yannick

-------------------------

https://wurmlab.github.io

Evolutionary genomics @ Queen Mary U London