New CATMAID release: 2020.02.15
29 views
Skip to first unread message
Tom Kazimiers
Feb 16, 2020, 11:49:17 AM2/16/20
to CATMAID
Hi all,
We just released a new CATMAID version, 2020.02.15:
https://github.com/catmaid/CATMAID/releases/tag/2020.02.15
You will find a list of changes as we well as some update notes at the
end of this email and on the website linked above. As always, please let
us know if you run into any problems, both as administrator and user.
The docker images tagged latest, stable and 2020.02.15 have been
(re)built.
If you update a CATMAID instance, please use the most recent commit of
the master branch in our GitHub repository, which includes all future
bug fixes for this release:
https://github.com/catmaid/CATMAID
If you are administering a CATMAID server, the "Notes" section provides
information on how to migrate to the new version. The file UPDATE.md
contains a copy of the migration steps as well. Please also note that
this CATMAID version drops support for Python 3.5. Use Python 3.6 or
newer. This release requires also Postgres 11+ and PostGIS 2.5+. If you
have to update your database, please make sure to update to the target
version of PostGIS in all databases *before* you update the database
cluster, it will make the update go more smoothly. This means installing
the new PostGIS version (e.g. v3), connecting to each database and
running `ALTER EXTENSION postgis UPDATE` (check version using \dx). The
same database version needs to be available for the target version of
Postgres. If you need to update your database, go to Postgres 12
directly, because the next version of CATMAID will require Postgres 12
and PostGIS 3.
Cheers,
Tom
2020.02.15
==========
Contributors: Chris Barnes, Andrew Champion, Stephan Gerhard,
Pat Gunn, Tom Kazimiers
Notes
-----
- Python 3.5 is not supported anymore. Use Python 3.6, 3.7 or 3.8.
- Postgres 11 and PostGIS 2.5 is required, Postgres 12 and PostGIS 3 is
recommended. If Postgres needs to be updated, update directly to
Postgres 12. If both needs to be updated, update PostGIS first and
run ``ALTER EXTENSION postgis UPDATE;`` in every database. For
docker-compose setups this database update is performed automatically.
If a replication setup is in use, the database configuration changes
for Postgres 12. CATMAID's replication documentation explains what
needs to be done.
- A virtualenv update is required. Before you start it, please remove
some packages that are not needed anymore first:
pip uninstall asgi-ipc asgi-rabbitmq
- If ASGI was set up before, make sure to install channels_rabbitmq or
channels_redis (depending on what yous use). The older asgi_rabbitmq
and asgi_redis packages aren't supported anymore. This also requires
an update of the CHANNELS_LAYERS in settings.py. The channels_rabbitmq
documentation for an example:
https://github.com/CJWorkbench/channels_rabbitmq/. This variable isn't
defined by default anymore. Therefore you likely have to replace any
`CHANNELS_LAYERS[…] = …` with something like `CHANNELS_LAYERS = { …
}`. The new format is (use custom credentials on any production
system!):
```
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_rabbitmq.core.RabbitmqChannelLayer",
"CONFIG": {
"host": "amqp://guest:gu...@127.0.0.1/asgi",
},
},
}
```
Also, if supervisord is in use to manage CATMAID process groups, the
main Daphne process needs an adjustment: instead of calling `daphne`
with the `mysite.asgi:channel_layer` parameter, use
`mysite.asgi:application`. A complete supervisord entry would then
look something like this:
```
[program:catmaid-daphne]
directory = /home/catmaid/catmaid/django/projects/
command = /home/catmaid/catmaid/django/env/bin/daphne --unix-socket=/var/run/daphne/catmaid.sock mysite.asgi:application
user = www-data
stdout_logfile = /var/log/daphne/catmaid-server.log
redirect_stderr = true
```
As last step, the supervisor entry for the `daphne worker` process has
to be removed. New types of workers can be added, but are not needed
in most cases. The channels documentation has more information on
this.
- GDAL v2 or newer is now needed. If your Ubuntu version doesn't support
this yet, there is an official PPA:
sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt-get update
- The next version of CATMAID will require Postgres 12 and PostGIS 3.
- The management command catmaid_populate_summary_tables is now known as
catmaid_refresh_node_statistics.
- When enabling and disabling both history tracking and spatial update
events on startup, advisory locks are now used. This should make the
startup of parallel CATMAID workers more robust.
- The application of migrations 88-91 and 98-99 might take a while to
complete, because they rewrite potentially big database table
(treenode_edge, treenode, class_instance, and more). Therefore, make
also sure that there is enough space available at the database storage
location (25% of database data directory should be plenty). If no
replication is used, setting the following Postgres options can speed
up the process: `wal_level = minimal`, `archive_mode = off` and
`max_wal_senders = 0`.
Due to this database update data consistency and correctness was
improved, because additional foreign key relationships have been added
that were missing before.
- Both `configuration.py.example` and `create_configuration.py` support
now the option `catmaid_default_enabled_tools`, which defines the list
of front-end tools that are enabled by default for new users. The
following options are supported: cropping, tagging, textlabel,
tracing, ontology and roi. By default the tracing tool is now enabled
for new users, because it seems in many setups users want to have this
enabled by default.
- Back-end errors result now in actual HTTP error status codes.
Third-party clients need possibly some adjustments to handle API
errors. In case of an error, status 400 is returned if an input data
or parameter problem, 401 for permission problems and 500 otherwise.
- On startup, CATMAID will now test if the output directory is
accessible as well as if its expected folder layout is present. If
expected subfolders are missing, they will now be created.
- Python code is now linted by flake8. The config is fairly relaxed (see
.travis.flake8), but contributions which do not conform with the
selected rules will fail CI checks.
Features and enhancements
-------------------------
Permissions:
- If data is imported by a user, e.g. from a remote CATMAID instance,
this user has all permissions over the imported data. If data was
imported or created by someone else, the regular rules apply:
permissions are granted if the user is creator, superuser or has
permission over the data's creator. If user A has permissions over
user B's data, user A will also have permission to edit user B's
imported skeletons. This is done in order to be able to maintain the
original creator and editor data of the imported data, should it be
available.
Node filters:
- Binary split: a new toggle labeled "inclusive" allows to decide
whether to include the split not in an upstream sub-arbor (default:
true).
CLI importer:
- Performance improvement: only the skeleton summary entries of the
imported skeletons are now created.
- Only update treenode edges and skeleton summary if treenodes were
actually imported (rather than e.g. only volumes).
- Add --auto-name-unknown-users option, to automatically generate names
for users that are not available from an import, but referenced in it
(by ID).
CLI exporter:
- The new --annotation-annotation option allows to specify a
meta-annotation that all exported annotations need to share
transitively. If for instance all all top-level annotations that
should be exported are annotated with "exportable" and
"--annotation-annotation exportable" is used, only annotations from
those annotation hierarchies marked as "exportable" will be exported.
Without this option, all annotations are exported.
- If users should be exported, also export users of volumes.
Project statistics:
- Requests to the back-end our now performed in parallel, resulting in
faster loading times.
- It is now possible to configure how many largest neurons should be
displayed as well as an optional name pattern.
- The user/time action table will now also list imported nodes and
imported cable length. These values are subtracted from the
corresponding regular counters. If "Include imports" is unchecked, the
same information that was displayed before this commit is shown, i.e.
the regular node count and cable length field include the imported
data.
Tracing tool:
- The tool bar button to refresh caches has been replaced with a button
to open a context menu titled "More tools". The context menu has an
entry to refresh cashes and a list of tracing data in other projects.
- Tracing data from other CATMAID instances (or other local projects)
can now be added to the active stack viewer by selecting the
respective remote project from the "More tools" menu item "Remote
data".
- To open a remote CATMAID instance at the same location as the current
view, find the remote project in the "Remote data" section of the
"More tools" menu, and click on "Open remote view".
- The marker type used fore remote tracing data can be configured
through the Settings Widget in the Tracing Overlay section, option
"Remote node marker". A ring is used by default.
- Merges will now by default respect the special annotation "stable". If
a neuron is marked as such it will always win a merge and if both
merge partners are marked "stable", the merge is canceled. This can be
disabled by setting the project or instance setting "Stable join
annotation" in the Tracing section of the Settings Widget to an empty
string. The merge dialog will show a warning if stable merge
annotations have any bearing on the merge direction.
- Fast merge mode can now optionally be applied only to
neurons/fragments with a particular name pattern (regular expression).
This can be configured in the Fast Merge section of the Tracing
section in the Settings Widget.
- Alt + G will now select the closest node in the active skeleton. If no
skeleton is selected, the globally closest node is select. Regular G
works as before in the current section.
- Skeleton splitting: if a skeleton sampler was in use, a split could so
far only be successfully executed if the split node is within a
sampler domain. This is fixed now and sampled skeletons can now be
split upstream or downstream of a domain as well. If split upstream,
the domain will be deleted, because it is part of the split-off
fragment.
- The default interaction mode (skeleton tracing, synapse dropping,
select, move, import) can now be configured in the Tracing section of
the Settings Widget. This interaction mode will be enabled when
opening the Tracing tool.
Tracing overlay:
- The remote mirror CATMAID instance configuration is now taken from the
central configuration, which can be edited in the "Other CATMAID
instances" section in the Settings Widget. The "Read-only mirror
index" settings is kept but refers to list elements in the central
configuration list.
- Joining two skeletons without confirmation ("fast mode"), will now
attach an annotation to the resulting skeleton that references the
merged in skeleton by name. The name of this annotation as well as the
suggested reference visible in the confirmation dialog have the form
"Merged: <neuron-name>".
- It is now possible order loaded tracing data by skeleton size, both in
ascending and descending order. This is useful with other node count
limiting filters. The layer settings accessible through the blue/white
box in the lower left corner of a stack viewer provides the "Data
ordering" select box to change this setting. By default no ordering is
applied.
- When merging a fragment that contains a sampler, it can now be merged
if the respective skeleton doesn't need to be rerooted.
- Visibility groups can now use a generic node creator test that
references the current user (logged in our anonymous). It is called
"(self)" in the list of creators for all three visibility groups. This
is useful to define visibility groups on the project or instance level
that include a hiding pattern while guaranteeing that users see their
own nodes.
- Shift + Click on a remote node from a local node (or vice versa) will
now bring up an import preview dialog which is used as a merge preview
at the same time. Confirming the change will import the remote
skeleton and merge it at the selected location, works with virtual
nodes.
Navigator:
- Two new buttons and a text input field on the right side of the
toolbar allow easier handling of locations: the first icon (the map
marker) is a button to copy the current location of the center of the
view in project (physical) coordinates. If clicked while the Alt key
is pressed, stack (pixel) coordinates will be copied. Both modes will
copy the coordinates in the form "X, Y, Z" to the clipboard.
- The text input box allows to enter coordinates and other information.
Pressing enter or clicking the angle brackets on its right side will
make CATMAID try to interpret the text and act accordingly. The
following interpretations are attempted: 1. `Project coordinates` of
the form "X, Y, Z" and "X, Y". Parentheses and brackets are removed,
2. `Skeleton ID`, 3. `Location ID` (treenode, connector, etc.), 4.
`Neuron ID`, 5. `Bookmark letter` from the bookmark system accessible
through `;` and 6. Project coordinates from `CATMAID URL`. These are
tested in order and whatever matches first, wins.
- Additionally, particular interpretations can be enforced by using any
of the following prefixes "stack:", "px:", "neuron:", "skeleton:",
"node:", "connector:", "url:" and "bookmark:". It is possible to use
only the beginning of a prefix, i.e. "stack: 1,2,3" is the same as "st
1,2,3". This will only try the referenced parser. It also makes
stack-coordinate parsing available, which is otherwise not checked
for.
- Annotation lists show now also annotation IDs.
Administration:
- The catmaid_update_cache_tables management command accepts now a
`--jobs=n` parameter, which allows it to run in parallel when set to
values `n` larger one. By default a single process is used. This can
speed up cache generation significantly. With the help of the option
`--chunk-size=n` it is possible to define how many cells should be
processed per CPU task. For large dataset it might be useful to raise
the default of 10.
- The catmaid_update_cache_tables management command accepts now a
`--depth-step=n` parameter, which makes the update process reevaluate
the number of grid cells to look at `n` times during a single run. For
instance, if set to 2, the upper bound of cells to look at is
reevaluated when the process is through with half the depth dimension.
In larger datasets, this can speed up cache generation significantly.
- The catmaid_update_cache_tables management command accepts now an
optional ``--order`` parameter, which can be set to either "cable-asc"
or "cable-desc". By default no order is applied.
- The group list will now also show a sortable column containing the
number of group members. It also provides a more useful filtering
option on the right. Clicking "Only non-user groups" will now only
show groups with a name that isn't a username. Additionally, it is
possible to filter by a minimum and maximum number of group members.
- Settings: add CLIENT_SETTINGS and FORCE_CLIENT_SETTINGS options so
that an initial instance-wide client configuration can be specified.
- Setup: the create_configuration.py script will now create backups of
existing settings.py and django.wsgi files.
Volumes:
- Volumes have now their surface area and volume associated with them as
well as whether they are watertight. This information is visible in
the Volume Manager widget and can be recomputed on demand if needed,
using the respective button in the action column.
- The base unit used for the ara and volume columns can be adjusted
using the "Reference unit" select box in the widget controls.
Neuron Search:
- Name queries can now be marked as exact, which prevents matching
against parts of other names. This is ignored for regular expressions.
- Name queries can now be marked as case sensitive using the respective
checkbox next to the name field. By default name queries are case
insensitive.
- The search can now be further constrained to only skeletons that
include at least partially imported fragments or to skeletons that are
fully imported.
Graph widget:
- Node filters are now supported. Like in many widgets, they available
through the funnel icon in the title bar. Only nodes that are allowed
by the selected filters, can contribute connections and the graph. At
the moment, this comes with a few caveats: if filters are in use, no
subgraphs (e.g. axon/dendrite, synapse clusters) can be used. Grouping
isn't available either and in addition, skeletons are removed from the
widget if they don't comply with the filters. All this is likely going
to chagne in future versions.
- The "Nodes" tab has now three new buttons to hide, show and remove
unconnected nodes. Show will nodes visible that have been hidden
before. Removing nodes will remove the skeletons/nodes completely from
the widget.
- The node size specified in an imported .graphml file is now respected.
This is useful if graph fine tuning (layout and node size) is done in
other tools, like Gephi.
Review widget:
- The new tab "Synapse completeness" can list synaptic connector nodes
don't have a presynaptic node associated with them as well the ones
without a postsynaptic node.
- The Skeleton analytics tab now also lists low confidence nodes/edges
with a confidence lower than 4. The label of each entry shows the
respective confidence value.
Import/Export widget:
- It is now possible to import neurons from other CATMAID instances. The
"Import from CATMAID" tab provides tools to search remote skeletons by
name or annotation. Imported neurons can automatically be annotated
with a set of annotations. If no annotation are provided, CATMAID will
add the annotation "Import" to imported skeletons.
- A custom set of default annotations that will be added to imported
skeletons can be defined through the "Default skeleton import
annotations" option in the Settings Widget. At the moment, this is set
to "Import" as well as "{source} upload {group}". The second
annotation uses two optional placeholders, which are replaced by the
skeleton import source reference and the user's primary group,
respectively. Should no primary group be set, the username name is
used.
- The original ID and source URL of all imported skeletons is tracked.
This allows the widget to display local skeleton IDs, if the remote ID
has already been imported from the current source URL.
- If the active skeleton is a remote skeleton (in a remote data tracing
layer), it can now be imported by clicking the "Import active
skeleton" button in the "Import from CATMAID" tab of the widget. Like
the other import, this iwll first ask for confirmation with a 3D
dialog.
- The NRRD export can now use the JRC2018U (unisex) target template
brain space as target.
- The NRRD export can now export more than one skeleton at a time. All
skeleton sources are now displayed.
- The NRRD export will now by default create a Zip file if multiple
skeletons are exported both in a synchronous and asynchronous export.
- SWC imports can now have a user defined initial name.
Neuron navigator:
- The new button "Imported nodes" will list all imported treenodes for
the displayed skeleton.
- The detail view of an annotation will now list the time a neuron was
annotated next to the listed neuron name. The last time this link was
edited is shown as well. Both columns can be used for sorting as well.
3D viewer:
- The new shading mode named "Imported fragments" will make only
imported edges visible. The new coloring mode "Imported (green)" will
highlight all imported fragments in green.
- Stored views (View tab > Save view) are now shared between all CATMAID
instances, not only between all 3D Viewers like it is done now. This
means they are also persisted on the back-end and can be retrieved at
a later point.
- 3D viewer: the line width has now an upper limit of 25px to prevent
crashes of the 3D viewer with larger values and many skeletons.
- Desmosome links can now be displayed. The color assigned in the
Settings Widget section for desmosomes in the Tracing Overlay is used
here as well.
Neuroglancer widget:
- CATMAID now includes a copy of Neuroglancer that can be accessed at
<catmaid_base_url>/neuroglancer
- The Neurglancer widget allows to open an instance of Neuroglancer in a
new window and its navigation (center location, zoom level) is
synchronized with the navigation happening in CATMAID. An URL to a
Neuroglancer instance containing the same dataset as shown in the
CATMAID stack viewer needs to be provided.
- The widget's info page has a step-by-step guide on how to use it for
the FAFB dataset
Treenode table:
- The UI elements to select active filter options like node type,
confidence or confidence operator now havea a colored background if an
active filter is selected. This is useful to draw more attention to
the fact that leaf nodes are selected by default.
- Remote skeletons are now supported. They can be mixed with local
skeletons.
- A new column "i" represents whether or not a particular treenode was
imported. This information is loaded separately, parallel to the
regular information. Which, is why initially "…" is shown. If a node
wasn't imported, the field is empty. If it was imported, "Y" is shown.
Morphology plot:
- Two new center computation modes have been added: Tagged node closest
to root, Tagged node most distant from root. Both use the tag input
field to the right of the center mode selector. Either pressing enter
or Redraw after a tag change will recompute the plot. If no node is
found with the respective tag, the root is used and a warning is
displayed.
- It is now possible to select the interpolation mode of the plot. So
far the plotted data was always interpolated with a basis spline. This
isn't necessarily what is required for a particular plot, e.g. if the
sample values have to be intersected. To allow this, a drop-down menu
with the following interpolation options has been added: basis spline,
linear, cardinal spline, monotone, step before and step after.
Spaces:
- Users with the new "can_fork" permission have a new user menu entry in
the upper right corner: "Create own space". This allows to create a
personal copy of a project, without any tracing data.
Miscellaneous:
- The active project title is now shown in the window title.
- There is a new menu item named "User info" in the menu displayed when
hovering the mouse cursor over the user's name in the upper right
corner. When clicked, it will show the basic information on a user:
name, username, user id, primary group and color.
- Projects can now be kept open if all last stack-viewers are closed. To
not automatically close the open project with the last stack-viewer,
uncheck the "Project closes with last stack viewer" option in the
Settings Widget.
- Neuron similarity widget: besides a regular arithmetic mean, the
geometric mean is now an additional option for how to combine
(normalized) forward and reverse scores.
- Neuron similarity detail widget: make CSV export of object IDs and
names configurable.
- Retrieving review information is now faster for bigger skeletons.
- Name search: the speed of search queries is now improved.
- Selection table: the summary info button now shows also the number of
imported nodes within the selected skeletons.
- Neuron search: search fields will now expand if their content is
bigger than the default size. Also copied fields don't contain the
content of the source field anymore.
- A CATMAID deep-link URL can now include the layout that should be
displayed using the `layout` parameter. It takes a layout
specification as it is used in the Settings Widget and the Layout
menu.
- The speed of bulk operations on skeletons has changed (e.g. splitting,
joins, reroot).
- Settings widget: it is now possible to clear both front-end and
back-end settings through the "Clear settings" button. It will ask for
confirmation and whether both front-end and back-end settings should
be cleared.
- The top toolbar of the web UI won't add line breaks anymore if only
little horizontal space is available. The part of the toolbar that
contains the widget buttons for the selected tool will shrink instead.
Only elements that fit the available width are displayed there now.
Invisible items are not a problem, because all widgets can be reached
through the "Open Widget" dialog or Ctrl + Space.
- Potential WebGL performance problems are now detected when stacks and
stack groups are loaded. If problems are expected by the browser, a
warning is shown.
- Spatial update worker: cache cells are not queued for updates anymore
if the source project of a spatial event doesn't have any caches.
- Data views: add option "show_empty_projects" to simple project list
data view. It is false by default and can be turned on to show empty
projects, including the classification dummy project.
- Project statistics: clicking the "User analytics" button with user
selected will open the respective user analytics views for all of them
(max. the first three).
Bug fixes
---------
- Connector list: dates are now display correctly.
- Tracing tool: don't allow moving nodes in select-only mode.
- Tracing layer: merged-in single nodes will now also have their name
remembered as a new annotation in a join if their name doesn't follow
the default pattern "neuron <number>".
- Project importer: the UNIX "glob" filter pattern that can be used to
filter imported projects is now also applied to imports from URLs and
files.
- Nblast widget: store tangent neighbor count specified in UI for
scoring matrix imported from CSV.
- Celery can now be restarted again without restarting all of CATMAID.
- Connectivity widget: fix error showing when doing many subsequent
Refresh button clicks.
- 3D viewer: the handling of intersecting transparent objects has
improved and works like expected in most situations.
- Synapse plot: a missing skeleton source interface method was added,
making it now possible to also remove skeletons from the widget.
- Synapse plot: skeletons can now be added to the widget again. This was
temporarily broken for CATMAID instances using a sub-folder in their
URL.
- Management command "catmaid_find_node_provider_config" to generate
NODE_PROVIDERS configurations for the `settings.py` file:The default
value for the "--orientation" parameter is now "xy". It was
accidentally "xz" before.
- Graph widget: pressing the Delete key with a node selected in an
active Graph Widget will now ask for the removal of the node from the
graph, not anymore whether the active node should be deleted
completely.
- Graph widget: showing fractions of inbound/outbound fractions of
connections as edge labels works again.
- Graph widget: refreshing a graph with undirected edges (e.g.
sub-graphs like axon/dendrite split, desmosomes, etc.) doesn't lead to
suddenly appearing arrow heads.
- Graph widget: undirected links like desmosomes will now only appear
once between two nodes. They would be shown twice before, once for
each of the partners being the "source" or "target".
- Graph widget: apply currently selected layout after graph update. This
fixes occasional fall-backs to the force-directed layout.
- SWC skeleton import: if `force = true`, treenodes of a replaced
skeleton are now correctly removed on a successful import.
- CLI Importer: creating unknown users manually works now as expected.
- Node filters: the application of node filters with rules applied in an
intersected fashion is fixed now and the filtered result of all
skeletons is shown.
- Spatial update notifications: only try to update the database
configuration if the database isn't running as a database replica.
- Synapse plot: fix read-out of settings input fields.
- Connectivity matrix: don't raise an error in "fraction" display mode
on a widget refresh (e.g. caused by a neuron name update).
- Connectivity matrix: don't show error when enabling "fractions"
display without neurons being loaded.
- Connectivity widget: fix error caused by quickly refreshing the table
successively.
- Graph widget: fix error caused by quickly refreshing the graph successively.
- Neuron navigator: fix name display after rename of neuron.
- Neuron search: the tab key focus order within one widget is now
maintained if multiple search windows are open.
- Neuron search: sort results ascending by name by default and keep the
sort order through table redraws.
- Settings widget: the width of multiline select elements (e.g.
annotation based neuron naming) is now adjusted automatically to fit
the content.
- 3D Viewer: fix skeleton vs. name CSV in "Synapses as CSV" export.
- 3D Viewer: fix accidental hiding of soma node sphere when hiding
radius meshes. This happened if the node tagged 'soma' wasn't the root
node. Now the test for whether to hide radius meshes for a node now
requires that the node isn't tagged with 'soma', which is more robust
than root node alone.
- Measurement table: fix CSV export for aggregated multi-fragment results.
API changes
-----------
Additions:
- GET `/{project_id}/stats/cable-length`:
Accepts now an optional 'name_pattern' argument that allows case
insensitive pattern matching in the respective neuron name. If a '/'
prefix is used, the pattern is interpreted as a regular expression.
- GET|POST `/{project_id}/skeletons/validity`:
Obtain a list of valid or invalid skeletons in a set pf passed in
skeleton IDs.
- `GET|POST /{project_id}/skeletons/from-origin`:
Looks up skeleton IDs based on a set of `source_ids`, a `source_url`
and a `source_project_id`..
- `GET|POST /{project_id}/skeletons/origin`:
Gets origin information on a list of `skeleton_ids`.
- `GET|POST /{project_id}/skeletons/import-info`:
Gets information on which and how many nodes are imported in a set of
skeletons.
- `POST /{project_id}/annotations/query-targets`:
Accepts now an optional boolean "with_timestamps" parameter, which is
set to false by default. If set to true, the returned annotations will
include their creation_time and edition_time.
- `POST /{project_id}/annotations/query-targets`:
Accepts now two additional sorting options: annotated_on,
last_annotation_link_edit. The first one sorts by the date the matched
annotations were added to the target. The second option sorts by when
this link was edited last.
- GET|POST `/{project_id}/skeletons/completeness`:
Obtain completeness information on a list of skeletons.
- `POST /{project_id}/skeletons/{skeleton_id}/find-labels`:
The new optional boolean `only_leaves` parameter (default false) can
be used to return only tagged leaf nodes (including root).
Modifications:
- All APIs: errors during a request are now indicated with a more useful
HTTP status code than 200: errors related to the request, input data
and the client will result in status 400, permission errors in status
403, unavailable resources in status 404 and internal server errors in
status 500.
- `POST|GET /{project_id}/node/list`:
Offers a new optional parameter "ordering", which can be used to order
the result set of nodes. The values cable-asc and cable-desc are
allowed. By default no ordering is applied.
- `POST|GET /{project_id}/volumes/`:
Information on area, volume and watertightness is now returned for
each volume as well as information on whether these details have been
computed (meta_computed). This meta data can be computed for each
volume using the `{project_id}/volumes/{volume_id}/update-meta-info`
API.
- `GET /{project_id}/volumes/{volume_id}/update-meta-info`:
Try to compute area, volume and watertightness for a volume. Sets
`meta_computed` to true. If this operation fails, all meta data fields
are set to NULL/None.
- `POST /{project_id}/annotations/query-targets`:
Accepts now an optional boolean "name_exact" parameter, which is set
to false by default. If set to true, the passed in name has to match
exactly. This is faster than using a regular expression like '/^name$'
for exact matches.
- `POST /{project_id}/skeletons/import`:
The new parameters `source_id` and `source_url` allow to associate a
source data reference with the imported skeleton. The new list
parameter "annotations" can have annotation names, which are added to
the import skeleton. By default only the annotation "Import" is set.
- `POST /{project_id}/skeleton/join`:
The new parameter `lose_sampler_handling` controls how to deal with
samplers defined for the merged in skeleton. Possible values are
'delete-sampler' and 'keep-sampler', the latter being the default. If
samplers should be kept, it is required that the sampler domains are
defined on parts of the skeleton that don't need local rerooting.
Otherwise an error is raised.
Deprecations:
None.
Removals:
None.
We just released a new CATMAID version, 2020.02.15:
https://github.com/catmaid/CATMAID/releases/tag/2020.02.15
You will find a list of changes as we well as some update notes at the
end of this email and on the website linked above. As always, please let
us know if you run into any problems, both as administrator and user.
The docker images tagged latest, stable and 2020.02.15 have been
(re)built.
If you update a CATMAID instance, please use the most recent commit of
the master branch in our GitHub repository, which includes all future
bug fixes for this release:
https://github.com/catmaid/CATMAID
If you are administering a CATMAID server, the "Notes" section provides
information on how to migrate to the new version. The file UPDATE.md
contains a copy of the migration steps as well. Please also note that
this CATMAID version drops support for Python 3.5. Use Python 3.6 or
newer. This release requires also Postgres 11+ and PostGIS 2.5+. If you
have to update your database, please make sure to update to the target
version of PostGIS in all databases *before* you update the database
cluster, it will make the update go more smoothly. This means installing
the new PostGIS version (e.g. v3), connecting to each database and
running `ALTER EXTENSION postgis UPDATE` (check version using \dx). The
same database version needs to be available for the target version of
Postgres. If you need to update your database, go to Postgres 12
directly, because the next version of CATMAID will require Postgres 12
and PostGIS 3.
Cheers,
Tom
2020.02.15
==========
Contributors: Chris Barnes, Andrew Champion, Stephan Gerhard,
Pat Gunn, Tom Kazimiers
Notes
-----
- Python 3.5 is not supported anymore. Use Python 3.6, 3.7 or 3.8.
- Postgres 11 and PostGIS 2.5 is required, Postgres 12 and PostGIS 3 is
recommended. If Postgres needs to be updated, update directly to
Postgres 12. If both needs to be updated, update PostGIS first and
run ``ALTER EXTENSION postgis UPDATE;`` in every database. For
docker-compose setups this database update is performed automatically.
If a replication setup is in use, the database configuration changes
for Postgres 12. CATMAID's replication documentation explains what
needs to be done.
- A virtualenv update is required. Before you start it, please remove
some packages that are not needed anymore first:
pip uninstall asgi-ipc asgi-rabbitmq
- If ASGI was set up before, make sure to install channels_rabbitmq or
channels_redis (depending on what yous use). The older asgi_rabbitmq
and asgi_redis packages aren't supported anymore. This also requires
an update of the CHANNELS_LAYERS in settings.py. The channels_rabbitmq
documentation for an example:
https://github.com/CJWorkbench/channels_rabbitmq/. This variable isn't
defined by default anymore. Therefore you likely have to replace any
`CHANNELS_LAYERS[…] = …` with something like `CHANNELS_LAYERS = { …
}`. The new format is (use custom credentials on any production
system!):
```
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_rabbitmq.core.RabbitmqChannelLayer",
"CONFIG": {
"host": "amqp://guest:gu...@127.0.0.1/asgi",
},
},
}
```
Also, if supervisord is in use to manage CATMAID process groups, the
main Daphne process needs an adjustment: instead of calling `daphne`
with the `mysite.asgi:channel_layer` parameter, use
`mysite.asgi:application`. A complete supervisord entry would then
look something like this:
```
[program:catmaid-daphne]
directory = /home/catmaid/catmaid/django/projects/
command = /home/catmaid/catmaid/django/env/bin/daphne --unix-socket=/var/run/daphne/catmaid.sock mysite.asgi:application
user = www-data
stdout_logfile = /var/log/daphne/catmaid-server.log
redirect_stderr = true
```
As last step, the supervisor entry for the `daphne worker` process has
to be removed. New types of workers can be added, but are not needed
in most cases. The channels documentation has more information on
this.
- GDAL v2 or newer is now needed. If your Ubuntu version doesn't support
this yet, there is an official PPA:
sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt-get update
- The next version of CATMAID will require Postgres 12 and PostGIS 3.
- The management command catmaid_populate_summary_tables is now known as
catmaid_refresh_node_statistics.
- When enabling and disabling both history tracking and spatial update
events on startup, advisory locks are now used. This should make the
startup of parallel CATMAID workers more robust.
- The application of migrations 88-91 and 98-99 might take a while to
complete, because they rewrite potentially big database table
(treenode_edge, treenode, class_instance, and more). Therefore, make
also sure that there is enough space available at the database storage
location (25% of database data directory should be plenty). If no
replication is used, setting the following Postgres options can speed
up the process: `wal_level = minimal`, `archive_mode = off` and
`max_wal_senders = 0`.
Due to this database update data consistency and correctness was
improved, because additional foreign key relationships have been added
that were missing before.
- Both `configuration.py.example` and `create_configuration.py` support
now the option `catmaid_default_enabled_tools`, which defines the list
of front-end tools that are enabled by default for new users. The
following options are supported: cropping, tagging, textlabel,
tracing, ontology and roi. By default the tracing tool is now enabled
for new users, because it seems in many setups users want to have this
enabled by default.
- Back-end errors result now in actual HTTP error status codes.
Third-party clients need possibly some adjustments to handle API
errors. In case of an error, status 400 is returned if an input data
or parameter problem, 401 for permission problems and 500 otherwise.
- On startup, CATMAID will now test if the output directory is
accessible as well as if its expected folder layout is present. If
expected subfolders are missing, they will now be created.
- Python code is now linted by flake8. The config is fairly relaxed (see
.travis.flake8), but contributions which do not conform with the
selected rules will fail CI checks.
Features and enhancements
-------------------------
Permissions:
- If data is imported by a user, e.g. from a remote CATMAID instance,
this user has all permissions over the imported data. If data was
imported or created by someone else, the regular rules apply:
permissions are granted if the user is creator, superuser or has
permission over the data's creator. If user A has permissions over
user B's data, user A will also have permission to edit user B's
imported skeletons. This is done in order to be able to maintain the
original creator and editor data of the imported data, should it be
available.
Node filters:
- Binary split: a new toggle labeled "inclusive" allows to decide
whether to include the split not in an upstream sub-arbor (default:
true).
CLI importer:
- Performance improvement: only the skeleton summary entries of the
imported skeletons are now created.
- Only update treenode edges and skeleton summary if treenodes were
actually imported (rather than e.g. only volumes).
- Add --auto-name-unknown-users option, to automatically generate names
for users that are not available from an import, but referenced in it
(by ID).
CLI exporter:
- The new --annotation-annotation option allows to specify a
meta-annotation that all exported annotations need to share
transitively. If for instance all all top-level annotations that
should be exported are annotated with "exportable" and
"--annotation-annotation exportable" is used, only annotations from
those annotation hierarchies marked as "exportable" will be exported.
Without this option, all annotations are exported.
- If users should be exported, also export users of volumes.
Project statistics:
- Requests to the back-end our now performed in parallel, resulting in
faster loading times.
- It is now possible to configure how many largest neurons should be
displayed as well as an optional name pattern.
- The user/time action table will now also list imported nodes and
imported cable length. These values are subtracted from the
corresponding regular counters. If "Include imports" is unchecked, the
same information that was displayed before this commit is shown, i.e.
the regular node count and cable length field include the imported
data.
Tracing tool:
- The tool bar button to refresh caches has been replaced with a button
to open a context menu titled "More tools". The context menu has an
entry to refresh cashes and a list of tracing data in other projects.
- Tracing data from other CATMAID instances (or other local projects)
can now be added to the active stack viewer by selecting the
respective remote project from the "More tools" menu item "Remote
data".
- To open a remote CATMAID instance at the same location as the current
view, find the remote project in the "Remote data" section of the
"More tools" menu, and click on "Open remote view".
- The marker type used fore remote tracing data can be configured
through the Settings Widget in the Tracing Overlay section, option
"Remote node marker". A ring is used by default.
- Merges will now by default respect the special annotation "stable". If
a neuron is marked as such it will always win a merge and if both
merge partners are marked "stable", the merge is canceled. This can be
disabled by setting the project or instance setting "Stable join
annotation" in the Tracing section of the Settings Widget to an empty
string. The merge dialog will show a warning if stable merge
annotations have any bearing on the merge direction.
- Fast merge mode can now optionally be applied only to
neurons/fragments with a particular name pattern (regular expression).
This can be configured in the Fast Merge section of the Tracing
section in the Settings Widget.
- Alt + G will now select the closest node in the active skeleton. If no
skeleton is selected, the globally closest node is select. Regular G
works as before in the current section.
- Skeleton splitting: if a skeleton sampler was in use, a split could so
far only be successfully executed if the split node is within a
sampler domain. This is fixed now and sampled skeletons can now be
split upstream or downstream of a domain as well. If split upstream,
the domain will be deleted, because it is part of the split-off
fragment.
- The default interaction mode (skeleton tracing, synapse dropping,
select, move, import) can now be configured in the Tracing section of
the Settings Widget. This interaction mode will be enabled when
opening the Tracing tool.
Tracing overlay:
- The remote mirror CATMAID instance configuration is now taken from the
central configuration, which can be edited in the "Other CATMAID
instances" section in the Settings Widget. The "Read-only mirror
index" settings is kept but refers to list elements in the central
configuration list.
- Joining two skeletons without confirmation ("fast mode"), will now
attach an annotation to the resulting skeleton that references the
merged in skeleton by name. The name of this annotation as well as the
suggested reference visible in the confirmation dialog have the form
"Merged: <neuron-name>".
- It is now possible order loaded tracing data by skeleton size, both in
ascending and descending order. This is useful with other node count
limiting filters. The layer settings accessible through the blue/white
box in the lower left corner of a stack viewer provides the "Data
ordering" select box to change this setting. By default no ordering is
applied.
- When merging a fragment that contains a sampler, it can now be merged
if the respective skeleton doesn't need to be rerooted.
- Visibility groups can now use a generic node creator test that
references the current user (logged in our anonymous). It is called
"(self)" in the list of creators for all three visibility groups. This
is useful to define visibility groups on the project or instance level
that include a hiding pattern while guaranteeing that users see their
own nodes.
- Shift + Click on a remote node from a local node (or vice versa) will
now bring up an import preview dialog which is used as a merge preview
at the same time. Confirming the change will import the remote
skeleton and merge it at the selected location, works with virtual
nodes.
Navigator:
- Two new buttons and a text input field on the right side of the
toolbar allow easier handling of locations: the first icon (the map
marker) is a button to copy the current location of the center of the
view in project (physical) coordinates. If clicked while the Alt key
is pressed, stack (pixel) coordinates will be copied. Both modes will
copy the coordinates in the form "X, Y, Z" to the clipboard.
- The text input box allows to enter coordinates and other information.
Pressing enter or clicking the angle brackets on its right side will
make CATMAID try to interpret the text and act accordingly. The
following interpretations are attempted: 1. `Project coordinates` of
the form "X, Y, Z" and "X, Y". Parentheses and brackets are removed,
2. `Skeleton ID`, 3. `Location ID` (treenode, connector, etc.), 4.
`Neuron ID`, 5. `Bookmark letter` from the bookmark system accessible
through `;` and 6. Project coordinates from `CATMAID URL`. These are
tested in order and whatever matches first, wins.
- Additionally, particular interpretations can be enforced by using any
of the following prefixes "stack:", "px:", "neuron:", "skeleton:",
"node:", "connector:", "url:" and "bookmark:". It is possible to use
only the beginning of a prefix, i.e. "stack: 1,2,3" is the same as "st
1,2,3". This will only try the referenced parser. It also makes
stack-coordinate parsing available, which is otherwise not checked
for.
- Annotation lists show now also annotation IDs.
Administration:
- The catmaid_update_cache_tables management command accepts now a
`--jobs=n` parameter, which allows it to run in parallel when set to
values `n` larger one. By default a single process is used. This can
speed up cache generation significantly. With the help of the option
`--chunk-size=n` it is possible to define how many cells should be
processed per CPU task. For large dataset it might be useful to raise
the default of 10.
- The catmaid_update_cache_tables management command accepts now a
`--depth-step=n` parameter, which makes the update process reevaluate
the number of grid cells to look at `n` times during a single run. For
instance, if set to 2, the upper bound of cells to look at is
reevaluated when the process is through with half the depth dimension.
In larger datasets, this can speed up cache generation significantly.
- The catmaid_update_cache_tables management command accepts now an
optional ``--order`` parameter, which can be set to either "cable-asc"
or "cable-desc". By default no order is applied.
- The group list will now also show a sortable column containing the
number of group members. It also provides a more useful filtering
option on the right. Clicking "Only non-user groups" will now only
show groups with a name that isn't a username. Additionally, it is
possible to filter by a minimum and maximum number of group members.
- Settings: add CLIENT_SETTINGS and FORCE_CLIENT_SETTINGS options so
that an initial instance-wide client configuration can be specified.
- Setup: the create_configuration.py script will now create backups of
existing settings.py and django.wsgi files.
Volumes:
- Volumes have now their surface area and volume associated with them as
well as whether they are watertight. This information is visible in
the Volume Manager widget and can be recomputed on demand if needed,
using the respective button in the action column.
- The base unit used for the ara and volume columns can be adjusted
using the "Reference unit" select box in the widget controls.
Neuron Search:
- Name queries can now be marked as exact, which prevents matching
against parts of other names. This is ignored for regular expressions.
- Name queries can now be marked as case sensitive using the respective
checkbox next to the name field. By default name queries are case
insensitive.
- The search can now be further constrained to only skeletons that
include at least partially imported fragments or to skeletons that are
fully imported.
Graph widget:
- Node filters are now supported. Like in many widgets, they available
through the funnel icon in the title bar. Only nodes that are allowed
by the selected filters, can contribute connections and the graph. At
the moment, this comes with a few caveats: if filters are in use, no
subgraphs (e.g. axon/dendrite, synapse clusters) can be used. Grouping
isn't available either and in addition, skeletons are removed from the
widget if they don't comply with the filters. All this is likely going
to chagne in future versions.
- The "Nodes" tab has now three new buttons to hide, show and remove
unconnected nodes. Show will nodes visible that have been hidden
before. Removing nodes will remove the skeletons/nodes completely from
the widget.
- The node size specified in an imported .graphml file is now respected.
This is useful if graph fine tuning (layout and node size) is done in
other tools, like Gephi.
Review widget:
- The new tab "Synapse completeness" can list synaptic connector nodes
don't have a presynaptic node associated with them as well the ones
without a postsynaptic node.
- The Skeleton analytics tab now also lists low confidence nodes/edges
with a confidence lower than 4. The label of each entry shows the
respective confidence value.
Import/Export widget:
- It is now possible to import neurons from other CATMAID instances. The
"Import from CATMAID" tab provides tools to search remote skeletons by
name or annotation. Imported neurons can automatically be annotated
with a set of annotations. If no annotation are provided, CATMAID will
add the annotation "Import" to imported skeletons.
- A custom set of default annotations that will be added to imported
skeletons can be defined through the "Default skeleton import
annotations" option in the Settings Widget. At the moment, this is set
to "Import" as well as "{source} upload {group}". The second
annotation uses two optional placeholders, which are replaced by the
skeleton import source reference and the user's primary group,
respectively. Should no primary group be set, the username name is
used.
- The original ID and source URL of all imported skeletons is tracked.
This allows the widget to display local skeleton IDs, if the remote ID
has already been imported from the current source URL.
- If the active skeleton is a remote skeleton (in a remote data tracing
layer), it can now be imported by clicking the "Import active
skeleton" button in the "Import from CATMAID" tab of the widget. Like
the other import, this iwll first ask for confirmation with a 3D
dialog.
- The NRRD export can now use the JRC2018U (unisex) target template
brain space as target.
- The NRRD export can now export more than one skeleton at a time. All
skeleton sources are now displayed.
- The NRRD export will now by default create a Zip file if multiple
skeletons are exported both in a synchronous and asynchronous export.
- SWC imports can now have a user defined initial name.
Neuron navigator:
- The new button "Imported nodes" will list all imported treenodes for
the displayed skeleton.
- The detail view of an annotation will now list the time a neuron was
annotated next to the listed neuron name. The last time this link was
edited is shown as well. Both columns can be used for sorting as well.
3D viewer:
- The new shading mode named "Imported fragments" will make only
imported edges visible. The new coloring mode "Imported (green)" will
highlight all imported fragments in green.
- Stored views (View tab > Save view) are now shared between all CATMAID
instances, not only between all 3D Viewers like it is done now. This
means they are also persisted on the back-end and can be retrieved at
a later point.
- 3D viewer: the line width has now an upper limit of 25px to prevent
crashes of the 3D viewer with larger values and many skeletons.
- Desmosome links can now be displayed. The color assigned in the
Settings Widget section for desmosomes in the Tracing Overlay is used
here as well.
Neuroglancer widget:
- CATMAID now includes a copy of Neuroglancer that can be accessed at
<catmaid_base_url>/neuroglancer
- The Neurglancer widget allows to open an instance of Neuroglancer in a
new window and its navigation (center location, zoom level) is
synchronized with the navigation happening in CATMAID. An URL to a
Neuroglancer instance containing the same dataset as shown in the
CATMAID stack viewer needs to be provided.
- The widget's info page has a step-by-step guide on how to use it for
the FAFB dataset
Treenode table:
- The UI elements to select active filter options like node type,
confidence or confidence operator now havea a colored background if an
active filter is selected. This is useful to draw more attention to
the fact that leaf nodes are selected by default.
- Remote skeletons are now supported. They can be mixed with local
skeletons.
- A new column "i" represents whether or not a particular treenode was
imported. This information is loaded separately, parallel to the
regular information. Which, is why initially "…" is shown. If a node
wasn't imported, the field is empty. If it was imported, "Y" is shown.
Morphology plot:
- Two new center computation modes have been added: Tagged node closest
to root, Tagged node most distant from root. Both use the tag input
field to the right of the center mode selector. Either pressing enter
or Redraw after a tag change will recompute the plot. If no node is
found with the respective tag, the root is used and a warning is
displayed.
- It is now possible to select the interpolation mode of the plot. So
far the plotted data was always interpolated with a basis spline. This
isn't necessarily what is required for a particular plot, e.g. if the
sample values have to be intersected. To allow this, a drop-down menu
with the following interpolation options has been added: basis spline,
linear, cardinal spline, monotone, step before and step after.
Spaces:
- Users with the new "can_fork" permission have a new user menu entry in
the upper right corner: "Create own space". This allows to create a
personal copy of a project, without any tracing data.
Miscellaneous:
- The active project title is now shown in the window title.
- There is a new menu item named "User info" in the menu displayed when
hovering the mouse cursor over the user's name in the upper right
corner. When clicked, it will show the basic information on a user:
name, username, user id, primary group and color.
- Projects can now be kept open if all last stack-viewers are closed. To
not automatically close the open project with the last stack-viewer,
uncheck the "Project closes with last stack viewer" option in the
Settings Widget.
- Neuron similarity widget: besides a regular arithmetic mean, the
geometric mean is now an additional option for how to combine
(normalized) forward and reverse scores.
- Neuron similarity detail widget: make CSV export of object IDs and
names configurable.
- Retrieving review information is now faster for bigger skeletons.
- Name search: the speed of search queries is now improved.
- Selection table: the summary info button now shows also the number of
imported nodes within the selected skeletons.
- Neuron search: search fields will now expand if their content is
bigger than the default size. Also copied fields don't contain the
content of the source field anymore.
- A CATMAID deep-link URL can now include the layout that should be
displayed using the `layout` parameter. It takes a layout
specification as it is used in the Settings Widget and the Layout
menu.
- The speed of bulk operations on skeletons has changed (e.g. splitting,
joins, reroot).
- Settings widget: it is now possible to clear both front-end and
back-end settings through the "Clear settings" button. It will ask for
confirmation and whether both front-end and back-end settings should
be cleared.
- The top toolbar of the web UI won't add line breaks anymore if only
little horizontal space is available. The part of the toolbar that
contains the widget buttons for the selected tool will shrink instead.
Only elements that fit the available width are displayed there now.
Invisible items are not a problem, because all widgets can be reached
through the "Open Widget" dialog or Ctrl + Space.
- Potential WebGL performance problems are now detected when stacks and
stack groups are loaded. If problems are expected by the browser, a
warning is shown.
- Spatial update worker: cache cells are not queued for updates anymore
if the source project of a spatial event doesn't have any caches.
- Data views: add option "show_empty_projects" to simple project list
data view. It is false by default and can be turned on to show empty
projects, including the classification dummy project.
- Project statistics: clicking the "User analytics" button with user
selected will open the respective user analytics views for all of them
(max. the first three).
Bug fixes
---------
- Connector list: dates are now display correctly.
- Tracing tool: don't allow moving nodes in select-only mode.
- Tracing layer: merged-in single nodes will now also have their name
remembered as a new annotation in a join if their name doesn't follow
the default pattern "neuron <number>".
- Project importer: the UNIX "glob" filter pattern that can be used to
filter imported projects is now also applied to imports from URLs and
files.
- Nblast widget: store tangent neighbor count specified in UI for
scoring matrix imported from CSV.
- Celery can now be restarted again without restarting all of CATMAID.
- Connectivity widget: fix error showing when doing many subsequent
Refresh button clicks.
- 3D viewer: the handling of intersecting transparent objects has
improved and works like expected in most situations.
- Synapse plot: a missing skeleton source interface method was added,
making it now possible to also remove skeletons from the widget.
- Synapse plot: skeletons can now be added to the widget again. This was
temporarily broken for CATMAID instances using a sub-folder in their
URL.
- Management command "catmaid_find_node_provider_config" to generate
NODE_PROVIDERS configurations for the `settings.py` file:The default
value for the "--orientation" parameter is now "xy". It was
accidentally "xz" before.
- Graph widget: pressing the Delete key with a node selected in an
active Graph Widget will now ask for the removal of the node from the
graph, not anymore whether the active node should be deleted
completely.
- Graph widget: showing fractions of inbound/outbound fractions of
connections as edge labels works again.
- Graph widget: refreshing a graph with undirected edges (e.g.
sub-graphs like axon/dendrite split, desmosomes, etc.) doesn't lead to
suddenly appearing arrow heads.
- Graph widget: undirected links like desmosomes will now only appear
once between two nodes. They would be shown twice before, once for
each of the partners being the "source" or "target".
- Graph widget: apply currently selected layout after graph update. This
fixes occasional fall-backs to the force-directed layout.
- SWC skeleton import: if `force = true`, treenodes of a replaced
skeleton are now correctly removed on a successful import.
- CLI Importer: creating unknown users manually works now as expected.
- Node filters: the application of node filters with rules applied in an
intersected fashion is fixed now and the filtered result of all
skeletons is shown.
- Spatial update notifications: only try to update the database
configuration if the database isn't running as a database replica.
- Synapse plot: fix read-out of settings input fields.
- Connectivity matrix: don't raise an error in "fraction" display mode
on a widget refresh (e.g. caused by a neuron name update).
- Connectivity matrix: don't show error when enabling "fractions"
display without neurons being loaded.
- Connectivity widget: fix error caused by quickly refreshing the table
successively.
- Graph widget: fix error caused by quickly refreshing the graph successively.
- Neuron navigator: fix name display after rename of neuron.
- Neuron search: the tab key focus order within one widget is now
maintained if multiple search windows are open.
- Neuron search: sort results ascending by name by default and keep the
sort order through table redraws.
- Settings widget: the width of multiline select elements (e.g.
annotation based neuron naming) is now adjusted automatically to fit
the content.
- 3D Viewer: fix skeleton vs. name CSV in "Synapses as CSV" export.
- 3D Viewer: fix accidental hiding of soma node sphere when hiding
radius meshes. This happened if the node tagged 'soma' wasn't the root
node. Now the test for whether to hide radius meshes for a node now
requires that the node isn't tagged with 'soma', which is more robust
than root node alone.
- Measurement table: fix CSV export for aggregated multi-fragment results.
API changes
-----------
Additions:
- GET `/{project_id}/stats/cable-length`:
Accepts now an optional 'name_pattern' argument that allows case
insensitive pattern matching in the respective neuron name. If a '/'
prefix is used, the pattern is interpreted as a regular expression.
- GET|POST `/{project_id}/skeletons/validity`:
Obtain a list of valid or invalid skeletons in a set pf passed in
skeleton IDs.
- `GET|POST /{project_id}/skeletons/from-origin`:
Looks up skeleton IDs based on a set of `source_ids`, a `source_url`
and a `source_project_id`..
- `GET|POST /{project_id}/skeletons/origin`:
Gets origin information on a list of `skeleton_ids`.
- `GET|POST /{project_id}/skeletons/import-info`:
Gets information on which and how many nodes are imported in a set of
skeletons.
- `POST /{project_id}/annotations/query-targets`:
Accepts now an optional boolean "with_timestamps" parameter, which is
set to false by default. If set to true, the returned annotations will
include their creation_time and edition_time.
- `POST /{project_id}/annotations/query-targets`:
Accepts now two additional sorting options: annotated_on,
last_annotation_link_edit. The first one sorts by the date the matched
annotations were added to the target. The second option sorts by when
this link was edited last.
- GET|POST `/{project_id}/skeletons/completeness`:
Obtain completeness information on a list of skeletons.
- `POST /{project_id}/skeletons/{skeleton_id}/find-labels`:
The new optional boolean `only_leaves` parameter (default false) can
be used to return only tagged leaf nodes (including root).
Modifications:
- All APIs: errors during a request are now indicated with a more useful
HTTP status code than 200: errors related to the request, input data
and the client will result in status 400, permission errors in status
403, unavailable resources in status 404 and internal server errors in
status 500.
- `POST|GET /{project_id}/node/list`:
Offers a new optional parameter "ordering", which can be used to order
the result set of nodes. The values cable-asc and cable-desc are
allowed. By default no ordering is applied.
- `POST|GET /{project_id}/volumes/`:
Information on area, volume and watertightness is now returned for
each volume as well as information on whether these details have been
computed (meta_computed). This meta data can be computed for each
volume using the `{project_id}/volumes/{volume_id}/update-meta-info`
API.
- `GET /{project_id}/volumes/{volume_id}/update-meta-info`:
Try to compute area, volume and watertightness for a volume. Sets
`meta_computed` to true. If this operation fails, all meta data fields
are set to NULL/None.
- `POST /{project_id}/annotations/query-targets`:
Accepts now an optional boolean "name_exact" parameter, which is set
to false by default. If set to true, the passed in name has to match
exactly. This is faster than using a regular expression like '/^name$'
for exact matches.
- `POST /{project_id}/skeletons/import`:
The new parameters `source_id` and `source_url` allow to associate a
source data reference with the imported skeleton. The new list
parameter "annotations" can have annotation names, which are added to
the import skeleton. By default only the annotation "Import" is set.
- `POST /{project_id}/skeleton/join`:
The new parameter `lose_sampler_handling` controls how to deal with
samplers defined for the merged in skeleton. Possible values are
'delete-sampler' and 'keep-sampler', the latter being the default. If
samplers should be kept, it is required that the sampler domains are
defined on parts of the skeleton that don't need local rerooting.
Otherwise an error is raised.
Deprecations:
None.
Removals:
None.
Reply all
Reply to author
Forward
0 new messages