trouble in loading flywire data.

[Li Guo]

Hi all,

I am trying to load the neuron from flywire and plot it in natverse. 

I have installed natverse, fafbseg in R studio, cloud volume in python, below are my codes:

library(natverse)

library(fafbseg)

gf.flywire=read_cloudvolume_meshes("720575940615763227")

First time, the error was reported as following:

 

Loading required namespace: readobj

  downloading meshes

Loading required namespace: reticulate

Error: Please install python cloudvolume module as described at:

https://github.com/seung-lab/cloud-volume#setup

This should normally work:

pip3 install cloud-volume

If you have already installed cloudvolume but it is not found

then do:

usethis::edit_r_environ()

 to point to the right python

e.g. RETICULATE_PYTHON="/usr/local/bin/python3"

Since I have installed the cloudvolume, so I used the second suggestion and do the modification to the .Renviron, which now is 

GITHUB_PAT=268f4d2ffcd68aa225c8c3eab47594cde8642518

RETICULATE_PYTHON="/Library/Frameworks/Python.framework/Versions/3.8/bin/python3"

The python path was got by 'which python3' command in the terminal.

Then I restarted R session and run the same codes again, I have the second error reported:

Loading required namespace: readobj

  downloading meshes

Loading required namespace: reticulate

Error in py_call_impl(callable, dots$args, dots$keywords) : 

  UnsupportedProtocolError: 

Cloud Path must conform to FORMAT://PROTOCOL://BUCKET/PATH

Examples: 

  precomputed://gs://test_bucket/em

  gs://test_bucket/em

  graphene://https://example.com/image/em

Supported Formats: None (precomputed), graphene, precomputed, boss

Supported Protocols: gs, file, s3, matrix, http, https

Cloud Path Recieved: brainmaps://772153499790:fafb_v14:fafb-ffn1-20190805

Detailed traceback: 

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/cloudvolume.py", line 205, in __new__

    path = strict_extract(cloudpath)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/paths.py", line 91, in strict_extract

    path = extract(cloudpath, windows, disable_toabs)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/paths.py", line 144, in extract

    raise error

I appreciate any solutions or suggestions!

Bests

Li 

[Greg Jefferis]

Dear Li Guo,

Thanks for your message. I'm surprised reticulate didn't find your system Python3. Do you know if it is the Apple install or was installed from somewhere else? I assume the problem was that it selected python2 rather than python 3. In any case you have solved that problem for now, so that's great.

In terms of the next steps, you need to choose the flywire segmentation before starting to read meshes. 

choose_segmentation("flywire31")

gf.flywire=read_cloudvolume_meshes("720575940615763227")

This is mentioned in the readme, but apparently not in the read_cloudvolume_meshes docs so I have added it there. In fact I have also decided to make flywire31 the default autosegmentation since it now seems to be the most commonly used. If you update fafbseg, this will now be the case. I would still recommend to use choose_segmentation in any script in case the default changes in future.

With best wishes,

Greg.

--
You received this message because you are subscribed to the Google Groups "nat-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nat-user+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nat-user/5ab7cc98-a22f-4c8a-9509-d3e729d08a55n%40googlegroups.com.

[Gregory Jefferis]

(For reference for others)

You need to provide a token as noted in the error message. You can do this with flywire_set_token. Please consult the docs here 

http://natverse.org/fafbseg/reference/read_cloudvolume_meshes.html

for example code and further details. All the best, Greg. 

Sent from my iPhone

On 21 Nov 2020, at 16:53, Li Guo <li_...@ucsb.edu> wrote:

Hi Greg,

Thanks for your information! Before I changed the Renviron, the python R pointed to is :

> library('reticulate')

> repl_python()

Python 3.6.11 (/Users/liguo/Library/r-miniconda/envs/r-reticulate/bin/python)

Reticulate 1.16 REPL -- A Python interpreter in R.

After modification, the python R used is :

> repl_python()

Python 3.8.5 (/Library/Frameworks/Python.framework/Versions/3.8/bin/python3)

Reticulate 1.16 REPL -- A Python interpreter in R.

Looks like the system one and what I installed were both python3, but the versions were different.

After I chose the flywire segmentation, I still got the error:

> choose_segmentation("flywire31")

> gf.flywire=read_cloudvolume_meshes("720575940615763227")

  downloading meshes

Error in py_call_impl(callable, dots$args, dots$keywords) : 

  AuthenticationError: No Graphene authentication token was provided. Does ~/.cloudvolume/secrets/chunkedgraph-secret.json exist?

Detailed traceback: 

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/cloudvolume.py", line 207, in __new__

    return REGISTERED_PLUGINS[path.format](**kwargs)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/__init__.py", line 43, in create_graphene

    meta = GrapheneMetadata(

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/metadata.py", line 92, in __init__

    "Authorization": "Bearer %s" % self.parse_token(auth_token)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/metadata.py", line 120, in parse_token

    raise exceptions.AuthenticationError(

Do you have any suggestions to solve this?

Bests

Li

On Nov 21, 2020, at 8:53 AM, Li Guo <li_...@ucsb.edu> wrote:

Hi Greg,

Thanks for your information! Before I changed the Renviron, the python R pointed to is :

> library('reticulate')

> repl_python()

Python 3.6.11 (/Users/liguo/Library/r-miniconda/envs/r-reticulate/bin/python)

Reticulate 1.16 REPL -- A Python interpreter in R.

After modification, the python R used is :

> repl_python()

Python 3.8.5 (/Library/Frameworks/Python.framework/Versions/3.8/bin/python3)

Reticulate 1.16 REPL -- A Python interpreter in R.

Looks like the system one and what I installed were both python3, but the versions were different.

After I chose the flywire segmentation, I still got the error:

> choose_segmentation("flywire31")

> gf.flywire=read_cloudvolume_meshes("720575940615763227")

  downloading meshes

Error in py_call_impl(callable, dots$args, dots$keywords) : 

  AuthenticationError: No Graphene authentication token was provided. Does ~/.cloudvolume/secrets/chunkedgraph-secret.json exist?

Detailed traceback: 

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/cloudvolume.py", line 207, in __new__

    return REGISTERED_PLUGINS[path.format](**kwargs)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/__init__.py", line 43, in create_graphene

    meta = GrapheneMetadata(

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/metadata.py", line 92, in __init__

    "Authorization": "Bearer %s" % self.parse_token(auth_token)

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cloudvolume/datasource/graphene/metadata.py", line 120, in parse_token

    raise exceptions.AuthenticationError(

Do you have any suggestions to solve this?

Bests

Li

[Greg Jefferis]

Dear Li,

If you have the coordinates in a single column of text (or 3 columns of a dataframe) you can extract them with the function xyzmatrix [1]. You can then plot them with points3d or spheres3d.

For example

syns=c("(136174, 77325, 5392)", "(136195, 77358, 5393)", "(136170, 77360, 5394)", 

"(136178, 77400, 5397)", "(136336, 77300, 5402)", "(146779, 71264, 4424)", 

"(146848, 71171, 4433)", "(146880, 71155, 4435)", "(146760, 71204, 4434)", 

"(146864, 71146, 4437)", "(146894, 71150, 4436)", "(146840, 71170, 4436)", 

"(146890, 71148, 4436)", "(146886, 71146, 4436)", "(146842, 71160, 4436)", 

"(146844, 71157, 4439)", "(146863, 71141, 4441)", "(146829, 71161, 4442)", 

"(146796, 71177, 4441)")

# scale to nm from raw voxels

syns.nm=scale(xyzmatrix(syns), center = F, scale = 1/c(4,4,40))

points3d(syns.nm)

# or ...

spheres3d(syns.nm, radius=100)

Best,

Greg.

[1] since https://github.com/natverse/nat/pull/449 merged to GitHub master on 16 Nov. So you may need to update the nat package.

On 22 Nov 2020, at 19:41, Li Guo <li_...@ucsb.edu> wrote:

Hi Greg,

Thanks for your suggestions! I have another question for synapses plotting. If I got all the synapse information and saved them in (x,y,z) format according to the flywire segmentation:

(136174, 77325, 5392)
(136195, 77358, 5393)
(136170, 77360, 5394)
(136178, 77400, 5397)
(136336, 77300, 5402)
(146779, 71264, 4424)
(146848, 71171, 4433)
(146880, 71155, 4435)
(146760, 71204, 4434)
(146864, 71146, 4437)
(146894, 71150, 4436)
(146840, 71170, 4436)
(146890, 71148, 4436)
(146886, 71146, 4436)
(146842, 71160, 4436)
(146844, 71157, 4439)
(146863, 71141, 4441)
(146829, 71161, 4442)
(146796, 71177, 4441)

How can I show these dots with neuron in the natverse?

Bests

Li

On Nov 21, 2020, at 10:55 AM, Gregory Jefferis <jeff...@gmail.com> wrote:

That is the correct template although you can convert it to FlyWire space if you want. See 

http://natverse.org/fafbseg/articles/articles/FAFB-FlyWire.html

For details. 

To avoid the axes, do

plot3d(gf.flywire[1], col='red', lwd=3)

Or

wire3d(gf.flywire[[1]], col='red', lwd=3)

shade3d(gf.flywire[[1]], col='red', lwd=3)

This has to do with whether you are plotting a neuronlist or an individual  mesh3d object. 

All the best,

Greg.

Sent from my iPhone

On 21 Nov 2020, at 18:17, Li Guo <li_...@ucsb.edu> wrote:

Hi Greg,

I think I figure out how to solve this problem by myself. I am excited to get my plot by: 

gf.flywire=read_cloudvolume_meshes("720575940615763227”)

plot3d(gf.flywire[[1]], col='red', lwd=3)

plot3d(FAFB14,alpha=0.1)

<Screen Shot 2020-11-21 at 10.15.09 AM.png>

Did I use the right template? If it is possible, can you show a example how to remove the axis on the figure? And can I save the plot into PDF?

[Li Guo]

Dear Greg,

Thanks! It works very well!

Bests

Li