precomputed dataset example

[mousav...@gmail.com]

Hi,

Is there any precomputed dataset (including all json and 3D volumes/slice) for download as an example?

Thanks

Best

[Jeremy Maitin-Shepard]

You can find the precomputed datasets used for the two examples linked from the README at

gs://neuroglancer-public-data/flyem_fib-25/image/

gs://neuroglancer-public-data/flyem_fib-25/ground_truth/

gs://neuroglancer-public-data/kasthuri2011/ground_truth/

gs://neuroglancer-public-data/kasthuri2011/image/

gs://neuroglancer-public-data/kasthuri2011/image_color_corrected/

You can access these directories in several ways:

1. using gsutil

2. from the Google Cloud Console, e.g.

https://console.cloud.google.com/storage/browser/neuroglancer-public-data/flyem_fib-25/image

(requires you be signed in with a Google account)

3. via HTTP directly, e.g.

https://storage.googleapis.com/neuroglancer-public-data/flyem_fib-25/image/info

You can also view the network requests made by Neuroglancer when viewing those examples by opening Chrome or Firefox developer tools.

--
You received this message because you are subscribed to the Google Groups "Neuroglancer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuroglancer...@googlegroups.com.
To post to this group, send email to neurog...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/2b19380e-bc52-432b-ad3a-63b149faa6f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[seyed mostafa mousavi kahaki]

Thank you Jeremy,

I am able to visualize my data based on precomputed data. Thanks for the information you provided. 

I have another question, how can I visualize brain map data? I have this URL referring to a brain map volume: http://brainmaps.org/HBP4/m.mulatta/cor/mmu29720-T1/. I tried to use brainmaps://http://brainmaps.org/HBP4/m.mulatta/cor/mmu29720-T1 but it didn't work. 

Thank you,

Best

Seyed Mostafa

[Jeremy Maitin-Shepard]

The brainmaps:// scheme refers to the Google Brainmaps API, for which there is limited external availability.

Neuroglancer doesn't currently support displaying brainmaps.org data directly --- depending on how it is made available from the server, it might be possible to add support for it, though.

--

You received this message because you are subscribed to the Google Groups "Neuroglancer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuroglancer...@googlegroups.com.
To post to this group, send email to neurog...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/c87c818a-2d60-4e04-a95c-15fb03362302%40googlegroups.com.

[seyed mostafa mousavi kahaki]

Thanks Jeremy for the instant reply,

So far I only could run neuroglancer using single Nifti file and precomputed sources. Are there any other sources that you could provide such as ndstore or dvid images/DBs?

[Jeremy Maitin-Shepard]

Are you asking about additional examples?  Unfortunately I don't have any.

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/5c42bd9f-cb97-40af-8660-8692fc364244%40googlegroups.com.

[seyed mostafa mousavi kahaki]

Yes, I was asking about other examples for neuroglancer using other methods such as dvid or ndstore (as neuroglancer input). for example ndstore://... or dvid://....

Thanks

[Jeremy Maitin-Shepard]

Actually, there is this publicly available DVID server:

http://emdata.janelia.org/#/

The web UI built into DVID already provides a way to view volumes in neuroglancer.

An example datasource URL is:

dvid://http://emdata.janelia.org:80/ee78982c87b14d008bb3f93e9e546c10/grayscalejpeg

Note that because the DVID server is accessed over HTTP rather than HTTPS, Firefox and Chrome will prevent you from accessing it if you use the https:// neuroglancer demo URL --- you have to access the neuroglancer client via HTTP as well, as in the below link:

http://neuroglancer-demo.appspot.com/#!{'layers':{'grayscalejpeg':{'source':'dvid://http://emdata.janelia.org:80/ee78982c87b14d008bb3f93e9e546c10/grayscalejpeg'_'type':'image'}}_'navigation':{'pose':{'position':{'voxelSize':[8_8_8]_'voxelCoordinates':[3807.5_5569.5_7327.5]}}_'zoomFactor':8}}

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/34d5b0ce-8fa9-4041-9cd1-a9776cc38fa6%40googlegroups.com.

[seyed mostafa mousavi kahaki]

Thanks Jeremy, it works using http as you said. Thank you very much. you helped me alot. unfortunately, i need to download their data but it seems it is not possible. I'm trying to generate different types of data (nifti, precomputed, dvid, brain map formats) to be able to use in neuroglancer using our dataset. I will study more about other formats to find it out. If you have some references or information which can help me on this, I would truly appreciate. 

Thank you very much

[Jeremy Maitin-Shepard]

It definitely is possible to download data from the emdata.janelia.org server --- that is what the server is there for, in addition to being a demonstration of DVID.

Can you explain more what you are trying to accomplish?  I don't understand what you mean by "trying to generate different types of data (nifti, precomputed, dvid, brain map formats) to be able to use in neuroglancer using our dataset".

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/e3925092-6cee-4571-a42a-0964668242c7%40googlegroups.com.

[seyed mostafa mousavi kahaki]

We are doing registration and generating thousands of image tiles (or chunks including their x,y,z positions). My application should be able to generate the output in different formats, so people can use in their system. so far, I generated the output in 1) tiff, 2)nifti  3) precomputed for neuroglancer 4) sqlite db structure. 

now I am interested to generate DVID format and brain map formats. So, let's say the input is a bunch of 3D volumes (including their spatial locations) and the output will be brain map format data. 

You can have a look to our data which I generated (precomputed) and run with neuroglancer: 

https://neuroglancer-demo.appspot.com/#!{'layers':{'image4':{'source':'precomputed://https://web.northeastern.edu/kahaki/diadem/image4'_'type':'image'}}_'navigation':{'pose':{'position':{'voxelSize':[128_128_128]_'voxelCoordinates':[1213.646240234375_1491.944580078125_23]}}_'zoomFactor':211.03632264961647}_'perspectiveOrientation':[-0.12398990988731384_0.9901304841041565_0.06461020559072495_0.0096799461171031]_'perspectiveZoom':1808.0424144560645}

Best

[Jeremy Maitin-Shepard]

I don't know anything about the brainmaps.org format.

Regarding DVID, if you want to support it, it may be easiest to use the existing software to work with it, rather than trying to write it directly.  See https://github.com/janelia-flyem/dvid and https://github.com/janelia-flyem/diced

There is also TeraStitcher (https://github.com/abria/TeraStitcher) which seems to be doing something similar to what you describe.

Another format that may be useful to support is the HDF5 + XML format used by Big Data Viewer (https://imagej.net/BigDataViewer).

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/9a4c195f-ac45-4577-9763-4c567555bfbb%40googlegroups.com.

[seyed mostafa mousavi kahaki]

Thank you very much Jeremy for the valuable information. I will have look carefully at the links you provided. 

Best wishes

[afried...@gmail.com]

Hi Seyed, 

I am trying to get Neuroglancer running on my local machine and accessing data locally  but I have run into a few problems.

I have been able to get the client running successfully in my browser. I run the python dev server and I can access the page, but when I select 'python://' from the Source drop down it displays "Fetching "http://localhost:8081/neuroglancer/info/" resulted in HTTP error 404: Not Found."  This is probably because I haven't set up that folder. I tried copying the json file located here: https://storage.googleapis.com/neuroglancer-public-data/flyem_fib-25/image/info into both the static and dist folders in my app, but I it doesn't fix the error. Would you be able to walk me through what you did to get it working with a little detail. Any resources would help too.

Thanks

Aaron 

[Jeremy Maitin-Shepard]

To use the Python integration, you need to manually start a Python process --- the Python process will run a local webserver and provide you with a URL for accessing it.  The examples are in python/examples/, refer to python/README.md.

The python-dev-server is primarily useful when actually developing the Neuroglancer client code responsible for the Python integration, but is otherwise not the most convenient way.  To use it with e.g. examples/example.py, pass in --static-content-url http://localhost:8081 assuming http://localhost:8081 is the URL to the local webserver printed when you run the python-dev-server.

Regarding accessing data on brainmaps.org, the brainmaps:// backend in Neuroglancer actually refers to something different.  Currently brainmaps.org is not a supported data source for Neuroglancer, though if you care to implement support for it (which may not be too difficult), a patch would certainly be welcome (and feel free to ask additional questions).

--

You received this message because you are subscribed to the Google Groups "Neuroglancer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuroglancer...@googlegroups.com.
To post to this group, send email to neurog...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/76305d29-d0be-4101-bcc6-25f9b5dee06b%40googlegroups.com.

[afried...@gmail.com]

Thank you Jeremy,

This gives me some context. I got the example working. I'm looking to download some data onto my local machine and run it as a proof of concept.  I'm guessing that it would be easier to run a precomputed dataset but I could be wrong. I was trying to access the neuorglancer-public-data you mentioned above. The data can come from anywhere, I just need to show that it is possible to load data locally. 

I have a number of questions regarding this:

Would this public data be a valid source for downloading a dataset?

Where would I host the data, in the python process?

Would I go about loading the data by modifying the example?

I am new to this app and appreciate any light you can shine.

Thanks!     

To unsubscribe from this group and stop receiving emails from it, send an email to neurog...@googlegroups.com.

[Jeremy Maitin-Shepard]

Using the Python integration, you can view any 3-d NumPy array loaded in memory (or other Python object, like an h5py dataset, that supports indexing like a Numpy array) directly.  To do that, you can just modify e.g. example.py.

You can also use the precomputed format (or n5 or nifti formats) to view local data --- in that case you don't need the Python integration (you can even use the public http://neuroglancer-demo.appspost.com client), but will need to use some other web server to host the data.  You can use the cors_webserver.py script for that purpose.  Note that if, as is likely, you will be serving the data from an http:// rather than https:// url, you need to make sure that the Neuroglancer client you use is also served from a localhost or http:// url, not an https:// url, due to restrictions placed by web browsers.  In particular, if you are serving the data from http://localhost:8000, make sure to use http://neuroglancer-demo.appspost.com rather than https://neuroglancer-demo.appspost.com

To write some existing dataset that you may have in the neuroglancer precomputed format, you can use the https://github.com/seung-lab/cloud-volume package.

You can certainly download one of the public datasets for local testing.  You can do that using e.g.:

gsutil -m cp -r gs://neuroglancer-public-data/flyem_fib-25_training2 .

That will download the small flyem_fib-25_training2 image and ground truth segmentation volumes.  You can also download the larger ones in the gs://neuroglancer-public-data bucket, but because of the large number of files involved it will take quite a long time using gsutil.

To unsubscribe from this group and stop receiving emails from it, send an email to neuroglancer...@googlegroups.com.

To post to this group, send email to neurog...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/2e972d34-35fc-462e-a488-cba4b5961d51%40googlegroups.com.

[mostafa.m...@gmail.com]

Hi Aaron,

I think you fixed the problem already. I created a sample data for Neuroglancer from our own dataset. You can access it here web.northeastern.edu/kahaki/diadem/image4

and access the visualization test here: https://neuroglancer-demo.appspot.com/#!%7B%22layers%22:%5B%7B%22source%22:%22precomputed://https://web.northeastern.edu/kahaki/diadem/image4

[afried...@gmail.com]

Ah, so I need only to host the precompiled dataset and then point the client at it (also check the cross origin request). So I can even serve the files up statically from a Node.js server. I can just use the client locally (or use http://neuroglancer-demo.appspost.com) and make sure the urls are either both using http or both using https to load the dataset.  Then I load as my Source: precompiled://http://localhost/<flyem_fib-25_training2 folder>

Is this right?

[Jeremy Maitin-Shepard]

Yeah, that's correct.

To unsubscribe from this group and stop receiving emails from it, send an email to neuroglancer...@googlegroups.com.

To post to this group, send email to neurog...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/neuroglancer/b454f5bc-b95b-4005-830c-4aadf5f8faa1%40googlegroups.com.

[afried...@gmail.com]

Thank you, Jeremy (and Mostafa), I got the images working. It looks like that flyem_fib-25_training2 sample data includes mesh information in the ground truth folder. Is there any way to get the 3D volume loaded in the viewer?

Thanks

Yeah, that's correct.