Re: cytoseg on jane
4 views
Skip to first unread message
Rick Giuly
Oct 31, 2012, 1:24:04 PM10/31/12
to Alex Perez, cyt...@googlegroups.com
Hi Alex,
Are the training image stack and the training segmentation stack two
different sizes? That might be what's causing the error. They'll need to
have exactly the same dimensions.
The final output will be in <your output folder>/voxelOutput/blobs/resized
However, since the error happened, the final output probably wasn't
produced.
Let me know how it goes
Thanks,
-Rick
Alex Perez wrote:
> Hey Rick,
>
> The process I started yesterday terminated with the error message:
>
> from 0 to 29, loading image index 20, stack has 20 images
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 345, in <module>
> contourSingleStep('findTrainingContours')
> File "run_pipeline_test.py", line 106, in contourSingleStep
> runSteps(**segmentationParams)
> File "../run_steps.py", line 129, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 132, in run
> self.componentDetector.runLoadTrainingProbabilityMap()
> File "../component_detector.py", line 1729, in
> runLoadTrainingProbabilityMap
> self.contourProcessingTrainingRegion)
> File "../component_detector.py", line 1761, in loadTrainingProbabilityMap
> loadImageStack(self.precomputedTrainingProbabilityMapFilePath, region)
> File "../data_viewer.py", line 2758, in loadImageStack
> raise Exception("Tried to load image number %d when there are only
> %d images in the stack at %s" % (i, len(fileList), path))
> Exception: Tried to load image number 20 when there are only 20 images
> in the stack at
> /ncmirdata5/aperez/cytoseg/data/ZT04/output/voxelOutput/training/mitochondria/resized
>
> Any thoughts on what this means? I also have a full log of the process,
> if that would be helpful.
>
> There are a bunch of files in the output folder, but none of them appear
> to be correct. Also, which files in the output folder are supposed to be
> the final output?
>
> Thanks,
> Alex
>
> On Tue, Oct 30, 2012 at 1:11 PM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>> wrote:
>
>
>
> So, for running cytoseg to detect mitochondria on jane, here's how
> you can do it from the command line:
>
> Download script:
> svn checkout */http/*://cytoseg.googlecode.com/svn/trunk/
> <http://cytoseg.googlecode.com/svn/trunk/> cytoseg-read-only
>
> Go into testing folder:
> cd cytoseg
> cd testing
>
> This will set your environmental variables:
> source /home/rgiuly/edp_source7.txt
>
> This runs the process (from the testing folder) - with these
> parameters, it will run the example:
> /usr/local/epd-7.1-2/bin/python run_pipeline_test.py
> data/example/input output --trainingImage=data/example/train_images
> --trainingSeg=data/example/train_seg
> --voxelTrainingLowerBound=*,*,* --voxelTrainingUpperBound=*,*,*
> --voxelProcessingLowerBound=*,*,* --voxelProcessingUpperBound=*,*,*
> --contourTrainingLowerBound=*,*,* --contourTrainingUpperBound=*,*,*
> --contourProcessingLowerBound=*,*,*
> --contourProcessingUpperBound=*,*,* --accuracyCalcLowerBound=*,*,*
> --accuracyCalcUpperBound=*,*,* --labelConfigFile=settings3.py
> --voxelWeights=0.0130,0.00064 --contourListWeights=7,1
> --contourListThreshold=0.8 --step1 --step2
>
> To use your data, you'll need to set these parameters. You can keep
> the rest of the command line just like it is above.
>
> input image stack parameter: data/example/input in example
>
> output folder: output in example
>
> training image stack parameter:
> --trainingImage=data/example/train_images in example
>
> training segmentation stack parameter:
> --trainingSeg=data/example/train_seg in example
>
>
> More documentation here: cytoseg.googlecode.com
> <http://cytoseg.googlecode.com>
>
> Let me know if you have any questions
>
>
> -Rick
>
>
>
Rick Giuly
Oct 31, 2012, 5:19:32 PM10/31/12
to Alex Perez, cyt...@googlegroups.com
OK, can you send the whole command line that you used - then I'll take a
closer look.
-Rick
Alex Perez wrote:
> Hi Rick,
>
> The training image and training segmentation stacks are both the same
> size and are both 8-bit.
>
> There are 30 images in both training stacks, but for some reason it's
> outputting only 20 images to the directories in:
>
> /ncmirdata5/aperez/cytoseg/data/ZT04/output/voxelOutput/training/mitochondria
>
> All the other output directories contain 30 images. I'm not sure if this
> means anything, just something I noticed looking through the log.
>
> Thanks,
> Alex
>
> On Wed, Oct 31, 2012 at 10:24 AM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>> wrote:
>
>
> Hi Alex,
>
> Are the training image stack and the training segmentation stack two
> different sizes? That might be what's causing the error. They'll
> need to have exactly the same dimensions.
>
> The final output will be in <your output
> folder>/voxelOutput/blobs/__resized
> On Wed, Oct 31, 2012 at 10:24 AM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>> wrote:
>
>
> Hi Alex,
>
> Are the training image stack and the training segmentation stack two
> different sizes? That might be what's causing the error. They'll
> need to have exactly the same dimensions.
>
> The final output will be in <your output
>
> However, since the error happened, the final output probably wasn't
> produced.
>
> Let me know how it goes
>
> Thanks,
> -Rick
>
>
> Alex Perez wrote:
>
> Hey Rick,
>
> The process I started yesterday terminated with the error message:
>
> from 0 to 29, loading image index 20, stack has 20 images
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 345, in <module>
> contourSingleStep('__findTrainingContours')
> However, since the error happened, the final output probably wasn't
> produced.
>
> Let me know how it goes
>
> Thanks,
> -Rick
>
>
> Alex Perez wrote:
>
> Hey Rick,
>
> The process I started yesterday terminated with the error message:
>
> from 0 to 29, loading image index 20, stack has 20 images
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 345, in <module>
> File "run_pipeline_test.py", line 106, in contourSingleStep
> runSteps(**segmentationParams)
> File "../run_steps.py", line 129, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 132, in run
> self.componentDetector.__runLoadTrainingProbabilityMap(__)
> runSteps(**segmentationParams)
> File "../run_steps.py", line 129, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 132, in run
> File "../component_detector.py", line 1729, in
> runLoadTrainingProbabilityMap
> self.__contourProcessingTrainingRegio__n)
> runLoadTrainingProbabilityMap
> File "../component_detector.py", line 1761, in
> loadTrainingProbabilityMap
>
> loadImageStack(self.__precomputedTrainingProbability__MapFilePath, region)
> loadTrainingProbabilityMap
>
> File "../data_viewer.py", line 2758, in loadImageStack
> raise Exception("Tried to load image number %d when there
> are only
> %d images in the stack at %s" % (i, len(fileList), path))
> Exception: Tried to load image number 20 when there are only 20
> images
> in the stack at
> /ncmirdata5/aperez/cytoseg/__data/ZT04/output/voxelOutput/__training/mitochondria/resized
> raise Exception("Tried to load image number %d when there
> are only
> %d images in the stack at %s" % (i, len(fileList), path))
> Exception: Tried to load image number 20 when there are only 20
> images
> in the stack at
>
> Any thoughts on what this means? I also have a full log of the
> process,
> if that would be helpful.
>
> There are a bunch of files in the output folder, but none of
> them appear
> to be correct. Also, which files in the output folder are
> supposed to be
> the final output?
>
> Thanks,
> Alex
>
> On Tue, Oct 30, 2012 at 1:11 PM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>
> Any thoughts on what this means? I also have a full log of the
> process,
> if that would be helpful.
>
> There are a bunch of files in the output folder, but none of
> them appear
> to be correct. Also, which files in the output folder are
> supposed to be
> the final output?
>
> Thanks,
> Alex
>
> On Tue, Oct 30, 2012 at 1:11 PM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>> wrote:
>
>
>
> So, for running cytoseg to detect mitochondria on jane,
> here's how
> you can do it from the command line:
>
> Download script:
> svn checkout */http/*://cytoseg.googlecode.__com/svn/trunk/
>
>
>
> So, for running cytoseg to detect mitochondria on jane,
> here's how
> you can do it from the command line:
>
> Download script:
> <http://cytoseg.googlecode.com/svn/trunk/>
>
> <http://cytoseg.googlecode.__com/svn/trunk/
> <http://cytoseg.googlecode.com/svn/trunk/>> cytoseg-read-only
>
> Go into testing folder:
> cd cytoseg
> cd testing
>
> This will set your environmental variables:
> source /home/rgiuly/edp_source7.txt
>
> This runs the process (from the testing folder) - with these
> parameters, it will run the example:
> /usr/local/epd-7.1-2/bin/__python run_pipeline_test.py
>
> Go into testing folder:
> cd cytoseg
> cd testing
>
> This will set your environmental variables:
> source /home/rgiuly/edp_source7.txt
>
> This runs the process (from the testing folder) - with these
> parameters, it will run the example:
> data/example/input output
> --trainingImage=data/example/__train_images
> --trainingSeg=data/example/__train_seg
> --voxelTrainingLowerBound=*,*,__*
> --voxelTrainingUpperBound=*,*,__*
> --voxelProcessingLowerBound=*,__*,*
> --voxelProcessingUpperBound=*,__*,*
> --contourTrainingLowerBound=*,__*,*
> --contourTrainingUpperBound=*,__*,*
> --contourProcessingLowerBound=__*,*,*
> --contourProcessingUpperBound=__*,*,*
> --accuracyCalcLowerBound=*,*,*
> --accuracyCalcUpperBound=*,*,* --labelConfigFile=settings3.py
> --voxelWeights=0.0130,0.00064 --contourListWeights=7,1
> --contourListThreshold=0.8 --step1 --step2
>
> To use your data, you'll need to set these parameters. You
> can keep
> the rest of the command line just like it is above.
>
> input image stack parameter: data/example/input in example
>
> output folder: output in example
>
> training image stack parameter:
> --trainingImage=data/example/__train_images in example
> --accuracyCalcUpperBound=*,*,* --labelConfigFile=settings3.py
> --voxelWeights=0.0130,0.00064 --contourListWeights=7,1
> --contourListThreshold=0.8 --step1 --step2
>
> To use your data, you'll need to set these parameters. You
> can keep
> the rest of the command line just like it is above.
>
> input image stack parameter: data/example/input in example
>
> output folder: output in example
>
> training image stack parameter:
>
> training segmentation stack parameter:
> --trainingSeg=data/example/__train_seg in example
> <http://cytoseg.googlecode.com__>
Rick Giuly
Nov 1, 2012, 4:03:44 AM11/1/12
to Alex Perez, cyt...@googlegroups.com
Hi Alex, please go ahead and download cytoseg again, because I made some
updates. You can redo the whole download or just do "svn update" to get
the changes.
I also thresholded the training segmentation. This was not a documented
requirement, but it should be documented now. Thresholded version is here:
/ncmirdata1/rgiuly/alex_train_seg
Finally, I'm using settings2.py for --labelConfigFile. This just tells
it to use nonzero values for the object and zero for non-object in the
training segmentation.
So here's the command. It should work now. Let me know if you have any
more problems.
I've already ran a test with contourListThreshold=0.8 and it looks to be
too high, so you might want to do like 0.6. You'll see the difference if
you try them out a bit.
/usr/local/epd-7.1-2/bin/python run_pipeline_test.py
/ncmirdata5/aperez/cytoseg/data/ZT04/input output3
--trainingImage=/ncmirdata5/aperez/cytoseg/data/ZT04/train_images
--trainingSeg=/ncmirdata1/rgiuly/alex_train_seg
--voxelTrainingLowerBound=*,*,* --voxelTrainingUpperBound=*,*,*
--voxelProcessingLowerBound=*,*,* --voxelProcessingUpperBound=*,*,*
--contourTrainingLowerBound=*,*,* --contourTrainingUpperBound=*,*,*
--contourProcessingLowerBound=*,*,* --contourProcessingUpperBound=*,*,*
--accuracyCalcLowerBound=*,*,* --accuracyCalcUpperBound=*,*,*
--labelConfigFile=settings2.py --voxelWeights=0.0130,0.00064
--voxelProcessingLowerBound=*,*,* --voxelProcessingUpperBound=*,*,*
--contourTrainingLowerBound=*,*,* --contourTrainingUpperBound=*,*,*
--contourProcessingLowerBound=*,*,* --contourProcessingUpperBound=*,*,*
--accuracyCalcLowerBound=*,*,* --accuracyCalcUpperBound=*,*,*
--contourListWeights=7,1 --contourListThreshold=0.8 --step2 &>
outputfile3.txt
Thanks,
-Rick
Alex Perez wrote:
>
> /usr/local/epd-7.1-2/bin/python run_pipeline_test.py
> /ncmirdata5/aperez/cytoseg/data/ZT04/input
> /ncmirdata5/aperez/cytoseg/data/ZT04/output
> --trainingImage=/ncmirdata5/aperez/cytoseg/data/ZT04/train_images
> --trainingSeg=/ncmirdata5/aperez/cytoseg/data/ZT04/train_seg
> --voxelTrainingLowerBound=*,*,* --voxelTrainingUpperBound=*,*,*
> --voxelProcessingLowerBound=*,*,* --voxelProcessingUpperBound=*,*,*
> --contourTrainingLowerBound=*,*,* --contourTrainingUpperBound=*,*,*
> --contourProcessingLowerBound=*,*,* --contourProcessingUpperBound=*,*,*
> --accuracyCalcLowerBound=*,*,* --accuracyCalcUpperBound=*,*,*
> --voxelProcessingLowerBound=*,*,* --voxelProcessingUpperBound=*,*,*
> --contourTrainingLowerBound=*,*,* --contourTrainingUpperBound=*,*,*
> --contourProcessingLowerBound=*,*,* --contourProcessingUpperBound=*,*,*
> --labelConfigFile=settings3.py --voxelWeights=0.0130,0.00064
> --contourListWeights=7,1 --contourListThreshold=0.8 --step1 --step2 2>&1
> | tee cytoseg.log
>
> Thanks,
> Alex
>
> On Wed, Oct 31, 2012 at 2:19 PM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>> wrote:
>
>
> OK, can you send the whole command line that you used - then I'll
> take a closer look.
>
> -Rick
>
> Alex Perez wrote:
>
> Hi Rick,
>
> The training image and training segmentation stacks are both the
> same
> size and are both 8-bit.
>
> There are 30 images in both training stacks, but for some reason
> it's
> outputting only 20 images to the directories in:
>
> /ncmirdata5/aperez/cytoseg/__data/ZT04/output/voxelOutput/__training/mitochondria
> On Wed, Oct 31, 2012 at 2:19 PM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>> wrote:
>
>
> OK, can you send the whole command line that you used - then I'll
> take a closer look.
>
> -Rick
>
> Alex Perez wrote:
>
> Hi Rick,
>
> The training image and training segmentation stacks are both the
> same
> size and are both 8-bit.
>
> There are 30 images in both training stacks, but for some reason
> it's
> outputting only 20 images to the directories in:
>
>
> All the other output directories contain 30 images. I'm not sure
> if this
> means anything, just something I noticed looking through the log.
>
> Thanks,
> Alex
>
> On Wed, Oct 31, 2012 at 10:24 AM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>
> All the other output directories contain 30 images. I'm not sure
> if this
> means anything, just something I noticed looking through the log.
>
> Thanks,
> Alex
>
> On Wed, Oct 31, 2012 at 10:24 AM, Rick Giuly <rgi...@gmail.com
> <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>> wrote:
>
>
> Hi Alex,
>
> Are the training image stack and the training segmentation
> stack two
> different sizes? That might be what's causing the error.
> They'll
> need to have exactly the same dimensions.
>
> The final output will be in <your output
> folder>/voxelOutput/blobs/____resized
>
>
> Hi Alex,
>
> Are the training image stack and the training segmentation
> stack two
> different sizes? That might be what's causing the error.
> They'll
> need to have exactly the same dimensions.
>
> The final output will be in <your output
>
>
> However, since the error happened, the final output
> probably wasn't
> produced.
>
> Let me know how it goes
>
> Thanks,
> -Rick
>
>
> Alex Perez wrote:
>
> Hey Rick,
>
> The process I started yesterday terminated with the
> error message:
>
> from 0 to 29, loading image index 20, stack has 20 images
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 345, in <module>
> contourSingleStep('____findTrainingContours')
>
> However, since the error happened, the final output
> probably wasn't
> produced.
>
> Let me know how it goes
>
> Thanks,
> -Rick
>
>
> Alex Perez wrote:
>
> Hey Rick,
>
> The process I started yesterday terminated with the
> error message:
>
> from 0 to 29, loading image index 20, stack has 20 images
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 345, in <module>
>
> File "run_pipeline_test.py", line 106, in
> contourSingleStep
> runSteps(**segmentationParams)
> File "../run_steps.py", line 129, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 132, in run
>
> self.componentDetector.____runLoadTrainingProbabilityMap(____)
> File "run_pipeline_test.py", line 106, in
> contourSingleStep
> runSteps(**segmentationParams)
> File "../run_steps.py", line 129, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 132, in run
>
>
> File "../component_detector.py", line 1729, in
> runLoadTrainingProbabilityMap
> self.____contourProcessingTrainingRegio____n)
> File "../component_detector.py", line 1729, in
> runLoadTrainingProbabilityMap
>
> File "../component_detector.py", line 1761, in
> loadTrainingProbabilityMap
>
>
> loadImageStack(self.____precomputedTrainingProbability____MapFilePath,
> File "../component_detector.py", line 1761, in
> loadTrainingProbabilityMap
>
>
> region)
>
> File "../data_viewer.py", line 2758, in loadImageStack
> raise Exception("Tried to load image number %d
> when there
> are only
> %d images in the stack at %s" % (i, len(fileList), path))
> Exception: Tried to load image number 20 when there are
> only 20
> images
> in the stack at
>
> /ncmirdata5/aperez/cytoseg/____data/ZT04/output/voxelOutput/____training/mitochondria/resized
>
> File "../data_viewer.py", line 2758, in loadImageStack
> raise Exception("Tried to load image number %d
> when there
> are only
> %d images in the stack at %s" % (i, len(fileList), path))
> Exception: Tried to load image number 20 when there are
> only 20
> images
> in the stack at
>
>
>
> Any thoughts on what this means? I also have a full log
> of the
> process,
> if that would be helpful.
>
> There are a bunch of files in the output folder, but
> none of
> them appear
> to be correct. Also, which files in the output folder are
> supposed to be
> the final output?
>
> Thanks,
> Alex
>
> On Tue, Oct 30, 2012 at 1:11 PM, Rick Giuly
> <rgi...@gmail.com <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>>> wrote:
>
>
>
> So, for running cytoseg to detect mitochondria on
> jane,
> here's how
> you can do it from the command line:
>
> Download script:
> svn checkout
> */http/*://cytoseg.googlecode.____com/svn/trunk/
>
> Any thoughts on what this means? I also have a full log
> of the
> process,
> if that would be helpful.
>
> There are a bunch of files in the output folder, but
> none of
> them appear
> to be correct. Also, which files in the output folder are
> supposed to be
> the final output?
>
> Thanks,
> Alex
>
> On Tue, Oct 30, 2012 at 1:11 PM, Rick Giuly
> <rgi...@gmail.com <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>
> <mailto:rgi...@gmail.com <mailto:rgi...@gmail.com>>>> wrote:
>
>
>
> So, for running cytoseg to detect mitochondria on
> jane,
> here's how
> you can do it from the command line:
>
> Download script:
> svn checkout
> <http://cytoseg.googlecode.__com/svn/trunk/
> <http://cytoseg.googlecode.com/svn/trunk/>>
>
> <http://cytoseg.googlecode.____com/svn/trunk/
>
> <http://cytoseg.googlecode.__com/svn/trunk/
> <http://cytoseg.googlecode.com/svn/trunk/>>> cytoseg-read-only
>
> Go into testing folder:
> cd cytoseg
> cd testing
>
> This will set your environmental variables:
> source /home/rgiuly/edp_source7.txt
>
> This runs the process (from the testing folder) -
> with these
> parameters, it will run the example:
> /usr/local/epd-7.1-2/bin/____python
> <http://cytoseg.googlecode.__com/svn/trunk/
> <http://cytoseg.googlecode.com/svn/trunk/>>> cytoseg-read-only
>
> Go into testing folder:
> cd cytoseg
> cd testing
>
> This will set your environmental variables:
> source /home/rgiuly/edp_source7.txt
>
> This runs the process (from the testing folder) -
> with these
> parameters, it will run the example:
> run_pipeline_test.py
> data/example/input output
> --trainingImage=data/example/____train_images
> --trainingSeg=data/example/____train_seg
> --voxelTrainingLowerBound=*,*,____*
> --voxelTrainingUpperBound=*,*,____*
> --voxelProcessingLowerBound=*,____*,*
> --voxelProcessingUpperBound=*,____*,*
> --contourTrainingLowerBound=*,____*,*
> --contourTrainingUpperBound=*,____*,*
> --contourProcessingLowerBound=____*,*,*
> --contourProcessingUpperBound=____*,*,*
>
> --accuracyCalcLowerBound=*,*,*
> --accuracyCalcUpperBound=*,*,*
> --labelConfigFile=settings3.py
> --voxelWeights=0.0130,0.00064 --contourListWeights=7,1
> --contourListThreshold=0.8 --step1 --step2
>
> To use your data, you'll need to set these
> parameters. You
> can keep
> the rest of the command line just like it is above.
>
> input image stack parameter: data/example/input in
> example
>
> output folder: output in example
>
> training image stack parameter:
> --trainingImage=data/example/____train_images in
> --accuracyCalcLowerBound=*,*,*
> --accuracyCalcUpperBound=*,*,*
> --labelConfigFile=settings3.py
> --voxelWeights=0.0130,0.00064 --contourListWeights=7,1
> --contourListThreshold=0.8 --step1 --step2
>
> To use your data, you'll need to set these
> parameters. You
> can keep
> the rest of the command line just like it is above.
>
> input image stack parameter: data/example/input in
> example
>
> output folder: output in example
>
> training image stack parameter:
> example
>
> training segmentation stack parameter:
> --trainingSeg=data/example/____train_seg in example
>
> training segmentation stack parameter:
>
>
>
> More documentation here: cytoseg.googlecode.com
> <http://cytoseg.googlecode.com>
> <http://cytoseg.googlecode.com__>
> <http://cytoseg.googlecode.__com__
>
>
> More documentation here: cytoseg.googlecode.com
> <http://cytoseg.googlecode.com>
> <http://cytoseg.googlecode.com__>
Reply all
Reply to author
Forward
0 new messages