Re: NodeDoesNotExist

[Rick Giuly]

Hi Kurt,

Yeah, it's not documented, so I can see why this would be confusing.

You can keep the deprecated one.

(1)
self.addPersistentVolumeAndRefreshDataTree(volume, volume_name)
This will save a volume with the name given by volume_name
volume - 3D volume (array)
volume_name - name of the node where the volume is to be stored

(2)
gui.getVolume(volume_name)
This will get a volume with the name given by volume_name
volume_name - name of the node where the volume is to be stored

Those volume_name's have to match for (1) and (2) or that error will occur.


There's a call (addPersistentVolumeAndRefreshDataTree) in
calculateDerivatives that creates the volume and gives it a name. Then
there's a call (getVolume) in getPointFeaturesAt that accesses it.


I'd recommend adding print statements to print out the parameters of
addPersistentVolumeAndRefreshDataTree, and by looking at them, you will
probably see that the volume name is slightly different or something.
Hope that helps, let me how it goes.


Best,
-Rick

[Rick Giuly]

Hi Kurt,

Actually, I think this may be the problem, in the function
recordLocalFeatures. Try just changing the "if 0" to "if 1". I may have
just removed that derivative calculation and now it needs to be put back.


Code snippet:


file = open(voxelExamplesFilename, "w")

filteredVolume =
self.getPersistentObject(voxelTrainingImageNodePath)
sh = filteredVolume.shape

#originalVolume = self.getPersistentObject(originalVolumeNodePath)

print "learnLocalFeatures dimensions", sh
if 0:
self.calculateDerivatives(filteredVolume, 'training')

volume = numpy.zeros(sh)
#selected x, y, and z

# get point features at the arbitrary point borderWidthFeatures
# to get a list of feature names
dictionary = getPointFeaturesAt(inputVolumeDict, filteredVolume,
'training', self,
borderWidthForFeatures)





kurt weiss wrote:
> Hi Rick,
> I tried to comment in the eigenvalues and structure tensor in
> cytoseg_classify.py, but I'm getting a Node DoesNotExist error at this
> line:
>
> xGVolume = gui.getVolume('%s_0Gradient_blur%d' %\
> (derivativeVolumesIdentifier, i))
>
> I attempted a few changes like calling gui.getVolume_new instead of the
> deprecated function and commenting in the i=0 statement above
> xGVolume=..., and well messing around with a whole bunch of stuff in
> hopes of figuring it out myself, but no luck. Any suggestions?
> Here is a snippet of the error followed by a snippet of the code I
> changed. The full terminal output and cytoseg_classify.py files are
> attached.
> Thank You very much,
> Kurt
>
> ERROR:
> from 0 to 4, loading image index 4, stack has 72 images
> data/example/in/cropped input004.tif
> volume.shape (615, 551, 5)
> subvolume box.cornerA [0, 0, 0]
> subvolume box.cornerB [615, 551, 5]
> subvolume box.shape() [615 551 5]
> volume shape (615, 551)
> full array shape (615, 551)
> cropped array shape (615, 551)
> ../data_viewer.py:493: UserWarning: deprecated
> warnings.warn("deprecated")
> runClassifyVoxels
> self.numberOfLayersToProcess None
> ../data_viewer.py:446: UserWarning: deprecated
> warnings.warn("deprecated")
> ../data_viewer.py:391: UserWarning: deprecated: addVolume
> warnings.warn("deprecated: addVolume")
> recordLocalFeatures
> learnLocalFeatures dimensions (350, 350, 75)
> ../data_viewer.py:416: UserWarning: deprecated: getVolume
> warnings.warn("deprecated: getVolume")
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 366, in <module>
> voxelStep('classifyVoxels')
> File "run_pipeline_test.py", line 101, in voxelStep
> voxelChunkSize)
> File "../batch_process.py", line 74, in batchProcessVoxels
> runSteps(**segmentationParams)
> File "../run_steps.py", line 152, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 115, in run
> self.componentDetector.runClassifyVoxels()
> File "../component_detector.py", line 1856, in runClassifyVoxels
> numberOfLayersToProcess=self.numberOfLayersToProcess)
> File "../component_detector.py", line 817, in classifyVoxels
> self.voxelWeightDict)
> File "../cytoseg_classify.py", line 531, in recordLocalFeatures
> 'training', self, borderWidthForFeatures)
> File "../cytoseg_classify.py", line 1285, in getPointFeaturesAt
> (derivativeVolumesIdentifier, i))
> File "../data_viewer.py", line 417, in getVolume
> node = getNode(self.mainDoc.dataRootNode, ('Volumes', name))
> File "../tree.py", line 295, in getNode
> currentNode = currentNode.getChild(name)
> File "../tree.py", line 140, in getChild
> raise NodeDoesNotExist, ("Tried to access node named %s but it
> wasn't there. (parent node name: %s) (parent node: %s)" % (name,
> self.name <http://self.name>, self))
> tree.NodeDoesNotExist: Tried to access node named
> training_0Gradient_blur0 but it wasn't there. (parent node name:
> Volumes) (parent node: Volumes( test volume 1() test volume 2()
> voxelTrainingImage() voxelTrainingLabel() default_testOriginalVolume()
> inputImage()))
>
> ********************************************************************************
> cytoseg_classify.py lines 1258 - 1331
>
> def getPointFeaturesAt(inputVolumeDict, volume,
> derivativeVolumesIdentifier, gui, point):
> """Get features (for classification) at a particular point"""
>
> # f is dictionary of features
>
> if not(isInsideVolumeWithBorder(volume, point,
> borderWidthForFeatures)):
> raise Exception, 'The point %s is not inside the volume enough.
> In needs to be away from the border by %d pixels for x, %d pixels for y,
> and %d pixels for z. Volume shape: %s' % (point,
> borderWidthForFeatures[0], borderWidthForFeatures[1],
> borderWidthForFeatures[2], str(volume.shape))
>
> f = odict()
>
>
> #sizeIdentifiers = ('3x3x3', '5x5x5', '7x7x7')
> sizeIdentifiers = ('(3)', '(5)', '(7)')
> v = [None, None, None]
>
> #for i in range(3):
> for i in range(1):
> size = i+1
> v =
> volume[point[0]-size:point[0]+size,point[1]-size:point[1]+size,point[2]-size:point[2]+size]
>
>
> # experimental features
> #if 0:
>
> #i = 0
> #todo: note that getVolume may be a slow operation
> xGVolume = gui.getVolume('%s_0Gradient_blur%d' %\
> (derivativeVolumesIdentifier, i))
> xG = at(xGVolume, point)
> yGVolume = gui.getVolume('%s_1Gradient_blur%d' %\
> (derivativeVolumesIdentifier, i))
> yG = at(yGVolume, point)
> zGVolume = gui.getVolume('%s_2Gradient_blur%d' %\
> (derivativeVolumesIdentifier, i))
> zG = at(zGVolume, point)
>
> # experimental features
> #if 0:
>
> if i == 0:
> f['grayValue'] = at(volume, point)
> #'differenceOfGaussian'
> f['gradientMagnitude'] = sqrt(pow(xG,2) + pow(yG,2) +
> pow(zG,2))
>
>
> # experimental features
> #if 0:
>
> stAtSelectedPoint = structureTensor(xG,yG,zG)
>
> sortedEigAtSelectedPoint = numpy.linalg.eigvals(stAtSelectedPoint)
> sortedEigAtSelectedPoint.sort()
>
> prefix = sizeIdentifiers[i] + '_'
>
> f[prefix + 'eig0'] = sortedEigAtSelectedPoint[0]
> f[prefix + 'eig1'] = sortedEigAtSelectedPoint[1]
> f[prefix + 'eig2'] = sortedEigAtSelectedPoint[2]
>
> values = v.flatten(1)
> #print "i", i, "values", values
>
> moments = statistics.moments(values)
> f[prefix + 'mean'] = moments[0]
> f[prefix + 'standardDeviation'] = moments[1]
> f[prefix + 'thirdMoment'] = moments[2]
> f[prefix + 'fourthMoment'] = moments[3]
>
> quantiles = statistics.sortAndReturnQuantiles(values)
> f[prefix + 'minimum'] = quantiles[0]
> f[prefix + '0.25-quantile'] = quantiles[1]
> f[prefix + 'median'] = quantiles[2]
> f[prefix + '0.75-quantile'] = quantiles[3]
> f[prefix + 'maximum'] = quantiles[4]
>

[Rick Giuly]

Hi Kurt,

This line gives you v:

v =
volume[point[0]-size:point[0]+size,point[1]-size:point[1]+size,point[2]-size:point[2]+size]

It's just a small 3D volume centered on "point". It's 3x3x3. You can
make it a larger volume by adjusting "i".



Regarding values:

values = v.flatten(1)

This means "get a list of of every value in the v array" - it's just to
get a list of numbers so you can do statistics on them.

So it looks like you're enabling the statistics related stuff, and for
that to work, you'll need the values. The length of the values list
should never be zero (it should be 3x3x3 = 27 in this case).

-rick


kurt weiss wrote:
> Thanks Rick. This got me past the node not existing, but now I'm stuck
> on properly filling the 'values' array. I think. Here is what happened:
> after your fix I got the error:

>
> Traceback (most recent call last):
> File "run_pipeline_test.py", line 366, in <module>
> voxelStep('classifyVoxels')
> File "run_pipeline_test.py", line 101, in voxelStep
> voxelChunkSize)
> File "../batch_process.py", line 74, in batchProcessVoxels
> runSteps(**segmentationParams)
> File "../run_steps.py", line 152, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 115, in run
> self.componentDetector.runClassifyVoxels()
> File "../component_detector.py", line 1856, in runClassifyVoxels
> numberOfLayersToProcess=self.numberOfLayersToProcess)
> File "../component_detector.py", line 817, in classifyVoxels
> self.voxelWeightDict)
> File "../cytoseg_classify.py", line 531, in recordLocalFeatures
> 'training', self, borderWidthForFeatures)

> File "../cytoseg_classify.py", line 1320, in getPointFeaturesAt
> moments = statistics.moments(values)
> File "../statistics.py", line 43, in moments
> numpy.sqrt(moment(values, meanValue, 2)),
> File "../statistics.py", line 35, in moment
> return total / len(values)
> ZeroDivisionError: float division by zero
>
> So I made a quick if statement to avoid dividing by zero and
> subsequently got the following error:

> ../data_viewer.py:416: UserWarning: deprecated: getVolume
> warnings.warn("deprecated: getVolume")

> *i 0 values [] *
> /usr/local/lib/python2.7/dist-packages/numpy-1.6.2-py2.7-linux-x86_64.egg/numpy/core/fromnumeric.py:2374:
> RuntimeWarning: invalid value encountered in double_scalars
> return mean(axis, dtype, out)
> lenth of values: 0
> lenth of values: 0
> lenth of values: 0

> Traceback (most recent call last):
> File "run_pipeline_test.py", line 366, in <module>
> voxelStep('classifyVoxels')
> File "run_pipeline_test.py", line 101, in voxelStep
> voxelChunkSize)
> File "../batch_process.py", line 74, in batchProcessVoxels
> runSteps(**segmentationParams)
> File "../run_steps.py", line 152, in runSteps
> detector.run(steps)
> File "../segmentation_manager.py", line 115, in run
> self.componentDetector.runClassifyVoxels()
> File "../component_detector.py", line 1856, in runClassifyVoxels
> numberOfLayersToProcess=self.numberOfLayersToProcess)
> File "../component_detector.py", line 817, in classifyVoxels
> self.voxelWeightDict)
> File "../cytoseg_classify.py", line 531, in recordLocalFeatures
> 'training', self, borderWidthForFeatures)

> File "../cytoseg_classify.py", line 1326, in getPointFeaturesAt
> quantiles = statistics.sortAndReturnQuantiles(values)
> File "../statistics.py", line 56, in sortAndReturnQuantiles
> return (values[0], values[N/4], values[N/2], values[(3*N)/4],
> values[N-1])
> IndexError: index out of bounds
>
> So it seems values is just empty by using the print statement. The only
> reference I see to it is at line 1317 of cytoseg_classify: How should I
> be filling values? and what is it exactly? and what is v?
> Thanks again,
> Kurt

>
> # experimental features
> #if 0:
>
> if i == 0:
> f['grayValue'] = at(volume, point)
> #'differenceOfGaussian'
> f['gradientMagnitude'] = sqrt(pow(xG,2) + pow(yG,2) +
> pow(zG,2))
>
>
> # experimental features
> #if 0:
>
> stAtSelectedPoint = structureTensor(xG,yG,zG)
>
> sortedEigAtSelectedPoint = numpy.linalg.eigvals(stAtSelectedPoint)
> sortedEigAtSelectedPoint.sort()
>
> prefix = sizeIdentifiers[i] + '_'
>
> f[prefix + 'eig0'] = sortedEigAtSelectedPoint[0]
> f[prefix + 'eig1'] = sortedEigAtSelectedPoint[1]
> f[prefix + 'eig2'] = sortedEigAtSelectedPoint[2]
>
> values = v.flatten(1)

> self.getPersistentObject(__voxelTrainingImageNodePath)

> sh = filteredVolume.shape
>
> #originalVolume =

> self.getPersistentObject(__originalVolumeNodePath)

>
> print "learnLocalFeatures dimensions", sh
> if 0:

> self.calculateDerivatives(__filteredVolume, 'training')

>
> volume = numpy.zeros(sh)
> #selected x, y, and z
>
> # get point features at the arbitrary point borderWidthFeatures
> # to get a list of feature names

> dictionary = getPointFeaturesAt(__inputVolumeDict,

> filteredVolume,
> 'training', self, borderWidthForFeatures)
>
>
>
>
>
> kurt weiss wrote:
>
> Hi Rick,
> I tried to comment in the eigenvalues and structure tensor in
> cytoseg_classify.py, but I'm getting a Node DoesNotExist error
> at this
> line:
>

> xGVolume = gui.getVolume('%s_0Gradient___blur%d' %\

> self.componentDetector.__runClassifyVoxels()

> File "../component_detector.py", line 1856, in runClassifyVoxels

> numberOfLayersToProcess=self.__numberOfLayersToProcess)

> File "../component_detector.py", line 817, in classifyVoxels
> self.voxelWeightDict)
> File "../cytoseg_classify.py", line 531, in recordLocalFeatures
> 'training', self, borderWidthForFeatures)
> File "../cytoseg_classify.py", line 1285, in getPointFeaturesAt
> (derivativeVolumesIdentifier, i))
> File "../data_viewer.py", line 417, in getVolume

> node = getNode(self.mainDoc.__dataRootNode, ('Volumes', name))

> File "../tree.py", line 295, in getNode
> currentNode = currentNode.getChild(name)
> File "../tree.py", line 140, in getChild
> raise NodeDoesNotExist, ("Tried to access node named %s but it
> wasn't there. (parent node name: %s) (parent node: %s)" % (name,

> self.name <http://self.name> <http://self.name>, self))

> tree.NodeDoesNotExist: Tried to access node named
> training_0Gradient_blur0 but it wasn't there. (parent node name:
> Volumes) (parent node: Volumes( test volume 1() test volume 2()
> voxelTrainingImage() voxelTrainingLabel()
> default_testOriginalVolume()
> inputImage()))
>

> ******************************__******************************__********************

> cytoseg_classify.py lines 1258 - 1331
>

> def getPointFeaturesAt(__inputVolumeDict, volume,

> derivativeVolumesIdentifier, gui, point):
> """Get features (for classification) at a particular point"""
>
> # f is dictionary of features
>

> if not(isInsideVolumeWithBorder(__volume, point,

> borderWidthForFeatures)):
> raise Exception, 'The point %s is not inside the
> volume enough.
> In needs to be away from the border by %d pixels for x, %d
> pixels for y,
> and %d pixels for z. Volume shape: %s' % (point,
> borderWidthForFeatures[0], borderWidthForFeatures[1],
> borderWidthForFeatures[2], str(volume.shape))
>
> f = odict()
>
>
> #sizeIdentifiers = ('3x3x3', '5x5x5', '7x7x7')
> sizeIdentifiers = ('(3)', '(5)', '(7)')
> v = [None, None, None]
>
> #for i in range(3):
> for i in range(1):
> size = i+1
> v =

> volume[point[0]-size:point[0]+__size,point[1]-size:point[1]+__size,point[2]-size:point[2]+__size]

>
>
> # experimental features
> #if 0:
>
> #i = 0
> #todo: note that getVolume may be a slow operation

> xGVolume = gui.getVolume('%s_0Gradient___blur%d' %\

>
> (derivativeVolumesIdentifier, i))
> xG = at(xGVolume, point)

> yGVolume = gui.getVolume('%s_1Gradient___blur%d' %\

>
> (derivativeVolumesIdentifier, i))
> yG = at(yGVolume, point)

> zGVolume = gui.getVolume('%s_2Gradient___blur%d' %\

>
> (derivativeVolumesIdentifier, i))
> zG = at(zGVolume, point)
>
> # experimental features
> #if 0:
>
> if i == 0:
> f['grayValue'] = at(volume, point)
> #'differenceOfGaussian'
> f['gradientMagnitude'] = sqrt(pow(xG,2) + pow(yG,2) +
> pow(zG,2))
>
>
> # experimental features
> #if 0:
>
> stAtSelectedPoint = structureTensor(xG,yG,zG)
>
> sortedEigAtSelectedPoint =

> numpy.linalg.eigvals(__stAtSelectedPoint)
> sortedEigAtSelectedPoint.sort(__)

>
> prefix = sizeIdentifiers[i] + '_'
>
> f[prefix + 'eig0'] = sortedEigAtSelectedPoint[0]
> f[prefix + 'eig1'] = sortedEigAtSelectedPoint[1]
> f[prefix + 'eig2'] = sortedEigAtSelectedPoint[2]
>
> values = v.flatten(1)
> #print "i", i, "values", values
>
> moments = statistics.moments(values)
> f[prefix + 'mean'] = moments[0]
> f[prefix + 'standardDeviation'] = moments[1]
> f[prefix + 'thirdMoment'] = moments[2]
> f[prefix + 'fourthMoment'] = moments[3]
>

> quantiles = statistics.__sortAndReturnQuantiles(values)

> f[prefix + 'minimum'] = quantiles[0]
> f[prefix + '0.25-quantile'] = quantiles[1]
> f[prefix + 'median'] = quantiles[2]
> f[prefix + '0.75-quantile'] = quantiles[3]
> f[prefix + 'maximum'] = quantiles[4]
>
>
>
>
>

> --
> .