Errors while running custom pipeline

42 views
Skip to first unread message

Y Wang

unread,
May 18, 2021, 2:25:01 PM5/18/21
to bob-devel
Hello, 
I defined my own preprocessor and extractor according to the example of the documentation in the following webpage: 

I wrote the preprocessor and extractor in different python files. The class of preprocessor and the class of extractor both have the 'transform' method. I tested them by one image, and both work fine. 

Here is my testing: 
I loaded an image data X,  then put X into the preprocessor,  and got the preprocessed data.  preprocessed_X = PerturbTransformer().transform(X)

Then I fed the preprocessed_X into my extractor AFFFE().transform(preprocessed_X), and it works fine as well. 

Below is my config file, the pipeline. I loaded the preprocessor and extractor, then wrap them by make_pipeline...define an algorithm, then wrap all.. just as the documentation shows.
1.png

I would like to test the pipeline on atnt database, so I run the pipeline command, and I got this error message:
2.png

As the preprocessor and extractor work fine, and the pipeline seems fine as well. What is this Error? How can I solve it? 
If you need more information to locate the problem, please tell me. 

Thank you so much!

Amir Mohammadi

unread,
May 19, 2021, 5:53:09 AM5/19/21
to bob-...@googlegroups.com, bob-devel
Please send the full error log.

On May 18 2021, at 8:25 pm, Y Wang <wangyu...@gmail.com> wrote:
Hello, 
I defined my own preprocessor and extractor according to the example of the documentation in the following webpage: 

I wrote the preprocessor and extractor in different python files. The class of preprocessor and the class of extractor both have the 'transform' method. I tested them by one image, and both work fine. 

Here is my testing: 
I loaded an image data X,  then put X into the preprocessor,  and got the preprocessed data.  preprocessed_X = PerturbTransformer().transform(X)

Then I fed the preprocessed_X into my extractor AFFFE().transform(preprocessed_X), and it works fine as well. 

Below is my config file, the pipeline. I loaded the preprocessor and extractor, then wrap them by make_pipeline...define an algorithm, then wrap all.. just as the documentation shows.


As the preprocessor and extractor work fine, and the pipeline seems fine as well. What is this Error? How can I solve it? 
If you need more information to locate the problem, please tell me. 

Thank you so much!


--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

Y Wang

unread,
May 19, 2021, 8:43:43 AM5/19/21
to bob-devel
Here is the complete error message
3.png
4.png

Y Wang

unread,
May 19, 2021, 9:03:49 AM5/19/21
to bob-devel
The error log said my preprocessor has no attribute 'fit', but it has. 
I defined the 'fit' inside the transform method just as the sample code.
5.png


Sample code of the Transformer:

6.png

Amir Mohammadi

unread,
May 19, 2021, 10:42:34 AM5/19/21
to bob-...@googlegroups.com, bob-devel
The error is raised because you don't have a fit method in PerturbTransformer.

On May 19 2021, at 2:43 pm, Y Wang <wangyu...@gmail.com> wrote:
Here is the complete error message


Amir Mohammadi

unread,
May 19, 2021, 11:34:14 AM5/19/21
to bob-...@googlegroups.com, bob-devel
That's a formatting issue from the docs. It should be like this:
from sklearn.base import TransformerMixin, BaseEstimator

class CustomTransformer(TransformerMixin, BaseEstimator):
    def transform(self, X):
        transformed_X = my_function(X)
        return transformed_X

    def fit(self, X, y=None):
        return self
or if you want to just define the transform method, this way is easier:
from sklearn.preprocessing import FunctionTransformer

def CustomTransformer(**kwargs):
    return FunctionTransformer(my_function, **kwargs)
On May 19 2021, at 3:03 pm, Y Wang <wangyu...@gmail.com> wrote:
The error log said my preprocessor has no attribute 'fit', but it has. 
I defined the 'fit' inside the transform method just as the sample code.


Y Wang

unread,
May 19, 2021, 1:07:09 PM5/19/21
to bob-devel
I revised and run the pipeline again, it seems the data has passed the preprocessor successfully, but an error raised when
going through my feature extractor. 
I have a calculation X = X/225. in my extractor code. According to what I see in the gitlab feature-extractor template, this calculation should be supported, but it raised an error: 

       X = X/255.

   TypeError: unsupported operand type(s) for /: 'SampleBatch' and 'float'


Here is the gitlab sample code:

https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/feature-extractors/bob/bio/face/extractor/PyTorchModel.py

 10.png


It doesn't matter whether it's 255 (int) or 255. (float), both raised the TypeError. 

Below is the complete error log:

8.png9.png


Sorry to bother you again, but could you check what is this problem?

Thank you.


Amir Mohammadi

unread,
May 20, 2021, 5:12:56 AM5/20/21
to bob-...@googlegroups.com, bob-devel
check input validation in

On May 19 2021, at 7:07 pm, Y Wang <wangyu...@gmail.com> wrote:
I revised and run the pipeline again, it seems the data has passed the preprocessor successfully, but an error raised when
going through my feature extractor. 
I have a calculation X = X/225. in my extractor code. According to what I see in the gitlab feature-extractor template, this calculation should be supported, but it raised an error: 
       X = X/255.

   TypeError: unsupported operand type(s) for /: 'SampleBatch' and 'float'



Here is the gitlab sample code:


 



Reply all
Reply to author
Forward
0 new messages