Some issues with strongly-typed GP ('NoneType' object is not subscriptable)

[Ben]

Hi all,

I'm having some real trouble here. I'm moving over from ECJ to DEAP.

I'm running into some real issues with DEAP that I've been stuck on all day. 

I'm getting the following error

'NoneType' object is not subscriptable

And can't figure out why.

Ive done some searching around, but each of the issues seems to be different to mine

https://github.com/DEAP/deap/issues/142: Not using this method of compile

https://groups.google.com/forum/#!topic/deap-users/NgL8_rYr4MI: I have an input to my tree defined

I created a minimal example that is causing this issue here: https://gist.github.com/ben-ix/284b301eba0e819c0a0722c8bcc1acbc

For completeness, heres a sample tree generated

So the structure is being enforced. The image (ARG0) is being used as input, a shape has also been selected as a terminal. 

These are then passed into the example function, with the correct branch ordering being used so the strongly typed structure is there.

The textual representation of the tree is also correct, giving: eg(ARG0, Shape(Ellipse)), and I have defined both a  __repr__ and 

a __str__ method for the two classes (as I was getting another error without this).

The output of this function/tree should just be 1, due to this being hardcoded for the example. However, the problem occurs when trying

to run an executable version of the tree, i.e.

 output = toolbox.compile(expr=individual)
 label = output(sample_img)

Running this will throw the following error on the output call

<string> in <lambda>(ARG0) TypeError: 'NoneType' object is not subscriptable

What can I do to solve this, any help would be greatly appreciated.

Thank you 

[Ben]

What i'm finding is: The input to GP can be a custom type, likely because DEAP renames this to i.e. ARG0. However typed function input/outputs only seem to be working using the built-in python types, not custom classes (even with a __repr__ defined). Any idea why this may be?

[Manuel Belmadani]

Hey Ben, I've tried your code and I can't track down the error either (I can reproduce is easily by doing output(1), just not sure why though). I'm wondering if the Enum extension is causing the issue, their __repr__ seems pretty specific, but I don't know enough about DEAP internals to figure how that would play out here. 

I'm pretty sure you should be able to define non-built in inputs/output. I've done it before. Here's an example of how I did it:

pset = deap.gp.PrimitiveSetTyped("MAIN", [], SuperCustomClass, "IN")

...

pset.addPrimitive(typed_super_custom_class,
                               [CustomClass1, CustomClass2],
                               SuperCustomClass)

...

def typed_super_custom_class(cc1, cc2):
    return SuperCustomClass(cc1, cc2)

1) Create a class to set as your MAIN type

2) Add a primitive with a function calling the constructor for your main type

3) The last function is just a wrapper to return a SuperCustomClass object

4) repeat for CustomClass1 and CustomClass2

This worked for me as far as I remember, though it might not be the most elegant way to do it. I would try that, and try to get rid of the Enum-extended class and maybe use something like inheritance instead (Shape super class is a plain object, and subclasses Ellipse, Row etc. extend them. Then your typed_shaped_class( integer ) function would just create the right one depending on the integer.) I'm not saying this is the way to do it, but hopefully it might be useful to debug this error. I'm pretty puzzled about this too.

[Ben]

Thank you for the suggestions Manuel. Unfortunately, the steps still didn't work for me as I believe the problem was with adding the Ephemeral Constants which I need to achieve my task. 

I also tried removing the enum class as you mentioned and did this with a simple wrapper class i.e.

class Wrapper:
    def __init__(self, value):
        self.value = value
   
    def __str__(self):
        return "Wrapper(" + repr(self.value) + ")"
   
    __repr__ = __str__

As I don't need all the functionality subclasses offers. This still gets the same issue, even being careful with the __repr__  method. 

The shapes can be represented simply as a list of strings, which solves the problem but unfortunately, I do need this wrapper/custom classes approach to work with other inputs as well.

Thank you for your time

[Ben]

I think i've figured this out. The pset is passed to gp.compile, and the pset.context dict is used for the context of eval.

The problem is, by default if an ephemeral context is using a custom type, i.e.

pset.addEphemeralConstant("Custom", lambda: Custom() , Custom)

The lambda code ("Custom": Custom()) will be added to the dict, but the type being used will not.

It seems I may be able to solve this by adding my classes to the pset context manually (I will try this shortly), but an ideal

situation would be if this was done automatically. Is this something that could be added? or at least a function added to pset

for this. Im happy to give this a go and open a pull request. 

Is this discussion more appropriate for the github issues now?

[Ben]

Just another update (sorry I'm spamming this a bit, would be nice if you could edit old posts).

This can be manually solved as mentioned above by doing something such as the following when you are adding the ephemeral constants. 

pset.context["Custom"] = Custom

Where Custom is your new class, and "Custom" is used in your __repr__ method. 

This solution still seems a bit hacky, but it works.

If code could be implemented to do this automatically when adding terminals or functions that would be great,

but I'm sure there's additional complexities there.