Teachable machine model not work correctly in Scratch3

[kk chan]

I have made several models and open them in different Scratch3 files.

None of them work properly.

In my model, there are two classes: Drink and Not Drink.

Each class there are over 300 photos. After training, the results are very good in Teachable machine page.

After the model is exported and opened in Scratch3, the results are 100% incorrect.

I tested several models and none of them work properly.

[Dale Lane]

I would be happy to investigate this for you, however I would need a little more detail to go on then "not work".

For example, can you provide an example Scratch project that demonstrates the problem that you're seeing, please?

Or an example of a Teachable Machine model that you think doesn't work?

Or a specific error message? 

Or something I could investigate?

Kind regards

Dale

[CX Aviation]

https://teachablemachine.withgoogle.com/models/DPi0tMG6v/
I trained a model as above and open it in Scratch.
The model has two labels. Phone and vacuum.
In Scratch, only "vacuum" is displayed even there is no phone nor vacuum in front of the web cam.

I have another model for "drink" and "not drink".

Similar situation, only "drink" is shown.

[Dale Lane]

> The model has two labels. Phone and vacuum.
> In Scratch, only "vacuum" is displayed even there is no phone nor vacuum in front of the web cam.

That is what I would expect, as if the model only has two labels then those are the only two responses that the model can give for any image. 

The model will only ever return "phone" and "vacuum". Hopefully if it's well-trained, you'd expect to see a low confidence score for images that don't resemble either of them (and you can use the confidence block to filter out such responses)

> I have another model for "drink" and "not drink".

> Similar situation, only "drink" is shown.

Is your code above on the sprite or the backdrop? 

It should really be on the backdrop.

If it's on the sprite, unless the sprite is very large, that would be a reason why you'd see unexpected results. 

[Dale Lane]

Let me try to explain that a little better.

-------------------------------------------------------------------------

Consider this Scratch project:

That green "costume image" sprite will be returning this image:

In other words, the image for the current costume of the sprite, regardless of whether the sprite is currently hidden, or if another sprite happens to be obscuring it. 

It is the current costume image for the sprite where the code block is running.

-------------------------------------------------------------------------

Consider this Scratch project:

That green "backdrop image" block will be returning this image:

In other words, it shows what is currently visible on the backdrop at the location of the sprite.

It will include any other sprites that are visible, and any current backdrop.

-------------------------------------------------------------------------

Consider this Scratch project:

That green "backdrop image" block will be returning this image:

In other words, it returns the current contents of the Stage, including the backdrop and all visible sprites. 

-------------------------------------------------------------------------

Does that help?

Which of those are you using in your project?

Kind regards

D

[kk chan]

Hi Dale,
I don't really understand.
My plan is to make a game with a backdrop and a cat.
I have trained a model with two labels by Teachable Machine. They labels are "Phone" and "Vacuum". It works fine on the page of Teachable Machine.
Then it is added to Scratch 3. My plan is, when there is a phone in front of the web cam, the cat moves right for 5 steps. When there is a vacuum, move the cat to the left for 5 steps. 
In my previous tests, only "vacuum" is returned and the confidence is over 90.  Another problem is, once it is added to Scratch 3, the game backdrop cannot be shown.

[kk chan]

This is the program and my demo. 

But it shows the first label all the times. I tried different models and find that only the first model label are returned.

On the other hand, I want to make a game backdrop there. Looks like it is impossible as the web cam image is loaded as a backdrop here.

Under Teachable Machine, it works fine.

This is how I upload the trained model

Finally, i open it like below.

I did all projects in the same way. But none of them works, only the first label is returned.

[Dale Lane]

I don't think the problem is with the Teachable Machine model, or how you're adding it to Scratch.

I think the problem is with what you're giving to your Teachable Machine model to recognize.

There are three image blocks you can use to get an image for your model to recognize.

From my last post:

A:      B:      C: 

From the screenshots of your code, I can see that you're using B.

Based on your description of what you're trying to do, I think you should be using C.

I've created a quick Scratch project to give you an example of how to do this which is attached.

Kind regards

D