Error: No space left on device

9 views
Skip to first unread message

Quinton Heney

unread,
Jul 20, 2021, 6:35:20 PMJul 20
to Machine Learning for Kids
Could you help with this error (causes model training to fail after a few seconds for an image model):

Error: wbuffer_write: write failed unix error: No space left on device vips2png: unable to write to target target

Only some of the students are seeing this error when training their models. Is it an issue with one of the image files in the training "buckets"?


Dale Lane

unread,
Jul 20, 2021, 7:23:19 PMJul 20
to Machine Learning for Kids
Sorry about that. 

That was an error on my servers - the error referred to a failure on my side when my code tried to resize their training image(s) to be the right size for training a model.

I can see from the logs on my side that two of the three instances of the ML4K servers had this problem - which is why some of your students will have been okay. They will have used the third server which was okay. (Requests are randomly distributed across all three instances)).

I do have some automation that looks out for this sort of problem, and from the logs on my side I can see that it was triggered 18 minutes ago - that restarted the two servers that had the problem. This will have resolved the problem at that time.

But I'm sorry for the inconvenience it caused your students in the meantime. 

I'll look into what I can do to prevent that happening again in future.

Kind regards

D

Reply all
Reply to author
Forward
0 new messages