OSError', OSError(24, 'Too many open files')

31 views

Skip to first unread message

unread,

Jun 5, 2024, 1:12:59 PM6/5/24

to idtracker.ai users group

I am getting this error persistantly in the 3rd Protocol. This time it was almost at the 90% pretraining mark.
88.65% of the images have been used during pretraining tracker.py:256
(if higher than 90.00% we stop pretraining)
New pretraining iteration, using the #16 global fragment tracker.py:241
Loading identification images from disk (24 pre_trainer.py:32
iterations). It took 0:00:00
Creating training dataloader with 47211 images labeled train.py:225
with 11 distinct classes
Creating validation dataloader with 5251 images labeled train.py:225
with 11 distinct classes
Reinitializing only fully connected layers models.py:56
Entering the training loop... train.py:115
12:51:58 ERROR An error occurred, saving data before printing run.py:74
traceback and exiting the program
Saving Session object in /Users/gilraitses/Documents/Mirna run.py:78
Lab/videos/session_Gr21a@Chrimson(3)_T_Re_Sq_219to436P_15_2_3
#T_Bl_Sq_2to7P_15_1_3_202402131442_3x_1/session.json
ERROR Error while saving data: [Errno 24] Too many open run.py:108
files: <'file.path'>

Here is the traceback log:

--- Logging error ---

Traceback (most recent call last):

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/run.py", line 43, in track_video

self.list_of_fragments = tracker_API(

^^^^^^^^^^^^

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/tracker/__init__.py", line 52, in tracker_API

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/tracker/tracker.py", line 51, in track

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/tracker/tracker.py", line 139, in accumulation_protocol

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/tracker/tracker.py", line 246, in pretrain

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/tracker/pre_trainer.py", line 78, in pretrain_global_fragment

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/network/train.py", line 119, in train_loop

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/idtrackerai/base/network/train.py", line 163, in evaluate

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 434, in __iter__

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 387, in _get_iterator

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1022, in __init__

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/multiprocessing/context.py", line 103, in Queue

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/multiprocessing/queues.py", line 49, in __init__

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/multiprocessing/context.py", line 88, in BoundedSemaphore

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/multiprocessing/synchronize.py", line 152, in __init__

File "/opt/anaconda3/envs/idtrackerai/lib/python3.11/multiprocessing/synchronize.py", line 57, in __init__

OSError: [Errno 24] Too many open files