Prepare CSV file with irregular number of columns using Pandas

Skip to first unread message


Feb 11, 2022, 10:20:33 AMFeb 11
to PyData

I am trying to prepare a Pandas dataset for a Machine Scheduling problem, the data entries I am using have been generated through iterations; meaning that we do iterate as follow:

5 Jobs with 2..3..4 Machines, then 6 Jobs with 2..3..4 Machines, then 7 Jobs with 2..3..4 Machines, etc..

My problem is : what should I do with the "empty columns" in smaller iterations ? Meaning that for the instances where Job = 5 for example, I only have 5 Columns, but for the instances where Job = 8 I have 8 Columns.. how to deal with the "missing" 3 values for the Jobs having 5 instances only ?

I know Pandas requires certain uniformity in the Dataset structure, which I cannot respect here, shall I make all remaining values NAN ? will this false the result and make the training process inaccurate ? Any solution to this ?

The picture should give a better understanding on what the data looks like :

scheduling Problem definition.JPG

Reply all
Reply to author
0 new messages