Multiprocessing using python shows error “I/O operation on closed file”

[Aurora Eugene]

Hi All, 

I Am new to concepts of programming, just started learning with Python. Started writing a program for multiprocessing using Python . The program is to print contents of CSV file parallel. 

CSV file contains :

folder path, EXTENSION 

for example 

C:\users , .CSV

C:\Windows, .PDF 

etc

so, the CSV file will contain more than 200 folder paths and their extensions. What I have to do in code is I should loop through the CSV file and print the contents of the CSV file parallelly by assigning it to workers in a POOL 

So below is my Code, but it throws "I/O operation on closed file", Can anyone help me to figure out this issue?

Thanks in Advance. 

Code :

def folderStatistic(t):
    j, dir_name = t
    print(t) # just need to print contents of CSV file here . Dynamically
    #have furthur operation to be performed using extension and filename

def get_directories():
    with open('CONFIG.csv') as f:
        reader = csv.reader(f)
    return [col for row in reader for col in row]

def folderstatsMain():
    freeze_support()
    start = time.time()
    pool = Pool()
    worker = partial(folderStatistic)
    pool.map(worker, enumerate(get_directories()))

def datatobechecked():
    try:
        folderstatsMain()
    except Exception as e:
        # pass
        print(e)

if __name__ == '__main__':
    datatobechecked()

[Joel Tanko]

For someone new to programming, that's a bit complicated for you don't you think?

Your problem lies in your "with" block

  with open('CONFIG.csv') as f:
    reader = csv.reader(f)
    return [col for row in reader for col in row]

1. You can only return once and then the function is exited, even if your code worked you'd get a generator object in a list

2. reader isn't accessible outside the with context, same as every context variable

You can rewrite it as

1.  

ls = []

with open('CONFIG.csv') as f:

    reader = csv.reader(f)

        ls.append(col for row in reader for col in row)

return ls

2. 

file = open('CONFIG.csv')
        reader = csv.reader(file)

    return [col for row in reader for col in row]

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFPSiMbseEq02W4S3F613H%3Denuk1TJ4Lac7CFodWuaO6iGE3hA%40mail.gmail.com.

[Aurora Eugene]

Hi , 

I updated the code , and currently is working fine 

Anyway thank you for your help :) 

Regards , 

Aurora 

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFPSiMbseEq02W4S3F613H%3Denuk1TJ4Lac7CFodWuaO6iGE3hA%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJ4Kmg7fy7ZS7dCaUpHLeDF8pvxOpAks2YQrpNPqinWuA89T%2BA%40mail.gmail.com.