os.path.exists thread safety?

893 views
Skip to first unread message

vapirix

unread,
May 26, 2011, 6:50:31 PM5/26/11
to web2py-users
I read in the docs about request.folder and not changing directory
with os.path. Since I have literally no idea how and what in python is
(and is not) thread safe, is the following thread safe? Do I need to
be using os.path.join anywhere in there?

if not os.path.exists(images+directory+resource): # <-- obviously this
being my question
image = Image.open(images+resource)
imgScaled = ImageOps.fit(image, (w,
h,),method=Image.ANTIALIAS)
imgScaled.save(images+directory+resource, 'jpeg')

If it isn't, kindly point me in the right direction. =D Thanks, guys.

ron_m

unread,
May 26, 2011, 9:05:21 PM5/26/11
to web...@googlegroups.com
os.path.join takes into account the path separation character - UNIX = '/' Windows = '\' which is why you should use it for when you want to run on both system types even if you don't expect to right now.

os.path.exists internally does a stat() of the file to determine if it exists. There are cases listed in the docs where it returns false such as broken symbolic links. If another thread removes the file right after you get a true return then your next operation will fail. The best option is to use try except blocks and gracefully handle the file went away errors in the except section and re-raise errors you can't handle there. On UNIX once you get the file open the file will remain accessible through that open file descriptor even if another thread removes the file which is actually completed by just removing the directory name-inode pair. No other process will ever open the file again once all the directory links are gone and the final close will clean up the contents and release the resources back to the free pool on the file system. Conversely, on Windows you can't remove a file open to a process is my understanding from using that system.

I digress but one of the hardest things to find on a UNIX system is a process writing blocks to a file which has been already removed from all directories. That file will continue to grow as long as the process writes to it and the blocks remain allocated as long as the process stays alive. The sys admin can see the file system free space disappearing but will have trouble finding out why because this file won't show up on du reports. In this case your friend is lsof. This isn't a bash of UNIX, I much prefer that system type, just one of the things that many people using it don't realize.

Reply all
Reply to author
Forward
0 new messages