Joseph Coffey wrote: > Has anyone else encountered issues writing to a file in reinteract?
> When I try to do this in a worksheet, I get a Value error, telling me > that I'm writing to a closed file. It works fine in a library.
I'm seeing this too. It's a bit weird, but I think I've worked out what's going on.
Firstly, if you enclose the file commands in another block, everything works just fine. For example, this always works for me: if True: f = file('temp.txt', 'w') f.write('Hi!\n') f.close() Enclosing the output commands in a try block, with f.close() in the finally block, is probably not a bad idea anyway, so this could be a usable work-around.
I also noticed that if I capture the output of the write command in another variable, I never get a ValueError. I think this is due to how Reinteract tracks modification of objects. I believe that when Reinteract sees something like "f.write('blah')", it assumes that the command will modify f. Therefore, before executing it, Reinteract makes a copy of f and executes write() on that copy. However, copy.copy() on an open file object returns a closed, uninitialized file object. Trying to write to this object raises the ValueError. On the other hand, when Reinteract sees a statement like "g = f.write('blah')", it assumes that write() does not modify f, so no copy is made.
(It may be worth noting that the same thing happens when reading files. f = file('temp.txt', 'r') f.read() raises a ValueError, while f = file('temp.txt', 'r') s = f.read() does not. The same logic applies.)
What I don't understand is that sometimes f = file('temp.txt', 'w') g = f.write('Hi!\n') f.close() fails to actually write to the file temp.txt, even though it executes without error. (Instead, I get an empty file.) I haven't been able to figure out why it works sometimes and not other times. Opening the file for both reading and writing and conducting a read operation seems to ensure that the write succeeds. Perhaps someone who actually understands file operations can explain this part.
Anyway, my feeling is that the best work-around is to enclose all the file output commands in a block. Besides fixing this problem, it will also help ensure that the file was opened before attempting to write to it.
On Thu, Jul 16, 2009 at 6:58 PM, Robert Schroll <rschr...@gmail.com> wrote:
> Joseph Coffey wrote: >> Has anyone else encountered issues writing to a file in reinteract?
>> When I try to do this in a worksheet, I get a Value error, telling me >> that I'm writing to a closed file. It works fine in a library.
I think it's worth pointing that there is a conceptual issue related to fitting file I/O into the Reinteract framework.
If I have:
f = open("somefile.txt", "w"); f.write("a\n"); f.write("b\n"); f.close();
there's no way I can execute just some of those lines and have things work out right. If I change f.write("b\n"); to f.write("c\n"); then the file needs to end up with:
a c
Not:
a b c
> I'm seeing this too. It's a bit weird, but I think I've worked out > what's going on.
> Firstly, if you enclose the file commands in another block, everything > works just fine. For example, this always works for me: > if True: > f = file('temp.txt', 'w') > f.write('Hi!\n') > f.close() > Enclosing the output commands in a try block, with f.close() in the > finally block, is probably not a bad idea anyway, so this could be a > usable work-around.
This would be my suggestion as well - make writing the file atomic from Reinteract's perspective. It also bypasses the problem with being unable to copy a file since if 'f' isn't a defined variable before the block is executed, Reinteract won't try to copy it.
> I also noticed that if I capture the output of the write command in > another variable, I never get a ValueError. I think this is due to how > Reinteract tracks modification of objects. I believe that when > Reinteract sees something like "f.write('blah')", it assumes that the > command will modify f. Therefore, before executing it, Reinteract makes > a copy of f and executes write() on that copy. However, copy.copy() on > an open file object returns a closed, uninitialized file object. Trying > to write to this object raises the ValueError. On the other hand, when > Reinteract sees a statement like "g = f.write('blah')", it assumes that > write() does not modify f, so no copy is made.
Exactly. But see above for why this isn't really a very satisfactory workaround for file I/O. File I/O has side effects on the outside world (the magnetization patterns on the hard disk), so neither copying nor not copying actually works out.
> What I don't understand is that sometimes > f = file('temp.txt', 'w') > g = f.write('Hi!\n') > f.close() > fails to actually write to the file temp.txt, even though it executes > without error. (Instead, I get an empty file.) I haven't been able to > figure out why it works sometimes and not other times. Opening the file > for both reading and writing and conducting a read operation seems to > ensure that the write succeeds. Perhaps someone who actually > understands file operations can explain this part.
I think it's just that f.close() is also a mutation, so again the f.close() is happening on a copy, so the f.write() is never flushed. It seems to work reliably (with the limitations discussed above) for me if I do:
h = f.close();
I think the long-term solution here is to have some way of "annotating" standard methods, so Reinteract would know:
>>> m = re.search("b+", "abbbb") >>> m.group(0) # Not a mutation, no need to copy >>> f = file('temp.txt', 'w') >>> f.write('Hi!\n') # A mutation, can't copy, need to re-execute from the point f was initially assigned
FYI: I opened Trac ticket #91 about this. Although the ticket links to this thread, I forgot to link the thread back to the ticket. So here it is: http://www.reinteract.org/trac/ticket/91
Owen Taylor wrote:
> On Thu, Jul 16, 2009 at 6:58 PM, Robert Schroll <rschr...@gmail.com> wrote:
>> Joseph Coffey wrote:
>>> Has anyone else encountered issues writing to a file in reinteract?
>>> When I try to do this in a worksheet, I get a Value error, telling me
>>> that I'm writing to a closed file. It works fine in a library.
> I think it's worth pointing that there is a conceptual issue related
> to fitting file I/O into the Reinteract framework.
> there's no way I can execute just some of those lines and have things
> work out right. If I change f.write("b\n"); to f.write("c\n"); then
> the file needs to end up with:
> a
> c
> Not:
> a
> b
> c
>> I'm seeing this too. It's a bit weird, but I think I've worked out
>> what's going on.
>> Firstly, if you enclose the file commands in another block, everything
>> works just fine. For example, this always works for me:
>> if True:
>> f = file('temp.txt', 'w')
>> f.write('Hi!\n')
>> f.close()
>> Enclosing the output commands in a try block, with f.close() in the
>> finally block, is probably not a bad idea anyway, so this could be a
>> usable work-around.
> This would be my suggestion as well - make writing the file atomic
> from Reinteract's perspective. It also bypasses the problem with being
> unable to copy a file since if 'f' isn't a defined variable before the
> block is executed, Reinteract won't try to copy it.
>> I also noticed that if I capture the output of the write command in
>> another variable, I never get a ValueError. I think this is due to how
>> Reinteract tracks modification of objects. I believe that when
>> Reinteract sees something like "f.write('blah')", it assumes that the
>> command will modify f. Therefore, before executing it, Reinteract makes
>> a copy of f and executes write() on that copy. However, copy.copy() on
>> an open file object returns a closed, uninitialized file object. Trying
>> to write to this object raises the ValueError. On the other hand, when
>> Reinteract sees a statement like "g = f.write('blah')", it assumes that
>> write() does not modify f, so no copy is made.
> Exactly. But see above for why this isn't really a very satisfactory
> workaround for file I/O. File I/O has side effects on the outside
> world (the magnetization patterns on the hard disk), so neither
> copying nor not copying actually works out.
>> What I don't understand is that sometimes
>> f = file('temp.txt', 'w')
>> g = f.write('Hi!\n')
>> f.close()
>> fails to actually write to the file temp.txt, even though it executes
>> without error. (Instead, I get an empty file.) I haven't been able to
>> figure out why it works sometimes and not other times. Opening the file
>> for both reading and writing and conducting a read operation seems to
>> ensure that the write succeeds. Perhaps someone who actually
>> understands file operations can explain this part.
> I think it's just that f.close() is also a mutation, so again the
> f.close() is happening on a copy, so the f.write() is never flushed.
> It seems to work reliably (with the limitations discussed above) for
> me if I do:
> h = f.close();
> I think the long-term solution here is to have some way of
> "annotating" standard methods, so Reinteract would know:
>>>> m = re.search("b+", "abbbb")
>>>> m.group(0) # Not a mutation, no need to copy
>>>> f = file('temp.txt', 'w')
>>>> f.write('Hi!\n') # A mutation, can't copy, need to re-execute from the point f was initially assigned