How to write a data file to Beaglebone from Python

[John Baker]

I want to store a little data on the eMMC from my Python code. My program is not getting errors when I try to write a very short data file but it doesn't actually seem to write it as I cannot read it back. I am using

f = open('MyFile.txt', w)

or

f = open('/home/debian/Desktop/MyFile.txt', w)

or

f = open('MyFile.txt', r)

or

f = open('/home/debian/Desktop/MyFile.txt', r)

to open the file and am not getting any errors but Python is not finding the file and I cannot find the file with a  search from WinSCP.

So how to create a data file and where does it go?

Thanks,

John

[William Hermans]

I'm not a python developer, but actually have done this many times in C. So I can tell you this with certainty. This is done no different than it is done on any Linux system.

The point here is that you should go out and find any good python tutorial, that covers writing to a file, and follow it.

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Baker]

That's what I figured William. As far as I can tell, I am doing the open and write and read like I have with C and as described in https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files. However, I am not getting any error message when I do the open for writing and do the write, just that the file is not getting created and written. It should be so straightforward but it's not working.

John

[William Hermans]

That's what I figured William. As far as I can tell, I am doing the open and write and read like I have with C and as described in https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files. However, I am not getting any error message when I do the open for writing and do the write, just that the file is not getting created and written. It should be so straightforward but it's not working.

Does this file already exist ? You should check the documentation for the file write() method and see what all is required for the method to work correctly. Also, as I said I'm no python developer, but the code shown there is really bad form. There is no error checking on the file object when attempting the open() call. In C this would be done by checking the value of f, but I'm not even sure this is possible in python.

So I'd attempt to help you trouble shoot this by writing code myself, here, and testing. But I'm very "allergic" to python. Or more correctly, I'm getting old, and set in my ways, and python just is not in my future plans. *ever* . . .

[Paul Wolfson]

python

>>> import os

>>> os.getcwd()

'home/pwolfson'

>>>

-------------------------------------------------
Paul Wolfson, Ph.D., TX LPI, #A17473
Dallas Legal Technology
3402 Oak Grove Avenue, Suite 300-A
Dallas, Texas 75204-2353
214-257-0984 (Tel)
214-838-7220 (Fax)
Send me an email.

-------------------------------------------------
The contents of this email are confidential to the sender and the ordinary user of the email address to which it was addressed, and may also be privileged.  If you are not the addressee of the email, you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever.  If you have received this email in error, please advise the sender at  214-257-0984.  Thank you.
-------------------------------------------------

[John Baker]

Thanks Paul.
getcwd() tells me that my cwd is /home/debian/Desktop when I use f = open('MyFile.txt', w). I am using Try/Except and not getting an exception with the open() statement. Perhaps I can't write a text file to the Desktop, but it seems like it should give me an error message.
John

You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/4eMyTYs_lnw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

[John Baker]

I made a new folder /home/baker and then changed my write open() command to f = open('/home/baker/MyFile.txt', w) and the read open command to f = open('/home/baker/MyFile.txt', w) but still am not able to create and write to the MyFile.txt. Now the getcwd() command tells me that the cwd is /home/baker. Also not getting any error message even for writing or reading, just the exception to reading the file that it cannot find the file. No exception when I try to write to the file. I am running my Python program with sudo python myprogram.py. I tried changing the permissions to my /home/baker directory with "sudo chmod -R 777 /home/baker" but WinSCP still says the owner is root and still my Python program cannot write a text file to /home/baker.

So I'm at wits end (a short trip to be sure) and cannot figure out how to write the data file.

John

[William Hermans]

John, the only other thign I can think of which may be a possibility. Is that your eMMC might perhaps be in read only mode ? Which does not explain the no error message thing, but why don't you try to write a simple C app to do the same and see what happens ?

[William Hermans]

Sorry I should have added this o my last post. In C you check the file descriptor "object" for -1, and if -1 you check errno, which should be a value indicating which error you're getting.

Another thing you can check, even before trying something with C, is dmesg | tail, and see if any errors crop up there. I would not think so, but I've been surprised before . . .

[John Baker]

Thanks William but although I have done some C programming and have read and written files in C, I don't know how to get C running on the Beaglebone. Hopefully I haven't painted myself in a corner but my Python program is running well otherwise. I am doing some Fuzzy Logic control for a paint flow project, have a GUI with an animated graph using matplotlib and Tkinter, am outputting a PWM signal and measuring the filtered PWM voltage back into the controller software with the BBB ADC. It's all looking good until I decided to write a very short text file for the system gain, just a single number, but am stymied by it. I just now made my own text file using nano and see that the file is in my /home/baker folder but my Python program cannot find it, cannot read it.
It seems likely that I'm just doing something wrong, some simple thing and perhaps I simply cannot read and write from Python to my text file, but that seems crazy.
John

[William Hermans]

It seems likely that I'm just doing something wrong, some simple thing and perhaps I simply cannot read and write from Python to my text file, but that seems crazy.

The thought that you're doing something wrong did cross my mind, but I usually try to give a person a benefit of the doubt, until their skill level makes its self plain.

Maybe if you show us what you have for a source file someone can figure what's going on. I can actually read through Python quite well despite never having write a single line of python myself. The syntax is not too far off other language I've used in the past.

[John Baker]

William,
I don't know what the dmesg command should show but I don't see anything exciting except for:
CAUTION: musb: Babble interrupt occurred

Otherwise there is something about a hub, usb, ethernet libphy: PHY .... not found and IPV6: phy nnnnn not found on slave 1.

Here's some of my code. Hopefully you'll see something newbie-wrong that I've done. It's been a while since I have done file I/O, so it's likely that I have some syntax error(s).


Thanks for your help,
John

[Dennis Lee Bieber]

On Tue, 5 Apr 2016 13:43:47 -0700 (PDT), John Baker
<bakerengi...@gmail.com> declaimed the
following:

>I want to store a little data on the eMMC from my Python code. My program
>is not getting errors when I try to write a very short data file but it
>doesn't actually seem to write it as I cannot read it back. I am using
>f = open('MyFile.txt', w)
>

First problem -- the mode is a string...

f = open("MyFile.txt", "w")

>or
>f = open('/home/debian/Desktop/MyFile.txt', w)
>
>or
>f = open('MyFile.txt', r)
>
>or
>f = open('/home/debian/Desktop/MyFile.txt', r)
>
>to open the file and am not getting any errors but Python is not finding
>the file and I cannot find the file with a search from WinSCP.
>

Did you ever write anything to the file, and did you close the file?

f.write("Some junk\n")
f.close()


The first open should create the file in whatever your current working
directory is. Without connecting my BBB I can't confirm if the second is a
valid path.

And last -- how are you running it? SSH to a command line?

--
Wulfraed Dennis Lee Bieber AF6VN
wlf...@ix.netcom.com HTTP://wlfraed.home.netcom.com/

[William Hermans]

Here's some of my code. Hopefully you'll see something newbie-wrong that I've done. It's been a while since I have done file I/O, so it's likely that I have some syntax error(s).

So, I would reduce the code you have, at least in a test file, and use only the absolute minimum code for printing a text string to a file. Then of course double check to make sure it works. I would also remove the Str conversion from the write() functional call, and make that an explicitly defined string before the call to write(). The "call" Str(some numerical value) also seems very odd to me. Usually with OOP languages you have something akin to value.toString(value to convert), but again this is where my lack of python BCL( base class library ) knowledge is going to leave me at a disadvantage.

[William Hermans]

You're also not writing a new line character after your value into the file, which may or may not cause a problem.

[Chad Baker]

Isn't the open syntax => open("filename.ext","r")?

--
Chad Baker Memphis, TN

[Jonathan Smith]

If the directory /home/baker is showing as owned by root, chaos won't change ownership of it. Try chown {user.group} /home/baker.  Then, assuming you are running the application as {user} you should at least have write permissions to that directory, which you would not have of it was owned as root.

Jonathan

[Dennis Lee Bieber]

On Tue, 5 Apr 2016 19:11:32 -0700, John Baker
<bakerengi...@gmail.com> declaimed the
following:

>Here's some of my code. Hopefully you'll see something newbie-wrong that
>I've done. It's been a while since I have done file I/O, so it's likely
>that I have some syntax error(s).
>
>

To reiterate, the first thing is that ALL of your open() calls are
erroneous. The second argument to open() is a string, you seem to be
providing some object -- unless the objects r and w were earlier bound to
strings of "r" and "w", the open calls should be failing.

Since you are using bare except clauses, you could be trapping on /any/
error and never identifying what the error is.

You state you are not seeing any messages... What are the values of
"PaintFlow" and "control_output" at the start of that image (really? a
screen grab rather than text cut&paste?)? Could it be that you aren't even
entering that block because the if statement is coming up false?

As often advised on comp.lang.python -- reduce your case to the minimal
that illustrates the problem. You think you are having problems creating a
file, so create a short test program that does nothing else.

-=-=-=-=-=-

debian@beaglebone:~$ ls Desktop/
debian@beaglebone:~$ ls
Desktop build example.py io.py t.py
debian@beaglebone:~$ cat io.py
f = open("/home/debian/Desktop/junk.txt", "w")
f.write("This is\njust\nsome junk\n")
f.close()
f = open("/home/debian/Desktop/junk.txt", "r")
for i,ln in enumerate(f):
print "line #%s = '%s'" % (i,ln)
f.close()
debian@beaglebone:~$ python io.py
line #0 = 'This is
'
line #1 = 'just
'
line #2 = 'some junk
'
debian@beaglebone:~$ ls Desktop/
junk.txt
debian@beaglebone:~$ cat Desktop/junk.txt
This is
just
some junk
debian@beaglebone:~$

-=-=-=-=-
Note that reading lines does not strip the trailing new-line marker, hence
the ' showing up on a line by itself.

[John Baker]

Thanks All for your help. It is much appreciated and shows how helpful the BB Forum users are and how valuable the Forum is.

John

[John Baker]

Gosh, how did I miss that, that the mode is a string. Well it had to be
something simple like that. Quoting the w and r fixed the problem. Very
embarrassing.
:-[
Thanks,
John

[John Baker]

Now after all that I've decided not to write anything to the eMMC, fearing that I'll cause some hiccup with the writes.

John

johnbakeree.blogspot.com

[William Hermans]

Now after all that I've decided not to write anything to the eMMC, fearing that I'll cause some hiccup with the writes.

John

Eh ? What do you mean ? You do realize that you can create a tmpfs directory, that is located in memory, where you could then create files  that are also placed in memory ?

[John Baker]

William,
No, I did not know about writing to the tmpfs directory. That's a great idea. I don't know how to do it so will search for info on it.

I decided that I don't need to write these particular files but can simply save the data to RAM. However, I will need to write a log file to a USB memory stick, so the tmpfs directory should fit the bill exactly to temporarily save the data before writing to USB.

Thanks very much for the suggestion.
John
johnbakeree.blogspot.com

You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/4eMyTYs_lnw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

[William Hermans]

http://www.cyberciti.biz/tips/what-is-devshm-and-its-practical-usage.html

One caveat however. Do not use /dev/shm. /dev/shm is meant to be used for POSIX IPC shared memory. So it'd probably work just fine, but using tmpfs is much better, and the directory used can be placed anywhere in a file system( within reason ).

So, at the command line: mount -t tmpfs -o size=512m tmpfs /mnt/ramdisk

Or, in /etc/fstab: tmpfs       /mnt/ramdisk tmpfs   nodev,nosuid,noexec,nodiratime,size=1024M   0 0

Also keep in mind, again, that the file / directory can be placed any where in the systems directory structure. So if you wanted a file somewhere in your home directory called "somefile" . . .


$ mkdir ~/test
$ mount -t tmpfs -o size=256m tmpfs /home/john/test

$ touch ~/test/somefile

One additional note I would like to leave on. I've used as much as 256m for a "ramdisk" on the beaglebone, so that does work. You can perhaps make it slightly bigger, but I would not push it. Typically though for a large single application, I've found that the beagelbone will use only up to around 120m. it really depends on what the beaglebone is doing.

[William Hermans]

And actually mount will need root permissions, so chown will need to be run on the tmpfs directory if one needs user rights to the directory.

[mickeyf]

I'm not a Python developer either, 'though I know it is very popular of late. Does it say something about Python itself that it did not throw up a big error in your face when the quotes were omitted? Or is the problem elsewhere?

On Wednesday, April 6, 2016 at 11:24:18 AM UTC-7, John Baker wrote:

[Melk933]

The Python on my machine throws up an error.  

$ python testopen.py 

Traceback (most recent call last):  

File "testopen.py", line 1, in <module>     f= open("T3.txt", w) NameError: name 'w' is not defined

[Paul Wolfson]

modeString = "w"

fileNameString = "out.txt"

f = open(fileNameString, modeString)

f.write("This is a test for text output.")

f.close()

[John Baker]

Yes indeed. Python 2.7 in the Debian 3.8.13 bone50 on my BBB does flag the error, telling me that w is not correct. 

I'm not sure as I haven't gone back to my old code to try it again, but I suspect my use of Try/Except sort of hid the error, just telling me that the Try failed and not telling me exactly what the error was and I simply assumed the error was in the name or path in the open command. Clearly I should have just used the open command outside of the Try/Except and then an error message would have popped up telling me that the w was incorrect. Hopefully I have learned from this, first to double-check the syntax and second to dig a little to find out why the open command (or other command) didn't work. I continue to be embarrassed at this dumb mistake but I am going on to probably make other mistakes. However, my code is working perfectly, GUI and all, including writing a logfile to the eMMC. 

Thanks for everyone's help. I certainly needed it.

John

[William Hermans]

Yes indeed. Python 2.7 in the Debian 3.8.13 bone50 on my BBB does flag the error, telling me that w is not correct. 

I'm not sure as I haven't gone back to my old code to try it again, but I suspect my use of Try/Except sort of hid the error, just telling me that the Try failed and not telling me exactly what the error was and I simply assumed the error was in the name or path in the open command. Clearly I should have just used the open command outside of the Try/Except and then an error message would have popped up telling me that the w was incorrect. Hopefully I have learned from this, first to double-check the syntax and second to dig a little to find out why the open command (or other command) didn't work. I continue to be embarrassed at this dumb mistake but I am going on to probably make other mistakes. However, my code is working perfectly, GUI and all, including writing a logfile to the eMMC. 

Thanks for everyone's help. I certainly needed it.

Usually, with try / catch blocks, you're given and error object back, from which you *can* print out the error.message text / character object. So unless you had something like . .

try{
. . .

} catch(e){

  print(e.message);

}

You were using the try catch block incorrectly ;)

[Dennis Lee Bieber]

On Thu, 14 Apr 2016 16:40:39 -0700 (PDT), John Baker
<bakerengi...@gmail.com> declaimed the
following:

>should have just used the open command outside of the Try/Except and then

>an error message would have popped up telling me that the *w *was

>incorrect. Hopefully I have learned from this, first to double-check the

The first thing to learn is: never use a bare "except" for production
code. If you are going to wrap something in "try/except" you should have
some idea of which exceptions might be raised and therefore should be named
in the "except" statement. That way, you intercept only the ones you expect
might be seen and for which you are prepared to do corrective action.

If the only purpose of the "except" is to print some simplified
message, a la "something failed", then you are better off to not use a
"try/except" block at all, and let the exception kill the program and dump
a traceback that would tell you what failed.

[John Baker]

Thanks All for the help and suggestions on Try/Catch and other. More evidence of the especially helpful folk on the Beagleboard users forum.
John

You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/4eMyTYs_lnw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

[William Hermans]

Thanks All for the help and suggestions on Try/Catch and other. More evidence of the especially helpful folk on the Beagleboard users forum.
John

No problem John, and hopefully we weren't too obvious. The key point one should take away from this whole discussion though is: Whenever in code *something* could fail, it needs to be tested for. Then if you're debugging, you should at minimum implement a method to print out that something failed, and why. In C, this is often referred to as printf() debugging, and actually can be used to trouble shoot most program errors.

In the case where you're forced to use try / catch blocks however. Remove these try  / catch blocks when done debugging *IF* you can. Some cases, in some languages, you're all but forced to use try / catch blocks for things such as checking if a network connection exists. Anything really, that your application may need, but may not necessarily be active at any given point in time - While your application is running.