Looking for file transfer tool with XMODEM on Linux
D. M. Herrmann
I am looking for a simple solution for receiving files with the XMODEM
protocol with a conventional V.32bis Modem on Linux.
Here's a short description of the necessary behaviour:
* The modem is waiting for a call.
* The modem answers a call when the line rings. Handshaking with the remote
modem takes place. The remote modem is a GSM modem sending with 9.6 kbps. It
receives a specific data package (about 76 KB) and hangs up.
* Hi-level Protocol for the file transfer will be XMODEM-1K (the only
protocol supported by the remote GSM modem)
* The data package is saved to disk into a unique filename
* The modem regains waiting state and is ready for another incoming
connection.
No linux user authentication shall take place. So, I believe I cannot use
mgetty in this case.
The process has to be carried out automatically, so a scriptable terminal
program might work. I have considered using minicom or kermit but I cannot
figure out whether these programs are fully scriptable to meet my needs.
Does anybody have an idea what tools are necessary here and how they are to
be combined?
Regards, Dominik
Frank da Cruz
: I am looking for a simple solution for receiving files with the XMODEM
minicon, no. Kermit, yes:
http://www.columbia.edu/kermit/ckscripts.html
C-Kermit does not support Xmodem protocol but it can "load" external
protocols, e.g. rzsz, lrzsz, or crzsz.
- Frank
Alan Connor
Minicom calls whatever tools are needed and then gets out of the way:
zmodem, ymodem, gkermit, and xmodem for sending and receiving files on
my system as it is currently configured.
It also has a scripting utility that comes with it called 'runscript'.
AC
Floyd L. Davidson
>
>I am looking for a simple solution for receiving files with the XMODEM
>protocol with a conventional V.32bis Modem on Linux.
>
>Here's a short description of the necessary behaviour:
>* The modem is waiting for a call.
>* The modem answers a call when the line rings. Handshaking with the remote
>modem takes place. The remote modem is a GSM modem sending with 9.6 kbps. It
>receives a specific data package (about 76 KB) and hangs up.
>* Hi-level Protocol for the file transfer will be XMODEM-1K (the only
>protocol supported by the remote GSM modem)
The modem is not what determines the transfer protocol. Perhaps
you just mean that whatever is controlling the modem uses Xmodem-1k
protocols?
>* The data package is saved to disk into a unique filename
>* The modem regains waiting state and is ready for another incoming
>connection.
>
>No linux user authentication shall take place. So, I believe I cannot use
>mgetty in this case.
Hmmm... I haven't looked at that for a *long* time. It might
be that you can use mgetty to configure the local modem, and
have it immediately pass any connected state off to another
process. Normally that process would be the login program, but
in this case it would something else...
The something else that you want is rz, which is part of the
zmodem software and will already be installed on almost any
Linux box.
>The process has to be carried out automatically, so a scriptable terminal
>program might work. I have considered using minicom or kermit but I cannot
>figure out whether these programs are fully scriptable to meet my needs.
As others are indicating, you can use various "modem" programs
to start rz, but I fail to see any advantage to that at all.
Whatever, both minicom or kermit could certainly be used, though
it probably is just an unnecessary complication.
--
FloydL. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl...@barrow.com
Alan Connor
Hey Floyd. Haven't seen you around in a while.
Isn't minicom or a similar program needed to establish the basic
connection before passing off control to a utility like zmodem?
AC
D. M. Herrmann
Yes, I came accross that already. But as far is I understood, this runscript
utility comes into play at the moment a connection has been established. But
before it comes to this there are several tasks which are to be carried out
(starting minicom non-interactively from a script, modem initialisation)...
Any further ideas?
Dominik
D. M. Herrmann
>> I am looking for a simple solution for receiving files with the
>> XMODEM protocol with a conventional V.32bis Modem on Linux.
>>
>> Here's a short description of the necessary behaviour:
>> * The modem is waiting for a call.
>> * The modem answers a call when the line rings. Handshaking with the
>> remote modem takes place. The remote modem is a GSM modem sending
>> with 9.6 kbps. It receives a specific data package (about 76 KB) and
>> hangs up. * Hi-level Protocol for the file transfer will be
>> XMODEM-1K (the only protocol supported by the remote GSM modem)
>
> The modem is not what determines the transfer protocol. Perhaps
> you just mean that whatever is controlling the modem uses Xmodem-1k
> protocols?
Yes, you are definitely right. The higher level of this device makes use of
XMODEM-1K, so I have to implement it here.
>> * The data package is saved to disk into a unique filename
>> * The modem regains waiting state and is ready for another incoming
>> connection.
>>
>> No linux user authentication shall take place. So, I believe I
>> cannot use mgetty in this case.
>
> Hmmm... I haven't looked at that for a long time. It might
> be that you can use mgetty to configure the local modem, and
> have it immediately pass any connected state off to another
> process. Normally that process would be the login program, but
> in this case it would something else...
>
> The something else that you want is rz, which is part of the
> zmodem software and will already be installed on almost any
> Linux box.
One further question: How does the rx tool (or the rz tool respectively)
know which ttyS[x] to use for the file transfer? There are no command line
parameters dealing with this issue...
>> The process has to be carried out automatically, so a scriptable
>> terminal program might work. I have considered using minicom or
>> kermit but I cannot figure out whether these programs are fully
>> scriptable to meet my needs.
>
> As others are indicating, you can use various "modem" programs
> to start rz, but I fail to see any advantage to that at all.
> Whatever, both minicom or kermit could certainly be used, though
> it probably is just an unnecessary complication.
I get your point. a respawning mgetty process comes very close to what I
need so I would be pleased to learn how to use it for this problem.
Thanks for your help so far.
Dominik
Alan Connor
Can't you just call minicom from a bash script? It initializes the
modem automatically and will, with the right commandline arguments
also dial a number automatically.
#!/bin/sh
minicom -d <alias>
(alias from the dialing directory in minicom)
And have the "E" entry in the dialing directory's Edit menu run
your script. That runscript can simply call a bash program with
the "!" argument...
Is that a 'further idea'?
AC
D. M. Herrmann
> Can't you just call minicom from a bash script? It initializes the
> modem automatically and will, with the right commandline arguments
> also dial a number automatically.
>
> #!/bin/sh
>
> minicom -d <alias>
>
> (alias from the dialing directory in minicom)
>
> And have the "E" entry in the dialing directory's Edit menu run
> your script. That runscript can simply call a bash program with
> the "!" argument...
I hope I got your point right, please correct me otherwise.
The problem with your idea is that I am sitting on the "local" station
(which receives the call) - dialling out doesn't happen here. The local
modem is just sitting around and waiting for a call from a remote station.
So, I guess, auto dialing won't help me much on the local station.
The "remote" station (which sends the data over the line to me) is not under
my control. It already works pretty fine and I am able to receive data with
a terminal program interactively/manually.
But I haven't checked the dialing directory yet, perhaps your idea will make
sense if I have had a look on it.
There remains the looping problem: After a successful file reception the
modem has to prepare for the next one.
So minicom would have to behave as follows:
Initialize the modem - Wait for call - receive call - handshaking - receive
some binary data - save received binary data to file - start from the
beginning (initialize the modem/wait for call).
A second process would wait simultaneously for the binary data to become
evident on disk and fetch it from there, erasing the file afterwards so it
can be replaced when the next call comes in.
I hope the scenario becomes clear gradually.
Regards, Dominik
Alan Connor
Not real familiar with using minicom to *receive* calls, but it seems like
you'd run minicom with a runscript script (called from the commandline with
-S the script will run before [almost] anything else happens) having it wait
for the call with an 'expect' command:
(from man runscript)
expect {
pattern [statement]
pattern [statement]
[timeout <value> [statement] ]
then take it from there when the expected string was received.
The examples in /usr/share/doc/minicom/demos/ (debian) are
pretty helpful.
Apparently you already know how to handle the login, the handshake, so
writing the script shouldn't be hard. I'm not clear on whether the
expect, waiting, part would need to loop with a short sleep between
iterations, or not. Will do some more reading and playing around.
You'll probably figure it out before I do.
I am beginning to think that you would be better off just having it
login as a regular user and replace the shell in /etc/passwd with
a script run in a bash shell called in restricted mode.
Sound familiar?
:-)
Later,
AC
Michael W. Cocke
xmodem will be mind-numbingly inefficient on a v.32 modem. Do you have
the option of using a different protocol? zmodem is available for
every platform I've ever seen and is a much better choice.
Mike-
--
If you're not confused, you're not trying hard enough.
--
Please note - Due to the intense volume of spam, we have installed
site-wide spam filters at catherders.com. If email from you bounces,
try non-HTML, non-encoded, non-attachments,
----== Posted via Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeed.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= 19 East/West-Coast Specialized Servers - Total Privacy via Encryption =---
D. M. Herrmann
>>>>>>> I am looking for a simple solution for receiving files with the
>>>>>>> XMODEM protocol with a conventional V.32bis Modem on Linux.
>>>>>>>
>>>>>>> Here's a short description of the necessary behaviour:
>>>>>>> * The modem is waiting for a call.
>>>>>>> * The modem answers a call when the line rings. Handshaking with
>>>>>>> the remote modem takes place. The remote modem is a GSM modem
>>>>>>> sending with 9.6 kbps. It receives a specific data package
>>>>>>> (about 76 KB) and hangs up. * Hi-level Protocol for the file
>>>>>>> transfer
>>>>>>> will
>>>>>>> be XMODEM-1K (the only protocol supported by the remote GSM
>>>>>>> modem)
>>>>>>> * The data package is saved to disk into a unique filename
>>>>>>> * The modem regains waiting state and is ready for another
>>>>>>> incoming connection.
>>>>>>>
>>>>>>> No linux user authentication shall take place. So, I believe I
>>>>>>> cannot use mgetty in this case.
>>>>>>> The process has to be carried out automatically, so a scriptable
>>>>>>> terminal program might work. I have considered using minicom or
>>>>>>> kermit but I cannot figure out whether these programs are fully
>>>>>>> scriptable to meet my needs.
> [...]
>
>
> Not real familiar with using minicom to *receive* calls, but it seems
> like you'd run minicom with a runscript script (called from the
> commandline with -S the script will run before [almost] anything else
> happens) having it wait for the call with an 'expect' command:
>
> (from man runscript)
>
> expect {
> pattern [statement]
> pattern [statement]
> [timeout <value> [statement] ]
>
>
> then take it from there when the expected string was received.
>
> The examples in /usr/share/doc/minicom/demos/ (debian) are
> pretty helpful.
So I am going to go looking for the following things:
* Writing a runscript, which does essentially the following:
- Initialize Modem with AT-Command-String
- Wait for RING (no idea how this is done in a runscript, though)
-- Answer call
-- Receive all the incoming binary data into a file at once
-- [hangup]
-- quit minicom (hopefully possible, too in the runscript)
* Minicom would be called from a shell / perl script in a neverending loop
with -S <runscript>.
* Questions:
How can I quit minicom from a runscript (AFAIR it is "Ctrl-A, X" in
interactive mode)?
How to wair for the incoming connection?
How to start file reception without having to enter a filename manually into
the interactive dialog box?
Or: Is it possible to redirect all incoming data into another shell script
which would be responsible for storing the data on disk?
Someone already pointed out that one could run a shell script with "!" from
a runscript.
> Apparently you already know how to handle the login, the handshake, so
> writing the script shouldn't be hard. I'm not clear on whether the
> expect, waiting, part would need to loop with a short sleep between
> iterations, or not. Will do some more reading and playing around.
Thanks a lot! Well, as I pointed out earlier there is no need for a login in
the prototype. The problem is that I cannot easily change the remote station
so it would simulate the typical Unix login handshaking with
username/password. The remote station is rather dumb and only capable of
sending out the data package...
Help in writing the runscript would be appreciated of course, because time
is not my friend (as always).
> You'll probably figure it out before I do.
>
> I am beginning to think that you would be better off just having it
> login as a regular user and replace the shell in /etc/passwd with
> a script run in a bash shell called in restricted mode.
Would be a great method, but as stated earlier I cannot make the remote
station send a username at the moment. This method would be something for
the future.
Some more hints and I might be able to get it going with minicom.
Regards, Dominik
Bill Marcum
<do...@exonet.de> wrote:
>
> The problem with your idea is that I am sitting on the "local" station
> (which receives the call) - dialling out doesn't happen here. The local
> modem is just sitting around and waiting for a call from a remote station.
> So, I guess, auto dialing won't help me much on the local station.
>
the letter O).
--
The truth you speak has no past and no future. It is, and that's all it
needs to be.
Alan Connor
I'm looking at getty(1). You can send an init string to the modem
on the commandline (like Bill Marcum's suggestion) with the -I
option and call a non-standard login program with the -l option.
Seems to me that Floyd is right and that minicom is just a useless
extra layer here.
Calling a bash script with the -l option that invokes rx and does
the file manipulation and resets the modem seems like it should
work.
I've never used rx but will go over the manpage and docs when I
get the chance.
Using netcat to just accept whatever shows up, regardless of protocol,
and dump it in a file might also be an option. Nc is very adaptable
to bash scripts.
It might help if you posted the details of the handshake.
AC
Alan Connor
Some more clues:
/sbin/getty -w -I 'ATE0Q1&D2&C1S0=1\015' 115200 ttyS?
to initialize the modem and tell it to answer any calls after the first ring.
The sending computer should send a sx command that activates rx.
This might work (in /etc/inittab):
/sbin/getty -w -I 'ATE0Q1&D2&C1S0=1\015' 115200 ttyS? -l script
#!/bin/sh
# script
/usr/bin/rx [-a dos2unix || -b binary(raw)] filename && exit 0
That should get the file and store it in under that filename then
kill getty which will respawn automatically. May be necessary to
kill the getty by its PID.
(i think)
lrzsz is available as a binary package on deb and rh. It has all 6
protocols sx rx sz rz sy ry.
AC
D. M. Herrmann
Your proposal works great!
I can establish a connection to the modem with the getty process attached
and can easily receive a file afterwards.
The getty command line is:
getty -w -i -n -I 'ATE0Q1&D2&C1S0=1\015' 115200 /dev/ttyS0 -l path-to-script
whereas
-w means "wait for CRLF before handing over control to the login program"
-i means "do not display issue file"
-n means "do not ask for a login name"
-I means "send init string to tty-device"
-l means "use special login program"
Only one more problem remains: The -w causes getty to wait for a CRLF before
handing over control to the "login" program.
So, the remote side has to send a \r\n, wait for the rx process coming up
and only then it may start the transmission.
In case I cannot make the remote side change its behaviour (and send the
CRLF before transmission), of course it would be great if there was an easy
way to get rid of this initial CRLF by putting my hands on the local
station.
Omission of the -w parameter does not work here as exprected, because
removing the -w makes getty invoke the login script way too early (when the
protocol handshaking process is not finished yet), the connection gets hung
up before it is fully established and rx remains waiting forever.
Inserting a "sleep 30" in the login bash script before rx doesn't help
either. Obviously, getty is just terminating ahead of time. (getty, rx are
from debian woody 3.0).
To be clear: The transmission itself works perfectly already.
You asked me for the handshaking - it's quite simple:
Local Station: Wait for incoming call, answer call, start xmodem data
reception immediately without login or a further initialization sequence.
Remote Station: Dial number, wait for protocol handshake being finished,
start xmodem-1k data transmission: send 76 KB of binary data at once. Hangup
line.
That's it.
Regards, Dominik
Alan Connor
>
>
> Hi Alan,
>
>> >>>>>>>>> I am looking for a simple solution for receiving files with the
>> >>>>>>>>> XMODEM protocol with a conventional V.32bis Modem on Linux.
>>
<snip>
[AC wrote:]
Wonderful! It's been fun working on this, Dominik. I've learned a lot.
Wouldn't have been possible without your thorough and well-organized
responses. This makes for a very useful record in the Archives too, for
others in the future: Usenet at its best. I know a lot of folks here
think that stuff like this archaic, but that's just because the net and
sys admins they rely on do all this sort of stuff for them behind the
scenes while they shoot space aliens and listen to music :-)
----------------------------------------------------------------------
Perhaps a sort of 'mini-daemon' would work here:
#!/bin/sh
while true ; do
if (activity on ttyS0); then echo -e "\r\n" > ttyS0 ; sleep N ; continue
else sleep 1 ; continue
done
Loop endlessly, one iteration per second until activity is detected on
tty0, then send a CRLF to ttyS0 and sleep for a while before resuming.
(activity on ttyS0) is a copout at this point. Not sure how to do that.
Perhaps a file recording all activity on ttyS0:
/dev/ttyS0 >> file
if [ -s file ] ; then ....
(if the file exists and isn't empty)
And empty the file after the "sleep N" with :> file.
AC
--
Pass-List -----> Block-List ----> Challenge-Response
The key to taking control of your mailbox. Design Parameters:
http://tinyurl.com/2t5kp || http://tinyurl.com/3c3ag
Challenge-Response links -- http://tinyurl.com/yrfjb
D. M. Herrmann
>>
>>>>>>>>>>>> I am looking for a simple solution for receiving files
>>>>>>>>>>>> with the XMODEM protocol with a conventional V.32bis Modem
>>>>>>>>>>>> on Linux.
>>>
>
> <snip>
>
>>>
>>>
>>> /sbin/getty -w -I 'ATE0Q1&D2&C1S0=1\015' 115200 ttyS? -l script
>>>
>>>
>>> #!/bin/sh
>>> # script
>>>
>>> /usr/bin/rx [-a dos2unix || -b binary(raw)] filename && exit 0
>>>
>>
>> Your proposal works great!
>>
>> I can establish a connection to the modem with the getty process
>> attached and can easily receive a file afterwards.
>>
>> The getty command line is:
>> getty -w -i -n -I 'ATE0Q1&D2&C1S0=1\015' 115200 /dev/ttyS0 -l
>> path-to-script
>>
>> whereas
>> -w means "wait for CRLF before handing over control to the login
>> program"
>> -i means "do not display issue file"
>> -n means "do not ask for a login name"
>> -I means "send init string to tty-device"
>> -l means "use special login program"
>>
>> Only one more problem remains: The -w causes getty to wait for a
>> CRLF before handing over control to the "login" program.
>>
>> So, the remote side has to send a \r\n, wait for the rx process
>> coming up and only then it may start the transmission.
>>
>>
>> To be clear: The transmission itself works perfectly already.
>>
>
> Wouldn't have been possible without your thorough and well-organized
> responses. This makes for a very useful record in the Archives too,
> for others in the future: Usenet at its best.
Well, it's fun reading your competent answers :) I hope many people will
find this discussion useful when they are looking for solutions to similar
problems. That's the way the usenet works, after all.
> Perhaps a sort of 'mini-daemon' would work here:
<snip>
> Loop endlessly, one iteration per second until activity is detected on
> tty0, then send a CRLF to ttyS0 and sleep for a while before resuming.
Nice idea, but this won't work here. I have already tried sending the \r\n
from another tty on the local machine to ttyS0 at the very special point
when getty is waiting for it.
The Bad News: It does not work: The \r\n is sent to the *remote* station (I
can see the cursor jump to the next line when I run a terminal program
there). But getty doesn't care at all. It keeps sitting there waiting for
the magic \r\n pair which obviousloy really has to come from the remote
side.
Well, this whole problem might prove difficult to work out without changing
the remote station.
So I have commenced trying out an approach Floyd sent me by mail some hours
ago:
A bash script consisting of
#!/bin/bash
stty 9600 --file=/dev/ttyS0
(echo -ne "ATE1Q0S0=1\r\n";exec rx -b filename) </dev/ttyS0 >/dev/ttyS0
#end script
(the modem init string may be different, depending on modem type)
I keep experiencing some funny behaviour, though:
If I execute the just mentioned getty on ttyS0 before I try receiving data
with Floyd's script, it will work beautifully: rx receives the file and
hangs up. Perfect.
But if I reboot the local linux box and try to use Floyd's script
immediately after reboot, it won't work:
The local modem answers the line (as expected). But then the protocol
handshaking apparently fails after about 2/3 of time. The local modem will
hang up without reason in the midst of the handshaking (while the remote
modem still tries to establish the connection for some time and gives up
finally because the line's dead).
Now, if I run your getty command line with the same initialization string as
in Floyd's script *one time*, kill the getty process immediately afterwards
(even without answering a call or receiving anything) and then run Floyd's
script from above again, ... it will work as expected. The handshaking takes
place and rx comes up waiting for the transmission to begin.
So, something is different after getty gets its hands on the ttyS0 device.
But I cannot figure it out at the moment.
I had a look at the terminal settings with stty --file=/dev/ttyS0 before
getty is run and afterwards. getty changes lots of parameters:
Afterwards:
speed 115200; line = 0
min = 1; time = 0
ignbrk -brkint -icrnl -imaxbel
-opost -onlcr
-isig -icanon -iexten -echo -echoe -echok -echoctl -echoke
It doesn't help if I set those parameters manually with stty ahead of
Floyd's one-liner. getty seems to do some additional initializing on the
ttyS0 device. Perhaps I am missing something obvious?
Any ideas?
Thanks for your time and help!
Regards, Dominik
Alan Connor
Yes, I am very good at fumbling in the dark :-)
> I hope many people will
> find this discussion useful when they are looking for solutions to similar
> problems. That's the way the usenet works, after all.
>
>> Perhaps a sort of 'mini-daemon' would work here:
>
><snip>
>
>> Loop endlessly, one iteration per second until activity is detected on
>> tty0, then send a CRLF to ttyS0 and sleep for a while before resuming.
>
> Nice idea, but this won't work here. I have already tried sending the \r\n
> from another tty on the local machine to ttyS0 at the very special point
> when getty is waiting for it.
> The Bad News: It does not work: The \r\n is sent to the *remote* station
I was afraid of that. Perhaps it could be sent directly to getty ??
>(I
> can see the cursor jump to the next line when I run a terminal program
> there). But getty doesn't care at all. It keeps sitting there waiting for
> the magic \r\n pair which obviousloy really has to come from the remote
> side.
>
> Well, this whole problem might prove difficult to work out without changing
> the remote station.
>
>
> So I have commenced trying out an approach Floyd sent me by mail some hours
> ago:
Floyd really knows his stuff. Wish he'd post to Usenet instead of mailing
people...
Only writing a custom version of rx in bash, with its builtin
/dev/tcp|udp/host/port or with bash and netcat.
What port is being used for the transaction?
Is this TCP or UDP?
What are the EXACT contents of the handshake? (as in tcpdump)
In-other-words, if you had a script that started by waiting for
input from that port:
read A
# What would $A be, so that you could test and make sure it wasn't
# just a wrong number?
if [ "$A" = "..." ] ; then echo -e ...
...
I'm not saying I could write the script, having no way to test the
various parts, but I could HELP you write it.
It's a simple protocol, and I've written similar scripts for POP
and SMTP.
AC
Floyd L. Davidson
>ago:
>A bash script consisting of
>#!/bin/bash
>stty 9600 --file=/dev/ttyS0
>(echo -ne "ATE1Q0S0=1\r\n";exec rx -b filename) </dev/ttyS0 >/dev/ttyS0
>#end script
>
>(the modem init string may be different, depending on modem type)
All of that was meant to be just an example of the technique,
and you have to fill in the appropriate arguments to commands
to fit your situation.
The "init string" is just the only three AT commands I could
remember off the top of my head. There are *several* others
that you will need in a real init string.
The same is true of stty arguments, but lets talk about what is
happening and I'll show you a nice little trick that'll have you
rolling on the floor laughing at how easy it is (and should
cause all of use to bow towards Bell Labs, where unix came from,
in appreciation of this sort of mind set for system software).
>I keep experiencing some funny behaviour, though:
>If I execute the just mentioned getty on ttyS0 before I try receiving data
>with Floyd's script, it will work beautifully: rx receives the file and
>hangs up. Perfect.
>
>But if I reboot the local linux box and try to use Floyd's script
>immediately after reboot, it won't work:
You *must* totally initialize the serial port. Obviously any
form of a getty program must do that too, as must any modem
program. Hence running any of them will necessarily configure
the serial port for you. (They can also initialize the modem
too, but that would be a /really/ /bad/ idea to make use of.)
However, if you get the serial port configured just one time in
a way that works, and then do this:
stty -g --file=/dev/ttyS0 > stty.out
It will generate a string that stty can use to reconfigure a port
to be *exactly* what it was when you ran the command. All you
have to do is put that string into your script as an argument to
the stty command (rather than the "9600" that I showed in the
example). The string will look something like this:
1:0:800008bd:0:3:1c:7f:15:4:5:1:0:11:13:1a:0:12:f:17:16:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0
Once again... The script has three, possibly four, commands in it.
1. Something to initialize the serial port.
2. Something to initialize the modem.
3. Something to
A. recognize that a data transfer is being initiated, and
B. save the data to a file.
4. Something to massage the data file.
Number 1 is done with stty. The second is done with echo. It
appears that rx will do everything needed for both 3 and 4.
The exact arguments for each of those commands will depend on
information that I do not have, and *you* will need to determine
based on what you have.
>I had a look at the terminal settings with stty --file=/dev/ttyS0 before
Use the -a option to stty and you'll see just how many
parameters there are in total (or count the ':' field separators
in the -g output for just a raw number).
>getty is run and afterwards. getty changes lots of parameters:
>Afterwards:
>speed 115200; line = 0
Virtually every serial port program changes the bit rate.
Needing to configure that is fairly obvious. The "line"
discipline value (which is not a POSIX defined part of termios)
is also changed by such things as pppd, and is the most commonly
missed configuration parameter that causes erratic results
(people notice that their program always works fine, unless it
is used right after ppp crashes or is killed with SIGKILL, and
figure that it is a bug in pppd that destroys the serial port;
but the bug is in their program).
>min = 1; time = 0
>ignbrk -brkint -icrnl -imaxbel
>-opost -onlcr
>-isig -icanon -iexten -echo -echoe -echok -echoctl -echoke
Several of those are "do or die" parameters that *must* be
correct. -canon is raw input mode, vs canonical input mode.
-opost causes processing of output. All of the echo parameters
relate to how much noise the serial port and the modem make (and
they can confuse each other totally).
>It doesn't help if I set those parameters manually with stty ahead of
>Floyd's one-liner. getty seems to do some additional initializing on the
>ttyS0 device. Perhaps I am missing something obvious?
Yep, you're missing just about the entire part of the learning
curve devoted to serial ports! That's the part which was going
to keep you up all night every night between now and the deadline
in July. :-(
Do "stty -g" and you can skip learning any of that. :-)
Alan Connor
>
>
Snipped article:
http://groups.google.com/groups?selm=87pt7ki...@barrow.com
>
> --
> FloydL. Davidson <http://web.newsguy.com/floyd_davidson>
> Ukpeagvik (Barrow, Alaska) fl...@barrow.com
Wonderful! So many questions answered.
AC
Frank da Cruz
: ...
: I hope I got your point right, please correct me otherwise.
:
: The problem with your idea is that I am sitting on the "local" station
: (which receives the call) - dialling out doesn't happen here. The local
: modem is just sitting around and waiting for a call from a remote station.
: So, I guess, auto dialing won't help me much on the local station.
:
: The "remote" station (which sends the data over the line to me) is not under
: my control. It already works pretty fine and I am able to receive data with
: a terminal program interactively/manually.
:
: But I haven't checked the dialing directory yet, perhaps your idea will make
: sense if I have had a look on it.
:
: There remains the looping problem: After a successful file reception the
: modem has to prepare for the next one.
: So minicom would have to behave as follows:
: Initialize the modem - Wait for call - receive call - handshaking - receive
: some binary data - save received binary data to file - start from the
: beginning (initialize the modem/wait for call).
: A second process would wait simultaneously for the binary data to become
: evident on disk and fetch it from there, erasing the file afterwards so it
: can be replaced when the next call comes in.
:
: I hope the scenario becomes clear gradually.
Let's suppose you were using C-Kermit for this:
http://www.columbia.edu/kermit/ckermit.html
and also that you had a program called rx installed that would receive files
using Xmodem-1K. Here is your script:
set modem type usrobotics ; or whatever else it is
set port /dev/ttyS0 ; or whatever other port you need to use
set speed 57600 ; or other desired speed
set flow rts/cts ; Xmodem and Xon/Xoff don't mix!
set protocol xmodem ; Tells Kermit to use rx for file transfer
cd xxxx ; CD to desired directory
while true { ; Loop for many calls
answer 0 ; Wait forever for a call to come in
if fail continue ; If some error just try again
receive yyyy ; Receive the file (*)
hangup ; Hang up the phone
}
That's it. Kermit takes care of all the messy modem details for you.
This way, you can change modems without having to rewrite your script,
except to change the name of the modem.
(*) When using any Xmodem-family protocol, the file sender does NOT send
the name of the file, so you have to tell Kermit the name to store it under.
Replace "yyyy" with the desired name. Now clearly if you are running this
in a loop, you won't want to use the same name each time. Therefore use
one or more of Kermit's many built-in variables and functions to construct
a unique date, e.g.
receive \v(ndate)_\flpad(\v(ntime),5,0).bin
This gives a filename composed of the numeric date and time, listed in
order of reception:
\v(ndate) is the numeric date, e.g. 20040627
\v(ntime) is the seconds since midnight
\flpad(x,5,0) left-pads its argument with 0's to make it 5 bytes long
If you want to keep a log, it's easy. If you want to rename, move, or
otherwise process the file after reception, that's easy too. Again, I refer
you to the Kermit script library for examples:
http://www.columbia.edu/kermit/ckscripts.html
For further information about any of the commands shown above, just type
"help xxx" at the C-Kermit> prompt, where "xxx" is the name of the command,
e.g. "help answer", "help set protocol".
- Frank
Alan Connor
>
>:
> Let's suppose you were using C-Kermit for this:
>
> http://www.columbia.edu/kermit/ckermit.html
>
Useful information, Frank. What do you think of gkermit, which
comes with many linux distros, and IS 'free'?
I've been using minicom for a long time, and it's performed very
well. Why do you think kermit is better?
AC
Frank da Cruz
: On 28 Jun 2004 16:30:49 GMT, Frank da Cruz <f...@columbia.edu> wrote:
:> ...
:> Let's suppose you were using C-Kermit for this:
:>
:> http://www.columbia.edu/kermit/ckermit.html
:
: Useful information, Frank. What do you think of gkermit, which
:
G-Kermit is not a communications program, only a stdio file-transfer protocol,
just like rzsz, except it uses Kermit protocol. C-Kermit, on the other hand:
. Makes connections (serial port, modem, Internet, some others)
. Includes a built-in FTP client.
. Supports several Internet security methods (Kerberos, SSL/TLS, ...)
for making secure Telnet, FTP, and other kinds of connections.
. Supports Unicode and many other character sets.
. Supports external transport protocols such as SSH and tn3270.
. Supports external file-transfer protocols such as XYZMODEM.
. Is platform independent (i.e. not tied to Unix).
. Includes its own built-in platform- and transport-independent
script programming language.
> ...and IS 'free'?
>
Freedom is in the eye of the beholder. Somebody (me, in fact) did the
work to design and implement it so there could be *some* Kermit software
presence in "free" or "open source" distributions that were reluctant
to carry C-Kermit despite the fact that its license allows them to do so:
ftp://kermit.columbia.edu/kermit/f/COPYING.TXT
In fact, C-Kermit is now included in most Linux distributions, although
you might have to do a "custom install" to get it. Of course you can also
just download it from here (Columbia U).
: I've been using minicom for a long time, and it's performed very
: well. Why do you think kermit is better?
:
Depends what you mean by "better" -- read this:
http://www.columbia.edu/kermit/kermit.html#minicom
- Frank
Alan Connor
Okay. I read it. Pretty impressive. But as I read I kept thinking: Emacs,
and sure enough, you confessed to this resemblence in short order.
I'm just one of those people that likes simple and discrete tools that
are knitted together loosely by shell scripts, if anything. That call
other utilities as needed.
Monolithic juggernauts like Emacs just turn me off completely.
I like to be able to troubleshoot problems in a hurry, by isolating
the various participants.
And change the various parts of something without waiting for the
developers to get around to it.
Sure, we need more standardization, but inbred and massive monolithic
multi-application monsters don't accomplish this.
I have a *great* ftp program. I do not need my communications application
to do anything but call it and get out of the way.
You *might* mention, in your discussion of minicom, that it too has
a scripting language.
Thanks for your time.
AC
Frank da Cruz
:
:
:
: Okay. I read it. Pretty impressive. But as I read I kept thinking: Emacs,
: and sure enough, you confessed to this resemblence in short order.
:
: I'm just one of those people that likes simple and discrete tools that
: are knitted together loosely by shell scripts, if anything. That call
: other utilities as needed.
:
: Monolithic juggernauts like Emacs just turn me off completely.
It's a tradeoff, and not just a matter of taste or aesthetics. In
communications applications, it often does not make sense to use an external
scripting tool. You really do want the same process that is in charge of
the communication medium to be the one that you control with your script, so
it can reliably sense and/or set error conditions, modem signals, protocol
policies, and whatnot. This is explained to some extent here:
http://www.columbia.edu/kermit/ftpscripts.html
: I like to be able to troubleshoot problems in a hurry, by isolating
: the various participants.
:
Kermit is nothing if not a good debugging tool.
: And change the various parts of something without waiting for the
: developers to get around to it.
:
Nothing is stopping you doing that with Kermit, the source code is in the
open. Anyway you should be thinking of Kermit as a language rather than
a communications program. Like EMACS it is totally extensible. You can
can usually "change" it to do what you want by writing writing programs
*for* it, rather than by hacking the source code. Take another at the
script library to see what I mean:
http://www.columbia.edu/kermit/ckscripts.html#init
: Sure, we need more standardization, but inbred and massive monolithic
: multi-application monsters don't accomplish this.
:
That's a value judgement that not everybody would make. If I can write a
script to perform some complicated communications task that can be freely
moved among different platforms (Linux, Mac OS X, all other Unixes, Windows,
OS/2, VMS, ...) as well as among different communication methods (serial,
modem, telnet, ssh, X.25, ...) I think I've indeed accomplished some kind
of standardization and, more to the point, saved myself and possibly others
some work.
: I have a *great* ftp program. I do not need my communications application
: to do anything but call it and get out of the way.
:
: You *might* mention, in your discussion of minicom, that it too has
: a scripting language.
:
It didn't at the time that exchange took place.
Btw, the script I posted earlier is just about a simple as you can get. I'd
be surprised to see a solution that is simpler or more elegant, and if it
were portable to many platforms and/or communication methods, that would
just be a bonus. I posted the script because the other proposed solutions
were becoming more and more convoluted. Pipelining a series of simple Unix
applications together is not always the best approach.
- Frank
D. M. Herrmann
sorry for the delay with this reply. Spent several hours in front of the
terminals playing with all the tools in hand now.
The good news: It's working now and it rocks! Read on for the details...
Floyd L. Davidson wrote:
> "D. M. Herrmann" <do...@exonet.de> wrote:
>> So I have commenced trying out an approach Floyd sent me by mail
>> some hours
>> ago:
>> A bash script consisting of
>> #!/bin/bash
>> stty 9600 --file=/dev/ttyS0
>> (echo -ne "ATE1Q0S0=1\r\n";exec rx -b filename) </dev/ttyS0
>> >/dev/ttyS0 #end script
>>
>> (the modem init string may be different, depending on modem type)
>
> All of that was meant to be just an example of the technique,
> and you have to fill in the appropriate arguments to commands
> to fit your situation.
>
> The "init string" is just the only three AT commands I could
> remember off the top of my head. There are *several* others
> that you will need in a real init string.
Yes, of course. I add &D&C1 to the above for my US Robotics model.
<snip>
>> I keep experiencing some funny behaviour, though:
>> If I execute the just mentioned getty on ttyS0 before I try
>> receiving data
>> with Floyd's script, it will work beautifully: rx receives the file
>> and
>> hangs up. Perfect.
>>
>> But if I reboot the local linux box and try to use Floyd's script
>> immediately after reboot, it won't work:
>
> You *must* totally initialize the serial port. Obviously any
> form of a getty program must do that too, as must any modem
> program. Hence running any of them will necessarily configure
> the serial port for you. (They can also initialize the modem
> too, but that would be a /really/ /bad/ idea to make use of.)
Yes, I considered running getty after reboot once and killing it
automatically shortly afterwards just before starting the bash script with
the rx-loop. I thought that to be a really quick and dirty approach and
didn't ponder on it any longer.
> However, if you get the serial port configured just one time in
> a way that works, and then do this:
>
> stty -g --file=/dev/ttyS0 > stty.out
>
> It will generate a string that stty can use to reconfigure a port
> to be *exactly* what it was when you ran the command. All you
> have to do is put that string into your script as an argument to
> the stty command (rather than the "9600" that I showed in the
> example). The string will look something like this:
>
>
1:0:800008bd:0:3:1c:7f:15:4:5:1:0:11:13:1a:0:12:f:17:16:0:0:0:0:0:0:0:0:0:0:
0:0:0:0:0:0
>
> Once again... The script has three, possibly four, commands in it.
>
> 1. Something to initialize the serial port.
> 2. Something to initialize the modem.
> 3. Something to
> A. recognize that a data transfer is being initiated, and
> B. save the data to a file.
> 4. Something to massage the data file.
>
> Number 1 is done with stty. The second is done with echo. It
> appears that rx will do everything needed for both 3 and 4.
>
> The exact arguments for each of those commands will depend on
> information that I do not have, and *you* will need to determine
> based on what you have.
<snip>
> Perhaps I am missing something obvious?
>
> Yep, you're missing just about the entire part of the learning
> curve devoted to serial ports! That's the part which was going
> to keep you up all night every night between now and the deadline
> in July. :-(
>
> Do "stty -g" and you can skip learning any of that. :-)
It is really amazing. This neat little -g argument to stty saved me hours of
serial ports programming research. Thank you very much. The man page hops
over this important -g parameter quite fast. The importance of it is not
mentioned there at all...
As stated above I have a running setup now consisting of the following:
A bash script called floydtest.sh (thinking about keeping this name as kind
of an homage to its main contributor *g*):
--- /root/floydtest.sh ---
#!/bin/bash
# first of all some paths and parameters
ECHO="/bin/echo" # echo tool
STTY="/bin/stty" # stty tool
RX="/usr/bin/rx" # xmodem receive tool
RXARGS="-c --overwrite -b" # CRC checking, overwrite existing, binary
MIMENCODE="/usr/bin/mimencode" # for base64-encoding of data
MAILX="/usr/bin/mailx" # send mail tool
DIP="/usr/bin/dip" # dial-in manager tool (for modem init)
DIPSCRIPT="/root/reset.dip" # contains modem init commands
DATFILE="/root/floydtest.dat" # where to store the received data
DEVICE="/dev/ttyS0" # device the modem is connected to
# file containing serial port settings from call to stty -g -F <dev>
# you have to save the output of the above call to this file
# at the time the serial port is in a working state (kind of a
# snapshot)
DEVINIT="/root/stty.out"
# send the data to this email address
MAILRECPT="mai...@theeye.ath.cx"
# Wait for data and receive...
sleep 2
# initialize modem (see below); ATS0=1 (Auto Answer Mode)
$DIP $DIPSCRIPT
# initialize serial port (must come after the call to dip!)
$STTY --file=$DEVICE `cat $DEVINIT`
# wait for incoming connection being established
( sleep 1;
$RX $RXARGS $DATFILE
) <$DEVICE >$DEVICE
# again, wait a second
sleep 1
# encode data and hand on by mail
$MIMENCODE $DATFILE | $MAILX -s "subject" $MAILRECPT
sleep 5
--- end floydtest.sh ---
This script is included in /etc/inittab as follows:
--- /etc/inittab ---
T0:23:respawn:/root/floydtest.sh
--- end /etc/inittab ---
A key question is the modem initalization after system reboot. I could not
get stable results with the earlier proposed
echo -ne "initstring\015" > /dev/ttyS0
This command would block and the modem would not auto answer because
apparently it had not received the init string at that moment yet. After
several tries the modem would have received the string and would answer the
line - but something was obviously wrong because the remote station always
received the initstring after the connection was established. A clear sign
that something was not okay.
The statement
echo -ne "initstring" >> /dev/ttyS0
didn't work either (proposed in a post some months ago found with
groups.google.com).
So I had to look for a new method to initialize the modem.
I didn't want to invoke minicom with runscript for initializing the modem
because at times it messed up the serial port setup (non deterministic). So
I went looking for a more simple solution.
The dip tool, which was used in early days of the internet as a dial in tool
was worth a try. It is fully scriptable. You can get it as DEB-Package (look
for dip_3.3.7p-4_i386.deb on your favourite debian mirror).
Important is the order of the stty and the dip call. It didn't work the
other way round. See below.
Here is my dip script which initializes the modem and enables auto answer
(found with groups.google.com, slightly modified):
--- /root/reset.dip ---
port ttyS0
speed 115200
reset
send ATZ\r
wait OK 2
if $errlvl != 0 goto modem_trouble
print Modem reset .... OK
send ATE0&D2&C1S0=1\r
wait OK 2
if $errlvl != 0 goto modem_trouble
print Modem init .... OK
goto exit
modem_trouble:
print Trouble resetting the modem...
print Modem is horribly snarled up!
exit:
print Modem reset done.
exit
--- end /root/reset.dip ---
That's all. The process respawns after completion which guarantees a stable
loop ready to receive connections.
Took me about 30-40 hours figuring out and your fancy guys' writing on my
screen!
The rest is easy: The recieved data package is base64-encoded and sent away
by mail to a remote server which pipes it into a php script. This script
dissembles the data package and writes it to a mysql database.
The deadline seems quite safe on the horizon now :-)
Thanks again for your insightful replies, you definitely saved my butt this
time, perhaps one day there's time to meet for a drink around the corner! I
hope the above code will help other people with similar problems.
If anyone is even more interested in the dirty details here on my setup,
feel free to contact me on the net or at domi ( at ) exonet.de.
Regards, Dominik
Alan Connor
>
>
> Hi guys,
>
> sorry for the delay with this reply. Spent several hours in front of the
> terminals playing with all the tools in hand now.
> The good news: It's working now and it rocks! Read on for the details...
<snip>
>
> That's all. The process respawns after completion which guarantees a stable
> loop ready to receive connections.
> Took me about 30-40 hours figuring out and your fancy guys' writing on my
> screen!
>
> The rest is easy: The recieved data package is base64-encoded and sent away
> by mail to a remote server which pipes it into a php script. This script
> dissembles the data package and writes it to a mysql database.
>
> The deadline seems quite safe on the horizon now :-)
>
> Thanks again for your insightful replies, you definitely saved my butt this
> time, perhaps one day there's time to meet for a drink around the corner! I
> hope the above code will help other people with similar problems.
>
> If anyone is even more interested in the dirty details here on my setup,
> feel free to contact me on the net or at domi ( at ) exonet.de.
>
> Regards, Dominik
>
>
Most excellent, Dominik. I'm saving your fine script to my serial docs.
Thanks for following through here.
We'll get together with Floyd in the Land of the Midnight Sun one of
these days, and toss back a few brews.
AC
Alan Connor
My version of minicom is at least 3 years old: Time to update your
webpage, I think...
> Btw, the script I posted earlier is just about a simple as you can get. I'd
> be surprised to see a solution that is simpler or more elegant, and if it
> were portable to many platforms and/or communication methods, that would
> just be a bonus. I posted the script because the other proposed solutions
> were becoming more and more convoluted. Pipelining a series of simple Unix
> applications together is not always the best approach.
>
> - Frank
No doubt you have written a fine program. It's just not the sort of
approach I like. REALLY don't want to learn another programming language
that has no use beyond a single program.
Let us know when ckermit is fully modularized and uses sh instead.
Have a good one.
AC
D. M. Herrmann
I would prefer the Bavarian Oktoberfest in Munich in the last few days of
September for that matter. If anyone of you is interested in meeting in Real
Life, let me know, I'll arrange a trip to Munich!
Have to talk to my client about honoring the help of my usenet fellows,
perhaps I can provide you with airplane tickets (as Floyd suggested in PM
earlier...).
So long,
Dominik
Alan Connor
It's a very good thought, Dominik, but for reasons too complex to
explain here, just not possible for me.
Hope to see you on the Usenet.
AC