DDP Not Retrieving Packets from fldigi?
Brendan O'Connor
initialized. I can send packets via DDP (say, the IM example), but DDP
*never* sees any incoming packets-- malformed or not-- from fldigi,
even though I can see that fldigi clearly gets them.
I've tried this OSX->Linux, Linux->OSX, and Linux->Linux, all with no
luck. I had this working a few days ago, but now I've clearly lost my
mind or something-- nothing works now. (Including fresh installs of
both fldigi and DDP.) Even in debug mode, DDP says nothing happens:
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Warning - Timeout waiting for packet.
DDP/receiveDataFromAny()/Warning - No valid packet was received from
receivePacket().
DDP/receiveDataFromAny()/Warning - No valid packet was received.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Warning - Timeout waiting for packet.
DDP/receiveDataFromAny()/Warning - No valid packet was received from
receivePacket().
DDP/receiveDataFromAny()/Warning - No valid packet was received.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Warning - Timeout waiting for packet.
DDP/receiveDataFromAny()/Warning - No valid packet was received from
receivePacket().
DDP/receiveDataFromAny()/Warning - No valid packet was received.
DDP/receiveDataFromAny()/Warning - Maximum retries (3) has been
exceeded.
By the way, do you have a VCS somewhere that you use? Assuming I can
get my universe back into gear, I'd love to help work on DDP.
Thanks,
---Brendan O'Connor, K3QB
Danny Knaggs
Ok, ensure your setup is working normally - ensure no wires have fell out the rig, audio levels dropped, try to have a qso using just plain fldigi. Sounds like something really basic has changed.
Ensure both ends are using the same version of ddp, frequency, mode, digimode, specification, compression, etc. Delete the settings XML and go with defaults, defaults work best while you get settled.
The os is runs on shouldn't matter, as long as all the packages are present. After ddp has displayed its license it will show the setup and any potential problems. Ensure debug mode is enabled to show extra details.
You say its working fine on OSX? If so, excellent I'll update the info.
No source revision as of yet. What do you have in mind?
73,
Dan - 2E0DPK.
Brendan O'Connor
over the ddp code itself. Essentially, what's happening is that DDP is
telling fldigi to transmit the data, but it's cutting off the last
character (instead of *DDP**, it sends *DDP* at the end), so DDP never
sees a full packet at the other end.
A workaround I found, in ddp.py about line 455:
if self.fldigi is not None:
if self.SPECIFICATION == 0:
tx2.extend(d)
elif self.SPECIFICATION == 1:
# Can't send 8-bit character set to fldigi, so we'll have to
hex-encode it instead
tx2.extend(self.PROTOCOL_HEADER)
tx2.extend(self.encodeStreamToBase128(self.scramble(d), 0, True))
tx2.extend(self.PROTOCOL_FOOTER)
tx2.extend(" ")#BFO
tx2.extend("^r")
(Only changed the #BFO line, but I included the rest for context)
Now, though, I'm having a more serious problem. I'll include logs from
the debug mode on the receiving end:
========
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Exception - <Fault -501: 'Integer parameter too high'>
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/BFO()/REC - ['**DDP*', '0420', 'OSX', '', 'QSO', '00000000',
'0001000000000010', '2f5f8e24f884f59100f5132d179e74ff704c54b61d',
'937f550e8ed8e274d0c7a1d7039230a35000acb0', '*DDP**']
DDP/BFO()/Received - Got something
DDP/BFO()/Received - packet is not none
DDP/BFO()/UDP - Got here
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/BFO()/REC - ['**DDP*', '0420', 'OSX', '', 'QSO', '00000001',
'0100000000000000', '456c',
'125f6afcf13241757a33d89c471a880da5abf678', '*DDP**']
DDP/BFO()/Received - Got something
DDP/BFO()/Received - packet is not none
DDP/BFO()/UDP - Got here
DDP/BFO()/AboutToReturn - [['**DDP*', '0420', 'OSX', '', 'QSO',
'00000000', '0001000000000010',
'2f5f8e24f884f59100f5132d179e74ff704c54b61d',
'937f550e8ed8e274d0c7a1d7039230a35000acb0', '*DDP**']]
DDP/decodeData()/Information - Running...
DDP/decodeBase128ToStream()/Information - Running...
DDP/offsetChar()/Information - Running...
DDP/descramble()/Information - Running...
DDP/decompressStream()/Information - Running...
DDP/BFO()/DEC - ??R?x-K???E6? MKX?
Exception: Not a gzipped file
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/BFO()/REC - ['**DDP*', '0420', 'OSX', '', 'QSO', '00000000',
'0001000000000010', '2f5f8e24f884e89b427dc75d184ba85b99b1f5',
'6205d447d5512c41b4775951637d43d7724a63e8', '*DDP**']
DDP/BFO()/Received - Got something
DDP/BFO()/Received - packet is not none
DDP/BFO()/UDP - Got here
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/BFO()/REC - ['**DDP*', '0420', 'OSX', '', 'QSO', '00000001',
'0100000000000000', '456c',
'125f6afcf13241757a33d89c471a880da5abf678', '*DDP**']
DDP/BFO()/Received - Got something
DDP/BFO()/Received - packet is not none
DDP/BFO()/UDP - Got here
DDP/BFO()/AboutToReturn - [['**DDP*', '0420', 'OSX', '', 'QSO',
'00000000', '0001000000000010',
'2f5f8e24f884e89b427dc75d184ba85b99b1f5',
'6205d447d5512c41b4775951637d43d7724a63e8', '*DDP**']]
DDP/decodeData()/Information - Running...
DDP/decodeBase128ToStream()/Information - Running...
DDP/offsetChar()/Information - Running...
DDP/descramble()/Information - Running...
DDP/decompressStream()/Information - Running...
DDP/BFO()/DEC - ??R?ro?x????yɰ?
Exception: Not a gzipped file
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
========
There's some extra log lines I've included in there (all marked with
the BFO sigil, so I can delete them later :) ), so you can see what
it's seeing. I'm not yet a protocol expert-- thoughts on what's
getting dropped?
On Sun, May 22, 2011 at 6:57 PM, Danny Knaggs <knagg...@gmail.com> wrote:
> Ok, ensure your setup is working normally - ensure no wires have fell out
> the rig, audio levels dropped, try to have a qso using just plain fldigi.
> Sounds like something really basic has changed.
>
> Ensure both ends are using the same version of ddp, frequency, mode,
> digimode, specification, compression, etc. Delete the settings XML and go
> with defaults, defaults work best while you get settled.
Right, doing all that. Not actually using a rig *at all*-- doing this
with soundcard-to-soundcard stuff, so I can ensure there's no QRM
involved.
>
> The os is runs on shouldn't matter, as long as all the packages are
> present. After ddp has displayed its license it will show the setup and any
> potential problems. Ensure debug mode is enabled to show extra details.
Yep, I've been doing that.
>
> You say its working fine on OSX? If so, excellent I'll update the info.
Let's wait until I've actually got it working again. :-)
>
> No source revision as of yet. What do you have in mind?
I love GitHub, not least as that's where the communities tend to hang
out and do a *lot* of cross-pollination, and it has lots of tools to
aid in taking help from the community (nice pull requests, patching
support, issue trackers, etc.). Much more up-to-date than Google Code,
and it lets you keep the *code* and the *community* together in one
place. Lots of communities have, therefore, redirected people from
their Google Code page over to GitHub. (And you can, of course, keep
posting releases over at gCode-- no need to make people change who
don't care about the code itself, at least at first, though GitHub
also has a great binary/package posting system). I'd be happy to help
you get set up over there, if you need a hand.
---BFO
Danny Knaggs
I'm seeing warnings about gzip, this means that both sides are not using the same compression. Put on pylzma on both sides and try again.
Gzip is used as a fallback, I put it in for testing.
Not seeing the problem with it clipping a character off the end, rather than a space use a \n instead.
Dan.
Brendan O'Connor
from one to the other, however.
Just to make sure I hadn't borked anything, I reverted back to your
distributed version of ddp on both sides. I changed just line 464 on
the sending computer to read:
tx2.extend("\n^r")#BFO
Rather than
tx2.extend("^r")
This was because after the revert, before that change, the receiving
side still only ever saw *DDP* at the end, rather than *DDP**, and
thus never saw a packet.
Here's the logs on the receiving end (Linux):
======
DDP/__init__()/Information - Initialising DDP v0.4.2...
DDP/__init__()/Information - Checking Python version...
DDP/__init__()/Information - Running under Python v2.6.6.
DDP/__init__()/Information - Checking platform...
DDP/__init__()/Information - You appear to be running on Linux, this
platform is fully supported.
DDP/__init__()/Information - Checking compression modules...
DDP/__init__()/Warning - PyLZMA is not available, falling back to GZIP
module instead for the default compressor.
DDP/__init__()/Information - Setting up for Fldigi mode using PSK500R...
DDP/setupFldigi()/Information - Running...
DDP/__init__()/Information - Running in 7-bit text mode.
DDP/__init__()/Information - Initialisation complete.
Please enter your callsign: LAP
Information: Starting RX thread...
Information: Ready to start chatting.
LAP: DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/decodeData()/Information - Running...
DDP/decodeBase128ToStream()/Information - Running...
DDP/offsetChar()/Information - Running...
DDP/descramble()/Information - Running...
DDP/decompressStream()/Information - Running...
Exception: Not a gzipped file
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/decodeData()/Information - Running...
DDP/decodeBase128ToStream()/Information - Running...
DDP/offsetChar()/Information - Running...
DDP/descramble()/Information - Running...
DDP/decompressStream()/Information - Running...
Exception: Not a gzipped file
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
======
That's over the four-message send triggered by a one-word message on
the sending computer. It keeps saying it's a valid packet-- but
nothing ever gets passed to the program, and that weird "not a gzipped
file" issue pops up still.
---BFO
Danny Knaggs
The log is still showing pylzma is not available, look at init part if the log. Which distro of linux are you using?
You could try not installing pylzma on the other side or making it use gzip all the time by tweaking the code.
Looks like it is processing packets fine though.
Dan.
Brendan O'Connor
and I installed python-lzma, which is *not* the same thing (as it
happens) as PyLZMA. (Python-lzma is actually PyLibLZMA.) The latter
isn't in the apt repositories, so you have to apt-get install
python-dev python-setuptools first, then easy_install pylzma. Now I
get "PyLZMA is available" on both ends.
So now, I'm back to the earlier problems: the cutting-off-the-last-*
problem, and the fact that packets don't decode even when they're not
cut off. The updated log is below. What system are you using to test
on your side? I feel like I'm missing something basic, but I can't
find what.
Thanks,
---BFO
======
DDP/__init__()/Information - Initialising DDP v0.4.2...
DDP/__init__()/Information - Checking Python version...
DDP/__init__()/Information - Running under Python v2.7.1.
DDP/__init__()/Information - Checking platform...
DDP/__init__()/Information - You appear to be running on Linux, this
platform is fully supported.
DDP/__init__()/Information - Checking compression modules...
DDP/__init__()/Information - PyLZMA is available, will be used as the
default compressor.
DDP/__init__()/Information - Setting up for Fldigi mode using PSK125R...
DDP/setupFldigi()/Information - Running...
DDP/__init__()/Information - Running in 7-bit text mode.
DDP/__init__()/Information - Initialisation complete.
Please enter your callsign: LAP
Information: Starting RX thread...
Information: Ready to start chatting.
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
LAP: DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/verifyPacket()/Warning - Wrong number of sections (71 != 10).
DDP/receivePacket()/Warning - Packet received is invalid.
DDP/receiveDataFromAny()/Warning - No valid packet was received from
receivePacket().
DDP/receiveDataFromAny()/Warning - No valid packet was received.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Exception - <Fault -501: 'Integer parameter too high'>
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
Warning: No packets have been received.
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
DDP/receivePacket()/Exception - <Fault -501: 'Integer parameter too high'>
DDP/receivePacket()/Information - A packet has been found in the text.
DDP/splitPacket()/Information - Running...
DDP/verifyPacket()/Information - Running...
DDP/sha1()/Information - Running...
DDP/receivePacket()/Information - Packet received is valid.
DDP/decodeData()/Information - Running...
DDP/decodeBase128ToStream()/Information - Running...
DDP/offsetChar()/Information - Running...
DDP/descramble()/Information - Running...
DDP/decompressStream()/Information - Running...
Exception: Error while decompressing: 1
DDP/receiveDataFromAny()/Information - Running...
DDP/receiveDataFromAny()/Information - Receiving packet...
DDP/receivePacket()/Information - Running...
DDP/ptt()/Information - Running...
======
Danny Knaggs
What version of fldigi is installed on the machine? Could be a bug,
however - add the extra \n as per previous message and see if that
solves it. If it does, I'll put it in the next version.
As for packets not decoding when complete, this is as per design.
Dan.
Brendan O'Connor
> OK, I'll update the README for this on Ubuntu.
>
> What version of fldigi is installed on the machine?
3.21.8 on Linux (current repository version), 3.21.9 on OSX.
> Could be a bug,
> however - add the extra \n as per previous message and see if that
> solves it. If it does, I'll put it in the next version.
It didn't-- the log I sent last time didn't make the packets decode,
at least, though DDP did recognize the packet as being valid when I
added that \n.
>
> As for packets not decoding when complete, this is as per design.
...wait, I'm confused. Shouldn't something show up in the im-chat
program when the packet is valid and is received?
Also, re: the next version, have you thought about that VCS issue? My
offer is still open to help in any way you need with a GitHub
transition.
---BFO
Danny Knaggs
> > OK, I'll update the README for this on Ubuntu.
>
> > What version of fldigi is installed on the machine?
>
> 3.21.8 on Linux (current repository version), 3.21.9 on OSX.
>
> > Could be a bug,
> > however - add the extra \n as per previous message and see if that
> > solves it. If it does, I'll put it in the next version.
>
> It didn't-- the log I sent last time didn't make the packets decode,
> at least, though DDP did recognize the packet as being valid when I
> added that \n.
test I use a couple of rigs connected to a pair of dummy loads and set
the output power to 500mW.
Could be a issue with the soundcard, what soundcard are using for
testing? Try increasing the samplerate in fldigi to 48KHz, many cards
prefer that. Also try changing the converter to "best sinc
interpolator". I've also set the "slow CPU" option as well even
though I don't need to set it.
>
>
>
> > As for packets not decoding when complete, this is as per design.
>
> ...wait, I'm confused. Shouldn't something show up in the im-chat
> program when the packet is valid and is received?
it will state this when in debug mode.
>
> Also, re: the next version, have you thought about that VCS issue? My
> offer is still open to help in any way you need with a GitHub
> transition.
the moment.
>
> ---BFO
Danny.