odr-zmq2edi problems

[pswiatki]

Hello,

I am testing the DAB transmission chain (well, part of it - no TX yet)  based on ODR tools.

Since I am particularly interested in EDI stream, I thought I would pick up ZMQ stream from odr-dabmux

using odr-zmq2edi and dump the resulting EDI stream into some file for further analysis.

So, I use:

odr-zmq2edi tcp://127.0.0.1:9100 test.edi -d 127.0.0.1 -p 9111 -s 9200 -D -v

My DabMux shows this when starting up:

      --- Output list ---
      Output      0
        URI: zmq: tcp://*:9100
      Output      1
        URI: simul://
      Start loop
      inputZMQ sub-2 input pre-buffering complete

I also feed compressed audio to it using ZMQ - on port 9002:

odr-audioenc -l -b 192 -c 2 -V -v file:///[...]test.flac -o tcp://localhost:9002

This audio stream seems to be picked up nicely by DabMux as indicated in bold above.

However, when I try odr-zmq2edi I get:

      ZMQ2EDI converter from ODR-DabMux v2.1.1 starting up
      Setting up EDI Sender with delay 500 ms
      Setup EDI
      EDI set up
      EDI
       verbose     1
       to 127.0.0.1:9111
        source port 9200
      Opening ZMQ input: tcp://127.0.0.1:9100
ERROR Error: Incomplete metadata received

which happens almost immediately after running  odr-zmq2edi (I guess with this 500 ms delay mentioned in the above log).

I tried to debug this problem, but got an interesting result:

in zmq2edi.cpp I added some debug in get_md_one_frame() function (attached) and then I get:

      ZMQ2EDI converter from ODR-DabMux v2.1.1-dirty starting up
      Setting up EDI Sender with delay 500 ms
      Setup EDI
      EDI set up
      EDI
       verbose     1
       to 127.0.0.1:9111
        source port 9200
      Opening ZMQ input: tcp://127.0.0.1:9100
WARN  In get_md_one_frame(), size: 12 bytes
WARN  Input buffer hex dump:
WARN  buf[0] = 0
WARN  buf[1] = 0
WARN  buf[2] = 0
WARN  buf[3] = 0
WARN  buf[4] = 0
WARN  buf[5] = 0
WARN  buf[6] = 0
WARN  buf[7] = 0
WARN  buf[8] = 0
WARN  buf[9] = 0
WARN  buf[10] = 0
WARN  buf[11] = 0
WARN  utc_offset_received: FALSE
WARN  edi_time_received FALSE
WARN  dlfc_received FALSE
WARN  ID: 0, len: 0
ERROR Error: Incomplete metadata received

Looks like all metadata in this first frame are zeroes. Why is that? I have metadata enabled in the MUX config:

    zeromq {
        ; Listen on all interfaces, on port 9100
        endpoint "tcp://*:9100"
   
        ; Transmit backward compatible metadata containing
        ; EDI time and UTC offset when TIST is enabled.
        ; WARNING! requires ODR-DabMux to be compiled with
        ; EDI output, and this will enable leap second download
        ; as for the EDI output!
        allowmetadata true

Can getting and sending ZMQ data on the same address (but different port) be a problem here?

[Matthias Brändli]

Hi,

thanks for the detailed report, that makes it easier for me to reproduce.

I set up a multiplexer with TIST enabled and zmq output with metadata,
and your odr-zmq2edi command line (even though it looks wrong to me, see
below) works. If I set TIST to false, I get your behaviour. If I set
allowmetadata to false, I get an another warning and error from ODR-zmq2edi:

WARN Insufficient data to parse metadata
ERROR Error: Insufficient data

So the summary seems to be that ODR-zmq2edi only supports a zmq stream
that contains ETI with valid TIST and metadata. I have to admit I never
tried it without, because the use case we have for this tool is when
driving several modulators in an SFN. In a single TX situation, we
normally have the mux PC right next to the mod, and the EDI output of
ODR-DabMod works fine.

Quick fix for you: enable TIST in the mux.
Proper fix: modify ODR-zmq2edi to support "no TIST" and properly
document the use case.

I would be happy to accept a pull request or a patch by email for the
proper fix!


Regarding the ODR-zmq2edi command line: What is the "test.edi" supposed
to do?

The usage screen says
odr-zmq2edi [options] <source>

So your command line should be

odr-zmq2edi -d 127.0.0.1 -p 9111 -s 9200 -D -v tcp://127.0.0.1:9100

but I guess yours also works fine


Regarding ODR-AudioEnc: If you encode from a file, there is no
throttling, and it will encode faster than realtime. Have a look at the
scenarios in the README:

https://github.com/Opendigitalradio/ODR-AudioEnc/blame/master/README.md#L162


Cheers
mpb

On 01/04/18 23:23, pswiatki wrote:
> Hello,
>
> I am testing the DAB transmission chain (well, part of it - no TX yet) 
> based on ODR tools.
> Since I am particularly interested in EDI stream, I thought I would pick

> up ZMQ stream from *odr-dabmux*
> using *odr-zmq2edi* and dump the resulting EDI stream into some file for

> further analysis.
> So, I use:
>
> |
> odr-zmq2edi tcp://127.0.0.1:9100 test.edi -d 127.0.0.1 -p 9111 -s 9200 -D -v
>
>
> |
>
> My DabMux shows this when starting up:
>
> |

>       ---Outputlist ---

>       Output     0
>         URI:zmq:tcp://*:9100
>       Output     1
>         URI:simul://
>       Startloop

>       *inputZMQ sub-2input pre-buffering complete**
> *

> |
>
> I also feed compressed audio to it using ZMQ - on port 9002:
>
> |

> odr-audioenc -l -b 192-c 2-V -v file:///[...]test.flac -o

> tcp://localhost:9002
>
>
> |
>
> This audio stream seems to be picked up nicely by DabMux as indicated in

> *bold *above.

> However, when I try odr-zmq2edi I get:
>
> |
>       ZMQ2EDI converter fromODR-DabMuxv2.1.1starting up
>       Settingup EDI Senderwithdelay 500ms
>       SetupEDI
>       EDI setup
>       EDI
>        verbose     1
>        to 127.0.0.1:9111
>         source port 9200

>       OpeningZMQ input:tcp://127.0.0.1:9100
> ERROR Error:Incompletemetadata received

>
> |
>
> which happens almost immediately after running  odr-zmq2edi (I guess
> with this 500 ms delay mentioned in the above log).
>
> I tried to debug this problem, but got an interesting result:
>
> in zmq2edi.cpp I added some debug in get_md_one_frame()function
> (attached) and then I get:
>
> |
>       ZMQ2EDI converter fromODR-DabMuxv2.1.1-dirty starting up
>       Settingup EDI Senderwithdelay 500ms
>       SetupEDI
>       EDI setup

>       EDI
>        verbose     1
>        to 127.0.0.1:9111
>         source port 9200

>       OpeningZMQ input:tcp://127.0.0.1:9100
> WARN  Inget_md_one_frame(),size:12bytes
> WARN  Inputbuffer hex dump:

> WARN  buf[0]=0
> WARN  buf[1]=0
> WARN  buf[2]=0
> WARN  buf[3]=0
> WARN  buf[4]=0
> WARN  buf[5]=0
> WARN  buf[6]=0
> WARN  buf[7]=0
> WARN  buf[8]=0
> WARN  buf[9]=0
> WARN  buf[10]=0
> WARN  buf[11]=0
> WARN  utc_offset_received:FALSE
> WARN  edi_time_received FALSE
> WARN  dlfc_received FALSE
> WARN  ID:0,len:0

> ERROR Error:Incompletemetadata received

>
> |
>
> Looks like all metadata in this first frame are zeroes. Why is that? I
> have metadata enabled in the MUX config:
>
> |
>     zeromq {

>         ;Listenon all interfaces,on port 9100
>         endpoint "tcp://*:9100"
>    
>         ;Transmitbackward compatible metadata containing

>         ;EDI time andUTC offset whenTIST isenabled.
>         ;WARNING!requires ODR-DabMuxto be compiled with
>         ;EDI output,andthiswill enable leap second download

>         ;asforthe EDI output!

>         allowmetadata true
>
>
>
> |
>
> Can getting and sending ZMQ data on the same address (but different
> port) be a problem here?
>
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "mmbtools" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crc-mmbtools...@googlegroups.com
> <mailto:crc-mmbtools...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[pswiatki]

Thank you very much, Matthias!

I enabled TIST and odr-zmq2edi started working properly. I can see
EDI stream with expected frames in the edi.debug dump file.

And... yeah - I somehow missed some part of the command line which

was obviously wrong (test.edi) - interestingly, it did not cause the immediate
fault. But I appreciate your pointing this to me.

If I only can figure out how to modify the code to allow odr-zmq2edi

to work  without TIST present in the output stream of the multiplexer

- I will issue a pull request on Github.

I have another comment on the issue you mentioned in your response.

It is about the multiplexer not applying any back-pressure on the audio encoder,

but I guess it will be better to take this subject to a new thread.

Thanks again,

Peter

> <mailto:crc-mmbtools+unsub...@googlegroups.com>.

[Matthias Brändli]

Glad it works now!


> If I only can figure out how to modify the code to allow *odr-zmq2edi*

> to work  without TIST present in the output stream of the multiplexer
> - I will issue a pull request on Github.

Thanks a lot, looking forward to it!

mpb