Announcing dabtools project

[Dave Chapman]

Hi,

I've started work on a project I've called "dabtools", the core of which
I hope to turn into a library for DAB reception on Linux.

It is based on David Crawley's old OpenDAB software for the Psion
Wavefinder, and David May's more recent (but incomplete - it only
decoded the FIC) "rtl-dab" project to implement a DAB receiver for
RTL-SDR dongles.

It's still in the early stages, and the code needs a lot of cleaning up,
but I've created a github repository and uploaded my current code here:

https://github.com/linuxstb/dabtools

The README file describes the project in more detail, but the current
status is that it can record (to an ETI stream) an off-air DAB/DAB+
broadcast using either the old Psion Wavefinder or any of the cheap
RTL-SDR USB dongles.

There is also a simple eti2mpa filter to extract the MPEG audio for the
specified subchannel.

Whilst working on this, I found a few small bugs in the FIG parser in
etisnoop, and submitted a PR to fix those earlier this evening.

As an example usage, you can just do something like:

./sdr2eti 218640000 > dump.eti

to tune an RTL-SDR dongle to 218.64MHz and save the entire DAB ensemble
to an ETI file.

Or to play the service with sub-channel ID 3 using madplay:

./sdr2eti 218640000 | eti2mpa 3 | madplay -


My initial priorities are to clean the code (creating a library/API to
the DAB reception functionality), ensure the ETI stream is as correct as
possible, handle errors in the recorded stream and to handle multiplex
reconfigurations.

Any feedback welcome!

Regards,

Dave.

[coin...@yahoo.com]

Hi Dave,

Thank you for that. This looks great and promising !

I did an early test with a Noxon stick but have problems with synchronisation.
What is strange is that it can synchronise using rtl-dab but I will continue to investigate.
There were some changes in rtl-dab since February but I don't know if you have applied them.

I have not yet checked if you can decode DAB+. At least having an option to dump the subchannel (without decoding the DAB+ superframes) could already be useful.

The modular approach is interesting because it offers the possibility to use different hardware, if sdr2eti could take any IQ input then it could be easily used with USRP, HackRF or any other platform.
On the other side, the possibility, to get raw suchannel and stream them elsewhere can be interesting for making a remote controlled receiver.
In the past, there was a nice project dabusb from Baycom (http://www.baycom.de/hardware/dabusb/ ) where you could see all the ensemble information, including slideshow on a webpage and could stream the programs on HTTP without decoding/re-encoding them. Unfortunately the hardware it was working on has disappeared since years.

Regards

Mathias Coinchon

[Dave Chapman]

On 14/04/2015 10:04, coin...@yahoo.com wrote:
> Hi Dave,
>
> Thank you for that. This looks great and promising !
>
> I did an early test with a Noxon stick but have problems with
> synchronisation.
> What is strange is that it can synchronise using rtl-dab but I will
> continue to investigate.

That's odd. The only difference I can think of is that dabtools (at the
moment) only locks onto a signal when it finds 10 consecutive
transmission frames with no CRC errors in the FIBs. So if your
reception is borderline, rtl-dab may indicate a lock, but sdr2eti will not.

To test that theory, try uncommenting line 82 in sdr2eti.c - that will
display the number of FIBs in each TF with correct CRCs.

> There were some changes in rtl-dab since February but I don't know if
> you have applied them.
>

I've just taken a quick look at the commits to rtl-dab made over the
past few days, and I don't think any of them should affect the sync.
But I'll look properly this evening, and merge any changes which affect
the files I've forked into dabtools.

> I have not yet checked if you can decode DAB+. At least having an
> option to dump the subchannel (without decoding the DAB+ superframes)
> could already be useful.
>

DAB+ worked OK in my single test, using a combination of sdr2eti and
etisnoop.

> The modular approach is interesting because it offers the possibility
> to use different hardware, if sdr2eti could take any IQ input then it
> could be easily used with USRP, HackRF or any other platform.

Yes, I'm intending to make it support as many input formats/hardware
devices as possible. Once I fully unify the code for the Wavefinder and
RTL-SDR, I'll look at other things.

> On the other side, the possibility, to get raw suchannel and stream
> them elsewhere can be interesting for making a remote controlled receiver.
> In the past, there was a nice project dabusb from Baycom
> (http://www.baycom.de/hardware/dabusb/ ) where you could see all the
> ensemble information, including slideshow on a webpage and could
> stream the programs on HTTP without decoding/re-encoding them.
> Unfortunately the hardware it was working on has disappeared since years.
>

Yes, that's what I'm trying to avoid with dabtools - there seem to have
been various DAB applications over the years, but most are tied to
specific hardware. By abstracting the decoding to the ETI layer, I'm
hoping more useful applications can be built.

Regards,

Dave.

[Rash]

I'll second that this is really interesting announcement. I am looking forward to testing these - thank you Dave.

Best regards,

Rash.

--
You received this message because you are subscribed to the Google Groups "mmbtools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crc-mmbtools...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthias Brändli]

On 14. 04. 15 01:07, Dave Chapman wrote:
> Whilst working on this, I found a few small bugs in the FIG parser in
> etisnoop, and submitted a PR to fix those earlier this evening.

I was wondering who this linuxstb person was :-) Thanks for telling us
about this interesting development !

I have accepted your pull request for etisnoop. Keep us updated on your
project, and thanks for mentioning our wiki in your README.

Cheers,
mpb

[Dave Chapman]

Hi All,

A little progress update and a request for help.

Firstly, I now have a unified "dab2eti" program which works with both
the Wavefinder and the RTL-SDR dongles.

I've managed to incorporate an x86-optimised viterbi decoder into
dab2eti, and this has reduced the CPU usage (on my Intel(R) Core(TM)2
Duo CPU E6750 @ 2.66GHz decoding the full BBC National DAB Ensemble)
from about 51% to about 7% with the Wavefinder, and from about 68% to
about 28% on the RTL-SDR dongles.

I would like to add an "eti2aac" filter to extract DAB+ audio streams,
but don't currently have access to any DAB+ broadcasts. I would be
grateful if anyone reading this with access to DAB+ would make a short
(perhaps 1 or 2 minutes) ETI capture with dab2eti and provide me with a
copy.

If you do that, please make sure you use the very latest version of
dabtools - I accepted a PR from Stephan Pöschel this morning which fixed
a couple of bugs in the ETI output.

I've added a TODO list to dabtools, but if anyone else has any ideas for
features, please let me know.

Thanks,

Dave.

[888...@gmail.com]

Hi Dave

Will be very cool to see SNR measurement and signal level :)

Regards

[Dave Chapman]

On 19/04/2015 09:17, 888...@gmail.com wrote:
> Hi Dave
>
> Will be very cool to see SNR measurement and signal level :)
>
> Regards
>

Hi,

Agreed, but that's unlikely to come from me (my interest is in the
higher-level aspects). But I've added it to the to-do list..

Dave.

[Rash]

Hi Dave,

Greetings from sunny Brighton...

Are you aware of the etitools on the Open Digital Radio wiki?

They were made to extract ETI from the MPEG-2 transport streams on certain satellites, and extract the subchannels to (amongst other things) stream to an Icecas2 server.

I thought I might mention this in case it saves you some work...

I'll send you a way to get an ETI containing a dabplus service directly.

R.

[Dave Chapman]

Hi,

Yes, I know about etitools, thanks. I was curious about the MPEG-TS
code though - do you know which satellite(s) are broadcasting DAB
streams that way?

Thanks,

Dave.

[Rash]

Hi Dave,

I couldn't possibly say!

I'm told that King of Sat is quite good.... ;-)

R.

[Stefan Pöschel]

Hi Dave,

there are two german DAB ensembles broadcasted on Astra 3B 23,5°E I am
aware of (german board):
http://radioforum.foren.mysnip.de/read.php?8773,1012759,1231675#msg-1231675

They are both transmitted in ETI-NA G.704 format (decoding this to
ETI-LI is not funny at all) and are embedded into a pseudo MPEG-TS: In
each packet after the sync byte 0x47, just 187 bytes of data follow.

Regards,
Stefan

[Rash]

Hi Stefan,

Sergiy made some tools for a Dreambox DM500 I had at RadioHack for extracting from TS and converting from NA. 

I believe he has some updated ones on his GitHub which also support DAB+

Best regards,

Rash.

http://wiki.opendigitalradio.org/Ensemble_Transport_Interface

[Dave Chapman]

Hi Stefan,

Thanks for the info. I have access to a dish pointing to 23.5E, but I
don't think a normal DVB-S card can tune to them. At least, I've not
been able to.

Dave.

[Craig Barber]

Has anyone tried this with the streams on 5e?

[Dave Chapman]

Hi Craig,

dabtools is for receiving "normal" over-the-air DAB broadcasts.  For the satellite feeds, you can use eti-tools:

https://github.com/piratfm/eti-tools

I've done some tests yesterday, and am happy to confirm that an "off-air" recording of the BBC's UK national ensemble recorded with dab2eti is identical (apart from the order of sub-channels, which is arbitrary) to a recording of the BBC's DAB satellite feed converted to NI format with eti-tools.

Dave.

--

[Dave Chapman]

All,

I should clarify that there is one problem with dab2eti compared to the satellite feeds - dab2eti doesn't yet handle multiplex reconfigurations.  But that's high up on my to-do list.

Dave.

[Craig]

Guys, can you recommend a tool/application for tuning to, and piping/saving the MPEG-TS data off satellite to a file so that I can use the TS2NA application? I have been playing with 'szap' but I cannot work out how I get the information from the data channel, into TS2NA?

[Dave Chapman]

Craig,

I use my own dvbstream app:

https://github.com/linuxstb/dvbtools/

The command-line I use is:

dvbstream -c 1  -D 2 -f 12303000 -p h -s 25547 -fec 7_8 -o 1061 > recording.ts

-c 1 selects /dev/dvb/adapter1  (default is 0)
-D 2 is Diseqc switch position 2

Dave.

[Craig Barber]

Thanks Dave, then you just run

ts2na -p 1061 -s 0 -i recording.ts -o test.eti ? for example?

[Dave Chapman]

Craig,

For the BBC (and I think D1) feeds, the correct value of the -s parameter is "12" (the default).

For the MuxCo feeds on the same transponder, use -s 0.

Dave.

[Craig Barber]

Thanks Dave, seems to work this end! I need to study more into what -s does etc, as the read me doesn't explain much. I plan to try this out with a Dutch DAB ETI if I can find one on a satellite eventually!

Craig 

[Sergiy]

Hi, Craig. It's seen visually, when you having dump of the pid, which contains eti-stream. This offset is always repeating same "dummy" bytes at the beginning of each ts-packet and amount of that bytes are always the same.

четвер, 30 квітня 2015 р. 20:46:28 UTC+3 користувач Craig написав:

[Craig]

Thanks Sergiy, it all makes sense now! Has anyone managed to get anything off 23.5E? I'm trying, but failing miserably... King Of Sat doesn't have anything, and I tried that German messageboard but had the same problem as Dave, cannot seem to see anything on those frequencies, so cannot even get a PID to work with from here.

Craig

[Craig]

Dave,

One of your tools, eti2mpa - I have been trying to decode a channel without success, what does it mean by 'channel number'? I tried stuff like 0,1 etc, also tried the service/subchannel ID number without any luck..

Craig

[Dave Chapman]

Craig,

The parameter to eti2mpa should be the "sub-channel ID". 

You can use the ni2http utility from eti-tools to view the services in an ETI stream:

ni2http --list -i file.eti

If that command doesn't list the services, then there is perhaps something wrong with your ETI stream.

Dave.