c2gmsk modem: 2400 bps version and some questions on FEC

[Kristoff Bonne]

Hi all,


(message crossposted on codec2 and digitalvoice lists. Follow-up set to
digitalvoice list)


First of all an announcement. I've started implementing the 2400 bps
modem in c2gmsk API, the GMSK based modem for codec2.

I have posted it on github on a seperate branch:
https://github.com/on1arf/gmsk/tree/c2gmsk_2400bps

This modem is currently assigned versionid 2400/15, so please
concidering this still to be development software (modem versionid "15"
= experimental). Once the code has been thoroughly tested, the modem
will be renumbered to modem version 2400/0.

Implemented is:
- The 2400 bps GMSK modem itself

- interleaving 2400-to-1400 bps (ordering the bits as such that the most
important bits are protected by the FEC; but still distributed in the
frame) (*)
- the golay based FEC (mainly based on the code developed by David and
Robert Morelos-Zaragoza as used in freeDV).
- scrambling
- transmission interleaving


I have a couple of question for the list on the issue of FEC.


First some information:
- the 2400 bps modem is based on 40 ms frames, so 96 bits each.
- There are two types of frames:
-> frames that include a sync-pattern: 24 bits sync pattern, 56 bits
codec2 information, 1 * 12 bits golay FEC + 4 bits of spare
-> frames that do not include a sync-pattern: 56 bits codec2
information, 3 * 12 bits golay FEC + 4 bits of spare
The first frame of a stream must be a "sync" type-frame. After that,
there can be up to 7 "non-sync" frames.


My question:
1/ The golay FEC as implemented now is a the 12/23 scheme, so 12 bits of
data + 11 bits of FEC. However, the frame structure actually has place
for 12 bits of fec information. I did find some information on the net
about a 12/24 golay scheme (that should be a little better then the
12/23 scheme) but sofar I have not find any source-code for this.
- Has somebody already implemented fec 12/24 using open-source?
- is the extra work worth the effort? Does golay 12/24 provide a big
advantage over golay 12/23?


2/ As noted above, every 40 ms frame has 4 bits of unused "spare"
information. I see two ways these bits can be used:
- use it to provide hamming-code FEC for 3 bits
- reserve it for some other use in the future. (like a very low speed
data channel).

Any comments, ideas or proposals?


3/ As the current 4800 bps (version 4800/0) has a very basis FEC system
(1/3 repetition code), I have been thinking about designing a better
modem for 4800 bps. My idea is to use the format of the 2400 bps modem,
and apply a 1/2 convolutional code on top of that.

The issue that comes up is this: the gmsk stream contains a sync-pattern
that is inserted into the stream at certain points. The goal is double
- allow the receiver to detect bitslip
- allow a receiver that is not syncronised to pick up a ongoing
conversation. For this, the receiver needs to read the stream and look
for these "sync" patterns to know where to start decoding the stream.

This sync-pattern is 24 bits.


So, concidering this requirements, I see three ways to implement a 4800
bps version of the 2400 bps modem:
- simply apply a 1/2 FEC ontop of the complete 2400 bps stream,
including the sync pattern. That way, the sync-pattern will actually
become 48 bits!

- divide the stream into "windows" of 1 up to 8 frames. Every windows
starting with one "sync" frame and followed by up to 7 "non-sync"
frames. The convolution code would then be applied per window. The sync
pattern would NOT be part of the 1/2 convolution code FEC and would
-therefor- be 24 bits.
This would mean that per window, we would gain 24 bits of space.
However, some of it them will have to be used for the "tailbiting" of
the convolution coding window.

- Apply the 1/2 convolutional code to the stream without any windowing.
However, do not apply it to the sync-pattern that would -therefor- still
be 24 bit. Simply take out the sync-pattern from convolutional coding.
This is a sort of halfway implementation between the first two senarios.




Anybody any comments, ideas or proposals on this?

Who can provide links to open-source libraries to implement a 1/2
convolutional coding sceme?


(*) The interleaving-matrix for the 1400-to-2400 and 2400-to-1400
conversion can available on my blog. However, due to discussion between
myself and my ISP, my blog is currently not accessable. I will move that
information to github in one of the comming days.




73
kristoff - ON1ARF

[Marciniak, Ed]

Well, the four extra bits could be used for a few things. Ascii encoding a call sign with a comma character (seperator indicating repeat) or an indicator that the next frame will be last in transmission (ptt release can be detected by encoding computer prior to actually having completed the encode of the frame - possibly in time to indicate channel is about to be free). A different seperator character could be used to indicate someone is calling you with their call to follow. (No call sign prior to seperator and your call sign would be interpreted as a cq, dxcc area and seperator then your call to be interpreted as calling cq for a particular area). Another delimeter could seperate a call sign from a grid square.

If only certain formatting of the extra bits is legal and they have enough structure, they could allow rapid acquisition, or monitoring of channel fades or multipath.

The value of 1/24 of the channel is a fraction of a db with ideal coding....or put differently, it is more valuable if you had 100 bps available that eliminated the need to identify for a controlled net handling a disaster or provide a calling function.

73,

Ed
NB0M

--
You received this message because you are subscribed to the Google Groups "digitalvoice" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.
To post to this group, send email to digita...@googlegroups.com.
Visit this group at http://groups.google.com/group/digitalvoice?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

[Tom King]

could that be used for some ALE functionality too?

[Kristoff Bonne]

Hi Ed,


As mentioned in my other message, I have started implemeting a very
basic 100 bps auxiliary data service.

However, note that the c2gmsk (althou currently not implemented) does
have a header put in front of the stream.
As certain actions to be taken on the stream -like routing in a repeater
or callsign squelch- are dependent on this information, it makes sence
to put this information in front of the stream.
Anycase, a 100 bps channel is probably to slow for this. (especially if
you want to apply a FEC on the header, thereby doubling the size of the
header).




Anycase, I do have another request.

As mentioned, the modem currently has the version-id 2400/15, where the
"15" indicates that this is still an experimental version. Althou I have
done some basic tests, I do would like to see the modem tested much
more. Especially the low-level DSP functions are new.

The apitest applications do allow quick and easy tests to modulate a raw
c2 file to a WAV-file and decode it back to a c2-file. I have done so
with a number of files, but I would therefor ask people to run much more
tests on the modem and provide feedback on it.
If possible, is somebody with some scripting skills is able to decode
large audio-chuncks to codec2 and run them through the modem, that would
be very much appriciated.



73
kristoff - ON1ARF

[Kristoff Bonne]

Hi list,



On 05-06-13 12:33, Steve wrote:
> One of the limitations of APRS is the cycle time, so that any new
> protocol would likely include a position/velocity report on the order
> of seconds (probably not less than 10). So along with digital packet
> modes, and/or voice modes could be interleaved a GPS position/velocity
> report (with no repeater function of course). It would be a good way
> to produce a table of stations within receiver range. Maybe even a bit
> to signify manned or unmanned. --

Concidering the replies on the list and allow people to experiment, I've
started to implement this as part of the c2gmsk API.

To give the child a name, I've called it "the auxiliary data service"
(yes, kind-of copied of auxiliary data service as found on DAB which
does have a simular purpose). As such, the 4 bits per 40 ms codec2 frame
are therefor called the "auxiliary data channel" (aka "ADC").


These are the specifications:
- bitrate: 4 bits / 40 ms = 100 bits / second
- no error-detection or error-correction implemented on modem level
- interleaving IS applied: (as it is part of the
transmission-interleaving sceme of c2gmsk).
- part of modem 2400/0 modem of c2gmsk (*)
- will probably also be implemented in the 4800/1 modem (i.e. the 2400/0
modem with a 1/2 FEC on top of that). Even thou the 4800/1 modem does
have twice the raw bitrate of the 2400/1 modem, the bitrate of the ADC
will still be 100 bps.
- implemented as part of c2gmsk API release 20130606


The current implementation is very basic:

* public API calls
c2gmsk_sendmessage (sender chain): Queue or replace a message to be send
via the auxiliary data channel.

* messages returned from the API
- "C2GMSK_MSG_AUXDATADONE" (sender chain queue): a message indicating
that the to-send message has been completely sent.

- "C2GMSK_MSG_AUXDATARECMSG" (receiver chain queue): a complete message
has been received.



A auxdata "message" can be any text, delimited by a terminating NULL
(0x00) character. In the current implementation, a message is limited to
1024 octets (which is equivalent to a message of 81 seconds).
When no data is to be send, the ADC contains all NULL characters.


This API can be extended in a number of ways (or another API can be
build on top of this one). Things to do:
- queing of multiple messages to be send over the ADC. (calling
"c2gmsk_sendmessage" when transmitting a message will interrupt the
current transfer and can potentionally also mislead the "nibble counter".
- more robust end-of-message detection
- error-detection + (if possible) error-correction
- transfer of binary data (including NULL-characters)
- multiplexing multiple data-streams over the ADC
- automatic cyclic repetition of ADS messages.

However, as this is -I think- not part of the core design of the c2gmsk
modem or codec2 DV, this is something that I do not want to spend to
much time on. I propose that somebody else in the community to take the
lead in this.
As mentioned, the code for this will be incorperated in API release
20130606 which -I hope- I will be able to put online this weekend.


For people who want to work on the source-code for this:
Most code of the auxiliary data service can be found in the file
"c2_auxdata.h". These are a couple of functions that are called from the
main "c2gmsk modem api" (found in "gmskmodemapi.c"). In addition, there
is some code in "gmskmodemapi.h" for the definition of the datafields in
the "gmsk session" structure and in "gmskmodemapi.c" for the
initialisation of these fields, plus one support function to queue
received data to the message-queue (so they can be returned by the API
to the main application)
Doing a simple "grep auxdata *" in the source-directory should give a
good overview where all the parts are located.



(*) Note: the 2400/0 modem of c2gmsk currently carries the version-id
2400/15 (where "15" indicates a experimental modem) as the modem still
has to be tested more.



73
kristoff - ON1ARF

[James Hall]

Is it possible to get a bit error rate and use that information to determine when the link is good enough to back down on error correction in order to increase the data channel? Could be handy if you're close enough to communicate with someone that the FEC isn't really doing a whole lot I guess.

--
You received this message because you are subscribed to the Google Groups "digitalvoice" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice+unsubscribe@googlegroups.com.

[Bruce Perens]

This is not effective for mobile operation unless you're using full
duplex. You can drive into a "hole" during a single transmission.

[Marciniak, Ed]

That would be great if only two parties were involved. If multiple people were listening and you cut the fec back some people may not longer be able to decode. Even if you could, you'd have to have some protocol handshake. You could, I suppose, while talking have your computer transmit the signal to noise margin for each call sign recently heard. The people listening could decide whether to reduce their transmit power if everyone listening to them has a good margin.

The optimum strategy depends on whether you are transmit power limited, data rate limited, or other factors like burst (impluse) noise versus gaussian noise.

73,
Ed
NB0M

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.

[Bruce Perens]

On 06/07/2013 07:15 AM, Kristoff Bonne wrote:
> the modem currently has the version-id 2400/15, where the "15"
> indicates that this is still an experimental version. Althou I have
> done some basic tests, I do would like to see the modem tested much
> more. Especially the low-level DSP functions are new.

Hi Kristoff,

Can you put some time into layering this software? Right now, it seems
that the modem, FEC, interleaver, and all of the code for D-STAR like
operations are mixed together.

I would like to exepriment with a fast-sync version similar to what
David has done for HF, with a repeater connection reservation scheme so
that we can reserve a data path and then only have to send one byte to
indicate the source and destination thereafter. This, hopefully, can
mitigate the lost-header problem. It is obviously very different from
what D-STAR does today. To make this work, I am going to have to pick
the modem code out of the rest.

Thanks

Bruce

[James Hall]

Both good points. It is neat imagining what could become possible now with the level of access having open-source software entails. Gotta watch out for logical errors.

[Steve]

Voice is sort of like UDP: fire and forget. You can't really have packet retries on either one, although you can loose packets to minimum effect if the packets are small.

Voice without FEC can be very ugly sounding (screeches and squawks).

But, for ham radio, I agree: the black boxes should be selectable by the operator. CRC or FEC ON/OFF, Interleave ON/OFF/DEPTH, Data or Voice, Data and Voice...

[Bruce Perens]

I don't plan on doing without FEC, although I might want to do the FEC
as David has for HF.

I do, however, want to arrange the packet entirely differently, and I
very definitely don't want to have the dropped header problem. Because
of the narrow bandwidth of our codec, I think a connection-based
protocol could be a big win for repeater use. The first key-up would
include a short negotiation with the repeater, and it's a one-byte ID
only for header information for the next 5 minutes after that. If we
send the connection ID every half second or so, we get rid of the
dropped-header problem.

Connection basing lets us be as complicated with our routing as we
desire. So, you could connect automatically to the nearest open node
with your radio going through a list of frequencies until it succeeds in
a negotiation with one node, and then that node would route you to any
node you desire. You could renegotiate automatically as you move from
the range of one node to another. We can do all sorts of fun telephony
things.

Thanks

Bruce

[Kristoff Bonne]

James,


If you look at the demodulate apitest application, you will see that the API (and especially the demodulation chain) actually returns quite a bit of information.

For every "demodulate" call you make (where you provide 40 ms of audio), you get back a "message queue" structure. This structure contains a number of messages (at least one), and one of them is "FEC statistics".

Depending on the number of golay blocks in the frame (1 if it is frame with also a sync-pattern, 3 if it is a frame without a sync pattern), it returns one or three of these messages. The value returned in the message is the distance between the 23 bit structure before and after golay decoding (read: you compair the 23 bit code you have before golay FEC decoding with the code after FEC decoding. The value returned is the number of bits that are different).


The problem with the setup you mention is however not to retrieve the FEC data from the retrieved stream; the problem is that any system where the FEC parameters are changed on the fly depend on a feedback channel from the receiving side to the sending side. And as we are mainly dealing with half-duplex communication, this is not possible.
(if you want to keep the loop short enough).



73
kristoff - ON1ARF

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.

To post to this group, send email to digita...@googlegroups.com.
Visit this group at http://groups.google.com/group/digitalvoice?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "digitalvoice" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.

[Kristoff Bonne]

Hi Bruce,

I guess you are mainly interested in the demodulation chain.


Actually, if you look at all the message that are send back in the
"message queue" , there are two that might already interest you.

There are two "printbit" messages which are mainly used for debugging
purposes.

There is a message that is simply a 96 bit text-stream containing the 96
bits (as the characters "0" or "1") as decoded by the GMSK demodulator.
This is the raw data as received, completely without any syncronisation,
audio-inversion detection, or any other processing.

Another one also contains a textual representation of the data received,
but is syncronised with the c2gmsk stream format. It (usualy) will be 96
bits together with a marker: "S" : frame with sync-matter, "M": missed
sync, "E" (end sync detected) or some certain other special ones (mainly
situations where the "bitslip detection mechanism" has kicked in).
This should gives you the raw bitstream of the c2gmsk stream, before
applying all the higher layer operations (audio-inversion,
transmission-interleaving, scrambling, FEC decoding, interleaving).

In theory, it would be possible to add even more debug messages in the
API and return the data at these points in the stack too; but I am
wondering if this is would not make the API quite heavy; especially as
the information gathered from this is only interesting for a very small
group of people.


When I was working on these layers (fixing bugs), I simply dumped the
data at certain points in the stack, by just placing "printf"s in the
source-code at certain points and dump everything to files.
If you want to bypass certain code, just comment out the lines, and do
"make".


Is this what you would require?




Note that the upper layer actions (interleaving, scrambling) are all
very much depend on syncronisation.
- the interleaving matrix used is different for a frame that has a sync
pattern and one that does not (as you do not want to interleave the
sync-pattern).
The fact of detecting if a frame has a sync-pattern or not, is directly
linked to the receiver being in sync with the stream that comes in.

- The scrambling pattern used of a frame is directly linked to the
"framecounter". There are 8 different scrambling pattern used in
sequence. Every time a frame with a sync-pattern is received, the
counter to indicate what scrambling-pattern to use is reset.

- As a frame that contains a sync pattern is formated differently from
one that does not; FEC decoding and concequent decoding also depends on
proper syncronisation.



Note that the GMSK stream is sequencial, in contrast to the FDMDV2
stream where all carriers are send in parallel. While using the HF
modem, the impact of the syncronisation channel is spread out over the
complete frame.


In GMSK, the syncronisation happens much more "bursty", by adding a
sync-pattern in a frame from time to time. But keep in mind that these
syncronisation bits do take up quite a bit of place (25 % of the frame
that has a sync-pattern), place that otherwize is used for two more
golay-blocks in that particular frame.
Syncronisation does come at a price!!! The more sync-information you
have, the less FEC you can implement and the higher the S/N ratio must
be for a good decode.



73
kristoff - ON1ARF

[Kristoff Bonne]

Hi Bruce,

On 07-06-13 20:47, Bruce Perens wrote:
> I don't plan on doing without FEC, although I might want to do the FEC
> as David has for HF.
>
> I do, however, want to arrange the packet entirely differently, and I
> very definitely don't want to have the dropped header problem. Because
> of the narrow bandwidth of our codec, I think a connection-based
> protocol could be a big win for repeater use. The first key-up would
> include a short negotiation with the repeater, and it's a one-byte ID
> only for header information for the next 5 minutes after that. If we
> send the connection ID every half second or so, we get rid of the
> dropped-header problem.

Can you explain this is little bit more?

> Connection basing lets us be as complicated with our routing as we
> desire. So, you could connect automatically to the nearest open node
> with your radio going through a list of frequencies until it succeeds
> in a negotiation with one node, and then that node would route you to
> any node you desire. You could renegotiate automatically as you move
> from the range of one node to another. We can do all sorts of fun
> telephony things.

OK, but keep in mind that in the ham world, you are dealing with
repeaters that are mainly operated and funded by some local group. This
is quite different from a "network" where all nodes are under control of
one single organisation.




> Thanks
> Bruce
73
kristoff - ON1ARF

[Kristoff Bonne]

Steve,

On 07-06-13 20:25, Steve wrote:
> But, for ham radio, I agree: the black boxes should be selectable by
> the operator. CRC or FEC ON/OFF, Interleave ON/OFF/DEPTH, Data or
> Voice, Data and Voice...

YES for the people doing experimenting.

But, from a "USER" point of view, the situation is different. It
probably does not make much sence to have a protocol with 10 options and
buttons that can all be changed.
It makes both the modem and the upper-layer application quite complex.

It probably makes more sence to limit the modem to (say) a limited
number of different "profiles", with certain preselected options ,
designed for a specific usage. E.g.
- line-of-sight VHF/UHF
- mobile use
- 10 meter or VHF-low "DX"
- NVIS HF
- satellite
- ...


For that reason, the idea is that c2gmsk does provide the flexibility to
support different "modems". Every modem can then be tailored for a
certain application.



73
kristoff - ON1ARF

[Bruce Perens]

Hi Kristoff,

This is my scheme for enabling repeater digital voice at low bandwidth. By low, I think this could work at 1800 Baud rather than 2400, given David's work on HF. And of course that would let us have more channels and more effective power, and adjacent channels to use for non-speech data when we want that.

A node sends a broadcast packet every 10 minutes a with its callsign, a simplex/repeater flag, an "open/closed" flag regarding its availability to non-members, a yes/no flag regarding its willingness to route to other nodes, and arbitrary additional information. This facilitates automatic collection of the available nodes you can hear without any additional infrastructure, and also fulfills rules for station identification.

We can assume that routing happens over the internet, but it can happen over radio too. Source routing (the stations to use between the source and destination) is not supported, it is assumed that the node is smart enough to figure out the route or forward to a station that can. Repeaters will take "Repeat" as a destination and send to all listening stations on their output channel. Simplex nodes can still route to/from other stations or repeaters while using only one channel.

To begin a communication with a node (which may be a repeater), a station automatically sends it a connect request with the requested destination. The node sends back a connection ID byte or a refusal. We make this happen as quickly as possible, so that the communication still feels like FM. Refusals may include other information, such as a list of other nodes to try or the state of the node you are attempting to connect to. So, the operator may hear the refusal tone or the radio may just renegotiate and put through the communication.

For the next 5 minutes, any communication the node receives with that connection ID byte is routed according to the information sent with the request. The station can renew the request every few minutes with a trailer at the end of a voice communication, or it can tell the node to tear down the connection if the operator hangs it up. Hang-up may come from the operator turning the channel knob, or an explicit button push, etc.

The connection ID is sent often enough that if the node loses sync during a deep fade, it can re-establish the connection quickly. Say every half second. This also makes doubling (a second station sending over one it can't hear) easier to determine. Stations will not double over a repeater that they can hear, the software will prevent that.

A really busy repeater can tell stations that their connection is no longer valid, if it has to reuse the connection ID byte before it times out. This would happen in least-recently-used order.

Obviously I am trying to think outside of the D-STAR, IP, or AX.25 box. Ideas appreciated.

Thanks

Bruce

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Bruce Perens]

On 6/7/2013 6:03 PM, Bruce Perens wrote:
>
> This is my scheme for enabling repeater digital voice at low bandwidth.

About sync, I was thinking of using one of the existing timing phase
recovery algorithms, plus a reasonably short sync symbol at the start of
a frame. So, these would require changes to the modem.

[Kristoff Bonne]

Hi Bruce,


I'm sorry for the late reply. It has been quite a strange weekend, so I
just had time to reply to you today.

(inline comments)

(I'll reply to both of your messages at the same time).

On 08-06-13 03:03, Bruce Perens wrote:
> Hi Kristoff,
>
> This is my scheme for enabling repeater digital voice at low
> bandwidth. By low, I think this could work at 1800 Baud rather than
> 2400, given David's work on HF. And of course that would let us have
> more channels and more effective power, and adjacent channels to use
> for non-speech data when we want that.
> A node sends a broadcast packet every 10 minutes a with its callsign,
> a simplex/repeater flag, an "open/closed" flag regarding its
> availability to non-members, a yes/no flag regarding its willingness
> to route to other nodes, and arbitrary additional information. This
> facilitates automatic collection of the available nodes you can hear
> without any additional infrastructure, and also fulfills rules for
> station identification.

I do have a bit problem try to understand your message.

I do understand your point of a 1800 bps modem, but I have a bit of an
issue why this should be related to your proposal on repeaters.
The latter is -correct me if I wrong- is basically that the header of
stream is replaced by a token which is used for later streams. But how
does this relate to the 1800 bps modem?

> This is my scheme for enabling repeater digital voice at low bandwidth.

> About sync, I was thinking of using one of the existing timing phase
> recovery algorithms, plus a reasonably short sync symbol at the start
> of a frame. So, these would require changes to the modem.

OK, let me see if I understand this.

GMSK works completely in the time domain. The two main issues with it are:
- correctly detecting the beginning of a frame
- keeping the receiver is sync with the sender.

Now, for the latter part. That is already done via the software PLL that
is in the receiver that keeps the receiver in sync with the sender. As
long as a signal with sufficient alternations of 0s and 1s is received,
this should be good enough. I guess this is what you mean with the
"timing phase recovery algorithms".

For the first part, there are three senarios:
- syncing on the beginning of a stream
- keep synced on a stream, even if it is not received for a small periode
- syncing on a stream where the receiver starts listening to a stream in
the middle of a QSO.

The problem is not so much "find the sync-pattern is the received
stream". The problem is finding is even in a stream with lots of QRM and
hence lots of bit-errors.

In the 2400 bps modem, the sync-pattern is 24 bits. Which is pretty
long, but it allows to define a rule like this:
A "valid" sync is
- if the sync-pattern is received at the place in the stream where we
thing it should be, with less then 6 bit-errors
- if the sync-pattern is received one position to the left or to the
right, with less then 3 bit-errors
- if the sync-pattern is received two positiions to the left or to the
right, with no biterrors

If you use a short sync-pattern, this really reduces the ability to deal
with errors, especially burst-errors, in the sync-pattern.
Also, if the sync-pattern is short, it is difficult to find a
bit-pattern that does not resemble itself to much when shifted one or
two positions to the left or right. Otherwize, you cannot detect bitslip.



Now, I just two question about a possible 1800 bps mode:

A 4800 bps GMSK signal is about rougly 8 Khz wide (if you place the edge
of your channel at -30 db). A 2400 bps is 4 Khz wide and a 1800 bps is
therefor about 3 Khz wide.
If you concider simplex channels. This means that you would need to have
at least 3 channels next to eachother in a 12.5 Khz raster to have the
advantage of being able to add a 4th channel in that raster by using
1800 bps instead of 2400 bps. Do you concider this to be a practicle
situation?

For me, the advantage of the 4 Khz bandwidth of a 2400 bps is that it
fits in both the 5 Khz raster on 10 meter or 4 meter and therefor also a
6.25 Khz raster on the other bands, something that cannot be said about
4800 bps GMSK.
For repeaters, I kind of fail to see the advantage of reducing the
bandwidth even further, as there is no smaller raster-size anyway. In
repeater-channel rasters, you place the stations in the middle of the
raster to limit the interference to other repeaters (especially analog
repeaters) in the next channel.


- For the 4800 bps modem, the spectrum of the AF signal is 0 to 2400 Hz.
For the 2400 bps modem, it's 0 to 1200 Hz and the 1800 bps modem would
reduce that further to 0 to 900 Khz.
I'm just wondering how flat the frequency-respond of a typical FM
receiver / transmittor (on the "9k6" dataport) is in that
frequency-range. (as we do are talking about using standard FM
transceivers; not

Remember that we are using FM here, not SSB. In FreeDV, you can place
the signal at (e.g.) 1000 Hz to 2250 Hz of the AF spectrum, avoiding the
lower frequencies of the AF spectrum. In GMSK, the AF frequencies
always go from 0 up-to bps/2 Hz.


- Thinking about alternative modulation scemes. Anybody any idea what is
the maximim bitrate you can practically get over a FM transceiver using
AFSK (read: packet, APRS)? Could you get 1800 bps over AFSK over a
FM-tranceiver without needing a 9k6 port?



73
kristoff - ON1ARF

[Phil Frost]

On 06/07/2013 09:03 PM, Bruce Perens wrote:
> This is my scheme for enabling repeater digital voice at low
> bandwidth.

It sounds to me that a lot of this functionality, and the implementation
details necessary to obtain it, may be defined in a good many
network protocols already, many of which have mature and free (as in
freedom) implementations available. Not all of them are impossible to
implement on a low bandwidth link.

I find it hard to make specific suggestions since there doesn't seem to
be a clear specification of requirements. But for example,
https://en.wikipedia.org/wiki/Mesh_networking mentions at least a dozen
mesh networking protocols that do something close to this:

> A node sends a broadcast packet every 10 minutes a with its callsign,
> a simplex/repeater flag, an "open/closed" flag regarding its
> availability to non-members, a yes/no flag regarding its willingness
> to route to other nodes, and arbitrary additional information.

I think a more deliberate attempt to re-use existing solutions will
advance the state of ham radio communications a lot faster than
inventing new protocols. Ham radio networking technology is about four
decades old, and we won't get anywhere new by making incremental
improvements on that.

[br...@perens.com]

On 6/10/2013 2:27 PM, Kristoff Bonne wrote:
>
> I do understand your point of a 1800 bps modem, but I have a bit of an
issue why this should be related to your proposal on repeaters.

My assumption is that one of the reasons we have for using more bandwidth
than the codec alone would need is that headers are large and have to be
transmitted often if we want to deal with drop-outs. The other reason is
FEC, and David has done a good job of showing that the S/N lost by using a
wider signal is not necessarily paid back by the FEC that you can put in
that wider signal. So, I am thinking about duplicating David's work on HF
and going for efficient use of power rather than error correction.

>
> Now, for the latter part. That is already done via the software PLL that
is in the receiver that keeps the receiver in sync with the sender. As
long as a signal with sufficient alternations of 0s and 1s is received,
this should be good enough. I guess this is what you mean with the
"timing phase recovery algorithms".

Yes.

>
> In the 2400 bps modem, the sync-pattern is 24 bits. Which is pretty
long, but it allows to define a rule like this:
> A "valid" sync is
> - if the sync-pattern is received at the place in the stream where we
thing it should be, with less then 6 bit-errors
> - if the sync-pattern is received one position to the left or to the
right, with less then 3 bit-errors
> - if the sync-pattern is received two positiions to the left or to the
right, with no biterrors
>
> If you use a short sync-pattern, this really reduces the ability to deal
with errors, especially burst-errors, in the sync-pattern.
> Also, if the sync-pattern is short, it is difficult to find a
bit-pattern that does not resemble itself to much when shifted one or
two positions to the left or right. Otherwize, you cannot detect
bitslip.

Understood. Now, let's take this 24-bit pattern and distribute it in time,
so that we have just a few bits at the start of every codec frame. We know
how far apart in time these bits are supposed to be. So, our confidence
that we actually have sync is a number that is integrated over time. Our
PLL can keep the clock going over some duration of drop-out and still
maintain the clock well enough to capture the next sync symbol.

We can also have a longer sync pattern at the beginning of a transmission,
and a separate algorithm for syncing on that. But I am most concerned with
being able to maintain sync as well as we do now on HF through a drop-out,
_and_ being able to maintain the connection (meaning the header data)
through a drop-out.

>
>
>
> Now, I just two question about a possible 1800 bps mode:
>

> This means that you would need to have at least 3 channels next to each
other in a 12.5 Khz raster to have the advantage of being able to add a
4th channel in that raster by using 1800 bps instead of 2400 bps. Do you
concider this to be a practicle situation?

In practice it's two local 3 KHz channels and two between them that are
allocated at some distance and thus work like guard channels. I am
certainly asking for more frequency precision than our current radios
might offer. A radio should not capture a signal on the adjacent channel,
and should not overlap the adjacent channel too badly, so we might have to
require a TCXO. Obviously I am talking about pushing the technology, not
retrofitting existing radios.

>
> For repeaters, I kind of fail to see the advantage of reducing the
bandwidth even further, as there is no smaller raster-size anyway.

But isn't this a chicken-and-egg problem? Give us radios capable of
narrower channels, and we will allocate narrow channels. I assure you that
the frequency coordinators out here are under enough pressure.

>
> I'm just wondering how flat the frequency-respond of a typical FM
receiver / transmittor (on the "9k6" dataport) is in that
frequency-range. (as we do are talking about using standard FM
transceivers

The target platform I'm shooting for is a Whitebox derivative
https://github.com/testaco/whitebox/#readme
I have one of these on hand, and will have a second soon, and am driving
them with an Android touchscreen system.

I think that existing FM or D*STAR radios are OK for experimenting with
Codec2, but they will mostly remain FM or D*STAR radios for their
lifetime. We really need the "HT of the Future" or at least a
purpose-built commercial HT to get large numbers of people on the air with
this mode.

Thanks

Bruce

[br...@perens.com]

Hi Phil,

There are definitely existing things we can use like OSPF. But I am at a
loss to come up with an existing connection-based protocol that is
optimized for ultra-low bandwidth and single turn-around per negotiation.
I think these would both be requirements for what I envision. SIP and
Jingle are obvious candidates, but they are too big and have too many
turn-arounds. However, we could design the required protocol so that it
maps in some way to SIP.

Thanks

Bruce

[jdow]

Just how are you proposing using existing FM radios and repeaters?

If you are talking about using FreeDV's audio output to drive an FM
radio your performance is going to be a somewhat wretched compromise
and low bit rates may not purchase you nearly as much performance
increase as you'd like. It will work, however. It just won't work as
well as you'd like. At high Eb/N0 figures the performance will be
good enough that a whole lot of FEC is wasting bits. At low Eb/N0
levels be aware that FM demodulator output is not pure WGN.

If you try to wrangle FreeDV as used on HF into the UHF spectrum I'd
advise against it. It's optimized for a selective fading HF path. It
needs to be tuned within tens of Hz for it to pull in and lock. If
you try to use narrow, even 3 kHz channels at UHF be aware that 1 ppm
is 10 times more error on 450 Mhz than the pull in range of FreeDV,
roughly speaking. 1 ppm oscillators are expensive. 0.1 ppm oscillators
are much more expensive. And they'll fall out at 23 cm.

For UHF work it's best to use bandwidth occupancy to best advantage.
For that, however, you must use techniques the FCC flat out frowns
upon like serious spread spectrum. Then 1 ppm or even 10 ppm can
become completely adequate for most communications needs. Orthogonal
codes become your friends. (e.g. GPS navigation signals.)

{^_^} Joanne/W6MKU

[Marciniak, Ed]

This would probably be a great time to bring up a couple items....

Shannon's limit theorem: to paraphrase a bit, a finite amount of power is required to transmit a piece of information. If you double the number of bits by transmitting fec (2:1), and had a perfect demodulator that could still recover a perfect output with a gross 50% error rate, you wouldn't actually come out ahead. Depending on the exact characteristics of the interfering signal and computational resources available, a suboptimal fec scheme may have a practical advantage.

There are several types of noise:

Random impulse noise (which tends to cause short bursts of errors)
Gaussian white noise (strictly average errors with no discernable bursts)

Repetitive errors from partial coherence with another signal could possibly be classified as a combination of the other two. A frequency hopper that jams your channel in a pseudorandom fashion with a finite (and short) sequence length would be an example.

In some cases a smaller amount of fec in one layer and interleaving data in time can result in a 'perfect channel' with less total bits sent, but there is a price in decoding delay. (Turbo codes are worth googling)

Other possibilities are things like protecting certain bits in a frame as a compromise in delay, bit rate, etc.

If a delay before the first bits decoded are available equals an entire frame, and enough computational resources are available, other techniques could be used such as avoiding a pll altogether by estimating parameters that result in a maximal output of a correlation function and iteratively solving for a frame that decodes correctly or giving up.

----- Original Message -----
From: jdow [mailto:jd...@earthlink.net]
Sent: Monday, June 10, 2013 06:32 PM
To: digita...@googlegroups.com <digita...@googlegroups.com>

[Matthew Pitts]

Joanne, Bruce and Kristoff are discussing VHF/UHF specific modems here, as well as software defined radio hardware to use it on. Matthew Pitts N8OHU Sent from Yahoo! Mail on Android

---

From: jdow <jd...@earthlink.net>;
To: <digita...@googlegroups.com>;

Subject: Re: [digitalvoice] Re: layering the 2400 Baud modem

Sent: Mon, Jun 10, 2013 11:32:59 PM

-- You received this message because you are subscribed to the Google Groups "digitalvoice" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice+unsub...@googlegroups.com.

[jdow]

I saw the mention of repeaters in his discussion. Existing repeaters are
not going to work unless you find a bent pipe repeater (RF to IF to RF
with no demodulation (RF to IF to Audio to IF to RF). (The term comes
from SatCom usage. Think of a flow of water (RF) through the satellite
envisioned as a U shaped pipe.)

I also saw the mention of narrow bandwidths. They're not going to work
very well. A wide band set of orthogonal codes with a moderately high
chipping rate and a low bit rate would, if legal, preserve the number
of channels and the lack of precise frequency control. Yes, this is an
anti-jamming approach. But, noise of many kinds can be considered to be
jamming as can signals on a different code.

Narrow band transmission is not really a technique suitable for UHF and
above and is dubious at VHF. Narrow band data, however, simply purchases
you a greater anti-jamming margin. GPS signals are a rather dramatic
example of this with their 50 dB give or take some AJ margin.

I saw two inappropriate approaches mentioned so thought I'd chip in with
a couple bits worth of information from long experience in the ECCM
field.

{^_^} Joanne/W6MKU

On 2013/06/10 17:14, Matthew Pitts wrote:
> Joanne,
>
> Bruce and Kristoff are discussing VHF/UHF specific modems here, as well as
> software defined radio hardware to use it on.
>
> Matthew Pitts
> N8OHU
>
> Sent from Yahoo! Mail on Android
>
>

> --------------------------------------------------------------------------------
> *From: * jdow <jd...@earthlink.net>;
> *To: * <digita...@googlegroups.com>;
> *Subject: * Re: [digitalvoice] Re: layering the 2400 Baud modem
> *Sent: * Mon, Jun 10, 2013 11:32:59 PM

[br...@perens.com]

> I saw the mention of repeaters in his discussion.

Yes. But not existing repeaters. SDR repeaters.

> A wide band set of orthogonal codes with a moderately high
> chipping rate and a low bit rate would, if legal, preserve the number
> of channels and the lack of precise frequency control.

Yes, but I can't wait for the rules to change that way.

Also, I am planning on use of nonlinear amplification for battery
efficiency, so some sort of FSK like GMSK would be appropriate.

> Narrow band transmission is not really a technique suitable for UHF and
> above and is dubious at VHF.

Just because of frequency precision, or some other issue?

Thanks

Bruce

[jdow]

This is generally due to frequency precision needs. If you adopt a
modulation that can tolerate a kilo Hertz or more of error then typical
ham radios will work through 450 MHz or so. Remember that at UHF one
ppm of error is a big number.

So you either waste power on a pilot carrier, waste time locking to the
signal, or some combination of the two. Standard direct sequence spread
spectrum offers better performance all around. Of course, you do still
have the acquisition problem.

Frequency precision might be mitigated with custom digital repeaters
that transmit a continuous synchronization signal when not repeating
data. Then a long synchronization would happen once on QSY with a
short synchronization for each new transmitter.

{^_^}

[Matthew Pitts]

In other words, take a page from the ETSI DMR specifications? Or perhaps from the NXDN specifications, since those are FDMA rather than TDMA. Personally, I would rather see ham gear go back to being focused on ham frequencies instead of being cheap (figuratively speaking) scanners.

Matthew Pitts N8OHU Sent from Yahoo! Mail on Android

---

From: jdow <jd...@earthlink.net>;
To: <digita...@googlegroups.com>;

Subject: Re: [digitalvoice] Re: layering the 2400 Baud modem
Sent: Tue, Jun 11, 2013 2:15:01 AM

[Marciniak, Ed]

Or you digitize at least one full frame, recover some analytic property, then decode the entire frame....

for example if the code had a limited number of zero or one bits, but the sync and end of frame had a specific number , that would leave behind a telltale spectral line as well as symmetric sidebands that are artifacts of it starting and stopping. The strongest spike could be recovered with phase and the data sliced. If a phase jump at end of frame is noted relative to synch, one might even be able estimate certain distortions and try and solve the integer ambiguity (if a device has excess cpu capacity to do so, or even skip the extra work if a good decode occurs on the first try).

Yeah, I know....deterministic requirements are good and what I'm suggesting doesn't have that property....with good power management, however, you might get away with putting the cpu to sleep when it isn't busy and that could offset increase power consumption of a faster processor.

----- Original Message -----
From: jdow [mailto:jd...@earthlink.net]

Sent: Monday, June 10, 2013 09:15 PM
To: digita...@googlegroups.com <digita...@googlegroups.com>
Subject: Re: [digitalvoice] Re: layering the 2400 Baud modem

--
You received this message because you are subscribed to the Google Groups "digitalvoice" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.

[jdow]

FWIW I remember the "good old days" of the 60s. They were not all that
golden good. I remember fondly in the 70s ad hoc QSOs from Glendora Ca
to mobiles in Oz on SSB. To an extent those days are still around. But
we also have new modes that trade off distortion of the audio for noise
on the audio. The experimentation is fun and generally worth it.

FM is nice for rag chewing - but, what repeaters encourage an hour
monopolized by a small group rag chewing about whatever? Yeah, I
used to do 6 meters and 2 meters SSB. Those days seem to be lost.
I remember some fun QSOs talking technology with other whizkids
up there - about 6 of us used to natter for hours about such things
without quite letting cats out of the bag to competitors for what
we were doing. (I was working calibrated to a few ppb oven standard
I'd spent months carefully tweaking in to WWV until tweaks were not
really needed on the leading edge of the 1 kHz tick. GPS rather spoiled
that fun. Mea culpa - I helped get those birds up there working. It
has some strange circuitry. I seem to approach some problems from off
beat directions.)

Technology moves on. Keeping abreast of it is a good thing, though.
That is a fun challenge, too.

Thinking of which, there is a mode of operation I don't fool with
that would be a nice challenge for CODEC based phone operation -
contesting. Can FreeDV be tweaked to improve its use in contesting?

{^_^}

On 2013/06/10 19:26, Matthew Pitts wrote:
> In other words, take a page from the ETSI DMR specifications? Or perhaps from
> the NXDN specifications, since those are FDMA rather than TDMA. Personally, I
> would rather see ham gear go back to being focused on ham frequencies instead of
> being cheap (figuratively speaking) scanners.
>
> Matthew Pitts
> N8OHU
>
> Sent from Yahoo! Mail on Android
>
>

> --------------------------------------------------------------------------------
> *From: * jdow <jd...@earthlink.net>;
> *To: * <digita...@googlegroups.com>;

> *Subject: * Re: [digitalvoice] Re: layering the 2400 Baud modem
> *Sent: * Tue, Jun 11, 2013 2:15:01 AM

> to digitalvoice...@googlegroups.com. <javascript:return>

> To post to this group, send email to digita...@googlegroups.com.

> <javascript:return>

> Visit this group at http://groups.google.com/group/digitalvoice?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "digitalvoice" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to digitalvoice...@googlegroups.com.

[Steve]

I think FreeDV needs a recorder function.  Basically you record something and assign it to a button. You could record "CQ DX DE XX" or the Contest joke "You're 59 QRZ".

But rather than store the audio, you just store the codec bytes, so when you push the button, the bytes are sent through the codec and off to the modem.

[Steve]

I was thinking also, since GMSK is probably going to have a bit scrambler, you're also going to have say 23 bits at the start, and end, for that purpose.

[Kristoff Bonne]

Hi Bruce,


With some delay due to other priorities. (coding, bugfixing, ... :-) )

On 11-06-13 00:58, br...@perens.com wrote:
> On 6/10/2013 2:27 PM, Kristoff Bonne wrote:
>> I do understand your point of a 1800 bps modem, but I have a bit of an
> issue why this should be related to your proposal on repeaters.
>
> My assumption is that one of the reasons we have for using more bandwidth
> than the codec alone would need is that headers are large and have to be
> transmitted often if we want to deal with drop-outs.

To be honest, hamradio is half-duplex so I don't think a 200 ms or 300
ms or 400 ms delay at the beginning of the stream due to the header is
such an issue.

> The other reason is
> FEC, and David has done a good job of showing that the S/N lost by using a
> wider signal is not necessarily paid back by the FEC that you can put in
> that wider signal. So, I am thinking about duplicating David's work on HF
> and going for efficient use of power rather than error correction.

OK, but -for me- the question is if it will work with with a normal
FM-radio; which is -for me- the main platform I aim for.

First designing an open-source modem, then writting software for it, and
... at the end ... demanding that people by a new radio .. is (IMHO) not
the best way to get people to use it. :-)


I do it could be usefull for a couple of other reasons:
- for any possible SDR-based radios
- for simulation purposes
- as the basis for a 3600 bps modem (simular like the 4800/1 modem I
want to do: take the 2400 bps modem and add a 1/2 convolution-code FEC
saus on top of it, so do the same thing here).

However, I will need some help from some other people about this, in
particular with the low-level DSP stuff for a 1800 bps modulator and
demodulator.

Also, for now, somebody can already start with a interleaving matrix for
this. I guess the FreeDV code should contain the information what 12
bits should be protected by the golay code. But the data still need to
be interleaved so that "important" or "related" bits are located as far
as possible from eachother to be as less as possible, this to make it
less vulnerable to burstnoise.





Another thing that might be interesting to look at (by people who know
better DSP then myself) is whether it would be possible to convert the
low-level filters in the API so that they input and output I/Q signals
directly. I don't know how easy / difficult it is to convert the filters
direclty and if this would be faster? This would probably make the API
more interesting for SDR applications.



So, people .... people with DSP and SDR knowledge wanted !!!! :-)

> Understood. Now, let's take this 24-bit pattern and distribute it in
time, so that we have just a few bits at the start of every codec frame.
> We know how far apart in time these bits are supposed to be.

I had also been thinking about that. It's more difficult to code bit I
do think it is be possible.


The only issue is that c2gmsk uses a special trick, as the sync-patterns
in the stream not only to determine the sync, but also to mark the end
of a stream. At the end of a stream, a use a different sync-pattern then
during an ongoing stream.

I also want to use the same trick to mark other types of data in the
future like the header or a "silence" frame (i.e. the audio is played
out as silence by the API but the radio-frame actually containing a
data-payload for a "slow-speed data" service).


If you pattern is spread out over multiple frames, we might need to add
a couple of frames of silence at the end of a stream before we are
really able to send the "end-of-stream" marker completely.
I don't know how what to do for the slow-speed data service.

>> I'm just wondering how flat the frequency-respond of a typical FM
> receiver / transmittor (on the "9k6" dataport) is in that
> frequency-range. (as we do are talking about using standard FM
> transceivers
>
> The target platform I'm shooting for is a Whitebox derivative
> https://github.com/testaco/whitebox/#readme
> I have one of these on hand, and will have a second soon, and am driving
> them with an Android touchscreen system.
>
> I think that existing FM or D*STAR radios are OK for experimenting with
> Codec2, but they will mostly remain FM or D*STAR radios for their
> lifetime. We really need the "HT of the Future" or at least a
> purpose-built commercial HT to get large numbers of people on the air with
> this mode.

As explained, my priority now is allowing people to experiment.

"Future radios" are all very nice, but I do not want to force people to
buy a new radio, just to be able to run a now modem.

For me, the 1800 bps in on the "to do" list, but surely not on the top.

On the top of my list (for the c2gmsk project, that is) is
1/ addapt the API so to input and output "GMSK-bits" instead of
modulated audio-signals. That is to support the hardware GMSK board
(arduino / raspi plugin) now being designed by Jim KI6ZUM (see here:
http://ki6zum.com/dstar/dv_overview.htm)

This means that the API would still do bitjugling, FEC, and what have
you, but no GMSK modulation / demodulation.


2a/ create a 4800 bps modem based on the 2400 bps modem, but with a FEC
of 1/2 on top of it. (as the CMX589 GMSK chip of the board being
designed by Jim only support bitrates of 4800 bps upwards).
2b/ A gnuradio module for c2gmsk.

I still need to descide which one of these two will come first.


3/ implement the header

4/ Implement the API in applications (like tlb or allstar).




This is unless somebody can provide a good reason to implement a 1800
bps version earlier. (what about 1800 bps over AFSK and FM-transceivers
without 9k6 ports? Is this possible)

[Matthew Pitts]

Kristoff, Regarding AllStar: I could probably do the channel driver for that, since I also need to enable the AllStar Link version of Asterisk to handle Codec2; there are some major differences between the 1.4 series and the 1.8 and later code that needs to be dealt with by someone, and I have the test platform in place for it already.

Matthew Pitts N8OHU Sent from Yahoo! Mail on Android

---

From: Kristoff Bonne <kris...@skypro.be>;
To: <digita...@googlegroups.com>;
Subject: Re: [digitalvoice] Re: layering the 2400 Baud modem
Sent: Sat, Jun 15, 2013 11:03:46 PM

-- You received this message because you are subscribed to the Google Groups "digitalvoice" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice+unsub...@googlegroups.com.

To post to this group, send email to digita...@googlegroups.com.

Visit this group at http://groups.google.com/group/digitalvoice.

[Kristoff Bonne]

Hi Matthew,


I just had a chat with Ken on the experimenter group about allstar and also about these channel drivers.


Actually, from what I understand, the software components are already in place. What the driver needs to do is
- codec2 encoding/decoding
- formatting for whatever digital voice system you want to use (c2gmsk, fdmdv2)
- modulation

So this is basically what the current c2gmsk modem plus the codec2 API written by David.


If we get c2gmsk into allstar, we have already one repeater platform capable of using it.



73
kristoff - ON1ARF

To unsubscribe from this group and stop receiving emails from it, send an email to digitalvoice...@googlegroups.com.

[Tony Langdon]

On 18/06/13 6:42 AM, Kristoff Bonne wrote:

Hi Matthew,


I just had a chat with Ken on the experimenter group about allstar and also about these channel drivers.


Actually, from what I understand, the software components are already in place. What the driver needs to do is
- codec2 encoding/decoding
- formatting for whatever digital voice system you want to use (c2gmsk, fdmdv2)
- modulation

So this is basically what the current c2gmsk modem plus the codec2 API written by David.


If we get c2gmsk into allstar, we have already one repeater platform capable of using it.

If we get it into thelinkbox, then any system based on that software could use it there too - as well as anything thelinkbox can transcode to (which includes IRLP and Echolink).  There's two open source platforms that are already out there.

-- 
73 de Tony VK3JED/VK3IRL
http://vkradio.com

[Matthew Pitts]

My thought was to make Codec2 one of the supported codecs for AllStar Link, much like David Rowe did with a later version of Asterisk, which would enable sip clients that have that as an optional codec could use it. That would leave the channel driver to handle the modulation and  formatting, though if c2gmsk used the same basic format as D-Star, this would be simpler to implement.

Matthew Pitts

N8OHU

---

From: Kristoff Bonne <kris...@skypro.be>
To: digita...@googlegroups.com

Sent: Monday, June 17, 2013 4:42 PM

[Kristoff Bonne]

Tony,


I have already been looking in the source-code of tlb to troubleshoot in issue I had on the pi.

In theory, it should not be to difficult.

The sound code now just has a PCM-stream that it fetches from the core of the application and it sends it to the OSS device. What need to be done is just call codec2 encoding and c2gmsk modulation in front of that (and the other way around on the receiving side), and that should be it.


However, I have been thinking about rewriting the audio-layer of tlb. Especially the OSS layer (or the oss-alsa compability) seams to be an issue on the pi. When switching from receiving to sending (i.e. when tlb needs to start transmitting on its radio interface), the OSS device sometimes gets stuck for a couple of seconds. Althou audio itself isn't lost (it's buffered by the application) I do fear that this will give issues when transmitting a gmsk stream.
A gmsk stream may not be interrupted, not even for a single frame. The requirements for the audio-layer when doing gmsk are much more strict then for analog audio.


Perhaps an issue would be to convert that part of the code to use portaudio and use the "callback" methode. That way, the audio-layer runs in its own thread and has its own timing. But I see no reason why it cannot be done.



However, for me, this is more down on my to-do list. My first priority is the modem itself.


73
kristoff - ON1ARF

--

[Tony Langdon]

On 18/06/13 4:54 PM, Kristoff Bonne wrote:

Tony,


I have already been looking in the source-code of tlb to troubleshoot in issue I had on the pi.

In theory, it should not be to difficult.

The sound code now just has a PCM-stream that it fetches from the core of the application and it sends it to the OSS device. What need to be done is just call codec2 encoding and c2gmsk modulation in front of that (and the other way around on the receiving side), and that should be it.

There's all sorts of possibilities, once you have Codec2 in the core of tlb.  You have access to Echolink, etc.  Now, is it best to have the modem as part of the tlb binary? (one binary does all) or is it better to have a separate modem application or module that talks over a socket?  The latter approach allows you to mix and match different modems - VHF to HF gateway anyone (i.e. GMSK to FDMDV modems), all hooked up to an Echolink conference? :)  Just thinking out loud as to the wild ideas possible.

[Kristoff Bonne]

Matthew,


Well, the way I understand allstar (Ed, please correct me if I am wrong) is that all traffic over allstar always gets converted to PCM. It goes like this: (e.g.)

D-STAR -> PCM -> allstar -> PCM -> echolink


If you have a conversion from (say) echolink to echolink, you should not get over allstar (or else you would have a unnecessairy double-conversion) and the conversion should stay completely on the "echolink" side.

I agree this works great when you deal with voip-systems like echolink or D-STAR which have their own routing-protocol. However, for codec2, we do not have this.
We have two radio-layer services (fdmdv2/freedv and c2gmsk) but no routing-service or conferencing-service.


Do note that I have written "c2enc" to capsulate codec2 over udp (like between the voice input/output application and the modem) but we do not have a conferencing-system that can natively switch / duplicate codec2 frames between one or multiple users.
This would require somebody to write something simular to tlb, but that works internally with codec2 frames instead of PCM.

[Matthew Pitts]

Good morning Kristoff, AllStar is basically Asterisk and uses the IAX2 (Inter-Asterisk eXchange) protocol for communication between AllStar Link systems. This means it can use any voice codec that is supported by Asterisk, as long as both ends of the chain support it. The main thing is that it is based on an older version of Asterisk that does not have some of the changes to the RTP code that exist in Asterisk 1.8 (which is what the existing Asterisk patch for Codec2 is for) so that patch won't work. To handle conversion from AllStar to D-Star, the current D-Star channel driver hands the audio stream to a DV-Dongle and receives audio from the dongle going into AllStar.

Matthew Pitts N8OHU Sent from Yahoo! Mail on Android

---

From: Kristoff Bonne <kris...@skypro.be>;
To: <digita...@googlegroups.com>;
Subject: Re: [digitalvoice] Re: layering the 2400 Baud modem

Sent: Tue, Jun 18, 2013 7:09:29 AM

[Steve]

Going a little bit off topic, I've been looking at my interface to the codec2 and find I want to just communicate in bytes. The conversion to shorts for audio being superfluous.

For example, I receive two bytes from the soundcard interface, combine these into shorts, send to codec2 (which does all its work in floats, but I digress further), and then, of course, receive a short from the codec2 that must be split into two bytes to send back to the soundcard interface.

I don't see where the short conversions are adding value.

[Kristoff Bonne]

Hi Matthew,


I may be wrong but for some reason, I get the impression that what is called "allstar" is actually two different systems:

- you talk of allstar as asterisk, so this is actually a VOIP switch. A system to do "one to one" calls over IP (with the addition that the end points are not POTS or ISDN lines but radio interfaces).

So this would mean that you follow the normal VoIP way of working: if you walk to talk to a remote line, you make a SIP connection, there is a capabilities-exchange and -once a common codec is agreed- you use RTP to exchange voip packets.

So,
- Case 1: endpoint "A" is a c2gmsk channel driver (+URI + radio), endpoint "B" is D-STAR channel driver.
Endpoint A says "I speak codec2 and PCM uLaw", endpoint B says "I speak AMBE and PCM uLaw".
The only common codec is PCM uLaw, so it is agreed that the conversation will be done using that codec. Both sides then have the responsability to do the conversion (PCM-codec2 or PCM-AMBE) and that's it.

- Case 2: endpoint "A" is a c2gmsk radio (channel driver), endpoint "B" is a FreeDV/FMDMV channel-driver.
Again, endpoint A says "I speak codec2 and PCM uLaw". However, here endpoint B also says "I speak codec2 and PCM uLaw".
Here, the best common codec is codec2, so the conversation is done using codec2 without any conversation.

Am I correct?



However, this only applies for communication between two endpoints.
What we where talking about is not a one-to-one conversation. It's a conference service. It is a system where multiple devices are all connected in one big room: clients speak all different kinds of codecs. However, the one and only common codec is PCM.

(my guess is that -for asterisk- the conference-server is just an endpoint like another. It is probably configured that is only supports one codec (PCM uLaw) so that any incoming connection to the conference is force to use PCM uLaw.

Am I correct?



73
kristoff - ON1ARF

[John D. Hays]

Kristoff,

I think that's a pretty reasonable set of assumptions.  Allstar does run on an older version of Asterisk.

---

John D. Hays
K7VE

PO Box 1223, Edmonds, WA 98020-1223 

  

[Bruce Perens]

On 6/18/2013 6:42 AM, Steve wrote:
> For example, I receive two bytes from the soundcard interface, combine
> these into shorts, send to codec2 (which does all its work in floats,
> but I digress further), and then, of course, receive a short from the
> codec2 that must be split into two bytes to send back to the soundcard
> interface.
>
> I don't see where the short conversions are adding value.

They definitely add value when the sound card data is more than 8 bits
wide. If we do any manipulation of that data before and after the codec,
short is the natural value.

Thanks

Bruce

[Kristoff Bonne]

Hi John,


OK, if this is correct, then
- this means that if you are on a conference, everything will always first get transcoded to PCM (the common codec for everybody). So, even if you are on (say) echolink and are listening to somebody else on echolink, the audio will have been transcoded twice (echolink->PCM and then PCM->echolink)


- However, more interestingly, it would allow us to set up a conference where the "native codec" is codec2 and let everybody who uses codec2 (c2gmsk or freedv) connect to that conference. It would allow people to communicate using codec2 without this double translation, no matter if he or she is using a PC-client, a c2gmsk radio of a FreeDV radio.

It would therefor not be necessairy to create a dedicated codec2 "reflector-system".





73
kristoff - ON1ARF

[Steve]

Obviously you'd need to send byte pairs, re-assembled/dis-assembled inside the DLL. The analyse_one_frame() function converts the short array into floats, so it wouldn't be onerous to re-assemble the bytes there. This is more along the lines of soundcards and PC-think though.

In firmware I've decided to just go from A/D straight to float, and then tack the new 80 samples onto the end of the analysis memory (Sn) using DMA (4 bytes * 80 samples). So DMA will move the last data left 80 floats, and then DMA will input 80 new floats to the end before hitting the "analyze" button...

regards...

[Tony Langdon]

On 19/06/13 8:50 AM, Kristoff Bonne wrote:

Hi John,


OK, if this is correct, then
- this means that if you are on a conference, everything will always first get transcoded to PCM (the common codec for everybody). So, even if you are on (say) echolink and are listening to somebody else on echolink, the audio will have been transcoded twice (echolink->PCM and then PCM->echolink)


- However, more interestingly, it would allow us to set up a conference where the "native codec" is codec2 and let everybody who uses codec2 (c2gmsk or freedv) connect to that conference. It would allow people to communicate using codec2 without this double translation, no matter if he or she is using a PC-client, a c2gmsk radio of a FreeDV radio.

It would therefor not be necessairy to create a dedicated codec2 "reflector-system".

tlb/tbd allow for the case where you can have half duplex communication over a conference without transcoding - so if all connections are Codec2, you can configure for no transcoding.  tbd can ONLY do the no transcoding case (since it can't handle PCM at all - afterall, it was developed as a conference server - tlb has the enhancements to allow it to be a repeater controller, which means being able to handle PCM and encode/decode the various codecs.

tlb/tbd can also transport text using the Echolink protocol, but so far, no one has implemented text in the non Echolink transports.

[Matthew Pitts]

Tony, AllStar generally uses the highest common codec available to the connected nodes, since it supports multiple codecs. In the case of my system, I have 4 nodes connected in a hub and spoke configuration; one node is a dedicated hub and handles any transcoding that might be needed within my system. One node is a dedicated link to my D-Star reflector, one is my Echolink node and the fourth node is for AllStar connections. If we add Codec2 as a native codec, it becomes a transport option if available, and if a node connects that doesn't understand it, that connection uses something it does. Kristoff, AllStar uses the base capabilities of Asterisk, modified by app_rpt, which functions as a repeater controller; I can give you more details tomorrow on that.

Matthew Pitts N8OHU Sent from Yahoo! Mail on Android

---

From: Tony Langdon <vk3...@gmail.com>;
To: <digita...@googlegroups.com>;
Subject: Re: [digitalvoice] codec2 and voip conference-services
Sent: Wed, Jun 19, 2013 1:29:56 AM

[Tony Langdon]

On 19/06/13 12:08 PM, Matthew Pitts wrote:

Tony, AllStar generally uses the highest common codec available to the connected nodes, since it supports multiple codecs. In the case of my system, I have 4 nodes connected in a hub and spoke configuration; one node is a dedicated hub and handles any transcoding that might be needed within my system. One node is a dedicated link to my D-Star reflector, one is my Echolink node and the fourth node is for AllStar connections. If we add Codec2 as a native codec, it becomes a transport option if available, and if a node connects that doesn't understand it, that connection uses something it does.

OK, sounds good to me.  I think I'm getting close to the design for my new system.  It's going to have at least 2 Linux systems - one will run Asterisk, one will run tlb, both on low powered embedded systems.  There will be some form of network file storage (just a small amount), and a link using chan_tlb between them.  Lockouts will be implemented, so that IRLP, Echolink and AllStar can't be cross connected.  The repeater will most likely be connected to tlb, because (1) I know the platform better, (2) it has the functionality I need, and (3) a more "natural" dialing scheme for IRLP and Echolink (no leading zeros, just a basic prefix to isolate the networks at most).  I also already have a script that can parse tlb's DTMF decoding for translation into other formats.

The processors will be embedded (might even sneak a R-Pi in there somewhere!) to save on power consumption.  I already have most of the hardware here.

The feasibility of doing this has already been researched on the app_rpt mailing list, as far as the scripting side goes.  Just have to see if I have the grunt to manage the codecs on the processors I'm likely to be using.

As for Codec2, it would be most convenient on tlb, but happy to use it on AllStar, once I have something compatible up.  I already have 2 configurations - one is a node, one is a hub (which I tried running for a while, but it kept messing up the VPS.  Might have to try it on its own server instance).  A tlb based inplementation of codec2 could be made immediately available on the 9555 channel of my reflector, once it's stable and bug free enough for beta testing - Codec2 linked to IRLP and Echolink. :)