How to encode/decode string to binary in golang

[Van Fury]

Hi,

I have two hexadecimal string values and would like to concatenate the two strings and

1. encode the result to binary
2. decode the resulting binary back to hexadecimal string

I did the following but I was finding it difficult to decode the result back. I ignore error check in this case.

What i did so far:

s1 := "1d28ed66824aa2593e1f2a4cf740343f"

s2 := "dee2bd5dde763885944bc9d65419"

s3 := s1 + s2

s1s2Byte, _ := hex.DecodeString(s3)

randAutnBin := fmt.Sprintf("%b", s1s2Byte)

result:

[11101 101000 11101101 1100110 10000010 1001010 10100010 1011001 111110 11111 101010 1001100 11110111 1000000 110100 111111 11011110 11100010 10111101 1011101 11011110 1110110 111000 10000101 10010100 1001011 11001001 11010110 1010100 11001]

I would like to decode the binary result back the hexadecimal string to get s1 and s2.

Any help?

Van

[Kurtis Rader]

Perhaps I have misunderstood your question but doesn't hex.EncodeToString(s1s2Byte) do what you want?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ad3a981b-cf22-47cd-9fe6-8db83a097b42n%40googlegroups.com.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Alex Howarth]

If s1 and s2 are a fixed length then you can just slice up the decoded string based on the lengths. If they are of a variable length, you'll need a separator in the input string to later split on when decoded (s3 := s1 + ":" + s2 etc)?

--

[Van Fury]

Sorry I did not frame my question properly but what I would like to do is to

encode concatenated s1 and s2 into raw binary and then decode the raw binary

back to s1 and s2.

[Volker Dobler]

On Monday, 13 March 2023 at 12:24:15 UTC+1 Van Fury wrote:

Sorry I did not frame my question properly but what I would like to do is to

encode concatenated s1 and s2 into raw binary and then decode the raw binary

back to s1 and s2.

The only problem is knowing where s1 ends / where s2 starts.

So your "encoded raw binary" should be:

16 byte of len(s1) (e.g. in decimal), s1 and s2.

fmt.Print("%016d%s%s", len(s1), s1,s2) should do.

Decoding by reading 16 byte, Atoi'ing, reading s1 and the

rest is s2.

V. 

[Alex Howarth]

You might be looking for strconv.ParseUint() https://pkg.go.dev/strconv#ParseUint

https://go.dev/play/p/qAO9LfLD41D

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1f66ccdc-6c58-48f0-9017-9614220f88d9n%40googlegroups.com.

[Van Fury]

Do you mean encoding should be

rawdata := fmt.Print("%016d%s%s", len(s1), s1,s2)

or

rawdata := fmt.Printf("%016d%s%s", len(s1), s1,s2)

[Van Fury]

Relating to my previous question, I have been reading but it is still not clear to me what raw binary is, how is it different from

text formatted binary (fmt.Sprintf("%b", s1s2Byte))?

In my problem above I need to encode s1+s2 to raw binary before sending the result to the server

which then decodes the raw binary back to s1+s2.

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/wCGYXo-r-uo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8c1be24c-8056-4b74-ac79-e1f8ae7bf935n%40googlegroups.com.

[Kurtis Rader]

On Mon, Mar 13, 2023 at 9:41 AM Van Fury <fury...@gmail.com> wrote:

Relating to my previous question, I have been reading but it is still not clear to me what raw binary is, how is it different from

text formatted binary (fmt.Sprintf("%b", s1s2Byte))?

In my problem above I need to encode s1+s2 to raw binary before sending the result to the server

which then decodes the raw binary back to s1+s2.

At this point I think we are looking at an XY problem. Your s1 and s2 string variables are sequences of raw bytes; i.e., "raw binary". It would probably help us help you if you more fully described the problem rather than focusing on your solution. Especially since your solution doesn't work. In fact, your solution can't work without imposing more structure on the encoding to make it possible for the recipient to know where s1 ends and s2 begins. 

[Bruno Albuquerque]

I wonder if what is needed here is just null-terminated strings.

-Bruno

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD_oDA%3DEB4My7NJ0W_%3D0UtTTJeDeGqRotiK%3DSTKTRv4Cxw%40mail.gmail.com.

[Volker Dobler]

On Monday, 13 March 2023 at 17:41:42 UTC+1 Van Fury wrote:

Relating to my previous question, I have been reading but it is still not clear to me what raw binary is, how is it different from

text formatted binary (fmt.Sprintf("%b", s1s2Byte))?

"text formated binary" takes a stream of bytes  and for each bit in

this stream output "0" or "1" as letters, i.e. the bytes 48==0x30

and 49==0x31. An eightfold blowup in size.

Try understanding how _text_ is encoded as bytes.

 

Computers work on streams of bytes.

The meaning/interpretation of a single byte can vary:

Not all number can be stored in a single byte and

text has to be encoded by bytes too.

Please forget about "raw binary", there is no such thing.

String variables are a stream of bytes and do not need any encoding,

especially not something like "raw binary" that doesn't exist.

A stream of bytes can be _printed_ (encoded) as decimal, hexadecimal,

octal, base64 or even as bit ('0' and '1'). sometimes this is necessary

because the underlying protocol cannot transmit possible byte values.

When dealing with "binary" streams this cannot happen (it happens e.g.

when using JSON to transmit data).

You have to make clear what the output/encoding space should be

(and why!): There is a inconsistency between wanting "binary" and

a textual bit stream (which is not "binary" but text) and hexadecimal

encoding (also not binary but text).

You manoeuvred yourself in a corner by focusing on the solution

instead of the actual problem.

V.

[robert engels]

Am arbitrary byte can be encoded in 2 HEX characters - only a 2x increase in size not 8x.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5cd8a906-5b73-4b86-abb1-8106fd9f73cdn%40googlegroups.com.

[Marcello H]

You can also use gob for that.

Here are 2 functions from my library that can help.

import "encoding/gob"

/*

encodeGob encodes a model to gob bytes.

*/

func encodeGob(data any) ([]byte, error) {

var (

buf bytes.Buffer

enc = gob.NewEncoder(&buf) // Will write to network.

)

err := enc.Encode(data)

return buf.Bytes(), errors.Wrap(err, "encodeGob")

}

/*

decodeGob decodes gob bytes to a model.

*/

func decodeGob(decVal []byte, data any) (err error) {

buf := bytes.NewBuffer(decVal)

dec := gob.NewDecoder(buf)

err = dec.Decode(data)

return errors.Wrap(err, "decodeGob")

}

Op dinsdag 14 maart 2023 om 00:06:31 UTC+1 schreef robert engels:

[Van Fury]

Am developing a server (diameter) which will response with an AVP SIP-Authenticate. In the specification

" The SIP-Authenticate AVP is of type OctetString and It shall contain, binary encoded, the concatenation of the authentication challenge RAND and the token AUTN" 

The RAND and the AUTN in this case are hexadecimal strings s1 and s2.

My problem is how to encode s1 and s2 concatenated and then be able to decode at the server side too.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3bf5ccfd-e007-4f8c-8d1e-a6e76800c403n%40googlegroups.com.

[Brian Candler]

On Tuesday, 14 March 2023 at 09:26:05 UTC Van Fury wrote:

Am developing a server (diameter) which will response with an AVP SIP-Authenticate. In the specification

" The SIP-Authenticate AVP is of type OctetString and It shall contain, binary encoded, the concatenation of the authentication challenge RAND and the token AUTN" 

I believe "binary encoded" here just means []byte.  Concatenate them using append().

 

The RAND and the AUTN in this case are hexadecimal strings s1 and s2.

I don't believe either is a "hexadecimal string" in the sense of being encoded using the ASCII symbols "0" to "9" and "a" to "f".  They may conventionally be *displayed* in this format, e.g. for debugging purposes, but internally and on-the-wire I think they are just sequences of bytes.

NOTE: my experience is with RADIUS rather than Diameter, so I *could* be wrong.  I'm just outlining what I *expect* to be the case; I haven't gone reading through Diameter RFCs.

[Brian Candler]

Presumably you are looking at something like this: https://www.etsi.org/deliver/etsi_ts/129200_129299/129229/13.01.00_60/ts_129229v130100p.pdf

6.3.10 SIP-Authenticate AVP

The SIP-Authenticate AVP is of type OctetString and contains specific parts of the data portion of the WWW-Authenticate or Proxy-Authenticate SIP headers that are to be present in a SIP response.
It shall contain, binary encoded, the concatenation of the authentication challenge RAND and the token AUTN. See
3GPP TS 33.203 [3] for further details about RAND and AUTN. The Authentication Information has a fixed length of
32 octets; the 16 most significant octets shall contain the RAND, the 16 least significant octets shall contain the AUTN.

I believe what it's saying is: in the [textual] SIP WWW-Authenticate header, there are some fields RAND and AUTN which are encoded in some textual form (hex? base64? Check the details of the auth scheme)

You need to convert those to their underlying binary data, giving you two [16]byte blobs, then concatenate them to give 32 bytes of data.

If so, you had roughly the right idea in the first place (except s2 was less than 16 bytes).  You could concatenate the hex representations and then decode, as you did; or I would be inclined to decode them separately:

https://go.dev/play/p/851IrsExn76

The result is [32]byte, which is 16 bytes from one field and 16 bytes from the other.  You just use it as-is for the AVP value; this is the "binary encoded" value.  It doesn't mean you should turn it into a string of ASCII "1" and "0" characters.

[Jesper Louis Andersen]

On Tue, Mar 14, 2023 at 10:25 AM Van Fury <fury...@gmail.com> wrote:

Am developing a server (diameter) which will response with an AVP SIP-Authenticate. In the specification

" The SIP-Authenticate AVP is of type OctetString and It shall contain, binary encoded, the concatenation of the authentication challenge RAND and the token AUTN" 

The RAND and the AUTN in this case are hexadecimal strings s1 and s2.

My problem is how to encode s1 and s2 concatenated and then be able to decode at the server side too.

You probably want to start thinking about representations. Your application has some internal representation of the (128-bit long) RAND and AUTN token. And there's a representation on the "wire" in the protocol. The protocol representation is decided for you, but you can pick any you'd like for the internal representation. OctetString is just a synonym for []byte in this case. And the output is going to be at length 32. You have a function pair which encodes/decodes from your internal representation to the protocol (and back).

Were it me, I'd represent the internal RAND and AUTN as [16]byte data. Then, the encoder/decoder is just the identity function in this case. If you need a hexadecimal representation elsewhere in the application, you'd hex-encode when you need it. It sounds to me you have chosen a hexadecimal encoding as your internal representation, but I'd advise against it. You end up having to do far more work getting into the right form all over in your app.

In any case, the advantage of thinking about internal/protocol representation is that the conversion points between the formats become explicit and controlled in the application. This makes it far easier to evolve the application internally without having to worry about the wire format. Both the wire format and the application can evolve independently of each other.

--

J.

[Van Fury]

Thank you all for the clarification. At least I now know in this case octetString is referring to []byte.

I did the conversion to []byte and the radius server accepted the result.

I was just confused when the one handling the radius server asked for "raw binary" in the response.

Help appreciated!!