Pseudo-binary parsing

[Gerhard Reithofer]

Hi TCLers,
I'm trying to write a program which manages 3d-printer settings ind a
database, there are 1000s of combinations as one profile has ca. 150
values defined.
The settings are used by the manufacturers slicing software which is
unfortunately not open source.

Looking at the stored data it is a typical ascii representation which is
a good idea to pass these files to different platforms.
The format is the typical "ini" file with "parameter=value" like
brimExtruder=2
brimEnable=true
heightTempsR="3,170;8,200;"

There are obviously strings ("PLA std 210-135"), integers (also as lists
"-1,-1") and the booleans "true" and "false".

But one type is coded in format:
infillCrossAngle=@Variant(\0\0\0\x87\x42\xb4\0\0)
The "@Variant" is constant, there are always 3 zero bytes at the
beginning.
After that always follows the byte "\x87"
and the last 4 bytes are the corr. value I think.

I have identified a few values:
byte pattern => supposed value
-----------------------------------------
\0\0\0\x87>\x8a=q => 0.27mm
\0\0\0\x87\x42\xa0\0\ => 70%
\0\0\0\x87\x42\xda\0\0 => 109%
\0\0\0\x87\x41 \0\0 => 10mm/s
\0\0\0\x87\x43H\0\0 => 200°
\0\0\0\x87\x42H\0\0 => 50°
\0\0\0\x87=\xcc\xcc\xcd => 0.1mm
\0\0\0\x87\x42\xc8\0\0 => 100
\0\0\0\x87\x42 \0\0 => 40
\0\0\0\x87\x42H\0\0 => 50mm^2
\0\0\0\x87@@\0\0 => 3.0mm
\0\0\0\x87@ \0\0 => 3.5mm

I did add the dimensions of the values as it may be that is some machine
specific constant included but I think it is not (see the 50 above)

My question is if someone can tell me how to convert the "Variant"
value to a corresponding mathematical numeric value?

TIA,
Gerhard

--
Gerhard Reithofer - Techn. EDV Reithofer - http://www.tech-edv.co.at

[js]

On 2/7/21 7:58 AM, Gerhard Reithofer wrote:
> Hi TCLers,
> I'm trying to write a program which manages 3d-printer settings ind a
> database, there are 1000s of combinations as one profile has ca. 150
> values defined.
> The settings are used by the manufacturers slicing software which is
> unfortunately not open source.
>

...

> \0\0\0\x87\x42\xc8\0\0 => 100
> \0\0\0\x87\x42 \0\0 => 40

> My question is if someone can tell me how to convert the "Variant"
> value to a corresponding mathematical numeric value?
>

I am not sure it is a direct binary representation of some sort. As you
mention, these values may be items in a limited set of acceptable
points. If so, the manufacturer may have simply enumerated (or encoded)
them for convenience. I often do this to avoid dealing with spaces,
weird characters or quoting during input/output. For example:

Berlin --> 1
London --> xyz
etc.

There is no correlation between 1 and xyz but they do translate to
Berlin and London within the application.

[Gerhard Reithofer]

On Sun, 7 Feb 2021, js wrote:

Hi js,

> On 2/7/21 7:58 AM, Gerhard Reithofer wrote:
> > Hi TCLers,

...

> ...
>
> > \0\0\0\x87\x42\xc8\0\0 => 100
> > \0\0\0\x87\x42 \0\0 => 40
>
>
> > My question is if someone can tell me how to convert the "Variant"
> > value to a corresponding mathematical numeric value?
> >
>
> I am not sure it is a direct binary representation of some sort. As you
> mention, these values may be items in a limited set of acceptable points. If
> so, the manufacturer may have simply enumerated (or encoded) them for
> convenience. I often do this to avoid dealing with spaces, weird characters or
> quoting during input/output. For example:
>
> Berlin --> 1
> London --> xyz
> etc.
>
> There is no correlation between 1 and xyz but they do translate to Berlin and
> London within the application.

that's unlikely.

If you look at my patterns there are with different dimensions but 2
equal numeric values.

byte pattern => supposed value
-----------------------------------------

\0\0\0\x87\x42H\0\0 => 50°

...

\0\0\0\x87\x42H\0\0 => 50mm^2

IMHO does this not work in your method as the binary patterns should be
unique (keys).
But I do just "assume" and not "know" ;-)

The complete lines are:
deceAreaThreshold=@Variant(\0\0\0\x87\x42H\0\0)
perimeterOuterSpeed=@Variant(\0\0\0\x87\x42H\0\0)

The 1st one a an area specification the 2nd is movement speed value.

Thank you very much,

[Christian Gollwitzer]

Am 08.02.21 um 12:24 schrieb Gerhard Reithofer:

> If you look at my patterns there are with different dimensions but 2
> equal numeric values.
>
> byte pattern => supposed value
> -----------------------------------------
> \0\0\0\x87\x42H\0\0 => 50°
> ...
> \0\0\0\x87\x42H\0\0 => 50mm^2

It is unclear to me, if you have literally this string

"\0\0\0\x87\x42H\0\0"

in your input, or if this is some representation which you or your
editor chose for a sequence of 8 bytes with the hex values

0x00 0x00 0x00 0x87 0x42 0x48 0x00 0x00

IOW, do you have backslashes in your input?

Christian

[js]

On 2/8/21 6:24 AM, Gerhard Reithofer wrote:
>
> The complete lines are:
> deceAreaThreshold=@Variant(\0\0\0\x87\x42H\0\0)
> perimeterOuterSpeed=@Variant(\0\0\0\x87\x42H\0\0)
>
> The 1st one a an area specification the 2nd is movement speed value.
>

Unless there is some formatting error in the post, the variant indexes
are identical; and according to you, they map to "50°" and "50mm^2". The
quantities are the same (50), but units are different (speed vs. area).
So in all likelihood this means that the unit is not encoded in the
variant but is assumed due to the hidden semantics behind "variable" it
is being assigned to (it looks like deceAreaThreshold goes with mm^2, etc.)

But if you look at the two examples I quoted in my first reply, there is
a space character in there. I have not seen any binary formats that
allow spaces in their encoding (which could just mean I just I don't
know those things well enough).

\0\0\0\x87\x42\xc8\0\0 => 100
\0\0\0\x87\x42 \0\0 => 40

In any case, I was just offering a different interpretation that seemed
just as likely as it being a binary parsing issue. It doesn't make sense
why the manufacturer would choose this encoding for certain numbers when
others are in plain text. (Would a careless out-of-range value destroy
the machine?) You could always simply ask them.

[Uwe Klein]

Am 08.02.21 um 15:21 schrieb Christian Gollwitzer:

> It is unclear to me, if you have literally this string
>
>     "\0\0\0\x87\x42H\0\0"
>

ascii printable with escaped unpritables IMU.

binary scan $string $format_spec variable
set variable

But I've tried S, s, I, i, c on it and I don't see the proper scaling
between values

using these:
\0\0\0\x87 >@@\0\0< => 3.0mm
\0\0\0\x87 >@ \0\0< => 3.5mm

I'd expect to be LSB first.

the resultant values should show a 6 : 7 relation
they don't.

Uwe

[Schelte]

On Sun, 07 Feb 2021 13:58:42 +0100, Gerhard Reithofer wrote:
> But one type is coded in format:
> infillCrossAngle=@Variant(\0\0\0\x87\x42\xb4\0\0)
> The "@Variant" is constant, there are always 3 zero bytes at the
> beginning.
> After that always follows the byte "\x87"
> and the last 4 bytes are the corr. value I think.
>
> I have identified a few values:
> byte pattern => supposed value
> -----------------------------------------
> \0\0\0\x87>\x8a=q => 0.27mm
> \0\0\0\x87\x42\xa0\0\ => 70%
> \0\0\0\x87\x42\xda\0\0 => 109%
> \0\0\0\x87\x41 \0\0 => 10mm/s
> \0\0\0\x87\x43H\0\0 => 200°
> \0\0\0\x87\x42H\0\0 => 50°
> \0\0\0\x87=\xcc\xcc\xcd => 0.1mm
> \0\0\0\x87\x42\xc8\0\0 => 100
> \0\0\0\x87\x42 \0\0 => 40
> \0\0\0\x87\x42H\0\0 => 50mm^2
> \0\0\0\x87@@\0\0 => 3.0mm
> \0\0\0\x87@ \0\0 => 3.5mm
>

Many of these look like big-endian single precision floating point values:
% binary scan \0\0\0\x87>\x8a=q x4R value;set value
0.27000001072883606
% binary scan \0\0\0\x87\x42\xda\0\0 x4R value;set value
109.0
% binary scan "\0\0\0\x87\x41 \0\0" x4R value;set value
10.0
% binary scan \0\0\0\x87\x43H\0\0 x4R value;set value
200.0
% binary scan \0\0\0\x87\x42H\0\0 x4R value;set value
50.0
% binary scan \0\0\0\x87=\xcc\xcc\xcd x4R value;set value
0.10000000149011612
% binary scan \0\0\0\x87\x42\xc8\0\0 x4R value;set value
100.0
% binary scan "\0\0\0\x87\x42 \0\0" x4R value;set value
40.0
% binary scan "\0\0\0\x87@@\0\0<" x4R value;set value
3.0

Except:
% binary scan "\0\0\0\x87\x42\xa0\0\0" x4R value;set value
80.0
% binary scan "\0\0\0\x87@ \0\0<" x4R value;set value
2.5
Which you claimed should be 70 and 3.5 respectively.


Schelte.

[Schelte]

Sorry for the mess my newsreader program made of that post. Let's try
that again. Otherwise I hope you can figure out where the newlines go.

[Gerhard Reithofer]

Hi Schelte,

On Tue, 9 Feb 2021, Schelte wrote:

> Sorry for the mess my newsreader program made of that post. Let's try
> that again. Otherwise I hope you can figure out where the newlines go.
>
> Many of these look like big-endian single precision floating point values:
> % binary scan \0\0\0\x87>\x8a=q x4R value;set value
> 0.27000001072883606

...

> Which you claimed should be 70 and 3.5 respectively.

yes, that's definitive the format :-) :-) :-) :-) :-)

Including your bugs which I "implemented" by looking at wrong lines.

I thought that it is floating point format therfore the bytes can't be
"calculated" due to the different internal representations.
But there are many versions, different conversions, sizes and formats.

You definitely saved much, much much time for me and my planned project.

Many thanks,