Query: RANGE Operator. Does it only work in base-10?

[TiddlyTweeter]

The range operator is a god-send for automation.

Yesterday TonyM assembled Unicode charts easily using it (https://anthonymuscio.github.io/PreReleaseGlyphs.html -- Takes time to load, it is an experiment).

I was very impressed by it. The only downside is the ranges he had to use are decimal. But Unicode is best thought in hexadecimal. Mixing the two bases in presentation is not ideal.

So. My query is whether there might be a way to feed hex numbers (convert to decimal first?) to Range so we could at least feed in hex even if the number crunching is digital?

Any thoughts appreciated!

Best wishes

TT

[TW Tones]

TT,

I found a decimal source, but a simple plugin could convert between the two, in fact it could even be written in wikitext.

However I believe Evans formulae plugin may have the facility, it would be nice to have it just for dealing with entities and other Hex use cases. With 16 values per character its can also be a way to marginally compress numeric strings such as serial numbers.

Regards

Tones/TonyM

[Mark S.]

Just for fun. Convert 0-255 to Hex.

<$vars hexlist="0 1 2 3 4 5 6 7 8 9 A B C D E F">

<$list filter="[range[31,40]]" variable=dec>

<$list filter="[<dec>divide[16]trunc[]add[1]]" variable="byte">

<$list filter="[<dec>remainder[16]add[1]]" variable="bit">

<$list filter="[enlist<hexlist>nth<byte>]=[enlist<hexlist>nth<bit>]+[join[]]" variable="hex1">

<<dec>> <<hex1>> <br/>

</$list></$list></$list></$list>

</$vars>

[TiddlyTweeter]

Ciao Tony

Mark S.' reply indicates the hex-dec translate numbers might be derivable without Evan in vanilla TW.

My main point was that dealing with Unicode code points you really need to be a hexadecimal :-). It is complicated in multiple ways. A major one being that UTF-16 breaks once beyond BMP. It becomes vital to have one standard numeric hook to hang on to navigate consistently. 

I think your tool is brilliant but entering decimal ranges manually is an invitation for cock-ups. 

Can we move towards auto convert to decimal-Ranges from Kosher Unicode code-point ranges you do provide?

Best wishes

TT

[TiddlyTweeter]

Ciao Mark S. 

Mark S. wrote:

Just for fun. Convert 0-255 to Hex.

<$vars hexlist="0 1 2 3 4 5 6 7 8 9 A B C D E F">

<$list filter="[range[31,40]]" variable=dec> ...

Thanks. That is useful! I just realised both TonyM & I have been round these hoops before with your guidance!

Actually you solved a very difficult issue for me in Base-27 ... in https://groups.google.com/g/tiddlywiki/c/mBm0nW95NzE/m/KNOcQR-eCQAJ

[TW Tones]

Mark,

On this occasion I need to convert a 4 digit Hex to a decimal, so that I can use it in the range widget, because we cant generate a range of hex numbers.

Hence this issue https://github.com/Jermolene/TiddlyWiki5/issues/5177

I think I can do it but if you know how please share.

Tones 

[Mark S.]

Here's my first attempt. It uses the 5.1.23 search-replace filter operator. It requires the input to be padded to 4 bytes.

Now that I've done it, I realize it could probably be written with a recursive loop and be open ended (not needing to be padded).

\define hex2dec2(hex)

<$vars myhex=<<__hex__>> myfilter="[split[]nth<place>search-replace:g[A],[10]search-replace:g[B],[11]search-replace:g[C],[12]search-replace:g[D],[13]search-replace:g[E],[14]search-replace:g[F],[15]]">

<$list filter="1 2 3 4" variable="place"><<place>>

<$list filter="[<myhex>subfilter<myfilter>] [<place>compare:number:eq[1]then[4096]else[1]] [<myhex>subfilter<myfilter>] [<place>compare:number:eq[2]then[256]else[1]] [<myhex>subfilter<myfilter>] [<place>compare:number:eq[3]then[16]else[1]]+[product[]]"/>

</$list>

</$vars>

\end

\define hex2dec(hex)

<$wikify text="<<hex2dec2 $hex$>>" name=dec>

<$list filter="[enlist<dec>sum[]]"/>

</$wikify>

\end

<<hex2dec EA0E>>

[coda coder]

Mark - you're adding 10 (decimal) somewhere.

FFFF -> 65545

0000 -> 10

[Saq Imtiaz]

If it helps there is a pad[] operator in the pre-release.

[coda coder]

Mark... I see it. You left some debugging in there: <<place>>

[coda coder]

10 is the sum of 1 2 3 4

[Mark S.]

Ok, cleaned it up a bit:

\define hex2dec2(hex)

<$vars myhex=<<__hex__>> myfilter="[split[]nth<place>search-replace:g[A],[10]search-replace:g[B],[11]search-replace:g[C],[12]search-replace:g[D],[13]search-replace:g[E],[14]search-replace:g[F],[15]]">

<$list filter="1 2 3 4" variable="place">

<$list filter="

[<myhex>subfilter<myfilter>] 

[<place>compare:number:eq[1]then[4096]else[1]]  [<place>compare:number:eq[2]then[256]else[1]]  [<place>compare:number:eq[3]then[16]else[1]]

[<place>compare:number:eq[4]then[1]else[1]]

+[product[]]

"/>

</$list>

</$vars>

\end

\define hex2dec(hex)

<$wikify text="<<hex2dec2 $hex$>>" name=dec>

<$list filter="[enlist<dec>sum[]]"/>

</$wikify>

\end

<<hex2dec FFFF>>

[TW Tones]

Mark,

Good start. I was thinking how to get 16^0 16^1 16^2 and 16^3 and you have hard coded it. If moving to a recursive process we may be able to handle variable length hex.

But it is a little mind bending.

Tones

[Mark S.]

Here's a recursive version that that can be any length of hex. Lightly tested. Hopefully it's working OK.

\define hex2dec2(byte,mult:1)

<$vars myfilter="[search-replace:g[A],[10]search-replace:g[B],[11]search-replace:g[C],[12]search-replace:g[D],[13]search-replace:g[E],[14]search-replace:g[F],[15]]">

<$list filter="

[<__byte__>subfilter<myfilter>] 

=[<__mult__>]

+[product[]]

"/>

</$vars>

\end

\define hex2dec-r(hex,mult)

<$list filter="[<__mult__>] =16 +[product[]]" variable="mult">

<$list filter="[<__hex__>split[]nth[1]]" variable="byte">

<$list filter="[<__hex__>split[]rest[]join[]]" variable="hex"

emptyMessage="<$macrocall $name=hex2dec2 byte=<<__hex__>> mult=<<__mult__>> />"

>

<$macrocall $name=hex2dec2 byte=<<byte>> mult=<<__mult__>> />

<$macrocall $name="hex2dec-r" hex=<<hex>> mult=<<mult>> />

</$list></$list></$list>

\end

\define hex2dec-setup(hex)

<$list filter="[<__hex__>split[]reverse[]join[]]" variable=rhex>

<$macrocall $name=hex2dec-r hex=<<rhex>> mult=1 />

</$list>

\end

\define hex2dec(hex)

<$wikify text="<<hex2dec-setup $hex$>>" name=dec>

<$list filter="[enlist<dec>sum[]]"/>

</$wikify>

\end

<<hex2dec F111>>

On Wednesday, December 2, 2020 at 5:42:55 PM UTC-8 TW Tones wrote:

Mark,

[TW Tones]

Testing,

Nice work, It takes a bit to follow the code, but I am sure I will learn something, thanks so much.

You may see it in a Unicode database soon.

Tones

[TW Tones]

Mark,

I have followed your code through, More testing but it seems to all make sense to me now, It would be great if we could turn such code into a hierarchy chart that also recognised the list re-iterations. The information is all there so automation would be possible. In this case I can use the decimals in the range operator to satisfy the OT.

Thank you

Regards

Tones

[TW Tones]

Mark et all,

I think the new reduce operator may simplify this with each byte being converted at a time, making use of the accumulator and index.

Regards

'Tones

[Mark S.]

I was about to explain that the reduce operator wouldn't help because we don't have a power operator when I thought to check ... and we do!

So, great idea! And now we can forget recursive! 

<$vars myhex="FFFF">

<$vars myfilter="[[16]power<index>]=[<currentTiddler>search-replace:g[A],[10]search-replace:g[B],[11]search-replace:g[C],[12]search-replace:g[D],[13]search-replace:g[E],[14]search-replace:g[F],[15]]+[product[]add<accumulator>]" >

<$list filter="[<myhex>split[]reverse[]reduce<myfilter>]" />

</$vars>

</$vars>

[TW Tones]

Mark,

I thought you would do this so well, and yes the power operator, I was about to ask for it.

Attached I have turned it into a macro that has different ways to pass the values and examples, needs pre-release 5.1.23.

I hope I am not too heavy "standing on your shoulders"?

Tones

[Mark S.]

That's great. I'm glad you can make use of it.