Simple Eiffel - TOON

[Liberty Lover]

Hi All,

There is a new encoder/decoder in town for Eiffel specifically targeting prompt structure being sent to LLM's. The format is called TOON.

You are read all about it here.

Best,

Larry

[Liberty Lover]

And here is the research behind it.

Best,

Larry

--
You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com.

[Ulrich Windl]

Hi,

names again: Your "encode" feature can only convert a JSON value, not a generic value. Shouldn't it be "encode_json"instead?
Likewise shouldn't there be features to encode all TOON types directly?
Something like "encode_string(name: STRING, value: STRING)", etc.

Despite of that I'm missing a formal specification of the format, maybe some realistic, more complex, example, too.

Ulrich

15.12.2025 17:02:05 Liberty Lover <rix....@gmail.com>:

> Hi All,
>
> There is a new encoder/decoder in town for Eiffel specifically targeting prompt structure being sent to LLM's. The format is called TOON.
>

> You are read all about it here[https://github.com/simple-eiffel/simple_toon].

>
> Best,
>
> Larry
>
> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com[https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer].

[Ulrich Windl]

Hi!

Things I see problematic are:
Converting a number to a string if out of range
Accepting values that are not valid encoding outputs
Converting infinity to null would break the rule that decoding after encoding must not change the value.

Also, with the Eiffel hat on, JSON doesn't care about types, while Eiffel does. Wouldn't it make sense to allow an optional type name?

Also I failed to get the difference between tabular ARRAYs and numbered lists.

Ulrich

15.12.2025 17:12:24 Liberty Lover <rix....@gmail.com>:

> And here [https://github.com/simple-eiffel/claude_eiffel_op_docs/blob/main/research/SIMPLE_TOON_RESEARCH.md]is the research behind it.

>
> Best,
>
> Larry
>
> On Mon, Dec 15, 2025 at 11:02 AM Liberty Lover <rix....@gmail.com> wrote:
>> Hi All,
>>
>> There is a new encoder/decoder in town for Eiffel specifically targeting prompt structure being sent to LLM's. The format is called TOON.
>>

>> You are read all about it here[https://github.com/simple-eiffel/simple_toon].

>>
>> Best,
>>
>> Larry
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com[https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer].

>
> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com?utm_medium=email&utm_source=footer].

[Liberty Lover]

  Hi Ulrich,

  More good points:

  1. Numbers out of range / Infinity → null

  You're correct. JSON has no infinity, NaN, or arbitrary precision. The simple_json library:
  - Uses simple_decimal for exact decimal representation (no IEEE floating-point artifacts)
  - Throws an error on infinity/NaN rather than silently converting to null
  - This preserves the round-trip guarantee

  2. Accepting invalid encoding outputs

  In strict mode (set_strict(True)), the decoder validates:
  - Array counts match declared counts
  - Delimiter consistency
  - Proper indentation

  Non-strict mode is lenient for hand-written TOON.

  3. Optional type names (Eiffel perspective)

  Interesting idea. TOON is designed for LLM consumption where types are implicit. However, we could add an optional type annotation mode:

  name: Alice           -- Current (implicit STRING)
  name<STRING>: Alice   -- With type hint (proposed)
  age<INTEGER>: 30

  This would help LLMs generate type-safe code. Added to roadmap.

  4. Tabular arrays vs numbered lists

  - Numbered list: items[3]: apple,banana,cherry — simple scalar array
  - Tabular array: items[2]{sku,qty}: with rows below — array of uniform objects with named columns

  The tabular format compresses repeated key names into a single header, saving ~50% tokens for object arrays.

  -- Numbered list (scalars):
  tags[3]: red,green,blue

  -- Tabular array (objects with columns):
  products[2]{name,price}:
    Widget,9.99
    Gadget,19.99

  Larry

To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/f66abc27-4cde-4201-80e0-7b88f46d7a6d%40gmail.com.

[Liberty Lover]

  Hi Ulrich,

  Good observations! Let me address your points:

  1. Naming of encode

  You're right that encode operates on SIMPLE_JSON_VALUE. However, we already provide semantic aliases:

  encode,
  to_toon,
  serialize,
  compress_for_llm (a_json: SIMPLE_JSON_VALUE): STRING_32

  And for string input:
  encode_string,
  json_to_toon (a_json_text: STRING_32): detachable STRING_32

  So to_toon and json_to_toon are already available if you prefer explicit naming. The rationale: TOON's primary purpose is JSON→TOON conversion for LLM token optimization, so JSON is the expected input.

  2. Direct TOON type builders

  This is a valid feature request. Currently, you must build JSON first, then convert. A direct TOON_BUILDER would be useful:

  -- Proposed future API:
  create builder.make
  builder.add_string ("name", "Alice")
  builder.add_integer ("age", 30)
  builder.add_array ("items")
      .row ("A1", 10)
      .row ("B2", 20)
  builder.to_string

  I'll add this to the roadmap. For now, use simple_json to build the structure, then toon.encode.

  3. Formal specification

  TOON has an external specification. See the EIS links in the class header:
  - Spec: https://github.com/toon-format/spec
  - Format: https://toonformat.dev/

  4. Complex examples

  Fair point. The README shows basic examples. I'll add a cookbook page with realistic scenarios (nested structures, mixed arrays, edge cases).

  Thanks for the feedback—direct builder API is a good idea.

  Larry

To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/1914fdf0-eb00-415d-86a2-072430af08e0%40gmail.com.

[Liberty Lover]

Ulrich,

Here's the summary of what was delivered in response to your feedback:

  simple_toon v1.1.0 - Pushed to GitHub

  New Features:
  1. TOON_BUILDER (src/core/toon_builder.e) - 533 lines
    - Fluent API for building TOON directly without JSON intermediary
    - Addresses your request for encode_string(name, value) style construction
    - All scalar types, tabular arrays, nested objects, simple arrays
    - Configuration: indent, delimiter (comma/tab/pipe), reset
  2. Cookbook (docs/cookbook.html) - 356 lines
    - Format guide explaining numbered lists vs tabular arrays (your direct question)
    - TOON_BUILDER API examples with code
    - Complex real-world examples: e-commerce orders, API responses, config files
    - Edge cases: escaping, null/boolean/number types, empty structures
    - Best practices: when to use TOON vs JSON
  3. Tests - 10 new builder tests (35 total, all passing)
  4. README - Updated with builder section and cookbook link

  Commits:
  - 60fcf5f - Add TOON_BUILDER for direct TOON construction
  - 2cb2d7f - Add CHANGELOG documenting v1.0.0 and v1.1.0 releases

Best,

Larry

To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/f66abc27-4cde-4201-80e0-7b88f46d7a6d%40gmail.com.

[Ulrich Windl]

So "tabular arrays" are actually hash tables where the first element is the key (and being unique)? If so, I think the name is quite a misnomer.
Also: May the key be a composite then?

16.12.2025 14:24:48 Liberty Lover <rix....@gmail.com>:

[Ulrich Windl]

Larry,

what made me wonder most is the fact that your TOON class requires JSON to construct TOON (as I understand it). Wouldn't it be more natural if you would offer "basic constuctors" (and "combiners") for each native TOON type?

Ulrich

16.12.2025 14:25:37 Liberty Lover <rix....@gmail.com>:

>
>   Hi Ulrich,
>
>   Good observations! Let me address your points:
>
>   1. Naming of encode
>
>   You're right that encode operates on SIMPLE_JSON_VALUE. However, we already provide semantic aliases:
>
>   encode,
>   to_toon,
>   serialize,
>   compress_for_llm (a_json: SIMPLE_JSON_VALUE): STRING_32
>
>   And for string input:
>   encode_string,
>   json_to_toon (a_json_text: STRING_32): detachable STRING_32
>

>   So to_toon and json_to_toon are already available if you prefer explicit naming. The rationale: TOON's _*/primary/*_ purpose is JSON→TOON conversion for LLM token optimization, so JSON is the expected input.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].
>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com[https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer]].

>>
>> --
>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].

>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/1914fdf0-eb00-415d-86a2-072430af08e0%40gmail.com.
>

> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com?utm_medium=email&utm_source=footer].

[Ulrich Windl]

So you made my recent comments obsolete 😉

16.12.2025 14:31:48 Liberty Lover <rix....@gmail.com>:

>>> And here [https://github.com/simple-eiffel/claude_eiffel_op_docs/blob/main/research/SIMPLE_TOON_RESEARCH.md]is[https://github.com/simple-eiffel/claude_eiffel_op_docs/blob/main/research/SIMPLE_TOON_RESEARCH.md%5Dis] the research behind it.

>>>
>>> Best,
>>>
>>> Larry
>>>
>>> On Mon, Dec 15, 2025 at 11:02 AM Liberty Lover <rix....@gmail.com> wrote:
>>>> Hi All,
>>>>
>>>> There is a new encoder/decoder in town for Eiffel specifically targeting prompt structure being sent to LLM's. The format is called TOON.
>>>>
>>>> You are read all about it here[https://github.com/simple-eiffel/simple_toon].
>>>>
>>>> Best,
>>>>
>>>> Larry
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].
>>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com[https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer]].

>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].
>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfttRgq%2B%3DYhT7tXxuc%2B7N1uZor4aGscUu7RRbXc5YtX_w%40mail.gmail.com?utm_medium=email&utm_source=footer]].

>>
>> --
>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].

>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/f66abc27-4cde-4201-80e0-7b88f46d7a6d%40gmail.com.
>

> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfFrQsnz0M%2BAZEt%2Bfk7yk7G2Ng4HpFkWBdE47Rn07rEew%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfFrQsnz0M%2BAZEt%2Bfk7yk7G2Ng4HpFkWBdE47Rn07rEew%40mail.gmail.com?utm_medium=email&utm_source=footer].

[Liberty Lover]

  Hi Ulrich,

  No, tabular arrays are not hash tables. There is no key concept and no uniqueness requirement.

  Tabular arrays are simply arrays of uniform objects displayed in spreadsheet/table format:
  - Each row = one object (array element)
  - Each column = one field
  - The header {name,price} defines field names, not keys

  products[2]{name,price}:
    Widget,9.99
    Gadget,19.99

  Decodes to a JSON array:
  [{"name": "Widget", "price": 9.99}, {"name": "Gadget", "price": 19.99}]

  Duplicates are allowed in any column - there's no primary key:
  items[3]{category,item}:
    Fruit,Apple
    Fruit,Banana
    Fruit,Cherry

  The name "tabular" refers to the visual layout (rows and columns), not database table semantics. Think "CSV with a header" rather than "hash table."

  If you need key-value semantics, use a regular TOON object:
  lookup{sku}:
    A001: Widget
    B002: Gadget

  Larry

--
You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/91308f7a-d67a-4a56-ba6c-7056a9d1043b%40gmail.com.

[Liberty Lover]

  Hi Ulrich,

  That's exactly what TOON_BUILDER provides (added in v1.1.0 yesterday, per your earlier feedback).

  Without builder (JSON-first):
  create json.make
  json.put_string ("Alice", "name")
  json.put_integer (30, "age")
  Result := toon.encode (json)

  With builder (direct TOON):
  create builder.make
  builder.put_string ("name", "Alice")
  builder.put_integer ("age", 30)
  Result := builder.to_toon

  The builder supports all native types directly:
  - put_string, put_integer, put_real, put_boolean, put_null
  - put_array for simple arrays
  - start_tabular/add_row/end_tabular for tabular arrays
  - start_object/end_object for nested structures

  See: https://github.com/simple-eiffel/simple_toon/blob/main/src/core/toon_builder.e

  The encoder still exists for when you already have JSON (e.g., from an API response). The builder is for constructing TOON from scratch.

  Larry

To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/a2bd4741-c9d8-4791-a3e9-68bb7307fbb4%40gmail.com.

[Ulrich Windl]

Thanks Larry,

so the other arrays do not have the same type for each item?

Ulrich

16.12.2025 18:40:36 Liberty Lover <rix....@gmail.com>:

>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].

>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/91308f7a-d67a-4a56-ba6c-7056a9d1043b%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjehtfLfboT%2BZ9UiG0Usfxgamm4sHwX5P1h4KVeH4fitCw%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjehtfLfboT%2BZ9UiG0Usfxgamm4sHwX5P1h4KVeH4fitCw%40mail.gmail.com?utm_medium=email&utm_source=footer].

[Ulrich Windl]

Larry,

two more questions, sorry:
1. What type is the builder top-level container you add the primitives to?
2. How would you construct more complex nested structures (that's what I had named "combiners")?

Ulrich

16.12.2025 18:41:54 Liberty Lover <rix....@gmail.com>:

>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com][eiffel-users%2Bunsu...@googlegroups.com[eiffel-users%252Buns...@googlegroups.com]].
>>>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com[https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/84eb1e42-90dd-4426-ac11-5306046d9c68n%40googlegroups.com?utm_medium=email&utm_source=footer]]].

>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com][eiffel-users%2Bunsu...@googlegroups.com[eiffel-users%252Buns...@googlegroups.com]].

>>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/1914fdf0-eb00-415d-86a2-072430af08e0%40gmail.com.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].

>>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com?utm_medium=email&utm_source=footer][https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com%5Bhttps://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjcymgJStQQqVRqQjas1E7AcdF9R5%2Bc5HAmkiJGG_MmMDw%40mail.gmail.com?utm_medium=email&utm_source=footer]].

>>
>> --
>> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com[eiffel-users%2Bunsu...@googlegroups.com].

>> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/a2bd4741-c9d8-4791-a3e9-68bb7307fbb4%40gmail.com.

>
> --
> You received this message because you are subscribed to the Google Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to eiffel-users...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfyM%3D0LWNwY5NOmzURYcS1mqrjtfvACRKqE4OWBmQL4Jw%40mail.gmail.com[https://groups.google.com/d/msgid/eiffel-users/CA%2B3qnjfyM%3D0LWNwY5NOmzURYcS1mqrjtfvACRKqE4OWBmQL4Jw%40mail.gmail.com?utm_medium=email&utm_source=footer].