Reading from SPECIAL arrays: Eiffel Studio version 25.12 VS 16.05

[Finnian Reilly]

Reading from SPECIAL arrays: Eiffel Studio version 25.12 VS 16.05

Lately I got my benchmarking suite up and running for the first time on v25.12. I ran this set of benchmarks STRING_8_VS_MANAGED_C_STRING_8 and had some surprising results compared to v16.05.  I had recorded the v16.05 results as a note in the source code.  I booted up my old machine again to double check and the results were consistent with some slight variation.

(Measuring passes in 1000 millisecs)

Compiler version 16.05 + gcc v4.8.4 on Linux Mint 19 (Mac Mini 2012)

   RESULTS: indexed_item

      C buffer indexed_item   :  788.0 times (100%)
      C buffer indexed_string :  782.0 times (-0.8%)
      SPECIAL indexed_item    :  780.0 times (-1.0%)

   RESULTS: starts_with

      C buffer starts_with : 2178.0 times (100%)
      SPECIAL starts_with  :  878.0 times (-59.7%)

   RESULTS: occurrences

      C buffer occurrences : 2170.0 times (100%)
      SPECIAL occurrences  : 1687.0 times (-22.3%)

   RESULTS: CSV line parsing

      C buffer parse_csv :  65.0 times (100%)
      SPECIAL parse_csv  :  52.0 times (-20.0%)

Compiler version: 25.12 + gcc v13.3.0 on Linux Mint 22.2 (Lenovo ThinkCentre M90a Pro Gen6 2025)

   RESULTS: indexed_item
      C buffer indexed_item   :  5689.0 times (100%)
      SPECIAL indexed_item    :  1434.0 times (-74.8%)
      C buffer indexed_string :   232.0 times (-95.9%)

   RESULTS: starts_with

      C buffer starts_with :  5199.0 times (100%)
      SPECIAL starts_with  :  2666.0 times (-48.7%)

   RESULTS: occurrences

      SPECIAL occurrences  : 1377.0 times (100%)
      C buffer occurrences :  441.0 times (-68.0%)

   BENCHMARKING: CSV line parsing

      SPECIAL parse_csv  : 127.0 times (100%)
      C buffer parse_csv : 107.0 times (-15.7%)

Impact on Xpact project

While this is not good news in general, it's actually good news for the Xpact project.  I changed the code  on the hot paths to read  from SPECIAL arrays using the base_address and some inline C code. The average speed went up from x1.292  to x1.311 relative to eXpat.

frozen c_read_character_8 (a_area: POINTER; i: INTEGER): CHARACTER_8
      -- Character at offset `i' in buffer `a_area'.
   external
      "C inline"
   alias
      "return ((EIF_CHARACTER_8 *)$a_area)[$i];"
   end

Average performance relative to eXpat

Using SPECIAL.item the average is x1.292 
Fastest benchmark: [x1.53 to eXpat] mandarin-names-and-text.xsl CRC-32-cdata: eXpat passes = 2069; Xpact-core passes = 3169

Using c_read_character_8 the average is x1.311
Fastest benchmark:  [x1.70 to eXpat] mandarin-names-and-text.xsl CRC-32-attribute: eXpat passes = 1564; Xpact-core passes = 2662

As an experiment, I tried replacing SPECIAL buffers with a class derived from MANAGED_POINTER, using some inline externals for the hot paths. It complicated the code and actually degraded performance, so I reverted to SPECIAL, which works better with STRING_8. I wasted a few days on this, but it was educational, and I did at least get it passing the full test suite before deciding to stick with SPECIAL buffers.

-- 
SmartDevelopersUseUnderScoresInTheirIdentifiersBecause_it_is_much_easier_to_read
(Eiffel = Security by Contract + C Speed)

[Eric Bezault]

Note that the differences in the benchmark results are not
necessarily due to the use of a new version of EiffelStudio
alone. There are several other factors which might have an
impact:

- not the same machine
- not the same OS version
- not the same C compiler version

Otherwise, I assume that you were compiling in non-void-safe
mode when using 16.05. Are compiling in void-safe mode with
25.12?

--
Eric Bezault <er...@gobosoft.com>
Eiffel expert - available for freelance work
https://www.gobosoft.com

On 30/08/2026 15:12, Finnian Reilly wrote:
>
> Reading from SPECIAL arrays: Eiffel Studio version 25.12 VS 16.05
>
> Lately I got my benchmarking suite up and running for the first time on

> *v25.12.* I ran this set of benchmarks STRING_8_VS_MANAGED_C_STRING_8
> <https://github.com/finnianr/Eiffel-Loop/blob/master/benchmark/source/
> comparison/string/string_8_vs_managed_c_string_8.e> and had some
> surprising results compared to *v16.05*.  I had recorded the *v16.05*

> results as a note in the source code. I booted up my old machine again
> to double check and the results were consistent with some slight variation.
>
> (Measuring passes in 1000 millisecs)
>

> *Compiler version 16.05 + gcc v4.8.4 on Linux Mint 19 (Mac Mini 2012)*
>
> RESULTS:*indexed_item*

>
> C buffer indexed_item : 788.0 times (100%)
> C buffer indexed_string : 782.0 times (-0.8%)
> SPECIAL indexed_item : 780.0 times (-1.0%)
>

> RESULTS:*starts_with*

>
> C buffer starts_with : 2178.0 times (100%)
> SPECIAL starts_with : 878.0 times (-59.7%)
>

> RESULTS:*occurrences*

>
> C buffer occurrences : 2170.0 times (100%)
> SPECIAL occurrences : 1687.0 times (-22.3%)
>

> RESULTS:*CSV line parsing*

>
> C buffer parse_csv : 65.0 times (100%)
> SPECIAL parse_csv : 52.0 times (-20.0%)
>

> *Compiler version: 25.12 + gcc v13.3.0 on Linux Mint 22.2 (Lenovo
> ThinkCentre M90a Pro Gen6 2025)*
>
> RESULTS:*indexed_item*

> C buffer indexed_item : 5689.0 times (100%)
> SPECIAL indexed_item : 1434.0 times (-74.8%)
> C buffer indexed_string : 232.0 times (-95.9%)
>

> RESULTS:*starts_with*

>
> C buffer starts_with : 5199.0 times (100%)
> SPECIAL starts_with : 2666.0 times (-48.7%)
>

> RESULTS:*occurrences*

>
> SPECIAL occurrences : 1377.0 times (100%)
> C buffer occurrences : 441.0 times (-68.0%)
>

> BENCHMARKING:*CSV line parsing*

>
> SPECIAL parse_csv : 127.0 times (100%)
> C buffer parse_csv : 107.0 times (-15.7%)
>
>
> Impact on Xpact project
>
> While this is not good news in general, it's actually good news for the
> Xpact project.  I changed the code  on the hot paths to read  from

> *SPECIAL* arrays using the *base_address* and some inline C code. The

> average speed went up from x1.292  to x1.311 relative to eXpat.
>
> frozen c_read_character_8 (a_area: POINTER; i: INTEGER): CHARACTER_8
> -- Character at offset `i' in buffer `a_area'.
> external
> "C inline"
> alias
> "return ((EIF_CHARACTER_8 *)$a_area)[$i];"
> end
>
>
> Average performance relative to eXpat
>

> Using *SPECIAL.item* the average is *x1.292
> *Fastest benchmark: [x1.53 to eXpat] mandarin-names-and-text.xsl CRC-32-

> cdata: eXpat passes = 2069; Xpact-core passes = 3169
>

> Using *c_read_character_8* the average is* x1.311
> *Fastest benchmark:  [x1.70 to eXpat] mandarin-names-and-text.xsl

> CRC-32-attribute: eXpat passes = 1564; Xpact-core passes = 2662
>

> As an experiment, I tried replacing *SPECIAL* buffers with a class
> derived from *MANAGED_POINTER*, using some inline externals for the hot

> paths. It complicated the code and actually degraded performance, so I

> reverted to *SPECIAL*, which works better with *STRING_8*. I wasted a

> few days on this, but it was educational, and I did at least get it

> passing the full test suite before deciding to stick with *SPECIAL* buffers.

>
> --
> SmartDevelopersUseUnderScoresInTheirIdentifiersBecause_it_is_much_easier_to_read
> (Eiffel = Security by Contract + C Speed)
>

> --
> You received this message because you are subscribed to the Google
> Groups "Eiffel Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to eiffel-users...@googlegroups.com <mailto:eiffel-
> users+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/eiffel-
> users/9fdbe74b-65a7-4d1b-8374-6fa5b4359e0c%40eiffel-loop.com <https://
> groups.google.com/d/msgid/eiffel-
> users/9fdbe74b-65a7-4d1b-8374-6fa5b4359e0c%40eiffel-loop.com?
> utm_medium=email&utm_source=footer>.

[Finnian Reilly]

Hi Eric,

that is true, there are many factors at play. 

What is constant is the codebase and the ECF. I only changed a few syntactical things to make it compile on 25.12, but it's essentially the same non-void safe mode on both. I haven't gotten around to migrating Eiffel-Loop to Void-safe.

The Xpact code is void safe, but it is a separate project and not part of Eiffel-Loop. However I pulled in few classes from Eiffel-Loop that were useful to Xpact and made them void-safe, and put them in a contrib folder in Xpact.

Regardless of what is causing the change in results I still find it surprising. I asked Claude "Has gcc undergone any changes in the last 10 years that would make a major performance difference ?"

Claude

Yes, quite a lot, actually, and the changes are exactly the kind that could plausibly explain a chunk of what you're seeing with raw pointer/array access.

Codegen quality at -O2/-O3 has genuinely improved. One independent benchmark comparing gcc5 to gcc14 found execution time decreased by roughly 27% at -O3, 32% at -O1, and as much as 47% at -Og, comparing gcc5 to gcc14, with more modest gains at -O0. So if EiffelStudio compiles its generated C at a reasonably high optimization level, you'd expect a real, non-trivial speedup from the gcc upgrade alone, independent of anything EiffelStudio changed. arxiv

Auto-vectorization has been a major focus area. Intel's own writeup on GCC 12 describes several auto-vectorization enhancements for Intel AVX-512, with GCC 12.1 delivering a 19% improvement over GCC 7.5 on the SPECrate 2017 Integer benchmark, and notes a very-cheap cost model was introduced to enable vectorization even at -O2, extending those gains to a much wider set of users. If any part of your SPECIAL or C-buffer access pattern is loop-like (scanning, copying, character-by-character comparison), this is squarely the kind of change that would speed it up on modern gcc without anyone touching EiffelStudio. IntelIntel

Recent releases keep finding gains in branch handling and inlining tuning, which is directly relevant to your indexed_item bounds-check story. A very recent example: an Intel engineer found a single-line change to GCC's branch misprediction cost model that produced a 12% improvement on a SPEC CPU 2017 benchmark on both modern Intel and AMD chips, by making the compiler more willing to generate branchless code instead of branching code. That's precisely the sort of micro-tuning that would shift the relative cost of a bounds-checked SPECIAL access versus a raw C dereference, in either direction, depending on which one has more branches for the optimizer to target. XDA Developers

The flip side: compile times have gone up substantially for that performance, so it's not a free lunch, but that doesn't affect your runtime numbers. One analysis found compilation time increased by about 42% at -O3 comparing gcc5 to gcc14 over a nine-year span, tracking the introduction of new optimization passes aimed at faster generated code. GitHub

Net: yes, "the modern compiler is faster at C-level access" is a credible and well-documented claim on its own terms, gcc really has gotten meaningfully better at exactly this kind of low-level work over the past decade. That supports your Xpact result standing on its own. It also means, as you noted, that untangling how much of the SPECIAL-vs-16.05 story is "gcc got better" versus "EiffelStudio's codegen changed" is a real, non-trivial attribution problem, not something you can wave away either way without controlling for gcc version directly.

Note that the differences in the benchmark results are not
necessarily due to the use of a new version of EiffelStudio
alone. There are several other factors which might have an
impact:

- not the same machine
- not the same OS version
- not the same C compiler version

Otherwise, I assume that you were compiling in non-void-safe
mode when using 16.05. Are compiling in void-safe mode with
25.12?

-- 
SmartDevelopersUseUnderScoresInTheirIdentifiersBecause_it_is_much_easier_to_read
(Eiffel = Security by Contract + C Speed)

[Finnian Reilly]

Hi Mischa,
in the new version the relationship between SPECIAL and the C buffer has
inverted. SPECIAL is counting occurrences much faster.
But yes in absolute terms it's actually got slower on a much more
powerful machine. I didn't notice. Very strange !
regards
Finnian
> Hi Finnian,
>
> That's odd, the 'occurrences' got  quite a bit slower, despite some 10
> years of hardware advances and software/compiler improvements... Any
> idea what might have happened?
>
> Best,
> Mischa

[Finnian Reilly]

Trying inlining = 5 VS previous 15

For the initial benchmark report I had inlining set to 15.  I was wondering if the strange results could be accounted for by an "excessive" amount of inlining. So I decided to try a more modest figure of 5. 

Performed September 2nd 2026 at 16:00 GMT

Inlining = 5 (previously 15)

Compiler version 16.05 + gcc v4.8.4 on Linux Mint 19 (Mac Mini 2012)

RESULTS: indexed_item
    C buffer indexed_item   : 1553.0 times (100%)
    SPECIAL indexed_item    : 1545.0 times (-0.5%)
    C buffer indexed_string : 1541.0 times (-0.8%)

RESULTS: starts_with
    C buffer starts_with : 1547.0 times (100%)
    SPECIAL starts_with  :  833.0 times (-46.2%)

RESULTS: occurrences
    C buffer occurrences :  973.0 times (100%)
    SPECIAL occurrences  :  846.0 times (-13.1%)

RESULTS: CSV line parsing
    C buffer parse_csv :  58.0 times (100%)
    SPECIAL parse_csv  :  51.0 times (-12.1%)

Compiler version: 25.12 + gcc v13.3.0 on Linux Mint 22.2 (Lenovo ThinkCentre M90a Pro Gen6 2025)

RESULTS: indexed_item
    C buffer indexed_item   :  5854.0 times (100%)
    SPECIAL indexed_item    :  1533.0 times (-73.8%)
    C buffer indexed_string :   238.0 times (-95.9%)

RESULTS: starts_with
    C buffer starts_with :  5235.0 times (100%)
    SPECIAL starts_with  :  2712.0 times (-48.2%)

RESULTS: occurrences
    SPECIAL occurrences  : 1402.0 times (100%)
    C buffer occurrences :  453.0 times (-67.7%)

RESULTS: CSV line parsing
    SPECIAL parse_csv  : 127.0 times (100%)
    C buffer parse_csv : 108.0 times (-15.0%)

Observation

As you can see, the result is substantially the same. A very precipitous drop in relative performance of SPECIAL.item VS reading with a C pointer using an inline external. 

Counting occurences

But at least SPECIAL is outperforming C inlining on counting occurrences. The C buffer code looks like

    frozen occurrences (c: CHARACTER_8): INTEGER
        -- Number of times `c' appears in `area'
        local
            i, l_count: INTEGER; l_area: POINTER
        do
            l_area := area; l_count := count
            from i := 0 until i = l_count loop
                if read_character_8 (l_area, i) = c then
                    Result := Result + 1
                end
                i := i + 1
            end
        end

read_character_8 is a contractual wrapper for external function c_read_character_8.

The SPECIAL code is found in READABLE_STRING_8

    occurrences (c: CHARACTER_8): INTEGER
            -- Number of times `c' appears in the string.
        local
            i, nb: INTEGER
            a: SPECIAL [CHARACTER_8]
        do
            from
                i := area_lower
                nb := count + i
                a := area
            until
                i = nb
            loop
                if a.item (i) = c then
                    Result := Result + 1
                end
                i := i + 1
            end
        ensure then
            zero_if_empty: count = 0 implies Result = 0
            recurse_if_not_found_at_first_position:
                (count > 0 and then item (1) /= c) implies
                    Result = substring (2, count).occurrences (c)
            recurse_if_found_at_first_position:
                (count > 0 and then item (1) = c) implies
                    Result = 1 + substring (2, count).occurrences (c)
        end

[Ulrich Windl]

Hi!

Back in 1992 we also had that effect that some program commercial was executing in less time on a slower computer. Nobody really understood, and nobody did analyze. That was the 80486 aera when 33MHz was a fast CPU...

Ulrich

30.08.2026 21:53:35 Finnian Reilly <fin...@eiffel-loop.com>: