Possible bug in Spring4D 2.0.2: Memoize causes empty enumeration on first iteration

[Nils]

Hi,

after upgrading to Spring4D 2.0.2, I encountered an issue where an IEnumerable combined with Memoize returns no elements on the first for..in iteration, but works correctly on the second iteration.
Without Memoize, everything behaves as expected.

I can reproduce the issue in both Delphi 11.3 and Delphi 12.3.

The problem did not occur with Spring4D 2.0.0 (I skipped 2.0.1, so I don't know exactly when it was introduced).

Important detail:

The bug only appears when the source sequence comes from a dictionary, specifically from Dict.Values.

I could not reproduce the issue when using IList<T>.

procedure TMyTestObject.Test;
type
  TTestRec = record
    Int: Integer;
  end;
var
  Rec: TTestRec;
begin
  var Dict := TCollections.CreateDictionary<Integer,TTestRec>;
  for var I := 10 downto 1 do
  begin
    Rec.Int := I;
    Dict[I] := Rec;
  end;

  var SomeValues := Dict.Values
    .Where(function(const ARec: TTestRec): Boolean
    begin
      Result := ARec.Int > 7;
    end)
    .Memoize; // Without Memoize -> works as expected

  var OrderedList := SomeValues.Ordered(
    function(const Left, Right: TTestRec): Integer
    begin
      Result := CompareValue(Left.Int, Right.Int);
    end);

  var IntEnum := TEnumerable.Select<TTestRec,Integer>(
    OrderedList,
    function(const ARec: TTestRec): Integer
    begin
      Result := ARec.Int;
    end);

  var Count1 := 0;
  for var Item in IntEnum do
    Inc(Count1);

  var Count2 := 0;
  for var Item in IntEnum do
    Inc(Count2);

  Assert.IsTrue(Count1 = Count2, 'Should be equal');
end;

Observed behavior:

• Count1 is 0

• Count2 is 3

• Expected: both counts should be equal.

This only happens when Memoize is used before the Ordered call.
It did not occur with previous Spring4D versions.

Is this a known issue, or is there something wrong with this usage pattern?

Thanks in advance for any insights!

[Stefan Glienke]

Hi, I cannot reproduce this behavior in either 2.0.2 or develop.

If you add messages to the code as follows:

  var SomeValues := Dict.Values
    .Where(function(const ARec: TTestRec): Boolean
    begin

      Writeln('Where: ', ARec.Int);

      Result := ARec.Int > 7;
    end)
    .Memoize; // Without Memoize -> works as expected

  var OrderedList := SomeValues.Ordered(
    function(const Left, Right: TTestRec): Integer
    begin

      Writeln('Compare: ', Left.Int, ' ', Right.Int);

      Result := CompareValue(Left.Int, Right.Int);
    end);

  var IntEnum := TEnumerable.Select<TTestRec,Integer>(
    OrderedList,
    function(const ARec: TTestRec): Integer
    begin

      Writeln('Select: ', ARec.Int);
      Result := ARec.Int;
    end);

What output do you get? Mine is this:

Where: 10
Where: 9
Where: 8
Where: 7
Where: 6
Where: 5
Where: 4
Where: 3
Where: 2
Where: 1
Compare: 10 9
Compare: 10 8
Compare: 9 8
Select: 8
Select: 9
Select: 10
Compare: 10 9
Compare: 10 8
Compare: 9 8
Select: 8
Select: 9
Select: 10

[Nils]

Thank you very much for your super quick reply! I had to experiment a bit more yesterday to shed some more light on the issue.

First of all, please excuse me for getting the version number wrong — I honestly thought I was using version 2.0.2, but in fact I was working with develop.

For me it actually also works when I reference the DCUs. However, if I directly add the directories source\Base and source\Base\collections to the search path, then I get the error.

What’s also interesting is the console output:

         Where: 10
         Where: 9
         Where: 8
         Where: 7
         Where: 6
         Where: 5
         Where: 4
         Where: 3
         Where: 2
         Where: 1

         Compare: 9 10
         Compare: 8 10
         Compare: 8 9

         Select: 8
         Select: 9
         Select: 10

At first I didn’t even notice it — but in your output (and in mine when I use the DCUs) the Compare/Select section is listed twice.

[Stefan Glienke]

The hint with compiling from sources was important - the call to DynArrayCopyRange behaves differently when compiling from sources, it inlines this small wrapper around _DynArrayCopyRange

But the actual bug was in GetCount, affecting fItems; the parameter-passing order is undefined behavior in Delphi. Fixed

[Nils]

I really didn’t expect this kind of behavior caused by inlining — definitely something new I’ve learned today.
Thanks a lot for your help and the quick fix; I really appreciate it.