[llvm-dev] FileCheck DAG match N lines

[David Stenberg via llvm-dev]

Hi!

If a test has output of the following form:

START
VALUE X1
VALUE X2
[...]
VALUE X_N
END

where the order of X_1..X_N is not deterministic, is it possible to
write FileCheck checks that verify that the "VALUE" lines occur in any
order, with only one occurrence per line, and no other "VALUE" lines in
between?

One idea that was suggested to me was to write checks of the following
form:

CHECK: START
CHECK-NOT: VALUE
CHECK-DAG: VALUE X1
CHECK-DAG: VALUE X2
[...]
CHECK-DAG: VALUE X_N
CHECK-NOT: VALUE
CHECK: END

However, that for example matches on:

START
VALUE X1
VALUE X1
VALUE Foo
VALUE X3
VALUE X1
VALUE X2
END

I have looked through the FileCheck documentation, but haven't found
anything that describes a similar case.

Are these sorts of checks achievable with FileCheck?

Best regards,
David
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Robinson, Paul via llvm-dev]

> If a test has output of the following form:
>
> START
> VALUE X1
> VALUE X2
> [...]
> VALUE X_N
> END
>
> where the order of X_1..X_N is not deterministic, is it possible to
> write FileCheck checks that verify that the "VALUE" lines occur in any
> order, with only one occurrence per line, and no other "VALUE" lines in
> between?
>
> One idea that was suggested to me was to write checks of the following
> form:
>
> CHECK: START
> CHECK-NOT: VALUE
> CHECK-DAG: VALUE X1
> CHECK-DAG: VALUE X2
> [...]
> CHECK-DAG: VALUE X_N
> CHECK-NOT: VALUE
> CHECK: END

That sequence will verify there's no VALUE prior to the first
matching CHECK-DAG, or after the last matching CHECK-DAG. It
won't look at anything between the first and last DAG match.

If VALUE doesn't occur anywhere else, it's possible that the
command-line option `-implicit-check-not` would do what you want.
Offhand I don't recall how it interacts with CHECK-DAG, though.
--paulr

[Joel E. Denny via llvm-dev]

Hi David,

This use case has come up several times.  I've not heard of any existing solution.

I've been thinking we should have CHECK-DAG-NEXT blocks.  Likewise, for unordered text on a single line, we'd have CHECK-DAG-SAME blocks.  In both these cases, we'd have the question of how to handle the first match.  Would it be bound by the NEXT or SAME rule relative to the match preceding the block?

If anyone wants to implement something like this, please cc me on the discussion/review.  I'd like to make use of such a feature, and I have ideas on how to address some of the subtleties, such as the question above.

Thanks.

Joel