Strange bug in xforms repeat in Orbeon 2018.1CE

[Gerrit Boers]

Hi,

We’re testing the Orbeon 2018.1 release and I ran into a strange issue with the xforms repeat.

1. The xxforms-nodeset-changed event no longer fires after first form load.

2. When other actions in the form are triggered the selection index changes to the last item in the set and the xxforms-index-changed-event fired thus changing the selection.

Relevant code snippet:

                            <xforms:repeat nodeset="instance('designation-resultset')/designation" id="designations" xxforms:external-events="keydown-pressed keyup-pressed">
                                <xhtml:tr class="{if (instance('focus')='search') then '' else 'not-selectable'}">
                                    <xhtml:td width="15%" style="padding-left:0.8em;">
                                        <xhtml:span class="{if (@inSet) then 'code-in-set' else()}">
                                            <xforms:output ref="@code"/>
                                        </xhtml:span>
                                        <!--                                        <xforms:output ref="@code"/>-->
                                    </xhtml:td>
                                    <xhtml:td width="55%">
                                        <xforms:output ref="."/>
                                    </xhtml:td>
                                    <xhtml:td width="30%">
                                        <xforms:output ref="@codeSystemName"/>
                                    </xhtml:td>
                                </xhtml:tr>
                                <xforms:action ev:event="xxforms-nodeset-changed">
                                    <xforms:action if="instance('focus')='search' and (instance('selected-concept')/@code!=instance('designation-resultset')/designation[index('designations')]/@code or instance('selected-concept')/@codeSystem!=instance('designation-resultset')/designation[index('designations')]/@codeSystem)">
                                        <xforms:setvalue ref="instance('selected-concept')/@code" value="instance('designation-resultset')/designation[index('designations')]/@code"/>
                                        <xforms:action if="instance('selected-concept')/@codeSystem!=instance('designation-resultset')/designation[index('designations')]/@codeSystem">
                                            <xforms:setvalue ref="instance('selected-concept')/@codeSystem" value="instance('designation-resultset')/designation[index('designations')]/@codeSystem"/>
                                            <xforms:send submission="get-codesystem-info"/>
                                        </xforms:action>
                                        <xforms:send if="(instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='complex' or instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &gt;30) and not(instance('compose')/include)" submission="get-concept"/>
                                        <xforms:send if="(instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='complex' or instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &gt;30) and instance('compose')/include" submission="get-concept-in-context"/>
                                        <xforms:send if="instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='simple' and instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &lt;31" submission="get-codesystem"/>
                                    </xforms:action>
                                </xforms:action>
                                <xforms:action ev:event="xxforms-index-changed">
                                    <xforms:setvalue ref="instance('focus')" value="'search'"/>
                                    <xforms:action if="instance('selected-concept')/@code!=instance('designation-resultset')/designation[index('designations')]/@code or instance('selected-concept')/@codeSystem!=instance('designation-resultset')/designation[index('designations')]/@codeSystem">
                                        <xforms:setvalue ref="instance('selected-concept')/@code" value="instance('designation-resultset')/designation[index('designations')]/@code"/>
                                        <xforms:action if="instance('selected-concept')/@codeSystem!=instance('designation-resultset')/designation[index('designations')]/@codeSystem">
                                            <xforms:setvalue ref="instance('selected-concept')/@codeSystem" value="instance('designation-resultset')/designation[index('designations')]/@codeSystem"/>
                                            <xforms:send submission="get-codesystem-info"/>
                                        </xforms:action>
                                        <xforms:setvalue ref="instance('focus')" value="'search'"/>
                                        <xforms:send if="(instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='complex' or instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &gt;30) and not(instance('compose')/include)" submission="get-concept"/>
                                        <xforms:send if="(instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='complex' or instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &gt;30) and instance('compose')/include" submission="get-concept-in-context"/>
                                        <xforms:send if="instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/@type='simple' and instance('codesystem-list')/codeSystem[@oid=instance('selected-concept')/@codeSystem]/xs:integer(@count) &lt;31" submission="get-codesystem"/>
                                    </xforms:action>
                                </xforms:action>
                            </xforms:repeat>
                        </xhtml:table>

The second issue is serious as it changes the application flow.

Here’s the action that causes the index of the repeat to change to the last item in the set:

                                    <xforms:trigger appearance="compact">
                                        <xforms:label class="control-label">
                                            <xhtml:img src="/img/gear16.png" alt=""/>
                                        </xforms:label>
                                        <xforms:hint ref="$resources/compose-valueset"/>
                                        <xforms:action ev:event="DOMActivate">
                                            <xforms:setvalue ref="instance('focus')" value="'detail'"/>
                                            <xforms:setvalue ref="instance('include')/system/@oid" value="instance('codesystem-info')/@oid"/>
                                            <xforms:setvalue ref="instance('include')/system/@value" value="instance('codesystem-info')/@url"/>
                                            <xforms:setvalue ref="instance('include')/system/@display" value="instance('codesystem-info')/name[1]"/>
                                            <xforms:setvalue ref="instance('include')/@home" value="instance('concept-codesystem')/@code"/>
                                            <xforms:insert context="instance('compose')" nodeset="*" origin="instance('include')"/>
                                            <xforms:send submission="get-valueset-expansion"/>
                                            <xforms:send submission="get-concept-in-context"/>
                                            <xforms:delete ref="instance('designation-resultset')/compose"/>
                                            <xforms:insert context="instance('designation-resultset')" origin="instance('compose')"/>
                                            <xforms:send submission="validate-search-result"/>
                                        </xforms:action>
                                    </xforms:trigger>

Any suggestions as to what could be done by me or is it indeed an error?

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit,

One change in 2018.1 is that the `xf:insert` and `xf:delete` actions don't
update repeats right away but on refresh, as was already done for any other
change to the data model. We think this is more consistent with what happens
with other updates to the data model, and avoids a number of corner cases.

Would this explain what you are seeing? If so, could you adapt your code to
this new behavior? If not easily done, you can set
`oxf.xforms.update-repeats` to `true` to get the old behavior, but I would
try to avoid this, as this will at some point most likely be deprecated.

https://doc.orbeon.com/configuration/properties/xforms#xforms-repeat-updates-upon-xf-insert-and-xf-delete

-Alex

-----
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
--
Sent from: http://discuss.orbeon.com/

[Gerrit Boers]

Hi Alex,

I tried setting the property. It did solve the minor issue of the xxforms-nodeset-changed not firing but the the index of the repeat is still changing to the last item in the set. The action that causes this to happen does not change the content of the repeat in any way, so why does the index move to the last item in the set?
I’d rather not use the property, the issue with the xxforms-nodeset-changed I can solve by moving the code to the submission but the more serious problem with index of the repeat changing remains.

King regards,

Gerrit

> --
> You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to orbeon+un...@googlegroups.com.
> To post to this group, send email to orb...@googlegroups.com.

[Alessandro Vernet]

OK Gerrit, for the index issue, could you create for me a minimal example
that I can run here, really keeping it as simple as possible, so I can
understand and possibly debug the issue?

[Alexander Henket]

Hi Alessandro. I can reproduce using this fragment under Orbeon 2018.1. As soon as you click Refresh, you'll see the index go the last item for some reason.

<xh:html xmlns:xf="http://www.w3.org/2002/xforms" xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events">
    <xh:head>
        <xh:title>orbeon-2018-1-last-index-issue</xh:title>
        <xf:model id="repro">
            <xf:instance id="test1">
                <tests>
                    <test>1</test>
                    <test>2</test>
                </tests>
            </xf:instance>
            <xf:instance id="test2">
                <tests>
                    <test>1</test>
                    <test>2</test>
                    <test>3</test>
                    <test>4</test>
                </tests>
            </xf:instance>
        </xf:model>
    </xh:head>
    <xh:body>
        <xh:h3>TEST</xh:h3>
        <xf:trigger appearance="compact">
            <xf:label>Refresh</xf:label>
            <xf:insert ev:event="DOMActivate" nodeset="instance('test1')" origin="instance('test2')"/>
        </xf:trigger>
        <xh:table bgcolor="white">
            <xf:repeat nodeset="instance('test1')/test" id="test-table">
                <xh:tr>
                    <xh:td><xf:output ref="."/></xh:td>
                </xh:tr>
            </xf:repeat>
        </xh:table>
    </xh:body>
</xh:html>

[General Brainworking]

Hi Alex,

I’ll get on it, could take some time though to come up with a simple form with this issue.

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Alexander,

Thank you for the example; and indeed, the index is updated to 4, which
seems to be inline with the following statement in the XForms spec: "If
items are added to the repeat collection, then the repeat index is set to
the position of the last item added.", from section 10.4.1.3. Is this what
you're referring to? If so, it is possible that earlier versions didn't
implement this correctly.

https://www.w3.org/community/xformsusers/wiki/XForms_2.0#Index_Processing

[Alexander Henket]

A couple of points:

1. This is new behavior for Orbeon 2018.1 right? It wasn't doing this in Orbeon 2017.2.2
   
2. We almost always refresh full instances through submissions. After xforms-submit-done we 'manually' set the index to what we want it to be and then leave it up to xxforms-index-changed to follow selections in the repeat. If the selected item that was in the set before still exists after xforms-submit-done, we go to that index, if not we go to the first, never the last. The problem is that since Orbeon 2018.1 this has become impossible because the Orbeon-machinery takes over whatever you do. In the fragment above, after clicking Refresh "2" is still in the set and so I don't want my index changed at all, but if it "has to" change then go to index 1, not last().

My/our main point is that behavior changed in a backward incompatible way and we are left without ways to influence the desired behavior.

[Alexander Henket]

Example: we have a tree with objects. Every time you click an object, this triggers a detail view and loads of interface elements become active or change status.

When you refresh the tree from the server, there may be new items in the list, others may have disappeared. After forms-submit-done we look up if the previously selected item is still present, and if so select it. If is not then we go to the first item in the list because that makes sense. Going to the last item in a list after a refresh, unless a user specifically chooses to do so, does not make sense.

The problem is that <xforms:action ev:event="xforms-submit-done"> ... </xforms:action> is completely ignored after refresh and so there's nothing I can do to implement desired refresh behavior.

[Gerrit Boers]

Hi Alex,

In the original case the content of the repeat is not changed at all but still the index moves to the last item in the repeat.

Kind regards,

Gerrit

[Alexander Henket]

Then you should probably add a second minimal example for Alessandro

You received this message because you are subscribed to a topic in the Google Groups "Orbeon Forms" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orbeon/YWM7tURmS8s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orbeon+un...@googlegroups.com.

[Alessandro Vernet]

Hi Alexander

1. This is new behavior for Orbeon 2018.1 right? It wasn't doing this in Orbeon 2017.2.2

I haven't tested this example on 2017.2.2, but if you're saying that it worked differently in that version, I trust you ;). As long as the current implementation is inline with the spec, and inline with what we think makes sense, it seems like we're on a good path.

 

2. We almost always refresh full instances through submissions. After xforms-submit-done we 'manually' set the index to what we want it to be and then leave it up to xxforms-index-changed to follow selections in the repeat.

This should work, because the `xf:setindex` triggers an immediate XForms refresh before setting the index per your instruction. The index will have been set to the last iteration during the refresh, but then it is set to whatever value you provided. If this isn't what you're seeing, could you send me a minimal example that shows the problem?

-Alex 

[Alexander Henket]

I think I did not explain myself right, so I extended the fragment to include setting the index to 1 after nodes-change. In 2017.2.2 this works as desired. In 2018.1 the setindex is performed and subsequently overruled and reset to last(). I trust you agree that ignoring my setindex is not desired behavior.

Sequence: -- click Refresh, this triggers the nodeset-changed and setindex 1. You will get xf:message for what is happening under the hood. You will also see and xforms-index-changed event being trigger to index 4. That last thing is not me: that is Orbeon.

<xh:html xmlns:xf="http://www.w3.org/2002/xforms" xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events">
    <xh:head>
        <xh:title>orbeon-2018-1-last-index-issue</xh:title>
        <xf:model id="repro">
            <xf:instance id="test1">
                <tests>
                    <test>1</test>
                    <test>2</test>
                </tests>
            </xf:instance>
            <xf:instance id="test2">
                <tests>
                    <test>1</test>
                    <test>2</test>
                    <test>3</test>
                    <test>4</test>
                </tests>
            </xf:instance>
        </xf:model>
    </xh:head>
    <xh:body>
        <xh:h3>TEST</xh:h3>
        <xf:trigger appearance="compact">
            <xf:label>Refresh</xf:label>

            <xf:action ev:event="DOMActivate">
                <xf:insert nodeset="instance('test1')" origin="instance('test2')"/>
                <xf:message>Set index 1</xf:message>
                <xf:setvalue ref="index('test-table')" value="1"/>
            </xf:action>
        </xf:trigger>
        <xh:table style="background-color: white; border: 1px solid black;">

            <xf:repeat nodeset="instance('test1')/test" id="test-table">
                <xh:tr>

                    <xh:td style="padding: 1em;"><xf:output ref="."/>&#160; - List item</xh:td>
                </xh:tr>
                <xf:action ev:event="xxforms-nodeset-changed">
                    <xf:message>nodeset change. &#160;<xf:output ref="index('test-table')"/></xf:message>
                </xf:action>
                <xf:action ev:event="xxforms-index-changed">
                    <xf:message>index change. &#160;<xf:output ref="index('test-table')"/></xf:message>
                </xf:action>

[Alessandro Vernet]

Hi Alexander,

This is because you'll want to use:

<xf:setindex repeat="test-table" index="1"/>

instead of

<xf:setvalue ref="index('test-table')" value="1"/>

The `index()` function returns an integer, not a node whose value you could
set. And if I do this change in your example, the index is indeed 1 at the
end, of the refresh.

https://www.w3.org/community/xformsusers/wiki/XPath_Expressions_Module#The_index.28.29_Function

[Alexander Henket]

Right. I guess that previous versions were forgiving for this, as I don't recall issues with setvalue like this.

:( Just airing my frustration for this counterintuitive part of the spec. Mostly because of this issue we have decided to go to 2017.2.2 so we have time to figure out how to deal with this major setback (796 repeats in 50 files)

Please note that Gerrit might still come up with a different fragment as apparently the part that hit me, was not exactly the part that hit him, so I kind of hijacked his thread.

Thank you for your answers even if the bigger picture isn't what I hoped it would be.

[Alessandro Vernet]

Got it Alexander, you or Gerrit will let me know if you can put together that
other example. And I didn't mean to downplay how disruptive changes in the
XForms engine can be! I hope that adding an `xf:setindex` after the instance
replacement will give you the behavior you need.

[Gerrit Boers]

Hi Alex,

I’m still testing and figuring out what’s going on and how to create an example but I’ve managed to pin it down to a nodeset replace.

You referred to this statement in the specification:

If items are added to the repeat collection, then the repeat index is set to the position of the last item added.

The issue I’m describing is not about adding items to the repeat but the index is set to the last item on a complete nodeset replacement. Is this indeed as intended? I can understand that the index should be set to the last item if items are added, but it doesn’t make sense to me to set the index to the last item if the complete nodeset is replaced by a submission.

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit,

I'll let you read section "10.4.1.3 Index Processing" of the spec (see link
below): it doesn't talk about instance replacement specifically, and I take
an instance replacement to be equivalent to removing all the nodes and then
adding all the nodes of the new document, which maps to doing the "If items
are removed…", and then the "If items are added…" from that paragraph.

https://www.w3.org/community/xformsusers/wiki/XForms_2.0#Index_Processing

Now, we can disagree with the spec ;). And if don't, we can implement
another behavior, or push for the spec to be changed. But then, what
behavior would you prefer? Keeping the index where it is, if possible?
Setting the index to the first item (if there is one, 0 otherwise)? How to
justify having a different behavior when an instance replacement is used vs.
when the same resulting instance is obtained through node deletion then
insertion?

[Gerrit Boers]

Hi Alex,

I fully agree that the specification is not clear on this issue and I’m also in favour of consistent behaviour of components. It would be great if the specification would be clear on this because the way it is working now there’s still some inconsistency because the first replace of the (empty) instance will have the index on the first item while all subsequent replacements will put the index on the last item.
Fortunately I managed to solve the issue by changing the code to store and restore the index position. Setting the index position explicitly (xforms-submit-done) also solved the issue that I had with the xxforms-nodeset-changed not firing.

Kind regards,

Gerrit

[Gerrit Boers]

It seems I was cheering prematurely as I’m running into problems again on further testing. The things is that I was assuming that the nodeset-changed events and index-changed events would be mutually exclusive thus allowing to distinguish user events from submission updates. In the new Orbeon this is no longer the case. How can I distinguish between a user changing the selection and the selection being changed by the system?

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit,

Indeed, maybe not. What is the use case for making such a distinction? That
is, what action would you like to take if users switch to another iteration
that you don't want to take if that switch is done because of, say, an
instance replacement?

[Gerrit Boers]

Hi Alex,

Here’s a screenshot of the application I’m working on:

It’s essentially a master-detail view with a search function and result view and below a detail view of the selected concept. When a user changes the selection in the result the detail should change accordingly, so far so good. The user can define a value set using the buttons in the detail view. After each action the search result is validated against the value set definition, this no longer works because the index is moved to the last position after instance replacement. Storing and restoring the index position does not work in this case because the user can navigate (using the links in the detail view) to a concept not in the search result, hence the need for a separate nodeset-changed event to distinguish user actions from system actions. The way it is now (Orbeon 2018) there’s no distinction anymore as the index-changed event will now always fire when the nodeset is replaced, this is a significant change of behaviour from the last version. IMHO the index-chaged event should not fire when a nodeset is replaced.

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit,

I am not sure I am following: if the "detail" part is updated when the
current selected line in the "master" is updated, wouldn't you want that to
happen if users change the selection, say with a click, but *also* if the
selection changes for some other reason, say a search returning new items?

[Gerrit Boers]

Hi Alex,

Sure, when the user changes the selection in the repeat the detail view should be updated. But while composing a value set in the detail view the the search result is validated against the value set, the content of the repeat does not change in this case just some item attribute indicating wether an item is in the set or not. The index should not change because the context in the detail view will be lost. Because the user can use the detail view to navigate to a concept not present in the repeat I cannot use a stored index position to restore the concept in detail view.
I guess I have to figure something out using a separate instance for the validation part of the items in the repeat.

Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit,

So if I understand you, as users type in fields inside the repeat, you are
submitting the whole instance, which comes back with some annotation
informing you about fields validity, and when that happens you don't want
the index be reset. Is that an accurate summary the problem?

[Gerrit Boers]

Hi Alex,

Yes, this is exactly the case.

Kind regards,

Gerrit

[Alessandro Vernet]

OK Gerrit, then I would try the following:

1. Before running the submission, call `xxf:index()` and save the current
index somewhere, in some node of an instance.
2. On `xforms-submit-done`, call `xf:setindex()` with the value you saved.

You'll let us know if this works.

https://doc.orbeon.com/xforms/xpath/extension-functions/extension-controls#xxf-index

[General Brainworking]

Hi Alex,

This will not work because the user can navigate (in the detail view) away from the concepts in the repeat so storing and restoring the index does not work. I came up with workaround by using a separate instance for the validation so I was able to solve this particular issue but then I ran into the next problem (see other mail).


Kind regards,

Gerrit

[Alessandro Vernet]

Hi Gerrit, I don't quite get what you mean by "the user can navigate away

from the concepts in the repeat so storing and restoring the index does not

work", but if you have a workaround (which we can discuss in another
thread), we can leave it at that ;).