Automatic validation using Arelle command line

[Bengt Rodehav]

I'm trying to put together an automatic process as follows:

• A user uploads data in Excel format
  
• We create an XBRL file from the Excel data
• We validate the created XBRL file using Arelle command line
• If the validation passed, then the user can download the XBRL file
• If the validation failed, then no download will be possible. Instead the validation errors shall be shown.

Although I have managed to validate the XBRL file using Arelle command line, I haven't found a way to programmatically verify whether the validation passed or not.

At first I thought I could just look at the exit code from arelleCmlLine.exe to see whether the validation succeeded or not. However, it seems like the exit code is always 0 regardless if the validation succeeded or not.

I strongly suggest that this is changed. You could introduce a command line option that causes arelleCmlLine.exe to use a non-zero exit code on validation failures. Alternatively, the exit code could be configurable and use 0 as default (as today) if you want to be backwards compatible.

I then thought I could use the log output to determine whether en error occurred or not. So I set the log level to "error" and check if the log is empty or not. This seemed to work quite well until I specified the "eba" disclosure system" (this is for EBA's Asset Encumbrance). This is necessary otherwise the formula validation does not occur (probably due to the filing indicators).

It turns out that in order to see the validation failures from the EBA formulas I have to use at least logging level "info". However then I also get a lot of informational stuff which makes it impossible for me to (programmatically) determine whether the validation failed or not.

I'm currently at lost. I thought my biggest problem would be how to make the users understand the cryptic validation error messages but as it stands I cannot even determine whether the XBRL validation succeeded or not.

This was very surprising to me. I believe using Arelle as the validation engine in automatic processes is a common scenario that must be supported.

Have I got this all wrong? Can someone advise me regarding this?

[Ryan]

Hi Bengt, 

The lead developer monitors this list sporadically, so you will have the best luck with feature requests or 

improvements by signing-up and submitting them on the Arelle issue tracker: https://arelle.atlassian.net/browse/ARELLE/

Regarding the message-level of formula failures, try adding the option `--formulaUnsatisfiedAsserError` to your Arelle command line. 

This should cause failed formula assertions to be reported as "error" rather than "info".

On detecting validation "success": 

I've always done subsequent processing of the Arelle validation messages in order to determine

success or failure. You may be able get by using the --logCodeFilter="..." and --logLevelFilter="..."

options which allow you to filter the messages by code and level using a regular expression.

To perform further processing you should output the validation log in a structured format like xml or json.

These formats will give you much greater detail, including line-numbers, xpointer references, and

breakdowns of the elements causing an error.

To output the messages to an xml file use the option:

--logFile="validation-log.xml" 

Be sure you use a file extension of ".xml" or the log with be output as plain text.

You can also start Arelle in its web-server mode (with option --webserver=localhost:8080)

and then use the web api to validate files and receive the validation messages as json or xml. 

Details on the web-api are self-hosted by the server at http://localhost:8080/help .

Examples of validating with the web-api from java are in: https://github.com/Arelle/Arelle/blob/master/arelle/examples/

A final option is extending Arelle programmatically from Python. There is a minimal example of writing a custom

Arelle controller and log-handler at: https://github.com/Arelle/Arelle/blob/master/arelle/examples/CustomLogger.py

I've used this technique with great success, but it does require a willingness to dig through the Arelle codebase.

[Bengt Rodehav]

Thanks a lot for your detailed and quick answer. I will try your suggestions ASAP.

/Bengt

--

---
You received this message because you are subscribed to a topic in the Google Groups "Arelle-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/arelle-users/r-bA3xGkMTo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to arelle-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Victor Berg]

Hi Ryan. 

My name is Victor and I am a College of Bengt. 

I tried to do as you told and singed up to the Arelle Issue tracker. But I cannot create an Issue and there is no way to contact the Admin...? 

What should I do? I will post my issue here in a separate thread for now.

Thanks

Victor

[Herm Fischer]

Victor,

I added you to the users group on Atlasssian (manually).  Something changed in their system that causes new accounts to need the users group added manually.  You should be able to create issues now, please try it.

    Herm

--

---
You received this message because you are subscribed to the Google Groups "Arelle-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arelle-users...@googlegroups.com.