Hi Matteo !
I've been wishing for times to write something about ENTSOE TP but didn't take the time to so far.
Thank you for providing such thoughts and tools.
Transparency platform is a great achievement that should be reinforced.
Some of its weaknesses and the amount of supplementary work they encourage downstream cause a little harm.
The short term continuous updates you mention in your readme is a direct consequence of 543/2013 policy.
In 16th article, §2 states that required information are published at after x or strictly after x. It encourages operators to provide data at their own pace (in France, every 15 minutes, in Switzerland, D+5)
We need a more robust protocol to expect data at precise times (and I really like every 15 minutes => electricitymap).
"The platform has been created with the aim of providing timely
information to markets and not to provide a consistent and reliable
source of information for analysts and scientists".
I may like to add "and from a policy perspective, not from data governance practice" to your readme.
I may also disagree about this strict opposition between market and scientists.
As any data usage, it improves data quality with consumers (including scientists and third parties) stumbling at every step, even if initially designed for markets.
Furthermore, the TP lacks of consistency in regard of the policy itself.
I currently spend my time discussing with several operators in different countries because of their lack of required data.
That's really time consuming to compare between generation units list and this section, report to operators then (not to mention on my free time sometimes)
But that's really simple to do as well. ENTSOE should actually observe data, expect inconsistencies and automatically report them to operators.
I hope this initiative could make the use of platform better and free more data about generation actual data.