Do you want people to use CP2K?
akohlmey
Here is one question to the CP2K developers team
that is bugging me and some other people for a while now:
Do you really want people to _use_ cp2k?
If not, please _tell_ us so that we don't
lose more time in trying to play catch-up
with compiler versions, libraries, unexpected
input file changes and don't need to suffer
every single time, when we try to compile it
on a new machine.
If yes, please take the following issues
under serious consideration:
- We need to have an official release! As much as it is
understandable, that many of you want to wait until you
have the perfect program, it is impossible to tell somebody
in a supercomputing center 'I need cp2k, but I need the version
of jan-33-2007, since that worked so well for regular QuickStep,
and please also compile feb-30-2007 because that was better for QM/
MM'...
A significant part of the functionality in cp2k is now quite
mature,
so why not pick a 'known-to-be-good(TM)' version, make a list of
working features, then create a branch on the cvs, make a (beta?)
release (2007-05.pl0) and apply only bug fixes to that branch and
make those available from time to time. If done with some
consideration
it would mean little additional work, but _much_ more consistent
behavior for the released code branch, which is exactly what users
need.
if you want people to use cp2k, you have to give them a chance.
- The compileability (portability?) of cp2k should be a priority, at
least on the release branch. It is especially difficult to port
cp2k
to many commonly used platforms at the major supercomputing centers
(US, Europe, Japan). It is a reality the many of the tools available
do not follow the latest greatest standards and sticking to a more
globally supported subset will allow of a greater exposure of cp2k
to high-end hardware and thus more ambitious and prestigious
calculations.
With the most extreme example being the libint interface (and
hybrid
functionals would actually be an application that needs to be run
on high-end hardware) and the fact that even a tool bundled with
and required to compile cp2k (forpar.x) chokes on it. This is a
point, where it is difficult to see the common sense in it.
Where do we get a g95 for a NEC-SX8, IBM BG/L, Cray XT3?
Do we really need a function pointer interface to C?
This is essentially an invitation for compilers to miscompile the
code and debugging becomes even more difficult as it is already.
- People that want to use the (great) features that are currently
under active development and that are willing to follow the cvs,
but are not developers, need to know what is happening.
Even the cp2k website says:
Using the code for production quality simulations is possible but
requires detailed knowledge about the active development.
So where do we _get_ this knowledge?
As an aside: in the last draft of this mail, there was a paragraph
here, suggesting to move the forum away from the disastrous
Berlios site where we can have something that works more like
a mailing list (i.e. you can quote previous messages). With the
new cp2k google group, this is no longer required. Thanks for that.
- Finally, if you want 'just users' to really put your code to use,
we need to have a some _real_ examples that produce 'good(TM)'
results, that people can follow them. The regtest directories are
not very helpful, as those inputs are engineered to test
reproducibility of previously implemented features.
And then some (simple) guidelines are needed on how to setup a
'normal' calculation, how to pick potentials, basis sets, force
fields,
avoid common problems, etc. so that people can actually start
working
on producing a good user's guide. In my opinion, a good user's
guide
is written by users (and not necessarily developers), thus 'early
adopter'
users need feedback on their inputs and the documentation they would
write.
'Help in exchange for Documentation' could be a motto for this.
Again, initially i would suggest here to use some wiki-style
web service (even berlios would provide this), but it seems the
solution in the google group would be more suitable, at least
for building an initial draft.
Certainly, some of you will now feel (a little?) offended, as you
have
put in all the hard work and made everything freely available, so why
should you bother? Right! You are totally entitled to say that, but
then please don't present cp2k as something that everybody can (and
should) use.
cp2k has obviously a lot of potential, but as long as only a few
people
can exploit it, it will stay a sideshow. If you _want_ people to use
it,
you have to occasionally make a step towards them.
Thanks for reading this far. We hope that you don't find this text
more thought provoking than inflammatory, and rather see it as a
conscientious effort by devoted users to help bringing cp2k into
mainstream use. Something that would benefit both, (current and
future) users and developers.
with best regards,
axel kohlmeyer.
supported by:
Mauro Boero
Guillaume Lamoureux
Nichols A. Romero
P.S.: The final edit is by me (axel), so any errors or
misunderstandings
are most likely my fault.
Teodoro Laino
I won't comment further otherwise it looks like just a discussion
between me and Axel.. while in the project there are
at least 16 people subscribed as developers..
> unexpected input file changes
This is somehow unavoidable. It's like a chess game.. it's difficult
to forecast a logical organization of keywords..
So.. as far as it is required, though I know that it can be quite
annoying for people, we have to do that...
Regarding all the rest I expressed my opinion several times... a
release is something doubtless needed..
But again is not something we can do quickly.. all peoples involved
in the project mostly agree with that..
so in the same way as we moved to googlegroups I'm certain that a
release will pop up one day...
cheers
Teo
Juerg Hutter
> Hi everybody!
>
> Here is one question to the CP2K developers team
> that is bugging me and some other people for a while now:
>
> Do you really want people to _use_ cp2k?
First of all, thanks for caring. I will take some time to
try to answer to your questions and comments. However,
these are my personal remarks and might vary from
other developers.
I want to start to say that your mail is rather polemic
and I suppose this was intentional in order to get
some of us to reply.
>
> If not, please _tell_ us so that we don't
> lose more time in trying to play catch-up
> with compiler versions, libraries, unexpected
> input file changes and don't need to suffer
> every single time, when we try to compile it
> on a new machine.
This doesn't really need an answer. Of course nobody
works on the code in order to prevent others to use it.
>
> If yes, please take the following issues
> under serious consideration:
>
> - We need to have an official release! As much as it is
> understandable, that many of you want to wait until you
> have the perfect program, it is impossible to tell somebody
> in a supercomputing center 'I need cp2k, but I need the version
> of jan-33-2007, since that worked so well for regular QuickStep,
> and please also compile feb-30-2007 because that was better for QM/
> MM'...
>
> A significant part of the functionality in cp2k is now quite
> mature,
> so why not pick a 'known-to-be-good(TM)' version, make a list of
> working features, then create a branch on the cvs, make a (beta?)
> release (2007-05.pl0) and apply only bug fixes to that branch and
> make those available from time to time. If done with some
> consideration
> it would mean little additional work, but _much_ more consistent
> behavior for the released code branch, which is exactly what users
> need.
> if you want people to use cp2k, you have to give them a chance.
To have a release has been discussed among the developers many times.
We even had once (some 4-5 years back) a date for a release. The main
problem I see is that nobody steps forward and takes the
responsability.
You also underestimate the work that comes with a release. First
the testing phase and then keeping up with the bugs.
I just want to make the comparison to CPMD where there was only
one release in 30 months since I stopped pushing them.
Maybe somebody from one of the large groups could take the job?
>
> - The compileability (portability?) of cp2k should be a priority, at
> least on the release branch. It is especially difficult to port
> cp2k
> to many commonly used platforms at the major supercomputing centers
> (US, Europe, Japan). It is a reality the many of the tools available
> do not follow the latest greatest standards and sticking to a more
> globally supported subset will allow of a greater exposure of cp2k
> to high-end hardware and thus more ambitious and prestigious
> calculations.
>
I don't think this is really our (the CP2K developers) problem. Except
for a few exceptions (I will comment on them in the next section)
CP2K uses the Fortran95 standard. The problem is much more with
the compiler builders. I assume by trying to target efficiently
the more and more complex CPUs they introduce new bugs with every
new release. It's then often our task to find and isolate those
problems. I can asure you that my group has spent a lot of time
hunting the bugs of IBM, Intel, NAG etc.
> With the most extreme example being the libint interface (and
> hybrid
> functionals would actually be an application that needs to be run
> on high-end hardware) and the fact that even a tool bundled with
> and required to compile cp2k (forpar.x) chokes on it. This is a
> point, where it is difficult to see the common sense in it.
> Where do we get a g95 for a NEC-SX8, IBM BG/L, Cray XT3?
> Do we really need a function pointer interface to C?
> This is essentially an invitation for compilers to miscompile the
> code and debugging becomes even more difficult as it is already.
>
The libint interface is only needed for the experimental HFX code.
The implementation was done such, that if you don't need the
functionality no problems should occur.
Currently this functionality is only available with the g95
compiler (it's promised for NAG and IBM).
To make this clear, HFX has been implemented in my group for
testing purposes. We have done this such that it costs us
the least amount of work. If this doesn't work for your
platform/compiler combinations we can't help you.
Again, we didn't choose this way because we thought it would be
cool to have a function pointer interface. This was the
optimal choice of tools for our needs. We made every step
necessary in order not to interfere with the stability of
the code.
> - People that want to use the (great) features that are currently
> under active development and that are willing to follow the cvs,
> but are not developers, need to know what is happening.
> Even the cp2k website says:
>
> Using the code for production quality simulations is possible but
> requires detailed knowledge about the active development.
>
> So where do we _get_ this knowledge?
>
This is certainly a valid point. The problem is that even within
the developers group it is difficult to get this information.
We more or less rely on the CVS for it. I hope we will be able
to improve this situation in the future.
> As an aside: in the last draft of this mail, there was a paragraph
> here, suggesting to move the forum away from the disastrous
> Berlios site where we can have something that works more like
> a mailing list (i.e. you can quote previous messages). With the
> new cp2k google group, this is no longer required. Thanks for that.
>
> - Finally, if you want 'just users' to really put your code to use,
> we need to have a some _real_ examples that produce 'good(TM)'
> results, that people can follow them. The regtest directories are
> not very helpful, as those inputs are engineered to test
> reproducibility of previously implemented features.
>
This has been discussed many times. Again, somebody has to step
forward and take up the task. It's not that we don't know this
or that we intentionally keep this information to ourselves.
Maybe the users from the bigger groups could start and maintain
a CP2K example repository?
> And then some (simple) guidelines are needed on how to setup a
> 'normal' calculation, how to pick potentials, basis sets, force
> fields,
> avoid common problems, etc. so that people can actually start
> working
> on producing a good user's guide. In my opinion, a good user's
> guide
> is written by users (and not necessarily developers), thus 'early
> adopter'
> users need feedback on their inputs and the documentation they would
> write.
> 'Help in exchange for Documentation' could be a motto for this.
Documentation is a problem that is on par with the release and the
examples. You have to find somebody to take the lead. These jobs
are not easy. I can tell you from my experience from CPMD that
only occasionally somebody will thank you for your efforts, but
many people will tell you more or less friendly what is (still)
wrong with what you provide.
BTW we are currently trying the third time in 5 years to get
some funding for such tasks from the EU. I fear that we will
again not succeed.
With other words a good part of the development of CP2K comes
from my personal time and my research money. It would be highly
appreaciated if friends could add more to the effort in the
future.
best regards
Juerg
fawzi
* release
we have discussed already several times about it, I was against a
release before the transition to the new input, but now I think that
it would be a good point to do it.
Obviously something that will remain with a release is that one should
*really* not add new features to it, otherwise the burden of merging
them in the head will get much bigger.
* compilability
compiler fault normally, and I don't see many ways to improve it
easily, we aleady have some cludges due to compiler problems, but I am
strongly against putting too much work working around compilers bugs.
About libint I think that Juerg already answered, and I personally
thnik that using the new standard interface was the right choice, even
if at the moment it gives headaches.
* documentation
With respect to the documentation it is something that should be done,
but is boring, anyway I have just checked in a first preliminary
version of a tutorial, and this before reading your post :).
In cp2k/doc/tutorialCp2k.html you can find it, I plan to complete it
next week or so and put it officially on the web...
So don't despair, obviously not everything will happen as fast as you
might like, but we know about the problems, but unfortunately nobody
of us really likes to do boring work.
If you want to help then I think that nobody is stopping you, but yes
as the developer group is spread around, everybody has its own
problems, it is not always easy...
Fawzi