New astropy.models sub-package!

[Thomas Robitaille]

Hi everyone,

This is just to let you know that Nadia Dencheva's pull request for
astropy.models, which is a generalized model-fitting framework, has
now been merged:

http://docs.astropy.org/en/latest/models/index.html

While we've reviewed the pull request, and tested out astropy.models
for a few test cases, what would be really helpful at this stage is if
anyone interested can update to the latest developer version of
astropy and really test out this sub-package and use it for 'real'
work.

Please report any issues that you come across, whether bugs, things
that are difficult to do, or anything that is unclear in the
documentation, so that we can make sure this sub-package is stable by
the time we do the 0.3 release!

Thanks to Nadia for a great contribution!

Cheers,
Tom

[Adrian Price-Whelan]

Awesome! Congrats.

I have to admit I didn't follow this PR *at all*, and I can't seem to find it in the PR history? Given that, I'm going to dump some comments and feel free to ignore them if they were discussed at length on github...

Most of my comments are just naming / consistency...

- Why Gauss1DModel and not Gaussian1DModel? It's not Cheb1DModel (instead of Chebyshev), so I vote we stick to long names.

- In the Gaussian model, why is the mean called xcen? If we use xsigma, shouldn't it be just mu (or xmu, why the 'x'?)? Either that, or I propose sigma -> std or stddev or something. In any case, I think xcen is cryptic...

- Related, why is amplitude spelled out but the other two kwargs abbreviated? My vote is to have 'amplitude', 'sigma', 'mu' or 'amplitude', 'stddev', 'mean' -- but *not* 'cen' or center...

- The model class has attribute 'paramdim', then 'parnames', then 'pset' -- we should really be consistent with our abbreviation of the word parameters or this is going to be confusing.

- For all functions like "Sky2Pix" or "RotateCelestial2Native", I propose we use the word "To" instead of the number "2" -- at quick glance, it may be confusing having so many 2's floating around when many of the models are names XXX2D, but this is a minor detail.

That's it for now -- looks fun though!

Thanks,

Adrian

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

// Adrian Price-Whelan -- Columbia Astronomy //

[Erik Bray]

On Tue, Apr 23, 2013 at 10:17 AM, Adrian Price-Whelan
<adri...@gmail.com> wrote:
> Awesome! Congrats.
>
> I have to admit I didn't follow this PR *at all*, and I can't seem to find
> it in the PR history? Given that, I'm going to dump some comments and feel
> free to ignore them if they were discussed at length on github...
>
> Most of my comments are just naming / consistency...
>
> - Why Gauss1DModel and not Gaussian1DModel? It's not Cheb1DModel (instead of
> Chebyshev), so I vote we stick to long names.
> - In the Gaussian model, why is the mean called xcen? If we use xsigma,
> shouldn't it be just mu (or xmu, why the 'x'?)? Either that, or I propose
> sigma -> std or stddev or something. In any case, I think xcen is cryptic...

I think the 'x' is for consistency with Gauss2DModel which has xcen,
ycen, xsigma, ysigma, etc.

I agree 'xcen' is cryptic though. I would prefer any of one of
'x_mean', 'x_mu', or 'x_center'. I think all of these should have
underscores after the 'x' and 'y'.

> - Related, why is amplitude spelled out but the other two kwargs
> abbreviated? My vote is to have 'amplitude', 'sigma', 'mu' or 'amplitude',
> 'stddev', 'mean' -- but *not* 'cen' or center...

I'm not sure I get your point here.

> - The model class has attribute 'paramdim', then 'parnames', then 'pset' --
> we should really be consistent with our abbreviation of the word parameters
> or this is going to be confusing.

I agree.

> - For all functions like "Sky2Pix" or "RotateCelestial2Native", I propose we
> use the word "To" instead of the number "2" -- at quick glance, it may be
> confusing having so many 2's floating around when many of the models are
> names XXX2D, but this is a minor detail.

I agree here too, but only +0.

[Adrian Price-Whelan]

On Tue, Apr 23, 2013 at 3:36 PM, Erik Bray <erik....@gmail.com> wrote:

On Tue, Apr 23, 2013 at 10:17 AM, Adrian Price-Whelan
<adri...@gmail.com> wrote:
> Awesome! Congrats.
>
> I have to admit I didn't follow this PR *at all*, and I can't seem to find
> it in the PR history? Given that, I'm going to dump some comments and feel
> free to ignore them if they were discussed at length on github...
>
> Most of my comments are just naming / consistency...
>
> - Why Gauss1DModel and not Gaussian1DModel? It's not Cheb1DModel (instead of
> Chebyshev), so I vote we stick to long names.
> - In the Gaussian model, why is the mean called xcen? If we use xsigma,
> shouldn't it be just mu (or xmu, why the 'x'?)? Either that, or I propose
> sigma -> std or stddev or something. In any case, I think xcen is cryptic...

I think the 'x' is for consistency with Gauss2DModel which has xcen,
ycen, xsigma, ysigma, etc.

I agree 'xcen' is cryptic though. I would prefer any of one of
'x_mean', 'x_mu', or 'x_center'.  I think all of these should have
underscores after the 'x' and 'y'.

I still don't think the x belongs in the 1D distribution -- one of the virtues of it being 1D is that there is no ambiguity to the "mean" of the distribution. In fact, if someone conceptually thinks about 1D distributions over z, for instance, then this is unnecessarily restrictive. Ok now I'm just being ridiculous :) ...but still, I think the parameters should be either ["mu", "sigma"] or ["mean", "stddev"].

 

> - Related, why is amplitude spelled out but the other two kwargs
> abbreviated? My vote is to have 'amplitude', 'sigma', 'mu' or 'amplitude',
> 'stddev', 'mean' -- but *not* 'cen' or center...

I'm not sure I get your point here.

My point is just that right now, 'xcen' is an abbreviation of 'x_center' or something, but 'amplitude' is not abbreviated...that's all.

 

> - The model class has attribute 'paramdim', then 'parnames', then 'pset' --
> we should really be consistent with our abbreviation of the word parameters
> or this is going to be confusing.

I agree.

> - For all functions like "Sky2Pix" or "RotateCelestial2Native", I propose we
> use the word "To" instead of the number "2" -- at quick glance, it may be
> confusing having so many 2's floating around when many of the models are
> names XXX2D, but this is a minor detail.

I agree here too, but only +0.

Well, at least it wasn't -0 :)

Thanks!

- Adrian

[Erik Tollerud]

The PR is at https://github.com/astropy/astropy/pull/493 - you
probably didn't find it because it's so far back...

I like most of Adrian's suggestions - Adrian, feel free to issue a PR
or add an issue about some or all of these (and probably do "@nden" to
get Nadia's attention on it). This is *not* intended to be a "final"
work by any means - the PR got merged partly because it's a lot easier
to make enhancements at this point against the main astropy repo, so
such improvements are definitely welcome!

(It's of course also fine to discuss it here, but I just wanted to
make it clear that this is *not* at all considered API-stable or
anything, so changes like this should easily and quickly get accepted
if there's agreement on them.)

On Tue, Apr 23, 2013 at 10:17 AM, Adrian Price-Whelan
<adri...@gmail.com> wrote:

Erik

[Adrian Price-Whelan]

Aha! Got it. Given that, I'll make these changes and issue a PR and we can discuss there.

Thanks!

- Adrian

[Thøger Rivera-Thorsen]

Just out of curiosity, why not avoid duplicating effort in the fitting
library by utlilizing existing code like e.g. lmfit? It seems very
similar in design to the current fitting functionality, from a
superficial look at the docs at least.

[Nadia Dencheva]

The goal was to create an extensible framework. It should be easy to add a custom fitter using an optimization algorithm, even one  which is not in scipy.

In addition, a separation of optimization algorithms, fit statistic and estimators would provide further flexibility. This is something that sherpa does very well. This has not been implemented in models yet but the current design makes it easy to add.

Nadia

email to astropy-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

// Adrian Price-Whelan -- Columbia Astronomy //

--
You received this message because you are subscribed to the Google Groups
"astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an

email to astropy-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

You received this message because you are subscribed to the Google Groups "astropy-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev+unsubscribe@googlegroups.com.

[Thøger Rivera-Thorsen]

I have been moving back and forth between Sherpa and lmfit myself. I agree that Sherpa is very powerful and flexible, but also quite, hmm, opaque (plus, I have had problems every single time I have installed it). If something could be made that would have the power and extensibility of Sherpa but the transparency of lmfit, I would be one very happy man.

email to astropy-dev...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

// Adrian Price-Whelan -- Columbia Astronomy //

--
You received this message because you are subscribed to the Google Groups
"astropy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an

email to astropy-dev...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "astropy-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to astropy-dev...@googlegroups.com.

[Erik Tollerud]

Can you clarify what you mean by "transparency of lmfit"? You mean
the interface, or how it's implemented?

If the latter, it's not inconceivable to write a Fitter class for the
models framework that actually uses lmfit (it would then be an
optional dependency for astropy).




On Thu, Apr 25, 2013 at 9:59 AM, Thøger Rivera-Thorsen

Erik

[Thøger Rivera-Thorsen]

As a programmer I am entirely self-taught and know almost nothing about
"good practice" when it comes to writing libraries and stuff. But I have
tried to write a wrapper for Sherpa and one for lmfit for an app I'm
working on and found the latter much easier and more flexible (but,
admittedly, not as powerful) as Sherpa, which on top of it tends to
break iPython tracebacks, have a very poorly documented object-oriented
interface etc., and although it kinda-sorta works, scripting the
interactive interface feels a little dirty and like asking for trouble.

In lmfit, it is clearer which objects are actually returned from a given
functiojn call, and it seems more clear when you are instantiating a
class and when you are calling a function, something that Sherpa seems
to try to hide because they want a matlab-like interactive command
interface. So lmfit in general feels more pythonic to me, with the
python interface to sherpa seeming like a bit of an aftertjhought
(which, of course, it also technically is).

On 2013-04-25 17:37, Erik Tollerud wrote:
> Can you clarify what you mean by "transparency of lmfit"? You mean
> the interface, or how it's implemented?
>
> If the latter, it's not inconceivable to write a Fitter class for the
> models framework that actually uses lmfit (it would then be an
> optional dependency for astropy).
>
>
>
>

> On Thu, Apr 25, 2013 at 9:59 AM, Th�ger Rivera-Thorsen

> <thoge...@gmail.com> wrote:
>> I have been moving back and forth between Sherpa and lmfit myself. I agree
>> that Sherpa is very powerful and flexible, but also quite, hmm, opaque
>> (plus, I have had problems every single time I have installed it). If
>> something could be made that would have the power and extensibility of
>> Sherpa but the transparency of lmfit, I would be one very happy man.
>>
>> On 2013-04-25 15:50, Nadia Dencheva wrote:
>>
>> The goal was to create an extensible framework. It should be easy to add a
>> custom fitter using an optimization algorithm, even one which is not in
>> scipy.
>>
>> In addition, a separation of optimization algorithms, fit statistic and
>> estimators would provide further flexibility. This is something that sherpa
>> does very well. This has not been implemented in models yet but the current
>> design makes it easy to add.
>>
>> Nadia
>>
>>

>> On Thu, Apr 25, 2013 at 7:16 AM, Th�ger Rivera-Thorsen

[Christoph Deil]

Hi Nadia,

would it be possible to do a Google hangout soon to discuss the astropy modelling / fitting API and how to proceed from what is in master now?

I'd like to help as much as I can with coding, or where license permits, adapt the good parts of existing Python fitting packages like Sherpa or lmfit for astropy, but I'm not sure where best to start.

Besides the question of how to restructure / rename things, there's also the question of how to get currently missing functionality like e.g. model convolution and integration or error estimation methods.

Other things that I would use but maybe are out of scope for astropy are e.g. unbinned fitting or fitting N-dimensional datasets.

Some of these things might be very difficult to add later without changing the API, so I think it would be good to do as much as time permits now before the astropy 0.3 release and it becomes harder to change things.

Thoger,

I've been using Sherpa a lot and tried out lmfit and iminuit and other modelling / fitting packages a bit and wanted to add some comments to what you said.

One thing that should be noted is that Sherpa has the concept of a  "session" for interactive work.

So there is a current data set, current model, current PSF and effective area, current fit statistic and current optimizer, … which is super-convenient for interactive work because it let's you do one thing after the other without having to remember how to hand things around and how everything is connected.

The design of Sherpa (object-oriented with procedural user interface and session for interactive work) is described pretty well in this proceeding:

http://conference.scipy.org/proceedings/scipy2009/paper_8/full_text.pdf

In my opinion the Sherpa object-oriented design with loosely coupled data, model, fit statistic, optimiser classes is very well thought out (one thing I currently don't like in the astropy API is that the fit statistic and optimiser are combined in a "Fitter" class), as is their user interface for interactive fitting sessions at the ipython prompt, so I don't think the python interface to Sherpa is an afterthought as you put it.

But I did have similar problems with Sherpa as you mentioned. E.g. Sherpa is very hard to build and install from source, which means you have to get their binary distribution and then it is very hard  to install other Python packages with C extensions like astropy to be able to use it with their Python.

And all their documentation is for the procedural UI, so it was too hard to figure out the object-oriented API from the code itself and I used the procedural UI for an analysis pipeline, which was a bit error prone because of the global session state. And I tried to add my (quite specialised) user-defined data types and fit statistics to Sherpa and in the end I didn't manage to use it for my application because their interfaces weren't flexible enough.

So let's see how modelling and fitting in astropy is evolving … I do think the experience (and code if licence permits for core parts like model convolution or integration, or error estimation methods … which are hard to implement efficiently) of the Sherpa devs is invaluable here to get a great API and all missing functionality in astropy quickly.

Christoph

On Apr 25, 2013, at 6:08 PM, Thøger Rivera-Thorsen <thoge...@gmail.com> wrote:

As a programmer I am entirely self-taught and know almost nothing about "good practice" when it comes to writing libraries and stuff. But I have tried to write a wrapper for Sherpa and one for lmfit for an app I'm working on and found the latter much easier and more flexible (but, admittedly, not as powerful) as Sherpa, which on top of it tends to break iPython tracebacks, have a very poorly documented object-oriented interface etc., and although it kinda-sorta works, scripting the interactive interface feels a little dirty and like asking for trouble.

In lmfit, it is clearer which objects are actually returned from a given functiojn call, and it seems more clear when you are instantiating a class and when you are calling a function, something that Sherpa seems to try to hide because they want a matlab-like interactive command interface. So lmfit in general feels more pythonic to me, with the python interface to sherpa seeming like a bit of an aftertjhought (which, of course, it also technically is).



On 2013-04-25 17:37, Erik Tollerud wrote:

Can you clarify what you mean by "transparency of lmfit"?  You mean
the interface, or how it's implemented?

If the latter, it's not inconceivable to write a Fitter class for the
models framework that actually uses lmfit (it would then be an
optional dependency for astropy).

On Thu, Apr 25, 2013 at 9:59 AM, Thøger Rivera-Thorsen
<thoge...@gmail.com> wrote:

I have been moving back and forth between Sherpa and lmfit myself. I agree
that Sherpa is very powerful and flexible, but also quite, hmm, opaque
(plus, I have had problems every single time I have installed it). If
something could be made that would have the power and extensibility of
Sherpa but the transparency of lmfit, I would be one very happy man.

On 2013-04-25 15:50, Nadia Dencheva wrote:

The goal was to create an extensible framework. It should be easy to add a
custom fitter using an optimization algorithm, even one  which is not in
scipy.

In addition, a separation of optimization algorithms, fit statistic and
estimators would provide further flexibility. This is something that sherpa
does very well. This has not been implemented in models yet but the current
design makes it easy to add.

Nadia

On Thu, Apr 25, 2013 at 7:16 AM, Thøger Rivera-Thorsen

--
Erik

[Thøger Emil Rivera-Thorsen]

I do not know much about under-the-hood software design, so I definitely believe that Sherpas design is good. I also do like the simplicity and ease-of-use for the procedural interface, but I almost always switch to scripting and/or programming when I'm past the initial data exploration (for the sake of reproducibility), and the procedural interface is less suited for that, and the OO interface is... Well, largely undocumented and hard to guess. And it seems like our frustrations have been of almost identical character.

I do not know whether it is a good design choice or not, technically, the way lmfit returns a Minimizer object and changes a Parameters object in-place. But what I know is that it was quick, easy and transparent to figure out *what* lmfit returns and changes in which cases, which makes it easy to write a simple wrapper for it for my program.

Besides, one thing I immediately liked about lmfit (which also applies to simple scipy omtimizatio, I guess) is the way it is easy to feed it pretty much any old python callable as the function to minimize (for example, I have made an implementation of Tepper Garcias Voigt profile which I haven't found anywhere else in a fitting program). It may not be super hard to feed a custom model to Sherpa, but it was definitely not immediately clear how.� Can I get that transparency of lmfit in Astropy, then it is fine for me if Sherpa code and/or design philosophy is used extensively.

I really didn't mean to make it sound like I'm belittling Sherpa, I have been using it quite a lot since I discovered it around a year ago. Especially, its optimization methods and error estimation methods seem much more powerful that what I've found elsewhere. But venturing outside the interactive session is quite painful.




On 04/25/2013 10:54 PM, Christoph Deil wrote:

[snip]

In my opinion the Sherpa object-oriented design with loosely coupled data, model, fit statistic, optimiser classes is very well thought out�(one thing I currently don't like in the astropy API is that the fit statistic and optimiser are combined in a "Fitter" class), as is their user interface for interactive fitting sessions at the ipython prompt, so I don't think the python interface to Sherpa is an afterthought as you put it.

But I did have similar problems with Sherpa as you mentioned. E.g. Sherpa is very hard to build and install from source, which means you have to get their binary distribution and then it is very hard �to install other Python packages with C extensions like astropy to be able to use it with their Python.

And all their documentation is for the procedural UI, so it was too hard to figure out the object-oriented API from the code itself and I used the procedural UI for an analysis pipeline, which was a bit error prone because of the global session state. And I tried to add my (quite specialised) user-defined data types and fit statistics to Sherpa and in the end I didn't manage to use it for my application because their interfaces weren't flexible enough.

So let's see how modelling and fitting in astropy is evolving � I do think the experience (and code if licence permits for core parts like model convolution or integration, or error estimation methods � which are hard to implement efficiently) of the Sherpa devs is invaluable here to get a great API and all missing functionality in astropy quickly.

Christoph

On Apr 25, 2013, at 6:08 PM, Th�ger Rivera-Thorsen <thoge...@gmail.com> wrote:

As a programmer I am entirely self-taught and know almost nothing about "good practice" when it comes to writing libraries and stuff. But I have tried to write a wrapper for Sherpa and one for lmfit for an app I'm working on and found the latter much easier and more flexible (but, admittedly, not as powerful) as Sherpa, which on top of it tends to break iPython tracebacks, have a very poorly documented object-oriented interface etc., and although it kinda-sorta works, scripting the interactive interface feels a little dirty and like asking for trouble.

In lmfit, it is clearer which objects are actually returned from a given functiojn call, and it seems more clear when you are instantiating a class and when you are calling a function, something that Sherpa seems to try to hide because they want a matlab-like interactive command interface. So lmfit in general feels more pythonic to me, with the python interface to sherpa seeming like a bit of an aftertjhought (which, of course, it also technically is).



On 2013-04-25 17:37, Erik Tollerud wrote:

Can you clarify what you mean by "transparency of lmfit"? �You mean

the interface, or how it's implemented?

If the latter, it's not inconceivable to write a Fitter class for the
models framework that actually uses lmfit (it would then be an
optional dependency for astropy).

On Thu, Apr 25, 2013 at 9:59 AM, Th�ger Rivera-Thorsen
<thoge...@gmail.com> wrote:

I have been moving back and forth between Sherpa and lmfit myself. I agree
that Sherpa is very powerful and flexible, but also quite, hmm, opaque
(plus, I have had problems every single time I have installed it). If
something could be made that would have the power and extensibility of
Sherpa but the transparency of lmfit, I would be one very happy man.

On 2013-04-25 15:50, Nadia Dencheva wrote:

The goal was to create an extensible framework. It should be easy to add a

custom fitter using an optimization algorithm, even one �which is not in

scipy.

In addition, a separation of optimization algorithms, fit statistic and
estimators would provide further flexibility. This is something that sherpa
does very well. This has not been implemented in models yet but the current
design makes it easy to add.

Nadia

On Thu, Apr 25, 2013 at 7:16 AM, Th�ger Rivera-Thorsen
<thoge...@gmail.com> wrote:

Just out of curiosity, why not avoid duplicating effort in the fitting
library by utlilizing existing code like e.g. lmfit? It seems very similar
in design to the current fitting functionality, from a superficial look at
the docs at least.




On 2013-04-25 04:00, Erik Tollerud wrote:

The PR is at https://github.com/astropy/astropy/pull/493 - you
probably didn't find it because it's so far back...

I like most of Adrian's suggestions - Adrian, feel free to issue a PR
or add an issue about some or all of these (and probably do "@nden" to

get Nadia's attention on it). �This is *not* intended to be a "final"

--
Erik

�
�

[Christoph Deil]

On Apr 25, 2013, at 11:47 PM, Thøger Emil Rivera-Thorsen <thoge...@gmail.com> wrote:

I do not know much about under-the-hood software design, so I definitely believe that Sherpas design is good. I also do like the simplicity and ease-of-use for the procedural interface, but I almost always switch to scripting and/or programming when I'm past the initial data exploration (for the sake of reproducibility), and the procedural interface is less suited for that, and the OO interface is... Well, largely undocumented and hard to guess. And it seems like our frustrations have been of almost identical character.

I do not know whether it is a good design choice or not, technically, the way lmfit returns a Minimizer object and changes a Parameters object in-place. But what I know is that it was quick, easy and transparent to figure out *what* lmfit returns and changes in which cases, which makes it easy to write a simple wrapper for it for my program.

Besides, one thing I immediately liked about lmfit (which also applies to simple scipy omtimizatio, I guess) is the way it is easy to feed it pretty much any old python callable as the function to minimize (for example, I have made an implementation of Tepper Garcias Voigt profile which I haven't found anywhere else in a fitting program). It may not be super hard to feed a custom model to Sherpa, but it was definitely not immediately clear how.  Can I get that transparency of lmfit in Astropy, then it is fine for me if Sherpa code and/or design philosophy is used extensively.

I really didn't mean to make it sound like I'm belittling Sherpa, I have been using it quite a lot since I discovered it around a year ago. Especially, its optimization methods and error estimation methods seem much more powerful that what I've found elsewhere. But venturing outside the interactive session is quite painful.

One reason I (and I think many others) use the Sherpa procedural UI a lot is that fitting / modelling is often a very interactive process:

- fit doesn't converge … fix this parameter first … try different starting values …

- ok, fit did converge … look at residuals in ds9 … too large, try more complex model, ...

Of course pipeline analysis from scripts it also very common and there using objects instead of a global session is cleaner as you said.

For astropy I think maybe we should try to get a clean object-oriented API first.

This will probably be quite verbose and not very suited for interactive use at the IPython prompt,

but I think the easy-to-use, possibly session-based procedural UI can be added as a second step and we don't have to do it now (this is just a guess)?

Christoph

On 04/25/2013 10:54 PM, Christoph Deil wrote:

[snip]

In my opinion the Sherpa object-oriented design with loosely coupled data, model, fit statistic, optimiser classes is very well thought out (one thing I currently don't like in the astropy API is that the fit statistic and optimiser are combined in a "Fitter" class), as is their user interface for interactive fitting sessions at the ipython prompt, so I don't think the python interface to Sherpa is an afterthought as you put it.

But I did have similar problems with Sherpa as you mentioned. E.g. Sherpa is very hard to build and install from source, which means you have to get their binary distribution and then it is very hard  to install other Python packages with C extensions like astropy to be able to use it with their Python.

And all their documentation is for the procedural UI, so it was too hard to figure out the object-oriented API from the code itself and I used the procedural UI for an analysis pipeline, which was a bit error prone because of the global session state. And I tried to add my (quite specialised) user-defined data types and fit statistics to Sherpa and in the end I didn't manage to use it for my application because their interfaces weren't flexible enough.

So let's see how modelling and fitting in astropy is evolving … I do think the experience (and code if licence permits for core parts like model convolution or integration, or error estimation methods … which are hard to implement efficiently) of the Sherpa devs is invaluable here to get a great API and all missing functionality in astropy quickly.

Christoph

On Apr 25, 2013, at 6:08 PM, Thøger Rivera-Thorsen <thoge...@gmail.com> wrote:

As a programmer I am entirely self-taught and know almost nothing about "good practice" when it comes to writing libraries and stuff. But I have tried to write a wrapper for Sherpa and one for lmfit for an app I'm working on and found the latter much easier and more flexible (but, admittedly, not as powerful) as Sherpa, which on top of it tends to break iPython tracebacks, have a very poorly documented object-oriented interface etc., and although it kinda-sorta works, scripting the interactive interface feels a little dirty and like asking for trouble.

In lmfit, it is clearer which objects are actually returned from a given functiojn call, and it seems more clear when you are instantiating a class and when you are calling a function, something that Sherpa seems to try to hide because they want a matlab-like interactive command interface. So lmfit in general feels more pythonic to me, with the python interface to sherpa seeming like a bit of an aftertjhought (which, of course, it also technically is).



On 2013-04-25 17:37, Erik Tollerud wrote:

Can you clarify what you mean by "transparency of lmfit"?  You mean

the interface, or how it's implemented?

If the latter, it's not inconceivable to write a Fitter class for the
models framework that actually uses lmfit (it would then be an
optional dependency for astropy).

On Thu, Apr 25, 2013 at 9:59 AM, Thøger Rivera-Thorsen
<thoge...@gmail.com> wrote:

I have been moving back and forth between Sherpa and lmfit myself. I agree
that Sherpa is very powerful and flexible, but also quite, hmm, opaque
(plus, I have had problems every single time I have installed it). If
something could be made that would have the power and extensibility of
Sherpa but the transparency of lmfit, I would be one very happy man.

On 2013-04-25 15:50, Nadia Dencheva wrote:

The goal was to create an extensible framework. It should be easy to add a

custom fitter using an optimization algorithm, even one  which is not in

scipy.

In addition, a separation of optimization algorithms, fit statistic and
estimators would provide further flexibility. This is something that sherpa
does very well. This has not been implemented in models yet but the current
design makes it easy to add.

Nadia

On Thu, Apr 25, 2013 at 7:16 AM, Thøger Rivera-Thorsen
<thoge...@gmail.com> wrote:

Just out of curiosity, why not avoid duplicating effort in the fitting
library by utlilizing existing code like e.g. lmfit? It seems very similar
in design to the current fitting functionality, from a superficial look at
the docs at least.




On 2013-04-25 04:00, Erik Tollerud wrote:

The PR is at https://github.com/astropy/astropy/pull/493 - you
probably didn't find it because it's so far back...

I like most of Adrian's suggestions - Adrian, feel free to issue a PR
or add an issue about some or all of these (and probably do "@nden" to

get Nadia's attention on it).  This is *not* intended to be a "final"

--
Erik