How about cleaning up for Eclipse PyDev?

[Joachim Durchholz]

It would be nice to be able to work on SymPy using Eclipse.

Current state of things is that I'm getting ~50,000 errors/warnings. I
tried cleaning this up a bit with some hopefully uncontroversial changes
- fixing the E30x warnings (too many/few empty lines between
class/function declarations) dropped that number by ~3,000, which is a
start but obviously no more than that.

I'm considering configuring PyDev to do somewhat less strict PEP-8
checking, but it didn't work (yet).
Also, this might mean adding a .project file or some other way of
helping people configure their PyDev into usefulness, so I'm not sure
what the best way ahead is.

My current working mode is to fix one kind of warning at a time, and do
a mega commit for that. This makes it easy to fix those things that can
be fixed in a mechanistic way, and it also makes it easy to review the
pull requests.
On the minus side, the commits can get pretty large, and they touch
everybody's work, so they should not sit unmerged for very long.

Thoughts? Opinions?

Regards,
Jo

[Sudhanshu Mishra]

Hey Joachim,

This is some good work. There's also an online service to monitor it. See this https://landscape.io/github/sympy/sympy/ 

Regards

Sudhanshu Mishra

www.sudhanshumishra.in

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscribe@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/549EAF02.9070806%40durchholz.org.
For more options, visit https://groups.google.com/d/optout.

[Jason Moore]

There is a PR for pep 8 changes already here: https://github.com/sympy/sympy/pull/8538

Jason
moorepants.info
+01 530-601-9791

To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAFW_KCQk9gvj4VqcesFT1PTiefSkZVaLAv7jN6DJBoSWdXt0ig%40mail.gmail.com.

[Francesco Bonazzi]

I think it would be nice to make tests easily runnable with PyDev's debugger. Unfortunately, SymPy's tests being non-standard means that PyDev is not able to start them.

I have prepared some script like this:

from sympy. <submodule> .tests import test_<something>

for i in dir(test_<something>):
    if not i.startswith("test_"):
        continue
    getattr(test_<something>, i)()

I think the real advantage of Eclipse is its debugger, and easy-to-use breakpoints in the source code. I also usually install Vrapper, which emulates the commands by Vim.

Anyways, I recently realized that PyCharm, which used to be a paid proprietary Python IDE, is now open source. I suggest to use PyCharm instead of Eclipse+PyDev, it works much better. PyCharm gets a lot less false warnings than Eclipse+PyDev. I used to write my SymPy contribution with Eclipse+PyDev+Vrapper, but now I definitely changed to PyCharm.

Both PyCharm and PyDev can detect Python tests written using the standard Python unit testing systems, unfortunately they don't recognize SymPy's tests. PyCharm recognizes the doctests, though, which is really comfortable. It would be nice to run the tests by simply click on the file in PyCharm's/Eclipse's file browser. That's my major worry.

[Aaron Meurer]

I personally don't use PEP 8 style warnings in my editor because I
always write code that is PEP 8 compliant, or if I don't, it's on
purpose. I do use pyflakes warnings, though, which warn about unused
variables and undefined variables. Fixing those is worthwhile, but in
my opinion, trying to fix whitespace issues is futile and a waste of
time. The pull request will be impossible to keep in a merged state,
and will become the source of much bikeshedding. And at the end, of
the day, does it really matter if there are one, two, or three spaces
between classes and functions?

Aaron Meurer

[Joachim Durchholz]

Am 27.12.2014 um 20:57 schrieb Aaron Meurer:
> I personally don't use PEP 8 style warnings in my editor because I
> always write code that is PEP 8 compliant, or if I don't, it's on
> purpose.

That may be the case, but I have seen several pieces of code where PEP8
compliance wasn't given more than a passing nod. Obviously not your
code, I assume (though I doubt that being PEP8-compliant in the way you
describe would border on the superhuman).

Still, other people will not apply PEP8 in the same way as you do.
Actually I thought of myself as pretty PEP8-compliant, but a quick check
revealed that I'm nowhere near that.
From that, I'm taking home that claims of compliance to any ruleset
need to be validated (actually just like any claim).
I'm also taking home that at least for people new to Python, running
pep8.py on a few occasions will help them write more idiomatic Python
code, even if only on a very superficial level. It's still valuable
because it will take stylistic matters off the mind of the more
experienced Pythonistas here, because they'll simply say "please run
your code through pep8.py and make sure that no unintentional violations
remain, that makes it easier on my eyes and we can then fix the really
important issues".

> I do use pyflakes warnings, though, which warn about unused
> variables and undefined variables. Fixing those is worthwhile, but in
> my opinion, trying to fix whitespace issues is futile and a waste of
> time.

I can see that point. Actually I have a lot of sympathy for it.

> And at the end, of
> the day, does it really matter if there are one, two, or three spaces
> between classes and functions?

Surprisingly enough, it does.

I did that empty-line correction just as a way to test my new PyDev
installation and see how easy it would be to do any clean-up tasks with
it. I certainly didn't expect any noticeable changes in code quality.

However, when looking at the code, I found it noticeably easier to
navigate it.

> The pull request will be impossible to keep in a merged state,

Not that I'd say "impossible", but yes it would be good to decide quickly.

> and will become the source of much bikeshedding.

Not really. The C crowd have their One True Brace Style to bikeshed
about, we have PEP 8.

Though I wouldn't take "bikeshedding" as a "no can do". In fact almost
any project in C land has decided on a specific color for their
bikeshed, and moved on to other issues.
So, yes, bikeshedding, but it quickly passes. Not a serious issue in the
long term.

[Joachim Durchholz]

Am 27.12.2014 um 17:33 schrieb Francesco Bonazzi:
> I think it would be nice to make tests easily runnable with PyDev's
> debugger. Unfortunately, SymPy's tests being non-standard means that PyDev
> is not able to start them.

Just run bin/test with the proper parameters.
Avoid options that cause bin/test to start the tests in a subprocess.

Also, you *should* be able to start the test programs directly.

Either way, you need to set up a proper PYTHONPATH so that the test will
find the sympy module. It took me a bit of fiddling to get everything
right, but I got it to work eventually, and I think it gets easier once
you know which setting means exactly what.
Alternatively, a good set of instructions would be helpful. Since I'm
currently setting up a new Eclipse install and getting SymPy to work on
it, I might actually write something :-)

> I think the real advantage of Eclipse is its debugger, and easy-to-use
> breakpoints in the source code.

I don't know PyCharm, so I can't compare.

> Anyways, I recently realized that PyCharm, which used to be a paid
> proprietary Python IDE, is now open source.

Ah, that's interesting.
For others, I fear - I'm too entrenched with Eclipse right now and would
not want to learn yet another IDE.

> I suggest to use PyCharm
> instead of Eclipse+PyDev, it works much better. PyCharm gets a lot less
> false warnings than Eclipse+PyDev.

PyDev *claims* that you can configure the warning.
I haven't manage to get that to work yet, unfortunately, so I don't
really know whether that point stands or not.

> Both PyCharm and PyDev can detect Python tests written using the standard
> Python unit testing systems, unfortunately they don't recognize SymPy's
> tests.

SymPy should become more standardized in that respect, I think.
But that's a separate discussion.

[Aaron Meurer]

On Sat, Dec 27, 2014 at 3:32 PM, Joachim Durchholz <j...@durchholz.org> wrote:
> Am 27.12.2014 um 17:33 schrieb Francesco Bonazzi:
>>
>> I think it would be nice to make tests easily runnable with PyDev's
>> debugger. Unfortunately, SymPy's tests being non-standard means that PyDev
>> is not able to start them.
>
>
> Just run bin/test with the proper parameters.
> Avoid options that cause bin/test to start the tests in a subprocess.

This is enabled by default. You'll have to use --no-subprocess to
avoid it. Without starting in a subprocess, it will be harder to debug
issues caused by hash randomization (Python 3.3+), or hash
randomization won't be enabled at all (Python 2 or Python 3.2).

>
> Also, you *should* be able to start the test programs directly.
>
> Either way, you need to set up a proper PYTHONPATH so that the test will
> find the sympy module. It took me a bit of fiddling to get everything right,
> but I got it to work eventually, and I think it gets easier once you know
> which setting means exactly what.
> Alternatively, a good set of instructions would be helpful. Since I'm
> currently setting up a new Eclipse install and getting SymPy to work on it,
> I might actually write something :-)

Our tests should work with py.test too, so if there is support for
that you can use it.

>
>> I think the real advantage of Eclipse is its debugger, and easy-to-use
>> breakpoints in the source code.
>
>
> I don't know PyCharm, so I can't compare.
>
>> Anyways, I recently realized that PyCharm, which used to be a paid
>> proprietary Python IDE, is now open source.
>
>
> Ah, that's interesting.
> For others, I fear - I'm too entrenched with Eclipse right now and would not
> want to learn yet another IDE.
>
>> I suggest to use PyCharm
>>
>> instead of Eclipse+PyDev, it works much better. PyCharm gets a lot less
>> false warnings than Eclipse+PyDev.
>
>
> PyDev *claims* that you can configure the warning.
> I haven't manage to get that to work yet, unfortunately, so I don't really
> know whether that point stands or not.
>
>> Both PyCharm and PyDev can detect Python tests written using the standard
>> Python unit testing systems, unfortunately they don't recognize SymPy's
>> tests.
>
>
> SymPy should become more standardized in that respect, I think.
> But that's a separate discussion.

-100000 to using unittest style tests.

Aaron Meurer

>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/549F3361.9050009%40durchholz.org.

[Francesco Bonazzi]

On Saturday, December 27, 2014 11:32:08 PM UTC+1, Joachim Durchholz wrote:

Am 27.12.2014 um 17:33 schrieb Francesco Bonazzi:
> I think it would be nice to make tests easily runnable with PyDev's
> debugger. Unfortunately, SymPy's tests being non-standard means that PyDev
> is not able to start them.

Just run bin/test with the proper parameters.
Avoid options that cause bin/test to start the tests in a subprocess.

Also, you *should* be able to start the test programs directly.

I would like to run or debug tests with just one click, as illustrated in this picture:

Either way, you need to set up a proper PYTHONPATH so that the test will
find the sympy module. It took me a bit of fiddling to get everything
right, but I got it to work eventually, and I think it gets easier once
you know which setting means exactly what.
Alternatively, a good set of instructions would be helpful. Since I'm
currently setting up a new Eclipse install and getting SymPy to work on
it, I might actually write something :-)

That's why I dumped Eclipse as soon as I realized that PyCharm has been made open source. Eclipse+PyDev and PyCharm are quite similar when it comes to functionality, but I'm much more comfortable with PyCharm. Concerning SymPy, with PyCharm I just had to open the folder of SymPy, the IDE recognized it was a git repository from github, when lines are edited, they get marked by a new color to visually track changes, compare/revert. Setting up git in Eclipse required some configuration work instead, and it was not as fussless as in PyCharm.

A review by someone else who switched to PyCharm from PyDev:

http://nicoddemus.github.io/articles/pycharm/

PyCharm also supports IPython notebook file editing.

SymPy should become more standardized in that respect, I think.
But that's a separate discussion.

I think that an IDE's killer feature is the easiness to debug the code, and that should be a primary issue. I would really like that SymPy tests could be debugged by simply one click on the test file name, that is way more comfortable than anything else.

[Joachim Durchholz]

Am 27.12.2014 um 23:52 schrieb Aaron Meurer:
> On Sat, Dec 27, 2014 at 3:32 PM, Joachim Durchholz <j...@durchholz.org> wrote:
>> Am 27.12.2014 um 17:33 schrieb Francesco Bonazzi:
>>>
>>> Both PyCharm and PyDev can detect Python tests written using the standard
>>> Python unit testing systems, unfortunately they don't recognize SymPy's
>>> tests.
>>
>> SymPy should become more standardized in that respect, I think.
>> But that's a separate discussion.
>
> -100000 to using unittest style tests.

Why?

Does "more standardized" necessarily imply "unittest style"?

[Joachim Durchholz]

Am 28.12.2014 um 10:35 schrieb Francesco Bonazzi:
>
> I think that an IDE's killer feature is the easiness to debug the code, and
> that should be a primary issue.

Sure.

I don't think that single-click procedures are necessary though.
What's more important is that the setup procedures behave, um,
"rational", i.e. they should not fail without apparent reason, and the
day-to-day procedures should be easy enough.

I.e. single-click "run all tests" certainly is nice, but SymPy tests
need configuration like --slow and random seed, which tends to break
single-click procedures. Something that requires two or three clicks
(and works reliably) should still be considered good enough for the
purpose, IMHO.

[Francesco Bonazzi]

On Sunday, December 28, 2014 12:13:51 PM UTC+1, Joachim Durchholz wrote:

I don't think that single-click procedures are necessary though.
What's more important is that the setup procedures behave, um,
"rational", i.e. they should not fail without apparent reason, and the
day-to-day procedures should be easy enough.

I.e. single-click "run all tests" certainly is nice, but SymPy tests
need configuration like --slow and random seed, which tends to break
single-click procedures. Something that requires two or three clicks
(and works reliably) should still be considered good enough for the
purpose, IMHO.

PyCharm can parse certain standardized test files, with the ability to run/debug a single test function by "left click" + "run/debug test" on the source code, this is comfortable when a developer has just made an edit but a test is failing, and he is trying to examine why the test is failing. I mean, you put a breakpoint by a mouse click, then you run the test in debug mode by clicking on the same location where the breakpoint lies, that's a killer feature.

Consider the potentiality for new users, maybe with little experience in SymPy or git. They would be able to learn how to run a test instantly, without any need to read SymPy's documentation. Git is really easy to use inside PyCharm.

Furthermore, I suggest that PyCharm be advised to new users having little experience with git. You just open SymPy's folder and it works. If you use Eclipse+PyDev it still needs some configuring, let alone integrate Git and Github into Eclipse.

[Aaron Meurer]

No, but in that case our tests are already "standardized" (they are just test_* functions with assertions). But if the suggestion is to use the unittest or unittest2 module with the class style of testing I am strongly opposed.

I.e. single-click "run all tests" certainly is nice, but SymPy tests need configuration like --slow and random seed, which tends to break single-click procedures. Something that requires two or three clicks (and works reliably) should still be considered good enough for the purpose, IMHO.

Sure but needing to care about those options is rare. The only flag you generally need to care about for testing is -k, which lets you run an exact test function, which can be faster. 

 

Aaron Meurer

>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sympy/549FE475.5010204%40durchholz.org.

[Joachim Durchholz]

Am 28.12.2014 um 23:14 schrieb Aaron Meurer:
> On Sun, Dec 28, 2014 at 4:07 AM, Joachim Durchholz <j...@durchholz.org> wrote:
>> Am 27.12.2014 um 23:52 schrieb Aaron Meurer:
>>>
>>> On Sat, Dec 27, 2014 at 3:32 PM, Joachim Durchholz <j...@durchholz.org>
>>> wrote:
>>>>
>>>> Am 27.12.2014 um 17:33 schrieb Francesco Bonazzi:
>>>>>
>>>>>
>>>>> Both PyCharm and PyDev can detect Python tests written using the
>>>>> standard
>>>>> Python unit testing systems, unfortunately they don't recognize SymPy's
>>>>> tests.
>>>>
>>>>
>>>> SymPy should become more standardized in that respect, I think.
>>>> But that's a separate discussion.
>>>
>>>
>>> -100000 to using unittest style tests.
>>
>>
>> Why?
>>
>> Does "more standardized" necessarily imply "unittest style"?
>
> No, but in that case our tests are already "standardized" (they are just
> test_* functions with assertions).

Ah. No, that wasn't what I meant.
I meant to say "if we follow standard patterns of testing frameworks
more closely, tools like PyDev and PyCharms should work better".

I have no knowledge of unittest and unittest2, and don't know if they
would be a good or bad idea.

>> I.e. single-click "run all tests" certainly is nice, but SymPy tests need
>> configuration like --slow and random seed, which tends to break
>> single-click procedures. Something that requires two or three clicks (and
>> works reliably) should still be considered good enough for the purpose,
>> IMHO.
>
> Sure but needing to care about those options is rare.

Well, not for me. --slow was the first option I needed.

> The only flag you
> generally need to care about for testing is -k, which lets you run an exact
> test function, which can be faster.

Which means that the point still stands: even the best IDE won't be able
to offer a single-click solution since you need to configure stuff
differently depending on what you want to know. I.e. you need at least a
second click to select the intended option(s).