Why is cloning so slow?

[Simon King]

Hi!

I installed sage-4.3.alpha0, and tried to make a clone. The cloning
process hanged, it did not finish within 8 hours!

It was on sage-math.

Cheers,
Simon

[Simon King]

In fact, it seems to me that sage-4.3.alpha0 is foobar.

After returning to sage-main, I tried to apply a patch:
sage: hg_sage.import_patch('patches/singular_prcopyr_no7577.patch')
cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg status
/home/SimonKing/sandbox/sage-4.3.alpha0/local/lib/python2.6/site-
packages/sage/misc/hg.py:240: DeprecationWarning: os.popen3 is
deprecated. Use the subprocess module.
x = os.popen3(s)
cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg status
cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg import
"/home/SimonKing/sandbox/patches/singular_prcopyr_no7577.patch"
abort: outstanding uncommitted changes

OK, a disturbing deprecation warning, and uncommitted changes.
Changes? Not really:
sage: hg_sage.commit()
cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg diff |
less
(END)

# by the way, it did not show me an empty page, it just seemed to do
nothing.
# After hitting "q":
cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg commit
nothing changed
sage:

So, nothing changed, but there are uncommitted changes.

Best regards,
Simon

[Carlo Hamalainen]

On Thu, Dec 3, 2009 at 9:32 AM, Simon King <simon...@nuigalway.ie> wrote:
> In fact, it seems to me that sage-4.3.alpha0 is foobar.

I can make a clone in 4.3.alpha0 from a clean install (takes about 1.5
minutes) but the 7577 patch doesn't do anything:

carlo@blah:~/sage/devel/sage$ hg qinit
carlo@blah:~/sage/devel/sage$ hg qimport
~/mpolynomial_rr_libsingular_fixes.patch
adding mpolynomial_rr_libsingular_fixes.patch to series file
carlo@blah:~/sage/devel/sage$ hg push
pushing to /home/carlo/sage-4.3.alpha0/devel/sage
searching for changes
no changes found


--
Carlo Hamalainen
http://carlo-hamalainen.net

[Simon King]

On Dec 3, 9:01 am, Carlo Hamalainen <carlo.hamalai...@gmail.com>
wrote:

> On Thu, Dec 3, 2009 at 9:32 AM, Simon King <simon.k...@nuigalway.ie> wrote:
> > In fact, it seems to me that sage-4.3.alpha0 is foobar.
>
> I can make a clone in 4.3.alpha0 from a clean install (takes about 1.5
> minutes) but the 7577 patch doesn't do anything:

That's a different patch.

But now, as you say it: Could it be the the patch from 7577 is already
merged? Although the ticket is "needs-info"?

Cheers,
Simon

[Carlo Hamalainen]

On Thu, Dec 3, 2009 at 10:11 AM, Simon King <simon...@nuigalway.ie> wrote:
> That's a different patch.

Oops.

> But now, as you say it: Could it be the the patch from 7577 is already
> merged? Although the ticket is "needs-info"?

I don't think 7577 has been applied in 4.3.alpha0, e.g. around line
213 of sage/libs/singular/ring.pyx there isn't something like

elif is_RealField(base_ring) and base_ring.rounding_mode() == 'RNDN':

which the patch is trying to bring in.

[Simon King]

Hi Carlo,

I'll elaborate a bit more what I did and what happened.

1. Untar and make sage-4.3.alpha0 on sage-math.
2. Wait a few hours, play with ./sage
(I didn't do ./sage -testall, though)
3. Do ./sage -clone infinitepoly before sleeping (in a screen session,
if that's relevant
4. Next morning, find that it is still not finished. I don't know
exactly where it was hanging, but my impression was that it
unsuccessfully tried to rebuild the documentation.
5. After hitting Ctrl-C several times, ./sage seemed to work *in the
new branch*.
6. I tried to apply the patch. One hunk failed. Strange enough, since
there are reports that it does apply.
7. Back to the main branch, ./sage -br main
8. Try to apply the patch to the main branch -- after all, the cloning
was not really a success. This time, mercurial complains that there
are uncommitted changes.
9. hg_sage.diff() showed nothing.
10. sage: hg_sage.status()
Getting status of modified or unknown files:

cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg
status

! doc/fr/a_tour_of_sage/eigen_plot.png
! doc/fr/a_tour_of_sage/sin_plot.png

Aha! There is a problem with the docs, right? Are these files missing?

Something else. When trying to apply the patch, there was a
deprecation warning:
sage: hg_sage.import_patch('http://trac.sagemath.org/sage_trac/raw-
attachment/ticket/7581/singular_prcopyr.patch')
Attempting to load remote file:
http://trac.sagemath.org/sage_trac/raw-attachment/ticket/7581/singular_prcopyr.patch
Loading: [.]

cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg status
/home/SimonKing/sandbox/sage-4.3.alpha0/local/lib/python2.6/site-
packages/sage/misc/hg.py:240: DeprecationWarning: os.popen3 is
deprecated. Use the subprocess module.
x = os.popen3(s)

Cheers,
Simon

[Minh Nguyen]

Hi Simon,

On Thu, Dec 3, 2009 at 8:34 PM, Simon King <simon...@nuigalway.ie> wrote:

<SNIP>

> 10. sage: hg_sage.status()
> Getting status of modified or unknown files:
> cd "/home/SimonKing/sandbox/sage-4.3.alpha0/devel/sage" && hg
> status
> ! doc/fr/a_tour_of_sage/eigen_plot.png
> ! doc/fr/a_tour_of_sage/sin_plot.png
>
> Aha! There is a problem with the docs, right? Are these files missing?

You get those two lines with the exclamation marks because the file
MANIFEST.in in Sage 4.3.alpha0 isn't configured to pick up those two
image files. When ticket #7190 (French translation: A Tour of Sage)
[1] was merged in Sage 4.3.alpha0, the file
devel/sage-main/MANIFEST.in wasn't also changed to take into account
the new image files, so these are not picked up when releasing the
alpha0 tarball. A result is that one would not see the image files in
devel/sage-main/doc/fr/a_tour_of_sage. You can fix the missing files
problem as follows:

[mvngu@sage sage-4.3.alpha0-7473-sphinx]$ cd devel/sage-main/
[mvngu@sage sage-main]$ hg st
! doc/fr/a_tour_of_sage/eigen_plot.png
! doc/fr/a_tour_of_sage/sin_plot.png
[mvngu@sage sage-main]$ hg revert -a
reverting doc/fr/a_tour_of_sage/eigen_plot.png
reverting doc/fr/a_tour_of_sage/sin_plot.png
[mvngu@sage sage-main]$ hg st
<no output>

At the moment, cloning in Sage 4.3.alpha0 is messed up. See ticket
#7473 [2] for a patch to fix this issue. You should apply that patch
to the script repository. From the look of it, that respository is a
bit messed up; some files are not yet under revision control:

[mvngu@sage sage-4.3.alpha0-7473-sphinx]$ cd local/bin/
[mvngu@sage bin]$ hg st
M sage-latest-online-package
? hmac256
? jmol
? pilconvert.py
? pildriver.py
? pilfile.py
? pilfont.py
? pilprint.py
? sphinx-autogen

A work around is to check in the changes with "hg ci", then apply the
patch at #7473:

[mvngu@sage bin]$ hg ci
<write a commit message>
[mvngu@sage bin]$ hg qimport
http://trac.sagemath.org/sage_trac/raw-attachment/ticket/7473/trac_7473-scripts_clone_v3.patch
adding trac_7473-scripts_clone_v3.patch to series file
[mvngu@sage bin]$ hg qpush
applying trac_7473-scripts_clone_v3.patch
now at: trac_7473-scripts_clone_v3.patch

Now make a clone:

[mvngu@sage sage-4.3.alpha0-7473-sphinx]$ ./sage -clone exp
<wait for about 20 minutes to 1 hour>


[1] http://trac.sagemath.org/sage_trac/ticket/7190

[2] http://trac.sagemath.org/sage_trac/ticket/7473

--
Regards
Minh Van Nguyen

[Simon King]

Dear Minh,

Thank you for your hints! It worked, and actually needed less than
"about 20 minutes to 1 hour".

So, now I can hopefully start to review the new libsingular patches.

Cheers,
Simon