CI Is (Generally) Broken
210 views
Skip to first unread message
Travis Scrimshaw
May 15, 2024, 9:01:45 PMMay 15
to sage-devel
As the title says. Most PRs I see end up with this initial build failure:
Run export MAKE="make -j2 --output-sync=recurse" SAGE_NUM_THREADS=4
+ cd /sage
+ export 'MAKE=make -j2 --output-sync=recurse' SAGE_NUM_THREADS=4
+ MAKE='make -j2 --output-sync=recurse'
+ SAGE_NUM_THREADS=4
+ ./sage -t --new -p4
no stored timings available
Running doctests with ID 2024-05-16-00-35-14-3cd651fa.
Running with SAGE_LOCAL='/sage/local' and SAGE_VENV='/sage/local/var/lib/sage/venv-python3.10'
Using --optional=bliss,coxeter3,debian,mcqd,meataxe,pip,sage,sage_spkg,sirocco,tdlib
Features to be detected: 4ti2,SAGE_SRC,benzene,bliss,buckygen,conway_polynomials,csdp,cvxopt,cvxopt,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_cubic_hecke,database_ellcurves,database_graphs,database_jones_numfield,database_knotinfo,dvipng,ecm,fpylll,fricas,gap_package_atlasrep,gap_package_design,gap_package_grape,gap_package_guava,gap_package_hap,gap_package_polycyclic,gap_package_qpa,gap_package_quagroup,gfan,graphviz,imagemagick,ipython,jmol,jupymake,kenzo,latte_int,lrcalc_python,lrslib,matroid_database,mcqd,meataxe,mpmath,msolve,nauty,networkx,numpy,palp,pandoc,pdf2svg,pdftocairo,pexpect,phitigra,pillow,plantri,polytopes_db,polytopes_db_4d,pplpy,primecountpy,ptyprocess,pynormaliz,pyparsing,python_igraph,requests,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.braiding,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.sat,sage.schemes,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,scipy,singular,sphinx,symengine_py,sympy,tdlib,threejs
fatal: not a git repository: /new/.git/worktrees/worktree-image
Doctesting files changed since last git commit
Traceback (most recent call last):
File "/sage/src/bin/sage-runtests", line 166, in <module>
err = DC.run()
File "/sage/src/sage/doctest/control.py", line 1571, in run
self.add_files()
File "/sage/src/sage/doctest/control.py", line 932, in add_files
change = subprocess.check_output(["git",
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', '--git-dir=/sage/.git', '--work-tree=/sage', 'status', '--porcelain']' returned non-zero exit status 128.
Previously, this was rare, but now it seems to be very common.
Also, when there is a docbuild failure, the output is completely useless. I can't even find what files are causing the failure, much less what the reason for the failure was. (The cryptic sphinx errors were at least marginally helpful for this.) However, this is unrelated to the main issue above.
Best,
Travis
Travis Scrimshaw
May 15, 2024, 9:03:09 PMMay 15
to sage-devel
Also, it has sometimes happened the CI is not correctly pulling in the changed files. This occurred specifically for a py -> pyx file change.
Best,
Travis
Matthias Koeppe
May 15, 2024, 9:43:36 PMMay 15
to sage-devel
Yes, various parts of the CI are broken.
Fixes for the CI ready for review:
- "Makefile, .ci/write-dockerfile.sh: Update for src/pyproject.toml after #36982" (https://github.com/sagemath/sage/pull/37926)
- "Update conda-lock files, rename macOS conda-lock files to match CI Conda" (https://github.com/sagemath/sage/pull/37998)
Other infrastructure PRs ready for review:
- "GH Actions: Fix build of macOS x86_64 wheels, build wheels for sagemath-{bliss,coxeter3,mcqd,tdlib} for PyPI" (https://github.com/sagemath/sage/pull/36525)
Matthias Koeppe
May 17, 2024, 10:58:37 PMMay 17
to sage-devel
Tobia...@gmx.de
May 19, 2024, 10:24:34 PMMay 19
to sage-devel
I've now set https://github.com/sagemath/sage/pull/37998 back to "need work" for the following reasons:
- Needs to be tested on all 6 os/python version combinations
- Updates of the conda lock files should never require additions to the "known bugs" exclusions. If a certain version upgrade results in problems, add a version constraint to the conda files to exclude that particular version.
- The changes related to "-latest" in ci-conda are no improvement
- The renaming of the macos lock files are now inconsistent with the naming of the linux lock files. We should not orient ourselves on what github actions use as the default, but what is the most common system - these files are primarily intended to be used by users, and not for the ci.
Matthias Koeppe
May 24, 2024, 11:21:42 AMMay 24
to sage-devel
Open for review:
- https://github.com/sagemath/sage/pull/37998 (revised)
Reply all
Reply to author
Forward
0 new messages