Errors during test, and operation

[Esteban Fonseca]

Hello all,

I'm getting errors during test, and then something really weird happens, I run the installation, and then I get errors when executing the scripts from /usr/local/bin, however, binaries under the source bin/ folder do actually work, and I can do a mkfs.s3ql, mount, umount, fsck, etc... I'm really puzzled as to what am I missing, all the required packages are there, I've also tried with pyfuse 1.3.1 and the previous version (3.4.1) with the same or similar results, which is really odd because a few months ago I successfully compiled that version on a previous install using the same procedure.

Here are some outputs:

1. Build

[root@hostname]# python3.6 setup.py build_ext --inplace
running build_ext
building 's3ql.deltadump' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/src
creating build/temp.linux-x86_64-3.6/src/s3ql
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c src/s3ql/deltadump.c -o build/temp.linux-x86_64-3.6/src/s3ql/deltadump.o -Wall -Wextra -Wconversion -Wsign-compare
src/s3ql/deltadump.c: In function ‘__pyx_pw_4s3ql_9deltadump_5load_table’:
src/s3ql/deltadump.c:7177:48: warning: ‘__pyx_v_int64’ may be used uninitialized in this function [-Wmaybe-uninitialized]
                 __pyx_v_int64 = (__pyx_v_int64 + ((__pyx_cur_scope->__pyx_v_col_args[__pyx_v_j]) + (__pyx_cur_scope->__pyx_v_int64_prev[__pyx_v_j])));
                                                ^
src/s3ql/deltadump.c:5904:11: note: ‘__pyx_v_int64’ was declared here
   int64_t __pyx_v_int64;
           ^
src/s3ql/deltadump.c:11993:13: warning: ‘__pyx_v_row_count’ may be used uninitialized in this function [-Wmaybe-uninitialized]
             return PyInt_FromLong((long) value);
             ^
src/s3ql/deltadump.c:5903:11: note: ‘__pyx_v_row_count’ was declared here
   int64_t __pyx_v_row_count;
           ^
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/s3ql
gcc -pthread -shared -Wl,-z,relro -g build/temp.linux-x86_64-3.6/src/s3ql/deltadump.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/s3ql/deltadump.cpython-36m-x86_64-linux-gnu.so -lsqlite3
copying build/lib.linux-x86_64-3.6/s3ql/deltadump.cpython-36m-x86_64-linux-gnu.so -> src/s3ql

2. Test

[root@hostname]# python3.6 -m pytest tests/
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /bin/python3.6
cachedir: .pytest_cache
rootdir: /usr/src/s3ql-3.5.0/tests, configfile: pytest.ini
plugins: trio-0.6.0
collected 351 items                                                                                                                                                                        

tests/t1_backends.py::test_read_write[local/aes] ERROR                                                                                                                                [  0%]
tests/t1_backends.py::test_read_write[local/aes] ERROR                                                                                                                                [  0%]

========================================================================================== ERRORS ===========================================================================================
_______________________________________________________________________ ERROR at setup of test_read_write[local/aes] ________________________________________________________________________
Traceback (most recent call last):
  File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 137, in pytest_runtest_setup
    check_output(item)
  File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 133, in check_output
    check_test_log(item.catch_log_handler)
AttributeError: 'Function' object has no attribute 'catch_log_handler'
______________________________________________________________________ ERROR at teardown of test_read_write[local/aes] ______________________________________________________________________
Traceback (most recent call last):
  File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 143, in pytest_runtest_teardown
    check_output(item)
  File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 133, in check_output
    check_test_log(item.catch_log_handler)
AttributeError: 'Function' object has no attribute 'catch_log_handler'
================================================================================== short test summary info ==================================================================================
ERROR tests/t1_backends.py::test_read_write[local/aes] - AttributeError: 'Function' object has no attribute 'catch_log_handler'
ERROR tests/t1_backends.py::test_read_write[local/aes] - AttributeError: 'Function' object has no attribute 'catch_log_handler'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 2 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================================== 2 errors in 1.13s =====================================================================================

3. Mount (or any other operation)

[root@hostname]# /usr/local/bin/mount.s3ql --compress none --backend-options no-ssl --authfile /etc/s3ql_authinfo s3://region/bucket /mnt
Traceback (most recent call last):
  File "/usr/local/bin/mount.s3ql", line 33, in <module>
    sys.exit(load_entry_point('s3ql==3.5.0', 'console_scripts', 'mount.s3ql')())
  File "/usr/local/bin/mount.s3ql", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 105, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/mount.py", line 18, in <module>
    from .metadata import (download_metadata, upload_metadata, dump_and_upload_metadata,
  File "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/metadata.py", line 13, in <module>
    from .deltadump import INTEGER, BLOB, dump_table, load_table
ImportError: cannot import name 'INTEGER'

Thanks in advance for any help :)

[Nikolaus Rath]

On Sep 03 2020, Esteban Fonseca <este...@gmail.com> wrote:
> *1. Build*

Nothing wrong here.


> *2. Test*

>
> [root@hostname]# python3.6 -m pytest tests/

[...]

> Traceback (most recent call last):
> File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 137, in
> pytest_runtest_setup
> check_output(item)
> File "/usr/src/s3ql-3.5.0/tests/pytest_checklogs.py", line 133, in
> check_output
> check_test_log(item.catch_log_handler)
> AttributeError: 'Function' object has no attribute 'catch_log_handler'

Maybe we are not compatible with pytest 6.x - can you try an earlier
version (say 5.x)?

> *3. Mount (or any other operation)*

>
> "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/mount.py",
> line 18, in <module>
> from .metadata import (download_metadata, upload_metadata,
> dump_and_upload_metadata,
> File
> "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/metadata.py",
> line 13, in <module>
> from .deltadump import INTEGER, BLOB, dump_table, load_table
> ImportError: cannot import name 'INTEGER'

This one confuses me, but I'd focus on the tests first.


Best,
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«

[Esteban Fonseca]

Hello Nikolaus, I was able to solve this by running the following:

python3.6 setup.py build_cython

Got this from here:

https://www.mail-archive.com/search?l=s3...@googlegroups.com&q=subject:%22Re%5C%3A+%5C%5Bs3ql%5C%5D+Error+when+compiling+s3ql+3.5.0%22&o=newest&f=1

So thanks to Daniel Jagszent as well.

Regards,

Esteban Fonseca.

Best,
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

--
You received this message because you are subscribed to the Google Groups "s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email to s3ql+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/87pn73uye7.fsf%40vostro.rath.org.

[Esteban Fonseca]

On Thu, Sep 3, 2020 at 5:12 PM Nikolaus Rath <Niko...@rath.org> wrote:

Sorry didn't see this comment, I can confirm that pytest 5.x works fine for the tests. 

> *3. Mount (or any other operation)*
>
> "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/mount.py",
> line 18, in <module>
>     from .metadata import (download_metadata, upload_metadata,
> dump_and_upload_metadata,
>   File
> "/usr/local/lib64/python3.6/site-packages/s3ql-3.5.0-py3.6-linux-x86_64.egg/s3ql/metadata.py",
> line 13, in <module>
>     from .deltadump import INTEGER, BLOB, dump_table, load_table
> ImportError: cannot import name 'INTEGER'

This one confuses me, but I'd focus on the tests first.


Best,
-Nikolaus

--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«