Error installing on EC2

[Matthew Shoemake]

I am installing s3ql on an Amazon EC2 instance, and I'm having a bit of trouble. I believe I've installed all of the libraries correctly:

[ec2-user]$ uname -a

Linux ip-10-64-42-69 3.2.30-49.59.amzn1.x86_64 #1 SMP Wed Oct 3 19:54:33 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

[ec2-user]$ sqlite3 -version

3.7.14.1 2012-10-04 19:37:12 091570e46d04e84b67228e0bdbcd6e1fb60c6bdb

[ec2-user]$ python2.7 -c 'import pycryptopp'

[ec2-user]$ python2.7 -c 'import apsw; print apsw.apswversion()'

3.7.14.1-r1

[ec2-user]$ python2.7 -c 'import lzma; print lzma.__version__'

0.5.3

[ec2-user]$ python2.7 -c 'import llfuse; print llfuse.__version__'

0.37.1

Here's the error I'm getting. Any help is much appreciated:

[ec2-user]$ sudo python2.7 setup.py test

running test

running egg_info

writing requirements to src/s3ql.egg-info/requires.txt

writing src/s3ql.egg-info/PKG-INFO

writing top-level names to src/s3ql.egg-info/top_level.txt

writing dependency_links to src/s3ql.egg-info/dependency_links.txt

writing entry points to src/s3ql.egg-info/entry_points.txt

reading manifest file 'src/s3ql.egg-info/SOURCES.txt'

writing manifest file 'src/s3ql.egg-info/SOURCES.txt'

running build_ext

copying build/lib.linux-x86_64-2.7/s3ql/_deltadump.so -> src/s3ql

Uncaught top-level exception:

Traceback (most recent call last):

  File "setup.py", line 317, in <module>

    main()

  File "setup.py", line 175, in main

    command_options={ 'sdist': { 'formats': ('setup.py', 'bztar') } },

  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "/usr/lib64/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/test.py", line 137, in run

    self.with_project_on_sys_path(self.run_tests)

  File "/usr/lib64/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/test.py", line 117, in with_project_on_sys_path

    func()

  File "setup.py", line 296, in run_tests

    testLoader=ScanningLoader())

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/main.py", line 97, in __init__

    self.parseArgs(argv)

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/main.py", line 152, in parseArgs

    self.createTests()

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/main.py", line 161, in createTests

    self.module)

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/loader.py", line 148, in loadTestsFromNames

    suites = [self.loadTestsFromName(name, module) for name in names]

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/loader.py", line 123, in loadTestsFromName

    return self.loadTestsFromModule(obj)

  File "setup.py", line 288, in loadTestsFromModule

    tests.append(self.loadTestsFromName(submodule))

  File "/home/ec2-user/s3ql-1.12/unittest2-0.5.1-py2.7.egg/unittest2/loader.py", line 120, in loadTestsFromName

    parent, obj = obj, getattr(obj, part)

AttributeError: 'module' object has no attribute 't1_dump'

[Francisco Reyes]

On Thursday, November 22, 2012 9:14:41 PM UTC-5, Matthew Shoemake wrote:

I am installing s3ql on an Amazon EC2 instance, and I'm having a bit of trouble. I believe I've installed all of the libraries correctly:

[ec2-user]$ uname -a

Linux ip-10-64-42-69 3.2.30-49.59.amzn1.x86_64 #1 SMP Wed Oct 3 19:54:33 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Sorry I am not experienced enough to help.. but was wondering, are you installing on any of the documented OSs here: http://code.google.com/p/s3ql/w/list ?

May be easier than to try and track every dependency.

Also check this:

https://groups.google.com/forum/?fromgroups=#!topic/s3ql/8jU_zopeMp4

and possibly this

https://groups.google.com/forum/?fromgroups=#!topic/s3ql/APA1rNwdYIs

If you are trying to install on any of the listed OSs on the wiki, may be easier to go that route.

Good luck..

[Matthew Shoemake]

Thanks for the guidance. I am going to try another Linux version on EC2. 

Is there a preferred distribution that s3ql is known to work particularly well on? Fedora, Ubuntu, Debian, etc.? 

[Francisco Reyes]

On Thursday, November 22, 2012 11:12:25 PM UTC-5, Matthew Shoemake wrote:

Is there a preferred distribution that s3ql is known to work particularly well on? Fedora, Ubuntu, Debian, etc.? 

I think any of the listed ones in the Wiki should work.

Personally I have installed it a couple of times in Ubuntu without any problems.

On Ubuntu if you use the instructions here:  http://code.google.com/p/s3ql/wiki/installation_ubuntu

And then as that wiki post suggests continue here: http://ubuntuservergui.com/ubuntu-server-guide/mount-s3-ubuntu-server

you should be ok.

Use the second link for examples of formatting and mounting.

Also very useful for day to day operation: http://www.rath.org/s3ql-docs/about.html

What I have been doing is creating shell scripts for the day to day operations like...

mount-script.sh

-------

#!/bin/sh

mount.s3ql --cachesize 2048000 --cachedir /S3ql-cache --max-cache-entries 8000 --allow-other --metadata-upload-interval 3600 s3c://s.greenqloud.com/backup-low-security /s3ls

umount-script

-------

#!/bin/sh

umount.s3ql /s3ls

fsck-script

-------

#!/bin/sh

fsck.s3ql --cachedir /S3ql-cache s3c://s.greenqloud.com/backup-low-security

I am using GreenQloud instead of amazon so it is slightly different syntax, but the above scripts are just to give you an idea..

The other good reason to use the scripts is in case you have multiple mounts. I have a lowsecurity mount (mounted at all times) and a adhocmount (only mount when copying data to it for increased security).. so I have scripts named

lsMNT -> Low security mount

lsUMNT -> Low security unmount

lsFSCK -> Low security fsck

amMNT -> Adhoc mount

amUMNT -> Adhoc unmount

amFSCK -> Adhoc fsck

If you are going to backup lots of files take a look into the cache parameters and also pay attention to ulimit. In particular notice that your limiting factor will likely be --max-cache-entries, before --cachesize.

Take care...

[Nikolaus Rath]

On 11/22/2012 09:14 PM, Matthew Shoemake wrote:
> I am installing s3ql on an Amazon EC2 instance, and I'm having a bit of
> trouble. I believe I've installed all of the libraries correctly:
>
> [ec2-user]$ uname -a
>
> Linux ip-10-64-42-69 3.2.30-49.59.amzn1.x86_64 #1 SMP Wed Oct 3 19:54:33
> UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
>
> [ec2-user]$ sqlite3 -version
>
> 3.7.14.1 2012-10-04 19:37:12 091570e46d04e84b67228e0bdbcd6e1fb60c6bdb
>
> [ec2-user]$ python2.7 -c 'import pycryptopp'
>
> [ec2-user]$ python2.7 -c 'import apsw; print apsw.apswversion()'
>
> 3.7.14.1-r1
>
> [ec2-user]$ python2.7 -c 'import lzma; print lzma.__version__'
>
> 0.5.3
>
> [ec2-user]$ python2.7 -c 'import llfuse; print llfuse.__version__'
>
> 0.37.1
>
>
> Here's the error I'm getting. Any help is much appreciated:
>
> [ec2-user]$ sudo python2.7 setup.py test
> running test
> running egg_info

[...]

> 120, in loadTestsFromName
> parent, obj = obj, getattr(obj, part)
> AttributeError: 'module' object has no attribute 't1_dump'

Try running this:

setup.py build_ext --inplace
bin/mount.s3ql --help


This should give you a more helpful error message.


Best,

-Nikolaus

--
»Time flies like an arrow, fruit flies like a Banana.«

PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

[David Harrison]

i've run s3ql on ubuntu 10.04 64bit instances without issue...

you can find a list of AMI's here...

http://uec-images.ubuntu.com/releases/10.04/release/

 

--