Dear experts,
I am doing a loadtest for my pycassa. I found that it raise Exceptions if
my application do over 1000 write at the same time.
My app:
import pycassa
from multiprocessing import Process

def singleton(cls):
    instances = {}
    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    return get_instance

@singleton
class Connection(pycassa.ConnectionPool):
    def __init__(self, keyspace='InviteAppRevamp2',
server_list=['192.168.1.41:9160'], pool_size=20):
        pycassa.ConnectionPool.__init__(self,
                                        keyspace,
                                        server_list=server_list,
                                        pool_size=pool_size)

def write():
    pool = Connection()
    cf = pycassa.ColumnFamily(pool, 'test2')
    attr = {('xyz', "abc", '123'): "colval"}
    cf.insert('haha', attr)

if __name__ == '__main__':
    for i in range(1, 10):
        p = Process(target=write, args=())
        p.start()

How can I solve the issue?
Cheers,
Cato

What kind of Exceptions?

Did you check your open file handle limit? (use ulimit -a)

  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/mul tiprocessing/process.py",
line 88, in run
    self._target(*self._args, **self._kwargs)
  File "cassandra.py", line 21, in write
    pool = Connection()
  File "cassandra.py", line 8, in get_instance
    instances[cls] = cls(*args, **kwargs)
  File "cassandra.py", line 18, in __init__
    pool_size=pool_size)
  File "/Library/Python/2.7/site-packages/pycassa/pool.py", line 359, in
__init__
    self.fill()
  File "/Library/Python/2.7/site-packages/pycassa/pool.py", line 420, in
fill
    conn = self._create_connection()
  File "/Library/Python/2.7/site-packages/pycassa/pool.py", line 408, in
_create_connection
    (exc.__class__.__name__, exc))
AllServersUnavailable: An attempt was made to connect to each of the
servers twice, but none of the attempts succeeded. The last failure was
TTransportException: Could not connect to 192.168.1.41:9160

This error occurs.
And file limit is 1024
Cheers,
Cato

I found that the error can be solved by changing the pool size to 1.
However, another error occurs:
[Fri Sep 21 10:12:20 2012] [alert] (13)Permission denied: mod_wsgi
(pid=12374): Couldn't bind unix domain socket
'/private/var/run/wsgi.12374.0.1.sock'.
[Fri Sep 21 10:12:20 2012] [notice] Digest: generating secret for digest
authentication ...
[Fri Sep 21 10:12:20 2012] [notice] Digest: done
[Fri Sep 21 10:12:20 2012] [notice] Apache/2.2.21 (Unix) PHP/5.3.6
mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 mod_wsgi/3.4 Python/2.7.1 configured --
resuming normal operations
INFO:pycassa.pool:Connection 4329685632 (192.168.1.41:9160) in pool
4495557264 failed: Could not connect to 192.168.1.41:9160

This is the error log of my apache
Cheers,
Cato

I have already fixed that problem too. Just follow the document here:
http://code.google.com/p/modwsgi/wiki/ConfigurationIssues
section
Location Of UNIX Sockets
However, some of my http request will generate error, but no error is shown
in apache error log.
Those http requests return something like this:
<urlopen error [Errno 54] Connection reset by peer>
<urlopen error [Errno 32] Broken pipe>

Cheers,
Cato

If your open file handle limit is 1024, that's probably the issue.  You
need to increase that using ulimit, which is temporary, or by editing
/etc/security/limits.conf, which requires logging out and logging back in
to take effect, and may be located somewhere else on non-debian based
distros.

There's an example of what to put in /etc/security/limits.conf here:
http://www.datastax.com/docs/1.0/troubleshooting/index#java-reports-a...