Help - Blob File upload

[Scott Reynolds]

Hi I'm trying to upload a large blod to Cassandra with Python and I'm stuck. I need to allow uploads of around 1MB to be loaded into a blob column but I keep getting a Cassandra error, 'cssandra.cluster.NoHostAvailable: ("Unable to complete the operation against any hosts', {})

I'm getting stuck using the following code, I presume its cause by the max query params (Query parameters in a query: 65535 (216-1)-- https://docs.datastax.com/en/cql/3.3/cql/cql_reference/refLimits.html). 

Does anyone know if there's a way around the limit? 

import ssl
import os
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
from cassandra.query import SimpleStatement
from cassandra.encoder import Encoder

def e_var(name):
try:
    return os.environ[name]
except KeyError:        
    print ("Please set the environment variable " + name)        
    sys.exit(1)
    
ssl_options = {
    'ssl_version': ssl.PROTOCOL_TLSv1}
    
user = e_var('CAS_USER')
pwd = e_var('CAS_PWD')
ip = e_var('CAS_IPS').split(',')
keyspace = e_var('CAS_KEYSPACE')
ap = PlainTextAuthProvider(username=user, password=pwd)

cluster = Cluster(ip, auth_provider=ap, ssl_options=ssl_options)
session = cluster.connect(keyspace)
session.execute('CREATE TABLE IF NOT EXISTS  sample (key text,data blob, PRIMARY KEY (key))')
print('***table done')
def insert(bytes_length, message ):
    e = Encoder()  
    data = ('a' * bytes_length).encode()  
    d = {    'key' : 'bucket_1', 'data' :bytearray(data)  }  
    session.execute("""INSERT INTO sample (key,data) VALUES (%(key)s, %(data)s)""",d)  
    print('***' + message)

insert(3000, 'done first insert')
insert(32727, 'done second insert')
insert(32728, 'done third insert')

[jaume.m...@datastax.com]

Hello Scott,

The error is not related to the max query parameter since you are only using two (keys and data). I've tried your script and it's working fine for me setting the appropriate ssl parameters, I'm not sure how you have set the authentication since you are not specifying the cert_file or the key file for the ssl options. Might help to run the driver using DEBUG mode to see the logs. This is done by adding at the beginning of the script;

import logging

log = logging.getLogger()
log.setLevel('DEBUG')
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s [%(levelname)s] %(name)s: %(message)s"))
log.addHandler(handler)

Also the cassandra logs (specifically the system.log file) might be useful to know what's going on.

Jaume

[Scott Reynolds]

I’ve been working through this more it seems to only occur on python 3.6 (haven’t tried any other python 3.*).  It seems to work fine in python 2.7.

I’m gong to try on another machine. (Working I’m getting logs).

---

From: python-dr...@lists.datastax.com <python-dr...@lists.datastax.com> on behalf of jaume.m...@datastax.com <jaume.m...@datastax.com>
Sent: Monday, November 6, 2017 1:48:24 PM
To: DataStax Python Driver for Apache Cassandra User Mailing List
Subject: Re: Help - Blob File upload

 

--
You received this message because you are subscribed to the Google Groups "DataStax Python Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-u...@lists.datastax.com.