Single node "cluster" with no shards

[Dayton Jones]

I've setup a single node running elassandra-6.8.4.5, the status of the "cluster" is green,  and I can see data via cqlsh (selects, desc keyspaces, etc)  But there are *no* shards -- I don't mean there are unallocated shards, I mean the shard count is 0.  I can connect to port 9200 and query (_cat, _cluster, etc) and I've ensured that the cassandra process is running with '-e' (though it didn't matter if that was there or not)  So I'm at a loss why I have no shards, and can't create indices.  Looking at elastic_admin.metadata, I can see the indices for all the keyspaces - so I'd assume once I have shards, I'll be able to use the indices (even if I have to rebuild them) and perform other tasks.

I did copy data/* and elasticsearch.data/* from another cluster (also single node) to seed this new node.  Any help/suggestions greatly appreciated.

_cluster/health
{
  "cluster_name" : "mycluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}


_cluster/state
{
  "cluster_name" : "mycluster",
  "cluster_uuid" : "8001ab07-31f5-4a78-a940-861243aa739a",
  "version" : 3,
  "state_uuid" : "uCL3UEWwTI65OCVzXoCdeQ",
  "master_node" : "py2tdBUKQAS5htt1lm2XGw",
  "blocks" : { },
  "nodes" : {
    "py2tdBUKQAS5htt1lm2XGw" : {
      "name" : "10.24.88.21",
      "status" : "ALIVE",
      "ephemeral_id" : "py2tdBUKQAS5htt1lm2XGw",
      "transport_address" : "10.24.88.21:9300",
      "attributes" : {
        "rack" : "b",
        "dc" : "Lab"
      }
    }
  },
  "metadata" : {
    "version" : 0,
    "cluster_uuid" : "8001ab07-31f5-4a78-a940-861243aa739a",
    "templates" : { },
    "indices" : { },
    "index-graveyard" : {
      "tombstones" : [ ]
    }
  },
  "routing_table" : {
    "indices" : { }
  },
  "routing_nodes" : {
    "unassigned" : [ ],
    "nodes" : {
      "py2tdBUKQAS5htt1lm2XGw" : [ ]
    }
  },
  "restore" : {
    "snapshots" : [ ]
  },
  "snapshot_deletions" : {
    "snapshot_deletions" : [ ]
  },
  "snapshots" : {
    "snapshots" : [ ]
  }
}


_cat/nodes
ip          heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.24.88.21            4          96  67    0.22    0.97     0.89 mdi       *      10.24.88.21

_cat/allocation
shards disk.indices disk.used disk.avail disk.total disk.percent host        ip          node
     0           0b    42.8gb     57.1gb     99.9gb           42 10.24.88.21 10.24.88.21 10.24.88.21

_cat/shards
index shard prirep state docs store ip node

_cat/nodeattrs
node        host        ip          attr value
10.24.88.21 10.24.88.21 10.24.88.21 rack b
10.24.88.21 10.24.88.21 10.24.88.21 dc   Lab

[Mitch Gitman]

The one thing you did that looks questionable is this: "I did copy data/* and elasticsearch.data/* from another cluster."

If you're migrating data from one Cassandra cluster to another, you should be using the standard Cassandra tools to do that, not hacking the filesystem. Among your options:

* Use nodetool snapshot to capture the data in the source cluster. Use sstableloader to load the captured data into the destination cluster.

* Use dsbulk unload to capture the data in the source cluster. Use dsbulk load to load the captured data into the destination cluster.

* Use CQL COPY FROM to export the data into the source cluster. Use CQL COPY TO to import the contents of the exported file into the destination cluster.

* Do the equivalent with Spark.

Once the data is loaded, you can submit the HTTP POST request to the Elasticsearch API exposed by Elassandra to define the index and let it get populated with the data.

--
You received this message because you are subscribed to the Google Groups "Elassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elassandra+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elassandra/76d2bf43-2eb6-48b3-bc95-bcc4eac41225n%40googlegroups.com.

[Dayton Jones]

Thanks, I 'll try that - I did the copy from one node to the other because I'd seen several comments that it should work (especially since it's just single node)

Is there a way to get the entire keyspace to copy or would I really have to specify each table in the keyspace  (copy from keyspace.table1, copy from keysapce.table2, etc) -- ie, is there a wildcard that cqlsh/copy recognizes, or a way to batch (there are 15 keyspaces, each with several tables) it?  I could script it, loop through keyspaces/tables but it would be nice if I could just do one command (or at least one command per keyspace)

[Mitch Gitman]

Commands like this tend to be executed on a single table and not on all the tables in a keyspace, let alone all the keyspaces in a cluster (which could be dangerous, considering the system keyspaces).

Here's some bash scripting one can use to cycle through all the tables in a keyspace and then do something with them:
 echo "SELECT table_name,id FROM system_schema.tables WHERE keyspace_name = '$KEYSPACE';" > get_table_identifiers_query
 cqlsh -u $USERNAME -p $PASSWORD $CQLSH_HOST -f get_table_identifiers_query | sed -e '1,3d' -e '$d' -e '/^$/d' -e 's/ //g' -e 's/-//g' -e 's/|/-/g'> table_identifiers
 while IFS='' read -r TABLE || [[ -n "$TABLE" ]]; do
  NEXT_TABLE=${TABLE%-*}
 done < table_identifiers

Once you have NEXT_TABLE, you can make it the argument of some command.

There may be a more Cassandra-native way to get all the tables for a keyspace, but I'm not aware of it.

To view this discussion on the web visit https://groups.google.com/d/msgid/elassandra/e19b6737-6576-4151-bd39-365724691860n%40googlegroups.com.