Indexer doesn't delete records in Solr

[Johannes Neubarth]

Hello,
we are currently migrating from Lily 2.0 to 2.4. I set up a new machine with CDH 4.2.2, Lily 2.4 and Solr 4.6.1.
Our index is up and running:

Number of indexes: 1

Post
  + General state: ACTIVE
  + Update state: SUBSCRIBE_AND_LISTEN
  + Batch build state: INACTIVE
  + Queue subscription ID: IndexUpdater_Post
  + Solr Mode: CLASSIC
    + Solr shards:
      + shard1: http://192.168.1.22:8983/solr

I can create records from Java successfully. Then I call the indexer with
  indexer.index("record", id);
This was not necessary in our Lily 2.0 installation, but anyway. Afterwards, I commit to Solr:
  curl http://localhost:8983/solr/update -H 'Content-type:text/xml' --data-binary '<commit/>'
Now the record shows up in Solr.
Next, I delete the record using the Lily API. A RecordScanner confirms that the record is deleted. Accordingly, I cannot call the indexer afterwards:
    "IndexerException: failed to read from repository"
And even if I start a batch update with
    lily-update-index -n Post --build-state BUILD_REQUESTED
and commit to Solr, the record is still there.

There are no error messages in the Solr log, and only a few for Lily:

[WARN   ] <2014-03-12 14:02:24,139> (org.apache.hadoop.util.NativeCodeLoader): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[ERROR  ] <2014-03-12 14:02:25,115> (com.ngdata.sep.impl.SepModelImpl): Requested to remove a subscription which does not exist, skipping silently: 'LinkIndexUpdater'
...

[WARN   ] <2014-03-12 14:02:29,580> (org.springframework.context.support.PropertySourcesPlaceholderConfigurer): Could not load properties from URL [file:/home/lily/.lily-overrides.properties]: /home/lily/.lily-overrides.properties (No such file or directory)
[WARN   ] <2014-03-12 14:02:30,969> (com.sun.jersey.spi.inject.Errors): The following warnings have been detected with resource and/or provider classes:
  WARNING: A sub-resource method, public void org.lilyproject.rest.index.IndexResource.index(java.lang.String,java.lang.String,java.lang.String,java.lang.String) throws java.lang.Exception, with URI template, "", is treated as a resource method
[WARN   ] <2014-03-12 14:23:23,637> (org.apache.hadoop.mapred.JobClient): Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
[WARN   ] <2014-03-12 14:23:24,589> (org.apache.hadoop.conf.Configuration): fs.default.name is deprecated. Instead, use fs.defaultFS
[ERROR  ] <2014-03-12 14:23:24,969> (org.apache.hadoop.hbase.client.HConnectionManager): Connection not found in the list, can't delete it (connection key=HConnectionKey{properties={hbase.zookeeper.property.clientPort=2181, hbase.client.instance.id=lilyclient-2, hbase.client.pause=1000, zookeeper.znode.parent=/hbase, hbase.client.retries.number=10, hbase.zookeeper.quorum=192.168.1.22}, username='lily'}). May be the key was modified?
 

Any advice on how to delete records from Solr via Lily would be appreciated.
Hannes

[Nishanth S]

Hi ,

I am having the same issue.Did you get some work around?.

[Johannes Neubarth]

No, unfortunately we didn't solve it.
The project I was working on is completed by now. Since our amount of data was lower than expected, we decided to only persist records to Solr, and not use HBase anymore.

Hannes

[Nishanth S]

Hey John,

Thanks for the reply.