No such unique ID for 'tagv'

[Sungho Park]

I suffer from the following error.

Are there someone who know how to handle it?
Can I remove the tagv([0, 23, -112]) in the error ?

The version of OpenTSDB: [2.0.0] built from revision 781cb61 in a MINT state

04:47:15.130 ERROR [HttpQuery.logError] - [id: 0x40db85de, /100.66.160.5:45628 => /100.66.162.219:4242] Internal Server Error on /q?start=24h-ago&m=sum:5m-avg:rate:target_process_cpu_se
conds%7Btarget=*%7D&o=&yrange=%5B0:%5D&nokey&wxh=800x400&autoreload=15&png
com.stumbleupon.async.DeferredGroupException: At least one of the Deferreds failed, first exception:
        at com.stumbleupon.async.DeferredGroup.done(DeferredGroup.java:169) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup.recordCompletion(DeferredGroup.java:158) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup.access$200(DeferredGroup.java:36) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup$1NotifyOrdered.call(DeferredGroup.java:97) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.callback(Deferred.java:1005) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup.done(DeferredGroup.java:169) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup.recordCompletion(DeferredGroup.java:158) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup.access$200(DeferredGroup.java:36) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.DeferredGroup$1NotifyOrdered.call(DeferredGroup.java:97) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.callback(Deferred.java:1005) ~[suasync-1.4.0.jar:fe17b98]
        at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:450) ~[asynchbase-1.4.1.jar:8c94a38]
        at org.hbase.async.RegionClient.decode(RegionClient.java:1185) ~[asynchbase-1.4.1.jar:8c94a38]
        at org.hbase.async.RegionClient.decode(RegionClient.java:82) ~[asynchbase-1.4.1.jar:8c94a38]
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[netty-3.6.2.Final.jar:na]
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) ~[netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[netty-3.6.2.Final.jar:na]
        at org.hbase.async.RegionClient.handleUpstream(RegionClient.java:1008) ~[asynchbase-1.4.1.jar:8c94a38]
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) ~[netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) ~[netty-3.6.2.Final.jar:na]
        at org.hbase.async.HBaseClient$RegionClientPipeline.sendUpstream(HBaseClient.java:2431) ~[asynchbase-1.4.1.jar:8c94a38]
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.2.Final.jar:na]
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.2.Final.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) [na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.6.0_27]
        at java.lang.Thread.run(Thread.java:679) [na:1.6.0_27]
Caused by: com.stumbleupon.async.DeferredGroupException: At least one of the Deferreds failed, first exception:
        ... 29 common frames omitted
Caused by: net.opentsdb.uid.NoSuchUniqueId: No such unique ID for 'tagv': [0, 23, -112]
        at net.opentsdb.uid.UniqueId$1GetNameCB.call(UniqueId.java:227) ~[tsdb-2.0.0.jar:781cb61]
        at net.opentsdb.uid.UniqueId$1GetNameCB.call(UniqueId.java:224) ~[tsdb-2.0.0.jar:781cb61]

[Sungho Park]

I tried the following command. But it is not recovered yet.

sudo /usr/share/opentsdb/bin/tsdb uid fsck fix

[Sungho Park]

I also failed deletion of the metric...

How can I recover the situation without modifying the source code ?

$ sudo /usr/share/opentsdb/bin/tsdb scan --delete 2000/01/01 sum xxxxxx

...
ab61e6b3 job=dea appid=1d9a2823add304d51a072073adc95e93 env=pro
2013-12-20 07:54:29,654 INFO  [main] TSDB: Flushing compaction queue
2013-12-20 07:54:29,660 INFO  [New I/O worker #1] HBaseClient: Lost connection with the -ROOT- region
Exception in thread "main" com.stumbleupon.async.DeferredGroupException: At least one of the Deferreds failed, first exception:
	at com.stumbleupon.async.DeferredGroup.done(DeferredGroup.java:169)
	at com.stumbleupon.async.DeferredGroup.recordCompletion(DeferredGroup.java:158)
	at com.stumbleupon.async.DeferredGroup.access$200(DeferredGroup.java:36)
	at com.stumbleupon.async.DeferredGroup$1NotifyOrdered.call(DeferredGroup.java:97)
	at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278)
	at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257)
	at com.stumbleupon.async.Deferred.callback(Deferred.java:1005)
	at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:450)
	at org.hbase.async.RegionClient.decode(RegionClient.java:1185)
	at org.hbase.async.RegionClient.decode(RegionClient.java:82)
	at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
	at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.hbase.async.RegionClient.handleUpstream(RegionClient.java:1008)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555)
	at org.hbase.async.HBaseClient$RegionClientPipeline.sendUpstream(HBaseClient.java:2431)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:679)
Caused by: net.opentsdb.uid.NoSuchUniqueId: No such unique ID for 'tagv': [0, 23, -108]
	at net.opentsdb.uid.UniqueId$1GetNameCB.call(UniqueId.java:227)
	at net.opentsdb.uid.UniqueId$1GetNameCB.call(UniqueId.java:224)
	... 25 more

On Friday, December 20, 2013 3:33:19 PM UTC+9, Sungho Park wrote:

[ManOLamancha]

On Friday, December 20, 2013 12:24:43 AM UTC-8, Sungho Park wrote:

I also failed deletion of the metric...

How can I recover the situation without modifying the source code ?

$ sudo /usr/share/opentsdb/bin/tsdb scan --delete 2000/01/01 sum xxxxxx

...
ab61e6b3 job=dea appid=1d9a2823add304d51a072073adc95e93 env=pro
2013-12-20 07:54:29,654 INFO  [main] TSDB: Flushing compaction queue
2013-12-20 07:54:29,660 INFO  [New I/O worker #1] HBaseClient: Lost connection with the -ROOT- region
Exception in thread "main" com.stumbleupon.async.DeferredGroupException: At least one of the Deferreds failed, first exception:

Hmm, that's a good question. We may need to modify the source to ignore the lookup on a scan with --delete so it would get rid of the data points for missing values.