Rhizome commandline problems

[®om]

Hi,

With 2 devices, I sent broadcast messages using MeshMS (one message from A and one message from B).
The messages are correctly received by the other.

But I have 2 problems with Rhizome in command line.

Problem 1: sometimes "rhizome list" does not work

I have 4 manifests on device A.

sqlite> select id, filehash from manifests;
id|filehash
14AAA069EB2CA36DED9F4F623D8C84BC0E9AD01F5D082CB0A05009B0CAE0C1B2|48A9797506D81D3F9FFA20E3FD8553951ED0913EFD310AC10AF846FB580EEB0A9A2EFD811F44A1983415029D007475C849EE20C84D9F4E96ADF0789805A354E7
F46F347CEFB9DECB066B6AAA493BBC24855E54C1B5427143061966B81AFE3C3D|30E68800BE49C3D79F048DFF9690DF3E03239557770490991C470D62014E7FA178F6832DF3B68A2C609B8DC23938A9A7D1C8CFEF2D9BC5B6B20AB42A2B0B80E4
9F817E8075850041F7CE43FFB7159F9C1F983E9C4B514FF2EB77BB0854FEC08F|D7FCB1E4B803C092EA236094F0825B3DE30040F9209DD419B529C32D77C3BD98E19B65EE9C18D7193C7C63FB03A5190D0E70296900902E785B479091BBF9EF4E
C3A6F23158727A49468349716BAB89720553A5BFC8F678839D7B6EC648332618|ED611240A2367FC2F6232A16191F4FE7230F62799A332A9142810DDB08F0056E9D61B70B3AA98EFC5268B6CACFFE0E9414B7AFF96BCE4C655513125652F5D73E

sqlite> select * from files;
id|length|highestpriority|datavalid|inserttime
48A9797506D81D3F9FFA20E3FD8553951ED0913EFD310AC10AF846FB580EEB0A9A2EFD811F44A1983415029D007475C849EE20C84D9F4E96ADF0789805A354E7|31|2|1|1364555583013
30E68800BE49C3D79F048DFF9690DF3E03239557770490991C470D62014E7FA178F6832DF3B68A2C609B8DC23938A9A7D1C8CFEF2D9BC5B6B20AB42A2B0B80E4|25|2|1|1364555585363
D7FCB1E4B803C092EA236094F0825B3DE30040F9209DD419B529C32D77C3BD98E19B65EE9C18D7193C7C63FB03A5190D0E70296900902E785B479091BBF9EF4E|29|2|1|1364555591877
ED611240A2367FC2F6232A16191F4FE7230F62799A332A9142810DDB08F0056E9D61B70B3AA98EFC5268B6CACFFE0E9414B7AFF96BCE4C655513125652F5D73E|25|2|1|1364555592691

sqlite> select * from fileblobs;
id|data
48A9797506D81D3F9FFA20E3FD8553951ED0913EFD310AC10AF846FB580EEB0A9A2EFD811F44A1983415029D007475C849EE20C84D9F4E96ADF0789805A354E7|
30E68800BE49C3D79F048DFF9690DF3E03239557770490991C470D62014E7FA178F6832DF3B68A2C609B8DC23938A9A7D1C8CFEF2D9BC5B6B20AB42A2B0B80E4|ͱC��ib\��)Lm"��N}l
D7FCB1E4B803C092EA236094F0825B3DE30040F9209DD419B529C32D77C3BD98E19B65EE9C18D7193C7C63FB03A5190D0E70296900902E785B479091BBF9EF4E|
ED611240A2367FC2F6232A16191F4FE7230F62799A332A9142810DDB08F0056E9D61B70B3AA98EFC5268B6CACFFE0E9414B7AFF96BCE4C655513125652F5D73E|��-�/��g��hX�

However, on the device A, "rhizome list" does not seem to work (other commands like "id self" work correctly):
# /data/data/org.servalproject/bin/servald rhizome list
13
_id:service:id:version:date:.inserttime:.author:.fromhere:filesize:filehash:sender:recipient:name

On the device B, "rhizome list" works:
# /data/data/org.servalproject/bin/servald rhizome list
13
_id:service:id:version:date:.inserttime:.author:.fromhere:filesize:filehash:sender:recipient:name
6:MeshMS1:F46F347CEFB9DECB066B6AAA493BBC24855E54C1B5427143061966B81AFE3C3D:1364560224182:1364560224182:1364560224280:53CDD96C9FFB6AFFFD7BCD8C03570331EAD051922F5E5850F15C1D08A8C17267:1:50:3E6ACC29D8F21E1164B8088B2909EC87142D68BDF2867E094FFBBD238CF6EF2E57ACA9043ABF29D8FD37799756031AADD1505F1556F525A0BEEE59EAA77370CF:53CDD96C9FFB6AFFFD7BCD8C03570331EAD051922F5E5850F15C1D08A8C17267:E28E28A12F2771CB5548BA43C172A86A374279C331F513754A6C8818D213A862:
5:MeshMS1:14AAA069EB2CA36DED9F4F623D8C84BC0E9AD01F5D082CB0A05009B0CAE0C1B2:1364560161174:1364560161174:1364560223442::0:59:4631D795341A895537536C4B2213DF252BE94E79004A9311031BA83D0FF759A6A1F6C467739C9E8A0A0F4C1B60C1C705FD1EE090FBF7D941FD14DF08FE2F43BB:E28E28A12F2771CB5548BA43C172A86A374279C331F513754A6C8818D213A862:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:
4:MeshMS1:C3A6F23158727A49468349716BAB89720553A5BFC8F678839D7B6EC648332618:1364555592295:1364555592295:1364555654624::0:25:ED611240A2367FC2F6232A16191F4FE7230F62799A332A9142810DDB08F0056E9D61B70B3AA98EFC5268B6CACFFE0E9414B7AFF96BCE4C655513125652F5D73E:E28E28A12F2771CB5548BA43C172A86A374279C331F513754A6C8818D213A862:53CDD96C9FFB6AFFFD7BCD8C03570331EAD051922F5E5850F15C1D08A8C17267:
3:MeshMS1:9F817E8075850041F7CE43FFB7159F9C1F983E9C4B514FF2EB77BB0854FEC08F:1364555652304:1364555652304:1364555652446:53CDD96C9FFB6AFFFD7BCD8C03570331EAD051922F5E5850F15C1D08A8C17267:1:29:D7FCB1E4B803C092EA236094F0825B3DE30040F9209DD419B529C32D77C3BD98E19B65EE9C18D7193C7C63FB03A5190D0E70296900902E785B479091BBF9EF4E:53CDD96C9FFB6AFFFD7BCD8C03570331EAD051922F5E5850F15C1D08A8C17267:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:

Problem 2: I am not able to extract decrypted messages

So I use my device B. The data associated to the manifests with _id 3 and 5 (having recipient ~0 (0xFF..FF)) are unencrypted:
# /data/data/org.servalproject/bin/servald rhizome extract file 14AAA069EB2CA36DED9F4F623D8C84BC0E9AD01F5D082CB0A05009B0CAE0C1B2 tempmanifest
service:MeshMS1
manifestid:14AAA069EB2CA36DED9F4F623D8C84BC0E9AD01F5D082CB0A05009B0CAE0C1B2
version:1364560161174
inserttime:1364560223442
.readonly:1
filesize:59
filehash:4631D795341A895537536C4B2213DF252BE94E79004A9311031BA83D0FF759A6A1F6C467739C9E8A0A0F4C1B60C1C705FD1EE090FBF7D941FD14DF08FE2F43BB

The content of extracted manifest give more information than printed on stdout:

# cat tempmanifest
BK=F9E54C359D2648FD47EA2A6C40B63D41253C946D0AB9EDC6ABE49D707BB5F4EF
crypt=0
id=14AAA069EB2CA36DED9F4F623D8C84BC0E9AD01F5D082CB0A05009B0CAE0C1B2
recipient=FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
sender=E28E28A12F2771CB5548BA43C172A86A374279C331F513754A6C8818D213A862
service=MeshMS1
filesize=59
date=1364560161174
version=1364560161174
filehash=4631D795341A895537536C4B2213DF252BE94E79004A9311031BA83D0FF759A6A1F6C467739C9E8A0A0F4C1B60C1C705FD1EE090FBF7D941FD14DF08FE2F43BB
�g�&���O=�=fqᕞ"y
                u���7�B�� )��☃�└b=����←��P     ��� #

As you can see, crypt=0. However, data seems encrypted, the resulting file is binary:
# hd tempfile                                                                 
00000000: 00 1a 02 00 00 01 3d b5 d9 cd 2d 00 05 32 32 32 s 37d
00000010: 32 32 00 01 2a 00 06 67 6e 65 78 75 73 00 1a 00 s 349
00000020: 17 02 00 00 01 3d b6 1f a8 ba 00 05 32 32 32 32 s 35b
00000030: 32 00 01 2a 00 03 61 61 61 00 17 s 19a
sum bbb

Do I need --keyring-pin or --entry-pin for extracting it? How to know these "pin"?

Thank you for your help.
®om

[®om]

I read the parsing code, and I now understand the content (for problem 2). I explain if someone needs it.

Here was my message log (given on my previous topic):

00000000: 00 1a 02 00 00 01 3d b5 d9 cd 2d 00 05 32 32 32

00000010: 32 32 00 01 2a 00 06 67 6e 65 78 75 73 00 1a 00

00000020: 17 02 00 00 01 3d b6 1f a8 ba

00 05 32 32 32 32
00000030: 32 00 01 2a 00 03 61 61 61 00 17

The two last bytes is a short representing length: 0x0017 (23 in decimal).
So we count -23 bytes from the end (-5 for headers, 2×2 for length + 1 for switch_byte), the length is written here too (0x0017).
The next byte is 0x02, which is RhizomeMessage.SWITCH_BYTE, so it is a rhizome message.
Then, the 8 following bytes are timestamp (0x0000013db61fa8ba), which is today:
date -d @$(echo 'div=1000;obase=10;ibase=16;13DB61FA8BA/div'|bc)
Fri Mar 29 13:29:20 CET 2013

Then the senderDID is in UTF.
The 2 first bytes is the length of DID (0x0005), which is only 5 spaces (0x3232323232).
Then the recipient DID in UTF: length=0x0001, value=0x2a (42, the answer, which is the ASCII code of '*' set in BroadcastPeer).
Then message is then written in UTF: length=0x0003, value=0x616161 ("aaa").

"aaa" was my message. So everything is OK, it is not encrypted.

However, I have no explanation for problem 1.

[Paul Gardner-Stephen]

Hello Rom,

Rhizome list and related commands can fail if the servald currently has the database locked.   

This is an sqlite limitation.

That said, we try to make sure the servald process does not hold any database locks for a long time, but our efforts don't seem to be perfect yet.

If you repeat the command, it will often work.

So I am curious as to what you are working on in this area?

(We are also still trying to schedule review/merging your existing commits as soon as we get the chance).

Paul.