Improving(Understanding) the xip tool

[pranjan...@gmail.com]

Hello Community,
                               I'm Pranjan Sana from Kolkata,India and i'm having a hard time understanding the xiaconf repository.I'm fairly profiecient in c code.My guess is this stems from my inexperience in linux kernel networking(?).
Could you suggest a more specific starting point for me? The prerequisites for playing around with the Xip tool? or a starting point in the xiaconf repository?
          
I've read Michel's paper and gained an overview,but cant figure out how the raw internals work.

[Hrishav Mukherjee]

Hi Pranjan,

I'm not a mentor, they'll surely provide you better help. But if by "raw internals" you mean linux networks functionality, then the book "Understanding Linux Network Internals"  would be a great resource. 

Regards

[pranjan...@gmail.com]

Has this group become suddenly inactive? been days,still no reply ! Have any of the new members made any contact with the mentors in the last 3-4 days?

[saurav kumar]

Hi Prajan,
Glad to hear about your interest. Generally mentors try to reply as fast as they can but sometimes they are busy with the work for organization and are unable to reply as quickly as you expect them to.
About your queries i would say that you don't need to look at the networking internal to understand xiaconf as it just sends rtnl messages to the kernel. I would suggest first reading upon rtnl msgs and then try going through some principal's xip file like xiphid.c inside xip and then working your way up the directory.
If there is any query you can post it here or mail me on sauravda...@gmail.com.
Best,
Saurav.

[pranjan...@gmail.com]

Thank you Saurav,
I do realize the mentors are busy.Thanks for the heads up,means a lot since I was totally lost.I'd bother you again with further queries.
Cheers,
Pranjan.

[pranjan...@gmail.com]

Hi mentors,
I've read about the rtnetlink messages from the man pages,and from the "How to write a principal" article in Wiki got a fair understanding of xipad.c.
However i can't figure out the xiphid.c file,although it looks quite similarly structured as xipad.c.

1) Can't understand what a hid_file is. Same with final_file_name(tmp_ffn,prv_ffn)
2)  Can't understand the function of different keys and their interaction with hid_file.I have gone through openssl library.

Please help me with relevant resources to go through. A short description of how this directory works would be nice

[saurav kumar]

Hi Pranjan,
I'm attaching a file which i used last year to get some basic description please browse through it if you want to know more about rtnetlink. Hope it helps.
1)HID XIDs are the hash of public keys of the machines they name. If you have read about the concept of public and private key, then you can probably relate or you can read more about it on wikipedia. So when a new hid is to be added,a new private key is created and then written in the file tmp_ffn which is then renamed to prv_ffn. Try and add an HID XID over your virtual machine and see what it returns. (https://github.com/AltraMayor/XIA-for-Linux/wiki/How-to-set-up#HID_Principal).
2) I think if you try and learn about private and public key communication mechanism, then you might be able to understand it better. You can find the tutorial for the same real easy, just learn about the basic mechanism and come back to the xiphid code.
Best,
Saurav.

[pranjan...@gmail.com]

Hi mentors,

I've gone through the xiphid.c file again and finally understood that part,After reading through your suggestions.

I've made a rough pen and paper documentation of my understanding of that file,which would get me through that file for future uses.

Although I understood addneigh | delneigh |showneighs and in xiphid.c , I dont see how it works overall .Shouuld a thorough read of this link make it better?

“https://nscpolteksby.ac.id/ebook/files/Ebook/Computer%20Engineering/Linux%20Kernel%20Networking%20-%20Implementation%20(2014)/chapter%207%20Linux%20Neighbouring%20Subsystem.pdf”

It’d be great if you give me a brief descriiption of neighbouring subsystems.

Also,that i have finished your initial recommendations of "first reading upon rtnl msgs and then try going through some principal's xip file like xiphid.c inside xip and then working your way up the directory.”,what should be my next course of action?

Please help me get a better understanding of the problem statement?

[Qiaobin Fu]

Hi Pranjan,

For the neighboring subsystem, briefly, it's responsible for discovering the nodes on the same link, and maintain the L3-to-L2 mapping. As Saurav suggested, you don't need to go over the details of the neighboring subsystem.

For the next steps, please follow our project description, which precisely documents what should be done. Basically, you need to find ways to refactor our xip code, and make it simpler, more extensible and flexible using libmnl. Also, please alter xip to automatically load the kernel modules of requested principals, the kni example in Gatekeeper is a good start point for this task. Reading the current source code of xip can help a lot.

Best,

Qiaobin

[saurav kumar]

Hi Prajan,
Adding on to the answer of Qiaobin, for the first point you need to work on finding the duplications among various principal. Focus on finishing this first by going through files like xipad, xipether, so that you can get to know what functions could be moved and the basic functionalities common to all.

[pranjan...@gmail.com]

Hid and ad principal has fairly commented out the dupicate functions


is the HID principal deprecated by Ethernet principal? if its not,They share some common funtions like print_neigh,show_neigh,add/del_neigh.

xiart.c lists some common rtnetlink  functions used all over the codebase,
like print_route,xrt_list_rt_redirect,dump.xrt_modify route. But these have been repeatedly duplicated in other files.Any specific reason for this?

xdp and serval are analogous to UDP and TCP i'd think. They share functions like dummp,print_socket,do_addroute,do_delroute,do_show

rtnetlink code is heavily used in these duplicated code,which i understand needs to be replaced by libmnl code. So should I go about making a library for these,without changing them to libmnl?

as this duplication is spread throughout the xip ,suggest me a starting point in the points i've highlighted?

I've mapped out all trivial duplications on pen and paper.

[saurav kumar]

Hi Pranjan,
I'm not sure if the hid principal is going to be deprecated this year or the next. But i would suggest you to mark out the functions with a similar functionality and also pay attention to the slight changes between some of them.
For the xiart functions, can you provide me some examples of where the functions have been duplicated inside the principal files?
Yes, xdp and serval are counterparts of udp and tcp as written in the xia documents.
I couldn't get your query related to rtnetlink as it seems contradictory that first you understand the aim of replacing the rtnetlink code with libmnl and then you are asking if you could just compose a library comprising of rtnetlink functions that have been duplicated?

[pranjan...@gmail.com]

"For the xiart functions, can you provide me some examples of where the functions have been duplicated inside the principal files?"

Xiart functions are

1. xrt_modify_route /Based on iproute2/ip/iproute.c:iproute_modify. /

is fairly similar with modify_local (in xipether.c and xipu4id.c)

modify_addr(in xip hid.c)

modify_neigh(in xiphid.c and xipether.c)

defining a filter for the routing table dump and the dump function itself is pretty common in the codebase,with some minor variations.

2. static int dump(__u32 tbl_id, xid_type_t ty, rtnl_filter_t print)

is fairly similar with static int showaddrs(void) (in xiphid.c) [with a minor change in do_showaddrs we can use a common dump function maybe]

and similar with static int dump(void) (in xipu4id.c)

and static int dump(__u32 tbl_id, xid_type_t ty, rtnl_filter_t print) (in xipserval)

and static int dump(__u32 tbl_id, rtnl_filter_t print) (in xipxdp)

3. static int print_route(const struct sockaddr_nl *who, struct nlmsghdr *n,void *arg)

is duplicated in xipad.c and has some similarity with the print_route in xipu4id

print_addr in xiphid.c also shares a similar structure

static int print_socket(const struct sockaddr_nl *who, struct nlmsghdr *n.void *arg) in xipxdp,xipserval also shares a similar structure, With minor changes we might implement a general one for all these xip

files.

I am attaching some of the differences/similarities in comparisons

[Michel Machado]

Hi Pranjan,

Try to package your finding into your proposal, and explain how you'll take advantage of them to reorganize the code.

--
You received this message because you are subscribed to the Google Groups "Linux XIA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-xia+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[pranjan...@gmail.com]

I can see the library i'd try to implement ofcourse eliminates duplication,But i don't know whether the code gets messier for people usually accustomed to monolithic structures of the xip codes.I'd upload something by tonight that shows how I plan to go about this implementation

I don't think there won't be enough time to detail my approach for changing rtnetlink code to libmnl code before the deadline of my proposal. So could you drop a few hints about the really important things i should do,to give an overall understanding of how to implement this change? at least the relevant directories in Net-eval i need to go through?

I've read the libmnl documentation.And i'm fairly skilled in socket programming.

Since i'm a bit short on time (i'm sorry) any suggestions at all would be helpful.

[Cody Doucette]

Hi Pranjan,

I would look at the libmnl documentation, especially the API, and try to figure out what functions it provides that will replace function calls in the xip code. Then look in the xip tool for those function calls, so you can discuss in your proposal which function calls will have to be replaced, and where they are in the code.

Best,

Cody

--

You received this message because you are subscribed to the Google Groups "Linux XIA" group.

To unsubscribe from this group and stop receiving emails from it, send an email to linux-xia+unsubscribe@googlegroups.com.

[pranjan...@gmail.com]

I realise how awfully late it is,but please answer my doubt regarding the automatically load the kernel modules of requested principals at runtime.
 I've analyzed how the init_kni() loads the kernel module.similarly we can design a init_ppal_kmod(const char *ppal_kmod_path),it's takes as input the path of requested principal in kernel,grab_file method reads the kernel bin file into a buffer and returns the buffer,
Now init_module(void *module_image, unsigned long len, const char *param_values)
 loads the  ELF image in the file(buffer) into kernel space, performs any necessary symbol relocations, initializes module parameters to values provided by the caller, and then runs the module's init function

The param_values argument is a string containing  specifications of the values for module parameter which the kernel parses  and initializes the specified parameters.

Can you validate the above and give me suggestions?

[Sachin Paryani]

Hi Pranjan,

You're thinking in the right direction but you will have to be a little more specific. For example, wow would you ensure that when a user tries manipulate a principal, the corresponding module is automatically loaded? Since there isn't too much time left before the proposal submission deadline, I'd suggest that you focus on the parts that you are sure of, in your proposal. Good luck!

Best,

Sachin  

[pranjan...@gmail.com]

I have submitted a draft proposal titled "Improving the xip tool" in the category xia mininet, comments would be quite helpful.

[pranjan...@gmail.com]

anything at all would be helpful

[Sachin Paryani]

Hi Pranjan,

I have left a few comments on your draft proposal. Please ensure that your turn your final proposal in at least an hour before the deadline.

Good luck :)

Best.

Sachin 

On Tue, Mar 27, 2018 at 8:27 AM, <pranjan...@gmail.com> wrote:

anything at all would be helpful

[Sachin Paryani]

You'll find out soon :)

On Tue, Mar 27, 2018 at 11:01 AM, Pranjan Sana <pranjan...@gmail.com> wrote:

Thank you,much appreciated.I will do that. The formality apart,did that proposal seem to have realistic chances? 

[pranjan...@gmail.com]

HI Sachin,

Wasn't that soon.

But I guess I'm selected now :)

May I contact you on your  personal email?
Cheers,
Pranjan.

[Sachin Paryani]

Hi Pranjan,

Congratulations and welcome to the Linux XIA family! Watch out for your inbox, I'll be sending you an email very soon. 

Best,

Sachin