Re: Re: hello:) on openwsn software architecture and libopen

[openwsn]

Hello, Weiwei

 

Since you're a CS Ph.D candidate, i suggest you do more on high level design and developing. It's not affordable for you to waste too much time on low level operations. it's ideal for you if you can integrate your thesis project and the open source project together.

 

the whole openwsn software can be divided into the following parts:

 

- openwsn library: running on ARM hardware now (you can port it to other platforms if you have interests)

     attention the final customer should develope an entire application based on this library. openwsn community doesn't require the application must be open source. we have give some demostrations in the "test" folder in the source code.

  for an entire wsn application and developing, we need at least the following 4 kinds of embedded applciations:

 

  1. fullnode: including sensing and routing functionality. (support routing)

  2. simplenode: can only interact with the fullnode. it is an ultra-weak node. similar to RFD device in ZigBee. no routing capability.

  3. sinknode: collect data from the network. openwsn targets to support multiple and mobile sink nodes. (still need careful design to achieve this goal) 

  4. sniffer node (useful for testing and developing)

    

- worldview: an GUI client based on libopen.

  the customer can view the data through this GUI application or send command to the wsn.

  we tongji group will not make too much effort on this application. because different user has different requirements. we cannot satisfy all of them. so we try to design a standard wsn interface. that's the born of "libopen"(formerly known as libsink).

 

- libopen

  it's the middleware between user's GUI application and the wsn network. the architecture is:

 

                     direct function call     rs232/usb                  wireless

  GUI Application <====> libopen <====> sink node <=====> other wsn nodes

     multiple                        multiple                 multiple                    from 10^0 to 10^4

 

 

for openwsn software on embedded node, we developed "hal" layer and "service" layer, and give some "application" layer examples in "test" folder. some of them are still not stable, but it support communication successfully now. at first, we had thought to implement a simple scheduler in the software, but in the end we give up and introduce uCOS. due to the license constraint, it's not a good choice. fortunately, we almost didn't use any functions in uCOS except hal_enter_critical()/hal_leave_critical(). so that we can give up uCOS in the future easily. another reason is uCOS is not a energy-friendly OS for wsn applications. you had to change its scheduler to make it acceptable in wsn applications.

 

in the future, we plan to integrate TinyOS as a TinyOS Service in openwsn. The tinyos service should be co-exist with other openwsn services. so we can enjoy from the functionality and tinyos community.  this isn't impossible. there's already an example by MANTIS project. MANTIS has successfully integrate TinyOS and bring a new version TinyMOS.  (we didn't choose MANTIS as our low level OS because it's not stable enough to support robus industry applications, but i do think MANTIS is wonderful)

 

we have implement RS232 manipulations in libopen. the old version of libsink has been tested and passed. but the architecture may not so clear. i'll do some revision on it. you can help me in this revision. libopen will have the following features in the end:

 

- an payload analysis API (known as BinaryXml format, but not tested. furthermore, the API is not complete)

- an data logging mechanism(log the received data onto disk and read them later. maybe an embedded database such as BerkeleyDB is a good choice).  (thread driven)  (not done)

- an data query refinement and optimization module. (not done)

   TinyDB is quite good. maybe we can get some reference from it. however, the API interface will not support SQL-like parsers. it's just an API.

  currently, we only need a API to tell the user "what" happend "when" and "where". the "when" and "where" parameter should be arbitrary value passed from the GUI application.

- an network API (support interaction with wsn network. it doesn't care the symantic meaning of the payload data. just sending and receiving) (done, needs heavy revision)

- an network simulation API. (some work has done in "simu" folder, not complete, needs testing )

   we need to provide a  simulation network to speed the developing of worldview GUI and other top level applications. it support the developers to start their work without a real wsn background. the simulation API can generate a virtual network topology and random payload or load real data from the disk to feed the simulation.

- an host driven routing (we have done it in another project using C#. so we need to port it into libopen.   need embedded software support.).

 

all our source code can be downloaded from http://code.google.com/p/openwsn/source

you can checkout it by TotoriseSVN to get the updated source code. i think you can start from the simulation and payload format analysis (BinaryXml).

 

pls tell me your google account. i'll add you so that you can commit your changes to the svn server.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

we use VS2005(ANSI C/C++) for libopen.   do not use VC specific features in the developing. we only adopt ANSI C/C++. furthermore, C is prefered because it's easy for porting to embedded devices.  pls make sure your work can be compiled successfully before commiting.

 

besides libopen, we also want to develope a driver for gumtree based on libopen. gumtree is an opensource data analysis tool similar to NI's product. it's one of the best eclipse plugin contribution in 2006. if the driver can be developed successfully, then we have a powerful open source GUI client to analyze data and take worldview instead. this is what i had communicate with Ma.

 

ps: i also interested in your QoS study. we'll integrate wire industry network and wireless network together. we have a industry field bus lab and have long history to use some control field bus such as Profibus and cclink. so the QoS problem is critical in our project. 

 

yours sincerely

 

 

---

openwsn

2007-05-25

---

发件人： weiwei fang

发送时间： 2007-05-25 16:20:55

抄送：

主题： Re: hello:)

[openwsn]

Hello, weiwei:

 

Q: Why TBinaryXml?

R:

let's consider the relationship between lower network layer and higher application layer.

 

network is quite like a truck. it only transmit the payload data to its destination. while, the payload format should be defined by the application. that's true.

then my questioni is: how to facilitate the developing of application to manipulate the payload buffer?

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

we can assume the payload is just an memory buf:

  payload = {buffer, buf_capacity}

 

then we had to consider how to put different kinds of information data into this buffer. this is not an easy task. because the data has their intrinsic internal relationships. XML's tree structure is a very good idea to describe the data item and data relationships than simple list. So i borrow the idea from XML. I think the best method to put all kinds data into payload buffer is using XML. we can develope a ultra-light XML parser to help us. By using the parser API, we can add a new node(=data item) into the buffer and then retrieve it out.  The node can be any value with any type.

 

  data_node = [parent, type, value, len]

  property "parent" is used to keep the tree relationship among nodes. "len" is the value's length.

 

we don't care the node type. it should be regulated by the application developer. such a xmlparser API targeted at helping the developer to organize their data items easily. not regulate the symantic meaning of each item.

 

however, the classical TEXT format XML is very inefficient in wireless transmission and storage. so I suggest BinaryXml. the data node is stored in their binary format directly.  now:

 

  data_node = [node_length, parent, data_type, data_value ]

 

we can deduce the data value's length from node_length.  we cannot use special TEXT tags such as <A></A> to judge a node. instead, we assume the first byte of a node in the buffer is its length. (i cannot remember clearly about it. the first three byte of a node are node_length, type, parent, but i cannot remember their order clearly. )

 

furthermore, we have such a rule:

  Rule: there must exist a root node in the buffer.  and we can get the buffer meta information (total buffer length) from the root node.

 

this is the key idea of TBinaryXml.

you see it has no relationship with WSN. it's just an utlity library to help you organized data in a binary buffer.  in most of other applications, the developer often assume the data items have unique fixed length, so we can simple using a structure array. TBinaryXml improve this because we can support variant data length and record their relationship now.

 

after introducing TBinaryXml, all the payload transmitted among nodes or even the host GUI will adopt it. the payload is essentially a BinaryXml node(root node) with some child nodes/child tree.  this will be easily for future extension. we needn't change our payload analyze program if we add or remove a new data item in it.

 

you can developed and test it seprately and merge with the main project after testing.

the old source code are for your idea reference. you can develope your own.

 

 

---

openwsn

2007-05-26

---

发件人： weiwei fang

发送时间： 2007-05-25 21:34:06

收件人： openwsn

抄送：

主题： Re: Re: hello:) on openwsn software architecture and libopen

 

Dr. Zhang, hello!
    Thanks for your explaination. I have imported the libopen into my VS2005 environment, and compiled it successfully. And I learned something about rtl_binaryxml.cpp. However, I think it is difficult to define a common api for the sensory data.

    Take the tinyos for example, you can look for the surge_reliable from crossbow for more. A sensor sends back some sensory data, and at the server side, the server use the xlisten to parse it, and get the real data from some formulation. However, when the sensor changes, the data parser should also be changed. It is not fixed, but changes with the sensor type and related data type. So you should left this for the users. You should just give a small example like surge_reliable and xlisten, but not try to define such function.

     Besides, what I should do after I get the codes, since I have no hardware to use.

2007/5/25, openwsn <ope...@gmail.com>:

--
Best Regards!

Weiwei Fang/方维维  Ph.D. Candidate
School of Computer Science & Engineering
Beihang University, Beijing, P.R.China.
Phone:(86)10-82315908
----------------------------------------------------------------

[alexmajy]

Hello Dr. Zhang,

I would like to do some conctribution about worldview which we'v
talked about before. I don't have VS2005, but it seems that the csharp
version of worldview is done. is there any compiled form or demo I
could download to try? Or I just need to install the big vs and do it
myself? thx~~~

On 5月26日, 下午12时47分, "openwsn" <open...@gmail.com> wrote:
> Hello, weiwei:
>
> Q: Why TBinaryXml?
> R:
> let's consider the relationship between lower network layer and higher application layer.
>
> network is quite like a truck. it only transmit the payload data to its destination. while, the payload format should be defined by the application. that's true.
> then my questioni is: how to facilitate the developing of application to manipulate the payload buffer?
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> we can assume the payload is just an memory buf:
> payload = {buffer, buf_capacity}
>
> then we had to consider how to put different kinds of information data into this buffer. this is not an easy task. because the data has their intrinsic internal relationships. XML's tree structure is a very good idea to describe the data item and data relationships than simple list. So i borrow the idea from XML. I think the best method to put all kinds data into payload buffer is using XML. we can develope a ultra-light XML parser to help us. By using the parser API, we can add a new node(=data item) into the buffer and then retrieve it out. The node can be any value with any type.
>
> data_node = [parent, type, value, len]
> property "parent" is used to keep the tree relationship among nodes. "len" is the value's length.
>
> we don't care the node type. it should be regulated by the application developer. such a xmlparser API targeted at helping the developer to organize their data items easily. not regulate the symantic meaning of each item.
>
> however, the classical TEXT format XML is very inefficient in wireless transmission and storage. so I suggest BinaryXml. the data node is stored in their binary format directly. now:
>
> data_node = [node_length, parent, data_type, data_value ]
>
> we can deduce the data value's length from node_length. we cannot use special TEXT tags such as <A></A> to judge a node. instead, we assume the first byte of a node in the buffer is its length. (i cannot remember clearly about it. the first three byte of a node are node_length, type, parent, but i cannot remember their order clearly. )
>
> furthermore, we have such a rule:
> Rule: there must exist a root node in the buffer. and we can get the buffer meta information (total buffer length) from the root node.
>
> this is the key idea of TBinaryXml.
> you see it has no relationship with WSN. it's just an utlity library to help you organized data in a binary buffer. in most of other applications, the developer often assume the data items have unique fixed length, so we can simple using a structure array. TBinaryXml improve this because we can support variant data length and record their relationship now.
>
> after introducing TBinaryXml, all the payload transmitted among nodes or even the host GUI will adopt it. the payload is essentially a BinaryXml node(root node) with some child nodes/child tree. this will be easily for future extension. we needn't change our payload analyze program if we add or remove a new data item in it.
>
> you can developed and test it seprately and merge with the main project after testing.
> the old source code are for your idea reference. you can develope your own.
>

> openwsn
> 2007-05-26

>
> 发件人： weiwei fang
> 发送时间： 2007-05-25 21:34:06
> 收件人： openwsn
> 抄送：
> 主题： Re: Re: hello:) on openwsn software architecture and libopen
>
> Dr. Zhang, hello!
> Thanks for your explaination. I have imported the libopen into my VS2005 environment, and compiled it successfully. And I learned something about rtl_binaryxml.cpp. However, I think it is difficult to define a common api for the sensory data.
>
> Take the tinyos for example, you can look for the surge_reliable from crossbow for more. A sensor sends back some sensory data, and at the server side, the server use the xlisten to parse it, and get the real data from some formulation. However, when the sensor changes, the data parser should also be changed. It is not fixed, but changes with the sensor type and related data type. So you should left this for the users. You should just give a small example like surge_reliable and xlisten, but not try to define such function.
>
> Besides, what I should do after I get the codes, since I have no hardware to use.
>

> 2007/5/25, openwsn <open...@gmail.com>:

> all our source code can be downloaded fromhttp://code.google.com/p/openwsn/source

> you can checkout it by TotoriseSVN to get the updated source code. i think you can start from the simulation and payload format analysis (BinaryXml).
>
> pls tell me your google account. i'll add you so that you can commit your changes to the svn server.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> we use VS2005(ANSI C/C++) for libopen. do not use VC specific features in the developing. we only adopt ANSI C/C++. furthermore, C is prefered because it's easy for porting to embedded devices. pls make sure your work can be compiled successfully before commiting.
>

> besides libopen, we also want to ...
>
> 阅读更多