[Contiki-developers] csma operation with full queue

172 views
Skip to first unread message

Mohammad Abdellatif

unread,
Dec 15, 2011, 9:34:31 AM12/15/11
to Contiki developer mailing list
hey all
iam using csma in a network where packets are always generated and sent to the sink.
when the nodes in the middle have their queue full they broadcast the new received packet directly "normal"
however since more packets are always coming, i find out that as soon as the queue gets full, it will never be emptied and new packets will always be directly sent. which affects the performance of the whole system.
i think there should be a mechanism  to avoid this
has anyone faced this problem before, i think there is a bug here somewhere but i cant put my hand on it just yet.
any help is welcomed,
Thanks,

Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 


David Kopf

unread,
Dec 15, 2011, 9:46:10 AM12/15/11
to Contiki developer mailing list
Can you increase the queue size? But if the high packet rate is a steady state the queue will always get full eventually.

Mohammad Abdellatif


------------------------------------------------------------------------------
10 Tips for Better Server Consolidation
Server virtualization is being driven by many needs.
But none more important than the need to reduce IT complexity
while improving strategic productivity. Learn More!
http://www.accelacomm.com/jaw/sdnl/114/51507609/

_______________________________________________
Contiki-developers mailing list
Contiki-d...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/contiki-developers

------------------------------------------------------------------------------
10 Tips for Better Server Consolidation
Server virtualization is being driven by many needs.
But none more important than the need to reduce IT complexity
while improving strategic productivity. Learn More!
http://www.accelacomm.com/jaw/sdnl/114/51507609/
_______________________________________________
Contiki-developers mailing list
Contiki-d...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/contiki-developers

Mohammad Abdellatif

unread,
Dec 15, 2011, 9:53:45 AM12/15/11
to Contiki developer mailing list
i increased it and it still gets full. and if i increase it more than 20 i have compiling error for the sky motes.

my question is that, shouldn't there be a limit on how much time a packet stays in the queue?
the packet should not be dropped only if it was transmitted or passed the max retransmit times


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Simon Duquennoy

unread,
Dec 15, 2011, 1:16:16 PM12/15/11
to contiki-d...@lists.sourceforge.net
If you use the latest csma.c (the git repos, not the 2.5 release), you can get very long queues (hundreds of packets) by enabling queuebuf storage in Coffee. Look at examples/udp_stream/project-conf.h for a configuration example.
However, as David wrote, this will be useful only if your data rate isn't steady. If it is steady, you have to work on improving the throughput of your network or decreasing the application data rate.

Cheers,
Simon Duquennoy, Postdoc @ SICS
http://www.simonduquennoy.net

Mohammad Abdellatif

unread,
Dec 15, 2011, 7:25:45 PM12/15/11
to Contiki developer mailing list
thanks for the tip Simon.. could you send me the link for this repo because i cant seem to find it
thanks,

Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Simon Duquennoy

unread,
Dec 16, 2011, 4:05:12 AM12/16/11
to contiki-d...@lists.sourceforge.net
------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for 
developers. It will provide a great way to learn Windows Azure and what it 
provides. You can attend the event by watching it streamed LIVE online.  
Learn more at http://p.sf.net/sfu/ms-windowsazure

Mohammad Abdellatif

unread,
Dec 16, 2011, 8:08:50 AM12/16/11
to Contiki developer mailing list
i have tested the updated version and while i am now able to increase the queue size, i found out that the performance became worse.
when enabling the debug in the csma file, i find out that it says:

csma: could not allocate packet, dropping packet

without the queue getting full.. i had to reduce the sending rate much much more than earlier "with the older csma version" in order to avoid this
i dont know what makes the csma drop the packet when the queue is not full?
anyone has any suggestion?


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Simon Duquennoy

unread,
Dec 16, 2011, 11:33:05 AM12/16/11
to contiki-d...@lists.sourceforge.net
This message is supposed to appear when the queue is full.
1) How do you know it isn't?
2) What configuration do you use? If you enabled queueing in flash, you have to set COFFEE_CONF_DYN_SIZE as done the project-conf.h of examples/udp-stream

Regards,

Simon Duquennoy, Postdoc @ SICS
http://www.simonduquennoy.net

Mohammad Abdellatif

unread,
Dec 16, 2011, 1:49:05 PM12/16/11
to Contiki developer mailing list
This message is supposed to appear when the queue is full.
No there is another message that says that the queue buffer is full not his one.


1) How do you know it isn't?
because it says the length of the queue before this and it is no where near full


2) What configuration do you use? If you enabled queueing in flash, you have to set COFFEE_CONF_DYN_SIZE as done the project-conf.h of examples/udp-stream
I havent tested this yet but i will and see what happens
thanks for your help


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Mohammad Abdellatif

unread,
Dec 21, 2011, 8:58:20 AM12/21/11
to Contiki developer mailing list
the
Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 

i have followed the configuration in the udp-stream example and  am still getting the same problem.
nodes drop packets because they cant allocate them in the memory, even though the queue is not full!
i dont know where is this problem is coming from and even with a very low rate it still happens.

any one knows why this is happening?
thanks
Mohammad

David Kopf

unread,
Dec 21, 2011, 9:14:45 AM12/21/11
to Contiki developer mailing list
Do you have debug printfs in csma.c? When a node starts up using contikimac
there are a lot of broadcast strobes, and with rpl they fly back and forth
for a few seconds. That could be the trigger for the queue filling. Are you
using a recent git? there was a problem with the callback that unloaded
packets a couple of months ago.

the
Mohammad Abdellatif


Mohammad Abdellatif

Regards,

Mohammad Abdellatif

http://sourceforge.net/projects/contiki/develop

Mohammad Abdellatif

Mohammad Abdellatif

------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create
new or port existing apps to sell to consumers worldwide. Explore the
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev

_______________________________________________
Contiki-developers mailing list
Contiki-d...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/contiki-developers


------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create
new or port existing apps to sell to consumers worldwide. Explore the
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev

Mohammad Abdellatif

unread,
Dec 21, 2011, 9:28:17 AM12/21/11
to Contiki developer mailing list
i have just cloned the code from the repo two days ago.
i am showing the pritfs from csma and i can see that the packets r being dropped while the queue is not full.
going through the csma.c i can see that this print is triggered when q = memb_alloc(&packet_memb); returns NULL

i am not using contikimac.. i am using sicslowmac.. rpl operation is working ok.. the problem happens when i start sending data..
the problem is that it happens randomly..
Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




David Kopf

unread,
Dec 21, 2011, 10:11:12 AM12/21/11
to Contiki developer mailing list
Well that is the indication that the queue is full, unless there is some dangling pointer or stack overflow that messes up the memb
list or pointer.
The memb_alloc routine just does

for(i = 0; i < m->num; ++i) {
if(m->count[i] == 0) {
/* If this block was unused, we increase the reference count to
indicate that it now is used and return a pointer to the
memory block. */
++(m->count[i]);
return (void *)((char *)m->mem + (i * m->size));
}
}

the
Mohammad Abdellatif


Mohammad Abdellatif

Regards,

Mohammad Abdellatif

http://sourceforge.net/projects/contiki/develop

Mohammad Abdellatif

unread,
Dec 21, 2011, 10:38:42 AM12/21/11
to Contiki developer mailing list
but there is another exit condition in the csma file that deals if the queuebuf is full
q->buf = queuebuf_new_from_packetbuf();
and if this q in NULL it means that the queue is full
and this never happens.. because the packet is dropped before it goes there

 i think something is messing with the memb like u say but i dont know how to catch it.. i used the stack watcher in cooja and no stack overflow happened..
do you know of anyway to check what is messing with it?
thanks

Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




David Kopf

unread,
Dec 21, 2011, 11:10:27 AM12/21/11
to Contiki developer mailing list
Well a full queue is consistent with random problems. There are several queues involved, and the debug prints are not distinct in
every case. Did you enable debug in quebuf.c?

Mohammad Abdellatif

unread,
Dec 21, 2011, 12:08:34 PM12/21/11
to Contiki developer mailing list
after enabling the debug in queuebuf.. i can see that the queue gets full.. however csma still says that the queue is not full which i cant seem to understand why.


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




David Kopf

unread,
Dec 21, 2011, 1:47:39 PM12/21/11
to Contiki developer mailing list
Do your own scan for free slots before adding and after removing an entry. A stack overwrite would make that number change
abruptly.

Mohammad Abdellatif

unread,
Dec 22, 2011, 9:15:07 AM12/22/11
to Contiki developer mailing list
i am pretty sure this is the problem.. can u help me with doing that if u have the time?
thanks,


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




David Kopf

unread,
Dec 22, 2011, 10:12:30 AM12/22/11
to Contiki developer mailing list
Do e.g. $ objdump -t --section=.bss webserver6.avr-raven. On avr builds the queue buffers are in the middle of .bss so stack
overflow would affect other things first (the last entry is uip_udp_conns).

However writing off the end of packetbuf would set the bufmem count array to show no free entries. Do you memcpy or sprintf to that
using a variable size?

00800cba l O .bss 00000002 packetbufptr
00800cbc l O .bss 000000b2 packetbuf_aligned
00800d6e l O .bss 0000000f bufmem_memb_count
00800d7d l O .bss 0000001e bufmem_memb_mem
00800d9b l O .bss 00000002 refbufmem_memb_count
00800d9d l O .bss 0000006a refbufmem_memb_mem
00800e07 l O .bss 0000000f buframmem_memb_count
00800e16 l O .bss 00000c4e buframmem_memb_mem
00801a68 l O .bss 00000002 lastport

-----Original Message-----
From: Mohammad Abdellatif
Sent: Thursday, December 22, 2011 9:15 AM
To: Contiki developer mailing list
Subject: Re: [Contiki-developers] csma operation with full queue

i am pretty sure this is the problem.. can u help me with doing that if u have the time?
thanks,

Mohammad Abdellatif


Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN
http://win.inescporto.pt/mma

On Wed, Dec 21, 2011 at 6:47 PM, David Kopf <dak...@embarqmail.com> wrote:
Do your own scan for free slots before adding and after removing an entry. A stack overwrite would make that number change
abruptly.


-----Original Message-----
From: Mohammad Abdellatif

Sent: Wednesday, December 21, 2011 12:08 PM
To: Contiki developer mailing list
Subject: Re: [Contiki-developers] csma operation with full queue

after enabling the debug in queuebuf.. i can see that the queue gets full.. however csma still says that the queue is not full which
i cant seem to understand why.

Mohammad Abdellatif

unread,
Dec 22, 2011, 1:52:08 PM12/22/11
to Contiki developer mailing list
when i send i just use uip_udp_packet_sendto .. it should handle all the packetbuf work. right?


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Mohammad Abdellatif

unread,
Dec 22, 2011, 2:00:19 PM12/22/11
to Contiki developer mailing list
i still dont know why does the prints from queuebuf.c and from csma.c shows different sizes of the queue buffer.. they decrease together when the csma does a retransmission.. however when this problem happens, the size from the queubuf.c increases while the size from the csma.c stays the same!!!!
as soon as the size reaches the max, csma start saying that it cant allocate the packet.


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Mohammad Abdellatif

unread,
Dec 27, 2011, 11:34:47 AM12/27/11
to Contiki developer mailing list
Iam beginning to think that the problem is related to the neighbor queue
the node sends the packets from one queue only while the the other queue backs up until it is full which causes the problem..
in my case, each nodes r arranged in a row, what happens is that the neighbor queue in the direction of the sink is not full while the other one is "in the direction of the other nodes" , then this node cant forward the packets to the sink however it can still generate its own packets and reply to messages coming from the sink.

can any one tell me where is the function responsible for sending queued packets and how does it select which neighbor queue to send from first?

thaks,


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Mohammad Abdellatif

unread,
Dec 27, 2011, 3:02:03 PM12/27/11
to Contiki developer mailing list
i think i managed a work around it..
in csma.c
the function free_first_packet
in the end after the main queue is finished and freed.. i added this in order to send any remaining packets in the other queue.
      else {
      /* This was the last packet in the queue, we free the neighbor */
      ctimer_stop(&n->transmit_timer);
      list_remove(neighbor_list, n);
      memb_free(&neighbor_memb, n);
     
// my addition
      struct neighbor_queue *n = list_head(neighbor_list);
      if(n != NULL) {
      transmit_packet_list(n);
}
//
   } 

so far the problem hasnt appeared, but i am still doing some tests.
will keep u posted


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Simon Duquennoy

unread,
Jan 2, 2012, 4:37:48 AM1/2/12
to Contiki developer mailing list
Hi,

As there is one transmit_timer per neighbor queue, you shouldn't have to do this. In the code snippet you're showing, the timer of the current neighbor is stopped before freeing the neighbor structure. Other neighbor queues remain active, and their timer will eventually trigger.
To answer the question in your previous email, there is no centralized decision selecting the next neighbor to send to. Instead, each neighbor has its own ctimer implementing the csma backoff, and the transmissions are triggered independently.

Regards,
Simon

Mohammad Abdellatif

unread,
Jan 2, 2012, 1:44:18 PM1/2/12
to Contiki developer mailing list
yeah i guess u r right.. however, what i do is a kinda of a speedup to the process. instead of waiting for the other queue's timer to expire.
it is still did not solve the problem completely, i will try to do more work on it and keep u posted if i reach something solid.

thanks,
 
Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox

Mohammad Abdellatif

unread,
Jan 18, 2012, 7:56:14 AM1/18/12
to Contiki developer mailing list
Hey
i am still having the same error
the csma freezes at some point "random" and so the queue gets full
it sends new generated packets however it doesnt  forward the ones who are already in the queue and drops any new incoming packets.
i tried reducing the sending rate to 1 packet per minute and still this error sometimes happen.
not sure what to do more.. any ideas?


Mohammad Abdellatif

Phd student at Faculty of Engineering University of Porto (FEUP)
Researcher at Inesc Porto, UTM, WiN 




Reply all
Reply to author
Forward
0 new messages