MicaZ Radio Communication Comparison
M.Schippling -- v0.1 -- 1/23/2006
schip@etantdonnes.com


Introduction

This is a brief extenstion to my exploration into the communication speed and reliability of the Mica2 Motes from Xbow technologies running the TinyOS embeded operating system. The original detailed report is at:
This page adds the results of some basic tests with the Micaz, which uses a Chipcon CC2420 RF tranceiver chip operating at 2.4GHz, and a comparison of the 'Z to the '2 in the same environment. It is not as detailed because I only have two Micaz's to play with.

See the MIca2 report for a description of the test system and methodology. The only differences are that these tests were done with TOS version 1.1.10, due to the fact that the Micaz is not entirely operational in 1.1.7, and the antenas used on both the 'Z and '2 are the wire whips supplied by Xbow attached via MMCX connectors. The environment is my spare bedroom rather than the attic because it's too cold to go upstairs now. The radio settings for the 'Z were:

Experiments

As in the Mica2 report the experiments explored the effects of the inter-message delay time and the use of the TOS message ACK mechanism. Since there were only two devices available, interdevice CSMA interference and the difference between Broadcast and RoundRobin messaging were not explored.
To review, the variables examined were:
This is the delay from the end of one send to the beginning of the next. In some situations (e.g., when ACKs are used) the send may take more or less time so this is not a fixed loop time. The Micaz's operate at a considerably higher message rate which makes timing problematic (see note below).
This also had some significant and surprising effects. As it stands the robot communication system is using ACKs to try to tame the unreliable nature of the Mica2 radio. However this may only be a good idea with Micaz's.

Timing

The Java system under Windows XP has a timer resolution of 10ms. The Micaz's apparently send and receive messages in under 10ms -- I did not attempt to determine the actual value. Therefore time measurements are rather ad hoc and should not be take as anything but indicative.

Table Field Description

The fields in the tables below are:
  1. Delay Time -- Message cycle throttle time as described above.
  2. msg/sec Total -- Total attempted message cycles (number of transmits from base-station).
  3. msg drop % -- Percent of messages which did not receive a reply.
  4. msg/sec Success -- Total successful messages per second.
  5. noACK % -- Percent of transmitted messages that did not receive an ACK at the base-station.
  6. msg miss % -- Percent of messages that were not received at the re-Mote
Remember that "Successful Messages" here means message cycles of one send and one reply, so the actual on-the-air number of messages is twice that shown.


Results

Message Cycle Using ACKs

In this set of experiments messages are sent point-to-point with ACKs enabled. The cycle is driven by the host using various inter-message delay times as shown:

MicaZ
Delay
Time
 msg/sec
Total
 msg drop
     %
 msg/sec
Success
 noACK
  %
msg miss
     %
 Notes
0
 65.93 23.02
 50.75 0.01
 0.0
 2m apart
10
 49.73
 1.6
 48.94 0.13
 0.03
 2m apart
20
 33.29
 0.0
 33.29 0.0
 0.0
 2m apart
20
 29.49
 10.37
 26.41
 9.67
 4.8
 5m apart, bad reception

Mica2
Delay
Time
 msg/sec
Total
 msg drop
     %
 msg/sec
Success
 noACK
  %
msg miss
     %
 Notes
0
 20.93 49.02
 10.67 76.45
 44.4
 2m apart
50
 11.11
 1.17
 10.98 12.52
 1.12
 2m apart
50
 11.07
 1.93
 10.86
 14.18
 1.75
 10m apart, good reception


The above shows the relationship between the message delay and the number of message cycles that were dropped. What we see is that, without throttling the send, a very high percentage of message cycles fail. From examination of the raw data it appears that on the Mica2, most messages are dropped at the re-Mote end (commands are not received); but, for the Micaz under good conditions, the failures seem to be in receiving the replies from the re-Mote. The throughput is quite a bit higher for the Micaz, as expected, but the failure rate is significantly lower. In fact with proper throttling (10-20ms), reception is perfect.

More interestingly, the Micaz's ACK behavior is much better. Not only are considerably fewer ACKs lost, but the ones that do go missing actually tend to indicate real lost messages. This is surmised from a cursory examination of their relationship in the raw data, but it can also be seen in the results for the "bad reception" test where the ACK and message failure rates are nearly the same (vis the Mica2 data where missing ACKS have very little to do with the failed message rate). Without more devices I can't determine if this will hold in general.

Message Cycle NOT Using ACKs

This set of experiments is identical to the above but ACKs were disabled at both ends. Note the maximum success rate of almost 70 msg/sec (at 0ms) and the reliable rate of  about 1/2 that at 20 ms delay under 'ideal' conditions for the MicaZ.

MicaZ
Delay
Time
 msg/sec
Total
 msg drop
     %
 msg/sec
Success
msg miss
     %
 Notes
0
 99.64 30.79
 68.97 8.56
 2m apart
10
 49.89
 0.1
 48.84 0.03
 2m apart
20
 33.28
 0.01
 33.27 0.01
 2m apart
20
 33.04
 38.53
 20.31
 9.06
 5m apart, bad reception

Mica2
Delay
Time
 msg/sec
Total
 msg drop
     %
 msg/sec
Success
msg miss
     %
 Notes
0
 19.96 26.82
 14.60 22.82
 2m apart
50
 9.99
 0.73
 9.92 0.57
 2m apart
50
 9.99
 0.87
 9.90
 0.73
 10m apart, good reception


Almost Raw Results and Test Code

The spread sheet with the accumulated data used to generate these graphs is here:
The TOS and Java code used to run the tests is available, along with an update to some of the Java files to quell a naughty threading bug found when testing the Micaz's:

Conclusions

With Micaz's, using a message request/reply system where a basestation requests a message from each re-Mote, by point-to-point request and reply I have arrived at the following conclusions:

Point-to-point request messages should be throttled to about 25 (request/replies) per second or many replies will be dropped. This amounts to something between 10 and 20ms delay between message request sends.

With message ACKs enabled in a point-to-point mode, under ideal conditions where there is little chance of transmit overlap and CSMA backoff, about 50 successful individual messages per second can be sent with almost no failures. ACKs are consistently received for successful messages.

Without ACKs, in a point-to-point mode, under ideal conditions where there is little chance of transmit overlap and CSMA backoff, successful messages are around 66 per second with almost no failures.

The ACK mechanism works very reliably in the MicaZ. When radio reception is marginal, ACK failures generally align with message failures (in contrast to the Mica2 where ACKS are dropped almost 10 times more often than messages. I don't know how this scales in a multi-re-Mote situation however.

An observation during testing of bad reception: The Micaz's seem to be quite sensitive to antenna positioning and object interference. The two communicating devices were in different rooms separated by some major plumbing, and slight re-postioning of the re-Mote had a large effect on successful message completion. Also just waving my hands around either the base-station or re-Mote caused many message failures. The Mica2's were much less sensitive on both counts in the same environment. YMMV of course...

xxx