USBPIC Serial Message System Design

Based on my reverse engineering of the Microchip reference code (see:
./MessageUSB.txt) the USBPIC system will poll all USB services in the
Scheduler loop (as done in the example USBTasks() and ProcessIO()).
It uses these functions supplied in the reference code:

   USBCheckBusStatus();        // Must use polling method --WTF??
   USBDriverService();        // handle USB "Interrupts"
   CDCTxService();            // executes pending Transmit-to-PC requests
   CDCRxService();            // fields pending Receive-from-PC requests

All the reference code is copied into system/usb and slightly modified
to implement things as described below.

Messages to be transmitted will be assembled in a user buffer and
then posted to the TX service using the mUSBUSARTTxRam() macro.
The TX service handles framing and copying of the message into the
Endpoint buffer for transmission.

Received messages will be fielded by the RX service and posted to
the user by calling the USBNotifyRxMsg() function in the core code.
The RX service handles de-framing and copying of the message from
the Endpoint buffer into a user message buffer.

All user level defines and functions are specified in USBmsg.h.


Memory Usage

At the user level a UPIC_Msg structure is used for every message.

The full message buffer is 64 bytes, where 6 bytes are used for
framing and header information. The actual user payload area should
be limited to around 50 bytes due to possible "escaped" bytes in the
message body (see Message Format below).

The internal memory layout is modified from the reference to make
better use of available RAM. Since we're using only 4 of the Endpoint
BDT descriptors the Endpoint TX and RX buffers will be located in
rambank 0x400 as well. This is all in the system/usb directory tree:

  #pragma udata usbram4=0x400    // usb4:0x400
    volatile far BDT ep0Bo;         //Endpoint #0 BD Out == USBCtrlEPService
    volatile far BDT ep0Bi;         //Endpoint #0 BD In  == USBCtrlEPService
    volatile far BDT ep1Bo;         //Endpoint #1 BD Out ?= HID_BD_OUT
    volatile far BDT ep1Bi;         //Endpoint #1 BD In  ?= HID_BD_IN
    volatile far BDT ep2Bo;         //Endpoint #2 BD Out ?= not used
    volatile far BDT ep2Bi;         //Endpoint #2 BD In  == CDC_INT_BD_IN
    volatile far BDT ep3Bo;         //Endpoint #3 BD Out == CDC_BULK_BD_OUT
    volatile far BDT ep3Bi;         //Endpoint #3 BD In  == CDC_BULK_BD_IN

Directly following the EPs are the transfer buffers:

    volatile far CTRL_TRF_SETUP SetupPkt;
    volatile far CTRL_TRF_DATA CtrlTrfData;
    volatile far unsigned char cdc_notice[CDC_INT_EP_SIZE];
    volatile far unsigned char cdc_data_rx[CDC_BULK_OUT_EP_SIZE]; // RX buffer
    volatile far unsigned char cdc_data_tx[CDC_BULK_IN_EP_SIZE];  // TX buffer

Then some buffer maintenance variables:
    POINTER pCDCTxSrc;    // pointer to TX message source buffer
    byte cdc_tx_len;    // TX source buffer count
    byte cdc_tx_state;    // TX transfer state
    byte cdc_mem_type;    // TX source memory type ROM/RAM
    byte* pCDCRxSrc;    // pointer to RX message buffer
    byte cdc_rx_len;    // current RX buffer counter
    byte cdc_rx_state;    // current RX transfer state
    
Then some misc maintenance variables:
    LINE_CODING line_coding;                     // line coding info
    CONTROL_SIGNAL_BITMAP control_signal_bitmap; // usb control bits
    byte dummy_encapsulated_cmd_response[dummy_length];

This leaves some RAM available in bank 0x400 if one really needs it...

Rambank 0x500 has 4 user message buffers of 64 bytes each.
These buffers are maintained by the RX and TX Services in support
of User messages:

  #pragma udata usbram5=0x500    // usb5:0x500
    byte USBMbufs[4][64];

The message buffers are accessed at the user level using the functions
described below.


Message Format
    
The contents of the message will be "framed and escaped" as per a
(slightly) modified tinyOS protocol, described here:
  http://web.archive.org/web/20060627154751/http://www.octavetech.com/pubs/TB5-01+Deciphering+TinyOS+Serial+Packets.pdf

To cut to the chase... see system/USBmsg.h and usb/cdc.c ...

Each USB message sent or received has this raw format:
  -------------------------- added by Packetizer.java framer protocol
    byte frame=0x7E;        // framing byte...gets escaped in body
    byte mtype=0x42;        // TOS message type -- used for status
    byte seqno;             // incremented sequence number
  -------------------------- user message
    byte length;            // length of user payload data
    byte type;              // user message type
    byte data[MAX_UPICPAY_LEN];    // user payload
  -------------------------- added by Packetizer.java framer protocol
    byte frame=0x7E;
  --------------------------
  The pre and post 0x7E frame bytes are added by the TX and RX
  services and never appear in user message buffers.

Because the 0x7E value is used to "frame" each message it cannot
appear elsewhere in the message body. If it does, the code in cdc.c
"escapes" the value by pre-pending a value of 0x7D and then XORs
the actual value with 0x20. This also makes 0x7D a special value,
so all appearences of 0x7D get the same pre-pend and XOR treatment. 

So... Any time a 0x7E appears (inside the frame bytes) the raw
message will contain: 0x7D, 0x5E (0x7E^0x20). And any time a 0x7D
appears in the data the raw message will contain: 0x7D, 0x5D.
This is handled by the cdc.c code for both TX and RX, so the user
should never see it on the controller side. You will have to deal
with it all on the PC host side however...


User Message Buffers and Functions

Each user message buffer in USBMbufs[] is cast to be a UPIC_Msg, where
the data[] array becomes the full payload length allowed in each buffer:

  typedef struct
  {
      // -------------------------- system header
    byte mtype;     // used for status indication
    byte seqno;
      // -------------------------- user message
    byte length;    // length of user data, not including hdr
    byte type;      // user message type
    byte data[2];   // first bytes of [MAX_UPICPAY_LEN] payload
                    //  note that this accounts for sync-bytes in final
                    //  message size when calculating framed sizes
  } UPIC_Msg;

The mtype field is used to indicate the current use/free status of
each buffer -- 0 is FREE, any other value is some kind of busy status.
The length field indicates how many bytes of the data[] area are in use,
and the type field is an arbitrary user specified indicator.


The user can get a Transmit message buffer using:
    
    UPIC_Msg* USBGetTxMsg( byte type );

which will return a UPIC_Msg pointer or NULL if they are all busy.
The type field will be set in the header and the length will be 0.

At a slightly higer level the user may call:
    
    UPIC_Msg* getSendMsg( byte type );

Which will behave exactly as USBGetTxMsg() if no message of the
given type is active, or return an active message of that type
if there is one. This eliminates the need for the application to
maintain pointers to messages that are being assembled "piecemeal".

To send the message when it is complete, the user should call:

    void USBPostTxMsg( UPIC_Msg *msg );

Alternately this function may be used:

    byte postSendMsg( UPIC_Msg *msg, byte length );

It will only send if the message is of the given length or larger.
It returns TRUE (1) if the message is actually sent or FALSE (0)
if it is not yet "filled". This can be used with getSendMsg()
to build up a multi-field message over multiple calls.


The user is notified of the reception of an Receive message using:
    
    void USBNotifyRxMsg( byte *data, byte len );

where data points to the relevant USBMbufs buffer and len is the
number of bytes used in that buffer, including the header fields.
This function is in scheduler.c and it in turn posts the user Task
that is named in the appconfig.h header file:

    USE_MessageTask( UPIC_Msg *msg );

A nominal implementation of this function is in stub/messageTask.c


When done with a received message the user should call this function
with the message (msg) pointer to be freed. This may also be used to
abandon a TX message buffer if you change your mind:

    void USBFreeMsg( UPIC_Msg *msg );

The TX service will free Transmit message buffers after they are sent.
   

Other messaging utility interfaces are:

  Format the given buffer as a UPIC_Msg structure with given type
   (note: no length checking is done). The underlying buffer will
   be set to a Busy state.

    void USBFmtMsg( UPIC_Msg *msg, byte type, byte length );

  Append len bytes of data to a message buffer. Maintains the
   msg->length field and checks for overflow. Returns the number
   of bytes written, which may be smaller if it overflows:

    byte USBAppendMsg( UPIC_Msg *msg, byte* data, byte len ) ;

  Return TRUE if USB system is connected and ready for messages:

    BOOL USBReady(void);
    

Necessary system interface functions are also in the header:

  Initialize the USB system. This actually just enables a bit of
  the hardware which then waits to get a connection to a host.
  It is called from maincode.c::main() if the USB system is
  enabled in appconfig.h:
  
    void USBInit(void);
    
  Service the USB system. This is called from scheduler.c::scheduler()
  every time the scheduler completes a pass through the task list
  (fairly frequently...):
  
    void USBService(void);

  Blink the status LED on the board in different patterns depending
  on the state of the USB connection. Also called from scheduler():

    void USBBlinkStatus(void);