Given the SLIP

Serial Line Internet Protocol goes back a long, long way. Back to the early days of the internet. The original internet connections used packet based transport protocols. What does that mean? Well, it means that data was sent in lumps. A lump had a header, the data and probably a checksum at the end. EtherNet connections (which were shiny new and used to underpin the early internet) shouted packets into the ether (actually a wire) and then waited for responses that made sense.

This was all fine and dandy for connecting together machines in the computer room, but what if you were using a serial connection? On a serial connection the unit of transmission is a single 8 bit byte, not a lump of stuff. If all you can receive is individual bytes, how do you know when you've got a packet's worth?

Enter SLIP. This is a protocol that lets you send blocks of data down a channel which can only send individual bytes. It works by nominating a character called "Frame End" (0xc0) that marks the end of a packet. A receiver can know that it has received a complete packet when it receives a frame end. Of course, the next question is "How do we send the character 0xc0 in a stream of data that could contain any 8 bit value". We do this by nominating another character as "Frame Escape" (0xdb). This character marks the start of an "escape sequence". There are actually only two escape sequences. One sequence (0xdb 0xdc) means "this is actually the character 0xC0" and the other sequence (0xdb 0xdd) means "this is actually the character 0xdb". (I'm giving my numbers in hex to show that I'm a real computer person. For the record 0xc0 is the value 192)

In olden, olden, times you would buy a modem so you could connect your home computer to a distant server and then you would have to write a SLIP driver for your computer that could assemble the internet packets recieved froom the phone line and then pass them into the TCP/IP stack on your home computer. Happy days.

But what has this got to do with the price of fish, or even sending data to an ESP device? Well, it turns out that ESP devices use SLIP as the protocol for their link to the PC. That packet that I carefully assembled yesterday now has to be made into a SLIP packet so that I can send it to the device. This is actually quite a simple piece of code. It has to do three things:

1. Look for the character 0xC0 in the input data and convert it into the sequence 0xDB 0xDC.
2. Look for the character 0xDB in the input data and convert it into the sequence 0xDB 0xDD.
3. Put the character 0xC0 on the end of the data packet.

There's a function in pytool (at line 394) that does this conversion:

def write(self, packet):
    buf = b'\xc0' \
            + (packet.replace(b'\xdb', b'\xdb\xdd').replace(b'\xc0', b'\xdb\xdc')) \
            + b'\xc0'
    self.trace("Write %d bytes: %s", len(buf), HexFormatter(buf))
    self._port.write(buf)

It uses some neat Python features that allow replacment of elements in arrays. You can see how it swaps the escape characters.

packSlip(message) {
    // find out how many extra spaces we need for escape
    // count the 0xdb and 0xc0 values in the message
    let extra = 0;
    message.map((i) => { if ((i == 0xdb) || (i == 0xC0)) extra++ });

    let inputLength = message.length;

    // add in extra plus space for 0xc0 at start and end
    let totalLength = inputLength + extra + 2;

    let out = new Uint8Array(totalLength);

    let wPos = 0;

    out[wPos++] = 0xc0;

    for (let b of message) {
        switch (b) {
            case 0xdb:
                out[wPos++] = 0xdb;
                out[wPos++] = 0xdd;
                break;

            case 0xc0:
                out[wPos++] = 0xdb;
                out[wPos++] = 0xdc;
                break;

            default:
                out[wPos++] = b;
        }
    }

    out[wPos++] = 0xc0;

    return out;
}

This is my JavaScript slip packing code. You give it an array of byte values and it gives you back a slip encoded string with 0xc0 on the end (and at the beginning too - which clears out any nastiness with old characters lying around in the input buffer of the receiver).

When you start using UInt8Arrays you discover that all the nice dynamic array stuff in JavaScript (which will make an array automatically grow when you put things it it) has disappeared. You have to make the array the exact length you need it to be.

I use the map function to count the number of characters that need to be encoded and use this to work out how long the result array needs to be. Then I spin through the array and do the encoding.

There are almost certainly much more efficient and cunning ways of doing this, but the above code does work. So now I have a nice block of data that I want to send to the ESP device. And then I'll want to see what comes back...