[vcf-midatlantic] A "100%" Efficient BBS Transfer Protocol

[Kenneth Gober]

On Mon, Oct 14, 2019 at 7:55 PM John Heritage via vcf-midatlantic <
vcf-midatlantic at lists.vcfed.org> wrote:

> This protocol would operate like ZModem or YModem-G where the uploader just
> keeps streaming, but no CRCs would be sent with the transfer.  Instead, the
> 'sender' computer would simply pre-calculate CRCs for every 1K or so;  the
> receiver computer would calculate after receiving that 1K, and send back to
> the uploader.  The uploader would go back and verify that it was getting
> the right CRCs in the right order (you could pre-calculate these and store
> in RAM before the transfer starts to reduce CPU load during transfer).   If
> no problems were encountered - you might hit that theoretical 1800 cps.
>

You will probably want to at least mark block boundaries, so that the sender
and receiver can stay in sync if any bytes are dropped in transmission.
Otherwise a single dropped byte could cause every successive block to
be sent again.  With a 1K block size, a 2-byte 'frame boundary' marker
would give you an efficiency of 1024/1026 (99.8%), assuming the block
doesn't contain embedded markers that need to be escaped.  For
random data you could guess that you'll get about 4 such bytes on
average per block, so call it 1024/1034 (99.0%) efficient.

I chose 2 bytes as the size of the frame marker so that a single escape
byte could handle all of these cases:
ESC n (n=0-63) - frame (x mod 64) follows (numbered starting from 0)
ESC n (n=64-127) - current frame contains n-63 literal ESC bytes here
ESC n (n=128-191) - end of file, file size should be (x mod 64) frames
ESC n (n=192-255) - please send the CRC for frame (x mod 64) again

The first 3 cases should be self-explanatory, the last case allows the
receiver to send a 'frame' CRC in addition to the 'payload' CRC, so the
sender can detect corrupt CRCs (or corrupt frame numbers).

-ken