[vcf-midatlantic] A "100%" Efficient BBS Transfer Protocol

[John Heritage]

Interesting Ken - thank you;

Ahh interesting - is that because a one byte slip/drop might not allow the
CRC algorithm to work properly?

I guess alternatively if you had a truly symmetrical connection you could
send the entire file back on the other channel as a brute force method of
ensuring 100% correct and 100% efficiency?




On Tue, Oct 15, 2019 at 2:33 PM Kenneth Gober <kgober at gmail.com> wrote:

> On Mon, Oct 14, 2019 at 7:55 PM John Heritage via vcf-midatlantic <
> vcf-midatlantic at lists.vcfed.org> wrote:
>
>> This protocol would operate like ZModem or YModem-G where the uploader
>> just
>> keeps streaming, but no CRCs would be sent with the transfer.  Instead,
>> the
>> 'sender' computer would simply pre-calculate CRCs for every 1K or so;  the
>> receiver computer would calculate after receiving that 1K, and send back
>> to
>> the uploader.  The uploader would go back and verify that it was getting
>> the right CRCs in the right order (you could pre-calculate these and store
>> in RAM before the transfer starts to reduce CPU load during transfer).
>>  If
>> no problems were encountered - you might hit that theoretical 1800 cps.
>>
>
> You will probably want to at least mark block boundaries, so that the
> sender
> and receiver can stay in sync if any bytes are dropped in transmission.
> Otherwise a single dropped byte could cause every successive block to
> be sent again.  With a 1K block size, a 2-byte 'frame boundary' marker
> would give you an efficiency of 1024/1026 (99.8%), assuming the block
> doesn't contain embedded markers that need to be escaped.  For
> random data you could guess that you'll get about 4 such bytes on
> average per block, so call it 1024/1034 (99.0%) efficient.
>
> I chose 2 bytes as the size of the frame marker so that a single escape
> byte could handle all of these cases:
> ESC n (n=0-63) - frame (x mod 64) follows (numbered starting from 0)
> ESC n (n=64-127) - current frame contains n-63 literal ESC bytes here
> ESC n (n=128-191) - end of file, file size should be (x mod 64) frames
> ESC n (n=192-255) - please send the CRC for frame (x mod 64) again
>
> The first 3 cases should be self-explanatory, the last case allows the
> receiver to send a 'frame' CRC in addition to the 'payload' CRC, so the
> sender can detect corrupt CRCs (or corrupt frame numbers).
>
> -ken
>