Interesting Ken - thank you; Ahh interesting - is that because a one byte slip/drop might not allow the CRC algorithm to work properly? I guess alternatively if you had a truly symmetrical connection you could send the entire file back on the other channel as a brute force method of ensuring 100% correct and 100% efficiency? On Tue, Oct 15, 2019 at 2:33 PM Kenneth Gober <kgober@gmail.com> wrote:
On Mon, Oct 14, 2019 at 7:55 PM John Heritage via vcf-midatlantic < vcf-midatlantic@lists.vcfed.org> wrote:
This protocol would operate like ZModem or YModem-G where the uploader just keeps streaming, but no CRCs would be sent with the transfer. Instead, the 'sender' computer would simply pre-calculate CRCs for every 1K or so; the receiver computer would calculate after receiving that 1K, and send back to the uploader. The uploader would go back and verify that it was getting the right CRCs in the right order (you could pre-calculate these and store in RAM before the transfer starts to reduce CPU load during transfer). If no problems were encountered - you might hit that theoretical 1800 cps.
You will probably want to at least mark block boundaries, so that the sender and receiver can stay in sync if any bytes are dropped in transmission. Otherwise a single dropped byte could cause every successive block to be sent again. With a 1K block size, a 2-byte 'frame boundary' marker would give you an efficiency of 1024/1026 (99.8%), assuming the block doesn't contain embedded markers that need to be escaped. For random data you could guess that you'll get about 4 such bytes on average per block, so call it 1024/1034 (99.0%) efficient.
I chose 2 bytes as the size of the frame marker so that a single escape byte could handle all of these cases: ESC n (n=0-63) - frame (x mod 64) follows (numbered starting from 0) ESC n (n=64-127) - current frame contains n-63 literal ESC bytes here ESC n (n=128-191) - end of file, file size should be (x mod 64) frames ESC n (n=192-255) - please send the CRC for frame (x mod 64) again
The first 3 cases should be self-explanatory, the last case allows the receiver to send a 'frame' CRC in addition to the 'payload' CRC, so the sender can detect corrupt CRCs (or corrupt frame numbers).
-ken