rec@advdev.Cambridge.NCR.COM (Rob Coutch) (02/18/89)
REQUEST FOR HELP: I'm looking for the complete explanation/description of the xmodem protocol for data transfer. (checksum and crc modes) If anyone with this data or info on how to obtain this data would please contact me with this information, I would be very grateful. Thanks in advance! BTW: this is for a possible PASCAL program. Rob Coutch rec@advdev.cambridge.ncr.com
Mannie@cup.portal.com (William Allison Guynes) (02/19/89)
I would also like and information on not only Xmodem, but Ymodem and Zmodem protocols also. If you have ANY information that might help me, please send it! (All information is welcome) Mannie@cup.portal.com
reid@hpmtlx.HP.COM ($Reid Trimble) (02/24/89)
Here is an OLD def'n of xmodem I think I got off the net some time ago:
MODEM PROTOCOL OVERVIEW 178 lines, 7.5K
1/1/82 by Ward Christensen. I will maintain a master copy of
this. Please pass on changes or suggestions via CBBS/Chicago
at (312) 545-8086, or by voice at (312) 849-6279.
NOTE this does not include things which I am not familiar with,
such as the CRC option implemented by John Mahr.
Last Rev: (none)
At the request of Rick Mallinak on behalf of the guys at
Standard Oil with IBM P.C.s, as well as several previous
requests, I finally decided to put my modem protocol into
writing. It had been previously formally published only in the
AMRAD newsletter.
Table of Contents
1. DEFINITIONS
2. TRANSMISSION MEDIUM LEVEL PROTOCOL
3. MESSAGE BLOCK LEVEL PROTOCOL
4. FILE LEVEL PROTOCOL
5. DATA FLOW EXAMPLE INCLUDING ERROR RECOVERY
6. PROGRAMMING TIPS.
-------- 1. DEFINITIONS.
<soh> 01H
<eot> 04H
<ack> 05H
<nak> 15H
<can> 18H
-------- 2. TRANSMISSION MEDIUM LEVEL PROTOCOL
Asynchronous, 8 data bits, no parity, one stop bit.
The protocol imposes no restrictions on the contents of the
data being transmitted. No control characters are looked for
in the 128-byte data messages. Absolutely any kind of data may
be sent - binary, ASCII, etc. The protocol has not formally
been adopted to a 7-bit environment for the transmission of
ASCII-only (or unpacked-hex) data , although it could be simply
by having both ends agree to AND the protocol-dependent data
with 7F hex before validating it. I specifically am referring
to the checksum, and the block numbers and their ones-
complement.
Those wishing to maintain compatibility of the CP/M file
structure, i.e. to allow modemming ASCII files to or from CP/M
systems should follow this data format:
* ASCII tabs used (09H); tabs set every 8.
* Lines terminated by CR/LF (0DH 0AH)
* End-of-file indicated by ^Z, 1AH. (one or more)
* Data is variable length, i.e. should be considered a
continuous stream of data bytes, broken into 128-byte
chunks purely for the purpose of transmission.
* A CP/M "peculiarity": If the data ends exactly on a
128-byte boundary, i.e. CR in 127, and LF in 128, a
subsequent sector containing the ^Z EOF character(s)
is optional, but is preferred. Some utilities or
programs still do not handle EOF without ^Zs.
* The last block sent is no different from others, i.e.
there is no "short block".
-------- 3. MESSAGE BLOCK LEVEL PROTOCOL
Each block of the transfer looks like:
<SOH><blk #><255-blk #><--128 data bytes--><cksum>
in which:
<SOH> = 01 hex
<blk #> = binary number, starts at 01 increments by 1, and
wraps 0FFH to 00H (not to 01)
<255-blk #> = blk # after going thru 8080 "CMA" instr, i.e.
each bit complemented in the 8-bit block number.
Formally, this is the "ones complement".
<cksum> = the sum of the data bytes only. Toss any carry.
-------- 4. FILE LEVEL PROTOCOL
---- 4A. COMMON TO BOTH SENDER AND RECEIVER:
All errors are retried 10 times. For versions running with
an operator (i.e. NOT with XMODEM), a message is typed after 10
errors asking the operator whether to "retry or quit".
Some versions of the protocol use <can>, ASCII ^X, to
cancel transmission. This was never adopted as a standard, as
having a single "abort" character makes the transmission
susceptible to false termination due to an <ack> <nak> or <soh>
being corrupted into a <can> and canceling transmission.
The protocol may be considered "receiver driven", that is,
the sender need not automatically re-transmit, although it does
in the current implementations.
---- 4B. RECEIVE PROGRAM CONSIDERATIONS:
The receiver has a 10-second timeout. It sends a <nak>
every time it times out. The receiver's first timeout, which
sends a <nak>, signals the transmitter to start. Optionally,
the receiver could send a <nak> immediately, in case the sender
was ready. This would save the initial 10 second timeout.
However, the receiver MUST continue to timeout every 10 seconds
in case the sender wasn't ready.
Once into a receiving a block, the receiver goes into a
ne-second timeout for each character and the checksum. If the
receiver wishes to <nak> a block for any reason (invalid
header, timeout receiving data), it must wait for the line to
clear. See "programming tips" for ideas
Synchronizing: If a valid block number is received, it
will be: 1) the expected one, in which case everything is fine;
or 2) a repeat of the previously received block. This should
be considered OK, and only indicates that the receivers <ack>
got glitched, and the sender re-transmitted; 3) any other block
number indicates a fatal loss of synchronization, such as the
rare case of the sender getting a line-glitch that looked like
an <ack>. Abort the transmission, sending a <can>
---- 4C. SENDING PROGRAM CONSIDERATIONS.
While waiting for transmission to begin, the sender has
only a single very long timeout, say one minute. In the
current protocol, the sender has a 10 second timeout before
retrying. I suggest NOT doing this, and letting the protocol
be completely receiver-driven. This will be compatible with
existing programs.
When the sender has no more data, it sends an <eot>, and
awaits an <ack>, resending the <eot> if it doesn't get one.
Again, the protocol could be receiver-driven, with the sender
only having the high-level 1-minute timeout to abort.
-------- 5. DATA FLOW EXAMPLE INCLUDING ERROR RECOVERY
Here is a sample of the data flow, sending a 3-block message.
des the two most common line hits - a garbaged block,
and an <ack> reply getting garbaged. <xx> represents the
checksum byte.
SENDER RECEIVER
times out after 10 seconds,
<--- <nak>
<soh> 01 FE -data- <xx> --->
<--- <ack>
<soh> 02 FD -data- xx ---> (data gets line hit)
<--- <nak>
<soh> 02 FD -data- xx --->
<--- <ack>
<soh> 03 FC -data- xx --->
(ack gets garbaged) <--- <ack>
<soh> 03 FC -data- xx ---> <ack>
<eot> --->
<--- <ack>
-------- 6. PROGRAMMING TIPS.
* The character-receive subroutine should be called with a
parameter specifying the number of seconds to wait. The
receiver should first call it with a time of 10, then <nak> and
try again, 10 times.
After receiving the <soh>, the receiver should call the
character receive subroutine with a 1-second timeout, for the
remainder of the message and the <cksum>. Since they are sent
as a continuous stream, timing out of this implies a serious
like glitch that caused, say, 127 characters to be seen instead
of 128.
* When the receiver wishes to <nak>, it should call a "PURGE"
subroutine, to wait for the line to clear. Recall the sender
tosses any characters in its UART buffer immediately upon
completing sending a block, to ensure no glitches were mis-
interpreted.
The most common technique is for "PURGE" to call the
character receive subroutine, specifying a 1-second timeout,
and looping back to PURGE until a timeout occurs. The <nak> is
then sent, ensuring the other end will see it.
* You may wish to add code recommended by Jonh Mahr to your
character receive routine - to set an error flag if the UART
shows framing error, or overrun. This will help catch a few
more glitches - the most common of which is a hit in the high
bits of the byte in two consecutive bytes. The <cksum> comes
out OK since counting in 1-byte produces the same result of
adding 80H + 80H as with adding 00H + 00H.reid@hpmtlx.HP.COM ($Reid Trimble) (02/24/89)
Here's an old turbo pascal (3.0?) program that implements xmodem. Have fun
deciphering it...
--------------------------------- cut ----------------------------------------
{.HEFile: Modem.Pas Listed: 4/20/87 Page: #}
{.PL66}
{$C-} {no user interrupts}
{$U-}
{$K-} {no stack checking - program works}
program Modem;
{ Written by Jack M. Wierda Chicago Illinois
Modified by Steve Freeman
LANGUAGE: TURBO Pascal
This program is in the public domain.
This program is basically a re-write in PASCAL of Ward Christensen's
Modem Program which was distributed in CP/M User's Group Volume 25. Identical
and compatible options are provided to allow this program to work directly
with XMODEM running under CP/M. }
const
Version = '12-Nov-84';
FredsPhone = '7-5038';
SignOnLine = 'ACGM10,RLIP,PSSWD';
MaxPhoneNums = 26;
COMport = 1;
NUL = 00; SOH = #$01; EOT = #$04; ACK = #$06;
TAB = 09; LF = #$0A; CR = #$0D; NAK = #$15;
Space = ' '; DELete = $7F;
lastbyte = 127;
timeout = 256;
errormax = 5;
retrymax = 5;
loopspersec = 6500;
Intseg: integer = 0; {filled with interrupt segment address}
type maxstr = string[255];
PhoneEntry = string[32];
PhoneStr = string[20];
BytePointer = ^byte;
var COMbase: integer; {this will point to the Communications base}
WorkFile: file;
PhoneFile: text;
PhoneList: array[1..MaxPhoneNums] of PhoneEntry;
option, hangup, return, mode, baudrate : char;
sector : array[0..lastbyte] of byte;
base, N_Phones: integer;
{ interrupt vectors and pointers to them }
newvec, oldvec: BytePointer;
INT3: BytePointer absolute $0000:$002C; {for COM2:}
INT4: BytePointer absolute $0000:$0030; {for COM1:}
rcvbuf: array[0..127] of byte;
inptr, outptr: integer;
datardy: boolean;
{.pa}
type hexstr = string[4];
function hex(num: integer): hexstr;
var i, j: integer;
h: string[16];
str: hexstr;
begin
str := '0000'; h := '0123456789ABCDEF'; j := num;
for i:=4 downto 1
do begin
str[i] := h[(j and 15)+1];
j := j shr 4;
end;
hex := str;
end;
{.cp10}
function GetYN: char;
var c: char;
begin
repeat
read(kbd,c);
c := upcase(c);
until c in ['Y','N'];
writeln(c);
GetYN := c
end;
{.cp4}
procedure SetDTR;
begin
port[base+4] := $09; {DTR on and INT enabled}
end;
{.cp4}
procedure HangUpPhone; {hang up by terminating the line}
begin
port[base+4] := 0;
end;
{.cp7}
function status: integer;
var st: integer;
begin
st := port[base+5];
st := st shl 8 + port[base+6];
status := st;
end;
{.cp6}
procedure send(ch: char);
var s: byte;
begin
repeat s := port[base+5] and $20 until (s=$20);
port[base] := ord(ch);
end;
{.cp6}
function get_rcv_char: char;
begin
get_rcv_char := chr(rcvbuf[outptr]);
outptr := (outptr + 1) and $7F;
if inptr=outptr then datardy := false;
end;
{.cp5}
function receive: char;
begin
repeat until datardy;
receive := get_rcv_char;
end;
{.cp9}
function ReadLine(seconds:integer): integer;
var j : integer;
begin
j := loopspersec * seconds;
repeat j := j-1 until datardy or (j = 0);
if j = 0
then readline := timeout
else readline := ord(get_rcv_char);
end;
{.cp8}
procedure PurgeLine; {purge the receive register}
var c: char;
begin
repeat
if datardy then c := get_rcv_char;
delay(35); { 300 baud time period for received char }
until not(datardy)
end;
{.cp42}
procedure Set_RS232_Vector;
procedure Int_Handler;
{ This routine buffers all incoming received data }
begin
inline($50/$52/$57/$1E/ {save registers}
$2E/ {CS:}
$8E/$1E/Intseg/ {MOV DS,[Intseg]} {get data segment pointer}
$BA/$FD/$03/ {MOV DX,$3FD} {is character ready?}
$EC/ {IN AL,DX}
$24/$01/ {AND AL,01}
$74/$19/ {JZ here} { no, skip entry}
$BA/$F8/$03/ {MOV DX,$3F8} { yes, get pointer}
$A1/inptr/ {MOV AX,[inptr]} {get index to buffer}
$97/ {XCHG DI,AX}
$EC/ {IN AL,DX} {get data from receiver}
$88/$85/rcvbuf/ {MOV [DI+rcvbuf],AL} {put data into buffer}
$97/ {XCHG DI,AX} {increment pointer}
$40/ {INC AX}
$24/$7F/ {AND AL,$7F}
$A3/inptr/ {MOV [inptr],AX}
$B8/$01/$00/ {MOV AX,1} {show data is ready}
$A2/datardy/ {MOV [datardy],AX}
{here}
$B0/$64/ {MOV AL,64} {EOI, level 4 on 8259}
$E6/$20/ {OUT 20,AL}
$1F/$5F/$5A/$58/$CF); {restore and return}
end;
begin
Intseg := Dseg;
COMbase := $0400 + 2 * (COMport - 1);
oldvec := INT4;
newvec := ptr(cseg,ofs(Int_Handler)+7+5);
INT4 := newvec;
inline($BA/$3F8/ {MOV DX,BASE}
$EC/$EC/$EC/$EC/ {IN AL,DX}
$BA/$3FD/$EC/ {MOV DX,BASE+5 ! IN AL,DX}
$BA/$3FE/$EC); {MOV DX,BASE+6 ! IN AL,DX}
datardy := false; inptr := 0; outptr := inptr;
inline($E4/$21/$24/$EF/$E6/$21); {turn off IRQ mask bit - enabled}
end;
{.cp16}
procedure Setup(md, brc: char);
var al: integer;
begin
base := memw[0:COMbase];
port[base+3] := $83; {access baud rate divisor and sets
8 data, no parity, 1 stop}
if md='O' then mode:=' ' else mode:='R';
baudrate := brc;
if baudrate='1'
then portw[base] := $0060 {1200 baud}
else portw[base] := $0180; { 300 baud}
port[base+3] := $03; {set access for xmt/rcv}
port[base+1] := $01; {enable receiver interrupts}
SetDTR; {put station on-line}
return := 'N';
end;
{.cp16}
procedure Initialize;
var mode, baudrate: char;
begin
repeat
write('Mode : A(nswer), O(riginate) ? ');
read(kbd,mode); mode := upcase(mode);
until mode in ['A','O'];
writeln(mode);
repeat
write('Baud rate : 3(00), 1(200) ? ');
read(kbd,baudrate);
until baudrate in ['1','3'];
writeln(baudrate);
Setup(mode,baudrate);
end;
{.cp19}
procedure terminal;
var s, t: byte;
c: char;
begin {$I-} {no I/O checking here}
writeln('Use ctrl-E to exit terminal mode.');
repeat
s := port[base+5]; {get status}
if datardy
then begin
t := ord(get_rcv_char); t := t and $7F;
if t<>$7F then write(chr(t));
end;
if keypressed and ((s and $20) = $20)
then begin
read(kbd,c);
port[base] := ord(c);
end;
until (c = ^E);
end; {$I+}
{.cp5}
procedure sendtext(str: maxstr);
var i: integer;
begin
for i:=1 to length(str) do send(str[i]);
end;
{.cp20}
function Dial(PhoneNumber: PhoneStr): char;
var c, kc: char;
t: integer;
begin
HangUpPhone; write(cr,lf,'Dialing: ',PhoneNumber);
delay(250); SetDTR; delay(250); sendtext(cr); delay(1000);
sendtext('AT '+mode+'M1V0DT'+PhoneNumber+cr); delay(2000);
c := receive; c := chr(0); repeat c := get_rcv_char until (c=cr);
write(', Waiting for carrier ...');
t := 60 * loopspersec;
repeat
t := t - 1;
if datardy then c := get_rcv_char;
if keypressed then read(kbd,kc);
until (c in ['0'..'5']) or (t=0) or (kc=^E);
if c='1'
then writeln(' connected.')
else if (t=0) or (kc=^E) then c := '9';
Dial := c
end;
{.cp15}
procedure SignOn;
var i: integer;
c: char;
begin
write('Signing on ... ');
delay(2000);
for i:=1 to 7
do begin
send('8');
delay(333);
end;
sendtext('('+cr);
delay(2500); sendtext(SignOnLine+cr);
writeln('all set !');
end;
{.pa}
procedure SendFile;
var j, sectornum, counter, checksum : integer;
filename : string[20];
c: char;
procedure SendIt;
begin
sectornum := 1;
repeat
counter := 0;
blockread(WorkFile,sector,1);
repeat
write(cr,'Sending sector ', sectornum);
send(SOH); send(chr(sectornum)); send(chr(-sectornum-1));
checksum := 0;
for j:=0 to lastbyte
do begin
send(chr(sector[j]));
checksum := (checksum + sector[j]) mod 256
end;
send(chr(checksum));
purgeline;
counter := counter + 1;
until (readline(10) = ord(ack)) or (counter = retrymax);
sectornum := sectornum + 1
until (eof(WorkFile)) or (counter = retrymax);
if counter = retrymax
then writeln(cr,lf,'No ACK on sector')
else begin
counter := 0;
repeat
send(EOT);
counter := counter + 1
until (readline(10)=ord(ack)) or (counter=retrymax);
if counter = retrymax
then writeln(cr,lf,'No ACK on EOT')
else writeln(cr,lf,'Transfer complete');
end;
end;
begin
write('Filename.Ext ? '); readln(filename);
if length(filename)>0
then begin
assign(WorkFile,filename);
reset(WorkFile);
SendIt;
close(WorkFile)
end;
end;
{.pa}
procedure readfile;
var j, firstchar, sectornum,sectorcurrent, sectorcomp, errors,
checksum : integer;
errorflag : boolean;
filename : string[20];
procedure ReceiveIt;
begin
sectornum := 0; errors := 0;
send(nak); send(nak); { send ready characters }
repeat
errorflag := false;
repeat
firstchar := readline(20)
until firstchar in [ord(SOH),ord(EOT),timeout];
if firstchar = timeout then writeln(cr,lf,'Error - No starting SOH');
if firstchar = ord(SOH)
then begin
sectorcurrent := readline(1); {real sector number}
sectorcomp := readline(1); {+ inverse of above}
if (sectorcurrent+sectorcomp)=255 {<-- becomes this #}
then begin
if (sectorcurrent=sectornum+1)
then begin
checksum := 0;
for j := 0 to lastbyte
do begin
sector[j] := readline(1);
checksum := (checksum+sector[j]) and $00FF
end;
if checksum=readline(1)
then begin
blockwrite(WorkFile,sector,1);
errors := 0;
sectornum := sectorcurrent;
write(cr,'Received sector ',sectorcurrent);
send(ack)
end
else begin
writeln(cr,lf,'Checksum error');
errorflag := true
end
end
else if (sectorcurrent=sectornum)
then begin
repeat until readline(1)=timeout;
writeln(cr,lf,'Received duplicate sector ', sectorcurrent);
send(ack)
end
else begin
writeln(cr,lf,'Synchronization error');
errorflag := true
end
end
else begin
writeln(cr,lf,'Sector number error');
errorflag := true
end
end;
if errorflag then begin
errors := errors+1;
repeat until readline(1)=timeout;
send(nak)
end;
until (firstchar in [ord(EOT),timeout]) or (errors = errormax);
if (firstchar=ord(EOT)) and (errors<errormax)
then begin
send(ack);
writeln(cr,lf,'Transfer complete')
end
else writeln(cr,lf,'Aborting');
end;
begin
write('Filename.Ext ? '); readln(filename);
if length(filename)>0
then begin
assign(WorkFile,filename);
rewrite(WorkFile);
ReceiveIt;
close(WorkFile);
end;
end;
{.cp17}
function ReadPhoneList: integer;
var index: integer;
begin
assign(PhoneFile,'MODEM.PHN');
index := 0;
{$I-} reset(PhoneFile); {$I+}
if IOresult=0
then begin
while (not eof(PhoneFile)) and (index<26)
do begin
index := index + 1;
readln(PhoneFile,PhoneList[index]);
end;
close(PhoneFile);
end;
ReadPhoneList := index;
end;
{.cp41}
procedure Call;
var rc: char;
selection, i, j, k: integer;
PhoneNo: PhoneStr;
begin
if N_Phones>0
then begin
clrscr; writeln;
for i:=1 to N_Phones
do begin
if (i mod 2)=0
then write(' ')
else writeln;
write(chr(i+64),' - ',PhoneList[i]);
end;
writeln; writeln; write('Enter selection letter: ');
repeat
repeat until keypressed;
read(kbd,rc); rc := upcase(rc);
selection := ord(rc) - ord('@');
until (selection in [1..N_Phones]);
writeln(rc);
mode := PhoneList[selection][31];
baudrate := PhoneList[selection][32];
Setup(mode,baudrate);
j := 30; PhoneNo := '';
while PhoneList[selection][j]<>'.' do j:=j-1;
for k:=j+1 to 30 do PhoneNo := PhoneNo + PhoneList[selection][k];
rc := Dial(PhoneNo);
end
else rc := Dial(FredsPhone);
if rc='1'
then begin
if N_Phones=0
then SignOn
else if selection=1 then Signon;
terminal;
end
else HangUpPhone;
end;
{.cp22}
procedure GetOption;
begin
clrscr;
writeln('Modem, ',Version);
gotoxy(7,4); writeln('Options:');
writeln;
writeln(' R - receive a file');
writeln(' S - send a file');
writeln(' T - terminal mode');
writeln;
writeln(' C - place a call');
writeln(' H - hang up the phone');
writeln(' O - option configuration');
writeln(' X - exit to system');
writeln; write('which ? ');
repeat
read(kbd,option);
option := upcase(option);
until option IN ['O','C','R','S','T','H','X'];
writeln(option);
end;
{.cp16}
begin {Modem}
Set_RS232_Vector;
N_Phones := ReadPhoneList;
Setup('O','1'); { default of Originate/1200 baud }
repeat
GetOption;
case option of
'T': Terminal;
'R': ReadFile;
'S': SendFile;
'O': Initialize;
'C': Call;
'H': HangUpPhone;
'X': return := 'Y';
end;
until return='Y';
inline($E4/$21/$0C/$10/$E6/$21); {turn on IRQ mask bit - disabled}
(* INT4 := oldvec; {restore the old RS232 vector} *)
end.
milne@ics.uci.edu (Alastair Milne) (03/01/89)
Mannie@cup.portal.com writes: > Isn't it funny how everyone knows how to USE XModem and no one >knows how it works? (Transfer protocols are the most closely guarded >secrets among programmers it seems) It may be true of x-modem (which, actually, I don't know how to use) but Kermit's protocol is extensively described in documentation available for download from Columbia University. I'd be interested in knowing whether XModem is as secure a protocol as Kermit -- the brief descriptions I've heard of it, compared with the description of Kermit, suggest that it isn't. A. Milne
ken@cs.rochester.edu (Ken Yap) (03/01/89)
|> Isn't it funny how everyone knows how to USE XModem and no one |>knows how it works? (Transfer protocols are the most closely guarded |>secrets among programmers it seems) There is no conspiracy involved. The protocol description is available from various places. I have a copy somewhere in my pack rat directories. And sources for various implementations are available too. (Source is truth :-).) Of course, hardware tends to intimidate some software types. | It may be true of x-modem (which, actually, I don't know how to use) but | Kermit's protocol is extensively described in documentation available for | download from Columbia University. I'd be interested in knowing whether | XModem is as secure a protocol as Kermit -- the brief descriptions I've | heard of it, compared with the description of Kermit, suggest that it isn't. And the Kermit protocol is also described in a book. By the way, I not sure what you mean by secure. Not as likely to get into strange states? Less susceptible to data errors? Now if you want something you use all the time but most people don't understand, you have to look no further than your compiler.