Date: Sun, 21 Dec 97 16:29:54 EST
From: Frank da Cruz <fdc@watsun.cc.columbia.edu>
To: wermit-l@wkuvx1.wku.edu
Subject: C-Kermit 6.1 Alpha.10
Message-ID: <CMM.0.90.4.882739794.fdc@watsun.cc.columbia.edu>
Here's Alpha.10 of C-Kermit 6.1.193. Still a closed Alpha, etc etc, like
last time, but big changes this time. Also, lots of new people added to the
mailing list (see notes at the end about how to find out what happened in
previous Alphas).
GENERAL CHANGES SINCE ALPHA.09 (13 Nov 1997)
. Major file-transfer speedups (explained below).
. New, more accurate and consistent calculation of transfer time and CPS.
. Various improvements and fixes to statistics reporting.
. Fixes to filename pattern-matching for automatic text/binary-mode switching.
. New SET TRANSFER DISPLAY BRIEF, one line per file, similar to FTP.
. New SET TRANSACTION-LOG BRIEF, one line per file, suitable for databases.
. GET /RECURSIVE added, as well as GET /RECURSIVE /DELETE.
. New dynamic server help text in response to REMOTE HELP.
. -V command line option = SET FILE PATTERNS OFF + SET XFER MODE MANUAL.
. New CLOSE command to close current connection (any kind).
. RETURN now works right when given from within SWITCH.
. Fixed SHOW MACRO and improved it to allow "?" to display macro names.
. Minor DELETE command editing and parsing problems fixed.
. Command recall fixed now, I hope.
. Assorted Telnet protocol fixes and improvements.
. Server end of REMOTE SET WINDOW fixed to not overallocate packet buffers.
. External protocols now properly set SUCCESS/FAILURE and \v(status).
. IF NUMERIC now properly fails if operand contains imbedded spaces.
. SET MODEM COMMAND VOLUME HIGH command fixed.
. No more "more?" prompting when run in batch or background.
. No more automatic CONNECT after DIAL when in batch or background.
. SHOW COMM now displays CARRIER-WATCH setting always.
. Scary messages no longer issued upon failure to set TCP socket options.
. A couple of minor memory leaks plugged.
. Numerous "keep compilers happy" changes.
Also, lots of problems with the fullscreen file-transfer display were fixed,
primarily those in which an interrupted or failed transfer would nevertherless
report "Transfer OK" as the percent bar would suddenly zoom out to 100%.
Oh yeah, and did I mention the new "drain" feature before? I forgot when I
put it in -- a few Alpha edits ago. Imagine you are downloading a file with
sliding windows (or, worse, streaming -- see below) and you interrupt it with
an Error packet (or this happens by itself because of a file-write error or
somesuch). When you CONNECT back to the remote (or, worse, when this happens
automatically because of autodownload), torrents of packets that were launched
by the sender before it got the E packet are gushing out at you, making your
terminal or emulator beep, lose its configuration, or freeze up. Now
C-Kermit, whenever it is in local mode and receiving files and it sends an E
packet, "drains" the communication channel, reading and discarding all those
queued-up packets until there are no more, before returning to command or
CONNECT mode.
UNIX-SPECIFIC CHANGES
. Improved file transfer performance on FIONREAD-deprived platforms.
. 16-bit QNX version now available.
. BIGBUFOK now predefined for Linux and SCO OpenServer.
. Removed references to Linux kernel header files, etc, that now conflict
with glib, hopefully with no ill effects (be sure to check that REDIRECT
and high serial speeds still work in Linux).
. Rlogin enabled for HP-UX 10.00 (but of course still requires privilege).
. Improved HP-UX makefile entries (from Peter Eichhorn).
. "cd ..<Esc>" now works as expected.
. "Union wait" warnings fixed in Digital UNIX 4.0 (from Lucas Hart).
. Lots of debugging added to CONNECT module to catch orphaned forks,
which have been reported in HP-UX, plus a failsafe mechanism to kill
any leftover forks.
. Man page updated.
I also happened to observe that when telnetted to an IRIX system (at least
versions 5.3 and 6.2), IRIX C-Kermit can't send files if the packet length is
greater than 4K; it can receive files with longer packets, but not send them.
This is evidently a peculiarity of the IRIX Telnet server, since when entering
IRIX in any other way, it doesn't happen.
Open UNIX issues:
. Orphan CONNECT forks reported in HP-UX.
. HP-UX 10.x version doesn't pop back to prompt if carrier drops on cua0p0
device; as far as I can tell, it never did, even though most other
SVR4-based versions do and HP-UX 10.x shares the same code.
. Still no C-Kermit 6.1 version for the BeBox.
. This version not yet checked out in Plan 9 (and when it is, that would be
a good opportunity to add TCP/IP support).
VMS-SPECIFIC CHANGES
. Outbound LAT connections fixed and tested.
. Fixed SEND /MAIL:<address> <filename>.
. NOVMSSHARE selection added to CKVKER.COM + much cleanup, from Lucas Hart.
. OLD_VMS conditionals updated for VMS 4.7, from Lucas Hart.
. Incoming filetype attribute was improperly overriding FILE TYPE IMAGE.
. No more testing for controller type on Alpha, so SHOW MODEM should work.
. Changed SET CARRIER and HELP SET CARRIER to print informative messages.
. VMS buffered serial-port read code changed to try not to lose the last few
incoming characters (like NO CARRIER) when a serial connection is hung up.
. VMS server now shows directory names when given the REMOTE DIR command.
. CKVKER.BWR (VMS C-Kermit Beware) file reorganized and augmented.
I learned after much experimentation -- and maybe some VMS experts will
explain this to me -- that when sending to remote-mode VMS C-Kermit (at least
when entering VMS via the UCX 2.0 Telnet server), the following control
characters can NOT be unprefixed:
Ctrl-C, Ctrl-M (obviously), Ctrl-N, Ctrl-O, Xon/Xoff and their high-bit
equivalents, Ctrl-X, Ctrl-Y, and CR with high bit.
Now I thought we were putting the terminal into the most transparent of modes.
I suppose I'm not surprised by Xon/Xoff being swallowed (after all, the
terminal must have HOSTSYNC/TTSYNC for file transfer to work at all, even on a
Telnet connection) but some of the others are surprising. For example, why
Ctrl-X and -Y but not Ctrl-Z? Why SO/SI? Is there some magic I can utter in
CKVTIO.C to make these pass through? (Yes, I do put the device in PASSTHRU
mode...)
In any case, I changed C-Kermit to never unprefix any of these characters when
it knows it's sending to VMS, no matter what the user said, and SET PREFIXING
CAUTIOUS to include the ones that weren't included previously (14,15,24,25).
(Obviously, this does not happen with all VMS systems, but it does happen
with enough of them to warrant these precautions.)
Open VMS issues:
. The new serial-port handling code is completely untested; I don't have a
way to test it myself.
. Inability to transfer certain types of files that Kermit-32 can transfer.
. Inability to transfer files under DECIntact or other environments that
"own" the TT: device and won't share it.
. Inability to creat or CD to a directory when the directory specification
includes a DECnet node.
. Getting/setting file permissions in VMS.
. Need subsecond timers like the ones added for UNIX. Does VMS have a system
call that returns the time of day in milliseconds or somesuch? (See UNIX
rftimer() and gftimer() in ckutio.c -- we need this for VMS too.)
. Receipt of broadcasts during CONNECT & file transfer still not disabled?
FILE-TRANSFER SPEEDUPS
1. Streaming
A new, experimental Kermit protocol option called "streaming" has been added
in Alpha.10. The idea is that if the two Kermits have a reliable transport
(such as TCP/IP or X.25) between them, then there is no need to send ACKs for
Data packets, since a reliable transport will, by definition, deliver all
packets in order and undamaged, and keeping the traffic all in one direction,
in turn, has certain benefits on various kinds of connections (TCP, modems
that allocate their bandwidth dynamically, etc).
Streaming is like using an infinite window size, with no timeouts, and zero
tolerance for transmission errors. It goes much faster than non-streaming
when a small packet length is used, and it also tends to go faster even with
the longest packet lengths; the results vary with the particular connection,
and so this is just another tuning knob, but one that will kick in
automatically when it is safe to do so.
For now, streaming can be used only between two copies of C-K 6.1 Alpha.10, or
between C-K 6.1 Alpha.10 and MS-DOS Kermit 3.16 Alpha.1. After the shakeout
period, it will also become available in future releases of Kermit 95 and
MS-DOS Kermit.
More about streaming (much more) in ckermit2.upd, Section 4.20.
2. Unprefixing NUL (0)
As of Alpha.10, C-Kermit can finally send and receive file-transfer packets
in which NUL (ASCII 0) is unprefixed (no more NUL-terminated packets!).
NUL-bearing packets are properly entered in the packet log, and shown
"symbolically" in the debug.log.
NUL is, of course, extremely prevalent in binary files such as executables,
and this has been a significant source of packet overhead. For example, when
transferring itself (the SunOS C-Kermit executable) with minimal prefixing and
9000-byte packets, we see:
File size: 1064960
Packet chars with 0 prefixed: 1199629 overhead = 12.65%
Packet chars with 0 unprefixed: 1062393 overhead = -0.03%
Transfer rates go up accordingly, not only because of the reduced amount of
i/o, but also because less computation is done on each end.
3. Clear-Channel Protocol
Now that C-Kermit itself is capable of sending and receiving any byte at all
on a clear channel, it is, for the first time, in a position to negotiate a
clear channel with the other Kermit, giving it permission to unprefix any and
all characters that it knows are safe. In general this means all but the
Kermit start-of-packet character (normally Ctrl-A), Carriage Return (not only
Kermit's end-of-packet character, but also treated specially on Telnet NVT
links), and IAC (255, also special to Telnet). By default, C-Kermit will say
it has a clear channel only if it has opened a TCP socket. More about this
in ckermit2.upd, Section 4.19.
For now, unprefixed 0's can be sent only between two copies of C-Kermit 6.1
Alpha.10, and/or MS-DOS Kermit 3.16 Alpha 1.
4. Other Speed Improvements
The last couple Alphas had a bug in which two identical characters in a row
were turned into a repeat prefix plus count (2) plus one copy of the
character, i.e. two characters replaced by three. This did not cause file
damage, but it hurt performance. This is fixed in Alpha.10.
There was also lots of under-the-hood tuning with respect to buffer sizes,
i/o methods, etc, plus continued microtuning: function-call removal, loop
tightening, registerizing, etc.
WHERE TO FIND IT
C-Kermit 6.1.193 Alpha.10 is in the usual places for C-Kermit test versions,
along with early drafts of all the update documentation, etc:
ftp://kermit.columbia.edu/kermit/test/tar/
cku193src.tar.Z Source code (UNIX and VMS), tar, UNIX compress
cku193src.tar.gz Source code (UNIX and VMS), tar, gzipped
cku193txt.tar.Z Other text files, tar, UNIX compress
cku193txt.tar.gz Other text files, tar, gzipped
ftp://kermit.columbia.edu/kermit/test/text/
Individual source and text files.
Among the interesting text files:
ckermit2.upd -- Detailed documentation of new features since 6.0.
ckc193.upd -- Program edit history since 6.0.
ckuins.doc -- UNIX installation instructions.
ckvins.doc -- VMS installation instructions.
ckvker.bwr -- VMS C-Kermit "beware file" (hints and tips).
ckuker.bwr -- UNIX C-Kermit beware file.
ckcplm.doc -- Program logic manual, API definition, etc.
alpha*.txt -- Announcements of each Alpha test.
INDIVIDUAL ALPHA.10 BINARIES:
The binaries marked with (*) are Alpha.10, transferred using itself (in
streaming mode this time) to the Kermit ftp site; the others are mostly
Alpha.09 or earlier (built at sites I can't reach any more or sent in by
others).
ftp://kermit.columbia.edu/kermit/test/bin/
* ckdker-aosvsii.pr: DG AOS/VS-II (it lives!)
* ckuker.bsd44c-hp9000_300 HP-9000/300 4.4BSD-Lite
* ckuker.bsdi2-2.1 PC, BSDI 2.1
ckuker.bsdi2-3.0 PC, BSDI 3.0
* ckuker.bsdi2-3.1 PC, BSDI 3.1
* ckuker.du32 DEC Alpha Digital UNIX 3.2
* ckuker.du40 DEC Alpha Digital UNIX 4.0
ckuker.hpux500wintcp HP-9000/550 HP-UX 5.21 + TWG-TCP/IP 1.2
* ckuker.hpux80c-hp9000_385 HP-9000/385 HP-UX 8.00 (not optimized)
* ckuker.hpux80oc-hp9000_385 HP-9000/385 HP-UX 8.00 (optimized)
* ckuker.hpux90o700 HP-9000/712 HP-UX 9.05
* ckuker.hpux100o-10.20 HP-9000/715 HP-UX 10.20
* ckuker.irix51-5.3 SGI IRIX 5.3
* ckuker.irix60-6.2 SGI IRIX 6.2
* ckuker.irix62-6.2 SGI IRIX 6.2 (**)
* ckuker.linux-1.2.13-i386 PC, Red Hat Linux 1.2.13
* ckuker.next-3.1-mc68040 NeXT 68040 NeXTSTEP 3.1
* ckuker.qnx16 QNX 4.24 16-bit
* ckuker.qnx32 QNX 4.24 32-bit
* ckuker.rs6aix41c IBM RS/6000 AIX 4.1
* ckuker.sco32v504net PC, SCO OpenServer 5.0.4
* ckuker.sinix542-5.42-mips SNI SINIX 5.4.2 MIPS (RM200)
* ckuker.solaris2x-2.4-sparc Sun Sparc, Solaris 2.4
* ckuker.solaris2x25-2.4-sparc Sun Sparc, Solaris 2.4 + SunLink X.25
* ckuker.solaris25-2.5.1-sparc Sun Sparc, Solaris 2.5.1
ckuker.solaris25-2.6-sparc Sun Sparc, Solaris 2.6
* ckuker.sunos41c-4.1.3-sparc Sun Sparc, SunOS 4.1.3_U1
* ckuker.ultrix42c-4.3-mips DECstation 5000, Ultrix 4.3
* ckuker.unixware2-2.1.1 PC, Unixware 2.1.1
* ckuker.unixware2-2.1.2 PC, Unixware 2.1.2
* ckvaker-vms62-nonet.exe: Alpha CPU, VMS 6.2, no TCP/IP
* ckvaker-vms62-tgv40a.exe: Alpha CPU, VMS 6.2, MultiNet 4.0A
* ckvaker-vms71-nonet.exe: Alpha CPU, VMS 7.1, no TCP/IP
* ckvaker-vms71-tgv40b.exe: Alpha CPU, VMS 7.1, MultiNet 4.0B
* ckvvker-vms55-nonet.exe: VAX CPU, VMS 5.5-2, no TCP/IP
* ckvvker-vms55-ucx20.exe: VAX CPU, VMS 5.5-2, UCX 2.0
* ckvvker-vms61-nonet.exe: VAX CPU, VMS 6.1, no TCP/IP
* ckvvker-vms61-tgv40a.exe: VAX CPU, VMS 6.1, MultiNet 4.0AX
* ckvvker-vms71-nonet.exe: VAX CPU, VMS 7.1, no TCP/IP
* ckvvker-vms71-tgv40b.exe: VAX CPU, VMS 7.1, MultiNet 4.0B
(**) The IRIX 6.2 version that I built on IRIX 6.2 does not support
high serial speeds because it was not built on an O-Class machine;
the header files did contain definitions for the higher speeds.
I'd be glad to do more of these myself and spare you the trouble -- for that
I'd need guest IDs on systems or configurations not listed above, to which I
can Telnet from here, and which have the necessary C compilers, header files,
& libraries.
Version 6.1 has not yet been checked out in OS-9, Amiga, VOS, Atari ST, or
Macintosh -- volunteers?
Thanks to all of you for your help with & comments on this version so far!
Please continue to beat on it and send any bug reports straight to me.
If you do succeed in building it on a platform not listed above, please let
me know the details (machine make & model, OS and version, separate TCP/IP
product, if any, and the size (in bytes) of the resulting executable (for
addition to the table at the end of the ckc193.upd file). And if you have
trouble, of course, let me know.
BONUS: MS-DOS KERMIT 3.16 ALPHA 1
This is a brand-new test version of MS-DOS Kermit that incorporates some of
the same new features found in C-Kermit 6.1:
. Recursive file transfers (descent through directory tree)
. \frfiles() and \frdirectories() functions
. Unprefixing 0
. Streaming
. Numerous other file-transfer speedups
. Accumulation of transfer CRC in \v(crc) variable
. Various fixes
In MS-DOS Kermit, streaming is activated by SET STREAMING ON, and deactivated
with SET STREAMING OFF. Find this version in:
ftp://kermit.columbia.edu/kermit/test/bin/msk316.zip
containing the msk316.exe executable and msk316.doc, some brief update notes.
- Frank
P.S. If you want off this mailing list, let me know.