[ Previous ] [ Next ] [ Index ] [ C-Kermit Home ] [ Kermit Home ]
Article: 12223 of comp.protocols.kermit.misc
From: fdc@watsun.cc.columbia.edu (Frank da Cruz)
Newsgroups: comp.protocols.kermit.misc
Subject: Case Study #23: Modem Scripting Tutorial
Date:Fri Feb 23 20:02:02 EST 2001
Organization: Columbia University
It's been a while since scripting tutorials have been posted here, but with C-Kermit 8.0 nearing release, with all its new scripting features, now seems like a good time to resurrect the practice.
Does your organization have a big modem pool? Of course (like any other big university) ours has has a pool of many hundreds of lines. As you know, phone lines aren't cheap so the annual cost of running a big modem pool is painful. Yet, even in this age of DSL and cable modems, the demand for modems never stops increasing, and therefore so do the complaints about busy signals, and the pressure to increase the capacity, and the counterpressure to contain contain costs. (Eventually some day every room will have an Internet jack and modems will have the same historical status as card punches and Teletypes, but that's another discussion -- for now DSL in the home is often not available, usually has a long wait when it is, and it's expensive).
One consequence of all this is the need to monitor and test the modem pool to make sure it's working at its best -- no broken or poorly-performing lines or servers -- plus the degree to which it is available, i.e. the answer-to-call ratio. There's a script for doing this in the Kermit script library, but it's undergone quite a few refinements in recent months as we reconfigure our modem pool, change telephone service providers, etc, and many of the refinements nicely illustrate new features of C-Kermit 8.0 (and the forthcoming next release of Kermit 95).
http://www.columbia.edu/kermit/ck80.html
The script is given a list of phone numbers and a list of terminal-server names. Each list can have one or more items. Obviously the script also needs to know the details of the calling computer: serial device and/or modem name, speed, etc. The script prompts for password (because it's a bad idea to store passwords in files) and runs in a loop, performing the following actions:
Here is bit of a log:
Date Time Number Line Speed Speed-After Blers Retr Upload Dnload ---------- ------- -------- ----- ----------- ----- ---- ------ ------ 02/22 0905 5551234 ccts5:080 49333 50666/28800 0 0 2951 5184 02/22 0907 5551235 ccts3:002 49333 49333/31200 1 0 3255 5143 02/22 0909 5551236 ccts2:096 49333 49333/31200 6 0 3243 5031 02/22 0911 5551237 ccts5:051 49333 50666/28800 1 0 3002 5220 02/22 0913 5551238 ccts5:069 49333 50666/28800 0 0 3008 5234In this case we're calling from a V.90 (56K) modem, which, as you know, supports higher speeds in the incoming direction than the outgoing. The terminal server names are ccts2, ccts3, etc; the combination of server name and line number specify a particular telephone circuit and modem.
Naturally, every imaginable kind of error is caught and logged:
02/22 0917 5551240 ccts3:076 49333 49333/31200 5 0 3217 5068 02/22 0919 5551241 ccts5:112 49333 UNKNOWN UNK UNK 3010 5215 02/22 1055 5551241 ccts4:UNK 31200 UNKNOWN UNK UNK FAILED FAILED 02/22 1119 5551243 FAILED: "BUSY" 02/22 1509 5551244 FAILED: "NO CARRIER" 02/22 1509 5551245 ccts5:183 49333 50666/31200 1 0 2925 5201
The script can be halted between calls by the operator from the keyboard, but you don't need to halt it to get at the log, which is opened and closed around each write so it can be examined, copied, uploaded, or whatever on platforms that might not allow shared access to open files.
The log format is such that headings and so forth can be separated out (in UNIX) with:
grep ^[0-9] modem.log > entries
leaving one line per call:
wc -l entries # number of calls grep -v FAILED entries | wc -l # number that were answered
The ratio of these two numbers gives the answer-to-call ration. (C-Kermit 8.0 also has its own internal GREP and line-counting commands in case you're not using UNIX.)
A log of successful calls can be sorted in various ways because the columns are fixed. For example, sorting on the "Line" column puts multiple calls to the same number together. The numeric data can be analyzed statistically too. For example, you can get the minimum, maximum, and mean upload or download speed and you can correlate it with telephone or line number to see (for example) if certain lines, terminal servers, or phone numbers perform better or worse than others over many calls (this would also jump out at you when eyeballing a sorted log).
The updated modem-test script is here:
ftp://ftp.kermit.columbia.edu/kermit/scripts/ckermit/modemtest2
Let's look at a few pieces of it:
dcl \&n[] = 5551234 5551235 5551236 5551237 5551238 5551239 5551240 -
5551241 5551242 5551243 5551244 5551245 5551246 5551247
dcl \&p[] = ccts1 ccts2 ccts3 ccts4 ccts5 ccts6
This declares an array \&n[] and loads it with 16 phone numbers, and another array \&p[], loading it with six terminal-server names. To change the script to use different lists of numbers and names requires changing only these two statements -- everything else adjusts itself automatically.
The terminal-server prompt turns out to be the terminal-server name followed by a ">" character, such as "ccts3>", so we construct a parallel array, \&q[], of prompts:
dcl \&q[\fdim(&p)]
for \%i 1 \fdim(&p) 1 {
.\&q[\%i] := \&p[\%i]>
}
The size of this array must be included in its declaration, since we are not initializing it in the declaration. But unlike in other languages (such as C), the size need not be a constant. In this case we say that the size is the "dimension" (size) of the array \&p[]. Then we loop through the new array, making the appropriate assignments.
Here's where we dial, illustrating (a) how we dial a number from the array; (b) how we handle failures based on Kermit's \v(dialstatus) variable (the \v(dialresult) variable is the modem's call-result message string):
dial \&n[\%i] ; \%i is the loop variable.
if fail {
switch \v(dialstatus) {
:8, logrecord {FAILED: Timed out}, break
:9, logrecord {FAILED: User canceled}, stop
:10, logrecord {FAILED: Modem not ready}, break
:default, logrecord {FAILED: "\v(dialresult)"}, break
}
continue
}
The terminal servers in question are Ciscos. Upon connection you can either send PPP negotiation data or else request a command prompt by sending a carriage return. The script sends a carriage return. The server prints a screenful of text and then the prompt:
ccts4 line 17 Welcome to blah blah blah blah blah blah blah blah blah blah ccts4>
The text might or might not contain a line like this:
ccts4 line 17
If it does, we need to pick up the line (port) number, but if it doesn't, we don't want to get stuck waiting for it. So instead, we scoop up everything up to and including the prompt:
clear input ; Clear the INPUT buffer
.\%x := 0 ; (explained below)
for \%j 1 10 1 { ; Try 10 times to get terminal server herald
output \13 ; Send Carriage Return (ASCII 13)
minput 10 \fjoin(&q,,2) ; Look for any of the prompts
if success break ; Quit this loop if we get one
}
if > \%j 10 { ; This means we didn't get one.
logrecord {FAILED: No terminal server prompt}
continue ; Go make another call.
}
The magic command in this section is:
minput 10 \fjoin(&q,,2)
The function \fjoin() is new to C-Kermit 8.0. It replaces itself with a text string consisting of all the elements of the given array, separated (in this case) by spaces (the "2" is a formatting code). Therefore, this statement is equivalent to:
minput 10 ccts1> ccts2> ccts3> ccts4> ccts5> ccts6>
This is not simply a notational convenience. It's one of the features that allows our script to be table-driven, in the sense that we only have to change the array initialization to make the script use different server names or phone numbers, and even different numbers of them, without having to change other lines in the script, such as the MINPUT that looks for all the possible prompts.
If the MINPUT command succeeded, we have the entire output of the terminal server in our \v(input) variable. Now we can scan it for the line-number text:
.tmp := \v(input) ; Copy the \v(input) variable .\%x = \v(minput) ; The item that MINPUT matched (1, 2, 3, ...)
The next statement searches for the line-number message. \&p[\%x] is the name of this server, corresponding to the prompt. So, for example, if the prompt was "ccts4>", the server name is "ccts4" and this statement searches the text we just read for the string "ccts4 line ".
.\%y = \findex({\&p[\%x] line },\m(tmp))
Next, we initialize the tsport (terminal-server port) variable to "UNK" in the desired line was not found:
.tsport = UNK
Then if the "ccts4 line " string was found, we use another function, \fword(), to get the third "word" from the string that starts with "ccts4 line ". This is the line number. Then we left-pad (\flpad()) it with 0's so it is exactly 3 digits:
if > \%y 0 {
.tsport := \flpad(\fword(\s(tmp[\%y]),3),3,0)
}
Now we know the server name and the port number:
.tsport := \&p[\%x]:\m(tsport)
which, continuing with our example, becomes:
ccts4:017
The rest is fairly straightforward: the regular OUTPUT / INPUT / IF FAIL sequence familiar to all Kermit script writers, to accomplish login, starting the Kermit program on the far end, transferring files, and logging out. There is one new feature in the data-transfer phase that puts some status information in the file-transfer display:
SET TRANSFER MESSAGE text
in which the text can contain variables. In this case it's used to show the call sequence number, the phone number, the terminal server name and port, and the time the transfer started (so you can easily tell if it's stuck):
set xfer message \m(seq): \v(dialnumber) (\m(tsport)) \v(time)
e.g.
Last Message: 712: 5554321 (ccts3:020) 16:02:56
After logout comes another interesting part, where we query the modem for statistics and capture them. Of course the details depend on the modem. For US Robotics modems we "output ATI6\13" and then parse the results as follows:
set flag off
while not flag {
minput 10 Blers Retrains Speed OK
switch \v(minput) {
:1, clear input, input 2 \10, .blers := \fword(\v(input),1), break
:2, clear input, input 2 \10, .retrains := \fword(\v(input),5), break
:3, clear input, input 2 \10, .ospeed := \fword(\v(input),1,,/), break
:4, set flag on, break
}
}
In other words, we look for any of "Blers", "Retrains", "Speed", and "OK" (which terminates the report). If we get OK, we're done. Otherwise we read the rest of the line ("input 2 \10", i.e. read up to a linefeed) and then use \fword() to extract the desired word.
The script requires about 2 minutes per call, so if all goes well it collects about 720 records per day, using a 100K test file and calling a 56K modem pool. It's best to collect several samples per server port, which could take a day or two or more, depending on the size of your pool and its hunt strategy.
Finally, obtaining statistics from the results is easier than ever using C-Kermit 8.0's new floating-point arithmetic and S-Expression syntax:
ftp://ftp.kermit.columbia.edu/kermit/scripts/ckermit/stats
All we need to do is filter out the error records, as shown at the top, and then use UNIX 'cut' to isolate the desired two columns of numbers. If the servername:port column is one of them, we can simply filter out the non-numeric characters, so (for example) ccts4:017 becomes 4017. Records that have ccts4:UNK are discarded completely since we don't know the port number.
Here we get statistics for upload and download speed versus server:port:
grep ^[0-9] modem.log | grep -v FAILED > tmp cut -c21-31,65-70 < tmp > up cut -c21-31,72-78 < tmp > dn tr -d "[a-z:]" < up | grep -v UNK > up2 tr -d "[a-z:]" < dn | grep -v UNK > dn2 stats up2 stats dn2
The last one ("stats dn2") prints something like this:
Points: 382
X Y
Minimum: 2001.00 1875.00
Maximum: 5192.00 5282.00
Mean: 3642.26 5016.66
Variance: 1811436.52 157155.99
Std Deviation: 1345.90 396.43
Correlation coefficient: 0.12
- Frank
[ Top ] [ Previous ] [ Next ] [ Index ] [ C-Kermit Home ] [ Kermit Home ]
Kermit Case Study 23 / Columbia University / kermit@kermitproject.org / 23 Feb 2001