Published 2024-04-03.
Last modified 2024-06-27.
Time to read: 3 minutes.
This page is part of the
mainframe collection.
Personal computers have excellent user interfaces and many good options for source code management. In contrast, mainframes are straight out of the last millennium. I needed a way to maintain JCL on my local machines but execute it on a remote system. Traditionally, users interact with mainframes using 3270 terminals; the ISPF menuing system and Vista TN3270 transfer are both extremely clunky.
In contrast, ftp has some features that most people are unaware of,
which makes it a good approach.
However, IBM’s documentation is incomplete and not designed to educate the reader; instead, it is a partial list of facts.
This article explains the bash script that I wrote, which uses ftp to upload JCL, execute it and download results.
I rented a z/OS LPAR from Maintec to perform this work.
My user ID was MIKES01.
Shell
$ export loadlib="$ibm/horswill/CDROM/cicsadp/Cobol Application/TSO/loadlib"
$ cd "$loadlib"
$ ftp ftp://MIKES01:$MAIN_PWD@maintec Connected to 130.250.184.212. Connected to maintec. 220-FTPD1 IBM FTP CS V2R4 at MTI1, 15:43:10 on 2024-04-03. 220 Connection will close if idle for more than 5 minutes. 331 Send password please. 230 MIKES01 is logged on. Working directory is "MIKES01.". Remote system type is MVS. 200 Representation type is Image
ftp>
Pre-Setting MVS Dataset Parameters
The WSL/Ubuntu tnftp documentation says this about the ftp quote subcommand:
From man ftp
quote [arg ...] The arguments specified are sent, verbatim, to the remote FTP server.
To get the help information on the mainframe’s site command,
use ftp to connect to the mainframe, then type quote help site.
This is the help information retrieved from Maintec’s ftp server:
Ftp session continued
ftp> quote help site 214-The SITE command sub parameters are: 214-ASAtrans Specifies that ASA control characters in ASA files opened 214- for text processing should be converted to C control 214- character sequences during file transfer. 214-NOASAtrans Specifies that ASA control characters in ASA files opened 214- for text processing should be transferred as ASA control 214- characters. 214-AUTOMount Permits automatic mounting of volumes for data sets on 214- volumes that are not mounted. 214-NOAUTOMount Prevents automatic mounting of volumes for data sets on 214- volumes that are not mounted. 214-AUTORecall Permits migrated data sets to be recalled automatically. 214-NOAUTORecall Prevents migrated data sets from being recalled 214- automatically. 214-BLKsize=value Specifies the blocksize for new data sets. If 'value' is 214- omitted, blocksize will not be used when allocating new 214- data sets. 214-BLocks Specifies primary and secondary space allocation are in 214- blocks. 214-BLOCKSIze=value Specifies the blocksize for new data sets. If 'value' is 214- omitted, blocksize will not be used when allocating new 214- data sets. 214-BUfno=value Number of access method buffers. 214-CHKptint=value Checkpoint interval - the number of records to transmit 214- before sending a restart marker. 214-CHMOD newmode filename changes the permission bits of an HFS file. 214- newmode may be specified as an octal mask, or may be 214- specified as {who}{operation}{permission} where {who} is 214- u, g, o, or a (user, group, other, or all), {operation} is 214- +, -, or = (add permissions, remove permissions, or 214- replace permissions), and {permission} is some combination 214- of r, w, x (read, write, or execute). filename is the name 214- of the HFS file whose permission bits are to be changed. 214-CONDdisp=disp Specifies the conditional disposition of a data set as 214- Catlg (catalogue) or Delete. 214-CTRLConn=value Specifies the ASCII to be used on the control connection. 214- 'value' may be '7bit' or an ASCII code page name. Omit 214- 'value' to reset. 214-CYlinders Specifies primary and secondary space allocation are in 214- cylinders. 214-DATAClass=name Default SMS data class for new data sets. Omit the 'name' 214- to reset the data class name. 214-DATAKEEPALIVE=value Specifies the number of seconds of inactivity on the 214- data connection before a keepalive packet is sent. 214-DATASetmode Treat all subsequent levels as a data set (disables 214- directory mode). 214-DB2=value DB2 subsystem name (default is DB2). 214-DCbdsn=data_set Specifies that FTP should allocate any new data sets with 214- the same attributes as this data set. Data_set is either a 214- fully qualified data set name in quotes or appended to the 214- present directory name prefix. Blocksize, lrecl, recfm and 214- retpd parameters will override the model data set 214- characteristics if they are specified. 214-DEBUG=value Sets the server trace options 214-DUMP=value Sets the server extended trace options 214-DESt=dest NJE destination for files to go to. Can be specified as 214- USER@NODE, NODE.USER, or JES symbolic destination. Omit 214- the 'dest' to reset. 214-DIrectory=value Specifies the number of directory blocks when allocating a 214- new PDS. If 'value' is omitted, directory blocks will not 214- be specified when allocating a new PDS. 214-DIRECTORYMode Treat each level of a data set as a directory. Only the 214- next lower level is used for MPUT or server MGET, LS and 214- DIR commands. 214-DSNTYPE={BASIC|LARGE|SYSTEM} Specifies the data set name type for new 214- physical sequential data sets. BASIC indicates basic 214- format. LARGE indicates large format. SYSTEM indicates 214- physical sequential data sets will be allocated with the 214- SMS data class value, or the system default value. 214-DSWAITTIME=value Specifies the number of minutes to wait for an MVS data 214- set to become available. 214-DSWAITTIMEREPLY=value Specifies how often to send a data set retry reply 214- to the client while the FTP server is waiting for access 214- to an MVS data set. 214-EATTR=[NO|OPT|SYSTEM] specifies whether new data sets can have extended 214- attributes and whether the data sets can reside in the 214- EAS. NO indicates that the data set cannot reside in the 214- EAS and its VTOC entry cannot have extended attributes. 214- OPT indicates that the data set can reside in the EAS and 214- its VTOC entry can have extended attributes if the volume 214- supports them. SYSTEM indicates the data set will use the 214- SMS data class EATTR value, or system default EATTR value 214- if no SMS data class is defined or if the data class 214- contains no EATTR specification. 214-ENcoding=value Specifies whether multi-byte or single-byte data 214- conversion is to be performed on ASCII data transfers. The 214- values allowed are Mbcs and Sbcs for multi-byte and 214- single-byte, respectively. 214-FIFOIOTIME=value Specifies the number of seconds the FTP server waits when 214- reading from or writing to a named pipe. 214-FIFOOPENTIME=value Specifies the number of seconds the FTP server waits 214- when opening a named pipe. 214-FILEtype=value Specifies file type (SEQ, JES, or SQL). SEQ is standard 214- MVS sequential or partitioned data sets, or HFS files. 214- This is the default and most common. JES is MVS spool used 214- for submitting Jobs and retrieving their output. SQL is 214- for submitting DB2 queries. 214-ISPFStat Update statistics for PDS member 214-NOISPFStat Do not update statistics for PDS member 214-JESLrecl=val|* Lrecl to use for JES internal reader. 214-JESGETBYDSN FTP retrieval when FILETYPE=JES and JESINTERFACELEVEL=2 214- allows the foreign file to be a job id followed by a JES 214- spool dataset name 214-NOJESGETBYDSN FTP retrieval when FILETYPE=JES and JESINTERFACELEVEL=2 214- allows a JESPUTGET for submitting a job and automatically 214- receiving output 214-JESJOBname=val Job name of jobs to return for FTP JES support. 214-JESOwner=val Job owner of jobs to return for FTP JES support. 214-JESSTatus=val Status of jobs to return for FTP JES support. 214-JESRecfm=val|* Recfm to use for JES internal reader. 214-LISTLEVEL={0|1|2} Specifies the format of the LIST reply. LISTLEVEL 0 214- specifies that PDS, PDSE and HFS data sets are displayed 214- with a DSORG value of PO. LISTLEVEL 1 specifies that PDS 214- data sets are displayed with a DSORG value of PO, PDSE 214- data sets are displayed with a DSORG value of PO-E, and 214- HFS data sets are displayed with a DSORG value of HFS. 214- LISTLEVEL 2 specifies LISTLEVEL 1 display options, and 214- also fewer but wider columns of output to accommodate 214- larger physical sequential data sets. 214-LISTSUBDIR Wildcard searches span one subdirectory. 214-NOLISTSUBDIR Wildcard searches span only the current working directory. 214-LRecl=value Specifies the logical record length for new data sets. If 214- 'value' is omitted, lrecl will not be specified when 214- allocating new data sets. 214-MBdataconn=value Specifies the conversion table names for the data 214- connection when ENcoding has a value of MBCS. 'value' is 214- a pair of code pages with the format 214- (file_system_cp,network_transfer_cp). 214-MBREQUIRELASTEOL Specifies that the last record of an incoming multibyte 214- file must include an EOL sequence. If the last record 214- received has no EOL sequence, the server will treat this 214- as an error. 214-NOMBREQUIRELASTEOL Specifies that the last record of an incoming multibyte 214- file need not include an EOL sequence. If the last record 214- received has no EOL sequence, the server will treat this 214- as a complete file transfer. No error will be reported. 214-MBSENDEOL=value Specifies the line terminator for outbound MBCS ASCII 214- data. Value is CRLF, CR, LF, or NONE 214-MGmtclass=name Default SMS management class for new data sets. Omit the 214- 'name' to reset the management class name. 214-MIGRatevol=vol Volid for migrated data sets. 214-PDSTYPE=[{PDS|PDSE}] Specifies whether to allocate new directories as PDS 214- or PDSE data sets. If value is omitted, PDSTYPE will not 214- be used when allocating new MVS directories. 214-PRImary=value Specifies the amount of storage for the primary allocation 214- for new data sets. If 'value' is omitted, primary 214- allocation will not be specified when allocating a new 214- data set. 214-Qdisk=volume Lists the amount of space available on a volume. If no 214- volume is given, all the storage volumes are listed. 214-QUOtesoverride Specifies that single quotes around a filename indicate 214- that the filename should override the current working 214- directory rather than be appended to the current working 214- directory 214-NOQUOtesoverride Specifies that single quotes at the beginning of a file 214- name should be treated as part of the file name. 214-RDw RDW from variable format data sets retained as data. 214-NORDw RDW from variable format data sets discarded as not data. 214-READTAPEFormat=value Format of input tape data sets. Valid formats are F 214- (for fixed), V (for variable), S (for spanned), X (for 214- lrecl X), and blank (unspecified). 214-RECfm=value Specifies the record format for new data sets. If 'value' 214- is omitted, record format will not be specified when 214- allocating a new data set. 214-REMOVEINBEOF UNIX EOF removed from ASCII inbound data transfers. 214-NOREMOVEINBEOF UNIX EOF is not removed from ASCII inbound data transfers. 214-RESTPUT Server supports checkpointing when receiving data. 214-NORESTPUT Server does not support checkpointing when receiving data. 214-RETpd=value Specifies retention period for newly created data sets. If 214- 'value' is omitted, retention period will not be specified 214- when allocating a new data set. 214-SBDataconn=value Specifies the translation tables for the data connection. 214- 'value' may be a pair of code pages (cp) with the format 214- (file_system_cp,network_transfer_cp), the name of an MVS 214- data set or HFS file that contains the translate tables, 214- or the string FTP_STANDARD_TABLE. Omit '=value' or give 214- value of * to reset. 214-SBSENDEOL=value Specifies the line terminator for outbound SBCS ASCII 214- data. Value is CRLF, CR, LF, or NONE 214-SECondary=value Specifies the amount of storage for the secondary space 214- allocation for new data sets. If 'value' is omitted, 214- secondary allocation will not be specified when allocating 214- a new data set. 214-SPRead SQL results sent in SPREADsheet format. 214-NOSPRead SQL results sent in REPORT format. 214-SQLCol=value Column headings use N(ames), L(abels), or A(ny). 214-STOrclass=name Default SMS storage class for new data sets. Omit the 214- 'name' to reset the storage class name. 214-SUBSYS Specifies the subsystem name for new allocations. 214-TAPEREADSTREAM A more efficient read path (read as stream) is used for 214- reading tape data sets. Restrictions apply. 214-NOTAPEREADSTREAM A common read path is used for reading tape data sets. 214-TRacks Specifies primary and secondary space allocation are in 214- tracks. 214-TRAILingblanks Returns trailing blanks for fixed format data sets that 214- are retrieved. 214-NOTRAILingblanks Removes trailing blanks for fixed format data sets that 214- are retrieved. 214-TRUNcate Truncated records will not be treated as an error. 214-NOTRUNcate Truncated records will be treated as an error and the file 214- transfer will fail. 214-UCOUNT=value Maximum number of units for UNIT statement. 214-UCSHostcs codepage Specifies the EBCDIC codepage to use for conversion 214- to/from UCS. 214-UCSSub Specifies that the EBCDIC substitution character will be 214- used to replace any Unicode character that can not be 214- converted. Data transfer will continue. 214-NOUCSSub Specifies data transfer will end if any Unicode character 214- cannot be successfully converted. 214-UCSTrunc Specifies data transfer will continue even if EBCDIC data 214- is truncated during Unicode conversion. 214-NOUCSTrunc Specifies data transfer will end if the logical record 214- length of the receiving data set is too small to contain 214- the data after converstion from Unicode to EBCDIC. 214-Umask mask Specifies the octal UMASK to be used when allocating new 214- HFS files. UMASK will restrict the setting of the 214- permission bits. 214-UNICODEFILESYSTEMBOM={ASIS|ALWAYS|NEVER} Specifies whether to include a 214- byte order mark when storing Unicode files. ASIS specifies 214- to store the file as received, byte order mark or not; 214- ALWAYS specifies to always store a byte order mark; NEVER 214- specifies to never store a byte order mark. Only the first 214- character of the incoming file is treated as a byte order 214- mark. 214-Unit=value Specifies the name of a unit for allocation. If no value 214- is given, the previous value is removed and the system 214- default unit is used. 214-UNIXFILETYPE={FIFO|FILE} Specifies whether the FTP server should treat 214- files in the Unix file system as regular files or as named 214- pipes during file transfer. FILE indicates the server 214- will treat files in the Unix file system as regular files. 214- Inbound files are stored into the Unix file system as 214- regular files, and outbound files are read from the Unix 214- file system as regular files. FIFO indicates the server 214- will treat files in the Unix file system as named pipes. 214- Inbound files are stored into the Unix file system as 214- named pipes, and outbound files are read from the Unix 214- file system as named pipes. 214-VCOUNT=value Maximum number of volumes for VOLUME statement. 214-VOLume=value Specifies the name of a volume or volumes used for 214- allocation. If no value is given, the previous value is 214- removed and the system default volume list is used. 214-WRAPrecord Wrapping data into next record. 214-NOWRAPrecord Not wrapping data into next record. 214-WRTAPEFastio Allow tape write to use BSAM I/O 214-NOWRTAPEFastio Do not allow tape write to use BSAM I/O 214-Xlate=name Specifies a file containing translate tables for the data 214- connection. Filename is defined by FTP server environment 214- variable _FTPXLATE_. Default filename is 214- . .TCPXLBIN. Omit '=name' (or use '=*') to reset. 214 Use server STAT command to display present values.
I found that the dsorg parameter was not accepted, and the blksize parameter was required:
Ftp session continued
ftp> quote site lrecl=80 dsorg=ps recfm=f 200-Unrecognized parameter 'dsorg=ps' on SITE command. 200-BLOCKSIZE must equal LRECL for RECFM F 200-BLOCKSIZE being set to 80 200 SITE command was accepted
ftp> quote site lrecl=80 blksize=80 recfm=f 200 SITE command was accepted
Manual Job Submission
The mainframe ftp site subcommand can access the JES2 spooler when the FILE=JES parameter is provided.
Some useful commands are:
quote site filetype=JES - Connect to JESPLEX, (conect to JES in layman terms) dir - List Jobs (after the above command is executed) get filename - Get job dataset from remote system put filename - Submit job
The quote stat subcommand shows the status of the remote system.
Shell
$ ftp ftp://MIKES01:$MAIN_PWD@maintec ftp> quote stat 211-Server FTP talking to host 70.30.247.170, port 49884 211-User: MIKES01 Working directory: MIKES01. 211-The control connection has transferred 935 bytes 211-There is no current data connection. 211-The next data connection will be actively opened 211-to host 70.30.247.170, port 49884, 211-using Mode Stream, Structure File, type ASCII, byte-size 8 211-Automatic recall of migrated data sets. 211-Automatic mount of direct access volumes. 211-Auto tape mount is allowed. 211-Inactivity timer is set to 300 211-Timer FTPKEEPALIVE is set to 0 211-Timer DATAKEEPALIVE is set to 0 211-Timer DSWAITTIME is set to 0 211-Server site variable DSWAITTIMEREPLY is set to 60 211-Timer FIFOOPENTIME is set to 60 211-Timer FIFOIOTIME is set to 20 211-VCOUNT is 59 211-ASA control characters in ASA files opened for text processing will be 211- transferred as ASA control characters. 211-Server site variable TAPEREADSTREAM is set to FALSE 211-Trailing blanks are removed from a fixed format data set when it is 211- retrieved. 211-Data set mode. (Do not treat each qualifier as a directory.) 211-ISPFSTATS is set to FALSE 211-Primary allocation 1 track. Secondary allocation 1 track. 211-Partitioned data sets will be created with 27 directory blocks. 211-FileType JES (MVS Job Spool). JES Name is JES2 211-Number of access method buffers is 5 211-RDWs from variable format data sets are discarded. 211-Records on input tape are unspecified format 211-SITE DB2 subsystem name is DB2 211-Data not wrapped into next record. 211-Tape write is not allowed to use BSAM I/O 211-Truncated records will be treated as an error and the file transfer will 211- fail 211-JESLRECL is 80 211-JESRECFM is Fixed 211-JESINTERFACELEVEL is 1 211-Server site variable JESTRAILINGBLANKS is set to TRUE 211-Confidence level in data transfers is neither checked nor reported 211-ENcoding is set to SBCS 211-SBDataconn codeset names: IBM-1047,IBM-850 211-Outbound SBCS ASCII data uses CRLF line terminator 211-Outbound MBCS ASCII data uses CRLF line terminator 211-Server site variable MBREQUIRELASTEOL is set to TRUE 211-Server site variable UNICODEFILESYSTEMBOM is set to ASIS 211-Server site variable UNIXFILETYPE is set to FILE 211-DBSUB is set to FALSE 211-Server site variable EXTDBSCHINESE is set to TRUE 211-SBSUB is set to FALSE 211-SBSUBCHAR is set to SPACE 211-SMS is active. 211-New data sets will be catalogued if a store operation ends abnormally 211-Single quotes will override the current working directory. 211-UMASK value is 027 211-Process id is 83886565 211-Checkpoint interval is 0 211-Server site variable RESTPUT is set to TRUE 211-Server site variable REMOVEINBEOF is set to FALSE 211-Authentication type: None 211-TLS security is supported at the DRAFT level 211-Server site variable READVB is set to LE 211-Port of Entry resource class for IPv4 clients is: TERMINAL 211-Record format F, Lrecl: 80 211-Server site variable EATTR is set to SYSTEM 211-Server site variable DSNTYPE is set to SYSTEM 211-Server site variable LISTSUBDIR is set to TRUE 211-Server site variable LISTLEVEL is set to 0 211 *** end of status ***
Here is a sample session where I manually typed the commands necessary to upload JCL, execute it on the mainframe, and download the results:
Shell
$ ftp ftp://MIKES01:$MAIN_PWD@maintec Connected to maintec. 220-FTPD1 IBM FTP CS V2R4 at MTI1, 14:00:30 on 2024-04-08. 220 Connection will close if idle for more than 5 minutes. 331 Send password please. 230 MIKES01 is logged on. Working directory is "MIKES01.". Remote system type is MVS. 200 Representation type is Image ftp> ascii 200 Representation type is Ascii NonPrint ftp> quote site file=jes 200 SITE command was accepted ftp> put sessions/cicsadp_loadlib_amblist.jcl local: sessions/cicsadp_loadlib_amblist.jcl remote: sessions/cicsadp_loadlib_amblist.jcl 229 Entering Extended Passive Mode (|||2339|) 125 Sending Job to JES internal reader FIXrecfm 80 100% |****************************************************************************************************************************| 399 350.71 KiB/s --:-- ETA 250-It is known to JES as JOB01475 250 Transfer completed successfully. 399 bytes sent in 00:00 (4.76 KiB/s) ftp> get J1475 my.downloaded.job.output.txt local: my.downloaded.job.output.txt remote: J373 229 Entering Extended Passive Mode (|||14511|) 125 Sending all spool files for requested Jobid 1128 48.17 KiB/s 250 Transfer completed successfully. 1128 bytes received in 00:00 (25.57 KiB/s) ftp> bye 221 Quit command received. Goodbye.
The syntax for retreiving the results is simply GET J followed by the job number; leading zeros must be omitted,
followed by the name of the file on the local machine to save the output as.
Translating Machine Codes
Mainframe formatted job output has a carriage control character at the start of each line,
and the JES2 spooler writes !! END OF JES SPOOL FILE !! at the end of each spool file.
These files can be retrieved individually
or retrieved as a group.
There are two types on carriage control codes: ASA and Machine. The last partial spool file has Machine control characters that will most likely display as unusual glyphs.
IBM's documentation is unhelpful.
The ASA control characters are:
- Blank
- space one line
- Zero (0)
- space 2 lines
- Minus (-)
- space 3 lines
- Plus (+)
- Suppress line advance (Used by old impact printers for boldfacing)
- One (1)
- Skip to new line on new page
IBM machine code printer control characters also precede each line. They are typically non-printable, one byte hexadecimal characters. These characters need to be translated.
run_jcl Bash Script
I wrote the run_jcl script to upload JCL, execute it, and download the results.
#!/bin/bash
# (C) Mike Slinn mslinn@mslinn.com All rights reserved
function help {
if [ "$1" ]; then printf "Error: $1\n\n"; fi
echo "$(basename $0) does the following:
- Uses FTP to upload a given local JCL file (encoded in ASCII),
to a temporary file on a mainframe (automagically encoded in EBCDIC),
- Where it is run,
- Using the mainframe user id specified by environment variable MAIN_UID,
- And the mainframe user password specified by environment variable MAIN_PWD,
- Downloads the resulting ASCII output files (which are concatenated together),
- Translates machine control codes into a UTF-8 equivalent,
- Saves the result as a UTF-8 file in the same local directory as the original JCL file.
The IP address or domain name of the mainframe ftp server can be specified on the command line,
otherwise the environment variable MAIN_IP_OR_DOMAIN is used to provide the default value of the IP address.
The JCL file:
- name must end with a .jcl filetype, in lower case.
- must only contain printable ASCII characters and spaces.
- must not contain tabs or control characters.
- Should have a job name on the job card consisting of the user id plus one character.
The path provided for the JCL file:
- can be relative or absolute.
- must not contain spaces.
The intermediate .ascii and .jobinfo files and the resulting .txt file will be created in the same
directory as the .jcl file used to create them.
Syntax:
$ $(basename $0) [OPTIONS] JCL_FILENAME_WITHOUT_EXT [IP_ADDRESS]
Where:
OPTIONS can include:
-d Delete intermediate files (.ascii and .jobinfo)
-f Overwrite output and intermediate file if specified.
-h Display this help text.
JCL_FILENAME_WITHOUT_EXT is the path to the JCL file, with or without the .jcl filetype.
IP_ADDRESS is the subdomain or IP address of the ftp server
Sample invocations:
$ $(basename $0) amblist_cicsadp_loadlib 123.456.789.012
Uploading 'amblist_cicsadp_loadlib.jcl' to 123.456.789.012
Saved job run information as 'amblist_cicsadp_loadlib.jobinfo'
Downloading JOB1234 results as 'amblist_cicsadp_loadlib.ascii'
Writing final result to 'amblist_cicsadp_loadlib.txt'
Deleting 'amblist_cicsadp_loadlib.ascii'
$ export MAIN_IP_OR_DOMAIN=123.456.789.012
$ $(basename $0) amblist_cicsadp_corba -df
Uploading 'amblist_cicsadp_corba.jcl' to 123.456.789.012
Saved job run information as 'amblist_cicsadp_corba.jobinfo'
Downloading JOB1234 results as 'amblist_cicsadp_corba.ascii'
Writing final result to 'amblist_cicsadp_corba.txt'
Deleting 'amblist_cicsadp_corba.ascii'
"
exit 2
}
unset DELETE_INTERMEDIATE
unset OVERWRITE
OPTSTRING=":dfh"
while getopts ${OPTSTRING} opt; do
case ${opt} in
d) export DELETE_INTERMEDIATE=true ;;
f) export OVERWRITE=true ;;
h) help ;;
?) help ;;
esac
done
shift $((OPTIND-1))
if [ -z "$1" ]; then help "No JCL file was specified."; fi
if [ "$2" ]; then
MAIN_IP_OR_DOMAIN="$2"
else
if [ -z "$MAIN_IP_OR_DOMAIN" ]; then
help "Environment variable MAIN_IP_OR_DOMAIN is undefined and the mainframe IP address was not specified on the command line"
fi
fi
if [ -z "$MAIN_PWD" ]; then help "Environment variable MAIN_PWD is undefined"; fi
if [ -z "$MAIN_UID" ]; then help "Environment variable MAIN_UID is undefined"; fi
# Ensure $PATH_WITHOUT_EXT has no filetype
if [[ "$1" == *.jcl ]]; then
PATH_WITHOUT_EXT="$( echo "$1" | rev | cut -f 2- -d '.' | rev )"
else
PATH_WITHOUT_EXT="$1"
fi
JCL_FILE="$PATH_WITHOUT_EXT.jcl"
if [ ! -f "$JCL_FILE" ]; then help "JCL file '$JCL_FILE' does not exist."; fi
TABS="$( grep -oP '\t' < "$JCL_FILE" | wc -l )"
if [ "$TABS" != 0 ]; then help "JCL file '$JCL_FILE' contains $TABS tab characters"; fi
STEP_1_OUTPUT="$PATH_WITHOUT_EXT.jobinfo"
ASCII_FILE="$PATH_WITHOUT_EXT.ascii"
UTF8_FILE="$PATH_WITHOUT_EXT.txt"
if [ -f "$UTF8_FILE" ] && [ ! "$OVERWRITE" ]; then
echo "Error: '$UTF8_FILE' already exists and the -f option was not specified. Aborting."
exit 3
fi
rm -rf "$STEP_1_OUTPUT" "$ASCII_FILE" "$UTF8_FILE"
echo "Uploading '$JCL_FILE'"
# Send JCL in ASCII mode, not binary FTP mode (which is the default).
# The mainframe FTP server will translate the incoming ASCII ftp subcommands to EBCDIC
# and returns results in ASCII.
# The mainframe responses are saved in $STEP_1_OUTPUT, which will be parsed for the job number
# in the next step.
ftp ftp://$MAIN_UID:$MAIN_PWD@$MAIN_IP_OR_DOMAIN > "$STEP_1_OUTPUT" <<EOF
ascii
quote site filetype=jes
put $JCL_FILE
dir
bye
EOF
echo "Saved job run information as '$STEP_1_OUTPUT'"
# $STEP_1_OUTPUT might have contents similar to the following:
# MIKES01X JOB01468 ACTIVE
# MIKES01S JOB00373 OUTPUT 3 Spool Files
# MIKES01X JOB00923 OUTPUT 3 Spool Files
# MIKES01X JOB00926 OUTPUT 3 Spool Files
# MIKES01X JOB01345 OUTPUT 4 Spool Files
# MIKES01X JOB01462 OUTPUT 4 Spool Files
#
# Successive jobs have higher numbers.
# Job numbers wrap, but it takes a while before JOB99999 wraps to JOB00000.
# Some systems list oldest jobs first, others list newest jobs first.
# Sometimes the current job is listed as ACTIVE, sometimes not. Unclear how that works.
# Parse the job number from $STEP_1_OUTPUT, remove the leading 'JOB' and leading zeros,
# and use the parsed job number in the ftp 'get' subcommand below.
# First try looking for active jobs, then look at the complete job history if no active jobs were found
JOB_NUMBER="$( grep ACTIVE < "$STEP_1_OUTPUT" | grep -oP "JOB\K[\d]+" | sed 's/^0*//' | sort -n | tail -n 1 )"
if [ -z "$JOB_NUMBER" ]; then # There is no active job, so fetch the most recent job results instead
JOB_NUMBER="$( grep -oP "JOB\K[\d]+" < "$STEP_1_OUTPUT" | sed 's/^0*//' | sort -n | tail -n 1 )"
fi
echo "Downloading JOB$JOB_NUMBER results as '${ASCII_FILE}'"
# Download the results of running the JCL in the first ftp session;
# the results from the mainframe are encoded in ASCII.
# All mainframe output files are concatenated together, with machine codes embedded.
# Each file is delimited by:
# !! END OF JES SPOOL FILE !!<U+0085>
ftp ftp://$MAIN_UID:$MAIN_PWD@$MAIN_IP_OR_DOMAIN <<EOF
ascii
quote site filetype=jes
get J${JOB_NUMBER} ${ASCII_FILE}
bye
EOF
echo "Writing final result to '$UTF8_FILE'"
# - Translate machine codes in "$ASCII_FILE" into UTF-8 equivalents
# - Save as "$UTF8_FILE"
sed -e "s/\x85/\f/g" \
-e "s/\x0B/\n/g" \
-e "s/\x13/\n\n/g" \
-e "s/\x1B/\n\n\n/g" \
< "$ASCII_FILE" > "$UTF8_FILE"
# Potentially delete the intermediate .ascii and .jobinfo files
if [ "$DELETE_INTERMEDIATE" ]; then
echo "Deleting $STEP_1_OUTPUT and $ASCII_FILE"
rm -f "$STEP_1_OUTPUT" "$ASCII_FILE"
fi
read -n 1 -s -r -p "Press any key to view '$UTF8_FILE', or Control-C to terminate."
less "$UTF8_FILE"
This script works fairly well, but bash is hard to maintain and sometimes this script does not return the proper results file.
jcl Ruby Script
I decided to rewrite the Bash script in Ruby.
After some experimentation, I looked at the source code for the net-ftp
project and realized that if I implemented the literal (aka quote subcommand) then
a simple Ruby script could replace the awkward Bash script.
The only other change that was necessary
was to modify the method call which returns to the calling routine for two methods, a change of 8 characters.
First, I created a new issue in the net-ftp project.
Then I wrote 4 short new lines
of code, add 4 short lines of comments, and incremented the version number of the net-ftp gem.
I only added a total of 312 characters to the source code.
The modifications in the pull request were tested against an IBM z/OS Communications Server.
The documentation consisted of the following code example that shows how to use the enhanced net-ftp library.
The code example had three times as many lines of code than the code changes I made to the library.
The following code fragment uses FTP to submit a batch job to the mainframe, and collects the results.
Sample usage
def parse_job_number_from(line)
tokens = line.split('JOB')
raise StandardError, "Error: The string 'JOB' was not found in the FTP response." unless tokens.length == 2
job_number = tokens[1].split.first
job_number.sub(/^0+/, '') # Remove leading zeros
end
Net::FTP.open(@ip_or_domain) do |ftp|
ftp.login @uid, @pwd
ftp.quote 'site filetype=jes'
job_info = ftp.puttextfile @local_jcl_filepath
job_number = parse_job_number_from job_info
ftp.gettextfile "J#{job_number}", @local_results_file
end
The version of the net-ftp Ruby gem containing the new feature was tagged as v0.3.6
and included in the v0.3.7 release.
The Ruby script works flawlessly!
