Variable Replacement in a file

[Squashman]

So some of you may know I work a lot on a mainframe so I use a lot of JCL (Job Control Language) which is nothing more than a text file that tells the Job Scheduler on the Mainframe what files to use and what programs to run.

A lot of our JCL programs are triggered from our Windows Application Servers with nothing more than a .BAT file. The batch file just runs an FTP script to upload the JCL.txt file to the Job Scheduler on the mainframe.

Most of our JCL programs are fixed text files but sometimes I have to do some variable substitution within the JCL.txt file before I upload it to the mainframe to run. I normally just do this with a VBscript because it works quite nicely to read the entire JCL.txt file into a variable and then just use the REPLACE command and then write it out to a new file to upload to the mainframe.

So my task today is to do that variable substitution within a normal batch file without using any hybrid techniques or external tools.

So lets examine this scenario. We have software that watches for a file in a specific directory. When it sees a new file in the directory it kicks off the batch file and passes the file name it found to the batch file. That file name needs to be substituted into the JCL.txt file by finding the variable %parmin% and write it out to a temp file that will then be uploaded to the job scheduler on the mainframe.

JCL.txt

Code: Select all

//$1CLTCAD JOB FORMAT,'CLT',CLASS=A,MSGLEVEL=(1,1),MSGCLASS=X                                                   00010001
//JOBLIB   DD DSN=GCP.MACH1.PROD.LINKLIB,DISP=SHR                                                               00020001
//         DD DSN=DIS.PROD.LINKLIB,DISP=SHR                                                                     00030001
//********************************************************************                                          00050001
//*   jcl test                                                      **                                          00060001
//********************************************************************                                          00080001
//DEL010   EXEC PGM=IEFBR14                                                                                     00320001
//IEF01    DD DSN=DMR.CLTNPROD.ALLOCAT.DOCMAST(+01),                                                            00330001
//             DISP=(MOD,DELETE),                                                                               00340001
//             UNIT=(DISK,,DEFER),                                                                              00350001
//             SPACE=(TRK,0)                                                                                    00360001
//IEF02    DD DSN=DMR.CLTNPROD.PROJECT.STATUS(+01),                                                             00370001
//             DISP=(MOD,DELETE),                                                                               00380001
//             UNIT=(DISK,,DEFER),                                                                              00390001
//             SPACE=(TRK,0)                                                                                    00400001
//*                                                                                                             00410001
//STEP010  EXEC SAS                                                                                             00420001
//MASTER   DD DSN=DMR.CLTNPROD.ALLOCAT.DOCMAST(0),DISP=SHR                                                      00430001
//RECONIN  DD DSN=DMR.CLTNPROD.PROJECT.STATUS(0),DISP=SHR                                                       00440001
//PARMIN   DD *                                                                                                 00450001
%parmin%                                                                                                        00460001
/*                                                                                                              00470001
//MASTOUT  DD DSN=DMR.CLTNPROD.ALLOCAT.DOCMAST(+01),                                                            00510001
//            DISP=(NEW,CATLG,DELETE),                                                                          00520001
//            UNIT=SMS,MGMTCLAS=TRANS,                                                                          00530001
//            SPACE=(CYL,(250,250),RLSE),                                                                       00540001
//            DCB=(LRECL=200,BLKSIZE=0,RECFM=FB,BUFNO=10)                                                       00550001
//RECONUP  DD DSN=DMR.CLTNPROD.PROJECT.STATUS(+01),                                                             00560001
//            DISP=(NEW,CATLG,DELETE),                                                                          00570001
//            UNIT=SMS,MGMTCLAS=TRANS,                                                                          00580001
//            SPACE=(CYL,(250,250),RLSE),                                                                       00590001
//            DCB=(LRECL=1000,BLKSIZE=0,RECFM=FB,BUFNO=10)                                                      00600001
//SYSIN    DD DSN=DMR.CLTNIFE.FILES(ALLOADHS),DISP=SHR                                                          00610001
//*                                                                                                             00620001
//  IF STEP010.SAS.RC > 4 THEN                                                                                  00630001
//*                                                                                                             00640001
//DEL020   EXEC PGM=IEFBR14                                                                                     00680001
//IEF02    DD DSN=DMR.CLTNPROD.ALLOCAT.DOCMAST(+01),                                                            00690001
//            DISP=(MOD,DELETE),                                                                                00700001
//            UNIT=(DISK,,DEFER),                                                                               00710001
//            SPACE=(TRK,0)                                                                                     00720001
//IEF03    DD DSN=DMR.CLTNPROD.PROJECT.STATUS(+01),                                                             00730001
//            DISP=(MOD,DELETE),                                                                                00740001
//            UNIT=(DISK,,DEFER),                                                                               00750001
//            SPACE=(TRK,0)                                                                                     00760001
//*                                                                                                             00770001
//  ENDIF                                                                                                       00780001
//                                                                                                              00790001

For the sake of testing I am just hard coding my parmin variable. And I thought I would just give the CALL ECHO a try to see if it would do the DOUBLE expansion of the variable.

Code: Select all

@echo off
setlocal
:: set parmin=%1
set parmin=foo.txt
for /f "delims=" %%G in (JCL.txt) do echo %%G
pause

Sure enough it expands the variable.

Code: Select all

//PARMIN   DD *                                                                                                 00450001
foo.txt                                                                                                        00460001
/*                                                                                                              00470001
//MASTOUT  DD DSN=DMR.METLPROD.ALLOCAT.DOCMAST(+01),                                                            00510001

Problem is the stupid special characters. It got rid of "> 4" on this line.

Code: Select all

//  IF STEP010.SAS.RC  THEN                                                                                  00630001

I thought I could do something like this but this does not work because it seems to be expanding the variable when it PIPES it to FIND so the FIND command never finds %parmin%.

Code: Select all

@echo off
setlocal
:: set parmin=%1
set parmin=foo.txt
for /f "delims=" %%G in (JCL.txt) do (
   echo %%G |find "%%parmin%%" && (call echo %%G) || (echo %%G)
)
pause

Not sure where to go from here.

[aGerman]

Doesn't work with && and || but works with
if errorlevel 1 ...
for whatever reason.

But even this doesn't look like a script with good performance. Let me think about it ...

Regards
aGerman

EDIT
Er ... no ... I tricked myself out.

EDIT2
What about

Code: Select all

@echo off &setlocal

set "parmin=foo.txt"


for /f "delims=" %%G in (JCL.txt) do (
  set "ln=%%G"
  setlocal EnableDelayedExpansion
  echo !ln:$$parmin$$=%parmin%!
  endlocal
)

pause

with the string $$parmin$$ in JCL.txt instead of %parmin%?

[Squashman]

Performance is not an issue as this file as you can see is very small. Most of the JCL files are not more than 100 lines.

[aGerman]

Sorry I edited again while you answered... Would the above be an alternative?

[Squashman]

with the string $$parmin$$ in JCL.txt instead of %parmin%?

I guess if I don't we will just be fighting the percent symbols. Most of our JCL files are already using the percent symbols for the variable replacement so I kind of wanted to keep it consistent but I think it will never work unless we change to a different symbol for the variable replacement.

[aGerman]

Well if you insist on the percent signs you could actually use them with the string replacement.

Code: Select all

  echo !ln:%%parmin%%=%parmin%!

It's more confusing but works.

[Squashman]

I guess I was trying to be too tricky with the expansion. Got to keep it simple.

[foxidrive]

Are hybrid tools not approved because parts of WSH are locked down?

Is VBS also locked down?

[Squashman]

Are hybrid tools not approved because parts of WSH are locked down?

Is VBS also locked down?

Definitely not locked down and I normally do write this types of scripts with VBscript but I already have an existing Batch file that is already doing a bunch of stuff for this automation task. I really want to keep it self contained to keep it simple. If I have it call out to another script that will just confuse the hell out of the two people that have to back me up. They barely understand batch and vbscript. If I throw in a hybrid script that will just be more confusing for them.

Any new tasks that I have been creating all get put into Vbscript when I have to manipulate files.

[foxidrive]

It would be harder when others also need to be on top of everything.

I was just thinking that a VBS replace would be more robust, if there are reasons why say repl.bat can't be added as a system wide utility.

[Squashman]

It would be harder when others also need to be on top of everything.

I was just thinking that a VBS replace would be more robust, if there are reasons why say repl.bat can't be added as a system wide utility.

I like Repl.bat because of the things I have seen everyone do with it but I have not had time to play with it or understand all the code inside of it. I don't want to add into any automation task until I fully understand how to use it. My primary function is to keep the workflow going with automated tasks so everything is hands off. This is a combination of in-house programs written in VB5, batch files, vbscripts, JCL pushed to the mainframe and program called WatchDirectory.

Not that I don't trust Dave's code I just need to feel warm and fuzzy before I use it.

[Samir]

I'm like you where I like to understand code (or at least all the test cases) before I implement or try to use it. I found it worth the time to figure out a 'replace' batch file that suited my purpose.

Then, to avoid calling external batches, I simply incorporated the code into my own batch. It's single file simplicity at its best.

[dbenham]

For the sake of testing I am just hard coding my parmin variable. And I thought I would just give the CALL ECHO a try to see if it would do the DOUBLE expansion of the variable.

Code: Select all

@echo off
setlocal
:: set parmin=%1
set parmin=foo.txt
for /f "delims=" %%G in (JCL.txt) do echo %%G
pause

Sure enough it expands the variable.

That can't work
I think you meant

Code: Select all

for /f "delims=" %%G in (JCL.txt) do CALL echo %%G

And yes, poison characters are unprotected, and quoted ^ are doubled. (and of course, empty lines are discarded, and lines starting with ; are discarded)

For a quick implementation of a template file with replaceable content, I like to use !parmin! instead of %parmin%. Then your replace loop can be as simple as:

Code: Select all

setlocal enableDelayedExpansion
set "parmin=foo.bar"
for /f "delims=" %%G in (JCL.txt) do echo %%G

Your only concerns become

- empty lines are discarded, easily solved if need be
- lines beginning with ; are discarded, easily solved if need be
- ! literals in template are not preserved
- ^ literals in template are not preserved if ! also appears on the line

! and ^ literals may be encoded within the template as !B! and !C!, and then define those replacement variables in your script

Code: Select all

setlocal enableDelayedExpansion
set "parmin=foo.bar"
set "B=^!"
set "C=^"
for /f "delims=" %%G in (JCL.txt) do echo %%G

Dave Benham

[Squashman]

Empty lines are not an issue with JCL. You basically cannot have an empty line in JCL because the first empty line it finds stops execution. Now that works for testing purposes when you want the job automation to stop at a certain point but once you go to full automation any empty lines have to be commented for it to continue processing.

[dbenham]

That's fine. I know nothing about JCL.

But I was speaking in terms of a general solution for a template file with replaceable parameters.

I forgot there is one other restriction - line length is limited to ~8k


Dave Benham