sub strings extraction from multiline output

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
Mahendra
Posts: 26
Joined: 23 Sep 2012 02:29

sub strings extraction from multiline output

#1 Post by Mahendra » 07 Jan 2014 10:48

i request you to please help to extract the sub strings from the multiline output. Thanks in advance

we have a command in MQ in display the channel status. that command display the status of different types of channels ( SVRCONN/CLUSSDR/CLUSRCVR/SDR/RCVR). all the types channels does not have all common properties. only few of the properties are common. i want to extract the value of property.

command :
dis chs(*) chltype chstada chstati msgs sslpeer

The above command displays the data in below format.( in multiple lines). here i have written sample outputs.

CHANNEL(ABCD) CHLTYPE(SVRCONN)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
msgs(1009) sslpeer(kjhdsfkjsd)
substate(Recive)



CHANNEL(DEFG) CHLTYPE(SDR)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
substate(MQGet) msgs(10020)
sslpeer(kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj)
XMITQ(ABCD.TEST)


CHANNEL(GHIJ) CHLTYPE(CLUSRCVR)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
current mcauser(ahghj)
substate(Recive) sslpeer(kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj)


i want to extract the property values

channel | chltype | chstada |chstati |status |msgs |sslpeer
ABCD | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd
DEFG | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd
GHIJwewe | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd



some channels have some extra properties and may not end all type channels with same property. for example SDR channel having extra property xmitq. but all the channels display start with channel name.

The proprty value may not standard length and position. it varies from channel to channel. suppose channel name can have max allowed upto 20 chars..

could you please some one guide me on this. with the below thread solution i tried ....
viewtopic.php?f=3&t=5227

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: sub strings extraction from multiline output

#2 Post by dbenham » 07 Jan 2014 12:03

Using REPL.BAT

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "channel="

:: Define $D to contain output delimiter. Any string could be used.
set "$D=|"

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Process the command output
setlocal enableDelayedExpansion
>output.txt (
  echo %$header%
  for /f "tokens=1,2 delims=()" %%A in (
    'dis chs(*) chltype chstada chstati msgs sslpeer^|repl "\) *" "\n" x'
  ) do (
    if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
    set "%%A=%%B"
  )
  if defined channel echo %$data%
)

The code assumes you do not have ! in your data. It should be easy to toggle delayed expansion on and off if you need to support ! in data.

EDIT - Simplified logic and added a few comments

Dave Benham
Last edited by dbenham on 07 Jan 2014 22:04, edited 1 time in total.

Aacini
Expert
Posts: 1885
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: sub strings extraction from multiline output

#3 Post by Aacini » 07 Jan 2014 15:30

The Batch file below use FindRepl.bat program.

Code: Select all

@echo off
set "caret=^"
setlocal EnableDelayedExpansion

rem Get parameters and assemble from they:
rem 1- The regexp to search for multiple terms
rem 2- The list of submatched subexpressions
rem 3- The headers of the columns
rem The first parameter "channel" is pre-inserted
set "regexp=(channel\([!caret!\)]*)\)"
set "subexprs=:1"
set header[1]=channel
set n=1
:nextParam
   set "regexp=!regexp!|%1\(([!caret!\)]*)\)"
   set /A n+=1
   set subexprs=%subexprs%:%n%
   set header[%n%]=%1
   shift
if "%1" neq "" goto nextParam

rem Get the command output
dis chs(*) %* > data.txt

rem Show headers
set "line="
for /L %%i in (1,1,%n%) do (
   rem Insert here code for width column alignment
   set "line=!line!  !header[%%i]!  |"
)
echo !line:~0,-1!

rem Perform the search. FindRepl program return one line per SUBEXPRESSION (%n% lines per channel)
rem with %n% values in each line enclosed in quotes: all values empty excepting the submatched one

for /L %%i in (1,1,%n%) do set var[%%i]=Undefined
for /F "delims=" %%a in ('^< data.txt FindRepl /I "!regexp!" /$%subexprs%') do (
   set i=0
   for %%b in (%%a) do (
      set /A i+=1
      if %%b neq "" (
         for /F "tokens=1,2 delims=(" %%c in (%%b) do set "next=%%c" & set "channel=%%d"
         if /I "!next!" equ "channel" (
            rem Data for next channel started: show the previous one, if any
            if "!var[1]!" neq "Undefined" (
               set "line="
               for /L %%i in (1,1,%n%) do (
                  rem Insert here code for width column alignment
                  set "line=!line!  !var[%%i]!  |"
               )
               echo !line:~0,-1!
            )
            rem Restart data for next channel
            set "var[1]=!channel!"
            for /L %%i in (2,1,%n%) do set var[%%i]=Undefined
         ) else (
            set "var[!i!]=%%~b"
         )
      )
   )
)

set "line="
for /L %%i in (1,1,%n%) do (
   rem Insert here code for width column alignment
   set "line=!line!  !var[%%i]!  |"
)
echo !line:~0,-1!


For example:

Code: Select all

C:\> test chltype chstada  chstati  status  msgs sslpeer
  channel  |  chltype  |  chstada  |  chstati  |  status  |  msgs  |  sslpeer
  ABCD  |  SVRCONN  |  10-12-2013  |  10:20:30  |  RETRYING  |  1009  |  kjhdsfkjsd
  DEFG  |  SDR  |  10-12-2013  |  10:20:30  |  RETRYING  |  10020  |  kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj
  GHIJ  |  CLUSRCVR  |  10-12-2013  |  10:20:30  |  RETRYING  |  Undefined  |  kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj


Antonio

berserker
Posts: 95
Joined: 18 Dec 2013 00:51

Re: sub strings extraction from multiline output

#4 Post by berserker » 07 Jan 2014 20:02

awk script

Code: Select all


function strip( string ){
    # remove anything from beginning to the first ( ,
    # and remove anything from last bracket ) till the end
    gsub(  /.*\(|\).*/ , "", string )   
    return string
}

BEGIN{
    RS="\n\n+"      # set record separator to be more than 1 newlines
    FS="[ \n]+"     # set field separator to be space or newline
    IGNORECASE=1    # case insensitive search
    sep       ="|"  # output delimiter
}
{
    for( i=1; i<=NF; i++){
        printf  $i ~ /CHANNEL/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHLTYPE/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHSTADA/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHSTATI/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /STATUS/  ? (strip( $i ) sep ) : ""
        printf  $i ~ /msgs/    ? (strip( $i ) sep ) : ""
        printf  $i ~ /sslpeer/ ? (strip( $i ) sep ) : ""
       
    }
    print ""

}



result

Code: Select all

C:\>awk -f  myScript.awk  myStatusFile.txt
ABCD|SVRCONN|10-12-2013|10:20:30|RETRYING|1009|kjhdsfkjsd|
DEFG|SDR|10-12-2013|10:20:30|RETRYING|10020|kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj|
GHIJ|CLUSRCVR|10-12-2013|10:20:30|RETRYING|kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj|


you can also pass your status to the script using pipe

Code: Select all

`your MQ command` | awk -f myScript.awk 

Mahendra
Posts: 26
Joined: 23 Sep 2012 02:29

Re: sub strings extraction from multiline output

#5 Post by Mahendra » 07 Jan 2014 21:22

is there anyway without using any third party tools..i.e pure batch script.

berserker
Posts: 95
Joined: 18 Dec 2013 00:51

Re: sub strings extraction from multiline output

#6 Post by berserker » 07 Jan 2014 21:40

Mahendra wrote:is there anyway without using any third party tools..i.e pure batch script.

then have you tried the batch solutions others posted?

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: sub strings extraction from multiline output

#7 Post by dbenham » 07 Jan 2014 21:58

Mahendra wrote:is there anyway without using any third party tools..i.e pure batch script.

sure :)

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "channel="

:: Define $D to contain output delimiter. Any string could be used.
set "$D=|"

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Define LF to contain linefeed character (0x10)
set ^"LF=^

^" Above empty line is critical - DO NOT REMOVE

:: Process the command output
setlocal enableDelayedExpansion
>output.txt (
  echo %$header%
  for %%L in ("!LF!") do for /f "delims==" %%X in (
    'dis chs(*) chltype chstada chstati msgs sslpeer'
  ) do (
    set "ln=%%X"
    for %%L in ("!LF!") do set "ln=!ln:) =%%~L!"
    for /f "tokens=1,2 delims=()" %%A in ("!ln:) =%%~L!") do (
      if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
      set "%%A=%%B"
    )
  )
  if defined channel echo %$data%
)


Dave Benham

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: sub strings extraction from multiline output

#8 Post by foxidrive » 07 Jan 2014 23:37

Mahendra wrote:i request you to please help to extract the sub strings from the multiline output. Thanks in advance


The quote in blue really means: "I am too lazy to come back and tell you if your solution worked, and to personally thank you for writing free code for me, in your spare time"

Don't mind me, it's just one of my petty annoyances. :D

berserker
Posts: 95
Joined: 18 Dec 2013 00:51

Re: sub strings extraction from multiline output

#9 Post by berserker » 08 Jan 2014 00:10

foxidrive wrote:
Mahendra wrote:Don't mind me, it's just one of my petty annoyances. :D

lol, foxi i think this happens quite common in forums.

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: sub strings extraction from multiline output

#10 Post by foxidrive » 08 Jan 2014 03:58

berserker wrote:lol, foxi i think this happens quite common in forums.


It does. I sometimes rant about it. :)

Squashman
Expert
Posts: 4465
Joined: 23 Dec 2011 13:59

Re: sub strings extraction from multiline output

#11 Post by Squashman » 08 Jan 2014 07:33

Mahendra wrote:is there anyway without using any third party tools..i.e pure batch script.

So what don't you like about REPL.bat and FINDREPL.bat? Sure they are hybrid scripts but they are still native to Windows. It isn't technically a 3rd party utility.

Mahendra
Posts: 26
Joined: 23 Sep 2012 02:29

Re: sub strings extraction from multiline output

#12 Post by Mahendra » 08 Jan 2014 10:28

due to limitations, The extra scripts are not allowed to copy in the release bundle with out documentation and information to all the teams. so, it will be preferable single script.

The solution provide in the earlier conversations is not working for all the scenarios. it is working for the first channel only.

here i have a actual output of the command. few minor property values changed. HIGHLIGHTED FEW OF THE COMPONENTS FOR DIFFERENCE. there is no space in the command output.

command output:

AMQ8417:Display Channel Status details
CHANNEL(MQSTE.SVCS.S) CHLTYPE(SVRCONN)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.1.234) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(234) SSLPEER(CN=mqt.abc.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(RECEIVE)
AMQ8417:Display Channel Status details
CHANNEL(SDR.MQSTE.SVCS.S) CHLTYPE(CLUSSDR)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.12.234(1234)) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(2) SSLPEER(CN=mqt.asdfghjj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(MQGET)
XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8417:Display Channel Status details
CHANNEL(RCVR.MQSTE.SVCS.S) CHLTYPE(CLUSRCVR)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.12.23) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(5) SSLPEER(CN=mqt.ghjjghyj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(RECEIVE)


required format:

CHANNEL|CHLTYPE|CHSTADA|CHSTATI|MSGS|STATUS | SSLPEER

MQSTE.SVCS.S | SVRCONN | 2014-01-08 | 01.02.46 | 234| RUNNING | CN=mqt.abc.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US
SDR.MQSTE.SVCS.S | CLUSSDR | 2014-01-08 | 01.02.46 | 2| RUNNING | CN=mqt.asdfghjj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US
RCVR.MQSTE.SVCS.S | CLUSRCVR | 2014-01-08 | 01.02.46 | 2| RUNNING | CN=mqt.ghjjghyj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US

Squashman
Expert
Posts: 4465
Joined: 23 Dec 2011 13:59

Re: sub strings extraction from multiline output

#13 Post by Squashman » 08 Jan 2014 10:59

Mahendra wrote:due to limitations, The extra scripts are not allowed to copy in the release bundle with out documentation and information to all the teams. so, it will be preferable single script.

Plenty of documentation for Repl.bat
viewtopic.php?f=3&t=3855

And for FindRepl.bat
viewtopic.php?f=3&t=4697

Mahendra
Posts: 26
Joined: 23 Sep 2012 02:29

Re: sub strings extraction from multiline output

#14 Post by Mahendra » 08 Jan 2014 11:06

There other limitations to use...., like...it will always preferable single script. because to reduce the dependency of other script. size of the release bundle.....

dbenham
Expert
Posts: 2461
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: sub strings extraction from multiline output

#15 Post by dbenham » 08 Jan 2014 11:47

I had a silly bug in my original pure batch solution. It also had a redundant FOR loop.

Below is fixed code, with the delimiter set to " | " instead of "|". I tested with a static file containing the sample output instead of command output.

Code: Select all

@echo off
setlocal disableDelayedExpansion

:: Define $D to contain output delimiter. Any string could be used.
set "$D= | "

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Define LF to contain linefeed character (0x10)
set ^"LF=^

^" Above empty line is critical - DO NOT REMOVE

:: Process the command output
setlocal enableDelayedExpansion
set "channel="
>output.txt (
  echo %$header%
  for %%L in ("!LF!") do for /f "delims=" %%X in (
    'dis chs(*) chltype chstada chstati msgs sslpeer'
  ) do (
    set "ln=%%X"
    for /f "tokens=1,2 delims=()" %%A in ("!ln:) =%%~L!") do (
      if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
      set "%%A=%%B"
    )
  )
  if defined channel echo %$data%
)


Dave Benham

Post Reply