FOR LOOP Question - Delimiters

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
SIMMS7400
Posts: 539
Joined: 07 Jan 2016 07:47

FOR LOOP Question - Delimiters

#1 Post by SIMMS7400 » 09 Jan 2019 19:54

Hi Folks -

I have what I believe is a very easy question.

If I have a file with a line:
,Total Tire Centers
And my FOR LOOP looks like this:

Code: Select all

FOR /F "USEBACKQ TOKENS=1-2 DELIMS=," %%A IN ("%CNTRL_FILE%") DO ECHO "%%~A" "%%~B"
Why does "%%~A" return "Total Tire Centers" instead of just ""? Ideally, I want to be able to check if %%A is NULL but as of now, it seems as though %%A would never be NULL, which is not what I want.

Sorry for the - i'm sure - very easy question!
Thanks!

penpen
Expert
Posts: 1991
Joined: 23 Jun 2013 06:15
Location: Germany

Re: FOR LOOP Question - Delimiters

#2 Post by penpen » 10 Jan 2019 04:37

A tokenization is defined to decompose a given string into multiple strings in a unique way.
When using a simple (delimiter based only) tokenizer, there are only two possible ways to achieve that:
1) Each delimiter starts a new token, so empty words are tokens.
2) The first character in a word starts a new token, so you empty words aren't recognized as tokens.

For whatever reasons Mocrosoft decided to use the second definition.

It's just a guess, so i don't know for sure, but:
It might be beacause you it is easier to simulate the first tokenizing using the second (you just need to replace each seperator with a seperator and any no-delimiter-character when calling for, and remove the first character accessing that value), than the other way around (you have to remove the empty words, so you first need to know how many tokens you will get - which is anything but trivial) .

Also it is more flexible when accessing tokens defined by length, using some well defined characters for padding, which was used for example to store bank account numbers and bank routing numbers; example:

Code: Select all

1234567890,1234567890,1234567890
         1,        12,       123
1000      ,1001      ,10002     

penpen

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: FOR LOOP Question - Delimiters

#3 Post by aGerman » 10 Jan 2019 11:16

There is no foolproof method to process csv data in batch. Try something like that:

Code: Select all

@echo off &setlocal
set "file=test.csv"

setlocal EnableDelayedExpansion
<"!file!" (
  for /f %%i in ('type "!file!"^|find /c /v ""') do for /l %%j in (1 1 %%i) do (
    set "line=" &set /p "line="
    if defined line (
      set "line=!line:,=","!"
      for /f "tokens=1,2 delims=," %%k in ("!line!") do echo "%%~k" "%%~l"
    )
  )
)
pause
It will fail if the csv file contains quoted values and it gets even worse if quoted values contain commas.

Steffen

Post Reply