Rules for how CMD.EXE parses numbers

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
carlos
Expert
Posts: 489
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Rules for how CMD.EXE parses numbers

#31 Post by carlos » 24 May 2014 01:36

On windows 8 try set -2147483648 direclty using decimal notation (instead 0x80000000) cause the overflow message:

Code: Select all

set /a -2147483648
Invalid number.  Numbers are limited to 32-bits of precision.


but you can cause the overflow for set this number using this:

Code: Select all

set /a 1+2147483647
-2147483648


Also:

Code: Select all

set /a "(1<<31)"
-2147483648

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Rules for how CMD.EXE parses numbers

#32 Post by dbenham » 24 May 2014 07:03

Yes, that was one of the quirks that led me to investigate and write this thread.

The inability to directly input -2147483648 is strictly a limitation of the decimal number parser. It has nothing to do with the inherit limits of SET /A math capabilities.


Dave Benham

Dragokas
Posts: 42
Joined: 30 Jul 2013 09:42
Location: Ukraine, USSR
Contact:

Re: Rules for how CMD.EXE parses numbers

#33 Post by Dragokas » 24 May 2014 10:24

Additional info about IF statement:

When a difference beetween the operands is greater than MAX int-type (2147483647 + 1),
operand LSS always return 'FALSE'
operand LEQ always return 'FALSE'
operand GTR always return 'TRUE'
operand GEQ always return 'TRUE'

Code: Select all

@echo off
SetLocal EnableExtensions

set MAX=2147483640

For %%o in (LSS LEQ GTR GEQ) do (
  For /L %%n in (-9 1 -6) do (
    set /p =%%n %%o %MAX% ? <NUL
    call :# %%n %%o %MAX% & rem 'IF' statement don't support delayed method for operator
  )
  echo -------------------------
)
pause & exit /B

:# [_in_operand1] [_in_operator] [_in_operand2]
if %~1 %~2 %~3 (echo true  +) else (echo FALSE -)


Result:
-9 LSS 2147483640 ? FALSE -
-8 LSS 2147483640 ? true +
-7 LSS 2147483640 ? true +
-6 LSS 2147483640 ? true +
-------------------------
-9 LEQ 2147483640 ? FALSE -
-8 LEQ 2147483640 ? true +
-7 LEQ 2147483640 ? true +
-6 LEQ 2147483640 ? true +
-------------------------
-9 GTR 2147483640 ? true +
-8 GTR 2147483640 ? FALSE -
-7 GTR 2147483640 ? FALSE -
-6 GTR 2147483640 ? FALSE -
-------------------------
-9 GEQ 2147483640 ? true +
-8 GEQ 2147483640 ? FALSE -
-7 GEQ 2147483640 ? FALSE -
-6 GEQ 2147483640 ? FALSE -

EQU and NEQ works well here.

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Rules for how CMD.EXE parses numbers

#34 Post by dbenham » 24 May 2014 10:54

Dragokas wrote:When a difference beetween the operands is greater than MAX int-type (2147483647 + 1),
operand LSS always return 'FALSE'
operand LEQ always return 'FALSE'
operand GTR always return 'TRUE'
operand GEQ always return 'TRUE'
:shock: :evil:
OMG - That is so terrible :!:

A great discovery, but such a horrendous bug/design flaw by MS is inexcusable.

Presumably MS implemented the comparison as a subtraction, and then they look for either negative, zero, or positive. Math overflow would cause the bug.


Dave Benham

penpen
Expert
Posts: 1726
Joined: 23 Jun 2013 06:15
Location: Germany

Re: Rules for how CMD.EXE parses numbers

#35 Post by penpen » 24 May 2014 14:28

:( :!:
This means that any math-like batch file should be revised... ... ... .

For example the great MD5 Algorithm (uses IF %~Z1 GTR 268435455, and other inequations):

Code: Select all

IF -1879048192 GTR 268435455 (echo yes) else (echo Oh No)

Or the Multi-process Advanced Encryption Standard (AES) (if !d! geq 16, ...):

Code: Select all

if -2147483647 GEQ 16 echo a


... ... (or all the other great math tools)

penpen

carlos
Expert
Posts: 489
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Rules for how CMD.EXE parses numbers

#36 Post by carlos » 25 May 2014 04:29

Seems that this implemented like this, using a substractions as dbenham says:

lss = ( n1 - n2 ) < 0
leq = ( n1 - n2 ) <= 0
gtr = ( n1 - n2 ) > 0
geq = ( n1 - n2 ) >= 0
neq = ( n1 - n2 ) != 0
equ = ( n1 - n2 ) == 0

what bad, the substraction is all unnecessary, and is the bug.
But only operators distinct than equ and neq, and if are on different sign are affected.

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Rules for how CMD.EXE parses numbers

#37 Post by dbenham » 26 May 2014 11:55

Below is an IF macro that eliminates the bug. Unfortunately it is >150 times slower :(

Usage is simple:

Code: Select all

(%if% val1 condition val2) && (TrueBlock) || (FalseBlock)

If the last statment in the TrueBlock might raise an error, then an additional command must be added that always succeeds. Otherwise the FalseBlock might fire after the TrueBlock. I like to use (CALL ) with a trailing space.

Code: Select all

(%if% -9 lss 2147483640) && (
  echo TRUE
  copy someFileThatMightNotExist newLocation
  (call )
) || (
  echo FALSE
)


Here is the macro with test cases

Code: Select all

@echo off
setlocal


:: ----------- Define the IF macro ----------------
set ^"LF=^

^"
set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

set if=for %%# in (1 2) do if %%#==2 (setlocal enableDelayedExpansion^&for /f "tokens=1-3 delims= " %%1 in ("!args!") do (%\n%
  set /a test=0, rtn=1%\n%
  if %%1 lss 0 if %%3 gtr 0 set test=-1%\n%
  if %%1 gtr 0 if %%3 lss 0 set test=1%\n%
  if /i %%2 equ lss (%\n%
    if !test! equ -1 set rtn=0%\n%
    if !test! equ 0 if %%1 lss %%3 set rtn=0%\n%
  ) else if /i %%2 equ leq (%\n%
    if !test! equ -1 set rtn=0%\n%
    if !test! equ 0 if %%1 leq %%3 set rtn=0%\n%
  ) else if /i %%2 equ gtr (%\n%
    if !test! equ 1 set rtn=0%\n%
    if !test! equ 0 if %%1 gtr %%3 set rtn=0%\n%
  ) else if /i %%2 equ geq (%\n%
    if !test! equ 1 set rtn=0%\n%
    if !test! equ 0 if %%1 geq %%3 set rtn=0%\n%
  ) else if /i %%2 equ equ (%\n%
    if %%1 equ %%3 set rtn=0%\n%
  ) else if /i %%2 equ neq (%\n%
    if %%1 neq %%3 set rtn=0%\n%
  )%\n%
  for %%N in (!rtn!) do (%\n%
    endlocal^&endlocal%\n%
    if %%N equ 0 (call ) else (call)%\n%
  )%\n%
)) else setlocal^&set args=

:: ------------ Test the IF macro ------------------
for %%S in (
  "-9 2147483640"
  "-8 2147483640"
  "8 -2147483640"
  "7 -2147483640"
  "9 2147483640"
  "2147483640 8"
  "-9 -2147483640"
  "-8 -2147483640"
  "-9 -9"
) do for /f "tokens=1,2" %%A in ("%%~S") do (
  for %%T in (lss leq gtr geq equ neq) do (
    (%if% %%A %%T %%B) && (echo macro:  %%A %%T %%B TRUE) || (echo macro:  %%A %%T %%B FALSE)
    call :test %%A %%T %%B
    echo(
  )
  echo(
)
exit /b

:test
if %1 %2 %3 (echo normal: %1 %2 %3 TRUE) else (echo normal: %1 %2 %3 FALSE)
exit /b

--OUTPUT--

Code: Select all

macro:  -9 lss 2147483640 TRUE
normal: -9 lss 2147483640 FALSE

macro:  -9 leq 2147483640 TRUE
normal: -9 leq 2147483640 FALSE

macro:  -9 gtr 2147483640 FALSE
normal: -9 gtr 2147483640 TRUE

macro:  -9 geq 2147483640 FALSE
normal: -9 geq 2147483640 TRUE

macro:  -9 equ 2147483640 FALSE
normal: -9 equ 2147483640 FALSE

macro:  -9 neq 2147483640 TRUE
normal: -9 neq 2147483640 TRUE


macro:  -8 lss 2147483640 TRUE
normal: -8 lss 2147483640 TRUE

macro:  -8 leq 2147483640 TRUE
normal: -8 leq 2147483640 TRUE

macro:  -8 gtr 2147483640 FALSE
normal: -8 gtr 2147483640 FALSE

macro:  -8 geq 2147483640 FALSE
normal: -8 geq 2147483640 FALSE

macro:  -8 equ 2147483640 FALSE
normal: -8 equ 2147483640 FALSE

macro:  -8 neq 2147483640 TRUE
normal: -8 neq 2147483640 TRUE


macro:  8 lss -2147483640 FALSE
normal: 8 lss -2147483640 TRUE

macro:  8 leq -2147483640 FALSE
normal: 8 leq -2147483640 TRUE

macro:  8 gtr -2147483640 TRUE
normal: 8 gtr -2147483640 FALSE

macro:  8 geq -2147483640 TRUE
normal: 8 geq -2147483640 FALSE

macro:  8 equ -2147483640 FALSE
normal: 8 equ -2147483640 FALSE

macro:  8 neq -2147483640 TRUE
normal: 8 neq -2147483640 TRUE


macro:  7 lss -2147483640 FALSE
normal: 7 lss -2147483640 FALSE

macro:  7 leq -2147483640 FALSE
normal: 7 leq -2147483640 FALSE

macro:  7 gtr -2147483640 TRUE
normal: 7 gtr -2147483640 TRUE

macro:  7 geq -2147483640 TRUE
normal: 7 geq -2147483640 TRUE

macro:  7 equ -2147483640 FALSE
normal: 7 equ -2147483640 FALSE

macro:  7 neq -2147483640 TRUE
normal: 7 neq -2147483640 TRUE


macro:  9 lss 2147483640 TRUE
normal: 9 lss 2147483640 TRUE

macro:  9 leq 2147483640 TRUE
normal: 9 leq 2147483640 TRUE

macro:  9 gtr 2147483640 FALSE
normal: 9 gtr 2147483640 FALSE

macro:  9 geq 2147483640 FALSE
normal: 9 geq 2147483640 FALSE

macro:  9 equ 2147483640 FALSE
normal: 9 equ 2147483640 FALSE

macro:  9 neq 2147483640 TRUE
normal: 9 neq 2147483640 TRUE


macro:  2147483640 lss 8 FALSE
normal: 2147483640 lss 8 FALSE

macro:  2147483640 leq 8 FALSE
normal: 2147483640 leq 8 FALSE

macro:  2147483640 gtr 8 TRUE
normal: 2147483640 gtr 8 TRUE

macro:  2147483640 geq 8 TRUE
normal: 2147483640 geq 8 TRUE

macro:  2147483640 equ 8 FALSE
normal: 2147483640 equ 8 FALSE

macro:  2147483640 neq 8 TRUE
normal: 2147483640 neq 8 TRUE


macro:  -9 lss -2147483640 FALSE
normal: -9 lss -2147483640 FALSE

macro:  -9 leq -2147483640 FALSE
normal: -9 leq -2147483640 FALSE

macro:  -9 gtr -2147483640 TRUE
normal: -9 gtr -2147483640 TRUE

macro:  -9 geq -2147483640 TRUE
normal: -9 geq -2147483640 TRUE

macro:  -9 equ -2147483640 FALSE
normal: -9 equ -2147483640 FALSE

macro:  -9 neq -2147483640 TRUE
normal: -9 neq -2147483640 TRUE


macro:  -8 lss -2147483640 FALSE
normal: -8 lss -2147483640 FALSE

macro:  -8 leq -2147483640 FALSE
normal: -8 leq -2147483640 FALSE

macro:  -8 gtr -2147483640 TRUE
normal: -8 gtr -2147483640 TRUE

macro:  -8 geq -2147483640 TRUE
normal: -8 geq -2147483640 TRUE

macro:  -8 equ -2147483640 FALSE
normal: -8 equ -2147483640 FALSE

macro:  -8 neq -2147483640 TRUE
normal: -8 neq -2147483640 TRUE


macro:  -9 lss -9 FALSE
normal: -9 lss -9 FALSE

macro:  -9 leq -9 TRUE
normal: -9 leq -9 TRUE

macro:  -9 gtr -9 FALSE
normal: -9 gtr -9 FALSE

macro:  -9 geq -9 TRUE
normal: -9 geq -9 TRUE

macro:  -9 equ -9 TRUE
normal: -9 equ -9 TRUE

macro:  -9 neq -9 FALSE
normal: -9 neq -9 FALSE


Dave Benham

carlos
Expert
Posts: 489
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Rules for how CMD.EXE parses numbers

#38 Post by carlos » 26 May 2014 12:42

This is other bug, on set /a

Hello. As you know in set /a you can do three types of expansion:

normal expansion : %var%
delayed expansion: !var!
name expansion: var

The problem with the name expansion is that if we have a negative number stored as hexadecimal notation in a variable called var, and we use set /a var that expansion will convert that hexadecimal negative number to the max positive number: 2147483647

Check this example:

Code: Select all

@Echo Off
SetLocal EnableExtensions EnableDelayedExpansion

Rem Set a negative number in variable var: -5

Cmd /C Exit /B -5
Echo The hexadecimal representation of -5 is: 0x!=ExitCode!
Rem -5 = 0xFFFFFFFB

Set "var=0xFFFFFFFB"

Echo Look the difference in the name expansion.
Echo It turn the negative number in hexadecimal notation to 2147483647
Set /A "normal_exp=%var%"
Set /A "delayed_exp=!var!"
Set /A "name_exp=var"

Echo Normal Expansion: %normal_exp%
Echo Delayed Expansion: %delayed_exp%
Echo Name Expansion: %name_exp%

Pause
Goto :Eof



It prints:
Normal Expansion: -5
Delayed Expansion: -5
Name Expansion: 2147483647

penpen
Expert
Posts: 1726
Joined: 23 Jun 2013 06:15
Location: Germany

Re: Rules for how CMD.EXE parses numbers

#39 Post by penpen » 26 May 2014 13:13

@dbenham: Nice macro!

Sad to say, that i've found another bug (at least on my win xp home 32 bit):

Code: Select all

Z:\>if 20000000000 lss 0 (echo a) else echo b
b

Z:\>if 0x80000000 lss 0 (echo a) else echo b
b

Z:\>if -2147483648 lss 0 (echo a) else echo b
a

penpen

dbenham
Expert
Posts: 2288
Joined: 12 Feb 2011 21:02
Location: United States (east coast)

Re: Rules for how CMD.EXE parses numbers

#40 Post by dbenham » 26 May 2014 13:47

penpen wrote:Sad to say, that i've found another bug (at least on my win xp home 32 bit):

Code: Select all

Z:\>if 20000000000 lss 0 (echo a) else echo b
b

Z:\>if 0x80000000 lss 0 (echo a) else echo b
b

Z:\>if -2147483648 lss 0 (echo a) else echo b
a


That behavior has already been described by Liviu within this thread at viewtopic.php?p=24799#p24799. I then updated the original post in this thread to reflect the SET /A variable rule as follows:
dbenham wrote:Variables (all versions)

The rules for parsing un-expanded numeric variables are different. All three numeric notations employ a similar strategy: First ignore any leading negative sign and convert the number into an unsigned binary representation. Then apply any leading negative sign by taking the 2's compliment.

The big difference is that overflow conditions no longer result in an error. Instead the maximum magnitude value is used. A positive overflow becomes 2147483647, and a negative overflow becomes -2147483648.

Undefined variables are treated as zero, and variables that do not contain a valid numeric format are treated as zero.


Dave Benham

carlos
Expert
Posts: 489
Joined: 20 Aug 2010 13:57
Location: Chile
Contact:

Re: Rules for how CMD.EXE parses numbers

#41 Post by carlos » 17 Mar 2015 10:22

I found that the %~z expansion of for command treat the size (tested only on windows 7 sp1) as text. For example a big file with size greather than the max 32 bit number 2147483647, like 4700000000 is used without the 32 bit limitation.

Code: Select all

C:\dev>set /a 4700000000
Número no válido. Los números están limitados a 32 bits de precisión.


Code: Select all

C:\dev>fsutil file createnew BIG 4700000000
El archivo C:\dev\BIG está creado

C:\dev>FOR %# IN (BIG) DO @ECHO %~Z#
4700000000


Post Reply