Page 1 of 1

A Critique on toDec function and its workaround

Posted: 05 Apr 2011 05:10
by ghostmachine4
Source: :toDec


Limitation: This function works, but does not cater for big hex numbers, eg (tested on WinXP)

Code: Select all

C:\work>test.bat F
15
C:\work>test.bat FFFFFFFF
Invalid number.  Numbers are limited to 32-bits of precision.


Here's an implementation in vbscript for big hex numbers. Note: vbscript comes installed by default on most Windows distribution. There is no reason one could not get to know it and harness its capabilities (same with powershell).

Code: Select all

Function Decimal(HexNumber)
   For x = 1 To Len(HexNumber)
      ch = CLng("&h" & Mid(HexNumber, x, 1)  )
      s = 16 ^ (Len(HexNumber) - x)
      accumulate = accumulate + (ch * s)
   Next
   Decimal = accumulate
End Function
WScript.Echo Decimal(WScript.Arguments(0) )


Code: Select all

C:\work>test.bat FFFFFFFF
Invalid number.  Numbers are limited to 32-bits of precision.

C:\work>cscript //nologo test.vbs FFFFFFFF
4294967295



Note also, to convert Hex to Decimal with vbscript, just this simple statement does the trick

Code: Select all

CLng("&h" & "F" )


however, it suffers from the same limitation as the batch :toDec

Code: Select all

C:\work>head -10 test.vbs
WScript.Echo  CLng("&h" & WScript.Arguments(0) )
WScript.Quit

C:\work>cscript //nologo test.vbs FFFFFFFF
-1


A suggestion for a workaround for big hex numbers similar to the vbscript version is advised for :toDec for which I am sure it could be done (right?)

DosItNotHelp