Page 1 of 1

A Critique on toHex function and its workaround

Posted: 05 Apr 2011 07:31
by ghostmachine4
Source: :toHex


Limitation: This function works, but does not cater for big decimal numbers (limitation with DOS), eg (tested on WinXP)

Code: Select all

C:\work>test.bat 16
00000010

C:\work>test.bat 9999999991
Invalid number.  Numbers are limited to 32-bits of precision.
00000000



Here's an implementation in vbscript for big decimal numbers. Note: vbscript comes installed by default on most Windows distribution. There is no reason one could not get to know it and harness its capabilities (same with powershell).

Code: Select all

Function DecToHex(DecimalNumber)  
  strResult = ""
  intValue = DecimalNumber
  If intValue <= 15 Then
     strResult = Hex(intValue)     
  Else
     While intValue > 0
        m = InStr( CStr(intValue / 16),"." )
        num = Split( CStr(intValue / 16) ,".")        
        If m > 0 Then              
          strResult  = Hex(CDbl( "0." & num(1)) * 16)  & strResult          
       Else          
          strResult = "0" & strResult          
       End If
       intValue = num(0)
     Wend
  End If
  DecToHex = strResult
End Function

WScript.Echo  DecToHex(WScript.Arguments(0) )


Code: Select all

C:\work>cscript //nologo test.vbs 16
10

C:\work>cscript //nologo test.vbs 9999999991
2540BE3F7



This works by converting the number to hex in String representation: source

Note also, to convert Decimal to Hex with vbscript, just this simple statement does the trick

Code: Select all

Hex(10)


however, it suffers from the same limitation as the batch :toHex

Code: Select all

C:\work>head -5 test.vbs
WScript.Echo  Hex(WScript.Arguments(0) )
WScript.Quit

C:\work>cscript //nologo test.vbs 9999999991
C:\work\test.vbs(2, 1) Microsoft VBScript runtime error: Overflow: '[string: "9999999991"]'


A suggestion for a workaround for big decimal numbers similar to the vbscript version is advised for :toHex

DosItNotHelp