The Limited length of CMD variables?

[Kvc]

I was working on one of my projects and I encountered a problem that - the CMD variable can only retain 8KBs of length (8000 approx. chars). And, if you try to add more chars then the'll just get terminated.

Is there any solution for it?

Can we save more chars in one variable in CMD? Is it possible?

[Aacini]

No.

The maximum size of a user-defined environment variable is 32,767 characters. There is no technical limitation on the size of the environment block. However, there are practical limits depending on the mechanism used to access the block. For example, a batch file cannot set a variable that is longer than the maximum command line length.

Antonio

[aGerman]

The maximum string length in batch is 8191 characters.

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set /a "zeros=0, ones=1"
for /l %%i in (1 1 12) do set "zeros=!zeros!!zeros!"
for /l %%i in (1 1 12) do set "ones=!ones!!ones!"
set a=!zeros!!ones:~-4093!
echo(!a!
pause

4096+4093+2(for "a=") = 8191

https://support.microsoft.com/en-us/help/830473/command-prompt-cmd--exe-command-line-string-limitation

Steffen

[Kvc]

Hi Aacini & aGerman,

The maximum string length in batch is 8191 characters.

Just verified This statement... And, Confirmed it using the Getlen function... The valid length of the String saved in a variable is 8177 characters. (see title bar)



And, i changed your code little... for verification purposes. (see numbers & Getlen function call)

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set /a "zeros=0, ones=1"
for /l %%i in (1 1 12) do set "zeros=!zeros!!zeros!"
for /l %%i in (1 1 12) do set "ones=!ones!!ones!"
set aaa=!zeros!!ones:~-4081!
echo(!aaa!
Call Getlen "%aaa%"
Title %errorlevel%
pause

And, even on increasing the var_name's length - the max. chars saved in a var stringare still of 8177 chars. (try changing the '4081' to '4082' - you'll get max. len error) But, with the above 4081.. you won't.

And, the max. length of var stored and displayed without error is 8177.
here's the project zip.

Var-length-test.zip

(1.38 KiB) Downloaded 685 times

@Aacini - I didn't understood the 32 KBs block concept. :/ Can you elaborate?
And, Can I increase this limit by any means / hacks or extraaCMD extension - dos9, otherCMD alternatives etc.? Any method willbe appreciated.

[aGerman]

The valid length of the String saved in a variable is 8177 characters.

Not quite.
As I wrote - the maximum string length is 8191 characters (for a confirmation see the Microsoft link I posted). But string length doesnt mean that you can save 8191 characters in a variable because also the assignment belongs to the string. In my example you can save 8189 characters because the remaining 2 characters are needed for "a=". Thus, the maximum content of your variable is getting shorter for a longer variable name. And because your call of Getlen results in longer strings you can't even use the length that is theoretically possibel.

I didn't understood the 32 KBs block concept.

There is nothing to understand because that's set by Microsoft and thus, a limitation on Windows. And as tests show - the more content you have in the environment (and all Batch variable are environment variables) the slower becomes the execution of your script.

Typical example where Batch is doomed to failure: Trying to process markup like XML or HTML.
- it is most likely saved in a charset other than the OEM code page that batch uses
- the line length isn't limited; theoretically you can save the whole HTML or XML content in only one line

Another example: Trying to save the whole content of a file in memory.
Often people try to to read the content of a file linewise and save it in umpteen variables of an assoziative array (line_1, line_2, ...). Whenever possible, only save one line and directly process this line! It doesn't make sense to iterate over the file content to save an array only to iterate over the array afterwards.


Steffen

[penpen]

And, i changed your code little... for verification purposes. (see numbers & Getlen function call)

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set /a "zeros=0, ones=1"
for /l %%i in (1 1 12) do set "zeros=!zeros!!zeros!"
for /l %%i in (1 1 12) do set "ones=!ones!!ones!"
set aaa=!zeros!!ones:~-4081!
echo(!aaa!
Call Getlen "%aaa%"
Title %errorlevel%
pause

And, even on increasing the var_name's length - the max. chars saved in a var stringare still of 8177 chars. (try changing the '4081' to '4082' - you'll get max. len error) But, with the above 4081.. you won't.

The issue in that case is not the variable length itself; the limiting bottleneck in your case is this line:

Code: Select all

Call Getlen "%aaa%"

The length of 'Call Getlen "' plus '"' is 14 characters, so in this case %aaa% is limited to 8177 (=8191-14).

You could easily see that aGerman is right by using the substring functionality of environment variables;
but you could retrieve the string without any line length dependencies using set and reading the file length:

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set /a "zeros=0, ones=1"
for /l %%i in (1 1 12) do set "zeros=!zeros!!zeros!"
for /l %%i in (1 1 12) do set "ones=!ones!!ones!"
set _=!zeros!!ones:~-4093!
echo(!_:~8188!

cmd /d /a /v:on /c">"getLength.txt" set _"
for %%a in ("getLength.txt") do set /a "_.length=%%~za"
set _.length

goto :eof

Notes:
The character at index 8188 is the 8189th character.
the output of set contains the complet environment variable inclusing '_='.
If you want to handle max sized environment variables, you should use delayed Expansion and pass variables by reference (== its name in this case).


penpen

Edit: Added the third note.

[Kvc]

That's a lot of theory to grasp at once. But, Thanks @penpen & @aGerman for explaining it nicely. And, it feels like - There's no trick By which we can minimize this limitation of the Batch. Thanks for all your time & inputs.

[aGerman]

To find a possible workaround we would have to know the actual problem behind your question.

Steffen