The 8+8=16k double memory combo.

[Ed Dyreen]

[edit]
This is the result from playing with 'jeb's append' and 'tooComplex's in the middle' technique to pass arguments.
I combine both and discover it harnesses jeb's simplification of passing args and tooComplex's splitting over multi vars.

Code: Select all

@echo off &setlocal disableDelayedExpansion

set $lf=^


::two line
set ^"$n1c=^^^%$lf%%$lf%^%$lf%%$lf%^^"

rem The 16k memory macro !

set ^"macro1=setlocal enableDelayedExpansion ^&for %%? in ( 1,2,3,4 ) do echo.%%? ^&if %%? == 2 ( %$n1c%

   echo.This is macro1 8k of mem. %$n1c%
   echo.we could setlocal here %$n1c%
   echo.$arg=!$arg!_ %$n1c%

) else if %%? == 4 ( %$n1c%

   echo.This is macro1 8k of mem. %$n1c%
   echo.we could endlocal here %$n1c%
   echo.$arg=!$arg!_ %$n1c%

) else ( set $arg="

set "macro2=) &if %%? == 3 echo.This is macro2 8k of mem. &echo.$arg=!$arg!_"

( %macro1% "This works, as you can see" %macro2% )

pause
exit

Code: Select all

1
2
This is macro1 8k of mem.
we could setlocal here
$arg= "This works, as you can see" _
3
This is macro2 8k of mem.
$arg= "This works, as you can see" _
4
This is macro1 8k of mem.
we could endlocal here
$arg= "This works, as you can see" _
Druk op een toets om door te gaan. . .

Macro1 8k macro header ( a reusable macro ), can deal with tokenizing and push over endlocal,
Macro2 8k macro specification don't have to deal with that.

Because this combo cannot be nested and requires to be called in an uncomfortable manner from user perspective,
I only use this as a last resort.

[dbenham]

Brilliant Ed
I think that should work.

Since each command line is limited to 8191 chars, I always assumed that a complex block within () was also limited to 8191. But I'm thrilled that I was wrong. I successfully executed a 128k block of code within a batch.

I don't see the need for the enclosing () at call time, but I haven't tested.

I'm thinking you can rearrange things to get a more natural looking syntax at call time. Something like:

Code: Select all

%CallMacro% %MacroName% args

CallMacro would set up the outer loop with 3 phases. It would also define the first and last phases. Phase 1 would setlocal and parse the args into individual variables named arg1, arg2, ...argN. Phase 3 would return the results over the endlocal. CallMacro would end with the conditional execution of an undefined block of code like so

Code: Select all

...
else if %%?==2 

Note there is a space after the 2.

MacroName would be responsible for the bulk of the processing. It would have to prepare the results in some standard way that phase 3 understands. MacroName would be within a (block), followed by the argV assignment that jeb invented.

Your strategy should allow nearly the full 8191 bytes for the primary logic.

But I don't think it will work well if you try to call a macro that uses this strategy from within the definition of another macro.


Dave Benham

[Ed Dyreen]

'
Thanks,

I'm thinking you can rearrange things to get a more natural looking syntax at call time.

Probably, if I find time to look

But I don't think it will work well if you try to call a macro that uses this strategy from within the definition of another macro.

Only applies to macro that won't be nested in other macro and then even only if no other option to make it fit,
I have several candidates that may no longer require 'call :macroEval'.

[orange_batch]

So, are you guys anywhere near developing a solid modular macro library? I've been avoiding this insanity like the plague, but I'm interested in the results lol.

By modular I mean, being able to pick and choose which functions to add to a script, rather than loading an entire library.

[jeb]

( %macro1% "This works, as you can see" %macro2% )

Brilliant Ed
I think that should work.

No, this way the 8k++ test only works with data less than 8k

@Ed test it with more than 8k and it will fail with:
Line is too long

The idea of Ed is good, but has some limitations.

A block of code can contain much more than 8k (I can't remember if I ever found a limit, but it must be huge).
BUT each line of the block still has a limit of 8k.

The 8k limit is efective after the percent expansion phase, it counts all characters and linefeeds haven't an effect to the coiunting here.

A 16k limit is more or less active before the percent expansion occurs.
What I mean is, you can build a line with nearly unlimited length,
it's only important that the line will be reduced to less than 8k bytes after the percent expansion

Like

Code: Select all

set "Q="
echo %Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q%%Q% ... 
set /a val=1+%Q%%Q%%Q%[... 2000 times %Q%]^
1+%Q%%Q%%Q%[... 2000 times %Q%]^
1+%Q%%Q%%Q%[... 2000 times %Q%]^
1+%Q%%Q%%Q%[... 2000 times %Q%]^
1+%Q%%Q%%Q%[... 2000 times %Q%]^
1+%Q%%Q%%Q%[... 2000 times %Q%]^

The second sample with set/a can be as long as you want, I've tested it with 240k length and it still calculates the correct value.

But each line itself seems to have a weak character each 8192 characters (before the expansion).
It's only a weak limit, as only a percent character at position n*8192 will be disabled.
It can't expand a variable, but it is still there, a %% will still produce a single percent,
other characters at this positions seems not to be affected.

That's the cause why expample of ED can't work this way, it can changed to this

Code: Select all

( 
%macro1% "This works, as you can see, even with more than 8k total" 
%macro2% 
)

hope it helps

jeb

[Ed Dyreen]

other characters at this positions seems not to be affected.

Like playing a casino, knowing it will work, but not always ? Lovely !
Thanks jeb, I didn't test any of this

You say every n*8192 char would fail to expand, and only if it's a percent char.
Then another solution is padding, where we fill the 1st macro so we'll know exactly where this failing char is situated.

Code: Select all

%macro1_paddedTo8k%< n is a space >%macro2%

As long as macro2 inclusive args is below 8k 'the next failure' we should be safe or ?

[jeb]

Then another solution is padding, where we fill the 1st macro so we'll know exactly where this failing char is situated.
Code:
%macro1_paddedTo8k%< n is a space >%macro2%
As long as macro2 inclusive args is below 8k 'the next failure' we should be safe or ?

For macro's this behaviour isn't important, as this effect can only occur if a line is longer than 8k before the percent expansion evaluates, after the percent phase the line have to be always less than 8k.

jeb

[jeb]

I've tested a bit more with the "weak" character at n*8192.
I hoped to find a way to replace a percent with a percent expansion, but I found the following.

It seems that not the character at the position itself is "weak", even a variable expansion fails if the first percent is at position 8190 and the last at 8195.

The position seems to stop any expansion, my guess is, that the line is read in chunks of 8192 bytes, and the the expansion only works in a chunk but can not cross the boundary.

jeb