Search found 26 matches

by vin97
03 Aug 2020 09:35
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Ok, going to start rewriting my program then.
Learning a lot here, thanks again!
by vin97
02 Aug 2020 07:22
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Ok, I will test that. How does CMD work out piping internally or more specifically, what counts as a "critical pipe" capable of causing memory issues? Is it just piping into a command (with "|") or does it apply to "redirecting" in general? For example, could I avoid the problems by writing to tempo...
by vin97
01 Aug 2020 13:21
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Just Windows defender.
by vin97
01 Aug 2020 11:58
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Ok.

Is it only the nonpaged pool that can cause a system crash or does this also happen when memory is low in general? In other words, is it enough to only monitor the nonpaged pool in the batch file?

What kind of memory % do you suggest for stopping the program?
by vin97
01 Aug 2020 06:06
Forum: DOS Batch Forum
Topic: JREPL: Cutting of a string after the n-th occurrence of specific character?
Replies: 17
Views: 1710

Re: JREPL: Cutting of a string after the n-th occurrence of specific character?

Yes, that's the one I am using.
More specifically: dir /s /a /n /tw /o-d /4 /-c
by vin97
01 Aug 2020 06:04
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Thanks for the tips!

I think the leak is coming from some driver. Deactivating the antivirus gives the same result.
So there is really no way to clear the nonpaged pool manually?
by vin97
31 Jul 2020 05:46
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

That seems to be it.
Problem now is, there doesn't seem to be a way to clear the NonPaged Pool without rebooting.
Maybe I'm missing something, though?
by vin97
31 Jul 2020 05:45
Forum: DOS Batch Forum
Topic: JREPL: Cutting of a string after the n-th occurrence of specific character?
Replies: 17
Views: 1710

Re: JREPL: Cutting of a string after the n-th occurrence of specific character?

I'm sorry, I thought I made it clear above but using DIR /B is not an option, since the ultimate goal is to scan for directories modified after a certain date.
by vin97
30 Jul 2020 04:49
Forum: DOS Batch Forum
Topic: JREPL: Cutting of a string after the n-th occurrence of specific character?
Replies: 17
Views: 1710

Re: JREPL: Cutting of a string after the n-th occurrence of specific character?

Thanks for the example but file.txt would look a little different.

Since the textfile is produced by the DIR command, each line has some irrelevant string preceding the directory path that needs to be removed.
by vin97
30 Jul 2020 04:30
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Thanks for those snippets but the FOR loop was just an example to show that even this simple version is creating memory issues. In the actual program, there is a lot of other stuff that needs to happen inside the loop. Nobody got an idea where those leaks are coming from in the first place or how th...
by vin97
29 Jul 2020 08:57
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Re: Memory leak when reading large text files

Strange because memory usage keeps steadily rising at a similar pace while the loop is running and stops as soon as the program is done.
It takes a few minutes to scan the text file and builds 2+GB memory usage in the process.
by vin97
29 Jul 2020 06:16
Forum: DOS Batch Forum
Topic: Memory leak when reading large text files
Replies: 18
Views: 1431

Memory leak when reading large text files

I get a bad buffer overflow when reading large text files (3+ MB) through a FOR loop. Looking at the task manager, the memory usage keeps rising and rising up the point of system crash. The strange thing is that no single process ever shows up with more than 50MB in the process list at any time. Als...
by vin97
29 Jul 2020 06:04
Forum: DOS Batch Forum
Topic: JREPL: Cutting of a string after the n-th occurrence of specific character?
Replies: 17
Views: 1710

Re: JREPL: Cutting of a string after the n-th occurrence of specific character?

This recursion depth check is only one part of the program.
The more important one is finding out which directories were modified after a certain date for doing incremental backups.


How would you modify your code for variable recursion depth?
by vin97
28 Jul 2020 15:55
Forum: DOS Batch Forum
Topic: JREPL: Cutting of a string after the n-th occurrence of specific character?
Replies: 17
Views: 1710

Re: JREPL: Cutting of a string after the n-th occurrence of specific character?

I know there are still a lot of bad things in my algorithm but I want to try improving it first myself.
I was purely wondering about the speed of JREPL in general and how optimized it is.
by vin97
28 Jul 2020 15:53
Forum: DOS Batch Forum
Topic: FOR /F Text Reading: Retain Empty Lines & Dynamic Skip?
Replies: 2
Views: 652

Re: FOR /F Text Reading: Retain Empty Lines & Dynamic Skip?

1 Would still have to read the line, no? Ideally I would like to avoid even having to read a flag at the beginning of each line.
2 Doing this trick in a for loop with the type command is very slow for me, so I guess I will stick to filling empty lines beforehand.