Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

File io memory leak?

Moderators: kris

Posted: 10/11/09 18:18:15

Ok, I was just messing around with an app i'm developing for a friend. Today i decided to randomly see how much memory it was using in ram, so i hit ctrl + alt + delete and it was far higher than I expected (~16,000kb). I proceeded to comment out anything but the base code (using Derelict SDL and Opengl). After doing so I gave it a run and got 11,000kb, much more reasonable. I uncommented some lines and came down to when i load in files. I solved some of it by putting in a delete command i forgot to, but the rest is not anywhere in my code.

This causes no increase in memory (or none large enough to register 1kb):

auto file = new File (file_name); //file_name defined elsewhere
if(file is null)
	return null;
//some commented out code
delete file;

This code increases my memory usage by ~900kb:

auto file = new File (file_name); //file_name defined elsewhere
if(file is null)
	return null;
ubyte[] fbuffer = cast(ubyte[]);
//some commented out code;
delete fbuffer;
delete file;

I'm using the old still, but rooted around and found out it just links to the new File code. Further rooting around didn't really result in much. I found the detach function for devices which the new File type inherits form, but it doesn't free any memory and niether does the read function of the new File type.

I'm loading in 10 files total at this point: ~65kb, 15kb, 2kb, 1kb, 2kb, 2kb, 3kb, 36kb, 1,025kb, 257kb

The 900kb memory increase happens when i reload the files that are: 65kb, 257kb, and 1,025kb

Not all of the 900kb is coming from the file reloading, i'm also changing the window size for sdl when i do it and sdl has a leak of about 200kb or something when i was looking but don't remember.

Can anyone confirm this?

Author Message

Posted: 10/11/09 21:02:00

Two things: one, 'new' never returns null in D, so there's no need to check it. If you do run out of memory, an exception will be thrown instead.

Two, Tango's GC does not return memory to the OS, even if you use 'delete'. 'delete' just tells the GC "okay, you can reuse this memory!" but that memory is kept around because chances are, you're gonna be using it again. If it returned the memory to the OS (reducing the virtual memory size), it'd just have to waste a bunch of time re-querying the OS for heap later.

The end result is that the memory footprint of D programs is slightly higher than what you might expect for a similar C or C++ program, but in reality, it's not that bad. Just don't be anal about it and you probably won't run into any issues. :)

I don't know why I'd make a new scripting language. I mean, I might as well just draw some lines in the sand with a stick.

Posted: 10/11/09 21:58:51

That if statement is left from converting the file it's in from c. Just never got around to removing it b/c I haven't had to deal with that file in some time.

That doesn't explain why my mem usage continues to grow. I don't like memory leaks in any form and don't really care about how long the OS may take to give me some memory, the GC has to return some at some point.

Posted: 10/12/09 04:11:23

I'd like to see a gc-option whereby heap usage could be compacted/released

Posted: 10/12/09 10:39:08

Automatically, you mean? Yeah, that'd be nice.

In the meantime, wolf, you can import tango.core.Memory and try GC.minimize(). It should release memory back to the OS.

I don't know why I'd make a new scripting language. I mean, I might as well just draw some lines in the sand with a stick.

Posted: 10/12/09 22:16:01

Thanks for the tips, but i finally found that the error is with Derelict and not with tangos file classes. I tried the GC stuff, but it didn't change anything, even in a test app that simply loaded in the same 10 files and reloaded them in a loop until i said stop. The memory started at 1,6k and then grew to 3,016k (about right when you add up the file sizes) and wouldn't go down any with GC commands (used GC collect and then minimize).

The issue is when you load a texture into opengl using derelict the memory is registered as being used by your ram even though opengl should be putting them on the vcard ram (i'm unfamiliar if the task manager keeps track of it like that though) and whenever I reload a texture it fails to free up the old texture data and it for some reason expands my mem usage.

Also, Derelict SDL has a memory leak somewhere, b/c everytime you resize the window while using opengl with it, it adds ~150 - ~200k to mem usage. But i'll take these bugs over to them.

Thanks for the help.