Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

Ticket #817 (closed enhancement: fixed)

Opened 9 months ago

Last modified 8 months ago

Add a GC.reserve() routine to tell the GC to pre-allocate a certain amount of memory from the OS

Reported by: sean Assigned to: sean
Priority: major Milestone: 0.99.5
Component: Core Functionality Version: 0.99.3 Triller
Keywords: Cc:

Description (Last modified by sean)

Based on this conversation in #D.Tango:

Jarrett: so I realized why the problem just now manifested itself
Jarrett: I made my program more memory-conscious
sean_k: how so?
Jarrett: so before, it allocated a lot of memory in the compiler (this is in MiniD), and that made the pool bigger, or something; now the compiler doesn't allocate nearly as much memory and so it doesn't have to make the pool bigger in the compilation phase, leading to poop performance when running the resulting code
Jarrett: s/poop/poor/ ... well I guess it fits still :P
sean_k: still weird that performance would differ so much
sean_k: how much memory does the app use at its max?
Jarrett: sean_k: the new program takes 4,456K max, the old version took ~5,300K
Jarrett: sean_k: but if I allocate that 4MB chunk in the new program, it'll go up to 8MB and then all the way to ~10MB when running
Jarrett: sean_k: that number for the old program should be 5,536K
sean_k: The way the GC allocates memory from the OS is perhaps a bit weird. It starts with small pools and as more memory is needed it allocates progressively larger pools, with the max pool size being 8 megs. After that it just allocates more 8 meg pools.
sean_k: So if I had to guess I'd say that your early big allocation is setting a sort of "high water mark" and later pool allocations grab a bunch of memory at once instead of little bite-sized pieces.
Jarrett: hmm
Jarrett: interesting :)
sean_k: (Walter originally wrote this code so I'm just relaying what I know about how it works)
sean_k: But it sounds like for an app that slowly increases its demand for memory, the current scheme may be somewhat slow, and maybe there should be some way to tell the GC to "reserve" more memory at the outset?
Jarrett: which makes sense :)
Jarrett: sean_k: that'd be great

Change History

12/11/07 16:29:15 changed by sean

  • status changed from new to assigned.
  • description changed.

12/11/07 16:30:19 changed by sean

  • description changed.

12/11/07 17:43:46 changed by keinfarbton

I also had that problem once. In my case i had a start phase where i read xml files and stored the content in memory. About 700MB.

Simply disabling the GC made the trick. After all files where read, I reenabled the GC.

runtime 20min -> less 1 min.

12/11/07 21:08:24 changed by sean

  • description changed.

12/21/07 09:11:50 changed by larsivi

  • milestone changed from 0.99.4 to 0.99.5.

01/07/08 14:34:54 changed by sean

  • status changed from assigned to closed.
  • resolution set to fixed.

(In [3073]) Added GC.reserve(size_t) as a means of telling the GC to obtain a specific amount of memory from the OS and mark it as free. This is an optimization feature intended to allow programmers to tune the performance of applications which are expected to use a specific or minimum amount of memory. Calling this routine ensures that the specified amount of memory is immediately available and no collections should occur until more than the requested number of bytes are allocated during the program run. This closes #817.

01/07/08 14:49:01 changed by sean

Oh, one thing. I noticed that the GCStats reported when using GC.reserve() seems to say that quite a few pages are marked as B_PAGE compared to the same app not using GC.reserve(). I can't explain this given the GC implementation, but it's something to possibly keep an eye on.

01/08/08 21:05:50 changed by sean

The GCStats thing was an error in the output code of my test app. All is well in Tango-land.

Roadmap Timeline Tickets New Ticket Help / Guide About Trac
Login  |  
Trac dsource.org
RSS
Copyright © 2006-2008 Tango. All Rights Reserved. | Page Width: Static or Dynamic