Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

GC again - some detailed questions

Moderators: larsivi kris

Posted: 03/22/10 18:21:13

Sorry for flooding the forum here, but I hope those questions could be useful for other people too. I've done some experiments over the weekend and found that my audio player doesn't quite run well enough (in fact it's a minor catastrophe), and I need some more info on how to work with the GC to minimize its effect. I must admit that my knowledge about GC is really weak and that has to change if I want to solve those problems.

The major problem I have is that after running smoothly for a couple of minutes the player just can't deliver data fast enough for continuous playback. CPU is at normal levels (meandering between ~5-10%) but audio stutters every 4 seconds or so. It looks to me like the GC starts kicking in and halts my render thread every 4 seconds for just too long. Now I'll have to find a way around this, so ...

(1) Is there some detailed description already available somewhere? I have looked through the D site, Wiki4D and this site here but I've found nothing useful, unfortunately. Related to that: what are the real differences between the "official" D GC and Tango? Are there even differences? The best case would be if someone here has a link to some "GC manual" that is applicable to D/Tango. Anyway:

(2) As far as I understand the GC runs in its own thread in the background. Is this true?

(3) How does collection get triggered? By a timer or when some memory gets allocated?

(4) As far as I understand the GC scans every block of allocated memory. What happens if I allocate a buffer via malloc and have a member pointer in a GC-newed object pointing to it? Does that block get scanned?

(5) What if I have a static class pointer pointing to a malloced block? Does that block get scanned?

(6) If this scanning happens could I disable it with removeRoot() or using void * and casting to the D type whenever needed? Related to that: what exactly is a root anyway?

(7) Would this allocate (not initialize) a usable instance of Foo:

class Foo {

// ...

}

Foo *foo = cast(Foo *)malloc(Foo.sizeof);

(8) Would this allocate (not initialize) a usable instance of Bar:

class Foo {

// ...

}

class Bar : Foo {

// ...

}

Bar *bar = cast(Bar *)malloc(Bar.sizeof);

My guess is that (7) works because it wouldn't need a vtbl, but (8) doesn't. Is this correct?

(9) How exactly do enable() and disable() work? Whenever I tried to disable() and later enable() the GC it just stopped working entirely. Is there some timer that gets reset every time enable() is called?

Those are more or less the questions I have. Thanks for reading that far!

-Mike

Author Message

Posted: 03/22/10 19:52:16 -- Modified: 03/22/10 19:52:35 by
kris

1. the only differences (I know of) between the phobos GC and the one in Tango is related to thread-contention ... Tango has fixed some deadlock situations

2. no, the GC runs on the thread that calls for memory allocation. At that time the GC will halt all other threads it knows about ("stop the world")

3. when memory is requested, and a set of criteria indicate a collection should occur first

4. as far as I'm aware, the GC scans only those blocks of memory to which it has a 'root'. If so, the answer would be no

5. same as 4 (above)

6. shouldn't be scanned

7. D has explicit support for custom class allocators, documented on the digitalmars site. Definitely worth looking at

8. see 7 (above)

9. disable() is expected to gate any additional collection, until a matching/balanced enable() is called. The enable/disable ratio must be equal in order to re-enable collections.

The best way to ensure collection doesn't occur is to (a) pre-allocate the memory you need (b) don't allocate anything from the time-critical thread (c) use malloc/free for all allocations.

All that aside, the D GC is somewhat crude. Best to avoid it for real-time functionality. There are a number of games written with D, and they manage to do this quite effectively. See if you can reach h3r3tic (for example) on the irc channels?

Posted: 03/23/10 14:39:11

I have written video (and so audio too) player in D + FFMPEG. I left it playing for hours and I had no problems whatsoever with the GC. I'm using Win32 waveOut functions to write the audio data in the sound card buffer (or wherever it is, I'm not really sure), but then it plays no matter if the GC runs or not. Just play enough audio chunks in advance...

Posted: 03/23/10 16:59:23

@kris: Thanks, that's very helpful!

@bobef: In earlier versions I had only one thread that decodes and renders audio, and it ran rather well too. Recently I've split the functionality into one thread to read, decode and scan files in the background and another one to resample and process the data, and it's pretty obvious now that this is the problem. I actually can't render ahead that far, since I'm building this thing for DJing and live looping (and possibly, far in the future, even music production), so latency needs to be as small as possible. That's why I split it into two threads, so one can fetch data far ahead and the other one can use it when it's needed.

Do you have a site for your player?

Posted: 03/24/10 11:00:01

I don't have a site for the player because it is part of a private project. It is not even exactly mine. I don't know what live looping etc is, I'm sorry my experience wasn't of any help.

Posted: 03/24/10 17:49:42

Live looping is what people like beardyman or dub fx are doing: http://www.youtube.com/watch?v=UiInBOVHpO8

Of course you can use the same looping concepts with "regular" instruments or synthesizers too, so it's possible to create whole tracks from scratch, live, on stage, even if you're a single musician. A friend of mine is doing that and we're planning on teaming up, also I'm experimenting on using looping techniques when I'm DJing. There are hardware solutions (like Korg KAOSS Pad for example, I've got one of those but it doesn't really work for me) or existing software solutions (like the looper in Ableton Live), which are quite nice. The whole thing is in its infancy right now, and it's really where the interesting stuff is happening at the moment.