GC and system callbacks

vertex Joined: 11/24/07 Posts: 8

Posted: 03/18/10 23:23:40 Hi! I'm hunting one of those nice heisenbugs in my audio player ... and I'm out of ideas. So I thought I'll ask here if someone can give me some tips or new ideas as to what I could do. So: I have the GC and I have an audio callback function (ASIO/Windows in this case). Sooner or later I get an access violation when those two things come in contact. Theoretically this shouldn't happen. My callback function doesn't touch memory handling at all, it neither creates nor destroys memory. But as soon as another thread in the background is busy the access violations occur. If I substitute my ASIO code with a dummy (a class based on a Windows timer that behaves like a real audio interface) it works flawlessly, even with GC. If I disable the GC it works flawlessly too. If I just have the callback running and disable the rest of the program (the background threads which decode audio files) it works too. But as soon as I open an ASIO driver, have GC enabled and have some audio data pushed through the system it crashes after some time (the higher the system load the more likely it crashes, so it crashes most likely when the program is still initializing or opening new files for example). So I'm guessing it's the GC that kicks in in one thread, while the system is busy in the audio callback. I have no idea why that can crash, but it's my best guess. I have no idea how the GC actually works, so that's basically why I'm asking here. If one of you who has some insight into the inner workings with respect to callbacks and multithreading it would be great if you could help me out here. It's really very frustrating now; I'm hunting this bug for over two days now and I can't think of anything ... so ... any help would be appreciated. Regards, Mike

Author	Message
kris Joined: 03/27/04 Posts: 581	Posted: 03/18/10 23:51:36 there's a function in Thread.d which wraps an external thread and treats it (from that point on) just like a native D thread. Not sure if that would help in this case or not, but thought I'd mention it
vertex Joined: 11/24/07 Posts: 8	Posted: 03/19/10 00:11:08 -- Modified: 03/19/10 01:10:38 by vertex -- Modified 2 Times Hmm. I don't think I can't assume that it's always the same thread calling the callback. One question: Could it be that the following scenario is what happens? 1. GC suspends all threads to begin a collect 2. While GC is collecting the callback happens 3. Callback does its work (which includes moving buffer references from the "ready" queue to the "reuse" queue 4. GC isn't aware that callback changed some references and gets confused 5. Crash This is my best guess at the moment. Is that a possible problem that step #3 up there could really confuse the GC by shuffling references around? EDIT: I just hacked in some quick workaround and that's indeed what happens. Moral of the story: you *must not* change references in a system callback in a garbage collected language!
kris Joined: 03/27/04 Posts: 581	Posted: 03/19/10 03:13:51 -- Modified: 03/19/10 03:16:26 by kris Yes, that would do it :) The D GC is a "stop-the-world" collector, which is less than ideal for all kinds of things. This is why it kinda' needs to know about all threads so it can manage them appropriately. Shouldn't be an issue as long a thread doesn't touch any D-managed memory. One way around all this is to (sadly) just use malloc() for everything
vertex Joined: 11/24/07 Posts: 8	Posted: 03/19/10 16:46:31 I think I've got it now how to work with the GC in this situation. I've played around with it today and I'm really satisfied with how stable it is now. All there is to do now is to work through the remaining bugs which are all fairly conventional, put in Wiimote support (I really want this in) and I'm good to release the first beta in a couple of weeks :) What I've learned about GC through this project: I think above a certain complexity you can't treat the GC as "fire and forget" anymore. It works great for small portions of memory (like strings) where you can really just ignore memory management, but when you need to move lots of data you need to have a good memory management strategy in addition to GC. Thanks for the malloc tip, that's good to know!