portable selector for both pipes and sockets?

schveiguy Joined: 08/02/07 Posts: 128

Posted: 10/22/07 18:38:30 I know this is not your fault because Microsoft engineers decided to implement sockets differently than their other resource handles, but I am faced with this problem nonetheless. My problem is that I have a process running smoothly on linux that is single threaded and listens for new data on both a socket and multiple pipes from child processes. This works very well as the process only wakes up when there is data available on any of those sources. Now, I need to port this code to Windows. I haven't actually tried it yet, but reading the docs from MS and from past experience, I believe the tango.io.selector.* will not work on Microsoft Pipes, including ones from child processes. The reason is because I think the only usable !ISelector is the SelectSelector, which uses select to wait for events. According to the documentation for select, it is only usable on sockets :( So I think that although the code will compile, I think I will get an exception when I try adding the pipe to the selector. (going to start porting this today, so I'll see if it complains or not. Maybe there is an undocumented feature of select :) Is there any thoughts on how to solve this problem other than resorting to multi-threading? I hate to have separate designs on windows and Linux, as I want to only maintain one code base. Simple differences that can be handled by version(Windows) are ok, but single vs. multi threaded is a drastic difference. -Steve

Author	Message
schveiguy Joined: 08/02/07 Posts: 128	Posted: 10/22/07 21:08:25 Update: yeah, I get WSAENOTSOCK when calling select :( Also, BTW, SelectSelector? does not handle this gracefully. It actually simply returns -1 instead of throwing an exeception. -Steve
kris Joined: 03/27/04 Posts: 581	Posted: 10/22/07 21:32:00 Yeah, that's a pain. Best person to speak to about this is the author, Juan. He goes by the name of juanjoc on IRC ...
jcomellas Joined: 01/30/06 Posts: 3	Posted: 10/24/07 14:10:17 You're right, the SelectSelector will not work with handles that are not sockets. A way to solve this problem without heavily refactoring the selectors is to create another Selector that internally uses the WaitForMultipleObjects() Windows API. The problem with this API is that it only allows you to wait for events on upto 64 "objects" (sockets, pipes, file handles, mutexes, semaphores, etc.). It wouldn't be very hard to create a new selector with this API based on PollSelector.
schveiguy Joined: 08/02/07 Posts: 128	Posted: 10/24/07 16:35:16 Just an update for folks who are following this. I was not able to get WaitForMultipleObjects? to work on the pipe between the parent and child process. This means I will probably have to switch to a multi-threaded design. On windows, I could use Overlapped I/O, but then I am in the same situation where it doesn't port well to linux. Anyone else have any ideas?
schveiguy Joined: 08/02/07 Posts: 128	Posted: 10/25/07 20:17:02 More updates: I could not use overlapped I/O and threads since Windows does not support overlapped I/O on anonymous pipes. Conclusion: using pipes for child processes sucks on windows. So here is what I did. Prepare to cringe :) For each pipe that I wanted to read, I connected a tcp socket from the process to itself. Then I started a thread to simply read the pipe and output the results to the write end of the socket. Since it's now a socket, my main thread can simply use the SelectSelector? the same way as it was on linux. The only really inefficient part is that because I cannot use overlapped I/O on the pipe threads, I have to use PeekNamedPipe? to look and see if there is any data. If so, read it and send it along, if not, sleep for 2 seconds. What a pain. -Steve
kris Joined: 03/27/04 Posts: 581	Posted: 10/26/07 00:40:10 :D