| View previous topic :: View next topic |
| Author |
Message |
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
 Posted: Tue Feb 14, 2006 5:00 pm Post subject: |
 |
|
| sean wrote: | | By the way, do you plan on supporting the proactor model as well? I ask because I typically use IOCP on Windows, regardless of what I use elsewhere. Also, it's probably worth noting that while the proactor and reactor concepts are indeed somewhat opposed, the only notable difference in how they are handled at the code level is call order |
I imagine the Proactor version would have to operate at the Buffer level rather than at the Conduit level?
(Buffer has an API to retrieve an optional Conduit, but not the other way around) |
|
| Back to top |
|
 |
jcomellas
Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina
|
| Posted: Wed Feb 15, 2006 8:50 am Post subject: |
|
|
| kris wrote: | I imagine the Proactor version would have to operate at the Buffer level rather than at the Conduit level?
(Buffer has an API to retrieve an optional Conduit, but not the other way around) |
It's going to be tricky to have the current Buffer model work with IOCP's, because currently the Mango Buffer class assumes that it can read data from the associated Conduit whenever it needs it. That's not the case here. When calling read() using overlapped I/O the method returns without having finished the operation.
When dealing with IOCP's, the thread in which the read operation starts and the thread where it is completed are not the same. Some thought will have to go into thinking how to handle this with what we have right now. |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Wed Feb 15, 2006 10:46 am Post subject: |
|
|
| jcomellas wrote: | | When dealing with IOCP's, the thread in which the read operation starts and the thread where it is completed are not the same ... |
The Reactor model seems easier to use in this respect (from a developer-standpoint), yet it could also be wrapped with a multithreaded shell as necessary. That is; generic usage can be very simple to use yet, when targeting a Niagara or equivalent, I can explicitly scale the behaviour via multiple threads. That may represent the best of both worlds? |
|
| Back to top |
|
|
sean
Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA
|
| Posted: Wed Feb 15, 2006 11:07 am Post subject: |
|
|
Aye. The only snag is that the truly high-end IO models on Windows use the Proactor model: IOCP and overlapped with callbacks (I've never actually used the second, but there was some talk in the Boost list a few years back that it doesn't have a limit on number of events either). The 64 event limit on reactors puts a huge damper on things IMO, as that really amounts to only 32 read/write sockets per event handler. And scaling beyond that means either spinning on multiple handlers or monitoring each in a separate thread. For anything beyond small scale Windows apps, the Proactor model is actually far easier to use.
I'll think about this a bit today and see if I can come up with something that seems workable, as I'm fairly motivated to use IOCP  |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Wed Feb 15, 2006 1:28 pm Post subject: |
|
|
| sean wrote: | | I'll think about this a bit today and see if I can come up with something that seems workable, as I'm fairly motivated to use IOCP |
Off the top of my head I can think of three ways to handle the Proactor approach:
(a) treat Buffer just like a typical array, such that a read-op past end-of-current-data results in failure rather than a reload of additional content. The developer would pass a Buffer/delegate combo to the IOCP handler instead of a void[]/delegate combo. Buffer supports this type of behaviour by removing the associated conduit (making it look just like a memory-only buffer, or memory-mapped buffer).
(b) subclass Buffer to instead support a synch/rendezvous model when the current content has been consumed. As soon as content arrives, it is concurrently appended to the Buffer (up to the available remaining space), and another concurrent read is immediately started. This takes advantage of the Buffer ability to extend the 'readable' content (there's both a read position and a write position). Of course, this approach would have to be synchronized to allow concurrent read/writes; but it's a possibility.
(c) apply a double (or triple) buffering scheme. As soon as one internal buffer is used, swap in a populated one and initiate an asynchronous read on an unused internal buffer. This would also require the synch/rendezvous approach noted in (b), but limits the synchronization constraints of producer & consumer to a single point where the current internal buffer is empty. The approach should work for asynchronous writing also, which is a somewhat jolly bonus  |
|
| Back to top |
|
|
sean
Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA
|
| Posted: Wed Feb 15, 2006 4:09 pm Post subject: |
|
|
I like method (c), but what if the user tries to read more data than is available? Does the call block or is Eof returned? And what if data is being pushed at the user like crazy but he doesn't perform any reads at all? Should there be an upper limit set on buffer size? And what should happen when this limit is reached?
I'd much rather simply have an event fired when data arrives, and figure out what to do with it. However, I think too much importance may be being attributed to the differences here. What if both the Proactor and Reactor models used Buffers instead of Conduits? Then the differences could be handled before the event handler is called:
| Code: | InternetAddress addr = new InternetAddress("127.0.0.1", 80);
SocketConduit socket;
try
{
auto ISelector selector = new IOCPSelector();
int timeout = 5000; // 5 seconds
int eventCount;
selector.open();
socket.connect(addr);
selector.register(socket, ISelectionSet.Read | ISelectionSet.Write);
eventCount = selector.wait(timeout);
if (eventCount > 0)
{
ISelectionSet selectedSet = selector.selectedSet();
foreach (Buffer buf, uint eventMask; selectedSet)
{
if (selectedSet.isRead(eventMask))
{
buf.fill( buf.getConduit() );
writefln("Received a Read event from conduit");
}
if (selectedSet.isWrite(eventMask))
{
buf.drain();
writefln("Received a Write event from conduit");
}
if (selectedSet.isError(eventMask))
writefln("Received an Error event from conduit");
}
}
socket.close();
selector.close();
}
catch (Exception e)
{
writefln("Exception caught:\n\n?s", e.toString());
} |
The important bit is that the selector Buffer would load available data on a fill call while the IOCP buffer would not (since it would already be in the buffer), and the IOCP buffer could issue one write immediately when requestsed and buffer successive writes, while the selector Buffer must buffer *all* writes until drain is called (in both cases, drain could issue another write if buffered data is available). All read calls only operate on buffered data, and a single read is always pending on the Conduit in both cases, to ensure data will be read if it is available at the socket layer. Also, the IOCP buffer must also preserve the buffer area passed to any read/write calls until they complete (perhaps a chain of read/write buffers for the IOCP version, so instead of growing buffers in place a new one is simply obtained--this could be fairly efficient if an slist of unused buffers is maintained).
The thread synch issue jcomellas noted may not be a problem because it's uncommon to issue multiple simultaneous reads/writes to a single Conduit, thus there should be no need for mutual exclusion. In fact, I would suggest not even supporting scatter/gather for now, as I don't think it's worth the trouble.
I admit this probably sounds complicated, but it's really just a matter of writing custom buffers for the proactor and reactor patterns. And the differences between the two Buffer implementations are really fairly slight, despite my inability to describe things well.
Does this sound reasonable? I'm still getting up to speed on how Mango actually works, so I'm basing this on a quick scan of the IBuffer and IConduit file comments. |
|
| Back to top |
|
|
sean
Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA
|
| Posted: Wed Feb 15, 2006 5:11 pm Post subject: |
|
|
| By the way, I think there's no way around the read buffer potentially growing over time if the user is expecting encoded object data, since object size is unbounded. But hopefully, this won't be a common occurrence |
|
| Back to top |
|
|
jcomellas
Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina
|
| Posted: Wed Feb 15, 2006 5:34 pm Post subject: |
|
|
| sean wrote: | The important bit is that the selector Buffer would load available data on a fill call while the IOCP buffer would not (since it would already be in the buffer), and the IOCP buffer could issue one write immediately when requestsed and buffer successive writes, while the selector Buffer must buffer *all* writes until drain is called (in both cases, drain could issue another write if buffered data is available). The IOCP buffer must also preserve the buffer area passed to any read/write calls until they complete (perhaps a chain of read/write buffers for the IOCP version, so instead of growing buffers in place a new one is simply obtained--this could be fairly efficient if an slist of unused buffers is maintained).
The thread synch issue jcomellas noted may not be a problem because it's uncommon to issue multiple simultaneous reads/writes to a single Conduit, thus there should be no need for mutual exclusion. In fact, I would suggest not even supporting scatter/gather for now, as I don't think it's worth the trouble.
Does this sound reasonable? I'm still getting up to speed on how Mango actually works, so I'm basing this on a quick scan of the IBuffer and IConduit file comments. |
I like your proposed solution a lot. There are just two things that bother me:
1) By doing what you propose we negate one of the main benefits of IOCP's, which is avoiding context switches. With this, the read would be performed on a thread blocking on GetQueuedCompletionStatus() and the actual processing of the read data on the original calling thread.
2) I don't like having different behaviors for the Selectors depending on their type, so I would at least change the class name for the IOCP one and divide them into two families of classes.
I think that we'd need to add some kind of callback mechanism via delegates to have an efficient interface for IOCP. |
|
| Back to top |
|
|
sean
Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA
|
| Posted: Wed Feb 15, 2006 6:27 pm Post subject: |
|
|
| jcomellas wrote: | I like your proposed solution a lot. There are just two things that bother me:
1) By doing what you propose we negate one of the main benefits of IOCP's, which is avoiding context switches. With this, the read would be performed on a thread blocking on GetQueuedCompletionStatus() and the actual processing of the read data on the original calling thread. |
Aren't they typically the same thread? I know they are in my IOCP implementation. Though I suppose with IOCP you really can't make any assumptions--the kernel will try to avoid a context switch if possible, but there are no guarantees.
| Quote: | 2) I don't like having different behaviors for the Selectors depending on their type, so I would at least change the class name for the IOCP one and divide them into two families of classes.
I think that we'd need to add some kind of callback mechanism via delegates to have an efficient interface for IOCP. |
I agree that the selector model isn't ideal for IOCP as the selection loop is completely unnecessary. In fact, this was the only major difference in the way I handled poll vs. completion-based IO: the poll version had its selection loop built into the main program loop while with the IOCP version everything was hidden. It would be nice if a somewhat more common syntax could be found however, or perhaps create a default selector function in such a way that the average user doesn't have to worry about how it works:
processSelectionEvents( uint timeout = 0 );
Though the user would probably still need a way to attach event handlers for read, write, and error events? It might even be that a similar function wrapper could be done for IOCP, though I don't remember if GetQueuedIoCompletionStatus takes a timeout parameter. |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Wed Feb 15, 2006 7:59 pm Post subject: |
|
|
Thinking aloud:
what if this whole business were built-in as part of a Buffer subclass? I mean, there might be a number of different subclasses that behave in different ways (such as Proactor, Reactor, etc). They'd each have appropriate default functionality for read, write, and error; which could be overridden by a user as necessary ... |
|
| Back to top |
|
|
sean
Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA
|
| Posted: Wed Feb 15, 2006 8:33 pm Post subject: |
|
|
| kris wrote: | Thinking aloud:
what if this whole business were built-in as part of a Buffer subclass? I mean, there might be a number of different subclasses that behave in different ways (such as Proactor, Reactor, etc). They'd each have appropriate default functionality for read, write, and error; which could be overridden by a user as necessary ... |
That's kind of what I was thinking--have separate ProactorBuffer, ReactorBuffer classes. Though I couldn't speculate on details beyond there |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Thu Feb 16, 2006 6:05 pm Post subject: |
|
|
| jcomellas wrote: | | Now that I think of it, do you plan to add Conduits for pipes? If so, we'd have to differentiate between the input handle and the output handle. |
Yes, you're right. A PipeConduit would have two handles. I think it'd be useful to have such a beastie. |
|
| Back to top |
|
|
jcomellas
Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina
|
| Posted: Thu Apr 06, 2006 6:32 am Post subject: |
|
|
| kris wrote: | | jcomellas wrote: | | Now that I think of it, do you plan to add Conduits for pipes? If so, we'd have to differentiate between the input handle and the output handle. |
Yes, you're right. A PipeConduit would have two handles. I think it'd be useful to have such a beastie. |
The other option is have a register() method directly in the Conduit. If we do that, the Conduit can choose the correct handle depending on the type of event that is being registered. This is what Java does. I'm not very fond of this but it's the simplest alternative.
BTW, I finished the Selectors several days ago, but I keep getting crashes in the garbage collector when working with threads under Linux. Have you experienced this? |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Fri Apr 07, 2006 8:07 pm Post subject: |
|
|
| jcomellas wrote: | The other option is have a register() method directly in the Conduit. If we do that, the Conduit can choose the correct handle depending on the type of event that is being registered. This is what Java does. I'm not very fond of this but it's the simplest alternative.
BTW, I finished the Selectors several days ago, but I keep getting crashes in the garbage collector when working with threads under Linux. Have you experienced this? |
I was experiencing some odditites recently, but not in the same scenario. Sean cleaned them all up with the latest version of Ares, so it might be worthwhile giving that a go to see if anything changes? I was actually seeing some deadlock in the Phobos allocator, along with some GPFs ~ but that was under Win32.
Great news on the Selector front, though. We could try the register() approach first if you like? Perhaps get a feel for how it operates?
Edit: I don't suppose the crashes have anything to do with prelink, as Dave has noted in the D forum? |
|
| Back to top |
|
|
jcomellas
Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina
|
| Posted: Tue Apr 18, 2006 8:03 pm Post subject: |
|
|
| kris wrote: | I was experiencing some odditites recently, but not in the same scenario. Sean cleaned them all up with the latest version of Ares, so it might be worthwhile giving that a go to see if anything changes? I was actually seeing some deadlock in the Phobos allocator, along with some GPFs ~ but that was under Win32.
Great news on the Selector front, though. We could try the register() approach first if you like? Perhaps get a feel for how it operates?
Edit: I don't suppose the crashes have anything to do with prelink, as Dave has noted in the D forum? |
No, the crashes are not related to prelinking. It looks like they are related to multithreading. If you can, try to compile and run the example/selector.d program I sent you on Win32 and see what happens. In my tests the program crashes 100? of the time.
Let me know what you think of the latest Selector interface I've sent you. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
