dsource.org - forums

dsource.org

Open Source Development for
the D Programming Language

FAQ

Memberlist

Usergroups

Profile

Selector support (select/poll/epoll/WaitForMultipleObjects)
Goto page Previous 1, 2, 3, 4 Next

Forum Index -> Mango

View previous topic :: View next topic

Author

Message

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Mon Feb 13, 2006 10:09 pm Post subject:

jcomellas wrote:

sean wrote:

Is it even necessary to access the handles directly?

It's either that or adding a method in the Conduit to register itself to the class that handles the call to select()/poll()/etc. (Selector).

We have several options:

1. The Selector registers the conduit to itself:

Code:

SocketConduit socket;
EpollSelector selector;

selector.register(socket, ISelectionSet.Read);

2. The Selector registers the Conduit's handle:

Code:

SocketConduit socket;
EpollSelector selector;

selector.register(socket.handle(), ISelectionSet.Read);

3. The Conduit registers to the Selector.

Code:

SocketConduit socket;
EpollSelector selector;

socket.register(selector, ISelectionSet.Read);

Which one do you prefer? I'm leaning towards option 1, whereas Java uses option 3.

I would lean that way also, since #3 would require Conduit to import Selector. I'll add the getHandle() method and check it in asap.

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Mon Feb 13, 2006 10:30 pm Post subject:

jcomellas wrote:

It still needs to be tested and debugged a little bit more on Linux, though.

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Mon Feb 13, 2006 10:39 pm Post subject:

jcomellas wrote:

I have finished the implementation of the first Selector, the EpollSelector for Linux. In order to use this with Mango I need to add some way of getting a Conduit's handle. Kris, do you think you could add that?

Here's a sample of what the usage of a Selector would look like (not compiled yet):

Code:

InternetAddress addr = new InternetAddress("127.0.0.1", 80);
SocketConduit socket;

try
{
auto ISelector selector = new EpollSelector();
int timeout = 5000; // 5 seconds
int eventCount;

selector.open();
socket.connect(addr);
selector.register(socket, ISelectionSet.Read | ISelectionSet.Write);

eventCount = selector.wait(timeout);
if (eventCount > 0)
{
ISelectionSet selectedSet = selector.selectedSet();

foreach (Conduit conduit, uint eventMask; selectedSet)
{
if (selectedSet.isRead(eventMask))
writefln("Received a Read event from conduit");
if (selectedSet.isWrite(eventMask))
writefln("Received a Write event from conduit");
if (selectedSet.isError(eventMask))
writefln("Received an Error event from conduit");
}
}
socket.close();
selector.close();
}
catch (Exception e)
{
writefln("Exception caught:\n\n?s", e.toString());
}

What do you think about it?

Yes, very nice indeed. I've just checked in the relevant changes for getHandle().

There's one thing that's perhaps worthy of discussion: while I find it really convenient to think in terms of milliseconds, and such values often fit nicely into 32bit ints, it's perhaps becoming a bit long-in-the-tooth for todays clock rates? I've tried nanoseconds, and microseconds, yet nothing else feels right for some odd reason Smile

Currently, mango.sys.System abstracts microseconds as the 'generic' time period, and this is applied in a few places within Mango. What do you think we should do with the whole time-period thingy?

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Mon Feb 13, 2006 10:58 pm Post subject:

sean wrote:

jcomellas wrote:

What do you think about it?

Very nice. So to access the data buffer I use conduit? And is there an easy way to know how many bytes are available if a read event occurred?

You could invoke Conduit.read(dst[]) and it will tell you how many bytes it read (up to the length of the provided buffer; will return Eof when there's nothing available). That doesn't quite get you all the way there, but it's pretty workable for the most part.

However, this is clearly at a level below Buffer ~ which you might have been hinting at, Sean? It might be nice if one could somehow associate a higher level construct with the selected Conduit for such things. Any ideas?

It would be certainly be possible to avoid the association, if the client were to perform a setConduit() on some pre-constructed Buffer, or Buffer/Reader pair. Still, that's perhaps a bit idealistic?

jcomellas

Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina

Posted: Tue Feb 14, 2006 10:57 am Post subject:

kris wrote:

Currently, mango.sys.System abstracts microseconds as the 'generic' time period, and this is applied in a few places within Mango. What do you think we should do with the whole time-period thingy?

It would surely be "cleaner" to use some kind of abstraction for timing, but I'm not very fond of multiplying or dividing by System.Interval.XXX whenever I need to pass a timeout to a method. I don't know if you've ever used the ACE C++ framework, but it solves this problem very well by using a class with microsecond precision (ACE_Time_Value) which is used whenever you need to pass a timeout or time interval. I think we need to do something like this and use it everywhere.

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Tue Feb 14, 2006 11:29 am Post subject:

jcomellas wrote:

kris wrote:

Currently, mango.sys.System abstracts microseconds as the 'generic' time period, and this is applied in a few places within Mango. What do you think we should do with the whole time-period thingy?

I haven't used ACE at all, but agree with the sentiment. What do you think, Sean? Eric?

jcomellas

Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina

Posted: Tue Feb 14, 2006 11:34 am Post subject:

sean wrote:

Can you explain how this works at a lower level? For example, using IOCP I typically dedicate one or more threads to act as a producer/consumer layer between socket and user code, and allow the thread pool to grow as needed to ensure at least one worker thread is always available (basically, a home grown version of BindIOCPCallback). Is that sort of thing going on behind selector.wait? Or is selector.wait simply a wrapper for GetQueuedCompletionStatus?

The pattern I'm implementing is a bit different from I/O completions ports or POSIX asynchronous I/O. Basically, select and its derivatives (poll, epoll in level triggered mode, WaitForMultipleObjects, etc.) fire an event to tell the application that the kernel is ready to perform the requested action (read, write, etc.), whereas IOCP's and POSIX AIO notify the application when the action has already been completed. With IOCP's you normally tell the kernel to, for example, read data from a socket into a buffer you provide, and the kernel lets you know when the data has been read.

This difference forces a complete different programming model for your application's I/O subsystem if you use one or the other APIs. In fact, this difference is what gave rise to two different patterns, the Reactor (select and company) and the Proactor (IOCP, POSIX AIO). You can find some additional information about them here.

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Tue Feb 14, 2006 11:49 am Post subject:

jcomellas wrote:

sean wrote:

Thanks; that's a succinct and very useful distinction.

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Tue Feb 14, 2006 12:02 pm Post subject:

jcomellas wrote:

I don't know if you've ever used the ACE C++ framework, but it solves this problem very well by using a class with microsecond precision (ACE_Time_Value) which is used whenever you need to pass a timeout or time interval. I think we need to do something like this and use it everywhere.

Another option would be to use a Double as the abstraction. I'd originally planned to go this route, but then decided against it after considering cell-phones and PDAs. Of course, those devices will ultimately get silicon FP support ~ perhaps Double would be worthy of consideration after all?

Alternatively, an ACE-like D struct would encapsulate the notion nicely, avoid the need for largely free-form conversion-functions, and avoid certain coercion issues.

sean

Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

Posted: Tue Feb 14, 2006 12:09 pm Post subject:

kris wrote:

jcomellas wrote:

kris wrote:

Currently, mango.sys.System abstracts microseconds as the 'generic' time period, and this is applied in a few places within Mango. What do you think we should do with the whole time-period thingy?

I haven't used ACE at all, but agree with the sentiment. What do you think, Sean? Eric?

I agree. I've been thinking about doing this for Thread.sleep in Ares as well.

jcomellas

Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina

Posted: Tue Feb 14, 2006 12:23 pm Post subject:

kris wrote:

Yes, very nice indeed. I've just checked in the relevant changes for getHandle().

BTW, wouldn't it be better to use a typedef for file descriptors/handles instead of int directly. I did something like this for my stuff:

Code:

public typedef int Handle = -1;

Other platforms may not use an int for this.

sean

Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

Posted: Tue Feb 14, 2006 12:58 pm Post subject:

jcomellas wrote:

Thanks. I'd been thinking the same pattern would be used for both styles of IO. Regarding WaitForMultipleObjects... so far as I'm aware, this is limited to tracking 64 events, and select on Windows has the same limitation. How does this model scale beyond that limit?

Last edited by sean on Tue Feb 14, 2006 12:59 pm; edited 1 time in total

kris

Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

Posted: Tue Feb 14, 2006 12:58 pm Post subject:

jcomellas wrote:

kris wrote:

Yes, very nice indeed. I've just checked in the relevant changes for getHandle().

BTW, wouldn't it be better to use a typedef for file descriptors/handles instead of int directly. I did something like this for my stuff:

Code:

public typedef int Handle = -1;

Other platforms may not use an int for this.

Indeed. Fixed & checked-in.

sean

Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

Posted: Tue Feb 14, 2006 2:44 pm Post subject:

By the way, do you plan on supporting the proactor model as well? I ask because I typically use IOCP on Windows, regardless of what I use elsewhere. Also, it's probably worth noting that while the proactor and reactor concepts are indeed somewhat opposed, the only notable difference in how they are handled at the code level is call order. Here's some old C++ library code of mine as an example. First, the select verison:

Code:

void connection::on_send( fd_set& set )
{
if( !socket::is_open() || !socket::signalled( set ) )
return;
socket::send( m_buf_send.buf, m_buf_send.len, m_buf_send.num_bytes );
switch( continue_send( m_buf_send ) )
{
case op_success:
begin_send( m_buf_send );
return;
}
}

And now the IOCP version:

Code:

void cp_connection::on_send( cp_overlapped& ovl )
{
switch( continue_send( ovl ) )
{
case op_success:
begin_send( ovl );
case op_more_data:
socket::send( ovl );
return;
}
}

As you can see, the difference is that the select (reactor) version calls socket::send at the beginning, while the IOCP (proactor) version calls socket::send at the end. The code for begin_send and continue_send is identical in both cases--separate implementations exist mostly to avoid the use of virtual function calls for performance (though were I to do it over again I'd use template code instead).

For reference, here is a copy of begin_send and continue_send. Only the parameter name differs between the two implementations:

Code:

void connection::begin_send( msg_buf& mbuf )
{
if( m_sending || m_send_queue.empty() )
   return;
m_sending = true;
m_send_queue.get( m_send_cur );
m_send_cur->read( mbuf.begin(), mbuf.size() );
mbuf = m_send_cur->gcount();
}

connection::op_result connection::continue_send( msg_buf& mbuf )
{
mbuf += mbuf.num_bytes;
if( mbuf.remaining() == 0 )
{
   m_send_cur->read( mbuf.begin(), mbuf.size() );
   mbuf = m_send_cur->gcount();
}
if( mbuf.remaining() > 0 )
{
   return op_more_data;
}
else
{
   m_sending = false;
   return op_success;
}
}

jcomellas

Joined: 30 Jan 2006
Posts: 22
Location: Buenos Aires, Argentina

Posted: Tue Feb 14, 2006 4:58 pm Post subject:

sean wrote:

AFAIK it does not scale on Windows. It's been a long time since I did this kind of work on Windows, but the last time I did the only acceptable option was to use IOCP's. The problem is that to do that we need to modify the Conduit interface to support overlapped I/O. I don't think it's possible to do that right now with the current Mango Conduit interface.

The next step is to build a real Reactor/Proactor with a unified interface for events from file descriptors, timers and Unix signals.

There's a paper here with an ACE compatible class to provide a Proactor interface on Unix and Windows. Maybe that could be our starting point.

Last edited by jcomellas on Tue Feb 14, 2006 5:01 pm; edited 1 time in total

Display posts from previous:

	Forum Index -> Mango	All times are GMT - 6 Hours Goto page Previous 1, 2, 3, 4 Next
Page 2 of 4

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum