| View previous topic :: View next topic |
| Author |
Message |
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
 Posted: Sat May 20, 2006 12:06 pm Post subject: Socket.accept().isTextual()? |
 |
|
I hope code can explain better than I can:
| Code: |
auto ia = new InternetAddress (port);
auto server = new ServerSocket (ia, 10);
auto client = server.accept ();
auto reader = new TextReader (client, new LineToken);
|
Everything is Ok, until I run it and I get: "Error: text/binary mismatch between Reader and Buffer". I discovered that Socket.accept() returns non-textual sockets (through a long series of calls).
So, my question is, how can I read from a socket line by line? (BTW, Phobos makes it very easy, as SocketStream has a readLine() method.) |
|
| Back to top |
|
 |
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Sun May 21, 2006 11:05 pm Post subject: |
|
|
Ach ... I missed that one completely. Trying to check in a fix, but dsource won't allow checkins right now.
Should be able to do this:
| Code: |
auto ia = new InternetAddress (port);
auto server = new TextServerSocket (ia);
auto client = server.accept ();
foreach (line; new LineIterator (client))
char[] content = line.get; |
|
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 11:13 am Post subject: |
|
|
For the time being, I had to write a ConduitStream and use stream functions, but I'd certainly prefer a Mango solution.
I thought about using a LineIterator but I have a few doubts about that approach:
- How does it behave? Will it wait forever until a newline is sent? I mean, what if the last 5 characters get lost or arrive late? What if more than one line is sent but some arrive a bit too late? Will it just return one line?
- Isn't it a bit of overkill when reading just one line?
- Will I have to create new LineIterators everytime I want to read?
Anyway, I'll try that and see if it fits what I'm trying to do.
Thanks! |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 11:39 am Post subject: |
|
|
| Carlos wrote: | For the time being, I had to write a ConduitStream and use stream functions, but I'd certainly prefer a Mango solution.
I thought about using a LineIterator but I have a few doubts about that approach:
How does it behave? Will it wait forever until a newline is sent? I mean, what if the last 5 characters get lost or arrive late? What if more than one line is sent but some arrive a bit too late? Will it just return one line? |
Iterators can be used with a conduit, a buffer, or just a simple array. This means you can hook them up to files, sockets, memory-mapped files, pipes, etc. Socket-based conduits have an optional timout that can be set by the programmer ~ if nothing arrives within the timout period, an EOF is assumed, and a flag is set to indicate the reason.
| Carlos wrote: | | Isn't it a bit of overkill when reading just one line? |
Do you mean the cost of creating an object? The alternative is to build the various iterator functionality directly into the conduit/stream itself. If I had a system where GC use should be avoided, I'd maintain a buffer + iterator on a per-thread basis (by subclassing Thread), and then attach the buffer to the incoming socket via setConduit(). This is partly how mango.servlet and mango.http.server avoid per-request allocations. SocketConduit maintains a linked-pool, so that it doesn't push any GC buttons once it's warmed up. I guess it's all about tradeoffs 
| Carlos wrote: | | Will I have to create new LineIterators everytime I want to read? |
Nope. Iterator has set...() methods, to attach them onto a new instance of something. |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 11:42 am Post subject: Re: Socket.accept().isTextual()? |
|
|
| Carlos wrote: | I hope code can explain better than I can:
| Code: |
auto ia = new InternetAddress (port);
auto server = new ServerSocket (ia, 10);
auto client = server.accept ();
auto reader = new TextReader (client, new LineToken);
|
Everything is Ok, until I run it and I get: "Error: text/binary mismatch between Reader and Buffer". I discovered that Socket.accept() returns non-textual sockets (through a long series of calls).
So, my question is, how can I read from a socket line by line? (BTW, Phobos makes it very easy, as SocketStream has a readLine() method.) |
BTW, this text/binary mismatch thing is a bit of a pain. It came into being to help catch cases where the programmer was making a mistake. Like many such efforts, this one gets in the way sometimes. I'd appreciate any ideas on a better solution |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 1:30 pm Post subject: |
|
|
| kris wrote: | Should be able to do this:
| Code: |
auto ia = new InternetAddress (port);
auto server = new TextServerSocket (ia);
auto client = server.accept ();
foreach (line; new LineIterator (client))
char[] content = line.get; |
|
Mmm... not exactly. Remember that the trunk doesn't work, so for Mango 2.0 it's:
| Code: |
foreach (char [] line; new BufferTokenizer (client, new LineToken))
{
...
}
|
So, if I'm waiting for the client to send something, one command in one line, I have something like this:
| Code: |
while (true)
{
char [] command;
foreach (char [] line; new BufferTokenizer (client, new LineToken))
{
command = line;
break;
}
...
}
|
It's plain ugly. I'll keep using my previous solution until a new version is released or the trunk becomes usable.
Oh, and isTextual() is also a problem for writers, as I can't use DisplayWriter and, when I send a string, Writer sends some funny bytes:
| Code: |
auto client = server.accept ();
// auto writer = new DisplayWriter (client); // no way
auto writer = new Writer (client);
writer ("some string")(CR); // here
|
In the marked line, in the other side, I have to trim the string as I get weird stuff (what, I don't know, I haven't checked that deep) before and after the data. |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 1:36 pm Post subject: Re: Socket.accept().isTextual()? |
|
|
| kris wrote: | | BTW, this text/binary mismatch thing is a bit of a pain. It came into being to help catch cases where the programmer was making a mistake. Like many such efforts, this one gets in the way sometimes. I'd appreciate any ideas on a better solution |
Maybe Socket.accept (bool textual), as in the constructor?
At least the programmer should know if the data is text or binary. However, and this regards every other conduit, what happens if there's mixed text and binary data? Can Conduit.textual be changed? I don't think so, but then there's the question again: how to deal with mixed cases? |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 1:45 pm Post subject: |
|
|
| Carlos wrote: | Oh, and isTextual() is also a problem for writers, as I can't use DisplayWriter and, when I send a string, Writer sends some funny bytes:
| Code: |
auto client = server.accept ();
// auto writer = new DisplayWriter (client); // no way
auto writer = new Writer (client);
writer ("some string")(CR); // here
|
In the marked line, in the other side, I have to trim the string as I get weird stuff (what, I don't know, I haven't checked that deep) before and after the data. |
That's the distinction between text output and binary output ~ the latter prepends an array with the number of following bytes (Phobos does the same with the stream.write() method, I recall). DisplayWriter would complain about the "text/binary mismatch" in this case, because the socket is binary oriented. Usage of Writer will prepend the char[] length, since that class is not text-based. Just checked-in a fix for ServerSocket, so that might help?
One resolution would be to avoid emitting array lengths, and so on. That way, there'd be no mistaking the content. However, the programmer would then be responsible for inserting the appropriate "wire-protocols" instead, and performing all allocations when reading. It's a bit tricky to do the right thing, overall; I'm open to suggestions |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 1:45 pm Post subject: |
|
|
| kris wrote: | | Iterators can be used with a conduit, a buffer, or just a simple array. This means you can hook them up to files, sockets, memory-mapped files, pipes, etc. Socket-based conduits have an optional timout that can be set by the programmer ~ if nothing arrives within the timout period, an EOF is assumed, and a flag is set to indicate the reason. |
But if the client is GUI where the user can go to take a coffee, answer the phone, read some stuff, etc., and it's going to take a while to click a button or something, I can't set a timeout. It works when connecting, but then that's it. Can this timeout be changed?
| kris wrote: | | Carlos wrote: | | Isn't it a bit of overkill when reading just one line? |
Do you mean the cost of creating an object? The alternative is to build the various iterator functionality directly into the conduit/stream itself. If I had a system where GC use should be avoided, I'd maintain a buffer + iterator on a per-thread basis (by subclassing Thread), and then attach the buffer to the incoming socket via setConduit(). This is partly how mango.servlet and mango.http.server avoid per-request allocations. SocketConduit maintains a linked-pool, so that it doesn't push any GC buttons once it's warmed up. I guess it's all about tradeoffs |
That also seems too much... hehe... I thought about extending AbstractServer to ease my development, but my server only works with one client at a time, so I don't need multiple threads.
| kris wrote: | | Carlos wrote: | | Will I have to create new LineIterators everytime I want to read? |
Nope. Iterator has set...() methods, to attach them onto a new instance of something. |
See my previous post: LineIterator is not in 2.0. |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 1:56 pm Post subject: |
|
|
Socket timeout is disabled by default, so EOF is usually indicated via socket termination.
What is it that stops you from using the trunk, Carlos? Is it that wierd problem with duplicated content in one file? |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 2:00 pm Post subject: |
|
|
| SocketConduit.setTimeout() is where to set the timeout. Values are in microseconds, and a value of zero disables the timout (the default). |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 2:26 pm Post subject: |
|
|
| kris wrote: | | What is it that stops you from using the trunk, Carlos? Is it that wierd problem with duplicated content in one file? |
At least that. I actually haven't checked too much, but I think there's something of that, and I don't know if there're more errors. |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 2:28 pm Post subject: |
|
|
| kris wrote: | | SocketConduit.setTimeout() is where to set the timeout. Values are in microseconds, and a value of zero disables the timout (the default). |
Ok, thanks. |
|
| Back to top |
|
|
kris
Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific
|
| Posted: Mon May 22, 2006 2:34 pm Post subject: |
|
|
| Carlos wrote: | | kris wrote: | | What is it that stops you from using the trunk, Carlos? Is it that wierd problem with duplicated content in one file? |
At least that. I actually haven't checked too much, but I think there's something of that, and I don't know if there're more errors. |
I looked at the source code (in dsource) for that file, and it seemed fine. There's something very odd going on in that instance 
Oh, part of the reason why readLine() is not present within Buffer.d (or within socket.d) is that one would need three versions of it for the utf variants. Alternatively, one would need to template instead. Neither of those are attractive at that level, so I partitioned responsibility to the Iterator pattern instead. |
|
| Back to top |
|
|
Carlos
Joined: 19 Mar 2004
Posts: 396
Location: Canyon, TX
|
| Posted: Mon May 22, 2006 3:27 pm Post subject: |
|
|
| kris wrote: | | I looked at the source code (in dsource) for that file, and it seemed fine. There's something very odd going on in that instance |
A "revert" seem to have solved it.
I tried to compile the trunk again, and my problem now is GDC. Since it's still at DMD 0.140, it doesn't know about scope(), smart foreach, etc., and it fails. Besides that, mango.log.Hierarchy seems to be Ok now (with revert).
Problems found so far:
mango/io/FileConduit.d:333 [scope (exit)]
mango/http/server/HttpHeaders.d:157: template instance LineIteratorT!(char) is used as a type [???]
mango/io/DisplayWriter.d:90: template instance FormatStructT!(char) is used as a type [???]
(many more related to templates)
mango/sys/Epoch.d:538: 'timezone ' is not an arithmetic type
mango/sys/Epoch.d:538: incompatible types for ((-timezone ) / (60)): 'timezone' and 'int'
So, it's just a matter of either do nothing and wait until is GDC updated, or somehow support GDC? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
