Reading structs from Buffer / Conduit

xammy Joined: 02/01/07 Posts: 23

Posted: 04/03/07 18:15:09 I want to read some structs from a self-built MyConduit?. The latter defines a reader() method which fills data, returns length or Conduit.Eof at the end. I've also attached a buffer to the conduit. Now I want to read some kind of struct one after another, not knowing the length! Which method of my buffer do I use? I tested Buffer.fill(), but it didn't use buffering, so for every read, MyConduit?.reader() was invoked. Then Buffer.extract seemed to be quite good, as it really buffers the data and takes pointer/length pair - ideally for structs. But if MyConduit?.reader() returns Eof, Buffer.extract() prints an error: "end-of-file whilst reading". Or is this an exception, I can catch? I haven't figured it out, as it calls error(). Which function is preferrable? PS: I also tested the Reader protocol, but it also shows the error!

Author	Message
xammy Joined: 02/01/07 Posts: 23	Posted: 04/03/07 19:19:15 okay, exceptions weren't such a bad guess :) This way I can do it: auto read = new Reader(conduit); while (1) { try { read (element); Stdout ("read: ") (element) ("\n") (); } catch { break; } } Or is there a better / more elegant way?
kris Joined: 03/27/04 Posts: 581	Posted: 04/06/07 06:03:20 Exceptions are a terrible way to approach this :) The general mechanism with Reader/Writer is for a class to implement IReadable and/or IWritable. At that point, the class can be passed as an argument to any Reader and/or Writer instance (you don't create Reader/Writer subclass to do this, or even another protocol -- that would severely limit flexibility). If I have a class Foo, and have it implement IReadable/IWritable, I can treat it like this: Foo foo; Writer write; Reader read; write (foo); read (foo); This is all documented in the reference manual. Catching exceptions when doing this is, as a rule, not a good idea. The reason being that there is a valid "expectation" by the code that a (readable) instance actually exists within the stream. If it is not there, it is truly an exceptional condition. This is quite different than, for example, testing a stream to see if there a newline right around the corner. To extend this notion to a struct, please follow the instructions in the ref manual? Hope this help :)
xammy Joined: 02/01/07 Posts: 23	Posted: 05/28/07 09:01:06 Can one achieve this with a Reader, although one does not know, if there is data on a stream, without using exceptions? AFAICS the get() routines return an IReader for chaining, so I guess there's no such way?!
kris Joined: 03/27/04 Posts: 581	Posted: 05/28/07 20:48:48 "no available data" is an exceptional condition for Reader/Writer. If there's a protocol requiring optional attributes, there should be flags in the stream to indicate this, or the protocol itself should hide the distinctions.
xammy Joined: 02/01/07 Posts: 23	Posted: 05/29/07 14:38:57 Okay, now I recognized (thanks to kris) that Readers don´t fit my needs (it´s for FastCGI, as you might have guessed): I don´t now for sure, whether there´s more data on my stream. I need to read arrays and structs in a mixed order and those arrays are not preceeded by their length (at least not the way, tango.io.protocol.Reader would expect it) Is there a way to achieve this in tango without reading via the tango.io.Buffer routines directly? I´ve tried to implement it via Readers and it was much more elegant than with direct Buffer access. The only problems are the two, I´ve mentioned above :(
r.lph50 Joined: 11/27/06 Posts: 21	Posted: 05/30/07 13:15:49 I'm familiar with the FastCGI protocol, what part of the protocol are you reading arrays for?
xammy Joined: 02/01/07 Posts: 23	Posted: 05/30/07 15:39:55 It's about reading the stream data itself (FCGI_STDIN / FCGI_DATA) - they have no fixed size, which you know from the header struct. but between those, there's also the padding field. and content length is 2 byte in FastCGI, the length of arrays you read via a tango Reader will be 4, maybe 8 bytes on 64bit archs, as I guess?! And reading it char-by-char seems to be not really efficient, as you have to be able to save them in memory.
kris Joined: 03/27/04 Posts: 581	Posted: 05/30/07 18:29:01 Reader/Writer are configurable via the associated Protocol. By default, the protocol is one of raw binary with prefixed arrays, but this can be reconfigured as required. You can also attach multiple Reader/Writer instances to a shared Buffer, and they will remain in lockstep with each other. You can combine that with reading directly from the Buffer too ... e.g. you might use a reader to extract key fields, and use buffer.readExact() to pull out some raw data? I'm not familiar with the FastCGI protocol, so can't really comment on how best to approach this. But I expect the facilities are all there to handle it efficiently.
xammy Joined: 02/01/07 Posts: 23	Posted: 05/30/07 21:16:19 The one with combining Reader and Buffers sounds nice, I'm gonna try this!
r.lph50 Joined: 11/27/06 Posts: 21	Posted: 05/31/07 01:09:20 -- Modified: 05/31/07 01:10:19 by r.lph50 I have no idea if this is good tango style.. but I am doings this: I have a protocol driver which has a tango.io.GrowBuffer?. While that buffer has enough readable content to form a fastcgi Header struct, I slice the same number of bytes as the Header.sizeof out of the buffer: void[] headerData = inputBuffer.slice(Header.sizeof, false); and convert this to a Header struct. You can then check the protocol version, message length, request id etc. If the buffer doesn't have enough readable content to read the message (Header.sizeof + contentLength + paddingLength), I just return to await more input. If it does have enough content for the message, then I slice the buffer to get content: void[] content = inputBuffer.slice()[Header.sizeof .. Header.sizeof+contentLength]; and call the appropriate function for the message type, and afterwards do inputBuffer.truncate(Header.sizeof + contentLength + paddingLength); If the message type is STDIN or DATA then their function handlers copy the data to their GrowBuffer?.