Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

tango.io.Buffer absolute read/write methods

Moderators: larsivi kris

Posted: 05/26/08 11:00:48 Modified: 05/26/08 16:29:48

Hi, guess the last message contains too much code. My problem is that io.Buffer is a bit unflexible regarding read/write 'cause io.Buffer has relative read/write support only. I.E

// java.nio.ByteBuffer allows ie. the follwing method calls:
 float  getFloat()
 float  getFloat(int index)  // absolute !
 void  putFloat(float f)
 void  putFloat(int index, float f) // absolute !

So :

tango.io.Buffer has no get(uint index)...

So what are possible work arounds. ?

tango.io.Buffer contains the following member :

protected uint index; // current read position

I think this is also the write position. Right ? Means adding : uint position (uint pos) {....}; is allready something.... probabely adding an optional parameter to put get / read write could help. ie uint pos = this.index

So it should be possible to modify tango.io.Buffer without breaking existing code.

However, it's a pity. Tango contains nearly all of the stuff I need :Threading, logging, sockets, memory mapped files, collections, plus things I may need in future. Except this simple minor thing.

So what is the best way to enhance tango.io.buffer to get what I need. Thanx in advance, bjoern

PS: I need this functionality to a create "view" onto a Buffer.

Author Message

Posted: 05/26/08 18:43:01

Buffer is merely a mechanism to support buffered streaming ... it's a sliding window on untyped data flowing by. It can also be used to collect data, like an automatically expanding array, but that's not necessarily the primary goal. Buffer is also explicitly typeless.

There are some explicitly typed wrappers that sit atop Buffer, and translate the content appropriately. DataStream is one such wrapper, and the protocol package represents another, though neither of those explicitly support indexing in the manner you suggest because they have to respect the notion of a stream (versus an array).

However, you can easily treat the current Buffer content as an array of whatever type you like. Simply retrieve the backing-array and manipulate it accordingly. Buffer.getContent() or Buffer.slice() should give you what you need, albeit as a void[]

Hope that helps?

Posted: 05/26/08 19:21:34

Yes, That helps.

But what you suggest requires a lot of cast() operations... May I suggest to introduce FlexBuffer? / FlexGrowBuffer? ? A Buffer which can be treaded as stream (like now) and as hmmm, what, As scliceable, index-accessable buffered stream. I 've no idea how much need there is for such datastructures so it's probabely just me who feels a bit limited with io.Buffer. However in order to port some Java code a FLexBuffer would be a big win.

Remaining question : What would be the most (speed) efficient way to put/get a value at position N by using getContent()/slice(). means also byte[], etc...

Thanks Kris, also thanks to Lars for spending his time.

Posted: 05/26/08 19:35:01

The IO library is organized in a way that tends to place such functionality in place other than Buffer (such as DataStream). The benefit is that anyone can write any kind of wrapper, to represent data in whatever way they want. We did something similar with reading lines also ... there's no embedded readLine() in the generic IO portions of Tango because we'd have to support three versions of it everywhere (char/wchar/dchar). Instead, the line parsing is isolated into a wrapper, such as LineIterator and friends. A little bit of convenience is potentially lost, in trade for some notably improved flexibility and extensibility (or customization).

Posted: 05/26/08 19:57:30

Thanks again. I guess a "problem" with Tango is due to the fact that Tango is a HUGE lib. tango.io. (for example) requires IMO a lot of time in order to be able to use all its capabilities. So your last msg does NOT help. In fact it confuses me a bit. hope the unit tests tell me more. Bjoern ps : ATM I thing writing a class mybuffer { private void[] data; is easier than understanding tango's io system. ( written while beeing frustrated ) ;) Ah No ! I am a Tango fan...

Posted: 05/26/08 21:52:26

Sorry :(

I suppose I was trying to say this: it's easy to do what you suggest by accessing the underlying array used by Buffer (via Buffer.slice etc)

Posted: 05/27/08 00:02:51

Aye, Pas de problème Oh mon capitaine.

I guess regarding the underlaying array you talk abounty :

protected void[] data; // the raw data buffer

Can you please offer a few lines of code showing a get/put of Type T at pos N ?

Bjoern

Posted: 05/27/08 06:07:44

First I suggest you take a peek at TypedStream, if you've not already done so. Then assuming content is already loaded into a Buffer, and assuming there's sufficient data within it, then it's a matter of converting a void[] to a T[] and reading it:

void[] content = buffer.slice;
T value = (cast(T*) content.ptr)[index];

There's other ways to do that sort of thing in D, but that one is a common pattern. You can also create a Buffer with an array of your own:

T[1000] myarray;
auto buffer = new Buffer(myarray);
...

But if all your doing is reading a file or whatever, then you perhaps don't need a Buffer at all? If so, just read directly from a Conduit instead. For example:

auto file = new FileConduit ("myfile");
T[1000] myarray;
auto bytes = file.read (myarray);
...

Buffers are generally used for streaming data through your program, and they're certainly not required. Still, they can be useful for many things: MappedBuffer, for example, maps an entire file into virtual memory, and gives you access to all of it as a void[]

Is this helping?

Posted: 05/29/08 08:53:59

Sorry for answering that late, and yes it helps.

ATM I am still convinced that a Buffer should have absolute and relative read/write support out of the box. Seems that you don't share this opinion :) However, many thanks for hints and tips. Bjoern