Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

Read/Write fill using buffer

Moderators: larsivi kris

Posted: 09/21/08 10:31:46

Hi,

Reading/writing files using a buffer is a gain of speed.

Does Tango have support to a "File IO buffer" which manage position's changes in files ?

Thanks in advance, TSalm

Author Message

Posted: 09/21/08 17:13:02

A buffer is really just a window looking onto a data-stream, and neither has any real notion of a file-position (files are a special-case of stream, since they're the only variation to support position manipulation via seek).

One way to deal with this is to not buffer the input. Another way is to always clear the input and/or output buffer(s) after a file seek. Another way is to make the buffer the same size as the file, and then use buffer.skip to move around (or simply treat the buffer content as an array). Yet another approach is to use MapBuffer to memory-map the file content in a similar fashion, and then treat the file content as an array.

Hope that helps?

Posted: 09/21/08 18:49:50

My files are too big to have it all in memory, so the two latest can't be use.

Because, the most frequently case is to read linearly the file, I think the best way is to "File Reader" which use a buffer which is clearing and reloading datas after a position change.

Thanks Kris.

Posted: 09/21/08 18:54:58

yes, could be. BTW, MapBuffer? uses virtual memory and the O/S paging mechanisms. It ought to work for ya :)

Posted: 09/21/08 19:02:16

oups, I missed it... Therefore, MapBuffer? is the perfect way, thanks !

Posted: 09/21/08 19:05:48

Hope it works out well :)

Posted: 09/26/08 11:08:33 -- Modified: 09/26/08 11:09:49 by
lars_kirchhoff

I would be interested in some sample code for using the MappedBuffer? to read a write large files. I read some of the forum posts here and the tutorials, but don't know how I should start.

And a second question is how to use it with a TypedStream?.

Any help is appreciated. Thanks

Lars

Some thoughts about switching from PHP to D: http://www.lars-kirchhoff.de/go/journal/section/from-php-to-d/