Stream, MemoryStream and File

dsoul Joined: 08/30/08 Posts: 1

Posted: 08/30/08 20:46:30 Hi. I'm new on Tango and I don't know how to do somethings that I used to do with Phobos. Concretely I need a seekable generic Stream. For example. How can I do this, using tango? import std.stdio, std.stream; void readTest(Stream s) { uint v; s.position = 4; s.read(v); assert(v == 2); } void main() { Stream s; s = new File("myfile", FileMode.In | FileMode.Out); { s.write(cast(uint)1); s.write(cast(uint)2); s.position = 0; readTest(s); } s.close(); s = new MemoryStream(); { s.write(cast(uint)1); s.write(cast(uint)2); s.position = 0; readTest(s); } s.close(); } I tried to figure out how to do it in Tango. But I have a mess with Conduits, and buffers and I don't understand how to do it. To solve it I made (temporally) a Stream subset similar to Phobos one: abstract class Stream { alias IConduit.Seek.Anchor Anchor; ubyte read1() { ubyte v; read((&v)[0..1]); return v; } ushort read2() { ushort v; read((&v)[0..1]); return v; } uint read4() { uint v; read((&v)[0..1]); return v; } ulong read8() { ulong v; read((&v)[0..1]); return v; } void write(ubyte v) { write((&v)[0..1]); return v; } void write(ushort v) { write((&v)[0..1]); return v; } void write(uint v) { write((&v)[0..1]); return v; } void write(ulong v) { write((&v)[0..1]); return v; } abstract uint read (void[] v); abstract uint write(void[] v); abstract long seek(long offset, Anchor anchor = Anchor.Begin); ulong length() { ulong back = position; ulong ret = seek(0, Anchor.End); position = back; return ret; } ulong position() { return seek(0, Anchor.Current); } ulong position(ulong set) { return seek(set, Anchor.Begin); } } class FileStream : Stream { FileConduit file; this(char[] name) { file = new FileConduit(name); } override uint read (void[] v) { return file.read(v); } override uint write(void[] v) { return file.write(v); } override long seek(long offset, Anchor anchor = Anchor.Begin) { return file.seek(offset, anchor); } } class MemoryStream : Stream { long _position; long _length; ubyte[] data; this(ubyte[] data) { this.data = data; _position = 0; _length = data.length; } this(int _length) { this.data = new ubyte[_length]; this._position = 0; this._length = _length; } override uint read(void[] v) { ubyte[] vu = cast(ubyte[])v; int read = min(vu.length, _length - _position); vu[0..read] = data[_position.._position + read]; _position += read; return read; } override uint write(void[] v) { ubyte[] vu = cast(ubyte[])v; int read = min(vu.length, _position + _length); data[_position.._position + read] = vu[0..read]; _position += read; return read; } override long seek(long offset, Anchor anchor = Anchor.Begin) { switch (anchor) { case Anchor.Begin: return (_position = offset); case Anchor.Current: return (_position += offset); case Anchor.End: return (_position = _length + offset); } return -1; } override ulong length() { return _length; } override ulong position() { return _position; } } Thanks in advance, and sorry about my bad english.

Author	Message
korDen Joined: 03/12/08 Posts: 13	Posted: 08/31/08 07:59:59 Below are my thought on the subject, they can differ from others' point of view. Get used to Conduits, that's a new idiom used in Tango. It took some time for me to change my way of thinking about streams, but it is worth it. A Stream abstraction is used in many languages and libraries. That's something you usually read from and write to (like FileStream?, MemoryStream? etc). That's a C# and D/Phobos way. Such a stream has both reading and writing methods. Many don't have Stream but separate InputStream? and OutputStream?. InputStream? has reading methods and OutputStream? has writing methods. That's a Java way. There are separate FileInputStream? and FileOutputStream? classes, for example. (In fact, you never ever want to both read and write from file simultaniously.) Both approaches are quite similar, but have some differences. Every Stream in C# is both readable and writable. It is arguably a not very good design, because some streams are better implemented with separate input/output objects (Socket stream, for example). That's why Java Socket has getInputStream() and getOutputStream() methods. Besides, there are streams that are read only, and those that are write-only. So having two separate interfaces for reading and writing is good. However, creating two separate classes for reading/writing is an overkill, since you should open a file twice. Sometimes it is not possible at all, the two should be created simultaniously (Socket stream). Now back to D/Tango. A Conduit is something that provides InputStream? and OuputStream?. A simplified model is as follows: interface IConduit { InputStream input(); OutputStream output(); } That's the most sensible solution for sockets: you create it once and have both streams available. And for file streams - you allocate file handle just once. But Tango takes it one step further. It is not very handy to write file.input.read() or file.output.write(). It is long and takes 2 virtual function calls instead of just one. That's why all Conduits subclass from InputStream? and OutputStream?, and often input() and output() methods are implemented as "return this;". As a conclusion, what you need is a Conduit, not Stream. In fact, they are quite the similar (both subclass from InputStream? and OutputStream?), but slightly different in idea. Look at a (simplified) difinition of Stream in Phobos (belov) and compare to the IConduit above: interface Stream : InputStream, OutputStream { } I'd say that a Stream is slightly more restrictive, because in case of read-only stream you can't cast it to OutputStream? and get a null pointer. But I'm kinda dissapointed that there is no MemoryConduit? class in Tango yet. Feel free to implement and contribute it, I believe it will be gladly accepted!
larsivi Joined: 03/27/04 Posts: 613	Posted: 08/31/08 08:32:48 Somewhere there is a ticket, look for seekable streams.
schveiguy Joined: 08/02/07 Posts: 128	Posted: 08/31/08 09:01:38 Ticket is #863