Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

LineIterator re-uses memory buffer

Moderators: kris

Posted: 10/17/07 17:51:37

I noticed with LineIterator? that if you save copies of the lines they get corrupted while iterating through a stream. I don't know exactly where the problem is, but it seems to occur with large files. For example:

import tango.io.FileConduit;
import tango.text.stream.LineIterator;
import tango.io.Stdout;

int main(char[][] args)
{
  FileConduit fc = new FileConduit(args[1]);
  char[][] lines;
  foreach(line; new LineIterator!(char)(fc))
  {
    lines ~= line;
  }

  foreach(char[] line; lines)
  {
    Stdout(line).newline;
  }
  return 0;
}

and if I use a large file with lots of lines, on my linux system, I'm using /lib/modules/<kernelversion>/modules.alias, the output is not the same as the input file. I'm not sure what size is the critical mass.

To "fix" this, you can use the .dup property like so:

    lines ~= line.dup;

So either this is by design, in which case there should be a documentation to this effect (i.e. "do not store the elements or slices of elements returned by the line iterator outside the foreach loop"), or this is a bug.

-Steve

Author Message

Posted: 10/17/07 19:04:27

yes, the iterator exposes slices of the content so you have to .dup where appropriate. I'll add a prominent note in the doc for this.

Thanks!