Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

Issue with tango.io.protocol.NativeProtocol.readArray and empty array

Moderators: kris

Posted: 08/07/07 17:58:39

Hi,

I found a bug, and I'm not exactly sure which is the best way to fix it because I do not fully comprehend the impact of fixing it. The bug is when reading an empty array using the NativeProtocol? object with prefixes turned on. In the implementation, the the NativeProtocol? reads a 4-byte integer, which should be 0 (an empty array), and then tries to read 0 bytes from the buffer. In the case that the buffer is now empty, and the source conduit is not ready for reading, the call waits for data to be on the stream, even though it has requested 0 bytes. I found it because in a packet that I am sending over a network conduit, a variable length array is the last thing I am writing. In the case that it is actually empty, the method which reads the full packet doesn't return until another packet is sent on the stream. Then both packets are returned in succession. You can see this demonstrated by the following two files (built on Linux). You have to run them to see the bug at work:

testclient.d:

import tango.net.SocketConduit;
import tango.net.Socket;
import tango.io.protocol.PickleProtocol;
import tango.io.protocol.Writer;
import tango.io.Stdout;
import tango.stdc.posix.unistd;

int main(char[][] args)
{
  //
  // connect to the server
  //
  SocketConduit mysock = new SocketConduit();
  mysock.connect(new IPv4Address("127.0.0.1", 0x1234));

  ubyte[] tmp;

  Writer w = new Writer(new PickleProtocol(mysock));
  while(true)
  {
    tmp.length = (tmp.length + 1) % 4;
    w(tmp);
    w.flush();
    Stdout("Just sent data of length ") (tmp.length).newline;
    sleep(2);
  }
  return 0;
}

testserver.d:

import tango.net.ServerSocket;
import tango.net.SocketConduit;
import tango.io.model.IConduit;
import tango.io.Stdout;
import tango.io.protocol.PickleProtocol;
import tango.io.protocol.Reader;

int main(char[][] args)
{
  //
  // create a TCP socket, bind to an address and wait for incoming connections
  //
  ServerSocket mysock = new ServerSocket(new InternetAddress(0x1234));
  SocketConduit clientSock;
  while((clientSock = mysock.accept()) !is null)
  {
    //
    // read all the data from the client socket, then close the socket
    //
    ubyte[] data;
    Reader r = new Reader(new PickleProtocol(clientSock));
    try
    {
      while(true)
      {
        r(data);
        Stdout("read something of length ") (data.length).newline;
      }
    }
    catch(Exception e)
    {
    }
    Stdout.newline()("Disconnected...").newline;
  }
  return 0;
}

I see two places to fix this. First is in NativeProtocol?.d, in NativeProtocol?.readArray. I have fixed it in my copy with this code:

        void[] readArray (void* dst, uint bytes, Type type, Allocator alloc)
        {
                if (prefix_)
                   {
                   read (&bytes, bytes.sizeof, Type.UInt);
                   if(bytes == 0)
                     //
                     // no need to read 0 bytes
                     //
                     return new void[0];
                   return alloc (&read, bytes, type);
                   }

                return read (dst, bytes, type);
        }

the second possibility is in Buffer.d, in the method Buffer.fill(void[]) with an empty buffer. In this case, the buffer calls read even though the condition for the loop is satisfied before it begins:

        uint fill (void[] dst)
        {
                uint len;

                do {
                   uint i = read (dst [len .. $]);
                   if (i is IConduit.Eof)
                       return (len > 0) ? len : IConduit.Eof;

                   len += i;
                   } while (len < dst.length);

                return len;
        }

I'm not sure if fixing it here is the best thing, because I'm not sure why a do loop was used instead of a while loop.

There may also be other places of fixing it. However, I'm satisfied with my NativeProtocol? fix, although other cases might creep up that don't use NativeProtocol?.

-Steve

Author Message

Posted: 08/10/07 14:32:30

Thanks ... will look into it when I return next week :)

Posted: 10/19/07 15:04:37

After downloading the newest version of Tango, I realized I had to fix this again in my local copy. Any update on this bug?

-Steve

Posted: 10/19/07 17:57:40

sorry, I lost track of this completely. Will address it this weekend for sure.

Thanks