ZlibStream Problem

lars_kirchhoff Joined: 05/27/08 Posts: 59

Posted: 01/27/09 15:44:41 Hey, I've tried to read a gzip file with the ZlibStream? class. But I always get a Z_DATA_ERROR. Here is the code: auto fi = new FileInput("file.gz"); auto zi = new ZlibInput(fi); Cout.stream.copy (zi); I could read the file with gzgets. Any help is welcome. thanks. /l

Author	Message
larsivi Joined: 03/27/04 Posts: 614	Posted: 01/27/09 17:36:26 Is Gzip really compatible with Zip (zlib)? If so, then I think you uses it wrongly. ZlibStream is for raw zipped data, use Zip or the VFS ZipFolder to open files.
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 01/27/09 23:43:47 I don't think, but I'm not sure, that zip is compatible with gz. The think I was wondering was that ZlibStream? uses libz, which is also used to open gz files. As far as I understand ZlibStream? uses the inflate function from libz, which support streams. This is exactly what I would like to do. I would like to stream a compressed stream from a gz file to ZlibStream?, decompress it and output it either on a standard out or a file or a socket. I could open and read the gz file with the following code: GrowBuffer readFile(char[] filepath) { auto gzipf = gzopen(toCString(filepath), toCString("rb")); auto buffer = new GrowBuffer; if (gzipf != null) { char[1024] tmp; while (gzeof(gzipf) != 1) { buffer.append(toDString(gzgets(gzipf, toCString(tmp), 1024))); } } gzclose(gzipf); return buffer; } Cout.stream.copy(readFile('somefile.gz')); This way the buffer will grow to the complete size of the uncompressed file. I would rather create a stream like interface for reading gz files, just as it is described in inflate and used in ZlibStream?. The function should read as long as everything is read from the input stream, while providing a buffer that holds the uncompressed data and which can be read from any other stream unless it is empty. I though ZlibInput? would provide just that. Any hints how to get there would be great? /lars
larsivi Joined: 03/27/04 Posts: 614	Posted: 01/28/09 09:41:44 Are you able to see if there is any header data in file? Data that shouldn't be part of the stream itself?
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 01/28/09 15:40:48 That might be the problem. The file contains only the raw compressed data. we created the files with php and I want to read it right now. So there is no header data in the file. That might be the reason why Zlibstream or zlib's inflate doesn't work as no header is available. Is there a way to tell zlibstream the header?
larsivi Joined: 03/27/04 Posts: 614	Posted: 01/28/09 16:13:43 Sorry, I think you misunderstood. What I mean is that zip files have a lot of header baggage that isn't needed for the zlibstream - whether there still needs to be headers of some kind I don't know but I thought not. But you say that it works with gzopen and that particular function isn't used in ZlibStream leading me to believe that gzip support needs some additional support to be supported? I would appreciate it a lot if you could investigate / write patches :)
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 01/29/09 11:24:55 OK, I will further investigate into the inflate function implementation of zlibstream and try getting it to work. /lars
tomni Joined: 08/25/08 Posts: 30	Posted: 02/06/09 12:20:02 The solution to your problem is: "You can give inflateInit2() a windowBits parameter of 31 or 47 to decode the gzip format or to automatically detect and decode the zlib or gzip format, respectively." See this post http://www.groupsrv.com/science/about361350.html Would be good if this could find its way into the ZLibStream documentation in the code itself.
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 08/15/09 11:01:54 -- Modified: 08/15/09 11:05:23 by lars_kirchhoff -- Modified 2 Times Thanks to the bug fix, gzip is now very well supported. But I have a question how to set another encoding, other then auto detect. There is a constructor that allows setting the encoding: this(InputStream stream, Encoding encoding, int windowBits = WINDOWBITS_DEFAULT) but the problem is that the enum (Encoding) is only declared within the class. How do I access it or how do I call the constructor, because this is not working: auto fp = new FileInput("file.gz"); auto zlib = new ZlibStream(fp, Encoding.None); I was thinking of moving the enum out of the class, but maybe there is a better solution. thanks a lot.. /lars
larsivi Joined: 03/27/04 Posts: 614	Posted: 08/15/09 11:10:58 auto zlib = new ZlibStream(fp, ZlibStream.Encoding.None);
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 08/15/09 11:43:29 -- Modified: 08/15/09 11:43:53 by lars_kirchhoff Thanks larsivi, that is what I tried before, but I got an error "undefined identifier ZlibStream?", while compiling, but your example showed me the problem ;). It should be auto zlib = new ZlibStream(fp, ZlibInput.Encoding.None);