[Lynn]

<alert comment="newbie">

I'm puzzled on the appropriate way to link a D program to a library made up of C code (such as mysql, sqlite, wxwindows, zlib, 7-zip, etc.) The documentation has several references that this is possible, but I haven't come across an actual example.

In the trivial sample code below, I can get it to work if I compile the C code with the DMC compiler and use the Digital Mars librarian.

But in the general (and probably more typical) case for real world software packages, that doesn't seem like it would be a practical approach. I wouldn't think that makefiles for DMC would typically be available, even if the source code was available.

I think I understand about making a D wrapper module for the function definitions, but eventually doesn't the D main program have to link with something that the DigitalMars linker/librarian can comprehend? I mostly use Visual C++ 6.0, and its .lib output doesn't seem acceptable to dmd or lib.

[JJR]

Hello Lynn,

From my experience, it is pure frustration using foreign libs with both dmc and dmd. This has been discussed many times, and you'll hear of tools such as coff2omf and implib often. The problem usually arises when you mush use a 3rd party library for which you've only been provided with the library archive and the headers. As you mention, often the library itself is compiled with some system other then dmc (ie. MS Visual C/C++). Well, the old remedy was to use coff2omf to convert the library to the format dmc could understand. But then MS came out with an updated coff format, so then you FIRST were supposed to track down a MS linker utility that supports the /convert option, use it on the lib file, then call coff2omf, then use it with dmc. Complicated? Yep. In fact that linker tool seems to be very hard to find. These days, it appears to be nigh to impossible to convert the old omf format reliably.... at least I've had little success with the utility myself.

So that leaves you with two choices that I can think of:

(1) Hope the library project you need to use is open source so that you can recompile it into a library with dmc. Then you can be assured that dmd will link with it appropriately. This is problematic if the source has not been stream-lined for dmc. You have more work cut out for you because you must prep the source/makefiles yourself.

(2) Hope the library, if not open source, is at least distributed as a dll. If this is so, you have a chance to create a compatible import lib that you can use to link with your dmd project. This in itself is a major pain also, but perhaps less so then option #1. In this instance, though, dmc still makes things tricky to work with sometimes. For example, the implib you create from the dll is immediately useless until you use a whole wack of tools to set up a proper module definition file (*.def) for the library. This is usually necessary for dll files that will have a __stdcall calling convention peculiar to windows because dmc has the habit of appending @## to all function symbols in the library (where ## is the bytes defining the functions parameter list; so the game here is to figure the ## for each function in the header file and insert that into the *.def file). If the __stdcall is not used in the dll, (a fact usually discoverable by the headers definition of WINAPI for each function declaration), then this probably isn't an issue and the implib will be immediatly useful. This would save a ton of work. In the case of those libraries that require the @## values, Jan Knepper provides a modified implib that supposed to do all the work of determing the *.def @## values for you. If it worked, this would make option #2 the preferred method in all cases. But I've failed to get it to work even once and very little instruction is availabe for it use.

As it stands, I've had to resort to using tools from Borland's toolset (implib, impdef) mixed with dmc's (implib, lib). I've also had to hand modify the *.def files to get things finally working appropriately with dmc. I may sometimes also resort to open watcom and gcc tools when necessary.

Really this is probably the most significant weakness of the digitalmars compiler system. They interface very poorly with outside formats. If you are developing utterly and completely with the dmc and dmd toolset, you're fine. But any mixing is a major headache.

Well, I've droned on long enough.... Perhaps someone can give you a little more helpful details.

Later,

John

PS. Much of my interest in this topic got resparked when, just recently, I worked many hours to get dmc to link with opengl32.lib and glu32.lib. Both libraries provided with the cd version digitalmars dmc were incomplete (ie. missing functions). I had to go through a fairly lengthy process to grab the win XP system versions of the dll and convert them to a linkable solution for dmc. I did it, but it was a painful experience. Nonetheless, it was rewarding to finally get it working.

[Lynn]

Hi John,

Thanks for the very informative "drone". I looked in the archives for how to link with foreign libs, but apparently didn't search well enough. Sigh.

I've noticed posts about re-writing a tool/utility in D (SqLite, wxWindows, zlib, mysql, 7-zip, etc.) and that seems like a marginal idea. It's hard to see how it would keep up with updates to the base tool/utility that was written in C. Q/A would be a problem, as well as performance.

For example, I was trying to use the D version of zlib and found a show-stopper bug that indicated to me that I was one of the first persons to ever use zlib.d ... decompress didn't work for anything beyond a trivial test case. That was zlib.d based on 1.1.4 ... so somebody tackled zlib 1.2.1 (Walter?). It seems to work correctly now, but also seems much slower that the C version. That seems like a pretty good example of the situation that we face.

Perhaps the most helpful effort would be dmc makefiles that were kept up to date for the actual C code ... which sounds like a tedious, thankless task with low prospects of qualified volunteers to wrestle with knarly makefiles.

[JJR]

[JJR]

FYI, I was able to compile a static library for the 1.2.1 release of the zlib source using dmc. The example included with the library compiled and linked successfully also. A dll version would require a bit more work (and more familiarity with dmc). I didn't make a dmc Makefile for the project, but doing so for zlib would be fairly simple. dmd should have no trouble linking with such a library.

Later,

John Reimer

[Lynn]

I'd appreciate getting a copy of a dmc prepared zlib.lib. I was considering tackling that task as it seemed doable, but would be glad to benefit from your expertise. I almost think there was a makefile.sc for the semantec compiler/maker that is more or less compatible with dmc (they share a common history?)

Zlib seems stable/mature enough so that a port isn't completely out of the question, but that doesn't seem like a particularly good use of time to convert the code of something that works well. Except as a learning exercise as you note.

The only calls I'm interested in for the D app I'm contemplating are gzopen, gzclose, gzread, compress, and uncompress, so the number of extern (C) statements isn't too bad. Plus there are the structs for gzfile and perhaps some #defines.

The D version of gzread seemed quite a bit slower than the C version.

I was taking a look at wzWindows.lib, but my emerging impression is that it is C++ and thus unusable by D unless converted. Sigh.

[JJR]

[sean]

[quote="Lynn"]I'd appreciate getting a copy of a dmc prepared zlib.lib. I was considering tackling that task as it seemed doable, but would be glad to benefit from your expertise.[quote]
One ships with the D compiler. It's in /src/phobos/etc/c/zlib.

[JJR]

[sean]

[JJR]

[sean]

Oops! I must have missed that one in my updating.

[Workaphobia]

I went through an extraordinary amount of trouble trying to get opengl to work with DMC (not even D) a few months ago. Eventually I was able to use a .def file to generate an import library. I don't even fully understand the library formats and their relation to .dlls and .defs, I just followed a sample. In fact, I believe it was you, JRR, who helped me with that major headache.

I wouldn't recommend asking for help about this sort of thing on the newsgroup. The only response I got, after posting a lengthy explanation of the problem and all the different trains of thought I pursued, was a shameless plug for the DMC CD by Walter himself. But then again, that was the DMC group, which has one hundredth of the activity of the D group.
_________________
"Nifty News Fifty: When news breaks, we give you the pieces."

[JJR]

[jcc7]