D oriented System library

xammy Joined: 02/01/07 Posts: 23

Posted: 03/08/07 12:06:24 There are some things, I don't like, when D/Tango just wraps the libc. First is the memory, as D is garbage collected, C is not, you must be very careful when interacting with both. The getcwd memory leak showed this, as getcwd allocated memory, you must free. Second are zero terminated strings... horrible thing, when you want to pass a slice of your favourite string to the kernel, as you cannot put a \0 after the end (as it's a slice, there might be something useful) and at least the linux kernel does not depend on zero terminated strings in IO, as you always pass the length of your buffer as a parameter (which has to be calculated via strlen() by the libc, although your D program exactly knows the length...) What I'd like to see, is a D-optimized kernel interface. I already started a litte C library, but glibc-code is nearly unreadable ;-) But for example, implementing linux syscalls is very easy! As one of the most important things of D is, that it's C-compatible, this system library must be written in C and must do all those things, the POSIX people want it to do (at least!). But the design would be completely different, as it's D oriented, so it defers the zero-terminated-string stuff to the C part and let's D access the kernel as fast as possible. Another idea is to make the complete memory management (also of the C part) garbage collected. (If we're already rewriting everything, why not this way <g> ) Or make it at least modular enough to test both variants. Maybe this should not be discussed here, but as the average D programmer just uses Tango and is not interested, how it's done inside, a D oriented system library is most interesting for the Tango team. Any ideas, recommendations?

Author	Message
larsivi Joined: 03/27/04 Posts: 614	Posted: 03/08/07 12:40:14 You can write it in D, but let the functions have extern (C) calling convention? Otherwise it's obvious that it would be nice if the whole chain (even the OS!) could be written in D, and I think Tango can make this easier than it will be with at least the current Phobos.
xammy Joined: 02/01/07 Posts: 23	Posted: 03/08/07 14:53:49 At least, all the algorithmical stuff can be written in D in extern "C" environment. I believe, there are some part's, where you have to handle some executable-format-specific things like hidden symbols, etc. I don't know, whether D has that much options (in gcc all those attribute (...) stuff) But this shouldn't be more than startup/exit-code, the more interesting part might contain much of D's syntactic sugar. Although I'm not sure, whether one should use the version(){} syntax for all those porting things. It's more readable than #define, but you have to consider architecture, OS and maybe some other things, so this might be unreadable, too. But this is something, one might think about, when the basic stuff is written :)
xammy Joined: 02/01/07 Posts: 23	Posted: 03/15/07 18:54:48 It seems, one cannot write it in D, because the compiler inserts some module references, which are not resolved, if you link it with a C-only program. So we have some opportunities: 1) Rewrite the whole libc but design it, that one can use both, C and D in an efficient way. Everything would be GC'ed and one would have to rewrite thread stuff, too. This is the most ugly one, as we reinvent the wheel and cannot use D-language features like version(), nice inline assembly, etc. 2) Only implement kernel-interface, use thread stuff from standard libc, but use GC'ed memory for itself. (maybe include the GC in the library or use that of Tango/Phobos). Everything will be written in D and we can enhance it with thread stuff, if D is ready to, not need c libraries, any more. The only problem here is, that you have different I/O models for your C libraries and your D programs. 3) Write a D-only system library. We can code everything in D but one cannot link C code against the library and this would drop C-library support completely, because when the D program uses our threads and a C library uses libc and their threads, this won't be nice :-) 4) We might ask the glibc coders to include an interface for languages, which don't use zero-terminated strings, although this does not seem realistic to me, as they would have to change much of their I/O implementation for this to work. I would prefer the 2nd idea, as it does not drop C support and one can use D to write the library itself.
sean Joined: 06/24/04 Posts: 109	Posted: 03/18/07 00:50:50 -- Modified: 03/18/07 00:53:17 by sean D does not have to be garbage collected if that's a feature you aren't interested in. Tango provides an allocator that uses malloc, which can be linked instead of the default garbage collector. Ultimately though, if you want to provide a stripped-down systems library you will either have to alter the compiler runtime or require the existence of some other libraries: the standard C library, plus either Posix or Win32 libraries depending on your target system. There are not a large number of interaction points with external libraries (libc uses is limited to memset and a rare few others in Tango, for example), but they do exist. Oh, the malloc-based allocator is in lib/gc/stub. If you use this you'll have to be careful about string management, because built-in string operations rely on the GC to clean up after them. So adjusting length via the length property, concatenation, etc, all risk leaking memory.
xammy Joined: 02/01/07 Posts: 23	Posted: 03/19/07 13:45:50 I think, it's not the problem (or one of those, I described), that D is garbage collected. The problem is, that libc is not GC'ed ;)