Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

Layout & co, cost of variadic templates

Moderators: kris

Posted: 05/28/08 11:47:05

* Uninitialized Stack Arrays

it was recently shown how bad the handling of big arrays can be due to inexact gc in D, in this light the uninitialized arrays used in tango.text.convert.Layout.convert (void*[64] always, with gdc worse) are particularly bad

* gdc Vararg Mess

gdc handles varargs badly, and in a way that is difficult to fix, tha actual implementation does not treate all structures correctly, and structures bigger than 8 bytes (16 on AMD64) cannot be used. Also in dmd I wonder if alignment handling is always correct in the present code.

* Variadic Template Solution

A possible solution to these problems is to use variadic templates, but (at least initially) just to set up the generic call instead of using varargs.

I did a quick and dirty variadic implementation of Layout.The patch is at http://homepage.mac.com/fmohamed/var/variadic.patch.txt . All test are still passing, but I did not spend big amounts of time in polishing it, because as said I think that the API should be streamlined more, and I did not want to do non useful work. This should be see as a way to assess the cost of variadic templates and assess if a future implementation using them would be viable. I did this after having tested a small program and having seen that used well variadic templates do not bring too much bloat in exe sizes wrt. vararg. So how much bloat do they bring? To test it I looked at the unit tests of tango with gdc and normal compilation (-g) with dsss, the exe size with the normal implementation, with variadic templates and the difference absolute and in % (both for non stripped and stripped executable):

exe_size template_exe diff diff_in% ------- stripped ----- -----
test_DG-tango-core 1438080 1437752 -328 -0.0% 540348 540348 0 0.0%
test_DG-tango-group 3173228 3374588 201360 6.3% 1090908 1167864 76956 7.1%
test_DG-tango-io 2270680 2389380 118700 5.2% 680084 728996 48912 7.2%
test_DG-tango-math 864724 864472 -252 -0.0% 338808 338808 0 0.0%
test_DG-tango-net 3701736 3918076 216340 5.8% 1138996 1221376 82380 7.2%
test_DG-tango-stdc 564568 564196 -372 -0.1% 156412 156412 0 0.0%
test_DG-tango-sys 891016 890712 -304 -0.0% 300620 300620 0 0.0%
test_DG-tango-sys-darwin 471580 471400 -180 -0.0% 138988 138988 0 0.0%
test_DG-tango-text 2455860 2662340 206480 8.4% 1129196 1208972 79776 7.1%
test_DG-tango-text-locale 1435044 1553608 118564 8.3% 605296 652972 47676 7.9%
test_DG-tango-time 755720 755456 -264 -0.0% 243272 243272 0 0.0%
test_DG-tango-util 1639552 1747460 107908 6.6% 551748 599248 47500 8.6%

the same for the optimized version (-g -O -release)

exe_size template_exe diff diff_in% ------ stripped ----------
test_DG-tango-core 1338116 1337784 -332 -0.0% 449324 449324 0 0.0%
test_DG-tango-group 2170044 2265396 95352 4.4% 561300 589732 28432 5.1%
test_DG-tango-io 812520 812268 -252 -0.0% 289656 289656 0 0.0%

so I think that it is kind of acceptable (<10%), at least to me, and one can think about a variadic template solution (maybe going even further and using templates also to convert the arguments, and typeinfo only as fallback solution). Variadic templates can be used also keeping the current interface unchanged (as I did with opCall), but the code is ugly, and I realized it later. So in the implementation I got rid of the overloading between the convert using a sink and the one to a string. Actually the change I did goes toward adressing another rub I have with the present interface.

* Excessive Overloading

Personally I think that when different functions are used depending on the order of the arguments, it would be better to use two different names, this especially when these functions call each other/themselves and can have a variable number of arguments (so that to be told apart one has to remember that putting typeinfo first to call one function, but last to call the other one...). In layout this case arises often, and all these function are public. I think that most of these convolutions are due to historical reasons connected with the (buggy) implementation of vararg in gdc. Anyway I think one should get rid of them. For example vprint cannot always accept an array of Typeinfo as argument to replace {} because it might match the other overloaded function.

* Sink

The variant using Sink (that are actually delegates adding a string) is clearly the most basic one, and the text one should be explicitly constructed on the top of it. I understand that one wants to use the call chaining mechanism with Layout, but normally a user uses either the sink based interface or the string returning variant, so I thinkt that it is ok to create a different object at the beginning for the string returning version. Actually one could even think about making the Sink an alias template parameter, as the calling them can be much faster. Probably IO is so slow that it does not matter, but still worth a thought.

* Print!(T)

As we are at general API improvements I want to have a way to declare functions that accept a generic easy to use "output stream". This output stream should be Stdout or a file, or a socket, an in memory string builder... As this is not necessarily for serialization the correct choice is (I think) Print!(T) (not Writer...). This works well, but if I have an object that has a member function

Print!(T) desc(T)(Print!(T)) {...}

then obj.desc(Stdout("obj:")).newline; looks strange It would be better to be able to write Stdout("obj:")(&obj.desc).newline; This would mean that Print should be able to accept delegates Print!(T) delegate(Print!(T)) (or even void delegate(Print!(T))). It is clear that when outputting large amounts of data such a function is clearly superior than one returning a string, and if the need arises one can transform it easily to a string using a special stream. Maybe there is already something to achieve this, but I didn't see it

* Template-Fu

I find myself always looking for some simple template functions (especially related to arrays). Variant defines many of them, but only as private functions. I think that makeing then public in some module would be nice. I have actually a "TemplateFu?" module with those functions + ctfe_a2i for myself, and I find it very useful.

There are no responses to display.