View previous topic :: View next topic |
Author |
Message |
bobef
Joined: 05 Jun 2005 Posts: 269
|
Posted: Tue Aug 21, 2007 2:43 pm Post subject: Profile |
|
|
I did a little profile on the following simple script:
Code: |
module test;
for( local i=0;i<100;i++ )
{
foreach(k,v;_SERVER) {write("_SERVER["~k~"]="~v~"<br />");}
foreach(k,v;_COOKIE) {write("_COOKIE["~k~"]="~v~"<br />");}
}
writef(`some <html> stuff here`); |
_SERVER and _COOKIE are tables. _SERVER has around 20 strings in it and cookie only 2 or 3.
Code: | ======== Timer Is 3579545 Ticks/Sec, Times are in Microsecs ========
Num Tree Func Per
Calls Time Time Call
42232 646294 346972 8 MDValue.as
3500 1011007 253261 72 MDString.concat
475328 245369 245369 0 MDValue.invariant
42232 230058 198768 4 MDValue.canCastTo
45752 245204 192066 4 MDValue.isString |
These are some very troubling numbers. 475328 calls for this simple loop!
And why concat is called 3500 times? I see only 100*8=800 concats in this loop? This program takes 12 seconds to execute and the rest of the code without MiniD take few millisecs. When I comment the invariant function it takes 7 seconds (the profiler's time is included here too). Of course the profiler slows things down a lot, but still... Just look at the numbers...
I am willing to participate in MiniD's optimization if you give me some guidelines... For example this invariant thing could be versioned. There could be some version for lazy people who don't need speed. And other version that goes with a warning in the docs "don't change the value of MDValue.nullValue" instead of wasting 245 milliseconds on a simple loop
Sorry, but I really need speed. MiniD is very easy to extend and looks like D, but if it is not designed for high performance applications I must look somewhere else... I'm going to use D + some scripting language to replace PHP in a web app and every millisecond matters when we are talking about high load server... |
|
Back to top |
|
|
bobef
Joined: 05 Jun 2005 Posts: 269
|
Posted: Tue Aug 21, 2007 3:35 pm Post subject: |
|
|
And I forgot to mention... for(i:0..100) fails (with or w/o spaces). It throws some error. I think "CatExp codeGen not collapsed". Not sure but this is in my server log. |
|
Back to top |
|
|
doob
Joined: 06 Jan 2007 Posts: 367
|
Posted: Tue Aug 21, 2007 4:36 pm Post subject: |
|
|
I've also been looking into using minid as php replacement, it would be interesting to hear how it goes for you. |
|
Back to top |
|
|
JarrettBillingsley
Joined: 20 Jun 2006 Posts: 457 Location: Pennsylvania!
|
Posted: Tue Aug 21, 2007 5:37 pm Post subject: |
|
|
Quote: | And why concat is called 3500 times? I see only 100*8=800 concats in this loop? |
Two things:
One, when you have a list of concatenations, like "a ~ b ~ c ~ d", this is translated into a call like "MDString.concat(a, b, c, d)". So there's only one cat happening for each of those write() calls.
Two, you forgot to multiply by the inner loop iterations. But even then, I'm wondering how many items you really have in _SERVER and _COOKIE.
Oh, and I guess a third: it'll be faster to do "write("blah", k, " foo", v)" than to do all that unnecessary concatenation.
Quote: | When I comment the invariant function it takes 7 seconds (the profiler's time is included here too). ... For example this invariant thing could be versioned. |
Invariants go away in release mode, though. Are you sure you're compiling with -release? MiniD has an awful lot of debugging checks which really slow it down in debug mode (or I should say, "non-release" mode, as when you compile without -debug or -release, it'll still include contracts like the invariant there). Throw in a -O and/or -inline for good measure. (though beware, sometimes -O and -inline cause odd bugs. use at your own risk.)
Quote: | And I forgot to mention... for(i:0..100) fails (with or w/o spaces). It throws some error. I think "CatExp codeGen not collapsed". Not sure but this is in my server log. |
AHA! You've found a bug. I've fixed it, but if you want the quick patch, go to minid.compiler.NumericForStatement.fold, and insert the line "mBody = mBody.fold();" right before the return. |
|
Back to top |
|
|
bobef
Joined: 05 Jun 2005 Posts: 269
|
Posted: Wed Aug 22, 2007 12:25 am Post subject: |
|
|
doob wrote: | I've also been looking into using minid as php replacement, it would be interesting to hear how it goes for you. |
Well it goes as breeze
I've built a cgi class based on fastcgi4d that handles post/get/cookie/escape/unescape/sessions so it works in both cgi/fastcgi mode. I thought of using wombat but it is too bloated and is not portable. Many things are hardcoded (including stdin) and it doesn't even unescape _GET... and my class is just one file. I've also built a simple class that parses html <? minid ?> html files and it works great. I just need to add some basic web-oriented functionality to the scripts and thats it. But I am worried about minid's performance. The above example takes 50+ milliseconds to run (thats with -O -inline) and more than 50 of these are eaten by minid. That's way too much for this simple page. I realize that it prints a lot of data (like 1mb) to the console which is slow. I could remove concats but in real app there will be lots of them anyway. Also I replace "write" with "local g=" and it still takes at least 30ms... So I am not yet sure if I am going to use MiniD. There are not many alternatives though. DMDScript is hell to extend and it requires a license. Walnut is not ready. The rest are C wrappers. But still if C is working faster why not... Two garbage collectors doesn't sound like the fastest thing but who knows... |
|
Back to top |
|
|
JarrettBillingsley
Joined: 20 Jun 2006 Posts: 457 Location: Pennsylvania!
|
Posted: Wed Aug 22, 2007 5:53 am Post subject: |
|
|
Quote: | I could remove concats but in real app there will be lots of them anyway. |
Always keep in mind, though, that string concatenation will always create new objects, and will always, because of that, be fairly slow. If you need to build up a lot of strings piecewise without sending them to the output bit by bit, look into the StringBuffer class in the baselib. It's basically a mutable string type which you can reserve space for to keep heap allocations to a minimum. You can even reuse the same string buffer over and over in order to avoid memory allocations. |
|
Back to top |
|
|
JarrettBillingsley
Joined: 20 Jun 2006 Posts: 457 Location: Pennsylvania!
|
Posted: Wed Sep 12, 2007 2:11 pm Post subject: |
|
|
Just so you know, I reproduced your example, and compiling with -release halves the execution time. Using a StringBuffer to build up the output HTML instead of concatenation also provides a nice speed boost.
Also, for what it's worth, MiniD isn't as fast as, say, Lua, but it consistently outperforms the common implementations of Ruby and PHP in benchmarks, both of which are very commonly used for server-side scripting. It might not be phenomenally fast but it might just be fast enough. |
|
Back to top |
|
|
JarrettBillingsley
Joined: 20 Jun 2006 Posts: 457 Location: Pennsylvania!
|
Posted: Tue Sep 25, 2007 8:31 pm Post subject: |
|
|
A few more things: under what circumstances did you perform this test? Was the output going to the console, or was it redirected to a file? Did you redirect the file from the console (like running it as "mdcl test > someFile") or programmatically, by reassigning Tango's Stdout? Or maybe you redirected Stdout to some kind of buffer, like in a real web server scenario? All of these have a HUGE impact on the execution speed of this code. I can get your exact code to run in anywhere from 9ms to 130ms based on how I redirect the output. Interesting. |
|
Back to top |
|
|
bobef
Joined: 05 Jun 2005 Posts: 269
|
Posted: Wed Sep 26, 2007 1:34 am Post subject: |
|
|
I am not sure it was like one month ago when I tested. I believe I redirected the output to file from the prompt like this "test.exe > a.txt" |
|
Back to top |
|
|
|