FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Profile

 
Post new topic   Reply to topic     Forum Index -> MiniD
View previous topic :: View next topic  
Author Message
bobef



Joined: 05 Jun 2005
Posts: 269

PostPosted: Tue Aug 21, 2007 2:43 pm    Post subject: Profile Reply with quote

I did a little profile on the following simple script:
Code:

module test;

for( local i=0;i<100;i++ )
{
foreach(k,v;_SERVER) {write("_SERVER["~k~"]="~v~"<br />");}
foreach(k,v;_COOKIE) {write("_COOKIE["~k~"]="~v~"<br />");}
}

writef(`some &lt;html&gt; stuff here`);


_SERVER and _COOKIE are tables. _SERVER has around 20 strings in it and cookie only 2 or 3.

Code:
======== Timer Is 3579545 Ticks/Sec, Times are in Microsecs ========

  Num          Tree        Func        Per
  Calls        Time        Time        Call

  42232      646294      346972           8     MDValue.as
   3500     1011007      253261          72     MDString.concat
 475328      245369      245369           0     MDValue.invariant
  42232      230058      198768           4     MDValue.canCastTo
  45752      245204      192066           4     MDValue.isString


These are some very troubling numbers. 475328 calls for this simple loop!
And why concat is called 3500 times? I see only 100*8=800 concats in this loop? This program takes 12 seconds to execute and the rest of the code without MiniD take few millisecs. When I comment the invariant function it takes 7 seconds (the profiler's time is included here too). Of course the profiler slows things down a lot, but still... Just look at the numbers...
I am willing to participate in MiniD's optimization if you give me some guidelines... For example this invariant thing could be versioned. There could be some version for lazy people who don't need speed. And other version that goes with a warning in the docs "don't change the value of MDValue.nullValue" instead of wasting 245 milliseconds on a simple loop Wink
Sorry, but I really need speed. MiniD is very easy to extend and looks like D, but if it is not designed for high performance applications I must look somewhere else... I'm going to use D + some scripting language to replace PHP in a web app and every millisecond matters when we are talking about high load server...
Back to top
View user's profile Send private message
bobef



Joined: 05 Jun 2005
Posts: 269

PostPosted: Tue Aug 21, 2007 3:35 pm    Post subject: Reply with quote

And I forgot to mention... for(i:0..100) fails (with or w/o spaces). It throws some error. I think "CatExp codeGen not collapsed". Not sure but this is in my server log.
Back to top
View user's profile Send private message
doob



Joined: 06 Jan 2007
Posts: 367

PostPosted: Tue Aug 21, 2007 4:36 pm    Post subject: Reply with quote

I've also been looking into using minid as php replacement, it would be interesting to hear how it goes for you.
Back to top
View user's profile Send private message
JarrettBillingsley



Joined: 20 Jun 2006
Posts: 457
Location: Pennsylvania!

PostPosted: Tue Aug 21, 2007 5:37 pm    Post subject: Reply with quote

Quote:
And why concat is called 3500 times? I see only 100*8=800 concats in this loop?


Two things:

One, when you have a list of concatenations, like "a ~ b ~ c ~ d", this is translated into a call like "MDString.concat(a, b, c, d)". So there's only one cat happening for each of those write() calls.

Two, you forgot to multiply by the inner loop iterations. But even then, I'm wondering how many items you really have in _SERVER and _COOKIE.

Oh, and I guess a third: it'll be faster to do "write("blah", k, " foo", v)" than to do all that unnecessary concatenation.

Quote:
When I comment the invariant function it takes 7 seconds (the profiler's time is included here too). ... For example this invariant thing could be versioned.


Invariants go away in release mode, though. Are you sure you're compiling with -release? MiniD has an awful lot of debugging checks which really slow it down in debug mode (or I should say, "non-release" mode, as when you compile without -debug or -release, it'll still include contracts like the invariant there). Throw in a -O and/or -inline for good measure. (though beware, sometimes -O and -inline cause odd bugs. use at your own risk.)

Quote:
And I forgot to mention... for(i:0..100) fails (with or w/o spaces). It throws some error. I think "CatExp codeGen not collapsed". Not sure but this is in my server log.


AHA! You've found a bug. I've fixed it, but if you want the quick patch, go to minid.compiler.NumericForStatement.fold, and insert the line "mBody = mBody.fold();" right before the return. Smile
Back to top
View user's profile Send private message
bobef



Joined: 05 Jun 2005
Posts: 269

PostPosted: Wed Aug 22, 2007 12:25 am    Post subject: Reply with quote

doob wrote:
I've also been looking into using minid as php replacement, it would be interesting to hear how it goes for you.


Well it goes as breeze Wink
I've built a cgi class based on fastcgi4d that handles post/get/cookie/escape/unescape/sessions so it works in both cgi/fastcgi mode. I thought of using wombat but it is too bloated and is not portable. Many things are hardcoded (including stdin) and it doesn't even unescape _GET... and my class is just one file. I've also built a simple class that parses html <? minid ?> html files and it works great. I just need to add some basic web-oriented functionality to the scripts and thats it. But I am worried about minid's performance. The above example takes 50+ milliseconds to run (thats with -O -inline) and more than 50 of these are eaten by minid. That's way too much for this simple page. I realize that it prints a lot of data (like 1mb) to the console which is slow. I could remove concats but in real app there will be lots of them anyway. Also I replace "write" with "local g=" and it still takes at least 30ms... So I am not yet sure if I am going to use MiniD. There are not many alternatives though. DMDScript is hell to extend and it requires a license. Walnut is not ready. The rest are C wrappers. But still if C is working faster why not... Two garbage collectors doesn't sound like the fastest thing but who knows...
Back to top
View user's profile Send private message
JarrettBillingsley



Joined: 20 Jun 2006
Posts: 457
Location: Pennsylvania!

PostPosted: Wed Aug 22, 2007 5:53 am    Post subject: Reply with quote

Quote:
I could remove concats but in real app there will be lots of them anyway.


Always keep in mind, though, that string concatenation will always create new objects, and will always, because of that, be fairly slow. If you need to build up a lot of strings piecewise without sending them to the output bit by bit, look into the StringBuffer class in the baselib. It's basically a mutable string type which you can reserve space for to keep heap allocations to a minimum. You can even reuse the same string buffer over and over in order to avoid memory allocations.
Back to top
View user's profile Send private message
JarrettBillingsley



Joined: 20 Jun 2006
Posts: 457
Location: Pennsylvania!

PostPosted: Wed Sep 12, 2007 2:11 pm    Post subject: Reply with quote

Just so you know, I reproduced your example, and compiling with -release halves the execution time. Using a StringBuffer to build up the output HTML instead of concatenation also provides a nice speed boost.

Also, for what it's worth, MiniD isn't as fast as, say, Lua, but it consistently outperforms the common implementations of Ruby and PHP in benchmarks, both of which are very commonly used for server-side scripting. It might not be phenomenally fast but it might just be fast enough.
Back to top
View user's profile Send private message
JarrettBillingsley



Joined: 20 Jun 2006
Posts: 457
Location: Pennsylvania!

PostPosted: Tue Sep 25, 2007 8:31 pm    Post subject: Reply with quote

A few more things: under what circumstances did you perform this test? Was the output going to the console, or was it redirected to a file? Did you redirect the file from the console (like running it as "mdcl test > someFile") or programmatically, by reassigning Tango's Stdout? Or maybe you redirected Stdout to some kind of buffer, like in a real web server scenario? All of these have a HUGE impact on the execution speed of this code. I can get your exact code to run in anywhere from 9ms to 130ms based on how I redirect the output. Interesting.
Back to top
View user's profile Send private message
bobef



Joined: 05 Jun 2005
Posts: 269

PostPosted: Wed Sep 26, 2007 1:34 am    Post subject: Reply with quote

I am not sure it was like one month ago when I tested. I believe I redirected the output to file from the prompt like this "test.exe > a.txt"
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> MiniD All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group