FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Application-Level objects

 
Post new topic   Reply to topic     Forum Index -> DSP
View previous topic :: View next topic  
Author Message
brad
Site Admin


Joined: 22 Feb 2004
Posts: 490
Location: Atlanta, GA USA

PostPosted: Fri Jun 18, 2004 11:25 pm    Post subject: Application-Level objects Reply with quote

In the Welcome thread, you mention session contexts and the like. I am only familiar with JSP, but I'm wondering if we could take the best ideas from the other technologies listed in that post (ASP, JSP, Cold Fusion, Clearsilver) and incorporate them. Response and Request would be there, right?

I'm not a good source of knowledge on those technologies, but one that I can think of is a user context, rather than, or in addition to session context. Maybe it already exists in all of them, but it would be fairly useful in an app at work, as well as to run dsource.org. Or should this just be implemented in your application that uses DSP?

Anyone know of other contexts that we (pragma) should think about implementing?
_________________
I really like the vest!
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Jun 21, 2004 7:57 am    Post subject: Re: Application-Level objects Reply with quote

brad wrote:
In the Welcome thread, you mention session contexts and the like. I am only familiar with JSP, but I'm wondering if we could take the best ideas from the other technologies listed in that post (ASP, JSP, Cold Fusion, Clearsilver) and incorporate them. Response and Request would be there, right?


Well first off, "Clearsilver" is a new one by me, so I'll wind up googling that later. Smile

As for Respone/Request, they're a given since I cannot envision a way forward without using that model. (There's obviously a lot of good engineering behind the deisgn of the Java Servlet Architecture).

By "Application Level Objects", I was trying to capture what happens when you want a group of servlets to behave as a single coherent application. Most architectures accomplish this by some form of session management. I was thinking that we could probably use this in DSP by designating one special kind of servlet that is assigned to a group of servlets, probably by directory config. The application object would then be passed, via the request object, to every servlet request under its control.

In most cases, a generic application object would suffice. Provided it has session management, generic database access, and some kind of varible 'buckets' for storing session and application vars, you'd rarely have to subclass it for most applications.

This much is already planned and in a quasi-prototyped state. What concerns me the most at this point are the implications that

Quote:

I'm not a good source of knowledge on those technologies, but one that I can think of is a user context, rather than, or in addition to session context.


Yep, here's an idea of what I have in mind.
I can see configuring an application 'servlet', in a standard way, to generate custom objects based on certain events. So perhaps attaching "new MyUserImplementation()" to the app's "OnNewSession", or "GetSessionObject".

Then any servlet within that application would simply do something like "User usr = cast(User)request.application.GetSessionObject();" and proceed from there.

The more real-world examples that are thrown at this, the stronger it becomes, as it rises to meet real-world expectations of such a technology. I think this is where DSP may come ahead of some other technologies, since they require mountains of code to do accomplish relatively common tasks.

Quote:

Anyone know of other contexts that we (pragma) should think about implementing?


Well "Server", for one, needs to be provided alongside "Application" to the servlet. This is already somewhat covered in my prototype, but will need some serious fleshing out soon. In my mind, that pretty much covers the bases. If we provide a way to subclass request and response from within DSP, then we're pretty much set.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
brad
Site Admin


Joined: 22 Feb 2004
Posts: 490
Location: Atlanta, GA USA

PostPosted: Mon Jun 21, 2004 9:23 am    Post subject: Reply with quote

Any thoughts as to the exposed methods or API's or web services that would be necessary to have a DSP app be able to more effectively use a bunch of servers? (For this discussion, cluster == load balanced server bank).

For instance, our JSP app is served in our production environment by two servers running Tomcat. Our load balancer chooses the least busy server in the cluster and routes all new connections to that server. However, because of our session variables, we have to keep affinity settings so the user that starts a session on Server A stays on Server A. It's where all of her session objects are resident in memory. The issue is when three users come into Server A and start their session at times when Server B is busy. Later on in their session, they all run huge queries. So now, Server A has three huge queries and Server B is twiddling its thumbs.

Is this just poor design on our part? Okay, not poor design, but a choice made when we only had one server that doesn't scale very well? Should we have done session management at the database layer? Have a session server that the two app servers talk to? (Note, I don't like any one single point of failure, like a session server, and I don't even know if that's possible or not).

Also, our metadata is cached into static objects. When this is refreshed on one server, would there be a way to refresh it on all servers in the "cluster?" We only have two servers now, but there will be more later.

The main theme here is: Can we solve issues like this with DSP features, or is this a deeper design issue within apps that need to scale like this? Ideally, you could hit any server for any request even within the same session, and that server would be able to serve the request appropriately.

Does this make sense? Are you all wondering how the hell this guy is running dsource, let alone holding a steady job?

BA
_________________
I really like the vest!
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Jun 21, 2004 10:46 am    Post subject: Reply with quote

brad wrote:
Any thoughts as to the exposed methods or API's or web services that would be necessary to have a DSP app be able to more effectively use a bunch of servers? (For this discussion, cluster == load balanced server bank).

...snip...

The main theme here is: Can we solve issues like this with DSP features, or is this a deeper design issue within apps that need to scale like this? Ideally, you could hit any server for any request even within the same session, and that server would be able to serve the request appropriately.


I've seen the same thing with ASP applications as well. Virtuall all platforms, except Cold Fusion, treat sessions as an exception to the rule rather than a robust feature. Personally, I can't stand the language, but if there's one thing it does right is make these things configurable before you even start writing code.

Sessions *have* to be persisted somewhere neutral so that farming can even approach usefulness. Using a database to persist Session data is the best general-case solution. Good databases can be clustered as well, so you can move horizontally should the DB become a bottleneck.

Add to this a load balancer that shuttles requests to machines on the farm, with idle ones having preference and there you go.

That having been said, DSP should probably do something along the lines of Cold Fusion, and assume that sessions are going to be persisted somewhere: either in a database, object server, or cookies.

Exclamation

BTW, you may have helped to solve the biggest threat to the project yet. Take at look at the DLL thread and tell me if forcing session/application data to persist to storage in-between requests solves a lot of our problems.
Smile

Quote:

Are you all wondering how the hell this guy is running dsource, let alone holding a steady job?


::chuckle:: Not at all. Besides, you *are* running Dsource.org. That's gotta count for something.

And not to worry, I've seen more bad design decisions in production software than I care to count. Sometimes, knowing when *not* to use a computationally optimal solution in favor of making a deadline (provided that it works bug-free) is the measure of a wise developer (IMO).
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
brad
Site Admin


Joined: 22 Feb 2004
Posts: 490
Location: Atlanta, GA USA

PostPosted: Mon Jun 21, 2004 12:41 pm    Post subject: Reply with quote

pragma wrote:

BTW, you may have helped to solve the biggest threat to the project yet. Take at look at the DLL thread and tell me if forcing session/application data to persist to storage in-between requests solves a lot of our problems.
Smile


Hopefully this persistence would be shielded from the developer completely. I'm thinking of either a custom-built file system persistence or using embedded SQLite database or something.

One of the key considerations here is cross-platform abilities. SQLite works here, as well as a custom file system method. I'd lean towards SQLite, myself. Another question on the cross-platform - how do you envision DSP working on Linux? Instead of DLL's, you'd use .so (shared libraries)? Are they as dynamic as is required?
_________________
I really like the vest!
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Jun 21, 2004 1:50 pm    Post subject: Reply with quote

brad wrote:

Hopefully this persistence would be shielded from the developer completely. I'm thinking of either a custom-built file system persistence or using embedded SQLite database or something.


One of the key considerations here is cross-platform abilities. SQLite works here, as well as a custom file system method. I'd lean towards SQLite, myself.


Why not all of the above? Smile

I'm all for using a tool that works well for what it's needed for. However, I don't see much merit in locking into just one type of database, especially seeing as how databases are mostly pluggable from one to the next.

I'd like to see an abstraction here, something like "IPersistentStore" or "IDatabase", under which we'll have concrete implementations for anything you could imagine: flatfiles, OS devices (pipes, sockets), XML-based providers, RPC, sql lite, sql server, postgres, mysql, oracle, etc, etc. And of course this opens the door for any other custom types that this project may leave out.

Obviously, these are going to branch into subprojects, so we'll have to narrow the scope for version 1. I'm all for sticking with SQL Lite for starters.

As a side note, it looks like Mango has some good interfaces for generic serialization.

I completely agree with you that the end-developer shouldn't need to know more than "my session data is safe", and "my session objects must be IPersistable" or somesuch. Smile

The DSP platform should take care of shuttling session data to and from the registered store for that application. The type or location of said store doesn't even have to be the same for all applications on a given server; which is nice given that you may want to use completely different vendors for different apps, or not use a database at all.

Quote:

Another question on the cross-platform - how do you envision DSP working on Linux? Instead of DLL's, you'd use .so (shared libraries)? Are they as dynamic as is required?


Ideally, I want to see DSP working everywhere that D compiles; Windows, Linux, you name it. Furthermore, I don't think anyone's choice of web server should affect your choice of technologies for development. So I feel it should run as a Mango server/extension, Apache plugin and an ISAPI plugin.

As far as I've seen the only major differences between .dll's and .so's are binary. Conceptually, they seem to be identical and should behave the same. I think phobos.loader (Matthew's planned extension for phobos) should come in real handy in assisting the portability of the project.

This leaves the compilation unit and the servlet dll/so stub files as the only "ported" pieces of code in the whole mess. The plugin adaptors for Apache and ISS of course will need to be ported too, but I want to push that way down the timeline for now.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Mon Jun 21, 2004 2:35 pm    Post subject: Reply with quote

brad wrote:
Any thoughts as to the exposed methods or API's or web services that would be necessary to have a DSP app be able to more effectively use a bunch of servers? (For this discussion, cluster == load balanced server bank).

For instance, our JSP app is served in our production environment by two servers running Tomcat. Our load balancer chooses the least busy server in the cluster and routes all new connections to that server. However, because of our session variables, we have to keep affinity settings so the user that starts a session on Server A stays on Server A ...

The whole issue of session management versus performance/scaling is one near and dear to my heart <g>

The easiest, and often most robust, mechanism is to simply persist the entire session to a DB between requests. However, this is only realistic if the traffic volume is "quite low", or you have gobs of money to spend on DB software and hardware.

More typically, one uses sticky-session (aka server affinity) to route related requests to a particular server and maintain state in a semi-persistent manner (high value is always persisted; low value is kept around on that server). This reduces the load on the DB back-end, but runs the risk of session failure if one's server dies a horrible death.

Other options combine these two in various ways and through various layers of servers.

It's all a tradeoff between robustness, content value, and operational cost. I once built a system that scaled to around 16000 requests per second on consumer-grade hardware whilst supporting failover. That's quite a high number, and you can imagine the issues that had to be resolved. Once we hit around 100 requests per second, the DB became the bottleneck. After that it was the LAN. You quickly reach a point where failover support consumes more horsepower than the application. The resolution turned out to be quite simple and elegant, and worked like a charm. Mango will include the appropriate mechanisms to do likewise (which is why it currently has no session support, other than the simple class-serialization & registry framework).

Before one considers getting buried in performance/failover issues, it can be beneficial to change the application container. For example, tests from a few years back showed Tomcat to be the poster-child for abysmal performance. I believe this to be the case today also. If one were to upgrade from Tomcat, I'd recommend considering Jetty, or pay a few dollars and get Resin instead. Resin includes a clustering model that can be used in place of expensive load-balancing hardware (if that were a concern). In simple HTTP tests, Resin was ~7 times faster than Tomcat, and Jetty ~20 times faster.

By the time you outstrip Jetty or Resin in terms of throughput, you'll likely be looking at ways to reduce the persistence load on the DB server.

As to APIs; Mango will expose several different flavours of a Session class. Each flavour will support different levels of the robustness/cost tradeoff, and do the clustering and synchronization behind the scenes. The actual persistence mechanism(s) would be provided by someone like yourself Brad, as a method override or delegate, and would be invoked only as necessary by the Session environment.

- Kris
Back to top
View user's profile Send private message
brad
Site Admin


Joined: 22 Feb 2004
Posts: 490
Location: Atlanta, GA USA

PostPosted: Mon Jun 21, 2004 3:52 pm    Post subject: Reply with quote

kris wrote:

Before one considers getting buried in performance/failover issues, it can be beneficial to change the application container. For example, tests from a few years back showed Tomcat to be the poster-child for abysmal performance. I believe this to be the case today also. If one were to upgrade from Tomcat, I'd recommend considering Jetty, or pay a few dollars and get Resin instead. Resin includes a clustering model that can be used in place of expensive load-balancing hardware (if that were a concern). In simple HTTP tests, Resin was ~7 times faster than Tomcat, and Jetty ~20 times faster.

By the time you outstrip Jetty or Resin in terms of throughput, you'll likely be looking at ways to reduce the persistence load on the DB server.


How do Jetty and Tomcat differ? http://jetty.mortbay.org/jsp-examples/ is from the Jetty website, and they recommend Tomcat. Are you saying that we could replace Catalina, a part of Tomcat, with Jetty? Does Jetty have all of the capability of Tomcat, including JIT compilation?

We didn't see much of an improvement in using an eval version of Resin. Not sure if that just means the bottleneck is somewhere else or what.

kris wrote:

As to APIs; Mango will expose several different flavours of a Session class. Each flavour will support different levels of the robustness/cost tradeoff, and do the clustering and synchronization behind the scenes. The actual persistence mechanism(s) would be provided by someone like yourself Brad, as a method override or delegate, and would be invoked only as necessary by the Session environment.


<drool>
<poker-face>
When do you think there'd be something to play with?
</poker-face>
</drool>
_________________
I really like the vest!
Back to top
View user's profile Send private message
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Mon Jun 21, 2004 5:27 pm    Post subject: Reply with quote

brad wrote:
How do Jetty and Tomcat differ? http://jetty.mortbay.org/jsp-examples/ is from the Jetty website, and they recommend Tomcat. Are you saying that we could replace Catalina, a part of Tomcat, with Jetty? Does Jetty have all of the capability of Tomcat, including JIT compilation?

Those relative numbers were for a simple HTTP request, not JSP oriented requests. AFAIK Jetty and Tomcat both use Jasper as the JSP engine so they should both be equivalent in terms of functionality.

However, if you didn't see any improvement using Resin then the bottleneck is (as you said) likely to be elsewhere. Fastest way to figure that out is to profile the code with either something like Borland JProfiler (I think that's the name) or add a raft of elapsed-time checks in the code and use Log4J (or whatever) to spew them out to a file for subsequent analysis.

I must apologize for dragging this off-topic ...

- Kris
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Jun 21, 2004 8:11 pm    Post subject: Reply with quote

Kris, thank you for lending your experience to the conversation. 16000 requests per second? I had to read that three times before realizing that it wasn't a typeo. My hat's off to you to be able to wrangle a project of such proportions.

Quote:

As to APIs; Mango will expose several different flavours of a Session class. Each flavour will support different levels of the robustness/cost tradeoff, and do the clustering and synchronization behind the scenes. The actual persistence mechanism(s) would be provided by someone like yourself Brad, as a method override or delegate, and would be invoked only as necessary by the Session environment.


Outstanding! Once I have the prototype converted over to Mango, I suppose the project will benefit hugely from these improvements.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DSP All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group