Re: [apache-plusplus] Process model ideas for C++ Apache.

[Bret <mersault@busboy.sped.ukans.edu>]

>The "fiber" stuff I'm talking about you can find in the doc at
>http://www.arctic.org/~dgaudet/apache/2.0/process-model
>or in the docs subdirectory of the apache-2.0 cvs repository (which I
>think has a few more updates in it).

	Looked it over _very_ briefly, but obviously didn't pay close attention...
I'll have to read it again...

>But in a nutshell, a "fiber" is a user-level thread, a thread implemented
>by user-level code.  A "thread" is a kernel-level thread, and a "process" 
>is a process.  There are a few models of multiprocessing that are
>"interesting".  One is to have multiple fibers inside a process (I call
>this SSM, single process, single thread, multiple fibers), this is the
>model that many unix thread libraries use -- they wrap system calls like
>read()/write() with non-blocking reads and writes and possibly stick
>things in select when the operation would block.  Then there's SMS where
>the kernel provides all the threads, and syscalls don't need any wrapping.
>
>Solaris, NT, IRIX, and OSF/1 provide us with pthreads implementations that
>are actually SMM -- a number of user-level threads are multiplexed on a
>number of kernel-level threads.  This (and MMM) are the models that I find
>the most interesting... mostly based on some of the papers mentioned at
>that URL, and on discussions with folks.  NSPR happens to provide a SMM
>implementation on some systems (it only needs an SMS pthreads
>implementation and it can provide an SMM implementation on top of that).

	Intriguing... you're earlier comments regarding threading models make a
bit more sense now.
	However, am I to understand then that NSPR only provides access to a
multi-fibered, multi-threaded model whenever pthreads is available?  While
I agree that SMM is a far more interesting model to work from, this raises
some concerns in the back of my head...
	We've talked some about portability issues, and this is probably too early
to be worrying about this excessively, but perhaps we should at least give
it some thought.  We're going to need a compiler which can handle templates
and handle them correctly; to me, that says egcs.  If we decide to use ACE,
we _really_ need egcs, although that kind of prevents the use of NSPR...
having two "fiber" libraries with different innards running simultaneously
seems to me to be a really wide open door just asking for gremlins...
	Obviously, C++ brings with it many more portability concerns, and we can
address those at a later date.  What concerns me now is how much we
restrict ourselves by focusing on only one of the models described above.
If I understand you correctly, Dean, NSPR can only give us SMM on a
pthreads system.  Having done little threaded programming I'm by no means
an expert, but it occurs to me to ask... how much trouble can we get into
coding for a specific model?  It sounds as if NSPR is quite adaptable (that
is, works well under several of the models you mention) and can function
adequately using either an SSM or an SMM model, to use your terms.  Is this
the case and, if so, is the interface any different from the programmers
perspective?

>One major fault with my paper above is that I don't think the programmer
>should know about more than "processes" and "threads" -- the whether a
>thread is user-level or kernel-level should be completely transparant.  So
>beyond the discussion of process models themselve I tend to just refer to
>processes and threads... and skip fibers.  Confusing yeah. 

	The docs for NSPR suggest they adopt the same approach; threads are
presented and the application programmer could care less how they work.
That kind of abstraction answers the question above... but I don't remember
for sure if ACE utilizes a similar abstraction.  I'll have to go back and
look at how their thread model works... I thought they just used pthreads,
but I don't remember for sure...
	Ooops... those ACE guys never cease to amaze me.  Looks like Schmidt's
design is even more robust than NSPR; a cursory glance suggests that you
can even chose the model you wish to use at run-time.  Wow...
	Due to "technical difficulties" at work, I can't really check the
documents they've got on the ACE site... I'll look later...

>> 	I must admit, I haven't had a chance to study your Apache-nspr stuff yet,
>> Dean, but I'm curious as to you're reasoning for ditching the pool of
>> threads concept.  This might not be appropriate for this mailing list, but
>> I would like to hear why you're going for what seems like, on a surface
>> glance anyway, a less elegant system.  Was it only out of concern for
>> simplicity, or was there some other reason?
>
>The "best" method used to dispatch incoming connections to threads depends
>on how the underlying thread system is implemented.  

	Guess this is the cause of my concern.  I could be reading this wrong, but
it sounds as if we're coding optimizations which are specific for certain
platforms, specifically platforms with pthreads.  Will we see a performance
hit elsewhere and, if so, will it be significant?
	The other obvious question is, if this is a problem, could we distribute a
uniform thread implementation somehow?  Okay, so I'm asking for divine
intervention, but hey, why not? :)

>Apache isn't the only application that needs this accept --> thread
>operation... in fact it's something pretty easy to abstract.  I'd like to
>eventually abstract it and place it in NSPR itself.  NSPR can easily
>manage a pool of threads and dispatch according to the best model
>appropriate for whatever implementation it's interested in.  When a thread
>"exits" NSPR doesn't have to reclaim the storage immediately, it can just
>hold onto it, and use it for another connection... or for any other
>PR_CreateThread call.  The application shouldn't have to maintain a pool
>of prespawned threads, it can easily be hidden. 

	Okay... things are becoming clearer.  Distributing a modified NSPR which
handles these issues beyond the programmers gaze solves quite a few
problems.  How much would it take to modify NSPR not just to do this sort
of book-keeping, but to do it efficiently, all by itself?  It would not
only be useful for Apache, I'm sure the Netscape folks would be _very_
excited about it... the idea is very intriguing...

>So yeah, I plan to do this for simplicity, and I do have a plan for how to
>make it go faster later.  What I'm trying to do is not dig myself into a
>hole -- so when I try to take away an "optimization" that we've done in
>the past I try to make sure that it'll be easy to re-implement in another
>way if it turns out we really do need it.  I've been guilty of doing
>optimizations for the sake of doing them, without benchmark information,
>in the past... I'm trying to stop that... it just makes things far too
>complex. 

	Good planning...

>> avoids the "dispatch" overhead you expressed concern about before.  I
>> missed the lock to avoid starvation in the Apache code (have to go look for
>> that...)
>
>Look at http://www.apache.org/docs/misc/perf-tuning.html, search for
>"accept", I discuss the starvation problem there.  There's another form of
>starvation which 1.2 is susceptable to, but 1.3 isn't, which depends on
>the ordering of the Listen statements.  If you've got a busy address first
>(or last, I forget) it can starve the rest.

	Found the docs... thanks for the pointers...
	I had assumed that the mutexes were there as a rough attempt at eventually
moving away from a pre-forked model into something more like a purely
multithreaded model, but your docs suggest otherwise.  Just out of
curiousity, is there some reason the Apache group is so devoted to a
pre-forked model?
	Later...

								- Bret -