Re: [apache-plusplus] Process model ideas for C++ Apache.

[Bret <mersault@busboy.sped.ukans.edu>]

	Since Dean included my comments as well, I just thought I'd reply to this
message and save everybody additional inbox space.  And like I said
earlier, I'm by no means an expert on things threaded, so I'll shut up on
most of this instead of just embarrasing myself...

>select() (well, rather poll()) will have to be there behind the scenes.
>You can't get rid of it and still implement the multiple fiber models
>(using my terminology).  I don't see what is wrong with it.

	The wording was a bit confusing earlier, but as I understand it now, your
not arguing for removing the select() function, but for replacing the
massive overhead of handling off file descriptors obtained from select() to
waiting processes.  Is that fair?  If that is the case, I would agree...
but you're using a single, multithreaded process, rather than Apache's
multiprocess model, so the "overhead" you're concerned about should just
disappear.  Dean, did you cover this already... this message is getting too
damn long... :)

>>2.  the number of threads are minimized by specializing and
>>    abstracting threads by "function" ( accept()or, read()er,
>>    write()r, processer, etc.),
>
>Ack!  I'm already imagining the state-machine hell that Squid is.
>Have you looked at Squid?  That's awful to understand, and I've coded
>to that model before and it's awful to code to.  This is why I keep
>emphasizing fibers -- they let folks program with a serial mindset,
>which is a mindset folks are used to programing in... and they do it
>with little performance loss as far as I can tell...

	Could one of you two gentlemen speak a bit more about this "fiber" notion
to the horribly uninformed among us?

>>4.  complete abstraction of messages (requests & responses)
>>    from connections (network I/O) and the processing of the
>>    messages. This scheme implies a bucket(s) of message records
>>    in common or shared memory over which read()ers, and
>>    write()rs, processors, and other thread-types can operate.
>>    Bucket records are self-contained objects which know their
>>    state and handle their own concurrency.
>
>There's that state word...
>
>If I understand you, you want to make the state explicit and managed
>by the programmer.  By contrast, I really think state should be
>implicit and managed behind the scenes by the run-time.

	From merely a cursory glance, I'm not sure this is required by this
model... theoretically, some of the "buckets" could have pre-designed
functions which control this and which _should_ be hidden from the
programmer, assuming no further C++ conventions are violated! :)  In that
hypothetical case, the programmer would have less to do when writing the
code... but I'm only guessing here...

>>Acceptors:
>>Readers:
>>Processors:
>>Writers:
>
>So let me give an example why I think this is so damn difficult to use.
>I was going to post an example similar to this to new-httpd regarding
>one of our Grails: layered i/o.  You've got this Reader thread, and
>suppose it manages to read 1 byte.  That's it, the client is being stingy.
>It doesn't know what that byte is, it just hands it off to the Processor
>pool.  The processor pool immediately says "wtf?  I need more than one
>byte!" and goes right back to the Reader pool.  They go back and forth
>like this until the Processor is finally satisfied with the bytes it has.
>
>Every step along the way you've got mutexes being flipped around.
>One byte is extreme, but 128 bytes isn't.  Neither is 512 if you think
>later during the passing of bytes to a CGI.

	Ouch... but couldn't this be avoided by placing minimums on the data
you've got in your buffer before you trigger a control variable?

>Ah, but we'll just hide this all behind a method that reads a line of
>input, right?  But, hey, that's exactly what the implicit state model
>does...

	Nevermind... answer my own damn question if I'd read a bit further...

>>I'd have to study the code and design docs again to identify my
>>various sins in regards to C++ conventions, but I believe the
>>main one's are these:
>>
>...
>>3.  class methods which return the address of private data,
>>    especially aggregated structures for direct manipulation
>>    outside the class methods.
>
>Doing that, of course, completely negates one of the advantages of C++
>(implementation hiding).  Wouldn't inlined get/set methods get you the
>same performance benefit while still gaining you the ability to modify
>the implementation later?

	Agreed... I read this and was frightened.  Kinda hard to hide information
if you're throwing the address where your data lives all over the place.
As Dean mentioned, that's a pretty significant feature of C++ that you're
throwing away...
	This is also relevant to what I said above... if you want to use your
approach and perform some abstraction between programmer and thread
dispatching, you'll probably want to avoid this like the plague...
	Just out of curiousity, which of your sources suggested this kind of
approach?

>>Read()ers wake-up when the pool condition-variable (CV) is
>>asserted; any read()er can handle any connection in the pool.
>>So read()ers are scheduled by the randomness of the
>>condition-variable (CV) assertion. Read()ers read the
>>connection into a message pool, assert the CV for that pool,
>>and return to wait for another connection which needs reading.
>
>It sounds like you are describing the NT completion-port/fiber model.
>That model can be easily hidden behind a serial programming interface
>like NSPR...

	"Readers are scheduled by the _randomness_ of the condition variable"?????
 This sounds highly suspect to me... how do you avoid interference?
	Perhaps I see what Dean is getting at... in the design you've used,
Michael, how do you avoid multiple readers being awakened by a CV change
and both attempt to perform the same action simultaneously?  Even worse,
what if we're talking about writing, not reading?  You either have to have
some kind of scheduler, or you have to do a lot of praying.  Guess from
that perspective, masking everything behind an interface would help...
although I still think you could do it without it...

>NSPR makes me really happy up front because it can hide ALL of these
>implementation details, and allows for all the models that I'm interested
>in seeing apache support... In fact I would rather extend NSPR to
>provide an "accept() which creates a thread to service each connection"
>abstraction, because NSPR is at least one step closer to knowing which
>of the dozens of models of accept/threadpools are appropriate to the
>underlying multiprocessing model.

	

>>Yes, that's the plan, although I've found in my testing that
>>a single accept()or dedicated only to accepting connections is
>>often enough. I haven't tested this under a heavy HTTP
>>simulation yet so it may well require a pool of accept()ors.
>
>A single accept()or cannot take advantage of the parallelism in an SMP box
>with multiple network cards.  High end stuff, that I think I'm about to
>start ignoring for a while in the apache-nspr development... I'm about
>to get rid of the entire pool of threads concept in the interest of a
>simple accept()/create thread dispatch loop.  Then later, after it's
>been put into a benchmark situation, and it has been shown that multiple
>accept()ors would win I'll consider changing it...

	I must admit, I haven't had a chance to study your Apache-nspr stuff yet,
Dean, but I'm curious as to you're reasoning for ditching the pool of
threads concept.  This might not be appropriate for this mailing list, but
I would like to hear why you're going for what seems like, on a surface
glance anyway, a less elegant system.  Was it only out of concern for
simplicity, or was there some other reason?

>>I'm not sure how Apache does it, but I presume it builds a mask
>>of selectable fds for select(), calls select(), searches for
>>and dispatches to an idle process, who then calls accept()
>>(correct me if I'm wrong.) All the select() processing plus
>>the search/dispatch seems like extra overhead.
>
>Apache 1.x, single socket servers never need to call select() to accept.
>Remember Apache 1.x is a multiple *process* model.  They all just sit
>in accept() until they get a request.  They use blocking-i/o.  The only
>use of select() is to probe pipelined connections for a network packet
>optimization.  In this case it's a single file descriptor, and could
>easily be poll() instead.
>
>Apache 1.x multiple socket servers need to use select() to accept.
>But remember, there's absolutely no dispatch.  To do that requires fd
>passing between processes, absolutely something to be avoided.  There is
>a lock that prevents multiple processes from doing this select()/accept()
>at the same time (because it has starvation conditions we don't want to
>deal with other ways).

	These two distinctions cannot be emphasized enough... Apache runs multiple
processes rather than a single-process, multithreaded model, which is
currently being discussed.  Having the children perform the accept() also
avoids the "dispatch" overhead you expressed concern about before.  I
missed the lock to avoid starvation in the Apache code (have to go look for
that...), but this is similar to what I was referring to above... Michael,
how do you avoid this without some synchronizing agent somewhere?

	Later...

								- Bret -