Re: [apache-plusplus] Applications for HTTP servers (was "Process model for C++ Apache")

[Bret <mersault@busboy.sped.ukans.edu>]

>> Taking Michael's ideas and playing along while
>> simultaneously trying to return to the notion of a
>> Web server ... it occurs to me that the main place
>> we're gonna be looking for optimization is going to
>> be the request handlers, whatever we're using to
>> actually process any requests.
>> ... it seems that the act of reading requests is
>> relatively simple compared to any processing involved,
>> particularly for non-static content.
>> If that is the case, from the perspective of an HTTP 
>> server I find it hard to justify a pool of reader threads
>> waiting for requests and throwing them into a queue.
>
>I'm no expert on httpd, but Dean wrote earlier:
>> There's another aspect to http requests... they
>> generally don't require a lot of computing time
>> in userland... most of what they do is tell the
>> kernel to send some bytes somewhere.

	I think if you'll double check, it was said that this is true for _static_
content.  Any type of dynamic content does _far_ more than this, like fire
up a Perl interpreter for example.  If all you're shipping out is static
content (aka a porn site :) ), then this would become significant...
otherwise, for any kind of requrest for dynamic content, more than this is
involved...

>However, let's assume that you are correct. Let's also
>loose our emphasis on thread pools, which is what we've been
>discussing almost exclusively up to now. Thread pools are a
>natural extension of the more-important abstraction of
>connection- and message-pools and the way they abstract
>connections from network I/O, and messages from the
>processing of messages. A connection is nothing more than
>a record in the connection queue; a message is nothing more
>than a record in the message-queuing system.

	I agree with your abstraction regarding a connection and/or message
queueing system.  That's fine... but how does a thread pool model become a
"natural extension" of that?  If by "thread pools" you mean your model of
thread pools (i.e. pools of threads performing a single function), I would
argue there is at least one other model which is just as natural... a
single thread handling a single request over it's entire lifetime.

>Organizing connections and messages in this form allows us
>to manupulate these records independent of their content;
>we have meta-connection and meta-message information
>available.

	Okay... we still agree... but independent of your method of processing
(i.e. the abstraction of each pool performing a "function" as opposed to a
single thread handling an entire request for it's entire lifetime), how is
this different from any other Web server on the planet?

>How is this useful in an http server? I don't know if its
>useful - you'll tell me that. But here are some contrived
>and exagerated examples that illustrate uses of connection-
>and message-queues. Lets also suppose that we have the thread
>pools as I proposed (acceptors(), readers(), processors() and
>writers() ).
>
>1. The acceptor() accepts a connection, and populates a
>connection-record in the queue. We want to filter connections
>by IP address, so we invent a filter() thread who traverses
>the queue tossing out undesired connections, before the
>readers() get the connection-records. This is trivial.
>
>2. A connection-record may contain multiple requests, so it
>probably contains a small header. The reader() reads-in each
>request into a separate message-record in the message queue,
>where thay are found by the processors(). Voila message
>multiplexing!

	You've now got(at least) three functions, with three condition variables,
along with three mutexes.  You can do the same thing with one mutex,
wrapped around your accept() call... why _introduce_ complexity if it
doesn't buy you anything?

>Bret wrote:
>> Multiple handler threads would be excellent here, however...
>> but what kind of threads?
>
>Here are some more contrived, ridiculous examples:
>
>Remember that connection/message records and the threads
>in thread pools (except for processors() ) are generic.
>Acceptors() service only one port, but it can be any port,
>including a connection to a database server that does
>user authentication or spits-out HTML. Readers() can read
>from any connection on the browser side or back-end; same
>for writers(). Now lets assume we have a httpd which can
>service 100 request per second. Based on measurements, we
>determine that we need 6 acceptors(), three for browsers,
>three for the back-end. Since browsers can be stingy, we
>need 15 readers() and 15 writers(). We are going to do heavy
>database access to three separate DBs on other hosts on the
>lan, so we need about 20 processors(). The DB processing is
>probably the bottleneck in this setup.

	I can do the same thing with some type of Socket object that doesn't
require the introduction of a new thread pool, again with all the overhead
that that brings.  I understand that the generic nature of your thread pool
is nice, but if I can do the same thing with objects _without_ having to
introduce a new thread every time I want to write somewhere, why not do that?

>3. Complicated requests arrive from the browser, each request
>requiring 3 database requests, one from each DB server. The
>acceptors() and readers() do their thing and the processors()
>get the requests. Each processor() parses his respective
>request, formulates three DB queries, and throws them
>simultaneously into the message queue.
>
>3 continued: Now the usual procedure for processing would be
>to make the first query, get the response, make the second
>query and response, and then the third. Lets say that each
>query/response takes 2 seconds (yes, contrived and ridiculous),
>then total request processing is 6 seconds.
>
>3. continued: However, since we have a message queue, all
>three queries/responses are done simultaneously so total
>request processing is 2 seconds. This parallelism is made
>possible by the abstraction of messages from message processing
>and explains why we have pools of readers(), writers(), etc.
>That pool of readers() is working one browser request and
>3 back-end DB responses which result from the browser request.

	I agree with you when you say "The DB processing is probably the
bottleneck in this setup..."
	In the case of multiple databases, maybe.  But in the case of multiple
queries to a _single_ database, this just doesn't work.  Your numbers
assume that the database can handle the concurrency you're looking for.
Let's assume the database only receives connections on a single, well-known
port... if it get's three requests simultaneously, it will most likely
queue the requests and process them one at a time.  At least that's my
suspicion.  So you're still waiting.  Only now you've got blocked threads
sitting around twiddling their thumbs while you're waiting for a database
to respond to your query.
	If that is the case, what has the concurrency in your program bought you
but excess overhead?
	On a side note, I'm not sure how well a design such as this lends itself
to modularization... you're doing _everything_ within the server core.  I
would say you would be better off coding that functionality into an
external module, perhaps giving the module an interface to the "thread
pools", and letting them perform the query.  The module can be implemented
however you'd like... but I guess I just keep wondering why I would use a
thread pool model if I can achieve the exact same effect with multiple
threads of Socket objects...

>4. Take the same setup as in example three, but add 3 more
>redundant DB servers. Since we can derive meta-connection
>information from the connection queue, we can tell if a
>DB server is bogging-down and load-balance to the redundant
>server.

	Agreed... I don't think anyone is challenging your connection abstraction
or your request abstraction... Apache makes hefty use of something similar
right now.  Look through the code and check out request_rec and conn_rec
(or is it connection_rec?  I never can remember)... these structs either do
or easily could perform the exact same functions you're talking about.
	But again, I would argue most of this should be modularized...

>I don't know if any of the capabilities described in these
>examples (I can give dozens more but its late and I'm tired)
>belong in a httpd, but the architecture (process-model)
>allows it to happen if desired. The process model is more
>than the threading model - it is also how the data is
>organized.

	This distinction is crucial... I accept the second part ("how the data is
organized"), but I'm highly suspicious of your threading model (actually,
in the terms you use below, I believe I should say "process model").  And,
apparently like you, I do not view the two as being impossibly intertwined...

>> As a side note, this entire discussion seems to justify
>> Dean's point; a thread interface which abstracts most of
>> this from the programmer and determines the best possible
>> model to run on the current system would be
>> wonderful.  If NSPR does that better than anybody else, ...
>
>Agreed, but the thread interface, NSPR or otherwise, should
>be able to handle whatever division of thread functionality
>is desired, whether start-to-end or what I suggested. Don't
>confuse the "thread interface" with the "thread model" with
>the "process model".

	I kinda thought that was Dean's point... the interface should hide most of
those details from the programmer, using the best available model for the
current architecture.  Seems to me like doing it any other way amount to an
optimization for the particular platforms that can handle it... if you use
method Foo in your program, while method Bar works better on my platform,
then I see a performance hit because you took it upon yourself to hard-code
method Foo into my server core.  Why bother if the threading interface can
do it for me?
	You're right, there is a distinction... but the "process model" should be
determined by me, the "thread interface" by whomever designed my library,
and the "thread model" by whatever is best for the platform I'm currently on.
	For example, you can say you want threads to perform a particular function
or to perform multiple functions over time... but how that's implemented
should be decided upon by the thread library.
	Just my two pennies... somebody please correct me in the numerous places
I'm sure I've made mistakes... :)


										- Bret -