Applications for HTTP servers (was "Process model for C++ Apache")

[Bret <mersault@busboy.sped.ukans.edu>]

	Okay, I've returned from la-la land and have made an attempt to catch
up... but before I can do so, somebody beats me to the punch.  This may 

>> If fibers can not be spread across multiple CPUs, isn't there
>> a good possibility that a process with many fibers (say 300)
>> will be starving many of its fibers since the process is
>> competing for CPU (one CPU at a time) with all the other
>> processes in the system?
>
>There's another aspect to http requests... they generally don't require a
>lot of computing time in userland... most of what they do is tell the
>kernel to send some bytes somewhere.  The user-level threads packages
>generally context switch immediately when a write/read would block.  Then
>they'll work a bit on another request, and maybe switch again... a bunch
>can happen within one kernel context-switch.

	Is this the case for hybrid systems as well?  I was under the impression
that they amount to basically a mapping 'twixt user-level threads and
kernel-threads, so I wouldn't expect to see the same problems... is that
actually the case?
	As I understand it, Michael, the problem you refer to can happen when
_only_ fibers are used.  I believe hybrid systems can compensate somewhat,
perhaps at the cost of many context-switches... but I thought this was not
the case.  Dean, can you help out here?

>As I see it the largest point you're making is in the L1 cache effects of
>the increased number of stacks.  On x86 this will be really apparent
>because the stack is so busy due to a lack of registers.
>
>I really do think we can push static-content stuff into a lightweight pool
>of threads like you propose.  Most of that amounts to mapping the request
>to a file and then shovelling the file out.  The "shovelling the file out" 
>part really only needs to be one thread that does poll()-based work.

	Michael, you commented a bit ago that we were getting a bit off the topic
of an HTTP server, but I'm not sure I agree completely.  In fact, this
message reintroduced a point I was going to make...
	Taking Michael's ideas and playing along while simultaneously trying to
return to the notion of a Web server as opposed to a "MOM" product, it
occurs to me that the main place we're gonna be looking for optimization is
going to be the request handlers, whatever we're using to actually process
any requests.  I haven't actually done the benchmarks or profiling to
verify this, but it seems that the act of reading requests is relatively
simple compared to any processing involved, particularly for non-static
content.  Even on busy sites, I would suspect that the major hit comes from
the processing...
	If that is the case, from the perspective of an HTTP server I find it hard
to justify a pool of reader threads waiting for requests and throwing them
into a queue.  Multiple handler threads would be excellent here, however...
but what kind of threads?  Or, for that matter, do we even have a choice?
	An objection was made to using fibers for start-to-end processing,
apparently as opposed to fibers for individual funtions as used in "MOM"
product Michael describes.  Assuming we're only using threads for the
processing and writing, this objection seems to me to be nullified...
start-to-end threads, implemented using hybrid user-kernel threads whenever
possible, should work well enough.  We'll take a small hit on cache misses
for request for static content, but assuming the number of threads is not
extravagent, this should not be too painful... I hope...  Of course, this
penalty might be worse on a system where our thread implementation (we'll
call it NSPR now to bring a smile to Dean's face... :) ) can't use a hybrid
arrangement... any suggestions about how to minimize this?
	As a side note, this entire discussion seems to justify Dean's point; a
thread interface which abstracts most of this from the programmer and
determines the best possible model to run on the current system would be
wonderful.  If NSPR does that better than anybody else, it works for me...  
	Again, if I'm being stupid, ignore me... I'm still bringing myself up to
speed on all things threaded (it's been a while...)

								- Bret -