re: Threading and Asynchronous Clients

Poster Content
nk4um Moderator
Posts: 901
February 6, 2012 12:26

Hi Jeff - good feedback as ever. I think Tony can give you a more detailed response on the sync/async API decisions - but a cursory view is that the existing async APIs do allow you to register a callback (listener) to receive either the response or its exception (if it fails). This can be an arbitrary class with the correct async listener interface. Its pretty simple to use, you'll find a concrete example of this is in the Throttle overlay implementation in layer1.

You are correct that the backend fulcrum issues its own threads into the system - so admin requests can't be hit by application thread starvation. Also in NK5 we made it the default that backend requests are at a higher priority - so they will "jump the queue" ahead of any existing applications requests. I think that combined this means that in NK5 the backend is unlikely to be prevented from working.

In fact this illustrates something else. Nothing says you have to use the kernel's threadpool. You can issue a synchronous request on your own instantiated thread at any time or place. Its just then you take on more responsibilities - especially with regard to managing long outbound IO waits. You probably know that with NKEE there is a deadlock detector (in the repository) which can be used to provide a safety net to detect at least half-dozen potential deadlock scenarios and mitigate them, including LONG_IO_WAIT.



Like · Post Reply
nk4um User
Posts: 112
February 3, 2012 19:12re: Threading and Asynchronous Clients

(from the newsletter )

Something that seems to be tiptoed around is the harsh reality that a synchronous request blocks the calling thread. It doesn't matter how an accessor is implemented, if it is called synchronously (i.e., with context.source()) then the caller can't proceed until the accessor posts a response. And furthermore, the worker thread does not become available to serve other requests until the accessor finishes executing (not necessarily posting a response). The normal way to deal with this in an event-based system is to set up callbacks, and indeed the interfaces are there (issueAsyncRequest and INKFAsyncRequestListener) but there seem to be very few examples of their use, and they're not very friendly to high-level accessors. It seems like it would be useful to have an asynchronous interface that took as arguments a request to make and a callback which is another request to post the result of the first request to. In pseudocode,

  callback.addParam("handle", context.getRequestHandle())
  context.asyncCall(subrequest, callback)

imagining that there is a "active:postresponse" accessor which takes as input some resource and sets that resource as the response of the request represented by the 'handle' parameter.

Also, on threads and monitoring, I've long thought it would be useful to have a guaranteed way to be able to access the admin interface or a monitoring interface, even if the system is busy or locked up. An anti-throttle if you will, or more concretely a separate thread pool. Throttles work well for controlled situations, but if you missed something and there happens to be some un-throttled resource that can get out of control, it can render your system unusable.

A simple example:


This is a simple reimplementation of functionality that unix has had for decades, you probably know it as a fork bomb (ok this doesn't fork, but that is trivial to change). What's interesting tho is the behavior. I can run this in the developer playpen, and it will sit there spinning forever without ever sending a response. I can open up a new window and check the 'requests and threads' page and see that the request is in LONG_WAIT, and all the worker threads are waiting. But if that's the case then how can I be getting anything from the server at all? Simple (I think): because the http fulcrum creates a new thread for each new request and the 'requests and threads' page doesn't make any async subrequests, and therefore doesn't require any worker threads. But if it did, then it would hang also. If the admin pages (not the developer pages, obviously!) used a separate threadpool for their subrequests, then this wouldn't be a problem for them in general, and they could safely issue asynchronous subrequests that would run even if the rest of the system is tied up in a knot.

Like · Post Reply