Linux async sockets - SOCKETS_INLINE_COMPLETIONS and scheduling SocketIOEvent on threadpool

[petrroll]

Let's say I have a client library which manages its own sockets (e.g. StackExchangeRedis) and I want to make sure managing these sockets (ping, pongs) happens even if the instance is overloaded and general threadpool stops being able to make fast enough progress due to too many tasks scheduled on it (e.g.: StackExchange/StackExchange.Redis#3060).

From what I understand, there's little a library author can do in that case, regrettably. As the async sockets work roughly like this in linux:

1. Caller schedules a task
2. Dedicated epoll threads (can't controll them) read from OS sockets and:

• If SOCKETS_INLINE_COMPLETIONS==true, run SocketIOEvent completion inline
• else (which is default): schedule the SocketIOEvent on general threadpool using global queue

3. SocketIOEvent runs and:

• if it's on general threadpool && no execution context, schedules the user code continuation on threadpool preferLocal: true
• else schedules via execution context on threadpool

Which means that, assuming SOCKETS_INLINE_COMPLETIONS == false (which is default), one cannot make sure all of the tasks from OS socket getting data to user's continuation of the read operation run outside of the shared ThreadPool.

One could control the execution context of user code continuation and force scheduling that continuation on not shared, dedicated, possibly higher prio threads threadpool.

But there'd still be the de-bounce of SocketIOEvent via common threadpool (epoll threads schedule it, and this contination then schedules our user code continuation). And even worse the debounce happens via the global queue. Which means that if the global queue is not being drained enough, we'd never get to the user code.

If SOCKETS_INLINE_COMPLETIONS was true, it'd be possible. We could have the SocketIOEvent run on dedicated IO thread (not bogged down by run-away common threadpool) and the user continuation could run by our-controlled threadpool which could have dedicated high prio threads.

---

1. Is above roughly correct understanding?

2. If it is, what's the status of SOCKETS_INLINE_COMPLETIONS? are there any plans to enable it globally? If not, what are the concerns?

3. What about the worry that socket reads can get stalled if the threadpool is bogged down? Wouldn't it make sense to schedule these on dedicated threadpool or (if on the common) at least on the high priority global queue?

• I understand that with the high prio queue there's a risk of starving user continuations if the user continuations are then scheduled with preferLocal: true which can skip the queue a bit; but isn't that worth it?

Any other thoughts?

[hamedrabah]

Your description is roughly correct. The current Unix socket engine explicitly says that socket continuations are dispatched from the event thread to the ThreadPool. The process-wide escape hatch is:

DOTNET_SYSTEM_NET_SOCKETS_INLINE_COMPLETIONS=1

It defaults to off because inline completion can run continuations on the epoll/kqueue event thread. One blocking or unexpectedly expensive continuation can then stall event delivery for many sockets, which is worse than ordinary ThreadPool delay.

There is no supported per-library public switch that routes SocketIOEvent work to a private or high-priority pool. The runtime has a per-context preference internally, but it is not a library-facing scheduling contract.

For a library, the practical choices are to keep the shared ThreadPool healthy, avoid sync-over-async/blocking work, monitor starvation, and, where justified, raise the process's minimum worker count. A library that requires strict isolation must own a dedicated synchronous socket loop or native event loop and dispatch from there. I would only enable inline completions process-wide after proving every continuation on that path is short and nonblocking.