Alternative supervision logic / custom nurseries

[miracle2k]

I saw this topic mentioned a couple times in various tickets. Working with trio over the last couple of weeks, there were two specific kinds of behaviour that I felt I wanted to have.

One was a kind of "main task" mode. If the task ends, without an exception, I want the nursery to shutdown, rather than wait until other tasks are complete. This is a basically just a protection against "this task should not really end without an exception, but due to some bug in does". It's a pretty minor thing, so I won't talk about it more.

The more useful behaviour is a kind of "daemon" mode. A background task that will not block the nursery from exiting. I had of cases where I needed this; it was always about trying to process events concurrently with task.

My intention was to implement this in a custom nursery, and then ask here about what trio could do to make this kind of think easier or nicer. But I think the result turned out to be pretty nice and usable already, so maybe no action is required.

The code is here: https://gist.github.com/miracle2k/8499df40a7b650198bbbc3038a6fb292

And I am using it like this:

async def wait_for_stop_signal(self):    
    await self.stop_event.wait()
    raise StopSignalReceived()

async with open_special_nursery() as nursery:
    nursery.start_soon(wait_for_stop_signal, daemon=True)
    nursery.start_soon(func)

Hopefully this is useful for some.

I know there was some talk about Erlang-style supervision trees; I am not sure what the current plan is regarding custom supervision logic, but from this experiment, it seems to me that working on top of the existing nursery API is pretty flexible.

[smurfix]

Just open two nurseries – the outer for daemon tasks and the inner for non-daemon tasks. After you leave the inner nursery, cancel the outer nursery's cancel scope. Something like this:

@asynccontextmanager
async def open_dual_nurseries():
    async with trio.open_nursery() as daemon_nursery:
        try:
            async with trio.open_nursery() as normal_nursery:
                yield (normal_nursery, daemon_nursery)
        finally:
            daemon_nursery.cancel_scope.cancel()

Also note that your SpecialNursery doesn't have an async start method, which is very useful in all kind of situations.

[miracle2k]

I never cease to be surprised what can be accomplished by strategic arrangement of nurseries (or in some cases queues)!

[njsmith]

I guess that can be simplified even further to:

@asynccontextmanager
async def open_daemon_nursery():
    async with trio.open_nursery() as nursery:
        try:
            yield nursery
        finally:
            nursery.cancel_scope.cancel()

(Note: the try/finally here also fix a bug in the version above)

[smurfix]

Thanks, bug in the version above fixed.

The idea is to enforce waiting until all tasks in the "normal" nursery are finished before cancelling the threads in the "daemon" nursery. My version encapsulates this idea instead of requiring the caller to remember in which order to set up the nurseries.

[oremanj]

I don't think there's an action item left here besides "the two-nurseries trick is useful and maybe we should put it as an example somewhere". I've added that to #472, so am closing this one.