Proposal: a consistent and safe way to handle nodeos "shutdown sprinkles"

[bhazzard]

Story

Short Description
As an Antelope core developer, I want a proposal for handling nodeos "shutdown sprinkles", so that nodeos code is easier to understand and the shutdown process is safer.

What
Shutdown sprinkles are small bits of code that attempt to make shutdown safer by performing various actions such as checking if the app is currently shutting down, increasing the lifetime of shared_ptrs, etc. This story aims to define a proposal for standardizing and consolidating shutdown sprinkles in Nodeos to make the shutdown process safer and more predictable.

Why
Shutdown sprinkles lack cohesion and consistency, making it difficult to understand their purpose and whether they are necessary. Standardizing and consolidating shutdown sprinkles in Nodeos is important for several reasons. First and foremost, it will make the shutdown process safer and more predictable, reducing the likelihood of unexpected behavior or errors during shutdown. It will also make the codebase easier to understand and maintain by reducing complexity and eliminating redundancy. By documenting the new guidelines and best practices, we will also ensure that all developers are on the same page and can adhere to the same standards when implementing shutdown sprinkles in the future.

How
As part of this story, we will first identify all instances of shutdown sprinkles in the codebase and categorize them based on their purpose and necessity. We will then define a set of guidelines and best practices for implementing shutdown sprinkles in a consistent and safe manner. These guidelines will include recommendations for handling shared_ptrs, event loops, and any other resources that need to be managed during shutdown.

Next, we will refactor the existing shutdown sprinkles to adhere to these guidelines, consolidating them where necessary to eliminate redundancy and reduce complexity. We will also remove any shutdown sprinkles that are deemed unnecessary or overly conservative.

Finally, we will document the new guidelines and best practices for implementing shutdown sprinkles in Nodeos to ensure that all developers are aware of the new standards and can adhere to them in the future.

Acceptance Criteria

• All instances of shutdown sprinkles in the codebase have been identified and categorized based on their purpose and necessity.
• A set of guidelines and best practices for implementing shutdown sprinkles in a consistent and safe manner have been - defined and documented.
• Existing shutdown sprinkles have been refactored to adhere to the new guidelines, and redundant or unnecessary shutdown sprinkles have been removed.
• The new guidelines and best practices for implementing shutdown sprinkles have been documented and shared with all developers working on the nodeos project.

Background

Originally part of AntelopeIO/spring#842

[bhazzard]

Goal is to eliminate "shutdown sprinkles".

[bhazzard]

@spoonincode Is this issue redundant to this: #1099

[bhazzard]

Work is in progress, no proposal required.