[core] Use boost iostream in pipe logger for cross-platform

[dentiny]

This PR resolves a TODO item, which leverages boost iostreams to achieve cross-platform.

[jjyao]

Actually why removing this test? Shouldn't we just make it run on all platforms?

[dentiny]

Reason is, the test case is made for (1) pipe write, and check pipe size's effect; (2) log rotation.
Remove because (1) pipe is no longer used in the code; (2) log rotation has been tested elsewhere.

[jjyao]

[2025-01-25T02:42:31Z] //src/ray/util/tests:stream_redirection_utils_test FAILED in 3 out of 3 in 2.2s

[dentiny]

Hi @jjyao , could you please help me merge the PR? Thanks!

[jjyao]

I'll merge after windows tests pass

[jjyao]

Windows tests failed

[2025-01-29T21:17:16Z] //src/ray/util/tests:pipe_logger_test                                    FAILED in 3 out of 3 in 0.1s
[2025-01-29T21:14:48Z] //src/ray/util/tests:stream_redirection_utils_test                       FAILED in 3 out of 3 in 2.1s

[rynewang]

Discussed offline.

1. we don't need a StdOstream class - instead we need to define a spdlog sink for FDs (1 for posix, 1 for windows) and just put then into the logger.
2. the logger should NOT be captured in any lambdas, instead it can be put into the Handle class as a member. on close() we unset it to release the file handles <- this fixes the windows test issue
3. close() and flush() lambdas are not needed. we can just use spdlog logger's flush, and unset to do close().
4. in the short term (before 3), we can change lambda signatures to take a Handle& to avoid capturing the logger.

[dentiny]

Windows build gets stuck for half a day, merge develop to trigger a build

[dentiny]

Hi @jjyao could you please help me merge it? Thank you!