KTOR-8105 Fix concurrent flush attempts corrupting segment pool

[bjhham]

Subsystem
Client, CIO

Motivation
Should solve all of our concurrency woes:

• KTOR-8105 Race condition when writing to a buffer leads to NPE inside CIOReaderKt.readFrom
• KTOR-8086 NPE in readBuffer
• KTOR-8096 ArrayIndexOutOfBounds kotlinx-io

Solution
I found that the NIOSocketImpl is flushing the same channel as the TLS encoder coroutine, which can potentially lead to corruption due to the encoder writing the "close" message, then performing its own flushAndClose.

The stack traces in question:

[1318671573] write @ call-context#1025
  io.ktor.utils.io.ByteChannel.flush(ByteChannel.kt:91)
  io.ktor.utils.io.ByteChannel.flushAndClose(ByteChannel.kt:127)
  io.ktor.utils.io.CloseHookByteWriteChannel.flushAndClose(CloseHookByteWriteChannel.kt:21)
  io.ktor.utils.io.ByteWriteChannelKt$close$1.invoke(ByteWriteChannel.kt:39)
  io.ktor.utils.io.ByteWriteChannelKt$close$1.invoke(ByteWriteChannel.kt:39)
  kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$1.invokeSuspend(IntrinsicsJvm.kt:223)
  kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  kotlinx.coroutines.internal.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:279)
  kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:16)
  io.ktor.utils.io.ByteWriteChannelOperationsKt.fireAndForget(ByteWriteChannelOperations.kt:215)
  io.ktor.utils.io.ByteWriteChannelKt.close(ByteWriteChannel.kt:39)
  io.ktor.network.sockets.NIOSocketImpl.close(NIOSocketImpl.kt:65)
  io.ktor.network.tls.TLSCommonKt.tls(TLSCommon.kt:43)
  io.ktor.network.tls.TLSCommonKt$tls$3.invokeSuspend(TLSCommon.kt)
  kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:98)
  kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:113)
  kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
  kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)
[1318671573] write @ cio-to-nio-writer#2264
  io.ktor.utils.io.ByteChannel.flush(ByteChannel.kt:91)
  io.ktor.utils.io.ByteChannel.flushAndClose(ByteChannel.kt:127)
  io.ktor.utils.io.ByteReadChannelOperationsKt$reader$job$1.invokeSuspend(ByteReadChannelOperations.kt:319)
  kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:98)
  kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:113)
  kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
  kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)
[1318671573] write @ cio-tls-encoder#2326
  io.ktor.utils.io.ByteChannel.getWriteBuffer(ByteChannel.kt:49)
  io.ktor.utils.io.ByteWriteChannelOperationsKt.writeByte(ByteWriteChannelOperations.kt:17)
  io.ktor.network.tls.RenderKt.writeRecord(Render.kt:18)
  io.ktor.network.tls.TLSClientHandshake$output$1.invokeSuspend(TLSClientHandshake.kt:120)
  kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
  kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:113)
  kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
  kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
  kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)

[bjhham]

FYI I am going to run my jury-rigged instrumentation to detect any other places where we might have two coroutines (writing/flusing) or reading on the same bytechannel instance.