[BUG] master branch arrow oom

[liyin]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

No response

Environment

master

Debug logs

2025-01-02 23:59:08.131 ERROR 3088 --- [pi-summary-6344] o.a.h.collector.dispatch.WorkerPool : [Important] WorkerPool workerExecutor has uncaughtException.

java.lang.OutOfMemoryError: Cannot reserve 4194304 bytes of direct buffer memory (allocated: 8493725651, limit: 8497659904)
at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:121)
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:717)
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:692)
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:215)
at io.netty.buffer.PoolArena.tcacheAllocateNormal(PoolArena.java:197)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:139)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:129)
at io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.newDirectBufferL(PooledByteBufAllocatorL.java:173)
at io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.directBuffer(PooledByteBufAllocatorL.java:206)
at io.netty.buffer.PooledByteBufAllocatorL.allocate(PooledByteBufAllocatorL.java:53)
at org.apache.arrow.memory.netty.NettyAllocationManager.(NettyAllocationManager.java:74)
at org.apache.arrow.memory.netty.NettyAllocationManager.(NettyAllocationManager.java:81)
at org.apache.arrow.memory.netty.NettyAllocationManager$1.create(NettyAllocationManager.java:38)
at org.apache.arrow.memory.BaseAllocator.newAllocationManager(BaseAllocator.java:376)
at org.apache.arrow.memory.BaseAllocator.newAllocationManager(BaseAllocator.java:372)
at org.apache.arrow.memory.BaseAllocator.bufferWithoutReservation(BaseAllocator.java:357)
at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:336)
at org.apache.arrow.memory.RootAllocator.buffer(RootAllocator.java:27)
at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:298)
at org.apache.arrow.memory.RootAllocator.buffer(RootAllocator.java:27)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateBytes(BaseVariableWidthVector.java:498)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:446)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:407)
at org.apache.arrow.vector.VectorSchemaRoot.allocateNew(VectorSchemaRoot.java:150)
at org.apache.hertzbeat.common.entity.message.CollectRep$MetricsData$Builder.build(CollectRep.java:400)
at org.apache.hertzbeat.collector.dispatch.MetricsCollect.validateResponse(MetricsCollect.java:433)
at org.apache.hertzbeat.collector.dispatch.MetricsCollect.run(MetricsCollect.java:185)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)

Anything else?

No response

[lixiaobaivv]

I'm having the same issue

The problem may occur in this place.

[MasamiYui]

我遇到了同样的问题

此问题可能发生在这个地方。

Do you have a way to reproduce it? I want to try to solve it

[lixiaobaivv]

Memory overflow issue recurrence report

---

Problem description

I added 9000 ping assets in bulk for testing and reproduced the problem of memory overflow；

---

JVM parameters

The following are the JVM parameters used at startup:

--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED
-Xms1024m
-Xmx1024m
-XX:+PrintGCDetails
-XX:MaxDirectMemorySize=1024m

application.yml File contents

---

Screenshots of the subject
1.

2.

---