Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wasm][AOT] emcc : error - received SIGKILL (-9) #89402

Open
pavelsavara opened this issue Jul 24, 2023 · 8 comments
Open

[wasm][AOT] emcc : error - received SIGKILL (-9) #89402

pavelsavara opened this issue Jul 24, 2023 · 8 comments
Assignees
@lewing
Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono Known Build Error Use this to report build issues in the .NET Helix tab os-browser Browser variant of arch-wasm wasm-aot-test WebAssembly AOT Test
Milestone
10.0.0

Comments

@pavelsavara
Copy link
Member

pavelsavara commented Jul 24, 2023 •
edited by build-analysis bot
Loading

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=653707

Error Blob

{
  "BuildRetry": false,
  "ErrorPattern": ["emcc", "SIGKILL"],
  "ExcludeConsoleLog": false
}

Reproduction Steps

AOT wasm Microsoft.Extensions.Options.SourceGeneration.Unit.Tests

Log

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=653707
Error message validated: [emcc SIGKILL]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/25/2024 7:29:32 AM UTC

Report

Build Definition Test Pull Request
945538 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution
945250 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111723
945210 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112260
944758 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112283
944670 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #110622
944205 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112199
944099 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112266
943428 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112209
943321 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111642
943272 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111676
943239 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111934
943231 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112213
943179 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112206
943083 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #98643
943132 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution
942807 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112209
942068 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
942000 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution #112121
941952 dotnet/runtime normal-System.Collections.Immutable.Tests.WorkItemExecution
941949 dotnet/runtime normal-System.Runtime.Tests.WorkItemExecution
941824 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112213
941736 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #107769
941742 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112201
941724 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
941660 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112170
941620 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #110755
941615 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112206
941519 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112209
941493 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111091
941455 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111050
941378 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #107769
941286 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112142
941235 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112173
940232 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112162
939681 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111934
939371 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution #112122
939065 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111318
938844 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #104999
938760 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #107118
936402 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111742
937913 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111651
937307 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112066
937288 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111934
937285 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111934
937281 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112050
937266 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112065
936833 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #107769
936750 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111934
936575 dotnet/runtime WasmTestOnChrome-ST-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
936410 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #112028
936269 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111910
935218 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111318
933488 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111953
932840 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #104999
932764 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111938
932398 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111808
932372 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111929
932350 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution
931338 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111882
930809 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111751
930490 dotnet/runtime normal-System.Runtime.Tests.WorkItemExecution #111434
928501 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111581
927775 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
927287 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #110471
926874 dotnet/runtime normal-System.Runtime.Tests.WorkItemExecution
926934 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111318
926670 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111580
925820 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #110471
925619 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111716
925491 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
922813 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111398
919888 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
919130 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution
919030 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111446
918309 dotnet/runtime normal-System.Runtime.Serialization.Xml.Tests.WorkItemExecution
918072 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111327
917144 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
917141 dotnet/runtime normal-System.Runtime.Tests.WorkItemExecution
916319 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
916112 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
915581 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution #111318
914849 dotnet/runtime WasmTestOnChrome-ST-System.Runtime.Tests.WorkItemExecution
913579 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution
913290 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution #110695
913151 dotnet/runtime normal-Microsoft.Extensions.Options.SourceGeneration.Unit.Tests.WorkItemExecution

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
3 41 85
@pavelsavara pavelsavara added arch-wasm WebAssembly architecture blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' area-Codegen-AOT-mono wasm-aot-test WebAssembly AOT Test Known Build Error Use this to report build issues in the .NET Helix tab os-browser Browser variant of arch-wasm labels Jul 24, 2023
@pavelsavara pavelsavara added this to the 8.0.0 milestone Jul 24, 2023
@ghost
Copy link

ghost commented Jul 24, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Error Blob

{
  "BuildRetry": false,
  "ErrorPattern": "emcc.*SIGKILL",
  "ExcludeConsoleLog": true
}

Reproduction Steps

AOT wasm Microsoft.Extensions.Options.SourceGeneration.Unit.Tests

Log

Author: pavelsavara
Assignees: -
Labels:

arch-wasm, blocking-clean-ci, area-Codegen-AOT-mono, wasm-aot-test, Known Build Error, os-browser
Milestone: 8.0.0

@pavelsavara pavelsavara mentioned this issue Jul 24, 2023
Merged
@lewing
Copy link
Member

lewing commented Jul 24, 2023

Is this it hitting the ulimit of the container?

@lewing lewing self-assigned this Jul 24, 2023
@radical
Copy link
Member

radical commented Jul 24, 2023 

[ 7455.672962] [ 2825] 1000 2825 158937 41558 684032 0 0 python3 
[ 7455.672964] [ 4160] 102 4160 67335 453 180224 0 0 rsyslogd 
[ 7455.672966] [ 4181] 0 4181 455864 16951 634880 0 0 amacoreagent 
[ 7455.672968] [ 4453] 0 4453 298556 3977 249856 0 0 auoms 
[ 7455.672969] [ 4495] 0 4495 215902 6240 204800 0 0 auomscollect 
[ 7455.672971] [ 4750] 102 4750 191707 14176 622592 0 0 mdsd 
[ 7455.672973] [ 4967] 0 4967 334103 3030 253952 0 0 azsecd 
[ 7455.672975] [ 5046] 0 5046 352600 4193 270336 0 0 azsecmond 
[ 7455.672977] [ 32457] 0 32457 317321 3577 258048 0 0 azsecd 
[ 7455.672979] [ 2226] 0 2226 376486 334278 2924544 0 0 clamscan 
[ 7455.672981] [ 22067] 0 22067 180189 824 114688 0 -998 containerd-shim 
[ 7455.672983] [ 22089] 1000 22089 1159 16 57344 0 0 helix_docker_wo 
[ 7455.672985] [ 22156] 1000 22156 1159 17 53248 0 0 execute.sh 
[ 7455.672987] [ 22212] 1000 22212 5047 87 81920 0 0 bash 
[ 7455.672989] [ 22215] 1000 22215 68517493 67031 1163264 0 0 dotnet 
[ 7455.672990] [ 23320] 1000 23320 1159 17 61440 0 0 sh 
[ 7455.672992] [ 23321] 1000 23321 34332 22818 335872 0 0 python3 
[ 7455.672994] [ 23345] 1000 23345 1420573 1386396 11264000 0 0 wasm-opt 
[ 7455.672995] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/docker/a2a4080691301bb74e3b282eab0b6f638e9d6f1227e62967b7fd46654706e554,task=wasm-opt,pid=23345,uid=1000 
[ 7455.673018] Out of memory: Killed process 23345 (wasm-opt) total-vm:5682292kB, anon-rss:5545584kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:11000kB oom_score_adj:0

@lewing
Copy link
Member

lewing commented Jul 25, 2023

lets disable these tests in aot for now

@lewing lewing mentioned this issue Aug 13, 2023
Open
3 tasks
@lewing
Copy link
Member

lewing commented Aug 13, 2023

closting as a duplicate of dotnet/dnceng#450

@lewing lewing closed this as completed Aug 13, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Sep 12, 2023
@pavelsavara
Copy link
Member Author

I see this again https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-99833-merge-af0760c542d34f95a8/normal-System.Runtime.Tests/1/console.f849dad3.log?helixlogtype=result

@pavelsavara pavelsavara reopened this Apr 2, 2024
@lewing
Copy link
Member

lewing commented Apr 24, 2024

Analysis claims there are recent failures but the links are not updated?

@pavelsavara
Copy link
Member Author

we run most of those problematic cases only in runtime-wasm pipeline. Which needs to be triggered manually.
I updated the filter

@lewing lewing removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label May 2, 2024
@dotnet dotnet unlocked this conversation May 27, 2024
@akoeplinger akoeplinger modified the milestones: 8.0.0, 9.0.0 May 27, 2024
@lewing lewing modified the milestones: 9.0.0, 10.0.0 Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lewing lewing

Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono Known Build Error Use this to report build issues in the .NET Helix tab os-browser Browser variant of arch-wasm wasm-aot-test WebAssembly AOT Test
Projects
None yet
Milestone
10.0.0
Development

No branches or pull requests
4 participants
@radical @lewing @pavelsavara @akoeplinger