JIT: Always aggressively compact blocks

[amanasifkhalid]

Part of #107749. Now that we only run block layout in the backend, we can freely compact all compactable blocks in the frontend without worrying about churning layout.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[amanasifkhalid]

cc @dotnet/jit-contrib, @AndyAyersMS PTAL. Small diffs from LSRA churn -- Linux x64 diffs seem to be inflated by duplicate entries in benchmarks.run_pgo:

Top method regressions (bytes):
          15 (2.77 % of base) : 103346.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 126722.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 137390.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 163348.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 163596.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 164460.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 104896.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 134841.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 163992.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 164420.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 172856.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 173384.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 173536.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 105880.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 119574.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 122324.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 130979.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.77 % of base) : 139112.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 163516.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)
          15 (2.74 % of base) : 164156.dasm - System.Text.Encoding:GetBytes(ulong,int,ulong,int,System.Text.EncoderNLS):int:this (Tier1)

TP diffs seem to be inflated on non-Windows platforms by some outlier methods in realworld, though pin is borked on my machine, so I cannot narrow down the culprit. I'm guessing we're hitting some pathological case where LSRA needs to split many more critical edges, because we're now compacting BBJ_ALWAYS blocks with multiple predecessors into successor blocks with multiple successors.

[amanasifkhalid]

Re: the TP outlier, the method with the most basic blocks in realworld is Interop+Sys:GetDriveType(System.String):int, with over a thousand blocks post-frontend. Running SPMI tpdiff on this method alone shows a 167% TP regression. The size of the method itself doesn't seem to be the problem, but its branchiness is: one merge block at the end of the method has 780 predecessors. This change unlocks additional compaction during a late fgUpdateFlowGraph call in lowering, during which 220 blocks are compacted, and for each compaction, we must iterate and redirect the 780 predecessors.

Since this is an extreme case, I don't think it warrants changing our compaction strategy, though since we seem to compact chains of blocks with some frequency, it might be worth looking into deferring flow edge updates to reduce duplicated work; I'll keep this in mind when we start rewriting fgUpdateFlowGraph altogether, which we plan to do as part of #107749. @AndyAyersMS are you ok with merging this as-is?

[amanasifkhalid]

ping @AndyAyersMS

[AndyAyersMS]

Can you add a note to #107749 so we don't forget about fixing the big fan-in case?

[amanasifkhalid]

Can you add a note to #107749 so we don't forget about fixing the big fan-in case?

Sure thing; added.

[amanasifkhalid]

/ba-g blocked by build timeout