Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temp fix for Android NDK REG_* conflicts #111681

Merged
merged 5 commits into from
Jan 22, 2025

Conversation

EgorBo
Copy link
Member

@EgorBo EgorBo commented Jan 21, 2025

Addresses build failures mentioned in this comment.

The Android NDK includes REG_* constants (and definitions like #define REG_RAX REG_RAX) in system headers, specifically ucontext.h, which is indirectly included in the JIT. This conflicts with JIT's

enum _regNumber_enum
{
    REG_RAX = 0,
    ...
}

There are several options to resolve this issue:

  1. Replace all REG_* with JITREG_* using regex. This is a simple solution but would change around 18k places in the JIT codebase + noise from clang-formatter.
  2. Replace enum _regNumber_enum with enum class Reg and update all uses from REG_RAX to Reg::RAX. This adds type safety, as we perform a lot of complex operations on REGs. However, it also means updating 18k places and additional work where we perform operations on registers (casts, comparisons, arithmetic, even shifts!), which can't be fully resolved with overridden operators (I tried). This is likely the most correct approach.
  3. Wrap all JIT code in a namespace, as suggested by @BruceForstall. This would require modifying 300+ files and dealing with clang-formatter.
  4. Use the approach in this PR with undef+def.

We decided to go with the fourth option for now (to unblock the Android folks) as it is the least intrusive and doesn't require many changes, thus minimizing merge conflicts for other PRs.

@dotnet-issue-labeler dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jan 21, 2025
Copy link
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

#define REG_FT11 JITREG_FT11
#undef REG_STK
#define REG_STK JITREG_STK
#endif // defined(TARGET_RISCV64)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this for all targets? Can we just do it for only the targets with the conflicts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that we only need this for x64 and arm32, so a few more can be removed. Also would suggest to self contain the workaround where the enum gets defined if that's possible (so do the undef+defs right after the enums)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that we only need this for x64 and arm32

that doesn't match with what I get, still needed for all four

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's needed for arm32, x86 and x64 at very least. It may be needed for RISC-V (depends on __USE_GNU define). It's not needed for arm64 with the NDK 27 version on my machine but there's no guarantee about different versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, odd. @janvorli #110471 (comment) that he was able to get it to build for arm64.

From what I read in that thread it used to build for x64 as well for a different NDK version

Copy link
Member Author

@EgorBo EgorBo Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway, if you have ideas how to narrow down the amount of undef, please feel free to send a PR once Android support is landed, so that can be validate on CI instead of creating build errors for that PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think it would not work? While it is not supported by us, I believe linux x86 it is actively being used by Samsung.

Copy link
Member

@filipnavara filipnavara Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe linux x86 it is actively being used by Samsung.

I think it was broken for a while and doesn't pass (some of) the tests. Hello World worked last time I tried (around the time of NativeAOT/win-x86 bring-up). I intend to look into it at some point and I would certainly prefer to keep the x86 code around.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think it would not work? While it is not supported by us, I believe linux x86 it is actively being used by Samsung.

I have not seen contributions to linux-x86 for a long time, and we have recently completed some quite substantial internal ABI representation changes within the JIT. Since it is untested I would be surprised if something hasn't bitrotted.

@EgorBo EgorBo marked this pull request as ready for review January 22, 2025 00:28
@EgorBo
Copy link
Member Author

EgorBo commented Jan 22, 2025

@dotnet/jit-contrib ready for review (to unblock android folks). Checked locally that this fixes build for #110471 (comment)

@steveisok
Copy link
Member

Thanks to @EgorBo and the JIT team for jumping on this quick!

@@ -133,6 +133,180 @@ REGDEF(K7, 7+KBASE, KMASK(7), "k7" )

REGDEF(STK, 8+KBASE, 0x0000, "STK" )

// Temporary workaround to avoid changing all the code that uses REG_* enum values.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mention "temporary" in the comments... Unless we want to find this comment and laugh about it in 10 years 😉

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment is still needed to explain why this copy-paste is needed, I am fine with removing the Temporary, just don't want to re-run the CI for it, I can remove it before merge with b-ag

Copy link
Member

@jakobbotsch jakobbotsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It LGTM, but you might want to get Bruce's sign-off too since he was participating in the internal discussion.

@EgorBo
Copy link
Member Author

EgorBo commented Jan 22, 2025

/ba-g latest build was green, merging comment-only change

@EgorBo
Copy link
Member Author

EgorBo commented Jan 22, 2025

/ba-g merging comment-only change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants