[Feature] User can use different hadoop-user to submit application

[lordk911]

What changes were proposed in this pull request

Issue Number: close #3222

Brief change log

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

Does this pull request potentially affect one of the following parts

• Dependencies (does it add or upgrade a dependency): no

[wolfboys]

Overall, it looks good 👍 Keep up the good work! I'm really looking forward to seeing the next update. 😊

[lordk911]

@wolfboys I think the key changes are in YarnApplicationClient class, setConfig method directly using UserGroupInformation.setLoginUser method is not correct, should use ProxyUser, This requires either changing the doSubmit method declaration or moving the setConfig changes into the doSubmit method. And config the hadoop-user's keytab on the job definition page is required in a Kerberos-enabled environment too.

[lordk911]

I think the key changes are in YarnApplicationClient class, setConfig method directly using UserGroupInformation.setLoginUser method is not correct, should use ProxyUser, This requires either changing the doSubmit method declaration or moving the setConfig changes into the doSubmit method. And config the hadoop-user's keytab on the job definition page is required in a Kerberos-enabled environment too.

@wolfboys Do you have any suggestions for this?
1、if use UserGroupInformation.setLoginUser ，The CurrentUser of the other tenant is changed.
2、if use ProxyUser , the user configed as streampark.hadoop-user-name should have the ability to represent other users.
3、if kerberos is enabled, each tenant needs to upload their own keytab, and the keytab needs to be protected

[wolfboys]

I think the key changes are in YarnApplicationClient class, setConfig method directly using UserGroupInformation.setLoginUser method is not correct, should use ProxyUser, This requires either changing the doSubmit method declaration or moving the setConfig changes into the doSubmit method. And config the hadoop-user's keytab on the job definition page is required in a Kerberos-enabled environment too.

@wolfboys Do you have any suggestions for this? 1、if use UserGroupInformation.setLoginUser ，The CurrentUser of the other tenant is changed. 2、if use ProxyUser , the user configed as streampark.hadoop-user-name should have the ability to represent other users. 3、if kerberos is enabled, each tenant needs to upload their own keytab, and the keytab needs to be protected

If Kerberos is involved, things will become complicated. How to manage the Kerberos configurations for different users is a problem we have to face. I suggest not considering Kerberos for now.

[lordk911]

@wolfboys The code has been modified to use proxyuser

[wolfboys]

@wolfboys The code has been modified to use proxyuser

Thanks for your contribution, I will review it later

[caicancai]

You can modify your title, for example [Feature] User can use different hadoop-user to submit application, the first letter is preferably capitalized, thank you

[wolfboys]

The hadoopUser also needs to be set in the application copy method

[wolfboys]

One more question, have you tested it? Will the set hadoopUser work ok?

[lordk911]

One more question, have you tested it? Will the set hadoopUser work ok?

I've tested it against streampark 2.1.2

[lordk911]

The hadoopUser also needs to be set in the application copy method

done

[wolfboys]

LGTM, good job🤗