Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core] [6/N] Fix shared pointer usage for gcs server #48990

Merged

Conversation

@dentiny dentiny requested a review from a team as a code owner November 29, 2024 05:30
@dentiny dentiny added the go add ONLY when ready to merge, run all tests label Nov 29, 2024
@dentiny dentiny force-pushed the hjiang/gcs-server-shared-pointer-nov-28 branch from 80ca126 to ec79f58 Compare November 29, 2024 09:22
Copy link
Contributor

@rynewang rynewang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. though I think if we move forward this way we need to do proper lifetime annotations, e.g.

  1. GcsServer outlives everyone.
  2. GcsTableStorage, all GcsManagers share a lifetime, that starts from a GcsServer DoStart to Stop

and so on.

@dentiny
Copy link
Contributor Author

dentiny commented Nov 30, 2024

LGTM. though I think if we move forward this way we need to do proper lifetime annotations, e.g.

  1. GcsServer outlives everyone.
  2. GcsTableStorage, all GcsManagers share a lifetime, that starts from a GcsServer DoStart to Stop

and so on.

Could you please tell me what should I add? C++ attributes? Or documentations?
Not sure if I understand correctly, my take is unique pointer itself is a lifecycle annotation, which implicit denotes class object outlives data member and owns B's lifecyle.

@dentiny dentiny requested a review from rynewang November 30, 2024 05:54
/// Grpc based pubsub's periodical runner.
PeriodicalRunner pubsub_periodical_runner_;
/// The runner to run function periodically.
PeriodicalRunner periodical_runner_;
/// The gcs table storage.
std::shared_ptr<gcs::GcsTableStorage> gcs_table_storage_;
std::unique_ptr<gcs::GcsTableStorage> gcs_table_storage_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what should the decision process be behind keeping things as unique ptr vs. just the obj and deleted copy constructors for the class

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to use unique pointer for all dependent class, which potentially involves IO operations.
I think the best strategy is to have

  • a base class, which defines the interface
  • an implementation class, which is the real production code
  • and a mock class/fake class for unit test purpose

In GCS, most of the data members have their own logic, should be wrapped in unique pointer.

@dentiny
Copy link
Contributor Author

dentiny commented Dec 3, 2024

  • GcsTableStorage, all GcsManagers share a lifetime, that starts from a GcsServer DoStart to Stop

Discussed offline, comments added, PTAL.

@dentiny dentiny assigned jjyao and unassigned jjyao Dec 3, 2024
@jjyao
Copy link
Collaborator

jjyao commented Dec 4, 2024

Merge confclit.

Signed-off-by: hjiang <[email protected]>
@dentiny
Copy link
Contributor Author

dentiny commented Dec 4, 2024

Merge confclit.

@jjyao Yeah this PR has been on hold for a while... conflict should be resolved now.

@@ -89,7 +89,7 @@ class GcsInitData {

protected:
/// The gcs table storage.
std::shared_ptr<gcs::GcsTableStorage> gcs_table_storage_;
gcs::GcsTableStorage &gcs_table_storage_;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see reference and pointer are both used. What's the rule you are following?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Prefer to use reference as data member, since it guarantee non-null;
  • Have to use pointer in some places due to our testing implementation.

Copy link
Contributor Author

@dentiny dentiny Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In our test case, quite a few places are implemented as

Class(/*member1=*/nullptr, /*member2=*/nullptr)

meanwhile mem1 and mem2 have their own dependency which I don't want to touch, so using nullptr in some places, but the ownership should be clear.

@dentiny dentiny requested a review from jjyao December 4, 2024 06:17
@dentiny dentiny changed the title [core] Continued PR to fix shared pointer usage for gcs server [core] [6/N] Fix shared pointer usage for gcs server Dec 4, 2024
@jjyao jjyao merged commit f5698a2 into ray-project:master Dec 4, 2024
5 checks passed
ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants