-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
slep000, slep workflow #30
Changes from 9 commits
e8f5c9c
6d24c52
d296bcb
2d3fca6
bf2191f
fb32fa0
0e01bb4
4c42863
5155b22
845255b
10f1a80
77cc077
dbb936d
8ca1a82
888aabe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,275 @@ | ||
.. _slep_000: | ||
|
||
============================== | ||
SLEP000: SLEP and its workflow | ||
============================== | ||
|
||
:Author: Adrin Jalali | ||
:Status: Draft | ||
:Type: Process | ||
:Created: 2020-02-13 | ||
|
||
Abstract | ||
######## | ||
|
||
This SLEP specifies details related to SLEP submission, review, and acceptance | ||
process. | ||
|
||
Motivation | ||
########## | ||
|
||
Without a predefined workflow, the discussions around a SLEP can be long and | ||
consume a lot of energy for both the author(s) and the reviewers. The lack of a | ||
known workflow also results in the SLEPs to take months (if not years) before | ||
it is merged as ``Under Review``. The purpose of this SLEP is to lubricate and | ||
ease the process of working on a SLEP, and make it a more enjoyable and | ||
productive experience. This SLEP borrows the process used in PEPs and NEPs | ||
which means there will be no ``Under Review`` status. | ||
|
||
|
||
What is a SLEP? | ||
############### | ||
|
||
SLEP stands for Scikit-Learn Enhancement Proposal, inspired from Python PEPs or | ||
Numpy NEPs. A SLEP is a design document providing information to the | ||
scikit-learn community, or describing a new feature for scikit-learn or its | ||
processes or environment. The SLEP should provide a concise technical | ||
specification of the proposed solution, and a rationale for the feature. | ||
|
||
We intend SLEPs to be the primary mechanisms for proposing major new features, | ||
for collecting community input on an issue, and for documenting the design | ||
decisions that have gone into scikit-learn. The SLEP author is responsible for | ||
building consensus within the community and documenting dissenting opinions. | ||
|
||
Because the SLEPs are maintained as text files in a versioned repository, their | ||
revision history is the historical record of the feature proposal. | ||
|
||
SLEP Audience | ||
############# | ||
|
||
The typical primary audience for SLEPs are the core developers of | ||
``scikit-learn`` and technical committee, as well as contributors to the | ||
project. However, these documents also serve the purpose of documenting the | ||
changes and decisions to help users understand the changes and why they are | ||
made. The SLEPs are available under `Scikit-learn enhancement proposals | ||
<https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/>`_. | ||
|
||
adrinjalali marked this conversation as resolved.
Show resolved
Hide resolved
|
||
SLEP Types | ||
########## | ||
adrinjalali marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
There are three kinds of SLEPs: | ||
|
||
1. A Standards Track SLEP describes a new feature or implementation for | ||
scikit-learn. | ||
|
||
2. An Informational SLEP describes a scikit-learn design issue, or provides | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just for clarification, these are new, right? Does anyone want to give an example of these, and when they are useful? The notion is a bit vague for me tbh. |
||
general guidelines or information to the scikit-learn community, but does not | ||
propose a new feature. Informational SLEPs do not necessarily represent a | ||
scikit-learn community consensus or recommendation, so users and implementers | ||
are free to ignore Informational SLEPs or follow their advice. | ||
|
||
3. A Process SLEP describes a process surrounding scikit-learn, or proposes a | ||
change to (or an event in) a process. Process SLEPs are like Standards Track | ||
SLEPs but apply to areas other than the scikit-learn library itself. They may | ||
propose an implementation, but not to scikit-learn’s codebase; they require | ||
community consensus. Examples include procedures, guidelines, changes to the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we explicitly include the governance document or is that encapsulated in the decision-making process? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does "community consensus" mean here? |
||
decision-making process, and changes to the tools or environment used in | ||
scikit-learn development. Any meta-SLEP is also considered a Process SLEP. | ||
|
||
|
||
SLEP Workflow | ||
############# | ||
|
||
A SLEP starts with an idea, which usually is discussed in an issue or a pull | ||
request on the main repo before submitting a SLEP. It is generally a good idea | ||
for the author of the SLEP to gauge the viability and the interest of the | ||
community before working on a SLEP, mostly to save author's time. | ||
|
||
A SLEP must have one or more champions: people who write the SLEP following the | ||
SLEP template, shepherd the discussions around it, and seek consensus in the | ||
community. | ||
|
||
The proposal should be submitted as a draft SLEP via a GitHub pull request to a | ||
``slepXXX`` directory with the name ``proposal.rst`` where ``XXX`` is an | ||
appropriately assigned three-digit number (e.g., ``slep000/proposal.rst``). The | ||
draft must use the `SLEP — Template and Instructions | ||
<https://github.com/scikit-learn/enhancement_proposals/blob/master/slep_template.rst>`_ | ||
file. | ||
|
||
Once the PR for the SLEP is created, a post should be made to the mailing list | ||
containing the sections up to “Backward compatibility”, with the purpose of | ||
limiting discussion there to usage and impact. Discussion on the pull request | ||
will have a broader scope, also including details of implementation. | ||
|
||
At the earliest convenience, the PR should be merged (regardless of whether it | ||
is accepted during discussion). Additional PRs may be made by the champions to | ||
adrinjalali marked this conversation as resolved.
Show resolved
Hide resolved
|
||
update or expand the SLEP, or by maintainers to set its status, discussion URL, | ||
etc. | ||
|
||
Standards Track SLEPs (see bellow) consist of two parts, a design document and | ||
a reference implementation. It is generally recommended that at least a | ||
prototype implementation be co-developed with the SLEP, as ideas that sound | ||
good in principle sometimes turn out to be impractical when subjected to the | ||
test of implementation. Often it makes sense for the prototype implementation | ||
to be made available as PR to the scikit-learn repo (making sure to | ||
appropriately mark the PR as a WIP). | ||
|
||
Review and Resolution | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 9 status are proposed here, but I am not sure which status would correspond to "Under review" or "Consolidated draft than is still under discussion". And what would be the current status of merged SLEP000 ? Should the SLEP be listed here ? |
||
--------------------- | ||
|
||
SLEPs are discussed on the mailing list or the PRs modifying the SLEP. The | ||
possible paths of the status of SLEPs are as follows: | ||
|
||
.. image:: pep-0001-process_flow.png | ||
:alt: SLEP process flow diagram | ||
|
||
All SLEPs should be created with the ``Draft`` status. | ||
|
||
Eventually, after discussion, there may be a consensus that the SLEP should be | ||
accepted – see the next section for details. At this point the status becomes | ||
``Accepted``. | ||
|
||
Once a SLEP has been ``Accepted``, the reference implementation must be | ||
completed. When the reference implementation is complete and incorporated into | ||
the main source code repository, the status will be changed to ``Final``. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Here, I think that we could add a section to give information about the scikit-learn version where the interface is considered stable. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. xref: #56 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done, WDYT? |
||
|
||
To allow gathering of additional design and interface feedback before | ||
committing to long term stability for a feature or API, a SLEP may also be | ||
marked as ``Provisional``. This is short for“Provisionally Accepted”, and | ||
adrinjalali marked this conversation as resolved.
Show resolved
Hide resolved
|
||
indicates that the proposal has been accepted for inclusion in the reference | ||
implementation, but additional user feedback is needed before the full design | ||
can be considered ``Final``. Unlike regular accepted SLEPs, provisionally | ||
accepted SLEPs may still be ``Rejected`` or ``Withdrawn`` even after the | ||
related changes have been included in a scikit-learn release. | ||
|
||
Wherever possible, it is considered preferable to reduce the scope of a | ||
proposal to avoid the need to rely on the ``Provisional`` status (e.g. by | ||
deferring some features to later SLEPs), as this status can lead to version | ||
compatibility challenges in the wider scikit-learn ecosystem. | ||
|
||
A SLEP can also be assigned status ``Deferred``. The SLEP author or a core | ||
developer can assign the SLEP this status when no progress is being made on the | ||
SLEP. | ||
|
||
A SLEP can also be ``Rejected``. Perhaps after all is said and done it was not | ||
a good idea. It is still important to have a record of this fact. The | ||
``Withdrawn`` status is similar; it means that the SLEP author themselves has | ||
decided that the SLEP is actually a bad idea, or has accepted that a competing | ||
proposal is a better alternative. | ||
|
||
When a SLEP is ``Accepted``, ``Rejected``, or ``Withdrawn``, the SLEP should be | ||
updated accordingly. In addition to updating the status field, at the very | ||
least the ``Resolution`` header should be added with a link to the relevant | ||
thread in the mailing list archives or where the discussion happened. | ||
|
||
SLEPs can also be ``Superseded`` by a different SLEP, rendering the original | ||
obsolete. The ``Replaced-By`` and ``Replaces`` headers should be added to the | ||
original and new SLEPs respectively. | ||
|
||
``Process`` SLEPs may also have a status of ``Active`` if they are never meant | ||
TomDLT marked this conversation as resolved.
Show resolved
Hide resolved
|
||
to be completed, e.g. SLEP 1 (this SLEP). | ||
|
||
How a SLEP becomes Accepted | ||
--------------------------- | ||
|
||
A SLEP is ``Accepted`` by the voting mechanism defined in the `governance model | ||
<https://scikit-learn.org/stable/governance.html?highlight=governance>`_. We | ||
need a concrete way to tell whether consensus has been reached. When you think | ||
a SLEP is ready to accept, create a PR changing the status of the SLEP to | ||
``Accepted``, then send an email to the scikit-learn mailing list with a | ||
subject like: | ||
|
||
Proposal to accept SLEP #<number>: <title> | ||
adrinjalali marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
In the body of your email, you should: | ||
|
||
- link to the latest version of the SLEP, and a link to the PR accepting the | ||
SLEP. | ||
|
||
- briefly describe any major points of contention and how they were resolved, | ||
|
||
- include a sentence like: “The vote will be closed in a month i.e. on | ||
<the_date>.” | ||
|
||
Generally the SLEP author will be the one to send this email, but anyone can do | ||
it; the important thing is to make sure that everyone knows when a SLEP is on | ||
the verge of acceptance, and give them a final chance to respond. | ||
|
||
In general, the goal is to make sure that the community has consensus, not | ||
provide a rigid policy for people to try to game. When in doubt, err on the | ||
side of asking for more feedback and looking for opportunities to compromise. | ||
|
||
If the final comment and voting period passes with the required majority, then | ||
the SLEP can officially be marked ``Accepted``. The ``Resolution`` header | ||
should link to the PR accepting the SLEP. | ||
|
||
If the vote does not achieve a required majority, then the SLEP remains in | ||
``Draft`` state, discussion continues as normal, and it can be proposed for | ||
acceptance again later once the objections are resolved. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need a mechanism for rejecting clearly a SLEP? The reason that I ask this is that I can be useful to avoid people loosing a lot of time on SLEP that is considered as not having chances of getting through? Maybe the "rejection" could and should be done via a pull request to the SLEP that explains the reason for rejecting it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The SLEP in its current state says:
I think it's important for the SLEP authors to make that decision if they wish, and otherwise leave it to the vote. rejected and withdrawn states are effectively the same. WDYT? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In other words, if not by the author's decision (not necessarily exhaustion) and not the vote, how else would you like to be able to reject a SLEP? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with @adrinjalali I think the mechanisms described here should be enough. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the case where the original author does not feel like withdrawing a SLEP after a failed vote and the majority things that it should be withdrawn or rejected, what should we do? Is it acceptable for someone else to do a PR to reject the slep and then use the usual decision making described in the governance doc to deal with the rejection PR? If so maybe we should make it a bit more explicit here. Maybe just amending the next section to:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the case where the original author does not feel like withdrawing a SLEP after a failed vote and the majority things that it should be withdrawn or rejected, what should we do? Is it acceptable for someone else to do a PR to reject the slep and then use the usual decision making described in the governance doc to deal with the rejection PR? If so maybe we should make it a bit more explicit here. Maybe just amending the next section to:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hm so the ambiguity on how to reach the "rejected" state is copied from the NEP. The PEP doesn't have that problem because they basically delegate the decision to an individual that then makes the call and can reject, or the steering council can reject. With the NEP version that goes back to draft there's no way to get rejected. Anyway I think it's fine the way it is for now, we can always clarify later. |
||
|
||
In unusual cases, with the request of the author, the scikit-learn technical | ||
committee may be asked to decide whether a controversial SLEP is ``Accepted``. | ||
|
||
Maintenance | ||
----------- | ||
|
||
In general, Standards track SLEPs are no longer modified after they have | ||
reached the ``Final`` state as the code and project documentation are | ||
considered the ultimate reference for the implemented feature. However, | ||
finalized Standards track SLEPs may be updated as needed. | ||
|
||
Process SLEPs may be updated over time to reflect changes to development | ||
practices and other details. The precise process followed in these cases will | ||
depend on the nature and purpose of the SLEP being updated. | ||
|
||
Format and Template | ||
------------------- | ||
|
||
SLEPs are UTF-8 encoded text files using the `reStructuredText | ||
<http://docutils.sourceforge.net/rst.html>`_ format. Please see the `SLEP — | ||
Template and Instructions | ||
<https://github.com/scikit-learn/enhancement_proposals/blob/master/slep_template.rst>`_ | ||
file and the `reStructuredTextPrimer | ||
<https://www.sphinx-doc.org/en/stable/rest.html>`_ for more information. We use | ||
`Sphinx <https://www.sphinx-doc.org/en/stable/>`_ to convert SLEPs to HTML for | ||
viewing on the web. | ||
|
||
Header Preamble | ||
--------------- | ||
|
||
Each SLEP must begin with a header preamble. The headers must appear in the | ||
following order. Headers marked with * are optional. All other headers are | ||
required:: | ||
|
||
:Author: <list of authors' real names and optionally, email addresses> | ||
:Status: <Draft | Active | Accepted | Deferred | Rejected | | ||
Withdrawn | Final | Superseded> | ||
:Type: <Standards Track | Informational | Process> | ||
:Created: <date created on, in yyyy-mm-dd format> | ||
* :Requires: <slep numbers> | ||
* :scikit-learn-Version: <version number> | ||
* :Replaces: <slep number> | ||
* :Replaced-By: <slep number> | ||
* :Resolution: <url> | ||
|
||
The Author header lists the names, and optionally the email addresses of all | ||
the authors of the SLEP. The format of the Author header value must be | ||
|
||
Random J. User <[email protected]> | ||
|
||
if the email address is included, and just | ||
|
||
Random J. User | ||
|
||
if the address is not given. If there are multiple authors, each should be on a | ||
separate line. | ||
|
||
Copyright | ||
--------- | ||
|
||
This document has been placed in the public domain [1]_. | ||
|
||
References and Footnotes | ||
------------------------ | ||
|
||
.. [1] _Open Publication License: https://www.opencontent.org/openpub/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😨