MNT add GitHub Action to add and remove no-contribution warning by AnneBeyer · Pull Request #33521 · scikit-learn/scikit-learn

AnneBeyer · 2026-03-11T15:11:52Z

Reference Issues/PRs

This implements an idea from the last Monthly Developer Meeting.

What does this implement/fix? Explain your changes.

Whenever a lable starting with "Needs" is added to an issue, a warning is added on top of the issue description like this:

Once all labels starting with "Needs" are removed, the warning is also removed.

I had some help from Codex on the bash part, but refined and refactored it (a lot) and played around with this in a separate repo to make sure the warning is

added only once
only removed if all issues with "Needs ..." are removed

@lesteve @betatim

AI usage disclosure

I used AI assistance for:

Code generation (e.g., when writing an implementation or fixing a bug)
Test/benchmark generation
Documentation (including examples)
Research and understanding

Any other comments?

lesteve

Thanks this seems super useful and not too complicated 🎉!

What is the behaviour when testing on your fork when the issue is created? Does it add the warning message or do you need to set a "Needs label"?

I would like that the warning when the issue is created. I am not 100% sure this is the case right now.

The wording may need a bit of tweaking, let's see what other people think before I jump in 😉.

.github/workflows/no-contribution-warning.yml

thomasjpfan · 2026-03-11T23:44:32Z

.github/workflows/no-contribution-warning.yml

+    steps:
+      - name: Remove no-contribution warning
+        run: |
+          has_needs_label="$(gh issue view "$ISSUE_NUMBER" --json labels --jq '.labels[].name | select(startswith("Needs"))' | head -n 1)"


With the startsWith(github.event.label.name, 'Needs') if conditional, do we need to check again with has_needs_label?

I think startsWith(github.event.label.name, 'Needs') only checks the label that was just removed but there may be more "Needs" labels left, so you need to check all the labels? For example you have "Needs Triage" and "Needs Reproducer" and you remove "Needs Triage", you still want the warning to be there and not remove it.

Side-comment: I think you can use github.event.issue.labels for all the labels, but I am not sure it would simplify things much compared to using the current logic with gh + jq.

Exactly, the if condition checks that the step is only executed when a "Needs ..." label is removed, and has_needs_label checks whether there are any other "Needs ..." labels left.

For reference, we currently have 9 labels staring with "Needs"

AnneBeyer · 2026-03-12T07:25:20Z

What is the behaviour when testing on your fork when the issue is created?

It is dependent on a "Needs ..." label, but since all our issues are automatically labelled "Needs Triage" when they are created, the message will be added a few seconds after they are created. Adding a condition for issue creation on top should be straight forward, though, if you think that makes more sense.

lesteve · 2026-03-12T07:51:30Z

It is dependent on a "Needs ..." label, but since all our issues are automatically labelled "Needs Triage" when they are created, the message will be added a few seconds after they are created

Can you test the behaviour in your separate repo by adding a workflow that adds the label on issue creation 🙏.

I seem to remember some kind of limitation that if the label is set by a workflow it will not trigger another one to prevent some kind of infinite loop e.g. labeled event => the workflow adds a label => labeled event => etc ...

See https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow#triggering-a-workflow-from-a-workflow or https://github.com/orgs/community/discussions/25565 for example.

AnneBeyer · 2026-03-12T08:57:38Z

I seem to remember some kind of limitation that if the label is set by a workflow it will not trigger another one to prevent some kind of infinite loop

Indeed, thanks for pointing that out! I added the Label Blank issues workflow to my test repo and nothing happens if the label is set by it. I'll add on issue opened as an additional condition.

lesteve · 2026-03-12T09:03:12Z

About the wording, I am completely fine with "not open for contribution (yet)" but in case others find it a bit harsh, here are a few alternatives I can think of:

"not PR-ready" or "not ready for a PR"
the issue needs to be understood and investigated before opening a PR makes sense

AnneBeyer · 2026-03-12T09:16:39Z

I agree that the wording is a bit too harsh. (I just copied it from another issue and focussed on the workflow first.)

I would vote for something like "not ready for a PR". It is more concise than the second point, but does not exclude contributions to the issue itself (like adding a reproducer, or whatever else might be needed.)

AnneBeyer · 2026-03-12T09:18:52Z

We could also add an explicit link to the "Needs triage" section in the docs?

lesteve · 2026-03-12T10:39:03Z

We could also add an explicit link to the "Needs triage" section in the docs?

You mean in the warning message, right? That seems reasonable. At the same time we should try to keep it short because otherwise are more like to skip over it.

Also I would use > [!IMPORTANT] to have it in red and may be make it a bit more meaningful

Important

this is some important things that you have to read

Hmmm actually it's violet for me not red, I guess orange is better 🤷

lesteve · 2026-03-12T10:41:00Z

Actually "CAUTION" is red see doc

Caution

now this is red and you should really pay attention

Slightly prefer read for it to stand out a bit more, but not a super strong opinion on this

betatim · 2026-03-12T13:46:13Z

I like the "not ready for a PR" wording suggestion. I would prefer using > [!IMPORTANT] because warning and caution feel a bit too strong and "in your face". But if others have a strong preference I'll submit to them.

The other thing to carefully look at is things like body="$(gh issue view "$ISSUE_NUMBER" --json body --jq .body)" - we take input from a (potentially evil) user. Then we do stuff with it, like use it later on when editing the issue comment. I don't remember what exactly the recommendation is to, for example, avoid the user putting something in their comment that ends up being run as an additional command. You somehow need to escape it, I think using " around things is "the thing to do". It would be good to triple check - if you find the right section of the docs they are pretty good and/or there are tools to scan actions. Commands that involve user controlled stuff make me nervous :D :D

AnneBeyer · 2026-03-12T14:45:49Z

I also think Caution would be a bit too much. I'd be in favour of keeping Warning, as this is also what people (mainly @lesteve and @lucyleeow) were adding manually before (see, e.g., #32739, #32781, #33045 or #33108)

lesteve · 2026-03-12T15:33:26Z

body="$(gh issue view "$ISSUE_NUMBER" --json body --jq .body)" - we take input from a (potentially evil) user. Then we do stuff with it, like use it later on when editing the issue comment. I don't remember what exactly the recommendation is to, for example, avoid the user putting something in their comment that ends up being run as an additional command. You somehow need to escape it, I think using " around things is "the thing to do

It's a good point to keep in mind indeed that we are using user input so we need to be careful about this, plus the workflow has elevated permissions (issues: write). So if you steal the token you can post plenty of comments into the repo, maybe delete comments, the GitHub doc (if you find it) has all the details.

I also think we are fine because we quote the $body variable. Writing the logic in Python with PyGitHub would also be a way to avoid being bitten by this if we want, because it's less easy to do a similar thing by mistake with Python.

lesteve · 2026-03-12T15:35:48Z

Between important and warning, I vote for warning. There is precedent and I feel orange may stand out a bit more than violet (super subjective opinion based on nothing much 😉).

AnneBeyer · 2026-03-12T16:12:36Z

I played around with the formatting a bit and found that, when new_body is empty (because the issue body was empty before adding the warning) the issue update is ignored instead of removing the body content (not sure if this would ever actually happen, though). Right now, I "solved" it by not removing the added new lines, because they are not rendered in the GitHub issues, but I'm not completely happy with that.

In general, I'm leaning towards checking out PyGitHub though, also after looking into all the things that could potentially go wrong 😅 but I will only manage to get to that next week.

AnneBeyer · 2026-03-18T09:00:28Z

Here is the version using PyGithub. It took me a while to figure out all the details, but I learned a lot, so feel free to ping me on all future things related to GitHub Actions. 🤓

AnneBeyer · 2026-03-18T09:00:34Z

The only potential problem I see is if the author edits the issue body right after the issue was created (to fix typos or the like). Then, on saving, an error is shown that the issue could not be updated, and the changes are not saved (but they will still be present in the edit window if the issue is edited again after the warning is added). Would we be willing to risk that? We're talking about a window of 10-20 seconds here.

AnneBeyer · 2026-03-18T09:01:19Z

One thing I wasn't sure is where the python script should live. Others are in:

build_tools/github/autoclose_prs.py
maint_tools/update_tracking_issue.py
.github/scripts/label_title_regex.py

I went for the last one, let me know if anywhere else would be preferred.

betatim

LGTM

I have no strong opinion on where the put the script. From your enumeration of locations it sounds like one day we could consolidate?

Don't tell too many people about your new found GitHub Action knowledge. It is in high demand :D

.github/scripts/add_or_remove_no_pr_warning.py

betatim · 2026-03-18T15:39:34Z

.github/scripts/add_or_remove_no_pr_warning.py

+    has_needs_labels = any(label.name.startswith("Needs") for label in issue.labels)
+    if not has_needs_labels:
+        if body_text.startswith(message):
+            new_body = body_text.removeprefix(f"{message}\n\n")


Whoop removeprefix being useful!

betatim · 2026-03-18T15:41:36Z

.github/workflows/not-ready-for-pr-warning.yml

+          python-version: '3.13'
+      - name: Install PyGithub
+        run: pip install -Uq PyGithub
+      - name: Checkout repository


Which commit does this checkout? My guess is that it is main because there is no commit to prefer for issues. I think this is what we want.

From the docs:

# The branch, tag or SHA to checkout. When checking out the repository that # triggered a workflow, this defaults to the reference or SHA for that event. # Otherwise, uses the default branch.

So I guess it's main either way in our case.

.github/workflows/not-ready-for-pr-warning.yml

Co-authored-by: Tim Head <betatim@gmail.com>

add workflow to add and remove warning

c88aa3f

lesteve reviewed Mar 11, 2026

View reviewed changes

.github/workflows/no-contribution-warning.yml Outdated Show resolved Hide resolved

.github/workflows/no-contribution-warning.yml Outdated Show resolved Hide resolved

thomasjpfan reviewed Mar 11, 2026

View reviewed changes

AnneBeyer added 2 commits March 12, 2026 14:28

also add warning when issue is opened

2f64ce0

add clearer variable name

571e9ce

AnneBeyer added 2 commits March 12, 2026 16:49

refactor newline handling

830dd67

adapt wording

21f83a1

AnneBeyer marked this pull request as draft March 16, 2026 10:39

AnneBeyer added this to Labs Mar 16, 2026

github-project-automation bot moved this to Todo in Labs Mar 16, 2026

AnneBeyer moved this from Todo to In progress in Labs Mar 16, 2026

AnneBeyer added 2 commits March 18, 2026 09:31

rename workflow and convert to using PyGithub

0c90b38

add python script for workflow and adapt warning message

069f7de

AnneBeyer marked this pull request as ready for review March 18, 2026 08:41

betatim approved these changes Mar 18, 2026

View reviewed changes

thomasjpfan reviewed Mar 19, 2026

View reviewed changes

.github/workflows/not-ready-for-pr-warning.yml Show resolved Hide resolved

AnneBeyer and others added 3 commits March 19, 2026 17:40

adapt wording

d52e327

Co-authored-by: Tim Head <betatim@gmail.com>

add sparse checkout to only checkout workflow script

343f479

Merge branch 'main' into add-no-contribution-warning

31cf1c6

betatim requested a review from lesteve March 20, 2026 10:36

betatim added Build / CI Waiting for Second Reviewer First reviewer is done, need a second one! labels Mar 20, 2026

lucyleeow added the No Changelog Needed label Mar 25, 2026

Uh oh!

Conversation

AnneBeyer commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement/fix? Explain your changes.

AI usage disclosure

Any other comments?

Uh oh!

lesteve left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thomasjpfan Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

lesteve Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnneBeyer Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

AnneBeyer Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

betatim commented Mar 12, 2026

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lesteve commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnneBeyer commented Mar 12, 2026

Uh oh!

AnneBeyer commented Mar 18, 2026

Uh oh!

AnneBeyer commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnneBeyer commented Mar 18, 2026

Uh oh!

betatim left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

betatim Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

betatim Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

AnneBeyer Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AnneBeyer commented Mar 11, 2026 •

edited

Loading

lesteve Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

lesteve commented Mar 12, 2026 •

edited

Loading

AnneBeyer commented Mar 18, 2026 •

edited

Loading