Task lists action breaks merge queues

[frenzibyte]

Queuing a pull request for merge shows the task-list-completed action hanging up not reporting any status at all, making the merge queue not merge any PRs due to no response.

[stilliard]

@frenzibyte sorry you're experiencing this currently.
could you send me the ID of your PR where you're seeing this? I can check the logs our side as we keep only the ID in the log against each step.
Generally in these situations we've sent the reply to github quickly but they may have been experiencing a backlog at he time.
One thing that we've seen help is to make a quick edit to the PR, such as just adding a label or adding a full stop to the PR name etc. This causes github to re-send the webhook to us and we send it back again but sometimes the 2nd request goes through faster than the first for whatever reason.

[frenzibyte]

@stilliard Sure, it's ppy/osu-framework#5753.

[BethanyBerkowitz]

Was there ever a solution for this? I tried editing the PR description and pushing a commit, but neither seemed to help. It's happening with most but not all PRs generated by Dependabot.

honeybadger-io/honeybadger-js#1113
honeybadger-io/honeybadger-js#1110
honeybadger-io/honeybadger-js#1111

cc @subzero10

[stilliard]

@frenzibyte sorry I missed your reply! All ok with your PRs now?

@BethanyBerkowitz thanks for reporting, no simple fix. From our side we send the check response back to github within a second as the check runs super fast.
I've checked the logs & I can see your checks all came through and ran in a few seconds but then lagged waiting for github to acknowledge the check response.

I'll keep an eye on it and I can try reporting to github again but I can see while typing that github's acknowledged the check response now and it's passed 👍

[frenzibyte]

We were unfortunately forced to disable merge queues due to this bug.

[stilliard]

@frenzibyte sorry to hear.
As far as I can see from logs we have, the issue happens on occasion when we send github the check response. Literally added logs before and after this line.

Normally it resolves again which to me hints it's temp slowness their side and normally by the time I've reported an issue to them it's resolved again.
More info and cases of it over on #25

Seems to be running ok now but I'll keep an eye on it and see if I can catch it running super slow again to report to them.

[ReinAchten-TomTom]

We're facing the same issue. For other similar status checks that come from an app or a 3rd-party tool which aren't triggered on a merge queue, we've added a workaround for now; a dummy workflow triggered on merge_group with a job that has the same name as the "missing" check. You can even skip that single job since GitHub considers a skipped job a passed status check. Something like:

name: Merge queue dummy

on:
  merge_group:

jobs:
  dummy-job:
    name: task-list-completed
    steps:
      - if: false
        run: echo "foo"

@stilliard for the task-list-completed app, wouldn't it help if you'd add the merge_group event as well over here? It will likely require some more (small) changes in the code and assumes you can derive PRs using the payload provided in this webhook event 🤔 (edit: the payload doesn't simply give you a list of pull requests unlike the payload for the other events used by the app, so might not be simple).

[stilliard]

Hi @ReinAchten-TomTom , sorry I'd missed your comment before.

Is the issue the same as above where sometimes the task shows as Expected or shows X of Y tasks complete but doesn't see the others?
If so it's likely part of #25 where sometimes it takes a long time for the final check to complete.

Or does this happen every time and it never completes?
I've not tried out merge queues myself yet in github but reading about them more now it looks like something I'd want to use.
I'll look to test this soon and see what happens.

[ReinAchten-TomTom]

Hi @stilliard, thanks for your response.

Hi @ReinAchten-TomTom , sorry I'd missed your comment before.

Is the issue the same as above where sometimes the task shows as Expected or shows X of Y tasks complete but doesn't see the others? If so it's likely part of #25 where sometimes it takes a long time for the final check to complete.

Or does this happen every time and it never completes? I've not tried out merge queues myself yet in github but reading about them more now it looks like something I'd want to use. I'll look to test this soon and see what happens.

It's like the issue described in the first comment:

Queuing a pull request for merge shows the task-list-completed action hanging up not reporting any status at all, making the merge queue not merge any PRs due to no response.

Unfortunately, GitHub's merge queue is lacking some imho important features at this moment, like specifying a different set of required checks for the merge queue. Given task-list-completed doesn't listen to the merge_group event and act accordingly, this results in the merge queue not being mergeable in case task-list-completed is setup as a required check.