Failed notarizations should not be put in downloads

[Tom Goddard]

Mac notarization failure is supposed to be a build failure so the build
does not go on the web. That should be fixed.

[Zach Pearson]

FWIW, notarization failures are considered an error on Actions so Mac daily builds that don't get signed just don't get uploaded. I'm not sure how the old build scripts work.

[Greg Couch]

The old build scripts just echoed "ERROR IN NOTARIZATION" and made the unnotarized application available to download. There is code to treat it as a failure, but the echo never fails.

[pett]

Yeah, my dim recollection was that notarization failures still went on the web site. Nonetheless, since running the app when the notarization failed is basically impossible for the average user, it would be good to treat notarization failures the same as other build failures and not put them on the web site.

[Zach Pearson]

My GitHub Actions script retries notarization until it succeeds, but even before I implemented that failing notarizations would fail builds. That meant that those ChimeraXes wouldn't get uploaded. I think it's safe to close this ticket.

[Tom Goddard]

Doesn't retrying until it succeeds just make the build hang when notarization fails because a change to ChimeraX makes it not pass notarization? Even for the case when the Apple service is down or overloaded retrying with no limit seems like a bad idea, exchanging one bad behavior (failed notarization) for an equally bad one, hanging the builds. The past behavior of retrying 5 times seemed like the sensible compromise.

[Zach Pearson]

It's always notarizing that fails and never signing, and double-checking the script I can see why: signing waits for 30 seconds before retrying, but notarizing doesn't! It fails 5 times fast and then my retry action waits ten seconds before retrying the whole step. I could just add a call to time.sleep where necessary and remove the retry action, but as it is it's limited to 100 tries or 60 minutes total, whichever ends up being lower. Notarizing takes ~5-7 minutes so probably the time limit will be hit first.

[Tom Goddard]

I'm not sure I was clear before. Notarization can fail because or distribution does not meet Apple requirements. Then trying over and over for 60 minutes does not make too much sense. But maybe it is ok given that there is a 60 minute limit. Since the previous method was try up to 5 times I think it also could not distinguish an Apple service failure from a non-compliant distribution failure, so that also would waste 5 retries if or distribution does not meet some notarization requirement. So it sounds like the current behavior is ok. I didn't know about 60 minute limit since a few comments ago you said it retries until it succeeds.

[Zach Pearson]

Yes, sorry, when I said that I failed to take into account the case where a build was doomed to fail because it didn't meet Apple requirements.