Incidencia #41871

Check for GNU Make 4.3 and its implications for grouped targets

Abrir Fecha: 2021-03-30 00:02 Última actualización: 2021-03-31 01:47

Informador:
Propietario:
Tipo:
Estado:
Cerrado
Componente:
Prioridad:
6
Gravedad:
5 - Medium
Resolución:
Fixed
Fichero:
Ninguno

Details

In the recent release of OCR 0.3 we discovered a problem where there would be a Make rule that builds two or more targets in the same recipe, and if a jobserver build attempted to build more than one target covered by the same recipe at the same time, it would invoke the recipe more than once simultaneously, causing the two invocations to stomp on each others' output and leave at least one output file corrupted. On research into how Make works it was discovered that there is a feature called "grouped targets" useful for this case: replace the ":" separator in the recipe with "&:" and Make will do the right thing, invoking the recipe only once to build all its targets. (Why this isn't the default, is not clear; probably for backwards compatibility.) So we added that to the relevant recipes in OCR and pushed the release.

On further research: it's clear that there are a lot of places in the main Tsukurimashou build that would also benefit from using this feature. However, the feature is new in version 4.3 of GNU Make, released January 2020, so we can't count on every user having it. As far as I can tell - tested in GNU Make 4.2.1 - the new syntax is compatible with older versions, they just won't do the desired behaviour of only invoking the recipe once. Maybe older Make is interpreting the "&" as the filename of another target that the recipe might build, and then (since that isn't a target we ever ask to build) not caring that it does not actually get built.

So:

We should implement this syntax wherever it seems useful throughout the system

We should check for GNU Make 4.3 (or possibly just for the only-invoke-once behaviour itself) in each configure script

At least in OCR, we should disable jobserver mode if we don't have GNU Make 4.3. Other parts of the system, which seemingly were not breaking before, should maybe get a warning instead of an automatic disable; it seems like they were in a race-condition state, but in a build like Tsukurimashou that isn't running at the high speed of the OCR build, the race condition may be unlikely to really bite the user and it's a shame to force jobserver off unnecessarily.

Ticket History (2/2 Histories)

2021-03-30 00:02 Updated by: mskala
  • New Ticket "Check for GNU Make 4.3 and its implications for grouped targets" created
2021-03-31 01:47 Updated by: mskala
  • Estado Update from Open to Cerrado
  • Resolución Update from Ninguno to Fixed
Comentario

Fixed in SVN. Turns out the only places grouped targets are currently relevant are in the root Tsukurimashou build, and in OCR. Earlier versions of Make basically ignore the new syntax (the & is indeed treated as a filename) in recipes that have both a body and explicit filenames, but they generate errors on pattern rules (because the filename & does not contain a %) and on the no-body recipes used to trigger pattern rules.

So, a new Autoconf test was implemented to check whether Make accepts the &: syntax and set a substitution variable equal to & if it is accepted, null if not. That can then be used where appropriate to invoke the &: syntax. Configure scripts changed to give a warning when grouped targets are unavailable and parallel build is enabled, at the root; and to just automatically disable parallel build if grouped targets are unavailable, inside OCR. From experiment, it appears that using parallel build without grouped targets is very likely to cause trouble in OCR but only hypothetically possible to cause trouble in the root build, hence the difference in handling.

Attachment File List

No attachments

Editar

You are not logged in. I you are not logged in, your comment will be treated as an anonymous post. » Entrar