Skip to content

Fix generate-direct race that ran apitypes generation twice#5442

Merged
janniklasrose merged 1 commit into
mainfrom
janniklasrose/taskfile-generate-direct-racecondition
Jun 5, 2026
Merged

Fix generate-direct race that ran apitypes generation twice#5442
janniklasrose merged 1 commit into
mainfrom
janniklasrose/taskfile-generate-direct-racecondition

Conversation

@janniklasrose
Copy link
Copy Markdown
Contributor

@janniklasrose janniklasrose commented Jun 4, 2026

What

  • Remove dependency on generate-direct-apitypes from generate-direct target, keep only generate-direct-resources

Why

  • generate-direct-resources already depends on generate-direct-apitypes which means apitypes is invoked twice concurrently

Testing

./task generate-direct --dry --force

Before

[generate-openapi-json]    wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-openapi-json]    wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-direct-resources] uv run ... > resources.generated.yml

After:

[generate-openapi-json]    wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-direct-resources] uv run ... > resources.generated.yml

This pull request was written by Isaac.

generate-direct listed both generate-direct-apitypes and generate-direct-resources as parallel deps, but generate-direct-resources already depends on generate-direct-apitypes. That diamond made go-task schedule generate-direct-apitypes (and its generate-openapi-json dep) twice, concurrently: two "python3 ... > apitypes.generated.yml" processes truncating and rewriting the same file that generate-direct-resources reads as input.

A resources run that read the half-truncated apitypes.generated.yml silently dropped every resource whose type comes from that file, emitting a resources.generated.yml with only the apitypes.yml override entries.

Depend only on generate-direct-resources, which already pulls in generate-direct-apitypes transitively, so the chain is linear and apitypes runs exactly once before resources reads its output. Verified with "task generate-direct --dry --force": apitypes is scheduled once after the fix versus twice before.

Co-authored-by: Isaac
@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

Commit: bce982d

Run: 26950252900

Copy link
Copy Markdown
Contributor

@denik denik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The problem is clear and fix is simple, this could benefit from shorter human-written description, this is too verbose:

Problem
generate-direct listed both generate-direct-apitypes and generate-direct-resources as parallel deps, but generate-direct-resources already declares deps: ['generate-direct-apitypes']. That diamond made go-task schedule generate-direct-apitypes (and its generate-openapi-json dep) twice, concurrently — two python3 … > bundle/direct/dresources/apitypes.generated.yml processes truncating and rewriting the same file that generate-direct-resources reads as input.

When the resources step read the half-truncated apitypes.generated.yml, it silently dropped every resource whose type comes from that file, emitting a resources.generated.yml containing only the apitypes.yml override entries. Because it depends on timing, the corruption is silent and intermittent — it only surfaced when diffing regenerated output.

Fix
Depend only on generate-direct-resources, which already pulls in generate-direct-apitypes transitively (which in turn depends on generate-openapi-json). The dependency chain is now strictly linear, so apitypes runs exactly once and completes before resources reads its output. The apitypes entry in the parent was both redundant and the source of the race.

Evidence
./task generate-direct --dry --force (dry-run prints the plan without executing wget/python):

Before — generate-direct-apitypes scheduled twice:

[generate-openapi-json] wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-openapi-json] wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-direct-resources] uv run ... > resources.generated.yml
After — scheduled once:

[generate-openapi-json] wget ... && mv ...
[generate-direct-apitypes] python3 generate_apitypes.py ... > apitypes.generated.yml
[generate-direct-resources] uv run ... > resources.generated.yml
A minimal standalone repro mirroring the dependency shape (go-task v3.50.0, the version this repo pins) confirmed the duplication is deterministic: the buggy diamond ran the apitypes subtree twice on 20/20 runs; the linear form ran it once on 20/20.

Test plan
./task generate-direct --dry --force shows generate-direct-apitypes scheduled exactly once (twice before).
yamlfmt -lint -conf yamlfmt.yml Taskfile.yml passes.
Standalone repro: buggy 2x, fixed 1x apitypes executions over 20 runs each.
This pull request and its description were written by Isaac.

@janniklasrose
Copy link
Copy Markdown
Contributor Author

nit: The problem is clear and fix is simple, this could benefit from shorter human-written description, this is too verbose:

this one was particularly wordy...

@janniklasrose janniklasrose added this pull request to the merge queue Jun 5, 2026
Merged via the queue into main with commit 46ced41 Jun 5, 2026
30 checks passed
@janniklasrose janniklasrose deleted the janniklasrose/taskfile-generate-direct-racecondition branch June 5, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants