Skip to content

fix(cloudformation): move task settings into Mappings so they apply on deploy - #418

Merged
credfeto merged 9 commits into
mainfrom
chore/417-reduce-task-memory
Oct 13, 2026
Merged

credfeto merged 9 commits into
mainfrom
chore/417-reduce-task-memory

Conversation

@prpixie

@prpixie prpixie commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The deploy action only passes CloudFormation parameters for which it has an environment value. Changing a parameter's Default therefore never reached the existing stack, so the change was silently ignored on deploy. This is the same cause as issue 3930 in the funfair-treasury-reporting repository.

To make the task settings apply on deploy, they now live in a Deployment mapping read with Fn::FindInMap:

  • ECSClusterName, LoadBalancerArn, ContainerPort, TaskDefinitionCPU, TaskDefinitionMemory, DesiredCount, MinimumHealthPercent, MaximumHealthPercent and SecretsManagerArn moved into the mapping.
  • Values are unchanged except task memory, which drops from 4096 MB to 1024 MB (CPU stays 512 / 0.5 vCPU). Observed peak memory is about 190 MB.

Other changes

  • The always-false TaskRole condition and the TaskRoleArn property are removed. This has no behaviour change because the stack's task role is blank.
  • SecurityGroupIDs, SubnetIDs, VpcID and TaskExecutionRole stay as parameters until issue 1030 in the funfair-server-template repository is synced.
  • COVERAGE.md is bootstrapped as the coverage ratchet baseline (.NET overall line coverage 99.0%).
  • Changelog entries updated.

Closes #417

@prpixie prpixie added the auto-pr Pull request created automatically label Oct 12, 2026
Reduce the Fargate task definition memory default from 4096 MB to 1024 MB. CPU stays at 512 (0.5 vCPU). Observed peak memory is 4.6% of 4 GB (about 190 MB).

Closes #417
@credfeto

credfeto commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Roslyn analyzer findings

Source Rule Level File Line Suppressed Message
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.CloudFormation.cs 26 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapPost(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet CA2000 error src/BuildBot/Helpers/ServerStartup.cs 122 yes Call System.IDisposable.Dispose on object created by 'CreateLogger()' before all references to it are out of scope
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.Watchtower.cs 19 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapPost(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.Test.cs 22 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapGet(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.GitHub.cs 19 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapPost(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.GitHub.cs 29 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapPost(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet IL2026 error src/BuildBot/Helpers/Endpoints.GitHub.cs 39 yes Using member 'Microsoft.AspNetCore.Builder.EndpointRouteBuilderExtensions.MapPost(IEndpointRouteBuilder, String, Delegate)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. This API may perform reflection on the supplied delegate and its parameters. These types may be trimmed if not directly referenced.
DotNet IL2026 error src/BuildBot/obj/Release/net11.0/Microsoft.AspNetCore.Http.RequestDelegateGenerator/Microsoft.AspNetCore.Http.RequestDelegateGenerator.RequestDelegateGenerator/GeneratedRouteBuilderExtensions.g.cs 1014 yes Using member 'Microsoft.AspNetCore.Http.HttpResponseJsonExtensions.WriteAsJsonAsync(HttpResponse, TValue, JsonSerializerOptions, CancellationToken)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.

@credfeto credfeto changed the title docs(changelog): add placeholder entry for #417 chore(cloudformation): reduce task memory to 1 GB Oct 12, 2026
@credfeto

Copy link
Copy Markdown
Member

Prompt from the repository owner (live chat, verbatim):

we use 'default' for parameters which means they don't get changed on update... see this related issue https://github.com/funfair-tech/funfair-treasury-reporting/issues/3930 we use a similar structure to deploy here

Finding: confirmed for this repo. .github/actions/cloudformation-deploy/action.yml builds TemplateParameters.json only from parameters that have a matching env: value in the action (ApplicationName, ApplicationContainer, ApplicationVersion, ApiPrefix, ApiHost, LoadBalancerHostName, LoadBalancerPriority, Direct1LoadBalancerPriority, Direct2LoadBalancerPriority, AwsContainerRegistry), then runs aws cloudformation deploy --parameter-overrides file://TemplateParameters.json. TaskDefinitionMemory is never passed, so on a stack update CloudFormation keeps the previous value and the changed Default in this PR would not reach the running task.

Status: the PR is on hold, not marked ready, while the owner decides how to re-scope it.

…n deploy

The deploy action only passes parameters it has an env value for, so a changed parameter Default never reached the existing stack. Move ECSClusterName, LoadBalancerArn, ContainerPort, TaskDefinitionCPU, TaskDefinitionMemory, TaskRole, DesiredCount, MinimumHealthPercent, MaximumHealthPercent and SecretsManagerArn into a Deployment mapping read with Fn::FindInMap, so every deploy applies the template's values, including the 1 GB task memory.

Refs #417
@credfeto

Copy link
Copy Markdown
Member

Owner confirmed in live chat that the running stack's stored TaskRole parameter is blank ("task role blank"), so mapping Task.Role to "" keeps HasTaskRole false and the deploy does not change the task role.

@credfeto

Copy link
Copy Markdown
Member

Coding Researcher

Question: Does the CloudFormation service (not just cfn-lint) accept an empty-string Mappings leaf value (Deployment.Task.Role = "") read via Fn::FindInMap, and is Fn::FindInMap valid inside the HasTaskRole condition?

Outcome:

  • Fn::FindInMap inside Fn::Not/Fn::Equals in Conditions: supported, high confidence (condition functions, conditions syntax).
  • Empty-string mapping value: moderate confidence it is accepted. The docs allow String or List values with no minimum length (mappings syntax). No report of the service rejecting it was found; the known Mappings errors are about intrinsic functions inside mappings or wrong nesting. The empty value never reaches TaskRoleArn because Fn::If/AWS::NoValue removes it.
  • To confirm: run aws cloudformation validate-template or create a change set before merging.

@credfeto

Copy link
Copy Markdown
Member

Prompt from the repository owner (live chat, verbatim), answering Q7 ("Should #418 remove the dead task-role logic: the Role mapping key, the HasTaskRole condition and TaskRoleArn?"):

remove it

Resolution: routed to Code Fixer as one change on this PR. Behaviour is unchanged, because HasTaskRole was always false and TaskRoleArn always resolved to AWS::NoValue (the owner confirmed the running stack's task role is blank).

@credfeto

Copy link
Copy Markdown
Member

Replying to #418 (comment) ("remove it"):

Done in the working tree, pending commit: cloudformation.json no longer has the Role key in the Deployment mapping's Task group, the HasTaskRole condition (and with it the now-empty top-level Conditions block), or the TaskRoleArn property on the task definition. TaskExecutionRole and ExecutionRoleArn are unchanged. Behaviour is identical, because HasTaskRole was always false and TaskRoleArn always resolved to AWS::NoValue.

Pattern Sweep: no other reference to HasTaskRole or Deployment.Task.Role exists in the repo (the only remaining TaskRole text is a CHANGELOG entry, left for the changelog step). cfn-lint --info --include-checks I --template cloudformation.json is clean and the JSON parses. A Fixed in <sha> follow-up will be posted once the commit is pushed.

Deployment.Task.Role was a constant empty string, so HasTaskRole was always false and TaskRoleArn always resolved to AWS::NoValue. Remove the mapping key, the condition and the property; behaviour is unchanged.

Refs #417
@credfeto

Copy link
Copy Markdown
Member

Prompt from the repository owner (live chat, verbatim), answering Q8 ("Should the changelog correction remove the Unreleased W1030 entry about TaskRoleArn, since the property is now gone?"):

yes

Resolution: the Changelog correction step removes "Resolved cfn-lint W1030 warning on the TaskRoleArn CloudFormation property by only setting it when a TaskRole is provided" from Unreleased.

@credfeto

Copy link
Copy Markdown
Member

Fixed in 928a720: removed the always-false HasTaskRole condition, the Deployment.Task.Role mapping key and the TaskRoleArn property; behaviour unchanged. (Re: #418 (comment))

@credfeto credfeto added AI-Work Work for an AI Agent Config Change Configuration files changes Urgent Urgent Priority labels Oct 12, 2026
@credfeto credfeto changed the title chore(cloudformation): reduce task memory to 1 GB fix(cloudformation): move task settings into Mappings so they apply on deploy Oct 12, 2026
Comment thread cloudformation.json
Comment thread cloudformation.json
Comment thread CHANGELOG.md
Comment thread cloudformation.json
Comment thread cloudformation.json
Comment thread cloudformation.json
Comment thread cloudformation.json
Comment thread cloudformation.json
@credfeto

Copy link
Copy Markdown
Member

AI Review Loop: Phase B round 1 triage

  • Live stack values may differ from the old Defaults (cloudformation.json:47, :43, :280): valid and open. Because the deploy never passed these parameters, the stack holds its creation-time values, and the Defaults changed after creation (CPU, memory, ContainerPort). Already confirmed by the owner: memory 4 GB, CPU 0.5 vCPU, task role blank. Still to confirm against aws cloudformation describe-stacks: ContainerPort, ECSClusterName, LoadBalancerArn, SecretsManagerArn, DesiredCount, MinimumHealthPercent, MaximumHealthPercent. Waiting on the owner, since the AWS CLI is not available to the agent.
  • CHANGELOG "4 GB to 1 GB" (CHANGELOG.md:30): not changed. The owner measured the running task at 4 GB, so the statement is accurate.
  • Secrets resolved into plaintext Environment (cloudformation.json:97): pre-existing on main, not introduced here. Filed as Security: ECS task definition exposes Secrets Manager values as plain environment variables #419.
  • Root cause is the shared deploy action (cloudformation.json:32): already tracked in cloudformation-deploy: pass shared network and execution-role settings on every deploy funfair-server-template#1030, per the approved plan.
  • ContainerPort also hard-coded in the health check, Dockerfile and ServerStartup.cs (cloudformation.json:247): pre-existing and unchanged by this PR; the mapping keeps 8080.
  • Template not linted on this PR (cloudformation.json:37): lint-code skips while the PR is a draft and runs once it is marked ready. cfn-lint (--info --include-checks I) also passed locally and in the pre-commit hook on every commit.

@credfeto

Copy link
Copy Markdown
Member

Owner confirmed in live chat (verbatim), after checking the stack's Parameters tab in the CloudFormation console:

they all match... they've never been deployed other than just deploy this template

So ContainerPort 8080, ECSClusterName DiscordBots, the testing listener ARN, the BuildBot-ph1YIJ secret ARN, DesiredCount 1 and health 100/200 all equal the mapping. With memory 4 GB, CPU 0.5 vCPU and the task role blank also confirmed earlier, the next deploy changes only task memory (4096 -> 1024). This resolves the open Phase B finding at cloudformation.json:47/:43/:280; Phase B round 1 has no remaining actionable findings, so the loop advances to Phase C.

@credfeto

Copy link
Copy Markdown
Member

AI Review Loop: Phase C security review: no findings. The only secret-handling concern (plaintext {{resolve:secretsmanager}} in Environment) is pre-existing on main and tracked in #419; removing the TaskRole and SecretsManagerArn parameters reduces what a deploy can override.

Coverage ratchet bootstrap measurement on ea570db for #417; .NET overall unchanged at 99.0%.
@credfeto

Copy link
Copy Markdown
Member

Coverage ratchet passed - advancing to Human Review (bootstrap: no COVERAGE.md on main yet; .NET overall 99.0% on the branch, unchanged).

@credfeto
credfeto marked this pull request as ready for review October 13, 2026 10:00
@credfeto
credfeto requested a review from a team as a code owner October 13, 2026 10:00
@credfeto

Copy link
Copy Markdown
Member

AI Review Loop: reviewed 4038765

@credfeto

Copy link
Copy Markdown
Member

Super-linter summary

Language Validation result
CLOUDFORMATION Pass ✅
JSON Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@credfeto

Copy link
Copy Markdown
Member

CI Debugger: build-pre-release failed on 4038765 (run https://github.com/funfair-tech/BuildBot/actions/runs/38380350897), attempt 1 of 3 - routed to CI Debugger for diagnosis.

@credfeto

Copy link
Copy Markdown
Member

CI Debugger: environment/infrastructure block for build-pre-release

Root cause: an external NuGet feed, not code in this PR. The build-pre-release check fails in the Install Build Version Tool / Install dotnet tool step: dotnet tool install --local FunFair.BuildVersion ... --ignore-failed-sources ends with Unhandled exception: Response status code does not indicate success: 403 (Forbidden). It does this on all 5 retry attempts (30s apart), so the step exits 1 before any build of the repository starts.

Evidence

  • The runner's NuGet sources include two public S3 feeds, https://s3-eu-west-1.amazonaws.com/dotnet-nuget/ and https://s3-eu-west-1.amazonaws.com/dotnet-nuget-prerelease/. Neither is configured in this repository; both come from the runner's NuGet source setup.
  • Passing run 38292680629 (head ea570db, 2026-10-12 18:43Z): both S3 feeds returned NotFound (404) for registration/funfair.buildversion/index.json, which NuGet treats as "package not on this source", and the tool installed from the local cache (FunFair.BuildVersion 6.2.19.2266).
  • Failing runs 38380350897 and 38380434177 (head 4038765, 2026-10-13 10:00Z): the same URLs now return Forbidden (403). NuGet treats that as a fatal source error, and --ignore-failed-sources does not stop it. The local cache api.nuget.local:5555 still answers OK for every registration page.
  • A direct probe of both S3 URLs at diagnosis time also returns HTTP 403.
  • The only commit between the passing and failing heads changes the COVERAGE.md footer. The PR's cloudformation.json, COVERAGE.md and CHANGELOG.md changes do not take part in this step.

Fix needed (outside this PR): restore anonymous GET (404 for missing keys) on the dotnet-nuget and dotnet-nuget-prerelease S3 buckets, or remove or replace those sources in the build runners' NuGet source configuration. Once that is done, re-run build-pre-release; no code change is needed. Other repositories using the same runners and sources will fail the same way.

Stopping under the CI Debugger environment/infrastructure block convention (the credfeto-ci-debugger "Environment/Infrastructure Block Marker" rule), and marking this PR Blocked. The machine-readable orchestrator:env-block marker is deliberately left out: this is an interactive session with no IMAGE_SHA_DEVELOPMENT_AGENT available, and a new agent image would not fix an S3 bucket policy anyway, so Blocked must be cleared by hand once the feed is fixed.

@credfeto credfeto added Blocked Blocked by a dependency or external factor and removed Blocked Blocked by a dependency or external factor labels Oct 13, 2026
@credfeto

Copy link
Copy Markdown
Member

Owner reported in live chat (verbatim):

its fixed

Verified: both S3 NuGet feeds (dotnet-nuget, dotnet-nuget-prerelease) now return 404 for registration/funfair.buildversion/index.json instead of 403. Re-ran the failed build-pre-release job (run 38380434177). The environment cause behind Blocked is resolved; the label is left for the owner to clear.

@credfeto
credfeto merged commit 0c762d1 into main Oct 13, 2026
47 of 49 checks passed
@credfeto
credfeto deleted the chore/417-reduce-task-memory branch October 13, 2026 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-Work Work for an AI Agent auto-pr Pull request created automatically Config Change Configuration files changes Urgent Urgent Priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce BuildBot ECS task memory from 4 GB to 1 GB

2 participants