EDU-1425 Document requirements for the Outputs tab in platform - #1837
Merged
Merged
Conversation
Document the Pipeline outputs sub-tab on the run details page and its requirements (data lineage enabled + Nextflow workflow output syntax), add a pipeline output visibility benefit to data lineage, and link to the Outputs tab documentation from the Pipelines overview. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
✅ Deploy Preview for seqera-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
justinegeffen
added
1. Editor review
Mirror the Pipeline outputs sub-tab documentation, data lineage benefit, and Pipelines overview link into the latest published enterprise doc set so the change is visible on production docs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
christopher-hakkaart
approved these changes
Sep 15, 2026
justinegeffen
added
2. Edu reviews complete
justinegeffen
added a commit
that referenced
this pull request
Sep 16, 2026
…s policies [EDU-1421] Two follow-ups on the Cloud half of #1847: - Restore the "Pipeline output visibility" bullet on the Cloud data lineage page. #1847 merged master in and then dropped it while rewriting the "Why use data lineage" list, silently reverting #1837 (EDU-1425). Restoring it also re-uses the [nextflow-workflow-outputs] link definition, which #1847 left defined but orphaned. - Add s3:ListBucket to the three Automatic-mode lineage policy surfaces that were missing it: aws-batch.mdx, aws-cloud.mdx, and both combined policy JSONs rendered inline and offered for download from those pages. #1847 documents the permission as required — without it, reindex fails with AccessDenied — but only added it to workspace-management.md, so anyone following a compute-env page or downloading the full policy got a set that cannot reindex. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
justinegeffen
added a commit
that referenced
this pull request
Sep 16, 2026
…ith Enterprise [EDU-1421] - Restore the "Pipeline output visibility" bullet on the Enterprise data lineage page, same regression already fixed on the Cloud page: the rewrite of "Why use data lineage" dropped it, reverting #1837 (EDU-1425) and orphaning the [nextflow-workflow-outputs] definition. Lands in the unversioned tree, so it ships with 26.2. - Add s3:GetObject and the object-level resource to the Cloud lineage statements in aws-batch.mdx, aws-cloud.mdx and both combined policy JSONs. This PR's own Enterprise page documents ManageLineageBuckets with s3:GetObject and s3:ListBucket over both seqera-lineage-* and seqera-lineage-*/*, and Platform reads each record from the bucket per Functional flow step 4, but the Cloud surfaces granted bucket-level actions only. The two products' integration credentials now match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
justinegeffen
added a commit
that referenced
this pull request
Sep 17, 2026
* docs: lineage event ingestion moves from SQS to SNS push [PLAT-6516] Platform no longer provisions or polls an SQS queue in the customer's AWS account for lineage record notifications. The lineage bucket publishes object-created events to an SNS topic, which pushes them to a per-workspace Platform webhook over HTTPS. No sqs:* grant remains in any documented permission set. Cloud and Enterprise: - Replace the sqs:* statements with the SNS set Platform actually calls: CreateTopic, SetTopicAttributes, Subscribe, ConfirmSubscription, Unsubscribe, DeleteTopic on seqera-lineage-*. - Document sns:ConfirmSubscription and s3:ListBucket as required, since both fail quietly: without the former a workspace reports as configured while indexing nothing, without the latter reindex fails with AccessDenied. - Add manual-mode setup: topic access policy, bucket notification rule, and subscribing the workspace webhook URL. Manual credentials now need only s3:GetObject, s3:ListBucket and sns:ConfirmSubscription. - Document the Event delivery status and Webhook URL surfaced on the lineage settings page, and rewrite the mode-switch and disable behavior in terms of the topic and its subscription. - Rework the functional flow, and drop the SQS cost figure. Enterprise only: - Upgrade guidance for the one-off SQS-to-SNS migration: it needs both permission sets while it runs, and the SQS grants can be dropped afterwards. Covers retrying from the settings page and Disable lineage as recovery, and states that the bucket and its records are never affected. - Note that TOWER_SERVER_URL must be publicly reachable over HTTPS. - Document TOWER_LINEAGE_STORE_PREFIX, TOWER_LINEAGE_SNS_MAX_RETRIES, TOWER_LINEAGE_SNS_MAX_DELAY_SECONDS and TOWER_LINEAGE_MIGRATE_SQS_TRANSPORT. Ref: seqeralabs/platform#12315 * Apply style guide * fix(cloud): restore dropped lineage bullet, align s3:ListBucket across policies [EDU-1421] Two follow-ups on the Cloud half of #1847: - Restore the "Pipeline output visibility" bullet on the Cloud data lineage page. #1847 merged master in and then dropped it while rewriting the "Why use data lineage" list, silently reverting #1837 (EDU-1425). Restoring it also re-uses the [nextflow-workflow-outputs] link definition, which #1847 left defined but orphaned. - Add s3:ListBucket to the three Automatic-mode lineage policy surfaces that were missing it: aws-batch.mdx, aws-cloud.mdx, and both combined policy JSONs rendered inline and offered for download from those pages. #1847 documents the permission as required — without it, reindex fails with AccessDenied — but only added it to workspace-management.md, so anyone following a compute-env page or downloading the full policy got a set that cannot reindex. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(enterprise,cloud): restore lineage bullet, align Cloud policies with Enterprise [EDU-1421] - Restore the "Pipeline output visibility" bullet on the Enterprise data lineage page, same regression already fixed on the Cloud page: the rewrite of "Why use data lineage" dropped it, reverting #1837 (EDU-1425) and orphaning the [nextflow-workflow-outputs] definition. Lands in the unversioned tree, so it ships with 26.2. - Add s3:GetObject and the object-level resource to the Cloud lineage statements in aws-batch.mdx, aws-cloud.mdx and both combined policy JSONs. This PR's own Enterprise page documents ManageLineageBuckets with s3:GetObject and s3:ListBucket over both seqera-lineage-* and seqera-lineage-*/*, and Platform reads each record from the bucket per Functional flow step 4, but the Cloud surfaces granted bucket-level actions only. The two products' integration credentials now match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Update data-lineage.md Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com> * Update last updated date in data-lineage.md Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com> * docs(enterprise): document lineage IAM on the AWS CE pages Enterprise 26.2 ships data lineage, and an admin must grant SNS topic and lineage-bucket permissions to use it, but neither enterprise compute environment page said so. platform-enterprise_docs/compute-envs/ has never carried any lineage content -- not in 26.1, not anywhere in git history -- so an admin assembling IAM from aws-batch.md or aws-cloud.md got no lineage permissions and no pointer to where they are documented. Cloud already documents this on both its CE pages and in its downloadable combined policies. This mirrors that structure on Enterprise: - aws-batch.md: new "Data lineage (optional)" section alongside the other optional permission sections, plus the matching statement in the "Full permissive policy (for reference)" block, which had no seqera-lineage-* statement at all. - aws-cloud.md: new "Data lineage (optional)" section under Optional permissions. Permissions match ManageNotificationTopics and ManageLineageBuckets as documented on platform-enterprise_docs/data/data-lineage.md, which stays the canonical reference for Manual provisioning. Depends on #1847 for the SNS transport; based on its branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(cloud): drop remaining public preview claims for data lineage The lineage page no longer calls data lineage a preview feature, but two other Cloud surfaces still did, contradicting it within the same product: - orgs-and-teams/workspace-management.md: the Lineage note - platform-cloud.md: the Manage data bullet on the product landing page Enterprise surfaces are deliberately untouched — 26.2 has not shipped and its pages still describe lineage as a preview feature. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com> Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Co-authored-by: Christopher Hakkaart <christopher.hakkaart@gmail.com> Co-authored-by: Justine Geffen <justinegeffen@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the documentation gap identified in EDU-1425: users who enable data lineage find the Outputs tab on the run details page shows only Reports, with no explanation of why the Pipeline outputs sub-tab (showing
FileOutputlineage records) does not appear. Both data lineage and the Nextflow workflow output syntax are required — neither was documented together before.Changes
run-details.mdx(cloud + enterprise)Expanded the Outputs tab section to document both sub-tabs:
tower.ymlconfigurationOutput Name,File Path,Type,Lineage ID,Lineage Labels) and a requirements note explaining that both data lineage and the Nextflow workflow output syntax (Nextflow 24.10.0+) must be enableddata-lineage.md(cloud + enterprise)Added a Pipeline output visibility bullet to the Why use data lineage section, linking the Outputs tab to the benefits of enabling lineage.
pipelines/overview.md(cloud + enterprise)Added a Pipeline outputs and reports subsection under Key features, with links to the Outputs tab documentation, the Reports configuration page, and the data lineage page. This satisfies the acceptance criterion to link to this documentation from the Pipelines section.