Repository navigation
feat: add CompetencyFramework bulk term CSV workflow (validate/commit/download) - #1335
Open
aimansharief wants to merge 57 commits into
Open
aimansharief wants to merge 57 commits into
aimansharief wants to merge 57 commits into
Conversation
feat: add CompetencyFramework bulk term CSV workflow (validate/commit/download)
…d-Knowlg/knowledge-platform into feat/competency-framework
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
…k schema The type field was copied verbatim from the Framework schema, carrying Framework's K-12/TPD enum and a "K-12" default. Those values describe curriculum taxonomies and mean nothing for a competency framework. Nothing reads a framework's type: no business logic in taxonomy-api, search-api or knowledge-platform-jobs branches on it, and NodeUtil's relation projection keys on an exact "framework" objectType match that CompetencyFramework never satisfied. CompetencyFramework is already a distinct object type, so objectType does the discriminating that K-12/TPD did for Framework. Because BaseSchemaValidator fills defaults before validating, a create that omitted type was silently stored as "K-12" -- including the create example in the API reference. Removing it from both properties and required avoids persisting a meaningless value. Framework's own schema is untouched.
19 tests were failing on this branch. 15 of them were test-harness defects, not product bugs; the other 4 need a running Redis. FrameworkActorTest (14): update/review/reject resolve the request-body channel to a Channel node before doing anything else. In production that value arrives from the x-channel-id header via commonHeaders(), but these tests build a bare Request and bypass the controller, so channel was "" -- yielding either ERR_INVALID_CHANNEL_ID or an unmatched mock call. Supply channel and stub the Channel node, as the already-passing tests in this file do. The .img test had a second, distinct cause: VersioningNode.getEditableNode deliberately re-reads ".img" after addNode creates it, so a stub that always reports not-found breaks that re-read. Make it state-aware. TermBulkManagerTest (1): a successful commit surfaces only rows carrying warnings, and the sheet's cm3 create row has no associations and nothing referencing it, so it gets the documented WARN_ORPHAN_TERM -- behaviour a sibling test already asserts. Expect that row instead of none. taxonomy-actors: 189 -> 204 passing; the 4 remaining failures are CategoryCacheTest/FrameworkCacheTest, which require Redis.
Identifiers are uniquely indexed in JanusGraph, so /framework/v3 and /competency-framework/v3 resolve the same vertex for a given id. The controllers differ only in the objectType/schemaName/bodyKey they put in the request context, and that context drives validation and serialization -- never the lookup. Neither endpoint checked what it got back, so each served and mutated the other's objects. Reads are gated at the response exit, after the cache, the hierarchy and the graph converge on one map, so no store can answer without being type-checked. Writes are gated on the node they load instead: by the time there is a response the node is already mutated, which makes a response-level check useless for them. That covers update, retire, review, reject and publish. A mismatch returns 404 rather than 400 -- for that endpoint the id genuinely isn't there. A record carrying no objectType is also rejected: every writer stamps it, so its absence means the record cannot be type-verified. Term APIs are deliberately left alone. They are type-agnostic by design so competency frameworks can hold terms, and FrameworkTermController puts objectType=Term in context. Gating the shared loaders (getLiveEditNode / assertFrameworkEditable) would have compared a framework node against "Term" and broken every term operation, so the check lives in the framework-specific operations and the loaders are untouched. assertFrameworkEditable now returns the node it already loaded so those callers can assert on it; term callers ignore it. No database data or structure is changed. taxonomy-actors: 216 tests, 212 passing; the 4 failures are the pre-existing CategoryCacheTest/FrameworkCacheTest ones that need Redis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bulk/validate→bulk/commitdiffs an uploaded CSV against the current graph state and writes it, attempting every create row regardless of earlier failures (a row that already succeeded stays persisted even if the overall commit is reported failed) — new rows land in Live status, updates/retires apply immediately.bulk/download(GET) rebuilds the current term set as CSV on demand, with category-name placeholder rows when a framework has zero terms yet./competency-framework/v3/...(term create/update/retire, category CRUD, bulk/create) now that the bulk CSV flow covers them — no functionality lost, no other route affected.No new external dependencies — CSV parsing reuses commons-csv, already transitively resolved via actor-core → platform-common (same path collection-csv-actors uses); cloud storage reuses the existing StorageService.
Type of change
How Has This Been Tested?
mvn test -pl taxonomy-apiTest Configuration:
Checklist: