Skip to content

Alignment of types and their docs - #9234

Merged
mandy-chessell merged 2 commits into
odpi:mainfrom
mandy-chessell:oak2026
Aug 21, 2026
Merged

Alignment of types and their docs#9234
mandy-chessell merged 2 commits into
odpi:mainfrom
mandy-chessell:oak2026

Conversation

@mandy-chessell

Copy link
Copy Markdown
Contributor

Align the open metadata types with their documentation, and add type-fvt

Working through the UML models in egeria-docs/site/docs/types page by page and comparing them against
the types this repo actually builds turned up a long tail of drift: attributes the beans read and wrote
but no type declared, attributes whose data type disagreed with the bean, relationship ends named
differently in the type system than in the API, and types drawn and documented but never implemented.

This PR fixes what it found, implements the types that were missing, and adds an FVT suite so the whole
model stays honest from now on. A companion PR in egeria-docs carries the diagram and markdown side.

Properties that existed everywhere except the type system

Each of these was readable and writable through the bean, builder and converter, and drawn on the
diagram, but no type declared it - so the value was silently dropped on the way to the repository.

  • embeddedMetadata added to DocumentStore and MediaCollection. MediaFile, the third type using
    it, already declared it.
  • securityRoles added to the SecurityListMembership classification, which previously wrote
    securityGroups only.
  • TypeEmbeddedAttribute's three missing attributes.
  • NetworkGatewayLink rebuilt: it now extends RelationshipBeanProperties and declares
    displayName, description, externalEndpointAddresses and internalEndpointAddresses. Its branch in
    OpenMetadataRelationshipBuilder was nested inside an instanceof LabeledRelationshipProperties check
    that could no longer be true, so it produced no properties at all - OpenMetadataTypeTest caught that.

Attribute types corrected

OpenMetadataProperty is the single definition, so these were fixed at source and the archive
regenerated rather than patched (a TypeDefPatch cannot change an attribute's type - the repository
rejects it with INCOMPATIBLE_PROPERTY_PATCH).

  • externalEndpointAddresses / internalEndpointAddresses - string to array<string>
  • size - int to long; securedProperties - to map<string,string>; securityProperties - to
    map<string,object>
  • the six CodeAnalysisAnnotation counters - int to long
  • timezone renamed to timeZone on Person and FixedLocation, matching the beans and diagrams. The
    old spelling stays as a RENAMED_ATTRIBUTE pointing at the new one so existing instances follow the
    value across the change.

Two attributes were removed rather than kept: SecurityRole.distinguishedName, which duplicates the
declaration it inherits from SecurityList (its sibling SecurityGroup never had one), and
Topic.topicName.

Relationship end names

Three sources name the ends of a relationship - the endDef attribute names, the field names in
AttributedMetadataElement, and the labels on the diagrams - and they had drifted apart. The API field
names are what users see in the JSON, so they win, except where one field serves several relationships,
where the diagram decides.

102 endDef names were renamed on that basis. Of the 149 relationships that have a field in
AttributedMetadataElement, 261 of their 282 ends now carry the API field name exactly; the 21
exceptions are the shared fields (parentSchemaElements alone serves seven relationships).

Two constraints are worth recording, both enforced by OMRSArchiveBuilder and both fatal at build time
rather than merely untidy:

  • an end name must be unique within the entity it hangs off, and
  • it must not collide with a property that entity already declares - Regulation has its own
    regulators attribute, so Regulator.endDef2 stays assignedRegulators and the diagram changed
    instead.

Note that the builder registers each end name in the other end's namespace, so a collision is reported
against a type that does not appear in the name at all.

New types

0280 / 0281 / 0282 - software development assets (8 types, previously drawn but not built):
GeneratedTarget and ReusableTechnique classifications and the ReusableTechniqueUse relationship;
SoftwareComponent under Collection and SoftwareModule under that; RunnableSoftwareComponent under
Asset, with DependentSoftwareComponent and SoftwareSource. Four new properties (buildId,
buildTool, buildToolVersion, runtimeEnvironmentType), eight beans in a new softwaredevelopment
package, and builder/converter branches for the two types carrying attributes.

The relationships and classifications are exposed through the devops-pipeline OMVS - a new
SoftwareDevelopmentHandler plus ten REST endpoints and .http samples. The three entities need no new
API, being Collection and Asset subtypes that collection-manager and asset-maker already serve.

RunnableSoftwareComponent was drawn with a version : string attribute, which is not implemented:
version is the instance header's own property and is a long there, so a string attribute of the same
name collides in the API JSON, and Referenceable already declares versionIdentifier, which the type
inherits. No type in the model declares version. The diagram was corrected instead.

0462 - governance action process specialisations (5 types): AnalyticalActionProcess,
CataloguingActionProcess, ExploringActionProcess, SurveyingActionProcess and
ProvisioningActionProcess, all extending GovernanceActionProcess. These are now used in place of the
generic type by the content pack writers, the Coco samples, the Unity Catalog pack, the Jacquard and
subscription connectors, and createProcessFromGovernanceActionType, which gains a processType
parameter.

Nine stale page links

OpenMetadataWikiPages had three constants pointing at pages that no longer exist
(MODEL_0013_ACTIONS, MODEL_0451_NOTIFICATIONS, MODEL_0457_CONNECTOR_ACTIVITY_REPORTS), plus
MODEL_0145_PERSPECTIVES; a new MODEL_0111_CONTACT_DETAILS was added, and nine types were repointed at
the page they are actually drawn on. Fixing the three broken constants brought previously unreachable
pages into the comparison.

New: type-fvt

A functional verification suite for the type system itself. Where query-fvt asks "does the query
surface behave correctly?"
, this asks "is every type in the model actually usable?" For every entity,
classification and relationship type it creates an instance with every declared attribute populated,
reads it back through the connector context, checks each property survived, updates it, and removes it.

./gradlew :open-metadata-test:open-metadata-fvt:type-fvt:test -PrunTypeFvt

634 test cases, one per type, so a failure names the type rather than just "something is broken":
342 entity types, 200 relationships, 86 classifications, plus 6 whole-model checks. About two minutes
against a local PostgreSQL server.

Nothing is hand-listed. The type list is read from the running server - not the archive file, because a
type's real shape is newTypeDefs merged with every later typeDefPatch and the server has already done
that merging. A type added to the model becomes a test case on the next run with no edit to the
suite
, which is how the five 0462 types and the eight software-development types were verified.

The whole-model checks are the ones that catch a model that compiles and loads but is quietly wrong:
every supertype resolves; every relationship end and classification target resolves; no two types share a
GUID; name and GUID lookups agree; and every OpenMetadataType constant names a type the server has,
with the same GUID
- the check that keeps the Java view and the archive from drifting apart.

It shares query-fvt's shape (in-process platform, no Kafka, PostgreSQL repository, permit-all chain) on
port 9447 and its own schema, so the two can run side by side, and it is opt-in - skipped by a normal
build.

Two platform behaviours it flushed out, both correct, both worth knowing: a purged element is reported
as an unknown GUID
, not an empty result, so a "check it is gone" step must expect a 404; and
Memento hides its element from ordinary calls, including the lookup a detach performs first, so
forLineage is needed on the write options as well as the reads.

Testing

  • OpenMetadataTypeTest passes - run after every archive and bean change.
  • type-fvt: 634 cases, 0 failures.
  • Archive and content packs regenerated; devops-pipeline builds; javadoc clean.

mandy-chessell and others added 2 commits August 21, 2026 23:46
Signed-off-by: Mandy Chessell <mandy.e.chessell@gmail.com>
@mandy-chessell
mandy-chessell merged commit e0af575 into odpi:main Aug 21, 2026
4 checks passed
* that fail: a type that cannot be created through the connector context, and is not listed here, is
* a finding this suite exists to report.
*/
private static final Map<String, String> UNINSTANTIABLE_TYPES = new LinkedHashMap<>();
* Empty today - kept so that a genuine exclusion has an obvious, documented home rather than being
* hidden inside a test.
*/
private static final Map<String, String> UNLINKABLE_RELATIONSHIPS = new LinkedHashMap<>();
@Bean
public SecurityFilterChain typeFvtSecurityFilterChain(HttpSecurity http) throws Exception
{
http.csrf(csrf -> csrf.disable())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants