MLH-1151 | optimise lineage calculation

[aarshi0301]

Change description

Description here

• this is behind a feature flag
• only for tenants which are enabled will see these changes

This code is an alternative to the previous implementation with improved performance:

Iterator<AtlasEdge> edgeIterator = assetVertex.query()
        .direction(AtlasEdgeDirection.BOTH)
        .label(PROCESS_EDGE_LABELS)
        .has(STATE_PROPERTY_KEY, ACTIVE.name())
        .edges()
        .iterator();

int processHasLineageCount = 0;

while (edgeIterator.hasNext()) {
    AtlasEdge edge = edgeIterator.next();
    if (!RequestContext.get().getDeletedEdgesIdsForResetHasLineage().contains(edge.getIdForDisplay()) && !currentEdge.equals(edge)) {
        AtlasVertex relatedProcessVertex = edge.getOutVertex();
        boolean processHasLineage = getEntityHasLineage(relatedProcessVertex);
        if (processHasLineage) {
            processHasLineageCount++;
            break;
        }
    }
}

if (processHasLineageCount == 0) {
    AtlasGraphUtilsV2.setEncodedProperty(assetVertex, HAS_LINEAGE, false);
}

Testing Notes

Initial Lineage Flows

• Table1 → ProcessA → Table2
  Status: Table1: true | Table2: true | ProcessA: true

• Table0, Table1 → ProcessA → Table2
  Status: Table0: true | Table1: true | Table2: true | ProcessA: true

Deletion Scenarios

• Delete Table2
  Status: Table0: false | Table1: false | Table2: true | ProcessA: false

• Table0 → Process0 → Table1 → ProcessA → Table2
  Status: Table0: true | Table1: true | Process0: true | ProcessA: true | Table2: true

• Delete Table2
  Status: Table0: true | Process0: true | Table1: true | ProcessA: false | Table2: true

• Table1 → ProcessA → Table2, Delete ProcessA
  Status: Table1: false | Table2: false | ProcessA: true

Type of change

• Bug fix (fixes an issue)
• New feature (adds functionality)

Related issues

Fix #1

Helm Config Changes for Running Tests (Staging PR)

Does this PR require Helm config changes for testing?

• Tests are NOT required for this commit. (You can proceed with the PR.) ✅
• No, Helm config changes are not needed. (You can proceed with the PR.) ✅
• Yes, I have already updated the config-values on enpla9up36. (You can proceed with the PR.) ✅
• Yes, but I have NOT updated the config-values. (Please update them before proceeding; or, tests will run with default values.)⚠️

Checklists

Development

• Lint rules pass locally
• Application changes have been tested thoroughly
• Automated tests covering modified code pass

Security

• Security impact of change has been considered
• Code follows company security practices and guidelines

Code review

• Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
• "Ready for review" label attached and reviewers assigned
• Changes have been reviewed by at least one other contributor
• Pull request linked to task tracker where applicable

---

Note

Optimizes lineage recalculation using Gremlin traversal with edge object-ID filtering and adds RequestContext support for deleted edge object IDs; also enables CI for an additional branch.

• Lineage recalculation performance:
  • Update DeleteHandlerV1.updateAssetHasLineageStatusV2(...) to filter edges by object IDs and use Gremlin traversal (hasId(P.without(...)), outV().has(HAS_LINEAGE, true).limit(1).hasNext()) instead of streaming projections.
  • Exclude current/removed edges using actual TinkerPop IDs.
• Request context enhancements:
  • Add deletedEdgesObjectIdsForResetHasLineage tracking with addToDeletedEdgesObjectIdsForResetHasLineage(...) and getDeletedEdgesObjectIdsForResetHasLineage(); clear it in clearCache().
  • Adjust callers to add removed edges by object ID.
• CI:
  • Enable workflow on branch mindbodylineage in .github/workflows/maven.yml.

^{Written by Cursor Bugbot for commit 865b2c7. This will update automatically on new commits. Configure here.}

[Copilot]

Pull Request Overview

This PR optimizes lineage calculation performance in the DeleteHandlerV1 class by replacing a stream-based approach with a more efficient Gremlin traversal pattern. The optimization moves from materializing all edges and checking them in memory to using native graph database operations for filtering.

• Replaces .project() and .toStream().anyMatch() pattern with direct Gremlin filtering
• Uses native graph traversal operations (hasId, not) instead of materializing results in memory
• Maintains the same functional behavior while improving performance for lineage calculations

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

This loop modifies the traversal object repeatedly, which could create performance issues if there are many deleted edge IDs. Consider building a single filter condition or using a more efficient batch filtering approach to avoid multiple traversal modifications.

Suggested change

	// Filter out current edge using Gremlin
	traversal = traversal.where(not(hasId(currentEdge.getIdForDisplay())));
	for (String deletedEdgeId : RequestContext.get().getDeletedEdgesIdsForResetHasLineage()) {
	traversal = traversal.where(not(hasId(deletedEdgeId)));
	}
	// Filter out current edge and all deleted edges using a single Gremlin filter
	Set<String> excludedEdgeIds = new HashSet<>();
	excludedEdgeIds.add(currentEdge.getIdForDisplay());
	excludedEdgeIds.addAll(RequestContext.get().getDeletedEdgesIdsForResetHasLineage());
	traversal = traversal.where(not(hasId(excludedEdgeIds.toArray(new String[0]))));

[sriram-atlan]

i dont see this change behind FF @aarshi0301