Operationalize/Finalize PDG Viz Workflow on Kubernetes

[alonakos]

The objective of this ticket is to make the PDG Viz Workflow fully operational on Kubernetes (K8s). Deploying on K8s will allow for scalability across different clusters, and converting the workflow into a service will enable team members to independently trigger and manage various stages, moving away from the current manual process.

Description

The objective is to deploy the entire Viz Workflow on K8s.

Primary Issue: Viz Workflow #1

Subtasks and Dependencies:: Certain issues are prerequisites for achieving full K8s deployment:
Issue #33
Issue #46

Modularize Configuration: Currently, the workflow uses a single configuration file, which limits independent deployment of individual stages. To address this, we need to create separate configurations for each stage.

• Coordinate with @Rushiraj to ensure alignment on configuration modularization.

Complete Processing of the Ice Wedge Polygon Dataset: Finalizing the processing of the IWP dataset.

• Additional Requirements: WMTS Capabilities Document: Update viz-raster to generate WMTS capabilities documents to support subset downloads. (completed)

TODO:

• Check with @Rushiraj about the API design document to confirm alignment.
• Begin testing with a smaller, simpler dataset to ensure functionality before moving to the IWP dataset.