Draft
Conversation
Louis-Frey
force-pushed
the
MRB-650-Maps-simplified
branch
from
2185fd6 to
9eb4643
Compare
…oard" This reverts commit cdefa16.
summary statistics. (No changes to code yet.)
of summary statistics.
For Bias, RMSE and MAE map plots.
earthkit instead.
Francesco. Got a long way towards the png plots. Co-authored-by: Francesco Zanetta <francesco.zanetta@meteoswiss.ch>
properly working). Output written to .png now working.
detailed inspection of results at smaller spatial scale.
symmetric colour map for bias.
to see if all of it still works.
and similar style for MAE and RMSE of T2m.
Now works!
Some SLURM optimizations.
Change back (clean up in the end)
Might have to slightly adjust for ICON (emulator).
Solves a Problem that occurred due to the merge.
frazane
reviewed
Mar 18, 2026
Contributor
frazane
left a comment
frazane
left a comment
There was a problem hiding this comment.
Beware you added some log files and some notebooks, I don't know if this was intended!
Contributor
Thanks for pointing out. No, it was not. I will do a cleanup commit at the end where this will be removed. |
The verification files (verif.nc) do now not contain spatial information any more. Instead, this information is read from the forecast and truth data (grib / zarr), aggregated over forecasts and written into a verif_spatial directory for every run. This has the following advantages: - The spatial information (redundant because it already pre-exists in .grib and .zarr files) is now not unnecessarily written to disk. - The spatial verification pipeline is now separated to some degree from the normal one, meaning that the normal one could be run without the heavier spatial one. - aggregation over forecasts now needs to be performed only for the parameters and lead times for which it is actually requested, instead of for all. This speeds up computations when only a specific subset of maps is of interest. Co-authored-by: Francesco Zanetta <francesco.zanetta@meteoswiss.ch>
(not thoroughly tested)
handling of the special case when a map is all-NaN (then a uniform grey map is plotted, with an annotation).
TOT_PREC GRIB values are cumulative totals that are disaggregated via diff(lead_time) inside load_fct_data_from_grib. Loading only the requested step meant the diff had a single element and always returned NaN, producing all-NaN error maps for all seasons. Fix: detect cumulative params via _CUMULATIVE_PARAMS and load the preceding available GRIB step alongside the requested one, so the diff produces a valid result at the target lead time. Also add first-iteration debug logging (fcst/truth/mapped/error shape, range, NaN count) and a per-iteration warning when the error is all-NaN, to make similar issues immediately diagnosable from the logs.
Introduce a dedicated `spatial_all` Snakemake target and a new `--spatial` flag on `evalml experiment`. Spatial verification (aggregation + map plots) is now only triggered on explicit request, keeping the default experiment run lean. Spatial parameters (params, leadtimes, metrics, regions, seasons) are configured via an optional `spatial_verification:` block in the config YAML, backed by a new `SpatialVerificationConfig` Pydantic model with sensible defaults. All existing configs have been updated with the block for easy customisation.
Remove later, only for frl development (feel free to use).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add maps of forecast verification scores
Changes
verif.ncverif.nctemporary to avoid storage of large data volumes