Bug Report: RDM validity time (Option 2) captures local browser string, breaking OpenEO extraction pipeline

Reference Data
1

Context & Issue
I am reporting a localization (L10n) bug in the Reference Data Module (RDM) harmonization frontend that causes a critical failure downstream during the EO data extraction pipeline in JupyterLab.

When harmonizing a private dataset via the RDM, if the user’s browser is set to a non-English locale (e.g., Spanish), the system saves the raw localized date string instead of parsing it to a standard ISO-8601 format.

Steps to Reproduce:

  1. Upload a dataset to the RDM for harmonization.
  2. At Step 2, select “Option 2: Define a single validity time for all observations” and pick a date using the calendar widget.
  3. If the browser is in Spanish, the RDM writes the localized string into the valid_time attribute of the output .gpkg file.
    Example: 31-05-2021 00:00:00 (Hora est. Sudamérica Pacífico)

The Impact (Downstream Failure):
When loading this harmonized .gpkg into the JupyterLab environment and running WorldCerealExtractionsApp.run(), the extraction pipeline eventually crashes.

The pandas library attempts to track the OpenEO jobs via an internal job_tracking.csv. However, the unescaped spaces, parentheses, and special characters in the localized date string break the CSV parsing. This throws the following sequence of errors:

WARNING: Error "Error tokenizing data. C error: EOF inside string starting at row 1", retrying in 10 seconds...

And eventually crashes the monitoring thread:
ValueError: cannot reindex on an axis with duplicate labels

Because the monitoring thread (Thread-10 run_loop) dies silently, the script assumes all tasks are complete, leaving OpenEO jobs orphaned/queued on the CDSE platform and resulting in an incomplete .geoparquet merge.

Expected Behavior:
The RDM backend should sanitize the input from the UI calendar widget and strictly enforce a standard ISO-8601 format (e.g., YYYY-MM-DD) for the valid_time column, regardless of the user’s local browser settings.

Current Workaround:
Before passing the .gpkg to the JupyterLab pipeline, I am manually overriding the valid_time column using a spatial script to force the string into a clean "YYYY-MM-DD" format, which allows the extraction to run perfectly.

Thanks for looking into this!

2

The final parquet output generated - 2024_bra_2023braziltest_point_110.parquet or sample_2024_bra_2023braziltest_point_110.parquet only stores validity time like below image. When you open this in apps liks QGIS, they show it localized. Could you elaborate a bit with screenshots where you are seeing this? Which parquet file. We do not generate .gpkg files

image
image624×543 29.5 KB