History range queries are off by the UTC offset (timestamps compared as local-time text) #11

Closed
opened 2026-10-28 11:38:12 +01:00 by pti · 0 comments
Owner

Problem

The history range queries compare timestamps as text, and the timestamps are stored in local time. The range bounds come from the API in whatever offset the caller sends, usually UTC (…Z). The range is then off by the UTC offset (1 h in winter, 2 h in summer).

The modernc SQLite driver stores time.Time as t.String(), e.g. 2026-10-28 07:58:08.038 +0100 CET m=+35740.25. The repositories query WHERE timestamp >= ? AND timestamp <= ? with the bound time.Time from parseTimeRangeParams. For …Z input that is 2026-10-28 07:03:18 +0000 UTC, which compares as text against the local-time strings.

Seen

  • /api/history/heating?start=…&end=2026-10-28T07:03:18Z misses the switch-off at 07:58:08 CET (06:58:08 UTC), which is inside the range.
  • The 30-day temperature export run at 09:33 local time ended at 08:33: the last hour was missing.
  • Callers sending the local offset (+01:00) get the right rows, which is how the release watcher works around it.

Affects all range queries: temperature history, heating history, events, anticipation samples and coast records. It also hits the history graph, the CSV/JSON exports and the analysis in #8. Around the DST change, local-time text also stops sorting correctly for an hour. ORDER BY timestamp has the same problem; the restore code already orders by id for that reason.

Fix options

  • Small: convert range bounds to time.Local in the repositories before querying. The text then compares like-for-like, except in the DST hour.
  • Proper: store timestamps in a sortable, zone-independent form, e.g. UTC RFC 3339 text or Unix milliseconds. The driver's _time_format DSN parameter can set the format. Existing rows (back to 2019) need a migration to rewrite them, parsing the t.String() text and dropping the m=+… monotonic suffix.
## Problem The history range queries compare timestamps as text, and the timestamps are stored in local time. The range bounds come from the API in whatever offset the caller sends, usually UTC (`…Z`). The range is then off by the UTC offset (1 h in winter, 2 h in summer). The modernc SQLite driver stores `time.Time` as `t.String()`, e.g. `2026-10-28 07:58:08.038 +0100 CET m=+35740.25`. The repositories query `WHERE timestamp >= ? AND timestamp <= ?` with the bound `time.Time` from `parseTimeRangeParams`. For `…Z` input that is `2026-10-28 07:03:18 +0000 UTC`, which compares as text against the local-time strings. ## Seen - `/api/history/heating?start=…&end=2026-10-28T07:03:18Z` misses the switch-off at 07:58:08 CET (06:58:08 UTC), which is inside the range. - The 30-day temperature export run at 09:33 local time ended at 08:33: the last hour was missing. - Callers sending the local offset (`+01:00`) get the right rows, which is how the release watcher works around it. Affects all range queries: temperature history, heating history, events, anticipation samples and coast records. It also hits the history graph, the CSV/JSON exports and the analysis in #8. Around the DST change, local-time text also stops sorting correctly for an hour. `ORDER BY timestamp` has the same problem; the restore code already orders by `id` for that reason. ## Fix options - **Small:** convert range bounds to `time.Local` in the repositories before querying. The text then compares like-for-like, except in the DST hour. - **Proper:** store timestamps in a sortable, zone-independent form, e.g. UTC RFC 3339 text or Unix milliseconds. The driver's `_time_format` DSN parameter can set the format. Existing rows (back to 2019) need a migration to rewrite them, parsing the `t.String()` text and dropping the ` m=+…` monotonic suffix.
pti closed this issue 2026-10-29 14:39:55 +01:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
pti/blubgo#11
No description provided.