monoscope/test/integration/Pages
Anthony Alaribe 924e8e4002
Query cache (#252)
* add intelligent query cache for KQL timeseries queries

- Cache summarize queries with bin/bin_auto time binning
- Store results in UNLOGGED PostgreSQL table for performance
- Delta fetch on partial cache hits (only query new time range)
- Merge cached + new data, deduplicate by timestamp
- 24h sliding window eviction via scheduled cleanup job
- Per-project cache namespace with query hash + bin interval key

* Auto-format code with fourmolu

* fix query cache issues and improve code quality

- Fix race condition: use LEAST/GREATEST for cache time range merging
- Fix partial hit logic: handle requests extending both directions
- Add LIMIT 1 to cache lookup query
- Use existing toXXHash from Utils instead of manual xxHash call
- Single-pass stats calculation with foldl'
- Use liftA2 for Maybe operations, comparing for point-free style
- Use viaNonEmpty from Relude instead of Safe package

* add hourly cache eviction to prevent unbounded growth

- Add cleanupExpiredCache function to remove stale entries
- Evict entries where cached_to > 24h ago or last_accessed_at > 1h ago
- Integrate with HourlyJob in BackgroundJobs
- Log cleanup count when entries are deleted

* Auto-format code with fourmolu

* refactor query cache for conciseness and correctness

- Use <&> and LambdaCase for cleaner lookupCache
- Extract toPosix helper for time conversions
- Use groupBy for declarative deduplication
- Remove redundant index (UNIQUE constraint creates it)
- Increase LRU threshold to 4h (from 1h)
- Remove redundant Parser qualified import in Charts.hs

* Auto-format code with fourmolu

* fix query cache stats calculation for multi-column data

- recalculateStats now processes all value columns, not just the first
- maxGroupSum correctly sums all values per row (timestamp)
- filterByTimestamp recalculates stats after filtering
- add integration tests for cache hit, partial hit, and multi-column stats

* Auto-format code with fourmolu

* fix cache metadata mismatch and backward-extending request handling

- Use slidingWindowStart instead of entry.cachedFrom when updating cache
  after trimming, ensuring metadata matches actual data range
- Remove PartialHit case for requests extending both directions, as only
  forward delta was fetched leaving backward data missing

* optimize query cache table for performance

- Use composite primary key instead of UUID PK + unique constraint
- Remove FK constraint (orphans cleaned by LRU eviction)
- Use JSON instead of JSONB (faster writes, always read whole blob)
- Add fillfactor=70 for HOT updates on hit_count/last_accessed_at
- Add aggressive autovacuum settings for high-update table
- Simplify eviction to just LRU (remove redundant cached_to check)
- Remove unnecessary indexes (table is small, hourly eviction)

* fix query cache correctness and performance issues

- Replace LEAST/GREATEST with direct replacement in updateCache to fix
  metadata mismatch after trimOldData removes old entries
- Make sliding window configurable based on bin interval (48 points or 24h min)
- Cap mode frequency map to 1000 entries for high-cardinality data
- Use vector ifilter for deduplication instead of list groupBy conversion

* Auto-format code with fourmolu

* use Relude words and one instead of qualified imports

* refactor: simplify QueryCache code

- Simplify parseBinIntervalToSeconds using unitToSeconds helper
- Extract rowTimestamp helper to dedupe join . V.headM usage
- Replace case () with multi-way if in lookupCache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
History 2026-01-01 14:51:17 +01:00
..
Endpoints Github 2-way sync with a repo (#244) 2025-12-26 01:19:12 +01:00
LogExplorer Github 2-way sync with a repo (#244) 2025-12-26 01:19:12 +01:00
Projects Alerts view & Error details page (#225) 2025-12-23 17:54:20 +01:00
AnomaliesSpec.hs Github 2-way sync with a repo (#244) 2025-12-26 01:19:12 +01:00
ApiSpec.hs Fix GHC 9.12 TypeAbstractions errors in test files 2025-11-27 23:29:55 +01:00
BusinessFlowsSpec.hs Github 2-way sync with a repo (#244) 2025-12-26 01:19:12 +01:00
DashboardsSpec.hs feat: add folder field to new dashboard modal 2025-12-26 16:31:11 +01:00
GitSyncSpec.hs fix: use relative paths in GitSync test assertions 2025-12-26 18:55:52 +01:00
MonitorsSpec.hs Teams rebased (#233) 2025-12-18 23:38:00 +01:00
QueryCacheSpec.hs Query cache (#252) 2026-01-01 14:51:17 +01:00
ReportsSpec.hs Github 2-way sync with a repo (#244) 2025-12-26 01:19:12 +01:00
ShareSpec.hs Refactor integration tests to use simplified testServant helper 2025-11-17 09:04:28 +01:00