Commit graph

127 commits

Author SHA1 Message Date
Anthony Alaribe
7bd351203a
Add widget-integrated alerts feature (#272)
* Add widget-integrated alerts feature

Enable creating and managing alerts directly from dashboard widget details:
- Add widget_id, dashboard_id, and recovery threshold columns to query_monitors
- Add Alerts tab to widget expand view with full configuration form
- Render threshold marklines on charts (configurable: always/on_breach/never)
- Show alert status indicator on widget headers (bell icon with colors)
- Add source column to Alerts page showing Widget vs Log Explorer origin
- Sync widget query changes to linked alerts automatically
- Delete alerts when widgets are removed from dashboards
- Support recovery thresholds (hysteresis) to prevent alert flapping
- Populate widget alert status at render time for real-time indicators

* Auto-format code with fourmolu

* Fix widget alerts: clone on duplicate, sync queries, persist status

- Clone alerts when duplicating widgets with new alert ID tied to new widget
- Sync widget query changes to associated alert's logQuery and logQueryAsSql
- Prevent direct query modification for widget-tied alerts
- Add current_status column to track alert state (migration 0026)
- Extract normalizeWidgetId, alertActiveWindow, mkMarkLine helpers
- Use stored current_status in getWidgetAlertStatuses instead of computing

* Auto-format code with fourmolu

* Fix widget alerts: add missing current_status field and lowercase status values

- Add current_status to SQL queries in getAlertsByTeamHandle, getActiveQueryMonitors,
  queryMonitorByWidgetId, queryMonitorsByDashboardId
- Fix case sensitivity: use lowercase status values to match SQL constraint
- Remove redundant pass statement, simplify wasAlerting/wasWarning with helper

* Cache current_value to avoid eval() on every dashboard load

- Add current_value column to query_monitors table
- Store evaluated value during background job instead of computing on read
- Replace eval(log_query_as_sql) with cached current_value in getWidgetAlertStatuses
- Remove unnecessary null guard from deleteMonitorsByWidgetIds
- Use foldMap for statusMap construction

* Simplify monitor status logic and extract updateTabBySlug helper

- Refactor monitorStatus to use breached/recovered predicates instead of
  duplicated comparison logic for triggerLessThan
- Use currentStatus field directly instead of time-based alerting checks
- Extract updateTabBySlug helper in Dashboards to reduce code duplication

* Auto-format code with fourmolu

* Consolidate widget alert migrations into single file

* Move showThresholdLines from monitor to widget configuration

showThresholdLines is a display setting that belongs in the widget JSON,
not the monitor database table. This change:
- Removes show_threshold_lines column from query_monitors table
- Updates widgetAlertUpsertH to save showThresholdLines to the widget
- Widget already has this field in its JSON schema

* Use DOUBLE PRECISION for threshold and metric values

Support decimal values (e.g., 0.5, 32.5°C) in monitor thresholds:
- Migration: Convert alert_threshold, warning_threshold to DOUBLE PRECISION
- Migration: New columns use DOUBLE PRECISION (recovery thresholds, current_value)
- Haskell: Update all threshold/value fields from Int to Double
- Remove fromIntegral conversions in BackgroundJobs

* Code cleanup: simplify BackgroundJobs and add NOT NULL constraints

- Simplify sum/map to list comprehension: sum [v | Only v <- results]
- Inline wasAlerting/wasWarning in monitorStatus call
- Use void consistently instead of _ <- pattern
- Simplify breached/recovered predicates in monitorStatus
- Use && instead of , in guards for clarity
- Add NOT NULL to current_status and current_value columns

* Auto-format code with fourmolu

* Fix: remove stale show_threshold_lines from SELECT queries

* Fix: rename 'first' variable to avoid shadowing Bifunctor.first

* Refactor: use _select from pg-entity instead of manual column lists

* Refactor: address code review feedback for alerts-in-widgets

- Add MonitorStatus ADT (MSNormal/MSWarning/MSAlerting) replacing string literals
- Simplify applyAlertStatus using fromMaybe/do pattern
- Add warning log when widget alert query parsing fails
- Fix race condition by deleting monitors before dashboard update
- Normalize widgetId once at function entry in dashboardWidgetPutH

* Auto-format code with fourmolu

* Fix: format threshold values with thousand separators

* Refactor: extract formatWithCommas to Utils.hs

* Refactor: extract shared alert UI helpers to Pages/Monitors.hs

- Add thresholdInput_, recoveryInput_, directionSelect_, frequencySelect_, collapsibleSection_
- Update Dashboards.hs widgetAlertConfig_ to use shared helpers
- Update Log.hs alertConfigurationForm_ to use shared helpers
- Net reduction of ~37 lines through DRY refactoring

* Auto-format code with fourmolu

* Style: use surface-raised for alert form in Log.hs

* Add integration tests and doctests for alert monitoring

- Add doctests to monitorStatus for hysteresis logic validation
- Add widget alert cascade deletion test
- Add widget alert query sync test
- Add monitor with recovery thresholds integration test
- Export checkTriggeredQueryMonitors and monitorStatus from BackgroundJobs
- Fix missing AlertUpsertForm fields in MonitorsSpec

* Refactor: extract functions and fix hlint warnings

- Extract syncWidgetAlert, normalizeWidget, updateDashboardWidgets from dashboardWidgetPutH
- Extract evaluateQueryMonitor with error handling for SQL failures
- Fix hlint warnings: use toText, one, and function composition

* Auto-format code with fourmolu

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-07 18:13:18 +01:00
Anthony Alaribe
a0ae548d33 Fix query parsing to detect invalid queries that return empty AST
The parser uses sepBy which returns [] on parse failure instead of
an error. Now we detect when user provided a non-empty query but
got empty AST, treating this as a parse error and returning 0 results.

Updated the "invalid query syntax" test to expect 0 results instead
of all results, making it consistent with other error handling tests.
2026-01-06 20:22:58 +01:00
Anthony Alaribe
bf6faa8cd9
Improve anomaly detail page layout and add AI chat (#249)
* Improve anomaly detail page layout and add AI chat

- Move action buttons (acknowledge, archive) to navbar as page actions
- Remove surface-raised from header, add more breathing room
- Update button styles to match navbar patterns (btn-sm)
- Add time stat boxes showing first/last seen with compact formatting
- Add AI chat functionality for investigating anomalies
- Add side-chevron-left icon (without box) for collapse toggle
- Use new icon for trace panel collapse button

* Auto-format code with fourmolu

* Refactor AI chat to use agentic tools and DB-persisted history

- Extend AgenticConfig with conversationId and conversationType fields
- Add runAgenticChatWithHistory for DB-backed conversations
- Refactor Anomalies.hs to use agentic query system (access to 8 tools)
- Add Slack/Discord thread persistence in DB instead of repeated API calls
- Add deterministic UUID5 helpers for thread conversation IDs
- One-time migration pattern: first thread access migrates from external API
- Log Explorer remains stateless for low latency

* Auto-format code with fourmolu

* Fix security issues and improve AI chat persistence

Security fixes:
- Use executeSecuredQuery for dashboard constants/variables SQL
- Add project_id filtering to prevent cross-project data access

AI chat improvements:
- Save user message before AI call for complete history
- Persist assistant response on error and parse failure paths
- Fix race condition in getOrCreateConversation (use listToMaybe)

Code style (CLAUDE.md):
- Remove unnecessary $ before lambdas in Anomalies.hs
- Clean up unused imports in Dashboards.hs

* Auto-format code with fourmolu

* Refactor time display and error handling for cleaner code

- Add abbreviateUnit helper with LambdaCase for cleaner word mapping
- Simplify compactTimeAgo using word-based abbreviation
- Refactor timeStatBox_ to use pattern matching on T.words
- Replace fromMaybe.error with explicit pattern match in getOrCreateConversation

* Auto-format code with fourmolu

* Fix hlint warnings in Anomalies and AI modules

- Replace T.unwords/T.words/T.unlines with Relude equivalents
- Simplify lambdas to operator sections
- Replace pure () with pass

* Optimize AI and anomaly code with better data structures

- Use Map for O(1) time unit abbreviation lookup in Anomalies.hs
- Extract dangerous SQL patterns to top-level constant with additional
  security patterns (copy, execute, explain, set)
- Replace V.toList conversions with pure Vector operations using vIntercalate
- Use V.take instead of take after V.toList for efficiency
- Reorder migration files for correct dependency order

* Auto-format code with fourmolu

* Optimize AI and anomaly code with better data structures

- Use Map for O(1) time unit abbreviation lookup in Anomalies.hs
- Extract dangerous SQL patterns to top-level constant with additional
  security patterns (copy, execute, explain, set)
- Replace V.toList conversions with pure Vector operations using vIntercalate
- Use V.take instead of take after V.toList for efficiency
- Reorder migration files for correct dependency order

* Auto-format code with fourmolu

* Use chunksOf and parseMaybe for cleaner code in Anomalies.hs

- Replace manual groupMessages recursion with chunksOf from extra
- Use parseMaybe from aeson instead of case on fromJSON result

* Fix security and code quality issues

- Fix SQL injection risk: wrap query in transaction with READ ONLY mode,
  add comment markers (--,/*,*/) to dangerous patterns validation
- Replace unsafe error call with safe fromMaybe in getOrCreateConversation
- Extract textToConversationId helper for cleaner UUID V5 generation
- Remove Control.Lens import, use record update syntax instead
- Add manage_updated_at trigger migration for ai_conversations table
- Update parser test expectations to match current output format

* Auto-format code with fourmolu

* Fix bugs and improve code quality in anomaly/issues modules

- Fix race condition in getOrCreateConversation: fail explicitly if
  RETURNING clause doesn't return a row instead of silent fallback
- Replace Map lookup with pattern matching for time unit abbreviation
- Fix V.take order: take before map for better performance
- Make chat history pairing more robust with explicit recursion
- Add doctests for abbreviateUnit function
- Remove obvious comments, keep only meaningful ones
- Add updated_at trigger for ai_conversations table

* Auto-format code with fourmolu

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-03 18:32:18 +01:00
Anthony Alaribe
cde6fc5c88
Enable agentic tool calling for query generation (#251)
* feat: Add agentic tool calling for AI query generation

Introduces an agentic LLM module that allows the AI to run exploratory
queries before generating the final KQL query. This improves accuracy
when users ask about specific data that the LLM may not know about.

Key changes:
- New Pkg.AI.Agentic module with tool definitions and execution
- Tools: GetFieldValues, GetServices, CountQuery, SampleLogs
- Context-aware mode selection (agentic vs fast mode)
- Web explorer uses agentic mode for complex queries
- Bot handlers continue using fast mode for lower latency

The system intelligently decides when to use tools vs direct response
based on query complexity and context (web UI vs bots).

* Auto-format code with fourmolu

* feat: Use FastMode by default with precomputed facet context

Changes:
- WebExplorer now uses FastMode by default for faster responses
- Pre-include facet context (services, status codes, etc.) in prompts
- Add GetFacets tool for agentic mode when deeper exploration needed
- Fetch precomputed facets in aiSearchH and pass to agentic config
- LLM now has real project data context without needing tool calls

This makes queries faster while still providing accurate results by
giving the LLM actual field values (services, methods, status codes)
from the precomputed facets stored in the database.

* Auto-format code with fourmolu

* refactor: Use KQL instead of raw SQL for agentic tool execution

- Update executeGetFieldValues to use KQL summarize query
- Update executeCountQuery to use selectLogTable for count queries
- Update executeGetServices to use KQL summarize by resource.service.name
- Update executeSampleLogs to use selectLogTable with KQL parsing
- Add helper functions for formatting query results
- Add Time and Log effects to tool execution constraints
- Remove unused SQL query building helpers

This change ensures all tool queries go through the KQL infrastructure,
enabling caching and optimizations built into the KQL workflow.

* Auto-format code with fourmolu

* refactor: Add RunQuery tool and apply code review improvements

Key changes:
- Add RunQuery tool that supports both KQL (preferred) and raw SQL
  for complex queries that KQL cannot express
- Add security validation for SQL queries (SELECT only)
- Extract constants (maxFieldValues, maxSampleLogs, etc.)
- Add helper functions (successResult, errorResult, getTextArg, etc.)
  to reduce code duplication in tool execution
- Apply LambdaCase extension for cleaner pattern matching
- Use mapMaybe instead of filter + map for better efficiency
- Extract keyFacetFields as a constant
- Reduce overall code complexity from ~289 lines to ~233 lines

The LLM can now choose between KQL and SQL based on query complexity,
with strong preference for KQL due to its caching and optimizations.

* Auto-format code with fourmolu

* security: Inject project_id filter into SQL queries

- Add injectProjectFilter to automatically add project_id filter
- Handles queries with and without existing WHERE clauses
- Wraps existing WHERE conditions in parentheses for correct precedence
- Updates tool description to inform LLM that project_id is auto-injected

This prevents any cross-project data access when using raw SQL queries.

* security: Use parameterized CTE for SQL query project filtering

Instead of string manipulation to inject project_id (which was vulnerable
to SQL injection), use a CTE wrapper with parameterized queries:

- Add executeSecuredQuery that wraps user queries in a CTE
- The CTE shadows otel_logs_and_spans with a pre-filtered version
- project_id is passed as a parameterized query value
- Remove unsafe injectProjectFilter and findInsertPoint functions
- Clean up unused imports

This approach is robust because:
1. project_id cannot be injected - it's a query parameter
2. Any query referencing otel_logs_and_spans gets the filtered version
3. No brittle string parsing of WHERE clauses

* Auto-format code with fourmolu

* fix: Add missing runKqlQuery helper and reduce tool duplication

- Add runKqlQuery helper function that was removed by linter
- Refactor executeCountQuery to use runKqlQuery helper
- Refactor executeGetServices to use runKqlQuery helper
- Refactor executeSampleLogs to use runKqlQuery helper
- Remove duplicate/leftover code from partial merge
- Net reduction of ~114 lines through DRY refactoring

* perf: Replace CTE with direct WHERE injection for project_id

CTEs can cause performance issues on large tables because PostgreSQL
may materialize the entire CTE before applying user predicates.

New approach:
- Inject "project_id = ?" directly into the WHERE clause
- project_id value is still parameterized (safe from SQL injection)
- limit is also parameterized
- Query structure manipulation is for WHERE placement only, not values
- PostgreSQL can now use indexes on project_id efficiently

* security: Block UNION/EXCEPT/INTERSECT to prevent project_id filter bypass

- Add UNION/EXCEPT/INTERSECT to blocked SQL patterns since they could
  allow querying data from other projects in the second query
- Improve keyword detection with word boundary checking (prefix, space,
  newline, tab) instead of just substring matching
- Provide specific error message for bypass attempts vs DDL/DML operations

* Auto-format code with fourmolu

* Refactor Agentic.hs for code succinctness and add unit tests

- Group magic constants into ToolLimits config record
- Simplify mode selection logic (~25 lines reduced)
- Simplify SQL validation and runSqlQuery (~25 lines reduced)
- Simplify format functions using ViewPatterns
- Use bimap/either for cleaner error handling
- Sanitize error messages (don't expose internal errors)
- Add unit tests for suggestAgenticMode, tool parsing, SQL validation
- Fix pre-existing build errors in RequestDumps.hs (try import, minimum)

Total: ~80 lines reduced (663 -> 581 LOC, 12% reduction)

* Auto-format code with fourmolu

* Fix case-sensitivity bug in executeSecuredQuery

The function was using lowerQuery for detection but applying
positions to userQuery. While this worked for ASCII (toLower
preserves length), it was fragile. Refactored to use a
breakOnCI helper that handles case-insensitivity consistently
and returns splits of the original text.

* Fix hlint warnings in Agentic.hs

* Refactor Agentic.hs to use langchain-hs native tool calling

- Use langchain-hs LLM.chat API with OpenAI tool definitions instead of
  custom text-based tool protocol
- Define tools using OAITool.Tool format passed to the chat API
- Implement agentic loop that handles tool calls via LLM.toolCalls
- Maintain conversation history using langchain-hs ChatHistory
- Add GetSchema tool for field discovery
- Update cabal.project to latest langchain-hs commit
- Simplify tests to match updated exports

* Auto-format code with fourmolu

* Fix hlint warnings in Agentic.hs

* Improve code quality in Agentic.hs

- Remove fake SQL validation tests that tested non-existent functions
- Add maxDisplayRows and maxBodyPreview to ToolLimits (removes magic numbers)
- Add getTextArg helper to reduce duplicate pattern matching
- Use bool function for jsonToText Bool case
- Pass limits to formatSampleLogs and formatQueryResults

* Improve code quality and reduce boilerplate in Agentic.hs

- Add module-level Haddock documentation explaining the agentic system
- Remove unused AgenticResponse type from exports
- Add getLimitArg helper to consolidate repeated limit calculation pattern
- Add mkSimpleTool and mkProp helpers to reduce tool definition boilerplate
- Add toolError helper for consistent error messages with context
- Rename getIntFromArgs to getIntArg for consistency with getTextArg

* Auto-format code with fourmolu

* Fix UTF-8 bug in executeSecuredQuery's breakOnCIKeyword

The previous implementation calculated text length on the lowercase
version but applied the split to the original text. This would fail
for Unicode characters that change length when lowercased (e.g.,
German ẞ → ss, Turkish İ → i).

Fixed by using character-by-character search with proper position
tracking on the original string, avoiding the length mismatch issue.

* Unify AI system with always-on tool calling

- Merge Pkg.AI.Agentic into Pkg.AI for single AI module
- Remove FastMode/AgenticMode distinction - always use tool calling
- Remove suggestAgenticMode brittle keyword matching - LLM decides when to use tools
- Add facet context fetching to all bots (Discord, Slack, WhatsApp)
- All channels now have unified behavior with full project awareness
- Simplify AgenticConfig by removing mode field
- Increase maxIterations to 5 for better tool exploration

* Auto-format code with fourmolu

* Refactor bot modules to share AI query processing logic

Extract common functionality into Pages.Bots.Utils:
- AIQueryResult data type for unified response handling
- processAIQuery for shared facet fetching and AI execution
- formatThreadsPrompt for common thread context formatting

Reduces duplication across Discord, Slack, and WhatsApp modules.

* Auto-format code with fourmolu

* Use langchain-hs TokenBufferMemory for thread context

Replace manual formatThreadsPrompt with TokenBufferMemory to provide
token-aware thread context management. This prevents prompt overflow
by trimming messages to 3000 tokens while preserving system messages.

* Auto-format code with fourmolu

* Improve AI tool calling with SQL support and code cleanup

- Add run_sql_query tool for complex queries not expressible in KQL
- Make helper functions more succinct using LambdaCase
- Use V.mapMaybe instead of mapMaybe . V.toList for better performance
- Preserve diagnostic info in error handling (show parse/execution errors)

* Use TokenBufferMemory consistently for chat history management

- Add addMessagesToMemory helper in AI.hs using TokenBufferMemory
- Replace manual message concatenation in runAgenticLoop with memory-based approach
- Simplify roleToText in Utils.hs to use show

* Simplify code patterns and reduce redundancy

- Simplify executeSecuredQuery using subquery wrapper (34 -> 9 lines)
- Use Semigroup for NonEmpty concatenation in addMessagesToMemory
- Replace verbose case pattern with <&> in processAIQuery
- Use tuple destructuring for Maybe handling

* Fix hlint warning: use nonEmpty from Relude

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* Remove redundant Data.List.NonEmpty import

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* Remove unused TimeSpec import

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-02 23:53:21 +01:00
Anthony Alaribe
3f385043ef
Support more KQL operators and functions (#259)
* Add KQL functions for _overview.yaml SQL migration

Implement P0 and P1 KQL features needed to migrate _overview.yaml dashboard
queries from SQL to KQL:

P0 Features:
- countif(condition): COUNT(*) FILTER (WHERE condition) for error rate calculations
- dcount(field): COUNT(DISTINCT field) for service/resource counts

P1 Features:
- coalesce(field, default): COALESCE for null handling
- strcat(field1, separator, field2): CONCAT for string concatenation
- iff(condition, then, else): CASE WHEN for simple conditionals
- case(cond1, val1, ..., default): Multi-branch CASE for categorization

SQL generation for each function:
- countif(status == "ERROR") → COUNT(*) FILTER (WHERE status = 'ERROR')
- dcount(service_name) → COUNT(DISTINCT service_name)
- coalesce(method, "unknown") → COALESCE(method, 'unknown')
- strcat(method, " ", path) → CONCAT(method, ' ', path)
- iff(x > 0, a, b) → CASE WHEN x > 0 THEN a ELSE b END
- case(x >= 500, e, x >= 400, w, s) → CASE WHEN x >= 500 THEN e WHEN x >= 400 THEN w ELSE s END

Example target KQL for _overview.yaml:
  kind == "server" AND duration != null
  | summarize
      error_rate = countif(status_code == "ERROR") * 100.0 / count(),
      p95_latency = percentile(duration / 1e6, 95)
    by resource.service.name

* Auto-format code with fourmolu

* Refactor KQL parsers to use applicative style

- Add pSimpleAgg helper for common function(subject) pattern
- Convert pCountIf, pCoalesce, pStrcat, pIff to applicative style
- Simplify pCase with inline tuple construction
- Remove redundant pDCount (now handled by pSimpleAgg)
- Net reduction of ~50 lines while maintaining same functionality

* Auto-format code with fourmolu

* Align KQL functions with Microsoft KQL dialect

Update function signatures to match official Microsoft KQL syntax:

- dcount(expr [, accuracy]): Add optional accuracy parameter (0-4)
  for HyperLogLog estimation precision

- coalesce(arg1, arg2, [arg3,...]): Now variadic (2-64 args)
  Returns first non-null expression

- strcat(arg1, arg2, [arg3,...]): Now variadic (1-64 args)
  Concatenates any scalar expressions

- iff(condition, then, else): Now accepts any Values for then/else
  Not limited to just field references

- case(pred1, val1, [pred2, val2, ...] else): Values are now any scalar
  Matches Microsoft KQL multi-branch conditional syntax

Added pScalarExpr helper to parse either literal values or field
references, enabling flexible value parsing in scalar functions.

Sources:
- https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/dcount-aggfunction
- https://learn.microsoft.com/en-us/kusto/query/coalesce-function
- https://learn.microsoft.com/en-us/kusto/query/strcat-function
- https://learn.microsoft.com/en-us/kusto/query/iff-function
- https://learn.microsoft.com/en-us/kusto/query/case-function

* Auto-format code with fourmolu

* Refactor KQL parsers for DRY and fix backtracking

Parser improvements:
- Extract `comma` helper for repeated `string "," *> space` pattern
- Add `pVariadicAgg` helper for variadic functions (coalesce, strcat)
- Convert pDCount to applicative style (7 lines -> 5 lines)
- Simplify pCoalesce and pStrcat to one-liners using pVariadicAgg
- Use `comma` consistently across all parsers

Bug fix:
- Add `try` to Count parser to prevent partial consumption breaking
  backtracking to countif/dcount parsers
- Reorder dcount before count in parser choice list

Code reduction: ~30 lines eliminated while improving consistency

* Auto-format code with fourmolu

* Refactor KQL parser for succinctness and convert percentile queries to KQL

Parser improvements:
- Add withoutAlias helper to eliminate repetitive <*> pure Nothing
- Add commaSep/commaSep1 for cleaner comma-separated parsing
- Refactor pPercentilesMulti/pPercentileSingle to applicative style
- Fix pScalarExpr to use toQText instead of display (preserves KQL notation)
- Extract pCount parser with cleaner parentheses handling
- Use foldMap for CASE WHEN display, simplify Plain display

Dashboard improvements:
- Convert Request Latency Percentiles widget to KQL (was 25 lines SQL)
- Convert Service Latency Percentiles widget to KQL (was 25 lines SQL)
- Both now use: summarize percentiles(duration, 50, 75, 90, 95, 99) by bin_auto(timestamp)

* Auto-format code with fourmolu

* Improve tests with full SQL assertions and fix KQL dot notation

Tests:
- Update countif, dcount, coalesce, strcat, iff, case tests to assert
  full generated SQL queries instead of partial string checks
- Add full SQL assertion for percentiles LATERAL unnest query
- Update where clause combination test with full SQL assertion

Dashboard KQL:
- Fix KQL queries to use "." separator (resource.service.name)
  instead of "___" which is only for raw SQL
- Add order by timestamp desc for proper ordering

* Fix ambiguous 'many' import and apply HLint suggestion

- Hide 'many' from Relude to resolve ambiguity with Text.Megaparsec
- Replace case with maybe in extractPercentilesInfo per HLint

* Refactor KQL parser and add dcount accuracy validation

- Remove redundant commaSep/commaSep1 wrappers, use sepBy directly
- Use ViewPatterns for cleaner extractPercentilesInfo pattern matching
- Add validation for dcount accuracy bounds (0-4 per KQL spec)
- Add comment explaining dcount accuracy is ignored in PostgreSQL
- Add edge case tests for variadic functions, nested calls, field paths

* Fix UnexpectedNull error for hashes field in OtelLogsAndSpans

Use COALESCE(hashes, '{}') in SQL queries to handle NULL values
in the hashes column, preventing PGArray parsing failures.

* Fix UnexpectedNull error for start_time field in OtelLogsAndSpans

Use COALESCE(start_time, timestamp) in SQL queries to handle NULL values
in start_time column. Logs may not have start_time set, falling back to
the main timestamp field.

* Fix field references in coalesce/strcat SQL generation

- Add Field constructor to Values type for unquoted field references
- Update pScalarExpr to use Field for subject references
- Remove catch-all from pValues that was consuming commas
- Add coalesce validation requiring 2+ arguments
- Clean up Stats.hs exports
- Update test expectations for correct SQL output

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-02 15:00:39 +01:00
Anthony Alaribe
4af007eb65
Add KQL support for latency percentile queries (#255)
* Add KQL percentiles() function for latency percentile queries

- Add Percentiles aggregation function to Stats.hs that generates
  p50, p75, p90, p95, p99 percentiles with optional divisor for
  unit conversion (e.g., percentiles(duration, 1e6) for ns to ms)
- Update Parser.hs to generate LATERAL unnest SQL pattern when
  percentiles() is used in summarize queries
- Update Log.hs to use KQL query instead of raw SQL for latency
  percentiles widget
- Update dashboard YAML files to use KQL percentile queries:
  - opentelemetry-tracing.yaml
  - endpoint-stats.yaml
  - mysql.yaml
  - postgresql.yaml
  - apache.yaml
  - cloudflare.yaml

This reduces SQL boilerplate significantly and makes percentile
queries much more readable and maintainable.

* Auto-format code with fourmolu

* Update KQL percentile/percentiles to match Microsoft KQL syntax

Update the implementation to match Microsoft KQL semantics:
- percentile(expr, percentile) - single percentile value
- percentiles(expr, p1, p2, p3, ...) - multiple custom percentile values

Key changes:
- Parser now accepts explicit percentile values (e.g., 50, 75, 90, 95)
- If last value > 100, it's treated as a divisor for unit conversion
- SQL generation dynamically builds ARRAY based on specified percentiles
- Dashboard queries now use explicit percentile values

Examples:
  percentile(duration, 95) - single 95th percentile
  percentiles(duration, 50, 75, 90, 95) - multiple percentiles
  percentiles(duration, 50, 75, 90, 95, 1e6) - with divisor for ns->ms

* Match Microsoft KQL syntax for percentile/percentiles functions

Update the implementation to exactly match Microsoft KQL semantics:
- percentile(expr, percentile) - single percentile value
- percentiles(expr, p1, p2, ...) - multiple percentile values

Now supports expressions in the first argument for unit conversion:
  percentiles(duration / 1e6, 50, 75, 90, 95)

Key changes:
- Add SubjectExpr type to support field / number and field * number
- Parser accepts expression as first arg, then percentile values
- No more magic divisor detection - use explicit expressions
- Dashboard queries updated to use proper expression syntax

Examples matching Microsoft KQL:
  percentile(duration, 95)
  percentiles(duration, 50, 75, 90, 95)
  percentiles(duration / 1e6, 50, 75, 90, 95, 99)

* Auto-format code with fourmolu

* Fix chart widget query updates and combine where clauses

- Fix event dispatch in query-editor.ts: dispatch update-query to window
  so chart widgets receive query change events
- Fix Parser.hs: combine multiple where clauses with AND instead of
  replacing them, so filters like "service == X | duration != null"
  correctly apply both conditions
- Add combineWhereClause helper function
- Add unit test for combined where clause behavior
- Refactor Stats.hs: extract setAlias helper for namedAggregation

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-01 18:24:49 +01:00
Anthony Alaribe
4c48772156
Fix parser unit tests to expect _total_count column (#256)
Update 7 test cases to include `count(*) OVER() as _total_count`
in expected SQL output, matching the change from commit 7b9515c
that combined data and count queries into one.

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-01 15:11:11 +01:00
Anthony Alaribe
924e8e4002
Query cache (#252)
* add intelligent query cache for KQL timeseries queries

- Cache summarize queries with bin/bin_auto time binning
- Store results in UNLOGGED PostgreSQL table for performance
- Delta fetch on partial cache hits (only query new time range)
- Merge cached + new data, deduplicate by timestamp
- 24h sliding window eviction via scheduled cleanup job
- Per-project cache namespace with query hash + bin interval key

* Auto-format code with fourmolu

* fix query cache issues and improve code quality

- Fix race condition: use LEAST/GREATEST for cache time range merging
- Fix partial hit logic: handle requests extending both directions
- Add LIMIT 1 to cache lookup query
- Use existing toXXHash from Utils instead of manual xxHash call
- Single-pass stats calculation with foldl'
- Use liftA2 for Maybe operations, comparing for point-free style
- Use viaNonEmpty from Relude instead of Safe package

* add hourly cache eviction to prevent unbounded growth

- Add cleanupExpiredCache function to remove stale entries
- Evict entries where cached_to > 24h ago or last_accessed_at > 1h ago
- Integrate with HourlyJob in BackgroundJobs
- Log cleanup count when entries are deleted

* Auto-format code with fourmolu

* refactor query cache for conciseness and correctness

- Use <&> and LambdaCase for cleaner lookupCache
- Extract toPosix helper for time conversions
- Use groupBy for declarative deduplication
- Remove redundant index (UNIQUE constraint creates it)
- Increase LRU threshold to 4h (from 1h)
- Remove redundant Parser qualified import in Charts.hs

* Auto-format code with fourmolu

* fix query cache stats calculation for multi-column data

- recalculateStats now processes all value columns, not just the first
- maxGroupSum correctly sums all values per row (timestamp)
- filterByTimestamp recalculates stats after filtering
- add integration tests for cache hit, partial hit, and multi-column stats

* Auto-format code with fourmolu

* fix cache metadata mismatch and backward-extending request handling

- Use slidingWindowStart instead of entry.cachedFrom when updating cache
  after trimming, ensuring metadata matches actual data range
- Remove PartialHit case for requests extending both directions, as only
  forward delta was fetched leaving backward data missing

* optimize query cache table for performance

- Use composite primary key instead of UUID PK + unique constraint
- Remove FK constraint (orphans cleaned by LRU eviction)
- Use JSON instead of JSONB (faster writes, always read whole blob)
- Add fillfactor=70 for HOT updates on hit_count/last_accessed_at
- Add aggressive autovacuum settings for high-update table
- Simplify eviction to just LRU (remove redundant cached_to check)
- Remove unnecessary indexes (table is small, hourly eviction)

* fix query cache correctness and performance issues

- Replace LEAST/GREATEST with direct replacement in updateCache to fix
  metadata mismatch after trimOldData removes old entries
- Make sliding window configurable based on bin interval (48 points or 24h min)
- Cap mode frequency map to 1000 entries for high-cardinality data
- Use vector ifilter for deduplication instead of list groupBy conversion

* Auto-format code with fourmolu

* use Relude words and one instead of qualified imports

* refactor: simplify QueryCache code

- Simplify parseBinIntervalToSeconds using unitToSeconds helper
- Extract rowTimestamp helper to dedupe join . V.headM usage
- Replace case () with multi-way if in lookupCache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-01 14:51:17 +01:00
Anthony Alaribe
4b25c95630 fix: use relative paths in GitSync test assertions
buildSyncPlan uses relative paths (without prefix) as dbState keys,
matching how the actual code stores filePath in the database.
2025-12-26 18:55:52 +01:00
Anthony Alaribe
d1e4600ed8 feat: add folder field to new dashboard modal 2025-12-26 16:31:11 +01:00
Anthony Alaribe
112063c5e9 feat: add folder field to dashboard rename modal for git sync
Users can now set a folder path for dashboards that will be used when
syncing to git. The filename is auto-generated from the dashboard title.
2025-12-26 16:27:20 +01:00
Anthony Alaribe
54eb6c581c feat: add git file path field to dashboard rename modal
Users can now view and edit the git file path for dashboards in the
rename modal. The path is relative to the dashboards/ folder.

Example: entering "reports/monthly.yaml" will sync to
"dashboards/reports/monthly.yaml" in the repo.

Also fixed path handling to consistently store relative paths in DB
and construct full paths at push time.
2025-12-26 16:23:59 +01:00
Anthony Alaribe
da704c17a2 fix: queue git sync push when dashboards are created/updated
Dashboard changes were not being pushed to GitHub because
queueGitSyncPush was never called after dashboard operations.

Changes:
- Created syncDashboardAndQueuePush to combine file info sync and job queuing
- Simplified GitSyncPushDashboard job to compute file path internally
- Updated all dashboard handlers to use the new consolidated function
- Simplified queueGitSyncPush signature (no file path param needed)
2025-12-26 15:53:02 +01:00
Anthony Alaribe
bccc1bbacd
Github 2-way sync with a repo (#244)
* add git sync for dashboards

* fix: improve git sync reliability and code quality

- Fix race condition: store root tree SHA instead of file entry SHA

- Fix empty token handling: preserve existing token when form empty

- Improve decryption: return Nothing on failure

- Replace inefficient JSON parsing with parseMaybe

- Replace manual toHex with base16-bytestring

- Add uuid-quasi for compile-time UUID validation in tests

* refactor: consolidate git sync helpers and reduce code duplication

* Auto-format code with fourmolu

* refactor: improve git sync code quality

- Webhook validation now returns Either for better error messages
- Add SQL column constant to reduce duplication
- Simplify Map construction with inline lambda
- Improve pattern matching exhaustiveness in gitSyncPushDashboard
- PackageImports kept for explicit crypto package disambiguation

* Auto-format code with fourmolu

* refactor: improve code quality and fix SQL injection risk

- Use pg-entity _selectWhere/_delete instead of manual column lists
- Fix SQL injection in selectDashboardsSortedBy with whitelist approach
- Add Entity instance for GitHubSync for type-safe queries
- Replace two-pass partition with single-pass foldl' in git sync
- Parallelize file fetching during git sync using Ki.scoped
- Qualify SqlQQ import to avoid collision with Variable.sql field
- Rename key params to encKey to avoid shadowing Data.Aeson.Lens.key

* Auto-format code with fourmolu

* feat: add comprehensive GitHub sync setup instructions

- Show full webhook URL with copy button
- Add step-by-step guide for creating GitHub personal access token
- Document expected repository structure (dashboards/*.yaml)
- Show example dashboard YAML format
- Include webhook setup instructions for auto-sync

* refactor: use markdown for git sync setup instructions

- Replace verbose HTML with markdown quasi-quotes
- Use MMark to render markdown to HTML
- Much cleaner and easier to maintain documentation
- Removes mmark from unused packages warning

* refactor: improve code quality and add decryption logging

- Add logging when GitHub sync token decryption fails to aid debugging
- Replace foldl' partition with idiomatic list comprehensions
- DRY duplicated SQL queries in selectDashboardsSortedBy
- Use pg-entity _selectWhere for type-safe query building
- Use RecordDot syntax instead of fromOnly in getDashboardByBaseTemplate

* Auto-format code with fourmolu

* refactor: improve code quality and security

- Add warning log when webhook accepted without secret validation
- Extract withDecryption helper to DRY decrypted getter functions
- Use foldMap for efficient buildSyncPlan implementation
- Extract delete fields in list comprehension to avoid pattern warnings
- Remove unused imports (foldl', Data.Map.Strict)

* style: fix hlint suggestions

- Use point-free style for Ki.fork composition
- Remove redundant toStrict call
- Parenthesize unary negation for clarity
- Remove redundant brackets around V.fromList

* style: remove unnecessary package qualification for base16-bytestring

* Auto-format code with fourmolu

* fix: resolve CI warnings and errors

- Add fromJust import to GitSyncSpec.hs
- Remove unused Job parameter from processBackgroundJob
- Fix ambiguous record fields in DashboardsSpec.hs using lens syntax
- Remove undefined usage in BusinessFlowsSpec.hs

* refactor: simplify withDecryption using maybe and either

* Auto-format code with fourmolu

* refactor: simplify git sync error handling and use pg-entity

* fix: resolve CI compile errors in tests

* fix: resolve DashboardVM field access ambiguity in tests

* fix: use record dot syntax for DashboardVM, qualified for DashboardForm

* security: use constant-time comparison for HMAC validation

* refactor: use point-free style for encryptToken

* refactor: flatten nested case with tuple pattern matching

* refactor: use lens for decryptSync, fix constEq type ambiguity

* refactor: use bimap for base64 errors, Ki.awaitAll for parallel exec

* Auto-format code with fourmolu

* refactor: simplify sort field handling with lookup table

* perf: fix N+1 query by pre-fetching teams before parallel sync

* fix: use mkDashboard helper to avoid ambiguous record update

* perf: use T.filter instead of T.replace for newline removal

* refactor: use point-free style for titleToFilePath

* feat: add GitHub sync E2E tests and fix empty repo handling

- Add comprehensive E2E integration tests for GitHub sync

- Handle 404 from GitHub tree API as empty tree (for repos with all files deleted)

- Add updateTags function to update dashboard tags during sync

- Fix SyncUpdate to also update tags when pulling from GitHub

- Add Default instance for GitHubSync

- Add detectDefaultBranch function for auto-detecting repo branch

* Auto-format code with fourmolu

* Github.hs

* Auto-format code with fourmolu

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-26 01:19:12 +01:00
Yussif Mohammed
cc51fda8a6
Alerts view & Error details page (#225)
* towards supporting teams in projects.

* Auto-format code with fourmolu

* improve team list

team view side page, alerts and dashbaord next

yeah well

validation checks and fixes

tiny details improvments

prepare endpoints for testing

Auto-format code with fourmolu

towards tests

Auto-format code with fourmolu

rm monoscope-test

more tests cases

Auto-format code with fourmolu

alerts page improvements bug fixes and teams link

Auto-format code with fourmolu

feat: alerts section, teams.

send notifications

Auto-format code with fourmolu

fix errs and slack install

Auto-format code with fourmolu

towards table component

Auto-format code with fourmolu

dashbaord bulk actions delete

dashboards with teams

team dashboard

Auto-format code with fourmolu

feat: teams bulk actions delete

Auto-format code with fourmolu

feat: add title field & teams to dashboard modal

Auto-format code with fourmolu

* remove stack files and configure hie.yaml for better IDE support

Auto-format code with fourmolu

fix and passing tests

Auto-format code with fourmolu

add dashboards spec

Auto-format code with fourmolu

notifications templates

query viz and alert notifications

Auto-format code with fourmolu

alert config stuff

fixes and towards issue detail page

Auto-format code with fourmolu

Reorganize project structure

Move config files to config/ directory (tailwind.config.js, workbox-config.js, weeder.toml). Move documentation to docs/ (DEVELOPMENT.md, DOCKER_COMPOSE_README.md). Update all references in Dockerfile, Makefile, tailwind.css, and documentation.

Fix CI disk space exhaustion by cleaning up unnecessary files and build artifacts

Optimize CI caching strategy to reduce disk usage

Use separate caches per test suite and cache only essential build
artifacts instead of entire dist-newstyle directory. This reduces
disk usage while maintaining fast compilation through effective caching.

Fix CSS build in deployment by using correct Tailwind v4 CLI

The project uses Tailwind CSS v4 syntax (@import, @source, @plugin
directives) but the build commands were using the old v3 CLI which
doesn't process v4 syntax correctly, causing CSS changes to not appear
in deployed builds.

Updated Dockerfile and CI workflows to use @tailwindcss/cli instead of
the legacy tailwindcss command, and removed the obsolete -c config flag
since v4 uses @config directive in the CSS file.

Use npx tailwindcss instead of @tailwindcss/cli for v4 compatibility

The tailwindcss package (v4.1.13) in devDependencies is the correct CLI
for Tailwind v4. Using @tailwindcss/cli was unnecessarily specific and
may have caused issues in the Docker build environment.

This ensures the CSS with all custom utilities (surface-raised, etc.)
builds correctly in production.

Cherry pick files (#232)

* cherry pick files

* Auto-format code with fourmolu

* add more files

* Auto-format code with fourmolu

* finx undefined var

* review changes

* Auto-format code with fourmolu

* discord channel' type rename

* Auto-format code with fourmolu

* claude rec fixes

* Auto-format code with fourmolu

* fix notification typo

* Auto-format code with fourmolu

* discord get channel log error

* fix typo

* rm reswap related code from System.Types.hs

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>

Teams rebased (#233)

* towards supporting teams in projects.

* Auto-format code with fourmolu

* fix htmx json encoder extension

* improve team list

* team view side page, alerts and dashbaord next

* yeah well

* validation checks and fixes

* tiny details improvments

* prepare endpoints for testing

* Auto-format code with fourmolu

* towards tests

* Auto-format code with fourmolu

* rm monoscope-test

* fix alert creation

* more tests cases

* Auto-format code with fourmolu

* alerts page improvements bug fixes and teams link

* Auto-format code with fourmolu

* feat: alerts section, teams.

* send notifications

* Auto-format code with fourmolu

* fix errs and slack install

* Auto-format code with fourmolu

* add teams column to dashboards table

* towards table component

* Auto-format code with fourmolu

* dashbaord bulk actions delete

* dashboards with teams

* team dashboard

* Auto-format code with fourmolu

* fix table

* feat: teams bulk actions delete

* Auto-format code with fourmolu

* feat: add title field & teams to dashboard modal

* Auto-format code with fourmolu

* remove stack files and configure hie.yaml for better IDE support

* Auto-format code with fourmolu

* fix and passing tests

* Auto-format code with fourmolu

* add more tests

* add dashboards spec

* Auto-format code with fourmolu

* fix pull request workflow

* fix: review change requests

* Auto-format code with fourmolu

* feat: tagify consolidation

* Auto-format code with fourmolu

* updates

* simplifications and other changes

* background changes and fixes

* Auto-format code with fourmolu

* package yaml

* Auto-format code with fourmolu

* fix typo

* towards supporting teams in projects.

* Auto-format code with fourmolu

* improve team list

* team view side page, alerts and dashbaord next

* yeah well

* validation checks and fixes

* tiny details improvments

* prepare endpoints for testing

* Auto-format code with fourmolu

* towards tests

* Auto-format code with fourmolu

* rm monoscope-test

* more tests cases

* Auto-format code with fourmolu

* alerts page improvements bug fixes and teams link

* Auto-format code with fourmolu

* feat: alerts section, teams.

* send notifications

* Auto-format code with fourmolu

* fix errs and slack install

* Auto-format code with fourmolu

* add teams column to dashboards table

* towards table component

* Auto-format code with fourmolu

* dashbaord bulk actions delete

* dashboards with teams

* team dashboard

* Auto-format code with fourmolu

* feat: teams bulk actions delete

* Auto-format code with fourmolu

* feat: add title field & teams to dashboard modal

* Auto-format code with fourmolu

* remove stack files and configure hie.yaml for better IDE support

* Auto-format code with fourmolu

* fix and passing tests

* Auto-format code with fourmolu

* add more tests

* add dashboards spec

* Auto-format code with fourmolu

* fix: review change requests

* Auto-format code with fourmolu

* feat: tagify consolidation

* Auto-format code with fourmolu

* updates

* simplifications and other changes

* background changes and fixes

* Auto-format code with fourmolu

* package yaml

* Auto-format code with fourmolu

* Migrate to master's Table API

  - Update dashboards and teams tables to use new column-based API
  - Replace ItemsList.ItemsRows with Table.TableRows
  - Add missing validateTeamDetails function
  - Fix import conflicts

* Migrate to master's Table API and fix migration idempotency

- Update dashboards and teams tables to use new column-based API
- Replace ItemsList.ItemsRows with Table.TableRows
- Add missing validateTeamDetails function
- Fix import conflicts (Wreq.header ambiguity)
- Add IF NOT EXISTS to index creation in teams migration

* Add frosted glass backdrop to modals

- Add backdrop-filter blur effect to .modal-backdrop class
- Applies 8px blur for frosted glass effect
- Includes webkit prefix for browser compatibility

* Remove padding from table container

- Remove px-6, pt-4, and pb-16 from containerClasses
- Table now renders edge-to-edge without padding

* Add padding to non-settings page content

- Remove padding from Table component's default containerClasses
- Add px-6 pt-4 pb-16 wrapper for non-settings pages in BodyWrapper
- Settings pages (Teams, etc.) remain without extra padding

* Remove duplicate padding from dashboards list page

- Remove pb-2 px-6 pt-8 from itemsListPage div
- Padding now comes from BodyWrapper parent wrapper
- Avoids double padding on dashboards list

* Align dashboard button and table checkbox with page header

* Fix double padding issue with standardized table padding system

- Remove global padding wrapper from BodyWrapper to fix double padding
- Add addPadding config option to Table component
- Update table pages to use addPadding: Dashboards, API Catalog, Issues, Monitors
- Standardize all table padding to px-6 pt-4 pb-2
- Update Monitors page section padding to match standard

* fix tables and compilation

* Auto-format code with fourmolu

* fix dulicate line

* Auto-format code with fourmolu

* fix build

* Refactor FromJSON instances to use deriving strategies

- Use deriving via with DAE.Rename for field name transformations

- Use deriving via with DAE.StripSuffix for simple underscore removal

- Extract SlackEventPayload constructors into separate record types

- Simplify TeamForm to use deriving anyclass for internal API

- Remove unused imports

Code reduction: ~80 lines of manual JSON parsing boilerplate eliminated

* Auto-format code with fourmolu

* Add team validation and permission checks

- Add admin-only restriction for team create/update
- Validate team members are project members
- Add email format validation for notification emails
- Add length limits to team name and handle
- Verify teams exist before adding to dashboards
- Extract dispatchTeamNotifications helper in BackgroundJobs
- Use TeamDetails record to simplify createTeam/updateTeam APIs
- Handle duplicate team handles with ON CONFLICT DO NOTHING

* Auto-format code with fourmolu

* Refactor teams feature with comprehensive improvements

Code Quality & Type Safety:

- Refactor createTeam/updateTeam to use TeamDetails record type

- Add compile-time safety with NonEmpty for softDeleteProjectMembers

- Standardize SQL array syntax (lowercase uuid[], consistent spacing)

- Add phone_numbers support to all team types and queries

Performance Optimizations:

- Optimize getTeamsVM/getTeamByHandle with LEFT JOIN + array_agg

- Use for_ instead of explicit null checks

Error Handling & Observability:

- Add logging for missing/deleted teams in monitor notifications

- Improve error handling for addTeamsToDashboards

- Add graceful fallback when configured teams are deleted

Code Cleanup:

- Extract dispatchTeamNotifications helper function

- Remove redundant project query

- Use idiomatic Haskell patterns

* Auto-format code with fourmolu

* Fix variable shadowing in email validation

* Refactor for code succinctness

- Simplify NonEmpty to Vector conversion (remove redundant toList)

- Extract handleQueryMonitorThreshold into focused helper functions

- Add logMissingTeams for team logging

- Add createAndInsertIssue for issue creation

- Add calculateThreshold for threshold calculation

- Add buildMonitorAlert for alert construction

- Add sendNotifications for notification dispatch

- Add sendFallbackNotifications for fallback handling

* Auto-format code with fourmolu

---------

Co-authored-by: Yussif Mohammed <yousiph77@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

towards supporting teams in projects.

Auto-format code with fourmolu

improve team list

validation checks and fixes

tiny details improvments

Auto-format code with fourmolu

towards tests

Auto-format code with fourmolu

more tests cases

Auto-format code with fourmolu

* alerts page improvements bug fixes and teams link

Auto-format code with fourmolu

feat: alerts section, teams.

send notifications

Auto-format code with fourmolu

fix errs and slack install

Auto-format code with fourmolu

towards table component

Auto-format code with fourmolu

dashbaord bulk actions delete

dashboards with teams

Auto-format code with fourmolu

feat: teams bulk actions delete

Auto-format code with fourmolu

feat: add title field & teams to dashboard modal

Auto-format code with fourmolu

remove stack files and configure hie.yaml for better IDE support

Auto-format code with fourmolu

fix and passing tests

Auto-format code with fourmolu

add dashboards spec

Auto-format code with fourmolu

alert config stuff

fixes and towards issue detail page

rmv duplicate declarations

Auto-format code with fourmolu

fix resizer

Auto-format code with fourmolu

make project compile/fix errors

* Auto-format code with fourmolu

feat: use log-list virtual table for monitor history tab

Auto-format code with fourmolu

fix issue detail tabs

Auto-format code with fourmolu

clean up unused imports

Auto-format code with fourmolu

parameterized trace/span sql queries

fix unkwon stack typo

more unused imports cleanup

Auto-format code with fourmolu

remove commented logs

Rebase onto master, resolve conflicts with master changes

fix tests after recent changes to alerts view

fix tests after changing ManageMembers handlers to return ProjectId

remove monoscope-test

towards fixing tests

Auto-format code with fourmolu

fix: sort starred dashboards first and add chart padding

- Dashboard list now always shows starred/favorited items at the top regardless of the selected sort option (newest, oldest, A-Z, etc.)

- Added p-2 padding to widget chart container to prevent bleeding edges

fix: exclude p-2 padding for timeseries_stat widgets

fix: dashboard variable tagify inputs not initializing correctly

- Update createTagify to accept both selector strings and DOM elements

- Wrap dashboard variable tagify init in DOMContentLoaded event

- Remove overflow-hidden from variable fieldset to allow dropdown display

fix: remove unnecessary border wrapper from table component

Auto-format code with fourmolu

towards fixing integration tests

Auto-format code with fourmolu

* fix more errors

* Auto-format code with fourmolu

* claude req fixes

* Auto-format code with fourmolu

* fix table

* Auto-format code with fourmolu

* bring back removed page due to rebase

* Auto-format code with fourmolu

* rm unused imports

* fix failing delim queries in BackgroundJobs.hs

* Auto-format code with fourmolu

* fix build after rebase

* Auto-format code with fourmolu

* test final fix

* rm log attention

* add index for first_trace_id and recent_trace_id in errors table

* fix index creation in 0019_add_trace_id.sql

* remove concurrently index

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-23 17:54:20 +01:00
Anthony Alaribe
64b3a8bc88
refactor: migrate from pg-transact to effectful-postgresql to support postgres open telemetry instrumentation (#236)
* refactor: migrate from pg-transact to effectful-postgresql

Replace pg-transact and pg-transact-effectful with effectful-postgresql for database operations. Key changes: replace DB effect with WithConnection, remove dbtToEff wrappers, use direct PG.query/PG.execute calls, update runDB to runWithConnectionPool, keep pg-entity for query building, add IOE constraint alongside WithConnection where needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* Auto-format code with fourmolu

* chore: use effectful-postgresql fork with OpenTelemetry support

Switch to tonyalaribe/effectful-postgresql fork (instrument-pg branch) and
enable the +enable-otel flag for OpenTelemetry instrumentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* fix: improve connection usage and error handling in BackgroundJobs

- Use PG.execute directly instead of multiple withConnection calls for
  CleanupDemoProject (more efficient connection usage)
- Handle missing project gracefully in ATRuntimeException processing
  instead of crashing with error
- Clean up unused imports (PGS, withConnection)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* fix: correct effectful-postgresql subdir path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* fix: add proper error handling and logging for critical operations

- Log advisory lock release failures instead of silently ignoring
- Log when error update doesn't match any span (potential data issue)
- Log when span hash update count mismatches expected (data integrity)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* Auto-format code with fourmolu

* refactor: simplify single-row query handling in BackgroundJobs

* refactor: return lists from model query functions instead of Vectors

PostgreSQL-simple returns lists naturally, so wrapping results with V.fromList was wasteful. Model query functions now return lists directly, with Vector conversion happening at call sites that actually need Vectors.

* Auto-format code with fourmolu

* perf: fix N+1 query pattern and improve advisory lock handling

- Replace individual UPDATE queries in forM_ loop with batched UPDATE
  using SQL unnest to reduce O(n) DB round-trips to O(1)
- Add 5-minute statement timeout to advisory lock to prevent hangs
- Add timeout reset in finally block for cleanup
- Improve comments explaining session-level lock auto-release behavior

* refactor: replace verbose Only unwrapping with coerce

- Use coerce @[Only T] @[T] instead of map (\(Only x) -> x)
- Use coerce @(Maybe (Only T)) @(Maybe T) instead of fmap (\(Only x) -> x)
- Update GenerateOtelFacetsBatch to use Projects.ProjectId instead of Text

* Auto-format code with fourmolu

* fix: advisory lock release and remove unnecessary vector conversion

- Use query instead of execute for pg_advisory_unlock (SELECT returns result)
- Add validation for unexpected unlock results
- Remove unnecessary V.fromList/V.toList round-trip in runHourlyJob

* refactor: simplify COUNT(*) query result handling

Use inline maybe/listToMaybe instead of case expression for cleaner code.

* Auto-format code with fourmolu

* refactor: use catch instead of try/case for advisory lock cleanup

Cleaner error handling with catch pattern instead of explicit try + case Left/Right.

* Auto-format code with fourmolu

* perf: use V.mapMaybe and V.unzip3 to avoid list conversions

Direct Vector operations for batch error updates instead of
list mapMaybe + V.fromList.

* refactor: introduce DB type alias and switch to pg-transact

* Auto-format code with fourmolu

* fix errors in specs

* fix: remove duplicate import in Dashboards.hs

* fix: resolve integration test failures

- BusinessFlowsSpec: fix column names for apis.subscriptions table

- TestUtils: add test user as admin to nil UUID project

- System/Types: increase static UUID pool to prevent collisions

- AnomaliesSpec: fix test assertions for correct filter views

* fix aroundAll

* fix: resolve integration test failures

- Fix insertProjectMembers to handle empty list (prevents 'no rows supplied' error)

- Fix DashboardsSpec UUID collisions by grouping creations and adding cleanup

- Fix ManageMembersSpec: correct userID to UUID 1, update expected member count and email

- Fix ProjectsSpec: update expected project count to include demo project

* remove PG_EFFECT_MIGRATION.md

* fix: use unique team handles in bulk add teams test to avoid conflicts

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-22 22:32:43 +01:00
Anthony Alaribe
5cd948697f
Teams improvement v2 (#234)
* Improve dashboards and teams UI with star functionality

- Add star/unstar functionality for dashboards with persistent state
- Update dashboard sorting to prioritize starred items
- Improve table component styling and layout
- Add icons for dashboards and teams in list views
- Streamline table columns and remove unnecessary checkboxes
- Update bulk actions to use new table component features

* Auto-format code with fourmolu

* Fix teams bulk actions to use itemId field

Update TBulkActionForm to use itemId instead of teamId to match
the Table component's standardized field name for bulk actions.

* Improve settings navigation layout and styling

- Reduce navigation width for better space utilization
- Update font sizes and spacing for cleaner appearance
- Improve icon styling with background colors
- Enhance navigation item padding and layout

* Refactor API keys page to use Table component and add client-side tab support

- Enhanced TabFilter component to support both client-side and server-side tabs
- Added clientSide boolean field to TabFilter for tab switching mode
- Added targetId field to TabFilterOpt for custom element targeting
- Refactored API keys page to use reusable Table component
- Updated all existing TabFilter usage to include new fields
- Improved consistency across table implementations

* Auto-format code with fourmolu

* nitpicks on

* add test for starring and unstarring, and dashboard default sorting

* Fix tagify scope issue in dashboard creation modal

The tagify variable was scoped inside DOMContentLoaded event listener,
causing a reference error when getSelectedTeams() tried to access it
during form submission. Moved declaration to outer scope.

* Fix gridStackInstance null error when adding dashboard widgets

* Fix teams badge WCAG contrast by using badge-neutral

* Fix compilation errors

- Fix do-notation errors by adding proper do blocks instead of using >> after $
- Rename local section_ helper to teamSection_ to avoid shadowing
- Remove unused ProjectId parameter from renderMonitorContent

* Use brackets instead of do-blocks for single-line expressions

- Keep code on one line by using parentheses and >> operator
- Applied to card_, notifRow_, lazySection_, teamSection_, field_, fieldIcon_ helpers

* Apply code formatting updates

- Replace font-mono with monospace class
- Minor whitespace adjustments

* Fix duplicate html assignment bug and simplify code

* Auto-format code with fourmolu

* Refactor tables to use standardized Table component with sorting

- Add containerId to Table Config for HTMX targeting
- Add renderAsTable mode for proper table rendering
- Add rowId and rowAttrs to TableRows for load-more consistency
- Fix th elements to include column attrs for alignment
- Fix class concatenation spacing in table headers
- Fix scrolling in BodyWrapper (overflow-y-hidden to overflow-y-auto)
- Add sort parameter to Issues.selectIssues
- Refactor Anomalies and ApiCatalog to use TableHeaderActions
- Remove flex classes from table cells

* Auto-format code with fourmolu

* Fix table load more by moving pagination inside tbody

- Move pagination link inside tbody for table mode (as a <tr>)
- Add renderTablePaginationLink with hx-target='closest tr'
- Add id to tbody for targeting
- Pagination row gets replaced with new rows + new pagination row

* Refactor Anomalies table to use built-in checkbox

- Remove custom checkbox column (use table's rowId feature instead)
- Add status indicator dot to Issue column title row
- Update TableRows to use proper rowId for load more

* Auto-format code with fourmolu

* Fix Anomalies issue column layout structure

- Fix indentation so metadata, stats, actions are separate blocks
- Status indicator dot in title row
- Condense code using single-line patterns where appropriate

* Auto-format code with fourmolu

* Merge renderPaginationLink and renderTablePaginationLink into single function

* Auto-format code with fourmolu

* Align row checkboxes to top and add vertical padding to issue column

* Improve settings sidebar: larger text, neutral icons, stronger selection highlight

* Widen settings sidebar

* Fix hlint warnings

* Fix all hlint warnings across codebase

* Auto-format code with fourmolu

* Fix GHC warnings: dodgy import, name shadowing, unused do-bind

* Use foldMap and zipWithM_ for cleaner iteration patterns

* Extract columnCount helper and add error handling for processVariable

* Auto-format code with fourmolu

* Optimize URL construction and use single-pass fold for change counting

* Extract issueTypeBadge helper to reduce code duplication

* Fix SQL injection vulnerabilities and semantic error in change counting

* Auto-format code with fourmolu

* Simplify SQL validation and remove redundant variable

* Auto-format code with fourmolu

* Rename selectDashboardsSorted to selectDashboardsSortedBy for clarity

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-19 23:44:55 +01:00
Anthony Alaribe
9521695299
Teams rebased (#233)
* towards supporting teams in projects.

* Auto-format code with fourmolu

* fix htmx json encoder extension

* improve team list

* team view side page, alerts and dashbaord next

* yeah well

* validation checks and fixes

* tiny details improvments

* prepare endpoints for testing

* Auto-format code with fourmolu

* towards tests

* Auto-format code with fourmolu

* rm monoscope-test

* fix alert creation

* more tests cases

* Auto-format code with fourmolu

* alerts page improvements bug fixes and teams link

* Auto-format code with fourmolu

* feat: alerts section, teams.

* send notifications

* Auto-format code with fourmolu

* fix errs and slack install

* Auto-format code with fourmolu

* add teams column to dashboards table

* towards table component

* Auto-format code with fourmolu

* dashbaord bulk actions delete

* dashboards with teams

* team dashboard

* Auto-format code with fourmolu

* fix table

* feat: teams bulk actions delete

* Auto-format code with fourmolu

* feat: add title field & teams to dashboard modal

* Auto-format code with fourmolu

* remove stack files and configure hie.yaml for better IDE support

* Auto-format code with fourmolu

* fix and passing tests

* Auto-format code with fourmolu

* add more tests

* add dashboards spec

* Auto-format code with fourmolu

* fix pull request workflow

* fix: review change requests

* Auto-format code with fourmolu

* feat: tagify consolidation

* Auto-format code with fourmolu

* updates

* simplifications and other changes

* background changes and fixes

* Auto-format code with fourmolu

* package yaml

* Auto-format code with fourmolu

* fix typo

* towards supporting teams in projects.

* Auto-format code with fourmolu

* improve team list

* team view side page, alerts and dashbaord next

* yeah well

* validation checks and fixes

* tiny details improvments

* prepare endpoints for testing

* Auto-format code with fourmolu

* towards tests

* Auto-format code with fourmolu

* rm monoscope-test

* more tests cases

* Auto-format code with fourmolu

* alerts page improvements bug fixes and teams link

* Auto-format code with fourmolu

* feat: alerts section, teams.

* send notifications

* Auto-format code with fourmolu

* fix errs and slack install

* Auto-format code with fourmolu

* add teams column to dashboards table

* towards table component

* Auto-format code with fourmolu

* dashbaord bulk actions delete

* dashboards with teams

* team dashboard

* Auto-format code with fourmolu

* feat: teams bulk actions delete

* Auto-format code with fourmolu

* feat: add title field & teams to dashboard modal

* Auto-format code with fourmolu

* remove stack files and configure hie.yaml for better IDE support

* Auto-format code with fourmolu

* fix and passing tests

* Auto-format code with fourmolu

* add more tests

* add dashboards spec

* Auto-format code with fourmolu

* fix: review change requests

* Auto-format code with fourmolu

* feat: tagify consolidation

* Auto-format code with fourmolu

* updates

* simplifications and other changes

* background changes and fixes

* Auto-format code with fourmolu

* package yaml

* Auto-format code with fourmolu

* Migrate to master's Table API

  - Update dashboards and teams tables to use new column-based API
  - Replace ItemsList.ItemsRows with Table.TableRows
  - Add missing validateTeamDetails function
  - Fix import conflicts

* Migrate to master's Table API and fix migration idempotency

- Update dashboards and teams tables to use new column-based API
- Replace ItemsList.ItemsRows with Table.TableRows
- Add missing validateTeamDetails function
- Fix import conflicts (Wreq.header ambiguity)
- Add IF NOT EXISTS to index creation in teams migration

* Add frosted glass backdrop to modals

- Add backdrop-filter blur effect to .modal-backdrop class
- Applies 8px blur for frosted glass effect
- Includes webkit prefix for browser compatibility

* Remove padding from table container

- Remove px-6, pt-4, and pb-16 from containerClasses
- Table now renders edge-to-edge without padding

* Add padding to non-settings page content

- Remove padding from Table component's default containerClasses
- Add px-6 pt-4 pb-16 wrapper for non-settings pages in BodyWrapper
- Settings pages (Teams, etc.) remain without extra padding

* Remove duplicate padding from dashboards list page

- Remove pb-2 px-6 pt-8 from itemsListPage div
- Padding now comes from BodyWrapper parent wrapper
- Avoids double padding on dashboards list

* Align dashboard button and table checkbox with page header

* Fix double padding issue with standardized table padding system

- Remove global padding wrapper from BodyWrapper to fix double padding
- Add addPadding config option to Table component
- Update table pages to use addPadding: Dashboards, API Catalog, Issues, Monitors
- Standardize all table padding to px-6 pt-4 pb-2
- Update Monitors page section padding to match standard

* fix tables and compilation

* Auto-format code with fourmolu

* fix dulicate line

* Auto-format code with fourmolu

* fix build

* Refactor FromJSON instances to use deriving strategies

- Use deriving via with DAE.Rename for field name transformations

- Use deriving via with DAE.StripSuffix for simple underscore removal

- Extract SlackEventPayload constructors into separate record types

- Simplify TeamForm to use deriving anyclass for internal API

- Remove unused imports

Code reduction: ~80 lines of manual JSON parsing boilerplate eliminated

* Auto-format code with fourmolu

* Add team validation and permission checks

- Add admin-only restriction for team create/update
- Validate team members are project members
- Add email format validation for notification emails
- Add length limits to team name and handle
- Verify teams exist before adding to dashboards
- Extract dispatchTeamNotifications helper in BackgroundJobs
- Use TeamDetails record to simplify createTeam/updateTeam APIs
- Handle duplicate team handles with ON CONFLICT DO NOTHING

* Auto-format code with fourmolu

* Refactor teams feature with comprehensive improvements

Code Quality & Type Safety:

- Refactor createTeam/updateTeam to use TeamDetails record type

- Add compile-time safety with NonEmpty for softDeleteProjectMembers

- Standardize SQL array syntax (lowercase uuid[], consistent spacing)

- Add phone_numbers support to all team types and queries

Performance Optimizations:

- Optimize getTeamsVM/getTeamByHandle with LEFT JOIN + array_agg

- Use for_ instead of explicit null checks

Error Handling & Observability:

- Add logging for missing/deleted teams in monitor notifications

- Improve error handling for addTeamsToDashboards

- Add graceful fallback when configured teams are deleted

Code Cleanup:

- Extract dispatchTeamNotifications helper function

- Remove redundant project query

- Use idiomatic Haskell patterns

* Auto-format code with fourmolu

* Fix variable shadowing in email validation

* Refactor for code succinctness

- Simplify NonEmpty to Vector conversion (remove redundant toList)

- Extract handleQueryMonitorThreshold into focused helper functions

- Add logMissingTeams for team logging

- Add createAndInsertIssue for issue creation

- Add calculateThreshold for threshold calculation

- Add buildMonitorAlert for alert construction

- Add sendNotifications for notification dispatch

- Add sendFallbackNotifications for fallback handling

* Auto-format code with fourmolu

---------

Co-authored-by: Yussif Mohammed <yousiph77@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-18 23:38:00 +01:00
Yussif Mohammed
a85be17184
Cherry pick files (#232)
* cherry pick files

* Auto-format code with fourmolu

* add more files

* Auto-format code with fourmolu

* finx undefined var

* review changes

* Auto-format code with fourmolu

* discord channel' type rename

* Auto-format code with fourmolu

* claude rec fixes

* Auto-format code with fourmolu

* fix notification typo

* Auto-format code with fourmolu

* discord get channel log error

* fix typo

* rm reswap related code from System.Types.hs

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
2025-12-18 12:13:39 +01:00
Anthony Alaribe
e212df0b96
Tests in prs. And fix hlint warnings and failing UI tests (#230)
* Enable tests and hlint in pull request workflow

- Add hlint job to run on all PRs

- Make test jobs run even if build fails (using if: always())

- Fix shadowing binding in Table.hs colAttrs function

- Fix LLM.hs type error by using langchain-hs standard pattern

- Fix TypeScript error in log-worker-functions.ts with optional chaining

This ensures CI checks run independently and provide feedback even when some jobs fail.

* Convert PR workflow from stack to cabal

- Replace stack with cabal commands throughout

- Update caching to use dist-newstyle and ~/.cabal/store

- Use awalsh128/cache-apt-pkgs-action for system deps

- Set OpenSSL env vars properly for GitHub Actions

- Add fail-fast: false to test matrix

- Add ::group:: annotations for better log organization

* Revert LLM.hs changes - they were stack-related, not actual bugs

* minor linting error

* run hlint refactor in prs

* Apply hlint fixes across codebase

Fixed 239 hlint warnings: use toText, simplify encoding chains, replace fromMaybe with maybe, use ordNub, convert to guards, use unless, eta reduction, list comprehensions, newtype conversions, and remove unused imports. Updated .hlint.yaml to allow flexible imports for Data.List and Data.Aeson.

* Auto-format code with fourmolu

* Fix UI test failures in GitHub Actions

Update Node.js version from 18 to 20 to satisfy happy-dom requirement

Add CSS.escape mock for Monaco Editor theme initialization

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-17 00:34:01 +01:00
Anthony Alaribe
e9f828e866
Attempt at a versatile table component (#229)
* attempt at a versatile table component

* Auto-format code with fourmolu

* pr updates based on feedback

* Auto-format code with fourmolu

* refactor

* fix ui differences in the tables

* Auto-format code with fourmolu

* remove redundancies

* Auto-format code with fourmolu

* remove redundancies

* Auto-format code with fourmolu

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-16 22:38:46 +01:00
Yussif Mohammed
03107e3adc fix tests and docker file 2025-11-28 10:28:18 +00:00
Anthony Alaribe
ba651f5040 Fix GHC 9.12 TypeAbstractions errors in test files
Replace type synonym Projects.ProjectId with UUIDId constructor
in all test files to fix "Illegal term-level use of type constructor"
errors that occur with GHC 9.12's TypeAbstractions extension.
2025-11-27 23:29:55 +01:00
Anthony Alaribe
4fe33fac2e Consolidate Telemetry, Anomalies, and Monitors modules
- Merged Pages/Telemetry/Metrics.hs and Trace.hs into Pages/Telemetry.hs
- Moved Pages/Anomalies/AnomalyList.hs to Pages/Anomalies.hs
- Moved Pages/Monitors/Alerts.hs to Pages/Monitors.hs
- Fixed FromRow type errors in several files (Only wrapper)
- Fixed import issues in Pages/Projects.hs
2025-11-27 23:29:54 +01:00
Anthony Alaribe
318b62d430 Consolidate Pages/Projects into single module
Merged CreateProject, Integrations, ListProjects, and ManageMembers
into Pages.Projects, reducing ~60 lines from import overhead.
2025-11-27 23:29:38 +01:00
Anthony Alaribe
ddf832f621 Consolidate modules to reduce codebase size
- Delete unused Pkg.Google (empty) and Models.Projects.ProjectsEmail (unused)
- Merge Pkg.Parser.Core into Pkg.Parser.Expr
- Merge Models.Apis.Fields.Query into Fields.Types, delete Fields.hs wrapper
- Merge Models.Projects.LemonSqueezy into Projects.hs
- Merge Pkg.Components.Modals into Pages.Components
- Merge Models.Apis.Formats into Models.Apis.Fields.Types

Reduces codebase from ~33k to ~32.8k lines by eliminating 9 modules.
2025-11-27 23:29:16 +01:00
Anthony Alaribe
e1397ea721 Fix type mismatch in ProjectsSpec integration tests
The ListProjectsGet handler returns a 3-tuple containing projects, demo project, and showDemoProject flag. Updated test destructuring to match the actual return type.
2025-11-23 21:01:20 +01:00
Anthony Alaribe
873b71e13c Refactor test infrastructure and fix onboarding session caching
- Add refreshSession and runTestEffect helpers to eliminate duplication
- Skip session cache for ONBOARDING projects (actively being updated)
- Use pattern guards in sessionAndProject for cleaner control flow
- Simplify test setup: UPDATE sudo status instead of INSERT with conflict handling
- Restructure BusinessFlowsSpec with TestContext and withTestProject
- Rename projectOnboarding to projectOnboardingH for consistency
- Fix LemonSqueezy payment plan enum (FREE vs Free)
2025-11-17 17:16:41 +01:00
Anthony Alaribe
4b0e92659b Fix integration test compilation errors after testServant refactoring
Update all integration tests to properly unpack tuple response from testServant helper, which now returns (RespHeaders a, a).

Key changes:
- Add servant and servant-htmx to integration test dependencies
- Fix tuple unpacking in all test specs (AnomalyList, Log, Alerts, ManageMembers, Projects, BusinessFlows, Reports, Api, ApiCatalog, GrpcIngestion)
- Fix BusinessFlowsSpec imports to use ResponseHeader data constructor
- Fix type-level string usage for header lookups
- Fix ProjectId to Text conversions using .toText
- Add missing pattern match cases to avoid incomplete pattern warnings
2025-11-17 11:30:07 +01:00
Anthony Alaribe
c13704adf3 Rename onboarding handlers for consistency and cleanup config
- Renamed onboardingInfoPost -> onboardingInfoPostH
- Renamed onboardingConfPost -> onboardingConfPostH
- Updated Routes to use renamed handlers
- Removed unused notifChannels function
- Cleaned up BusinessFlowsSpec comments
- Updated auto-instrument config (commented out ATBackgroundCtx target and some span attributes)
2025-11-17 09:25:29 +01:00
Anthony Alaribe
f10795b158 Refactor integration tests to use simplified testServant helper
Simplified test code by:
- Replacing verbose toServantResponse pattern with concise testServant helper
- Changed test lambdas from \tr@TestResources{..} to \tr for cleaner syntax
- Updated field access to use dot notation (tr.trPool, tr.trATCtx, etc.)

This reduces boilerplate and improves test readability across all integration
test suites while maintaining the same functionality.

Affected test suites:
- Pages (ApiSpec, LogExplorer, Anomalies, Projects, Endpoints, Reports,
  Monitors, BusinessFlows, Share)
- MonitoringSpec
- ProcessMessageSpec
2025-11-17 09:04:28 +01:00
Anthony Alaribe
04e21f37ee Add business flows integration test suite
Add comprehensive integration tests for business-critical workflows:

- Onboarding flow: Test all steps (Info, Survey, NotifChannel,
  Integration) through handlers, verifying data persistence via
  GET endpoints
- LemonSqueezy billing: Table-driven tests for webhook events
  (subscription_created, cancelled, resumed, expired)
- Replay session recording: Test event ingestion and retrieval
- S3 configuration: Validation and removal tests (pending Minio setup)

Tests use the new structured response types to verify handler behavior
without HTML parsing. Includes table-driven testing patterns for
webhook scenarios and helper functions for creating test payloads.
2025-11-16 01:46:02 +00:00
Anthony Alaribe
c8a9007920 Refactor OTLP test helpers into TestUtils module
Move test helper functions for creating API keys and ingesting
telemetry data (logs, traces, metrics) from GrpcIngestionSpec into
the shared TestUtils module. This includes:
- createTestAPIKey: Create API keys via handler
- ingestLog/Trace/Metric: Ingest via resource attributes
- ingestLogWithHeader/TraceWithHeader/MetricWithHeader: Ingest via gRPC Authorization header
- toBaseServantResponse: Run base context handlers in tests

Add necessary imports (Proto, OTLP modules, Api module) and derive
Generic for AuthContext to support the new test infrastructure.
2025-11-16 01:43:22 +00:00
Anthony Alaribe
8477228369 Update integration tests to pass logger to logging helper
Pass logger parameter to logBackgroundJobsInfo calls to support
the new structured logging interface.
2025-11-15 21:48:42 +00:00
Anthony Alaribe
c28d1f6f20 Update test infrastructure to support log levels
Load environment config in test setup to respect LOG_LEVEL setting.
Update logBackgroundJobsInfo to use structured logging and accept logger parameter.
2025-11-15 21:48:12 +00:00
Anthony Alaribe
8c2b7e1b3e Add gRPC metadata authentication and fix grapesy API compatibility
Metadata authentication:
- Add OtlpRequestMetadata type to parse Authorization header from gRPC metadata
- Support both 'Bearer <token>' and raw token formats
- Update processTraceRequest, processLogsRequest, processMetricsRequest to accept
  metadata API key parameter
- Add RPC handlers that extract API key from gRPC metadata
- Combine metadata API key with resource attribute keys for authentication
- Add test helpers for ingesting telemetry with gRPC headers

Grapesy API compatibility (for upgraded library):
- Replace NoMethods with NoMoreMethods constructor
- Update sendFinalOutput calls to pass tuple (defMessage, NoMetadata)
- Add type instance declarations for response metadata
2025-11-15 20:06:37 +00:00
Anthony Alaribe
499b7038c5 Fix test assertions and exception handling in GrpcIngestionSpec
Update expectLogsJson helper to use fail instead of expectationFailure
for proper IO type compatibility, add handling for LogsPatternList
constructor, and fix Test 3.2 to properly expect GrpcUnauthenticated
exception when API key is missing.
2025-11-15 05:59:13 +00:00
Anthony Alaribe
32a4be21c4 Fix type signatures and add missing import in GrpcIngestionSpec
- Add Data.Aeson qualified import
- Fix queryLogs return type: LogsResp -> LogsGet
- Make expectLogsJson type more specific
- Add LogsQueryLibrary case to expectLogsJson pattern match
2025-11-15 05:59:12 +00:00
Anthony Alaribe
8b0d315756 Add test infrastructure for gRPC ingestion tests
- Export gRPC handlers and process functions from OtlpServer for testing
- Add runQueryEffect helper in TestUtils for chart query testing
- Fix ProjectsSpec test to verify updates in list instead of relying on index
2025-11-15 05:59:12 +00:00
Anthony Alaribe
b977108665 Refactor GrpcIngestionSpec with helper functions
- Add testTimeRange constant for common time range setup
- Add helper functions: ingestLog, ingestTrace, ingestMetric
- Add queryLogs helper to replace repetitive 22-parameter apiLogH calls
- Add expectLogsJson helper for unified response extraction
- Use traverse and zip/zip3 for cleaner iteration patterns
- Reduced from 447 lines to 176 lines (60% reduction)
- All tests still target complete handler chains
2025-11-15 05:59:11 +00:00
Anthony Alaribe
a4ccb14bd2 Refactor integration tests to use TestResources pattern
Update integration tests to use the new TestResources pattern for better
test organization and consistency.

Changes:
- Replace withSetup callback with withTestResources
- Use runTestBg helper instead of runTestBackground
- Access test resources via TestResources record fields
- Remove redundant imports (execute from DBT)
- Simplify test setup code

This refactoring improves test readability and reduces boilerplate.
2025-11-15 05:59:09 +00:00
Anthony Alaribe
5ab5c35edd improve background job handling and make tests more precise in job handling 2025-11-15 05:54:10 +00:00
Anthony Alaribe
614715b313 fix tests and reduce default limit to 500 for weaker conections 2025-10-26 23:10:32 +01:00
Anthony Alaribe
b9ed4c58be fix unit tests 2025-10-26 22:42:05 +01:00
Yussif Mohammed
d6fa7c1f66 fix integration tests with events table updates enabled 2025-10-22 21:00:39 +00:00
Anthony Alaribe
c66d11b7c3 fix unit test unsuported test case 2025-10-18 02:21:48 +02:00
Yussif Mohammed
446604315f fix test and remove redundant imports 2025-10-14 18:14:58 +00:00
Anthony Alaribe
f344f88dac fix the argument issues from apiLogH in the tests, so tests compile 2025-10-12 20:17:13 +02:00
Yussif Mohammed
172d56083d fix tests 2025-10-05 07:58:27 +00:00
Yussif Mohammed
3b2fcae21b fmt and fix tests 2025-10-02 11:22:14 +00:00