Commit graph

4,223 commits

Author SHA1 Message Date
Anthony Alaribe
0c48177549 Add LLM effect with golden test caching for issue enhancement
Implements a new effectful LLM module that enables caching of OpenAI API
calls for golden tests while making real API calls in production.

Changes:
- Add Data.Effectful.LLM module with runLLMReal and runLLMGolden interpreters
- Update Pkg.AI to add effectful version (callOpenAIAPIEff)
- Update Models.Apis.Issues.Enhancement to use LLM effect
- Add LLM to ATBackgroundCtx effect stack
- Wire runLLMReal in production runner (System.Types)
- Wire runLLMGolden in test runner (Pkg.TestUtils)
- Update BackgroundJobs to call effectful Enhancement functions
- Load OPENAI_API_KEY from environment for tests

The LLM effect caches API responses in ./tests/golden/ directory during
tests, preventing unnecessary API calls and making tests deterministic.
2025-11-15 05:59:08 +00:00
Anthony Alaribe
0916880780 enable parallel test execution and fix race conditions
- Add dynamic CPU detection to Makefile for optimal parallel execution
- Make background job concurrency configurable (default: 4)
- Implement concurrent job processing in tests using Ki structured concurrency
- Fix race condition in insertIssue by using correct ON CONFLICT constraint

This reduces test execution time while ensuring thread-safe database operations.
2025-11-15 05:59:08 +00:00
Anthony Alaribe
8a08da4b46 optimize span processing performance and add tracing instrumentation
- Replace O(n²) nubBy with O(n) HashMap-based deduplication for endpoints, shapes, fields, and formats
- Batch UUID generation upfront to avoid repeated IO operations per span
- Make entity extraction pure computation by pre-generating all UUIDs
- Add OpenTelemetry tracing spans for pubsub and kafka batch processing with error tracking
- Add hashable dependency for HashMap-based deduplication
- Remove unnecessary threadDelay in test utilities
- Add strictness annotations to improve memory usage

These changes significantly reduce the algorithmic complexity and IO overhead of span processing while improving observability.
2025-11-15 05:59:07 +00:00
Anthony Alaribe
a4184a3f27 fix span hash updates to use correct array type conversion
The hashes column in otel_logs_and_spans is text[] but was being updated
with jsonb arrays, causing type errors. Use LATERAL JOIN with
jsonb_array_elements_text to properly convert variable-length JSONB arrays
to PostgreSQL text[] arrays, avoiding multidimensional array constraints.
2025-11-15 05:55:21 +00:00
Anthony Alaribe
dfc2752af1 optimize span processing with batched database updates
Replace sequential per-span UPDATE queries with a single batched update
using PostgreSQL UNNEST. This reduces processing time from ~150ms per span
to a single bulk operation, improving throughput from 30+ seconds for 200
spans to under 1 second.

Changes:
- Batch span hash updates using UNNEST with uuid[] and jsonb[] arrays
- Filter spans with empty hashes before batching
- Maintain same safety guarantees with timestamp and project_id filters
2025-11-15 05:54:10 +00:00
Anthony Alaribe
5ab5c35edd improve background job handling and make tests more precise in job handling 2025-11-15 05:54:10 +00:00
Anthony Alaribe
f6a0ca6ca0 increase virtual list work timeout duration 2025-11-15 05:54:10 +00:00
Yussif Mohammed
d67610f7ec format 2025-11-14 12:55:40 +00:00
Yussif Mohammed
a4d1aad8f7 fx test 2025-11-14 12:40:22 +00:00
Yussif Mohammed
f1f34172b2 fix reports page now working in prod 2025-11-14 11:32:41 +00:00
Yussif Mohammed
b90acccce1 fix empty state 2025-11-13 13:14:46 +00:00
Yussif Mohammed
60aa4ae653
New Reports UI (#207)
* checkpoint

* span overview checkpoint

* reports checkpoint, slow queries

* empty state

* auto load first report

* format

* fetch and save neccesary chart data for report

* feat: issues breakdown

* touches

* issues id

* format
2025-11-13 07:45:20 +00:00
Yussif Mohammed
0f9ef86e8c fix span detail loader in the middle of screen 2025-11-09 16:57:39 +00:00
Yussif Mohammed
11e37767a0 fmt 2025-11-09 13:20:29 +00:00
Yussif Mohammed
7bdc84f94c traces widget touches 2025-11-09 13:19:41 +00:00
Yussif Mohammed
58bac979bd fix name error 2025-11-09 12:30:32 +00:00
Yussif Mohammed
31fd5edb4d fix telemetry trace rendering issue 2025-11-09 09:11:39 +00:00
Yussif Mohammed
291777966f get all necessary data in one call 2025-11-08 12:18:48 +00:00
Yussif Mohammed
f0cf1cef63 widget span list view 2025-11-07 11:27:43 +00:00
Yussif Mohammed
392e341c0f fmt 2025-11-07 09:06:42 +00:00
Yussif Mohammed
0f15612621 use single query param for trace id and timestamp 2025-11-06 12:44:58 +00:00
Yussif Mohammed
90c687f707 feat: add timestamp to trace view 2025-11-06 12:33:03 +00:00
Yussif Mohammed
343cc3f665 fix flamegraph colors 2025-11-05 18:57:57 +00:00
Yussif Mohammed
acf2dfc7d8 fix view parent trace view 2025-11-05 13:12:15 +00:00
Yussif Mohammed
a6f1260c52 fmt 2025-11-04 21:21:28 +00:00
Yussif Mohammed
2b129af05e towards traces widget 2025-11-04 21:18:20 +00:00
Anthony Alaribe
1e1a3ca9a3 make fmt 2025-11-03 19:53:15 +01:00
Anthony Alaribe
3d1ea9b47f fix deployment 2025-11-03 19:25:54 +01:00
Anthony Alaribe
90ec5eed5e standardize the navTabs styles 2025-11-02 15:26:12 +01:00
Anthony Alaribe
6efccdecb3 fix sizing inconsistencies 2025-11-02 14:34:48 +01:00
Anthony Alaribe
7d4d27e15c reduce the paddings of the tabs 2025-11-02 13:57:50 +01:00
Anthony Alaribe
ab5bb2b369 fix bug so the visual range is displayed on chart load 2025-11-02 13:17:47 +01:00
Anthony Alaribe
94bfe42f81 more transparent mark area 2025-11-02 12:59:09 +01:00
Anthony Alaribe
16d87ac16b fix the issue of latency widths starting as wider than they actually should be, and resizing on scroll 2025-11-01 18:21:18 +01:00
Anthony Alaribe
7cbc95e917 fix the issue of latency widths starting as wider than they actually should be, and resizing on scroll 2025-11-01 18:20:25 +01:00
Anthony Alaribe
ce89f5943c fix clearing the create alert form on submission 2025-11-01 18:15:05 +01:00
Anthony Alaribe
92bd57a47e add the color to new rows as an indicator 2025-11-01 15:34:05 +01:00
Anthony Alaribe
f66ce4e6d5 fix query for streaming in new data. set the from query param, not to 2025-11-01 15:11:13 +01:00
Anthony Alaribe
acb1b658e0 Fix issue where when a query is deleted, the suggestions don't get triggered. 2025-11-01 14:48:48 +01:00
Anthony Alaribe
b97b0e9273 remove some unused code and optimize height calc for query-editor 2025-11-01 14:12:00 +01:00
Anthony Alaribe
0bd66375ac remove some of the perf measures temporarily 2025-11-01 13:57:09 +01:00
Anthony Alaribe
8c4045e001 disable expensive monaco configs 2025-11-01 13:52:30 +01:00
Anthony Alaribe
a125d7f187 more regex optimizations 2025-11-01 13:45:18 +01:00
Anthony Alaribe
011781ae5f remove unused query-editor functions 2025-11-01 13:29:58 +01:00
Anthony Alaribe
81321acb9c only trigger the query event with a 500ms debounce and if the query has changed 2025-11-01 13:18:55 +01:00
Anthony Alaribe
07705b0510 prepare the regexes and reuse them to improve performance on keypress 2025-11-01 13:00:39 +01:00
Anthony Alaribe
42a7a30261 fix the cursor handling logic for query logs. And add performance logic and checks so we can find out whats making the query-editor sluggish 2025-10-31 18:11:11 +01:00
Anthony Alaribe
ed441d1337 fix the cursor handling 2025-10-31 18:11:11 +01:00
Anthony Alaribe
dc83b7627b remove query editor debounce delays 2025-10-31 18:11:10 +01:00
Yussif Mohammed
41eabf4b2d fmt 2025-10-29 22:10:23 +00:00