From a5a6458defaa7216f6349783c2b441ebbc0e55a5 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 18:15:54 +0200 Subject: [PATCH 001/297] docs: plan Slack Agent consolidation --- plans/2026-09-30-slack-agent-consolidation.md | 115 ++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 plans/2026-09-30-slack-agent-consolidation.md diff --git a/plans/2026-09-30-slack-agent-consolidation.md b/plans/2026-09-30-slack-agent-consolidation.md new file mode 100644 index 000000000..a88b0d989 --- /dev/null +++ b/plans/2026-09-30-slack-agent-consolidation.md @@ -0,0 +1,115 @@ +# Slack Agent consolidation plan + +Status: proposed implementation plan. Research and code inspection are complete; this plan does not claim live Slack acceptance or authorize implementation. + +This plan narrows the [Slack Agent roadmap](2026-09-08-slack-agent.md) to the unfinished conversation and tool experience. The [implementation log](2026-09-08-slack-agent-implementation.md) records earlier work. Current code, tests, and live behavior take precedence over old status notes in that log. + +## Goal and user contract + +Monoscope answers a mention in a channel or a message in its Agent view. An issue alert opens the same investigation that users can continue in Slack or Monoscope. The agent can discover and use every meaningful project operation exposed by Monoscope's MCP server and CLI, subject to the linked user's permissions. + +The first acceptance question comes from the reported screenshot: `@Monoscope what metrics can you access`. In a newly joined private channel, a linked user must receive a visible response. The response must list real metric names from the authorized project's catalog, with scope and recency stated. An unlinked user must receive a private account-link action. Both paths must produce a visible outcome. + +“Slack Agent” means Slack's native Agent experience, configured through the Slack app's Agent features. The `APP` badge in a message does not prove that `agent_view` is enabled. The [manifest template](../docs/slack/agent-manifest.json) has not been applied or accepted in a live workspace. + +## Current state and gaps + +| Area | Current server path | Gap to close | +| --- | --- | --- | +| Slack intake | `Pages.Bots.Slack` verifies signed events, saves a receipt, and queues `processSlackEvent`. | Trace a first private-channel mention to a visible outcome. The screenshot does not identify the failing boundary. | +| Conversation | Slack thread binding, issue conversation IDs, history, turn checkpoints, and reply reservations exist. | Make the issue conversation the stable record across Slack and Monoscope. Prove recovery after retries, interruption, and follow-ups. | +| Native Agent | `agent_view` exists in a template; the worker calls `agents.sessions.setStatus`. | Apply the manifest in a sandbox and prove the Agent view, session lifecycle, stop action, and channel mentions. | +| Tools | `Pkg.AI.allToolDefs` and `executeToolCall` are a small hand-written list. `Web.MCP.allTools` derives API tools and adds composites. | Use one project capability catalog for agent discovery and execution. Remove duplicate tool contracts. | +| Metrics | MCP exposes `query_metrics`. `Telemetry.getMetricCatalogPage` serves the web catalog. | Add a bounded metric-discovery operation. Querying a known metric cannot answer which metrics exist. | +| Authorization | Slack identity resolution rechecks active project membership. | Carry the member's permission into each tool call. Membership alone cannot authorize administrative writes. | + +The first-mention failure and the metric-discovery gap are separate defects. A missing metric tool can produce an incomplete answer; it does not explain the absence of any reply. Production logs or a controlled Slack replay are needed to locate that failure. + +## Feature tree + +1. **Reliable entry and delivery** + - Mention in a public or private channel; direct message; Agent view prompt. + - Private account link, reconnect notice, working state, answer, or actionable failure. + - Durable retry without duplicate answers or abandoned `processing` status. +2. **One issue conversation** + - Bind an alert root, Slack thread, incident episode, issue, and in-app chat to one project-scoped conversation. + - Keep user messages, evidence, tool outcomes, progress, decisions, and final answers in order. + - Accept follow-ups and stop requests while work is active; resume from a saved boundary. +3. **Complete project tools** + - Discover tools by task, then load the selected schema and execute it. + - Cover telemetry, metric catalog and queries, issues, incidents, monitors, dashboards, endpoints, log patterns, project settings, repository evidence, and other MCP/API project operations. + - Map CLI project operations to the same server capabilities. CLI-local login, configuration, completion, and terminal rendering remain client functions. + - Apply project scope, current membership, permission, action intent, and required confirmation at execution time. +4. **Evidence-led replies** + - Show the query window, metric or event source, links, observed facts, hypotheses, and missing evidence. + - Keep progress in the originating thread. Report later action outcomes there. +5. **Native Agent experience** + - Agent view, suggested prompts, session status, stop behavior, and threaded replies. + - Add streaming or richer task progress only after the durable answer path passes acceptance. + +Ambient channel suggestions and autonomous fixes are later features. They require their own noise, authority, and outcome rules. + +## Server design + +### Keep one conversation worker + +Extend the existing signed receipt, thread lock, turn checkpoint, and reply-publication path. Do not add a second Slack-specific agent executor. A queue item identifies stored work; persisted state decides what the worker runs. The worker owns input ordering, model work, tool calls, and delivery for one conversation at a time. + +Treat Slack as an input and delivery adapter. The issue conversation owns history and tool evidence. A later Slack message joins that conversation by the stored thread and issue binding, not by text matching. A new mention in an unrelated thread starts a new conversation. A notification root with an issue ID binds to that issue's conversation. + +Record each input and output boundary. A retry must distinguish “Slack accepted the reply” from “Slack did not accept it.” Unknown delivery outcomes use the existing reservation and reconciliation mechanism before another post. + +### Share the capability contract + +Use the MCP/API operation catalog as the coverage baseline. Each operation needs a stable name, description, input schema, result shape, project scope, and action class. Agent discovery returns short matching summaries; the selected operation returns its full schema. The model must not receive the entire MCP catalog on every turn. + +`Web.MCP.runTool` currently dispatches some operations through an in-process Servant application under `ATBaseCtx`; the Slack worker uses `ATBackgroundCtx`. Directly calling that function from the worker is not a safe consolidation plan. Share operation definitions and underlying domain actions through an explicit adapter. Keep one authorization and result contract for MCP and Agent callers, with their distinct principals supplied at the boundary. + +Add metric discovery from `Telemetry.getMetricCatalogPage`. Return bounded pages with name, type, unit, description, labels, last-seen time, and active status. Accept the catalog's service filter. State when a page is incomplete. Do not enumerate metric names by scanning raw datapoints; the existing catalog is the intended bounded source. + +Generate a coverage inventory from the MCP registry and CLI command tree. Every project operation must map to a shared capability, or have a documented client-only reason. A parity test must fail when a new MCP project operation lacks an Agent policy. A missing policy must not grant access by default. + +### Authorize each action + +Resolve the Slack user to a project member before work and again before each tool and reply. Include the member's permission, not only the project ID. Read tools use the current user's allowed project scope. Writes use the same domain permission rules as the product operation. + +An explicit user request is required for a write. Destructive or administrative actions, such as key management, member changes, and deletion, require a reviewable confirmation bound to the operation, arguments, project, and requester. Recheck permission after confirmation and before execution. Tool results must state what the server actually changed. + +The tool catalog can describe an unavailable action without executing it. The agent explains the missing permission or required confirmation. It must not use a service credential to bypass the linked user's role. + +### Implementation touchpoints + +- `Pages.Bots.Slack` owns signed intake, identity prompts, native session status, history backfill, and reply publication. +- `Pkg.AI` owns the model loop, checkpoints, Agent access checks, and current tool dispatch. +- `Web.MCP`, `Web.Routes`, and `Web.ApiHandlers` define the existing project API and MCP coverage baseline. +- `Models.Telemetry.Telemetry` supplies metric catalog pages. `Models.Apis.Integrations` resolves the linked Slack principal. +- `test/integration/Pages/Bots/WorkflowsSpec.hs` exercises signed Slack handlers and workers. Add regressions there for visible first-turn outcomes and issue continuity. + +## Delivery sequence and acceptance + +| Slice | Work | Evidence required before the next slice | +| --- | --- | --- | +| 1. First mention | Reproduce the screenshot path with a signed `app_mention` in a newly joined private channel. Inspect receipt, identity, scopes, startup status, thread history, model start, and publication. Add the failing regression before the fix. | Linked and unlinked users each get the expected visible outcome. Retry produces one answer. A failure records a boundary and leaves no silent pending turn. | +| 2. Metric answer | Expose the paged metric catalog and metric query through the Agent tool path. Keep project scope fixed by the linked identity. | “What metrics can you access?” names seeded metrics, reports recency and pagination, and never claims that an empty or failed read means no metrics exist. | +| 3. Tool parity | Add shared discovery and execution contracts, a CLI/MCP coverage inventory, and permission policy. Move existing Agent tools onto that contract without changing their result meaning. | Representative read and write operations agree across MCP, CLI, and Agent. Cross-project, revoked-member, lower-role, and missing-confirmation cases fail before side effects. | +| 4. Issue conversation | Use the existing issue binding, stored history, checkpoints, and delivery reservations for Slack and in-app follow-ups. | An alert, diagnosis, human reply, tool result, stop, retry, and recovery form one ordered issue conversation without duplicate posts. | +| 5. Native rollout | Apply the reviewed manifest to a sandbox installation, reconnect scopes, and run the Agent acceptance matrix. Keep notification delivery intact. | The Agent view, suggested prompts, DM, channel mention, session status, stop, and thread follow-up work in Slack. Verify private-channel history and rate limits on the actual installation. | + +Use `build.log` and `build-test-dev.log` watchers for Haskell feedback. Keep integration regressions at the signed handler, worker, database, and Slack HTTP boundary. Use model fakes only at the LLM effect boundary. Before any pull-request update, run the repository's Haskell reviews and `make ci-signoff` as `AGENTS.md` requires. + +## Live diagnosis and rollout gates + +For the reported no-reply case, collect the workspace, channel, message timestamp, event ID, installation ID, and receipt ID. The screenshot contains none of these identifiers. Inspect whether Slack delivered `app_mention`, whether Monoscope queued and ran the receipt, and which external call last succeeded. Do not label a guessed failure as the root cause. + +The sandbox matrix covers a linked member, an unlinked user, a revoked member, an admin, a lower-role member, a new private channel, an existing issue thread, a DM, and the Agent view. Repeat each case after a duplicate event, worker interruption, Slack rate limit, and ambiguous publication outcome. Verify that the original alert transport still delivers notifications. + +Slack's Agent migration can be irreversible. Review the live manifest against the template before changing it. Record the workspace plan, app settings, granted scopes, event subscriptions, and observed Agent behavior. Code and OAuth scopes alone do not prove native Agent availability. + +## Prior art and decisions + +- [Slack's Agent guide](https://docs.slack.dev/ai/developing-agents/) defines the native response loop, Agent features, session status, and streaming methods. +- [Junior's conversation worker](https://github.com/getsentry/junior/blob/0dfd0e805789103be20ad20338b2117407f9c397/packages/junior/src/chat/task-execution/README.md) uses durable input, one lease per conversation, and delivery state. Monoscope can apply these rules in its existing server worker. +- [Junior's MCP search](https://github.com/getsentry/junior/blob/0dfd0e805789103be20ad20338b2117407f9c397/packages/junior/src/chat/tools/skill/search-mcp-tools.ts) separates tool discovery from execution. Monoscope needs that split for full tool parity. +- [Polylane's Slack flow](https://polylane.com/product/slack/) shows mentions, threaded follow-ups, a live checklist, and outcomes in the originating thread. Its [investigation model](https://docs.polylane.com/investigation/investigations) keeps one issue's evidence and fix outcome in one thread. These are product references, not claims about its internal implementation. + +The recommended launch boundary is reliable requested conversations and authorized tools. Ambient replies, autonomous fixes, and automatic deployment follow only after the requested path passes live acceptance. From 166647e023b979018aea3290e7c3827e26dc61bc Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 19:17:27 +0200 Subject: [PATCH 002/297] Sessions view: truncated identities carry a tooltip; a nonzero rate never prints as 0.0% The row's user or session id is clipped at 24ch with no way to read the rest; and 3 errored sessions of 10K rounded to "0.0%" on the Errored tile. fmtPct1 (all ten callers) now says "<0.1%" for a rate that would round to zero but is not zero. --- src/Pages/LogExplorer/Log.hs | 8 +++++++- web-components/src/log-list.ts | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/Pages/LogExplorer/Log.hs b/src/Pages/LogExplorer/Log.hs index ad13f0bbb..04ef82592 100644 --- a/src/Pages/LogExplorer/Log.hs +++ b/src/Pages/LogExplorer/Log.hs @@ -1148,8 +1148,14 @@ logLatencyWidget pid = -- "100.0%" -- >>> LL.fmtPct1 (-1.25) -- "-1.2%" +-- | One-decimal percentage; a rate that would round to zero but is not zero says so. +-- +-- >>> map fmtPct1 [0, 0.03, 0.05, 4.14, 100] +-- ["0.0%","<0.1%","0.1%","4.1%","100.0%"] fmtPct1 :: Double -> Text -fmtPct1 x = toText (showFFloat (Just 1) x "") <> "%" +fmtPct1 x + | x > 0 && x < 0.05 = "<0.1%" + | otherwise = toText (showFFloat (Just 1) x "") <> "%" -- | Shimmer placeholder mirroring 'sessionsHeader_' (6-KPI grid + over-time bar diff --git a/web-components/src/log-list.ts b/web-components/src/log-list.ts index 97dc50cec..e77871318 100644 --- a/web-components/src/log-list.ts +++ b/web-components/src/log-list.ts @@ -3923,7 +3923,7 @@ export class LogList extends LitElement { parts.push(content); }; - if (user) add(html`${user}`); + if (user) add(html`${user}`); if (url) { // Middle-truncate: keep the last path segment visible since it usually // identifies the page ("/checkout/cart" is more useful than "/api/v2/…"). From 2cfba01b6d8100eebe7579be963ab27f98136b78 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 19:23:12 +0200 Subject: [PATCH 003/297] Speed up local CI signoff (#608) * Skip attested checks before starting local CI services * Overlap integration and browser checks in local signoff * Bring full local CI signoff under four minutes * Keep browser checks responsive under full signoff load * Run built test binaries during local signoff * Trim redundant signoff work and stabilize browser readiness --- .github/workflows/pullrequest.yml | 2 +- docs/local-ci.md | 4 +- e2e/playwright.config.ts | 2 +- e2e/tests/container-identity.spec.ts | 1 + e2e/tests/dashboard-add-widget.spec.ts | 10 +- monoscope.cabal | 4 +- package.yaml | 2 - scripts/ci/ci.sh | 214 +++++++++++++++++++------ scripts/e2e.sh | 2 +- test/integration/Main.hs | 13 +- 10 files changed, 192 insertions(+), 62 deletions(-) diff --git a/.github/workflows/pullrequest.yml b/.github/workflows/pullrequest.yml index 9a36f2284..2cc8f4165 100644 --- a/.github/workflows/pullrequest.yml +++ b/.github/workflows/pullrequest.yml @@ -285,7 +285,7 @@ jobs: if: needs.gate.outputs.skip_hlint != 'true' id: refactor run: | - hlint src/ --refactor --refactor-options="--inplace" || true + make fix-lint || true # Recorded before the push is attempted, because the push is what forks # and dependabot can't do — the tree still changed either way, and that # is what decides whether the check below may be attested. diff --git a/docs/local-ci.md b/docs/local-ci.md index 53b28b9ec..7d31fb19a 100644 --- a/docs/local-ci.md +++ b/docs/local-ci.md @@ -100,7 +100,7 @@ Two deliberate differences from your normal `make test`: - **Build directories are container-private.** `dist-newstyle`, all three `node_modules` directories, npm's download cache, and Playwright's browser download are named volumes. Your host's macOS/arm64 artifacts would corrupt the Linux build. - The volumes persist, so the second `make ci` is + Volumes are separate for each worktree and persist, so the second `make ci` there is fast — but the **first one is a cold build** and takes as long as a cold CI run. Start it and go do something else; every run after that is incremental. `make ci-down` keeps them; `make ci-clean` deletes them and buys you the cold @@ -111,7 +111,7 @@ Two deliberate differences from your normal `make test`: Checks that do not need services start only the runner; `e2e` starts Postgres, and `integration-tests` starts all three services. When every selected check is already attested, `make ci` skips Docker entirely. - Local CI runs serialize automatically because they share build volumes and service databases. + Local CI runs serialize automatically to avoid overloading the host. Worktree runs mount their shared Git metadata so attestation fingerprints work in containers. If HLint is installed on the host, `make ci` runs that check there, as the GitHub workflow does. The host passes its attestation-ref list to the runner, which has no SSH keys. diff --git a/e2e/playwright.config.ts b/e2e/playwright.config.ts index 5a0e66deb..73408b24c 100644 --- a/e2e/playwright.config.ts +++ b/e2e/playwright.config.ts @@ -4,7 +4,7 @@ export default defineConfig({ testDir: "./tests", fullyParallel: true, // Keep browser concurrency bounded so layout and latency checks have headroom. - workers: 4, + workers: 2, forbidOnly: !!process.env.CI, retries: 0, use: { diff --git a/e2e/tests/container-identity.spec.ts b/e2e/tests/container-identity.spec.ts index 50d090c06..51794c1d1 100644 --- a/e2e/tests/container-identity.spec.ts +++ b/e2e/tests/container-identity.spec.ts @@ -35,6 +35,7 @@ test("navbar refresh menu applies Off and interval options", async ({ page }) => const menuToggle = page.locator(width < 768 ? '[popovertarget="n-timepicker-popover"]' : '[data-live-data-trigger]'); await expect(transport).toHaveAttribute("data-interval", "15000"); await expect(transport).toHaveAttribute("data-state", "live"); + await expect(page.locator(`${width < 768 ? '#n-timepicker-popover' : '#n-live-data-pop'} [data-refresh-option][data-value="15000"]`)).toHaveAttribute("aria-pressed", "true"); await menuToggle.click(); await expect(page.getByRole("button", { name: "15 seconds", exact: true })).toHaveAttribute("aria-pressed", "true"); await menuToggle.click(); diff --git a/e2e/tests/dashboard-add-widget.spec.ts b/e2e/tests/dashboard-add-widget.spec.ts index 375624870..a3d29fa1c 100644 --- a/e2e/tests/dashboard-add-widget.spec.ts +++ b/e2e/tests/dashboard-add-widget.spec.ts @@ -403,8 +403,14 @@ test("dashboard teams default to @everyone and support bulk add and removal", as const team = `e2e-team-${Date.now()}`; await page.goto(`/p/${DEMO_PROJECT}/manage_teams`); await page.getByText("New Team", { exact: true }).click(); - await page.locator('[name="teamName"]').fill(team); - await page.locator('[name="teamHandle"]').fill(team); + const teamName = page.locator('[name="teamName"]'); + const teamHandle = page.locator('[name="teamHandle"]'); + await expect(teamName).toBeFocused(); + await teamName.fill(team); + await expect(teamName).toHaveValue(team); + await teamHandle.fill(team); + await expect(teamHandle).toHaveValue(team); + await expect(teamName).toHaveValue(team); await page.getByRole("button", { name: "Create Team", exact: true }).click(); await expect(page.getByRole("link", { name: team, exact: true })).toBeVisible(); await page.goto(`/p/${DEMO_PROJECT}/dashboards`); diff --git a/monoscope.cabal b/monoscope.cabal index 6c175b3ca..6b4378f6f 100644 --- a/monoscope.cabal +++ b/monoscope.cabal @@ -793,7 +793,7 @@ test-suite integration-tests TypeFamilies UndecidableInstances ViewPatterns - ghc-options: -fwrite-ide-info -threaded -Weverything -Werror -fno-defer-typed-holes -Wno-error=deprecations -Wno-error=unused-packages -Wno-error=implicit-lift -Wno-error=missing-poly-kind-signatures -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-monomorphism-restriction -Wno-missing-safe-haskell-mode -Wno-partial-fields -Wno-prepositive-qualified-module -Wno-missing-export-lists -Wno-orphans -Wno-missing-deriving-strategies -Wno-missing-role-annotations -Wno-unused-matches -Wno-missing-kind-signatures -Wno-type-defaults -Wno-unused-top-binds -Wno-unused-imports -Wno-error=incomplete-record-selectors -Wno-ambiguous-fields -fno-write-ide-info -threaded -rtsopts "-with-rtsopts=-N -A64m" -O0 + ghc-options: -fwrite-ide-info -threaded -Weverything -Werror -fno-defer-typed-holes -Wno-error=deprecations -Wno-error=unused-packages -Wno-error=implicit-lift -Wno-error=missing-poly-kind-signatures -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-monomorphism-restriction -Wno-missing-safe-haskell-mode -Wno-partial-fields -Wno-prepositive-qualified-module -Wno-missing-export-lists -Wno-orphans -Wno-missing-deriving-strategies -Wno-missing-role-annotations -Wno-unused-matches -Wno-missing-kind-signatures -Wno-type-defaults -Wno-unused-top-binds -Wno-unused-imports -Wno-error=incomplete-record-selectors -Wno-ambiguous-fields -threaded -rtsopts "-with-rtsopts=-N -A64m" -O0 build-tool-depends: hspec-discover:hspec-discover , proto-lens-protoc:proto-lens-protoc @@ -1366,7 +1366,7 @@ test-suite unit-tests TypeFamilies UndecidableInstances ViewPatterns - ghc-options: -fwrite-ide-info -threaded -Weverything -Werror -fno-defer-typed-holes -Wno-error=deprecations -Wno-error=unused-packages -Wno-error=implicit-lift -Wno-error=missing-poly-kind-signatures -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-monomorphism-restriction -Wno-missing-safe-haskell-mode -Wno-partial-fields -Wno-prepositive-qualified-module -Wno-missing-export-lists -Wno-orphans -Wno-missing-deriving-strategies -Wno-missing-role-annotations -Wno-unused-matches -Wno-missing-kind-signatures -Wno-type-defaults -Wno-unused-top-binds -Wno-unused-imports -Wno-error=incomplete-record-selectors -Wno-ambiguous-fields -fno-write-ide-info -threaded -rtsopts "-with-rtsopts=-N -A64m" -O0 + ghc-options: -fwrite-ide-info -threaded -Weverything -Werror -fno-defer-typed-holes -Wno-error=deprecations -Wno-error=unused-packages -Wno-error=implicit-lift -Wno-error=missing-poly-kind-signatures -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-monomorphism-restriction -Wno-missing-safe-haskell-mode -Wno-partial-fields -Wno-prepositive-qualified-module -Wno-missing-export-lists -Wno-orphans -Wno-missing-deriving-strategies -Wno-missing-role-annotations -Wno-unused-matches -Wno-missing-kind-signatures -Wno-type-defaults -Wno-unused-top-binds -Wno-unused-imports -Wno-error=incomplete-record-selectors -Wno-ambiguous-fields -threaded -rtsopts "-with-rtsopts=-N -A64m" -O0 build-tool-depends: proto-lens-protoc:proto-lens-protoc build-depends: diff --git a/package.yaml b/package.yaml index 72583a01c..cbd3dab1c 100644 --- a/package.yaml +++ b/package.yaml @@ -199,7 +199,6 @@ tests: main: Main.hs source-dirs: test/integration ghc-options: - - -fno-write-ide-info - -threaded - -rtsopts - '"-with-rtsopts=-N -A64m"' @@ -300,7 +299,6 @@ tests: main: Main.hs source-dirs: test/unit ghc-options: - - -fno-write-ide-info - -threaded - -rtsopts - '"-with-rtsopts=-N -A64m"' diff --git a/scripts/ci/ci.sh b/scripts/ci/ci.sh index 6c0288427..aa690f88a 100755 --- a/scripts/ci/ci.sh +++ b/scripts/ci/ci.sh @@ -248,16 +248,33 @@ record_result() { # — attest now, or stage for a caller that has push # Publish the results staged by a run that could not push (the local CI container # has no credentials — origin is ssh and the image has no keys). cmd_publish() { # [file] - local f c fp caps plat + local f c fp caps plat spec + local -a refspecs=() f=${1:-${CI_ATTEST_OUT:-.ci/attest.tsv}} [ -s "$f" ] || { note "nothing to publish ($f)"; return 0; } while IFS=$'\t' read -r c fp caps plat; do - [ -n "$c" ] && publish_attestation "$c" "$fp" "$caps" "$plat" + if [ -n "$c" ]; then + if spec=$(attestation_refspec "$c" "$fp" "$caps" "$plat"); then + if remote_refs | grep -Fx "${spec#*:}" >/dev/null; then + note "already attested ${spec#*:}" + else + refspecs+=("$spec") + fi + else + note "could not prepare attestation for $c — result is still valid, just not cached" + fi + fi done < "$f" rm -f "$f" + [ "${#refspecs[@]}" -gt 0 ] || return 0 + if git push -q "$REMOTE" "${refspecs[@]}" 2>/dev/null; then + for spec in "${refspecs[@]}"; do note "attested ${spec#*:}"; done + else + note "could not publish attestations (no push access to $REMOTE?) — results are still valid, just not cached" + fi } -publish_attestation() { # [fingerprint] [caps] [platform] +attestation_refspec() { # [fingerprint] [caps] [platform] local fp caps plat ref tree commit runner_id fp=${2:-$(fingerprint "$1")} caps=${3:-$(detect_caps)} @@ -276,8 +293,15 @@ caps=$caps platform=$plat commit=$(git rev-parse HEAD) runner=$runner_id") - if git push -q "$REMOTE" "${commit}:${ref}" 2>/dev/null; then - note "attested $1 → $ref" + printf '%s:%s' "$commit" "$ref" +} + +publish_attestation() { # [fingerprint] [caps] [platform] + local spec + if ! spec=$(attestation_refspec "$@"); then + note "could not prepare attestation for $1 — result is still valid, just not cached" + elif git push -q "$REMOTE" "$spec" 2>/dev/null; then + note "attested $1 → ${spec#*:}" else note "could not publish attestation for $1 (no push access to $REMOTE?) — result is still valid, just not cached" fi @@ -324,12 +348,13 @@ run_body() { # # those libraries are absent from its package database. Build them here so # this check remains independently reproducible when build is skipped. doctests) - cabal build monoscope:lib:monoscope monoscope-shared:lib:monoscope-shared monoscope-cli:lib:monoscope-cli -j $CABAL_FLAGS "$CABAL_OPTS" + [ -n "${CI_UNIT_BIN:-}" ] || cabal build monoscope:lib:monoscope monoscope-shared:lib:monoscope-shared monoscope-cli:lib:monoscope-cli -j $CABAL_FLAGS "$CABAL_OPTS" cabal test doctests -j $CABAL_FLAGS "$CABAL_OPTS" --test-show-details=direct ;; - unit-tests) cabal test unit-tests -j $CABAL_FLAGS "$CABAL_OPTS" --test-show-details=direct ;; - cli-tests) cabal test monoscope-cli:cli-tests -j $CABAL_FLAGS "$CABAL_OPTS" --test-show-details=direct ;; + unit-tests) if [ -n "${CI_UNIT_BIN:-}" ]; then "$CI_UNIT_BIN"; else cabal test unit-tests -j $CABAL_FLAGS "$CABAL_OPTS" --test-show-details=direct; fi ;; + cli-tests) if [ -n "${CI_CLI_BIN:-}" ]; then "$CI_CLI_BIN"; else cabal test monoscope-cli:cli-tests -j $CABAL_FLAGS "$CABAL_OPTS" --test-show-details=direct; fi ;; weeder) + cabal build integration-tests unit-tests -j $CABAL_FLAGS "$CABAL_OPTS" command -v weeder >/dev/null 2>&1 || cabal install weeder --install-method=copy --installdir=/usr/local/bin --overwrite-policy=always weeder --config weeder.toml --hie-directory dist-newstyle ;; @@ -338,13 +363,14 @@ run_body() { # # Drives the real server in a real browser. scripts/e2e.sh starts that server itself on # 8081 against a throwaway database, so this only has to supply the binary and chromium. e2e) - cabal build monoscope-server -j $CABAL_FLAGS "$CABAL_OPTS" + [ -n "${E2E_SERVER_BIN:-}" ] || cabal build monoscope-server -j $CABAL_FLAGS "$CABAL_OPTS" (cd e2e && npm ci --prefer-offline --no-audit && \ { [ -e /usr/local/share/monoscope-playwright-deps ] || npx playwright install-deps chromium; } && \ npx playwright install chromium) scripts/e2e.sh ;; integration-tests) run_integration ;; + selftest-fail) false; true ;; *) die "no body for check '$1'" ;; esac } @@ -355,6 +381,7 @@ run_body() { # # under the server's max_connections. run_integration() { local shards bin i green + local -a shard_pids=() shards=${CI_SHARDS:-4} # Without a reachable TimeFusion the URL must go, or every example dies dialling # a dead host instead of taking TestUtils' Postgres-as-TimeFusion fallback. Only @@ -363,16 +390,17 @@ run_integration() { export USE_EXTERNAL_DB=true LOG_LEVEL=${LOG_LEVEL:-warn} (cd web-components && npm ci --prefer-offline --no-audit) make build-chart-cli - cabal build integration-tests -j $CABAL_FLAGS "$CABAL_OPTS" + [ -n "${CI_INTEGRATION_BIN:-}" ] || cabal build integration-tests -j $CABAL_FLAGS "$CABAL_OPTS" # Same flags as the build above: list-bin resolves against a plan, and a # different one points at a path the build never wrote. - bin=$(cabal list-bin integration-tests $CABAL_FLAGS) + bin=${CI_INTEGRATION_BIN:-$(cabal list-bin integration-tests $CABAL_FLAGS)} rm -f build-shard-*.log for i in $(seq 0 $((shards - 1))); do - ( start=$(date +%s); SHARD_INDEX=$i SHARD_TOTAL=$shards "$bin" --color > "build-shard-$i.log" 2>&1 + ( start=$(date +%s); SHARD_INDEX=$i SHARD_TOTAL=$shards "$bin" +RTS -N1 -RTS --color > "build-shard-$i.log" 2>&1 echo "[shard-time] $(( $(date +%s) - start ))s" >> "build-shard-$i.log" ) & + shard_pids+=("$!") done - wait + for i in "${shard_pids[@]}"; do wait "$i" || true; done echo "=== per-shard (wall-clock | result) — wide spreads ⇒ rebalance ===" for i in $(seq 0 $((shards - 1))); do printf "shard %s: %-6s | %s\n" "$i" "$(sed -n 's/.*\[shard-time\] //p' "build-shard-$i.log" | tail -1)" \ @@ -430,13 +458,45 @@ cmd_gate() { } cmd_run() { - local c req ref rc=0 unrunnable='' degraded='' + local c req ref bin i rc=0 unrunnable='' degraded='' e2e_pid='' e2e_log='' e2e_result='' + local integration_pid='' integration_log='' integration_result='' + local checks + checks=$(selected_checks "$@") CAPS=$(detect_caps) note "capabilities: ${CAPS:-none}" # shellcheck disable=SC2046 - [ -n "${CI_FINGERPRINTS:-}" ] || pin_fingerprints $(selected_checks "$@") + [ -n "${CI_FINGERPRINTS:-}" ] || pin_fingerprints $checks [ "${CI_FORCE:-}" = "true" ] || remote_refs >/dev/null - for c in $(selected_checks "$@"); do + for c in $checks; do + if [ "$c" = integration-tests ] && [ -n "$integration_pid" ]; then + if wait "$integration_pid"; then + [ ! -s "$integration_result" ] || cat "$integration_result" >> "$CI_ATTEST_OUT" + else + rc=1 + fi + cat "$integration_log" + rm -f "$integration_log" "$integration_result" + [ "$rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = "true" ] || [ -n "$e2e_pid" ] || return 1 + continue + fi + if [ "$c" = e2e ] && [ -n "$e2e_pid" ]; then + if wait "$e2e_pid"; then + [ ! -s "$e2e_result" ] || cat "$e2e_result" >> "$CI_ATTEST_OUT" + else + rc=1 + fi + cat "$e2e_log" + rm -f "$e2e_log" "$e2e_result" + [ "$rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = "true" ] || return 1 + continue + fi + if [ "$c" = integration-tests ] && [ "${CI_LOCAL_ASYNC_SERVICES:-}" = true ]; then + for i in $(seq 1 60); do + CAPS=$(detect_caps) + caps_satisfy "$CAPS" "$(check_requires integration-tests)" && break + sleep 2 + done + fi req=$(check_requires "$c") # A missing capability is never a silent pass — this run can say nothing about # that check. CI_ALLOW_DEGRADED still runs it for the feedback (e.g. the suite @@ -454,10 +514,29 @@ cmd_run() { note "$c already proven ($ref) — skipping (CI_FORCE=true to override)" continue fi + # If build was skipped, still overlap the two suites when both need to run. + if [ "$c" = integration-tests ] && [ -n "${CI_ATTEST_OUT:-}" ] \ + && [ -z "$e2e_pid" ] \ + && case " $(echo "$checks" | tr '\n' ' ') " in *" integration-tests "*" e2e "*) true ;; *) false ;; esac; then + e2e_log=$(mktemp -t ci-e2e-log.XXXXXX) + e2e_result=$(mktemp -t ci-e2e-result.XXXXXX) + CI_ATTEST_OUT="$e2e_result" "$0" run e2e > "$e2e_log" 2>&1 & + e2e_pid=$! + fi note "running $c" local started=$SECONDS - if run_body "$c"; then + if CAPS="$CAPS" "$0" body "$c"; then note "passed $c in $((SECONDS - started))s" + if [ "$c" = build ] && [ -n "${CI_ATTEST_OUT:-}" ]; then + CI_UNIT_BIN=$(cabal list-bin unit-tests $CABAL_FLAGS) + CI_CLI_BIN=$(cabal list-bin monoscope-cli:cli-tests $CABAL_FLAGS) + CI_INTEGRATION_BIN=$(cabal list-bin integration-tests $CABAL_FLAGS) + E2E_SERVER_BIN=$(cabal list-bin monoscope-server $CABAL_FLAGS) + export CI_UNIT_BIN CI_CLI_BIN CI_INTEGRATION_BIN E2E_SERVER_BIN + for bin in "$CI_UNIT_BIN" "$CI_CLI_BIN" "$CI_INTEGRATION_BIN" "$E2E_SERVER_BIN"; do + [ -x "$bin" ] || die "build did not produce $bin" + done + fi if ! inputs_unchanged "$c"; then note "PASSED $c, but inputs changed during the run — NOT attested; rerun required" rc=1 @@ -467,10 +546,25 @@ cmd_run() { # the network and the object store; neither is part of what the check proved. case " $degraded " in *" $c "*) ;; *) [ "${CI_NO_ATTEST:-}" = "true" ] \ || record_result "$c" || note "could not record $c — it still passed, just isn't cached" ;; esac + # Build has populated Cabal's cache; start integration while the remaining + # checks run. Browser tests start after doctests to avoid peak compiler work. + if [ "$c" = build ] && [ -n "${CI_ATTEST_OUT:-}" ] \ + && case " $(echo "$checks" | tr '\n' ' ') " in *" build "*" integration-tests "*" e2e "*) true ;; *) false ;; esac; then + integration_log=$(mktemp -t ci-integration-log.XXXXXX) + integration_result=$(mktemp -t ci-integration-result.XXXXXX) + CI_ATTEST_OUT="$integration_result" "$0" run integration-tests > "$integration_log" 2>&1 & + integration_pid=$! + fi + if [ "$c" = doctests ] && [ -n "$integration_pid" ] && [ -z "$e2e_pid" ]; then + e2e_log=$(mktemp -t ci-e2e-log.XXXXXX) + e2e_result=$(mktemp -t ci-e2e-result.XXXXXX) + CI_ATTEST_OUT="$e2e_result" "$0" run e2e > "$e2e_log" 2>&1 & + e2e_pid=$! + fi else rc=1 note "FAILED $c" - [ "${CI_KEEP_GOING:-}" = "true" ] || return 1 + [ "${CI_KEEP_GOING:-}" = "true" ] || [ -n "$e2e_pid" ] || return 1 fi done [ -z "$unrunnable" ] || note "not run here (CI will still have to):$unrunnable" @@ -506,11 +600,12 @@ cmd_gc() { # ---------------------------------------------------------------- local docker COMPOSE_FILE=ci/compose.yml -compose() { docker compose -f "$COMPOSE_FILE" --project-name monoscope-ci "$@"; } +COMPOSE_PROJECT="monoscope-ci-$(printf '%s' "$ROOT" | sha256 | cut -c1-12)" +compose() { docker compose -f "$COMPOSE_FILE" --project-name "$COMPOSE_PROJECT" "$@"; } cmd_local() { - # Every local run shares the Cabal/Node volumes and ephemeral service databases. - # Lock before even resetting services; parallel runs corrupt GHC interface files. + # Serialize local runs to avoid oversubscribing the host. Each worktree has its + # own volumes so a checkout on another branch cannot replace this one's HIE files. if [ "${CI_LOCAL_LOCKED:-}" != true ]; then python3 -c ' import fcntl, os, sys @@ -525,30 +620,40 @@ with open(sys.argv[1], "w") as lock: return fi # The dependency image has no HLint; the host tool is also how CI runs it. - local c host_rc=0 + local c host_rc=0 host_hlint=false host_pid='' host_log='' host_fp='' local -a container_checks=() for c in $(selected_checks "$@"); do if [ "$c" = hlint ] && command -v hlint >/dev/null 2>&1; then - "$0" run hlint || host_rc=$? + host_hlint=true else container_checks+=("$c") fi done - [ "$host_rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = true ] || return "$host_rc" - [ "${#container_checks[@]}" -gt 0 ] || return "$host_rc" + if [ "${#container_checks[@]}" -eq 0 ]; then + [ "$host_hlint" = false ] || "$0" run hlint + return + fi + if [ "${CI_FORCE:-}" != "true" ]; then + remote_refs >/dev/null + local -a pending=() + for c in "${container_checks[@]}"; do + find_attestation "$c" >/dev/null || pending+=("$c") + done + if [ "${#pending[@]}" -eq 0 ]; then + [ "$host_hlint" = false ] || "$0" run hlint + note "all selected checks are already attested — skipping local containers" + return 0 + fi + container_checks=("${pending[@]}") + fi set -- "${container_checks[@]}" command -v docker >/dev/null 2>&1 || die "docker is required for \`ci.sh local\`" docker compose version >/dev/null 2>&1 || die "docker compose v2 is required" - if [ "${CI_FORCE:-}" != "true" ]; then - remote_refs >/dev/null - local cached=true c - for c in $(selected_checks "$@"); do - find_attestation "$c" >/dev/null || { cached=false; break; } - done - if [ "$cached" = true ]; then - note "all selected checks are already attested — skipping local containers" - return "$host_rc" - fi + if [ "$host_hlint" = true ]; then + host_log=$(mktemp -t ci-hlint-log.XXXXXX) + host_fp=$(mktemp -t ci-hlint-fp.XXXXXX) + CI_FINGERPRINTS_PATH="$host_fp" "$0" run hlint > "$host_log" 2>&1 & + host_pid=$! fi # Postgres and MinIO are deliberately cache-free integration fixtures (Postgres # is tmpfs-backed). Keeping their *containers* after a run made a second local @@ -566,27 +671,24 @@ with open(sys.argv[1], "w") as lock: note "resetting ephemeral integration services" compose rm -sf postgres minio timefusion >/dev/null 2>&1 || true fi - if [ "$needs_postgres" = true ]; then - note "starting Postgres…" - compose up -d --wait postgres - fi if [ "$needs_integration" = true ]; then - note "starting integration services…" - compose up -d --wait minio # Pick up a native arm64 TimeFusion image when available. if [ -z "${MONOSCOPE_CI_TF_IMAGE:-}" ] && [ "$(uname -m)" = arm64 ] \ && docker image inspect "$TF_LOCAL_IMAGE" >/dev/null 2>&1; then export MONOSCOPE_CI_TF_IMAGE="$TF_LOCAL_IMAGE" MONOSCOPE_CI_TF_PLATFORM=linux/arm64 note "using locally built $TF_LOCAL_IMAGE (native arm64)" fi - # Probe from the runner: TimeFusion's distroless image has no healthcheck shell. - if compose up -d timefusion >/dev/null 2>&1 \ - && compose run --rm -T --no-deps runner bash -c \ - 'for _ in $(seq 1 60); do (exec 3<>/dev/tcp/timefusion/5432) 2>/dev/null && exit 0; sleep 2; done; exit 1' >/dev/null 2>&1; then - note "TimeFusion is accepting pgwire connections" - else - note "TimeFusion did not start (${MONOSCOPE_CI_TF_PLATFORM:-linux/amd64} on $(uname -m)) — integration-tests will be left to CI; see docs/local-ci.md" - fi + note "starting integration services…" + compose up -d postgres timefusion || note "TimeFusion did not start — integration-tests will report missing capabilities" + elif [ "$needs_postgres" = true ]; then + note "starting Postgres…" + compose up -d --wait postgres + fi + if [ -n "$host_pid" ]; then + wait "$host_pid" || host_rc=$? + cat "$host_log" + rm -f "$host_log" "$host_fp" + [ "$host_rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = true ] || return "$host_rc" fi rm -f .ci/attest.tsv local rc=0 @@ -608,7 +710,7 @@ with open(sys.argv[1], "w") as lock: compose run --rm ${git_mount[@]+"${git_mount[@]}"} ${refs_mount[@]+"${refs_mount[@]}"} \ -e CI_ROOT=/build \ -e "CI_ALLOW_DEGRADED=${CI_ALLOW_DEGRADED:-}" -e "CI_FORCE=${CI_FORCE:-}" -e "CI_KEEP_GOING=${CI_KEEP_GOING:-}" \ - -e "CI_NO_ATTEST=${CI_NO_ATTEST:-}" \ + -e "CI_NO_ATTEST=${CI_NO_ATTEST:-}" -e "CI_LOCAL_ASYNC_SERVICES=$needs_integration" \ runner sh -c 'cp scripts/ci/ci.sh /tmp/ci-run.sh && exec bash /tmp/ci-run.sh run "$@"' ci-run "$@" || rc=$? # Publish whatever passed even if a later check failed — a green check is green. cmd_publish .ci/attest.tsv @@ -904,6 +1006,10 @@ assert() { # cmd_selftest() { SELFTEST_RC=0 + local body_rc=0 + "$0" body selftest-fail >/dev/null 2>&1 || body_rc=$? + assert "body stops at first failure" 1 "$body_rc" + assert "built unit binary runs without Cabal" ok "$(CI_UNIT_BIN=$(command -v true) "$0" body unit-tests && echo ok)" assert "caps superset" ok "$(caps_satisfy 'ghc pg minio tf-real' 'ghc pg' && echo ok)" assert "caps exact" ok "$(caps_satisfy 'ghc' 'ghc' && echo ok)" @@ -930,6 +1036,13 @@ cmd_selftest() { "$(echo $seen | tr ' ' '\n' | sort -u | wc -l | tr -d ' ')" assert "fingerprint stable" "$(fingerprint hlint)" "$(WORKTREE_TREE=''; fingerprint hlint)" + local cached_ref + cached_ref=$(mktemp -t ci-selftest-refs.XXXXXX) + attest_ref frontend "$(fingerprint frontend)" node > "$cached_ref" + assert "cached local skips containers" skipped \ + "$(CI_LOCAL_LOCKED=true CI_REMOTE_REFS_FILE="$cached_ref" "$0" local frontend 2>&1 | sed -n 's/.*all selected checks are already attested.*/skipped/p')" + rm -f "$cached_ref" + # A change under a check's inputs must move its fingerprint; one outside must not. # Probe with NEW files only — never edit-and-restore a tracked file, which would # discard whatever the developer has uncommitted in it. @@ -1022,6 +1135,7 @@ case "$cmd" in caps) detect_caps; echo ;; gate) cmd_gate "$@" ;; run) cmd_run "$@" ;; + body) run_body "$@" ;; attest) cmd_attest "$@" ;; publish) cmd_publish "$@" ;; local) cmd_local "$@" ;; diff --git a/scripts/e2e.sh b/scripts/e2e.sh index 561e8c50b..45152a4ba 100755 --- a/scripts/e2e.sh +++ b/scripts/e2e.sh @@ -37,7 +37,7 @@ psql -h "$PGHOST" -p "$PGPORT" -U postgres -tAc \ "SELECT 1 FROM pg_database WHERE datname='$DB'" | grep -q 1 || psql -h "$PGHOST" -p "$PGPORT" -U postgres -q -c "CREATE DATABASE \"$DB\"" -BIN=$(cabal list-bin monoscope-server 2>/dev/null || true) +BIN=${E2E_SERVER_BIN:-$(cabal list-bin monoscope-server 2>/dev/null || true)} [ -x "$BIN" ] || { echo "monoscope-server not built. Run: cabal build monoscope-server" >&2; exit 1; } # The server's working directory. Built from .env.example so no real credential is ever diff --git a/test/integration/Main.hs b/test/integration/Main.hs index 915c8d67c..014e167aa 100644 --- a/test/integration/Main.hs +++ b/test/integration/Main.hs @@ -1,5 +1,6 @@ module Main (main) where +import Data.List (lookup) import Relude import Spec qualified import Test.Hspec.Runner (Config (..), defaultConfig, hspecWith) @@ -31,4 +32,14 @@ main = do -- Deterministic hash of the outermost group; keeps a spec's examples (and any -- shared state) together in one shard. inShard :: Int -> Int -> ([String], String) -> Bool - inShard i n (groups, _) = foldl' (\a c -> a * 31 + ord c) 7 (concat (take 1 groups)) `mod` n == i + inShard i n (groups, _) = (if n == 12 then fromMaybe hashed (lookup groupName balanced) else hashed) == i + where + groupName = concat (take 1 groups) + hashed = foldl' (\a c -> a * 31 + ord c) 7 groupName `mod` n + balanced = + [ ("Pages.LogExplorer.Log", 4) + , ("Pages.Bots.Workflows", 5) + , ("BackgroundJobs.ReportUsage", 2) + , ("EndpointDiscovery", 7) + , ("Pages.Endpoints.ApiCatalog", 0) + ] From 1d1930e7695718a4fe45dcf3e139b70fc215af3b Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 19:42:39 +0200 Subject: [PATCH 004/297] Log: one fmtPct1 doctest block --- src/Pages/LogExplorer/Log.hs | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/src/Pages/LogExplorer/Log.hs b/src/Pages/LogExplorer/Log.hs index 04ef82592..a0175fbd9 100644 --- a/src/Pages/LogExplorer/Log.hs +++ b/src/Pages/LogExplorer/Log.hs @@ -1137,21 +1137,11 @@ logLatencyWidget pid = } --- | One-decimal percent formatter. +-- | One-decimal percent formatter; a rate that would round to zero but is not zero says so. -- -- >>> import "monoscope" Pages.LogExplorer.Log qualified as LL --- >>> LL.fmtPct1 0 --- "0.0%" --- >>> LL.fmtPct1 5.24 --- "5.2%" --- >>> LL.fmtPct1 100 --- "100.0%" --- >>> LL.fmtPct1 (-1.25) --- "-1.2%" --- | One-decimal percentage; a rate that would round to zero but is not zero says so. --- --- >>> map fmtPct1 [0, 0.03, 0.05, 4.14, 100] --- ["0.0%","<0.1%","0.1%","4.1%","100.0%"] +-- >>> map LL.fmtPct1 [0, 0.03, 0.05, 5.24, 100, -1.25] +-- ["0.0%","<0.1%","0.1%","5.2%","100.0%","-1.2%"] fmtPct1 :: Double -> Text fmtPct1 x | x > 0 && x < 0.05 = "<0.1%" From 6ffc45affa15e9aeb2c86624f89691d6a14abdfe Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 19:43:01 +0200 Subject: [PATCH 005/297] Clean up CI background jobs and shard failures (#613) --- .github/workflows/pullrequest.yml | 1 + Makefile | 2 +- docs/local-ci.md | 5 ++- scripts/ci/ci.sh | 74 ++++++++++++++++++------------- test/integration/Main.hs | 2 + 5 files changed, 50 insertions(+), 34 deletions(-) diff --git a/.github/workflows/pullrequest.yml b/.github/workflows/pullrequest.yml index 2cc8f4165..740d9a89a 100644 --- a/.github/workflows/pullrequest.yml +++ b/.github/workflows/pullrequest.yml @@ -285,6 +285,7 @@ jobs: if: needs.gate.outputs.skip_hlint != 'true' id: refactor run: | + # Refactor may report hints as a nonzero exit; Run hlint below gates the result. make fix-lint || true # Recorded before the push is attempted, because the push is what forks # and dependabot can't do — the tree still changed either way, and that diff --git a/Makefile b/Makefile index 7eb8a383f..517894d59 100644 --- a/Makefile +++ b/Makefile @@ -230,7 +230,7 @@ lint: hlint src fix-lint: - find ./src -name '*.hs' | xargs -L1 hlint --refactor --refactor-options="--inplace" + hlint src/ --refactor --refactor-options="--inplace" # Add a FontAwesome icon to the sprite sheet. # Usage: make fa-add ICON=arrow-right STYLE=solid diff --git a/docs/local-ci.md b/docs/local-ci.md index 7d31fb19a..288fd674e 100644 --- a/docs/local-ci.md +++ b/docs/local-ci.md @@ -269,7 +269,7 @@ QEMU, so it runs at a useful fraction of native rather than 10× slower. Budget | `CI_KEEP_GOING=true` | don't stop the sweep at the first failure | | `CI_ALLOW_DEGRADED=true` | run checks whose capabilities are missing, unattested | | `CI_NO_ATTEST=true` | run, publish nothing | -| `CI_SHARDS=n` | integration-test shard count (local defaults to 6; GitHub uses 4) | +| `CI_SHARDS=n` | integration-test shard count (local defaults to 12; GitHub uses 4) | | `CI_ATTEST_DISABLED=true` | ignore all attestations — set as a repo variable to force full CI runs | | `MONOSCOPE_CI_TF_IMAGE` / `MONOSCOPE_CI_TF_PLATFORM` | point at a locally built TimeFusion image (auto-detected after `make tf-image`) | | `TF_TARGET_CPU` | CPU baseline used by `make tf-image`; defaults to `neoverse-n1` on arm64 and `x86-64-v3` on amd64. Override only for a known compatible target. | @@ -277,6 +277,9 @@ QEMU, so it runs at a useful fraction of native rather than 10× slower. Budget | `MONOSCOPE_BUILDER` | buildx builder to build the image with; falls back to the default builder if absent | | `SHIP_ANY_BRANCH=1` | let `make ship` deploy something other than master | +`CI_UNIT_BIN`, `CI_CLI_BIN`, `CI_INTEGRATION_BIN`, `E2E_SERVER_BIN`, and +`CI_LOCAL_ASYNC_SERVICES` are internal handoffs between local checks; leave them unset. + ## When you need to invalidate everything The fingerprint covers repository content, not the toolchain. If the deps image diff --git a/scripts/ci/ci.sh b/scripts/ci/ci.sh index aa690f88a..34c4eba9d 100755 --- a/scripts/ci/ci.sh +++ b/scripts/ci/ci.sh @@ -297,14 +297,10 @@ runner=$runner_id") } publish_attestation() { # [fingerprint] [caps] [platform] - local spec - if ! spec=$(attestation_refspec "$@"); then - note "could not prepare attestation for $1 — result is still valid, just not cached" - elif git push -q "$REMOTE" "$spec" 2>/dev/null; then - note "attested $1 → ${spec#*:}" - else - note "could not publish attestation for $1 (no push access to $REMOTE?) — result is still valid, just not cached" - fi + local f + f=$(mktemp -t ci-attest.XXXXXX) + printf '%s\t%s\t%s\t%s\n' "$1" "${2:-$(fingerprint "$1")}" "${3:-$(detect_caps)}" "${4:-$(platform_tag)}" > "$f" + cmd_publish "$f" } # ---------------------------------------------------------------- check bodies @@ -370,7 +366,6 @@ run_body() { # scripts/e2e.sh ;; integration-tests) run_integration ;; - selftest-fail) false; true ;; *) die "no body for check '$1'" ;; esac } @@ -380,7 +375,7 @@ run_body() { # # resource-pool lifecycle (see test/integration/Main.hs). Keep shards * ~9 conns # under the server's max_connections. run_integration() { - local shards bin i green + local shards bin i green shard_rc=0 local -a shard_pids=() shards=${CI_SHARDS:-4} # Without a reachable TimeFusion the URL must go, or every example dies dialling @@ -400,7 +395,7 @@ run_integration() { echo "[shard-time] $(( $(date +%s) - start ))s" >> "build-shard-$i.log" ) & shard_pids+=("$!") done - for i in "${shard_pids[@]}"; do wait "$i" || true; done + for i in "${shard_pids[@]}"; do wait "$i" || shard_rc=1; done echo "=== per-shard (wall-clock | result) — wide spreads ⇒ rebalance ===" for i in $(seq 0 $((shards - 1))); do printf "shard %s: %-6s | %s\n" "$i" "$(sed -n 's/.*\[shard-time\] //p' "build-shard-$i.log" | tail -1)" \ @@ -409,7 +404,7 @@ run_integration() { green=$(grep -lE "examples?, 0 failures" build-shard-*.log | wc -l | tr -d ' ') # green==N already means every shard printed a clean summary; don't also grep for # "error:" — the app logs error: lines at LOG_LEVEL=warn on passing runs. - if [ "$green" -ne "$shards" ] || grep -qE "[1-9][0-9]* failures?|Interrupted" build-shard-*.log; then + if [ "$shard_rc" -ne 0 ] || [ "$green" -ne "$shards" ] || grep -qE "[1-9][0-9]* failures?|Interrupted" build-shard-*.log; then echo "SHARDED RUN FAILED ($green/$shards green):" for f in build-shard-*.log; do grep -qE "examples?, 0 failures" "$f" && continue @@ -457,6 +452,17 @@ cmd_gate() { echo "skip_all=$skip_all" >> "$out" } +stop_early() { [ "$1" -ne 0 ] && [ "${CI_KEEP_GOING:-}" != true ]; } + +collect_background() { # + local status=0 + wait "$1" || status=1 + [ "$status" -ne 0 ] || [ ! -s "$3" ] || cat "$3" >> "$CI_ATTEST_OUT" + cat "$2" + rm -f "$2" "$3" + return "$status" +} + cmd_run() { local c req ref bin i rc=0 unrunnable='' degraded='' e2e_pid='' e2e_log='' e2e_result='' local integration_pid='' integration_log='' integration_result='' @@ -469,25 +475,15 @@ cmd_run() { [ "${CI_FORCE:-}" = "true" ] || remote_refs >/dev/null for c in $checks; do if [ "$c" = integration-tests ] && [ -n "$integration_pid" ]; then - if wait "$integration_pid"; then - [ ! -s "$integration_result" ] || cat "$integration_result" >> "$CI_ATTEST_OUT" - else - rc=1 - fi - cat "$integration_log" - rm -f "$integration_log" "$integration_result" - [ "$rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = "true" ] || [ -n "$e2e_pid" ] || return 1 + collect_background "$integration_pid" "$integration_log" "$integration_result" || rc=1 + integration_pid='' + stop_early "$rc" && break continue fi if [ "$c" = e2e ] && [ -n "$e2e_pid" ]; then - if wait "$e2e_pid"; then - [ ! -s "$e2e_result" ] || cat "$e2e_result" >> "$CI_ATTEST_OUT" - else - rc=1 - fi - cat "$e2e_log" - rm -f "$e2e_log" "$e2e_result" - [ "$rc" -eq 0 ] || [ "${CI_KEEP_GOING:-}" = "true" ] || return 1 + collect_background "$e2e_pid" "$e2e_log" "$e2e_result" || rc=1 + e2e_pid='' + stop_early "$rc" && break continue fi if [ "$c" = integration-tests ] && [ "${CI_LOCAL_ASYNC_SERVICES:-}" = true ]; then @@ -506,7 +502,9 @@ cmd_run() { if ! caps_satisfy "$CAPS" "$req"; then if [ "${CI_ALLOW_DEGRADED:-}" != "true" ]; then note "CANNOT RUN $c here — needs [$req], have [$CAPS]" - unrunnable="$unrunnable $c"; rc=1; continue + unrunnable="$unrunnable $c"; rc=1 + stop_early "$rc" && break + continue fi note "running $c DEGRADED — needs [$req], have [$CAPS]; result will not be attested" degraded="$degraded $c" @@ -540,6 +538,7 @@ cmd_run() { if ! inputs_unchanged "$c"; then note "PASSED $c, but inputs changed during the run — NOT attested; rerun required" rc=1 + stop_early "$rc" && break continue fi # A green check stays green even if we cannot record it. Publishing touches @@ -564,9 +563,11 @@ cmd_run() { else rc=1 note "FAILED $c" - [ "${CI_KEEP_GOING:-}" = "true" ] || [ -n "$e2e_pid" ] || return 1 fi + stop_early "$rc" && break done + [ -z "$integration_pid" ] || collect_background "$integration_pid" "$integration_log" "$integration_result" || rc=1 + [ -z "$e2e_pid" ] || collect_background "$e2e_pid" "$e2e_log" "$e2e_result" || rc=1 [ -z "$unrunnable" ] || note "not run here (CI will still have to):$unrunnable" [ -z "$degraded" ] || note "run degraded, NOT attested (CI will still run these):$degraded" return $rc @@ -1006,10 +1007,19 @@ assert() { # cmd_selftest() { SELFTEST_RC=0 - local body_rc=0 - "$0" body selftest-fail >/dev/null 2>&1 || body_rc=$? + local body_rc=0 bg_log bg_result + CI_UNIT_BIN=false "$0" body unit-tests >/dev/null 2>&1 || body_rc=$? assert "body stops at first failure" 1 "$body_rc" assert "built unit binary runs without Cabal" ok "$(CI_UNIT_BIN=$(command -v true) "$0" body unit-tests && echo ok)" + bg_log=$(mktemp -t ci-selftest-bg-log.XXXXXX) + bg_result=$(mktemp -t ci-selftest-bg-result.XXXXXX) + (exit 1) & + body_rc=0 + collect_background "$!" "$bg_log" "$bg_result" || body_rc=$? + assert "background failure propagates" 1 "$body_rc" + assert "background files cleaned" no "$([ -e "$bg_log" ] || [ -e "$bg_result" ] && echo yes || echo no)" + assert "failure stops sweep" yes "$(stop_early 1 && echo yes)" + assert "keep-going continues sweep" no "$(CI_KEEP_GOING=true stop_early 1 && echo yes || echo no)" assert "caps superset" ok "$(caps_satisfy 'ghc pg minio tf-real' 'ghc pg' && echo ok)" assert "caps exact" ok "$(caps_satisfy 'ghc' 'ghc' && echo ok)" diff --git a/test/integration/Main.hs b/test/integration/Main.hs index 014e167aa..608690b81 100644 --- a/test/integration/Main.hs +++ b/test/integration/Main.hs @@ -36,6 +36,8 @@ main = do where groupName = concat (take 1 groups) hashed = foldl' (\a c -> a * 31 + ord c) 7 groupName `mod` n + -- Measured overrides for the local CI default of 12 shards; other totals + -- use the stable hash so changing CI_SHARDS never drops a spec. balanced = [ ("Pages.LogExplorer.Log", 4) , ("Pages.Bots.Workflows", 5) From 5f4739f4c3a7c1f022b27191fa3c2f6991d2bb0e Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 20:02:49 +0200 Subject: [PATCH 006/297] Overlap serialized browser checks with doctests (#614) --- scripts/ci/ci.sh | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/scripts/ci/ci.sh b/scripts/ci/ci.sh index 34c4eba9d..34739937d 100755 --- a/scripts/ci/ci.sh +++ b/scripts/ci/ci.sh @@ -545,16 +545,13 @@ cmd_run() { # the network and the object store; neither is part of what the check proved. case " $degraded " in *" $c "*) ;; *) [ "${CI_NO_ATTEST:-}" = "true" ] \ || record_result "$c" || note "could not record $c — it still passed, just isn't cached" ;; esac - # Build has populated Cabal's cache; start integration while the remaining - # checks run. Browser tests start after doctests to avoid peak compiler work. + # Build has populated Cabal's cache; overlap the independent test suites. if [ "$c" = build ] && [ -n "${CI_ATTEST_OUT:-}" ] \ && case " $(echo "$checks" | tr '\n' ' ') " in *" build "*" integration-tests "*" e2e "*) true ;; *) false ;; esac; then integration_log=$(mktemp -t ci-integration-log.XXXXXX) integration_result=$(mktemp -t ci-integration-result.XXXXXX) CI_ATTEST_OUT="$integration_result" "$0" run integration-tests > "$integration_log" 2>&1 & integration_pid=$! - fi - if [ "$c" = doctests ] && [ -n "$integration_pid" ] && [ -z "$e2e_pid" ]; then e2e_log=$(mktemp -t ci-e2e-log.XXXXXX) e2e_result=$(mktemp -t ci-e2e-result.XXXXXX) CI_ATTEST_OUT="$e2e_result" "$0" run e2e > "$e2e_log" 2>&1 & From 873edb61082c0826c32f445b3e4c36b7233edd05 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 20:36:32 +0200 Subject: [PATCH 007/297] Avoid redundant browser fixture cache waits (#615) --- e2e/tests/container-identity.spec.ts | 9 ++++----- e2e/tests/host-map.spec.ts | 9 ++++----- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/e2e/tests/container-identity.spec.ts b/e2e/tests/container-identity.spec.ts index 51794c1d1..b26e16226 100644 --- a/e2e/tests/container-identity.spec.ts +++ b/e2e/tests/container-identity.spec.ts @@ -1,5 +1,5 @@ import { test, expect } from "@playwright/test"; -import { DEMO_PROJECT, sql, awaitInfraSnapshotExpiry } from "./helpers"; +import { DEMO_PROJECT, sql } from "./helpers"; const CONTAINER = "e2e-duplicate-container"; const CLUSTER = "8b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d"; @@ -11,14 +11,12 @@ const identities = [ ["identity-a", CLUSTER, "identity-node-a", "identity-pod-b"], ]; const cleanup = `DELETE FROM otel_metrics WHERE project_id='${DEMO_PROJECT}' AND resource->'k8s'->'container'->>'name'='${CONTAINER}';`; -test.beforeAll(async () => { - test.setTimeout(60_000); +test.beforeAll(() => { if (!process.env.E2E_BASE_URL) return; sql(cleanup + identities.map(([namespace, cluster, node, pod], index) => ` INSERT INTO otel_metrics (project_id,id,series_id,timestamp,metric_name,metric_type,value,resource___k8s___container___name,resource___k8s___pod___name,resource___k8s___namespace___name,resource) VALUES ('${DEMO_PROJECT}',gen_random_uuid(),'e2e-container-${index}','2024-12-31T23:59:00Z','container.cpu.usage','GAUGE',${index + 1},'${CONTAINER}','${pod}','${namespace}', '${JSON.stringify({ k8s: { container: { name: CONTAINER }, pod: { name: pod }, namespace: { name: namespace }, cluster: { uid: cluster }, node: { name: node } } })}');`).join("")); - await awaitInfraSnapshotExpiry(); }); test.afterAll(() => { if (process.env.E2E_BASE_URL) sql(cleanup); @@ -28,9 +26,10 @@ test.describe.configure({ mode: "serial" }); test("navbar refresh menu applies Off and interval options", async ({ page }) => { test.skip(!process.env.E2E_BASE_URL, "Requires a disposable fixture database"); + // Lowercase keeps this cache key separate from the later 5M inventory fixture. for (const width of [390, 1440]) { await page.setViewportSize({ width, height: 900 }); - await page.goto(`/p/${DEMO_PROJECT}/infrastructure/containers?since=5M`, { waitUntil: "domcontentloaded" }); + await page.goto(`/p/${DEMO_PROJECT}/infrastructure/containers?since=5m`, { waitUntil: "domcontentloaded" }); const transport = page.locator("[data-time-transport]"); const menuToggle = page.locator(width < 768 ? '[popovertarget="n-timepicker-popover"]' : '[data-live-data-trigger]'); await expect(transport).toHaveAttribute("data-interval", "15000"); diff --git a/e2e/tests/host-map.spec.ts b/e2e/tests/host-map.spec.ts index 2ee4ee8f4..6d310309c 100644 --- a/e2e/tests/host-map.spec.ts +++ b/e2e/tests/host-map.spec.ts @@ -1,17 +1,16 @@ import { test, expect } from "@playwright/test"; -import { DEMO_PROJECT, awaitInfraSnapshotExpiry, sql } from "./helpers"; +import { DEMO_PROJECT, sql } from "./helpers"; -const HOST_MAP_URL = `/p/${DEMO_PROJECT}/infrastructure/host-map?since=5M`; +// 300S is a five-minute window with its own raw cache key among fixture suites. +const HOST_MAP_URL = `/p/${DEMO_PROJECT}/infrastructure/host-map?since=300S`; const HOST = "e2e-host-without-usage"; const cleanup = `DELETE FROM otel_metrics WHERE project_id='${DEMO_PROJECT}' AND resource___host___name='${HOST}';`; -test.beforeAll(async () => { - test.setTimeout(60_000); +test.beforeAll(() => { sql(cleanup + ` INSERT INTO otel_metrics (project_id,id,series_id,timestamp,metric_name,metric_type,value,resource___host___name,resource) VALUES ('${DEMO_PROJECT}',gen_random_uuid(),'e2e-host',now(),'system.uptime','GAUGE',60,'${HOST}', '{"host":{"name":"${HOST}"},"os":{"type":"linux"}}');`); - await awaitInfraSnapshotExpiry(); }); test.afterAll(() => sql(cleanup)); From 84b1affc818f6ad96a1036b13b28bb2fc9e32120 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 20:55:01 +0200 Subject: [PATCH 008/297] Wait for visible refresh menu state in browser test (#616) --- e2e/tests/container-identity.spec.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/e2e/tests/container-identity.spec.ts b/e2e/tests/container-identity.spec.ts index b26e16226..b3a9ffbfe 100644 --- a/e2e/tests/container-identity.spec.ts +++ b/e2e/tests/container-identity.spec.ts @@ -34,7 +34,6 @@ test("navbar refresh menu applies Off and interval options", async ({ page }) => const menuToggle = page.locator(width < 768 ? '[popovertarget="n-timepicker-popover"]' : '[data-live-data-trigger]'); await expect(transport).toHaveAttribute("data-interval", "15000"); await expect(transport).toHaveAttribute("data-state", "live"); - await expect(page.locator(`${width < 768 ? '#n-timepicker-popover' : '#n-live-data-pop'} [data-refresh-option][data-value="15000"]`)).toHaveAttribute("aria-pressed", "true"); await menuToggle.click(); await expect(page.getByRole("button", { name: "15 seconds", exact: true })).toHaveAttribute("aria-pressed", "true"); await menuToggle.click(); From 27916661c0a405232546f8db1caf58c2e6284455 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Wed, 30 Sep 2026 21:08:16 +0200 Subject: [PATCH 009/297] Remove final browser fixture cache wait (#617) --- e2e/tests/container-identity.spec.ts | 3 +-- e2e/tests/helpers.ts | 7 ------- e2e/tests/infrastructure-time-window.spec.ts | 7 +++---- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/e2e/tests/container-identity.spec.ts b/e2e/tests/container-identity.spec.ts index b3a9ffbfe..68b472df6 100644 --- a/e2e/tests/container-identity.spec.ts +++ b/e2e/tests/container-identity.spec.ts @@ -26,10 +26,9 @@ test.describe.configure({ mode: "serial" }); test("navbar refresh menu applies Off and interval options", async ({ page }) => { test.skip(!process.env.E2E_BASE_URL, "Requires a disposable fixture database"); - // Lowercase keeps this cache key separate from the later 5M inventory fixture. for (const width of [390, 1440]) { await page.setViewportSize({ width, height: 900 }); - await page.goto(`/p/${DEMO_PROJECT}/infrastructure/containers?since=5m`, { waitUntil: "domcontentloaded" }); + await page.goto(`/p/${DEMO_PROJECT}/infrastructure/containers?since=5M`, { waitUntil: "domcontentloaded" }); const transport = page.locator("[data-time-transport]"); const menuToggle = page.locator(width < 768 ? '[popovertarget="n-timepicker-popover"]' : '[data-live-data-trigger]'); await expect(transport).toHaveAttribute("data-interval", "15000"); diff --git a/e2e/tests/helpers.ts b/e2e/tests/helpers.ts index 4fb4a3278..117e54ba1 100644 --- a/e2e/tests/helpers.ts +++ b/e2e/tests/helpers.ts @@ -21,13 +21,6 @@ export async function assertStripeCheckout( } } -// These fixtures only use the disposable database owned by scripts/e2e.sh. -// Infrastructure pages serve one container snapshot per (project, window) for at least 30s -// (containersInWindowCached / TimePicker.cacheTtl). Specs in the serialized project seed the -// shared demo project back to back, so rows seeded after a neighbour's page load stay -// invisible until that snapshot expires. Call after seeding, before the first page load. -export const awaitInfraSnapshotExpiry = () => new Promise(resolve => setTimeout(resolve, 31_000)); - export function sql(query: string) { execFileSync('psql', ['-h', process.env.E2E_PGHOST ?? process.env.DB_HOST ?? 'localhost', '-p', process.env.E2E_PGPORT ?? process.env.DB_PORT ?? '5432', '-U', 'postgres', diff --git a/e2e/tests/infrastructure-time-window.spec.ts b/e2e/tests/infrastructure-time-window.spec.ts index 6e1bd6bd2..f7a73b298 100644 --- a/e2e/tests/infrastructure-time-window.spec.ts +++ b/e2e/tests/infrastructure-time-window.spec.ts @@ -1,19 +1,18 @@ import { test, expect } from "@playwright/test"; -import { DEMO_PROJECT, sql, awaitInfraSnapshotExpiry } from "./helpers"; +import { DEMO_PROJECT, sql } from "./helpers"; const namespace = "e2e-infra-preset"; const cluster = "8b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d"; const provider = "e2e-infra-provider"; const cleanup = `DELETE FROM otel_metrics WHERE project_id='${DEMO_PROJECT}' AND resource->'k8s'->'namespace'->>'name'='${namespace}';`; test.beforeAll(async ({ request }) => { - test.setTimeout(90_000); // Fixture setup waits out the shared snapshot's 30s TTL. if (!process.env.E2E_BASE_URL) return; sql(cleanup + [30, 600].map(age => `INSERT INTO otel_metrics (project_id,id,series_id,timestamp,metric_name,metric_type,value,resource___k8s___container___name,resource___k8s___pod___name,resource___k8s___namespace___name,resource) VALUES ('${DEMO_PROJECT}',gen_random_uuid(),'infra-preset-${age}','${new Date(Date.now() - age * 1000).toISOString()}','container.cpu.usage','GAUGE',1,'infra-preset-${age}','infra-preset-pod-${age}','${namespace}', '${JSON.stringify({ k8s: { container: { name: `infra-preset-${age}` }, pod: { name: `infra-preset-pod-${age}` }, namespace: { name: namespace }, cluster: { uid: cluster }, node: { name: `infra-preset-host-${age}` } }, container: { image: { name: `registry.example/infra-preset-${age}` } }, cloud: { provider, region: "e2e-infra-region" } })}');`).join("")); - await awaitInfraSnapshotExpiry(); + // Match the page's raw cache key; earlier fixture pages omit from/to. await expect.poll(async () => { - const response = await request.get(`/p/${DEMO_PROJECT}/infrastructure/hosts?${new URLSearchParams({ provider, group: "region", since: "5M", deferred: "1" })}`); + const response = await request.get(`/p/${DEMO_PROJECT}/infrastructure/hosts?${new URLSearchParams({ provider, group: "region", from: "", to: "", since: "5m", deferred: "1" })}`); expect(response.status()).toBe(200); return (await response.text()).includes("infra-preset-host-30"); }, { timeout: 40_000 }).toBe(true); From 250cb9bfb14272cdc64f0c1d297c825180eae489 Mon Sep 17 00:00:00 2001 From: Anthony Alaribe Date: Thu, 1 Oct 2026 11:56:14 +0200 Subject: [PATCH 010/297] Fix dashboard bug reports; exploratory sweep fixes and tests (#618) * Fix dashboard list scroll, favorites star, widget counts, widget editor saves/toggles, AI palette time range - #itemsListPage no longer clips the table (h-full overflow-hidden) - unstarred favorites star is always visible as a grey outline - widget count covers tab and group widgets (Dashboards.allWidgets) - expand drawer saves to the widget's tab; PUT resolves the tab itself, updates group children, and 404s instead of toasting success for a widget it never found - widget editor keeps non-chart widget types, drops the explorer-only Hide timeline / Create monitor toggles, and links to its Monitors tab instead - command palette: first Cmd+K no longer closes itself, Ask AI is clickable, keeps the AI time range and sends the timezone - every time picker on a page reflects the URL range (AI results, widget drawer) * Widget monitors refuse SQL-only widgets; e2e for dashboard list, drawer rename and palette AI A raw-SQL widget posts an empty KQL query, which compiled to an all-traffic count. The Monitors tab now explains instead, and the handler rejects the empty query. * Modals close on Escape; dashboard tabs keep the picked range; overview Infra stats decode - modalWith_ heard Escape only on its hidden toggle, never from focused modal content; closeModal left body.overflow-hidden behind - tab and variable links sent the server-rendered since/from/to, so a range picked client-side reset on the next tab switch; they now take it from the live picker - getTimeRange read a nonexistent #custom_range_input and always returned empty - _overview Infra stats selected ::text || '%' into the numeric stat decoder; every one rendered an error overlay * e2e: a picked range survives a dashboard tab switch * Lazy-load Tagify so the explorer's monitor teams picker works The log explorer skips Tagify for load time, but its lazily swapped monitor form has a tag input; init threw 'Tagify is not a constructor' and teams could not be picked. Pages without it now carry the asset URLs and fetch it on first use. * Issue page: live log-pattern seen/count; keep multi-word summary values in one chip A log-pattern issue row is written at detection, so Last seen and the all-time count froze while the pattern kept firing (prod: '7 hrs ago, 8 all time' vs 2 min, 92). Summary text was split on spaces, orphaning '<*> min' outside its chip. * AI chat widgets get addressable ids; pickers share the URL range; Tagify init stays sync for live instances - model widgets carry no id, so chat tables targeted '#' and threw in htmx - time-picker vitest now states the new contract: a URL range relabels every picker - an input that already holds a Tagify instance is reconciled synchronously * Review: Plated traversals for the widget tree, exhaustive payload case, chat-id doctest - allWidgets/overWidgetTree via universeOf/transformOf; dashboardQueryText and countWidgetsWithMetric reuse allWidgets (the latter now sees group children) - patternM matches every IssuePayload constructor instead of a catch-all - the chat widget-id regression is a doctest on aiChatHistoryView_ * Widget editor links to Monitors only for KQL widgets * Widget save keeps the sent tab only when the widget is on it; repair stored overview Infra stats - a stale or missing tab slug falls back to the widget's own tab; a widget id that repeats across tabs still updates the tab the client named - 0209 gives the 3 dashboards holding a copy of the old overview template the numeric Infra stat SQL (they rendered 'Query execution failed') - DashboardTemplatesSpec reuses Dashboards.allWidgets * Auto-format code with fourmolu * Improve dashboard template creation previews * Document production impact review scope * Build CI test executables during signoff * Wait for both infrastructure fixture windows * Allow inventory shell to finish before browser assertion * Auto-format code with fourmolu * Repair saved overview stat units and simplify template selection --------- Co-authored-by: github-actions[bot] --- docs/plans/2026-09-30-pr-impact-review.md | 486 ++++++++++++++++++ .../assets/pr-impact-checkout-volume.svg | 44 ++ e2e/tests/dashboard-add-widget.spec.ts | 86 +++- e2e/tests/infrastructure-time-window.spec.ts | 15 +- e2e/tests/query-editor.spec.ts | 16 + e2e/tests/sidebar-navigation.spec.ts | 13 + scripts/ci/ci.sh | 2 +- src/Models/Projects/Dashboards.hs | 10 +- src/Pages/BodyWrapper.hs | 4 + src/Pages/CommandPalette.hs | 13 +- src/Pages/Components.hs | 4 +- src/Pages/Dashboards.hs | 256 +++++---- src/Pages/Issues.hs | 53 +- src/Pages/Telemetry.hs | 2 +- src/Pkg/Components/LogQueryBox.hs | 3 +- src/Utils.hs | 18 +- .../0209_overview_infra_stats_numeric.sql | 12 + .../0210_overview_infra_stat_units.sql | 24 + static/public/dashboards/_overview.yaml | 10 +- test/integration/MonitoringSpec.hs | 3 + .../integration/Pages/DashboardWidgetsSpec.hs | 33 ++ test/integration/Pages/DashboardsSpec.hs | 23 + test/integration/Pages/IssuesSpec.hs | 18 + .../integration/System/MigrationRunnerSpec.hs | 22 + test/unit/Models/DashboardTemplatesSpec.hs | 22 +- web-components/src/main.ts | 66 ++- web-components/test/time-picker-state.test.ts | 11 +- 27 files changed, 1098 insertions(+), 171 deletions(-) create mode 100644 docs/plans/2026-09-30-pr-impact-review.md create mode 100644 docs/plans/assets/pr-impact-checkout-volume.svg create mode 100644 static/migrations/0209_overview_infra_stats_numeric.sql create mode 100644 static/migrations/0210_overview_infra_stat_units.sql diff --git a/docs/plans/2026-09-30-pr-impact-review.md b/docs/plans/2026-09-30-pr-impact-review.md new file mode 100644 index 000000000..4c2b11c29 --- /dev/null +++ b/docs/plans/2026-09-30-pr-impact-review.md @@ -0,0 +1,486 @@ +# Production impact reviews — working scope (2026-09-30) + +Status: decision-ready draft pending launch order and shared-repository +ownership. This document scopes automatic GitHub pull request reviews, a daily +codebase scan, and an on-demand reliability check. + +## Outcome + +For a connected repository, Monoscope reviews each new pull request revision against +the project's production evidence. It calls out changes that may break observability, +reliability, or performance, and shows why the affected path matters. It does not +review syntax, style, or other code trivia. + +The reviewer separates three claims: + +1. **Observed:** what logs, metrics, traces, issues, or deployments actually show. +2. **Inferred:** the effect the proposed diff could have, with the mechanism and + uncertainty stated. +3. **Unknown:** what missing mapping, instrumentation, or history prevents a conclusion. + +Pre-merge telemetry establishes current behavior and impact. It cannot by itself +prove the proposed code will fail after merge. + +Polylane's impact review sets a useful bar: one comment updated per push, a +changed line tied to live production evidence, and an optional blocking check. +It passes when a concrete break cannot be proved. Monoscope should keep that +high bar for an **action needed** finding, while naming unknown coverage +explicitly because blind spots are part of this product's purpose. A PR aimed +at an existing issue should also say whether the changed path plausibly fixes +that issue, with the same observed/inferred distinction. + +## Customer workflow + +1. Connect the existing GitHub App to a Monoscope project. Working assumption: + repositories granted to that installation start receiving reviews without + another Monoscope form. The member can disable reviews for a repository. +2. Monoscope proposes repository-to-service mappings from available source, + deployment, and telemetry metadata. It shows ambiguous mappings as missing + coverage until a user resolves them; they do not silently prevent a review. +3. PR events automatically queue a review for the latest head commit. A later commit supersedes + an unfinished review. The GitHub output identifies the reviewed commit. +4. One PR summary is updated in place. The review is advisory at launch; whether + customers can make a check required is an open decision. +5. A user can correct a mapping, disable reviews for a repository, and rerun a review. +6. The project's settings control whether production evidence is included in + GitHub comments. It is included by default; the alternative posts the + verdict and links while keeping production details in Monoscope. + +Existing code context mappings can seed step 2, but today they are entered by the +customer and an optional service means a repository may match several services. +They improve a review; they do not gate it. Automatic setup therefore needs a +visible mapping result, not an assumed match. +The project page should show each eligible repository's review state and inferred +services, with a one-step correction for an ambiguous or missing service. A +repository can still receive a coverage-unknown review before that correction. +If one installation or repository is connected to several Monoscope projects, +the publishing owner needs an explicit rule; do not silently combine projects' +telemetry in a single GitHub comment. + +### Proposed first release slice (pending product choices) + +Start with connected GitHub source repositories and observability continuity: +changed instrumentation, service identity, attributes, and names that existing +telemetry, monitors, or dashboards depend on. Include a blind-spot finding when +the changed path cannot be mapped to a service or measured. This yields a +specific, testable review promise before expanding to reliability and performance +claims. Keep the broader finding families in the review contract below. + +For each PR head revision: verify the delivery and project/repository grant; +load the changed files and current mapping; collect bounded telemetry and +monitor/dashboard references; build structured observed/inferred/unknown +findings; publish one comment for that revision; and store the run and its +evidence links. Repeated deliveries reuse the same run. A new head supersedes +older work, and an older worker cannot overwrite the newer comment. If a query +fails, the comment reports incomplete coverage rather than a clean verdict. + +Use the existing project AI and routine surfaces for related investigations, +but keep the review run separate from a chat transcript. Project memory is +retrieved before evidence collection and updated only after a sourced finding +is confirmed. This is a proposed rollout sequence, not a product decision yet. + +Proposed delivery order: (1) project-scoped evidence links, memory, and verified +GitHub delivery; (2) automatic PR reviews with an advisory comment and check; +(3) daily codebase scans using the same evidence path; (4) on-demand reliability +reports with Monoscope signals; (5) CloudWatch, Datadog, and Sentry adapters. +Each stage has an end-to-end acceptance case below. The final scope includes +all five stages; launch grouping and order remain open to the customer priority. + +## Review contract + +Every finding contains a changed file and line, a concrete failure mechanism, +the affected service or endpoint, the measured production context, and the next +check or fix. It cites direct links to Monoscope evidence with explicit time range +and environment. A claim that a change *will* break production needs stronger +proof than a historical correlation. + +### Evidence join and confidence + +- **Direct dependency:** a changed telemetry name or attribute appears in an + existing monitor or dashboard query, and production data confirms the old + signal is emitted by the mapped service. Show the exact query reference, + changed line, time window, and whether a query covering both names would keep + historical data visible after the change. + This can support an action-needed finding about continuity. +- **Mapped service:** the repository is linked to a service, but the specific + changed path has no known runtime link. Show that service's traffic, issues, + and representative traces as context; label the path-to-impact link as + inferred. High traffic alone does not prove the diff will cause a failure. +- **Repository only:** the App can read the diff, but a catch-all code mapping, + missing deployment revision, or ambiguous services prevent a sound production + join. Publish coverage unknown and a one-step mapping action. Do not present + another service's trace as proof for this PR. + +Revision evidence matters separately from service mapping. A SHA-like +`service.version`, `vcs.repository.ref.revision`, or `git.commit.sha` can tie +observed telemetry to a deployed revision after the identifier is resolved +against the repository; a version string or a repository's current branch +cannot. A trace shown in the comment must name whether it comes +from the exact endpoint or only the mapped service. Query-reference detection +should use parsed KQL where possible; raw SQL or template-driven queries may +need an explicit unknown instead of a text-match verdict. + +Initial finding families: + +- **Observability break:** removing or renaming spans, logs, metric names, labels, + trace correlation, service identity, or attributes used by existing charts, + monitors, and investigations. Check whether old and new data would remain + queryable as one historical series, or whether dashboards and baselines split. + For a proposed rename, show whether a query that includes both names preserves + the series and whether the affected monitor/chart can use it during migration. + OpenTelemetry's semantic-convention migrations use dual emission as a phased + option, which is a useful repair pattern to suggest when it fits. +- **Reliability impact:** a change to a path with meaningful live traffic, + errors, open issues, incident history, or downstream dependencies; explain the + diff's specific risk, not just that the path is busy. +- **Performance impact:** query, dependency, retry, timeout, or high-volume path + changes with relevant latency, volume, or cost baselines. +- **Blind spot:** a changed path cannot be assessed because relevant telemetry, + source mapping, deployment version, or trace linkage is absent. + +The summary distinguishes **action needed**, **worth checking**, **coverage +unknown**, and **no finding**. “No finding” means the review completed with +adequate coverage; it is not a substitute for a failed query or absent data. +Documentation-only and test-only changes need no production finding. The review +must suppress duplicate or weak claims and state its evidence limits. + +## Evidence shown in the PR + +- A short impact statement and the changed line that motivated it. +- An embedded chart image with values, window, environment, units, and sample + size, plus short relevant trace and historical issue context. The reviewer + can understand the finding from the comment alone. Links open the + interactive chart, trace, or issue for deeper inspection. +- A statement of what was observed versus inferred, plus missing coverage. + +The default GitHub output includes bounded production aggregates, a short +sanitized trace excerpt when useful, and the relevant historical context. +The project can switch to links-only output. Neither mode posts raw log bodies, +secrets, or personal data. The embedded image is an immutable PNG chart snapshot, +with the same approved aggregates as the comment and no additional sensitive +data. GitHub cannot render images served behind authentication, and anyone +with an anonymized image URL may view it, so treat the snapshot as shareable +by URL. The links-only project setting suppresses both the chart and its +numbers in GitHub. Monoscope's existing widget PNG endpoint is unsuitable: +it is not a fixed snapshot, and the URL's time range is not covered by its +signature. Do not reuse it for PR evidence. GitHub CLI can attach images, but its documented +workflow requires repository push access; do not expand the App's repository +write permission solely to obtain chart attachments. + +### Illustrative PR comment (fictional data) + +> **Action needed · checkout-api · reviewed `abc1234`** +> +> `src/checkout/instrumentation.ts:84` replaces `http.route` with +> `route.name`. The checkout latency monitor and its dashboard still group by +> `http.route`. If the old attribute stops being emitted, those views will no +> longer match new spans, and their historical series will be split from new +> observations. +> +> **Observed:** 1.2 million checkout spans carried `http.route` in the last +> seven days; the monitor and dashboard both reference it. An incident on +> September 18 used this monitor. Daily volume: 164k · 169k · 171k · +> 174k · 170k · 176k · 176k. A representative trace spans the checkout +> handler and payment call, both grouped under the old route attribute. +> +> ![Bar chart of daily checkout spans in production, Sep 24–30: 164k, 169k, 171k, 174k, 170k, 176k, and 176k.](assets/pr-impact-checkout-volume.svg) +> +> [Open interactive chart] · [View monitor] · [View trace] +> +> **Change to check:** keep the old attribute during migration, or update +> the monitor and dashboard with a query that covers both names. Verify a +> post-deploy span before removing the old name. + +The real comment must link to exact project-scoped evidence and show its +time window. The example uses invented counts and a local SVG illustration; +publishing substitutes an immutable PNG snapshot URL for the local asset. + +## Project memory + +The reviewer, daily scan, chat, and incident investigations share memory +scoped to the Monoscope project. Conversation transcripts and run records stay +as history; they are not themselves trusted project memory. + +- **Project briefing:** a short, editable overview of stable architecture, + service boundaries, links to canonical repository-to-service mappings, + telemetry conventions, and known operational rhythms. Agents read it at + the start of a project task. They can propose a briefing update when new + evidence changes that overview; the change and its source remain visible. +- **Fact records:** one sourced codebase fact, operational decision, or confirmed + failure mode per record, tagged with service, repository, and environment. + Examples are "checkout workers publish to this queue," "the team retains + `http.route` during migration," and "this dependency times out during the + morning import." Each record has a source (code revision and path, query and + time window, trace, issue, deployment, or human author), an observed date, + and a status: current, needs recheck, or archived. Agents retrieve relevant + records when reviewing a changed path or investigating an issue. + +Each review, scan, chat, or investigation can propose new facts or an update to +an existing one. Deduplicate by subject and claim before promotion. A record +becomes reusable memory automatically when its source confirms the claim; +members can correct it. A member's own decision or note is marked as authored +by that member, not presented as measured telemetry. A dismissed PR finding is +feedback, not proof of the opposite claim. Keep uncertain hypotheses in the run +record, where they remain accessible without becoming trusted memory. + +Project members can browse, search, edit, pin, and archive records from a +project page, and export the briefing and records through a project-scoped API. +The PR reviewer, daily scan, chat, and incident investigator use the same +project-scoped search tool; agents see the record's source and status alongside +its text. A memory remains accessible even if its originating conversation is +deleted or hidden from the agent's current thread. +An edit keeps the earlier version and its source. A new deployment, changed +instrumentation, a change to the cited code path, contradictory telemetry, or +a broken source link marks the affected record for recheck; agents then verify +it against current code or live evidence before citing it. Do not duplicate +current telemetry values or repository contents as durable facts when the +source can be queried directly. + +Memory retrieval must enforce project membership and project ID before search +or prompt assembly. A source record from one project never becomes context +for another. A PR comment may quote a memory only when its source and current +status support the specific claim. + +This follows Polylane's confirmed, sourced memories and editable workspace +notes, LangGraph's separation of thread history from cross-thread memory, +and Letta's shared context blocks for information several agents need. +The project briefing is kept short; specific findings are retrieved when +relevant instead of loading an ever-growing transcript into every review. +Daily scan and review runs remain accessible as dated history; only durable, +confirmed conclusions are promoted into memory. + +### Review engine and prior art + +Use Monoscope's existing project-scoped agent and GitHub integration for the +first release. Add read-only, revision-pinned source tools and a bounded review +run that joins source changes to Monoscope evidence. Keep project memory in +Monoscope so review, scan, chat, and investigations read the same sourced facts. +An agent transcript or condensed context is run history, not project memory. + +- OpenCodeReview's deterministic diff selection, related-file grouping, + read-only code tools, coverage warnings, and final finding filter are useful + review patterns. Its context compression only shortens one run; durable + domain-specific memory is on its roadmap. Its default defect rules would + need replacement with the production-impact contract here. +- PR-Agent shows how to maintain one updated PR comment, preserve finding + state across reruns, expose review coverage, read trusted repository context + from the default branch, and keep per-repository settings. Its repository + instruction files and PR finding state can complement Monoscope memory but + do not replace sourced, project-scoped facts shared across workflows. +- Pi offers an embeddable agent SDK, custom tools, and persistent sessions. + It is a candidate for a separately deployed source-reading worker if the + existing Haskell loop proves limiting. It adds a Node/Bun runtime and requires + an external sandbox for filesystem, process, network, and credential access. + Its session persistence serves transcripts rather than authoritative + cross-workflow project memory. + +The boundary remains stable if the review engine changes: supply a pinned PR +diff, bounded read-only source access, relevant memory records, and verified +telemetry evidence; receive structured observed/inferred/unknown findings with +source references. Monoscope validates the evidence, applies project policy, +and publishes the comment and chart snapshot. + +## Daily codebase scan + +Read each selected repository's current revision daily, even when no PR is open. +Reuse the same service mapping and evidence rules to find instrumentation gaps, +bad metadata, missing trace correlation, and performance regressions. Compare +code instrumentation with the signals Monoscope actually receives, and name +the affected service and code or configuration location. The result destination +and launch order relative to PR reviews are open. + +The scan compares the current default-branch revision with the previous scan, +then checks candidates against recent production signals. Examples: a span or +log attribute emitted in code but absent from observed telemetry; missing +`service.name`, environment, version, or trace IDs in a measured share of +records; unstable labels that fragment a metric; and a code change near a +measured latency regression. A scan result includes the revision and source +location, affected service, sample size and time window, the observed defect, +the proposed fix, and what remains unknown. A code smell alone is a hypothesis, +not a confirmed production regression. + +Deduplicate recurring results across days and recheck them after a fix or new +deployment. Keep a dated run record even when there is no new finding. Promote +only confirmed, durable findings into project memory. A missing source mapping, +unavailable telemetry, or an unscannable repository is reported as incomplete +coverage, not a clean scan. + +## On-demand reliability check + +A customer can request a check for problems their alerts missed. The agent +examines available telemetry, open issues, monitors, and recent changes, +then returns evidence-backed findings and the blind spots that prevented an +alert. CloudWatch, Datadog, and Sentry are possible additional sources; whether +they are required at launch is open. A connector must distinguish absent data, +permission failure, and a genuinely healthy result. + +The check starts with the project's connected signals and a stated lookback +window. It finds concrete candidates such as a rising error group, a latency +or saturation regression, or an unhealthy dependency; compares each with a +relevant baseline; and inspects active monitors and incident history. Call a +problem **unalerted** only when the observed issue and alert gap are both +supported: no matching active monitor, a disabled or muted monitor, a threshold +that missed the observed data, or a delivery failure. Show the actual query, +chart, trace or issue, affected service, onset, and why the alert did not cover +it. An uncertain monitor match is stated as unknown, not as proof that no alert +exists. + +The result is a project-scoped report with ranked findings, suggested monitor +or instrumentation changes, source links, and a coverage table for Monoscope +and each connected provider. Providers that are not connected remain explicit +gaps. Repeat runs compare with prior findings and do not repeatedly announce +the same unchanged problem. The customer can run it again after changing an +alert, then inspect whether the coverage gap closed. + +## Technical starting point + +- The GitHub App installation, credential, and repository picker already exist. +- `projects.code_mappings` links repositories to a project and optionally a + service; deployment evidence and source reads use those mappings. Its current + path mapping resolves stack-frame paths into repository files, not arbitrary + changed files into runtime endpoints. A mapping with no service is a repo + link, not evidence that every service in the project uses its code. +- Telemetry carries `service.name`, `service.version`, and trace IDs when the + instrumented application emits them. The existing source-context logic also + recognizes SHA-like revision attributes; semver alone is not a commit link. + Monitor and dashboard KQL queries are available to inspect for signal + dependencies, while raw SQL and query templates need separate handling. +- The GitHub webhook currently accepts push events for dashboard sync and ignores + pull request events. It also looks up only the single dashboard-sync repository + before verifying the delivery. Source repositories linked through + `code_mappings` do not reach this handler's accepted path. The current GitHub + verifier accepts a missing per-repository secret, so PR publishing needs a + configured GitHub App webhook secret and signature verification before any + project lookup or queued work. After verification, resolve the installation + and repository to a linked project, then fetch, persist the run, and publish. + Those are new paths. GitHub's App guidance recommends verifying webhook + signatures and includes the installation ID in App deliveries. +- The source-code picker currently presents one preferred credential even though + a project can hold several account grants. Auto setup must discover eligible + repositories across the project's grants and show which installation owns + each repository; an account that sorts or was installed most recently cannot + silently determine the PR review target. +- The GitHub App will need PR read, PR comment write, and Checks write for the + proposed advisory check. Treat expanded permissions as part of installation + and upgrade UX. +- Existing agent tools can query telemetry, issues, incidents, endpoints, + monitors, repositories, and deployments. The review needs bounded, + project-scoped access and a structured evidence result before publishing. +- Existing project AI conversations and routine runs retain history, but no + project-memory search or fact-promotion tool appears in the agent tool list. + Shared briefing, fact records, their source/status, and scoped retrieval are + new paths rather than more text appended to conversation metadata. +- Existing routine templates cover telemetry quality, gaps, correlation, + performance, and deployment regressions; the daily scan should reuse their + evidence logic where it applies. Those routines do not read a repository's + current revision or track code findings across daily runs yet. +- No CloudWatch, Datadog, or Sentry provider client is present in the current + server code. Reading those providers and evaluating their alert coverage are + separate connector work; their names in the current UI/code are comparisons. + +## Acceptance checks + +- Installing the app with repository access is sufficient to start automatic + reviews; an existing source mapping improves evidence without a second form. + Ambiguous service mapping yields explicit coverage unknown. +- A source repository linked outside dashboard sync receives verified PR events. + An unsigned delivery queues no review. Multiple account grants and a repository + linked to several projects follow the chosen ownership rule without mixing + project evidence. +- A PR open or head update produces one current review for that exact revision; + duplicate deliveries do not produce duplicate comments. +- A finding can be traced from changed line to service to time-bounded + production evidence. A reviewer can open every cited chart, issue, or trace. +- In full-evidence mode, the comment renders a chart image for each relevant + time series with its window, environment, units, and sample size visible; + the chart link remains available. In links-only mode it renders no aggregate + numbers or chart image. Reopening the same PR later shows the same snapshot, + and changing image URL parameters cannot broaden its time or project scope. +- A catch-all repository mapping does not turn unrelated services' telemetry + into proof. A missing or non-SHA service version downgrades revision certainty, + and a representative service trace is labeled as such. +- Missing telemetry, unmapped repos, query failures, and low sample sizes are + visibly different from a clean result. +- A changed attribute used by a monitor or dashboard is caught in an end-to-end + fixture, including its effect on older telemetry; a stylistic diff produces + no finding. +- The daily scan identifies a known missing correlation or metadata defect in + an end-to-end fixture and cites the affected service. A second unchanged run + does not announce it again, while its dated run remains accessible. +- An on-demand check reports an unalerted issue with supporting telemetry and + identifies the absent or ineffective monitor. Disconnected and failed + providers appear as coverage gaps; rerunning after an alert change can show + whether the gap closed. +- A confirmed finding from one run is available to a later PR review in the + same project, with its source and status. It is absent from another project; + a retracted or stale finding is not cited as current evidence. +- A sourced codebase fact learned in one scan is searchable by a project member + and by a later agent run. Changing its cited code path marks it for recheck; + deleting the originating conversation does not silently erase it. + +Before making reviews a required check, measure review completion time, +repository-to-service coverage, evidence-link validity, duplicate comments, +findings engineers accepted or dismissed, and missed problems found after merge. +Review a sample of clean verdicts as well as flagged PRs; a low finding count +alone does not prove the reviewer is accurate. + +## Recommended defaults and decisions to confirm + +These defaults make the scope implementable without treating unanswered product +choices as agreed decisions. + +| Choice | Working default | Reason to revisit | +| --- | --- | --- | +| Setup | Review all repositories granted to the project's GitHub App installation; allow a per-repository off switch. | A customer may grant repositories used only for dashboard sync or code context. | +| GitHub output | One updated comment with all evidence, plus a non-blocking check that mirrors its verdict. A project may later make the check required. | Extra Checks permission and launch complexity. | +| Service mapping | Infer from source mappings, deployment/version metadata, and telemetry. Show an explicit coverage gap when ambiguous, with one-step correction. | Some monorepos may need a path-to-service map. | +| Finding priority | Make observability continuity and blind spots excellent first, then expand to concrete reliability and performance impacts. Keep all families in the final product. | A customer may value live reliability risk first. | +| Scan and check delivery | Project report history; notify only for new, evidence-backed findings. | Team may prefer Slack or GitHub issues. | +| External telemetry | Use Monoscope signals first; add CloudWatch, Datadog, and Sentry as explicit connected sources for the on-demand check. | Customers may require these connectors before adopting the check. | +| Project memory | Monoscope project is canonical, with member UI, project-scoped search/API, and export. Automatically promote sourced facts; members can edit or archive them. | Repository-owned memory may be important for coding agents outside Monoscope. | +| Chart proof | Embed an immutable chart snapshot with the same bounded aggregates shown in the comment; link the interactive chart too. The existing project links-only setting removes both. | A GitHub image URL is shareable by anyone who has it, so chart content must stay within the approved aggregate boundary. | + +The highest-impact confirmations are launch order across the three workflows +and how one repository shared by multiple Monoscope projects chooses its +publishing owner. The latter cannot be +chosen from an arbitrary credential order or by mixing project telemetry. + +## References + +- [GitHub image proxy and private image limits][github-images] +- [GitHub CLI image attachment permissions][github-attachments] +- [GitHub App permission for PR comments][github-comments] +- [GitHub App permission for check runs][github-checks] +- [GitHub App webhook signature guidance][github-webhooks] +- [GitHub App webhook installation identity][github-installation] +- [Polylane memories and notes][polylane-memories] +- [Polylane production impact review][polylane-impact] +- [Superlog proactive reliability and daily observability review][superlog] +- [OpenTelemetry semantic convention migration guidance][otel-migration] +- [LangGraph memory overview][langgraph-memory] +- [Letta shared memory blocks][letta-blocks] +- [OpenCodeReview architecture][ocr-architecture] +- [OpenCodeReview roadmap][ocr-roadmap] +- [PR-Agent review behavior][pr-agent-review] +- [PR-Agent repository context][pr-agent-context] +- [Pi SDK][pi-sdk] +- [Pi permissions][pi-permissions] + +[github-images]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/about-anonymized-urls +[github-attachments]: https://docs.github.com/en/github-cli/github-cli/attaching-files-with-github-cli +[github-comments]: https://docs.github.com/en/rest/issues/comments +[github-checks]: https://docs.github.com/en/rest/guides/using-the-rest-api-to-interact-with-checks +[github-webhooks]: https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/best-practices-for-creating-a-github-app +[github-installation]: https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/building-ci-checks-with-a-github-app +[polylane-memories]: https://docs.polylane.com/context/memories +[polylane-impact]: https://polylane.com/product/impact-intelligence/ +[superlog]: https://superlog.sh/ +[otel-migration]: https://opentelemetry.io/docs/specs/semconv/non-normative/db-migration/ +[langgraph-memory]: https://docs.langchain.com/oss/python/concepts/memory +[letta-blocks]: https://docs.letta.com/v1-sdk/memory/memory-blocks +[ocr-architecture]: https://github.com/alibaba/open-code-review/blob/main/pages/src/content/docs/en/architecture.md +[ocr-roadmap]: https://github.com/alibaba/open-code-review/blob/main/ROADMAP.md +[pr-agent-review]: https://github.com/The-PR-Agent/pr-agent/blob/main/docs/docs/tools/review.md +[pr-agent-context]: https://github.com/The-PR-Agent/pr-agent/blob/main/docs/docs/usage-guide/additional_configurations.md +[pi-sdk]: https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/sdk.md +[pi-permissions]: https://github.com/earendil-works/pi#permissions--containerization diff --git a/docs/plans/assets/pr-impact-checkout-volume.svg b/docs/plans/assets/pr-impact-checkout-volume.svg new file mode 100644 index 000000000..6d65401a7 --- /dev/null +++ b/docs/plans/assets/pr-impact-checkout-volume.svg @@ -0,0 +1,44 @@ + + Checkout spans per day + Production, September 24 to 30. Daily counts in thousands: 164, 169, 171, 174, 170, 176, 176. Total 1.2 million spans. + + Checkout spans per day + Production · Sep 24–30 · 1.2M spans + + + + + + + 200k + 100k + 0 + + + + + + + + + + + + 164k + 169k + 171k + 174k + 170k + 176k + 176k + + + Sep 24 + 25 + 26 + 27 + 28 + 29 + 30 + + diff --git a/e2e/tests/dashboard-add-widget.spec.ts b/e2e/tests/dashboard-add-widget.spec.ts index a3d29fa1c..32be4a294 100644 --- a/e2e/tests/dashboard-add-widget.spec.ts +++ b/e2e/tests/dashboard-add-widget.spec.ts @@ -52,7 +52,7 @@ type Dash = { id: string; title: string }; async function makeDashboard(page: Page, title: string, template = "Blank dashboard"): Promise { await page.goto(`/p/${DEMO_PROJECT}/dashboards`); await page.locator('label[for="newDashboardMdl"]').first().click(); - await page.getByText(template, { exact: true }).click(); + await page.locator("#dashListItemParent").getByText(template, { exact: true }).click(); await page.getByRole("textbox", { name: "Dashboard name *", exact: true }).fill(title); await page.getByRole("button", { name: "Create" }).first().click(); await page.waitForURL(/\/dashboards\/[0-9a-f-]{36}/i, { timeout: 60000 }); @@ -94,6 +94,31 @@ async function openWidgetDrawer(page: Page) { await expect(page.locator("#visualizationTabs")).toBeVisible({ timeout: 20000 }); } +test("dashboard templates prefill the name and preview their own widgets", async ({ page }) => { + await page.setViewportSize({ width: 1440, height: 900 }); + await page.goto(`/p/${DEMO_PROJECT}/dashboards?new=true`); + const modal = page.locator("#newDashboardMdl + .modal .modal-box"); + const preview = page.locator("#dashboardTemplatePreviews"); + const create = page.getByRole("button", { name: "Create", exact: true }); + const initial = { modal: await modal.boundingBox(), preview: await preview.boundingBox(), create: await create.boundingBox() }; + const name = page.getByRole("textbox", { name: "Dashboard name *", exact: true }); + await expect(name).toHaveValue("Blank dashboard"); + await page.locator("#dashListItemParent").getByText("Nginx", { exact: true }).click(); + await expect(name).toHaveValue("Nginx"); + await expect(page.locator('#dashboardTemplatePreviews [data-template="nginx.yaml"]')).toBeVisible(); + await expect(page.locator('#dashboardTemplatePreviews [data-template="nginx.yaml"]')).toContainText("Requests"); + const selected = { modal: await modal.boundingBox(), preview: await preview.boundingBox(), create: await create.boundingBox() }; + for (const part of ["modal", "preview", "create"] as const) { + for (const edge of ["x", "y", "width", "height"] as const) { + expect(Math.abs(selected[part]![edge] - initial[part]![edge]), `${part} ${edge} moved`).toBeLessThan(1); + } + } + await name.fill("My infrastructure"); + await page.locator("#dashListItemParent").getByText("Redis", { exact: true }).click(); + await expect(name).toHaveValue("My infrastructure"); + await expect(page.locator('#dashboardTemplatePreviews [data-template="redis.yaml"]')).toBeVisible(); +}); + test.describe("adding widgets to a dashboard", () => { // Every test here writes to one shared dashboard. test.describe.configure({ mode: "serial" }); @@ -587,3 +612,62 @@ test("widget monitors persist independently across dashboards and reopen with sa await expect(form.locator('[name="alertThreshold"]')).toHaveValue("10.0"); await expect(form.getByRole("button", { name: "Update monitor", exact: true })).toBeVisible(); }); + +// Regressions from one bug report, all on a tabbed template (root `widgets: []`): the list +// counted 0 widgets and clipped rows below the fold, unstarred stars were invisible until +// hover, and the expand drawer saved edits to the root list, so a rename silently vanished. +test("a tabbed dashboard lists its widgets and keeps a rename made in the widget drawer", async ({ page }) => { + test.slow(); + const dash = await makeDashboard(page, `E2E Tabbed ${Date.now()}`, "Overview"); + + await page.setViewportSize({ width: 1280, height: 360 }); + await page.goto(`/p/${DEMO_PROJECT}/dashboards`); + const row = page.locator("#dashboardsTable tbody tr", { hasText: dash.title }); + expect(Number(await row.locator("td").last().innerText())).toBeGreaterThan(0); + await expect(row.getByRole("button", { name: "Add dashboard to favorites" })).toHaveCSS("opacity", "1"); + const lastRow = page.locator("#dashboardsTable tbody tr").last(); + await page.locator("#main-content").hover(); + await page.mouse.wheel(0, 5000); + await expect.poll(async () => (await lastRow.boundingBox())!.y + (await lastRow.boundingBox())!.height).toBeLessThanOrEqual(360); + + // Escape used to be heard only by the hidden toggle, never by the focused modal content. + await page.locator('label[for="newDashboardMdl"]').first().click(); + await expect(page.locator("#newDashboardMdl")).toBeChecked(); + await page.keyboard.press("Escape"); + await expect(page.locator("#newDashboardMdl")).not.toBeChecked(); + + await page.setViewportSize({ width: 1440, height: 900 }); + await page.goto(`/p/${DEMO_PROJECT}/dashboards/${dash.id}`); + // Tab links used to carry the server-rendered range, so a range picked here reset on switch. + await page.locator("#n-currentRange").click(); + await page.locator("#n-timepicker-popover button", { hasText: "Last 24 hours" }).click(); + await page.getByRole("tab", { name: "Service", exact: true }).click(); + await expect(page).toHaveURL(/\/tab\/service\?.*since=24H/); + await page.getByRole("tab", { name: "Overview", exact: true }).click(); + + await page.getByRole("button", { name: "Expand widget", exact: true }).first().click(); + await page.getByRole("tab", { name: "Edit", exact: true }).click(); + const editor = page.locator(".widget-editor"); + // The log explorer's timeline/monitor toggles did nothing here; the Monitors tab owns alerts. + await expect(editor.getByText("Hide timeline")).toHaveCount(0); + const renamed = `Renamed ${Date.now()}`; + await editor.getByRole("textbox", { name: "Widget title" }).fill(renamed); + await Promise.all([ + page.waitForResponse((r) => r.request().method() === "PUT" && r.url().includes("tab=") && r.ok()), + editor.getByRole("button", { name: "Save changes" }).click(), + ]); + await page.reload(); + await expect(page.getByText(renamed).first()).toBeVisible(); + + // The reload keeps ?expand=, so the drawer reopens on the same widget. + await page.getByRole("tab", { name: "Edit", exact: true }).click(); + // The editor links to its Monitors tab only for a KQL widget; a raw-SQL one cannot be monitored. + const monitorLink = editor.locator('label[for$="-tab-monitors"]'); + const hasKql = await page.evaluate(() => Boolean((window as any).widgetJSON.query?.trim())); + await expect(monitorLink).toHaveCount(hasKql ? 1 : 0); + if (hasKql) { + await monitorLink.click(); + await expect(editor.locator(".page-drawer-tab-monitors")).toBeChecked(); + } + await deleteDashboard(page, dash); +}); diff --git a/e2e/tests/infrastructure-time-window.spec.ts b/e2e/tests/infrastructure-time-window.spec.ts index f7a73b298..03f3078c1 100644 --- a/e2e/tests/infrastructure-time-window.spec.ts +++ b/e2e/tests/infrastructure-time-window.spec.ts @@ -6,15 +6,16 @@ const cluster = "8b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d"; const provider = "e2e-infra-provider"; const cleanup = `DELETE FROM otel_metrics WHERE project_id='${DEMO_PROJECT}' AND resource->'k8s'->'namespace'->>'name'='${namespace}';`; test.beforeAll(async ({ request }) => { + test.setTimeout(90_000); if (!process.env.E2E_BASE_URL) return; sql(cleanup + [30, 600].map(age => `INSERT INTO otel_metrics (project_id,id,series_id,timestamp,metric_name,metric_type,value,resource___k8s___container___name,resource___k8s___pod___name,resource___k8s___namespace___name,resource) VALUES ('${DEMO_PROJECT}',gen_random_uuid(),'infra-preset-${age}','${new Date(Date.now() - age * 1000).toISOString()}','container.cpu.usage','GAUGE',1,'infra-preset-${age}','infra-preset-pod-${age}','${namespace}', '${JSON.stringify({ k8s: { container: { name: `infra-preset-${age}` }, pod: { name: `infra-preset-pod-${age}` }, namespace: { name: namespace }, cluster: { uid: cluster }, node: { name: `infra-preset-host-${age}` } }, container: { image: { name: `registry.example/infra-preset-${age}` } }, cloud: { provider, region: "e2e-infra-region" } })}');`).join("")); - // Match the page's raw cache key; earlier fixture pages omit from/to. - await expect.poll(async () => { - const response = await request.get(`/p/${DEMO_PROJECT}/infrastructure/hosts?${new URLSearchParams({ provider, group: "region", from: "", to: "", since: "5m", deferred: "1" })}`); + // Match both page cache keys before asserting the picker changes the inventory. + for (const [since, host] of [["5m", "infra-preset-host-30"], ["1H", "infra-preset-host-600"]]) await expect.poll(async () => { + const response = await request.get(`/p/${DEMO_PROJECT}/infrastructure/hosts?${new URLSearchParams({ provider, group: "region", from: "", to: "", since, deferred: "1" })}`); expect(response.status()).toBe(200); - return (await response.text()).includes("infra-preset-host-30"); + return (await response.text()).includes(host); }, { timeout: 40_000 }).toBe(true); }); test.afterAll(() => { if (process.env.E2E_BASE_URL) sql(cleanup); }); @@ -122,10 +123,10 @@ for (const [route, container, filters] of [...views, ["containers", "containersC try { await page.goto(`/p/${DEMO_PROJECT}/infrastructure/${route}?${new URLSearchParams({ ...filters, from: "", to: "", since: "5M" })}&since=5m`, { waitUntil: "commit" }); const entries = page.locator(`#${container} ${route === "host-map" ? "button[data-hx-get]" : 'tr[role="button"]'}`); - await page.waitForFunction(() => Boolean((window as any).htmx && document.querySelector("[data-deferred-shell]"))); + await page.waitForFunction(() => Boolean((window as any).htmx)); await expect.poll(() => page.evaluate(() => typeof (window as any).params)).toBe("undefined"); - // Process the actual shell before deferred application globals arrive. - await page.evaluate(() => (window as any).htmx.process(document.querySelector("[data-deferred-shell]"))); + // The shell may have loaded automatically before this test reaches it. + await page.evaluate(() => { const shell = document.querySelector("[data-deferred-shell]"); if (shell) (window as any).htmx.process(shell); }); await expect(entries).toHaveCount(1); expect(new URL((await entries.first().getAttribute("data-hx-get"))!, page.url()).searchParams.get("since")).toBe("5m"); } finally { release(); } diff --git a/e2e/tests/query-editor.spec.ts b/e2e/tests/query-editor.spec.ts index 216b8db62..72f81ebcc 100644 --- a/e2e/tests/query-editor.spec.ts +++ b/e2e/tests/query-editor.spec.ts @@ -146,3 +146,19 @@ test.describe("Query editor", () => { await expect(library.getByRole("tab", { name: tab })).toBeVisible(); }); }); + +// Regression: the explorer ships without Tagify, so the monitor panel's teams picker threw +// "Tagify is not a constructor" and never initialised. Also the only coverage of creating a +// monitor from the explorer. +test("the explorer creates a monitor with a working teams picker", async ({ page }) => { + const title = `E2E Explorer Monitor ${Date.now()}`; + await page.goto(`/p/${DEMO_PROJECT}/log_explorer#create-alert-toggle`); + const form = page.locator("#alert-form"); + await expect(form).toBeVisible({ timeout: 20000 }); + await expect.poll(() => page.evaluate(() => Boolean((document.getElementById("alert-form-teams") as any)?._tagifyInstance))).toBe(true); + await form.locator('[name="title"]').fill(title); + await form.locator('[name="alertThreshold"]').fill("10"); + await form.getByRole("button", { name: /Create monitor/ }).click(); + await page.goto(`/p/${DEMO_PROJECT}/monitors`); + await expect(page.getByText(title, { exact: true })).toBeVisible(); +}); diff --git a/e2e/tests/sidebar-navigation.spec.ts b/e2e/tests/sidebar-navigation.spec.ts index a0e8ac9ba..5225c8c46 100644 --- a/e2e/tests/sidebar-navigation.spec.ts +++ b/e2e/tests/sidebar-navigation.spec.ts @@ -53,3 +53,16 @@ test('Explorer stays highlighted through tab navigation and browser history', as await expect(page).toHaveURL(/\/service_map(?:\?|$)/); await expect(explorer).toHaveAttribute('aria-current', 'page'); }); + +// Regressions: the palette's own lazy fetch closed it on the first Cmd+K, the fuzzy filter +// hid the Ask AI row, and Ask AI navigated with the query alone, dropping its time range. +test('command palette Ask AI opens the explorer with the AI time range', async ({ page }) => { + await page.route('**/log_explorer/ai_search', (route) => + route.fulfill({ contentType: 'application/json', body: JSON.stringify({ query: 'level == "ERROR"', time_range: { since: '1H' } }) })); + await page.goto(project + '/dashboards'); + await page.locator('body').press('Control+k'); + await page.locator('#cmd-palette-input').fill('errors from checkout in the last hour'); + await page.locator('[data-ai-action]').click(); + await expect(page).toHaveURL(/\/log_explorer\?.*since=1H/); + expect(new URL(page.url()).searchParams.get('query')).toBe('level == "ERROR"'); +}); diff --git a/scripts/ci/ci.sh b/scripts/ci/ci.sh index 34739937d..86e23d025 100755 --- a/scripts/ci/ci.sh +++ b/scripts/ci/ci.sh @@ -338,7 +338,7 @@ run_body() { # npx tailwindcss -i ./static/public/assets/css/tailwind.css -o ./static/public/assets/css/tailwind.min.css --minify (cd web-components && npm ci --prefer-offline --no-audit && NODE_ENV=production npx vite build --mode production --sourcemap false) ;; - build) cabal build all -j $CABAL_FLAGS "$CABAL_OPTS" ;; + build) cabal build all unit-tests monoscope-cli:cli-tests integration-tests -j $CABAL_FLAGS "$CABAL_OPTS" ;; # doctest shells out to GHC with `-package` for each local library. A fresh # runner can legitimately reuse the separate build attestation, but then # those libraries are absent from its package database. Build them here so diff --git a/src/Models/Projects/Dashboards.hs b/src/Models/Projects/Dashboards.hs index 90ac0a278..2fa02e6cb 100644 --- a/src/Models/Projects/Dashboards.hs +++ b/src/Models/Projects/Dashboards.hs @@ -17,6 +17,7 @@ module Models.Projects.Dashboards ( readDashboardsFromDirectory, readDashboardEndpoint, validateDashboard, + allWidgets, replaceDashboardVariables, deleteDashboardsByIds, addTeamsToDashboards, @@ -257,6 +258,11 @@ readDashboardEndpoint uri = do >>= validateDashboard +-- | Every widget across root and tabs, group children included. +allWidgets :: Dashboard -> [Widget.Widget] +allWidgets dashboard = foldMap (universeOf (#children . _Just . folded)) $ dashboard.widgets <> foldMap (concatMap (.widgets)) dashboard.tabs + + -- | Reject handwritten sortable SQL unless it exposes the trusted sort slot -- and a fallback order. Without both, the browser can display a sortable header -- while the database still limits the original order, which is a false control. @@ -265,11 +271,9 @@ validateDashboard dashboard = case violations of [] -> Right dashboard xs -> Left $ "sortable SQL widgets require {{table_sort}} and default_sort: " <> T.intercalate ", " xs where - widgets = concatMap flatten $ dashboard.widgets <> maybe [] (concatMap (.widgets)) dashboard.tabs - flatten widget = widget : maybe [] (concatMap flatten) widget.children violations = [ fromMaybe "" widget.title - | widget <- widgets + | widget <- allWidgets dashboard , any ((== Just True) . (.sortable)) $ fromMaybe [] widget.columns , Just sql <- [widget.sql] , isNothing widget.defaultSort || not ("{{table_sort}}" `T.isInfixOf` sql) diff --git a/src/Pages/BodyWrapper.hs b/src/Pages/BodyWrapper.hs index 9af43380a..47deee128 100644 --- a/src/Pages/BodyWrapper.hs +++ b/src/Pages/BodyWrapper.hs @@ -320,6 +320,10 @@ bodyWrapper bcfg child = do void $ script_ "window.gridStackReady = new Promise(resolve => { window.__resolveGridStack = resolve; });" script_ [src_ src, defer_ "true", onload_ "window.__resolveGridStack?.(window.GridStack)"] ("" :: Text) deferredCss $ assetUrl "/public/assets/css/thirdparty/rrweb.css" + -- Without Tagify up front, main.ts fetches it from here when a tag input first appears. + unless bcfg.needsTagify do + meta_ [name_ "tagify-js", content_ $ assetUrl "/public/assets/deps/tagify/tagify.min.js"] + meta_ [name_ "tagify-css", content_ $ assetUrl "/public/assets/css/thirdparty/tagify.min.css"] mapM_ css $ [assetUrl "/public/assets/css/thirdparty/tagify.min.css" | bcfg.needsTagify] <> [assetUrl "/public/assets/deps/gridstack/gridstack.min.css" | bcfg.needsGridStack] diff --git a/src/Pages/CommandPalette.hs b/src/Pages/CommandPalette.hs index b6523abd3..42db42558 100644 --- a/src/Pages/CommandPalette.hs +++ b/src/Pages/CommandPalette.hs @@ -92,7 +92,7 @@ paletteShell_ pid = do , class_ "cmd-palette-backdrop hidden fixed inset-0 flex flex-col items-center pt-[15vh] bg-black/40" , style_ "z-index:99999" , [__|on click if event.target is me add .hidden to me end - on htmx:before:request from add .hidden to me end|] + on htmx:before:request from if event.target.id is not 'cmd-palette-dynamic' add .hidden to me end|] ] do -- Header @@ -143,9 +143,10 @@ paletteShell_ pid = do [ data_ "ai-action" "true" , [__|on click set :q to #cmd-palette-input.value - fetch `/p/${me.closest('.cmd-palette').dataset.pid}/log_explorer/ai_search` {method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify({input: :q})} - then set :r to it.json() - then set window.location to `/p/${me.closest('.cmd-palette').dataset.pid}/log_explorer?query=${encodeURIComponent(:r.query)}` + fetch `/p/${me.closest('.cmd-palette').dataset.pid}/log_explorer/ai_search` {method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify({input: :q, timezone: Intl.DateTimeFormat().resolvedOptions().timeZone})} as json + then set :r to it + then set tr to :r.time_range + then set window.location to `/p/${me.closest('.cmd-palette').dataset.pid}/log_explorer?query=${encodeURIComponent(:r.query or '')}&since=${tr.since or ''}&from=${encodeURIComponent(tr.from or '')}&to=${encodeURIComponent(tr.to or '')}` |] ] $ itemBody_ "sparkles" "cmd-ai-label" "Ask AI: \"\"" "AI" @@ -215,8 +216,8 @@ paletteShell_ pid = do end end else - -- Root: filter direct + category items - for item in in :palette + -- Root: filter direct + category items; the log/AI shortcuts echo the query, so they never filter + for item in in :palette set :typ to item.dataset.cmdType if :typ === 'child' then -- Show matching children inline during global search diff --git a/src/Pages/Components.hs b/src/Pages/Components.hs index 15e236625..440251a5d 100644 --- a/src/Pages/Components.hs +++ b/src/Pages/Components.hs @@ -937,8 +937,8 @@ modalWith_ modalId cfg triggerM contentHtml = do , id_ modalId , type_ "checkbox" , Aria.label_ "Toggle modal" - , [__|on keyup if the event's key is 'Escape' set my.checked to false trigger keyup end - on closeModal from body set my.checked to false end + , -- Escape is heard on window: focus sits inside the modal, which is not this input's subtree. + [__|on keyup[key is 'Escape'] from window or closeModal from body if my.checked set my.checked to false then send change to me end on change if my.checked then add .overflow-hidden to wait 50ms then diff --git a/src/Pages/Dashboards.hs b/src/Pages/Dashboards.hs index 8d4212c0f..fa653f7f7 100644 --- a/src/Pages/Dashboards.hs +++ b/src/Pages/Dashboards.hs @@ -108,7 +108,7 @@ import Pkg.Components.Table (Table (..)) import Pkg.Components.Table qualified as Table import Pkg.Components.TimePicker qualified as TimePicker import Pkg.Components.Widget qualified as Widget -import Pkg.DeriveUtils (UUIDId (..), WrappedEnumSC (..), assetUrl, bulkActionSlug) +import Pkg.DeriveUtils (UUIDId (..), WrappedEnumSC (..), assetUrl, bulkActionSlug, encodeEnumSC) import Pkg.Metrics qualified as Metrics import Pkg.Parser (QueryComponents (..), SqlQueryCfg (..), applyScopedQuery, binDensityFor, constantToKQLList, constantToSQLList, defSqlQueryCfg, fixedUTCTime, mkScopedQuery, parseQueryToComponents, replacePlaceholders, variablePresetsKQL) import Pkg.SchemaLearning.Catalog qualified as Catalog @@ -1145,8 +1145,15 @@ dashboardWidgetPutH pid dashId widgetIdM tabSlugM widget = do uid <- UUID.genUUID <&> UUID.toText let normalizedWidgetIdM = normalizeWidgetId <$> widgetIdM widgetUpdated = normalizeWidget widget normalizedWidgetIdM uid + -- An edit keeps the sent tab only if the widget is on it (ids repeat across tabs); + -- a missing or stale slug falls back to where the widget actually lives. + tabSlugM' <- case normalizedWidgetIdM of + Nothing -> pure tabSlugM + Just nwid -> + let onTab slug = any (any (any (widgetMatches nwid) . universeOf (#children . _Just . folded)) . (.widgets) . snd) $ findTabBySlug (fold dash.tabs) slug + in maybe (throwError err404{errBody = "Widget not found in dashboard"}) (pure . (mfilter onTab tabSlugM <|>) . fst) $ findWidgetInDashboard nwid dash - _ <- Dashboards.updateSchema dashId (updateDashboardWidgets dash tabSlugM normalizedWidgetIdM widgetUpdated) Nothing + _ <- Dashboards.updateSchema dashId (updateDashboardWidgets dash tabSlugM' normalizedWidgetIdM widgetUpdated) Nothing syncDashboardAndQueuePush pid dashId whenJust normalizedWidgetIdM \nwid -> do existingMonitor <- Monitors.queryMonitorByWidgetId pid (Just dashId.unUUIDId) nwid @@ -1179,11 +1186,16 @@ normalizeWidget widget normalizedWidgetIdM generatedId = updateDashboardWidgets :: Dashboards.Dashboard -> Maybe Text -> Maybe Text -> Widget.Widget -> Dashboards.Dashboard updateDashboardWidgets dash tabSlugM normalizedWidgetIdM widgetUpdated = - overDashWidgets tabSlugM (\ws -> maybe (ws <> [widgetUpdated]) (\nwid -> map (updateWidget nwid) ws) normalizedWidgetIdM) dash + overDashWidgets tabSlugM (\ws -> maybe (ws <> [widgetUpdated]) (\nwid -> overWidgetTree (updateWidget nwid) ws) normalizedWidgetIdM) dash where updateWidget nwid w = if widgetMatches nwid w then mergeWidgetPreservingQuery w widgetUpdated else w +-- | Apply @f@ to every widget, group children included. +overWidgetTree :: (Widget.Widget -> Widget.Widget) -> [Widget.Widget] -> [Widget.Widget] +overWidgetTree = map . transformOf (#children . _Just . traversed) + + -- | A widget is addressed either by its explicit id or by its slugified title. widgetMatches :: Text -> Widget.Widget -> Bool widgetMatches wid w = w.id == Just wid || maybeToMonoid (slugify <$> w.title) == wid @@ -1399,7 +1411,6 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW , term "_" [text| on htmx:before:request - set widgetJSON.type to document.getElementById('${widgetPreviewId}').closest('[class~="group/wgtexp"]').querySelector('#visualizationTabs input:checked').value then set widgetJSON.title to #{'${widgetTitleInputId}'}.value then if not widgetJSON.id gridStackInstance.removeWidget('#add_a_widget_label', true, false) @@ -1436,6 +1447,7 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW $ [ type_ "radio" , value_ tabName , class_ $ "hidden page-drawer-tab-" <> T.toLower tabName + , id_ $ widPrefix <> "-tab-" <> T.toLower tabName , name_ $ wid <> "-drawer-tab" ] <> [checked_ | isActive] @@ -1483,10 +1495,9 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW , source = Nothing , targetSpan = Nothing , query = widgetToUse.rawQuery <|> widgetToUse.query - , vizType = Just $ case widgetToUse.wType of - Widget.WTTimeseriesLine -> "timeseries_line" - Widget.WTLogs -> "logs" - _ -> "timeseries" + , -- Non-chart types (table, stat, …) match no viz tab, so none is checked + -- and the widget keeps its own type instead of saving as a bar chart. + vizType = if isNewWidget then Nothing else Just $ toText $ encodeEnumSC @"WT" widgetToUse.wType , updateUrl = False , targetWidgetPreview = Just widgetPreviewId , alert = False @@ -1495,6 +1506,9 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW , mobileExtra = Nothing , parseError = Nothing } + unless (isNewWidget || maybe True (T.null . T.strip) widgetToUse.query) + $ label_ [Lucid.for_ $ widPrefix <> "-tab-monitors", class_ "self-end cursor-pointer text-xs text-textBrand hover:underline"] + $ if isJust monitorM then "Edit monitor" else "Create monitor" details_ [class_ "text-xs text-textWeak"] do summary_ [class_ "cursor-pointer select-none transition-colors hover:text-textStrong"] "Show generated SQL" div_ @@ -1542,7 +1556,6 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW , term "_" [text| on 'update-widget-query' - set widgetJSON.type to me.closest('[class~="group/wgtexp"]').querySelector('#visualizationTabs input:checked').value then set widgetJSON.query to event.detail.value then set widgetJSON.title to #{'${widgetTitleInputId}'}.value then trigger 'update-widget' on me |] @@ -1550,54 +1563,59 @@ widgetViewerEditor_ pid paymentPlan dashboardIdM tabSlugM currentRange existingW Components.chartSkeleton_ script_ [text| var widgetJSON = ${widgetJSON}; - widgetJSON.type = document.getElementById('${widgetPreviewId}')?.closest('[class~="group/wgtexp"]')?.querySelector('#visualizationTabs input:checked')?.value || widgetJSON.type; |] + widgetJSON.type = document.getElementById('${widgetPreviewId}')?.closest('[class~="group/wgtexp"]')?.querySelector('#visualizationTabs input:checked')?.value || widgetJSON.type; + { const p = new URLSearchParams(location.search), since = p.get('since'), from = p.get('from'), to = p.get('to'); + if (since || (from && to)) window.updateTimePicker({since, from, to}, {targetPr: 'widget', skipSetParams: true}); } |] -- Alerts tab content unless isNewWidget do let alertFormId = widPrefix <> "-alert-form" alertEndpoint = flip foldMap dashboardIdM \dashId -> "/p/" <> pid.toText <> "/widgets/" <> sourceWid <> "/alert?dashboard_id=" <> dashId.toText - div_ [class_ "group/walert hidden group-has-[.page-drawer-tab-monitors:checked]/wgtexp:block mt-6"] do - let hasAlert = isJust monitorM - defaultTitle = maybe (fromMaybe "Widget Alert" widgetToUse.title <> " - Threshold Alert") (.alertConfig.title) monitorM - -- Enable Alert toggle - label_ [class_ "flex items-center justify-between p-4 bg-fillWeaker rounded-xl border border-strokeWeak cursor-pointer mb-4"] do - div_ [] do - h4_ [class_ "font-medium text-textStrong"] "Enable Alert" - p_ [class_ "text-xs text-textWeak"] "Get notified when this widget's value crosses thresholds" - input_ $ [type_ "checkbox", name_ "alertEnabled", form_ alertFormId, class_ "toggle toggle-primary alert-enable"] <> [checked_ | hasAlert] - form_ - [ id_ alertFormId - , hxPost_ alertEndpoint - , hxSwap_ "none" - , hxTrigger_ "submit" - , hxVals_ $ "js:{teams: window.getTagValues('#" <> alertFormId <> "-teams')}" - , class_ "flex flex-col gap-3 hidden group-has-[.alert-enable:checked]/walert:flex" - ] - do - input_ [type_ "hidden", name_ "widgetId", value_ sourceWid] - input_ [type_ "hidden", name_ "query", value_ $ fromMaybe "" widgetToUse.query] - input_ [type_ "hidden", name_ "vizType", value_ $ case widgetToUse.wType of Widget.WTTimeseriesLine -> "timeseries_line"; _ -> "timeseries"] + div_ [class_ "group/walert hidden group-has-[.page-drawer-tab-monitors:checked]/wgtexp:block mt-6"] + $ if maybe True (T.null . T.strip) widgetToUse.query + then emptyState_ def{icon = Just "bell", size = ESCompact} "This widget can't be monitored" "Monitors evaluate a KQL query, and this widget is built from raw SQL." + else do + let hasAlert = isJust monitorM + defaultTitle = maybe (fromMaybe "Widget Alert" widgetToUse.title <> " - Threshold Alert") (.alertConfig.title) monitorM + -- Enable Alert toggle + label_ [class_ "flex items-center justify-between p-4 bg-fillWeaker rounded-xl border border-strokeWeak cursor-pointer mb-4"] do + div_ [] do + h4_ [class_ "font-medium text-textStrong"] "Enable Alert" + p_ [class_ "text-xs text-textWeak"] "Get notified when this widget's value crosses thresholds" + input_ $ [type_ "checkbox", name_ "alertEnabled", form_ alertFormId, class_ "toggle toggle-primary alert-enable"] <> [checked_ | hasAlert] + form_ + [ id_ alertFormId + , hxPost_ alertEndpoint + , hxSwap_ "none" + , hxTrigger_ "submit" + , hxVals_ $ "js:{teams: window.getTagValues('#" <> alertFormId <> "-teams')}" + , class_ "flex flex-col gap-3 hidden group-has-[.alert-enable:checked]/walert:flex" + ] + do + input_ [type_ "hidden", name_ "widgetId", value_ sourceWid] + input_ [type_ "hidden", name_ "query", value_ $ fromMaybe "" widgetToUse.query] + input_ [type_ "hidden", name_ "vizType", value_ $ case widgetToUse.wType of Widget.WTTimeseriesLine -> "timeseries_line"; _ -> "timeseries"] - Components.formField_ Components.FieldSm def{Components.value = defaultTitle, Components.placeholder = "e.g. High error rate monitor"} "Name" "title" True Nothing - -- Monitor Schedule section (shared component) - Alerts.monitorScheduleSection_ paymentPlan (maybe 5 (.checkIntervalMins) monitorM) (maybe 5 (.timeWindowMins) monitorM) (Just "threshold_exceeded") - -- Thresholds section (shared component) - Alerts.thresholdsSection_ ((monitorM >>= (.alertConfig.unit)) <|> widgetToUse.unit) (Just wid) (((.alertThreshold) <$> monitorM) <|> widgetToUse.alertThreshold) ((monitorM >>= (.warningThreshold)) <|> widgetToUse.warningThreshold) (maybe False (.triggerLessThan) monitorM) (monitorM >>= (.alertRecoveryThreshold)) (monitorM >>= (.warningRecoveryThreshold)) - -- Widget-specific: Show threshold lines option - let currentLines = fromMaybe "always" widgetToUse.showThresholdLines - div_ [class_ "bg-bgBase rounded-xl border border-strokeWeak p-3"] - $ Components.formSelectField_ Components.FieldSm "Show threshold lines on chart" "showThresholdLines" False - $ forM_ ([("always", "Always"), ("on_breach", "Only when breached"), ("never", "Never")] :: [(Text, Text)]) \(v, lbl) -> - option_ ([value_ v] <> [selected_ "" | v == currentLines]) $ toHtml lbl + Components.formField_ Components.FieldSm def{Components.value = defaultTitle, Components.placeholder = "e.g. High error rate monitor"} "Name" "title" True Nothing + -- Monitor Schedule section (shared component) + Alerts.monitorScheduleSection_ paymentPlan (maybe 5 (.checkIntervalMins) monitorM) (maybe 5 (.timeWindowMins) monitorM) (Just "threshold_exceeded") + -- Thresholds section (shared component) + Alerts.thresholdsSection_ ((monitorM >>= (.alertConfig.unit)) <|> widgetToUse.unit) (Just wid) (((.alertThreshold) <$> monitorM) <|> widgetToUse.alertThreshold) ((monitorM >>= (.warningThreshold)) <|> widgetToUse.warningThreshold) (maybe False (.triggerLessThan) monitorM) (monitorM >>= (.alertRecoveryThreshold)) (monitorM >>= (.warningRecoveryThreshold)) + -- Widget-specific: Show threshold lines option + let currentLines = fromMaybe "always" widgetToUse.showThresholdLines + div_ [class_ "bg-bgBase rounded-xl border border-strokeWeak p-3"] + $ Components.formSelectField_ Components.FieldSm "Show threshold lines on chart" "showThresholdLines" False + $ forM_ ([("always", "Always"), ("on_breach", "Only when breached"), ("never", "Never")] :: [(Text, Text)]) \(v, lbl) -> + option_ ([value_ v] <> [selected_ "" | v == currentLines]) $ toHtml lbl - Alerts.notificationSettingsSection_ ((.alertConfig.severity) <$> monitorM) ((.alertConfig.subject) <$> monitorM) ((.alertConfig.message) <$> monitorM) (maybe True (.alertConfig.emailAll) monitorM) teams (maybe V.empty (.teams) monitorM) alertFormId monitorM + Alerts.notificationSettingsSection_ ((.alertConfig.severity) <$> monitorM) ((.alertConfig.subject) <$> monitorM) ((.alertConfig.message) <$> monitorM) (maybe True (.alertConfig.emailAll) monitorM) teams (maybe V.empty (.teams) monitorM) alertFormId monitorM - -- Action buttons - div_ [class_ "flex items-center justify-end gap-2 pt-4 pb-20 mt-4 border-t border-strokeWeak"] do - when hasAlert $ button_ [type_ "button", class_ "btn btn-ghost btn-sm", hxDelete_ alertEndpoint, hxSwap_ "none"] "Remove monitor" - primaryButton_ [type_ "submit"] do - faSprite_ "plus" "regular" "w-3.5 h-3.5" - if hasAlert then "Update monitor" else "Create monitor" + -- Action buttons + div_ [class_ "flex items-center justify-end gap-2 pt-4 pb-20 mt-4 border-t border-strokeWeak"] do + when hasAlert $ button_ [type_ "button", class_ "btn btn-ghost btn-sm", hxDelete_ alertEndpoint, hxSwap_ "none"] "Remove monitor" + primaryButton_ [type_ "submit"] do + faSprite_ "plus" "regular" "w-3.5 h-3.5" + if hasAlert then "Update monitor" else "Create monitor" -------------------------------------------------------------------- @@ -1641,6 +1659,11 @@ widgetAlertUpsertH pid _widgetIdPath dashboardIdM form = do (session, _) <- Projects.sessionAndProject pid now <- Time.currentTime + -- An empty query compiles to a count over all traffic, not the widget's series. + when (isJust form.alertEnabled && T.null (T.strip form.query)) do + addErrorToast "This widget can't be monitored" (Just "Monitors need a KQL query; this widget is built from raw SQL.") + throwError err400{errBody = "Widget monitors need a KQL query"} + -- Reuse the widget's existing monitor id when there is one queryMonitorId <- Monitors.queryMonitorByWidgetId pid dashboardIdM form.widgetId @@ -1651,7 +1674,7 @@ widgetAlertUpsertH pid _widgetIdPath dashboardIdM form = do let dashboardId = UUIDId dashId (_, dash) <- getDashAndVM pid dashboardId Nothing let updateWidget w = if widgetMatches form.widgetId w then w{Widget.id = Just form.widgetId, Widget.showThresholdLines = form.showThresholdLines, Widget.unit = mfilter (not . T.null) $ T.strip <$> form.unit} else w - dash' = dash & #widgets %~ map updateWidget & #tabs %~ fmap (map (\t -> t & #widgets %~ map updateWidget)) + dash' = dash & #widgets %~ overWidgetTree updateWidget & #tabs %~ fmap (map (#widgets %~ overWidgetTree updateWidget)) void $ Dashboards.updateSchema dashboardId dash' Nothing -- If alertEnabled is not checked, delete the monitor @@ -1744,19 +1767,35 @@ instance ToHtml DashboardsGet where toHtmlRaw = toHtml -renderDashboardListItem :: Bool -> Text -> Text -> Maybe Text -> Maybe Text -> Maybe Text -> Html () -renderDashboardListItem checked title value description icon prview = label_ +renderDashboardListItem :: Bool -> Text -> Text -> Maybe Text -> Maybe Text -> Html () +renderDashboardListItem checked title value description icon = label_ [ class_ - [text| cursor-pointer group/it text-sm border border-transparent hover:bg-fillWeaker hover:border-strokeWeak rounded-lg flex p-1.5 gap-2 items-center + [text| cursor-pointer group/it text-sm border border-transparent hover:bg-fillWeaker hover:border-strokeWeak rounded-lg flex p-1.5 gap-2 items-center group-has-[input:focus-visible]/it:outline-2 group-has-[input:focus-visible]/it:outline-offset-2 group-has-[input:checked]/it:bg-fillWeaker group-has-[input:checked]/it:border-strokeWeak dashboardListItem|] , term "data-title" title , term "data-description" $ maybeToMonoid description - , term "data-preview" $ fromMaybe "/public/assets/svgs/screens/dashboard_blank.svg" prview - , term "hx-on:mouseover" "dItemPreview.src = this.dataset.preview; dItemTitle.innerText = this.dataset.title; dItemDescription.innerText = this.dataset.description" - , term "hx-on:mouseout" "const c = document.querySelector('.dashboardListItem:has(input:checked)'); if (c) { dItemPreview.src = c.dataset.preview; dItemTitle.innerText = c.dataset.title; dItemDescription.innerText = c.dataset.description }" ] do - input_ $ [class_ "hidden", type_ "radio", name_ "file", value_ value] <> [checked_ | checked] + input_ + $ [ class_ "sr-only" + , type_ "radio" + , name_ "file" + , value_ value + , [__| + on change + set :item to closest