mirror of
https://github.com/monoscope-tech/monoscope.git
synced 2026-11-16 10:18:01 +00:00
15 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
72a7b4b47e |
repo-lint check: actionlint over the workflows and hpack-check over monoscope.cabal (#866)
Nothing parsed the workflows (#859 broke Deploy with an expression no check read), and CI regenerates monoscope.cabal before building, so a stale committed file (#858) passed every gate. |
||
|
|
d5671d62ee |
Unify autonomous agents across Slack, web and native jobs (#619)
Slack, web conversations, routines and native review/fix jobs share the capability catalog, current authority checks, run ledger and operation receipts. Runs, charts and saved reports open inside the conversation inspector. Consolidates duplicate regression suites and documentation while retaining the product contracts, native recovery evidence and migration baseline transition. Validation: all eleven exact-input local checks pass, including 1,216 integration examples (25 pending VM cases), 2,105 doctests, 315 unit examples, 16 CLI examples, 1,041 UI tests and 172 browser tests (one skip). Separate native/KVM acceptance is recorded in the review. The exact Linux AMD64 image was built locally on this Mac with OrbStack/Rosetta. |
||
|
|
f3a3a9ce89 |
Add RUM audience breakdowns and reusable segments (#740)
* Highlight the selected visualization tab with brand colors * Add RUM audience breakdowns, investigation filters, and saved segments * Declare form decoding dependency for integration tests * Check expanded RUM templates for query scoping * Supply required summaries in RUM browser fixtures * Prepare shared npm dependencies before parallel CI lanes * Target the segment disclosure explicitly in browser coverage |
||
|
|
8890344e96 |
Share local CI caches across checkouts (#711)
* Share local CI dependency caches across checkouts Each checkout path got its own cabal/npm/playwright volume set, so every worktree and signoff clone leaked ~8GB of content-keyed caches; 35 sets had piled up. Share those. dist-newstyle stays per checkout: sharing it was tried and weeder then reported live functions as dead, because unchanged modules lost their .hie files after another branch built into the volume. * Note that local sign-off must stay fast * Share dist-newstyle across checkouts; weeder repairs missing .hie files After another branch built into a shared dist, GHC can leave a live module compiled without its .hie, so weeder misses every reference it makes and reports live functions dead. The weeder check now recompiles any tracked module missing one (dropping cabal's build stamp, since a touch leaves the source hash unchanged) instead of every checkout paying a cold build. |
||
|
|
d8b3bf853a |
Add project wiki with revision history and permission checks (#689)
* Add project wiki with revision history and permission checks * Address wiki review and disclose bounded lists * Use typed wiki pages and shared morph navigation |
||
|
|
ffd46239ad |
Faster local sign-off: build the deploy image alongside the checks
- make ci-signoff builds the deploy image in parallel with the checks when the tree is committed, so make ship only has to deploy. - Deploy images build on this machine by default; the SSH builder (the production host) is opt-in via MONOSCOPE_BUILDER. - Local builds skip the registry cache export (minutes per image); GitHub runners, which start without a cache, keep it. - Local e2e runs with more Playwright workers (E2E_WORKERS, default 6); GitHub keeps 2. - Doctests join integration and e2e as background lanes after build, running the prebuilt binary; release-tests run on the host meanwhile. - CI Postgres runs without fsync: each example's CREATE DATABASE drops ~1s. - Dockerfile: GIT_HASH is declared after the runtime apt layer (it rebuilt that layer, ~2 min emulated, on every commit), and COPY --chown replaces a chown -R layer that duplicated 170MB into every push. - e2e: the template-modal layout check measures after the open transition. Signoff 5.5 min -> under 3 min with the image built in parallel. |
||
|
|
2d6b08019b |
Reuse PR production images and reject stale deployments (#651)
* Reuse PR production images and reject stale deployments * Use existing Node tooling for release regression tests * Cover digest deployments and explicit rollbacks * Seed Cabal cache mounts from prebuilt dependency image * Wait for dashboard tab arrival before renaming widgets * Use Docker digest output and clarify deployment guards * Preserve registry lookup diagnostics during image builds * Dismiss onboarding notices before switching framework guides |
||
|
|
5bee041d4c |
Widget: stat tiles skip the no-data overlay (#608)
* Speed up local CI signoff with parallel builds and browser tests * Widget: stat tiles skip the no-data overlay Inside a 7rem stat tile the two-line "No data in this time range" overlay overflowed and was clipped under the value (production RUM, Browser errors at 0). The value already says 0; the overlay is for chart panels. |
||
|
|
37d5a59513 |
Improve RUM, sessions and infrastructure: performance, correctness and live updates (#604)
* Fix RUM sessions, cache concurrency, and infrastructure drilldowns * Exercise RUM guards against native cache and independent fixtures * Preserve session list focus and scroll on live refresh * Cache explicit empty RUM searches briefly * Verify fresh empty RUM searches reuse cached results * Fix RUM page links and preserve Explorer regex queries * Preserve container identity through inventory and drilldowns Keep namespace, cluster, node and typed scope through inventory grouping, drawer selection, charts and log links. Preserve time ranges and show recovery for ambiguous legacy links. Regression reproduced collapsed rows and cross-container chart values before fixing. All 13 Containers examples pass on PostgreSQL and fresh native TimeFusion; browser guards pass at 390 and 1280px. Native chart tests use a bracketed explicit test pool because the existing test harness keeps its postgres-simple TF pool on PostgreSQL. * Preserve RUM session search URLs and service scope * Address cache review and verify CPU charts through the selected backend * Revalidate stale RUM session lists inside their swap target * Expose container identity in narrow table rows Show full pod, namespace, cluster and node labels beneath container names when the corresponding columns collapse. Wrap UUIDs while preserving the existing desktop columns. Retained e2e regression covers five same-name identities, including replicas on the same node. Missing namespace and Pod checks failed before the fixes; normal deferred navigation, full metadata visibility and viewport fit now pass at 390 and 1280px. All 13 focused Containers examples pass; formatter, Hlint and review loops are clean. * Count distinct RUM sessions across the full time window * Use exposed Servant API parser in container regression * Preserve native JSON predicates and invalidate stale compiled filters * Keep loaded RUM summary values visible during live refresh * Keep Kubernetes container drilldowns within their cluster * Keep error lifecycle fixture clocks and job stages ordered * Keep session form windows aligned with the time picker * Keep container inventory focused on finding entities * fix(rum): refresh panels with the current picker window * fix(rum): replace initial panel requests when the window changes * test(kql): round-trip escaped quoted regex patterns * Show singleton observations in line charts * Preserve inventory facets when changing filters * Keep inventory search accessible on mobile * Compute RUM web vitals from observation populations Share scalar and explicit-histogram populations across field, trend and exact URL views. Preserve cumulative baselines, coverage, scalar atoms and metric siblings when capping pages. Validated: native handler 9/0, PostgreSQL RUM 28/0, formatting and HLint, two review rounds. Styled browser proof remains pending after capture harness timeouts; local doctests failed before examples because the monoscope package was not registered. Combined CI must verify codecs and integrated behavior. * Keep primary container CPU visible with mobile filters * Fix mixed vital atoms above finite histogram bounds * Keep vital trend axes and coverage accurate * Accept optional starts for delta vital populations * Keep collector container CPU visible on mobile * Preserve explicit RUM windows when since is absent * Reuse RUM cache entries for blank window fields * Match container summary to its selected time slice * Use singular labels for individual vital observations * Reuse RUM populations across equivalent explicit window requests * Keep RUM vital observations and coverage visible on phones * Keep partial vital trend buckets inside the selected axis * test(rum): check singleton point within short time axis Extend the existing axis browser guard with a fixed 30-second window and actual plotted point containment, scrolling the lazy chart into view. Browser proof remains pending authoritative CI: isolated WAI attempts failed before the existing 1H axis check, so they do not reproduce clipping. The b4 handler regression remains verified. * Refresh container inventory when its time window changes * fix(time-picker): apply automatic refresh menu choices Use native querySelector in the existing hyperscript click handler; keep the hx-live proxy bindings unchanged. The browser regression reproduces Off retaining interval 15000, then verifies default, Off, 30s and 15s states and selection at mobile and desktop widths after the fix. Focused browser: 1 passed (4.0s); formatting and lint passed. Full CI remains pending parent verification. * Load RUM panels before application globals initialize * Refresh infrastructure inventory when its time window changes * Keep page vital measurements readable on mobile * Keep infrastructure refresh hyperscript covered by parser checks * Preserve refreshed chart registry through HTMX settling * Preserve captured error identity in regression lifecycle fixture * Keep initial RUM requests alive during automatic ticks * Await infrastructure fixture visibility through shared snapshot cache * Preserve pending RUM updates and remove redundant prewarming * Preserve ownership of replacement session searches * Avoid a separate epoch read for noncumulative RUM vitals * Scope RUM query-count guard to its handler * Load selected session details independently of the list * Send RUM summary widgets as SQL so TimeFusion serves them from rollups The four summary stats and the activity chart now send the exact browserScope, pageViewPredicate and errorPredicate SQL that TimeFusion's rum_* rollup measures declare. KQL's lowering (startswith as a regex, exception.type != null as a JSON search) could never match them. The chart becomes one aggregate unpivoted to two series; a page view that errored now counts in both. * RUM improvements * Auto-format code with fourmolu * Simplify RUM population queries and type query failures * Harden RUM vital SQL and consolidate regex parsing * improve signoff * Consolidate repeated RUM, infrastructure and test logic - Derive SQL encoding for VitalCoverage/VitalReadKind instead of 12 label bindings - panelSwapAttrs owns the panel URL and hx-sync; share timeWindowVals_ with Components - One kqlFilter plus node/cluster fields for container, host and Kubernetes pivots - linkParams/renderedWidget test helpers; shared histogram and panel-loading builders * Cache npm downloads and e2e node_modules in local CI * Auto-format code with fourmolu * RUM sessions list: paint the newest slice first and bound the follow-up scans A cold session list over a wide window scanned the whole window three times in sequence (core, enrichment by id, recording cross-lookup); the panel took 22-97s on a busy project. Wide cold lists now paint the newest three hours (1.7s) and revalidate the full range through the stale-serve path; the follow-up scans are bounded to the rows already found; the recording cross-lookup runs only for text searches. * RUM: morph live-tick panel refreshes, keep the pulse widgets across revalidation, one empty state for missing Web Vitals Every panel replaced itself with outerHTML on the 15s tick, and a stale pulse panel re-rendered its four self-fetching widgets on revalidation — a 2-10s spinner flash on first load. Rendered panels now morph; a populated pulse skips revalidation. A project that reports no Web Vitals gets one explanation on the Performance tab instead of five No-data rows and two empty panels. * Infrastructure snapshot: reduce to the newest datapoint per series before the blob lookups The pivot evaluated ~20 resource/attributes JSON lookups on every datapoint in the 15-minute slice (49.9K rows on the demo project) although only the newest row per series survives. A first window over series_id keeps those survivors, so the lookups run over ~750 rows: projection 1.37s -> 40ms, cold infra pages 2.4-3.5s -> 0.85-0.92s. * Charts: update a live echarts instance in place instead of dispose+init; stop the 401 reload loop A morphed panel keeps its chart element, but every re-init disposed the instance and drew it again, which blanked all five Web Vitals charts on each 15s tick and cleared the stat tiles' values while their first fetch was in flight. chartWidget now updates a live instance in place (hx-morph-skip on the slot), the server-rendered stat value stays until the response replaces it, and a 401 from /chart_data is verified once before reloading, at most once a minute, instead of reloading the page on every occurrence. * RUM: local times, a second signal on rated numbers, family hues off the status hues, widen action on the empty vitals state Top pages and browser errors printed UTC beside a header showing the viewer's zone; they render through local-time now. P75 cells carried their rating by colour only and get a dot plus a screen-reader label. Audience chips used the warning, success and error hues for Linux, Android and Firefox. The sessions workspace gives the last-page column room to show the distinguishing tail. The empty Web Vitals state's primary action now does what its copy advises. * RUM: never cache the newest-3h slice; clamp span pads to the window; release charts a morph left detached A cached slice would be served as the final list whenever the full revalidation failed. The recording pad could reach outside the selected range. Charts kept alive across a morph were never disposed when the fragment that held them left with a tab switch. * RUM: spanScope doctest builds its scope from the module's own imports * RUM: show the refreshing hint at the top of a provisional session list * e2e: a refreshed vital chart keeps its instance on its element * Infrastructure pages: denser host list, honest drawer empty state, one live region per table, status color only for status Integration chips no longer wrap into 65px rows or clip the last column, and the empty Configuration column appears only when a host reports a provider or region. The host drawer's metrics empty state reloads the drawer at one hour instead of navigating to the host map, and its copy states the 15-minute freshness slice. Group-by and resource controls drop their duplicate count live region. Brand and success fills used as decoration (drawer header, count pill, Ready badges) become plain text with a dot; the one Not ready badge stands alone. Pod names and workload badges truncate at the end. CPU renders as cores to three decimals on every list. * RUM: rated cells keep their coverage tooltip; pin that a failed revalidation never serves the 3h slice as final * e2e: the host drawer's one-hour retry is a button * Charts: one sweep for registrations whose element is gone, on after:swap and after:settle An outerMorph settles without firing after:swap, so charts kept alive across a morph and then removed with the fragment were never released; the two sweeps are now one. * RUM: refreshing hint on the Overview's recent sessions too; the session list fits a phone The Overview's recent-sessions panel is served from the same newest-3h slice and now says so. At phone width the list dropped its Duration column behind a 28rem minimum; the minimum applies from sm up and the Activity counts hide below it. * Speed up local CI signoff with cache reuse and serialized runners * Keep RUM vital reads and session cache behavior correct * Auto-format code with fourmolu * Handle empty CI mounts and align host layout assertion * Infrastructure: drawer retry reuses hostDetailUrl, empty state names the real freshness slice, wrapper divs folded * Monitors: drop an unused import that fails -Werror * RUM replay workspace: fact row in the header for every session, selected row scrolls into view, duration column fits a phone The header showed only avatar, name and id for a session with a recording; duration, page views, events, errors, last page and service now sit in one fact row for every session. A shared link's selected row sat thousands of pixels down an unscrolled list; it scrolls into view once on init and stays put on live ticks. * Session replay player: fit the frame to the recording's aspect ratio, one-row compact toolbar Full-width embeds used a fixed 550px frame, so a wide recording sat between black bands (280px at 1440, 367px on a phone). The frame now follows the recording's aspect ratio, clamped to the viewport. Below 760px of player width the toolbar keeps one row: icon buttons with labels in tooltips, and the identity no longer truncates. * RUM: Web Vitals coverage notes say what the export did and what fixes it * RUM: the Overview's recent sessions are the Sessions tab's list, one cache entry for both The Overview ran its own span and recording queries under separate cache keys, so visiting both tabs scanned the window twice. Both now read the unfiltered session search, which already merges recordings; the two Overview-only query variants are gone. * RUM panel cache: serve stale entries for twelve hours instead of one The first visit of the morning paid a cold 24h scan because last night's entries were past the one-hour horizon. Entries now serve stale and revalidate for twelve hours; the on-write prune keeps the same bound. * Infrastructure images: registry names stay on one line and tag pills wrap instead of clipping * RUM: trim three comments the review pass found restating the code * local-time: re-localize after a morph swap A morph patches the element's text back to the server's UTC fallback without touching its attributes or reconnecting it, so every live tick reverted the page's localized timestamps. * RUM Overview: one pulse scan serves presence, the session count and the P75 tile The presence probe was a LIMIT 1 while the four stat tiles each fetched their own number, so the tiles spun 2–10s after the panel and never shared its cache. The probe is now one aggregate over the window; the session and P75 tiles render with the panel (and are cached with it), and the two sparkline tiles keep their own fetch, whose series is the same request as their number. * RUM pulse: no row means no telemetry; the overview test pins the served tile values * RUM Overview: all four headline tiles render from the pulse scan A timeseries stat tile now shows a server-supplied value until its sparkline response brings the stats, so page views and browser errors arrive with the panel like sessions and P75. * Log explorer sessions: one grouped scan plus a page-bounded context read The Sessions viz referenced a wide CTE four times (DataFusion inlines each reference) with body::text in the projection: 54s on the demo project over 24h. Now one scan per session key with only rollup-servable aggregates (any per-group distinct count forces the raw store: 7s vs 0.15s), sort, paging and the header summary in Haskell, then landing url / user agent / first error / services for the page only, bounded to the page's own span. The header's service count is a GROUP BY service read on the first page. traceCount is gone from SessionRow; the [+N] chip counts events, which is what the expand endpoint returns, and grows with its count. sessionSortColumn was left with only its doctest and is deleted. Demo project, 24h: 54s -> 7-28s (scan cold vs warm), load-more 7.6s, 3h 2s. * Log explorer sessions: a live poll skips the header and its service read The client's 15s newer-poll sends cursor=… without a window; the sessions route ignored it, so every tick re-ran the header summary and the service read on top of the aggregation. The route now takes cursor and a poll gets rows only; fetchSessions returns Maybe SessionSummary gated in one place. Poll 1.4s -> ~1.0s on the demo project. Also from the second review pass: uniqueServices pinned in Test 11.1, an orphaned Default deriving dropped, the row copy via RecordWildCards. * Log explorer sessions: run the header's service read under the grouped scan On projects whose sessions span the whole day the page context read is a window scan too, so the view was three raw TimeFusion scans in sequence (7s + 5s + 10s on the main project). The service read is independent of the scan, so it forks alongside it in a ki scope: 24h view 31.6s -> 16-19s. * log-list: a live tick during the initial load no longer re-runs it With the sessions viz on a big project the initial load took 24s, so the 15s tick fired before any row existed; without a cursor it issued a second full aggregation (15s more TimeFusion time) for nothing. A tick while the initial or refresh load is in flight is dropped. * Log explorer sessions: retry transient TimeFusion errors like the data read does Two concurrent raw scans from one request is the burst shape that trips TimeFusion's transient resource errors, and the sessions handler was the one explorer read without retryTransientEff around it. * Apply the branch review: typed regex lowering, versioned RUM cache keys, explicit vital coverage rank, keyboard-reachable session actions hs-distill / hs-evasion-review / hs-lob-review over the branch, applied: - Expr: CaseMode reaches the jsonpath lowering as a type, not a string op; compileRegex lives in evalExpr; seedColumns is one modifyIORef'. - RUM: PulseQuery + queryVersion/rumCacheDbKey replace per-constructor key suffixes in the handler; VitalEstimate is a record with optional bounds; VitalCoverage priority is an explicit rank, not spelling order; interpTimefusion folds the pool routing into one call. - Widget: hx-morph-skip is static markup; a served stand-in lives in its own servedValue field with a null source so the sparkline still loads. - Components: the table live-tick listener fires on after:settle (an outerMorph never fires after:swap); timeWindowVals_ uses window.params(). - Explorer sessions: SessionsRead sum type; the [+N] chip and Replay run on click so Enter works; the list header no longer prints a page-local event count as the total; histogram bars carry aria-labels; durations roll into min/h/d; URLs cap at 40ch. - Tests: deferredBody/shellHtml shared in TestUtils; RUM spec fixtures deduped; doctests for routeKql, VitalEstimate, getDurationNSMS. * Auto-format code with fourmolu * Utils: integer literals in the getDurationNSMS doctest * Shells read the query string inline again; a served stat value wins for any tile type Two e2e contracts the review pass broke: deferred shells must resolve before the application script arrives (window.params is not there yet), and the Sessions / P75 tiles are plain stats whose served value the type-selected fallback skipped, leaving the tiles blank. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
|
bf5d4a670a |
Lint the shared and CLI packages too, and fix what that turned up
CI ran `hlint src/`, so monoscope-shared and monoscope-cli — 22 modules, including the KQL parser and the binary we ship to users — were never linted. shared/src was already clean; the CLI had eight issues, one of them error severity. All mechanical, all Relude preferences the rest of the codebase already follows: T.pack over a String where toText does, T.unwords/T.unlines where Relude's are in scope, the Data.Set alias the hlint config pins, and a lambda that is a <=< . cabal test monoscope-cli:cli-tests passes 16/16 and the binary links. |
||
|
|
3fde4bba24 |
Fingerprint the frontend checks on the Haskell they actually read
ui-tests and frontend declared @fe, which is web-components plus tailwind config. But
the hyperscript guard parses every [__|…|] quasiquote in src/, issues-highlight-theme
reads a script out of Pages/Issues.hs, tailwind scans src/**/*.hs for class names, and
two specs load the vendored htmx builds. A Haskell-only change therefore left both
fingerprints unchanged, the gate reused an older pass, and those guards never ran —
which is how a test pointing at a file deleted in
|
||
|
|
3bf1388abd |
Ship the whole deploy locally, so CI stops being the critical path
CI's image build takes ~71 minutes; the same image built natively on an amd64 box takes ~36 cold, and far less warm. Waiting on a remote queue for the slow half of a deploy is what makes "push it and see what CI says" tempting, which is how untested work reaches master. `make ship` does all four steps here: checks (attesting each pass), build and push the image, push the commit, deploy to CapRover. The order is deliberate — checks first because everything after is expensive, image before push so CI's probe skips its own build, push before deploy because production must run a commit that exists on origin. It refuses to deploy a commit origin/master does not contain, and refuses when any deploy-path check lacks an attestation for this exact tree. weeder and hlint failures do not block it: they gate pull requests, not deploys. `make builder-setup` points buildx at a native amd64 host over SSH, capped so a -j48 GHC build cannot starve whatever else that machine runs. Without one the build falls back to the emulated local one, and a stale builder name falls back too rather than failing a deploy. The build now shares CI's registry cache in both directions, so a laptop build no longer starts cold. `make tf-image` builds TimeFusion for this architecture, which is the only reason integration-tests could not run on a Mac; ci.sh detects the result with no exported variables. Its image is distroless, so the compose healthcheck could never pass — readiness is now probed on the pgwire port from the runner, which also proves the thing the check needs rather than that a container exists. CI's deploy job asks `ci.sh deployed <sha>` and skips a commit already deployed from a machine, so this cannot restart production twice with the same image. Deploy credentials are env-only, via the per-app deploy token rather than the admin password. |
||
|
|
55ad13e747 | Render failed chart queries explicitly without caching them | ||
|
|
c2860941b2 |
Put the e2e suite in CI, and fix make ci resolving its root to /
`make ci` has been broken since
|
||
|
|
f8514ee295 |
ci: let anyone run CI, and let CI skip what was already proven (#491)
CI re-runs work that has already been done: a PR run proves a tree, the squash merge keeps that exact tree, and master builds and tests all of it again. On 4-vCPU runners that is ~20 minutes per push, and none of that capacity is available to a developer whose laptop is faster than the runner. ci/checks.tsv is now the one definition of what CI is; the workflows and scripts/ci/ci.sh both read it. Each check is fingerprinted over the git blob hashes of its declared inputs, and a pass publishes an attestation as a ref: refs/ci-attest/v1/<check>/<fingerprint>/<platform>/<capabilities>/<date> A gate job resolves those with one ls-remote and no object fetches, and skips any check already proven for the tree it is about to test — whether by an earlier run, a re-run, or `make ci` on someone's machine. Producing one needs push access, the same trust boundary as pushing code; forks can neither create nor be affected by them. Reuse is deliberately conservative because this gates a production deploy. Checks declare the capabilities they need and attestations record the ones the environment had, so a run that fell back to a stub service can never satisfy a check that needs the real one. Fingerprints are pinned from the gate, so steps that rewrite the tree (hpack, npm ci, hlint --refactor) cannot desync what gets attested from what was looked up. The deploy condition is spelled out rather than implied by `needs`: build-and-test may be skipped, but only when the gate says every Haskell check was already proven. Also switches the minio service to bitnamilegacy/minio. The bitnami/minio repository was emptied in the Aug-2025 catalog change and now has no tags at all; CI has only been working off a runner-side image cache, and an eviction would have taken the integration suite down with nothing in the diff to blame. Verified end to end: all six checks ran through ci.sh and published attestations, and a clean checkout on macOS/arm64 then resolved CI's linux-amd64 attestations and skipped the entire Haskell suite. See docs/local-ci.md. |