0
Fork 0
mirror of https://github.com/oblien/openship.git synced 2026-11-16 06:57:12 +00:00
Commit graph

2,735 commits

Author SHA1 Message Date
Hydra
4314eb8daa fix(dashboard): read runtime source GitHub metadata from API contract 2026-11-13 18:19:38 +02:00
Hydra
a1089e12f3 fix: align monthly and yearly pricing across web and dashboard 2026-11-13 17:58:11 +02:00
Hydra
a90fe0904a fix(web): clarify Cloud pricing and included management 2026-11-13 17:27:09 +02:00
Mohammed Mostafa
300045a0e9
feat(plugins): prepare ChatGPT/Codex directory submission (#1746)
- Add openWorldHint to MCP tool annotations with per-route override
- Document readOnly/destructive/openWorld hints in MCP docs
- Add OpenAI review test cases, publication notes, and support/privacy/terms URLs to plugin manifest
- Bump plugin version to 0.1.2
- Add SUBMISSION.md guide for directory draft creation and review setup
2026-11-13 16:40:23 +02:00
Johannes Rueschel
49fd139c81
fix(cli): use Node for container host-channel probe (#1745)
Co-authored-by: chbndrhnns <7534547+chbndrhnns@users.noreply.github.com>
2026-11-13 11:49:11 +02:00
Hydra
5caea3b6d5 test(edge): allow realistic CLI startup when verifying replacement v0.9.12 2026-11-12 22:39:26 +02:00
Hydra
18f12075f7 test(scaling): retain redacted operator diagnostics after database failure 2026-11-12 22:32:13 +02:00
Hydra
3561b6ca0a ci(release): split Docker journeys across four complete shards 2026-11-12 22:23:56 +02:00
Hydra
2d874048aa test(deployments): supply saved environment in inline build fixture 2026-11-12 22:22:26 +02:00
Hydra
0af1d7ec91 test(release): freeze v0.9.11 upgrade baseline 2026-11-12 22:04:43 +02:00
Hydra
a97baa1f0d Bump to v0.9.12 + announce 2026-11-12 21:32:29 +02:00
Hydra
ca4aef9f99 fix(test): isolate worker-local edge upgrade compatibility checks 2026-11-12 21:31:53 +02:00
Hydra
79bb4b895d fix(ci): update build fixtures and finalize v0.9.12 notes 2026-11-12 21:24:23 +02:00
Hydra de lerne
2454fe4509
fix(dashboard): keep destination selection consistent for one server (#1741) 2026-11-12 21:20:36 +02:00
Hydra de lerne
e19711e1c5
feat: add domain virtual responses with a compact editor (#1739)
* feat: add domain virtual responses with a compact editor and bounded edge matching

* fix: keep traffic working during rolling edge script updates

* refactor: centralize edge management transport
2026-11-12 21:20:24 +02:00
Hydra de lerne
739728ce11
fix(migration): preserve project services and reuse domain routing (#1740) 2026-11-12 21:06:52 +02:00
Hydra
d695e84427 chore: pause unpublished v0.9.12 release 2026-11-12 18:42:40 +02:00
Hydra
a174d7dc7b Bump to v0.9.12 + announce 2026-11-12 18:35:04 +02:00
Hydra
c1341e9fc9 fix(migration): keep environment fields visible while editing 2026-11-12 18:34:47 +02:00
Hydra
a18878316c docs: prepare critical v0.9.12 release notes 2026-11-12 18:30:02 +02:00
Hydra de lerne
f8e8829c3a
fix: unify project settings, refine web navigation, and restore CI (#1738)
* chore: keep generated review artifacts out of git

* fix(dashboard): keep single apps in the service settings flow

* fix(web): refine Arabic landing and navigation

* fix(core): align host channel branding with dashboard copy
2026-11-12 18:03:33 +02:00
Michael Ogunjimi
232b51c52d
fix(deploy): keep token clones on https when git config rewrites to ssh (#1736)
Token clones allow only HTTPS (protocol.allow=never, GIT_ALLOW_PROTOCOL=https), but
git still reads the user's global and system config. A common rule such as
url."git@github.com:".insteadOf https://github.com/ turns the https clone URL into
ssh, and git fails with "transport 'ssh' not allowed".

Map the repository URL and its namespace onto themselves in the scoped clone
config. Git applies the longest matching insteadOf, so these outrank a broader
rewrite while the rest of the user's config (proxy, CA bundle) keeps working.

Fixes #1731
2026-11-12 17:16:32 +02:00
Hydra de lerne
47431eb567
update (#1737) 2026-11-12 17:15:42 +02:00
Hydra de lerne
db8a5cf4f4
fix(mail): complete local backups and portable server restores (#1734)
* fix(mail): complete local backups and portable server restores

* test(mail): report engine diagnostics on round-trip failure

* fix(mail): stop Postfix writers before container snapshots

* fix(mail): apply Postfix shutdown in the daemon action renderer

* test(mail): install GNU tar on the remote fixture host
2026-11-12 17:03:11 +02:00
Hydra de lerne
367afdec91
fix: sync dashboard setup, instance imports, and migration pages (#1729)
* fix(import): restore workspace visibility and verified edge status

* fix: complete dashboard setup and migration website updates
2026-11-12 11:10:49 +02:00
Hydra de lerne
0976d9d082
fix(migration): improve platform import setup and cancellation (#1728) 2026-11-12 08:14:23 +02:00
Hydra de lerne
679848601d
fix(db): repair missing workspace and advisory fields on upgrade (#1727) 2026-11-12 08:04:47 +02:00
Saim Shafique
057d65de7a
fix(deploy): let webhook and CLI deploys approve reviewed Compose changes (#1675)
* fix(deploy): let webhooks and CLI accept reviewed Compose changes

Webhook and CLI deploys hit COMPOSE_REVIEW_REQUIRED with no way through the gate, and the refusal hid what changed. Accept the current Compose source explicitly per deploy (--accept-compose-changes, or acceptComposeChanges on the hook), and surface the added/changed/removed services in the CLI error and the webhook delivery record.

* test(contracts): include acceptComposeChanges in deployment schema flags

The Compose review gate added acceptComposeChanges to
CreateDeploymentSchema without updating the contract test that
enumerates the schema's public flags, so the contracts suite failed
on CI. Add the flag to the expected list.

* docs(cli): list --accept-compose-changes in the deploy reference

The new deploy flag was missing from the generated CLI reference,
so docs:check failed with a stale reference. Add the row in
declaration order.

* Retain explicit Compose approval through shared deployment admission

---------

Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 18:57:58 +02:00
chriscrosby
65583e0c56
Accept bounded YAML aliases in GitHub workflow imports (#1661)
Co-authored-by: Chris Crosby <1213497+chriscrosby@users.noreply.github.com>
2026-11-11 18:17:07 +02:00
Rudi
9a20c0b23b
perf(edge): avoid redundant route writes and remote reads (#1660)
* fix(edge): stop trimming edge file content read through exec

Executors resolve command output with `stdout.trim()`, which is right for
command parsers and wrong for file content. Three file reads go through
`exec` + `cat`: the sudo fallback when the login user cannot read a file
(root-owned 0600 vhosts on a non-root SSH login), the container fallback in
`readMaybeInContainer`, and `DockerEdgeExecutor` in container file mode.
Each returned the file without its trailing newline.

That breaks every comparison against the bytes about to be written:
`writeEdgeChallenge`'s "already on disk" check never matched, so each
`serveEdgeChallenge` rewrote the vhost and reloaded the edge, and a
rollback that restored such a snapshot wrote the truncated text back.

Frame the payload between two markers so the trim cannot reach it. The
closing marker is printed only after `cat` succeeds, and unframed output
is an error rather than a short file; anything a login shell prints
before the opening marker is ignored. `exec` itself is unchanged.

(SshExecutor still decodes each output chunk separately, so a multi-byte
UTF-8 character split across chunks can decode wrongly. That predates
this change, affects every command, and vhosts are ASCII; left for a
separate fix.)

Refs #1536

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(edge): skip route writes and reloads that would change nothing

Every route registration rewrote its vhost and sidecar and reloaded the
edge, even when the bytes on disk were already exactly right. On a
deploy `registerResolvedRoutes` registers each hostname and then, when a
verified certificate already exists, `provisionCert` re-registers it
through `activateCert`: two write-and-reload cycles per hostname for no
change. Over SSH on a remote server that is most of the deploy's routing
phase.

`updateConfig` now publishes only the files that differ: a change is
skipped when the file already holds exactly that content as a regular
file with the desired mode, or when it removes a path that is already
gone. The mode comes from a new optional `CommandExecutor.lstat` (SFTP
lstat on SSH, fs.lstat locally and through DockerEdgeExecutor's mount),
which also answers the existence probe, so an unchanged file costs the
same two file operations as before. Where the mode cannot be read,
content is the only evidence, as in `writeEdgeChallenge` today.

Skipping the reload needs more than disk matching: a process that died
between write and reload leaves the edge running something else. The
provider records whether it has seen the edge load what is on disk (a
successful reload sets it; a failed reload or incomplete rollback clears
it), and a transaction that writes nothing still reloads until then.
Every write path to edge config or certificate files clears that record
before it touches disk, so a writer that fails part-way, inside or
outside this transaction, cannot leave it vouching for unloaded files.
Writes bump a generation counter, and a reload only vouches for the
generation it started with; certbot and certificate staging, which run
outside the config lock, bump it again when they finish, so once they
return no reload that overlapped them vouches for what they wrote.

Remote servers get a new provider per deployment (the native local
platform keeps one for the life of the process), so an unchanged deploy
of HTTP-01 hostnames costs one confirming reload instead of two per
hostname. DNS-01 and wildcard hostnames, which the platform activates
directly, still reload once each. That confirming reload also syncs the
container edge's trusted-proxy include, which comes from the API's
environment and so changes only with a restart.

`writeEdgeChallenge` drops its own content-only "already on disk" return
and lets `updateConfig` decide, so it honours the mode and the record
above too. Challenge tokens are served, not loaded, and do not clear the
record.

`activateCert` always reloads: issuance and uploads put new certificate
material at the paths the vhost already names, so the vhost text cannot
show the change. Only `provisionCert`'s reuse of an existing, verified
certificate may skip. Rollback restores only the files this transaction
attempted to write, with their original mode when known; a symlink, even
a dangling one, is replaced by a regular file (and restored as a link
where `readlink` can see it).

Refs #1536

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(edge): stop repeating the same remote file operations per route

On a non-root SSH login whose vhosts are root:root 0600, every read of a
route file is a refused SFTP read plus a `sudo cat`, and every probe for a
path that does not exist ends in a `sudo test -e`. With the previous two
fixes an unchanged registration writes and reloads nothing, but it still
paid about 25 such round trips per hostname, most of them repeats.

- Remember, for one turn of the edge config lock, what was read or written
  under the sites directory. Slug resolution, the protection check and the
  transaction snapshot read the same vhost and sidecar up to seven times.
  Only the provider's own file helpers change those files, all under that
  lock; certificates stay uncached because certbot changes them.
- `elevatedExecutor.exists` asks `lstat` before `sudo test -e`. SFTP's "no
  such file" is authoritative; a path behind a directory the login cannot
  search reports a refusal and still gets root's answer.
- Create the sites directory once per provider; the atomic writes create
  missing parents anyway.
- Skip removing a route's bootstrap certificate directory when it is
  absent: lstat says so and `exists` (which also looks inside a container
  edge) agrees. An unknown answer still removes.

Remote operations for an unchanged redeploy of nine HTTP-01 TLS
hostnames through a non-root login, counted at the provider level
(registerRoute + provisionCert per host, excluding the platform's
certificate pre-check) with a fake executor that refuses unelevated reads
of the sites and certificate directories:

                    remote execs (sudo)  SFTP ops  reloads
  main                       459 (351)       414       18
  + untrimmed reads          387 (315)       396       18
  + skip unchanged           226 (226)       326        1
  + this commit               56  (56)       317        1

Refs #1536

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(edge): report a refused lstat in elevatedExecutor.exists

The new lstat shortcut turned a rejected probe into "unknown" without
observing the error, which `errors:check` (audit-error-boundaries) flags
as an unobserved rejection. Report it through observeCaughtError like
the neighbouring `exists` fallback; behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 18:10:22 +02:00
Hydra de lerne
1bb5948243
fix(dashboard): refresh deployment state after navigation and browser resume (#1673)
* fix(dashboard): reconcile deployment status after navigation

* Avoid opening a retry stream before its destination page loads

* Preserve buffered build logs during browser resume checks

* Retain a trailing status check when resume overlaps an older read
2026-11-11 18:09:58 +02:00
Hydra de lerne
5d5621d7b6
Fix mail account overflow and list all signed-in mailboxes (#1726) 2026-11-11 17:01:35 +02:00
Johannes Rueschel
1f9fab13e4
test(edge): verify private vhosts with unprivileged workers (#1725)
* fix(edge): publish vhost configurations with 0644 permissions

* Keep edge route secrets private and verify unprivileged workers

---------

Co-authored-by: chbndrhnns <7534547+chbndrhnns@users.noreply.github.com>
Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 16:59:59 +02:00
Hydra de lerne
4ab89b9510
fix(build): keep environment values out of Dockerfiles and logs (#1674)
* fix(build): keep environment values out of Dockerfiles and logs

* test(build): exercise secret transport in deployment CI

* test(build): align cloud fixture file and command identities

* fix(build): validate secrets before allocating build resources
2026-11-11 16:54:35 +02:00
Saim Shafique
ea0d015b86
Preserve mail logs and ban history safely across engine updates (#1705)
* fix(mail): persist mail logs, fail2ban database and ignoreip across engine updates

Mail engine updates recreate the container (docker rm -f + docker run),
so anything not on a bind mount is lost: /var/log (mail.log, dovecot and
fail2ban logs), fail2ban's SQLite database, and operator ignoreip
customisations in /etc/fail2ban.

- Bind-mount /var/log and /var/lib/fail2ban from the host mail state dir.
  Deliberately not /etc/fail2ban: a seed-once mount would hide later image
  changes (same trap as #392's master.cf).
- Render operator ignoreip additions (OPENSHIP_MAIL_F2B_IGNOREIP, passed
  through engine.env) into a jail.d override on every boot, as baked
  jail.local value + operator value, so image defaults are never dropped.
- Run cron under supervisord so logrotate's daily job actually fires for
  the now-persisted logs; logrotate is installed explicitly (only a
  Recommends of rsyslog) and build-gated.

Fixes #1701

* Preserve existing mail state before replacement and repair supervised log rotation

---------

Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 16:54:11 +02:00
chriscrosby
45f8758540
Avoid duplicate webhook payloads for GitHub-owned workflows (#1690)
* Avoid duplicate GitHub observer webhook payloads

* Let periodic reconciliation cover old observer events

---------

Co-authored-by: Chris Crosby <1213497+chriscrosby@users.noreply.github.com>
2026-11-11 16:46:23 +02:00
Saim Shafique
a0230bc4f5
fix(platform): match BullMQ repeatables by name so cron edits remove the old schedule (#1697)
getRepeatableJobs() returns repeatables whose `id` is null (BullMQ 5.70.4
stores them under an opaque hashed key and never writes an `id` into the
repeat hash), so the `r.id === jobId` check in scheduleRecurring and
removeRecurring never matched. Editing a job's cron added a second
repeatable that kept firing, and disabling a policy left a stale
repeatable in Redis. scheduleRecurring adds the job under its jobId as
the name, so match on `r.name` instead.

Adds a regression test with a mocked recurring queue shaped like real
BullMQ 5.70.4 output: the stale repeatable is removed on reschedule and
on disable, and other jobs' repeatables are left alone.

Fixes #1695
2026-11-11 16:38:44 +02:00
chriscrosby
36239dd5ff
Allow reviewed repository patches to delete regular files (#1693)
* Allow reviewed repository patches to delete regular files

* Show file deletions in repository patch review

---------

Co-authored-by: Chris Crosby <1213497+chriscrosby@users.noreply.github.com>
2026-11-11 16:38:34 +02:00
Hydra de lerne
3ca648f039
fix(edge): survive SSH disconnects during upgrades (#1699)
* fix(edge): finish upgrades and rollback independently of SSH

* Check retained edge backups before stopping traffic
2026-11-11 16:25:12 +02:00
Saim Shafique
0df888e864
fix(updates): report image services with an unknown deployed digest instead of up to date (#1706)
* fix(updates): report image services with an unknown deployed digest instead of up to date

When the active service_deployment row has no image_digest — a carried
service loses it when a per-service update upserts the row — the feed
compared digests only when both were known and fell back to the equal
ref strings, so the service read as up to date and `?behind=1` dropped
the project entirely.

The deployed side is now marked unknown, not current: evaluateDrift
flags image services whose row has no digest, and the feed resolves
those against the live daemon before rendering — the running
container's image ID against the image ID the configured tag resolves
to right now, the same test the deploy pipeline uses before carrying
an unchanged service. A mismatch is behind; matching images recover
the tag's RepoDigest onto the row when the registry comparison agrees;
anything the daemon cannot answer stays unknown and the project is
kept visible with an `unverified` flag instead of being dropped.

Fixes #1702

* fix(updates): narrow the live runtime to DockerRuntime before resolving unknown digests

`resolveDeploymentRuntimeForRead` returns a `RuntimeAdapter`, and
`prepareDeploymentImage` is Docker-only, so the Typecheck job failed
with TS2345 at local-image-drift.ts:79. Guard the call with the same
`instanceof DockerRuntime` check `recordDeploymentArtifacts` uses:
other runtimes leave the services unknown, and the runtime is still
disposed on every path.

---------

Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 16:04:58 +02:00
Hydra de lerne
5bf164283d
Check source-update PR access and guide GitHub permission recovery (#1721)
* Check source PR permissions and provide connection recovery

* Update assistant PR fixtures for permission preflight
2026-11-11 15:46:45 +02:00
Saim Shafique
fb2a1f9e22
fix(mail): drop Dovecot's stale pid file before supervisord starts (#1704)
After a `docker restart` (not recreate) the openship-mail container
keeps the previous run's /run/dovecot/master.pid, and a fresh PID
namespace hands the old PID to another daemon — usually amavisd.
Dovecot then refuses to start with "Dovecot is already running",
supervisord burns its retries and marks dovecot FATAL, and nothing
listens on 993/995 until someone removes the pid file by hand.

The entrypoint already did this for Amavis (PR #885); Dovecot is the
mirror image. A new prepare-dovecot-runtime.sh removes
/var/run/dovecot/master.pid at boot, called from the entrypoint next
to the Amavis step, and a Docker e2e test proves the refusal and the
recovery against real Dovecot.

Fixes #1700
2026-11-11 15:43:59 +02:00
Hydra de lerne
41d4d76543
Fix canonical GitHub OAuth routing in same-origin proxy builds (#1723) 2026-11-11 15:30:09 +02:00
Rohan Das
6565726c01
feat(dns): add DigitalOcean DNS provider (#1707)
* feat(dns): add DigitalOcean DNS provider support

- Add DigitalOcean DNS management for custom domains and DNS-verified HTTPS
- Preserve record ownership with paired TXT markers and update provider setup guidance

* fix(dns): parse DigitalOcean record IDs from formatted JSON

- Handle compact and pretty-printed API responses, with a fallback when Python is unavailable
- Cover record cleanup and API error responses in DNS hook tests

* fix(dns): harden DigitalOcean DNS provider after review

- Decode TXT input before writing so quoted values compare in-sync
- Treat zones not delegated to DigitalOcean nameservers as pending
- Resolve zones with one paginated domain list and strip wildcard labels
- Carry credential id and provider on unauthorized lookups so the
  reconnect modal targets the rejected credential
- Clean up ownership markers when the marked record is already gone
- Report read-only token verification accurately and surface missing
  write scope on 403
- Guard token redaction and requests against an empty token
- Share DNS name helpers and the DNS-01 cleanup hook template

* fix(dns): release owned records from inactive DigitalOcean zones

- Allow DNS cleanup to resolve inactive zones while keeping writes restricted to active zones
- Fetch DigitalOcean records and ownership markers concurrently

* fix(dns): resolve provider credentials oldest first

* Require strict JSON record IDs before DNS challenge cleanup

---------

Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 15:23:24 +02:00
Hydra de lerne
f537a4fdb3
Fix custom API setup state and reuse managed agent setup (#1722) 2026-11-11 15:11:18 +02:00
Saim Shafique
3fd39c5dc5
fix(api): stamp PAT token id on REST audit rows and audit token reads (#1708)
* fix(api): stamp PAT token id on REST audit rows and audit token reads

Two gaps in the audit log, both from the same root: a REST call made
with a personal access token left no trace of WHICH token made it, and
a successful read made with a token left no row at all — while the
same read through MCP was audited with the token id attached.

- resolveCallClientId now falls back to the credential the auth layer
  already resolved: a token principal's session id IS its canonical
  principal id (pat:<id> / oauth:<clientId>), minted from the
  hashed-token lookup, not from anything the caller supplies. The
  nonce-signed MCP claim still wins. Browser sessions never reach the
  fallback (no principalKind), so a cookie session id can never land
  in the column. Shared operations and the deployment create path use
  the same resolver, so their rows gain the token id too. No schema
  change: the column was documented for exactly this value.
- The route emitter now audits successful reads/lists by token
  principals (PAT / OAuth) — a script holding a token gets the same
  "what did it look at" treatment MCP already gives its reads.
  Browser reads stay unaudited (too noisy), failed reads are still
  not logged, and operations that already recorded the call still
  skip the emitter. One exception: reads of the audit log itself are
  not audited — every such read would append a row to the log, so its
  total could never be stable and its facets would be dominated by
  whoever reads it most.

Fixes #1703

* fix(api): clear the error-boundary gate and the tokens parity expectation

The error-boundary gate flagged the pre-auth catch in
resolveCallClientId as unobserved. It only runs when no
RequestContext exists yet, so there is no credential to derive
from: mark it with the repo's diagnostics-ignore marker.

The tokens SDK-parity audit test listed only the three MCP rows,
but token reads are now audited by design, so the two remote
token-client reads in that test add two settings:read rows.
Expect them, matching the settings/webhooks parity updates.

Fixes #1703

* fix(api): avoid duplicate MCP read audit events

* fix(api): simplify audit coverage and reject malformed nonce bytes

---------

Co-authored-by: Hydra <abdelhamedmo59@gmail.com>
2026-11-11 14:43:44 +02:00
Hydra de lerne
a2384998ba
Propose source image updates through reviewed GitHub PRs (#1717)
* feat(updates): propose reviewed image changes in repository PRs

* Reuse deployment image validation and publish source-update API references

* Keep shared and inherited Compose image edits in the source editor

* Offer reviewed source updates for pinned advisory images
2026-11-11 13:55:46 +02:00
Hydra de lerne
dcf893c125
Fix webmail message colors across light and dark themes (#1716) 2026-11-11 13:36:41 +02:00
Hydra de lerne
01b9fbe739
Protect support intake and add mailbox sender blocking (#1715)
* Protect support intake and add mailbox sender blocking

* Regenerate support operator route reference

* Wire contact widget key into the website runtime

* Fix repeat contact challenges and classify private support blocks
2026-11-11 13:35:46 +02:00
Hydra de lerne
430a5ca900
Fix webmail printing and refresh security dependencies (#1718)
* Render mail print metadata as text and isolate print documents

* Update application dependencies for published security advisories

* Require patched Dovecot and Pigeonhole in mail images
2026-11-11 13:35:06 +02:00