* add unit tests to LocalRuntimeController
* Refactor miniflare-remote-bindings e2e test
This changes the test so that there is only a single remote-binding proxy worker deployment
used for all the remote-bindings (except for mtls, which needs special configuration).
* remove dev-with-resources as this is covered by other tests such as dev.test.ts and miniflare-remove-bindings.test.ts
* remove unnecessary line
* fixups
* Move findWranglerConfig tests to workers-utils
What appeared to be a simple move turned into quite a lot of changes.
But they are all for the betterment of society and general wellbeing of civilization.
* rename `test` entry-point to `test-helpers`
* add changeset
* fix: ensure that the `sourcemap-codec` library has been transformed correctly to be imported in workerd during Vitest runs
Prior to this the inline snapshot tests were failing because they use the `magic-string` library, which in turn relies upon the `sourcemap-codec` library.
This resulted in errors that look like:
```
Error running worker: SyntaxError: The requested module '@jridgewell/sourcemap-codec' does not provide an export named 'encode'
```
* refactor: run C3 e2e tests more easily locally and individually
* fixup! refactor: run C3 e2e tests more easily locally and individually
* move to-exist helper back to setup file
* rename the filtering env vars
* fixup! refactor: run C3 e2e tests more easily locally and individually
* fix c3 e2e ci groups
* remove unnecessary step
* skip gatsby on yarn
* Run windows tests in a normal directory
* skip Workers tests on Windows
* match the previous test descriptions
* feat(wrangler): add support for FedRAMP High compliance region
Now it is possible to target Wrangler at the FedRAMP High compliance region.
There are two ways to signal to Wrangler to run in this mode:
- set `"compliance_region": "fedramp_high"` in a Wrangler configuration
- set `CLOUDFLARE_COMPLIANCE_REGION=fedramp_high` environment variable when running Wrangler
If both are provided then the environment variable overrides the configured value.
When in this mode OAuth authentication is not supported.
It is necessary to authenticate using a Cloudflare API Token acquired from the Cloudflare FedRAMP High dashboard.
Most bindings and commands are supported in this mode.
- Unsupported commands may result in API requests that are not supported - possibly 422 Unprocessable Entity responses.
- Unsupported bindings may work in local dev, as there is no local validation, but will fail at Worker deployment time.
Resolves DEVX-1921.
* update workers.dev domain for fed mode
* error when env var and config don't line up
* fix env-var factory regression from earlier commits in this PR
* ensure that the local runtime controller passes through region to start mixed mode
* Check the logged-in state in the remote runtime controller rather than at the start of `wrangler dev`.
By this point we have the resolved config, so we can user the compliance region
but also it handles the case where a user toggles remote mode on during a dev session.
* Print compliance region in `wrangler whoami`
* fixups
* move ComplianceConfig type to misc-variables
* more fixes
* agh
* tighten ComplianceConfig so that it must be defined
* revert unnecessary change to `usingLocalNamespace`
* fix up the tests
* remove unnecessary empty line
* chore: configure eslint to ignore CommonJS config files (.cjs) as well as normal files (.js)
* chore: suppress logs of cached builds and tests
* chore: ensure wrangler can be published from Windows
* chore: reduce noise on Windows tests using `run-in-tmp`
* chore: ensure that NODE_EXTRA_CA_CERTS gets passed through turbo to jobs
This was causing the C3 e2e tests to fail locally for people running behind WARP, since framework generators like Remix and Solid were trying to make Internet requests to get template files and crashing with self-signed certificates errors.
* test: add new mock-npm-registry helper package
* test: use new mock-npm-registry helper in vitest-pool-workers tests
* test: use new mock-npm-registry helper in create-cloudflare e2e tests
* add to known spell check words
* add preview script to docusaurus
* refactor C3 e2e tests
* C3: don't overwrite the template's wrangler.toml in e2e tests
* test: run C3 e2e tests locally without having to deploy to cloudflare
This still tests that the frameworks are generated, compile and can be run locally;
while avoiding the time-const, flakiness, and denial of external contributors from running the tests.
* fix up C3 e2e preview test for next experimental
* uncomment PNPM cache directory
* remove unnecessary console.log()
* feat: experimental workers assets can be ignored by adding a .cfassetsignore file
* fixup! feat: experimental workers assets can be ignored by adding a .cfassetsignore file
* refactor: use the computed `compatibilityFlags` variable
* refactor: do not mutate the compatibility flags array in `validateNodeCompat()`
This makes it possible to call `validateNodeCompat()` mulitple times without breaking anything.
* refactor: move the Pages assets esbuild plugin to its own function
* refactor: remove duplication from hybrid nodejs compat plugin
* fix: ensure that `process` global works correctly in node.js compat v2
* test: add fixture and tests for nodejs_compat_v2 on Pages
* PR review updates
* Add TODO for removing experimental: prefix
* chore: add launch.json to debug single (current) test file
* chore: better docblock
* chore: pnpm install using 9.1.2
---------
Co-authored-by: Andy Jessop <ajessop@cloudflare.com>
* Implement additional E2E tests for `wrangler dev (--remote)`
* Rebase + rework e2e to use readUntil
* Kill Wrangler
* Use prebuilt node-pty to speedup CI installation
---------
Co-authored-by: Samuel Macleod <smacleod@cloudflare.com>
* fix: ensure C3 shell scripts work on Windows
Our use of `shell-quote` was causing problems on Windows where it was
escaping character (such as `@`) by placing a backslash in front.
This made Windows think that such path arguments, were at the root.
For example, `npm install -D @cloudflare/workers-types` was being converted to
`npm install -D \@cloudflare/workers-types`, which resulted in errors like:
```
npm ERR! enoent ENOENT: no such file or directory, open 'D:\@cloudflare\workers-types\package.json'
```
Now we just rely directly on the Node.js `spawn` API to avoid any shell quoting
concerns. This has resulted in a slightly less streamlined experience for people
writing C3 plugins, but has the benefit that the developer doesn't have to worry
about quoting spawn arguments.
Closes https://github.com/cloudflare/workers-sdk/issues/4282
* C3 e2e - test on CI with Windows + pnpm
* C3 e2e - clean up test projects according to their type
Previously we only deleted Pages projects - in fact we were trying to delete
the Hono (non-Pages) Worker as a Pages project.
Now we delete the project based on its type.
* C3 - framework template fixes for Windows
- do not use bash-style environment variable setting in Docusaurus scripts
- do not run TTY interactive e2e tests on Windows
* C3 e2e - make cleaning up e2e test files more resilient
* C3 - update Nuxt template to avoid requiring Windows incompatible shell environment variable
* C3 e2e - remove noise from expected test errors
* C3 e2e - Only cleanup workers and projects that are over 1 hour old
These e2e test workers and projects should be cleaned up as part of the normal test completion.
But if the test crashes they may be left orphaned.
This change ensures that we do not clean up projects to early while they are still being used.
* C3 e2e - only clean up test projects once per day at 3am
* C3 e2e - fix commit message tests on Windows
* C3 e2e - recreate test folders for each e2e retry
Previously we were reusing folders after clearing them when retrying a
failed test. But this could lead to problems, especially on Windows where
clearing out the folder did not always work.
* C3 e2e - bump test timeout longer
* C3 e2e - do not run C3 e2e tests on unsupported OSes
* C3 e2e - store e2e logs separately by OS
* C3 e2e - Retry some of the checks a few times to reduce flakiness
* C3 e2e - Record vitest results as a json artifact for downloading
* C3 e2e - disable global caches for package managers
This commit uses environment variables to tell package managers to put
cached files in a local directory rather than a global shared one, which
can cause problems with race conditions when running multiple installs
at the same time.
* C3 e2e - don't cancel other matrix jobs if one fails
It is a false optimization to cancel jobs that are likely to pass when
another job flakes out.
* ci: retry the external-durable-objects-app fixture if it flakes
* C3 e2e - drop docusaurus and react tests
These jobs tend to flake out and are not providing much of a useful signal.
* ci: skip fixture tests that exercise the "dev registry"
Reenable when https://github.com/cloudflare/workers-sdk/pull/4241 lands and improves reliability of this test.
* C3 - ensure shell commands in logging are reasonably quoted
C3 often outputs log messages to the user of commands that are being
executed. Users tend to cut and paste these into their terminal to run
themselves. This makes sure that these are likely to just work went
pasted into their shell.
* fixup! C3 - ensure shell commands in logging are reasonably quoted
* fixup! C3 - ensure shell commands in logging are reasonably quoted
* refactor: move deployment-bundle related files in their own directory
* refactor: move CfAccount to create-worker-preview
It is not really related to the types in worker.ts
* refactor: Do not export local function`fileExists`
It is not used outside the `entry.ts` file
* refactor: Move runCustomBuild (and associated functions) to its own file
* refactor: move `guessWorkerFormat()` to its own file
* refactor: Move plugins to their own directories
* refactor: Move duplicate code into `getEntryPointFromMetafile()` function
* Update packages/wrangler/src/deployment-bundle/run-custom-build.ts
Co-authored-by: MrBBot <me@mrbbot.dev>
* Ensure esbuild plugins have consistent naming
* Move source file extensions into a constant
---------
Co-authored-by: MrBBot <me@mrbbot.dev>
* [C3] fix: use a valid compatibility date for worker templates
Previously, we changed wrangler.toml to use the current date for the
compatibility_date setting in wrangler.toml when generating workers.
But this is almost always going to be new recent and results in a warning.
Now we look up the most recent compatibility date via npm on the workerd
package and use that instead.
Fixes https://github.com/cloudflare/workers-sdk/issues/2385
* Improve workerd date matching and support non-npm runners.
* fixup! [C3] fix: use a valid compatibility date for worker templates
* [wrangler] test: ensure that mocks are cleared between tests
* chore: add additional known words to spelling dictionary
* [wrangler] test: mock console logs in traverse-module-graph tests
This avoids unwanted excess logging output during tests.
* [wrangler] test: allow the C3 command to be modified via an environment variable.
`WRANGLER_C3_COMMAND` can override the command used by `wrangler init` when delegating to C3.
By default this will use `create cloudflare@2`.
If you want to test the integration between wrangler and C3 locally, you can do the following:
```sh
# Ensure both Wrangler and C3 are built
npm run build
# Run the command from the Wrangler package directory
cd packages/wrangler
# Tell Wrangler to use the local version of create-cloudflare
WRANGLER_C3_COMMAND="run create-cloudflare" npx wrangler init temp
```
This makes use of the fact we have a monorepo and npm will delegate to the local
version of a package if possible.
Since the way `wrangler init` is written, it will always use `npm` (or equivalent) to run C3, rather than `npx` (or equivalent).
Therefore we have added a `create-cloudflare` script to the Wrangler `package.json` that will work with `npm run create-cloudflare`.
But that means that you must run the `npx wrangler init` command from the `wrangler` package directory.
* Add unusual words to workspace dictionary
* Add update() method to spinner
* [C3] feat: poll DNS and URL to find if project is ready
This code uses a custom DNS resolver to avoid polluting the OS caches.
* Fix up the glob path for the c3 CI job
* Fixing tests
* Change node version to 16.14 in test-c3 workflow to fix execa
* Fix issue with projectName arg parsing
* Temporarily disable c3 tests on windows
* Clean up loop on spinner
* Add logging to auth and account retrieval
* Retry create cloudflare command
* Tighten up temp directory usage in C3 e2e tests
* Temporarily disable MacOS jobs in CI for C3
---------
Co-authored-by: James <jculveyhouse@cloudflare.com>
* Change how Devtools works
* Depend on patch-package
* Revert settings inclusion
* Give git a name
* Tabs
* Build Wrangler first?
* Ensure thi only runs on relevant branches
* Ensure thi only runs on relevant branches, including this PR
* Use correct pages project name
* refactor: add overloads to the `getEnvironmentVariableFactory()` helper
Ensure that the return value is a function that returns a guaranteed defined string if a `defaultValue` is provided.
* feat: enable Wrangler to target the staging API by setting WRANGLER_API_ENVIRONMENT=staging
If you are developing Wrangler, or an internal Cloudflare feature, and during testing,
need Wrangler to target the staging API rather than production, it is now possible by
setting the `WRANGLER_API_ENVIRONMENT` environment variable to `staging`.
This will update all the necessary OAuth and API URLs, update the OAuth client ID, and
also (if necessary) acquire an Access token for to get through the firewall to the
staging URLs.
* feat: autogenerated config
Leveraging the initialization features currently built-in we can
now generate a config file for the user. This will allow them to
further the improvement of DX and local development best practices.
Aggregated endpoints with various points of data used in Wrangler.toml to
provide a seamless experience for the user from dashboard.
resolves#1638
Co-authored-by: Pete Bacon Darwin <pete@bacondarwin.com>
* feat: add opt-in usage metrics gathering
This change adds support in Wrangler for sending usage metrics to Cloudflare.
This is an opt-in only feature. We will ask the user for permission only once per device.
The user must grant permission, on a per device basis, before we send usage metrics to Cloudflare.
The permission can also be overridden on a per project basis by setting `send_metrics = false` in the `wrangler.toml`.
If Wrangler is running in non-interactive mode (such as in a CI job) and the user has not already given permission
we will assume that we cannot send usage metrics.
The aim of this feature is to help us learn what and how features of Wrangler (and also the Cloudflare dashboard)
are being used in order to improve the developer experience.
This is a first go-round of emitting type declarations alongside the
bundled JS output, which makes it easier to use wrangler as a library.
- Added `wrangler/tsconfig.emit.json` for emitting type declarations
- Added dependency on `@microsoft/api-extractor` for bundling emitted
types into a single file
- Added `wrangler/api-extractor.json`, the api-extractor config file
- Added `wrangler/scripts/emit-types.ts`, which runs api-extractor
- Modified `.gitattributes` and `.vscode/settings.json` to treat
`api-extractor.json` and `tsconfig.emit.json` as `jsonc`
- Added `wrangler/tsconfig.emit.json` which extends `wrangler/tsconfig.json`
but emits type declarations to `emitted-types`
- Added `emitted-types` to `.gitignore`
- Extracted external dependency list from `wrangler/scripts/bundle.ts`
to keep type definitions consistent with JS bundle
* refactor: switch to tab indentation rather than spaces
* chore: update to latest prettier
* style: update files to new tab indentation
* style: convert all source code indentation to tabs
Fixes#1298
This enables simpler testing, as we do not have to spawn new child processes
to avoid the `process.exit()` from killing the jest process.
As part of the refactor, some of the `Error` classes have been moved to a
shared `errors.ts` file.
This change adds full support for the setting the protocol that the localhost proxy server listens to.
Previously, it was only possible to use `HTTP`. But now you can set it to `HTTPS` as well.
To support `HTTPS`, Wrangler needs an SSL certificate.
Wrangler now generates a self-signed certificate, as needed, and caches it in the `~/.wrangler/local-cert` directory.
These certificates expire after 30 days and are regenerated by Wrangler as needed.
Note that if you use HTTPS then your browser will complain about the self-signed and you must tell it to accept the certificate before it will let you access the page.
Some cleanups on our infra packages
- updated typescript ([some great new features](https://devblogs.microsoft.com/typescript/announcing-typescript-4-6/)), typescript-eslint-*
- updated jest, types/jest
- deleted `@types/estree`/`@babel/types` since we don't use them anymore
- setup `tsconfig.json` to explicitly include the `@types/jest` package (so we don't check every `@types/*` package by default
- setup githubs actions to fail on lint warnings
The configuration for a Worker is complicated since we can define different "environments", and each environment can have its own configuration.
There is a default ("top-level") environment and then named environments that provide environment specific configuration.
This is further complicated by the fact that there are three kinds of environment configuration:
- **non-overridable**: these values are defined once in the top-level configuration, apply to all environments and cannot be overridden by an environment.
- **inheritable**: these values can be defined at the top-level but can also be overridden by environment specific values.
Named environments do not need to provide their own values, in which case they inherit the value from the top-level.
- **non-inheritable**: these values must be explicitly defined in each environment if they are defined at the top-level.
Named environments do not inherit such configuration and must provide their own values.
All configuration values in `wrangler.toml` are optional and will receive a default value if not defined.
This change adds more strict interfaces for top-level `Config` and `Environment` types,
as well as validation and normalization of the optional fields that are read from `wrangler.toml`.
Previously the snapshots were capturing blank lines that only contained whitespace,
which was conflicting with VS Code settings that wanted to remove trailing whitespace.
Now we strip this whitespace before checking snapshots so it does not appear in our
test files, and we can safely turn on `trimTrailingWhitespace` in the VS Code settings.
See https://github.com/cloudflare/wrangler2/pull/411/files#r802597313