Cache extracted styles across Turbopack PostCSS workers - #521
Open
borisyankov wants to merge 4 commits into
Open
borisyankov wants to merge 4 commits into
borisyankov wants to merge 4 commits into
Conversation
The builder keeps file keys as paths relative to cwd. The bundler keeps rules by absolute path. When a file was deleted, the builder gave the relative path to bundler.remove(), so the rules stayed in the CSS output until the process stopped. Resolve the path before the call. Also use a Set for the lookup of current files. The old Array.includes() lookup made each build O(n^2). A file can also be deleted during a build, after the glob finds it. The builder then could not read the file, and the build failed with ENOENT, also in watch mode. Now the builder treats such a file as deleted.
In watch mode, the bundler replaced the styles of a changed file only when the new transform created styles. The old styles stayed in the CSS output in two cases: - The file still imports react-strict-dom, but has no styles now. - The file no longer contains "react-strict-dom", so the builder did not transform it. Remove the stored styles in both cases. When a transform fails in watch mode, keep the old styles as before. The error is often a temporary syntax error during an edit. Babel returns null for a file that it ignores (the `ignore` and `only` options, or a .babelignore file). The bundler then failed with a TypeError, also in watch mode. Now it treats such a file as a file with no styles.
Turbopack runs PostCSS in short-lived worker processes. The builder loses its in-memory state (file mtimes and extracted styles) after each rebuild, so the plugin transforms all included files again on each change. In large projects, one style change can take more than a minute. In development with Turbopack, the builder now writes its state to node_modules/.cache/postcss-react-strict-dom and loads it in a new process. The builder does not transform files that did not change. The plugin finds Turbopack from the TURBOPACK environment variable, which Next.js sets. Other bundlers keep the builder in one process and do not use the cache. Each dev server session has its own cache. Turbopack starts all PostCSS workers of a session from the dev server process, so the cache file name contains the parent process id. A restart of the dev server starts with an empty cache, like the in-memory state of other bundlers. A restart then also fixes changes that the cache cannot find: a content change that keeps the mtime, a new Babel config file, a change to a file that another file imports, or a change to a linked package. The builder removes the cache files of sessions whose process no longer runs. The cache file name also contains a hash of the inputs that change the styles of an unchanged file: - the versions of postcss-react-strict-dom, @babel/core, react-strict-dom, and @stylexjs/babel-plugin, from the copies that the process loads - the Babel options The include and exclude patterns are not in the hash, because the builder removes the files that they no longer match. The hash keeps the source and flags of a RegExp in the Babel options. A function has no stable form, so the builder does not use the cache if the Babel options contain one. The cache also keeps the mtimes of the Babel config files that Babel loads for each transformed file. These include the babel.config.* and .babelrc files that Babel finds without the configFile option. If one of these files changes, the builder does not use the cache. The bundler loads the Babel config once for each file, for the transform and for the list of config files. Each set of plugin options now has its own builder. Before, one builder got the options of each build, so builds with other options in the same process changed each other's state, also at the same time. Equal options share one builder, because some bundlers create the plugin again for each build. The builder keeps the new mtime of a file only after a successful transform. It does not transform a file that failed again until the file changes. The cache does not keep the new mtime of such a file, so a new process shows the error in its first build. Concurrent builds wait for the same transform of a file, and do not transform it twice. The builder writes the cache only when its state changes. It writes to a temporary file and then renames it, so workers never read a partial file. The temporary file name contains the process id, the thread id, and a random part. The builder ignores a cache file that has missing fields. Production builds do not use the cache. Fixes react#520
The setup guides and the Vite example app set "useLayers", but the plugin reads "useCSSLayers". The plugin ignored the option. It had no effect only because the default value is also true.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #520.
Turbopack runs PostCSS in short-lived worker processes. Each new worker loses the plugin's in-memory state (file mtimes and extracted styles), so every included file is transformed again on each change. In large projects this makes one style change take more than a minute. In development with Turbopack (found through
TURBOPACK, which Next.js sets), the builder now keeps its state innode_modules/.cache/postcss-react-strict-domand transforms only the files whose mtime changed. Webpack, Vite, and production builds keep the builder in memory and do not use the cache.The cache belongs to one dev server session. Turbopack starts all PostCSS workers of a session from the dev server process (I checked this with Next.js 15.5.9 in
apps/nextjs-app), so the cache file name containsprocess.ppid. A restart starts with an empty cache, the same as the in-memory state of other bundlers. A restart also fixes anything that the mtime check cannot find: a content change that keeps the mtime, a new Babel config file, a change to a file that another file imports, or an edit to a linked package. The builder removes the cache files of sessions whose process has exited.In a session, the cache key also contains:
The include and exclude patterns are not in the key, because the builder removes the files that they no longer match.
Commits
Each commit passes the tests on its own.
bundler.remove()with a relative path, but the bundler keys rules by absolute path, so the styles of a deleted file stayed until a restart. Also, a file that is deleted between the glob and the read no longer fails the build with ENOENT.react-strict-dom, and a file that Babel ignores. Before, an ignored file caused a TypeError, because the code destructured thenullresult.useLayers, but the option isuseCSSLayers.Testing
flow checkpass at each commit. The new tests fail on the earlier version of this branch.apps/nextjs-appwithnext dev --turbopack. The cache file name contains the dev server's pid, the stale cache file was removed, and the styles of a newly added file were written to the cache.