Support .mjs output

Experimental support for ES modules just landed in Node 8.5 (changelog, PR).
Since ES modules have some parsing and semantic differences, Node decided to use the mjs extension for ES modules (while js is for the "script" target and commonjs modules).

The current Typescript version (2.5.2) supports ES modules emission but uses the js extension by default. It means that to use it with Node, a post-compilation step is required to change the extension from js to mjs. This adds undesirable complexity to use native ES modules support with Node.

A solution would be to add a compiler option to output *.mjs files when emitting ES modules.

Edit (2018-03-22): The propositions below are a bit outdated. I recommend reading the issue to see the progression. See this comment for my current proposition.

Notes:

It is possible that new parse goals will be added in the next versions of the ES spec. Letting the user provide the extension explicitly (as opposed to a boolean switch) would allow to be forward compatible with other new extension.
It should still be possible to emit ES modules with the *.js extension, many tools rely on the js extension.
Another solution would be to have a different extension for the source files: *.mts files would compile to *.mjs, this would be similar to *.tsx and *.jsx.

See #10939 (and #9839, #9551, #7926, #7699 and #9670) and this comment.

Kitson Kelly (@kitsonk)

Kitson Kelly (@kitsonk)
Thanks for posting these link but I am not sure if they are relevant, could you explain them? I found these while checking if the issue was already opened: most of the issues you listed are about input files. This issue is about compiling ts files to mjs instead of js (adding an options to change the compiler output).

Basically those say that TypeScript does not want to concern itself with managing extensions like that, as it is already complicated enough. In particular the output extensions run afoul of the TypeScript non-goal of:

Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.

If you want .mjs files, it would be best to do something like:

$ npm install renamer -g
$ renamer -regex --find '\.js^' --replace '.mjs' './outDir/**/*.js'

Kitson Kelly (@kitsonk)

Thank you very much for clarifying your comment.
I understand that adding any new feature is a burden since it means that it then must be supported for a long time, but I believe that the benefits of the mjs support are worth it: it won't turn tsc in a complex build tool but help TS users.

The long term goal of the Node team is to allow users to author their code using ES modules without paying a cost due to the commonJs modules and script target. This is one of the reasons why the proposals to require "use module" or an export statement (even an empty one) were dropped in favour of using a new extension. See the Node EPS discussions in nodejs/node-eps#57 and nodejs/node-eps#60. If a post-compilation step to rename the files is still required in two years only because .mjs was introduced later then it will just add another burden to remember for years (like the BOM or the different line endings)... Even if there are some discussions to support ES modules with the .js extension ("module" property), the current position is that - if such support is added - it should be used as a fallback for old tooling. .mjs is promoted for new code.

Complex build tools have their place for assets management, dead-code elimination, bundling, minification, etc. These all fall out of scope for TS, but being able to produce runnable code does not. Big projects have their own workflows and use the TS library (either directly or through plugins for other task runners), adding the renaming step is not a big deal for them. On the other hand, a sizeable chunk of projects using Typescript are small/medium sized libraries. These libraries usually only need a single tool: tsc. If they want to support native ES modules, they'll currently have to come up with a command similar to the one posted above by Kitson Kelly (@kitsonk). The problems are that it raises the barrier to entry, causes duplicate effort and makes the build more expensive.

It raises the barrier to entry because newcomers can no longer simply use tsc -p && node index.mjs (--experimental-modules will no longer be required when the use-case for this issue will be relevant) and run their code to try out Typescript. For existing projects, the "build and run" command is usually already defined as an npm script or gulp/grunt/webpack/whatever task, writing this command (or understanding it) will be more difficult: for some persons it will be trivial, for others it'll require a few hours of research. The rate of change in the JS ecosystem is already pretty high, let's try to not worsen it.
Now, a related problem is that it may cause some inconsistencies and duplication: different projects will use different ways to rename their file. I discovered renamer thanks to the message above, but I would have written a gulp task otherwise. Someone else would have used a POSIX-only shell command with broken edge cases or rolled their own helper Node script. To be honest, I think that most good projects would have a good implementation but this lack of standard way to deal with file renames when no build system is already there may increase the "barrier to entry" problem.
Finally, once the manual renaming is configured, it still represents an additional cost. I don't have any measures but for medium projects and using Node to rename the files, it may be longer to start the VM than running the file rename: why not doing it in the same process as TS? Also: what about --watch? How do I make it play well with my custom command? Do you have to have to watch the build directory to rename as the files are emitted?

So far, most of my arguments were about the build complexity: having it done once in TS is better than having each project coming up with its own solution. There is another important reason to have ES modules working out of the box, and I think that it aligns with the goals of TS:

Align with current and future ECMAScript proposals.

Preserve runtime behavior of all JavaScript code.

ES modules are part of the spec and the TS syntax uses them. These modules have a specified runtime behaviour that cannot be fully replicated in commonJs. It affects among other things circular dependencies, early errors, mutation of exported namespace, etc. Many people won't care, until it bites them. When setting the module option to commonjs, TS does a pretty good job of generating an output with the expected behaviour but it does not trump using real ES modules. So we are in a situation where we can either set an option and have a good approximation using commonJs but if we want the exact spec and runtime behavior we have to jump through hoops and loops.

One last argument is that even if browsers support native ES modules without requiring .mjs, real-world usage of native ES modules will start server-side with Node because you can control its version. Node is important for the whole ecosystem so it would be damageable to just ignore that it uses .mjs for ES modules. You can't just treat it as a "platform detail" when this platform is used by the majority of the TS audience.

To summarize, here are my main points:

Now that native support exists for ES modules, people will want to use it out-of-the-box.
Node settled on .mjs and hopes that it'll gain traction. .mjs is Node-specific, but Node is important
Many libraries only need tsc, requiring a second tool for renaming increases barrier to entry and causes duplicated effort (of varying quality).
Using a post-compilation command leads to higher build times and does not play well with the watch mode.
Using native ES modules is the only way to match the spec exactly, TS should not make it hard to use

Just an Opinion

As a very strong fan of TypeScript, and obviously a user of node (I guess even TypeScript is) I would really like to see some agreement between them on an issue that has had so much disagreements over many years of rational and sometimes awkward discussions.

I would have loved it if node had a transition phase to make common js (the non-standard) become .cjs over a period of two years.

That said, mjs seems to be the current direction, obviously that non-js file extension (either one) itself does not matter, but the ability for TypeScript to provide a way to control this inline as the originator of the transpiled files is something that really falls to TypeScript.

That said (again), mjs as a locked extension is kind of very opinionated, someone must have realized that forcing js was okay, it was already called js, but forcing mjs, come on, even my spelling checker is nagging me on that one. I hope nodejs will become a little more flexible on that one.

Proposal

Please make it possible to specify mjs as a module format that simply means compiling es2015 modules and calling them mjs :)

It's important to note here that .mjs isn't only about file extensions. running node with the
--experimental-modules flag also means that ES6 module specifiers must be valid urls, so the extension has to be included in the import statement, e.g. import x from './x.mjs

If I'm understanding this right, renaming the files isn't a solution, TS would also need to compile module specifiers with the .mjs extension in the ES2015+/mjs mode. We're keen to support newer versions of node by publishing non-downleveled iterators as es2015+ modules for IxJS, so it would be great to get an answer on this soon.

TypeScript already allows you to include a .js extension on your imports (even when you're actually importing .ts TypeScript files). The change needed would be to expand that to also allow .mjs.

It's not about the source files including the extension, it's about the compiled files.

If I compile import x from './x' into ES5/CommonJS, TS emits:

var x = require("./x");

If I compile it to ES2015/ESModules with an .mjs extension, TS should emit:

import x from "./x.mjs"

If a library is trying to support both old and new node with CommonJS and ESM side-by-side (as *.js and *.mjs extensions respectively), node only imports the .mjs files if the extension is included in the module specifier.

There are some potential updates in the works… including a --loader option that would you to specify a file with a resolve hook that will take the specifier and parent module path, then it could handle extension prioritization as needed. It seems to be very flexible but their goal is to make it declarative and not have people overloading the actual loader functions.

In terms of picking mjs over js, that is only the case in the CJS loader system which uses the now legacy Module prototype with it's hooks, the ones that everyone likes override all the time.

In the current --experimental-modules release, the ESM loader is hard coded to '.mjs' or falls back to the CJS loader.

What I've seen so far over the past few days makes me believe that the ESM loading system will take over with pluggable CJS loading (unless opting to use the legacy-modules by flag or based on the main file).

Check out this PR/discussion

Paul Taylor (@trxcllnt)

Paul Taylor (@trxcllnt) I guess regarding the part about baking the extension right into the import… this applies to import/export everywhere now as part of loader specs.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

So if TypeScript would continue to work without third-party tooling, it will need to find a strategy to write out standard (not ISO but platform-specific) out-of-the-"compiler"-box projects that will just work in either node or browsers at least depending on the compilerOptions intent specified by the user, and do so without asking the user to mockup some hack to get it to work.

This is just an opinion, but honestly, it feels like it for TypeScript to address.

Checkout this MDN reference but make sure you notice the part where it first said "excluding the .js extension" then in the examples included the extension anyway.

Saleh Abdel Motaal (@daflair)

I'd like to include here that the IANA has an Internet Draft which specifically adds .mjs for COMMON usage that represents the Module goal of ECMAScript . .mjs is not purely a Node.js concern and is even included in an example for browser specs and is supported by a variety MIME DBs already like shared-mime-info.

Saleh Abdel Motaal (@daflair) can you expand on what

(unless opting to use the legacy-modules by flag or based on the main file).

means?

The next step after that PR is to setup per package loaders to be able to guard against global loader mutation. So, it might be going the direction you think.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

Interestingly the WHATWG Loader Spec had a very early version in the ECMAScript spec that was removed at the last minute before ES2015! There are interesting other things like the original CommonJS Spec which mandated not to have extensions. We should probably avoid dwelling on the past so much since the different loaders vary so much on these opinions.

Node's EP specced a superset of the WHATWG resolve algorithm that does do various Node idioms like file extension completion. However, the browser has a subset that is safe to use in Node. Still, even with that subset there are problems with dependency trees since "bare" imports are waiting on userland feedback (you can get involved in the hook here), but is mostly left up to intelligent servers and service workers for now.

I'd recommend trying to compile down to the WHATWG compatible specifiers except for bare specifiers for now.

Bradley Farias (@bmeck)

Bradley Farias (@bmeck) what I meant by:

unless opting to use the legacy-modules by flag or based on the main file

I was making an assumption that at some point you will have to use something like --disable-es-modules or --legacy-modules which is really an option that simply does the opposite of --experimental-modules (when this flag is the default behaviour) to resort the current loader system and completely bypass anything related to the new loader (ie legacy applications that simply find ways to be incompatible with this existential change to their eco system).

But now that I dug in a little deeper in your recent PR's this might not be your intent…

So it is best to ask you about your intent here 😉?

Saleh Abdel Motaal (@daflair)

Saleh Abdel Motaal (@daflair) I'm still not understanding. ESM support in --experimental-modules completely avoids touching CJS except by overtaking the defaulting of .mjs to CJS (it now throws). I can't think of a reason to introduce such a flag. Also, it would be a very hard sell to change the default behavior from CJS to ESM or vice versa and I doubt that will ever happen.

adding .mts etc doesn't really help at all for package maintainers who want to provide .mjs and .cjs exports to be consumed by either audience

Node is the only platform which cares about using the mjs (or cjs) extension (optionally). For anywhere else, just using .js is fine. In node, you should almost never compile your package into both formats; this causes two copies of your package (potentially) to get loaded into node at runtime, which, in addition to being performance inefficient, can cause issues with state, class nominality, and unique symbols. Instead, in node, you should really just provide a format-specific convenience wrapper around a single source-of-truth package format (eg, a handwritten esm wrapper that imports the cjs core). Ergo, you really shouldn't need to compile the same source to both a .js cjs file and a .mjs esm file - doing so likely means you're quietly breaking something in node. (Now, if you're building for node and another platform, that's another story, but then you don't need the mjs extension anymore, since it's only used for format disambiguation in node.)

Worst case, if you really really badly want to make node load two full, separate copies of your library, you could always compile once into a lib/cjs dir and once into a lib/esm dir, and drop a package.json with "type": "module" into lib/esm. Still don't actually need .mjs at all, let alone output extension overrides for it. It is actually incredibly valuable to know that .ts always maps to .js - it just makes resolution logic so much more easy and reliable.

Hm, if you can just have a package.json in a dist-esm folder with just {"type":"module"} inside that doesn't break things in weird and wonderful new ways then that neatly solves any objections I have.. the situation I'm thinking about is indeed the package maintainers that need to build 3 versions of the package to distribute on npm because they want to support consumption from node cjs, node esm, webpack, roll-up, <script type=module>, ie11 all at the same time with a single package...

jamie-pate

jamie-pate thats exactly my observation i am researcher in that fild and member of the nodejs package-maintainance group. We Really did break evolution at some points. But the Good thing is when everything would be offered ESM Only with .js extension even without a package.json we would make the whole ecosystem healty again.

when using NPM packages you should dist only one flavor -esm -cjs -webpack everything a own package and consider even sharing packages the esm code for example could when it is a really old package you could start offering a ESM package that loads and depends on the CJS package do not even transpil that to ESM you gain nothing from that.

to load a single file CJS package inside ESM it would be enough to know that both use the same globals so we can shim like this

const resetLoader = () => {
globalThis.exports = {};
globalThis.module = { exports: globalThis.exports };
}
resetLoader()
await import('singleFile.cjs') // i do not want to over complicate it
const myCjsModule = globalThis.module.exports;
resetLoader()

there is ongoing work on the import.meta.resolve feature that will complet the above shim with resolving capabilitys

Trying to give a practical example here: I have been working on a deep dependency in our code that has been written in CommonJs a long time ago. I used it as a test to see if I could update it to be a package that is written in Typescript and supports both to be required as cjs (to not break when I drop it in the greater application) and to offer esm for the whole system to be step-by-step updated to ESM-only code. → @tradle/typeforce

My plan was to use tsc to compile teach variant of the src folder into a cjs and mjs folder with source-maps for better debugging experiences.

|- cjs
|  \- index.js
|- mjs
|  \- index.js
\- src
   \- index.ts

In the package.json I defined both the main and module field with the respective .js file. Using two different tsconfig.json variants (1, 2) as previously suggested with a shared base file.

In order to be able to test the generated files I wanted to be able to load them directly in the CLI. import('./mjs/index.js') will fail with the obvious (node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension. error message. So using a script I fixed this to rename all .js files to .mjs. This script needed to be expanded as with the change to .mjs also I better change the type definitions to .d.mts and the source maps to .mjs.map for consistency. The ties between the source and the source-maps had both to be updated as well. A pretty interesting problem that I found was that a .mjs file can not import a relative .mjs file.

(node:12628) UnhandledPromiseRejectionWarning: Error [ERR_UNSUPPORTED_DIR_IMPORT]

Of course in typescript I can not change import { compile } from './compile' to from './compile.ts as I would be running into ts(2691): An import path cannot end with a '.ts' extension. so I had to add some additional fixture in the rename script.

The initial tests were written with tape and to run the tests I wanted to be able to use just run the tests using ts-node test/index.ts. In order for this to work I set the ./tsconfig.json to have module = commonjs as ts-node doesn't support ESM yet(?).

To make sure that all code works same I originally I tried to put the tests in src/test and compile it for each variant and run them in that variant but I havn't managed to compile the tests with the external commonjs libary (fresh-tape) as module = nodejs, which is why I put it in the test folder.

I think it would help quite a bit if typescript would build node esm modules with a .mjs ending and also replace the directory import statements with direct .mjs links.

Martin Heidegger (@martinheidegger)

In order to be able to test the generated files I wanted to be able to load them directly in the CLI. import('./mjs/index.js') will fail with the obvious (node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension

Martin Heidegger (@martinheidegger) check out how https://github.com/jwalton/solox does this. The build process for solox is very similar to what you proposed with an esm tsconfig.json and a cjs tsconfig.json. These get used to compile the source into ./dist/esm and ./dist/cjs. There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains {"type": "module"}, and in the cjs folder it's {"type": "commonjs"}. Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.

The result is that all files are ".js" files, but if you clone and build this project you'll find that await import('./dist/esm/index.js') from the node REPL works exactly as you'd expect, as does require('./dist/cjs/index.js'). No rename scripts are required.

My tests are in /test, but they import from ../src, and tests run fine out of the test folder using jest and ts-jest. (If you run into problems with tests, you might want to flip things so the tsconfig.json is commonjs, and have a tsconfig.esm.json that's "module": "es6".)

Note that one potential downside of this approach (and your original approach) is that it's possible for projects to end up with two copies of your code in memory (if they both import your code and require it in different places). This approach should definitely be avoided if you have singleton instances or other global state.

Jason Walton (@jwalton)

Thank you Jason Walton (@jwalton) for sharing notes.

There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains {"type": "module"}

Oh! Thank you for pointing this out. This will indeed make my solution a lot simpler. Wouldn't this be a good feature to add to the typescript compile process?

Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.

Actually, I also was under the requirement to support deep imports like import * from 'typeforce/async' as part of backwards compatibility and I needed to conditional exports per file. (manual task atm.) It would be nice to have a good automation for that as well.

In a simple experiment with solox (found here martinheidegger/solox@e0a5edd) I tried to add a simple dependency and ran into a catch-22 where lint would break without a .ts declaration and compile now breaks with. 😅

In a simple experiment with solox (found here martinheidegger/solox@e0a5edd) I tried to add a simple dependency and ran into a catch-22 where lint would break without a .ts declaration and compile now breaks with. 😅

This is a idiosyncracy of typescript, and it's not very intuative, but try making this change:

-import foo from './test.ts';
+import foo from './test.js';

(i.e. change the "ts" to a "js" in your import) and your experiment should build and run correctly.

For deep imports in nodejs you can use the exports property of package.json
https://nodejs.org/api/packages.html#exports

    "exports": {
        "./*": "./dist/*.js",
        "./*/*": "./dist/*/*.js"
    },

NodeJS does not recognize the "modules" property, only bundlers will recognize that.

I'm using that with the same import 'test.js'; approach in typescript and the {"type":"module"} micro package.json approach as shared above, and finally after a week of futzing i'm pretty happy with what I've got. (except that the webpack/angular part of the project still uses import 'test' because they haven't agreed on the manditory import extensions idea 😭 )

jamie-pate

(i.e. change the "ts" to a "js" in your import) and your experiment should build and run correctly.

Interesting. This seems to be not yet supported by ts-node, though this is actively being worked on TypeStrong/ts-node#1361 I will revisit this and post an update using yours and jamie-pate input when that landed.

I use swc to transpile my ts files to mjs.

yarn add -D @swc/cli @swc/core
yarn run swc next.config.ts -o next.config.mjs -C module.type=es6 -C sourceMaps=inline

Martin Heidegger (@martinheidegger)

In order to be able to test the generated files I wanted to be able to load them directly in the CLI. import('./mjs/index.js') will fail with the obvious (node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension

Martin Heidegger (@martinheidegger) check out how https://github.com/jwalton/solox does this. The build process for solox is very similar to what you proposed with an esm tsconfig.json and a cjs tsconfig.json. These get used to compile the source into ./dist/esm and ./dist/cjs. There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains {"type": "module"}, and in the cjs folder it's {"type": "commonjs"}. Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.

The result is that all files are ".js" files, but if you clone and build this project you'll find that await import('./dist/esm/index.js') from the node REPL works exactly as you'd expect, as does require('./dist/cjs/index.js'). No rename scripts are required.

My tests are in /test, but they import from ../src, and tests run fine out of the test folder using jest and ts-jest. (If you run into problems with tests, you might want to flip things so the tsconfig.json is commonjs, and have a tsconfig.esm.json that's "module": "es6".)

Note that one potential downside of this approach (and your original approach) is that it's possible for projects to end up with two copies of your code in memory (if they both import your code and require it in different places). This approach should definitely be avoided if you have singleton instances or other global state.

Jason Walton (@jwalton) This proposal looks like a duplicate of probably the most insightful post on the topic to date cc Michael O'Brien (@mobsense) : https://www.sensedeep.com/blog/posts/2021/how-to-create-single-source-npm-module.html

By the way, for those who may be interested in having a look at the rewrite strategy path, it also seems worth considering the 2 packages below :

• tsc-multi

• tsukuru

Guillaume FORTAINE (@gfortaine)

Guillaume FORTAINE (@gfortaine) Excellent suggestion. By placing a { "type": "module" } into my /dist/esm folder I no longer have to have the "type": "module" at my root package.json but my ESM is now properly importing, assuming I manually specify the .js extensions, but this is close.

azu (@azu)

tsconfig-to-dual-package cc azu (@azu)

build: add fixup script for dual modules

Ensures CJS/ESM bundle is compatible with Alpha (sonic), pagesJS (vit…

Navigation Menu

Repository navigation

Support `.mjs` output #18442

200 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support .mjs output #18442

Description

Activity

kitsonk commented on Sep 13, 2017

demurgos commented on Sep 13, 2017

kitsonk commented on Sep 14, 2017

demurgos commented on Sep 15, 2017

SMotaal commented on Sep 28, 2017

trxcllnt commented on Oct 3, 2017

dpogue commented on Oct 3, 2017

trxcllnt commented on Oct 3, 2017

SMotaal commented on Oct 3, 2017

SMotaal commented on Oct 3, 2017

bmeck commented on Oct 3, 2017

SMotaal commented on Oct 3, 2017

bmeck commented on Oct 3, 2017

200 remaining items

weswigham commented on Feb 12, 2022

jamie-pate commented on Feb 12, 2022

frank-dspeed commented on Feb 12, 2022

martinheidegger commented on Feb 16, 2022

jwalton commented on Feb 16, 2022

martinheidegger commented on Feb 16, 2022

jwalton commented on Feb 16, 2022

jamie-pate commented on Feb 16, 2022

martinheidegger commented on Feb 16, 2022

lucasvazq commented on Mar 19, 2022

gfortaine commented on Jan 2, 2023

owenallenaz commented on Jan 10, 2023

gfortaine commented on Jan 14, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Support `.mjs` output #18442