upgrading/whats-new-4.0/es6-modules.rst
.. _v4-es6-modules:
When MathJax was first released, the current version of JavaScript was
ES5, so when the code base was moved to Typescript for v3, it was
down-compiled to produce ES5 code. Modern browsers support ES6, which
include many new features, such as true object class creation and
inheritance, proper import and export commands, Set and
Map objects, promises, iterators, and many other features that
make JavaScript programs faster and more reliable.
Along with new language features, ES6 introduced a new module
structure that affects how individual javascript files obtain values
from other files, and how they make their own definitions available to
others. ES6 modules (which we will refer to as MJS modules) use the
new import and export commands to do this, while the older
CommonJS module format (which we will call CJS) used require()
and the :data:module.exports object to perform those functions.
MathJax v3 uses CommonJS modules with ES5 code (though this was not quite pure ES5, since one of its dependencies was actually ES6), but modern JavaScript applications are moving more and more to MJS format. Beginning with version 4, MathJax offers both MJS and CJS versions, with the MJS version being ES6, but the CJS version remaining ES5, as in past versions. The webpacked components for use in web pages are now based on the MJS versions.
.. _v4-ESM-implications:
The webpacked MJS files are smaller than the earlier webpacked CJS
files, so that should mean faster download and compile times, and the
ES6 code is more efficient, so should run faster. But since this
version is no longer ES5, the es5 directory that was part of the
URLs for accessing MathJax from a CDN is no longer correct. The
details of how the directories have been adjusted are given in the
next section, but for use on the web, the only important difference
introduced by the change to ES6 is that you simply remove the /es5
from the url. For example, you would use
.. code-block:: shell
https://cdn.jsdelivr.net/npm/mathjax@4/tex-mml-chtml.js
to load the tex-mml-chtml.js combined component.
Support for IE11 has been dropped in v4, as it does not support enough
of the ES6 standards. (It is possible to webpack the CJS versions, so
you can build your own ES5 version if that is necessary for you. This
is described at the end of this section below.) In version 3, we
recommended a link to polyfill.io in order to support IE11. This
should now be removed since version 4 will not work with IE11 even
with the polyfill, and especially since the polyfill.io has been
bought by a Chinese entity that has used it to insert malignant code
into web sites that use it.
If your only usage is in a web browser, you can skip the :ref:next section <v4-accessing>.
.. _v4-ESM-directory-structure:
MathJax's new dual distribution of both MJS and CJS modules requires a
new directory structure in the mathjax/MathJax-src <https://github.com/mathjax/MathJax-src>__ repository and its
associated @mathjax/src npm package in order to accommodate both
versions. In the past, the compiled JavaScript code was found in the
js directory, and the webpacked components were in the es5
directory. Now that there are both CommonJS and ES-module versions of
the compiled code, these are stored in the cjs and mjs
directories, respectively.
The webpacked components are now based on the new mjs files, hence
they are ES6 files, and so the es5 directory has been removed,
with the components now being placed in the new generically named
bundle directory. That way, if there is a move to ES7 or higher,
the directory name doesn't need to change again. The mathjax/MathJax <https://github.com/mathjax/MathJax>__ repository and associated
mathjax package have also eliminated the es5 directory, and
the combined components and component directories are now at the top
level of the repository. That means you can access them without the
need for /es5 in the URL that was needed in v3. (See the
:ref:v4-accessing section for more information on how to access v4.0
in a browser.)
Existing node applications that use MathJax (e.g., the v3 examples in
the MathJax-demos-node <https://github.com/mathjax/MathJax-demos-node>__ repository) may
have code that refers to the mathjax-full/js,
mathjax-full/es5, or mathjax/es5 directories that no longer
exist. To accommodate these, v4 includes an exports section in
its package.json file that maps these references to the proper new
locations. In particular, references to @mathjax/src/es5 are
routed to @mathjax/src/bundle automatically, and similarly,
mathjax/es5 is routed to mathjax. For the @mathjax/src/js
directory, references will be routed to @mathjax/src/mjs or
@mathjax/src/cjs depending on whether the reference is from an
import statement or a require() call. That means that ES
modules (using import) will get the mjs versions, while
CommonJS modules (using require()) will get the cjs ones.
The main package.json file now includes a "type": "module"
line so that the .js files are considered to be MJS files
automatically. The cjs directory (and other directories that need
to be marked as CJS files) contain separate package.json files
that set the type to commonjs so that the .js files they
contain will be treated as CJS files.
Similarly, the font directories in the font packages have mjs and
cjs directories, with exports sections in their
package.json files to route import to the former and
require() to the latter.
.. _v4-ESM-components-src:
components/srcThe files that are used to create the webpacked component files are
ES6 modules, since they use import and export, and in previous
versions of MathJax, you needed to use node -r esm to be able to
require() these in your own programs as you can't use require()
to load ES modules directly. Although you could load the webpacked
versions of the component files in either MJS or CJS applications, the
fact that the source component files are MJS modules made it difficult
to use the source versions of the components in CJS applications.
Now that MathJax provides both MJS and CJS versions, the source
component files are made available in both forms as well. Originally,
the component files were found in components/src; with this beta
version, those are now in components/mjs, since they are ES
modules.
Prior to version 4, MathJax used Babel to convert these to ES5 during
the webpack process, but since the webpacked versions are now ES6,
that is no longer necessary, and Babel is no longer needed as a
dependency for MathJax. Instead, for those who wish to use the
components from source in a CommonJS node application, we use
Typescript to down-compile the components/mjs files into the
components/cjs directory as CJS modules of ES5 code.
In previous versions, require('mathjax-full') would load
components/src/node-main/node-main.js, which would load the
components from source rather than the webpacked versions. With the
mathjax package, which only includes the webpacked versions,
require('mathjax') would get es5/node-main.js, the webpacked
version. In version 4, the two have been standardized so that they
both load the webpacked version. When used with require(), you
will get bundle/node-main.cjs, while import will load
bundle/node-main.mjs. This is accomplished via the exports
section of the package.json file. Of course, you use
@mathjax/src in place of mathjax-full.
In order to get the source versions in @mathjax/src, use
.. code-block:: js
const MathJax = require('@mathjax/src/source');
or
.. code-block:: js
import MathJax from '@mathjax/src/source';
and then call MathJax.init(...). These load
components/mjs/node-main/node-main.mjs or
components/cjs/node-main/node-main.cjs, respectively.
For those who have been using components/src to load individual
components from source, we map components/js to components/mjs
when included via an import command, and to components/cjs
when included via require(), so you can use components/js to
get the correct files in either case. For backward compatibility,
components/src has been mapped the same way.
The end result is that you should always get an appropriate version for your situation, whether you are importing MathJax into an MJS application or requiring it into a CJS one.
.. _v4-ESM-more-issues:
Since MathJax now needs to produce javascript files in two different
formats, we use different typescript configuration files for the
different setups. These are stored in the tsconfig directory. The
MJS files are produced using tsconfig/mjs.json and the CJS one use
tsconfig/cjs.json. Both of these call in tsconfig/common.json to
set the parameters that are common to both, and then specify the
target and module values to be correct for the desired JavaScript
version and module format. The main tsconfig.json file simply calls
in tsconfig/mjs.def and is there as a convenience for tools that
expect a tsconfig.json file in the main directory.
In order to support both MJS and CJS versions, MathJax's dependencies
also must provide both versions. The speech-rule-engine,
mj-context-menu, and mhchemparser packages all now include
both module formats using dual directories similar to MathJax itself.
This means that the imports used by MathJax for these packages need to
change depending on which module version is being created. In order
to accomplish this, the references to those packages are handled using
pseudo-package references that are remapped to the correct locations
via the tsconfig.json and package.json files.
To this end, MathJax now uses the #sre, #menu, and #mhchem
pseudo-package names to refer to these packages. The main
package.json file uses the imports section to map these to the
actual package directories that contain their MJS JavaScript files.
E.g., #mhchem/* is mapped to mhchemparser/esm/*, to obtain the
ES module versions of the parser. Conversely, the package.json
file that is placed in the cjs directory maps #mhchem/* to
mhchemparser/js/* to obtain the CommonJS versions. Similar
mappings are done for the other two packages.
In addition to the mappings in the package.json file to let node
(and webpack) know which directory to use, Typescript must also be
told where to look for the .d.ts files for these packages. This
is accomplished through the tsconfig json files via the paths
array.
MathJax takes its default font from one of the MathJax font packages,
and that also has separate MJS and CJS directories, so the MathJax
code uses a pseudo-package, #default-font, to link to the proper
mjs or cjs directory in the font package. This also provides
a means of specifying what the default font is (mathjax-newcm by
default), as changing the mappings in the package.json files and
the tsconfig files would change the default font.
For the most part, MathJax's typescript source code can be used to
produce either ES modules or CommonJS modules without alteration. But
there are a few differences between MJS and CJS code that do need to
be taken into account. For example, CommonJS code provides
__dirname for the location of the file being compiled, but this is
not available in MJS modules; meanwhile, MJS files must use new URL(import.meta.url).pathname to get that data, and import is
not available in CJS modules. That means there is no common method
that can be used for this in both cases, and so MathJax has some
module-specific files to handle the few instances where
module-specific code is needed.
To accommodate this, we introduce additional pseudo-package names that
can be used to select between MJS and CJS files that export the needed
data using the module-specific mechanisms. The #root and
#mml3 pseudo-package names are used for these two situations in
the Typescript code, and #js and #source are used in the
components/mjs definitions to link to the mjs or cjs
JavaScript code and to module-specific code for obtaining the
directory name. These are mapped to the proper locations in the
package.json files and the tsconfig files. The
module-specific code that these link to are stored in mjs and
cjs directories where they are needed, so that #root/root.js
gets mapped to ts/components/mjs/root.js when MJS files are being
produced, but to ts/components/cjs/root.js for CJS files, and
similarly for #root/sre-root.js. The tsconfig files exclude
the directories for the other format so that they are not compiled
when not needed.
Finally, since modern browsers can import MJS files, it is possible to
load the MathJax files into a browser directly via a <script type="module"> tag. To do so, however, you need to include a
<script type="importmap"> tag that tells the browser how to find
the pseudo-packages described above. Something like
.. code-block:: html
<script type="importmap"> { "imports": { "#js/": "./node_modules/@mathjax/src/mjs/", "#source/source.cjs": "./node_modules/@mathjax/src/components/mjs/source-lab.js", "#root/": "./node_modules/@mathjax/src/mjs/components/mjs/", "#mml3/": "./node_modules/@mathjax/src/mjs/input/mathml/mml3/mjs/", "#default-font/": "./node_modules/mathjax-modern-font/mjs/", "#sre/": "./node_modules/speech-rule-engine/js/", "#menu/": "./node_modules/mj-context-menu/js/", "#mhchem/": "./node_modules/mhchemparser/esm/", "@mathjax/src/components/js/a11y/util.js": "./node_modules/@mathjax/src/components/mjs/a11y/util-lab.js", "@mathjax/src/components/js/": "./node_modules/@mathjax/mjs/components/mjs/", "@mathjax/src/js/": "./node_modules/@mathjax/src/mjs/", "@mathjax/src/": "./node_modules/@mathjax/src/" } } </script>should do the trick. Then you can set up your MathJax configuration
in a file mathjax-config.js as in
.. code-block:: js
import {source} from '@mathjax/src/components/js/source.js';
window.MathJax = { loader: { load: ['input/tex', 'output/chtml'], source: source } };
and then use
.. code-block:: html
<script type="module"> import './mathjax-config.js'; import '@mathjax/src/components/js/startup/startup.js'; </script>to load MathJax components via their source code rather than webpacked files.
This is useful for testing changes to MathJax, but should not be used in production, as the number of files that will be loaded can be quite large, and each file will need to be retrieved separately, making for unneeded network overhead.
.. _v4-ESM-component-json:
In past versions of MathJax, the components/src directory
contained files that control the production of the webpacked component
files in the es5 directory. Each component had a subdirectory
that contained files that told the MathJax build tools how to
construct the component. These included at least one .js file
that imported the needed MathJax modules and did any setup needed for
the component, along with one or more of build.json,
copy.json, and webpack.config.js that contain the data needed
for the build tools to process the component.
In version 4, these three .json files have been combined into a
single config.json file that contains sections for each of the
three original ones. The build property of config.json
contains the data that used to be in build.json, the copy
property holds what was in copy.json, and the webpack property
holds the data needed to pack the component.
The webpack data primarily consists of the data that used to be
passed to the :meth:PACKAGE() function in webpack.config.js, but
as named properties, and only those that differ from the defaults need
to be included (unlike the calls to :meth:PACKAGE() where all
arguments were needed). The defaults are set up so that most
properties don't need to be specified, just the name of the component
and the libs array, in most cases. There are some additional
properties that control whether the default font should be included in
the packed file, or whether a function from an external file should be
called to modify the default webpack configuration after it has been
constructed. See the config.json files in the various
components/mjs subdirectories for examples.
.. _v4-ESM-build-mjs-cjs:
There are a large number of new package scripts used for building the files used by MathJax. The main changes are that there are separate commands for building the MJS and CJS files, along with new commands to make everything all in one step, and for making single components individually.
The pnpm -s compile and pnpm -s make-components scripts
perform these steps for the MJS versions, and there is a new
.. code-block:: shell
pnpm -s build
command that does both of these at once. The command
.. code-block:: shell
pnpm -s build-all
not only compiles and packs the MJS versions, but also compiles the CJS versions.
Additional commands and details are given in the section on
:ref:v4-build-tools.
.. _v4-ESM-es5-webpack:
The webpacked components in the bundle directory are based on
the MJS files, which are ES6 JavaScript files. In contrast, the
earlier versions of MathJax had CJS versions of ES5 code. If you need
to support an ES5 environment, it is possible to build that using
.. code-block:: shell
npm run -s compile-cjs npm run -s make-cjs-components
which will create a bundle-cjs directory that contains ES5 versions
of the webpacked components, comparable to the old es5 directory.
The tsconfig.json file has the settings for the MJS versions of
the JavaScript files, and if you use an editor like emacs, the
Typescript editing mode automatically compiles the .ts files based
on tsconfig.json into the mjs directory. If you need to make
modifications to the typescript files and your application links to
the CJS versions of the compiled files, it may be convenient to switch
the tsconfig.json file to produce the CJS versions instead. To do
that, use
.. code-block:: shell
npm run -s use-cjs
which will point the tsconfig.json file to the parameters for
compiling into the cjs directory instead. When you are done,
.. code-block:: shell
npm run -s use-mjs
will set things back to the original arrangement.
|-----|