Amstramgram Rollup Plugins - A workflow to build website

Why ?

Rollup.js is great and already has a lot of available plugins.

BUT...

But those plugins require that you import all your stuff inside your javascript.
There is no easy way to process external html or css files.
So how to do when you simply want to build a website and serve optimized assets in your pages ?
What if you want to write your javascript in ES2022 and provide an alternative version bundled with Babel to old browsers ?
Or use latest CSS specifications and compile your stylesheets with PostCSS Preset Env ?

That was the problem I'm facing and the only solution I found to solve it was to build my own plugins collection...
At least one to process HTML files and a second to compile the CSS...
Since I still had some time to spare, I also built a little utility plugin to replace text in files based on regular expressions and another to watch whatever I need.

I will take advantage of this article to detail as precisely as possible the configuration that I use.

Goal

Usually, when I work on a multipage website, my project tree structure looks something like this :

root

dev
prod
src
package.json
rollup.config.mjs

This is the structure I used for this repo, except that :
- src folder has been renamed to docs_src,
- dev to docs_dev
- and prod to docs to ensure Github compatibility.

It also has three additional amstramgramRollupPlugin folders containing my own plugins.

So, to be clear, here's how our working structure actually is :

root

docs
docs_dev
docs_src
node_modules
rollup-plugin-postcss-amstramgram
rollup-plugin-posthtml-amstramgram
rollup-plugin-replace-amstramgram
rollup-plugin-watcher-amstramgram
package.json
rollup.config.mjs

Let's take a look at the docs_src folder :

docs_src

css
- common
  - aside.css
  - code.css
  - const.css
  - header.css
  - main.css
  - prism-vsc-dark-plus.css
- index.css
html
- common
  - head.html
  - header-aside.html
  - template.html
- content
  - css.html
  - html.html
  - index.html
  - replace.html
  - watcher.html
js
- common
  - aside.js
  - code.js
  - main.js
- polyfills
  - polyfills.js
- index.js
css.html
error.html
html.html
index.html
replace.html
watcher.html

CSS

The css/common subfolder contains the files that are imported in the index.css file.
What we should be able to do is, from the index.css file and all its imported elements, build one file processed with postcss-import and PostCSS Preset Env (and cssnano if needed) and put it in the right place in docs_dev (or docs) folder.

That's a job for rollup-plugin-postcss-amstramgram !

Finally, the result will be included at the bottom of the body of each html page and loaded asynchronously thanks to a well-known solution.

<link rel="stylesheet" href="css/index.css" media="print" onload="this.media='all';">

The most curious among you will have noticed that each of the css files or almost is in fact doubled by a little brother scss.
This allows us to test an scss-based bundle in practice.
It is besides him which is charged in the page which deals with the plugin postcss.

HTML

The html/common subfolder contains the fragments of html code that are included in all the pages.
In fact, these pages share the same meta in head, the same inline css (except for the page about the postcss plugin), as well as the same header and aside.
All these elements are included in each page with posthtml-include...

The html/content subfolder holds the html code for the specific content of each page.
The pages located at the root of the src folder are identical except for their title and description AND the fact that they each include a different content (and fortunately !!!).
Note also and once again that a different stylesheet is delivered to the css page.

Each of the main html files is just a few lines long and looks something like this :

<include src="common/template.html">
	{
		"title" : "Amstramgram Rollup Plugins - A workflow to build website",
		"description" : "An introduction to processing external HTML and CSS files with Amstramgram Rollup Plugins",
		"content" : "index",
		"style" : "" 
	}
</include>

The style key allows us to specify a particular stylesheet bundled from the scss version for the css page.

<include src="common/template.html">
	{
		"title" : "Amstramgram PostCSS Rollup Plugin",
		"description" : "Amstramgram PostCss Rollup Plugin documentation",
		"content" : "css" ,
		"style": "-from-scss"
	}
</include>

The template grabs this information and assembles the page you're reading right now...

<!doctype html>
<html lang="en" class="loading">

<head>
	<!-- A specific title -->
	<title>{{ title }}</title>
	<!-- A specific description -->
	<meta name="description" content="{{ description }}">
	<include src="common/head.html"></include>
</head>

<body>
	<include src="common/header-aside.html"></include>
	<div class="content-wrapper">
		<div class="content">
			<!-- A specific content -->
			<include src="content/{{content}}.html"></include>
		</div>
	</div>
	<div class="up"></div>
	<!-- A specific style : the style of the page about the postcss plugin is originally written in scss-->
	<link rel="stylesheet" href="css/index{{style}}.css" media="print" onload="this.media='all';">
	<script type="module" src="js/index.mjs"></script>
	<script nomodule src="js/noModule/index.js"></script> 
</body> 

</html>

We have to process the five html files located in the root of the docs_src folder with posthtml-include (and htmlnano if we are building production files) and put the resulting files in docs_dev (or docs) folder.

This is the mission that rollup-plugin-posthtml-amstramgram bravely accomplishes .

JS

The js/common subfolder contains all the javascript that are imported in the index.js file.
The main javascript file is included at the bottom of the body in script tags (see below at the end of htlm/common/template.html file) and minified for the production version.

Two versions are delivered :
- A modern one provided as a module bundled by Babel just because I use Prism to display the code and therefore take advantage of babel-plugin-prismjs.
- An old fashioned one marked with a nomodule attribute and bundled with Babel according to the browserslist specified in the package.json file.

Some oldies need a little more help.
In particular, Internet Explorer 11 does not understand Custom Events or Array.from and also has difficulty with regular expressions used extensively by prism.js (see here).
To fix this, we provide a global polyfill if the browser does not support any of these required features.
The test is performed in the <head> block of each page :

var needRegExpPolyfill = false;
//Catch IE regexp error in prism.js
try {
  RegExp('(?<test>a)');
} catch (e) {
  needRegExpPolyfill = true;
}
if (needRegExpPolyfill || !Array.from || typeof window.CustomEvent !== "function") {
  var d = document, s = d.createElement('script');
  s.async = "false";
  s.src = "js/polyfills/polyfills.js";
  d.head.appendChild(s);
}

Assets

All the assets (images, icons, etc) are stored in an assets folder located at the root of the docs_dev directory.
When building for production, we'll simply use rollup-plugin-copy to mirror it in the docs folder.

Server

All this is developed in Visual Studio Code.
I'm using the Live Server extension which provides a local development server with live reload.
The only thing we need to do is set the docs_dev folder as its root in the .code-workspace file.
To avoid unnecessary browser reloading, we also add some files to ignore.

...
  "settings": {
    "liveServer.settings.root": "/docs_dev",
    "liveServer.settings.ignoreFiles": [
      "**/*.css",
      "**/*.html",
      "**/*.map",
      "docs_dev/js/noModule/**/*",
      "docs_dev/js/polyfills/**/*",
    ],
  }
...

Rollup

Rollup has a lot to do for us :
- bundle our main javascript file in two versions as mentioned above (one in esm, another in iife).
- copy (and minify if in production) the polyfill(s).
- process the html and css files.
- bundle a cjs version of the plugins for distribution.

Our rollup config file exports an array of four objects : module, noModule, polyfills and plugins.

module and noModule have the same input but a different output format and a specific babel configuration.
polyfills simply compiles (and minifies if in production mode) our polyfill file(s) from the docs_src/js/polyfills to the docs_dev/js/polyfills (or docs/js/polyfills) folder.
plugins bundles a cjs version of the original esm version for each of our plugins.

The babelModule object defines the babel configuration for the module build whereas the babelNoModule object extends babelModule with @babel/preset-env and is used for the non module build.

Note that the browserslist entry of package.json is used by @babel/preset-env and postcss-preset-env.

//noderesolve and commonjs are needed for prism.js
import noderesolve from '@rollup/plugin-node-resolve'
import commonjs from '@rollup/plugin-commonjs'

//JS
import babel from '@rollup/plugin-babel'
import terser from '@rollup/plugin-terser'

//Replace
//Here we use the es6 version of the plugin
import replace from './rollup-plugin-replace-amstramgram/esm/index.mjs'
//If the plugin has been installed from npm :
//import replace from 'rollup-plugin-replace-amstramgram'

//CSS
//Here we use the es6 version of the plugin
import css from './rollup-plugin-postcss-amstramgram/esm/index.mjs'
//If the plugin has been installed from npm :
//import css from 'rollup-plugin-postcss-amstramgram'
import postcssImport from 'postcss-import'
import postcssPresetEnv from 'postcss-preset-env'
import cssnano from 'cssnano'

//SCSS
//https://github.com/postcss/postcss-scss
import scssParser from 'postcss-scss'
//https://github.com/csstools/postcss-sass
import sass from '@csstools/postcss-sass'


//HTML
import html from './rollup-plugin-posthtml-amstramgram/esm/index.mjs'
import htmlinclude from 'posthtml-include'
import htmlnano from 'htmlnano'

//ASSETS
//Watch changes in plugins folders
//Here we use the es6 version of the plugin
import watcher from './rollup-plugin-watcher-amstramgram/esm/index.mjs'
//If the plugin has been installed from npm :
//import watcher from 'rollup-plugin-watcher-amstramgram'

//PRODUCTION
import fsExtra from 'fs-extra'//To empty prod folder when building for production
import fg from "fast-glob"//Used for plugins export
import copy from 'rollup-plugin-copy'//Copy assets folder from dev to prod folder
import autoExternal from 'rollup-plugin-auto-external';


/**
 * Project is developed in Visual Studio Code.
 * Server with live reload is provided by the VSCode Live Server extension
 * https://github.com/ritwickdey/vscode-live-server
 * Server settings are stores in the .code-workspace file :
 * "liveServer.settings.root": "/docs_dev"
 * To prevent multiple reloads on some changes, we may add :
 */
//"liveServer.settings.ignoreFiles": [
//  "**/*.css",
//  "**/*.html",
//  "**/*.map",
//  "docs_dev/js/noModule/**/*",
//  "docs_dev/js/polyfills/**/*"
//]


const
  src = 'docs_src/',
  dev = 'docs_dev/',
  prod = 'docs/',
  dest = process.env.BUILD === 'development' ? dev : prod,
  //Babel basic configuration
  babelModule = {
    babelHelpers: 'bundled',
    plugins: [
      ['prismjs', {
        'languages': ['html', 'javascript', 'js-extras', 'json', 'scss', 'css'],
      }]
    ]
  },
  //Babel configuration to support old browsers
  //Note that browserslist is set in package.json
  babelNoModule = Object.assign({
    presets: [
      [
        "@babel/preset-env"
      ]
    ]
  }, babelModule)

//FIRST ROLLUP TASK :
//- bundle js in a module
//- compile html with minification if in production
//- compile css with minification if in production
//- watch rollup Plugins folders if in development
//- copy assets if in production
const module = {
  input: `${src}js/index.js`,
  output: {
    file: `${dest}js/index.mjs`,
    format: 'esm',
    sourcemap: process.env.BUILD === 'development',
  },
  plugins: [
    //noderesolve and commonjs are needed for prism.js
    noderesolve(),
    commonjs(),
    //Replace beginning and ending of tags by html entities :
    //  & becomes &amp;
    //  < becomes &lt;
    //  > becomes &gt;
    //The resulting text files are then included in final html files with htmlinclude
    replace({
      jobs: {
        from:
          [
            `${src}index.html`,
            `${src}css.html`,
            `${src}html/common/template.html`,
            `rollup.config.mjs`,
            `package.json`
          ],
        to: `${src}html/code`,
        rename: (name) => name + '.txt'
      },
      replace:
        [
          [/&/g, '&amp;'],  //  & becomes &amp;
          [/</g, '&lt;'],   //  < becomes &lt;
          [/>/g, '&gt;']    //  > becomes &gt;
        ]
    }),
    html({
      jobs: { from: src, to: dest },
      watch: process.env.BUILD === 'development',
      verbose: true,
      plugins: [
        htmlinclude({
          root: `${src}html/`
        }),
        ...(process.env.BUILD === 'production' ? [htmlnano()] : [])
      ]
    }),
    css({
      jobs: { from: `${src}css/index.css`, to: `${dest}css` },
      sourcemap: process.env.BUILD === 'development',
      //Watch is useless since folder src is already watched by html plugin
      verbose: true,
      plugins: [
        postcssImport(),
        //Note that browserslist is set in package.json
        postcssPresetEnv({
          stage: 1,
        }),
        ...(process.env.BUILD === 'production' ? [cssnano()] : [])
      ]
    }),
    css({//SCSS processing
      jobs: { from: `${src}css/index.scss`, to: `${dest}css`, rename: 'index-from-scss' },
      sourcemap: process.env.BUILD === 'development',
      verbose: true,
      parser: scssParser,
      plugins: [
        sass(),
        postcssPresetEnv({
          stage: 2,
        }),
      ]
    }),
    babel(babelModule),
    ...(process.env.BUILD === 'development' ?
      //If in development
      [
        //Watch the plugins folders
        //Watch package.json and rollup.config.mjs
        //since they are included in code tags in html files
        watcher({
          files: [`rollup-plugin-*`, `package.json`, `rollup.config.mjs`],
        }),
      ]
      :
      //If in production
      [
        //Copy assets folder
        copy({
          targets: [
            { src: `${dev}assets`, dest: prod }
          ]
        }),
        //Minify
        terser(),
      ]
    )
  ],
  //Comment/Uncomment if you need
  watch: {
    clearScreen: process.env.BUILD === 'production',
  },
}

//SECOND ROLLUP TASK : bundle js in IIFE format
const noModule = {
  input: `${src}js/index.js`,
  output: {
    file: `${dest}js/noModule/index.js`,
    format: 'iife',
    sourcemap: process.env.BUILD === 'development',
  },
  plugins: [
    //noderesolve and commonjs are needed for prism.js
    noderesolve(),
    commonjs(),
    //Note that browserslist is set in package.json
    babel(babelNoModule),
    //Minify if in production
    ...(process.env.BUILD === 'production' ? [terser()] : [])
  ]
}

//THIRD ROLLUP TASK : bundle polyfills
const polyfill = {
  input: `${src}js/polyfills/polyfills.js`,
  output: {
    file: `${dest}js/polyfills/polyfills.js`,
    format: 'iife',
    sourcemap: process.env.BUILD === 'development',
  },
  plugins: [
    //Minify if in production
    ...(process.env.BUILD === 'production' ? [terser()] : [])
  ]
}

//Set up plugins bundles when in production
const buildPluginsExport = _ => {
  const plugins = []
  if (process.env.BUILD === 'production') {
    //Clean production directory before production build
    fsExtra.emptyDirSync(prod)
    //bundle common js versions for each plugin
    //Use fast-glob to find all the rollup-plugin folders
    fg.sync(['rollup-plugin-*'], { onlyDirectories: true }).forEach(folder => {
      plugins.push({
        input: `${folder}/esm/index.mjs`,
        //Set the dependencies from package.json as external
        external: Object.keys(fsExtra.readJsonSync(`${folder}/package.json`).dependencies),
        output: {
          file: `${folder}/cjs/index.cjs`,
          format: 'cjs',
        },
        plugins: [autoExternal()]
      })
    })
  }
  return plugins
}

//Export rollup tasks
export default [module, noModule, polyfill, ...buildPluginsExport()]

Package.json

Just a few last words about package.json.

As the name suggests, the dev script, launched by the npm run dev command, is used for development.
It calls rollup with its config file whose default name is rollup.config.js (that's why the -c parameter) in watch mode (that's why the -w parameter) and set the node environment variable BUILD to development.

The dev-cjs script just checks that the plugins cjs versions are working properly.

The prod script (npm run prod) calls rollup with the same rollup.config.js file and set the node environment variable BUILD to production.

{
  "name": "rollup-plugin-amstramgram",
  "version": "2.0.0",
  "description": "Amstramgram Rollup Plugins repo",
  "author": "Amstramgram <contact@onfaitdessites.fr>",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/Amstramgram75/Amstramgram-Rollup-Plugins.git"
  },
  "bugs": {
    "url": "https://github.com/Amstramgram75/Amstramgram-Rollup-Plugins/issues"
  },
  "homepage": "https://github.com/Amstramgram75/Amstramgram-Rollup-Plugins#readme",
  "license": "MIT",
  "scripts": {
    "ava-check-options": "ava ./tests/ava-check-options.mjs",
    "ava-check-process": "ava ./tests/ava-check-process.mjs",
    "dev": "rollup -c -w --environment BUILD:development",
    "dev-cjs": "rollup -c rollup.cjs.js -w --bundleConfigAsCjs --environment BUILD:development",
    "prod": "rollup -c --environment BUILD:production"
  },
  "devDependencies": {
    "@babel/preset-env": "^7.21.4",
    "@csstools/postcss-sass": "^5.0.1",
    "@rollup/plugin-babel": "^6.0.3",
    "@rollup/plugin-commonjs": "^24.1.0",
    "@rollup/plugin-node-resolve": "^15.0.2",
    "@rollup/plugin-terser": "^0.4.1",
    "ava": "^5.2.0",
    "babel-plugin-prismjs": "^2.1.0",
    "cssnano": "^6.0.0",
    "fast-glob": "^3.2.12",
    "fs-extra": "^11.1.1",
    "htmlnano": "^2.0.4",
    "postcss": "^8.4.23",
    "postcss-import": "^15.1.0",
    "postcss-preset-env": "^8.3.2",
    "postcss-scss": "^4.0.6",
    "posthtml": "^0.16.6",
    "posthtml-include": "^1.7.4",
    "prismjs": "^1.29.0",
    "rollup": "^3.20.7",
    "rollup-plugin-auto-external": "^2.0.0",
    "rollup-plugin-copy": "^3.4.0",
    "terser": "^5.17.1"
  },
  "browserslist": [
    "defaults",
    "ie 11",
    "Safari >= 9"
  ]
}