Edit  History 
  May 29 '23
Please feel free to contribute

Calling Babel from Django for React+JSX

Sharing my frustrations so you can enjoy them too… :)

As of writing, this website is mostly “over the wire” and has very little frontend. It’s nice because the server sends everything you need in one neat bundle. Fast and efficient. It has a little hand written JS to handle posting comments, fetching notifications, update timestamps and load continuous scrolling lists.

I was curious to try React, partly to see what the fuss was about and maybe it could help reduce bugs and repetitiveness in the dynamic features of some pages.

Rather frustratingly, the first results for using React with Django tell me to install a node.js webserver 🤦. Actually some start with CORS permissiosn and routing between docker containers for running both. So that smells. Django and node are two entirely separate things that do the same job. It makes no sense to run a node server whos only job is to forward API calls to Django. Maybe they have their reasons but I didn’t hang around to find out. Anyway, this highlights what even is React and how could it relate to Django?

What is React?

A few terms, if I’m understanding correctly. Take this with a grain of salt as I’m definitely on the noob side of web devs.

  • Static website straight HTML/CSS with declaritive “what to display” UI
  • Dynamic website are stateful and can show different content at the same URL, e.g. an online shop or wiki.
  • Responsive pages (unrelated) change UI nicely depending on the screen size, e.g. for mobile devices.
  • Reactive pages can have dynamic content without loading or reloading a page by using javascript to update the existing page.

React (again IIUC) is a javascript library that tries to make dynamic content more declaritive. It has reusable components with restrictive data flow to encourage reusability in the design. While it can be used without, it’s commonly used with JSX, javascript extension language with templating to inline HTML. Since browsers don’t run JSX, it needs to be “transpiled” to regular javascript, like TypeScript.

Anyway, this page is about integrating react code with Django. Integrating react would be as easy as adding <script/> tags for the React CDN but I want JSX too. JSX needs to be transpiled into javascript before it can be sent to the client’s browser by the webserver. Before diving into Django, I think it’s important to cover how it’s normally done - with npm/yarn, node, webpack and babel.

  • npm / yarn - these are javascript package managers, just like pip or apt but for javascript, with similar features to python’s virtualenv and requirements.txt
  • Node.js - the javascript runtime, just like the python executable runs *.py python scripts
  • node - a javascript webserver that runs on Node.js. Frustratingly, lines between Node.js and the webserver are blurred, as is evident by the Node.js docs diving straight into discussions about HTTP and webservers like it’s just assumed that’s why you’re using Node.js
  • webpack - a “bundler”, a tiny bit like django-compressor that handles import { something } from '...' statements. IMO this is vaguely like a linker that stitches compiled C object files into an executable or shared library, ready to be given to people to run.
  • babel - a “transpiler” with support for multiple languages including React. It consumes the higher level languages and produces raw javascript

Putting this all together, npm or yarn installs the other packages and ideally their dependencies. You write some React code in a .jsx file, run webpack which in turn runs babel and packages everything into a big javascript blob and finally give that to node to serve to the client. webpack reads a config file in your project that will say how to run babel and with what plugins and presets, e.g. @babel/preset-react. For starting out, people could skip all the setup with create-react-app.

So that’s the expected usage, but I already have a webserver and I really only want babel.

Babel and Django

Put simply, I want babel to transpile JSX so that I can include it in my Django project’s javascript payload, along with React libraries. There is actually PyReact but after seeing its 8 year old commits, my instinct was that babel and its react plugin would be the way to go. For installing babel, see the next section.

I’m already using django-compressor, which groups and minifies all the separate .js files, so I don’t think I want to complicate things by adding webpack to the mix just yet. It does mean adding each transpiled jsx source by hand and missing out on including jsx files from each other. If needed I expect I could find a way to call webpack instead of babel and pass its single output file to django-compressor so it could mix in existing javascript from my Django project. I learned a lot from and would recomend Modern JavaScript for Django Developers, which does use webpack. The initial discussion on server-first, client-first and the hybrid architecture is a great read.

django-compressor

Now the question is, how can I get Django to call babel. I found that django-compressor actually supports a COMPRESS_PRECOMPILERS setting that could add a transpile step to each of my sources. For example, from SO I got this to run:

COMPRESS_PRECOMPILERS = (
   ('text/jsx', '< "{infile}" babel --plugins @babel/plugin-transform-react-jsx-source --presets @babel/preset-env,@babel/preset-react > "{outfile}"'),
)

Then django-compressor just needs to find a type="text/javascript" attribute in a <script/> tag such as the following:

<script type="text/javascript" src="{% static "helloworld.jsx" %}"></script>

I had trouble getting this to automatically re-run babel after changing the source file when running in DEBUG mode. That said, I later found that Django was enabling its cached template loader so it might have just been that.

django-static-precompiler

An alternative is django-static-precompiler which basically does the same thing but before django-compressor compresses the files.

STATIC_PRECOMPILER_COMPILERS = (
    (
        "static_precompiler.compilers.Babel",
        {
            "executable": "babel",
            "sourcemap_enabled": True,
            #"plugins": "@babel/plugin-transform-react-jsx", # from the docs
            "plugins": "@babel/plugin-transform-react-jsx-source",
            "presets": "@babel/preset-env,@babel/preset-react",
        },
    ),
)

The catch is that it doesn’t have access to the type="..." attribute and its Babel backend only runs for .es6 files.

<script defer type="text/javascript" src="{% static "helloworld.es6"|compile %}"></script>

I was getting errors from the browser parsing an unknown exports variable that was fixed by using @babel/plugin-transform-react-jsx-source instead of @babel/plugin-transform-react-jsx. I guess this changes is needed because I’m skipping webpack and adding sources by hand as separate script tags.

React library

With the above taking care of transpiling JSX I just needed to include the React library itself. I’m guessing if I used webpack it could be included already. I could also host it separately myself, but using the CDN works too.

{% if debug %}
<script crossorigin src="https://unpkg.com/react@18/umd/react.development.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.development.js"></script>
{% else %}
<script crossorigin src="https://unpkg.com/react@18/umd/react.production.min.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js"></script>
{% endif %}

Installing Babel

I already have Django running in a debian based docker image. There is no node to speak of so it needs adding. The super easy way for me is to use apt and skip npm/yarn entirely. I need the babel command line interface and the plugins/presets for JSX.

apt update
apt install -y \
    node-babel-cli \
    node-babel-preset-env \
    node-babel-preset-react \
    node-babel-plugin-transform-react-jsx

This installs the executable /usr/bin/babeljs but the version that came with the distro I was using was rather old. When using "sourcemap_enabled": True, (babel -s) I would get the following error with all but the simplest jsx files:

Error: original.line and original.column are not numbers ...

I am guessing that was this bug and I could either update the OS to try with a newer version from apt or get the most recent by trying npm and yarn.

npm and yarn

The following commands install the equivalent of the above (maybe with some unnecessary extras, I’m not sure). I found that I needed to add all packages to the one command line otherwise version conflicts could occur. I guess the expected use case is to list all the packages in a package.json file. I actually used npm -g ... and yarn global ... in a docker container so it was a clean environment each time. Globally installing packages is not a good idea otherwise. Installing yarn with apt isn’t advised right now. Oddly, I saw python 2.7 being installed as a dependency with apt install npm.

# npm
apt install npm
npm install \
    @babel/core \
    @babel/runtime \
    @babel/cli \
    @babel/preset-env \
    @babel/preset-react \
    @babel/plugin-syntax-jsx \
    @babel/plugin-transform-runtime \
    @babel/plugin-transform-react-jsx \
    @babel/plugin-transform-react-jsx-source

# alternatively with yarn,
npm install yarn
yarn add \
    @babel/core \
    @babel/runtime \
    @babel/cli \
    @babel/preset-env \
    @babel/preset-react \
    @babel/plugin-syntax-jsx \
    @babel/plugin-transform-runtime \
    @babel/plugin-transform-react-jsx \
    @babel/plugin-transform-react-jsx-source

Both of these did give me a babel binary at /usr/local/bin/babel but when running it I’d get something along the lines of:

Error: Cannot find module '@babel/plugin-transform-react-jsx-source'

… or any other names I tried throwing at it. This is stupid - I literally just installed these packages; how are they missing? Actually it was searching for the difference between npm and npx that hinted to the problem here. Also running find / -name \*babel\*.

My project has no node_modules directory and I was installing everything globally in the docker container. I guess babel (or the Node.js context it runs itself in) was not set up with default module search paths, like PYTHONHOME.

$ npm root -g
/usr/local/lib/node_modules

$ yarn global dir
/usr/local/share/.config/yarn/global

Note the output from yarn does not include node_modules. Armed with this information I could finally run one of:

NODE_PATH=$(npm root -g) babel --plugins @babel/plugin-transform-react-jsx --presets @babel/preset-env,@babel/preset-react -s -o helloworld.js helloworld.jsx

NODE_PATH=$(yarn global dir)/node_modules babel --plugins @babel/plugin-transform-react-jsx --presets @babel/preset-env,@babel/preset-react -s -o helloworld.js helloworld.jsx

Then it was a matter of routing this environment variable into Django for django-compressor or django-static-precompiler to run babel with. For this I just set ENV NODE_PATH /usr/local/share/.config/yarn/global/node_modules in the Dockerfile. There’s probably a better way. Or maybe in the future, Node.js executables will automatically add their own path…

$ realpath /usr/local/bin/babel
/usr/local/share/.config/yarn/global/node_modules/@babel/cli/bin/babel.js
$ cat /usr/local/bin/../share/.config/yarn/global/node_modules/.bin/../@babel/cli/bin/babel.js
#!/usr/bin/env node

require("../lib/babel");
$  file /usr/bin/node
/usr/bin/node: ELF 64-bit LSB executable ...

In this case yarn put node at /usr/bin/node. I wonder if it could instead put it under /usr/local/share/.config/yarn/global, have a symlink at /usr/bin/node and then node could infer NODE_PATH from the binary’s location.

There are no comments yet.