Building Python packages

PyCafe supports installing any pure Python wheel directly from PyPI by including it in your requirements.txt. However, some packages include C, C++ or Rust code, and requires a binary wheel to be built. These binary wheel packages cannot be installed from PyPI. Also, some packages do not have a wheel available on PyPI, so even those packages cannot always be installed directly on PyCafe.

There are three main sources of binary wheel packages:

• Pyodide
• PyCafe
• User uploaded

We first discuss where the packages come from, and then how to build them.

Sources of binary wheel packages

Packages from Pyodide

PyCafe supports all packages built for Pyodide, check the list of packages for each version we support here:

• https://pyodide.org/en/0.23.0/usage/packages-in-pyodide.html
• https://pyodide.org/en/0.26.1/usage/packages-in-pyodide.html
• https://pyodide.org/en/0.26.2/usage/packages-in-pyodide.html

Packages from PyCafe

PyCafe comes with an extra set of packages that are not part of Pyodide, but are available on PyCafe.

User uploaded packages

Projects can contain files that are uploaded by the user. These files can be scripts, images, but also wheels. The user can upload a wheel file to the project, and then install it in the PyCafe environment.

An example project can be found at https://py.cafe/maartenbreddels/jiter-demo where the uploaded wheel is referenced in the requirements.txt file as follows:

jiter @ https://py.cafe/files/maartenbreddels/jiter-demo/jiter-0.6.1-cp312-cp312-pyodide_2024_0_wasm32.whl

This is part of our API and can be used to install any wheel that is uploaded to PyCafe, even across projects.

Building wheels

We show how to build a wheel for a particular package so you can easily reproduce it. It might be that the specific package you want to build requires a different approach. In that case, you should consult the documentation or CI script (e.g. GitHub Actions workflow file) to consult their documentation or maintainers to find out more details about the build process.

In the examples below we used Python 3.12.

Building pure Python wheels

We use the pyperclip package as an example, which is a pure Python package but does not have a wheel on PyPI, as you can see from this list.

The simplest way to build a Python wheel is to use the build package.

$ cd /tmp
$ git clone https://github.com/asweigart/pyperclip
$ cd pyperclip
$ pip install build
$ python -m build --wheel
$ ls -al dist
pyperclip-1.9.0-py3-none-any.whl

You can now upload this wheel to your project on PyCafe and include it in the requirements.txt file. PyCafe already comes with support for pyperclip version 1.8.2, so in case you need a different version, you can build it yourself.

Note that you can right click a file on the file browser in PyCafe to get the URL to the file.

Building binary packages using Rest

To build packages with a binary extension, we will use jiter as an example.

See the Pyodide documentation about general documentation on building packages for Pyodide.

Pyodide environment

Install the pyodide-build package that matches the version of Pyodide you want to build for. In this case, we use version 0.26.3.

$ pip install pyodide-build==0.26.3

Install emscripten

PyCafe (or Pyodide) uses the emscripten platform, to build a binary that runs on emscripten, we need to install the emscripten sdk, and more specifically install and activate the version of emscripten that matches the version of Pyodide you want to build for.

$ cd /tmp
$ git clone https://github.com/emscripten-core/emsdk.git
$ cd emsdk
$ PYODIDE_EMSCRIPTEN_VERSION=$(pyodide config get emscripten_version)
$ ./emsdk install ${PYODIDE_EMSCRIPTEN_VERSION}
$ ./emsdk activate ${PYODIDE_EMSCRIPTEN_VERSION}
$ source "/tmp/emsdk/emsdk_env.sh"

Install Rust

You can install Rust any way you like, but the easiest way is to use the following command:

$ curl https://sh.rustup.rs -sSf | sh

Install Rust emscripten toolchain

In order to build a binary for emscripten, we need to install the relevant toolchain and targets.

$ rustup toolchain install nightly
$ rustup target add --toolchain nightly wasm32-unknown-emscripten

Build the binary wheel

Now that all toolchains are install and configured, we can build the wheel.

$ git clone git@github.com:pydantic/jiter.git
$ cd jiter/crates/jiter-python
$ pyodide venv .venv-pyodide
$ source .venv-pyodide/bin/activate
$ RUSTUP_TOOLCHAIN=nightly pyodide build
$ ls dist
jiter-0.6.1-cp312-cp312-pyodide_2024_0_wasm32.whl