Mojo: From Art to Analytics (part III)

Deconstructed

If any missed the previous series here is part 1 and part 2 links.

Once finished with Matlab from MathWorks I decided to port my Deconstructed code base from Matlab to Modular Mojo. My initial reasons were speed and the existing Python code base to help me porting code: import Matlab files, handle matrices (via numpy) or axiliary tasks like reading images and writing videos.

Let’s start with some general information aboutModular. It is a start-up founded by Chris Lattner, which you may know from his work in ground-breaking projects like LLVM, Clang and Swift, i.e. a career dedicated to low-level software and optimization.

Modular unveiled Mojo approximately in May 2023. A new language sharing the syntax of Python but compiled, extremely fast and customizable at low level; PLUS wrapping compatibility with all the Python existing code. Sounds like a dream !

Some of the early acceleration examples demonstrated a x10,000+ increase in speed over Python on matrix multiplication (unfortunately, the blog entry is no longer available, that is the issue withMojo, material becomes obsolete fast). The main product of Modular is currently MAX, composed of 3 components:

  1. MAX (Modular Accelerated Xecution). It’s a set of tools and API to lie between open models and CPU/GPU hardware. It offers inference improvements on well-known LLMs.
  2. Mojo, as discussed
  3. Magic, a packet and virtual environment manager.

That’s the theory. As a disclaimer this is experience, which is a few months old. MAX moves fast, so some parts may not be 100% up-to-date. Let’s go with the good, the bad and the ugly.

The good.

  • Mojo is very very fast.
  • Most of the speed gains do not come from the sole fact of being a compiled language. It natively supports Single Instruction Multiple Data (SIMD) instructions. Yes, that’s something I had heard since the MMX extensions of Pentium, but never had the chance to use. Using SIMD is a game changer for high-performance algorithms involving vectors and matrices.
  • Mojo natively handles vectorization and parallelization (multi-thread) in a way that is very simple for the user.
  • Mojo documentation is very good for being a new product, partially open sourced.
  • The future may be bright, with “democratized” access to GPU hardware beyond CUDA.

The bad

  • SIMD is great but you need to change your code design philosophy to leverage it. It will not be done automatically. You also need to learn the associated operations made available, like gather and scatter.
  • I missed badly not having a similar equivalent to Numpy or a native matrix type. It helped me appreciate the huge work behind something like data slicing. There is a community package under the build but still very far from Numpy. Also MAX is taking some of those roles.
  • There is no direct installer for Windows. You can find your way using Windows Subsystem Linux (WSL) but it is not yet natively supported. Only Linux and Mac.
  • If you come from the old school of languages there are new things to learn, new concepts like traits and value ownership.
  • There is no built-in type conversion and that is painful, believe me.

The ugly

  • Compatibility with Python is still far from complete. Expect many packages not working a working with limitations: an example is Numpy.
  • Tech Support is limited. There is a nice community but I guess they have too many open fronts and you will not have quick answers to your questions/bugs.
  • Oh boy, compiler errors … Some of the many compiler errors are really difficult to figure out what’s causing them. I heard something similar from the Rust users.
  • Debugging in Windows via WSL was painful. It made me feel in 1992 debugging like in the good old times of printing traces.
  • Expect that some aspects may evolve and be superseded without backward compatibility. I gave up when the Pointer base classes were redefined.
  • Limited code base. There are some examples in their website but the lack of examples make it a bit more difficult to learn. Also LLMs will not be great experts on it.

It took me a LOT of time to port and debug the code to the point of having some proof of concepts that showed the code could run below the 1 us per iteration. Unfortunately an upgrade of Mojo, rendered my code obsolete and decided to park it as it had become a time drain. I have great hopes for the future of Mojo but it can be wise to leave it a bit of time to stabilize.

This took me to the next step: going back to the good old C++.