AI/ML Field Guide
Comparison table

Framework matrix

Fifteen ML frameworks and inference engines on one page, with the version at the time of writing, licence, hardware support, what each is good at and what to watch for.

PyTorchJAXHugging FaceNVIDIAAppleMicrosoftOpen sourceLast verified 13 Nov 2026· PyTorch 2.15, JAX 0.12, TensorFlow 2.21, scikit-learn 1.9, Transformers 5.23, PEFT 0.21, TRL 1.18, vLLM 0.33, llama.cpp 1.0, Ollama 0.40, ONNX Runtime 1.32, MLX 0.32, TensorRT 11.4, DeepSpeed 0.20, Megatron Core 0.20; verified 13 November 2026

This page is for anyone picking a training framework or an inference engine, or checking whether the one they already use still fits. Versions were read from PyPI and the GitHub releases pages on 13 November 2026 and will drift within weeks. Licences are for the code, not for any model weights you load with it. Hardware columns say what the project supports in its own builds at the time of writing, not what a fork or a vendor plugin adds.

The matrix

Framework Current version Licence Hardware support Best at Watch for
PyTorch 2.15.0 BSD-3-Clause CUDA, ROCm, Apple MPS, CPU, TPU via torch_xla, Intel XPU Research code, custom models, the default for most open models torch.compile graph breaks on dynamic Python, MPS gaps in some ops, large wheels (CUDA builds over 2 GB)
JAX (with Flax and Optax) JAX 0.12.0.1, Flax 0.12.12, Optax 0.2.9 Apache-2.0 CPU, CUDA, TPU, ROCm plugin, experimental Metal plugin TPU training, functional code, vmap and pmap style parallelism, research at scale Functional purity rules (no in-place mutation), recompilation on new shapes, smaller ecosystem of pretrained checkpoints
TensorFlow and Keras TensorFlow 2.21.0, Keras 3.15.1 Apache-2.0 CPU, CUDA, TPU, Apple Metal via a plugin, ROCm builds from AMD Production serving with TF Serving and TFLite (LiteRT), Keras 3 runs on PyTorch and JAX too Fewer new models land here first, Windows GPU builds dropped after 2.10, Keras 3 is not Keras 2
scikit-learn 1.9.1 BSD-3-Clause CPU, some estimators dispatch to PyTorch or CuPy via the array API Tabular data, classical models, pipelines, metrics, model selection No deep learning, no GPU for most estimators, large data needs chunking or another tool
Hugging Face Transformers 5.23.0 Apache-2.0 Follows PyTorch (PyTorch only since v5) Loading any open model by name, tokenisers, generation, the Trainer Fast release cadence breaks pinned code, trust_remote_code runs arbitrary Python, memory use of generate defaults
PEFT 0.21.3 Apache-2.0 Follows PyTorch LoRA, QLoRA, prompt tuning and other adapters on top of Transformers models Adapter merging changes numerics slightly, target module names differ per architecture
TRL 1.18.0 Apache-2.0 Follows PyTorch SFT, DPO, GRPO and reward-model training with Transformers models Trainer arguments renamed between releases, defaults tuned for multi-GPU, read the config classes
vLLM 0.33.0 Apache-2.0 CUDA, ROCm, CPU (x86 and ARM), TPU, Intel XPU, AWS Neuron High-throughput serving with paged KV cache, continuous batching, an OpenAI-compatible server Needs a GPU to be worth it, long first start (graph capture), quantised kernels vary per GPU generation
llama.cpp 1.0.0 (13 November 2026, build b12364) MIT CPU, CUDA, Metal, Vulkan, ROCm (HIP), SYCL Running GGUF models on laptops and small servers, mixed CPU and GPU offload 1.0 removed the deprecated C API, so older bindings break, prompt processing is slow on CPU for long contexts
Ollama 0.40.16 MIT CPU, CUDA, ROCm, Metal One-command local models with a REST API at port 11434 Default context window is short unless you set it, model library tags hide the quantisation level
ONNX Runtime 1.32.0 MIT CPU, CUDA, TensorRT, DirectML, Core ML, ROCm, OpenVINO, QNN, WebGPU Portable inference for exported models across vendors, mobile and browser Export is the hard part: unsupported ops and dynamic shapes, LLM support lags PyTorch native
MLX 0.32.4 MIT Apple Silicon (Metal) and CPU, experimental CUDA backend Training and inference on Apple Silicon with unified memory, mlx-lm for LLMs Apple only in practice, smaller ecosystem, no ROCm or TPU
TensorRT and TensorRT-LLM TensorRT 11.4.0, TensorRT-LLM 1.2.1 TensorRT core is proprietary (NVIDIA licence), open-source components and TensorRT-LLM are Apache-2.0 NVIDIA GPUs only Lowest latency on NVIDIA hardware, FP8 and FP4 kernels on Hopper and Blackwell Engines are built per GPU model and version, long build times, no portability
DeepSpeed 0.20.0 Apache-2.0 CUDA, ROCm, Intel XPU, partial CPU ZeRO sharded training of large models across many GPUs, offload to CPU and NVMe Configuration by JSON file, stage 3 changes how you touch parameters, custom op builds at install
Megatron-LM (Megatron Core) core_v0.20.0 BSD-3-Clause style NVIDIA licence CUDA (AMD maintains a ROCm fork) Tensor, pipeline and expert parallel pretraining of transformers at hundreds of GPUs NVIDIA-shaped: assumes NCCL, Transformer Engine and NVIDIA containers, steep setup

Choosing between them

Training a model from scratch or fine-tuning one in Python: PyTorch, unless you are on TPUs or you want functional style, in which case JAX. Keras 3 is a reasonable high-level layer that runs on either. TensorFlow by itself is now mostly a maintenance choice for teams with existing TF Serving or TFLite deployments.

Tabular data: scikit-learn, with gradient boosting from LightGBM or XGBoost when trees win (they usually do on tables, see /learn/beginner/05-trees-and-gradient-boosting/). Reach for a neural network only when the features are text, images or audio.

Working with open language models: Transformers to load and tokenise, PEFT to fine-tune cheaply, TRL for preference training. See /topics/fine-tuning-and-alignment/ for the methods.

Serving an LLM: on one or more NVIDIA GPUs with many concurrent users, vLLM. On a laptop or a single machine for a few users, llama.cpp directly or through Ollama. On Apple Silicon, MLX is often faster than llama.cpp for the same model, at the cost of the GGUF ecosystem. When latency on NVIDIA hardware matters more than portability, TensorRT-LLM.

Pretraining at scale: Megatron Core for the parallelism, DeepSpeed ZeRO when you want sharding with less code change. Both are specialist tools and most teams never need them.

Interoperability

Format or path What it carries Produced by Consumed by Notes
safetensors Tensors only, with a JSON header, memory-mappable Transformers, PEFT, most trainers Transformers, vLLM, MLX, converters Safe to load from untrusted sources because there is no pickle. The default on the Hub.
PyTorch state_dict (.pt, .bin) Tensors via pickle torch.save torch.load Load with weights_only=True, which is the default since PyTorch 2.6.
ONNX Graph plus weights torch.onnx.export (the dynamo-based exporter is the default), tf2onnx, skl2onnx ONNX Runtime, TensorRT, OpenVINO, Core ML tools Op coverage lags new PyTorch features. Check opset versions.
torch.export A traced graph in PyTorch IR torch.export.export AOTInductor, ExecuTorch, the ONNX exporter The newer path. Dynamic shapes have to be declared.
GGUF Quantised weights with metadata and tokeniser convert_hf_to_gguf.py in llama.cpp, then llama-quantize llama.cpp, Ollama, many desktop apps, Transformers can read it for dequantised loading One file per quantisation level. The format is versioned.
TensorRT engine A compiled plan for one GPU model trtexec, TensorRT-LLM build TensorRT runtime only Rebuild per GPU family and per TensorRT version.
MLX safetensors Tensors plus an MLX config mlx_lm.convert MLX Shares the safetensors container with different layouts for quantised layers.

The common route for an open LLM is: train or fine-tune in PyTorch, save safetensors, then convert once per target (GGUF for llama.cpp, an MLX conversion for Apple Silicon, an AWQ or GPTQ checkpoint for vLLM). See /reference/quantisation-formats/ for what each conversion costs.

Version pinning advice

  • Pin the big three to a minor version in your lock file: torch==2.15.*, transformers==5.23.*, scikit-learn==1.9.*. Patch releases are safe, minors are not always.
  • Pin CUDA with the PyTorch wheel index you install from, and record the driver version. A torch wheel built for CUDA 12.x on a machine with an older driver fails at first kernel launch, not at import.
  • Transformers and PEFT and TRL move together. Upgrade them in one commit and run your smoke test.
  • vLLM pins its own torch version. Install it in its own environment rather than fighting the resolver.
  • llama.cpp has no stable ABI between builds. Pin the build tag (b12364) or the 1.x release, and pin the Python binding separately.
  • Record the versions in the model card or the experiment log. A result you cannot reproduce six months later is a result you do not have.

Sources