Jetson Orin Nano Super によるローカルMLLM活用についてA new engineer at Medley shares how they built a local multimodal LLM…

A newly hired engineer at the Japanese healthcare technology company Medley has published a walkthrough of a personal project that runs a multimodal large language model entirely on a compact edge device, offering a grounded look at how capable on-device AI inference has become. The post, written by an engineer named Yamamoto who joined Medley this year as a new graduate and works on the company's Job Medley recruitment service, appears as the 14th entry in the MEDLEY Summer Tech Blog Relay.

The core of the project is a local multimodal LLM (MLLM) environment built around Gemma 4 running on NVIDIA's Jetson Orin Nano Super. The author structures the article into three parts, beginning with a verification of what the Jetson Orin Nano Super can actually handle. The recurring theme is running inference locally, without sending data to a cloud service, which the summary highlights as a route to privacy-focused AI processing that avoids external dependencies. Because the device operates on the edge, the workload stays on hardware the user controls, an approach that is attractive for applications where latency, offline operation, or data sensitivity matter.

The Jetson Orin Nano Super is part of NVIDIA's Jetson line of small form-factor computers aimed at edge AI and robotics. The "Super" refresh raised the performance ceiling of the earlier Orin Nano while keeping the developer kit at an accessible price point, and it is frequently positioned as an entry route for developers who want to experiment with generative AI at the edge. These boards pair an Arm-based CPU with an integrated NVIDIA GPU and run JetPack, NVIDIA's Linux-based software stack, which provides access to CUDA and the libraries commonly used for accelerated inference. That combination is what makes it feasible to load and serve a model of Gemma's size on a board that draws far less power than a desktop workstation.

Gemma is Google's family of open-weight models, released so that developers can download and run them on their own hardware rather than only through a hosted API. Multimodal variants are designed to accept images alongside text, which is what enables the "MLLM" framing of the article: the system can, in principle, reason over visual inputs as well as written prompts. Running such a model on constrained edge hardware typically involves trade-offs, and projects of this kind often rely on techniques like quantization to reduce memory footprint and keep inference responsive. The author's verification step appears intended to test where those practical limits fall on this particular board.

For readers considering similar builds, the local LLM ecosystem now offers several adjacent tools that make experimentation easier. Runtimes such as Ollama and llama.cpp have popularized running open models on personal machines, while NVIDIA's own TensorRT-LLM targets optimized inference on its GPUs. On Jetson devices specifically, NVIDIA has promoted tutorials and containers for generative AI, reflecting a broader industry push to move at least some inference away from centralized data centers. This shift is driven partly by cost and partly by the steady improvement of smaller models, which increasingly deliver useful results without the parameter counts of frontier systems.

The wider context is a growing interest in on-device and edge AI across the industry. As open-weight models proliferate and edge accelerators become cheaper, more developers are testing whether tasks once assumed to require cloud infrastructure can run locally. A personal project like this one is a single data point rather than a benchmark, and the article is presented as an individual exploration rather than a Medley product. Still, it illustrates the kind of hands-on evaluation that helps establish what is realistic on affordable hardware today.

It is worth noting that the piece is a personal blog entry, and its findings reflect one engineer's setup rather than a formal study, so results are likely to vary with configuration, model version, and workload. For an audience weighing edge deployment, the value lies less in a definitive verdict than in a documented, reproducible attempt. It also serves as a reminder that company engineering blogs, including relay-style series like Medley's, remain a useful source of practical, real-world notes on emerging tools.