Senior Software Engineer - LLM Compression Infrastructure

Ora Computing · Vienna · Full-time

We compress large language models (LLMs). Our information-theoretic structural pruning and quantization algorithm shrinks model footprints by over 80% without retraining, in hours rather than weeks.

We closed a €3.5M seed in 2026 (Constructor, Greencode Ventures, XISTA) and are working with customers in automotive, edge inference, and cloud.

The role

You'll own how our software stack is built. Today the codebase reflects four people moving fast, it works, but it needs structure. Your job is to give it that structure: well-designed libraries, robust packages, environments, the kind of codebase that scales as we grow the team and ship more to customers.

This is not a glue-code role. You'll work between the algorithm and inference layer: designing a compression pipeline that is fully automated and takes target runtimes into account. You'll design the abstractions our compression pipeline runs on and make them fast.

What you'll work on

• Designing and refactoring our core libraries — pruning, quantization, retraining, evaluation — into clean, well-scoped packages
• Building the internal tooling that lets the team move quickly without breaking things — CI, benchmarks, reproducible runs
• Integrating our compression output with inference engines (vLLM, TensorRT-LLM, llama.cpp) and customer deployment targets
• Setting the engineering bar for the team as we hire

Looking to advance your Development & Programming career with relocation support? Explore Development & Programming Jobs with Relocation Packages that include comprehensive packages to help you move and settle in your new role.

What we're looking for

• Bachelor’s/Master's in computer science or equivalent, plus 2+ years of professional software engineering
• Strong opinions about code design. You know what a well-structured library looks like and why
• GPU experience — memory hierarchy, kernels, what bottlenecks performance — even if you don't write CUDA daily
• Production-grade Python. You write code others can read, extend, and trust
• You finish things and you care about the codebase you leave behind

Bonus

• Open-source contributions to ML infrastructure (vLLM, llama.cpp, transformers, TensorRT-LLM, PyTorch internals)
• CUDA, Triton, or kernel-level work

Discover our full range of relocation jobs with comprehensive support packages to help you relocate and settle in your new location.

• Experience designing a library from scratch that other engineers ended up using
• Familiarity with model serving and inference optimization

Practical

• Vienna-based. Hybrid or fully remote
• Working language is English
• We sponsor visas and support relocation
• Compensation: €70–120k base + equity. Austrian minimum disclosed per Kollektivvertrag: €45,738/year
• You'll set the engineering standards we hire against next

How to apply

Send CV, a code sample you're proud of, and any open-source links to [email protected]. Tell us in two paragraphs what you'd want to build at Ora and why. We respond within a week.