Open-weight vs closed models: answering the trade-off in interviews
In this article (9 sections)
- Get the terms right first
- The dimensions that actually decide it
- Three scenario answers you can adapt
- Scenario 1: an internal assistant at a regulated lender
- Scenario 2: a startup testing whether users want a feature
- Scenario 3: an application in several Indian languages
- Answers that lose marks
- The follow-up to expect
- Practise saying it aloud
Related programme: the Generative & Agentic AI course (3 months).
When an interviewer asks whether you would use an open-weight or a closed model, they are testing whether you decide from constraints rather than preference. A strong answer names the deciding factors for the specific case — data residency and control, quality on the actual task, total cost at the expected volume, latency, licence terms and the team's ability to operate it — then picks one and says what evidence would change your mind.
Get the terms right first
Interviewers notice loose vocabulary. Three terms are worth using precisely:
- Closed (proprietary) model: accessed through a provider's API or product; you do not get the weights.
- Open-weight model: weights are downloadable and can be run on your own or rented infrastructure. The licence may still restrict use.
- Open source in the strict sense also covers access to the code and terms that allow use, study, modification and sharing. Many "open" models are open-weight without meeting that definition.
Saying "open-weight, under its specific licence" instead of "open source, so free" signals that you have read a licence.
The dimensions that actually decide it
| Dimension | Pushes towards closed | Pushes towards open-weight |
|---|---|---|
| Data control | Provider's enterprise terms are acceptable to your security and legal teams | Data must not leave your environment, or regulators and clients require self-hosting |
| Task quality | The strongest available model is clearly better on your evaluation set | A smaller model, possibly fine-tuned, meets your quality bar |
| Volume and cost | Low or unpredictable volume; paying per call is simpler | High, steady volume where running your own inference may cost less overall |
| Latency | Provider's latency is fine for the use case | You need tight, predictable latency close to your systems |
| Operating skill | Team has little experience running GPU inference | Team can deploy, scale, monitor and patch model servers |
| Licence | Provider terms fit your use | The model's licence permits your commercial use and scale |
| Change control | You can tolerate provider model updates | You need to pin an exact version for audit or reproducibility |
Notice that none of these rows says one type is better. Each is a condition.
Three scenario answers you can adapt
Scenario 1: an internal assistant at a regulated lender
"I'd start by asking what the bank's risk and security teams allow. If customer data cannot leave its own infrastructure, that settles the first question: I'd shortlist open-weight models whose licences permit commercial use and evaluate them on a set of real, anonymised internal questions. If an approved enterprise API with suitable data terms exists, I'd include one closed model in the same evaluation as a quality ceiling. Then I'd compare quality against operating cost, including the GPU hosting and the people to run it. I'd expect data control to be decisive here, but I'd want the evaluation before committing."
Scenario 2: a startup testing whether users want a feature
"I'd use a closed model through an API for the first version. The question at this stage is whether users want the feature, and an API lets us learn that in days without building inference infrastructure. I'd log prompts and outputs, subject to our privacy policy, and build an evaluation set from real usage. If the feature works and volume becomes large and predictable, I'd test whether an open-weight model meets the quality bar at lower total cost. I'd write the application so the model call sits behind one interface, so switching is contained."
Scenario 3: an application in several Indian languages
"Language coverage would be my first test, because quality in Hindi, Tamil or Bengali varies a lot between models and I wouldn't trust a general leaderboard for it. I'd build a small evaluation set per language with native speakers checking outputs, and run both closed and open-weight candidates on it. If an open-weight model is strong in the languages we need, the ability to fine-tune it on domain text could matter. If not, the closed model may be the only option that meets the bar, at least initially."
Answers that lose marks
- "Open source is always better because it's free." Ignores hosting and operating cost, and licence limits.
- "Always use the best model available." Ignores cost, latency and data constraints.
- "Closed models are a security risk." Too absolute; enterprise terms and deployments vary, and self-hosting has its own risks.
- Naming a model without a reason. Model names change quickly; reasoning ages well.
The follow-up to expect
Interviewers often ask, "How would you test that?" Have a short answer ready: a task-specific evaluation set with representative and hard cases, the same set run on each candidate, results reported per slice, plus cost and latency per request at expected volume. That is the evidence that turns a preference into a decision.
Practise saying it aloud
Pick one scenario, set a two-minute timer and answer out loud. Record it. Check whether you named at least three dimensions, chose an option and said what would change your mind. If you are preparing for broader GenAI interviews, the skill check can show which skills your resume already evidences. Model selection by evaluation is covered in the Generative & Agentic AI add-on, and the data scientist to AI engineer guide explains why this kind of trade-off becomes a weekly decision.
Frequently asked questions
What is the difference between open-source and open-weight models?
An open-weight model publishes its trained weights so you can run it yourself, but its licence, training data and code may not meet the definition of open source. Licences vary a lot, including limits on commercial use or on users above a certain size, so read the specific licence rather than assuming.
Is an open-weight model always cheaper?
Not necessarily. You avoid per-call fees but take on hosting, GPUs or inference services, scaling, monitoring and updates. Which is cheaper depends on your volume, latency needs and the team's operating skill, so it has to be estimated for the case at hand.
Pankit Kumar has 10 years in Data Science & AI, building and shipping production systems in regulated pharma and clinical environments. He is a freelance trainer at Boston Institute of Analytics, AnalytixLabs and Scaler, and has taught this material to thousands of working professionals.
This article is part of our Generative & Agentic AI programme — 3 months. Add practical GenAI, retrieval and agent-building skills to your existing toolkit.
Explore Generative & Agentic AI