NUS Introduces Jeva: A Native Architecture for Decision-Centric LLMs

When the endpoint of a task is decision-making, should models start from the decision itself?

Finding relevant evidence in a set of documents, determining whether a condition holds, or scoring according to criteria—many language model requests ultimately require not a long answer, but a clear and reliable judgment.

The System One model is designed precisely for such needs: directly outputting choices, probabilities, or scores without generating lengthy reasoning text.

However, simply considering "directly providing answers" is insufficient. For the same question and the same set of answers, if we merely change the order of the options, should the model's decision change?

Consider another scenario: A user wants to process a refund, but the system only offers three options: "Change Address," "Check Logistics," and "Reset Password." Even if the model provides a clear ranking, is the option ranked first necessarily the correct answer?

Recent work from the xML Lab at the National University of Singapore places these questions back at the starting point of architectural design.

Paper Link: https://arxiv.org/pdf/2610.18773

GitHub: https://github.com/LiQiiiii/Jeva

Model: https://huggingface.co/LIQIIIII/Jeva-4B

Figure 1 | Jeva's architecture and four structural properties. The top shows an architectural comparison, while the bottom provides an intuitive illustration; based on Figure 1 of the original paper.

What fundamental capabilities should decision-making models possess?

Jeva focuses not on specialized tricks for specific tasks, but on four structural properties commonly required across different decision-making tasks.

Null-aware｜No suitable answer is also an answer

"The best option" does not equal "the appropriate option." Suppose a user needs to process a refund, but the candidates are only Change Address, Check Logistics, and Reset Password: even if one has the highest score, it doesn't mean it solves the problem. Jeva separates the judgment of "whether an acceptable candidate exists" from "which candidate is better among them," providing explicit null output when no answer is present, rather than forcing a selection.

Permutation-invariant｜Changing order shouldn't change judgment

The meaning of the same set of candidates should not change if they are reordered from A, B, C to C, A, B. Jeva performs symmetric aggregation over the candidate set and scores them using the same rules, ensuring that probabilities follow candidate identity rather than list position. What truly matters is not whether the model still selects the "second item," but whether it still selects the same answer.

Option-scalable｜More options don't change the decision method

Candidates might be a dozen labels or thousands of documents. Jeva applies a shared scoring rule to each candidate, eliminating the need to expand a fixed set of answer symbols for new candidates. In the paper's large-candidate experiments, it directly processes the complete set without needing to narrow down the scope through multiple rounds of elimination.

Cacheable｜Reuse computation for options, not old answers

Documents, policy clauses, and scoring criteria are often more stable than user queries. Jeva can retain the computational results for these options and continue using them when queries change or candidates are recombined; however, the judgment for the current request is still re-executed. What is cached is reusable information, not the conclusion from the previous instance.

These four properties are not unrelated feature lists, but questions that the same decision-making architecture must jointly address.

How does Jeva translate these requirements into architecture?

Jeva separates "understanding text" from "making the current judgment": a shared encoder represents the query and options separately, and a lightweight decision module then reads the entire candidate set, updating only the query state. This ensures that option representations remain reusable, while judgments update according to the current query and candidate content.

First, let the candidate set participate in judgment

Separate encoding does not imply independent scoring. As shown in the figure below, by fixing the initial query state and candidates A and B, and only replacing the third candidate from C₊ to C₋, the updated state changes direction, shifting preference from A to B.

This construction demonstrates that a single decision block can already express set-dependent preferences; in contrast, relying solely on independent scoring between the query and individual candidates cannot reverse the relative ranking of A and B due to changes in a third candidate.

Figure 2 | Fixed options, changing preferences. (a) Keeping A, B, and the initial state unchanged, replace the third candidate; (b) The query state updates along different directions; (c) The score difference between A and B shifts from positive to negative; based on Figure 3 of the original paper.

Changing positions should not alter judgment; changing available content may adjust judgment accordingly.

Then distinguish "who is better" from "is there a suitable answer"

After obtaining candidate scores, it is necessary to determine whether the entire set of options is applicable. Jeva uses $q$ to describe relative preferences among candidates, estimates the existence of a suitable answer via coverage probability $\rho$, and synthesizes this into a complete distribution $p$ that includes null.

The upper part of the figure below lowers the scores of two candidates by the same magnitude simultaneously: the relative probabilities remain 73% and 27%, but the coverage probability drops from 88% to 12%, causing the null probability to rise. The lower part fixes existing scores and adds a candidate with a score lower than the current maximum: the coverage probability remains unchanged, while probabilities among candidates are redistributed.

Figure 3 | Relative preference and answer presence are determined separately. (a) Uniformly lowering candidate scores leaves $q$ unchanged but reduces the coverage probability $\rho$; (b) Adding low-scoring candidates to existing ones keeps $\rho$ constant while redistributing candidate probabilities within the full distribution $p$. Adapted from Figure 4 of the original paper.

Finally, Reuse Computation for Invariant Options

The state updates and output computations described above are executed per request. However, the semantic representations of options and their layer projections can be cached. When the model, decision type, and encoding settings remain fixed, new requests can retrieve these cached memories to reconstruct the current candidate set.

In the figure below, processing A, B, and C for the first time requires preparing three options. For the second round facing B, C, and D, only D needs to be added. In the third round using A, C, and D, all three options can be reused. Each time, the request is re-encoded, and the current judgment is computed anew.

Figure 4 | New requests and combinations reuse existing candidates. The three candidate sets are ABC, BCD, and ACD, with single-option cache hit rates of 0/3, 2/3, and 3/3 respectively. Adapted from Figure 5 of the original paper.

The same computational framework supports Choice, Noul, and Score outputs. Given a report stating "Production login page returns HTTP 503, preventing all customers from logging in or completing purchases," the three output modes can address the questions in the table below. Specifically, Noul returns the probability that the proposition is true, while null indicates no suitable candidate exists.

Experimental Validation

The Jeva-4B model in the paper uses a Qwen3.5-4B backbone and a decision block. Evaluation focuses on task quality while also examining structural issues such as absent answers, option permutation, and scaling candidate size.

Can "None of the Above" Be Directly Identified?

The research team constructed 3,080 matched requests on BANKING77, each providing 16 candidates, including 588 cases where the correct answer was absent. Jeva achieved an AUROC of 98.66% for null detection, outperforming baselines KEV-BF16 (93.61%), InternDecision-4B (84.27%), and JevK5 (79.44%).

AUROC measures the model's overall ability to distinguish between "answer present" and "answer absent" across different thresholds. More intuitively, at a threshold where approximately 5% of valid-answer requests are misclassified as having no answer, Jeva correctly identifies about 94% of the absent-answer requests.

This corresponds to a clear experimental goal derived from the refund question mentioned earlier: the model must not only rank options but also identify when none of the provided options are applicable.

Does Changing the Order Alter the Answer for the Same Question?

Order sensitivity experiments fix the question and candidate content, changing only the arrangement of candidates. On 600 Choice samples, the team tested the original order, reverse order, and up to three random permutations, aligning results by candidate identity.

Jeva exhibited no decision flips; both the original accuracy and the accuracy under "correct in every tested order" were 76.67%. In contrast, the flip rates for Jev-1.13.0, InternDecision-4B, and JevK5 were 17.00%, 22.67%, and 26.17%, respectively.

Figure 5 | Left: ROC curve for identifying absent answers; closer to the top-left corner is better. Right: Original accuracy versus accuracy correct under all tested orders; the gap between them illustrates the impact of order changes. Excerpted from Figures 6(b,c) of the original paper.

These two sets of experiments refine the concept of "reliable selection": rejecting options when they are inappropriate, and maintaining stable judgments when options merely change position.

Can It Scale Natively and Effectively as Options Increase?

As the number of candidates grows, a new challenge arises: Is the model truly comparing the complete set, or is it forced to decompose it into many rounds of small-scale selections?

In the paper's ToolRet diagnostic, request distributions span six settings ranging from 32 to 2,048 candidates. Jeva supports all requests; candidate text can be encoded in batches, but once prepared, the full set participates in comparison within a single decision head computation.

In the 2,048-candidate setting, Jeva achieves 69.88% accuracy with a median request latency of 12.04 seconds on a single H200 GPU. Recursive selection-based JevK5 achieves 67.47% accuracy with a median latency of approximately 28.35 seconds. Thus, Jeva simultaneously delivers higher accuracy and roughly 2.35× speed advantage in this setting.

Figure 6 | Top: Accuracy. Middle: Median request latency; the dashed line represents Jeva's previous batched setting, where 7.85× corresponds to the batching optimization gain for the same model. Bottom: Proportion of processable requests. Different candidate arrays use different questions; accuracy comparisons are restricted to fully supported settings. Adapted from Figure 8 of the original paper.

A more critical question remains: For the same problem, as more candidates are gradually added, can the model consistently provide the correct answer? To address this, the paper fixes 83 questions and progressively expands the candidate set for each question from 32 to 2,048.

Figure 7 | Each column represents the same question, while each row corresponds to a different number of candidates. Colored cells indicate correct answers; adapted from Figure 7 of the original paper.

Jeva correctly answered 58 of these questions across all tested scales, compared to 48 for JevK5 and only 7 for CLM. Rather than focusing solely on accuracy at a specific scale, this experiment more directly demonstrates how well correct decisions are preserved as the candidate set expands.

If the questions vary, why repeatedly process the same document?

Long documents in retrieval tasks, policy clauses in business systems, and scoring criteria in evaluation tasks may be reused by many different requests. As candidates grow longer and their quantity increases, the cost of redundant encoding becomes increasingly significant. Jeva’s reuse mechanism supports not only identical candidate sets but also allows assembling new sets by retrieving existing options from the cache.

In the MLDR long-document experiments described in the paper, when facing newly combined candidate sets where all candidates were already cached, the median request latency dropped from approximately 2.10 seconds to 75.1 milliseconds—a 27.98x speedup. No changes in decision outcomes were observed under different caching conditions.

Figure 8 | Top: Speedup under different candidate reuse ratios; Middle: Request latency with shared candidate libraries; Bottom: Proportion of candidates avoiding redundant encoding; adapted from Figure 9 of the original paper.

Summary

Jeva unifies answer-absence recognition, order invariance, candidate expansion, and computational reuse within its model design. Its core philosophy can be summarized as:

Making fundamental decision principles an intrinsic capability of the architecture.

Decision principles, built into the architecture.

Jeva offers more than just faster single-inference performance; it provides a new design perspective: treating candidate sets, answer absence, and computational reuse as structural components of decision-making itself, ensuring the architecture is designed from the ground up to meet these requirements.

References:

https://arxiv.org/pdf/2610.18773

Follow ASI Instantly

⭐ Like, Share, and "Watching" – The One-Click Trio ⭐

Star us to lock in instant updates from New Intelligence Era!