Skip to content
OpenJevX wiki
Every statement on this page is cited and was checked against jevresearch HEAD (4634ffa), jevx HEAD (4a4722d), openjevx v0.5.9 (ee2a1f4) on 2026-10-04.

Question types

The three types

Section titled “The three types”
type ask criteria read the answer from
noul yes/no none (optional {"true": "...", "false": "..."}) noul: probability of yes 1
choice pick one {"key": "description", ...} choice and confidence 2
score rating ["lowest", ..., "highest"] score (expected level, 0 = first) and probabilities 3

A type other than these three returns the error unknown type. 4 Every question needs at least two options. 5

noul

Section titled “noul”

noul has two keys, false and true. 6 Without criteria, the options read “no, the statement does not hold” and “yes, the statement holds”. 7 Set criteria to {"true": "...", "false": "..."} to replace the default option texts. 8 9 The noul field is the probability of true. 6 10 For noul, confidence is the larger of P(true) and 1 - P(true). 11

choice

Section titled “choice”

choice criteria are {"key": "description", ...}. 2 Each option reads key: description, or only key when the description is empty. 12 criteria may also be a JSON list; each item then becomes a key. 13 choice is the key of the option with the highest probability (best). 14

score

Section titled “score”

score criteria are a JSON list of levels, from lowest to highest. 15 16 Each score level gets the option text level <i>: <text>. 17 score is the expected level, sum(j * p_j) over the levels j. 18 So score is a decimal, with 0 as the first level. 18 16

Answer fields

Section titled “Answer fields”

Every answer has type, probabilities, confidence and answer_confidence. 19 probabilities maps each key to its probability, rounded to 4 decimals. 20 21 confidence and answer_confidence are both set to the probability of the best option. 22 For noul, confidence is then replaced by the larger of P(true) and 1 - P(true). 6 23 action.act_probability comes from the model’s second output, a two-way softmax over its action logits. 24 25 action is only added when both action outputs are finite numbers. 24

Note: the training script adds the action output to the loss with weight 0.0. 26 In the several-judgements recipe (model 0.5.2, server 0.5.7, CPU), act_probability is 1 on all three answers. 27

How the input is laid out

Section titled “How the input is laid out”

Each question is encoded as its own item, with the same state tokens. 28 The question head is the text <type> question: <instructions>. 29 Each option is cut to 48 tokens. 30 Each option starts with a [MASK] token. 31 If the options leave fewer than 16 tokens of the head_max budget, each option longer than max(4, (head_max - 16) / number of options) tokens is cut to that length. 32 The head is then cut to the remaining budget, keeping at least 8 tokens. 33 The item is [CLS], the head, [SEP], the options, [SEP], the state, [SEP]. 34 The state is cut to max_len minus the encoded head and options (with their CLS and SEP tokens), minus one. 35 Option markers at or past max_len are dropped. 36 Literal [MASK] text in the state, instructions or options is replaced by a space. 37 38

The model folder’s config.json sets max_len and head_max. 39 The shipped model 0.5.2 folder has max_len 512 and head_max 256. 40 The 0.5.2 config.json has max_len 512, the length the model was trained at. 40 41 Longer inputs are now cut at 512 tokens. 41 A bare .onnx file with no config.json gets MaxLen 1024 and HeadMax 256. 42

Watch out: a long state is truncated to fit max_len. 43

Put the part the question is about near the start of a long state.

Calibration

Section titled “Calibration”

Each model folder carries its own calibration temperature for choice, score and noul. 44 45 A model folder whose config.json has no temperature is refused. 46 Temperatures must be > 0. 47 Each logit is divided by its type’s temperature before a softmax. 48 The training job writes the temperatures into its eval report, and make_model_folder.py copies them into the folder’s config.json. 49 50 51 52

A temperature cannot fix confident mistakes. 53 On model 0.5.0, right answers and confidently wrong answers sat at the same raw margin. 54 55

Default thresholds in jevx

Section titled “Default thresholds in jevx”
you get the agent should
noul at or above 0.8, or confidence at or above 0.6 for a choice or score act on it 56
noul at or below 0.2 act on the no 57
anything in between rewrite the question more narrowly, check it itself, or ask the user 58
an HTTP error report that the call failed; never treat it as a no 59

These are the jevx CLI’s default thresholds. 60 In jevx code the built-in defaults are Yes 0.8, No 0.2 and MinConfidence 0.6. 61

Writing good questions

Section titled “Writing good questions”

When the answer is unsure, the fix is usually a narrower question. 62 Name the exact thing you are looking for, say what counts as yes, and let code do arithmetic and dates. 63

An example of letting code do the dates: compute the candidate due dates in code, and let the model only pick one. 64 The model judges text, it does not do date arithmetic. 65 On this recipe model 0.5.2 was wrong, and confidently: it picked the issue date (0.90) instead of the due date. 66 67

The bench caveat holds: concrete, high-signal questions work. 68 Fuzzy, judgment-heavy ones (answerable, lead quality, email urgency) fail on the local model (model version not recorded). 68

Your inputs will score differently: copy the pattern, not the numbers, and test on your own data. 69

Sources

Section titled “Sources”

Footnotes

Section titled “Footnotes”

  1. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L37–39 ↩

  2. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L37–40 ↩ ↩2

  3. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L37–41 ↩

  4. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L117–118 ↩

  5. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L117–121 ↩

  6. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L98–100 ↩ ↩2 ↩3

  7. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L101–114 ↩

  8. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L39 ↩

  9. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L107–114 ↩

  10. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L244–245 ↩

  11. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L244–246 ↩

  12. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L82–86 ↩

  13. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L275–286 ↩

  14. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L218–237 ↩

  15. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L88–93 ↩

  16. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L41 ↩ ↩2

  17. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L94–97 ↩

  18. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L238–243 ↩ ↩2

  19. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L229–234 ↩

  20. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L224–231 ↩

  21. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L263–268 ↩

  22. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L218–233 ↩

  23. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L235–246 ↩

  24. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L248–253 ↩ ↩2

  25. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/main.go L391 ↩

  26. openjevx @ v0.5.9 (ee2a1f4) · finetuning/train/train_openjevx.py L257 ↩

  27. openjevx @ v0.5.9 (ee2a1f4) · recipes/17-several-judgements.md L13–16 ↩

  28. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L126–134 ↩

  29. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L138–139 ↩

  30. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L141–146 ↩

  31. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L147 ↩

  32. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L153–166 ↩

  33. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L167–169 ↩

  34. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L170–183 ↩

  35. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L170–181 ↩

  36. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L187–192 ↩

  37. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L127 ↩

  38. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L138–142 ↩

  39. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/model.go L30–39 ↩

  40. openjevx @ v0.5.9 (ee2a1f4) · README.md L70–72 ↩ ↩2

  41. openjevx @ v0.5.9 (ee2a1f4) · llmresults/13-v0.5.2-gate-misses.md L120–121 ↩ ↩2

  42. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L25–28 ↩

  43. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L178–181 ↩

  44. openjevx @ v0.5.9 (ee2a1f4) · README.md L70–71 ↩

  45. openjevx @ v0.5.9 (ee2a1f4) · README.md L79–80 ↩

  46. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/model.go L90–92 ↩

  47. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/model.go L93–97 ↩

  48. openjevx @ v0.5.9 (ee2a1f4) · internal/decide/decide.go L199–217 ↩

  49. openjevx @ v0.5.9 (ee2a1f4) · finetuning/export/make_model_folder.py L1 ↩

  50. openjevx @ v0.5.9 (ee2a1f4) · finetuning/export/make_model_folder.py L6–9 ↩

  51. openjevx @ v0.5.9 (ee2a1f4) · finetuning/export/make_model_folder.py L53 ↩

  52. openjevx @ v0.5.9 (ee2a1f4) · finetuning/export/make_model_folder.py L64–68 ↩

  53. openjevx @ v0.5.9 (ee2a1f4) · llmresults/13-v0.5.2-gate-misses.md L18–22 ↩

  54. openjevx @ v0.5.9 (ee2a1f4) · llmresults/13-v0.5.2-gate-misses.md L3 ↩

  55. openjevx @ v0.5.9 (ee2a1f4) · llmresults/13-v0.5.2-gate-misses.md L20–21 ↩

  56. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L77–79 ↩

  57. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L80 ↩

  58. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L81 ↩

  59. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L82 ↩

  60. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L82–84 ↩

  61. jevx @ 4a4722d (4a4722d) · core/core.go L105–107 ↩

  62. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L84–85 ↩

  63. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L85 ↩

  64. openjevx @ v0.5.9 (ee2a1f4) · recipes/04-pick-a-value.md L3 ↩

  65. openjevx @ v0.5.9 (ee2a1f4) · recipes/04-pick-a-value.md L44 ↩

  66. openjevx @ v0.5.9 (ee2a1f4) · recipes/04-pick-a-value.md L24 ↩

  67. openjevx @ v0.5.9 (ee2a1f4) · recipes/04-pick-a-value.md L48 ↩

  68. jevresearch @ 4634ffa (4634ffa) · jevresearch/USECASES.md L50 ↩ ↩2

  69. openjevx @ v0.5.9 (ee2a1f4) · recipes/README.md L50 ↩