Skip to content
OpenJevX wiki
Every statement on this page is cited and was checked against jev-cloud origin/main (10a744c), openjevx v0.5.9 (ee2a1f4) on 2026-10-04.

Model from object storage

What model can point at

Section titled “What model can point at”

"model" (or -model / OPENJEVX_MODEL) can be an object-store URL: s3://bucket/prefix/ holding the three files, or one s3://bucket/model.tar.gz (or .tar) with them at its root or under model/. 1

The code also treats a key ending in .tgz as an archive; any other key is read as a folder prefix. 2

The model is one folder; nothing is embedded in the binary. 3

gs:// and azblob:// are registered and say “not supported yet”. 4

Any other :// value fails with only s3:// URLs and local paths are supported. 5

For a folder prefix the server checks the graph, config.json and tokenizer.json, and only the third one, tokenizer.json, may be missing. 6 7

Terminal window
OPENJEVX_MODEL=s3://models/current/ ./openjevx

Credentials and IAM

Section titled “Credentials and IAM”

The server downloads with the AWS default credential chain (env, profile, EC2 instance role, EKS IRSA / Pod Identity, ECS task role); no keys go in the config. 8

It only calls GetObject and HeadObject, both covered by s3:GetObject. 9

The role needs s3:GetObject on the objects and s3:ListBucket on the prefix, so that a missing object is a 404. 10

The region is AWS_REGION or the profile’s when set; otherwise the server starts at us-east-1 and follows the bucket region S3 names in its x-amz-bucket-region header, which needs no extra permission. 11

It does not call GetBucketLocation, which would need s3:GetBucketLocation. 12

The jev-cloud contract gives the server’s role this least-privilege, read-only policy: 13 14

{ "Version": "2012-10-17", "Statement": [
  { "Effect": "Allow", "Action": ["s3:GetObject"], "Resource": "arn:aws:s3:::<bucket>/models/current/*" },
  { "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": "arn:aws:s3:::<bucket>",
    "Condition": { "StringLike": { "s3:prefix": ["models/current/", "models/current/*"] } } }
]}

Gotcha

On a 401 or 403 the server reports access denied to s3://<bucket> (the server's role needs s3:GetObject on the model objects and s3:ListBucket on the prefix). 15 Only a 404 counts as “no model there yet”, which is what triggers the fallback; a 403 is an error. 16 17 18 With nothing cached, that error is returned as is, so the server fails to start. 19 20 21

Cache and sha256

Section titled “Cache and sha256”

The server downloads the model, verifies it, and serves it from a local cache. 8

The cache defaults to the user cache dir/openjevx/models, else $TMPDIR/openjevx-models; set it with model_cache / OPENJEVX_MODEL_CACHE. 22

Downloads go into <cache>/<hash of url>/v-<hash of etags>-<time>/ with a source.json manifest, are verified, then renamed into place. 23

Pointer files current and previous are written with rename; other versions are deleted. 24

If the cache folder cannot be created, the error tells you to set model_cache / OPENJEVX_MODEL_CACHE to a writable folder. 25

The jev-cloud container runs as uid 10001 with a writable /tmp, so the contract asks the server to allow OPENJEVX_MODEL_CACHE=/tmp/jev-cache. 26

Verify: config.json’s sha256 must match the graph, and the pin (if set) must match. 27

A mismatch fails loudly. 27

The pin model_sha256 / OPENJEVX_MODEL_SHA256 is the sha256 of the .tar.gz, or of the folder’s openjevx.w8.onnx. 28

Start, reload and hot swap

Section titled “Start, reload and hot swap”

Start: a cached model whose ETags still match starts without downloading. 29

If the store is unreachable, denies access or holds a bad upload, a valid cache is served with a WARNING; with no valid cache the server refuses to start. 30

model_reload / OPENJEVX_MODEL_RELOAD defaults to off. 31

Set, it checks the ETag this often, for example 5m, and it must be at least 10s. 31

A model_reload below 10s stops the server at startup with at least 10s. 32 33

Reload: on a new ETag the server downloads into a new cache folder and verifies it. 34

It then opens a new session, swaps it in between requests, and keeps the previous folder for rollback. 35

A bad upload is logged and the old model keeps serving. 36

The swap happens under the inference lock, and the old session is destroyed after it. 37

The store’s ETag is the version identity; a multipart re-upload of the same bytes looks new and is downloaded once more. 38

GET /health reports version, sha256, source (the s3 URL or local path), fallback, loaded_at, dir, remote_version (the ETags), offline, and previous (the model it replaced). 39

Fallback for an empty bucket

Section titled “Fallback for an empty bucket”

Empty bucket (fresh deploy): the fallback folder is served, and the prefix is checked every model_reload (every minute if reload is off) until a model appears. 40

The fallback is model_fallback / OPENJEVX_MODEL_FALLBACK, by default the model/ lookup next to the executable. 41

In the Docker image that default is /app/model. 42

With an empty bucket and no fallback model, startup fails with ..., and no fallback model: .... 43

MinIO, Ceph and Cloudflare R2

Section titled “MinIO, Ceph and Cloudflare R2”

Set the endpoint with AWS_ENDPOINT_URL_S3 (or AWS_ENDPOINT_URL) and turn on path-style (AWS_S3_USE_PATH_STYLE=true or "model_s3_path_style": true) unless the store has wildcard bucket DNS. 44

For R2, also set AWS_REGION=auto. 45

Terminal window
AWS_ENDPOINT_URL_S3=http://minio:9000 AWS_S3_USE_PATH_STYLE=true OPENJEVX_MODEL=s3://models/current/ ./openjevx

Promote and rollback layout

Section titled “Promote and rollback layout”

The jev-cloud contract keeps one bucket per deployment, in the customer’s own account. 46

The server loads models/current/, which holds openjevx.w8.onnx, config.json and tokenizer.json. 47

Every trained or delivered version lives in models/<version>/, immutable once written, with an eval_report.json the server does not read. 48

models/history.log gets one line per promote; the server does not read it. 49

s3://<bucket>/
  models/
    current/            <- model = s3://<bucket>/models/current/
    <version>/          <- immutable once written
    history.log
  training/
    <job>/input/decisions.csv
    <job>/output/model.tar.gz

Promote = copy the 3 files of models/<version>/ into models/current/; the server’s ETag reload or a rolling restart picks it up. 50

Rollback = promote the previous version again. 51

Gotchas

Section titled “Gotchas”

Gotcha

The pin (model_sha256) and reload are at odds: a pinned server rejects every new upload. 52 Pin for a fixed version; use reload for a moving current/. 52

Gotcha

A promote that changes only config.json still downloads the graph again (about 600 MB). 53

Gotcha

ListBucket is needed so a missing object returns 404, not 403. 54

Sources

Section titled “Sources”

Footnotes

Section titled “Footnotes”

  1. openjevx @ v0.5.9 (ee2a1f4) · README.md L88–89 ↩

  2. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/source.go L97–102 ↩

  3. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L18–19 ↩

  4. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L21–22 ↩

  5. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/source.go L86–89 ↩

  6. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/source.go L106–108 ↩

  7. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/source.go L119–124 ↩

  8. openjevx @ v0.5.9 (ee2a1f4) · README.md L89–91 ↩ ↩2

  9. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/s3.go L18–20 ↩

  10. openjevx @ v0.5.9 (ee2a1f4) · README.md L91–92 ↩

  11. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/s3.go L20–22 ↩

  12. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L31 ↩

  13. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L1 ↩

  14. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L29–38 ↩

  15. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/s3.go L115–116 ↩

  16. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/s3.go L113–116 ↩

  17. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/cache.go L138–141 ↩

  18. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/source.go L101–107 ↩

  19. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/cache.go L116–126 ↩

  20. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/source.go L115–117 ↩

  21. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/main.go L122–125 ↩

  22. openjevx @ v0.5.9 (ee2a1f4) · README.md L98 ↩

  23. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L25–27 ↩

  24. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L26–27 ↩

  25. openjevx @ v0.5.9 (ee2a1f4) · internal/modelsrc/cache.go L62–64 ↩

  26. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L48–49 ↩

  27. openjevx @ v0.5.9 (ee2a1f4) · README.md L108 ↩ ↩2

  28. openjevx @ v0.5.9 (ee2a1f4) · README.md L97 ↩

  29. openjevx @ v0.5.9 (ee2a1f4) · README.md L109 ↩

  30. openjevx @ v0.5.9 (ee2a1f4) · README.md L109–110 ↩

  31. openjevx @ v0.5.9 (ee2a1f4) · README.md L99 ↩ ↩2

  32. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/source.go L73–75 ↩

  33. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/main.go L112–115 ↩

  34. openjevx @ v0.5.9 (ee2a1f4) · README.md L113 ↩

  35. openjevx @ v0.5.9 (ee2a1f4) · README.md L113–114 ↩

  36. openjevx @ v0.5.9 (ee2a1f4) · README.md L114 ↩

  37. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/main.go L140–152 ↩

  38. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L41–42 ↩

  39. openjevx @ v0.5.9 (ee2a1f4) · README.md L115–116 ↩

  40. openjevx @ v0.5.9 (ee2a1f4) · README.md L111–112 ↩

  41. openjevx @ v0.5.9 (ee2a1f4) · README.md L100 ↩

  42. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L29–30 ↩

  43. openjevx @ v0.5.9 (ee2a1f4) · cmd/openjevx/source.go L102–106 ↩

  44. openjevx @ v0.5.9 (ee2a1f4) · README.md L103–105 ↩

  45. openjevx @ v0.5.9 (ee2a1f4) · README.md L106 ↩

  46. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L4 ↩

  47. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L7–11 ↩

  48. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L6–14 ↩

  49. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L15 ↩

  50. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L20–21 ↩

  51. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L21 ↩

  52. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L39–40 ↩ ↩2

  53. openjevx @ v0.5.9 (ee2a1f4) · docs/adr/0012-model-from-object-store.md L37–38 ↩

  54. jev-cloud @ origin/main (10a744c) · docs/model-s3-contract.md L39 ↩