Skip to content
OpenJevX wiki
Every statement on this page is cited and was checked against jev-cloud origin/main (10a744c) on 2026-10-04.

AWS Marketplace

Two container products

Section titled “Two container products”

Jev runs the OpenJevX server on Amazon ECS Fargate in your account. 1 openjevx Server runs on Amazon ECS Fargate in your account and serves your own OpenJevX-format model. 2

product what it serves delivery
Jev the OpenJevX server behind an API-key gate, with a built-in base model 1 3 Amazon ECS Fargate via CloudFormation 4
openjevx Server your own OpenJevX-format model from your S3 bucket, with no bundled model 2 5 ECS Fargate via CloudFormation (openjevx-server-ecs.cfn.yaml, required parameter ModelUri) 6

Jev on EKS uses a CloudFormation stack that creates an EKS cluster, a versioned-models S3 bucket and a pod identity, and then you install the Jev Helm chart. 7

The server image in both products is version 0.5.6. 8 9

openjevx Server needs an OpenJevX-format model

Section titled “openjevx Server needs an OpenJevX-format model”

You need an OpenJevX-format model in S3 in the same account before you launch. 10 It is a model.tar(.gz) holding model/ (openjevx.w8.onnx, config.json, tokenizer.json), or a prefix holding those three files. 11 openjevx Server serves OpenJevX-format models only, not arbitrary ONNX models. 12 Without a readable model the server does not start. 13

Launch parameters

Section titled “Launch parameters”

Launch takes about 10 minutes. 14 Choose “Continue to Launch”, then the ECS Fargate delivery option, then Launch CloudFormation. 15 16

parameter meaning
ModelUri s3://<bucket>/<path>/model.tar or s3://<bucket>/<prefix>/; the task gets read access to that bucket only 17
ApiKey 24+ characters; clients send it as Authorization: Bearer <ApiKey> on every /v1 call 18
DashboardPassword 12+ characters, no double quote or backslash 19
Scheme internal (recommended; reachable only inside the new VPC) or internet-facing 20
AllowedCidr who may reach the load balancer; default the VPC only 21
TaskCpu / TaskMemory 2048 / 4096 recommended 22
DesiredCount number of tasks, 1 to 10, default 1 23

The stack sets one inference thread per vCPU: 1 for 1024 CPU units, 2 for 2048, 4 for 4096. 24

When the stack is CREATE_COMPLETE, the Outputs tab shows Endpoint. 25 The Jev stack also outputs ModelsBucket. 26 The model loads 1-2 minutes after the task starts. 26

Use

Section titled “Use”
Terminal window
curl -s <Endpoint>/health
curl -s <Endpoint>/v1/systemone -H "Authorization: Bearer <ApiKey>" \
  -d '{"state": {"order_total_rs": 501}, "questions": {"discount": {"type": "noul",
       "instructions": "Does the order get the discount? Orders over Rs 500 get 10% off."}}}'

Every answer returns a probability per option. 27 Calls without the right key get 401. 28 /health is open; every /v1/* path needs Authorization: Bearer <JEV_API_KEY>. 29

The request format is on the API reference page.

Your own model in S3

Section titled “Your own model in S3”

For Jev, the service loads s3://<ModelsBucket>/models/current/model.tar, or the built-in base model while that does not exist. 30 Keep versions at models/<version>/model.tar, and promote or roll back by copying one to models/current/model.tar. 31 The server swaps within a minute between requests, with no restart, and rejects a bad upload. 32 /health shows the version. 32

To update the model in openjevx Server, overwrite the object at ModelUri. 33 The server checks it every minute and swaps to the new model between requests. 34 It keeps the previous model in memory. 35 A bad upload is rejected and the current model keeps serving. 36 Roll back by uploading the previous version again; S3 versioning is recommended. 37

To make a model of your own, see Fine-tune.

Sizing

Section titled “Sizing”

The model is a 598 MB 8-bit model that uses about 1.6 GB RAM, on CPU only. 38 In vendor tests on AWS Fargate, server-side, per vCPU: 0.3-1.05 s for a short decision of 50 tokens, and 3.6-10.5 s for a 512-token input. 39

Model version not recorded for these figures.

The range follows the host CPU that Fargate assigns. 39 Add tasks (DesiredCount) for throughput. 40 You pay only for your AWS resources: the Fargate task, the load balancer and logs. 41

On EKS, each Jev pod requests 1.5 vCPU and 2.5 GiB. 42 A c6i.xlarge node fits two pods, which a zero-downtime rollout needs; a c6i.large fits only one. 42

Removal

Section titled “Removal”

For openjevx Server, delete the CloudFormation stack; your model bucket is never touched. 16 43 For Jev, delete the stack; the models bucket is kept, so empty and delete it yourself if you no longer need your models. 44

Security

Section titled “Security”

The image runs as non-root on a distroless base. 45 46 It makes no outbound calls except reading your model from S3. 47 There is no telemetry, and request bodies are never logged. 47 Secrets live in AWS Secrets Manager. 48 The gate refuses to start when JEV_API_KEY is not set, unless JEV_ALLOW_NO_API_KEY=1. 49 The gate checks the Bearer key with a constant-time compare. 50

Warning: the Endpoint output is http:// plus the load balancer’s DNS name. 51 The load balancer takes calls on port 80. 52 With Scheme set to internet-facing, the load balancer is public; the API key still applies. 53

Support: [email protected]. 54

Sources

Section titled “Sources”

Footnotes

Section titled “Footnotes”

  1. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L2–5 ↩ ↩2

  2. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/openjevx-server-ecs.cfn.yaml L2–5 ↩ ↩2

  3. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L14–15 ↩

  4. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/listing.json L48–59 ↩

  5. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/listing-openjevx-server.json L7 ↩

  6. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/listing-openjevx-server.json L40–48 ↩

  7. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-eks.cfn.yaml L2–5 ↩

  8. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/listing.json L44–46 ↩

  9. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/listing-openjevx-server.json L37–39 ↩

  10. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L1–4 ↩

  11. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L2–3 ↩

  12. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L3–4 ↩

  13. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L15–16 ↩

  14. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L1 ↩

  15. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L2 ↩

  16. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L7 ↩ ↩2

  17. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L7–9 ↩

  18. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L10 ↩

  19. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L11 ↩

  20. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L12 ↩

  21. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L13 ↩

  22. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L14 ↩

  23. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L53–57 ↩

  24. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L60–65 ↩

  25. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L15 ↩

  26. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L6 ↩ ↩2

  27. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L23 ↩

  28. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L24 ↩

  29. jev-cloud @ origin/main (10a744c) · image/gate/main.go L5–6 ↩

  30. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L15 ↩

  31. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L16 ↩

  32. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L17 ↩ ↩2

  33. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L26–27 ↩

  34. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L27 ↩

  35. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L27–28 ↩

  36. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L28–29 ↩

  37. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L29 ↩

  38. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L35 ↩

  39. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L36–37 ↩ ↩2

  40. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L34–37 ↩

  41. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L38 ↩

  42. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-eks.cfn.yaml L16 ↩ ↩2

  43. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L40–41 ↩

  44. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions.txt L47 ↩

  45. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L44 ↩

  46. jev-cloud @ origin/main (10a744c) · image/Dockerfile L42–46 ↩

  47. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L45 ↩ ↩2

  48. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L46 ↩

  49. jev-cloud @ origin/main (10a744c) · image/gate/main.go L57–60 ↩

  50. jev-cloud @ origin/main (10a744c) · image/gate/main.go L5 ↩

  51. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L236–238 ↩

  52. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L26–30 ↩

  53. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/jev-ecs.cfn.yaml L21–25 ↩

  54. jev-cloud @ origin/main (10a744c) · marketplace/aws/listing/usage-instructions-openjevx-server.txt L47 ↩