Skip to main content

MCP-compatible guardrails server basing on eva-judge for production runtime.

Quick Start​

git clone https://github.com/eva-llm/eva-guard
cd eva-guard
nvm use
pnpm i
pnpm run server

API​

GET /list-tools​

Returns the list of available remote tools and the schema how to call.

POST /call-tool​

API is called according to schema from GET /list-tools, for example:

curl -X POST http://localhost:3000/call-tool \
  -H "Content-Type: application/json" \
  -d '{
    "name": "G-Eval",
    "arguments": {
      "prompt": "What is the capital of France?",
      "criteria": "question should be correct"
    }
  }'

Response Example:

{
  "reason": "The question is grammatically correct...",
  "passed": true,
  "score": 1
}

Settings​

NOTE! It uses temperature=0 to be closer to determenistic result.

Environment Variables​

  • PROVIDER - default openai;
  • MODEL - default gpt-4.1-mini;
  • LLM_PROVIDER_CONCURRENCY - size of workers pool to call LLM provider, default 100;
  • REDIS_URL- if defined it uses Redis as permanent storage of G-Eval / B-Eval Evaluation Steps.