Redraven
Redraven Quickstart
Run a Redraven evaluation from Python and read the result.
Use Redraven to generate or run red-team evaluations against your AI system. Your LLM key stays in your process; the SDK submits generated responses back to Redraven for scoring.
Install
pip install redravenor:
uv add redravenConfigure
export REDRAVEN_API_KEY="rr_..."
export REDRAVEN_ORGANIZATION_ID="00000000-0000-0000-0000-000000000000"
export REDRAVEN_BASE_URL="https://app.redraven.fireraven.ai"Run an Existing Test
import asyncio
import redraven
async def my_llm(prompt: str, messages: list[dict] | None = None) -> str:
# Call your own provider here.
_ = messages
return f"echo: {prompt}"
async def main():
async with redraven.Client() as client:
run = await client.call_agent(
test_id="<your-test-id>",
llm=my_llm,
concurrency=4,
retries=2,
)
await client.wait_for_evaluation_ready(test_id="<your-test-id>")
result = await client.get_eval_summary(
test_id="<your-test-id>",
expected_cases=run.expected_cases,
allow_partial=True,
)
print(result.state, result.received, result.failed)
asyncio.run(main())Read Outputs Over HTTP
After evaluation, use the HTTP API to export metrics, recommendations, and reports:
curl -sS \
-H "X-API-Key: $REDRAVEN_API_KEY" \
-H "X-Organization-Id: $REDRAVEN_ORGANIZATION_ID" \
"$REDRAVEN_BASE_URL/api/v1/tests/$TEST_ID/results"