Turn expensive prompts into cheap fine-tuned models.

Hosted App - Running Locally - Experiments

• Experiment

  • Bulk-test wide-reaching scenarios using code templating.
  • Seamlessly translate prompts across different model APIs.
  • Tap into autogenerated scenarios for fresh test perspectives.

• Fine-Tune (Beta)

  • Easy integration with OpenPipe's SDK in both Python and JS.
  • Swiftly query logs using intuitive built-in filters.
  • Export data in multiple training formats, including Alpaca and ChatGPT, with deduplication.

These are sample experiments users have created that show how OpenPipe works. Feel free to fork them and start experimenting yourself.

• Twitter Sentiment Analysis
• Reddit User Needs
• OpenAI Function Calls
• Activity Classification

• GPT 3.5 Turbo
• GPT 3.5 Turbo 16k
• GPT 4

• 7b chat
• 13b chat
• 70b chat

• Open-Orca/OpenOrcaxOpenChat-Preview2-13B
• Open-Orca/OpenOrca-Platypus2-13B
• NousResearch/Nous-Hermes-Llama2-13b
• jondurbin/airoboros-l2-13b-gpt4-2.0
• lmsys/vicuna-13b-v1.5
• Gryphe/MythoMax-L2-13b
• NousResearch/Nous-Hermes-llama-2-7b

• Claude 1 Instant
• Claude 2

1. Install Postgresql.
2. Install NodeJS 20 (earlier versions will very likely work but aren't tested).
3. Install pnpm: npm i -g pnpm
4. Clone this repository: git clone https://github.com/openpipe/openpipe
5. Install the dependencies: cd openpipe && pnpm install
6. Create a .env file (cp .env.example .env) and enter your OPENAI_API_KEY.
7. Update DATABASE_URL if necessary to point to your Postgres instance and run pnpm prisma migrate dev to create the database.
8. Create a GitHub OAuth App, set the callback URL to <your local instance>/api/auth/callback/github, e.g. http://localhost:3000/api/auth/callback/github.
9. Update the GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET values from the Github OAuth app (Note: a PR to make auth optional when running locally would be a great contribution!).
10. Start the app: pnpm dev.
11. Navigate to http://localhost:3000

1. Copy your .env file to .env.test.
2. Update the DATABASE_URL to have a different database name than your development one
3. Run DATABASE_URL=[your new datatase url] pnpm prisma migrate dev --skip-seed --skip-generate
4. Run pnpm test