Git Product home page Git Product logo

ai-tdd's Introduction

aitdd logo

AI + TDD

Follow the bird

GPT powered CLI for TDD

You write the test — GPT writes the code until it passes the test ✅

Prompting GPT with a test suite makes it write code impressively accurate


Setup

AITDD runs on Bun, make sure you first install the latest Bun version.

  1. Install AITDD globally as a CLI:

    curl -sSL https://raw.githubusercontent.com/di-sukharev/AI-TDD/master/install.sh | bash
  2. Get your API key from OpenAI. Make sure you add payment details, so API works.

  3. Set the key to AITDD config:

    aitdd config set OPENAI_API_KEY <your_api_key>

    Your api key is stored locally in ~/.aitdd/config config file and is not stored anywhere in any other way.

  4. Set the command to run the tests:

    aitdd config set RUN_TESTS "npm run test"

Your api key is stored locally in ~/.aitdd/config config file and is not stored anywhere in any other way.

Example

Here is a frontend test suite written in Jest + Testing Library. Yes, AITDD easily passes even frontend tests:

import React from "react";
import { rest } from "msw";
import { setupServer } from "msw/node";
import { render, fireEvent, waitFor, screen } from "@testing-library/react";
import "@testing-library/jest-dom";
import Fetch from "../fetch";

const server = setupServer(
  rest.get("/greeting", (req, res, ctx) => {
    return res(ctx.json({ greeting: "hello there" }));
  })
);

beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

test("loads and displays greeting", async () => {
  render(<Fetch url="/greeting" />);

  fireEvent.click(screen.getByText("Load Greeting"));

  await waitFor(() => screen.getByRole("heading"));

  expect(screen.getByRole("heading")).toHaveTextContent("hello there");
  expect(screen.getByRole("button")).toBeDisabled();
});

test("handles server error", async () => {
  server.use(
    rest.get("/greeting", (req, res, ctx) => {
      return res(ctx.status(500));
    })
  );

  render(<Fetch url="/greeting" />);

  fireEvent.click(screen.getByText("Load Greeting"));

  await waitFor(() => screen.getByRole("alert"));

  expect(screen.getByRole("alert")).toHaveTextContent("Oops, failed to fetch!");
  expect(screen.getByRole("button")).not.toBeDisabled();
});

Ready? Here is the code generated by AITDD (GPT-4) to pass the test:

import React, { useState } from "react";

function Fetch({ url }) {
  const [data, setData] = useState(null);
  const [error, setError] = useState(null);
  const [loading, setLoading] = useState(false);

  async function fetchData() {
    setLoading(true);
    try {
      const response = await fetch(url);
      if (!response.ok) throw new Error("Failed to fetch");
      const data = await response.json();
      setData(data.greeting);
    } catch (error) {
      setError(error.message);
    } finally {
      setLoading(false);
    }
  }

  return (
    <div>
      {data && <h1 role="heading">{data}</h1>}
      {error && <div role="alert">{error}</div>}
      <button onClick={fetchData} disabled={loading}>
        Load Greeting
      </button>
    </div>
  );
}

export default Fetch;

Usage

You can call aitdd like this:

aitdd run

Payments

You pay for your own requests to OpenAI API. AITDD uses latest GPT model by default, check it's pricing. Maximum response tokens are set to 2000, you can adjust it via ait config set maxTokens=<number>.

I couldn't manage ChatGPT model to solve the problem. I tried to few shot it with a response example, it doesn't understand what I want. If you want to try manage it via ChatGPT — test it and open a PR 🚀

ai-tdd's People

Contributors

di-sukharev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ai-tdd's Issues

Support actually running tests and feeding back results

For non-trivial / implementations where they could be interpreted in many different ways, GPT4 will still miss in a zero-shot attempt. Perhaps you could run the test file and feed the results into a second execution of aitdd. Recursive AI coding—the dream!

Generate test from debugger in VSCode

Would be cool to make it possible in VSCode to:

  1. Generate a unit test from debugger call stack or variable (everything for Arrange, Act and Assert (AAA) pattern is already there)
  2. Slightly modify this test to satisfy new requirements
  3. Generate code using ai-tdd based on generated unit test

feature: optimize work on a single test suite with some sort of cache and smaller patch responses

i'm thinking of something like this:

  1. you send a test suite filepath
  2. lib saves the filepath and caches the file
  3. you send the same filepath again (working on a particular test suite)
  4. lib finds the difference between the cached file content and the new file content and send only diff to openAI api
  5. GPT responses with a JSON something like {type: "replace", from: "some_code_from_cached_file_content", "to": "new_content"}, other types may be append (to add line after a line), insert with {startLine: <code_string>, endLine: <code_string>} (to add code between the lines)

Request failed with status code 404

I always receive the message Error: Request failed with status code 404
The api-key is correct.
When i am sending requests via Postman it is working.

Unable to set up aitdd with my API key

I followed the straightforward README, and setup my config as such (API changed for obvious reason)

~/.aitdd/config

aitdd config set OPENAI_API_KEY sk-ManyLettersWithMixedUpperCaseAndLowerCase
aitdd config set RUN_TESTS "npm run test"

When I try to run it in the console:

➜  TiedSiren git:(main) aitdd run
│
└  'aitdd config set OPENAI_API_KEY sk-ManyLettersWithMixedUpperCaseAndLowerCase' name is invalid, it should be either 'AITDD CONFIG SET OPENAI_API_KEY SK-MANYLETTERSWITHMIXEDUPPERCASEANDLOWERCASE' or it doesn't exist.

│
└  Manually fix the '.env' file or global '~/.aitdd/config' config file.

I am not sure where I got it wrong. How can I fix it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.