entropy-research / devon Goto Github PK

Devon: An open-source pair programmer

License: GNU Affero General Public License v3.0

Python 86.75% Dockerfile 0.04% Shell 0.18% TypeScript 12.49% JavaScript 0.28% CSS 0.24% HTML 0.01%

code-generation llama3 agent agent-based-framework ai-developer ai-software ai-software-engineer code-assistant developer-tools gpt-4

devon's Introduction

Devon: An open-source pair programmer

demo.mp4

How do y'all ship so quickly?

← We have a community-driven Dev Team for this repo. Come join us! It's great.

Installation

Prerequisites

node.js and npm
pipx, if you don't have this go here
API Key (just one is required)
- Anthropic
- OpenAI
- Groq (not released in package yet, run locally)

We're currently working on supporting Windows! (Let us know if you can help)

Installation commands

To install, simply run:

curl -sSL https://raw.githubusercontent.com/entropy-research/Devon/main/install.sh | bash

Or to install using pipx + npm:

# For the backend
pipx install devon_agent

# For the terminal ui
npm install -g devon-tui

# And for the main UI
npx devon-ui

If you already have devon_agent or devon-tui installed, update it by running:

npm uninstall -g devon-tui
npm install -g devon-tui

pipx install --force devon_agent

This installs the Python backend, and the cli command to run the tool

Thats it! Happy building :)

Running the agent

Then to run the main ui, the command is:

npx devon-ui

It's that simple.

For the terminal ui:

Navigate to your project folder and open the terminal.
Set your Anthropic API or OpenAI API key as an environment variable:

export ANTHROPIC_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#OR

export OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#OR

export GROQ_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then to run the terminal-ui, the command is:

devon-tui

It's as easy as that.

Note

Don't worry, the agent will be able to only access files and folders in the directory you started it from. You can also correct it while it's performing actions.

To run in debug mode, the command is:

devon-tui --debug

To run in local mode:

Warning

The current version of local model support is not mature, proceed with caution, and expect the performance to degrade significantly compared to the other options.

Get deepseek running with ollama
Start the local ollama server by running

ollama run deepseek-coder:6.7b

Then configure devon to use the model

devon-tui configure

Configuring Devon CLI...
? Select the model name: 
  claude-opus 
  gpt4-o 
  llama-3-70b 
❯ ollama/deepseek-coder:6.7b

And finally, run it with:

devon-tui --api_key=FOSS

For a list of all commands available:

devon-tui --help

Features

Multi-file editing
Codebase exploration
Config writing
Test writing
Bug fixing
Architecture exploration
Local Model Support

Limitations

Minimal functionality for non-Python languages
Sometimes have to specify the file where you want the change to happen
Local mode is not good right now. Please try to avoid using it.

Progress

This project is still super early and we would love your help to make it great!

Current goals

Multi-model support
- Claude 3 Opus
- GPT4-o
- Groq llama3-70b
- Ollama deepseek-6.7b
- Google Gemini 1.5 Pro
Launch plugin system for tool and agent builders
Improve our self-hostable Electron app
Set SOTA on SWE-bench Lite

View our current thoughts on next steps here

Star history

Past milestones

June 14, 2024 - Launch Electron UI v0.0.13
June 1, 2024 - Devon V2 Beta Electron UI
May 19, 2024 - GPT4o support + better interface support v0.1.7
May 10, 2024 - Complete interactive agent v0.1.0
May 10, 2024 - Add steerability features
May 8, 2024 - Beat AutoCodeRover on SWE-Bench Lite
Mid April, 2024 - Add repo level code search tooling
April 2, 2024 - Begin development of v0.1.0 interactive agent
March 17, 2024 - Launch non-interactive agent v0.0.1

Current development priorities

Improve context gathering and code indexing abilities ex:
- Adding memory modules
- Improved code indexing
Add alternative models and agents to:
- a) Reduce end user cost and
- b) Reduce end user latency
Electron app
- Better code diff view
- Timeline interface
- Send user file events/changes to Devon

How can I contribute?

Devon and the entropy-research org are community-driven, and we welcome contributions from everyone! From tackling issues to building features to creating datasets, there are many ways to get involved:

Core functionality: Help us develop the core agents, user experience, tool integrations, plugins, etc.
Research: Help us research agent performance (including benchmarks!), build data pipelines, and finetune models.
Feedback and Testing: Use Devon, report bugs, suggest features, or provide feedback on usability.

For details, please check CONTRIBUTING.md.

If you would like to contribute to the project, please join the discord: Discord

Feedback

We would love feedback! Feel free to drop us a note on our Discord in the #feedback channel, or create issues!

We collect basic event type (i.e. "tool call") and failure telemetry to solve bugs and improve the user experience, but if you want to reach out, we would love to hear from you!

To disable telemetry, set the environment variable DEVON_TELEMETRY_DISABLED to true

export DEVON_TELEMETRY_DISABLED=true

Community

Join our Discord server and say hi! Discord

License

Distributed under the AGPL License. See LICENSE for more information.

devon's People

Contributors

Stargazers

Watchers

Forkers

jobygitgud christopherhwood josephrp diogodsa squanto 3ool0ne touristshaun anishthite ruthvik-17 yalton twilwa latteliem tcoyze anomius ototao pasula007 teamtonic objectjosh dotpyu rminz ramzimalhas tomchapin jthodge zeroxclem tocasoft lalomorales22 ahmetkilinc miscarriage87 orinocoz cocobeach quickpanda iamkomen cyborgmarina professeurfalken gmh5225 thomascherickal stophobia veryvanya edwardshez nalaso mdwoicke joemocha fiditenemini biomassives wahaze businistry goddmood dependify decentralised-ai sitais ach1916 antoninolorenzo cwwang2 keppler55b vital121 styner2023 gronioss nazariyv mz0in bixoryai kustomzone agent-uncommon seshakiran babybirdprd mihir1003 hbcbh1999 ghenghis zb0413 luisfx fergusmeiklejohn docxology xc0r nxtrevgen ifitsmanu gptog mbrukman taocao kazhuyo feodorfitsner f901107 zhanglin77 mustafaskyer mayi140611 yinfi gino2013 zhangliaoxiaoqin umer4447 lorna988 sunilgitb lamardealmaker jaroslav-ks ghgoodreau ukaserge jansystemic solijafari pabg92 hamzaaylak mr-narender osheitchang sdan

devon's Issues

agent attempts to open files that do not exist, edit loops with indentation error

Installed and tried my first prompt:

"I'd like to improve the test coverage in this project for the python files in src/shared"

Agent decided to try to open a file that does not exist and then hangs:

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                                                                                                                                        |
|   The user requested improving test coverage for the Python files in the src/shared directory. From the directory listing, I can see there are Python files in src/shared, but no test files specifically   |
|    for that directory.                                                                                                                                                                                      |
|                                                                                                                                                                                                             |
|   To improve test coverage, I should:                                                                                                                                                                       |
|                                                                                                                                                                                                             |
|   1. Open and review the Python files in src/shared to understand what functionality needs testing                                                                                                          |
|   2. Create a new test file in the tests/shared directory for the src/shared code                                                                                                                           |
|   3. Write test cases that exercise the key functionality and edge cases of the src/shared code                                                                                                             |
|   4. Run the new tests to make sure they pass                                                                                                                                                               |
|   5. Check the test coverage report to confirm coverage has improved                                                                                                                                        |
|   6. Submit the changes                                                                                                                                                                                     |
|                                                                                                                                                                                                             |
|   I'll start by opening the Python files in src/shared to see what needs testing.                                                                                                                           |
|                                                                                                                                                                                                             |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-------------------------------------------------------------------------------------------------------------------------------------------------------+
|   Devon ran: Running command:                                                                                                                         |
|   open_file "src/shared/utils.py"                                                                                                                     |
|                                                                                                                                                       |
|   > Failed to open file: /Users/rwalters/Documents/GitHub.nosync/python-robb/src/shared/utils.py. Error: Could not open file, file does not exist:    |
|   /Users/rwalters/Documents/GitHub.nosync/python-robb/src/shared/utils.py                                                                             |
+-------------------------------------------------------------------------------------------------------------------------------------------------------+
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                               |
|   Waiting for Devon...                                                                                                                                                                                        |
|                                                                                                                                                                                                               |
|                                                                                                                                                                                                               |
|                                                                                                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I hit rate limit using Claude 3

initializing error, takes alot of time

when i type devon into my cmd it keeps giving me initializing for 3 hours and still the same.. iam using windows 10 any help ?

GROQ not implemented

error:
Please provide an API key using the --api_key option or by setting OPENAI_API_KEY or ANTHROPIC_API_KEY.

The code also shows that the check does not include GROQ_API_KEY

Regression Testing Spec document

Basically we want to be able to define some testing spec document with N steps and test points for regression testing.

This task should:

Implement regression tests for tools and model (in headless mode)
Implement a system by which the model can know which steps to execute and can be tested on its execution of those steps

Multi-Model Config

Goal here I think is to make sure that we can easily allow users to swap out endpoints, prompts, and models. So to complete this,
we should have a basic config system that captures:

default agent name
model name

and then, some sort of abstraction that acts as a agent:model:prompt mapping

Local llm support

This issue requires adding ollama and supported models with prompts as an alternative model choice.

Would also be nice to add default model selection to the devon command with devon set-default

In prompts, "I should perform command2" in CoT but then "command1" is used

https://github.com/entropy-research/Devon/blob/87b0461bc37d045a53acdda5470bcc29a6a72991/devon_agent/prompt.py#L150C1-L158C13

  CORRECT:
  <THOUGHT>
  ...thought 1 ...
  ...thought 2 ...
  I should perform command2 in the next step
  </THOUGHT>
  <COMMAND>
  command1 arg1
  </COMMAND>

Note command2 vs command1. While the model gets the gist of it, probably less confusing to it to make this consistent. This occurs twice in prompts.py

devon_agent not added to path

I dont know what way devon_agent is supposed to be added to path. If its a symlink or moving the file to bin. But it does not happen.
Arch Linux x86_64

Exception: Could not find api key on colab

Hi, I'm getting this error on colab using Groq API, here is a summary of the code:

!sudo apt install pipx -qy

# update Node to v18
%%bash
dpkg --configure -a
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
NODE_MAJOR=18 && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
sudo apt-get update
sudo apt-get install nodejs -qy

Install xterm then run the following commands on the terminal

>python -m venv devon
>source devon/bin/activate

curl -sSL https://raw.githubusercontent.com/entropy-research/Devon/main/install.sh | bash
>pipx ensurepath
>source ~/.bashrc
# re-login and activate "devon" env again, then:

>export GROQ_API_KEY=gsk_...

>devon_agent server

  File "/root/.local/pipx/venvs/devon-agent/lib/python3.10/site-packages/devon_agent/__main__.py", line 23, in server
    raise Exception("Could not find api key")
Exception: Could not find api key

There is no devon command, it's devon_agent, I'm doing something wrong here?

Maing websites (python http.server) prevents form continuation

Devon is waiting for the server to stop - but user can not do that. Perhaps there is an bug for killing the server.

I needed to kill devon, start again and continue each time. The old port was taken (so the server was still alive), devon was able to use next free port (8001, 8002).

Expected: devon should close the http server after user confirms that the page looks good. Or keep the server alive and do changes to the html/js files.

I can provide full log if needed.

Just wanna say thank you

Thank you for choosing Anthropic as the primary endpoint. OpenAI has recently been acting as a monster, being hostile towards open source community, proposing hardware spying etc.

I might contribute to Devon in my free time.

Thanks

It's stuck

Running into Claude limits

Error: Error code: 429 - {'type': 'error', 'error': {'type': 'rate_limit_error', 'message': 'Number of request tokens has exceeded your per-minute rate limit (https://docs.anthropic.com/claude/reference/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase.'}}

Compatibly issue, Windows 11

Hello, yeah, I got this error after running pipx install devon_agent:

(env) PS C:\Users\sonic\Downloads\Devon> pipx install devon_agent
Fatal error from pip prevented installation. Full pip output in file:
C:\Users\sonic.local\pipx\logs\cmd_2024-05-15_23.37.05_pip_errors.log

Some possibly relevant errors from pip install:
ERROR: Ignored the following versions that require a different python version: 0.0.1 Requires-Python <4.0,>=3.10; 0.0.2 Requires-Python <4.0,>=3.10; 0.0.3 Requires-Python <4.0,>=3.10; 0.0.4 Requires-Python <4.0,>=3.10; 0.0.5 Requires-Python <4.0,>=3.10; 0.0.6 Requires-Python <4.0,>=3.10; 0.0.7 Requires-Python <4.0,>=3.10; 0.1.0 Requires-Python <4.0,>=3.10; 0.1.1 Requires-Python <4.0,>=3.10; 0.1.2 Requires-Python <4.0,>=3.10; 0.1.3 Requires-Python <4.0,>=3.10; 0.1.4 Requires-Python <4.0,>=3.10; 0.1.5 Requires-Python <4.0,>=3.10; 0.1.6 Requires-Python <4.0,>=3.10
ERROR: Could not find a version that satisfies the requirement devon-agent (from versions: none)
ERROR: No matching distribution found for devon-agent

Error installing devon-agent.

Any help, is much appreciated.

devon

NOT RUN ON WSL WIDWODS

Refactor tools

By looking inside devon_agent folder it comes out that there is tools.py module and tools folder module, I don't know what are specifically the redundant parts in the first one, however I think it should be checked and tools.py should be removed, mainly for code maintainability.

Add support for using via an sdk

AutoGPT is about to expand the way we do components and give agents access to tools. If it's not a ton of trouble, I'd be interested in how you'd go about creating a Component to provide your tool to AutoGPT (and other agents we build) as a component.

Docs still in progress, things in movement and such but trying to get some conversations started on how to actually integrate other agents.

I would say don’t start coding this on our behalf until we chat :)

Agent regression tests

In order to make it easy for people to contribute, we need a good system for checking regressions.

At the moment, I propose using a simple file editing task or something where the model has to search, create and edit files. Running this with each supported model on a PR or something like that. Unsure of frequency as of now.

Devon server requires SIGKILL instead of SIGTERM

I am making this issue to bring attention to the fact that the cli sends a SIGKILL but should send a SIGTERM

Local ollama issues: loops forever on random problems: problems with saving/ovewriting, no closing fence

Running local ollama run deepseek-coder:6.7b
Kubuntu 24.04, RTX 2060 6GB VRAM

Fail to generate files
It loops forever trying to run CreateFileTool.function() but got too many arguments.

+------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                     |
|   I understand that we are creating a website about some of my indie games which were    |
|   created using Assembly language for the x86 boot sector. I need to create the          |
|   necessary files and directories for this project.                                      |
|                                                                                          |
+------------------------------------------------------------------------------------------+
+--------------------------------------------------------------------------------------------+
|   Devon ran: Running command:                                                              |
|   create_file index.html                                                                   |
|   ```                                                                                      |
|   &lt;DOCTYPE html&gt;                                                                     |
|   &lt;html&gt;                                                                             |
|   &lt;head&gt;                                                                             |
|        &lt;title&gt;My Indie Games&lt;/title&gt;                                           |
|   &lt;/head&gt;                                                                            |
|   &lt;body&gt;                                                                             |
|        &lt;h1&gt;Welcome to my indie games website&lt;/h1&gt;                              |
|        &lt;p&gt;Here you can find information about some of my favorite indie games, all   |
|    made using Assembly language for the x86 boot sector.&lt;/p&gt;                         |
|   &lt;/body&gt;                                                                            |
|   &lt;/html&gt;                                                                            |
|   ```                                                                                      |
|                                                                                            |
|   > CreateFileTool.function() takes from 3 to 4 positional arguments but 44 were given     |
+--------------------------------------------------------------------------------------------+

Generate first file but fails with updating it
After restarting and given same prompt it now creates index.html but then loops forever trying to update the file.

+-------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                      |
|   I understand the user's request to create a website about his indie games, which are    |
|   written in Assembly language and targeting an x86 boot sector. I will start by          |
|   creating a basic HTML structure for the site.                                           |
|                                                                                           |
+-------------------------------------------------------------------------------------------+
+--------------------------------------------------------------------------------------+
|   Devon ran: Running command:                                                        |
|   create_file index.html                                                             |
|   <<<                                                                                |
|   &lt;DOCTYPE html&gt;                                                               |
|   &lt;html lang="en"&gt;                                                             |
|   &lt;head&gt;                                                                       |
|        &lt;meta charset="UTF-8"&gt;                                                  |
|        &lt;title&gt;My Indie Games&lt;/title&gt;                                     |
|   &lt;/head&gt;                                                                      |
|   &lt;body&gt;                                                                       |
|        &lt;h1&gt;Welcome to my indie games website&lt;/h1&gt;                        |
|   &lt;/body&gt;                                                                      |
|   &lt;/html&gt;                                                                      |
|   >>>                                                                                |
|                                                                                      |
|   > Failed to create file: index.html. Error: Could not create file, file already    |
|   exists: /home/kj/Code/devon/games-web/index.html                                   |
+--------------------------------------------------------------------------------------+

Not valid HTML files
The file itself lacks proper TAGs:

&lt;!DOCTYPE html&gt;
&lt;html lang="en"&gt;
&lt;head&gt;
    &lt;meta charset="UTF-8"&gt;
    &lt;title&gt;My Indie Games&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
    &lt;h1&gt;Welcome to my indie games website!&lt;/h1&gt;
&lt;/body&gt;
&lt;/html&gt;

This only happens once.

Not closing fence

Third restart (clean folder) it generates valid index.html:

+--------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                       |
|   I understand the requirement of creating a website for my indie games. I will start by   |
|    creating an HTML file that serves as the main page of our website. This will include    |
|   information about the game catalog, introduction and instructions on how to run them     |
|   (in emulator or from floppy).                                                            |
|                                                                                            |
+--------------------------------------------------------------------------------------------+
+--------------------------------------------------------------------------------------------+
|   Devon ran: Running command:                                                              |
|   create_file index.html                                                                   |
|   <<<                                                                                      |
|   <!DOCTYPE html>                                                                          |
|   <html>                                                                                   |
|       <head>                                                                               |
|           <title>Indie Games Catalog</title>                                               |
|       </head>                                                                              |
|       <body>                                                                               |
|           <h1>Welcome to our Indie Games Catalog</h1>                                      |
|           <p>Here you can find a collection of indie games developed in assembly for x86   |
|    bootsector.                                                                             |
|           Each game comes with an introduction and instructions on how to run them (in     |
|   emulator or from floppy).</<｜begin▁of▁sentence｜>                                       |
|       </body>                                                                              |
|   </html>                                                                                  |
|   >>>                                                                                      |
|                                                                                            |
|   > Successfully created file /home/kj/Code/devon/games-web/index.html                     |
+--------------------------------------------------------------------------------------------+

..but then loops forever on No closing fence (>>>) :

+--------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                       |
|   I understand the requirement of creating a website for my indie games. I will start by   |
|    creating an HTML file that serves as the main page of our website. This will include    |
|   information about the game catalog, introduction and instructions on how to run them     |
|   (in emulator or from floppy).                                                            |
|                                                                                            |
+--------------------------------------------------------------------------------------------+
+------------------------------------------------------------+
|   Devon ran:                                               |
|   > No closing fence (>>>) found for multiline argument.   |
+------------------------------------------------------------+

Clean the folder (because it can not overwrite files) and it's the same thing with good html and then No closing fence loop.

Javascript Error on running Electron

Installed following the steps mentioned in README.md.

Current Behaviour

Then ran,

npx devon-ui

Then Electron window popped up and the below error message came in a box.

Context (Environment)

Ran the above on, Macbook M3

OpenAI Support

Add support for OpenAI models.

devon

not work on wsl windows 10

Ollama Support: litellm dependency issue with uvicorn

Context
I am working to add Ollama support for Devon, so I started working on the prompts in my fork inside the ollama-support branch; at the moment of writing there is the OllamaModel class implemented as follows inside devon_agent/agents/model.py:

import os
import litellm
import logging
from litellm import completion
from dataclasses import dataclass
from typing import Optional

# ...
class OllamaModel:

    def __init__(self, args: ModelArguments):
        self.args = args
        self.api_model = args.model_name  # fixed
        self.model_metadata = {
            "max_tokens": 4096,
        }

        self.api_key = "ollama"

    def query(self, messages: list[dict[str, str]], system_message: str = "") -> str:
        model_completion = completion(
            messages=[{"role": "system", "content": system_message}] + messages,
            max_tokens=self.model_metadata["max_tokens"],
            model=self.api_model,
            temperature=self.args.temperature,
            stop=["</COMMAND>"],
            api_base="http://localhost:11434"
        )

        response = model_completion.choices[0].message.content.rstrip("</COMMAND>")
        return response + "</COMMAND>"

Issue
To run litellm the following command should be runned:

litellm --model ollama_chat/model_name

It relies on litellm[proxy] library, to install it the following command should be runned:

poetry add litellm[proxy]

The problem is that it causes a dependency error with poetry, as follows:

$ poetry add litellm[proxy]

Updating dependencies

Because no versions of litellm match >1.37.16,<2.0.0
 and litellm[proxy] (1.37.16) depends on uvicorn (>=0.22.0,<0.23.0), litellm[proxy] (>=1.37.16,<2.0.0) requires uvicorn (>=0.22.0,<0.23.0).
So, because devon-agent depends on both uvicorn (^0.29.0) and litellm[proxy] (^1.37.16), version solving failed.

Possible Solution
I think that the problem could be solved by removing litellm dependency in models.py and directly using ollama for python.
If there are other solutions I would appreciate some feedback, however until a solution is found I am unable to work one ollama support.

`poetry install` loop that degenerates into `scroll_down "main.py"` loop

I asked Devon to first prepare its environment properly and it seems there was an issue with Poetry, which kept indicating there were no dependencies to install or update (which is not true). Separate from this issue, how should we reason about Devon's Python environment?

(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:15.441780+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:15.441882+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:15.442616+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:15.443602+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:16.039170+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:16.039222+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:16.039242+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:16.039260+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
INFO:     127.0.0.1:63087 - "GET /session/cli/events HTTP/1.1" 200 OK

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                                                                                              |
|   I have reviewed the `README.md` and `main.py` files. The `README.md` provides an overview of the project architecture, installation instructions, and usage.    |
|   The `main.py` file contains the main logic for setting up and running the Telegram bot.                                                                         |
|                                                                                                                                                                   |
|   To proceed, I'll set up the environment using `poetry` as instructed in the `README.md`. This involves running `poetry install` to install the dependencies     |
|   and then `poetry shell` to activate the virtual environment.                                                                                                    |
|                                                                                                                                                                   |
|   Since I cannot run the code due to missing API keys, I'll focus on identifying potential improvements by reviewing the code and documentation. I'll start by    |
|   installing the dependencies using `poetry`.                                                                                                                     |
|                                                                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+----------------------------------------------+
|   Devon ran: Running command:                |
|   poetry install                             |
|                                              |
|   > Installing dependencies from lock file   |
|                                              |
|   No dependencies to install or update       |
+----------------------------------------------+
INFO:     127.0.0.1:63088 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63090 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63101 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63102 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63103 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63105 - "GET /session/cli/events HTTP/1.1" 200 OK






****************


NAME: Devon

THOUGHT:
I have reviewed the `README.md` and `main.py` files. The `README.md` provides an overview of the project architecture, installation instructions, and usage. The `main.py` file contains the main logic for setting up and running the Telegram bot.

To proceed, I'll set up the environment using `poetry` as instructed in the `README.md`. This involves running `poetry install` to install the dependencies and then `poetry shell` to activate the virtual environment.

Since I cannot run the code due to missing API keys, I'll focus on identifying potential improvements by reviewing the code and documentation. I'll start by installing the dependencies using `poetry`.


ACTION:
poetry install


OBSERVATION: Installing dependencies from lock file

No dependencies to install or update

SCRATCHPAD: None


****************





(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:22.625999+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:22.626130+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:22.626211+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:22.626319+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:23.200441+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:23.200488+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:23.200508+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:23.200526+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
INFO:     127.0.0.1:63107 - "GET /session/cli/events HTTP/1.1" 200 OK

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                                                                                              |
|   I have reviewed the `README.md` and `main.py` files. The `README.md` provides an overview of the project architecture, installation instructions, and usage.    |
|   The `main.py` file contains the main logic for setting up and running the Telegram bot.                                                                         |
|                                                                                                                                                                   |
|   To proceed, I'll set up the environment using `poetry` as instructed in the `README.md`. This involves running `poetry install` to install the dependencies     |
|   and then `poetry shell` to activate the virtual environment.                                                                                                    |
|                                                                                                                                                                   |
|   Since I cannot run the code due to missing API keys, I'll focus on identifying potential improvements by reviewing the code and documentation. I'll start by    |
|   installing the dependencies using `poetry`.                                                                                                                     |
|                                                                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+----------------------------------------------+
|   Devon ran: Running command:                |
|   poetry install                             |
|                                              |
|   > Installing dependencies from lock file   |
|                                              |
|   No dependencies to install or update       |
+----------------------------------------------+
INFO:     127.0.0.1:63108 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63121 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63128 - "GET /session/cli/events HTTP/1.1" 200 OK

INFO:     127.0.0.1:63133 - "GET /session/cli/events HTTP/1.1" 200 OK






****************


NAME: Devon

THOUGHT:
I have successfully reviewed the `README.md` and `main.py` files. The next step is to identify potential improvements to the library. Since I cannot run the code due to missing API keys, I will focus on code review and documentation.

I'll start by examining the `main.py` file for any potential improvements or refactoring opportunities.


ACTION:
scroll_down "main.py"


OBSERVATION: Installing dependencies from lock file

No dependencies to install or update

SCRATCHPAD: None


****************





Traceback (most recent call last):
  File "/Users/elazzarin/.local/pipx/venvs/devon-agent/lib/python3.12/site-packages/devon_agent/session.py", line 321, in step_event
    response = env.tools[toolname]({
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elazzarin/.local/pipx/venvs/devon-agent/lib/python3.12/site-packages/devon_agent/tool.py", line 91, in __call__
    response = self.function(context, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elazzarin/.local/pipx/venvs/devon-agent/lib/python3.12/site-packages/devon_agent/tools/editortools.py", line 568, in function
    ctx["state"].editor.file[abs_path]["page"] = new_page_number
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable


Error routing tool call: 'NoneType' object is not subscriptable

(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.248566+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.248695+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.248770+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.248870+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
/Users/elazzarin/code/gptgenie main.py
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252091+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252180+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252231+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252273+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252313+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'EnvironmentResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252351+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ToolResponse', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252390+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
(True, {'properties': {'batch_size': 1, 'event_type': 'ModelRequest', 'message': '', 'server_context': 'FastAPI', '$lib': 'posthog-python', '$lib_version': '3.5.0', '$geoip_disable': True}, 'timestamp': '2024-05-20T15:33:28.252430+00:00', 'context': {}, 'distinct_id': '9389db25-00e0-489b-9b2a-27dd602686d9', 'event': 'SessionEventEvent'})
INFO:     127.0.0.1:63134 - "GET /session/cli/events HTTP/1.1" 200 OK

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|   Devon is thinking:                                                                                                                                              |
|   I have successfully reviewed the `README.md` and `main.py` files. The next step is to identify potential improvements to the library. Since I cannot run the    |
|   code due to missing API keys, I will focus on code review and documentation.                                                                                    |
|                                                                                                                                                                   |
|   I'll start by examining the `main.py` file for any potential improvements or refactoring opportunities.                                                         |
|                                                                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+----------------------------------------------+
|   Devon ran: Running command:                |
|   scroll_down "main.py"                      |
|                                              |
|   > 'NoneType' object is not subscriptable   |
+----------------------------------------------+

Enforce output format for local/oss models

What the title says. This should solve #55 and allow the model to at least follow an agentic loop.

Add headless mode

what the title says.

WSL installation errors

WSL 22.04 ubuntu
during installation this happens

Trouble identifying path

it appears devon has a hard time identifying the path of the items I have asked it to do. Devon ran: Running command: |
| open_file |
| "/Users/username/devopsgpt/backend/app/pkgs/tools/llm_basic.py" |
| |
| > Failed to open file: /Users/username/DevOpsGPT/username/devopsgpt/ |
| backend/app/pkgs/tools/llm_basic.py. Error: Could not open file, file |
| does not exist: /Users/username/DevOpsGPT/username/devopsgpt/backend |
| /app/pkgs/tools/llm_basic.py |
+----------------------------------------------------------------------------+
+----------------------------------------------------------------------------+
| Devon is thinking: |
| The error message indicates that the file llm_basic.py could not be |
| opened because it does not exist at the specified path. It seems there |
| was a mismatch between the path returned by the find_file command and |
| the actual file path. |
| |
| To resolve this, I should: |
| 1. Double-check the file path returned by find_file |
| 2. Verify that the file exists at the correct location |
| 3. If the file path is incorrect, update it to the correct path |
| 4. Open the file using the correct path |
| 5. Check if the LLMBase class is defined in the file |
| 6. Make any necessary changes to fix the ImportError |
| 7. Run tests to ensure the fix resolves the issue |
| |
| I'll start by verifying the file path and updating it if needed. |
| |
+----------------------------------------------------------------------------+
+----------------------------------------------------------------------+
| Devon ran: Running command: |
| find_file "llm_basic.py" |
| |
| > /Users/username/devopsgpt/backend/app/pkgs/tools/llm_basic.py |
+----------------------------------------------------------------------+
+----------------------------------------------------------------------------+
| Devon is thinking: |
| The find_file command returned the correct path for the llm_basic.py |
| file:

Loop

'NoneType' object has no attribute 'request'

My apologies, yes I am a noob, but i usually manage to get everything running fine that i invest time to, it seems the CUI is running with ollama but it hangs when starting, I understand it says please dont use the local version but i cant afford api keys....paid too much for the PC.
Anyway.....
sievo666@SigmaTau:/mnt/f/AI$ devon --api_key=FOSS
Using model name: ollama/deepseek-coder:6.7b
+-----------------------------------+
| Task: ask user for what to do |
+-----------------------------------+
'NoneType' object has no attribute 'request'
╔═════════════════════════════════════════════════════════╗
║ Error: 'NoneType' object has no attribute 'request' ║
╚═════════════════════════════════════════════════════════╝
+--------------------------------------------------------------------------------------------+
| Devon is thinking: Exit due to exception: 'NoneType' object has no attribute 'request' |
+--------------------------------------------------------------------------------------------+

Add a way to ignore some folders from scanning

Devon appears to be attempting to read the entire directory, including both 'node_modules' and '.venv' folders. If so, is there a method to create a '.devonignore' file or utilize the existing '.gitignore' file to selectively 'hide' specific content from Devon?

It attempted to open a directory as a file and got stuck

Edit tool fails on empty files

Adding this because empty files cause the edit tool to fail.

Stuck on initializing

I installed everything and when I ran the command "devon" it gets stuck on initailizing.

Support for Google Gemini

It'd be great to get support for Google Gemini too

Exceeding per-minute rate limit

It would be great if there could be some way that Devon outputs a history so that if the workflow is stopped it can be resumed again upon reloading. I Have tried a few tricks to keep my workflow going but I still keep hitting rate limits. My attempt was to create an instructions.txt with a very detailed explanation of what I wanted. In my instructions I told Devon to keep a running log in a separate txt file so that the workflow could be easily restarted in case of a crash. Within my instructions.txt I tell Devon to first read the previous log and then continue working. It kind of works, but not well enough to get me through the entire project.

LM Studio integration

👋 I see that local LLMs are on the roadmap, are there any plans for LM Studio as well? Is there anything needed to make adding it easier?

Benchmark on SWE-Bench

It would be interesting to see the performance on SWE-Bench benchmarks, so that this project can be more clearly differentiated from the increasing number of other coding agents.

https://www.swebench.com/
https://github.com/princeton-nlp/SWE-bench
[ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues?
https://arxiv.org/abs/2310.06770
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan
Language models have outpaced our ability to evaluate them effectively, but for their future development it is essential to study the frontier of their capabilities. We consider real-world software engineering to be a rich, sustainable, and challenging testbed for evaluating the next generation of language models. We therefore introduce SWE-bench, an evaluation framework including 2,294 software engineering problems drawn from real GitHub issues and corresponding pull requests across 12 popular Python repositories. Given a codebase along with a description of an issue to be resolved, a language model is tasked with editing the codebase to address the issue. Resolving issues in SWE-bench frequently requires understanding and coordinating changes across multiple functions, classes, and even files simultaneously, calling for models to interact with execution environments, process extremely long contexts and perform complex reasoning that goes far beyond traditional code generation. Our evaluations show that both state-of-the-art proprietary models and our fine-tuned model SWE-Llama can resolve only the simplest issues. Claude 2 and GPT-4 solve a mere 4.8% and 1.7% of instances respectively, even when provided with an oracle retriever. Advances on SWE-bench represent steps towards LMs that are more practical, intelligent, and autonomous.

Devon sometimes doesnt actually write to files

In certain scenarios, the agent successfully edits but the changes arent reflected.
Platform: MacOS
Potential causes:

permissions (can try to create a env where the agents doesn't have any permissions)

(for reference discord: support)

allow Devon's history to be written and read from disk

Often Devon gets stuck with something — either a loop or other bug — and the only path out is to terminate the process and start over. Ideally, the thought process log, scratchpad, etc. that represents Devon's work could be written to disk so it can resume where it left off should the process need to be terminated.

This could also be useful for running tests, forking Devons, or even collaboration between Devons in the future.

entropy-research / devon Goto Github PK

devon's Introduction

Devon: An open-source pair programmer

How do y'all ship so quickly?

Installation

Prerequisites

Installation commands

Thats it! Happy building :)

Running the agent

Features

Limitations

Progress

This project is still super early and we would love your help to make it great!

Current goals

Star history

Past milestones

Current development priorities

How can I contribute?

Feedback

Community

License

devon's People

Contributors

Stargazers

Watchers

Forkers

devon's Issues

Current Behaviour

Context (Environment)

Recommend Projects

Recommend Topics

Recommend Org