lavague-ai / lavague Goto Github PK

#29 highlighted some recent issue with llama-index streaming not working anymore.
Have to fix it

Is your feature request related to a problem? Please describe.

Currently the docker has only 2 commands, build and launch, and it launches the config.py file, what If I want to use huggingface_interface.py?

Can we modify the entrypoint.sh? and add the support for huggingface and possibly for others?

Describe the solution you'd like
Modifying the entrypoint.sh file and adding the support for the same.

Describe alternatives you've considered
I am not sure about the alternatives, but modifying the entrypoint.sh seems a good solution at this point.

Additional context

Problem: not clear how to install and run locally on macOS.

Proposed solution: Allow to install the tool via the Homebrew package manager on macOS. Example: brew install lavague

The command center module should:

• Get HTML source code from URL
• Make calls to actionEngine using config file for customizable parameters such as models to use, prompt template to use, etc.
• Execute the generated code

The browser simply won't load. Tried different urls, but still no joy.

Do you plan to support playwright?

Hello folks,
I'm trying to get the project off its feet with the source code.
I gave the file path in the lavague launch command parameters, this did not work.
I tried the commands in the normal document.

I started with the setup.sh file, there was no error here.
Chrome driver unzip was done successfully.
I am working in a virtual environment with conda.

There's openai.py file on path lavague-files and examples/api

When the command tried on those paths, chromedriver error.

 lavague-launch --file_path hf.txt  --config_path openai.py                                                                                                                    
Traceback (most recent call last):
  File "/opt/anaconda3/envs/LaVague/bin/lavague-launch", line 8, in <module>
    sys.exit(launch())
  File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/__init__.py", line 121, in launch
    action_engine, get_driver = load_action_engine(config_path, streaming=True)
  File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/utils.py", line 18, in load_action_engine
    config_module = import_from_path(path)
  File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/utils.py", line 13, in import_from_path
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 846, in exec_module
  File "<frozen importlib._bootstrap_external>", line 982, in get_code
  File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'openai.py'
(LaVague) 

LaVague 
❯ conda info

     active environment : LaVague
    active env location : /opt/anaconda3/envs/LaVague
            shell level : 5
       user config file : /Users/serdarkaracay/.condarc
 populated config files : /Users/serdarkaracay/.condarc
          conda version : 24.1.2
    conda-build version : 24.1.2
         python version : 3.11.7.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.1.2=0
                          __osx=14.4.1=0
                          __unix=0=0
       base environment : /opt/anaconda3  (writable)
      conda av data dir : /opt/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/anaconda3/pkgs
                          /Users/serdarkaracay/.conda/pkgs
       envs directories : /opt/anaconda3/envs
                          /Users/serdarkaracay/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.1.2 requests/2.31.0 CPython/3.11.7 Darwin/23.4.0 OSX/14.4.1 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.6 aau/0.4.3 c/QkeNtkUrgwz7nVYVSzIrAg s/5f_5979BIZ-G0e4oRXaKzg e/ATRt1h_Xl4cCAzCR1M6zcQ
                UID:GID : 501:20
             netrc file : None
           offline mode : False

MacOS M1 Max Sonoma 14.4.1
Python 3.9.19

It gives a TypeError saying ,
Missing "Prompt" argument.

Kindly suggest what am I doing wrong.

Hi,
First of all excellent work. I am facing issue with timeouts. I am able to install and run the demo. That's good. However when go the gradio app like this.
I typed in https://youtube.com

Then I supplied the following query

It went on to execute this statement forever. Moreover it does not display any errors or stack trace.

It hasn't been effective on most of the sites I've attempted to use it on, as it's often identified as a bot. Are there any plans to enhance the agent's resilience against such detections?

Is it possible to connect LM Studio?

If not, can you add it?

Thanks.

Is your feature request related to a problem? Please describe.
It would be nice to support local llms as most companies have data privacy policies especially around recent AI developments

Describe the solution you'd like
It would be nice to point to a local llm or have some integration with standard libraries like ollama

We have a monolithic installation process, for instance cuda is installed by default, which is too heavy for some use cases.

We will provide soon a version where the Action Engine can be provided through a SaaS to make it easier for people who don't want to do computing locally, so their package will be very lightweight.

@lyie28 Could you have a look? I guess this can be done in conjunction with #59

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Take code from Google Colab example and make it into a modular Python library

Describe the bug
When simply running all fields one for one in the google colab I get the following:

NoSuchDriverException Traceback (most recent call last)
in <cell line: 40>()
38
39 # Choose Chrome Browser
---> 40 driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
41
42 # action_engine = ActionEngine(llm, embedder)

2 frames
/usr/local/lib/python3.10/dist-packages/selenium/webdriver/common/driver_finder.py in get_path(service, options)
42
43 if path is None or not Path(path).is_file():
---> 44 raise NoSuchDriverException(f"Unable to locate or obtain driver for {options.capabilities['browserName']}")
45
46 return path

NoSuchDriverException: Message: Unable to locate or obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

I was able to fix it by removing the service from the initialization so instead of # Choose Chrome Browser
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options) I did driver = webdriver.Chrome( options=chrome_options)

Assigning an ID to Selenium drivers and locking them for individual users

@HiImMadness : see with @mbrunel to integrate the work you did on improving the retriever and the new prompt template to increase performance

• automated testing needed after pushes to main

Add modular different export options that will interact with hub when built: python, pytests, FastAPI etc.

I realize that depending on the model, the generated code can have different issues and needs to be cleaned differently.

For instance, with Hugging Face API, the generation continues after the end of the markdown cell

# First we need to identify the component first, then we can click on it.

# Based on the HTML, the link can be uniquely identified using the class "w-full rounded-full border border-gray-200 text-sm placeholder-gray-400 shadow-inner outline-none focus:shadow-xl focus:ring-1 focus:ring-inset dark:bg-gray-950 h-7 pl-7"
# Let's use this class with Selenium to identify the link
search_bar = driver.find_element(By.XPATH, "//*[@class='w-full rounded-full border border-gray-200 text-sm placeholder-gray-400 shadow-inner outline-none focus:shadow-xl focus:ring-1 focus:ring-inset dark:bg-gray-950 h-7 pl-7']")

search_bar.click()

# Now we can type the asked input
search_bar.send_keys("The Stack")

# Finally we can press the 'Enter' key
search_bar.send_keys(Keys.ENTER)

---

HTML:
<!DOCTYPE html>
<html>
<head>
    <title>Mock Page</title>
</head>```

For Azure OpenAI, it starts with ```python, with Mixtral on Fireworks AI it starts with a natural answer like "Here is an answer".

We should have the Action Engine be initialized with a cleaning function.

Litellm is open source project you can read about them if you want https://github.com/BerriAI/litellm
Many projects are integrating with it because with this we can use easily 100+ different LLMs as it helps in creating local proxy server which api structure is open ai compatible and with one apo structure it becomes easy to change model and api key and use easily without additional configurations and reading and implementing from individual llm docs so please think about it

-> Specs: doesn't require developer mode

I tried running the gradio demo in colab. It is not working. Just shows an error message without telling what is causing the error.

• Place your custom config file for integration with X API or model in examples/api
• Add an integration notebook to docs/docs/integrations following the other integration notebooks as an example

Is your feature request related to a problem? Please describe.
Add a way for people to test and run this locally - not in a codelab. Docker is an easy path for folks.

Describe the solution you'd like
I'd like clear instructions on how to run this locally.

When I run this, irrespective of which site I simply get "error" inside the virual screen. Tried several different websites and browsers and I get the same thing. No error in the code so presumably all dependencies are installed correctly... This example is the IRS site. Happens within the link and within colab.

Add telemetry to help us improve performance/reduce failures:

Collect:
URL
Date
HTML code
Retrieved nodes by Llama Index
LLM used
Error message
Code produced
Screenshot