This document provides an overview of the data flow for our web application, InstaCook, which processes a recipe video URL input to generate and return the instructions and ingredient list to the user interface (UI).

• Introduction
• Demo video
• Data Flow Steps
• Flowchart
• Technologies Used
• Installation
• Usage

InstaCook takes a video URL input from the user, extracts the audio, transcribes it, processes the transcription with Groq to generate a JSON response, and sends this JSON to the Instacart API. The API returns an ingredient list that is displayed in the UI.

Loom link

1. UI: The user provides a video URL through the application's UI.
2. Audio Extraction: The audio from the provided video URL is extracted.
3. Whisper Speech to Text/Transcription: The extracted audio is transcribed into text using Whisper.
4. Groq Processing: The transcribed text is processed using Groq to extract instructions and ingredients into a JSON format.
5. Send to Instacart API: The JSON object is sent to the Instacart API (pending approval).
6. Return Ingredient List to UI: The Instacart API processes the JSON and returns an ingredient list.
7. UI: The returned ingredient list is displayed back in the UI for the user.

flowchart TD
    A[UI] -->|User Inputs Video URL| B[Audio Extraction]
    B --> C[Whisper Speech to Text/Transcription]
    C --> D[Groq converts instructions and ingredients into JSON format]
    D --> E[Send to Instacart API]
    E --> F[Return Instacart Recipe Page]
    F --> A[UI]

1. yt-dlp Youtube downloader - Download and extract audio from youtube video link.
2. ffmpeg audio conversion - Convert the extracted audio to 16khz sample rate which is required by openai/whisper.
3. transformer speech2text - OpenApi open source whisper model (until groq enables public access for their speech2text api).
4. groq fast ai inference - Llama3-70b-8192 using groq api to extract recipe title, instructions and ingredients; generate instacart api request
5. instacart developer api - Creating a recipe page with developer api.

1. Create a new conda environment using

conda create -n insta-cook python=3.11

2. Activate the conda environment

conda activate insta-cook

3. Install the requiered python modules using

pip install -r requirements.txt

1. Create a groq account and get api key.
2. Input the generated key in .env file in the project directory and add the value as GROQ_API_KEY=<your_generated_key>

Note: I have delete my api key ;)

3. Follow installation instructions above
4. Run

python yt_instacart_recipie.py