Would you like to train a manim code generation model yourself ? about generative-manim HOT 12 OPEN

I have sent an email

from generative-manim.

Hi @Yangget and @360macky ! I'm currently working on a project that involves converting text queries to Manim-specific codes, just like @Yangget mentioned. Then, I stumbled upon this repository. Hahaha.

I would really love to collaborate with you two regarding this project and bring it to life. It would help increase the precision and flexibility of a no-code to low-code programmatic animation platform.

I'll send you guys an email soon so we can discuss on this further! 😄

from generative-manim.

Hello @Yangget! Sorry about delayed answer. Sure, I'd like to hear your idea. Feel free to share it in a new reply for this issue, or on my personal email: [email protected].

from generative-manim.

from generative-manim.

Hi @waizwafiq!

That sounds great. It's a good idea that I'm working on.

You can add me to Discord, if it's okay for you: 360macky • Marcelo#7552

from generative-manim.

Let's create a Discord Server !

@360macky @waizwafiq

from generative-manim.

Hi!
Can you tell me your latest news? I am very curious about this project

from generative-manim.

Hi! Can you tell me your latest news? I am very curious about this project

Hi @Puiching-Memory!

We are currently testing an implementation of a flow to generate code inspired by LangGraph:

Our Discord server is small, I invite you to join us. I am updating this experiment continuously these weeks.

from generative-manim.

This link shows that the invitation is invalid, it could be that the link has expired, or there is a problem with my network.
Your idea inspired me. This is my first contact with LangGraph, and I can't help you for the time being.

Recently I have been working on a similar task, hoping to use GPT to drive manim to generate animations. Existing GPT's still make frequent errors. I've heard that there are ways to fine-tune an LLM to turn it into an expert model. I wonder if this method is suitable for the present situation.

If so, what data sets do we need to prepare and which fine-tuning framework should we choose?

from generative-manim.

This link shows that the invitation is invalid, it could be that the link has expired, or there is a problem with my network. Your idea inspired me. This is my first contact with LangGraph, and I can't help you for the time being.

Recently I have been working on a similar task, hoping to use GPT to drive manim to generate animations. Existing GPT's still make frequent errors. I've heard that there are ways to fine-tune an LLM to turn it into an expert model. I wonder if this method is suitable for the present situation.

If so, what data sets do we need to prepare and which fine-tuning framework should we choose?

Thanks for the heads up on the Discord link. It's updated!

I'm glad to read that the idea inspires you too. I think there are many systems to explore, so I couldn't determine which is best at this point. We started with Zero-Shot Learning, but Fine-Tuning and LangGraph are ideas we explored. Then we will evaluate which is better.

For the datasets: We would need datasets that contain the following properties: prompt, code, type.

Basically describe as a human would, a video scene or an image scene for the prompt, categorize it into what would be a video or an image, and the code itself. After this I plan to transform all the code and the prompt (which, in addition to the instruction, will specifically include the type of media to be generated) into the JSONL format that is used for fine tuning.

We have 27 Python files (from Manim Community), usually at least 50+ are preferred (at this point: more is better).

Feel free to propose new ideas, add new files to our recent small dataset, make new things on this workflow or follow the project closely. I'm happy to answer if you have more questions.

from generative-manim.