Git Product home page Git Product logo

Hi ! Here is Coobiw 👋

🙋‍♂️ About Me:

  • 👨‍🦰 I’m currently a Master of Science candidate of Peking University (PKU).
  • 👦 Before that, I received the Honours Bachelor, Huazhong University of Science and Technology (HUST).
  • ❤️‍🔥 Now, I am intersted in Multi-modal Learning especially MLLM.

😋 Projects:

  • 💥 In 2023 summer, I take part in OSPP(Open Source Promotion Plan) Summer Camp , with the honor of contributing for MMPretrain to build prompt-based classifier.
    • Now, the implement of zero-shot CLIP classifier has been merged to the main branch. PR Link
    • The implement of RAM(Recognize Anything Model) has been merged to the dev branch. Welcome to use the gradio WebUI to test it on MMPretrain! PR Link
  • 💥 2023.10: I implement MiniGPT4Qwen, which is a toy model aligning MiniGPT4 with Qwen-Chat LLM model. I just use 18.8k high quality instruction-tuning data(bi-lingual, selected from minigpt4 and llava). Just fine-tuning the projection layer (3M trainable parameters), this model support Chinese and English! MiniGPT4Qwen
  • 💥 2024.2: I extend MiniGPT4Qwen to MPP-Qwen14B(Multimodal Pipeline Parallel), scaling up both the LLM(to Qwen-14B-Chat) and pretrain-data(using LLaVA-pretrain-data). I also unfreeze the whole LLM during SFT-stage. All training is conducted on 3090/4090 GPUs. To prevent poverty (24GB of VRAM) from limiting imagination, I implemented an MLLM version based on deepspeed Pipeline Parallel. Pre-training can be completed in 22 hours on 2x4090s, while SFT requires training on 6x4090s (because it needs to fully activate the LLM), but due to the small amount of data, it only takes several hours.MPP-Qwen14B
  • 💥 2024.6: MPP-Qwen-Next is released! Support {video/image/multi-image} {single/multi-turn} conversations. All training periods are conducted on 8 RTX3090(24GB) GPUs. MPP-Qwen-Next.


Anurag's GitHub stats

Coobiw's Projects

coobiw.github.io icon coobiw.github.io

Github Pages template for academic personal websites, forked from mmistakes/minimal-mistakes

fire_detection_system icon fire_detection_system

It is a system used to detect the fire object based on cascade-classifier,Yolov1 Network Architecture and their combination.By the way,In order to improve the accuracy,we also use FCN Network.Actually,it is better for you to use Yolov3,v4,v5 to solve this problem.We just want to introduce an idea to enhance the accurary on small object,such as detecting on the segmentation results.

ip-iqa icon ip-iqa

[ICME2024, Official Code] for paper "Bringing Textual Prompt to AI-Generated Image Quality Assessment"

knn_with_maxheap icon knn_with_maxheap

I use MaxHeap to optimize the KNN algorithm.The detail and the code are shown in this repository.

learngit icon learngit

教程→ https://www.liaoxuefeng.com/wiki/896043488029600 推送请使用UTF-8编码

mcm_2021a_code icon mcm_2021a_code

The repository is about some of my codes which I do in 2021 MCM Question A.

minisora icon minisora

The Mini Sora project aims to explore the implementation path and future development direction of Sora.

mpp-llava icon mpp-llava

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

pytorch-word2vec-glove icon pytorch-word2vec-glove

This repository implements the Word2Vec and GloVe by pytorch deep learning framework. Refer to the d2l(https://zh-v2.d2l.ai/) and self-implement the GloVe. So maybe this is a record of my learning start for NLP.:)

self-learning-codes-notes icon self-learning-codes-notes

A repo which is served as a record of my learning in DL,CV,NLP, including various codes and notes. Please keep fighting!

torch-like_numpy_mlp icon torch-like_numpy_mlp

A torch-like only-numpy-implement of MLP without any deep learning framework. A simple computation graph is also realized.

trivqa icon trivqa

[CVPRW2024, Official Code] for paper "Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap".

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.