Gather AIGC most useful tools, materials, publications and reports
Report | Link | Date | Institution |
---|---|---|---|
Stanford AI index Report 2023 | Link | Stanford | |
Sparks of Artificial General Intelligence: Early experiments with GPT-4 | Link | Microsoft | |
A Survey of Large Language Models | Link | April 2023 | Renmin University, China & University of Montreal, Canada |
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond | Link | Amazon & many others | |
A Cookbook of Self-Supervised Learning | Link | Meta & many others | |
Let’s Verify Step by Step | Link | May 2023 | OpenAI |
A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering | Link | May 2023 | Kyung Hee University and many |
A Comprehensive Survey on Segment Anything Model for Vision and Beyond | Link | May 2023 | Hong Kong University of Science and Technology and many |
On the Design Fundamentals of Diffusion Models: A Survey | Link | June 2023 | Durham University |
Open LLM Leaderboard | Link | Update in real time | Huggingface |
A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering | Link | May 2023 | JKyung Hee University and many |
Alpaca Open Source Code Stanford March 2023
Dolly Open Source Code Databricks March 2023 Note: OK to use commercially
Vicuna Open Source Code UC Berkeley, CMU, Stanford, and UC San Diego March 2023
ChatPDF March 2023
Bard Google March 2023
Langchain Community Effort March 2023
Microsoft 365 Copilot Microsoft March 2023
AutoGPT Community Effort April 2023
Grounded SAM IDEA April 2023
DeepSpeed Chat Microsoft April 2023
AgentGPT Community Effort April 2023
MiniGPT King Abdullah University of Science and Technology April 2023
DeepFloyd IF Stability.ai April 2023
Open Llama Berkeley May 2023
SoftVC VITS Singing Voice Conversion Community May 2023
Falcon Tii May 2023
UltraLM Tsinghua University June 2023
COS597G Understanding Large Language Models Princeton 2022
CS324 Large Language Models Stanford 2023
ChatGPT, LangChain and DS Courses Deeplearning.ai Jun 2023
Large Multimodal Models: Notes on CVPR 2023 Tutorial Microsoft Jun 2023
OpenAI Cookbook
Llama Index
PrivateGPT
Llama.cpp
1. Data is still king - LLMs are great but if you don't have quality clean data you won’t go far.
2. Smaller models can be just as good as larger general models at specific tasks. And cheaper!
3. Fine-tuning is becoming cheaper.
4. Evaluation of LLMs is very hard - feels very subjective still.
5. Managed APIs are expensive.
6. "Traditional" ML isn't going anywhere.
7. Memory matters - for both serving and training.
8. Information retrieval w/ vector databases is becoming standard pattern.
9. Start w/ prompt engineering and push that to its limits before fine-tuning w/ smaller models.
10. Use agents/chains only when necessary. They are unruly.
11. Latency is critical for a good user experience.
12. Privacy is critical.