Git Product home page Git Product logo

awesome-sketch-based-applications's Introduction

Awesome-Sketch-Based-Applications

A collection of sketch based applications.

Feel free to create a PR or an issue.

Outline

0. Survey

Paper Source Code/Project Link
Deep Learning for Free-Hand Sketch: A Survey TPAMI 2022 [code]

1. Sketch Based Image Synthesis

1.1 Automatic Synthesis

  • Natural Image or Object
Paper Source Code/Project Link
SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis CVPR 2018 [code]
Image Generation from Sketch Constraint Using Contextual GAN ECCV 2018 [code] [project]
Multi-Instance Sketch to Image Synthesis With Progressive Generative Adversarial Networks IEEE Access 2019
Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation ICCV 2019 [code] [project]
SketchyCOCO: Image Generation from Freehand Scene Sketches CVPR 2020 [code]
Unsupervised Sketch-to-Photo Synthesis ECCV 2020 [code] [project]
Sketch Your Own GAN ICCV 2021 [code] [webpage]
Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis WACV 2022 [code]
Customizing GAN Using Few-Shot Sketches ACM MM 2022
DiffSketching: Sketch Control Image Synthesis with Diffusion Models BMVC 2022 [code]
MaskSketch: Unpaired Structure-guided Masked Image Generation CVPR 2023 [project] [code]
Picture that Sketch: Photorealistic Image Generation from Abstract Sketches CVPR 2023 [project] [code]
  • Human Face / Portrait / Body
Paper Source Code/Project Link
LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network ACM MM 2019
DeepFaceDrawing: Deep Generation of Face Images from Sketches SIGGRAPH 2020 [project] [code]
DeepFacePencil: Creating Face Images from Freehand Sketches ACM MM 2020 [project] [code]
Controllable Sketch-to-Image Translation for Robust Face Synthesis TIP 2021
DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN TVCG 2022
DeepPortraitDrawing: Generating Human Body Images from Freehand Sketches C&G 2023
Semantics-Preserving Sketch Embedding for Face Generation TMM 2023 [project] [code]
Parsing-Conditioned Anime Translation: A New Dataset and Method TOG 2023 [code]
Sketch2Human: Deep Human Generation with Disentangled Geometry and Appearance Constraints TVCG 2024
  • 3D image
Paper Source Code/Project Link
3D-aware Conditional Image Synthesis CVPR 2023 [project] [code]

1.2 Style-conditioned

Paper Source Code/Project Link
TextureGAN: Controlling Deep Image Synthesis with Texture Patches CVPR 2018 [code]
Multimodal Unsupervised Image-to-Image Translation ECCV 2018 [code]
SketchPatch: Sketch Stylization via Seamless Patch-level Synthesis SIGGRAPH Asia 2020
Self-Supervised Sketch-to-Image Synthesis AAAI 2021 [code]
CoGS: Controllable Generation and Search from Sketch and Style ECCV 2022
Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model WACV 2023 [project] [code]
DemoCaricature: Democratising Caricature Generation with a Rough Sketch CVPR 2024 [project]
SMFS-GAN: Style-Guided Multi-class Freehand Sketch-to-Image Synthesis CGF 2024

1.3 Text-conditioned

Paper Source Code/Project Link
Sketch-Guided Text-to-Image Diffusion Models SIGGRAPH 2023 [project]
Adding Conditional Control to Text-to-Image Diffusion Models ICCV 2023 [code]
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models arxiv 23.02 [code]
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild NeurIPS 2023 [code] [project]
Stable Doodle (ClipDrop) [demo]
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing arxiv 24.02
Block and Detail: Scaffolding Sketch-to-Image Generation arxiv 24.02
One-Step Image Translation with Text-to-Image Models arxiv 24.03 [code]
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models CVPR 2024 [code]

2. Sketch Based Image Editing

  • Arbitrary Image
Paper Source Code/Project Link
Free-Form Image Inpainting with Gated Convolution (DeepFill v2) ICCV 2019 [project] [code]
Fashion Editing with Multi-scale Attention Normalization 1906.00884
DeFLOCNet: Deep Image Editing via Flexible Low-level Controls CVPR 2021 [code]
DeepSIM: Image Shape Manipulation from a Single Augmented Training Sample ICCV 2021 [code] [project]
SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches CVPR 2022 [code] [project]
Draw2Edit: Mask-Free Sketch-Guided Image Manipulation ACM MM 2023 [code]
Towards Interactive Image Inpainting via Robust Sketch Refinement TMM 2024 [code] [project]
  • Human Face / Portrait / Hair
Paper Source Code/Project Link
FaceShop: Deep Sketch-based Face Image Editing SIGGRAPH 2018 [project]
CaricatureShop: Personalized and Photorealistic Caricature Sketching TVCG 2018
Sparse, Smart Contours to Represent and Edit Images CVPR 2018 [project]
Example-Guided Style-Consistent Image Synthesis from Semantic Labeling CVPR 2019 [code]
SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and Color ICCV 2019 [code]
Intuitive, Interactive Beard and Hair Synthesis with Generative Models CVPR 2020
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches ECCV 2020 [code] [project]
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control SIGGRAPH 2021 [code] [project]
SketchHairSalon: Deep Sketch-based Hair Image Synthesis SIGGRAPH Asia 2021 [project]
Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editing ECCV 2022 [code] [project]
NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields SIGGRAPH Asia 2022 [project]
SketchFaceNeRF: Sketch-based Facial Generation and Editing in Neural Radiance Fields SIGGRAPH 2023
DemoCaricature: Democratising Caricature Generation with a Rough Sketch arxiv 23.12 [project] [code]
  • Anime Editing
Paper Source Code/Project Link
AniFaceDrawing: Anime Portrait Exploration during Your Sketching SIGGRAPH 2023 [project]
  • Video Editing
Paper Source Code/Project Link
DeepFaceVideoEditing: Sketch-based Deep Editing of Face Videos SIGGRAPH 2022 [project]

3. Sketch Based Image Retrieval (SBIR)

  • Object-level
Paper Source Code/Project Link
Sketch Me That Shoe CVPR 2016 [code-caffe] [code-tf] [project]
Deep Multi-task Attribute-driven Ranking for Fine-grained Sketch-based Image Retrieval BMVC 2016
Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval CVPR 2017 [code]
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval ICCV 2017 [project]
Zero-Shot Sketch-Image Hashing CVPR 2018
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval CVPR 2018
Generative Domain-Migration Hashing for Sketch-to-Image Retrieval ECCV 2018 [code]
A Zero-Shot Framework for Sketch Based Image Retrieval ECCV 2018 [code]
Generalising Fine-Grained Sketch-Based Image Retrieval CVPR 2019
Doodle to Search: Practical Zero-Shot Sketch-Based Image Retrieval CVPR 2019 [project] [code]
LiveSketch: Query Perturbations for Guided Sketch-Based Visual Search CVPR 2019
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-Based Image Retrieval CVPR 2019 [code]
Learning Structural Representations via Dynamic Object Landmarks Discovery for Sketch Recognition and Retrieval TIP 2019
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval ICCV 2019
TC-Net for iSBIR: Triplet Classification Network for Instance-level Sketch Based Image Retrieval ACM MM 2019
Sketch-Based Image Retrieval With Multi-Clustering Re-Ranking TCSVT 2019
Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval TCSVT 2019
Zero-Shot Sketch-Based Image Retrieval via Graph Convolution Network AAAI 2020
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval CVPR 2020 [code]
Solving Mixed-modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image Retrieval CVPR 2020
Fine-Grained Instance-Level Sketch-Based Image Retrieval IJCV 2020
StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval CVPR 2021
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch-Based Image Retrieval CVPR 2021 [code]
DLA-Net for FG-SBIR: Dynamic Local Aligned Network for Fine-Grained Sketch-Based Image Retrieval ACM MM 2021
Domain-Aware SE Network for Sketch-based Image Retrieval with Multiplicative Euclidean Margin Softmax ACM MM 2021 [code]
Relationship-Preserving Knowledge Distillation for Zero-Shot Sketch Based Image Retrieval ACM MM 2021
Transferable Coupled Network for Zero-Shot Sketch-Based Image Retrieval TPAMI 2021 [project]
TVT: Three-Way Vision Transformer through Multi-Modal Hypersphere Learning for Zero-Shot Sketch-Based Image Retrieval AAAI 2022
Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval CVPR 2022 [code]
Sketch3T: Test-time Training for Zero-Shot SBIR CVPR 2022
Augmented Multi-Modality Fusion for Generalized Zero-Shot Sketch-based Visual Retrieval TIP 2022 [code]
Adaptive Fine-Grained Sketch-Based Image Retrieval ECCV 2022 [code]
Conditional Stroke Recovery for Fine-Grained Sketch-Based Image Retrieval ECCV 2022 [code]
A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch ECCV 2022 [code] [project]
Multi-Level Region Matching for Fine-Grained Sketch-Based Image Retrieval ACM MM 2022 [code]
Prototype-based Selective Knowledge Distillation for Zero-Shot Sketch Based Image Retrieval ACM MM 2022
DLI-Net: Dual Local Interaction Network for Fine-Grained Sketch-Based Image Retrieval TCSVT 2022 [code]
Data-Free Sketch-Based Image Retrieval CVPR 2023 [code]
Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style CVPR 2023 [code]
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR CVPR 2023 [project]
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Not CVPR 2023 [project]
Cross-Domain Alignment for Zero-Shot Sketch-Based Image Retrieval TCSVT 2023
Semi-transductive Learning for Generalized Zero-Shot Sketch-Based Image Retrieval AAAI 2023
Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers CVPR 2024 [project]
You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval CVPR 2024 [project]
How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval? CVPR 2024 [project]
Asymmetric Mutual Alignment for Unsupervised Zero-Shot Sketch-Based Image Retrieval AAAI 2024
Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrieval ECCV 2024
Elevating All Zero-Shot Sketch-Based Image Retrieval Through Multimodal Prompt Learning ECCV 2024 [code]
Zero-Shot Sketch Based Image Retrieval via Modality Capacity Guidance IJCAI-24 [code]
  • Scene-level
Paper Source Code/Project Link
SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches ECCV 2020
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input CVPR 2022
SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval using Adaptive GCNs TIP 2022
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context ECCV 2022 [code] [Dataset]
Scene-Level Sketch-Based Image Retrieval with Minimal Pairwise Supervision AAAI 2023
FreestyleRet: Retrieving Images from Style-Diversified Queries ECCV 2024 [code]
  • Video Retrieval
Paper Source Code/Project Link
Fine-Grained Instance-Level Sketch-Based Video Retrieval TCSVT 2020
Fine-Grained Video Retrieval with Scene Sketches TIP 2023 [project]

4. Sketch Based 3D Shape Retrieval

Paper Source Code/Project Link
Sketch-based 3D Shape Retrieval using Convolutional Neural Networks CVPR 2015
Learning Cross-Domain Neural Networks for Sketch-Based 3D Shape Retrieval AAAI 2016
Deep Correlated Metric Learning for Sketch-based 3D Shape Retrieval AAAI 2017
Learning Barycentric Representations of 3D Shapes for Sketch-based 3D Shape Retrieval CVPR 2017
Deep Cross-modality Adaptation via Semantics Preserving Adversarial Learning for Sketch-based 3D Shape Retrieval ECCV 2018
Unsupervised Learning of 3D Model Reconstruction from Hand-Drawn Sketches ACMMM 2018
Towards 3D VR-Sketch to 3D Shape Retrieval 3DV 2020 [code] [project]
Uncertainty Learning for Noise Resistant Sketch-Based 3D Shape Retrieval TIP 2021
Towards Fine-Grained Sketch-Based 3D Shape Retrieval TIP 2021
Domain Disentangled Generative Adversarial Network for Zero-Shot Sketch-Based 3D Shape Retrieval AAAI 2022
Retrieval-Specific View Learning for Sketch-to-Shape Retrieval TMM 2023
Doodle to Object: Practical Zero-Shot Sketch-Based 3D Shape Retrieval AAAI 2023 [code]
Democratising 2D Sketch to 3D Shape Retrieval Through Pivoting ICCV 2023

5. Sketch Based 3D Shape Modeling

  • Free-hand sketch
Paper Source Code/Project Link
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks 3DV 2017 [code] [project]
Shape Synthesis from Sketches via Procedural Models and Convolutional Networks TVCG 2017
DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling SIGGRAPH 2017 [project] [code]
BendSketch: Modeling Freeform Surfaces Through 2D Sketching SIGGRAPH 2017 [project]
Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling SIGGRAPH Asia 2018 [code] [project]
DeepSketchHair: Deep Sketch-based 3D Hair Modeling TVCG 2019
Interactive Liquid Splash Modeling by User Sketches SIGGRAPH Asia 2020
Monster Mash: A Single-View Approach to Casual 3D Modeling and Animation SIGGRAPH Asia 2020 [project] [code] [demo]
SAniHead: Sketching Animal-like 3D Character Heads Using a View-surface Collaborative Mesh Generative Network TVCG 2020
Towards Practical Sketch-Based 3D Shape Generation: The Role of Professional Sketches TCSVT 2020
Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches CVPR 2021
Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches ICCV 2021
Real-time Skeletonization for Sketch-based Modeling SMI 2021 [code]
Sketch2Pose: Estimating a 3D Character Pose from a Bitmap Sketch SIGGRAPH 2022 [project] [code]
Sketch2PQ: Freeform Planar Quadrilateral Mesh Design via a Single Sketch TVCG 2022
Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation ECCV 2022 [code]
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling ECCV 2022 [code]
Deep Reconstruction of 3D Smoke Densities from Artist Sketches EG 2022 [project] [code]
A Diffusion-ReFinement Model for Sketch-to-Point Modeling ACCV 2022 [code]
RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent Dataset CVPR 2023 [project]
SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling TVCG 2023 [project] [code]
Reality3DSketch: Rapid 3D Modeling of Objects from Single Freehand Sketches TMM 2023
GA-Sketching: Shape Modeling from Multi-View Sketching with Geometry-Aligned Deep Implicit Functions PG 2023 [code]
Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes CVPR 2024 [project] [code]
SENS: Part-Aware Sketch-based Implicit Neural Shape Modeling EG 2024 [code] [project]
3D Reconstruction from a Single Sketch via View-dependent Depth Sampling TPAMI 2024 [code]
  • Scene Generation
Paper Source Code/Project Link
GroundUp: Rapid Sketch-Based 3D City Massing ECCV 2024 [code]
Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches arxiv 24.08 [webpage]
  • NeRF
Paper Source Code/Project Link
Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation arxiv 24.01
SKED: Sketch-guided Text-based 3D Editing ICCV 2023 [project] [code]
SketchDream: Sketch-based Text-to-3D Generation and Editing SIGGRAPH 2024 [project]
  • CAD sketch
Paper Source Code/Project Link
Lifting Freehand Concept Sketches into 3D SIGGRAPH Asia 2020 [project] [code]
Sketch2CAD: Sequential CAD Modeling by Sketching in Context SIGGRAPH Asia 2020 [project] [code]
Free2CAD: Parsing Freehand Drawings into CAD Commands SIGGRAPH 2022 [project] [code]
Symmetry-driven 3D Reconstruction From Concept Sketches SIGGRAPH 2022 [project]
Piecewise-smooth Surface Fitting Onto Unstructured 3D Sketches SIGGRAPH 2022 [project]
Reconstruction of Machine-Made Shapes from Bitmap Sketches SIGGRAPH Asia 2023 [project]
CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention CVPR 2024
SfmCAD: Unsupervised CAD Reconstruction by Learning Sketch-based Feature Modeling Operations CVPR 2024 [code]

6. Sketch Based Garment Design

Paper Source Code/Project Link
Sketching Folds: Developable Surfaces from Non-Planar Silhouettes TOG 2015 [project]
FoldSketch: Enriching Garments with Physically Reproducible Folds SIGGRAPH 2018 [project]
Learning a Shared Shape Space for Multimodal Garment Design SIGGRAPH Asia 2018 [project]
Garment Ideation: Iterative view-aware sketch-based garment modeling 3DV 2022 [code]
Controllable Visual-Tactile Synthesis ICCV 2023 [project] [code]
Controllable Garment Image Synthesis Integrated with Frequency Domain Features PG 2023
Toward Intelligent Interactive Design: A Generation Framework Based on Cross-domain Fashion Elements ACM MM 2023
FashionDiff: A Controllable Diffusion Model Using Pairwise Fashion Elements for Intelligent Design ACM MM 2023
TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion Model NICOGRAPH 2024

7. Sketch Completion

  • Sketch completion
Paper Source Code/Project Link
SketchGAN: Joint Sketch Completion and Recognition with Generative Adversarial Network CVPR 2019
SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human Sketches BMVC 2020 [code]
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt CVPR 2020 [code]
Generative Sketch Healing IJCV 2022
  • Sketch gap / connectivity detection
Paper Source Code/Project Link
Joint Gap Detection and Inpainting of Line Drawings CVPR 2017 [project] [code]
Detecting Viewer-Perceived Intended Vector Sketch Connectivity SIGGRAPH 2022 [project]

8. Sketch Restoration, Retargeting and Inpainting

  • Manga
Paper Source Code/Project Link
Exploiting Aliasing for Manga Restoration CVPR 2021 [webpage] [code]
Seamless Manga Inpainting with Semantics Awareness SIGGRAPH 2021 [webpage] [code]
Screentone-Preserved Manga Retargeting arxiv 22.03
Manga Rescreening with Interpretable Screentone Representation arxiv 23.06
  • Hand-Drawn Drawings
Paper Source Code/Project Link
Restoration of Hand-Drawn Architectural Drawings using Latent Space Mapping with Degradation Generator CVPR 2023

9. Sketch and Shading

Paper Source Code/Project Link
Learning to Shadow Hand-drawn Sketches CVPR 2020 [project] [code]
SmartShadow: Artistic Shadow Drawing Tool for Line Drawings ICCV 2021 [project]

10. Sketch Enhancement / Beautification

  • Sketch Enhancement
Paper Source Code/Project Link
SketchMan: Learning to Create Professional Sketches ACM MM 2020 [code]
  • Sketch Beautification
Paper Source Code/Project Link
Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made Objects TVCG 2024
PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision arxiv 24.07
  • Super Resolution
Paper Source Code/Project Link
APISR: Anime Production Inspired Real-World Anime Super-Resolution CVPR 2024 [code]

11. Sketch-Guided Object Localization

Paper Source Code/Project Link
Sketch-Guided Object Localization in Natural Images ECCV 2020 [project] [code]
Localizing Infinity-shaped fishes: Sketch-guided object localization in the wild arxiv 21.09 [code]
What Can Human Sketches Do for Object Detection? CVPR 2023 [project]
Sketch-based Video Object Segmentation: Benchmark and Analysis BMVC 2023 [code]
Query-guided Attention in Vision Transformers for Localizing Objects Using a Single Sketch WACV 2024 [project] [code]

12. Sketch-Guided Video Synthesis

Paper Source Code/Project Link
Deep Sketch-guided Cartoon Video Inbetweening TVCG 2021 [code]
ToonCrafter: Generative Cartoon Interpolation arxiv 24.05 [code] [webpage]

13. Sketch Recognition

Paper Source Code/Project Link
Sketch-a-Net that Beats Humans BMVC 2015 [code]
Sketch-a-Net: A Deep Neural Network that Beats Humans IJCV 2017 [code]
Deep Self-Supervised Representation Learning for Free-Hand Sketch TCSVT 2020 [code]
S3Net:Graph Representational Network For Sketch Recognition ICME 2020 [code]
Sketchformer: Transformer-based Representation for Sketched Structure CVPR 2020 [code]
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt CVPR 2020 [code]
Multi-Graph Transformer for Free-Hand Sketch Recognition TNNLS 2021 [code]
Sketch-R2CNN: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition TVCG 2021 [code]
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting CVPR 2021 [code]
SketchAA: Abstract Representation for Abstract Sketches ICCV 2021
SketchLattice: Latticed Representation for Sketch Manipulation ICCV 2021 [code]
Multi-Stage Spatio-Temporal Networks for Robust Sketch Recognition TIP 2022

14. Sketch Segmentation and Perceptual Grouping

  • Semantic / Instance Segmentation
Type Paper Source Code/Project Link
Pixelwise SPFusionNet: Sketch Segmentation Using Multi-modal Data Fusion ICME 2019
SketchyScene: Richly-Annotated Scene Sketches (scene-level) ECCV 2018 [code]
Exploring Local Detail Perception for Scene Sketch Semantic Segmentation (scene-level) TIP 2022
Open Vocabulary Semantic Scene Sketch Understanding (scene-level) CVPR 2024 [code] [project]
Stroke-level Free Hand-Drawn Sketch Segmentation ECCV 2012
Data-driven Segmentation and Labeling of Freehand Sketches SIGGRAPH Asia 2014 [code] [project] [dataset]
Example-Based Sketch Segmentation and Labeling Using CRFs TOG 2016
SketchSegNet+: An End-to-End Learning of RNN for Multi-Class Sketch Semantic Segmentation IEEE Access 2019
Fast Sketch Segmentation and Labeling With Deep Learning CGA 2019
Stroke-based sketched symbol reconstruction and segmentation CGA 2020
SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks TOG 2021 [code]
One Sketch for All: One-Shot Personalized Sketch Segmentation TIP 2022
Sketch-Segformer: Transformer-Based Segmentation for Figurative and Creative Sketches TIP 2023 [code]
CreativeSeg: Semantic Segmentation of Creative Sketches TIP 2024 [code]
ContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention CVPR 2024 [webpage]
  • Perceptual Grouping
Type Paper Source Code/Project Link
Stroke-level Sketching by perceptual grouping ICIP 2013
Making Better Use of Edges via Perceptual Grouping CVPR 2015
Universal Sketch Perceptual Grouping / Toward Deep Universal Sketch Perceptual Grouper ECCV 2018 / TIP 2019 [code]
  • FloorPlan
Type Paper Source Code/Project Link
Vector Recognizing Vector Graphics without Rasterization NeurIPS 2021 [code]
FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting ICCV 2021 [project]
GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings CVPR 2022
CADTransformer: Panoptic Symbol Spotting Transformer for CAD Drawings CVPR 2022 [code]
VectorFloorSeg: Two-Stream Graph Attention Network for Vectorized Roughcast Floorplan Segmentation CVPR 2023 [code]
Hierarchical Recognizing Vector Graphics and A New Chart-based Vector Graphics Dataset TPAMI 2024 [code]

15. Sketch Representation Learning

  • Stroke order importance/saliency, sketch abstraction
Paper Source Code/Project Link Data Manner Related Applications
Learning Deep Sketch Abstraction CVPR 2018 Vector/stroke-level FG-SBIR
Goal-Driven Sequential Data Abstraction ICCV 2019 Vector/stroke-level Sketch recognition
Pixelor: a competitive sketching AI agent. So you think you can sketch? SIGGRAPH Asia 2020 [Project] [Code] Vector/stroke-level Sketch synthesis and recognition
SketchAA: Abstract Representation for Abstract Sketches ICCV 2021 Vector/stroke-level Sketch recognition, FB-SBIR, sketch healing
CLIPasso: Semantically-Aware Object Sketching SIGGRAPH 2022 [project] [code] Vector/stroke-level
Abstracting Sketches through Simple Primitives ECCV 2022 [code] Vector/stroke-level Sketch recognition, FG-SBIR
Learning Geometry-aware Representations by Sketching CVPR 2023 [code] Vector/stroke-level object attribute classification, domain transfer, stroke-based generation, FG-SBIR
SketchXAI: A First Look at Explainability for Human Sketches CVPR 2023 [project] Vector/stroke-level sketch recognition
Prediction with Visual Evidence: Sketch Classification Explanation via Stroke-Level Attributions TIP 2023 Vector/stroke-level
What Sketch Explainability Really Means for Downstream Tasks CVPR 2024 Vector/stroke-level
Do Generalised Classifiers really work on Human Drawn Sketches? ECCV 2024 Pixel-level
  • Conventional Representation Learning
Paper Source Code/Project Link Data Manner Related Applications
A Neural Representation of Sketch Drawings (Sketch-RNN) ICLR 2018 [code] Vector/stroke-level Reconstruction and interpolation
SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human Sketches BMVC 2020 [code] Vector/stroke-level Sketch recognition, retrieval, completion and analogy
Sketchformer: Transformer-based Representation for Sketched Structure CVPR 2020 [code] Vector/stroke-level Sketch classification, SBIR, reconstruction and interpolation
CoSE: Compositional Stroke Embeddings NeurIPS 2020 [Code] Vector/stroke-level Auto-completing diagrams
SketchLattice: Latticed Representation for Sketch Manipulation ICCV 2021 [code] Lattice graph Sketch healing and image-to-sketch synthesis
SketchODE: Learning neural sketch representation in continuous time ICLR 2022 [Project] Vector/Stroke-level Continuous-time representation, reconstruction & interpolation
Linking Sketch Patches by Learning Synonymous Proximity for Graphic Sketch Representation AAAI 2023 [code] Vector/Stroke-level Sketch synthesis and sketch healing
SketchKnitter: Vectorized Sketch Generation with Diffusion Models ICLR 2023 [code] Vector/Stroke-level vectorized sketch generation
ChiroDiff: Modelling chirographic data with Diffusion Models ICLR 2023 [Project] Vector/Stroke-level vectorization, de-noising/healing, abstraction
Enhance Sketch Recognition's Explainability via Semantic Component-Level Parsing AAAI 2024 [code] Vector/Stroke-level recognition and segmentation
Modelling complex vector drawings with stroke-clouds ICLR 2024 [code] Vector/Stroke-level reconstruction and generation
Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation CVM 2024 Vector/Stroke-level reconstruction, recognition and semantic segmentation
Paper Source Code/Project Link Data Manner Related Applications
A Learned Representation for Scalable Vector Graphics ICCV 2019 [code] SVG Font design
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation NeurIPS 2020 [Code] [Project] SVG Vector Graphics Animation, reconstruction and interpolation
SVGformer: Representation Learning for Continuous Vector Graphics using Transformers CVPR 2023 SVG Reconstruction, classification, interpolation, retrieval
StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis arxiv 24.01 SVG generation
  • Self-supervised or few/zero-shot learning
Paper Source Code/Project Link Data Manner Related Applications
Sketch Fewer to Recognize More by Learning a Co-Regularized Sparse Representation TCSVT 2019 few-shot classification
On Learning Semantic Representations for Large-Scale Abstract Sketches TCSVT 2020 [Code] Bitmap and Vector Retrieval and recognition
Deep Self-Supervised Representation Learning for Free-Hand Sketch TCSVT 2020 [Code] Vector/stroke-level Retrieval and recognition
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt CVPR 2020 [Code] Vector/stroke-level Sketch recognition, retrieval, and gestalt
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting CVPR 2021 [Code] Both Vector and Raster-level Recognition
Self-Supervised Learning for Enhancing Spatial Awareness in Free-Hand Sketches IJCAI 2024 [Code] Vector-level Synthesis
  • Scene-level
Paper Source Code/Project Link
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text CVPR 2023
  • Implicit Neural Representations
Paper Source Code/Project Link
SketchINR: A First Look into Sketches as Implicit Neural Representations CVPR 2024

16. Sketch and Visual Correspondence

  • Datasets
Name Paper Source Code/Project Link
CreativeFlow+ Creative Flow+ Dataset CVPR 2019 [code]
ATD-12K Deep animation video interpolation in the wild CVPR 2021 [code]
AnimeRun AnimeRun: 2D Animation Correspondence from Open Source 3D Movies NeurIPS 2022 [code]
PaintBucket-Character Learning Inclusion Matching for Animation Paint Bucket Colorization CVPR 2024 [code]
  • Methods
Matching Type Paper Source Code/Project Link
Region Globally optimal toon tracking SIGGRAPH 2016 [project]
Region The Animation Transformer: Visual Correspondence via Segment Matching ICCV 2021
Region Learning Inclusion Matching for Animation Paint Bucket Colorization CVPR 2024 [code] [project]
Point SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence TCSVT 2020
Point SketchZooms: Deep Multi-view Descriptors for Matching Line Drawings CGF 2021 [project]
Stroke Differential Operators on Sketches via Alpha Contours SIGGRAPH 2023 [code]

17. Sketch Animation/Inbetweening

  • Inbetweening
Paper Source Representation Code/Project Link
BetweenIT: An Interactive Tool for Tight Inbetweening CGF 2010 stroke
Context-Aware Computer Aided Inbetweening TVCG 2017 stroke
FTP-SC: Fuzzy Topology Preserving Stroke Correspondence SCA 2018 stroke [webpage] [video]
Cacani: 2d animation and inbetween software / stroke [software]
Stroke-Based Drawing and Inbetweening with Boundary Strokes CGF 2021 stroke
Joint Stroke Tracing and Correspondence for 2D Animation TOG 2024 stroke [code] [webpage]
Deep Geometrized Cartoon Line Inbetweening ICCV 2023 vertex/point [code]
Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support Inbetweenings ICIP 2019 raster
Bridging the Gap: Sketch-Aware Interpolation Network for High-Quality Animation Sketch Inbetweening ACM MM 2024 raster
ToonCrafter: Generative Cartoon Interpolation arxiv 24.05 raster [code] [webpage]
  • Animation
Paper Source Representation Code/Project Link
Autocomplete Hand-drawn Animations SIGGRAPH Asia 2015 stroke [webpage] [video]
ToonSynth: Example-Based Synthesis of Hand-Colored Cartoon Animations SIGGRAPH 2018 image [webpage]
Live Sketch: Video-driven Dynamic Deformation of Static Drawings CHI 2018 vector [video]
Animating Portrait Line Drawings from a Single Face Photo and a Speech Signal SIGGRAPH 2022 image [code]
A Method for Animating Children’s Drawings of the Human Figure TOG 2023 image [code] [project] [demo]
Non-linear Rough 2D Animation using Transient Embeddings CGF 2023 stroke
Sketch Video Synthesis EG 2024 stroke [code] [project]
Breathing Life Into Sketches Using Text-to-Video Priors CVPR 2024 stroke [code] [project]
AniClipart: Clipart Animation with Text-to-Video Priors arxiv 24.04 image [project]
Dynamic Typography: Bringing Text to Life via Video Diffusion Prior arxiv 24.04 curve [project] [code]
DrawingSpinUp: 3D Animation from Single Character Drawings SIGGRAPH Asia 2024 image [project] [code]

18. Sketch and AR/VR

Paper Source Code/Project Link
SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image UIST 2017 [video]
Model-Guided 3D Sketching TVCG 2018 [video]
Mobi3DSketch: 3D Sketching in Mobile AR CHI 2019 [video]
Interactive Liquid Splash Modeling by User Sketches SIGGRAPH Asia 2020 [video]
Towards 3D VR-Sketch to 3D Shape Retrieval 3DV 2020 [code] [project]
3D Curve Creation on and around Physical Objects with Mobile AR TVCG 2021 [video]
HandPainter - 3D Sketching in VR with Hand-based Physical Proxy CHI 2021 [video]
Fine-Grained VR Sketching: Dataset and Insights 3DV 2021 [code]
Structure-Aware 3D VR Sketch to 3D Shape Retrieval 3DV 2022 [code]
GestureSurface: VR Sketching through Assembling Scaffold Surface with Non-Dominant Hand TVCG 2023
3D VR Sketch Guided 3D Shape Prototyping and Exploration ICCV 2023 [code]

19. Sketch Based Incremental Learning

Paper Source Code/Project Link
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches CVPR 2022 [code]

20. Sketch Quality Measurement

Paper Source Code/Project Link
Annotation-Free Human Sketch Quality Assessment IJCV 2024 [code]
Finding Badly Drawn Bunnies CVPR 2022 [code]

21. Cloud Augmentation with Sketches

Paper Source Code/Project Link
Cloud2Sketch: Augmenting Clouds with Imaginary Sketches ACM MM 2022 [project]

22. Sketch and Re-identification

Paper Source Code/Project Link
Cross-Compatible Embedding and Semantic Consistent Feature Construction for Sketch Re-identification ACM MM 2022 [code]
SketchTrans: Disentangled Prototype Learning with Transformer for Sketch-Photo Recognition TPAMI 2023 [code]

23. Sketch-based Salient Object Detection

Paper Source Code/Project Link
Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings CVPR 2023

24. Study on Strokes

Paper Source Code/Project Link
Neural Brushstroke Engine: Learning a Latent Style Space of Interactive Drawing Tools SIGGRAPH Asia 2022 [project]
Ciallo: GPU-Accelerated Rendering of Vector Brush Strokes SIGGRAPH 2024 [code]

awesome-sketch-based-applications's People

Contributors

ashok-arjun avatar ayankumarbhunia avatar dasayan05 avatar kiteretsu77 avatar markmohr avatar wangqiang9 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

spiritbear000 shitou7630 zeitgeistqian loganlxb doc22940 ayankumarbhunia biswassanket jdc08161063 willpansutd sauradip peterzhousz liuqiskan ashok-arjun wjlee-barco dasayan05 peterzs wn9081 jaedukseo ml-lab albertotono luoxz-ai danielcho-hk wangqiang9 suzhoushr sahil301290 tangyibo123 shubhgurukul learntostick simuji soumitri2001 jackzhousz rajpratyush xdu-aero-association chenweibo212 springtiger achiga-wuhu caobinbc luh1124 warld-alien maelh jona42-ui mdahao zxhxixi sunyuxi spekkiodancer kiteretsu77 dlinwl felixchan9527 phymucs whuhxb yunseok-kang

awesome-sketch-based-applications's Issues

[Novel Applications]

Sketch-a-Classifier: Sketch-based Photo Classifier Generation, CVPR 2018.

Learning to Sketch with Shortcut Cycle Consistency, CVPR 2018.

This is a great effort to organize all the sketch-related papers. Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.