Git Product home page Git Product logo

Layout Generation and Baseline Implementation

Contents

Layout VAE

LayoutVAE is a variational autoencoder based model . It is a probabilistic and autoregressive model which generates the scene layout using latent variables in lower dimensions . It is capable of generating different layouts using the same data point.

  • CountVAE: This is the first part of the layoutVAE model; it takes the label set as input and predicts the counts of bounding boxes for corresponding labels. The input is provided as multilabel encoding.
  • BBox VAE: This the second part of the model was BBox VAE with LSTM based Embedding Generation. Similar to Countvae here also previous predictions along with the label set and label counts are used as conditioning info for current predictions.

Layout VAE Model

modelvae

Flow Diagram

Architecture

Results Obtained

VAE_result

Layout Transformer

Layout Transformer is a model proposed for generating structured layouts which can be used for documents, websites, apps, etc. It uses the decoder block of the Transformer Model, which is able to capture the relation of the document boxes with the previously predicted boxes (or inputs). Since it is an auto-regressive model, it can be used to generate entirely new layouts or to complete existing partial layouts. The paper also emphasized on the fact that this model performs better than the existing models (at that time) and is better in the following aspects:

  • Able to generate layouts of arbitrary lengths
  • Gives better alignment due to the discretized grid
  • Is able to effectively capture the relationships between boxes in a single layout, which gives meaningful layouts

Layout Transformer Model Architecture

Trans_model

Results

Trans_result

LayoutGAN

LayoutGAN uses a GAN network , with the generator taking randomly sampled inputs (class probabilities and geometric parameters) as parameters, arranging them and thus producing refined geometric and class parameters.

Architecture

Results on MNIST

Results on single column layouts

Quantitative Comparison

A total of three metrics were used to compare the models.

  • Overlapping Loss
  • Interection over Union (IoU)
  • Alignment Loss

After Calculating the losses for each model, the following comparison table was obtained:

Overlap IOU Alignment
Original Data 1.000000 1.000000 1.000000
LayoutGAN 1172.005234 2745.437529 1.164882
LayoutVAE 119.320127 185.864381 3.493406
Layout Transformer 1.090315 1.422297 0.739862

layout-generation's Projects

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.