Git Product home page Git Product logo

roomnet's Introduction

roomnet

This is a tensorflow implementation of room layout paper: RoomNet: End-to-End Room Layout Estimation.

New: You can find pre-trained model here and sample.npz for get_res.py here

Note: This is a simply out-of-interest experiemnt and I cannot guarantee to get the same effect of the origin paper.

Network

Roomnet network Architecture Here I implement two nets: vanilla encoder-decoder version and 3-iter RCNN refined version. As the author noted, the latter achieve better results.

Data

I use LSUN dataset and please download and prepare the RGB images and get a explorationo of the .mat file it includs because they contain layout type, key points and other information. Here I simply resize the image to (320, 320) with cubic interpolation and do the flip horizontally. (Note: When you flip the image, the order of layout key points should also be fliped.) You can see the preparation of data in prepare_data.py

Pre-requests:

You need to install tensorflow>=1.2, opencv, numpy, scipy and other basic dependencies.

How to use:

Training:

python main.py --train 0 --net vanilla (or rcnn) --out_path path-to-output 

Testing:

python main.py --test 0 --net vanilla (or rcnn) --out_path path-to-output 

Modifications

  1. Classification loss: The author only use loss on the ground truth label while I consider whole classes and I use the common cross entropy loss.
  2. Layout loss: I split the layout map to 3 level according to the numerical value which means the foreground and background. The bigger the value is, the larger weight its corresponding loss takes.
  3. Upsampling layer: Since I don't find upsampling operation that remember the pooling positions and reproject those back, I simply use the conv_transpose operation in tf.

Some Results:

RCNN:

RCNN Results

Vanilla

Vanilla Results

roomnet's People

Contributors

gitbosun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

roomnet's Issues

Mat file link is invalid

Hi!

The link[http://lsun.cs.princeton.edu/challenge/2015/roomlayout/data/training.mat]and [http://lsun.cs.princeton.edu/challenge/2015/roomlayout/data/validation.mat]are invalid.When I searched for this link,the page displays "The requested URL was not found on this server." Can you upload mat files to github?

Thank you very much!

Not getting correct outputs

Hi, thank you for the code. I followed your instructions and managed to train a model using the vanilla network. However, my test results are not correct. The room classification label is mostly correct, but the layout is almost always wrong. See the examples below...
0de2e3da9aea62b94141039401b6205c08800063_pred_4
01fe1b8ceed678b27fd45aca2bde6b8c7f0cad40_pred_5
See my training loss curves below. The classification loss goes down to almost zero but the layout loss is still quite high. Do you know why this might be?
graphg

File Missing 'rcnn/test/2.npz'

I get this error of file missing "out=np.load('rcnn/test/2.npz')"
Kindly guide me how to overcome this.
Regards

File "/home/ahmad/anaconda3/lib/python3.6/site-packages/numpy/lib/npyio.py", line 372, in load
fid = open(file, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'rcnn/test/2.npz'

screenshot from 2018-08-08 22-01-11

Pre-trained model

Hi, I tried to run your pre-trained model with my own images, but I don't imagine how to do that. Can you give me maybe some instructions or tips?

Can you please share the code or pretrained-model working well?

I've been working implementation of RoomNet with your code, but i found there is a few bugs and it still doesn't make a satisfied result. I think there must be a few more bugs that should be modified, but I can't figure it out at all.

Can you please share the code working well??????

sample.npz

Hi, Could you please reshare the sample.npz file ?

In this part:
New: You can find pre-trained model here and sample.npz for get_res.py here

when I click on (Here) I get ERROR 404

Thank you so much

Sample.npz in get_res.py file

In
get_res.py file
out np.load(sample.npz)
Can you please tell me how to take this
sample.npz
Either I need to download or need to prepare like other .npz files using prepare_data.py

how to run inference on single image

i am able to train the model using RCNN and vanilla.

  1. while training and validation, the script uses .mat files to get the image information. But i dont have any clue which information it contains and can we generate .mat file for own images? if so, which parameters I need to consider the to save in MAT file.

  2. After training, How can I run inference on single image or set of images?

please help me out, Thanks

Predictions reproducibility

Hi!
I have a problem with reproducibility of the predictions on inference. Below I attached the code I use to make predictions on samples from the file you provided: sample.npz.

I made one more function in addition to yours train and test for inference:

def inference(args):
    out_path = Path(args.out_path)

    outdir = out_path / 'inference'
    model_dir = out_path / 'model'

    if not outdir.exists():
        os.makedirs(outdir)

    config = tf.ConfigProto()
    config.gpu_options.allow_growth = True
    config.allow_soft_placement = True

    sess = tf.Session(config=config)
    device = '/gpu:0'
    if args.gpu == 1:
        device = '/gpu:1'
    with tf.device(device):
        if args.net == 'vanilla':
            net = RoomnetVanilla()
        if args.net == 'rcnn':
            net = RcnnNet()
        net.build_model()

        i = 0

        net.restore_model(sess, model_dir)
        print('restored')

        start_time = time.time()

        npz = np.load('sample.npz')
        print('sample.npz is now loaded')

        im_in, lay_gt, label_gt, names = npz['im'], npz['gt_lay'], npz['gt_label'], npz['names']
        net.set_feed(im_in, lay_gt, label_gt, i)

        pred_class, pred_lay = net.run_result(sess)
        c_out = np.argmax(pred_class, axis=1)
        c_gt = np.argmax(label_gt, axis=1)
        acc = np.mean(np.array(np.equal(c_out, c_gt), np.float32))

        for j in range(batch_size):
            img = im_in[j]
            outim = get_im(img, pred_lay[j], c_out[j], str(j))
            outim2 = get_im(img, lay_gt[j], c_gt[j], str(j))
            outpath = outdir / str(i)

            if not outdir.exists():
                os.makedirs(outpath)

            f, ax = plt.subplots(1, 2)
            ax = iter(ax.flatten())
            plt.sca(next(ax))
            plt.imshow(outim)
            plt.title(f'predicted {j}')
            plt.xlabel(f'class: {c_out[j]}')
            plt.sca(next(ax))
            plt.imshow(outim2)
            plt.title(f'ground truth')
            plt.xlabel(f'class: {c_gt[j]}')
            plt.show()

        print('[step: %d] [time: %s] [acc: %s]' % (i, time.time() - start_time, acc))
        net.print_loss_acc(sess)

and one more change in main:

    if not args.train == -1:
        train(args)
    if not args.test == -1:
        test(args)
    if not args.inference == -1:
        inference(args)

Then I noticed that after each time I run inference i get different results. For example, couple of predictions for image with index 11 in sample.npz:
image
image
image

P.S. I used the following command to run the inference:
python main.py --inference 0 --net vanilla --out_path pretrained_model --gpu 1.

So, the question is how to make results on inference reproducible?

Mat file in prepare_data.py

Hi!

In your code there exists a trainig.mat file in prepare_data.py line 10 mat='/home/mcg/Data/LSUN/data/training.mat' which isn't explained what exactly it is or how to compose it. Can you elaborate further about it, so that we can try the algorithm?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.