CSV Import about property_web_builder HOT 15 CLOSED

I've been thinking a lot recently about the different options for importing existing sites. The csv import was something I hacked together a while back but it is not being used very much and I don't think I'll be making many changes to it soon ( if more people ask or someone can help then I will)

What I want to spend more time on is to make it easier to import directly from existing websites. I've been working on this in a separate gem and I will soon include that in PWB:

https://github.com/RealEstateWebTools/property_web_scraper

If you let me know the url of your old website I can have a quick look to see how easy it would be to import directly from it.

from property_web_builder.

http://carusoimmobiliare.it/

from property_web_builder.

That should be quite easy to scrape - I will have a look later

from property_web_builder.

Actually, that site does not have much info about a property and it is not so easy to scrape. Do you maybe have listings on idealista.it or some other website that might be easier to scrape?

from property_web_builder.

Hi @Zaldor, the latest build should allow you to import properties from your website.

Try it out here:
https://pwbimporter.herokuapp.com/en/admin/io/properties/from_webpage

Enter this url and try importing it:
http://www.carusoimmobiliare.it/immobiliare/frame.php?page=dettaglio_annuncio&_cod_annuncio=100038

It is still a proof of concept but I will try to polish it up soon.

from property_web_builder.

Return a error...

would anyway be a ONE by ONE process?

from property_web_builder.

mmm, not quite sure what the problem is there. Yes, it would be a one by one process at the moment. To be honest, right now for your use case you might as well just add the entries manually.

When I get the time I will improve on it with better error handling and the ability to retrieve multiple properties at once. That isn't likely to be in the next few weeks though.

from property_web_builder.

from property_web_builder.

Fair point, I will do that when I get the chance.

from property_web_builder.

I received the same error importing from a web page. Ill make a gist of it before eod today. But as far as I can tell its our rails setup, not a pwb error. The log file reports something to the effect
of a remote server not being allowed to open a console. Only the localhost is defined in the messages I saw.

from property_web_builder.

from property_web_builder.

I sent Ed my error log yesterday, lets wait for his opinion. :)

from property_web_builder.

Hi guys, sorry for the delay in replying.

I've implemented the functionality for retrieving info from external websites in a gem which needs to be imported and mounted. I haven't had the time to document it properly but if you are creating a new website from the heroku project it will just work.

If are updating a website you created on heroku using my heroku project then you first need to update it via the heroku dashboard. After doing that you will need to run these commands from the console:

rails db:migrate
rails app:property_web_scraper:db:seed

If you are not using my heroku project then it is a bit more complicated:

1. Add property_web_scraper with this line in your gemfile:
   gem 'property_web_scraper', github: 'RealEstateWebTools/property_web_scraper'
2. Mount it at /io by adding this to your routes.rb:
   mount PropertyWebScraper::Engine => '/io/'
3. Copy the migrations:
   rails property_web_scraper:install:migrations
4. Migrate:
   rails db:migrate
5. Seed the database
   rails property_web_scraper:db:seed
6. Add this line to your config/initializers/assets.rb file:
   Rails.application.config.assets.precompile += %w( pwb/themes/default.css )

from property_web_builder.

I think in my case i should do the update, which I guess is as before:

1. MY git fork > Pull Request > Merge
2. Heroku Dash > Deploy > Update from git

But I don't see any difference from my PWB-for-heroku to etewiah/pwb-for-heroku
Am I missing something?

from property_web_builder.

It looks like you've updated your application successfully. You now need to run the 2 commands I mention.

You can do this by clicking the "more" button on the heroku dashboard (it is in the top righthand corner) and select "Run console". In the form that appears enter "rails db:migrate" and hit run.
Do the same again for the "rails app:property_web_scraper:db:seed".

BTW, I am aiming to improve the documentation so if you can help that will be really appreciated. I have just created a page about how to update heroku installs here:

https://github.com/etewiah/pwb-for-heroku/wiki/Updating-your-heroku-installation

It is a wiki so if there is anything you feel should be added feel free to edit it or create a new page. Add documentation in Italian if you wish ;)

from property_web_builder.