Antenati update overnight changed the URL's about antenati HOT 6 CLOSED

Yeah they're being tricky about it by some sort of filter or something. I went to that example URL above, and using the SaveAllResources plug-in with the option "Include all assets by XHR requests", I was able to save the manifest, but they're definitely doing something to make it harder. For that example, in the "dam-antenati.san.beniculturali" folder in this zip:
https://www.dropbox.com/s/004yjx06omtnf1q/www.antenati.san.beniculturali.it.zip
Btw, thanks for all your hard work on this tool, it's awesome and has been a lifesaver for me in my research!

from antenati.

Works perfect again - thanks!!! You're the best!

from antenati.

Thanks for finding a work-around for their new security. I had made a similar program many years ago in Excel - one that allows you to just list comuni, record types, and years, and the program would go through the website to find each collection before downloading it, without even having to open a browser - but it stopped working since they added password protection to the manifest. Adding the headers as you found still does not work in VBA either, because XHR uses javascript, so I wrote a python program that downloads the html of the manifest given its URL, and then wrapped that in a C++ dll. My Excel program is working perfectly again. Thank you! Now I'm off to download all the record collections in several provinces before they fix the current security hole.

from antenati.

Very curious this effort to make harder our job. I'll have a look at it, we have to figure out why manifest URL return a 403. Few months ago they added geographic and user agent filters. Let's guess what's going on now.

from antenati.

I compared the headers sent by the browser during a standard session, with the headers sent by entering the manifest URL directly in the URL bar. The main difference was the presence of these two headers in the standard case:

	referer: https://www.antenati.san.beniculturali.it/
	origin: https://www.antenati.san.beniculturali.it

Seems that "referer" is enough to fix this issue. Thanks a lot, @jbellanca.

from antenati.

Interestingly, User-Agent seems not needed anymore. I've kept it, adding also Origin, just in case new filters are added in the future. See fa965f2

from antenati.