In order to know whether a change occurred, the old options.json must be checked.

Make the scanner build once for ever the composite image used when sharing albums

All these images go in a subfolder of cache folder

Hi,

Compared to the "original" PhotoFloat, I'm not sure if I like the CSS-changes regarding font size (smaller some places), font colors (white vs. blue, and green for the image names), spacing between thumbnails, and showing image names in the album view (the minimalistic "show all thumbnails" w/o anything "extra" was kinda neat). Example in difference between color/font size can be seen in the screenshots in #4.

I guess it's a personal preference, and easily changeable, but it would probably an idea to see what most people prefer, and use that as the standard.

It seems that photofloat tries now to read files that it is not supposed to "read".
so there are two things:

• unreadable -> does that meant that it is skipping the file on purpose?
• the gif at the end seems to have some other issue? (i'll anyways see that i remove the files from my albums folder...)

2017-07-24T12:46:11.097254   |  |  |  |  |--[Next level folder:]                           index.php_files
2017-07-24T12:46:11.097735   |  |  |  |  |  |--[ processing image/video]                      index.htm
2017-07-24T12:46:11.104400   |  |  |  |  |  |  |--[unreadable]                                   :-(
2017-07-24T12:46:11.104536   |  |  |  |  |  |--[Next level folder:]                           index_data
2017-07-24T12:46:11.106926   |  |  |  |  |  |  |--[ processing image/video]                      genericLib.js
2017-07-24T12:46:11.115294   |  |  |  |  |  |  |  |--[unreadable]                                   :-(
2017-07-24T12:46:11.115433   |  |  |  |  |  |  |--[ processing image/video]                      navigation.css
2017-07-24T12:46:11.121287   |  |  |  |  |  |  |  |--[unreadable]                                   :-(
2017-07-24T12:46:11.121404   |  |  |  |  |  |  |--[ processing image/video]                      navigationLib.js
2017-07-24T12:46:11.126207   |  |  |  |  |  |  |  |--[unreadable]                                   :-(
2017-07-24T12:46:11.126322   |  |  |  |  |  |  |--[ processing image/video]                      navigation_a.gif
Traceback (most recent call last):
  File "/var/www/download.kos.li/private/photofloat/scanner/main.py", line 164, in <module>
    main()
  File "/var/www/download.kos.li/private/photofloat/scanner/main.py", line 157, in main
    TreeWalker(Options.config['album_path'], Options.config['cache_path'])
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 28, in __init__
    folders_album = self.walk(self.album_path)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 133, in walk
    next_walked_album = self.walk(entry)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 133, in walk
    next_walked_album = self.walk(entry)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 133, in walk
    next_walked_album = self.walk(entry)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 133, in walk
    next_walked_album = self.walk(entry)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 133, in walk
    next_walked_album = self.walk(entry)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 167, in walk
    photo = Media(entry, self.cache_path)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/PhotoAlbum.py", line 184, in __init__
    self._photo_thumbnails(image, media_path, thumbs_path)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/PhotoAlbum.py", line 372, in _photo_thumbnails
    self._photo_thumbnails_cascade(image, photo_path, thumbs_path)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/PhotoAlbum.py", line 456, in _photo_thumbnails_cascade
    thumb = self._thumbnail(image_to_start_from, photo_path, thumbs_path, thumb_size)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/PhotoAlbum.py", line 555, in _thumbnail
    start_image_copy = self.resize_canvas(start_image_copy, thumb_size, Options.config['background_color'], False)
  File "/var/www/clients/client3/web131/private/photofloat/scanner/PhotoAlbum.py", line 338, in resize_canvas
    newImage = Image.new(mode, (canvas_width, canvas_height), background_color)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1763, in new
    return Image()._new(core.fill(mode, size, color))
TypeError: an integer is required

When a new version changes json files structure, the scanner hasn't any way to detect it.

A label is required in the json file, indicating the json structure, so that the scanner can detect a previous json version and recreate it.

Should be added:

• metadata titles
• metadata values

I like the folder/date switch feature, but I think the folder view should be default. If one needs "by date", you could just view an image, and then navigate from there (or provide a link to "by date" in the bottom of the page or similar). I would probably use the folder/album view 90% of the time, and that means one extra mouse click 90% of the time I visit the gallery.

give the user the possibility to choose a custom favicon

The scanner generates the cache name of every media "normalizing" the characters: everything lower case, space to underscore, etc.

This way media names which are traduced to the same normalized name do not produce errors in scanner, but when seeing them, js code cannot move from one another

This is a long standing bug, from Jason's times

Currently you need to comment out the translation-thingie (011-translations.js) from index.html in order for it not to translate. This causes issues when changing things in the git repo, or when updating and things have changed in index.html.

011-translations.js should be renamed to 011-translations.js.dist or similar, and 011-translations.js added to .gitignore. This would leave it out of the logic in js-css-minify.sh, causing it not to be included by-default, unless someone decides to copy/rename it to 011-translations.js.

I'd prefer to separate: a reduced_sizes array or integers and a thumbnail_size integer value for the square little thumbnail

options.json cannot be in the web site root folder, it's not secure to let the scanner write to that directory, it must be in cache folder.

This way, server_cache_path cannot be an option, because js must know where to find options.json. So cache folder on server will be always "cache"

Currently all "cache" objects are stored directly in the root of the cache folder. This works just fine, but can be a bit cumbersome to "manage"/browse when having a large amount of images. I have about ~50k images in the gallery, and this results in about 250k files in the cache folder (making basic browsing with "ls" commands etc a bit cumbersome).

Suggestion; put all cache files related to an album (thumbnails + json), within a folder bearing the same name, that is placed in the cache folder.

Example; given a cache file with the name this_is_a_test_folder_with_spaces-img_6454.jpg_150s.jpg.
Current path: $webroot/cache/this_is_a_test_folder_with_spaces*
New path: $webroot/cache/this_is_a_test_folder_with_spaces/this_is_a_test_folder_with_spaces*

If you have nested albums, it would be sufficient to put them all in the same folder.

hitting crl-c doen't stop the python app

When you exclude or come upon a folder without read permission, None is returned for mtime. This causes an error when comparing mtime. See error below and suggested patch. I guess it can be fixed in several ways...

Great work Paulo, I really appreciate your work and the improvements you are introducing!

thomas@zombie:/mnt/media/various/photoshare/ps/myphotoshare/scanner$ ./main.py ../../myphotoshare.conf > /tmp/ps.log
Traceback (most recent call last):
  File "./main.py", line 258, in <module>
    main()
  File "./main.py", line 250, in main
    TreeWalker()                                                                                                                                           File "/mnt/media/various/photoshare/ps/myphotoshare/scanner/TreeWalker.py", line 50, in __init__
    [folders_album, num, max_file_date] = self.walk(Options.config['album_path'], album_cache_base, origin_album)
  File "/mnt/media/various/photoshare/ps/myphotoshare/scanner/TreeWalker.py", line 318, in walk
    max_file_date = max(max_file_date, sub_max_file_date)
TypeError: can't compare datetime.datetime to NoneType

thomas@zombie:/mnt/media/various/photoshare/ps/myphotoshare/scanner$ git diff
diff --git a/scanner/TreeWalker.py b/scanner/TreeWalker.py
index 8eb5cb2..0c28d33 100644
--- a/scanner/TreeWalker.py
+++ b/scanner/TreeWalker.py
@@ -189,14 +189,14 @@ class TreeWalker:
                if not os.access(absolute_path, os.R_OK | os.X_OK):
                        message("access denied to directory", os.path.basename(absolute_path), 1)
                        back_level()
-                       return [None, 0, None]
+                       return [None, 0, datetime(1900,1,1)]
                listdir = os.listdir(absolute_path)
                if Options.config['exclude_tree_marker'] in listdir:
                        next_level()
                        message("excluded with subfolders by marker file", Options.config['exclude_tree_marker'], 4)
                        back_level()
                        back_level()
-                       return [None, 0, None]
+                       return [None, 0, datetime(1900,1,1)]
                skip_files = False
                if Options.config['exclude_files_marker'] in listdir:
                        next_level()
thomas@zombie:/mnt/media/various/photoshare/ps/myphotoshare/scanner$

Videos are not working/displayed/playable when browsing albums in fullscreen. This has "never" worked, but should, if possible, be solved/fixed.

Improvement suggestion, make crf for video configurable. I want to keep down the size of the transcoded media.

diff --git a/myphotoshare.conf.defaults b/myphotoshare.conf.defaults
index 906fa07..8c765f3 100644
--- a/myphotoshare.conf.defaults
+++ b/myphotoshare.conf.defaults
@@ -55,6 +55,9 @@ subdir_method = md5
 # The transcode bitrate used for videos: an integer, the bitrate in bits/s, even in notation nnk, nnM
 video_transcode_bitrate = 1M

+# The crf used for videos
+video_crf = 20
+
 # The jpg quality parameter for generating the reduced size images and the tumbnails
 # an integer number 1 -100
 jpeg_quality = 95
diff --git a/scanner/PhotoAlbum.py b/scanner/PhotoAlbum.py
index a8eaa11..61f6a2a 100644
--- a/scanner/PhotoAlbum.py
+++ b/scanner/PhotoAlbum.py
@@ -795,7 +795,7 @@ class Media(object):
                        '-preset', 'slow',                                      # set specific preset that provides a certain encoding speed to compression ratio
                        '-profile:v', 'baseline',                               # set output to specific h264 profile
                        '-level', '3.0',                                        # sets highest compatibility with target devices
-                       '-crf', '20',                                           # set quality
+                       '-crf', Options.config['video_crf'],
                # set quality
                        '-b:v', Options.config['video_transcode_bitrate'],      # set videobitrate
                        '-strict', 'experimental',                              # allow native aac codec below
                        '-c:a', 'aac',                                          # set aac as audiocodec

now in master a parameter can be set in order to choose parallel or cascade processing of thumbnails

If parallel is set (the method is derived from Joachim's code), stating that a thumbnail exists requieres about a fraction of a second, while in cascading mode (derived from paolo code) it takes about 1/10 time

@jallakim could you investigate why?

is there a way that i could exclude certain folders, e.g. an idea would be to add a file ".photofloat.exclude" or so in a folder that shouldn't be scanned by photofloat. this would make it flexible and i wouldn't need to keep a central list of folders that should be excluded, but the information is "self-included".

any idea what i am doing wrong? I used the default conf-file and only changed the paths. I am running it on debian wheezy, may that be a problem due to too old version? Python 2.7.3 and php-gd is at 5.6.31

2017-07-23T16:07:42.112233 |--[Next level folder:] albums
2017-07-23T16:07:42.112586 | |--[Next level folder:] 2000
2017-07-23T16:07:42.127229 | | |--[Next level folder:] 2000-08-12 - Bünzli 9 im Bierhübeli
2017-07-23T16:07:42.131751 | | | |--[Next level folder:] Dima
2017-07-23T16:07:42.145290 | | | | |--[ processing image/video] Dsc00066b.jpg
2017-07-23T16:07:42.437353 | | | | | |--[reducing size] 1600
2017-07-23T16:07:42.748535 | | | | | |--[reducing size] 1200
2017-07-23T16:07:42.900639 | | | | | |--[reducing size] 800
2017-07-23T16:07:42.930821 | | | | | |--[thumbing for albums] 200, fit size
2017-07-23T16:07:42.960879 | | | | | |--[thumbing for albums] 200, square
Traceback (most recent call last):
File "/var/www/download.kos.li/private/PhotoFloat/scanner/main.py", line 164, in
main()
File "/var/www/download.kos.li/private/PhotoFloat/scanner/main.py", line 157, in main
TreeWalker(Options.config['album_path'], Options.config['cache_path'])
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/TreeWalker.py", line 28, in init
folders_album = self.walk(self.album_path)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/TreeWalker.py", line 133, in walk
next_walked_album = self.walk(entry)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/TreeWalker.py", line 133, in walk
next_walked_album = self.walk(entry)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/TreeWalker.py", line 133, in walk
next_walked_album = self.walk(entry)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/TreeWalker.py", line 167, in walk
photo = Media(entry, self.cache_path)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/PhotoAlbum.py", line 184, in init
self._photo_thumbnails(image, media_path, thumbs_path)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/PhotoAlbum.py", line 372, in _photo_thumbnails
self._photo_thumbnails_cascade(image, photo_path, thumbs_path)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/PhotoAlbum.py", line 476, in _photo_thumbnails_cascade
thumb = self._thumbnail(smallest_reduced_size_image, photo_path, thumbs_path, thumb_size, thumb_type)
File "/var/www/clients/client3/web131/private/PhotoFloat/scanner/PhotoAlbum.py", line 551, in _thumbnail
start_image_copy.thumbnail((thumb_size, thumb_size), Image.ANTIALIAS)
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1562, in thumbnail
im = self.resize(size, resample)
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1298, in resize
im = self.im.stretch(size, resample)
TypeError: integer argument expected, got float

`

Not sure if the "folders view"/"year view"/"month view"/"day view" is very consistent?

I.e. when are which of them supposed to be displayed? They keep changing around without a clear pattern? It also seems that the number of links changes between one and two, depending on where you are (only one is shown when in "year view", but two for all the other modes).

Actually this thumbnail is a square one.

The option could permit a rectangular thumbnail, with two options:

• a fixed height one (h trailing letter)
• a square canvas one (c trailing letter)

I'd like to implement social share buttons, at least for facebook and twitter

the problem now is that if you share any photofloat page you are sharing the nude html.

We must solve two problems:

• look for a way to have a static page with at least the image: an url where the image is shown independently of js
• implement the buttons who share the correct url

In order to have the static url I was thinking in generating in cache folder an html page for each image, it could be the simple index.html where the img is inserted by python with a replace command

Hi,

There seems to be a long-standing problem that some photos cannot be transposed. In old versions (of Joachim patches to photofloat), photo is not transposed on my machine but the treewalker seem to handle the exception. This is not the case for myphotoshare / Paulo photofloat fork.

See printouts below. Unfortunately, the photo is too big to attach... (41MB). It is a panorama from several individual photos I manual composed.

Regards,

Thomas

thomas@zombie:$ identify  20160410_100829.jpg
20160410_100829.jpg JPEG 23392x1600 23392x1600+0+0 8-bit sRGB 43.26MB 0.010u 0:00.010
identify-im6.q16: Invalid SOS parameters for sequential JPEG `20160410_100829.jpg' @ warning/jpeg.c/JPEGWarningHandler/352.

2017-08-08 14:51:22.033351   |  |  |--[ processing image/video]                      20160410_100829.jpg
Traceback (most recent call last):
  File "./main.py", line 248, in <module>
    main()
  File "./main.py", line 241, in main
    TreeWalker()
  File "/mnt/media/various/photoshare/myphotoshare/scanner/TreeWalker.py", line 29, in __init__
    [folders_album, num] = self.walk(Options.config['album_path'])
  File "/mnt/media/various/photoshare/myphotoshare/scanner/TreeWalker.py", line 168, in walk
    [next_walked_album, num] = self.walk(entry_with_path, album)
  File "/mnt/media/various/photoshare/myphotoshare/scanner/TreeWalker.py", line 210, in walk
    media = Media(album, entry_with_path, Options.config['cache_path'])
  File "/mnt/media/various/photoshare/myphotoshare/scanner/PhotoAlbum.py", line 222, in __init__
    self._photo_thumbnails(image, media_path, Options.config['cache_path'])
  File "/mnt/media/various/photoshare/myphotoshare/scanner/PhotoAlbum.py", line 369, in _photo_thumbnails
    mirror = image.transpose(Image.ROTATE_180)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1966, in transpose
    self.load()
  File "/usr/lib/python2.7/dist-packages/PIL/ImageFile.py", line 244, in load
    raise_ioerror(err_code)
  File "/usr/lib/python2.7/dist-packages/PIL/ImageFile.py", line 59, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file

The link in the footer is wrong (pointing to an old/renamed GitHub repo).

implemente an options to include google analytics (or other service) code in page via php

Folders/albums when in the folder view should just be displayed as normal thumbnails (equal to the images). IMHO it only makes sense to make folders/albums and images distinguishable when in the dates view (where you can have both folders/albums and images in the same page).

http://palmaro.qumran2.net/?m=cache%2Fcache_album%2F_folders-matrimoni-pinna_pirastu_2017.jpg#!/_folders-matrimoni-pinna_pirastu_2017

We should show image immediately, before preloading album, if we visit a link to an album directly.

Something like this; https://git.jocke.no/photofloat/commit/?h=patches&id=20fe79eb4e9fb1fa3e295f15542b1e2900fa4253

Now translations are in the index.html, and they are selected via jquery calls

They must be put in a separate file (json file?)

Forgot to have "Math." before max()

--- a/web/js/012-display.js
+++ b/web/js/012-display.js
@@ -634,7 +634,7 @@ $(document).ready(function() {
                                                }
                                                calculatedWidth = thumbWidth;
                                        } else if (Options.media_thumb_type == "square") {
-                                               if (max(width, height) < Options.media_thumb_size) {
+                                               if ( Math.max(width, height) < Options.media_thumb_size) {
                                                        thumbHeight = height;
                                                        thumbWidth = width;
                                                } else {

i get the following error with the latest version and don't know where to look at for the cause of the error?
58 2017-09-25 11:27:10.822554 |--[piwik_id] [DEFAULT ]
Traceback (most recent call last):
File "/var/www/download.kos.li/private/photofloat/scanner/main.py", line 260, in
main()
File "/var/www/download.kos.li/private/photofloat/scanner/main.py", line 252, in main
TreeWalker()
File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 63, in init
by_date_album = self.generate_date_album(origin_album)
File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 155, in generate_date_album
self.generate_composite_image(by_date_album, by_date_max_file_date)
File "/var/www/clients/client3/web131/private/photofloat/scanner/TreeWalker.py", line 544, in generate_composite_image
random_thumbnails.append(random_thumbnails[i])
IndexError: list index out of range

This way we remove the metadata bar links to switch to by date view

... and I still don't understandy sufficiently the code in order to work on it

Images seems to "overflow" on the top, going behind the album/path at the top.

Old behaviour:

New behaviour:

Should be fixed so that it uses the old behaviour.

This way media in the same tree would be grouped in the same subdir

We shouldn't have to rethumb/transcode all images/videos in an album if we interrupt photofloat mid-cycle. Currently, if main.py was half way through an album scan/run, hitting interrupt would cause you to start all over again for that album (even if ~50% of the thumbs/transcodes already was made, it would re-make them).

We should maybe check if file already exists (even if there is no .json-file), and if so, re-use it. Would probably have to do some kind of integrity check so that we don't get corrupt images, or scenarios where only 2 out of 4 thumbs of an image was created (in which we would have to re-make all the thumbs for that image).

detect link type from m parameter file extension

Hi,

Why are we not using the minified .js/.css files in index.html?

And why are you using them individually, rather than the one blob "scripts.min.js"/"styles.min.css"?

You'd want as few HTTP-requests as possible, so by using them individually, it's theoretical slower. In addition (as a side effect); by not using the minified files, access is blocked by the .htaccess-files (if you're using apache), since they only allow the ".min.js" and ".min.css" files.

from image 1 go to date view -> you see all the copies of the same image, but you cannot pass from one another

We should add a piece of hash for album viewed images, where the 2nd piece is the album, and is discarded in album view, while in date view it permits to switch between the various copies of the same image

I thumbnail_generation_mode=parallel, thumnails of tif images are reported as currupt like this:

2017-07-06T07:40:33.525596   |  |  |  |--[ processing image/video]                 1999104.tif

2017-07-06T07:40:34.983790   |  |  |  |  |--[corrupt image]                           1999104.tif

2017-07-06T07:40:35.091845   |  |  |  |  |--[corrupt image]                           1999104.tif

2017-07-06T07:40:35.203525   |  |  |  |  |--[corrupt image]                           1999104.tif

2017-07-06T07:40:35.317236   |  |  |  |  |--[corrupt image]                           1999104.tif

2017-07-06T07:40:35.454228   |  |  |  |  |--[corrupt image]                           1999104.tif

Switching back to cascade makes thumbnail generation work as normal.

actually they are left there

JS should reduce buttons size if only one fits in a row or in a column

actually they are always sorted past -> future

The scanner generates the cache name of every folder "normalizing" the characters: everything lower case, space to underscore, etc.

This way album names which are traduced to the same normalized name produce errors in scanner

This is a long standing bug, from Jason's times

ios safari doesn't support fullscreen api

As part of your modifications, all thumbnail sizes other than 150s and 1600 were removed. Getting rid of 75s seems fair (not even sure when that was used; not even on mobile), and also bumping the size to 1600px. However, we should produce two sizes by default, where one of them is smaller than 1600px, as that would be preferable when browsing via mobile.

Viewing the album on a mobile (iPhone6), it prefers 1024px before 1600px when viewing single images, and frankly, I cannot spot a difference with my bare eye. I haven't tested on a retina iPad, which might prefer the 1600px.

In any case; loading 1024px is about 50% the size of the 1600px, making loading over 3G/4G much faster.

Not saying we should do 1024px + 1600px, but we should do $somesize + 1600px.

I also realize that this is easy to customize, but I think it would be a good idea that the default setting is to produce two thumbs of different size.