Comments (7)
Just to test I upgraded to v3.2.3 and the problem still existing there, I would be happy to try to debug it, but at this point unsure how, any pointers would be appreciated.
from organize.
Please update to the new v3, where your script runs fine:
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ SIMULATION │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Config: ".configs/issue375-python.yml"
⚙ Rule #0: New images ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
.venv/docx-template/docProps/thumbnail.jpeg
- (echo) Based on Filesystem ctime .venv/docx-template/docProps/thumbnail.jpeg should go to 2024/02/16/
- (move) Move to /main/pictures/2024/02/16/thumbnail.jpeg
tests/resources/images-with-exif/1.jpg
- (echo) Based on Exif.Image.DateTime tests/resources/images-with-exif/1.jpg should go to 2018/09/29/
- (move) Move to /main/pictures/2018/09/29/1.jpg
tests/resources/images-with-exif/2.jpg
- (echo) Based on Exif.Image.DateTime tests/resources/images-with-exif/2.jpg should go to 2018/04/29/
- (move) Move to /main/pictures/2018/04/29/2.jpg
tests/resources/images-with-exif/3.jpg
- (echo) Based on Exif.Image.DateTime tests/resources/images-with-exif/3.jpg should go to 2017/08/12/
- (move) Move to /main/pictures/2017/08/12/3.jpg
tests/resources/images-with-exif/4.jpg
- (echo) Based on Exif.Image.DateTime tests/resources/images-with-exif/4.jpg should go to 2018/02/22/
- (move) Move to /main/pictures/2018/02/22/4.jpg
tests/resources/images-with-exif/5.jpg
- (echo) Based on Exif.Image.DateTime tests/resources/images-with-exif/5.jpg should go to 2015/07/08/
- (move) Move to /main/pictures/2015/07/08/5.jpg
success 6 / fail 0
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ SIMULATION │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
In v3 you will also see any exceptions raised in your custom python filter.
from organize.
Please note the new and very much improved exif
filter in v3 where you can use exiftool
for parsing your images by setting an environment variable: ORGANIZE_EXIFTOOL_PATH="exiftool"
. Also in this version the dates and times are automatically parsed and available as datetime.datetime
objects.
from organize.
I tested the new exif tool filter as an option but unfortunately I couldn't find a way to reproduce the desired logic.
As per the example config above I am trying to date the images , and the method for doing it so doesn't only depends on exif data, if it exist it is the primary but if it doesn't then I check the file name too and if neither of them give me a date just use file create date.
I am still considering the add more options for example OCR from the bottom corner the date, as old photos had the date burnt in for a few years between full manual and digital cameras. I scan quite a few of those too.
I considered to just package it up into a python library and maybe add video support too , however I am way more a script kiddie rather than a programmer and writing it just for the sake of it makes no sense , unless it is used somewhere other than my own organize config .
However back to the issue , I still see skipped files and I have no means to debug the issue , any suggestions ?
from organize.
Thanks for the hints so far @tfeldmann I narrowed down the error to the exif filter, here is my test setup
The folder has only a test file and a test config
/tmp/issue375$ ls -1
1ae75a1e661e44eda7e1f920d8e5e82c.jpg
test-config.yaml
The config is validated
/tmp/issue375$ organize check test-config.yaml
No problems found in "test-config.yaml".
Setting up and running the test config
/tmp/issue375$ ORGANIZE_EXIFTOOL_PATH="exiftool"
/tmp/issue375$ organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 1 / fail 0
running exiftool manually on the file in Q
/tmp/issue375$ exiftool 1ae75a1e661e44eda7e1f920d8e5e82c.jpg
ExifTool Version Number : 12.40
File Name : 1ae75a1e661e44eda7e1f920d8e5e82c.jpg
Directory : .
File Size : 111 KiB
File Modification Date/Time : 2024:04:13 19:11:18+02:00
File Access Date/Time : 2024:04:13 19:24:28+02:00
File Inode Change Date/Time : 2024:04:13 19:11:18+02:00
File Permissions : -rw-r--r--
File Type : JPEG
File Type Extension : jpg
MIME Type : image/jpeg
JFIF Version : 1.01
Resolution Unit : None
X Resolution : 1
Y Resolution : 1
Image Width : 1080
Image Height : 1080
Encoding Process : Progressive DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2)
Image Size : 1080x1080
Megapixels : 1.2
Also tried with full path for exiftool, same result
/tmp/issue375$ command -v exiftool
/usr/bin/exiftool
/tmp/issue375$ ORGANIZE_EXIFTOOL_PATH="/usr/bin/exiftool"
/tmp/issue375$ organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 1 / fail 0
I am on the latest release from organize and 12.40 on exiftool
/tmp/issue375$ organize --version
organize v3.2.3
/tmp/issue375$ exiftool -ver
12.40
And the config itself
---
rules:
- name: "image with exif filter"
subfolders: true
locations:
- "."
filters:
- extension:
- jpg
- exif
actions:
- echo: "This doesn't work"
- name: "image without exif filter"
subfolders: true
locations:
- "."
filters:
- extension:
- jpg
actions:
- echo: "This works"
or in debug format
Config(
rules=[
Rule(
name='image with exif filter',
enabled=True,
targets='files',
locations=[
Location(
path=[
'.'
],
min_depth=0,
max_depth='inherit',
search='breadth',
exclude_files=set(),
exclude_dirs=set(),
system_exclude_files={
'.localized',
'~$*',
'desktop.ini',
'.DS_Store',
'thumbs.db'
},
system_exclude_dirs={
'.git',
'.svn'
},
filter=None,
filter_dirs=None,
ignore_errors=False
)
],
subfolders=True,
tags=set(),
filters=[
Extension(
extensions={
'jpg'
}
),
Exif(
filter_tags={},
lowercase_keys=True
)
],
filter_mode='all',
actions=[
Echo(
msg="This doesn't work"
)
]
),
Rule(
name='image without exif filter',
enabled=True,
targets='files',
locations=[
Location(
path=[
'.'
],
min_depth=0,
max_depth='inherit',
search='breadth',
exclude_files=set(),
exclude_dirs=set(),
system_exclude_files={
'.localized',
'~$*',
'desktop.ini',
'.DS_Store',
'thumbs.db'
},
system_exclude_dirs={
'.git',
'.svn'
},
filter=None,
filter_dirs=None,
ignore_errors=False
)
],
subfolders=True,
tags=set(),
filters=[
Extension(
extensions={
'jpg'
}
)
],
filter_mode='all',
actions=[
Echo(
msg='This works'
)
]
)
],
_config_path=PosixPath('test-config.yaml')
)
from organize.
What is the output of echo $ORGANIZE_EXIFTOOL_PATH
? I ask because your environment variable might not be set correctly. Can you try: ORGANIZE_EXIFTOOL_PATH='exiftool' organize run test-config.yaml
(in a single command)?
from organize.
The one liner version works fine
/tmp/issue375$ ORGANIZE_EXIFTOOL_PATH='exiftool' organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #0: image with exif filter ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This doesn't work
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 2 / fail 0
but simple variable set does not work
/tmp/issue375$ ORGANIZE_EXIFTOOL_PATH="/usr/bin/exiftool"
/tmp/issue375$ echo $ORGANIZE_EXIFTOOL_PATH
/usr/bin/exiftool
organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 1 / fail 0
and
/tmp/issue375$ ORGANIZE_EXIFTOOL_PATH="exiftool"
/tmp/issue375$ echo $ORGANIZE_EXIFTOOL_PATH
exiftool
/tmp/issue375$ organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 1 / fail 0
The good news is if I export it then it works
/tmp/issue375$ export ORGANIZE_EXIFTOOL_PATH="exiftool"
/tmp/issue375$ organize run test-config.yaml
Config: "test-config.yaml"
⚙ Rule #0: image with exif filter ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This doesn't work
⚙ Rule #1: image without exif filter ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
./
./1ae75a1e661e44eda7e1f920d8e5e82c.jpg
- (echo) This works
success 2 / fail 0
Note, if it's just set as variable then it does not show up in env
So the "fix" can, a simple documentation update to replace
ORGANIZE_EXIFTOOL_PATH="exiftool"
to
export ORGANIZE_EXIFTOOL_PATH="exiftool"
or alternatively the method for detecting the path could be more try to run in as a shell something like command -v exiftool
, and set it based on that.
As far as my issue goes I actually do not use the exif data from organize at all, ( as described above the complex fallback logic ) so removing the filter fixed it for me, but also added the variable to .bashrc
just in case I will use it at some point.
I could not find the relevant doc string in the repo, if you point me at it happy to send a PR on that, or if you consider the detection logic, where the os.environ.get
default fallback value is from command -v
instead of nothing, happy to attempt that path too.
from organize.
Related Issues (20)
- Consistent chronological order Issue in File Renaming HOT 6
- rename action error: ERROR! Missing value for template: 'builtin_function_or_method object' has no attribute 'strftime' HOT 3
- Original Name Variable Issue in Move and Rename Operation HOT 3
- Error: Config problem HOT 4
- Integration of Third-Party Plugin Interface HOT 1
- XDG_CONFIG_HOME ignored by default on macOS HOT 2
- Include multiple actions and mix move and shell crashes organize-tool HOT 3
- Moving a file to a NAS server triggers an error HOT 10
- Have Organize start up and stay running for interaction at the command line, (rather than run a config.yml and close)? HOT 5
- macos_tags problem HOT 5
- Renaming files HOT 5
- Copy folder structure from parent folder, monitored folder, or root folder HOT 4
- Persist logs HOT 1
- Remove macOS tags
- TypeError: unhashable type: 'list' when using aliases HOT 7
- Shell command fails HOT 4
- Help installing in Linux Mint virtual python env HOT 12
- Dockerfile Image Creation Error : COPY failed: file not found in build context or excluded by .dockerignore: stat organize: file does not exist HOT 1
- Dockerfile - Organize Build Errors & Failed Install ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from organize.