Future of the project / collaboration about lodestone HOT 15 OPEN

@James-Firth Yes, it shall be like that. It is up to the user to choose what functionalities fit his needs.

Regarding the UI, this is very good point. I was thinking about it somewhere along the lines how Google Drive / Docs does it. You have one lets say more generic approach (Google Drive), showing you all the possible files with timeline, full text search, etc. However you would also have specialized views (Google Docs, Google Sheets, ...) showing you a tailored perspective - subset of the files. For example for photos I imagine this would be primarily structured along albums and identified people, objects, places, ... Under the hood, the BE part, is pretty much agnostic to it. It is all build around tags and properties. But on FE I believe we really need to tailor it for decent UX. Mimir has it in form of configurable indices. I am expecting that you can just choose what "view" fits your case for each index.

I welcome any contributors, both BE, FE or even people interested developing individual processors :)
The FE is Typescript + React + Chakra UI if you are interested.

from lodestone.

Wow! That looks incredible. The tagging rules engine is specifically something that I had hoped to add to Lodestone at some point in the future. I also like the ML possibilities as well as I think that would be an amazing addition if we can make the administration of it easy for end users.

Were you thinking of continuing with Mimir as a separate, standalone project complete with UI or combining the 2 projects into one? From what you've said and after looking over your docs, so far Mimir is a reorganized and upgraded version of Lodestone's backend components. Does it make sense to start a completely new project, or join the two together and take advantage of the (admittedly small so far) community that Lodestone already has?

from lodestone.

Thanks, I am glad you like it. UI for the rules will surely be a challenge :)

Honestly, I haven't got that far to think about the project organization just yet. I just wanted to reach out and put it out there for discussion :) I think I am ok with both variants. For me the most important thing is for it to start getting some traction. It is big project and we will need to build active community if we want it ever to reach its potential.

I don't really mind if it is going to be called Lodestone 2.0 or Mimir or whatever. But I put decent effort into setting up the project on GitLab with all the CI / integrated package repositories goodness coming with it. So I would rather if it stayed there as I don't have that much experience with GitHub actions or willingness to migrate it just for the sake of it.

from lodestone.

I can take care of the automation pieces. That's one of the areas that really interest me (and honestly I'm better at that stuff these days than I am at writing Angular code). Let's definitely keep talking and see where things go.

from lodestone.

Ok, fair enough. But for now, migration itself won't really help us move forward.

For me the biggest question to solve is how to bootstrap the FE development. Without UI it has no way of reaching wider audience. We are pretty much looking at a new codebase giving the fact how much the API changed with Mimir. I can surely start but I was hoping you would drive it. There is still plenty BE stuff for me to cover.

Now that you mention Angular, it is probably the right time to ask - I am actually a TS/React developer as my main profession. I initially thought I would go the React way ™️. Would make sense to me personally, given my background, and the fact that I think we could reach maximum amount of possible contributors that way. I am not set against Angular, but it would mean you would have to be the one to drive that forward. I am sure I would be able to contribute in some way, but I would definitely not be proficient enough to set the project course with the best practices in mind.

from lodestone.

Ah, the age-old debate (at least in front end development) :) I actually know next-to-nothing about actual React development. For our needs at work, the Angular ecosystem was the better approach to support the dozen plus teams that need to work in our monorepo. Obviously any technology can be good or bad depending on the engineers writing the code and no one tech is the right answer for everything. I do have a fundamental dislike of the idea of JSX though :)

from lodestone.

Uff, that kind of puts us in the tough spot 😆
You know what, it would not hurt me to learn a bit of Angular. I have nothing against it personally. It's just as you mentioned, React was the way I have chosen for projects at my work :) But one day you will have to tell me, why from all the things you don't like about React is JSX the one which hurts you the most ;)

from lodestone.

Hahaha...fair enough! DM me on Twitter (same username) or via the Gitter link in the Readme and I'll tell you (so I don't get hate mail here).

from lodestone.

Hi @dskaggs and @adam-stanek I was just looking at getting a nice home server set up recently and stumbled across Lodestone

Has most focus moved on to Mimir at this point on Gitlab or was there still thoughts of merging, etc? I'm eyeing the repos and seeing a bit more activity there but want to know where I should be looking :)

from lodestone.

@James-Firth Honestly, I think when I agreed to take over as maintainer on this project, I had a "my eyes are bigger than my stomach" moment. I've been remiss about moving it forward. @adam-stanek helped out a bunch on the Go services some time ago but I just haven't had the time to put into Lodestone. I've actually been experimenting with PaperlessNG in my home lab and am probably going to go that route for my uses.

from lodestone.

@dskaggs Totally fair, I've jumped on my fair share of projects with good intent but not enough time! (I'm a team lead and sometimes even get to write code as a fullstack dev sometimes!)

I know the original author mentioned PaperlessNG messes with the files, any chance you could elaborate on what it does?
I initially saw PaperlessNG and was thinking of it for my homelab, but liked the UI of Lodestone more (despite seeing the deprecation etc)

I'll have to either check out @adam-stanek's Mimir when it's ready for use or check out PaperlessNG myself :)

from lodestone.

Hi @James-Firth, I did decent amount of work on Mimir, but I think it is still a few months away from prime time. It is completely new platform, not a Lodestone evolution. It builds upon similiar principles - Elasticsearch and S3 backend on microservice architecture, imagick + tika for processing under the hood. But my ambition was much bigger then DMS. I am building general file indexing platform. It does not matter if you are indexing documents, your photos, videos, or whatever files you have. From each you can extract the metadata and search through them. I was aiming for making the whole thing extendable so that it is very easy to write your own processor for specific files you might have (there is an exposed API + library for that). It is built around of set of rules of what the system should do with individual files.

From the end-user features:

• Multi-level tagging (you can have different types of tags)
• Tagging and processing based on file path - you can leverage your existing directory structure and tag the files by it or process them differently based on their file location
• User defined properties - you can define whatever you want for your files - ie. due dates for invoices
• Faceted search - you can narrow down your full-text searches by tags and properties
• Downloading results of search as ZIP archive
• Sharing links to download individual files
• Native upload directly in the app
• Authentication - integration with proxies and OIDC
• Index level permissions, document / tag level probably later this year

Majority of that stuff is fully implemented on backend, but I am only slowly getting there on the frontend part. The thing which is holding it back the most is any frontend for the configuration, which makes it quite hard to start with in the current state. I would suggest to check back in few months unless you want to make your hands dirty :)

from lodestone.

@adam-stanek (whoops this sate in drafts all day) Thanks for the info! Yeah I can see this expanding to more than DMS. It seems modular though so if someone was focused on DMS they could "just" use those features right? Love the extensible idea.

For the UI are you thinking of also going modular with a "documents tab" and a "photos tab" etc? I could see photo management being a very different UX for instance.

I can't promise any time but are you looking for help via issues or PRs or anything? I'm intrigued by the backend languages but I do React work when I'm wearing my developer hat at work.

from lodestone.

Thanks for the discussion you two! I'll keep Mimir discussion on the Mimir repo and Lodestone info here if I do any more poking around :)

from lodestone.

Did anything ever come of this? It seems like maybe Loadstone is dead at this point, since @dskaggs has moved to paperless-NG?

One thing (among others) that Loadstone offers which isn't available in Paperless-NGX is that it leaves files alone on the disk instead of reorganizing/renaming them like Paperless-NGX does.

from lodestone.