archlinux AUR package and more detailed install instructions about davis HOT 39 CLOSED

Hey, thanks for the feedback. We did a little digging in the meantime, too.

We tried the caldav reminder setup from nextcloud with the same result. When that happened I searched for the issue in nextcloud context and stumbled upon the same link that you posted. But the content is not true. It was an early iOS13 beta restriction/issue.

Reading through the nextcloud issues brought me on the right track ;) I always test my connections with local network https connections and lets encrypt certificates from a virtual box before implementing the production usage. THAT does not work any more. So it was a certificate issue. Apple changed that with iOS 13. Reminders from iOS 13 do only accept ssl connections from the web with a valid certificate, not from local lan ips. For this to work you would have to add and trust it manually.

https://support.apple.com/en-us/HT204477

iOS ask for trusting the certificate when connecting to the server via local network, but that does not seem to be enough for the reminders app and it does not add the certificate to the iOS trust settings. Calendar via the same connection do work.

Then I set the test environment to forward the port to my testbox and connected via web, and it WORKS ;)))

A lot learned again today ;)

I will document everything in the next days and we will do some testing.

Thanks again for the project and all the help. I will get back after documenting, testing and diving deeper in the nginx settings and the dotenv documentation.

Thanks a lot - have a great day ;)))

from davis.

Glad it works fine !
v1.4.6 is available now, see https://github.com/tchapi/davis/releases/tag/v1.4.6

I'll close this issue for now :)

from davis.

Hello

Thanks for your feedback.

Is it possible to submit an AUR package for archlinux for easier installation?

Unfortunately I will not do that — because I don't have the time / resource to maintain packages first, and also because this web app is not linked / dependent on any architecture or linux distro. It's just plain PHP.

1. You mean the whole project ? It's a Symfony app, so just grab the source code (download a release, untar it) and do composer install to fetch the dependencies. You can then serve the public/index.php file via your webserver of choice and it will work. That's it. See https://symfony.com/doc/current/setup/file_permissions.html for more details and if you don't know Symfony or how to deploy it : https://symfony.com/doc/current/deployment.html
   1b. Also, there's a Dockerfile in the repo that you can have a look at to see how it is deployed. It's quite easy.
2. If you're not migrating from Baikal, just run bin/console migrate as indicated in the README. You need to fill in the correct database url beforehand in the .env file of course
3. I don't think you need a very custom pool - and it depends on your server more than on the app (the available cpus etc). The default one should do fine.
4. Yes, it's this secret. But you need to change it for your environments of course. The .env file is just the default placeholder value.

Don't hesitate if you have other questions
BR

from davis.

Hey,

thanks for the fast answer. I`m not familiar with symfony or composer.

1. So I first instll composer and symfony-cli apps from the repos?
   The root of the webserver is in /usr/share/webapps. So I check out the source to /usr/share/webapps/davis?
   Then I cd /usr/share/webapps/davis an run composer install?
2. O.k., I`ll try that.
3. I think I probably need one and will play with the settings. Needing pdo_mysql and intl extensions is a good start.
4. I´m not familiar with .env files. What exact changes would have to be made to the file? How do I get a valid secret?

Thanks again. I will try as soon as I know how to install and give feedback.

from davis.

1 -> You just need composer, nothing else. You can install in /usr/share/webapps/davis if you wish, just change the root of your webserver accordingly.
4 -> Well you need to change the values according to your environment : where is your database, the no-reply email, the auth method, etc ... A valid secret is just a string that you, and only you, know.

I would recommend that you make yourself familiar with environment variables, and how PHP-FPM and Nginx interact so that you can understand better how to deploy this tool. A primer in Symfony would help, too. It is not like Baïkal, as it's aimed at people that already know a bit how to deploy and work with web apps in PHP behind a webserver. If you're looking for a web interface to change the settings, then stick to Baïkal for now, it will be easier to maintain for you.

from davis.

Thanks for the next round of input. I dived into it yesterday and achieved first results. I will get back with them and perhaps with a few more questions in the next days.

from davis.

Hey,

I really got everything setup and installed after quite some learning, trial and error ;)

When I'm done I will hand over the install script, the php and the nginx config. I think this will help a lot if other (not so experienced) users would like to try/use this.

I installed to a subdirectory /usr/share/webapps/davis where the root of the web server is /usr/share/webapps/.

Unfortunately I discovered a few problems and questions on the way:

1 permissions
After installation I gave the following ownerships/permissions. Do they sound reasonable in regards to functionality and security?

chown -R http:http /usr/share/webapps/davis
chmod 750 /usr/share/webapps/davis

2 href and src changes
Due to the subdirectory installation I had do modify href and src and add a location block to the nginx config.

for HTML_FILE in $(find /usr/share/webapps/davis/templates -name "*.html.twig")
do
	sed -i 's|="\/images\/|="/davis/public/images/|' "$HTML_FILE"
	sed -i 's|="\/css\/|="/davis/public/css/|' "$HTML_FILE"
	sed -i 's|="\/js\/|="/davis/public/js/|' "$HTML_FILE"
done

nginx location block below fastcgi block

    location ~ ^\/davis/public\/.+[^\/]\.(?:css|png|js)$ {
        try_files $uri /davis/public/index.php$request_uri;
        add_header Cache-Control "public, max-age=15778463";
        # documentation see above
        #include configs-enabled/header/headers_https.conf;
        #include configs-enabled/header/header_csp.conf;
        # optional: don't log access to assets
        access_log off;
    }

Can you change the href and src to work in the root dir and in subdirs?

3 .env files
Is it meant to delete .env and env.local after running composer dump-env prod?
The command created a env.local.php successfully.

The .env.local file is created with the following permissions. Do they sound reasonable?

chmod 640 /usr/share/webapps/davis/.env.local
chown root:http /usr/share/webapps/davis/.env.local

Is it really ment to store the marinade database password in plain text in the .env.local file?

4 wrong davis version in dashboard
The dashboard shows davis version 1.2.0 instead of current installed version.

5 fail2ban
Did you already work out a failed2ban rule to protect the login site or is it meant to disable it after configuration?

6 functionality and reminders issue
I use archlinux as a server and macOS and iOS as clients. Here a little overview what I found working (in short tests).

macOS (default apple onboard tools)
Calendar - working
Contacts - working (only with the default address book of the user)
Reminders - working

iOS latest version (default apple onboard tools)
Calendar - working
Contacts - working (only with the default address book of the user)
Reminders - NOT WORKING

So the main problem/issue in the moment is that reminders on iOS are not working. iOS connects to the server, but doesn't show the calendar as reminder.

If I remember correctly in baikal and radicale there is a possibility to add an "entry" to the user and decide if it is a calender, a reminder, a note or a combination of them. There are no log entries in the nginx log about the iOS device trying to connect to the reminder. Perhaps it does not notify the reminder as one.

All you see in the reminders app is the server entry without reminder lists.
And there are two more issues with reminders.
Adding a reminder list on iOS adds a local list entry to this server which does not get synced and not created on the server.
Adding a reminder list on macOS adds a local list entry for a second and disappears. It also doesn't get synced or created on the server.

This is my main problem and actually important to me as we all use this feature. It would be very nice if you could look into this. I will help testing and providing what I can. I tried the latest stable and git version of davis.

7 php 7.4
Archlinux upgraded to php 7.4 this weekend. When I upgrade all php packages I get an error on the login site.

	Oops! An Error Occurred
	The server returned a "500 Internal Server Error".
	Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

Downgrading to 7.3 fixes the functionality as described in point 6 with not working iOS reminders.


Thanks a lot again as I would like to use this in production after fixing the iOS reminders issue.


Have a nice sunday afternoon.


Kind regards

from davis.

In addition to number 6 there is one more reason to separate calendar, notes and reminders functionality (even if it is not the cause of the issue that it is currently not working with iOS). Currently if you add a calendar to a user and enable the option for reminder list "todo" then (if you enable calendar and reminders for this caldav account) the same entry also appears as a calendar and a reminders list in macOS. It would make sense to have different options available for an "entry":

reminders only
notes only
calendar only
reminder + notes
reminder + calendar
calendar + notes
reminder + notes + calendar

from davis.

Hello !

1 - I don't know your webserver and how you handle permissions so I can't really tell. Your webserver needs to be able to read the files of the application, and write logs in var/log and cache in var/cache, that's it. See https://symfony.com/doc/current/setup/file_permissions.html

2 - You shouldn't do this, this is a recipe for disaster (and plain wrong). Adapt your nginx config to point to your public directory, and rewrite the urls directly with Nginx if you need. The provided Nginx config should be everything you need really (Have a look in the docker/configurations folder). Moreover, you shoud deploy a web app like this at the root of your domain.

3 - In production, you would rather indicate your environment via fastcgi_params in Nginx. See https://symfony.com/doc/current/components/dotenv.html

4 - Fixed in latest release, thanks.

5 - I don't use fail2ban, and I don't know how it works — this is out of scope of this repo

6 - I will have a look when I have a chance. Probably a sabre/dav issue, though. Regarding your second point, it's up to the client, not the server, to make this distinction.

7 - Stick to PHP 7.3 for now. 7.4 is literaly 4 days old, you should expect things to break with it (Although I don't think your 500 is linked to PHP 7.4, check the logs for more info — I would guess you didn't install the necessary libs like php-intl);

from davis.

PS for point 6, after enabling TODOs on the calendar, it works for me on macOS (I can't test on iOS though).

from davis.

Hey, thanks for the answers.

1 - ok, I double checked and it seems alright.

2 - What would be the correct way of handling the href and src when using in a subdirectory?

What is the reason to use a new root server location? We use other services, e.g. nextcloud in a subdirectory, too. And it is a supported type of installation.

Otherwise we would need a new subdomain, new certificates, ...

3 - Wouldn't that mean adding a plain text mysql database password to the nginx server config?

If I would like to stick with the .env and env.local would I delete them after compiling .env.local.php?

4 - perfect, thank you ;)

5 - o.k., I will have a look at the failed2ban rule. Do you suggest to disable the login and only enable when configurations need to be done? If yes, how?

6 - This is the main issue/problem I'm having with this and a dealbreaker until it is fixed. Yes, macOS works. In iOS the server connects (and calendars work), but in the reminders app there is only the grey headline without an actual reminders list.
As stated above additional problem(s):
Adding a reminder list on iOS adds a local list entry to this server which does not get synced and not created on the server.
Adding a reminder list on macOS adds a local list entry for a second and disappears. It also doesn't get synced or created on the server. The one created on the server works, only the one created on macOS does not work.

7 - Yes, we stick with php 7.3 for now, thanks.

from davis.

PS for point 6, after enabling TODOs on the calendar, it works for me on macOS (I can't test on iOS though).

Do you mean in the davis interface? I did that and it leads to these results:

macOS (default apple onboard tools)
Calendar - working
Contacts - working (only with the default address book of the user)
Reminders - working (when reminders list is added in davis, adding new reminder lists from macOS does not work)

iOS latest version (default apple onboard tools)
Calendar - working
Contacts - working (only with the default address book of the user)
Reminders - NOT WORKING (reminders list not appearing in iOS at all)

from davis.

Thanks for all your help. Again learning a lot and I hope the iOS error can be fixed and I can edge out the last few installation issues to get this into production use. Would be really nice ;)

from davis.

2 - What would be the correct way of handling the href and src when using in a subdirectory?

A subdomain and a certificate are commodities nowadays (and free), you should use it. If you don't want, well, you need to change your nginx configuration, but not touch the application files.

3 - Wouldn't that mean adding a plain text mysql database password to the nginx server config?

Exactly. Either your nginx config or your app needs to know the password, one way or another...

If I would like to stick with the .env and env.local would I delete them after compiling .env.local.php?

I don't know, check the dotenv documentation. I don't use dotenv in production for obvious perf reasons.

5 - I don't understand why you would need to disable the login, just choose a secure password. Your installation in subdirectories is already less secure than a public, password-protected login page.

6 - Check the latest 1.4.3 release to see if it works better. Might help.

from davis.

2 - ok, thanks, I will work on that.

3 - ok, I will have a look at both solutions.

5 - ok, I will work on the fail2ban rule.

6 - Thanks for the latest update. On macOS I now actually can add new reminder lists and they appear in davis.
Unfortunately the iOS issue still persists. We tried on two different phones with the latest iOS 13.2.3. Still no reminder lists in the reminders app.
We also tried on an "older" phone and iOS 12.4.3 seems to work fine.
So it seems either to be an issue of davis or sabre/dav with iOS13 and reminders or an apple bug.

from davis.

"Starting with iOS 13 and macOS Catalina, the new Reminders app no longer supports CalDAV."

in https://support.busymac.com/blog/112990

from davis.

So you're out of luck

from davis.

As an addition I don't know if the functionality for local network ssl connections for reminders can be restored from saber/dav or only from apple.

from davis.

Today I restructured my nginx installation and php-fpm pools to install davis to a separate server with separate server root and certificate (as you suggested).

When trying to install the latest version I got the following error:

Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for ocramius/proxy-manager 2.5.0 -> satisfiable by ocramius/proxy-manager[2.5.0].
    - ocramius/proxy-manager 2.5.0 requires php ^7.4.0 -> your PHP version (7.3.12) does not satisfy that requirement.
  Problem 2
    - ocramius/proxy-manager 2.5.0 requires php ^7.4.0 -> your PHP version (7.3.12) does not satisfy that requirement.
    - doctrine/migrations 2.2.0 requires ocramius/proxy-manager ^2.0.2 -> satisfiable by ocramius/proxy-manager[2.5.0].
    - Installation request for doctrine/migrations 2.2.0 -> satisfiable by doctrine/migrations[2.2.0].

As I understand correctly the needed packages are no longer compatible with php 7.3.

Will you specify different dependency lists for the different php versions or do I have to upgrade to 7.4? This would most probably break other stuff like nextcloud.

And one more question. When the database is already set up and I want to upgrade to a newer version of davis is it recommended to run the database migration on each update?

Thanks

EDIT: I went back in the releases and the latest one installing is 1.4.3.

from davis.

When configuring nginx I stumbled across a configuration problem with the Content-Security-Policy.

I had to loosen it in regards to script-src and set it to

script-src 'self' https://code.jquery.com/ 'unsafe-inline'

all in all the most restricted I got davis to work with is this config:

add_header Content-Security-Policy "default-src 'none'; script-src 'self' https://code.jquery.com/ 'unsafe-inline'; style-src 'self'; frame-src 'self'; img-src 'self' data:; font-src 'self' data:; media-src 'self'; connect-src 'self'; object-src 'self';" always;

does this look ok to you?

EDIT

I also had to loosen style-src to
style-src 'self' 'unsafe-inline'

all in all that is my working Content-Security-Policy:

add_header Content-Security-Policy "default-src 'none'; script-src 'self' https://code.jquery.com/ 'unsafe-inline'; style-src 'self' 'unsafe-inline'; frame-src 'self'; img-src 'self' data:; font-src 'self' data:; media-src 'self'; connect-src 'self'; object-src 'self';" always;

from davis.

Hello,

There is no difference between 1.4.3 and latest regarding the dependencies (the composer.json file is exactly identical) so both should break, in fact. This is due to the latest release in ocramius/proxy-manager and the fact that the lock file was created in PHP 7.4.

I've set the platform to PHP 7.3.12 for now in my latest commit (master), can you test that ?

is it recommended to run the database migration on each update?

Yes, exactly. It won't break anything, just modify some table columns / indexes if needed.

Regarding Content-Security-Policy, I don't have that in my nginx config, and I never put it, but it seems reasonable.

from davis.

Hey,

thanks, the current git master installs again with php 7.3.

Also thanks for the feedback regarding the updates and the Content-Security-Policy. I will implement both in the installation/update.

from davis.

I just tested the latest git version a bit. Thanks for implementing the separation of calendar / reminders / notes. When only reminders (todos) is selected, the entry does no longer appear in the calendars in macOS any more. Nice ;) Thanks, great work ;)

After finishing nginx and php-fpm yesterday I will hopefully figure the rest of the installation (dotenv and fail2ban) out soon and we will do more intense testing next week...

Until then I have two more questions:
1 Is it necessary to make E-Mail an obligatory field when creating a user?
2 What is the purpose of the APP_SECRET and will a random string, let's say 12 to 16 digits be ok?

from davis.

• I'm not sure about email, but I guess it's in the Caldav spec. If you find some place where it says that it's not compulsory, I'd be happy to change it
• For the secret : https://stackoverflow.com/a/34887007/1741150

from davis.

Sorry, I don't know about the Caldav specs regarding email.

Thanks for the link explaining the secret.

from davis.

Today I took some time for the rest of the installation.

I managed to get fail2ban working for the website admin interface and for calder/carddav connections.

Regarding the .env.local file I have to more questions:

1 MAILER_URL

Do I understand correctly that the MAILER_URL is used for sending invitations from the calendars? Or is it used for anything else?

I tested my MAILER_URL config using bin/console swiftmailer:email:send and it refuses to send an email with the specified address. I found it that it is due to having special characters in the email password. According to the documentation it has to use the urlencode function to work with special characters. It would be nice if you could implement that. If not I have to use another email address with a password that does not contain special characters. Sending test emails with bin/console swiftmailer:email:send works if the password does not contain special characters.

2 tmp and public
I do not understand the purpose of the /tmp and /public entry in the .env file. I got that it is only used if using webdav. If the absolute path to the installation is /path/to/davis/public and the root of the webserver is also /path/to/davis/public where should they be located and what ownership and permissions would be required?
The .env file only says it has to be writable for the webserver.
And is the entry in the .env file relative to the webserver root path or an absolute filesystem path?

After figuring these two things out we will start more intense testing next week.

Thanks again for this very nice project, all your patience and your help.

Have a nice evening ;)

from davis.

1. The MAILER_URL purpose is specifically and only to send invitations. You are right (as is the documentation) for the encoding. But I have nothing to implement, just urlencode your password before writing it in the .env file. Ex : if your password is p@ssword, just put p%40ssword in the file (You can use https://www.urlencoder.org/ for instance, to do it quickly)

2. The paths can be relative or absolute. When they are relative, they are relative to /path/to/davis, but that might depend on your webserver config. That's why I recommend using absolute paths always, so it's no surprise. You can omit these variables if you don't use webdav though.

Happy to know that Davis is useful for you.

from davis.

1. Thanks, I got it to working converting the password with
   php -r "echo urlencode('$SMTP_EMAIL_PASSWORD');"

The test email and the invitations work. How do I change the language and the content of the invitation template?

2. I would like to try the webdav feature at least. So do I understand correctly that if the root of the davis webserver is /path/to/davis/public, I would have to add

/path/to/davis/public/tmp
/path/to/davis/public/public

and specify these exact two absolute paths in the .env.local file?

3. Is it possible to give each calendar entry a notification by default, let's say 30 minutes before the event? I can add that in macOS calendar in Calendar - Preferences - Notifications. But can that be set in davis so that every client connecting to davis pulls the setting from there?

Thanks

from davis.

1. Create a new translation, see https://symfony.com/doc/current/translation.html
2. You can specify the path you want, as long as they are absolute, you're quite safe
3. Use your client to set this preference. I think it's automatically saved in sabre/dav (depending on the client I guess)

from davis.

Thanks

1. In which local file of the installation can content of the invitation email be changed?

from davis.

If you mean to change the content, and not only texts, you can fork this repository and change the email template in templates/mails

from davis.

If you mean to change the content, and not only texts, you can fork this repository and change the email template in templates/mails

ok, thanks.

When testing I saw that the email address configured in URL_MAILER in the dotev file was used in the from field for the invitation email.

I don't know how this would be for you or other users that have multiple users and calendars defined. I think that would confuse the recipient of the invitation.

When testing sending emails by using bin/console swiftmailer:email:send the from can be defined with any valid email address and it is shown as the one specified there. So for our usage it would be best and most flexible solution if there were two fields in the admin interface for each calendar (not for the user) to specify a sender name (independent from the calendar name) and a sender email address that is being used by the from field. If the fields are left empty the default values from now could be used or it could be necessary fields.

All emails could still be send by the URL_MAILER.

It would be very nice if you could implement that. That would make it very flexible, comfortable and not confusing for the recipient.

Thanks a lot for looking into it. Have a great weekend ;)

from davis.

If you look closely in DavisIMipPlugin.php, the From field is the INVITE_FROM_ADDRESS (generally, a no-reply address), and the Reply-to is the actual name and email address of the originator of the mail (that is, the organizer).

This is expected and reasonable for me, and clearly not confusing for the user (invitations are not sent by a user, but by an automated process, hence the "no-reply" From field, but if you want to reply, you can do so and it's likely that it's the organizer of the event you want to reach).

It's exactly the behaviour of Baïkal if I remember well, and of other stacks.
If you want to change it feel free (you can fork), but I won't touch this for now.

from davis.

If you look closely in DavisIMipPlugin.php, the From field is the INVITE_FROM_ADDRESS (generally, a no-reply address), and the Reply-to is the actual name and email address of the originator of the mail (that is, the organizer).

ok, I get how this is supposed to work.

As in our workgroup we are a few people that all have read/write access to all calendars. I wanted to add all calendars to one user and specify a reply address per calendar. I do not have the programming skills to add that feature and want to stick to the original development.

If this is not working every user needs his own account (and therefore its own reply-to address) and has to add delegates to all other users. This gets pretty ugly in the calendar. I attached to screenshots to show the difference.

Do you see any other way to do that? Is there a way to show the delegates directly without all the subgroups?

After testing my wish would be to only add a "reply to" field to each calendar and use it if filled out, if empty stick with the default. If you do not like that I guess I have to stick with the delegates, right?

When testing delegates I stumbled across two other things.
a. The same delegates are always activated for all calendars of the user. Is that correct behavior? I would assume to only share the calendars I add the delegates to.
b. The reminders apps in macOS and iOS do not have an option to activate delegates (calendar does). Does this mean there have to be separate user accounts for the reminders to share them in different combinations via different users?

Sorry about all the questions. Hope this helps for future questions from other users. And I hope you can use a part of my script and documentation for the readme (I will give you everything when all is set up and working). I really like davis and we will most likely use it in production in early 2020. I have tried a few caldav/carddav servers now and davis seems to be the best in many aspects currently out there ;)

from davis.

Well, indeed every user needs its own account, this is how the protocol is designed. In your case, it's just your client that displays it in a way you don't like, unfortunately. Hence I won't implement any specifics deviating from the standard — you might want to check other clients (like Busycal for instance) that might display delegation the way you think it should be.

a. Yes. Delegation is at the principals level, not the calendar level
b. I guess so yes, but I'm not familiar with Reminders nor how it should work

Thanks for your kind words ! ;)

from davis.

Hey,

thanks again for all your help, we managed to structure, install and configure macOS and iOS clients successfully and all (short) tests with only two persons went well.

As promised I sent you an email with the install script for davis and fail2ban, the nginx configuration, the php configuration and the documentation.

It would be nice if you could have a look at it and see if you find anything that is just wrong or insecure and give me a short feedback.

Otherwise it hopefully helps you or with other users that have questions in the future.

Have a nice weekend ;)

Thanks a lot, kind regards

from davis.

Hey,

our tests are going on for more than two weeks now and everything seems to work fine with iOS and macOS clients using calendar, contacts and reminders.

I've seen that sabre/dav merged your pull requests and released version 4.0.3.

After nextcloud 18 was released we also moved our complete php installation to version 7.4.

Would be nice to see an update compatible to php 7.4 including the latest dav version.

Again thanks for all the help and recommendations. We will stick with davis for daily production usage and hope it will be maintained for a long time ;)

Hopefully a lot of people find and use this really nice work. Perhaps it can somehow be integrated in wikis and/or distributions. I know that you do not want to release a "package", but if someone is looking for a dav server tool the first view perhaps goes to the packages of the distros or to their wikis.

I only found this by mere chance when reading something on the baikal GitHub site.

Thanks, kind regards

from davis.

And I sent you version 3 of the install script via email. Please delete all old versions and only keep this version. All other files from the first bunch of files I sent you are still up-to-date ;)

from davis.

Thanks.

I ran my script to update and it worked perfectly ;)

It seems there are two changes necessary to the symfony.lock.

During composer install there is the following output:

Package zendframework/zend-eventmanager is abandoned, you should avoid using it. Use laminas/laminas-eventmanager instead.
Package zendframework/zend-code is abandoned, you should avoid using it. Use laminas/laminas-code instead.

Thanks in advance for looking into it.

Have a nice weekend ;)

from davis.