Account / Address book integration about conversations HOT 48 CLOSED

ok... this is going to be a rather difficult topic. Let me start by admitting that I know very little about Contacts Provider and Accounts. I glanced over the official documentation once or twice but never wrote actual code or tried to get a deeper understanding of the workings.

Just let me write down some random thoughts on that topic. Don't regard them as final. I'm willing to have an open discussion about any of this.

Everything needed to sync contacts and update the presence in the phone book

I know the address bock has the ability to display online statuses but I don't know any app that uses them. (GTalk did this at some point I believe??! - but hangouts doesn't) I don't know if Conversations should be the first (and only) app to do that.
Furthermore as I mentioned before on different occasions I'm not a fan of presences in general (thus I'm trying to hide them in Conversations or at least don't make them too obvious) Presences - especially with stream management are lying anyway and it only confuses the user if the other party is displayed as online but something like OTR doesnt work. Furthermore most of my contacts are online 100% of the time since the don't like the privacy implications of having something like auto idle. (Plus how do you do auto idle on mobile phone? be online every time the app is in the foreground like whatsapp does? This would mean a huge impact on privacy. I know for a fact that a lot of whatsapp user abuse the "last time online" thing as a "he has read my message why isn't he answering")

What users (jids) should be moved to the address book?

This is in fact the big question to which I don't have an answer yet. I know a lot of jabber users which huge rosters full of unimportant users. We certainly don't want our address book full of boring contacts which just contain a jid (and no phone number) and thus are basically worthless.

Should we auto-merge with existing users?

Definitively. This I believe is the only reason to provide a contact provider any way. But I never understood on what grounds the address book is merging contacts. This I will have to find out before I can say more about this.

Should/Can we keep groups alive?

I rather not. I believe groups are a poor excuse for a lack of a good search functionality which both the address book and Conversations as it is right now provide by themselves.

Should we integrate with the account sync?

I don't even know what this is :-)

from conversations.

Just let me write down some random thoughts on that topic. Don't regard them as final. I'm willing to have an open discussion about any of this.

This is why I raised this issue in the first place. Start a discussion about what would be useful.

I know the address bock has the ability to display online statuses but I don't know any app that uses them. (GTalk did this at some point I believe??! but hangouts doesn't) I don't know if Conversations should be the first (and only) app to do that.

I'm also not sure if it's still available. Anyway, I can live without it.

Furthermore as I mentioned before on different occasions I'm not a fan of presences in general (thus I'm trying to hide them in Conversations or at least don't make them too obvious) Presences - especially with stream management are lying anyway and it only confuses the user if the other party is displayed as online but something like OTR doesnt work. Furthermore most of my contacts are online 100% of the time since the don't like the privacy implications of having something like auto idle.

Ok, I've quite the opposite of a roaster. I've ~3x more users than I see online at any given time. And those who are online are usually somehow important to me.

How about this: Let's add a "last seen" and "last talked", where "last seen" is the last time we saw some traffic from that person and last talked is the time of the last chat.

This is, IMHO, the better metric than presence because "seen 5min ago" means "He'll most likely see what you write to him" whereas "Last seen 2 weeks ago" means "try to reach him by other means, he might have abandoned this account".

This would also solve the question what contacts should be synced: The jid has to have some traffic (recent last seen) AND I have to be able to fetch user information.

I could, alternately, switch to a "sync only contacts that do have an active chat" if the list of seen users is too large (100?).

Should we auto-merge with existing users?

Definitively. This I believe is the only reason to provide a contact provider any way. But I never understood on what grounds the address book is merging contacts. This I will have to find out before I can say more about this.

This one is rather easy. You have Contacts (e.g. iNPUTmice) that consist of RawContacts (e.g. via synced via github account) which have metadata (e.g. github account url).

It's basically a matter of finding the right Contact (based on Metadata) for your RawContact (or creating a new Contact).

Should/Can we keep groups alive?

I rather not. I believe groups are a poor excuse for a lack of a good search functionality which both the address book and Conversations as it is right now provide by themselves.

From a Conversations POV definetly right. Now here is how groups integrate into the contacts list: Open your Contacts app, choose the 3 dots and open Contacts to display, choose personalize and pick the groups you'd like to view. This has been available since the beginning of the public Android Contacts API, it's hidden in different places depending on your UI skin, but it's available.

It would enable you to say "Nah, I don't want to see my coworkers on this phone". Anyway, I'd happy to skip this and go for a "only sync important/alive contacts".

Should we integrate with the account sync?

I don't even know what this is :-)

This is the Circle-like 2-arrow symbol you'll see e.g. on the power controls bar. Android will notify all accounts roughly once a day that they can/should now sync their data (e.g. contacts). The fun is that all Account apps do this. Facebook, GMail/G+, Twitter. You won't drain much battery because all apps hog the CPU/network during that event anyway. It's thus a great opportunity to update e.g. the roaster or user information.

from conversations.

IMHO this should be optional. I for example don't want any of my contacts in conversations to be in my android address book and vice versa. This also would raise some privacy issues. We all know that there are apps that access the address book and even upload the contacts to some servers. This means that those apps will also see my conversations contacts if they are integrated to the android address book. This is not good for many reasons. The same goes for account integration.

from conversations.

That argument is quite bogus IMHO.

Yes, there were malicious apps that uploaded contacts. BUT there were also malicious apps that used root exploits. We don't protect ourselfs against root exploits. Which means we aren't protected against malicious apps right now.

That said most uploaders tried to fetch some known-good metadata, specifically e-mail, phone number and name. (As name + number is very likely on a phone). This means that for most uploaders you would not even lose any information. Why? Because you have to know what the metadata means to be usefull. See https://github.com/rtreffer/AsmackService/blob/master/res/xml/contacts.xml on how I've defined a XMPP metadata type. You'll simply get garbage if you try to steal that.

Furthermore, if you don't want to share your information in the first place don't put it into your XMPP vCard.

That said it might be a candidate for some "expert" settings.

PS: This issue describes Conversation -> Address Book integration. We could do the other direction, Address Book -> Conversation, too.

from conversations.

This is clearly not only a problem of malicious apps. The majority of people do use apps like WhatsApp or Facebook. And those already upload your contacts.

And you are not making a good point there. You totally rely on security by obscurity. Which clearly is a bad strategy. Actually everybody who knows how to find the source code of conversations could easily find out how to make use of the information.

You also say "for most uploaders you would not even lose any information". Sorry, but security is not about being secure from 'most' uploaders or 'most' attacks and relying on the fact that people don't care anyway.

from conversations.

Ok, I've quite the opposite of a roaster. I've ~3x more users than I see online at any given time. And those who are online are usually somehow important to me.

Oh yeah me too. I meant to say that most of the users who are online are "online/available" instead of away, na, or dnd. Of course I have a lot of users in my roster who are never online and probably don't use their account any longer. This is why we should think about whether or not to expose the entire roster over a contact provider.

How about this: Let's add a "last seen" and "last talked", where "last seen" is the last time we saw some traffic from that person and last talked is the time of the last chat.
This is, IMHO, the better metric than presence because "seen 5min ago" means "He'll most likely see what you write to him" whereas "Last seen 2 weeks ago" means "try to reach him by other means, he might have abandoned this account".

This sounds great. I was thinking about doing something like this myself before but never came around to do that.

Maybe this could be one of the first features to implement since it is relatively easy and we could get rid of the presence display entirely. (Just the display within the Contacts Details - not the internal representation of presences - we need those to make something like OTR work)

Android will notify all accounts roughly once a day that they can/should now sync their data (e.g. contacts).

If something like this exists we should be using it.

....Groups....

If we decide to expose the roster groups over the contact provider then we will have to display them in Conversations as well. Anything else would confuse the user. And thus we would have to find a UI/UX concept for that. I am definitely against the "classic roster" view with a tree like representation. This will give groups to much of an importance. (If you catch my drift. Not sure how to express this properly).
Since a roster entry can have multiple groups we could maybe think of groups as more like 'Tags' maybe. Then each contact in the 'Contacts Activity' could have a comma separated list of Tags/Groups next to them/below them. This won't change the current behaviour (as in we won't have to do a tree like representation) and we could simply add a 'filter by tag/group' menu item.

The majority of people do use apps like WhatsApp or Facebook. And those already upload your contacts.

Sorry but this is not our fault. If you don't trust an app with your data don't f***ing install the app or at the very least use something like privacy guard.

That being said the entire contacts provider thing should be optional for different reason - mostly people who don't like to have "garbage contacts' in the address book and/or use the address book only for people with phone numbers (as a phone book if you will)

Another point - not specific to anything - but I still wanna mention it. One of the design goals of Conversations is to require as little permissions as possible (ie everything that could be done with an intent should be done with an intent)

from conversations.

Sorry but this is not our fault. If you don't trust an app with your data don't f***ing install the app or at the very least use something like privacy guard.

I see so Conversations is an app for experienced users only and doesn't make security available to everybody? That really is a pity.

from conversations.

Ok, so how would you protect yourself against copy pasting a conversation to pastebin? This is a true security issue!

Knowingly uploading data is not a security issue. FB and WhatsApp ask if they should upload your contacts. If you say yes uploading it is not a security issue.

from conversations.

@iNPUTmice

Yes, I'd keep the 2 things (contacts in address book and conversation db) separated but in sync. This way we can decide what should end up where.
I'd say it would be nice to have at least the active conversations accessible from the address book. Anyway, I can live with any other default too. Let's see how the different options work out and decide based on that? Changing the defaults on what to sync should be easy.

Permission wise... Adding accounts is a rather heavy permission. But we could also do stuff like fetching a vCard from the address book if mail and jid match (I usually tried to do this for corp installs, jid == mail address).

Regarding presence: good to hear that you had similar ideas. That means it can't be too much off. We should do s.th. like "relatime", only updating the information if the delta is > 1minute. We'll otherwise put too much (useless) load on the sqlite.

from conversations.

Ok, so how would you protect yourself against copy pasting a conversation to pastebin? This is a true security issue!
Knowingly uploading data is not a security issue. FB and WhatsApp ask if they should upload your contacts. If you say yes uploading it is not a security issue.

I got it Conversations is not going to be secure for unexperienced people.

We could do the other direction, Address Book -> Conversation, too.

This is not good either. I don't necessarily want all my android contacts to be in conversations. This is ridiculous to have.

from conversations.

Let's see how the different options work out and decide based on that?

Yes. This i really something we should experience before we decide on sane defaults. I think what this will boil down to how good the (auto?) merging of our raw contacts and the existing contacts will work. Because what I would really like to avoid is that contacts of mine appear twice (once as a contact with a phone number and once as a contact with a jabber id) - But i think this is all something we can decide upon once we have some code to toy with.

from conversations.

@geileszeuch are you basically saying that users should be prevented from adding contacts to the phone book at all because there are apps out there that "steal" those information? So based on your argumentation the "People" App is unsafe for the inexperienced user as well because it allows you to create contacts which then can be read by the facebook app? So you shouldn't put money in your bank account because there is malware out there which takes screenshots while you are on your online banking website?

from conversations.

@geileszeuch Furthermore I already said that the contact provider thing should be optional. So you basically already got what you wanted.

from conversations.

@iNPUTmice I am saying that if people want to use Conversations as a secure place without having to care about all the other apps they have installed, then Conversations shouldn't have access to the phone book in either way.

@geileszeuch are you basically saying that users should be prevented from adding contacts to the phone book at all because there are apps out there that "steal" those information?

In my eyes the whole android system did something wrong in this respect. Apps shouldn't be able to access the whole address book as easily as they are now, and that's why I believe Conversations also should not do that.

from conversations.

In my eyes the whole android system did something wrong in this respect. Apps shouldn't be able to access the whole address book as easily as they are now, and that's why I believe Conversations also should not do that.

This app should be worse because I dislike features of the platform it's running on. That was quite obvious from the last posts, but thank you for pointing it out, again. YMMD.

from conversations.

It's good to know that you @rtreffer are pretty open for suggestions and discussions.

from conversations.

@geileszeuch I see your points. Please have a look at issue #47 - I however see this more like a don't require the user to trust us then a fixing the android ecosystem.

from conversations.

Sorry but this is not our fault. If you don't trust an app with your data don't f***ing install the app or at the very least use something like privacy guard.

I know we are done with this topic, but I wanted add one last thing. This argument doesn't solve the problem. The problem is it is not up to you. As soon as someone adds you in his address book and uses apps like Whatsapp or Facebook you are basically screwed and there is nothing you can do about it.

from conversations.

Sorry but this is not our fault. If you don't trust an app with your data don't f***ing install the app or at the very least use something like privacy guard.

I know we are done with this topic, but I wanted add one last thing. This argument doesn't solve the problem. The problem is it is not up to you. As soon as someone adds you in his address book and uses apps like Whatsapp or Facebook you are basically screwed and there is nothing you can do about it.

I really really hope for everyone else that you've stopped using E-Mail altogether. Because, you know, Facebook, LinkedIn and just about every CRM allows you to scan E-Mail for contacts and you're screwed if anyone uses it. True story. So please stop using Mail.

Oh and for everyone else health stop giving your phone number away. Adding it to a contact will most likely sync it to google.

While we are at it: Do you even notice how screwed your logic is? I dislike Facebook and WhatsApp. Because of that open source apps should not be build to be viable replacements. Thjis is what you're advocating.

Tell me when that worked out.

from conversations.

Now, you made perfectly clear that you don't care about security and your data at all. You are willing to give those up at any cost to make the app just a little bit more comfortable. If we follow your logic we should stop using encryption altogether, because its complicated, uses our CPU unnecessary and therefore eats battery life, you have to verify people, OTR is not async, all of these arguments make the app not a viable replacement. And in addition everybody could just upload the conversation in pastebin or make a screenshot. So let's just forget about encryption at all its useless. This is what you're advocating. Tell me when THAT worked out in a project which focuses on security foremost, which Conversations does.

If you don't like giving up just a little bit of comfort for security and privacy and you don't care about those things at all then I believe you're wrong here, because, I don't know if you even noticed by now, Conversations focuses on privacy and security.

from conversations.

@geileszeuch There is a difference between "I give person X data and it abuses it" and "Data is lost due to mussing security". Case 1 is not a security issue. Yet you are coming back with it all the time. I can publish any data I get from you. This is NOT a security issue. This is why you could legally go after me in that case.

Given your logic E-Mail should never be distributed between server because it might get abused on the other end. That's the very side effect of giving it away.

Btw: do you have a single google talk user in your roster? In that case you are screwed as described.

from conversations.

Let me quote from my own mission statement

Be as beautiful and easy to use as possible without sacrificing security or privacy

Conversations will always be about protecting your privacy. But it only can account for what happens on your phone. If you don't want Whatsapp or Facebook to leak data from your phone either don't install those apps or use privacy guard.

If you don't want third parties to upload your data to Facebook or Google please talk to them before giving away your phone number or Jabber ID. Tell them straight away that you know they use Facebook and you know that Facebook 'steals' your data and they please shouldn't do that or else you don't give them your phone number.

Besides. We are already planning on making the contact provider thing optional in one way or another! So could we please stop talking about that?

from conversations.

@rtreffer That's why you use encryption and things like that, to build a world in which you don't have to trust the servers.

FYI I don't have any Google talk user in my roster and don't worry I do not use any CRM or any app that has more access to my data than I would like to. And for the sake of future questions, yes I only use open source software and yes I check the sources and built them myself.

from conversations.

@geileszeuch Come back if you have a paper on how encryption can help to protect data from be abused by the recipient.

from conversations.

YMMD. You clearly have no clue about encryption do you? Ever heard of deniability?
Here is the paper you wanted https://otr.cypherpunks.ca/otr-wpes.pdf
Please read it, before talking any more nonsense.

from conversations.

@geileszeuch you have no clue where encryption ends. You send me an OTR message with credentials. I upload it to pastebin because I'm not trustworthy.

That's the problem we're talking about. You give me data (your jid, your vCard). I abuse it. Come back if you can provide how I could be blocked from abusing it that way. Hint: it's not encryption.

from conversations.

Like pointed out thousand times already I just don't give you my data.

from conversations.

First of all, can we please stop using such inflammatory language ("you don't care about security at all", "you have no clue", "talking nonsense", all the sarcasm) and stay civil? This really only serves to further solidify the fronts and prevents calm, reasoned discussion.

There really seems to be a disconnect between different notions of privacy/security in this thread. The issue of personal information of some sort being "leaked" by third parties (e.g. the upload to Facebook/Whatsapp/...) scenario is of course a very real and important problem. It is however not a technological one. At a basic level, the issue that is being discussed here is not whether to trust the app. The question of how productive it is to question the trustworthiness of the app has been very well covered in the discussion of #47. Rather what we are talking about is whether we can trust other users (e.g. trust them not to upload our info to some third party against our will).
Thus we have to now consider malicious contacts, making this a fundamentally unsolvable problem. No amount of encryption can prevent another contact to share the info they have on you with anyone they please, as all of this has to be unpacked at some point. This is just an information theoretical reality.
The only real solution for this, as pointed out previously, is to simply not share this info (i.e. JID/vCard/...) with such untrusted third parties. Any attempt to solve this through use of technology is doomed to fail. For example, it is reasonable to assume that - in absence of automatic contact merging - users may add the JIDs to their address book manually, and there is simply no way to prevent this or any similar attack vector on our data. As such, we are not talking about how to behave in a secure way, but rather how to do damage control.
Furthermore, OTR is completely unrelated to the issue at hand, as it is pretty much only concerned with transport security.

I certainly see the argument that automatic address book merging removes some of the required maliciousness on the side of the other contact, and I am all for sane defaults. The user should not have to be bothered or have to inform themselves on the particulars of privacy mechanisms in order to be secure. This sense of simplicity is very much in line with the philosophy behind Conversations as well.
However, (perceived) privacy/security is neither the only nor the single most important criterion to evaluate. The goal should not be absolute, ultimate security at any cost, as this pretty much always heavily impacts usability. Rather, we should aim to strike a balance between secure defaults and a pleasant user experience. Where this balance lies is really a somewhat subjective decision.

There really is no point in beating this dead horse any further as this is ultimately a judgement call of the head developer. IMO, the suggested solution of making merging opt-out is really the best we can do here and most of you seem to agree. This seems like another good candidate for the "expert mode" settings menu that has come up in a few other threads.

from conversations.

@strb and every one else. You are just wasting your time with geileszeuch. He is stupid or troll, or both. Just ban him :)

from conversations.

• can "last seen / last talked" be stored serverside?
• storing stuff in android contacts: This may be bad, as "malicious" apps can read contact by permission, but conversations data only by exploits. There IS some kind of security on android (even better than an average pc), so let the user profit from it by not defaulting to store everything in contacts. (But i do not want to join your flamewar)

from conversations.

I'd like to see my conversation contacts in my address book, too and if conversations account would be integrated in android accounts simply because this would be more androidish. And I like conversations because of being androidish ;)

from conversations.

Is there in the current implementation now a function to switch sync off? Android 6 keeps asking me for access to the address book, even when i denied it (i'd like to avoid syncing contacts and JIDs, mostly for not mixing this lists together, as XMPP does a decent job at keeping a roster).

from conversations.

from conversations.

And the Conversations message "please accept the next question" before?

from conversations.

from conversations.

Ah, need to re-test. The last times i just declined without setting the checkmark and thought conversations will have some checkmark not to ask by itself. If setting "do not ask again" works for all messages, it's fine :).

from conversations.

This is an excellent app. I am trying to integrate with my address book. That's what I want and I need to do. I have seen some part of code that enables this but I am yet to find an interface to activate it.

public void onPhoneContactsLoaded(final List<Bundle> phoneContacts) { if (mPhoneContactMergerThread != null) { mPhoneContactMergerThread.interrupt(); } mPhoneContactMergerThread = new Thread(new Runnable() { @Override public void run() { Log.d(Config.LOGTAG, "start merging phone contacts with roster"); for (Account account : accounts) { List<Contact> withSystemAccounts = account.getRoster().getWithSystemAccounts(); for (Bundle phoneContact : phoneContacts) { if (Thread.interrupted()) { Log.d(Config.LOGTAG, "interrupted merging phone contacts"); return; } Jid jid; try { jid = Jid.fromString(phoneContact.getString("jid")); } catch (final InvalidJidException e) { continue; } final Contact contact = account.getRoster().getContact(jid); String systemAccount = phoneContact.getInt("phoneid") + "#" + phoneContact.getString("lookup"); contact.setSystemAccount(systemAccount); if (contact.setPhotoUri(phoneContact.getString("photouri"))) { getAvatarService().clear(contact); } contact.setSystemName(phoneContact.getString("displayname")); withSystemAccounts.remove(contact); } for (Contact contact : withSystemAccounts) { contact.setSystemAccount(null); contact.setSystemName(null); if (contact.setPhotoUri(null)) { getAvatarService().clear(contact); } } } Log.d(Config.LOGTAG, "finished merging phone contacts"); updateAccountUi(); } }); mPhoneContactMergerThread.start(); }

from conversations.

The code @bktowett has pasted can be found here: https://github.com/siacs/Conversations/blob/11666195398b8c673b47863ab07744c8926b54d0/src/main/java/eu/siacs/conversations/services/XmppConnectionService.java#L1229
(just as a side note)

from conversations.

I am using the addressbook integration. But I have contacts with more then one JID.
Here now the problem.
I add a contact with his skype jabber id (XMPP transport) to my roster and also with his real jabber-ID. Now the problem is, that I add this two JID to the contact of my android addressbook, but I am seeing only on entry in the contacts of conversations and not two entries.
In a other XMPP client application (pidgin), I see two entries on my roster.

from conversations.

@fanningert how are you able to do address book integration? I have not been able to figure out yet.

from conversations.

@bktowett Sorry for the late response. But I don't find a solution for this problem.

from conversations.

Just noticed there was no 'Accounts' settings menu integration. IMO this should be separate issue.

For the address book, I wouldn't like Conversations to modify it in any way (by default). It is perfect what it's doing at the moment — read-only integration, to bring names&avatars to roster.

from conversations.

Read only is not the problem. The problem is, what is when one contact has two JIDs. Where can I selected the right JID for a conversation.

from conversations.

@fanningert, what do you mean by 'contact has two JIDs'?

from conversations.

@muchweb I have some contacts with two XMPP-IDs ([email protected], [email protected]) in there contact details. (JID = Jabber Identifiers)

from conversations.

hello

using this integration if new contact added on XMPP server and that contact number already in my phone book than That contact is automatically seen in my conversations application contact?

thanks

from conversations.

@BavarvaSajan There is no number involved. That's not how it works...

from conversations.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from conversations.