adobe-apiplatform / user-sync.py Goto Github PK

If you use an extension to add a group to a user, then that group is not removed on later runs if the user is removed or if the extension changes to no longer add that group.

The other day, a customer asked if we have a Docker image for the User Sync Tool. I told him we don't that I know of, but I'd check with the dev team to see if it's something we'd look into supporting. I don't know how much demand we'll get for one, but it might be a nice-to-have feature sometime in the future.

Here's a quick proof-of-concept Dockerfile I threw together, in case anyone wants to mess with it:

FROM python:2.7
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential libssl-dev libffi-dev libsasl2-dev libldap2-dev
RUN pip install --upgrade pip
COPY misc/build/python-ldap-requirements.txt /opt/app/misc/build/python-ldap-requirements.txt
COPY misc/build/requirements.txt /opt/app/misc/build/requirements.txt
COPY . /opt/app
RUN pip install -r /opt/app/misc/build/python-ldap-requirements.txt
RUN pip install -r /opt/app/misc/build/requirements.txt
WORKDIR /opt/app
ENV PYTHONPATH="/opt/app"
ENTRYPOINT ["python", "user_sync/app.py"]

To build, just put the Dockerfile in the main user-sync dir and run this command:

docker build -t user-sync .

To run, mount a volume to the config dir, and another to the log dir. Params to the sync tool are supplied after the the image name. Example:

ubuntu@ubuntu-xenial:~$ docker run -v /home/ubuntu/user-sync.py/logs:/opt/app/logs -v /home/ubuntu/user-sync.py/config:/opt/app/config user-sync:latest --process-groups --users group ADOBE-SIGN,ADOBE-VIDEO,ADOBE-STOCK,ADOBE-ACROBAT-PRO-DC --config-filename /opt/app/config/user-sync-config.yml

note - In the config files, paths to log dir, additional config files, private key, etc should be based on volume path (e.g. /opt/app/logs).

eg, for the sample example.users.list:

2017-02-01 15:23:44 15140 DEBUG processor - Group work list: {None: {'[email protected]': set([]), '[email protected]': set([]), '[email protected]': set([])}}

Seems like the list was not loaded.

If you remove an active AD user from an AD group that is being mapped for user sync,
PLC does not get removed from the user.

However, if I disabled AD user that will remove PLC in the console just fine

Customer is getting the following error:

  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\app.py", line 270,
in main
    begin_work(config_loader)
  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\app.py", line 176,
in begin_work
    rule_processor.run(directory_groups, directory_connector, dashboard_connectors)
  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\rules.py", line 10
2, in run
    self.read_desired_user_groups(directory_groups, directory_connector)
  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\rules.py", line 16
2, in read_desired_user_groups
    user_key = RuleProcessor.get_directory_user_key(directory_user)
  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\rules.py", line 65
9, in get_directory_user_key
    return RuleProcessor.get_user_key(directory_user['username'], directory_user['domain'], directory_user['email'])
  File "c:\user-sync\.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\rules.py", line 67
8, in get_user_key
    return username + ',' + domain
TypeError: cannot concatenate 'str' and 'NoneType' objects
2017-03-13 14:13:38 2448 INFO main - ========== End Run (Total time: 0:00:16) ===================

I don't want to include the whole log for the run because it contains PII stuff that we don't want in Github.

It occurs after the directory user queries. I haven't had time to do much investigation, but I current guess is that we're getting a malformed "mail" attribute that is causing problems with this function in rules.py:

    @staticmethod
    def get_user_key(username, domain, email, isAdobeID=False):
        username = user_sync.helper.normalize_string(username)
        domain = user_sync.helper.normalize_string(domain)
        email = user_sync.helper.normalize_string(email)

        if isAdobeID:
            return '<AdobeID>' + email
        if (username == None):
            return email
        if (username.find('@') >= 0):
            return username
        return username + ',' + domain

The user_email_format is set to {mail}. So the username should always be an email, and the condition to concatenate w/ the comma and domain should never be hit. For some reason it is.

One thing to note is the customer is syncing one AD group containing two users. Both users have a valid email address. Are we pulling any data for users that don't belong to the sync group?

Is there anything we can do to anticipate this condition so the application doesn't crash?

I would like to see if I can reproduce this in my test environment. I'll post additional information as I work on that.

Seems like these messages should be ERROR rather than WARNING

2017-02-01 14:37:37 15876 WARNING dashboard.owning.action - Error requestID: action_1 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"
2017-02-01 14:37:37 15876 WARNING dashboard.owning.action - Error requestID: action_2 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"
2017-02-01 14:37:37 15876 WARNING dashboard.owning.action - Error requestID: action_3 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"

I get this warning all the time.

2017-03-21 17:57:37 2608 WARNING processor - Found user with no identity type, using enterpriseID: {'username': '[email protected]', 'source_attributes': {'username': None, 'domain': None, 'uid': None, 'identity_type': None, 'c': 'MX', 'sn': 'Doe 2', 'givenName': 'Sam', 'email': '[email protected]'}, 'domain': None, 'groups': [], 'firstname': 'Sam', 'country': 'MX', 'identity_type': None, 'lastname': 'Doe 2', 'email': '[email protected]'}
2017-03-21 17:57:37 2608 INFO processor - Adobe user not in input user set: enterpriseID,[email protected],
2017-03-21 17:57:37 2608 WARNING processor - Found user with no identity type, using enterpriseID: {'username': '[email protected]', 'source_attributes': {'username': None, 'domain': None, 'uid': None, 'identity_type': None, 'c': 'MX', 'sn': 'Doe 3', 'givenName': 'Sam', 'email': '[email protected]'}, 'domain': None, 'groups': [], 'firstname': 'Sam', 'country': 'MX', 'identity_type': None, 'lastname': 'Doe 3', 'email': '[email protected]'}

Is it possible can to add the following line to Iter_users() for directory_ldap.py
to suppress this message. I found that return user dict value of iter_users() method does not contain identitytype key which caused it to throw this warning all the time.

user['identitytype'] = options['user_identity_type']

Steps

1. Sync and add a new user from AD with a first name: "Name1"
2. Go to the dashboard and delete the user
3. In AD, change that user's name from "Name1" to "Name2"
4. Sync the user again with --update-user-info

Results

• After the second sync, it would display "Name1" as the first name on the dashboard. Only after you run --update-user-info again would the name be updated to "Name2".
• It seems like you cannot re-add a previously existing user and change their name at the same time.

user-sync isn't passing its logger to umapi_client, in order to make sure they are using the same settings. That should be done.

The docs should all consistently use the terms "product configuration" and "user group" where the specific type of group is being referred to.

Need a way to prevent server disk space from fill up with log files from UST.

add log rotation interval of daily, weekly, monthly, yearly, or by # of day would solve this issue.

When no default country code is specified in the config file, the script will crash with an unhandled exception when creating a user with a blank country code.

2017-02-13 11:13:19 26645 ERROR main - Unhandled exception
Traceback (most recent call last):
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/app.py", line 261, in main
    begin_work(config_loader)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/app.py", line 167, in begin_work
    rule_processor.run(directory_groups, directory_connector, dashboard_connectors)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/rules.py", line 93, in run
    self.process_dashboard_users(dashboard_connectors)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/rules.py", line 189, in process_dashboard_users
    self.add_dashboard_user(user_key, dashboard_connectors)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/rules.py", line 389, in add_dashboard_user
    dashboard_connectors.get_owning_connector().send_commands(commands, callback)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/connector/dashboard.py", line 120, in send_commands
    action = action_manager.create_action(commands)
  File "/home/adorton/.pex/install/user_sync-1.0-py2-none-any.whl.93526b61b26b84904dc82e42af6e4e60d8bd70d9/user_sync-1.0-py2-none-any.whl/user_sync/connector/dashboard.py", line 243, in create_action
    command_function(**command_param)
  File "/home/adorton/.pex/install/umapi_client-2.0.2-py2.py3-none-any.whl.78eb80b1f5c32494273374a205db3c5917772c76/umapi_client-2.0.2-py2.py3-none-any.whl/umapi_client/functional.py", line 175, in create
    raise ValueError("You must specify country when creating a Federated ID")
ValueError: You must specify country when creating a Federated ID
2017-02-13 11:13:19 26645 INFO main - ========== End Run (Total time: 0:00:11) ===================

It would probably be better to handle the error by writing an error message to the log and continuing the sync process. Crashing the script could prevent valid users from being created (e.g. if those users come after an invalid user in a list of users to create).

If a customer opts to not provide a default country code, logging instances of users with no country code will allow them to use their logs to identify user records that need to be fixed in their directory systems.

Customers with centralized user directories should be able to have users of different types in those directories (e.g., different domains some of which are enterpriseID and some of which are federatedID, or adobeID users mixed in with enterpriseID users).

To support this, we should get the identity type from a directory attribute, in the same way we use the type column in CSV input. There is already a customization mechanism for customers to pick the attribute (and format) used for email address info in the directory; that mechanism should be extended to support username, domain, and identity type.

This would override the user_identity_type option at both the connector and global levels.

This appears in the example config files but when uncommented result in the following message:

2017-02-01 14:43:53 14292 WARNING config - Found unused keys: ['ldap'] in: <user-sync-config.yml>.directory.connectors

I've copied the example files, and just edited dashboard-config.yml & user-sync-config.yml (to try and get something minimal working) - but I'm seeing an Unhandled Exception from the user-sync tool, and not sure how to proceed.

Here are my edits to the two yml files (actual values removed and replaced with X's):

$ diff dashboard-config.yml.ORIG dashboard-config.yml
9,13c9,13
<   org_id: "Org ID goes here"
<   api_key: "API key goes here"
<   client_secret: "Client secret goes here"
<   tech_acct: "Tech account ID goes here"
<   priv_key_path: "Path to private.key goes here"
---
>   org_id: "96AEE1CE543800880A4C98A5@AdobeOrg"
>   api_key: "XXXXXXXXXXXXXXXXXXXXXX"
>   client_secret: "XXXXXXX-XXXXX-XXXXX-XXXXX-XXXXXXXX"
>   tech_acct: "[email protected]"
>   priv_key_path: "/home/p/prynhart/private.key"

$ diff user-sync-config.yml.ORIG user-sync-config.yml
12c12
<   # owning: example.dashboard-config.yml
>   owning: dashboard-config.yml
<   # user_identity_type: enterpriseID
>   user_identity_type: federatedID
>   console_log_level: debug

I thought that this would be enough to establish a secure connection - not sure what to make of the crash. The actual traceback that I'm getting is pasted inline below. Any hints would be appreciated.

Thanks,

Patrick

$ ./user-sync
2017-03-09 10:20:59 30841 INFO config - Using main config file: user-sync-config.yml
2017-03-09 10:20:59 30841 INFO main - ========== Start Run =======================================
2017-03-09 10:20:59 30841 INFO dashboard.owning - Creating connection for org id: "96AEE1CE543800880A4C98A5@AdobeOrg" using private key file: "/home/p/prynhart/private.key"
2017-03-09 10:20:59 30841 ERROR main - Unhandled exception
Traceback (most recent call last):
  File "/home/p/prynhart/.pex/install/user_sync-1.1.1-py2-none-any.whl.21a3a678dccecc3c248aaab94e34af462a77008d/user_sync-1.1.1-py2-none-any.whl/user_sync/app.py", line 270, in main
    begin_work(config_loader)
  File "/home/p/prynhart/.pex/install/user_sync-1.1.1-py2-none-any.whl.21a3a678dccecc3c248aaab94e34af462a77008d/user_sync-1.1.1-py2-none-any.whl/user_sync/app.py", line 166, in begin_work
    dashboard_owning_connector = user_sync.connector.dashboard.DashboardConnector("owning", owning_dashboard_config)
  File "/home/p/prynhart/.pex/install/user_sync-1.1.1-py2-none-any.whl.21a3a678dccecc3c248aaab94e34af462a77008d/user_sync-1.1.1-py2-none-any.whl/user_sync/connector/dashboard.py", line 93, in __init__
    user_agent="user-sync/" + APP_VERSION
  File "/home/p/prynhart/.pex/install/umapi_client-2.0.3-py2.py3-none-any.whl.27fb02ed364b2eb12931a3425c557b9f3d7fbe82/umapi_client-2.0.3-py2.py3-none-any.whl/umapi_client/connection.py", line 114, in __init__
    self.auth = self._get_auth(ims_host=ims_host, ims_endpoint_jwt=ims_endpoint_jwt, **auth_dict)
  File "/home/p/prynhart/.pex/install/umapi_client-2.0.3-py2.py3-none-any.whl.27fb02ed364b2eb12931a3425c557b9f3d7fbe82/umapi_client-2.0.3-py2.py3-none-any.whl/umapi_client/connection.py", line 130, in _get_auth
    token = AccessRequest("https://" + ims_host + ims_endpoint_jwt, api_key, client_secret, jwt())
  File "/home/p/prynhart/.pex/install/umapi_client-2.0.3-py2.py3-none-any.whl.27fb02ed364b2eb12931a3425c557b9f3d7fbe82/umapi_client-2.0.3-py2.py3-none-any.whl/umapi_client/auth.py", line 51, in __call__
    jwt_token = jwt.encode(payload, self.key, algorithm='RS256')
  File "/home/p/prynhart/.pex/install/PyJWT-1.4.2-py2.py3-none-any.whl.e2814ae802fc1ffaca54777c9702d5c27e3f22ee/PyJWT-1.4.2-py2.py3-none-any.whl/jwt/api_jwt.py", line 56, in encode
    json_payload, key, algorithm, headers, json_encoder
  File "/home/p/prynhart/.pex/install/PyJWT-1.4.2-py2.py3-none-any.whl.e2814ae802fc1ffaca54777c9702d5c27e3f22ee/PyJWT-1.4.2-py2.py3-none-any.whl/jwt/api_jws.py", line 98, in encode
    key = alg_obj.prepare_key(key)
  File "/home/p/prynhart/.pex/install/PyJWT-1.4.2-py2.py3-none-any.whl.e2814ae802fc1ffaca54777c9702d5c27e3f22ee/PyJWT-1.4.2-py2.py3-none-any.whl/jwt/algorithms.py", line 167, in prepare_key
    key = load_pem_private_key(key, password=None, backend=default_backend())
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/hazmat/backends/__init__.py", line 71, in default_backend
    _default_backend = MultiBackend(_available_backends())
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/hazmat/backends/__init__.py", line 28, in _available_backends
    "cryptography.backends"
  File "/home/p/prynhart/user-sync/.bootstrap/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/hazmat/backends/openssl/backend.py", line 17, in <module>
    from cryptography import utils, x509
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/x509/__init__.py", line 7, in <module>
    from cryptography.x509.base import (
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/x509/base.py", line 16, in <module>
    from cryptography.x509.extensions import Extension, ExtensionType
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/x509/extensions.py", line 20, in <module>
    from cryptography.hazmat.primitives import constant_time, serialization
  File "/home/p/prynhart/.pex/install/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl.97c7929f138bb30736fdb7bae240a1582722a8f2/cryptography-1.7.2-cp27-cp27m-linux_x86_64.whl/cryptography/hazmat/primitives/constant_time.py", line 9, in <module>
    from cryptography.hazmat.bindings._constant_time import lib
ImportError: /home/p/prynhart/.pex/install/cffi-1.9.1-cp27-cp27m-linux_x86_64.whl.9f1fc2a07650a8df3e6ebc7cd4f5dd4d7a6eb9d9/cffi-1.9.1-cp27-cp27m-linux_x86_64.whl/_cffi_backend.so: undefined symbol: PyUnicodeUCS2_FromUnicode
2017-03-09 10:20:59 30841 INFO main - ========== End Run (Total time: 0:00:00) ===================

Accumulate statistics and write them to the log file. Including:

Number of directory users
Number of Adobe users
Number of users created
Number of users updated
Number of users with updated groups
Number of users removed from mapped groups because they were not in directory
Number of users removed/deleted
Number of users with no changes

Steps

1. Set up AD with both enterpriseID users and federatedID users (@ensemble.ca = enterprise and @ensemble-systems.com = federated)
2. Add the config setting user_identity_type: federatedID under dashboard:
3. Run python user-sync.pex --users group "CCE AD Group 1" --process-groups. (Only the federatedID users will be added to the dashboard)
4. Remove user_identity_type: federatedID (So it would default to enterpriseID)
5. Run python user-sync.pex --users group "CCE AD Group 1" --process-groups. again. (The enterpriseID users will be added to the dashboard)
6. Add back the user_identity_type: federatedID under dashboard:
7. Run python user-sync.pex --users group "CCE AD Group 1" --process-groups. again.

Results

• I see a "ServerError: Server Error (500)"
  server error log.txt
• It seems I can only trigger this error with the --process-group argument.

Question.
Is it possible to sync both enterpriseID users and federatedID users from AD at the same time? Or does this need the user to add custom extension code in the config for it to work? It seems to only allow one user type per run.

user-sync-config.txt

When a user who was a member of a mapped group is removed from the directory, the group membership is not removed. This is when run as

python user-sync.pex --users file example.users-file.csv --process-groups

From the behavior, the loop over Adobe group members checking that each is still in the directory and removing from group if not is missing or incorrect.

Should be an error if unrecognized stuff on the command line. eg

python user-sync.pex --users file example.users-file.csv --process-group

use case:

• [email protected] is added to adobe.com as Adobe ID and is not part of any Org.
• my Org has the "somedomain.com" owned now, and I want to add the same e-mail to it as federatedID
• this user is added to an AD group and I try to sync up with AC running it with --users group "Adobe Acrobat DC" --update-user-info --process-groups arguments.
  Result: This is an ERROR with a message like "user not in Org".

On Admin Console side if I search for this user, it is not there.

Expected: user should be added to my Org as federatedID, since the AdobeID is not there either and only one of them can be there using same e-mail address.

Workaround: I can manually add the user as federatedID into Admin Console. In case there are many I can use bulk upload in Admin Console.

As Python 2 end-of-life is approaching, we need to port user-sync to Python 3 for customers who are transitioning. As with umapi-client, we should support both Python 2 and 3.

A number of security vulnerabilities were fixed.
That should say Python in the title.

If you have
limits:
max_deletions_per_run= 10

you get
2017-03-02 17:31:55 17952 ERROR main - Unhandled exception
Traceback (most recent call last):
File "c:\user-sync.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\app.py", line 256, in main
config_loader = create_config_loader(args)
File "c:\user-sync.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\app.py", line 184, in create_config_loader
config_loader = user_sync.config.ConfigLoader(config_bootstrap_options)
File "c:\user-sync.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\config.py", line 75, in init
config_content = self.load_from_yaml(self.main_config_path)
File "c:\user-sync.pex\install\user_sync-1.1.1-py2-none-any.whl.1d38cd7281dcf25d7c48135e8d87ce573a9f8c9f\user_sync-1.1.1-py2-none-any.whl\user_sync\config.py", line 260, in load_from_yaml
return yaml.load(input_file)
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml_init_.py", line 71, in load
return loader.get_single_data()
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\composer.py", line 36, in get_single_node
document = self.compose_document()
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\composer.py", line 127, in compose_mapping_node
while not self.check_event(MappingEndEvent):
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\parser.py", line 98, in check_event
self.current_event = self.state()
File "c:\user-sync.pex\install\pyyaml-3.12-cp27-cp27m-win_amd64.whl.95f7f7154d83db289311a6943ed874bc14dae938\pyyaml-3.12-cp27-cp27m-win_amd64.whl\yaml\parser.py", line 439, in parse_block_mapping_key
"expected , but found %r" % token.id, token.start_mark)
ParserError: while parsing a block mapping
in "user-sync-config.yml", line 1, column 1
expected , but found ''
in "user-sync-config.yml", line 81, column 5

Sometimes there is a perfectly good error message at the end of stack trace though not in this case.

We should get as good a message as possible with the file/line/column if known and not unhandled exception.

• Create the ability to mock UMAPI call results so that manually created results can be returned from UMAPI for specific tests.

• Create the ability to mock directory call results so that manually created results can be returned from directory query calls.

• It will probably also be necessary to mock authentication and access calls.

• Create some tests using these capabilities:

  • Basic create function where directory returns some users and Adobe org has no users. Check that expected user create and add to groups are called.
  • Basic update function where directory returns some users already in Adobe but group memberships and some fields are different.
  • Basic remove function where Adobe returns some users not in the directory and check actions run based on different removal options --remove-nonexistent-users, --delete-nonexistent-users, --remove-entitlements-for-nonexistent-users, --generate-remove-list
  • Cover enterprise and federated users

Think about using record and playback with smart compare as a mechanism.

logging:
file_log_level: errorx
-- no error message

logging:
file_log_level: error
-- still shows debug messages

Steps

1. Run the user-sync without any arguments (python user-sync.pex) or reading from a csv file only (python user-sync.pex --users file users-file.csv --process-groups)

Results

• There would be a warning message: "WARNING config - Found unused keys: ['ldap'] in: <user-sync-config.yml>.directory.connectors"

(Found in v1.1.1) When doing --process-groups and an Adobe ID user is found in the dashboard in a group that should be removed, user-sync issues the appropriate remove command but gets an error result from the server indicating the user is not found.

It's not clear whether this is a client problem (specifying the wrong ID type, perhaps?) or a server problem. Here's the relevant log snippet running against the "Adobe Production Validation: Stress Test 1" organization, and I've attached the config and the input users file below:

2017-03-01 03:58:36 8665 INFO processor - Total directory users after filtering: 8
2017-03-01 03:58:36 8665 DEBUG processor - Group work list: {None: {'jsmith3,k.on-the-side.net': set(['dbrotsky test all apps']), '[email protected]': set([]), '[email protected]': set(['dbrotsky test ug']), '[email protected]': set(['dbrotsky test all apps']), '[email protected]': set(['dbrotsky test all apps', 'dbrotsky test ug']), 'jsmith1,k.on-the-side.net': set(['dbrotsky test ug']), 'jsmith4,k.on-the-side.net': set([]), 'jsmith2,k.on-the-side.net': set(['dbrotsky test ug'])}}
2017-03-01 03:58:36 8665 INFO processor - ---------- End Load from Directory (Total time: 0:00:00) ---
2017-03-01 03:58:36 8665 INFO processor - ---------- Start Sync Dashboard ----------------------------
2017-03-01 03:58:36 8665 INFO processor - Syncing owning...
2017-03-01 03:58:36 8665 DEBUG requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): usermanagement.adobe.io
2017-03-01 03:58:37 8665 DEBUG requests.packages.urllib3.connectionpool - https://usermanagement.adobe.io:443 "GET /v2/usermanagement/users/210DB41957FFDC210A495E53@AdobeOrg/0 HTTP/1.1" 200 1391
2017-03-01 03:58:37 8665 DEBUG umapi_client - Ran multi-user query: [] {} (page 0: 35 found)
2017-03-01 03:58:37 8665 INFO processor - Adobe User not in Directory: [email protected]
2017-03-01 03:58:37 8665 INFO processor - Removed from Groups: set(['dbrotsky test ug'])
2017-03-01 03:58:37 8665 INFO processor - Managing groups for user key: [email protected] organization: None added: None removed: set(['dbrotsky test ug'])
2017-03-01 03:58:37 8665 INFO dashboard.owning.action - Added action: {"do": [{"remove": {"product": ["dbrotsky test ug"]}}], "requestID": "action_1", "user": "[email protected]"}
2017-03-01 03:58:37 8665 DEBUG requests.packages.urllib3.connectionpool - https://usermanagement.adobe.io:443 "POST /v2/usermanagement/action/210DB41957FFDC210A495E53@AdobeOrg HTTP/1.1" 200 192
2017-03-01 03:58:37 8665 ERROR dashboard.owning.action - Error requestID: action_1 code: "error.user.not_found" message: "No valid users were found in the request"
2017-03-01 03:58:37 8665 INFO processor - ---------- End Sync Dashboard (Total time: 0:00:01) --------

users-file.csv
user-sync-config.yml

In particular, --remove-entitlements-for-nonexistent-users

If syncing from a file where the file looks like this:

firstname,lastname,email,country,groups,type,user,domain
John,Smith,[email protected],US,"all_apps",federatedID,jsmith

then you get an unhandled exception (see below). It should either take the domain from the email address or give a more reasonable error message.

2017-02-23 12:12:08 16808 ERROR main - Unhandled exception
Traceback (most recent call last):
File "c:\users....pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\app.py", line 262, in main
begin_work(config_loader)
File "c:\users....pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\app.py", line 168, in begin_work
rule_processor.run(directory_groups, directory_connector, dashboard_connectors)
File "c:\users....pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\rules.py", line 84, in run
self.read_desired_user_groups(directory_groups, directory_connector)
File "c:\users....pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\rules.py", line 143, in read_desired_user_groups
user_key = RuleProcessor.get_directory_user_key(directory_user)
File "c:\users....pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\rules.py", line 555, in get_directory_user_key
return RuleProcessor.get_user_key(directory_user['username'], directory_user['domain'], directory_user['email'])
File "c:\users...t.pex\install\user_sync-1.0rc1-py2-none-any.whl.ea43aaba8f06fbb809d14379ef8554f20ee5a174\user_sync-1.0rc1-py2-none-any.whl\user_sync\rules.py", line 574, in get_user_key
return username + ',' + domain
TypeError: cannot concatenate 'str' and 'NoneType' objects
2017-02-23 12:12:08 16808 INFO main - ========== End Run (Total time: 0:00:01) ===================

If a federated domain is set up for username based login (rather than email based), then user sync does not seem to recognize users. If you run the same sync repeatedly, it creates the same users and adds them to the same PLCs on every run.

Currently, the UMAPI error messages are displayed without being near the parameters that are being sent. This makes it really hard to figure out what call the error message corresponds to. The requestID connects them. Can you put the call info with the return info?

2017-02-01 17:14:20 13396 INFO processor - Adding user with user key: [email protected]
2017-02-01 17:14:20 13396 INFO dashboard.owning.action - Added action: {"do": [{"createEnterpriseID": {"lastname": "Roe", "country": "US", "email": "[email protected]", "firstname": "Richard", "option": "ignoreIfAlreadyExists"}}], "requestID": "action_2", "user": "[email protected]"}
2017-02-01 17:14:20 13396 INFO processor - Adding user with user key: [email protected]
2017-02-01 17:14:20 13396 INFO dashboard.owning.action - Added action: {"do": [{"createEnterpriseID": {"lastname": "Smith", "country": "US", "email": "[email protected]", "firstname": "John", "option": "ignoreIfAlreadyExists"}}], "requestID": "action_3", "user": "[email protected]"}
2017-02-01 17:14:20 13396 DEBUG requests.packages.urllib3.connectionpool - https://usermanagement.adobe.io:443 "POST /v2/usermanagement/action/620049C857E4735B7F000101@AdobeOrg?testOnly=true HTTP/1.1" 200 248
2017-02-01 17:14:20 13396 WARNING dashboard.owning.action - Error requestID: action_1 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"
2017-02-01 17:14:20 13396 WARNING dashboard.owning.action - Error requestID: action_2 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"
2017-02-01 17:14:20 13396 WARNING dashboard.owning.action - Error requestID: action_3 code: "error.user.belongs_to_another_org" message: "Illegal to invite user from another organization's owned auth src"

If there is an entry in the dashboard-config.yml file like this:

priv_key_path: private.key

then the file should be interpreted as relative to the directory containing the dashboard-config.yml file. Now it appears it is relative to the current directory so that if there is a -c option for the config file location, this file ends up not being found unless it is fully qualified.

Create a feature to protect Adobe users from being updated or removed by User Sync. The purpose of this feature is to provide a means for customers to prevent User Sync from interfering with users that they are administering manually.

We extend the existing dashboard configuration with the ability to identify Adobe accounts that are immune from update, group change, and removal. Users so identified are protected from any alteration by User Sync including
--remove-nonexistent-users,
--delete-nonexistent-users,
--remove-entitlements-for-nonexistent-users
--update-user-info - updating fields of the user
--process-groups - updating group membership
--remove-list (etc)

The additional configuration options under dashboard are as follows:

dashboard:
  owning: dashboard-config.yml  # existing key
  exclude_groups:  # new key: list of dashboard-side groups (user group or product configuration)
    - special_users   # members of any group in this list are excluded from updates/remove/etc.
    - more special users   # can have more than one group in a list
  exclude_users:   # new key: list of regular expressions matched against dashboard-side usernames
    - ".*@example.com"  # any matching user is excluded from updates/remove/etc.
    - pattern2                   # can have more than one regular expression in a list
  exclude_identity_types: # new key: list of identity types
    - adobeID                     # default in config file: means exclude Adobe IDs from update/remove/etc. 
    - enterpriseID               # can exclude additional identity types
    - federatedID

exclude_groups defines a list of Adobe user groups or PCs. Adobe users who are members of such groups are not removed, updated, nor have their group membership changed.

exclude_users gives a list of patterns. Adobe users with usernames that match (default case-insensitive, unless the pattern specifies case-sensitive) any of the specified patterns (if there are more than one) are not removed, updated, nor have their group membership changed.

exclude_identity_types gives a list of identity types. Adobe users who have one of these identity types are not removed, updated, nor have their group membership changed.

Note that directory side users are still created on the Adobe side even if one of the exclude- parameters would then exclude the user on the Adobe side from updates in successive runs. That is, these parameters apply only to Adobe users who exist when the Adobe directory is being matched against the customer directory.

All of these additional configuration options are optional and default to empty (that is, no dashboard users are excluded from update/remove/etc.). They should appear in the example main config file commented out, except for exclude_identity_types, which should have the sole member value adobeID.

Accounts that would have been removed or updated but were not because of this feature should be listed as info level log entries.

UserSync is not creating (i.e., using the UMAPI to invite) new users of type Adobe ID. In fact there is no way to specify Adobe ID as the identity type for a new user.

This isn't an issue. It is a test for notifications.

Steps

1. Set up a custom attribute in the config:

extensions:

• context: per-user
  extended_attributes:
  • sn
    extended_dashboard_groups:
  • Ensemble Test
  • Ensemble Test 2
    after_mapping_hook: |
    sn = source_attributes.get('sn')
    target_attributes['firstname'] = 'Ensemble123'
    if int(sn) > 3:
    target_groups.add('Ensemble Test')
    elif int(sn) <= 3 :
    target_groups.add('Ensemble Test 2')

2. Run the user-sync while pointing to a csv user file: python user-sync.pex --users file users-file.csv

Results

• This error would appear:

2017-03-24 15:10:06 6164 ERROR main - Unhandled exception
Traceback (most recent call last):
File "c:\users\raymondf.pex\install\user_sync-1.2rc2-py2-none-any.whl.f8ddbb45621af04445afc502d8a
9ef65e0bf86fb\user_sync-1.2rc2-py2-none-any.whl\user_sync\app.py", line 363, in main
begin_work(config_loader)
File "c:\users\raymondf.pex\install\user_sync-1.2rc2-py2-none-any.whl.f8ddbb45621af04445afc502d8a
9ef65e0bf86fb\user_sync-1.2rc2-py2-none-any.whl\user_sync\app.py", line 209, in begin_work
rule_processor.run(directory_groups, directory_connector, dashboard_connectors)
File "c:\users\raymondf.pex\install\user_sync-1.2rc2-py2-none-any.whl.f8ddbb45621af04445afc502d8a
9ef65e0bf86fb\user_sync-1.2rc2-py2-none-any.whl\user_sync\rules.py", line 157, in run
self.read_desired_user_groups(directory_groups, directory_connector)
File "c:\users\raymondf.pex\install\user_sync-1.2rc2-py2-none-any.whl.f8ddbb45621af04445afc502d8a
9ef65e0bf86fb\user_sync-1.2rc2-py2-none-any.whl\user_sync\rules.py", line 291, in read_desired_user_
groups
exec(options['after_mapping_hook'], self.after_mapping_hook_scope)
File "", line 3, in
TypeError: int() argument must be a string or a number, not 'NoneType'
2017-03-24 15:10:06 6164 INFO main - ========== End Run (User Sync version: 1.2rc2) (Total time: 0:0
0:02)

I'm testing the sync tool with a couple of users that I've synced from my Active Directory sandbox. I was able to create and provision them successfully.

When I mark those users as disabled in my sandbox AD, the sync tool doesn't do anything to deprovision them.

Here's the log I get when I run it after disabling the users in AD:

log.txt

What I expect to happen, since I'm not running with the "remove users" option, is for the disabled accounts to have their product entitlements removed, essentially disabling those accounts in the admin console.

For reference, here is my product config. I've ensured that those users only belong to product config groups that are mapped in the config file.

    - directory_group: ADOBE-ALL-APPS
      dashboard_groups:
        - All Apps
    - directory_group: ADOBE-PHOTOSHOP
      dashboard_groups:
        - Photoshop
    - directory_group: ADOBE-VIDEO
      dashboard_groups:
        - Default Custom plan - Video - 6 GB configuration

This error occurs when creating a new EnterpriseID user in the stage environment. I was not able to replicate this with FederatedID or in the production environment.

ERROR dashboard.owning.action - Error requestID: action_3 code: "error.internal.exceptionflys" message: "Severe exception: null"

The users were created successfully regarding the error message.

I ran the following command:

./user-sync --process-groups --users group "SG-Adobe-All-App"

user-sync-config.txt
2017-03-02.txt

The docs say that all users can be removed/deleted, but in fact the code in process_orphaned_dashboard_users still restricts itself to processing Federated ID users.

(I noticed this when fixing a different issue; now I'm going to try to repo.)

Please update the pdf and word documentation on page 10
in "Example configurations, under user-sync.config.yml:
“dashboard" should not contain attribute “user-identity-type”. Should be under "directory"

Thanks!

I don't see any mention of user_username_format or user_domain_format in the Sync Guide. I'm helping a customer that uses username-based auth on their federated domain, and we had to search the source code to figure out how to set username and domain on user records.

Steps

1. Run python user-sync.pex --input-unmatched-users test-remove.txt --delete-unmatched-users

Results

• There's a line that says "2017-03-24 15:59:16 9468 INFO main - Read 1 umatched users."

Call UMAPI client directly as user operations are identified. It will automatically bundle operations and run them in batches. Clean up code so there isn't duplicate functionality for creating and managing JSON requests; let UMAPI client handle this.

If the file specified using the --source-filter command line option is valid YAML but contains no valid filter, then no exception is thrown and no warning is given. For example, in one test the command line specified

--source-filter ldap:filter.yml

where the specified file's content was just

(cn=Smoke*)

This content is faulty (should have been all_users_filter: "(cn=Smoke*)" ) and doesn't define a valid filter. It's valid YAML though, parsing as a single string, so no exception is thrown.

It would be useful to produce a warning in the log for such a case.

The "Enterprise Dashboard" has been rebranded to the "Enterprise Admin Console" everywhere but, apparently, in User Sync. While this would be a rather widespread change, affecting not only the code but also the configuration options and configuration file names, and while it would not be backward compatible, it's one that I believe we should make sooner rather than later.

Question: if we did this change, should we maybe bump the major version number? After all, it is backwards-incompatible, and we have been talking for quite a while about the "2.0" release of user-sync. I'm thinking, what with all the features in this 1.2 release, we should maybe rename it the 2.0 release and do the rename...

It appears that mapped PLCs are removed from a disabled user (disabled in the directory) even when those PLCs are not specified in the --users group argument.

Example-

Here are two users disabled in my test AD environment:

Test User 1 exists in my admin console, and is assigned two PLCs - All Apps and Adobe Sign

Here's the config I'm using - user-sync-config.yml.txt

And the command to run sync-

py -2 .\user-sync.pex --process-groups --users group ADOBE-SIGN

When I run the sync tool, I expect it to remove the Sign PLC, but not All Apps, since All Apps is not specified in the --user groups param. Here's the log showing that the user is removed from both PLCs:

2017-03-01 16:26:29 7716 INFO config - Using main config file: user-sync-config.yml
2017-03-01 16:26:29 7716 INFO main - ========== Start Run =======================================
2017-03-01 16:26:29 7716 INFO connector.ldap - Connecting to: ldap://10.51.19.15 using username: ldapuser
2017-03-01 16:26:30 7716 INFO connector.ldap - Connected
2017-03-01 16:26:30 7716 INFO dashboard.owning - Creating connection for org id: "5190B7C05746859A7F000101@AdobeOrg" using private key file: "C:\Users\adorton\training\user-sync\private.key"
2017-03-01 16:26:31 7716 INFO dashboard.owning - API initialized on: https://usermanagement-stage.adobe.io/v2/usermanagement
2017-03-01 16:26:31 7716 INFO processor - ---------- Start Load from Directory -----------------------
2017-03-01 16:26:31 7716 INFO processor - Building work list...
2017-03-01 16:26:31 7716 INFO connector.ldap - Loading users...
2017-03-01 16:26:31 7716 WARNING connector.ldap - No email attribute: mail for dn: CN=Administrator,CN=Users,DC=ccestestdomain,DC=com
2017-03-01 16:26:31 7716 WARNING connector.ldap - No email attribute: mail for dn: CN=itcloud,CN=Users,DC=ccestestdomain,DC=com
2017-03-01 16:26:31 7716 WARNING connector.ldap - No email attribute: mail for dn: CN=BladeLogicRSCD,CN=Users,DC=ccestestdomain,DC=com
2017-03-01 16:26:31 7716 WARNING connector.ldap - No email attribute: mail for dn: CN=ldapuser,CN=Users,DC=ccestestdomain,DC=com
2017-03-01 16:26:31 7716 WARNING connector.ldap - No email attribute: mail for dn: CN=Kevin Bhunut,CN=Users,DC=ccestestdomain,DC=com
2017-03-01 16:26:31 7716 INFO connector.ldap - Total users loaded: 0
2017-03-01 16:26:32 7716 INFO processor - Total directory users after filtering: 0
2017-03-01 16:26:32 7716 INFO processor - ---------- End Load from Directory (Total time: 0:00:00) ---
2017-03-01 16:26:32 7716 INFO processor - ---------- Start Sync Dashboard ----------------------------
2017-03-01 16:26:32 7716 INFO processor - Syncing owning...
2017-03-01 16:26:33 7716 INFO processor - Adobe User not in Directory: [email protected]
2017-03-01 16:26:33 7716 INFO processor - Removed from Groups: set([u'adobe sign', u'all apps'])
2017-03-01 16:26:33 7716 INFO processor - Managing groups for user key: [email protected] organization: None added: None removed: set([u'adobe sign', u'all apps'])
2017-03-01 16:26:33 7716 INFO dashboard.owning.action - Added action: {"do": [{"remove": {"product": ["adobe sign", "all apps"]}}], "requestID": "action_1", "user": "[email protected]"}
2017-03-01 16:26:35 7716 INFO processor - ---------- End Sync Dashboard (Total time: 0:00:02) --------
2017-03-01 16:26:35 7716 INFO main - ========== End Run (Total time: 0:00:05) ===================

If I remove the All Apps mapping from my config and run through this use case again, then the All Apps PLC is not removed. Should it also be retained if it is in the config, but not specified in --users group?

It is hard to diagnose problem when customers provide logs without this.

Added more info in my comment below.

Steps

1. Set up AD users with country (ie: CA, US or none)
2. In the after_mapping_hook, add: target_attributes['country'] = 'GB'
3. Run the user-sync with python user-sync.pex --users group "CCE AD Group 1" --update-user-info --process-groups
4. Go to the dashboard user management and export the user list to confirm the country codes

Results

• The country codes for the users remain the same as AD (ie: CA, US or UD)
• In the log, I see that the GB is in the
  2017-03-27 17:16:47 10648 DEBUG processor - Target attrs, after: {'username': '[email protected]', 'domain': None, 'uid': None, 'firstname': '!Custom', 'lastname': 'Tes', 'country': 'GB', 'email': '[email protected]'}

extensions:
  - context: per-user
    extended_attributes:
      - sn
      - title
    extended_dashboard_groups:
      - Ensemble Test
      - Ensemble Test 2
    after_mapping_hook: |
      sn = source_attributes.get('sn')
      title = source_attributes.get('title')
      if title is not None:
        target_attributes['lastname'] = title[0:3]
      target_attributes['firstname'] = '!Custom'
      target_attributes['country'] = 'GB'
      if int(sn) > 3:
        target_groups.add('Ensemble Test')
      elif int(sn) <= 3 :
        target_groups.add('Ensemble Test 2')

Full config and log:
user-sync-config.txt
country code log.txt

Please provide comment on the below

"In my testing, the only way to get SyncTool to work is to point the connector-ldap.yml config file to the root of our Active Directory so as to encompass both locations (Obviously I could place the Adobe groups in the "accounts" OU with the users but this messes with our structure). With this setup, SyncTool then syncs my entire directory of users, meaning we get generic email accounts, test accounts, service accounts etc, into the Adobe console when they don't really need to be there.

The command I've been testing with is the following: (all config files, certs etc are all in the same directory as the user-sync executable and I'm running the SyncTool from that directory)

user-sync --users all --process-groups

This works perfectly and does what I expect when I have group mappings configured in user-sync-config.yml

Ideally I'd like to know if it's possible that only users that are actually in one of the mapped AD groups to sync to the Adobe Console or to restrict where SyncTool pulls users from. I did note that there is a "—users group group1, group2" switch but that appears to sync users only and does not apply any Adobe user group management which is not really what we're after.

For us (and I realise this is a specific use case but I thought I'd run it by you in any case) ideally we'd have some way to specify where to look for users, and some way to exclude some OUs we don't want examined for users.

e.g
users: "OU=accounts,DC=domain,DC=com,DC=au"
groups: "OU=Adobe,OU=SecurityGroups,DC=domain,DC=com,DC=au"
exclude:

• "OU=generic_accounts,OU=accounts,DC=domain,DC=com,DC=au"
• "OU=generic_emails,OU=accounts,DC=domain,DC=com,DC=au"

Some other things we ran into and would like to query is around how SyncTool treats expired users and disabled users.

If a user is disabled and SyncTool has not seen it before, that disabled user does not sync to the Adobe Console – perfect and as expected.
If a user is disabled and already exists in the Adobe Console and remains in a mapped AD groups, SyncTool ignores that user account does not remove it from the Adobe User Group.
If a user is disabled and already exists in the Adobe Console and is removed from a mapped AD groups. SyncTool ignores that user account and does not remove it from the Adobe User Group

We have lots and lots of freelancers come in and out of the business, we don't necessarily disable their accounts but set the account expiry based on their contract length. Expired users are pretty much treated as active users in that they will Sync to the Adobe Console and be placed in an Adobe User group if they are in a mapped AD group. If we are using the Adobe Console for licensing numbers, the above scenarios could be problematic"

The --remove-nonexistent-users command-line param only applies to Federated users. Should it also work for Enterprise users?

For details, see #9 (comment)