Hi, I am currently working on a terraform provider for thebastion. H

I try the branch of <a href="https://github.com/ovh/the-bastion/pull/377" data-hoverca

Hey <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Hey <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

Could you try the branch of PR <a class="issue-link js-issue-link" data-error-text="Fa

Deleted account name and uid blocked about the-bastion HOT 8 CLOSED

Pierrelefort commented on June 27, 2024

Deleted account name and uid blocked

from the-bastion.

Comments (8)

Pierrelefort commented on June 27, 2024 1

I try the branch of PR with the same operations i did for the log and it work fine !

from the-bastion.

Pierrelefort commented on June 27, 2024

UPDATE: My custom terraform provider create two accounts using the "--uid-auto" option. Both tried to create an account with the same uid, creating an error. But, i don't know why the bastion has blocked the name or the uid.

from the-bastion.

speed47 commented on June 27, 2024

Hey @Pierrelefort ,

Nice to hear that you're working on a terraform provider! This is actually on my backlog because we also have the need internally, especially to handle add/removing of IPs in bastion groups (in that case, the account used by terraform would be an aclkeeper of these groups). I'll opensource it when it's ready (as we've done with the ansible wrapper).

Now about your issue, do you have an exact list of steps to reproduce? I've tried creating two accounts with --uid-auto, deleting the first one, then creating a third one with --uid-auto, I can't seem to stumble upon it.

On your screenshots, it would imply that the "test" account no longer exists, but its primary group (also named "test") is still there, as if the account deletion process had been interrupted; but that's just what I'm trying to infer based on your screenshots.

To get more info, you might want to try running the /opt/bastion/bin/admin/check-consistency.pl as root on the bastion server, it doesn't modify anything, just checks several things around the accounts and groups and reports inconsistencies when there are.

from the-bastion.

ogirardot commented on June 27, 2024

Hi @speed47 (I'm working with @Pierrelefort ), I've sent you an email related to this ongoing work which is going to be open source as well, we've focused first on users, groups and server ips if you'd like to sync up, it seems we'd both benefit to join forces.

from the-bastion.

speed47 commented on June 27, 2024

Hey @ogirardot , just replied to your email, I missed it originally!

from the-bastion.

Pierrelefort commented on June 27, 2024

Nice to hear that you're working on a terraform provider! This is actually on my backlog because we also have the need internally, especially to handle add/removing of IPs in bastion groups (in that case, the account used by terraform would be an aclkeeper of these groups). I'll opensource it when it's ready (as we've done with the ansible wrapper).

We are close to open the repository to the public! And in our case we went with an admin account for his impersonate method (adminSudo) to remove/add ingress keys to user.

Now about your issue, do you have an exact list of steps to reproduce? I've tried creating two accounts with --uid-auto, deleting the first one, then creating a third one with --uid-auto, I can't seem to stumble upon it.

It happened when we try to create two users the option --uid-auto on parallel calls. When terraform want to create resources (here missing users), he create them on parallel. When we debug the issue, we found out thebastion return the same uid on those parallel calls. Since we cannot change terraform comportement easily we decided to avoid --uid-auto option.

I managed to reproduce the error with log:

2023-03-20T17:46:09.402+0100 [INFO]  provider.terraform-provider-thebastion: Request bastion: --osh accountCreate --account "test2" --uid-auto --public-key "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcjliyS0gOlGrxz0bX0S6GV1roGW2beEiIB+/yzygXzL7vzRU3u6Ty/wODC+kABNebtgJ7TCFj387drS3A14bojFlbSlS+r9bdToczfc0ZxwV89ToEGkw4hWIsTSw2ADg9aTIDclAZjNtE+SQUZLSS1gKJSHKah4SWaMf7CSHy7zKg4Q70qHEXJ+UCPfR30glX7joH5kny81aY9vRtRQKs6/RbG8Zd2CoxBkNAYA2k9NPVKEv3eUhiwkK+c1Zf9L5Fk2mW1jhvOwQ4auvZdV/mh/mY5uWqV2Q7KjhpucnVVgv87Uv6drL2lvQyDOvl1G03ab+rXS7eKD3aX1MkphxCrSsNaG4lTT0NB72Wa64CrCHGMcqPrdAhHkRnze/XdmXW7FOlo+nmLPRBZlBME+XT9yyQFNxksJpTAZEK33Xwccoq9PwqPsOFIHPS8PiVifQMarLXonlCz++wzoFEsdYCxdvU/jJmjBvsBcFXV+V5whtOc9JGAJ6JrtnEJJd774c=" --json: tf_provider_addr=hashicorp.com/ovh/thebastion tf_req_id=56f88413-8cad-c58e-3206-3baa5a60c9d9 tf_rpc=ConfigureProvider thebastion_host=*** @caller=/Users/pierre/Documents/terraform-provider-thebastion/thebastion/clients/client_ssh.go:64 thebastion_path_private_key=/Users/pierre/Documents/terraform-provider-thebastion/idthebastion thebastion_username=poweruser @module=thebastion thebastion_path_known_host=/Users/pierre/.ssh/known_hosts timestamp=2023-03-20T17:46:09.401+0100
2023-03-20T17:46:09.402+0100 [INFO]  provider.terraform-provider-thebastion: Request bastion: --osh accountCreate --account "test1" --uid-auto --public-key "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcjliyS0gOlGrxz0bX0S6GV1roGW2beEiIB+/yzygXzL7vzRU3u6Ty/wODC+kABNebtgJ7TCFj387drS3A14bojFlbSlS+r9bdToczfc0ZxwV89ToEGkw4hWIsTSw2ADg9aTIDclAZjNtE+SQUZLSS1gKJSHKah4SWaMf7CSHy7zKg4Q70qHEXJ+UCPfR30glX7joH5kny81aY9vRtRQKs6/RbG8Zd2CoxBkNAYA2k9NPVKEv3eUhiwkK+c1Zf9L5Fk2mW1jhvOwQ4auvZdV/mh/mY5uWqV2Q7KjhpucnVVgv87Uv6drL2lvQyDOvl1G03ab+rXS7eKD3aX1MkphxCrSsNaG4lTT0NB72Wa64CrCHGMcqPrdAhHkRnze/XdmXW7FOlo+nmLPRBZlBME+XT9yyQFNxksJpTAZEK33Xwccoq9PwqPsOFIHPS8PiVifQMarLXonlCz++wzoFEsdYCxdvU/jJmjBvsBcFXV+V5whtOc9JGAJ6JrtnEJJd774c=" --json: tf_provider_addr=hashicorp.com/ovh/thebastion tf_req_id=56f88413-8cad-c58e-3206-3baa5a60c9d9 tf_rpc=ConfigureProvider thebastion_username=poweruser @caller=/Users/pierre/Documents/terraform-provider-thebastion/thebastion/clients/client_ssh.go:64 thebastion_host=*** thebastion_path_known_host=/Users/pierre/.ssh/known_hosts thebastion_path_private_key=/Users/pierre/Documents/terraform-provider-thebastion/idthebastion @module=thebastion timestamp=2023-03-20T17:46:09.401+0100
2023-03-20T17:46:09.929+0100 [DEBUG] provider.terraform-provider-thebastion: Called provider defined Resource Create: tf_req_id=80f6d29a-6d74-bc7e-eef1-648e4b6370d5 @caller=/Users/pierre/go/pkg/mod/github.com/hashicorp/[email protected]/internal/fwserver/server_createresource.go:98 @module=sdk.framework tf_provider_addr=hashicorp.com/ovh/thebastion tf_resource_type=thebastion_user tf_rpc=ApplyResourceChange timestamp=2023-03-20T17:46:09.929+0100
2023-03-20T17:46:09.930+0100 [ERROR] provider.terraform-provider-thebastion: Response contains error diagnostic: tf_proto_version=6.3 tf_provider_addr=hashicorp.com/ovh/thebastion tf_rpc=ApplyResourceChange diagnostic_summary="Error creating user" @caller=/Users/pierre/go/pkg/mod/github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail="Could not create user, unexpected error: thebastion error code: ERR_USERADD_FAILED / msg: Error while running useradd for test1 UID/GID 9997 (Command exited with status 4)" diagnostic_severity=ERROR tf_req_id=80f6d29a-6d74-bc7e-eef1-648e4b6370d5 tf_resource_type=thebastion_user timestamp=2023-03-20T17:46:09.930+0100
2023-03-20T17:46:09.954+0100 [ERROR] vertex "thebastion_user.test1" error: Error creating user
2023-03-20T17:46:10.439+0100 [DEBUG] provider.terraform-provider-thebastion: Response bastion: ---e54a37e12616---------------------------------------the-bastion-3.09.00-rc3---
=> create a new bastion account
--------------------------------------------------------------------------------
~ Creating group test2 with GID 9997...
~ Creating user test2 with UID 9997...
~ Creating tty group of account...
~ Adding account to potential supplementary groups...
~ Creating needed files and directories with proper permissions in home...
~ Creating some more directories...
~ Applying proper ownerships...
~ Adding provided public key in authorized_keys file...
~ Generating account personal bastion key...
~ Account successfully created!
~ Configuring sudoers for this account

*** Regenerating account 'test2' sudoers file from templates

`-> ... generating /etc/sudoers.d/osh-account-test2_126a8a

`-> [�[32m OK �[0m]
~ ==> alias fix-my-config-please-missing-bastion-name='ssh test2@e54a37e12616 -t -- '
~ To test his access, ask this user to set the above alias in their .bash_aliases, then run `fix-my-config-please-missing-bastion-name --osh info'


JSON_START
{"error_message":"OK","error_code":"OK","value":null,"command":"accountCreate"}
JSON_END
-------------------------------------------------------------</accountCreate>---

This log is quite verbose but you can see with :

ERR_USERADD_FAILED / msg: Error while running useradd for test1 UID/GID 9997 (Command exited with status 4)
Creating group test2 with GID 9997... ~ Creating user test2 with UID 9997...
That they are created with the same uid/gid.

Then i have done the following commands on my bastion server:

poweruser@fix-my-config-please-missing-bastion-name(master)> accountList
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
╭──e54a37e12616───────────────────────────────────────the-bastion-3.09.00-rc3───
│ ▶ list bastion accounts
├───────────────────────────────────────────────────────────────────────────────
│ healthcheck          9999
│ poweruser            9998
│ test2                9997
╰──────────────────────────────────────────────────────────────</accountList>───
poweruser@fix-my-config-please-missing-bastion-name(master)> accountDelete --account test2 --no-confirm
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
╭──e54a37e12616───────────────────────────────────────the-bastion-3.09.00-rc3───
│ ▶ delete an existing bastion account
├───────────────────────────────────────────────────────────────────────────────
│ ❗ Hint: account test2 is currently ACTIVE (i.e. not disabled), think twice before removing it!
│
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
│ Backing up home directory...
*** Deleting account 'test2' sudoers file
`-> ... deleting /etc/sudoers.d/osh-account-test2_126a8a
`-> [ OK ]
│ Backup done
│ Removing 'test2' group membership from 'keyreader' user
│ Deleting system user 'test2'...
│ Deleting group test2-tty...

│ Account test2 has been deleted
╰────────────────────────────────────────────────────────────</accountDelete>───
poweruser@fix-my-config-please-missing-bastion-name(master)> accountCreate --account test1 --uid 9997 --no-key
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
╭──e54a37e12616───────────────────────────────────────the-bastion-3.09.00-rc3───
│ ▶ create a new bastion account
├───────────────────────────────────────────────────────────────────────────────
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_TERMINAL = "iTerm2",
	LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

│
│ ⛔ The group test1 already exists
╰────────────────────────────────────────────────────────────</accountCreate>───

Hope that will help you understand the issue !

On your screenshots, it would imply that the "test" account no longer exists, but its primary group (also named "test") is still there, as if the account deletion process had been interrupted; but that's just what I'm trying to infer based on your screenshots.

To get more info, you might want to try running the /opt/bastion/bin/admin/check-consistency.pl as root on the bastion server, it doesn't modify anything, just checks several things around the accounts and groups and reports inconsistencies when there are.

We got the following response from the bastion server:

root@arkhn-bastion: ~# /opt/bastion/bin/admin/check-consistency.pl
found 3 key groups
found 22 bastion users
found 166 groups

from the-bastion.

speed47 commented on June 27, 2024

Okay, so this is clearly a race condition when two creations happen exactly at the same time.
The --uid-auto option doesn't pick a random UID, it picks the highest-still-available one. This is to avoid "stealing" a lower UID that you might need later. This is because, in our use case, we have specific UIDs for humans (the same UID across all the infrastructures), but don't have those for M2M/automation accounts, and we use --uid-auto on those.

I can add a mutex there to avoid two simultaneous creations from picking the same UID. Thanks for the detailed report! 👍 . I'll have a branch for you to test with Terraform, using --uid-auto.

from the-bastion.

speed47 commented on June 27, 2024

Could you try the branch of PR #377 ?

from the-bastion.

Deleted account name and uid blocked about the-bastion HOT 8 CLOSED

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent