Comments (17)
kaushalshriyan
commented on July 24, 2024
Hi,
Is there a way to set a service account and scopes in config.yaml file?
as per https://github.com/GoogleCloudPlatform/healthcare/blob/master/deploy/project_config.yaml.schema? Thanks in advance.
Best Regards,
Kaushal
from healthcare.
xingao267
commented on July 24, 2024
Hi @kaushalshriyan, DPT use Terraform do make deployments under the hood, so for GCE, it's using Terraform's google_compute_instance resource.
To your first question, I think you might not need the items keyword. Try directly
metadata:
- startup-script: sudo apt-get update
- enable-oslogin: TRUE
- ...
And to your second question:
To create a new service account, use service_accounts, and to set the scope for a service account, set it in the place where it is being used. So for example, for compute image, set the scopes as in google_compute_image.
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 Thanks a lot for the reply. I have follow up questions
- I have enabled the array service_accounts: in config.yaml file. I am facing this issue -> https://paste.centos.org/view/b7d929c8
- Is there a way to run "sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh" which is located on my local system. I tried this
metadata-from-file: "/Users/kaushalshriyan/ghcdrupalproject/healthcare/deploy/installnginxmariadbnewtestdrupalghc.sh"in config.yaml file. It does not work.
I look forward to hearing from you. Thanks in advance.
from healthcare.
xingao267
commented on July 24, 2024
@kaushalshriyan you are very welcome
-
Please follow properties for the list of properties that
service_accountsupports. The two supported attributes areaccount_idanddisplay_name. It will create a service account in the format<account_id>@my-project.iam.gserviceaccount.com -
Unfortunately,
metadata-from-fileis not supported by Terraform, or DPT. And I don't think it's currently possible to run an arbitrary script during DPT execution.
from healthcare.
kaushalshriyan
commented on July 24, 2024
Releasing state lock. This may take a few moments...
2020/04/14 20:29:40 Failed to apply configs: failed to apply "ghcdrupalprojectdpt": failed to apply resources: failed to apply plan: exit status 1:
Error: Extraneous JSON object property
on main.tf.json line 165, in resource[5].google_compute_instance.ghcdrupalprojectdpt-instance:
165: "service_accounts": {
No argument or block type is named "service_accounts". Did you mean
"service_account"?
cat config.yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This sample configuration provides the minimum configuration required by the DPT scripts.
# Audit resources will be created locally in the project.
overall:
organization_id: '7954763295'
billing_account: 89NJCG-KL987Y-LPIU76
domain: mydomain.com
generated_fields_path: ./generated_fields.yaml
projects:
- project_id: ghcdrupalprojectdpt
owners_group: [email protected]
auditors_group: [email protected]
audit:
logs_bigquery_dataset:
dataset_id: digitalapicraft_ghcdrupalprojectdpt001_logs # Bigquery Dataset names must use underscores.
location: US
devops:
state_storage_bucket:
name: digitalapicraft-ghcdrupalprojectdpt-state
location: US
compute_instances:
- name: ghcdrupalprojectdpt-instance
zone: us-central1-a
machine_type: n1-standard-1
boot_disk:
initialize_params:
image: centos-cloud/centos-7-v20200309
network_interface:
network: default
access_config: {}
metadata:
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
service_accounts:
accountId: "ghcdrupalproject-owners-digita@ghcdrupalprojectdpt.iam.gserviceaccount.com"
from healthcare.
kaushalshriyan
commented on July 24, 2024
2020/04/14 20:47:33 Running: [terraform apply]
Releasing state lock. This may take a few moments...
2020/04/14 20:47:38 Failed to apply configs: failed to apply "ghcdrupalprojectdpt": failed to apply resources: failed to apply plan: exit status 1:
Error: Extraneous JSON object property
on main.tf.json line 166, in resource[5].google_compute_instance.ghcdrupalprojectdpt-instance.service_account:
166: "accountId": "ghcdrupalproject-owners-digita@ghcdrupalprojectdpt.iam.gserviceaccount.com"
No argument or block type is named "accountId".
Error: Missing required argument
on main.tf.json line 167, in resource[5].google_compute_instance.ghcdrupalprojectdpt-instance.service_account:
167: },
The argument "scopes" is required, but no definition was found.
cat config.yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This sample configuration provides the minimum configuration required by the DPT scripts.
# Audit resources will be created locally in the project.
overall:
organization_id: '7954763295'
billing_account: 89NJCG-KL987Y-LPIU76
domain: mydomain.com
generated_fields_path: ./generated_fields.yaml
projects:
- project_id: ghcdrupalprojectdpt
owners_group: [email protected]
auditors_group: [email protected]
audit:
logs_bigquery_dataset:
dataset_id: digitalapicraft_ghcdrupalprojectdpt001_logs # Bigquery Dataset names must use underscores.
location: US
devops:
state_storage_bucket:
name: digitalapicraft-ghcdrupalprojectdpt-state
location: US
compute_instances:
- name: ghcdrupalprojectdpt-instance
zone: us-central1-a
machine_type: n1-standard-1
boot_disk:
initialize_params:
image: centos-cloud/centos-7-v20200309
network_interface:
network: default
access_config: {}
metadata:
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
service_account:
accountId: "ghcdrupalproject-owners-digita@ghcdrupalprojectdpt.iam.gserviceaccount.com"
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 It worked now.
service_accounts:
- account_id: ghcdrupaldptprojectowners
display_name: "ghcdrupal Service Account"
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
when I ran it by hand
gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan
ServiceException: 401 Anonymous caller does not have storage.objects.get access to ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh.
How to use the service account which will run the below command
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
cat config.yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This sample configuration provides the minimum configuration required by the DPT scripts.
# Audit resources will be created locally in the project.
overall:
organization_id: '7954763295'
billing_account: 89NJCG-KL987Y-LPIU76
domain: mydomain.com
generated_fields_path: ./generated_fields.yaml
generated_fields_path: ./generated_fields.yaml
projects:
- project_id: ghcdrupalprojectdpt
owners_group: [email protected]
auditors_group: [email protected]
audit:
logs_bigquery_dataset:
dataset_id: digitalapicraft_ghcdrupalprojectdpt001_logs # Bigquery Dataset names must use underscores.
location: US
devops:
state_storage_bucket:
name: digitalapicraft-ghcdrupalprojectdpt-state
location: US
service_accounts:
- account_id: ghcdrupaldptprojectowners
display_name: "ghcdrupal Service Account"
compute_instances:
- name: ghcdrupalprojectdpt-instance
zone: us-central1-a
machine_type: n1-standard-1
boot_disk:
initialize_params:
image: centos-cloud/centos-7-v20200309
network_interface:
network: default
access_config: {}
metadata:
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
Thanks in advance.
from healthcare.
kaushalshriyan
commented on July 24, 2024
cat config.yaml
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This sample configuration provides the minimum configuration required by the DPT scripts.
# Audit resources will be created locally in the project.
overall:
organization_id: '7954763295'
billing_account: 89NJCG-KL987Y-LPIU76
domain: mydomain.com
generated_fields_path: ./generated_fields.yaml
projects:
- project_id: ghcdrupalproject
owners_group: [email protected]
auditors_group: [email protected]
audit:
logs_bigquery_dataset:
dataset_id: digitalapicraft_ghcdrupalproject001_logs # Bigquery Dataset names must use underscores.
location: US
devops:
state_storage_bucket:
name: digitalapicraft-ghcdrupalproject-state
location: US
service_accounts:
- account_id: ghcdrupaldptprojectowners
display_name: "ghcdrupal Service Account"
compute_firewalls:
- name: ghcdrupal-firewall
network: default
allow:
protocol: icmp
compute_instances:
- name: ghcdrupalinstance
zone: us-central1-a
machine_type: n1-standard-1
boot_disk:
initialize_params:
image: centos-cloud/centos-7-v20200309
network_interface:
network: default
access_config: {}
metadata:
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcode/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
I am currently facing two issues which are as below:-
Issue No. 1. the service account account_id: ghcdrupaldptprojectowners is reporting ServiceException: 401 Anonymous caller does not have storage.objects.get access. However i have given Storage Admin role to [email protected]
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
gsutil cp gs://ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan
ServiceException: 401 Anonymous caller does not have storage.objects.get access to ghcdrupalcodedpt/installnginxmariadbnewtestdrupalghc.sh.
Issue No. 2. firewall ruleset is not getting created. I have set the below firewall rules to allow http and https in config.yaml file to allow inbound traffic to the new VM instance created using the below command
compute_firewalls:
- name: ghcdrupal-firewall
network: default
allow:
protocol: http
compute_firewalls:
- name: ghcdrupal-firewall
network: default
allow:
protocol: https
It reports the below error
2020/04/15 22:17:31 Failed to apply configs: failed to apply "ghcdrupalproject": failed to apply resources: failed to apply plan: exit status 1:
Error: Error updating Firewall "projects/ghcdrupalproject/global/firewalls/ghcdrupal-firewall": googleapi: Error 400: Invalid value for field 'resource.allowed[0].IPProtocol': 'http'. Invalid IP protocol specification., invalid
on main.tf.json line 156, in resource[5].google_compute_firewall.ghcdrupal-firewall:
156: }
Thanks in Advance.
Best Regards,
Kaushal
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 I have been able to fix the firewall issue by providing the below configuration
compute_firewalls:
- name: ghcdrupal-firewall
network: default
allow:
protocol: "tcp"
ports: ["22","80","443"]
I have not been able to fix this issue
the service account account_id: ghcdrupaldptprojectowners is reporting ServiceException: 401 Anonymous caller does not have storage.objects.get access.
However i have given Storage Admin role to [email protected]
Thanks in advance.
from healthcare.
xingao267
commented on July 24, 2024
You've already created the new service account, but has not yet assign it to the compute_instance, you need to add
compute_instances:
- name: ghcdrupalinstance
zone: us-central1-a
machine_type: n1-standard-1
boot_disk:
initialize_params:
image: centos-cloud/centos-7-v20200309
network_interface:
network: default
access_config: {}
metadata:
startup-script: "cd /home/kaushalshriyan; gsutil cp gs://ghcdrupalcode/installnginxmariadbnewtestdrupalghc.sh /home/kaushalshriyan; sh -xv /home/kaushalshriyan/installnginxmariadbnewtestdrupalghc.sh"
service_account:
email:${google_service_account.ghcdrupaldptprojectowners.email}
scopes:
- cloud-platform
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 Thanks a lot. It worked perfectly.
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 I have a gitlab repository. Does the DPT toolkit support git clone like the below example.
git clone https://gitlab.mydomain.com/sourcecode/appplication.git?
from healthcare.
xingao267
commented on July 24, 2024
@kaushalshriyan how and where would like to execute this git clone command? And is this gitlab repo private?
from healthcare.
kaushalshriyan
commented on July 24, 2024
bazel run cmd/apply:apply -- --config_path=config.yaml --projects=demoproject will execute the below git clone command on the newly spawned VM instance.
git clone https://gitlab.mydomain.com/sourcecode/appplication.git
Yes, the GitLab repo is private.
from healthcare.
xingao267
commented on July 24, 2024
@kaushalshriyan you can do that by include that in the startup script of the VM you created. If it's private, you need additional script in the startup script to first download a github token, which could be stored in a Cloud Storage bucket.
from healthcare.
kaushalshriyan
commented on July 24, 2024
@xingao267 Thanks a lot for the explanation.
from healthcare.
umairidris
commented on July 24, 2024
Closing this as the issue seems to be resolved. I would encourage you to check our early preview of the new version of the toolkit available at https://github.com/GoogleCloudPlatform/healthcare-data-protection-suite.
Thanks!
from healthcare.
Related Issues (20)
- Required IAM permission is not mentioned anywhere HOT 4
- resource not found error on creating immunization with reaction HOT 1
- IAM group can't be assigned owner role HOT 1
- I import image in gcp dicom store, but on my pubsub topic assigned dicom store is not publishing message HOT 1
- deployment of inference module is failing in breast_density_auto_ml.ipynb HOT 4
- Ingesting HL7v2 data from GCS failing HOT 1
- Exposing EHR data via google cloud FHIR api
- Flutter Access HOT 2
- Immunization_demo: based on FHIR stu3
- `mimic_eicu/tutorials/BigQuery_ML.ipynb` notebook needs some changes HOT 1
- Serving healthcare api from a custom domain name
- Security Policy violation Binary Artifacts HOT 119
- Dependency errors building immunisation_demo/frontend. HOT 1
- Update Readme with instructions for contributing/how to contribute HOT 1
- Update license on auditing utility
- FHIR store auditing utility
- Consumption FHIR store to Pubsub and GCS utility
- Ingestion Pubsub to fhir store utility
- Create ReadMe for Healthcare Toolkit
- Error in generate_synthea_dataset.ipynb
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from healthcare.