Integrate DLF into ODH JupyterHub about datashim HOT 9 CLOSED

Thanks for moving this forward @vpavlin !
For the read-only, a simple solution would be to extend the CRD to something like this:

apiVersion: com.ie.ibm.hpsys/v1alpha1
kind: Dataset
metadata:
  name: your-dataset
spec:
  local:
    type: "COS"
    accessKeyID: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    secretAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    endpoint: "https://s3.eu.cloud-object-storage.appdomain.cloud"
    bucket: "kubeflow-yiannis"
    region: "" #it can be empty
    readonly: true

Regarding the labeling etc, the current way to mount in /mnt/datasets/* is as follows:

  labels:
    dataset.0.id: "your-dataset"
    dataset.0.useas: "mount"

Do you need a different functionality about the pods labeling convention?

from datashim.

@YiannisGkoufas The labels should be fine, I would suggest to move the readonly up to spec.readonly instead of spec.local.readonly - I believe the readonly option applies to all the backends so it makes more sense to make it more "global" on dataset level. WDYT?

from datashim.

Oh yes the local bit is going away in any case. Will work on this and update the current issue. Thanks @vpavlin

from datashim.

Is there any progress on readonly access?

from datashim.

Hey @vpavlin yes yes, not documented yet, but if you are feeling brave, try the following images for the dataset operator and csi-s3.

yiannisgkoufas/dataset-operator:readonly-amd64
yiannisgkoufas/csi-s3:readonly-amd64

Example readonly dataset (only working on COS)

apiVersion: com.ie.ibm.hpsys/v1alpha1
kind: Dataset
metadata:
  name: your-dataset
spec:
  local:
    type: "COS"
    accessKeyID: "xxxx"
    secretAccessKey: "xxxxxx"
    endpoint: "http://s3.eu.cloud-object-storage.appdomain.cloud"
    bucket: "kubeflow-yiannis"
    readonly: "true"
    region: "" #it can be empty

edit: removing the local bit is part of a bigger effort to restructure the CRD, but its on the pipeline

from datashim.

Hi @vpavlin can you check the branch https://github.com/IBM/dataset-lifecycle-framework/tree/readonly it support the readonly functionality in a Dataset as shown in the example

from datashim.

Hi @YiannisGkoufas we are pushing a release of ODH and trying to finish some changes in JupyterHub, which is preventing me from looking at this. Just wanted to let you know this is still on the list for me!

from datashim.

No worries @vpavlin ! thanks for letting me know. I have actually gathered some questions/ideas on ODH integration so would love discuss more when you have time

from datashim.

Closing this issue as this is stale

from datashim.