Specifying artifact regististry images with tags fails about python-aiplatform HOT 7 OPEN

I have also similar problem.
In my case, the service account for setting to job.submit needs to have roles/artifactregistry.reader to the target artifact registry for uploading pipeline template.

Anyway in my understanding, the service account called in job.submit is for executing vertex ai but for call pipeline template.

from python-aiplatform.

Any update on this? I seem to have the same issue

from python-aiplatform.

A workaround for anyone with the same issue - first use the KFP SDK to resolve the tag to an exact version, then pass the exact version as template_path:

import re
from kfp.registry import RegistryClient
from google.cloud import aiplatform

_VALID_AR_URL = re.compile(
    r"https://([\w\-]+)-kfp\.pkg\.dev/([\w\-]+)/([\w\-]+)/([\w\-]+)/([\w\-.]+)",
    re.IGNORECASE,
)

template_path = f"https://{region}-kfp.pkg.dev/{PROJECTID}/{REPOSITORY_NAME}/{PIPELINE_NAME}/{TAG}"

match = _VALID_AR_URL.match(template_path)
if match and "sha256:" not in template_path:
    region = match.group(1)
    project = match.group(2)
    repo = match.group(3)
    package_name = match.group(4)
    tag = match.group(5)
    host = f"https://{region}-kfp.pkg.dev/{project}/{repo}"
    client = RegistryClient(host=host)
    metadata = client.get_tag(package_name, tag)
    version = metadata["version"][metadata["version"].find("sha256:") :]
    template_path = f"{host}/{package_name}/{version}"

# Instantiate PipelineJob object
pl = aiplatform.pipeline_jobs.PipelineJob(
    template_path=template_path,
    ...
)

from python-aiplatform.

Thanks for the detailed report! I've filed this as an internal bug and I'll get back to you when I have further updates about this.

from python-aiplatform.

For anyone finding this via google. I encountered a very similar error to this when trying to run a template as a pipeline job.

Turns out that I was not specifying the service account in the job submit call. Adding the service account to that call (i.e job.submit(service_account=pipeline_service_account)) fixed the issue for me.

from python-aiplatform.

BTW downloading the pipeline YAML from Artifact Registry via tag works fine using the KFP SDK registry functions

from python-aiplatform.

Similarly it works fine using a curl request as per the docs

from python-aiplatform.