Packaging & Distribution of Operators about astronomer-providers HOT 11 CLOSED

Did a POC on jfrog private pypi repository.

1.Created a cloud based local jfrog repo, was able to upload a wheel package to the jfrog pypi repo.

phanikv@Phanis-MBP python-helloworld % python3 setup.py bdist_wheel upload -r local
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/helloworld
copying helloworld/init.py -> build/lib/helloworld
copying helloworld/main.py -> build/lib/helloworld
copying helloworld/VERSION.txt -> build/lib/helloworld
installing to build/bdist.macosx-10.14-arm64/wheel
running install
running install_lib
creating build/bdist.macosx-10.14-arm64
creating build/bdist.macosx-10.14-arm64/wheel
creating build/bdist.macosx-10.14-arm64/wheel/helloworld
copying build/lib/helloworld/init.py -> build/bdist.macosx-10.14-arm64/wheel/helloworld
copying build/lib/helloworld/VERSION.txt -> build/bdist.macosx-10.14-arm64/wheel/helloworld
copying build/lib/helloworld/main.py -> build/bdist.macosx-10.14-arm64/wheel/helloworld
running install_egg_info
running egg_info
writing helloworld.egg-info/PKG-INFO
writing dependency_links to helloworld.egg-info/dependency_links.txt
writing entry points to helloworld.egg-info/entry_points.txt
writing top-level names to helloworld.egg-info/top_level.txt
reading manifest file 'helloworld.egg-info/SOURCES.txt'
writing manifest file 'helloworld.egg-info/SOURCES.txt'
Copying helloworld.egg-info to build/bdist.macosx-10.14-arm64/wheel/helloworld-0.1-py3.8.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.macosx-10.14-arm64/wheel/helloworld-0.1.dist-info/WHEEL
creating 'dist/helloworld-0.1-py3-none-any.whl' and adding 'build/bdist.macosx-10.14-arm64/wheel' to it
adding 'helloworld/VERSION.txt'
adding 'helloworld/init.py'
adding 'helloworld/main.py'
adding 'helloworld-0.1.dist-info/LICENSE'
adding 'helloworld-0.1.dist-info/METADATA'
adding 'helloworld-0.1.dist-info/WHEEL'
adding 'helloworld-0.1.dist-info/entry_points.txt'
adding 'helloworld-0.1.dist-info/top_level.txt'
adding 'helloworld-0.1.dist-info/RECORD'
removing build/bdist.macosx-10.14-arm64/wheel
running upload
Submitting dist/helloworld-0.1-py3-none-any.whl to https://myenv1.jfrog.io/artifactory/api/pypi/default-pypi-local
Server response (200): OK

2.It provides download metrics for each package in "List" and "Graph" views.

from astronomer-providers.

@kaxil @phanikumv Please pause on this

from astronomer-providers.

Some more options to consider.

• https://pypicloud.readthedocs.io/en/latest/index.html
• https://testdriven.io/blog/private-pypi/
• https://medium.com/packagr/creating-and-sharing-private-python-packages-151a95e10735

from astronomer-providers.

Some more options to consider

• https://pypicloud.readthedocs.io/en/latest/index.html
• https://testdriven.io/blog/private-pypi/
• https://medium.com/packagr/creating-and-sharing-private-python-packages-151a95e10735

All these does not seem to have SSO (Auth0/Okta) integration or download metrics unfortunately

from astronomer-providers.

There is also support for okta integration, refer to https://jfrog.com/knowledge-base/how-to-configure-an-artifactory-saml-sso-with-okta/

For publishing private docker images, refer to https://www.jfrog.com/confluence/display/JFROG/Docker+Registry

from astronomer-providers.

@ernest-kr had evaluated this earlier and says Jfrog is expensive so we will need to check out the pricing model.

from astronomer-providers.

The Enterprise version costs $699 per month and has below terms.

• 200GB Transfer/month
• 125GB Storage
• 25,000 CI/CD Minutes/month

Do we really need 25,000 CI/CD Minutes ?
There is also a custom quote available, but we need to discuss our requirements with them

from astronomer-providers.

Wow, that's expensive. we definitely don't need 25,000 CI/CD Minutes not 125GB storage or 200GB Transfer.

Let's discuss about this next week and we can reach out to them.

from astronomer-providers.

Rough numbers to atleast start a conversation and get an initial quote with them:

• 10GB Transfer/month
• 10GB Storage
• 500 CI/CD Minutes/month

Questions to ask:

• Can we increase the quota mid-month?
• Do they have a per user quote or just resource-based quotes?

from astronomer-providers.

Let's try out https://cloud.google.com/artifact-registry in the meantime and https://www.sonatype.com/products/pricing too and compare those 3 (including jFrog).

from astronomer-providers.

Reached out to JFrog with our requirement.

---------- Forwarded message ---------
From: Phani Kumar [email protected]
Date: Wed, Jan 5, 2022 at 7:22 PM
To: Anil Patra [email protected]

Hello Anil,
Thanks for reaching out. Happy new year !
Would you mind providing a custom quote for the below requirement?

10GB Transfer/month
10GB Storage
500 CI/CD Minutes/month
Please let me know if you have any questions. Would be happy to get on a call to discuss further

Thanks,
Phani

from astronomer-providers.