[Feature Request]: Support using local file system as storage about boring-registry HOT 7 OPEN

Hi @skyf0cker ! :) Generally, we welcome a storage backend implementation for the local file system. This is especially true for development and experimentation, but we have some reservations regarding its usage in production environments. What do you think about adding a disclaimer in the documentation, that this storage backend should only be used for development or non-critical scenarios?

Yes, i agree with you.
And i have created a draft pull request and implement the local file system storage backend. I will turn it into a formal pull request after adding necessary unit tests and disclaimer you said above.
In the end, i hope you can assaign this issue to me and help to review the subsequent PR. ;)

from boring-registry.

I generally don't have any objections to adding an HTTP file server, because as you pointed out, it's required to be able to serve the files :) All I'm concerned about is that if the HTTP file server is part of the boring-registry, it shouldn't leak out into the other storage backends but be part of the local storage backend.

For me, there are two acceptable solutions right now:

• Extend the local storage to include HTTP handlers for the download URLs which serve the file
• An HTTP backend which implements all the Terraform-specific logic and uses an HTTP file server. That file server would have to map HTTP paths to files in the filesystem, e.g. something like http.FileServer

I think the first one is fine if it's only used for development, but the second solution might be cleaner

I apologize for the delayed response.

Regarding the two methods I mentioned, upon further consideration, I agree that the second method using an HTTP backend may have unresolved issues. One of the concerns is retrieving information about available versions of the provider, where the registry needs to access the directory structure of a specified location. However, different file servers often have different responses when accessing directories. For example, Golang's FileServer and Node's http-server return HTML, but with different structures. This discrepancy would make it challenging for the registry to uniformly parse the directory structure. If we were to define an interface for the HTTP file server, it would require additional design considerations and customization costs for users who adopt it.

At this point, I agree that the first method I mentioned is a quick and definite solution, and it does not conflict with the second method. By having a local backend that supports local testing, it is still possible to develop a more general HTTP backend. Therefore, I would suggest focusing on enhancing the current local storage implementation by adding a file-serving handler to address this issue.

I hope this aligns with your thoughts.

from boring-registry.

@oliviermichaelis Can you help to review this request? If this proposal allows, i can put into development

from boring-registry.

Hi @skyf0cker ! :)
Generally, we welcome a storage backend implementation for the local file system.
This is especially true for development and experimentation, but we have some reservations regarding its usage in production environments.
What do you think about adding a disclaimer in the documentation, that this storage backend should only be used for development or non-critical scenarios?

from boring-registry.

Hi @oliviermichaelis .
During my testing, I realized that I might have made a mistake. Terraform cannot directly access local files and can only handle download URLs for files available via HTTP or HTTPS. This means that if we want to fetch local provider files using Terraform, we need to have a server to host these files. Furthermore, during the initialization of LocalStorage, we need to configure the server endpoint related settings. Things are getting complicated. Taking into account the implementation of the boring-registry, maybe what we need is not something called LocalStorage, but a more generic Storage Backend, such as an Http Storage Backend. The Http Storage Backend would be configured with an Http File Server. The Registry would handle Terraform requests and return the URL of the Http File Server to Terraform. The registry does not concern itself with how the Http File Server performs storage.
In fact, when I looked at other similar implementations of registries, they integrated the HTTP server module directly into their own registry. This allows the registry to directly support capabilities such as provider upload and download.
Now, I would like to understand your thoughts on Boring-Registry. From your perspective, is the approach of using an Http Storage Backend acceptable to you?

from boring-registry.

I generally don't have any objections to adding an HTTP file server, because as you pointed out, it's required to be able to serve the files :) All I'm concerned about is that if the HTTP file server is part of the boring-registry, it shouldn't leak out into the other storage backends but be part of the local storage backend.

For me, there are two acceptable solutions right now:

• Extend the local storage to include HTTP handlers for the download URLs which serve the file
• An HTTP backend which implements all the Terraform-specific logic and uses an HTTP file server. That file server would have to map HTTP paths to files in the filesystem, e.g. something like http.FileServer

I think the first one is fine if it's only used for development, but the second solution might be cleaner

from boring-registry.

Good point, and I do not have any concerns :) Thanks for your insights!

from boring-registry.