Should we add a Dockerfile for development and testing? about classifier-reborn HOT 49 CLOSED

@envygeeks any progress on this? Is there anything I can help you with?

from classifier-reborn.

This could be useful. It's a bit of a departure from the usual development workflow on the gem, what do you think @parkr?

from classifier-reborn.

I would note here that I am using Ruby 2.3 because gsl gem won't install in Ruby 2.4 due to the Fixnum -> Integer change. They have a merged PR (SciRuby/rb-gsl#42) in the repo, but the gem is not released with the fix yet.

from classifier-reborn.

@Ch4s3: It's a bit of a departure from the usual development workflow on the gem

Are there anything specific that you have in your mind which is not directly available in this workflow? We can perhaps change the default command to watch for file changes and run tests automatically, if that is something desired as the default behavior.

On the other hand, I think it can be very useful if the Dockerfile is added in the repo and a corresponding image is published at jekyll/classifier-reborn via automatic builds. This will make the development environment portable and all the tagged releases will be preserved in the DockerHub for historical purposes when someone needs them for debugging.

from classifier-reborn.

Interesting. Yeah, nothing is missing I just wanted to have parker weigh in on this.

from classifier-reborn.

I think you should use whatever tools make it easiest to develop. If you're constantly rebuilding images, then it might be too much friction. If you build the image once and it provides a nice base for cross-platform development, then go for it. Do whatever makes development for you and our contributors easiest! Whatever you choose, don't forget to document it. 😄

from classifier-reborn.

Alright then, I'll call that consensus! Let's make the docker image available!

from classifier-reborn.

Alright then, I'll call that consensus! Let's make the docker image available!

You asked for #104. Now, go ahead and setup automated builds (if you have write access to https://hub.docker.com/u/jekyll/) with two types, one for the Branch and the other for the Tag. Leave both types with default configurations.

from classifier-reborn.

Now that PR #104 is merged, this issue can be closed after adding the automated builds on DockerHub. Without the availability of the jekyll/classifier-reborn image the documentation will not be valid.

from classifier-reborn.

Unfortunately I don't have write access there. @parkr do you have access?

from classifier-reborn.

Unfortunately I don't have write access there. @parkr do you have access?

I that is not an individual's account, instead an organization then ideally there should be more than one team members added for ongoing maintenance and to avoid locking out if the owner of the organization is not accessible.

from classifier-reborn.

PR #106 improves the documentation wordings while it assumes no reliance on the automated build located at jekyll/classifier-reborn. However, this will still be a good idea to setup an automated build to create a history of all tagged releases in DockerHub.

from classifier-reborn.

@ibnesayeed anything else you'd like to do on this?

from classifier-reborn.

@Ch4s3: @ibnesayeed anything else you'd like to do on this?

Yes, the automated build please. I know you don't have access to the jekyll DockerHub account/organization, but whoever has it should add the automated build corresponding to this. Although the current documentation does not rely on it, but it will be really nice to have a history of all the released versions automatically being preserved on DockerHub.

from classifier-reborn.

@parkr know anything about the jekyll DockerHub account/organization?

from classifier-reborn.

I'm not entirely sure, but my hunch is that we set that up to support jekyll/docker.

from classifier-reborn.

@parkr: ... but my hunch is that we set that up to support jekyll/docker.

That sounds right based on the linkage of the repo and available images in the hub. By looking at the repo history, it looks like @envygeeks might have some idea about who has credentials.

On a side note, it seems like the jekyll docker image is not updated in ages. The last time it was updated in the hub was a year ago and no change has happened in the corresponding GitHub repo in last five months. Although the jekyll docker image has got 110 stars and 100K+ pulls. I do believe that it should be actively maintained. It is worth making an official docker image in my opinion.

from classifier-reborn.

Jekyll Docker doesn't automate builds through Dockerhub because we use Docker-Template to build complex images that are templated and it is actively maintained, while I can see that you would assume that it's not because Docker's website is generally terrible at showing proper statistics, I can assure you that every time a commit is made to jekyll/docker by an authorized team member it does update the images, that should be clear from the "build passing" through Travis-CI which is fully transparent in both the build process and the push process.

What are you needing? I do have my own credentials (of which I will not be sharing) and admin on our Docker but so do 2 other team members (@parkr included) and I do have the credentials for the bot but that's specifically for jekyll/docker, I would have to adjust it's permissions and commit those outside of any commit you make in encrypted form. If you need a repository I can create it if @parkr doesn't remember his credentials and setup the automated build if you want Dockerhub to do it.

from classifier-reborn.

Thanks @envygeeks for the explanation. I was certainly not aware of the workflow and assumed the standard mechanism used for automatic builds, especially because I guess in the past, that is how it was done as apparent from the build history.

For this repo, we don't want any custom method. It is a simple Dockerfile that does not need any complex structure, so standard automated build setup will do just fine (and preferred). We just want an automated build setup as jekyll/classifier-reborn, connected to this repo. Just make sure that you set up both the Branch and the Tag types for triggers. Leave both types with default configurations.

from classifier-reborn.

Is @parkr and @mattr- fine with this?

from classifier-reborn.

More importantly than that is @Ch4s3 fine with maintaining the image?

from classifier-reborn.

As per @Ch4s3 above:

Alright then, I'll call that consensus! Let's make the docker image available!

from classifier-reborn.

Well, once @parkr or @mattr- give the go ahead I'll create it for ya'll unless he wants to.

from classifier-reborn.

If you read messages above, @parkr is not against it. However, I haven't seen @mattr- commenting in this matter.

from classifier-reborn.

I need explicit permission directly from @parkr regardless, sorry.

from classifier-reborn.

That's perfectly fine. Please follow your protocol as set by the organization. I am not in hurry and this is not stopping me from doing anything. I just thought it would be a good to have thing for the reasons described above.

from classifier-reborn.

👍 from me!

from classifier-reborn.

@envygeeks any news?

from classifier-reborn.

I'll get it setup tonight, it's on my todo for today after I get some other high-priority stuff done.

from classifier-reborn.

thanks

from classifier-reborn.

That reminds me, @parkr want's this put into the Docker repository, can you ship the pull there that way we can have that build server handle it? You can add a simple Dockerfile with an opts.yml file and (if you want even base it off of our Jekyll) image.

from classifier-reborn.

@envygeeks, Docker images should be as self-contained with the least dependencies as possible. Basing it on Jekyll image will bring nothing except complications and unnecessarily increased image size. Additionally, adding another intermediate build server will introduce yet another layer of indirection which will cause yet another point of failure and should be avoided unless it has some real advantages.

In my opinion, a simple automated build configuration under jekyll's account on DockerHub with the two configurations (one for branch and one for tag) is all you need here.

from classifier-reborn.

Parker made the decision and superficial arguments and invalid claims will not change that unless he makes the decision. And if it's going into my repository it will follow the laws of the land, like Europe, doesn't matter what country you are in, the laws of that country apply. Like the United States, doesn't matter what state you are from, the laws of the state you are in apply.

from classifier-reborn.

Sir @envygeeks, you made it very interesting. I thought I was contributing to the project not just by writing code, tests, and introducing new capabilities, but also by sharing my opinion and thoughts via healthy comments. However, you sound like I am stepping into your and/or Jekyll's land to impose my "superficial arguments and invalid claims". That said, now, for the sake of gaining knowledge and insights, I am really curious to learn which of my arguments were "superficial" and claims "invalid" and how/why? Which, you are in no way forced to tell, if the law of "your land" does not permit, but I would really appreciate if you do enlighten me. Please consider that as my humble request.

As far as the decision of making the image available and the way to build it is concerned, please enjoy the law of your land. It was a request that was made 33 days ago, agreed on by @parkr and @Ch4s3 33 days ago, implemented and merged 32 days ago, given 👍 explicitly by @parkr 28 days ago, and still dealing with some efficient management and logistic jargon. I am not aware of @parkr's decisions beyond what he said here in this conversation, so I can't speak about them.

from classifier-reborn.

You are in no way being constructive at this point, our communications are being disconnected, I wish you the best but I will not be assisting you further, another member of Jekyll can deal with it from here on out.

from classifier-reborn.

I guess I run a tighter ship than I thought! I didn't mean for my decision to cause any sort of miscommunication or escalation of tensions here. My decisions are not like laws, and I am more than happy to listen to counter arguments. Ultimately, we should do whatever is best for the project.

My suggestion was only to keep everything in the jekyll/docker repo because there is already tooling setup there to generate images. If you think adding a Dockerfile or two for this repository, perhaps one for development and one for production, then I'd be happy to see that. If you can get the automated process setup, I am happy to help however I can. Is this something Docker Hub builds?

from classifier-reborn.

Oh man... I'm totally cool with whatever everyone else wants to do here. If we can set up something sensible under jekyll/docker that's great, but if we need something more specific I'm open to that as well. Do you have further input @ibnesayeed and @parkr?

from classifier-reborn.

@parkr: I guess I run a tighter ship than I thought! I didn't mean for my decision to cause any sort of miscommunication or escalation of tensions here. My decisions are not like laws, and I am more than happy to listen to counter arguments.

Such miscommunication is inevitable when managing a team internally while also dealing with an open community at large. It became frustrating to me that for last several weeks we are saying similar things repeatedly and in return getting arguments with no action plan. I would be totally cool with whatever decisions are made (even if it is to abandon the feature), but it should not go into void, there should be some actionable plans put forth, be it for the internal team or for the community at large.

@parkr: Ultimately, we should do whatever is best for the project.

This is precisely what I was doing and still trying to do.

@parkr: My suggestion was only to keep everything in the jekyll/docker repo because there is already tooling setup there to generate images. If you think adding a Dockerfile or two for this repository, perhaps one for development and one for production, then I'd be happy to see that.

It is perfectly fine to keep the Dockerfile in a separate repository then use the automated build or custom build server to push the built images to the hub. I am not aware of how Jekyll's build server gets triggered and what all it does that is not possible by vanilla automated build. There are a few reasons however, why one might choose not to use the automated builds feature of the DockerHub such as, 1) the build process needs some special things that can't be done within Dockerfile or limited by the DockerHub and 2) one does not want to link the GitHub account with the DockerHub account or setup a GitHub webhook.

In my opinion, separate build servers might be useful for production images, but in this case our focus is to have a development image. I don't even see a good reason to provide a production image for this project as it is not a stand-alone software (yet). It is only a library that will be used as a dependency in other projects. And if someone wants to use docker for their project, one would go for some other base images and install it as a gem (note that Docker does not allow multiple inheritance or mix-in when writing a Dockerfile). Keeping this Dockerfile in this repo will not only enable automated builds for bleeding edge version based on the master branch, but also an archived version of each new release. Additionally, it will allow developers to build their images locally from within this repo if they prefer not to fetch from the DockerHub or when they make some changes in the dependency that requires changes in the Dockerfile (that would otherwise require making change in the other repo).

Having said that, I am still fine if there are good reasons to keep it with other Jekyll images. In my opinion, this image does not have to be based on (i.e., inherited from) the Jekyll image as it is not dependent on Jekyll. In fact the dependency is the other way.

@parkr: If you can get the automated process setup, I am happy to help however I can. Is this something Docker Hub builds?

For me or anyone else to be able to setup an automated build for this, there are two requirements, 1) have write access to this GitHub repository and 2) included as the member of the jekyll organization on DockerHub (my id is the same in both the places). These permissions can be granted temporarily and revoked once the automated builds are set up (which will hardly take a couple minutes).

from classifier-reborn.

@Ch4s3: Do you have further input @ibnesayeed

I am totally fine if it is added under jekyll/docker if that helps better manage images, except, I am not aware of the build process of that repo and what all it does, so can't speak much about that. As far as my thoughts are concerned, I have just elaborated them in the previous comment. Again, I totally understand that there could be other good reasons from the organizational or management perspective to do it differently form what is proposed here.

from classifier-reborn.

I am totally fine if it is added under jekyll/docker if that helps better manage images, except, I am not aware of the build process of that repo and what all it does

For somebody who claims to know nothing about jekyll/docker you sure did make a lot of assertions about it and it's image earlier and then got cheeky when I called that out. The irony is thick here.

from classifier-reborn.

@envygeeks: For somebody who claims to know nothing about jekyll/docker you sure did make a lot of assertions about it and it's image earlier and then got cheeky when I called that out. The irony is thick here.

I am really sorry for being sarcastic and getting cheeky, and I mean it. I hope you will forgive me for that and help the project in the best way possible. Thanks!

from classifier-reborn.

If rolling it into jekyll/docker simplifies things, then let's do that for now.

from classifier-reborn.

@Ch4s3: If rolling it into jekyll/docker simplifies things, then let's do that for now.

My suggestion would be to keep this Dockerfile in this repo that would make it easier to build images locally while developing. Notice that this Dockerfile is tailored towards development environment and documented accordingly. I don't see any usefulness of a production oriented image just yet, until we start shipping a binary with CLI or server implementations as described in #99. If image builds based on jekyll/docker repo can be triggered automatically every time something is pushed or released in this repo then that would do the trick. Please take necessary steps, as I have very little interest in learning another workflow of building Docker images that is project specific and not practiced widely in the Docker community.

from classifier-reborn.

For a development image, add a Dockerfile to this repository. I think we misunderstood the intent. Production images should go to jekyll/docker.

As for automatic building, if it's for development, let's keep the process super simple and strike the automatic building. Instead, provide a script/build or script/docker script which builds the image and runs a container with a common entrypoint like bundle exec pry. No need to bother pushing out a development image when you'll need the repo and building the image from scratch won't take longer than a minute or two locally.

from classifier-reborn.

@parkr: For a development image, add a Dockerfile to this repository.

That is already there. We have added it a while ago.

@parkr: I think we misunderstood the intent. Production images should go to jekyll/docker.

The title of this request talks about development and testing environments. As long as it remain a library and not a stand-alone application, what would a production image even be or do?

@parkr: As for automatic building, if it's for development, let's keep the process super simple and strike the automatic building.

I was advocating automated builds because it would bring more visibility to the project and more importantly it will have an archive of the development environment for each release for easier debugging and replication of issues on older versions as the project moves forward. I am coming from web archiving background, so perhaps it is my obsession of preservation, but if it causes management burden then we can strike it off. I would note though, the replication is still possible by checking out a specific commit and building the image then as long as Docker build process itself does not change over time.

@parkr: Instead, provide a script/build or script/docker script which builds the image and runs a container with a common entrypoint like bundle exec pry.

I personally don't see a need for such scripts as they will fail if Docker is not installed. I have documented how to use Docker for development and documentation. Please have a look at the documentation, especially on sections named "Development Using Docker" and "Documentation" and provide feedback to make it better. For now the default entrypoint is default rake task, but how to access bash or pry inside the container is also documented.

I think we can safely close this issue now.

from classifier-reborn.

If everyone is satisfied, I'm closing this for now.

from classifier-reborn.

I was advocating automated builds because it would bring more visibility to the project and more importantly it will have an archive of the development environment for each release for easier debugging

Just wanted to chime in and say this is done through Git already, all Jekyll projects git tag releases.

from classifier-reborn.

@envygeeks: Just wanted to chime in and say this is done through Git already, all Jekyll projects git tag releases.

That's roughly what I meant when I said:

@ibnesayeed: I would note though, the replication is still possible by checking out a specific commit and building the image then as long as Docker build process itself does not change over time.

from classifier-reborn.

Ah yes, I should have kept reading, I tend to quote early.

from classifier-reborn.