generate_new_metadata in tidy_backup_clean.rb needs to preserve metadata.json about knife-tidy HOT 2 OPEN

I think the only thing that should happen here is:

1. if metadata.rb and metadata.json both exist, do nothing at all.
2. if metadata.json only exists, try to construct a simplified metadata.rb
3. if metadata.rb only exists, generate metadata.json using the knife cookbook metadata approach most likely (this should never happen, don't worry about error conditions, just let it fail hard if it doesn't work)
4. if metadata.rb and metadata.json both don't exist, hard fail (this should never ever happen)

Most of the logic around hacking up metadata.rb in knife tidy should be eliminated from the codebase.

from knife-tidy.

Okay so I now have a bit more complete information on what is going on.

The problem is that this runs on a checkout of the filesystem from "knife ec backup".

For old chef servers there may have been no enforcement of the metadata.json file being uploaded to the manifest of the cookbook. All the information is accessible from the API, but knife ec backup will only download the filesystem contents. Since it is throwing away necessary data, this seems to violate the principle that it should be a "backup" tool.

So what needs to happen is that knife ec backup should GET <org>/cookbooks/NAME/version and dump the metadata field in the response to metadata.json and use that as the source of truth of the metadata.json of the cookbook instead of the filesystem
contents.

If knife download/upload aka chef_fs need to be changed at the same time that would probably be a good way to implement this.

Once that is done THEN we can assume in knife tidy that cookbooks MUST always have a valid metadata.json (and can raise and suggest the user upgrade their knife-ec-backup gem if the cookbook does not).

from knife-tidy.