jgstew / besapi Goto Github PK

There is existing functionality for export_all_sites that can be used to "backup" custom content on a regular basis to GIT.

• https://github.com/jgstew/besapi/blob/master/examples/export_all_sites.py

• besapi/src/besapi/besapi.py

  Line 744 in e0e5058

  def export_all_sites(

It should be enhanced to only export contents if the last_modified time is newer in the origin site than in the already exported file.

The goal is to allow one way "backup" of custom site(s) to GIT, then allow one way ingestion of the same site from GIT.

Related:

• https://stackoverflow.com/questions/4565700/how-to-specify-the-private-ssh-key-to-use-when-executing-shell-command-on-git

When using the BigFix REST API to upload a file to the BigFix Server, you can just send the file to the root and if it already exists with the exact same SHA1 and filename, then it will not create a duplicate file on the remote server.

Before sending the file to the root server, the file could be hashed locally to determine it's SHA1 hash value, then use that value along with the filename in order to determine if the file should be uploaded again, or if an existing copy of the file already exists and that step can be skipped.

Doing this check should save time for any file over a fairly small size, and also have less impact on the root server than sending it a file that it must first hash on the root server itself in order to determine the file's SHA1 hash value in order to determine if the file already exists on the root itself.

Can get the full list of all uploads on the root using: https://bigfix:52311/api/uploads but this could be slow if there are large number of uploads already there.

It should be faster to look for the specific upload in question like this: https://bigfix:52311/api/upload/092bd8ef7b91507bb3848640ef47bb392e7d95b1/test_besapi_upload.txt

The besapi.upload() function should have a parameter added that would be the file's sha1. If the sha1 isn't provided, then the file should be hashed to determine it, otherwise use the provided hash in case we have already hashed the file previously and do not need to do it again, which would be slow for large files.

Use the SHA1 hash of the file, along with it's filename in order to query the root to see if we need to upload the file or not.

The previous behavior of always sending the file each time could be achieved by giving an invalid sha1 value in the parameter, since it would never match a real file, though perhaps it should do that without querying the root first if it knows the sha1 is not valid.

---

Upload RESTAPI reference:

• https://developer.bigfix.com/rest-api/api/upload.html

Approximate location for upload function in the code:

• https://github.com/jgstew/besapi/blob/master/src/besapi/besapi.py#L507

Examples that uses besapi to upload files:

• https://github.com/jgstew/besapi/blob/master/tests/tests.py#L122
• https://github.com/jgstew/jgstew-recipes/blob/main/SharedProcessors/BigFixFileUploader.py