bbrcan / zm2drive Goto Github PK

This code:

export async function listFiles(mimeType, folderId, auth) {
  const drive = google.drive({version: 'v3', auth})
  const response = await drive.files.list({
    pageSize: 1000,
    q: (`mimeType = '${mimeType}'`
      + ' and trashed = false '
      + `and '${folderId}' in parents`),
    fields: `files(id, name, md5Checksum)`,
  })
  return response.data.files
}

only returns 1000 files. It would be necessary to use a page token and get multiple pages to return more.

If there are many zip files to create, and /tmp is of limited size, it may be exceeded and the uploads will fail. Many distributions put /tmp into memory for performance reasons, but then it is limited by the allocated size. A reasonable workaround is to set TMPDIR to point to a location where there is sufficient space, for example:
export TMPDIR=/var/tmp
has been added to the shell script.

files.list returns 100 files per page by default. I added

pageSize: 1000,

and it returns 1000 max, better but still may be exceeded.

Google drive has a limit of 10 queries per second per IP address. This can be exceeded by deleting many files at once. I changed the delete code to be:

if (args.deleteOld) {

  // delete files that exist on Drive, but not locally

  const zipFileNames = zipFiles.map(f => path.basename(f))

  //added
  const util = require('util');
  const sleep = util.promisify(setTimeout);
  //

  const filesToDelete = existingFiles
    .filter(f => !zipFileNames.includes(f.name))

  console.log(`Deleting ${filesToDelete.length} existing files.`)

  // Google drive has a quota of 10 transactions per second per IP address.
  // Wait 200 ms between deletes so a maximum of 5 delete transactions occur per second.
  filesToDelete.forEach(f => drive.deleteFile(f.id, auth), await sleep(200))
}

and it seems to work.

Dates do not seem to work for me.

Entering dates in various formats does not seem to find any events.

Example:

--fromDate 2018-10-01T00:00:00 --toDate 2018-10-04T23:59:59

finds no events on 4 October 2018. Entering no dates finds and correctly uploads all events.