Description:
The ProcessCarImage function is invoked when a new image appears in the blob storage. The function aims to send a license plate read message to Service Bus. It first invokes the plate recognition service, copies the image blob to an archive folder, and deletes the original blob. If the plate recognition result has a "low" quality, the function invokes a durable orchestrator to wait for human approval for further processing. However, if the function runtime crashes after deleting the original image but before sending the message to Service Bus, when the function is retried, no image will be found for the workflow. Function runtime crashes can occur due to a variety of reasons. Some common causes include: software, hardware, and, network timeout.

Issue:
The ProcessCarImage function does not ensure idempotency when the function runtime untimely crashes after deleting the image but before the function successfully finishes. This may result in the Service Bus message never being sent, even when the function is retried.

Suggested fix:
Adjust the steps of the workflow to make the workflow idempotent:

1. (a) If the original blob exists, invoke the recognition service using the original image. (b) Otherwise, use the archived blob as the input of the service.
2. Archive the blob if the original blob exists.
3. Delete the blob if it exists (use imageBlob.DeleteIfExistsAsync() instead of imageBlob.DeleteAsync()).
4. Invoke an orchestrator if the read quality is low.
5. Send the plate info to Service Bus.

By following this adjusted workflow, we can ensure idempotency and prevent the issue of Service Bus messages not being sent due to function runtime crashes. If a crash happens before step 3, the retry will follow the same steps from 1(a). If a crash happens after step 3 (so the original blob has been deleted), when the function retries, the source of the image will be the previously archived blob (i.e., the workflow starts from 1(b)).

Hi,

Just wondering if you could upload any documents such as the mind map created during this video: https://www.youtube.com/watch?v=yPK3wYKtRss

Thanks!

[ " ", "X", " ",
" ", "X", " ",
" ", "X", " " ]

I send a struct with 3 booleans from C# to C++ with PINVOKE.

There is no example for this.
But you have to add [MarshalAs(UnmanagedType.I1)] to get the right size. Otherwise the values are not correct on the C++ side.

Example:

public struct Bla
{
        public float fooFloat;
        [MarshalAs(UnmanagedType.I1)] public bool foo1;
        [MarshalAs(UnmanagedType.I1)] public bool foo2;
        [MarshalAs(UnmanagedType.I1)] public bool foo3;
}

Can you add a example for that in PInvokeLib?

Description:

I would like to kindly bring attention to a potential issue in the BlobProcessFunctions class, specifically the ProcessCsv function. This function is triggered by a BlobTrigger and processes a CSV file, then sends the result to a Service Bus topic. However, the current implementation does not guarantee idempotency. Suppose the function crashes after successfully processing the CSV file and sending the result to the Service Bus topic (but before acknowledging the event source that the function has finished). In that case, when the function is retried, it will process the CSV file and send the result to the Service Bus topic again, causing duplicate messages.

Suggested Fix:

To avoid duplicate messages in the Service Bus topic due to untimely retries, a deduplication mechanism can be implemented. Specifically, one can attach a unique id to each processed CSV result when it is being generated. The Service Bus topic should be configured to utilize the deduplication feature based on this unique id. By doing so, the Service Bus will automatically prevent the delivery of duplicate messages to the topic.

Additionally, you can consider implementing a check in the receiver's end to verify if the message with the same unique id has already been processed. If the id is already present in a deduplication set holding all previous processed message ids, the receiver should ignore the message. Otherwise, the receiver should process the message as usual.

Thank you for considering this feedback. I hope that this suggestion could help improve the idempotency and reliability of the ProcessCsv function. Please feel free to reach out if you have any questions or concerns.

First of all excellent resource and i really like this repo.

I have a question related to Burn Payload example. Do you know if we can use * wilcard in payload. My requirement is, i have bunch of files ending with *.lic. When i install the exe the *.lic files has to be copied over to the installation location. This way we can control before running what *.lic can exist next to the exe in the same directory so that the files get installed. Do you know if we can do this?

Note: I am able to use CopyFile element in MSI and it works great. When i bundle the same MSI into an exe the *.lic files are not copied. I can think only one reason for this problem, burn caches the installer to a location other than the sourcedirectory and all *.lic files are not available in the cache location so they are not being copied. Not sure how to avoid this problem