aws-samples / amazon-sagemaker-asynchronous-inference-computer-vision Goto Github PK

I have a case where the output is not just metadata in json but transformed image/audio, I am uploading the transformed media file to s3 and returning the s3 location via flask app, but what I receive on invoking endpoint is s3 location of the metadata file containing s3 location of the media file. Is there a simpler way to accomplish this?

Thanks for the great and informative tutorial.
I followed your tutorial step by step but i got in this error after executing these lines code.

print(model_name) endpoint_config_name = f"DetectionEndpointConfig-{strftime('%Y-%m-%d-%H-%M-%S', gmtime())}" create_endpoint_config_response = sm_client.create_endpoint_config( EndpointConfigName=endpoint_config_name, ProductionVariants=[ { "VariantName": "variant1", "ModelName": model_name, "InstanceType": "ml.g4dn.xlarge", "InitialInstanceCount": 1 } ] ) print(f"Created EndpointConfig: {create_endpoint_config_response['EndpointConfigArn']}")

endpoint_name = f"sm-{strftime('%Y-%m-%d-%H-%M-%S', gmtime())}" response = sm_client.create_endpoint( EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name, )

waiter = boto3.client('sagemaker').get_waiter('endpoint_in_service') print("Waiting for endpoint to create...") waiter.wait(EndpointName=endpoint_name)

My end point is not creating and it is in waiting state for more then 30 mins.
It output cloudwatch logs:

2023-07-23T15:05:42.086-07:00 | Please specify --force/-f option to overwrite the model archive output file. 2023-07-23T15:05:42.086-07:00CopySee -h/--help for more details. | See -h/--help for more details. 2023-07-23T15:05:42.087-07:00 | ERROR - /.sagemaker/ts/models/model.mar already exists. 2023-07-23T15:05:42.087-07:00 | Please specify --force/-f option to overwrite the model archive output file. 2023-07-23T15:05:42.087-07:00 | See -h/--help for more details.

is it possible to do a tuning on the internal queue? is there any visibility timeout or something like that?

If I'm using BYOC framework with async endpoint, should the /invocations receive the s3 address or the image/video file? Should I have to include functions for downloading image/video from s3 or uploading the output? What should I return from /invocations?