aws-samples / aws-lambda-sap-odp-extractor Goto Github PK

Can you kindly lower the default lambda timeout to e.g. 1 min? Especially when dealing with Lambda private connectivity and related troubleshooting the turnaround times are high (especially for beginners it would simplify it).

timeout: cdk.Duration.seconds(300) -> timeout: cdk.Duration.seconds(60)

I have several SAP ODP data sources that I would need to extract. And I want to schedule the Lambda job for doing that for every data source. But this package produces ONE extractor and Lambda function, which is difficult to schedule in production with multiple data source.

Can I create multiple Lambda functions with the extractors with different settings? If yes, then how? If no, is there a way to dynamically input the function parameters while scheduling?\

Please help. Thank you.

Hi @kumarrk21 ,

We are trying to pull data with more than 8 million records in SAP. SAP chunks this data available in several parts of ~50k size each. The ODATA extractor is connecting with the service, pulling the Data, but it is extracting only 1st part (~50k) and does not extract the rest of the 39 parts with 50k records.

Is there anything in the code that we can modify to release this restriction? How can we extract all the parts and the entire data from SAP without facing this issue?

It will be great help if you could kindly reply. Thank you.

@kumarrk21 We have multiple ODP data sources in SAP with same parameters (Entity name, Service name, Host and Port) Even the Username is same. Only difference is the Password. The password in the SAP system differentiates the Login to the respective Clients.

But when I am implementing this in AWS Lambda, the system is working for one client, but as I am changing the password to connect to the other client keeping all other parameters same, it is failing to connect.

Is there any way I can tell my Lambda how to differentiate different client sources with same parameters and different password?

Hello,

I am trying to use ODP on AWS , but when i test the AWS lambda function, i get the following error

{ "statusCode": 200, "body": { "success": false, "message": "Expecting value: line 1 column 1 (char 0)", "traceback": "Traceback (most recent call last):\n File \"/opt/python/extractor.py\", line 69, in extract\n _extract(\" \",True)\n File \"/opt/python/extractor.py\", line 96, in _extract\n sapresponsebody = json.loads(sapresponse.text)\n File \"/var/lang/lib/python3.6/json/__init__.py\", line 354, in loads\n return _default_decoder.decode(s)\n File \"/var/lang/lib/python3.6/json/decoder.py\", line 339, in decode\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n File \"/var/lang/lib/python3.6/json/decoder.py\", line 357, in raw_decode\n raise JSONDecodeError(\"Expecting value\", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n", "data": null, "numberofrecs": 0 } }

Has anyone encountered this before?

@kumarrk21 We are doing a POC to extract data from ECC standard extractor using oDATA service.

Initally we tried for Master datasource( 0MATERIAL_ATTR) and we were able to extract data out of ECC using oDATA service and also delta capabilities worked fine for such extractors.

Now we are planning to extract data for LO datasource like 2LIS_02_HDR.
I understand the scenario for a collective run' and 'setup table filling' when we are extracting the data in BW system using DTP and Infopackage where we first fill the setup table and run the full infopackage/DTP and then initialize the delta pointer by running a delta info package/DTP. By doing this a entry is created in ODQMON/RAS7 in ECC .

I want to know if we extract LO datasource using oDATA service and LO datasource being different from other datasource where we have setup table and Delta queue functionality how will the delta functionality works for LO datasource? If u run my Lambda function for such oDATAservice where in ECC the delta pointer will be maintained.