fourtheorem / ai-as-a-service Goto Github PK

View Code? Open in Web Editor NEW

55.0 55.0 19.0 2.32 MB

Code for the book "AI as a Service" from Manning Publications

Home Page: https://www.manning.com/books/ai-as-a-service

JavaScript 89.95% Shell 7.98% HTML 2.07%

ai-as-a-service's People

Contributors

Stargazers

Watchers

Forkers

chattg1 univrs arpits9 arpitsharma9 allensmile bodysuperman nirupam1sharma edydkim lutherkhill phoitack jacksadigov wanderingminds xiaobai-fighting yunho0130 bigdatasciencegroup proton0210 unimbekar shawnexample lewisville-flow

ai-as-a-service's Issues

Redirect fails

Hi ,

I need some help with redirect, for me it fails though I get response in browser with jwt token,

Issues with deploying code in Chapter 7

Issue 1:

Started DynamoDB Local, ran the POST curl command to create the seed URL, I got success:

curl -X POST http://localhost:4000/frontier-url/dummy-seed | jq '.'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    95  100    95    0     0    416      0 --:--:-- --:--:-- --:--:--   416
{
  "depth": 0,
  "seed": "dummy-seed",
  "url": "dummy-seed",
  "status": "PENDING",
  "createdAt": 1565494586240
}

But then when I ran the GET curl command to list the URLs for the dummy seed, I got a failure:

curl http://localhost:4000/frontier-url/dummy-seed
Event object failed validation

The process running serverless local shows an error with a long stacktrace with this at the bottom:

message: 'Event object failed validation',
  details: 
   [ { keyword: 'required',
       dataPath: '.queryStringParameters',
       schemaPath: '#/properties/queryStringParameters/required',
       params: [Object],
       message: 'should have required property status' } ]

It talks about a "status" parameter but I don't know what it's talking about based on what I read in the book so far. I examined the code and found its validation schema included a status property with an enum type that looks like it expects the following values:

const STATUS_VALUES = ['PENDING', 'FETCHED', 'FAILED']

I tried the following GET curl command which worked:

curl http://localhost:4000/frontier-url/dummy-seed?status=PENDING
[]

Looks like the book may need to be changed to reflect this parameter that must be provided.

Issue 2:

When running the command sls invoke local -f fetch --path test-events/load-request.json to fun the fetcher locally, it loads the fourTheorem site in my Chrome, but then I get this error in a stack trace:

message: 'The specified bucket is not valid.',
  code: 'InvalidBucketName',
  region: null,
  time: 2019-08-11T04:02:32.932Z,
  requestId: 'D2C6C9E014C3F17F',
  extendedRequestId: 'tSb494uf3oQbz/vYZXVW2iCEVa03JoI/ATWPfBf5o5wnupg2sWe+OgiYLDSrtB1NTl8VB8iOcqg=',
  cfId: undefined,
  statusCode: 400,
  retryable: false,
  retryDelay: 94.45364589611742

I added the following code to handler.js in the storeItems function to debug:

...
log.debug({ keyPrefix }, 'Storing assets')

  console.info('***item store bucket***', itemStoreBucket);

  return Promise.all([
...

and this was that output:

***item store bucket*** #{AWS::AccountId}-dev-item-store

So it looks like the bucket name is not being generated properly when it runs.

Tracing it backwards, I see this code:

const itemStoreBucket = process.env.ITEM_STORE_BUCKET

This tells me it's looking for an env var for the bucket name. And I see this in serverless.yml:

environment:
  STAGE: ${self:provider.stage}
  ITEM_STORE_BUCKET: ${self:custom.itemStoreBucket}

custom:
  itemStoreBucket: '#{AWS::AccountId}-${self:provider.stage}-item-store'

I found a GitHub issue (serverless/serverless#3967) that makes it sound like this is a dead end because it's not supported without adding a library to support it ("psuedo" variable?).

I googled Serverless Framework variables and found this page in the docs (https://serverless.com/framework/docs/providers/aws/guide/variables/) and at the very bottom they include something on "Psuedo Parameters Reference" which includess an example showing AWS::AccountId. I'm not sure what to make of it right now though. This is more Serverless Framework knowledge than I have right now.

But something that crosses my mind right now. I have a bucket called <my_aws_account_id>-dev-item-store as of this step and it looks like the goal of using the account ID in the bucket name is to make it globally unique. Why not instruct the reader to choose a name themselves as an env var in a .env file like the previous chapters did? That approach seems simple and worked so far as I read the book and followed along deploying.

I decided to take a stab at switching to this approach myself. I hard coded a bucket prefix, changing the code in serverless.yml to:

custom:
  itemStoreBucket: 'mattwelke-sandbox-ai-as-a-service-chap7-${self:provider.stage}-item-store'

Before trying the local invoke again, I went back to the item-store directory and changed the bucket name there too, since it would be a dependency. I ran sls deploy there to create the new bucket.

When I go back to the fetch-service and run the local invoke it works now (though it hangs after fetching, needs ctrl-c and the book doesn't mention this). Side note, at this point I realize that oddly enough the AWS::AccountId worked fine for the bucket creating step. Not sure why. It looks like you load the psuedo parameter plugin in both serverless.yml files.

At the end of the chapter I was able to deploy and run the step function, and the book tells me to check the item store. I can tell that the functions ran a few times to get the screenshots so I should see more than one, and the book shows an example of "ai-and-machine-learning" as a page crawled, but I only have one crawled page at this point. It could be related to the bucket issue.

My full bucket path is: Amazon S3/mattwelke-sandbox-ai-as-a-service-chap7-dev-item-store/https%3A%2F%2Ffourtheorem.com/https%3A%2F%2Ffourtheorem.com (/ is the arrow icon at the top) and there are just two files:

page.html
screenshot.png

Issue with the distributor service serverless.yml file

I picked the latest code ( where Chapter 2-3 are combined). I changed the region to us-east-1.
It would be nice if you could parameterize the region, it is not fair to expect everyone to be by default in eu-west-1
I am getting a Time out error:
Serverless Error ---------------------------------------

connect ETIMEDOUT 92.242.140.21:443

error "redirect_mismatch" when clicking "login" in Chapter 3 front end app

I was able to follow along with Chapter 3 to the point where I work with the AWS web console to set up Cognito. The instructions became unclear around the point where you create a user and update the .env file.

The book says "To do this, select
the Users and Groups and click on the Create User button." but stops short of explaining what to do to create the user properly. I used my own judgement and created a user with my personal email address, a temporary password, and turned off the SMS related features. Then I had a user.

The next part of the chapter has us fill in the env vars by checking their values in the web console:

# environment defintion for Chapter 3
TARGET_REGION=eu-west-1 1
CHAPTER3_BUCKET=<your bucket name>
CHAPTER3_DATA_BUCKET=<Your data bucket name>
CHAPTER3_DOMAIN=<your development domain>
CHAPTER3_COGNITO_BASE_DOMAIN=<your cognito domain>
CHAPTER3_COGNITO_DOMAIN=<your cognito domain>.auth.eu-west-1.amazoncognito.com 2
CHAPTER3_POOL_ARN=<your user pool ARN>
CHAPTER3_POOL_ID=<your user pool ID>
CHAPTER3_POOL_CLIENT_ID=<your app integration client ID>
CHAPTER3_IDPOOL=<your identity pool ID>

The web console for CHAPTER3_IDPOOL does not look like the book's screenshots. I found that I had to click on my user pool, and find an "edit" button to get to a screen that looked like this. Additionally, my screen had some warnings about roles that I'm not sure are relevant or not:

When I finished filling out the env vars and deployed the new version of the todo-service and front end app, when I click "login" I get the following error:

I don't think there's anything in the book that helps me learn why this error occurred. Unfortunately, because it's so tutorial oriented without explaining the AWS services we're creating, I don't know enough about the AWS services to debug it.

Issue with downloaded images when running chapter 2 code

I'm in stage 2 of the Manning review right now, and I wanted to rerun chapter 2 to refresh my memory. I'm starting with the code in this repo as of July 28th. I was able to deploy the services for the distributor, crawler, and analysis, and it looks like they're being invoked properly, but the crawler service isn't getting the images properly. The bucket ends up with a few files with UUID names and no extension. Each file is 37.0 B in size. The status.json after the analysis service finishes shows errors coming back from Rekognition because it isn't able to use them because there's no recognized file format.

My status.json after the test message in SQS before deploying the analysis service:

{
    "url": "https://www.fourtheorem.com",
    "stat": "downloaded",
    "downloadResults": [
        {
            "url": "https://user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_630,w_1200,f_auto,q_auto/1683166/fourTheorem-Icon-Colour_ffdaqr.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1887969/336615_754571.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/803591_493588.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/627290_748283.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/991972_577611.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/262334_782130.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1683166/4TheoremHR-206c_atmqag.jpg",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1683166/IMG_2607_s3mgsc.jpg",
            "stat": null
        }
    ]
}

My status.json after deploying the analysis service and sending the second message to the SQS queue:

{
    "url": "https://www.fourtheorem.com",
    "stat": "analyzed",
    "downloadResults": [
        {
            "url": "https://user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_630,w_1200,f_auto,q_auto/1683166/fourTheorem-Icon-Colour_ffdaqr.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_300,w_300,f_auto,q_auto/1683166/464788_60136.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1887969/336615_754571.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/803591_493588.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/627290_748283.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/991972_577611.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1305285/262334_782130.png",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1683166/4TheoremHR-206c_atmqag.jpg",
            "stat": null
        },
        {
            "url": "data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==",
            "stat": "ok"
        },
        {
            "url": "https://www.fourtheorem.com//user-images.strikinglycdn.com/res/hrscywv4p/image/upload/c_limit,fl_lossy,h_1440,w_720,f_auto,q_auto/1683166/IMG_2607_s3mgsc.jpg",
            "stat": null
        }
    ],
    "analysisResults": [
        {
            "image": "www.fourtheorem.com/145e4580-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.559Z",
                "requestId": "f9f3299c-b185-11e9-911a-5f092d135c3f",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 0.6624038143925759
            }
        },
        {
            "image": "www.fourtheorem.com/146152c0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.557Z",
                "requestId": "f9f377be-b185-11e9-b7db-31d96e79290b",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 40.707824820991
            }
        },
        {
            "image": "www.fourtheorem.com/1461a0e0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.560Z",
                "requestId": "f9f37794-b185-11e9-ae21-030afa8ccd80",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 72.60318403685527
            }
        },
        {
            "image": "www.fourtheorem.com/1461c7f0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.579Z",
                "requestId": "fa03081c-b185-11e9-b458-61c9b4f3c044",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 75.50197600936184
            }
        },
        {
            "image": "www.fourtheorem.com/1461ef00-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.578Z",
                "requestId": "fa02e0eb-b185-11e9-99cf-fde9012fdaea",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 89.12434889307994
            }
        },
        {
            "image": "www.fourtheorem.com/14621610-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.559Z",
                "requestId": "f9f6d338-b185-11e9-950c-a56b94f0b5c8",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 7.097429440553493
            }
        },
        {
            "image": "www.fourtheorem.com/146438f0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.578Z",
                "requestId": "fa02e080-b185-11e9-8f5f-c1e9f6e2144c",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 54.803948292451764
            }
        },
        {
            "image": "www.fourtheorem.com/f900b290-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.560Z",
                "requestId": "f9f6fa10-b185-11e9-a239-f7c0997dcd57",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 88.11870915598827
            }
        },
        {
            "image": "www.fourtheorem.com/f90100b0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.560Z",
                "requestId": "f9f6ac27-b185-11e9-92d4-2f7f37e9347d",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 9.06639965443401
            }
        },
        {
            "image": "www.fourtheorem.com/f9014ed0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.581Z",
                "requestId": "fa030802-b185-11e9-8df6-c5fa97e5dca4",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 87.23615918185952
            }
        },
        {
            "image": "www.fourtheorem.com/f902ae60-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.579Z",
                "requestId": "fa0355bb-b185-11e9-996c-17153a389d8b",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 0.9124073598199667
            }
        },
        {
            "image": "www.fourtheorem.com/f902d570-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.579Z",
                "requestId": "fa0307dc-b185-11e9-ba7d-bbb63ed177b0",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 43.74453268373555
            }
        },
        {
            "image": "www.fourtheorem.com/f90609c0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.578Z",
                "requestId": "fa0355ab-b185-11e9-8f76-b17b3ef1cf31",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 20.091979676529604
            }
        },
        {
            "image": "www.fourtheorem.com/f90630d0-b185-11e9-a028-d73a3911036d",
            "labels": [],
            "err": {
                "message": "Request has invalid image format",
                "code": "InvalidImageFormatException",
                "time": "2019-07-28T22:20:59.580Z",
                "requestId": "fa032eed-b185-11e9-85ed-1d87e10b7330",
                "statusCode": 400,
                "retryable": false,
                "retryDelay": 94.550262322752
            }
        }
    ],
    "wordCloudList": []
}

Some system info:

> aws --version
aws-cli/1.16.148 Python/3.6.7 Linux/5.0.0-21-generic botocore/1.12.138
> sls --version
1.48.4 (Enterprise Plugin: 1.3.3, Platform SDK: 2.1.0)
> node --version
v8.16.0
> npm --version
6.10.2

Issues deploying code in Chapter 6

I encountered some issues when reading Chapter 6 and deploying the code.

I decided not to do a pull request for these issues since not all of them have simple fixes and you may want to rearrange names of env vars instead of using the workaround I used to get the region env var in for it to run.

Issue 1:

Right before 6.6. Testing the Pipeline, on page 27 of the current PDF, there is a deploy command. When I ran it I got an error:

Serverless Error ---------------------------------------
 
  An error occurred: TranslateEventSourceMappingKinesisC6ptransstream - Unrecognized event source, must be kinesis, dynamodb stream or sqs. Expected arn of format arn:aws:kinesis:us-east-1:111122223333:stream/my-stream (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: e3a2c3e3-9415-4edf-bdfc-070188f96e47).

The error message included us-east-1 so I thought it might an issue with how the region was substituted.

I replaced AWS_REGION with TARGET_REGION in the .env file we're preparing and then it worked. My .env file included vars for Kinesis that looked like this after this change:

CHAPTER6_PIPELINE_TRANSLATE_STREAM_ARN=arn:aws:kinesis:${TARGET_REGION}:${AWS_ACCOUNT_ID}:stream/${CHAPTER6_PIPELINE_TRANSLATE_STREAM}
CHAPTER6_PIPELINE_SENTIMENT_STREAM_ARN=arn:aws:kinesis:${TARGET_REGION}:${AWS_ACCOUNT_ID}:stream/${CHAPTER6_PIPELINE_SENTIMENT_STREAM}
CHAPTER6_CLASSIFIER_NAME=chap6classifier
CHAPTER6_CLASSIFIER_ARN=arn:aws:comprehend:${TARGET_REGION}:${AWS_ACCOUNT_ID}:document-classifier/${CHAPTER6_CLASSIFIER_NAME}

Issue 2:

I ran streamReader.js where we're instructed to, and I got the following error:

ConfigError: Missing region in config
at Request.VALIDATE_REGION

That was the top of the stacktrace.

I found a solution here: https://stackoverflow.com/questions/47009074/configuration-error-missing-region-in-config-aws

I added this code:

AWS.config.update({
  region: env.TARGET_REGION,
});

in streamReader.js after the requires but before the AWS SDK is called. It worked.

Issue 3:

I had the same problem in train-classifier.js later in the chapter when I ran train.sh. Solved the same way, adding the region override code. This script didn't use the dotenv library so I had to use process.env to access the target region.

AWS.config.update({
  region: process.env.TARGET_REGION
})

I also had to add export TARGET_REGION to train.sh so that it would carry the env var through to the Node script. Then it ran properly.

I had to make this same change to status.sh and check-status.js to be able to run it.

Issue 4:

When deploying the classify service, I got an error:

Serverless Error ---------------------------------------
 
  Invalid variable reference syntax for variable env.CHAPTER6_DATA_ACCESS_ARN. You can only reference env vars, options, & files. You can check our docs for more info.

I examined the different serverless.yml files to compare them and noticed that this variable was being accessed with a period instead of a colon. I changed env. to env: in this service and then the deploy worked.

Issue deploying distributor-service on Chapter 2

I am trying to use the "serverless deploy" in the distributor-service on Chapter 2. I am in Windows 10 and I am using the Region - us-east-1. I get the following error:

Serverless Error ---------------------------------------

An error occurred: IamRoleLambdaExecution - The policy failed legacy parsing (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: 012cafcf-fd87-41bb-82c4-970969900287).

Attached is the serverless.yml file that I am using

serverless.zip

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.