collactionteam / collaction_backend Goto Github PK

Issue: https://github.com/CollActionteam/collaction_backend/runs/4453953747

Potential solution: Explicitly define the dependencies in the template (see aws/serverless-application-model#192 (comment))

The crowdaction IDs in the example event JSON files do not match the pattern of IDs in the table (e.g. sustainability%23food%23veganMonth22-12)

see docs/api2.yaml

⚠ Please notify the frontend team about the changed endpoint in the API when merging.

If the template parameter for FirebaseProjectId is empty, allow unsigned JWTs for easier testing.

see /auth/auth.go

Use Hashcash (or a derivative algorithm) to protect the email contact endpoint against spam.

The following regex pattern should be enforced for the field app_version: ^(?:ios|android) [0-9]+\.[0-9]+\.[0-9]+\+[0-9]+$

Pagination in the API should be implemented as an extension to JSend.
A successful response may include the field nextPage (null, if it does not exist) and should accept the query parameter page with a default value if not present.

Example request:

GET /some/path?page=<PAGE_ID>

Example response:

{
   "status": "success",
   "data": { ... },
   "nextPage": <PAGE_ID>
}

⚠ Please update the API documentation when implementing pagination.
(This ticket should remain open until all "legacy" features have been switched to pagination)

• Apply JSend with schema version checking
• Returned different schema for listing crowdactions
• Private crowdaction password handling
• #24

Replace all uses of APIGatewayProxyRequest with APIGatewayV2HTTPRequest

Encoding issue when fetching values from DynamoDB ("&" becomes "\u0026" in the image URLs e.g. on /crowdactions?status=ended).

(DynamoDB uses UTF-8 which should also be what GoLang uses by default)

Create and document a GNU Make makefile that invokes all frequently used commands as suggested in #90.

• Update MFA credentials (invoke python script)
• Build SAM application
• Deploy developer test stack
• Run tests
• etc?

see docs/api2.yaml

⚠ Please notify the frontend team about the changed endpoint in the API when merging.

Should we use godoc to generate and host code documentation using the CI/CD pipeline?

This is confusing: https://github.com/CollActionteam/collaction_backend/blob/development/docs/api.yml#L101
Change to: "Get participatiON..."

⚠ Blocked until friend or follower feature is defined and planned

• endpoint maps list of (name, phone_number) to list of (name, userID)
• Authentication required
• Prevent scraping using Hashcash (see #58)
• API documentation is updated
• GSI for user profile (phone_number -> userID)
• Privacy: Opt-In/Opt-Out

It may take up to 24h for an updated profile picture to show up in the app because this is the default expiration time for the Cloudfront distribution.
Instead, the cache should be invalidated as soon as the new profile picture was uploaded (see https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html).

https://github.com/CollActionteam/collaction_backend/blob/development/profile-crud/README.md?plain=1#L72

We should filter the fields that we return when looking up a user profile.
It is not necessary to include the user id, since it was already used in the request.
We should definitely not include the phone number, since this may be abused to collect phone numbers of our users.

https://github.com/CollActionteam/collaction_backend/blob/development/email_contact/main.go#L58

We should not return the MessageId, since it is internal information and not a useful API response for the user.
Instead, return a "generic" confirmation as JSON like { "message": "message sent successfully" }.

Tasks:

• Return crowdaction list (see /crowdactions) under /profiles/{userID}/participations
• Return participant list (see field top_participants) under /crowdactions/{crowdactionID}/participations
• Pagination
• API documentation is updated (!)

Event for /whoami seems to be missing.
Create and test in AWS console.

⚠ Blocked by #111

• Claim/award (app API vs CMS API ❓ 👉 TBD ❗) badges for crowdactions
• See badges in profile
• Update API documentation

From Jira CAN-99:

Currently, there is no pagination needed as the list of actions is small but hopefully, there will be a lot of them so pagination will be mandatory in the end
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.Pagination.html

Outline:
New query parameter page: e.g. api-dev.collaction.org/crowdactions?status=ended&page=eyJwayI6ImFjdCIsInNrIjoic3VzdGFpbmFiaWxpdHkjZm9vZCN2ZWdhbk1vbnRoMjAtMTIifQ==

New response:

{
  data: [...], // The list of crowdactions returned by the backend
  next_page: "eyJwayI6ImFjdCIsInNrIjoic3VzdGFpbmFiaWxpdHkjZm9vZCN2ZWdhbk1vbnRoMjAtMTIifQ==" // Next page key as Base64 or null
}

Obtaining value for next_page to return from models/crowdactions.go#listCrowdactions:

1. Variable startFrom is of type utils.PrimaryKey (aka map[string]*dynamodb.AttributeValue)
2. If this variable is null or empty, return null
3. Marshal this variable as a JSON string (avoid optional whitespace characters. E.g. {"pk":"act","sk":"sustainability#food#veganMonth20-12"})
4. Convert JSON string to base64
   Obtain parameter startFrom for models/crowdactions.go#listCrowdactions:
   (See above, but just in reverse)

Shorter page IDs:
Instead of converting the primary key to a JSON file, concatenate pk and sk and obtain the base64 string which should reduce the length of the generated page ID. Here is an example using the same primary key:
JSON → eyJwayI6ImFjdCIsInNrIjoic3VzdGFpbmFiaWxpdHkjZm9vZCN2ZWdhbk1vbnRoMjAtMTIifQ==
pk.sk → YWN0LnN1c3RhaW5hYmlsaXR5I2Zvb2QjdmVnYW5Nb250aDIwLTEy

Use $ref: './part.yaml' as described here.

From Slack:
Thoughts on using a page out of Java's book with the class StringBuilder (and maybe method chaining) to consistently and maintainably implement request handling on the backend?
Example:

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
   handlerChain := new(HandlerChain)
            .append(validateRequestMyHandler) // Check parameters
            .append(authentiate) // (optional) Store user info in ctx
            .append(processRequestMyHandlerPart1)
            .append(processRequestMyHandlerPart2) // There could be multiple steps to processing a request
            .append(processRequestMyHandlerPart3)
   return handlerChain.handle(ctx, req)
}

Every handler only invokes the next one if there is no error, otherwise it returns the error response.
This way we can return an error using shared code without adding unrelated logic to the handlers.

Please discuss! 🙂

To ensure that all features are working, we should perform regression testing on the entire backend:

• Spin up a temporary Cloud Formation stack and tear it down after the test
• Create code to test cases on the live backend over HTTP
• ❓ How to deal with timeouts or partial test fails
• (Optional) Scheduled workflow on GitHub actions

Depends on #56

see docs/api2.yaml

⚠ Please notify the frontend team about the changed endpoint in the API when merging.

Issue: See #48 (May occur elsewhere)
Solution: See #50

Use the cfn-include preprocessor to combine several smaller template files into the final template file to be deployed.
(Similar to #98)

This requires NodeJS.
I we actually end up using this, we might want to look into using NodeJS/NPM for other things as well, such as porting scripts to NodeJS and also using it for other automation tasks/tools (see #91)

RegEx: ^(?:ios|android) [0-9]+\\.[0-9]+\\.[0-9]+\\+[0-9]+$
(see: https://github.com/CollActionteam/collaction_backend/blob/development/internal/models/contact.go#L8)

Also: Add line break between message and app version.

• Investigate the option of using localstack for development instead of deploying to AWS for testing.
• Create setup documentation
• (Create utility scripts for convenience)

Some crowdactions should require/offer periodic check-in by participants to continuously track/confirm their participation.
This requires some changes to the API, backend logic and internal state of the participations.

Changes:

• Track "streak" of participation
• Allow check-in to "extend streak"
• Reset streak if check in was missed (should the maximum streak still be stored?)
• Update API docs

• Consolidate readme files
• Document project structure in readme (see Confluence)

The key of the cache used to store the workflow build artifact does not contain the commit hash which may lead to the wrong build being used in subsequent jobs.

• Switch to On-Demand provisioning model (PAY_PER_REQUEST) for DynamoDB (see docs)
• Investigate the possibility to switch this on for all stacks except "prod" using the template.yaml file.

After a user joins a crowdaction, they may see a "pop-up" with additional steps to take in order to improve retention and increase engagement. (see design)

The content is specific to each crowdaction and is stored (DB field post_join_call_to_action containing JSON string).
The following example shows the required fields and implied schema:

{
   "title": "Where to start?",
   "subtitle": "Need help hetting started? Here is an idea for your first groery list, let's go shopping!"
   "action_items": {
      "is_numbered": true,
      "items": ["Garlic", "Rosemary", "Thyme"]
   },
   "call_to_action_subtitle": "Need more ideas? Feel free to join our WhatsApp group and ask around, we would love to help you!",
   "button_dismiss": "Ready to get started",
   "button_action": {
      "text": "Join us on WhatsApp",
      "link": "https://chat.whatsapp.com/..."
   }
}

Steps:

• Crowdaction schema in database is updated
• The schema is validated when creating/editing a crowdaction.
• Documentation in Confluence is updated
• API documentation is updated

There should only be one place in which the API is documented.
Delete individual README.md files for functions and update docs/api.yml.

⚠ Confirm with design and frontend if the specification for this feature is ok!

Tasks:

• Comment threads are fetched by crowdaction(id) (one thread per crowdaction)
• Comments can be deleted* by the author (return placeholder for deleted comment in thread)
• Comments can be edited (?) (indicate with additional boolean field in response item)
• Pagination
• API documentation is updated (!)
• Spam protection e.g. rate limit (How would we implement this? Investigate!)

*Implement using flag, so comments can be "un-deleted".

• Utility function to generate HTTP response for auth error
• Utility function to generate HTTP response for internal server error
• Use utility functions for errors in every handler

Tasks:

• /crowdactions/{crowdactionID} only returns if the correct password is provided in the request (only if password not empty)
• review fields returned by /crowdactions (e.g. commitment options should not be returned)
• Only /crowdactions?status=featured should return the top_participants

see docs/api2.yaml

⚠ Please notify the frontend team about the changed endpoint in the API when merging.

• Remove deprecate endpoints (/hello, /whoami)
• Remove corresponding event JSON files
• Update API documentation

Bug:

Error: Failed to create changeset for the stack: dev-some-developer, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Template format error: Unresolved resource dependencies [StaticContentDistribution] in the Resources block of the template

How to reproduce it:
Follow the instructions for deploying a developer stack here.

Issue:
StaticContentDistribution is not defined when deploying without the parameter DomainParameter.

Solution:
Conditionally set value of the environment variable CLOUDFRONT_DISTRIBUTION using:
!If [shouldUseCustomDomainNames, !Ref StaticContentDistribution, ""] (ternary operator).
A condition to ignore empty values for this environment variable parameter already exists, so no further action should be required. (Nonetheless the corresponding feature should be tested after this change)

https://github.com/CollActionteam/collaction_backend/blob/development/models/crowdaction.go#L36
The name of the JSON-field should be subcategory, not sub_category.

(see API and CollActionteam/collaction_app#91 (comment))

Currently, date values are stored as serialized strings.
This has a few disadvantages:

• Off-by-one errors are not insignificant (error of 24h)
• Events occur only in one timezone at the "correct" time (Midnight in Frankfurt is not midnight in other parts of the world)
• (Parsing date strings may fail, if different formats are used)

To mitigate this, we should switch to using unix timestamps instead.
Things this approach would not solve:

• A deadline that is set for midnight in Europe would still be at a different time of day in other parts of the world
  (However this is also not desired behavior, because it would give more or less time depending on the location of the user. Instead, we should allow for higher precision accurate to at least one hour)

❗ This is merely a suggestion that has pros and cons. Feel free to discuss 🤔💬

Steps

• Date/time variables are stored as unix timestamps
• Business logic evaluates and compares unix timestamps
• API is updated (coordinate with frontend)

see /contact in docs/api2.yaml

• Successful response matches API spec
• Error response matches API spec
• Request is validated according to API spec (nonce refers to feature #58 and should not be a required field for now, but it should be allowed)
• Header X-CollActionAPI-Version is optional in request and loaded, but not further processed (maybe leave todo in the code)