dittofeed / dittofeed Goto Github PK

Amend the /journeys/stats endpoint to include the proportion of users that traverse along a given path within a journey.

Render these proportions as percentages on Journeys.

These percentages can be calculated by querying the user_events_v2 clickhouse table for DFJourneyNodeProcessed events which are tracked here.

_{From SyncLinear.com | DF-316}

Add "test" sms and email providers to the /dashboard/settings provider selectors. These are already implemented on the backend and should only require frontend changes.

See the test provider types:

What makes the test providers useful is that they:

1. don't require contacting an external service
2. don't require additional credentials

This enables local developers to more easily test journeys, broadcasts, and deliveries involving SMS, as well as users of the application in production.

_{From SyncLinear.com | DF-445}

I deployed dittofeed with render and connected it to send track events from our segment account. As events started to arrive, loading events page always results in a frontend crash.

Browser console reports this:

TypeError: Cannot read properties of null (reading 'journeyId')
    at G (events-eadfafb0b5ef748e.js:1:3782)
    at Object.renderCell (events-eadfafb0b5ef748e.js:1:4648)
    at 7271-611cc4c3d29c6cc6.js:63:24093
    at 7271-611cc4c3d29c6cc6.js:63:26369
    at ab (framework-f29e48ae95cae5a3.js:9:60916)
    at uo (framework-f29e48ae95cae5a3.js:9:71227)
    at i (framework-f29e48ae95cae5a3.js:9:121559)
    at oO (framework-f29e48ae95cae5a3.js:9:99113)
    at framework-f29e48ae95cae5a3.js:9:98980
    at oF (framework-f29e48ae95cae5a3.js:9:98987)

Looking at some relevant sections of the stacktrace tells me the original is the 3rd line here

G = e=>{
                var t, n;
                let r = e.journeyId || ""
                  , i = e.nodeId || "";
                if ("broadcast-message" === i) {
                    let e = z(r);
                    return e
                }
                let s = e.templateId || ""
                  , a = A.find(e=>e.id === s)
                  , o = null !== (t = null == a ? void 0 : a.type) && void 0 !== t ? t : null
                  , u = null !== (n = null == a ? void 0 : a.name) && void 0 !== n ? n : null
                  , c = W(r, s, u, o);
                return c

After a few thousand events, the events page started to load fine, but when i search for something it crashes.

Implement support for https://resend.com/ as an email service provider.

Add a new PersistedEmailProvider type:

Add new EmailServiceProviderSuccess type:

https://github.com/dittofeed/dittofeed/blob/01a6db73e237897b7e961c07969ea5d60a8bcbd0/packages/isomorphic-lib/src/types.ts#L1946-L1950C4

Add new EmailServiceProviderFailure type:

Add new EmailProviderSecret type:

Error handling for the /templates/test api endpoint:

A webhook for updating the status of sent emails:

Handle the email provider type when sending messages:

Create the Resend email provider:

Add Place in Settings to configure email provider:

_{From SyncLinear.com | DF-387}

Hey, found dittofeed on HN. Looks really cool and promising. Something that I think would enormously increase the value for many small businesses would be to integrate whatsapp, via something like wawebjs or openwa as opposed to the conventional whatsapp api since that isn'v very feasable for small businesses. Just an input.

Implement support for https://postmarkapp.com/ as an email service provider.

Add a new PersistedEmailProvider type:

Add new EmailServiceProviderSuccess type:

https://github.com/dittofeed/dittofeed/blob/01a6db73e237897b7e961c07969ea5d60a8bcbd0/packages/isomorphic-lib/src/types.ts#L1946-L1950C4

Add new EmailServiceProviderFailure type:

Implement support for https://postmarkapp.com/ as an email service provider.

Add new EmailProviderSecret type:

Error handling for the /templates/test api endpoint:

A webhook for updating the status of sent emails:

Handle the email provider type when sending messages:

Create the Postmark email provider:

Add Place in Settings to configure email provider:

_{From SyncLinear.com | DF-437}

When user properties section is collapsed, text editor should expand left:

Collapse icon should be double left arrows inside of a circle, overlayed on the user properties section's right border (near the top), with a short, informative tooltip:

https://fonts.google.com/icons?selected=Material%20Icons%20Outlined%3Akeyboard_double_arrow_left%3A

Expand icon should be double right arrows inside of a circle, overlayed on the border between the message editor and the Dittofeed menu bar, with a short, informative tooltip:

https://fonts.google.com/icons?selected=Material%20Icons%20Outlined%3Akeyboard_double_arrow_right%3A

"Publish Changes" and "Send Test Message" buttons should appear as the following icons respectively, also inside of circles, overlayed on the same border as the expand icon (vertically, below expand button), with short, informative tooltips:

1. Publish Changes https://fonts.google.com/icons?selected=Material%20Icons%20Outlined%3Apublish%3A
2. Send Test Message https://fonts.google.com/icons?selected=Material%20Icons%20Outlined%3Aforward_to_inbox%3A

_{From SyncLinear.com | DF-428}

Implement a new /twilio endpoint in the packages/api/src/controllers/webhooksController.ts. It should consume webhook events from Twilio.

docs:

• Twilio tracking overview: https://www.twilio.com/docs/messaging/guides/track-outbound-message-status
• On the meaning of different statuses: https://www.twilio.com/docs/messaging/guides/outbound-message-status-in-status-callbacks
• Webhook standards: https://www.twilio.com/docs/usage/webhooks/webhooks-connection-overrides
• Webhooks with advanced Opt-Out Opt-In feature: https://www.twilio.com/docs/messaging/tutorials/advanced-opt-out#keeping-track-of-your-users-status
• Opt out error code 21610: https://www.twilio.com/docs/api/errors/21610

You should use the I-Twilio-Idempotency-Token as the message id if available, when submitting internal events corresponding to dittofeed webhook events.

In particular, the events we care about are:

• Delivered: The message has been delivered to the recipient's device.
• Failed: The message could not be delivered.

Along with handling the OptOutType on webhook payloads, to determine if the user has changed their subscription status. When an SMS requests to twilio fails with error code, the user's subscription status should also be updated.

To complete this ticket, record a video of you sending multiple test messages with the sms template editor:

1. A generic message e.g. "hello world"
2. A message to unsubscribe e.g. "STOP"
3. A message to re-subscribe e.g. "START"

Use a tool like ngrok to consume the webhook payloads locally, and show that they produce events locally.

_{From SyncLinear.com | DF-328}

When changes have been published, this check icon should appear beside the "Publish Changes" text:

https://fonts.google.com/icons?selected=Material%20Icons%20Outlined%3Acheck_circle_outline%3A

When changes haven't been published, the check icon should disappear

_{From SyncLinear.com | DF-427}

Apologies if I missed this somewhere, but I searched through all the docs and couldn't find it.

Would be great to have a way to manage templates (really everything) via an API. For example, I've got a platform that already has a way for users to build email templates (in a more WYSIWYG way), but I'd love to use Dittofeed for the eventing and sending of messages. Ideally, I'd just POST/PUT the template when they edit it, and otherwise let them build their workflows/etc the same way.

Similarly, I'd love to be able to embed various parts of the dittofeed admin/management area within the app. If it was directly embeddable, that'd be nice, but I could also make do with an API as well.

Thanks!

While following the instructions for docker installation, I am getting an error:

$ ./scripts/admin.sh bootstrap --workspace-name="Our Name"
[04:57:37 UTC] INFO: Upserting workspace.
...
[04:57:38 UTC] INFO: creating write key
...
[04:57:38 UTC] INFO: Bootstrapping worker.
[04:57:48 UTC] ERROR: Failed to bootstrap worker.
    err: {
      "type": "Error",
      "message": "Failed to connect before the deadline",
      "stack":
          Error: Failed to connect before the deadline
              at checkState (/service/node_modules/@temporalio/client/node_modules/@grpc/grpc-js/src/client.ts:176:18)
              at Timeout._onTimeout (/service/node_modules/@temporalio/client/node_modules/@grpc/grpc-js/src/channel.ts:745:9)
              at listOnTimeout (node:internal/timers:564:17)
              at processTimers (node:internal/timers:507:7)
    }

Everything prior in the script seems to have gone fine.

These are the running containers at the time:

$ docker ps
CONTAINER ID   IMAGE                                           COMMAND                  CREATED        STATUS        PORTS                                                                                                                             NAMES
8932c7714910   dittofeed/dittofeed-dashboard:v0.7.4            "docker-entrypoint.s…"   12 hours ago   Up 12 hours   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                                                                                         dittofeed-install-dashboard-1
b5134cb6689d   dittofeed/dittofeed-api:v0.7.4                  "docker-entrypoint.s…"   12 hours ago   Up 12 hours   0.0.0.0:3001->3001/tcp, :::3001->3001/tcp                                                                                         dittofeed-install-api-1
8cb6f42b84ae   dittofeed/dittofeed-admin-cli:v0.7.4            "tail -f /dev/null"      12 hours ago   Up 12 hours                                                                                                                                     dittofeed-install-admin-cli-1
236562f2c454   postgres:15                                     "docker-entrypoint.s…"   12 hours ago   Up 12 hours   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                                                         dittofeed-install-postgres-1
a5abfded5ba2   clickhouse/clickhouse-server:22.9.5.25-alpine   "/entrypoint.sh"         12 hours ago   Up 12 hours   0.0.0.0:8123->8123/tcp, :::8123->8123/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:9009->9009/tcp, :::9009->9009/tcp   dittofeed-install-clickhouse-server-1

I checked the logs of these and nothing pops up as an error.

Any ideas?

Thanks!

The url state state should include the page number, the page size, the searchTerm, and the currently selected event if there is one for the events table: packages/dashboard/src/components/eventsTable.tsx

_{From SyncLinear.com | DF-436}

Hey team,
I am Kaiwalya, Developer Advocate at Elestio. Elestio has been providing options of fully deploying and managing Dittofeed application as shown here. I think it would be a great idea if we can add it to official readme/documentation here.
In addition to this, if you are interested we provide collaboration opportunities with tools we support by revenue share upon addition of this method in docs. If you would like to collaborate, just drop me an email at [email protected] :)

Currently, the pretteri eslint plugin doesn't seem to be working. This code:

from this PR

https://github.com/dittofeed/dittofeed/pull/603/files#

doesn't fail linting due to prettier errors, when it should

_{From SyncLinear.com | DF-419}

When on the /dashboard/users page or the /dashboard/users/[id] page, user properties with the Internal resourceType should not be rendered.

_{From SyncLinear.com | DF-362}

Implement a search bar in the events page. This should allow you to search events by:

• event name (or page/screen)
• event type
• message id

Selecting an individual event should bring up a sidebar showing the event in greater detail.

Should be featured on the /dashboard/events page.

_{From SyncLinear.com | DF-322}

_{From SyncLinear.com | DF-451}

Issue:
The current default from address behaviour in message templates can lead to conflicts when switching email providers while using the same template.

Use Case:
For example, if a user creates a message template with a specific default from address and later switches from SendGrid to SMTP as the email provider, the default from address specified in the template may no longer be valid or appropriate for the new provider, leading to potential delivery issues.

Proposal:
To address this issue, we propose implementing dynamic default from address fetching based on the selected email provider. This would ensure consistency and accuracy in default from address usage across different providers. Additionally, providing user guidance and fallback mechanisms will help handle cases where the default address is not specified or cannot be fetched.

Modify the packages/dashboard/src/components/deliveriesTable.tsx component by adding a preview column. This column should use an "eye" icon.

Clicking on this column should bring up a full screen preview of what the delivery looked like for users. This should reuse the preview portion of the packages/dashboard/src/components/templateEditor.tsx component, including the rising animation.

_{From SyncLinear.com | DF-435}

• add default secretkey value
• provide warnings on login screen that env values haven't been overridden

_{From SyncLinear.com | DF-450}

The resend integration's endpoint for tracking is giving 400's in production. Create a resend application, and use ngrok or the equivalent to test locally.

_{From SyncLinear.com | DF-446}

Add a delete button in the /dashboard/users/[id] page to hit the DELETE /api/users endpoint. Make sure it has a confirmation modal.

_{From SyncLinear.com | DF-417}

I am using EC2 instance with Amazon Linux and getting the following error.

Am I doing something wrong?

Create a CI job to validate that the packages/docs/open-api.json file is up to date, when modifications to packages/api , packages/backend-lib or packages/isomorphic-lib are made. This will ensure that the PR author has updated the mintlify open-api docs if necessary.

_{From SyncLinear.com | DF-438}

Ensure /api/journeys/stats can serve tracking information about SMS message nodes.

_{From SyncLinear.com | DF-448}

_{From SyncLinear.com | DF-452}

Filter users by user property value or segment value e.g.

• user property - email:name
• segment - new users:true

Should be featured on the /dashboard/users page. Will likely require searching over the computed_property_assignments_v2 clickhouse table, which will likely require a new skip index.

The filtering UX should be modeled after Linear's filtering, as shown below.

https://www.loom.com/share/97674e0a0aea4c459f768fc21ea2199c?sid=da3456c0-e48b-4045-97e8-855b7d937330

The computed_property_assignments_v2table should be queried by providing values for all columns in the sort key,

(workspace_id, type, computed_property_id, user_id)

_{From SyncLinear.com | DF-299}

Where's all the code?

Create table component for paginated resource list views:

• /dashboard/templates
• /dashboard/segments
• /dashboard/journeys
• /dashboard/broadcasts
• /dashboard/user-properties

Component should have columns for

• Name
• Updated At

and allow for additional columns e.g.

Segments

• Journeys Used By
• Last Re-Computed

Templates

• Journeys Used By

User Properties

• Last Re-Computed
• Templates Used By

Broadcasts

• Sent At

_{From SyncLinear.com | DF-415}

This issue will contain the proposal to add IOS push, and Android push channels.

Backend

New required models / concepts:

• New "Channel" entries, with device id's as the relevant identifier rather than email address.
• For mobile push, we need a new convention for recording and calculating device specific subscription state.
  • Standard subscriptions are device independent and configured within the upstream application.
  • Device specific subscriptions, which are configured within the end user's device settings.
• Some standard set of user properties to be aggregated in clickhouse, including device subscription status (better name pending), and device token.
• A new "Application" model. This model will have an assigned auth token secret, which will be used to retrieve user's device tokens.
• Channel specific analytics.

We will also need to amend our user journey temporal worker to support additional message types.

Frontend

New requirements:

• A way to select different messaging channels from the journey builder message node.
• A way to create channel specific templates from the UI.
• A way to restrict the available the available templates in the message node based on the selected message type.
• A way add the relevant mobile auth token in the application settings.

When user clicks the delete button on the list view on items on the follow pages:

• /dashboard/journeys
• /dashboard/templates
• /dashboard/segments
• /dashboard/subscription-groups

Open a confirmation modal before deleting, with a relevant message.

_{From SyncLinear.com | DF-344}

Implement a ui element in /dashboard/settings for selecting SMS providers. This should look similar to the ui element available for selecting among email service providers.

_{From SyncLinear.com | DF-444}

the dev API server changes in response to file changes in the dashboard package. scope the tsc-watch command so that it only changes in response to changes in the backend-lib isomorphic-lib or api packages.

for acceptance, record a video demonstrating that making changes to the dashboard package doesn't restart the api server.

_{From SyncLinear.com | DF-432}

Hi,

We have deployed using Self-Host with Docker Compose. However, the value of API base is coming as "apiBase": "http://localhost:3001"

Can you please provide how to set this value in docker-compose?

This issue will describe the work that needs to go into implementing git based workflows, as described in Dittofeed's docs. This is a major initiative and will require several new internal features.

Resource Mapping

A way to map database resources to and from yaml configuration.

• A function for mapping resource values held in postgres to a YAML based dsl / config, resourcesToConfig.
• A function for diffing resource config files and updating resource values held in postgres, updateResourcesFromConfig.
• A new branch column on resource data models, updating their unique columns from @@unique([workspaceId, name]) to @@unique([workspaceId, name, branch]). This value would default to "main".

Authorization

An authorization scheme to protect resources on the main branch. When workspaces are "protected", you should not be able to update resources on the main branch directly.

• A function canUpdateResource which accounts for the branch of the resource that's being updated.
• Using this function in the API to prevent updates to protected resources.
• Incorporate this function into the dashboard, to prevent forbidden updates.
• A way to make a workspace protected in the settings.

Branching

• An API endpoint for creating new branches, and switching between them, to be integrated into the dashboard.
• An API endpoint for deleting branches, to be integrated into the dashboard.
• An API endpoint for rendering diffs between workspace branches.

Github App.

We will need a git app reading from, and writing to github. This app will be used to write config to git when changes are made, and read them back from main.

The previously discussed updateResourcesFromConfig will be called on the main branch when changes to the config are made.

This column in the events table component will contain links to resources for InternalEventType events. For example BadWorkspaceConfiguration events for MessageTemplateRenderError should contain a templateId in its payload. This id should be used to render a link to the related template with its name as the link title.

Moreover the "individual event view" rendered by clicking on an event, should also contain a related resources section.

Note that this should also relate journeys by journeyId. Journey resources should link back to a broadcast page rather than a journey page, if the journey belongs to a broadcast.

_{From SyncLinear.com | DF-421}

currently, when editing a segment, the segment's configuration disappears if an operator is added

_{From SyncLinear.com | DF-470}

Create a section of /dashboard/settings in which users can set a default "From" address for emails. When creating new email templates, the From section should be populated with this value by default.

_{From SyncLinear.com | DF-317}