multiwoven / multiwoven Goto Github PK

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

In FullRefresh sync mode, for every sync run, we fetch all records from the source and send them to the destination. Since there's no need to preserve the sync records once the data is moved to the destination, we need to flush or clean the sync records after each completed run.

Hi !

Congratulation for this project, it will be awesome!

I just want to open this feature request to allow developers building custom destination connectors.
Feel free to choose the best framework for this feature ;)

Regards

Is your feature request related to a problem? Please describe.
Releasing an update for our repo requires manual intervention to release with a semver version update. It also takes some time to create a change log. Because these are manually done we have a discrepancy in our application release. Also when we update the docker images with the tag, the tags created are commit sha which should ideally be release versions. All these issues will be solved by automating the release process.

Describe the solution you'd like
We will cut a weekly release with a minor version. We will also set up a workflow which can be used to cut patches or major releases by manually initiating the GitHub action. This GitHub action will create a tag and push it to the repo. This tag will push a docker image to the docker hub with the same tag. We will also create PR with release notes using Git Cliff which will create a release not on merge.

validation for the primary key in the ActiveRecord model involves checking if the primary key exists for the source connector

model create payload

{
  "model": {
    "connector_id": 25,       // Replace with the appropriate connector_id
    "name": "users",          // Replace with the desired model name
    "query": "SELECT Id,User.Username, RecordType.Name, Account.AccountNumber, Account.OnboardedAccountNumber__c, PlannedVisitStartTime, PlannedVisitEndTime FROM Visit",  // Replace with the desired query
    "query_type": "soql",     // Replace with the desired query type
    "primary_key": "Id"
  }
}

Is your feature request related to a problem? Please describe.
Add MySQL connecter or Give the option to add a configurable connector on user choice

In open source there is not option to create own connector thourgh UI please provide atleat mysql connector because most of the people use mysql as it there source .
Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

**Is your feature request related to a problem?
We are showing queried, success and failed count in the sync run listing. In incremental refresh we skip the rows based on the primary key and creates confusion for user eg:total queried is 1000, success =2, failed = 0. The question is where the other 998 records are. why it is not showing in the sync run list metrics.

Describe the solution you'd like
We need to show a new metrics called skipped which states how many records in the sync due to duplicate primary of the model.

Good to have
We need to show skipped rowns count in the reporting dashboard.

**Is your feature request related to a problem? **
I would like to disable sync from source to destination for some time when especially when there schema changes going on the source/destination

Describe the solution you'd like
Frontend should give an option to disable or enable a sync

Create a helm chart for deploying the multiwoven app.