This issue is to outline some goals of and work to do for a 1.0 stable release. For specific issues, see those labeled with the 1.0 milestone. Everything described here is up for discussion - if there are features would like to see or have an interesting use case, please leave a comment below.

There are a few questions in the "moving forward" section below - please chime in if you have any thoughts!

Scope

• The main goal of this package is to sync data to Sanity, not back to Shopify.
• Additionally, the focus will only be on syncing products & collections, and limited to what the Storefront API can read. The primary reason for this is security: by default, Sanity datasets are public. To sync orders, customer data, and so on would make it easy for users of the package to expose sensitive business information. The dashboard would also need to store Shopify API keys in its dataset. While datasets can be set up with more fine-grained access control, this package won't directly support handling sensitive data at all. (But, there will be a way to roll your own solution, see the notes under @sane-shopify/server below)

1.0 Features

overall

• General stability. There are a lot of potential issues between the two services, and a lot of edge cases to be found and handled.
• Testing. So far, this package has no automated tests. As the work below is completed, test coverage needs to be improved. Most of this testing will need to mock out requests and responses from Shopify & Sanity, which is kind of a drag, and could also mean more maintenance down the line as the API of the services changes. But for e-commerce, stability is crucial, so this needs to happen to some degree.
• Project management / roadmap. I'll either use Github Projects or a Notion document to track features, their progress, and a general roadmap.
• Documentation. This all needs better docs! Please open an issue or PR with anything you'd like to see in this regard - the whole thing is in my head at this point, so it's hard for me to see the missing pieces.

@sane-shopify/sanity-plugin

• Sync products & collections to sanity from the dashboard
• Create sanity document references between products & collections that match the relationships in Shopify
• Configure what fields you would like to sync (collections and products) #17
• Add custom fields to product variants #33
• Add a "sync item" button to each collection/product document #51

@sane-shopify/server

• "ready to go" serverless functions for accepting Shopify webhooks. #31
  For 1.0, this will include:

  • Product: products/create, products/update, products/delete
  • Collection: collections/create, collections/update, collections/delete

  These functions will assume some kind of standard sync. Versions will be available for AWS Lambdas (which are also used on Netlify) and Zeit.

• A more configurable webhook, or some boilerplate to use the sync-utils API directly to write your own webhook logic.

@sane-shopify/sync-utils
This package is the core of the project and is used within the webhooks and the Sanity plugin. Most of its use will be internal, but the API will be documented and available for more advanced usage.

• reconfigure sync-utils internal API #39

Moving Forward

Beyond that, the project will grow more depending on the needs and interest of those using it. Please let me know what you'd like to see out of sane shopify!

@sane-shopify/sync-utils

• Sync basic shop data in a sanity document #42

  Read-only versions of the shop policies/counties/payment settings might be useful to store (mainly to avoid duplication and API rate limits)

@sane-shopify/server

• Shop data webhook #42

@sane-shopify/sanity-plugin

• A better dashboard experience. Right now, it's pretty basic looking, and only gives simple syncing progress information.
  • You also can't leave the tab until the syncing is complete. This may need to wait until this issue is handled by the Sanity team.
  • What sort of information would you like to see in a dashboard?
  • A recent activity feed would be great to give peace of mind that everything is working correctly

nevermind, closing. Feel free to open separate issues for ideas & suggestions.

Hey guys,

So I'm trying to use the new Product Variant fields that was added and I noticed that there's no ShopifyId at the variant level, it doesn't appear to be in sanityShopifyProduct nor allSanityShopifyProduct.

The use case here is I need the variant ShopifyId in order to add the right variant to cart.

This is everything I see under sanityShopifyProduct:

A recent activity feed would be great to give peace of mind that everything is working correctly

sync-utils

• add a lastSync field to all documents, and update with all creates & patches

sanity-plugin

• Use GROQ to query the datastore for the last N updates, based on the lastSync field

• add yarn build to all package's prepare scripts
• add git commands to CI scripts:

  git checkout master
  lerna bootstrap
  git reset --hard
  lerna publish from-package --yes
  

  • make sure this pushes the updated version & commit to github.
  • The reference below mentions that developers do a manual lerna version. Make sure that the commands above take care of the versioning, so any push to master will trigger a publish.

References:

• https://medium.com/mitterio/multirepo-to-lerna-js-monorepo-80f6657cb443

The plugin should accept some configuration that will affect the in-dashboard syncing tool. For instance, users may want to sync products, but not collections.

The sync-utils API should also be updated to reflect this, and offer fewer, yet more configurable functions. Probably just one function for syncing single documents, and another for syncing many.

All of the webhooks will use this API as well, so their use cases will need to be taken into account.

Sync

• Show a progress bar for syncing operations
• disable sync button during sync
• hide sync button after sync
• Don't show until setup is complete
• Show progress for fetching initial documents, then both syncing document
• Show progress bar for archiving items

Summary

• Display summary ("3 products updated", "10 collections archived")

Activity

• Display list of recently synced products & collections

Setup

• Show confirmation/error after submitting key
• add "are you sure" confirm when removing keys

General

• just make things look better!

Rate limiting is currently implemented by a simple delay(100) for sanity PUT updates. The get updates and calls to the Shopify api should have proper rate limiting. Maybe https://www.npmjs.com/package/rxjs-ratelimiter

Read-only versions of the shop policies/counties/payment settings might be useful to store (mainly to avoid duplication and API rate limits)

• Create sanity document type
• include shop info in sync
• set up webhooks
• determine what data to sync

@homerjam could you put together a GraphQL query for the Storefront API that would fetch the information you think would be useful here?

I discovered this after updating the URL in use from https://your-storefront-name.myshopify.com/api/graphql to /api/2020-01/graphql. According to the shopify docs, this should work, but it doesn't. Anyway, it shows the Setup screen instead of a displaying some kind of error about connecting to the API:

A request with a bad token will also return an error - so, we'll need to differentiate between that error and other that are just not connecting in the first place, either because the URL is somehow bad (maybe the storefront name changed, but it hasn't been updated in sanity), or the request is somehow blocked in another way.

https://github.com/jaredpalmer/tsdx

I'm planning to use the Storefront API as the auth layer for a GraphQL API that wraps Sanity/Shopify API requests. I'll then store additional sensitive data for each customer in Sanity which customers will have access to if they are logged in.

In order to do this I'll need to sync customer data - initially only the ID will be strictly necessary but name/email etc would obviously be useful.

With this in mind is there a way we can make the syncing system more flexible ie. not tied to products/collections so that it can be used for variants, customers etc? For instance there appears to be some repeated code in the fetching utils.

I'm getting the following error when running sanity graphql-deploy.
Anyone else encounter the same thing?

➜  studio git:(master) npm run graphql-deploy

> [email protected] graphql-deploy /Users/raf/code/gatsby-shopify-sanity-alpha/studio
> sanity graphql deploy --playground

✖ Generating GraphQL schema
/Users/raf/code/gatsby-shopify-sanity-alpha/studio/node_modules/@sane-shopify/sanity-plugin/lib/index.js:1
import { createElement, Fragment, Component } from 'react';
^^^^^^
SyntaxError: Cannot use import statement outside a module
    at compileFunction (<anonymous>)

Hey there — 

I'm in the situation of having multiple instances of Shopify — they're exactly the same, but ideally I'd manage the content in one place. So for products, I need to have the product id's from both stores link.

Would that be a remote possibility, even with some moderate/heavy modification?

Cheers

Hey @good-idea, thanks for adding in product variants!

So I got it compiling fine now, however when I try to resync my shopify products (my products on Shopify has actual variants so I'm trying to resync to see if they match up to the new variants schemas), collections will sync fully, however when it gets to syncing products it gets stuck at 0, and I see the following error in console:

Here's my latest schema.js:

import createSchema from 'part:@sanity/base/schema-creator'
import schemaTypes from 'all:part:@sanity/base/schema-type'

import {
  createProductDocument,
  createCollectionDocument,
  createProductVariant,
  saneShopifyObjects,
} from '@sane-shopify/sanity-plugin'

const product = createProductDocument({
  fields: [
    {
      title: 'Tags',
      name: 'tags',
      type: 'array',
      of: [{ type: 'string' }],
      options: {
        layout: 'tags',
      },
    },
  ],
})
const productVariant = createProductVariant()
const collection = createCollectionDocument()

export default createSchema({
  name: 'mySchema',
  types: schemaTypes.concat([
    ...saneShopifyObjects,
    product,
    productVariant,
    collection,
  ]),
})

• Zeit/Micro
• AWS Lambdas
• configure dataset

From https://community.shopify.com/c/API-Versioning/Cannot-access-versioned-graphql-API/m-p/641464#M128:

I am able to connect to my shop's Storefront API with the URL:

https://my-shop-name.myshopify.com/api/graphql

But not with a versioned URL, i.e.:

https://my-shop-name.myshopify.com/api/2020-01/graphql
https://my-shop-name.myshopify.com/api/2019-07graphql

I'm using my storefront token in the headers. Am I missing something obvious?

0.6.3 added 2020-01, which broke the sanity plugin. 0.6.6 removes it. But, this API version should be locked in, it changes often and some changes could be breaking. :(

• Allow users to provide a custom query for what additional data to sync from Shopify when syncing

Potential issue: both the Sanity plugin and the proxy server will need to be configured with the same options.

use these to define the names in the sanity plugin, as well as typenames in places like syncingClient.ts -> getItemType

Hey Joseph,

I was trying to transform Product Images using

allSanityShopifyProduct() {
  edges {
    node {
      sourceData {
        images {
          edges {
            node {
              localFile {
                childImageSharp {
                  fluid(maxWidth: 910) {
                    ...GatsbyImageSharpFluid_withWebp
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

but looks like that's not supported. Is this something you're interested in having in the package, or are we better off adding an image field in Sanity?

I'm thinking ahead about clients having to add images twice: Once in Shopify from (needed to see the thumbnail on checkout) and then once in Sanity.

Interested to see what you think.

• keep a log of sync events in a private sanity document
• display logs in shopify tool pane

Hey there — giving this plugin a go again! Initial setup does look much cleaner in 0.8! Unfortunately once I get to localhost:3333/shopify, I'm hit with the "Connected to shopify storefront" screen and can't Unlink. Haven't set it up yet so it's odd that that's happening.

Any ideas on a workaround?

• Currently, the document types are shopifyCollection and shopifyProduct. This gets confusing when using the GraphQL proxy, where Product is coming from shopify and ShopifyProduct is coming from sanity.
• Also, change sanityObjects to saneShopifyObjects

Re: #46

• Update plugin UI to be able to add, sync, & remove multiple storefronts.
• Include storefront info in all synced items
• include in sync-utils API

for products & collections that have no images:

While this project is in its, obvious, early stages, there is currently no license attached the project.

We're looking around for some solutions for an upcoming project, and while this one seems like something we'd like to contribute to, I'm hesitant to push my developers here without some project licensing information.

Right now, the plugin handles all of this, but these should just be moved to the sync-utils.

This might be annoying because to create the sync utils, you need to pass in credentials. If you pass in undefined credentials, calls to any method will result in an error.
But, that should be OK, users can always use testSecrets to confirm that they are in good shape

In progress - working on adding product variants as an array of sub-fields to the Sanity documents, allowing you to extend on these variants. For instance, adding a separate descriptions or galleries of images.

If it's all connected in Sanity, there's no need to deal with the shopify GraphQL server at all. I wish i had thought of this sooner!

Clients will still need to use the Storefront API for adding & creating checkouts, customer login, etc. But all public Product & Collection info should be synced to their Sanity documents.

eg:

Updated 10 products
Created 2 products
99 products already up-to-date

These are mostly used internally, but might be helpful for others to use without the whole setup.

Until this project gets off the ground, this will be my unofficial "mailing list" :) I'll make announcements here.

Please restrict comments to a quick hello to sign up so others who have commented aren't getting too many emails.

If you have any questions about or suggestions for the project, feel free to open an issue, or find me @joseph on the Sanity.io slack channel.

Hey Joseph,

So I tried removing and reinstalling node modules and I'm still getting this error on 0.6.3:

✖ Generating GraphQL schema
/Users/mathew/Dev/esqido-gatsby-sanity/studio/node_modules/@sane-shopify/sanity-plugin/lib/index.js:1
import { createElement, createContext, Component, useContext, Fragment } from 'react';
^^^^^^
SyntaxError: Cannot use import statement outside a module
    at compileFunction (<anonymous>)
    at Object.<anonymous> (~/Dev/esqido-gatsby-sanity/studio/schemas/schema.js:4:1)

Here's my schema.js:

import createSchema from 'part:@sanity/base/schema-creator'
import schemaTypes from 'all:part:@sanity/base/schema-type'

import {
  createProductDocument,
  createCollectionDocument,
  saneShopifyObjects,
} from '@sane-shopify/sanity-plugin'

const product = createProductDocument({
  fields: [
    {
      title: 'Tags',
      name: 'tags',
      type: 'array',
      of: [{ type: 'string' }],
      options: {
        layout: 'tags',
      },
    },
  ],
})
const collection = createCollectionDocument()

export default createSchema({
  name: 'mySchema',
  types: schemaTypes.concat([...saneShopifyObjects, product, collection]),
})

Hi there,

First of all thanks for putting this project together!

I'm currently working on a custom storefront and after following the instructions in the readme, I'm running into this compile error which didn't exist prior to installing this package:

Error in ./node_modules/graphql-tag/src/index.js
Module not found: Error: Can't resolve 'graphql/language/parser' in '/Users/mathew/Dev/esqido-gatsby-sanity/studio/node_modules/graphql-tag/src'
 @ ./node_modules/graphql-tag/src/index.js 1:13-47
 @ ./node_modules/@sane-shopify/sync-utils/lib/index.es.js
 @ ./node_modules/@sane-shopify/sanity-plugin/lib/index.js
 @ ./node_modules/@sane-shopify/sanity-plugin/lib (all:part:@sanity/base/tool)
 @ ./node_modules/@sanity/default-layout/lib/defaultLayoutRouter.js
 @ ./node_modules/@sanity/default-layout/lib/components/DefaultLayoutContainer.js (part:@sanity/base/root)
 @ ./node_modules/@sanity/base/lib/components/SanityRoot.js (part:@sanity/base/sanity-root)
 @ ./node_modules/@sanity/server/lib/browser/entry-dev.js
 @ multi ./node_modules/normalize.css/normalize.css ./node_modules/@sanity/server/lib/browser/entry-dev.js

Here's my schema.js:

import createSchema from 'part:@sanity/base/schema-creator'
import schemaTypes from 'all:part:@sanity/base/schema-type'

import {
  createProductDocument,
  createCollectionDocument,
  saneShopifyObjects,
} from '@sane-shopify/sanity-plugin'

const product = createProductDocument()
const collection = createCollectionDocument()

export default createSchema({
  name: 'mySchema',
  types: schemaTypes.concat([...saneShopifyObjects, product, collection]),
})

My sanity.json:

"plugins": [
    "@sanity/base",
    "@sanity/components",
    "@sanity/default-layout",
    "@sanity/default-login",
    "@sanity/dashboard",
    "@sanity/desk-tool",
    "dashboard-widget-structure-menu",
    "dashboard-widget-document-list",
    "dashboard-widget-netlify",
    "@sane-shopify/sanity-plugin"
  ],
  "parts": [
    {
      "name": "part:@sanity/base/schema",
      "path": "./schemas/schema.js"
    },
    {
      "name": "part:@sanity/desk-tool/structure",
      "path": "./deskStructure.js"
    },
    {
      "implements": "part:@sanity/dashboard/config",
      "path": "./dashboardConfig.js"
    }
  ]

Any advice would be appreciated!

• After a product is deleted (via webhook)
• After a full sync, if the existing document cannot match one found in the Shopify inventory

Setup button stays in loading state perpetually, even though setup has finished

CMS users should be able to manually trigger a sync of an individual document from the dashboard

BLOCKED, needs a single syncDocument export from sync-utils

Hi,

I'm working on a proposal for a client project which would combine Shopify and Sanity - if it goes ahead I think it would be great to make use of sane-shopify and contribute to it.

There's a feature I was wondering about to reduce friction for the client - do you think it's possible to create products in Sanity and push these to Shopify via the Shopify Admin API?

Have you considered this and are there any technical constraints preventing it that I might not have considered?

Thanks,
James

The frontend side of the sanity plugin is pretty messy, now is the time to refactor to make it more readable & use hooks.

• set react deps to 16.8

To get an error, mess up the GraphQL queries by removing one of the fragments