vendure-ecommerce / vendure Goto Github PK

Description

Consider the following:

1. Bob is an authenticated Customer
2. At some point in the past, Bob was signed in and added 3 items to the active Order
3. Bob logged out, his session expired, or he is browsing on a different device
4. Bob (as anonymous user) Adds 2 items to an active Order.
5. Bob logs in.

What do we do at step 5? We have now 2 active Orders - the current anonymous Order with 2 items, and the former Order with 3 items.

There are the following possibilities:

1. The more recent Order (2 items) replaces the old Order (anonymous priority).
2. The old Order is reinstated and the recent 2 items are discarded (authenticated priority).
3. The contents are merged (merge).

Merging Strategy

I believe there should be a Strategy approach to this, because in different circumstances, different strategies makes sense. Consider:

• If Bob logs in during regular browsing, the merge strategy makes most sense.
• If Bob logs in during the checkout process, the anonymous priority strategy makes most sense - otherwise Bob will be shocked to find he is actually buying 5 items rather than the 2 he was expecting.

The choice of strategy should be configurable, as different retailers may have different ideas about the best way to handle this.

The ProductVariantTranslation entity is not being updated.

See product.service.ts line 102. Also this whole method should be moved to product-variant.service.ts.

Plugins are a way to customize the behaviour of Vendure. They would allow the developer to replace certain default behaviours with new ones, and to extend the capabilities of the default Vendure install.

For example, a plugin could:

• Change the storage of assets from the local file system to a cloud provider like s3
• Integrate with payment gateways
• Implement custom shipping logic
• Implement custom promotion logic
• (potentially) alter the state machine which will be used to control the orders process and add new states and transitions and hook custom logic into those transitions.
• Add custom fields to entities and apply logic based on those fields
• (potentially) define entirely new entities and their relationships with the default entities.

Design

We have a single VendureConfig object which contains all configurable aspects of the server.

The config object has three broad kinds of configurable property:

1. Plain values, such as port or channelTokenKey.
2. Strategies which are used for things like determining the way entity IDs are handled, or how image previews are generated.
3. Hooks, which are just callbacks used to interject some custom logic at certain points of certain processes.

The idea is that a "plugin" is just a way to set a bunch of related config rules all at once, as well as perform any other set-up or logic which might be needed when the app bootstraps.

The simplest implementation would be:

export interface VendurePlugin {
    init(config: VendureConfig): VendureConfig | Promise<VendureConfig>;
}

A plugin implements this interface and when the app bootstraps, each plugin is run in series (they would be defined as an array in the config passed to the bootstrap() function).

Each plugin can modify the config and then that modified config is passed on to the next plugin in the array.

Currently we only have CRU but no D operations. I have intentionally left our "delete" operations because I think we need a "soft delete" (i.e. do not physically remove from the DB, but just mark as "deleted").

This is due in an upcoming version of TypeORM (typeorm/typeorm#534) so I will wait until this ships before implementing deletes.

For queries which yield a PaginatedList response (e.g. products(), customers() etc.) it will be a common requirement to filter and sort those lists.

There should be a generic format for requests which allow filtering and sorting to be expressed in the query arguments.

Sorting

1. Sorting should be possible on any publicly-exposed field which has a primitive value (i.e. you can sort products by "name", but not by "variants")
2. Multiple sorts should be possible, e.g. sort first by name, and then by price.
3. Sort direction may be specified but would default to ascending.
4. Sorting should be possible on custom fields

Filtering

1. Filtering should be possible on any publicly-exposed field which has a primitive value (i.e. you can filter products by "name", but not by "variants")
2. Multiple filters should be possible, e.g. filter by name match, and then by price comparison, but a truly flexible, nestable API is not required (or even desirable). Instead, complex nested filtering should be the responsibility of a dedicated search feature, which would e.g. pass through an Elastic query to an instance of Elastic. For the simple filter, multiple filters would be applied with an implicit AND operator.
3. To implement filtering there will need to be some kind of DSL which provides a set of operators (string contains, string match, starts with, number equals, number greater than etc.) which are applied to a specified field.
4. Filtering should be possible on custom fields.

API Design

First of all we should put all list options into a single object. Currently we have individual take and skip args. These should be moved to a ListOptions type:

interface ListQueryOptions {
  take: number;
  skip: number;
  sort: SortParameter[];
  filter: FilterParameter[];
}

This approach also has the advantage that we can more easily add options in the future.

interface SortParameter {
   field: string;
   order: 'asc' | 'desc';
}

interface FilterParameter {
  field: string;
  operator: FilterOperator;
  value: string | number;
}

type FilterOperator = 
  'gt'  |
  'gte' |
  'lt'  |
  'lte' |
  'eq'  |
  'contains' |
  'startsWith';

Currently the Vendure server returns various error messages when encountering exceptions which have a i18n message but no specific status code or error code.

For a better developer experience for those building front-ends for the API, we should have a system of language-independent error codes and also be able to explicitly set the status code when throwing an I18nError.

For example, there are places in resolvers where we need to throw a "Forbidden" error, but there is currently no way to set the status of the response to 403.

Also, status codes allow the UI developer to implement conditional logic based on a well-defined set of possible error codes that may be returned from a given query/mutation.

These error codes would look like ERR_SHORT_DESCRIPTION, e.g.:

ERR_FORBIDDEN
ERR_ENTITY_DOES_NOT_EXIST
ERR_INVALID_SORT_FIELD
ERR_CANNOT_TRANSITION

This would also be a good opportunity to think about how to make the I18n (which is currently only used for error messages) statically typed so we know when a string has no translation.

We need a way for developers to add arbitrary custom fields to any entity.

Use Cases

• We have some products which use flammable materials which cannot be shipped by air. Therefore we want to add a custom boolean "noShipByAir" field to all ProductVariants. This can then be used in some custom shipping logic.
• We have a CMS with extra info about certain products and we want a custom url attached to a Product which we can render as a link to the corresponding info page.
• We want to attach a boolean "premium" field to a Customer and change behaviour of the app based on that.

Ideal Developer Ergonomics

Adding a field to an entity involves the following files (example for Product):

• product.entity.ts
• product.graphql
• (potentially) product-translation.entity.ts

It would be ideal to not require the developer to have to update multiple files and keep them in sync in order to add fields. Better would be to add the fields via config when bootstrapping the server, e.g.

bootstrap({ 
    port: API_PORT,
    apiPath: API_PATH,
    dbConnectionOptions: {
        //...
    },
    customFields: {
        Product: [
            { name: 'infoUrl', type: 'string' } // this is a minimal example. In practice there will 
                                                              // need to be more config options for each field
        ],
    },
);

Then on bootstrap, we dynamically add those fields to the appropriate entities and also extend the GraphQL schema using the JS API.

Field Types

• string
• localeString (will require modification to the *-translation files also)
• int
• float
• boolean
• datetime
• json

Other types could be added (e.g. geo-spacial types, blob types) but the above would be the most common.

Custom Filtering & Sorting

It would be quite likely that the developer would want to sort or filter by a custom field. This would then require dynamic extensions to the GraphQL resolvers for that entity. There needs to be some generic way of passing in custom sort and filter arguments to all list queries.

Admin UI

A form input will need to be generated in the admin ui app for the detail view of that entity (where such a detail view exists) and a "editable" boolean config option could be used to signify that the field value is read-only in the admin ui.

(Relates to #26)

The price of a ProductVariant is stored as an integer representing the price of the item in cents (or pence etc) without any taxes applied.

When putting together an order, the final price of the OrderItem is therefore subject to:

1. Taxes (e.g. VAT)
2. Promotions (buy 1, get 1 free etc.)

Furthermore, the overall Order price is the aggregate of each OrderItem as well as:

1. Shipping
2. Promotions (10%discount voucher etc.)

The price modifications listed above are known as Adjustments.

How Adjustments work

1. An AdjustmentSource defines the type of adjustment (tax, promotion or shipping) and also holds logic used to determine whether it should be applied to a given OrderItem or Order.
2. If it should be applied, then a new Adjustment is created for that OrderItem or Order (known as target).
3. The Adjustment holds the amount in cents by which the price of the target should be modified - positive for taxes and shipping and negative for promotions.

Determining whether to apply to a target

The basic idea is that each AdjustmentSource would have a function which if called for each potential target (e.g. each OrderItem in an Order) and this function should return true if an Adjustment should be applied or false if not.

The hard part will be figuring out how to allow the administrator to write this function. We don't want arbitrary JavaScript to be written and executed. A couple of alternatives are:

• Create a form-based interface which can express the common conditions (minimum quantity, minimum value, date restrictions etc) and common results (fixed discount, percentage discount etc).
  ✅ Foolproof - no arbitrary or unexpected code allowed
  ✅ Forms inputs are familiar
  👎 In order to express all possible combinations of condition & result, the form becomes very complex
  👎 Limited expressiveness and less efficient for power users.
• Create a DSL which can be used to write a written description of the conditions and results. This DSL would have to be verifiable and highlight errors in real-time like how VSCode works with TypeScript.
  ✅ Very expressive and potentially efficient.
  ✅ Potentially much more readable than forms.
  👎 Very complex to implement - if no existing solution exists, I'd have to create our own DSL -> JavaScript "compiler".

Research is needed to figure out the real costs & benefits of each approach, including:

• Tabulate all possible combinations of adjustment condition / adjustment result to see just how much we need to support
• Research existing DSL solutions. If there is nothing readily available, get an idea of how much work it would take to implement one, including an editor interface which provides good US for the administrator.

The new version 14 of graphql-js was recently released.

We have already been using the RC version for a while on the server but now should upgrade to the final release and also upgrade any dependencies which have it as a peerDependency etc.

Currently we have Jest unit tests for the server, but no tests for the API itself (i.e. tests which make GraphQL API calls and assert the response)

Now that the general API parts already implemented are (hopefully) somewhat stable, it makes sense to set up API testing infrastructure.

This would involve:

• A method of setting up a pre-defined test database. Look into using sql.js (which is supported by TypeORM) to allow all DB operations to be performed in-memory, which should ensure the tests run reasonably quickly.
• Setting up the test runner to make real HTTP calls and asserting on results. Jest snapshots would probably make sense for this. Look into supertest for making the HTTP calls.
• Setting up Jenkins to run the tests on push.
• Writing tests to cover all existing queries and mutations.

The common createdAt / updatedAt fields are missing from all entity types currently.

Some points of note if we want to support digital goods:

Customers in the EU must pay VAT on digital goods at the rate applicable in their own country regardless of where the seller is located. Make sure that your tax settings are correct to sell digital products to customers in the EU. source

Currently we are using the Apollo CLI codegen capabilities to generate interfaces based on the server schema & client operations. This works well but has the limitation that types are only generated if they are used in a client-side query.

This is not too much of a problem for admin-focused operations, since the admin ui will inevitably contain the queries/mutations required to produces the types for those operations. The problem comes for the shopfront-oriented operations (e.g. addItemToOrder()), which might not ever be used in the admin ui app.

There is an open issue (apollographql/rover#351) on the Apollo CLI repo to be able to generate types for all server schema operations, but there is currently no resolution.

Look into alternatives for generating the server schemas, such as:

• https://github.com/dangcuuson/graphql-schema-typescript
• https://github.com/avantcredit/gql2ts/tree/master/packages/cli

Shipping represents the price of sending the goods in an Order to the delivery address.

A typical ecommerce store may use several shipping providers (e.g. Royal Mail, DPD, FedEx, TNT). Each provider has particular properties which vary:

• Zones in which they operate
• Pricing structure (flat rates, by weight, surcharges, base charges etc)
• Constraints (cannot ship certain goods by airmail, do not ship past a certain volumetric weight etc)

ShippingMethod

Therefore it makes sense to create a new entity, ShippingMethod, which encode the above information. The ShippingMethod would fulfill 2 primary functions:

1. Determine whether the given Order can be shipped by that method based on Order contents and destination Zone.
2. If yes, determine the cost of shipping.

Shipping price calculation

Different shipping providers use differing pricing structures. Some examples:

UK example (flat rate)

Here is a real example from a UK-based ecommerce store.

In this example, the Zone comprises the country "United Kindom (GB)" only. However, there are variations depending on postcode, since some addresses on islands and the Scottish highlands are subject to extra charges by Royal Mail. So a given UK address can be considered either "Mainland", "Highlands & Islands" or "Channel Islands" depending on the post code. Also, the package is "small" if its total weight is under 1500g AND its volumetric size is less than 15" x 22"

Europe example

These are also actual shipping rules from a UK shop relating to shipping to Europe:

The main restriction here is that certain products contain solvents which cannot be sent via air, so this affects the available ShippingMethods. All the rules are quite complex so here is a non-exhaustive list of rules to illustrate the complexity of real-world shipping calculations:

• Small orders to "Western Europe" (a subset of all European countries) which do not contain restricted solvents: 7.00 - 11.05 depending on exact weight.
• Small orders to "Eastern Europe" which do not contain restricted solvents: 10.00 - 16.28 depending on exact weight.
• Large orders up to 25kg or containing restricted solvents: Flat rate per destination country

Method of Calculation

As the 2 examples above illustrate, real-world shipping can be pretty complex and it would be nigh on impossible to model a general solution which could account for all of the above rules for most businesses.

Therefore, a useful and flexible design would be to use the idea of a ShippingCalculator for each ShippingMethod. A ShippingCalculator would be a function (or method of a class implementing an interface) which takes the Order and returns a price.

This would allow an arbitrary level of complexity to be expressed according to the business rules of the store. For example, the "no solvents" rule can be implemented as a custom field on the ProductVariant. This would be more flexible than an idea of "shipping categories" since it can take in many dimensions at once.

For example: it would allow prices to be specified in an excel spreadsheet (a common way to enumerate a matrix of weight/price values) which would then be read by the ShippingCalculator and used to return the corresponding price.

Likewise, the "determine whether a ShippingMethod applies" function could also be a custom function, ShippingEligibilityChecker

Currently we are using JWT tokens (& refresh tokens - see #19) for client authentication.

After doing more research (see this thread, or Why JWTs Suck as Session Tokens for example), I am wondering whether we can really simplify the auth implementation.

Current Solution

• User logs in, generate JWT which encodes user's identifier & permissions. Also generate refresh token.
• Client stores both authToken & refreshToken and includes them as headers in each request.
• Server reads JWT and if valid & not expired, forwards request
• If expired, reads refreshToken & attempts to re-create a new set of tokens and then attaches these are headers in the next response.

Problems

• We store some data in the JWT but never actually use it - we always do a DB lookup of the user anyway, so actually all the data (apart from the user ID) stored in the JWT is redundant.
• The client has to store & send 2 tokens in order for the refresh to work.
• The tokens are large (~500 - 700 bytes)
• Sessions cannot be invalidated without changing a user's password.
• The backend implementation is complex and difficult to follow.

So since we always perform a DB lookup to get user data for each request, we don't seem to get much benefit from JWT.

Simpler proposal

1. Create a "Session" entity which has a randomly-generated token, a reference to a User, and an expiration datetime.
2. When a user logs in, create a new Session for that user, set the expiry according the app config, and return the token (or set a cookie).
3. Client uses the token directly as a bearer token header for next request.
4. Perform a .findOne() on the Session repository with the token. Check the expiry date and if not expired, set expiry date to "now + configured session duration". Get the User from the Session and attach to Req context and forward the request.
5. Create a logOut mutation which takes a token and deletes (or sets to 'complete') the Session and deletes the cookie.

Currently a ProductVariant has a price property with a single integer value.

It may be desirable to allow a ProductVariant to have multiple price values, one for each supported currency.

• One way to achieve this would be to use Channels (see #12) with a separate Channel representing each currency zone.
• Another approach is like that of Moltin products where the price field is an array of currency-value objects.

In practice, both of the above imply a one-to-many relationship of ProductVariant -> Price objects, in which case we may as well opt for a Channel-based approach, since Channels also unlock other desirable features.

Check out https://github.com/basecamp/trix

I need to do some research on the best way to store and represent date types. Some questions:

• What format should be used in the DB?
• What format should we send over the API?
• What format should be used in GraphQL inputs?
• Do we need a GraphQL DateTime scalar? What's the advantage over the String type?

Product Categories are a way of grouping Products. A category in essence a collection of Facets (#2). Any ProductVariant which to which all of those Facets are assigned would then be considered to be in that category.

Use Cases

Some shops may choose to organize their products in a tree-like taxonomy, starting with the most general grouping at the top and getting ever-more specific. E.g. "electrical goods" -> "phones" -> "iPhones"

Another use is for applying promotions to a particular category.

Example

Hierarchy

There should be a way of having a hierarchy of categories. E.g. The "electrical goods" category contains "phones" and "laptops". Another way to say it is that "phones" has "electrical goods" as a parent.

To implement this, the ProductCategory would have a "parent" one-to-many relationship. TypeORM has built-in support for such tree structures: http://typeorm.io/#/tree-entities

Most countries have some kind of sales tax applied to goods being purchased.

In the UK for example, the current VAT rates are zero (0%), reduced (5%) and standard (20%). Here are rates in other EU countries

A tax is modeled as a type of Adjustment (#29) (AdjustmentType.TAX) and should be applied before any other adjustments, since other adjustment conditions would typically query the tax-inclusive prices.

Design

1. The ProductVariant entity should have a taxCategory property which points to a tax AdjustmentSource.
2. The price property should be tax-inclusive. There should be a new property, priceBeforeTax which contains the price before taxes are applied.

A ProductVariant should have assets just like a Product.

Use case:

Product = Winsor & Newton artists' watercolours.
ProductVariant = tube of Cerulean Blue. Asset is a color swatch image.

The following needs to be done:

• Add assets and featuredAsset props to the TS entity and the GraphQL type
• Update the service layer to set the assets
• Add asset display & selection to the admin ui.

This issue is a discussion of the high-level implementation of Orders

Overview

Orders represent a collection of ProductVariants which a Customer is in the process of purchasing or has purchased.

Prior Art

• https://guides.spreecommerce.org/developer/orders.html
• https://guides.solidus.io/developers/orders/order-state-machine.html
• https://docs.sylius.com/en/1.2/components_and_bundles/components/Order/state_machine.html

Order States

An Order can be thought of as a finite state machine with the following states:

1. AddingItems: The first time a Customer adds a ProductVariant to an order, a new Order entity is created in the "AddingItems" state. A new OrderItem entity is created for each ProductVariant which represents the variant in the given quantity. This state continues until the checkout stage.
2. ArrangingShipping: Once all items are added, the shipping can be arranged. At this stage the shipping destination is set so that the shipping cost may be calculated.
3. ArrangingPayment: Once shipping has been calculated, we know the final cost of the Order and can arrange payment. At this point the order details can be sent to a payment provider. This state can be skipped if the order total is 0.
4. OrderComplete: Payment has been received and the order is completed.
5. Cancelled: A completed order may be cancelled after it is placed.

User-defined states

It would be useful to allow the developer to define additional states for the Order to match the particular business requirements of the shop. I'll look for existing FSM libs to see if this can be accommodated in a safe and intuitive manner.

Transition Hooks

When transitioning from one state into another, one or more functions can be called to execute some logic before the transition is made, and possibly also have the opportunity to stop the transition.

For example, when an Order transitions from "ArrangingPayment" to "Completed", a hook could generate and send a transactional email confirming the order.

Cost Calculation

The cost of an order must take into account the following factors:

1. The price of the ProductVariant * quantity for each OrderItem
2. Taxes applied to each OrderItem
3. Adjustments applied to each OrderItem (e.g. promotional discounts - buy 1 get one free)
4. Adjustments applied to the entire order (e.g. promotional discounts - £5 off order with voucher code)
5. Shipping costs (which is another type of Adjustment)

Currently FacetValues can only be added to ProductVariants (see design in #2). With further development it seems apparent that in many cases it makes more sense to have FacetValues primarily set at the Product level:

• In the product list view we can also list FacetValues
• It is much simpler to add them to a Product rather than selecting all Variants and then adding to all.
• Reduce the visual noise of the ProductVariant list by eliminating the shared FacetValues
• In certain cases where individual Variants have FacetValues not shared by other Variants in the Product, they can still be added at that level.

This should be a relatively simple change:

1. Add [FacetValues!]! property to Product type schema + create / udapte Inputs.
2. Implement the logic to link them in the service layer.
3. Display them in the UI, list/detail components.
4. Update the product import format to account for the 2 types of facet.

How should the admin ui be deployed?

Initial thought is to bundle it with the server npm package, so that a single install can get everything up an running. However, in real production apps, it is possible that the admin ui is run on a separate server than the API, so it should also be available stand-alone somehow.

For the initial release, it would probably be sufficient to bundle it with the server and serve it up on a pre-defined endpoint such as http://localhost:3000/admin (the path could be a configurable property of the VendureConfig).

It is missing from recently-implemented resolver methods.

Current config allows setting the JWT secret only. We should expose other options as in: https://github.com/auth0/node-jsonwebtoken#jwtsignpayload-secretorprivatekey-options-callback to allow more fine-grained control.

There are a number of circumstances in which an ecommerce shop might want to generate and send an email to a customer, e.g:

• Account creation / welcome
• Email address verification
• Order confirmation
• Shipping confirmation
• Cart abandonment
• Request for feedback on an order

There are 3 aspects to transaction emails: events, email creation, and sending.

Events

An event would be some point in a process where listeners could be notified and then decide whether or not to create and send an email. Typical events would include: account created, order transitions to a certain state.

We already have hooks in those processes governed by the FiniteStateMachine, but this is not enough. For example, a cart abandonment email is not necessarily triggered by any kind of state transition, but perhaps an automated task that runs periodically.

Therefore it would make sense to have some kind of generic event bus service (#40), whereby various events are published and then the email handler can subscribe to any of these and thereby send an email at the right moment.

Email creation

This step occurs in response to an event and is concerned with generating the body of the email. It would use some kind of templating system such as Handlebars which could be passed pertinent information from the event object (e.g. the Order and Customer entities in the case of a hypothetical OrderStateTransitionEvent) and would use this data to interpolate into the template. The result would be typically a string of HTML.

Email sending (transport)

A "transport" is a method of actually sending the mail to the recipient.

SMTP is the most common way of sending an email. A business will either use their own SMTP server, or use a service such as Mandrill or SendGrid

Local - A local email server may be used such as sendmail may be used to directly send the email.

Other - For testing purposes a mock transport may be used which simply writes the email to a file on the local file system.

Nodemailer supports all of the above transport out of the box, and new transports can be written.

The permissions handling in the RolesGuard decorator is buggy and needs a comprehensive set of e2e tests which:

1. Create a bunch of different Roles with various permissions
2. Attempt various queries & mutations
3. Assert 200 / 403 depending on whether that role should be allowed to perform the given operation.

First depends on building out the createRole & assignRoleToUser mutations, which we can then use in the e2e test script to set up the tests.

While designing the Emails system, it became apparent that a generic event bus would be beneficial.

Here is the problem statement that EventBus solves in a nutshell:

"I want an easy, centralized way to notify code that's interested in specific types of events when those events occur without any direct coupling between the code that publishes an event and the code that receives it." source

The rationale of why is makes sense for email handling is outlined in #39.

Other use-cases:

• Logging
• Analytics
• Integration with 3rd party services which want to know about specific events

Design

The basic design is a singleton service, EventBus, which has a subscribe method and a publish method:

abstract class VendureEvent {}

class OrderStateTransitionEvent extends VendureEvent {
  constructor(private order: Order, private customer: Customer) {}
}

class EventBus {
  subscribe<T extends VendureEvent>(event: T, handler: (event: T) => void) {
    // ...
  }
  publish(event: VendureEvent) {
    // ...
  }
}

// order-state-machine.ts
onTransitionEnd: (fromState, toState, data) => {
  // ...
  this.eventBus.publish(new OrderStateTransitionEvent(
    data.order,
    data.customer,
  ));
  return this.onTransitionEnd(fromState, toState, data);
}

Built-in events

The build-in events should include:

• OrderStateTransitionEvent
• PaymentStateTransitionEvent
• AccountRegistrationEvent
• LogInEvent
• LogOutEvent

There will probably be a few more that make sense, which can be added as needed.

Currently we are using shared types & utils from both server & admin-ui, and we end up with ugly import paths like:

import { CustomFieldConfig } from '../../../../../../shared/shared-types';

I think there is a way to configure TS to resolve the "shared" dir so we could instead use:

import { CustomFieldConfig } from 'shared/shared-types';

Sometimes the loading indicator in the admin ui gets "stuck" in the loading state, which means that the loading counter never decrements back to zero.

When this occurs and yet there are no API requests in flight, then there is a logic error somewhere, which should be found and fixed.

It is currently in beta, so once it hits stable, we should upgrade: https://vmware.github.io/clarity/news/1.0.0-beta.2

Create a basic administrator module for the admin ui, including:

• CRUD on Administrators
• CRUD on Roles
• Assigning Administrators to Roles

Note - this module should not deal with Customers, even though Customers and Administrators are both a type of User. This is because, to the end-user, they are very different kinds of entities. Plus, only the SuperAdmin would typically be dealing with the Administrator CRUD operations, whereas viewing and editing Customer data would be routine for any kind of administrator.

Reference: https://en.wikipedia.org/wiki/Payment_gateway#Typical_transaction_processes

Payment would typically be handled by a 3rd party payment provider such as PayPal, Stripe, WorldPay, etc.

The typical flow is to make a call to the payment API with the payment information (and possibly the order details). The payment is processed asynchronously and then on success a token is returned. The payment source (card) may be charged immediately or at some later point (e.g. upon shipping).

High-level process

1. Customer checks out and sets shipping address & shipping method.
2. Customer enters payment information somewhere (directly in store app or on payment gateway website)
3. Payment is authorized.
4. Payment is settled (money is moved from customer account to merchant)

Payment Workflows

There are a number of possible payment workflows used by various payment gateways and methods:

1. Client-side redirect

In this workflow, the customer is redirected to the gateway. The customer then enters all data on the gateway website. Upon completion (approval, clearing or declined etc), the customer is redirected to some specified page where the completion payload is delivered e.g by a POST request, or by a JavaScript callback.
Example: WorldPay Business Gateway

2. Client-side gateway API

In this workflow, the payment data is entered into a form on the storefront (typically provided as a ready-made JavaScript lib by the payment gateway), and then this data is securely posted via XHR to an API endpoint provided by the gateway. The card is authorized and the payment is settled in a single step. The response contains the token etc. which is then stored as a Payment in Vendure.
Example: PayPal Checkout

3. Client-side authorization, server-side settlement

In this workflow, an initial call is made to the payment gateway from the client side to set up and authorize the payment. A token is returned but the charge has not been settled. This token is then used to create a Payment in Vendure with the "authorized" state. This token can then be used for more advanced workflows, such as only settling the payment upon shipment or storing s reference to the customer & card for easy repeat payments (e.g. Stripe Checkout.

4. Server-side-only processing

Some websites store payment data for convenient reuse in repeat orders. This means that the payment data is sent to the server, and the authorization process is carried out between the server and gateway, rather than the client and gateway. This workflow implies PCI DSS compliance which we will not get into.

Vendure should support the first 2 workflows natively. The third can be supported with custom code by those businesses which have the resources and will.

Case study: PayPal Checkout (client-side only)

1. After shipping method is selected, storefront implements the PayPal Checkout integration which opens a popup which allows user to make payment via PayPal.
2. PayPal authorizes and settles the payment and returns a token representing the successful transaction.
3. The token is sent to Vendure via a call to an addPaymentToOrder mutation.
4. A Payment entity is created in the Settled state.
5. The order would now be in a state of PaymentsSettled.
6. The addPaymentToOrder mutation returns the Order, and since it is now settled, the customer is shown the "thank you for your order" page.

Case study: Stripe (client & server side action)

1. After shipping method is selected, storefront implements one of the various Stripe widgets for taking payment, e.g. https://stripe.com/docs/quickstart#collecting-payment-information
2. Stripe process the payment and return a token authorizing the payment. At this point, the funds have not yet been settled.
3. The token is sent to Vendure via a call to an addPaymentToOrder mutation.
4. A Payment entity is created in the state Authorized.
5. The Payment is transitioned to the Settled state, and in the step a hook is used to settle the charge against the Stripe charges API using the authorization token. Upon success the hook returns true so that the transition can complete.
6. The order would now be in a state of PaymentsSettled.
7. The addPaymentToOrder mutation returns the Order, and since it is now settled, the customer is shown the "thank you for your order" page.

Action items

PaymentMethod

The PaymentMethod defines the states and transition hooks for a given payment gateway. When the addPaymentToOrder mutation is called, Vendure would look up the matching PaymentMethod and then apply a method to create the Payment entity and transition its state as per the case studies above.

The PaymentMethod would be a TypeScript class passed into the VendureConfig, which would then auto-generate a corresponding entity in the database which would allow customization of variables (such as API key used in calls to the Stripe API) which would be stored and manipulated in the same way as Promotion conditions / actions.

Payment entity

A Payment represents an amount paid towards the Order total. Usually it will equal the total, but might not in the case that multiple payments are made for a single Order.

The Payment would store the state of the transaction (authorized, cleared, declined etc), the token returned on success, plus any other pertinent information which might be useful to keep such as IP address and specific info returned by the payment provider.

addPaymentToOrder mutation

This mutation would take a token, the code of the PaymentMethod and possibly other pertinent data and create a Payment entity associated with the given order.

Rework default Order states

The default states should include:

• "PaymentsAuthorized"
• "PaymentsSettled"

A simple, easy-to-use search tool is critical for a successful ecommerce site. If people can’t find your products, they can’t buy them. A bad in-site search can frustrate users enough to make them abandon you for a competitor. source: Neilsen Norman

Search functionality is one of the most important factors of an ecommerce app. Vendure will ship with a built-in SQL-based search solution, and will be easily extended with more specialized search plugins.

Implementation

There should be an interface which is common to all search plugins, so that plugins can be easily swapped out. This interface would include:

• A method for creating the search index (see below)
• A method for updating the search index selectively upon changes to any of the input entities.
• A method to perform a search, given a search term, optional facets, optional categories, optional sorting, optional filtering by any property.

Search Index

The existing products endpoint can be used for a rudimentary search but is not ideal because it involves potentially expensive joins of many tables.

Instead, a separate search index would be created which denormalizes these entities into simple rows for easy search and filtering. For external tools such as Elastic, this is also the approach used.

Since the index is denomalized, it will become stale upon changes to any of the related entities. For this reason it must be updated when a Product, ProductVariant, Facet, FacetValue or Category changes. The naive approach would be to rebuild the entire index on any change, but this will likely be inefficient. A better way to do it would be to create a format which provides the changed entity, and the indexer can then determine which records are affected by that change and update only those rows.

Potentially the EventBus system can be used to trigger updates to the index.

External search services

Some sites may opt for a hosted 3rd party search solution such as Algolia. In this case, the indexing would still be handled by the plugin. In the case of Algolia, the plugin would use the Algolia API to push records into the index.

On the other hand, the actual search request by the customer may go direct to the search provider (as is recommended by Algolia), rather than going through the Vendure server which would act as a proxy. Both patterns should be supported.

Search endpoint

The search endpoint is what would be used by the storefront client app to return search results. Since different search engines may have vastly differing query formats, it makes sense to have a generic JSON input type for the search input, and then have the search plugin transform and forward that data on to the search engine.

Search signals

The following signals should be supported by the default search plugin:

• weighted keyword matches
• facets
• categories
• applicable fields of Products & ProductVariants such as price
• custom fields of Products & ProductVariants
• weighted review score, i.e. score derived from avg review score and number of reviews (review functionality yet to be implemented).

"Advanced" search

Advanced search includes things like boolean operators in a search term. Research indicates that the vast majority of ecommerce users do not use such features.

In our recent search study, the mean query length was 2.0 words. Other studies also show a preponderance of simple searches. Most users cannot use advanced search or Boolean query syntax. source: Nielsen Norman

Therefore we will not support things like operators in the default search. Custom plugins are free to implement such features.

When creating new entities with the create... mutations, the newly-created entities are not added to the Apollo cache.

Looks like we need to use the update or updateQueries property of the mutate() method: https://www.apollographql.com/docs/angular/features/cache-updates.html#directAccess

Currently, auth is still handled via a REST call. The AuthController should be replaced by a GraphQL resolver.

Channels represent distinct sales outlets. Many web shops will just have a single, default channel, which is the website. However, there are cases when it would be desirable to be able to define distinct sales channels, such as:

• Websites for a foreign markets
• Alternative websites marketed to different audiences selling subset of all products in catalog
• Mobile app with different (but overlapping) product range

Thus the following aspects could vary from one channel to the next:

• Product availability
• Taxes
• Payment methods
• Shipping methods
• Default language
• Default currency code
• User role

Prior Art

• Sylius has a comprehensive Channels implementation which covers all of the above: https://docs.sylius.com/en/1.2/book/configuration/channels.html
• Magento has "stores" which seem analogous to Channels: https://docs.magento.com/m1/ce/user_guide/store-operations/stores-multiple.html
• VevoCart Multi-Store feature: http://www.vevocart.com/multi_store_ecommerce-content.aspx

Design

Adding Channels would quite significantly increase the complexity of the models. Here is a rough idea for an implementation:

• A Channel entity is created, which has (among other properties) an id and name. By default, there is a "default" channel which must exist. Any number of additional channels may be created.
• A Product has an available property relative to each Channel. This could be as simple as a one-to-many relationship from Product -> Channels.
• A ProductVariant's price would now be associated with a particular Channel.
• Likewise, TaxAdjustment and ShippingAdjustment entities (once implemented) would need to relate to a particular Channel.
• A User's role would be relative to a given Channel. Thus you can have Administrators who can view and administrate only selected Channels. This will be tricky to get right.
• The current Channel should somehow be implicit, so that we don't need to go and add a channelId argument to every query / mutation. Perhaps the active channel id can be stored in the JWT info so that it can be figured out once and then just persists for the session.

A Customer can be a guest (has no associated User) or registered (has an associated User).

Account creation

An account can be created in one of two ways:

1. Ad-hoc account creation by the customer outside the order process
2. Conversion of a guest account into an authenticated account after the placing of an order.

As the diagram shows, a customer does not need to create an account after placing an order. Indeed, the customer is free to only every do guest checkouts. For the checkout, however, the email address would be used to create a guest Customer, and then on subsequent checkouts with the same email address, the same guest Customer would be assumed.

At any time, the guest Customer can be converted into an authenticated Customer by registering in either of the two ways outlined above.

Registration process

There are 2 questions to resolve regarding registration:

1. Do we support 3rd party authentication methods in the future (social login, Google account etc)? If so, what (if any) changes to the current User entity would be needed?
2. Do we require email account verification? If so, what is the purpose? Do unverified users have a restricted set of possibilities compared to verified ones?

Account Deletion (GDPR)

A user should have the ability to delete their account, as should an administrator. In this case, we want to keep the physical row in the DB for the purposes of data consistency and reporting, but we should completely remove any personally identifiable information (PII):

“[A]n identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person.” source

Overview

• A typical web shop has one or more images for each product. Each individual variant may also have one or more images (e.g. a paint range, where each variant is a colour with its own colour-swatch image).
• Videos are also a common requirement, and might be hosted on the server itself, or linked from a dedicated video server or video service like YouTube.
• Other binaries might be used, such as pdfs with spec sheets or instruction manuals.
• An administrator should be able to easily create new media objects by uploading or supplying a link.
• There should be some means to browse existing media objects in the admin-ui, so that existing objects can be re-used.

Design

A Media entity would represent all types of binary files. It would have a type discriminator whose value would be a MediaType (image, video, etc).

Physical storage

The Media entity would have a pointer to the actual binary file somewhere on a disk. The physical storage location should be configurable - it should not have to reside on the same machine as the Vendure API server. (look into existing Node libs which abstract away the file system).

Previews / thumbnails

There should be a built-in way to generate different sizes of image whilst preserving the original. This could be generalized into a "preview" (or better name) property which would be a collection of one or more MediaPreview entities. This would also then handle the case for preview images of videos, pdfs and other file types.

Another way would be to have a single "preview" image which can be resized on-the-fly with query parameters. These generated previews would then be cached so that the re-sampling only need take place for the initial call. This is much simpler but has performance implications.

Maybe the way to go would be to automatically generate a default set of previews (according to some config), and then further permutations can be generated on the fly.

It would be good to abstract the actual image-manipulation lib away so that the image processing capabilities can be extended by the developer, so in essence we pass through the query string to the configured "image plugin" which can then pass off the work to some library. For example, a developer may want to use ImageMagick rather than a JavaScript-based lib for improved performance.

Linking with other entities

The Product and ProductVariant entities would have the existing image string property replaced by a media property which would be a collection of Media entities.

Additionally, the custom fields API should be extended to allow Media fields with optional type filter.

Where there is more than 1 Media entity, there must be a way to choose the "default" one, i.e. the image to show in a list view of Products, for example.

This could be done by:

• adding an additional defaultMedia property which stores the id of one of the linked Media entities.
• using the order of the collection, so that the 0th element is considered the default.

I would tend to favour the first solution, since it is more explicit.

Media browser

A media browser is a UI component which is used in the admin-ui to select an existing Media entity. As a first implementation this would be akin to the media browser in WordPress - essentially a flat (no folders), filterable list of all Media entities.

For large shops this might become unwieldy - our reference shop has over 6,000 individual images. Therefore a future extension will have to address the issue of organizing large media collections, with folders or tags or something like that.

References

• Handling file uploads with Apollo Server: https://www.apollographql.com/docs/guides/file-uploads.html
• JIMP - image manipulation lib: https://www.npmjs.com/package/jimp
• Perf of JIMP vs native-based image libs: jimp-dev/jimp#46

The JWT generated on login is time-limited. Once it expires, the session is no longer valid and the user must login newly.

For a long, continuous user session, this is undesirable. The token should be refreshed during use. Needs research on how this is best achieved.

Secondly, if user is inactive for an expended period (e.g. puts laptop to sleep, opens next day and admin ui is still open in browser), then the next API call will fail with a 403 error. In this case, the app should automatically re-route to the login page, and upon successful login, redirect the user back to the route they were last requesting before the error.

Facets are properties which can be assigned to a product / product variant and are used to encode properties of that product along arbitrary dimensions. Reference: https://www.sitepoint.com/cms-content-organization-structures-trees-vs-facets-vs-tags/

Examples of facets

• Brand: HP, Dell, Apple, Microsoft
• Form Factor: Laptop, Desktop, Tablet, Hybrid
• Memory: 4GB, 8GB, 16GB, 32GB

Purpose of facets

Facets are used to add structured metatdata to a product for the purpose of categorization and filtering.

A typical use-case is to have a filter-based product-discovery interface e.g. like Asos which allows filtering by various facets. Amazon also use this approach.

If a shop would prefer a tree-like category-based navigation, this can be built up via pre-defined combinations of facets which are grouped into a category. Further design on facet-based categories to follow.

Questions to resolve

1. Should facets be associated with a Product, a ProductVariant, or both?
2. Should the range of facet values be fixed to a pre-defined list of values, or should new values be generated on an ad-hoc basis (like how tags work in WordPress)?
3. How should Facets and FacetValues be stored in the database? A Facet entity with a one-to-many collection of FacetValues? Or is there some simpler way to store them?

Up until now all development and testing has been done against the source using ts-node.

Ultimately, Vendure will be an npm package and we will be delivering transpiled JavaScript with accompanying d.ts files.

To build for distribution, the following needs to be solved:

1. Transpile into a portable package. Currently, the use of "paths" for the shared (server / client) code causes problems on a simple tsc build. It is likely an additional build step (e.g. Gulp) will be needed to do a bit of work on getting the paths to work correctly in the dist build.
2. Include any needed non-code files such as package.json in the dist build
3. Use barrel files to re-export those types which may be required by the end-user, so that they do not just have to deep-import everything from the original folder structure.
4. Test with yarn link (or similar tool) to simulate a real npm install to ensure all works as expected.

There is currently no way to set which languages should be available to translate translatable entities into.

Where should the available languages be set?

• Per Channel. This would not work if there are translatable entities which are not channel aware.
• Globally. Then we would need some global settings location. We already have the VendureConfig object, but perhaps there should be a database-level config registry for dynamic settings which can be changed via the admin ui.

We are currently on 1.3.6. v2 is broadly backwards-compatible but since we are using the Nest GraphQLModule, we need to wait until v2 is properly supported.

There is an thread in which some people get it to work, but there currently seems to be an issue with the Passport.js integration.

Best to wait for a bit to see if these issues get resolved.

Then we can have the same version of TypeScript for both server and UI, which will make development a bit smoother.

I read this article: https://refactoringui.com/previews/building-your-color-palette/ and now I want to use this system for the admin ui. We already have something not too far off, but we are missing the explicit shades of each colour.

Hiho, I am just followed the readme (populated via yarn and then started server and admin-ui). When I try to login I will get: "No valid channel was specified".

{"errors":[{"message":"No valid channel was specified","locations":[{"line":2,"column":3}],"path":["config"],"extensions":{"code":"NO_VALID_CHANNEL","exception":{"stacktrace":["Error: error.no-valid-channel-specified"," at RequestContextService.getChannelToken (/home/uter/src/vendure/server/src/api/common/request-context.service.ts:59:19)"," at RequestContextService.fromRequest (/home/uter/src/vendure/server/src/api/common/request-context.service.ts:33:35)"," at AuthGuard.canActivate (/home/uter/src/vendure/server/src/api/middleware/auth-guard.ts:40:65)"," at process._tickCallback (internal/process/next_tick.js:68:7)"]}}}],"data":null}
Cheers,
Christoph

Currently the channel code is generated in a random way:

This means that every time the "populate" script is run, a new code is generated and makes the population of data non-deterministic. This then adds complexity to e.g. the test client which needs to obtain the new code each time. Also the storefront app always needs to be updated.

Better would be to allow the code to be supplied in some way, probably in the VendureConfig object.

Prettier v1.15 supports pretty-printing of Angular templates: https://prettier.io/blog/2018/11/07/1.15.0.html