The general practice to terminate any console process is to use Ctrl+c one more multiple times. Even in the SharePoint world, see SPFx's gulp serve, yo teams gulp serve & gulp ngrok-serve, etc.

Please be consistent... currently the instructions say to press ENTER... please change to follow established conventions for console apps: Ctrl+C

This feature introduces a file system monitoring hook on the responses file, so that when the responses file is modified, the proxy will automatically refresh the responses without having to restart it.
If the responses file has been deleted, the proxy will clear the responses it read previously from its memory.
Proxy also monitors the current working directory for changes, so that when a new file responses.json is created, it will automatically read its contents and setup mock responses.

We should move the Move to the Graph JS SDK guidance to our wiki so that it's in the same place as the rest of our docs and we can update it more easily.

Right now, we put the rootCert.pfx file in the ~/.config folder which is a generic folder that many apps on macOS use to store their config. By looking at the file, it's not clear that it belongs to the proxy, and could lead to folks deleting it. We should see if it's possible to adjust the config path to ~/.confg/msgraph-developer-proxy/ so that it's clear which files belong to the proxy and are safe to delete after uninstalling the proxy.

Graph Chaos Proxy can play a vital role in helping developers test their applications. By responding with preconfiguring responses, it can mimic different states of the Graph API and M365 data allowing developers to test their application with different conditions.

This feature allows developers to define responses for specific Graph calls.

Responses are defined in a JSON file named responses.json located in the current working directory from which the proxy has been started. This allows developers to use different responses with different projects.
The structure of the responses file is as follows:

{
  "responses": [
    {
      "url": "https://graph.microsoft.com/v1.0/me",
      "responseBody": {
        // ... mock response
      }
    },
    {
      "url": "https://graph.microsoft.com/v1.0/me/photo",
      "responseCode": 404
    },
    {
      "url": "https://graph.microsoft.com/v1.0/users/*",
      "responseBody": {
        // ... mock response
      }
    },
    {
      "url": "https://graph.microsoft.com/v1.0/users/*/photo/$value",
      "responseBody": "@picture.png"
    }
  ]
}

Requests are matched in the order they’re defined in the responses.json file, so it’s up to developers to define which responses take precedence.

If an intercepted request matches a pattern in the responses.json file, the proxy responds with the mock response for that request as defined in the responses.json file. If there’s no matching mock response, proxy will reply with a random error response, the same way as if there was no responses.json defined.

Responses from the responses.json file are read and stored in memory when the proxy starts. After changing responses file, you need to restart the proxy to read the new contents.

Providing mock responses can be disabled even when a responses.json file is present by using the --no-mocks option on the command line.

• extend configuration with specifying mock responses
• load configuration from the current working directory
• respond to matching URLs with mocked responses when available
• match URLs using wildcards
• add support for setting response headers on mock responses (needed for properly handling binary responses)
• add support for binary responses using @-tokens
• extend configuration with the --no-mocks flag
• reload config when responses changed

If an intercepted call falls within the failure rate %, the proxy returns one of the faulty responses at random (429, 500, 502, 503, 504). If the call is outside the failure rate %, the call is passed as-is to Graph and the response from Graph is returned to the caller.

By using the proxy in CI/CD pipelines, dev teams can verify that all committed code properly handles service errors. We should research the scenario of how the proxy could be used in a GitHub action that's a part of a PR validation workflow. Things to research:

• how would developers install the proxy in a GH workflow
• how would developers configure and start it
• how would developers capture results and expose them in build results (artifacts)
• how can we streamline this process to make it as seamless as possible
• how we can express test suites and -cases in proxy

We currently provide install instructions for Windows and macOS in the Get started section.

We should also provide instructions for Linux (Ubuntu).

Microsoft Graph Developer Proxy supports the use of national cloud deployments, however we don't call out what steps developers need to take to take advantage of this in our guides, we should add a How-to guide to cover this.

Because the proxy supports more than just chaos, let's rename the project and all its artifacts to Graph Developer Proxt to properly reflect this change.

Enabling the proxy on macOS requires manual steps. In comparison, on Windows, after starting the proxy, it automatically registers as a system-wide proxy. We should look for ways to simplify the steps, either via a script people can run, or have the proxy automatically register as a system-wide proxy, just like it does on Windows.

We should consider adding support for plugins so that third parties can add functionality beyond what we're offering and support scenarios specific to their organization.

There is an error in our yml:
https://github.com/microsoftgraph/msgraph-developer-proxy/actions/runs/3567052507/workflow

When a $batch request fails due to a throttling issue the request itself has a response status code of 424, while one or more requests within the batch have a status 429. When failing a $batch request the proxy should fail all requests within the batch with differing Retry-After values, the highest of these should be used to determine if the retried request may succeed.

Source: https://learn.microsoft.com/en-us/graph/throttling#throttling-and-batching

By default, the proxy will run on port xyz. This feature allows configuring the proxy to run on the number specified using the --port option in the command line, when starting the proxy, eg. --port 1234.
If the port is already in use, the proxy will close with an error saying: “Port xyz is already in use. Please use a different port”.
To specify a different port or use the default port, you need to restart the proxy.

To highlight the value of using the Graph SDK and the Chaos Proxy, we should include samples in the repo that we can share with developers and highlight in the video that we submit with this project.

We should focus on samples that

• Have minimal dependencies (nodejs)
• Use nodejs
• Use script to create Azure AD app registration
• Use Graph JS SDK
• Are contained within a single file
• Reuse sample content in developer tenants

To simplify acquisition, we’ll publish the proxy to winget and homebrew. That way, customers will be able to acquire the proxy using the tools that they’re already familiar with. As P0, we’ll publish the release on GitHub.

• #636
• #637
• #647

Extend the proxy with ability to test slow API responses.

Rationale

One aspect of testing how applications use an API is how they respond to slow API responses. Is it obvious that content is loading? Is the whole application unresponsive? Is the application failing because it expects a response in a certain timeframe? Graph Developer Proxy can help developers validate the user experience when the API responds slowly.

Spec

• introduce app setting named minDelayMs, that defines the lower range of the delay in milliseconds added to API response. Default value 500
• introduce app setting named maxDelayMs, that defines the upper range of the delay in milliseconds added to API response. Default value 10000
• introduce option named --min-delay that allows users to override the value of minDelayMs as configured in appSettings. When not specified, uses the value from appSettings.
• introduce option named --max-delay that allows users to override the value of maxDelayMs as configured in appSettings. When not specified, uses the value from appSettings.
• introduce option named --slow-responses which is a flag. When used, it will cause the proxy to add a random delay from range minDelayMs <= x <= maxDelayMs to each API response (mocked or not)

Add a page to the docs to explain what the proxy is and how to use it

This feature extends monitoring of Graph requests with default responses for the US government cloud (https://graph.microsoft.us), US DOD cloud (https://dod-graph.microsoft.us) and China cloud (https://microsoftgraph.chinacloudapi.cn).

Customers can start the proxy on their machine. They can register it either as a global proxy for all network traffic or as a proxy for a specific application.

Right now, in order to mock Graph API responses with the Proxy, you need to manually build the responses file. We should allow creating responses from a HAR file, to simplify the process. To start, we'd support HAR files that are generated by Chromium-based browsers (Edge, Chrome) and consider extending to other sources in the future.

Spec:

• introduce new option --har-file that customers can use to point the proxy to a .har file
• when customer uses the --har-file options, the proxy starts, generates mock responses and closes
• generated responses are sent to stdout. That way customers can decide themselves how they want to handle them: pipe to another command or save to a file using output redirection > responses.json

Right now, proxy's UI is a continuously scrolling output with messages of different levels. When used with apps that execute many requests to Graph, it's hard to find the issues and suggestions reported by the proxy. We should research the opportunity of terminal UI (TUI), such as Terminal.Gui. It would give us the ability to build a UI and separate the scrolling output, from messages that we want to highlight. It would also allow us to show counters and other information relevant when we start looking into implementing behaviors to make throttling more realistic.

Document the steps to chaos test Azure Functions that use Graph:

• deploy chaos proxy so that it's accessible by Azure Functions
• update Azure Functions code to use the proxy when calling Graph

Today when it comes to SharePoint/OneDrive applications we still see a many applications that take aa hybrid approach in which they use Graph for core scenarios but fall back to either SharePoint REST or SharePoint CSOM for the operations that covered via Graph APIs.

In order to test such applications having support for SharePoint REST/CSOM would really help. The proxy could dynamically detect SharePoint URLs being hit (*.sharepoint.com, *.sharepoint.us,...) and that would work for the majority of cases. For vanity URL tenants ideally there's an option to specify the wildcard URL on which the proxy will take action.

This is a great tool to help developers, looking forward to see it's scope widen by supporting SharePoint REST/CSOM calls.

Following a recent Twitter discussion between @gavinbarron and Chris Johnson on this topic, we should consider adding a page to our Wiki in the Background information section to explain the position of using the proxy alongside other testing approaches to help inform developers of the value add from a testing perspective.

Chris: Why this vs. a unit test? It would have thought simulating errors etc should be part of a test suite? This seems good for dev quick play, but not for full dev cycle?

Gavin : Great question!
Unit test should absolutely be part of your testing suite. This allows you to simulate failures in your running application. Everything from 404s and 429s to 507s.
It's a lot easier to understand a failure case when you can trigger it on demand.

Chris: I guess when running an app, loaded up with usage, and being able to introduce errors would be helpful. For things like testing your code when the APIs return 429s etc. I usually build a test that exercises those scenarios vs. running the whole app.

Gavin: Depending on your codebase, in some cases mocking an http response with a specific status and body can be very hard.
I remember it being a collosal yak shaving effort when we were using RestSharp as our Http request library.

Chris: for sure. but to know you don't break your error handling etc. you gotta shave dem damn yaks! Now, shipping a test graph client lib (to simulate errors on demand from tests) would really really help.

Document the steps needed to use Playwright to automate chaos testing a web app connected to Microsoft Graph.

Running a single page application in the browser that uses the /v1.0/me request with the proxy configured to use mocked responses and a failureRate of 0 i.e. --failure-rate 0 --port 8080 results in CORS errors.

We should not intercept OPTIONS requests and ensure that Access-Control-Accept-* headers are set on mock responses.

We provide an appSettings.json file which stores default configuration for the proxy, we should add a How-to guide which shows how developers can update the default configuration, so that developers don't need to remember the command line operations to configure the proxy to their needs.

The proxy intercepts HTTP calls to graph.microsoft.com. Calls to other domains are passed as-is to the server and the reply is returned to the caller.

Set up automated releases with GitHub actions, so that we can automatically get a release for Windows, macOS and Linux.

When debugging multiple page loads of a web app, it would be helpful to be able to clear the previous proxy output to more easily see what's current.

Update readme with the new name and any other information that's needed when the repo becomes public.

When we release a new version of the proxy, we want to be able the old version to show a message on startup notifying of the new version being available and instructions to download and update it.

Before we commit actual engineering resources to this new project, I think it'd be a good thing to run a competitive analysis with these existing solutions:

• https://stoplight.io/open-source/prism
• https://saucelabs.com/platform/api-testing (the api mock part)
• ...

The goal would be to understand:

• what we want to achieve with this new project
• how much existing solutions cover
• is it worth building our own thing, or should we add to what's existing

In the proxy when we detect that a request to Microsoft Graph has been made without using an SDK, we provide guidance on how to migrate your your app to use the SDK in the form of a page in the Wiki, which will be converted to a How-to guide #59.

We should also provide a one pager that explains why we make this suggestion and the benefits of migrating to use an SDK, this way we can focus on the explanation without getting bogged down into the details, which we can refer to as a CTA for the page.

When a 429 failure is returned to the caller it may provide a Retry-After header, e.g. Retry-After: 10. The proxy will hold an in-memory set of 429 failed requests by HTTP verb and URL, repeated calls to the proxy for a failed request which have not waited the indicated time period will automatically be failed with the same Retry-After value.
A small percentage of 429 responses will not report a Retry-After value to the caller but will internally track this value to encourage the developer using the proxy to implement an exponential back-off when no Retry-After is provided in the response.

When building apps that use Microsoft Graph, developers struggle with figuring out which set of permissions their app needs. They need to consider all the different Graph API calls that are issued on a page, for each one find out the necessary scopes, combine the scopes for the different calls and from there deduct the set of minimal permissions needed for the app. They need to do this for the different routes of their app, or for the whole app if they don't want to/can't use dynamic consent. This process is labor-intensive and error-prone.

The Proxy could simplify the process as follows:

• developers put the proxy in record mode, where the proxy keeps track of the different Graph API calls
• developers use their app to issue Graph API calls. Here, they can decide if they want to get scopes for the whole app or just a part of it
• developers stop the recording
• Proxy gets the list of unique endpoints called in the recorded session. For each endpoint, it retrieves the necessary permissions from the Graph dev API.
• Proxy makes a unique list of scopes necessary for the recorded endpoints and shows it to developers so that they can include it in their app

Depends on #93

By default, the proxy fails 70% of calls to Graph. This feature allows configuring the failure rate using the --failure option in the command line, when starting the proxy, eg. --failure 90, to set the failure rate to 90%.
If the specified value is invalid (less than 0, more than 100 or not a number), the proxy will close with an error saying: “xyz is not a valid failure rate. Specify a number between 0 and 100”.
To specify a different failure rate, you need to restart the proxy.

Hosted runners for actions are taking a very long time to pick up an agent

https://github.com/microsoftgraph/msgraph-developer-proxy/actions/runs/3590639181/jobs/6044235558

When intercepting Graph GET requests that don't include the $select parameter, we should highlight it and instruct developers to optimize them by adding $select with the list of properties they'd like to retrieve. Request without $select are slower because they potentially retrieve more data than actually needed in the app. Specifying the set of properties to retrieve would not only lower the load on Graph but also speed up the app.

Consideration: since there might be cases where apps need the whole data set, we should either allow to suppress this suggestion on specific requests or disable it altogether to avoid returning messages that aren't relevant.

Currently, developers can save proxy configuration centrally by updating the appSettings.json file, which apply these settings globally. However, there may be occasions where developers want to save different configurations for specific projects, which we currently don't support.

We should consider, in the same way which we do for responses.json, providing a project specific way of configuring the proxy.

To further simplify acquisition, we’ll publish the proxy as a Docker container so that customers can pull it from Docker hub without having to run custom executables on their host. This allows them to isolate the runtime but also have a shared instance of the proxy running for the whole dev team.

Please configure a 1ES hosted runner so that we can do cool things

Right now, it's hard to discern the different types of messages in the proxy. What's more, indenting messages makes them even harder to read. We should consider color-coding different types of messages.

Here's our output at the moment:

In comparison, here's an example from the Node proxy:

This is debug output where arrows describe different types of messages (proxy response →/request ←, pass-thru response/request ↑).

Let's look at the different types of messages we log and how we could code them to make them easier to follow.

We should consider including some content into the Microsoft Learn documentation so that developers can discover the proxy when looking at our SDKs and learning material.

This will help get the proxy in front of developers early to use when building apps using Microsoft Graph and include it in their development cycles.

We should consider adding high level content and link directly to our Wiki, so that we don't duplicate content and have to maintain in two places.

Some suggestions, there could and most probably be more.

• Microsoft Graph SDK overview
• Explore Microsoft Graph scenarios for JavaScript development

SharePoint API calls (Graph, REST, CSOM) return RateLimit headers whenever the application is about the consume its resource unit quota (see https://aka.ms/sharepoint/throttling). It would be good if the proxy allowed to mimic that by allowing the user to set the resource unit quota and then the proxy can track the request coming in, calculate the resource units for the request and verify if the quota is about to expire...if so, the proxy can start to include rate limit headers in the response.

.NET 7 has support for tracking and applying rate limits, see https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/

Right now we support throttling using a failure rate: customers specify the rate (likelihood) to fail each requests and if the random value we get matches the rate, we fail the request. While this is convenient for easily testing apps against specific issues, it's not representative of how Microsoft Graph behaves in practice. We should look for ways to implement behaviors, so that customers can configure the proxy either to fail requests like it does now, or follow the realistic limitations as defined at https://learn.microsoft.com/en-us/graph/throttling-limits.