microsoft / dev-proxy Goto Github PK
View Code? Open in Web Editor NEWDev Proxy is an API simulator that helps you effortlessly test your app beyond the happy path.
Home Page: https://aka.ms/devproxy
License: MIT License
Dev Proxy is an API simulator that helps you effortlessly test your app beyond the happy path.
Home Page: https://aka.ms/devproxy
License: MIT License
Extend the proxy with ability to test slow API responses.
One aspect of testing how applications use an API is how they respond to slow API responses. Is it obvious that content is loading? Is the whole application unresponsive? Is the application failing because it expects a response in a certain timeframe? Graph Developer Proxy can help developers validate the user experience when the API responds slowly.
minDelayMs
, that defines the lower range of the delay in milliseconds added to API response. Default value 500
maxDelayMs
, that defines the upper range of the delay in milliseconds added to API response. Default value 10000
--min-delay
that allows users to override the value of minDelayMs
as configured in appSettings. When not specified, uses the value from appSettings.--max-delay
that allows users to override the value of maxDelayMs
as configured in appSettings. When not specified, uses the value from appSettings.--slow-responses
which is a flag. When used, it will cause the proxy to add a random delay from range minDelayMs <= x <= maxDelayMs
to each API response (mocked or not)When a $batch request fails due to a throttling issue the request itself has a response status code of 424, while one or more requests within the batch have a status 429. When failing a $batch request the proxy should fail all requests within the batch with differing Retry-After
values, the highest of these should be used to determine if the retried request may succeed.
Source: https://learn.microsoft.com/en-us/graph/throttling#throttling-and-batching
SharePoint API calls (Graph, REST, CSOM) return RateLimit headers whenever the application is about the consume its resource unit quota (see https://aka.ms/sharepoint/throttling). It would be good if the proxy allowed to mimic that by allowing the user to set the resource unit quota and then the proxy can track the request coming in, calculate the resource units for the request and verify if the quota is about to expire...if so, the proxy can start to include rate limit headers in the response.
.NET 7 has support for tracking and applying rate limits, see https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/
Customers can start the proxy on their machine. They can register it either as a global proxy for all network traffic or as a proxy for a specific application.
We provide an appSettings.json
file which stores default configuration for the proxy, we should add a How-to guide which shows how developers can update the default configuration, so that developers don't need to remember the command line operations to configure the proxy to their needs.
To highlight the value of using the Graph SDK and the Chaos Proxy, we should include samples in the repo that we can share with developers and highlight in the video that we submit with this project.
We should focus on samples that
Right now, it's hard to discern the different types of messages in the proxy. What's more, indenting messages makes them even harder to read. We should consider color-coding different types of messages.
Here's our output at the moment:
In comparison, here's an example from the Node proxy:
This is debug output where arrows describe different types of messages (proxy response →/request ←, pass-thru response/request ↑).
Let's look at the different types of messages we log and how we could code them to make them easier to follow.
We should consider including some content into the Microsoft Learn documentation so that developers can discover the proxy when looking at our SDKs and learning material.
This will help get the proxy in front of developers early to use when building apps using Microsoft Graph and include it in their development cycles.
We should consider adding high level content and link directly to our Wiki, so that we don't duplicate content and have to maintain in two places.
Some suggestions, there could and most probably be more.
Right now, in order to mock Graph API responses with the Proxy, you need to manually build the responses file. We should allow creating responses from a HAR file, to simplify the process. To start, we'd support HAR files that are generated by Chromium-based browsers (Edge, Chrome) and consider extending to other sources in the future.
Spec:
--har-file
that customers can use to point the proxy to a .har file--har-file
options, the proxy starts, generates mock responses and closes> responses.json
When a 429 failure is returned to the caller it may provide a Retry-After
header, e.g. Retry-After: 10
. The proxy will hold an in-memory set of 429 failed requests by HTTP verb and URL, repeated calls to the proxy for a failed request which have not waited the indicated time period will automatically be failed with the same Retry-After
value.
A small percentage of 429 responses will not report a Retry-After
value to the caller but will internally track this value to encourage the developer using the proxy to implement an exponential back-off when no Retry-After
is provided in the response.
Right now we support throttling using a failure rate: customers specify the rate (likelihood) to fail each requests and if the random value we get matches the rate, we fail the request. While this is convenient for easily testing apps against specific issues, it's not representative of how Microsoft Graph behaves in practice. We should look for ways to implement behaviors, so that customers can configure the proxy either to fail requests like it does now, or follow the realistic limitations as defined at https://learn.microsoft.com/en-us/graph/throttling-limits.
Running a single page application in the browser that uses the /v1.0/me
request with the proxy configured to use mocked responses and a failureRate of 0 i.e. --failure-rate 0 --port 8080
results in CORS errors.
We should not intercept OPTIONS
requests and ensure that Access-Control-Accept-*
headers are set on mock responses.
Following a recent Twitter discussion between @gavinbarron and Chris Johnson on this topic, we should consider adding a page to our Wiki in the Background information section to explain the position of using the proxy alongside other testing approaches to help inform developers of the value add from a testing perspective.
Chris: Why this vs. a unit test? It would have thought simulating errors etc should be part of a test suite? This seems good for dev quick play, but not for full dev cycle?
Gavin : Great question!
Unit test should absolutely be part of your testing suite. This allows you to simulate failures in your running application. Everything from 404s and 429s to 507s.
It's a lot easier to understand a failure case when you can trigger it on demand.
Chris: I guess when running an app, loaded up with usage, and being able to introduce errors would be helpful. For things like testing your code when the APIs return 429s etc. I usually build a test that exercises those scenarios vs. running the whole app.
Gavin: Depending on your codebase, in some cases mocking an http response with a specific status and body can be very hard.
I remember it being a collosal yak shaving effort when we were using RestSharp as our Http request library.
Chris: for sure. but to know you don't break your error handling etc. you gotta shave dem damn yaks! Now, shipping a test graph client lib (to simulate errors on demand from tests) would really really help.
Hosted runners for actions are taking a very long time to pick up an agent
https://github.com/microsoftgraph/msgraph-developer-proxy/actions/runs/3590639181/jobs/6044235558
Enabling the proxy on macOS requires manual steps. In comparison, on Windows, after starting the proxy, it automatically registers as a system-wide proxy. We should look for ways to simplify the steps, either via a script people can run, or have the proxy automatically register as a system-wide proxy, just like it does on Windows.
To further simplify acquisition, we’ll publish the proxy as a Docker container so that customers can pull it from Docker hub without having to run custom executables on their host. This allows them to isolate the runtime but also have a shared instance of the proxy running for the whole dev team.
Today when it comes to SharePoint/OneDrive applications we still see a many applications that take aa hybrid approach in which they use Graph for core scenarios but fall back to either SharePoint REST or SharePoint CSOM for the operations that covered via Graph APIs.
In order to test such applications having support for SharePoint REST/CSOM would really help. The proxy could dynamically detect SharePoint URLs being hit (*.sharepoint.com, *.sharepoint.us,...) and that would work for the majority of cases. For vanity URL tenants ideally there's an option to specify the wildcard URL on which the proxy will take action.
This is a great tool to help developers, looking forward to see it's scope widen by supporting SharePoint REST/CSOM calls.
Because the proxy supports more than just chaos, let's rename the project and all its artifacts to Graph Developer Proxt to properly reflect this change.
By default, the proxy fails 70% of calls to Graph. This feature allows configuring the failure rate using the --failure
option in the command line, when starting the proxy, eg. --failure 90, to set the failure rate to 90%.
If the specified value is invalid (less than 0, more than 100 or not a number), the proxy will close with an error saying: “xyz is not a valid failure rate. Specify a number between 0 and 100”.
To specify a different failure rate, you need to restart the proxy.
The general practice to terminate any console process is to use Ctrl+c one more multiple times. Even in the SharePoint world, see SPFx's gulp serve, yo teams gulp serve & gulp ngrok-serve, etc.
Please be consistent... currently the instructions say to press ENTER... please change to follow established conventions for console apps: Ctrl+C
In the proxy when we detect that a request to Microsoft Graph has been made without using an SDK, we provide guidance on how to migrate your your app to use the SDK in the form of a page in the Wiki, which will be converted to a How-to guide #59.
We should also provide a one pager that explains why we make this suggestion and the benefits of migrating to use an SDK, this way we can focus on the explanation without getting bogged down into the details, which we can refer to as a CTA for the page.
We should move the Move to the Graph JS SDK guidance to our wiki so that it's in the same place as the rest of our docs and we can update it more easily.
Right now, proxy's UI is a continuously scrolling output with messages of different levels. When used with apps that execute many requests to Graph, it's hard to find the issues and suggestions reported by the proxy. We should research the opportunity of terminal UI (TUI), such as Terminal.Gui. It would give us the ability to build a UI and separate the scrolling output, from messages that we want to highlight. It would also allow us to show counters and other information relevant when we start looking into implementing behaviors to make throttling more realistic.
When we release a new version of the proxy, we want to be able the old version to show a message on startup notifying of the new version being available and instructions to download and update it.
Set up automated releases with GitHub actions, so that we can automatically get a release for Windows, macOS and Linux.
Currently, developers can save proxy configuration centrally by updating the appSettings.json
file, which apply these settings globally. However, there may be occasions where developers want to save different configurations for specific projects, which we currently don't support.
We should consider, in the same way which we do for responses.json
, providing a project specific way of configuring the proxy.
By using the proxy in CI/CD pipelines, dev teams can verify that all committed code properly handles service errors. We should research the scenario of how the proxy could be used in a GitHub action that's a part of a PR validation workflow. Things to research:
We currently provide install instructions for Windows and macOS in the Get started section.
We should also provide instructions for Linux (Ubuntu).
Microsoft Graph Developer Proxy supports the use of national cloud deployments, however we don't call out what steps developers need to take to take advantage of this in our guides, we should add a How-to guide to cover this.
When building apps that use Microsoft Graph, developers struggle with figuring out which set of permissions their app needs. They need to consider all the different Graph API calls that are issued on a page, for each one find out the necessary scopes, combine the scopes for the different calls and from there deduct the set of minimal permissions needed for the app. They need to do this for the different routes of their app, or for the whole app if they don't want to/can't use dynamic consent. This process is labor-intensive and error-prone.
The Proxy could simplify the process as follows:
Depends on #93
When debugging multiple page loads of a web app, it would be helpful to be able to clear the previous proxy output to more easily see what's current.
The proxy intercepts HTTP calls to graph.microsoft.com. Calls to other domains are passed as-is to the server and the reply is returned to the caller.
Add a page to the docs to explain what the proxy is and how to use it
This feature extends monitoring of Graph requests with default responses for the US government cloud (https://graph.microsoft.us), US DOD cloud (https://dod-graph.microsoft.us) and China cloud (https://microsoftgraph.chinacloudapi.cn).
There is an error in our yml:
https://github.com/microsoftgraph/msgraph-developer-proxy/actions/runs/3567052507/workflow
Document the steps to chaos test Azure Functions that use Graph:
This feature introduces a file system monitoring hook on the responses file, so that when the responses file is modified, the proxy will automatically refresh the responses without having to restart it.
If the responses file has been deleted, the proxy will clear the responses it read previously from its memory.
Proxy also monitors the current working directory for changes, so that when a new file responses.json is created, it will automatically read its contents and setup mock responses.
We should consider adding support for plugins so that third parties can add functionality beyond what we're offering and support scenarios specific to their organization.
Right now, we put the rootCert.pfx file in the ~/.config
folder which is a generic folder that many apps on macOS use to store their config. By looking at the file, it's not clear that it belongs to the proxy, and could lead to folks deleting it. We should see if it's possible to adjust the config path to ~/.confg/msgraph-developer-proxy/
so that it's clear which files belong to the proxy and are safe to delete after uninstalling the proxy.
When intercepting Graph GET requests that don't include the $select
parameter, we should highlight it and instruct developers to optimize them by adding $select
with the list of properties they'd like to retrieve. Request without $select
are slower because they potentially retrieve more data than actually needed in the app. Specifying the set of properties to retrieve would not only lower the load on Graph but also speed up the app.
Consideration: since there might be cases where apps need the whole data set, we should either allow to suppress this suggestion on specific requests or disable it altogether to avoid returning messages that aren't relevant.
Please configure a 1ES hosted runner so that we can do cool things
Graph Chaos Proxy can play a vital role in helping developers test their applications. By responding with preconfiguring responses, it can mimic different states of the Graph API and M365 data allowing developers to test their application with different conditions.
This feature allows developers to define responses for specific Graph calls.
Responses are defined in a JSON file named responses.json located in the current working directory from which the proxy has been started. This allows developers to use different responses with different projects.
The structure of the responses file is as follows:
{
"responses": [
{
"url": "https://graph.microsoft.com/v1.0/me",
"responseBody": {
// ... mock response
}
},
{
"url": "https://graph.microsoft.com/v1.0/me/photo",
"responseCode": 404
},
{
"url": "https://graph.microsoft.com/v1.0/users/*",
"responseBody": {
// ... mock response
}
},
{
"url": "https://graph.microsoft.com/v1.0/users/*/photo/$value",
"responseBody": "@picture.png"
}
]
}
Property | Required | Description |
---|---|---|
url | true | Graph request URL to match the request. Can contain one or more wildcards to match multiple requests |
responseCode | false | HTTP response code. If omitted, defaults to 200 OK |
responseBody | false | Body to send along with response. If set to an object, will send it as serialized JSON string. If set to a string, will send it as-is. If the string starts with @, the proxy will treat the string that follows as a file system path and read the response from the specified file. This is useful for sending large or binary responses (file, photo). If the specified path doesn’t exist, proxy will fail with the error: “Specified response path xyz doesn’t exist”. |
Requests are matched in the order they’re defined in the responses.json file, so it’s up to developers to define which responses take precedence.
If an intercepted request matches a pattern in the responses.json file, the proxy responds with the mock response for that request as defined in the responses.json file. If there’s no matching mock response, proxy will reply with a random error response, the same way as if there was no responses.json defined.
Responses from the responses.json file are read and stored in memory when the proxy starts. After changing responses file, you need to restart the proxy to read the new contents.
Providing mock responses can be disabled even when a responses.json file is present by using the --no-mocks
option on the command line.
@
-tokens--no-mocks
flagBefore we commit actual engineering resources to this new project, I think it'd be a good thing to run a competitive analysis with these existing solutions:
The goal would be to understand:
By default, the proxy will run on port xyz. This feature allows configuring the proxy to run on the number specified using the --port
option in the command line, when starting the proxy, eg. --port 1234.
If the port is already in use, the proxy will close with an error saying: “Port xyz is already in use. Please use a different port”.
To specify a different port or use the default port, you need to restart the proxy.
Update readme with the new name and any other information that's needed when the repo becomes public.
If an intercepted call falls within the failure rate %, the proxy returns one of the faulty responses at random (429, 500, 502, 503, 504). If the call is outside the failure rate %, the call is passed as-is to Graph and the response from Graph is returned to the caller.
Document the steps needed to use Playwright to automate chaos testing a web app connected to Microsoft Graph.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.