Comments (7)
@MarcoRossignoli please respond here.
from vstest.
Hi @daveMueller I did a sample here
...\merge>dotnet test --collect "MyDataCollector" --test-adapter-path "....\MyDataCollector\bin\Debug\netstandard2.0" --diag log.txt
In the processor I've added the code to attach in case we're called with 2 attachments (one per test).
The processor is called also once per project "in process". So in my sample 3 times.
from vstest.
OK thanks a lot for that. I made a mistake with the return value of ProcessAttachmentSetsAsync
that led to the false behavior. I'll get back to that again in a minute. I still have some small questions:
-
I still don't fully understand when the post processing is triggered. As described here, it says that it is triggered after
any two test executions are done
. But from my experiments and also in your sample it additionally is triggered after finalizing each project report? -
Does the newly merged report needs to be passed back to the test platform in addition to the input attachments? Something like:
public Task<ICollection<AttachmentSet>> ProcessAttachmentSetsAsync(XmlElement configurationElement, ICollection<AttachmentSet> attachments, IProgress<int> progressReporter, IMessageLogger logger, CancellationToken cancellationToken) { if(attachments.Count <= 1) return Task.FromResult(attachments); // merge reports to new report attachments = new[] { new AttachmentSet(new Uri(filePath), report.fileName) }.Concat(attachments).ToList(); return Task.FromResult(attachments); }
This would also mean that the actual merge operation could be executed multiple times with intermediate merge results depending on the amount of test projects? Like also seen in here.
If it is like that, I would love to know the last time the post processing is triggered to clean up the intermediate merge result files. Or can I do this in another way?
-
Is there a specific operation needed to tell the platform that there is a new post processed attachment? Like for instance inside the collector
Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.DataCollectionSink.SendFileAsync(...)
?
from vstest.
I still don't fully understand when the post processing is triggered. As described here, it says that it is triggered after any two test executions are done. But from my experiments and also in your sample it additionally is triggered after finalizing each project report?
Yes, the merge works the same way inside VS and there when we run multi-tfm the same "server" is running more modules, this is not the case for dotnet test
. For that reason on dotnet test
you'll see a call at the end of every project and another call at the end of all projects. From your question I suppose your doubt is if it's possible use the json format as intermediate and the real one at the end. I suppose that this is not possible so my suggestion on coverlet repo could be wrong. It's possible we need to merge every format separately.
Does the newly merged report needs to be passed back to the test platform in addition to the input attachments? Something like:
The processor select "your" assets(using the uris GetExtensionUris) and the input will be composed by that assets, you can elaborate and return a new set(one or more) you don't need to return the input if are no more useful and you can discard those.
If it is like that, I would love to know the last time the post processing is triggered to clean up the intermediate merge result files. Or can I do this in another way?
Unfortunately nope, we don't have at the moment a signal of "last merge", I suppose this is linked to the intermediate format as I've discussed above.
Is there a specific operation needed to tell the platform that there is a new post processed attachment? Like for instance inside the collector Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.DataCollectionSink.SendFileAsync(...)?
This question is not clear, why you need it? Post processing is done at the end of "the execution", after the data collectors are shootdown.
from vstest.
This question is not clear, why you need it? Post processing is done at the end of "the execution", after the data collectors are shootdown.
I was just thinking about where to store the final result then. For the collectors the file storage location is selected by the platform. For the post processing we might need to add another runsettins option to specify the location of the merged result.
from vstest.
I was just thinking about where to store the final result then. For the collectors the file storage location is selected by the platform. For the post processing we might need to add another runsettins option to specify the location of the merged result.
I would expect that all the attachments you receive inside the post processor come from the same result folder(default or user specified), so you can take the first one directory and create the final file there and return.
from vstest.
OK thanks for that. I guess everything is clear now.
from vstest.
Related Issues (20)
- InProcessVsTestConsoleWrapper creates a VsTestConsoleProcessManager but doesn't Dispose() it HOT 1
- Dump file not created when process crashes HOT 2
- Test Adapter for Google Test never works! HOT 1
- DTAExecutionHost.exe failed with code 1 - Error occurred while publishing the test results : OutOfMemoryException HOT 1
- Upgrade to Visual Studio 17.10.x now has internal exception in Azure DevOps log HOT 1
- Blame dotnet test - System.IO.FileNotFoundException HOT 1
- Unsuppressable overwrite warning when using <VSTestLogger> MSBuild property which confuses users HOT 1
- Would you please increase the maximum length of grouping in Test Explorer? HOT 2
- Microsoft.Testing.Platform vs Microsoft.TestPlatform.TestHost package HOT 1
- vstest.console.exe cannot resolve assembly for extension with target framework .net8.0, however dotnet test (vstest.console.dll) resolves it correctly HOT 3
- Testhost is blocked on system error message dialog HOT 2
- dotnet msbuild /t:VSTest does not work with terminal logger HOT 2
- Exception thrown in TestHostTraceListener.Fail does not include stack trace or method name
- Findings source corresponding to `Microsoft.CodeCoverage` 17.8.0 HOT 1
- [FeatureRequest] Possibilty to run Unit or Integration Test using dotnet test HOT 4
- Required build arguments should not be specified in pipeline YML
- IFrameworkHandle.LaunchProcessWithDebuggerAttached allows null for workingDirectory in signature but throws
- IFrameworkHandle.LaunchProcessWithDebuggerAttached needs ability to control ProcessStartInfo HOT 5
- Unable to disable determistic source mapping in Microsoft.CodeCoverage HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vstest.