Arcade build scripts run on whatever version of Powershell is available on the machine. Today that means we need to support Powershell v3.0, which leads to required changes like #42.

As @natemcmaster suggested, we should be explicit about the minimum supported Powershell version that we've tested (even if that's back to 3.0).

That makes sense. Maybe we should add #requires

#requires -version 5.0

https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_requires

The build.cmd prints out -restore as an option if I run build.cmd -?. But if I use it I get the following error

>E:\code\arcade> .\Build.cmd -restore
E:\code\arcade\build\Build.ps1 : Cannot bind parameter because parameter 'restore' is specified more than once. To provide multiple
values to parameters that can accept multiple values, use the array syntax. For example, "-parameter value1,value2,value3".
At line:1 char:48

Uploading a lot of code from difference sources now. What license should we be using here?

Currently, there's just one task in the repo, as this grows, we will likely need to begin sharing code between packages. When that occurs, we should consider using an shproj to share code rather than manual linking.

There is some discussion around support for using shared projects.

jaredpar 21 hours ago
Why do all the manual likning here vs. creating a shared source project?

natemcmaster 21 hours ago
Is shproj compatible with dotnet-CLI now?

chcosta 21 hours ago
At the moment, this isn't even needed, these could just be moved into the only build project in the repo. I'll do that for now.

tmat 21 hours ago
Is shproj compatible with dotnet-CLI now?
Were there any issues? We use shared projects in Roslyn.

natemcmaster 20 hours ago
Last I checked, $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\CodeSharing\ doesn't exist in the msbuild that ships in .NET Core CLI

tmat commented 20 hours ago
@natemcmaster Microsoft.CodeSharing.*.targets don't seem to do anything useful for the build. They are just setting project capabilities and such, presumably for design-time build in VS. Since they are imported conditionally it should work without them. @nguerrera @davkean Do shared projects work in VS Code/VS Mac?

jaredpar 3 hours ago
Unsure about that particular MSBuild mechanism but we definitely use shared source projects in Roslyn + dotnet/CLI to build them.

Looks like there will be a number of places where we need to specify a desktop target framework in our projects. This target framework essentially establishes the minimum desktop runtime necessary to use artifacts from the arcade repository. Rather than pick project per project think we should establish the minimum runtime we desire and use it consistently through our projects.

I believe we should use net461 for the following reasons:

1. Allows any utility libraries we write to target netstandard2.0. Choosing any lower desktop runtime means we have to use netstandard1.3.
2. It's present in over 97% of all window machines in existence.
3. It's present in 99% of all windows machines which also install Visual Studio.

In the .CS file which is generated by the Microsoft.DotNet.BuildTools.GenAPI package, is it possible to include the xml documentation of the methods which are listed for each types listed in the file?

Feed package currently has a dependency on VersionTools which is published out of BuildTools. Until that dependency is removed, we should look into a maestro type solution for taking updates. Also, in general we'll need some solution for this going forward.

Add build.sh / etc scripts.

tmat reviewed an hour ago

src/Microsoft.DotNet.Build.Tasks.Feed/Microsoft.DotNet.Build.Tasks.Feed.csproj 

+    <PackageReference Include="System.Runtime.InteropServices.RuntimeInformation" Version="4.4.0-beta-24813-03" /> 
+  </ItemGroup>   
+ 
+  <ItemGroup Condition="'$(TargetFramework)' == 'netstandard1.5'"> 

tmat an hour ago • edited
Why do we conditionally reference different versions of packages (like SRM 1.4.1 vs 1.3.0)? NuGet should automatically pick the right binary from each package based on TargetFramework version.

This is considered "done" when:

• The core set of packages are building and publishing out of arcade
• A dependency uptake mechanism is in place (e.g. Maestro)
• Sufficient functionality available for Tier 1 repos on-boarding onto Arcade

@chcosta commented on Tue Jun 13 2017

Tracking issue https://github.com/dotnet/corefx/issues/20869#issuecomment-307464151

/cc @ellismg

src/Microsoft.DotNet.Build.Tasks.Feed/Microsoft.DotNet.Build.Tasks.Feed.nuspec 
@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="utf-8"?> 
+<package> 

weshaggard 4 hours ago
Given we are an SDK based project why do we need to manually create the nuspec?

chcosta 3 hours ago
I'm not aware of any reason that we need to. I slightly prefer the nuspec just because it's easier to read, makes it feel like I have more control, and creates a separation between the build / packaging. That's probably just me not used to the new standard. I can move this into the project.

chcosta an hour ago
It actually seems to be a bit of a pain (taking more time that I would have hoped) to get the lock file assemblies to be included in the package build outputs. I'll play around with it a bit more but likely end up filing an issue to follow up on this.

natemcmaster 42 minutes ago
I'm okay with using nuspec. I've hit the same issues and discussed them before with NuGet. It's not on their list of priorities. See NuGet/Home#5063, NuGet/Home#6321. Using nuspec is easier IMO.

weshaggard 34 minutes ago
Given there are issues with the default generated nuspec's from SDK projects for tool package I can live with a nuspec. Thanks for trying to use the shipping tools.

weshaggard 32 minutes ago
I wonder if for this repo we should try to use a nuspec template of sorts and write a simple target to fill it the blanks from the project properties. That isn't an issue for one tool package but when we get more it will help avoid duplication and inconsistencies.

natemcmaster 28 minutes ago
Might work. I've been down this road before. IIRC the nuspec pack step in NuGet is pretty fragile. You must keep the NuspecPropertiesmsbuild prop and the nuspec substitutions closely aligned. If you have empty properties in MSBuild, or unused substitutions, pack will fail. In aspnet, we worked around by just implementing our own PackNuspec task. Would love to contribute it back to NuGet, but haven't found the time.

Creating a nuspec template sdk might be useful for creating our task packages.

I presume we'll want to use signtool, but there are some details that need to be worked out. Once there's a plan, I'd like to see that funded soon so we can start using it for ESRP and nupkg signing.

ASP

• DownloadFile
• ZipArchive
• UnzipArchive
• GenerateFileFromTemplate
• ComputeChecksum
• Chmod

Roslyn/CLI

• Repack
• Signtool

CoreFx/CoreCLR

• BlobFeed
  • VersionTools/Dependency update
  • Repo Tools
  • ILAsm
• ILLinker

We should rename the build directory to something else because it always gets in the way whenever I try to tab complete build.cmd.

The core repo's call this directory tools-local which would seem reasonable.

@tmat @chcosta thoughts?

Today we use SCEP Installer in our CoreCLR build in the attempt to fix https://github.com/dotnet/core-eng/issues/2225 in the form of a package which

This tool is used to install and uninstall Security Center Endpoint Protection which we think is causing the above issue. At the moment of filing this issue this is just an assumption and there is no hard data to confirm this will fix it.

If we've validated that this is the fix and we want to keep doing this we need to formalize the packaging process which today happens in someones computer and once the package is manually created we then manually upload it to the VSTS feed.

Documentation around how do we create the package and its contents can be found here: https://github.com/dotnet/core-eng/tree/master/Documentation/SCEP%20Installer

See dotnet/buildtools#1943. Without this resolver it will become very difficult to align our assembly dependencies across our tasks when running on .NET Framework.

natemcmaster reviewed 3 hours ago

Directory.Build.props 
+  <PropertyGroup> 
+    <!-- Respect environment variable for the NuGet Packages Root if set; otherwise, use the current default location --> 
+    <NuGetPackageRoot Condition="'$(NuGetPackageRoot)' == ''">$(NUGET_PACKAGES)</NuGetPackageRoot> 
+    <NuGetPackageRoot Condition="'$(RepoRoot)' != ''">$(RepoRoot)packages\</NuGetPackageRoot> 

natemcmaster 3 hours ago
This will overwrite NuGetPackageRoot even if already set. Did you mean to add '$(NuGetPackageRoot)' != '' to the condition?

tmat an hour ago
I suggest removing this altogether and just use global cache. Saves space and restore time on dev machine if I don't need to have dozens of copies of the same nuget packages all over my harddrive.

chcosta 27 minutes ago
This was intentional. RepoToolset sets NuGetPackageRoot before evaluating Directory.Build.props. Took me a bit to figure out why I couldn't control this locally until I realized that was happening. So I'm just overriding RepoToolset here.

tmat 25 minutes ago
I suggest you do not override it and let the packages restore to global cache.

chcosta 24 minutes ago
We've hit quite a few issues in the build lab with repo's not playing nicely and/or cleaning up after / before themselves which leads to difficult to diagnose package issues. Locally, as a dev, using the cache seems reasonable, but running so many builds in the build lab under the same lab account, this has fallen apart. Perhaps the state of things has gotten better, but I'd be hesitant to rely on that given how many repo's / branches / releases / msrc's / etc... the build machines execute.

jaredpar 22 minutes ago
If that's the casse then why not only make this change for non-deveolper builds.

tmat 22 minutes ago • edited
Then only set the nuget root when running in CI. That's possible in build.ps1. There you can pass /p:NuGetPackageRoot=XXX to the Build.proj driver.

tmat 21 minutes ago
It'd be also good to check with @jaredpar on how this is handled in Roslyn CI.

@chcosta commented on Tue Feb 20 2018

The feed package has a dependency on VersionTools which it consumes to make build manifest generation "easy" / transparent. The dependency is not strictly necessary and not every consumer of the feed package may care about the build manifest generation. We should remove the feed task package dependency on VersionTools.

I think that it'd make sense to have a "prodcon v1 push" task that encapsulates the PushToBlobFeed plus manifest generation, and brings in whatever packages are needed--which would be the Feed and VersionTools packages. -dagood

I think that, for now, I'm just going to update versiontools package to include the desktop assets and have the feed package consume it as a packagereference during build. we can pull it apart down the road -chcosta

This is the kind of thing that it's easy to let slide because it's somewhat difficult (may involve quite a few moving pieces and updating how consumption works across many repo's), but I think we should invest in this sooner than later.

/cc @dagood @markwilkie

FYI @weshaggard @mmitche

@chcosta commented on Tue Feb 20 2018

Additional info dotnet/buildtools#1836

What level of support do we want to provide for building on Linux.

Building for the desktop framework on a non-Windows platform is not currently supported.

dotnet/sdk#335

There's a workaround to make this work, but it's not clear to me that this is even a scenario that we really care about for the Arcade repo. I think we'll be doing all official publishing from a Windows box, so this won't apply there, but a dev building on Linux will produce a different package that won't run on Windows.

As a temporary work around, I'm going to add something like this to /Directory.Build.props

<PropertyGroup Condition="'$(OS)' == 'Windows_NT'">
    <!-- Place 'NetFxTfm' in the 'TargetFramework' property of a csproj to include building framework version if building on Windows but to skip building against the framwork on non-Windows platforms.
         Building targeting the framework is not supported on Linux
         ie. <TargetFrameworks>netcoreapp2.0;$(NetFxTfm)</TargetFrameworks> -->
    <NetFxTfm>net45</NetFxTfm>
  </PropertyGroup>

Do we think it's important to build framework assets on Non-Windows platforms in the Arcade repo?

/cc @weshaggard @tmat @jaredpar @natemcmaster @markwilkie

How about not using a silly project name and give this repo a meaningful name? Let's continue to use meaningful names within the dotnet organization. It really helps newbies understand things when we aren't using meaningless project names.

Currently empty:

Possible wording:

Build tools that are designed to be shared across multiple .NET Foundation projects

We have a bunch of scripts that are duplicated across repos that build native code:

https://github.com/dotnet/coreclr/blob/master/src/pal/tools/gen-buildsys-win.bat
https://github.com/dotnet/corefx/blob/master/src/Native/Windows/gen-buildsys-win.bat
https://github.com/dotnet/core-setup/blob/master/src/corehost/Windows/gen-buildsys-win.bat

https://github.com/dotnet/coreclr/blob/master/src/pal/tools/probe-win.ps1
https://github.com/dotnet/corefx/blob/master/src/Native/Windows/probe-win.ps1
https://github.com/dotnet/core-setup/blob/master/src/corehost/Windows/probe-win.ps1

We should consolidate these scripts in a single place so bug fixes can be applied in one spot and all dependent places get the fixes.

This also applies to a bunch of common CMake functions like strip symbols:

https://github.com/dotnet/coreclr/blob/c1bbdae7964b19b7063074d36e6af960f0cdc3a0/functions.cmake#L125
https://github.com/dotnet/corefx/blob/a9f4a46f48baeaadd515d85ac9de1249b8023a5a/src/Native/Unix/CMakeLists.txt#L193
https://github.com/dotnet/core-setup/blob/b5dd381dd3a586640e4bfe4804d6563861eaab20/src/settings.cmake#L71

In general - we need better native code building infrastructure so we aren't copying these files all over any repo that needs to build native code.

/cc @weshaggard @dagood @markwilkie @janvorli

build.sh currently has version 1.0.5 of the shared framework hardcoded. Look at updating this and making it configurable.

Clone the repo and run build.cmd and you will get the following:

E:\code\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Framework (>= 0.1.0-preview-00022) but Microsoft.Build.Framework 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Framework 0.1.0-preview-00038-160914 was resolved. [E:\code\arcade\Arcade.sln]
E:\code\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Utilities.Core (>= 0.1.0-preview-00022) but Microsoft.Build.Utilities.Core 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Utilities.Core 0.1.0-preview-00038-160914 was resolved. [E:\code\arcade\Arcade.sln]

Validate use as packagereference
Validate use as sdk
Validate closure (probably grab this from buildtools)

How should we version packages in this repo?

In particular, Microsoft.DotNet.Build.Tasks.Feed is currently being published from BuildTools as 2.2.0-prerelease-xxxxx-xx.

W.R.T. packages in arcade:

• Do we want to set the prerelease label to prerelease for everything? Currently it's beta.
• Should we rev the patch version or leave it as 0?

As an example, the test package I produced today out of Arcade is 1.0.0-beta-62706-01. Note that the build number part of the prerelease version is 62706-01 vs 02431-01 from the most recent package published out of buildtools.

I don't imagine that all packages in the repo need to version the same though they'll probably want to share the same prerelease version moniker.

@weshaggard @natemcmaster @markwilkie

@chcosta commented on Mon Mar 05 2018

@Chrisboh commented on Thu Mar 08 2018

@chcosta could you please move it to the Arcade repo?

This is a commandline version of ApiReviewer and helpful for generating assembly metadata diffs.

In order to build arm binaries, we currently have roughly the same script copied into the following places:

https://github.com/dotnet/coreclr/blob/master/cross/build-rootfs.sh
https://github.com/dotnet/corefx/blob/master/cross/build-rootfs.sh
https://github.com/dotnet/core-setup/blob/master/cross/build-rootfs.sh

And they are all slightly out of date with each other because someone will update coreclr for a new platform or to fix a bug, and then forget to update the other duplicates.

Also, if/when ASP.NET and the SDK need to build for arm, they will need these scripts as well.

We should consolidate this infrastructure into a single place - this repo - where the things that are common across repos should live.

/cc @weshaggard @janvorli

What is the strategy for maintaining the SignToolData.json (https://github.com/dotnet/arcade/blob/master/build/SignToolData.json) file? I'm not a fan of manually maintaining a list of files that we need to sign. People always forget to update it and we have build failures because some binary is unsigned. In the core repo's we have moved to a model of generating and dropping files next to the binaries (foo.dll.requires_signing) based on a property, as part of the build process and then gather the list of files to sign from those. It has worked pretty well over the last few years of doing it.

We'll consider this portion of the work "done" when Arcade is:

• Enabled in CI for all supported platforms
• Building and signed officially for all supported platforms
• Publishing
• Repo is no longer private and has all the appropriate licenses
• "Getting Started" documentation is available
• Package tests are in place to give some confidence

buildtools VersionTools package does not include desktop implementation (even though it's being built). This means that we can't build the feed package without either forking the source into arcade, or publishing the desktop implementation for compilation.

I'm preferring consuming a packagereference as it seems less messy if / when we remove the feed package dependency on versiontools.

Decoupling VersionTools: https://github.com/dotnet/core-eng/issues/2699

PR: dotnet/buildtools#1920

@weshaggard commented on Thu Dec 07 2017

In order to make our builds more reliable and maintainable long term we need to be able to bootstrap our toolsets as part of our build initialization and stop relying on globally installed tools. This includes things like CMake and Visual Studio but for this issue I want to focus on what it would take to break our dependency on requiring Visual Studio. Note that I think we should support using Visual Studio if it is present but we should be able to build without it and allow folks to isolate themselves and build without it even if it is installed.

There are a few things I'm aware of that need to be figured out in order to break our VS dependency:

1. We need msbuild along with the managed tools and targets.
   • We should be able to use what comes with the SDK to handle the managed builds.
2. We need C++ compiler as well as the headers and libs.
   • We should be able to look https://aka.ms/dailyMSVC and switch to downloading their nuget packages.
3. We need the Windows SDK to be able to build things targeting UWP.
   • We need to investigate our options. Assuming it is just some libs and header perhaps we create our own custom package to provide this.

cc @shawnro @dleeapho

@dleeapho commented on Tue Jan 31 2017

This is the second part of implementing cmake toolset. This is mainly around uptake in the official builds.

@shawnro commented on Fri Feb 03 2017

@dleeapho can you please link the first part of the work to this one?

@ravimeda commented on Fri Feb 03 2017

First part that covers dev workflow is #224

@ravimeda commented on Mon Feb 27 2017

Scope of the work that will be done in this task is limited to -

1. Single repository. Probing and acquisition scripts will be limited to dotnet/CoreFx.
2. Single prerequisite i.e., CMake, This means scripts will probe, and acquire missing CMake.
3. Single source for CMake acquisition. Acquire CMake from OSS repository.

Tasks:

• Process to host a version of CMake on OSS repository is defined
  • Document a repeatable and reliable process of downloading the source of the declared CMake version, performing a security audit on the source, build the sanitized source, package and host it on OSS Tools repository.
  • Ensure the package corresponding to the declared version of CMake for each .NET Core repository is hosted in OSS Tools repository.

• Official build step to acquire declared version of CMake from OSS repository is prepared
  • In official builds, prepare a build step that will acquire the package corresponding to the declared version of the repository being built. Package should be acquired from the OSS Tools repository.

• Developer guidelines are updated
  • Update developer guidelines document to point to .toolsversions, which will contain the declared version of CMake for the repository.

@ravimeda commented on Mon Mar 06 2017

@dleeapho I suggest splitting this into two User Stories as follows:

1. As a repo owner, I have a defined process to host a build tool at OSS Tools repository.
2. As a repo owner, I can follow a contract and process to declare a version of a build tool, and provision it for official and CI builds.

First one will be in Sprint 115, and the other in Sprint 116. Please let me know if this is acceptable.

@ravimeda commented on Mon Mar 06 2017

#549 tracks the work for determining the process for hosting a package at OSS Tools repository. I work on it in this sprint.

Moving this task (#404) to Sprint 116 since it is dependent on #549

Create a document to describe how to onboard, or get started with Arcade

Seems two things should be done at a minimum:

1. Settle on long term package versioning plan
2. Settle on versioning plan for Arcade itself (especially the scripts)

The versioning scheme needs to work for our servicing scenarios

Once Feed package is publishing from Arcade, disable publishing from BuildTools. Note, there is still work to be done (official builds, signing, etc...) before we're ready for that.

No matter what tools/infra we currently use there is always a need for some boiler-plate scripts, config files, etc that need to be maintained across many repos. Today that is mostly manual and is a very tedious process to ensure everything is in-sync and consistent. We need some process, perhaps maestro, that will monitor changes to common files and then submit auto-PRs to all repo's that subscribe to the engineering system.

We have issue dotnet/buildtools#1709 tracking a similar feature but I think this is going to be key for getting people to arcade and keeping them updated so I've filed the issue here.

Some general requirements:

1. Any auto-updated files should be blindly updated by this tool and never have any customizations from the individual repos. If customizations are needed then it should be handled by an extension point where repo owners can plug into from a separate file.
2. These files should be in well known locations in each repo so the updating is just a simple overwrite.
3. We may need the flexibility in the system to move files if we decide we need to rename or move a file for some reason.

The feed package has a dependency on Microsoft.Build.Framework and Microsoft.Build.Utilities.Core packages versioned 0.1.0-preview-00022. Those packages are not present in the dotnetfeed package source, so we have to acquire them from the dotnet.myget.org dotnet-buildtools package source.

We should look into either adding the packages to the dotnetfeed source or moving to a version which is already on dotnetfeed so we can remove the additional restoresource from the repo.

E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Framework (>= 0.1.0-preview-00022) but Microsoft.Build.Framework 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Framework 0.1.0-preview-00038-160914 was resolved. [E:\gh\dotnet\arcade\Arcade.sln]
E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Utilities.Core (>= 0.1.0-preview-00022) but Microsoft.Build.Utilities.Core 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Utilities.Core 0.1.0-preview-00038-160914 was resolved. [E:\gh\dotnet\arcade\Arcade.sln]
E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Framework (>= 0.1.0-preview-00022) but Microsoft.Build.Framework 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Framework 0.1.0-preview-00038-160914 was resolved.
E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Utilities.Core (>= 0.1.0-preview-00022) but Microsoft.Build.Utilities.Core 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Utilities.Core 0.1.0-preview-00038-160914 was resolved.
E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Framework (>= 0.1.0-preview-00022) but Microsoft.Build.Framework 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Framework 0.1.0-preview-00038-160914 was resolved.
E:\gh\dotnet\arcade\src\Microsoft.DotNet.Build.Tasks.Feed\Microsoft.DotNet.Build.Tasks.Feed.csproj : error NU1603: Microsoft.DotNet.Build.Tasks.Feed depends on Microsoft.Build.Utilities.Core (>= 0.1.0-preview-00022) but Microsoft.Build.Utilities.Core 0.1.0-preview-00022 was not found. An approximate best match of Microsoft.Build.Utilities.Core 0.1.0-preview-00038-160914 was resolved.

What namespace should we be using for projects in this repo? Microsoft is the universal namespace but we also only have one project. Is that the right choice? Remember at one point there were rules on this but don't remember what they are.

Sorry for bike shedding 😦

Microsoft.DotNet.Build.Tasks.IO

Charter: includes tasks related file system IO

• DownloadFile - downloads a file.
• ZipArchive - creates a .zip file
• UnzipArchive - unzips a .zip file
• ComputeChecksum - computes the SHA256 or SHA512 checksum for files
• Chmod - change Unix permissions

@ravimeda commented on Mon Mar 06 2017

The goal of this task is to understand the steps involved in hosting a tool at OSS Tools repository. This is required for official and CI builds that will consume a tool from a trustable source, and not download directly from internet. Work is scoped to single tool i.e., CMake. Tasks towards this goal are as follows:

• Register an existing version of CMake already available in OSS Tools repository for consumption in dotnet builds.
• Add a new version of CMake to OSS Tools repository. Understand and document the end-to-end process.
• Define the URL format for dotnet build tools. This means given the name of a tool and version, and using a base URL, the full URL to the tool package can be determined.

@shawnro commented on Thu Mar 16 2017

@MichaelSimons, @ravimeda should this be a User Story instead of a Task?

@ravimeda commented on Thu Mar 16 2017

Compiler and Languages Platform team already has a process in place to distribute CMake 3.6 (https://osstool.microsoft.com/palamida/RequestDetails.htm?rid=43314&projectId=1&from=requests). Evaluate if this can be extended to dotnet.

We should try to tee the console from RemoteExecutorHost so it gets into the XML test log and thus into the correct test result in Helix. As it is you just get eg (note the code = "some managed exceptoin")

Assert.Equal() Failure\r
    Expected: 42\r
    Actual:   -532462766

        Stack Trace:

           at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose()

but if you look at the full console output, aggregated for all the tests in this library, you see the actual exception --

2017-05-09 01:33:05,362: INFO: proc(54): run_and_log_output: Output: Hello World
2017-05-09 01:33:05,440: INFO: proc(54): run_and_log_output: Output: Exception from RemoteExecutorConsoleApp(System.Console.Tests, Version=4.1.0.0, Culture=neutral, PublicKeyToken=9d77cc7ad39b68eb, ConsoleEncoding+<>c, <OutputEncoding_SetWithErrorAndOutputInitialized_ResetsErrorAndOutput>b__6_0):
2017-05-09 01:33:05,440: INFO: proc(54): run_and_log_output: Output: Assembly: System.Console.Tests, Version=4.1.0.0, Culture=neutral, PublicKeyToken=9d77cc7ad39b68eb
2017-05-09 01:33:05,440: INFO: proc(54): run_and_log_output: Output: Type: ConsoleEncoding+<>c
2017-05-09 01:33:05,440: INFO: proc(54): run_and_log_output: Output: Method: Int32 <OutputEncoding_SetWithErrorAndOutputInitialized_ResetsErrorAndOutput>b__6_0()
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output: Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.IOException: The request is not supported
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.ConsolePal.SetConsoleOutputEncoding(Encoding enc)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.Console.set_OutputEncoding(Encoding value)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at ConsoleEncoding.<>c.<OutputEncoding_SetWithErrorAndOutputInitialized_ResetsErrorAndOutput>b__6_0()
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    --- End of inner exception stack trace ---
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at RemoteExecutorConsoleApp.Program.Main(String[] args)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output: 
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output: Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.IOException: The request is not supported
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.ConsolePal.SetConsoleOutputEncoding(Encoding enc)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at System.Console.set_OutputEncoding(Encoding value)
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at ConsoleEncoding.<>c.<OutputEncoding_SetWithErrorAndOutputInitialized_ResetsErrorAndOutput>b__6_0()
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    --- End of inner exception stack trace ---
2017-05-09 01:33:05,487: INFO: proc(54): run_and_log_output: Output:    at RemoteExecutorConsoleApp.Program.Main(String[] args)
2017-05-09 01:33:05,596: INFO: proc(54): run_and_log_output: Output:    ConsoleEncoding.OutputEncoding_SetWithErrorAndOutputInitialized_ResetsErrorAndOutput [FAIL]
2017-05-09 01:33:05,596: INFO: proc(54): run_and_log_output: Output:       Assert.Equal() Failure
2017-05-09 01:33:05,596: INFO: proc(54): run_and_log_output: Output:       Expected: 42
2017-05-09 01:33:05,596: INFO: proc(54): run_and_log_output: Output:       Actual:   -532462766
2017-05-09 01:33:05,596: INFO: proc(54): run_and_log_output: Output:       Stack Trace:
2017-05-09 01:33:05,721: INFO: proc(54): run_and_log_output: Output:             at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose()
2017-05-09 01:33:05,957: INFO: proc(54): run_and_log_output: Output: Exception from RemoteExecutorConsoleApp(System.Console.Tests, Version=4.1.0.0, Culture=neutral, PublicKeyToken=9d77cc7ad39b68eb, ConsoleEncoding+<>c, <InputEncoding_SetWithInInitialized_ResetsIn>b__3_0):
2017-05-09 01:33:05,957: INFO: proc(54): run_and_log_output: Output: Assembly: System.Console.Tests, Version=4.1.0.0, Culture=neutral, PublicKeyToken=9d77cc7ad39b68eb