[Feature]: Item lifetime tracing

[svew]

This is a longshot, please close if it's far out of scope,

I often have trouble tracing how a file copy in a tree of project dependencies flows from one project to the next. Builds are complicated, and sometimes it's hard to see the big picture of what targets and MSBuild calls are used in order to get a Content file from project A all the way to the output folder of project D.

It would be very helpful to see a graph of how a project transforms an item via various targets, how it receives items from other projects, etc. You might select a specific item from the log, or a string to search for (e.g. 'file.txt'), and the program will attempt to trace how the item containing that phrase gets modified, gets copied to new items, gets used by certain targets, etc:

In my head, such a graph might look something like this:

flowchart TD

    subgraph MyProject.csproj
        C[<b>@ContentWithTargetPath</b><br/>'file.txt'<br/>%CopyToOutputDirectory='PreserveNewest'<br/>%TargetPath='file.txt'<br/>%OriginalItemSpec='file.txt']
        direction TB
        subgraph Evaluation
            Z["<b>ItemGroup</b><br/>@Content+='resources/*.txt'<br/>%CopyToOutputDirectory='PreserveNewest'"]
        end
        A[<b>@Content</b><br/>'file.txt'<br/>%CopyToOutputDirectory='PreserveNewest']
        subgraph AssignTargetPaths
            B["<b>AssignTargetPath</b><br/>Files='@(Content)'"]
        end
        subgraph PrepareForBuild
            E["<b>FindAppConfigFile</b></br>SecondaryList='@(Content)'<br/><br/>"No outputs]
        end
        subgraph ResolveAssemblyReferences
            F["<b>ResolveAssemblyReference</b><br/>CandidateAssemblyFiles='@(Content)'<br/><br/>"No outputs contain keyword 'file.txt'<br/>Click here to add items to graph]
        end
        subgraph GetItemTargetPaths
            D["Returns='@(ContentWithTargetPath)'"]
        end
        Z:::X --> A
        A:::X --> B:::X
        A --> E:::X
        B --> C:::X
        C --> D:::X
        A --> F:::X
    end

    classDef X text-align:start;

Loading

The backing code used to render such a graph could also be used in the future for new panels which could show a diagram of target dependencies, project dependencies, etc.

I imagine a feature like this might be somewhat complicated to implement, but if the resources were ever found it would be immensely helpful!

[baronfel]

I think we're lacking the knowledge of the transformation in the binlog at this point in time, but @YuliiaKovalova may know more - I think she's trying to implement something that could give us the data to power a view like this, but for Properties specifically (since more users interact with properties we think). If successful, Items could be next.

I do agree with you that tracking the chain of changes is a very valuable activity that is hard to do in the viewer at this time.

[svew]

I imagine that it's possible to correctly trace items and properties as modified via ItemGroup and PropertyGroup elements in targets, though it would probably require reading the source build files directly to get the raw instructions. It might be a bit bug prone too, but I assume that's what @YuliiaKovalova would have to do in some cases to trace property changes?

Tasks would certainly be more opaque, but if the user is given the ability to review the outputs of a task and select output items they think best matches their query, I think that would be perfectly fine.

Is there any room for contributing to this kind of work?

[KirillOsenkov]

I've always wanted something like this. Item flow is difficult to investigate and I imagine a lot of it can be automated.

The closest we've gotten so far is this:
https://github.com/KirillOsenkov/MSBuildStructuredLog/wiki/Search-Syntax#copy-search-file-copy-map

When you search for $copy filename.txt it will track the file across transitive project references until it finds the project it originated from.

Here's the logic for it:

MSBuildStructuredLog/src/StructuredLogger/Analyzers/FileCopyMap.cs

Line 251 in 6e220d0

private void TryExplainSingleFileCopy(Project project, string filePath, IList<SearchResult> resultSet)