Support multi file verification on Etherscan about truffle-plugin-verify HOT 19 CLOSED

I tried to do this today, but I ran into some issues in the step "compute the artifact path for this source path", since it's pretty difficult to find out the artifact file(s) corresponding to a Solidity file, since a file can include one or more contracts with different names. I think for now I'll publish the multi file verification with the alphabetical file order soon (rather than the "intuitive" file order) and make an issue of fixing the file order in the future.

from truffle-plugin-verify.

Hey @abcoathup! Thanks for moving this issue here. I started work on it yesterday and it's a priority to get done. Will post updates on this issue.

from truffle-plugin-verify.

Unfortunately, the metadata will contain everything that was compiled at the same time; solc doesn't do any pruning.

from truffle-plugin-verify.

Looking at the linked buidler issue it does as long as it includes all the same files as in the local compilation (as I understand it that includes any superfluous files).

from truffle-plugin-verify.

Sounds like the main issue with it is that it clutters the Etherscan interface. But I'd say even with superfuous files it is easier to browse than the flattened code.

from truffle-plugin-verify.

It would be worth letting users know if they don't want superfluous files how to do this.

That's a very good point.

@gnidan earlier you said:

Unfortunately, the metadata will contain everything that was compiled at the same time; solc doesn't do any pruning.

When I look at the MetaCoin artifacts for this extended MetaCoin example, it only includes MetaCoin and ConvertLib, excluding the irrelevant WrappedMetaCoin (and Migrations). I'm assuming this is because truffle compile compiles each contract one by one?

It looks like this means that only the relevant files are included in the verification (which would be great!). See these verified contracts (MetaCoin, WrappedMetaCoin), that shows how they are only including the relevant files.

from truffle-plugin-verify.

@abcoathup I tested it out on your example, which showed that it didn't work yet with node_modules. After fixing that, it appears to verify (SimpleToken).

One thing that I did notice is that the order of the files is a bit different than what you'd expect. This is caused by the order in which they appear in the Truffle artifact's metadata field, where the contracts are listed in alphebetical order (by file path), rather than the order in which they occur in the code.

One (a bit complicated) way I've thought of to get the "expected" order is:

for node in artifact.ast.nodes:
  if node.nodeType == "ImportDirective":
    add node.absolutePath to ordered list of source paths
    compute the artifact path for this source path
    recursively apply the same algorithm to this artifact's AST

I think that should be doable though, and I do think it is worth the effort to have the more "intuitive" file ordering. Any comments on that?

cc @gnidan

from truffle-plugin-verify.

I just published the new release without the "intuitive" file order. I will close this issue, and create a new one to try to achieve a better ordering.

from truffle-plugin-verify.

Hi @rkalis,

That sounds great. I am suggesting the community use multi-file verification, so will be great to have this as a truffle plugin.

One potential issue to be aware of is that if other (unrelated) contracts are included as part of the verification compilation, then Etherscan will also display these: NomicFoundation/hardhat#804

from truffle-plugin-verify.

That's a good caveat to be aware of, thanks for the heads up!

To double check, @gnidan, when using the metadata in the Truffle artifacts, that should only contain the relevant files for the contract represented by the artifact, right?

from truffle-plugin-verify.

Hmm I see. Looking at the linked buidler issue, it appears that solidity versions <0.7.0 also can't correctly be verified if it doesn't use the same files in Etherscan as it does in the local compilation.

Might still be worthwhile for the plugin to check if the Solidity version >=0.7.0 if it can do some analysis of which files actually need to be included and which can be left out.

from truffle-plugin-verify.

Does Etherscan not support metadata-based compilation pre-0.7.0?

from truffle-plugin-verify.

Seems like there are trade-offs, then. Are these superfluous sources a big problem? (What's the harm in including them?)

from truffle-plugin-verify.

I have most code for this ready in the v0.5.0 branch. I've tested it with some different parameters (e.g. optimisation settings, evm version, libraries). @abcoathup or any of the other interested parties: do you have any interesting test cases that I should try out before pushing the new version live?

from truffle-plugin-verify.

It would be worth letting users know if they don't want superfluous files how to do this. I have been manually removing contracts not required for verification and the build directory prior to verification.

@rkalis I have mostly tried verifying simple ERC20 tokens extending from OpenZeppelin Contracts. (OpenZeppelin Contracts v3).

// SPDX-License-Identifier: MIT
pragma solidity ^0.6.0;

import "@openzeppelin/contracts/token/ERC20/ERC20.sol";

/**
 * @title SimpleToken
 * @dev Very simple ERC20 Token example, where all tokens are pre-assigned to the creator.
 * Note they can later distribute these tokens as they wish using `transfer` and other
 * `ERC20` functions.
 */
contract SimpleToken is ERC20 {

    /**
     * @dev Constructor that gives msg.sender all of existing tokens.
     */
    constructor () public ERC20("Simple Token", "SIM") {
        _mint(msg.sender, 1000000 * (10 ** uint256(decimals())));
    }
}

from truffle-plugin-verify.

When I look at the MetaCoin artifacts for this extended MetaCoin example, it only includes MetaCoin and ConvertLib, excluding the irrelevant WrappedMetaCoin (and Migrations). I'm assuming this is because truffle compile compiles each contract one by one?

Truffle doesn't compile contracts one-by-one; instead, it looks for what files have changed and then compiles all files necessary for those changed files (so if A imports B, and B changes, then both A and B will compile)

More often than not, we'll want to assume that users will compile all their contracts at once, as if they ran truffle compile --all

from truffle-plugin-verify.

@gnidan then why is e.g. the Migrations never included in the metadata field of other artifacts. From what I understood earlier is that the metadata field includes all source files that were compiled at the same time, even if they don't import each other. But what I'm seeing is that only the source file of the artifact and all files it (recursively) imports are included.

I mean the behaviour I'm seeing is perfect for what we need for the plugin, but I'm just trying to understand the expected behaviour.

from truffle-plugin-verify.

Oh that's interesting @rkalis. I would expect Migrations to be present.

I'll play with this later and see if I can figure out what we can expect here for real.

from truffle-plugin-verify.

One (a bit complicated) way I've thought of to get the "expected" order is: ...

You might just skip the manual traversal here and use @truffle/compile-solidity/profiler - since that module houses logic for determining imports. Not sure if it will help determine source order or not, but it might?

from truffle-plugin-verify.