Third party LICENSE collection error when build wheel package about pytorch HOT 8 CLOSED

And I tried removing them and rebuilding the whl package now. It may takes a bit of time.

It works, thank you very much!

from pytorch.

Tentatively assigning to @c-p-i-o , who added the dependency in #122999

@ezyang just curios, is it used for anything yet? If not, then perhaps revert?

P.S. I wonder how it works with the nightly builds though?

from pytorch.

I found this issue when building directly whl on my ubuntu machine (without using docker), or maybe I need to use docker (follow the ci script) to build it?
And if I set "Unknown license" as return value rather than value error, it works.

        if 'ApacheLicense' in txt:
            # Hmm, do we need to check the text?
            return 'Apache-2.0'
        .
        .
        .
        elif all([squeeze(m) in txt.lower() for m in bsd3_src_txt]):
            return 'BSD-Source-Code'
        elif any([squeeze(m) in txt.lower() for m in mit_txt]):
            return 'MIT'
        else:
            # A license may not be on this list, but is still a valid license.
            raise ValueError('unknown license')

from pytorch.

We did attempt to fix this in build_bundled.py script (https://github.com/pytorch/pytorch/blob/main/third_party/build_bundled.py#L117)
Is this fix not working?

from pytorch.

Tentatively assigning to @c-p-i-o , who added the dependency in #122999

@ezyang just curios, is it used for anything yet? If not, then perhaps revert?

We are waiting for some higher level abstractions for metrics to land. Once landed, for the OSS community, we plan on implementing those higher level abstractions via OTEL.

P.S. I wonder how it works with the nightly builds though?

Works on nightly because of (https://github.com/pytorch/pytorch/blob/main/third_party/build_bundled.py#L117)

from pytorch.

@xiaoran007 FWIW, the issue didn't repro for me on main.

python setup.py bdist_wheel

...
adding license file 'LICENSE'
adding license file 'NOTICE'
...
adding 'torch-2.4.0a0+gitc5d83bc.dist-info/LICENSE'
adding 'torch-2.4.0a0+gitc5d83bc.dist-info/METADATA'
adding 'torch-2.4.0a0+gitc5d83bc.dist-info/NOTICE'
adding 'torch-2.4.0a0+gitc5d83bc.dist-info/WHEEL'
adding 'torch-2.4.0a0+gitc5d83bc.dist-info/entry_points.txt'

I think what's happening here is that

1. you might have checked out main. This fetches third-party/opentelemetry-cpp.

2. Now, when you switch over to tag v2.1.0, you end up pulling an oldbuild_bundled.py from the tag but you still have third-party/opentelemetry-cpp in your directory left there.

3. The old build_bundled.py script tries to iterate over the new third-party/opentelemetry-cpp library looking for LICENSE files and fails as it doesn't understand the weird sigslot file.

To confirm this, you'd need to do git status after you checkout v2.1.0 tag and you should see third-party/opentelemetry-cpp as a untracked files

Can you confirm this is what might be happening?

from pytorch.

Yes, when I checkout to tag v2.1.0, the files are untracked:

HEAD detached at v2.1.0
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        third_party/cpp-httplib/
        third_party/opentelemetry-cpp/

And I tried removing them and rebuilding the whl package now. It may takes a bit of time.

from pytorch.

Thanks @xiaoran007!
I'm closing this issue for now - but re-open it if you run into any further issues.

from pytorch.