jasonwhite / ducible Goto Github PK

I would like to run pdbdump and get structured and easily parsable output for use in tracecode-debug, a tool for an upcoming static analysis companion to https://github.com/nexB/tracecode-build

A number of test cases need to be added to verify that builds are indeed deterministic. This will involve building every set of sources files twice and comparing the checksums to verify they are identical. The negative case also needs to be tested. That is, modifying a source file should affect the output and produce non-identical files.

The plan is to write this test suite in Python as we already have a dependency on it.

It allows to trim the leading part of the path from __FILE__, thereby adding determinism.

Currently, PDB files are not patched to remove non-determinism.

This presents a larger challenge than just patching the PE file alone, because:

1. The PDB file format is not well documented.
2. The size of the PDB file seems to change with every build. Thus, we cannot use a memory map to make modifications to the PDB file as we will need to remove data.

References:

1. https://github.com/Microsoft/microsoft-pdb

The timestamp of a DLL seems to have at least some kind of role as a unique identifier that is relevant to the loader. Using the same timestamp for different versions of a DLL might introduce weird side effects.

Instead, the timestamp should be set to the hash of the input file to guarantee a unique value, see https://devblogs.microsoft.com/oldnewthing/20180103-00/?p=97705 which describes that Windows 10 DLLs now also use a hash of the file as a timestamp.

The source tarballs (or at least v1.2.1.tar.gz) have no version.h, so make tries to configure version.h.in, which fails because there is no git checkout.

Source tarballs are important for projects that package binaries such as Linux distributions or MSYS2. They are more prone to include a package that is built from a source tarball that corresponds to a version number. Packages built from version control checkouts are frowned upon because that is a sign of instability.

Hello I have a problem. We have the original .exe and .pdb file of an application we released - that was digitally signed due to being released to customer desktops. Now the application randomly crashed and we have a dump file from the crash.

Now my question is does that mean there is no way for us to look at the dump file because the exe was digitally signed or is there some way around this with the help of your tool or in general?

The Windows build does not have any version information. This should be added to make things more polished.

Downloaded trunk using SVN, but prebuild step fails to generate "version.h" if repository is not a GIT one. Using GIT worked fine.

When building incrementally, the linker fails with the error LNK1209: program database 'foo.pdb' differs from previous link; relink or rebuild.

This could be caused by a number of reasons, one being that the resulting PDB is simply not valid. The linker may also keep information about the state of the PDB separately (maybe in the .ilk file). This needs to be researched more.

This is part feature request/part question.

Even after running the DLL and PDB through ducible, I found that the PDB hashes would differ, so the PE hashes would also differ, leading to cache misses.
Turns out that the linker writes out absolute paths to .obj files in the module info substream!
This is particularly obvious across different developer machines where their usernames are different.

Have you considered adding support to replace those with relative or fake absolute paths?
Do you know if there are any downsides of doing something like that?

Hi Jason,

First, I would like to warmly thanks you for this program. We added ducible to our build pipeline and it is a huge gain of time while debugging.

I would like to know if it is possible to make ducible work for .lib file? AFAIK it is generating a pdb as well, that has the same limitations as the .dll one.

Thank you for your time!