cms-sw / cmssw-config Goto Github PK

A first step should be to fix the call to nvcc for linking device code.
The current rule

  $(VERB_ECHO) $(SCRAM_PREFIX_COMPILER_COMMAND) "$(NVCC) $(call AdjustFlags,$1,,CUDA_LDFLAGS CUDA_FLAGS)  $? -o $@" &&\
              ($(SCRAM_PREFIX_COMPILER_COMMAND) $(NVCC) $(call AdjustFlags,$1,,CUDA_LDFLAGS CUDA_FLAGS) $? -o $@) $(endlog_$(1))

should become

  $(VERB_ECHO) $(SCRAM_PREFIX_COMPILER_COMMAND) "$(NVCC) -dlink $(call AdjustFlags,$1,,CUDA_LDFLAGS CUDA_FLAGS) --compiler-options '$(call AdjustCudaHostFlags,$1,CXXFLAGS) $(CXXSHAREDOBJECTFLAGS)' $? -o $@" &&\
              ($(SCRAM_PREFIX_COMPILER_COMMAND) $(NVCC) -dlink $(call AdjustFlags,$1,,CUDA_LDFLAGS CUDA_FLAGS) --compiler-options '$(call AdjustCudaHostFlags,$1,CXXFLAGS) $(CXXSHAREDOBJECTFLAGS)' $? -o $@) $(endlog_$(1))

where we explicitly pass -dlink, and also pass the full set of flags to the host compiler.
This should go together with the changes to the cuda toolfile at cms-sw/cmsdist#4168 .

Then, to avoid some confusion between the existing $(1)_cudaobj variable and $(1)_cudaobjs that I propose to add, I suggest to rename the former to $(1)_cudadlink (short for device link):

-$(1)_cudaobj      := $$($(1)_objdir)/$(1)_scramcuda.$(OBJEXT)
+$(1)_cudadlink    := $$($(1)_objdir)/$(1)_cudadlink.$(OBJEXT)
...
-$(1)_objs         := $$(addprefix $$($(1)_objdir)/, $$(addsuffix .$(OBJEXT),$$($(1)_files))) $$(if $(SYNTAX_ONLY),,$$($(1)_cudaobj))
+$(1)_objs         := $$(addprefix $$($(1)_objdir)/, $$(addsuffix .$(OBJEXT),$$($(1)_files))) $$(if $(SYNTAX_ONLY),,$$($(1)_cudadlink))
...
-ifneq ($(strip $($(1)_cudaobj)),)
-$($(1)_cudaobj): $$(addprefix $($(1)_objdir)/, $(addsuffix .$(OBJEXT), $($(1)_cudafiles)))
+ifneq ($(strip $($(1)_cudadlink)),)
+$($(1)_cudadlink): $$(addprefix $($(1)_objdir)/, $(addsuffix .$(OBJEXT), $($(1)_cudafiles)))

Finally, to support linking CUDA device code across different libraries, for each package with CUDA source files we need to:

• define a corresponding list of device object files and a static library (I picked the suffix _nv.o because of the ELF sections they are going to have, but feel free to use anything else):

   ifneq ($(strip $$($(1)_cudafiles)),)
   $(1)_cudadlink    := $$($(1)_objdir)/$(1)_cudadlink.$(OBJEXT)
  +$(1)_cudaobjs     := $$(addprefix $$($(1)_objdir)/, $$(addsuffix _nv.$(OBJEXT),$$($(1)_cudafiles)))
   endif

• copy the ELF sections __nv_relfatbin, __nv_module_id and .nvFatBinSegment to the device object files:

  $($(1)_cudaobjs): %_nv.$(OBJEXT): %.$(OBJEXT)
      objcopy -j '.nv*' -j '__nv*' $^ $@
  

• link these device object files into a static library:

  lib$(1)_nv.a:
      ar crs ... $($(1)_cudaobjs)
  

• during the device link step, take into account the full dependencies from the BuildFiles, but use these _nv.a libraries instead of the standard .so libraries.

One question: do we want to keep these _nv.a files in the same place as the .so libraries, or in a separate directory ? They are only used for linking, not at run time.

Hi,

I wanted to know if there is a possibility to have something like the output of git describe --tags available at compile time so as to be able to version executables. This could be achieved via a compile time -D name=definition option.

The use case is for tools which are loosely bound to the actual CMSSW version used, like https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit where we struggle with users using old versions of the tool.

I am not a SCRAM expert, do I do not know if something like this is already in place (for instance for $SCRAM_ARCH or $CMSSW_VERSION) and, if not, where to hook it up.

We need to concentrate on getting the actual code work multithreaded, before we bother about tests and binary executables. I would therefore propose we have an option to limit clang-static-analyzer to stuff which gets executed by cmsRun.

See cms-sw/cmssw#44506

Make sure that a CMS package A can only include public headers of other package B (i.e. B/interface). With in a package disallow to include headers from non interface sub-directories e.g. src, bin, plugins, test etc.

See cms-sw/cmssw#40792 (comment) i.e.

<flags ALPAKA_BACKENDS="cuda rocm"/> works. For longer term, I'm wondering would it be feasible
to add a possibility for an inverted logic, i.e. "all backends except the listed ones"? Although we should
re-think the relation of "platform" concept in DataFormats  (and ESProducers(?) and Services) and 
"backend" concept in EDModules, and the impact of that for the library organization and build
rules anyway, so I'm not if this detail is worth of addressing before the outcome of the
platform-backend thought process.

Add a buildrule/check to make sure that multiple cms edm plugins of a package do not generate same cfi python modules. Plugins from different packages can generate same cfi.py modules.

Just like plugins directory , shut the scram warning for test directory.
src//test contains only BuildFile.xml without any sources. Better to remove BuildFile.xml too or do not tag this directory

git cms-checkdeps allows selecting different kinds of dependencies (header, python, or BuildFile). Right now, scram b checkdeps always checks all of them. Often, I find cases where I want to skip the python dependencies (the python dependency check is very broad and often gives false positives) but keep the others. Could arguments similar to the git version be provided for the scram version?