Git Product home page Git Product logo

buildserver's Introduction

DEPRECATED

This repository contains instructions and script files for building Mixxx's dependencies and producing a Mixxx build server. This was used for Mixxx =< 2.3. Mixxx's dependencies are now built with vcpkg from our vcpkg repository (vcpkg was first used for Mixxx 2.3 on Windows, but Mixxx 2.3 still used the scripts in this repository for macOS).

Build Server Architecture

Our build environment consists of a master machine and worker machines that compile and package Mixxx. The master and workers need not be on the same machine. However, we typically run the master as a "real" host machine and the workers as virtualized guests on that machine.

Standalone Build Environment

If you're looking to build Mixxx on your own computer, you don't need this repository. You can follow instructions on our wiki for each platform:

Pre-built Environments

Pre-built environments are available for Windows and macOS at:

To see which environment Mixxx 2.3.x builds are currently using, check packaging/macos/build_environment and packaging/windows/build_environment files in the Mixxx source tree.

buildserver's People

Contributors

be-ing avatar daschuer avatar ferranpujolcamins avatar holzhaus avatar joergatgithub avatar josepmajaz avatar palakis avatar pegasus-rpg avatar rryan avatar sblaisot avatar uklotzde avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

buildserver's Issues

Win: Missing DLLs in dynamic build

It looks like the Windows buildserver is not working properly for dynamic builds. Various dependencies are built in static mode, even though build_environment.bat is given the Dynamic argument. This results in unmet dependencies when compiling Mixxx.

This is my build.bat script for Mixxx: https://pastebin.com/cd1bctuX
And this is the output: https://pastebin.com/RABLSfCY

I ran build_environment.bat x64 ReleaseFastbuild Dynamic, which fails to build dynamic libraries for:

  • libid3tag
  • libmad
  • libopus
  • libshout
  • libsndfile
  • portaudio
  • portmidi (porttime)
  • protobuf
  • pthreads
  • rubberband
  • sqlite3
  • taglib

For completeness, these are the output files in buildserver/lib per dependency.

*(!)  = Static version of library built instead of shared one.

.
├── build_chromaprint.bat
│   ├── chromaprint.dll
│   ├── chromaprint.lib
│   └── chromaprint.pdb
├── build_fftw3.bat
│   ├── libfftw-3.3.dll
│   ├── libfftw-3.3.lib
│   ├── libfftw-3.3.pdb
│   ├── libfftwf-3.3.dll
│   ├── libfftwf-3.3.lib
│   └── libfftwf-3.3.pdb
├── build_hss1394.bat
│   ├── libHSS1394.dll
│   ├── libHSS1394.lib
│   └── libHSS1394.pdb
├── build_libflac.bat
│   ├── libFLAC.dll
│   ├── libFLAC.lib
│   └── libFLAC.pdb
├── build_libid3tag.bat (!)
│   ├── libid3tag.lib
│   └── libid3tag.pdb
├── build_libmad.bat (!)
│   ├── libmad.lib
│   └── libmad.pdb
├── build_libogg.bat
│   ├── libogg.dll
│   ├── libogg.lib
│   └── libogg.pdb
├── build_libopus.bat (!)
│   ├── opus.lib
│   └── opus.pdb
├── build_libshout.bat (!)
│   ├── libshout.lib
│   └── libshout.pdb
├── build_libsndfile.bat (!)
│   ├── g72x.lib
│   ├── libsndfile.lib
│   └── libsndfile.pdb
├── build_libvorbis.bat
│   ├── libvorbis.dll
│   ├── libvorbisfile.dll
│   ├── libvorbisfile.lib
│   ├── libvorbisfile.pdb
│   ├── libvorbis.lib
│   └── libvorbis.pdb
├── build_openssl.bat
│   ├── libeay32.dll
│   ├── libeay32.lib
│   ├── ssleay32.dll
│   └── ssleay32.lib
├── build_portaudio.bat (!)
│   ├── portaudio.lib
│   └── portaudio.pdb
├── build_portmidi.bat (portmidi=OK, porttime=(!))
│   ├── portmidi.dll
│   ├── portmidi.lib
│   ├── portmidi.pdb
│   ├── porttime.lib
│   └── porttime.pdb
├── build_protobuf.bat (!)
│   ├── libprotobuf-lite.lib
│   └── libprotobuf-lite.pdb
├── build_pthreads.bat (!)
│   ├── pthread.lib
│   └── pthread.pdb
├── build_qt4.bat (no output files, but that's OK, libs are in Qt folder)
├── build_rubberband.bat (!)
│   ├── rubberband.lib
│   └── rubberband.pdb
├── build_sqlite3.bat (!)
│   ├── sqlite3.lib
│   └── sqlite3.pdb
├── build_taglib.bat (!)
│   ├── tag.lib
│   └── tag.pdb
└── build_zlib.bat
    ├── zdll.lib
    ├── zlibwapi.dll
    ├── zlibwapi.lib
    └── zlibwapi.pdb

Absolute path dependencies

The following libraries still put and expect files to be located in C:\mixxx\environments\prototype\build

  • chromaprint-1.1
  • taglib-1.9.1

Workaround: Move the buildserver contents into C:\mixxx\environments\prototype

move state from Jenkins configuration scripts to this repository

Currently, some of the Jenkins configuration scripts rely on specific version numbers of libraries. This is cumbersome to maintain because it requires changing versions numbers scattered in different places (build scripts, Jenkins config scripts for PRs, Jenkins config scripts for release branches, Jenkins build scripts for master branch) when a library is updated. Instead, everything that changes regularly with dependency versions should be in this repository.

link warnings with opusfile PDBs

e.g. ```opusfile.lib(bands.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(bands.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(celt.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(celt.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(celt_decoder.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(celt_decoder.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(celt_lpc.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(celt_lpc.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(cwrs.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(cwrs.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(entcode.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(entcode.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(entdec.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(entdec.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(entenc.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(entenc.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(kiss_fft.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(kiss_fft.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(laplace.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(laplace.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(mathops.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(mathops.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(mdct.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(mdct.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(modes.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(modes.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(pitch.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(pitch.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(quant_bands.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(quant_bands.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(rate.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(rate.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(vq.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(vq.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(pitch_sse.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(pitch_sse.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(x86cpu.obj) : warning LNK4099: PDB 'celt.pdb' was not found with 'opusfile.lib(x86cpu.obj)' or at 'C:\projects\mixxx\win64_build\celt.pdb'; linking object as if no debug info
opusfile.lib(bwexpander.obj) : warning LNK4099: PDB 'silk_common.pdb' was not found with 'opusfile.lib(bwexpander.obj)' or at 'C:\projects\mixxx\win64_build\silk_common.pdb'; linking object as if no debug info
opusfile.lib(bwexpander_32.obj) : warning LNK4099: PDB 'silk_common.pdb' was not found with 'opusfile.lib(bwexpander_32.obj)' or at 'C:\projects\mixxx\win64_build\silk_common.pdb'; linking object as if no debug info
opusfile.lib(CNG.obj) : warning LNK4099: PDB 'silk_common.pdb' was not found with 'opusfile.lib(CNG.obj)' or at

macOS environment is not relocatable

The dylibs and binaries produced by the macOS environment are not movable since their dependencies are encoded with absolute paths instead of relative paths. This means others cannot download the environment and use it, since the binaries (protoc, moc, etc.) won't work. The dylibs get fixed to be bundle-relative with install_name_tool by Mixxx's packaging process, so this is why it went unnoticed.

e.g.

otool -L environments/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libshout.dylib
environments/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libshout.dylib:
	/Users/mixxx/bs-2.1-mac/amd64/environment/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libshout.3.dylib (compatibility version 6.0.0, current version 6.0.0)
	/Users/mixxx/bs-2.1-mac/amd64/environment/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libvorbis.0.dylib (compatibility version 5.0.0, current version 5.8.0)
	/Users/mixxx/bs-2.1-mac/amd64/environment/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/mixxx/bs-2.1-mac/amd64/environment/2.1-j00016-21856a6-osx10.7-x86_64-release/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

Mac: Remove universal binary support.

Now that macOS is pretty much all amd64, we don't need to make universal PPC/i386/amd64 builds, so we can remove a bunch of code that supports that.

update macOS builder to 10.13

The macOS builder is still running macOS 10.11. Eventually this will prevent us from upgrading the macOS SDK to the version required by Qt.

libshout PDBs aren't named right

We get warnings like this when building Mixxx:

'C:\projects\mixxx\win64_build\lib.pdb'; linking object as if no debug info
libshout.lib(buffer.obj) : warning LNK4099: PDB 'lib.pdb' was not found with 'libshout.lib(buffer.obj)' or at 'C:\projects\mixxx\win64_build\lib.pdb'; linking object as if no debug info

Windows Jenkins configuration script assumes some dependencies present on server

The Jenkins configuration scripts assume that the ASIO SDK and MSVC Redistributable are present on the build runner and assume specific filenames. This setup requires write access to the build runner's storage to update those dependencies. Instead, those should be downloaded by the build scripts if they are not already cached in the Jenkins workspace like how the macOS scripts work.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.