I wonder whether the University can provide the TotalView debugger. An efficient use of Iceberg is of course parallelization. However, support for developing parallel code is rather limited on Iceberg. It'd be extremely useful to make something like TotalView available to the cluster and to individual developers.

this page: http://rcg.group.shef.ac.uk/iceberg/software/compilers/git.html
references the install script https://github.com/rcgsheffield/iceberg_software/blob/master/software/install_scripts/compilers/git/install_git_5.2.sh but that install script doesn't exist there.

I found it in https://github.com/rcgsheffield/iceberg_software/blob/master/software/install_scripts/apps/git/install_git_2.5.sh (apps instead of compilers)

So we could fix the URL, sure, and I was about to submit a PR for that, but really the URL for the original page is wrong. Should be http://rcg.group.shef.ac.uk/iceberg/software/app/git.html and I'm not sure how to change that.

In the Abaqus section, I suggested that /scratch was used as the scratch directory. Is this correct?

I'm wondering if /fastdata would be an option. Any reason to not use fastdata for Abaqus scratch? How reliable is it for example?

Supportworks request from: Marco Mcsweeney

http://www.broadinstitute.org/igv/
Requested by Afsaneh Maleki-Dizaji for teaching. Deadline 26th October.

There are no installation notes for MATLAB available. I understand that a basic install is straightforward but integration with the SGE is rather more tricky. How was it done for the current installs?

Requested by Stephen McCord

Website: https://orcaforum.cec.mpg.de/

I've added a HelloWorld example in .cpp

https://github.com/mikecroucher/iceberg_software/blob/master/software/compilers/examples/helloworld/hello.cpp

to allow us to refer to something explicit when writing how to use the various compilers.

How should we be referring to such files in the text? Options include

• Link to the github url (but may lead to people struggling to download it)
• Include the code explicitly in the text so users copy and paste
• A separate iceberg-examples repo - we could encourage users to git-clone this to get all examples.

As I write this, I'm favouring the final option. Thoughts?

While writing the new user documentation for Abaqus, I noted an anomaly in that even though I had selected Intel Compiler 12.1 in my job submission file, it compiled using Intel 14.0
Investigation determined that we are doing something non-standard in our use of the abaqus command. Namely ::

#!/bin/bash
## 
#Record the use of this script by Stuart Mumford.
# logger -p local1.debug "iceberg_software: USER=$USER TYPE=monitor SOFTWARE=/usr/local/bin/abaqus"
#DS straight link will not work 
module load compilers/intel

export XLIB_SKIP_ARGB_VISUALS=1
mesa=" "
if [ "$1" == "cae" ] ;then
    mesa="-mesa"
fi
if [ -z $ABAQUSCOMMAND ] ; then 
   echo " Please use 'module load apps/abaqus' command first "
   exit 1
fi

 $ABAQUSCOMMAND $* $mesa

My main issue with this is that the abaqus command will always load the latest version of the Intel compiler. The practical upshot of this is that if we add a new version of the Intel Compiler, we will be changing the abaqus environment for users without them necessarily knowing! We have needlessly coupled Intel Compiler to Abaqus! Such tight coupling is rarely a good idea

This could lead to issues with reproducibility.

In summary:

• It is impossible for us to upgrade the Intel Compiler without running the risk of breaking this Abaqus command.
• We could inadvertently ruin a user’s set of results. E.g. if we update the Intel Compiler during a parameter sweep, different versions of the compiler will be used for different runs. This a Bad Idea!
• A user from a different site will expect to control which version of the Intel Compiler they use. Here, we always force the latest.
• This wrapper to abaqus is non-standard and may lead to further problems. It forces the -mesa switch for example. This switch is often, but not always, useful.

I suggest a change where we move a more standard way of working. Failing that, we should fully document this non-standard modification and warn users of the implications.

For now, I have modified the documentation such that it explicitly references the abq133 command.

Deadline 26th october. Requested by Afsaneh Maleki-Dizaji

http://cole-trapnell-lab.github.io/cufflinks/install/

We should distill something useful from the discussion at https://groups.google.com/a/sheffield.ac.uk/forum/#!topic/hpc/WEtdsOFP6fs and put it into the docs.

In the old documentation: https://www.shef.ac.uk/wrgrid/gpu/iceberg/applications

ansys-linemode and runansys appear to be a local developments but I can't find any documentation about it.

Furthermore, where do bench01.mac and bench05.mac come from?
I'd like to give details on how to reproduce the results shown in the trable.

Some random facts that need including (taken from a recent support call)

It's possible to use the hybrid openmpi environments to get exclusive access to a node, (this PE is intended for mixed Openmpi-OpenMP jobs, but can also be used for normal MPI jobs which require specific placement)
e.g.

$ -pe openmpi-hybrid-16 [number of cores]

will work like the normal openmpi-ib parallel environment, but place your processes in groups of 16 per node, basically filling up each node. (note this will likely increase your job wait time). The 16-core hybrid environment is only available on Iceberg's newest nodes.

You can also use 12-core hybrid PE to run on the older nodes, but you'd also need to add the arch=intel-x5650 option to make sure that the job ran on a machine with 12 cores only.

$ -pe openmpi-hybrid-12 [number of cores]

$ -l arch=intel-x5650

http://ccb.jhu.edu/software/tophat/tutorial.shtml

Requested by Afsaneh Maleki-Dizaji

Looking at the install instructions, I'm wondering if a binary install might be the way forward.
Tophat has re-reqs too. Some of which we don't have yet. bowtie2 for example

https://www.broadinstitute.org/gatk/

Requsted by Afsaneh S Maleki-Dizaji for teaching. Deadline October 26th

I installed it as an emergency request. Need to follow up with the documentation.

http://su2.stanford.edu/

Requested by Michael Donnelly

I had a query about MOSEK. The user wanted to use it within MATLAB.
We might be eligible to get an academic license for free! One of us should follow this up perhaps?

https://www.mosek.com/resources/academic-license

It would be nice to have some examples of using Bowtie2 in batch mode. The current documentation is a little sparse http://rcg.group.shef.ac.uk/iceberg/software/apps/bowtie2.html

Also, a benchmark that takes a while (no more than a day) which we could use to demonstrate multithreading mode and try out different compiler switches at build time.

Hopefully such examples can be provided that aren't too large?

Perhaps the bioinfomatics community can help us out?

The Compilers section currently has CMake and git along with the compilers. I suggest that we rename this section 'Development Tools'.

The background colour is wrong for a start.

CiCS advertise courses for various HPC-related stuff here

https://sites.google.com/a/sheffield.ac.uk/rcg/my-blog/researchitcoursesfor2015-2016autumnsemester

We should have links to the course material from the relevant Iceberg documentation pages.

This is a pre-req for #153

http://sourceforge.net/projects/bowtie-bio/files/bowtie2/

The license server will need updating first though!

I can't find any details in the old docs.

We need:-

• The modulefile
• Instructions on batch usage. Last time I checked, you couldn't run a notebook in batch.

and it really should be!

The current front page of the website is very sparse. What else is required?

Hi,

Asking on behalf of the Materials Science team, if we can have CASTEP 9.0 (it's labelled as 16.1 now for some reason?) installed on Iceberg, and for it to run along side 8.0 until we are happy that it is a stable version.

Thanks,

Chris

I want to build something that links to it. I'm guessing that the gcc version matters...but could be wrong of course.

The old docs are at https://www.shef.ac.uk/wrgrid/using/access/browser

Given that this no longer works on Chrome, are we still going to support it?
If so, the docs need writing.

It would be useful to add a small but complete example of how to compile and run an OpenACC CUDA job using the PGI Compilers.

I have installed a basic version of Octave 4.0. Its usable and includes the GUI but is missing support for several libraries (all of which will need to be compiled separately). I need to document this.

At the moment we have one page per version, is this the best way?

What about subsections on one pages?

http://sourceforge.net/projects/bio-bwa/files/

requested by Afsaneh Maleki-Dizaji for teaching. Deadline Monday 26th October

This stuff

https://www.shef.ac.uk/wrgrid/data

Perl 5.x - was requested. We have it installed as part of the system but there is no documentation.

We sometimes get calls like this:

Today I cannot see my Iceberg /data/username directory on Filezilla/WinSCP. I
can navigate to it via Exceed (so I know it is there) but it isn't showing
up on the FTP programs.

Need to

• ensure that the automounting behaviour is documented (yes; see PR #449)
• also mention this in the troubleshooting section (PR #553)

The changes to the html table formatting have broken the references and software sidebar.

Something to run to help exercise our Code Saturne install is required.

http://sourceforge.net/p/bless-ec/wiki/Home/

Requested by Suo Qui

I'm building a library that depends on HDF5. Its config tool is asking me if HDF5 needs SZIP and ZLIB. I have no idea how to answer this!

I note that pre-built binaries at https://www.hdfgroup.org/HDF5/release/obtain5.html have been built with these enabled. How were ours built? How might I find out?

Please could we install this program for analysis and annotation of DNAseq data. Here is all the info: http://gemini.readthedocs.org/en/latest/

Thanks

Johnathan

Current look is horrible!

So I created a very basic fortran code that calls the NAG library just so I can figure how to compile it.

PROGRAM ExampleNAG1


 USE nag_library, ONLY : A00AAF

        CALL A00AAF

END PROGRAM

I followed the old doc and the new doc.

So after loading the intel compiler and using both,

ifort sourcefile.f90    $NAGLIB    -I$NAGINC

and

ifort your_code.f90 -lnag_mkl -o your_code.exe

it errored with

catastrophic error: Too many errors, exiting
compilation aborted

However the examples worked when ran (nag_example a00aaf and nagexample24 d03phf)

The compile line for the first example is,

ifort -I/usr/local/packages5/nag/fll6i24dcl/nag_interface_blocks a00aaf.f90 -lnag_mkl -o a00aaf

and if I do the same with my example code, it compiles.

So is the nag_interface_blocks needed to compile the code every time?

Information regarding the Salford Fortran compiler for Windows is currently on the old documentation site. Since this is not installed on Iceberg, it should not be included in the new documentation but should, instead, be migrated to the Sheffield software download service.

https://www.sheffield.ac.uk/cics/software

bedtools 2.22.1 https://github.com/arq5x/bedtools2/releases

For Afsaneh Maleki-Dizaji. Deadline 26th October.

picard 1.129 https://github.com/broadinstitute/picard/releases/tag/1.129

(requires Java 1.6 there is info on this here http://broadinstitute.github.io/picard/building.html#BuildingPicardUtilities)

For Afsaneh S Maleki-Dizaji for teaching. Deadline October 26th

Documentation concerning some of the SGE accounting mechanism would be useful. Here's a random collection of facts:

qacct -j [jobid]

...you may end up with multiple jobs matching because our JobIDs have wrapped around.

The accounting file is at /usr/local/sge/live/default/accounting.
Records look like this

interactive.q:testnode03.iceberg.shef.ac.uk:fe:fe1abc:INTERACTIVE:26833:sge:0:1433190433:1433190450:1433190450:0:1:0:0.003999:0.001999:1612.000000:0:0:0:0:710:0:0:16.000000:8:0:0:0:11:3:SHEFFIELD:defaultdepartment:NONE:1:0:0.005998:0.000000:0.000000:-U cstest -u fe1mpc -l h_vmem=16G -I y -P SHEFFIELD:0.000000:NONE:0.000000:0:0

See man accounting for the fields.

Apparently this info also ends up on our splunk server. What's that? Where is it? How does it end up there? How can it be used. So many questions.....

https://vcftools.github.io/index.html

Requested by Eran Elhaik