FragPipe-equivalent workflow on Linux for Brucker .d files about msfragger HOT 28 CLOSED

Hi Natalia,

Looks like your C++ std library is not compatible with Bruker's library. We had the same problem on our RedHat. I solved this by using a latest Ubuntu docker (https://hub.docker.com/_/ubuntu/). Basically, you need to pull the docker image and start a docker container with it. Then, in the docker environment (it is basically an Ubuntu), run MSFragger's command. Again, remember to always copy/move the ext folder with MSFragger.jar.

It need some basic knowledge about docker. Feel free to let me know if you have any problem starting a docker container.

Best.

Fengchao

from msfragger.

Hi Natalia,

Thanks for your interest in MSFragger and IMQuant. You need to keep the ext folder sitting alongside the MSFragger.jar so that it can find the library to read raw files.

If you want to use IMQuant in the command line, you can find the jar files in your operate system's temp folder by looking at the command printed in FragPipe. Here is an example from my computer:

IMQuant [Work dir: C:\Users\yufe\Desktop]
java -Xmx108G -Dbruker.lib.path="C:\Users\yufe\Desktop\JAR\ext\bruker" -cp "C:\Users\yufe\AppData\Local\Temp\fragpipe\batmass-io-1.17.2.jar;C:\Users\yufe\AppData\Local\Temp\fragpipe\imquant-1.6.5.jar" imquant.IMQuant --mztol 10 --imtol 0.05 --rttol 0.4 --minfreq 0.50 --plot 1 --psm C:\Users\yufe\Desktop\psm.tsv E:\data\20180819_TIMS2_12-2_AnBr_SA_200ng_HeLa_50cm_120min_100ms_11CT_1_A1_01_2767.d 20180819_TIMS2_12-2_AnBr_SA_200ng_HeLa_50cm_120min_100ms_11CT_1_A1_01_2767.pepXML

As you can see, the jar files are in C:\Users\yufe\AppData\Local\Temp\fragpipe\.

After getting your jar file, you can type similar commands in Linux by mimicking what FragPipe does and changing the paths accordingly.

Again, remember to copy/move the ext folder along with MSFragger.jar and also put IMQuant.jar in the same folder (because it also needs it).

Best,

Fengchao

from msfragger.

Hi Fengchao,

Thanks a lot for a quick reply.

The ext folder already is together with MSFragger.jar.
I am struggling however with adapting this line from you script:
java -Xmx64G -jar $msfraggerPath $fraggerParamsPath $dataDirPath/*.mzML

to work with the Brucker files. I tried to add the path to the Brucker libraries but I did not succeed.

I tried:

java -Xmx64G -jar -Dbruker.lib.path="/cl_tmp/FragPipe/MSFragger-2.2/ext/bruker/" $msfraggerPath $fraggerParamsPath $dataDirPath/*.d

but it was obviously wrong. I'm afraid I'm not much of a java person yet.

I checked in the C:\Users\admin\AppData\Local\Temp\fragpipe folder as indicated by the logfiles, but there are no .jar files in there... Is there somewhere where I could download it from?

Many thanks,

Natalia

from msfragger.

Hi Natalia,

You only need to replace $dataDirPath/*.mzML with $dataDirPath/*.d, where .d is your Bruker raw folder.

You don't need to add ext path to the command, MSFragger will be able to find it as long as they are in the same folder.

BTW, you need to replace $msfraggerPath, $fraggerParamsPath, and $dataDirPath with your real paths.

In order to get the jar files in your temp folder, you need to click Run in FragPipe (and check dry run if you don't want to really run it).

Best,

Fengchao

from msfragger.

Hi Fengchao,

I'm afraid that is exactly what I have been doing all along. I have now reloaded the MSFragger and the ext folder, on the off chance that something had gotten corrupted and tried again, but with the same results. It definitely cannot find Brucker native libraries. Do you have any ideas why it might be so?

Could that perhaps be an os issue? Does MSFragger run on CentOS 7.7?
What are the system requirements?

I'm asking because if I try:

java -Xmx64G -agentpath:/cl_tmp/MSFragger-2.2/ext/bruker/libtimsdata-2-4-4.so -jar $msfraggerPath $fraggerParamsPath $dataDirPath/*.d

(after defining all the variables, like in your script, of course), I get:

Error occurred during initialization of VM
Could not find agent library /cl_tmp/MSFragger-2.2/ext/bruker/libtimsdata-2-4-4.so in absolute path, with error: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /cl_tmp/MSFragger-2.2/ext/bruker/libtimsdata-2-4-4.so)

We currently have GLIBCXX_3.4.19 and I'm afraid there might be problems with updating it for the whole system...

Cheers,

Natalia

from msfragger.

Hi Fengchao,

Unfortunately, I would need to run it under SGE on which I'm not a root either. But I think I found a different workaround and MSFragger and Philosopher seem to be running alright. However, the IMQuant does not, which sadly renders all the previous steps useless...

I couldn't find any documentation for IMQuant, which is part of the problem: I'm not sure if the syntax is quite right. Do you have a "Fragpipe" script with the IMQuant you could share perhaps?

However, I'm afraid there's a more serious underlying problem. When I tried to run:
$ java -jar imquant-1.6.5.jar

I got:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: umich/ms/fileio/filetypes/xmlbased/AbstractXMLBasedDataSource
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: umich.ms.fileio.filetypes.xmlbased.AbstractXMLBasedDataSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more

Any ideas what the problem might be? Looks like some compatibility issue again.

Many thanks for all your help,

Natalia

from msfragger.

Hi Natalia,

You need to use something like java -Xmx108G -Dbruker.lib.path="C:\Users\yufe\Desktop\JAR\ext\bruker" -cp "C:\Users\yufe\AppData\Local\Temp\fragpipe\batmass-io-1.17.2.jar;C:\Users\yufe\AppData\Local\Temp\fragpipe\imquant-1.6.5.jar" imquant.IMQuant --mztol 10 --imtol 0.05 --rttol 0.4 --minfreq 0.50 --plot 1 --psm C:\Users\yufe\Desktop\psm.tsv E:\data\20180819_TIMS2_12-2_AnBr_SA_200ng_HeLa_50cm_120min_100ms_11CT_1_A1_01_2767.d 20180819_TIMS2_12-2_AnBr_SA_200ng_HeLa_50cm_120min_100ms_11CT_1_A1_01_2767.pepXML.

Please note this part -cp "C:\Users\yufe\AppData\Local\Temp\fragpipe\batmass-io-1.17.2.jar;C:\Users\yufe\AppData\Local\Temp\fragpipe\imquant-1.6.5.jar" imquant.IMQuant, it needs batmass-io-1.17.2.jar.

Best,

Fengchao

from msfragger.

Hi,

This is what I have been doing. Is there any IMQuant documentation available anywhere, including the system requirements?

I think the real issue is the JNI error. I don't think anything will run until that is sorted Any ideas hat might be causing it?

Many thanks,

Natalia

from msfragger.

Hi Natalia,

You will be able to see printed help document after you can run IMQuant.jar. The system requirement of IMQuant is the same as MSFragger. Since you can run MSFragger, you should be able to run IMQuant.

Your error is NoClassDefFoundError which implies that IMQuant cannot find certain classes, which is very likely due to the missed batmass-io-1.17.2.jar in calling imquant.jar (according to your previous message, you used java -jar imquant-1.6.5.jar which was WRONG.). You should get batmass-io-1.17.2.jar from FragPipe's temp folder and append it to your command line.

Best,

Fengchao

from msfragger.

batmass-io-1.17.2.jar is there alright. Why is imquant-1.6.5.jar wrong? I was the only imquantxxx.jar file in the temp folder.

from msfragger.

Is your command line java -jar imquant-1.6.5.jar from your previous message? If not, could you please send me your real command line?

from msfragger.

yes, but thats the name of the file from the Temp folder after the local run on Windows

from msfragger.

But the correct command line should be something like java -Xmx108G -Dbruker.lib.path="ext\bruker" -cp "batmass-io-1.17.2.jar;imquant-1.6.5.jar" imquant.IMQuant

from msfragger.

no, that was just to get some details, as per your previous instruction to a different enquiry to somebody else:

"You may get detailed help info by typing

java -jar IMQuant.jar"

But even this does not work.
So I'm afraid the quantification won't either...

from msfragger.

I am sorry but I don't think you read the whole thread of that issue. I amended that reply by saying (https://github.com/Nesvilab/FragPipe/issues/158#issuecomment-563330554)

Sorry I forgot to mention that you can find those jar files by checking the command line printed by FragPipe. Here is an example from my computer:

java -Xmx119G -cp "C:\Users\yufe\AppData\Local\Temp\fragpipe\batmass-io-1.17.0.jar;C:\Users\yufe\AppData\Local\Temp\fragpipe\imquant-1.6.2.jar" imquant.IMQuant ......

Best,

Fengchao

And the issue was addressed. I think you should also use the similar command line rather than java -jar IMQuant.jar, which I already explained several times in my previous replies.

from msfragger.

This does not really matter, I get a "A JNI error has occurred, please check your installation and try again" in all cases and this is what really troubles me. I see now reason why

java -jar imquant-1.6.5.jar

would throw this error.

(same error is thrown when I try to run the full script)

from msfragger.

Could you please try typing a command line like java -Xmx108G -Dbruker.lib.path="ext\bruker" -cp "batmass-io-1.17.2.jar;imquant-1.6.5.jar" imquant.IMQuant (by replacing your real path) and send a screen shot to me?

from msfragger.

I did it and it doesn't work. throwing:
Error: Could not find or load main class imquant.IMQuant

However, it works fine when I do it locally on windows (where FragPipe runs as well)

I now have the same (latest) java version on both...

from msfragger.

Hmm. That's weird. IMQuant is a pure java program except for the Bruker library which is written in C++ and provided by the vendor. If you can run it in Windows but not Linux, one possible reason is that the Bruker library could not work properly due to some reasons but Java returned a misleading error message (I had this before actually.). But you also said that you MSFragger could read Bruker's .d folder in your Linux (Am I right?), which implies that the Bruker library works well in your Linux.

If there is not problem in the command and the environment, I have no clue about this error right now. But batmass-io has some options to print diagnostic/debug messages. I need @chhh 's help on this.

@chhh Could you please take a look and provide some suggestions?

Thanks,

Fengchao

from msfragger.

MSFragger seem to have run through ok. I got very reasonable number of identifications on all levels. I used a workaround that was successfully used for maxquant and the Brucker libraries, so I think this should be alright...

from msfragger.

Just in case. I compiled a jar file with all libraries: https://umich.box.com/s/e6nzs6htx4h5bhouz2c0l08z3z3kas2k You should be able to run it by typing java -jar IMQuant-1.6.5-jar-with-dependencies.jar with adjusted paths. But you still need to keep the ext folder in the same folder of the jar file. Please let me know if this one works well in your Linux.

Best,

Fengchao

from msfragger.

from msfragger.

Hi Alexey,

I think we need one. I also need to create a IMQuant GitHub page so that people can download a complete jar rather than copy it from FragPipe's temp folder. I will work with @hayse1 regarding the tutorial.

Best,

Fengchao

from msfragger.

Just in case. I compiled a jar file with all libraries: https://umich.box.com/s/e6nzs6htx4h5bhouz2c0l08z3z3kas2k You should be able to run it by typing java -jar IMQuant-1.6.5-jar-with-dependencies.jar with adjusted paths. But you still need to keep the ext folder in the same folder of the jar file. Please let me know if this one works well in your Linux.

Best,

Fengchao

Hi Fengchao,

java -jar IMQuant-1.6.5-jar-with-dependencies.jar doesn't cause any errors anymore, thanks a lot!

Does it mean that I should now skip -Dbruker.lib.path and -cp for batmass-io-1.17.2.jar in my script, as they are included in the new .jar file?

Also - so sorry to bother you even more - how do I specify the experiments and fractions on linux? At the moments it is being collated all in one file, which in my set up makes no biological sense.

Thanks a lot!

Natalia

from msfragger.

Glad to see that it works for you.

No, you don't need to put -Dbruker.lib.path or -cp for batmass-io-1.17.2.jar in your command line. But, again, you need to put the ext folder along with the jar file so that it can read the raw format.

Multi-experiments report is not a easy thing with shell script. You have to manually create those folders and move files to their corresponding folders. Then, run Philosopher and IMQuant accordingly. I suggest you borrow the commands from FragPipe (after clicking Run / dry run) and create a custom script based on it.

Best,

Fengchao

from msfragger.

Hello,

I am also having "Error: Could not find or load main class" with IonQuant. This is part of the command I used :
java -Xmx100G -Dbruker.lib.path="/projappl/project_2000752/Rigbe/TIMS-TOF/MSFragger-3.0/ext/bruker" -cp "/projappl/project_2000752/Rigbe/TIMS-TOF/MSFragger-3.0/batmass-io-1.17.4.jar;/projappl/project_2000752/Rigbe/TIMS-TOF/MSFragger-3.0/IonQuant-1.3.0.jar" ionquant.IonQuant

Is there an IonQuant jar with all the dependencies I could use too?

BR,
Rigbe

from msfragger.

If you want to run IonQuant standalone, you need to download the jar from https://github.com/Nesvilab/IonQuant/releases/latest.

Best,

Fengchao

from msfragger.

That worked, thanks.

Best,
Rigbe

from msfragger.