FishTaco (Functional Shifts Taxonomic Contributors) is a metagenomic computational framework that aims to identify the driver taxa of microbiome functional shifts
License: Other
The official FishTaco source code repository. For details on FishTaco, see http://borenstein-lab.github.io/fishtaco/.
FishTaco is a metagenomic computational framework, aiming to identify the taxa that are driving the functional shifts we observe in microbiomes of different individuals or disease states.
For FishTaco announcements and questions, including notification of new releases, you can visit the FiShTaCo users forum.
Hello,
I am unable to get the test_fishtaco.py file to work... I installed version 1.1.6 and followed the install instructions at https://borenstein-lab.github.io/fishtaco/installation.html . I get about 10 errors when I run the test file. Is there anybody that would be able to assist me? I am not a computer programmer so it's been a huge learning curve trying to learn to code and run this program for my honors thesis. I really appreciate any help that can be provided.
I've pasted the error below:
Traceback (most recent call last):
File "test_fishtaco.py", line 570, in test_is_output_correct_for_fishtaco_FDR_correction
with open("fishtaco_out_FDR_correction_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_FDR_correction_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 342, in test_is_output_correct_for_fishtaco_de_novo_inference
with open("fishtaco_out_de_novo_inf_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_de_novo_inf_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 456, in test_is_output_correct_for_fishtaco_filtering_by_function_list
with open("fishtaco_out_filtering_by_function_list_STAT_run_log_SCORE"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_filtering_by_function_list_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 227, in test_is_output_correct_for_fishtaco_no_inference
with open("fishtaco_out_no_inf_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_no_inf_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 398, in test_is_output_correct_for_fishtaco_predict_functional_profile
with open("fishtaco_out_predict_function_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_predict_function_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 285, in test_is_output_correct_for_fishtaco_prior_based_inference
with open("fishtaco_out_prior_based_inf_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_prior_based_inf_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 523, in test_is_output_correct_for_fishtaco_shapley_value
test_output = pd.read_csv('fishtaco_out_shapley_'
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 575, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 933, in init
self._engine = self._make_engine(f, self.engine)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine
self.handles = get_handle( # type: ignore[call-overload]
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/common.py", line 789, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_shapley_STAT_taxa_contributions_SCORE_wilcoxon_ASSESSMENT_multi_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 630, in test_is_output_correct_for_fishtaco_stricter_FDR_filter
with open("fishtaco_out_stricter_FDR_filter_STAT_run_log_SCORE_wilcoxon"
FileNotFoundError: [Errno 2] No such file or directory: 'fishtaco_out_stricter_FDR_filter_STAT_run_log_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 182, in test_learn_non_neg_elastic_net_de_novo
test_output = pd.read_csv('test_de_novo_based_inf_STAT_taxa_learned_'
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 575, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 933, in init
self._engine = self._make_engine(f, self.engine)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine
self.handles = get_handle( # type: ignore[call-overload]
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/common.py", line 789, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'test_de_novo_based_inf_STAT_taxa_learned_copy_num_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Traceback (most recent call last):
File "test_fishtaco.py", line 137, in test_learn_non_neg_elastic_net_with_prior
test_output = pd.read_csv('test_prior_based_inf_STAT_taxa_learned_'
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 575, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 933, in init
self._engine = self._make_engine(f, self.engine)
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine
self.handles = get_handle( # type: ignore[call-overload]
File "/home/aadeg/.local/lib/python3.8/site-packages/pandas/io/common.py", line 789, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'test_prior_based_inf_STAT_taxa_learned_copy_num_SCORE_wilcoxon_ASSESSMENT_single_taxa.tab'
Ran 12 tests in 6.815s
FAILED (errors=10)
`
Hi,
I am trying to install fishtaco and have run into a few problems.
Conda fishtaco package did not work. I found out the fishtaco code in the conda package is outdated and does not have the changes made on the github page for updates to sklearn.model_selection.
I installed it from github but when I run test_fishtaco, it fails for elastic net with this error message:
Traceback (most recent call last):
File "/home/hena.ramay/miniconda3/envs/fishtaco/bin/run_fishtaco.py", line 186, in <module>
main(vars(given_args))
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/compute_contribution_to_DA.py", line 734, in main
enet, validation_rsqr = learn_non_neg_elastic_net_with_prior.learn(cov_train, res_train, params)
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/learn_non_neg_elastic_net_with_prior.py", line 121, in learn
_ = enet_path(cov_inner_train, response_inner_train,
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py", line 509, in enet_path
raise ValueError("Unexpected parameters in params", params.keys())
ValueError: ('Unexpected parameters in params', dict_keys(['fit_intercept', 'normalize', 'return_models']))
Following is the output form test_fishtaco.py. I have printed out the values for variables being sent to enet_path. Can you please help me in figuring out what I need to do here?
==============================================================
Testing compute_differential_abundance.py
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters: {'input_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'class_header': True, 'row_metadata': None, 'output_file': 'test_compute_differential_abundance.tab', 'method': 'Wilcoxon', 'control_label': '0', 'case_label': '1', 'verbose': True, 'alpha': 0.05}
Loading files... Done.
Number of samples: 213
Number of controls: 107
Number of cases: 106
Number of functions: 10
Computing differential abundance... Done.
Writing output... Done.
.==============================================================
Testing compute_pathway_abundance.py
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters: {'ko_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/KO_vs_SAMPLE_MUSiCC.tab', 'ko_to_pathway_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/data/KOvsPATHWAY_BACTERIAL_KEGG_2013_07_15.tab', 'output_file': 'test_compute_pathway_abundance.tab', 'output_counts_file': 'test_compute_pathway_abundance_counts.tab', 'mapping_method': 'naive', 'compute_method': 'sum', 'transpose_ko_abundance': False, 'transpose_output': False, 'verbose': True}
Reading files...
Done.
Writing output...
Done.
.==============================================================
Testing compute_contribution_to_DA.py (FDR correction)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001.tab', 'apply_inference': False, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_FDR_correction', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'FDR', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Calculating agreement between metagenome-based and taxa-based functional profiles...
Calculating differential abundance values for each function...
Creating 5 permutations...
Done.
0:K00001 took 0.056569814682006836 seconds to run.
Writing output...
Done.
Testing output...
Deleting temporary files...
.==============================================================
Testing compute_contribution_to_DA.py (de novo inference)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': None, 'apply_inference': True, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_de_novo_inf', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
No input of genomic content given to FishTaco, inferring the mapping of taxa to functions from taxonomic and functional profiles
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Inferring the genomic content of each taxa...
0 : K00001
************************************************
cov_inner_train:
[[0. 0. 0.02 ... 0. 0. 0. ]
[0.03 0.05 0.41 ... 0.03 0.04 0.06]
[0. 0.05 0.08 ... 0.02 0.01 0.01]
...
[0. 0. 0.04 ... 0. 0. 0.01]
[0.03 0.02 0.45 ... 0.01 0.01 0.1 ]
[0. 0. 0.02 ... 0. 0. 0. ]]
response_inner_train:
[1.26 0.73 0.65 1.06 0.35 0.25 1.01 0.27 1. 0.54 0.27 0.44 0.56 0.87 0.51 0.79 0.43 0.73 0.32 1.25 0.52 0.36 1.01 0.47 0.89 1.19 0.92 0.69 0.26 1.13 0.78 0.89 0.3 0.74 0.81 0.46 0.27 1.17 1.13
0.51 0.72 0.85 0.38 0.43 0.88 0.72 0.88 0.7 0.42 1.19 0.87 0.28 0.84 0.64 0.71 0.32 0.26 0.76 1.26 0.3 0.24 0.5 0.58 0.56 0.62 0.33 0.6 1.15 0.33 0.43 0.87 0.84 0.73 0.39 0.56 0.37 0.25 0.78
1.24 0.4 0.8 0.56 0.78 1.15 0.33 0.48 0.26 0.34 0.86 0.63 0.51 1.34 0.42 0.43 0.51 0.86 0.51 0.57 0.57 0.61 0.23 0.72 0.45 0.26 0.51 0.69 0.24 0.77 0.47 0.76 0.37 0.4 0.3 0.52 0.41 0.77 0.6
0.75 0.51 0.73 0.37 0.27 0.35 1.02 0.48 0.4 0.52 0.55 0.51 0.37 0.82 0.71 1.07 0.69 1.13]
l1_ratio
0.5
************************************************
Traceback (most recent call last):
File "/home/hena.ramay/miniconda3/envs/fishtaco/bin/run_fishtaco.py", line 186, in <module>
main(vars(given_args))
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/compute_contribution_to_DA.py", line 734, in main
enet, validation_rsqr = learn_non_neg_elastic_net_with_prior.learn(cov_train, res_train, params)
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/learn_non_neg_elastic_net_with_prior.py", line 121, in learn
_ = enet_path(cov_inner_train, response_inner_train,
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py", line 509, in enet_path
raise ValueError("Unexpected parameters in params", params.keys())
ValueError: ('Unexpected parameters in params', dict_keys(['fit_intercept', 'normalize', 'return_models']))
Testing output...
F==============================================================
Testing compute_contribution_to_DA.py (filter by list)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001_K00054.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001_K00054.tab', 'apply_inference': False, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_filtering_by_function_list', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': None, 'multi_function_filter_list': 'K00001,K00007,K00020', 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for the following functions: K00001,K00007,K00020
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:3 #Taxa:10 #Samples:213
Calculating agreement between metagenome-based and taxa-based functional profiles...
Calculating differential abundance values for each function...
Creating 5 permutations...
Done.
0:K00001 took 0.05357718467712402 seconds to run.
1:K00007 took 0.05191183090209961 seconds to run.
2:K00020 took 0.05183863639831543 seconds to run.
Writing output...
Done.
Testing output...
Deleting temporary files...
.==============================================================
Testing compute_contribution_to_DA.py (no inference)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001.tab', 'apply_inference': False, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_no_inf', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Calculating agreement between metagenome-based and taxa-based functional profiles...
Calculating differential abundance values for each function...
Creating 5 permutations...
Done.
0:K00001 took 0.055394649505615234 seconds to run.
Writing output...
Done.
Testing output...
Deleting temporary files...
.==============================================================
Testing compute_contribution_to_DA.py (predict function)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': None, 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001.tab', 'apply_inference': False, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_predict_function', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': False, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
No input of functional abundance given to FishTaco, predicting from taxonomic abundance and genomic content...
Reading files...
Done.
Writing output...
Done.
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Calculating agreement between metagenome-based and taxa-based functional profiles...
Calculating differential abundance values for each function...
Creating 5 permutations...
Done.
0:K00001 took 0.051674604415893555 seconds to run.
Writing output...
Done.
Testing output...
Deleting temporary files...
.==============================================================
Testing compute_contribution_to_DA.py (prior-based inference)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001.tab', 'apply_inference': True, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_prior_based_inf', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'single_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '5', 'number_of_shapley_orderings_per_taxa': '100', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Inferring the genomic content of each taxa...
0 : K00001
************************************************
cov_inner_train:
[[2.85e-04 3.32e-03 5.29e-03 ... 1.41e-03 6.04e-04 5.15e-04]
[2.64e-04 7.47e-04 5.05e-02 ... 4.71e-04 1.58e-03 4.56e-03]
[1.01e-04 3.42e-04 7.23e-03 ... 4.70e-04 4.93e-04 8.96e-04]
...
[2.83e-04 9.90e-05 1.08e-02 ... 9.90e-05 9.90e-05 9.90e-05]
[1.75e-03 1.16e-03 2.97e-02 ... 3.36e-04 9.57e-04 6.65e-03]
[1.84e-04 1.22e-04 1.05e-03 ... 9.43e-05 1.04e-04 1.60e-04]]
response_inner_train:
[0.65 0.44 1.06 0.35 1.07 0.25 1. 0.79 0.27 0.44 0.56 0.51 0.79 0.43 1.69 1.25 0.49 0.52 1.01 0.47 0.89 0.69 0.47 1.13 0.89 1.38 0.27 0.78 1.13 0.52 0.51 0.43 0.72 0.48 0.54 0.88 0.88 0.7 0.42
1.19 0.87 0.34 0.84 1.04 0.64 0.71 0.32 0.69 0.43 0.75 0.67 0.56 0.62 0.6 1.15 0.33 0.59 0.87 0.61 0.84 0.5 0.5 0.73 0.39 1.27 0.37 1.07 1.1 0.46 0.33 0.78 0.62 0.44 1.24 0.4 0.8 0.78 0.33
0.68 0.48 1.16 0.28 0.61 0.96 0.26 0.39 0.74 0.86 0.63 0.51 0.74 0.42 0.43 0.86 0.57 0.57 0.61 0.91 0.23 0.71 1.28 0.45 0.26 0.33 0.51 0.69 0.24 0.77 0.47 0.76 0.37 0.3 0.52 0.71 0.41 0.7 0.77
0.68 0.58 0.36 0.73 0.72 0.31 0.37 0.27 0.88 0.4 0.76 1.07 0.37 0.82 0.28 0.71 0.69 1.13]
l1_ratio
0.5
************************************************
Traceback (most recent call last):
File "/home/hena.ramay/miniconda3/envs/fishtaco/bin/run_fishtaco.py", line 186, in <module>
main(vars(given_args))
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/compute_contribution_to_DA.py", line 734, in main
enet, validation_rsqr = learn_non_neg_elastic_net_with_prior.learn(cov_train, res_train, params)
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/learn_non_neg_elastic_net_with_prior.py", line 121, in learn
_ = enet_path(cov_inner_train, response_inner_train,
File "/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py", line 509, in enet_path
raise ValueError("Unexpected parameters in params", params.keys())
ValueError: ('Unexpected parameters in params', dict_keys(['fit_intercept', 'normalize', 'return_models']))
Testing output...
F==============================================================
Testing compute_contribution_to_DA.py (Shapley value)
Path to examples:/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco
==============================================================
Given parameters:
{'taxa_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_SAMPLE_for_K00001.tab', 'function_abun_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/WGS_KO_vs_SAMPLE_MUSiCC_only_K00001.tab', 'class_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/SAMPLE_vs_CLASS.tab', 'taxa_to_function_file': '/home/hena.ramay/miniconda3/envs/fishtaco/lib/python3.9/site-packages/fishtaco/examples/METAPHLAN_taxa_vs_KO_only_K00001.tab', 'apply_inference': False, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out_shapley', 'map_function_level': 'none', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'Bonf', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'multi_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': '0', 'number_of_permutations': '100', 'number_of_shapley_orderings_per_taxa': '10', 'da_result_file': None, 'single_function_filter': 'K00001', 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': True, 'write_log': True, 'num_cv': 5, 'alpha': 0.05, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
Filtering for a single function: K00001
Done.
#controls = 107, #cases = 106
Computing a differential abundance score for each taxa...
Done.
Computing a differential abundance score for each function...
Done.
Selecting only functions that are enriched in samples with the label: 1...
Done.
#DA functions:1 #Taxa:10 #Samples:213
Calculating agreement between metagenome-based and taxa-based functional profiles...
Calculating differential abundance values for each function...
Creating 100 permutations...
Done.
Computing permuted shapley orderings scores for 100 orderings...
compute_contribution_to_DA.py::355 reads:
pd.read_csv(function_mapping_file, dtype={0: str}, sep="]t")
which causes the file to be read incorrectly and therefore FishTaco fails to aggregate KOs to pathways or modules.
The sep argument should be \t.
Dear all
I have been trying to compare the use of MetaPhlan2 and other classification tools (e.g. kraken2 and others). Will feeding a taxonomic and functional output fils generated by kraken2 and HUMAnN2 be problematic for FishTaco?
Thanks
Marc
Hi author,
If i run fishtaco PICRUSt-derived metagenomic functional profile, I found the PICRUSt pre-calculated files for genomic content file is too large, and now i use the picrust2 that output file will some different .How can I run FishTaco with QIIME2 and PICRUSt2 output files?
Thanks for your answer
Hello,
I am using your tool, thanks for your sharing. The programme proceeded with error when the Function abundance file had several (e.g., 1, 2, 3, 5) records (error: .Unexpected error while running MUSiCC on functional data (make sure you have KEGG orthology groups (KOs) as rows):
(<type 'exceptions.ValueError'>, ValueError('Cannot have number of folds n_folds=5 greater than the number of samples: 1.',), <traceback object at 0x7f70f5b832d8>)
), but it worked well more than 200 records. I wanna know why and is there any relationship between different records?
Good afternoon,
I tried installing fishtaco today with the following pip install -U fishtaco command, but for some reason when I try to run the test, I get the following error:
pkg_resources.ResolutionError: No scrip named 'test_fishtaco.py' even though I have the script in my file path.
Any suggestions to fix this would be very helpful.
Thank you,
Susheel
Hi,
Thank you for an excellent approach to functional metagenomics. I have a major concern with the normalization method based on relative abundance. It has been demonstrated that is one of the worst approaches. Can you suggest other normalization methods compatible with MUSSic and FishTaco? For differential abundance analysis I usually use Log Center Ratio (CoDa, Gloor, 2017) or Variation stabilization (DESeq2), but neither of them give values in the range of 0 and 1.
Thank you,
Dear all,
I was installing FishTaco but got a problem with the cross_validation function
attached the output of test function of fishtaco
Error Fishtac.txt
Is that a problem on python 3.6 or on Scikit-learn?
Thanks in advance.
Marcio
Hi author,
When I run the script of run_fishtaco.py: run_fishtaco.py -ta gg13.5_otu_table_norm.txt -fu metagenome_predictions_MUSiCC_Normalized.txt -l mapping.txt -gc ko_13_5_precalculated.tab -op fishtaco_out_no_inf -map_function_level none -functional_profile_already_corrected_with_musicc -assessment single_taxa -log, I encountered the following error message：
Traceback (most recent call last):
File "/home/apps/miniconda3/envs/qiime1/bin/run_fishtaco.py", line 100, in
main(vars(given_args))
File "/home/apps/miniconda3/envs/qiime1/lib/python2.7/site-packages/fishtaco/compute_contribution_to_DA.py", line 174, in main
if np.sum(np.isnan(taxa_to_function_data.values)) > 0:
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''.
I checked the format of all files, and I found that all txt flie are separated by tab. The profile of ko_13_5_precalculated.tab was downloaded through the website (http://picrust.github.io/picrust/picrust_precalculated_files.html#id1). I modified the GreenGenes IDs in both the taxonomic abundance (gg13.5_otu_table_norm.txt) and genomic content tables (ko_13_5_precalculated.tab) to prefix them with a non-numeric character (e.g., changing “228054” to “t228054”).
The following is a screenshot of the relevant information for each input file：
Thanks for your answer
I have just got FishTaco running on my linux system and it is working correctly with your example data. However I am running into an error with my data. The output:
Given parameters:
{'taxa_abun_file': 'fishtaco_taxabund_12_nozero.csv', 'function_abun_file': 'fishtaco_funabund.tsv', 'class_file': 'fishtaco_samples_12.tsv', 'taxa_to_function_file': None, 'apply_inference': True, 'case_label': '1', 'control_label': '0', 'output_pref': 'fishtaco_out', 'map_function_level': 'pathway', 'map_function_file': None, 'perform_inference_on_ko_level': False, 'multiple_hypothesis_correction': 'FDR-0.05', 'max_da_functions_cases_controls': None, 'taxa_assessment_method': 'multi_taxa', 'score_to_compute': 'wilcoxon', 'max_score_cutoff': '100', 'na_rep': 'NA', 'number_of_permutations': '100', 'number_of_shapley_orderings_per_taxa': '5', 'da_result_file': None, 'single_function_filter': None, 'multi_function_filter_list': None, 'functional_profile_already_corrected_with_musicc': False, 'write_log': False, 'residual_mode': 'remove_residual', 'normalization_mode': 'scale_permuted', 'permutation_mode': 'blocks'}
Reading input files...
Running MUSiCC...
Loading data using pandas module...
Done.
Performing MUSiCC Correction...
Learning sample-specific models
..........................................................................................................................................................................................................................................................................Done.
Performing MUSiCC Normalization...
Done.
Done.
Running time was 20 seconds.
No input of genomic content given to FishTaco, inferring the mapping of taxa to functions from taxonomic and functional profiles
Mapping functions to pathway/module level...
Reading files...
Done.
Writing output...
Done.
Done.
Done.
Reducing taxa, function, and class data to contain the exact same set of samples...
Done.
#controls = 66, #cases = 70
Computing a differential abundance score for each taxa...
Traceback (most recent call last):
File "run_fishtaco.py", line 177, in
main(vars(given_args))
File "/home/amy/microbiome/picrust2_standalone/fishtaco/envFT/lib/python3.6/site-packages/fishtaco/compute_contribution_to_DA.py", line 468, in main
'alpha': args['alpha']}
KeyError: 'alpha'
I have checked my input file formats and they are identical to examples. I have tried to remove any taxa lines that have zeros in all samples (I had some as a subset of a larger dataset).
A little of my tab-delimited taxa abundance file:
Taxa | AE010719-479 | AE010719-480 | AE010719-481
43fddf1528d4a98928fd8c3a8ac23bfd | 0.411748379294621 | 0.07942961346447 | 0.5158757202156
0eb88b722902316b11770922bcdaca7b | 0.237564679526576 | 0.043107829362351 | 0.211278731181463
efbe1f58b1e2984ddc53a64f047d94ff | 0.124541543585575 | 0.015993542793937 | 0.077922681370423
496ecde24f9ab698992413d3d4f04b5f | 0.037925497115442 | 0.025350512690204 | 0.022458335914751
637b9b3f4d1cbb1a10c07817619cdf69 | 0.042802482107809 | 0.002570924636035 | 0.040874171364847
d46e2205f0c6ecf67b51f83d111c509c | 0.001645486806368 | 0.022121909658904 | 0.001254569109721
99e433a3ce4d5290445f668df2c9147e | 0.031403025316707 | 0.007294251292936 | 0.023108853230903
487de539f50da640cd8914cea7821561 | 0.020519022223985 | 0.002122507548354 | 0.017424570968342
A little of my tab-delimited function abundance file:
Function | AE010719-479 | AE010719-480 | AE010719-481
K00001 | 5.51279E-05 | 0.0002265119 | 4.20007E-05
K00002 | 0 | 8.654E-07 | 0
K00003 | 0.0003856851 | 0.0005163871 | 0.0003739182
K00004 | 3.7417E-06 | 2.20775E-05 | 4.9638E-06
K00005 | 2.286E-07 | 0.000106112 | 1.758E-07
A little of my tab-delimited sample information file:
Sample | Site
AE010719-479 | 1
AE010719-480 | 1
AE010719-481 | 1
AE010719-482 | 1
AE010719-483 | 1
AE010719-484 | 0
AE010719-485 | 0
AE010719-486 | 0
Could you please advise what this error relates to? The only thing I can think of is that there are no significant abundances of taxa. I have tried with no FDR correction and get the same. I would be surprised if this is the case as other programs using same data find many sig differences.
Many thanks in advance for you help!
Hi,
When trying to change the num_cv using "--num_cv". I get this error:
File "/home/qiime2/miniconda/envs/qiime2-2020.8/lib/python3.6/site-packages/fishtaco/compute_contribution_to_DA.py", line 653, in main
if number_of_cases < num_cv:
TypeError: '<' not supported between instances of 'numpy.ndarray' and 'str'
From line 618 in compute_contribution_to_DA.py
I think this is because python is reading the argument from --num_cv as string, not int.
I would be grateful if this issue can be resolved. Thanks!
With version 1.1.0 the license has been changed from BSD 3-clause to the UW license.
The availability of those versions on GitHub as well as PyPI somewhat contradict (or at the very least make questionable) the following term of the license:
1. [...] The FishTaco software remains at your University and is not published, distributed, or otherwise transferred or made available to other than Academic Users.
Also note that by using GitHub you agreed to their Terms which state (among other things):
- License Grant to Other Users
Any User-Generated Content you post publicly, including issues, comments, and contributions to other Users' repositories, may be viewed by others. By setting your repositories to be viewed publicly, you agree to allow others to view and "fork" your repositories (this means that others may make their own copies of Content from your repositories in repositories they control).
If you set your pages and repositories to be viewed publicly, you grant each User of GitHub a nonexclusive, worldwide license to use, display, and perform Your Content through the GitHub Service and to reproduce Your Content solely on GitHub as permitted through GitHub's functionality (for example, through forking). You may grant further rights if you adopt a license. If you are uploading Content you did not create or own, you are responsible for ensuring that the Content you upload is licensed under terms that grant these permissions to other GitHub Users.
Your license does not agree with that, I suppose (IANAL).
Furthermore other parts of the license (e.g., 2. You may not distribute FishTaco or any modification to FishTaco to any third party.) prevent to make FishTaco 1.1.0+ available on other platforms, e.g., http://bioconda.github.io/ / https://anaconda.org/bioconda, which could make the software easily obtainable by academic users.
The "self-destructing" part of the license
8. This Software License Agreement and all rights granted under it terminate on December 31st, 2020. Upon termination, you agree to remove so as to make unrecoverable the original FishTaco software, all copies and all modifications thereof.
also prevents generating reproducible research results in academics in the future, i.e., year 2021 and later.
Would you consider a license change which could address all or some of the above?
Hi,
I have been trying to visualize the results from running fishtaco but while running the MultiFunctionTaxaContributionPlots function I get the following error:
Error in if (taxa_da[taxa_da$Taxa == curr_taxa, "StatValue"] > 0) { :
missing value where TRUE/FALSE needed
I would be grateful for any suggestions on how to fix this error.
Thank you!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.