lanl / t-elf Goto Github PK

Tensor Extraction of Latent Features (T-ELF). Within T-ELF's arsenal are non-negative matrix and tensor factorization solutions, equipped with automatic model determination (also known as the estimation of latent factors - rank) for accurate data modeling. Our software suite encompasses cutting-edge data pre-processing and post-processing modules.

Home Page: https://lanl.github.io/T-ELF/

License: Other

Python 100.00%

blind-source-separation dimensionality-reduction feature-extraction gpu high-performance-computing hpc latent-variables machine-learning matrix matrix-factorization

t-elf's Issues

Improve X GPU transfer - reduce overhead

Add practical examples using T-ELF (such as topic modeling, recommender system, pattern extraction, etc.)

Add Bunny

SymNMFk Sparse support

Upload T-ELF to Zenodo for DOI number. Create Badge for DOI on README and update Citations on README and Documentation pages.

Add HNFMk

Bug in RESCALk (probably in NMFk and SymNMFk and TriNMFk as well), where perturbation parallel n_jobs assigned based on k parallel rules. For example, results in n_jobs=1 if decomposing for a single k, while we could have run many perturbations in parallel

Add WNMFk

reordered_con_mat does not fit in MPI gather when the matrix is too large and when using multi-node decomposition

Add TF-IDF + NMFk Example Script

Create a script that uses Vulture, Beaver, NMFk to clean a CSV file, create a TF-IDF matrix, decompose with NMFk

Add Peacock

Slow nnsvd fix

Add NTDS_FAPG

RESCAL logging and graph aggregation

Add CP-APR

Updating Vulture to Support Substitutions, Modularization

Add BNMFk

Add documentation (README.md) to examples

Add Orca

Add RNMFk

Add iPenguin

Add SPLIT NMFk

Error in plot_NMFk.plot_consensus_mat: ValueError: Argument must be an image or collection in this Axes matplotlib

Irregular error when plotting consensus matrix in TELF.factorization.utilities.plot_NMFk.plot_consesnsus_mat at plt.imshow(C).

Error: ValueError: Argument must be an image or collection in this Axes matplotlib

I believe might be caused by some issue in matplotlib. Added temporary hot-fix where plotting is skipped. If we re-run the code the missing plots are generated without error.

Replicate:

import os
os.environ["OMP_NUM_THREADS"] = "1" # export OMP_NUM_THREADS=1
os.environ["OPENBLAS_NUM_THREADS"] = "1" # export OPENBLAS_NUM_THREADS=1
os.environ["MKL_NUM_THREADS"] = "1" # export MKL_NUM_THREADS=1
os.environ["VECLIB_MAXIMUM_THREADS"] = "1" # export VECLIB_MAXIMUM_THREADS=1
os.environ["NUMEXPR_NUM_THREADS"] = "1" # export NUMEXPR_NUM_THREADS=1
from TELF.factorization import NMFk
import sys; sys.path.append("../../scripts/")
from generate_X import gen_data,gen_data_sparse

X = gen_data(R=4, shape=[100, 200])["X"]

params = {
    "n_perturbs":12,
    "n_iters":100,
    "epsilon":0.015,
    "n_jobs":-1,
    "init":"nnsvd", 
    "use_gpu":False,
    "save_path":"../../results/", 
    "save_output":True,
    "collect_output":True,
    "predict_k_method":"sill",
    "verbose":True,
    "nmf_verbose":False,
    "transpose":False,
    "sill_thresh":0.8,
    "pruned":True,
    'nmf_method':'nmf_fro_mu', # nmf_fro_mu, nmf_recommender
    "calculate_error":True,
    "predict_k":True,
    "use_consensus_stopping":0,
    "calculate_pac":True,
    "consensus_mat":True,
    "perturb_type":"uniform",
    "perturb_multiprocessing":False,
    "perturb_verbose":False,
    "simple_plot":True
}
Ks = range(1,9,1)
name = "Example_NMFk"
note = "This is an example run of NMFk"

model = NMFk(**params)
results = model.fit(X, Ks, name, note)

SymNMFk missing documentation (https://github.com/lanl/T-ELF/blob/main/TELF/factorization/SymNMFk.py). Need to add documentation and re-generate Sphinx docs.
Also, documentation website missing example usage for SymNMFk (https://github.com/lanl/T-ELF/blob/main/docs/source/SymNMFk.rst). See for example NMFk (https://github.com/lanl/T-ELF/blob/main/docs/source/NMFk.rst). Need to add example usage to documentation website for SymNMFk and regenerate Sphinx docs.

lanl / t-elf Goto Github PK

t-elf's Issues

Recommend Projects

Recommend Topics

Recommend Org