p-koo / libre Goto Github PK

The function accepts an activation parameter but does not use it.

https://github.com/p-koo/cipher/blob/f90c43bc00f9c47e2f5e0cd4478ff519421cd182/cipher/model_zoo/residualbind.py#L7

the fasta file format apparently supports ; as comments. we can implement this by ignoring lines that begin with ;.

https://github.com/p-koo/cipher/blob/0d8f52bc77686367628d6dd50722ddf2d1edd400/cipher/preprocess/wrangle.py#L248-L289

d885338 moves the one-hot functions. the tests need to be moved too. tests do not pass at this time.

https://github.com/p-koo/cipher/blob/e1ad5bb4ca86a7c45f54bb4fdb6f05d6c4a1b1a9/cipher/preprocess/tests/test_wrangle.py#L164

The parse_fasta function assumes that each sequence takes up one line only, but that is not always the case. Sequences can span multiple lines. I have an implementation here that handles multiple lines correctly.

https://github.com/p-koo/cipher/blob/bbfaaf454eff4c700a0d6eaec50997a072c2e2ce/cipher/preprocess/wrangle.py#L178-L211

the function convert_one_hot can return wrong results if any letter in the sequences is not within the alphabet. a check should be added to make sure all the values in the sequences are in the alphabet.

the check could be this:

unknown = set(np.unique(s)).difference(alphabet)
if unknown:
    raise ValueError("letter found that is not in alphabet")

https://github.com/p-koo/cipher/blob/e1ad5bb4ca86a7c45f54bb4fdb6f05d6c4a1b1a9/cipher/utils.py#L96-L142

I have an implementation to one-hot encode sequences that is 3x faster than the current implementation.

https://github.com/kaczmarj/rotation-koo-lab/blob/310bdf589c59f4ef9ef8511a479e69925c4bbcf3/chip-seq-to-hdf5-dataset/chipseq_utils.py#L317-L363

This is the current implementation:

https://github.com/p-koo/cipher/blob/48b1dccdd42576b2bf556edbc7243a838d9deaf0/cipher/preprocess/singletask.py#L160-L226

My implementation does not strip or pad the sequences, but i can add that.

filter_encode_metatable has some hard-coded queries, which might be better suited as variables. for example, the function keeps rows with File assembly GRCh38, but sometimes I can imagine something else might be desired.

perhaps we can re-work this function to allow the user to enter a query? pandas.DataFrame.query() might be useful here.

https://github.com/p-koo/cipher/blob/03e184824cc64d38bb626e80ecc326f1f45653c7/cipher/preprocess/wrangle.py#L59-L60

jax has an example of a one-hot implementation: https://jax.readthedocs.io/en/latest/notebooks/neural_network_with_tfds_data.html#utility-and-loss-functions

def one_hot(x, k, dtype=jnp.float32):
  """Create a one-hot encoding of x of size k."""
  return jnp.array(x[:, None] == jnp.arange(k), dtype)

we could try a similar implementation. might be faster than the current version

several functions in wrangle.py are also defined in singletask.py. we should choose one implementation and remove the other one.

The function enforce_constant_size accepts a path to a bed file and writes a bed file. This feels more like command-line behavior, where input and output are files. I would suggest that this function should take in a pandas dataframe representation of a bed file, and it should return a modified dataframe. I suggest that the processing script that uses this function should take care of loading and saving files, if it is necessary.

I have an implementation of this here: https://github.com/kaczmarj/rotation-koo-lab/blob/310bdf589c59f4ef9ef8511a479e69925c4bbcf3/chip-seq-to-hdf5-dataset/chipseq_utils.py#L102-L132

https://github.com/p-koo/cipher/blob/bbfaaf454eff4c700a0d6eaec50997a072c2e2ce/cipher/preprocess/wrangle.py#L7

pandas.read_csv infers the compression used (if any) by default. See the documentation at https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html. So we can remove the compression argument.

the .to_csv() method also infers compression. if you pass a filename with a .gz extension, the file will be gzip-compressed.

https://github.com/p-koo/cipher/blob/bbfaaf454eff4c700a0d6eaec50997a072c2e2ce/cipher/preprocess/wrangle.py#L34-L37

The function make_directory can be replaced with

pathlib.Path("path/to/directory").mkdir(exist_ok=True)

See https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir. Pathlib has lots of useful operations for paths.

https://github.com/p-koo/cipher/blob/bbfaaf454eff4c700a0d6eaec50997a072c2e2ce/cipher/utils.py#L11-L31