yoshida-lab / xenonpy Goto Github PK
View Code? Open in Web Editor NEWXenonPy is a Python Software for Materials Informatics
Home Page: http://xenonpy.readthedocs.io
License: BSD 3-Clause "New" or "Revised" License
XenonPy is a Python Software for Materials Informatics
Home Page: http://xenonpy.readthedocs.io
License: BSD 3-Clause "New" or "Revised" License
When a molecule is found not to be able to convert to Mol,
current proposal class directly skip the generated smiles string.
Instead, we should either fill in with None or return back to the old smiles string,
and throw out a warning for that.
I installed development mode with this site,
https://xenonpy.readthedocs.io/en/latest/installation.html
and tried to check the preset.elements with this page.
https://xenonpy.readthedocs.io/en/latest/tutorial/1-dataset.html
then I got the following error message.
>>> preset.elements
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/hagawa/codes/xenonpy/XenonPy/xenonpy/datatools/preset.py", line 230, in elements
self._check('elements')
File "/home/hagawa/codes/xenonpy/XenonPy/xenonpy/datatools/preset.py", line 192, in _check
"data {0} not exist, please run <Preset.sync('{0}')> to download from the repository".format(data))
RuntimeError: data elements not exist, please run <Preset.sync('elements')> to download from the repository
I cannot calculate descriptor without this preset.elements data.
How can I get it?
>>> from xenonpy.datatools import preset
doesn't make any error.
I'm using Ubuntu18.04 and Anaconda.
Hello. Thank you for the awesome package.
I have founded trivial things regarding the raise exception.
mdl = MDL()
models = mdl("_", save_to=False)
C:\Anaconda3\lib\json\decoder.py in raw_decode(self, s, idx)
355 obj, end = self.scan_once(s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
When 504 error, ret.json()
doesn't success because the return value is not json, since
JSONDecodeError
overwrite raise HTTPError
.
def query(self, query, variables):
payload = json.dumps({'query': query, 'variables': variables})
ret = requests.post(url=self._url, headers=self._headers, data=payload)
if ret.status_code != 200:
raise HTTPError('status_code: %s, %s' %
(ret.status_code, ret.json()))
ret = ret.json()['data']
return ret
ipdb> ret
<Response [504]>
ipdb> ret.content
b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body bgcolor="white">\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx/1.14.2</center>\r\n</body>\r\n</html>\r\n'
ipdb> ret.json()
*** json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I'm looking forward for your kind reply. Thank you very much.
When training for NGram, some SMILES may failed.
Instead of raising error directly, recording the SMILES that caused the error and continue on is important!
Implement rdkit fingerprint.
xenonpy.descriptor.Fingerprints
classI failed to install xenonpy on anaconda3 on linux and failed to use the docker xenonpy on linux. (It looks that the docker xenonpy doesn't work because of a proxy problem.)
I am afraid that xenonpy is now too integrated to install. How about separating them into layers, e.g. preset feature, feature calculators, and MDL, and so on, depending on users' purposes?
We need to add more querying functions to MDL class. At here, I want to collect some suggestions for what function should be be given priority.
Update the descriptor class to support sparse vector/matrix and the use of them during model training and prediction (with convertor?)
The modifier needs a try-catch to avoid stopping the iteration due to only one failure of molecule modification (caused by not able to find a substring in the N-gram table inside the get_prob function).
Instead of raising an error when an element is not found, you should have an option to fill in with NaN.
In XenonPy/xenonpy/inverse/iqspr/estimator.py, there is a bug.
XenonPy/xenonpy/inverse/iqspr/estimator.py
Lines 72 to 77 in fefc5cc
'desc' and 'y' are merged if length of them are different.
In previous sample/iQSPR.ipynb, there are bug relating this problem in cell In[11]: (Already fixed).
I guess you should check length of smiles list and property DataFrame.
Following is my opinion.
if len(smiles) != y.shape[0]:
raise RuntimeError('len(smiles) != y.shape[0]')
Thank you.
And I apologize for my poor English.
This refactor will import RNN training and training losses automatic saving.
Material Project and OQMD have no terms for redistribution.
The official conda channel of rdkit have not supported python 3.7 on OSX. Just remove it temporarily.
We need to add some helper functions to help users when building sample dataset.
sample
.Preset
class.Fingerprint featurizers will have a init parameter to control the input type.
Set descriptor default to None.
Estimator can input dictionary of tuple now, first must be BaseDescriptor and second is BaseEstimator
File extension will be changed.
before | after | description |
---|---|---|
pkl.pd_ |
pd(.*) |
pandas.DataFrame object file |
csv |
csv |
csv file |
xlsx |
xlsx or xls |
excel file |
pkl.z_ |
pkl(.*) |
common pickled files |
This will also remove Splitter.index property.
Merge OFM codes
Allow weighted combination of multiple Ngram, trained separately.
(Need to watch out for how to track the number of trained data in each case)
set_para will not function properly in BaseDescriptor because the values assigned will not pass to BaseFeaturizers inside.
E.g., when setting on_errors with .set_para in BaseDescriptor, the on_errors actually will not pass to the BaseFeaturizers defined inside.
A new def for set_para in BaseDescriptor is needed
We designed the BaseDescriptor
as a container of BaseFeaturizer
calculators. By using this, user can batch a lot of featurizers as a preset for pipelining. This proposal for add feature selection function to BaseDescriptor
class.
Proposal
Assume we have some class like this:
class BaseDescriptor:
def __init__(sefl, *, featurizers='all'):
....
class NewDescriptor(BaseDescriptor):
def __init__(sefl, *, featurizers):
super().__init__(featurizers=featurizers)
....
sefl.input = Featurizer1()
sefl.input = Featurizer2()
sefl.input = Featurizer3()
descriptor = NewDescriptor()
In this case, for the input has column named input, descriptor
will calculate all features that associate with self.input
then concatenate them. This is exactly what we did in current version.
In this proposal, user can initialize the NewDescriptor
with a parameter called featurizers
. This parameter contains the name of features. Only the featurizer which have name in the featurizers
will be activated.
In following example, only the specific features 'Featurizer1' and 'Featurizer3' will be calculated.
descriptor = NewDescriptor(featurizers=['Featurizer1', 'Featurizer3'])
I found that I can download the models for "stable inorganic compounds for materials project" but cannot download those for "QM9 Dataset" or "PolymerGenome Dataset" either.
When I tried mdl.pull(urls), it responded as follows:
FileNotFoundError: [Errno 2] No such file or directory: '~\S3\organic.nonpolymer.mu_debye\rcdk.fp.fingerprint\mxnet.nn.neural_network\shotgun_mu_Debye_randFP4975_corr-0.7528_mxnet_294-101-28-1_2018-06-13\model-175724d\shotgun_mu_Debye_randFP4975_corr-0.7528_mxnet_294-101-28-1_2018-06-13-045255-symbol.json'
Does anyone have any idea for this problem?
When property NaN rows and descriptor NaN rows are not matching (not subset of one of the other), there may be mismatch of rows and causing error in fitting?
Because of #20, users have to move their own data from ~/.xenonpy/dataset
to userdata
where specific in the ~/.xenonpy/conf.yml
.
We should add some migration functions to help that.
$ python -m xenonpy migrate
New versions of rdkit, pandas and pymatgen are no longer supported in python 3.5 officially. It's time for us to finish the python 3.5 support now.
Until now BaseDescriptor
class use a fixed name to get input data from pd.DataFrame
object like:
class YouDescriptor(BaseDescriptor):
def __init__(self, n_jobs=-1):
self.descriptor1 = Feature1(n_jobs)
Here descriptor1
must be the column name of input. To use this class is something like:
descriptor = YouDescriptor()
input = pd.Series(<'list of samples'>, name='descriptor1')
output = descriptor.fit_transform(input)
That's not flexible in the following 2 case:
These issue is a proposal to improve user experience.
inner_name: user_given_name
.In the last part of XenonPy/samples/mdl.ipynb, it is announced the R tutorials will be released.
I want to use the R model, since the python model is available only for inorganic crystals.
Please disclose how to calculated the fingerprint value input to the R model.
If there is some reference code, it is ok.
We specify the db version in conf.yml
file to sync user' dataset with us, but this is not a flexible way.
This is a proposal to refactor the data loader which allow to load the user determined version of dataset. This can be used like this:
from xenonpy.datatools import Preset
with Preset(ver='0.1.1') as preset:
data = preset.mp_structure
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.