The hbutils from hansbug

Function && classes going to be included.

Reflection - class functions
- wraps for class (no final name yet, will be redesigned, deprecated version: https://github.com/opendilab/treevalue/blob/main/treevalue/utils/clazz.py#L9)
- 【deprecate】common parent class (See https://github.com/opendilab/treevalue/blob/main/treevalue/utils/clazz.py#L143)
- common direct parent class (See https://github.com/opendilab/treevalue/blob/main/treevalue/utils/clazz.py#L160)
- class decorator for quickly class build (such as autorepr, autostate, autojson, etc)
Random - random functions
- better shuffle function
- better choice function (including multiple choice, non-duplicated multiple choice)
- quick random string (such as random_digits, random_alphabets)
- quick random patterned string (such as random_base64, random_md5)
- timestamp-based random token
Concurrency - concurrent utilities
- double event (can open or close, see https://github.com/opendilab/DI-engine/blob/main/ding/interaction/base/threading.py#L5)
- Concurreny executor with topological order
Color - color system
- color system classes (see: https://github.com/opendilab/treevalue/blob/main/treevalue/utils/color.py)
- color generating functions (see: https://github.com/opendilab/treevalue/blob/main/treevalue/tree/tree/graph.py#L176)
String - string utilities
- truncate (processing too-long string)
- pluralize (see: https://pypi.org/project/inflect/)
- templated_string (often used in cli or configuration system)

Bug with pip_install function on Colab

Only happens on colab

from hbutils.system import pip_install
pip_install(['onnxruntime-gpu'], silent=False)

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
[<ipython-input-9-7d731497eb49>](https://localhost:8080/#) in <module>
      1 from hbutils.system import pip_install
      2 
----> 3 pip_install(['onnxruntime-gpu'], silent=False)

4 frames
[/usr/local/lib/python3.8/dist-packages/hbutils/system/python/package.py](https://localhost:8080/#) in pip_install(reqs, silent, force, user)
    173     """
    174     if force or not check_reqs(reqs):
--> 175         pip('install', *(('--user',) if user else ()), *reqs, silent=silent)
    176 
    177 

[/usr/local/lib/python3.8/dist-packages/hbutils/system/python/package.py](https://localhost:8080/#) in pip(silent, *args)
     89     with open(os.devnull, 'w') as sout:
     90         try:
---> 91             process = subprocess.run(
     92                 [sys.executable, '-m', 'pip', *args],
     93                 stdin=sys.stdin if not silent else None,

[/usr/lib/python3.8/subprocess.py](https://localhost:8080/#) in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    491         kwargs['stderr'] = PIPE
    492 
--> 493     with Popen(*popenargs, **kwargs) as process:
    494         try:
    495             stdout, stderr = process.communicate(input, timeout=timeout)

[/usr/lib/python3.8/subprocess.py](https://localhost:8080/#) in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)
    806         (p2cread, p2cwrite,
    807          c2pread, c2pwrite,
--> 808          errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    809 
    810         # We wrap OS handles *before* launching the child, otherwise a

[/usr/lib/python3.8/subprocess.py](https://localhost:8080/#) in _get_handles(self, stdin, stdout, stderr)
   1487             else:
   1488                 # Assuming file-like object
-> 1489                 c2pwrite = stdout.fileno()
   1490 
   1491             if stderr is None:

UnsupportedOperation: fileno

And the stdout is this thing

import sys
print(sys.stdout)
print(sys.__stdout__)

<ipykernel.iostream.OutStream object at 0x7faffd26fee0>
<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>

Problem with auto importing.

opendilab/treevalue#30

Maybe need a warning or exception when importing is failed?

Check if there are some missing of weakref

See:

We should weak_ref some of the objects when we create method in closure, or the related instance will be never destroyed.

AETG Error when no pairs are provided.

Traceback (most recent call last):
  File "/home/hansbug/oo-projects/data_app/dataapp/gui/gene/result.py", line 9, in <lambda>
    self.clicked.connect(lambda: aetg.aetg_result(result))
  File "/home/hansbug/oo-projects/data_app/dataapp/gui/gene/aetg.py", line 117, in aetg_result
    for i, pair in enumerate(g.tuple_cases()):
  File "/home/hansbug/.pyenv/versions/3.6.3/lib/python3.6/site-packages/hbutils/testing/generator/base.py", line 62, in tuple_cases
    for d in self.cases():
  File "/home/hansbug/.pyenv/versions/3.6.3/lib/python3.6/site-packages/hbutils/testing/generator/aetg.py", line 146, in cases
    m = len(self.__pairs[-1])
IndexError: list index out of range

Add creation of symlink and link

symlink function (if the parent directory of dst not exist, just create it)
link function (if the parent directory of dst not exist, just create it)
add link/symlink option into isolated_directory method

Add color distance and generating module.

Color distance based on RGB color space
Color auto generate module, like the treevalue.

中文版说明文档

从知乎来的，感觉每个功能都很神奇，跪求中文版说明文档😝

python_version()'s unittest will fail when setuptools<50

When setuptools==49.0.0

test/system/python/test_implementation.py::TestSystemPythonImplementation::test_is_cpython PASSED
test/system/python/test_package.py::TestSystemPythonPackage::test_package_version PASSED
test/system/python/test_version.py::TestSystemPythonVersion::test_python_version FAILED

================================================================================================ FAILURES =================================================================================================
_______________________________________________________________________________ TestSystemPythonVersion.test_python_version _______________________________________________________________________________

self = <test.system.python.test_version.TestSystemPythonVersion object at 0x7fcb3ec47f50>

    def test_python_version(self):
        _actual_version = sys.version_info
        _get_version = python_version()
        assert isinstance(_get_version, _Version)
>       assert _get_version.major == _actual_version.major
E       AttributeError: 'Version' object has no attribute 'major'

test/system/python/test_version.py:17: AttributeError

----------- coverage: platform linux, python 3.7.6-final-0 -----------
Name                                      Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------
hbutils/system/python/__init__.py             3      0   100%
hbutils/system/python/implementation.py      23      0   100%
hbutils/system/python/package.py             10      0   100%
hbutils/system/python/version.py              6      0   100%
-----------------------------------------------------------------------
TOTAL                                        42      0   100%
Coverage XML written to file coverage.xml

========================================================================================= short test summary info =========================================================================================
FAILED test/system/python/test_version.py::TestSystemPythonVersion::test_python_version - AttributeError: 'Version' object has no attribute 'major'

When setuptools is 50 or higher version, this problem disappeared.

Release v0.0.1

Reflection - class functions
- wraps for class (no final name yet, will be redesigned, deprecated version: https://github.com/opendilab/treevalue/blob/main/treevalue/utils/clazz.py#L9)
Random - random functions
- better shuffle function
- better choice function (including multiple choice, non-duplicated multiple choice)
Color - color system
- color system classes (see: https://github.com/opendilab/treevalue/blob/main/treevalue/utils/color.py)

Compatibility:

py3.6-3.9
pypy3.6
pypy3.7

The mission above is the task of release v0.0.1.

Other mission can be found in #1 .

Bug of pip_install function when run unittest with multiple workers

See here: https://github.com/opendilab/DI-engine/actions/runs/4031645083/jobs/6931053414

__________________ ERROR collecting test/file/test_stream.py ___________________
hbutils/string/plural.py:10: in <module>
    import inflect
E   ModuleNotFoundError: No module named 'inflect'

During handling of the above exception, another exception occurred:
test/file/test_stream.py:7: in <module>
    from hbutils.testing import isolated_directory
hbutils/testing/__init__.py:6: in <module>
    from .compare import *
hbutils/testing/compare/__init__.py:1: in <module>
    from .text import *
hbutils/testing/compare/text.py:12: in <module>
    from ...string import plural_word
hbutils/string/__init__.py:6: in <module>
    from .plural import *
hbutils/string/plural.py:14: in <module>
    pip_install(['inflect>=5.2.0'], silent=True)
hbutils/system/python/package.py:181: in pip_install
    pip('install', *(('--user',) if user else ()), *reqs, silent=silent)
hbutils/system/python/package.py:96: in pip
    assert not process.returncode, f'Error when calling {process.args!r}{os.linesep}' \
E   AssertionError: Error when calling ['/Users/runner/hostedtoolcache/Python/3.9.16/x[64](https://github.com/HansBug/hbutils/actions/runs/4032245347/jobs/6932003704#step:14:65)/bin/python', '-m', 'pip', 'install', 'inflect>=5.2.0']
E   Error Code - 1
E   Stdout:
E   Collecting inflect>=5.2.0
E     Downloading inflect-6.0.2-py3-none-any.whl (34 kB)
E   Collecting pydantic>=1.9.1
E     Downloading pydantic-1.10.4-cp39-cp39-macosx_10_9_x86_64.whl (2.9 MB)
E        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 14.1 MB/s eta 0:00:00
E   Collecting typing-extensions>=4.2.0
E     Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
E   Installing collected packages: typing-extensions, pydantic, inflect
E   
E   
E   Stderr:
E   ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pydantic/types.cpython-39-darwin.so'

Refactor color system

Make the color model immutable
Make the color model compariable (the current model is based on float rgb values, so it is actually not compariable)
Add blend, lighten, substraction for the colors

Error when some names are not covered.

Traceback (most recent call last):
  File "/home/hansbug/oo-projects/data_app/dataapp/gui/gene/result.py", line 9, in <lambda>
    self.clicked.connect(lambda: aetg.aetg_result(result))
  File "/home/hansbug/oo-projects/data_app/dataapp/gui/gene/aetg.py", line 117, in aetg_result
    for i, pair in enumerate(g.tuple_cases()):
  File "/home/hansbug/.pyenv/versions/3.6.3/lib/python3.6/site-packages/hbutils/testing/generator/base.py", line 62, in tuple_cases
    for d in self.cases():
  File "/home/hansbug/.pyenv/versions/3.6.3/lib/python3.6/site-packages/hbutils/testing/generator/aetg.py", line 170, in cases
    xk = (non_exists, node_cnt[ituple], self.__rnd.random())
KeyError: <_NameValueTuple name: 'abc', value: '1'>

hansbug / hbutils Goto Github PK

hbutils's Introduction

hbutils's People

Contributors

Stargazers

Watchers

Forkers

hbutils's Issues

Recommend Projects

Recommend Topics

Recommend Org