hunterap23 / amped Goto Github PK

Describe the feature that is being tested/implemented
Add automated release/deployment GitHub Actions workflows.

List any relations to other issues in this repository or in other repositories
Requires https://github.com/HunterAP23/PythonAutomatedDeployment to be completed

Describe the feature that is being tested/implemented
Add functionality to replicate the following process functions from the multiprocessing library into the wrapped concurrent.futures classes:

• imap
• imap_unordered
• starmap / starmap_async

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
Add the following functionality (and possibly test the performance of) to the PoolExecutor wrapper class:

• No affinity handling / affinity is handled by the OS
• List vs Tuple vs Queue performance
  • Affinity handled in a list (multiprocessing manager list)
  • Affinity stored in a tuple (requires multiprocessing manager RLock)
  • Affinity stored in a queue (multiprocessing.queue)
  • Affinity stored in deque (handle choosing between appending/popping from left or right sides)
• Standard vs random affinity sorting
• Standard pop/get vs random choice

List any relations to other issues in this repository or in other repositories
Related to https://github.com/HunterAP23/Python_Optimization_Tester
Might be better to test the functionality there instead of here.

Describe the feature that is being tested/implemented
Redesign the architecture for creating individual pools and pool groups.
Current implementation is confusing and hard to work with.

Miscellaneous changes required:

• Pools should have the same inputs & functionality as the underlying ThreadPoolExecutor and ProcessPoolExecutor classes, with the added ability to start with / change CPU core affinity.
• Pools should have the following two arguments:
  • Iterable core_iter listing the individual core numbers that the pool can use. IF not definied then assume all available cores that the process can use.
  • Integer core_limit to limit how many CPU cores can be utilized at any given time. If not defined then assume all provided cores can be utilized at once.
• Pool groups should just handle holding onto pools with an overall affinity
• Pools inside pool groups should be given a name and/or a unique ID
• Pool groups can use either a dictionary key or a namespace for accessing individual pools within the pool group
• Telling a pool group to run a function without giving a name of a specific pool should run the function across all pools within the pool group.
• Pools within pool groups can have their own affinities based on the pool group's affinity

List any relations to other issues in this repository or in other repositories
Relates to #5

Describe the feature that is being tested/implemented
Add the following functionality to the PoolExecutor wrapper class:

• Wraps around combination of Thread and Process pool classes
• Raise "not implemented" exception when using functions not available on the specified class type
• Add optional variable used when creating thread pools to act as a multiplier for the specified number of threads (default multiplier is 5).
  • IE: if the given number of threads was 6, and the multiplier was 1.5, then the total number provided to the super ThreadPoolExecutor would be 9.
  • This should only be applied to ThreadPoolExecutors, NOT ProcessPoolExecutors

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
submit function should user wrapper logic to reduce call overhead.

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
Switch to poetry tool for installing and updating package dependencies, and for creating release wheels on Pypi

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
Find a clear and consistent method of deducing if the system uses the big.LITTLE CPU core architecture design (where the CPU has a mix of performance and efficiency cores that function differently), and if so then deduce what cores are of what type.

This method would preferably work across different operating systems and hardware.
Operating System include:

• Windows
• Linux
• MacOS

Hardware (at the time of writing) includes:

• Intel Core 12th gen CPU's
• Apple M1 ARM SoC's (present on newer Macs / Macbooks / iPads)
• A number of ARM-based CPU's utilized on Android devices

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
Validate function arguments passed into the pools and pool groups.

Use inspect.signature().parameters to find the names of the arguments used in the given functions

• Validate initializer function, if it was provided
• Validate function given by user

List any relations to other issues in this repository or in other repositories
N/A

Describe the feature that is being tested/implemented
Replace any usage of the multiprocessing library with the concurrent.futures library.

List any relations to other issues in this repository or in other repositories
Relates to #1

Describe the feature that is being tested/implemented
Write useful code comments and generate documentation with the sphinx tool to host on readthedocs.org

List any relations to other issues in this repository or in other repositories
N/A