limdor / p2p-lending Goto Github PK

Instead of creating several DataFrames according to the investment platform, this issue will aggregate all of them in a single one. Then, the computations of statistics will be modified to use the unified DataFrame.

The diversification report depends on the overall one. I think that reports must be independent between them.

Add new unit tests.

The values in the table that represent currency, should be right aligned and formated.

should be
19.970,59€

@limdor , you must use the following command to open the dashboard service:

python dashboard.py

that has the following expected output:

 * Serving Flask app 'dashboard' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)

Then, open the dashboard in a browser.

Finally, upload the data.csv file by dropping it in the appropriate box. The expected result is:

I believe that a data structure is needed to make more straightforward statistics computation and reporting. I want to discuss if it is necessary and how it should be.

I propose to have a class with the aggregated investment data frame. Then, the functions that calculate statistics can work with this class to set the necessary information.

I think that this will help to split the reporting functions without duplicating some operations. For instance, we can compute country and loan originator percentages by computing once the total investment.

@limdor, what do you think? Can we arrange a call to talk about this possibility?

The following information should be added to the dashboard

Change the text that appears on the pie chart on mouse over to show only the outstanding principal.
Instead of the following image:

It should show only the value underlined in read formatted with 2 decimals with the euro sign. (13.499,23€)

There is an error in filter_investment_files_by_date when a list of dates is sent as an input argument. The error is because datetime.date doesn't has the method lower.

I obtain the following error:

Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Repository pip instantiated at:
  /Users/aquintanas/BSC/git/p2p-lending/WORKSPACE:11:12: in <toplevel>
  /private/var/tmp/_bazel_root/258ef8fd9a91de21d575463e402686bb/external/rules_python/python/pip.bzl:53:19: in pip_install
Repository rule pip_repository defined at:
  /private/var/tmp/_bazel_root/258ef8fd9a91de21d575463e402686bb/external/rules_python/python/pip_install/pip_repository.bzl:67:33: in <toplevel>
ERROR: An error occurred during the fetch of repository 'pip':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_root/258ef8fd9a91de21d575463e402686bb/external/rules_python/python/pip_install/pip_repository.bzl", line 63, column 13, in _pip_repository_impl
                fail("rules_python_external failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: rules_python_external failed:  (Timed out)
ERROR: Error fetching repository: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_root/258ef8fd9a91de21d575463e402686bb/external/rules_python/python/pip_install/pip_repository.bzl", line 63, column 13, in _pip_repository_impl
                fail("rules_python_external failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: rules_python_external failed:  (Timed out)
ERROR: Skipping '//:main': no such package '@pip//': rules_python_external failed:  (Timed out)
WARNING: Target pattern parsing failed.
ERROR: no such package '@pip//': rules_python_external failed:  (Timed out)
INFO: Elapsed time: 616.653s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: 

Build label: 4.2.1-homebrew
MacOS: Big Sur 11.4

The function filter_investment_files_by_date is unnecessary complex for the current functionalities of the program. Moreover, this function is not correct when a list of dates is passed as an input argument.

Date column is always the same value. We could remove the Date column and just add the date next to the title.

We should decide what to do if there are files for different dates. One option would be to show a message and make it fail (do not allow multiple files), the other would be to show just under the title the date for each platform.

The application provides two modes, on to show past investements and another one to show only the latest.
The point here is how we calculate the latest date. If for one platform we have information of January and February and for the other we have January, February and March, the latest date will be different for both platforms.
This will mean that we will aggregate information for Platform A from February and Platform B from March. This is not necessary intuitive and can generate confusing.
We should consider in case we show only the latest information, to do it for the latest date that we have common for all platforms.

This issue discusses how it should be reported the information parsed and computed. In the last meeting (22 September 2021), it was agreed that the most feasible option seems to be a JSON. Accordingly, I propose the following structure:

date_of_report = {
    overall_statistics = {
        overall_investment = X,
        number_loan_parts = X,
    },
    country_statistics = {
        percentage_loans_in_single_country = X,
        percentage_loans_in_three_countries = X,
    },
    originator_statistics = {
         percentage_loans_in_single_originator = X,
         percentage_loans_in_five_originators = X,
    },
    country_investment_data = { 
        dataFile = Y,
    },
    originator_investment_data = {
        dataFile = Z,
    }
}

@limdor , we should discuss the following points:

• General structure
• Key names
• How to store the data. Now I propose in a different file, but It could be included in the JSON.

Finally, I am not 100% sure that in the future, we won't have to change to an RDBM.

For a given point in time we have the list of all credits with the amount of each of them and the interest.
This information is enough to report a rough estimation on how much income will be generated the next month.

For example:
100€ at 7%
50€ at 2%
would generate each
100 * 0.07 / 12 = 0.58€
50 * 0.02 / 12 = 0.08€
and in this case we would report that for next month there will be an income of 0.66€.

Of course this is not correct because a lot of parameters should be consider but this would give a rough estimation.
Some of the cases where this would fail:

• if the payment is delayed
• if the investment finishes before of the end of the month

The current approach is not going deeper into the dependencies. Next try would be to test the bazel linting system

https://github.com/thundergolfer/bazel-linting-system

This is a inconsistent with what we have currently that all tests are in a test folder but we might consider if this is better because then the test is next to the file and we can clearly see what file has tests and which one not.

Originally posted by @limdor in #43 (comment)

We should add an automatic CI that at least runs
bazel build //:main