Git Product home page Git Product logo

Comments (6)

emretezel avatar emretezel commented on September 18, 2024

I have added a zero size empty __global_capital.parquet file under data/parquet/capital but now getting a different error

 File "/home/pysystemtrade/opt/pysystemtrade/sysdata/parquet/parquet_capital.py", line 36, in get_capital_pd_df_for_strategy
    pd_df = self.parquet.read_data_given_data_type_and_identifier(
  File "/home/pysystemtrade/opt/pysystemtrade/sysdata/parquet/parquet_access.py", line 54, in read_data_given_data_type_and_identifier
    return pd.read_parquet(filename)
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-user/lib/python3.10/site-packages/pandas/io/parquet.py", line 670, in read_parquet
    return impl.read(
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-user/lib/python3.10/site-packages/pandas/io/parquet.py", line 272, in read
    pa_table = self.api.parquet.read_table(
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-user/lib/python3.10/site-packages/pyarrow/parquet/core.py", line 1776, in read_table
    dataset = ParquetDataset(
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-user/lib/python3.10/site-packages/pyarrow/parquet/core.py", line 1343, in __init__
    [fragment], schema=schema or fragment.physical_schema,
  File "pyarrow/_dataset.pyx", line 1367, in pyarrow._dataset.Fragment.physical_schema.__get__
  File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Could not open Parquet input source '<Buffer>': Parquet file size is 0 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 548, in <module>
    backup_arctic_to_parquet()
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 95, in backup_arctic_to_parquet
    backup_capital(backup_data)
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 451, in backup_capital
    parquet_data = data.parquet_capital.get_capital_pd_df_for_strategy(
  File "/home/pysystemtrade/opt/pysystemtrade/sysdata/parquet/parquet_capital.py", line 40, in get_capital_pd_df_for_strategy
    raise missingData(
syscore.exceptions.missingData: Unable to get capital data from parquet for strategy __global_capital
``

from pysystemtrade.

emretezel avatar emretezel commented on September 18, 2024

After adding the empty missing file, ran interactive update capital manual and getting the following error

(pysystemtrade-user) pysystemtrade@emre-OptiPlex-3080:~/opt/pysystemtrade/sysinit/transfer$ python backup_arctic_to_parquet.py 
Configuring sim logging
2024-03-29 16:17:10 DEBUG config {'type': 'config', 'stage': 'config'} Adding config defaults
2024-03-29 16:17:10 DEBUG backup_arctic_to_parquet Dumping from arctic, mongo to parquet files
Do futures contract prices?n
FX?n
Multiple prices?n
Adjusted prices?n
Strategy positions?n
Contract positions?n
Capital?y
Traceback (most recent call last):
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 548, in <module>
    backup_arctic_to_parquet()
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 95, in backup_arctic_to_parquet
    backup_capital(backup_data)
  File "/home/pysystemtrade/opt/pysystemtrade/sysinit/transfer/backup_arctic_to_parquet.py", line 454, in backup_capital
    if len(parquet_data) > strategy_capital_data:
  File "/home/pysystemtrade/anaconda3/envs/pysystemtrade-user/lib/python3.10/site-packages/pandas/core/generic.py", line 1519, in __nonzero__
    raise ValueError(
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

from pysystemtrade.

PurpleHazeIan avatar PurpleHazeIan commented on September 18, 2024

Hit the same problem during my own migration. Line 454 needs to read 'if len(parquet_data) > len (strategy_capital_data):'

from pysystemtrade.

emretezel avatar emretezel commented on September 18, 2024

Would it be possible for a patch to be committed to the master branch?

Thank you,
Emre

from pysystemtrade.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.