fprime-community / fprime-tools Goto Github PK

View Code? Open in Web Editor NEW

16.0 9.0 34.0 684 KB

F´ Python tooling and helpers.

Home Page: https://github.com/nasa/fprime

License: Apache License 2.0

Python 88.00% CMake 1.38% C++ 6.96% Fortran 3.66%

fprime-tools's Introduction

F´ Tools

A package containing tools to work with F´. Installation can be done using:

pip install fprime-tools

Issues for these tools should be reported here: File an issue See also: https://nasa.github.io/fprime to see the F´ framework.

Developer Installation

In order to install a branch not released on PIP, follow these steps:

Clone this repo or the developer fork containing the desired branch
Checkout the desired branch
Remove previous installation with pip uninstall fprime-tools
Install local checkout with pip install .

Developers can add the -e flag when local changes need to affect the tool install immediately. -e links the current directory into the PIP install rather than copying the files over thus allowing local edits to affect the installed tools. This is commonly done during the development cycle.

Installing devel

These instructions will install the devel branch without allowing local edits to affect the installation.

git clone https://github.com/fprime-community/fprime-tools.git
cd fprime-tools
git checkout devel
pip uninstall fprime-tools
pip install .

Black Formatter

To automatically format code, the Black Formatter can be installed with: pip install black

Then it can be run on the project using:

black ./

fprime-tools's People

Contributors

Stargazers

Watchers

fprime-tools's Issues

StringType Crashes on "Too Long"


*F´ Version*	devel
*Affected Component*

Problem Description

StringType uses a non-existant value when validating strings. Thus it crashes when trying to validate rather than producing a proper message:

How to Reproduce

Send a CMD_NO_OP_STRING of length longer than 40 in the GDS (version 2.0)

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "fprime-gds/src/fprime_gds/flask/commands.py", line 122, in put
    self.sender.send_command(command, arg_list)
  File "prime-gds/src/fprime_gds/common/pipeline/standard.py", line 149, in send_command
    cmd_data = fprime_gds.common.data_types.cmd_data.CmdData(
  File "fprime-gds/src/fprime_gds/common/data_types/cmd_data.py", line 70, in __init__
    self.convert_arg_value(val, typ)
  File "fprime-gds/src/fprime_gds/common/data_types/cmd_data.py", line 184, in convert_arg_value
    arg_type.val = arg_val
  File "fprime-tools/src/fprime/common/models/serialize/type_base.py", line 82, in val
    self.validate(val)
  File "fprime-tools/src/fprime/common/models/serialize/string_type.py", line 40, in validate
    raise StringSizeException(len(self.val), self.__max_string_len)
TypeError: object of type 'NoneType' has no len()

Python exception with incorrect command-line arguments


*F´ Version*	3.x
*Affected Component*	n/a

Problem Description

When an fprime-util command is executed with an incorrect argument, the Python code generates an unhelpful exception stack:

How to Reproduce

$: fprime-util generate -ut
Traceback (most recent call last):
  File "/xxxx/bin/fprime-util", line 8, in <module>
    sys.exit(main())
  File "/xxxx/lib/python3.8/site-packages/fprime/util/__main__.py", line 14, in main
    return fprime.util.build_helper.utility_entry(args=sys.argv[1:])
  File "/xxxx/lib/python3.8/site-packages/fprime/util/build_helper.py", line 219, in utility_entry
    parsed, cmake_args, make_args, parser, runners = parse_args(args)
  File "/xxxx/lib/python3.8/site-packages/fprime/util/build_helper.py", line 209, in parse_args
    cmake_args, make_args = validate(parsed, unknown)
  File "/xxxx/lib/python3.8/site-packages/fprime/util/build_helper.py", line 115, in validate
    d_args = {
  File "/xxxx/lib/python3.8/site-packages/fprime/util/build_helper.py", line 116, in <dictcomp>
    match.group(1): match.group(2)
AttributeError: 'NoneType' object has no attribute 'group'

Expected Behavior

Should print an error like invalid argument "-ut"

F´ New Fails on PIP Installs


*F´ Version*
*Affected Component*

Problem Description

To reproduce (user was using a virtual environment):

Install from pip without source. pip install fprime-tools
Run new component: fprime-util new --component

Error:

fprime-util new –component
[WARNING] **** fprime-util new is prototype functionality ****
[INFO] Cookiecutter source: using builtin

Traceback (most recent call last):
  File ".../bin/fprime-util", line 8, in <module>
    sys.exit(main())
  File ".../lib/python3.8/site-packages/fprime/util/__main__.py", line 14, in main
    return fprime.util.build_helper.utility_entry(args=sys.argv[1:])
  File ".../lib/python3.8/site-packages/fprime/util/build_helper.py", line 155, in utility_entry
    runners[parsed.command](build, parsed, cmake_args, make_args)
  File ".../lib/python3.8/site-packages/fprime/util/cli.py", line 112, in template
    return new_component(build.deployment, parsed.platform, parsed.verbose, build)
  File ".../lib/python3.8/site-packages/fprime/fbuild/interaction.py", line 265, in new_component
    cookiecutter(source, extra_context={"component_namespace": deployment.name})
  File ".../lib/python3.8/site-packages/cookiecutter/main.py", line 67, in cookiecutter
    repo_dir, cleanup = determine_repo_dir(
  File ".../lib/python3.8/site-packages/cookiecutter/repository.py", line 131, in determine_repo_dir
    raise RepositoryNotFound(
cookiecutter.exceptions.RepositoryNotFound: A valid repository for ".../lib/python3.8/site-packages/fprime/fbuild/../cookiecutter_templates/cookiecutter-fprime-component" could not be found in the following locations:
.../lib/python3.8/site-packages/fprime/fbuild/../cookiecutter_templates/cookiecutter-fprime-component

Cache Regen Not Showing Errors


*F´ Version*
*Affected Component*

Problem Description

An internal cache-regen that detects an error does not show up the error in the output making debug without verbose impossible.

Reproduce:

fprime-util generate
fprime-util build
add "get_module_name()" to some cmale list
fprime-util build # masked error
fprime-util build --verbose # visible error

Typo in github repository description


*F´ Version*	devel
*Affected Component*	repository

Problem Description

The github repo descriptions is: F´ python tooling and helpers. Formally Fw/Pyhon. There is typo in word Pyhon. It looks like it should be Python.

Documentation


*F´ Version*
*Affected Component*

Feature Description

We should have sphinx run, and include nasa/fprime#499

`fprime-util new` not working since v3.1.0


*F´ Version*	fprime-tools >=3.1.0
*Affected Component*	`fprime-util new`

Problem Description

fptools v3.1.0 added the pass_through argument to the runners[parsed.command] call for calling the runner (see diff). However the callback for the new runner was not updated accordingly, and is still only taking 4 arguments instead of 5 (see here)

How to Reproduce

$ pip install fprime-tools==3.1.0 (or any version >=3.1.0)
$ fprime-util new --component throws

[ERROR] template() takes 4 positional arguments but 5 were given

Comment

Has this not been caught because this command is not used with F` v3, since it's generating XML? The fix is very easy so I'll make a PR. Should help with #37.

(F841) Local variable name is assigned to but never used


*F´ Version*	devel
*Affected Component*	string_util.py

Problem Description

Ruff linter spotted one issue in:

https://github.com/fprime-community/fprime-tools/blob/58e28dfa4a1c93e55aff533747def2b5386aa40f/src/fprime/util/string_util.py#L95

src/fprime/util/string_util.py:95:26: F841 [*] Local variable `e` is assigned to but never used

According to Flake 8: "A local variable in your function was defined but is not being used. This local variable should be removed."

Ref: https://www.flake8rules.com/rules/F841.html

Expected Behavior

The variable should be used or be removed.

PS: can be tagged as Good First Issue

Enum Schematron error


*F´ Version*	2.0
*Affected Component*	all

Problem Description

The schematron rule checker for enums crashes if no enum values are specified.

/fprime/Autocoders/Python/schema/default/enum_value_schematron.rng

How to Reproduce

Create an enum without values
Compile

Expected Behavior

I would expect:

It would give me an error telling me about a missing value field.

Problem is that the rule has the following comment:

Either all enum items should have a value or none should

or, 2) based on the comment, it would allow the enum with no values.

The schematron error produces an exception that should be handled cleanly:

WARNING: XML file XXXXEnumAi.xml is not valid according to schematron x/fprime/Autocoders/Python/src/fprime_ac/parsers/../../../../../Autocoders/Python/schema/default/enum_value_schematron.rng.
Parsing Enum XXXX
'value'
Traceback (most recent call last):
  File "x/fprime/Autocoders/Python/bin/codegen.py", line 1338, in <module>
    main()
  File "x/fprime/Autocoders/Python/bin/codegen.py", line 1294, in main
    if EnumGenerator.generate_enum(xml_filename):
  File "x/fprime/Autocoders/Python/src/fprime_ac/utils/EnumGenerator.py", line 64, in generate_enum
    enum_xml = XmlEnumParser.XmlEnumParser(xml_file)
  File "x/fprime/Autocoders/Python/src/fprime_ac/parsers/XmlEnumParser.py", line 115, in __init__
    self.__items.append((item["name"], item["value"], item["comment"]))
  File "src/lxml/etree.pyx", line 2479, in lxml.etree._Attrib.__getitem__
KeyError: 'value'

(Directory names were edited out)

Remove unnecessary parsers arguments when applicable


*F´ Version*
*Affected Component*

Feature Description

Parsers are architected in a way where they all inherit from a top level parser that defines a bunch of stuff which is supposedly shared. Only, some of this stuff if not used in some commands, bloating the help text with unnecessary and wrong information, because those options don't actually do anything.
E.g.: fprime-util format --help will print those options

  -d DEPLOY, --deploy DEPLOY
                        F prime deployment directory to use. May contain multiple build directories.
  -p PATH, --path PATH  F prime directory to operate on. Default: cwd, /Users/chammard/Work/fp/fprime-tools.
  --build-cache BUILD_CACHE
                        Overrides the build cache with a specific directory
  -v, --verbose         Turn on verbose output.
  --ut                  Run command against unit testing build type

We should find a way to remove those extra parsing options when they are not needed. Somehow, it isn't as straightforward as you'd think to remove an argument from a parser. This may be a hint that our parser architecture is not adapted to the growing functionality of fprime-util, which doesn't only revolve around builds anymore.

Rationale

Un-bloat help texts.

fprime-util build when run with parallel jobs fails to build multiple deployments


*F´ Version*	3.1
*Affected Component*	build

Problem Description

fprime-util build fails when parallel jobs are used (-j2) and multiple deployments are defined in te Fprime project. The error is not appearing if instead of fprime-util build, make is used.

Error:

-- Installing: /home/user/my_proj/default/build-artifacts/Linux/lib/static/libPro1.a
-- Installing: /home/user/my_proj/default/build-artifacts/Linux/lib/static/libPro2.a
-- Installing: /home/user/my_proj/default/build-artifacts/Linux/lib/static/libFlight_Topology.a
-- Installing: /home/user/my_proj/default/build-artifacts/Linux/dict/FlightTopologyAppDictionary.xml
CMake Error at Simulator/cmake_install.cmake:47 (file):
  file INSTALL cannot find
  "/home/user/my_proj/default/build-fprime-automatic-native/bin/Linux/Simulator":
  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:52 (include)


make[3]: *** [Makefile:74: install] Error 1
make[2]: *** [Flight/CMakeFiles/Flight.dir/build.make:165: bin/Linux/Flight] Error 2
make[2]: *** Deleting file 'bin/Linux/Flight'
make[1]: *** [CMakeFiles/Makefile2:10642: Flight/CMakeFiles/Flight.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
[ERROR] CMake erred with return code 2

How to Reproduce

Create a Fprime project with two deployments with one topology each.
Run fprime-util build -j2

Expected Behavior

fprime-util build should run with no errors.

Missions's `config_directory` in `settings.ini` is not taken into account


*F´ Version*	v3.1.0
*Affected Component*	settings.py

Problem Description

A user's project cannot have its own parameter sets. The user may set the value config_directory to his configuration folder, but the tool still points to the configuration folder of F'.

Unless I'm mistaken, the only way to play with these parameters is to directly modify in the configuration of F' existing as a sub-module of the user's project.

Given the fprime-sphinx settings.ini, this feature was available before:

config_directory: Cfg

How to Reproduce

In the project settings.ini file, set a random value to config_directory, like "azerty".
Run fprime-util generate.
Look at the command log:

-- Autocoder constants file: /home/fprime-poc/App/config/AcConstants.ini
-- Configuration header directory: /home/fprime-poc/fprime/config
-- [python3] python3 found at: /home/fprime-venv/bin/python3

Expected Behavior

The user's path must be taken into account to obtain a configuration of the parameters different from the default one of the framework.

By browsing a bit the code base, I think the issue is around these lines.

https://github.com/fprime-community/fprime-tools/blob/2ba8103175423f1de26dacb3e2508207027a516b/src/fprime/fbuild/settings.py#L64

E712 - Comparison to true/false should be 'if cond is true/false:'


*F´ Version*	devel
*Affected Component*	cli.py

Problem Description

Ruff linter spotted one issue in:

https://github.com/fprime-community/fprime-tools/blob/3f7865a8a8b980b16a0aff969e8150c632670281/src/fprime/fbuild/cli.py#L54

src/fprime/fbuild/cli.py:54:25: E712 Comparison to `True` should be `cond is True`
src/fprime/fbuild/cli.py:54:63: E712 Comparison to `False` should be `cond is False`

According to Flake 8: "When comparing a variable to True, you should use the form if x is True or simply if x. The most common incorrect form is if x == True."

Ref: https://www.flake8rules.com/rules/E712.html

Expected Behavior

The line should become something like:

if parsed.ut and not parsed.disable_sanitizers:

fprime-locate-uses


*F´ Version*
*Affected Component*

Feature Description

Add fpp-locs-uses informative. Introspect locations.

include `.gitignore` as part of project scaffold with `fprime-util new --project`


*F´ Version*
*Affected Component*

Feature Description

New projects generated with fprime-util new --project should include a .gitignore

Rationale

As to ensure artifacts, s.a., the build cache generated from fprime-util generate, don't accidentally get committed into git

`fprime-util -p` Does Not Respect CWD


*F´ Version*
*Affected Component*

Problem Description

When running with fprime-util -p it searches for a build cache starting at that directory (e.g -p). It does not look for a build cache from cwd. Thus -d and -p are not inverses.

cd <component>
fprime-util build -d <path to deployment> # builds with deployment

cd <deployment>
fprime-util build -p <path to component> # Does not build with deployment

This should be the case because to build with the detected deployment of a component is:

cd <component>
fprime-util build #Detected deployment

Collect Terminal Mirroring Code


*F´ Version*
*Affected Component*

Feature Description

The code to run CMake in the terminal w/ code highlighting and capturing is fairly sophisticated. The code to run FPP in terminal with capture and mirroring is also sophisticated.

We should collect this code into one library that supports both as it is two solutions to the same problem.

FPP uses FD select to prevent blocking when capturing output
CMake uses pseudo-terminals to maintain the colorization in the output from CMake

Final code features:

Maintain colors
Maintain efficiency of select
Allow for mirroring output to terminal and a capture
Allow "no terminal" to turn off mirroring
Return captured lines as (stderr, stdout)

String-parameter default generates incorrect XML


*F´ Version*	3.1.1
*Affected Component*	Custom F-Prime Component

Problem Description

When establishing a parameter on my custom component of type string, I do it like so:

param TN_HOST : string default "localhost"

Then upon running fprime-util build

The tool crashes with the following error:

attributes construct error, line 143, column 114 (PDUComponentAi.xml, line 143)
Traceback (most recent call last):
  File "./fprime/Autocoders/Python/bin/codegen.py", line 1334, in <module>
    main()
  File "./fprime/Autocoders/Python/bin/codegen.py", line 1255, in main
    xml_type = XmlParser.XmlParser(xml_filename)()
  File "./fprime/Autocoders/Python/src/fprime_ac/parsers/XmlParser.py", line 45, in __init__
    element_tree = etree.parse(fd)
  File "src/lxml/etree.pyx", line 3538, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1897, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1917, in lxml.etree._parseFilelikeDocument
  File "src/lxml/parser.pxi", line 1811, in lxml.etree._parseDocFromFilelike
  File "src/lxml/parser.pxi", line 1201, in lxml.etree._BaseParser._parseDocFromFilelike
  File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
make[3]: *** [CGSClient/PDU/CMakeFiles/CGSClient_PDU.dir/build.make:75: CGSClient/PDU/PDUComponentAc.hpp] Error 255
make[2]: *** [CMakeFiles/Makefile2:7581: CGSClient/PDU/CMakeFiles/CGSClient_PDU.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:7589: CGSClient/PDU/CMakeFiles/CGSClient_PDU.dir/rule] Error 2
make: *** [Makefile:2249: CGSClient_PDU] Error 2
  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
  File "./CGSClient/build-fprime-automatic-native/CGSClient/PDU/PDUComponentAi.xml", line 143
lxml.etree.XMLSyntaxError: attributes construct error, line 143, column 114

When manually inspecting ./CGSClient/build-fprime-automatic-native/CGSClient/PDU/PDUComponentAi.xml line 143, I can clearly see that any quotes surround a string in an *.fpp file are included as part of a string-literal in the generated XML.

This is the line in question:

<parameters>
    <parameter id="0x0" set_opcode="0x2" save_opcode="0x3" name="TN_HOST" data_type="string" size="80" default=""localhost""/>
</parameters>

Specifically drawing attention to

default=""localhost""

Which if I manually adjust to:

default="localhost"

It all works like a charm.

How to Reproduce

Set a string-type default for a parameter such as: param TN_HOST : string default "localhost"
Run fprime-util build
Manually inspect corresponding *ComponentAi.xml file

Expected Behavior

I expect the generated XML to only contain one-pair of strings surrounding the default field instead of two.

fprime-util new --project error in folder without permissions


*fprime-tools Version*	3.2.1
*Affected Component*

Problem Description

When trying to create a new project inside a directory without the right permission, a general error is returned.

How to Reproduce

Create a directory
Remove the permission to create file in the directory
Run fprime-util new --project inside the directory created
You should have the error
[ERROR] [Errno 2] No such file or directory: '/your_folder/new_fprime_project/project.cmake

Expected Behavior

I would expect an error more related to the problem. Something like

[ERROR] Cannot created a directory, permission denied

Clean up passing flags to CMake


*F´ Version*
*Affected Component*

Problem Description

Running fprime-tools generate should implement -Dxyz KEY=VAL to pass flags to CMake, however, this functionality doesn't work as expected :

$ fprime-tools generate --help
# ....
  -Dxyz DXYZ            Pass -D flags through to CMakes

Instead, it's implemented as -DKEY=VAL.

How to Reproduce

doens't work, but 2) does:

$ # 1) bad
$ fprime-tools generate -Dxyz KEY=VAL
$ # 2) good
$ fprime-tools generate -DKEY=VAL

Expected Behavior

Either 1) should work and 2) not, or 2) should be documented rather than 1).

Root cause

Relevant files:

Extra

Also, the CMake flags are passed with syntax -D <var>[:<type>]=<value>, consider adding the parameter to the regex.

fprime-util purge native does not purge native


*F´ Version*	47afb72e1f2cb79c1ab475458b0ed3faca5d317c
*Affected Component*	fprime-tools Version: 3.0.1

Problem Description

I have my default_toolchain defined as vxworks. When I purge native, fprime-tools tries to remove my vxworks build cache instead of my native build cache.

How to Reproduce

Define default_toolchain in settings.ini to a non-native toolchain
fprime-util generate native
fprime-util purge native

Expected Behavior

I expect purge native to remove the build-fprime-automatic-native build cache instead of build-fprime-automatic-vxworks.

add license option when generating project scaffold with `fprime-util new --project`


*F´ Version*
*Affected Component*

Feature Description

Option to add an OSI approved license when generating project scaffold with fprime-util new --project - perhaps with default to Apache 2.0 to match core fprime codebase.

Rationale

As to allow for ease of integration and contributions as an fprime community project

pip install fprime-tools error


*F´ Version*
*Affected Component*

Problem Description

I'm trying to on my M1 Mac and get the error below. Note that my pip was associated with python3.7, however, I was trying to work around that by using <sudo /usr/bin/pip3 install fprime-tools>.

Error Message

Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Developer/CommandLineTools/usr/bin/python3 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/0d/h10rbhy15y5_l_f69gtct5s40000gn/T/tmphcdhm6qo
       cwd: /private/var/folders/0d/h10rbhy15y5_l_f69gtct5s40000gn/T/pip-install-r7e5c70m/pyyaml
  Complete output (44 lines):
  running egg_info
  writing lib/PyYAML.egg-info/PKG-INFO
  writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
  writing top-level names to lib/PyYAML.egg-info/top_level.txt
  Traceback (most recent call last):
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 288, in <module>
      setup(
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 296, in run
      self.find_sources()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 303, in find_sources
      mm.run()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 534, in run
      self.add_defaults()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 570, in add_defaults
      sdist.add_defaults(self)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/command/sdist.py", line 228, in add_defaults
      self._add_defaults_ext()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/command/sdist.py", line 312, in _add_defaults_ext
      self.filelist.extend(build_ext.get_source_files())
    File "setup.py", line 204, in get_source_files
      self.cython_sources(ext.sources, ext)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/cmd.py", line 103, in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources
  ----------------------------------------
ERROR: Command errored out with exit status 1: /Library/Developer/CommandLineTools/usr/bin/python3 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/0d/h10rbhy15y5_l_f69gtct5s40000gn/T/tmphcdhm6qo Check the logs for full command output.
WARNING: You are using pip version 19.2.3, however version 23.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

FPP Visualization Automation


*F´ Version*
*Affected Component*

Feature Description

Automate the fop-visualizations. nasa/fprime#1167

Sub-repo cloning script


*F´ Version*	n/a
*Affected Component*	n/a

Feature Description

It is more frequent that new projects are using F Prime as a sub-repo. It would be convenient to have a cloner script that sets up the sub-repos and local configs for the user. Here are the suggested features:

Create an F Prime sub-repo with the correct release, tag, or commit
Create a local config directory with a copy of what is in the F Prime repo.
Copy Ref deployment to act as placeholder
Create correct cmake files in new Ref.
Check out F Prime sub-repo
Build new Ref deployment (maybe?)

Rationale

Make creating a sub-repo based repository easier and less error-prone.

Update serialization and deserialization of enum types

The enum type is always serialized and deserialized to a single type (the Python ‘i’ type):

https://github.com/fprime-community/fprime-tools/blob/a65f0140e0e1f0ba40522767839059a78eaa3067/src/fprime/common/models/serialize/enum_type.py#L72-L100

We need to revise this code as follows:

Store the integer width in the Python enum (fprime-tools).
Represent the integer width in the AppDictionary.xml (nasa/fprime).
Get the integer width from the AppDictionary.xml when creating the Python enum (fprime-gds).
Use the stored integer width to serialize and deserialize the enum (fprime-tools).

Logging the issue here, but three repos are involved (fprime-tools, fprime-gds, and fprime).

ImportError with MarkupSafe version 2.1.1


*F´ Version*	v3.0.1
*Affected Component*

Problem Description

fprime-util encounters ImportError with MarkupSafe version 2.1.1

$ fprime-util --help
Traceback (most recent call last):
  File "/usr/local/bin/fprime-util", line 5, in <module>
    from fprime.util.__main__ import main
  File "/usr/local/lib/python3.8/dist-packages/fprime/util/__main__.py", line 9, in <module>
    import fprime.util.build_helper
  File "/usr/local/lib/python3.8/dist-packages/fprime/util/build_helper.py", line 31, in <module>
    from fprime.fbuild.cli import add_fbuild_parsers, get_target
  File "/usr/local/lib/python3.8/dist-packages/fprime/fbuild/cli.py", line 12, in <module>
    from fprime.fbuild.interaction import confirm
  File "/usr/local/lib/python3.8/dist-packages/fprime/fbuild/interaction.py", line 12, in <module>
    from cookiecutter.main import cookiecutter
  File "/usr/local/lib/python3.8/dist-packages/cookiecutter/main.py", line 15, in <module>
    from cookiecutter.generate import generate_context, generate_files
  File "/usr/local/lib/python3.8/dist-packages/cookiecutter/generate.py", line 15, in <module>
    from jinja2 import FileSystemLoader
  File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
    from jinja2.utils import Markup
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)

How to Reproduce

pip install fprime-tools
fprime-util --help

Expected Behavior

Should display help text

Add fpp to `fprime-util new`


*F´ Version*
*Affected Component*

Feature Description

Currently fprime-util new generates XML. We should augment it to generate FPP or XML per a flag.

e.g

fprime-util new  # FPP (default)
fprime-util new --xml # XML old setup

invalid F prime deployment


*F´ Version*
*Affected Component*	Camera

Problem Description

I'm trying to create the implementation files for my camera component, and I keep getting an error.

A description of the problem with sufficient detail to understand the issue.
the command I used is "fprime-util impl" and I was in the camera component pathway when using the command. the error statement say's "[ERROR] /home/djaldridge/fprime/CaNOP_FSW/build-fprime-automatic-native is an invalid F prime deployment. Does not define any of the cache fields: FPRIME_PROJECT_ROOT,FPRIME_LIBRARY_LOCATIONS,FPRIME_FRAMEWORK_PATH"

Here's some images of the Linux terminal, the cmake files for the camera, and cmake files for the project:

Continued cmake files for camera:

cmake files for project:

How to Reproduce

clone GitHub project.
went into component pathway.
use command "fprime-util impl".

Expected Behavior

the component creates the implentation files.

A description of the expected behavior.
since I used the command "fprime-util impl", the camera component creates the implementation files.

Explicitly opening CMakeLists.txt with UTF-8 encoding

This is a minor thing, but we had copy-pasted a comment into our deployment's CMakeList.txt which included non-ASCII quotes:

# FPP needs location files to detect where symbols come from.  This locations file needs to be built.  In order to do so, we run an internal CMake build specifically to generate these files.  It runs the same CMake files as the normal build.  FPRIME_FPP_LOCS_BUILD is set to true when it is running that build and false otherwise.
# You need to run that check in v3.RC2 because that special build does not define a “${PROJECT_NAME}” target, and thus the target_compile_options will all fail with “missing target”.  I have a fix submitted for v3 that will make that check unnecessary in the final release candidate.
# TODO (vnguyen): When we baseline against the officially release fprime v3 this should be removed.

The open() Python built-in (https://docs.python.org/3/library/functions.html#open) by default opens with rt which is read text mode which uses some default encoding. Apparently this encoding in our cross compilation environment is ASCII which caused fprime-util to barf on fprime-util generate:

$ fprime-util generate -DCMAKE_TOOLCHAIN_FILE=/opt/cross_toolchain/aarch64-gnu-4.9.toolchain.cmake
Traceback (most recent call last):
  File "/usr/bin/fprime-util", line 33, in <module>
    sys.exit(load_entry_point('fprime-tools==2.0.2', 'console_scripts', 'fprime-util')())
  File "/usr/lib/python3.6/site-packages/fprime_tools-2.0.2-py3.6.egg/fprime/util/__main__.py", line 14, in main
  File "/usr/lib/python3.6/site-packages/fprime_tools-2.0.2-py3.6.egg/fprime/util/build_helper.py", line 330, in utility_entry
  File "/usr/lib/python3.6/site-packages/fprime_tools-2.0.2-py3.6.egg/fprime/fbuild/builder.py", line 519, in find_nearest_deployment
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2000: ordinal not in range(128)

The following git patch explicitly opens with utf8 encoding. Not sure what the ramifications of this would be on other projects and users, but here it is for your consideration.

diff --git a/src/fprime/fbuild/builder.py b/src/fprime/fbuild/builder.py
index 5e7a2f8..76b66df 100644
--- a/src/fprime/fbuild/builder.py
+++ b/src/fprime/fbuild/builder.py
@@ -524,7 +524,7 @@ class Build:
         if not full_path.parents:
             raise UnableToDetectDeploymentException()
         if list_file.exists():
-            with open(list_file) as file_handle:
+            with open(list_file, encoding='utf8') as file_handle:
                 text = file_handle.read()
             if Build.VALID_CMAKE_LIST.search(text):
                 return full_path
diff --git a/src/fprime/fbuild/cmake.py b/src/fprime/fbuild/cmake.py
index 3e215ef..b7e85c8 100644
--- a/src/fprime/fbuild/cmake.py
+++ b/src/fprime/fbuild/cmake.py
@@ -393,7 +393,7 @@ class CMakeHandler:
         if not os.path.isfile(cmake_file):
             raise CMakeProjectException(source_dir, "No CMakeLists.txt is defined")
         # Test the cmake_file for project(
-        with open(cmake_file) as file_handle:
+        with open(cmake_file, encoding='utf8') as file_handle:
             project_lines = list(
                 filter(lambda line: "project(" in line, file_handle.readlines())
             )

EDIT: I should note the patch is based on 3819631ece03008d75bfa9ad978b9298f651c85c

Cheetah version warning


*F´ Version*
*Affected Component*	All

Problem Description

A description of the problem with sufficient detail to understand the issue.

How to Reproduce

Do a new install of the tools:

pip install --upgrade fprime-tools fprime-gds
...
WARNING: The candidate selected for download or install is a yanked version: 'Cheetah3' candidate (version 3.2.6.post2 at https://files.pythonhosted.org/packages/75/8f/54099fb1d2bc830ed0a1e345aca10762fa412e400e13347920d7af991670/Cheetah3-3.2.6.post2-cp38-cp38-manylinux1_x86_64.whl#sha256=7fe156842964c5650a1a583fd958a2e6f2c99720b2b139ecfc69d1d3c1727578 (from https://pypi.org/simple/cheetah3/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*))
Reason for being yanked: Failed

Expected Behavior

No warnings...

fprime-util generate does not forward CMake exit code


*F´ Version*	3.0.0
*Affected Component*	`fprime-tools`

Problem Description

fprime-util generate does not forward the exit code from the CMake configuration step. This means a failed CMake configuration step does not mean fprime-util generate fails (according to exit code). This was caught because my CI proceeded despite CMake configuration errors.

How to Reproduce

git clone --recursive -b dev-non-forwarded-exit-code [email protected]:vnguyen/tmp-fprime-build.git
cd tmp-fprime-build
python3 -m venv venv
source venv/bin/activate
pip install fprime-tools
cd fprime-cadre
fprime-util generate
echo $? # note that it is zero despite CMake failure

Expected Behavior

The exit codes for subprocesses that fail (such as the CMake configuration invocation) should be returned by the fprime-util generate tool.

fprime-community / fprime-tools Goto Github PK

fprime-tools's Introduction

F´ Tools

Developer Installation

Black Formatter

fprime-tools's People

Contributors

Stargazers

Watchers

Forkers

fprime-tools's Issues

Problem Description

How to Reproduce

Problem Description

How to Reproduce

Expected Behavior

Problem Description

Problem Description

Problem Description

Feature Description

Problem Description

How to Reproduce

Comment

Problem Description

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Feature Description

Rationale

Problem Description

How to Reproduce

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Problem Description

Expected Behavior

Feature Description

Feature Description

Rationale

Problem Description

Feature Description

Problem Description

How to Reproduce

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Root cause

Extra

Problem Description

How to Reproduce

Expected Behavior

Feature Description

Rationale

Problem Description

Error Message

Feature Description

Feature Description

Rationale

Problem Description

How to Reproduce

Expected Behavior

Feature Description

Problem Description

How to Reproduce

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Problem Description

How to Reproduce

Expected Behavior

Recommend Projects

Recommend Topics

Recommend Org