grunt-lucas / porytiles Goto Github PK

It would be nice to have pre-baked TAB completion scripts for bash, zsh, and fish (any other popular shells?).

• public interface needs to have good header documentation comments
• Set up auto-generated documentation: doxygen? RTD?

Apparently, certain maps that use overworld vendors (see Slateport Market for an example) make use of BG palette 12 for the mart menu. If any tiles are using palette 12, they will end up looking distorted. Porytiles should support an option like -forbid-hardware-palette=12 so that the compilation algorithm is forced to keep palette 12 unused. Users can specify this option on specific maps where they intend to have an overworld vendor. For maps with no vendors, then palette 12 (or whatever the top palette is based on fieldmap settings) can be used with no issues.

https://discord.com/channels/976252009114140682/976252039380209694/1223264978010902588

Describe the feature

• -disable-metatile-generation skips generation of metatiles.bin
• -disable-attribute-generation skips generation of metatile_attributes.bin

Description
Delete old nightly releases after some limit? What is a reasonable limit to save? 14?
https://github.com/dev-drprasad/delete-older-releases

Describe the feature
Currently, the decompile-secondary command is stubbed. Finish this implementation.

Description

• Check all in-code TODOs, FIXMEs, FEATUREs, use todo.sh to find them
• Address those that are critical to a polished 1.0.0 product, they are marked 1.0.0

See #46 . This is obviously not a good user experience. Porytiles should provide a way for users to specify that their paired primary is pre-compiled.

• Add support for Windows via MSVC? MinGW?
  • Apparently it is possible to build a cross-compiled version with minGW on a macbook:

  brew install mingw-w64
  cd porytiles
  make clean && CC="x86_64-w64-mingw32-gcc" CXX="x86_64-w64-mingw32-g++" make
  

  • you can extract this and steal the libpng.a file https://packages.msys2.org/package/mingw-w64-x86_64-libpng
• set up package caches so installs don't have to run every time
  • probably too hard to do with homebrew, apt
• Mac universal binary?
  • https://stackoverflow.com/questions/67945226/how-to-build-an-intel-binary-on-an-m1-mac-from-the-command-line-with-the-standar
• better build system? (cmake? autotools? etc)
  • https://cmake.org/cmake/help/latest/guide/tutorial/index.html
  • https://stackoverflow.com/questions/68105648/configure-cmake-to-work-with-homebrew-libraries-instead-system-provided-librarie
  • https://cliutils.gitlab.io/modern-cmake/
  • https://github.com/ttroy50/cmake-examples/tree/master
• static analysis: https://nrk.neocities.org/articles/c-static-analyzers

It seems that libpng is dependent on zlib. MacOS ships with a default zlib, but Porytiles uses the latest via homebrew. This should be made clear in the install instructions, or I should change MacOS Porytiles to use the default zlib. Note that homebrew libpng declares its zlib dependency to be the MacOS shipped zlib: https://github.com/Homebrew/homebrew-core/blob/master/Formula/lib/libpng.rb

I think I added it explicitly because for linux builds, I link statically. And hence libpng needs zlib installed to link properly. However, for the dynamically linked MacOS builds, I don't actually need to specify zlib specifically, since it is specified as a private requirement in libpng. I should clean up the makefile and build scripts accordingly so it's not so confusing.

• the entire warning parsing system is a hot flaming dumpster fire mess, fix it somehow?
  • https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
• warnings-as-errors shouldn't bail until the very end, is there an easy way to do this?
• clang/gcc will show "And 100 similar warnings..." if there are too many warnings, something like this

• https://github.com/ocornut/imgui

Description
Many newer features do not have test coverage. Fix this.

• -dump-anim-code option?
• instead of outputting all the files, just write C code to the console
• the C code should be copy-paste-able into tileset_anims.h/c and src/data/tilesets/headers.h
• is it possible to generate the code and insert it automatically? This would be even better. Would need some kind of syntax tree parsing.

Using decompile-primary to decompile the primary general tileset of pokeemerald generates top/middle/bottom images, but all seem to be created with the wrong palettes.

This is the main green color of the grass tile from the bottom.png that is generated-

This should be the color that matches color 13 in palette 2, but it does not match, as shown here in porymap's palette editor-

This isn't the only color that doesn't match. I found that at least colors 9, 11, 12, and 14 from palette 2 also did not match, but did not test all.
I believe this issue may be causing tilesets to not compile correctly when custom tiles are used in combination with decompiled tiles. For example: Input vs. Output in porymap

Environment:

• amd64
• Porytiles release version: nightly-623dc85a9cfdb1c493e92cd298a9bfb2dabc7cf6

In order to be more clear:

Compiled tilesets should be referred to as "Porymap-format tilesets"

Decompiled tilesets should be referred to as "Porytiles-format tilesets".

Thus, the compiler takes you from a Porytiles-format tileset to a Porymap-format tileset. And the decompiler takes you from a Porymap-format tileset to a Porytiles-format tileset.

We can have documentation that officially defines a Porymap-format tileset and a Porytiles-format tileset.

• input PNGs of any dimension, e.g. users can "draw" the map the way they want it to look
• instead of outputting files for Porymap, output a top, middle, bottom PNG that can then be fed thru compile-primary to get the Porymap files

• hide this behind an optimization flag, -Otile-sharing (will make it easier to test)
• canonical example: Pokémart and Pokécenter roofs are identical tiles with different colorings
  • ideally, two palettes will be color-aligned such that we only have to save a single indexed roof tile

Additional assign algorithms may help. E.g. some kind of A* with a decent heuristic. What else? Can we add better heuristics to the assign algos we already have?

• https://theory.stanford.edu/~amitp/GameProgramming/Heuristics.html
• https://realtimecollisiondetection.net/blog/?p=56

Do more research into heuristics and approximation strategies for solving bin packing, since palette assignment is just a slight variant of this problem.

I can probably make a brew tap for MacOS, or for Linux users who use Homebrew (why? lol)

• https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap
• https://brew.sh/2020/11/18/homebrew-tap-with-bottles-uploaded-to-github-releases/

Other package managers to support, figure out how (probably not via their central package list)

• apt
• pacman
• yum
• any more??

Lots of TODOs, FIXMEs, and NOTEs exist. Clean them up.

• freestanding mode would allow input PNG of any dimension, would only generate a tiles.png and pal files
• might be useful for some people who drew a scene they want to tile-ize
• low-priority feature

It would be nice if we could publish some kind of libporytiles, maybe in the form of a .a/.so/.dylib, as well as a header-only library. That way e.g. Porymap could vendor the header-only lib for integration, or other developers could use the library for their own purposes, etc.

• see how Porymap does this
• this would be an amazing feature to support those who have a customized attribute setup

Add an -exclude-animations option to the Tileset Compilation Options menu. This will be useful for those experimenting with the animation system, and for demo/debugging purposes.

The CImg library provides similar functionality to png++, but it newer and better supported. Png++ is no longer updated and has some issues compiling on certain platforms. This overhaul will require significant testing.

Describe the bug
Trying to decompile the compiled version of SmokingArmy's combined tileset gives:

porytiles: internal compiler error: array::at
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a bug. Please file an issue here:
https://github.com/grunt-lucas/porytiles/issues

In the issue body, please include the following info:
  - the above error message
  - the full command line you ran
  - any relevant input files
  - the version / commit of Porytiles you are using
  - the compiler (and settings) you built with (if you built from source)

Including these items makes it more likely a maintainer will be able to
reproduce the issue and create a fix release.

Note: I had to set the primary tiles override to 1024. I think this may have something to do with the bug. Will need to run it thru the debugger to see why the array access issue occurs.

Additional context

porytiles decompile-primary -Wall -o foo smoking_army_porytiles metatile_behaviors.h 

Original compilation command:

porytiles compile-primary -Wall -o foo -tiles-output-pal=true-color -tiles-primary-override=1024 smoking_army ./metatile_behaviors.h

Environment (please complete the following information):

• Used commit 1af1364 but bug probably occurs with all versions

After decompiling the general primary tileset for use with a custom secondary tileset it seems that any generated metatiles that should be reusing existing general tiles/palettes may be incorrect. All metatiles using custom palettes/tiles seem to be created correctly.
I am using dual-layer tiles, if that helps at all.

• For the overlapping snow/grass metatiles the snow/grass tiles are correct, but any sections that should be reusing palette 2 (or the custom palette generated for the snow/grass tile) and tiles 0x002/0x003 are wrong. Palette 3 is used and tile 0x002 is always flipped horizontally and vertically while tile 0x003 is always flipped vertically.

• The vanilla tree+snow/grass overlapping metatiles are rather strange. For the top edge of the vanilla trees, tile 0x0CD should be used with palette 2, but porytiles is generating a tile in the secondary tileset and uses palette 3, while no appropriate palette is generated for that tile.

• The inside tiles of the vanilla trees in the secondary tileset all seem to be set to vanilla tiles, but the wrong tiles and wrong palettes, almost like it recognized vanilla tiles but chose wrong. Tiles 0x009, 0x019, and 0x029 with palette 2 all should have worked there.

Command used to decompile general primary tileset:
porytiles decompile-primary -o $HOME/general $HOME/pokeemerald-expansion/data/tilesets/primary/general $HOME/pokeemerald-expansion/include/constants/metatile_behaviors.h

Command being used to compile the secondary tileset:
porytiles compile-secondary -dual-layer -Wall -o $HOME/pokeemerald-expansion/data/tilesets/secondary/snow $HOME/porytiles-snow $HOME/general $HOME/pokeemerald-expansion/include/constants/metatile_behaviors.h

Here are the bottom/middle/top images used in compilation

Environment (please complete the following information):

• amd64
• Porytiles release version: nightly-623dc85a9cfdb1c493e92cd298a9bfb2dabc7cf6

Some features don't have coverage:

• todo

Currently, the --verbose option enables logging to stderr. Let's overhaul this to a traditional log level system using the spdlog library. Users can supply e.g. --verbose=INFO to display INFO level logs and above.

In addition to this overhaul, add lots more logging everywhere.

• will require key frame detection / key frame hints at CLI?

Description
Create a basic YouTube tutorial walkthrough.

Me again lol,
I'm trying to generate a tileset using compile-primary and I have the following error:

warning: ./my-tileset/assign.cfg: cached compilation settings not found [-Wmissing-assign-config]
note: running full parameter search matrix, this may take awhile...

fatal error: palette assignment parameter search matrix failed to find any suitable parameters
note: please see the following wiki page for help with working through this error:
      https://wiki-page-link-goes-here.com
terminating compilation of ./my-tileset/

This might be relevant: I was having the "running full parameter search matrix, this may take awhile..." message for a long while, and I accidentally closed the terminal. When I ran the command again, I got this error.

[EDITED] I'd like to avoid losing all the maps I've made using the previous iteration of this tileset; is there a way to rollback what Porytiles has done without losing everything?

Just for context, these are my three .PNG images:

Describe the feature

• -print-report option that prints out various statistics
  • Number of tiles, metatiles, unique colors, etc
  • Palette efficiency in colors-per-palette-slot: a value of 1 means we did a perfect allocation
    • calculate this by taking the Number Unique Colors / Number Slots In Use
  • Print all animation start tiles
  • dump pal files and tile.png to CLI? See e.g.:
    • https://github.com/eddieantonio/imgcat
    • https://github.com/stefanhaustein/TerminalImageViewer

Describe the bug
Decompiler chokes when attributes contain invalid values for target base game. For example, if a TerrainType attribute has an out-of-range value in the compiled tileset, the decompiler fails with an internal error. Ideally, it should simply present the user with some sort of warning and substitute a sane default value.

Additional context
See rusturf_tunnel tileset from Josh

Environment (please complete the following information):
N/A

Describe the feature

• -Wpalette-alloc-efficiency
  • warn user if palette allocation was not 100% efficient
• -Wno-transparency-present
  • warn user if the tileset did not have any of the selected transparency color
  • this is a common mistake if user decompiles a vanilla tileset that uses a different transparency color
    • user may forget to set -transparency-color which will break compilation
• -Wstray-pixel
  • Idea from user Fexty: "Only thing I could imagine is if the algo detects a tile having only a some very few pixels of a different color, it could try without those (change them to the next closest color maybe or just transparent), and if palettes can be assigned with that change let the user know about that. I would say in a lot of cases a situation like this is some accidental miscoloring of pixels."
• -Wsimilar-color-coalesce
  • Another idea from Fexty: "Also one thing I think could be useful is to allow the user to specify a threshold at which "similar" colors get collapsed together."

• palette-overrides folder in the input folder
  • in this folder, numbered JASC PAL files containing exactly 16 colors are copied directly into the final palette
    • e.g. 1.pal will become palette 1
    • fail build if we can't assign all tiles given the override
    • user can put a - on a line to indicate that this could be any color
    • the primary use case for this would be so that users could:
      • 1. e.g. fix a given color slot in a given palette, sometimes this is nice for certain DNS systems, etc
      • 2. manually construct their palettes if they prefer, but still benefit from Porytiles automatic metatile generation and tiles.png compression

I'm having trouble with attribute assignment (like, this metatile is tall grass, this one's a puddle, etc).
I have a primary and a secondary tileset. On my primary tileset all the metatile attributes work fine, but on the secondary tileset there's a house with a door and I can't for the life of me tell it that the door is indeed a door.

These are the images for the tileset:

This is Porymap lying to me:

Note that Porymap also sees the whole house as layer type "middle/top", while it should mainly be bottom (grass) / middle (house), shouldn't it? Again, layer type works fine on my primary tileset.

The attribute file contains:

id,behavior
0,MB_PUDDLE // <---- I added this just to see if it'd change the first tile's attribute: it does not, either
35,MB_NON_ANIMATED_DOOR

The command line I'm running is the following (from inside pokeemerald-expansion/tools/porytiles/, which contains the tileset images in the secondary and primary folders):

porytiles compile-secondary -dual-layer -Wall -o ../../data/tilesets/secondary/test-secondary ./secondary ./primary ../../include/constants/metatile_behaviors.h

Am I missing something or is this a bug?

Description
Finish the wiki.

• Refactor CLI parsing, it's a mess
  • CXXOpts lib may be helpful here