Git Product home page Git Product logo

arduboy_toolset's Introduction

Arduboy toolset

Main window Cart builder

Package editor Image converter

A set of personal tools derived heavily from https://github.com/MrBlinky/Arduboy-Python-Utilities. This is a fan project and not an official too, use at your own risk (see license).

The toolset allows you to:

  • Upload and backup sketches + eeprom
  • Upload and backup FX flash data (carts of games)
  • Add/update games in the flashcart of your Arduboy (FX or otherwise)
  • Create/edit custom flashcarts with custom categories
  • Create/edit .arduboy package files
  • Convert images into code or data for use with the Arduboy2 or FX library
  • Update your Arduboy FX with the latest games and updates from the official cart website

Quickstart

  • Get the latest release from https://github.com/randomouscrap98/arduboy_toolset/releases.
  • For Windows: just run the exe
  • For MacOS (requires a modern version, sorry!): mount the .dmg and run the tool directly from within. There is no installer
  • For Linux, you probably need to add yourself to a dialout group, or run the program as sudo:
    • sudo usermod -a -G dialout <username>, just once
    • There are no releases for Linux. Clone the repo (or download the source from release) and if you're unsure, just run sh linux_easyrun.sh. Feel free to modify the script if you don't like it! All requirements are satisfied by pip and python 3.8 or greater

The first window is the basic toolset, letting you upload and download stuff from your Arduboy. Most of the tools are here, including the .arduboy package creator and image converter.

If you want to add or update games on your Arduboy FX, use the File menu to open the cart builder. From here, you can use the newly opened window's File menu to read the FX cart off your Arduboy FX. It will take sometime to load. Once it's complete, you can browse the local copy of your FX games in the window. Changes you make here are not immediately put onto your Arduboy.

You can add games by dragging and dropping .hex or .arduboy files into the window. They will be inserted below the currently focused menu item. If you accidently place a game where you don't want it, you can drag the games around in the list, or press Ctrl-delete or use the Cart menu to remove them. When you're ready to put everything back on your Arduboy, use the File menu again to Flash to Arduboy.

Get the latest games by using the Network menu to update your cart. This connects to the offical cart website and calculates which games you're missing and which need an update. WARNING: there are no unique identifiers for games, so a best attempt is made to match games against the official cart website, but it can get it wrong! Always check the update window for correctness before applying the update!

Caveats

While I've done as much as I can to ensure the tool works appropriately, I can't promise it will work flawlessly, I don't take any responsibility for lost data (I'm sorry!).

Prince of Arabia + other FX titles, 2023

Using the Cart Editor will usually preserve the FX saves used by the more complex "FX" games, such as Prince Of Arabia. However, we've found that sometimes the carts were not configured properly from the cart builder website, and your save data may be located in the development area at the end of the flashcart. As such, there is a chance that updating your cart will make you lose this save. You can check if your Prince of Arabia game is correct by loading the cart in the cart builder, using Ctrl-F to search for Arabia, then checking the 3 numbers under the game's title image on the left. If the last number is 4096 or higher, your game is correct. If it's 0, it is development mode and you may lose the save.

If you wish to backup a "Development" save, you will need to make a complete backup of your FX, without trimming. The Arduboy toolset defaults to trimming the FX backup, just uncheck the trim option before you backup you flash and the save data will be included. This is because the save is stored at the very end of the cart, so trimming the backup would remove that part from the file.

Running From Source

If you have python already and just want to run from source, or want to run the command line (clone repo first ofc):

cd arduboy_toolset
# Optional: create and activate a virtual environment
# python -m venv .venv
# source .venv/bin/activate
pip install -r requirements.txt
python main_gui.py
# You can also run the cart builder directly (anything named main_* can be run directly)
python main_cart.py

Creating a standalone

This has to be done for each operating system, there is no cross compiler. Note that because of weird nonsense with Windows, I have to build the CLI and the GUI separately.

cd arduboy_toolset
# Optional: create and activate a virtual environment
# python -m venv .venv
# source .venv/bin/activate
pip install -r requirements.txt
pyinstaller arduboy_toolset.spec
# For MacOS, need to then package it into a dmg (command will be added later)

Notes:

  • There used to be a CLI app, and the code is still there if you want to use it, but I'm no longer supporting it. I plan on making a separate, far more robust CLI
  • The GUI is a "onefile" app, it will startup slower but you can take that single file and put it anywhere. This may change in the future to increase startup time
  • The GUI has the console removed; if you're looking for logs, the program logs to a txt file next to the executable
  • Windows will often mark it as some kind of "dangerous file". Please see pyinstaller/pyinstaller#5854

arduboy_toolset's People

Contributors

randomouscrap98 avatar steka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

steka eggfly

arduboy_toolset's Issues

Image converter

  • A new tab dedicated to image conversion
  • A big area to display the image X:1 (needs scrollbars)
  • A way to set the zoom (default 4?)
  • An area for configuring tile + spacing + transparency
  • Image is redrawn with bounding boxes as settings change
  • A readonly box at the bottom for output, must manually convert
  • Button to convert, button to convert + save (adds pragma and stdio.h)
  • Modifying any image values should mark the box as "invalid" (or maybe just clear it?)
  • Should maybe mark the parts of the image that will become the mask? Seems difficult...

Add FX uploader

I asked Mr.Blinky and they said the "flashcart-upload.py" is the correct file (the old one), so I will use that. There's a lot of "dev" options there, I'm not sure if I want to add all those flags, even though they are probably pertinent to someone. I'll have to think about it; it makes sense to create it "on the fly" but then we have all these special flags JUST for fx uploads. It's easy to do in the UI, and if the CLI should mirror the UI, then the flags SHOULD exist. An alternative is to run the tool once to generate special fx data with extras and another to flash that newly created file, but that's a lot of extra work, especially if that's a common workflow. It also doesn't match the current setup of flags for "on the fly" patches, like for the screen and LEDs.

Chances are, this flash will take a long time. It's not even 8KB per second I think, and it has to flash 16MB. But someone online said 5 minutes for 9MB... far shy of the 22 minutes I calculated assuming no losses. Hm... who knows

Sketch backup incorrect bootloader size

The sketch backup function from Mr.Blinky's scripts assumes a bootloader size of 4k, meaning a backup of a larger program will not include the last 1k. This may need to be remedied after adding other functions to detect bootloader size + type

Add ability to generate save blocks

I don't know if this'll be useful, it's mostly for carts that are missing a save when there should be one. Just having a save in the cart should automatically patch the program, so it could "fix" development mode I think? Hopefully nobody will need more than the 4k save, I don't want to complicate that interface...

More visual cues on category + FX games

It's hard to see which slots are categories at a glance, it should be way more obvious. Maybe categories could have a spacer at the bottom that's a very ridiculous color? That might also fix the glitchy height issues if you can somehow get it to be the size of two textboxes...

Also, there should be some visual indicator if a game is FX enabled. Just check if there's FX data or save data.

In Utilities, add a .arduboy creator/editor

Might as well have an easier (gui) way to create .arduboy files. Could be a simple form that accepts .hex files, data + save files, a title screen (I'm aware arduboy files allow more than one image), and the various bits of information that goes inside the info.json file. More complex things can be added later (like multiple images, etc).

One important feature is that the data .bin file can be scanned to see if there's potentially a dev save file at the end of it, and ask if you want to strip it out into a proper save. The form should have the usual user error stuff, like not having a .hex file, maybe parsing the .hex file to see if it's valid, auto conversion of images to the proper 1 bit format and size, maybe an auto-generation feature for the title screen, etc. For now, I want to only support a subset of the fields, namely the ones that go into the cart file.

Add function to read bootloader only, check it with sketch analysis

Sketch analysis now does a rudimentary test for fx, mini, or plain arduboy. It might not be entirely trustworthy, but let's assume it is. Sketch analysis runs against raw bytes, so it will work on the bootloader too.

Create a function which reads the correct amount of data out of the bootloader (4kb for older, 3kb for V > 13 (?)) and returns the bootloader itself. Perhaps there's a function in mr.blinky's scripts, otherwise read the bootloader instructions.

Once you have that function, create a wrapper function (perhaps in shortcuts? idk) which will detect what device is connected to a port. Later, use this to improve the entire toolset (not yet though, want to test it in the field)

Add shortcut (and menu option?) to jump between categories

Not sure what the keyboard shortcut should be, maybe ctrl+up or down? I kinda wanted to use that to move individual slots up and down, maybe reserve that for ctrl+shift+up? Those two might not make sense put together like that though.

Make arduboy parser prefer title.png as title

I don't see any way to mark a screenshot as a title image, so I'm going to assume anything called "title.png" is the title. What I think should happen is:

  • If you find a .png file and there's no image (and it's not banner.png), set it as the image
  • If you find a title.png, always set it as the image regardless if it has been set before
    This should still allow us to do a single pass on the zip and get an image, but prefer the title.png

Bootloader info not updated

Since you removed the bootloader from the update list, it is no longer linked against missing data in the category

Update QOL

Need:

  • To alert the user when no updates are found
  • To apply games with no category to SOME location (or remove them from the update list? maybe that's bad)
  • Maybe add more information to the loading screens?
  • Try to avoid that freeze on "applying" the update? Is there any way to do that?

This may require updating the "simple" form of the progress thing to still allow information to show in the box somehow.

Add a shortcut which detects the dragged fileset for certain features + auto-creates a slot

There are certain oft-used drags which could be made easier. Right now, each file is treated as an individual slot, so you can't for instance drag a .hex and a .bin into the cart builder and expect the .bin to work. However, I could detect if "data" or "save" is in the .bin file, and if the only dragged files are a single .hex file and one or both of those bin files, I could put them all together into a single slot.

But this would require users to name their files appropriately, as there's no good way to tell if a file should be a save file or a data file (at least, not without guessing, which might be bad).

Create an SQLite schema for flashcart saves

Take a look at the .csv outputs from the actual cart builder (http://www.bloggingadeadhorse.com/cart/Cart.html) and figure out a good format for some tables in SQLite. It might be as easy as just two tables: one for categories and one for programs. There should be fields to store blobs for the images and the flashable data, whatever format you decide to store it in (is there a conversion from bin to hex? If stored as bin, you'll need to add the ability to flash raw bin to arduboy sketch area). Remember, this isn't meant to be a replacement for the official cart format, and isn't meant to go on the web, so you don't need anything fancy. But make sure it COULD be extended in the future just in case.

After creating the format, consider looking at the flashcart decompiler from Mr.Blinky and seeing how difficult it would be to have it decompile to sqlite data instead. Probably simple... I hope.

Request for midi converter

I'll have to look into existing tools and what's even available out there and familiarize myself with the arduboy libraries for sound

arduboy connection issues on linux/opensuse

When connecting to my arduboy mini for upload/backup/etc, the device shows as getting reset. During this reset process, the ttyACM0 device is owned by root for a second instead of dialout. I think this is why I get could not open port /dev/ttyACM0: [Errno 5] Input/output error: '/dev/ttyACM0'. MrBlinky's python scripts work for me, so it could be there are some timeouts in place there that allow the device reset to settle.

Massive oversight with binaries + support for Arduboy Mini

So, I think large portions of the cart builder, the UI, and arduboy parsing may need to be completely changed because of this oversight.

There needs to be a way to ensure that the device for the current cart is KNOWN. Always known, like... if you load it from the device, then the device can be set. If you load a bin from the filesystem, then the device must be autodetected (if possible) or the user will have to choose.

Furthermore, parsing AND creation of arduboy files needs to be changed to support multiple binaries. There need to be systems in place to determine if a given binary is safe for use on a given device. The devices must be standardized + constants (hopefully there are standard naming conventions chosen in the arduboy community for these).

  • When creating a binary, nothing needs to be changed
  • When creating the .arduboy package (any time), need to indicate the system it's for. Since the ArduboyParsed format is changing anyway, just need whatever is constructing the ArduboyParsed object to put the right binaries in the right places.
  • Will probably need to entirely reinvent the package creator interface, can't reuse the slot item (unless...?)
  • When parsing arduboy files, need to inspect list of binaries and create proper list. Always need to check device.
  • When parsing hex files into ArduboyParsed, need to auto-detect device (if possible) and if not possible, device must be set to "unknown".
  • When loading .arduboy into the cart, need to load the appropriate binaries. If none are available for the current system (remember, system must ALWAYS be chosen), need to throw up appropriate errors.
  • What happens when a default .hex is loaded? Assume it's correct for the system?

Auto-detect dev save area in data and ask if you want to create a save

When adding data to the cart, IF there is no save already added to a slot and you're adding data, if the data has a section at the end that is precisely the size of a save (4k, 8k, etc when looking at page chunks), ask if the user would like to trim that save file out and add it as a save. Or, if a save already exists, just ask if you want to trim the save without overwriting what's there.

Maybe add "add game here" button to categories?

There's a lot of wasted space with categories. Perhaps there could be some room made for a button for adding games to a category? It's a bit clunky to click on a category then add the game from the menu, especially because clicking on the big yellow area doesn't select the item.

Maybe also look into making ANY focus select the current item, so it's more intuitive. Might be a manual process.

Add a way to move categories around

If we're going to stick with the list format for a while, there should be a way to move categories around and bring all their games with them. Should be easy, since I calculate the category index values on compile and ignore whatever was there before.

Add "inspect" for header + save

Have a very minimally featured hex display for both the header and the save file. Perhaps there's something you can import which will be a hex editor already...

Anyway, add this inspect feature to the debug menu.

Add link to cart builder in utilities tab

Finding the cart builder is a bit more confusing on Mac (or maybe I'm just not used to it) and so it'd be nice to have a second way to get to the cart builder by going to the utilities tab.

Add a way to backup fx saves (and reapply them?)

It'd be cool if there were some quick way to pull all the fx saves off (maybe even including the dev save at the end?). I don't know what people would do with them, it might be difficult to automatically reapply them later, but they could definitely manually apply them.

Add shortcut (and menu option?) to move current slot up or down

Dragging and dropping is all well and good, but it'd be nice to quickly move a slot through the list. Since a lot of stuff is going to share this "remove and readd" slot code, it should definitely be made generic.

Not sure what the keyboard shortcut is. Ctrl+shift+u already moves whole categories up and down. Ah, idea:

ctrl+u/d moves up and down through categories
ctrl+shift+u/d shifts categories up and down
ctrl+shift+up/down shifts slots up and down

(nothing assigned to ctrl+up/down maybe)

Image convert drag + drop

Let drag+drop work for... maybe just the preview area? I was thinking the whole thing but IDK how it'll function with inputs.

Add ability to show image on arduboy

There's a script in Mr.Blinky's repo that shows images on the arduboy. You should do that; readd the "utilities" section and maybe have something like that there?

Add some kind of cart utility to the CLI

I'm thinking I could have a command that parses a bin file into an extremely simple sqlite database. The user could then use their own commands to modify the database, then use the tools to recompile the database into a bin. Maybe there should be a shortcut step too that somehow updates games on the cart? Seems difficult, the sqlite thing might be easier.

Add option for pragma or header guard

When exporting image to .h, have option to choose between header guard or pragma. Or... just generate the header guard? Not sure yet, does anyone actually prefer the pragma?

Patch checkboxes + radios in menu for cart editor

In the file menu, there should be a "Patches" option that opens a submenu with a checkbox for ssd1309 and a radio for contrast. One option should be "no change". If there's no radio option for menus and it must be done manually, just allow double checking to remove and don't include the "no change"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.