balvald / artemisscannertracker Goto Github PK

Is your feature request related to a problem? Please describe.
I'd like to be able to look at and search through the scanned and sold biodata.
So I can see what I have already scanned in a system without being in the system.
Because some systems are many light-years away from where I currently am.

Describe the solution you'd like
A new window that shows a table of data from the current cmdr.
Also a search function that limits the entries that I can scroll through.

Describe alternatives you've considered
A new window that shows the biodata in a hierarchical form (first system that can be expanded to planet then bio signals.)
Also a search function that limits the entries that I can look at.

Theres probably still quite the amount of duplicated code since the original event handling was just a full copypaste of the journalcrawling code, (ew I know.)

To amend this:

• Identify duplications
• Create helper functions
• Use those helper functions to replace these duplications.

Helper functions will be stored in the eventhandling.py and will be imported into respective files once thats finished.

unfixable scenario No.1:

Selling a single system with biodata that could be from a selection of different systems that the player has currently scanned and hasn't sold yet. then die and lose the rest of the unsold biodata. - With bad luck this will switch up the exobiology entries that were sold from one system to another.

Prerequesites for this case: unsold data when just looking at amount of each species of system A must be a subset of the unsold data of system B.

unfixable scenario No.2:

Sell a singular organic data scan only from a single system where several more organic scans have been scanned to be sold from the same species but different body. You have to sell the one that isn't on the body that'd come first when they're alphanumerically sorted. Then realize that of course the one sold is not the one tracked as sold. Then die so the state ends up not retrieveable.

Prerequesite for this one: Scan the same species on different planets in the same system.

What "SellOrganicData" needs to make it possible to fix these: Info of the system/body the biodata is from when sold.

Workarounds:

1. Sell the whole batch everytime.
2. Don't ever die with biodata partially sold.

Describe the bug
Since updating to the latest, 0.2.4, release the in-progress scans only show "N/A" value. The accumulation of Unsold Scan Value does happen as expected.

To Reproduce
Steps to reproduce the behavior:

1. Enable in-progress scan value, if it isn't.
2. Scan something.

Screenshots

cough I thought I had a 1/3 scan there, but this is 3/3. anyway, the "N/A" is present all through the scan process.

Desktop (please complete the following information):

• OS: windows
• AST Version: 0.2.4
• EDMC Version: 5.7.0

Additional context
Have verified that finishing a scan, and starting a new one, on another planet, wasn't going to kick-start the process.

Is your feature request related to a problem? Please describe.
Sometimes I lose track in my head of where I am on scanning a planet; I know it has five bio signals, but I can't remember if I have scanned all of them or not. (eg: I exited the game on a planet after scanning, went to sleep, work, and then next evening have no idea where exactly I was at.)

Right now I open up the "Scans in this System", then mentally cross-reference the number of recorded scans for the planet I'm on and the number of bio signals found on the planet. Works, but is mental effort.

Describe the solution you'd like
Show "on this planet:" as well as "in this system:" in the summary line for scans:

In the illustration I show "1/3" scans on the planet; I'd love if this kept track of the number of biological signals on the planet and showed progress like that.
Simply showing the count "for the current planet" when that context exists would be super useful alone, though.

Describe alternatives you've considered
I actually get some of this feedback from the EDMC-Canonn plugin, which tracks the total number of biological signals for the current planet (if you open that part of the UI) and if I have personally hit them with the comp-scanner.

Sadly, there is a big difference between "comp-scanned" and "took three successful bio samples", meaning it sometimes falls down because (a) I comp-scan something to record the location, since it is hard to find and I'm in the middle of sampling another bio type, or (b) I comp-scan something, but don't actually sample it, or (c) I sample something, and completely forget to comp-scan any of the three locations.

Additional context
I considered making a separate feature request for remembering the number of biological signals on the planet and/or asking spansh/EDSM about it. I didn't think it was worth it, but if you want I'll split it off.

I'd like it if the "on this planet" and "on this system" indicators went green, or showed a nice ticked box, when they were fully sampled, too. Something to say "job done, time to go home on to the next system."

PS: thank you again for the wonderful tool. I am extremely happy to have it, and it is one of the most frequently used tools in my EDMC stack.

Is your feature request related to a problem? Please describe.
When I want to check "have I cleared range of previous sample" I spend a lot longer working to read and understand a number – which often changes at least once while I'm looking, triggering a mental "restart" of the work – to determine a boolean.

Sometimes the exact number is very meaningful to me, but maybe 70 percent of the time I just care about the boolean "good or bad range"

Describe the solution you'd like
I'd like to see this one bit of information exposed through the use of colour, which would make it easy to understand at a glance, without having to understand the text itself.

I use the standard windows "white" background, so for me having the text in dark red while too close, and normal black otherwise, would work well. If the text is red I know I have to go further, otherwise it just tells me exactly how far I have gone. (Which is very valuable to know if, eg, I should give up on this direction because I've moved out of the biome.)

Describe alternatives you've considered
I considered "green when far enough", but that highlights the "everything normal, nothing to do" situation. Generally, that isn't the right choice, so using "red, too close, no colour, everything fine" seemed a better option.

I considered if using multiple colours would be useful: red, too close, orange, almost there, green, good. It was instantly attractive, because more information, right? ...but then I thought, "what do I do when it switches red to orange?", the answer being nothing, of course. It is information, but there is no reasonable action to take from it.

I also considered if some extra indicator, like a little square of colour, or an outline, or an image of a checkbox, might work better. I don't think so, but they are certainly valid options, and would probably meet my needs fine. I just like the coloured text idea better.

thank you. :)

Update 14 reportedly comes with buffs to the amount CMDR's earn for different biodata scans

Update the table on the dev branch needed for #3 to still work after Update 14

Current Format on dev branches:

{<cmdr> : [ { species:<species>, system : <system>, body : <body> }, ... , {...} ], ...}

The amount of different biodata scans ought to become big, huge even.
If they get too big searching through the list of dicts that are assigned to a cmdr name might take ages.

Rather have some sort of search register so we can reduce the amount of entries we have to skim through to get to the relevant ones when we want to display already sold biodata.

{<cmdr> : { <letter> : { system : [ { species : <species>, system : <system>, body : <body> } ] }, <letter>: ..., <nonletter/other>: {[...]} }}

then we only need to search through the systems that start with the same letter.
(which is quite significant because currently if we don't have any sold or unsold data in a system we have to check every single entry!)

since #9 breaks the compatability of the old soldbiodata.json to the new version I might aswell introduce this for v0.2.0

Describe the bug
When using the 'Scan game journals for exobiology' the plugin goes into a long procedure that takes a while.
During this time nothing else can be done by EDMC.

To Reproduce
Steps to reproduce the behavior:

1. Make sure there are some game journals available for the plugin to read through.
2. Go to the 'AST' tab in the EDMC settings.
3. Click on 'Scan game journals for exobiology' or 'Scan local journals for exobiology'

Expected behavior
The journal crawling starts and does not prevent EDMC or other EDMC plugins from working while it reads through journals.

Desktop (please complete the following information):

• OS: any
• AST Version: v0.2.9 and earlier
• EDMC Version: 5.10.4 and earlier

Additional context
Add any other context about the problem here.

This issue is for me to check out threading in python because it will come in handy.

• read this
• and subsequently check through this
• alternatively check out other resources that epxlain how threading can be used.
• learn to use threading in the context of an edmc plugin.

Possible use cases here: have a different thread do api calls, and not more plugin locking up when scanning through a lot of journal files

Is your feature request related to a problem? Please describe.
From a Forum post:
I am out in the black so is there not anyway for this to scan my journals for just unsold data so I can see what I currently have is worth? I have only been out for a week so there are less than 20 journals to deal with.

Describe the solution you'd like
The journal crawler should not only update the soldbiodata.json but also the notsoldbiodata.json

As of now the plugin still needs to be restarted for the changed plugin settings to take effect.

This is annoying because any single change (box tick) in the plugins' settings needs atleast a restart of EDMC to take effect.

Implement a way to dynamically add and remove widgets without the need to restart EDMC everytime.

Check https://github.com/EDCD/EDMarketConnector/blob/main/PLUGINS.md#display and https://github.com/EDCD/EDMarketConnector/blob/main/PLUGINS.md#avoiding-potential-pitfalls

Is your feature request related to a problem? Please describe.
I'd sometimes like not to see the scans in a system for a short moment. Especially if there are a lot of different species I've already found.

Describe the solution you'd like
A button on the ui in the 3rd column next to the "Scans in this system" text that can switch wheter or not the scans will get shown.
When there are scans the text in the 2nd column won't say None and instead is just empty, have it say the amount of scans in this system that are in any way finished so I still have a feel of how much is done in the system without having to show the full list (which if there are a lot of species will be quite long)

Describe alternatives you've considered
None.

Describe the bug
Unsold scanned value misreporting after using the journal scan buttons in the settings tab.

To Reproduce
Steps to reproduce the behavior:

1. Go to AST tab in settings
2. Click on "Scan game journals for exobiology"
3. plugin reports unsold exobiology
4. go to vista genommics and try to sell
5. See that there is actually no unsold exobiology left!

OR

1. Go to AST tab in settings
2. Click on "Scan game journals for exobiology"
3. plugin reports unsold exobiology
4. go to vista genommics and try to sell
5. See that there is unsold exobiology but the amount reported is wrong!

Expected behavior

1. Go to AST tab in settings
2. Click on "Scan game journals for exobiology"
3. plugin reports unsold exobiology
4. go to vista genommics and try to sell
5. See that there is unsold exobiology left to sell.

OR

1. Go to AST tab in settings
2. Click on "Scan game journals for exobiology"
3. plugin reports no unsold exobiology
4. go to vista genommics and try to sell
5. See that there is actually no unsold exobiology left.

Desktop (please complete the following information):

• OS: Any
• AST Version: 0.2.5, 0.2.6 and onwards
• EDMC Version: 5.8.1 (shouldn't matter on this one honestly.)

Additional context
Please add cases where this happened, if this happened to you.
Preferably with a set of continous journals that trigger said behaviour.
(Meaning there is no information missing on what the character has done in between. Important are only biodata scan events, deaths, biodata sell events.)

The plugin should check if theres been another release.

If there is another release it should show a button that'll bring the user to the latest release.

• As in the title

The settings options "Scan game journals for exobiology" does not find files and therefor fails to update status.

Steps to reproduce

1. Go to "Files > Settings > AST"
2. Click on "Scan game journals for exobiology"
3. See error

Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python3.10/tkinter/__init__.py", line 1921, in __call__
    return self.func(*args)
  File "/home/poinck/.local/share/EDMarketConnector/plugins/ArtemisScannerTracker/load.py", line 439, in buildsoldbiodatajson
    self.rawvalue = build_biodata_json(logger, config.default_journal_dir)
  File "/home/poinck/.local/share/EDMarketConnector/plugins/ArtemisScannerTracker/journalcrawler.py", line 64, in build_biodata_json
    edlogs = [f for f in os.listdir(journaldir) if f.endswith(".log")]
FileNotFoundError: [Errno 2] No such file or directory: 'None'

Expected behavior:
Update the exobiology status on main EDMC window

System:

• OS: Linux
• AST Version: 0.2.5
• EDMC Version: 5.8.1

Additional context:

• I can rule out that EDMC might not find the journal files, because other plugin do find them.
• Could it be that some of the json files that track the bio-status are not there yet?
• I had to restart EMDC twice, because first time AST did not find it's own config, after clicking "OK" in the settings menu and restarting it, it started to work.

It would be nice if the plugin shows the minimum distance we should travel before make the next scan for this species.

Vista genomics base prices seem to have changed again as of Update 14.01

As a result of this the plugins unsold scan value tracking is probably off. :c

For example, stratum tectonitas apparently slightly over 19M base price (increase compared to Update 14), Bacterium Aurasus at only 1M now (decrease compared to Update 14).

Gotta check what the UI says, and what is the base value they actually contribute towards CMDRs balance.
If theres discrepancy focus on the latter for now.

Or just wait for some time until Canonn has an exhaustive and updated list. (would have the unsold scanning value tracker be inaccurate for some time.)

Describe the bug
I wasn't sure if I should even file this as a bug, but: the third scan of any biological target will reset the "Distance to Scan 2" marked to point to itself. That also turns the text red, because it is now within a couple of meters.

Then, a short while later, the sample is fully processed, and the distance markers clear. So the red text vanishes, and the problem goes away.

Note: I actually decided to file this, finally, after I twice got fooled by this sequence of events:

1. perform the third scan on something.
2. notice the red text for scan 2.
3. immediately jump in the SRV, cancelling the animation for the final sample processsing.
4. drive 1,500m and hunt around for a few minutes to find my third sample.
5. pull out my sample tool, and see it go into the "finishing sampling process" animation.
6. scan 3/3 is recorded and the distance clears.

...yeah, I genuinely forgot I had already gotten two scans of something in the time it took me to find the third sample. These are usually things like bacteria or osseus where the "high density" populations are 1.0 to 1.5KM apart or more, and I'm doing this on SRV/foot. It is surprisingly easy to get distracted and lose state when this is the third or fourth sample in a row on a planet. (well, for me.)

To Reproduce
Steps to reproduce the behavior:

1. Sample any biological thing, twice, to get to 2/3 samples.
2. Find a third instance, at a good sampling range from the other two.
3. Observe that "distance to scan 1, and 2" are both green.
4. perform bio sample 3/3.
5. observe that distance to sample 2 goes red briefly, while the sampling process works through

Expected behavior
the 3/3 sample shouldn't overwrite the 2/3 sample location.

Screenshots
if necessary I'll reproduce this and supply them, but I didn't grab any.

Desktop (please complete the following information):

• OS: windows 11
• AST Version: all since the colour changed for distance to scan X; I think before that too, but I never noticed.
• EDMC Version: 5.7.0

Is your feature request related to a problem? Please describe.
Sometimes, it is very annoying or difficult to find three samples of something. For example, bacterial colonies are sometimes nothing but a vague difference in pattern on a busy surface, or a fungoida lives up in the hills hidden from everyone.

Right now I find myself wondering "is it worth the effort to track down the third sample?"; I'd certainly put in more work for a $19M sample than a $1M sample.

Describe the solution you'd like
If I could see the value of the thing I was currently sampling that'd make it easier to prioritise ("drop this one sample, with no other growths in evidence, and take another thing") or decide to abandon a search ("I'm not getting a headache staring at this dirt for a $1M bacteria")

Describe alternatives you've considered
Right now I look up the value by hand, which is much more cumbersome. :)

Additional context
In my imagination it'd be something like this, but I'm sure there are other ways to display the information that'd work well too.

Unsold Scan Value:  158,209,100 Cr.  (plus 1/3 of 17,900,000 Cr.)

plugin needs to know which cmdr is the one that acutally scanned the exobiology.

Show the profits of currently unsold but fully scanned biodata.

Track the scanned profits for biodata and subtract the amount sold, since you don't have to sell all data at once.

Set the value to 0 Cr. instead of it going negative for some cases when things were scanned while the plugin was unable to track it because EDMC was not used/the plugin was not loaded/active.

No tracking of the bonus the player can get when selling bio data that because its impossible to know if the player gets it beforehand.

edit; when this was written it was in fact not finished on the dev branch.

Is possible in next version create dot in CR (money) and copy icon to export.
Is most simple to see the value and make calcs

Track all biodata. Whats already scanned but unsold and sold.

In E:D Odyssey you can sell biodata in 3 different ways:

• Sell all of the unsold data at once.
• Sell all data of a single system.
• Sell a single particular exobiology scan. (ew.)
  (The last two need some exhaustive testing, minus the impossible edge cases)

The case that seems impossible to catch:
if the player sells the data by system and 2 systems have the same amount of the same species then no one can tell which system was actually sold at vista genomics. This is because the SellOrganicData does not have any info about from where exactly the sold data is in the biodata array (neither system or planet info is anywhere in the respective event in the journal).

Botched as of now "good enough" solution until the "SellOrganicData" event gets changed (so never):
In described case whatever is the first system we encounter thats still possible to be the system that was sold is chosen to have its portion of data be pushed from an unsoldbiodata.json to an soldbiodata.json

unfixable horror scenario:
Selling a single biodata entry from a single system that could be from a selection of different systems which data haven't been sold then die and lose the rest of the unsold biodata. - With bad luck this will switch up two exobiology data entries

Bonus unfixable horror scenario:
Sell a singular organic data scan only from a single system where several more organic scans are ready to be sold from the same species but different body. Then realize that of course the one sold is not the one tracked as sold

The "horror scenarios" are generally avoidable when selling atleast a whole system at once (as people rarely scan only the exact same amount and type of exobiology in two different systems.) And completely avoidable when they sell everything at once.

Missing from current dev branch as of now:

• Add a possibility to have the plugin scan through journals and create/update the respective soldbiodata.json and unsoldbiodata.json
  for any biodata that was scanned prior to installing the plugin.

Is your feature request related to a problem? Please describe.
I'd love to have information on my scan progress while I'm scanning things. I do love it. Thank you!

I'd also like to keep the information EDMC displays relevant to my current activities. Ideally, nothing that I don't care about should be present (eg: mission status, when I'm not doing them, system scan status when not exploring), but definitely nothing that I don't care about should be there.

Right now AST displays a bunch of information slots that I don't care about except when I'm actively scanning, all the time. 😿

Describe the solution you'd like
I can tell you what I'd do to make this happen, but really, all I care about is that I don't see "empty" slots where data could be, but currently isn't. So, to me the "time when a field is relevant" is:

Describe alternatives you've considered
This is my first port of call. If you don't like the idea, which is completely cool BTW, I'll probably
just go ahead and write my own. I am just lazy, haha ;)

but seriously, I can't think of a better alternative than "only show the line if" as per the table above,
but I'd be fine with anything that achieved my goal of only showing information when I care about it,
and probably perfectly happy with 80 percent of that lot, TBH.

Additional context
I hacked this locally to test it, just hiding everything when plugin.AST_current_scan_progress > 0:, which more or less works. It just isn't very elegant, and isn't a full implementation ... and I don't really want to carry a local patch or fork.

That is just dumb, when someone built an awesome tool already. Especially because I bet it isn't as easy as it seems from the outside, haha. ;)

Describe the bug
When scanning the second plant on the same planet EDMC works as intended with the exception to not adding the entry to the notyetsolddata.json.
This then shows in the "Scans in this system" section of the plugin where the recently finished scan does not show up.
(not sure if same planet is relevant, Not sure if a third scan would work again)

Specifically the code that it should run but does not for some reason is:

            if currententrytowrite not in plugin.notyetsolddata[cmdr]:
                # If there is no second Sample scantype event
                # we have to save the data here.
                plugin.notyetsolddata[cmdr].append(currententrytowrite)
                file = plugin.AST_DIR + "\\notsoldbiodata.json"
                with open(file, "r+", encoding="utf8") as f:
                    notsolddata = json.load(f)
                    if cmdr not in notsolddata.keys():
                        notsolddata[cmdr] = []
                    notsolddata[cmdr].append(currententrytowrite)
                    f.seek(0)
                    json.dump(notsolddata, f, indent=4)
                    f.truncate()
                currententrytowrite = {}

located at line 83 within bioscan_event() in eventhandling.py

To Reproduce
Steps to reproduce the behavior:

1. Start EDMC with plugin
2. Do the first full scan of any Exobiology on a planet.
3. Do a second full scan of any Exobiology on that same planet.
4. See error

Expected behavior
the plugin should add the entry of the exobiology scan immediately to the notyetsolddata.json and diplay the finished but unsold scan below the "Scans in this system" line.

Screenshots

Compare what the line "Last Exobiology Scan" says with both the current system and body and see that an entry is missing below the "Scans in this system" line.

Desktop (please complete the following information):

• OS: Windows 11
• AST Version: dev 0.2.7
• EDMC Version: 5.8.1+d81c93c

Additional Context
I can remedy this locally by using the journal crawling function in the settings though it is not great to wait up to a minute after every second scan

Refactor the remaining code in load.py to use ui_label, ui_entry, ui_button instead of tk.Label and tk.Button for consistency and brevity.

Describe the bug
The current version, 0.2.7, does not appear in EDMC 5.9.4.

To Reproduce
Steps to reproduce the behavior:
Install version 0.2.7 with the most recent version of EDMC. No AST section appears in either the window or the plugin settings.

Desktop (please complete the following information):

• OS: Windows 10
• AST Version: 0.2.7
• EDMC Version: 5.9.4

Additional context
0.2.6 does appear with the update available notice.

Is your feature request related to a problem? Please describe.
Once I'm looking at an unsold value of 158,209,100 Cr. a lot of those numbers stop being really meaningful. The difference between 159,209,000 and 159,208,100 Cr. is quite small.

The extra detail makes it harder to understand, at a glance, what my balance is. (For me; I'm sure others don't experience the same.)

Describe the solution you'd like
I'd prefer to see "up to four significant digits" of value. So, for example:

Value             Display
1,234             1,234 Cr.
12,345            12.34k Cr.
123,456           123.5k Cr.
1,234,567         1,234k Cr.
12,345,678        12.35m Cr.
123,456,789       123.5m Cr.
1,234,567,890     1,235m Cr.

Round to nearest (.0 - .499 => 0, .5 - .999 => 1) seems to work best with that.

Describe alternatives you've considered
Any shortening would be fine, really, if it kept to three or four digits. Even five is probably fine to take in quickly, at a glance.

Additional context
I've used this before in displaying numbers where quick reference, at a glance comprehension was needed. It was the best way I found to manage that; people "get" four numbers fast, without getting overwhelmed, but it still shows the "right" amount of detail for humans at each stage.

Update Readme for v0.2.7

Should reflect changes made of #36
and any other changes/features that I might have missed before.

• Fix #53 first
• Add new Screenshots for the Readme to reflect changes in the UI
• Update explanations for v0.2.7 in regards to #36
• Check if there is anything missing in the documentation.
• Do another test to check if theres anything obvious thats wrong with it before making it the next release.

Describe the bug
If you reset a scan with two out of three samples, the "Distance to Scan #2:" field remains pointed at the second sample of the old exobio item.

The "Distance to Scan #1:" field works fine, and when you take the second sample it correctly overwrites the second scan field. No functional problems, just a somewhat confusing display.

To Reproduce
Steps to reproduce the behavior:

1. Scan any random exobio thing twice. eg: get to Tussock Vigram (2/3)
2. Observe that Distance to Scan #1 and #2 fields are both populated.
3. Scan any other exobio thing, accepting the "will discard the partial sample" warning.
4. Observe that Distance to Scan #1 is your newly scanned thing, but #2 is still pointing to the previous sample.

(For maximum effect, travel to a new system between steps 2 and 3 ;)

Expected behavior
The second scan should be cleared.

Desktop (please complete the following information):

• OS: Win 11
• AST Version: 0.2.1
• EDMC Version: latest. sorry, can't quickly find the version, and I don't think it matters.

Thank you so much for all your hard work.

I as the developer would like to have both unit tests and some other tests that simulate certain common and uncommon user behaviours that the plugin would need to face so that when I work on every next iteration of the plugin I don't have to do some tests manually and end up forgetting some obvious or important case and introduce new bugs into the project.

This one is free form no milestone attached to it. Though I've hinted I'll do this after v0.2.5 though probably will do atleast some of it during this.

• #41
• #42
• #43

Best would be also to introduce tests to prevent bugs from resurfacing so a list of bugs that need their tests on the other repo once its created:

• #12
• #32
• #37
• #35