fachat / geckos-v2 Goto Github PK

lmon is rather large, source-wise and code-wise; source is mostly artifact from old C64-based development.

• reformat
• use common.a65
• Output redirection
• "scripting" (either cmdline option, or pipe to stdin)

instead if calls "term" directly, so even restart by init is not triggered

The recent Issue #14 has shown that fsiec does not care about the state of the disk drive. As it seems some operations seem to be "interruptive" when done in the middle of other operations, like closing a file while loading a directory.

While that does not should be a problem in general, some protection should be throught of:

1. don't load a directory in parallel (this has secaddr =0 and a second open would conflict with a previous one
2. Defer some operations while loading a directory (e.g. closing a file)

The streams interface is horribly slow as it requires one kernel call per character for reading and another for writing. Previous attempts to speed up have not brought the necessary results (see the GETB/PUTB calls).

A new approach will be as follows:

• STRMAP: map a user space buffer to the receiving resp. sending end of a stream. A buffer can be more than 256 byte in length.
• STRPUSH / STRPULL to send resp. receive data into these mapped buffers, where only the number of bytes put to / taken from the buffer are exchanged as parameters, and the kernel copies the data between the user space and kernel space buffers

The buffer can be of different types:

• contiguous: like a single fread() resp. fwrite() to/from a single buffer
• circular: a circular buffer
• send/receive: STRPUSH and STRPULL are interleaved and a two-way send/receive communication is established

Advantages:

• (hopefully) better speed, e.g. lib6502 can map the full segment buffer and the only loop over STRPULLs on load. The filesystem can use a circular buffer, the kernel will translate.
• the buffer mapping is stored in the kernel and does not need to be re-calculated every time
• potential to extend to zero-copy transfers (but unclear yet how the remapping protocol will work)
• the send/receive code can replace the global SYSBUF/PCBUF mechanism, and related semaphore handling

Disadvantages:

• kernel/library size increases
• only a single task should allow to map a buffer (for resource reasons). the others (if any) still need to use the single-byte interface. Devices will also use the single-byte interfaces
• some sys calls /may/ require mapping the buffer on 256 byte boundaries (e.g. FORK) to avoid remapping in-between processing

Open issues:

• send/receive protocol not fully clear yet

... to provide some standard texts for the known error numbers.

lib6502 should use SBRK to manage memory (at least try to expand when needed)

... which is somewhat counter to what it should be....

Steps to reproduce:

1. Go to the old shell on console 2, type "c:ls". Shell returns to a prompt without running the program (this works in 2.0.9).
2. Enter the command again. The shell does not accept any further input, and the lsh shell on console 1 also becomes unresponsive.

improve TTY handling based on MORE Fix (Issue #27 )

• Tasks get a PROCESS GROUP (PGRP), inherited from parent
• Fork can create new/use PGRP so tasks can be grouped into jobs by shell (creted PGRP Id is same as task ID of forked child, so returned task ID value can be reused)
• STREAMS get an optional "line discipline"
  -- if set
  --- STREAM gets a PGRP attached
  ---- On GETC, if PGRP does not match block
  ---- On PUTC,
  ----- if Ctrl-C -> signal (all) PGRP tasks SIGTERM
  ----- if Ctrl-D -> EOF

It would be good if there were a build chain that would automatically run some test programs in an emulator, and check the results to see if something got broken.

This could potentially be implemented using e.g. an emulator like the VICE Commodore emulator.

This is related to the fact that one command (e.g. PS) uses PCBUF to get information from the kernel, tries, to write it to STDOUT, so it locks PCBUF during this time, while the shell still tries to open the down-pipe process, which is also using PCBUF to open the file and FORK.

Or at least some other recognized console codes (vt52?) so that it can be easily used when telnetting to other computers.

In VICE, the keyboard mapping seems to be broken for pet32k and cbm8296:

pet32k (graphics keyboard):

• "z" seems to be always upper case (maybe with the exception of the first time
• "x" does almost not work at all, needs many presses until it appears

cbm8296 (business keyboard):

• "z" does not work

... to print out diagnostic information on the environment (like map of allocated zeropage, map of allocated memory, statistics on tasks like total used memory etc)
Would probably need some entry point into lib6502 to get the necessary information.

Thus pipes use a different char, which should not be needed

• use alternate ROM pages as file source (like in init)
• allow reading of binary files from ROM

The C64 serial bus is very slow. My usual workaround is to use my IEEE488 interface for any serious work (which is also supported by GeckOS of course :-)

However, not everyone has this, so maybe we can employ a derivation of a fastloader protocol when reading files from disk. (which would help a lot, as basically all commands are loaded from disk before execution).

Open issues:

• can we partially load a file (or load it blockwise) using such a protocol, to not block the OS when multitasking? E.g. creating a pipe that inputs from disk, and loads multiple commands will have to have interleaved reads from disk.

As taken from Glenn Holmer's commentary on https://www.youtube.com/watch?v=jtlAOdJmeDI&t=134s

This would allow at some point completely remove the old "shell" code

wc, cat, hexdump, and more use the basically the same code to iterate through the input parameters, interpret them as files to read, and read them. The file loop is a copy/paste code, so should be factored out into a common included code file.

steps to reproduce:

1. On cbm8x96, ps shows the same exec address for lsh and ps, which is either in the lib6502, or not set correctly
2. when, in c64, running two instances of ps at the same time (one "ps" from the lsh, one "c:ps" from the old style shell on the second console), the exec address of the second ps is inconsistent from different runs, and seemingly unplausible values (like below the one started at the beginning of memory (with exec currently at 237b)

As taken from Glenn Holmer's presentation and comments on GeckOS here https://www.youtube.com/watch?v=jtlAOdJmeDI&t=134s

Allows the GETINFO call to retrieve the command name and implement a "ps" like functionality

Implement Job Control in lsh, i.e. being able to stop, put in background and put in foreground of commands (i.e. jobs = PGRPs):
Use STREAMS "line discipline"

• On PUTC,
  -- if Ctrl-Z -> SIGSTP to PGRP tasks, send SIGCHLD to parent
• Shell manages jobs (PGRP)
  -- SIGCONT to children on fg/bg

The documentation currently is in the doc/ folder in HTML format.
Markdown is a better fit these days.

E.g.
"help ' more"
(Note: use ' as piping so far, see other issues about not being able to enter pipe symbol)

To avoid unnecessary task switching when Streams are full or (at least) empty

@Cenbe reported a bug when trying to setup the internet daemon. Can you pls add details here?

subtask of issue #4

... so e.g. filesystem binaries could be linked to device-/architecture-specific part on load, not build time.

For example the fsibm could be linked to a block device driver that could be either wd1770, scsi or vc1571, vc1571 could pull in an IEC driver, and scsi could be linked to either the hardware scsi driver, or an USB-driver...
fsiec could be linked to the hardware device driver on load as well

As taken from Glenn Holmer's commentary on https://www.youtube.com/watch?v=jtlAOdJmeDI&t=134s

This greatly helps debugging stuff after a program has been loaded - and relocated somewhere into the memory.

A "right" place would be the thread table, and a new call to get some information from there.
A "sufficient" place would probably be the task table, and extend the existing GETINFO call, but that only gives the address of the first thread, not the others.

• starting ROM program is only possible in the old-style shell
  and it doesn't fill in the command line.

Similar to KILL - take some cmdline params and call the DEVCMD kernel call.

which leads to every command (e.g. filters) to print out a newline on their own, which kind of defeats their purpose

An extended "ps" command (e.g. -t option) could show information on threads as well, esp. if they are waiting e.g. on semaphores etc.
Needs an extended GETINFO or new THREADINFO call (preferrably the first)

... instead returns E_FNODRV

Some files still have non-ASCII/non-ISO-Latin-1/non-UTF-8 character in them, most likely coming from edits ages ago on the Atari ST, using its character encoding (e.g. ö as $94).

The target encoding should be UTF-8 but it needs to be checked if xa65 can cope with this.

When building the slipd, the xa from ubuntu (version 2.3.5) bails out with
rsh.a65:line 688: 2ed0:65816 mode used/required error rsh.a65:line 701: 2eed:65816 mode used/required error
that refer to uses of "flib2osa_w" and "flib2osa_r", which translate to
lib6502/libjmp.a65:#define flib2osa_r LIB6502-6 /* maps lib fileno to OS/A65 stream */ lib6502/libjmp.a65:#define flib2osa_w LIB6502-3 /* maps lib fileno to OS/A65 stream */
where "LIB6502" is defined as late-binding label, i.e. using "-LLIB6502"

when a program is started from lsh (e.g. the "ps" command), each time, you can see that the memory is reused, but zeropage addresses are always relocated to new addresses, so it seems zeropage is not freed when the program terminates.

• ignore VFAT long filenames (or handle them... :-)
• I had problems copying file from HD to IEC? Stream error feature?
• implement OSA2 stream error feature
• check filesystem structure (esp. on FAT16) (bytes free count differs from what is expected)

In the monitor, there is
.asc "COPYCON^@dup^@cd^@DevCmd^@copy^@"
This breaks if there is a pre-processor define like
#define DUP 123
As a mitigation you have to split the string constant like so
.asc "COPYCON^@d","UP^@cd^@DevCmd^@copy^@"

The MORE command reads files and pages them on output. After each page MORE waits for user input to show the next page of content.
When MORE is used to pipe files via stdin, however, it still reads the user input from stdin, where the actual data is coming from - so it immediately continues.

In early unix programs, stdin, stdout and stderr were actually the very same file, and commands like the unix "more" were reading from stderr.

As taken from Glenn Holmer's commentary on https://www.youtube.com/watch?v=jtlAOdJmeDI&t=134s

Currently there are two shells - the old "shell" and the new "lsh". Not all functionality has been implemented in lsh yet, so shell keeps being around.
Extending lsh to completely take over from "shell" would allow to clean up the "shell" code, and allow to separate out the "mon" (which is currently mangled with the old shell) to be come a first class program on its own.

Semaphores are still held when a process dies while holding them.
This is especially disastrous with the SEM_SENDBUF semaphore...

This is also the reason why init does not restart the lsh when it exits - it waits for the SEM_SENDBUF still held by the now dead lsh.

The build process for boot images is a mess.

For example, the C64 boot "ROM" segment addresses need to be manually managed in the Makefile depending on the size of the actually assembled segments. And then the available RAM needs to be fed back into c64rom.a65 to let lib6502 know what it can allocate.

I'd rather have some (set of) defined kernel and program images, and build them into a boot image using a proper tool automagically.

Could be initially adapted from the CAT command.

As taken from Glenn Holmer's commentary on https://www.youtube.com/watch?v=jtlAOdJmeDI&t=134s

This is a discussion item. Actual work will happen on subsequent issue items.
Here are some initial thoughts:

• an REU would require a full filesystem built, if used as filesystem.
• A further extension for the REU would be a "swap" functionality to swap out tasks in case of low memory (but that's rather ambitious)
• the CMD HD is an IEC device - it should work somewhat out of the box. Extra functionality could be bolted onto the IEC filesystem, like CHDIR?
• SImilarly for uIEC
• I have no idea about ultimate1541