How to support other languages on top of ChrysaLisp? about chrysalisp HOT 19 CLOSED

What @vygr said.

Disclaimer: I am not a programmer. I'm a technical writer and journalist, following a line of research that will I hope lead to a book and so far has spawned a FOSDEM talk among others, and a lot of blog posts.

Programming languages are cultures. 2 are dominant in the world today: Windows and Unix. And modern Windows is a lot like Unix: it's written in C, with a programming model that implies, and higher-level software on top of it is also in C or C derivatives such as C++.

This has led many people to believe that C is in some way fundamental. It isn't. It isn't even a low-level language. It was in the 1960s but it isn't now.

C is a culture and it is a pervasive, infectious culture. Everything built on C tends to inherit that culture, and because that's all anyone under 50 has seen, they don't see it because they think it's universal. It isn't.

There are dozens to hundreds of interesting operating systems and programming languages that are nothing to do with C.

Non-C-like low-level languages for building OSes in include Oberon, Forth and Lisp.

Interesting OSes that I like that were not built on C include ColorForth, Oberon (the language and the OS, they share a name), Symbolics Genera and Taos.

The result are OSes with a profoundly different flavour, right from the ground up.

In my research into other schools of computing, I found that Lisp has a whole rich history that's now mostly forgotten. It's now a weird language that influenced Clojure and is used for scripting Emacs and that's about it.

It is much, much more than that.

Some writing on this:
http://www.catb.org/esr/faqs/hacker-howto.html
http://www.paulgraham.com/avg.html
http://www.marktarver.com/bipolar.html

from chrysalisp.

This is an interesting question. It brings up a lot of issues ! Liam is correct in his statement, but I'm not totally hard line on this.

My reason for doing this project was stated in the last HackerNews post where I commented. But in short I wanted to create a purely assembler based system, for my own personal therapy ;), and the Lisp angle came about due to a long standing debate with a good friend who's crazy about Lisp. As I ended up needing to replace NASM as my assembler I decided to try writing a Lisp to code the new assembler in. The C-Script stuff happened purely as a way to fast prototype assembler level functions.

So do I want to even encourage other languages onto the system ? Mmmm, tricky. I personally have no desire to use another language, on ChrysaLisp, this combo proves highly effective for me. However that might not be the case for other people.

I have a TODO item to look into getting LLVM to produce ChrysaLisp function format, and that would be my desired route to get other language support. But one generally dosn't just end up with the language, you get dragged into all the extra libraries and OS support features etc etc and before you know it your just a Unix clone or whatever....oh but you must support the fork call.......

I have been here before ! Taos in 1990 was a nano kernel parallel OS that was 12KB with a 170KB translator and was truly great (yeah I would say that wouldn't I ;) ). But Tao Group gave in to pressure to implement C/C++ and Unix like features and eventually it was just yet another not quite Unix with multi-megabyte footprint. We even started dropping features that were the entire point of Taos as they didn't really fit the C/Unix model.

We should have stuck with what we had and just used it to create products that succeeded on there own merit. It's all still a tender area for me, hence my reference to working out some historical Angst by doing this project.

So, what to do ? I don't want people to think I disparage other languages or OS's, I don't. I use C/C++ day in day out for all my other coding and I think LLVM and Clang are really superb projects and that Linux is what it is, Windows is what it is, and OSX is what it is etc etc. But do I want ChrysaLisp to end up being them ? Mmmm.

I would encourage you, Martyn or any other, if your inclined, to look into a back end for LLVM that can write VP functions. First goal would be to allow the compilation of individual functions, not C/C++ main.exe's but just get to the point of allowing C, with no use of globals, or statics or anything that brings up the whole OS support for thread level storage and so forth. Just isolated, local vars only, with clean in and outs. That would give us the ability to see just how good the code produced is ! I'm not 100% sold on the merits of this either after my ChrysaLisp++ project , but it would be the first thing to explore.

from chrysalisp.

Since IMHO the entire point is to get away from the evils of C and its kin, that is not desirable, but then, it's not my project.

If you want new interesting OS projects that are more modern takes on C-family programming languages, there are plenty:

https://github.com/JehanneOS/jehanne -- derived from Plan 9, i.e. UNIX 2.0
https://github.com/redox-os/redox -- new, microkernel-based, in Rust
https://github.com/haiku/haiku -- modern BeOS-alike in C++
https://github.com/google/gvisor -- Google's user-space kernel, in Go, for running in containers
https://github.com/achilleasa/gopher-os -- proof-of-concept Go kernel
https://github.com/HelenOS/helenos -- clean modern Unix-alike
https://github.com/minix3/minix -- FOSS microkernel Unix by Dr Andy Tanenbaum

I'm sure there are more.

from chrysalisp.

Thanks for your detailed response. My interest is that I'd like to find out more about how everything works and I'm much more comfortable at the c asm level than with lisp.

Although to be fair I haven't really put any effort into learning it....

My thought process was that by implementing something I'm comfortable with I could focus on the bits I'm not 100% on.

I also don't want to reimplement anything more than a c type language, no std libs, no system calls just the same functionality that can be called from lisp using c syntax.

As everything goes to vp assembly I thought this would be doable.

from chrysalisp.

Before I wrote the Lisp interpreter in ChrysaLisp I had always heard about Lisp, how it was supposedly the best thing since sliced bread, but that all the parens made it really terrible etc, all the usual stuff.

Then I actually started to try it, and yes, it really is a good thing (tm) :) Well worth having a go with. So I would push you to give it a try, it's not that difficult to get into, and you might just end up loving it as I did. ;)

from chrysalisp.

Sounds like a plan, are there any resources that you'd recommend?

from chrysalisp.

For LLVM just head to their website..

For Lisp I can highly recommend http://www.gigamonkeys.com/book/, very good book indeed. But do remember that ChrysaLisp don't do any of that car/cdr/cons stuff ! But this is a great read and gives you a good set of examples as to why Lisp is so good.

ChrysaLisp tends to stick with the higher level functional ways to manipulate lists, map, reduce, each, some, etc. Googling 'lisp map' etc will normally land you on some lisp reference site that will explain what they do.

from chrysalisp.

I've had a long ponder on this and I think the best way for me to approach this is to try and implement something somewhat substantial. The current ideas I have are: disassembler and possibly binary modifier tool, a game engine something along the lines of scumm vm, things in that vein. Any suggestions comments on these as being a good fit for ChrysaLisp and/or other suggestions.

from chrysalisp.

disassembler shell command would be nice :) Let you try some Lisp and give us a nice additional tool :)

Would that also be able to disassemble a boot_image file, so multiple functions in an unbound block, and show you x86_64 and Aarch64 too ? As well as individual functions.

from chrysalisp.

Those seems like a reasonable set of goals. Let me know if you have any pointers about where to hook in, or things to look at.

from chrysalisp.

Take a look first at cmd/dump.lisp good starting point that reads in a byte stream from stdin or a file.

Look through the doc/FUNCTION.md doc to get all the info on the function format. And look at sys/func.inc to see how functions are put together by the assembler.

Your aiming to reverse that. I’d start by trying to disassemble a function into its 5 sections, the code block can just be hex dumped for now. But take apart the header and paths and print them out nice etc. Then start to think about turning the code section into a dissembled listing etc.

You should be well on your way with the Lisp by the time your at this stage.

Also i’d Have the main cmd/dis.lisp file plus a folder cmd/dis/xxxx for all the other code and things that go with that app. Probably set this as the way command style apps organise things going forward.

I’d have the cmd/dis.lisp use the filename and split it to extract the cpu and dynamically chose to read in the disassembler parts from cmd/dis/... for that cpu type. So only pulling in the stuff that’s needed.

The (code str [offset width]) function is going to be your friend while doing this !

from chrysalisp.

Fantastic - thanks very much. I'll keep you posted.

from chrysalisp.

Also these macros are defined in boot.lisp and you will find them most useful as well !

I’d gulp the whole function in with a single (load name) call and then take it apart with these macros. No need to redefine them they will be loaded allready.

(defmacro get-byte (s o)
;(get-byte str index) -> num
(list 'code s 1 o))
(defmacro get-short (s o)
;(get-short str index) -> num
(list 'code s 2 o))
(defmacro get-int (s o)
;(get-int str index) -> num
(list 'code s 4 o))
(defmacro get-long (s o)
;(get-long str index) -> num
(list 'code s 8 o))
(defun get-cstr (s o)
;(get-cstr str index) -> str
(defq k o)
(while (ne 0 (get-byte s o))
(setq o (inc o)))
(slice k o s))

from chrysalisp.

Start a dis branch and push stuff to that as you go along. Every time you happy to merge over to master do a PR etc.

from chrysalisp.

Sure thing - thanks again.

from chrysalisp.

One last tip. Take a look at the cmd/asm.inc file at the (make-boot) stuff. That does a huge amount of similar things to pull functions apart and glue them back together into the single boot_image file.

You’ll find lots of interesting things there that mirror a lot of what you’ll be doing.

from chrysalisp.

You might find this old project by @tvdstorm of interest:

https://github.com/tvdstorm/CherryLisp

from chrysalisp.

@TheMartynBliss:

How about getting this up and running:
https://greenfield.app/ (Repo by @udevbe: https://github.com/udevbe/greenfield)

😈

from chrysalisp.

Martyn I think we have come far enough now that this issue is agreed on. :)

from chrysalisp.