NXT Forth compiler

[pepijndevos]

For the RCX there was pbForth, which used a custom firmware afaict. My plan is to write a Forth compiler that compiles to NBC, using the standard/enhanced firmware.

I read this guide, but I'm still thinking about a few problems. http://www.eighty-twenty.org/repos/jone ... nesforth.S I never used pbForth or made a compiler before, so any help is appreciated.

First of all, NBC is a fairly high level assembly, if such a thing exists

It is quite easy to use arrays for the stacks, but there are a few difficulties.

One problem is the immediate mode. It is troublesome to evaluate code on the NXT, even more so because it is not possible to allocate memory on runtime. No doubt someone has tackled the problem of AOTing Forth, but I haven't figured it out yet. It would be cool to have a REPL on the NXT though.

Most Forth implementations use an indirect threaded interpreter. One option is that I use subroutines and forget about it, the other is filling my code with labels. Or is there a reliable way to do the required pointer magic in NBC?

[HaWe]

what are the advantages of this language over NXC or Java?
is it faster, can it allocate memory for code and variables on external memory devices, or has it got pointers and a stack and a heap for dynamic memory allocation or for recursion?

[pepijndevos]

Forth is a (forgotten?) branch of computer languages, the other important one being Lisp. It doesn't use variables and function parameters, but is still a high level language in a way. (the concept of levels is unimportant to a Forth programmer)

It cannot do anything NBC can't do, since I will compile to it. It has a stack and pointers, but no allocation. Tail recursion is just a compiler optimization, so I might add that sometime.

The lack of allocation causes some trouble though. In most Forths, word definitions are added to a linked list, but such a simple data structure is not possible(at runtime). This means I'll either have to define all words at compile time, or use an array instead.

Quoting the software guide:

The program can resize arrays at run-time

How is this done in NBC?

[HaWe]

for recursions you'll need a stack, but the NBC-fw hasn't got a stack.
For dynamic memory allocation by pointers you'll ned a heap, but the NBC-fw has got no heap.
The linked lists during runtime are indeed a big advantage of Forth (or maybe e.g., Prolog IIRC), but maybe pblua is more efficient in that respect.
Data structures cannot become a part of the code in NBC.

It surely will be a big personal challenge and a lot of fun to you to implement Forth, and for this I wish you all the best, but based on the Lego fw it will do only one half of the job, at least I am afraid so.

[pepijndevos]

The thing is that you can implement stacks and heaps with arrays, and since arrays can be resized, you can get pretty creative with that. The question is how, because no such function is listed in the NBC array operations, but the firmware guide says it is possible in multiple occasions.

Meanwhile, I think the solution for one of my problems is obviously to run immediate words on the host. This might mean that these have to be written in another language than Forth. Or maybe I should write the compiler in Forth, for a real challenge.

I also figured you only need this linked list of definitions at compile time. The host can easily support these. A the NBC level, these just become labels or subroutines or whatever.

So what is the problem with recursion in NBC/NXC? What happens if you make a NBC subroutine or a NXC function call itself? Ultimately, you could implement a y-combinator.

I need to look a bit more into how the firmware actually works. It seems that unlike assembly, this firmware has a strong separation between code and data, which makes things complicated.

[afanofosc]

The replace opcode is one option for growing an array in-place. If you replace a single element with an array of elements it will make the array larger before copying into it the values from the replacement array that you passed into the opcode.

You can grow and shrink an array if you don't care about its contents using arrinit. It will replace existing elements with the value you specify.

Most of the other array operations are not in-place. arrbuild requires a separate output array from any of its input arguments. You can't use any of the inputs as the output or it will erase the input values before it adds them to the newly allocated output array. arrsubset lets you create a new smaller array that contains a subset of the values of another array.

You may want to contact Ralph Hempel to see if he can give you some pointers about creating a version of Forth for the NXT. Personally, I think you would be better off making it a custom firmware, like pbLua, rather than trying to build it on top of NBC and the standard NXT firmware. Ralph not only has written pbLua, but he has told me in the past that he has worked on a pbScheme or pbLisp (one of the two, iirc) and perhaps he has investigated creating a pbForth for the NXT.

John Hansen

[pepijndevos]

Thanks for clearing that up. I'll have to check if any of those allow me to make a stack.

I had contact with Ralph, but at some point he stopped responding. What he seems to do is take an ARM compiler and an existing implementation, and then add the API to the language. That is kind of the opposite of what I'm interested in. I want to experiment with writing a compiler, and don't care to much about the API.

Using the standard/enhanced firmware is a big plus for me. That way I can experiment with NXT-G, NBC/NXC and in the future my Forth.

[HaWe]

nxtOSEK has a firmware with all of the C++ functionality, e.g. a stack, a heap, pointers, recursions, use of additional flash memory for code and variables.
Maybe your exit to Eden^^

[tcwan]

Using the standard/enhanced firmware is a big plus for me. That way I can experiment with NXT-G, NBC/NXC and in the future my Forth.

The Enhanced Firmware with NxOS (and I think nxtoSEK) allows you to run stuff from RAM without having to reflash the the firmware with new code each time. Of course you're limited to 64K for everything, but I don't think that is a problem at the start.

If you're interested in using NxOS, I've also developed a USB-based ARM instruction GDB stub which allows you to debug ARM code without having to use JTAG. Info can be found in my post in this thread https://sourceforge.net/apps/phpbb/mind ... f=4&t=1205

[pepijndevos]

Waaaitamoment. How can nxtoSEK and nxOS run on the enhanced firmware, and do all these crazy things, while NBC can't? Does the enhanced firmware allow native code to be executed somehow, or do they just use clever compiler tricks?

I'm going to read the firmware guide from LEGO for a second time to see what it is really capable of.