Building the "Perfect" Programming Language

[bdk6]

This thread has already spawned one other thread and now has prompted me to start this one. Although discussing religion and politics at the same time is probably safer than "best" programming languages, I will nevertheless dive right in.
Here is the short story. I have been working on a good language for beginners for some time that would be easy to learn ( like BASIC) yet "modern", structured, "fast", and powerful. The idea came about in discussions about Arduino and the suitability of the Arduino mix of C and C++ for the typical Arduino user (short story, it's a bad idea.) With my recent re-entry to the VIC 20, 6502 world, and the previous linked discussion, I've found a new interest in that project or something similar targeted primarily at the 6502. What I have done up to now is on my web site.
My question to all you 6502 gurus is this: based on what I have done so far, what would YOU want in a general purpose, efficient, simple yet powerful language for use on your vintage 6502 equipment? Would it be useful to you? Would you use it? What are the most important attributes? What would you NOT want it do have/do? Cross compiler, resident compiler, or something else? Or would you want something completely different from what I'm thinking?
Keep in mind that should I go forward with this it will most likely be based on work I've already done, sprinkled with ideas given here that fit my original vision. So it likely won't be anyone's "perfect" language other than perhaps mine. I'm sure many of you have plenty of ideas that could be useful. I ain't too smart, but I can follow smart people!

[BigEd]

(I'm sure you won't get any kind of majority view here - so whatever interesting views you do get, you'll do best (I think) if you continue to follow your own vision.)

In our parallel universe nearby where we're exploring new CPU designs within a one-page coding constraint, one of the tracks we're following is a BCPL compiler. That's a predecessor of C, well-suited to word-based machines, and not especially well-suited to 6502 which is why you'd usually see it as a VM implementation rather than native. In BCPL there's one data type: the word. Also arrays. (But we're nowhere near a self-hosting BCPL, and I think self-hosting is very attractive. There is VSPL which might be a good target for self-hosting.)

You're more on the Algol track - more types available - and that seems very attractive to me, and we know there were Algol compilers for machines as small as 8k words in the distant past.

With Basic and C, there's a big advantage in that people won't have a new language to learn. Personally I'm interested in the more general case - an effective language that's easy to learn, efficient to target our favourite small machines with native code output, and preferably self-hosted. Even better of course if the language is also bootstrapped - written in itself!

[GARTHWILSON]

<thumbs up>

BigEd wrote:

(I'm sure you won't get any kind of majority view here - so whatever interesting views you do get, you'll do best (I think) if you continue to follow your own vision.)

Agreed. Our brains don't all work the same way, and what is the "perfect" programming language for one person, won't be for the the next.

Quote:

With Basic and C, there's a big advantage in that people won't have a new language to learn. Personally I'm interested in the more general case - an effective language that's easy to learn, efficient to target our favourite small machines with native code output, and preferably self-hosted. Even better of course if the language is also bootstrapped - written in itself!

Again I'll add my hearty agreement, especially about the self-hosting idea.

BASIC gets a lot of bad rap because there have been so many "toy" BASICs out there giving it a bad name. At the other end of the range, there have also been some excellent, professional-quality ones. And a lot in between. I've used four different ones myself, spanning the range. Although I'm not particularly a huge BASIC fan, I would like to see Lee's very good EhBASIC combined with all the best of other basics to make a really outstanding BASIC for the '816.

[bdk6]

BigEd wrote:

(I'm sure you won't get any kind of majority view here - so whatever interesting views you do get, you'll do best (I think) if you continue to follow your own vision.)

Following my own "vision" is certainly the intent. If you design a language by committee you end up with horrors like PL/1 or Algol 68 or Visual Basic. The main ideas that I'm looking for consensus on (or at least a few viewpoints) are cross vs self hosted, native vs vm, and should it support features (like recursion) that make it more convenient but less efficient. Right now, I expect the language will look much as the version I linked to. I've spent a lot of time on that and mostly like what I've come up with. I will have to review it in the new context of 6502 which wasn't considered at the time.

BigEd wrote:

You're more on the Algol track - more types available - and that seems very attractive to me, and we know there were Algol compilers for machines as small as 8k words in the distant past.

Yes, more data types is definitely a plus and a requirement. I toyed with the idea of something like "B" (successor to BCPL and predecessor of C) for simplicity, but on an 8 bit machine (especially 6502) you certainly want to have a byte-sized type but you need at least a 16 bit type to do much useful work. As for BCPL my opinion is that's it's close enough to C you might as well do C if only for familiarity but not using C was what started the project
Thanks a lot for your input. It's very helpful.

[bdk6]

GARTHWILSON wrote:

Agreed. Our brains don't all work the same way, and what is the "perfect" programming language for one person, won't be for the the next.

Absolutely. it won't even be perfect for me! Hence the quotes in the title. I'm already mostly decided on what the language should look like (somewhat like a mix betwen BASIC, Pascal, and C) but trying to fine-tune it for use on 6502.

GARTHWILSON wrote:

Again I'll add my hearty agreement, especially about the self-hosting idea.

BASIC gets a lot of bad rap because there have been so many "toy" BASICs out there giving it a bad name. At the other end of the range, there have also been some excellent, professional-quality ones. And a lot in between. I've used four different ones myself, spanning the range. Although I'm not particularly a huge BASIC fan, I would like to see Lee's very good EhBASIC combined with all the best of other basics to make a really outstanding BASIC for the '816.

The self-hosting vs cross-compiled was a big question I had and looking for opinions on. That isn't really a language issue, but an implementation issue. However, it will have an effect on the language itself because self-hosting will make it much more difficult to do some things. Gigahertz processors with Gigabytes of RAM and really fast mass storage can simply do things that the 8 bit machines can't.
I'm not really a fan of BASIC either. But it has some nice features and a LOT of us first learned on it. I'm trying to capture the "flavor" of it with a "better" language.

I appreciate your input. This is exactly the type of stuff I'm looking for.

[BitWise]

I've been looking at BCPL and Modula2 as potential bootstrapping languages for 6502/65816 systems for a while. C has a few grammatical issues that make it hard to implement without a full LALR based parser unless you want to go the Small C route.

Porting the CINTCODE interpreter is an easy way to get BCPL working quickly but its internal word based addressing makes memory access slow with every address having to be scaled by 2 or 4. Not such a problem on ARM or x86 with 'base + scaled index' addressing mode instructions but it does not suit the 65 family very well.

My big problem is time -- I don't have enough of it with a full time job and a couple of spare time ones as well.

[BigEd]

I would guess that it's a good idea to include recursion and local variables, partly from a perspective that this is good way to improve structure and maintainability of some kinds of programs, and partly because it might help your language look attractive to someone experienced intending to help someone else learn to program, and that increases your potential user base.

[bdk6]

BigEd wrote:

I would guess that it's a good idea to include recursion and local variables, partly from a perspective that this is good way to improve structure and maintainability of some kinds of programs, and partly because it might help your language look attractive to someone experienced intending to help someone else learn to program, and that increases your potential user base.

In my personal opinion, local variables are a must. That's probably my biggest complaint with most BASICs. Recursion is expensive, and typically not that useful. Although for self-hosting and writing the compiler in the language itself, recursive descent parsing is VERY useful. (Even GCC has switched to mostly recursive descent parsing.) So I think that making it possible but not default is a good idea. That is something I need to look into.
Having the largest possible user base is secondary to making it useful for writing "real" programs. By that I suppose I mean could you write a decent game or a good application. The COWGOL language (posted just after my original post!) looks interesting, but the goals are much different. And I wouldn't consider it a beginner's language.
One of the biggest problems I see is getting efficient code on a self-hosted compiler. A big machine has lots of resources for creating good code that just aren't available (in reasonable space and time) on small machines. Right now I'm leaning toward something like subroutine threaded code that will give reasonable speed and small code size, with an option to stick some assembly code in for critical stuff.
EDIT: In some dreamland in an orthogonal universe, I would have time to write TWO compilers for the same language: one self-hosted and one cross-compiled from a large machine. The self-hosted could be used for piddling and quick development and the cross compiler could be used to get the best code for "release." But that's a crazy dream.

[bdk6]

BitWise wrote:

My big problem is time -- I don't have enough of it with a full time job and a couple of spare time ones as well.

"All my possessions for a moment of time." -- Elizabeth I

[GARTHWILSON]

bdk6 wrote:

In my personal opinion, local variables are a must. That's probably my biggest complaint with most BASICs.

One of the best BASICs I've used, that on my hand-held Hewlett-Packard HP-71B from the mid-1980's, does allow local variables and environments.  You can have lots of program files in memory at the same time, even active, and any one of them can call subprograms in itself or any other other BASIC file, or even itself, up to the limit of memory, meaning you can have recursion.  The local variables and environments are rather necessary for making sure one program doesn't step on variables and I/O assignments etc. that may still be in use by other ones that are pending.

When I followed up on Ed's comment about self-hosting, I had in mind that the initial system would be formed on a bigger machine, but after that, it would be independent of the bigger machine to develop new software to run under it.

[BigEd]

I suppose I'm thinking of this progression:

For execution of the user's program:
- an interpreter of a HLL
- a VM, interpreting a dense code like a p-code or bytecode
- some kind of threaded code
- native code like an expert might have written in assembly

For compilation of the users' program:
- compiled on a modern PC
- compiled on the 6502 machine itself

Substitute the compiler for the user's program, and you see that one level would be a native-code 6502 compiler which was unable to compile itself on the 6502 due to capacity, and another level would be one which is not only written in itself and produces native code, but also has the capacity to compile itself on the target machine.

Put another way, first question is whether a user can do development entirely on the 6502 machine, and the second question is whether the compiler writer can do compiler development entirely on the 6502 machine. That's a higher bar.

Not that this matters in a deep sense. Can we write code? Do we get acceptable performance? That's what really matters.

I think some of the commentary in the thread about C is really commentary about the gap between the code a compiler produces and the code a skilled assembly language programmer would produce, if they tackled the same program. I think the road to a happier life is not to worry so much about that gap. Worry about whether the code is too big to fit in the machine, or too slow to wait for the answer.

[whartung]

For something to be self hosted requires a lot of infrastructure, specifically it requires mass storage and a way to access it.

Since something like CP/M never took root in the 6502 world, there's no common denominator for a basic system based on the 6502. If you were to build a bare bones Z80 system with mass storage today, even as a hobbyist, it would run CP/M (or a derivative) and open the door to a host of legacy software. If you were to build a 6502 machine today, you wouldn't have anything. You'd be an island in terms of a self hosting system. Hardware with no software.

So, is it going to be self-hosted on a C64? An Atari? An Apple? At some level it shouldn't matter, but there's a baseline (for lack of a better term) "BIOS" that the underlying code needs to assume.

Next, the problem with simple compiler is crummy code generation. We all know this. The problem with a 6502 hosted solution is that it's simply limited to simple compilers. While CPU speed is certainly a barrier to a "sufficiently smart compiler", it turns out that RAM is the more precious resource. A 6502 is starved for both.

Action! was unique. It was a cartridge based system, it compiled from RAM to RAM, it was a simple language that manage to skip over the problems of dealing with call frames and what not (it simply didn't have them). If things got locked up, a quick RESET would preserve the editor content and warm start the environment. Fast system with fast code and fast turnaround.

I honestly don't know how good the compiler was, I don't even know if it did mundane things such as constant folding etc. A single pass compiler is limited to what optimizations it can do.

On the Atari, there was a C65 system. I used it once. It was terrible. It was a classic implementation, very disk bound. It was glacial. There was a reason much development on 6502 machines were done with cross compilers on larger hosts.

Obviously a '816 based system, i.e. one that's actually got memory rather than just a faster 6502 with a better instruction set, has more resources available to make a better compiler. But, again, there's no common baseline common OS to host on a '816 SBC.

A Forth can clearly be self-hosted via a meta compiler.

UCSD was really novel this way, as it was self-hosting. They had the entire tool chain: compiler, assembler, linkers, editor. It's one reason I find still find the p-system compelling on small machines today. You could also create code for another machine, if you had a mechanism to move the files. I think an expanded P-Machine on a '816 SBC would be pretty neat. It wouldn't necessarily access all of the MB's ram a '816 can get (at least not "natively"), but you could offer up 64K of heap, 64K of code space, 64K of code cache, and the VM. You could easily use the excess RAM for a disk cache.

Then you could use it to make the environment less horrible -- ca. 1978 user interfaces...it's kind of nasty, frankly .

[rwiker]

Plasma looks like it may be a good choice for a high(ish) level language on a 6502: reasonably high level, tolerable performance and has an interactive environment that runs on the 6502. (Note: I haven't actually got round to trying out Plasma yet.)

Among the various BASIC dialects, surely BBC Basic should be mentioned: it's fast, has some high-level language constructs, and has a built-in assembler. Since the source code for BBC Basic is available as a BBC Basic program (actually, assembler code using the built-in assembler), it is also demonstrably self-hosting.

[BigEd]

An amusing point! I think you need a serious machine or special tactics to assemble a 16k ROM on a 6502 system: even 64k RAM is going to be tight. The source file I found is 200k which is big for a floppy, let alone for loading into RAM...

It's true, the various 6502 camps are somewhat mutually isolated. In the land of Acorn there's a healthy story of an MOS with a filesystem abstraction. There's something similar in the land of Commodore - which means we have the additional problem of more than one standard to choose from. Presumably other camps have their own too. Perhaps some of the homebrew systems support FAT filesystems, but must have to cook up their own API for file access.

I suppose each of us will do whatever seems comfortable: build on our personal favourite existing scheme, or roll our own. Or just use serial I/O with no idea of a filesystem at all, which is OK for some purposes.

[whartung]

BigEd wrote:

An amusing point! I think you need a serious machine or special tactics to assemble a 16k ROM on a 6502 system: even 64k RAM is going to be tight. The source file I found is 200k which is big for a floppy, let alone for loading into RAM...

Heh. The Atari floppies were 80K back in the day.

Memory has been the "scarce" resource since day 1, so it's not untoward to be able to stream a file from storage, and assemble it (or whatever), and write the the artifacts back out to disk, rinse and repeat for the different phases of the operations. But, especially on the Atari (notably in reference to the C65 comment above), the disk is just flat slow and awful to make it essentially unusable for "work". it's fine as a fancy cassette drive to load and save programs, but part and parcel to actual work...no. Just too slow.

It was interfaced at 19.2K baud, so it would take ~40s to fill the disk. Just takes forever. Now, I had a build take 10 minutes yesterday, but that was 320K lines of code, and that including downloading half the internet.

Quote:

It's true, the various 6502 camps are somewhat mutually isolated. In the land of Acorn there's a healthy story of an MOS with a filesystem abstraction. There's something similar in the land of Commodore - which means we have the additional problem of more than one standard to choose from. Presumably other camps have their own too. Perhaps some of the homebrew systems support FAT filesystems, but must have to cook up their own API for file access.

On the Atari, we had "CIO", for its central I/O system. Device identifiers, pluggable drivers, stable, common interface, the works.

If you opened "S:" and started writing text to it, you'd show text on the screen. "P:", the printer, "C:" the cassette, and "D:FILE.TXT", the disk drive. Same code, just the file/device spec changed. It would be a non-issue from a program point of view to mount a DOS file system on to an Atari. Have potential issues dealing with directories and what not, but you'd be able to read and write a file. The code wouldn't know the difference. Just replace the "D" driver and let fly.

Device drivers were a first class concept in the system ROM. You could create your own devices (X: if you wanted, for example), or override the built in devices.

I never really had an idea how it worked on the Commodore. I know (I think) that the disk was Device 8, but beyond that, I'm not sure how things like drivers and such were handled there. I don't even know if the Apple had anything close to what the Atari (or even the Commodore) had. Seems like it was just a bunch of well know addresses that it relied on, rather than a first class abstraction.