introducing minimOS

[zuiko21]

Hi all,

So I finally built my first 6502 computer, and now I need software for it... While I blew some EPROMs with specifically tailored firmware to do some fancy (and sometimes useful!) things, there was a clear need for a common platform for future applications to rely on -- especially since I'm involved in some technical stuff that would benefit from a simple but highy controllable computer, but I'm not willing to program it from the scratch every time.

So... here's the answer: minimOS

As the name suggests, it's intended to be a minimalistic Operating System; however, there are three main goals in my mind:

• Scalability: from a simple embedded system to a (almost) full-featured desktop with GUI
• Modularity: the ability to program drivers, kernel extensions and applications without knowing each other; and build the kernel with just the needed features (see above)
• Portability: intended to run on my (let's hope, several) machines but also adaptable to existing 65xx architectures... or even other processors (obviously without binary compatibility then)

After some experimental versions (more of a design stage), current (and first "working") release is 0.4, including the LED-Keypad driver as the sole supported device; will run on SDd and Chihuahua (and also Chihuahua Plus)

I have put the source code (and some binaries) provisionally at http://cjss.sytes.net/lib/minimOS/, although I'll try to set a GitHub account for it ASAP... In the meanwhile, I'll try to explain my file structure:

• kernel_0.4rc.s is the main file, the one to be assembled (I use xa). It has #includes for all the remaining components.
• macros_0.4.h includes automatic implementation of CMOS-only opcodes; if NMOS is selected, they're replaced for equivalent NMOS code fragments. There are some conditional-assembly definitions, but that may change in the future...
• api_0.4.h defines several constants: function names, error codes, port numbers... currently uses preprocessor macros, may be labels in the future.
• zeropage_0.4.h reserves the first three bytes, and the last ones for function parameters, etc. -- currently from $EA, but unfortunately the user space may shrink a bit in more mature versions...
• sysvars_0.4.h are the system globals, from $0200 (unless there's very liitle RAM!). Addresses may change within versions or depending on installed drivers, thus shouldn't be directly accessed by applications. Device drivers may include their own variables after this (for instance, drivers/drv_led.h) because currently they're assembled within the kernel... but I don't rule out the possibility of loading drivers on the fly, will have to devise a method for dinamically assigning variable space for them, though.
• shell.s is the code that gets executed after the POST (boot)... so far is just like a "typewriter" (get char from default input device, put it on default output device, loop forever) for demonstration purposes.

And the folders:

• drivers/ contains both the driver code (like drv_led.s for the LED Keypad) and associate variables (drv_led.h). Also included is a dirty replacement for it, drv_debug.s which uses the bus sniffer as a rudimentary output device for debugging purposes.
• firmware/ contains some code for specific tasks on my machines; nothing to do with the OS, really.
• bin/ has ready-to-blow ROM contents of older versions (and the current release minimOS_0.4rc.bin). They're all 2 KiB, suitable for a 27C16 EPROM.
• to_do/ has source code for future components; this is under development and won't assemble right now.
• old/ (also available inside other folders) has, obviously, older (probably non-working) versions.

You may read on history.txt some scheduled features for the forthcoming 0.4.1 version... but there's much more on the road: filesystem, multitasking...

While this is certainly in a very early stage, it's a start...

[White Flame]

While writing an OS is fun in and of itself, serious thought should be given to what it will take to create decent applications for it.

Memory allocation, supporting banked or other types of expanded memory, and address independence are all fundamental tools you want to get right.

There are also a lot of advantages to including some sort of bytecoded utility language. First, it can save you a lot of memory which is critical in multitasking. Since dealing with an OS and dynamic memory can involve shuffling around a lot of 16-bit pointers, that can cause a lot of code bloat in raw 6502. Also, having things like safepoints for context switches, enforcement of invariants, and using memory "properly" to your OS assumptions can be a lot easier in a more predictable scripting environment. And finally, it's obviously easier to write larger and more complex applications that use your OS APIs from a higher level language.

[GARTHWILSON]

Although you've already made a lot of progress, you might want to check out André Fachat's GeckOS/A65 v.2.0. From the description:

• • OS/A65 is a full-featured Multitasking/Multithreading operating system for the 6502. It is preemptive and implements some Unix-like features, like signals, semaphores, relocatable fileformat, standard library, internet support via a kind of simplified sockets and last but not least virtual consoles.
    
    It is extremly scalable. Stripped down to the scheduler and interrupt handling the kernel is only slightly above 2k. In normal embedded systems the kernel has around 4k, with only application programs running. Full featured systems have a 4k kernel, and several support tasks provided system services like TCP/SLIP and (different) filesystems.
    
    The kernel is almost completely hardware-independent. All the architecture-specific stuff is in a separate subdirectory for each architecture.
    
    The lib6502 as standard library allows easy access to the system services. Parts of this library are already implemented in another 6502 operating system, Lunix by Daniel Dallmann. This way source code compatibility is achieved.
    
    Version 2.0.0 features a "slipd" server process that brings easy internet access to all lib6502 programs, that can now access TCP connections like files. A stable WWW server running on the OS is built into the slipd daemon. Also a remote login can be done. This way the OS can run programs to for example read sensors and write the stuff to files, which are exported by the WWW server.
    
    The relocatable o65 fileformat used by the lib6502 standard library in version 2.0.0 allows more than one instance of a program being run at the same time without interference, even without virtual memory. Also the very same binaries runs on all supported platforms (if they do not use architecture specific stuff, but lib6502 calls only).

I've done multitasking, but not with a true preemptive multitasking OS, so that part is something I cannot speak on with authority.

Quote:

And finally, it's obviously easier to write larger and more complex applications that use your OS APIs from a higher-level language.

And, while still in assembly, the level of the language can be raised a lot with usually no penalty in memory or performance, by using structure macros.

[zuiko21]

White Flame wrote:

While writing an OS is fun in and of itself, serious thought should be given to what it will take to create decent applications for it.

Of course! This is barely one half of the road...

In the current state of things, with a rather inmature API, I haven't written a single application so far but several things are already in the forge: first of all, a musical score editor -- well, sort-of it's best described as a programmable beeper (at PB7) allowing a sequence of musical notes to be typed in and played afterwards. Intended for use with the LED Keypad, it should allow a somewhat more convenient interface if/when a "proper" keyboard and/or screen is available. Playing basics and internal storage format are already sorted out, even the menu system is thought of; now I have to deploy the key-sequence detecting algorithm...

Quote:

Memory allocation, supporting banked or other types of expanded memory, and address independence are all fundamental tools you want to get right.

Certainly. While they aren't currently implemented, bankswitching support (already in a pretty mature design stage) will be included -- two 16 kiB windows selected thru 16-bit registers, up to 1 GiB addressing.

But with a bit of support from the software, some address independence may be achieved: I plan to make the bankswitching routines (for "long" jumps and subroutine calls) specify banks in a relative fashion, while the address within the 16 K window might be absolute (say, $8000-$BFFF). Memory is cheap nowadays, thus a longer-than-16K program would select banks from number 0, no matter where (which bank) the OS placed the code. You have to think on 16K chunks, but not that long ago the x86 architecture had to live thru 64K segments, right?

Another feature would be sequential code auto bank switching: if the (fixed) ROM starts at $C000, the very first instructions on it (unreachable otherwise) would just switch to the next code bank and then jump to $8000, continuing program execution... this will work as long as no opcode crosses a bank boundary, but that could be solved with no more than a couple of NOPs, even automatically by the compiler/assembler -- anyway, I remember (late 80s) having to manually place the directive $SEGMENT on TurboBasic programs thus shouldn't be that much of an issue...

Quote:

There are also a lot of advantages to including some sort of bytecoded utility language
<snip>
And finally, it's obviously easier to write larger and more complex applications that use your OS APIs from a higher level language.

But when you're on limited resources (like any 6502 system, at least compared to current architectures) there's the risk of performance penalty... Anyway, FORTH is intended to be the preferred highish-level language, because of swift performance and, mostly, ease of implementation.

GARTHWILSON wrote:

Although you've already made a lot of progress, you might want to check out André Fachat's GeckOS/A65 v.2.0

Some time ago, when I was gathering information about the 6502 in order to complete my system's design, I ran into this site... I was very encouraged by the fact that I wasn't alone here, but also much impressed by Fachat's system (both hard and soft) since it was a close match for my idea...

...but I want to get my hands real dirty so I need to make my own. Seriously, I haven't looked deeply into GeckOS, but (at least at first glance) it seems to be somewhat Commodore-centric... I have no experience with that brand (never had any 6502 computer before, really!) so it feels somewhat strange to me

Even if I know that porting would be more difficult, I know for sure that I don't want yet another UNIX-like thingy... but anyway, GeckOS (a stunning work, BTW) seems a great source of inspiration, and for the future I don't rule out some interoperability (like making a minimOS-compliant lib6502 module)

Quote:

And, while still in assembly, the level of the language can be raised a lot with usually no penalty in memory or performance, by using structure macros.

Well, that looks very nice, and certainly will be most useful when things get complicated... but after teaching my students about the virtues of structured programming and source code indentation, when I see a piece of paper written by me on assembly, with such streamlined opcodes, I feel something like a wicked relief But will definitely consider them.

In the meanwhile, I want to refine the basic I/O as much as possible, before entering into process and/or memory management, filesystems, etc. Let's hope all these come soon!

[GARTHWILSON]

zuiko21 wrote:

first of all, a musical score editor -- well, sort-of it's best described as a programmable beeper (at PB7) allowing a sequence of musical notes to be typed in and played afterwards

You could do MIDI with a 6551 ACIA, and that would allow you to input the notes from a MIDI keyboard too, and play the music with sounds synthesized by the quality of the MIDI keyboard. (I've done it, but only enough to prove that it can be done, not enough to make significant music.) The MIDI bit rate is 31,250bps, which is not a standard rate option for the 6551 ACIA with a 1.8432 MHz crystal, so to get it, set the 6551 for one-sixteenth of the XTAL clock input rate, and then feed it with 500kHz instead of 1.8432MHz. Get the 500kHz off of a 6522's PB7 from its free-running T1.

Quote:

but I want to get my hands real dirty so I need to make my own.

You could get dirty by re-writing GeckOS for the '816 which gives a lot of benefits for relocatable code, multitasking, multithreading, etc..

Quote:

Quote:

And, while still in assembly, the level of the language can be raised a lot with usually no penalty in memory or performance, by using structure macros.

Well, that looks very nice, and certainly will be most useful when things get complicated... but after teaching my students about the virtues of structured programming and source code indentation, when I see a piece of paper written by me on assembly, with such streamlined opcodes, I feel something like a wicked relief But will definitely consider them.

Part of the purpose of course is to keep the project from getting out of control. The structures make you a lot more productive, and, if you have to put it aside for whatever reason (life happens!), it's much easier to see what you were doing and pick up again where you left off and be productive right away. The structure macros never take away your ability to use the lowest level of assmebly language.

[White Flame]

It's great to hear your goals for this! I'm looking forward to your progress, and I think those are some good memory model ideas.

zuiko21 wrote:

In the meanwhie, I want to refine the basic I/O as much as possible, before entering into process and/or memory management, filesystems, etc. Let's hope all these come soon!

I'd actually recommend working on them in parallel. The way you handle memory management especially can have a great effect on how your basic I/O works, what buffer handling tools are available to it (including how memory could affect blocking vs async), and how it can hand the data to processes. It would suck to come up with some cool I/O ideas, but then come up with better memory management utilities that your I/O stuff couldn't really take advantage of without a rethink or would require extra copying overhead.

Details from all sorts of different places in your system can inspire awesome ideas for other parts. Once you have multiple systems fleshed out, but not super-committed, you can think of space-saving integrated ways of handling multiple features with the same code that might not have been obvious in isolation.

[enso]

Just to introduce a dissenting opinion, OS's are so 90's! Personally I think a Forth-like environment is better then these OS things that just sit there eating up memory and cycles. Before you know it, you will spend 3 years writing a GUI windowing system so that your users can jump through hoops just to do a simple 'hello world' one-liner in a few hundred lines of C!

I like retrocomputing because of the ability to control the machine. Having layers of drivers, ABIs, loaders and linkers and memory managers and things like that just makes me tired.

Just an opinion.

EDIT: The last thing I want is to start a battle. I do like what you are doing. I just don't want to lose the retro feel of our 6502 machines. It is easy, especially with the 16 and 32-bit varieties that seem to invite OS things. But consider that you can either write code for a machine directly (yes, its tedious to 'start from scratch') or an operating system that shields you from hardware details but introduces orders of magnitude more arbitrary restrictions and binds you to software instead. That software has many more bugs, and as it changes, everything breaks, making it impossible to 'stop the insanity'.

In other words, I can boot a C64 in a second and run Gorf without any problems only because there is no OS, just some ROM routines. Getting an old Macintosh or a PC program to work is much harder - try finding that PharLap DOS extender and setting up a modern or an old PC to work with it just right!

I will shut up now.

[White Flame]

But there is an OS on the C64. It interrupts the running user program (if any) all the time, as well as contains a default editor and provides an easy I/O abstraction layer over a number of varying peripherals. You can use the same routines to talk to the serial bus, user port RS232, and even address the keyboard and screen via the same Kernal device APIs!

While I guess it doesn't have a linker (relocating to $0801 on ",0" and setting up memory after the BASIC program really doesn't count), it still has drivers, ABIs, loaders, and memory managers.

[GARTHWILSON]

I'm absolutely not an OS expert-- but I have similar functions (RS-232, display, keypad, etc.) on my workbench computer running Forth, but I would not particularly call it an OS, as any program(s) compiled on top of those functions have full access to everything and can for example set up interrupts and insert, delete, and priority-sort interrupt-service routines as well as alarms for things that need to be done on a schedule, affording a kind of pseudo-multitasking without a multitasking OS. The only real difference between kernel code and user code is that the kernel is in ROM whereas user code gets compiled into RAM. The same dictionary encompasses both. The one thing that would be rather impractical to do is to delete a program that's not the last one compiled, leaving an unused hole in RAM usage, and re-use that RAM space. So far I've never had to do that for my uses. [Edit, 5/15/14: I posted an article on simple methods of doing multitasking without a multitasking OS, at http://wilsonminesco.com/multitask/index.html.]

[enso]

There are many ways to look at it. I would like to avoid a religious fight - we all have opinions.

Consider a small system with some code in ROM. Is it an OS or not? Here are some questions:
-Can you program directly to hardware or do you have to call functions in the 'kernel' or whatever?
-Is there housekeeping, key-scanning, video or whatever, code running 'or else'?
-Can you take over the CPU or is the 'OS' calling you and scheduling your code?
-Is there memory management at least partially outside your control?
-Is the code in ROM (or loaded as part of the 'OS') just helpful routines you can call or ABIs, APIs other A*Is you have to use or else?
-Is the system multitasking in any reasonable way
-Do you have to perform some sort of shutdown?

So as I see it (by no means a fact, just an opinion), a C64, and Apple ][, and Atari 6502 machines have no real OS (thank God), just some helpful routines,maybe interrupt code and conventions - helpful but more or less optional. Amiga, Atari ST, PCs and Macs - definitely an OS. With an OS, it's all or nothing.

Forth is a interesting environment. I don't think a normal Forth is an OS, just some routines you could call and an environment to make it happen. You could write an OS in Forth.

I think an OS is like a government or the courts - it has more to do with restriction and regulation. In that sense, I don't want an OS. An optional set of 'helper' routines and an optional interrupt framework is great.

There is also a temptation to expand infinitely once an OS is in place. Writing 'just in case' code like libraries just leads to not understanding the underlying hardware. In fact, it leads to not understanding the libraries, and writing layers upon layers around simple hardware, slowing everything down and requiring ever-increasing resources and endless 'updates' and rewrites for new versions.

The reason I love retro hardware is simplicity...

[BigEd]

I'd say Acorn's Beebs have an OS, because the services and calling convention were well defined and documented, and it was documented that you weren't supposed to do things yourself, and in a succession of quite different machines and OS versions those programs which behaved accordingly were in fact portable. The services offered were adequate for programs to not need to bypass them.

An OS is an abstraction: unless things in reality are changing, it's hard to judge the value of the abstraction.

But that particular OS does not offer task-switching or memory protection, and does not prevent any random code from running on the CPU. So your questions are useful, but it's not true that all of the conditions need be met. I'm sure you'd agree - it's a fuzzy question overall.

Cheers
Ed

[enso]

I didn't mean that list as a list of required conditions for an OS, just some questions to ask.

It is a completely subjective decision, especially in smaller and minimal cases. It's easy to agree that Linux or Windows an OS. In retromachines, it's not that simple, especially if you are not required to follow the rules.

[GARTHWILSON]

The Wikipedia article on operating systems does support the idea of it being kind of fuzzy. It does however say,

Quote:

All user software needs to go through the operating system in order to use any of the hardware, whether it be as simple as a mouse or keyboard or as complex as an Internet component.

which I'm not sure I can fully agree with, although I am super far from being an expert in OSs like maybe Samuel Falvo or Toshi Morita. It was kind of fun to read near the bottom,

Quote:

In some cases, hobby development is in support of a "homebrew" computing device, for example, a simple single-board computer powered by a 6502 microprocessor. Or, development may be for an architecture already in widespread use. Operating system development may come from entirely new concepts, or may commence by modeling an existing operating system. In either case, the hobbyist is his/her own developer, or may interact with a small and sometimes unstructured group of individuals who have like interests.

We've had various discussions on OSs, including in the 65Org32 topic ("Improving the 6502, some ideas"), and I always find them interesting, but at the same time, my workbench applications absolutely cannot tolerate the overhead of having the user program go through the OS to access hardware when things have to happen on microsecond intervals. In other applications there does need to be a way to prevent problems like priority inversions though.

There is obviously a legitimate place for both going with an OS and without one. "With" is not always the better way.

[BigEd]

Yes, let's agree that Wikipedia has that wrong. An OS is some software which provides abstractions or marshals resources for some other software, which we call user programs or application programs. There might be no essential feature that you can say that every OS has to have!

Agreed, it's fuzzy and it's subjective.

Cheers
Ed

[enso]

That quote is quite amusing.

I think it's a good idea to give the user as much flexibility as possible. It's nice to write directly to the hardware with helpful routines in ROM. It's nice to not have to rewrite the obvious - keyboard scanning, disk saving routines etc. It's nice to be able to use pieces as you see fit and not have to buy into someone's idea. A monolith of an OS that completely blocks your way around the system is less nice, as I see it.

For a system with a variety of possible hardware it's nice to abstract away the hardware to make the code easier to port. For a homemade one-off system abstraction is less important. For a large team effort, defining clear interfaces between each other's code is sensible. For a one-person effort it's probably silly to define layers of APIs.

It's easy to assume that successful companies are making sensible decisions and follow their lead. I remember reading a lot of books and thinking that the corporate software-development models made sense. Then I thought about it more, and decided that it's not so good at all. An OS to me is an encapsulation of techniques to lock in the customer base, bringing very little value to the end-user.

Even an open OS like Linux is a strange idea - competing with Windows and Apple using the same bullying techniques for no gain at all. It's like a bunch of hippies building themselves a jail while singing songs of freedom.