6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Sun May 05, 2024 9:11 am

All times are UTC




Post new topic Reply to topic  [ 264 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 18  Next
Author Message
PostPosted: Thu Apr 10, 2014 7:06 am 
Offline

Joined: Tue Jul 24, 2012 2:27 am
Posts: 672
BigDumbDinosaur wrote:
It shouldn't take millions of lines of code to control mass storage and I/O, schedule processes, keep time and manage memory. UNIX did all that and then some 30 years ago. Where did everything go wrong?


Ah, but it does take millions of lines to make it fast, configurable, robustly handle errors, secure, report status properly, and work with any number of hardware combinations.

Things like caching systems and analyzing actions for predictive behavior add significant footprint to a program, but end up making it run much faster. I've heard some statistic that 80% of modern source code is error management. And from personal experience, security, status reporting, configurability, etc are things that are incredibly difficult to insert into code after the fact; they have significant structural requirements if you want to get them right.



GARTHWILSON wrote:
Many here know far more about operating systems than I; but where is the dividing line between the kernel and the GUI that accompanies it for example?


If the GUI is how the user is expected to launch programs (not just application-specific sub-processes), then I'd consider it part of the OS. I wouldn't consider it part of the kernel, unless the low-level kernel graphics support is explicitly associated with supporting that particular GUI. Exposing a framebuffer or 2d/3d accelerators does not mean the particular running GUI is part of the kernel, to me.

_________________
WFDis Interactive 6502 Disassembler
AcheronVM: A Reconfigurable 16-bit Virtual CPU for the 6502 Microprocessor


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 12:35 pm 
Offline

Joined: Sun Apr 10, 2011 8:29 am
Posts: 597
Location: Norway/Japan
The GUI is not part of the kernel in Linux (unlike some Windows versions), although there are some hooks in the kernel to support X11.

As for the horrible amounts of RAM now needed - it isn't really the OS itself which requires that. I'm sitting here on a very ancient laptop computer with very little RAM compared to almost anything now (this Android tablet I'm writing this on has much more RAM). The laptop has supported me for software development for a decade, and I inherited it from someone who inherited it from someone else..
No, the RAM requirement comes from things like Firefox. It used to be no problem running lots of Firefox windows on this machine. Now I have to regularly kill Firefox and restart, to release memory (kill, because then Firefox can restore the pages I had running), with much less memory footprint until it's bloated itself up again. Serious memory leaks, I suppose. Been like that for a long time now. I've tried another browser with much less footprint, Midori, it works really well (better than Chrome on gmail.. imagine that), but it crashes sometimes so it's back to Firefox. Sigh.

But the Linux kernel and OS itself runs fine on this machine. Many years back I got a full X11 system working without swapping on 8MB 386 machines (I was helping out at an Italian company and 8MB 386 machines were what they had got). I doubt 8MB is still possible, but I bet I can get a modern Linux system running on 32MB or less. Just not with any Firefox or similar bloated graphical tools.

My old Nokia N800 internet tablet runs basically a full Debian Linux with 128MB RAM.. only the windowing environment is different (called Hildon). I can have 15 browser windows w/graphics open, no problem. My N900 phone is also a full Linux system, although it has 256MB RAM.

Of course, all this is still vastly different from the 8/16-bit era. or the original Unix which ran on PDP.

But then there are other, valid reasons for needing RAM.. data size. I process satellite imagery data. this requires huge amount of RAM (or you would be swapping and paging for days). The datasets are from 2GB up to (more commonly) 20-30GB, sometimes more. It may require sophisticated decompressing of data embedded inside these huge datasets. There's simply no way to get around that without RAM. My old laptop can't.. which is why I also carry a small notebook, mostly useless to work with but I run 64-bit Linux and it has lots of RAM (and disk).

-Tor


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 2:01 pm 
Offline
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10797
Location: England
The STEPS research project ran for about 5 years had a goal of making a productive GUI environment, from drivers to OS to applications, in 20 thousand lines of code. It has released various interim reports but not a final report. Some code is available but not final code. I think they succeeded in showing that the goal was not outrageous. They use dynamic and domain-specific languages to achieve the goal - there may be philosophical similarities to the approach used by a Forth expert.

See for example
http://www.vpri.org/pdf/tr2013002_KSonward.pdf (especially Table 1)
http://www.vpri.org/pdf/tr2011004_steps11.pdf

Cheers
Ed


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 5:00 pm 
Offline
User avatar

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8432
Location: Southern California
Quote:
No, the RAM requirement comes from things like Firefox.

You mean you don't like the fact that I typically have 60-80 tabs on Firefox with Linux? :lol: I do have 2.5GB of RAM on this cheap 8-year-old PC with an AMD Sempron though. I mostly only re-start Firefox when an update requires it, but I have to re-start my Thunderbird email client maybe every few weeks because something seems to gets messed up in its task scheduling and it gets to where the effects of key presses get delayed for as much as a few seconds, while there's no problem with other programs.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 5:22 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8173
Location: Midwestern USA
GARTHWILSON wrote:
Many here know far more about operating systems than I; but where is the dividing line between the kernel and the GUI that accompanies it for example? My DOS-based CAD is point-and-click, and obviously graphics, but there's no GUI in the OS itself. There was minor GUI when I was running GEOS under DOS. On my Linux machine, I use audio- and photo-editing software, streaming video, etc. which are amazingly complex yet not part of the OS. The GUI that came with the kernel does have things like thumbnail photo previews, and the mp3 files' icons, not to be outdone, start playing the music if you mouse over them (since it's not very helpful if they all play at once when you're looking at the folder's contents). It does not seem that the OS has any business being as big as modern ones. But where is the line between the kernel and the GUI? is the GUI part of the OS? Do programs that are very graphics-oriented particularly need a GUI that's external to the program? For desktop applications, we enjoy a lot of things that won't come simply; yet I'm not convinced that the way it's done is the way it should be done. Further, it is inexcusable that modern OSs want the amount of RAM and disc space that they do.

The Linux GUI runs as a user space application above the kernel, meaning it's not part of the kernel. All versions of Windows prior to Windows 95 ran the GUI as an application on top of DOS. Stability was a (big) problem because DOS has no concept of memory protection, preemption, etc.

Linux uses loadable modules to support hardware discovered during boot. This is different from, say, SCO UNIX, the latter which runs a monolithic kernel with the device drivers statically linked in. SCO also also uses the concept of boot-time loaded drivers (BTLD) to add support for new hardware that doesn't have a device driver integrated into the kernel—the kernel is actually relinked on the fly prior to being started. SCO UNIX kernels tend to be relatively compact, since they usually didn't carry around anything that isn't needed to run the target machine. If fact, back when we sold and support SCO on our servers, we usually unlinked all device drivers that weren't pertinent to the installed hardware, a step that could significantly shrink the kernel's size and memory footprint.

Unfortunately, a fair amount of complication exists in modern operating systems due to networking and especially in the case of x86 machines, the myriad combinations of hardware that may be encountered. That has added quite a bit of bloat. Adding more bloat is the fact that networking opens the doors to security intrusions that only the kernel can efficiently prevent, something that Microsoft was slow in realizing with Windows.

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 5:36 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8173
Location: Midwestern USA
White Flame wrote:
BigDumbDinosaur wrote:
It shouldn't take millions of lines of code to control mass storage and I/O, schedule processes, keep time and manage memory. UNIX did all that and then some 30 years ago. Where did everything go wrong?

Ah, but it does take millions of lines to make it fast, configurable, robustly handle errors, secure, report status properly, and work with any number of hardware combinations.

Things like caching systems and analyzing actions for predictive behavior add significant footprint to a program, but end up making it run much faster. I've heard some statistic that 80% of modern source code is error management. And from personal experience, security, status reporting, configurability, etc are things that are incredibly difficult to insert into code after the fact; they have significant structural requirements if you want to get them right.

I'd have to disagree, and I do so from many years of experience in the UNIX and Linux world. Kernel bloat is a relatively recent phenomenon that seems to have coincided with the infusion of object oriented programming philosophies into operating system design (thanks a lot, Plauger!). Creeping featurism and sloppy coding are also causes. The latter is often a byproduct of using languages like C++, which tend to be inefficient and result in the computer analog of using a library to say the cow jumped over the moon.

Quote:
GARTHWILSON wrote:
Many here know far more about operating systems than I; but where is the dividing line between the kernel and the GUI that accompanies it for example?

If the GUI is how the user is expected to launch programs (not just application-specific sub-processes), then I'd consider it part of the OS.

I wouldn't say that just because programs are being started from the GUI. Saying otherwise would imply that BASH is part of the Linux operating system, which it isn't—it runs as a user-space application. Now, in the Windows world, all bets are off, since Microsoft stupidly decided to more-or-less integrate the GUI into the kernel.

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Thu Apr 10, 2014 10:20 pm 
Offline

Joined: Tue Jul 24, 2012 2:27 am
Posts: 672
Well, "kernel bloat" and the overall memory footprint of the OS can be considered different things. I'm haven't been inside the kernel in a long time.

As far as choice of language goes, I think using things like C++ yields a much larger and more unwieldy source code size, but probably similar or smaller runtime memory footprint than many other options, especially dynamic languages. The latter often use large headers on allocated objects and tend to have much larger runtime support binaries. Any good JIT language (ie, dynamic runtime optimization & recompliation) will of course also add the memory overhead of the compiler, storage for heuristics, and holding on to at least two representations of the code. So C++ is likely a positive for smaller memory footprint, but a failure in terms of bugs & security (something Forth would fail at as well), and often a wash on speed nowadays (manual cache alignment notwithstanding).


I'd technically agree that BASH isn't part of Linux, since Linux is just the kernel. But it is a part of the Ubuntu, RedHat, Slackware, etc OSes. It's bundled in and performs for the user the fundamental computer-user interaction of launching programs. That's sort of like saying the Windows Control Panel isn't part of the OS, because it's just another launched program. Sure, it's not part of the kernel (I think?) but it's still part of the OS.


And yes, the web browsers are by far the biggest memory sinks on my system. Quassel (the IRC client I recently started using) is terrible at holding on to EVERYTHING in memory as well, crawling up to 1GB usage often.

_________________
WFDis Interactive 6502 Disassembler
AcheronVM: A Reconfigurable 16-bit Virtual CPU for the 6502 Microprocessor


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 11, 2014 1:39 am 
Offline

Joined: Sun Apr 10, 2011 8:29 am
Posts: 597
Location: Norway/Japan
GARTHWILSON wrote:
Quote:
No, the RAM requirement comes from things like Firefox.

You mean you don't like the fact that I typically have 60-80 tabs on Firefox with Linux? :lol: I do have 2.5GB of RAM on this cheap 8-year-old PC with an AMD Sempron though. I mostly only re-start Firefox when an update requires it, but I have to re-start my Thunderbird email client maybe every few weeks because something seems to gets messed up in its task scheduling and it gets to where the effects of key presses get delayed for as much as a few seconds, while there's no problem with other programs.

My desktop PC back at the office has 16GB of RAM now (a million times more than the first IBM PC model 5150 announced at $1600 in 1981), Firefox still manages to make the system start swapping. I have lots of stuff open on that one though, out of necessity. I work on lots of different things, but with sometimes days or weeks or even months where I do other things. Many, many things. So to be able to switch context I keep the context.. in the browser windows. I now wish for a box with 196GB RAM, there's one system at work that can take that amount, and a guy in another place has a box with 256GB RAM (he does Altera FPGA progamming on Windows though).
One problem (and it affects Chrome the same way) is apparently that a lot of Javascript stuff can keep eating memory. I don't know much about that, but some folks over on another forum tries to run a compiler in a browser and ran into that issue. All I know is that the memory goes up, and up, and up, until I kill and restart. The it reloads all but with a more sensible memory footprint. Yes I have to use the same procedure on the 16GB box as on this old laptop.. just on another scale.

Ah, Thunderbird.. that one definitely has a memory leak. Don't know which versions but it seems the newer ones aren't fixed. I simply can't use it on the laptop anymore. Sylpheed works better. In fact the Android tablet works best for email, with K9.

-Tor


Top
 Profile  
Reply with quote  
PostPosted: Sun Apr 13, 2014 7:40 pm 
Offline

Joined: Mon Jan 07, 2013 2:42 pm
Posts: 576
Location: Just outside Berlin, Germany
Okay, I'll have to explain the joke for the English speakers, but first the original quotes for the Germans here: After hours of painful debugging I finally get the most primitive version of the DO/LOOP to work (stupid bug in POSTPONE - calling LITERAL instead of (LITERAL) - combined with forgetting POSTPONE for UNLOOP in the definition of LOOP). Startled by my cry of triumph, my wife looks up from her book.

Ich kann jetzt Schleifen machen! I say proudly ("I can do loops now").
Das konnte Dein Sohn schon mit sechs Jahren, she says dryly. ("Your son could already do that when he was six years old.")

The joke here is that die Schleife is not only the loop in a computer program, but how you tie your shoe laces (eine Schuhschleife binden). I tell y'all, it's hard to get respect as a programmer around here :wink: ...


Top
 Profile  
Reply with quote  
PostPosted: Sun Apr 13, 2014 9:45 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8173
Location: Midwestern USA
scotws wrote:
Okay, I'll have to explain the joke for the English speakers, but first the original quotes for the Germans here: After hours of painful debugging I finally get the most primitive version of the DO/LOOP to work (stupid bug in POSTPONE - calling LITERAL instead of (LITERAL) - combined with forgetting POSTPONE for UNLOOP in the definition of LOOP). Startled by my cry of triumph, my wife looks up from her book.

Ich kann jetzt Schleifen machen! I say proudly ("I can do loops now").
Das konnte Dein Sohn schon mit sechs Jahren, she says dryly. ("Your son could already do that when he was six years old.")

The joke here is that die Schleife is not only the loop in a computer program, but how you tie your shoe laces (eine Schuhschleife binden). I tell y'all, it's hard to get respect as a programmer around here :wink: ...

Leave it to your wife to be a killjoy! :lol:

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Mon Apr 14, 2014 4:54 am 
Offline
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10797
Location: England
Lest BDD's casual stereotyping should appear normal and acceptable, I offer my own case: my other half is a more accomplished software engineer than I am, and is entirely supportive of my hobby. Looks like she'll earn more than I will this year too, which is fine.

Just because we have an interest in a micro from the 70s doesn't mean we need to espouse the values of the 50s.

Cheers
Ed


Top
 Profile  
Reply with quote  
PostPosted: Mon Apr 14, 2014 6:01 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8173
Location: Midwestern USA
BigEd wrote:
Lest BDD's casual stereotyping should appear normal and acceptable, I offer my own case: my other half is a more accomplished software engineer than I am, and is entirely supportive of my hobby. Looks like she'll earn more than I will this year too, which is fine.

Just because we have an interest in a micro from the 70s doesn't mean we need to espouse the values of the 50s.

Cheers
Ed

Huh?

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 25, 2014 6:11 am 
Offline

Joined: Sat Aug 21, 2010 7:52 am
Posts: 231
Location: Arlington VA
scotws wrote:
A note on the dictionary structure, which, er, is slightly different that other Forths...
This got me thinking, what if the dictionary structure were optimized for memory? Say, for a ROMmable Forth. The Name Length, Name and Link Fields aren't needed by the final executable binary. These symbols and links could be loaded temporarily into RAM for compiling new code, and removed from the binary target. I'm sure someone has done this before, but who, and what else must be considered with this approach?


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 25, 2014 6:47 am 
Offline
User avatar

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8432
Location: Southern California
That's compiling headerless code. It's fairly common, and I've done it with a metacompiler and with an assembler, but never on the same computer that will run the resulting code.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 25, 2014 1:29 pm 
Offline

Joined: Sat Aug 21, 2010 7:52 am
Posts: 231
Location: Arlington VA
Right now my dictionary is straight Fig with the exceptions of the CFA being replaced by executable 6502 code (e.g. jsr enter for a secondary), and it's multiple linked lists not just one.

I was a little bummed when I got CREATE working and couldn't do something like
: {spadecharacter} ;
because the graphics have the bit7 set and that makes them look like the final character of the word name.

Separating the NFA and LFA from the rest of the dictionary, could look like this:
Code:
BEFORE 
PETTIL dictionary word structure
+---------------+
|   link[low]   |  +0      Link field
|   link[high]  |
+-+-+-+-+-+-+-+-+
[1]   [   len   ]  +2      Name field length & flags
| [I]           |          Immediate bit
|   [S]         |          Smudge bit
+-+-+-+-+-+-+-+-+
|0| name[1]     |  +3      Name field
|0|         ... |
|1| name[len]   |
+-+-------------+
| code[1]       |  +4+len  Code field
|           ... |          (always executable code)
| code[n]       |
+---------------+
| data[1]       |          Parameter field
|           ... |
| data[n]       |   
+---------------+
Code:
AFTER (split dictionary)
PETTIL dictionary word structure
+---------------+
|   addr[low]   |  +0      Code field pointer
|   addr[high]  |          points to code[1]
+-+-+-+-+-+-+-+-+
|     [   len   ]  +2      Name field length & flags
[I]             |          Immediate bit
| [S]           |          Smudge bit
+-+-+-+-+-+-+-+-+
|   name[1]     |  +3      Name field
|           ... |
|   name[len]   |
+---------------+

Immediate and Smudge bit move to the left. 
BIT7 of length =1 and BIT7 of Name[last] =1 are both gone

+---------------+
| code[1]       |          Code field address
|           ... |          (always executable code)
| code[n]       |
+---------------+
| data[1]       |          Parameter field
|           ... |
| data[n]       |   
+---------------+
With headerless code, some possibilities suddenly emerge.
    The Name field no longer uses bit7 of the text portion, so names could use PETSCII graphics
    There's no LFA and NFA in the way.
    The code area is just code, so falling through to the next word wouild work, and other intertwingling
    The symbol table is always length+3 per symbol, which dramatically simplifies FIND
    words could have a maximum length 63 instead of 31 (but who would want that?)
    the aforementioned memory savings in the target code
Hmm... getting from the CFA back to the NFA, though, that's tricky. Traversing the entire name list would be slow. Could intersperse pointers between the code blocks, but that's exactly what this design is trying to move us away from. Also there would be three different things trying to grow in memory. Some allocation would be required. Dictionary grows up from the bottom, block buffers grow down from the top, and now the symbol table.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 264 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 18  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: