6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Thu Nov 21, 2024 12:38 pm

All times are UTC




Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Stack being overrun
PostPosted: Sun Dec 09, 2018 2:30 am 
Offline
User avatar

Joined: Tue Mar 05, 2013 4:31 am
Posts: 1385
Well, I've been doing a fair amount of testing with the CMOS instruction set of EhBasic and found an odd stack and memory overflow problem. To see if it was something I caused when modifying multiple routines with the CMOS instructions, I went back to a very stock version of EhBasic to verify. the "very stock version" only has changes to page zero to avoid conflicts with my BIOS and Monitor code. Turns out the problem still exists.

The basic program being used is based on some of the benchmarking programs that Arne posted, which can be found here:

viewtopic.php?f=1&t=5198

In short, here's my version for one of the programs:

Code:
5 DIM P(4000)
6 P(0)=3: P(1)=3: P(2)=5: P(3)=7
10 ZS=7: INPUT A,B
12 CALL 65328
15 T3=3:T5=5
20 FOR C = 7 TO A STEP 2
22 IF C-T3=6 THEN T3=C : GOTO 80
24 IF C-T5=10 THEN T5=C : GOTO 80
26 IF C-T5=20 THEN T5=C : GOTO 80
30 I=3
35 D=P(I)
40 Q=INT(C/D)
45 IF Q*D=C THEN 80
50 IF Q>D THEN I=I+1:GOTO 35
55 IF P(0) < 4000 THEN P(0)=P(0)+1: P(P(0))=C
60 IF C-ZS >= B THEN PRINT C,ZS,C-ZS, : GOTO 100
70 ZS = C
80 NEXT C
90 PRINT "No Solution",
100 PRINT""
105 CALL 57371
110 GOTO 6


The CALL instructions start and stop a BIOS benchmarking timer but doesn't do anything else. Running the above program, it calculates the numbers, then restarts effectively. After each iteration, the stack pointer continues to go down until you overflow the stack and it crashes. Roughly 13-14 times will do it.

If I add a CLEAR statement to the beginning of the program, the stack and memory stay clean and no overflow happens.

I'm wondering if this is something on the buggy side of EhBasic versus "the way it works". I'm wondering if anyone who has run these benchmarks on EhBasic found a similar issue by running it many iterations.

_________________
Regards, KM
https://github.com/floobydust


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Sun Dec 09, 2018 2:59 am 
Offline

Joined: Mon May 21, 2018 8:09 pm
Posts: 1462
This is a typical fault of BASIC interpreters, including BBC BASIC. It occurs because its control structures are only "pseudo structured" instead of the true structured programming you get with Algol-family languages.

What's happening at the low level is that you're jumping out of the FOR C loop with a GOTO, instead of with the NEXT C that would clean up the stack when it terminates. The subsequent entry of the same loop on the next run doesn't check for a prior loop already on the stack referencing the same index variable.

To solve this, you need to insert a NEXT that always terminates the loop into the escape path. In this case you could replace the PRINT-GOTO combination with C=A : GOTO 80, then alter the post-loop printing to compare C and ZS for equality. Don't forget to account for the modification to C that the NEXT will do before testing for termination.

Another way is to enclose the offending loop and escape code with FOR DUMMY = 1 TO 1 and NEXT DUMMY. This should unwind all inner nested loops when the NEXT DUMMY is executed, without significant side-effects.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Sun Dec 09, 2018 3:35 am 
Offline
User avatar

Joined: Tue Mar 05, 2013 4:31 am
Posts: 1385
Chromatix wrote:
This is a typical fault of BASIC interpreters, including BBC BASIC. It occurs because its control structures are only "pseudo structured" instead of the true structured programming you get with Algol-family languages.

What's happening at the low level is that you're jumping out of the FOR C loop with a GOTO, instead of with the NEXT C that would clean up the stack when it terminates. The subsequent entry of the same loop on the next run doesn't check for a prior loop already on the stack referencing the same index variable.

To solve this, you need to insert a NEXT that always terminates the loop into the escape path. In this case you could replace the PRINT-GOTO combination with C=A : GOTO 80, then alter the post-loop printing to compare C and ZS for equality. Don't forget to account for the modification to C that the NEXT will do before testing for termination.

Another way is to enclose the offending loop and escape code with FOR DUMMY = 1 TO 1 and NEXT DUMMY. This should unwind all inner nested loops when the NEXT DUMMY is executed, without significant side-effects.


Thanks for the very quick reply, that makes sense... but there's no doubt I'm not a BASIC programmer guy :shock:
I'm just relieved that all of my changes to the EhBasic source haven't broke anything. The CMOS version is a bit smaller and a bit quicker as well, so overall, I'm satisfied with the results to date, Cheers!

_________________
Regards, KM
https://github.com/floobydust


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Sun Dec 09, 2018 11:29 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
I tested the benchmark on my latest version of EhBASIC (2.22p4) where I increased the stack floor protection to cater for background interrupts. Sure enough I get only 12 iterations before it throws "Out of memory Error in line 55" due to the exhausted stack.

When you experience a crash instead of the error message in version 2.22p4 you should increase the Stack_floor constant (defaults to 16). In versions prior to 2.22p4 the stack floor protection may or may not catch the out of memory condition as not all internal and no external use of the stack is protected.

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Sun Dec 09, 2018 1:14 pm 
Offline
User avatar

Joined: Tue Mar 05, 2013 4:31 am
Posts: 1385
Thanks Klaus,

For the CMOS version I've been working on (I show as 2.22p4c) it is based on your latest 2.22p4 source with the stack floor protection. As my BIOS/Monitor shows 19 bytes of stack having been used after bootup, etc., I default the stack floor constant to 20, so I get an out of memory error as well.

Also, a big thank you (Danke!) for making patches and such on the original code base, it's been very helpful. I removed all of the interrupt code as I don't need it for my SBC and added an EXIT command to go back to the monitor. Code size has decreased to 9884 bytes due to some other changes as well.

_________________
Regards, KM
https://github.com/floobydust


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Sun Dec 09, 2018 3:37 pm 
Offline

Joined: Mon May 21, 2018 8:09 pm
Posts: 1462
Looking at this from a language-design perspective…

BASIC interpreters normally parse and interpret the program strictly line-by-line. This makes for a simple and compact parser, but it means that the statements delimiting control structures (FOR, NEXT, REPEAT, UNTIL, etc) need to establish and examine program state in order to function. They do not impose any genuine structure on the program; it is syntactically legal to sprinkle these statements at random without bothering to pair them. Mismatching them results in runtime errors due to the program state being inconsistent. This is also why most BASIC dialects can't do multi-line IF-THEN-ELSE structures. (Some can, but they usually change to Algol-family syntax design to achieve this - see below.)

By contrast, an Algol-family language parses the file as a whole, and the structure of the program is enforced syntactically. Newlines are just another kind of whitespace. Your C compiler will complain about mismatched braces and brackets before almost anything else (except the preprocessor, which strips out comments as well as handling includes and macros). Only once the structure of the program has been established will the compiler start examining the individual statements. Specific keywords are provided for breaking out of an inner scope to an outer one, or continuing with the next iteration of a loop, and it's thus an error to provide multiple "loop end" markers for a single loop.

Having understood this fundamental difference of design, you can start to see why Real Programmers disparage BASIC for not supporting proper structured programming.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 7:05 am 
Offline

Joined: Sat Jun 04, 2016 10:22 pm
Posts: 483
Location: Australia
I wondered why BASIC was bashed as much for quite a long time. Now I know why it is, and why I was as confused as I was.
I started programming with Qbasic, which is very different to the BASICs that were being bashed. It supports proper subroutines and functions, going so far as to hide the rest of the program if you edit or create a subroutine/function. It also makes it an error to have such things as a NEXT without a FOR, if I remember right, which means it's probably architectured as you describe, Chromatix.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 7:13 am 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8503
Location: Midwestern USA
Chromatix wrote:
Looking at this from a language-design perspective…

Have you ever used one of the timesharing BASICs found in the UNIX/Linux world? They are very much designed with structured programming in mind. I've written major vertical applications in Thoroughbred Dictionary-IV, which is a very powerful BASIC environment that runs natively on a variety of operating systems. You can make your programs as structured (or unstructured, if you are a programming "slob") as anything written in C.

A key characteristic of Thoroughbred (and others like it) that separates it from the Microsoft-type BASIC engines is program statements are evaluated for syntactical errors during entry—the programmer will immediately know if he's got a problem with a statement. Also, internal tables identify branch, jump and subroutine targets by address, which means the interpreter works that out at the time the program is created or edited, not during runtime. Branch, jump and subroutine targets can be labeled, which is not common in BASIC.

Expression evaluation also occurs during statement entry, and if an expression is dubious in structure (mismatch parentheses, etc.) the interpreter will report it before the program is even run. The evaluation will also simplify a complex math expression as much as possible, for example, eliminating redundant parentheses. Errors that cannot be detected during program creation/editing will be flagged at runtime as is typical of BASIC, but with a sophisticated error-trapping mechanism. Speaking of runtime, variables are organized into a sorted lookup table, which substantially reduces processing time when referring to a variable.

The entire language's makeup is designed to encourage structured programming, such as CALLing remote subroutines (functions in C), WHILE-WEND loops, multi-way decision-making and many other features. It's definitely not your grandma's BASIC!

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 9:18 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
EhBASIC makes no claim whatsoever to be a modern BASIC. It simply matches the capabilities of a 6502 CPU and its limited address space. If you want to run an interpreter of some 100k in size the 6502 is the wrong choice. Just don't compare apples with oranges!

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 10:11 am 
Offline
User avatar

Joined: Wed Feb 14, 2018 2:33 pm
Posts: 1488
Location: Scotland
Chromatix wrote:
Having understood this fundamental difference of design, you can start to see why Real Programmers disparage BASIC for not supporting proper structured programming.


I'm a real programmer (I get paid for it) and I really do not have any sort of disparaging thoughts towards BASIC whatsoever.

BASIC is traditionally the way it is due to its history - it was designed in the days of teletypes (c1964) when there were no visual editors. It was also designed to run and support multiple users on what by todays standards would be considered no better than a pocket calculator. It was also designed to be interactive - you get instant results - none of that waiting for cards or tapes to load or the program to be batch run and a printout being given, hours (or days!) later...

I've studied a lot of old BASIC programs and guess what - they're actually remarkably well structured - given the limitations of those early BASICs - more so the "micro" BASICs rather than the business ones that were also evolving on that generations mini and mainframe computers.

So why, today, does BASIC have this bad reputation? Mostly due to (a) snobbery from the university types, who from 2000 years ago have always pushed for their way to be the best way, and (b) self-taught programmers.

When (micro) computers became accessible in the 70's, clubs sprung up, people even bought their own home computer and just started to program - without the guidance from others. Some of these self-taught programmers write the best games and now run the worlds biggest companies - can you still believe that both Apple and MS started with BASIC? Woz himself wanted to write the first BASIC for the 6502 - and succeeded!

But what now? BASIC has evolved - there are ANSI standards, but also 100's of BASIC-Like languages, most still having an interactive part to them, some still using line numbers, some labels, some both. We have graphical BASIC environment, or, lets call that Visual Basic, or VB. Make it a small part of another application - say a spread sheet and you have Visual BASIC for Applications; VBA. There are billions of VB and VBA programs out there. Some might be used to calculate your pay-cheque..

Acorn/BBC Basic in the early 80's had structured ideas - multi-line if/then/else and do/while loops. Long variable names and named procedures and subroutines that supported recursion with local variables and more modern BASICs have switch type constructs too, some even have a not towards OOP with classes or at least a way to coine blocks of data in a structure (like a C struct)

Personally, I have a fondness towards BASIC - mostly because it was the first language I learned (in 1978) - it was my new Lego - it allowed me to be instantly creating, and to tear ideas apart and re-assemble them. After that, I learned FORTRAN (which was just BASIC on punched cards). My first introduction to a structured algol-like language was something that I suspect only one other person here might have heard of - imp77. Since then (1980) it's been C more or less all the way, although I did spend some time writing lots of BCPL and Pascal and a year doing c++ (never again) as well as some weird stuff like occam and many, many different assemblers.

My fondness extended to writing my own BASIC some 6/7 years ago - it's about 30K lines of C and it's my "ideal" BASIC. I write (and sell) programs in it, and I've even sold the interperter system which now appears as a separate commercial system which you can buy today. My BASIC doesn't even need line numbers, but you can use them if you like.

It does suffer from the issue that this thread is about though - if you jump out of a for/while loop using GOTO then eventually you'll run out of stack space, however it does have a break instruction for loops which does the right thing.

I am considering porting a cut-down version to the 6502, although it would be a big job - it has lots of assumptions built-in, such as 32-bit tokens and 32-bit integers. These are not hard to change though, however I may wait until I have my 65816 system going first.

So lets not completely rubbish BASIC - its evolving, we should look to helping and encouraging those old self-taught programmers, not to show them the error of their ways (error? Hah - some are now billionaires!) but maybe to present alternatives, if appropriate.

Here is an example of an old, structured BASIC program re-done in a modern BASIC (mine): https://unicorn.drogon.net/wumpus.rtb

-Gordon the Punk-Rock BASIC programmer. (See or rather listen to: https://www.bbc.co.uk/programmes/b05pnvmh )

_________________
--
Gordon Henderson.
See my Ruby 6502 and 65816 SBC projects here: https://projects.drogon.net/ruby/


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 4:20 pm 
Offline

Joined: Mon May 21, 2018 8:09 pm
Posts: 1462
I feel obliged to point out that BBC BASIC of the early 1980s did not have multi-line IF-THEN-ELSE or DO-WHILE (though REPEAT-UNTIL is structurally equivalent to the latter). It did have PROC and FN, which were considered notable advances for a microcomputer BASIC at the time, as more structured alternatives to GOSUB (ie. you could pass values as parameters instead of in globals). Better control-flow statements (WHILE-ENDWHILE, multi-line IF-THEN-ELSE-ENDIF, CASE-WHEN-OTHERWISE-ENDCASE) came along with the 32-bit RISC port of the late 1980s, which also allowed programs to be written without explicit line numbers.

And, as I pointed out, such advanced features usually went with a change to Algol-style syntax parsing, which established the structure of the program explicitly at an early stage, rather than implicitly through runtime state. It's fairly clear that VB, QBASIC and pretty much all "compiled BASICs" fall into that category. I think later versions of VB are basically C# with a macro veneer.

I think BBC BASIC remained unusual in that it still used runtime state even for these new features (assisted by an acceleration cache); that would have been done to retain maximum backwards compatibility with earlier versions of BBC BASIC, but it was still an exceptionally fast BASIC interpreter, making maximum use of the ARM CPU's features. And that backwards compatibility meant that using GOTO to jump out of a FOR-NEXT loop could still unbalance the stack.

I use "Real Programmer", capitalised as in the Jargon File, to refer to people who really knew what they were doing and tended to find most high-level languages limiting (not just BASIC). They were particularly scathing about BASIC because its limited features actively got in the way of their accustomed mode of solving problems, which usually involved Algol-style structure even if thy were working in assembly.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 4:31 pm 
Offline
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10985
Location: England
Don't forget LOCAL, which, with PROC, gave you good support for a more modular style of programming.

BBC Basic is still maintained and there's now a multi-platform free version too: see BBC Basic for SDL here. In fact Richard Russell has started freeing the source too.

(The other notable thing about the BBC Basics is their embedded assembler, which allows some high-level approaches to assembly code, as well as making assembly programming accessible to any user of BBC Basic.)


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Mon Dec 10, 2018 6:36 pm 
Offline

Joined: Sat Dec 13, 2003 3:37 pm
Posts: 1004
drogon wrote:
Chromatix wrote:
Having understood this fundamental difference of design, you can start to see why Real Programmers disparage BASIC for not supporting proper structured programming.


I'm a real programmer (I get paid for it) and I really do not have any sort of disparaging thoughts towards BASIC whatsoever.

I +1 what drogon wrote. There are too many billions of lines of functioning, working BASIC code (including BDDs Thoroughbred, MS's VB, MS Access, MS Excel, legacy PICK systems, plus who knows what else) in the world doing real work, solving real problems, efficiently, for it to be criticized so harshly.

I did Great Things in BASIC on the PDP-11 back in the day, I then went on to Great Things in VAX BASIC. I did Great Things on an Alpha Micro.

On the VAX, we had one process, it was essentially a BOM problem (Bill Of Materials, which is processing on a hierarchy of dependent parts). One guy saw that and said "I MUST use Pascal cuz pointers!", and tackled the problem.

He then wrote his code, fired it off, and we watch the counters tick by.

We watched them, and watched them. I stopwatched what I saw and went to our boss let him know that this process was going to take 23 days.

I rewrote it in BASIC, leveraging the VAX RMS, which is essentially ISAM. And my process took 23 hours.

The problem wasn't Pascal, it was HOW it was implemented in Pascal. I chose BASIC because it had the best interface in to RMS (you could use it from Pascal, it's just easier from BASIC). Simply, the Pascal option used far, far to much native RAM (since the developer relied on direct pointers) and swapped to death. Mine relied on keys and ISAM records. And any benefits that Pascal offered didn't overcome the better interface to RMS that BASIC had.
Chromatix wrote:
I use "Real Programmer", capitalised as in the Jargon File, to refer to people who really knew what they were doing and tended to find most high-level languages limiting (not just BASIC). They were particularly scathing about BASIC because its limited features actively got in the way of their accustomed mode of solving problems, which usually involved Algol-style structure even if thy were working in assembly.

Craftsmen don't blame their tools. "Real Programmers" get the job done, they find a way. There's a great story of a guy used Common Lisp to simulate a Forth system embedded on a space probe so they could upload a fix to it in flight.

I've written very clever BASIC code. I've written a multi-user scheduling system controlling 16 simultaneous displays, wall mounted (via some smart controller code I wrote in C) and desktops, while interfaced via remote messaging to Yet Another system (over a very crude, serial based network). All in BASIC. BASIC didn't support tree structures, so I wrote some. Didn't have built it sorting, so I wrote that as well. All of those algorithms from Knuth can be written in BASIC. Folks would select the resources they wanted, my system would propose a scheduling option using a simple best fit algorithm.

I've written truck routing systems in BASIC. Yes, with addresses, and lat long, etc. etc. all in BASIC. I had to parse addresses (thankfully I didn't have to process Salt Lake City), I had to geocode them all myself. I asked the client if I could run the routing software over night and they said "No" because in order to do that, you had to know how many drivers you had. Well, being a charity, they didn't KNOW how many drivers they had until they showed up in the morning (always possible a recovering driver relapsed and didn't show up). So, I had to route my 500 or so daily pickups across the LA Basin in 15 minutes. In BASIC. On a 20Mhz mini computer.

I certainly appreciate BDDs all assembly network of Commodore computers, but what makes that interesting in my eye is the architecture of the network of machines via the shared drive, not that he did it all in assembly.

As I said, "Real Programmers" find a way.

I like toys, I like tools, just like everyone else. I look at a measly RSTS/E PDP-11, or even a CP/M machine, and just stand in awe of the raw capability a high level tool like BASIC can provide. What an extraordinarily simple system that bring so much value out of this little box with just the right software, solving just the right problems.

BASIC is an unsung hero. BASIC was the beginning of the home grown programmer. Whether it was my Dad typing in formulas and stock numbers from the newspapers, using clever variable names like A, AA, B, BB, and complaining how he couldn't use AAA or BBB. A Doctor crafting their own Medical system after a busy day at the office (did your doctor write their own office system? Mine did.) Some kid typing in Wumpus for the 100,000th time.

Empowering people, solving problems. That's what these things are for. It's nice we can surf the web, shop, and watch videos. But I'd much rather beam with pride over a well printed invoice or an accurate aging that has the accounting folks nodding and smiling than any of that stuff.

Once walking through the office, a colleague and I came upon one of the people that gets reports from our system. She was cutting one of them up, reordering it, taping it together, and photocopying it. We said: "You know, we can do that for you." "Oh really!??" Now you can see how a simple sort can bring light to someone's day.

And, no, given greenfield, I would probably not choose BASIC today. But I wouldn't turn it away either.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Tue Dec 11, 2018 12:18 am 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
Wow! :shock:

_________________
Michael A.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack being overrun
PostPosted: Wed Dec 12, 2018 5:00 pm 
Offline

Joined: Sun Feb 22, 2004 9:01 pm
Posts: 108
Chromatix wrote:
I feel obliged to point out that BBC BASIC of the early 1980s did not have multi-line IF-THEN-ELSE or DO-WHILE (though REPEAT-UNTIL is structurally equivalent to the latter). It did have PROC and FN, which were considered notable advances for a microcomputer BASIC at the time, as more structured alternatives to GOSUB (ie. you could pass values as parameters instead of in globals). Better control-flow statements (WHILE-ENDWHILE, multi-line IF-THEN-ELSE-ENDIF, CASE-WHEN-OTHERWISE-ENDCASE) came along with the 32-bit RISC port of the late 1980s, which also allowed programs to be written without explicit line numbers.

ARM BASIC came in 1985/6, so mid-1980s not late 1980s.

And you can write pre-ARM BBC BASIC without line numbers, just use a non-line-numbering editing system. The requirement for using line numbers is dependent on the system you use for entering the code, not the code itself. Even 'C' programs were originally written with line numbers as in a line-by-line environment that is the only way to tell the editor where in the program you want what you've just typed to be inserted.

_________________
--
JGH - http://mdfs.net


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 22 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: