Benchmarking: VTL-2 vs. GIBL (TinyBasic)

[drogon]

Thought it might be fun (for some value of fun!) to run some of the classic benchamarks on my new port of VTL-2.

I'll spare you the suspense.. VTL-2 is faster.

I used the old Rugg/Feldman tests from 1977. Previously I did a write-up with them here:

https://projects.drogon.net/retro-basic ... enchmarks/

But this is here and now, so ...

VTL-2 is generally faster than GIBL (By TinyBasic). If you know how they work this is unsurprising. TinyBasic has to do a string compare for everything. There is no tokenisation, so the speed of an operation will depend just how deep down the chain the operation might be. VTL-2 also doesn't tokenise, but as every operation is essentially a single byte then it's basically pre-tokenised by you, the user as you type in it.

I'm not going to list all the benchmarks - see that link above, but to compare VTL-2 against GIBL, here is the last one I could run: R/F 7:

GIBL/TinyBasic:

Code: Select all

300PRINT"S"
400K=0
430M=TOP
500K=K+1
510A=K/2*3+4-5
520GOSUB 820
530FOR L=1 TO 5
535!(M+L) = A
540NEXT L
600IF K<1000 GOTO 500
700PRINT"E"
800END
820RETURN

and VTL-2:

Code: Select all

300 ?="S"
400 K=0
430
500 K=K+1
510 A=K/2*3+4-5
520 #=820
530 L=1
535 :L)=A
537 L=L+1
540 #=L<5*535
600 #=K<1000*500
700 ?="E"
800 #=999
820 #=!

And here are the results:

Code: Select all

        VTL-2                    | GIBL
---------------------------------+--------------------------
rf1     2.48  2.46  2.50 =  2.48 |  1.41  1.39  1.40 =  1.40
rf2                         2.48 |  7.94  7.87  7.87 =  7.89
rf3     3.88  3.91  3.97 =  3.92 | 12.36 12.30 12.27 = 12.31
rf4     4.22  4.13  4.13 =  4.16 | 12.41 12.41 12.30 = 12.37 
rf5     5.56  5.48  5.54 =  5.53 | 15.71 15.73 15.75 = 15.30
rf6    13.19 13.14 13.14 = 13.16 | 26.04 26.11 26.18 = 26.11
rf7    15.02 14.99 15.01 = 15.01 | 45.51 45.70 45.47 = 45.56

There is no VTL-2 run for rf2 as its the same as rf1 - due to VTL-2 not having a FOR loop construct.

Note: This is all running on a 2Mhz 6507 CPU and timings were with a stopwatch - I ran each one 3 times and took the average.

GIBL initially was faster but that was just because of the FOR loop construct in R/F 1. It went downhill after that when the benchmarks used a manually counted loop and GOTO.

I did think that the multiplication that you need to use in VTL-2 for every loop might have slowed it down, but it didn't appear to. (It's also an unsigned multiply). GIBL uses stacks to keep track of the FOR loop data and the GOSUB/RETURN data while there is no equivalent in VTL-2, so that's also going to add overhead into GIBL.

Conclusions?

Same as what we know already - as we move from assembler to an early symbolic code to a more feature full symbolic code the execution time is slower and slower - that's the penalty for using a higher (and higher) level language. (In the world of the interpreter, anyway)

It sort of broke down a few years later when BBC Basic was faster out of the box than any other interpreted Basic at the time - and it just got faster as the years went on, however that speed advantage did require larger and larger code spaces - 1K for VTL-2, 3.5K for GIBL/TinyBasic, 8K or 10K for MS Basic, 16K + another 14K of operating system for BBC Basic.

Cheers,

-Gordon

[barrym95838]

drogon wrote:

I did think that the multiplication that you need to use in VTL-2 for every loop might have slowed it down, but it didn't appear to. (It's also an unsigned multiply).

I spent a few precious code bytes to make 0*X and 1*X faster in VTL02C, knowing that it would have a favorable cost-to-benefit ratio.

Quote:

BBC Basic was faster out of the box than any other interpreted Basic at the time - and it just got faster as the years went on ...

I wasn't exposed to it in my formative years, but in hindsight it looks very impressive to me.

[BigEd]

Nice comparison, Gordon. And notable, Mike, the idea of special-casing some of the most common multiplications.