Storing binary data in screens.

[JimBoyd]

GARTHWILSON wrote:

It's an interesting idea, and one I had never thought of. It should make certain operations easier, with storage in flash memories for example. One thing I had thought of, but never got around to doing anything with yet, is making screens to be bigger, rather than 64 characters by 16 lines. 64 is rather limiting for source code with lots of comments. That is of course why there were shadow screens, back when displays couldn't do 128 characters across; but how inconvenient! So I thought why not have a screen be 2K or 4K, being 128 characters by 16 or 32 lines. Storage is plenty cheap now for that. 128 columns is hardly beyond my threshold of where I really start to feel comfortable.

I prefer using 1K blocks when using them for virtual memory on the Commodore 64. With 4 buffers I have 4 different 1K windows into different locations in virtual memory at the same time.
Here is something for consideration that I may not use myself because 1K screens seem a good fit for the C64's limited physical screen real-estate.
As for screens of 1K being annoying, whartung mentioned something in another thread about screens not being the same size as blocks on some Forth systems. What about making screens a fixed multiple of 1K blocks? They could be 128 characters long. The first 8 lines could be on the first block, the next 8 lines on the next block and so on. The editor could handle treating the 2, 4 or even 5 blocks as a single screen.
In the place of LOAD and THRU , or even in addition to them, there could be SLOAD ( screen load) and STHRU ( screen thru) to load a screen or a range of screens respectively. With these two words and the editor words handling the details, the user would not even need to know the underlying block size was still 1K.

[JimBoyd]

SamCoVT wrote:

The only note I would add about doing this is to make sure you FLUSH or SAVE-BUFFERS before powering down your computer, but that's true any time when using blocks.

When I've used the Commodore 64, whether real or a simulation, I've gotten into the habit of using BYE before shutting it down.

Code: Select all

: BYE  ( -- )
   SAVE-BUFFERS  ACLOSE
   [ 0FFFC , ] ; -2 ALLOT

[SamCoVT]

JimBoyd wrote:

IamRob wrote:

Storing a zero as the first byte of the screen also prevents the screen from being accidentally loaded, but does not affect the screen being displayed, and the LIST command is modified to display Control characters in inverse video.

Oh really?

I believe that's FIG-FORTH type behavior with the null character being the magical word "X" that stops interpreting a block or line of text. That was handy for FIG where the buffers were often smaller than a block, so the end of a buffer did not mean the end of a block, and refill would just get the next buffer and continue interpreting. FIG also would put a null at the end of the input buffer as it read in lines, so it worked well there too.

The newer-ish forths (I think starting with '94) do input differently and process things with "address count" type strings/buffers that I believe can have null and other characters in there. DPANS94 states that control characters shall be treated as a space (verified on Tali2 which lets me type null characters inside strings, but I can't use it as part of a word name because it is treated like a space). A zero at the beginning of a block/screen would then just be treated like a space and the rest of the block would continue to be evaluated if LOADed.

The zero at the beginning of a screen seems like a nice trick for FIG based forths, though.

[IamRob]

SamCoVT wrote:

I believe that's FIG-FORTH type behavior with the null character being the magical word "X" that stops interpreting a block or line of text. That was handy for FIG where the buffers were often smaller than a block, so the end of a buffer did not mean the end of a block, and refill would just get the next buffer and continue interpreting. FIG also would put a null at the end of the input buffer as it read in lines, so it worked well there too.

The newer-ish forths (I think starting with '94) do input differently and process things with "address count" type strings/buffers that I believe can have null and other characters in there. DPANS94 states that control characters shall be treated as a space (verified on Tali2 which lets me type null characters inside strings, but I can't use it as part of a word name because it is treated like a space). A zero at the beginning of a block/screen would then just be treated like a space and the rest of the block would continue to be evaluated if LOADed.

The zero at the beginning of a screen seems like a nice trick for FIG based forths, though.

If that is the case, then that is one behavior I like above other Forths.

Also, if Control Characters are getting automatically converted to an Ascii displayable character, that takes away from the ability to use pointers and Control Characters in screen blocks.

I don't see the purpose of new Forths making Control Characters into printable ascii characters. Does that mean there will need to be different word definitions for different devices?

For example, displaying text to a display screen will display the Control Characters as an ascii character, but printing the text to a printer will print the Control Character as is.

[JimBoyd]

IamRob wrote:

SamCoVT wrote:

I believe that's FIG-FORTH type behavior with the null character being the magical word "X" that stops interpreting a block or line of text. That was handy for FIG where the buffers were often smaller than a block, so the end of a buffer did not mean the end of a block, and refill would just get the next buffer and continue interpreting. FIG also would put a null at the end of the input buffer as it read in lines, so it worked well there too.

The newer-ish forths (I think starting with '94) do input differently and process things with "address count" type strings/buffers that I believe can have null and other characters in there. DPANS94 states that control characters shall be treated as a space (verified on Tali2 which lets me type null characters inside strings, but I can't use it as part of a word name because it is treated like a space). A zero at the beginning of a block/screen would then just be treated like a space and the rest of the block would continue to be evaluated if LOADed.

The zero at the beginning of a screen seems like a nice trick for FIG based forths, though.

If that is the case, then that is one behavior I like above other Forths.

Also, if Control Characters are getting automatically converted to an Ascii displayable character, that takes away from the ability to use pointers and Control Characters in screen blocks.

I don't see the purpose of new Forths making Control Characters into printable ascii characters. Does that mean there will need to be different word definitions for different devices?

For example, displaying text to a display screen will display the Control Characters as an ascii character, but printing the text to a printer will print the Control Character as is.

From what I can find in the Forth-83 Standard and the Ansi Forth Standard, the use of ASCII control characters is an environmental dependency. In other words, go ahead and support the use of control characters. Someone writing for your system, or mine, will need to make the choice between the extra functionality and convenience of using control characters versus portability. Fleet Forth is a Forth-83 Standard Forth and it does support control characters, as can be seen in my two screen shots in this thread.
Fleet Forth also has a "magic" word "X" that stops loading a block or parsing the text input buffer ( or evaluating a string). It is not an inline null. parsing text in Fleet Forth, including loading a block or evaluating a string, will stop when the text stream is exhausted or if EXIT is executed. When the input stream is exhausted, the string returned by WORD has a count of zero and a blank. This word is found in the Forth vocabulary and is an alias for EXIT that is immediate.
It is possible to add the Fig Forth behavior to stop parsing if a null is encountered. It will need to be space delimited.
Here is the session log of where I did just that.

Code: Select all

 OK
FORTH  OK
CREATE X  OK

In Fleet Forth, this basically creates a variable with no storage. Now to make the code field of X point to the body of EXIT

Code: Select all

' EXIT @ ' X !  OK
SEE X 
X
 ' (LOOP) >BODY 24 +
  2870          PLA
  2871   251    STA IP
  2873          PLA
  2874   252    STA IP 1+
  2876  2629    JMP NEXT
9 
 OK

Now to change the name field.

Code: Select all

$80C1 ' X >NAME !  OK
HEX LATEST 4 DUMP 
5721  C1 80 36  B  4 44 55 4D  50 20 54 20 20 4D 45 20   ..6KDDUMP T  ME 
 OK

And store a zero in the first byte of the block I'm using for this test.

Code: Select all

1 RAM LIST 
SCR# 8001 
0: @
1: 
2: CR .S
3: 
4: 
5: 
6: 
7: 
8: 
9: 
A: 
B: 
C: 
D: 
E: 
F: 
 OK
1 RAM LOAD  OK
2 1 RAM LINELOAD 
EMPTY  OK

As can be seen, the block doesn't load until I use LINELOAD to load the block starting at line 2, skipping the null character.
Having said that, it is not a technique I will use. When I use blocks for data, it is usually for virtual memory. The value in the first bytes of a block used for virtual memory depends on what is written to virtual memory.

[GARTHWILSON]

GARTHWILSON wrote:

It's an interesting idea, and one I had never thought of. It should make certain operations easier, with storage in flash memories for example. One thing I had thought of, but never got around to doing anything with yet, is making screens to be bigger, rather than 64 characters by 16 lines. 64 is rather limiting for source code with lots of comments. That is of course why there were shadow screens, back when displays couldn't do 128 characters across; but how inconvenient! So I thought why not have a screen be 2K or 4K, being 128 characters by 16 or 32 lines. Storage is plenty cheap now for that. 128 columns is hardly beyond my threshold of where I really start to feel comfortable.

In the facebook group "Forth2020 Users-Group" Zoom meeting yesterday (Sat, Mar 13, 2021) with Leo Brodie, author of "Starting Forth" and "Thinking Forth" which you can watch at https://www.youtube.com/watch?v=--IJEl6HV2k, he said that Charles Moore (inventor of Forth) was not concerned about what others thought about how something should be done, and that what was of interest to him was "What is the best way to do this for me?", and I seem to remember he said in another part of the video that there was nothing magical about the 1K block size. It kind of opens the way all the more for making 128x32 (or other size) blocks if you want to—not that you really needed permission, but it's easier to justify if you want to.

[BruceRMcF]

If using the first "pseudo-line" of a block as an index, with supporting tools:

\ This block contains definitions supporting BMW style --[ comments

... would be a way to make sure the index line is not interpreted.

The "Comus" equivalent to \ for "end processing source in file" would be \\ ... and then if you have a version that ends processing of a block, an index line starting with \\ would be a block that is never interpreted.

\\ This block contains binary tables used for fast multiply.

Of course, the block is not printable if it is raw binary, and so if it is assumed that the index line listing word(s) require printable characters, that means that only 960 bytes is available for raw binary storage in a 1K block.

If the C64 equivalent for \ is // then an equivalent for \\ may be ///

If it is desired for the block to be printable or transmittable as text, 7 base-32 digits is a maximum of 35 bits, so can hold 32bits of data, so with a trailing space would be 2 printable characters per 1 byte of data ... so a binary block of all bits set (2^32-1) would be:

3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV
3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV 3VVVVVV
...

Note that allocating 32 bits with 35 available means the the most significant digit is 0-3.

As a raw block, that would be 512 binary bytes of data per printable block ... with index line, 480 binary bytes per printable block.

An advantage of the format for a 16bit Forth is that the leading digit is 0-3, and three base-32 digits is at most 32,767 (VVV), so it can be processed with the leading digit, the next trio, and the last trio. So suppose you have LEAD@+ ( addr1 -- addr2 n ) TRIO@+ ( addr1 -- addr2 n ), TOPBITS gets the bottom two bits of a cell into the top two bits, TOPBIT gets the bottom bit into the top bit, and PUT-DOUBLE ( n1 n2 -- ) puts a double into a target buffer, incrementing the buffer index by four.

HEX
LEAD@+ TOPBITS >R
TRIO@+ DUP 2/ >R TOPBIT >R
TRIO@+ R> OR
R> R> OR
PUT-DOUBLE
CHAR+
... and you are ready for the next chunk of data.

[JimBoyd]

IamRob wrote:

Also, if Control Characters are getting automatically converted to an Ascii displayable character, that takes away from the ability to use pointers and Control Characters in screen blocks.

I don't see the purpose of new Forths making Control Characters into printable ascii characters. Does that mean there will need to be different word definitions for different devices?

For example, displaying text to a display screen will display the Control Characters as an ascii character, but printing the text to a printer will print the Control Character as is.

To clarify, Fleet Forth's TYPE does NOT convert control characters, QTYPE does. This is so I can LIST a screen without the listing doing things like changing the text color or clearing the screen because a BLOCK of binary data was listed by mistake, or was indexed.
My metacompiler actually uses a certain range of blocks for virtual memory while building a new Forth kernel.
Fleet Forth's blocks can hold any type of data, not just screens of source code.

[BruceRMcF]

Note that you don't have to give up the convenience of 1KB Blocks to have bigger screens ... you can define a "QuadScreen" as four Blocks on a divisible by four block number boundary and just build a QSCREEN editor wordset to work with four blocks and a display unit. 64x64 is plenty of room to write liberally commented sets of word definitions.

[JimBoyd]

BruceRMcF wrote:

Note that you don't have to give up the convenience of 1KB Blocks to have bigger screens ... you can define a "QuadScreen" as four Blocks on a divisible by four block number boundary and just build a QSCREEN editor wordset to work with four blocks and a display unit. 64x64 is plenty of room to write liberally commented sets of word definitions.

I've mentioned something like this here and here.