I have been absent from the 6502 Forum for multiple reasons and I'd like to thank the one person concerned about my wellbeing. From
Jul 2022 to Sep 2022, I was primarily avoiding a stalker on the Internet. Given the complete lack of boundaries (and the light touch regulation from administrators), I would be highly unsurprised if my stalker is also the
first person to be banned from the 6502 Forum after constructing an 8 bit system. Either way, I am unimpressed that the ban was temporary. Aptitude with hardware or software is not a license for toxic behavior and - unlike the situation in the 1990s - the 6502 Forum is not the only venue for 6502 systems. From
Sep 2022 to Dec 2022, I worked off-line with Sheep20 developing a tile video codec for 6502 systems. 8MHz 6502 with MicroSD connected via 6522 has enough bandwidth to block copy uncompressed 40*25 tile video and uncompressed 3D surround sound. Development slowed partly because my health declined. Over winter, I felt particularly cold and tired. However, from
Jan 2023 to Jul 2023, I've endured an undiagnosed illness which has repeatedly caused me to scream in pain. This is in addition to all of my previous health problems. Understandably, this has prevented me from programming or writing. Given my 2000+ word ramblings, some might say that my absence was a bonus.
Ultimately, I have searched for a 64-256 character successor to PETSCII. The result has several limitations but the minimal configuration (monochrome and silent) should work on Commodore64, Commander X16, AndersNielsen's
ABN6502 and other systems. The preferred configuration has been subject to scope creep and is slightly outside the base specification of the Commander X16. Sheep20 notes that any specification with "... and a PCM audio FIFO" is a failure to run even on unmodified niche hardware. Regardless, implementation has been mocked with a patch to
ffmpeg and it is suitable to watch cartoons and action films. Indeed, it is particularly good with flames and debris. However, the tile video system is completely incapable of representing two or more pieces of detail within one tile. This is most notable with failed attempts to render text or faces with any fidelity. A further quirk is that animation and live action from the 1990s is most suitable for conversion.
Old cartoons, such as
Popeye, have ropey color but many cartoons from the 1970s and 1980s (
Sailor Moon,
Ulysses 31,
Centurions) may use 20% or more of the luma for over-bright effects. With 3 bit per channel color, the loss of the top 2-3 values is hugely significant. 8*8*8 (512) colors are reduced to 6*6*6 (216) or 5*5*5 (125) colors. That reduces the effective color-space by 1-2 bits. Pastel cartoons from the late 1990s and early 2000s (Totally Spies, Powerpuff Girls, Winx Club seasons 1-3) use the full luma but more recent cartoons - drawn on computer - have an increasing amount of fine detail which a crude codec fails to render in any meaningful manner.
The situation is equally bad for live action but for different reasons. Legacy 525i NTSC transcoded to 360p MPEG2 (or similar) often has washed-out chroma. Examples include
Kenneth Branagh and Emma Thompson in a scene from William Shakespeare's Much Ado About Nothing,
Rupert Sheldrake, The Homing Pigeon Enigma, 1992, Utah Saints'
Something Good Is Gonna Happen and
ABBA In Japan: If It Wasn't For The Nights. When converting this to a low palette suitable for block copying, the result is one base color with random speckling. Whereas, late 1990s video was often shot in HD with the expectation that it would also be shown at SD. As examples, MTV Europe's rotation from the late 1990s (Britney Spears'
Baby, One More Time, Justin Timberlake's
Rock Your Body, Lenny Kravitz's
Are You Gonna Go My Way?, Jamiroquai's
Virtual Insanity, Prodigy's
Breathe) are all excellent after conversion. (Chemical Brothers'
Setting Sun is dodgy but it is intended to be unsettling.) Unfortunately, 4K HDR video (such as an
action sequence from 2011 Transformers: Dark Side Of The Moon or
Bumblebee) has reverted to the washed-out chroma of old video.
Since at least
Mon 21 Dec 2020, I have idly described a tile video codec suitable for 8 bit computers and 6502 in particular. On
Fri 12 Aug 2022, my idleness advanced to taking screen-shots of videos, down-sampling them in GIMP and reducing them to
my preferred 8 bit palette of RRGGBBII. Totem video player has the annoying "feature" that left arrow skips forward 15 seconds while right arrow only skips back 5 seconds. Regardless, I was able to move to the end of a video, skip backwards in 5 second offsets and manually build 40*25 pixel GIF animation with the correct ordering within GIMP. I repeated this process almost 20 times on short videos. This was for the purpose of viewing the lower bound of video quality. From previous failed experiments to convert 3840*2160 pixel video into 16*16 blocks, I found that the very worst case was flat texture with an effective resolution of 240*135 pixels. While many would strive to implement 240 JPEG tiles or something more seamless, I assure you that everything looks awesome in 240 columns of PETSCII. My new target of 40*25 pixels is 1/6 of the horizontal resolution, approximately 1/5 of the vertical resolution and 1/3 of the color-space. However, with a loose superset of PETSCII, some of this loss can be recovered and the result is quite watchable.
Sheep20 was wholly unimpressed with my animated examples and was particularly unimpressed that I hadn't written any software to perform the conversion. After a further four weeks of idle talk, Sheep20 effectively said "put up or shut up." Therefore, on
Thu 8 Sep 2022, I began work on the Klax1 video codec. The name is a reference to
The Clacks communication network in Terry Prachett's
Discworld and I hope that 6502 tile video may be transferred over network in the near future. Klax is also a reference to the "clackers" of the
Difference Engine where one of the many consequences of a successful
Analytical Engine would be a cottage industry of people developing compression algorithms. Furthermore, public lectures would likely use a semi-standard presentation system; much like PowerPoint and a projector is regarded as standard infrastructure. However, if we consider a hypothetical Victorian PowerPoint using cogs, gears and lensed candles, it is highly likely to use an addressable array of tiles. A vector scheme with deflection mirrors is desirable but infeasible. I have also just discovered that
Klax is a block game on Atari 2600.
I have previously worked on audio and video codecs with mixed success. One of the most prominent problems is a quest for efficiency. Specifically, an audio or video codec which can encode or decode in real-time has considerable practical benefit. Unfortunately, barriers to entry have increased while expectations have also increased. In particular, for video codecs, increased horizontal resolution is typically accompanied with increased vertical resolution. With the exception of a transition to widescreen, doubling the width also doubles the height and this quadruples the total number of pixels. For video encode and decode, the large volume of data is typically handled with parallel processing. This is commonly described as "embarrassingly parallel" given that video can be partitioned and assigned to separate processing cores with almost no loss of efficiency. However, a paradigm of embarrassingly parallel processing works against the plucky amateur who wishes to implement video on a single core, 8 bit system. Bandwidth is already constrained by whatever can be soldered at home or programmed on affordable FPGA. Options to scale home-brew hardware linearly are limited or absent. A further impediment is sourcing example material. When an increasing proportion of video is 4K HDR or better, getting it into an 8 bit system is also a minor challenge.
Understandably, my previous work with video codecs suffered from premature optimization. Specifically, I was too focused on byte streams and marshalling with the expectation that I could make it run at 30Hz - if only I could solve the immediate problems. Unfortunately, I could not get different features of my video codec to interact gracefully despite writing an elaborate assertion system. With hindsight, the assertion system was more focus on detail and offered little help with structure or semantics. This time around, I have been careful to avoid previous traps. Helpfully, tile video eliminates many of the fancy features typically found in video codecs, such as motion deltas. Unhelpfully, multiplication and compression are also excluded. Such minimalism has led to constant consideration about upward compatibility and this leads to the next most obvious traps of portability and over-abstraction. I could have written the absolute minimum amount of software to encode and decode one video frame. Instead, I was slowed by making my work open ended. My standard workflow for writing a video codec is to boot-strap using ffmpeg, libgd, libpng and zlib. ffmpeg has features to split and join frames in PNG and many other still image formats. libgd allows manipulation of bitmap images and this includes rectangle drawing. libgd calling libpng calling zlib performs read and write of GZip compressed PNG images. Therefore, a small amount of C, an unremarkable Makefile, some standard libraries and the external invocation of ffmpeg is sufficient to start writing a video codec. However, I didn't choose this route. Instead, I started work on a laptop with a live-boot, no compiler, no make system and without ffmpeg. In an attempt to make a video codec which was portable rather than fast, I spent two evenings attempting to decode PNG files using Unix shell scripts and Perl. I made shockingly good progress. Specifically, I decompressed chunks of PNG's
RIFF variant and obtained the raw RGB byte stream of the first chunk.
This tedious approach was abandoned when, by chance, I discovered that my live-boot system had an ImageMagick binary. ImageMagick was hugely significant in the 1990s because it allowed batch processing of pictures not restricted to cropping, resampling, titling and - most significantly - image format conversion. This was a very intensive task when servers typically ran at 50MHz and had 16MB RAM. It did not occur to me that ImageMagick would be the type of fossil which survives in the default Debian Linux distribution or derivatives. Regardless, I quickly formed a cunning plan. I could make a Perl program which has ffmpeg and ImageMagick as dependencies and use Perl's shell pipeline spawning functionality to read and write all PNGs as uncompressed TGA. Specifically, to decompress PNG:
Code:
open(IN,sprintf("cat 'foo/%08d.png' | convert - tga:- |",$frame)) || return 0;
This provides an input stream with a fixed length header and raw RGB data. The header describes the size of the image and the bit depth. Dependency on ImageMagick's convert binary is not ideal. However, it is not particularly onerous when ffmpeg is also required. Unfortunately, I encountered a multiple problems across different systems. The root problem is that versions of ffmpeg and ImageMagick weren't consistent and functionality differs across versions. ffmpeg has a deprecated set of parameter names for splitting and joining images while different versions of ImageMagick "helpfully" reduce RGB input to bi-level or monochrome. The initial work-around was to use PNG for input and JPEG for output. This work-flow reduced error but it was insufficient across all versions of ImageMagick. The subsequent work-around was to add SMTPE style chroma boxes (black, red, yellow, green, blue, purple, cyan, white) to inputs or outputs. I also added the black and white boxes for the bottom 10 bits of the frame number. (This was greatly appreciated by Sheep20 who has previously edited 16mm film and dubbed the audio separately.) The forced color prevented, for example, black frames being output as a bi-level PNG or monochrome JPEG. Many videos start with a black frame but the subsequent switch to color encoding caused ffmpeg to fail. It took a while for me to identify the root cause and this was not helped by encoding and decoding a broad selection of test videos on multiple computers. A further problem was truncation of frames caused by the processing delay of ImageMagick. I was reluctant to add a one second sleep in a video codec - and especially so when this only makes the problem less likely rather than solve it. A null stream is trivial to identify and redo. However, there is a more insidious case where the header is complete but the payload is short.
A codec requires an encoder and a decoder - and these components require data format. Following the example of TGA, Klax1.0 video format has fixed length header and stores tile data in 7 byte tuples where 1 texture byte is followed by 8 bit per channel RGB foreground and background. I hoped to store textures as two nybbles representing a XOR of two Walsh basis functions (horizontal and vertical) to approximate an area of 16*16 pixels. For downward compatibility, I hoped to store these nybbles in a non-obvious, non-contiguous format. Nybble encodings 8-15 represent the highest frequency encodings. The order of the bits is unimportant because they are typically used to select one pattern from a character ROM. The top bits of each nybble can be grouped together and this allows the highest frequency encodings to be mapped down. On 8*8 pixel systems, such as Commodore64 or Commander X16, NV bits can be tested or cleared with the least number of cycles. This allows 64 encodings to be sought in the most rapid fashion on systems with smaller tiles. This is not original thinking. I merely follow the examples of IEEE-754 floating point format and 65xx peripheral chip registers. The original part is that the scheme can be nested. Any encoding can be trimmed down to an encoding where each Walsh function has less bits. Regarding texture and color together, 16*16 pixels at 3 bytes per pixel (768 bytes uncompressed video) are approximated as 7 bytes per tile. This fixed length data reduction allows linear seek and read/write operation within an unindexed file. The overall scheme may have similarity to Avid's old proxy video editing format but I do not have sufficient information to confirm or refute it. Uncompressed data is reduced to less than 1% of its original size and this is a sufficient reduction to display video on 6502. It can be squashed further but the 7 byte tuple format is more flexible for experimentation. By retaining maximum RGB depth at the intensive encode step, we minimize the need to repeat encode. Whereas, decode has very little to perform. Experiments with palette mapping can be performed rapidly and repeatedly on 7 byte tuples. Although the tuples could be smaller, the gain is a false economy.
By
Mon 12 Sep 2022 - three days after I started - I encoded and decoded my first video frame. After that, I encoded and decoded my second video frame. I then cursed for two hours. Speed was a concern. Klax1.0 encoding took 40 minutes per frame. It was a brute force technique where each 16*16 pixel tile was partitioned using a XOR of two Walsh basis functions (horizontal and vertical) and the minimum
RMS error found. This is standard codec theory to minimize encoding error.
My greatest concern was that trivial examples failed horribly. I assembled a small library of video stills which were challenging but representative of anticipated inputs. This included a selection of frames from the standard
Bad Apple animation, Winx Club
Believix Transformation,
AKB48's distinctly NSFW [Not Safe For Work] Baby! Baby! Baby! music video and others. Unfortunately, one frame from Bad Apple revealed that more than 95% of the tiles were either solid, horizontal half tiles, vertical half tiles or the PETSCII Battenberg symbol. The frame could be more efficiently encoded using two bits per tile with minimal loss of quality. It looked moderately better than the output of an Atari 2600. Encoding 40*25 tiles, I had an effective resolution of 80*50 pixels - with the exception that one corner of a tile could not be set independently. AKB48 was blurry and also highly skewed.
I was in a very bad mood. I planned to re-use this tile encoding in a large number of other contexts including textured polygon blitting, video sampling and Virtual Reality. I spent many hours drawing diagrams and it was all undermined by one empirical result. I was reluctant to continue because more than one year of doodling had been dashed. Unfortunately, I was accustomed to this type of failure in the field of video codecs. To salvage the situation, I needed to increase effective resolution; the illusion of independent choice. I had
already described this problem to AndersNielsen:
Sheep64 on Sat 9 Jul 2022 wrote:
While tiles $00-$0F could be allocated to 2*2 blocks and allow 102*60 pixel bitmap display, 3*3 blocks can be assigned such that the middle square is the aggregate of the other squares. This would allow 153*90 pixel display with minor restriction. Other allocations are also possible.
My problem was analogous to a well known problem in JPEG/H.261. Specifically, there are a very large number of valid encodings which represent high frequency noise. Even worse, one tile of high frequency noise looks much like any other. JPEG has quantize thresholds which discourage these large, inter-changable encodings. With an encoding which only represents the most prominent wave, I observe a huge skew towards low frequency encodings. The most immediate improvement was that I could reduce the brute force search-space of Walsh functions. Reducing the maximum frequency of the horizontal and vertical Walsh functions by a factor of two reduced encoding time by a factor of four with minimal loss of encoding quality. This reduced frame encoding time from 40 minutes to 10 minutes. Down-sampling 1280*720 to 640*360 (or, more accurately, 640*352) further reduced encoding time to 2.5 minutes per frame. The down-sampling step is performed by passing appropriate parameters to ImageMagick.
Given the long encode time of each test image, I often tweaked the codec while I waited. On multiple occasions and often within the span of 10 minutes, the encoder became incompatible with the decoder. Furthermore, I used test data with incompatible aspect ratio or frame rate. Under such circumstances, I burned through a few step revisions of Klax encoding before arriving at a stable Klax1.3 encoding. This defines a 36 byte header which includes width and height in tiles, frame rate and tile set. Such is my laziness, each binary byte of parameter is preceded with ASCII. As an example, width is defined as 'X' followed by high byte of the width followed by 'x' followed by the low byte of the width. This greatly aids debug when reading hex dumps and should be imitated more widely.
I sought a better encoding. My initial plan was to have 48 types of Walsh tile where either the horizontal or vertical frequency may be large. This leaves 16 miscellaneous encodings which may define the most obvious omissions. This subsequently became a residual pool of 4-16 Walsh encodings, 48-60 other encodings and - potentially - 192 upwardly compatible encodings. This leaves the mere problem of defining 48-60 tile types. I thought that I was original to define diagonal chords. However, I already published a variant of this idea when suggesting an
alpha channel of tiles to ElEctric_EyE:
Sheep64 on Mon 28 Dec 2020 wrote:
A non-obvious arrangement is a square with nodes along all four edges at the 1/8, 3/8, 5/8 and 7/8 mark. This leads to 192 interesting permutations where it is possible to draw a chord diagonally across the tile. Furthermore, there are two encodings for each pair or points. This is sufficient to indicate which side of the chord should be solid or transparent. The remaining 64 permutations may indicate chords drawn at the 1/4 and 3/4 mark. The remaining 16 permutations may indicate chords in a diamond along the 1/2 mark of each edge. Of the remaining four permutations, one may indicate fully solid, one may indicate fully transparent and two encodings are unused. These unused encodings may not be useful for a basic hardware implementation but they are sufficient to indicate an escape code for future functionality.
A more obvious method to chord a square tile includes all four corners and nodes along all edges and 1/4, 1/2 and 3/4 mark. This provides 176 interesting permutations. The remaining 80 permutations may be allocated to curved alpha masks.
64 encodings seems generous but they are rapidly consumed because the typical encoding requires four-way rotational symmetry in addition to a mirrored form. Therefore, symbols are typically defined in groups of eight. Only special cases require four, two or one encoding. A blank encoding is not specifically needed because it can be defined as any tile with the same foreground and background. However, the video codec is intended to work as an overlay for ASCII text where both planes share the same palette values. This multi-plane scheme is also intended to be upward compatible with Greek, Cyrillic, CJK [Chinese, Japanese, Korean] and a variety of dingbat symbols which would be primarily useful for a video player and circuit CAD. So, a blank encoding in the video plane is not strictly required but it allows other planes to be viewed without obstruction. It is desirable for the blank symbol to use encoding 32 because this would be consistent with ASCII space. It is desirable for a corresponding reserved-for-future-expansion symbol to use encoding zero because this can be rapidly tested on 6502. It is also desirable to group symbols into powers of two because this allows, for example, FPGA character generation to use the least logic. It is also desirable to group symbols such that the most likely transitions are adjacent encodings. I have suggested a
palette encoding where INC/DEC operations simulate MPEG lighten/darken operations. INC/DEC on suitably arranged textures may simulate motion deltas. Sheep20 recommends that horizontal motion should take priority on the basis that horizontal panning of video is more common than vertical.
There are a few more constraints which should be enumerated. A smattering of high frequency encodings is desirable but not at the expense of rendering diagonal impulse - as occurred with Walsh basis functions. The union of PETSCII and Walsh functions would be desirable for display but the intersection is essential. The best precision for scroll-bars is desirable. In particular, 1/2 tile precision is grossly inadequate. The foremost application for scroll-bars is to convey position and duration of a video. This can be achieved using encodings 0-31 in the ASCII plane but this excludes other uses for that range. Most importantly, all encodings should be chosen with equal probability. Obviously, a blank frame will use one encoding. However, good tile encodings must be chosen frequently while also minimizing error. An encoding which is never used is obviously useless but it is less obvious that two or more marginal choices will maximize encoding error rather than minimize it.
After getting encode time down to 2.5 minutes and then improving image quality, it was time for a second round of improvements. The codec had become an elaborate Perl script around ffmpeg and ImageMagick's convert binary and it offered multiple modes of operation. It was apparent that the work-flow of ripping examples from YouTube and then MP4 -> PNG -> Klax1.3 -> JPEG -> MJPEG (for viewing) required approximately 1GB storage per minute of video. Splitting frames was a huge overhead and it failed to work with file system
read-ahead. One mode of operation for the Perl script was to rasterize PNG frames into one compressed stream of 768 byte blocks. This saved a moderate amount of storage because compression continued from one frame to the next while also saving the unused partial sector at the end of each file. More importantly, the raw video could be decompressed faster than it could be retrieved from storage with read-ahead. Unfortunately, when working from FAT32 storage, I now encountered FAT32's 4GB file limit. Regardless, using GZip compression, I was able to encode 5GB of raw video data from
SSX Cheer's Sharkbite routine. Sheep20 was irked that the text on the logo was only readable when the camera panned and has described the inter-dependent choices of pixel encoding like looking through patterned glass. Specifically, the ribbed type which makes a square lens effect. Sheep20 was more impressed that the tile video codec handled fades between cameras and, in other examples, transparent smoke and fire. Sheep20 was completely unimpressed with
SSX Cheer's tick-tock move (not to be confused with
Intel's tick tock or
ByteDance's TikTok) which I've performed successfully in competition conditions. Actually, I've used a very peculiar set of skills to improve the arts and sciences. In 2007, I daisy-chained 4017 counters to make a clean-room clone of a motion capture system exhibited at SIGGRAPH 2006. This was used to make the
jump in the
Dark Knight trailer. After joining a cheerleading team in 2013, I used a re-implementation of the motion capture system to improve my gymnastics. From 2022 to present, I now use my knowledge of ballet, jazz, cheer and urban to rapidly find errors in video encoding. This is why I've been using video such as Taylor Swift's
Shake It Off as test data. The encoded video looks impressive, sounds impressive and it is a convenient length to encode and demonstrate. However, I can coach many of the moves which I am unable to perform myself. Errors may be fleeting. Sometimes, the subject doesn't flow gracefully. Even with a blocky codec, that remains apparent. With suitable choice of test data, I can maximize the overlap of skill and write an encoder which minimizes subjective error.
Sheep20 was surprised that I would write more than 600 lines of Perl and 400 lines of C before invoking an interpreter or compiler. Sheep20 was more surprised that I could write more than 1000 lines with matched round brackets, square brackets, curly brackets and no dropped semi-colons. Admittedly, the C had missing array indexes and a few stray sigils. However, indexes present were matched. I work in this manner because it allows me to focus on algorithm rather than grammar. Too many people develop "works for me" brittle software in REPL - or force software through a compiler in ignorance. An example of grammatically correct, semantically deadly software - specific to C - is the break statement which caused network cascade failure in the
Martin Luther King Day Telephone Crash. 70 million telephone calls were dropped. 60000 people had no access to emergency services. Some people probably died. Admittedly, I'm only tinkering with pattern matching. However, it is a dramatic example of how focus on bytes, cycles and grammar is premature optimization. For me, an incomplete program, especially in the wrong language, is premature.
On
Wed 21 Sep 2022, I started to split the brute force tile encoder into a separate program. Initially, the sub-program was written in the same language. This paste job minimized disruption. It was then trivial to convert the "engine" into compiled C. I expected a small speed boost because the single threaded application would now be split across two cores and one would be free to perform blocking I/O while the other used 100% CPU. Furthermore, the communication channel (Unix pipe) is typically implemented in an efficient manner. For example, on Linux, I believe that a circular buffer of four memory pages are transferred between two application-spaces. In my case, with 768 byte blocks, this will be greatly aided by every third page being a block boundary. However, speed improvement was far more impressive. I did not expect performance to increase by more than 100%. The cache locality of the Perl bytecode interpreter and the 768 integer data-set provided a huge boost. This left conversion to C with a less stellar advantage. Regardless, encode time was under 30 seconds per frame.
For additional speed, it was trivial to make Perl spawn multiple copies of the engine and round-robin 768 byte blocks. Rather than reconcile the 7 byte tuples into one stream, they were written independently as striped data. Unfortunately, this parallelism has limits. I was unaware that ffmpeg on x86 already had assembly optimization for GZip PNG encode but I assumed that it was the bottleneck. Excluding ffmpeg's PNG encode, Klax1 encode time was down to 4.5 seconds per frame. Sheep20 wondered how I had striped the data. Sheep20 was under the false assumption that tile encoding used adjacent pixels in the encode process. I had to explain that it worked like JPEG and MJPEG. Every tile is encoded separately. However, the encoding scheme is such that an edge in one tile usually lines up with an edge in an adjacent tile. Likewise, fades, smoke and fire work because each encoded tile represents the minimum error of a given impulse. The encoder doesn't detect two layers and encode them separately. The encoding scheme has one or more variables which can be minimized. This may be part of a hierarchical encoding. It may be followed by a residual pass which reduces error to zero. Somehow, a scheme should rigged to approximate input.
On
Fri 21 Oct 2022, I began writing small programs to manipulate the 7 byte tuples, such as palette rotation, mirror and resize. Most of these programs are less than 500 bytes of Perl due to sneaky use of regular expressions. The 36 byte header can be pattern matched while extracting pertinent fields, such as width and height:
Code:
binmode(STDIN);
binmode(STDOUT);
read(STDIN,$head,36);
if($head!~/^\x7F\xFF\t\n\r\r\n[\010\011]KLAX1\.[34] o.F.f.J.j.X(.)x(.)Y(.)y(.)\0\0/s) {
exit(18);
} else {
$mx=ord($1)<<8+ord($2)<<0;
$my=ord($3)<<8+ord($4)<<0;
and the tuples can be split with:
Code:
read(STDIN,$tupl,7);
$tupl=~s/^(.)(.)(.)(.)(.)(.)(.)//os;
From here, it is possible to output negative colors, rotate color-space, reduce bits per channel, set foreground or background to zero, swap foreground and background or perform statistics on the data, such as frequency of textures and distribution of each color channel. Spatial and temporal processing is also possible. For example, it is possible to crop a cuboid of tuples within (x0,y0,t0)-(x1,y1,t1). Horizontal flip can be implemented by reading one row of tuples and outputting them backwards. Vertical flip works on larger quantities because entire rows are written in reverse.
A strange effect occurs if video is flipped but texture is unmodified. It is more accurate to describe the video as divided into textured strips and re-assembled as strips in the reverse order but with the texture otherwise unchanged. It kinda works but it is noticeably odd. The most informative program by far was resize. Blocky resize of bitmaps is bad. Blocky resize of textures is surely going to be worse? Not necessarily. Increasing the size of a video causes duplicate rows and columns to be inserted. Likewise, decreasing the size of a video omits rows and columns. However, this is tolerable over surprising extremes. Indeed, from my experiments, 40*25 tile video can be viewed comfortably at 12*7 or smaller, especially if accompanying audio is dominant. Obviously, the reduction is permanent loss of data but the total loss may be minimal when audio is also considered. Furthermore, 12*7 is a very convenient size and ratio. 84 bytes per channel for texture, foreground and background makes 252 bytes per frame for uncompressed video thumbnails. The aspect ratio is also very convenient. I strongly considered 8*8 SHEEPSCII icons for the forthcoming SheepOS™ (and prototyped a few designs) but I have definitely chosen 12*7 instead. The cool kids regard 40*25 as 16:10. 12*7 very closely approximates 16:9. It is the ideal size and aspect ratio for widescreen video icons. This is especially true when they are arranged in a grid. On 40 column display, 12 columns for Icon0, 1 column gap, another 12 columns for Icon1, 1 column gap, another 12 columns for Icon2 and 1 column gap leaves 1 column for vertical scroll bar. The remainder is exactly the same for 25 row display. 7 rows for Icon0, 1 column gap, another 7 rows for Icon3, 1 column gap, another 7 rows for Icon6 and 1 row gap leaves 1 column for horizontal scroll bar. Astounding!
With a 3*3 grid of 12*7 tile video thumbnails, it is possible to play them all concurrently (but silently) from storage or network and zoom to full screen when a video is selected. For streaming media, full screen video continues from the thumbnailed frame. For pre-recorded media, play starts from the beginning. For an example of this in practice, see the Glam Metal Detectives which used 3*3 and 4*3 video thumbnails for
comedy sketches in the form of retro-futuristic, dystopian 500 channel television feed.
At the other extreme, AndersNielsen's
ABN6502 with 51*30 tile display also approximates 16:9 and 16:10 display. Furthermore, the 1530 byte display is highly suited to 512 byte block storage, although payloads may have to be trimmed by one row or column to work over Ethernet. Actually, the loss of rows or columns may not be detrimental because 51*29 approximates 16:9 while 48*30 is 16:10 exactly. Although AndersNielsen maintains fastidious compatibility with 2MHz NMOS 6502, moderate extension to ABN6502 allows 8MHz 65C02 to multiplex texture at $0800-$0FFF, foreground at $1000-17FF and background at $1800-$1FFF using one 74HC161 and two 74HC574 or similar. A far less modest extension with dual port RAM provides PCM audio output. With no modification to hardware, ABN6502 can implement monochrome Klax1.3 in character ROM. Specifically, the 64 symbols would be characters $80-$BF and the inverted versions would be $C0-$FF. Patches have been sent to AndersNielsen with the suggestion to demo Bad Apple from
open reel tape or demo Justin Timberlake's
Rock Your Body from MicroSD. However, AndersNielsen has wisely spent time on an even more impressive
6507/6532 RIOT system which is
no larger than an Arduino Uno. On Wed 2 Feb 2022,
I said that AnderNielsen had great potential after getting WS2812B Digital LEDs working with 6502. The subsequent 17 months have been impressive.