Aha, someone else who built video on an AVR... I recall that to get the timing right, I had a program which was almost always in interrupt, except for the bit after each active line for a few microseconds before a timer put it to sleep - so the next sync could be correctly timed.
Neil
Another video on AVR do-er here... I did it as part of my original 6502 project, but abandoned it.
I did have a look at many different "racing the beam" solutions including the early ArduTV thing. e.g. Things like the original ZX80 were ... "interesting" and potentially "broken" but they did seem to work for the most part.
I wanted graphics so rather than using a "rom" lookup for the font in an e.g. 1KB display buffer, I had a ~9K display buffer for 320x220 pixels, so 40 bytes per row. Took an interrupt every 64µS and used that to fill the SPI port with the bytes, polling for the free flag to load up the next byte. Used 2 output pins to get the correct voltage levels as per the ArduTV thing. It all seemed to work just fine and I could adjust the image left/right and up/down.
The biggest drawback was the lack of time left over to do any actual processing - hence some systems (ZX...) which did a screen flicker on e.g. character entry and just blanked during program runs. I think it was the ZX81 that introduced fast/slow modes which could keep the video running during program execution - at the expense of speed...
I latterly abandoned it - purely for the speed issues as the AVR was also acting as a serial interface and filing system for the 6502 but it was a good experiment while it lasted.
https://youtu.be/09zhGUbVxdU
-Gordon