It works very like a Markov chain, rather like the pastiches one would see back in the 80s or 90s, with one difference being that the generator has a very deep history, and another that the probability for the next token is based on the prompt and the output so far. Another difference is that it's trained on an enormous body of text taken mostly from the web.
The more specific the prompt, and the more popular the content, the better it will do. I would expect it would do better writing x86 code than 1802 code, for example. If the prompt mentions Commodore and quarter squares, it might do better yet.
The algorithm is terribly simple: it's the data which contains all the complexity. The advances come in being able to digest terrifically large amounts of data.
Markov chains on Usenet in 1994:
https://groups.google.com/g/comp.sys.ap ... XL8ZuHTI4Jand here from 1991
https://groups.google.com/g/soc.motss/c ... wvsdsci5gJIn BYTE magazine November 1984
A Travesty Generator for MicrosFred Brooks in the 1950s, as an undergrad, programming the Mark IV at Harvard:
Quote:
So we undertook for our first year project to write a program that would analyze melodies and create synthetic melodies using a eightfold Markovian process. And that took us three years to get done but when—- and the problem is what do you use for a sample. What we needed is a big a corpus as we could get. So we chose common meter hymns because we could find a lot of them that already had the same metrical structure. We transposed them all to the same key then we analyzed the transition probabilities of the melodies. And the interesting question was, is there some order at which you get something that sounds like a human could have written it and yet doesn't replicate a chunk of your sample. And what we found was that at orders five and six that did happen. At seven and eight we replicated sample halves, and at two and three and so forth you wouldn't have mistaken it for human music. But at five and six we got stuff you could pass off on any choir.
Oral History of Fred Brooks; 2007-09-16Edit: for more detail, perhaps see this article
(via
this discussion which is worth a look too especially for the illustrations of failures in simple arithmetic. If you can't always get arithmetic right, surely not a good idea to expect correct code generation.)
Login
Register
FAQ
Search
