6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Thu Nov 21, 2024 5:26 pm

All times are UTC




Post new topic Reply to topic  [ 44 posts ]  Go to page 1, 2, 3  Next
Author Message
PostPosted: Wed Mar 01, 2023 7:28 pm 
Offline

Joined: Fri Mar 18, 2022 6:33 pm
Posts: 491
Over in this post I included an old and not very efficient subroutine I wrote for converting an ASCII digit into the equivalent binary value. Today, just for kicks, I asked ChatGPT it it could analyze 6502 machine code, and gave it that subroutine to chew on.

ChatGPT wrote:
This code appears to be a routine that converts an ASCII character representing a hexadecimal digit into its corresponding numeric value. Here is how the code works:

1. The routine begins by pushing the processor status register onto the stack (phy).
2. The LDY #$00 instruction initializes the Y register to zero, which will be used as an index into the HEX_TABLE lookup table.
3. The .next_upper label begins a loop that compares the value in the HEX_TABLE table at the index specified by the Y register to the character being converted. If a match is found, the code jumps to the .found label.
4. If no match is found, the INY instruction increments the Y register, and the CPY #$10 instruction checks whether the Y register has reached the end of the table. If it has, the code jumps to the .try_lower label.
5. If the Y register has not reached the end of the table, the code branches back to the .next_upper label to continue searching.
6. The .try_lower label initializes the Y register to 10 (the index of the first lowercase hexadecimal digit in the HEX_TABLE table).
7. The .next_lower label begins a loop that compares the value in the HEX_TABLE table at the index specified by the Y register to the character being converted. If a match is found, the code jumps to the .found label.
8. If no match is found, the CPY #$10 instruction checks whether the Y register has reached the end of the table. If it has, the code jumps to the .not_digit label.
9. If the Y register has not reached the end of the table, the code branches back to the .next_lower label to continue searching.
10. The .not_digit label restores the processor status register from the stack (PLY), sets the accumulator to #$FF (indicating that the character is not a valid hexadecimal digit), and returns from the routine (RTS).
11. If a match is found, the TYA instruction copies the index in the Y register (corresponding to the numeric value of the hexadecimal digit) to the accumulator. The processor status register is restored from the stack (PLY), and the routine returns with the result in the accumulator (RTS).

As for how to improve this code, that depends on what your specific goals are for the routine. Are you trying to optimize for speed, minimize code size, or improve readability? Are there any specific issues or limitations you're running into?


I was fairly impressed by this level of understanding, especially given that in our previous conversation about violin etudes it consistently was confused about technical specifics, and at one point told me about an etude book that it made up on the spot. (It was a bit like talking to Borges...). Here there are a few little errors, but it is mostly right!

So I asked it: Can we both improve speed and shrink code size?

Its reply was accurate, but disappointingly general (Optimize loops, use ZP addressing, etc.). But then I asked it to write its own version:

"Can you show me some 6502 machine language code that will convert an ASCII character representing a hexadecimal digit into its corresponding numeric value? It should return $FF if the input is not an ASCII character representing a hexadecimal digit."

ChatGPT wrote:
Certainly! Here's an example implementation of such a function:
Code:
; Input:  A = ASCII character to convert
; Output: A = corresponding hexadecimal digit value, or $FF if not valid
charToDigit:
    cmp #$30   ; Is input < "0"?
    bmi .not_hex_digit
    cmp #$3a   ; Is input >= "a"?
    bcc .found
    cmp #$41   ; Is input >= "A"?
    bcc .not_hex_digit
    cmp #$47   ; Is input <= "F"?
    bcs .not_hex_digit
    sec
    sbc #$30   ; Convert from ASCII to numeric value
    rts

.not_hex_digit:
    lda #$ff   ; Return $FF for invalid input
    rts

.found:
    sec
    sbc #$37   ; Convert from ASCII to numeric value for letters
    rts
You can call this function with the input character in the A register and it will return the corresponding hexadecimal digit value in A, or $FF if the input is not a valid hexadecimal digit.


It's a good try! I asked it some leading questions to see if I could get it to produce one that would actually work, but the tries got progressively worse. The second try used a "bhi" instruction, and the third one was basically gibberish.

This seems to be a theme with ChatGPT. It's pretty good with concepts, but falls down on specific technical details.

_________________
"The key is not to let the hardware sense any fear." - Radical Brad


Top
 Profile  
Reply with quote  
PostPosted: Wed Mar 01, 2023 7:55 pm 
Offline
User avatar

Joined: Tue Dec 21, 2010 9:16 pm
Posts: 14
Paganini wrote:
This seems to be a theme with ChatGPT. It's pretty good with concepts, but falls down on specific technical details.

That's because it can't invent anything new that it hasn't seen already, and it doesn't understand anything. All it does is remix text that it has come across on the Internet.

Compare with a programmer like you or me who does have an understanding of the 6502 instruction set. We can implement original algorithms without having seen any code for said algorithm. The ChatGPT can't do that.


Top
 Profile  
Reply with quote  
PostPosted: Wed Mar 01, 2023 7:55 pm 
Online
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10985
Location: England
[crossed in post - this is a response to the head post]

I am troubled by this... the only way to know if it's right about something is to have started by knowing what's right. When it's wrong, it expresses the same confidence that it does when it's right, so there's no clue in the language.

We all know that software is rather unforgiving, especially low level software. If it's written without understanding, and only modified when we check it and find it to be wrong, how can we ever be comfortable with the output? It's a random walk, more or less, stopping when we run out of energy in trying to correct it.

I feel we have to be terribly careful in how we use this tool, unless we're asking it something of no importance at all.

And the next generation is likely to be wrong in more subtle ways...


Top
 Profile  
Reply with quote  
PostPosted: Wed Mar 01, 2023 9:01 pm 
Offline
User avatar

Joined: Fri Aug 03, 2018 8:52 am
Posts: 746
Location: Germany
Dog Cow wrote:
That's because it can't invent anything new that it hasn't seen already

i mean humans are exactly the same in that regard.

Dog Cow wrote:
and it doesn't understand anything

i'd argue that's not true.
take HBML for example, which is an alternative syntax for HTML that some guy just made up.
he explained the sxntax of it to ChatGPT, and then asked it to write some basic webpage componenets using it.
HBML didn't exist anywhere online for it to copy/paste or draw any information from, all it had was a syntax and a comparison to HTML, and it was able to apply that knowledge to solve given problems. i would consider that "creating something new" and being able to understand something.
(link to the reddit post, which includes the whole chat log)

but i think a big problem with trying to get it to write 6502 assembly is just the muddiness of the term "assembly" since it refers to all existing ISA's, so even when you tell it "6502 assembly" it sometimes just uses some ARM instructions, or mixes up the syntax or addressing modes.
i often run into the problem that it assumes the accumulator or index registers are 16 or 32-bits wide depending on the function i request.
who knows, maybe a version of ChatGPT that can read the actual 6502 Primer online (like Bing maybe?) will be a lot better at writing code... or maybe chatGPT's strength would be more with helping people to debug their code (idk if you ever get stuck just paste your code to it and ask if it can see anything wrong or weird, maybe it'll work?)


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 28, 2023 6:35 am 
Offline

Joined: Wed Oct 07, 2020 6:43 pm
Posts: 6
This is a first try by GPT-4:

Code:
; Input:  ASCII character representing a hexadecimal digit in the A register
; Output: Corresponding numeric value in the A register, or $FF if not a valid hex digit

        ORA #$20          ; Make sure the input is lowercase (handles both uppercase and lowercase hex digits)
        CMP #$30          ; Compare the value to ASCII '0'
        BCC NotValid      ; If the value is less than '0', branch to NotValid
        CMP #$3A          ; Compare the value to ASCII '9' + 1
        BCC ConvertDigit  ; If the value is less than '9' + 1, branch to ConvertDigit
        CMP #$61          ; Compare the value to ASCII 'a'
        BCC NotValid      ; If the value is less than 'a', branch to NotValid
        CMP #$67          ; Compare the value to ASCII 'f' + 1
        BCS NotValid      ; If the value is greater or equal to 'f' + 1, branch to NotValid

ConvertDigit:
        SBC #$30          ; Subtract ASCII '0' from the value
        CMP #$0A          ; Compare the adjusted value to 10
        BCC ValidHex      ; If the adjusted value is less than 10 (0-9), it's a valid hex digit
        ADC #$30          ; Add ASCII '0' back to the value (restore the original input)
        SBC #$57          ; Subtract (ASCII 'a' - 10) from the value to handle 'a'-'f' range

ValidHex:
        RTS ; Return with the converted value in the A register

NotValid:
        LDA #$FF ; Load $FF into the A register to indicate an invalid input
        RTS ; Return with $FF in the A register



Asked to optimize, it suggested this change:
Code:
; Before:
; ADC #$30          ; Add ASCII '0' back to the value (restore the original input)
; SBC #$57          ; Subtract (ASCII 'a' - 10) from the value to handle 'a'-'f' range

; Updated:
SBC #$27          ; Subtract the difference between the two values ($57 - $30 = $27) to handle 'a'-'f' range



So, still not perfect, but getting there.


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 28, 2023 7:02 am 
Offline

Joined: Thu Jan 21, 2016 7:33 pm
Posts: 282
Location: Placerville, CA
Dog Cow wrote:
That's because it can't invent anything new that it hasn't seen already, and it doesn't understand anything. All it does is remix text that it has come across on the Internet.

(Shhhh!!! You're not supposed to actually say it!)


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 28, 2023 4:07 pm 
Offline

Joined: Mon Jan 19, 2004 12:49 pm
Posts: 983
Location: Potsdam, DE
Ignore the man behind the curtain!


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 28, 2023 4:45 pm 
Offline

Joined: Wed Oct 07, 2020 6:43 pm
Posts: 6
commodorejohn wrote:
Dog Cow wrote:
That's because it can't invent anything new that it hasn't seen already, and it doesn't understand anything. All it does is remix text that it has come across on the Internet.

(Shhhh!!! You're not supposed to actually say it!)


This is demonstrably not true. Just come up with a description of a genuinely new instruction that does something unusual, but useful for the task, and ask it to rewrite the code.


Top
 Profile  
Reply with quote  
PostPosted: Tue Mar 28, 2023 8:20 pm 
Offline

Joined: Fri Jul 09, 2021 10:12 pm
Posts: 741
Yes I think it's important not to dismiss it as a copy-and-paste machine. I think it's more accurate to say it is designed to understand language and data formatting and translate between languages. English is one of those languages, and it is very good at formatting data in English, as well as parsing it. It's also pretty good with computer languages but it will depend how well it knows the language. It is possible to temporarily teach it new concepts or languages during a conversation.

One thing it is very much not good at is facts, I guess because that's not what it is for. If it doesn't know something, it will happily make something up and pretend very convincingly that it is true. It will build it in to programs or sentences and make up other lies and offer them as supporting evidence - for example linking to nonexistent online documentation for functions it has made up. Again it is very good at presenting information in various formats, whether that information is true or not.


Top
 Profile  
Reply with quote  
PostPosted: Thu Mar 30, 2023 2:32 pm 
Offline
User avatar

Joined: Sun Jun 30, 2013 10:26 pm
Posts: 1949
Location: Sacramento, CA, USA
gfoot wrote:
... It will build it in to programs or sentences and make up other lies and offer them as supporting evidence - for example linking to nonexistent online documentation for functions it has made up ...

Telling a harmless tall tale is one thing. On the other hand, blatant liars, human or otherwise, give me the creeps.

_________________
Got a kilobyte lying fallow in your 65xx's memory map? Sprinkle some VTL02C on it and see how it grows on you!

Mike B. (about me) (learning how to github)


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 14, 2023 10:15 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
I asked Google Bard to make a similar routine and it responded with similar code. Lots of faults, but interesting enough it corrects itself when you point out those faults (I only tried a couple).

I then asked it to mathematically express the transformation and it gave this result:

Code:
decimal_value = (ascii_value - 0x30) * 16 ^ (position - 1)

where:
*ascii_value is the ASCII value of the hexadecimal digit
*position is the position of the hexadecimal digit in the string, starting from 1
*decimal_value is the corresponding numeric value of the hexadecimal digit

Which is mostly correct (disregarding inaccuracies such as not explaining that 0x30 is hex and so on..).

I am impressed by some of the responses such AI gives, but the creativity and self sceptisism they exhibit is not very interesting. Not very surprising since they are trained as a search engine and therefore exhibit all the traits of such.

A more interesting thing is to try to guide the AI through its faults to see if it can correct itself. For example I asked it to make 6502 code of the above mathematical expression which it did (with some faults). I then stated: "Can I suggest that you add a branch over the 4 asl instructions instead of clc?" And it actually did do that. Here is the code:

Code:
lda #ascii_value
sec
sbc #0x30
sta temp
lda #position
sec
sbc #1
sta position
beq skip_asl
asl temp
asl temp
asl temp
asl temp
skip_asl:
lda temp
sta decimal_value


Top
 Profile  
Reply with quote  
PostPosted: Fri Apr 14, 2023 10:41 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
I also asked it if a loop may help to ensure that position above 2 can be used, and a way to make the answer 32-bit. And to add enough ASL and ROLs. This is the code it gave:

Code:
lda #ascii_value
sec
sbc #0x30
sta temp_low
lda #0
sta temp_high
lda #position
sec
sbc #1
sta position

loop:
  asl temp_low
  rol temp_high
  asl temp_low
  rol temp_high
  asl temp_low
  rol temp_high
  asl temp_low
  rol temp_high
  dec position
  bne loop

end_loop:
  lda temp_low
  sta decimal_value
  lda temp_high
  sta decimal_value+1


Its actually extended to 16-bit so that position up to 4 should give valid result (I haven't tested but it looks like it). So still some faults... but its slightly more impressive than previous attempts.


Top
 Profile  
Reply with quote  
PostPosted: Sat Apr 15, 2023 8:20 am 
Offline

Joined: Thu Jan 21, 2016 7:33 pm
Posts: 282
Location: Placerville, CA
A boring, old-fashioned human being with a brain might point out that, for a start, the core loop could be made both smaller and faster simply by moving one of the temporary values and the loop index into the appropriate registers.


Top
 Profile  
Reply with quote  
PostPosted: Mon May 01, 2023 4:41 am 
Offline
User avatar

Joined: Sat Dec 01, 2018 1:53 pm
Posts: 730
Location: Tokyo, Japan
The other day I asked ChatGPT 3 to give me a 6502 routine to add two 16-bit numbers together. It came up with the following (and for some reason applied SQL syntax highlighting to it):

Code:
LDA high_byte1 ; Load the high byte of the first number into the accumulator
CLC           ; Clear the carry flag
ADC high_byte2 ; Add the high byte of the second number to the accumulator
STA result_high_byte ; Store the result in the high byte of the result

LDA low_byte1  ; Load the low byte of the first number into the accumulator
ADC low_byte2  ; Add the low byte of the second number to the accumulator, carrying if necessary
STA result_low_byte ; Store the result in the low byte of the result


Given how obviously, stupidly wrong it got one of the simplest and most common things a programmer does on a 6502, no further comment should be necessary.

That said, I'll go on a bit anyway.

The main issue with large language models (LLMs) is that they have no understanding of logic, or understanding period. When you ask it what 2 + 2 is, it comes up with 4 not because it's doing any arithmetic, but because that's a plausible continuation of your input text based on the data set it's been trained on. (This is why it's mostly correct with small numbers, but mostly wrong with large numbers.) This is why LLMs have been referred to as stochastic parrots. (If you really want the details of how LLMs work, Stephen Wolfram has a good, albeit long, explanation.) Essentially, its job is to produce text that sounds convincing; the truth of the text is irrelevant. Combine this with the human tendency to fall into the pathetic fallacy, not to mention simply wanting to believe in Star Trek-level magic, and it's clear why people are so (wrongly) keen on ChatGPT and similar tools.

The stochastic nature of ChatGPT became particularly clear to me when I was asking it how much a game development company spent on testing in 2013. It came up with something that looked quite plausible, though further research showed it to be wrong. It had found annual reports for the company filed with the SEC, but it generated an answer that looked very plausibly like the data from those annual reports that did not actually correspond to any of the real data in the reports. I asked it for references and it produced URLs into the SEC online database of reports that didn't exist; it turns out that all the URLs had similar formats so it simply combined various bits from all the URLs to produce new ones that looked similar to real ones but were not actually URLs from the database.

Also note that ChatGPT 3, at least, cannot accept corrections: if you tell it that a particular line of poetry has ten syllables, not eleven, it will likely still go on about that line having eleven syllables later in the conversation. Again, there's no logic processing involved, it's just stringing words together.

_________________
Curt J. Sampson - github.com/0cjs


Last edited by cjs on Mon May 01, 2023 3:02 pm, edited 2 times in total.

Top
 Profile  
Reply with quote  
PostPosted: Mon May 01, 2023 11:20 am 
Offline
User avatar

Joined: Fri Aug 03, 2018 8:52 am
Posts: 746
Location: Germany
hmm, you just got unlucky it seems. i asked it the same thing and got this:
Attachment:
chrome_swWEnqEsdP.png
chrome_swWEnqEsdP.png [ 29.8 KiB | Viewed 5798 times ]

of course the highlighting is wrong, but that's just an aesthetic thing.

overall Language models like chatGPT can understand logic to a degree (as seen by the HBML example), it's just that they aren't consistent enough that you can ask them a question and get a good answer 100% of the time. sometimes you get unlucky and it spits out garbage, other times you get something actually good and usable.

same thing with corrections, it can accept them, just not consistently.
for example i once asked it to write a powershell script for me that would extract all zip files in the current directory and sort the extracted files by extension into seperate folders.

the first version didn't work and just threw an error at me. so i told it what line errored and what the error was, it apologised for the mistake but didn't really fix it, even after multiple attempts.
so i looked up the error and what was causing it and fixed it myself. then i told chatGPT about the fix i applied and then it was suddenly able to do the same fix to it's own version of the script. after that i was able to tell it to do changes to the script and it didn't break it again...

.

though i'd still say that programming isn't a Language Model's specialty. if you had a language model like that as a translation layer between the user and a more specialized/programming centric AI/ML model, then it would porbably me much more powerful as a programming tool.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 44 posts ]  Go to page 1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron