6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Sat Sep 21, 2024 4:30 am

All times are UTC




Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 12:26 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
Hi

I am trying to compute CRC-16 for a SD Card, and I found the table-based version given in the source repository (http://www.6502.org/source/integers/crc.htm) was fast.

Anyway, the results I get are not the same as those reported by the SD Card. Anyone knows if the code is ok? :?


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 1:23 pm 
Offline
User avatar

Joined: Wed Feb 14, 2018 2:33 pm
Posts: 1467
Location: Scotland
kakemoms wrote:
Hi

I am trying to compute CRC-16 for a SD Card, and I found the table-based version given in the source repository (http://www.6502.org/source/integers/crc.htm) was fast.

Anyway, the results I get are not the same as those reported by the SD Card. Anyone knows if the code is ok? :?


Are you sure the SD card is using CRC? My (somewhat new and possibly limited) experience is that they don't use CRCs when used in SPI mode - that's assuming you're using SPI mode. However you do need a few CRCs in the initial setup commands but you don't need to calculate these as they're fixed and well known anyway.

My experience is based on writing an SD card filesystem in the past few weeks which uses SPI to communicate with the SD card and other than two (I think) cases for the setup commands, I don't use CRCs at all.

You may actually want CRCs though, in which case just ignore my witterings :)

Cheers,

-Gordon

_________________
--
Gordon Henderson.
See my Ruby 6502 and 65816 SBC projects here: https://projects.drogon.net/ruby/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 2:53 pm 
Offline
User avatar

Joined: Fri Aug 30, 2002 9:02 pm
Posts: 1738
Location: Sacramento, CA
I used that CRC code with my xmodem routines and it worked properly for me. I read somewhere that some CRC's use different initial values. Not sure if that is true or if that might apply to your case.

Daryl

_________________
Please visit my website -> https://sbc.rictor.org/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 4:30 pm 
Offline

Joined: Mon May 21, 2018 8:09 pm
Posts: 1462
Not only different IVs, but different polynomials, output XOR filters, and even evaluation directions. CRC can be a real mess, *especially* the 16-bit ones.

Find out exactly which CRC variant is in use here. Until you know that, you won't be able to produce matching values.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 5:07 pm 
Offline
User avatar

Joined: Tue Mar 02, 2004 8:55 am
Posts: 996
Location: Berkshire, UK
kakemoms wrote:
Hi

I am trying to compute CRC-16 for a SD Card, and I found the table-based version given in the source repository (http://www.6502.org/source/integers/crc.htm) was fast.

Anyway, the results I get are not the same as those reported by the SD Card. Anyone knows if the code is ok? :?

Only the initialisation commands need a CRC byte and those are all fixed sequences. You don't need to send CRCs when reading or writing sectors.

_________________
Andrew Jacobs
6502 & PIC Stuff - http://www.obelisk.me.uk/
Cross-Platform 6502/65C02/65816 Macro Assembler - http://www.obelisk.me.uk/dev65/
Open Source Projects - https://github.com/andrew-jacobs


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 5:10 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
drogon wrote:
kakemoms wrote:
Hi

I am trying to compute CRC-16 for a SD Card, and I found the table-based version given in the source repository (http://www.6502.org/source/integers/crc.htm) was fast.

Anyway, the results I get are not the same as those reported by the SD Card. Anyone knows if the code is ok? :?


Are you sure the SD card is using CRC? My (somewhat new and possibly limited) experience is that they don't use CRCs when used in SPI mode - that's assuming you're using SPI mode. However you do need a few CRCs in the initial setup commands but you don't need to calculate these as they're fixed and well known anyway.

My experience is based on writing an SD card filesystem in the past few weeks which uses SPI to communicate with the SD card and other than two (I think) cases for the setup commands, I don't use CRCs at all.

You may actually want CRCs though, in which case just ignore my witterings :)

Cheers,

-Gordon


Well, I have been using it without CRC16 checking until now, but the block data is sometimes corrupted, so I want to check it for that reason.

I am not sure its a valid CRC code, but all 512-byte block transmissions (from the card) end with two extra bytes. If those are not the CRC16 code, I don't know what they are. I have also checked the block data with Active Disk Editor, and it looks ok (and it does not contain the two extra bytes).

BitWise wrote:
Only the initialisation commands need a CRC byte and those are all fixed sequences. You don't need to send CRCs when reading or writing sectors.


I am not talking about command CRC checking. That uses CRC7, not CRC16.


Last edited by kakemoms on Mon Mar 04, 2019 5:17 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 5:16 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
Chromatix wrote:
Not only different IVs, but different polynomials, output XOR filters, and even evaluation directions. CRC can be a real mess, *especially* the 16-bit ones.

Find out exactly which CRC variant is in use here. Until you know that, you won't be able to produce matching values.


All docs indicate CCITT, meaning a 1021 polynomial (normal coding). Maybe I should check the other polynomial variants..

I also found an online coder so I will try to run some test strings with that: https://www.scadacore.com/tools/programming-calculators/online-checksum-calculator/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 5:22 pm 
Offline
User avatar

Joined: Wed Feb 14, 2018 2:33 pm
Posts: 1467
Location: Scotland
kakemoms wrote:
drogon wrote:
kakemoms wrote:
Hi

I am trying to compute CRC-16 for a SD Card, and I found the table-based version given in the source repository (http://www.6502.org/source/integers/crc.htm) was fast.

Anyway, the results I get are not the same as those reported by the SD Card. Anyone knows if the code is ok? :?


Are you sure the SD card is using CRC? My (somewhat new and possibly limited) experience is that they don't use CRCs when used in SPI mode - that's assuming you're using SPI mode. However you do need a few CRCs in the initial setup commands but you don't need to calculate these as they're fixed and well known anyway.

My experience is based on writing an SD card filesystem in the past few weeks which uses SPI to communicate with the SD card and other than two (I think) cases for the setup commands, I don't use CRCs at all.

You may actually want CRCs though, in which case just ignore my witterings :)

Cheers,

-Gordon


Well, I have been using it without CRC16 checking until now, but the block data is sometimes corrupted, so I want to check it for that reason.

I am not sure its a valid CRC code, but all 512-byte block transmissions (from the card) end with two extra bytes. If those are not the CRC16 code, I don't know what they are. I have also checked the block data with Active Disk Editor, and it looks ok (and it does not contain the two extra bytes).

I am not talking about command CRC checking. That uses CRC7, not CRC16.


Right. You get those extra 2 bytes even with CRCs turned off. At least I do - at which point I ignore them. I read the 512 bytes, then do 2 more dummy reads. Same for writing, I send the 512 bytes, then 2 0xFF bytes. The disk editor is probably hiding them from you as it's only going to present the 512 data bytes and not the CRCs (I guess - I've just been block dumping my cards from a Linux PC to test my writes and reads, and from Linux I only see the 512 byte data blocks).

I've not noticed any data corruption though - but then I'm not quite ready for stress testing it to any great degree yet.

From what I gathered, CRC is off when using SPI by default (except the initialisation commands), so you need to send the right command to turn it on and then presumably write the correct crc16 bytes for each block written too. I've never turned it on in my code.

Cheers,

-Gordon

_________________
--
Gordon Henderson.
See my Ruby 6502 and 65816 SBC projects here: https://projects.drogon.net/ruby/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 6:33 pm 
Offline

Joined: Sat Dec 13, 2003 3:37 pm
Posts: 1004
Chromatix wrote:
Not only different IVs, but different polynomials, output XOR filters, and even evaluation directions. CRC can be a real mess, *especially* the 16-bit ones.

Find out exactly which CRC variant is in use here. Until you know that, you won't be able to produce matching values.

Yea, "CRC" is not enough to be specific as to WHICH CRC. MD5, for example, is quite specific, but CRC is more a technique than a specific algorithm or values, so knowing the precise CRC equation is important.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 7:37 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
Ok. I have tested Paul Guertin's code with these databytes:

Code:
TESTDATA
        byte $01,$02,$03,$04,$05,$01,$02,$03,$04,$05,$01,$02,$03,$04,$05,$06


Those 16 bytes give me a CRC of $19 $8D.

Running the same 16 bytes through https://www.scadacore.com/tools/programming-calculators/online-checksum-calculator/ gives:

CRC-16-CCITT
(X.25, V.41, HDLC, XMODEM, Bluetooth, SD, many others; known as CRC-CCITT)
Normal 0x1021 86 0C

Strangely enough, this CRC generator: https://www.lammertbies.nl/comm/info/crc-calculation.html gives:

CRC-CCITT (0xFFFF) 0x8D19

Very strange.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 8:09 pm 
Offline

Joined: Wed Mar 02, 2016 12:00 pm
Posts: 343
drogon wrote:
Right. You get those extra 2 bytes even with CRCs turned off. At least I do - at which point I ignore them. I read the 512 bytes, then do 2 more dummy reads. Same for writing, I send the 512 bytes, then 2 0xFF bytes. The disk editor is probably hiding them from you as it's only going to present the 512 data bytes and not the CRCs (I guess - I've just been block dumping my cards from a Linux PC to test my writes and reads, and from Linux I only see the 512 byte data blocks).

I've not noticed any data corruption though - but then I'm not quite ready for stress testing it to any great degree yet.

From what I gathered, CRC is off when using SPI by default (except the initialisation commands), so you need to send the right command to turn it on and then presumably write the correct crc16 bytes for each block written too. I've never turned it on in my code.

Cheers,

-Gordon


Ok. It suggests I have to turn CRC on.

I only have reading working at the moment, so it is basically to check for errors without having to re-read each block which would take more time. Actually for a slow 6502 it may take less time if you run the SPI at 25MHz, but I plan to run the 6502 at around 28MHz.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 8:33 pm 
Offline
User avatar

Joined: Wed Feb 14, 2018 2:33 pm
Posts: 1467
Location: Scotland
kakemoms wrote:
drogon wrote:
Right. You get those extra 2 bytes even with CRCs turned off. At least I do - at which point I ignore them. I read the 512 bytes, then do 2 more dummy reads. Same for writing, I send the 512 bytes, then 2 0xFF bytes. The disk editor is probably hiding them from you as it's only going to present the 512 data bytes and not the CRCs (I guess - I've just been block dumping my cards from a Linux PC to test my writes and reads, and from Linux I only see the 512 byte data blocks).

I've not noticed any data corruption though - but then I'm not quite ready for stress testing it to any great degree yet.

From what I gathered, CRC is off when using SPI by default (except the initialisation commands), so you need to send the right command to turn it on and then presumably write the correct crc16 bytes for each block written too. I've never turned it on in my code.

Cheers,

-Gordon


Ok. It suggests I have to turn CRC on.

I only have reading working at the moment, so it is basically to check for errors without having to re-read each block which would take more time. Actually for a slow 6502 it may take less time if you run the SPI at 25MHz, but I plan to run the 6502 at around 28MHz.


I have the luxury of coding all mine in C on an ATmega that's the host to my 6502 system. Currently running the SPI clock at 8Mhz and using one of the sparkfun level shifter boards - 8Mhz is the max. possible on a 16Mhz ATmega. (after running it at 400Khz during initialisation) I've no plans to turn data CRCs on if I can possibly avoid it...

What sort of filing system are you looking at? I'm writing one loosely based on ProDOS8... So max. 32MB 'disk' but it reads a standard MBR, so 4 disks of 32MB each is possible - for now.

(currently have an 8GB microSD plugged in - all that space ... ah well!)

Cheers,

-Gordon

_________________
--
Gordon Henderson.
See my Ruby 6502 and 65816 SBC projects here: https://projects.drogon.net/ruby/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 9:13 pm 
Offline
User avatar

Joined: Tue Mar 02, 2004 8:55 am
Posts: 996
Location: Berkshire, UK
drogon wrote:
What sort of filing system are you looking at? I'm writing one loosely based on ProDOS8... So max. 32MB 'disk' but it reads a standard MBR, so 4 disks of 32MB each is possible - for now.

(currently have an 8GB microSD plugged in - all that space ... ah well!)

I bought a load of second hand 128MB and 256MB microSD cards a few years ago for retro computing projects - Largely pulls from old Nokia and Blackberry phones it seems. I also have some (possibly too many) 256MB compact flash cards and IDE adapters.

When I tried to buy cheap low capacity USB thumb drives last summer the smallest I could get was 2GB (White porcelain with a blue Chinese print - quite fetching). A quick search just now shows 4GB is becoming the new low limit.

_________________
Andrew Jacobs
6502 & PIC Stuff - http://www.obelisk.me.uk/
Cross-Platform 6502/65C02/65816 Macro Assembler - http://www.obelisk.me.uk/dev65/
Open Source Projects - https://github.com/andrew-jacobs


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 9:28 pm 
Offline
User avatar

Joined: Wed Feb 14, 2018 2:33 pm
Posts: 1467
Location: Scotland
BitWise wrote:
drogon wrote:
What sort of filing system are you looking at? I'm writing one loosely based on ProDOS8... So max. 32MB 'disk' but it reads a standard MBR, so 4 disks of 32MB each is possible - for now.

(currently have an 8GB microSD plugged in - all that space ... ah well!)

I bought a load of second hand 128MB and 256MB microSD cards a few years ago for retro computing projects - Largely pulls from old Nokia and Blackberry phones it seems. I also have some (possibly too many) 256MB compact flash cards and IDE adapters.

When I tried to buy cheap low capacity USB thumb drives last summer the smallest I could get was 2GB (White porcelain with a blue Chinese print - quite fetching). A quick search just now shows 4GB is becoming the new low limit.


I know :-( It's a bit ridiculous, but I really didn't want to go to floppys or my little retro project, much as I love my 40 year old Apple II ones that still work.. So I'm sort of pretending an SD card is a floppy - after all, it's about the same flexibility as a 3.5" disk is... ;-)

-Gordon

_________________
--
Gordon Henderson.
See my Ruby 6502 and 65816 SBC projects here: https://projects.drogon.net/ruby/


Top
 Profile  
Reply with quote  
 Post subject: Re: Problems with CRC-16
PostPosted: Mon Mar 04, 2019 10:39 pm 
Offline

Joined: Sat Dec 13, 2003 3:37 pm
Posts: 1004
drogon wrote:
I know :-( It's a bit ridiculous, but I really didn't want to go to floppys or my little retro project, much as I love my 40 year old Apple II ones that still work.. So I'm sort of pretending an SD card is a floppy - after all, it's about the same flexibility as a 3.5" disk is... ;-)

I know what you're talking about. All that "wasted space".

For a Forth standpoint, a 16 bit Forth that is, 65MB is the peak, since you can only have 65,000 "screens" (which are 1K each).

When I think about 65,000 screens - man, that's a lot. On the other, it's a lot of wasted space (screens are simple, not efficient :) ).

But the key point, to me, when thinking about this stuff, and off putting to me, was that, yea, using an SD Card as a floppy (or, pretty much anything), is all well and good, but what you (I) really want is the ability to copy data off of the system. For that, you need TWO "floppies".

Yea, I know, lots of work was done with one floppy back in the day, I, too, had a single floppy drive in the dark ages.

But, it sure is nice to NOT have to "swap floppies", especially something as fiddly as a SD Card. (Edit -- can you imagine trying to copy a 8MB section with a 32K RAM buffer? 256 swaps later..."Is that a callous on my thumb? Why is my flash connector loose?")

So, that's where I always stalled out. "Just put in an IDE SD Card". Sure ok, but, then..hey -- how do I make copies. "What a pain!"

Which is why I'm where I am today fixated on USB. One USB connector and a Hub, and "suddenly", I have 4 "floppies" (or whatever, more than one certainly). From here everything collapsed in to the singularity of using the RPI as an I/O controller. Maybe I'll call the project Singularity instead of SuperPI/O.

(Which brings up "If you have network ability, who needs a second drive?". Gah...)


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: