Problems with SD file operations (writing)

Hello,

Is it possible that the SD I/O libraries for the Sakura have a bug for writing right now? It seems that we only get files filled with wrong characters and even corrupt files at the moment.

The codes for writing in our specific case have not been changed, but the output is changed dramatically. (as of 9-1-2018 for as far as we know)

 

-美月

  • Hi Mitsuki,

    Were you able to resolve the issue already? Please wait a little bit more until someone can confirm about your doubt on the Sakura SD I/O library. Thanks for understanding.

    JB
    RenesasRulz Forum Moderator
  • In reply to JB:

    Hello JB,

    No luck yet, the issue is still present. I will wait until someone can confirm about the libraries. Meanwhile I will try to resolve it myself, too.


    -美月
  • In reply to Mitsuki:

    Hello JB,

    I have since uploaded the example program and that one gives problems too. It can be read by the Sakura before it is close();'d, but after that I can't read it back on PC. Is it perhaps known whether there are demands for certain cluster sizes, SD card sizes or anything? I do know about the 8.3 names requirement.

    As for saving within my project, does it give problems if I first manually SD.flush(); and then SD.close();, since SD.close(); flushes as well? The reference does not mention anything about it not being allowed.

    Quick update: I just removed the SD.flush(); attempt to make it work and it stillproduces a file that cannot be opened on a PC. This also means that the file is not read by the Sakura. 


    -美月

  • In reply to JB:

    Hello JB,

    In my last message I told that I tried the demo program and that the data in it cant be read back on a PC (which is a requirement, by the way). I have since tried some more things, and at the moment I can write a file ..sometimes. When I try opening a file, it will not always do it. Generally, from the 2nd attempt on it will succesfully open a file for writing. Once, it worked the first time, but it produced a file of nearly 4GB, that was 0B when I tried to delete it.

    I will post my piece of code below. Note that I changed variable names.

     

    if (SD.exists("filename.txt") && !xOnlyDeleteOnce){ 
    SD.remove("filename.txt");
    xOnlyDeleteOnce = true;
    }

    fileVar = SD.open("filename.txt", FILE_WRITE);

    if (fileVar) {
    fileVar.print(intVar1);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar2);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar3);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar4);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar5);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar6);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(floatVar1, 2); // print two decimal places
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(floatVar2, 2); // print two decimal places
    delay(5);
    fileVar.println(","); // Ends line and inserts cr/lf
    delay(5);
    delay(300); //@TODO test this -> didn't work
    fileVar.close();
    Serial.println("Done writing!");
    }
    else {
    errorFlag = true;
    errorCode = 11;
    }

    // This is followed by 

    if (!fileVar) { //code to return to main scenario }

    I hope that you, or anyone else, can help me fix this problem

    -美月

  • In reply to Mitsuki:

    Solved the problem by adding a 1 second delay before starting to write. It writes normally now, without any errors, and the files can be read back from a PC. I guess the MCU was still busy and thus things went wrong when it tried to write to the SPI bus.

    -美月
  • In reply to Mitsuki:

    ,

    And just when I thought it all worked again, it starts to misbehave. i "solved" my problems by adding delays, suspecting the Sakura was still busy updating registers when it got the SPI requests. It all went very well, until yesterday. Now, the Sakura starts to delete random files from the SD card, even files I am not touching. The only change I have made is that I have inserted tabs in my printed text, as follows
    fileVar.print("\t\ttabbed text\t\tnext field");
    One would assume this is a valid operation that does not remove random files.

    Also, is it possible to do something equivalent to "safely remove hardware" in Windows, for example before I shut the MCU down? I know it is supposed to not be needed for FAT16, but a corrupt filesystem would explain this strange behaviour.

    -美月
  • In reply to Mitsuki:

    I'm sorry for no response to this forum. I would like to try your condition. Could you let me know the version or the development environment. The current version of libraries is V2.20. Do you use web compiler or e2studio project?
  • In reply to Okamiya Yuuki:

    Dear Okamiya Yuuki-san,

    I am currently using e2studio. Could you tell me where I can find the library version?

    -美月
  • In reply to Mitsuki:

    Ok. There is no description in libraries. The version is only described template name or default sketch as below.

    If you could, try the latest version V2.20 downloadable at GR-SAKURA web. http://gadget.renesas.com/en/product/sakura.html

    Note that the projects can be used with e2studio V5.4, not for V6.x.

  • In reply to Okamiya Yuuki:

    Dear Okamiya Yuuki-san,

    I just tried using that sketch, but it does not want to work at all. Even the sketch.cpp gives errors. Inserting my files gives errors in that it can't find anything. (header files mostly)
    I am using e2studio version 5.4 already, so that can't be the problem.
    The SD library was not changed between my version and the new one, with the only exception being that GRSAKURA is now __RX600__; not a functional change.

    The SD card problems have been "fixed" in my old project again by increasing the delay I put after my fileVar.close(), as follows
    delay(100); // shorter delays might result in the file not opening. RX63 still busy changing registers maybe?
    File fileVar = SD.open("filename.ext", FILE_WRITE);
    fileVar.print("someText\t\tnext Field\nfirstEntry\t\tsecondEntry");
    fileVar.close();
    delay(100); // 50ms is too little, 100ms feels sluggish.

    Even though delays work, it does make the entire device feel sluggish (you have to wait till the delays and writing is done before the next screen appears and you can continue). It would be really nice if I could get rid of those delays in some ways. Do you have any advice as for how I can accomplish this?

    -美月
  • In reply to Mitsuki:

    I've tried the below code on my e2studio 5.4 (GNURX 14.03).

    /* GR-SAKURA Sketch Template V2.20 */
    #include <Arduino.h>
    #include <SD.h>
    static int intVar1, intVar2, intVar3, intVar4, intVar5, intVar6;
    static float floatVar1, floatVar2;
    static bool errorFlag = false;
    static bool xOnlyDeleteOnce = false;
    static uint8_t errorCode = 0;
    void setup() {
    Serial.begin(9600);

    if (!SD.begin()) {
    Serial.println("Card failed, or not present.");
    while (1)
    ;
    }
    }

    void loop() {
    Serial.println("Any key to start writing.");
    while (!Serial.available())
    ; // wait to press key
    Serial.read(); //dummy

    if (SD.exists("filename.txt") && !xOnlyDeleteOnce) {
    SD.remove("filename.txt");
    xOnlyDeleteOnce = true;
    }
    File fileVar = SD.open("filename.txt", FILE_WRITE);

    if (fileVar) {
    fileVar.print(intVar1);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar2);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar3);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar4);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar5);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(intVar6);
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(floatVar1, 2); // print two decimal places
    delay(5);
    fileVar.print(",");
    delay(5);
    fileVar.print(floatVar2, 2); // print two decimal places
    delay(5);
    fileVar.println(","); // Ends line and inserts cr/lf
    delay(5);
    delay(300); //@TODO test this -> didn't work
    fileVar.close();
    Serial.println("Done writing!");
    }
    else {
    errorFlag = true;
    errorCode = 11;
    }
    }


    The below image show that "FILENAME.TXT" was written, and show the contents in the file. It works fine.

     

     

    Next, I modified the program as below, "delay" was removed.

    /* GR-SAKURA Sketch Template V2.20 */
    #include <Arduino.h>
    #include <SD.h>
    static int intVar1, intVar2, intVar3, intVar4, intVar5, intVar6;
    static float floatVar1, floatVar2;
    static bool errorFlag = false;
    static bool xOnlyDeleteOnce = false;
    static uint8_t errorCode = 0;
    void setup() {
    Serial.begin(9600);

    if (!SD.begin()) {
    Serial.println("Card failed, or not present.");
    while (1)
    ;
    }
    }

    void loop() {
    Serial.println("Any key to start writing.");
    while (!Serial.available())
    ; // wait to press key
    Serial.read(); //dummy

    if (SD.exists("NODELAY.txt") && !xOnlyDeleteOnce) {
    SD.remove("NODELAY.txt");
    xOnlyDeleteOnce = true;
    }
    File fileVar = SD.open("NODELAY.txt", FILE_WRITE);

    if (fileVar) {
    fileVar.print(intVar1);
    fileVar.print(",");
    fileVar.print(intVar2);
    fileVar.print(",");
    fileVar.print(intVar3);
    fileVar.print(",");
    fileVar.print(intVar4);
    fileVar.print(",");
    fileVar.print(intVar5);
    fileVar.print(",");
    fileVar.print(intVar6);
    fileVar.print(",");
    fileVar.print(floatVar1, 2); // print two decimal places
    fileVar.print(",");
    fileVar.print(floatVar2, 2); // print two decimal places
    fileVar.println(","); // Ends line and inserts cr/lf
    fileVar.close();
    Serial.println("Done writing!");
    }
    else {
    errorFlag = true;
    errorCode = 11;
    }
    }


    The below image shows that "NODELAY.TXT" was written, and show the contents in the file. It works fine.

    Could you tell me the version of compiler ? I used GNURX v14.03, that is little bit old.

    Anyway, __RX600__ is activated by -mcpu=rx600. We had integrated the library for GR-SAKURA and GR-CITRUS(RX631) so we use __RX600__ macro.

     
  • In reply to Okamiya Yuuki:

    Dear Okamiya Yuuki-san,

    The version of GNURX I use is 14.03, same as you. 

    Also, when I run the SD writing in its own program, all runs fine. It is only when I remove delays in the entire application that it starts doing strange things. I suspect the program is still busy writing registers or doing something else while the SD card is first being accessed. If I remove the delay after file.close();, random files that I have not accessed seem to disappear, almost as if the FAT table is not yet written properly.

    I also experimented with delays and everything in other places. Right now, I only have a delay before opening a file, then one when I split up long strings (else the string gets cut off), and after closing the file:

    delay(200);
    if (!SD.exists(fileName)) {
    // write a little header
    delay(200);
    file = SD.open(fileName, FILE_WRITE);
    // Below strings are kept at approximately equal length as real code
    file.print("day-month-year hrs:minutes\tfirstInt pos1\tsecondInt pos1");
    delay(100); // 100 ms delay to write its buffer.
    file.print("\tthirdInt pos2\tfourthInt pos2\tfifthInt");
    delay(100); // another 100ms to clear buffer
    file.println("\tnumeralsInt\tfinalInt");
    file.close();
    }
    delay(200); // 200ms delay
    getTime(); // RTCC function, retrieve internal RTCC time and cast to separate variables
    delay(100);
    file = SD.open(fileName, FILE_WRITE);
    // Write data
    if (file) {
    /* block for date and time, then
    * a space, a dash and then the
    * actual logging items.
    */
    file.print(day);
    file.print("-");
    file.print(month);
    file.print("-");
    file.print(year);
    file.print(" ");
    if (hours < 10) {
    file.print("0");
    }
    file.print(hours);
    file.print(":");
    if (minutes < 10) {
    file.print("0");
    }
    file.print(minutes);
    file.print("\t\t\t");
    file.print(internalVar1/divisionFactor1);
    file.print("\t\t\t");
    file.print(internalVar2/divisionFactor1);
    file.print("\t\t\t");
    file.print(internalVar3/divisionFactor2);
    file.print("\t\t\t");
    file.print(internalVar4/divisionFactor2);
    file.print("\t\t\t");
    file.print(fifthInt);
    file.print("\t\t");
    if (time_minutes< 10) {
    file.print("0");
    }
    file.print(time_minutes);
    file.print(":");
    if (time_seconds< 10) {
    file.print("0");
    }
    file.print(time_seconds);
    file.print("\t\t");
    file.println(finalInt);
    // if reading back is made, the \t should be filtered out.
    }
    //file.flush(); // Should not be needed, flushes data to SD card
    file.close();
    delay(200); // Should not be needed, waits 200 ms to close the file. If this isn't done, this (or other) file might get corrupt

    If I isolate this in its own program and run it without delays, everything works fine. If I then put it back in the complete program, things get screwed up. Also, it seems that if I remove most of the tabs in the header writing bit, it can go on for longer without needing the delay to (presumably) write the buffer.

    Hope you can clear up some of my confusions with these new details.

    Yours,

     

    -美月

  • In reply to Mitsuki:

    Mitsuki-san, sorry for delay reply. I'm in biz trip. Let me have time until next week.
  • In reply to Okamiya Yuuki:

    Dear Okamiya Yuuki-san,

    I have since replaced "FILE_WRITE" with "O_WRITE | O_CREAT | O_APPEND", as per the creator of sd2fat's suggestions ( forum.arduino.cc/index.php ). After that, the wriets go a lot faster, and I think that this is what has solved most of my problems. Now, it only really messes up files when it loses power while writing. (presumably the FAT table was being written to.) The O_WRITE | O_CREAT | O_APPEND parameters allow for writing multiple bytes between flushing and, thus, remove a lot of overhead. The FILE_WRITE flushes after every byte.

    Still, I would like to hear your opinion on whether writing to registers for I/O could have caused problems as well.

    As an extra, there is apparently a much newer version of sdfat, you can find it at https://github.com/greiman/SdFat . Is there any possibility it will be included in the Sakura libraries in the near future? Apparently features are much more advanced, but the complexity has also grown a bit.


    Yours,

    -美月

  • In reply to Mitsuki:

    Hi Mitsuki-san,

    Thank you for your confirmation and suggestion.

    CS pin for SD card is assigned 22 pin. Do you use 22 pin for other function?
    Also please let me know whether other SPI pins, 11, 12 and 13 pin are used for other use.

    About SdFat, I will consider to use newer version for the next update opportunity.
    Basically we use Arduino library on their IDE. If there is a difference from original, it would confuse users sometimes.

    Regards,
    Okamiya.