LevelX performance

Hello,
we are using LevelX+FileX+USBX Mass storage device based on Winbond W25Q64FV (8 megabyte) QSPI Flash.
The MCU is Renesas Synergy S5D9.
If we read 2 megabyte file (USBX mass storage device) without LevelX it takes for several seconds.
But if we add LevelX the speed drops dramatically, the same file reads for 4 minutes (It is very low speed for our application).
In our application we writes this file (from MCU side) in fragments of approximately 200 bytes.
The maximum size of used memory is our application is less than 3 megabytes.
Please look at thread stack settings attached.
Is it any way to speed up LevelX performance?

Thanks, dxwak.

 

Parents
  • I suspect the issue is that when LevelX erases a QSPI block, it has to wait for the block erase to finish beofre it can do any other operations on the QSPI memory, and by default a delay of 1 threadx timer tick (10ms) is used in between the polling of  the QSPI to get it's status. In SSP 1.5.1 you can provide a user callback to do what you want, that will bypass this 10ms delay :-

     

    to specify the callback, add it the the configuration of the sf_memory_qspi_nor :-

    and just provide a function that has no delay in it, then the QSPI will be polled repeatedly until it has completed the erase (that should take alot less time than 10ms!)

  • Dear Jeremy, thank you for fast reply.
    As you can see we do not have problem with writing speed, however we could use your useful information in another projects. 
    Our problem is with reading speed.
    We tried to speed up reading speed using LevelX by using LX_DIRECT_READ define. 
    During testing we espied file corruption using LevelX.
    Sorry for late reply we have to made some tests to clarify.
    The summary is given underneath
    1. If we do not use LevelX there is no problem with reading file speed and file corruption, 2 Megabyte file reading time using FileX+MSC USBX FS is 4-5 seconds.
    2. If we use LevelX without LX_DIRECT_READ 2 Megabyte file reading time is 4 minutes ( The file is written in chunks of approximately 200 bytes)
    and file reading time is about 20-30 seconds if it was written in block.
    3. If we use LevelX the file sometimes corrupts when we write in chunks of 200 bytes.
    When file corrupts we could read garbage data(0x00 or 0xff) or we got error while opening file error code is FX_FILE_CORRUPT or FX_NOT_FOUND.
    Please look over example of corrupted file:

    The Fault tolerant data, fault tolerant and fault tolerant service properties of fileX sourse are enabled.
    During startup of the program fx_fault_tolerant_enable is called.

    1. How could we liquidate file errors using LevelX?
    2. Could we use LX_DIRECT_READ? Maybe we also could speedup reading speed by another options?

    SSC screenshots below:

     

    Media open function:

    ERROR_CODE media_open()
    {
    UINT status_connectivity_operation = fx_media_open(&g_fx_media0, (CHAR *)"g_fx_media0", SF_EL_FX_BlockDriver,
    &g_sf_el_fx0_cfg, g_media_memory_g_fx_media0, sizeof(g_media_memory_g_fx_media0));
    if (status_connectivity_operation != FX_SUCCESS) return MEDIA_OPEN_ERROR;

    status_connectivity_operation = g_sf_block_media_lx_nor0.p_api->open(g_sf_block_media_lx_nor0.p_ctrl, g_sf_block_media_lx_nor0.p_cfg);
    if(status_connectivity_operation != SSP_SUCCESS && status_connectivity_operation != SSP_ERR_ALREADY_OPEN) return NOR_OPEN_ERROR;

    status_connectivity_operation = fx_fault_tolerant_enable(&g_fx_media0, fault_tolerant_memory, sizeof(fault_tolerant_memory));
    if (status_connectivity_operation != FX_SUCCESS) return FAULT_TOLERANT_ERROR;

    return NO_ERROR;
    }

    Media test function:

    for (int i = 0; i < 9990; i++)
    {
    ERROR_CODE status_error = _save_results_in_archive("archive_basic.txt");
    if (status_error != NO_ERROR) while(1);
    }

    Save test function:

    ERROR_CODE _save_results_in_archive(char* name_file)
    {

    char buffer_file[200] = { 0 };

    FX_FILE archive_file;
    status_filex_operation = fx_file_open(&g_fx_media0, &archive_file, name_file, FX_OPEN_FOR_WRITE);
    if (status_filex_operation != FX_SUCCESS) return WRITE_FILE_ERROR;

    ...

    status_filex_operation = fx_file_write(&archive_file, buffer_file, 200);
    if (status_filex_operation != FX_SUCCESS) return WRITE_FILE_ERROR;

    fx_media_flush(&g_fx_media0);
    fx_file_close(&archive_file);

    return NO_ERROR;
    }

     

    Best Regards, dxwak.

  • We use winbond W25Q64FV 64 MBIt (8 Megabytes) QSPI device. The filesystem takes about 25% of QSPI device (5-6 files with total size of 2 Megabytes).
    The SSP version is 1.5.3 with IAR Embedded workbench.
    Maybe Logical sector mapping cache size (Level NOR Common) should be increased?

  • The levelX docuementation for SSP 1.5.3 states :-

     

    The generated code doesn't seem to do this for SSP 1.5.3, it reads the number of sectors in the QSPI device and formats it with this number of sectors, if format media is enabled.

     

     

  • So, how could we modify generated code to get result?

     

    We tried:

    1. Increasing LX_NOR_SECTOR_MAPPING_CACHE_SIZE to 128

    2. Disabling LX_DIRECT_READ

    3. Format media during initialization is disabled

    We decreased total sectors of FileX from 16384 to 8192 total sectors.

    The media is formatted with parameters:

     

    uint32_t sector_size = 512;

    uint32_t sector_count = 8192;

    status_operation = fx_media_format(&g_fx_media0, //Pointer to FileX media control block.

    SF_EL_FX_BlockDriver, //Driver entry

    &g_sf_el_fx0_cfg, //Pointer to Block Media Driver

    g_media_memory_g_fx_media0, //Media buffer pointer

    sizeof(g_media_memory_g_fx_media0), //Media buffer size

    (CHAR *)"Volume 1", //Volume Name

    1, //Number of FATs

    256, //Directory Entries

    0, //Hidden sectors

    sector_count, //Total sectors - Hidden Sectors

    sector_size, //Sector size

    8, //Sectors per cluster

    1, //Heads

    1); //Sectors per track

     

    Result the same - file corrupted.

    Now we are using it with custom board.

    We plan to make test project for SK-S7G2 and share it asap.

  • Hello, Jeremy.
    We tried on SK-S7G2 rev 3.1 board, the result is same.
    Please have a look at the projects (IAR + SSP 1.6.0)
    1. With LevelX (MediaTest NOR.zip)
    2. Without LevelX (MediaTest QSPI.zip)
    3. File written with LevelX archive_NOR.txt
    4. File written without LevelX archive_QSPI.txt

    drive.google.com/open
  • Hi Jeremy,
    do you have any update?
    Did you look our example code for SK-S7G2 ?
    We still got corrupted data when use LevelX with FileX and everything is good without LevelX.
    Regarding your note from LevelX documentation:
    "The FileX NOR Flash format should be one full block size of sectors less than the NOR flash provides".
    You told generated code doesn't do it.
    So could we do it from our part of code?
  • Hi Jeremy,
    today we tried:
    1. Change the size of the chunk written (512 bytes, 20000 bytes, 20480 bytes)
    2. Write only English one-byte characters
    3. Call lx_nor_flash_defragment betweentimes
    4. Change the timeout after file write operation
    5. Change media buffer size
    6. Disable fault tolerant
    7. Resolder new QSPI flash
    Everything without luck,
    When we wrote in chunks of 211 bytes the error appeared after 4500-5000 writes.
    When we wrote in chunks of 20 kbytes no error appeared if filesystem had 4 Megabyte size,
    and we had error again if filesystem had 8 Megabyte size.

    So we do not have any idea what we could try more, If you have try please inform us.
    Could we expect it will be fixed in the next SSP version?

    Regards, dxwak
  • Hi Dxwak

    Let me present your information to our LevelX developer next week when he is back from vacation, and see what he recommends. We can't make too many recommendations based on the Synergy MCU but we can probably evaluate your LevelX settings and file operations.

    Regards,
    Janet
    Express Logic
  • Hi Janet,
    Thank you for your reply.
    Looking forward for your recommendations on next week.

    Regards, dxwak
  • Hi Dxwak,

    I am going through the release notes for 1.6.0 for known issues with LevelX, and sadly there is only one:

    7.19 sf_memory_qspi_nor
    Issue ID: 14137
    If QSPI devices larger than 64 MB are used, QSPI (and LevelX) Block media driver reads the data incorrectly
    for addresses greater than 64 MB.
    Applies to: S7, S5, and S3 MCU Series
    Workaround: None

    Since your filesystem is only 8mB this doesn't apply. That's all I could find of known issues. As you probably already know, there aren't many posts to the KB or RR regarding LevelX either. But I sent this information, summarized, to our LevelX developer and should hear something back early next week. At the very least, he can confirm that your usage is reasonable, and possibly make suggestions for finding the problem. I am curious if Synergy may have not implemented it correctly, as Jeremy commented.

    Janet
  • One other comment:

    Next time you post your projects use the export utility (right click on the project panel and look for this usility). Then uncheck debug and synergy files. This will keep the file size down. Your zip files are quite large. Further, anybody downloading these files will have to build the project anyway, and in so doing overwrite the synergy file. And never export debug files. Just bloat.

    Cheers
  • There is an issue using LX_DIRECT_READ with LevelX on Synery. With LX_DIRECT_READ enabled, there is a write in lx_nor_flash_block_reclaim.c that uses the source address for the write, that is in QSPI (This write operation sector copies data from the old mapping location of a sector, to a new mapping location of the sector). However, on Synergy using QSPI with LevelX and LX_DIRECT_READ, this write actually ends up corrupting the sector data. The Synergy QSPI driver doesn't support a write operation where the source data for the write is in the QSPI device.

    I have included a fix in the attached project, in the file src/sf_el_lx_nor/sf_el_lx_nor.c, the original version of this file (synergy\ssp\src\framework\sf_el_lx_nor\sf_el_lx_nor.c is excluded from being built).

    The Dave2D hardware accelerator wasn't enabled in the GUIX studio project, neither were there transfer drivers for the USB controller in the USB Mass Storage Device stack, I have added these.

    I had to disable fault tolerance for Filex (I commented out the call to fx_fault_tolerant_enable() in filex_thread_entry.c) otherwise the USB mass storage device wouldn't open on my Windows 10 PC. Also, there is a note in the FileX manual about using fault tolerance:-

    I am also not 100% sure you will not run into further issues with the way you have your project configured. You have the file system mounted R/W both locally on the Synergy device, and on the host PC via the USB Mass Storage device class. I can see a situation where a filesystem write from the PC could be interrupted by a write from FileX (the Mutex protection in ux_device_msc_media_read() and ux_device_msc_media_write() will only protect individual writes from the PC from being interrupted, but it doesn't ensure the complete filesystem write of file data, and FAT update, will occur uninterrupted by a write from Filex). Also, there is no way to inform the other side that the filesystem has been updated, so that any cached information (e.g. FAT table) can be updated with the information that has just been written to the filesystem, so that both sides are using the latest filesystem data.

     

     

    MediaTest NOR.zip

  • I checked with our LevelX developer and he is not aware of the synergy/driver side of things, which sound like the real culprit.But he did suggest that in general to always make sure LevelX is working before adding FileX perform e.g. r/w logical sector tests to make sure LevelX integration is solid.

    I will get this fix request into the Renesas database so that it can be addressed as soon as possible. It looks like Jeremy has a reasonable workaround. Let me know if you have any questions about excluding files from builds in e2 studio.

    Regards,
    janet
  • Hi Jeremy and Janet.
    After applying Jeremy's fix the problem was solved, so now we could use LevelX and LX_DIRECT_READ in our product.
    Thank you for support provided.
    In our project Synergy device write data on QSPI based FileX system but for host PC the disk is read only.
    Also there is separate mode of operation of our equipment when we connect to the PC we don't write on FileX.
    so I hope we won't have further issues.
    We have no problems with opening USB mass storage on Windows PCs (tested with several PCs from W7 to W10), so we leave fault tolerance as it is.

    Regards, dxwak.
  • Glad to hear Jeremy's fix worked out! I think we will likely post his program in the Knowledge Base as well as apply those fixes for the next release so the next LevelX user doesn't have to run into this. Thanks for bringing this to our attention.

    Regards,
    Janet
  • Hello Jeremy, Janet.
    It seems we found one more bug using Synergy with LevelX .
    If we delete file using fx_file_delete the file system corrupted
    and fx_media_open return FX_BOOT_ERROR.

    1. There is no error if we delete via USB MSC from windows explorer
    2. We got error from time to time if file size less than 1MB.
    3. If we delete file after fill test (2MB file) we always got error.
    Just press the TEST button, wait while test finished with 9991 records,
    after it delete the file, so the file system got corrupted.
    4. LX_DIRECT_READ doesn't matter.
    5. There is no problem if we do not use LevelX.

    I add delete file button to GUI, updated project attached.
    Please check.MediaTest NOR delete.zip

Reply
  • Hello Jeremy, Janet.
    It seems we found one more bug using Synergy with LevelX .
    If we delete file using fx_file_delete the file system corrupted
    and fx_media_open return FX_BOOT_ERROR.

    1. There is no error if we delete via USB MSC from windows explorer
    2. We got error from time to time if file size less than 1MB.
    3. If we delete file after fill test (2MB file) we always got error.
    Just press the TEST button, wait while test finished with 9991 records,
    after it delete the file, so the file system got corrupted.
    4. LX_DIRECT_READ doesn't matter.
    5. There is no problem if we do not use LevelX.

    I add delete file button to GUI, updated project attached.
    Please check.MediaTest NOR delete.zip

Children
  • Hi dxwak

    First, why are you trying to (are you trying to ) run FileX locally on a Synergy device concurrently with mass storage device g_ux_device_class_storage?

    I am not sure why LevelX makes a difference. But it would be good to understand what you are trying to do.

    Thanks,
    Janet
  • Hi Janet,
    Thank you for your reply.
    I needn't to run FileX and MSC USBX concurrently.

    My device works on three independent modes
    1. Log the data using FileX LevelX QSPI storage
    2. View logged data on the device and delete the
    data if needed. So here I got the error. If
    I delete the file using fx_file_delete next
    time booting the device up with fx_media_open
    I got FX_BOOT_ERROR
    3. View and copy saved data on the PC while connect
    the device to the PC. The user cant delete the
    data from the PC. The disk runs as read-only device.

    In my test project it runs concurrently because it is
    just convinient to observe the disk after making some
    operations from FileX.
    I didn't connect the board to the USB while make the test.
    Moreover I delete USBX MSC stack from project and made the
    test again, the problem still exist.
    Hope it could be solved.

    Regards, dxwak.
  • Hi dxwak,

    I am trying to reproduce your actions  on start up, but I'm afraid the details above are a bit ambiguous to me. I can see the files from the PC, and as you observed I cannot delete a file from the PC. 

    Here is how I am running it (no changes to your project except to turn off optimization, always a BAD idea during debugging):

    1. The device is connected to PC by the J5 connector. I  boot the project from the debugging. 

    2. Am I supposed to format the media? If I do the button changes to Wait...and eventually completes. 

    3. What is the purpose of "test". By the way, what records is the "test" counting? It too eventually completes with no error.  Anyway, when "test" is running, only the "Menu" button is visible. 

    4. Then I try to delete a file. But I don't see anything happening. I put a breakpoint in delete_file() and it errors out (0x04 which is file not found). I tried this test a dozen times so the exact details and order of are perhaps confused in my head.  

    Please clarify 

    "Log the data using FileX LevelX QSPI storage" - who where what is this data logged? 

    "View logged data on the device and delete the data if needed." Are you viewing the data from the PC /windows explorer or from the debugger? 

    "So here I got the error. " Please clarify which this error is, is it reported on the LCD or do you set a breakpoint in the debugger somewhere?

    "If I delete the file using fx_file_delete next time booting the device up with fx_media_open I got FX_BOOT_ERROR" Still trying to reproduce this

     

    Thanks, apologies if I seem obtuse.

     

    Janet

  • Hi Janet,
    sorry for misunderstanding.
    For the first there are two projects, one is for our custom board and another one is uploaded example project for SK-S7G2 board where I could also reproduce the problem.
    Previously you asked me why need I to run FileX and MSC USBX concurrently, so I told that I needn't and described about my custom board project.
    Let's concetrate with SK-S7G2 current project because you have hardware to check.
    Regarding your questions:
    1. > The device is connected to PC by the J5 connector. I boot the project from the debugging.
    ok, but connectioan via J5 is optional, if you don't connect to PC via J5 connector, the problem still exist.
    2. > Am I supposed to format the media? If I do the button changes to Wait...and eventually completes.
    Yes, when you run first time you need to format the media. While media not formatted you couldn't observe disc or use FileX.
    Please press format media button and wait, I recorded a video:
    www.dropbox.com/.../format.mp4
    If you power cycle the board there are no errors and if you connect to PC you could observe disk with empty archive.txt file.
    3. > What is the purpose of "test".
    > By the way, what records is the "test" counting? It too eventually completes with no error.
    > Anyway, when "test" is running, only the "Menu" button is visible.
    Now I would like to fill archive.txt file with some data.
    It writes 200 bytes (from 0x00 till 0xc7) for 9991 times.
    Please wait while writes completes.
    If you power cycle the board everythigs seems to be okay, the number of records readed from FileX succesfully.
    If you connect the board to the PC you could observe the disk with about 2 Megabyte file.
    And if you look into the file (I used Fhred) you could see the file saved without any errors.
    I recorded videos:
    Start TEST:
    www.dropbox.com/.../test_start.mp4
    End TEST:
    www.dropbox.com/.../test_end.mp4
    4. > Then I try to delete a file. But I don't see anything happening. I put a breakpoint in delete_file() and it errors out (0x04 which is file not found).
    > I tried this test a dozen times so the exact details and order of are perhaps confused in my head.
    Now delete the file by pressing the DELETE FILE button.

    By default I do not put GUI messages out. 
    If you would like to see GUI output modify menu_window_handler by adding show_error() function in FILE_DELETE_BUTTON_EVENT:

    case GX_SIGNAL(FILE_DELETE_BUTTON_ID, GX_EVENT_CLICKED):
    if (current_status.code == NO_ERROR && !format_button_wait && !test_button_wait && !delete_button_wait)
    {
    delete_button_wait = true;
    current_status = delete_file();
    delete_button_wait = false;
    show_error();
    }
    break;

    For the first time fx_file_delete return SSP_SUCCESS, for the second time its return file not found.
    But if I power cycle the board to call fx_media_open, or open fx_media_open without power cycle it returns FX_BOOT_ERROR (Code 0x01).
    So the file system is broken, also I couldn't observe the file system on PC.
    Thats the problem after deleting testing file the file system is got corrupted.

    The video without error messages:

    https://www.dropbox.com/s/aty7ah0ziv5vg8o/delete_file_media_open_error.mp4?dl=0

    In the start of the video I power cycle the board and fx_media_open return FX_SUCCESS, but if I delete the file the file system is broken and fx_media_open return FX_BOOT_ERROR.

    And with error messages from delete button:

    www.dropbox.com/.../delete_file_status.mp4


    If you would like to retry the error please make steps 2-4 (format, write, delete) again.

    About clarifying questions I told about my custom board and project.


    If you have any other questions please do not hesistate to ask.
    Please inform me if you could reproduce the problem on your SK-S7G2 board.

    Regards, dxwak.

  • Hi Dxwak,

    Thanks, that helps to clarify things. Appreciate the short 'cameo' videos. A picture is worth a thousand words.

    When I power cycle the device, or reboot in the debugger, the 'auto initialization' fx_media_open always fails with the FX_BOOT_ERROR. When I click on Format Media button it always succeeds. This is regardless if the previous session included deleteing the file or not:

    Format Media -> Test -> Delete File

    Do you call fx_media_flush before closing everything down?

    Also, is there any way to shorten the test? It takes a super long time to wait for the 9991 records. Is it necessary to generate 2mB test file or can we get by with something smaller? Would smaller tests have the same problem with LevelX enabled or only the larger files. This sounds reminiscent of your previous issue with LevelX where the QSPI/(or synergy driver) is limited an memory source locations it can process. But Jeremy's patch should have resolved that, right? So this is something different now.

    I'll work on it more tomorrow.
  • Hi Janet,
    thank you for your patience.

    > When I power cycle the device, or reboot in the debugger, the 'auto initialization' fx_media_open always fails with the FX_BOOT_ERROR.
    > When I click on Format Media button it always succeeds.
    > This is regardless if the previous session included deleteing the file or not:
    This is very strange, if I don't delete the file or just create the file everything works well.

    > Do you call fx_media_flush before closing everything down?
    Yes I call fx_media_flush after checking, writing, or delete the file.

    > Also, is there any way to shorten the test? It takes a super long time to wait for the 9991 records. Is it necessary to >generate 2mB test file or can we get by with something smaller? Would smaller tests have the same problem with LevelX >enabled or only the larger files.

    Yes, you could make the test faster. If I make 10 records everything works well, but if I make 100 records the problem appeared.
    Please change the number of records in main_thread_entry() function to:
    if (test && count_records <= 99)

    > This sounds reminiscent of your previous issue with LevelX where the QSPI/(or synergy driver) is limited an memory
    > source locations it can process. But Jeremy's patch should have resolved that, right? So this is something different now.

    Yes these problem is alike but now unfortunately not the same.
    I used Jeremy's patch, without it writing is not reliable.

    I made a full test video, it started with Media open error, so I FORMAT_MEDIA, so I wait it done.
    Then I power down and up the system several times without any error, the file system is ok, and number of records is zero. After it I made test with 100 records. So again power cycle board several time, the file system is ok and shows 100 records. In the end I delete the file, FX_SUCCESS status received, and after power cycle the board I got FX_BOOT_ERROR from fx_media_open.
    Link to the video:
    www.dropbox.com/.../test_full.mp4

    Regards, dxwak.
  • The issue occurs when FileX starts to release sectors after the file has been deleted by FileX. In sf_el_fx, the ioctl for sector release is called with the sector number (start_sector value is passed in) :-

            ret_val = p_block_media->p_api->ioctl(p_block_media->p_ctrl,
                                                  SSP_COMMAND_CTRL_SECTOR_RELEASE,
                                                  (void *) start_sector);

    However, in sf_block_media_lx_nor, the value passed in is used as a pointer :-

     

    and that pointer is used to call lx_nor_flash_sector_release():-

    so levelx releases the wrong sector.

    However, if in sf_el_fx, the ioctl for sector release is called passing the address of the variable holding the sector number to release :-

            ret_val = p_block_media->p_api->ioctl(p_block_media->p_ctrl,
                                                  SSP_COMMAND_CTRL_SECTOR_RELEASE,
                                                  (void *) &start_sector);

    then the correct sector is passed to lx_nor_flash_sector_release() and the filesystem is not corrupted. The attached project has this fix in src\sf_el_fx\sf_el_fx.c the original version should be excluded from the build.

    SSP_1_6_0_S7G2_SK_LevelX_Sector_Release.zip

  • Dear Janet and Jeremy,
    after applying fix provided we do not see any problem now.
    Thank you for your support.
    Hope it could be helpful for other Synergy platfom users.

    Regards, dxwak.