Register

Bfile and system errors

Discuss issues related to the Casio Prizm 3rd party development
Senior Member
Posts: 141
Joined: Wed May 16, 2012 2:50 pm
Location: Portugal
Calculators: Casio fx-CG20

Bfile and system errors

Postby gbl08ma » Fri Jul 12, 2013 1:54 pm

I am rewriting my Utilities add-in and this time I am pretty sure there are no memory leaks. This add-in heavily uses the storage memory (it writes and deletes files in a short period of time).

Of course, because of this heavy usage, the filesystem gets slower and needs optimization. When it is optimized, my code runs absolutely fine. However, as it gets "unoptimized", my add-in starts to give system errors all the way. I am sure it's not memory leaks in my code! I have traced down the problem to the Bfile syscalls.
For example, calling writefile a lot of times on the same handle sometimes yields a system error. Calling Bfile_DeleteFile (on a closed file of course!) sometimes also results on an error.

I'd like to know if there's a way to avoid these errors and if they may be related to updating the screen at the same time that we use the Bfile calls (maybe Simon can tell if the DMA operations break the flash writing process).
If these errors can't be avoided (the OS avoids them somehow, even if the filesystem becomes slower!), I'd like to know I there's a way to tell if the filesysten needs optimization so that my add-in can call APP_OPTIMIZE when needed. (as I said, the errors do not happen when the filesystem is optimized)

Please help!

Senior Member
User avatar
Posts: 605
Joined: Sat Sep 15, 2012 6:59 am
Location: Krautland ****
Calculators: Casio fx-7400GII, Casio fx-7400GII (SH4), Casio fx-9750GII, Casio fx-9750GII (SH4), Casio fx-9860G, Casio fx-9860G SD, Casio fx-9860G Slim, Casio fx-9860GII SD, Casio fx-9860GII SD Power Graphic 2, Casio Classpad 330 plus, Casio fx-CG20, Casio fx-CG50, Casio Classpad fx-CP400

Re: Bfile and system errors

Postby SimonLothar » Fri Jul 12, 2013 5:58 pm

Without the source, I can only guess.

The first and rather nasty trap is writing an odd number of bytes. The low level flash routines work word-oriented (2-aligned) and go kinda berserk, once you write an odd number of bytes and try to write repeately to the same file. But usually they hang instead of entering some system error. The nasty thing is, that the error does not occur immediately, but after the next try to write to the file.

Another trap may be, that you cannot overwrite a file. To overwrite, you have to delete, create, write. If I remember well, trying to overwrite a zero-bit with a one-bit leads to a system error.

Or it is some "out-of-space"-problem with the storage memory. Do you check for the free space, before creating a file?

The DMAC shouldn't be the problem...unless you use some self-written timer-handlers, which are not cooperative and use DMA.

Do you use MPU-overclocking?
I'll be back!

Senior Member
Posts: 141
Joined: Wed May 16, 2012 2:50 pm
Location: Portugal
Calculators: Casio fx-CG20

Re: Bfile and system errors

Postby gbl08ma » Fri Jul 12, 2013 9:05 pm

I know it's hard to find the problem without looking at the code, but I have limited internet access right now (only on the phone) and the project is already quite big (over 300 KB of code, and maybe over 10K lines).
I will post the code later, but note that I'm not using miniSDK but rather GCC with libfxcg (the Bfile relevant code should be compatible, anyway).

Leaving alone the word-aligned problem for now, I am sure I am not overwriting any file as I always delete and recreate before writing, even when I'm just appending contents to a file. And most of the time I write everything at once, and next time I'm going to do anything with that file, it just gets deleted and recreated before being written again.
So I don't think the word-alignment would be of importance in this case.

I currently do not check for the amount of free space in memory, but my code does check if the create and open operations fail or not (create would fail if there was no space left) and only
proceeds to writing when they report success. Anyway, both the emulator I use for testing and my calculator have multiple megabytes of free space.

I am not using any custom timer handlers (I'm using the Timer_* syscalls for managing timers).

Furthermore, I have already traced a error once or twice to a DeleteFile call, which takes no arguments other than a file path. What bugs me is that the errors don't always occur even if I repeat exactly the same conditions (same content, same code, same size), giving the user the impression my add-in only works sometimes when the fault seems to be on the OS calls.

I read on the CG 20 manual a section that says that the system would automatically optimize the filesystem when it's needed (it is page 423 or 11-13 of the software manual). I don't know if that's actually true (never saw it working), but if it is, that "magic" isn't handled automatically by Bfile, or so it seems.

On my tests I don't have anything overclocked (even because overclocking does nothing on the emulator other than making the RTC go crazy).

Senior Member
User avatar
Posts: 605
Joined: Sat Sep 15, 2012 6:59 am
Location: Krautland ****
Calculators: Casio fx-7400GII, Casio fx-7400GII (SH4), Casio fx-9750GII, Casio fx-9750GII (SH4), Casio fx-9860G, Casio fx-9860G SD, Casio fx-9860G Slim, Casio fx-9860GII SD, Casio fx-9860GII SD Power Graphic 2, Casio Classpad 330 plus, Casio fx-CG20, Casio fx-CG50, Casio Classpad fx-CP400

Re: Bfile and system errors

Postby SimonLothar » Fri Jul 12, 2013 9:53 pm

I will try to reproduce the error with a minimum program, which creates, writes and deletes repeatedly.
Until I finished this, I do not need the source for the time being.
Perhaps you should post the system error information (f. i. address).
I'll be back!

Senior Member
Posts: 141
Joined: Wed May 16, 2012 2:50 pm
Location: Portugal
Calculators: Casio fx-CG20

Re: Bfile and system errors

Postby gbl08ma » Sat Jul 13, 2013 12:14 am

I too have thought of building such a small test program. But I thought I should ask first so that if someone knew the reason of these errors it would save me the work.

Just in case it is relevant, I'm always manipulating files inside a folder (one level, not more).

It's hard to debug because the target and PC of the system error change every time. I've seen cases where both are just x00000000. I think it is always a TLB error. It's as if the execution had jumped to some illegal address as a consequence of the Bfile call.

I rarely reboot the calc when a system error appears. I usually press Menu and then switch to another app, then back to mine. This has allowed me to see that sometimes the error with Bfile is so bad that the filesystem goes into some strange mode, as if it had been unmounted on the Test Mode (add-ins won't appear in the main menu, for example). Rebooting the calc or unmounting and mounting the SMem usually fixes the problem when this happens.

Could it be, that as the filesystem is modified, the add-in position in flash changes, causing the target to be invalid? (similar to what may happen when calling the optimization app from an add-in)
That could also explain why the error doesn't happen when the filesystem is optimized (as things are organized in SMEM, relocation of the add-in file doesn't happen).

Senior Member
User avatar
Posts: 605
Joined: Sat Sep 15, 2012 6:59 am
Location: Krautland ****
Calculators: Casio fx-7400GII, Casio fx-7400GII (SH4), Casio fx-9750GII, Casio fx-9750GII (SH4), Casio fx-9860G, Casio fx-9860G SD, Casio fx-9860G Slim, Casio fx-9860GII SD, Casio fx-9860GII SD Power Graphic 2, Casio Classpad 330 plus, Casio fx-CG20, Casio fx-CG50, Casio Classpad fx-CP400

Re: Bfile and system errors

Postby SimonLothar » Sat Jul 13, 2013 5:39 am

gbl08ma wrote:Could it be, that as the filesystem is modified, the add-in position in flash changes, causing the target to be invalid? (similar to what may happen when calling the optimization app from an add-in)
That could also explain why the error doesn't happen when the filesystem is optimized (as things are organized in SMEM, relocation of the add-in file doesn't happen).
Good point! Your observations lead to this conclusion. If optimization is called automatically from inside of the corresponding Bfile-calls, Prizm add-ins, which change the storage memory, will fatally fail eventually, indeed. Perhaps there is a way to suspend auto-optimization temporarily...time to investigate.
I'll be back!

Senior Member
Posts: 141
Joined: Wed May 16, 2012 2:50 pm
Location: Portugal
Calculators: Casio fx-CG20

Re: Bfile and system errors

Postby gbl08ma » Sat Jul 13, 2013 10:03 am

Even more interesting than disabling auto-optimization, would be to get its effects mitigated: it would be optimal if the filesystem could still be optimized, but the static add-in memory location would be automatically adjusted to the correct one after optimization (so that the target doesn't become invalid).

My biggest question is, "how does the OS do it?". For OS apps (eActivity, etc.) there would be no problem with auto-optimization, since these apps don't reside in the filesystem but rather in the big binary blob that the OS is, and so they never change position.
Then how do add-ins do it? I went looking and from all the Casio-made add-ins for the Prizm, the ones that write to the SMEM are Geometry (when you open a g3p file, add geometry things to it, then save), and Picture Plot (after opening a g3p file, you can do FILE then Save or Save-As).

I have tried "abusing" the Geometry add-in by overwriting multiple times the same g3p file, and of course there was no system error. So add-ins definitely know how to work with the SMEM, perhaps they just do it in a different way from static OS apps.

Senior Member
User avatar
Posts: 605
Joined: Sat Sep 15, 2012 6:59 am
Location: Krautland ****
Calculators: Casio fx-7400GII, Casio fx-7400GII (SH4), Casio fx-9750GII, Casio fx-9750GII (SH4), Casio fx-9860G, Casio fx-9860G SD, Casio fx-9860G Slim, Casio fx-9860GII SD, Casio fx-9860GII SD Power Graphic 2, Casio Classpad 330 plus, Casio fx-CG20, Casio fx-CG50, Casio Classpad fx-CP400

Re: Bfile and system errors

Postby SimonLothar » Sun Jul 14, 2013 9:21 am

I failed to reproduce the error, yet.

I used the following code to exhaust the storage memory.
Code: Select all
unsigned short filename[ FILENAMESIZE ];
int size, i;
int handle;

   Bdisp_SetBacklightLevel( 1 );  // save battery capacity during the long running file-tests

//   size = 0x100000;
   size = 120000;
   for ( i = 0; i < 3; i++ ){
      Bfile_StrToName_ncpy( filename, "\\\\fls0\\CXXY", 0x10A );
      filename[ 11 ] = 0x30 + i;
      filename[ 12 ] = 0;
      DisplayStatusArea();
      Bfile_DeleteEntry( filename );
      Bfile_CreateEntry( filename, 1, &size );
      handle = Bfile_OpenFile( filename, 3, 0 );
      if ( handle >= 0 ){
         Bfile_WriteFile( handle, (void*)0x80000000, size );
         Bfile_CloseFile( handle );
      };
   };

   return;

But even with a remaining space of about 77824 bytes (which is smaller than the 128 KiB flash-sectorsize) the system worked without complaints.
First I wrote some 512 KiB files, then several 1 MiB files. Loop-count always adjusted to nearly exhaust the memory. Deleted, created, wrote. Finally, with about 440000 remaining bytes, I created three 120000 byte-files, which must have led to fragmentation, because 120000 bytes cannot fill a 128 KiB flash-sector completely. No failures.
Undoubtly the system manages to re-use deleted files successfully without manual optimization. But up to now I did not observe, that any harm has been done to the actually running G3A.

As for reinstating an active G3A, which has been moved by optimization, while still running:
That is a stunt! I once wrote an add-in for fx-9860-types, which called the optimization syscall. To achieve this, the calling code has to be run from RAM. The add-in copies a special program to this RAM-area, drops some information in the add-in's main memory directory and passes control to the RAM-code. From RAM the optimization function can be called securely. After the optimization has been finished (the optimization syscall returns to the special program in RAM), the add-ins have to be registered again. Then (we are still in RAM) the system must be instructed to restart the original add-in. Finally the restarted add-in must know (via the above mentioned information in main memory), that it has been restarted after "self-optimization". I suppose, a similar concept will apply to the Prizm.
I'll be back!

Senior Member
Posts: 141
Joined: Wed May 16, 2012 2:50 pm
Location: Portugal
Calculators: Casio fx-CG20

Re: Bfile and system errors

Postby gbl08ma » Sun Jul 14, 2013 9:56 am

I wonder if the size of the g3a and its position in memory influence anything. I think your code actually failed to produce any move of the add-in file in flash, because the g3a was probably the first file to be put in the calc (so it should have come first in the flash), and all those files your add-in created and deleted were written after the flash portion where the add-in resides. So the optimization process never had any need to touch the add-in file (that is, if optimization works like I think it does).

I suggest the following experience, which reproduces conditions more similar to those on which my Utilities add-in runs: put some files in flash before putting there the test add-in. Then optimize the calc to make sure these files come first (or at least are all together and unfragmented). Then copy the test add-in and have it rename, delete and recreate these files that are before it in flash. This way I think the add-in will move more than it did with your setup. This is what I am going to try when I get back to my development environment.

Also a bigger add-in may be more likely to get moved (or not, after all smaller files are easier to move).

And in case you didn't know, the filesystem used by the Prizm appears to be the Fugue FAT filesystem (judging by some strings found in memory). This is a proprietary filesystem and I couldn't find much openly acessible documentation about it.

Hopefully tomorrow I'll be able to release a test version of my add-in along with source code.

Senior Member
User avatar
Posts: 605
Joined: Sat Sep 15, 2012 6:59 am
Location: Krautland ****
Calculators: Casio fx-7400GII, Casio fx-7400GII (SH4), Casio fx-9750GII, Casio fx-9750GII (SH4), Casio fx-9860G, Casio fx-9860G SD, Casio fx-9860G Slim, Casio fx-9860GII SD, Casio fx-9860GII SD Power Graphic 2, Casio Classpad 330 plus, Casio fx-CG20, Casio fx-CG50, Casio Classpad fx-CP400

Re: Bfile and system errors

Postby SimonLothar » Sun Jul 14, 2013 11:35 am

gbl08ma wrote:I wonder if the size of the g3a and its position in memory influence anything. I think your code actually failed to produce any move of the add-in file in flash, because the g3a was probably the first file to be put in the calc (so it should have come first in the flash), and all those files your add-in created and deleted were written after the flash portion where the add-in resides. So the optimization process never had any need to touch the add-in file (that is, if optimization works like I think it does).

I suggest the following experience, which reproduces conditions more similar to those on which my Utilities add-in runs: put some files in flash before putting there the test add-in. Then optimize the calc to make sure these files come first (or at least are all together and unfragmented). Then copy the test add-in and have it rename, delete and recreate these files that are before it in flash. This way I think the add-in will move more than it did with your setup. This is what I am going to try when I get back to my development environment.

Also a bigger add-in may be more likely to get moved (or not, after all smaller files are easier to move).
My G3A is INSIGHT.G3A (104 KiB). I did as you proposed. Deleted INSIGHT.G3A, optimized the storage memory, transferred INSIGHT.G3A again and then ran the test-function on 77824 bytes free space. No errors.

gbl08ma wrote:And in case you didn't know, the filesystem used by the Prizm appears to be the Fugue FAT filesystem (judging by some strings found in memory). This is a proprietary filesystem and I couldn't find much openly acessible documentation about it.
fx_calculators_SuperH_based.chm: Start/fx-CG20/FAT. That's what I found.
I'll be back!

Next

Return to Casio Prizm SDK

Who is online

Users browsing this forum: No registered users and 16 guests