Omnimaga
Calculator Community => Discontinued => Major Community Projects => KnightOS => Topic started by: SirCmpwn on September 27, 2010, 10:52:34 pm
-
Attached is the specification for KFS, please review this if you have time and let me know what kind of feedback you can give. Garbage collection is not fully thought out at the moment, so I know this portion of the document is inaccurate. In addition, one error in this version is the page byte, this should be added so that the file entry looks as follows:
Type ID (11111110b for file) Byte
Directory ID Word ; The directory containing the file
Name (0 delimited)
Page byte ; The flash page the file starts on
Offset Word ; The offset from the start of that page (+4000h) the file is on
-
Will read when I have the chance. Good luck on figuring out garbage collection!
(On an unrelated note, I'm done with the main planning and will begin coding (my OS :P) :))
-
Yay ^_^
-
I'll check this when I have a minute and nice to hear Calcdude84se :D
-
wow nice! Those things really are the most beautifull things to make in life IMO :D Your own working data storage system. Won't this have compatibility issues with the TIOS files? And will it be possible to send data to your calc when KnightOS is installed on it using TIconnect/TiLP? When this won't be possible, which I guess being true, will you write your own data transferring program?
-
To answer your question to the best of my knowledge, yes, the TI-OS file system is wholly incompatible with the KFS. However, he plans a feature that, on first setup, allows you to do a special reformatting. (I don't know if that will be done.)
Providing he implements the protocol, it will be possible to transfer data via TI-Connect or TiLP. (I presume files will be "wrapped" into TI-OS formats for linking programs/TI-OS based calcs.)
-
First question, I plan on having the installer move all of your TIOS files into a directory in KFS when it is reformatted. The VAT (TIOS Filesystem) is completely incompatible (the main reason dual boot is out of the question). I will possibly write my own linking program, but TI Connect support is also planned.
-
Thanks for clarifying. It also appears I forgot to explicitly state the effects/purpose of the special reformatting. :P
-
Yay, does this mean you're done working on the KFS? :D
Also, a random question (I probably don't know what I'm talking about): How would you tell the difference between garbage entries and simply nonexistent ones?
And would 2ND+[OFF] actually turn the calc completely off in the OS?
-
Sadly, there's no way to turn the calc completely off without removing all the batteries. That's just the hardware.
And that random question is confusing me. (Not that I am confused about the FS, but rather the question itself) ???
-
Well, it says that 0000000b is for "garbage" entries, but what if it's already deleted?
EDIT: Oh, wait, they're all in flash. Never mind.
-
Oh, you thought it was RAM. Okay, that makes more sense.
Glad to know you've figured it out. :)
-
Yeah. Well, about that, if you can't turn the calc completely off, why keep everything in flash? Is it just to protect against mem clears?
-
Pretty much, not to mention flash has so much more storage space. (Or in case you have to change the batteries.)
Rather important also if the system can be brought down in two bytes.
di
halt
-
Oh, okay. And just wondering: If it were to delete vars instead of marking them for garbage, would that be more damaging to the flash chip? Meaning does each write to it do just as much wearing no matter how small it is?
-
What wears a flash chip is erasing it (setting all the 0 bits back to 1's)
However, having it clean up after each deletion rather than all at once would be slower and yes, it would probably wear the flash more.
-
I'm thinking of doing a garbage collection at boot, to avoid locking up the entire system for a garbage collection while saving a file. Thoughts?
-
Well, how long will a garbage collection take? If it's no more than ~30sec, I see no problem in making it an atomic operation, IMO.
-
I would estimate it at 10-20 seconds.
-
At least IMO it's fine. It's not like they'll happen at a very great frequency anyway... and since you can completely pause any running program (or, more advanced, just those that are using files in the FS being garbage collected ;)) it doesn't seems like it'd be too bad.
-
Ooo, it might be a good idea to freeze file streams when garbage collections are in progress, instead of only allowing one file at a time. This is something I've been struggling with: how to edit multiple files at once? I know I will probably use filestreams to make it easier on the user to cross page boundaries and such. But I'm still not sure how to handle multiple pages.
-
Ooo, it might be a good idea to freeze file streams when garbage collections are in progress, instead of only allowing one file at a time.
Elaborate, please, I'm confused. ???
This is something I've been struggling with: how to edit multiple files at once?
What's getting in your way?
I know I will probably use filestreams to make it easier on the user to cross page boundaries and such.
Do programs handle flash almost directly, or am I misunderstanding?
But I'm still not sure how to handle multiple pages.
As above, but also, sectors might be a worry too.
-
I think that a solution to the multiple files problem is making programs define the size beforehand. As for programs handling flash directly, I would prefer they use filestreams in the OS to handle it, because the OS will handle the page boundaries. A filestream is a means to reading and writing files or memory, or in KOSes case, only files.
-
You still haven't answered my question of how are multiple files a problem. :P
Also, I thought you earlier had an idea of "edit buffers" or something like that where programs edited pieces of a file one piece at a time in RAM. Was that scrapped?
-
It will probably be scrapped, seeing as most files will probably end up being too big. I don't want to edit files that take up most of available RAM. In KFS, three bytes are dedicated to filesize, and I want to make sure I support that.
As for editing multiple files, if two files are growing in flash alongside each other, there will be significant problems unless I garbage collect every time a byte is written.
-
Why would you be garbage collecting? If one file encroaches on another, you'll have to delete and move it (unless you can't ???), and I recommend you make provisions to be able to skip a certain amount of extra space after file to prevent what you and I described from happening :)
-
Delete+move would mean that the garbage would pile up very quickly.
-
Definitely, hence why I recommend providing extra space at the end of a file :D (And make sure you don't update the size byte in flash until the file is closed :P)
-
Yeah, I'm not sure about that. I'll think it over for a bit. Imagine if two programs were writing large files, multi-page, perhaps. It might garbage collect 10 times over the course of saving them.
-
Garbage collecting at boot seems like a good idea, as long as it won't wear the flash chip faster if it garbage collects even when the memory is not very fragmented
Keep up the good work on KnightOS :D
-
It may count up the garbage and see if it's worth garbage collecting or not.
-
Ah sounds good :)
If someone doesn't reboot for a long while, I assume it will automatically GC when needed?
-
Yes.
-
What if someone decides to replace the batteries?
-
Nothing would be different... O_o
-
Ok good. Also, if you make your own linking software, will we be able to send any kind of file to the calc like on the TI-Nspire? On the TI-Nspire, if you want to send a Gameboy Color ROM, all you do is add a .tns extension at the end and you're set. I assume TI-Connect would not accept such file, though, right?
Also for batteries, will there be some protection against battery removal/dying during GC'ing? Or will it do like on the 83+ and random archived files vanish?
-
It will check the battery state before starting, and probably inform the user that they should change the batteries really quick before the GC starts, and inform them of possible data loss.
As for files, you can send any file you like to KOS.
-
Great!
-
It may count up the garbage and see if it's worth garbage collecting or not.
Sounds good, then. But what if it really needs to garbage collect in the middle of an operation or something? It's still gotta take that into account, and then it might as well just wait until then to do it instead of checking at boot.
-
Another thing: it would be nice to not call it Garbage Collecting. I always found this weird. It might be best to call it defragmenting or something, like when deleting an APP. In French, Garbage Collect is Memory Reorganizing, anyway. The first time I saw GC, I thought the calc was asking me to do a mem reset...
-
Yeah, I guess it does sound a bit weird...
-
Agreed. Calling it Defragment is much more unambiguous.
-
Another thing: it would be nice to not call it Garbage Collecting. I always found this weird. It might be best to call it defragmenting or something, like when deleting an APP. In French, Garbage Collect is Memory Reorganizing, anyway. The first time I saw GC, I thought the calc was asking me to do a mem reset...
This may be the case, but I didn't make the term: http://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29 (http://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29)
-
Ah, right x.x. Well, if you call it that way, then it might be good to make it very visible somewhere in the readme or OS that this is just about fragmented mem that needs to be reorganized or something. That said, most ppl who may use your OS will probably be experienced calc users but some people with less experience may start using the shell later too if many software/games they want are made available only for KnightOS or if they get the OS from someone at school.
-
Yeah, my first reactions after finding the GarbageCollect feature when I got my calc were (in this order):
- Freaking out, thinking that everything was gonna be cleared
- Trying it again to see what it did
- Thinking it was fun and GarbageCollecting dozens of times a day just to see the message x.x
Also, wouldn't it be a waste to check for garbage at boot? It would have to check while saving stuff no matter what, and at the rate some of us use our calcs, we might have to garbage collect several times during use anyway.
-
Well, I still haven't implemented it yet. We'll see.
-
I personally feel it should maybe only do it if the isn't a lot of remaining archive sectors that aren't fragmented. I guess it might be best to wait a bit, though. What are you working on next on KOS, btw, Sir?
-
Next, I'm working on libraries, then making the filesystem conform better to KFS and be writable, then I'm releasing the kernel.
-
Ok, cool to hear. Good luck :)
I can,t wait to try it :D
-
Just noticed this topic, and I had a thought, isn't it also defragmenting? Garbage Collection in the way we are doing it seems to fit Defragmentation more.
Also, How about having it scan the filesystem and see if defragementing is a good idea before doing it? Like see how fragmented it is first, and if it is below a certain percent then it wont defragment
-
Well, it should only do this when the filesystem is full.
-
Ah ok, that makes more sense :)