Omnimaga

Calculator Community => Other Calc-Related Projects and Ideas => TI Z80 => Topic started by: Xeda112358 on January 02, 2013, 04:17:46 pm

Title: FileSyst
Post by: Xeda112358 on January 02, 2013, 04:17:46 pm
Hey everybody, I believe this is my next big project while I take a break from Grammer. In fact, this program used some code from my current progress on GrammerOS, but I will probably end up using this project for GrammerOS later. Anyways, what is it?

FileSyst provides some core routines for handling a filesystem on the TI-83+/84+/SE calculators and in the plans are a full folder system support, along with navigation, naming of the variables, icons, and a slew of other abilities. Currently, progress is in the planning stage. A few days ago I made a program and last night I turned it into an App. I basically took BatLib, hollowed it out except for about 50 BatLib commands, then I inserted what I had done with the program version and it is an undocumented mess at the moment. However, it has this much completed:

-Run the app and select the program to run (this is only for the developers version)
-The program will be executed using the FileSyst parser hook (hooks are restored when the program finishes)
-The dim() token is used to execute the included BatLib routines or the FileSyst routines.
-FileSyst takes arguments as a string. For example, "BASIC(Zeda:Doc Prog)" .
-FileSyst can open folders and files and automatically creates folder/files if you try to access them and they don't exist.
-There are only two commands at the moment:

BASIC() will open a file and try to execute it as a BASIC program
VPTR() will make a shortcut to an OS variable (the var doesn't actually need to exist)

An example might be:
Code: [Select]
dim("VPTR(prgmTEST,Zeda:Doc Test) BASIC(Zeda:Doc Test)
However, once I add in the CD command (which is basically only three bytes of code at this point):
Code: [Select]
dim("CD(Zeda:Doc) VPTR(prgmTEST, Test) BASIC( Test)
Anyways, since the program isn't very presentable, I won't release a program yet. Currently, there are two other projects that are already in the works to use this program-- ScopeOS by Codebender (with help from myself and Roguebantha) and Linux which is being done by Roguebantha (originally using BatLib, but that is why we have included certain BatLib commands in this app).

ScopeOS will hopefully provide a nice GUI to view the files and folders as well as all of the OS variables. I plan to add the following commands shortly:
Code: [Select]
CD(path)         This will open a folder (or use ".." to go up one folder level)
CF(filepath,field,data)     This will change a field in the header data of a file. Fields could be for passwords, descriptions, or specifying what programs to open the file with.
CI(filepath,icon#|icondata) This will change the icon number, or if you supply hex data, it will change the icon to a custom icon.
CN(filepath,newname)            This will change the name of a file.
COMP(filepath)              This will compress a file
COPY(filepath,newpath|newfilepath) This will copy a file from one location to another
DADD()                      Returns a string with the current directory
DEL(path) or DEL(filepath)   This will delete a folder or file.
GETD(path,#)                 Returns info aboutt he nth item in the directory (so you can create a list of variables or whatnot)
HIDE(filepath)              This will mark a file or folder as hidden
LOCK(filepath)              This will mark a file or folder as locked
MOVE(filepath,newpath)       Moves a file to another directory
ND(loc)                     This adds a new directory
SCUT(lfilepath,newloc)       This creates a shortcut of a file.
VPTR(VarName,filepath)      This will create a file that points to an OS variable

It currently supports relative paths where it starts from the current directory and absolute paths where it starts from the main directory. It also allows you to navigate back through directories by using ".." (or navigate back another directory for every additional period).

I wish it looked prettier so that I could supply some cool screenshots, but really, you don't see much happening. I've spent many hours over the past three days working on this working out any potential bugs. I hope the new years brings success because I had a really cool thought about how to use this when I first started.

Say you were making an RPG and you wanted to use a folder to store or retrieve monster data, player data, or anything else like that. You could use this program to make a directory in the main folder (named after your game or something), then in that folder you could put all of the other data you need. Then, to retrieve the data, you can use the FileSyst program to access the data through paths. Since the app has a bunch of graphics and data manipulating commands, I can make commands like SPRITE() so that you can make a script for certain parts of your program.

Here is a screenie from yesterday's version:
(http://img.removedfromgame.com/imgs/FileSystEx0.gif)
Title: Re: FileSyst
Post by: Sorunome on January 02, 2013, 04:32:48 pm
Wow, pretty cool! :D
And GrammerOS? :crazy:
Title: Re: FileSyst
Post by: Xeda112358 on January 02, 2013, 04:34:33 pm
Yes, I have been working on GrammerOS and Grammer3 every no and then, but I have basically taken a break from them (kind of like what I did with BatLib, probably a year from now I will put some serious work into the two).
Title: Re: FileSyst
Post by: Xeda112358 on January 02, 2013, 10:51:29 pm
The developers version has been uploaded to the Google project page for Scope OS for anybody that wants to try it in WabbitEmu. I doubt it is error free, but you can get an idea of how it works.

Download (http://scope-os.googlecode.com/files/FileSyst%20v1.004.zip)
Documentation is included, naturally, as well as the mangled source.

I added the GETD() function and CD(). CD() simply changes the default path and GETD() returns a string with information on the files in the given directory, such as the file names, types, and sizes. Now I can play with the included BatLib commands to try to get a simple GUI started :3
(http://img.removedfromgame.com/imgs/FileSystEx1.gif)

If you want the parser hook installed, you have to NOT let the program finish when executing a program from the app(I just press ON or try to run an assembly program).
Title: Re: FileSyst
Post by: Xeda112358 on January 03, 2013, 04:08:31 pm
This is an update for v1.006! I added the two commands DEL() and GICON() and I fixed the BatLib functions. On top of that, I modified one of the BatLib routines, so now commands like DBRead or GetStats can be performed on OS vars or FileSyst files. There was also an issue with GETD() that I fixed (it crashed when you tried to read an empty folder) and I also made it throw an error if you try to use it on a file instead of a folder. There was a bug with using ".." as well, to navigate back one folder. It was actually more of a really bad coding issue than a bug, but I fixed it.

Download (http://scope-os.googlecode.com/files/FileSyst%20v1.006.zip)

To expand on the two new commands:

I am hoping that some examples can soon be made, but for now you can watch the screenshot below :)
I basically run through these commands in the screenshot:
Code: [Select]
dim("GETD()
dim("CD(ZEDA)
dim("GETD()
dim("GETD(ZEDA)
dim("CD(ZEDA:DOC) CD(..:PROG) GETD(..)
dim("VPTR(prgmA,ZEDA:PROG PROGGY)
dim(8,"EA
dim(8,"(ZEDA:PROG PROGGY)
dim("GICON(ZEDA:PROG PROGGY)
dim("GICON(ZEDA)
dim(22.1,dim("GICON(ZEDA:PROG PROGGY)"),16,1,1,0,99
dim("CD(ZEDA) DEL(:DOC) GETD()
dim("DEL()
Note that dim(8) returns the contents of the variable and dim(22) draws a sprite
Title: Re: FileSyst
Post by: Xeda112358 on January 04, 2013, 10:43:56 am
Since the last post, I have done the following


My To Do list looks like this currently:


I also have in the plans:

For the file associations, here is an example. Say you had an ION program. If you use VPTR() to create an OS shortcut, you can then change Field 0 to include the phrase 'ION'. When you try to use OPEN() on the file, FileSyst will try to open DoorsCS7, MirageOS, ION, or NoShell to run the program (these are the default associations for ION programs). There will also be an external appvar that holds data from other installed programs. For example, if a program is made in the future for a text editor or image editor, they can edit the appvar to associate certain file types with the program.

Attached is version 1.010 for anybody that wants to play around with it.
Title: Re: FileSyst
Post by: Xeda112358 on January 04, 2013, 10:20:04 pm
 Filesyst v1.013 (http://scope-os.googlecode.com/files/FileSyst%20v1.013.zip) is ready for download. I changed CDIR() to PWD() by a suggestion from Roguebantha. So I have added PWD(), COPY(), and CFT() (change filt type). CP() is accepted as a substitute for COPY(), too.

COPY() doesn't copy the header of the file, so I plan to fix that tomorrow. All this means for now is that icon information isn't preserved when copying.

EDIT: FileSyst can now create folders and files, navigate through folders, and copy files/folders, so I think that means it has the basics for file management :3
I fixed COPY() to copy the file header and I am now working on CF() to change a field. For example, Field 0 will be for file associations, Field 1 will probably be for comments. So if I want to change the comment of file Hi:
Code: [Select]
CF(Zeda:Prog Hi,1,'This is a Hello World program.')
Which reminds me, I changed string inputs to require single quotation marks
Title: Re: FileSyst
Post by: Xeda112358 on January 05, 2013, 07:07:04 pm
FileSyst v1.017 (http://scope-os.googlecode.com/files/FileSyst%20v1.017.zip) has been uploaded. It now supports use of inline code in BASIC programs and you can store intermediate results to OS strings. I added CF(), GETF(), FDEL(), and AFA() (stands for Add File Association) today as well. Again, I'm hoping to get an example program sometime, but seriously, you can access a file system in BASIC now :D I plan to eventually add support for accessing Group variables as a special type of folder, so think of how cool that could be o.o You could make an RPG and create a folder for it and access all of the group data as part of the folder.
Title: Re: FileSyst
Post by: Xeda112358 on January 06, 2013, 09:15:08 am
Luckily nobody tried last nights version >.> After putting it on my calc and relinquishing my internets, I discovered that the documentation I provided for the order of the Extended BASIC commands was wrong and that they did not work if they had >1 argument x.x

The good news: The FileSyst commands still worked and I started to design a simple command prompt like thing :D The whole prompt thingy is written in BASIC and the shifting of the screen is an Assembly program from my B4001 contest entry (all it does is shift the lower 49 pixels up 6 pixels, so it preserves the top 9 rows). Sorry for the really long screenie, I kept forgetting that to delete files, you need to have an extra space >.>

EDIT: I uploaded a version that would freeze your calculator (some debugging code) .___. Sorry to the person that downloaded the app !_! Here is a fixed version (again) Hopefully I got it right this time.
Title: Re: FileSyst
Post by: shmibs on January 06, 2013, 11:41:32 am
ooh, that's really fancy =)
how long have those other OSs you mentioned been worked on? i don't see much about scope os on the project page there
Title: Re: FileSyst
Post by: Xeda112358 on January 06, 2013, 11:48:23 am
Thanks! Now that I have the eBASIC commands fixed, I can probably make faster/fancier (I hope).
how long have those other OSs you mentioned been worked on? i don't see much about scope os on the project page there
ScopeOS has been worked on for about a month, though I've seen no code, screenshots, or examples of it XD The Linux OS one I have seen some pretty cool screenshots of:
(http://img.removedfromgame.com/imgs/1-screenshot000.gif)
I'm assuming he won't mind me posting a screenshot >.>
 Linux OS (http://tibasicdev.wikidot.com/forum/t-598385/linux-os-in-hybrid-ti-basic)
Scope OS I (http://tibasicdev.wikidot.com/forum/t-593647/scope-os-i) <--That quickly degenerated into FileSyst conversation
Title: Re: FileSyst
Post by: Sorunome on January 06, 2013, 02:46:22 pm
wow, pretty cool!
And how did you get the screenie scaled up like that O.o
Title: Re: FileSyst
Post by: Xeda112358 on January 06, 2013, 03:18:41 pm
I'm not sure what he used :/
Title: Re: FileSyst
Post by: chickendude on January 06, 2013, 03:52:49 pm
It's probably TilEm (2). You can pick several sizes as well as the colors to use. The default color is that bright white (and a rather large resolution).
Title: Re: FileSyst
Post by: Xeda112358 on January 06, 2013, 11:50:45 pm
I added more to the command-prompt interface that I made and I finally fixed up the bugs that I introduced in some of my tests today. Now the "working directory" is saved to an external file and kept track of. This means that when you do CD() in one code block, the effect remains, even after the program is finished. This makes file navigation in BASIC much easier. Attached is a screenie and download
Title: Re: FileSyst
Post by: DJ Omnimaga on January 06, 2013, 11:55:08 pm
Looks pretty nice and interesting. Also off-topic: Is GrammerOS an actual OS or is it a shell like MirageOS or DCS?
Title: Re: FileSyst
Post by: Xeda112358 on January 06, 2013, 11:56:01 pm
My goal is to make it an actual OS. Grammer 3 will be more like a shell with a few extra features.
Title: Re: FileSyst
Post by: Yeong on January 06, 2013, 11:56:03 pm
Command prompt O.o
Nicely done!
Title: Re: FileSyst
Post by: Xeda112358 on January 07, 2013, 12:08:37 am
I am only using 8 of the FileSyst commands to do that, too :D I wanted to add in a file icon editor, too, but I haven't gotten to that yet XD I have the BASIC code in my B4000 program because that edits the player icon.

I wish I had some actual games to create that use the filesystem, though. It would be easy to store highscores and stats as well as subprograms into folders that you name after the game. Plus, it would be nice to show off the app and figure out what commands I need to better integrate FileSyst with BASIC.
Title: Re: FileSyst
Post by: Xeda112358 on January 08, 2013, 01:20:52 am
I won't be releasing an update since I got caught up with other issues (and I didn't have computer access most of the day). I modified the AFA() command (Add File Association) so that it would check if the file association has been made (basically, it won't make the same association twice, that is redundant and takes up memory). I also added RFA() (Remove File Association) and I have started work on the OPEN() function, which opens files based on file associations. I am currently coding in all of the default stuff, so ION programs will open with either DoorsCS7, MirageOS, ION, or Noshell, for example. You can also do something like AFA( Game,'BATB') and then when you do OPEN( Game), FileSyst will install the BatLib hook before running the game. The filetypes I plan to add as defaults:

ION
MOS (MirageOS)
DCS7 (anything DoorsCS7 can run)
DCS6
C3 (Celtic3)
OMNI (Omicalc)
XLIB (xLIB)
BATB (BatLib Basic)
BAT (BatLib assembly)
GRAM (Grammer programs)

I will probably add more later, but those are my main concerns I am testing DCS7 first.
Title: Re: FileSyst
Post by: Matrefeytontias on January 08, 2013, 09:37:26 am
Looking nice :D I find the text display in the command prompt too slow though. The manual takes a while to show up entirely.
Title: Re: FileSyst
Post by: Xeda112358 on January 08, 2013, 09:39:27 am
Yeah, I am trying to make it a fixed width-font, but since I am using BASIC, it is slow. I will probably add back in the text display commands in FileSyst to make it much faster and smoother.
Title: Re: FileSyst
Post by: Matrefeytontias on January 08, 2013, 09:40:19 am
Wait what, FileSyst is actually BASIC !? O.O I thought it was ASM :crazy:
Title: Re: FileSyst
Post by: Xeda112358 on January 08, 2013, 09:41:24 am
The program in the screenshot uses FileSyst (which is an app) to access the folders and files, otherwise, the program is coded in BASIC.
Title: Re: FileSyst
Post by: Matrefeytontias on January 08, 2013, 09:42:21 am
Oh yeah, I was speaking of the file system :P however that's a pretty awesome performance to make a command prompt in Basic O.O
Title: Re: FileSyst
Post by: Xeda112358 on January 08, 2013, 09:51:21 am
Hehe, thanks :) I want FileSyst to be completely accessible to BASIC programs and programmers.

As a side note, about 9 hours ago, before I went to bed, I had made progress on the OPEN() command. I need to fix a few minor issues, but basically, if you have DoorsCS7 on your calculator and it can open a file type, OPEN() will work. I still need to program in all of the other apps and programs, though, such as for BatLib, Grammer, Omnicalc, and Axe Fusion programs.

To think of File Associatons, it is basically like adding a ".txt" or ".exe" or ".py" to the end of a file name. FileSyst does not actually check the header or contents of the program to see what it needs to be run with. Instead, if you want to assign a program the BatLib file type, for example:
Code: [Select]
dim(0
CD(Zeda:Games)
VPTR(prgmRPG, RPG)   ;creates a file that links to the program "RPG"
AFA( RPG,'BATB')        ;Add the file association  for BatLib BASIC
Return            ;Exist the FileSyst parser and return to BASIC
Then, whenever you do OPEN( RPG), FileSyst will try to install BatLib, then run the program.
Title: Re: FileSyst
Post by: Matrefeytontias on January 08, 2013, 10:30:24 am
With DoorsCS7, you can seek the 3-bytes type of the file which is at (pointer_on_file+5), so you can quickly open it with DCS7's AP. Moreover, if a file starts with BB6DC93180, then it's a DCS7 associated file and must only be opened with DoorsCS7.
Title: Re: FileSyst
Post by: Xeda112358 on January 08, 2013, 10:45:48 am
I know, but FileSyst doesn't use DoorsCS7 except for when the user wants to run a DoorsCS7 program (or some program that DoorsCS7 can run). For example, if the user doesn't have MirageOS on their calculator, but they have DoorsCS7, and they are trying to run a MirageOS program, FileSyst will open up DoorsCS7 and use its RunProg call.

Using the File Association feature, you can open files with certain apps or programs. I even plan to make it so that even BASIC programs can be used to open files o.o (For example, if you try to open up game data, you can make it auto-launch the actual BASIC game, simply by telling FileSyst that the BASIC program is what opens that file type)
EDIT:
To expand a little more, FileSyst stores files with the following format:
1 byte
1 byte
n bytes
1 byte
33 bytes
1 byte
2 bytes
n
2 bytes
n
Type (1Fh indicates it is a folder, all other types are open)
Name Size
Name
Icon (If this is -1, it is a custom icon, 0 is default, anything else is a preset)
IconData (If the Icon is not a custom Icon, this is omitted
Info (this is for info about Hidden,Locked, Shortcut, or OS var, among other things)
Fields Size
Fields Data
Data Size
File Data
Fields have a header byte so there are up to 256 different fields. Field 0 contains information about file associations, Field 1 contains a file description. If the field doesn't have any data, it is removed. If there is no field data, then Field Data is removed and Field Size is set to zero.

Field 0 is where FileSyst figures out which program  or app to open to handle the file.

Also, as an update, here is a screenie where I run the BASIC program MORPION40  (Tic-Tac-Toe) from DCS7 and DK5ION (DonkeyKong) as an ION program from DCS7.
Title: Re: FileSyst
Post by: Sorunome on January 08, 2013, 08:37:33 pm
Looking nice, but can you also use it in basic programs?
And if yes, is it compatible with BatLib as it uses dim( too?
Title: Re: FileSyst
Post by: Xeda112358 on January 09, 2013, 12:43:10 am
Looking nice, but can you also use it in basic programs?
Sorunome, that screenshot is of it being used in a BASIC program.
And if yes, is it compatible with BatLib as it uses dim( too?
The answer is yes and no. The two cannot work at the same time, but in this update I will introduce a new command OPEN() that is able to open BatLib programs (among other things). Also, a bunch of BatLib commands are provided at the moment with FileSyst, though some may be removed later.

So as to the update, I have OPEN() working for a handful of file types! These include BASIC, Assembly, xLIB, Omnicalc, BatLib, Celtic 3, Grammer, DoorsCS7 BASIC, DoorsCS7 Assembly, MirageOS, ION, DoorsCS6 Assembly. I will probably be making more adjustments, but it is pretty well functioning at the moment. On top of that it allows you to run programs that are archived and I spent a good portion of today making it so that it would return to the calling program properly. I updated prgmCMD to include the OPEN command and you can see as it opens DK5ION that this should be very useful!
EDIT: Also, I started work on a tutorial.
Title: Re: FileSyst
Post by: shmibs on January 09, 2013, 03:35:15 am
I updated prgmCMD to include the OPEN command and you can see as it opens DK5ION that this should be very useful!

indeed! i bet basic programmers out there will love this, and the code could be really useful elsewhere as well. is it calling those other apps to open the selected files? if so, it would be nice to have a hook that's called whenever a program is run from the homescreen, seeing as there isn't really anything out there that chooses what to use in opening a file dynamically like that.
Title: Re: FileSyst
Post by: Xeda112358 on January 09, 2013, 10:45:24 am
is it calling those other apps to open the selected files? if so, it would be nice to have a hook that's called whenever a program is run from the homescreen, seeing as there isn't really anything out there that chooses what to use in opening a file dynamically like that.
Yes, the program checks for the associated app or program and has it loaded (it also installs the necessary hook and restores the parser hook after running). However, since the app does this based on data stored in the file header and not the program, it would be difficult to install a hook to intercept the prgm token to do this. From the homescreen, people can do dim("OPEN( <<file>>) and it will work, so long as the file has the proper file association set.
Title: Re: FileSyst
Post by: Sorunome on January 09, 2013, 07:02:29 pm
Looking nice, but can you also use it in basic programs?
Sorunome, that screenshot is of it being used in a BASIC program.
It was off the homescrean :P
And if yes, is it compatible with BatLib as it uses dim( too?
The answer is yes and no. The two cannot work at the same time, but in this update I will introduce a new command OPEN() that is able to open BatLib programs (among other things). Also, a bunch of BatLib commands are provided at the moment with FileSyst, though some may be removed later.
Oh, that is pretty cool, does batlibs chainhook still work then?
Title: Re: FileSyst
Post by: Xeda112358 on January 09, 2013, 10:08:31 pm
Oh, I think you missed this (http://ourl.ca/18065/333397) post, then :P Or, you can see what I started working on earlier today in this post

EDIT: Oh, I forgot to mention the important part of what I did today. I added in actual icons instead of the two dummy icons, as you can see from the example. GICON() also now looks for more info when it reads a "default" icon. For example, it will check if the file has any file association and then it checks if it is a BASIC or assembly program, returning the appropriate icon.
Title: Re: FileSyst
Post by: Xeda112358 on January 10, 2013, 03:44:26 pm
I figured that I would ask for input on the following changes to be made to FileSyst:

-Using "/" instead of ":" for folder paths
-Getting rid of the File Association field and just using file extensions. For example, DonkeyKong.ION

Do these sound like good ideas?
Title: Re: FileSyst
Post by: Deep Toaster on January 10, 2013, 03:52:35 pm
-Getting rid of the File Association field and just using file extensions. For example, DonkeyKong.ION
That's actually a great idea for any shell...
Title: Re: FileSyst
Post by: Xeda112358 on January 10, 2013, 03:58:09 pm
Thanks :) And since FileSyst is going to be used in at least three projects (LinuxOS, ScopeOS, and LSDOS), all of which will have shell features, that should be greatly useful. Plus, as it is, it isn't easy to show the user what file associations there are. However, if it is part of the file name as a file extension, users will be able to clearly see what kind of file they are opening.
Title: Re: FileSyst
Post by: Sorunome on January 10, 2013, 05:36:38 pm
I like the idea with the file extensions :D
And the / for folder path is a must anyways :P
Title: Re: FileSyst
Post by: Xeda112358 on January 11, 2013, 12:28:35 am
Okay, so I added in file extensions and using / for filepaths. It as actually a lot easier than I thought it would be to convert to using file extensions. I also added in the .DOC7 to open up DocDE7 files and I removed AFA() and RFA() as they are no longer needed.

I think I should probably mention that in the screenie running prgmGUI, prgmGUI keeps its subprograms archived. It creates a folder in the main directory called GUI, creates OS shortcuts there, then runs them when it needs to using OPEN(). But what is even cooler is that GUI recognises hidden files or folders and won't display them in the menus. That is why you don't see its folder-- it is marked as hidden using the HIDE() command.
Title: Re: FileSyst
Post by: ben_g on January 20, 2013, 04:10:59 pm
This looks cool. Do you need to have DCS installed for DocDE to work or is this fully compatible with DCS programs?
Title: Re: FileSyst
Post by: Xeda112358 on January 20, 2013, 04:13:59 pm
You need DoorsCS as well as DocDE7 to open .DOC7 files.
Title: Re: FileSyst
Post by: tr1p1ea on January 20, 2013, 07:12:06 pm
Very handy feature :).

I was wondering if you had put any thought into case sensitivity at all? Is it required? (could it be toggled?)
Title: Re: FileSyst
Post by: Xeda112358 on January 21, 2013, 06:15:01 am
Currently it is case sensitive. I was thinking of making it not case sensitive for the extension part of the name, but I am trying to figure out the best approach for the rest of the name. For example, DOnkeyKong.ION is different from DonkeyKong.ION. I could make the file searching code recognise the two as the same name, though. Do you think this would be a good idea?
Title: Re: FileSyst
Post by: Sorunome on January 23, 2013, 08:08:27 pm
I would go with storing case sensitive but searching case in-sensitive
Title: Re: FileSyst
Post by: tr1p1ea on January 23, 2013, 11:51:31 pm
I think case-insensitivity is possibly a good idea if input is calc driven, since it would save tedious typing time? But i guess it comes down to your preference Xeda ... is the necessary work required to implement worth the benefit? - I guess you can see it as both an increase and decrease in flexibility.
Title: Re: FileSyst
Post by: Xeda112358 on January 26, 2013, 09:57:29 am
I have modified the program a bit so input is no longer case-sensitive. New files will be created case senstive, though, so you can create Partex2.gram and refer to it as PARTEX2.GRAM. This will hopefully be useful to those who don't want to activate lowercase or use lowercase.

Do you think that converting the file extensions to lowercase is a good idea? I think it looks nicer and it wouldn't be much of a hassle to convert uppercase to lowercase. You could still refer to files using uppercase file extensions, but when you use GETD(), it will return the extensions as lowercase if I do this.

I am also going to add in a new command that I was supposed to do last week. It will be an eBASIC command similar to one that already exists. It basically returns variable names in the OS VAT so that the other folks using FileSyst can easily get the names of all the vars in the OS and create appropriate folders with all the shortcuts. The differences between this and the GetVar command that already exists:

Once I modify the code to be more flexible (it currently only searches for appvars), I will probably post it in the Useful Routines thread.
Title: Re: FileSyst
Post by: Xeda112358 on April 28, 2013, 12:26:07 pm
Update!

There was a pretty bad bug that I ran into a while ago while modifying the program and I wasn't sure what caused it. I finally tracked it to the root of the problem today by luck because I had no idea what to look for. It turned out that four bytes of code that I commented out were actually needed in a certain rare situation.

So that is fixed, and I added the NEW() command so that folders and files could be created whenever they were needed. As well, I added the SINSRT() command to insert a string of data into a file. It is not complete as it only inserts at the beginning of the file, but it is the start of a set of commands that I have been planning on working on. As well, you can now use the OS Ans variable, if it is a string, in place of string arguments and you can use OS String variables.
Here is an example:
Code: [Select]
:Input "Name:",Str1
:dim(-1             ;start a block of FileSyst code
:CD(Game/Str)
:NEW( Name.str)
:SINSRT( Name.str,Str1)
:Return             ;Exit the block of FileSyst code
That will create the file Name.str in folder Game/Str and fill it with the contents of Str1.


I am sure there are still bugs, so take care, but the download is here. (https://code.google.com/p/scope-os/downloads/detail?name=FileSyst%20v1.022.zip&can=2&q=) I also updated the example program a little bit, but not much. In particular, renaming files seems to have problems and I haven't looked to see if that is a problem in FileSyst or prgmGUI.

EDIT: I added in the DELB() command to delete a given number of bytes from a file at a given offset. It properly handles when the offset is beyond the file data, when the number of bytes deleted would go beyond the file and other potential issues. I also added an offset argument to the SINSRT() command and it handles when the offset goes beyond the file by simply appending the string to the file. I won't be back for 5 or 6 hours, but if I am not too tired, I will try to add in commands for line reading/writing/overwriting. I will then want to add the ability to parse FileSyst code directly from archive and have user defined functions. I already have a bunch of other things planned, though, so I should try not to get too far ahead of myself :P Needless to say, the commands and a few of the routines that I added today have been on my To Do list since week 1.
Title: Re: FileSyst
Post by: Joshuasm32 on April 28, 2013, 06:42:13 pm
Huuraay!  :D
Title: Re: FileSyst
Post by: TheMachine02 on April 29, 2013, 11:57:07 am
this is a very great program  :thumbsup: good job  :)
Title: Re: FileSyst
Post by: Xeda112358 on April 29, 2013, 12:39:38 pm
The meeting actually lasted a lot longer than I expected, so I was too tired last night. So instead, I will just upload what I have and hope for more time later :)
Title: Re: FileSyst
Post by: willrandship on June 03, 2013, 05:39:36 pm
How have I not seen this project until now?

Request if it's not already there: Archive support.

By the way, don't be surprised that your code runs faster than TI-OS. There are many areas where their code could be optimized heavily.
Title: Re: FileSyst
Post by: Xeda112358 on June 05, 2013, 10:21:54 am
When you say Archive support, do you mean having the ability to keep files and folders in archive? Currently,if you make any OS pointers, then the OS vars can be archived. An example is if you have prgmSUB01 archived, this will still work:
Code: [Select]
CD(Sub)         ;open the folder Sub.
VPTR(prgmSUB01, 01)   ;create an OS pointer to prgmSUB01 named 01 stored in folder Sub.
OPEN(Sub/ 01)      ;use the OPEN() command to open the file. The file type is auto detected and opened with the appropriate routines.
I use this technique to keep sub programs in archive. As well, you can have some sub-programs that require different parser hooks, so if prgmSUB01 is a regular BASIC program, but prgmSUB02 needs Celtic 3, prgmSUB03 needs BatLib, and prgmSUB04 needs DoorsCS7, you could do:
Code: [Select]
CD(Sub)
VPTR(prgmSUB01, 01)
VPTR(prgmSUB02, 02.c3)
VPTR(prgmSUB03, 03.batb)
VPTR(prgmSUB04, 04.dcs7)
As long as the appropriate apps are on the calculator,the programs will run. For a Celtic 3 program, FileSyst first checks if the Celtic 3 app is on the calculator, if it isn't, it checks if DoorsCS7 is on the calculator. If one of them is on, then FileSyst will try to install their hooks and then run the program. From the readme:
Code: [Select]
/==============================================================\
|     OPEN(File)                                               |
\==============================================================/
|  This can be used to open files based on its file extension. |
|If it does not have a file association, FileSyst will try to  |
|open it as a nostub assembly program or BASIC program. Built  |
|in file types and the programs that open them are:            |
+======+=======================================================+
|Assoc:| Programs (In descending order of priority)            |
+======+=======================================================+
| ION  | DoorsCS7,DoorsCS6                                     |
| MOS  | DoorsCS7,DoorsCS6                                     |
| DCS  | DoorsCS7,DoorsCS6                                     |
| DOC7 | DoorsCS7+DocDE7                                       |
| BAT  | BatLib                                                |
| BATB | BatLib                                                |
| C3   | DoorsCS7,Celtic 3                                     |
| XLIB | DoorsCS7,Celtic 3,xLIB                                |
| DCS6 | DoorsCS6,DoorsCS7                                     |
| DCS7 | DoorsCS7                                              |
| DCSB | DoorsCS7                                              |
| OMNI | Omnicalc                                              |
| GRAM | Grammer                                               |
+======+=======================================================+
|  For example, if you try to run an xLIB program, FileSyst    |
|first searches for DoorsCS7, if that isn't available, it tries|
|Celtic 3 and then xLIB.                                       |
|  Further, xLIB and Omnicalc don't have any actual installers |
|aside from manual installation. If FileSyst has to install    |
|these, it uses a method that should only work on specific     |
|versions, xLIB v.602b and Omnicalc v1.26 and v1.26MP.         |
|  Now, for some example code:                                 |
|     CD(Zeda/Games)                   ;Change the directory.  |
|     VPTR(prgmDK5ION, DonkeyKong.ION) ;Makes an OS shortcut.  |
|     OPEN( DonkeyKong.ION)            ;Run DonkeyKong.        |
|                                                              |
|  OPEN() should return to the program properly, but there is  |
|always a chance that it won't if an assembly program doesn't  |
|quit properly (or if a BASIC program uses Stop).              |
\==============================================================/
Title: Re: FileSyst
Post by: Xeda112358 on July 04, 2013, 11:15:30 am
Hey! I was trying to come up with ideas for things that I could add today. I added in a command called OSNEW() that creates an OS variable with an optional size argument. I was thinking of adding in a complicated method for handling variables specific to FileSyst. Basically, here is the idea and I think it is too complicated and I won't add it:

-Create a special type of hidden folder with the name of the main program being run.
-Have a command to define a subroutine
-This creates a folder with the name of the subroutine, and the relative location in the variable for quick lookup
-Inside this folder, will contain named variables used by the routine such as floats, ints, and strings.

This means that if I add an ability to SUB(LBL), or whatever, they can have local variables (and possibly access variables in other subroutines). This could also mean that such a language would be slower than TI-BASIC since variables can have custom names and they are located in folders (then again, this could potentially be faster than the OSes VAT lookup).

So yeah, just some food for thought :)
Title: Re: FileSyst
Post by: tr1p1ea on July 04, 2013, 05:55:13 pm
You know, Xeda, you should just get it over with and write a whole Operating System :).
Title: Re: FileSyst
Post by: Xeda112358 on July 04, 2013, 07:17:03 pm
The problem is that I still don't understand Flash protocol and USB protocol. I am not sure, though, but I think SirCmpwn released a template for an operating system, so I could probably use that.
Title: Re: FileSyst
Post by: DrDnar on July 05, 2013, 01:40:57 am
The problem is that I still don't understand Flash protocol and USB protocol. I am not sure, though, but I think SirCmpwn released a template for an operating system, so I could probably use that.
Raw flash commands are documented on the Wiki. (http://wikiti.brandonw.net/index.php?title=83Plus:OS:Raw_Flash_Commands) BrandonW is the only documentation for USB.
Title: Re: FileSyst
Post by: Sorunome on July 05, 2013, 03:39:31 am
erm, what is the file extention then if you use xlib, celtic3 AND picarc all in one file?
Would it be DCSB as you need DCS7 to run?
Title: Re: FileSyst
Post by: Xeda112358 on July 05, 2013, 07:55:20 am
You could use a .C3 extension since Celtic 3 supports all of those. An extension of .C3 will look for DCS7 first, since DCS7 has the most complete and bug-free[citation needed] version, else it will look for Celtic 3, and if that isn't available, then it won't execute.

EDIT:Also, thanks DrDnar!
Title: Re: FileSyst
Post by: Sorunome on July 05, 2013, 08:04:01 am
Oh cool! (On a side note, i never saw the C3 app anywhere, lol, same with picarc)
And yeah, let's all just hope that DCS8 will have a broader omnicalc support like the play command :)
Title: Re: FileSyst
Post by: Xeda112358 on July 08, 2013, 06:34:44 pm
I am starting to change some things around, like the look-up routine to find a function. To speed up the search a little in most cases, I had the names stored alphabetically. Now I have a 52-byte LUT to jump to a given spot in the table based on the first letter of the function.

I was also thinking that it might be a good idea to store folders and files alphabetically. While that may sound like a pain to actually code, I realised earlier that it should actually be quite easy and it would make searches faster. Instead of checking every file in the folder to find out it doesn't exist, it only needs to start where the first letter matches and go until the first letter no longer matches. This will mean that users can get an alphabetically sorted list of names. This should make it possible to have folders indexed in a similar way as how the function list is indexed. These all together should make lookup speed a bit faster than the system that TI uses and long program lists will be able to load just as quickly as short ones.
(tl;dr : files and folders can be accessed faster and new files can be created faster -- the code is already there)

I also wrote the brunt of a routine to tokenise source codes. This doesn't help much for speed, yet, but if control structures like loops are implemented later, this should greatly improve speed (by a gazillion-fold).

Hopefully all of this will make it easier to add variable support :)
Title: Re: FileSyst
Post by: Xeda112358 on January 11, 2014, 01:22:53 pm
I am one step closer to making a calculator OS.

This isn't a FileSyst update, but rather an allusion to what will hopefully come in the future.

Today I worked on putting together the basics of YAI (Yet Another Interpreter), but with more goals in mind, specifically working with variables. I am trying to put together a multiply command as a proof of concept, and it is not done. The parser currently does all of the following:

- It uses a GDB var for RAM storage (instead of an Appvar)
- It parses the input string by checking if it is a number or letter to start. If it is a number, it converts the number either to an integer or to a float (the float conversion isn't working yet). If it is a letter, it searches the built in command list using a binary search, allowing alphanumeric values (uppercase, lowercase, and numbers). If no match is found, it searches user defined variables and functions.
- Integers are arbitrary precision, up to 2040 bits.
- The GDB variable holds a stack for intermediate calculation and for general purpose (similar to the OS floating point stack), followed by an index for the variables, which are stored alphabetically

Because memory management is going to be so dynamic, I made a custom InsertMem/DelMem routine to update all appropriate pointers and the size of the GDB var. I made a PushAnsStack and PopScratch routine which is used by the parser when parsing inputs an outputs to functions. These use the custom insertmem/delmem routine, of course.

I tried to make my current system easy for me to work with, I am trying to get the biggest hurdles out of the way first, but I still need to write the floating point conversion routines (string→float, float→string). After this, making the commands should be much easier.

As it currently is, I can get right in and start making integer routines. I also have code ready for lists, arrays, and string variable types. I have not yet added creating and storing to variables, but reading from them should work. I only just finished all of the memory management, so the rest will be for the next time I get a few days off from work :P

For an example, I have put together an integer multiplication routine. The interface isn't the best, but for testing, it works okay. It shows using the MUL() command for multiplying integers. I put the code in prgmTEST and then the app "Interpre" reads prgmTEST and executes whatever is inside. Once it is finished, it passes the last result into Ans as a string.
Title: Re: FileSyst
Post by: Xeda112358 on January 16, 2014, 02:19:19 pm
I haven't merged this with FileSyst yet, but I am making some more progress. Last night I finally added the Str->Float routine, so now floats can be used. The way I am handling floats is by satisfying the following two conditions by default:
-There must be at least 32 bits of precision for the fractional part of the number.
-There must be at least a total of 128 bits of precision.

So for example, if you have a number whose integer part is already 112 bits before the decimal point is encountered, it will allocate 32 bits more for the decimal part making 144 bits.

For math, I only have multiplication. I have int_times_int and float_times_float, but I still need int_times_string and int_times_float and once I add in arrays and lists, I will need to accommodate for those. The way I handle precision with multiplication is by using the size of the highest precision input. So if one float was 128 bits and the other was 144, it would output a 144-bit number. I plan to make a command to modify the accuracy and size of floats.

I actually still need to complete one other part of the floating point code, and that is to handle negative exponents. I am sure I will get to it some time. Aside from floats, I have some variable support. You can make a var from scratch, but I didn't add in the code for updating a var with new data. That should be done easily enough.

I have a few commands that I added for testing purposes. Those are XORSPRITE(), UPDATELCD() and WAIT(). They work :) The way I have coded this interpreter should make it easy to incorporate a file system. I was already rewriting FileSyst to incorporate features I am using here (like an indexed binary search for files and folders) so I will probably merge the two.
Title: Re: FileSyst
Post by: Matrefeytontias on January 16, 2014, 02:23:29 pm
Wow ... This is more and more awesome :D by the way, why are there G and H letters in the sprite ?
Title: Re: FileSyst
Post by: Xeda112358 on January 16, 2014, 02:29:21 pm
The sprite isn't hexadecimal. I was just using a simple sprite. I don't have a "hex2bin" command yet. Also, I don't have the sprite variable type defined yet, so I had to cheat by using a string. The two empty spaces are just padding where the sprite width or height would be.

Also, I should point out that the float->str command is not finished (as seen in the screenshots). It only converts the integer part. However, it is actually storing the decimal part.
Title: Re: FileSyst
Post by: chickendude on January 16, 2014, 02:43:09 pm
Wow, that's great! Will be seeing a full-fledged OS one of these days?
Title: Re: FileSyst
Post by: Eiyeron on January 16, 2014, 03:10:34 pm
Are there any hooks can be reached with some Axe? I'd like to test it by doing a graphical wrapper.
Title: Re: FileSyst
Post by: Xeda112358 on January 16, 2014, 03:21:21 pm
I don't think I will ever finish writing an OS, but I am trying to get a lot of the key components done. Of the related things I have done that I would like to bring all together:
-EnG (http://ourl.ca/16802) which was never finished, but provided a very simple file system spanning the extra RAM pages (detecting whether there were 1 or more RAM pages).
-Grammer as a simple interpreted language.
-FileSyst which exposes a file system and related housekeeping commands.
Title: Re: FileSyst
Post by: Eiyeron on January 16, 2014, 03:25:19 pm
Xeda, are they viable for a full-fledged OS?
Title: Re: FileSyst
Post by: Xeda112358 on January 16, 2014, 03:38:24 pm
Are there any hooks can be reached with some Axe? I'd like to test it by doing a graphical wrapper.
Are you talking about the latest side project or FileSyst? FileSyst does have a jump table to handle a bunch of things. LangZ80 does not have one yet, but I can easily add in some jumps to the parser and a few of the routines. I have it so that it really can take HL as the location of the code, BC as the size of the code, and it does the interpreting from there. I also have a routine that converts the output to a 16-bit integer instead of a string, and I can make a call that reads from a 0-terminated string (like Axe uses) so you could do something like this in Axe:
Code: [Select]
LANG("VAR1")→A
And then it would either return a pointer to VAR1 if it is a string, or it would return int(VAR1) for Axe.
Xeda, are they viable for a full-fledged OS?
If I understand correctly, yes, these routines are currently adequate as a calculator. It defaults to a precision of 128 bits for floating point math (the TI-OS uses <48 bits) so it is much more accurate, and integers are arbitrary precision, up to 2040-bits (614 digits). I plan to add a few more variable types, including Sprite, Array, and List. As well, custom variable types so that types like Complex, Complex List, Complex Array, Matrix, can be made from these.