Omnimaga
Calculator Community => Major Community Projects => The Axe Parser Project => Topic started by: Quigibo on June 19, 2010, 03:02:56 am
-
I've finished the compiling to applications feature and I need a few people to help me do some more extensive testing. When I release the next version, which I think will be tomorrow night, application compiling will not be included, but I will also have an identical version with app compiling that I will give to a selected few.
Conditions:
- I've already tested this on an emulator, so I need this tested on real hardware (especially non-83+SEs).
- I need persons who will use this feature regularly.
- You should not have anything important in ram OR rom! Its very possible to corrupt rom if my routine is buggy. You may have to do a full rom clear to restore your calculator back to normal or even resend the OS so backups are mandatory!
- You should not share this version of Axe with anyone because I don't want to be liable for any of the possibilities above.
Although it might sounds dangerous, it should be physically impossible to brick your calculator becasue I'm only using one routine to write to flash and it explicitly checks to make sure it does not accidentally write to the certificate page, which means the worst case scenario be that you have to resend the OS. I haven't had any problems at all with the feature so far and have never needed to do any of that, so its unlikely I made any critical mistakes, but that's what I need to confirm before I make this feature public.
So any volunteers? :P
-
Me!
-
Sure, I'd love to try out this -
I need this tested on real hardware (especially non-83+SEs).
Oh come on ... that's just mean. :'(
-
Hmm, I wonder if it signs apps on-calc, too.
-
Oooh Oooh Me Me Me! I would love this! Especially because I am already running out of space for Portal :P and this would prevent me from having to do crazy page switching. I'll need to back up all my stuff tho o.O just in case
-
Hey, speaking of which, while we're in the business of passing out closed betas, how's Portal X coming, BuilderBoy? ;)
-
Its actually fine if you have an 83+SE, that's just the emulator I'm using. Optimally I would pick one person from each category of 83+, 83+SE, 84+, and 84+SE.
Although I probably could sign on-calc, I don't want to. Its too crazy for what its worth. I'm hoping apps can be sent to the computer and signed there (something that will be tested in this trial).
-
I have a TI-84+, by the way.
-
Oh, cool, then I'd love to be the 83+SE guy. My RPG is already getting awfully close to that 8k limit, so I'd like to give this a go.
-
If I was a more advanced programmer and actually had projects and such I would love to test this. I wish you guys luck on your testing (to who ever gets to test).
-
I would love to give this a try.
I have a 84+ and a 84+Se
Both running BrendanW's 2.53MP
-
BRING IT! >:D
My specs are in my sig.
-
Has no one offered to test on a 83+?
If no one does, I'd be glad to try it, but I'm not sure how regularly I'd use it, so I don't know if I'd be a prime candidate.
-
Would a TI-Nspire do?
I also got my 83+ but it has full of stuff on it I'm using regulary.
-
Can we have the on-calc signing? I'm not really sure what the advantage of compiling on-calc is when you can't even finish the job...
-
Well, right now I'm pretty sure the compiling is just in the test stages, so why would you want to wait 10 minutes to sign a potentially bad app on calc, without even knowing if compiling works successfully?
Plus, the point of the this closed beta is to make sure compiling works. Once it works, I'm sure Quigibo will provide a way to sign apps.
-
From what I could read somewhere, I am not sure if he is gonna add such feature right away, because it's gonna take like 10-15 minutes to sign and not a lot of people will probably want to wait this long for an app to sign. I think in later versions, it might be a good idea to implement, though, in case. Just make sure that the option needs to be selected everytime you choose to compile in APP format and that if you selected it, it forces you to confirm your choice, in case you accidentally selected it.
-
I am sure oncalc apps signing will display first a warning and prompt for confirmation. Most people do not know the details about APPS.
APPS would be primarily used on large projects and will be harder to debug because of having to use an emulator or resigning the apps each time you recompile the source.
Good luck to you.
-
Btw, if you have too much data in your program and the total code+data is over 16 KB compiled, will it just put the data on a second page?
-
I'm not saying it'll be fun, I'm just saying it'll be beneficial for people like me who can go days without seeing a computer.
-
ah yeah I guess you have a point.
-
Btw, if you have too much data in your program and the total code+data is over 16 KB compiled, will it just put the data on a second page?
I don't think so, because multi-page apps cause headaches even for ASM programmers. We have to figure out how we're going to read data from the other pages, since they aren't mapped into memory at the same time. It would complicate things a lot.
-
Well you don't have to sign an app to play it on your own calculator. The only time you do, from what I understand, is when you want to distribute it to other calculators. Which I think what Quigibo's logic is since you have to upload to a computer to distribute it you can just sign it in seconds on the computer instead of waiting ten minutes for the calculator.
-
Btw, if you have too much data in your program and the total code+data is over 16 KB compiled, will it just put the data on a second page?
I don't think so, because multi-page apps cause headaches even for ASM programmers. We have to figure out how we're going to read data from the other pages, since they aren't mapped into memory at the same time. It would complicate things a lot.
Ouch, ok x.x. I guess data will need to remain in appvar/program format then x.x
Also in APP form, will the game requires as much RAM as an ASM 8xp file? Would we be able for example to run an Axe app containing 15 KB of code and have a game appvar unarchived taking like 22 KB of RAM?
-
Btw, if you have too much data in your program and the total code+data is over 16 KB compiled, will it just put the data on a second page?
I don't think so, because multi-page apps cause headaches even for ASM programmers. We have to figure out how we're going to read data from the other pages, since they aren't mapped into memory at the same time. It would complicate things a lot.
Ouch, ok x.x. I guess data will need to remain in appvar/program format then x.x
Also in APP form, will the game requires as much RAM as an ASM 8xp file? Would we be able for example to run an Axe app containing 15 KB of code and have a game appvar unarchived taking like 22 KB of RAM?
I think that would be possible, yes. APPs don't take up any user RAM when executing.
-
That would be cool. Code is read from archive, right?
-
To answer some questions:
An unsigned app runs fine on the calculator. It does not need to be signed to use, only to transfer.
In order to compile the app, a copy of the executable is created in RAM first and then turned into an app at the end of compiling. This is becasue the ROM can not be modified without erasing the whole thing (or most of it) becasue that's just how flash works. So you're still limited by the roughly 24kb limit of free ram which is okay since the apps are only 16kb. They will not be 2 page apps becasue 1) there is not enough ram on the calculator to compile such a large program and 2) I currently don't have any commands that handle page swapping.
A check is done on your battery level before writing to flash and there is a warning not to remove the power while it saves just like the game boys used to do. Writing to flash is a little slow, it takes about an extra second per kilobyte I'd say on a 15MHz calc, but that's not so bad.
There are actually a lot of details about apps that change. Fortunately I take care of most of them automatically so you won't even be aware of them:
- You can't display strings using the bcall since they need to be in ram (I changed the routine so it works)
- Apps have to be exited a special way (the entire program is actually one big subroutine so you can exit normally)
- You can not write over data in the program (nothing I can do about that)
-
That would be cool. Code is read from archive, right?
That is correct.
-
Ok good. I was worried it was copied into RAM somewhere during execution, which would have made it very hard to run games with large amount of data (more appvars to archive/unarchive during loading sequences, depending of which you need)
@Quigibo, in future versions, do you plan any syntax changes that are specific to APP format? Or will the Axe source remain the exact same between both 8xp and 8xk files?
-
I doubt there would be any syntax changes, unless I support page swapping which might complicate things a bit, but I don't plan to. As I said before, most of the differences are corrected behind the scenes so the transition should be as simple as possible.
-
So, will we be able to use Programs/Appvars for data or do we have to put it all into the App?
-
For external data, you can use programs or Appvars. But you have a whole page to burn.
Remember that no matter what, your program will end up being 16,384 bytes if you compile it as an App. That means you will probably have space to burn.
-
Alright. I guess that makes it easier to just pack it all into the source and compile it as an App.
Quigibo, you mentioned that Apps have to be exited a special way. Do we have to code that into the source or is it automatically enabled when it's compiled as an App, sort of like the On break that Basic programs had?
-
I doubt there would be any syntax changes, unless I support page swapping which might complicate things a bit, but I don't plan to. As I said before, most of the differences are corrected behind the scenes so the transition should be as simple as possible.
aaah ok ^^
For external data, you can use programs or Appvars. But you have a whole page to burn.
Remember that no matter what, your program will end up being 16,384 bytes if you compile it as an App. That means you will probably have space to burn.
Another thing I want to add is that even if your game has the advantage of taking mostly archive instead of RAM in APP form, IMHO I think wasting 16 KB of archive on a 2 KB code game just to have it ran from archive is a big waste of space. Something to consider. Do not add unnecessary bloat or features either just to fill the space as an excuse. For example, do not make a 16 level grayscale splash screen for a number guessing game
-
For example, do not make a 16 level grayscale splash screen for a number guessing game
You know someone is going to do that now that you mentioned it :P
Quigibo, you mentioned that Apps have to be exited a special way. Do we have to code that into the source or is it automatically enabled when it's compiled as an App, sort of like the On break that Basic programs had?
Apps have to exit with bjump JForceCmdNoChar instead of ret. The way that Qugibo was mentioning it, it sounds like Axe will do this:
call code
bjump JForceCmdNoChar
code:
...
ret
This would work pretty well.
-
This sounds all well thought out :) Signing on-calc i think will eventualy be a nice feature to have, for those without computers or without ways to connect to them for example. It would be a last resort though, since its so slow, and needs not be added for a bit.
-
That's exactly what I'm doing Sir. I also have a ReloadAppEntryVecs bcall in there becasue its recommended in the guide.
-
I volunteer! :P
(I've been having memory issues with my project for the contest)
-
Well, if you need me I have a regular TI 83+ and a TI 83+ SE for testing.
-
I have a ti-84+se, and don't mind undergoing any crashes. Also, my projected app size is fairly large. The only reason I could possibly see multiple page use is for map data or text for individual sections that are copied to an appvar for the actual level. Other than that, multiple pages would just slow it down, so I don't think it should be added just yet. Since you got rid of setupeditor, we could use that for multiple page use. Eg. Setupeditor 2, for page 2.
Btw, I seriously want to help you test it!
-
I can help testing with another nspire...
Will there be a way to run one app from within another? That would be a fairly simple way to overcome the 16kb limit without using external variables.
This sounds really handy. Now I can have Portal X, Half Life: OC, and all those other axe games without archiving and unarchiving all the time!
Edit: DJ, will this private release be usable in contests? I could see it going both ways.
-
Not unless Quigibo confirms that the next public version will contains every feature this private beta has.
-
Yeah, otherwise it wouldn't be fair. So, has quigibo decided who he will release the beta to yet?
-
Not sure how it works. I believe if you show any interest, he may send you a beta, but again I don't know if there is a limit on beta-testers
-
Well, I got the beta, just sent it to my calc, and here are my results so far (with an Nspire)
It loads, looks great, acts great.
Compiles regular programs just fine
Compiles apps suprisingly fast, nearly as fast as progs!
Runs progs at exactly the same speed as asm( compiled ones!
So far, I give it 2 thumbs up!
One Question: Will Recompiling have the same negative effect as deleting? It probably won't affect me, since I'm using an nspire, but it would still be good to know.
Edit: I've got one issue now. When compiling to an app and getting an error message, it reports the type properly, but won't go to the program if it's archived. It gives ERR: UNKNOWN ERR Code: 3A24569 when you press prgm. Not much of a change over 2.6, though. It doesn't crash the calc, and in 2.6 archived progs would output gibberish anyways. Maybe it could be changed to ERR: Archived in future axe versions.
-
I'm probably exaggerating when I say that erasing flash pages wears down the flash chip. Unless your calc is already 10 years old, I wouldn't even worry about it at all. I only mention it becasue if I don't and then someone's flash chip is no longer able to be written to, that wouldn't be pretty. Defragmenting has the same effect as garbage collecting. Both erase flash but in different areas.
I forgot to fix the error messages. I'll do that now. I'm glad this is working on an NSpire, that was one of the calculators I was worried about.
-
Well, got to play around with the beta a bit. Everything was smooth and no problems occurred.
My calculator: TI 83+SE with 1.19
So, here are some things I tested for and worked just fine:
(All of these were of course used with the compile set to Application.)
-Compiled normally
-Archived source and compiled
-Forced errors and tried to compile
-Compiled using lowercase letters in name
-Compiled running entire program in Full
-Compiled running entire program in Normal
-Compiled running Full/Normal toggling in program
-Compiled program larger than 8KB
Everything seems to be working just fine with the app compiling.
Oh, as for willrandship's error, I was able to recreate the same thing that happened to him on my calc, so it's not just an Nspire issue.
-
I'm probably exaggerating when I say that erasing flash pages wears down the flash chip. Unless your calc is already 10 years old, I wouldn't even worry about it at all. I only mention it becasue if I don't and then someone's flash chip is no longer able to be written to, that wouldn't be pretty. Defragmenting has the same effect as garbage collecting. Both erase flash but in different areas.
I forgot to fix the error messages. I'll do that now. I'm glad this is working on an NSpire, that was one of the calculators I was worried about.
I think it's about 100000 writes, but we do not know if it's a myth or reality. Even BrandonW is not sure, because he never saw a bricked calc this way. We had a member who managed to do it, but maybe his calc was defective and his flash chip weared out earlier.
-
No, it's possible. I seem to remember an article about a device that constantly erased the flash chip over and over. I do believe that it failed after a few million writes. I don't believe the average user has any reason to worry about this.
-
Except that now I remember a Wikipedia article talking about Casio AFX calcs saying that the flash chip lasted only a few years in them. Maybe that calc erase stuff over and over for absolute no reason, like when turning it ON or something, though. It made me worried, though.
http://en.wikipedia.org/wiki/Casio_graphic_calculators#Algebra_FX_series
-
Just wanted a little follow-up. Has anyone had any problems? And has anyone tried to send the app to the computer and sign it? It sounds like so far, the routine is working great. The one other thing I wanted to test though was to make sure it gives a memory error if you try to compile an app when you have less that 16kb of archive left over. Would anyone be able to test that?
-
No problems here. I haven't tried signing it, but I really like the App compile feature. It's quite nice. I can try to compile one with <16Kb of ROM left.
-
I can try signing mine and see how wabbitemu takes it. As for the memory, seeing as how I've got 1381k free, I don't plan on getting below 16k anytime soon.
EDIT: Okay, problem. whenever I try to transfer the App to my computer, it will only send about 400 bytes. I've sent it around 5 times and it always stops there.
-
Stupid TI-Connect, it seems like it will only receive signed apps, too (400 bytes is about the length of the signature). Better luck with TILP? Waiting fifteen or however many minutes to sign it on calc is ridiculous.
-
Bah at TI x.x
Now how do people who got a 64 Bit edition of Windows and can't use TiLP without much hassle attempt at sending Axe apps to their computer without having to sign them on-calc?
-
Oh boy, I forgot about them... this is ridiculous. It shouldn't check whether the app is signed when receiving it! I don't need content control on my computer, too.
-
Perhaps... And this is just a thought from an inexperienced newbie... but...
Could you just make a temporary signature to slap onto the end of the app for receiving only?
Then, have a program that looks for that specific pattern of bytes and rips it out?
-
I'm not the authority on this, but I don't think so.
-
I'm pretty sure you can't. There is only one way a certain app can be validly signed, so you can't have a temporary signature. Note that the generated apps have a signature, just a very wrong one :P
-
TI-Connect patch? o.O
-
Well, I switched to TiLP, but I must have a faulty driver or something, because it's not detecting my calc.
-
I think TiLP cannot be installed at the same time as TI-Connect. I am not sure if Uninstallign TI-Connect is a good idea, though. I heard horror stories about that, involving TI-Connect never working again if you ever decided to reinstall it later.
-
Uh oh, maybe i should have read that before uninstalling it for TiLP ... :(
-
Well I could be wrong. I just heard those stories before. You may need to ask on the help and support forum, or check old topics about it if you find any. Some people migth be able to help you find a solution, and we would be less off-topic in this thread :P
-
What about doing it in a roundabout way, like when you want to send it to your computer, it creates a program that is identical to the App, but just a program. And then on the computer, you would have some sort of utility that changes it into an App, and also signs it.
-
I've also thought of sending my source to wabbitemu and compiling it there, but it doesn't let me export it (or anything for that matter).
-
That could work well I guess. You would just need to warn the user to not execute the program, since it wouldn't work due to the 8 KB limit
@Magic Banana: try pressing F7
-
What about doing it in a roundabout way, like when you want to send it to your computer, it creates a program that is identical to the App, but just a program. And then on the computer, you would have some sort of utility that changes it into an App, and also signs it.
Hey, that's a great idea! Quigibo, what do you think? ;D
-
The variables menu isn't in some of the older versions of wabbit and it will crash wabbit when I try to use it.
(Windows 7 64-bit)
-
Really? Strange. x.x You should PM buckeye about it or hit him on OmnomIRC/#omnimaga@EFnet IRC
-
That could work well I guess. You would just need to warn the user to not execute the program, since it wouldn't work due to the 8 KB limit
@Magic Banana: try pressing F7
Yeah, I've tried that. I rightclick it and click export - nothing happens. I try to export anything and nothing happens. I checked my folders and they don't pop up anywhere. :(
-
Weird x.x. Definitively something you might need to report to BuckeyeDude
-
Just drag and drop the vars/programs to a folder. Clicking export does nothing.
-
Oh wait he clicked Export. Sorry I misread :(
Drag N Drop should normally work. If it makes your computer case turn pink, then what version of Windows are you using?
-
What about doing a full backup of the calculator? Maybe that will be able to group the app.
I can try adding a dummy signature to see if it will change anything and yes, I can always export the binary and write a simple program to convert it to the app format, but I'd rather find a way to do it directly. If it's the calculator that is doing the validating and not the computer (presumably) I might be able to overwrite the checking of the signature somehow.
-
A full backup? All files are transfered individually, and put together into one group at the end by the computer. I doubt the calc knows it's doing a full backup. It may be worth a try, though.
Whatever works. :)
-
On a semi-unrelated note, I found the latest version of wabbit and I can use the variables menu now! ;D
-
Glad to hear. Weirdly enough, though, I always got it to work fine as soon as I discovered it, even with a build from late 2009
@Quigibo I wish you good luck. I hope you figure out a way to fix the issue.
-
Alright, I tried the drag and drop, but that only lets me drag anything green to export it. All of the apps are red and it doesn't let me drag/drop it.
I tried the full backup for my calc and strangely enough, it does the exact same thing as if I only transfer the app. Everything else comes out fine (including all the other apps) but the app compiled in Axe still comes out at 400 bytes.
-
ouch :/ sorry to hear then :/
This is strange. I hope there is a solution
-
I'll analyze the file later to see what it sent.
-
I have what may be a bug!
This may only happen on nspires, not sure. When I recompile an app, so it deletes itself, it still increases the number of apps by one, but since there isn't an extra app, it duplicates the last app that was there. Right now I'm at 6 duplicates on my nspire. All duplicate apps lead to the finance app.
ie.
Apps on calc
Finance
Axe
AXED
Puzzpack
Periodic
RushHour
PONG
Transfrm
Francais
List
Top to bottom
Finance
Axe
AXED
Puzzpack
Periodic
RushHour
PONG
Transfrm
Francais
Francais
Francais
Francais
Francais
Scrolling Bottom to top
Finance
Finance
Finance
Finance
Finance
Finance
Axe
AXED
Puzzpack
Periodic
RushHour
PONG
Transfrm
Francais
Not sure if it's axe, but I have a sneaking suspicion it is, as I haven't done anything else AT ALL with it but axe.
Maybe a Keypad swap will clear it.....
Is anyone else experiencing this?
-
Is this with the old beta? I have fixed many things since then with 0.3.2 including adding a dummy signature and updating the size field of the app. Those might have something to do with it if the Nspire handles things differently. Does this happen every time or just sometimes?
I would do a rom clear to get everything back to normal. Backup your stuff.
-
Quigibo, you do delete the previous app if it already exists, right?
-
Yes of course. You should see the "Defragmenting..." message every time you compile an app that already exists. That means that the old app is getting deleted.
-
Would it be possible to merely overwrite the data instead of deleting it, defragmenting, then writing it?
-
No, the way flash works is that all the data has to be cleared at the same time, or at least in very large chunks. After that, the entire page is 1s in binary. From then on, you can only write 0's over the 1's and you wouldn't be able to write a 1 over a 0 since the memory is kind of "one way". Think about it like the voltage potential is put high and it is easy to put it low (by writing a 0) but it can't be put high again unless the entire chip is put high.
Its really interesting actually. There is a good article on Wikipedia http://en.wikipedia.org/wiki/Flash_memory#Block_erasure (http://en.wikipedia.org/wiki/Flash_memory#Block_erasure)
-
Ah i see, thats too bad. Oh well, with the benefit of Application compiling, its sure worth the tiny wait ^^
-
No, the way flash works is that all the data has to be cleared at the same time, or at least in very large chunks. After that, the entire page is 1s in binary. From then on, you can only write 0's over the 1's and you wouldn't be able to write a 1 over a 0 since the memory is kind of "one way". Think about it like the voltage potential is put high and it is easy to put it low (by writing a 0) but it can't be put high again unless the entire chip is put high.
Its really interesting actually. There is a good article on Wikipedia http://en.wikipedia.org/wiki/Flash_memory#Block_erasure (http://en.wikipedia.org/wiki/Flash_memory#Block_erasure)
Oh! That makes sense now! Thanks for the great explanation. ;D