Omnimaga
Calculator Community => Major Community Projects => The Axe Parser Project => Topic started by: FloppusMaximus on October 15, 2010, 09:59:07 pm
-
Like I said over in the Mimas thread, I've been experimenting a bit to see how practical it would be to sign apps on the calculator. So I figured I'd take a look at how Axe generates applications.
(App signing on the calc is never going to be instantaneous (except maybe on the Nspire in ARM mode), so you probably don't want to do it every time you build an app, but it would be nice to have it as an option when you want to distribute your app to others.)
Thus, a few points:
1. It seems that when Axe generates an app, the "signature" is filled with zeroes. It would make life a lot easier, from my perspective, if you used FF bytes instead (and since the signature is invalid either way, it shouldn't matter what you put there initially.)
2. App signatures on the 83+ must begin with the bytes 02 2D 40 (followed by 66 bytes of data), not 02 0D (the latter is used for OS signatures as well as for app signatures on the 89/92+.)
3. Does Axe deal with the 55-mod-64 bug? (An application must not be exactly 55 bytes long mod 64; if it is, it can't be installed on a TI-73 or 83+ BE due to a bug in the boot code. So if an app is 55 bytes mod 64, RabbitSign will add an extra byte to the end, and increase the app length field accordingly. This would be easy to do at the time the app is compiled, and hard to do later.)
4. Only tangentially related to app signing, but does Axe deal with the FF-at-the-start-of-a-page bug? (If you have FF at the beginning of any app page, the OS will set the entire page to FFs when apps get defragmented. I wrote a program to patch the OS and fix this bug, but it's best to avoid that if at all possible.)
-
Perhaps this could be edited to a more friendly tone or risk a tempban?
-
Um. I'm sorry, I certainly don't mean any offense. What part of my comment did you find unfriendly?
-
Like I said over in the Mimas thread, I've been experimenting a bit to see how practical it would be to sign apps on the calculator. So I figured I'd take a look at how Axe generates applications.
(App signing on the calc is never going to be instantaneous (except maybe on the Nspire in ARM mode), so you probably don't want to do it every time you build an app, but it would be nice to have it as an option when you want to distribute your app to others.)
Thus, a few points:
1. It seems that when Axe generates an app, the "signature" is filled with zeroes. Why?! Why couldn't you use FFs? Don't you realize how much harder that makes it? :(
2. App signatures on the 83+ must begin with the bytes 02 2D 40 (followed by 66 bytes of data), not 02 0D (the latter is used for OS signatures as well as for app signatures on the 89/92+.)
3. Does Axe deal with the 55-mod-64 bug? (An application must not be exactly 55 bytes long mod 64; if it is, it can't be installed on a TI-73 or 83+ BE due to a bug in the boot code. So if an app is 55 bytes mod 64, RabbitSign will add an extra byte to the end, and increase the app length field accordingly. This would be easy to do at the time the app is compiled, and hard to do later.)
4. Only tangentially related to app signing, but does Axe deal with the FF-at-the-start-of-a-page bug? (If you have FF at the beginning of any app page, the OS will set the entire page to FFs when apps get defragmented. I wrote a program to patch the OS and fix this bug, but it's best to avoid that if at all possible.)
From the rules:
1: Flaming, bigotry, trolling or any other provocative comment aimed toward an user or group of users.
-
OK, I was just joking. I wasn't really expecting Quigibo to have thought of that, just trying to point out, in a slightly facetious way, that it would have been nice if he had.
-
Thanks, I appreciate it.
-
1. Yeah, you're probably right. I wasn't really expecting to sign the app on-calc at the time I wrote that part so I agree, if app signing becomes feasible on-calc then of course I would have to fill it with FFs instead. But yeah, it was mostly because I was trying to save one byte using an xor a instead of a load, not thinking about the consequences... That's something I'll definitely change next version now that you mention it.
2. I was unaware of this. I just looked it up on WikiTi I think and it seemed to work after that so I didn't think anything was wrong. I guess I will also have to change that then.
3. No, I have never heard of that bug... do I have to add an extra byte to the executing code itself or can I tack it onto another field? And is that size mod 64 including the signature and header?
4. Yeah, I used to have that bug in the compiler itself, but I fixed it one I realized that that was the problem. But
Axe currently only allows single paged apps and so I don't have this situation when compiling.
-
It's the size of the app including the header, but not including the signature (so, starting from 4000h, up to the 02h byte that marks the beginning of the signature.) Or to put it a different way, the app length field in the header should be set to something other than 49 mod 64.
You can just stick an extra byte onto the end, before the signature - that's what RabbitSign does, and it shouldn't make any difference to the app, unless you're depending on that byte being set to 2 for some reason.
-
Sounds great! okay, will it have a progress bar, because if it is going to take longer than 2 seconds, a percentage bar/count would be nice. How exactly are you going to do this? Hash the app, then the signing part is probably going to take the longest. Well, good luck with that!
-
I agree with Graph. When something is processing and there are no progress bar, it's hard to tell if the calc froze or if it's just taking a while. Some people may think the former then remove a battery, which I think isn't good when writing to Flash.
-
Awesome! I hope that Floppus and Quigibo can get this working. It's going to be wonderful for Axe and Mimas. :)
-
A progress bar? Sure, that would be easy enough to do. I guess I should point out, though, there are two distinct parts to app signing: MD5 hashing, and the Rabin algorithm or "signing" proper. The MD5 calculation takes time proportional to the size of the application, and will be significantly faster on an 83+ SE or 84+ due to hardware acceleration (I'm not sure about the Nspire, but it's probably comparable to the 84+.) The Rabin algorithm, in contrast, will take almost exactly the same amount of CPU time regardless of the application. So the two parts aren't directly comparable.
-
Ah right. Hopefully it should be bearable for the programmer :)
-
An update for those who care: my app signing code works, and works pretty well; the remaining difficulties are with the practical side of writing and erasing signatures in existing apps.
Compiling apps with Axe seems to be a little flaky...
- Sometimes the application is deleted as soon as it's created. I'm not entirely sure, but I think what is happening is that the "app-installed" bit isn't getting cleared, or isn't getting cleared soon enough. At some point, Axe calls _Arc_Unarc (or maybe some other archive-related routine), and somewhere deep inside that routine, a routine is called that checks the app-installed bits, and deletes any apps that don't have their corresponding bits cleared.
(In case you weren't aware: For an app whose base page is (16h - k), (2Ah - k), or (6Ah - k), depending on the model, the app-installed bit is bit (k % 8) at address 5FE0h + (k / 8) or 7FE0h + (k / 8), depending on which certificate sector is in use. Thus, if the 4000h sector is in use, bit 1 of 5FE0h always corresponds to the first installed app, regardless of the calc model. If that's a one-page app, bit 2 of 5FE0h corresponds to the second installed app. And so forth.)
So, you should be sure to clear the bit as soon as you write the app header and before you make any other OS calls, particularly anything archive-related.
- Sometimes, rather than an all-zeroes signature, the signature contains the bytes 02 0D 40 (not quite correct, as we discussed above) followed by a bunch of apparently-random data. I'm actually not 100% sure this is Axe's doing, though, as it's proving difficult to reproduce.
-
FloppusMaximus, unfortunately I am not experienced with that stuff, but could what you are describing be why app compiling doesn't work on my TI-83 Plus and someone else's calc? (I forgot who, but if I remember it was a 84+). Everyone else who has a 83+ can compile fine. When I compile on my 83+, sometimes it freezes on the creating app message, sometimes the calc turns off then greets me with a RAM clear, sometimes it exits fine, but in all cases, the APP is never created, and if I get away with no RAM clear, opening the Y= menu causes one. (There's no Y= junk in the MEM menu). If I can find the video I'll post the link here.
-
Hmm, I wouldn't think so. What seems to happen in this case is that the app gets compiled as usual, it just gets marked as deleted right away (so it doesn't show up in the apps menu, and as soon as you try to run another app, the OS will start defragmenting.)
Following up... This seems not to be an issue on 84+es at all, and may not be an issue on newer 83+es either. I don't really know very much about the internals of that part of the OS, and the more I look at it the weirder it gets. I'm not sure when exactly these status bits are supposed to be set or cleared in normal operation; the one thing I can say for sure is that if the bit is set, the app isn't considered valid and will end up getting deleted. I think maybe this was an old security measure that TI decided to stop using, or maybe it was never fully implemented to begin with.
-
Ah ok. I wonder if you might know what might be my problem. People suggest me it might be my flash chip wearing out. This calc was used since 2001 but barely until 2004 (it used to be my bro's calc). What is weird is that I have no problems sending stuff to my archive, same for archiving/unarchiving or during GC.
-
It is possible, I guess, that you have something weird going on in the certificate page. If you send me a copy I could take a look - I think BrandonW has tools for backing up and restoring the certificate.
Edit: Yes, he does: http://www.brandonw.net/calcstuff/certtools.zip
-
Mhmm, the certificate... I haven't thought about that. I'll try to get a dump from mine and send you a copy via e-mail or PM.
EDIT: Hmm it freezes on CERTIMG created ??? (even after a full mem clear). I had to remove a battery, then I got a RAM clear, and no appvar was created.
-
Well, that's a pretty straightforward program. Perhaps it's Brandon's unlocking technique that your calculator doesn't like (I believe Axe uses the same technique.) You could try this version instead.
-
Cool that one worked fine. I'll send you a copy of my certificate via e-mail. :)
EDIT: Sent. On a side note I'm not sure if this can be useful but my TI-83 Plus serial number ends with N-0700A.
-
It looks fine to me. Though, it looks like you probably would run into the apps-getting-deleted issue I was talking about, if you could actually get Axe to compile apps.
But it sounds to me like for whatever reason, the Flash unlocking code isn't working correctly. The problem is, the way Brandon's method works involves sending some invalid commands to the Flash chip, and different chips may respond differently. The method used in the program I posted above should be somewhat more robust, although I haven't tested it on all the possible platforms yet.
(By the way, my 83+ BE is even older - 1199A - and Brandon's method seems to work fine. So, who knows.)
-
Weird. I wonder if it's random from calc to calc...
Where does your calc come from, though? Most people here had their calc made in China, but mine was made in Taiwan. Could they have used different or even failing hardware there?
-
It says it's from China.
-
Actually disregard what I said... it's my other 83+ (with a broken link port) that is from Taiwan. I got confused because I use the other one so much now so it almost feels like my original 83+. X.x. I guess factories might not make a difference, then. My bro got this one at Staples and I got my broken one at Radio Shack back when Radio Shack still existed over here.
-
Bump!
Here is the first version of my on-calculator app signer. The name was kind of inevitable, following the grand tradition of silly names for third-party app signing tools.
I've tested this with a number of different configurations, in emulators and on real calculators, but I can't test everything. So please try it out and let me know if it works (i.e., after signing an application, can you then send it to another calculator?) You should also, of course, let me know if it crashes and wipes out all your programs.
It should be mentioned that this program currently does not work on "Chameleonized" calculators.
Quigibo, if you're reading this, you should take a look at the example code (example.asm) which is included in the package. It shows how to set the expiration count and app-installed bit for a newly installed app. Anyone else who's interested in compiling apps on the calculator may want to have a look at it, too.
(Trying again...)
-
Attachment coming soon to a forum near you...
How long does it take to sign it? Does it work on multi page apps?
-
Yes, it should work for multi-page apps.
Computing the MD5 hash takes about 4 seconds per page on an 83+ BE, 1 second per page on an SE or 84+. After that it takes about 50 seconds on a BE, or 20 seconds on an SE, to compute the signature.
-
That's faster than I thought! I remember Quiguibo said it might take about 10-15 minutes.
-
Yeah, BrandonW wrote a program to re-sign the calculator OS (not apps, but it's a similar amount of work); I don't remember offhand how long that program was supposed to take, but it was somewhere on the order of 10-20 minutes. Anyway, that program wasn't very optimized. :)
-
I see, I think I remember the OS signer now. Gonna download your file now. :)
-
Oh ho!, good sir - this shall be teletranscombulated to my calculator with rapid speed for prompt testing!
Edit: And, alas, it did not work for me. I created a short program that would display some text then wait for a key press to quit. I tested it to ensure it would operate normally, used Ducksign, then tried using it for about 16 times, but it defragmented. Could the fact that I used Axe have anything to do with it?
-
Sorry, I should have been more clear. DuckSign is for signing the application so that you can send it to another calculator. Setting the expiration count is a separate issue, and one that should probably be handled by the Axe Parser itself.
-
Ah, I see.
I had assumed that the two were related to each other, so that signing the application would also handle the expiration count.
So if I were to sign the app and transfer it to another calculator, would it still have the expiration count?
-
If you send it to another calculator, it will not expire. What I'm calling the expiration count is a field on the certificate page; it's not stored within the app, but it's set by the OS at the time the app is installed. For normal, non-limited-trial apps, it should be set to the value 80 00. Since Axe circumvents the normal installation procedure, this doesn't happen, so the field is left at its default value (FF FF), which is interpreted to mean that the app should expire after being run 16 times.
-
If you send it to another calculator, it will not expire. What I'm calling the expiration count is a field on the certificate page; it's not stored within the app, but it's set by the OS at the time the app is installed. For normal, non-limited-trial apps, it should be set to the value 80 00. Since Axe circumvents the normal installation procedure, this doesn't happen, so the field is left at its default value (FF FF), which is interpreted to mean that the app should expire after being run 16 times.
So, if axe changed those two bytes, would it fix it? Wait, can we even write to the certificate that easily?
-
Well, as "easily" as writing to other areas in Flash. :P (And yes, that's kind of dangerous; if you screw up the certificate, it's possible to crash the boot code and make it impossible to install an OS.)
-
Well, as "easily" as writing to other areas in Flash. :P (And yes, that's kind of dangerous; if you screw up the certificate, it's possible to crash the boot code and make it impossible to install an OS.)
Yeah, true. It's not completely a brick, though, is it?
-
I'm not entirely sure, actually. BrandonW devised a technique that you can use to run arbitrary code by sending specially crafted link packets. But in order to actually fix the damage and reinstall an OS, you would then need to unlock Flash somehow; this can always be done on an 83+ SE or 84+, but I don't know if there's any way to do it on an 83+ BE or 73 with no OS installed.
-
Yeah, I guess you could just make a backup, reset the flash manually, and burn it in there if you ever need to. ;-) But I don't know, software wise. Considering it might not even respond to packets at that point.
-
I'm not entirely sure, actually. BrandonW devised a technique that you can use to run arbitrary code by sending specially crafted link packets. But in order to actually fix the damage and reinstall an OS, you would then need to unlock Flash somehow; this can always be done on an 83+ SE or 84+, but I don't know if there's any way to do it on an 83+ BE or 73 with no OS installed.
Well, invalidating the certificate does not erase ROM. The OS would still be there, and the boot page certainly would. You could use any flash exploit you want if you got arbitrary code running correctly, it would just be hard.
-
If you were ever in a situation where you needed to use that technique, some or all of the OS would not be present.
This is getting way off topic...
-
What makes you say so? It's not like invalidating the certificate will insta-erase everything.
-
No, of course it doesn't. We were talking about what to do if you get into the situation where you cannot install an OS. Which implies that your OS has been partially or completely erased somehow.
-
Wasn't BrandonW program something that re-installed a certificate in case yours got corrupted/broken? I wonder what do we do if both the entire OS and the certificate are gone, though...
-
I wonder if it is possible to reinstall the OS if every single bit of writable information has been reset o.O
-
By the way, I should mention that I found some stupid bugs in the Axe app compiling that I'm in the process of fixing, it should make them finally signable on-calc without needing to flash a sector in the next version. I'm not sure if I would want to merge application signing into the Parser, but I would definitely like to include it in the tools section of the Axe suite with your permission once I get it working.
Also, I really don't want to mess with the certificate page. But if I understand correctly, couldn't that issue be averted by sending the app to the computer/another calculator and then sending it back? It seems like there would have to be some kind of mechanism to prevent that or the whole trial-run business would be pointless.
-
It would be great if the tool could be included in the future. I remember some people couldn't figure out how to use Rabbitsign and Wabbitsign. As for Wappsign I heard it sometimes gives bad signatures so I don't know if we can rely on it.
-
Well, I had never heard of ON-CALC Sign until one week ago and did my first Hello World Application, I was really thankful for it.
What I have to say about this...
It looks very good but I sometimes regret compiling it as an Application because I usually have 10 programs, 1 of them is to be an App and the others 9 aren't and I compile not-to-be-applications programs to Apps. Maybe a shortcut key (MODE+2ND to compile as App?)
That would pretty neat
-
I don't think there can ever be a situation when you cannot install an OS, unless you explicitly used BrandonW's special brick code.
-
It would be great if the tool could be included in the future. I remember some people couldn't figure out how to use Rabbitsign and Wabbitsign. As for Wappsign I heard it sometimes gives bad signatures so I don't know if we can rely on it.
Wabbitsign I can't get to work either in my batch file. Though Wappsign seems okay for me, worked 20/20 of the times I built my application for testing.
-
By the way, I should mention that I found some stupid bugs in the Axe app compiling that I'm in the process of fixing, it should make them finally signable on-calc without needing to flash a sector in the next version.
That's good to hear. By the way, I believe that you can simply end the app with the bytes 02 2D 40, and stop there, leaving the rest of the page blank. (I originally thought it might be necessary to include the "f" value - the extra 1-2 bytes that come after the signature proper - but this seems not to be the case.) You can still send the unsigned app to a PC, or at least you can using TiLP - somebody should probably check that TI-Connect doesn't have a problem with it.
I'm not sure if I would want to merge application signing into the Parser, but I would definitely like to include it in the tools section of the Axe suite with your permission once I get it working.
It's free software. All I ask is that you include the source code for the tool, or link to it.
Also, I really don't want to mess with the certificate page. But if I understand correctly, couldn't that issue be averted by sending the app to the computer/another calculator and then sending it back? It seems like there would have to be some kind of mechanism to prevent that or the whole trial-run business would be pointless.
I can understand your hesitation. In this case, I'm pretty confident that the example code I gave will do the right thing, or at least won't break anything too badly. Otherwise I wouldn't recommend it.
I don't really know much about how the limited-trial system is supposed to work; I've never seen an app that used it.
-
I wonder why TI just won't remove it altogether? They don't even sell apps anymore anyway.
-
I wonder why TI just won't remove it altogether? They don't even sell apps anymore anyway.
It would break compatibility. There are still a few non-TI apps that use this system, actually.
-
Really? O.O I wonder which apps does...
-
I remember seeing a math app for sale on some website, but I'm don't remember a lot of it... Other than that you had to download the trial app to get a code you had to enter on the website before you could purchase the app. The normal calc ID didn't work.
-
sort of on/off topic, but Axe apps get erased after 16 runs? That's (sort of) a pain...(but it gets fixed after they've been signed on-comp, right?)
-
Its weird because i have a Portal App thats been run many more times than that o.O
-
sort of on/off topic, but Axe apps get erased after 16 runs? That's (sort of) a pain...(but it gets fixed after they've been signed on-comp, right?)
Essentially, yes. In reality, it's more that they get erased after a "trial period". The signature, IIRC, is only checked when the os is being sent.
-
I remember seeing a math app for sale on some website, but I'm don't remember a lot of it... Other than that you had to download the trial app to get a code you had to enter on the website before you could purchase the app. The normal calc ID didn't work.
Weird, maybe they still sell apps, but only very few of them. I remember they sold many, back then, such as Periodic. Some apps costed around $20.
As for Axe apps being deleted after 16 runs, it seems like when you compile an app on-calc with Axe, the TI-OS thinks it's an actual trial app.
-
Really? O.O I wonder which apps does...
ZoomMath is an example of a third party App that costs money. ;) http://www.zoommath.com/
-
Oh right, but I mean TI ones. I was sure TI stopped selling apps althogether.
-
I think they have, but I don't expect they'll ever take those bits of code out. And even if they did - even if they released new OSes tomorrow that did away with all the old protection mechanisms - I'd still say it's worthwhile to remain compatible with the older OSes. I mean, personally, I'm still using 2.30 on my 84+, and given the state of the newer OSes, I don't plan on upgrading that in the forseeable future. I think both my 83+es are running 1.19 now, but I kept 1.14 for a very long time, for the sake of program compatibility. I'm sure I'm not the only one.
-
I think BrandonW always recommends to update to 2.43 and 1.19 but discourages using 2.53 MP. However, I guess if you never have issues with older OSes you don't really have to bother. I personally used 1.14 for ages as well, too, then stuck to 1.18 for 2 years. The only problem with older 83+ OSes is that they have a different ASCII charmap I think, or something related to ASCII is different, and as result, certain games using some characters will only send on a calc running 1.15 or higher.
-
The only problem with older 83+ OSes is that they have a different ASCII charmap I think, or something related to ASCII is different, and as result, certain games using some characters will only send on a calc running 1.15 or higher.
They are missing a few characters, namely ~@#$&`;\|_%. They were added to allow compatibility with the TI Keyboard. 1.16 also added a few characters, but they are rarely used. Some of them are the arrows, the … character and also ß, which is used in German.
-
Those OSes also broke compatibility with old ones, though. The 256 char-long string of 3x3 black squares is gone, for example. Illusiat 6 final dungeon, for instance, no longer shows up properly past OS 1.15.
-
Is it completely gone, or is it just replaced with a more normal token?
-
It became an up arrow, if I remember. You can see how the original token looked like at 2:37 in this video:
This is a 2 bytes token, by the way.
-
So I sent this to my calculator and tested it out on an Axe app, but it said the length was wrong and refused to sign it.
-
That's probably because of the bug in 0.4.7 that causes it to not update the length field. This has been fixed for the patch.