Omnimaga

Calculator Community => TI Calculators => General Calculator Help => Topic started by: Deep Toaster on January 01, 2011, 03:39:43 pm

Title: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 03:39:43 pm
Found a bug with MirageOS (v1.2) where it glitches when renaming stuff. And so I did exactly what no one else would do: I played around with the bug until I got something more serious, and now I've corrupted my archive with no backups at all :(

I can't boot my calculator anymore without using CLEAR+ON (it flashes a random screen for half a second, then disappears). If I disable the archive with that CLEAR thing, the calculator seems to work fine, so it's probably only some part of the archive that's corrupted, not the OS.

But I lost about 12 projects in group vars. Most of them are small, and I could probably get back to where I was in those projects in a few days, but Contra, XDE, PapiJump, and Ultimate Falldown were all really far in (Contra in particular already had a nearly complete engine with enemy AI and aiming that I'd just added in the past few days), and it would take a long time before I can get all of it coded again.

So here's my question: Anyone willing to give me directions on how to get as much of it back as I can? I've never really done anything with Flash before, except some simple reading. I haven't GarbageCollect'd or done anything else to my archive since the crash, so the data should still be there, but some of it's definitely corrupted. Thanks in advance for any info!
Title: Re: Recovering from a corrupted archive
Post by: jnesselr on January 01, 2011, 03:42:54 pm
Well, I guess an asm program could be written to make all the flash pages valid.
Title: Re: Recovering from a corrupted archive
Post by: Silver Shadow on January 01, 2011, 03:43:57 pm
Sorry to hear, but I doubt you can recover them if this is this bad. :( Just hope that your calc isn't completely broken.
Title: Re: Recovering from a corrupted archive
Post by: Happybobjr on January 01, 2011, 05:27:58 pm
what do you have in your ram?
anything inportant?
Title: Re: Recovering from a corrupted archive
Post by: Fast Crash on January 01, 2011, 05:33:00 pm
Did you make a RAM clear ?
Title: Re: Recovering from a corrupted archive
Post by: Munchor on January 01, 2011, 05:43:59 pm
@graphmastur: I once managed to get EVERYTHING from a corrupted archive. HOW? I don't know, I crashed my calculator and when I turned it back on it was good :D So, it was a random day, a few days after getting it corrupted. It was a random occasion.
Title: Re: Recovering from a corrupted archive
Post by: souvik1997 on January 01, 2011, 05:44:46 pm
I don't think that would help, considering his calc can't boot without ON-CLEAR. My calculator had the same problem about 2 months ago, and I had to reset the entire memory (archive+ram).
Title: Re: Recovering from a corrupted archive
Post by: jnesselr on January 01, 2011, 05:55:48 pm
@graphmastur: I once managed to get EVERYTHING from a corrupted archive. HOW? I don't know, I crashed my calculator and when I turned it back on it was good :D So, it was a random day, a few days after getting it corrupted. It was a random occasion.
If you turned it on and it was good, without you doing anything, then it wasn't a corrupted archive at all.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 06:01:45 pm
Sorry to hear, but I doubt you can recover them if this is this bad. :( Just hope that your calc isn't completely broken.

Yeah, it still works, thankfully :D And all my apps are still there. It's just that since I have to boot with CLEAR, all my archived/grouped vars are hidden.

what do you have in your ram?
anything inportant?
Did you make a RAM clear ?

Yeah, of course. I clear my RAM at least a dozen times a day anyway, so I don't care at all. What I'm trying to do is recover all my projects from the archive.

@graphmastur: I once managed to get EVERYTHING from a corrupted archive. HOW? I don't know, I crashed my calculator and when I turned it back on it was good :D So, it was a random day, a few days after getting it corrupted. It was a random occasion.

That's not a corrupted archive. If your calculator still works, it's only your RAM that got messed up, and after a RAM clear the RAM's set back to default. I do that just about every day too :D

Well, I guess an asm program could be written to make all the flash pages valid.

I don't know how... What makes a flash page "valid"?
Title: Re: Recovering from a corrupted archive
Post by: Fast Crash on January 01, 2011, 06:06:27 pm
Flash pages are mostly for apps
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 06:09:24 pm
Flash pages are mostly for apps

Flash is all the memory the calculator has that's not RAM. It's used for the OS, apps, and variables. The reason my apps are still there is because I used TI's CLEAR+ON boot, which only searches for apps in the archive.

All other flash variables are not added to the VAT, and that's why they're gone from my memory list. They're still physically there in the flash memory, just "hidden" in a way.

And so basically right now some of my flash pages are corrupted, and that's why I can't boot normally, but I don't know how that works.
Title: Re: Recovering from a corrupted archive
Post by: Fast Crash on January 01, 2011, 06:11:41 pm
Ok.

Can you link the calc with the PC after a RAM clear or the link is still btoken ?
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 06:14:07 pm
Ok.

Can you link the calc with the PC after a RAM clear or the link is still btoken ?

What do you mean by RAM clear? ??? My link's not broken, though.
Title: Re: Recovering from a corrupted archive
Post by: Fast Crash on January 01, 2011, 06:16:33 pm
usually when the RAM is corrupted you cannot link it to the computer ( it's detected but you can't transfer data ). So if you reset your RAM it should work and then you could backup your files.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 06:19:11 pm
usually when the RAM is corrupted you cannot link it to the computer ( it's detected but you can't transfer data ). So if you reset your RAM it should work and then you could backup your files.

It's not my RAM that's corrupted. I corrupt my RAM every day :D It's the archive that's corrupted, and I don't want to reset my Flash, because then my groups would definitely be gone.

On a related note, my calculator crashes every time I RAM clear now because it tries to boot normally afterwards.
Title: Re: Recovering from a corrupted archive
Post by: Fast Crash on January 01, 2011, 06:21:03 pm
if your Flash is corrupted, your RAM is also corrupted
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 01, 2011, 06:29:03 pm
Flash is flash and RAM is RAM, thoughh... Unless you mean it doesn't get recorded in the VAT correctly? I'm using the CLEAR boot, which doesn't look for variables in flash when the calculator boots, so the VAT is empty of archived vars. If I were to make it boot normally, the calculator would get stuck partway through the flash screening and crash, because my archive is corrupted. That's what I'm trying to fix right now.
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 01, 2011, 10:19:23 pm
Dumping your ROM might help, but it'll be slow. You'll have to dump a page, transfer it to PC, dump the next one, transfer it, and so on for as many pages as their are (32 on 83+, 64 on 84+, 128 on 83/4+SE)
Do you have GroupTool (http://www.ticalc.org/archives/files/fileinfo/385/38560.html) on your calc? It might be able to extract groups, even if they aren't in the VAT. (Not sure about this, though, and it's not like you are able to send an application x.x)
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 02, 2011, 09:51:27 am
Darn, I don't...
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 02, 2011, 10:17:23 am
Too bad. :(
If I have time later today I'll write a quick program to dump a ROM page to an appvar so you can start to dump your ROM. No promises, though.
Title: Re: Recovering from a corrupted archive
Post by: ztrumpet on January 02, 2011, 11:09:03 am
What was the name (before and after) of the program that you changed that killed your calc?  Also, was it Archived/Hidden/Locked?

I wish you luck, Deep Thought! :)
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 02, 2011, 11:14:27 am
I renamed Simul and the Impossible Game to something I don't remember (both were archived and locked), then renamed Contra to CON and back (locked). The last one was probably the one that messed things up because I had a group at the time called CON. Also, Mirage points to a random location in memory after a rename, so renaming it again does something weird to that random spot in memoroy (that's the bug I was playing with).

Too bad. :(
If I have time later today I'll write a quick program to dump a ROM page to an appvar so you can start to dump your ROM. No promises, though.

Thanks, calcdude! Do I just copy it over straight from flash? If so I'll try writing one.

EDIT: BrandonW gave me some tips (more like a full page explanation) here (http://www.cemetech.net/forum/viewtopic.php?p=130411#130411) (in case anyone else ever gets stuck here :P).
Title: Re: Recovering from a corrupted archive
Post by: Lionel Debroux on January 02, 2011, 02:33:05 pm
FWIW, TILP has long been able to perform ROM dumps. Thanks to BrandonW, TILP II 1.14 can dump a 84+(SE) through direct USB connection :)

[EDIT: using TILP is exactly what BrandonW is suggesting on Cemetech :D]
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 02, 2011, 02:45:11 pm
I'd do that, but I have a TI-83 Plus with no cable :(

I'll have t copy one flash page at a time to an appvar and piece it back together on-comp.
Title: Re: Recovering from a corrupted archive
Post by: Lionel Debroux on January 02, 2011, 02:52:44 pm
Ah, yes, if you're transferring to the computer through another calculator, ROM dumping won't work for you...
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 02, 2011, 04:08:31 pm
He can still transfer page by page, fortunately.
I'll write that program now.
Edit: Here you go :) When you run it, type the page number to dump in hex (no need to press alpha). It will create a 16KiB appvar called PAGEXX, where XX is the page. Note that you need enough free RAM. ;D
Edit 2: Note that the pages for an 83+ are from 00 to 1F
Edit 3: This program cannot dump the certificate (page $1E on an 83+). However, the program GETCERT in the Free83P (http://www.ticalc.org/archives/files/fileinfo/420/42048.html) zip file can.
Title: Re: Recovering from a corrupted archive
Post by: jnesselr on January 02, 2011, 09:29:18 pm
Can you give me the source for that, I'd like to learn it.
Title: Re: Recovering from a corrupted archive
Post by: Builderboy on January 03, 2011, 01:54:42 am
I hope everything can be recovered D: or at least the most important things, this sounds promising so i wish you the best of luck!
Title: Re: Recovering from a corrupted archive
Post by: thepenguin77 on January 03, 2011, 12:25:23 pm
Dr.Dnar actually wrote a program to recover deleted things in the archive. Here (http://www.ticalc.org/archives/files/fileinfo/411/41120.html) it is. If it encounters a problem and says archive corrupted, you, I, or someone can change the code to make it persist. (I remember Dr.Dnar saying that was one problem he didn't fix.)

I have probably recovered about 20 programs with this, so it definitely works.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 03, 2011, 02:08:41 pm
I know someone who has that on their calculator already... I'll try it. Thanks for the idea.
Title: Re: Recovering from a corrupted archive
Post by: Builderboy on January 03, 2011, 02:10:35 pm
I hope it can save group files as well as program O.O
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 03, 2011, 02:11:41 pm
Oh, wait, I just remembered: It doesn't recognize groups (I've tried it before and it didn't work). Only programs and appvars, I think :(
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 03, 2011, 06:26:45 pm
Yeah, that's why I didn't suggest it :(
graph: here's the source. If it fails to compile blame the fact I reformatted it from Mimas. I can post a commented version later if you want.
Title: Re: Recovering from a corrupted archive
Post by: jnesselr on January 03, 2011, 09:42:59 pm
Thank you very much, it was a good read.
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 03, 2011, 10:09:21 pm
You're welcome :)
My favorite part was my (ab)use of textShadow :P
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 03, 2011, 11:10:53 pm
He can still transfer page by page, fortunately.
I'll write that program now.
Edit: Here you go :) When you run it, type the page number to dump in hex (no need to press alpha). It will create a 16KiB appvar called PAGEXX, where XX is the page. Note that you need enough free RAM. ;D
Edit 2: Note that the pages for an 83+ are from 00 to 1F
Edit 3: This program cannot dump the certificate (page $1E on an 83+). However, the program GETCERT in the Free83P (http://www.ticalc.org/archives/files/fileinfo/420/42048.html) zip file can.

Whoa, I didn't even see those edits! Whoops :-[

Thanks calcdude! Btw, which pages do I need if all I want to do is get my archive space? 08h through 15h?
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 04, 2011, 12:55:43 am
Pages $08-$0B ought to be the swap sector, so just $0C-$15 should suffice. However, you should probably dump $08-$0C and $16-$17 as well, and your certificate. That should be enough to reconstruct your ROM (along with $1F from rom8x) as long as your OS isn't corrupted. Good luck!
Title: Re: Recovering from a corrupted archive
Post by: Builderboy on January 04, 2011, 01:17:59 am
I hope this works O.O
Title: Re: Recovering from a corrupted archive
Post by: DJ Omnimaga on January 04, 2011, 03:35:09 am
Me too, I would hate to see Contra and your other projects die, or seeing you having to restart from scratch. Tell us if you got any luck with this!
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 04, 2011, 12:13:37 pm
No time to post, but just an update: Unsquished the program and ran it on my calc. It works, thanks calcdude!
Title: Re: Recovering from a corrupted archive
Post by: thepenguin77 on January 04, 2011, 03:17:48 pm
Pages $08-$0B ought to be the swap sector, so just $0C-$15 should suffice. However, you should probably dump $08-$0C and $16-$17 as well, and your certificate. That should be enough to reconstruct your ROM (along with $1F from rom8x) as long as your OS isn't corrupted. Good luck!

Btw, you can't dump the certificate without unlocking flash first. ;)
Title: Re: Recovering from a corrupted archive
Post by: DJ Omnimaga on January 04, 2011, 03:21:28 pm
No time to post, but just an update: Unsquished the program and ran it on my calc. It works, thanks calcdude!
Really? Did you manage to recover anything so far?
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 04, 2011, 04:02:39 pm
No time to post, but just an update: Unsquished the program and ran it on my calc. It works, thanks calcdude!
Really? Did you manage to recover anything so far?

Nope, still dumping ot anothr calc.

And I just finished! I'll uploaded as soon as I can.

As a reality check, CERTIMG should be 3984 bytes, right?
Title: Re: Recovering from a corrupted archive
Post by: JosJuice on January 04, 2011, 04:04:56 pm
As a reality check, CERTIMG should be 3984 bytes, right?
I think it's supposed to be 16 kb (the size of one page)...
Title: Re: Recovering from a corrupted archive
Post by: DJ Omnimaga on January 04, 2011, 04:52:52 pm
No time to post, but just an update: Unsquished the program and ran it on my calc. It works, thanks calcdude!
Really? Did you manage to recover anything so far?

Nope, still dumping ot anothr calc.

And I just finished! I'll uploaded as soon as I can.

As a reality check, CERTIMG should be 3984 bytes, right?
Ok. I hope your Axe source can be retrieved.
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 04, 2011, 05:41:30 pm
As a reality check, CERTIMG should be 3984 bytes, right?
I think it's supposed to be 16 kb (the size of one page)...
Yeah, it should be 16KiB.
As for getting the ROM someone should be able to write a script to combine all those appvars and the OS file into a ROM image.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 04, 2011, 06:28:34 pm
Darn, that means I typed something in wrong D: I had to type in the hex for GETCERT by hand (probably a bad idea). I also typed your dumper by hand, though, and it turned out well, thankfully. 16399 bytes, right? 16384 + two size bytes + VAT entry with 6-letter name?
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 04, 2011, 06:36:20 pm
Yep, mine should generate appvars of that size. :)
That's not good about GETCERT. :/ I guess you should try retyping it... I'm not sure what else you could do.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 04, 2011, 06:37:09 pm
Does the certification ever change? I want to clear my archive, just in case... Then I'll run it.
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 04, 2011, 06:40:09 pm
The certificate changes most importantly upon receiving/deleting an app. If you clear your archive, the certificate image you generate cannot be used to generate an accurate ROM image of your corrupted archive. If all you wish to do is scan User Archive for your files, though, and not mess around with it in an emulator, then you can just clear your archive and not bother getting the certificate.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 04, 2011, 06:41:37 pm
Oh, I see. I already accidentally archived a variable again, though. Would that change it?
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 04, 2011, 06:43:08 pm
That shouldn't change the certificate. But again, unless you want to mess around in an emulator, you don't need it.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 10:38:27 am
Here they are. I don't know how to build a ROM from it, so can someone give me info/make one for me? TIA!

I don't have CERTIMG, though, since I somehow made a corrupted version. I just got a working one done, so I'll post that in a few days.

And I'll try to look for those groups... How are they stored? Do they just have a list of VAT entries to the actual programs or somethign?

EDIT: Just realized: CERTIMG is copyrighted or something, right? In that case I won't actually post it here.
Title: Re: Recovering from a corrupted archive
Post by: JosJuice on January 06, 2011, 10:54:12 am
EDIT: Just realized: CERTIMG is copyrighted or something, right? In that case I won't actually post it here.
There's not any code in CERTIMG - there's just data (such as your calc ID, and permissions for your installed apps).
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 11:52:13 am
Update: Found my group CON!

The last copy (which should be the newest) is stored completely on page $11, which should make it easier to get it off. But I don't know exaclty how groups are stored, so anyone have info/link? Thanks!

EDIT: Another update: Found XDE on page $0C.

EDIT2: Got them back! Figured out how the groups were stored and used Calcsys to hack them back. I've gotten Contra and PapiJump back already, and now I'm looking for XDE :D
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 06, 2011, 08:02:00 pm
Nice! Good luck finding the rest! :D
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 08:03:52 pm
Got Contra, PapiJump, XDE, and a bunch of misc. programs back. I tried to recover Ultimate Falldown the same way, but my calc glitched (which probably means that's the group that got corrupted). The ones I didn't get were minor projects, and I could probably recode them in a few minutes. So now I've reset my entire archive, and everything's good :)
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 06, 2011, 08:16:19 pm
Nice. I guess you didn't need to make those appvars after all :P
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 08:17:06 pm
Yeah, I did need to :D It's a lot easier to search for strings of hex on the computer than on the calc.
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 06, 2011, 08:20:00 pm
Ah. Calcsys does have a SEARCH command, though ;D
I guess you could briefly post the method you used like you said in the Contra thread?
Title: Re: Recovering from a corrupted archive
Post by: turiqwalrus on January 06, 2011, 08:22:17 pm
good to see that you have that stuff back, as most of it looked really good.
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 08:23:43 pm
Ah. Calcsys does have a SEARCH command, though ;D
I guess you could briefly post the method you used like you said in the Contra thread?


Yeah, but only for a single byte. I used Hex Fiend (on Mac) to search for the entire name of each project :)
Title: Re: Recovering from a corrupted archive
Post by: calcdude84se on January 06, 2011, 08:24:45 pm
No, I meant that in the console, if you type SEARCH followed by a hex string, it will search for those bytes in ROM. :D
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 06, 2011, 08:32:42 pm
Oh, really? Whoops...

Darn, didn't know that. I really should start RTFM.
Title: Re: Recovering from a corrupted archive
Post by: DJ Omnimaga on January 07, 2011, 06:26:14 pm
Glad you got your stuff back!
Title: Re: Recovering from a corrupted archive
Post by: Deep Toaster on January 08, 2011, 12:28:37 am
/me is glad too :D

Btw, some info on recovery here: http://ourl.ca/8664/161736 (http://ourl.ca/8664/161736)

Still have to add the actual info for recovering archived vars, though. Comments?
Title: Re: Recovering from a corrupted archive
Post by: DJ Omnimaga on January 09, 2011, 08:01:39 pm
Cool I'll check the topic :D