Author Topic: Athena: Calculator project installer  (Read 13194 times)

0 Members and 1 Guest are viewing this topic.

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6265
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: Athena: Calculator project installer
« Reply #30 on: December 15, 2010, 12:01:05 am »
Hmm, strange...so Control Panel\Programs\Default Programs\Set Associations doesn't work? I would think it would...maybe I'm just crazy
/e

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #31 on: December 15, 2010, 12:07:52 am »
Nope it doesn't work. It never could set it again to the same application and icon it did before.

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6265
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: Athena: Calculator project installer
« Reply #32 on: December 15, 2010, 12:22:07 am »
Hmm, strange...I would have thought that would work. I guess the only way would be to do it the way Iambian said.
/e

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #33 on: December 15, 2010, 02:01:35 am »
I believe I have the Athena packer generating files correctly, so I guess it's largely a success?
The only difference is that the data files start with a "Z" instead of a theta. Doesn't make that much of a difference, since you can pack to appvars instead of programs.

For anyone interested, here's some documentation about the files themselves.
Code: [Select]
-------------------------------------------------------------------------------
Documentation regarding the package file format for the data
portion of the TI-83/84+(SE) installer software. All instances of multibyte
numeric variables are stored little-endian.

Installations using the Athena packager/installer will contain at least two
files in addition to the installer: (1) Metadate file (2) Data files.

Metadata files contain information about the install. This includes, in no
particular order, the number of data files there are, their filenames, the
number of files that are archived inside the package, the estimated amount of
FlashROM the install will use, the amount of RAM it will use, and a slightly
longer (14 character) description field.

Metadata file structure:

Ofs Siz Desc
+00 08: header. $BB,$6D,$C9,$31,$80,$00,$01,$09
+08 01: number of installer files (64KB chunks)
+09 02: number of files to be installed (can be quite a few!)
+11 03: number of bytes of FlashROM the installation will take up
+14 02: number of bytes of RAM the installation will take up
+16 01: Install type flag (not used yet)
+17 14: 14 bytes ASCII text project project name
+31 01: RESERVED
+32 --: Names of installer files (all 8 char long). This is sized
        as number of installer files times eight.

The data files that come after the metadata file contains a number of streams
formatted in the following way:

+00    1b: filetype + flags. Matches TI-OS file types, except bit 7: 0=In RAM
+01    2b: checksum. Used for comparing installed files with this one.
+03    2b: file size (includes possible filesize field that comes with file)
+05    1b: filename length
+06    nb: filename
+06+n    : data stream

The number of these streams will not be known until EOF is encountered. The
amount of data in this file can be up to about 65500 bytes. None of the data
streams are split for the purpose of extracting data from the file. These data
streams are pucrunch-compressed as a solid archive, so it will be up to the
reader to decompress only bytes at a time and then read the information for
its formatting and its EOF.

The name of these data files is derived from the name of the project and the
uncompressed contents of the file. Every file name will be of the form:

Zfffnnnn
Z is just the letter Z
fff are 3 characters from 0-9 A-V. Hashed from name, so will remain constant.
nnnn are 4 characters from 0-9 A-V, hashed from data, so will likey vary.

Examples: These are true examples from an experiment done on packaging
Keven Oullet's (DJ Omnimaga) somewhat incomplete Illusiat 13.

On entering into the command line:
athena -a I13 "Illusiat 13"

I get the following files out:
I13.8xp          Size (bytes) : 132
ZNG0H7DI.8xv     Size (bytes) : 24716
ZNG0LK35.8xv     Size (bytes) : 4846
ZNGOLRSF.8xv     Size (bytes) : 11936

The input was sized collectively at 138327 bytes, so this is also a great
compression ratio. Understand that the actual filesizes on-calc are going to
be a slight bit smaller since calculator files on the computer contain
additional information that is useful for the computer to calculator linking
program.

-------------------------------------------------------------------------------
Documentation regarding the creation of DCS folders

Not created yet.
A Cherry-Flavored Iambian draws near... what do you do? ...

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #34 on: December 15, 2010, 04:16:14 am »
Wow this compressed A LOT O.O

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #35 on: December 16, 2010, 04:34:18 pm »
Spent all day yesterday working on the Z80 ASM side of Athena. It will now correctly detect metadata files and their associated data files, and I can have the program print out statistics written directly into the metadata file. Attached to this post is an image of what I'm seeing in the statistics view.

What I've got left to do is to edit the decompression routine to get it working for Athena, then start writing out the data that's in there. I can imagine that the hardest parts would be getting the decompression routine to work for me, and creating the appropriate files from the data that is read out. After that, it's rather trivial to print the progress to the screen and archive the files as necessary.

After looking for bugs, I'll then put this thing up for people to test. DJ Omnimaga's probably gonna be the first to get it because he's got the resources and the knowledge to fully test this Athena. I pretty much need all the large BASIC games I can find so this thing can be tested out.
A Cherry-Flavored Iambian draws near... what do you do? ...

Offline thepenguin77

  • z80 Assembly Master
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1594
  • Rating: +823/-5
  • The game in my avatar is bit.ly/p0zPWu
    • View Profile
Re: Athena: Calculator project installer
« Reply #36 on: December 16, 2010, 06:34:18 pm »
For me, windows is convinced 8xp are text files and will refuse to treat them as anything else x.x
easy fix, just go to the control panel and change file associations

Iambian, I'm really looking forward to this :)
This doesn't always work I think. I accidentally made 8xk files be openable by Notepad once and I never could find the original TI-Connect app icon again in control panel afterward, so now there's no send to TI device option anymore, I have to double-click the APP. Reinstalling TI-Connect doesn't do the trick. Once you accidentally change TI-Connect file associations, it's over.

I did the exact same thing. The problem is that you want the default file extension which is not strictly attached to a program. Use this. Worked like a charm.
zStart v1.3.013 9-20-2013 
All of my utilities
TI-Connect Help
You can build a statue out of either 1'x1' blocks or 12'x12' blocks. The 1'x1' blocks will take a lot longer, but the final product is worth it.
       -Runer112

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #37 on: December 16, 2010, 11:44:12 pm »
Ok thanks I'll have to try this.

Iambian nice! I don't think I'll have time to test every program I made, though, but I could maybe try some of the largest games like Zelda DLQ and Illusiat 13.
« Last Edit: December 16, 2010, 11:46:38 pm by DJ Omnimaga »

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #38 on: December 18, 2010, 11:19:08 am »
Well... I sorta ran into a snag. It relates to the compression and the limits of the edits I can make with the decompressor. Basically, I'm going to have to opt for weaker compression to make the edits, so projects won't get the best possible results when using pucrunch.

I'm sorry, but I knew these great results were a little ... too promising. It just won't work for what I've gotta do with these files. (pucrunch expects all decompressed data to remain where it is in one contiguous block. I cannot have it that way)

Still... for Illusiat 13, it goes from 135KB to 55.4KB instead of the much rosier 40KB. The results will most certainly vary depending on how far apart the LZ matches are. For those who know anything about the LZ77 algorithm ( I have marginal knowledge :P ), I had to restrict the search range to 255. I know this isn't very optimal, but pucrunch does a good job in making it work well. There's still decent compression.
« Last Edit: December 18, 2010, 11:20:07 am by Iambian »
A Cherry-Flavored Iambian draws near... what do you do? ...

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #39 on: December 18, 2010, 06:35:33 pm »
Sorry to hear. That said, the new compression still seems excellent to me. 135 KB to 55.4 KB is pretty great. Nice job, still. :D

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #40 on: December 18, 2010, 10:09:56 pm »
Just to make it clear, I'm still using pucrunch. It's just that I've had to set an option in the compiler to force it to restrict its search range. Thankfully, I can do that. At the moment, the edits are already made in the ASM program, though I will still need to test it all out to see if it works out right. I'm hoping it will, but who knows?
Spoiler For the curious:
This is the line of code used in Athena to run pucrunch:
pucrunch -d -c0 -r255 Athout_i.bin Athout_c.bin
Where the underscore characters are filled in with some character 0-9,A-Z
If you really want to know, this is the function I wrote to do the job. Some of the casts just aren't necessary, but still. I'm a newb when it comes to the C thing.

const char *alphanums = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
const char *tiheader  = "**TI83F*\x1A\x0A\0Built using the Athena Packager Software  "; //53 bytes
const char *tifileheader = "**TI83F*\x1A\x0A\0";
const char *athenaheader = "\xBB\x6D\xC9\x31\x80\x00\x01\x09";
int flushtofile(char *inputstream,int appendfilenum,int outfilelength)
{
    char tempfilename[]="Athout_i.bin\0";
    tempfilename[6] = alphanums[appendfilenum];
    FILE *newfileptr;
    newfileptr = fopen(tempfilename,"wb");
    if (newfileptr == NULL) return 1;
    if (outfilelength!=fwrite(inputstream,1,outfilelength,newfileptr)) {
        printf("Err: Could not flush buffer to file!");
        (void) fclose(newfileptr);
        return 1;
    }
    (void) fclose(newfileptr);
    if (!system(NULL)) return 1;
    char tempbatch[]="pucrunch -d -c0 -r255 Athout_i.bin Athout_c.bin\0";
    tempbatch[28]= alphanums[appendfilenum];
    tempbatch[41]= alphanums[appendfilenum];
    system(tempbatch);      //pucrunch will always output some sort of error. Newp.
    remove(tempfilename);  //assumes it can be removed. We just created and closed it...
    return 0;
}

« Last Edit: December 18, 2010, 10:10:41 pm by Iambian »
A Cherry-Flavored Iambian draws near... what do you do? ...

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #41 on: December 18, 2010, 10:25:39 pm »
I see. Do you think there will be a release, soon? (beta-test?)

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #42 on: December 18, 2010, 10:34:04 pm »
I see. Do you think there will be a release, soon? (beta-test?)
Really depends on how reliable the modified pucrunch decompressor is. If it passes with flying colors, I can have something out by this Monday or Tuesday. If not I'm not really sure.

The initial test will be decompressing blocks of 256 at $8000 and examining it against the same file on the computer opened up under a hex editor.

If that passes, I'll start with actually decompressing the files and comparing the results against the built-in checksums.

If *that* passes, I'll start writing in functions and start taking care of the various file formats that one might possibly throw into Athena. I'll be sure to exclude some formats (like window settings), but I should be able to cover the majority of the stuff that most projects contain.
A Cherry-Flavored Iambian draws near... what do you do? ...

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55942
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Athena: Calculator project installer
« Reply #43 on: December 18, 2010, 11:24:06 pm »
Ah ok. Hopefully things work out well.

Offline Iambian

  • Coder Of Tomorrow
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 739
  • Rating: +216/-3
  • Cherry Flavoured Nommer of Fishies
    • View Profile
Re: Athena: Calculator project installer
« Reply #44 on: December 19, 2010, 05:50:58 pm »
Well, I've been chatting it up with the (mostly) good people over at the Cemetech IRC channel, and I've figured from trying to get FFTOM2 to pack, that it does not fully support group files (.8xg , which are NOT the same as .tig). It seems to only grab the first file in the group (since these files are formatted in such a way that makes it extremely easy to mistake it as some other type of file), so I figure I should get that taken care of before I bring the ASM program out of test mode and into an actual install mode.

Once that's taken care of, Athena will separate and package the contents of a group rather than the group itself. I believe some minor changes are needed in the code, not a total overhaul. This is a good thing. I suppose.

I have other plans for use by advanced users in case they want to make the most of out Athena, including executing flagged ASM programs during an install to perform other functions that Athena cannot do by itself.

So all in all, this project is coming around nicely. If it's possible, I might have something rather basic sometime tonight.
« Last Edit: December 19, 2010, 05:51:50 pm by Iambian »
A Cherry-Flavored Iambian draws near... what do you do? ...