Omnimaga

Calculator Community => Other Calc-Related Projects and Ideas => TI Z80 => Topic started by: Iambian on December 08, 2010, 04:16:49 pm

Title: Athena: Calculator project installer
Post by: Iambian on December 08, 2010, 04:16:49 pm
Originally started on Cemetech (since there's a load of C coders there who are pretty willing to help), but I'm bringing the idea here too.

As stated in the title, "Athena" is a project to help simplify the installation of large BASIC games. It started when I was reading some thread on Cemetech and DJ Omnimaga mentioned something about me building a RoL3 installer in TI-Basic/CIII. So I figured I would try my hand at building something better. Something that could possibly be used by just about anyone. The project is targeted for the TI-83+SE/TI-84+(SE) users, since they've got crazy-insane archive space for which this is possible.

For users who will be installing these games, you will only need prgmATHENA and some number of data files that are distributed. These data files act as compressed group files. Once you get all that stuff to your graphing calculator (at least read the ReadME.txt that the developer will most likely have put in), run Asm(prgmATHENA) and it will guide you through the installation process. Once it starts, no further user interaction will be needed unless you have to do stuff like garbage collect. No longer do you have to sit in front of a computer and have to send each and every individual file and possibly have to manually archive them. No longer will you have to send group files just to extract then archive the variables. Athena will perform much of this management for you!

If you accidentally delete a few of the files used by the game and if you still have both prgmATHENA and its data files, you may repair the installation by running prgmATHENA again.

Planned is the ability to install files to work with DCS. Which is to say, it will be folder-aware and let you install files in any folder of your choosing (or create your own right then and there if you don't have the folder).

Since the data files are compressed, you'll be able to have multiple large BASIC games on your calculator at the same time. Just uninstall one and install the other if you really want to.

More information will be made available for developers interested in using Athena for their projects. So... tell me what you think.
Title: Re: Athena: Calculator project installer
Post by: Munchor on December 08, 2010, 04:18:36 pm
Originally started on Cemetech (since there's a load of C coders there who are pretty willing to help), but I'm bringing the idea here too.

As stated in the title, "Athena" is a project to help simplify the installation of large BASIC games. It started when I was reading some thread on Cemetech and DJ Omnimaga mentioned something about me building a RoL3 installer in TI-Basic/CIII. So I figured I would try my hand at building something better. Something that could possibly be used by just about anyone. The project is targeted for the TI-83+SE/TI-84+(SE) users, since they've got crazy-insane archive space for which this is possible.

For users who will be installing these games, you will only need prgmATHENA and some number of data files that are distributed. These data files act as compressed group files. Once you get all that stuff to your graphing calculator (at least read the ReadME.txt that the developer will most likely have put in), run Asm(prgmATHENA) and it will guide you through the installation process. Once it starts, no further user interaction will be needed unless you have to do stuff like garbage collect. No longer do you have to sit in front of a computer and have to send each and every individual file and possibly have to manually archive them. No longer will you have to send group files just to extract then archive the variables. Athena will perform much of this management for you!

If you accidentally delete a few of the files used by the game and if you still have both prgmATHENA and its data files, you may repair the installation by running prgmATHENA again.

Planned is the ability to install files to work with DCS. Which is to say, it will be folder-aware and let you install files in any folder of your choosing (or create your own right then and there if you don't have the folder).

Since the data files are compressed, you'll be able to have multiple large BASIC games on your calculator at the same time. Just uninstall one and install the other if you really want to.

More information will be made available for developers interested in using Athena for their projects. So... tell me what you think.

It's a kind of installer and uninstaller program?

Looks nice :)
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 08, 2010, 05:05:40 pm
Yup. People can build archives using this project and then distribute them as their game. It won't take up a whole lot of space since each archive is compressed. The only real limitation is the memory on the calculator. What could end up happening is that some large BASIC game that takes up some 130KB of memory can be archived in like 60KB or something like that.

The limitation with respect to the individual archives is that each archive can decompress up to 64KB. The archive itself may be quite small, but each one will expand to up to 64KB, which means that I can't have some huge as hell single archive, each 64KB (yes, they *will* send to the calc if they were that large. You just can't unarchive them).
Title: Re: Athena: Calculator project installer
Post by: AngelFish on December 08, 2010, 05:08:08 pm
Does it check to make sure there's enough space to decompress?
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 08, 2010, 05:10:46 pm
Does it check to make sure there's enough space to decompress?
In short, yes. Your RAM has got to be pretty much empty, though. There are checks in the data stream that tells the installer a rough estimate as to how much space the install will take, and then compare that to the calc's memory.

I intend on modifying the pucrunch decompressor so that it'll extract on a file basis. It'll then move stuff to FlashROM as necessary. As such, all files that go into FlashROM will be extracted first, then files that need to reside in RAM will go there.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 08, 2010, 10:07:45 pm
Got in some early results. While I haven't coded everything yet, I am able to get .bin files that contain all the compressed data. I tried to pack Illusiat 13, which weighs in at over 130KB. The end result? Three files that collectively take up around 42KB. (12KB, 25KB, 5KB)

Now, to verify that it's all good and start writing the .8xp packager into the C program. After that, the Z80 part of Athena can start up.
Title: Re: Athena: Calculator project installer
Post by: Builderboy on December 08, 2010, 10:17:38 pm
Wow thats some impressive stats right there O.O Why does it compress into 3 files instead of just one large 42K out of curiosity? 
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 08, 2010, 11:24:46 pm
This is awesome! Sure, maybe there's a steep size requirement if you decompress/install a massive game like Metroid II, but a lot of people complained about those games being hard to install back then, due to being in multiple steps, so this might come handy.

Will this support hacked pictures? This is one major issue when installing Zelda: Dark Link Quest and to a lesser extent Reuben Quest: The Lost Mirror.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 08, 2010, 11:41:42 pm
Wow thats some impressive stats right there O.O Why does it compress into 3 files instead of just one large 42K out of curiosity? 
The version of pucrunch that I'm using can only compress up to 64KB at a time. Since I13 is around 130KB and each file is reasonably sized, so it'll create three archives.

As far as hacked pics are concerned, if you can get them to the computer, it'll pack just fine. If it really becomes an issue (because I hear that hacked pics can only sent to the compy by group), I might add support for automatic group extraction so it can be included into the project. Maybe. I just want something that works at the moment.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 08, 2010, 11:43:27 pm
Oh right, I forgot about getting them on the comp. They have to be grouped. Alternatively maybe you could maybe have an on-calc hacked pic manager for use with your installer? The person could choose which hacked pic package to import in his project on the computer.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on December 10, 2010, 10:21:47 am
Just FYI, you can un-group hacked pics once they are on the PC with the WinRAR utility (if you were thinking that that's an issue). I'm not sure how the file names would show up on PC though.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 10, 2010, 11:37:50 am
Just FYI, you can un-group hacked pics once they are on the PC with the WinRAR utility (if you were thinking that that's an issue). I'm not sure how the file names would show up on PC though.
If it can do that, you should probably be able to rename them to whatever you want. The variable's name (when it is sent to the calc) is actually dependent on some name field somewhere near the start of the file.

On another note, I've got Athena generating valid .8xp files now, so I guess I can start working on the Z80 side of things pretty soon.
Title: Re: Athena: Calculator project installer
Post by: meishe91 on December 10, 2010, 09:48:15 pm
This sounds really cool, good luck on it :) Can't wait to see how this turns out.

Just out of curiosity, isn't pucrunch a compression method?
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 10, 2010, 10:06:21 pm
This sounds really cool, good luck on it :) Can't wait to see how this turns out.

Just out of curiosity, isn't pucrunch a compression method?
Thanks for the support!

Oh, and pucrunch is a compressor, not a method. pucrunch uses some LZ77/RLE hybrid, as explained in (way too much) detail here: http://www.cs.tut.fi/~albert/Dev/pucrunch/

If you can fully understand that, then you'll be all ready to implement it in Axe :)
Title: Re: Athena: Calculator project installer
Post by: meishe91 on December 10, 2010, 10:29:56 pm
Ah ok, I just remember seeing it somewhere before while compression was talked about so that's why I asked. But ah ok, thanks :) It'll be a long time before I incorporate any of that into anything I do though :P
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 10, 2010, 10:42:03 pm
Ah ok, I just remember seeing it somewhere before while compression was talked about so that's why I asked. But ah ok, thanks :) It'll be a long time before I incorporate any of that into anything I do though :P
It's not that big of a deal, though. They've already got a ready-made decompressor for the purpose of sticking it in your (Z80 ASM) program so you can use it to mess with chunks of data fitting of some really huge FlashAPP.

On a note related to the project, I have just enough to start working on the Z80 ASM side of things.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 11, 2010, 01:19:35 am
Just FYI, you can un-group hacked pics once they are on the PC with the WinRAR utility (if you were thinking that that's an issue). I'm not sure how the file names would show up on PC though.
If it can do that, you should probably be able to rename them to whatever you want. The variable's name (when it is sent to the calc) is actually dependent on some name field somewhere near the start of the file.

On another note, I've got Athena generating valid .8xp files now, so I guess I can start working on the Z80 side of things pretty soon.
It doesn't work on Windows XP or below, only on Vista and 7. On older OSes, some chars are not supported and the files won't ungroup.

Also these files can't be renamed while inside groups, it seems. (at least, for me), and even if I managed to do so, I noticed that when you rename a pic file, it seems WabbitEmu grabs the new windows file name, not the file name inside the pic. I noticed this after renaming _PRGMNAM.8xp to APRGMNAM.8xp by just renaming the file under Windows. At first, the file on Wabbitemu would be _PRGMNAM in the PRGM menu and wouldn't be executable, then it was APRGMNAM. It might be because my version of Wabbitemu isn't very up to date, though.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on December 11, 2010, 08:28:55 am
@Omni: That's good to know, I've only used WinRAR to un-group group files on Windows 7. I've never tried it on XP or below. I've also never fooled with Hacked pics either so I was unsure how those would be handled by the PC. I'd also imagine that the file would need to be un-grouped first before attempting any name changes.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 12, 2010, 04:35:15 am
To ungroup 8xg files with WinRAR do you just rename the extension ot .rar then extract the files? I wonder if the format is really that similar?

EDIT: Weird, could you clarify? I renamed a 8xg file to .rar and .zip and in both cases I got a corrupted/damager archive error on Windows 7 ???
Title: Re: Athena: Calculator project installer
Post by: JosJuice on December 12, 2010, 04:44:16 am
To ungroup 8xg files with WinRAR do you just rename the extension ot .rar then extract the files? I wonder if the format is really that similar?
The file formats are completely different. Maybe there's some kind of plug-in for zipping programs?
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 12, 2010, 05:33:51 am
I'm not sure. I would be surprised if WinRAR supported calculator formats natively, though. O.O
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on December 13, 2010, 10:08:17 am
I guess the group format uses a similar procedure to files which winrar supports natively. No renaming is necessary. Just right click the group file and choose extract all. Bam- you have a folder with the group name with all the files contained therein. =)
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 13, 2010, 10:40:14 pm
Oh, this isn't done by WinRAR, this is done by TI-Connect, which adds an extract all option in right-click.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on December 13, 2010, 10:51:27 pm
Oh, heh...  :-[
I knew that... I was just testing you.... yea... that's it!/me seems to have forgotten that TI Connect did that...
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 13, 2010, 11:00:41 pm
Lol :P

Also, double-clicking the group file opens its content in a Windows 98 looking way. It doesn't work in Windows 7 and can crash, though.
Title: Re: Athena: Calculator project installer
Post by: JosJuice on December 14, 2010, 09:26:20 am
Also, double-clicking the group file opens its content in a Windows 98 looking way. It doesn't work in Windows 7 and can crash, though.
For me, it doesn't work if the group file is in a zip file.
Title: Re: Athena: Calculator project installer
Post by: Builderboy on December 14, 2010, 02:11:05 pm
For me, windows is convinced 8xp are text files and will refuse to treat them as anything else x.x
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 14, 2010, 09:09:44 pm
Still reworking the packer part to Athena. Got a newer idea of generating these files. The initial file will instead have its metadata and data sections split so the metadata will be in its own file.

The data files are going to have names that are generated. The first character will be a theta, the next three are hex digits generated from the project name, and the last four are generated from the checksum of each file. In the event of a collision, the last four digits will just be incremented until there's no longer a problem.

EDIT: As for the windows problem, just look for the .8xp entry in the registry and delete that. It should solve your problem. Make sure you delete *all* entries, too. Some may be more obscure than others, and there may be more than one that's trying to associate your .8xp files with notepad.
Title: Re: Athena: Calculator project installer
Post by: Eeems on December 14, 2010, 10:56:20 pm
For me, windows is convinced 8xp are text files and will refuse to treat them as anything else x.x
easy fix, just go to the control panel and change file associations

Iambian, I'm really looking forward to this :)
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 14, 2010, 10:58:48 pm
For me, windows is convinced 8xp are text files and will refuse to treat them as anything else x.x
easy fix, just go to the control panel and change file associations

Iambian, I'm really looking forward to this :)
This doesn't always work I think. I accidentally made 8xk files be openable by Notepad once and I never could find the original TI-Connect app icon again in control panel afterward, so now there's no send to TI device option anymore, I have to double-click the APP. Reinstalling TI-Connect doesn't do the trick. Once you accidentally change TI-Connect file associations, it's over.
Title: Re: Athena: Calculator project installer
Post by: Eeems on December 15, 2010, 12:01:05 am
Hmm, strange...so Control Panel\Programs\Default Programs\Set Associations doesn't work? I would think it would...maybe I'm just crazy
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 15, 2010, 12:07:52 am
Nope it doesn't work. It never could set it again to the same application and icon it did before.
Title: Re: Athena: Calculator project installer
Post by: Eeems on December 15, 2010, 12:22:07 am
Hmm, strange...I would have thought that would work. I guess the only way would be to do it the way Iambian said.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 15, 2010, 02:01:35 am
I believe I have the Athena packer generating files correctly, so I guess it's largely a success?
The only difference is that the data files start with a "Z" instead of a theta. Doesn't make that much of a difference, since you can pack to appvars instead of programs.

For anyone interested, here's some documentation about the files themselves.
Code: [Select]
-------------------------------------------------------------------------------
Documentation regarding the package file format for the data
portion of the TI-83/84+(SE) installer software. All instances of multibyte
numeric variables are stored little-endian.

Installations using the Athena packager/installer will contain at least two
files in addition to the installer: (1) Metadate file (2) Data files.

Metadata files contain information about the install. This includes, in no
particular order, the number of data files there are, their filenames, the
number of files that are archived inside the package, the estimated amount of
FlashROM the install will use, the amount of RAM it will use, and a slightly
longer (14 character) description field.

Metadata file structure:

Ofs Siz Desc
+00 08: header. $BB,$6D,$C9,$31,$80,$00,$01,$09
+08 01: number of installer files (64KB chunks)
+09 02: number of files to be installed (can be quite a few!)
+11 03: number of bytes of FlashROM the installation will take up
+14 02: number of bytes of RAM the installation will take up
+16 01: Install type flag (not used yet)
+17 14: 14 bytes ASCII text project project name
+31 01: RESERVED
+32 --: Names of installer files (all 8 char long). This is sized
        as number of installer files times eight.

The data files that come after the metadata file contains a number of streams
formatted in the following way:

+00    1b: filetype + flags. Matches TI-OS file types, except bit 7: 0=In RAM
+01    2b: checksum. Used for comparing installed files with this one.
+03    2b: file size (includes possible filesize field that comes with file)
+05    1b: filename length
+06    nb: filename
+06+n    : data stream

The number of these streams will not be known until EOF is encountered. The
amount of data in this file can be up to about 65500 bytes. None of the data
streams are split for the purpose of extracting data from the file. These data
streams are pucrunch-compressed as a solid archive, so it will be up to the
reader to decompress only bytes at a time and then read the information for
its formatting and its EOF.

The name of these data files is derived from the name of the project and the
uncompressed contents of the file. Every file name will be of the form:

Zfffnnnn
Z is just the letter Z
fff are 3 characters from 0-9 A-V. Hashed from name, so will remain constant.
nnnn are 4 characters from 0-9 A-V, hashed from data, so will likey vary.

Examples: These are true examples from an experiment done on packaging
Keven Oullet's (DJ Omnimaga) somewhat incomplete Illusiat 13.

On entering into the command line:
athena -a I13 "Illusiat 13"

I get the following files out:
I13.8xp          Size (bytes) : 132
ZNG0H7DI.8xv     Size (bytes) : 24716
ZNG0LK35.8xv     Size (bytes) : 4846
ZNGOLRSF.8xv     Size (bytes) : 11936

The input was sized collectively at 138327 bytes, so this is also a great
compression ratio. Understand that the actual filesizes on-calc are going to
be a slight bit smaller since calculator files on the computer contain
additional information that is useful for the computer to calculator linking
program.

-------------------------------------------------------------------------------
Documentation regarding the creation of DCS folders

Not created yet.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 15, 2010, 04:16:14 am
Wow this compressed A LOT O.O
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 16, 2010, 04:34:18 pm
Spent all day yesterday working on the Z80 ASM side of Athena. It will now correctly detect metadata files and their associated data files, and I can have the program print out statistics written directly into the metadata file. Attached to this post is an image of what I'm seeing in the statistics view.

What I've got left to do is to edit the decompression routine to get it working for Athena, then start writing out the data that's in there. I can imagine that the hardest parts would be getting the decompression routine to work for me, and creating the appropriate files from the data that is read out. After that, it's rather trivial to print the progress to the screen and archive the files as necessary.

After looking for bugs, I'll then put this thing up for people to test. DJ Omnimaga's probably gonna be the first to get it because he's got the resources and the knowledge to fully test this Athena. I pretty much need all the large BASIC games I can find so this thing can be tested out.
Title: Re: Athena: Calculator project installer
Post by: thepenguin77 on December 16, 2010, 06:34:18 pm
For me, windows is convinced 8xp are text files and will refuse to treat them as anything else x.x
easy fix, just go to the control panel and change file associations

Iambian, I'm really looking forward to this :)
This doesn't always work I think. I accidentally made 8xk files be openable by Notepad once and I never could find the original TI-Connect app icon again in control panel afterward, so now there's no send to TI device option anymore, I have to double-click the APP. Reinstalling TI-Connect doesn't do the trick. Once you accidentally change TI-Connect file associations, it's over.

I did the exact same thing. The problem is that you want the default file extension which is not strictly attached to a program. Use this (http://www.vistax64.com/tutorials/91920-unassociate-file-extention-type.html). Worked like a charm.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 16, 2010, 11:44:12 pm
Ok thanks I'll have to try this.

Iambian nice! I don't think I'll have time to test every program I made, though, but I could maybe try some of the largest games like Zelda DLQ and Illusiat 13.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 18, 2010, 11:19:08 am
Well... I sorta ran into a snag. It relates to the compression and the limits of the edits I can make with the decompressor. Basically, I'm going to have to opt for weaker compression to make the edits, so projects won't get the best possible results when using pucrunch.

I'm sorry, but I knew these great results were a little ... too promising. It just won't work for what I've gotta do with these files. (pucrunch expects all decompressed data to remain where it is in one contiguous block. I cannot have it that way)

Still... for Illusiat 13, it goes from 135KB to 55.4KB instead of the much rosier 40KB. The results will most certainly vary depending on how far apart the LZ matches are. For those who know anything about the LZ77 algorithm ( I have marginal knowledge :P ), I had to restrict the search range to 255. I know this isn't very optimal, but pucrunch does a good job in making it work well. There's still decent compression.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 18, 2010, 06:35:33 pm
Sorry to hear. That said, the new compression still seems excellent to me. 135 KB to 55.4 KB is pretty great. Nice job, still. :D
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 18, 2010, 10:09:56 pm
Just to make it clear, I'm still using pucrunch. It's just that I've had to set an option in the compiler to force it to restrict its search range. Thankfully, I can do that. At the moment, the edits are already made in the ASM program, though I will still need to test it all out to see if it works out right. I'm hoping it will, but who knows?
Spoiler For the curious:
This is the line of code used in Athena to run pucrunch:
pucrunch -d -c0 -r255 Athout_i.bin Athout_c.bin
Where the underscore characters are filled in with some character 0-9,A-Z
If you really want to know, this is the function I wrote to do the job. Some of the casts just aren't necessary, but still. I'm a newb when it comes to the C thing.

const char *alphanums = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
const char *tiheader  = "**TI83F*\x1A\x0A\0Built using the Athena Packager Software  "; //53 bytes
const char *tifileheader = "**TI83F*\x1A\x0A\0";
const char *athenaheader = "\xBB\x6D\xC9\x31\x80\x00\x01\x09";
int flushtofile(char *inputstream,int appendfilenum,int outfilelength)
{
    char tempfilename[]="Athout_i.bin\0";
    tempfilename[6] = alphanums[appendfilenum];
    FILE *newfileptr;
    newfileptr = fopen(tempfilename,"wb");
    if (newfileptr == NULL) return 1;
    if (outfilelength!=fwrite(inputstream,1,outfilelength,newfileptr)) {
        printf("Err: Could not flush buffer to file!");
        (void) fclose(newfileptr);
        return 1;
    }
    (void) fclose(newfileptr);
    if (!system(NULL)) return 1;
    char tempbatch[]="pucrunch -d -c0 -r255 Athout_i.bin Athout_c.bin\0";
    tempbatch[28]= alphanums[appendfilenum];
    tempbatch[41]= alphanums[appendfilenum];
    system(tempbatch);      //pucrunch will always output some sort of error. Newp.
    remove(tempfilename);  //assumes it can be removed. We just created and closed it...
    return 0;
}

Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 18, 2010, 10:25:39 pm
I see. Do you think there will be a release, soon? (beta-test?)
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 18, 2010, 10:34:04 pm
I see. Do you think there will be a release, soon? (beta-test?)
Really depends on how reliable the modified pucrunch decompressor is. If it passes with flying colors, I can have something out by this Monday or Tuesday. If not I'm not really sure.

The initial test will be decompressing blocks of 256 at $8000 and examining it against the same file on the computer opened up under a hex editor.

If that passes, I'll start with actually decompressing the files and comparing the results against the built-in checksums.

If *that* passes, I'll start writing in functions and start taking care of the various file formats that one might possibly throw into Athena. I'll be sure to exclude some formats (like window settings), but I should be able to cover the majority of the stuff that most projects contain.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 18, 2010, 11:24:06 pm
Ah ok. Hopefully things work out well.
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 19, 2010, 05:50:58 pm
Well, I've been chatting it up with the (mostly) good people over at the Cemetech IRC channel, and I've figured from trying to get FFTOM2 to pack, that it does not fully support group files (.8xg , which are NOT the same as .tig). It seems to only grab the first file in the group (since these files are formatted in such a way that makes it extremely easy to mistake it as some other type of file), so I figure I should get that taken care of before I bring the ASM program out of test mode and into an actual install mode.

Once that's taken care of, Athena will separate and package the contents of a group rather than the group itself. I believe some minor changes are needed in the code, not a total overhaul. This is a good thing. I suppose.

I have other plans for use by advanced users in case they want to make the most of out Athena, including executing flagged ASM programs during an install to perform other functions that Athena cannot do by itself.

So all in all, this project is coming around nicely. If it's possible, I might have something rather basic sometime tonight.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 19, 2010, 06:41:13 pm
Oh I thought tig was the same? Since both opens with Extract all...

And grabbing only the first file of a group seems familiar. When ungrouping files in early versions of WabbitEmu or Flash Debugger, that happened as well. Seems like it's hard to deal with group files. X.x
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 19, 2010, 07:19:19 pm
It's not *that* hard. The one thing in the file that you're supposed to look at (which isn't directly stated in the linkguide) is that the data section size covers the entire data section, which can contain multiple files. Most people won't look past the first file since they assume that this size is equal to the header for the packed file plus its data. When you've gone past the first file in the group, your count doesn't match up with the aforementioned size. When this happens, there's more to be read. I ought to look forward for further contents, which are the other files in the group.

Most of the .8xx files are formatted this way, but the vast majority of them just have one entry. Which is to say, if you have something that'll read grouped files, it'll also read any other file, be it a .8xv, .8xp, or whatever in a very consistent way. In fact, it may not be necessary to know the extension if one just read the contents of the file, since filetypes and all other relevant data is kept within.

And in keeping with what you said, that revelation clicked for me once it started happening to Athena as well. I feel like I understand the file format a bit more now.

Oh, and I really should get back to work on Athena. Been playing Stepmania since I last posted or something like that and ... yeah. Too many breaks :P
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 19, 2010, 10:19:56 pm
Lol ok, well good luck with the release. ;D
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 21, 2010, 01:23:46 am
It goes well. A little more work putting in extra features and ironing things out, and it should be golden.
At the moment, you can test and install archives. For this exercise, I'm using Illusiat 13 as the example. See screenshot below.
Title: Re: Athena: Calculator project installer
Post by: calcdude84se on December 21, 2010, 10:51:12 am
Looks nice! :D Two questions: What's the 40803 when installing for, and can TI-Connect deal with those programs over 32KiB in size?
Title: Re: Athena: Calculator project installer
Post by: Munchor on December 21, 2010, 10:53:49 am
It goes well. A little more work putting in extra features and ironing things out, and it should be golden.
At the moment, you can test and install archives. For this exercise, I'm using Illusiat 13 as the example. See screenshot below.

Looks very good :)

I ask myself the same questions as calcdude84se, though...
Title: Re: Athena: Calculator project installer
Post by: ztrumpet on December 21, 2010, 11:04:28 am
Looks great! ;D  Nice job Iambian. :D
Title: Re: Athena: Calculator project installer
Post by: aeTIos on December 21, 2010, 11:05:47 am
Wow really nice! hold on!
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 21, 2010, 12:26:02 pm
Looks nice! :D Two questions: What's the 40803 when installing for, and can TI-Connect deal with those programs over 32KiB in size?
The 40803 is a bug. A really stupid one, but one that doesn't mess up anything. It's actually supposed to display the current file, but I guess I ought to change it anyway. Needs to say something like "File x of y"

And I believe it should. I did manage to get 60KB files to my (real) calc, but I don't quite remember what I used to do it. It was either TI-Connect or TiLP.

I've also gotta work on, of all things, the documentation. Athena won't do anyone any good if no one knows how to use it.
Title: Re: Athena: Calculator project installer
Post by: Builderboy on December 21, 2010, 01:40:42 pm
Wow this is looking great O.O The compression you are getting is pretty spectacular, awesome job! ;D
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 21, 2010, 02:52:29 pm
Awesome Iambian! This shall make installing large games and keeping them all on the same calc much easier now. ;D
Title: Re: Athena: Calculator project installer
Post by: Eeems on December 21, 2010, 02:55:50 pm
looking great Iambian :D can't wait to use this!
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 21, 2010, 07:43:06 pm
The Z80 Athena seems to be mostly done now. The only thing that doesn't really work is the "Repair" command, but then again, you can always uninstall and re-install to achieve the same effect.

The PC Athena still needs to get some group support, but once that's done, I'll be releasing Athena for testing. It shouldn't be much longer now.
Title: Re: Athena: Calculator project installer
Post by: Munchor on December 21, 2010, 07:45:31 pm
The Z80 Athena seems to be mostly done now. The only thing that doesn't really work is the "Repair" command, but then again, you can always uninstall and re-install to achieve the same effect.

The PC Athena still needs to get some group support, but once that's done, I'll be releasing Athena for testing. It shouldn't be much longer now.

That reminds me of Window's uninstalling/reparing programs program. What about program compatibility? Any program(s) that don't work? Or all of them do?
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 21, 2010, 08:59:16 pm
[...]
That reminds me of Window's uninstalling/reparing programs program. What about program compatibility? Any program(s) that don't work? Or all of them do?
They all should, since they're mostly hybrid BASIC programs that are being packaged here. Thing is, it won't install FlashAPPs, and according to some sources, installing them from file can be a major pain. Not worth it unless someone could write up some example code. I recall that the way Axe does it is rather hackish, so I wouldn't want to bloat the installer any more than it is unless I can figure out a way to keep its RAM footprint low.

On another note, adding in group support wasn't /that/ painful. I got RoL3 to package and install. It just takes forever to install due to the sheer number of programs there are. The screenshot I'll enclose will show a memory view of the archives and the testing portion, but not the actual install. No one has ten minutes to watch a screenshot.

When I get the whole thing put together, I'll be releasing the packer and installer.

EDIT: Also attached is a copy of Reign of Legends 3 (Thanks DJ). It is an example of what my program is capable of. Just run Athena as an ASM program. Like Asm(prgmATHENA) and then follow along. Controls are easy to figure out. Just try it.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 22, 2010, 02:20:05 am
Nice Iambian! I'll really have to try this! It will really make installing those games easier. I think there will be another feature on ticalc.org soon since it will make most large games with many files much easier to install. :)

What's with the weird files such as 'v, though? O.O
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 22, 2010, 02:24:55 am
Those strange files are things that don't display right. In the case of RoL3, those are your pic files. May want to fix it eventually, but I think it's fine like it is.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 22, 2010, 02:29:42 am
Ah ok, well this should be fine I guess if they work fine. :D
Title: Re: Athena: Calculator project installer
Post by: FinaleTI on December 22, 2010, 06:29:20 am
This is pretty cool and it works fine in Wabbit, but when I try to send an archive to my physical calculator, it says the file is corrupt or invalid. I'm using TI-Connect, btw.

Edit: Exporting them from Wabbit allows them to be sent, though... The Axe MIDI Converter had a similiar problem. Maybe something with the checksum? I dunno.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on December 22, 2010, 07:28:15 am
This is awesome Iambian! I can't believe how quickly this has come together! =)
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 22, 2010, 10:22:16 pm
This is pretty cool and it works fine in Wabbit, but when I try to send an archive to my physical calculator, it says the file is corrupt or invalid. I'm using TI-Connect, btw.

Edit: Exporting them from Wabbit allows them to be sent, though... The Axe MIDI Converter had a similiar problem. Maybe something with the checksum? I dunno.
If possible, would you attach both files, the one that won't send and the one that will? Much appreciated, since something like this can help me come to a solution much sooner.

EDIT: Scratch that. Replace the file you're using with the one attached below.
EDIT2: Found another error. Replacing TIPACK01 with TIPACK02
Title: Re: Athena: Calculator project installer
Post by: Munchor on December 23, 2010, 04:47:12 pm
Just saw this got added to ticalc.org archives, great!
Title: Re: Athena: Calculator project installer
Post by: Iambian on December 23, 2010, 08:07:36 pm
Still finding bugs with the installer. FFTOM2 is a great project to test this thing on, and it's revealing that Athena won't handle lists properly. I figure I'll get that thing to install first and then I'll update the release on ticalc.org.

EDIT: It actually writes the list out now (test version) but something is destabilizing Athena. Not sure what yet.

EDIT2: It seems CreateVar doesn't like making lists. Will have to use the romcall specific for that, then.

EDIT3: Now the game won't run. "This is quite a conundrum."
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on December 26, 2010, 02:49:55 am
Sorry to hear about the problems. I hope eventually you can figure them out.
Title: Re: Athena: Calculator project installer
Post by: Iambian on February 02, 2013, 01:33:16 pm
Necropost.

After just over two years, I've decided to revisit this little project. The major change is the packager interface, which is now much more user-friendly. This time around, instead of having to mess around with file folders and batch files, all you have to do is drag and drop into the window and click the packager button to make your files.

This has already been done and the application can now generate data files that can be read by the original Athena installer. I haven't released this version yet because I'm still working on overhauling the installer so it has a far smaller memory footprint during installation (ideally, zero. Yes, it's possible). Along with this, I want to implement the ability to make Athena packages self-extracting.

That's going to take some doing.

EDIT: Added screenshots of the packager window. Win7 makes them look prettier than they should be. Illusiat 13 is being used here.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on February 02, 2013, 01:36:53 pm
Glad to see it's still worked on. I was also wondering if you would ever cross-post about this here lol :P. I need to give this a try on Zelda DLQ when finished.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on February 06, 2013, 06:02:23 pm
Very nice Iambian. You should post some progress on some of those other projects of yours. :P *nudge nudge* *wink wink*
Title: Re: Athena: Calculator project installer
Post by: Iambian on February 19, 2013, 04:04:00 pm
A semi-crosspost from Cemetech.

This project is nearing a releasable form. I've given a pre-release copy to geekboy, and posted a quick link to the download in both the Cemetech and Omnimaga IRC channels for quick testing, and I've found one thing out.

The SWT libraries that drive the interface will not work in a 64-bit JVM. I can think of a couple of solutions for this, but what would be the best one? I could...
(1) Package the 64-bit SWT libraries and have the app choose which to load at run-time
(2) Build two packages, one 32-bit and the other 64-bit.
(3) Tell the user to download a 32-bit JVM if they're using a 64-bit machine.

Other possible bugs weren't found, since the only tester to report back told me he couldn't run the program on his 64-bit machine.

So far, the visible changes made to the packager since the last time I posted includes checkboxes next to the names of each file that lets you exclude files for packing. While that feature is next to useless if you have nothing but individual files, this is almost essential if you're putting in group files but need to exclude certain parts of it. (No, I will not add support for .tig files. You can use WinZip or your favorite .zip program for unpack those)

I've also finished up a working version of the Athena self-extracting installer. It works in a test case where I have the metadata file in the build, but now I need to get it so the packager can apply stub needed to make the output files self-extracting.

----
The .jar file is 1.92MB large, with less than 40KB of it being the code I wrote. Everything else is dependencies. I find this disgusting, but what can I do other than not use Java?

If you can think of any related utilities that might go well with this project, please suggest them.
Title: Re: Athena: Calculator project installer
Post by: TIfanx1999 on February 19, 2013, 05:37:28 pm
I think options 1 and 2 are both very good choices. 1 probably being the best as it requires the least from the user.
Title: Re: Athena: Calculator project installer
Post by: DJ Omnimaga on February 21, 2013, 12:51:52 am
1 might be best. It would be less confusing I think and 3 might be annoying if it requires the user to constantly switch between a 32 bits and 64 bit JVM (if he runs a 64-bit OS and that some programs only run in a 64-bit JVM.