### Author Topic: The .hpprgm format  (Read 6662 times)

0 Members and 1 Guest are viewing this topic.

#### Hasse

• LV0 Newcomer (Next: 5)
• Posts: 3
• Rating: +0/-0
##### The .hpprgm format
« on: August 05, 2014, 04:41:36 am »
I spent some time reverse engineering the .hpprgm format (the format of the user programs of the Prime).
0x0000-0x0003:
size of the header, excludes itself
(so the next header begins at size+4)
0x0004-0x0005:
Amount of variables in table
0x0006-0x0007:
Amount of something? (Perhaps views or something?)
0x0008-0x0009:
Amount of exported functions in table
0x000A-0x000F:
unneeded?
Conn. kit generates
7F 01 00 00 00 00
but all zeros seems to work too.
0x0010-0x----:
Exported item table.
Entry format is as follows:
Type of item:
30 00 for variable,
31 00 for exported function
Name of item:
UTF-16, until 00 00 00 00
Then the next entry follows.

VARIABLE VALUES:
(There are as many blocks of this type as you have exported variables and the
blocks are in the same order as the exported variables)
0x0000-0x0003:
size of the value, excludes itself
0x0004-0x0005:
01 00 for detecting that this is a list
02 00 for single value entry
IF single value entry:
0x0006-0x0007:
type:
10 01 for base-10 integer or float
11 20 for base-16 integer
12 02 for string
IF base-10 integer or float:
0x0008-0x000B:
exponent
signed little endian 32-bit integer
0x000C-0x0013:
mantissa
little endian weird stuff. Hexadecimal to be interpreted as decimal... WTF?
00 00 00 00 00 00 00 25 01 is supposed to be 1.25 in decimal,
00 00 00 00 00 00 00 28 06 -> 6.28 and so on

The value is mantissa*10^exponent
IF base-16 integer:
0x0008-0x000B:
02 00 00 00 (why?)
0x000C-0x0013
55 63 62 00 00 00 00 00 becomes #626355
25 06 00 00 00 00 00 00 becomes #625
IF string:
0x0008-0x0009:
Length of string in characters, excludes the tailing 00 00
0x000A-:
string itself, ends in 00 00
IF list:
0x0006-0x0007:
(Tends to be 16 00, 16 01 or 16 02)
0x0008-0x0009:
32-bit LE
Amount of members in list, let's call this N
0x000A-0x000B:
(Probably reserved for something, 7F 01 or 00 00)
N 4-byte values:

Actual list of values follows, they are in reverse order compared to what
they are in the source.
An entry in the list follows this formula:
Stuff gets clever and recursive, every entry in here is handled like a
"VARIABLE VALUE" itself minus the size integer at the beginning

0x0000-0x0003:
size of the header, excludes itself
0x0004-:
Code in UTF-16 LE until 00 00

Sorry if this is common knowledge already, I did try to search the forums (no results for hpprgm).  I did find this: http://tiplanet.org/hpwiki/HP_Prime/File_Format#User_BASIC_programs, but it seems that the information there is true only for some very simpe cases. I have no wiki editing skills so I can't update that.

I hope someone else find this useful and that someone continues the reverse engineering of the format (if that stuff truly is bytecode, it could be very useful). My own motivation is has dropped due to other, more interesting projects unrelated to calculators, which is why I stopped working on this.

EDIT: updated, that stuff is not bytecode, they are the encoded objects like Tim Wessman said.
« Last Edit: August 06, 2014, 06:34:44 am by Hasse »

• Editor
• LV10 31337 u53r (Next: 2000)
• Posts: 1708
• Rating: +229/-17
##### Re: The .hpprgm format
« Reply #1 on: August 05, 2014, 07:24:05 am »
Hi,

Thanks for the info - the link you gave has some info about the program format as seen by [MOHPC] member Eried - maybe it was only on simple programs, indeed, I don't know. However, it may be enough for file transfers. One would have to check with complex programs, though, with his program (see https://github.com/eried/PrimeComm/blob/master/PrimeLib/PrimeUsbFile.cs )

If you still have a tiny bit of motivation, you're welcome to register on the wiki you linked and try editing on the sandbox for example (you can try anything there, it won't matter).

If you decide to do that and edit/contribute to the file format page, let us know (just reply here telling so), otherwise we can wait for a bit (maybe people will have some suggestions) and then edit it ourselves from what you've posted here
« Last Edit: August 05, 2014, 07:35:51 am by Adriweb »
My calculator programs
TI-Nspire Lua programming : Tutorials  |  API Documentation

#### DJ Omnimaga

• Now active at https://codewalr.us
• CoT Emeritus
• LV15 Omnimagician (Next: --)
• Posts: 55821
• Rating: +3151/-232
• CodeWalrus founder & retired Omnimaga founder
##### Re: The .hpprgm format
« Reply #2 on: August 05, 2014, 11:37:38 am »
Yeah I remember someone there made a tool to send such program and I think optimize them on MoHPC board, mentionned in the post above, although I don't remember if it could edit files. Also, when opening such file in Wordpad, I noticed that we could still somewhat dechiper some of the code, so the format is probably not too hard to dechiper.

Welcome to the forums by the way!
« Last Edit: August 05, 2014, 11:39:39 am by DJ Omnimaga »
In case you are wondering where I went, I left Omni back in 2015 to form CodeWalrus due to various reasons explained back then, but I stopped calc dev in 2016 and am now mostly active on the CW Discord server at https://discord.gg/cuZcfcF

#### Streetwalrus

• LV12 Extreme Poster (Next: 5000)
• Posts: 3820
• Rating: +80/-8
##### Re: The .hpprgm format
« Reply #3 on: August 05, 2014, 11:56:10 am »
Lol that reminds me of HP40G programs. I'd some times print out the file to type it on calc.
send it

#### timwessman

• LV3 Member (Next: 100)
• Posts: 94
• Rating: +32/-0
##### Re: The .hpprgm format
« Reply #4 on: August 05, 2014, 06:40:25 pm »
Hello,

I'll copy in a post I made over on the MoHPC.

Quote
A gentle reminder to people about the hpprgm extension and others in use in the calculator - these are not plain text files!

I've seen several people now post files labeled with an hpprgm extension when in fact they are plain text. Please remember that hpprgm files are in fact BINARY files and not meant for editing with a text editor. While you can open an exe with a text editor, making changes will cause problems and most likely crashes. The same WILL happen if you are manually tweaking hpprgm files.

When the format changes as new features are added, your programs will confuse people as they try to load them and they fail. The fact that you can currently open them and they seem to show plain text primarily does not mean you should go ahead and edit things.

You have been cautioned and please remember this! For the sake of people in the future getting totally perplexed as to why things aren't working, I'd ask you to keep your plain text source files as source, and only have the hpprgm extension on files that were generated by HP software (or some other specialty program).

We don't have any issues with giving out information about the files, but you can be pretty certain the format will be changing as things get added. The program file format is far from ready to lock everything down permanently yet. You pretty much have everything of interest right now correct. There is not any arm functions or calls saved in the file. I'd recommend just keeping out of the object table at the moment as those will be the most likely to change. The source area is probably pretty safe.

The basic structure/process for saving is this:

Code: [Select]
 Save Total Size // save descriptor info Save All Vars/Exports //saves the names for use in accessing things Save the source //as described
I haven't checked for certain, but your "optional" headers might just be the HP Objs encoded. Reals, lists, etc. That is probably what you are seeing I would guess.
« Last Edit: August 05, 2014, 06:47:41 pm by timwessman »
TW

Although I work for the HP calculator group, the comments and opinions I post here are my own.

#### timwessman

• LV3 Member (Next: 100)
• Posts: 94
• Rating: +32/-0
##### Re: The .hpprgm format
« Reply #5 on: August 05, 2014, 06:46:36 pm »
*dup* How do I delete my dups? Possible?
« Last Edit: August 05, 2014, 06:48:58 pm by timwessman »
TW

Although I work for the HP calculator group, the comments and opinions I post here are my own.

#### Streetwalrus

• LV12 Extreme Poster (Next: 5000)
• Posts: 3820
• Rating: +80/-8
##### Re: The .hpprgm format
« Reply #6 on: August 05, 2014, 06:53:06 pm »
You can no longer do it yourself since the upgrade to smf 2.0. Just wait for a mod to do it.
send it

#### DJ Omnimaga

• Now active at https://codewalr.us
• CoT Emeritus
• LV15 Omnimagician (Next: --)
• Posts: 55821
• Rating: +3151/-232
• CodeWalrus founder & retired Omnimaga founder
##### Re: The .hpprgm format
« Reply #7 on: August 05, 2014, 07:41:29 pm »
We don't have any issues with giving out information about the files, but you can be pretty certain the format will be changing as things get added. The program file format is far from ready to lock everything down permanently yet. You pretty much have everything of interest right now correct. There is not any arm functions or calls saved in the file. I'd recommend just keeping out of the object table at the moment as those will be the most likely to change. The source area is probably pretty safe.
Does it mean that older .hpprgm files will eventually no longer send in newer OS versions?
In case you are wondering where I went, I left Omni back in 2015 to form CodeWalrus due to various reasons explained back then, but I stopped calc dev in 2016 and am now mostly active on the CW Discord server at https://discord.gg/cuZcfcF

#### cyrille

• LV1 Newcomer (Next: 20)
• Posts: 8
• Rating: +0/-0
##### Re: The .hpprgm format
« Reply #8 on: August 06, 2014, 01:28:02 am »
Hello,

note that a lot of the data in the header is optional, you can send a program without it's variables or any of the exported data info, put the source nevertheless and, once on the calculator, open, add a space and exit to compile... that makes it much easier if what you want to do is to edit programs on the PC...

note, prety much all the saves data in the calc follows a format of the type:
[header] [blob size   blob of data]*

in a program, there is no fixed size header, and the source is always the last entry in the file. This should make it easy to get to it without having to understand the format...

cyrille

#### Hasse

• LV0 Newcomer (Next: 5)
• Posts: 3
• Rating: +0/-0
##### Re: The .hpprgm format
« Reply #9 on: August 06, 2014, 02:19:08 am »
I originally wanted to make a C to HP PPL translator (obviously without pointer support), since I like C more. I wanted the "compiler" to output valid .hpprgm files which is why I tried to reverse engineer the format. However, I know it takes a long time to get something usable and the difference between the languages isn't great enough to make the project worthwhile.

The "optional headers" were only in some bigger .hpprgms like the hangman game over at MoHPC. I probably should have tried to make large programs myself and modify them to see how the header changes, but I was too lazy  . None of the small programs I wrote myself contained those headers.

Interestingly the hangman game has all zeroes where connection kit generates 7F 01 00 00 00 00. Could this be caused by the program being written on an older connection kit?

#### DJ Omnimaga

• Now active at https://codewalr.us
• CoT Emeritus
• LV15 Omnimagician (Next: --)
• Posts: 55821
• Rating: +3151/-232
• CodeWalrus founder & retired Omnimaga founder
##### Re: The .hpprgm format
« Reply #10 on: August 06, 2014, 02:27:09 am »
That said, it's also possible to send a modified firmware file to the calculator, so someone could just modify the official file with a command to launch C/ASM programs then release an IPS patch version of the mod, then you would be set.
In case you are wondering where I went, I left Omni back in 2015 to form CodeWalrus due to various reasons explained back then, but I stopped calc dev in 2016 and am now mostly active on the CW Discord server at https://discord.gg/cuZcfcF

• Editor
• LV10 31337 u53r (Next: 2000)
• Posts: 1708
• Rating: +229/-17
##### Re: The .hpprgm format
« Reply #11 on: August 06, 2014, 04:35:00 am »
Anyway, the format is known enough for several third-party tools (Eried's program, Critor's mViewer GX Generator, for example...) to generate valid files, so there's that. If the unsupported parts are actually not necessary and not long-term safe, well, we may not need to take time to understand them, it wouldn't be useful
My calculator programs
TI-Nspire Lua programming : Tutorials  |  API Documentation

#### Hasse

• LV0 Newcomer (Next: 5)
• Posts: 3
• Rating: +0/-0
##### Re: The .hpprgm format
« Reply #12 on: August 06, 2014, 06:40:17 am »
Sorry, I was being a little melodramatic yesterday. Reverse engineering is like a puzzle, and not completely understanding the file is like giving up for me. I have problems with giving up. I continued the work and updated the post accordingly.  Turns out there is no bytecode and Tim Wessman was correct, they are encoded values.

Regarding the usefulness of all of this, I agree that this is probably worthless but it is a fun puzzle nevertheless.

#### DJ Omnimaga

• Now active at https://codewalr.us
• CoT Emeritus
• LV15 Omnimagician (Next: --)
• Posts: 55821
• Rating: +3151/-232
• CodeWalrus founder & retired Omnimaga founder
##### Re: The .hpprgm format
« Reply #13 on: August 06, 2014, 12:32:12 pm »
Indeed. There are a bunch of things that aren't necessarily useful on the TI-84+, but it didn't stop anyone from documenting the entire calculator hardware and software, so it wouldn't hurt if the same was done with HP calcs.
In case you are wondering where I went, I left Omni back in 2015 to form CodeWalrus due to various reasons explained back then, but I stopped calc dev in 2016 and am now mostly active on the CW Discord server at https://discord.gg/cuZcfcF

#### cyrille

• LV1 Newcomer (Next: 20)
• Posts: 8
• Rating: +0/-0
##### Re: The .hpprgm format
« Reply #14 on: August 07, 2014, 02:19:28 am »
Hello,

well, I don't remember the specifics from the top of my head, but basicaly:
- First blob: header with number of exported function, variables (exported and not) and views.
This first blob also contains the list of strings with the names of these exported (or not) objects.

- n blobs: the content/value of the program global variables. This is where it is complicated as a varaible can contain expressions/functions and the like, even CAS objects... not that the format is complciated, it's just has a lot... from numbers, to matrices, lists, complex...

- 1 blob: source code.

cyrille