Omnimaga

General Discussion => Technology and Development => Computer Programming => Topic started by: BlakPilar on July 26, 2011, 04:52:19 pm

Title: 8xp File Structure
Post by: BlakPilar on July 26, 2011, 04:52:19 pm

I have a document (I can't remember where I got it from) and I was just wondering if it is 100% accurate. If it is, can someone explain the checksum? Thanks.

Code: [Select]

All values are in decimal.

8XP File:
(1-11)Header (11 bytes): 42,42,84,73,56,51,70,42,26,10,0
(12-52)Comment (42 bytes): 42 bytes of ASCII, if the comment is <42 bytes then fill the rest of the space with 0s
(53)Comment Delimiter: 0
(54-55)Length of the Data Section (2 bytes): Length of the data section+19
(56-57)Random Stuff (2 bytes): 13,0
(58-59)Length of the Data Section (2 bytes): Length of the data section+2
(60)Protected (1 byte): 6 if yes, 5 if no
(61-68)Program Name (8 bytes): Program Name (in ASCII)-fill with 0s if <8 bytes
(69-70)More Random Stuff (2 bytes): 0,0
(71-72)Length of the Data Section (2 bytes): Length of the data section+2
(73-74)Length of the Data Section (2 bytes): Length of the data section
(75-?)Data Section (Varies): The program's data
Checksum: Checksum of 56 bytes to the byte before the checksum

Title: Re: 8xp File Structure
Post by: Munchor on July 26, 2011, 05:01:39 pm

That is not very reliable, use merthsoft's tables, they are better.

Checksum:
A byte has a value (numerical), it's the sum of the numerical value of all the bytes.

Title: Re: 8xp File Structure
Post by: Scipi on July 26, 2011, 05:06:24 pm

Would the checksum include itself? And the would the checksum's value = 74 + bytes of data?

Title: Re: 8xp File Structure
Post by: Munchor on July 26, 2011, 05:09:12 pm

Quote from: HOMER-16 on July 26, 2011, 05:06:24 pm

Would the checksum include itself? And the would the checksum's value = 74 + bytes of data?

Not sure if it's 74, but it makes sense.

It's like the whole file is 74 bytes (header), the program code is X bytes. So the checksum is 74+numericalvalueOf(X).

Title: Re: 8xp File Structure
Post by: Michael_Lee on July 26, 2011, 05:13:26 pm

This is what I use (all values that start with '$' are in hex):

Code: [Select]

METADATA:

Sizes:
      Offset    Length  Description

Packet
|
| Header
| |   0         8       **TI83F*
| |   8         3       $1A $0A $00
| |   11        42      Comment - padded with zeros or spaces.
| |   53        2       Length of data section.  Should be equal to filesize - 57
| End Header
|
| Data
| |   55        2       Always has a value of $0B or $0D? (prefer $0B?)
| |   57        2       Length of Variable Data Section
| |   59        1       Variable type ID ($05 for programs)
| |   60        8       Variable name (padded with zeros)
| |   68        1       Version (??), usually $00.  (Present if byte 55 == $0D)
| |   69        1       Archive flag.  $80 archived, $00 if not. (Present if byte 55 == $0D)
| |   70        2       Length of Variable Data Section (repeat of bytes @ 57)
| |
| | Variable Data
| | | 72        2       Length of Actual Data / 2
| | | 74        n       Actual Data
| | End Variable Data
| |
| End Data
|
|    74 + n    2       Checksum (sum of Data section)
|
End Packet

And the checksum is the sum of everything in the Data section only.

Title: Re: 8xp File Structure
Post by: BlakPilar on July 26, 2011, 08:00:26 pm

@ephan, I tried their tables but they weren't very in-depth, unless I was looking at the wrong ones. By numerical value, do you mean like if the program was >DMS>DMS then the data section would be 01-01, so that value would be 2? Or is it just the length of the DataSection?

@Michael_Lee, thanks! I'll try doing everything that way when I get back to my computer. Just a question though... Isn't the part where it says "final size - 57" the same as Data Section + 19 (76, 74 bytes before the Data Section and the checksum, minus that 57)?

Title: Re: 8xp File Structure
Post by: Michael_Lee on July 26, 2011, 08:19:25 pm

Quote from: BlakPilar on July 26, 2011, 08:00:26 pm

@Michael_Lee, thanks! I'll try doing everything that way when I get back to my computer. Just a question though... Isn't the part where it says "final size - 57" the same as Data Section + 19 (76, 74 bytes before the Data Section and the checksum, minus that 57)?

Erm, I'm not really sure what you mean -- could you clarify?

I do know that in the Data section, the length of the Variable Data section is repeated twice, if that's what I think you're asking.

Title: Re: 8xp File Structure
Post by: BlakPilar on July 26, 2011, 08:23:46 pm

Okay, um... In yours, at the end of its header section, it says "filesize - 57". Wouldn't that be the same as where in mine it says "Data Section + 19"? Because the 74 bytes before the data section plus the 2 from the checksum is 76. Then 76 minus 57, from yours, is +19. It's a dumb question lol, I'm just checking.

Title: Re: 8xp File Structure
Post by: Michael_Lee on July 26, 2011, 08:31:24 pm

Ok, I see what you mean.  Yep, I think the two should be the same.

A quick comment: in your document, you have a comment delimiter at byte 53: I don't think there's actually one.  If you remove that, and shift down all the offsets accordingly, your file format should output something identical to mine.

Title: Re: 8xp File Structure
Post by: BlakPilar on July 26, 2011, 09:18:11 pm

Hmm... My exporting seems to be slightly off... It works generally, but it's a little broken...

Here's what happens with DRIFTER as an example program. I open it, then export it as a different file. Opening works perfectly.

Spoiler For Open:

Code: [Select]

:"Drifter
StoreGDB 1
/E/9rand→W
Float:Full:FnOff :AxesOff:GridOff:PlotsOff
ZStandard
104→Xmax
⁻72→Ymin
ZInteger
"(X]X(:@y:@y:[[X[[ [YY ++@y(++|cross|[|cross|++(@y+++^[^+ ]:+ ^+|cross|+^ +:] |cross|+^+|cross|í℮J  x√^oY    Y^ox√  J℮í→Str2
SetUpEditor DRIFT
1→dim(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{abs(A/(8Ans+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
DelVar ZClrDraw
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,sub("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,Ans,"1. PLAY DRIFTER
Text(28,Ans,"2. PLAY CUSTOM LEVEL
Text(36,Ans,"3. LEVEL EDITOR
Text(44,Ans,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22 or 1=abs(Ans-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End


Spoiler For Open, then save, then open again:

Code: [Select]

:"D
StoreGDB 1
/E/9→W
Float:F:FO :AO:GO:PO
ZStandard
104→X
⁻72→Y
ZInteger
"(X]X(:::[[X[[ [YY ++(++[++(+++^[^+ ]:+ ^++^ +:] +^+J  ^Y    Y^  J→S2
DRIFT
1→(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{(A/(8A+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
ZCD
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,A,"1. PLAY DRIFTER
Text(28,A,"2. PLAY CUSTOM LEVEL
Text(36,A,"3. LEVEL EDITOR
Text(44,A,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22  1=(A-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End


Title: Re: 8xp File Structure
Post by: Munchor on July 27, 2011, 04:26:53 am

Quote from: BlakPilar on July 26, 2011, 09:18:11 pm

Hmm... My exporting seems to be slightly off... It works generally, but it's a little broken...

Quote from: BlakPilar on July 26, 2011, 09:22:06 pm

Here's what happens with DRIFTER as an example program. I open it, then export it as a different file. Opening works perfectly.

Spoiler For Open:

Code: [Select]

:"Drifter
StoreGDB 1
/E/9rand→W
Float:Full:FnOff :AxesOff:GridOff:PlotsOff
ZStandard
104→Xmax
⁻72→Ymin
ZInteger
"(X]X(:@y:@y:[[X[[ [YY ++@y(++|cross|[|cross|++(@y+++^[^+ ]:+ ^+|cross|+^ +:] |cross|+^+|cross|í℮J  x√^oY    Y^ox√  J℮í→Str2
SetUpEditor DRIFT
1→dim(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{abs(A/(8Ans+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
DelVar ZClrDraw
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,sub("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,Ans,"1. PLAY DRIFTER
Text(28,Ans,"2. PLAY CUSTOM LEVEL
Text(36,Ans,"3. LEVEL EDITOR
Text(44,Ans,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22 or 1=abs(Ans-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End


Spoiler For Open, then save, then open again:

Code: [Select]

:"D
StoreGDB 1
/E/9→W
Float:F:FO :AO:GO:PO
ZStandard
104→X
⁻72→Y
ZInteger
"(X]X(:::[[X[[ [YY ++(++[++(+++^[^+ ]:+ ^++^ +:] +^+J  ^Y    Y^  J→S2
DRIFT
1→(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{(A/(8A+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
ZCD
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,A,"1. PLAY DRIFTER
Text(28,A,"2. PLAY CUSTOM LEVEL
Text(36,A,"3. LEVEL EDITOR
Text(44,A,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22  1=(A-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End


Double Posting is not allowed in those conditions, edit your post. Sigh, you should read the rule number 4:

Quote

4: The only time double-posting is allowed (posting two messages in a row in the same topic) is either if there has been 6 hours since your last post, 3 hours if your thread has been bumped out of the New Posts page, 1 hour if it's a project update or immediately if you need to upload more than 10 file attachments at once. Otherwise, update the last post using the "EDIT" button. It will be marked as unread for all members (altough it won't bump the thread).

Read all of them by the way:

http://www.omnimaga.org/index.php?action=ezportal;sa=page;p=2 (http://www.omnimaga.org/index.php?action=ezportal;sa=page;p=2)

Title: Re: 8xp File Structure
Post by: BlakPilar on July 27, 2011, 04:31:01 pm

Alright, I did, but could you clarify on the checksum? Say a program was just

Code: [Select]

9->AThe data section would be 39-04-41 (in bytes). Would the checksum be 124, or 3?

Title: Re: 8xp File Structure
Post by: Michael_Lee on July 28, 2011, 01:46:07 am

(Disclaimer: it's late at night, and I probably got something wrong.  Any corrections or modifications will be welcomed with open arms.)

The checksum is the sum of all the data from offsets starting from byte 55.

That means the sum of all the data in this region:

Code: [Select]

METADATA:

Sizes:
      Offset    Length  Description
<snip>
| Data
| |   55        2       Always has a value of $0B or $0D? (prefer $0B?)
| |   57        2       Length of Variable Data Section
| |   59        1       Variable type ID ($05 for programs)
| |   60        8       Variable name (padded with zeros)
| |   68        1       Version (??), usually $00.  (Present if byte 55 == $0D)
| |   69        1       Archive flag.  $80 archived, $00 if not. (Present if byte 55 == $0D)
| |   70        2       Length of Variable Data Section (repeat of bytes @ 57)
| |
| | Variable Data
| | | 72        2       Length of Actual Data / 2
| | | 74        n       Actual Data
| | End Variable Data
| |
| End Data
<snip>
...is the checksum.

For example, if I had the program,

Code: [Select]

3->A

the full 8xp (in hex) would be the following:

Code: [Select]

Packet
|
| Header
| | **TI83F*                   2A2A54493833462A
| | (part of signature)        1A0A00
| | Comment                    546F6B656E697A656420
| |                            7573696E67205449436F
| |                            6E766572742076302E35
| |                            00000000000000000000
| |                            0000
| | Length of Data section     1600
| End Header
|
| Data
| | Mystery value              0B00
| | Length of Variable Data    0500
| | Variable type ID           05
| | Variable name (TEST)       5445535400000000
| | Version (??)               00
| | Archive flag               00
| | Length of Variable Data    0500
| |
| | Variable Data
| | | Length of actual data    0300
| | | Actual data: 3->A        330441
| | End Variable Data
| |
| End Data
|
| Checksum                     D501
|
End Packet

The length of the Data section (present in the header) and the length of the Variable Data (present in the Data section) all are the number of bytes in their respective sections.  For example, if you were to count all the bytes in the Data section, you would get 22, which is 16 in hexadecimal, which, as you can see, is the the value recorded right after the comment.

The checksum, however, is the sum of all the bytes. 

0B + 00 + 05 + 00 + 05 + 54 + 45 + 53 + (etc, might have missed a few numbers there) + 33 + 04 + 41 = 1D5 (in hex).

The least significant byte goes first, so the checksum is D501.

(Random question: why is it that in the 8xp file, the least significant byte goes first, but looking at the actual data, specifically two-byte-tokens, the least significant byte goes last?  Wouldn't it be more consistent for the least significant byte to go last in the 8xp file format too?)

Title: Re: 8xp File Structure
Post by: BlakPilar on July 28, 2011, 10:33:30 am

Quote

The checksum, however, is the sum of all the bytes.

Ok, thanks!