« previous next »
Pages: [1] 2   Go Down

Author Topic: TNS and ZIP file differences  (Read 8720 times)

0 Members and 1 Guest are viewing this topic.

Offline Deep Toaster

  • So much to do, so much time, so little motivation
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 8217
  • Rating: +758/-15
    • View Profile
    • ClrHome
TNS and ZIP file differences
« on: October 19, 2011, 12:19:31 am »
For my BBify'r project, I made a short PHP snippet to open a TNS file as a ZIP archive and read the contents of Problem1.xml. Following the rough description on Hackspire's page on the [hackspire]TNS File Format[/hackspire], I replaced the first ten bytes (*TIMLP followed by four version bytes) with \x50\x4B\x03\x04 and then used ZipArchive to open it. Unfortunately, it didn't work, and ZipArchive didn't even give me an error.

What surprised me was that I had been able to do that with 7-zip before with no problems (using XVI32 to change the first few bytes). But since GUI apps all have a habit of pretending nothing happened when an error occurs, I decided to try it with the command-line unzip command, and here's what it gave me:
Quote from: unzip on bash on Debian
warning [cube.tns.txt]:  362 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  inflating: Problem1.xml
So apparently there is an error, and it's insignificant enough that unzip and 7-zip pass by with no problem, but large enough that ZipArchive stumbles.

Does anyone know what it is? Hackspire mentions nothing of the sort, but it seems outdated anyway (it says the only version numbers known are 0120, 0200, and 0300, while I'm working with a file with version number 0500). For now, I'm cheating in my script and using exec to run unzip on the uploaded file, but if possible I'd like to take that bit out and replace it with native PHP/ZipArchive. Thanks for any help!
« Last Edit: October 19, 2011, 12:20:55 am by Deep Thought »
Logged



Offline AngelFish

  • Is this my custom title?
  • Administrator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 3242
  • Rating: +270/-27
  • I'm a Fishbot
    • View Profile
Re: TNS and ZIP file differences
« Reply #1 on: October 19, 2011, 02:23:06 am »
Is the tns file 512 byte aligned? That might be your error right there.
Logged
∂²Ψ    -(2m(V(x)-E)Ψ
---  = -------------
∂x²        ℏ²Ψ

Offline Jim Bauwens

  • Lua! Nspire! Linux!
  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1881
  • Rating: +206/-7
  • Linux!
    • View Profile
    • nothing...
Re: TNS and ZIP file differences
« Reply #2 on: October 19, 2011, 03:13:42 am »
I had the same problem yesterday, but I only tried it with new documents. I managed to list the content, but it error'ed when I tried to extract some stuff.
Maybe because of encryption? (although kinda weird)
Logged

Offline Netham45

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2103
  • Rating: +213/-4
  • *explodes*
    • View Profile
Re: TNS and ZIP file differences
« Reply #3 on: October 19, 2011, 03:16:55 am »
I'm not familiar with the tns's, but try cutting everything out before the first 0x50,0x4B, as that should be the start of the zip. The central directory might need its offsets updated, too. Check Wikipedia's article on zip files, it's actually pretty damn good.

Edit: If you want to post a small tns, I can check what is off standard. I've worked quite a bit with zips.
« Last Edit: October 19, 2011, 03:18:53 am by Netham45 »
Logged
Omnimaga Admin

Offline Jim Bauwens

  • Lua! Nspire! Linux!
  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1881
  • Rating: +206/-7
  • Linux!
    • View Profile
    • nothing...
Re: TNS and ZIP file differences
« Reply #4 on: October 19, 2011, 03:46:51 am »
Code: [Select]

 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
     659  Unk:013     320  51% 2001-01-08 21:17 44684573  Document.xml
   61206  Unk:013    3465  94% 2001-01-08 21:17 2941b95f  Problem1.xml
--------          -------  ---                            -------
Code: [Select]
  skipping: Document.xml            unsupported compression method 13
   skipping: Problem1.xml            unsupported compression method 13
It looks like the compression method is not supported (or wrong marked?) Maybe because of the encryption.

Edit: I didn't see the error you gave. Try using the script on hackspire to add the correct header.
« Last Edit: October 19, 2011, 03:47:49 am by jimbauwens »
Logged

Offline Deep Toaster

  • So much to do, so much time, so little motivation
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 8217
  • Rating: +758/-15
    • View Profile
    • ClrHome
Re: TNS and ZIP file differences
« Reply #5 on: October 19, 2011, 09:37:05 am »
Quote from: Qwerty.55 on October 19, 2011, 02:23:06 am
Is the tns file 512 byte aligned? That might be your error right there.
No idea what that means :/
Quote from: Netham45 on October 19, 2011, 03:16:55 am
I'm not familiar with the tns's, but try cutting everything out before the first 0x50,0x4B, as that should be the start of the zip. The central directory might need its offsets updated, too. Check Wikipedia's article on zip files, it's actually pretty damn good.

Edit: If you want to post a small tns, I can check what is off standard. I've worked quite a bit with zips.
Thanks. I'll try that when I get home today. The TNS file I tested with was jimbauwens's unencrypted build of Chockasta's CubeField.
Quote from: jimbauwens on October 19, 2011, 03:46:51 am
It looks like the compression method is not supported (or wrong marked?) Maybe because of the encryption.
Hackspire does say TNS files use a "compression method 13" that doesn't really exist, but 7-zip and unzip run it fine ???
Logged



Offline Adriweb

  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1708
  • Rating: +229/-17
    • View Profile
    • TI-Planet.org
Re: TNS and ZIP file differences
« Reply #6 on: October 19, 2011, 10:12:26 am »
Extended has worked on that unknown 13 (0D field) compression algorithm (a part of the encryption has been patented and is thus public).

http://ndlessly.wordpress.com/2011/05/22/d-method-encryption-and-lua-hopes-for-os-3-0-2/
and
http://ndlessly.wordpress.com/2011/07/31/lua-to-tns-converter-a-little-bit-further/

He said that since he made a program to compress correctly (bypassing (and not breaking) the patented stuff, that's why it's not illegal), it wouldn't be so hard technically to build a de-compress tool.
« Last Edit: October 19, 2011, 10:17:02 am by adriweb »
Logged
My calculator programs
TI-Planet.org co-admin.
TI-Nspire Lua programming : Tutorials  |  API Documentation

Offline Deep Toaster

  • So much to do, so much time, so little motivation
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 8217
  • Rating: +758/-15
    • View Profile
    • ClrHome
Re: TNS and ZIP file differences
« Reply #7 on: October 19, 2011, 10:32:18 am »
Why did 7-zip and unzip work then? I don't know much about encryption and compression :/
Logged



Offline Jim Bauwens

  • Lua! Nspire! Linux!
  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1881
  • Rating: +206/-7
  • Linux!
    • View Profile
    • nothing...
Re: TNS and ZIP file differences
« Reply #8 on: October 19, 2011, 10:36:43 am »
They don't work with encrypted tns files, they just list the content.
Logged

Offline Netham45

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2103
  • Rating: +213/-4
  • *explodes*
    • View Profile
Re: TNS and ZIP file differences
« Reply #9 on: October 19, 2011, 02:03:39 pm »
As far as the zip goes (I'm no good with compression) if you remove the first 367 bytes, it looks to be a normal zip that even opens with Windows' built-in zip tool, which is extraordinarily bitchy.

The document.xml at the top appears to be a valid zip entry, once you patch the header. The rest of the zip seems to be running off of the assumption that it's not there (the central directory start is offset by -367 bytes, problem1.xml starts at 0x00, etc...) It's also compressed with the 0x0D compression type.

I repaired the central directory (Added an entry for document, corrected the offset of problem, added one to the file counters, updated the start of directory offset, updated the size of directory, searched for stupid typo for 5 minutes; I didn't update the timestamps on document.xml) in a copy of it and properly placed everything. Here's my resultant zip. Other than that 0x0D compression type, everything opens it fine.

Edit: The -367 byte offset assumes that you haven't fixed the entry for document.zip (remove first 10 bytes, add 0x50,0x4B,0x03,0x04).
« Last Edit: October 19, 2011, 02:05:48 pm by Netham45 »
Logged
Omnimaga Admin

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: TNS and ZIP file differences
« Reply #10 on: October 19, 2011, 02:07:34 pm »
In addition to the TIMLPxxxx at the beginning, where xxxx should be replaced by PK\x03\x04, at the bottom of the file, TIPD needs to be replaced by PK\x05\x06.
Logged
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline Netham45

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2103
  • Rating: +213/-4
  • *explodes*
    • View Profile
Re: TNS and ZIP file differences
« Reply #11 on: October 19, 2011, 02:08:39 pm »
Quote from: Lionel Debroux on October 19, 2011, 02:07:34 pm
In addition to the TIMLPxxxx at the beginning, where xxxx should be replaced by PK\x03\x04, at the bottom of the file, TIPD needs to be replaced by PK\x05\x06.

I didn't have to fix those on mine. o.O
Logged
Omnimaga Admin

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: TNS and ZIP file differences
« Reply #12 on: October 19, 2011, 02:15:28 pm »
Then it was probably not generated by TI's software (unless it was a file suitable for OS 1.1) :)
I've checked that the 8 obfuscated files I generated for a user of the tinspire Google group have TIPD instead of PK\x05\x06.
Logged
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline Deep Toaster

  • So much to do, so much time, so little motivation
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 8217
  • Rating: +758/-15
    • View Profile
    • ClrHome
Re: TNS and ZIP file differences
« Reply #13 on: October 19, 2011, 03:39:15 pm »
Thanks Netham45, I'll try that.

Lionel, jimbauwens built it with a compside script, so it might not have made the output in perfect TNS format.

EDIT: Does anyone have more unencrypted TNS files they could send me so I can play around with them? ;D
« Last Edit: October 19, 2011, 03:45:14 pm by Deep Thought »
Logged



Offline Jim Bauwens

  • Lua! Nspire! Linux!
  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1881
  • Rating: +206/-7
  • Linux!
    • View Profile
    • nothing...
Re: TNS and ZIP file differences
« Reply #14 on: October 19, 2011, 04:16:27 pm »
Depends
Spoiler For On What?:
On how big your pocketbook is

http://bwns.be/jim/term.tns should be unencrypted.
« Last Edit: October 19, 2011, 04:16:45 pm by jimbauwens »
Logged
Pages: [1] 2   Go Up
« previous next »