File Packing

[Blaze]

How does file packing work? I've searched google and haven't found anything other then SP2 fixes :-\.

I'm working on a project similiar to winzip/rar for the fun/experience and I'm wondering if its amazingly difficult or not..

[Newby]

Check out this document. It might help you.

[Joe]

I wrote a little packing thing called JoPaQ (ripped from MoPaQ (MPQ)) once. It was pretty simple.

(DWORD) Number of Files
For each file {
  (DWORD) Name Length (word?)
  (VOID) File Name
  (DWORD) File Length (might want to make this a qword, whatever)
  (VOID) File Data
}

The DWORD+VOID combination is somewhat of a pascal string. I still have the code laying arround somewhere, although it never really worked.

[iago]

google://huffman encoding

Huffman encoding is compression, not packing, but I assume that's what you're looking for. 

[Koga73]

Well, if your looking for a packer (itll recude the size of exes, works great, the exe will still run normally once packed), look up a program called UPX or ProcDump. And as for file compression, search for RLE, run length encryption. It works good for files with lots of repeats, such as:

111111111123456
becomes
10~123456

simple

[Blaze]

...?

[Koga73]

ok, you read the binary of a file your wanting to compress. Then, you look for areas that have 4 or more of the same character all next to each other, so lets say you load it in as binary, and you see the following:

abcddddddddefghhhhijklmnnnnnnopqrstuvwxyzzzzzz

Well, using rle, you would make it:

abc8~defg4~hijklm6~nopqrstuvwxy6~z

It reduced the size from 47 characters to 34.

So, now, save the file as a .BLZ or something, lol. Then, to make the file usable again... go thro, and reexpand all the stuff you compressed.

Thats for the compression part, as for packing files and making a bunch of files into one... just take each individual files biary, and put it all into one file. The depacker will just rebreak it apart into many files.

[Blaze]

I  understood what you orginally posted..  :-[

I was just wondering why you were suggesting I use another program.  I also don't think rle is very realalistic for using on excutable.

[Sidoh]

I also don't think rle is very realalistic for using on excutable.

It's not.

[Koga73]

i just figured id mention some exe compressors, there pretty cool. They can comperss an exe, and the exe can still run perfectly without havign to be decompressed. How do these work?

UPX and ProcDump are some examples.

[Sidoh]

i just figured id mention some exe compressors, there pretty cool. They can comperss an exe, and the exe can still run perfectly without havign to be decompressed. How do these work?

UPX and ProcDump are some examples.

I suppose it would depend on the operating system.  If it had a feature that detected and decrompressed the file data on the fly when it detected it was compressed, that would work.  Test the execution speeds of the same programs compressed and un-compressed and see if they're around the same.

[iago]

i just figured id mention some exe compressors, there pretty cool. They can comperss an exe, and the exe can still run perfectly without havign to be decompressed. How do these work?

UPX and ProcDump are some examples.

I suppose it would depend on the operating system.  If it had a feature that detected and decrompressed the file data on the fly when it detected it was compressed, that would work.  Test the execution speeds of the same programs compressed and un-compressed and see if they're around the same.

Generally, they're uncompressed when they're loaded.  So the load time will be slightly higher, but the overall run time will be mostly the same. 

[rabbit]

Yep.  The file is compressed into itself with instructions of how to decompress the file when it runs, then run it (instead of just running).  I love UPX :P

[Sidoh]

Generally, they're uncompressed when they're loaded.  So the load time will be slightly higher, but the overall run time will be mostly the same. 

Sorry, that's what I meant.