Here are the results of tests performed in March 2002 to compare lossless compression of "binary" files by all known good enough programs developed for such purpose, including RK, DC, YBS, Bzip2, RAR, ACE and 7-zip. See Archive Comparison Test by J.Gilchrist for more details: http://compression.ca If anybody wants to start or continue such tests, or can suggest some other sets of files, or other compression programs, (not sources or algorithm descriptions, executable programs only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 etc.) please let us know immediately: artest@inbox.ru Thank you!
The Art of Lossless Data Compression vol. 23b [[1]] COMPRESSION QUALITY
(see also [[2]] Speed [[3]] Details [[4]] Comments) Last eleventh line shows results for the sum of all 5960 files in ten sets. Original UHARC PPMonstr RK 7-zip 777 RAR PPMN ACE SBC 254.20% 103.24 100% 105.71 107.05 108.06 108.61 108.27 111.21 112.97 259.38% 100.93 102.18 102.53 102.35 100% 107.32 105.78 108.24 111.35 185.31% 100% 100.99 102.15 102.17 101.75 104.90 102.49 105.14 107.08 339.00% 100% 110.71 114.83 113.91 118.60 110.79 122.91 124.48 114.73 174.39% 100% 109.00 108.38 111.50 111.75 104.96 112.36 103.99 105.04 389.63% 100.96 100% 100.65 107.18 108.17 115.78 116.15 118.99 118.23 191.78% 100% 100.28 101.18 100.20 102.34 105.48 102.28 106.41 105.54 267.56% 100% 100.02 100.91 102.03 104.17 106.83 110.45 108.45 109.53 229.99% 100% 100.69 101.12 101.67 101.53 104.84 102.46 105.65 107.49 361.81% 103.19 100% 102.53 104.84 108.40 115.55 112.50 116.44 119.65 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 240.45% 100% 101.28 102.67 103.58 104.21 106.58 106.60 108.17 109.12 PPMd ERI IMP DC BOA ZZip GCAC YBS BZip2 pkzip 110.73 115.35 117.10 112.62 114.10 116.28 115.93 115.25 119.53 123.67 111.09 114.44 110.13 117.70 113.89 114.51 116.30 114.52 121.72 129.42 106.99 108.56 106.39 110.23 107.67 108.73 108.95 109.18 112.44 111.78 122.57 118.19 120.49 119.88 128.28 127.70 126.34 124.93 135.50 152.39 115.92 100.92 116.28 107.18 115.74 114.33 111.67 115.87 116.89 124.06 116.73 126.50 125.49 120.78 120.91 124.11 125.68 125.40 130.47 132.09 106.74 107.48 107.42 108.70 108.69 106.85 109.01 106.86 113.36 115.78 114.07 112.75 119.61 113.70 115.52 115.28 116.18 118.63 121.74 125.01 108.14 108.72 106.82 110.81 109.44 112.10 111.01 108.88 114.78 115.97 116.47 123.37 123.01 124.07 120.70 121.64 122.66 127.39 132.24 137.17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 110.43 111.14 112.01 112.39 112.46 113.20 113.42 113.52 118.27 121.89[[2]] Speed
4th set, all from ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/sface10.zip was used for this test, and a 400MHz PC with 256Mb RAM and Windows98. Programs, Compression/ Overall Average Users' Compressed options Extraction, Score Score Size seconds seconds, % seconds, % bytes , % no compression 0 0 7613 363 7613 405 27408516 422 777 a -mg 1683 1009 5354 255 3840 204 9586586 148 7za a -t7z 184 13 2906 139 2740 146 9752090 150 7za a -t7z -mx 319 13 2817 134 2531 135 8948766 138 7za a -tzip 62 9 3432 164 3377 180 12104526 186 7za a -tzip -mx 127 10 3487 166 3373 180 12062635 186 acb B 729 799 4424 211 3767 201 10426759 161 acb b 1020 1106 4974 237 4056 216 10254633 158 ace32 a 106 13 2984 142 2888 154 10315479 159 ace32 a -d4096 112 13 2664 127 2563 136 9141767 141 ace32 a -d4096 -m1 94 13 2659 127 2574 137 9190071 142 ace32 a -d4096 -m5 119 12 2675 128 2568 137 9158751 141 ace a -d4096 -m5 115 11 2669 127 2566 137 9158631 141 arh a -2 -mm 523 470 3824 182 3354 179 10196406 157 arh a -1 -mm 162 17 3130 149 2984 159 10625915 164 arh a -2 -1 817 491 4476 213 3741 199 11408516 176 ba -k -1 83 55 3116 149 3041 162 10721194 165 ba -k -10 99 59 3033 145 2944 157 10353854 160 ba -k -50 116 61 3121 149 3017 161 10601741 163 bee a -m1 358 414 3754 179 3432 183 10736631 165 bee a -m3 777 732 4312 206 3613 192 10094993 156 bee a -m3 -d3 807 808 4381 209 3655 195 9963046 153 bix a -mdg 160 9 3092 147 2948 157 10524303 162 bix a -mdg -s 211 8 2939 140 2749 146 9792685 151 boa -m15 1117 1068 5065 242 4060 216 10368786 160 boa -m15 -s 1130 1220 5227 249 4210 224 10359887 160 boa -m7 925 966 4796 229 3963 211 10458903 161 bzip2 -k -1 78 23 3323 158 3254 173 11605029 179 bzip2 -k -9 79 25 3189 152 3119 166 11111475 171 dc e 208 35 2918 139 2731 145 9630736 148 dc e -b16300 286 33 2510 120 2253 120 7888232 122 dc e -b16300 -mb5 272 34 2497 119 2252 120 7888421 122 eri a 114 69 2879 137 2776 148 9706022 150 eri a -m3 140 79 2872 137 2747 146 9555684 147 eri a -m6 156 80 2897 138 2757 147 9582523 148 gcac a 111 46 2994 143 2893 154 10212207 157 gcac s 110 51 3032 145 2933 156 10337588 159 imp98 a -mm 73 8 2794 133 2728 145 9770193 151 imp98 a -mm -m3 123 8 2833 135 2722 145 9730312 150 pkzip -es 10 6 3963 189 3954 211 14212209 219 pkzip -a 24 5 3459 165 3437 183 12347196 190 pkzip -exx 38 6 3466 165 3431 183 12320267 190 ppmd e -o3 -m250 83 93 3333 159 3258 173 11366925 175 ppmd e -o9 -m250 130 132 2680 128 2563 136 8709209 134 ppmd e -o10 -m250 133 137 2629 125 2509 134 8492774 131 ppmd e -o11 -m250 146 156 2676 128 2545 136 8549225 132 ppmn e -o6 -M:50 360 400 3474 166 3150 168 9774051 151 ppmn e -o9 -M:50 414 429 3536 169 3163 168 9694915 149 ppmonstr e -o3 -m250 704 744 4170 199 3536 188 9802452 151 ppmonstr e -o9 -m250 728 845 3730 178 3076 164 7771437 120 ppmonstr e -o10 -m250 761 799 3675 175 2990 159 7613446 117 ppmonstr e -o11 -m250 830 807 3763 179 3015 161 7653034 118 ppmy /o6 /m220 7030 7609 17725 845 11398 607 11114335 171 ppmy /o8 /m220 6405 6395 16238 774 10473 558 12376377 191 ppmy /o9 /m220 6579 6931 17053 813 11131 593 12752730 196 qlfc a 89 43 3100 148 3020 161 10689214 165 rar a -m1 70 9 3191 152 3128 167 11203280 173 rar a -m3 85 11 2593 124 2516 134 8990790 139 rar a -m5 96 11 2595 124 2508 134 8956695 138 rar a -m5 -s 130 13 2437 116 2320 124 8260158 127 rar a -mct- -s 92 11 2415 115 2332 124 8323381 128 rar32 a -mct- -s 92 9 2413 115 2331 124 8326507 128 rk -mf1 79 75 3194 152 3123 166 10946164 169 rk -mf3 260 172 3046 145 2812 150 9410072 145 rk -mx1 1146 1218 5037 240 4006 213 9627056 148 rk -mx3 1486 1606 5476 261 4138 220 8584940 132 sbc c -a -b63 124 53 2493 119 2381 127 8338208 128 sbc c -oa -b63 146 76 2357 112 2226 119 7689250 118 sbc c -of -b63 166 88 2410 115 2260 120 7762214 120 szip -o8 100 54 3209 153 3120 166 11000521 169 szip -o8 -b41 108 54 3126 149 3029 161 10671313 164 ufa a -mg 474 87 3311 158 2884 154 9900254 153 ufa a -mg -s 622 64 3472 166 2912 155 10030871 155 uharc a -m1 -md32768 226 44 2097 100 1893 101 6577686 101 uharc a -m2 -md32768 271 44 2126 101 1882 100 6522084 100 uharc a -m3 -md32768 317 44 2163 103 1878 100 6490844 100 uharc a -mz -md32768 57 60 2707 129 2656 141 9325552 144 uharc a -mx -md32768 383 313 2536 121 2191 117 6626726 102 ybs -m15mu 97 34 2757 131 2670 142 9457057 146 ybs -m15mu -r 100 33 2754 131 2664 142 9436768 145 zzip a -mm 106 48 2956 141 2861 152 10091114 155 zzip a -mm -a 107 44 2843 136 2747 146 9692674 149 Overall score is calculated by adding compression time, extraction time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 Average Users' score is calculated by adding (compress_time/10)+ extract_time + time it would take to transfer the compressed file over a 28,800bps network. Compression time is divided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or Internet. That's why compression time is not so actual for them.[[3]] Details
are no longer put to this main text (6294 lines reporting 238840 results on 5960 files in 10 sets), but can be found in FULL version with BINARIES.DAT and *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest23.zip or http://artest1.tripod.com/artest23.zip[[4]] Comments
Links to download programs:
7-Zip 2.30b17 :W http://www.7-zip.org/dl/7zip230b17.exe 891K ACE 2.04 :W http://winace.host.sk/ace204.exe 593K ERI 5.1fre :e http://geocities.com/eri32/eri51fre.zip 89K GCac 0.9k :W http://www.emit.jp/gca/gca_v09k.exe 295K PPMN 1.00b1+ :W http://compression.graphicon.ru/ms/ppmnb1+.rar 40K PPMY 0.3b :W http://www.pilabs.org.ua/sh/ppmy_3b.zip 15K RAR32 3.00b4 :e ftp://pubftp.slovenska.sk/pub/rarsoft/rar/rarx30b4.exe 415K WinRAR 3.00b4 :W ftp://pubftp.slovenska.sk/pub/rarsoft/rar/wrar30b4.exe 942K SBC_d 0.910b :e http://personal.inet.fi/musiikki/sjm/sbc0910b_dos.zip 195K SBC 0.910b :W http://personal.inet.fi/musiikki/sjm/sbc0910b_win32.zip 154K UHArc 0.4b :eW ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc04.zip 220K QLFC 6.6W :W http://ghido.shelter.ro/DownloadQLFC.php 49K ZZip 0.36c :W http://www.via.ecp.fr/~damien/downloads/zzip-win32.zip 35K PPMD var.H, PPmonstr v.H :W ftp://ftp.elf.stuba.sk/pub/pc/pack/ppmdh.rar 57K PkzipC 4.00 :W ftp://ftp.pkware.com/pkzc400s.exe 3470K PkZip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K RK-dos 1.04.1 :e http://rksoft.virtualave.net/downloads/rk104a1d.exe 461K RK 1.04.1 :W http://rksoft.virtualave.net/downloads/rk104a1w.exe 380K BA 1.01b5 :e http://hem.spray.se/mikael.lundqvist/ba101br5.zip 61K BIX 1.00b7 :W http://www.7-zip.org/dl/ufa/bix100b7.zip 89K 777 0.04b1 :W http://www.7-zip.org/dl/ufa/777004b1.zip 72K UFA 0.04b1 :W http://www.7-zip.org/dl/ufa/ufa004b1.zip 64K ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K Imp 1.1 :e http://www.technelysium.com.au/imp110d.zip 266K Imp-win 1.12 :W http://www.technelysium.com.au/imp112.exe 122K ACB 2.00c :e ftp://ftp.simtel.net/pub/simtelnet/msdos/compress/acb_200c.zip 42K BOA 0.58b :e ftp://ftp.elf.stuba.sk/pub/pc/pack/boa058.zip 74K DC 0.98b :W ftp://ftp.elf.stuba.sk/pub/pc/pack/dc124.zip 55K Bzip2 1.0.2 :W ftp://sourceware.cygnus.com/pub/bzip2/v102/bzip2-102-x86-win32.exe 72K SZip 1.12a :W http://www.compressconsult.com/szip/szip_112a_win32.zip 71K YBS 0.03e :e http://compression.graphicon.ru/ybs/ybs003ed.zip 56K YBS 0.03e :W http://compression.graphicon.ru/ybs/ybs003ew.zip 44K BEE 0.4.8 :W mailto:Andrew.Filinsky@p11.f4.n452.z2.fidonet.org :a - any DOS - DOS programs, will run under pure DOS or in a DOS box :e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - windows - Windows95/98/NT/etc programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename).Homepages:
Arhangel : http://geocities.com/SiliconValley/Lab/6606 BA : http://hem.spray.se/mikael.lundqvist Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK : http://rksoft.virtualave.net Imp,WinImp : http://www.technelysium.com.au/winimp.html ACE,WinACE : http://www.winace.com PkZip : http://www.pkware.com RAR,WinRAR : http://www.rarsoft.com BZip2 : http://sources.redhat.com/bzip2 SZip : http://www.compressconsult.com/szip PPMN : http://compression.graphicon.ru/ms YBS : http://compression.graphicon.ru/ybs SBC : http://sbcarchiver.netfirms.com QLFC : http://ghido.shelter.ro GCAC : http://emit.jp/gca/gca.html PPMy : http://www.pilabs.org.ua/sh ZZip : ? http://www.zzip.f2s.com Ufa,777, BIX,7-Zip : http://www.7-zip.orgWhat's new:
12 new programs were tested: UHArc 0.4 beta PPMd v.Ipre PPMonstr v.Ipre 7-zip 2.30b17 RAR 3.00b4 PPMN 1.00b1 ACE 2.04 SBC 0.910 beta ERI 5.1fre ZZip 0.36c GCAC 0.9k QLFC 6.6W Latest beta versions of PPMonstr, BEE, DC, are available from authors by e-mail request: PPMonstr: shkarin@arstel.ru BEE: Andrew.Filinsky@p11.f4.n452.z2.fidonet.org DC: EdgarBinder@t-online.de ACB and BEE are not tested on all binary files any more, both are too slow and not efficient on binary data, especially with uncompressed multimedia, their results can be found in previous versions of ARTest. Results of BIX, BA, Arhangel, SZip, QLFC and 7-zip with .zip format are in full version only, BINARIES.DAT file. FULL version contains all *.BAT and *.DAT files you'll need to build 10 directories with 5960 files (artest23.zip\COOK_SET\*.*) and to repeat all our tests (BINS.BAT, BINARIES.DAT). Some old or less efficient programs will not be tested next time: UFA, BIX, ACB, BA, BEE, PPMy.WARNINGS:
ACB refuses to take files shorter than 257 bytes. About 400 such files were processed with "rar a -m0 name.acb name.bin" (see do_acb_u.bat and do_acb_r.bat in artest19.zip). BA 1.00beta5 can't losslessly decompress many files: CUN5IF~1.bin, CUN6IF~1 etc It says nothing like "CRC fails". BEE can't decompress some files compressed in "solid" mode. DC 0.99.158b fails to decompress HLPCOPY..bin, but only if you compress with "-mb5" switch and [<output>] filename like "any.dc5". Says nothing like "CRC fails". SBC is the only compressor converting "non-portable" symbols like "+", "&", "$" or "," to "_". Inconvenient feature. Problems in all other compressors were not found. The LATEST RELEASE, and all previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/ Send your suggestions, comments to artest@inbox.ru With best kind regards, A.Ratushnyak, RAO Inc. Back to main ARTest page
