Here are the results of tests performed in June 2000 to compare lossless compression of True Color (24-bit) images by all known programs developed for such purpose, including BMF, JPEG-LS and PNG software, 7-Zip, Pkzip, and archivers with special algorithms for 24-bit images. See Archive Comparison Test by Jeff Gilchrist for more info: http://act.by.net If anybody wants to start or continue such tests, or can suggest some other sets of images, or other compression programs, (not sources or algorithm descriptions, executable programs only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 and lossy compression of up to 10000:1, we hear about it from time to time since our XTs) please let us know immediately: artest@hotmail.ru Thank you!
The Art of Lossless Data Compression vol. 19g [[1]] COMPRESSION QUALITY
(see also [[2]] Speed [[3]] Details [[4]] Comments) The last eleventh line shows results for the sum of all ten tests (336 images) Original BMF RKim RK UHIC ArHanGeL NK Eri RAR Imp 275.95% 100% 101.59 120.20 100.55 110.52 112.65 117.81 133.15 154.65 289.11% 100% 124.50 127.28 104.88 114.61 135.54 122.36 140.60 152.88 245.22% 100% 101.38 123.16 107.93 112.92 111.48 123.50 124.25 144.52 238.64% 100% 112.66 124.20 104.20 110.53 120.45 121.23 132.94 146.59 457.28% 101.93 100% 133.41 110.74 112.34 120.22 125.22 146.11 157.63 234.93% 100% 101.72 119.42 107.07 110.08 106.85 121.92 125.63 143.10 278.95% 100% 103.17 123.14 103.40 109.60 110.16 119.57 135.88 146.37 332.49% 100% 102.12 126.63 105.92 111.58 112.18 120.18 139.49 153.56 225.19% 100% 104.47 118.81 104.00 108.82 110.47 118.20 122.22 141.08 547.50% 100% 104.25 131.46 104.84 107.76 127.75 115.26 170.38 157.56 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 286.32% 100% 102.98 122.95 105.47 110.35 111.15 120.43 133.89 147.68 JPEG-LS SPIHT BTPC { Display } SEA PngCrush 7-Zip PkZip 149.81% 115.50 164.36 157.93 156.06 153.89 150.94 150.33 159.16 169.90 156.41% 136.11 170.42 177.21 169.62 164.86 163.71 151.73 205.21 211.56 119.20% 111.39 129.21 140.98 141.56 139.53 146.28 129.83 193.86 196.55 127.45% 122.73 137.49 146.42 145.46 144.00 147.31 134.68 182.24 185.85 172.49% 224.13 201.52 187.28 156.67 156.16 160.57 145.35 185.82 188.99 112.56% 111.61 119.89 129.58 132.73 129.30 137.85 120.59 170.09 172.74 118.69% 119.12 131.29 140.90 139.04 137.13 143.73 129.64 171.53 175.85 123.33% 117.86 136.63 145.92 143.78 142.44 151.77 133.46 197.76 202.49 117.84% 109.95 122.53 133.78 135.20 132.58 139.93 121.86 179.50 181.44 151.17% 132.31 176.94 164.21 150.26 148.74 156.87 142.19 186.28 195.55 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 121.48% 117.50 132.78 140.67 139.78 137.42 144.92 128.95 179.40 183.54[[2]] Speed
A set of 230 images totalling 271,331,460 bytes from ftp://ipl.rpi.edu/pub/image/still/KodakImages/color/ was used for this test and an AMD-K6-400 machine with 64M RAM and Windows98. Well-known 23 Kodak images copied ten times (see icopy.bat in artest19.zip) Programs,options Overall Average Compress Extract Compressed score, Users' time, time, size, score, seconds seconds bytes seconds % seconds % +7-zip a 44471 146% 43736 147% 817 137 156662222 +7-zip a -mx 44960 148% 43743 147% 1352 138 156492462 +arhangel a -mm -mz 34295 113% 31998 108% 2552 1557 108669974 +arhangel a -mm 34260 112% 31989 108% 2523 1551 108669974 +arhangel a -mm1 48908 161% 45779 154% 3476 2633 154074824 +arhangel a -mm3 42870 141% 40214 135% 2951 1879 136944664 bmf -f -s -q9 38628 127% 30978 104% 8500 2816 98323280 bmf -f -s 33477 110% 30467 103% 3345 2820 98323280 bmf -f -q9 33681 111% 29829 100% 4280 249 104947560 bmf -f 30324 100% 29572 100% 836 258 105229600 eri a -3ma8 31885 105% 30118 101% 1963 449 106102020 eri a -3ma8 -m3 31413 103% 30078 101% 1483 435 106182410 eri a -3ma8 -m2 31255 103% 30290 102% 1072 355 107379300 eri a -3ma8 -m1 32073 105% 31299 105% 860 299 111292130 +imp98 a -2 38040 125% 37366 126% 749 222 133449566 +imp98 a -2 -s4 38123 125% 37453 126% 744 218 133779179 +imp98 a -mm 43242 142% 42363 143% 977 110 151758474 +imp98 a -mm -s4 43204 142% 42299 143% 1006 109 151521544 +imp98 a -mm -m3 43528 143% 42174 142% 1505 119 150856163 +imp98 a -mm -m1 46105 152% 45522 153% 648 111 163246850 +imp-dos a -mm -m3 43565 143% 42194 142% 1524 137 150855710 Nk 35338 116% 31666 107% 4080 489 110767370 pkzip -exx 46871 154% 46394 156% 530 146 166302252 pkzip -a 46708 154% 46380 156% 365 146 166309392 pkzip -es 55426 182% 55211 186% 238 152 198128322 +pkzip32 -add -max 47059 155% 46592 157% 519 138 167047382 +pkzip32 -add 47018 155% 46662 157% 396 136 167349362 +WinRar a -mm -mde 37138 122% 36685 124% 504 259 130951080 +WinRar a -mm -m5 37149 122% 36678 124% 523 259 130921330 +WinRar a -mm -s- 37137 122% 36685 124% 503 259 130951080 +WinRar a -mm -s 37124 122% 36648 123% 529 258 130813550 +WinRar a -mmf 36740 121% 36324 122% 462 267 129638820 +rar32 a -mm -mde 37254 122% 36731 124% 582 297 130951080 +rar32 a -mm -m5 37263 122% 36724 124% 599 297 130921330 +rar32 a -mm -s- 37253 122% 36730 124% 581 297 130951080 +rar32 a -mm -s 37236 122% 36692 124% 605 294 130813550 +rar32 a -mmf 36860 121% 36371 122% 543 306 129638820 +rk -mx2 27554 90%* 16839 56%* 11906 12305 12035652 +rk -mx1 46827 154%* 37353 126%* 10527 11174 90452920 +rk -mf3 7011 23%* 5469 18%* 1713 1191 14784112 +rk -mf2 7863 25%* 6571 22%* 1435 1176 18906828 +rk -mf1 39048 128%* 37937 128%* 1235 887 132934712 +rkim cx 35399 116% 30769 104% 5144 2506 99896080 +rkim c 32861 108% 31261 105% 1778 2178 104059250 uhic m 36260 119% 32373 109% 4319 4477 98870920 uhic e 32812 108% 30565 103% 2496 2501 100132460 Alchemy ---n 46843 154% 44143 149% 3000 526 155940956 ---n 3 46602 153% 44584 150% 2242 648 157361419 ---n 2 46338 152% 44670 151% 1853 437 158572986 ---n 1 45301 149% 43610 147% 1879 434 154758370 SEA /png:100 42487 140% 41480 140% 1119 142 148414950 Display -i --same -bpng + filter=5,level=9 44739 147% 42446 143% 2547 159 151317180 + filter=5,level=1 44203 145% 43805 148% 442 168 156934080 + filter=1,level=9 44739 147% 42537 143% 2447 149 151714120 + filter=1,level=5 43955 144% 43263 146% 769 150 154930330 + filter=1,level=1 44605 147% 44292 149% 348 156 158763080 +pngcrush -q 49026 161% 42265 142% 7512 455 147813040 +pngcrush -brute 165325 545% 53632 181% 124103 460 146744530 +btpc 100 45618 150% 45209 152% 454 273 161606980 +lsp -g 42530 140% 42119 142% 456 375 150115060 +lsp 32507 107% 32034 108% 526 434 113570440 +locoe -N -c1 41544 137% 41271 139% 303 323 147305210 +locoe -N -c2 41732 137% 41406 140% 362 342 147701630 -PhotoJazz ????? ? ????? ? ??? ??? 111673670 + indicates new lines in current volume of ARTest - indicates oldest lines, that will not be present in next volume * only because most files are identical, and RK *always* uses "solid" archiving Overall score is calculated by adding compression time, extraction time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 , because 28800 bits_per_second is 3600 bytes_per_second Average Users' score is calculated by adding (compress_time/10)+ extract_time + time it would take to transfer the compressed file over a 28,800bps network. Compression time is divided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or Internet. That's why compression time is not so actual for them.[[3]] Details
are no longer put to this main text (380 lines reporting 10080 results on 336 files in 10 sets), but can be found in FULL version with GRAPHICS.DAT and *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest19.zip or http://artest1.tripod.com/artest19.zip 1,2,3,4,9 image sets were taken from ftp://ipl.rpi.edu/pub/image/ and /image2/ converted from Sun Raster format to Windows Bitmap, (this conversion changes image headers only, 32 bytes are replaced by 54 bytes, but image data is not modified) This site ipl.rpi.edu is described in comp.compression FAQ by Jean-loup Gailly, in 29 lines of "Subject: [55] Where can I find Lenna and other images?" Eight images in the 5th set are from the famous Waterloo Color Set, ( losslessly compressed with Eri32 4.8fre to 2,893,654 bytes, it can be found at http://geocities.com/eri32 ) 6,7,8th image sets are from ftp://download.intel.com/pressroom/images/ 10th set was downloaded from ftp://ftp.kodak.com/www/images/dental/ They were converted from TIFF format to Windows Bitmap. Some images were slightly renamed. See GRAPHICS.DAT from artest19.zip for more details. Some new images (from download.intel.com) were not added yet.[[4]] Comments
Links to download programs:
7-Zip 2.11 :W http://www.7-zip.com/dl/7zip211.exe 493K ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K BMF 1.1 :e ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/bmf_1_10.zip 132K ERI32 4.7fre :e http://geocities.com/eri32/eri47fre.zip 91K Imp 1.1 :e http://www.winimp.com/imp110d.zip 266K Imp-win 1.12 :W http://www.winimp.com/imp112.exe 122K PkZip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K PkZip32 2.50 :W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/pk250c32.exe 300K RK 1.02a5 :W http://malcolmt.tripod.com/downloads/rk102a05.exe 191K RKim 1.06 :W http://malcolmt.tripod.com/downloads/rkim106.zip 57K RAR32 2.71 :e ftp://ftp.netlab.sk/public/rarsoft/rar/rarx271.exe 257K WinRAR 2.71 :W ftp://ftp.netlab.sk/public/rarsoft/rar/wrar271.exe 588K UHIC 1.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip 52K NK 1.0 :W ftp://tu.tusur.ru/pub/ImageTools/NK1.ZIP 81K JPEG-LS 1.00 :W http://www.hpl.hp.com/loco/jlsrefV100.zip 325K PngCrush 1.5.1:e http://download.sourceforge.net/pmt/pngcrush-1.5.1-DOS.zip 143K SPIHT Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe 72K BTPC 4.1 :W http://www.engr.mun.ca/~john/btpc4.1.zip 715K Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe 811K Alchemy 1.10 :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip 1462K SEA 1.3 :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip 705K :a - any DOS - DOS programs, will run under pure DOS or in a DOS box :e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - windoze - Windows95/98/NT/etc programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename).Homepages:
Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK, RKim : http://malcolmt.tripod.com Imp,WinImp : http://www.technelysium.com.au mirror : http://www.winimp.com PkZip : http://www.pkware.com Ufa,777,7-Zip: http://www.7-zip.com RAR,WinRAR : http://www.rarsoft.com JPEG-LS : http://www.hpl.hp.com/loco PngCrush : http://pmt.sourceforge.net/pngcrush SPIHT : http://www.ipl.rpi.edu/research/SPIHT BTPC : http://www.engr.mun.ca/~john/btpc.html Display : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html Alchemy : http://www.handmadesw.com/Products/Image_Alchemy.htm SEA : http://www.photodex.com/products/dosWhat's new:
PngCrush 1.5.1, ERI32 4.8fre, WinRAR 2.71, RAR32 for DOS and OS/2 2.71, RK 1.03b1, ACE 2.00b3 and NK ver.1 were released. They will be tested soon. Results of Pngcrush 1.4.4, RK 1.02a5 and RAR 2.70 are given.WARNINGS:
ESP, Rkive and some other programs are not tested any more, their results and links can be found in previous versions: ESP - artest3...artest12 Rkive - 3...12 UFA - 3...12 777 - 13...14 BOA - 7...14 UHARC - 5...10 CompuShow 2000! - 3...11 ESP 1.92 has some bugs and was unable to correctly uncompress some test images: they are (re)named in our tests as AYMAR,GELSNGR,NACHTSH,OTELLIN,VADASZ,YASSO,YU1, DP08189,DP2179D,L217C-3,CONCEP3,PS09157,V42798C. ESP doesn't even report something like "CRC fails"...as if everything is OK. Rkive 1.92b1 fails to losslessly uncompress most images from 6th, 7th and 8th tests, and also Clegg, Frymire, Serrano from 5th. It reports ERROR: File fails CRC check!. Archive failed integrity check. The LATEST RELEASE, and all previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/The FINAL PART
> [[5]] PLEASE read THIS before replying to this article was removed from this text, but can be easily found at http://geocities.com/SiliconValley/Bay/1995/artest10.html http://artest1.tripod.com/artest10.html Send your suggestions, comments to artest@hotmail.ru With best kind regards, RAO Inc.